00010000:00000010:3.1F:1776177485.121462:0:0:0:(ldlm_resource.c:1463:__ldlm_resource_free()) slab-freed '(res)': 224 at 0000000074c3c20f. 00010000:00000010:3.1:1776177485.121489:0:0:0:(ldlm_lock.c:197:lock_handle_free()) slab-freed 'lock': 560 at 00000000c5439ae9. 00010000:00000010:3.1:1776177485.427199:0:0:0:(ldlm_resource.c:1463:__ldlm_resource_free()) slab-freed '(res)': 224 at 00000000e55908f4. 00010000:00000010:3.1:1776177485.427208:0:0:0:(ldlm_lock.c:197:lock_handle_free()) slab-freed 'lock': 560 at 00000000540a8f8b. 00010000:00000010:2.1F:1776177485.428100:0:2:0:(ldlm_lock.c:197:lock_handle_free()) slab-freed 'lock': 560 at 000000004c4563c4. 00010000:00000010:2.1:1776177485.428112:0:2:0:(ldlm_lock.c:197:lock_handle_free()) slab-freed 'lock': 560 at 0000000080db64b1. 00010000:00000010:2.1:1776177485.428116:0:2:0:(ldlm_resource.c:1463:__ldlm_resource_free()) slab-freed '(res)': 224 at 00000000333a4d01. 00080000:00000010:2.1:1776177485.464090:0:6233:0:(osd_handler.c:1584:osd_oxc_free()) kfreed 'oxe': 68 at 00000000460814bc. 00010000:00010000:0.0:1776178092.436854:0:3340:0:(ldlm_lockd.c:1903:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: mdt-lustre-MDT0001_UUID lock: 000000005c3ea2a3/0xa40083ab9bf52a89 lrc: 2/0,0 mode: --/COS res: [0x240000402:0x26:0x0].0x883f89b2 bits 0x2/0x0 rrc: 2 type: IBT gid 0 flags: 0x44a19400000000 nid: local remote: 0x0 expref: -99 pid: 8250 timeout: 0 lvb_type: 0 00010000:00010000:0.0:1776178092.436863:0:3340:0:(ldlm_lock.c:216:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: mdt-lustre-MDT0001_UUID lock: 000000005c3ea2a3/0xa40083ab9bf52a89 lrc: 0/0,0 mode: --/COS res: [0x240000402:0x26:0x0].0x883f89b2 bits 0x2/0x0 rrc: 2 type: IBT gid 0 flags: 0x44a19400000000 nid: local remote: 0x0 expref: -99 pid: 8250 timeout: 0 lvb_type: 0 00010000:00010000:0.0:1776178092.436875:0:3340:0:(ldlm_lock.c:820:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(COS) ns: mdt-lustre-MDT0001_UUID lock: 000000001a7d1fc0/0xa40083ab9bf52a90 lrc: 3/0,1 mode: COS/COS res: [0x240000402:0x442:0x0].0x0 bits 0x3/0x0 rrc: 2 type: IBT gid 0 flags: 0x50210000000000 nid: local remote: 0x0 expref: -99 pid: 8250 timeout: 0 lvb_type: 0 00010000:00010000:0.0:1776178092.436882:0:3340:0:(ldlm_lock.c:879:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: mdt-lustre-MDT0001_UUID lock: 000000001a7d1fc0/0xa40083ab9bf52a90 lrc: 2/0,0 mode: COS/COS res: [0x240000402:0x442:0x0].0x0 bits 0x3/0x0 rrc: 2 type: IBT gid 0 flags: 0x50210400000000 nid: local remote: 0x0 expref: -99 pid: 8250 timeout: 0 lvb_type: 0 00010000:00010000:0.0:1776178092.436892:0:3340:0:(ldlm_lockd.c:1879:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: mdt-lustre-MDT0001_UUID lock: 000000001a7d1fc0/0xa40083ab9bf52a90 lrc: 3/0,0 mode: COS/COS res: [0x240000402:0x442:0x0].0x0 bits 0x3/0x0 rrc: 2 type: IBT gid 0 flags: 0x40210400000000 nid: local remote: 0x0 expref: -99 pid: 8250 timeout: 0 lvb_type: 0 00010000:00010000:0.0:1776178092.436899:0:3340:0:(ldlm_lockd.c:1893:ldlm_handle_bl_callback()) Lock 000000001a7d1fc0 already unused, calling callback (0000000079e78348) 00010000:00010000:0.0:1776178092.436901:0:3340:0:(ldlm_request.c:364:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: mdt-lustre-MDT0001_UUID lock: 000000001a7d1fc0/0xa40083ab9bf52a90 lrc: 3/0,0 mode: COS/COS res: [0x240000402:0x442:0x0].0x0 bits 0x3/0x0 rrc: 2 type: IBT gid 0 flags: 0x40210400000000 nid: local remote: 0x0 expref: -99 pid: 8250 timeout: 0 lvb_type: 0 00010000:00010000:0.0:1776178092.436909:0:3340:0:(ldlm_request.c:1257:ldlm_cli_cancel_local()) ### server-side local cancel ns: mdt-lustre-MDT0001_UUID lock: 000000001a7d1fc0/0xa40083ab9bf52a90 lrc: 4/0,0 mode: COS/COS res: [0x240000402:0x442:0x0].0x0 bits 0x3/0x0 rrc: 2 type: IBT gid 0 flags: 0x40218400000000 nid: local remote: 0x0 expref: -99 pid: 8250 timeout: 0 lvb_type: 0 00010000:00010000:0.0:1776178092.436918:0:3340:0:(ldlm_inodebits.c:94:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x240000402:0x442:0x0].0x0 (000000005a05ecaa) 00010000:00010000:0.0:1776178092.436920:0:3340:0:(ldlm_inodebits.c:96:ldlm_reprocess_inodebits_queue()) Hint 3 00010000:00010000:0.0:1776178092.436923:0:3340:0:(ldlm_lockd.c:1903:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: mdt-lustre-MDT0001_UUID lock: 000000001a7d1fc0/0xa40083ab9bf52a90 lrc: 2/0,0 mode: --/COS res: [0x240000402:0x442:0x0].0x0 bits 0x3/0x0 rrc: 2 type: IBT gid 0 flags: 0x44a19400000000 nid: local remote: 0x0 expref: -99 pid: 8250 timeout: 0 lvb_type: 0 00010000:00010000:0.0:1776178092.436930:0:3340:0:(ldlm_lock.c:216:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: mdt-lustre-MDT0001_UUID lock: 000000001a7d1fc0/0xa40083ab9bf52a90 lrc: 0/0,0 mode: --/COS res: [0x240000402:0x442:0x0].0x0 bits 0x3/0x0 rrc: 2 type: IBT gid 0 flags: 0x44a19400000000 nid: local remote: 0x0 expref: -99 pid: 8250 timeout: 0 lvb_type: 0 00010000:00010000:0.0:1776178092.436954:0:3340:0:(ldlm_lock.c:820:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(COS) ns: mdt-lustre-MDT0001_UUID lock: 0000000095145bec/0xa40083ab9bf52aa5 lrc: 3/0,1 mode: COS/COS res: [0x240000402:0x26:0x0].0x97c4dddd bits 0x2/0x0 rrc: 2 type: IBT gid 0 flags: 0x50210000000000 nid: local remote: 0x0 expref: -99 pid: 8250 timeout: 0 lvb_type: 0 00010000:00010000:0.0:1776178092.436968:0:3340:0:(ldlm_lock.c:879:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: mdt-lustre-MDT0001_UUID lock: 0000000095145bec/0xa40083ab9bf52aa5 lrc: 2/0,0 mode: COS/COS res: [0x240000402:0x26:0x0].0x97c4dddd bits 0x2/0x0 rrc: 2 type: IBT gid 0 flags: 0x50210400000000 nid: local remote: 0x0 expref: -99 pid: 8250 timeout: 0 lvb_type: 0 00010000:00010000:0.0:1776178092.436977:0:3340:0:(ldlm_lockd.c:1879:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: mdt-lustre-MDT0001_UUID lock: 0000000095145bec/0xa40083ab9bf52aa5 lrc: 3/0,0 mode: COS/COS res: [0x240000402:0x26:0x0].0x97c4dddd bits 0x2/0x0 rrc: 2 type: IBT gid 0 flags: 0x40210400000000 nid: local remote: 0x0 expref: -99 pid: 8250 timeout: 0 lvb_type: 0 00010000:00010000:0.0:1776178092.436986:0:3340:0:(ldlm_lockd.c:1893:ldlm_handle_bl_callback()) Lock 0000000095145bec already unused, calling callback (0000000079e78348) 00010000:00010000:0.0:1776178092.436988:0:3340:0:(ldlm_request.c:364:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: mdt-lustre-MDT0001_UUID lock: 0000000095145bec/0xa40083ab9bf52aa5 lrc: 3/0,0 mode: COS/COS res: [0x240000402:0x26:0x0].0x97c4dddd bits 0x2/0x0 rrc: 2 type: IBT gid 0 flags: 0x40210400000000 nid: local remote: 0x0 expref: -99 pid: 8250 timeout: 0 lvb_type: 0 00010000:00010000:0.0:1776178092.436994:0:3340:0:(ldlm_request.c:1257:ldlm_cli_cancel_local()) ### server-side local cancel ns: mdt-lustre-MDT0001_UUID lock: 0000000095145bec/0xa40083ab9bf52aa5 lrc: 4/0,0 mode: COS/COS res: [0x240000402:0x26:0x0].0x97c4dddd bits 0x2/0x0 rrc: 2 type: IBT gid 0 flags: 0x40218400000000 nid: local remote: 0x0 expref: -99 pid: 8250 timeout: 0 lvb_type: 0 00010000:00010000:0.0:1776178092.437006:0:3340:0:(ldlm_inodebits.c:94:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x240000402:0x26:0x0].0x97c4dddd (00000000d1cf4cd1) 00010000:00010000:0.0:1776178092.437009:0:3340:0:(ldlm_inodebits.c:96:ldlm_reprocess_inodebits_queue()) Hint 2 00010000:00010000:0.0:1776178092.437011:0:3340:0:(ldlm_lockd.c:1903:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: mdt-lustre-MDT0001_UUID lock: 0000000095145bec/0xa40083ab9bf52aa5 lrc: 2/0,0 mode: --/COS res: [0x240000402:0x26:0x0].0x97c4dddd bits 0x2/0x0 rrc: 2 type: IBT gid 0 flags: 0x44a19400000000 nid: local remote: 0x0 expref: -99 pid: 8250 timeout: 0 lvb_type: 0 00010000:00010000:0.0:1776178092.437019:0:3340:0:(ldlm_lock.c:216:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: mdt-lustre-MDT0001_UUID lock: 0000000095145bec/0xa40083ab9bf52aa5 lrc: 0/0,0 mode: --/COS res: [0x240000402:0x26:0x0].0x97c4dddd bits 0x2/0x0 rrc: 2 type: IBT gid 0 flags: 0x44a19400000000 nid: local remote: 0x0 expref: -99 pid: 8250 timeout: 0 lvb_type: 0 00010000:00010000:0.0:1776178092.437037:0:3340:0:(ldlm_lock.c:820:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(COS) ns: mdt-lustre-MDT0001_UUID lock: 00000000db989acb/0xa40083ab9bf52aac lrc: 3/0,1 mode: COS/COS res: [0x240000402:0x73e:0x0].0x0 bits 0x3/0x0 rrc: 2 type: IBT gid 0 flags: 0x50210000000000 nid: local remote: 0x0 expref: -99 pid: 8250 timeout: 0 lvb_type: 0 00010000:00010000:0.0:1776178092.437045:0:3340:0:(ldlm_lock.c:879:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: mdt-lustre-MDT0001_UUID lock: 00000000db989acb/0xa40083ab9bf52aac lrc: 2/0,0 mode: COS/COS res: [0x240000402:0x73e:0x0].0x0 bits 0x3/0x0 rrc: 2 type: IBT gid 0 flags: 0x50210400000000 nid: local remote: 0x0 expref: -99 pid: 8250 timeout: 0 lvb_type: 0 00010000:00010000:0.0:1776178092.437053:0:3340:0:(ldlm_lockd.c:1879:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: mdt-lustre-MDT0001_UUID lock: 00000000db989acb/0xa40083ab9bf52aac lrc: 3/0,0 mode: COS/COS res: [0x240000402:0x73e:0x0].0x0 bits 0x3/0x0 rrc: 2 type: IBT gid 0 flags: 0x40210400000000 nid: local remote: 0x0 expref: -99 pid: 8250 timeout: 0 lvb_type: 0 00010000:00010000:0.0:1776178092.437108:0:3340:0:(ldlm_lockd.c:1893:ldlm_handle_bl_callback()) Lock 00000000db989acb already unused, calling callback (0000000079e78348) 00010000:00010000:0.0:1776178092.437111:0:3340:0:(ldlm_request.c:364:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: mdt-lustre-MDT0001_UUID lock: 00000000db989acb/0xa40083ab9bf52aac lrc: 3/0,0 mode: COS/COS res: [0x240000402:0x73e:0x0].0x0 bits 0x3/0x0 rrc: 2 type: IBT gid 0 flags: 0x40210400000000 nid: local remote: 0x0 expref: -99 pid: 8250 timeout: 0 lvb_type: 0 00010000:00010000:0.0:1776178092.437120:0:3340:0:(ldlm_request.c:1257:ldlm_cli_cancel_local()) ### server-side local cancel ns: mdt-lustre-MDT0001_UUID lock: 00000000db989acb/0xa40083ab9bf52aac lrc: 4/0,0 mode: COS/COS res: [0x240000402:0x73e:0x0].0x0 bits 0x3/0x0 rrc: 2 type: IBT gid 0 flags: 0x40218400000000 nid: local remote: 0x0 expref: -99 pid: 8250 timeout: 0 lvb_type: 0 00010000:00010000:0.0:1776178092.437127:0:3340:0:(ldlm_inodebits.c:94:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x240000402:0x73e:0x0].0x0 (000000002fef49bf) 00010000:00010000:0.0:1776178092.437129:0:3340:0:(ldlm_inodebits.c:96:ldlm_reprocess_inodebits_queue()) Hint 3 00010000:00010000:0.0:1776178092.437131:0:3340:0:(ldlm_lockd.c:1903:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: mdt-lustre-MDT0001_UUID lock: 00000000db989acb/0xa40083ab9bf52aac lrc: 2/0,0 mode: --/COS res: [0x240000402:0x73e:0x0].0x0 bits 0x3/0x0 rrc: 2 type: IBT gid 0 flags: 0x44a19400000000 nid: local remote: 0x0 expref: -99 pid: 8250 timeout: 0 lvb_type: 0 00010000:00010000:0.0:1776178092.437136:0:3340:0:(ldlm_lock.c:216:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: mdt-lustre-MDT0001_UUID lock: 00000000db989acb/0xa40083ab9bf52aac lrc: 0/0,0 mode: --/COS res: [0x240000402:0x73e:0x0].0x0 bits 0x3/0x0 rrc: 2 type: IBT gid 0 flags: 0x44a19400000000 nid: local remote: 0x0 expref: -99 pid: 8250 timeout: 0 lvb_type: 0 00010000:00010000:0.0:1776178092.437148:0:3340:0:(ldlm_lock.c:820:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(COS) ns: mdt-lustre-MDT0001_UUID lock: 000000009a100e86/0xa40083ab9bf52ac1 lrc: 3/0,1 mode: COS/COS res: [0x240000402:0x26:0x0].0xa062ba40 bits 0x2/0x0 rrc: 2 type: IBT gid 0 flags: 0x50210000000000 nid: local remote: 0x0 expref: -99 pid: 6257 timeout: 0 lvb_type: 0 00010000:00010000:0.0:1776178092.437153:0:3340:0:(ldlm_lock.c:879:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: mdt-lustre-MDT0001_UUID lock: 000000009a100e86/0xa40083ab9bf52ac1 lrc: 2/0,0 mode: COS/COS res: [0x240000402:0x26:0x0].0xa062ba40 bits 0x2/0x0 rrc: 2 type: IBT gid 0 flags: 0x50210400000000 nid: local remote: 0x0 expref: -99 pid: 6257 timeout: 0 lvb_type: 0 00010000:00010000:0.0:1776178092.437157:0:3340:0:(ldlm_lockd.c:1879:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: mdt-lustre-MDT0001_UUID lock: 000000009a100e86/0xa40083ab9bf52ac1 lrc: 3/0,0 mode: COS/COS res: [0x240000402:0x26:0x0].0xa062ba40 bits 0x2/0x0 rrc: 2 type: IBT gid 0 flags: 0x40210400000000 nid: local remote: 0x0 expref: -99 pid: 6257 timeout: 0 lvb_type: 0 00010000:00010000:0.0:1776178092.437162:0:3340:0:(ldlm_lockd.c:1893:ldlm_handle_bl_callback()) Lock 000000009a100e86 already unused, calling callback (0000000079e78348) 00010000:00010000:0.0:1776178092.437163:0:3340:0:(ldlm_request.c:364:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: mdt-lustre-MDT0001_UUID lock: 000000009a100e86/0xa40083ab9bf52ac1 lrc: 3/0,0 mode: COS/COS res: [0x240000402:0x26:0x0].0xa062ba40 bits 0x2/0x0 rrc: 2 type: IBT gid 0 flags: 0x40210400000000 nid: local remote: 0x0 expref: -99 pid: 6257 timeout: 0 lvb_type: 0 00010000:00010000:0.0:1776178092.437168:0:3340:0:(ldlm_request.c:1257:ldlm_cli_cancel_local()) ### server-side local cancel ns: mdt-lustre-MDT0001_UUID lock: 000000009a100e86/0xa40083ab9bf52ac1 lrc: 4/0,0 mode: COS/COS res: [0x240000402:0x26:0x0].0xa062ba40 bits 0x2/0x0 rrc: 2 type: IBT gid 0 flags: 0x40218400000000 nid: local remote: 0x0 expref: -99 pid: 6257 timeout: 0 lvb_type: 0 00010000:00010000:0.0:1776178092.437175:0:3340:0:(ldlm_inodebits.c:94:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x240000402:0x26:0x0].0xa062ba40 (00000000e1806ab9) 00010000:00010000:0.0:1776178092.437177:0:3340:0:(ldlm_inodebits.c:96:ldlm_reprocess_inodebits_queue()) Hint 2 00010000:00010000:0.0:1776178092.437179:0:3340:0:(ldlm_lockd.c:1903:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: mdt-lustre-MDT0001_UUID lock: 000000009a100e86/0xa40083ab9bf52ac1 lrc: 2/0,0 mode: --/COS res: [0x240000402:0x26:0x0].0xa062ba40 bits 0x2/0x0 rrc: 2 type: IBT gid 0 flags: 0x44a19400000000 nid: local remote: 0x0 expref: -99 pid: 6257 timeout: 0 lvb_type: 0 00010000:00010000:0.0:1776178092.437184:0:3340:0:(ldlm_lock.c:216:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: mdt-lustre-MDT0001_UUID lock: 000000009a100e86/0xa40083ab9bf52ac1 lrc: 0/0,0 mode: --/COS res: [0x240000402:0x26:0x0].0xa062ba40 bits 0x2/0x0 rrc: 2 type: IBT gid 0 flags: 0x44a19400000000 nid: local remote: 0x0 expref: -99 pid: 6257 timeout: 0 lvb_type: 0 00010000:00010000:0.0:1776178092.437195:0:3340:0:(ldlm_lock.c:820:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(COS) ns: mdt-lustre-MDT0001_UUID lock: 00000000fe617411/0xa40083ab9bf52ac8 lrc: 3/0,1 mode: COS/COS res: [0x240000402:0x57b:0x0].0x0 bits 0x3/0x0 rrc: 2 type: IBT gid 0 flags: 0x50210000000000 nid: local remote: 0x0 expref: -99 pid: 6257 timeout: 0 lvb_type: 0 00010000:00010000:0.0:1776178092.437201:0:3340:0:(ldlm_lock.c:879:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: mdt-lustre-MDT0001_UUID lock: 00000000fe617411/0xa40083ab9bf52ac8 lrc: 2/0,0 mode: COS/COS res: [0x240000402:0x57b:0x0].0x0 bits 0x3/0x0 rrc: 2 type: IBT gid 0 flags: 0x50210400000000 nid: local remote: 0x0 expref: -99 pid: 6257 timeout: 0 lvb_type: 0 00010000:00010000:0.0:1776178092.437207:0:3340:0:(ldlm_lockd.c:1879:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: mdt-lustre-MDT0001_UUID lock: 00000000fe617411/0xa40083ab9bf52ac8 lrc: 3/0,0 mode: COS/COS res: [0x240000402:0x57b:0x0].0x0 bits 0x3/0x0 rrc: 2 type: IBT gid 0 flags: 0x40210400000000 nid: local remote: 0x0 expref: -99 pid: 6257 timeout: 0 lvb_type: 0 00010000:00010000:0.0:1776178092.437213:0:3340:0:(ldlm_lockd.c:1893:ldlm_handle_bl_callback()) Lock 00000000fe617411 already unused, calling callback (0000000079e78348) 00010000:00010000:0.0:1776178092.437215:0:3340:0:(ldlm_request.c:364:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: mdt-lustre-MDT0001_UUID lock: 00000000fe617411/0xa40083ab9bf52ac8 lrc: 3/0,0 mode: COS/COS res: [0x240000402:0x57b:0x0].0x0 bits 0x3/0x0 rrc: 2 type: IBT gid 0 flags: 0x40210400000000 nid: local remote: 0x0 expref: -99 pid: 6257 timeout: 0 lvb_type: 0 00010000:00010000:0.0:1776178092.437220:0:3340:0:(ldlm_request.c:1257:ldlm_cli_cancel_local()) ### server-side local cancel ns: mdt-lustre-MDT0001_UUID lock: 00000000fe617411/0xa40083ab9bf52ac8 lrc: 4/0,0 mode: COS/COS res: [0x240000402:0x57b:0x0].0x0 bits 0x3/0x0 rrc: 2 type: IBT gid 0 flags: 0x40218400000000 nid: local remote: 0x0 expref: -99 pid: 6257 timeout: 0 lvb_type: 0 00010000:00010000:0.0:1776178092.437227:0:3340:0:(ldlm_inodebits.c:94:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x240000402:0x57b:0x0].0x0 (0000000002803edd) 00010000:00010000:0.0:1776178092.437229:0:3340:0:(ldlm_inodebits.c:96:ldlm_reprocess_inodebits_queue()) Hint 3 00010000:00010000:0.0:1776178092.437231:0:3340:0:(ldlm_lockd.c:1903:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: mdt-lustre-MDT0001_UUID lock: 00000000fe617411/0xa40083ab9bf52ac8 lrc: 2/0,0 mode: --/COS res: [0x240000402:0x57b:0x0].0x0 bits 0x3/0x0 rrc: 2 type: IBT gid 0 flags: 0x44a19400000000 nid: local remote: 0x0 expref: -99 pid: 6257 timeout: 0 lvb_type: 0 00010000:00010000:0.0:1776178092.437242:0:3340:0:(ldlm_lock.c:216:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: mdt-lustre-MDT0001_UUID lock: 00000000fe617411/0xa40083ab9bf52ac8 lrc: 0/0,0 mode: --/COS res: [0x240000402:0x57b:0x0].0x0 bits 0x3/0x0 rrc: 2 type: IBT gid 0 flags: 0x44a19400000000 nid: local remote: 0x0 expref: -99 pid: 6257 timeout: 0 lvb_type: 0 00010000:00010000:0.0:1776178092.437267:0:3340:0:(ldlm_lock.c:820:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(COS) ns: mdt-lustre-MDT0001_UUID lock: 000000008987f296/0xa40083ab9bf52add lrc: 3/0,1 mode: COS/COS res: [0x240000402:0x26:0x0].0x980417b2 bits 0x2/0x0 rrc: 2 type: IBT gid 0 flags: 0x50210000000000 nid: local remote: 0x0 expref: -99 pid: 8250 timeout: 0 lvb_type: 0 00010000:00010000:0.0:1776178092.437275:0:3340:0:(ldlm_lock.c:879:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: mdt-lustre-MDT0001_UUID lock: 000000008987f296/0xa40083ab9bf52add lrc: 2/0,0 mode: COS/COS res: [0x240000402:0x26:0x0].0x980417b2 bits 0x2/0x0 rrc: 2 type: IBT gid 0 flags: 0x50210400000000 nid: local remote: 0x0 expref: -99 pid: 8250 timeout: 0 lvb_type: 0 00010000:00010000:0.0:1776178092.437284:0:3340:0:(ldlm_lockd.c:1879:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: mdt-lustre-MDT0001_UUID lock: 000000008987f296/0xa40083ab9bf52add lrc: 3/0,0 mode: COS/COS res: [0x240000402:0x26:0x0].0x980417b2 bits 0x2/0x0 rrc: 2 type: IBT gid 0 flags: 0x40210400000000 nid: local remote: 0x0 expref: -99 pid: 8250 timeout: 0 lvb_type: 0 00010000:00010000:0.0:1776178092.437291:0:3340:0:(ldlm_lockd.c:1893:ldlm_handle_bl_callback()) Lock 000000008987f296 already unused, calling callback (0000000079e78348) 00010000:00010000:0.0:1776178092.437294:0:3340:0:(ldlm_request.c:364:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: mdt-lustre-MDT0001_UUID lock: 000000008987f296/0xa40083ab9bf52add lrc: 3/0,0 mode: COS/COS res: [0x240000402:0x26:0x0].0x980417b2 bits 0x2/0x0 rrc: 2 type: IBT gid 0 flags: 0x40210400000000 nid: local remote: 0x0 expref: -99 pid: 8250 timeout: 0 lvb_type: 0 00010000:00010000:0.0:1776178092.437302:0:3340:0:(ldlm_request.c:1257:ldlm_cli_cancel_local()) ### server-side local cancel ns: mdt-lustre-MDT0001_UUID lock: 000000008987f296/0xa40083ab9bf52add lrc: 4/0,0 mode: COS/COS res: [0x240000402:0x26:0x0].0x980417b2 bits 0x2/0x0 rrc: 2 type: IBT gid 0 flags: 0x40218400000000 nid: local remote: 0x0 expref: -99 pid: 8250 timeout: 0 lvb_type: 0 00010000:00010000:0.0:1776178092.437313:0:3340:0:(ldlm_inodebits.c:94:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x240000402:0x26:0x0].0x980417b2 (00000000022e9407) 00010000:00010000:0.0:1776178092.437317:0:3340:0:(ldlm_inodebits.c:96:ldlm_reprocess_inodebits_queue()) Hint 2 00010000:00010000:0.0:1776178092.437320:0:3340:0:(ldlm_lockd.c:1903:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: mdt-lustre-MDT0001_UUID lock: 000000008987f296/0xa40083ab9bf52add lrc: 2/0,0 mode: --/COS res: [0x240000402:0x26:0x0].0x980417b2 bits 0x2/0x0 rrc: 2 type: IBT gid 0 flags: 0x44a19400000000 nid: local remote: 0x0 expref: -99 pid: 8250 timeout: 0 lvb_type: 0 00010000:00010000:0.0:1776178092.437327:0:3340:0:(ldlm_lock.c:216:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: mdt-lustre-MDT0001_UUID lock: 000000008987f296/0xa40083ab9bf52add lrc: 0/0,0 mode: --/COS res: [0x240000402:0x26:0x0].0x980417b2 bits 0x2/0x0 rrc: 2 type: IBT gid 0 flags: 0x44a19400000000 nid: local remote: 0x0 expref: -99 pid: 8250 timeout: 0 lvb_type: 0 00010000:00010000:0.0:1776178092.437344:0:3340:0:(ldlm_lock.c:820:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(COS) ns: mdt-lustre-MDT0001_UUID lock: 0000000027cb0800/0xa40083ab9bf52ae4 lrc: 3/0,1 mode: COS/COS res: [0x240000402:0x60a:0x0].0x0 bits 0x3/0x0 rrc: 2 type: IBT gid 0 flags: 0x50210000000000 nid: local remote: 0x0 expref: -99 pid: 8250 timeout: 0 lvb_type: 0 00010000:00010000:0.0:1776178092.437358:0:3340:0:(ldlm_lock.c:879:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: mdt-lustre-MDT0001_UUID lock: 0000000027cb0800/0xa40083ab9bf52ae4 lrc: 2/0,0 mode: COS/COS res: [0x240000402:0x60a:0x0].0x0 bits 0x3/0x0 rrc: 2 type: IBT gid 0 flags: 0x50210400000000 nid: local remote: 0x0 expref: -99 pid: 8250 timeout: 0 lvb_type: 0 00010000:00010000:0.0:1776178092.437366:0:3340:0:(ldlm_lockd.c:1879:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: mdt-lustre-MDT0001_UUID lock: 0000000027cb0800/0xa40083ab9bf52ae4 lrc: 3/0,0 mode: COS/COS res: [0x240000402:0x60a:0x0].0x0 bits 0x3/0x0 rrc: 2 type: IBT gid 0 flags: 0x40210400000000 nid: local remote: 0x0 expref: -99 pid: 8250 timeout: 0 lvb_type: 0 00010000:00010000:0.0:1776178092.437374:0:3340:0:(ldlm_lockd.c:1893:ldlm_handle_bl_callback()) Lock 0000000027cb0800 already unused, calling callback (0000000079e78348) 00010000:00010000:0.0:1776178092.437378:0:3340:0:(ldlm_request.c:364:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: mdt-lustre-MDT0001_UUID lock: 0000000027cb0800/0xa40083ab9bf52ae4 lrc: 3/0,0 mode: COS/COS res: [0x240000402:0x60a:0x0].0x0 bits 0x3/0x0 rrc: 2 type: IBT gid 0 flags: 0x40210400000000 nid: local remote: 0x0 expref: -99 pid: 8250 timeout: 0 lvb_type: 0 00010000:00010000:0.0:1776178092.437385:0:3340:0:(ldlm_request.c:1257:ldlm_cli_cancel_local()) ### server-side local cancel ns: mdt-lustre-MDT0001_UUID lock: 0000000027cb0800/0xa40083ab9bf52ae4 lrc: 4/0,0 mode: COS/COS res: [0x240000402:0x60a:0x0].0x0 bits 0x3/0x0 rrc: 2 type: IBT gid 0 flags: 0x40218400000000 nid: local remote: 0x0 expref: -99 pid: 8250 timeout: 0 lvb_type: 0 00010000:00010000:0.0:1776178092.437396:0:3340:0:(ldlm_inodebits.c:94:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x240000402:0x60a:0x0].0x0 (00000000581be35c) 00010000:00010000:0.0:1776178092.437399:0:3340:0:(ldlm_inodebits.c:96:ldlm_reprocess_inodebits_queue()) Hint 3 00010000:00010000:0.0:1776178092.437402:0:3340:0:(ldlm_lockd.c:1903:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: mdt-lustre-MDT0001_UUID lock: 0000000027cb0800/0xa40083ab9bf52ae4 lrc: 2/0,0 mode: --/COS res: [0x240000402:0x60a:0x0].0x0 bits 0x3/0x0 rrc: 2 type: IBT gid 0 flags: 0x44a19400000000 nid: local remote: 0x0 expref: -99 pid: 8250 timeout: 0 lvb_type: 0 00010000:00010000:0.0:1776178092.437409:0:3340:0:(ldlm_lock.c:216:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: mdt-lustre-MDT0001_UUID lock: 0000000027cb0800/0xa40083ab9bf52ae4 lrc: 0/0,0 mode: --/COS res: [0x240000402:0x60a:0x0].0x0 bits 0x3/0x0 rrc: 2 type: IBT gid 0 flags: 0x44a19400000000 nid: local remote: 0x0 expref: -99 pid: 8250 timeout: 0 lvb_type: 0 00010000:00010000:0.0:1776178092.437434:0:3340:0:(ldlm_lock.c:820:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(COS) ns: mdt-lustre-MDT0001_UUID lock: 000000001b3f1324/0xa40083ab9bf52af9 lrc: 3/0,1 mode: COS/COS res: [0x240000402:0x26:0x0].0x681a89b2 bits 0x2/0x0 rrc: 2 type: IBT gid 0 flags: 0x50210000000000 nid: local remote: 0x0 expref: -99 pid: 8250 timeout: 0 lvb_type: 0 00010000:00010000:0.0:1776178092.437444:0:3340:0:(ldlm_lock.c:879:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: mdt-lustre-MDT0001_UUID lock: 000000001b3f1324/0xa40083ab9bf52af9 lrc: 2/0,0 mode: COS/COS res: [0x240000402:0x26:0x0].0x681a89b2 bits 0x2/0x0 rrc: 2 type: IBT gid 0 flags: 0x50210400000000 nid: local remote: 0x0 expref: -99 pid: 8250 timeout: 0 lvb_type: 0 00010000:00010000:0.0:1776178092.437452:0:3340:0:(ldlm_lockd.c:1879:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: mdt-lustre-MDT0001_UUID lock: 000000001b3f1324/0xa40083ab9bf52af9 lrc: 3/0,0 mode: COS/COS res: [0x240000402:0x26:0x0].0x681a89b2 bits 0x2/0x0 rrc: 2 type: IBT gid 0 flags: 0x40210400000000 nid: local remote: 0x0 expref: -99 pid: 8250 timeout: 0 lvb_type: 0 00010000:00010000:0.0:1776178092.437466:0:3340:0:(ldlm_lockd.c:1893:ldlm_handle_bl_callback()) Lock 000000001b3f1324 already unused, calling callback (0000000079e78348) 00010000:00010000:0.0:1776178092.437470:0:3340:0:(ldlm_request.c:364:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: mdt-lustre-MDT0001_UUID lock: 000000001b3f1324/0xa40083ab9bf52af9 lrc: 3/0,0 mode: COS/COS res: [0x240000402:0x26:0x0].0x681a89b2 bits 0x2/0x0 rrc: 2 type: IBT gid 0 flags: 0x40210400000000 nid: local remote: 0x0 expref: -99 pid: 8250 timeout: 0 lvb_type: 0 00010000:00010000:0.0:1776178092.437478:0:3340:0:(ldlm_request.c:1257:ldlm_cli_cancel_local()) ### server-side local cancel ns: mdt-lustre-MDT0001_UUID lock: 000000001b3f1324/0xa40083ab9bf52af9 lrc: 4/0,0 mode: COS/COS res: [0x240000402:0x26:0x0].0x681a89b2 bits 0x2/0x0 rrc: 2 type: IBT gid 0 flags: 0x40218400000000 nid: local remote: 0x0 expref: -99 pid: 8250 timeout: 0 lvb_type: 0 00010000:00010000:0.0:1776178092.437490:0:3340:0:(ldlm_inodebits.c:94:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x240000402:0x26:0x0].0x681a89b2 (00000000c1869e41) 00010000:00010000:0.0:1776178092.437493:0:3340:0:(ldlm_inodebits.c:96:ldlm_reprocess_inodebits_queue()) Hint 2 00010000:00010000:0.0:1776178092.437495:0:3340:0:(ldlm_lockd.c:1903:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: mdt-lustre-MDT0001_UUID lock: 000000001b3f1324/0xa40083ab9bf52af9 lrc: 2/0,0 mode: --/COS res: [0x240000402:0x26:0x0].0x681a89b2 bits 0x2/0x0 rrc: 2 type: IBT gid 0 flags: 0x44a19400000000 nid: local remote: 0x0 expref: -99 pid: 8250 timeout: 0 lvb_type: 0 00010000:00010000:0.0:1776178092.437504:0:3340:0:(ldlm_lock.c:216:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: mdt-lustre-MDT0001_UUID lock: 000000001b3f1324/0xa40083ab9bf52af9 lrc: 0/0,0 mode: --/COS res: [0x240000402:0x26:0x0].0x681a89b2 bits 0x2/0x0 rrc: 2 type: IBT gid 0 flags: 0x44a19400000000 nid: local remote: 0x0 expref: -99 pid: 8250 timeout: 0 lvb_type: 0 00010000:00010000:0.0:1776178092.437525:0:3340:0:(ldlm_lock.c:820:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(COS) ns: mdt-lustre-MDT0001_UUID lock: 00000000575572c2/0xa40083ab9bf52b00 lrc: 3/0,1 mode: COS/COS res: [0x240000402:0x619:0x0].0x0 bits 0x3/0x0 rrc: 2 type: IBT gid 0 flags: 0x50210000000000 nid: local remote: 0x0 expref: -99 pid: 8250 timeout: 0 lvb_type: 0 00010000:00010000:0.0:1776178092.437535:0:3340:0:(ldlm_lock.c:879:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: mdt-lustre-MDT0001_UUID lock: 00000000575572c2/0xa40083ab9bf52b00 lrc: 2/0,0 mode: COS/COS res: [0x240000402:0x619:0x0].0x0 bits 0x3/0x0 rrc: 2 type: IBT gid 0 flags: 0x50210400000000 nid: local remote: 0x0 expref: -99 pid: 8250 timeout: 0 lvb_type: 0 00010000:00010000:0.0:1776178092.437543:0:3340:0:(ldlm_lockd.c:1879:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: mdt-lustre-MDT0001_UUID lock: 00000000575572c2/0xa40083ab9bf52b00 lrc: 3/0,0 mode: COS/COS res: [0x240000402:0x619:0x0].0x0 bits 0x3/0x0 rrc: 2 type: IBT gid 0 flags: 0x40210400000000 nid: local remote: 0x0 expref: -99 pid: 8250 timeout: 0 lvb_type: 0 00010000:00010000:0.0:1776178092.437552:0:3340:0:(ldlm_lockd.c:1893:ldlm_handle_bl_callback()) Lock 00000000575572c2 already unused, calling callback (0000000079e78348) 00010000:00010000:0.0:1776178092.437555:0:3340:0:(ldlm_request.c:364:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: mdt-lustre-MDT0001_UUID lock: 00000000575572c2/0xa40083ab9bf52b00 lrc: 3/0,0 mode: COS/COS res: [0x240000402:0x619:0x0].0x0 bits 0x3/0x0 rrc: 2 type: IBT gid 0 flags: 0x40210400000000 nid: local remote: 0x0 expref: -99 pid: 8250 timeout: 0 lvb_type: 0 00010000:00010000:0.0:1776178092.437563:0:3340:0:(ldlm_request.c:1257:ldlm_cli_cancel_local()) ### server-side local cancel ns: mdt-lustre-MDT0001_UUID lock: 00000000575572c2/0xa40083ab9bf52b00 lrc: 4/0,0 mode: COS/COS res: [0x240000402:0x619:0x0].0x0 bits 0x3/0x0 rrc: 2 type: IBT gid 0 flags: 0x40218400000000 nid: local remote: 0x0 expref: -99 pid: 8250 timeout: 0 lvb_type: 0 00010000:00010000:0.0:1776178092.437576:0:3340:0:(ldlm_inodebits.c:94:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x240000402:0x619:0x0].0x0 (00000000c340a5fc) 00010000:00010000:0.0:1776178092.437579:0:3340:0:(ldlm_inodebits.c:96:ldlm_reprocess_inodebits_queue()) Hint 3 00010000:00010000:0.0:1776178092.437582:0:3340:0:(ldlm_lockd.c:1903:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: mdt-lustre-MDT0001_UUID lock: 00000000575572c2/0xa40083ab9bf52b00 lrc: 2/0,0 mode: --/COS res: [0x240000402:0x619:0x0].0x0 bits 0x3/0x0 rrc: 2 type: IBT gid 0 flags: 0x44a19400000000 nid: local remote: 0x0 expref: -99 pid: 8250 timeout: 0 lvb_type: 0 00010000:00010000:0.0:1776178092.437591:0:3340:0:(ldlm_lock.c:216:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: mdt-lustre-MDT0001_UUID lock: 00000000575572c2/0xa40083ab9bf52b00 lrc: 0/0,0 mode: --/COS res: [0x240000402:0x619:0x0].0x0 bits 0x3/0x0 rrc: 2 type: IBT gid 0 flags: 0x44a19400000000 nid: local remote: 0x0 expref: -99 pid: 8250 timeout: 0 lvb_type: 0 00010000:00010000:0.0:1776178092.437615:0:3340:0:(ldlm_lock.c:820:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(COS) ns: mdt-lustre-MDT0001_UUID lock: 00000000dcea299c/0xa40083ab9bf52b15 lrc: 3/0,1 mode: COS/COS res: [0x240000402:0x26:0x0].0x29771d40 bits 0x2/0x0 rrc: 2 type: IBT gid 0 flags: 0x50210000000000 nid: local remote: 0x0 expref: -99 pid: 8250 timeout: 0 lvb_type: 0 00010000:00010000:0.0:1776178092.437625:0:3340:0:(ldlm_lock.c:879:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: mdt-lustre-MDT0001_UUID lock: 00000000dcea299c/0xa40083ab9bf52b15 lrc: 2/0,0 mode: COS/COS res: [0x240000402:0x26:0x0].0x29771d40 bits 0x2/0x0 rrc: 2 type: IBT gid 0 flags: 0x50210400000000 nid: local remote: 0x0 expref: -99 pid: 8250 timeout: 0 lvb_type: 0 00010000:00010000:0.0:1776178092.437633:0:3340:0:(ldlm_lockd.c:1879:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: mdt-lustre-MDT0001_UUID lock: 00000000dcea299c/0xa40083ab9bf52b15 lrc: 3/0,0 mode: COS/COS res: [0x240000402:0x26:0x0].0x29771d40 bits 0x2/0x0 rrc: 2 type: IBT gid 0 flags: 0x40210400000000 nid: local remote: 0x0 expref: -99 pid: 8250 timeout: 0 lvb_type: 0 00010000:00010000:0.0:1776178092.437641:0:3340:0:(ldlm_lockd.c:1893:ldlm_handle_bl_callback()) Lock 00000000dcea299c already unused, calling callback (0000000079e78348) 00010000:00010000:0.0:1776178092.437644:0:3340:0:(ldlm_request.c:364:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: mdt-lustre-MDT0001_UUID lock: 00000000dcea299c/0xa40083ab9bf52b15 lrc: 3/0,0 mode: COS/COS res: [0x240000402:0x26:0x0].0x29771d40 bits 0x2/0x0 rrc: 2 type: IBT gid 0 flags: 0x40210400000000 nid: local remote: 0x0 expref: -99 pid: 8250 timeout: 0 lvb_type: 0 00010000:00010000:0.0:1776178092.437651:0:3340:0:(ldlm_request.c:1257:ldlm_cli_cancel_local()) ### server-side local cancel ns: mdt-lustre-MDT0001_UUID lock: 00000000dcea299c/0xa40083ab9bf52b15 lrc: 4/0,0 mode: COS/COS res: [0x240000402:0x26:0x0].0x29771d40 bits 0x2/0x0 rrc: 2 type: IBT gid 0 flags: 0x40218400000000 nid: local remote: 0x0 expref: -99 pid: 8250 timeout: 0 lvb_type: 0 00010000:00010000:0.0:1776178092.437663:0:3340:0:(ldlm_inodebits.c:94:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x240000402:0x26:0x0].0x29771d40 (00000000cdb4c68d) 00010000:00010000:0.0:1776178092.437667:0:3340:0:(ldlm_inodebits.c:96:ldlm_reprocess_inodebits_queue()) Hint 2 00010000:00010000:0.0:1776178092.437669:0:3340:0:(ldlm_lockd.c:1903:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: mdt-lustre-MDT0001_UUID lock: 00000000dcea299c/0xa40083ab9bf52b15 lrc: 2/0,0 mode: --/COS res: [0x240000402:0x26:0x0].0x29771d40 bits 0x2/0x0 rrc: 2 type: IBT gid 0 flags: 0x44a19400000000 nid: local remote: 0x0 expref: -99 pid: 8250 timeout: 0 lvb_type: 0 00010000:00010000:0.0:1776178092.437681:0:3340:0:(ldlm_lock.c:216:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: mdt-lustre-MDT0001_UUID lock: 00000000dcea299c/0xa40083ab9bf52b15 lrc: 0/0,0 mode: --/COS res: [0x240000402:0x26:0x0].0x29771d40 bits 0x2/0x0 rrc: 2 type: IBT gid 0 flags: 0x44a19400000000 nid: local remote: 0x0 expref: -99 pid: 8250 timeout: 0 lvb_type: 0 00010000:00010000:0.0:1776178092.437698:0:3340:0:(ldlm_lock.c:820:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(COS) ns: mdt-lustre-MDT0001_UUID lock: 0000000035cf61b1/0xa40083ab9bf52b1c lrc: 3/0,1 mode: COS/COS res: [0x240000402:0x546:0x0].0x0 bits 0x3/0x0 rrc: 2 type: IBT gid 0 flags: 0x50210000000000 nid: local remote: 0x0 expref: -99 pid: 8250 timeout: 0 lvb_type: 0 00010000:00010000:0.0:1776178092.437707:0:3340:0:(ldlm_lock.c:879:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: mdt-lustre-MDT0001_UUID lock: 0000000035cf61b1/0xa40083ab9bf52b1c lrc: 2/0,0 mode: COS/COS res: [0x240000402:0x546:0x0].0x0 bits 0x3/0x0 rrc: 2 type: IBT gid 0 flags: 0x50210400000000 nid: local remote: 0x0 expref: -99 pid: 8250 timeout: 0 lvb_type: 0 00010000:00010000:0.0:1776178092.437718:0:3340:0:(ldlm_lockd.c:1879:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: mdt-lustre-MDT0001_UUID lock: 0000000035cf61b1/0xa40083ab9bf52b1c lrc: 3/0,0 mode: COS/COS res: [0x240000402:0x546:0x0].0x0 bits 0x3/0x0 rrc: 2 type: IBT gid 0 flags: 0x40210400000000 nid: local remote: 0x0 expref: -99 pid: 8250 timeout: 0 lvb_type: 0 00010000:00010000:0.0:1776178092.437726:0:3340:0:(ldlm_lockd.c:1893:ldlm_handle_bl_callback()) Lock 0000000035cf61b1 already unused, calling callback (0000000079e78348) 00010000:00010000:0.0:1776178092.437729:0:3340:0:(ldlm_request.c:364:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: mdt-lustre-MDT0001_UUID lock: 0000000035cf61b1/0xa40083ab9bf52b1c lrc: 3/0,0 mode: COS/COS res: [0x240000402:0x546:0x0].0x0 bits 0x3/0x0 rrc: 2 type: IBT gid 0 flags: 0x40210400000000 nid: local remote: 0x0 expref: -99 pid: 8250 timeout: 0 lvb_type: 0 00010000:00010000:0.0:1776178092.437738:0:3340:0:(ldlm_request.c:1257:ldlm_cli_cancel_local()) ### server-side local cancel ns: mdt-lustre-MDT0001_UUID lock: 0000000035cf61b1/0xa40083ab9bf52b1c lrc: 4/0,0 mode: COS/COS res: [0x240000402:0x546:0x0].0x0 bits 0x3/0x0 rrc: 2 type: IBT gid 0 flags: 0x40218400000000 nid: local remote: 0x0 expref: -99 pid: 8250 timeout: 0 lvb_type: 0 00010000:00010000:0.0:1776178092.437749:0:3340:0:(ldlm_inodebits.c:94:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x240000402:0x546:0x0].0x0 (00000000d4d1ab24) 00010000:00010000:0.0:1776178092.437752:0:3340:0:(ldlm_inodebits.c:96:ldlm_reprocess_inodebits_queue()) Hint 3 00010000:00010000:0.0:1776178092.437755:0:3340:0:(ldlm_lockd.c:1903:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: mdt-lustre-MDT0001_UUID lock: 0000000035cf61b1/0xa40083ab9bf52b1c lrc: 2/0,0 mode: --/COS res: [0x240000402:0x546:0x0].0x0 bits 0x3/0x0 rrc: 2 type: IBT gid 0 flags: 0x44a19400000000 nid: local remote: 0x0 expref: -99 pid: 8250 timeout: 0 lvb_type: 0 00010000:00010000:0.0:1776178092.437765:0:3340:0:(ldlm_lock.c:216:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: mdt-lustre-MDT0001_UUID lock: 0000000035cf61b1/0xa40083ab9bf52b1c lrc: 0/0,0 mode: --/COS res: [0x240000402:0x546:0x0].0x0 bits 0x3/0x0 rrc: 2 type: IBT gid 0 flags: 0x44a19400000000 nid: local remote: 0x0 expref: -99 pid: 8250 timeout: 0 lvb_type: 0 00010000:00010000:0.0:1776178092.437782:0:3340:0:(ldlm_lock.c:820:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(COS) ns: mdt-lustre-MDT0001_UUID lock: 00000000250b39ef/0xa40083ab9bf52b31 lrc: 3/0,1 mode: COS/COS res: [0x240000402:0x26:0x0].0x53564696 bits 0x2/0x0 rrc: 2 type: IBT gid 0 flags: 0x50210000000000 nid: local remote: 0x0 expref: -99 pid: 8250 timeout: 0 lvb_type: 0 00010000:00010000:0.0:1776178092.437791:0:3340:0:(ldlm_lock.c:879:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: mdt-lustre-MDT0001_UUID lock: 00000000250b39ef/0xa40083ab9bf52b31 lrc: 2/0,0 mode: COS/COS res: [0x240000402:0x26:0x0].0x53564696 bits 0x2/0x0 rrc: 2 type: IBT gid 0 flags: 0x50210400000000 nid: local remote: 0x0 expref: -99 pid: 8250 timeout: 0 lvb_type: 0 00010000:00010000:0.0:1776178092.437795:0:3340:0:(ldlm_lockd.c:1879:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: mdt-lustre-MDT0001_UUID lock: 00000000250b39ef/0xa40083ab9bf52b31 lrc: 3/0,0 mode: COS/COS res: [0x240000402:0x26:0x0].0x53564696 bits 0x2/0x0 rrc: 2 type: IBT gid 0 flags: 0x40210400000000 nid: local remote: 0x0 expref: -99 pid: 8250 timeout: 0 lvb_type: 0 00010000:00010000:0.0:1776178092.437800:0:3340:0:(ldlm_lockd.c:1893:ldlm_handle_bl_callback()) Lock 00000000250b39ef already unused, calling callback (0000000079e78348) 00010000:00010000:0.0:1776178092.437801:0:3340:0:(ldlm_request.c:364:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: mdt-lustre-MDT0001_UUID lock: 00000000250b39ef/0xa40083ab9bf52b31 lrc: 3/0,0 mode: COS/COS res: [0x240000402:0x26:0x0].0x53564696 bits 0x2/0x0 rrc: 2 type: IBT gid 0 flags: 0x40210400000000 nid: local remote: 0x0 expref: -99 pid: 8250 timeout: 0 lvb_type: 0 00010000:00010000:0.0:1776178092.437807:0:3340:0:(ldlm_request.c:1257:ldlm_cli_cancel_local()) ### server-side local cancel ns: mdt-lustre-MDT0001_UUID lock: 00000000250b39ef/0xa40083ab9bf52b31 lrc: 4/0,0 mode: COS/COS res: [0x240000402:0x26:0x0].0x53564696 bits 0x2/0x0 rrc: 2 type: IBT gid 0 flags: 0x40218400000000 nid: local remote: 0x0 expref: -99 pid: 8250 timeout: 0 lvb_type: 0 00010000:00010000:0.0:1776178092.437814:0:3340:0:(ldlm_inodebits.c:94:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x240000402:0x26:0x0].0x53564696 (00000000738b6aff) 00010000:00010000:0.0:1776178092.437816:0:3340:0:(ldlm_inodebits.c:96:ldlm_reprocess_inodebits_queue()) Hint 2 00010000:00010000:0.0:1776178092.437819:0:3340:0:(ldlm_lockd.c:1903:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: mdt-lustre-MDT0001_UUID lock: 00000000250b39ef/0xa40083ab9bf52b31 lrc: 2/0,0 mode: --/COS res: [0x240000402:0x26:0x0].0x53564696 bits 0x2/0x0 rrc: 2 type: IBT gid 0 flags: 0x44a19400000000 nid: local remote: 0x0 expref: -99 pid: 8250 timeout: 0 lvb_type: 0 00010000:00010000:0.0:1776178092.443280:0:3340:0:(ldlm_lock.c:216:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: mdt-lustre-MDT0001_UUID lock: 00000000250b39ef/0xa40083ab9bf52b31 lrc: 0/0,0 mode: --/COS res: [0x240000402:0x26:0x0].0x53564696 bits 0x2/0x0 rrc: 2 type: IBT gid 0 flags: 0x44a19400000000 nid: local remote: 0x0 expref: -99 pid: 8250 timeout: 0 lvb_type: 0 00010000:00010000:0.0:1776178092.443459:0:3340:0:(ldlm_lock.c:820:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(COS) ns: mdt-lustre-MDT0001_UUID lock: 0000000054cf1a02/0xa40083ab9bf52b38 lrc: 3/0,1 mode: COS/COS res: [0x240000402:0x7e5:0x0].0x0 bits 0x3/0x0 rrc: 2 type: IBT gid 0 flags: 0x50210000000000 nid: local remote: 0x0 expref: -99 pid: 8250 timeout: 0 lvb_type: 0 00010000:00010000:0.0:1776178092.443468:0:3340:0:(ldlm_lock.c:879:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: mdt-lustre-MDT0001_UUID lock: 0000000054cf1a02/0xa40083ab9bf52b38 lrc: 2/0,0 mode: COS/COS res: [0x240000402:0x7e5:0x0].0x0 bits 0x3/0x0 rrc: 2 type: IBT gid 0 flags: 0x50210400000000 nid: local remote: 0x0 expref: -99 pid: 8250 timeout: 0 lvb_type: 0 00010000:00010000:0.0:1776178092.443476:0:3340:0:(ldlm_lockd.c:1879:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: mdt-lustre-MDT0001_UUID lock: 0000000054cf1a02/0xa40083ab9bf52b38 lrc: 3/0,0 mode: COS/COS res: [0x240000402:0x7e5:0x0].0x0 bits 0x3/0x0 rrc: 2 type: IBT gid 0 flags: 0x40210400000000 nid: local remote: 0x0 expref: -99 pid: 8250 timeout: 0 lvb_type: 0 00010000:00010000:0.0:1776178092.443489:0:3340:0:(ldlm_lockd.c:1893:ldlm_handle_bl_callback()) Lock 0000000054cf1a02 already unused, calling callback (0000000079e78348) 00010000:00010000:0.0:1776178092.443494:0:3340:0:(ldlm_request.c:364:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: mdt-lustre-MDT0001_UUID lock: 0000000054cf1a02/0xa40083ab9bf52b38 lrc: 3/0,0 mode: COS/COS res: [0x240000402:0x7e5:0x0].0x0 bits 0x3/0x0 rrc: 2 type: IBT gid 0 flags: 0x40210400000000 nid: local remote: 0x0 expref: -99 pid: 8250 timeout: 0 lvb_type: 0 00010000:00010000:0.0:1776178092.443502:0:3340:0:(ldlm_request.c:1257:ldlm_cli_cancel_local()) ### server-side local cancel ns: mdt-lustre-MDT0001_UUID lock: 0000000054cf1a02/0xa40083ab9bf52b38 lrc: 4/0,0 mode: COS/COS res: [0x240000402:0x7e5:0x0].0x0 bits 0x3/0x0 rrc: 2 type: IBT gid 0 flags: 0x40218400000000 nid: local remote: 0x0 expref: -99 pid: 8250 timeout: 0 lvb_type: 0 00010000:00010000:0.0:1776178092.443523:0:3340:0:(ldlm_inodebits.c:94:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x240000402:0x7e5:0x0].0x0 (00000000f86873a9) 00010000:00010000:0.0:1776178092.443525:0:3340:0:(ldlm_inodebits.c:96:ldlm_reprocess_inodebits_queue()) Hint 3 00010000:00010000:0.0:1776178092.443531:0:3340:0:(ldlm_lockd.c:1903:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: mdt-lustre-MDT0001_UUID lock: 0000000054cf1a02/0xa40083ab9bf52b38 lrc: 2/0,0 mode: --/COS res: [0x240000402:0x7e5:0x0].0x0 bits 0x3/0x0 rrc: 2 type: IBT gid 0 flags: 0x44a19400000000 nid: local remote: 0x0 expref: -99 pid: 8250 timeout: 0 lvb_type: 0 00010000:00010000:0.0:1776178092.443538:0:3340:0:(ldlm_lock.c:216:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: mdt-lustre-MDT0001_UUID lock: 0000000054cf1a02/0xa40083ab9bf52b38 lrc: 0/0,0 mode: --/COS res: [0x240000402:0x7e5:0x0].0x0 bits 0x3/0x0 rrc: 2 type: IBT gid 0 flags: 0x44a19400000000 nid: local remote: 0x0 expref: -99 pid: 8250 timeout: 0 lvb_type: 0 00010000:00010000:0.0:1776178092.443562:0:3340:0:(ldlm_lock.c:820:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(COS) ns: mdt-lustre-MDT0001_UUID lock: 00000000d24e9b73/0xa40083ab9bf52b4d lrc: 3/0,1 mode: COS/COS res: [0x240000402:0x26:0x0].0x7752dbce bits 0x2/0x0 rrc: 2 type: IBT gid 0 flags: 0x50210000000000 nid: local remote: 0x0 expref: -99 pid: 8250 timeout: 0 lvb_type: 0 00010000:00010000:0.0:1776178092.443568:0:3340:0:(ldlm_lock.c:879:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: mdt-lustre-MDT0001_UUID lock: 00000000d24e9b73/0xa40083ab9bf52b4d lrc: 2/0,0 mode: COS/COS res: [0x240000402:0x26:0x0].0x7752dbce bits 0x2/0x0 rrc: 2 type: IBT gid 0 flags: 0x50210400000000 nid: local remote: 0x0 expref: -99 pid: 8250 timeout: 0 lvb_type: 0 00010000:00010000:0.0:1776178092.443574:0:3340:0:(ldlm_lockd.c:1879:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: mdt-lustre-MDT0001_UUID lock: 00000000d24e9b73/0xa40083ab9bf52b4d lrc: 3/0,0 mode: COS/COS res: [0x240000402:0x26:0x0].0x7752dbce bits 0x2/0x0 rrc: 2 type: IBT gid 0 flags: 0x40210400000000 nid: local remote: 0x0 expref: -99 pid: 8250 timeout: 0 lvb_type: 0 00010000:00010000:0.0:1776178092.443580:0:3340:0:(ldlm_lockd.c:1893:ldlm_handle_bl_callback()) Lock 00000000d24e9b73 already unused, calling callback (0000000079e78348) 00010000:00010000:0.0:1776178092.443582:0:3340:0:(ldlm_request.c:364:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: mdt-lustre-MDT0001_UUID lock: 00000000d24e9b73/0xa40083ab9bf52b4d lrc: 3/0,0 mode: COS/COS res: [0x240000402:0x26:0x0].0x7752dbce bits 0x2/0x0 rrc: 2 type: IBT gid 0 flags: 0x40210400000000 nid: local remote: 0x0 expref: -99 pid: 8250 timeout: 0 lvb_type: 0 00010000:00010000:0.0:1776178092.443588:0:3340:0:(ldlm_request.c:1257:ldlm_cli_cancel_local()) ### server-side local cancel ns: mdt-lustre-MDT0001_UUID lock: 00000000d24e9b73/0xa40083ab9bf52b4d lrc: 4/0,0 mode: COS/COS res: [0x240000402:0x26:0x0].0x7752dbce bits 0x2/0x0 rrc: 2 type: IBT gid 0 flags: 0x40218400000000 nid: local remote: 0x0 expref: -99 pid: 8250 timeout: 0 lvb_type: 0 00010000:00010000:0.0:1776178092.443597:0:3340:0:(ldlm_inodebits.c:94:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x240000402:0x26:0x0].0x7752dbce (00000000e5cf8f1c) 00010000:00010000:0.0:1776178092.443600:0:3340:0:(ldlm_inodebits.c:96:ldlm_reprocess_inodebits_queue()) Hint 2 00010000:00010000:0.0:1776178092.443602:0:3340:0:(ldlm_lockd.c:1903:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: mdt-lustre-MDT0001_UUID lock: 00000000d24e9b73/0xa40083ab9bf52b4d lrc: 2/0,0 mode: --/COS res: [0x240000402:0x26:0x0].0x7752dbce bits 0x2/0x0 rrc: 2 type: IBT gid 0 flags: 0x44a19400000000 nid: local remote: 0x0 expref: -99 pid: 8250 timeout: 0 lvb_type: 0 00010000:00010000:0.0:1776178092.443607:0:3340:0:(ldlm_lock.c:216:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: mdt-lustre-MDT0001_UUID lock: 00000000d24e9b73/0xa40083ab9bf52b4d lrc: 0/0,0 mode: --/COS res: [0x240000402:0x26:0x0].0x7752dbce bits 0x2/0x0 rrc: 2 type: IBT gid 0 flags: 0x44a19400000000 nid: local remote: 0x0 expref: -99 pid: 8250 timeout: 0 lvb_type: 0 00010000:00010000:0.0:1776178092.443621:0:3340:0:(ldlm_lock.c:820:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(COS) ns: mdt-lustre-MDT0001_UUID lock: 000000006cf86792/0xa40083ab9bf52b54 lrc: 3/0,1 mode: COS/COS res: [0x240000402:0x493:0x0].0x0 bits 0x3/0x0 rrc: 2 type: IBT gid 0 flags: 0x50210000000000 nid: local remote: 0x0 expref: -99 pid: 8250 timeout: 0 lvb_type: 0 00010000:00010000:0.0:1776178092.443627:0:3340:0:(ldlm_lock.c:879:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: mdt-lustre-MDT0001_UUID lock: 000000006cf86792/0xa40083ab9bf52b54 lrc: 2/0,0 mode: COS/COS res: [0x240000402:0x493:0x0].0x0 bits 0x3/0x0 rrc: 2 type: IBT gid 0 flags: 0x50210400000000 nid: local remote: 0x0 expref: -99 pid: 8250 timeout: 0 lvb_type: 0 00010000:00010000:0.0:1776178092.443633:0:3340:0:(ldlm_lockd.c:1879:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: mdt-lustre-MDT0001_UUID lock: 000000006cf86792/0xa40083ab9bf52b54 lrc: 3/0,0 mode: COS/COS res: [0x240000402:0x493:0x0].0x0 bits 0x3/0x0 rrc: 2 type: IBT gid 0 flags: 0x40210400000000 nid: local remote: 0x0 expref: -99 pid: 8250 timeout: 0 lvb_type: 0 00010000:00010000:0.0:1776178092.443639:0:3340:0:(ldlm_lockd.c:1893:ldlm_handle_bl_callback()) Lock 000000006cf86792 already unused, calling callback (0000000079e78348) 00010000:00010000:0.0:1776178092.443641:0:3340:0:(ldlm_request.c:364:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: mdt-lustre-MDT0001_UUID lock: 000000006cf86792/0xa40083ab9bf52b54 lrc: 3/0,0 mode: COS/COS res: [0x240000402:0x493:0x0].0x0 bits 0x3/0x0 rrc: 2 type: IBT gid 0 flags: 0x40210400000000 nid: local remote: 0x0 expref: -99 pid: 8250 timeout: 0 lvb_type: 0 00010000:00010000:0.0:1776178092.443646:0:3340:0:(ldlm_request.c:1257:ldlm_cli_cancel_local()) ### server-side local cancel ns: mdt-lustre-MDT0001_UUID lock: 000000006cf86792/0xa40083ab9bf52b54 lrc: 4/0,0 mode: COS/COS res: [0x240000402:0x493:0x0].0x0 bits 0x3/0x0 rrc: 2 type: IBT gid 0 flags: 0x40218400000000 nid: local remote: 0x0 expref: -99 pid: 8250 timeout: 0 lvb_type: 0 00010000:00010000:0.0:1776178092.443652:0:3340:0:(ldlm_inodebits.c:94:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x240000402:0x493:0x0].0x0 (0000000036f59a41) 00010000:00010000:0.0:1776178092.443654:0:3340:0:(ldlm_inodebits.c:96:ldlm_reprocess_inodebits_queue()) Hint 3 00010000:00010000:0.0:1776178092.443656:0:3340:0:(ldlm_lockd.c:1903:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: mdt-lustre-MDT0001_UUID lock: 000000006cf86792/0xa40083ab9bf52b54 lrc: 2/0,0 mode: --/COS res: [0x240000402:0x493:0x0].0x0 bits 0x3/0x0 rrc: 2 type: IBT gid 0 flags: 0x44a19400000000 nid: local remote: 0x0 expref: -99 pid: 8250 timeout: 0 lvb_type: 0 00010000:00010000:0.0:1776178092.443666:0:3340:0:(ldlm_lock.c:216:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: mdt-lustre-MDT0001_UUID lock: 000000006cf86792/0xa40083ab9bf52b54 lrc: 0/0,0 mode: --/COS res: [0x240000402:0x493:0x0].0x0 bits 0x3/0x0 rrc: 2 type: IBT gid 0 flags: 0x44a19400000000 nid: local remote: 0x0 expref: -99 pid: 8250 timeout: 0 lvb_type: 0 00010000:00010000:0.0:1776178092.443678:0:3340:0:(ldlm_lock.c:820:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(COS) ns: mdt-lustre-MDT0001_UUID lock: 0000000068850c85/0xa40083ab9bf52b69 lrc: 3/0,1 mode: COS/COS res: [0x240000402:0x26:0x0].0xe452dddd bits 0x2/0x0 rrc: 2 type: IBT gid 0 flags: 0x50210000000000 nid: local remote: 0x0 expref: -99 pid: 8250 timeout: 0 lvb_type: 0 00010000:00010000:0.0:1776178092.443684:0:3340:0:(ldlm_lock.c:879:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: mdt-lustre-MDT0001_UUID lock: 0000000068850c85/0xa40083ab9bf52b69 lrc: 2/0,0 mode: COS/COS res: [0x240000402:0x26:0x0].0xe452dddd bits 0x2/0x0 rrc: 2 type: IBT gid 0 flags: 0x50210400000000 nid: local remote: 0x0 expref: -99 pid: 8250 timeout: 0 lvb_type: 0 00010000:00010000:0.0:1776178092.443689:0:3340:0:(ldlm_lockd.c:1879:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: mdt-lustre-MDT0001_UUID lock: 0000000068850c85/0xa40083ab9bf52b69 lrc: 3/0,0 mode: COS/COS res: [0x240000402:0x26:0x0].0xe452dddd bits 0x2/0x0 rrc: 2 type: IBT gid 0 flags: 0x40210400000000 nid: local remote: 0x0 expref: -99 pid: 8250 timeout: 0 lvb_type: 0 00010000:00010000:0.0:1776178092.443695:0:3340:0:(ldlm_lockd.c:1893:ldlm_handle_bl_callback()) Lock 0000000068850c85 already unused, calling callback (0000000079e78348) 00010000:00010000:0.0:1776178092.443697:0:3340:0:(ldlm_request.c:364:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: mdt-lustre-MDT0001_UUID lock: 0000000068850c85/0xa40083ab9bf52b69 lrc: 3/0,0 mode: COS/COS res: [0x240000402:0x26:0x0].0xe452dddd bits 0x2/0x0 rrc: 2 type: IBT gid 0 flags: 0x40210400000000 nid: local remote: 0x0 expref: -99 pid: 8250 timeout: 0 lvb_type: 0 00010000:00010000:0.0:1776178092.443702:0:3340:0:(ldlm_request.c:1257:ldlm_cli_cancel_local()) ### server-side local cancel ns: mdt-lustre-MDT0001_UUID lock: 0000000068850c85/0xa40083ab9bf52b69 lrc: 4/0,0 mode: COS/COS res: [0x240000402:0x26:0x0].0xe452dddd bits 0x2/0x0 rrc: 2 type: IBT gid 0 flags: 0x40218400000000 nid: local remote: 0x0 expref: -99 pid: 8250 timeout: 0 lvb_type: 0 00010000:00010000:0.0:1776178092.443709:0:3340:0:(ldlm_inodebits.c:94:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x240000402:0x26:0x0].0xe452dddd (00000000039e9c9f) 00010000:00010000:0.0:1776178092.443711:0:3340:0:(ldlm_inodebits.c:96:ldlm_reprocess_inodebits_queue()) Hint 2 00010000:00010000:0.0:1776178092.443713:0:3340:0:(ldlm_lockd.c:1903:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: mdt-lustre-MDT0001_UUID lock: 0000000068850c85/0xa40083ab9bf52b69 lrc: 2/0,0 mode: --/COS res: [0x240000402:0x26:0x0].0xe452dddd bits 0x2/0x0 rrc: 2 type: IBT gid 0 flags: 0x44a19400000000 nid: local remote: 0x0 expref: -99 pid: 8250 timeout: 0 lvb_type: 0 00010000:00010000:0.0:1776178092.443719:0:3340:0:(ldlm_lock.c:216:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: mdt-lustre-MDT0001_UUID lock: 0000000068850c85/0xa40083ab9bf52b69 lrc: 0/0,0 mode: --/COS res: [0x240000402:0x26:0x0].0xe452dddd bits 0x2/0x0 rrc: 2 type: IBT gid 0 flags: 0x44a19400000000 nid: local remote: 0x0 expref: -99 pid: 8250 timeout: 0 lvb_type: 0 00010000:00010000:0.0:1776178092.443729:0:3340:0:(ldlm_lock.c:820:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(COS) ns: mdt-lustre-MDT0001_UUID lock: 00000000782e253e/0xa40083ab9bf52b70 lrc: 3/0,1 mode: COS/COS res: [0x240000402:0x73c:0x0].0x0 bits 0x3/0x0 rrc: 2 type: IBT gid 0 flags: 0x50210000000000 nid: local remote: 0x0 expref: -99 pid: 8250 timeout: 0 lvb_type: 0 00010000:00010000:0.0:1776178092.443738:0:3340:0:(ldlm_lock.c:879:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: mdt-lustre-MDT0001_UUID lock: 00000000782e253e/0xa40083ab9bf52b70 lrc: 2/0,0 mode: COS/COS res: [0x240000402:0x73c:0x0].0x0 bits 0x3/0x0 rrc: 2 type: IBT gid 0 flags: 0x50210400000000 nid: local remote: 0x0 expref: -99 pid: 8250 timeout: 0 lvb_type: 0 00010000:00010000:0.0:1776178092.443743:0:3340:0:(ldlm_lockd.c:1879:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: mdt-lustre-MDT0001_UUID lock: 00000000782e253e/0xa40083ab9bf52b70 lrc: 3/0,0 mode: COS/COS res: [0x240000402:0x73c:0x0].0x0 bits 0x3/0x0 rrc: 2 type: IBT gid 0 flags: 0x40210400000000 nid: local remote: 0x0 expref: -99 pid: 8250 timeout: 0 lvb_type: 0 00010000:00010000:0.0:1776178092.443747:0:3340:0:(ldlm_lockd.c:1893:ldlm_handle_bl_callback()) Lock 00000000782e253e already unused, calling callback (0000000079e78348) 00010000:00010000:0.0:1776178092.443749:0:3340:0:(ldlm_request.c:364:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: mdt-lustre-MDT0001_UUID lock: 00000000782e253e/0xa40083ab9bf52b70 lrc: 3/0,0 mode: COS/COS res: [0x240000402:0x73c:0x0].0x0 bits 0x3/0x0 rrc: 2 type: IBT gid 0 flags: 0x40210400000000 nid: local remote: 0x0 expref: -99 pid: 8250 timeout: 0 lvb_type: 0 00010000:00010000:0.0:1776178092.443755:0:3340:0:(ldlm_request.c:1257:ldlm_cli_cancel_local()) ### server-side local cancel ns: mdt-lustre-MDT0001_UUID lock: 00000000782e253e/0xa40083ab9bf52b70 lrc: 4/0,0 mode: COS/COS res: [0x240000402:0x73c:0x0].0x0 bits 0x3/0x0 rrc: 2 type: IBT gid 0 flags: 0x40218400000000 nid: local remote: 0x0 expref: -99 pid: 8250 timeout: 0 lvb_type: 0 00010000:00010000:0.0:1776178092.443762:0:3340:0:(ldlm_inodebits.c:94:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x240000402:0x73c:0x0].0x0 (00000000890b13bb) 00010000:00010000:0.0:1776178092.443764:0:3340:0:(ldlm_inodebits.c:96:ldlm_reprocess_inodebits_queue()) Hint 3 00010000:00010000:0.0:1776178092.443765:0:3340:0:(ldlm_lockd.c:1903:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: mdt-lustre-MDT0001_UUID lock: 00000000782e253e/0xa40083ab9bf52b70 lrc: 2/0,0 mode: --/COS res: [0x240000402:0x73c:0x0].0x0 bits 0x3/0x0 rrc: 2 type: IBT gid 0 flags: 0x44a19400000000 nid: local remote: 0x0 expref: -99 pid: 8250 timeout: 0 lvb_type: 0 00010000:00010000:0.0:1776178092.443769:0:3340:0:(ldlm_lock.c:216:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: mdt-lustre-MDT0001_UUID lock: 00000000782e253e/0xa40083ab9bf52b70 lrc: 0/0,0 mode: --/COS res: [0x240000402:0x73c:0x0].0x0 bits 0x3/0x0 rrc: 2 type: IBT gid 0 flags: 0x44a19400000000 nid: local remote: 0x0 expref: -99 pid: 8250 timeout: 0 lvb_type: 0 00010000:00010000:0.0:1776178092.443783:0:3340:0:(ldlm_lock.c:820:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(COS) ns: mdt-lustre-MDT0001_UUID lock: 000000006c464689/0xa40083ab9bf52b85 lrc: 3/0,1 mode: COS/COS res: [0x240000402:0x26:0x0].0x9777dbce bits 0x2/0x0 rrc: 2 type: IBT gid 0 flags: 0x50210000000000 nid: local remote: 0x0 expref: -99 pid: 6257 timeout: 0 lvb_type: 0 00010000:00010000:0.0:1776178092.443788:0:3340:0:(ldlm_lock.c:879:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: mdt-lustre-MDT0001_UUID lock: 000000006c464689/0xa40083ab9bf52b85 lrc: 2/0,0 mode: COS/COS res: [0x240000402:0x26:0x0].0x9777dbce bits 0x2/0x0 rrc: 2 type: IBT gid 0 flags: 0x50210400000000 nid: local remote: 0x0 expref: -99 pid: 6257 timeout: 0 lvb_type: 0 00010000:00010000:0.0:1776178092.443793:0:3340:0:(ldlm_lockd.c:1879:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: mdt-lustre-MDT0001_UUID lock: 000000006c464689/0xa40083ab9bf52b85 lrc: 3/0,0 mode: COS/COS res: [0x240000402:0x26:0x0].0x9777dbce bits 0x2/0x0 rrc: 2 type: IBT gid 0 flags: 0x40210400000000 nid: local remote: 0x0 expref: -99 pid: 6257 timeout: 0 lvb_type: 0 00010000:00010000:0.0:1776178092.443803:0:3340:0:(ldlm_lockd.c:1893:ldlm_handle_bl_callback()) Lock 000000006c464689 already unused, calling callback (0000000079e78348) 00010000:00010000:0.0:1776178092.443805:0:3340:0:(ldlm_request.c:364:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: mdt-lustre-MDT0001_UUID lock: 000000006c464689/0xa40083ab9bf52b85 lrc: 3/0,0 mode: COS/COS res: [0x240000402:0x26:0x0].0x9777dbce bits 0x2/0x0 rrc: 2 type: IBT gid 0 flags: 0x40210400000000 nid: local remote: 0x0 expref: -99 pid: 6257 timeout: 0 lvb_type: 0 00010000:00010000:0.0:1776178092.443811:0:3340:0:(ldlm_request.c:1257:ldlm_cli_cancel_local()) ### server-side local cancel ns: mdt-lustre-MDT0001_UUID lock: 000000006c464689/0xa40083ab9bf52b85 lrc: 4/0,0 mode: COS/COS res: [0x240000402:0x26:0x0].0x9777dbce bits 0x2/0x0 rrc: 2 type: IBT gid 0 flags: 0x40218400000000 nid: local remote: 0x0 expref: -99 pid: 6257 timeout: 0 lvb_type: 0 00010000:00010000:0.0:1776178092.443819:0:3340:0:(ldlm_inodebits.c:94:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x240000402:0x26:0x0].0x9777dbce (00000000763e897c) 00010000:00010000:0.0:1776178092.443821:0:3340:0:(ldlm_inodebits.c:96:ldlm_reprocess_inodebits_queue()) Hint 2 00010000:00010000:0.0:1776178092.443823:0:3340:0:(ldlm_lockd.c:1903:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: mdt-lustre-MDT0001_UUID lock: 000000006c464689/0xa40083ab9bf52b85 lrc: 2/0,0 mode: --/COS res: [0x240000402:0x26:0x0].0x9777dbce bits 0x2/0x0 rrc: 2 type: IBT gid 0 flags: 0x44a19400000000 nid: local remote: 0x0 expref: -99 pid: 6257 timeout: 0 lvb_type: 0 00010000:00010000:0.0:1776178092.443837:0:3340:0:(ldlm_lock.c:216:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: mdt-lustre-MDT0001_UUID lock: 000000006c464689/0xa40083ab9bf52b85 lrc: 0/0,0 mode: --/COS res: [0x240000402:0x26:0x0].0x9777dbce bits 0x2/0x0 rrc: 2 type: IBT gid 0 flags: 0x44a19400000000 nid: local remote: 0x0 expref: -99 pid: 6257 timeout: 0 lvb_type: 0 00010000:00010000:0.0:1776178092.443852:0:3340:0:(ldlm_lock.c:820:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(COS) ns: mdt-lustre-MDT0001_UUID lock: 00000000a3795b7d/0xa40083ab9bf52b8c lrc: 3/0,1 mode: COS/COS res: [0x240000402:0x41b:0x0].0x0 bits 0x3/0x0 rrc: 2 type: IBT gid 0 flags: 0x50210000000000 nid: local remote: 0x0 expref: -99 pid: 6257 timeout: 0 lvb_type: 0 00010000:00010000:0.0:1776178092.443858:0:3340:0:(ldlm_lock.c:879:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: mdt-lustre-MDT0001_UUID lock: 00000000a3795b7d/0xa40083ab9bf52b8c lrc: 2/0,0 mode: COS/COS res: [0x240000402:0x41b:0x0].0x0 bits 0x3/0x0 rrc: 2 type: IBT gid 0 flags: 0x50210400000000 nid: local remote: 0x0 expref: -99 pid: 6257 timeout: 0 lvb_type: 0 00010000:00010000:0.0:1776178092.443863:0:3340:0:(ldlm_lockd.c:1879:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: mdt-lustre-MDT0001_UUID lock: 00000000a3795b7d/0xa40083ab9bf52b8c lrc: 3/0,0 mode: COS/COS res: [0x240000402:0x41b:0x0].0x0 bits 0x3/0x0 rrc: 2 type: IBT gid 0 flags: 0x40210400000000 nid: local remote: 0x0 expref: -99 pid: 6257 timeout: 0 lvb_type: 0 00010000:00010000:0.0:1776178092.443868:0:3340:0:(ldlm_lockd.c:1893:ldlm_handle_bl_callback()) Lock 00000000a3795b7d already unused, calling callback (0000000079e78348) 00010000:00010000:0.0:1776178092.443870:0:3340:0:(ldlm_request.c:364:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: mdt-lustre-MDT0001_UUID lock: 00000000a3795b7d/0xa40083ab9bf52b8c lrc: 3/0,0 mode: COS/COS res: [0x240000402:0x41b:0x0].0x0 bits 0x3/0x0 rrc: 2 type: IBT gid 0 flags: 0x40210400000000 nid: local remote: 0x0 expref: -99 pid: 6257 timeout: 0 lvb_type: 0 00010000:00010000:0.0:1776178092.443876:0:3340:0:(ldlm_request.c:1257:ldlm_cli_cancel_local()) ### server-side local cancel ns: mdt-lustre-MDT0001_UUID lock: 00000000a3795b7d/0xa40083ab9bf52b8c lrc: 4/0,0 mode: COS/COS res: [0x240000402:0x41b:0x0].0x0 bits 0x3/0x0 rrc: 2 type: IBT gid 0 flags: 0x40218400000000 nid: local remote: 0x0 expref: -99 pid: 6257 timeout: 0 lvb_type: 0 00010000:00010000:0.0:1776178092.443883:0:3340:0:(ldlm_inodebits.c:94:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x240000402:0x41b:0x0].0x0 (00000000ba9048ca) 00010000:00010000:0.0:1776178092.443886:0:3340:0:(ldlm_inodebits.c:96:ldlm_reprocess_inodebits_queue()) Hint 3 00010000:00010000:0.0:1776178092.443887:0:3340:0:(ldlm_lockd.c:1903:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: mdt-lustre-MDT0001_UUID lock: 00000000a3795b7d/0xa40083ab9bf52b8c lrc: 2/0,0 mode: --/COS res: [0x240000402:0x41b:0x0].0x0 bits 0x3/0x0 rrc: 2 type: IBT gid 0 flags: 0x44a19400000000 nid: local remote: 0x0 expref: -99 pid: 6257 timeout: 0 lvb_type: 0 00010000:00010000:0.0:1776178092.443893:0:3340:0:(ldlm_lock.c:216:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: mdt-lustre-MDT0001_UUID lock: 00000000a3795b7d/0xa40083ab9bf52b8c lrc: 0/0,0 mode: --/COS res: [0x240000402:0x41b:0x0].0x0 bits 0x3/0x0 rrc: 2 type: IBT gid 0 flags: 0x44a19400000000 nid: local remote: 0x0 expref: -99 pid: 6257 timeout: 0 lvb_type: 0 00010000:00010000:0.0:1776178092.443906:0:3340:0:(ldlm_lock.c:820:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(COS) ns: mdt-lustre-MDT0001_UUID lock: 00000000a211f94c/0xa40083ab9bf52bb6 lrc: 3/0,1 mode: COS/COS res: [0x240000402:0x26:0x0].0x0 bits 0x2/0x0 rrc: 2 type: IBT gid 0 flags: 0x50210001000000 nid: local remote: 0x0 expref: -99 pid: 6257 timeout: 0 lvb_type: 0 00010000:00010000:0.0:1776178092.443912:0:3340:0:(ldlm_lock.c:879:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: mdt-lustre-MDT0001_UUID lock: 00000000a211f94c/0xa40083ab9bf52bb6 lrc: 2/0,0 mode: COS/COS res: [0x240000402:0x26:0x0].0x0 bits 0x2/0x0 rrc: 2 type: IBT gid 0 flags: 0x50210401000000 nid: local remote: 0x0 expref: -99 pid: 6257 timeout: 0 lvb_type: 0 00010000:00010000:0.0:1776178092.443918:0:3340:0:(ldlm_lockd.c:1879:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: mdt-lustre-MDT0001_UUID lock: 00000000a211f94c/0xa40083ab9bf52bb6 lrc: 3/0,0 mode: COS/COS res: [0x240000402:0x26:0x0].0x0 bits 0x2/0x0 rrc: 2 type: IBT gid 0 flags: 0x40210401000000 nid: local remote: 0x0 expref: -99 pid: 6257 timeout: 0 lvb_type: 0 00010000:00010000:0.0:1776178092.443923:0:3340:0:(ldlm_lockd.c:1893:ldlm_handle_bl_callback()) Lock 00000000a211f94c already unused, calling callback (0000000079e78348) 00010000:00010000:0.0:1776178092.443925:0:3340:0:(ldlm_request.c:364:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: mdt-lustre-MDT0001_UUID lock: 00000000a211f94c/0xa40083ab9bf52bb6 lrc: 3/0,0 mode: COS/COS res: [0x240000402:0x26:0x0].0x0 bits 0x2/0x0 rrc: 2 type: IBT gid 0 flags: 0x40210401000000 nid: local remote: 0x0 expref: -99 pid: 6257 timeout: 0 lvb_type: 0 00010000:00010000:0.0:1776178092.443930:0:3340:0:(ldlm_request.c:1257:ldlm_cli_cancel_local()) ### server-side local cancel ns: mdt-lustre-MDT0001_UUID lock: 00000000a211f94c/0xa40083ab9bf52bb6 lrc: 4/0,0 mode: COS/COS res: [0x240000402:0x26:0x0].0x0 bits 0x2/0x0 rrc: 2 type: IBT gid 0 flags: 0x40218401000000 nid: local remote: 0x0 expref: -99 pid: 6257 timeout: 0 lvb_type: 0 00010000:00010000:0.0:1776178092.443937:0:3340:0:(ldlm_inodebits.c:94:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x240000402:0x26:0x0].0x0 (0000000046f6aa15) 00010000:00010000:0.0:1776178092.443939:0:3340:0:(ldlm_inodebits.c:96:ldlm_reprocess_inodebits_queue()) Hint 2 00010000:00010000:0.0:1776178092.443941:0:3340:0:(ldlm_lockd.c:1903:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: mdt-lustre-MDT0001_UUID lock: 00000000a211f94c/0xa40083ab9bf52bb6 lrc: 2/0,0 mode: --/COS res: [0x240000402:0x26:0x0].0x0 bits 0x2/0x0 rrc: 2 type: IBT gid 0 flags: 0x44a19401000000 nid: local remote: 0x0 expref: -99 pid: 6257 timeout: 0 lvb_type: 0 00010000:00010000:0.0:1776178092.443951:0:3340:0:(ldlm_lock.c:216:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: mdt-lustre-MDT0001_UUID lock: 00000000a211f94c/0xa40083ab9bf52bb6 lrc: 0/0,0 mode: --/COS res: [0x240000402:0x26:0x0].0x0 bits 0x2/0x0 rrc: 2 type: IBT gid 0 flags: 0x44a19401000000 nid: local remote: 0x0 expref: -99 pid: 6257 timeout: 0 lvb_type: 0 00010000:00010000:0.0:1776178092.450171:0:20763:0:(ldlm_lock.c:216:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 000000001506c4ff/0xa40083ab9bf34066 lrc: 0/0,0 mode: --/PR res: [0x280000400:0x2b0:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x44801400010020 nid: 192.168.204.3@tcp remote: 0x771f68aa1eb22fe6 expref: 45 pid: 8228 timeout: 899 lvb_type: 1 00000100:00100000:0.0:1776178092.450201:0:20763:0:(client.c:2215:ptlrpc_check_set()) Completed RPC req@0000000021b0d516 pname:cluuid:pid:xid:nid:opc:job ll_ost00_014:lustre-OST0000_UUID:20763:1862456998009984:192.168.204.3@tcp:104: 00010000:00010000:0.0:1776178092.450232:0:20763:0:(ldlm_request.c:283:ldlm_completion_ast()) ### client-side enqueue returned a blocked locksleeping ns: filter-lustre-OST0000_UUID lock: 00000000624ff71f/0xa40083ab9bf52c2d lrc: 3/0,1 mode: PW/PW res: [0x280000400:0x2b0:0x0].0x0 rrc: 2 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x40010080000000 nid: local remote: 0x0 expref: -99 pid: 20763 timeout: 0 lvb_type: 0 00010000:00010000:0.0:1776178092.450245:0:20763:0:(ldlm_request.c:199:ldlm_completion_tail()) ### client-side enqueue: granted ns: filter-lustre-OST0000_UUID lock: 00000000624ff71f/0xa40083ab9bf52c2d lrc: 3/0,1 mode: PW/PW res: [0x280000400:0x2b0:0x0].0x0 rrc: 2 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x40010080000000 nid: local remote: 0x0 expref: -99 pid: 20763 timeout: 0 lvb_type: 0 00010000:00010000:0.0:1776178092.450252:0:20763:0:(ldlm_request.c:528:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 00000000624ff71f/0xa40083ab9bf52c2d lrc: 3/0,1 mode: PW/PW res: [0x280000400:0x2b0:0x0].0x0 rrc: 2 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x40010080000000 nid: local remote: 0x0 expref: -99 pid: 20763 timeout: 0 lvb_type: 0 00010000:00010000:0.0:1776178092.450264:0:20763:0:(ldlm_lock.c:820:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0000_UUID lock: 00000000624ff71f/0xa40083ab9bf52c2d lrc: 3/0,1 mode: PW/PW res: [0x280000400:0x2b0:0x0].0x0 rrc: 2 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x50010080000000 nid: local remote: 0x0 expref: -99 pid: 20763 timeout: 0 lvb_type: 0 00010000:00010000:0.0:1776178092.450271:0:20763:0:(ldlm_lock.c:879:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 00000000624ff71f/0xa40083ab9bf52c2d lrc: 2/0,0 mode: PW/PW res: [0x280000400:0x2b0:0x0].0x0 rrc: 2 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x50010480000000 nid: local remote: 0x0 expref: -99 pid: 20763 timeout: 0 lvb_type: 0 00010000:00010000:0.0:1776178092.461557:0:20763:0:(ldlm_lockd.c:2159:__ldlm_bl_to_thread()) added 1/3 locks to regular blp list, 3 blwis in pool 00000100:00100000:0.0:1776178092.461775:0:20763:0:(service.c:2372:ptlrpc_server_handle_request()) Handled RPC req@00000000339a9a57 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_014:lustre-MDT0001-mdtlov_UUID+15:3356:x1862456998008384:12345-0@lo:6:osp-syn-0-1.0 Request processed in 47727us (47839us total) trans 4294969298 rc 0/0 00000100:00100000:0.0:1776178092.461785:0:20763:0:(nrs_fifo.c:241:nrs_fifo_req_stop()) NRS stop fifo request from 12345-0@lo, seq: 5153 00010000:00010000:0.0:1776178092.461943:0:20755:0:(ldlm_lock.c:216:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 0000000006069837/0xa40083ab9bf3f48a lrc: 0/0,0 mode: --/PR res: [0x280000400:0x41e:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x44801400010020 nid: 192.168.204.3@tcp remote: 0x771f68aa1eb26bf2 expref: 44 pid: 8228 timeout: 899 lvb_type: 1 00000100:00100000:0.0:1776178092.461955:0:20755:0:(client.c:2215:ptlrpc_check_set()) Completed RPC req@0000000069ebab60 pname:cluuid:pid:xid:nid:opc:job ll_ost00_006:lustre-OST0000_UUID:20755:1862456998010368:192.168.204.3@tcp:104: 00010000:00010000:0.0:1776178092.461981:0:20755:0:(ldlm_request.c:283:ldlm_completion_ast()) ### client-side enqueue returned a blocked locksleeping ns: filter-lustre-OST0000_UUID lock: 000000002f1f7bd5/0xa40083ab9bf52c50 lrc: 3/0,1 mode: PW/PW res: [0x280000400:0x41e:0x0].0x0 rrc: 2 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x40010080000000 nid: local remote: 0x0 expref: -99 pid: 20755 timeout: 0 lvb_type: 0 00010000:00010000:0.0:1776178092.461988:0:20755:0:(ldlm_request.c:199:ldlm_completion_tail()) ### client-side enqueue: granted ns: filter-lustre-OST0000_UUID lock: 000000002f1f7bd5/0xa40083ab9bf52c50 lrc: 3/0,1 mode: PW/PW res: [0x280000400:0x41e:0x0].0x0 rrc: 2 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x40010080000000 nid: local remote: 0x0 expref: -99 pid: 20755 timeout: 0 lvb_type: 0 00010000:00010000:0.0:1776178092.461993:0:20755:0:(ldlm_request.c:528:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 000000002f1f7bd5/0xa40083ab9bf52c50 lrc: 3/0,1 mode: PW/PW res: [0x280000400:0x41e:0x0].0x0 rrc: 2 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x40010080000000 nid: local remote: 0x0 expref: -99 pid: 20755 timeout: 0 lvb_type: 0 00010000:00010000:0.0:1776178092.461999:0:20755:0:(ldlm_lock.c:820:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0000_UUID lock: 000000002f1f7bd5/0xa40083ab9bf52c50 lrc: 3/0,1 mode: PW/PW res: [0x280000400:0x41e:0x0].0x0 rrc: 2 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x50010080000000 nid: local remote: 0x0 expref: -99 pid: 20755 timeout: 0 lvb_type: 0 00010000:00010000:0.0:1776178092.462005:0:20755:0:(ldlm_lock.c:879:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 000000002f1f7bd5/0xa40083ab9bf52c50 lrc: 2/0,0 mode: PW/PW res: [0x280000400:0x41e:0x0].0x0 rrc: 2 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x50010480000000 nid: local remote: 0x0 expref: -99 pid: 20755 timeout: 0 lvb_type: 0 00010000:00010000:0.0:1776178092.462013:0:20755:0:(ldlm_lockd.c:2159:__ldlm_bl_to_thread()) added 1/6 locks to regular blp list, 6 blwis in pool 00000100:00100000:0.0:1776178092.462601:0:20755:0:(service.c:2372:ptlrpc_server_handle_request()) Handled RPC req@0000000045560455 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_006:lustre-MDT0001-mdtlov_UUID+15:3356:x1862456998010304:12345-0@lo:6:osp-syn-0-1.0 Request processed in 38479us (38784us total) trans 4294969301 rc 0/0 00000100:00100000:0.0:1776178092.462607:0:20755:0:(nrs_fifo.c:241:nrs_fifo_req_stop()) NRS stop fifo request from 12345-0@lo, seq: 5158 00010000:00010000:0.0:1776178092.462636:0:20762:0:(ldlm_lock.c:216:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 00000000467d820e/0xa40083ab9bf41290 lrc: 0/0,0 mode: --/PR res: [0x280000400:0x45b:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x44801400010020 nid: 192.168.204.3@tcp remote: 0x771f68aa1eb275f4 expref: 42 pid: 8645 timeout: 899 lvb_type: 1 00000100:00100000:0.0:1776178092.462646:0:20762:0:(client.c:2215:ptlrpc_check_set()) Completed RPC req@0000000038c09127 pname:cluuid:pid:xid:nid:opc:job ll_ost00_013:lustre-OST0000_UUID:20762:1862456998009920:192.168.204.3@tcp:104: 00010000:00010000:0.0:1776178092.462659:0:20762:0:(ldlm_request.c:283:ldlm_completion_ast()) ### client-side enqueue returned a blocked locksleeping ns: filter-lustre-OST0000_UUID lock: 00000000372a97b2/0xa40083ab9bf52c26 lrc: 3/0,1 mode: PW/PW res: [0x280000400:0x45b:0x0].0x0 rrc: 2 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x40010080000000 nid: local remote: 0x0 expref: -99 pid: 20762 timeout: 0 lvb_type: 0 00010000:00010000:0.0:1776178092.462666:0:20762:0:(ldlm_request.c:199:ldlm_completion_tail()) ### client-side enqueue: granted ns: filter-lustre-OST0000_UUID lock: 00000000372a97b2/0xa40083ab9bf52c26 lrc: 3/0,1 mode: PW/PW res: [0x280000400:0x45b:0x0].0x0 rrc: 2 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x40010080000000 nid: local remote: 0x0 expref: -99 pid: 20762 timeout: 0 lvb_type: 0 00010000:00010000:0.0:1776178092.462672:0:20762:0:(ldlm_request.c:528:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 00000000372a97b2/0xa40083ab9bf52c26 lrc: 3/0,1 mode: PW/PW res: [0x280000400:0x45b:0x0].0x0 rrc: 2 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x40010080000000 nid: local remote: 0x0 expref: -99 pid: 20762 timeout: 0 lvb_type: 0 00010000:00010000:0.0:1776178092.462680:0:20762:0:(ldlm_lock.c:820:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0000_UUID lock: 00000000372a97b2/0xa40083ab9bf52c26 lrc: 3/0,1 mode: PW/PW res: [0x280000400:0x45b:0x0].0x0 rrc: 2 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x50010080000000 nid: local remote: 0x0 expref: -99 pid: 20762 timeout: 0 lvb_type: 0 00010000:00010000:0.0:1776178092.462686:0:20762:0:(ldlm_lock.c:879:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 00000000372a97b2/0xa40083ab9bf52c26 lrc: 2/0,0 mode: PW/PW res: [0x280000400:0x45b:0x0].0x0 rrc: 2 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x50010480000000 nid: local remote: 0x0 expref: -99 pid: 20762 timeout: 0 lvb_type: 0 00010000:00010000:0.0:1776178092.462693:0:20762:0:(ldlm_lockd.c:2159:__ldlm_bl_to_thread()) added 1/11 locks to regular blp list, 11 blwis in pool 00000100:00100000:0.0:1776178092.462780:0:20762:0:(service.c:2372:ptlrpc_server_handle_request()) Handled RPC req@00000000d335a548 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_013:lustre-MDT0001-mdtlov_UUID+15:3355:x1862456998008320:12345-0@lo:6:osp-syn-0-1.0 Request processed in 49276us (56221us total) trans 4294969303 rc 0/0 00000100:00100000:0.0:1776178092.462785:0:20762:0:(nrs_fifo.c:241:nrs_fifo_req_stop()) NRS stop fifo request from 12345-0@lo, seq: 5152 00000004:00080000:0.0:1776178092.462844:0:3355:0:(osp_sync.c:562:osp_sync_interpret()) reply req 00000000fa8483d9/2, rc 0, transno 2003 00000100:00100000:0.0:1776178092.462856:0:3355:0:(client.c:2215:ptlrpc_check_set()) Completed RPC req@00000000fa8483d9 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_02:lustre-MDT0001-mdtlov_UUID:3355:1862456998008576:0@lo:6:osp-syn-0-1.0 00000100:00100000:0.0:1776178092.462871:0:3355:0:(client.c:2215:ptlrpc_check_set()) Completed RPC req@000000004e8fbd9b pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_02:lustre-MDT0001-mdtlov_UUID:3355:1862456998008832:0@lo:103:jbd2/dm-1-8.0 00000004:00080000:0.0:1776178092.462882:0:3355:0:(osp_sync.c:562:osp_sync_interpret()) reply req 0000000008c9fb19/2, rc 0, transno 2007 00000100:00100000:0.0:1776178092.462885:0:3355:0:(client.c:2215:ptlrpc_check_set()) Completed RPC req@0000000008c9fb19 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_02:lustre-MDT0001-mdtlov_UUID:3355:1862456998008320:0@lo:6:osp-syn-0-1.0 00000004:00080000:0.0:1776178092.462926:0:3353:0:(osp_sync.c:562:osp_sync_interpret()) reply req 00000000232b3a5e/2, rc 0, transno 2004 00000100:00100000:0.0:1776178092.462929:0:3353:0:(client.c:2215:ptlrpc_check_set()) Completed RPC req@00000000232b3a5e pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_00:lustre-MDT0001-mdtlov_UUID:3353:1862456998008512:0@lo:6:osp-syn-0-1.0 00000004:00080000:0.0:1776178092.462937:0:3353:0:(osp_sync.c:562:osp_sync_interpret()) reply req 00000000c63a98c1/2, rc 0, transno 2006 00000100:00100000:0.0:1776178092.462940:0:3353:0:(client.c:2215:ptlrpc_check_set()) Completed RPC req@00000000c63a98c1 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_00:lustre-MDT0001-mdtlov_UUID:3353:1862456998008768:0@lo:6:osp-syn-0-1.0 00000004:00080000:0.0:1776178092.462951:0:3353:0:(osp_sync.c:562:osp_sync_interpret()) reply req 00000000638772fd/2, rc 0, transno 1983 00000100:00100000:0.0:1776178092.462959:0:3353:0:(client.c:2215:ptlrpc_check_set()) Completed RPC req@00000000638772fd pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_00:lustre-MDT0001-mdtlov_UUID:3353:1862456998009024:0@lo:6:osp-syn-1-1.0 00000004:00080000:0.0:1776178092.462969:0:3353:0:(osp_sync.c:562:osp_sync_interpret()) reply req 0000000035fd3386/2, rc 0, transno 1984 00000100:00100000:0.0:1776178092.462972:0:3353:0:(client.c:2215:ptlrpc_check_set()) Completed RPC req@0000000035fd3386 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_00:lustre-MDT0001-mdtlov_UUID:3353:1862456998009280:0@lo:6:osp-syn-1-1.0 00000100:00100000:0.0:1776178092.463005:0:14440:0:(client.c:2215:ptlrpc_check_set()) Completed RPC req@000000001d51eec7 pname:cluuid:pid:xid:nid:opc:job ll_ost00_004:lustre-OST0001_UUID:14440:1862456998010624:192.168.204.3@tcp:104: 00010000:00010000:0.0:1776178092.463018:0:14440:0:(ldlm_request.c:283:ldlm_completion_ast()) ### client-side enqueue returned a blocked locksleeping ns: filter-lustre-OST0001_UUID lock: 0000000073903591/0xa40083ab9bf52c65 lrc: 3/0,1 mode: --/PW res: [0x2c0000400:0x3f7:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x40010080000000 nid: local remote: 0x0 expref: -99 pid: 14440 timeout: 0 lvb_type: 0 00000100:00100000:0.0:1776178092.466513:0:3354:0:(client.c:1733:ptlrpc_send_new_req()) Sending RPC req@000000009c55a259 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_01:lustre-MDT0001-mdtlov_UUID:3354:1862456998011008:0@lo:6:osp-syn-0-1.0 00000100:00100000:0.0:1776178092.466554:0:3354:0:(events.c:361:request_in_callback()) peer: 12345-0@lo (source: 12345-0@lo) 00000100:00100000:0.0:1776178092.466580:0:3354:0:(ptlrpcd.c:415:ptlrpcd_check()) transfer 1 async RPCs [0->1] 00000100:00100000:0.0:1776178092.466584:0:3354:0:(client.c:1733:ptlrpc_send_new_req()) Sending RPC req@0000000008b3a3ad pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_01:lustre-MDT0001-mdtlov_UUID:3354:1862456998010944:0@lo:6:osp-syn-0-1.0 00000100:00100000:0.0:1776178092.466601:0:3354:0:(events.c:361:request_in_callback()) peer: 12345-0@lo (source: 12345-0@lo) 00000100:00100000:0.0:1776178092.466624:0:20759:0:(service.c:2152:ptlrpc_server_handle_req_in()) got req x1862456998010944 00010000:00010000:0.0:1776178092.471666:0:14440:0:(ldlm_request.c:199:ldlm_completion_tail()) ### client-side enqueue: granted ns: filter-lustre-OST0001_UUID lock: 0000000073903591/0xa40083ab9bf52c65 lrc: 3/0,1 mode: PW/PW res: [0x2c0000400:0x3f7:0x0].0x0 rrc: 2 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x40010080000000 nid: local remote: 0x0 expref: -99 pid: 14440 timeout: 0 lvb_type: 0 00010000:00010000:0.0:1776178092.471677:0:14440:0:(ldlm_request.c:528:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0001_UUID lock: 0000000073903591/0xa40083ab9bf52c65 lrc: 3/0,1 mode: PW/PW res: [0x2c0000400:0x3f7:0x0].0x0 rrc: 2 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x40010080000000 nid: local remote: 0x0 expref: -99 pid: 14440 timeout: 0 lvb_type: 0 00010000:00010000:0.0:1776178092.471687:0:14440:0:(ldlm_lock.c:820:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0001_UUID lock: 0000000073903591/0xa40083ab9bf52c65 lrc: 3/0,1 mode: PW/PW res: [0x2c0000400:0x3f7:0x0].0x0 rrc: 2 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x50010080000000 nid: local remote: 0x0 expref: -99 pid: 14440 timeout: 0 lvb_type: 0 00010000:00010000:0.0:1776178092.471698:0:14440:0:(ldlm_lock.c:879:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0001_UUID lock: 0000000073903591/0xa40083ab9bf52c65 lrc: 2/0,0 mode: PW/PW res: [0x2c0000400:0x3f7:0x0].0x0 rrc: 2 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x50010480000000 nid: local remote: 0x0 expref: -99 pid: 14440 timeout: 0 lvb_type: 0 00010000:00010000:0.0:1776178092.471712:0:14440:0:(ldlm_lockd.c:2159:__ldlm_bl_to_thread()) added 1/2 locks to regular blp list, 2 blwis in pool 00000100:00100000:0.0:1776178092.471922:0:14440:0:(service.c:2372:ptlrpc_server_handle_request()) Handled RPC req@00000000a30eb964 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_004:lustre-MDT0001-mdtlov_UUID+14:3353:x1862456998010560:12345-0@lo:6:osp-syn-1-1.0 Request processed in 41058us (41223us total) trans 4294969284 rc 0/0 00000100:00100000:0.0:1776178092.471930:0:14440:0:(nrs_fifo.c:241:nrs_fifo_req_stop()) NRS stop fifo request from 12345-0@lo, seq: 5161 00000100:00100000:0.0:1776178092.471951:0:14440:0:(service.c:2152:ptlrpc_server_handle_req_in()) got req x1862456998010816 00000100:00100000:0.0:1776178092.471992:0:14440:0:(nrs_fifo.c:179:nrs_fifo_req_get()) NRS start fifo request from 12345-0@lo, seq: 5165 00000100:00100000:0.0:1776178092.471997:0:14440:0:(service.c:2324:ptlrpc_server_handle_request()) Handling RPC req@0000000001be1ac2 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_004:lustre-MDT0001-mdtlov_UUID+17:3355:x1862456998010816:12345-0@lo:6:osp-syn-0-1.0 00002000:00080000:0.0:1776178092.472016:0:14440:0:(ofd_dev.c:1822:ofd_destroy_hdl()) lustre-OST0000: Destroy object 0x280000400:824 count 1 00010000:00010000:0.0:1776178092.472042:0:14440:0:(ldlm_lock.c:766:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0000_UUID lock: 00000000d999afdf/0xa40083ab9bf52c88 lrc: 3/0,1 mode: --/PW res: [0x280000400:0x338:0x0].0x0 rrc: 3 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 14440 timeout: 0 lvb_type: 0 00010000:00010000:0.0:1776178092.472051:0:14440:0:(ldlm_lock.c:675:ldlm_add_bl_work_item()) ### lock incompatible; sending blocking AST. ns: filter-lustre-OST0000_UUID lock: 00000000b3096574/0xa40083ab9bf38356 lrc: 2/0,0 mode: PR/PR res: [0x280000400:0x338:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x40000000000000 nid: 192.168.204.3@tcp remote: 0x771f68aa1eb24636 expref: 41 pid: 8645 timeout: 0 lvb_type: 1 00010000:00010000:0.0:1776178092.472073:0:14440:0:(ldlm_extent.c:318:ldlm_check_contention()) contended locks = 0 00010000:00010000:0.0:1776178092.472074:0:14440:0:(ldlm_extent.c:318:ldlm_check_contention()) contended locks = 0 00010000:00010000:0.0:1776178092.472076:0:14440:0:(ldlm_resource.c:1650:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 00000000d999afdf/0xa40083ab9bf52c88 lrc: 4/0,1 mode: --/PW res: [0x280000400:0x338:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x50010080000000 nid: local remote: 0x0 expref: -99 pid: 14440 timeout: 0 lvb_type: 0 00010000:00010000:0.0:1776178092.472097:0:14440:0:(ldlm_lockd.c:953:ldlm_server_blocking_ast()) ### server preparing blocking AST ns: filter-lustre-OST0000_UUID lock: 00000000b3096574/0xa40083ab9bf38356 lrc: 3/0,0 mode: PR/PR res: [0x280000400:0x338:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x50000000010020 nid: 192.168.204.3@tcp remote: 0x771f68aa1eb24636 expref: 41 pid: 8645 timeout: 0 lvb_type: 1 00010000:00010000:0.0:1776178092.472107:0:14440:0:(ldlm_lockd.c:489:ldlm_add_waiting_lock()) ### adding to wait list(timeout: 100, AT: on) ns: filter-lustre-OST0000_UUID lock: 00000000b3096574/0xa40083ab9bf38356 lrc: 4/0,0 mode: PR/PR res: [0x280000400:0x338:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x70000400010020 nid: 192.168.204.3@tcp remote: 0x771f68aa1eb24636 expref: 41 pid: 8645 timeout: 899 lvb_type: 1 00000100:00100000:0.0:1776178092.472118:0:14440:0:(client.c:1733:ptlrpc_send_new_req()) Sending RPC req@000000005967fd64 pname:cluuid:pid:xid:nid:opc:job ll_ost00_004:lustre-OST0000_UUID:14440:1862456998011328:192.168.204.3@tcp:104: 00000100:00100000:0.0:1776178092.472153:0:14440:0:(client.c:2500:ptlrpc_set_wait()) set 000000008a94139b going to sleep for 43 seconds 00000100:00100000:0.0:1776178092.472186:0:3356:0:(client.c:1733:ptlrpc_send_new_req()) Sending RPC req@00000000e837a7cd pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_03:lustre-MDT0001-mdtlov_UUID:3356:1862456998010880:0@lo:6:osp-syn-0-1.0 00000100:00100000:0.0:1776178092.472210:0:3356:0:(events.c:361:request_in_callback()) peer: 12345-0@lo (source: 12345-0@lo) 00000100:00100000:0.0:1776178092.472231:0:20759:0:(service.c:2152:ptlrpc_server_handle_req_in()) got req x1862456998010880 00000100:00100000:0.0:1776178092.472250:0:20759:0:(nrs_fifo.c:179:nrs_fifo_req_get()) NRS start fifo request from 12345-0@lo, seq: 5167 00000100:00100000:0.0:1776178092.472254:0:20759:0:(service.c:2324:ptlrpc_server_handle_request()) Handling RPC req@000000009b5eeec2 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_010:lustre-MDT0001-mdtlov_UUID+18:3356:x1862456998010880:12345-0@lo:6:osp-syn-0-1.0 00002000:00080000:0.0:1776178092.472264:0:20759:0:(ofd_dev.c:1822:ofd_destroy_hdl()) lustre-OST0000: Destroy object 0x280000400:674 count 1 00010000:00010000:0.0:1776178092.472275:0:20759:0:(ldlm_lock.c:766:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0000_UUID lock: 000000006b206cc9/0xa40083ab9bf52c8f lrc: 3/0,1 mode: --/PW res: [0x280000400:0x2a2:0x0].0x0 rrc: 3 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 20759 timeout: 0 lvb_type: 0 00010000:00010000:0.0:1776178092.472285:0:20759:0:(ldlm_lock.c:675:ldlm_add_bl_work_item()) ### lock incompatible; sending blocking AST. ns: filter-lustre-OST0000_UUID lock: 00000000bffc9b26/0xa40083ab9bf33982 lrc: 2/0,0 mode: PR/PR res: [0x280000400:0x2a2:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x40000000000000 nid: 192.168.204.3@tcp remote: 0x771f68aa1eb22d9a expref: 41 pid: 8228 timeout: 0 lvb_type: 1 00010000:00010000:0.0:1776178092.472291:0:20759:0:(ldlm_extent.c:318:ldlm_check_contention()) contended locks = 0 00010000:00010000:0.0:1776178092.472292:0:20759:0:(ldlm_extent.c:318:ldlm_check_contention()) contended locks = 0 00010000:00010000:0.0:1776178092.472293:0:20759:0:(ldlm_resource.c:1650:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 000000006b206cc9/0xa40083ab9bf52c8f lrc: 4/0,1 mode: --/PW res: [0x280000400:0x2a2:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x50010080000000 nid: local remote: 0x0 expref: -99 pid: 20759 timeout: 0 lvb_type: 0 00010000:00010000:0.0:1776178092.472322:0:20759:0:(ldlm_lockd.c:953:ldlm_server_blocking_ast()) ### server preparing blocking AST ns: filter-lustre-OST0000_UUID lock: 00000000bffc9b26/0xa40083ab9bf33982 lrc: 3/0,0 mode: PR/PR res: [0x280000400:0x2a2:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x50000000010020 nid: 192.168.204.3@tcp remote: 0x771f68aa1eb22d9a expref: 41 pid: 8228 timeout: 0 lvb_type: 1 00010000:00010000:0.0:1776178092.472329:0:20759:0:(ldlm_lockd.c:489:ldlm_add_waiting_lock()) ### adding to wait list(timeout: 100, AT: on) ns: filter-lustre-OST0000_UUID lock: 00000000bffc9b26/0xa40083ab9bf33982 lrc: 4/0,0 mode: PR/PR res: [0x280000400:0x2a2:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x70000400010020 nid: 192.168.204.3@tcp remote: 0x771f68aa1eb22d9a expref: 41 pid: 8228 timeout: 899 lvb_type: 1 00000100:00100000:0.0:1776178092.472343:0:20759:0:(client.c:1733:ptlrpc_send_new_req()) Sending RPC req@0000000092028508 pname:cluuid:pid:xid:nid:opc:job ll_ost00_010:lustre-OST0000_UUID:20759:1862456998011456:192.168.204.3@tcp:104: 00000100:00100000:0.0:1776178092.472364:0:20759:0:(client.c:2500:ptlrpc_set_wait()) set 000000006fc35fcf going to sleep for 43 seconds 00000100:00100000:0.0:1776178092.472379:0:3356:0:(client.c:1733:ptlrpc_send_new_req()) Sending RPC req@00000000f95e9da5 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_03:lustre-MDT0001-mdtlov_UUID:3356:1862456998011136:0@lo:6:osp-syn-1-1.0 00000100:00100000:0.0:1776178092.472403:0:3356:0:(events.c:361:request_in_callback()) peer: 12345-0@lo (source: 12345-0@lo) 00000100:00100000:0.0:1776178092.472422:0:20762:0:(service.c:2152:ptlrpc_server_handle_req_in()) got req x1862456998011136 00000100:00100000:0.0:1776178092.472439:0:20762:0:(nrs_fifo.c:179:nrs_fifo_req_get()) NRS start fifo request from 12345-0@lo, seq: 5168 00000100:00100000:0.0:1776178092.472442:0:20762:0:(service.c:2324:ptlrpc_server_handle_request()) Handling RPC req@000000008c469ec0 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_013:lustre-MDT0001-mdtlov_UUID+15:3356:x1862456998011136:12345-0@lo:6:osp-syn-1-1.0 00002000:00080000:0.0:1776178092.472452:0:20762:0:(ofd_dev.c:1822:ofd_destroy_hdl()) lustre-OST0001: Destroy object 0x2c0000400:784 count 1 00010000:00010000:0.0:1776178092.472464:0:20762:0:(ldlm_lock.c:766:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0001_UUID lock: 000000001735c8e5/0xa40083ab9bf52c96 lrc: 3/0,1 mode: --/PW res: [0x2c0000400:0x310:0x0].0x0 rrc: 3 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 20762 timeout: 0 lvb_type: 0 00010000:00010000:0.0:1776178092.472473:0:20762:0:(ldlm_lock.c:675:ldlm_add_bl_work_item()) ### lock incompatible; sending blocking AST. ns: filter-lustre-OST0001_UUID lock: 00000000f9398497/0xa40083ab9bf38ee7 lrc: 2/0,0 mode: PR/PR res: [0x2c0000400:0x310:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x40000000000000 nid: 192.168.204.3@tcp remote: 0x771f68aa1eb24a11 expref: 34 pid: 8227 timeout: 0 lvb_type: 1 00010000:00010000:0.0:1776178092.472479:0:20762:0:(ldlm_extent.c:318:ldlm_check_contention()) contended locks = 0 00010000:00010000:0.0:1776178092.472480:0:20762:0:(ldlm_extent.c:318:ldlm_check_contention()) contended locks = 0 00010000:00010000:0.0:1776178092.472481:0:20762:0:(ldlm_resource.c:1650:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0001_UUID lock: 000000001735c8e5/0xa40083ab9bf52c96 lrc: 4/0,1 mode: --/PW res: [0x2c0000400:0x310:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x50010080000000 nid: local remote: 0x0 expref: -99 pid: 20762 timeout: 0 lvb_type: 0 00010000:00010000:0.0:1776178092.472497:0:20762:0:(ldlm_lockd.c:953:ldlm_server_blocking_ast()) ### server preparing blocking AST ns: filter-lustre-OST0001_UUID lock: 00000000f9398497/0xa40083ab9bf38ee7 lrc: 3/0,0 mode: PR/PR res: [0x2c0000400:0x310:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x50000000010020 nid: 192.168.204.3@tcp remote: 0x771f68aa1eb24a11 expref: 34 pid: 8227 timeout: 0 lvb_type: 1 00010000:00010000:0.0:1776178092.472505:0:20762:0:(ldlm_lockd.c:489:ldlm_add_waiting_lock()) ### adding to wait list(timeout: 100, AT: on) ns: filter-lustre-OST0001_UUID lock: 00000000f9398497/0xa40083ab9bf38ee7 lrc: 4/0,0 mode: PR/PR res: [0x2c0000400:0x310:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x70000400010020 nid: 192.168.204.3@tcp remote: 0x771f68aa1eb24a11 expref: 34 pid: 8227 timeout: 899 lvb_type: 1 00000100:00100000:0.0:1776178092.472520:0:20762:0:(client.c:1733:ptlrpc_send_new_req()) Sending RPC req@00000000a2dca46e pname:cluuid:pid:xid:nid:opc:job ll_ost00_013:lustre-OST0001_UUID:20762:1862456998011520:192.168.204.3@tcp:104: 00000100:00100000:0.0:1776178092.472539:0:20762:0:(client.c:2500:ptlrpc_set_wait()) set 0000000096fec317 going to sleep for 6 seconds 00000004:00080000:0.0:1776178092.472560:0:3356:0:(osp_sync.c:562:osp_sync_interpret()) reply req 000000002b0faed1/2, rc 0, transno 2005 00000100:00100000:0.0:1776178092.472568:0:3356:0:(client.c:2215:ptlrpc_check_set()) Completed RPC req@000000002b0faed1 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_03:lustre-MDT0001-mdtlov_UUID:3356:1862456998010304:0@lo:6:osp-syn-0-1.0 00000100:00100000:0.0:1776178092.472615:0:3354:0:(ptlrpcd.c:415:ptlrpcd_check()) transfer 1 async RPCs [0->1] 00000100:00100000:0.0:1776178092.472622:0:3354:0:(client.c:1733:ptlrpc_send_new_req()) Sending RPC req@0000000095214b8e pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_01:lustre-MDT0001-mdtlov_UUID:3354:1862456998011584:0@lo:6:osp-syn-0-1.0 00000100:00100000:0.0:1776178092.472645:0:3354:0:(events.c:361:request_in_callback()) peer: 12345-0@lo (source: 12345-0@lo) 00000100:00100000:0.0:1776178092.472666:0:20755:0:(service.c:2152:ptlrpc_server_handle_req_in()) got req x1862456998011584 00000100:00100000:0.0:1776178092.472685:0:20755:0:(nrs_fifo.c:179:nrs_fifo_req_get()) NRS start fifo request from 12345-0@lo, seq: 5169 00000100:00100000:0.0:1776178092.472688:0:20755:0:(service.c:2324:ptlrpc_server_handle_request()) Handling RPC req@00000000ddc1dc02 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_006:lustre-MDT0001-mdtlov_UUID+19:3354:x1862456998011584:12345-0@lo:6:osp-syn-0-1.0 00002000:00080000:0.0:1776178092.472700:0:20755:0:(ofd_dev.c:1822:ofd_destroy_hdl()) lustre-OST0000: Destroy object 0x280000400:696 count 1 00010000:00010000:0.0:1776178092.472710:0:20755:0:(ldlm_lock.c:766:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0000_UUID lock: 000000008813523d/0xa40083ab9bf52c9d lrc: 3/0,1 mode: --/PW res: [0x280000400:0x2b8:0x0].0x0 rrc: 3 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 20755 timeout: 0 lvb_type: 0 00010000:00010000:0.0:1776178092.472720:0:20755:0:(ldlm_lock.c:675:ldlm_add_bl_work_item()) ### lock incompatible; sending blocking AST. ns: filter-lustre-OST0000_UUID lock: 000000008de4758e/0xa40083ab9bf34456 lrc: 2/0,0 mode: PR/PR res: [0x280000400:0x2b8:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x40000000000000 nid: 192.168.204.3@tcp remote: 0x771f68aa1eb23136 expref: 41 pid: 8645 timeout: 0 lvb_type: 1 00010000:00010000:0.0:1776178092.472726:0:20755:0:(ldlm_extent.c:318:ldlm_check_contention()) contended locks = 0 00010000:00010000:0.0:1776178092.472728:0:20755:0:(ldlm_extent.c:318:ldlm_check_contention()) contended locks = 0 00010000:00010000:0.0:1776178092.472729:0:20755:0:(ldlm_resource.c:1650:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 000000008813523d/0xa40083ab9bf52c9d lrc: 4/0,1 mode: --/PW res: [0x280000400:0x2b8:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x50010080000000 nid: local remote: 0x0 expref: -99 pid: 20755 timeout: 0 lvb_type: 0 00010000:00010000:0.0:1776178092.472746:0:20755:0:(ldlm_lockd.c:953:ldlm_server_blocking_ast()) ### server preparing blocking AST ns: filter-lustre-OST0000_UUID lock: 000000008de4758e/0xa40083ab9bf34456 lrc: 3/0,0 mode: PR/PR res: [0x280000400:0x2b8:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x50000000010020 nid: 192.168.204.3@tcp remote: 0x771f68aa1eb23136 expref: 41 pid: 8645 timeout: 0 lvb_type: 1 00010000:00010000:0.0:1776178092.472759:0:20755:0:(ldlm_lockd.c:489:ldlm_add_waiting_lock()) ### adding to wait list(timeout: 100, AT: on) ns: filter-lustre-OST0000_UUID lock: 000000008de4758e/0xa40083ab9bf34456 lrc: 4/0,0 mode: PR/PR res: [0x280000400:0x2b8:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x70000400010020 nid: 192.168.204.3@tcp remote: 0x771f68aa1eb23136 expref: 41 pid: 8645 timeout: 899 lvb_type: 1 00000100:00100000:0.0:1776178092.472767:0:20755:0:(client.c:1733:ptlrpc_send_new_req()) Sending RPC req@00000000f141ac55 pname:cluuid:pid:xid:nid:opc:job ll_ost00_006:lustre-OST0000_UUID:20755:1862456998011648:192.168.204.3@tcp:104: 00000100:00100000:0.0:1776178092.472792:0:20755:0:(client.c:2500:ptlrpc_set_wait()) set 00000000fe3ee5f3 going to sleep for 43 seconds 00000004:00080000:0.0:1776178092.472813:0:3356:0:(osp_sync.c:562:osp_sync_interpret()) reply req 000000005f1d3350/2, rc 0, transno 2002 00000100:00100000:0.0:1776178092.472820:0:3356:0:(client.c:2215:ptlrpc_check_set()) Completed RPC req@000000005f1d3350 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_03:lustre-MDT0001-mdtlov_UUID:3356:1862456998008384:0@lo:6:osp-syn-0-1.0 00000004:00080000:0.0:1776178092.472872:0:3356:0:(osp_sync.c:562:osp_sync_interpret()) reply req 000000005d95861f/2, rc 0, transno 2001 00000100:00100000:0.0:1776178092.472879:0:3356:0:(client.c:2215:ptlrpc_check_set()) Completed RPC req@000000005d95861f pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_03:lustre-MDT0001-mdtlov_UUID:3356:1862456998008640:0@lo:6:osp-syn-0-1.0 00000100:00100000:0.0:1776178092.472904:0:3355:0:(client.c:1733:ptlrpc_send_new_req()) Sending RPC req@000000001554c6b4 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_02:lustre-MDT0001-mdtlov_UUID:3355:1862456998011776:0@lo:6:osp-syn-0-1.0 00000100:00100000:0.0:1776178092.472931:0:3355:0:(events.c:361:request_in_callback()) peer: 12345-0@lo (source: 12345-0@lo) 00000100:00100000:0.0:1776178092.472953:0:20757:0:(service.c:2152:ptlrpc_server_handle_req_in()) got req x1862456998011776 00000100:00100000:0.0:1776178092.472976:0:20757:0:(nrs_fifo.c:179:nrs_fifo_req_get()) NRS start fifo request from 12345-0@lo, seq: 5170 00000100:00100000:0.0:1776178092.472980:0:20757:0:(service.c:2324:ptlrpc_server_handle_request()) Handling RPC req@00000000f6af7e5b pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_008:lustre-MDT0001-mdtlov_UUID+20:3355:x1862456998011776:12345-0@lo:6:osp-syn-0-1.0 00002000:00080000:0.0:1776178092.472990:0:20757:0:(ofd_dev.c:1822:ofd_destroy_hdl()) lustre-OST0000: Destroy object 0x280000400:845 count 1 00010000:00010000:0.0:1776178092.473000:0:20757:0:(ldlm_lock.c:766:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0000_UUID lock: 000000000e7a37e0/0xa40083ab9bf52ca4 lrc: 3/0,1 mode: --/PW res: [0x280000400:0x34d:0x0].0x0 rrc: 3 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 20757 timeout: 0 lvb_type: 0 00010000:00010000:0.0:1776178092.473011:0:20757:0:(ldlm_lock.c:675:ldlm_add_bl_work_item()) ### lock incompatible; sending blocking AST. ns: filter-lustre-OST0000_UUID lock: 000000001e512550/0xa40083ab9bf38dac lrc: 2/0,0 mode: PR/PR res: [0x280000400:0x34d:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x40000000000000 nid: 192.168.204.3@tcp remote: 0x771f68aa1eb249a8 expref: 41 pid: 8645 timeout: 0 lvb_type: 1 00010000:00010000:0.0:1776178092.473017:0:20757:0:(ldlm_extent.c:318:ldlm_check_contention()) contended locks = 0 00010000:00010000:0.0:1776178092.473019:0:20757:0:(ldlm_extent.c:318:ldlm_check_contention()) contended locks = 0 00010000:00010000:0.0:1776178092.473020:0:20757:0:(ldlm_resource.c:1650:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 000000000e7a37e0/0xa40083ab9bf52ca4 lrc: 4/0,1 mode: --/PW res: [0x280000400:0x34d:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x50010080000000 nid: local remote: 0x0 expref: -99 pid: 20757 timeout: 0 lvb_type: 0 00010000:00010000:0.0:1776178092.473037:0:20757:0:(ldlm_lockd.c:953:ldlm_server_blocking_ast()) ### server preparing blocking AST ns: filter-lustre-OST0000_UUID lock: 000000001e512550/0xa40083ab9bf38dac lrc: 3/0,0 mode: PR/PR res: [0x280000400:0x34d:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x50000000010020 nid: 192.168.204.3@tcp remote: 0x771f68aa1eb249a8 expref: 41 pid: 8645 timeout: 0 lvb_type: 1 00010000:00010000:0.0:1776178092.473045:0:20757:0:(ldlm_lockd.c:489:ldlm_add_waiting_lock()) ### adding to wait list(timeout: 100, AT: on) ns: filter-lustre-OST0000_UUID lock: 000000001e512550/0xa40083ab9bf38dac lrc: 4/0,0 mode: PR/PR res: [0x280000400:0x34d:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x70000400010020 nid: 192.168.204.3@tcp remote: 0x771f68aa1eb249a8 expref: 41 pid: 8645 timeout: 899 lvb_type: 1 00000100:00100000:0.0:1776178092.473054:0:20757:0:(client.c:1733:ptlrpc_send_new_req()) Sending RPC req@00000000a2f460e7 pname:cluuid:pid:xid:nid:opc:job ll_ost00_008:lustre-OST0000_UUID:20757:1862456998011840:192.168.204.3@tcp:104: 00000100:00100000:0.0:1776178092.473420:0:20757:0:(client.c:2500:ptlrpc_set_wait()) set 0000000020a93216 going to sleep for 43 seconds 00000004:00080000:0.0:1776178092.473466:0:3356:0:(osp_sync.c:562:osp_sync_interpret()) reply req 00000000544c54c4/2, rc 0, transno 1981 00000100:00100000:0.0:1776178092.473470:0:3356:0:(client.c:2215:ptlrpc_check_set()) Completed RPC req@00000000544c54c4 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_03:lustre-MDT0001-mdtlov_UUID:3356:1862456998008896:0@lo:6:osp-syn-1-1.0 00000004:00080000:0.0:1776178092.473480:0:3356:0:(osp_sync.c:562:osp_sync_interpret()) reply req 00000000b8464782/2, rc 0, transno 1982 00000100:00100000:0.0:1776178092.473482:0:3356:0:(client.c:2215:ptlrpc_check_set()) Completed RPC req@00000000b8464782 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_03:lustre-MDT0001-mdtlov_UUID:3356:1862456998009152:0@lo:6:osp-syn-1-1.0 00000004:00080000:0.0:1776178092.473513:0:3353:0:(osp_sync.c:562:osp_sync_interpret()) reply req 000000005927b2c3/2, rc 0, transno 1988 00000100:00100000:0.0:1776178092.473517:0:3353:0:(client.c:2215:ptlrpc_check_set()) Completed RPC req@000000005927b2c3 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_00:lustre-MDT0001-mdtlov_UUID:3353:1862456998010560:0@lo:6:osp-syn-1-1.0 00000004:00080000:0.0:1776178092.473527:0:3353:0:(osp_sync.c:562:osp_sync_interpret()) reply req 000000006f8bc860/2, rc 0, transno 1987 00000100:00100000:0.0:1776178092.473530:0:3353:0:(client.c:2215:ptlrpc_check_set()) Completed RPC req@000000006f8bc860 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_00:lustre-MDT0001-mdtlov_UUID:3353:1862456998009216:0@lo:6:osp-syn-1-1.0 00000100:00100000:0.0:1776178092.473616:0:3354:0:(client.c:1733:ptlrpc_send_new_req()) Sending RPC req@000000006247494d pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_01:lustre-MDT0001-mdtlov_UUID:3354:1862456998012096:0@lo:6:osp-syn-1-1.0 00000100:00100000:0.0:1776178092.473642:0:3354:0:(events.c:361:request_in_callback()) peer: 12345-0@lo (source: 12345-0@lo) 00000100:00100000:0.0:1776178092.473657:0:3354:0:(ptlrpcd.c:415:ptlrpcd_check()) transfer 2 async RPCs [0->1] 00000100:00100000:0.0:1776178092.473662:0:3354:0:(client.c:1733:ptlrpc_send_new_req()) Sending RPC req@0000000068adf51c pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_01:lustre-MDT0001-mdtlov_UUID:3354:1862456998012032:0@lo:6:osp-syn-1-1.0 00000100:00100000:0.0:1776178092.473678:0:3354:0:(events.c:361:request_in_callback()) peer: 12345-0@lo (source: 12345-0@lo) 00000100:00100000:0.0:1776178092.473690:0:3354:0:(client.c:1733:ptlrpc_send_new_req()) Sending RPC req@00000000ae838296 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_01:lustre-MDT0001-mdtlov_UUID:3354:1862456998012288:0@lo:6:osp-syn-1-1.0 00000100:00100000:0.0:1776178092.473707:0:3354:0:(events.c:361:request_in_callback()) peer: 12345-0@lo (source: 12345-0@lo) 00000100:00100000:0.0:1776178092.473729:0:20761:0:(service.c:2152:ptlrpc_server_handle_req_in()) got req x1862456998012288 00000100:00100000:0.0:1776178092.473747:0:20761:0:(nrs_fifo.c:179:nrs_fifo_req_get()) NRS start fifo request from 12345-0@lo, seq: 5173 00000100:00100000:0.0:1776178092.473750:0:20761:0:(service.c:2324:ptlrpc_server_handle_request()) Handling RPC req@000000007fbc2285 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_012:lustre-MDT0001-mdtlov_UUID+19:3356:x1862456998011968:12345-0@lo:6:osp-syn-1-1.0 00002000:00080000:0.0:1776178092.473762:0:20761:0:(ofd_dev.c:1822:ofd_destroy_hdl()) lustre-OST0001: Destroy object 0x2c0000400:813 count 1 00010000:00010000:0.0:1776178092.473775:0:20761:0:(ldlm_lock.c:766:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0001_UUID lock: 00000000a211f94c/0xa40083ab9bf52cb2 lrc: 3/0,1 mode: --/PW res: [0x2c0000400:0x32d:0x0].0x0 rrc: 3 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 20761 timeout: 0 lvb_type: 0 00010000:00010000:0.0:1776178092.473786:0:20761:0:(ldlm_lock.c:675:ldlm_add_bl_work_item()) ### lock incompatible; sending blocking AST. ns: filter-lustre-OST0001_UUID lock: 0000000024905eb4/0xa40083ab9bf39d2d lrc: 2/0,0 mode: PR/PR res: [0x2c0000400:0x32d:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x40000000000000 nid: 192.168.204.3@tcp remote: 0x771f68aa1eb24ed3 expref: 34 pid: 8230 timeout: 0 lvb_type: 1 00010000:00010000:0.0:1776178092.473792:0:20761:0:(ldlm_extent.c:318:ldlm_check_contention()) contended locks = 0 00010000:00010000:0.0:1776178092.473793:0:20761:0:(ldlm_extent.c:318:ldlm_check_contention()) contended locks = 0 00010000:00010000:0.0:1776178092.473794:0:20761:0:(ldlm_resource.c:1650:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0001_UUID lock: 00000000a211f94c/0xa40083ab9bf52cb2 lrc: 4/0,1 mode: --/PW res: [0x2c0000400:0x32d:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x50010080000000 nid: local remote: 0x0 expref: -99 pid: 20761 timeout: 0 lvb_type: 0 00010000:00010000:0.0:1776178092.473815:0:20761:0:(ldlm_lockd.c:953:ldlm_server_blocking_ast()) ### server preparing blocking AST ns: filter-lustre-OST0001_UUID lock: 0000000024905eb4/0xa40083ab9bf39d2d lrc: 3/0,0 mode: PR/PR res: [0x2c0000400:0x32d:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x50000000010020 nid: 192.168.204.3@tcp remote: 0x771f68aa1eb24ed3 expref: 34 pid: 8230 timeout: 0 lvb_type: 1 00010000:00010000:0.0:1776178092.473824:0:20761:0:(ldlm_lockd.c:489:ldlm_add_waiting_lock()) ### adding to wait list(timeout: 100, AT: on) ns: filter-lustre-OST0001_UUID lock: 0000000024905eb4/0xa40083ab9bf39d2d lrc: 4/0,0 mode: PR/PR res: [0x2c0000400:0x32d:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x70000400010020 nid: 192.168.204.3@tcp remote: 0x771f68aa1eb24ed3 expref: 34 pid: 8230 timeout: 899 lvb_type: 1 00000100:00100000:0.0:1776178092.473847:0:20761:0:(client.c:1733:ptlrpc_send_new_req()) Sending RPC req@000000005fdea934 pname:cluuid:pid:xid:nid:opc:job ll_ost00_012:lustre-OST0001_UUID:20761:1862456998012352:192.168.204.3@tcp:104: 00000100:00100000:0.0:1776178092.473877:0:20761:0:(client.c:2500:ptlrpc_set_wait()) set 00000000e015c18b going to sleep for 6 seconds 00000100:00100000:0.0:1776178092.473894:0:8228:0:(nrs_fifo.c:179:nrs_fifo_req_get()) NRS start fifo request from 12345-0@lo, seq: 5174 00000100:00100000:0.0:1776178092.473898:0:8228:0:(service.c:2324:ptlrpc_server_handle_request()) Handling RPC req@00000000a9cfe89c pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_001:lustre-MDT0001-mdtlov_UUID+19:3354:x1862456998012032:12345-0@lo:6:osp-syn-1-1.0 00002000:00080000:0.0:1776178092.473910:0:8228:0:(ofd_dev.c:1822:ofd_destroy_hdl()) lustre-OST0001: Destroy object 0x2c0000400:926 count 1 00010000:00010000:0.0:1776178092.473921:0:8228:0:(ldlm_lock.c:766:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0001_UUID lock: 00000000782e253e/0xa40083ab9bf52cc0 lrc: 3/0,1 mode: --/PW res: [0x2c0000400:0x39e:0x0].0x0 rrc: 3 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 8228 timeout: 0 lvb_type: 0 00010000:00010000:0.0:1776178092.473932:0:8228:0:(ldlm_lock.c:675:ldlm_add_bl_work_item()) ### lock incompatible; sending blocking AST. ns: filter-lustre-OST0001_UUID lock: 0000000000bde6bf/0xa40083ab9bf3d4cb lrc: 2/0,0 mode: PR/PR res: [0x2c0000400:0x39e:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x40000000000000 nid: 192.168.204.3@tcp remote: 0x771f68aa1eb2615d expref: 34 pid: 8230 timeout: 0 lvb_type: 1 00010000:00010000:0.0:1776178092.473938:0:8228:0:(ldlm_extent.c:318:ldlm_check_contention()) contended locks = 0 00010000:00010000:0.0:1776178092.473939:0:8228:0:(ldlm_extent.c:318:ldlm_check_contention()) contended locks = 0 00010000:00010000:0.0:1776178092.473941:0:8228:0:(ldlm_resource.c:1650:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0001_UUID lock: 00000000782e253e/0xa40083ab9bf52cc0 lrc: 4/0,1 mode: --/PW res: [0x2c0000400:0x39e:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x50010080000000 nid: local remote: 0x0 expref: -99 pid: 8228 timeout: 0 lvb_type: 0 00010000:00010000:0.0:1776178092.473958:0:8228:0:(ldlm_lockd.c:953:ldlm_server_blocking_ast()) ### server preparing blocking AST ns: filter-lustre-OST0001_UUID lock: 0000000000bde6bf/0xa40083ab9bf3d4cb lrc: 3/0,0 mode: PR/PR res: [0x2c0000400:0x39e:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x50000000010020 nid: 192.168.204.3@tcp remote: 0x771f68aa1eb2615d expref: 34 pid: 8230 timeout: 0 lvb_type: 1 00010000:00010000:0.0:1776178092.473965:0:8228:0:(ldlm_lockd.c:489:ldlm_add_waiting_lock()) ### adding to wait list(timeout: 100, AT: on) ns: filter-lustre-OST0001_UUID lock: 0000000000bde6bf/0xa40083ab9bf3d4cb lrc: 4/0,0 mode: PR/PR res: [0x2c0000400:0x39e:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x70000400010020 nid: 192.168.204.3@tcp remote: 0x771f68aa1eb2615d expref: 34 pid: 8230 timeout: 899 lvb_type: 1 00000100:00100000:0.0:1776178092.473973:0:8228:0:(client.c:1733:ptlrpc_send_new_req()) Sending RPC req@0000000078c1d864 pname:cluuid:pid:xid:nid:opc:job ll_ost00_001:lustre-OST0001_UUID:8228:1862456998012480:192.168.204.3@tcp:104: 00000100:00100000:0.0:1776178092.473994:0:8228:0:(client.c:2500:ptlrpc_set_wait()) set 0000000010774d6e going to sleep for 6 seconds 00000100:00100000:0.0:1776178092.474132:0:3355:0:(client.c:1733:ptlrpc_send_new_req()) Sending RPC req@00000000124cea36 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_02:lustre-MDT0001-mdtlov_UUID:3355:1862456998012160:0@lo:6:osp-syn-1-1.0 00000100:00100000:0.0:1776178092.474159:0:3355:0:(events.c:361:request_in_callback()) peer: 12345-0@lo (source: 12345-0@lo) 00000100:00100000:0.0:1776178092.474179:0:20763:0:(service.c:2152:ptlrpc_server_handle_req_in()) got req x1862456998012160 00000100:00100000:0.0:1776178092.474195:0:20763:0:(nrs_fifo.c:179:nrs_fifo_req_get()) NRS start fifo request from 12345-0@lo, seq: 5176 00000100:00100000:0.0:1776178092.474199:0:20763:0:(service.c:2324:ptlrpc_server_handle_request()) Handling RPC req@000000000c1c3037 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_014:lustre-MDT0001-mdtlov_UUID+20:3355:x1862456998012160:12345-0@lo:6:osp-syn-1-1.0 00002000:00080000:0.0:1776178092.474210:0:20763:0:(ofd_dev.c:1822:ofd_destroy_hdl()) lustre-OST0001: Destroy object 0x2c0000400:737 count 1 00010000:00010000:0.0:1776178092.474221:0:20763:0:(ldlm_lock.c:766:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0001_UUID lock: 0000000068850c85/0xa40083ab9bf52cce lrc: 3/0,1 mode: --/PW res: [0x2c0000400:0x2e1:0x0].0x0 rrc: 3 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 20763 timeout: 0 lvb_type: 0 00010000:00010000:0.0:1776178092.474233:0:20763:0:(ldlm_lock.c:675:ldlm_add_bl_work_item()) ### lock incompatible; sending blocking AST. ns: filter-lustre-OST0001_UUID lock: 000000003f12accd/0xa40083ab9bf377c5 lrc: 2/0,0 mode: PR/PR res: [0x2c0000400:0x2e1:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x40000000000000 nid: 192.168.204.3@tcp remote: 0x771f68aa1eb2425b expref: 34 pid: 8645 timeout: 0 lvb_type: 1 00010000:00010000:0.0:1776178092.474240:0:20763:0:(ldlm_extent.c:318:ldlm_check_contention()) contended locks = 0 00010000:00010000:0.0:1776178092.474241:0:20763:0:(ldlm_extent.c:318:ldlm_check_contention()) contended locks = 0 00010000:00010000:0.0:1776178092.474242:0:20763:0:(ldlm_resource.c:1650:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0001_UUID lock: 0000000068850c85/0xa40083ab9bf52cce lrc: 4/0,1 mode: --/PW res: [0x2c0000400:0x2e1:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x50010080000000 nid: local remote: 0x0 expref: -99 pid: 20763 timeout: 0 lvb_type: 0 00010000:00010000:0.0:1776178092.474263:0:20763:0:(ldlm_lockd.c:953:ldlm_server_blocking_ast()) ### server preparing blocking AST ns: filter-lustre-OST0001_UUID lock: 000000003f12accd/0xa40083ab9bf377c5 lrc: 3/0,0 mode: PR/PR res: [0x2c0000400:0x2e1:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x50000000010020 nid: 192.168.204.3@tcp remote: 0x771f68aa1eb2425b expref: 34 pid: 8645 timeout: 0 lvb_type: 1 00010000:00010000:0.0:1776178092.474271:0:20763:0:(ldlm_lockd.c:489:ldlm_add_waiting_lock()) ### adding to wait list(timeout: 100, AT: on) ns: filter-lustre-OST0001_UUID lock: 000000003f12accd/0xa40083ab9bf377c5 lrc: 4/0,0 mode: PR/PR res: [0x2c0000400:0x2e1:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x70000400010020 nid: 192.168.204.3@tcp remote: 0x771f68aa1eb2425b expref: 34 pid: 8645 timeout: 899 lvb_type: 1 00000100:00100000:0.0:1776178092.474280:0:20763:0:(client.c:1733:ptlrpc_send_new_req()) Sending RPC req@000000009126d8c6 pname:cluuid:pid:xid:nid:opc:job ll_ost00_014:lustre-OST0001_UUID:20763:1862456998012608:192.168.204.3@tcp:104: 00000100:00100000:0.0:1776178092.474310:0:20763:0:(client.c:2500:ptlrpc_set_wait()) set 00000000955e624b going to sleep for 6 seconds 00010000:00010000:0.0:1776178092.502317:0:20757:0:(ldlm_lock.c:216:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 000000001e512550/0xa40083ab9bf38dac lrc: 0/0,0 mode: --/PR res: [0x280000400:0x34d:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x44801400010020 nid: 192.168.204.3@tcp remote: 0x771f68aa1eb249a8 expref: 35 pid: 8645 timeout: 899 lvb_type: 1 00000100:00100000:0.0:1776178092.502331:0:20757:0:(client.c:2215:ptlrpc_check_set()) Completed RPC req@00000000a2f460e7 pname:cluuid:pid:xid:nid:opc:job ll_ost00_008:lustre-OST0000_UUID:20757:1862456998011840:192.168.204.3@tcp:104: 00010000:00010000:0.0:1776178092.502344:0:20757:0:(ldlm_request.c:283:ldlm_completion_ast()) ### client-side enqueue returned a blocked locksleeping ns: filter-lustre-OST0000_UUID lock: 000000000e7a37e0/0xa40083ab9bf52ca4 lrc: 3/0,1 mode: PW/PW res: [0x280000400:0x34d:0x0].0x0 rrc: 2 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x40010080000000 nid: local remote: 0x0 expref: -99 pid: 20757 timeout: 0 lvb_type: 0 00010000:00010000:0.0:1776178092.502369:0:8228:0:(ldlm_lock.c:216:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0001_UUID lock: 0000000000bde6bf/0xa40083ab9bf3d4cb lrc: 0/0,0 mode: --/PR res: [0x2c0000400:0x39e:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x44801400010020 nid: 192.168.204.3@tcp remote: 0x771f68aa1eb2615d expref: 30 pid: 8230 timeout: 899 lvb_type: 1 00000100:00100000:0.0:1776178092.502380:0:8228:0:(client.c:2215:ptlrpc_check_set()) Completed RPC req@0000000078c1d864 pname:cluuid:pid:xid:nid:opc:job ll_ost00_001:lustre-OST0001_UUID:8228:1862456998012480:192.168.204.3@tcp:104: 00010000:00010000:0.0:1776178092.502392:0:8228:0:(ldlm_request.c:283:ldlm_completion_ast()) ### client-side enqueue returned a blocked locksleeping ns: filter-lustre-OST0001_UUID lock: 00000000782e253e/0xa40083ab9bf52cc0 lrc: 3/0,1 mode: PW/PW res: [0x2c0000400:0x39e:0x0].0x0 rrc: 2 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x40010080000000 nid: local remote: 0x0 expref: -99 pid: 8228 timeout: 0 lvb_type: 0 00010000:00010000:0.0:1776178092.502399:0:8228:0:(ldlm_request.c:199:ldlm_completion_tail()) ### client-side enqueue: granted ns: filter-lustre-OST0001_UUID lock: 00000000782e253e/0xa40083ab9bf52cc0 lrc: 3/0,1 mode: PW/PW res: [0x2c0000400:0x39e:0x0].0x0 rrc: 2 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x40010080000000 nid: local remote: 0x0 expref: -99 pid: 8228 timeout: 0 lvb_type: 0 00010000:00010000:0.0:1776178092.502405:0:8228:0:(ldlm_request.c:528:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0001_UUID lock: 00000000782e253e/0xa40083ab9bf52cc0 lrc: 3/0,1 mode: PW/PW res: [0x2c0000400:0x39e:0x0].0x0 rrc: 2 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x40010080000000 nid: local remote: 0x0 expref: -99 pid: 8228 timeout: 0 lvb_type: 0 00010000:00010000:0.0:1776178092.502412:0:8228:0:(ldlm_lock.c:820:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0001_UUID lock: 00000000782e253e/0xa40083ab9bf52cc0 lrc: 3/0,1 mode: PW/PW res: [0x2c0000400:0x39e:0x0].0x0 rrc: 2 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x50010080000000 nid: local remote: 0x0 expref: -99 pid: 8228 timeout: 0 lvb_type: 0 00010000:00010000:0.0:1776178092.502417:0:8228:0:(ldlm_lock.c:879:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0001_UUID lock: 00000000782e253e/0xa40083ab9bf52cc0 lrc: 2/0,0 mode: PW/PW res: [0x2c0000400:0x39e:0x0].0x0 rrc: 2 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x50010480000000 nid: local remote: 0x0 expref: -99 pid: 8228 timeout: 0 lvb_type: 0 00010000:00010000:0.0:1776178092.502432:0:8228:0:(ldlm_lockd.c:2159:__ldlm_bl_to_thread()) added 1/3 locks to regular blp list, 3 blwis in pool 00000100:00100000:0.0:1776178092.508303:0:8228:0:(service.c:2372:ptlrpc_server_handle_request()) Handled RPC req@00000000a9cfe89c pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_001:lustre-MDT0001-mdtlov_UUID+25:3354:x1862456998012032:12345-0@lo:6:osp-syn-1-1.0 Request processed in 34401us (34621us total) trans 4294969289 rc 0/0 00000100:00100000:0.0:1776178092.508315:0:8228:0:(nrs_fifo.c:241:nrs_fifo_req_stop()) NRS stop fifo request from 12345-0@lo, seq: 5174 00000100:00100000:0.0:1776178092.508353:0:8228:0:(service.c:2152:ptlrpc_server_handle_req_in()) got req x1862456998013760 00000100:00100000:0.0:1776178092.508386:0:8228:0:(nrs_fifo.c:179:nrs_fifo_req_get()) NRS start fifo request from 12345-0@lo, seq: 5186 00000100:00100000:0.0:1776178092.508390:0:8228:0:(service.c:2324:ptlrpc_server_handle_request()) Handling RPC req@00000000efb13deb pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_001:lustre-MDT0001-mdtlov_UUID+27:3354:x1862456998013760:12345-0@lo:6:osp-syn-0-1.0 00002000:00080000:0.0:1776178092.508412:0:8228:0:(ofd_dev.c:1822:ofd_destroy_hdl()) lustre-OST0000: Destroy object 0x280000400:900 count 1 00010000:00010000:0.0:1776178092.508439:0:8228:0:(ldlm_lock.c:766:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0000_UUID lock: 00000000f90c1e39/0xa40083ab9bf52d14 lrc: 3/0,1 mode: --/PW res: [0x280000400:0x384:0x0].0x0 rrc: 3 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 8228 timeout: 0 lvb_type: 0 00010000:00010000:0.0:1776178092.508456:0:8228:0:(ldlm_lock.c:675:ldlm_add_bl_work_item()) ### lock incompatible; sending blocking AST. ns: filter-lustre-OST0000_UUID lock: 000000002d78274e/0xa40083ab9bf3a8be lrc: 2/0,0 mode: PR/PR res: [0x280000400:0x384:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x40000000000000 nid: 192.168.204.3@tcp remote: 0x771f68aa1eb252ae expref: 34 pid: 8645 timeout: 0 lvb_type: 1 00010000:00010000:0.0:1776178092.508465:0:8228:0:(ldlm_extent.c:318:ldlm_check_contention()) contended locks = 0 00010000:00010000:0.0:1776178092.508467:0:8228:0:(ldlm_extent.c:318:ldlm_check_contention()) contended locks = 0 00010000:00010000:0.0:1776178092.508469:0:8228:0:(ldlm_resource.c:1650:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 00000000f90c1e39/0xa40083ab9bf52d14 lrc: 4/0,1 mode: --/PW res: [0x280000400:0x384:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x50010080000000 nid: local remote: 0x0 expref: -99 pid: 8228 timeout: 0 lvb_type: 0 00010000:00010000:0.0:1776178092.508497:0:8228:0:(ldlm_lockd.c:953:ldlm_server_blocking_ast()) ### server preparing blocking AST ns: filter-lustre-OST0000_UUID lock: 000000002d78274e/0xa40083ab9bf3a8be lrc: 3/0,0 mode: PR/PR res: [0x280000400:0x384:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x50000000010020 nid: 192.168.204.3@tcp remote: 0x771f68aa1eb252ae expref: 34 pid: 8645 timeout: 0 lvb_type: 1 00010000:00010000:0.0:1776178092.508508:0:8228:0:(ldlm_lockd.c:489:ldlm_add_waiting_lock()) ### adding to wait list(timeout: 100, AT: on) ns: filter-lustre-OST0000_UUID lock: 000000002d78274e/0xa40083ab9bf3a8be lrc: 4/0,0 mode: PR/PR res: [0x280000400:0x384:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x70000400010020 nid: 192.168.204.3@tcp remote: 0x771f68aa1eb252ae expref: 34 pid: 8645 timeout: 899 lvb_type: 1 00000100:00100000:0.0:1776178092.508521:0:8228:0:(client.c:1733:ptlrpc_send_new_req()) Sending RPC req@0000000078c1d864 pname:cluuid:pid:xid:nid:opc:job ll_ost00_001:lustre-OST0000_UUID:8228:1862456998013952:192.168.204.3@tcp:104: 00000100:00100000:0.0:1776178092.508862:0:8228:0:(client.c:2500:ptlrpc_set_wait()) set 00000000835d6110 going to sleep for 43 seconds 00000004:00080000:0.0:1776178092.508892:0:3354:0:(osp_sync.c:562:osp_sync_interpret()) reply req 0000000068adf51c/2, rc 0, transno 1993 00000100:00100000:0.0:1776178092.508902:0:3354:0:(client.c:2215:ptlrpc_check_set()) Completed RPC req@0000000068adf51c pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_01:lustre-MDT0001-mdtlov_UUID:3354:1862456998012032:0@lo:6:osp-syn-1-1.0 00000100:00100000:0.0:1776178092.508935:0:3354:0:(ptlrpcd.c:415:ptlrpcd_check()) transfer 1 async RPCs [0->1] 00000100:00100000:0.0:1776178092.508945:0:3354:0:(client.c:1733:ptlrpc_send_new_req()) Sending RPC req@00000000a2f460e7 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_01:lustre-MDT0001-mdtlov_UUID:3354:1862456998014016:0@lo:6:osp-syn-1-1.0 00000100:00100000:0.0:1776178092.508982:0:3354:0:(events.c:361:request_in_callback()) peer: 12345-0@lo (source: 12345-0@lo) 00000100:00100000:0.0:1776178092.509013:0:20758:0:(service.c:2152:ptlrpc_server_handle_req_in()) got req x1862456998013824 00000100:00100000:0.0:1776178092.509038:0:20758:0:(service.c:2152:ptlrpc_server_handle_req_in()) got req x1862456998014016 00000100:00100000:0.0:1776178092.509055:0:20758:0:(nrs_fifo.c:179:nrs_fifo_req_get()) NRS start fifo request from 12345-0@lo, seq: 5187 00000100:00100000:0.0:1776178092.509081:0:20758:0:(service.c:2324:ptlrpc_server_handle_request()) Handling RPC req@0000000015095c30 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_009:lustre-MDT0001-mdtlov_UUID+28:3355:x1862456998013824:12345-0@lo:6:osp-syn-0-1.0 00002000:00080000:0.0:1776178092.509096:0:20758:0:(ofd_dev.c:1822:ofd_destroy_hdl()) lustre-OST0000: Destroy object 0x280000400:711 count 1 00010000:00010000:0.0:1776178092.509110:0:20758:0:(ldlm_lock.c:766:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0000_UUID lock: 000000000b903578/0xa40083ab9bf52d1b lrc: 3/0,1 mode: --/PW res: [0x280000400:0x2c7:0x0].0x0 rrc: 3 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 20758 timeout: 0 lvb_type: 0 00010000:00010000:0.0:1776178092.509139:0:20758:0:(ldlm_lock.c:675:ldlm_add_bl_work_item()) ### lock incompatible; sending blocking AST. ns: filter-lustre-OST0000_UUID lock: 00000000465ae959/0xa40083ab9bf34bb8 lrc: 2/0,0 mode: PR/PR res: [0x280000400:0x2c7:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x40000000000000 nid: 192.168.204.3@tcp remote: 0x771f68aa1eb233ac expref: 34 pid: 8645 timeout: 0 lvb_type: 1 00010000:00010000:0.0:1776178092.509147:0:20758:0:(ldlm_extent.c:318:ldlm_check_contention()) contended locks = 0 00010000:00010000:0.0:1776178092.509149:0:20758:0:(ldlm_extent.c:318:ldlm_check_contention()) contended locks = 0 00010000:00010000:0.0:1776178092.509151:0:20758:0:(ldlm_resource.c:1650:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 000000000b903578/0xa40083ab9bf52d1b lrc: 4/0,1 mode: --/PW res: [0x280000400:0x2c7:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x50010080000000 nid: local remote: 0x0 expref: -99 pid: 20758 timeout: 0 lvb_type: 0 00010000:00010000:0.0:1776178092.509170:0:20758:0:(ldlm_lockd.c:953:ldlm_server_blocking_ast()) ### server preparing blocking AST ns: filter-lustre-OST0000_UUID lock: 00000000465ae959/0xa40083ab9bf34bb8 lrc: 3/0,0 mode: PR/PR res: [0x280000400:0x2c7:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x50000000010020 nid: 192.168.204.3@tcp remote: 0x771f68aa1eb233ac expref: 34 pid: 8645 timeout: 0 lvb_type: 1 00010000:00010000:0.0:1776178092.509179:0:20758:0:(ldlm_lockd.c:489:ldlm_add_waiting_lock()) ### adding to wait list(timeout: 100, AT: on) ns: filter-lustre-OST0000_UUID lock: 00000000465ae959/0xa40083ab9bf34bb8 lrc: 4/0,0 mode: PR/PR res: [0x280000400:0x2c7:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x70000400010020 nid: 192.168.204.3@tcp remote: 0x771f68aa1eb233ac expref: 34 pid: 8645 timeout: 899 lvb_type: 1 00000100:00100000:0.0:1776178092.509188:0:20758:0:(client.c:1733:ptlrpc_send_new_req()) Sending RPC req@000000001f0b1867 pname:cluuid:pid:xid:nid:opc:job ll_ost00_009:lustre-OST0000_UUID:20758:1862456998014080:192.168.204.3@tcp:104: 00000100:00100000:0.0:1776178092.509214:0:20758:0:(client.c:2500:ptlrpc_set_wait()) set 000000004f92f0a0 going to sleep for 43 seconds 00000100:00100000:0.0:1776178092.509236:0:14440:0:(nrs_fifo.c:179:nrs_fifo_req_get()) NRS start fifo request from 12345-0@lo, seq: 5188 00000100:00100000:0.0:1776178092.509243:0:14440:0:(service.c:2324:ptlrpc_server_handle_request()) Handling RPC req@00000000599b0cf2 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_004:lustre-MDT0001-mdtlov_UUID+25:3354:x1862456998014016:12345-0@lo:6:osp-syn-1-1.0 00002000:00080000:0.0:1776178092.509258:0:14440:0:(ofd_dev.c:1822:ofd_destroy_hdl()) lustre-OST0001: Destroy object 0x2c0000400:1031 count 1 00010000:00010000:0.0:1776178092.509273:0:14440:0:(ldlm_lock.c:766:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0001_UUID lock: 0000000030bf6b1e/0xa40083ab9bf52d22 lrc: 3/0,1 mode: --/PW res: [0x2c0000400:0x407:0x0].0x0 rrc: 3 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 14440 timeout: 0 lvb_type: 0 00010000:00010000:0.0:1776178092.509285:0:14440:0:(ldlm_lock.c:675:ldlm_add_bl_work_item()) ### lock incompatible; sending blocking AST. ns: filter-lustre-OST0001_UUID lock: 0000000067990ad0/0xa40083ab9bf40879 lrc: 2/0,0 mode: PR/PR res: [0x2c0000400:0x407:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x40000000000000 nid: 192.168.204.3@tcp remote: 0x771f68aa1eb27297 expref: 29 pid: 8645 timeout: 0 lvb_type: 1 00010000:00010000:0.0:1776178092.509291:0:14440:0:(ldlm_extent.c:318:ldlm_check_contention()) contended locks = 0 00010000:00010000:0.0:1776178092.509293:0:14440:0:(ldlm_extent.c:318:ldlm_check_contention()) contended locks = 0 00010000:00010000:0.0:1776178092.509294:0:14440:0:(ldlm_resource.c:1650:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0001_UUID lock: 0000000030bf6b1e/0xa40083ab9bf52d22 lrc: 4/0,1 mode: --/PW res: [0x2c0000400:0x407:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x50010080000000 nid: local remote: 0x0 expref: -99 pid: 14440 timeout: 0 lvb_type: 0 00010000:00010000:0.0:1776178092.509313:0:14440:0:(ldlm_lockd.c:953:ldlm_server_blocking_ast()) ### server preparing blocking AST ns: filter-lustre-OST0001_UUID lock: 0000000067990ad0/0xa40083ab9bf40879 lrc: 3/0,0 mode: PR/PR res: [0x2c0000400:0x407:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x50000000010020 nid: 192.168.204.3@tcp remote: 0x771f68aa1eb27297 expref: 29 pid: 8645 timeout: 0 lvb_type: 1 00010000:00010000:0.0:1776178092.509323:0:14440:0:(ldlm_lockd.c:489:ldlm_add_waiting_lock()) ### adding to wait list(timeout: 100, AT: on) ns: filter-lustre-OST0001_UUID lock: 0000000067990ad0/0xa40083ab9bf40879 lrc: 4/0,0 mode: PR/PR res: [0x2c0000400:0x407:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x70000400010020 nid: 192.168.204.3@tcp remote: 0x771f68aa1eb27297 expref: 29 pid: 8645 timeout: 899 lvb_type: 1 00000100:00100000:0.0:1776178092.509333:0:14440:0:(client.c:1733:ptlrpc_send_new_req()) Sending RPC req@000000008c4bfd71 pname:cluuid:pid:xid:nid:opc:job ll_ost00_004:lustre-OST0001_UUID:14440:1862456998014144:192.168.204.3@tcp:104: 00000100:00100000:0.0:1776178092.509358:0:14440:0:(client.c:2500:ptlrpc_set_wait()) set 000000004d9ad1c9 going to sleep for 43 seconds 00010000:00010000:0.0:1776178092.525112:0:20754:0:(ldlm_lock.c:216:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0001_UUID lock: 00000000a2a1d81a/0xa40083ab9bf37bb5 lrc: 0/0,0 mode: --/PR res: [0x2c0000400:0x2e9:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x44801400010020 nid: 192.168.204.3@tcp remote: 0x771f68aa1eb243ab expref: 27 pid: 14440 timeout: 899 lvb_type: 1 00000100:00100000:0.0:1776178092.525134:0:20754:0:(client.c:2215:ptlrpc_check_set()) Completed RPC req@0000000006fcb74e pname:cluuid:pid:xid:nid:opc:job ll_ost00_005:lustre-OST0001_UUID:20754:1862456998012544:192.168.204.3@tcp:104: 00010000:00010000:0.0:1776178092.525161:0:20754:0:(ldlm_request.c:283:ldlm_completion_ast()) ### client-side enqueue returned a blocked locksleeping ns: filter-lustre-OST0001_UUID lock: 000000006c464689/0xa40083ab9bf52cc7 lrc: 3/0,1 mode: PW/PW res: [0x2c0000400:0x2e9:0x0].0x0 rrc: 2 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x40010080000000 nid: local remote: 0x0 expref: -99 pid: 20754 timeout: 0 lvb_type: 0 00010000:00010000:0.0:1776178092.525174:0:20754:0:(ldlm_request.c:199:ldlm_completion_tail()) ### client-side enqueue: granted ns: filter-lustre-OST0001_UUID lock: 000000006c464689/0xa40083ab9bf52cc7 lrc: 3/0,1 mode: PW/PW res: [0x2c0000400:0x2e9:0x0].0x0 rrc: 2 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x40010080000000 nid: local remote: 0x0 expref: -99 pid: 20754 timeout: 0 lvb_type: 0 00010000:00010000:0.0:1776178092.525180:0:20754:0:(ldlm_request.c:528:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0001_UUID lock: 000000006c464689/0xa40083ab9bf52cc7 lrc: 3/0,1 mode: PW/PW res: [0x2c0000400:0x2e9:0x0].0x0 rrc: 2 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x40010080000000 nid: local remote: 0x0 expref: -99 pid: 20754 timeout: 0 lvb_type: 0 00010000:00010000:0.0:1776178092.525188:0:20754:0:(ldlm_lock.c:820:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0001_UUID lock: 000000006c464689/0xa40083ab9bf52cc7 lrc: 3/0,1 mode: PW/PW res: [0x2c0000400:0x2e9:0x0].0x0 rrc: 2 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x50010080000000 nid: local remote: 0x0 expref: -99 pid: 20754 timeout: 0 lvb_type: 0 00010000:00010000:0.0:1776178092.525201:0:20754:0:(ldlm_lock.c:879:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0001_UUID lock: 000000006c464689/0xa40083ab9bf52cc7 lrc: 2/0,0 mode: PW/PW res: [0x2c0000400:0x2e9:0x0].0x0 rrc: 2 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x50010480000000 nid: local remote: 0x0 expref: -99 pid: 20754 timeout: 0 lvb_type: 0 00010000:00010000:0.0:1776178092.525229:0:20754:0:(ldlm_lockd.c:2159:__ldlm_bl_to_thread()) added 1/1 locks to regular blp list, 1 blwis in pool 00000100:00100000:0.0:1776178092.525534:0:20754:0:(service.c:2372:ptlrpc_server_handle_request()) Handled RPC req@00000000ee45921e pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_005:lustre-MDT0001-mdtlov_UUID+27:3354:x1862456998012288:12345-0@lo:6:osp-syn-1-1.0 Request processed in 51578us (51824us total) trans 4294969291 rc 0/0 00000100:00100000:0.0:1776178092.525547:0:20754:0:(nrs_fifo.c:241:nrs_fifo_req_stop()) NRS stop fifo request from 12345-0@lo, seq: 5175 00000004:00080000:0.0:1776178092.525626:0:3354:0:(osp_sync.c:562:osp_sync_interpret()) reply req 00000000ae838296/2, rc 0, transno 1995 00000100:00100000:0.0:1776178092.525641:0:3354:0:(client.c:2215:ptlrpc_check_set()) Completed RPC req@00000000ae838296 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_01:lustre-MDT0001-mdtlov_UUID:3354:1862456998012288:0@lo:6:osp-syn-1-1.0 00000100:00100000:0.0:1776178092.525750:0:3354:0:(client.c:1733:ptlrpc_send_new_req()) Sending RPC req@000000009126d8c6 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_01:lustre-MDT0001-mdtlov_UUID:3354:1862456998014272:0@lo:6:osp-syn-1-1.0 00000100:00100000:0.0:1776178092.525787:0:3354:0:(events.c:361:request_in_callback()) peer: 12345-0@lo (source: 12345-0@lo) 00000100:00100000:0.0:1776178092.525836:0:20754:0:(service.c:2152:ptlrpc_server_handle_req_in()) got req x1862456998014272 00000100:00100000:0.0:1776178092.525884:0:20754:0:(nrs_fifo.c:179:nrs_fifo_req_get()) NRS start fifo request from 12345-0@lo, seq: 5190 00000100:00100000:0.0:1776178092.525891:0:20754:0:(service.c:2324:ptlrpc_server_handle_request()) Handling RPC req@000000005fdea934 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_005:lustre-MDT0001-mdtlov_UUID+27:3354:x1862456998014272:12345-0@lo:6:osp-syn-1-1.0 00002000:00080000:0.0:1776178092.525924:0:20754:0:(ofd_dev.c:1822:ofd_destroy_hdl()) lustre-OST0001: Destroy object 0x2c0000400:644 count 1 00010000:00010000:0.0:1776178092.525949:0:20754:0:(ldlm_lock.c:766:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0001_UUID lock: 00000000f51fd0d7/0xa40083ab9bf52d30 lrc: 3/0,1 mode: --/PW res: [0x2c0000400:0x284:0x0].0x0 rrc: 3 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 20754 timeout: 0 lvb_type: 0 00010000:00010000:0.0:1776178092.525966:0:20754:0:(ldlm_lock.c:675:ldlm_add_bl_work_item()) ### lock incompatible; sending blocking AST. ns: filter-lustre-OST0001_UUID lock: 0000000000a3384d/0xa40083ab9bf349ff lrc: 2/0,0 mode: PR/PR res: [0x2c0000400:0x284:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x40000000000000 nid: 192.168.204.3@tcp remote: 0x771f68aa1eb23319 expref: 26 pid: 8228 timeout: 0 lvb_type: 1 00010000:00010000:0.0:1776178092.525975:0:20754:0:(ldlm_extent.c:318:ldlm_check_contention()) contended locks = 0 00010000:00010000:0.0:1776178092.525978:0:20754:0:(ldlm_extent.c:318:ldlm_check_contention()) contended locks = 0 00010000:00010000:0.0:1776178092.525981:0:20754:0:(ldlm_resource.c:1650:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0001_UUID lock: 00000000f51fd0d7/0xa40083ab9bf52d30 lrc: 4/0,1 mode: --/PW res: [0x2c0000400:0x284:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x50010080000000 nid: local remote: 0x0 expref: -99 pid: 20754 timeout: 0 lvb_type: 0 00010000:00010000:0.0:1776178092.526019:0:20754:0:(ldlm_lockd.c:953:ldlm_server_blocking_ast()) ### server preparing blocking AST ns: filter-lustre-OST0001_UUID lock: 0000000000a3384d/0xa40083ab9bf349ff lrc: 3/0,0 mode: PR/PR res: [0x2c0000400:0x284:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x50000000010020 nid: 192.168.204.3@tcp remote: 0x771f68aa1eb23319 expref: 26 pid: 8228 timeout: 0 lvb_type: 1 00010000:00010000:0.0:1776178092.526032:0:20754:0:(ldlm_lockd.c:489:ldlm_add_waiting_lock()) ### adding to wait list(timeout: 100, AT: on) ns: filter-lustre-OST0001_UUID lock: 0000000000a3384d/0xa40083ab9bf349ff lrc: 4/0,0 mode: PR/PR res: [0x2c0000400:0x284:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x70000400010020 nid: 192.168.204.3@tcp remote: 0x771f68aa1eb23319 expref: 26 pid: 8228 timeout: 899 lvb_type: 1 00000100:00100000:0.0:1776178092.526046:0:20754:0:(client.c:1733:ptlrpc_send_new_req()) Sending RPC req@00000000f141ac55 pname:cluuid:pid:xid:nid:opc:job ll_ost00_005:lustre-OST0001_UUID:20754:1862456998014336:192.168.204.3@tcp:104: 00000100:00100000:0.0:1776178092.526130:0:20754:0:(client.c:2500:ptlrpc_set_wait()) set 0000000006bd1f0a going to sleep for 6 seconds 00000100:00100000:0.0:1776178092.529391:0:3353:0:(client.c:1733:ptlrpc_send_new_req()) Sending RPC req@00000000cc80b69e pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_00:lustre-MDT0001-mdtlov_UUID:3353:1862456998014592:0@lo:6:osp-syn-0-1.0 00000100:00100000:0.0:1776178092.529428:0:3353:0:(events.c:361:request_in_callback()) peer: 12345-0@lo (source: 12345-0@lo) 00000100:00100000:0.0:1776178092.529454:0:3353:0:(ptlrpcd.c:415:ptlrpcd_check()) transfer 1 async RPCs [1->0] 00000100:00100000:0.0:1776178092.529459:0:3353:0:(client.c:1733:ptlrpc_send_new_req()) Sending RPC req@00000000cb1b73cb pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_00:lustre-MDT0001-mdtlov_UUID:3353:1862456998014656:0@lo:6:osp-syn-0-1.0 00000100:00100000:0.0:1776178092.529478:0:3353:0:(events.c:361:request_in_callback()) peer: 12345-0@lo (source: 12345-0@lo) 00010000:00010000:0.0:1776178092.534443:0:14440:0:(ldlm_lock.c:216:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0001_UUID lock: 0000000067990ad0/0xa40083ab9bf40879 lrc: 0/0,0 mode: --/PR res: [0x2c0000400:0x407:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x44801400010020 nid: 192.168.204.3@tcp remote: 0x771f68aa1eb27297 expref: 25 pid: 8645 timeout: 899 lvb_type: 1 00000100:00100000:0.0:1776178092.534458:0:14440:0:(client.c:2215:ptlrpc_check_set()) Completed RPC req@000000008c4bfd71 pname:cluuid:pid:xid:nid:opc:job ll_ost00_004:lustre-OST0001_UUID:14440:1862456998014144:192.168.204.3@tcp:104: 00010000:00010000:0.0:1776178092.534476:0:14440:0:(ldlm_request.c:283:ldlm_completion_ast()) ### client-side enqueue returned a blocked locksleeping ns: filter-lustre-OST0001_UUID lock: 0000000030bf6b1e/0xa40083ab9bf52d22 lrc: 3/0,1 mode: PW/PW res: [0x2c0000400:0x407:0x0].0x0 rrc: 2 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x40010080000000 nid: local remote: 0x0 expref: -99 pid: 14440 timeout: 0 lvb_type: 0 00010000:00010000:0.0:1776178092.534484:0:14440:0:(ldlm_request.c:199:ldlm_completion_tail()) ### client-side enqueue: granted ns: filter-lustre-OST0001_UUID lock: 0000000030bf6b1e/0xa40083ab9bf52d22 lrc: 3/0,1 mode: PW/PW res: [0x2c0000400:0x407:0x0].0x0 rrc: 2 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x40010080000000 nid: local remote: 0x0 expref: -99 pid: 14440 timeout: 0 lvb_type: 0 00010000:00010000:0.0:1776178092.534490:0:14440:0:(ldlm_request.c:528:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0001_UUID lock: 0000000030bf6b1e/0xa40083ab9bf52d22 lrc: 3/0,1 mode: PW/PW res: [0x2c0000400:0x407:0x0].0x0 rrc: 2 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x40010080000000 nid: local remote: 0x0 expref: -99 pid: 14440 timeout: 0 lvb_type: 0 00010000:00010000:0.0:1776178092.534500:0:14440:0:(ldlm_lock.c:820:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0001_UUID lock: 0000000030bf6b1e/0xa40083ab9bf52d22 lrc: 3/0,1 mode: PW/PW res: [0x2c0000400:0x407:0x0].0x0 rrc: 2 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x50010080000000 nid: local remote: 0x0 expref: -99 pid: 14440 timeout: 0 lvb_type: 0 00010000:00010000:0.0:1776178092.534507:0:14440:0:(ldlm_lock.c:879:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0001_UUID lock: 0000000030bf6b1e/0xa40083ab9bf52d22 lrc: 2/0,0 mode: PW/PW res: [0x2c0000400:0x407:0x0].0x0 rrc: 2 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x50010480000000 nid: local remote: 0x0 expref: -99 pid: 14440 timeout: 0 lvb_type: 0 00010000:00010000:0.0:1776178092.534529:0:14440:0:(ldlm_lockd.c:2159:__ldlm_bl_to_thread()) added 1/1 locks to regular blp list, 1 blwis in pool 00000100:00100000:0.0:1776178092.534710:0:14440:0:(service.c:2372:ptlrpc_server_handle_request()) Handled RPC req@00000000599b0cf2 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_004:lustre-MDT0001-mdtlov_UUID+29:3354:x1862456998014016:12345-0@lo:6:osp-syn-1-1.0 Request processed in 25466us (25727us total) trans 4294969294 rc 0/0 00000100:00100000:0.0:1776178092.534719:0:14440:0:(nrs_fifo.c:241:nrs_fifo_req_stop()) NRS stop fifo request from 12345-0@lo, seq: 5188 00000100:00100000:0.0:1776178092.534757:0:8228:0:(client.c:2215:ptlrpc_check_set()) Completed RPC req@0000000078c1d864 pname:cluuid:pid:xid:nid:opc:job ll_ost00_001:lustre-OST0000_UUID:8228:1862456998013952:192.168.204.3@tcp:104: 00010000:00010000:0.0:1776178092.534769:0:8228:0:(ldlm_request.c:283:ldlm_completion_ast()) ### client-side enqueue returned a blocked locksleeping ns: filter-lustre-OST0000_UUID lock: 00000000f90c1e39/0xa40083ab9bf52d14 lrc: 3/0,1 mode: --/PW res: [0x280000400:0x384:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x40010080000000 nid: local remote: 0x0 expref: -99 pid: 8228 timeout: 0 lvb_type: 0 00000004:00080000:0.0:1776178092.534804:0:3354:0:(osp_sync.c:562:osp_sync_interpret()) reply req 00000000a2f460e7/2, rc 0, transno 1998 00000100:00100000:0.0:1776178092.534816:0:3354:0:(client.c:2215:ptlrpc_check_set()) Completed RPC req@00000000a2f460e7 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_01:lustre-MDT0001-mdtlov_UUID:3354:1862456998014016:0@lo:6:osp-syn-1-1.0 00000100:00100000:0.0:1776178092.534905:0:20758:0:(client.c:2215:ptlrpc_check_set()) Completed RPC req@000000001f0b1867 pname:cluuid:pid:xid:nid:opc:job ll_ost00_009:lustre-OST0000_UUID:20758:1862456998014080:192.168.204.3@tcp:104: 00010000:00010000:0.0:1776178092.534917:0:20758:0:(ldlm_request.c:283:ldlm_completion_ast()) ### client-side enqueue returned a blocked locksleeping ns: filter-lustre-OST0000_UUID lock: 000000000b903578/0xa40083ab9bf52d1b lrc: 3/0,1 mode: --/PW res: [0x280000400:0x2c7:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x40010080000000 nid: local remote: 0x0 expref: -99 pid: 20758 timeout: 0 lvb_type: 0 00010000:00010000:0.0:1776178092.536136:0:20765:0:(ldlm_lock.c:216:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 000000008ca18537/0xa40083ab9bf34d32 lrc: 0/0,0 mode: --/PR res: [0x280000400:0x2ca:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x44801400010020 nid: 192.168.204.3@tcp remote: 0x771f68aa1eb2342a expref: 30 pid: 8645 timeout: 899 lvb_type: 1 00000100:00100000:0.0:1776178092.536150:0:20765:0:(client.c:2215:ptlrpc_check_set()) Completed RPC req@0000000022c7a99e pname:cluuid:pid:xid:nid:opc:job ll_ost00_016:lustre-OST0000_UUID:20765:1862456998013312:192.168.204.3@tcp:104: 00010000:00010000:0.0:1776178092.536170:0:20765:0:(ldlm_request.c:283:ldlm_completion_ast()) ### client-side enqueue returned a blocked locksleeping ns: filter-lustre-OST0000_UUID lock: 000000001e6d7318/0xa40083ab9bf52cf8 lrc: 3/0,1 mode: PW/PW res: [0x280000400:0x2ca:0x0].0x0 rrc: 2 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x40010080000000 nid: local remote: 0x0 expref: -99 pid: 20765 timeout: 0 lvb_type: 0 00010000:00010000:0.0:1776178092.536177:0:20765:0:(ldlm_request.c:199:ldlm_completion_tail()) ### client-side enqueue: granted ns: filter-lustre-OST0000_UUID lock: 000000001e6d7318/0xa40083ab9bf52cf8 lrc: 3/0,1 mode: PW/PW res: [0x280000400:0x2ca:0x0].0x0 rrc: 2 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x40010080000000 nid: local remote: 0x0 expref: -99 pid: 20765 timeout: 0 lvb_type: 0 00010000:00010000:0.0:1776178092.536182:0:20765:0:(ldlm_request.c:528:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 000000001e6d7318/0xa40083ab9bf52cf8 lrc: 3/0,1 mode: PW/PW res: [0x280000400:0x2ca:0x0].0x0 rrc: 2 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x40010080000000 nid: local remote: 0x0 expref: -99 pid: 20765 timeout: 0 lvb_type: 0 00010000:00010000:0.0:1776178092.536191:0:20765:0:(ldlm_lock.c:820:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0000_UUID lock: 000000001e6d7318/0xa40083ab9bf52cf8 lrc: 3/0,1 mode: PW/PW res: [0x280000400:0x2ca:0x0].0x0 rrc: 2 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x50010080000000 nid: local remote: 0x0 expref: -99 pid: 20765 timeout: 0 lvb_type: 0 00010000:00010000:0.0:1776178092.536198:0:20765:0:(ldlm_lock.c:879:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 000000001e6d7318/0xa40083ab9bf52cf8 lrc: 2/0,0 mode: PW/PW res: [0x280000400:0x2ca:0x0].0x0 rrc: 2 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x50010480000000 nid: local remote: 0x0 expref: -99 pid: 20765 timeout: 0 lvb_type: 0 00010000:00010000:0.0:1776178092.536214:0:20765:0:(ldlm_lockd.c:2159:__ldlm_bl_to_thread()) added 1/1 locks to regular blp list, 1 blwis in pool 00000100:00100000:0.0:1776178092.536440:0:20765:0:(service.c:2372:ptlrpc_server_handle_request()) Handled RPC req@000000006be5e5b9 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_016:lustre-MDT0001-mdtlov_UUID+31:3354:x1862456998013248:12345-0@lo:6:osp-syn-0-1.0 Request processed in 36640us (36727us total) trans 4294969315 rc 0/0 00000100:00100000:0.0:1776178092.536448:0:20765:0:(nrs_fifo.c:241:nrs_fifo_req_stop()) NRS stop fifo request from 12345-0@lo, seq: 5182 00000004:00080000:0.0:1776178092.536488:0:3354:0:(osp_sync.c:562:osp_sync_interpret()) reply req 00000000e0d60cbc/2, rc 0, transno 2019 00000100:00100000:0.0:1776178092.536500:0:3354:0:(client.c:2215:ptlrpc_check_set()) Completed RPC req@00000000e0d60cbc pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_01:lustre-MDT0001-mdtlov_UUID:3354:1862456998013248:0@lo:6:osp-syn-0-1.0 00000100:00100000:0.0:1776178092.536572:0:3354:0:(ptlrpcd.c:415:ptlrpcd_check()) transfer 1 async RPCs [0->1] 00000100:00100000:0.0:1776178092.536587:0:3354:0:(client.c:1733:ptlrpc_send_new_req()) Sending RPC req@000000006466ee18 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_01:lustre-MDT0001-mdtlov_UUID:3354:1862456998014976:0@lo:6:osp-syn-0-1.0 00000100:00100000:0.0:1776178092.536626:0:3354:0:(events.c:361:request_in_callback()) peer: 12345-0@lo (source: 12345-0@lo) 00000100:00100000:0.0:1776178092.536654:0:14440:0:(service.c:2152:ptlrpc_server_handle_req_in()) got req x1862456998014976 00000100:00100000:0.0:1776178092.536682:0:14440:0:(nrs_fifo.c:179:nrs_fifo_req_get()) NRS start fifo request from 12345-0@lo, seq: 5194 00000100:00100000:0.0:1776178092.536685:0:14440:0:(service.c:2324:ptlrpc_server_handle_request()) Handling RPC req@0000000043a010c5 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_004:lustre-MDT0001-mdtlov_UUID+31:3354:x1862456998014976:12345-0@lo:6:osp-syn-0-1.0 00002000:00080000:0.0:1776178092.536710:0:14440:0:(ofd_dev.c:1822:ofd_destroy_hdl()) lustre-OST0000: Destroy object 0x280000400:971 count 1 00010000:00010000:0.0:1776178092.536735:0:14440:0:(ldlm_lock.c:766:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0000_UUID lock: 0000000095e53696/0xa40083ab9bf52d4c lrc: 3/0,1 mode: --/PW res: [0x280000400:0x3cb:0x0].0x0 rrc: 3 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 14440 timeout: 0 lvb_type: 0 00010000:00010000:0.0:1776178092.536750:0:14440:0:(ldlm_lock.c:675:ldlm_add_bl_work_item()) ### lock incompatible; sending blocking AST. ns: filter-lustre-OST0000_UUID lock: 00000000156434a9/0xa40083ab9bf3cbb0 lrc: 2/0,0 mode: PR/PR res: [0x280000400:0x3cb:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x40000000000000 nid: 192.168.204.3@tcp remote: 0x771f68aa1eb25e54 expref: 29 pid: 8227 timeout: 0 lvb_type: 1 00010000:00010000:0.0:1776178092.536757:0:14440:0:(ldlm_extent.c:318:ldlm_check_contention()) contended locks = 0 00010000:00010000:0.0:1776178092.536758:0:14440:0:(ldlm_extent.c:318:ldlm_check_contention()) contended locks = 0 00010000:00010000:0.0:1776178092.536760:0:14440:0:(ldlm_resource.c:1650:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 0000000095e53696/0xa40083ab9bf52d4c lrc: 4/0,1 mode: --/PW res: [0x280000400:0x3cb:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x50010080000000 nid: local remote: 0x0 expref: -99 pid: 14440 timeout: 0 lvb_type: 0 00010000:00010000:0.0:1776178092.536780:0:14440:0:(ldlm_lockd.c:953:ldlm_server_blocking_ast()) ### server preparing blocking AST ns: filter-lustre-OST0000_UUID lock: 00000000156434a9/0xa40083ab9bf3cbb0 lrc: 3/0,0 mode: PR/PR res: [0x280000400:0x3cb:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x50000000010020 nid: 192.168.204.3@tcp remote: 0x771f68aa1eb25e54 expref: 29 pid: 8227 timeout: 0 lvb_type: 1 00010000:00010000:0.0:1776178092.536790:0:14440:0:(ldlm_lockd.c:489:ldlm_add_waiting_lock()) ### adding to wait list(timeout: 100, AT: on) ns: filter-lustre-OST0000_UUID lock: 00000000156434a9/0xa40083ab9bf3cbb0 lrc: 4/0,0 mode: PR/PR res: [0x280000400:0x3cb:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x70000400010020 nid: 192.168.204.3@tcp remote: 0x771f68aa1eb25e54 expref: 29 pid: 8227 timeout: 899 lvb_type: 1 00000100:00100000:0.0:1776178092.536799:0:14440:0:(client.c:1733:ptlrpc_send_new_req()) Sending RPC req@00000000dff3309c pname:cluuid:pid:xid:nid:opc:job ll_ost00_004:lustre-OST0000_UUID:14440:1862456998015040:192.168.204.3@tcp:104: 00000100:00100000:0.0:1776178092.536842:0:14440:0:(client.c:2500:ptlrpc_set_wait()) set 00000000835d6110 going to sleep for 6 seconds 00000100:00100000:0.0:1776178092.539851:0:3347:0:(events.c:361:request_in_callback()) peer: 12345-192.168.204.3@tcp (source: 12345-192.168.204.3@tcp) 00000100:00100000:0.0:1776178092.539943:0:3347:0:(events.c:361:request_in_callback()) peer: 12345-192.168.204.3@tcp (source: 12345-192.168.204.3@tcp) 00000100:00100000:0.0:1776178092.540521:0:6243:0:(service.c:2152:ptlrpc_server_handle_req_in()) got req x1862456979072192 00010000:00010000:0.0:1776178092.540574:0:6243:0:(ldlm_lockd.c:2638:ldlm_cancel_hpreq_check()) ### hpreq cancel/convert lock ns: filter-lustre-OST0001_UUID lock: 000000000b4a063d/0xa40083ab9bf409f3 lrc: 4/0,0 mode: PR/PR res: [0x2c0000400:0x40a:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x60000400010020 nid: 192.168.204.3@tcp remote: 0x771f68aa1eb27315 expref: 25 pid: 8645 timeout: 899 lvb_type: 1 00000100:00100000:0.0:1776178092.540612:0:6243:0:(service.c:2152:ptlrpc_server_handle_req_in()) got req x1862456979072320 00010000:00010000:0.0:1776178092.540625:0:6243:0:(ldlm_lockd.c:2638:ldlm_cancel_hpreq_check()) ### hpreq cancel/convert lock ns: filter-lustre-OST0000_UUID lock: 000000002d78274e/0xa40083ab9bf3a8be lrc: 4/0,0 mode: PR/PR res: [0x280000400:0x384:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x60000400010020 nid: 192.168.204.3@tcp remote: 0x771f68aa1eb252ae expref: 30 pid: 8645 timeout: 899 lvb_type: 1 00000100:00100000:0.0:1776178092.540646:0:6243:0:(nrs_fifo.c:179:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.204.3@tcp, seq: 2029 00000100:00100000:0.0:1776178092.540655:0:6243:0:(service.c:2324:ptlrpc_server_handle_request()) Handling RPC req@0000000053234520 pname:cluuid+ref:pid:xid:nid:opc:job ldlm_cn00_002:b50963bd-8eba-4432-9a54-8888d00e3fa1+25:2270:x1862456979072192:12345-192.168.204.3@tcp:103:ldlm_bl_02.0 00010000:00010000:0.0:1776178092.540673:0:6243:0:(ldlm_lockd.c:1720:ldlm_request_cancel()) ### server-side cancel handler START: 1 locks, starting at 0 00010000:00010000:0.0:1776178092.540678:0:6243:0:(ldlm_lockd.c:1771:ldlm_request_cancel()) ### server cancels blocked lock after 0s ns: filter-lustre-OST0001_UUID lock: 000000000b4a063d/0xa40083ab9bf409f3 lrc: 4/0,0 mode: PR/PR res: [0x2c0000400:0x40a:0x0].0x0 rrc: 4 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x60000400010020 nid: 192.168.204.3@tcp remote: 0x771f68aa1eb27315 expref: 25 pid: 8645 timeout: 899 lvb_type: 1 00010000:00010000:0.0:1776178092.540689:0:6243:0:(ldlm_lockd.c:562:ldlm_del_waiting_lock()) ### removed ns: filter-lustre-OST0001_UUID lock: 000000000b4a063d/0xa40083ab9bf409f3 lrc: 3/0,0 mode: PR/PR res: [0x2c0000400:0x40a:0x0].0x0 rrc: 4 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x50000400010020 nid: 192.168.204.3@tcp remote: 0x771f68aa1eb27315 expref: 25 pid: 8645 timeout: 899 lvb_type: 1 00010000:00010000:0.0:1776178092.540718:0:6243:0:(ldlm_lock.c:216:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0001_UUID lock: 000000000b4a063d/0xa40083ab9bf409f3 lrc: 0/0,0 mode: --/PR res: [0x2c0000400:0x40a:0x0].0x0 rrc: 4 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x44801400010020 nid: 192.168.204.3@tcp remote: 0x771f68aa1eb27315 expref: 25 pid: 8645 timeout: 899 lvb_type: 1 00010000:00010000:0.0:1776178092.540737:0:6243:0:(ldlm_extent.c:318:ldlm_check_contention()) contended locks = 0 00010000:00010000:0.0:1776178092.540739:0:6243:0:(ldlm_extent.c:318:ldlm_check_contention()) contended locks = 0 00010000:00010000:0.0:1776178092.540744:0:6243:0:(ldlm_lock.c:704:ldlm_add_cp_work_item()) ### lock granted; sending completion AST. ns: filter-lustre-OST0001_UUID lock: 00000000499bab84/0xa40083ab9bf52d06 lrc: 3/0,1 mode: PW/PW res: [0x2c0000400:0x40a:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x40090080000000 nid: local remote: 0x0 expref: -99 pid: 20762 timeout: 0 lvb_type: 0 00010000:00010000:0.0:1776178092.540753:0:6243:0:(ldlm_resource.c:1650:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0001_UUID lock: 00000000499bab84/0xa40083ab9bf52d06 lrc: 4/0,1 mode: PW/PW res: [0x2c0000400:0x40a:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x40090080000000 nid: local remote: 0x0 expref: -99 pid: 20762 timeout: 0 lvb_type: 0 00010000:00010000:0.0:1776178092.540790:0:6243:0:(ldlm_lockd.c:1782:ldlm_request_cancel()) ### server-side cancel handler END 00000100:00100000:0.0:1776178092.540851:0:6243:0:(service.c:2372:ptlrpc_server_handle_request()) Handled RPC req@0000000053234520 pname:cluuid+ref:pid:xid:nid:opc:job ldlm_cn00_002:b50963bd-8eba-4432-9a54-8888d00e3fa1+24:2270:x1862456979072192:12345-192.168.204.3@tcp:103:ldlm_bl_02.0 Request processed in 196us (1012us total) trans 0 rc 0/0 00000100:00100000:0.0:1776178092.540862:0:6243:0:(nrs_fifo.c:241:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.204.3@tcp, seq: 2029 00000100:00100000:0.0:1776178092.540882:0:6243:0:(nrs_fifo.c:179:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.204.3@tcp, seq: 2030 00000100:00100000:0.0:1776178092.540887:0:6243:0:(service.c:2324:ptlrpc_server_handle_request()) Handling RPC req@0000000018b5e3c6 pname:cluuid+ref:pid:xid:nid:opc:job ldlm_cn00_002:b50963bd-8eba-4432-9a54-8888d00e3fa1+30:2270:x1862456979072320:12345-192.168.204.3@tcp:103:ldlm_bl_02.0 00010000:00010000:0.0:1776178092.540896:0:6243:0:(ldlm_lockd.c:1720:ldlm_request_cancel()) ### server-side cancel handler START: 1 locks, starting at 0 00010000:00010000:0.0:1776178092.540901:0:6243:0:(ldlm_lockd.c:1771:ldlm_request_cancel()) ### server cancels blocked lock after 0s ns: filter-lustre-OST0000_UUID lock: 000000002d78274e/0xa40083ab9bf3a8be lrc: 4/0,0 mode: PR/PR res: [0x280000400:0x384:0x0].0x0 rrc: 4 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x60000400010020 nid: 192.168.204.3@tcp remote: 0x771f68aa1eb252ae expref: 30 pid: 8645 timeout: 899 lvb_type: 1 00010000:00010000:0.0:1776178092.540909:0:6243:0:(ldlm_lockd.c:562:ldlm_del_waiting_lock()) ### removed ns: filter-lustre-OST0000_UUID lock: 000000002d78274e/0xa40083ab9bf3a8be lrc: 3/0,0 mode: PR/PR res: [0x280000400:0x384:0x0].0x0 rrc: 4 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x50000400010020 nid: 192.168.204.3@tcp remote: 0x771f68aa1eb252ae expref: 30 pid: 8645 timeout: 899 lvb_type: 1 00010000:00010000:0.0:1776178092.540927:0:6243:0:(ldlm_lock.c:216:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 000000002d78274e/0xa40083ab9bf3a8be lrc: 0/0,0 mode: --/PR res: [0x280000400:0x384:0x0].0x0 rrc: 4 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x44801400010020 nid: 192.168.204.3@tcp remote: 0x771f68aa1eb252ae expref: 30 pid: 8645 timeout: 899 lvb_type: 1 00010000:00010000:0.0:1776178092.540941:0:6243:0:(ldlm_extent.c:318:ldlm_check_contention()) contended locks = 0 00010000:00010000:0.0:1776178092.540943:0:6243:0:(ldlm_extent.c:318:ldlm_check_contention()) contended locks = 0 00010000:00010000:0.0:1776178092.540945:0:6243:0:(ldlm_lock.c:704:ldlm_add_cp_work_item()) ### lock granted; sending completion AST. ns: filter-lustre-OST0000_UUID lock: 00000000f90c1e39/0xa40083ab9bf52d14 lrc: 3/0,1 mode: PW/PW res: [0x280000400:0x384:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x40090080000000 nid: local remote: 0x0 expref: -99 pid: 8228 timeout: 0 lvb_type: 0 00010000:00010000:0.0:1776178092.540951:0:6243:0:(ldlm_resource.c:1650:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 00000000f90c1e39/0xa40083ab9bf52d14 lrc: 4/0,1 mode: PW/PW res: [0x280000400:0x384:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x40090080000000 nid: local remote: 0x0 expref: -99 pid: 8228 timeout: 0 lvb_type: 0 00010000:00010000:0.0:1776178092.540977:0:6243:0:(ldlm_lockd.c:1782:ldlm_request_cancel()) ### server-side cancel handler END 00000100:00100000:0.0:1776178092.540994:0:6243:0:(service.c:2372:ptlrpc_server_handle_request()) Handled RPC req@0000000018b5e3c6 pname:cluuid+ref:pid:xid:nid:opc:job ldlm_cn00_002:b50963bd-8eba-4432-9a54-8888d00e3fa1+29:2270:x1862456979072320:12345-192.168.204.3@tcp:103:ldlm_bl_02.0 Request processed in 106us (1052us total) trans 0 rc 0/0 00000100:00100000:0.0:1776178092.541000:0:6243:0:(nrs_fifo.c:241:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.204.3@tcp, seq: 2030 00010000:00010000:0.0:1776178092.541024:0:8228:0:(ldlm_request.c:199:ldlm_completion_tail()) ### client-side enqueue: granted ns: filter-lustre-OST0000_UUID lock: 00000000f90c1e39/0xa40083ab9bf52d14 lrc: 3/0,1 mode: PW/PW res: [0x280000400:0x384:0x0].0x0 rrc: 2 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x40010080000000 nid: local remote: 0x0 expref: -99 pid: 8228 timeout: 0 lvb_type: 0 00010000:00010000:0.0:1776178092.541033:0:8228:0:(ldlm_request.c:528:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 00000000f90c1e39/0xa40083ab9bf52d14 lrc: 3/0,1 mode: PW/PW res: [0x280000400:0x384:0x0].0x0 rrc: 2 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x40010080000000 nid: local remote: 0x0 expref: -99 pid: 8228 timeout: 0 lvb_type: 0 00010000:00010000:0.0:1776178092.541044:0:8228:0:(ldlm_lock.c:820:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0000_UUID lock: 00000000f90c1e39/0xa40083ab9bf52d14 lrc: 3/0,1 mode: PW/PW res: [0x280000400:0x384:0x0].0x0 rrc: 2 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x50010080000000 nid: local remote: 0x0 expref: -99 pid: 8228 timeout: 0 lvb_type: 0 00010000:00010000:0.0:1776178092.541051:0:8228:0:(ldlm_lock.c:879:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 00000000f90c1e39/0xa40083ab9bf52d14 lrc: 2/0,0 mode: PW/PW res: [0x280000400:0x384:0x0].0x0 rrc: 2 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x50010480000000 nid: local remote: 0x0 expref: -99 pid: 8228 timeout: 0 lvb_type: 0 00010000:00010000:0.0:1776178092.541090:0:8228:0:(ldlm_lockd.c:2159:__ldlm_bl_to_thread()) added 1/2 locks to regular blp list, 2 blwis in pool 00000100:00100000:0.0:1776178092.541436:0:8228:0:(service.c:2372:ptlrpc_server_handle_request()) Handled RPC req@00000000efb13deb pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_001:lustre-MDT0001-mdtlov_UUID+32:3354:x1862456998013760:12345-0@lo:6:osp-syn-0-1.0 Request processed in 33044us (34312us total) trans 4294969316 rc 0/0 00000100:00100000:0.0:1776178092.541445:0:8228:0:(nrs_fifo.c:241:nrs_fifo_req_stop()) NRS stop fifo request from 12345-0@lo, seq: 5186 00000004:00080000:0.0:1776178092.541507:0:3354:0:(osp_sync.c:562:osp_sync_interpret()) reply req 0000000092028508/2, rc 0, transno 2020 00000100:00100000:0.0:1776178092.541521:0:3354:0:(client.c:2215:ptlrpc_check_set()) Completed RPC req@0000000092028508 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_01:lustre-MDT0001-mdtlov_UUID:3354:1862456998013760:0@lo:6:osp-syn-0-1.0 00000100:00100000:0.0:1776178092.541617:0:3354:0:(client.c:1733:ptlrpc_send_new_req()) Sending RPC req@00000000a62daf9a pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_01:lustre-MDT0001-mdtlov_UUID:3354:1862456998015104:0@lo:6:osp-syn-0-1.0 00000100:00100000:0.0:1776178092.541660:0:3354:0:(events.c:361:request_in_callback()) peer: 12345-0@lo (source: 12345-0@lo) 00000100:00100000:0.0:1776178092.541693:0:8228:0:(service.c:2152:ptlrpc_server_handle_req_in()) got req x1862456998015104 00000100:00100000:0.0:1776178092.541717:0:8228:0:(nrs_fifo.c:179:nrs_fifo_req_get()) NRS start fifo request from 12345-0@lo, seq: 5195 00000100:00100000:0.0:1776178092.541721:0:8228:0:(service.c:2324:ptlrpc_server_handle_request()) Handling RPC req@00000000af523f53 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_001:lustre-MDT0001-mdtlov_UUID+32:3354:x1862456998015104:12345-0@lo:6:osp-syn-0-1.0 00002000:00080000:0.0:1776178092.541751:0:8228:0:(ofd_dev.c:1822:ofd_destroy_hdl()) lustre-OST0000: Destroy object 0x280000400:723 count 1 00010000:00010000:0.0:1776178092.541779:0:8228:0:(ldlm_lock.c:766:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0000_UUID lock: 0000000015ae4ee6/0xa40083ab9bf52d53 lrc: 3/0,1 mode: --/PW res: [0x280000400:0x2d3:0x0].0x0 rrc: 3 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 8228 timeout: 0 lvb_type: 0 00010000:00010000:0.0:1776178092.541794:0:8228:0:(ldlm_lock.c:675:ldlm_add_bl_work_item()) ### lock incompatible; sending blocking AST. ns: filter-lustre-OST0000_UUID lock: 00000000eaa9fbf1/0xa40083ab9bf351a0 lrc: 2/0,0 mode: PR/PR res: [0x280000400:0x2d3:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x40000000000000 nid: 192.168.204.3@tcp remote: 0x771f68aa1eb235a4 expref: 28 pid: 8645 timeout: 0 lvb_type: 1 00010000:00010000:0.0:1776178092.541800:0:8228:0:(ldlm_extent.c:318:ldlm_check_contention()) contended locks = 0 00010000:00010000:0.0:1776178092.541803:0:8228:0:(ldlm_extent.c:318:ldlm_check_contention()) contended locks = 0 00010000:00010000:0.0:1776178092.541805:0:8228:0:(ldlm_resource.c:1650:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 0000000015ae4ee6/0xa40083ab9bf52d53 lrc: 4/0,1 mode: --/PW res: [0x280000400:0x2d3:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x50010080000000 nid: local remote: 0x0 expref: -99 pid: 8228 timeout: 0 lvb_type: 0 00010000:00010000:0.0:1776178092.541842:0:8228:0:(ldlm_lockd.c:953:ldlm_server_blocking_ast()) ### server preparing blocking AST ns: filter-lustre-OST0000_UUID lock: 00000000eaa9fbf1/0xa40083ab9bf351a0 lrc: 3/0,0 mode: PR/PR res: [0x280000400:0x2d3:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x50000000010020 nid: 192.168.204.3@tcp remote: 0x771f68aa1eb235a4 expref: 28 pid: 8645 timeout: 0 lvb_type: 1 00010000:00010000:0.0:1776178092.541856:0:8228:0:(ldlm_lockd.c:489:ldlm_add_waiting_lock()) ### adding to wait list(timeout: 100, AT: on) ns: filter-lustre-OST0000_UUID lock: 00000000eaa9fbf1/0xa40083ab9bf351a0 lrc: 4/0,0 mode: PR/PR res: [0x280000400:0x2d3:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x70000400010020 nid: 192.168.204.3@tcp remote: 0x771f68aa1eb235a4 expref: 28 pid: 8645 timeout: 899 lvb_type: 1 00000100:00100000:0.0:1776178092.541866:0:8228:0:(client.c:1733:ptlrpc_send_new_req()) Sending RPC req@000000009016e75e pname:cluuid:pid:xid:nid:opc:job ll_ost00_001:lustre-OST0000_UUID:8228:1862456998015168:192.168.204.3@tcp:104: 00000100:00100000:0.0:1776178092.541906:0:8228:0:(client.c:2500:ptlrpc_set_wait()) set 000000004d9ad1c9 going to sleep for 6 seconds 00010000:00010000:0.0:1776178092.543774:0:6245:0:(ldlm_lockd.c:2789:ldlm_bl_get_work()) Got 1 locks of 0 total in blp. (0 blwis in pool) 00010000:00010000:0.0:1776178092.543780:0:6245:0:(ldlm_lockd.c:1879:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 000000007ec324d4/0xa40083ab9bf52cf1 lrc: 2/0,0 mode: PW/PW res: [0x280000400:0x467:0x0].0x0 rrc: 2 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x40010480000000 nid: local remote: 0x0 expref: -99 pid: 20755 timeout: 0 lvb_type: 0 00010000:00010000:0.0:1776178092.543790:0:6245:0:(ldlm_lockd.c:1893:ldlm_handle_bl_callback()) Lock 000000007ec324d4 already unused, calling callback (0000000063c97258) 00010000:00010000:0.0:1776178092.543792:0:6245:0:(ldlm_request.c:364:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 000000007ec324d4/0xa40083ab9bf52cf1 lrc: 2/0,0 mode: PW/PW res: [0x280000400:0x467:0x0].0x0 rrc: 2 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x40010480000000 nid: local remote: 0x0 expref: -99 pid: 20755 timeout: 0 lvb_type: 0 00010000:00010000:0.0:1776178092.543800:0:6245:0:(ldlm_request.c:1257:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 000000007ec324d4/0xa40083ab9bf52cf1 lrc: 3/0,0 mode: PW/PW res: [0x280000400:0x467:0x0].0x0 rrc: 2 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x40018480000000 nid: local remote: 0x0 expref: -99 pid: 20755 timeout: 0 lvb_type: 0 00010000:00010000:0.0:1776178092.543818:0:6245:0:(ldlm_lockd.c:1903:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 000000007ec324d4/0xa40083ab9bf52cf1 lrc: 1/0,0 mode: --/PW res: [0x280000400:0x467:0x0].0x0 rrc: 2 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x44819480000000 nid: local remote: 0x0 expref: -99 pid: 20755 timeout: 0 lvb_type: 0 00010000:00010000:0.0:1776178092.543833:0:6245:0:(ldlm_lock.c:216:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 000000007ec324d4/0xa40083ab9bf52cf1 lrc: 0/0,0 mode: --/PW res: [0x280000400:0x467:0x0].0x0 rrc: 2 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x44819480000000 nid: local remote: 0x0 expref: -99 pid: 20755 timeout: 0 lvb_type: 0 00010000:00010000:0.0:1776178092.543857:0:6245:0:(ldlm_lockd.c:2792:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00010000:00010000:0.0:1776178092.543858:0:6245:0:(ldlm_lockd.c:2792:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00010000:00010000:0.0:1776178092.543860:0:6245:0:(ldlm_lockd.c:2792:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000004:00080000:0.0:1776178092.543882:0:3354:0:(osp_sync.c:562:osp_sync_interpret()) reply req 00000000b7aa289e/2, rc 0, transno 2021 00000100:00100000:0.0:1776178092.543891:0:3354:0:(client.c:2215:ptlrpc_check_set()) Completed RPC req@00000000b7aa289e pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_01:lustre-MDT0001-mdtlov_UUID:3354:1862456998013120:0@lo:6:osp-syn-0-1.0 00010000:00010000:0.0:1776178092.547528:0:9224:0:(ldlm_lockd.c:2792:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00010000:00010000:0.0:1776178092.551907:0:6245:0:(ldlm_lockd.c:2789:ldlm_bl_get_work()) Got 1 locks of 1 total in blp. (1 blwis in pool) 00010000:00010000:0.0:1776178092.551919:0:6245:0:(ldlm_lockd.c:1879:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0001_UUID lock: 00000000f51fd0d7/0xa40083ab9bf52d30 lrc: 2/0,0 mode: PW/PW res: [0x2c0000400:0x284:0x0].0x0 rrc: 2 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x40010480000000 nid: local remote: 0x0 expref: -99 pid: 20754 timeout: 0 lvb_type: 0 00010000:00010000:0.0:1776178092.551933:0:6245:0:(ldlm_lockd.c:1893:ldlm_handle_bl_callback()) Lock 00000000f51fd0d7 already unused, calling callback (0000000063c97258) 00010000:00010000:0.0:1776178092.551938:0:6245:0:(ldlm_request.c:364:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0001_UUID lock: 00000000f51fd0d7/0xa40083ab9bf52d30 lrc: 2/0,0 mode: PW/PW res: [0x2c0000400:0x284:0x0].0x0 rrc: 2 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x40010480000000 nid: local remote: 0x0 expref: -99 pid: 20754 timeout: 0 lvb_type: 0 00010000:00010000:0.0:1776178092.551949:0:6245:0:(ldlm_request.c:1257:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0001_UUID lock: 00000000f51fd0d7/0xa40083ab9bf52d30 lrc: 3/0,0 mode: PW/PW res: [0x2c0000400:0x284:0x0].0x0 rrc: 2 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x40018480000000 nid: local remote: 0x0 expref: -99 pid: 20754 timeout: 0 lvb_type: 0 00010000:00010000:0.0:1776178092.551982:0:6245:0:(ldlm_lockd.c:1903:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0001_UUID lock: 00000000f51fd0d7/0xa40083ab9bf52d30 lrc: 1/0,0 mode: --/PW res: [0x2c0000400:0x284:0x0].0x0 rrc: 2 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x44819480000000 nid: local remote: 0x0 expref: -99 pid: 20754 timeout: 0 lvb_type: 0 00010000:00010000:0.0:1776178092.551989:0:6245:0:(ldlm_lock.c:216:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0001_UUID lock: 00000000f51fd0d7/0xa40083ab9bf52d30 lrc: 0/0,0 mode: --/PW res: [0x2c0000400:0x284:0x0].0x0 rrc: 2 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x44819480000000 nid: local remote: 0x0 expref: -99 pid: 20754 timeout: 0 lvb_type: 0 00010000:00010000:0.0:1776178092.552018:0:6245:0:(ldlm_lockd.c:2789:ldlm_bl_get_work()) Got 1 locks of 0 total in blp. (0 blwis in pool) 00010000:00010000:0.0:1776178092.552022:0:6245:0:(ldlm_lockd.c:1879:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0001_UUID lock: 0000000039ef0a83/0xa40083ab9bf52d37 lrc: 2/0,0 mode: PW/PW res: [0x2c0000400:0x380:0x0].0x0 rrc: 2 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x40010480000000 nid: local remote: 0x0 expref: -99 pid: 20764 timeout: 0 lvb_type: 0 00010000:00010000:0.0:1776178092.552033:0:6245:0:(ldlm_lockd.c:1893:ldlm_handle_bl_callback()) Lock 0000000039ef0a83 already unused, calling callback (0000000063c97258) 00010000:00010000:0.0:1776178092.552035:0:6245:0:(ldlm_request.c:364:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0001_UUID lock: 0000000039ef0a83/0xa40083ab9bf52d37 lrc: 2/0,0 mode: PW/PW res: [0x2c0000400:0x380:0x0].0x0 rrc: 2 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x40010480000000 nid: local remote: 0x0 expref: -99 pid: 20764 timeout: 0 lvb_type: 0 00010000:00010000:0.0:1776178092.552041:0:6245:0:(ldlm_request.c:1257:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0001_UUID lock: 0000000039ef0a83/0xa40083ab9bf52d37 lrc: 3/0,0 mode: PW/PW res: [0x2c0000400:0x380:0x0].0x0 rrc: 2 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x40018480000000 nid: local remote: 0x0 expref: -99 pid: 20764 timeout: 0 lvb_type: 0 00010000:00010000:0.0:1776178092.552053:0:6245:0:(ldlm_lockd.c:1903:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0001_UUID lock: 0000000039ef0a83/0xa40083ab9bf52d37 lrc: 1/0,0 mode: --/PW res: [0x2c0000400:0x380:0x0].0x0 rrc: 2 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x44819480000000 nid: local remote: 0x0 expref: -99 pid: 20764 timeout: 0 lvb_type: 0 00010000:00010000:0.0:1776178092.552074:0:6245:0:(ldlm_lock.c:216:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0001_UUID lock: 0000000039ef0a83/0xa40083ab9bf52d37 lrc: 0/0,0 mode: --/PW res: [0x2c0000400:0x380:0x0].0x0 rrc: 2 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x44819480000000 nid: local remote: 0x0 expref: -99 pid: 20764 timeout: 0 lvb_type: 0 00010000:00010000:0.0:1776178092.552089:0:6245:0:(ldlm_lockd.c:2792:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00010000:00010000:0.0:1776178092.552091:0:6245:0:(ldlm_lockd.c:2792:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00010000:00010000:0.0:1776178092.552092:0:6245:0:(ldlm_lockd.c:2792:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000004:00080000:0.0:1776178092.552155:0:3354:0:(osp_sync.c:562:osp_sync_interpret()) reply req 000000009126d8c6/2, rc 0, transno 2000 00000100:00100000:0.0:1776178092.552163:0:3354:0:(client.c:2215:ptlrpc_check_set()) Completed RPC req@000000009126d8c6 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_01:lustre-MDT0001-mdtlov_UUID:3354:1862456998014272:0@lo:6:osp-syn-1-1.0 00010000:00010000:0.0:1776178092.552194:0:20758:0:(ldlm_request.c:199:ldlm_completion_tail()) ### client-side enqueue: granted ns: filter-lustre-OST0000_UUID lock: 000000000b903578/0xa40083ab9bf52d1b lrc: 3/0,1 mode: PW/PW res: [0x280000400:0x2c7:0x0].0x0 rrc: 2 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x40010080000000 nid: local remote: 0x0 expref: -99 pid: 20758 timeout: 0 lvb_type: 0 00010000:00010000:0.0:1776178092.552203:0:20758:0:(ldlm_request.c:528:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 000000000b903578/0xa40083ab9bf52d1b lrc: 3/0,1 mode: PW/PW res: [0x280000400:0x2c7:0x0].0x0 rrc: 2 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x40010080000000 nid: local remote: 0x0 expref: -99 pid: 20758 timeout: 0 lvb_type: 0 00010000:00010000:0.0:1776178092.552212:0:20758:0:(ldlm_lock.c:820:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0000_UUID lock: 000000000b903578/0xa40083ab9bf52d1b lrc: 3/0,1 mode: PW/PW res: [0x280000400:0x2c7:0x0].0x0 rrc: 2 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x50010080000000 nid: local remote: 0x0 expref: -99 pid: 20758 timeout: 0 lvb_type: 0 00010000:00010000:0.0:1776178092.552224:0:20758:0:(ldlm_lock.c:879:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 000000000b903578/0xa40083ab9bf52d1b lrc: 2/0,0 mode: PW/PW res: [0x280000400:0x2c7:0x0].0x0 rrc: 2 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x50010480000000 nid: local remote: 0x0 expref: -99 pid: 20758 timeout: 0 lvb_type: 0 00010000:00010000:0.0:1776178092.552250:0:20758:0:(ldlm_lockd.c:2159:__ldlm_bl_to_thread()) added 1/1 locks to regular blp list, 1 blwis in pool 00000100:00100000:0.0:1776178092.552742:0:20758:0:(service.c:2372:ptlrpc_server_handle_request()) Handled RPC req@0000000015095c30 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_009:lustre-MDT0001-mdtlov_UUID+35:3355:x1862456998013824:12345-0@lo:6:osp-syn-0-1.0 Request processed in 43658us (44210us total) trans 4294969318 rc 0/0 00000100:00100000:0.0:1776178092.552752:0:20758:0:(nrs_fifo.c:241:nrs_fifo_req_stop()) NRS stop fifo request from 12345-0@lo, seq: 5187 00010000:00010000:0.0:1776178092.552782:0:9224:0:(ldlm_lockd.c:2789:ldlm_bl_get_work()) Got 1 locks of 0 total in blp. (0 blwis in pool) 00010000:00010000:0.0:1776178092.552786:0:9224:0:(ldlm_lockd.c:1879:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 000000000b903578/0xa40083ab9bf52d1b lrc: 2/0,0 mode: PW/PW res: [0x280000400:0x2c7:0x0].0x0 rrc: 2 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x40010480000000 nid: local remote: 0x0 expref: -99 pid: 20758 timeout: 0 lvb_type: 0 00010000:00010000:0.0:1776178092.552793:0:9224:0:(ldlm_lockd.c:1893:ldlm_handle_bl_callback()) Lock 000000000b903578 already unused, calling callback (0000000063c97258) 00010000:00010000:0.0:1776178092.552796:0:9224:0:(ldlm_request.c:364:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 000000000b903578/0xa40083ab9bf52d1b lrc: 2/0,0 mode: PW/PW res: [0x280000400:0x2c7:0x0].0x0 rrc: 2 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x40010480000000 nid: local remote: 0x0 expref: -99 pid: 20758 timeout: 0 lvb_type: 0 00010000:00010000:0.0:1776178092.552802:0:9224:0:(ldlm_request.c:1257:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 000000000b903578/0xa40083ab9bf52d1b lrc: 3/0,0 mode: PW/PW res: [0x280000400:0x2c7:0x0].0x0 rrc: 2 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x40018480000000 nid: local remote: 0x0 expref: -99 pid: 20758 timeout: 0 lvb_type: 0 00010000:00010000:0.0:1776178092.552815:0:9224:0:(ldlm_lockd.c:1903:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 000000000b903578/0xa40083ab9bf52d1b lrc: 1/0,0 mode: --/PW res: [0x280000400:0x2c7:0x0].0x0 rrc: 2 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x44819480000000 nid: local remote: 0x0 expref: -99 pid: 20758 timeout: 0 lvb_type: 0 00010000:00010000:0.0:1776178092.552821:0:9224:0:(ldlm_lock.c:216:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 000000000b903578/0xa40083ab9bf52d1b lrc: 0/0,0 mode: --/PW res: [0x280000400:0x2c7:0x0].0x0 rrc: 2 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x44819480000000 nid: local remote: 0x0 expref: -99 pid: 20758 timeout: 0 lvb_type: 0 00010000:00010000:0.0:1776178092.552847:0:9224:0:(ldlm_lockd.c:2792:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00010000:00010000:0.0:1776178092.552848:0:9224:0:(ldlm_lockd.c:2792:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00010000:00010000:0.0:1776178092.552850:0:9224:0:(ldlm_lockd.c:2792:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00010000:00010000:1.0:1776178092.554873:0:20759:0:(ldlm_request.c:528:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0001_UUID lock: 000000002f317bcb/0xa40083ab9bf52d0d lrc: 3/0,1 mode: PW/PW res: [0x2c0000400:0x285:0x0].0x0 rrc: 2 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x40010080000000 nid: local remote: 0x0 expref: -99 pid: 20759 timeout: 0 lvb_type: 0 00010000:00010000:1.0:1776178092.554886:0:20759:0:(ldlm_lock.c:820:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0001_UUID lock: 000000002f317bcb/0xa40083ab9bf52d0d lrc: 3/0,1 mode: PW/PW res: [0x2c0000400:0x285:0x0].0x0 rrc: 2 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x50010080000000 nid: local remote: 0x0 expref: -99 pid: 20759 timeout: 0 lvb_type: 0 00010000:00010000:1.0:1776178092.554891:0:20759:0:(ldlm_lock.c:879:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0001_UUID lock: 000000002f317bcb/0xa40083ab9bf52d0d lrc: 2/0,0 mode: PW/PW res: [0x2c0000400:0x285:0x0].0x0 rrc: 2 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x50010480000000 nid: local remote: 0x0 expref: -99 pid: 20759 timeout: 0 lvb_type: 0 00010000:00010000:1.0:1776178092.554906:0:20759:0:(ldlm_lockd.c:2159:__ldlm_bl_to_thread()) added 1/1 locks to regular blp list, 1 blwis in pool 00000100:00100000:0.0:1776178092.555656:0:3348:0:(events.c:361:request_in_callback()) peer: 12345-192.168.204.3@tcp (source: 12345-192.168.204.3@tcp) 00000100:00100000:0.0:1776178092.555694:0:3348:0:(events.c:361:request_in_callback()) peer: 12345-192.168.204.3@tcp (source: 12345-192.168.204.3@tcp) 00000100:00100000:0.0:1776178092.555729:0:20770:0:(service.c:2152:ptlrpc_server_handle_req_in()) got req x1862456979072832 00010000:00010000:0.0:1776178092.555765:0:20770:0:(ldlm_lockd.c:2638:ldlm_cancel_hpreq_check()) ### hpreq cancel/convert lock ns: filter-lustre-OST0000_UUID lock: 00000000d9898ccd/0xa40083ab9bf3d8fa lrc: 5/0,0 mode: PR/PR res: [0x280000400:0x3e6:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x60000400010020 nid: 192.168.204.3@tcp remote: 0x771f68aa1eb262c2 expref: 26 pid: 14440 timeout: 899 lvb_type: 1 00000100:00100000:0.0:1776178092.555796:0:20770:0:(service.c:2152:ptlrpc_server_handle_req_in()) got req x1862456979072896 00010000:00010000:0.0:1776178092.555805:0:20770:0:(ldlm_lockd.c:2638:ldlm_cancel_hpreq_check()) ### hpreq cancel/convert lock ns: filter-lustre-OST0000_UUID lock: 00000000156434a9/0xa40083ab9bf3cbb0 lrc: 4/0,0 mode: PR/PR res: [0x280000400:0x3cb:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x60000400010020 nid: 192.168.204.3@tcp remote: 0x771f68aa1eb25e54 expref: 27 pid: 8227 timeout: 899 lvb_type: 1 00000100:00100000:0.0:1776178092.555836:0:20770:0:(nrs_fifo.c:179:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.204.3@tcp, seq: 2037 00000100:00100000:0.0:1776178092.555843:0:20770:0:(service.c:2324:ptlrpc_server_handle_request()) Handling RPC req@000000009d117b34 pname:cluuid+ref:pid:xid:nid:opc:job ldlm_cn00_005:b50963bd-8eba-4432-9a54-8888d00e3fa1+27:2271:x1862456979072832:12345-192.168.204.3@tcp:103:ldlm_bl_01.0 00010000:00010000:0.0:1776178092.555862:0:20770:0:(ldlm_lockd.c:1720:ldlm_request_cancel()) ### server-side cancel handler START: 1 locks, starting at 0 00010000:00010000:0.0:1776178092.555867:0:20770:0:(ldlm_lockd.c:1771:ldlm_request_cancel()) ### server cancels blocked lock after 0s ns: filter-lustre-OST0000_UUID lock: 00000000d9898ccd/0xa40083ab9bf3d8fa lrc: 5/0,0 mode: PR/PR res: [0x280000400:0x3e6:0x0].0x0 rrc: 4 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x60000400010020 nid: 192.168.204.3@tcp remote: 0x771f68aa1eb262c2 expref: 27 pid: 14440 timeout: 899 lvb_type: 1 00010000:00010000:0.0:1776178092.555879:0:20770:0:(ldlm_lockd.c:562:ldlm_del_waiting_lock()) ### removed ns: filter-lustre-OST0000_UUID lock: 00000000d9898ccd/0xa40083ab9bf3d8fa lrc: 4/0,0 mode: PR/PR res: [0x280000400:0x3e6:0x0].0x0 rrc: 4 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x50000400010020 nid: 192.168.204.3@tcp remote: 0x771f68aa1eb262c2 expref: 27 pid: 14440 timeout: 899 lvb_type: 1 00010000:00010000:0.0:1776178092.555911:0:20770:0:(ldlm_extent.c:318:ldlm_check_contention()) contended locks = 0 00010000:00010000:0.0:1776178092.555914:0:20770:0:(ldlm_extent.c:318:ldlm_check_contention()) contended locks = 0 00010000:00010000:0.0:1776178092.555919:0:20770:0:(ldlm_lock.c:704:ldlm_add_cp_work_item()) ### lock granted; sending completion AST. ns: filter-lustre-OST0000_UUID lock: 000000002cc4819c/0xa40083ab9bf52d45 lrc: 3/0,1 mode: PW/PW res: [0x280000400:0x3e6:0x0].0x0 rrc: 4 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x50090080000000 nid: local remote: 0x0 expref: -99 pid: 8645 timeout: 0 lvb_type: 0 00010000:00010000:0.0:1776178092.555932:0:20770:0:(ldlm_resource.c:1650:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 000000002cc4819c/0xa40083ab9bf52d45 lrc: 4/0,1 mode: PW/PW res: [0x280000400:0x3e6:0x0].0x0 rrc: 4 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x50090080000000 nid: local remote: 0x0 expref: -99 pid: 8645 timeout: 0 lvb_type: 0 00010000:00010000:0.0:1776178092.555963:0:20770:0:(ldlm_lockd.c:1782:ldlm_request_cancel()) ### server-side cancel handler END 00000100:00100000:0.0:1776178092.556007:0:20770:0:(service.c:2372:ptlrpc_server_handle_request()) Handled RPC req@000000009d117b34 pname:cluuid+ref:pid:xid:nid:opc:job ldlm_cn00_005:b50963bd-8eba-4432-9a54-8888d00e3fa1+27:2271:x1862456979072832:12345-192.168.204.3@tcp:103:ldlm_bl_01.0 Request processed in 164us (356us total) trans 0 rc 0/0 00000100:00100000:0.0:1776178092.556016:0:20770:0:(nrs_fifo.c:241:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.204.3@tcp, seq: 2037 00000100:00100000:0.0:1776178092.556040:0:20770:0:(nrs_fifo.c:179:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.204.3@tcp, seq: 2038 00000100:00100000:0.0:1776178092.556044:0:20770:0:(service.c:2324:ptlrpc_server_handle_request()) Handling RPC req@00000000f720f286 pname:cluuid+ref:pid:xid:nid:opc:job ldlm_cn00_005:b50963bd-8eba-4432-9a54-8888d00e3fa1+26:2271:x1862456979072896:12345-192.168.204.3@tcp:103:ldlm_bl_01.0 00010000:00010000:0.0:1776178092.556052:0:20770:0:(ldlm_lockd.c:1720:ldlm_request_cancel()) ### server-side cancel handler START: 1 locks, starting at 0 00010000:00010000:0.0:1776178092.556056:0:20770:0:(ldlm_lockd.c:1771:ldlm_request_cancel()) ### server cancels blocked lock after 0s ns: filter-lustre-OST0000_UUID lock: 00000000156434a9/0xa40083ab9bf3cbb0 lrc: 4/0,0 mode: PR/PR res: [0x280000400:0x3cb:0x0].0x0 rrc: 4 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x60000400010020 nid: 192.168.204.3@tcp remote: 0x771f68aa1eb25e54 expref: 26 pid: 8227 timeout: 899 lvb_type: 1 00010000:00010000:0.0:1776178092.556084:0:20770:0:(ldlm_lockd.c:562:ldlm_del_waiting_lock()) ### removed ns: filter-lustre-OST0000_UUID lock: 00000000156434a9/0xa40083ab9bf3cbb0 lrc: 3/0,0 mode: PR/PR res: [0x280000400:0x3cb:0x0].0x0 rrc: 4 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x50000400010020 nid: 192.168.204.3@tcp remote: 0x771f68aa1eb25e54 expref: 26 pid: 8227 timeout: 899 lvb_type: 1 00010000:00010000:0.0:1776178092.556098:0:20770:0:(ldlm_lock.c:216:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 00000000156434a9/0xa40083ab9bf3cbb0 lrc: 0/0,0 mode: --/PR res: [0x280000400:0x3cb:0x0].0x0 rrc: 4 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x44801400010020 nid: 192.168.204.3@tcp remote: 0x771f68aa1eb25e54 expref: 26 pid: 8227 timeout: 899 lvb_type: 1 00010000:00010000:0.0:1776178092.556113:0:20770:0:(ldlm_extent.c:318:ldlm_check_contention()) contended locks = 0 00010000:00010000:0.0:1776178092.556114:0:20770:0:(ldlm_extent.c:318:ldlm_check_contention()) contended locks = 0 00010000:00010000:0.0:1776178092.556117:0:20770:0:(ldlm_lock.c:704:ldlm_add_cp_work_item()) ### lock granted; sending completion AST. ns: filter-lustre-OST0000_UUID lock: 0000000095e53696/0xa40083ab9bf52d4c lrc: 3/0,1 mode: PW/PW res: [0x280000400:0x3cb:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x40090080000000 nid: local remote: 0x0 expref: -99 pid: 14440 timeout: 0 lvb_type: 0 00010000:00010000:0.0:1776178092.556123:0:20770:0:(ldlm_resource.c:1650:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 0000000095e53696/0xa40083ab9bf52d4c lrc: 4/0,1 mode: PW/PW res: [0x280000400:0x3cb:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x40090080000000 nid: local remote: 0x0 expref: -99 pid: 14440 timeout: 0 lvb_type: 0 00010000:00010000:0.0:1776178092.556151:0:20770:0:(ldlm_lockd.c:1782:ldlm_request_cancel()) ### server-side cancel handler END 00000100:00100000:0.0:1776178092.556172:0:20770:0:(service.c:2372:ptlrpc_server_handle_request()) Handled RPC req@00000000f720f286 pname:cluuid+ref:pid:xid:nid:opc:job ldlm_cn00_005:b50963bd-8eba-4432-9a54-8888d00e3fa1+25:2271:x1862456979072896:12345-192.168.204.3@tcp:103:ldlm_bl_01.0 Request processed in 127us (478us total) trans 0 rc 0/0 00000100:00100000:0.0:1776178092.556178:0:20770:0:(nrs_fifo.c:241:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.204.3@tcp, seq: 2038 00000100:00100000:1.0:1776178092.558748:0:20759:0:(service.c:2372:ptlrpc_server_handle_request()) Handled RPC req@0000000023210d1c pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_010:lustre-MDT0001-mdtlov_UUID+35:3353:x1862456998013504:12345-0@lo:6:osp-syn-1-1.0 Request processed in 51894us (51965us total) trans 4294969298 rc 0/0 00000100:00100000:1.0:1776178092.558756:0:20759:0:(nrs_fifo.c:241:nrs_fifo_req_stop()) NRS stop fifo request from 12345-0@lo, seq: 5185 00000100:00100000:1.0:1776178092.558773:0:20759:0:(nrs_fifo.c:179:nrs_fifo_req_get()) NRS start fifo request from 12345-0@lo, seq: 5203 00000100:00100000:1.0:1776178092.558776:0:20759:0:(service.c:2324:ptlrpc_server_handle_request()) Handling RPC req@00000000ed90ff75 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_010:lustre-MDT0001-mdtlov_UUID+34:3355:x1862456998015488:12345-0@lo:6:osp-syn-1-1.0 00002000:00080000:1.0:1776178092.558786:0:20759:0:(ofd_dev.c:1822:ofd_destroy_hdl()) lustre-OST0001: Destroy object 0x2c0000400:859 count 1 00010000:00010000:1.0:1776178092.558797:0:20759:0:(ldlm_lock.c:766:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0001_UUID lock: 00000000b2e83690/0xa40083ab9bf52d8b lrc: 3/0,1 mode: --/PW res: [0x2c0000400:0x35b:0x0].0x0 rrc: 3 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 20759 timeout: 0 lvb_type: 0 00010000:00010000:1.0:1776178092.558806:0:20759:0:(ldlm_lock.c:675:ldlm_add_bl_work_item()) ### lock incompatible; sending blocking AST. ns: filter-lustre-OST0001_UUID lock: 0000000072dc9eba/0xa40083ab9bf3b3d1 lrc: 2/0,0 mode: PR/PR res: [0x2c0000400:0x35b:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x40000000000000 nid: 192.168.204.3@tcp remote: 0x771f68aa1eb2565f expref: 19 pid: 8645 timeout: 0 lvb_type: 1 00010000:00010000:1.0:1776178092.558813:0:20759:0:(ldlm_extent.c:318:ldlm_check_contention()) contended locks = 0 00010000:00010000:1.0:1776178092.558814:0:20759:0:(ldlm_extent.c:318:ldlm_check_contention()) contended locks = 0 00010000:00010000:1.0:1776178092.558815:0:20759:0:(ldlm_resource.c:1650:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0001_UUID lock: 00000000b2e83690/0xa40083ab9bf52d8b lrc: 4/0,1 mode: --/PW res: [0x2c0000400:0x35b:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x50010080000000 nid: local remote: 0x0 expref: -99 pid: 20759 timeout: 0 lvb_type: 0 00010000:00010000:1.0:1776178092.558845:0:20759:0:(ldlm_lockd.c:953:ldlm_server_blocking_ast()) ### server preparing blocking AST ns: filter-lustre-OST0001_UUID lock: 0000000072dc9eba/0xa40083ab9bf3b3d1 lrc: 3/0,0 mode: PR/PR res: [0x2c0000400:0x35b:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x50000000010020 nid: 192.168.204.3@tcp remote: 0x771f68aa1eb2565f expref: 19 pid: 8645 timeout: 0 lvb_type: 1 00010000:00010000:1.0:1776178092.558858:0:20759:0:(ldlm_lockd.c:489:ldlm_add_waiting_lock()) ### adding to wait list(timeout: 100, AT: on) ns: filter-lustre-OST0001_UUID lock: 0000000072dc9eba/0xa40083ab9bf3b3d1 lrc: 4/0,0 mode: PR/PR res: [0x2c0000400:0x35b:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x70000400010020 nid: 192.168.204.3@tcp remote: 0x771f68aa1eb2565f expref: 19 pid: 8645 timeout: 899 lvb_type: 1 00000100:00100000:1.0:1776178092.558866:0:20759:0:(client.c:1733:ptlrpc_send_new_req()) Sending RPC req@00000000a59bfc89 pname:cluuid:pid:xid:nid:opc:job ll_ost00_010:lustre-OST0001_UUID:20759:1862456998016192:192.168.204.3@tcp:104: 00000100:00100000:1.0:1776178092.558887:0:20759:0:(client.c:2500:ptlrpc_set_wait()) set 000000006356692e going to sleep for 6 seconds 00010000:00010000:1.0:1776178092.558912:0:9224:0:(ldlm_lockd.c:2792:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:1.0:1776178092.558924:0:20766:0:(nrs_fifo.c:179:nrs_fifo_req_get()) NRS start fifo request from 12345-0@lo, seq: 5205 00000100:00100000:1.0:1776178092.558927:0:20766:0:(service.c:2324:ptlrpc_server_handle_request()) Handling RPC req@00000000fa6d10e9 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_017:lustre-MDT0001-mdtlov_UUID+34:3354:x1862456998016064:12345-0@lo:6:osp-syn-1-1.0 00002000:00080000:1.0:1776178092.558937:0:20766:0:(ofd_dev.c:1822:ofd_destroy_hdl()) lustre-OST0001: Destroy object 0x2c0000400:619 count 1 00010000:00010000:1.0:1776178092.558946:0:20766:0:(ldlm_lock.c:766:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0001_UUID lock: 00000000daa24072/0xa40083ab9bf52d99 lrc: 3/0,1 mode: --/PW res: [0x2c0000400:0x26b:0x0].0x0 rrc: 3 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 20766 timeout: 0 lvb_type: 0 00010000:00010000:1.0:1776178092.558955:0:20766:0:(ldlm_lock.c:675:ldlm_add_bl_work_item()) ### lock incompatible; sending blocking AST. ns: filter-lustre-OST0001_UUID lock: 00000000344b695c/0xa40083ab9bf33db1 lrc: 2/0,0 mode: PR/PR res: [0x2c0000400:0x26b:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x40000000000000 nid: 192.168.204.3@tcp remote: 0x771f68aa1eb22eff expref: 19 pid: 8228 timeout: 0 lvb_type: 1 00010000:00010000:1.0:1776178092.558961:0:20766:0:(ldlm_extent.c:318:ldlm_check_contention()) contended locks = 0 00010000:00010000:1.0:1776178092.558962:0:20766:0:(ldlm_extent.c:318:ldlm_check_contention()) contended locks = 0 00010000:00010000:1.0:1776178092.558963:0:20766:0:(ldlm_resource.c:1650:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0001_UUID lock: 00000000daa24072/0xa40083ab9bf52d99 lrc: 4/0,1 mode: --/PW res: [0x2c0000400:0x26b:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x50010080000000 nid: local remote: 0x0 expref: -99 pid: 20766 timeout: 0 lvb_type: 0 00010000:00010000:0.0:1776178092.561157:0:9225:0:(ldlm_lockd.c:2792:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00010000:00010000:1.0:1776178092.561275:0:20766:0:(ldlm_lockd.c:953:ldlm_server_blocking_ast()) ### server preparing blocking AST ns: filter-lustre-OST0001_UUID lock: 00000000344b695c/0xa40083ab9bf33db1 lrc: 3/0,0 mode: PR/PR res: [0x2c0000400:0x26b:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x50000000010020 nid: 192.168.204.3@tcp remote: 0x771f68aa1eb22eff expref: 19 pid: 8228 timeout: 0 lvb_type: 1 00010000:00010000:1.0:1776178092.561285:0:20766:0:(ldlm_lockd.c:489:ldlm_add_waiting_lock()) ### adding to wait list(timeout: 100, AT: on) ns: filter-lustre-OST0001_UUID lock: 00000000344b695c/0xa40083ab9bf33db1 lrc: 4/0,0 mode: PR/PR res: [0x2c0000400:0x26b:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x70000400010020 nid: 192.168.204.3@tcp remote: 0x771f68aa1eb22eff expref: 19 pid: 8228 timeout: 899 lvb_type: 1 00000100:00100000:0.0:1776178092.561329:0:3346:0:(events.c:361:request_in_callback()) peer: 12345-192.168.204.3@tcp (source: 12345-192.168.204.3@tcp) 00000100:00100000:0.0:1776178092.561365:0:3346:0:(events.c:361:request_in_callback()) peer: 12345-192.168.204.3@tcp (source: 12345-192.168.204.3@tcp) 00000100:00100000:0.0:1776178092.561399:0:3346:0:(events.c:361:request_in_callback()) peer: 12345-192.168.204.3@tcp (source: 12345-192.168.204.3@tcp) 00000100:00100000:1.0:1776178092.561462:0:20766:0:(client.c:1733:ptlrpc_send_new_req()) Sending RPC req@00000000a30eb964 pname:cluuid:pid:xid:nid:opc:job ll_ost00_017:lustre-OST0001_UUID:20766:1862456998016896:192.168.204.3@tcp:104: 00000100:00100000:1.0:1776178092.561487:0:20766:0:(client.c:2500:ptlrpc_set_wait()) set 000000000f3ac011 going to sleep for 6 seconds 00000100:00100000:1.0:1776178092.561505:0:20767:0:(service.c:2152:ptlrpc_server_handle_req_in()) got req x1862456979073024 00010000:00010000:1.0:1776178092.561523:0:20767:0:(ldlm_lockd.c:2638:ldlm_cancel_hpreq_check()) ### hpreq cancel/convert lock ns: filter-lustre-OST0001_UUID lock: 00000000814b93ef/0xa40083ab9bf32b7b lrc: 5/0,0 mode: PR/PR res: [0x2c0000400:0x246:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x60000400010020 nid: 192.168.204.3@tcp remote: 0x771f68aa1eb228ed expref: 20 pid: 8228 timeout: 899 lvb_type: 1 00000100:00100000:0.0:1776178092.561530:0:20754:0:(client.c:2215:ptlrpc_check_set()) Completed RPC req@0000000084030222 pname:cluuid:pid:xid:nid:opc:job ll_ost00_005:lustre-OST0001_UUID:20754:1862456998015360:192.168.204.3@tcp:104: 00010000:00010000:0.0:1776178092.561545:0:20754:0:(ldlm_request.c:283:ldlm_completion_ast()) ### client-side enqueue returned a blocked locksleeping ns: filter-lustre-OST0001_UUID lock: 00000000014a8a18/0xa40083ab9bf52d61 lrc: 3/0,1 mode: --/PW res: [0x2c0000400:0x246:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x40010080000000 nid: local remote: 0x0 expref: -99 pid: 20754 timeout: 0 lvb_type: 0 00000100:00100000:1.0:1776178092.561545:0:20767:0:(service.c:2152:ptlrpc_server_handle_req_in()) got req x1862456979072960 00010000:00010000:1.0:1776178092.561554:0:20767:0:(ldlm_lockd.c:2638:ldlm_cancel_hpreq_check()) ### hpreq cancel/convert lock ns: filter-lustre-OST0000_UUID lock: 00000000a376ee44/0xa40083ab9bf3bb72 lrc: 5/0,0 mode: PR/PR res: [0x280000400:0x3aa:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x60000400010020 nid: 192.168.204.3@tcp remote: 0x771f68aa1eb258ea expref: 24 pid: 8645 timeout: 899 lvb_type: 1 00000100:00100000:0.0:1776178092.561564:0:6241:0:(service.c:2152:ptlrpc_server_handle_req_in()) got req x1862456979073088 00000100:00100000:1.0:1776178092.561570:0:20767:0:(nrs_fifo.c:179:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.204.3@tcp, seq: 2039 00000100:00100000:1.0:1776178092.561574:0:20767:0:(service.c:2324:ptlrpc_server_handle_request()) Handling RPC req@00000000a770fd7b pname:cluuid+ref:pid:xid:nid:opc:job ldlm_cn00_004:b50963bd-8eba-4432-9a54-8888d00e3fa1+20:2270:x1862456979073024:12345-192.168.204.3@tcp:103:ldlm_bl_03.0 00010000:00010000:0.0:1776178092.561576:0:6241:0:(ldlm_lockd.c:2638:ldlm_cancel_hpreq_check()) ### hpreq cancel/convert lock ns: filter-lustre-OST0000_UUID lock: 00000000eaa9fbf1/0xa40083ab9bf351a0 lrc: 5/0,0 mode: PR/PR res: [0x280000400:0x2d3:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x60000400010020 nid: 192.168.204.3@tcp remote: 0x771f68aa1eb235a4 expref: 25 pid: 8645 timeout: 899 lvb_type: 1 00010000:00010000:1.0:1776178092.561587:0:20767:0:(ldlm_lockd.c:1720:ldlm_request_cancel()) ### server-side cancel handler START: 1 locks, starting at 0 00000100:00100000:0.0:1776178092.561590:0:6241:0:(nrs_fifo.c:179:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.204.3@tcp, seq: 2040 00010000:00010000:1.0:1776178092.561591:0:20767:0:(ldlm_lockd.c:1771:ldlm_request_cancel()) ### server cancels blocked lock after 0s ns: filter-lustre-OST0001_UUID lock: 00000000814b93ef/0xa40083ab9bf32b7b lrc: 4/0,0 mode: PR/PR res: [0x2c0000400:0x246:0x0].0x0 rrc: 4 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x60000400010020 nid: 192.168.204.3@tcp remote: 0x771f68aa1eb228ed expref: 20 pid: 8228 timeout: 899 lvb_type: 1 00000100:00100000:0.0:1776178092.561594:0:6241:0:(service.c:2324:ptlrpc_server_handle_request()) Handling RPC req@000000007e26d6b1 pname:cluuid+ref:pid:xid:nid:opc:job ldlm_cn00_000:b50963bd-8eba-4432-9a54-8888d00e3fa1+25:2270:x1862456979072960:12345-192.168.204.3@tcp:103:ldlm_bl_03.0 00010000:00010000:0.0:1776178092.561599:0:6241:0:(ldlm_lockd.c:1720:ldlm_request_cancel()) ### server-side cancel handler START: 1 locks, starting at 0 00010000:00010000:1.0:1776178092.561600:0:20767:0:(ldlm_lockd.c:562:ldlm_del_waiting_lock()) ### removed ns: filter-lustre-OST0001_UUID lock: 00000000814b93ef/0xa40083ab9bf32b7b lrc: 3/0,0 mode: PR/PR res: [0x2c0000400:0x246:0x0].0x0 rrc: 4 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x50000400010020 nid: 192.168.204.3@tcp remote: 0x771f68aa1eb228ed expref: 20 pid: 8228 timeout: 899 lvb_type: 1 00010000:00010000:0.0:1776178092.561602:0:6241:0:(ldlm_lockd.c:1771:ldlm_request_cancel()) ### server cancels blocked lock after 0s ns: filter-lustre-OST0000_UUID lock: 00000000a376ee44/0xa40083ab9bf3bb72 lrc: 5/0,0 mode: PR/PR res: [0x280000400:0x3aa:0x0].0x0 rrc: 4 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x60000400010020 nid: 192.168.204.3@tcp remote: 0x771f68aa1eb258ea expref: 25 pid: 8645 timeout: 899 lvb_type: 1 00010000:00010000:0.0:1776178092.561610:0:6241:0:(ldlm_lockd.c:562:ldlm_del_waiting_lock()) ### removed ns: filter-lustre-OST0000_UUID lock: 00000000a376ee44/0xa40083ab9bf3bb72 lrc: 4/0,0 mode: PR/PR res: [0x280000400:0x3aa:0x0].0x0 rrc: 4 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x50000400010020 nid: 192.168.204.3@tcp remote: 0x771f68aa1eb258ea expref: 25 pid: 8645 timeout: 899 lvb_type: 1 00010000:00010000:0.0:1776178092.561626:0:6241:0:(ldlm_extent.c:318:ldlm_check_contention()) contended locks = 0 00010000:00010000:0.0:1776178092.561628:0:6241:0:(ldlm_extent.c:318:ldlm_check_contention()) contended locks = 0 00010000:00010000:0.0:1776178092.561631:0:6241:0:(ldlm_lock.c:704:ldlm_add_cp_work_item()) ### lock granted; sending completion AST. ns: filter-lustre-OST0000_UUID lock: 00000000bbda5115/0xa40083ab9bf52d5a lrc: 3/0,1 mode: PW/PW res: [0x280000400:0x3aa:0x0].0x0 rrc: 4 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x50090080000000 nid: local remote: 0x0 expref: -99 pid: 20769 timeout: 0 lvb_type: 0 00010000:00010000:0.0:1776178092.561641:0:6241:0:(ldlm_resource.c:1650:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 00000000bbda5115/0xa40083ab9bf52d5a lrc: 4/0,1 mode: PW/PW res: [0x280000400:0x3aa:0x0].0x0 rrc: 4 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x50090080000000 nid: local remote: 0x0 expref: -99 pid: 20769 timeout: 0 lvb_type: 0 00010000:00010000:0.0:1776178092.561656:0:6241:0:(ldlm_lockd.c:1782:ldlm_request_cancel()) ### server-side cancel handler END 00000100:00100000:0.0:1776178092.561681:0:6241:0:(service.c:2372:ptlrpc_server_handle_request()) Handled RPC req@000000007e26d6b1 pname:cluuid+ref:pid:xid:nid:opc:job ldlm_cn00_000:b50963bd-8eba-4432-9a54-8888d00e3fa1+25:2270:x1862456979072960:12345-192.168.204.3@tcp:103:ldlm_bl_03.0 Request processed in 86us (316us total) trans 0 rc 0/0 00000100:00100000:0.0:1776178092.561688:0:6241:0:(nrs_fifo.c:241:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.204.3@tcp, seq: 2040 00000100:00100000:0.0:1776178092.561703:0:6241:0:(nrs_fifo.c:179:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.204.3@tcp, seq: 2041 00000100:00100000:0.0:1776178092.561706:0:6241:0:(service.c:2324:ptlrpc_server_handle_request()) Handling RPC req@00000000ba273b2a pname:cluuid+ref:pid:xid:nid:opc:job ldlm_cn00_000:b50963bd-8eba-4432-9a54-8888d00e3fa1+24:2270:x1862456979073088:12345-192.168.204.3@tcp:103:ldlm_bl_03.0 00010000:00010000:0.0:1776178092.561714:0:6241:0:(ldlm_lockd.c:1720:ldlm_request_cancel()) ### server-side cancel handler START: 1 locks, starting at 0 00010000:00010000:0.0:1776178092.561717:0:6241:0:(ldlm_lockd.c:1771:ldlm_request_cancel()) ### server cancels blocked lock after 0s ns: filter-lustre-OST0000_UUID lock: 00000000eaa9fbf1/0xa40083ab9bf351a0 lrc: 5/0,0 mode: PR/PR res: [0x280000400:0x2d3:0x0].0x0 rrc: 4 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x60000400010020 nid: 192.168.204.3@tcp remote: 0x771f68aa1eb235a4 expref: 24 pid: 8645 timeout: 899 lvb_type: 1 00010000:00010000:0.0:1776178092.561726:0:6241:0:(ldlm_lockd.c:562:ldlm_del_waiting_lock()) ### removed ns: filter-lustre-OST0000_UUID lock: 00000000eaa9fbf1/0xa40083ab9bf351a0 lrc: 4/0,0 mode: PR/PR res: [0x280000400:0x2d3:0x0].0x0 rrc: 4 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x50000400010020 nid: 192.168.204.3@tcp remote: 0x771f68aa1eb235a4 expref: 24 pid: 8645 timeout: 899 lvb_type: 1 00010000:00010000:0.0:1776178092.561738:0:6241:0:(ldlm_extent.c:318:ldlm_check_contention()) contended locks = 0 00010000:00010000:0.0:1776178092.561739:0:6241:0:(ldlm_extent.c:318:ldlm_check_contention()) contended locks = 0 00010000:00010000:0.0:1776178092.561741:0:6241:0:(ldlm_lock.c:704:ldlm_add_cp_work_item()) ### lock granted; sending completion AST. ns: filter-lustre-OST0000_UUID lock: 0000000015ae4ee6/0xa40083ab9bf52d53 lrc: 3/0,1 mode: PW/PW res: [0x280000400:0x2d3:0x0].0x0 rrc: 4 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x50090080000000 nid: local remote: 0x0 expref: -99 pid: 8228 timeout: 0 lvb_type: 0 00010000:00010000:0.0:1776178092.561748:0:6241:0:(ldlm_resource.c:1650:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 0000000015ae4ee6/0xa40083ab9bf52d53 lrc: 4/0,1 mode: PW/PW res: [0x280000400:0x2d3:0x0].0x0 rrc: 4 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x50090080000000 nid: local remote: 0x0 expref: -99 pid: 8228 timeout: 0 lvb_type: 0 00010000:00010000:1.0:1776178092.561750:0:20767:0:(ldlm_lock.c:216:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0001_UUID lock: 00000000814b93ef/0xa40083ab9bf32b7b lrc: 0/0,0 mode: --/PR res: [0x2c0000400:0x246:0x0].0x0 rrc: 4 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x44801400010020 nid: 192.168.204.3@tcp remote: 0x771f68aa1eb228ed expref: 20 pid: 8228 timeout: 899 lvb_type: 1 00010000:00010000:0.0:1776178092.561759:0:6241:0:(ldlm_lockd.c:1782:ldlm_request_cancel()) ### server-side cancel handler END 00010000:00010000:1.0:1776178092.561763:0:20767:0:(ldlm_extent.c:318:ldlm_check_contention()) contended locks = 0 00010000:00010000:1.0:1776178092.561764:0:20767:0:(ldlm_extent.c:318:ldlm_check_contention()) contended locks = 0 00010000:00010000:1.0:1776178092.561766:0:20767:0:(ldlm_lock.c:704:ldlm_add_cp_work_item()) ### lock granted; sending completion AST. ns: filter-lustre-OST0001_UUID lock: 00000000014a8a18/0xa40083ab9bf52d61 lrc: 3/0,1 mode: PW/PW res: [0x2c0000400:0x246:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x40090080000000 nid: local remote: 0x0 expref: -99 pid: 20754 timeout: 0 lvb_type: 0 00010000:00010000:1.0:1776178092.561772:0:20767:0:(ldlm_resource.c:1650:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0001_UUID lock: 00000000014a8a18/0xa40083ab9bf52d61 lrc: 4/0,1 mode: PW/PW res: [0x2c0000400:0x246:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x40090080000000 nid: local remote: 0x0 expref: -99 pid: 20754 timeout: 0 lvb_type: 0 00000100:00100000:0.0:1776178092.561773:0:6241:0:(service.c:2372:ptlrpc_server_handle_request()) Handled RPC req@00000000ba273b2a pname:cluuid+ref:pid:xid:nid:opc:job ldlm_cn00_000:b50963bd-8eba-4432-9a54-8888d00e3fa1+24:2270:x1862456979073088:12345-192.168.204.3@tcp:103:ldlm_bl_03.0 Request processed in 66us (375us total) trans 0 rc 0/0 00000100:00100000:0.0:1776178092.561780:0:6241:0:(nrs_fifo.c:241:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.204.3@tcp, seq: 2041 00010000:00010000:1.0:1776178092.561788:0:20767:0:(ldlm_lockd.c:1782:ldlm_request_cancel()) ### server-side cancel handler END 00000100:00100000:1.0:1776178092.561803:0:20767:0:(service.c:2372:ptlrpc_server_handle_request()) Handled RPC req@00000000a770fd7b pname:cluuid+ref:pid:xid:nid:opc:job ldlm_cn00_004:b50963bd-8eba-4432-9a54-8888d00e3fa1+19:2270:x1862456979073024:12345-192.168.204.3@tcp:103:ldlm_bl_03.0 Request processed in 228us (476us total) trans 0 rc 0/0 00010000:00010000:0.0:1776178092.561808:0:8228:0:(ldlm_lock.c:216:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 00000000eaa9fbf1/0xa40083ab9bf351a0 lrc: 0/0,0 mode: --/PR res: [0x280000400:0x2d3:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x44801400010020 nid: 192.168.204.3@tcp remote: 0x771f68aa1eb235a4 expref: 23 pid: 8645 timeout: 899 lvb_type: 1 00000100:00100000:1.0:1776178092.561810:0:20767:0:(nrs_fifo.c:241:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.204.3@tcp, seq: 2039 00000100:00100000:0.0:1776178092.561818:0:8228:0:(client.c:2215:ptlrpc_check_set()) Completed RPC req@000000009016e75e pname:cluuid:pid:xid:nid:opc:job ll_ost00_001:lustre-OST0000_UUID:8228:1862456998015168:192.168.204.3@tcp:104: 00010000:00010000:0.0:1776178092.561842:0:8228:0:(ldlm_request.c:283:ldlm_completion_ast()) ### client-side enqueue returned a blocked locksleeping ns: filter-lustre-OST0000_UUID lock: 0000000015ae4ee6/0xa40083ab9bf52d53 lrc: 3/0,1 mode: PW/PW res: [0x280000400:0x2d3:0x0].0x0 rrc: 2 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x40010080000000 nid: local remote: 0x0 expref: -99 pid: 8228 timeout: 0 lvb_type: 0 00010000:00010000:0.0:1776178092.561850:0:8228:0:(ldlm_request.c:199:ldlm_completion_tail()) ### client-side enqueue: granted ns: filter-lustre-OST0000_UUID lock: 0000000015ae4ee6/0xa40083ab9bf52d53 lrc: 3/0,1 mode: PW/PW res: [0x280000400:0x2d3:0x0].0x0 rrc: 2 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x40010080000000 nid: local remote: 0x0 expref: -99 pid: 8228 timeout: 0 lvb_type: 0 00010000:00010000:0.0:1776178092.561856:0:8228:0:(ldlm_request.c:528:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 0000000015ae4ee6/0xa40083ab9bf52d53 lrc: 3/0,1 mode: PW/PW res: [0x280000400:0x2d3:0x0].0x0 rrc: 2 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x40010080000000 nid: local remote: 0x0 expref: -99 pid: 8228 timeout: 0 lvb_type: 0 00010000:00010000:0.0:1776178092.561863:0:8228:0:(ldlm_lock.c:820:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0000_UUID lock: 0000000015ae4ee6/0xa40083ab9bf52d53 lrc: 3/0,1 mode: PW/PW res: [0x280000400:0x2d3:0x0].0x0 rrc: 2 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x50010080000000 nid: local remote: 0x0 expref: -99 pid: 8228 timeout: 0 lvb_type: 0 00010000:00010000:0.0:1776178092.561869:0:8228:0:(ldlm_lock.c:879:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 0000000015ae4ee6/0xa40083ab9bf52d53 lrc: 2/0,0 mode: PW/PW res: [0x280000400:0x2d3:0x0].0x0 rrc: 2 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x50010480000000 nid: local remote: 0x0 expref: -99 pid: 8228 timeout: 0 lvb_type: 0 00010000:00010000:0.0:1776178092.561884:0:8228:0:(ldlm_lockd.c:2159:__ldlm_bl_to_thread()) added 1/1 locks to regular blp list, 1 blwis in pool 00010000:00010000:1.0:1776178092.561915:0:20754:0:(ldlm_request.c:199:ldlm_completion_tail()) ### client-side enqueue: granted ns: filter-lustre-OST0001_UUID lock: 00000000014a8a18/0xa40083ab9bf52d61 lrc: 3/0,1 mode: PW/PW res: [0x2c0000400:0x246:0x0].0x0 rrc: 2 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x40010080000000 nid: local remote: 0x0 expref: -99 pid: 20754 timeout: 0 lvb_type: 0 00010000:00010000:1.0:1776178092.561922:0:20754:0:(ldlm_request.c:528:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0001_UUID lock: 00000000014a8a18/0xa40083ab9bf52d61 lrc: 3/0,1 mode: PW/PW res: [0x2c0000400:0x246:0x0].0x0 rrc: 2 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x40010080000000 nid: local remote: 0x0 expref: -99 pid: 20754 timeout: 0 lvb_type: 0 00010000:00010000:1.0:1776178092.561928:0:20754:0:(ldlm_lock.c:820:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0001_UUID lock: 00000000014a8a18/0xa40083ab9bf52d61 lrc: 3/0,1 mode: PW/PW res: [0x2c0000400:0x246:0x0].0x0 rrc: 2 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x50010080000000 nid: local remote: 0x0 expref: -99 pid: 20754 timeout: 0 lvb_type: 0 00010000:00010000:1.0:1776178092.561934:0:20754:0:(ldlm_lock.c:879:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0001_UUID lock: 00000000014a8a18/0xa40083ab9bf52d61 lrc: 2/0,0 mode: PW/PW res: [0x2c0000400:0x246:0x0].0x0 rrc: 2 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x50010480000000 nid: local remote: 0x0 expref: -99 pid: 20754 timeout: 0 lvb_type: 0 00010000:00010000:1.0:1776178092.561946:0:20754:0:(ldlm_lockd.c:2159:__ldlm_bl_to_thread()) added 1/2 locks to regular blp list, 2 blwis in pool 00000100:00100000:0.0:1776178092.562122:0:8228:0:(service.c:2372:ptlrpc_server_handle_request()) Handled RPC req@00000000af523f53 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_001:lustre-MDT0001-mdtlov_UUID+40:3354:x1862456998015104:12345-0@lo:6:osp-syn-0-1.0 Request processed in 20399us (20461us total) trans 4294969322 rc 0/0 00000100:00100000:0.0:1776178092.562130:0:8228:0:(nrs_fifo.c:241:nrs_fifo_req_stop()) NRS stop fifo request from 12345-0@lo, seq: 5195 00000004:00080000:0.0:1776178092.562164:0:3354:0:(osp_sync.c:562:osp_sync_interpret()) reply req 00000000a62daf9a/2, rc 0, transno 2026 00000100:00100000:0.0:1776178092.562175:0:3354:0:(client.c:2215:ptlrpc_check_set()) Completed RPC req@00000000a62daf9a pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_01:lustre-MDT0001-mdtlov_UUID:3354:1862456998015104:0@lo:6:osp-syn-0-1.0 00000100:00100000:0.0:1776178092.562234:0:3355:0:(client.c:1733:ptlrpc_send_new_req()) Sending RPC req@000000009016e75e pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_02:lustre-MDT0001-mdtlov_UUID:3355:1862456998016960:0@lo:6:osp-syn-0-1.0 00000100:00100000:0.0:1776178092.562261:0:3355:0:(events.c:361:request_in_callback()) peer: 12345-0@lo (source: 12345-0@lo) 00000100:00100000:0.0:1776178092.562290:0:8228:0:(service.c:2152:ptlrpc_server_handle_req_in()) got req x1862456998016960 00000100:00100000:0.0:1776178092.562310:0:8228:0:(nrs_fifo.c:179:nrs_fifo_req_get()) NRS start fifo request from 12345-0@lo, seq: 5211 00000100:00100000:0.0:1776178092.562314:0:8228:0:(service.c:2324:ptlrpc_server_handle_request()) Handling RPC req@00000000812a7ebe pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_001:lustre-MDT0001-mdtlov_UUID+40:3355:x1862456998016960:12345-0@lo:6:osp-syn-0-1.0 00002000:00080000:0.0:1776178092.562334:0:8228:0:(ofd_dev.c:1822:ofd_destroy_hdl()) lustre-OST0000: Destroy object 0x280000400:645 count 1 00010000:00010000:0.0:1776178092.562351:0:8228:0:(ldlm_lock.c:766:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0000_UUID lock: 00000000319537bc/0xa40083ab9bf52dc3 lrc: 3/0,1 mode: --/PW res: [0x280000400:0x285:0x0].0x0 rrc: 3 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 8228 timeout: 0 lvb_type: 0 00010000:00010000:0.0:1776178092.562362:0:8228:0:(ldlm_lock.c:675:ldlm_add_bl_work_item()) ### lock incompatible; sending blocking AST. ns: filter-lustre-OST0000_UUID lock: 00000000485847f3/0xa40083ab9bf32b3c lrc: 2/0,0 mode: PR/PR res: [0x280000400:0x285:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x40000000000000 nid: 192.168.204.3@tcp remote: 0x771f68aa1eb228d8 expref: 22 pid: 8645 timeout: 0 lvb_type: 1 00010000:00010000:0.0:1776178092.562368:0:8228:0:(ldlm_extent.c:318:ldlm_check_contention()) contended locks = 0 00010000:00010000:0.0:1776178092.562369:0:8228:0:(ldlm_extent.c:318:ldlm_check_contention()) contended locks = 0 00010000:00010000:0.0:1776178092.562370:0:8228:0:(ldlm_resource.c:1650:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 00000000319537bc/0xa40083ab9bf52dc3 lrc: 4/0,1 mode: --/PW res: [0x280000400:0x285:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x50010080000000 nid: local remote: 0x0 expref: -99 pid: 8228 timeout: 0 lvb_type: 0 00010000:00010000:0.0:1776178092.562389:0:8228:0:(ldlm_lockd.c:953:ldlm_server_blocking_ast()) ### server preparing blocking AST ns: filter-lustre-OST0000_UUID lock: 00000000485847f3/0xa40083ab9bf32b3c lrc: 3/0,0 mode: PR/PR res: [0x280000400:0x285:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x50000000010020 nid: 192.168.204.3@tcp remote: 0x771f68aa1eb228d8 expref: 22 pid: 8645 timeout: 0 lvb_type: 1 00010000:00010000:0.0:1776178092.562398:0:8228:0:(ldlm_lockd.c:489:ldlm_add_waiting_lock()) ### adding to wait list(timeout: 100, AT: on) ns: filter-lustre-OST0000_UUID lock: 00000000485847f3/0xa40083ab9bf32b3c lrc: 4/0,0 mode: PR/PR res: [0x280000400:0x285:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x70000400010020 nid: 192.168.204.3@tcp remote: 0x771f68aa1eb228d8 expref: 22 pid: 8645 timeout: 899 lvb_type: 1 00000100:00100000:0.0:1776178092.562407:0:8228:0:(client.c:1733:ptlrpc_send_new_req()) Sending RPC req@000000008bb01090 pname:cluuid:pid:xid:nid:opc:job ll_ost00_001:lustre-OST0000_UUID:8228:1862456998017024:192.168.204.3@tcp:104: 00000100:00100000:0.0:1776178092.562431:0:8228:0:(client.c:2500:ptlrpc_set_wait()) set 00000000418e8d44 going to sleep for 6 seconds 00010000:00010000:0.0:1776178092.563348:0:6244:0:(ldlm_lockd.c:2789:ldlm_bl_get_work()) Got 1 locks of 1 total in blp. (1 blwis in pool) 00010000:00010000:0.0:1776178092.563354:0:6244:0:(ldlm_lockd.c:1879:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 0000000015ae4ee6/0xa40083ab9bf52d53 lrc: 2/0,0 mode: PW/PW res: [0x280000400:0x2d3:0x0].0x0 rrc: 2 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x40010480000000 nid: local remote: 0x0 expref: -99 pid: 8228 timeout: 0 lvb_type: 0 00010000:00010000:0.0:1776178092.563366:0:6244:0:(ldlm_lockd.c:1893:ldlm_handle_bl_callback()) Lock 0000000015ae4ee6 already unused, calling callback (0000000063c97258) 00010000:00010000:0.0:1776178092.563370:0:6244:0:(ldlm_request.c:364:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 0000000015ae4ee6/0xa40083ab9bf52d53 lrc: 2/0,0 mode: PW/PW res: [0x280000400:0x2d3:0x0].0x0 rrc: 2 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x40010480000000 nid: local remote: 0x0 expref: -99 pid: 8228 timeout: 0 lvb_type: 0 00010000:00010000:0.0:1776178092.563379:0:6244:0:(ldlm_request.c:1257:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 0000000015ae4ee6/0xa40083ab9bf52d53 lrc: 3/0,0 mode: PW/PW res: [0x280000400:0x2d3:0x0].0x0 rrc: 2 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x40018480000000 nid: local remote: 0x0 expref: -99 pid: 8228 timeout: 0 lvb_type: 0 00010000:00010000:0.0:1776178092.563401:0:6244:0:(ldlm_lockd.c:1903:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 0000000015ae4ee6/0xa40083ab9bf52d53 lrc: 1/0,0 mode: --/PW res: [0x280000400:0x2d3:0x0].0x0 rrc: 2 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x44819480000000 nid: local remote: 0x0 expref: -99 pid: 8228 timeout: 0 lvb_type: 0 00010000:00010000:0.0:1776178092.563408:0:6244:0:(ldlm_lock.c:216:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 0000000015ae4ee6/0xa40083ab9bf52d53 lrc: 0/0,0 mode: --/PW res: [0x280000400:0x2d3:0x0].0x0 rrc: 2 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x44819480000000 nid: local remote: 0x0 expref: -99 pid: 8228 timeout: 0 lvb_type: 0 00010000:00010000:0.0:1776178092.563425:0:6244:0:(ldlm_lockd.c:2789:ldlm_bl_get_work()) Got 1 locks of 0 total in blp. (0 blwis in pool) 00010000:00010000:0.0:1776178092.563428:0:6244:0:(ldlm_lockd.c:1879:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0001_UUID lock: 00000000014a8a18/0xa40083ab9bf52d61 lrc: 2/0,0 mode: PW/PW res: [0x2c0000400:0x246:0x0].0x0 rrc: 2 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x40010480000000 nid: local remote: 0x0 expref: -99 pid: 20754 timeout: 0 lvb_type: 0 00010000:00010000:0.0:1776178092.563434:0:6244:0:(ldlm_lockd.c:1893:ldlm_handle_bl_callback()) Lock 00000000014a8a18 already unused, calling callback (0000000063c97258) 00010000:00010000:0.0:1776178092.563436:0:6244:0:(ldlm_request.c:364:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0001_UUID lock: 00000000014a8a18/0xa40083ab9bf52d61 lrc: 2/0,0 mode: PW/PW res: [0x2c0000400:0x246:0x0].0x0 rrc: 2 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x40010480000000 nid: local remote: 0x0 expref: -99 pid: 20754 timeout: 0 lvb_type: 0 00010000:00010000:0.0:1776178092.563442:0:6244:0:(ldlm_request.c:1257:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0001_UUID lock: 00000000014a8a18/0xa40083ab9bf52d61 lrc: 3/0,0 mode: PW/PW res: [0x2c0000400:0x246:0x0].0x0 rrc: 2 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x40018480000000 nid: local remote: 0x0 expref: -99 pid: 20754 timeout: 0 lvb_type: 0 00010000:00010000:0.0:1776178092.563452:0:6244:0:(ldlm_lockd.c:1903:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0001_UUID lock: 00000000014a8a18/0xa40083ab9bf52d61 lrc: 1/0,0 mode: --/PW res: [0x2c0000400:0x246:0x0].0x0 rrc: 2 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x44819480000000 nid: local remote: 0x0 expref: -99 pid: 20754 timeout: 0 lvb_type: 0 00010000:00010000:0.0:1776178092.563462:0:6244:0:(ldlm_lock.c:216:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0001_UUID lock: 00000000014a8a18/0xa40083ab9bf52d61 lrc: 0/0,0 mode: --/PW res: [0x2c0000400:0x246:0x0].0x0 rrc: 2 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x44819480000000 nid: local remote: 0x0 expref: -99 pid: 20754 timeout: 0 lvb_type: 0 00010000:00010000:0.0:1776178092.563474:0:6244:0:(ldlm_lockd.c:2792:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00010000:00010000:0.0:1776178092.563475:0:6244:0:(ldlm_lockd.c:2792:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00010000:00010000:0.0:1776178092.563477:0:6244:0:(ldlm_lockd.c:2792:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00010000:00010000:0.0:1776178092.564118:0:9224:0:(ldlm_lockd.c:2792:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:1.0:1776178092.565513:0:20754:0:(service.c:2372:ptlrpc_server_handle_request()) Handled RPC req@0000000021d5d4e8 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_005:lustre-MDT0001-mdtlov_UUID+37:3355:x1862456998014912:12345-0@lo:6:osp-syn-1-1.0 Request processed in 15280us (21918us total) trans 4294969300 rc 0/0 00000100:00100000:1.0:1776178092.565526:0:20754:0:(nrs_fifo.c:241:nrs_fifo_req_stop()) NRS stop fifo request from 12345-0@lo, seq: 5197 00010000:00010000:1.0:1776178092.565569:0:6245:0:(ldlm_lockd.c:2792:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000004:00080000:0.0:1776178092.566485:0:3355:0:(osp_sync.c:562:osp_sync_interpret()) reply req 000000007d5bffd0/2, rc 0, transno 2004 00000100:00100000:0.0:1776178092.566536:0:3355:0:(client.c:2215:ptlrpc_check_set()) Completed RPC req@000000007d5bffd0 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_02:lustre-MDT0001-mdtlov_UUID:3355:1862456998014912:0@lo:6:osp-syn-1-1.0 00000004:00080000:0.0:1776178092.566557:0:3355:0:(osp_sync.c:562:osp_sync_interpret()) reply req 00000000d2194868/2, rc 0, transno 2027 00000100:00100000:0.0:1776178092.566605:0:3355:0:(client.c:2215:ptlrpc_check_set()) Completed RPC req@00000000d2194868 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_02:lustre-MDT0001-mdtlov_UUID:3355:1862456998014720:0@lo:6:osp-syn-0-1.0 00000100:00100000:1.0:1776178092.566913:0:3354:0:(ptlrpcd.c:415:ptlrpcd_check()) transfer 1 async RPCs [0->1] 00000100:00100000:1.0:1776178092.566923:0:3354:0:(client.c:1733:ptlrpc_send_new_req()) Sending RPC req@00000000478fd8d5 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_01:lustre-MDT0001-mdtlov_UUID:3354:1862456998017152:0@lo:6:osp-syn-0-1.0 00000100:00100000:1.0:1776178092.566952:0:3354:0:(events.c:361:request_in_callback()) peer: 12345-0@lo (source: 12345-0@lo) 00000100:00100000:0.0:1776178092.567729:0:20754:0:(nrs_fifo.c:179:nrs_fifo_req_get()) NRS start fifo request from 12345-0@lo, seq: 5213 00000100:00100000:0.0:1776178092.567734:0:20754:0:(service.c:2324:ptlrpc_server_handle_request()) Handling RPC req@00000000b5702a7a pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_005:lustre-MDT0001-mdtlov_UUID+41:3354:x1862456998017152:12345-0@lo:6:osp-syn-0-1.0 00002000:00080000:0.0:1776178092.567750:0:20754:0:(ofd_dev.c:1822:ofd_destroy_hdl()) lustre-OST0000: Destroy object 0x280000400:941 count 1 00010000:00010000:0.0:1776178092.567766:0:20754:0:(ldlm_lock.c:766:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0000_UUID lock: 000000000b903578/0xa40083ab9bf52dd1 lrc: 3/0,1 mode: --/PW res: [0x280000400:0x3ad:0x0].0x0 rrc: 3 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 20754 timeout: 0 lvb_type: 0 00010000:00010000:0.0:1776178092.567780:0:20754:0:(ldlm_lock.c:675:ldlm_add_bl_work_item()) ### lock incompatible; sending blocking AST. ns: filter-lustre-OST0000_UUID lock: 00000000a6b36e6d/0xa40083ab9bf3bcec lrc: 2/0,0 mode: PR/PR res: [0x280000400:0x3ad:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x40000000000000 nid: 192.168.204.3@tcp remote: 0x771f68aa1eb25968 expref: 21 pid: 8645 timeout: 0 lvb_type: 1 00010000:00010000:0.0:1776178092.567786:0:20754:0:(ldlm_extent.c:318:ldlm_check_contention()) contended locks = 0 00010000:00010000:0.0:1776178092.567788:0:20754:0:(ldlm_extent.c:318:ldlm_check_contention()) contended locks = 0 00010000:00010000:0.0:1776178092.567789:0:20754:0:(ldlm_resource.c:1650:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 000000000b903578/0xa40083ab9bf52dd1 lrc: 4/0,1 mode: --/PW res: [0x280000400:0x3ad:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x50010080000000 nid: local remote: 0x0 expref: -99 pid: 20754 timeout: 0 lvb_type: 0 00010000:00010000:0.0:1776178092.567808:0:20754:0:(ldlm_lockd.c:953:ldlm_server_blocking_ast()) ### server preparing blocking AST ns: filter-lustre-OST0000_UUID lock: 00000000a6b36e6d/0xa40083ab9bf3bcec lrc: 3/0,0 mode: PR/PR res: [0x280000400:0x3ad:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x50000000010020 nid: 192.168.204.3@tcp remote: 0x771f68aa1eb25968 expref: 21 pid: 8645 timeout: 0 lvb_type: 1 00010000:00010000:0.0:1776178092.567817:0:20754:0:(ldlm_lockd.c:489:ldlm_add_waiting_lock()) ### adding to wait list(timeout: 100, AT: on) ns: filter-lustre-OST0000_UUID lock: 00000000a6b36e6d/0xa40083ab9bf3bcec lrc: 4/0,0 mode: PR/PR res: [0x280000400:0x3ad:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x70000400010020 nid: 192.168.204.3@tcp remote: 0x771f68aa1eb25968 expref: 21 pid: 8645 timeout: 899 lvb_type: 1 00000100:00100000:0.0:1776178092.567837:0:20754:0:(client.c:1733:ptlrpc_send_new_req()) Sending RPC req@00000000c1934814 pname:cluuid:pid:xid:nid:opc:job ll_ost00_005:lustre-OST0000_UUID:20754:1862456998017280:192.168.204.3@tcp:104: 00000100:00100000:0.0:1776178092.567862:0:20754:0:(client.c:2500:ptlrpc_set_wait()) set 000000004d9ad1c9 going to sleep for 43 seconds 00000100:00100000:0.0:1776178092.588685:0:3347:0:(events.c:361:request_in_callback()) peer: 12345-192.168.204.3@tcp (source: 12345-192.168.204.3@tcp) 00000100:00100000:1.0:1776178092.589101:0:6243:0:(service.c:2152:ptlrpc_server_handle_req_in()) got req x1862456979073152 00010000:00010000:1.0:1776178092.589145:0:6243:0:(ldlm_lockd.c:2638:ldlm_cancel_hpreq_check()) ### hpreq cancel/convert lock ns: filter-lustre-OST0000_UUID lock: 00000000eb6c8a25/0xa40083ab9bf419f2 lrc: 4/0,0 mode: PR/PR res: [0x280000400:0x46a:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x60000400010020 nid: 192.168.204.3@tcp remote: 0x771f68aa1eb2786a expref: 22 pid: 8228 timeout: 899 lvb_type: 1 00000100:00100000:1.0:1776178092.589176:0:6243:0:(nrs_fifo.c:179:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.204.3@tcp, seq: 2042 00000100:00100000:1.0:1776178092.589183:0:6243:0:(service.c:2324:ptlrpc_server_handle_request()) Handling RPC req@0000000017d54e92 pname:cluuid+ref:pid:xid:nid:opc:job ldlm_cn00_002:b50963bd-8eba-4432-9a54-8888d00e3fa1+22:2270:x1862456979073152:12345-192.168.204.3@tcp:103:ldlm_bl_04.0 00010000:00010000:1.0:1776178092.589200:0:6243:0:(ldlm_lockd.c:1720:ldlm_request_cancel()) ### server-side cancel handler START: 1 locks, starting at 0 00010000:00010000:1.0:1776178092.589205:0:6243:0:(ldlm_lockd.c:1771:ldlm_request_cancel()) ### server cancels blocked lock after 0s ns: filter-lustre-OST0000_UUID lock: 00000000eb6c8a25/0xa40083ab9bf419f2 lrc: 4/0,0 mode: PR/PR res: [0x280000400:0x46a:0x0].0x0 rrc: 4 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x60000400010020 nid: 192.168.204.3@tcp remote: 0x771f68aa1eb2786a expref: 22 pid: 8228 timeout: 899 lvb_type: 1 00010000:00010000:1.0:1776178092.589216:0:6243:0:(ldlm_lockd.c:562:ldlm_del_waiting_lock()) ### removed ns: filter-lustre-OST0000_UUID lock: 00000000eb6c8a25/0xa40083ab9bf419f2 lrc: 3/0,0 mode: PR/PR res: [0x280000400:0x46a:0x0].0x0 rrc: 4 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x50000400010020 nid: 192.168.204.3@tcp remote: 0x771f68aa1eb2786a expref: 22 pid: 8228 timeout: 899 lvb_type: 1 00010000:00010000:1.0:1776178092.589240:0:6243:0:(ldlm_lock.c:216:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 00000000eb6c8a25/0xa40083ab9bf419f2 lrc: 0/0,0 mode: --/PR res: [0x280000400:0x46a:0x0].0x0 rrc: 4 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x44801400010020 nid: 192.168.204.3@tcp remote: 0x771f68aa1eb2786a expref: 22 pid: 8228 timeout: 899 lvb_type: 1 00010000:00010000:1.0:1776178092.589257:0:6243:0:(ldlm_extent.c:318:ldlm_check_contention()) contended locks = 0 00010000:00010000:1.0:1776178092.589259:0:6243:0:(ldlm_extent.c:318:ldlm_check_contention()) contended locks = 0 00010000:00010000:1.0:1776178092.589263:0:6243:0:(ldlm_lock.c:704:ldlm_add_cp_work_item()) ### lock granted; sending completion AST. ns: filter-lustre-OST0000_UUID lock: 000000002a990613/0xa40083ab9bf52d68 lrc: 3/0,1 mode: PW/PW res: [0x280000400:0x46a:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x40090080000000 nid: local remote: 0x0 expref: -99 pid: 20764 timeout: 0 lvb_type: 0 00010000:00010000:1.0:1776178092.589271:0:6243:0:(ldlm_resource.c:1650:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 000000002a990613/0xa40083ab9bf52d68 lrc: 4/0,1 mode: PW/PW res: [0x280000400:0x46a:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x40090080000000 nid: local remote: 0x0 expref: -99 pid: 20764 timeout: 0 lvb_type: 0 00010000:00010000:1.0:1776178092.589306:0:6243:0:(ldlm_lockd.c:1782:ldlm_request_cancel()) ### server-side cancel handler END 00010000:00010000:0.0:1776178092.589921:0:6244:0:(ldlm_lockd.c:2789:ldlm_bl_get_work()) Got 1 locks of 0 total in blp. (0 blwis in pool) 00010000:00010000:0.0:1776178092.589927:0:6244:0:(ldlm_lockd.c:1879:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 000000002a990613/0xa40083ab9bf52d68 lrc: 2/0,0 mode: PW/PW res: [0x280000400:0x46a:0x0].0x0 rrc: 2 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x40010480000000 nid: local remote: 0x0 expref: -99 pid: 20764 timeout: 0 lvb_type: 0 00010000:00010000:0.0:1776178092.589935:0:6244:0:(ldlm_lockd.c:1893:ldlm_handle_bl_callback()) Lock 000000002a990613 already unused, calling callback (0000000063c97258) 00010000:00010000:0.0:1776178092.589938:0:6244:0:(ldlm_request.c:364:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 000000002a990613/0xa40083ab9bf52d68 lrc: 2/0,0 mode: PW/PW res: [0x280000400:0x46a:0x0].0x0 rrc: 2 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x40010480000000 nid: local remote: 0x0 expref: -99 pid: 20764 timeout: 0 lvb_type: 0 00010000:00010000:0.0:1776178092.589946:0:6244:0:(ldlm_request.c:1257:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 000000002a990613/0xa40083ab9bf52d68 lrc: 3/0,0 mode: PW/PW res: [0x280000400:0x46a:0x0].0x0 rrc: 2 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x40018480000000 nid: local remote: 0x0 expref: -99 pid: 20764 timeout: 0 lvb_type: 0 00010000:00010000:0.0:1776178092.589960:0:6244:0:(ldlm_lockd.c:1903:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 000000002a990613/0xa40083ab9bf52d68 lrc: 1/0,0 mode: --/PW res: [0x280000400:0x46a:0x0].0x0 rrc: 2 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x44819480000000 nid: local remote: 0x0 expref: -99 pid: 20764 timeout: 0 lvb_type: 0 00010000:00010000:0.0:1776178092.589965:0:6244:0:(ldlm_lock.c:216:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 000000002a990613/0xa40083ab9bf52d68 lrc: 0/0,0 mode: --/PW res: [0x280000400:0x46a:0x0].0x0 rrc: 2 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x44819480000000 nid: local remote: 0x0 expref: -99 pid: 20764 timeout: 0 lvb_type: 0 00010000:00010000:0.0:1776178092.589982:0:6244:0:(ldlm_lockd.c:2792:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00010000:00010000:0.0:1776178092.589983:0:6244:0:(ldlm_lockd.c:2792:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00010000:00010000:0.0:1776178092.589984:0:6244:0:(ldlm_lockd.c:2792:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000004:00080000:0.0:1776178092.590026:0:3355:0:(osp_sync.c:562:osp_sync_interpret()) reply req 00000000fed3e037/2, rc 0, transno 2028 00000100:00100000:0.0:1776178092.590051:0:3355:0:(client.c:2215:ptlrpc_check_set()) Completed RPC req@00000000fed3e037 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_02:lustre-MDT0001-mdtlov_UUID:3355:1862456998014528:0@lo:6:osp-syn-0-1.0 00000100:00100000:0.0:1776178092.590569:0:20764:0:(service.c:2152:ptlrpc_server_handle_req_in()) got req x1862456998017344 00000100:00100000:0.0:1776178092.590603:0:20764:0:(nrs_fifo.c:179:nrs_fifo_req_get()) NRS start fifo request from 12345-0@lo, seq: 5214 00000100:00100000:0.0:1776178092.590607:0:20764:0:(service.c:2324:ptlrpc_server_handle_request()) Handling RPC req@0000000003cebab2 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_015:lustre-MDT0001-mdtlov_UUID+42:3353:x1862456998017344:12345-0@lo:6:osp-syn-0-1.0 00002000:00080000:0.0:1776178092.590630:0:20764:0:(ofd_dev.c:1822:ofd_destroy_hdl()) lustre-OST0000: Destroy object 0x280000400:922 count 1 00010000:00010000:0.0:1776178092.590652:0:20764:0:(ldlm_lock.c:766:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0000_UUID lock: 0000000015ae4ee6/0xa40083ab9bf52dd8 lrc: 3/0,1 mode: --/PW res: [0x280000400:0x39a:0x0].0x0 rrc: 3 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 20764 timeout: 0 lvb_type: 0 00010000:00010000:0.0:1776178092.590667:0:20764:0:(ldlm_lock.c:675:ldlm_add_bl_work_item()) ### lock incompatible; sending blocking AST. ns: filter-lustre-OST0000_UUID lock: 00000000d8929059/0xa40083ab9bf3b392 lrc: 2/0,0 mode: PR/PR res: [0x280000400:0x39a:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x40000000000000 nid: 192.168.204.3@tcp remote: 0x771f68aa1eb2564a expref: 21 pid: 8645 timeout: 0 lvb_type: 1 00010000:00010000:0.0:1776178092.590674:0:20764:0:(ldlm_extent.c:318:ldlm_check_contention()) contended locks = 0 00010000:00010000:0.0:1776178092.590676:0:20764:0:(ldlm_extent.c:318:ldlm_check_contention()) contended locks = 0 00010000:00010000:0.0:1776178092.590677:0:20764:0:(ldlm_resource.c:1650:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 0000000015ae4ee6/0xa40083ab9bf52dd8 lrc: 4/0,1 mode: --/PW res: [0x280000400:0x39a:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x50010080000000 nid: local remote: 0x0 expref: -99 pid: 20764 timeout: 0 lvb_type: 0 00010000:00010000:0.0:1776178092.590698:0:20764:0:(ldlm_lockd.c:953:ldlm_server_blocking_ast()) ### server preparing blocking AST ns: filter-lustre-OST0000_UUID lock: 00000000d8929059/0xa40083ab9bf3b392 lrc: 3/0,0 mode: PR/PR res: [0x280000400:0x39a:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x50000000010020 nid: 192.168.204.3@tcp remote: 0x771f68aa1eb2564a expref: 21 pid: 8645 timeout: 0 lvb_type: 1 00010000:00010000:0.0:1776178092.590708:0:20764:0:(ldlm_lockd.c:489:ldlm_add_waiting_lock()) ### adding to wait list(timeout: 100, AT: on) ns: filter-lustre-OST0000_UUID lock: 00000000d8929059/0xa40083ab9bf3b392 lrc: 4/0,0 mode: PR/PR res: [0x280000400:0x39a:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x70000400010020 nid: 192.168.204.3@tcp remote: 0x771f68aa1eb2564a expref: 21 pid: 8645 timeout: 899 lvb_type: 1 00000100:00100000:0.0:1776178092.590717:0:20764:0:(client.c:1733:ptlrpc_send_new_req()) Sending RPC req@000000009d117b34 pname:cluuid:pid:xid:nid:opc:job ll_ost00_015:lustre-OST0000_UUID:20764:1862456998017408:192.168.204.3@tcp:104: 00000100:00100000:1.0:1776178092.591206:0:6243:0:(service.c:2372:ptlrpc_server_handle_request()) Handled RPC req@0000000017d54e92 pname:cluuid+ref:pid:xid:nid:opc:job ldlm_cn00_002:b50963bd-8eba-4432-9a54-8888d00e3fa1+21:2270:x1862456979073152:12345-192.168.204.3@tcp:103:ldlm_bl_04.0 Request processed in 2020us (2526us total) trans 0 rc 0/0 00000100:00100000:1.0:1776178092.591217:0:6243:0:(nrs_fifo.c:241:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.204.3@tcp, seq: 2042 00000100:00100000:0.0:1776178092.591914:0:20764:0:(client.c:2500:ptlrpc_set_wait()) set 00000000be3d70c9 going to sleep for 6 seconds 00000100:00100000:0.0:1776178092.595796:0:3346:0:(events.c:361:request_in_callback()) peer: 12345-192.168.204.3@tcp (source: 12345-192.168.204.3@tcp) 00000100:00100000:0.0:1776178092.595821:0:3346:0:(events.c:361:request_in_callback()) peer: 12345-192.168.204.3@tcp (source: 12345-192.168.204.3@tcp) 00000100:00100000:0.0:1776178092.595854:0:3346:0:(events.c:361:request_in_callback()) peer: 12345-192.168.204.3@tcp (source: 12345-192.168.204.3@tcp) 00000100:00100000:0.0:1776178092.595872:0:3346:0:(events.c:361:request_in_callback()) peer: 12345-192.168.204.3@tcp (source: 12345-192.168.204.3@tcp) 00000100:00100000:0.0:1776178092.595897:0:3346:0:(events.c:361:request_in_callback()) peer: 12345-192.168.204.3@tcp (source: 12345-192.168.204.3@tcp) 00000100:00100000:0.0:1776178092.595915:0:3346:0:(events.c:361:request_in_callback()) peer: 12345-192.168.204.3@tcp (source: 12345-192.168.204.3@tcp) 00000100:00100000:0.0:1776178092.595934:0:3346:0:(events.c:361:request_in_callback()) peer: 12345-192.168.204.3@tcp (source: 12345-192.168.204.3@tcp) 00000100:00100000:0.0:1776178092.595946:0:3346:0:(events.c:361:request_in_callback()) peer: 12345-192.168.204.3@tcp (source: 12345-192.168.204.3@tcp) 00000100:00100000:0.0:1776178092.595958:0:3346:0:(events.c:361:request_in_callback()) peer: 12345-192.168.204.3@tcp (source: 12345-192.168.204.3@tcp) 00000100:00100000:0.0:1776178092.595970:0:3346:0:(events.c:361:request_in_callback()) peer: 12345-192.168.204.3@tcp (source: 12345-192.168.204.3@tcp) 00000100:00100000:0.0:1776178092.595982:0:3346:0:(events.c:361:request_in_callback()) peer: 12345-192.168.204.3@tcp (source: 12345-192.168.204.3@tcp) 00000100:00100000:0.0:1776178092.595995:0:3346:0:(events.c:361:request_in_callback()) peer: 12345-192.168.204.3@tcp (source: 12345-192.168.204.3@tcp) 00000100:00100000:0.0:1776178092.596008:0:3346:0:(events.c:361:request_in_callback()) peer: 12345-192.168.204.3@tcp (source: 12345-192.168.204.3@tcp) 00000100:00100000:0.0:1776178092.596027:0:3346:0:(events.c:361:request_in_callback()) peer: 12345-192.168.204.3@tcp (source: 12345-192.168.204.3@tcp) 00000100:00100000:0.0:1776178092.596105:0:3346:0:(events.c:361:request_in_callback()) peer: 12345-192.168.204.3@tcp (source: 12345-192.168.204.3@tcp) 00000100:00100000:0.0:1776178092.596149:0:7252:0:(service.c:2152:ptlrpc_server_handle_req_in()) got req x1862456979073216 00010000:00010000:0.0:1776178092.596194:0:7252:0:(ldlm_lockd.c:2638:ldlm_cancel_hpreq_check()) ### hpreq cancel/convert lock ns: filter-lustre-OST0000_UUID lock: 000000006b1e294a/0xa40083ab9bf367c6 lrc: 4/0,0 mode: PR/PR res: [0x280000400:0x300:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x60000400010020 nid: 192.168.204.3@tcp remote: 0x771f68aa1eb23d06 expref: 21 pid: 8227 timeout: 899 lvb_type: 1 00000100:00100000:0.0:1776178092.596223:0:7252:0:(service.c:2152:ptlrpc_server_handle_req_in()) got req x1862456979073344 00010000:00010000:0.0:1776178092.596235:0:7252:0:(ldlm_lockd.c:2638:ldlm_cancel_hpreq_check()) ### hpreq cancel/convert lock ns: filter-lustre-OST0001_UUID lock: 00000000e2bfb5b3/0xa40083ab9bf32bf9 lrc: 5/0,0 mode: PR/PR res: [0x2c0000400:0x247:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x60000400010020 nid: 192.168.204.3@tcp remote: 0x771f68aa1eb22917 expref: 19 pid: 8645 timeout: 899 lvb_type: 1 00000100:00100000:0.0:1776178092.596246:0:7252:0:(service.c:2152:ptlrpc_server_handle_req_in()) got req x1862456979073472 00010000:00010000:0.0:1776178092.596254:0:7252:0:(ldlm_lockd.c:2638:ldlm_cancel_hpreq_check()) ### hpreq cancel/convert lock ns: filter-lustre-OST0001_UUID lock: 000000008acf7024/0xa40083ab9bf3f643 lrc: 5/0,0 mode: PR/PR res: [0x2c0000400:0x3e2:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x60000400010020 nid: 192.168.204.3@tcp remote: 0x771f68aa1eb26c85 expref: 20 pid: 14440 timeout: 899 lvb_type: 1 00000100:00100000:0.0:1776178092.596264:0:7252:0:(service.c:2152:ptlrpc_server_handle_req_in()) got req x1862456979073600 00010000:00010000:0.0:1776178092.596273:0:7252:0:(ldlm_lockd.c:2638:ldlm_cancel_hpreq_check()) ### hpreq cancel/convert lock ns: filter-lustre-OST0000_UUID lock: 0000000043c27e7c/0xa40083ab9bf39dea lrc: 5/0,0 mode: PR/PR res: [0x280000400:0x36e:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x60000400010020 nid: 192.168.204.3@tcp remote: 0x771f68aa1eb24f12 expref: 22 pid: 8645 timeout: 899 lvb_type: 1 00000100:00100000:0.0:1776178092.596288:0:7252:0:(service.c:2152:ptlrpc_server_handle_req_in()) got req x1862456979073728 00010000:00010000:0.0:1776178092.596295:0:7252:0:(ldlm_lockd.c:2638:ldlm_cancel_hpreq_check()) ### hpreq cancel/convert lock ns: filter-lustre-OST0001_UUID lock: 0000000072dc9eba/0xa40083ab9bf3b3d1 lrc: 5/0,0 mode: PR/PR res: [0x2c0000400:0x35b:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x60000400010020 nid: 192.168.204.3@tcp remote: 0x771f68aa1eb2565f expref: 21 pid: 8645 timeout: 899 lvb_type: 1 00000100:00100000:0.0:1776178092.596305:0:7252:0:(service.c:2152:ptlrpc_server_handle_req_in()) got req x1862456979073856 00010000:00010000:0.0:1776178092.596312:0:7252:0:(ldlm_lockd.c:2638:ldlm_cancel_hpreq_check()) ### hpreq cancel/convert lock ns: filter-lustre-OST0000_UUID lock: 00000000e59f0070/0xa40083ab9bf35512 lrc: 5/0,0 mode: PR/PR res: [0x280000400:0x2da:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x60000400010020 nid: 192.168.204.3@tcp remote: 0x771f68aa1eb236ca expref: 23 pid: 8228 timeout: 899 lvb_type: 1 00000100:00100000:0.0:1776178092.596322:0:7252:0:(service.c:2152:ptlrpc_server_handle_req_in()) got req x1862456979073984 00010000:00010000:0.0:1776178092.596330:0:7252:0:(ldlm_lockd.c:2638:ldlm_cancel_hpreq_check()) ### hpreq cancel/convert lock ns: filter-lustre-OST0000_UUID lock: 00000000485847f3/0xa40083ab9bf32b3c lrc: 5/0,0 mode: PR/PR res: [0x280000400:0x285:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x60000400010020 nid: 192.168.204.3@tcp remote: 0x771f68aa1eb228d8 expref: 24 pid: 8645 timeout: 899 lvb_type: 1 00000100:00100000:0.0:1776178092.596340:0:7252:0:(service.c:2152:ptlrpc_server_handle_req_in()) got req x1862456979074112 00010000:00010000:0.0:1776178092.596349:0:7252:0:(ldlm_lockd.c:2638:ldlm_cancel_hpreq_check()) ### hpreq cancel/convert lock ns: filter-lustre-OST0000_UUID lock: 00000000a6b36e6d/0xa40083ab9bf3bcec lrc: 5/0,0 mode: PR/PR res: [0x280000400:0x3ad:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x60000400010020 nid: 192.168.204.3@tcp remote: 0x771f68aa1eb25968 expref: 25 pid: 8645 timeout: 899 lvb_type: 1 00000100:00100000:0.0:1776178092.596359:0:7252:0:(service.c:2152:ptlrpc_server_handle_req_in()) got req x1862456979073280 00010000:00010000:0.0:1776178092.596369:0:7252:0:(ldlm_lockd.c:2638:ldlm_cancel_hpreq_check()) ### hpreq cancel/convert lock ns: filter-lustre-OST0000_UUID lock: 0000000012177dd1/0xa40083ab9bf33bf8 lrc: 5/0,0 mode: PR/PR res: [0x280000400:0x2a7:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x60000400010020 nid: 192.168.204.3@tcp remote: 0x771f68aa1eb22e6c expref: 26 pid: 8228 timeout: 899 lvb_type: 1 00000100:00100000:0.0:1776178092.596379:0:7252:0:(service.c:2152:ptlrpc_server_handle_req_in()) got req x1862456979073408 00010000:00010000:0.0:1776178092.596387:0:7252:0:(ldlm_lockd.c:2638:ldlm_cancel_hpreq_check()) ### hpreq cancel/convert lock ns: filter-lustre-OST0001_UUID lock: 000000003ed46f13/0xa40083ab9bf36901 lrc: 5/0,0 mode: PR/PR res: [0x2c0000400:0x2c3:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x60000400010020 nid: 192.168.204.3@tcp remote: 0x771f68aa1eb23d6f expref: 22 pid: 8645 timeout: 899 lvb_type: 1 00000100:00100000:0.0:1776178092.596398:0:7252:0:(service.c:2152:ptlrpc_server_handle_req_in()) got req x1862456979073536 00010000:00010000:0.0:1776178092.596405:0:7252:0:(ldlm_lockd.c:2638:ldlm_cancel_hpreq_check()) ### hpreq cancel/convert lock ns: filter-lustre-OST0001_UUID lock: 0000000076588d9d/0xa40083ab9bf36787 lrc: 5/0,0 mode: PR/PR res: [0x2c0000400:0x2c0:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x60000400010020 nid: 192.168.204.3@tcp remote: 0x771f68aa1eb23cf1 expref: 23 pid: 8227 timeout: 899 lvb_type: 1 00000100:00100000:0.0:1776178092.596420:0:7252:0:(service.c:2152:ptlrpc_server_handle_req_in()) got req x1862456979073664 00010000:00010000:0.0:1776178092.596428:0:7252:0:(ldlm_lockd.c:2638:ldlm_cancel_hpreq_check()) ### hpreq cancel/convert lock ns: filter-lustre-OST0001_UUID lock: 00000000adf1bb1a/0xa40083ab9bf38deb lrc: 5/0,0 mode: PR/PR res: [0x2c0000400:0x30e:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x60000400010020 nid: 192.168.204.3@tcp remote: 0x771f68aa1eb249bd expref: 24 pid: 8227 timeout: 899 lvb_type: 1 00000100:00100000:0.0:1776178092.596437:0:7252:0:(service.c:2152:ptlrpc_server_handle_req_in()) got req x1862456979073792 00010000:00010000:0.0:1776178092.596443:0:7252:0:(ldlm_lockd.c:2638:ldlm_cancel_hpreq_check()) ### hpreq cancel/convert lock ns: filter-lustre-OST0000_UUID lock: 0000000006d4cafa/0xa40083ab9bf3768a lrc: 5/0,0 mode: PR/PR res: [0x280000400:0x31e:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x60000400010020 nid: 192.168.204.3@tcp remote: 0x771f68aa1eb241f2 expref: 27 pid: 14440 timeout: 899 lvb_type: 1 00000100:00100000:0.0:1776178092.596452:0:7252:0:(service.c:2152:ptlrpc_server_handle_req_in()) got req x1862456979073920 00010000:00010000:0.0:1776178092.596459:0:7252:0:(ldlm_lockd.c:2638:ldlm_cancel_hpreq_check()) ### hpreq cancel/convert lock ns: filter-lustre-OST0001_UUID lock: 00000000344b695c/0xa40083ab9bf33db1 lrc: 5/0,0 mode: PR/PR res: [0x2c0000400:0x26b:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x60000400010020 nid: 192.168.204.3@tcp remote: 0x771f68aa1eb22eff expref: 25 pid: 8228 timeout: 899 lvb_type: 1 00000100:00100000:0.0:1776178092.596469:0:7252:0:(service.c:2152:ptlrpc_server_handle_req_in()) got req x1862456979074048 00010000:00010000:0.0:1776178092.596476:0:7252:0:(ldlm_lockd.c:2638:ldlm_cancel_hpreq_check()) ### hpreq cancel/convert lock ns: filter-lustre-OST0001_UUID lock: 000000007adb6341/0xa40083ab9bf38af7 lrc: 5/0,0 mode: PR/PR res: [0x2c0000400:0x308:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x60000400010020 nid: 192.168.204.3@tcp remote: 0x771f68aa1eb248c1 expref: 26 pid: 8645 timeout: 899 lvb_type: 1 00000100:00100000:0.0:1776178092.596489:0:7252:0:(nrs_fifo.c:179:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.204.3@tcp, seq: 2043 00000100:00100000:0.0:1776178092.596496:0:7252:0:(service.c:2324:ptlrpc_server_handle_request()) Handling RPC req@00000000f720f286 pname:cluuid+ref:pid:xid:nid:opc:job ldlm_cn00_003:b50963bd-8eba-4432-9a54-8888d00e3fa1+27:2271:x1862456979073216:12345-192.168.204.3@tcp:103:ldlm_bl_03.0 00010000:00010000:0.0:1776178092.596510:0:7252:0:(ldlm_lockd.c:1720:ldlm_request_cancel()) ### server-side cancel handler START: 1 locks, starting at 0 00010000:00010000:0.0:1776178092.596515:0:7252:0:(ldlm_lockd.c:1771:ldlm_request_cancel()) ### server cancels blocked lock after 0s ns: filter-lustre-OST0000_UUID lock: 000000006b1e294a/0xa40083ab9bf367c6 lrc: 4/0,0 mode: PR/PR res: [0x280000400:0x300:0x0].0x0 rrc: 4 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x60000400010020 nid: 192.168.204.3@tcp remote: 0x771f68aa1eb23d06 expref: 27 pid: 8227 timeout: 899 lvb_type: 1 00010000:00010000:0.0:1776178092.596523:0:7252:0:(ldlm_lockd.c:562:ldlm_del_waiting_lock()) ### removed ns: filter-lustre-OST0000_UUID lock: 000000006b1e294a/0xa40083ab9bf367c6 lrc: 3/0,0 mode: PR/PR res: [0x280000400:0x300:0x0].0x0 rrc: 4 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x50000400010020 nid: 192.168.204.3@tcp remote: 0x771f68aa1eb23d06 expref: 27 pid: 8227 timeout: 899 lvb_type: 1 00010000:00010000:0.0:1776178092.596548:0:7252:0:(ldlm_lock.c:216:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 000000006b1e294a/0xa40083ab9bf367c6 lrc: 0/0,0 mode: --/PR res: [0x280000400:0x300:0x0].0x0 rrc: 4 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x44801400010020 nid: 192.168.204.3@tcp remote: 0x771f68aa1eb23d06 expref: 27 pid: 8227 timeout: 899 lvb_type: 1 00010000:00010000:0.0:1776178092.596566:0:7252:0:(ldlm_extent.c:318:ldlm_check_contention()) contended locks = 0 00010000:00010000:0.0:1776178092.596568:0:7252:0:(ldlm_extent.c:318:ldlm_check_contention()) contended locks = 0 00010000:00010000:0.0:1776178092.596572:0:7252:0:(ldlm_lock.c:704:ldlm_add_cp_work_item()) ### lock granted; sending completion AST. ns: filter-lustre-OST0000_UUID lock: 000000000396420a/0xa40083ab9bf52da0 lrc: 3/0,1 mode: PW/PW res: [0x280000400:0x300:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x40090080000000 nid: local remote: 0x0 expref: -99 pid: 20760 timeout: 0 lvb_type: 0 00010000:00010000:0.0:1776178092.596581:0:7252:0:(ldlm_resource.c:1650:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 000000000396420a/0xa40083ab9bf52da0 lrc: 4/0,1 mode: PW/PW res: [0x280000400:0x300:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x40090080000000 nid: local remote: 0x0 expref: -99 pid: 20760 timeout: 0 lvb_type: 0 00010000:00010000:0.0:1776178092.596610:0:7252:0:(ldlm_lockd.c:1782:ldlm_request_cancel()) ### server-side cancel handler END 00000100:00100000:0.0:1776178092.596657:0:7252:0:(service.c:2372:ptlrpc_server_handle_request()) Handled RPC req@00000000f720f286 pname:cluuid+ref:pid:xid:nid:opc:job ldlm_cn00_003:b50963bd-8eba-4432-9a54-8888d00e3fa1+26:2271:x1862456979073216:12345-192.168.204.3@tcp:103:ldlm_bl_03.0 Request processed in 160us (874us total) trans 0 rc 0/0 00000100:00100000:0.0:1776178092.596665:0:7252:0:(nrs_fifo.c:241:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.204.3@tcp, seq: 2043 00000100:00100000:0.0:1776178092.596681:0:7252:0:(nrs_fifo.c:179:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.204.3@tcp, seq: 2044 00000100:00100000:0.0:1776178092.596685:0:7252:0:(service.c:2324:ptlrpc_server_handle_request()) Handling RPC req@00000000a770fd7b pname:cluuid+ref:pid:xid:nid:opc:job ldlm_cn00_003:b50963bd-8eba-4432-9a54-8888d00e3fa1+26:2271:x1862456979073344:12345-192.168.204.3@tcp:103:ldlm_bl_03.0 00010000:00010000:0.0:1776178092.596693:0:7252:0:(ldlm_lockd.c:1720:ldlm_request_cancel()) ### server-side cancel handler START: 1 locks, starting at 0 00010000:00010000:0.0:1776178092.596697:0:7252:0:(ldlm_lockd.c:1771:ldlm_request_cancel()) ### server cancels blocked lock after 0s ns: filter-lustre-OST0001_UUID lock: 00000000e2bfb5b3/0xa40083ab9bf32bf9 lrc: 5/0,0 mode: PR/PR res: [0x2c0000400:0x247:0x0].0x0 rrc: 4 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x60000400010020 nid: 192.168.204.3@tcp remote: 0x771f68aa1eb22917 expref: 26 pid: 8645 timeout: 899 lvb_type: 1 00010000:00010000:0.0:1776178092.596705:0:7252:0:(ldlm_lockd.c:562:ldlm_del_waiting_lock()) ### removed ns: filter-lustre-OST0001_UUID lock: 00000000e2bfb5b3/0xa40083ab9bf32bf9 lrc: 4/0,0 mode: PR/PR res: [0x2c0000400:0x247:0x0].0x0 rrc: 4 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x50000400010020 nid: 192.168.204.3@tcp remote: 0x771f68aa1eb22917 expref: 26 pid: 8645 timeout: 899 lvb_type: 1 00010000:00010000:0.0:1776178092.596719:0:7252:0:(ldlm_extent.c:318:ldlm_check_contention()) contended locks = 0 00010000:00010000:0.0:1776178092.596721:0:7252:0:(ldlm_extent.c:318:ldlm_check_contention()) contended locks = 0 00010000:00010000:0.0:1776178092.596722:0:7252:0:(ldlm_lock.c:704:ldlm_add_cp_work_item()) ### lock granted; sending completion AST. ns: filter-lustre-OST0001_UUID lock: 000000007ee917b5/0xa40083ab9bf52dae lrc: 3/0,1 mode: PW/PW res: [0x2c0000400:0x247:0x0].0x0 rrc: 4 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x50090080000000 nid: local remote: 0x0 expref: -99 pid: 8227 timeout: 0 lvb_type: 0 00010000:00010000:0.0:1776178092.596733:0:7252:0:(ldlm_resource.c:1650:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0001_UUID lock: 000000007ee917b5/0xa40083ab9bf52dae lrc: 4/0,1 mode: PW/PW res: [0x2c0000400:0x247:0x0].0x0 rrc: 4 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x50090080000000 nid: local remote: 0x0 expref: -99 pid: 8227 timeout: 0 lvb_type: 0 00010000:00010000:0.0:1776178092.596743:0:7252:0:(ldlm_lockd.c:1782:ldlm_request_cancel()) ### server-side cancel handler END 00000100:00100000:0.0:1776178092.596758:0:7252:0:(service.c:2372:ptlrpc_server_handle_request()) Handled RPC req@00000000a770fd7b pname:cluuid+ref:pid:xid:nid:opc:job ldlm_cn00_003:b50963bd-8eba-4432-9a54-8888d00e3fa1+26:2271:x1862456979073344:12345-192.168.204.3@tcp:103:ldlm_bl_03.0 Request processed in 72us (937us total) trans 0 rc 0/0 00000100:00100000:0.0:1776178092.596764:0:7252:0:(nrs_fifo.c:241:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.204.3@tcp, seq: 2044 00000100:00100000:0.0:1776178092.596776:0:7252:0:(nrs_fifo.c:179:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.204.3@tcp, seq: 2045 00000100:00100000:0.0:1776178092.596779:0:7252:0:(service.c:2324:ptlrpc_server_handle_request()) Handling RPC req@0000000053234520 pname:cluuid+ref:pid:xid:nid:opc:job ldlm_cn00_003:b50963bd-8eba-4432-9a54-8888d00e3fa1+25:2271:x1862456979073472:12345-192.168.204.3@tcp:103:ldlm_bl_03.0 00010000:00010000:0.0:1776178092.596787:0:7252:0:(ldlm_lockd.c:1720:ldlm_request_cancel()) ### server-side cancel handler START: 1 locks, starting at 0 00010000:00010000:0.0:1776178092.596790:0:7252:0:(ldlm_lockd.c:1771:ldlm_request_cancel()) ### server cancels blocked lock after 0s ns: filter-lustre-OST0001_UUID lock: 000000008acf7024/0xa40083ab9bf3f643 lrc: 5/0,0 mode: PR/PR res: [0x2c0000400:0x3e2:0x0].0x0 rrc: 4 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x60000400010020 nid: 192.168.204.3@tcp remote: 0x771f68aa1eb26c85 expref: 25 pid: 14440 timeout: 899 lvb_type: 1 00010000:00010000:0.0:1776178092.596797:0:7252:0:(ldlm_lockd.c:562:ldlm_del_waiting_lock()) ### removed ns: filter-lustre-OST0001_UUID lock: 000000008acf7024/0xa40083ab9bf3f643 lrc: 4/0,0 mode: PR/PR res: [0x2c0000400:0x3e2:0x0].0x0 rrc: 4 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x50000400010020 nid: 192.168.204.3@tcp remote: 0x771f68aa1eb26c85 expref: 25 pid: 14440 timeout: 899 lvb_type: 1 00010000:00010000:0.0:1776178092.596809:0:7252:0:(ldlm_extent.c:318:ldlm_check_contention()) contended locks = 0 00010000:00010000:0.0:1776178092.596810:0:7252:0:(ldlm_extent.c:318:ldlm_check_contention()) contended locks = 0 00010000:00010000:0.0:1776178092.596813:0:7252:0:(ldlm_lock.c:704:ldlm_add_cp_work_item()) ### lock granted; sending completion AST. ns: filter-lustre-OST0001_UUID lock: 000000006cc9c357/0xa40083ab9bf52d76 lrc: 3/0,1 mode: PW/PW res: [0x2c0000400:0x3e2:0x0].0x0 rrc: 4 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x50090080000000 nid: local remote: 0x0 expref: -99 pid: 20755 timeout: 0 lvb_type: 0 00010000:00010000:0.0:1776178092.596818:0:7252:0:(ldlm_resource.c:1650:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0001_UUID lock: 000000006cc9c357/0xa40083ab9bf52d76 lrc: 4/0,1 mode: PW/PW res: [0x2c0000400:0x3e2:0x0].0x0 rrc: 4 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x50090080000000 nid: local remote: 0x0 expref: -99 pid: 20755 timeout: 0 lvb_type: 0 00010000:00010000:0.0:1776178092.596839:0:7252:0:(ldlm_lockd.c:1782:ldlm_request_cancel()) ### server-side cancel handler END 00000100:00100000:0.0:1776178092.596854:0:7252:0:(service.c:2372:ptlrpc_server_handle_request()) Handled RPC req@0000000053234520 pname:cluuid+ref:pid:xid:nid:opc:job ldlm_cn00_003:b50963bd-8eba-4432-9a54-8888d00e3fa1+25:2271:x1862456979073472:12345-192.168.204.3@tcp:103:ldlm_bl_03.0 Request processed in 74us (1000us total) trans 0 rc 0/0 00000100:00100000:0.0:1776178092.596861:0:7252:0:(nrs_fifo.c:241:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.204.3@tcp, seq: 2045 00000100:00100000:0.0:1776178092.596872:0:7252:0:(nrs_fifo.c:179:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.204.3@tcp, seq: 2046 00000100:00100000:0.0:1776178092.596876:0:7252:0:(service.c:2324:ptlrpc_server_handle_request()) Handling RPC req@0000000018b5e3c6 pname:cluuid+ref:pid:xid:nid:opc:job ldlm_cn00_003:b50963bd-8eba-4432-9a54-8888d00e3fa1+25:2271:x1862456979073600:12345-192.168.204.3@tcp:103:ldlm_bl_03.0 00010000:00010000:0.0:1776178092.596883:0:7252:0:(ldlm_lockd.c:1720:ldlm_request_cancel()) ### server-side cancel handler START: 1 locks, starting at 0 00010000:00010000:0.0:1776178092.596887:0:7252:0:(ldlm_lockd.c:1771:ldlm_request_cancel()) ### server cancels blocked lock after 0s ns: filter-lustre-OST0000_UUID lock: 0000000043c27e7c/0xa40083ab9bf39dea lrc: 5/0,0 mode: PR/PR res: [0x280000400:0x36e:0x0].0x0 rrc: 4 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x60000400010020 nid: 192.168.204.3@tcp remote: 0x771f68aa1eb24f12 expref: 25 pid: 8645 timeout: 899 lvb_type: 1 00010000:00010000:0.0:1776178092.596895:0:7252:0:(ldlm_lockd.c:562:ldlm_del_waiting_lock()) ### removed ns: filter-lustre-OST0000_UUID lock: 0000000043c27e7c/0xa40083ab9bf39dea lrc: 4/0,0 mode: PR/PR res: [0x280000400:0x36e:0x0].0x0 rrc: 4 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x50000400010020 nid: 192.168.204.3@tcp remote: 0x771f68aa1eb24f12 expref: 25 pid: 8645 timeout: 899 lvb_type: 1 00010000:00010000:0.0:1776178092.596907:0:7252:0:(ldlm_extent.c:318:ldlm_check_contention()) contended locks = 0 00010000:00010000:0.0:1776178092.596909:0:7252:0:(ldlm_extent.c:318:ldlm_check_contention()) contended locks = 0 00010000:00010000:0.0:1776178092.596911:0:7252:0:(ldlm_lock.c:704:ldlm_add_cp_work_item()) ### lock granted; sending completion AST. ns: filter-lustre-OST0000_UUID lock: 000000007e7ea26f/0xa40083ab9bf52d7d lrc: 3/0,1 mode: PW/PW res: [0x280000400:0x36e:0x0].0x0 rrc: 4 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x50090080000000 nid: local remote: 0x0 expref: -99 pid: 8230 timeout: 0 lvb_type: 0 00010000:00010000:0.0:1776178092.596917:0:7252:0:(ldlm_resource.c:1650:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 000000007e7ea26f/0xa40083ab9bf52d7d lrc: 4/0,1 mode: PW/PW res: [0x280000400:0x36e:0x0].0x0 rrc: 4 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x50090080000000 nid: local remote: 0x0 expref: -99 pid: 8230 timeout: 0 lvb_type: 0 00000100:00100000:1.0:1776178092.598296:0:20767:0:(nrs_fifo.c:179:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.204.3@tcp, seq: 2047 00000100:00100000:1.0:1776178092.598304:0:20767:0:(service.c:2324:ptlrpc_server_handle_request()) Handling RPC req@000000004eb4fa98 pname:cluuid+ref:pid:xid:nid:opc:job ldlm_cn00_004:b50963bd-8eba-4432-9a54-8888d00e3fa1+23:2271:x1862456979073728:12345-192.168.204.3@tcp:103:ldlm_bl_03.0 00010000:00010000:1.0:1776178092.598318:0:20767:0:(ldlm_lockd.c:1720:ldlm_request_cancel()) ### server-side cancel handler START: 1 locks, starting at 0 00010000:00010000:1.0:1776178092.598324:0:20767:0:(ldlm_lockd.c:1771:ldlm_request_cancel()) ### server cancels blocked lock after 0s ns: filter-lustre-OST0001_UUID lock: 0000000072dc9eba/0xa40083ab9bf3b3d1 lrc: 5/0,0 mode: PR/PR res: [0x2c0000400:0x35b:0x0].0x0 rrc: 4 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x60000400010020 nid: 192.168.204.3@tcp remote: 0x771f68aa1eb2565f expref: 23 pid: 8645 timeout: 899 lvb_type: 1 00010000:00010000:1.0:1776178092.598333:0:20767:0:(ldlm_lockd.c:562:ldlm_del_waiting_lock()) ### removed ns: filter-lustre-OST0001_UUID lock: 0000000072dc9eba/0xa40083ab9bf3b3d1 lrc: 4/0,0 mode: PR/PR res: [0x2c0000400:0x35b:0x0].0x0 rrc: 4 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x50000400010020 nid: 192.168.204.3@tcp remote: 0x771f68aa1eb2565f expref: 23 pid: 8645 timeout: 899 lvb_type: 1 00010000:00010000:1.0:1776178092.598350:0:20767:0:(ldlm_extent.c:318:ldlm_check_contention()) contended locks = 0 00010000:00010000:1.0:1776178092.598352:0:20767:0:(ldlm_extent.c:318:ldlm_check_contention()) contended locks = 0 00010000:00010000:1.0:1776178092.598354:0:20767:0:(ldlm_lock.c:704:ldlm_add_cp_work_item()) ### lock granted; sending completion AST. ns: filter-lustre-OST0001_UUID lock: 00000000b2e83690/0xa40083ab9bf52d8b lrc: 3/0,1 mode: PW/PW res: [0x2c0000400:0x35b:0x0].0x0 rrc: 4 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x50090080000000 nid: local remote: 0x0 expref: -99 pid: 20759 timeout: 0 lvb_type: 0 00010000:00010000:1.0:1776178092.598361:0:20767:0:(ldlm_resource.c:1650:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0001_UUID lock: 00000000b2e83690/0xa40083ab9bf52d8b lrc: 4/0,1 mode: PW/PW res: [0x2c0000400:0x35b:0x0].0x0 rrc: 4 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x50090080000000 nid: local remote: 0x0 expref: -99 pid: 20759 timeout: 0 lvb_type: 0 00010000:00010000:1.0:1776178092.598377:0:20767:0:(ldlm_lockd.c:1782:ldlm_request_cancel()) ### server-side cancel handler END 00000100:00100000:1.0:1776178092.598401:0:20767:0:(service.c:2372:ptlrpc_server_handle_request()) Handled RPC req@000000004eb4fa98 pname:cluuid+ref:pid:xid:nid:opc:job ldlm_cn00_004:b50963bd-8eba-4432-9a54-8888d00e3fa1+23:2271:x1862456979073728:12345-192.168.204.3@tcp:103:ldlm_bl_03.0 Request processed in 98us (2505us total) trans 0 rc 0/0 00000100:00100000:1.0:1776178092.598408:0:20767:0:(nrs_fifo.c:241:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.204.3@tcp, seq: 2047 00000100:00100000:1.0:1776178092.598424:0:20767:0:(nrs_fifo.c:179:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.204.3@tcp, seq: 2048 00000100:00100000:1.0:1776178092.598428:0:20767:0:(service.c:2324:ptlrpc_server_handle_request()) Handling RPC req@00000000dff3309c pname:cluuid+ref:pid:xid:nid:opc:job ldlm_cn00_004:b50963bd-8eba-4432-9a54-8888d00e3fa1+25:2271:x1862456979073856:12345-192.168.204.3@tcp:103:ldlm_bl_03.0 00010000:00010000:1.0:1776178092.598435:0:20767:0:(ldlm_lockd.c:1720:ldlm_request_cancel()) ### server-side cancel handler START: 1 locks, starting at 0 00010000:00010000:1.0:1776178092.598440:0:20767:0:(ldlm_lockd.c:1771:ldlm_request_cancel()) ### server cancels blocked lock after 0s ns: filter-lustre-OST0000_UUID lock: 00000000e59f0070/0xa40083ab9bf35512 lrc: 4/0,0 mode: PR/PR res: [0x280000400:0x2da:0x0].0x0 rrc: 4 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x60000400010020 nid: 192.168.204.3@tcp remote: 0x771f68aa1eb236ca expref: 25 pid: 8228 timeout: 899 lvb_type: 1 00010000:00010000:1.0:1776178092.598448:0:20767:0:(ldlm_lockd.c:562:ldlm_del_waiting_lock()) ### removed ns: filter-lustre-OST0000_UUID lock: 00000000e59f0070/0xa40083ab9bf35512 lrc: 3/0,0 mode: PR/PR res: [0x280000400:0x2da:0x0].0x0 rrc: 4 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x50000400010020 nid: 192.168.204.3@tcp remote: 0x771f68aa1eb236ca expref: 25 pid: 8228 timeout: 899 lvb_type: 1 00010000:00010000:1.0:1776178092.598461:0:20767:0:(ldlm_lock.c:216:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 00000000e59f0070/0xa40083ab9bf35512 lrc: 0/0,0 mode: --/PR res: [0x280000400:0x2da:0x0].0x0 rrc: 4 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x44801400010020 nid: 192.168.204.3@tcp remote: 0x771f68aa1eb236ca expref: 25 pid: 8228 timeout: 899 lvb_type: 1 00010000:00010000:1.0:1776178092.598469:0:20767:0:(ldlm_extent.c:318:ldlm_check_contention()) contended locks = 0 00010000:00010000:1.0:1776178092.598470:0:20767:0:(ldlm_extent.c:318:ldlm_check_contention()) contended locks = 0 00010000:00010000:1.0:1776178092.598472:0:20767:0:(ldlm_lock.c:704:ldlm_add_cp_work_item()) ### lock granted; sending completion AST. ns: filter-lustre-OST0000_UUID lock: 000000002f1f7bd5/0xa40083ab9bf52dbc lrc: 3/0,1 mode: PW/PW res: [0x280000400:0x2da:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x40090080000000 nid: local remote: 0x0 expref: -99 pid: 20761 timeout: 0 lvb_type: 0 00010000:00010000:1.0:1776178092.598477:0:20767:0:(ldlm_resource.c:1650:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 000000002f1f7bd5/0xa40083ab9bf52dbc lrc: 4/0,1 mode: PW/PW res: [0x280000400:0x2da:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x40090080000000 nid: local remote: 0x0 expref: -99 pid: 20761 timeout: 0 lvb_type: 0 00010000:00010000:1.0:1776178092.598495:0:20767:0:(ldlm_lockd.c:1782:ldlm_request_cancel()) ### server-side cancel handler END 00000100:00100000:1.0:1776178092.598512:0:20767:0:(service.c:2372:ptlrpc_server_handle_request()) Handled RPC req@00000000dff3309c pname:cluuid+ref:pid:xid:nid:opc:job ldlm_cn00_004:b50963bd-8eba-4432-9a54-8888d00e3fa1+24:2271:x1862456979073856:12345-192.168.204.3@tcp:103:ldlm_bl_03.0 Request processed in 84us (2597us total) trans 0 rc 0/0 00000100:00100000:1.0:1776178092.598519:0:20767:0:(nrs_fifo.c:241:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.204.3@tcp, seq: 2048 00000100:00100000:1.0:1776178092.598530:0:20767:0:(nrs_fifo.c:179:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.204.3@tcp, seq: 2049 00000100:00100000:1.0:1776178092.598534:0:20767:0:(service.c:2324:ptlrpc_server_handle_request()) Handling RPC req@0000000056dbc32b pname:cluuid+ref:pid:xid:nid:opc:job ldlm_cn00_004:b50963bd-8eba-4432-9a54-8888d00e3fa1+23:2271:x1862456979073984:12345-192.168.204.3@tcp:103:ldlm_bl_03.0 00010000:00010000:1.0:1776178092.598541:0:20767:0:(ldlm_lockd.c:1720:ldlm_request_cancel()) ### server-side cancel handler START: 1 locks, starting at 0 00010000:00010000:1.0:1776178092.598545:0:20767:0:(ldlm_lockd.c:1771:ldlm_request_cancel()) ### server cancels blocked lock after 0s ns: filter-lustre-OST0000_UUID lock: 00000000485847f3/0xa40083ab9bf32b3c lrc: 5/0,0 mode: PR/PR res: [0x280000400:0x285:0x0].0x0 rrc: 4 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x60000400010020 nid: 192.168.204.3@tcp remote: 0x771f68aa1eb228d8 expref: 23 pid: 8645 timeout: 899 lvb_type: 1 00010000:00010000:1.0:1776178092.598558:0:20767:0:(ldlm_lockd.c:562:ldlm_del_waiting_lock()) ### removed ns: filter-lustre-OST0000_UUID lock: 00000000485847f3/0xa40083ab9bf32b3c lrc: 4/0,0 mode: PR/PR res: [0x280000400:0x285:0x0].0x0 rrc: 4 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x50000400010020 nid: 192.168.204.3@tcp remote: 0x771f68aa1eb228d8 expref: 23 pid: 8645 timeout: 899 lvb_type: 1 00010000:00010000:1.0:1776178092.598570:0:20767:0:(ldlm_extent.c:318:ldlm_check_contention()) contended locks = 0 00010000:00010000:1.0:1776178092.598571:0:20767:0:(ldlm_extent.c:318:ldlm_check_contention()) contended locks = 0 00010000:00010000:1.0:1776178092.598573:0:20767:0:(ldlm_lock.c:704:ldlm_add_cp_work_item()) ### lock granted; sending completion AST. ns: filter-lustre-OST0000_UUID lock: 00000000319537bc/0xa40083ab9bf52dc3 lrc: 3/0,1 mode: PW/PW res: [0x280000400:0x285:0x0].0x0 rrc: 4 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x50090080000000 nid: local remote: 0x0 expref: -99 pid: 8228 timeout: 0 lvb_type: 0 00010000:00010000:1.0:1776178092.598579:0:20767:0:(ldlm_resource.c:1650:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 00000000319537bc/0xa40083ab9bf52dc3 lrc: 4/0,1 mode: PW/PW res: [0x280000400:0x285:0x0].0x0 rrc: 4 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x50090080000000 nid: local remote: 0x0 expref: -99 pid: 8228 timeout: 0 lvb_type: 0 00010000:00010000:1.0:1776178092.598591:0:20767:0:(ldlm_lockd.c:1782:ldlm_request_cancel()) ### server-side cancel handler END 00000100:00100000:1.0:1776178092.598608:0:20767:0:(service.c:2372:ptlrpc_server_handle_request()) Handled RPC req@0000000056dbc32b pname:cluuid+ref:pid:xid:nid:opc:job ldlm_cn00_004:b50963bd-8eba-4432-9a54-8888d00e3fa1+23:2271:x1862456979073984:12345-192.168.204.3@tcp:103:ldlm_bl_03.0 Request processed in 73us (2673us total) trans 0 rc 0/0 00000100:00100000:1.0:1776178092.598614:0:20767:0:(nrs_fifo.c:241:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.204.3@tcp, seq: 2049 00000100:00100000:1.0:1776178092.598627:0:20767:0:(nrs_fifo.c:179:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.204.3@tcp, seq: 2050 00000100:00100000:1.0:1776178092.598631:0:20767:0:(service.c:2324:ptlrpc_server_handle_request()) Handling RPC req@00000000f4d20636 pname:cluuid+ref:pid:xid:nid:opc:job ldlm_cn00_004:b50963bd-8eba-4432-9a54-8888d00e3fa1+22:2271:x1862456979074112:12345-192.168.204.3@tcp:103:ldlm_bl_02.0 00010000:00010000:1.0:1776178092.598639:0:20767:0:(ldlm_lockd.c:1720:ldlm_request_cancel()) ### server-side cancel handler START: 1 locks, starting at 0 00010000:00010000:1.0:1776178092.598644:0:20767:0:(ldlm_lockd.c:1771:ldlm_request_cancel()) ### server cancels blocked lock after 0s ns: filter-lustre-OST0000_UUID lock: 00000000a6b36e6d/0xa40083ab9bf3bcec lrc: 5/0,0 mode: PR/PR res: [0x280000400:0x3ad:0x0].0x0 rrc: 4 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x60000400010020 nid: 192.168.204.3@tcp remote: 0x771f68aa1eb25968 expref: 22 pid: 8645 timeout: 899 lvb_type: 1 00010000:00010000:1.0:1776178092.598651:0:20767:0:(ldlm_lockd.c:562:ldlm_del_waiting_lock()) ### removed ns: filter-lustre-OST0000_UUID lock: 00000000a6b36e6d/0xa40083ab9bf3bcec lrc: 4/0,0 mode: PR/PR res: [0x280000400:0x3ad:0x0].0x0 rrc: 4 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x50000400010020 nid: 192.168.204.3@tcp remote: 0x771f68aa1eb25968 expref: 22 pid: 8645 timeout: 899 lvb_type: 1 00010000:00010000:1.0:1776178092.598663:0:20767:0:(ldlm_extent.c:318:ldlm_check_contention()) contended locks = 0 00010000:00010000:1.0:1776178092.598665:0:20767:0:(ldlm_extent.c:318:ldlm_check_contention()) contended locks = 0 00010000:00010000:1.0:1776178092.598667:0:20767:0:(ldlm_lock.c:704:ldlm_add_cp_work_item()) ### lock granted; sending completion AST. ns: filter-lustre-OST0000_UUID lock: 000000000b903578/0xa40083ab9bf52dd1 lrc: 3/0,1 mode: PW/PW res: [0x280000400:0x3ad:0x0].0x0 rrc: 4 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x50090080000000 nid: local remote: 0x0 expref: -99 pid: 20754 timeout: 0 lvb_type: 0 00010000:00010000:1.0:1776178092.598673:0:20767:0:(ldlm_resource.c:1650:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 000000000b903578/0xa40083ab9bf52dd1 lrc: 4/0,1 mode: PW/PW res: [0x280000400:0x3ad:0x0].0x0 rrc: 4 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x50090080000000 nid: local remote: 0x0 expref: -99 pid: 20754 timeout: 0 lvb_type: 0 00010000:00010000:1.0:1776178092.598683:0:20767:0:(ldlm_lockd.c:1782:ldlm_request_cancel()) ### server-side cancel handler END 00000100:00100000:1.0:1776178092.598699:0:20767:0:(service.c:2372:ptlrpc_server_handle_request()) Handled RPC req@00000000f4d20636 pname:cluuid+ref:pid:xid:nid:opc:job ldlm_cn00_004:b50963bd-8eba-4432-9a54-8888d00e3fa1+22:2271:x1862456979074112:12345-192.168.204.3@tcp:103:ldlm_bl_02.0 Request processed in 67us (2752us total) trans 0 rc 0/0 00000100:00100000:1.0:1776178092.598705:0:20767:0:(nrs_fifo.c:241:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.204.3@tcp, seq: 2050 00000100:00100000:1.0:1776178092.598719:0:20767:0:(nrs_fifo.c:179:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.204.3@tcp, seq: 2051 00000100:00100000:1.0:1776178092.598723:0:20767:0:(service.c:2324:ptlrpc_server_handle_request()) Handling RPC req@000000007fcab602 pname:cluuid+ref:pid:xid:nid:opc:job ldlm_cn00_004:b50963bd-8eba-4432-9a54-8888d00e3fa1+21:2271:x1862456979073280:12345-192.168.204.3@tcp:103:ldlm_bl_03.0 00010000:00010000:1.0:1776178092.598731:0:20767:0:(ldlm_lockd.c:1720:ldlm_request_cancel()) ### server-side cancel handler START: 1 locks, starting at 0 00010000:00010000:1.0:1776178092.598736:0:20767:0:(ldlm_lockd.c:1771:ldlm_request_cancel()) ### server cancels blocked lock after 0s ns: filter-lustre-OST0000_UUID lock: 0000000012177dd1/0xa40083ab9bf33bf8 lrc: 5/0,0 mode: PR/PR res: [0x280000400:0x2a7:0x0].0x0 rrc: 4 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x60000400010020 nid: 192.168.204.3@tcp remote: 0x771f68aa1eb22e6c expref: 21 pid: 8228 timeout: 899 lvb_type: 1 00010000:00010000:1.0:1776178092.598743:0:20767:0:(ldlm_lockd.c:562:ldlm_del_waiting_lock()) ### removed ns: filter-lustre-OST0000_UUID lock: 0000000012177dd1/0xa40083ab9bf33bf8 lrc: 4/0,0 mode: PR/PR res: [0x280000400:0x2a7:0x0].0x0 rrc: 4 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x50000400010020 nid: 192.168.204.3@tcp remote: 0x771f68aa1eb22e6c expref: 21 pid: 8228 timeout: 899 lvb_type: 1 00010000:00010000:1.0:1776178092.598756:0:20767:0:(ldlm_extent.c:318:ldlm_check_contention()) contended locks = 0 00010000:00010000:1.0:1776178092.598757:0:20767:0:(ldlm_extent.c:318:ldlm_check_contention()) contended locks = 0 00010000:00010000:1.0:1776178092.598759:0:20767:0:(ldlm_lock.c:704:ldlm_add_cp_work_item()) ### lock granted; sending completion AST. ns: filter-lustre-OST0000_UUID lock: 0000000096c1e297/0xa40083ab9bf52da7 lrc: 3/0,1 mode: PW/PW res: [0x280000400:0x2a7:0x0].0x0 rrc: 4 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x50090080000000 nid: local remote: 0x0 expref: -99 pid: 20765 timeout: 0 lvb_type: 0 00010000:00010000:1.0:1776178092.598765:0:20767:0:(ldlm_resource.c:1650:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 0000000096c1e297/0xa40083ab9bf52da7 lrc: 4/0,1 mode: PW/PW res: [0x280000400:0x2a7:0x0].0x0 rrc: 4 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x50090080000000 nid: local remote: 0x0 expref: -99 pid: 20765 timeout: 0 lvb_type: 0 00010000:00010000:1.0:1776178092.598782:0:20767:0:(ldlm_lockd.c:1782:ldlm_request_cancel()) ### server-side cancel handler END 00000100:00100000:1.0:1776178092.598798:0:20767:0:(service.c:2372:ptlrpc_server_handle_request()) Handled RPC req@000000007fcab602 pname:cluuid+ref:pid:xid:nid:opc:job ldlm_cn00_004:b50963bd-8eba-4432-9a54-8888d00e3fa1+21:2271:x1862456979073280:12345-192.168.204.3@tcp:103:ldlm_bl_03.0 Request processed in 75us (2840us total) trans 0 rc 0/0 00000100:00100000:1.0:1776178092.598804:0:20767:0:(nrs_fifo.c:241:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.204.3@tcp, seq: 2051 00000100:00100000:1.0:1776178092.598816:0:20767:0:(nrs_fifo.c:179:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.204.3@tcp, seq: 2052 00000100:00100000:1.0:1776178092.598820:0:20767:0:(service.c:2324:ptlrpc_server_handle_request()) Handling RPC req@00000000956deaca pname:cluuid+ref:pid:xid:nid:opc:job ldlm_cn00_004:b50963bd-8eba-4432-9a54-8888d00e3fa1+22:2271:x1862456979073408:12345-192.168.204.3@tcp:103:ldlm_bl_03.0 00010000:00010000:1.0:1776178092.598840:0:20767:0:(ldlm_lockd.c:1720:ldlm_request_cancel()) ### server-side cancel handler START: 1 locks, starting at 0 00010000:00010000:1.0:1776178092.598845:0:20767:0:(ldlm_lockd.c:1771:ldlm_request_cancel()) ### server cancels blocked lock after 0s ns: filter-lustre-OST0001_UUID lock: 000000003ed46f13/0xa40083ab9bf36901 lrc: 5/0,0 mode: PR/PR res: [0x2c0000400:0x2c3:0x0].0x0 rrc: 4 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x60000400010020 nid: 192.168.204.3@tcp remote: 0x771f68aa1eb23d6f expref: 22 pid: 8645 timeout: 899 lvb_type: 1 00010000:00010000:1.0:1776178092.598854:0:20767:0:(ldlm_lockd.c:562:ldlm_del_waiting_lock()) ### removed ns: filter-lustre-OST0001_UUID lock: 000000003ed46f13/0xa40083ab9bf36901 lrc: 4/0,0 mode: PR/PR res: [0x2c0000400:0x2c3:0x0].0x0 rrc: 4 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x50000400010020 nid: 192.168.204.3@tcp remote: 0x771f68aa1eb23d6f expref: 22 pid: 8645 timeout: 899 lvb_type: 1 00010000:00010000:1.0:1776178092.598868:0:20767:0:(ldlm_extent.c:318:ldlm_check_contention()) contended locks = 0 00010000:00010000:1.0:1776178092.598870:0:20767:0:(ldlm_extent.c:318:ldlm_check_contention()) contended locks = 0 00010000:00010000:1.0:1776178092.598873:0:20767:0:(ldlm_lock.c:704:ldlm_add_cp_work_item()) ### lock granted; sending completion AST. ns: filter-lustre-OST0001_UUID lock: 000000007da3b625/0xa40083ab9bf52d6f lrc: 3/0,1 mode: PW/PW res: [0x2c0000400:0x2c3:0x0].0x0 rrc: 4 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x50090080000000 nid: local remote: 0x0 expref: -99 pid: 20763 timeout: 0 lvb_type: 0 00010000:00010000:1.0:1776178092.598878:0:20767:0:(ldlm_resource.c:1650:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0001_UUID lock: 000000007da3b625/0xa40083ab9bf52d6f lrc: 4/0,1 mode: PW/PW res: [0x2c0000400:0x2c3:0x0].0x0 rrc: 4 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x50090080000000 nid: local remote: 0x0 expref: -99 pid: 20763 timeout: 0 lvb_type: 0 00010000:00010000:1.0:1776178092.598891:0:20767:0:(ldlm_lockd.c:1782:ldlm_request_cancel()) ### server-side cancel handler END 00000100:00100000:1.0:1776178092.598904:0:20767:0:(service.c:2372:ptlrpc_server_handle_request()) Handled RPC req@00000000956deaca pname:cluuid+ref:pid:xid:nid:opc:job ldlm_cn00_004:b50963bd-8eba-4432-9a54-8888d00e3fa1+22:2271:x1862456979073408:12345-192.168.204.3@tcp:103:ldlm_bl_03.0 Request processed in 83us (2934us total) trans 0 rc 0/0 00000100:00100000:1.0:1776178092.598911:0:20767:0:(nrs_fifo.c:241:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.204.3@tcp, seq: 2052 00000100:00100000:1.0:1776178092.598923:0:20767:0:(nrs_fifo.c:179:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.204.3@tcp, seq: 2053 00000100:00100000:1.0:1776178092.598928:0:20767:0:(service.c:2324:ptlrpc_server_handle_request()) Handling RPC req@0000000097ae2a31 pname:cluuid+ref:pid:xid:nid:opc:job ldlm_cn00_004:b50963bd-8eba-4432-9a54-8888d00e3fa1+21:2271:x1862456979073536:12345-192.168.204.3@tcp:103:ldlm_bl_03.0 00010000:00010000:1.0:1776178092.598935:0:20767:0:(ldlm_lockd.c:1720:ldlm_request_cancel()) ### server-side cancel handler START: 1 locks, starting at 0 00010000:00010000:1.0:1776178092.598940:0:20767:0:(ldlm_lockd.c:1771:ldlm_request_cancel()) ### server cancels blocked lock after 0s ns: filter-lustre-OST0001_UUID lock: 0000000076588d9d/0xa40083ab9bf36787 lrc: 4/0,0 mode: PR/PR res: [0x2c0000400:0x2c0:0x0].0x0 rrc: 4 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x60000400010020 nid: 192.168.204.3@tcp remote: 0x771f68aa1eb23cf1 expref: 21 pid: 8227 timeout: 899 lvb_type: 1 00010000:00010000:1.0:1776178092.598949:0:20767:0:(ldlm_lockd.c:562:ldlm_del_waiting_lock()) ### removed ns: filter-lustre-OST0001_UUID lock: 0000000076588d9d/0xa40083ab9bf36787 lrc: 3/0,0 mode: PR/PR res: [0x2c0000400:0x2c0:0x0].0x0 rrc: 4 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x50000400010020 nid: 192.168.204.3@tcp remote: 0x771f68aa1eb23cf1 expref: 21 pid: 8227 timeout: 899 lvb_type: 1 00010000:00010000:1.0:1776178092.598962:0:20767:0:(ldlm_lock.c:216:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0001_UUID lock: 0000000076588d9d/0xa40083ab9bf36787 lrc: 0/0,0 mode: --/PR res: [0x2c0000400:0x2c0:0x0].0x0 rrc: 4 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x44801400010020 nid: 192.168.204.3@tcp remote: 0x771f68aa1eb23cf1 expref: 21 pid: 8227 timeout: 899 lvb_type: 1 00010000:00010000:1.0:1776178092.598972:0:20767:0:(ldlm_extent.c:318:ldlm_check_contention()) contended locks = 0 00010000:00010000:1.0:1776178092.598973:0:20767:0:(ldlm_extent.c:318:ldlm_check_contention()) contended locks = 0 00010000:00010000:1.0:1776178092.598975:0:20767:0:(ldlm_lock.c:704:ldlm_add_cp_work_item()) ### lock granted; sending completion AST. ns: filter-lustre-OST0001_UUID lock: 000000000b5c0143/0xa40083ab9bf52db5 lrc: 3/0,1 mode: PW/PW res: [0x2c0000400:0x2c0:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x40090080000000 nid: local remote: 0x0 expref: -99 pid: 8645 timeout: 0 lvb_type: 0 00010000:00010000:1.0:1776178092.598981:0:20767:0:(ldlm_resource.c:1650:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0001_UUID lock: 000000000b5c0143/0xa40083ab9bf52db5 lrc: 4/0,1 mode: PW/PW res: [0x2c0000400:0x2c0:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x40090080000000 nid: local remote: 0x0 expref: -99 pid: 8645 timeout: 0 lvb_type: 0 00010000:00010000:1.0:1776178092.599000:0:20767:0:(ldlm_lockd.c:1782:ldlm_request_cancel()) ### server-side cancel handler END 00000100:00100000:1.0:1776178092.599013:0:20767:0:(service.c:2372:ptlrpc_server_handle_request()) Handled RPC req@0000000097ae2a31 pname:cluuid+ref:pid:xid:nid:opc:job ldlm_cn00_004:b50963bd-8eba-4432-9a54-8888d00e3fa1+20:2271:x1862456979073536:12345-192.168.204.3@tcp:103:ldlm_bl_03.0 Request processed in 84us (3031us total) trans 0 rc 0/0 00000100:00100000:1.0:1776178092.599019:0:20767:0:(nrs_fifo.c:241:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.204.3@tcp, seq: 2053 00000100:00100000:1.0:1776178092.599032:0:20767:0:(nrs_fifo.c:179:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.204.3@tcp, seq: 2054 00000100:00100000:1.0:1776178092.599037:0:20767:0:(service.c:2324:ptlrpc_server_handle_request()) Handling RPC req@00000000cf0fe5ca pname:cluuid+ref:pid:xid:nid:opc:job ldlm_cn00_004:b50963bd-8eba-4432-9a54-8888d00e3fa1+19:2271:x1862456979073664:12345-192.168.204.3@tcp:103:ldlm_bl_03.0 00010000:00010000:1.0:1776178092.599045:0:20767:0:(ldlm_lockd.c:1720:ldlm_request_cancel()) ### server-side cancel handler START: 1 locks, starting at 0 00010000:00010000:1.0:1776178092.599050:0:20767:0:(ldlm_lockd.c:1771:ldlm_request_cancel()) ### server cancels blocked lock after 0s ns: filter-lustre-OST0001_UUID lock: 00000000adf1bb1a/0xa40083ab9bf38deb lrc: 5/0,0 mode: PR/PR res: [0x2c0000400:0x30e:0x0].0x0 rrc: 4 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x60000400010020 nid: 192.168.204.3@tcp remote: 0x771f68aa1eb249bd expref: 19 pid: 8227 timeout: 899 lvb_type: 1 00010000:00010000:1.0:1776178092.599090:0:20767:0:(ldlm_lockd.c:562:ldlm_del_waiting_lock()) ### removed ns: filter-lustre-OST0001_UUID lock: 00000000adf1bb1a/0xa40083ab9bf38deb lrc: 4/0,0 mode: PR/PR res: [0x2c0000400:0x30e:0x0].0x0 rrc: 4 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x50000400010020 nid: 192.168.204.3@tcp remote: 0x771f68aa1eb249bd expref: 19 pid: 8227 timeout: 899 lvb_type: 1 00010000:00010000:1.0:1776178092.599119:0:20767:0:(ldlm_extent.c:318:ldlm_check_contention()) contended locks = 0 00010000:00010000:1.0:1776178092.599121:0:20767:0:(ldlm_extent.c:318:ldlm_check_contention()) contended locks = 0 00010000:00010000:1.0:1776178092.599123:0:20767:0:(ldlm_lock.c:704:ldlm_add_cp_work_item()) ### lock granted; sending completion AST. ns: filter-lustre-OST0001_UUID lock: 00000000fcb32840/0xa40083ab9bf52d84 lrc: 3/0,1 mode: PW/PW res: [0x2c0000400:0x30e:0x0].0x0 rrc: 4 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x50090080000000 nid: local remote: 0x0 expref: -99 pid: 20756 timeout: 0 lvb_type: 0 00010000:00010000:1.0:1776178092.599130:0:20767:0:(ldlm_resource.c:1650:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0001_UUID lock: 00000000fcb32840/0xa40083ab9bf52d84 lrc: 4/0,1 mode: PW/PW res: [0x2c0000400:0x30e:0x0].0x0 rrc: 4 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x50090080000000 nid: local remote: 0x0 expref: -99 pid: 20756 timeout: 0 lvb_type: 0 00010000:00010000:1.0:1776178092.599141:0:20767:0:(ldlm_lockd.c:1782:ldlm_request_cancel()) ### server-side cancel handler END 00000100:00100000:1.0:1776178092.599157:0:20767:0:(service.c:2372:ptlrpc_server_handle_request()) Handled RPC req@00000000cf0fe5ca pname:cluuid+ref:pid:xid:nid:opc:job ldlm_cn00_004:b50963bd-8eba-4432-9a54-8888d00e3fa1+19:2271:x1862456979073664:12345-192.168.204.3@tcp:103:ldlm_bl_03.0 Request processed in 120us (3161us total) trans 0 rc 0/0 00000100:00100000:1.0:1776178092.599163:0:20767:0:(nrs_fifo.c:241:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.204.3@tcp, seq: 2054 00000100:00100000:1.0:1776178092.599199:0:20767:0:(nrs_fifo.c:179:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.204.3@tcp, seq: 2055 00000100:00100000:1.0:1776178092.599203:0:20767:0:(service.c:2324:ptlrpc_server_handle_request()) Handling RPC req@000000003493d657 pname:cluuid+ref:pid:xid:nid:opc:job ldlm_cn00_004:b50963bd-8eba-4432-9a54-8888d00e3fa1+20:2271:x1862456979073792:12345-192.168.204.3@tcp:103:ldlm_bl_03.0 00010000:00010000:1.0:1776178092.599210:0:20767:0:(ldlm_lockd.c:1720:ldlm_request_cancel()) ### server-side cancel handler START: 1 locks, starting at 0 00010000:00010000:1.0:1776178092.599215:0:20767:0:(ldlm_lockd.c:1771:ldlm_request_cancel()) ### server cancels blocked lock after 0s ns: filter-lustre-OST0000_UUID lock: 0000000006d4cafa/0xa40083ab9bf3768a lrc: 4/0,0 mode: PR/PR res: [0x280000400:0x31e:0x0].0x0 rrc: 4 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x60000400010020 nid: 192.168.204.3@tcp remote: 0x771f68aa1eb241f2 expref: 20 pid: 14440 timeout: 899 lvb_type: 1 00010000:00010000:1.0:1776178092.599223:0:20767:0:(ldlm_lockd.c:562:ldlm_del_waiting_lock()) ### removed ns: filter-lustre-OST0000_UUID lock: 0000000006d4cafa/0xa40083ab9bf3768a lrc: 3/0,0 mode: PR/PR res: [0x280000400:0x31e:0x0].0x0 rrc: 4 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x50000400010020 nid: 192.168.204.3@tcp remote: 0x771f68aa1eb241f2 expref: 20 pid: 14440 timeout: 899 lvb_type: 1 00010000:00010000:1.0:1776178092.599236:0:20767:0:(ldlm_lock.c:216:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 0000000006d4cafa/0xa40083ab9bf3768a lrc: 0/0,0 mode: --/PR res: [0x280000400:0x31e:0x0].0x0 rrc: 4 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x44801400010020 nid: 192.168.204.3@tcp remote: 0x771f68aa1eb241f2 expref: 20 pid: 14440 timeout: 899 lvb_type: 1 00010000:00010000:1.0:1776178092.599246:0:20767:0:(ldlm_extent.c:318:ldlm_check_contention()) contended locks = 0 00010000:00010000:1.0:1776178092.599248:0:20767:0:(ldlm_extent.c:318:ldlm_check_contention()) contended locks = 0 00010000:00010000:1.0:1776178092.599250:0:20767:0:(ldlm_lock.c:704:ldlm_add_cp_work_item()) ### lock granted; sending completion AST. ns: filter-lustre-OST0000_UUID lock: 00000000540a8f8b/0xa40083ab9bf52d92 lrc: 3/0,1 mode: PW/PW res: [0x280000400:0x31e:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x40090080000000 nid: local remote: 0x0 expref: -99 pid: 20758 timeout: 0 lvb_type: 0 00010000:00010000:1.0:1776178092.599256:0:20767:0:(ldlm_resource.c:1650:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 00000000540a8f8b/0xa40083ab9bf52d92 lrc: 4/0,1 mode: PW/PW res: [0x280000400:0x31e:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x40090080000000 nid: local remote: 0x0 expref: -99 pid: 20758 timeout: 0 lvb_type: 0 00010000:00010000:1.0:1776178092.599274:0:20767:0:(ldlm_lockd.c:1782:ldlm_request_cancel()) ### server-side cancel handler END 00000100:00100000:1.0:1776178092.599288:0:20767:0:(service.c:2372:ptlrpc_server_handle_request()) Handled RPC req@000000003493d657 pname:cluuid+ref:pid:xid:nid:opc:job ldlm_cn00_004:b50963bd-8eba-4432-9a54-8888d00e3fa1+19:2271:x1862456979073792:12345-192.168.204.3@tcp:103:ldlm_bl_03.0 Request processed in 84us (3279us total) trans 0 rc 0/0 00000100:00100000:1.0:1776178092.599294:0:20767:0:(nrs_fifo.c:241:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.204.3@tcp, seq: 2055 00000100:00100000:1.0:1776178092.599306:0:20767:0:(nrs_fifo.c:179:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.204.3@tcp, seq: 2056 00000100:00100000:1.0:1776178092.599311:0:20767:0:(service.c:2324:ptlrpc_server_handle_request()) Handling RPC req@00000000fd0161d4 pname:cluuid+ref:pid:xid:nid:opc:job ldlm_cn00_004:b50963bd-8eba-4432-9a54-8888d00e3fa1+18:2271:x1862456979073920:12345-192.168.204.3@tcp:103:ldlm_bl_03.0 00010000:00010000:1.0:1776178092.599318:0:20767:0:(ldlm_lockd.c:1720:ldlm_request_cancel()) ### server-side cancel handler START: 1 locks, starting at 0 00010000:00010000:1.0:1776178092.599322:0:20767:0:(ldlm_lockd.c:1771:ldlm_request_cancel()) ### server cancels blocked lock after 0s ns: filter-lustre-OST0001_UUID lock: 00000000344b695c/0xa40083ab9bf33db1 lrc: 5/0,0 mode: PR/PR res: [0x2c0000400:0x26b:0x0].0x0 rrc: 4 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x60000400010020 nid: 192.168.204.3@tcp remote: 0x771f68aa1eb22eff expref: 18 pid: 8228 timeout: 899 lvb_type: 1 00010000:00010000:1.0:1776178092.599330:0:20767:0:(ldlm_lockd.c:562:ldlm_del_waiting_lock()) ### removed ns: filter-lustre-OST0001_UUID lock: 00000000344b695c/0xa40083ab9bf33db1 lrc: 4/0,0 mode: PR/PR res: [0x2c0000400:0x26b:0x0].0x0 rrc: 4 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x50000400010020 nid: 192.168.204.3@tcp remote: 0x771f68aa1eb22eff expref: 18 pid: 8228 timeout: 899 lvb_type: 1 00010000:00010000:1.0:1776178092.599341:0:20767:0:(ldlm_extent.c:318:ldlm_check_contention()) contended locks = 0 00010000:00010000:1.0:1776178092.599343:0:20767:0:(ldlm_extent.c:318:ldlm_check_contention()) contended locks = 0 00010000:00010000:1.0:1776178092.599344:0:20767:0:(ldlm_lock.c:704:ldlm_add_cp_work_item()) ### lock granted; sending completion AST. ns: filter-lustre-OST0001_UUID lock: 00000000daa24072/0xa40083ab9bf52d99 lrc: 3/0,1 mode: PW/PW res: [0x2c0000400:0x26b:0x0].0x0 rrc: 4 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x50090080000000 nid: local remote: 0x0 expref: -99 pid: 20766 timeout: 0 lvb_type: 0 00010000:00010000:1.0:1776178092.599354:0:20767:0:(ldlm_resource.c:1650:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0001_UUID lock: 00000000daa24072/0xa40083ab9bf52d99 lrc: 4/0,1 mode: PW/PW res: [0x2c0000400:0x26b:0x0].0x0 rrc: 4 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x50090080000000 nid: local remote: 0x0 expref: -99 pid: 20766 timeout: 0 lvb_type: 0 00010000:00010000:1.0:1776178092.599364:0:20767:0:(ldlm_lockd.c:1782:ldlm_request_cancel()) ### server-side cancel handler END 00000100:00100000:1.0:1776178092.599376:0:20767:0:(service.c:2372:ptlrpc_server_handle_request()) Handled RPC req@00000000fd0161d4 pname:cluuid+ref:pid:xid:nid:opc:job ldlm_cn00_004:b50963bd-8eba-4432-9a54-8888d00e3fa1+18:2271:x1862456979073920:12345-192.168.204.3@tcp:103:ldlm_bl_03.0 Request processed in 65us (3356us total) trans 0 rc 0/0 00000100:00100000:1.0:1776178092.599382:0:20767:0:(nrs_fifo.c:241:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.204.3@tcp, seq: 2056 00000100:00100000:1.0:1776178092.599393:0:20767:0:(nrs_fifo.c:179:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.204.3@tcp, seq: 2057 00000100:00100000:1.0:1776178092.599397:0:20767:0:(service.c:2324:ptlrpc_server_handle_request()) Handling RPC req@00000000c5909a76 pname:cluuid+ref:pid:xid:nid:opc:job ldlm_cn00_004:b50963bd-8eba-4432-9a54-8888d00e3fa1+17:2271:x1862456979074048:12345-192.168.204.3@tcp:103:ldlm_bl_02.0 00010000:00010000:1.0:1776178092.599403:0:20767:0:(ldlm_lockd.c:1720:ldlm_request_cancel()) ### server-side cancel handler START: 1 locks, starting at 0 00010000:00010000:1.0:1776178092.599407:0:20767:0:(ldlm_lockd.c:1771:ldlm_request_cancel()) ### server cancels blocked lock after 0s ns: filter-lustre-OST0001_UUID lock: 000000007adb6341/0xa40083ab9bf38af7 lrc: 5/0,0 mode: PR/PR res: [0x2c0000400:0x308:0x0].0x0 rrc: 4 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x60000400010020 nid: 192.168.204.3@tcp remote: 0x771f68aa1eb248c1 expref: 17 pid: 8645 timeout: 899 lvb_type: 1 00010000:00010000:1.0:1776178092.599415:0:20767:0:(ldlm_lockd.c:562:ldlm_del_waiting_lock()) ### removed ns: filter-lustre-OST0001_UUID lock: 000000007adb6341/0xa40083ab9bf38af7 lrc: 4/0,0 mode: PR/PR res: [0x2c0000400:0x308:0x0].0x0 rrc: 4 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x50000400010020 nid: 192.168.204.3@tcp remote: 0x771f68aa1eb248c1 expref: 17 pid: 8645 timeout: 899 lvb_type: 1 00010000:00010000:1.0:1776178092.599426:0:20767:0:(ldlm_extent.c:318:ldlm_check_contention()) contended locks = 0 00010000:00010000:1.0:1776178092.599427:0:20767:0:(ldlm_extent.c:318:ldlm_check_contention()) contended locks = 0 00010000:00010000:1.0:1776178092.599429:0:20767:0:(ldlm_lock.c:704:ldlm_add_cp_work_item()) ### lock granted; sending completion AST. ns: filter-lustre-OST0001_UUID lock: 000000006c464689/0xa40083ab9bf52dca lrc: 3/0,1 mode: PW/PW res: [0x2c0000400:0x308:0x0].0x0 rrc: 4 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x50090080000000 nid: local remote: 0x0 expref: -99 pid: 20757 timeout: 0 lvb_type: 0 00010000:00010000:1.0:1776178092.599435:0:20767:0:(ldlm_resource.c:1650:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0001_UUID lock: 000000006c464689/0xa40083ab9bf52dca lrc: 4/0,1 mode: PW/PW res: [0x2c0000400:0x308:0x0].0x0 rrc: 4 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x50090080000000 nid: local remote: 0x0 expref: -99 pid: 20757 timeout: 0 lvb_type: 0 00010000:00010000:1.0:1776178092.599445:0:20767:0:(ldlm_lockd.c:1782:ldlm_request_cancel()) ### server-side cancel handler END 00000100:00100000:1.0:1776178092.599457:0:20767:0:(service.c:2372:ptlrpc_server_handle_request()) Handled RPC req@00000000c5909a76 pname:cluuid+ref:pid:xid:nid:opc:job ldlm_cn00_004:b50963bd-8eba-4432-9a54-8888d00e3fa1+17:2271:x1862456979074048:12345-192.168.204.3@tcp:103:ldlm_bl_02.0 Request processed in 59us (3352us total) trans 0 rc 0/0 00000100:00100000:1.0:1776178092.599463:0:20767:0:(nrs_fifo.c:241:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.204.3@tcp, seq: 2057 00010000:00010000:1.0:1776178092.599513:0:20759:0:(ldlm_lock.c:216:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0001_UUID lock: 0000000072dc9eba/0xa40083ab9bf3b3d1 lrc: 0/0,0 mode: --/PR res: [0x2c0000400:0x35b:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x44801400010020 nid: 192.168.204.3@tcp remote: 0x771f68aa1eb2565f expref: 16 pid: 8645 timeout: 899 lvb_type: 1 00000100:00100000:1.0:1776178092.599523:0:20759:0:(client.c:2215:ptlrpc_check_set()) Completed RPC req@00000000a59bfc89 pname:cluuid:pid:xid:nid:opc:job ll_ost00_010:lustre-OST0001_UUID:20759:1862456998016192:192.168.204.3@tcp:104: 00010000:00010000:1.0:1776178092.599538:0:20759:0:(ldlm_request.c:283:ldlm_completion_ast()) ### client-side enqueue returned a blocked locksleeping ns: filter-lustre-OST0001_UUID lock: 00000000b2e83690/0xa40083ab9bf52d8b lrc: 3/0,1 mode: PW/PW res: [0x2c0000400:0x35b:0x0].0x0 rrc: 2 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x40010080000000 nid: local remote: 0x0 expref: -99 pid: 20759 timeout: 0 lvb_type: 0 00010000:00010000:1.0:1776178092.599545:0:20759:0:(ldlm_request.c:199:ldlm_completion_tail()) ### client-side enqueue: granted ns: filter-lustre-OST0001_UUID lock: 00000000b2e83690/0xa40083ab9bf52d8b lrc: 3/0,1 mode: PW/PW res: [0x2c0000400:0x35b:0x0].0x0 rrc: 2 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x40010080000000 nid: local remote: 0x0 expref: -99 pid: 20759 timeout: 0 lvb_type: 0 00010000:00010000:1.0:1776178092.599550:0:20759:0:(ldlm_request.c:528:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0001_UUID lock: 00000000b2e83690/0xa40083ab9bf52d8b lrc: 3/0,1 mode: PW/PW res: [0x2c0000400:0x35b:0x0].0x0 rrc: 2 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x40010080000000 nid: local remote: 0x0 expref: -99 pid: 20759 timeout: 0 lvb_type: 0 00010000:00010000:1.0:1776178092.599559:0:20759:0:(ldlm_lock.c:820:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0001_UUID lock: 00000000b2e83690/0xa40083ab9bf52d8b lrc: 3/0,1 mode: PW/PW res: [0x2c0000400:0x35b:0x0].0x0 rrc: 2 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x50010080000000 nid: local remote: 0x0 expref: -99 pid: 20759 timeout: 0 lvb_type: 0 00010000:00010000:1.0:1776178092.599565:0:20759:0:(ldlm_lock.c:879:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0001_UUID lock: 00000000b2e83690/0xa40083ab9bf52d8b lrc: 2/0,0 mode: PW/PW res: [0x2c0000400:0x35b:0x0].0x0 rrc: 2 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x50010480000000 nid: local remote: 0x0 expref: -99 pid: 20759 timeout: 0 lvb_type: 0 00010000:00010000:1.0:1776178092.599581:0:20759:0:(ldlm_lockd.c:2159:__ldlm_bl_to_thread()) added 1/1 locks to regular blp list, 1 blwis in pool 00000100:00100000:1.0:1776178092.599778:0:20759:0:(service.c:2372:ptlrpc_server_handle_request()) Handled RPC req@00000000ed90ff75 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_010:lustre-MDT0001-mdtlov_UUID+38:3355:x1862456998015488:12345-0@lo:6:osp-syn-1-1.0 Request processed in 41000us (41403us total) trans 4294969302 rc 0/0 00000100:00100000:1.0:1776178092.599786:0:20759:0:(nrs_fifo.c:241:nrs_fifo_req_stop()) NRS stop fifo request from 12345-0@lo, seq: 5203 00010000:00010000:1.0:1776178092.599818:0:20766:0:(ldlm_lock.c:216:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0001_UUID lock: 00000000344b695c/0xa40083ab9bf33db1 lrc: 0/0,0 mode: --/PR res: [0x2c0000400:0x26b:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x44801400010020 nid: 192.168.204.3@tcp remote: 0x771f68aa1eb22eff expref: 15 pid: 8228 timeout: 899 lvb_type: 1 00000100:00100000:1.0:1776178092.599852:0:20766:0:(client.c:2215:ptlrpc_check_set()) Completed RPC req@00000000a30eb964 pname:cluuid:pid:xid:nid:opc:job ll_ost00_017:lustre-OST0001_UUID:20766:1862456998016896:192.168.204.3@tcp:104: 00010000:00010000:1.0:1776178092.599862:0:20766:0:(ldlm_request.c:283:ldlm_completion_ast()) ### client-side enqueue returned a blocked locksleeping ns: filter-lustre-OST0001_UUID lock: 00000000daa24072/0xa40083ab9bf52d99 lrc: 3/0,1 mode: PW/PW res: [0x2c0000400:0x26b:0x0].0x0 rrc: 2 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x40010080000000 nid: local remote: 0x0 expref: -99 pid: 20766 timeout: 0 lvb_type: 0 00010000:00010000:1.0:1776178092.599868:0:20766:0:(ldlm_request.c:199:ldlm_completion_tail()) ### client-side enqueue: granted ns: filter-lustre-OST0001_UUID lock: 00000000daa24072/0xa40083ab9bf52d99 lrc: 3/0,1 mode: PW/PW res: [0x2c0000400:0x26b:0x0].0x0 rrc: 2 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x40010080000000 nid: local remote: 0x0 expref: -99 pid: 20766 timeout: 0 lvb_type: 0 00010000:00010000:1.0:1776178092.599874:0:20766:0:(ldlm_request.c:528:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0001_UUID lock: 00000000daa24072/0xa40083ab9bf52d99 lrc: 3/0,1 mode: PW/PW res: [0x2c0000400:0x26b:0x0].0x0 rrc: 2 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x40010080000000 nid: local remote: 0x0 expref: -99 pid: 20766 timeout: 0 lvb_type: 0 00010000:00010000:1.0:1776178092.599880:0:20766:0:(ldlm_lock.c:820:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0001_UUID lock: 00000000daa24072/0xa40083ab9bf52d99 lrc: 3/0,1 mode: PW/PW res: [0x2c0000400:0x26b:0x0].0x0 rrc: 2 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x50010080000000 nid: local remote: 0x0 expref: -99 pid: 20766 timeout: 0 lvb_type: 0 00010000:00010000:1.0:1776178092.599885:0:20766:0:(ldlm_lock.c:879:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0001_UUID lock: 00000000daa24072/0xa40083ab9bf52d99 lrc: 2/0,0 mode: PW/PW res: [0x2c0000400:0x26b:0x0].0x0 rrc: 2 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x50010480000000 nid: local remote: 0x0 expref: -99 pid: 20766 timeout: 0 lvb_type: 0 00010000:00010000:1.0:1776178092.599899:0:20766:0:(ldlm_lockd.c:2159:__ldlm_bl_to_thread()) added 1/2 locks to regular blp list, 2 blwis in pool 00000100:00100000:1.0:1776178092.600023:0:20766:0:(service.c:2372:ptlrpc_server_handle_request()) Handled RPC req@00000000fa6d10e9 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_017:lustre-MDT0001-mdtlov_UUID+38:3354:x1862456998016064:12345-0@lo:6:osp-syn-1-1.0 Request processed in 41094us (41366us total) trans 4294969303 rc 0/0 00000100:00100000:1.0:1776178092.600029:0:20766:0:(nrs_fifo.c:241:nrs_fifo_req_stop()) NRS stop fifo request from 12345-0@lo, seq: 5205 00010000:00010000:1.0:1776178092.600048:0:6245:0:(ldlm_lockd.c:2789:ldlm_bl_get_work()) Got 1 locks of 1 total in blp. (1 blwis in pool) 00010000:00010000:1.0:1776178092.600053:0:6245:0:(ldlm_lockd.c:1879:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0001_UUID lock: 00000000b2e83690/0xa40083ab9bf52d8b lrc: 2/0,0 mode: PW/PW res: [0x2c0000400:0x35b:0x0].0x0 rrc: 2 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x40010480000000 nid: local remote: 0x0 expref: -99 pid: 20759 timeout: 0 lvb_type: 0 00010000:00010000:1.0:1776178092.600086:0:6245:0:(ldlm_lockd.c:1893:ldlm_handle_bl_callback()) Lock 00000000b2e83690 already unused, calling callback (0000000063c97258) 00010000:00010000:1.0:1776178092.600090:0:6245:0:(ldlm_request.c:364:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0001_UUID lock: 00000000b2e83690/0xa40083ab9bf52d8b lrc: 2/0,0 mode: PW/PW res: [0x2c0000400:0x35b:0x0].0x0 rrc: 2 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x40010480000000 nid: local remote: 0x0 expref: -99 pid: 20759 timeout: 0 lvb_type: 0 00010000:00010000:1.0:1776178092.600099:0:6245:0:(ldlm_request.c:1257:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0001_UUID lock: 00000000b2e83690/0xa40083ab9bf52d8b lrc: 3/0,0 mode: PW/PW res: [0x2c0000400:0x35b:0x0].0x0 rrc: 2 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x40018480000000 nid: local remote: 0x0 expref: -99 pid: 20759 timeout: 0 lvb_type: 0 00010000:00010000:1.0:1776178092.600112:0:6245:0:(ldlm_lockd.c:1903:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0001_UUID lock: 00000000b2e83690/0xa40083ab9bf52d8b lrc: 1/0,0 mode: --/PW res: [0x2c0000400:0x35b:0x0].0x0 rrc: 2 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x44819480000000 nid: local remote: 0x0 expref: -99 pid: 20759 timeout: 0 lvb_type: 0 00010000:00010000:1.0:1776178092.600118:0:6245:0:(ldlm_lock.c:216:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0001_UUID lock: 00000000b2e83690/0xa40083ab9bf52d8b lrc: 0/0,0 mode: --/PW res: [0x2c0000400:0x35b:0x0].0x0 rrc: 2 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x44819480000000 nid: local remote: 0x0 expref: -99 pid: 20759 timeout: 0 lvb_type: 0 00010000:00010000:1.0:1776178092.600134:0:6245:0:(ldlm_lockd.c:2789:ldlm_bl_get_work()) Got 1 locks of 0 total in blp. (0 blwis in pool) 00010000:00010000:1.0:1776178092.600136:0:6245:0:(ldlm_lockd.c:1879:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0001_UUID lock: 00000000daa24072/0xa40083ab9bf52d99 lrc: 2/0,0 mode: PW/PW res: [0x2c0000400:0x26b:0x0].0x0 rrc: 2 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x40010480000000 nid: local remote: 0x0 expref: -99 pid: 20766 timeout: 0 lvb_type: 0 00010000:00010000:1.0:1776178092.600142:0:6245:0:(ldlm_lockd.c:1893:ldlm_handle_bl_callback()) Lock 00000000daa24072 already unused, calling callback (0000000063c97258) 00010000:00010000:1.0:1776178092.600144:0:6245:0:(ldlm_request.c:364:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0001_UUID lock: 00000000daa24072/0xa40083ab9bf52d99 lrc: 2/0,0 mode: PW/PW res: [0x2c0000400:0x26b:0x0].0x0 rrc: 2 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x40010480000000 nid: local remote: 0x0 expref: -99 pid: 20766 timeout: 0 lvb_type: 0 00010000:00010000:1.0:1776178092.600150:0:6245:0:(ldlm_request.c:1257:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0001_UUID lock: 00000000daa24072/0xa40083ab9bf52d99 lrc: 3/0,0 mode: PW/PW res: [0x2c0000400:0x26b:0x0].0x0 rrc: 2 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x40018480000000 nid: local remote: 0x0 expref: -99 pid: 20766 timeout: 0 lvb_type: 0 00010000:00010000:1.0:1776178092.600158:0:6245:0:(ldlm_lockd.c:1903:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0001_UUID lock: 00000000daa24072/0xa40083ab9bf52d99 lrc: 1/0,0 mode: --/PW res: [0x2c0000400:0x26b:0x0].0x0 rrc: 2 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x44819480000000 nid: local remote: 0x0 expref: -99 pid: 20766 timeout: 0 lvb_type: 0 00010000:00010000:1.0:1776178092.600164:0:6245:0:(ldlm_lock.c:216:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0001_UUID lock: 00000000daa24072/0xa40083ab9bf52d99 lrc: 0/0,0 mode: --/PW res: [0x2c0000400:0x26b:0x0].0x0 rrc: 2 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x44819480000000 nid: local remote: 0x0 expref: -99 pid: 20766 timeout: 0 lvb_type: 0 00010000:00010000:1.0:1776178092.600188:0:6245:0:(ldlm_lockd.c:2792:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00010000:00010000:1.0:1776178092.600190:0:6245:0:(ldlm_lockd.c:2792:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00010000:00010000:1.0:1776178092.600191:0:6245:0:(ldlm_lockd.c:2792:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00010000:00010000:1.0:1776178092.600216:0:20763:0:(ldlm_lock.c:216:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0001_UUID lock: 000000003ed46f13/0xa40083ab9bf36901 lrc: 0/0,0 mode: --/PR res: [0x2c0000400:0x2c3:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x44801400010020 nid: 192.168.204.3@tcp remote: 0x771f68aa1eb23d6f expref: 14 pid: 8645 timeout: 899 lvb_type: 1 00000100:00100000:1.0:1776178092.600227:0:20763:0:(client.c:2215:ptlrpc_check_set()) Completed RPC req@00000000456acdaf pname:cluuid:pid:xid:nid:opc:job ll_ost00_014:lustre-OST0001_UUID:20763:1862456998015808:192.168.204.3@tcp:104: 00010000:00010000:1.0:1776178092.600239:0:20763:0:(ldlm_request.c:283:ldlm_completion_ast()) ### client-side enqueue returned a blocked locksleeping ns: filter-lustre-OST0001_UUID lock: 000000007da3b625/0xa40083ab9bf52d6f lrc: 3/0,1 mode: PW/PW res: [0x2c0000400:0x2c3:0x0].0x0 rrc: 2 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x40010080000000 nid: local remote: 0x0 expref: -99 pid: 20763 timeout: 0 lvb_type: 0 00010000:00010000:1.0:1776178092.600246:0:20763:0:(ldlm_request.c:199:ldlm_completion_tail()) ### client-side enqueue: granted ns: filter-lustre-OST0001_UUID lock: 000000007da3b625/0xa40083ab9bf52d6f lrc: 3/0,1 mode: PW/PW res: [0x2c0000400:0x2c3:0x0].0x0 rrc: 2 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x40010080000000 nid: local remote: 0x0 expref: -99 pid: 20763 timeout: 0 lvb_type: 0 00010000:00010000:1.0:1776178092.600251:0:20763:0:(ldlm_request.c:528:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0001_UUID lock: 000000007da3b625/0xa40083ab9bf52d6f lrc: 3/0,1 mode: PW/PW res: [0x2c0000400:0x2c3:0x0].0x0 rrc: 2 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x40010080000000 nid: local remote: 0x0 expref: -99 pid: 20763 timeout: 0 lvb_type: 0 00010000:00010000:1.0:1776178092.600258:0:20763:0:(ldlm_lock.c:820:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0001_UUID lock: 000000007da3b625/0xa40083ab9bf52d6f lrc: 3/0,1 mode: PW/PW res: [0x2c0000400:0x2c3:0x0].0x0 rrc: 2 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x50010080000000 nid: local remote: 0x0 expref: -99 pid: 20763 timeout: 0 lvb_type: 0 00010000:00010000:1.0:1776178092.600262:0:20763:0:(ldlm_lock.c:879:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0001_UUID lock: 000000007da3b625/0xa40083ab9bf52d6f lrc: 2/0,0 mode: PW/PW res: [0x2c0000400:0x2c3:0x0].0x0 rrc: 2 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x50010480000000 nid: local remote: 0x0 expref: -99 pid: 20763 timeout: 0 lvb_type: 0 00010000:00010000:1.0:1776178092.600285:0:20763:0:(ldlm_lockd.c:2159:__ldlm_bl_to_thread()) added 1/1 locks to regular blp list, 1 blwis in pool 00000100:00100000:1.0:1776178092.600387:0:20763:0:(service.c:2372:ptlrpc_server_handle_request()) Handled RPC req@0000000024989815 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_014:lustre-MDT0001-mdtlov_UUID+38:3354:x1862456998015616:12345-0@lo:6:osp-syn-1-1.0 Request processed in 42536us (42831us total) trans 4294969304 rc 0/0 00000100:00100000:1.0:1776178092.600394:0:20763:0:(nrs_fifo.c:241:nrs_fifo_req_stop()) NRS stop fifo request from 12345-0@lo, seq: 5199 00010000:00010000:1.0:1776178092.600416:0:20761:0:(ldlm_request.c:199:ldlm_completion_tail()) ### client-side enqueue: granted ns: filter-lustre-OST0000_UUID lock: 000000002f1f7bd5/0xa40083ab9bf52dbc lrc: 3/0,1 mode: PW/PW res: [0x280000400:0x2da:0x0].0x0 rrc: 2 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x40010080000000 nid: local remote: 0x0 expref: -99 pid: 20761 timeout: 0 lvb_type: 0 00010000:00010000:1.0:1776178092.600424:0:20761:0:(ldlm_request.c:528:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 000000002f1f7bd5/0xa40083ab9bf52dbc lrc: 3/0,1 mode: PW/PW res: [0x280000400:0x2da:0x0].0x0 rrc: 2 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x40010080000000 nid: local remote: 0x0 expref: -99 pid: 20761 timeout: 0 lvb_type: 0 00010000:00010000:1.0:1776178092.600431:0:20761:0:(ldlm_lock.c:820:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0000_UUID lock: 000000002f1f7bd5/0xa40083ab9bf52dbc lrc: 3/0,1 mode: PW/PW res: [0x280000400:0x2da:0x0].0x0 rrc: 2 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x50010080000000 nid: local remote: 0x0 expref: -99 pid: 20761 timeout: 0 lvb_type: 0 00010000:00010000:1.0:1776178092.600437:0:20761:0:(ldlm_lock.c:879:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 000000002f1f7bd5/0xa40083ab9bf52dbc lrc: 2/0,0 mode: PW/PW res: [0x280000400:0x2da:0x0].0x0 rrc: 2 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x50010480000000 nid: local remote: 0x0 expref: -99 pid: 20761 timeout: 0 lvb_type: 0 00010000:00010000:1.0:1776178092.600457:0:20761:0:(ldlm_lockd.c:2159:__ldlm_bl_to_thread()) added 1/2 locks to regular blp list, 2 blwis in pool 00000100:00100000:1.0:1776178092.600585:0:20761:0:(service.c:2372:ptlrpc_server_handle_request()) Handled RPC req@000000007a321680 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_012:lustre-MDT0001-mdtlov_UUID+43:3354:x1862456998016768:12345-0@lo:6:osp-syn-0-1.0 Request processed in 39950us (39994us total) trans 4294969325 rc 0/0 00000100:00100000:1.0:1776178092.600592:0:20761:0:(nrs_fifo.c:241:nrs_fifo_req_stop()) NRS stop fifo request from 12345-0@lo, seq: 5210 00010000:00010000:0.0:1776178092.601088:0:7252:0:(ldlm_lockd.c:1782:ldlm_request_cancel()) ### server-side cancel handler END 00010000:00010000:0.0:1776178092.601107:0:6244:0:(ldlm_lockd.c:2789:ldlm_bl_get_work()) Got 1 locks of 0 total in blp. (0 blwis in pool) 00010000:00010000:0.0:1776178092.601111:0:6244:0:(ldlm_lockd.c:1879:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0001_UUID lock: 000000007ee917b5/0xa40083ab9bf52dae lrc: 2/0,0 mode: PW/PW res: [0x2c0000400:0x247:0x0].0x0 rrc: 2 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x40010480000000 nid: local remote: 0x0 expref: -99 pid: 8227 timeout: 0 lvb_type: 0 00010000:00010000:0.0:1776178092.601189:0:6244:0:(ldlm_lockd.c:1893:ldlm_handle_bl_callback()) Lock 000000007ee917b5 already unused, calling callback (0000000063c97258) 00010000:00010000:0.0:1776178092.601192:0:6244:0:(ldlm_request.c:364:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0001_UUID lock: 000000007ee917b5/0xa40083ab9bf52dae lrc: 2/0,0 mode: PW/PW res: [0x2c0000400:0x247:0x0].0x0 rrc: 2 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x40010480000000 nid: local remote: 0x0 expref: -99 pid: 8227 timeout: 0 lvb_type: 0 00010000:00010000:0.0:1776178092.601205:0:6244:0:(ldlm_request.c:1257:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0001_UUID lock: 000000007ee917b5/0xa40083ab9bf52dae lrc: 3/0,0 mode: PW/PW res: [0x2c0000400:0x247:0x0].0x0 rrc: 2 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x40018480000000 nid: local remote: 0x0 expref: -99 pid: 8227 timeout: 0 lvb_type: 0 00010000:00010000:0.0:1776178092.601222:0:6244:0:(ldlm_lockd.c:1903:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0001_UUID lock: 000000007ee917b5/0xa40083ab9bf52dae lrc: 1/0,0 mode: --/PW res: [0x2c0000400:0x247:0x0].0x0 rrc: 2 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x44819480000000 nid: local remote: 0x0 expref: -99 pid: 8227 timeout: 0 lvb_type: 0 00010000:00010000:0.0:1776178092.601228:0:6244:0:(ldlm_lock.c:216:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0001_UUID lock: 000000007ee917b5/0xa40083ab9bf52dae lrc: 0/0,0 mode: --/PW res: [0x2c0000400:0x247:0x0].0x0 rrc: 2 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x44819480000000 nid: local remote: 0x0 expref: -99 pid: 8227 timeout: 0 lvb_type: 0 00010000:00010000:0.0:1776178092.601244:0:6244:0:(ldlm_lockd.c:2792:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00010000:00010000:0.0:1776178092.601248:0:6244:0:(ldlm_lockd.c:2792:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00010000:00010000:0.0:1776178092.601250:0:6244:0:(ldlm_lockd.c:2792:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00010000:00010000:0.0:1776178092.601305:0:8230:0:(ldlm_lock.c:216:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 0000000043c27e7c/0xa40083ab9bf39dea lrc: 0/0,0 mode: --/PR res: [0x280000400:0x36e:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x44801400010020 nid: 192.168.204.3@tcp remote: 0x771f68aa1eb24f12 expref: 18 pid: 8645 timeout: 899 lvb_type: 1 00000100:00100000:0.0:1776178092.601314:0:8230:0:(client.c:2215:ptlrpc_check_set()) Completed RPC req@000000001f0b1867 pname:cluuid:pid:xid:nid:opc:job ll_ost00_002:lustre-OST0000_UUID:8230:1862456998015936:192.168.204.3@tcp:104: 00010000:00010000:0.0:1776178092.601325:0:8230:0:(ldlm_request.c:283:ldlm_completion_ast()) ### client-side enqueue returned a blocked locksleeping ns: filter-lustre-OST0000_UUID lock: 000000007e7ea26f/0xa40083ab9bf52d7d lrc: 3/0,1 mode: PW/PW res: [0x280000400:0x36e:0x0].0x0 rrc: 2 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x40010080000000 nid: local remote: 0x0 expref: -99 pid: 8230 timeout: 0 lvb_type: 0 00010000:00010000:0.0:1776178092.601331:0:8230:0:(ldlm_request.c:199:ldlm_completion_tail()) ### client-side enqueue: granted ns: filter-lustre-OST0000_UUID lock: 000000007e7ea26f/0xa40083ab9bf52d7d lrc: 3/0,1 mode: PW/PW res: [0x280000400:0x36e:0x0].0x0 rrc: 2 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x40010080000000 nid: local remote: 0x0 expref: -99 pid: 8230 timeout: 0 lvb_type: 0 00010000:00010000:0.0:1776178092.601337:0:8230:0:(ldlm_request.c:528:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 000000007e7ea26f/0xa40083ab9bf52d7d lrc: 3/0,1 mode: PW/PW res: [0x280000400:0x36e:0x0].0x0 rrc: 2 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x40010080000000 nid: local remote: 0x0 expref: -99 pid: 8230 timeout: 0 lvb_type: 0 00010000:00010000:0.0:1776178092.601342:0:8230:0:(ldlm_lock.c:820:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0000_UUID lock: 000000007e7ea26f/0xa40083ab9bf52d7d lrc: 3/0,1 mode: PW/PW res: [0x280000400:0x36e:0x0].0x0 rrc: 2 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x50010080000000 nid: local remote: 0x0 expref: -99 pid: 8230 timeout: 0 lvb_type: 0 00010000:00010000:0.0:1776178092.601348:0:8230:0:(ldlm_lock.c:879:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 000000007e7ea26f/0xa40083ab9bf52d7d lrc: 2/0,0 mode: PW/PW res: [0x280000400:0x36e:0x0].0x0 rrc: 2 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x50010480000000 nid: local remote: 0x0 expref: -99 pid: 8230 timeout: 0 lvb_type: 0 00010000:00010000:0.0:1776178092.601365:0:8230:0:(ldlm_lockd.c:2159:__ldlm_bl_to_thread()) added 1/1 locks to regular blp list, 1 blwis in pool 00000100:00100000:0.0:1776178092.602656:0:8230:0:(service.c:2372:ptlrpc_server_handle_request()) Handled RPC req@0000000039ff8335 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_002:lustre-MDT0001-mdtlov_UUID+43:3355:x1862456998015232:12345-0@lo:6:osp-syn-0-1.0 Request processed in 44178us (44406us total) trans 4294969326 rc 0/0 00000100:00100000:0.0:1776178092.602669:0:8230:0:(nrs_fifo.c:241:nrs_fifo_req_stop()) NRS stop fifo request from 12345-0@lo, seq: 5201 00010000:00010000:0.0:1776178092.602733:0:20754:0:(ldlm_lock.c:216:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 00000000a6b36e6d/0xa40083ab9bf3bcec lrc: 0/0,0 mode: --/PR res: [0x280000400:0x3ad:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x44801400010020 nid: 192.168.204.3@tcp remote: 0x771f68aa1eb25968 expref: 17 pid: 8645 timeout: 899 lvb_type: 1 00000100:00100000:0.0:1776178092.602748:0:20754:0:(client.c:2215:ptlrpc_check_set()) Completed RPC req@00000000c1934814 pname:cluuid:pid:xid:nid:opc:job ll_ost00_005:lustre-OST0000_UUID:20754:1862456998017280:192.168.204.3@tcp:104: 00010000:00010000:0.0:1776178092.602770:0:20754:0:(ldlm_request.c:283:ldlm_completion_ast()) ### client-side enqueue returned a blocked locksleeping ns: filter-lustre-OST0000_UUID lock: 000000000b903578/0xa40083ab9bf52dd1 lrc: 3/0,1 mode: PW/PW res: [0x280000400:0x3ad:0x0].0x0 rrc: 2 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x40010080000000 nid: local remote: 0x0 expref: -99 pid: 20754 timeout: 0 lvb_type: 0 00010000:00010000:0.0:1776178092.602780:0:20754:0:(ldlm_request.c:199:ldlm_completion_tail()) ### client-side enqueue: granted ns: filter-lustre-OST0000_UUID lock: 000000000b903578/0xa40083ab9bf52dd1 lrc: 3/0,1 mode: PW/PW res: [0x280000400:0x3ad:0x0].0x0 rrc: 2 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x40010080000000 nid: local remote: 0x0 expref: -99 pid: 20754 timeout: 0 lvb_type: 0 00010000:00010000:0.0:1776178092.602785:0:20754:0:(ldlm_request.c:528:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 000000000b903578/0xa40083ab9bf52dd1 lrc: 3/0,1 mode: PW/PW res: [0x280000400:0x3ad:0x0].0x0 rrc: 2 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x40010080000000 nid: local remote: 0x0 expref: -99 pid: 20754 timeout: 0 lvb_type: 0 00010000:00010000:0.0:1776178092.602792:0:20754:0:(ldlm_lock.c:820:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0000_UUID lock: 000000000b903578/0xa40083ab9bf52dd1 lrc: 3/0,1 mode: PW/PW res: [0x280000400:0x3ad:0x0].0x0 rrc: 2 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x50010080000000 nid: local remote: 0x0 expref: -99 pid: 20754 timeout: 0 lvb_type: 0 00010000:00010000:0.0:1776178092.602797:0:20754:0:(ldlm_lock.c:879:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 000000000b903578/0xa40083ab9bf52dd1 lrc: 2/0,0 mode: PW/PW res: [0x280000400:0x3ad:0x0].0x0 rrc: 2 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x50010480000000 nid: local remote: 0x0 expref: -99 pid: 20754 timeout: 0 lvb_type: 0 00010000:00010000:0.0:1776178092.602821:0:20754:0:(ldlm_lockd.c:2159:__ldlm_bl_to_thread()) added 1/1 locks to regular blp list, 1 blwis in pool 00000100:00100000:0.0:1776178092.603024:0:20754:0:(service.c:2372:ptlrpc_server_handle_request()) Handled RPC req@00000000b5702a7a pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_005:lustre-MDT0001-mdtlov_UUID+43:3354:x1862456998017152:12345-0@lo:6:osp-syn-0-1.0 Request processed in 35289us (36071us total) trans 4294969327 rc 0/0 00000100:00100000:0.0:1776178092.603031:0:20754:0:(nrs_fifo.c:241:nrs_fifo_req_stop()) NRS stop fifo request from 12345-0@lo, seq: 5213 00010000:00010000:0.0:1776178092.603087:0:8228:0:(ldlm_lock.c:216:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 00000000485847f3/0xa40083ab9bf32b3c lrc: 0/0,0 mode: --/PR res: [0x280000400:0x285:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x44801400010020 nid: 192.168.204.3@tcp remote: 0x771f68aa1eb228d8 expref: 16 pid: 8645 timeout: 899 lvb_type: 1 00000100:00100000:0.0:1776178092.603099:0:8228:0:(client.c:2215:ptlrpc_check_set()) Completed RPC req@000000008bb01090 pname:cluuid:pid:xid:nid:opc:job ll_ost00_001:lustre-OST0000_UUID:8228:1862456998017024:192.168.204.3@tcp:104: 00010000:00010000:0.0:1776178092.603114:0:8228:0:(ldlm_request.c:283:ldlm_completion_ast()) ### client-side enqueue returned a blocked locksleeping ns: filter-lustre-OST0000_UUID lock: 00000000319537bc/0xa40083ab9bf52dc3 lrc: 3/0,1 mode: PW/PW res: [0x280000400:0x285:0x0].0x0 rrc: 2 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x40010080000000 nid: local remote: 0x0 expref: -99 pid: 8228 timeout: 0 lvb_type: 0 00010000:00010000:0.0:1776178092.603957:0:8228:0:(ldlm_request.c:199:ldlm_completion_tail()) ### client-side enqueue: granted ns: filter-lustre-OST0000_UUID lock: 00000000319537bc/0xa40083ab9bf52dc3 lrc: 3/0,1 mode: PW/PW res: [0x280000400:0x285:0x0].0x0 rrc: 2 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x40010080000000 nid: local remote: 0x0 expref: -99 pid: 8228 timeout: 0 lvb_type: 0 00010000:00010000:0.0:1776178092.603964:0:8228:0:(ldlm_request.c:528:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 00000000319537bc/0xa40083ab9bf52dc3 lrc: 3/0,1 mode: PW/PW res: [0x280000400:0x285:0x0].0x0 rrc: 2 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x40010080000000 nid: local remote: 0x0 expref: -99 pid: 8228 timeout: 0 lvb_type: 0 00010000:00010000:0.0:1776178092.603972:0:8228:0:(ldlm_lock.c:820:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0000_UUID lock: 00000000319537bc/0xa40083ab9bf52dc3 lrc: 3/0,1 mode: PW/PW res: [0x280000400:0x285:0x0].0x0 rrc: 2 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x50010080000000 nid: local remote: 0x0 expref: -99 pid: 8228 timeout: 0 lvb_type: 0 00010000:00010000:0.0:1776178092.603977:0:8228:0:(ldlm_lock.c:879:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 00000000319537bc/0xa40083ab9bf52dc3 lrc: 2/0,0 mode: PW/PW res: [0x280000400:0x285:0x0].0x0 rrc: 2 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x50010480000000 nid: local remote: 0x0 expref: -99 pid: 8228 timeout: 0 lvb_type: 0 00010000:00010000:0.0:1776178092.603993:0:8228:0:(ldlm_lockd.c:2159:__ldlm_bl_to_thread()) added 1/1 locks to regular blp list, 1 blwis in pool 00000100:00100000:0.0:1776178092.604521:0:8228:0:(service.c:2372:ptlrpc_server_handle_request()) Handled RPC req@00000000812a7ebe pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_001:lustre-MDT0001-mdtlov_UUID+43:3355:x1862456998016960:12345-0@lo:6:osp-syn-0-1.0 Request processed in 42203us (42257us total) trans 4294969328 rc 0/0 00000100:00100000:0.0:1776178092.604531:0:8228:0:(nrs_fifo.c:241:nrs_fifo_req_stop()) NRS stop fifo request from 12345-0@lo, seq: 5211 00000100:00100000:0.0:1776178092.604558:0:8228:0:(nrs_fifo.c:179:nrs_fifo_req_get()) NRS start fifo request from 12345-0@lo, seq: 5216 00000100:00100000:0.0:1776178092.604562:0:8228:0:(service.c:2324:ptlrpc_server_handle_request()) Handling RPC req@000000007886b426 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_001:lustre-MDT0001-mdtlov_UUID+39:3354:x1862456998017600:12345-0@lo:6:osp-syn-1-1.0 00002000:00080000:0.0:1776178092.604580:0:8228:0:(ofd_dev.c:1822:ofd_destroy_hdl()) lustre-OST0001: Destroy object 0x2c0000400:831 count 1 00010000:00010000:0.0:1776178092.604601:0:8228:0:(ldlm_lock.c:766:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0001_UUID lock: 00000000f51fd0d7/0xa40083ab9bf52de6 lrc: 3/0,1 mode: --/PW res: [0x2c0000400:0x33f:0x0].0x0 rrc: 3 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 8228 timeout: 0 lvb_type: 0 00010000:00010000:0.0:1776178092.604612:0:8228:0:(ldlm_lock.c:675:ldlm_add_bl_work_item()) ### lock incompatible; sending blocking AST. ns: filter-lustre-OST0001_UUID lock: 00000000e391698d/0xa40083ab9bf3a609 lrc: 2/0,0 mode: PR/PR res: [0x2c0000400:0x33f:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x40000000000000 nid: 192.168.204.3@tcp remote: 0x771f68aa1eb251c7 expref: 12 pid: 8645 timeout: 0 lvb_type: 1 00010000:00010000:0.0:1776178092.604619:0:8228:0:(ldlm_extent.c:318:ldlm_check_contention()) contended locks = 0 00010000:00010000:0.0:1776178092.604622:0:8228:0:(ldlm_extent.c:318:ldlm_check_contention()) contended locks = 0 00010000:00010000:0.0:1776178092.604624:0:8228:0:(ldlm_resource.c:1650:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0001_UUID lock: 00000000f51fd0d7/0xa40083ab9bf52de6 lrc: 4/0,1 mode: --/PW res: [0x2c0000400:0x33f:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x50010080000000 nid: local remote: 0x0 expref: -99 pid: 8228 timeout: 0 lvb_type: 0 00010000:00010000:0.0:1776178092.604653:0:8228:0:(ldlm_lockd.c:953:ldlm_server_blocking_ast()) ### server preparing blocking AST ns: filter-lustre-OST0001_UUID lock: 00000000e391698d/0xa40083ab9bf3a609 lrc: 3/0,0 mode: PR/PR res: [0x2c0000400:0x33f:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x50000000010020 nid: 192.168.204.3@tcp remote: 0x771f68aa1eb251c7 expref: 12 pid: 8645 timeout: 0 lvb_type: 1 00010000:00010000:0.0:1776178092.604663:0:8228:0:(ldlm_lockd.c:489:ldlm_add_waiting_lock()) ### adding to wait list(timeout: 100, AT: on) ns: filter-lustre-OST0001_UUID lock: 00000000e391698d/0xa40083ab9bf3a609 lrc: 4/0,0 mode: PR/PR res: [0x2c0000400:0x33f:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x70000400010020 nid: 192.168.204.3@tcp remote: 0x771f68aa1eb251c7 expref: 12 pid: 8645 timeout: 899 lvb_type: 1 00000100:00100000:0.0:1776178092.604677:0:8228:0:(client.c:1733:ptlrpc_send_new_req()) Sending RPC req@000000001bbe82bd pname:cluuid:pid:xid:nid:opc:job ll_ost00_001:lustre-OST0001_UUID:8228:1862456998017856:192.168.204.3@tcp:104: 00000100:00100000:0.0:1776178092.604723:0:8228:0:(client.c:2500:ptlrpc_set_wait()) set 000000004d9ad1c9 going to sleep for 43 seconds 00010000:00010000:0.0:1776178092.604742:0:6244:0:(ldlm_lockd.c:2789:ldlm_bl_get_work()) Got 1 locks of 0 total in blp. (0 blwis in pool) 00010000:00010000:0.0:1776178092.604746:0:6244:0:(ldlm_lockd.c:1879:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 00000000319537bc/0xa40083ab9bf52dc3 lrc: 2/0,0 mode: PW/PW res: [0x280000400:0x285:0x0].0x0 rrc: 2 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x40010480000000 nid: local remote: 0x0 expref: -99 pid: 8228 timeout: 0 lvb_type: 0 00010000:00010000:0.0:1776178092.604758:0:6244:0:(ldlm_lockd.c:1893:ldlm_handle_bl_callback()) Lock 00000000319537bc already unused, calling callback (0000000063c97258) 00010000:00010000:0.0:1776178092.604761:0:6244:0:(ldlm_request.c:364:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 00000000319537bc/0xa40083ab9bf52dc3 lrc: 2/0,0 mode: PW/PW res: [0x280000400:0x285:0x0].0x0 rrc: 2 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x40010480000000 nid: local remote: 0x0 expref: -99 pid: 8228 timeout: 0 lvb_type: 0 00010000:00010000:0.0:1776178092.604768:0:6244:0:(ldlm_request.c:1257:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 00000000319537bc/0xa40083ab9bf52dc3 lrc: 3/0,0 mode: PW/PW res: [0x280000400:0x285:0x0].0x0 rrc: 2 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x40018480000000 nid: local remote: 0x0 expref: -99 pid: 8228 timeout: 0 lvb_type: 0 00010000:00010000:0.0:1776178092.604786:0:6244:0:(ldlm_lockd.c:1903:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 00000000319537bc/0xa40083ab9bf52dc3 lrc: 1/0,0 mode: --/PW res: [0x280000400:0x285:0x0].0x0 rrc: 2 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x44819480000000 nid: local remote: 0x0 expref: -99 pid: 8228 timeout: 0 lvb_type: 0 00010000:00010000:0.0:1776178092.604793:0:6244:0:(ldlm_lock.c:216:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 00000000319537bc/0xa40083ab9bf52dc3 lrc: 0/0,0 mode: --/PW res: [0x280000400:0x285:0x0].0x0 rrc: 2 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x44819480000000 nid: local remote: 0x0 expref: -99 pid: 8228 timeout: 0 lvb_type: 0 00010000:00010000:0.0:1776178092.604809:0:6244:0:(ldlm_lockd.c:2792:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00010000:00010000:0.0:1776178092.604810:0:6244:0:(ldlm_lockd.c:2792:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00010000:00010000:0.0:1776178092.604811:0:6244:0:(ldlm_lockd.c:2792:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00010000:00010000:0.0:1776178092.605414:0:20756:0:(ldlm_lock.c:216:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0001_UUID lock: 00000000adf1bb1a/0xa40083ab9bf38deb lrc: 0/0,0 mode: --/PR res: [0x2c0000400:0x30e:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x44801400010020 nid: 192.168.204.3@tcp remote: 0x771f68aa1eb249bd expref: 12 pid: 8227 timeout: 899 lvb_type: 1 00000100:00100000:0.0:1776178092.605428:0:20756:0:(client.c:2215:ptlrpc_check_set()) Completed RPC req@00000000f6af7e5b pname:cluuid:pid:xid:nid:opc:job ll_ost00_007:lustre-OST0001_UUID:20756:1862456998016128:192.168.204.3@tcp:104: 00010000:00010000:0.0:1776178092.605448:0:20756:0:(ldlm_request.c:283:ldlm_completion_ast()) ### client-side enqueue returned a blocked locksleeping ns: filter-lustre-OST0001_UUID lock: 00000000fcb32840/0xa40083ab9bf52d84 lrc: 3/0,1 mode: PW/PW res: [0x2c0000400:0x30e:0x0].0x0 rrc: 2 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x40010080000000 nid: local remote: 0x0 expref: -99 pid: 20756 timeout: 0 lvb_type: 0 00010000:00010000:0.0:1776178092.605455:0:20756:0:(ldlm_request.c:199:ldlm_completion_tail()) ### client-side enqueue: granted ns: filter-lustre-OST0001_UUID lock: 00000000fcb32840/0xa40083ab9bf52d84 lrc: 3/0,1 mode: PW/PW res: [0x2c0000400:0x30e:0x0].0x0 rrc: 2 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x40010080000000 nid: local remote: 0x0 expref: -99 pid: 20756 timeout: 0 lvb_type: 0 00010000:00010000:0.0:1776178092.605465:0:20756:0:(ldlm_request.c:528:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0001_UUID lock: 00000000fcb32840/0xa40083ab9bf52d84 lrc: 3/0,1 mode: PW/PW res: [0x2c0000400:0x30e:0x0].0x0 rrc: 2 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x40010080000000 nid: local remote: 0x0 expref: -99 pid: 20756 timeout: 0 lvb_type: 0 00010000:00010000:0.0:1776178092.605472:0:20756:0:(ldlm_lock.c:820:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0001_UUID lock: 00000000fcb32840/0xa40083ab9bf52d84 lrc: 3/0,1 mode: PW/PW res: [0x2c0000400:0x30e:0x0].0x0 rrc: 2 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x50010080000000 nid: local remote: 0x0 expref: -99 pid: 20756 timeout: 0 lvb_type: 0 00010000:00010000:0.0:1776178092.605477:0:20756:0:(ldlm_lock.c:879:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0001_UUID lock: 00000000fcb32840/0xa40083ab9bf52d84 lrc: 2/0,0 mode: PW/PW res: [0x2c0000400:0x30e:0x0].0x0 rrc: 2 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x50010480000000 nid: local remote: 0x0 expref: -99 pid: 20756 timeout: 0 lvb_type: 0 00010000:00010000:0.0:1776178092.605499:0:20756:0:(ldlm_lockd.c:2159:__ldlm_bl_to_thread()) added 1/1 locks to regular blp list, 1 blwis in pool 00000100:00100000:1.0:1776178092.609030:0:20761:0:(service.c:2152:ptlrpc_server_handle_req_in()) got req x1862456998018624 00000100:00100000:1.0:1776178092.609090:0:20761:0:(nrs_fifo.c:179:nrs_fifo_req_get()) NRS start fifo request from 12345-0@lo, seq: 5224 00000100:00100000:1.0:1776178092.609096:0:20761:0:(service.c:2324:ptlrpc_server_handle_request()) Handling RPC req@00000000228025f2 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_012:lustre-MDT0001-mdtlov_UUID+47:3355:x1862456998018304:12345-0@lo:6:osp-syn-0-1.0 00002000:00080000:1.0:1776178092.609318:0:20761:0:(ofd_dev.c:1822:ofd_destroy_hdl()) lustre-OST0000: Destroy object 0x280000400:1041 count 1 00010000:00010000:1.0:1776178092.609339:0:20761:0:(ldlm_lock.c:766:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0000_UUID lock: 00000000499bab84/0xa40083ab9bf52e1e lrc: 3/0,1 mode: --/PW res: [0x280000400:0x411:0x0].0x0 rrc: 3 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 20761 timeout: 0 lvb_type: 0 00010000:00010000:1.0:1776178092.609356:0:20761:0:(ldlm_lock.c:675:ldlm_add_bl_work_item()) ### lock incompatible; sending blocking AST. ns: filter-lustre-OST0000_UUID lock: 00000000f247d55d/0xa40083ab9bf3ee24 lrc: 2/0,0 mode: PR/PR res: [0x280000400:0x411:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x40000000000000 nid: 192.168.204.3@tcp remote: 0x771f68aa1eb269d0 expref: 14 pid: 8230 timeout: 0 lvb_type: 1 00010000:00010000:1.0:1776178092.609363:0:20761:0:(ldlm_extent.c:318:ldlm_check_contention()) contended locks = 0 00010000:00010000:1.0:1776178092.609365:0:20761:0:(ldlm_extent.c:318:ldlm_check_contention()) contended locks = 0 00010000:00010000:1.0:1776178092.609367:0:20761:0:(ldlm_resource.c:1650:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 00000000499bab84/0xa40083ab9bf52e1e lrc: 4/0,1 mode: --/PW res: [0x280000400:0x411:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x50010080000000 nid: local remote: 0x0 expref: -99 pid: 20761 timeout: 0 lvb_type: 0 00010000:00010000:1.0:1776178092.609396:0:20761:0:(ldlm_lockd.c:953:ldlm_server_blocking_ast()) ### server preparing blocking AST ns: filter-lustre-OST0000_UUID lock: 00000000f247d55d/0xa40083ab9bf3ee24 lrc: 3/0,0 mode: PR/PR res: [0x280000400:0x411:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x50000000010020 nid: 192.168.204.3@tcp remote: 0x771f68aa1eb269d0 expref: 14 pid: 8230 timeout: 0 lvb_type: 1 00010000:00010000:1.0:1776178092.609406:0:20761:0:(ldlm_lockd.c:489:ldlm_add_waiting_lock()) ### adding to wait list(timeout: 100, AT: on) ns: filter-lustre-OST0000_UUID lock: 00000000f247d55d/0xa40083ab9bf3ee24 lrc: 4/0,0 mode: PR/PR res: [0x280000400:0x411:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x70000400010020 nid: 192.168.204.3@tcp remote: 0x771f68aa1eb269d0 expref: 14 pid: 8230 timeout: 899 lvb_type: 1 00000100:00100000:1.0:1776178092.609418:0:20761:0:(client.c:1733:ptlrpc_send_new_req()) Sending RPC req@00000000a30eb964 pname:cluuid:pid:xid:nid:opc:job ll_ost00_012:lustre-OST0000_UUID:20761:1862456998018752:192.168.204.3@tcp:104: 00000100:00100000:1.0:1776178092.609455:0:20761:0:(client.c:2500:ptlrpc_set_wait()) set 00000000b0f5acc3 going to sleep for 43 seconds 00010000:00010000:1.0:1776178092.609472:0:6245:0:(ldlm_lockd.c:2792:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:1.0:1776178092.609486:0:20759:0:(nrs_fifo.c:179:nrs_fifo_req_get()) NRS start fifo request from 12345-0@lo, seq: 5225 00000100:00100000:1.0:1776178092.609490:0:20759:0:(service.c:2324:ptlrpc_server_handle_request()) Handling RPC req@0000000080e4108d pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_010:lustre-MDT0001-mdtlov_UUID+44:3354:x1862456998018624:12345-0@lo:6:osp-syn-1-1.0 00002000:00080000:1.0:1776178092.609503:0:20759:0:(ofd_dev.c:1822:ofd_destroy_hdl()) lustre-OST0001: Destroy object 0x2c0000400:576 count 1 00010000:00010000:1.0:1776178092.609517:0:20759:0:(ldlm_lock.c:766:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0001_UUID lock: 0000000097975a3b/0xa40083ab9bf52e25 lrc: 3/0,1 mode: --/PW res: [0x2c0000400:0x240:0x0].0x0 rrc: 3 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 20759 timeout: 0 lvb_type: 0 00010000:00010000:1.0:1776178092.609528:0:20759:0:(ldlm_lock.c:675:ldlm_add_bl_work_item()) ### lock incompatible; sending blocking AST. ns: filter-lustre-OST0001_UUID lock: 00000000501110f8/0xa40083ab9bf32887 lrc: 2/0,0 mode: PR/PR res: [0x2c0000400:0x240:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x40000000000000 nid: 192.168.204.3@tcp remote: 0x771f68aa1eb227f1 expref: 11 pid: 8228 timeout: 0 lvb_type: 1 00010000:00010000:1.0:1776178092.609534:0:20759:0:(ldlm_extent.c:318:ldlm_check_contention()) contended locks = 0 00010000:00010000:1.0:1776178092.609535:0:20759:0:(ldlm_extent.c:318:ldlm_check_contention()) contended locks = 0 00010000:00010000:1.0:1776178092.609536:0:20759:0:(ldlm_resource.c:1650:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0001_UUID lock: 0000000097975a3b/0xa40083ab9bf52e25 lrc: 4/0,1 mode: --/PW res: [0x2c0000400:0x240:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x50010080000000 nid: local remote: 0x0 expref: -99 pid: 20759 timeout: 0 lvb_type: 0 00010000:00010000:1.0:1776178092.609553:0:20759:0:(ldlm_lockd.c:953:ldlm_server_blocking_ast()) ### server preparing blocking AST ns: filter-lustre-OST0001_UUID lock: 00000000501110f8/0xa40083ab9bf32887 lrc: 3/0,0 mode: PR/PR res: [0x2c0000400:0x240:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x50000000010020 nid: 192.168.204.3@tcp remote: 0x771f68aa1eb227f1 expref: 11 pid: 8228 timeout: 0 lvb_type: 1 00010000:00010000:1.0:1776178092.609565:0:20759:0:(ldlm_lockd.c:489:ldlm_add_waiting_lock()) ### adding to wait list(timeout: 100, AT: on) ns: filter-lustre-OST0001_UUID lock: 00000000501110f8/0xa40083ab9bf32887 lrc: 4/0,0 mode: PR/PR res: [0x2c0000400:0x240:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x70000400010020 nid: 192.168.204.3@tcp remote: 0x771f68aa1eb227f1 expref: 11 pid: 8228 timeout: 899 lvb_type: 1 00000100:00100000:1.0:1776178092.609573:0:20759:0:(client.c:1733:ptlrpc_send_new_req()) Sending RPC req@00000000a59bfc89 pname:cluuid:pid:xid:nid:opc:job ll_ost00_010:lustre-OST0001_UUID:20759:1862456998018816:192.168.204.3@tcp:104: 00000100:00100000:1.0:1776178092.609592:0:20759:0:(client.c:2500:ptlrpc_set_wait()) set 000000007da7ee0c going to sleep for 6 seconds 00000100:00100000:1.0:1776178092.609697:0:3348:0:(events.c:361:request_in_callback()) peer: 12345-192.168.204.3@tcp (source: 12345-192.168.204.3@tcp) 00000100:00100000:1.0:1776178092.609975:0:20764:0:(client.c:2215:ptlrpc_check_set()) Completed RPC req@000000009d117b34 pname:cluuid:pid:xid:nid:opc:job ll_ost00_015:lustre-OST0000_UUID:20764:1862456998017408:192.168.204.3@tcp:104: 00010000:00010000:1.0:1776178092.609996:0:20764:0:(ldlm_request.c:283:ldlm_completion_ast()) ### client-side enqueue returned a blocked locksleeping ns: filter-lustre-OST0000_UUID lock: 0000000015ae4ee6/0xa40083ab9bf52dd8 lrc: 3/0,1 mode: --/PW res: [0x280000400:0x39a:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x40010080000000 nid: local remote: 0x0 expref: -99 pid: 20764 timeout: 0 lvb_type: 0 00010000:00010000:1.0:1776178092.610099:0:20760:0:(ldlm_request.c:199:ldlm_completion_tail()) ### client-side enqueue: granted ns: filter-lustre-OST0000_UUID lock: 000000000396420a/0xa40083ab9bf52da0 lrc: 3/0,1 mode: PW/PW res: [0x280000400:0x300:0x0].0x0 rrc: 2 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x40010080000000 nid: local remote: 0x0 expref: -99 pid: 20760 timeout: 0 lvb_type: 0 00010000:00010000:1.0:1776178092.610108:0:20760:0:(ldlm_request.c:528:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 000000000396420a/0xa40083ab9bf52da0 lrc: 3/0,1 mode: PW/PW res: [0x280000400:0x300:0x0].0x0 rrc: 2 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x40010080000000 nid: local remote: 0x0 expref: -99 pid: 20760 timeout: 0 lvb_type: 0 00010000:00010000:1.0:1776178092.610117:0:20760:0:(ldlm_lock.c:820:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0000_UUID lock: 000000000396420a/0xa40083ab9bf52da0 lrc: 3/0,1 mode: PW/PW res: [0x280000400:0x300:0x0].0x0 rrc: 2 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x50010080000000 nid: local remote: 0x0 expref: -99 pid: 20760 timeout: 0 lvb_type: 0 00010000:00010000:1.0:1776178092.610128:0:20760:0:(ldlm_lock.c:879:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 000000000396420a/0xa40083ab9bf52da0 lrc: 2/0,0 mode: PW/PW res: [0x280000400:0x300:0x0].0x0 rrc: 2 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x50010480000000 nid: local remote: 0x0 expref: -99 pid: 20760 timeout: 0 lvb_type: 0 00010000:00010000:1.0:1776178092.610145:0:20760:0:(ldlm_lockd.c:2159:__ldlm_bl_to_thread()) added 1/1 locks to regular blp list, 1 blwis in pool 00000100:00100000:0.0:1776178092.610658:0:20756:0:(service.c:2372:ptlrpc_server_handle_request()) Handled RPC req@0000000056ddcb09 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_007:lustre-MDT0001-mdtlov_UUID+44:3355:x1862456998015680:12345-0@lo:6:osp-syn-1-1.0 Request processed in 51914us (52370us total) trans 4294969306 rc 0/0 00000100:00100000:0.0:1776178092.610667:0:20756:0:(nrs_fifo.c:241:nrs_fifo_req_stop()) NRS stop fifo request from 12345-0@lo, seq: 5202 00000100:00100000:0.0:1776178092.612331:0:7252:0:(service.c:2372:ptlrpc_server_handle_request()) Handled RPC req@0000000018b5e3c6 pname:cluuid+ref:pid:xid:nid:opc:job ldlm_cn00_003:b50963bd-8eba-4432-9a54-8888d00e3fa1+13:2271:x1862456979073600:12345-192.168.204.3@tcp:103:ldlm_bl_03.0 Request processed in 15448us (16452us total) trans 0 rc 0/0 00000100:00100000:0.0:1776178092.612343:0:7252:0:(nrs_fifo.c:241:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.204.3@tcp, seq: 2046 00010000:00010000:0.0:1776178092.612895:0:9225:0:(ldlm_lockd.c:2789:ldlm_bl_get_work()) Got 1 locks of 0 total in blp. (0 blwis in pool) 00010000:00010000:0.0:1776178092.612901:0:9225:0:(ldlm_lockd.c:1879:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0001_UUID lock: 000000006c464689/0xa40083ab9bf52dca lrc: 2/0,0 mode: PW/PW res: [0x2c0000400:0x308:0x0].0x0 rrc: 2 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x40010480000000 nid: local remote: 0x0 expref: -99 pid: 20757 timeout: 0 lvb_type: 0 00010000:00010000:0.0:1776178092.612912:0:9225:0:(ldlm_lockd.c:1893:ldlm_handle_bl_callback()) Lock 000000006c464689 already unused, calling callback (0000000063c97258) 00010000:00010000:0.0:1776178092.612915:0:9225:0:(ldlm_request.c:364:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0001_UUID lock: 000000006c464689/0xa40083ab9bf52dca lrc: 2/0,0 mode: PW/PW res: [0x2c0000400:0x308:0x0].0x0 rrc: 2 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x40010480000000 nid: local remote: 0x0 expref: -99 pid: 20757 timeout: 0 lvb_type: 0 00010000:00010000:0.0:1776178092.612925:0:9225:0:(ldlm_request.c:1257:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0001_UUID lock: 000000006c464689/0xa40083ab9bf52dca lrc: 3/0,0 mode: PW/PW res: [0x2c0000400:0x308:0x0].0x0 rrc: 2 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x40018480000000 nid: local remote: 0x0 expref: -99 pid: 20757 timeout: 0 lvb_type: 0 00010000:00010000:0.0:1776178092.612954:0:9225:0:(ldlm_lockd.c:1903:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0001_UUID lock: 000000006c464689/0xa40083ab9bf52dca lrc: 1/0,0 mode: --/PW res: [0x2c0000400:0x308:0x0].0x0 rrc: 2 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x44819480000000 nid: local remote: 0x0 expref: -99 pid: 20757 timeout: 0 lvb_type: 0 00010000:00010000:0.0:1776178092.612962:0:9225:0:(ldlm_lock.c:216:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0001_UUID lock: 000000006c464689/0xa40083ab9bf52dca lrc: 0/0,0 mode: --/PW res: [0x2c0000400:0x308:0x0].0x0 rrc: 2 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x44819480000000 nid: local remote: 0x0 expref: -99 pid: 20757 timeout: 0 lvb_type: 0 00010000:00010000:0.0:1776178092.612980:0:9225:0:(ldlm_lockd.c:2792:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00010000:00010000:0.0:1776178092.612982:0:9225:0:(ldlm_lockd.c:2792:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00010000:00010000:0.0:1776178092.612983:0:9225:0:(ldlm_lockd.c:2792:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:0.0:1776178092.613434:0:3346:0:(events.c:361:request_in_callback()) peer: 12345-192.168.204.3@tcp (source: 12345-192.168.204.3@tcp) 00000100:00100000:0.0:1776178092.613463:0:3346:0:(events.c:361:request_in_callback()) peer: 12345-192.168.204.3@tcp (source: 12345-192.168.204.3@tcp) 00000100:00100000:0.0:1776178092.613497:0:3346:0:(events.c:361:request_in_callback()) peer: 12345-192.168.204.3@tcp (source: 12345-192.168.204.3@tcp) 00000100:00100000:0.0:1776178092.613527:0:20770:0:(service.c:2152:ptlrpc_server_handle_req_in()) got req x1862456979074240 00010000:00010000:0.0:1776178092.613556:0:20770:0:(ldlm_lockd.c:2638:ldlm_cancel_hpreq_check()) ### hpreq cancel/convert lock ns: filter-lustre-OST0001_UUID lock: 0000000098fc7f5c/0xa40083ab9bf4173d lrc: 5/0,0 mode: PR/PR res: [0x2c0000400:0x425:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x60000400010020 nid: 192.168.204.3@tcp remote: 0x771f68aa1eb27783 expref: 11 pid: 14440 timeout: 899 lvb_type: 1 00000100:00100000:1.0:1776178092.613564:0:20760:0:(service.c:2372:ptlrpc_server_handle_request()) Handled RPC req@00000000b9a3856b pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_011:lustre-MDT0001-mdtlov_UUID+49:3354:x1862456998016000:12345-0@lo:6:osp-syn-0-1.0 Request processed in 54578us (54863us total) trans 4294969330 rc 0/0 00000100:00100000:1.0:1776178092.613576:0:20760:0:(nrs_fifo.c:241:nrs_fifo_req_stop()) NRS stop fifo request from 12345-0@lo, seq: 5206 00000100:00100000:0.0:1776178092.613592:0:20770:0:(service.c:2152:ptlrpc_server_handle_req_in()) got req x1862456979074304 00010000:00010000:1.0:1776178092.613600:0:20764:0:(ldlm_request.c:199:ldlm_completion_tail()) ### client-side enqueue: granted ns: filter-lustre-OST0000_UUID lock: 0000000015ae4ee6/0xa40083ab9bf52dd8 lrc: 3/0,1 mode: PW/PW res: [0x280000400:0x39a:0x0].0x0 rrc: 2 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x40010080000000 nid: local remote: 0x0 expref: -99 pid: 20764 timeout: 0 lvb_type: 0 00010000:00010000:0.0:1776178092.613606:0:20770:0:(ldlm_lockd.c:2638:ldlm_cancel_hpreq_check()) ### hpreq cancel/convert lock ns: filter-lustre-OST0001_UUID lock: 0000000053542eb4/0xa40083ab9bf3d549 lrc: 5/0,0 mode: PR/PR res: [0x2c0000400:0x39f:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x60000400010020 nid: 192.168.204.3@tcp remote: 0x771f68aa1eb26187 expref: 12 pid: 8230 timeout: 899 lvb_type: 1 00000100:00100000:0.0:1776178092.613632:0:20770:0:(service.c:2152:ptlrpc_server_handle_req_in()) got req x1862456979074560 00010000:00010000:0.0:1776178092.613644:0:20770:0:(ldlm_lockd.c:2638:ldlm_cancel_hpreq_check()) ### hpreq cancel/convert lock ns: filter-lustre-OST0001_UUID lock: 00000000e391698d/0xa40083ab9bf3a609 lrc: 5/0,0 mode: PR/PR res: [0x2c0000400:0x33f:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x60000400010020 nid: 192.168.204.3@tcp remote: 0x771f68aa1eb251c7 expref: 13 pid: 8645 timeout: 899 lvb_type: 1 00000100:00100000:0.0:1776178092.613673:0:20770:0:(nrs_fifo.c:179:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.204.3@tcp, seq: 2059 00000100:00100000:0.0:1776178092.613680:0:20770:0:(service.c:2324:ptlrpc_server_handle_request()) Handling RPC req@0000000088356013 pname:cluuid+ref:pid:xid:nid:opc:job ldlm_cn00_005:b50963bd-8eba-4432-9a54-8888d00e3fa1+13:2272:x1862456979074240:12345-192.168.204.3@tcp:103:ldlm_bl_02.0 00010000:00010000:1.0:1776178092.613690:0:20764:0:(ldlm_request.c:528:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 0000000015ae4ee6/0xa40083ab9bf52dd8 lrc: 3/0,1 mode: PW/PW res: [0x280000400:0x39a:0x0].0x0 rrc: 2 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x40010080000000 nid: local remote: 0x0 expref: -99 pid: 20764 timeout: 0 lvb_type: 0 00010000:00010000:0.0:1776178092.613699:0:20770:0:(ldlm_lockd.c:1720:ldlm_request_cancel()) ### server-side cancel handler START: 1 locks, starting at 0 00010000:00010000:1.0:1776178092.613701:0:20764:0:(ldlm_lock.c:820:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0000_UUID lock: 0000000015ae4ee6/0xa40083ab9bf52dd8 lrc: 3/0,1 mode: PW/PW res: [0x280000400:0x39a:0x0].0x0 rrc: 2 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x50010080000000 nid: local remote: 0x0 expref: -99 pid: 20764 timeout: 0 lvb_type: 0 00010000:00010000:0.0:1776178092.613706:0:20770:0:(ldlm_lockd.c:1771:ldlm_request_cancel()) ### server cancels blocked lock after 0s ns: filter-lustre-OST0001_UUID lock: 0000000098fc7f5c/0xa40083ab9bf4173d lrc: 5/0,0 mode: PR/PR res: [0x2c0000400:0x425:0x0].0x0 rrc: 4 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x60000400010020 nid: 192.168.204.3@tcp remote: 0x771f68aa1eb27783 expref: 13 pid: 14440 timeout: 899 lvb_type: 1 00010000:00010000:1.0:1776178092.613714:0:20764:0:(ldlm_lock.c:879:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 0000000015ae4ee6/0xa40083ab9bf52dd8 lrc: 2/0,0 mode: PW/PW res: [0x280000400:0x39a:0x0].0x0 rrc: 2 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x50010480000000 nid: local remote: 0x0 expref: -99 pid: 20764 timeout: 0 lvb_type: 0 00010000:00010000:0.0:1776178092.613725:0:20770:0:(ldlm_lockd.c:562:ldlm_del_waiting_lock()) ### removed ns: filter-lustre-OST0001_UUID lock: 0000000098fc7f5c/0xa40083ab9bf4173d lrc: 4/0,0 mode: PR/PR res: [0x2c0000400:0x425:0x0].0x0 rrc: 4 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x50000400010020 nid: 192.168.204.3@tcp remote: 0x771f68aa1eb27783 expref: 13 pid: 14440 timeout: 899 lvb_type: 1 00010000:00010000:0.0:1776178092.613749:0:20770:0:(ldlm_extent.c:318:ldlm_check_contention()) contended locks = 0 00010000:00010000:0.0:1776178092.613751:0:20770:0:(ldlm_extent.c:318:ldlm_check_contention()) contended locks = 0 00010000:00010000:0.0:1776178092.613756:0:20770:0:(ldlm_lock.c:704:ldlm_add_cp_work_item()) ### lock granted; sending completion AST. ns: filter-lustre-OST0001_UUID lock: 00000000d3e952db/0xa40083ab9bf52ddf lrc: 3/0,1 mode: PW/PW res: [0x2c0000400:0x425:0x0].0x0 rrc: 4 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x50090080000000 nid: local remote: 0x0 expref: -99 pid: 20754 timeout: 0 lvb_type: 0 00010000:00010000:0.0:1776178092.613767:0:20770:0:(ldlm_resource.c:1650:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0001_UUID lock: 00000000d3e952db/0xa40083ab9bf52ddf lrc: 4/0,1 mode: PW/PW res: [0x2c0000400:0x425:0x0].0x0 rrc: 4 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x50090080000000 nid: local remote: 0x0 expref: -99 pid: 20754 timeout: 0 lvb_type: 0 00010000:00010000:0.0:1776178092.613787:0:20770:0:(ldlm_lockd.c:1782:ldlm_request_cancel()) ### server-side cancel handler END 00000100:00100000:0.0:1776178092.613845:0:20770:0:(service.c:2372:ptlrpc_server_handle_request()) Handled RPC req@0000000088356013 pname:cluuid+ref:pid:xid:nid:opc:job ldlm_cn00_005:b50963bd-8eba-4432-9a54-8888d00e3fa1+13:2272:x1862456979074240:12345-192.168.204.3@tcp:103:ldlm_bl_02.0 Request processed in 165us (414us total) trans 0 rc 0/0 00000100:00100000:0.0:1776178092.613852:0:20770:0:(nrs_fifo.c:241:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.204.3@tcp, seq: 2059 00000100:00100000:0.0:1776178092.613867:0:20770:0:(nrs_fifo.c:179:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.204.3@tcp, seq: 2060 00000100:00100000:0.0:1776178092.613870:0:20770:0:(service.c:2324:ptlrpc_server_handle_request()) Handling RPC req@00000000a5677c3e pname:cluuid+ref:pid:xid:nid:opc:job ldlm_cn00_005:b50963bd-8eba-4432-9a54-8888d00e3fa1+12:2273:x1862456979074304:12345-192.168.204.3@tcp:103:ldlm_bl_02.0 00010000:00010000:0.0:1776178092.613883:0:20770:0:(ldlm_lockd.c:1720:ldlm_request_cancel()) ### server-side cancel handler START: 1 locks, starting at 0 00010000:00010000:0.0:1776178092.613887:0:20770:0:(ldlm_lockd.c:1771:ldlm_request_cancel()) ### server cancels blocked lock after 0s ns: filter-lustre-OST0001_UUID lock: 0000000053542eb4/0xa40083ab9bf3d549 lrc: 5/0,0 mode: PR/PR res: [0x2c0000400:0x39f:0x0].0x0 rrc: 4 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x60000400010020 nid: 192.168.204.3@tcp remote: 0x771f68aa1eb26187 expref: 12 pid: 8230 timeout: 899 lvb_type: 1 00010000:00010000:0.0:1776178092.613896:0:20770:0:(ldlm_lockd.c:562:ldlm_del_waiting_lock()) ### removed ns: filter-lustre-OST0001_UUID lock: 0000000053542eb4/0xa40083ab9bf3d549 lrc: 4/0,0 mode: PR/PR res: [0x2c0000400:0x39f:0x0].0x0 rrc: 4 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x50000400010020 nid: 192.168.204.3@tcp remote: 0x771f68aa1eb26187 expref: 12 pid: 8230 timeout: 899 lvb_type: 1 00010000:00010000:0.0:1776178092.613910:0:20770:0:(ldlm_extent.c:318:ldlm_check_contention()) contended locks = 0 00010000:00010000:0.0:1776178092.613912:0:20770:0:(ldlm_extent.c:318:ldlm_check_contention()) contended locks = 0 00010000:00010000:0.0:1776178092.613914:0:20770:0:(ldlm_lock.c:704:ldlm_add_cp_work_item()) ### lock granted; sending completion AST. ns: filter-lustre-OST0001_UUID lock: 00000000a71646e9/0xa40083ab9bf52ded lrc: 3/0,1 mode: PW/PW res: [0x2c0000400:0x39f:0x0].0x0 rrc: 4 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x50090080000000 nid: local remote: 0x0 expref: -99 pid: 8645 timeout: 0 lvb_type: 0 00010000:00010000:0.0:1776178092.613921:0:20770:0:(ldlm_resource.c:1650:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0001_UUID lock: 00000000a71646e9/0xa40083ab9bf52ded lrc: 4/0,1 mode: PW/PW res: [0x2c0000400:0x39f:0x0].0x0 rrc: 4 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x50090080000000 nid: local remote: 0x0 expref: -99 pid: 8645 timeout: 0 lvb_type: 0 00010000:00010000:0.0:1776178092.613933:0:20770:0:(ldlm_lockd.c:1782:ldlm_request_cancel()) ### server-side cancel handler END 00000100:00100000:0.0:1776178092.613953:0:20770:0:(service.c:2372:ptlrpc_server_handle_request()) Handled RPC req@00000000a5677c3e pname:cluuid+ref:pid:xid:nid:opc:job ldlm_cn00_005:b50963bd-8eba-4432-9a54-8888d00e3fa1+12:2273:x1862456979074304:12345-192.168.204.3@tcp:103:ldlm_bl_02.0 Request processed in 81us (493us total) trans 0 rc 0/0 00000100:00100000:0.0:1776178092.613959:0:20770:0:(nrs_fifo.c:241:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.204.3@tcp, seq: 2060 00000100:00100000:0.0:1776178092.613976:0:20770:0:(nrs_fifo.c:179:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.204.3@tcp, seq: 2061 00000100:00100000:0.0:1776178092.613981:0:20770:0:(service.c:2324:ptlrpc_server_handle_request()) Handling RPC req@000000002ebdbed7 pname:cluuid+ref:pid:xid:nid:opc:job ldlm_cn00_005:b50963bd-8eba-4432-9a54-8888d00e3fa1+11:2271:x1862456979074560:12345-192.168.204.3@tcp:103:ldlm_bl_01.0 00010000:00010000:0.0:1776178092.613989:0:20770:0:(ldlm_lockd.c:1720:ldlm_request_cancel()) ### server-side cancel handler START: 1 locks, starting at 0 00010000:00010000:0.0:1776178092.613993:0:20770:0:(ldlm_lockd.c:1771:ldlm_request_cancel()) ### server cancels blocked lock after 0s ns: filter-lustre-OST0001_UUID lock: 00000000e391698d/0xa40083ab9bf3a609 lrc: 5/0,0 mode: PR/PR res: [0x2c0000400:0x33f:0x0].0x0 rrc: 4 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x60000400010020 nid: 192.168.204.3@tcp remote: 0x771f68aa1eb251c7 expref: 11 pid: 8645 timeout: 899 lvb_type: 1 00010000:00010000:0.0:1776178092.614002:0:20770:0:(ldlm_lockd.c:562:ldlm_del_waiting_lock()) ### removed ns: filter-lustre-OST0001_UUID lock: 00000000e391698d/0xa40083ab9bf3a609 lrc: 4/0,0 mode: PR/PR res: [0x2c0000400:0x33f:0x0].0x0 rrc: 4 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x50000400010020 nid: 192.168.204.3@tcp remote: 0x771f68aa1eb251c7 expref: 11 pid: 8645 timeout: 899 lvb_type: 1 00010000:00010000:0.0:1776178092.614015:0:20770:0:(ldlm_extent.c:318:ldlm_check_contention()) contended locks = 0 00010000:00010000:0.0:1776178092.614016:0:20770:0:(ldlm_extent.c:318:ldlm_check_contention()) contended locks = 0 00010000:00010000:0.0:1776178092.614018:0:20770:0:(ldlm_lock.c:704:ldlm_add_cp_work_item()) ### lock granted; sending completion AST. ns: filter-lustre-OST0001_UUID lock: 00000000f51fd0d7/0xa40083ab9bf52de6 lrc: 3/0,1 mode: PW/PW res: [0x2c0000400:0x33f:0x0].0x0 rrc: 4 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x50090080000000 nid: local remote: 0x0 expref: -99 pid: 8228 timeout: 0 lvb_type: 0 00010000:00010000:0.0:1776178092.614029:0:20770:0:(ldlm_resource.c:1650:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0001_UUID lock: 00000000f51fd0d7/0xa40083ab9bf52de6 lrc: 4/0,1 mode: PW/PW res: [0x2c0000400:0x33f:0x0].0x0 rrc: 4 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x50090080000000 nid: local remote: 0x0 expref: -99 pid: 8228 timeout: 0 lvb_type: 0 00010000:00010000:0.0:1776178092.614041:0:20770:0:(ldlm_lockd.c:1782:ldlm_request_cancel()) ### server-side cancel handler END 00000100:00100000:0.0:1776178092.614087:0:20770:0:(service.c:2372:ptlrpc_server_handle_request()) Handled RPC req@000000002ebdbed7 pname:cluuid+ref:pid:xid:nid:opc:job ldlm_cn00_005:b50963bd-8eba-4432-9a54-8888d00e3fa1+11:2271:x1862456979074560:12345-192.168.204.3@tcp:103:ldlm_bl_01.0 Request processed in 105us (593us total) trans 0 rc 0/0 00000100:00100000:0.0:1776178092.614094:0:20770:0:(nrs_fifo.c:241:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.204.3@tcp, seq: 2061 00000100:00100000:1.0:1776178092.616217:0:20767:0:(service.c:2152:ptlrpc_server_handle_req_in()) got req x1862456979074368 00010000:00010000:1.0:1776178092.616254:0:20767:0:(ldlm_lockd.c:2638:ldlm_cancel_hpreq_check()) ### hpreq cancel/convert lock ns: filter-lustre-OST0000_UUID lock: 000000006cc7b751/0xa40083ab9bf3a9ba lrc: 4/0,0 mode: PR/PR res: [0x280000400:0x386:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x60000400010020 nid: 192.168.204.3@tcp remote: 0x771f68aa1eb25302 expref: 14 pid: 8645 timeout: 899 lvb_type: 1 00000100:00100000:1.0:1776178092.616286:0:20767:0:(service.c:2152:ptlrpc_server_handle_req_in()) got req x1862456979074496 00000100:00100000:0.0:1776178092.616450:0:20770:0:(service.c:2152:ptlrpc_server_handle_req_in()) got req x1862456979074432 00010000:00010000:0.0:1776178092.616468:0:20770:0:(ldlm_lockd.c:2638:ldlm_cancel_hpreq_check()) ### hpreq cancel/convert lock ns: filter-lustre-OST0001_UUID lock: 0000000071df586c/0xa40083ab9bf3460f lrc: 5/0,0 mode: PR/PR res: [0x2c0000400:0x27c:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x60000400010020 nid: 192.168.204.3@tcp remote: 0x771f68aa1eb231c9 expref: 11 pid: 8645 timeout: 899 lvb_type: 1 00000100:00100000:0.0:1776178092.616490:0:20770:0:(nrs_fifo.c:179:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.204.3@tcp, seq: 2062 00000100:00100000:0.0:1776178092.616497:0:20770:0:(service.c:2324:ptlrpc_server_handle_request()) Handling RPC req@00000000803dbce0 pname:cluuid+ref:pid:xid:nid:opc:job ldlm_cn00_005:b50963bd-8eba-4432-9a54-8888d00e3fa1+15:2272:x1862456979074368:12345-192.168.204.3@tcp:103:ldlm_bl_02.0 00010000:00010000:0.0:1776178092.616513:0:20770:0:(ldlm_lockd.c:1720:ldlm_request_cancel()) ### server-side cancel handler START: 1 locks, starting at 0 00010000:00010000:0.0:1776178092.616520:0:20770:0:(ldlm_lockd.c:1771:ldlm_request_cancel()) ### server cancels blocked lock after 0s ns: filter-lustre-OST0000_UUID lock: 000000006cc7b751/0xa40083ab9bf3a9ba lrc: 4/0,0 mode: PR/PR res: [0x280000400:0x386:0x0].0x0 rrc: 4 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x60000400010020 nid: 192.168.204.3@tcp remote: 0x771f68aa1eb25302 expref: 15 pid: 8645 timeout: 899 lvb_type: 1 00010000:00010000:0.0:1776178092.616532:0:20770:0:(ldlm_lockd.c:562:ldlm_del_waiting_lock()) ### removed ns: filter-lustre-OST0000_UUID lock: 000000006cc7b751/0xa40083ab9bf3a9ba lrc: 3/0,0 mode: PR/PR res: [0x280000400:0x386:0x0].0x0 rrc: 4 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x50000400010020 nid: 192.168.204.3@tcp remote: 0x771f68aa1eb25302 expref: 15 pid: 8645 timeout: 899 lvb_type: 1 00010000:00010000:0.0:1776178092.616562:0:20770:0:(ldlm_lock.c:216:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 000000006cc7b751/0xa40083ab9bf3a9ba lrc: 0/0,0 mode: --/PR res: [0x280000400:0x386:0x0].0x0 rrc: 4 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x44801400010020 nid: 192.168.204.3@tcp remote: 0x771f68aa1eb25302 expref: 15 pid: 8645 timeout: 899 lvb_type: 1 00010000:00010000:0.0:1776178092.616582:0:20770:0:(ldlm_extent.c:318:ldlm_check_contention()) contended locks = 0 00010000:00010000:0.0:1776178092.616585:0:20770:0:(ldlm_extent.c:318:ldlm_check_contention()) contended locks = 0 00010000:00010000:0.0:1776178092.616591:0:20770:0:(ldlm_lock.c:704:ldlm_add_cp_work_item()) ### lock granted; sending completion AST. ns: filter-lustre-OST0000_UUID lock: 00000000801fce1a/0xa40083ab9bf52df4 lrc: 3/0,1 mode: PW/PW res: [0x280000400:0x386:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x40090080000000 nid: local remote: 0x0 expref: -99 pid: 20758 timeout: 0 lvb_type: 0 00010000:00010000:0.0:1776178092.616604:0:20770:0:(ldlm_resource.c:1650:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 00000000801fce1a/0xa40083ab9bf52df4 lrc: 4/0,1 mode: PW/PW res: [0x280000400:0x386:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x40090080000000 nid: local remote: 0x0 expref: -99 pid: 20758 timeout: 0 lvb_type: 0 00010000:00010000:0.0:1776178092.616644:0:20770:0:(ldlm_lockd.c:1782:ldlm_request_cancel()) ### server-side cancel handler END 00000100:00100000:0.0:1776178092.616706:0:20770:0:(service.c:2372:ptlrpc_server_handle_request()) Handled RPC req@00000000803dbce0 pname:cluuid+ref:pid:xid:nid:opc:job ldlm_cn00_005:b50963bd-8eba-4432-9a54-8888d00e3fa1+14:2272:x1862456979074368:12345-192.168.204.3@tcp:103:ldlm_bl_02.0 Request processed in 208us (771us total) trans 0 rc 0/0 00000100:00100000:0.0:1776178092.616717:0:20770:0:(nrs_fifo.c:241:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.204.3@tcp, seq: 2062 00010000:00010000:1.0:1776178092.616723:0:20767:0:(ldlm_lockd.c:2638:ldlm_cancel_hpreq_check()) ### hpreq cancel/convert lock ns: filter-lustre-OST0000_UUID lock: 000000002cc2995d/0xa40083ab9bf3bbf0 lrc: 4/0,0 mode: PR/PR res: [0x280000400:0x3ab:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x60000400010020 nid: 192.168.204.3@tcp remote: 0x771f68aa1eb25914 expref: 14 pid: 8227 timeout: 899 lvb_type: 1 00000100:00100000:0.0:1776178092.616741:0:20770:0:(nrs_fifo.c:179:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.204.3@tcp, seq: 2063 00000100:00100000:1.0:1776178092.616750:0:20767:0:(nrs_fifo.c:179:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.204.3@tcp, seq: 2064 00000100:00100000:0.0:1776178092.616751:0:20770:0:(service.c:2324:ptlrpc_server_handle_request()) Handling RPC req@000000009d00d508 pname:cluuid+ref:pid:xid:nid:opc:job ldlm_cn00_005:b50963bd-8eba-4432-9a54-8888d00e3fa1+11:2272:x1862456979074432:12345-192.168.204.3@tcp:103:ldlm_bl_02.0 00000100:00100000:1.0:1776178092.616759:0:20767:0:(service.c:2324:ptlrpc_server_handle_request()) Handling RPC req@00000000ca033a09 pname:cluuid+ref:pid:xid:nid:opc:job ldlm_cn00_004:b50963bd-8eba-4432-9a54-8888d00e3fa1+13:2272:x1862456979074496:12345-192.168.204.3@tcp:103:ldlm_bl_02.0 00010000:00010000:0.0:1776178092.616765:0:20770:0:(ldlm_lockd.c:1720:ldlm_request_cancel()) ### server-side cancel handler START: 1 locks, starting at 0 00010000:00010000:1.0:1776178092.616770:0:20767:0:(ldlm_lockd.c:1720:ldlm_request_cancel()) ### server-side cancel handler START: 1 locks, starting at 0 00010000:00010000:0.0:1776178092.616772:0:20770:0:(ldlm_lockd.c:1771:ldlm_request_cancel()) ### server cancels blocked lock after 0s ns: filter-lustre-OST0001_UUID lock: 0000000071df586c/0xa40083ab9bf3460f lrc: 5/0,0 mode: PR/PR res: [0x2c0000400:0x27c:0x0].0x0 rrc: 4 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x60000400010020 nid: 192.168.204.3@tcp remote: 0x771f68aa1eb231c9 expref: 11 pid: 8645 timeout: 899 lvb_type: 1 00010000:00010000:1.0:1776178092.616776:0:20767:0:(ldlm_lockd.c:1771:ldlm_request_cancel()) ### server cancels blocked lock after 0s ns: filter-lustre-OST0000_UUID lock: 000000002cc2995d/0xa40083ab9bf3bbf0 lrc: 4/0,0 mode: PR/PR res: [0x280000400:0x3ab:0x0].0x0 rrc: 4 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x60000400010020 nid: 192.168.204.3@tcp remote: 0x771f68aa1eb25914 expref: 13 pid: 8227 timeout: 899 lvb_type: 1 00010000:00010000:0.0:1776178092.616791:0:20770:0:(ldlm_lockd.c:562:ldlm_del_waiting_lock()) ### removed ns: filter-lustre-OST0001_UUID lock: 0000000071df586c/0xa40083ab9bf3460f lrc: 4/0,0 mode: PR/PR res: [0x2c0000400:0x27c:0x0].0x0 rrc: 4 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x50000400010020 nid: 192.168.204.3@tcp remote: 0x771f68aa1eb231c9 expref: 11 pid: 8645 timeout: 899 lvb_type: 1 00010000:00010000:1.0:1776178092.616805:0:20767:0:(ldlm_lockd.c:562:ldlm_del_waiting_lock()) ### removed ns: filter-lustre-OST0000_UUID lock: 000000002cc2995d/0xa40083ab9bf3bbf0 lrc: 3/0,0 mode: PR/PR res: [0x280000400:0x3ab:0x0].0x0 rrc: 4 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x50000400010020 nid: 192.168.204.3@tcp remote: 0x771f68aa1eb25914 expref: 13 pid: 8227 timeout: 899 lvb_type: 1 00010000:00010000:0.0:1776178092.616813:0:20770:0:(ldlm_extent.c:318:ldlm_check_contention()) contended locks = 0 00010000:00010000:0.0:1776178092.616816:0:20770:0:(ldlm_extent.c:318:ldlm_check_contention()) contended locks = 0 00010000:00010000:0.0:1776178092.616819:0:20770:0:(ldlm_lock.c:704:ldlm_add_cp_work_item()) ### lock granted; sending completion AST. ns: filter-lustre-OST0001_UUID lock: 000000007e5ef416/0xa40083ab9bf52dfb lrc: 3/0,1 mode: PW/PW res: [0x2c0000400:0x27c:0x0].0x0 rrc: 4 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x50090080000000 nid: local remote: 0x0 expref: -99 pid: 20755 timeout: 0 lvb_type: 0 00010000:00010000:1.0:1776178092.616836:0:20767:0:(ldlm_lock.c:216:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 000000002cc2995d/0xa40083ab9bf3bbf0 lrc: 0/0,0 mode: --/PR res: [0x280000400:0x3ab:0x0].0x0 rrc: 4 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x44801400010020 nid: 192.168.204.3@tcp remote: 0x771f68aa1eb25914 expref: 13 pid: 8227 timeout: 899 lvb_type: 1 00010000:00010000:0.0:1776178092.616838:0:20770:0:(ldlm_resource.c:1650:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0001_UUID lock: 000000007e5ef416/0xa40083ab9bf52dfb lrc: 4/0,1 mode: PW/PW res: [0x2c0000400:0x27c:0x0].0x0 rrc: 4 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x50090080000000 nid: local remote: 0x0 expref: -99 pid: 20755 timeout: 0 lvb_type: 0 00010000:00010000:1.0:1776178092.616860:0:20767:0:(ldlm_extent.c:318:ldlm_check_contention()) contended locks = 0 00010000:00010000:1.0:1776178092.616864:0:20767:0:(ldlm_extent.c:318:ldlm_check_contention()) contended locks = 0 00010000:00010000:0.0:1776178092.616865:0:20770:0:(ldlm_lockd.c:1782:ldlm_request_cancel()) ### server-side cancel handler END 00010000:00010000:1.0:1776178092.616867:0:20767:0:(ldlm_lock.c:704:ldlm_add_cp_work_item()) ### lock granted; sending completion AST. ns: filter-lustre-OST0000_UUID lock: 00000000b27cf00e/0xa40083ab9bf52e02 lrc: 3/0,1 mode: PW/PW res: [0x280000400:0x3ab:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x40090080000000 nid: local remote: 0x0 expref: -99 pid: 14440 timeout: 0 lvb_type: 0 00010000:00010000:1.0:1776178092.616875:0:20767:0:(ldlm_resource.c:1650:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 00000000b27cf00e/0xa40083ab9bf52e02 lrc: 4/0,1 mode: PW/PW res: [0x280000400:0x3ab:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x40090080000000 nid: local remote: 0x0 expref: -99 pid: 14440 timeout: 0 lvb_type: 0 00000100:00100000:0.0:1776178092.616888:0:20770:0:(service.c:2372:ptlrpc_server_handle_request()) Handled RPC req@000000009d00d508 pname:cluuid+ref:pid:xid:nid:opc:job ldlm_cn00_005:b50963bd-8eba-4432-9a54-8888d00e3fa1+11:2272:x1862456979074432:12345-192.168.204.3@tcp:103:ldlm_bl_02.0 Request processed in 138us (927us total) trans 0 rc 0/0 00000100:00100000:0.0:1776178092.616896:0:20770:0:(nrs_fifo.c:241:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.204.3@tcp, seq: 2063 00010000:00010000:1.0:1776178092.616899:0:20767:0:(ldlm_lockd.c:1782:ldlm_request_cancel()) ### server-side cancel handler END 00000100:00100000:1.0:1776178092.616922:0:20767:0:(service.c:2372:ptlrpc_server_handle_request()) Handled RPC req@00000000ca033a09 pname:cluuid+ref:pid:xid:nid:opc:job ldlm_cn00_004:b50963bd-8eba-4432-9a54-8888d00e3fa1+12:2272:x1862456979074496:12345-192.168.204.3@tcp:103:ldlm_bl_02.0 Request processed in 165us (973us total) trans 0 rc 0/0 00000100:00100000:1.0:1776178092.616930:0:20767:0:(nrs_fifo.c:241:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.204.3@tcp, seq: 2064 00000100:00100000:1.0:1776178092.616966:0:20759:0:(client.c:2215:ptlrpc_check_set()) Completed RPC req@00000000a59bfc89 pname:cluuid:pid:xid:nid:opc:job ll_ost00_010:lustre-OST0001_UUID:20759:1862456998018816:192.168.204.3@tcp:104: 00010000:00010000:1.0:1776178092.616982:0:20759:0:(ldlm_request.c:283:ldlm_completion_ast()) ### client-side enqueue returned a blocked locksleeping ns: filter-lustre-OST0001_UUID lock: 0000000097975a3b/0xa40083ab9bf52e25 lrc: 3/0,1 mode: --/PW res: [0x2c0000400:0x240:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x40010080000000 nid: local remote: 0x0 expref: -99 pid: 20759 timeout: 0 lvb_type: 0 00000100:00100000:1.0:1776178092.617011:0:20761:0:(client.c:2215:ptlrpc_check_set()) Completed RPC req@00000000a30eb964 pname:cluuid:pid:xid:nid:opc:job ll_ost00_012:lustre-OST0000_UUID:20761:1862456998018752:192.168.204.3@tcp:104: 00010000:00010000:0.0:1776178092.617099:0:8645:0:(ldlm_lock.c:216:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0001_UUID lock: 0000000053542eb4/0xa40083ab9bf3d549 lrc: 0/0,0 mode: --/PR res: [0x2c0000400:0x39f:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x44801400010020 nid: 192.168.204.3@tcp remote: 0x771f68aa1eb26187 expref: 10 pid: 8230 timeout: 899 lvb_type: 1 00000100:00100000:0.0:1776178092.617255:0:8645:0:(client.c:2215:ptlrpc_check_set()) Completed RPC req@000000004a9818ac pname:cluuid:pid:xid:nid:opc:job ll_ost00_003:lustre-OST0001_UUID:8645:1862456998018048:192.168.204.3@tcp:104: 00010000:00010000:0.0:1776178092.617272:0:8645:0:(ldlm_request.c:283:ldlm_completion_ast()) ### client-side enqueue returned a blocked locksleeping ns: filter-lustre-OST0001_UUID lock: 00000000a71646e9/0xa40083ab9bf52ded lrc: 3/0,1 mode: PW/PW res: [0x2c0000400:0x39f:0x0].0x0 rrc: 2 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x40010080000000 nid: local remote: 0x0 expref: -99 pid: 8645 timeout: 0 lvb_type: 0 00010000:00010000:0.0:1776178092.617294:0:8645:0:(ldlm_request.c:199:ldlm_completion_tail()) ### client-side enqueue: granted ns: filter-lustre-OST0001_UUID lock: 00000000a71646e9/0xa40083ab9bf52ded lrc: 3/0,1 mode: PW/PW res: [0x2c0000400:0x39f:0x0].0x0 rrc: 2 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x40010080000000 nid: local remote: 0x0 expref: -99 pid: 8645 timeout: 0 lvb_type: 0 00010000:00010000:0.0:1776178092.617302:0:8645:0:(ldlm_request.c:528:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0001_UUID lock: 00000000a71646e9/0xa40083ab9bf52ded lrc: 3/0,1 mode: PW/PW res: [0x2c0000400:0x39f:0x0].0x0 rrc: 2 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x40010080000000 nid: local remote: 0x0 expref: -99 pid: 8645 timeout: 0 lvb_type: 0 00010000:00010000:0.0:1776178092.617309:0:8645:0:(ldlm_lock.c:820:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0001_UUID lock: 00000000a71646e9/0xa40083ab9bf52ded lrc: 3/0,1 mode: PW/PW res: [0x2c0000400:0x39f:0x0].0x0 rrc: 2 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x50010080000000 nid: local remote: 0x0 expref: -99 pid: 8645 timeout: 0 lvb_type: 0 00010000:00010000:0.0:1776178092.617315:0:8645:0:(ldlm_lock.c:879:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0001_UUID lock: 00000000a71646e9/0xa40083ab9bf52ded lrc: 2/0,0 mode: PW/PW res: [0x2c0000400:0x39f:0x0].0x0 rrc: 2 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x50010480000000 nid: local remote: 0x0 expref: -99 pid: 8645 timeout: 0 lvb_type: 0 00010000:00010000:0.0:1776178092.617331:0:8645:0:(ldlm_lockd.c:2159:__ldlm_bl_to_thread()) added 1/2 locks to regular blp list, 2 blwis in pool 00000100:00100000:0.0:1776178092.617482:0:8645:0:(service.c:2372:ptlrpc_server_handle_request()) Handled RPC req@00000000ef2381ea pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_003:lustre-MDT0001-mdtlov_UUID+44:3355:x1862456998017472:12345-0@lo:6:osp-syn-1-1.0 Request processed in 10996us (11223us total) trans 4294969309 rc 0/0 00000100:00100000:0.0:1776178092.617490:0:8645:0:(nrs_fifo.c:241:nrs_fifo_req_stop()) NRS stop fifo request from 12345-0@lo, seq: 5217 00010000:00010000:0.0:1776178092.617533:0:20754:0:(ldlm_lock.c:216:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0001_UUID lock: 0000000098fc7f5c/0xa40083ab9bf4173d lrc: 0/0,0 mode: --/PR res: [0x2c0000400:0x425:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x44801400010020 nid: 192.168.204.3@tcp remote: 0x771f68aa1eb27783 expref: 9 pid: 14440 timeout: 899 lvb_type: 1 00000100:00100000:0.0:1776178092.617554:0:20754:0:(client.c:2215:ptlrpc_check_set()) Completed RPC req@00000000de27a229 pname:cluuid:pid:xid:nid:opc:job ll_ost00_005:lustre-OST0001_UUID:20754:1862456998017920:192.168.204.3@tcp:104: 00010000:00010000:0.0:1776178092.617570:0:20754:0:(ldlm_request.c:283:ldlm_completion_ast()) ### client-side enqueue returned a blocked locksleeping ns: filter-lustre-OST0001_UUID lock: 00000000d3e952db/0xa40083ab9bf52ddf lrc: 3/0,1 mode: PW/PW res: [0x2c0000400:0x425:0x0].0x0 rrc: 2 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x40010080000000 nid: local remote: 0x0 expref: -99 pid: 20754 timeout: 0 lvb_type: 0 00010000:00010000:0.0:1776178092.617577:0:20754:0:(ldlm_request.c:199:ldlm_completion_tail()) ### client-side enqueue: granted ns: filter-lustre-OST0001_UUID lock: 00000000d3e952db/0xa40083ab9bf52ddf lrc: 3/0,1 mode: PW/PW res: [0x2c0000400:0x425:0x0].0x0 rrc: 2 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x40010080000000 nid: local remote: 0x0 expref: -99 pid: 20754 timeout: 0 lvb_type: 0 00010000:00010000:0.0:1776178092.617583:0:20754:0:(ldlm_request.c:528:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0001_UUID lock: 00000000d3e952db/0xa40083ab9bf52ddf lrc: 3/0,1 mode: PW/PW res: [0x2c0000400:0x425:0x0].0x0 rrc: 2 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x40010080000000 nid: local remote: 0x0 expref: -99 pid: 20754 timeout: 0 lvb_type: 0 00010000:00010000:0.0:1776178092.617590:0:20754:0:(ldlm_lock.c:820:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0001_UUID lock: 00000000d3e952db/0xa40083ab9bf52ddf lrc: 3/0,1 mode: PW/PW res: [0x2c0000400:0x425:0x0].0x0 rrc: 2 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x50010080000000 nid: local remote: 0x0 expref: -99 pid: 20754 timeout: 0 lvb_type: 0 00010000:00010000:0.0:1776178092.617596:0:20754:0:(ldlm_lock.c:879:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0001_UUID lock: 00000000d3e952db/0xa40083ab9bf52ddf lrc: 2/0,0 mode: PW/PW res: [0x2c0000400:0x425:0x0].0x0 rrc: 2 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x50010480000000 nid: local remote: 0x0 expref: -99 pid: 20754 timeout: 0 lvb_type: 0 00010000:00010000:0.0:1776178092.617610:0:20754:0:(ldlm_lockd.c:2159:__ldlm_bl_to_thread()) added 1/4 locks to regular blp list, 4 blwis in pool 00000100:00100000:0.0:1776178092.617724:0:20754:0:(service.c:2372:ptlrpc_server_handle_request()) Handled RPC req@000000002b69594e pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_005:lustre-MDT0001-mdtlov_UUID+44:3354:x1862456998017664:12345-0@lo:6:osp-syn-1-1.0 Request processed in 14000us (14151us total) trans 4294969310 rc 0/0 00000100:00100000:0.0:1776178092.617732:0:20754:0:(nrs_fifo.c:241:nrs_fifo_req_stop()) NRS stop fifo request from 12345-0@lo, seq: 5215 00010000:00010000:1.0:1776178092.617743:0:20761:0:(ldlm_request.c:283:ldlm_completion_ast()) ### client-side enqueue returned a blocked locksleeping ns: filter-lustre-OST0000_UUID lock: 00000000499bab84/0xa40083ab9bf52e1e lrc: 3/0,1 mode: --/PW res: [0x280000400:0x411:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x40010080000000 nid: local remote: 0x0 expref: -99 pid: 20761 timeout: 0 lvb_type: 0 00010000:00010000:0.0:1776178092.617773:0:20755:0:(ldlm_lock.c:216:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0001_UUID lock: 0000000071df586c/0xa40083ab9bf3460f lrc: 0/0,0 mode: --/PR res: [0x2c0000400:0x27c:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x44801400010020 nid: 192.168.204.3@tcp remote: 0x771f68aa1eb231c9 expref: 8 pid: 8645 timeout: 899 lvb_type: 1 00010000:00010000:1.0:1776178092.617774:0:6245:0:(ldlm_lockd.c:2789:ldlm_bl_get_work()) Got 1 locks of 3 total in blp. (3 blwis in pool) 00010000:00010000:1.0:1776178092.617781:0:6245:0:(ldlm_lockd.c:1879:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 00000000801fce1a/0xa40083ab9bf52df4 lrc: 2/0,0 mode: PW/PW res: [0x280000400:0x386:0x0].0x0 rrc: 2 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x40010480000000 nid: local remote: 0x0 expref: -99 pid: 20758 timeout: 0 lvb_type: 0 00000100:00100000:0.0:1776178092.617787:0:20755:0:(client.c:2215:ptlrpc_check_set()) Completed RPC req@00000000fb56871d pname:cluuid:pid:xid:nid:opc:job ll_ost00_006:lustre-OST0001_UUID:20755:1862456998018368:192.168.204.3@tcp:104: 00010000:00010000:1.0:1776178092.617793:0:6245:0:(ldlm_lockd.c:1893:ldlm_handle_bl_callback()) Lock 00000000801fce1a already unused, calling callback (0000000063c97258) 00010000:00010000:1.0:1776178092.617798:0:6245:0:(ldlm_request.c:364:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 00000000801fce1a/0xa40083ab9bf52df4 lrc: 2/0,0 mode: PW/PW res: [0x280000400:0x386:0x0].0x0 rrc: 2 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x40010480000000 nid: local remote: 0x0 expref: -99 pid: 20758 timeout: 0 lvb_type: 0 00010000:00010000:1.0:1776178092.617807:0:6245:0:(ldlm_request.c:1257:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 00000000801fce1a/0xa40083ab9bf52df4 lrc: 3/0,0 mode: PW/PW res: [0x280000400:0x386:0x0].0x0 rrc: 2 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x40018480000000 nid: local remote: 0x0 expref: -99 pid: 20758 timeout: 0 lvb_type: 0 00010000:00010000:0.0:1776178092.617815:0:20755:0:(ldlm_request.c:283:ldlm_completion_ast()) ### client-side enqueue returned a blocked locksleeping ns: filter-lustre-OST0001_UUID lock: 000000007e5ef416/0xa40083ab9bf52dfb lrc: 3/0,1 mode: PW/PW res: [0x2c0000400:0x27c:0x0].0x0 rrc: 2 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x40010080000000 nid: local remote: 0x0 expref: -99 pid: 20755 timeout: 0 lvb_type: 0 00010000:00010000:1.0:1776178092.617833:0:6245:0:(ldlm_lockd.c:1903:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 00000000801fce1a/0xa40083ab9bf52df4 lrc: 1/0,0 mode: --/PW res: [0x280000400:0x386:0x0].0x0 rrc: 2 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x44819480000000 nid: local remote: 0x0 expref: -99 pid: 20758 timeout: 0 lvb_type: 0 00010000:00010000:0.0:1776178092.617835:0:20755:0:(ldlm_request.c:199:ldlm_completion_tail()) ### client-side enqueue: granted ns: filter-lustre-OST0001_UUID lock: 000000007e5ef416/0xa40083ab9bf52dfb lrc: 3/0,1 mode: PW/PW res: [0x2c0000400:0x27c:0x0].0x0 rrc: 2 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x40010080000000 nid: local remote: 0x0 expref: -99 pid: 20755 timeout: 0 lvb_type: 0 00010000:00010000:1.0:1776178092.617852:0:6245:0:(ldlm_lock.c:216:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 00000000801fce1a/0xa40083ab9bf52df4 lrc: 0/0,0 mode: --/PW res: [0x280000400:0x386:0x0].0x0 rrc: 2 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x44819480000000 nid: local remote: 0x0 expref: -99 pid: 20758 timeout: 0 lvb_type: 0 00010000:00010000:0.0:1776178092.617853:0:20755:0:(ldlm_request.c:528:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0001_UUID lock: 000000007e5ef416/0xa40083ab9bf52dfb lrc: 3/0,1 mode: PW/PW res: [0x2c0000400:0x27c:0x0].0x0 rrc: 2 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x40010080000000 nid: local remote: 0x0 expref: -99 pid: 20755 timeout: 0 lvb_type: 0 00010000:00010000:0.0:1776178092.617874:0:20755:0:(ldlm_lock.c:820:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0001_UUID lock: 000000007e5ef416/0xa40083ab9bf52dfb lrc: 3/0,1 mode: PW/PW res: [0x2c0000400:0x27c:0x0].0x0 rrc: 2 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x50010080000000 nid: local remote: 0x0 expref: -99 pid: 20755 timeout: 0 lvb_type: 0 00010000:00010000:0.0:1776178092.617880:0:20755:0:(ldlm_lock.c:879:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0001_UUID lock: 000000007e5ef416/0xa40083ab9bf52dfb lrc: 2/0,0 mode: PW/PW res: [0x2c0000400:0x27c:0x0].0x0 rrc: 2 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x50010480000000 nid: local remote: 0x0 expref: -99 pid: 20755 timeout: 0 lvb_type: 0 00010000:00010000:1.0:1776178092.617883:0:6245:0:(ldlm_lockd.c:2789:ldlm_bl_get_work()) Got 1 locks of 2 total in blp. (2 blwis in pool) 00010000:00010000:1.0:1776178092.617888:0:6245:0:(ldlm_lockd.c:1879:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0001_UUID lock: 00000000a71646e9/0xa40083ab9bf52ded lrc: 2/0,0 mode: PW/PW res: [0x2c0000400:0x39f:0x0].0x0 rrc: 2 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x40010480000000 nid: local remote: 0x0 expref: -99 pid: 8645 timeout: 0 lvb_type: 0 00010000:00010000:0.0:1776178092.617892:0:20755:0:(ldlm_lockd.c:2159:__ldlm_bl_to_thread()) added 1/3 locks to regular blp list, 3 blwis in pool 00010000:00010000:1.0:1776178092.617898:0:6245:0:(ldlm_lockd.c:1893:ldlm_handle_bl_callback()) Lock 00000000a71646e9 already unused, calling callback (0000000063c97258) 00010000:00010000:1.0:1776178092.617900:0:6245:0:(ldlm_request.c:364:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0001_UUID lock: 00000000a71646e9/0xa40083ab9bf52ded lrc: 2/0,0 mode: PW/PW res: [0x2c0000400:0x39f:0x0].0x0 rrc: 2 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x40010480000000 nid: local remote: 0x0 expref: -99 pid: 8645 timeout: 0 lvb_type: 0 00010000:00010000:1.0:1776178092.617906:0:6245:0:(ldlm_request.c:1257:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0001_UUID lock: 00000000a71646e9/0xa40083ab9bf52ded lrc: 3/0,0 mode: PW/PW res: [0x2c0000400:0x39f:0x0].0x0 rrc: 2 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x40018480000000 nid: local remote: 0x0 expref: -99 pid: 8645 timeout: 0 lvb_type: 0 00010000:00010000:1.0:1776178092.617916:0:6245:0:(ldlm_lockd.c:1903:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0001_UUID lock: 00000000a71646e9/0xa40083ab9bf52ded lrc: 1/0,0 mode: --/PW res: [0x2c0000400:0x39f:0x0].0x0 rrc: 2 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x44819480000000 nid: local remote: 0x0 expref: -99 pid: 8645 timeout: 0 lvb_type: 0 00010000:00010000:1.0:1776178092.617923:0:6245:0:(ldlm_lock.c:216:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0001_UUID lock: 00000000a71646e9/0xa40083ab9bf52ded lrc: 0/0,0 mode: --/PW res: [0x2c0000400:0x39f:0x0].0x0 rrc: 2 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x44819480000000 nid: local remote: 0x0 expref: -99 pid: 8645 timeout: 0 lvb_type: 0 00010000:00010000:1.0:1776178092.617948:0:6245:0:(ldlm_lockd.c:2789:ldlm_bl_get_work()) Got 1 locks of 2 total in blp. (2 blwis in pool) 00010000:00010000:1.0:1776178092.617951:0:6245:0:(ldlm_lockd.c:1879:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 00000000b27cf00e/0xa40083ab9bf52e02 lrc: 2/0,0 mode: PW/PW res: [0x280000400:0x3ab:0x0].0x0 rrc: 2 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x40010480000000 nid: local remote: 0x0 expref: -99 pid: 14440 timeout: 0 lvb_type: 0 00010000:00010000:1.0:1776178092.617957:0:6245:0:(ldlm_lockd.c:1893:ldlm_handle_bl_callback()) Lock 00000000b27cf00e already unused, calling callback (0000000063c97258) 00010000:00010000:1.0:1776178092.617960:0:6245:0:(ldlm_request.c:364:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 00000000b27cf00e/0xa40083ab9bf52e02 lrc: 2/0,0 mode: PW/PW res: [0x280000400:0x3ab:0x0].0x0 rrc: 2 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x40010480000000 nid: local remote: 0x0 expref: -99 pid: 14440 timeout: 0 lvb_type: 0 00010000:00010000:1.0:1776178092.617966:0:6245:0:(ldlm_request.c:1257:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 00000000b27cf00e/0xa40083ab9bf52e02 lrc: 3/0,0 mode: PW/PW res: [0x280000400:0x3ab:0x0].0x0 rrc: 2 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x40018480000000 nid: local remote: 0x0 expref: -99 pid: 14440 timeout: 0 lvb_type: 0 00010000:00010000:1.0:1776178092.617976:0:6245:0:(ldlm_lockd.c:1903:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 00000000b27cf00e/0xa40083ab9bf52e02 lrc: 1/0,0 mode: --/PW res: [0x280000400:0x3ab:0x0].0x0 rrc: 2 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x44819480000000 nid: local remote: 0x0 expref: -99 pid: 14440 timeout: 0 lvb_type: 0 00010000:00010000:1.0:1776178092.617982:0:6245:0:(ldlm_lock.c:216:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 00000000b27cf00e/0xa40083ab9bf52e02 lrc: 0/0,0 mode: --/PW res: [0x280000400:0x3ab:0x0].0x0 rrc: 2 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x44819480000000 nid: local remote: 0x0 expref: -99 pid: 14440 timeout: 0 lvb_type: 0 00010000:00010000:1.0:1776178092.617996:0:6245:0:(ldlm_lockd.c:2789:ldlm_bl_get_work()) Got 1 locks of 1 total in blp. (1 blwis in pool) 00010000:00010000:1.0:1776178092.617998:0:6245:0:(ldlm_lockd.c:1879:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0001_UUID lock: 00000000d3e952db/0xa40083ab9bf52ddf lrc: 2/0,0 mode: PW/PW res: [0x2c0000400:0x425:0x0].0x0 rrc: 2 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x40010480000000 nid: local remote: 0x0 expref: -99 pid: 20754 timeout: 0 lvb_type: 0 00010000:00010000:1.0:1776178092.618009:0:6245:0:(ldlm_lockd.c:1893:ldlm_handle_bl_callback()) Lock 00000000d3e952db already unused, calling callback (0000000063c97258) 00010000:00010000:1.0:1776178092.618012:0:6245:0:(ldlm_request.c:364:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0001_UUID lock: 00000000d3e952db/0xa40083ab9bf52ddf lrc: 2/0,0 mode: PW/PW res: [0x2c0000400:0x425:0x0].0x0 rrc: 2 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x40010480000000 nid: local remote: 0x0 expref: -99 pid: 20754 timeout: 0 lvb_type: 0 00000100:00100000:0.0:1776178092.618019:0:20755:0:(service.c:2372:ptlrpc_server_handle_request()) Handled RPC req@0000000015f7661d pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_006:lustre-MDT0001-mdtlov_UUID+44:3354:x1862456998018176:12345-0@lo:6:osp-syn-1-1.0 Request processed in 10915us (11177us total) trans 4294969311 rc 0/0 00000100:00100000:0.0:1776178092.618026:0:20755:0:(nrs_fifo.c:241:nrs_fifo_req_stop()) NRS stop fifo request from 12345-0@lo, seq: 5219 00010000:00010000:0.0:1776178092.618076:0:8228:0:(ldlm_lock.c:216:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0001_UUID lock: 00000000e391698d/0xa40083ab9bf3a609 lrc: 0/0,0 mode: --/PR res: [0x2c0000400:0x33f:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x44801400010020 nid: 192.168.204.3@tcp remote: 0x771f68aa1eb251c7 expref: 7 pid: 8645 timeout: 899 lvb_type: 1 00000100:00100000:0.0:1776178092.618088:0:8228:0:(client.c:2215:ptlrpc_check_set()) Completed RPC req@000000001bbe82bd pname:cluuid:pid:xid:nid:opc:job ll_ost00_001:lustre-OST0001_UUID:8228:1862456998017856:192.168.204.3@tcp:104: 00010000:00010000:0.0:1776178092.618102:0:8228:0:(ldlm_request.c:283:ldlm_completion_ast()) ### client-side enqueue returned a blocked locksleeping ns: filter-lustre-OST0001_UUID lock: 00000000f51fd0d7/0xa40083ab9bf52de6 lrc: 3/0,1 mode: PW/PW res: [0x2c0000400:0x33f:0x0].0x0 rrc: 2 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x40010080000000 nid: local remote: 0x0 expref: -99 pid: 8228 timeout: 0 lvb_type: 0 00010000:00010000:0.0:1776178092.618592:0:8228:0:(ldlm_request.c:199:ldlm_completion_tail()) ### client-side enqueue: granted ns: filter-lustre-OST0001_UUID lock: 00000000f51fd0d7/0xa40083ab9bf52de6 lrc: 3/0,1 mode: PW/PW res: [0x2c0000400:0x33f:0x0].0x0 rrc: 2 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x40010080000000 nid: local remote: 0x0 expref: -99 pid: 8228 timeout: 0 lvb_type: 0 00010000:00010000:0.0:1776178092.618607:0:8228:0:(ldlm_request.c:528:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0001_UUID lock: 00000000f51fd0d7/0xa40083ab9bf52de6 lrc: 3/0,1 mode: PW/PW res: [0x2c0000400:0x33f:0x0].0x0 rrc: 2 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x40010080000000 nid: local remote: 0x0 expref: -99 pid: 8228 timeout: 0 lvb_type: 0 00010000:00010000:0.0:1776178092.618614:0:8228:0:(ldlm_lock.c:820:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0001_UUID lock: 00000000f51fd0d7/0xa40083ab9bf52de6 lrc: 3/0,1 mode: PW/PW res: [0x2c0000400:0x33f:0x0].0x0 rrc: 2 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x50010080000000 nid: local remote: 0x0 expref: -99 pid: 8228 timeout: 0 lvb_type: 0 00010000:00010000:0.0:1776178092.618621:0:8228:0:(ldlm_lock.c:879:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0001_UUID lock: 00000000f51fd0d7/0xa40083ab9bf52de6 lrc: 2/0,0 mode: PW/PW res: [0x2c0000400:0x33f:0x0].0x0 rrc: 2 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x50010480000000 nid: local remote: 0x0 expref: -99 pid: 8228 timeout: 0 lvb_type: 0 00010000:00010000:0.0:1776178092.618637:0:8228:0:(ldlm_lockd.c:2159:__ldlm_bl_to_thread()) added 1/1 locks to regular blp list, 1 blwis in pool 00000100:00100000:0.0:1776178092.618855:0:8228:0:(service.c:2372:ptlrpc_server_handle_request()) Handled RPC req@000000007886b426 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_001:lustre-MDT0001-mdtlov_UUID+44:3354:x1862456998017600:12345-0@lo:6:osp-syn-1-1.0 Request processed in 14291us (15226us total) trans 4294969312 rc 0/0 00000100:00100000:0.0:1776178092.618865:0:8228:0:(nrs_fifo.c:241:nrs_fifo_req_stop()) NRS stop fifo request from 12345-0@lo, seq: 5216 00010000:00010000:1.0:1776178092.618900:0:6245:0:(ldlm_request.c:1257:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0001_UUID lock: 00000000d3e952db/0xa40083ab9bf52ddf lrc: 3/0,0 mode: PW/PW res: [0x2c0000400:0x425:0x0].0x0 rrc: 2 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x40018480000000 nid: local remote: 0x0 expref: -99 pid: 20754 timeout: 0 lvb_type: 0 00010000:00010000:1.0:1776178092.618917:0:6245:0:(ldlm_lockd.c:1903:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0001_UUID lock: 00000000d3e952db/0xa40083ab9bf52ddf lrc: 1/0,0 mode: --/PW res: [0x2c0000400:0x425:0x0].0x0 rrc: 2 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x44819480000000 nid: local remote: 0x0 expref: -99 pid: 20754 timeout: 0 lvb_type: 0 00010000:00010000:1.0:1776178092.618923:0:6245:0:(ldlm_lock.c:216:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0001_UUID lock: 00000000d3e952db/0xa40083ab9bf52ddf lrc: 0/0,0 mode: --/PW res: [0x2c0000400:0x425:0x0].0x0 rrc: 2 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x44819480000000 nid: local remote: 0x0 expref: -99 pid: 20754 timeout: 0 lvb_type: 0 00010000:00010000:1.0:1776178092.618936:0:6245:0:(ldlm_lockd.c:2789:ldlm_bl_get_work()) Got 1 locks of 0 total in blp. (0 blwis in pool) 00010000:00010000:1.0:1776178092.618939:0:6245:0:(ldlm_lockd.c:1879:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0001_UUID lock: 00000000f51fd0d7/0xa40083ab9bf52de6 lrc: 2/0,0 mode: PW/PW res: [0x2c0000400:0x33f:0x0].0x0 rrc: 2 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x40010480000000 nid: local remote: 0x0 expref: -99 pid: 8228 timeout: 0 lvb_type: 0 00010000:00010000:1.0:1776178092.618945:0:6245:0:(ldlm_lockd.c:1893:ldlm_handle_bl_callback()) Lock 00000000f51fd0d7 already unused, calling callback (0000000063c97258) 00010000:00010000:1.0:1776178092.618947:0:6245:0:(ldlm_request.c:364:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0001_UUID lock: 00000000f51fd0d7/0xa40083ab9bf52de6 lrc: 2/0,0 mode: PW/PW res: [0x2c0000400:0x33f:0x0].0x0 rrc: 2 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x40010480000000 nid: local remote: 0x0 expref: -99 pid: 8228 timeout: 0 lvb_type: 0 00010000:00010000:1.0:1776178092.618952:0:6245:0:(ldlm_request.c:1257:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0001_UUID lock: 00000000f51fd0d7/0xa40083ab9bf52de6 lrc: 3/0,0 mode: PW/PW res: [0x2c0000400:0x33f:0x0].0x0 rrc: 2 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x40018480000000 nid: local remote: 0x0 expref: -99 pid: 8228 timeout: 0 lvb_type: 0 00010000:00010000:1.0:1776178092.618960:0:6245:0:(ldlm_lockd.c:1903:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0001_UUID lock: 00000000f51fd0d7/0xa40083ab9bf52de6 lrc: 1/0,0 mode: --/PW res: [0x2c0000400:0x33f:0x0].0x0 rrc: 2 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x44819480000000 nid: local remote: 0x0 expref: -99 pid: 8228 timeout: 0 lvb_type: 0 00010000:00010000:1.0:1776178092.618969:0:6245:0:(ldlm_lock.c:216:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0001_UUID lock: 00000000f51fd0d7/0xa40083ab9bf52de6 lrc: 0/0,0 mode: --/PW res: [0x2c0000400:0x33f:0x0].0x0 rrc: 2 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x44819480000000 nid: local remote: 0x0 expref: -99 pid: 8228 timeout: 0 lvb_type: 0 00010000:00010000:1.0:1776178092.618983:0:6245:0:(ldlm_lockd.c:2792:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00010000:00010000:1.0:1776178092.618984:0:6245:0:(ldlm_lockd.c:2792:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00010000:00010000:1.0:1776178092.618985:0:6245:0:(ldlm_lockd.c:2792:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00010000:00010000:1.0:1776178092.619001:0:20764:0:(ldlm_lockd.c:2159:__ldlm_bl_to_thread()) added 1/1 locks to regular blp list, 1 blwis in pool 00000100:00100000:1.0:1776178092.619450:0:20767:0:(nrs_fifo.c:179:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.204.3@tcp, seq: 2065 00000100:00100000:1.0:1776178092.619460:0:20767:0:(service.c:2324:ptlrpc_server_handle_request()) Handling RPC req@00000000aab3f15f pname:cluuid+ref:pid:xid:nid:opc:job ldlm_cn00_004:b50963bd-8eba-4432-9a54-8888d00e3fa1+11:2270:x1862456979074624:12345-192.168.204.3@tcp:103:ldlm_bl_03.0 00010000:00010000:1.0:1776178092.619475:0:20767:0:(ldlm_lockd.c:1720:ldlm_request_cancel()) ### server-side cancel handler START: 1 locks, starting at 0 00010000:00010000:1.0:1776178092.619480:0:20767:0:(ldlm_lockd.c:1771:ldlm_request_cancel()) ### server cancels blocked lock after 0s ns: filter-lustre-OST0000_UUID lock: 000000003b55bbe6/0xa40083ab9bf33220 lrc: 4/0,0 mode: PR/PR res: [0x280000400:0x293:0x0].0x0 rrc: 4 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x60000400010020 nid: 192.168.204.3@tcp remote: 0x771f68aa1eb22b24 expref: 11 pid: 8645 timeout: 899 lvb_type: 1 00010000:00010000:1.0:1776178092.619490:0:20767:0:(ldlm_lockd.c:562:ldlm_del_waiting_lock()) ### removed ns: filter-lustre-OST0000_UUID lock: 000000003b55bbe6/0xa40083ab9bf33220 lrc: 3/0,0 mode: PR/PR res: [0x280000400:0x293:0x0].0x0 rrc: 4 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x50000400010020 nid: 192.168.204.3@tcp remote: 0x771f68aa1eb22b24 expref: 11 pid: 8645 timeout: 899 lvb_type: 1 00010000:00010000:1.0:1776178092.619507:0:20767:0:(ldlm_lock.c:216:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 000000003b55bbe6/0xa40083ab9bf33220 lrc: 0/0,0 mode: --/PR res: [0x280000400:0x293:0x0].0x0 rrc: 4 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x44801400010020 nid: 192.168.204.3@tcp remote: 0x771f68aa1eb22b24 expref: 11 pid: 8645 timeout: 899 lvb_type: 1 00010000:00010000:1.0:1776178092.619518:0:20767:0:(ldlm_extent.c:318:ldlm_check_contention()) contended locks = 0 00010000:00010000:1.0:1776178092.619519:0:20767:0:(ldlm_extent.c:318:ldlm_check_contention()) contended locks = 0 00010000:00010000:1.0:1776178092.619523:0:20767:0:(ldlm_lock.c:704:ldlm_add_cp_work_item()) ### lock granted; sending completion AST. ns: filter-lustre-OST0000_UUID lock: 000000007ae7b5f6/0xa40083ab9bf52e09 lrc: 3/0,1 mode: PW/PW res: [0x280000400:0x293:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x40090080000000 nid: local remote: 0x0 expref: -99 pid: 20762 timeout: 0 lvb_type: 0 00010000:00010000:1.0:1776178092.619530:0:20767:0:(ldlm_resource.c:1650:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 000000007ae7b5f6/0xa40083ab9bf52e09 lrc: 4/0,1 mode: PW/PW res: [0x280000400:0x293:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x40090080000000 nid: local remote: 0x0 expref: -99 pid: 20762 timeout: 0 lvb_type: 0 00010000:00010000:1.0:1776178092.619557:0:20767:0:(ldlm_lockd.c:1782:ldlm_request_cancel()) ### server-side cancel handler END 00000100:00100000:1.0:1776178092.619601:0:20767:0:(service.c:2372:ptlrpc_server_handle_request()) Handled RPC req@00000000aab3f15f pname:cluuid+ref:pid:xid:nid:opc:job ldlm_cn00_004:b50963bd-8eba-4432-9a54-8888d00e3fa1+10:2270:x1862456979074624:12345-192.168.204.3@tcp:103:ldlm_bl_03.0 Request processed in 143us (4544us total) trans 0 rc 0/0 00000100:00100000:1.0:1776178092.619608:0:20767:0:(nrs_fifo.c:241:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.204.3@tcp, seq: 2065 00000100:00100000:1.0:1776178092.623312:0:3346:0:(events.c:361:request_in_callback()) peer: 12345-192.168.204.3@tcp (source: 12345-192.168.204.3@tcp) 00000100:00100000:0.0:1776178092.623404:0:20765:0:(service.c:2372:ptlrpc_server_handle_request()) Handled RPC req@00000000ddc1dc02 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_016:lustre-MDT0001-mdtlov_UUID+51:3355:x1862456998016384:12345-0@lo:6:osp-syn-0-1.0 Request processed in 64106us (64180us total) trans 4294969331 rc 0/0 00000100:00100000:1.0:1776178092.623407:0:3346:0:(events.c:361:request_in_callback()) peer: 12345-192.168.204.3@tcp (source: 12345-192.168.204.3@tcp) 00000100:00100000:0.0:1776178092.623420:0:20765:0:(nrs_fifo.c:241:nrs_fifo_req_stop()) NRS stop fifo request from 12345-0@lo, seq: 5207 00000100:00100000:1.0:1776178092.623447:0:3346:0:(events.c:361:request_in_callback()) peer: 12345-192.168.204.3@tcp (source: 12345-192.168.204.3@tcp) 00000100:00100000:1.0:1776178092.623536:0:3346:0:(events.c:361:request_in_callback()) peer: 12345-192.168.204.3@tcp (source: 12345-192.168.204.3@tcp) 00000100:00100000:0.0:1776178092.624227:0:6241:0:(service.c:2152:ptlrpc_server_handle_req_in()) got req x1862456979074816 00010000:00010000:0.0:1776178092.624248:0:6241:0:(ldlm_lockd.c:2638:ldlm_cancel_hpreq_check()) ### hpreq cancel/convert lock ns: filter-lustre-OST0000_UUID lock: 00000000f247d55d/0xa40083ab9bf3ee24 lrc: 4/0,0 mode: PR/PR res: [0x280000400:0x411:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x60000400010020 nid: 192.168.204.3@tcp remote: 0x771f68aa1eb269d0 expref: 10 pid: 8230 timeout: 899 lvb_type: 1 00000100:00100000:0.0:1776178092.624270:0:6241:0:(nrs_fifo.c:179:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.204.3@tcp, seq: 2067 00000100:00100000:0.0:1776178092.624275:0:6241:0:(service.c:2324:ptlrpc_server_handle_request()) Handling RPC req@00000000a59bfc89 pname:cluuid+ref:pid:xid:nid:opc:job ldlm_cn00_000:b50963bd-8eba-4432-9a54-8888d00e3fa1+8:2271:x1862456979074880:12345-192.168.204.3@tcp:103:ldlm_bl_01.0 00010000:00010000:0.0:1776178092.624285:0:6241:0:(ldlm_lockd.c:1720:ldlm_request_cancel()) ### server-side cancel handler START: 1 locks, starting at 0 00010000:00010000:0.0:1776178092.624289:0:6241:0:(ldlm_lockd.c:1771:ldlm_request_cancel()) ### server cancels blocked lock after 0s ns: filter-lustre-OST0001_UUID lock: 00000000501110f8/0xa40083ab9bf32887 lrc: 4/0,0 mode: PR/PR res: [0x2c0000400:0x240:0x0].0x0 rrc: 4 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x60000400010020 nid: 192.168.204.3@tcp remote: 0x771f68aa1eb227f1 expref: 8 pid: 8228 timeout: 899 lvb_type: 1 00010000:00010000:0.0:1776178092.624303:0:6241:0:(ldlm_lockd.c:562:ldlm_del_waiting_lock()) ### removed ns: filter-lustre-OST0001_UUID lock: 00000000501110f8/0xa40083ab9bf32887 lrc: 3/0,0 mode: PR/PR res: [0x2c0000400:0x240:0x0].0x0 rrc: 4 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x50000400010020 nid: 192.168.204.3@tcp remote: 0x771f68aa1eb227f1 expref: 8 pid: 8228 timeout: 899 lvb_type: 1 00010000:00010000:0.0:1776178092.624319:0:6241:0:(ldlm_lock.c:216:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0001_UUID lock: 00000000501110f8/0xa40083ab9bf32887 lrc: 0/0,0 mode: --/PR res: [0x2c0000400:0x240:0x0].0x0 rrc: 4 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x44801400010020 nid: 192.168.204.3@tcp remote: 0x771f68aa1eb227f1 expref: 8 pid: 8228 timeout: 899 lvb_type: 1 00010000:00010000:0.0:1776178092.624330:0:6241:0:(ldlm_extent.c:318:ldlm_check_contention()) contended locks = 0 00010000:00010000:0.0:1776178092.624332:0:6241:0:(ldlm_extent.c:318:ldlm_check_contention()) contended locks = 0 00010000:00010000:0.0:1776178092.624334:0:6241:0:(ldlm_lock.c:704:ldlm_add_cp_work_item()) ### lock granted; sending completion AST. ns: filter-lustre-OST0001_UUID lock: 0000000097975a3b/0xa40083ab9bf52e25 lrc: 3/0,1 mode: PW/PW res: [0x2c0000400:0x240:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x40090080000000 nid: local remote: 0x0 expref: -99 pid: 20759 timeout: 0 lvb_type: 0 00010000:00010000:0.0:1776178092.624341:0:6241:0:(ldlm_resource.c:1650:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0001_UUID lock: 0000000097975a3b/0xa40083ab9bf52e25 lrc: 4/0,1 mode: PW/PW res: [0x2c0000400:0x240:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x40090080000000 nid: local remote: 0x0 expref: -99 pid: 20759 timeout: 0 lvb_type: 0 00010000:00010000:0.0:1776178092.624366:0:6241:0:(ldlm_lockd.c:1782:ldlm_request_cancel()) ### server-side cancel handler END 00000100:00100000:0.0:1776178092.626106:0:6241:0:(service.c:2372:ptlrpc_server_handle_request()) Handled RPC req@00000000a59bfc89 pname:cluuid+ref:pid:xid:nid:opc:job ldlm_cn00_000:b50963bd-8eba-4432-9a54-8888d00e3fa1+6:2271:x1862456979074880:12345-192.168.204.3@tcp:103:ldlm_bl_01.0 Request processed in 1828us (2700us total) trans 0 rc 0/0 00000100:00100000:0.0:1776178092.626261:0:6241:0:(nrs_fifo.c:241:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.204.3@tcp, seq: 2067 00000100:00100000:0.0:1776178092.626287:0:6241:0:(nrs_fifo.c:179:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.204.3@tcp, seq: 2069 00000100:00100000:0.0:1776178092.626293:0:6241:0:(service.c:2324:ptlrpc_server_handle_request()) Handling RPC req@00000000dd1281fe pname:cluuid+ref:pid:xid:nid:opc:job ldlm_cn00_000:b50963bd-8eba-4432-9a54-8888d00e3fa1+9:2271:x1862456979074816:12345-192.168.204.3@tcp:103:ldlm_bl_01.0 00010000:00010000:0.0:1776178092.626303:0:6241:0:(ldlm_lockd.c:1720:ldlm_request_cancel()) ### server-side cancel handler START: 1 locks, starting at 0 00010000:00010000:0.0:1776178092.626310:0:6241:0:(ldlm_lockd.c:1771:ldlm_request_cancel()) ### server cancels blocked lock after 0s ns: filter-lustre-OST0000_UUID lock: 00000000f247d55d/0xa40083ab9bf3ee24 lrc: 4/0,0 mode: PR/PR res: [0x280000400:0x411:0x0].0x0 rrc: 4 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x60000400010020 nid: 192.168.204.3@tcp remote: 0x771f68aa1eb269d0 expref: 9 pid: 8230 timeout: 899 lvb_type: 1 00010000:00010000:0.0:1776178092.626323:0:6241:0:(ldlm_lockd.c:562:ldlm_del_waiting_lock()) ### removed ns: filter-lustre-OST0000_UUID lock: 00000000f247d55d/0xa40083ab9bf3ee24 lrc: 3/0,0 mode: PR/PR res: [0x280000400:0x411:0x0].0x0 rrc: 4 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x50000400010020 nid: 192.168.204.3@tcp remote: 0x771f68aa1eb269d0 expref: 9 pid: 8230 timeout: 899 lvb_type: 1 00010000:00010000:0.0:1776178092.626342:0:6241:0:(ldlm_lock.c:216:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 00000000f247d55d/0xa40083ab9bf3ee24 lrc: 0/0,0 mode: --/PR res: [0x280000400:0x411:0x0].0x0 rrc: 4 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x44801400010020 nid: 192.168.204.3@tcp remote: 0x771f68aa1eb269d0 expref: 9 pid: 8230 timeout: 899 lvb_type: 1 00010000:00010000:0.0:1776178092.626354:0:6241:0:(ldlm_extent.c:318:ldlm_check_contention()) contended locks = 0 00010000:00010000:0.0:1776178092.626356:0:6241:0:(ldlm_extent.c:318:ldlm_check_contention()) contended locks = 0 00010000:00010000:0.0:1776178092.626359:0:6241:0:(ldlm_lock.c:704:ldlm_add_cp_work_item()) ### lock granted; sending completion AST. ns: filter-lustre-OST0000_UUID lock: 00000000499bab84/0xa40083ab9bf52e1e lrc: 3/0,1 mode: PW/PW res: [0x280000400:0x411:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x40090080000000 nid: local remote: 0x0 expref: -99 pid: 20761 timeout: 0 lvb_type: 0 00010000:00010000:0.0:1776178092.626372:0:6241:0:(ldlm_resource.c:1650:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 00000000499bab84/0xa40083ab9bf52e1e lrc: 4/0,1 mode: PW/PW res: [0x280000400:0x411:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x40090080000000 nid: local remote: 0x0 expref: -99 pid: 20761 timeout: 0 lvb_type: 0 00010000:00010000:0.0:1776178092.626399:0:6241:0:(ldlm_lockd.c:1782:ldlm_request_cancel()) ### server-side cancel handler END 00000100:00100000:0.0:1776178092.626425:0:6241:0:(service.c:2372:ptlrpc_server_handle_request()) Handled RPC req@00000000dd1281fe pname:cluuid+ref:pid:xid:nid:opc:job ldlm_cn00_000:b50963bd-8eba-4432-9a54-8888d00e3fa1+8:2271:x1862456979074816:12345-192.168.204.3@tcp:103:ldlm_bl_01.0 Request processed in 132us (2888us total) trans 0 rc 0/0 00000100:00100000:0.0:1776178092.626433:0:6241:0:(nrs_fifo.c:241:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.204.3@tcp, seq: 2069 00010000:00010000:0.0:1776178092.626462:0:20759:0:(ldlm_request.c:199:ldlm_completion_tail()) ### client-side enqueue: granted ns: filter-lustre-OST0001_UUID lock: 0000000097975a3b/0xa40083ab9bf52e25 lrc: 3/0,1 mode: PW/PW res: [0x2c0000400:0x240:0x0].0x0 rrc: 2 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x40010080000000 nid: local remote: 0x0 expref: -99 pid: 20759 timeout: 0 lvb_type: 0 00010000:00010000:0.0:1776178092.626475:0:20759:0:(ldlm_request.c:528:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0001_UUID lock: 0000000097975a3b/0xa40083ab9bf52e25 lrc: 3/0,1 mode: PW/PW res: [0x2c0000400:0x240:0x0].0x0 rrc: 2 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x40010080000000 nid: local remote: 0x0 expref: -99 pid: 20759 timeout: 0 lvb_type: 0 00010000:00010000:0.0:1776178092.626483:0:20759:0:(ldlm_lock.c:820:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0001_UUID lock: 0000000097975a3b/0xa40083ab9bf52e25 lrc: 3/0,1 mode: PW/PW res: [0x2c0000400:0x240:0x0].0x0 rrc: 2 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x50010080000000 nid: local remote: 0x0 expref: -99 pid: 20759 timeout: 0 lvb_type: 0 00010000:00010000:1.0:1776178092.626487:0:6245:0:(ldlm_lockd.c:2789:ldlm_bl_get_work()) Got 1 locks of 0 total in blp. (0 blwis in pool) 00010000:00010000:1.0:1776178092.626494:0:6245:0:(ldlm_lockd.c:1879:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 00000000bbda5115/0xa40083ab9bf52e17 lrc: 2/0,0 mode: PW/PW res: [0x280000400:0x40f:0x0].0x0 rrc: 2 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x40010480000000 nid: local remote: 0x0 expref: -99 pid: 8227 timeout: 0 lvb_type: 0 00010000:00010000:0.0:1776178092.626495:0:20759:0:(ldlm_lock.c:879:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0001_UUID lock: 0000000097975a3b/0xa40083ab9bf52e25 lrc: 2/0,0 mode: PW/PW res: [0x2c0000400:0x240:0x0].0x0 rrc: 2 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x50010480000000 nid: local remote: 0x0 expref: -99 pid: 20759 timeout: 0 lvb_type: 0 00010000:00010000:1.0:1776178092.626515:0:6245:0:(ldlm_lockd.c:1893:ldlm_handle_bl_callback()) Lock 00000000bbda5115 already unused, calling callback (0000000063c97258) 00010000:00010000:1.0:1776178092.626519:0:6245:0:(ldlm_request.c:364:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 00000000bbda5115/0xa40083ab9bf52e17 lrc: 2/0,0 mode: PW/PW res: [0x280000400:0x40f:0x0].0x0 rrc: 2 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x40010480000000 nid: local remote: 0x0 expref: -99 pid: 8227 timeout: 0 lvb_type: 0 00010000:00010000:1.0:1776178092.626529:0:6245:0:(ldlm_request.c:1257:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 00000000bbda5115/0xa40083ab9bf52e17 lrc: 3/0,0 mode: PW/PW res: [0x280000400:0x40f:0x0].0x0 rrc: 2 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x40018480000000 nid: local remote: 0x0 expref: -99 pid: 8227 timeout: 0 lvb_type: 0 00010000:00010000:0.0:1776178092.626532:0:20759:0:(ldlm_lockd.c:2159:__ldlm_bl_to_thread()) added 1/1 locks to regular blp list, 1 blwis in pool 00010000:00010000:1.0:1776178092.626546:0:6245:0:(ldlm_lockd.c:1903:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 00000000bbda5115/0xa40083ab9bf52e17 lrc: 1/0,0 mode: --/PW res: [0x280000400:0x40f:0x0].0x0 rrc: 2 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x44819480000000 nid: local remote: 0x0 expref: -99 pid: 8227 timeout: 0 lvb_type: 0 00010000:00010000:1.0:1776178092.626551:0:6245:0:(ldlm_lock.c:216:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 00000000bbda5115/0xa40083ab9bf52e17 lrc: 0/0,0 mode: --/PW res: [0x280000400:0x40f:0x0].0x0 rrc: 2 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x44819480000000 nid: local remote: 0x0 expref: -99 pid: 8227 timeout: 0 lvb_type: 0 00010000:00010000:1.0:1776178092.626569:0:6245:0:(ldlm_lockd.c:2789:ldlm_bl_get_work()) Got 1 locks of 0 total in blp. (0 blwis in pool) 00010000:00010000:1.0:1776178092.626571:0:6245:0:(ldlm_lockd.c:1879:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0001_UUID lock: 0000000097975a3b/0xa40083ab9bf52e25 lrc: 2/0,0 mode: PW/PW res: [0x2c0000400:0x240:0x0].0x0 rrc: 2 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x40010480000000 nid: local remote: 0x0 expref: -99 pid: 20759 timeout: 0 lvb_type: 0 00010000:00010000:1.0:1776178092.626577:0:6245:0:(ldlm_lockd.c:1893:ldlm_handle_bl_callback()) Lock 0000000097975a3b already unused, calling callback (0000000063c97258) 00010000:00010000:1.0:1776178092.626579:0:6245:0:(ldlm_request.c:364:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0001_UUID lock: 0000000097975a3b/0xa40083ab9bf52e25 lrc: 2/0,0 mode: PW/PW res: [0x2c0000400:0x240:0x0].0x0 rrc: 2 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x40010480000000 nid: local remote: 0x0 expref: -99 pid: 20759 timeout: 0 lvb_type: 0 00010000:00010000:1.0:1776178092.626585:0:6245:0:(ldlm_request.c:1257:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0001_UUID lock: 0000000097975a3b/0xa40083ab9bf52e25 lrc: 3/0,0 mode: PW/PW res: [0x2c0000400:0x240:0x0].0x0 rrc: 2 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x40018480000000 nid: local remote: 0x0 expref: -99 pid: 20759 timeout: 0 lvb_type: 0 00010000:00010000:1.0:1776178092.626595:0:6245:0:(ldlm_lockd.c:1903:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0001_UUID lock: 0000000097975a3b/0xa40083ab9bf52e25 lrc: 1/0,0 mode: --/PW res: [0x2c0000400:0x240:0x0].0x0 rrc: 2 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x44819480000000 nid: local remote: 0x0 expref: -99 pid: 20759 timeout: 0 lvb_type: 0 00010000:00010000:1.0:1776178092.626601:0:6245:0:(ldlm_lock.c:216:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0001_UUID lock: 0000000097975a3b/0xa40083ab9bf52e25 lrc: 0/0,0 mode: --/PW res: [0x2c0000400:0x240:0x0].0x0 rrc: 2 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x44819480000000 nid: local remote: 0x0 expref: -99 pid: 20759 timeout: 0 lvb_type: 0 00010000:00010000:1.0:1776178092.626645:0:6245:0:(ldlm_lockd.c:2792:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00010000:00010000:1.0:1776178092.626647:0:6245:0:(ldlm_lockd.c:2792:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00010000:00010000:1.0:1776178092.626648:0:6245:0:(ldlm_lockd.c:2792:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:0.0:1776178092.626913:0:20759:0:(service.c:2372:ptlrpc_server_handle_request()) Handled RPC req@0000000080e4108d pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_010:lustre-MDT0001-mdtlov_UUID+44:3354:x1862456998018624:12345-0@lo:6:osp-syn-1-1.0 Request processed in 17420us (18919us total) trans 4294969313 rc 0/0 00000100:00100000:0.0:1776178092.626928:0:20759:0:(nrs_fifo.c:241:nrs_fifo_req_stop()) NRS stop fifo request from 12345-0@lo, seq: 5225 00000100:00100000:1.0:1776178092.626928:0:20764:0:(service.c:2372:ptlrpc_server_handle_request()) Handled RPC req@0000000003cebab2 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_015:lustre-MDT0001-mdtlov_UUID+52:3353:x1862456998017344:12345-0@lo:6:osp-syn-0-1.0 Request processed in 36317us (36415us total) trans 4294969336 rc 0/0 00000100:00100000:1.0:1776178092.626938:0:20764:0:(nrs_fifo.c:241:nrs_fifo_req_stop()) NRS stop fifo request from 12345-0@lo, seq: 5214 00010000:00010000:0.0:1776178092.626956:0:9224:0:(ldlm_lockd.c:2792:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00010000:00010000:0.0:1776178092.626983:0:20761:0:(ldlm_request.c:199:ldlm_completion_tail()) ### client-side enqueue: granted ns: filter-lustre-OST0000_UUID lock: 00000000499bab84/0xa40083ab9bf52e1e lrc: 3/0,1 mode: PW/PW res: [0x280000400:0x411:0x0].0x0 rrc: 2 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x40010080000000 nid: local remote: 0x0 expref: -99 pid: 20761 timeout: 0 lvb_type: 0 00010000:00010000:0.0:1776178092.626992:0:20761:0:(ldlm_request.c:528:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 00000000499bab84/0xa40083ab9bf52e1e lrc: 3/0,1 mode: PW/PW res: [0x280000400:0x411:0x0].0x0 rrc: 2 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x40010080000000 nid: local remote: 0x0 expref: -99 pid: 20761 timeout: 0 lvb_type: 0 00000004:00080000:1.0:1776178092.626992:0:3353:0:(osp_sync.c:562:osp_sync_interpret()) reply req 00000000e1965319/2, rc 0, transno 2040 00000100:00100000:1.0:1776178092.627000:0:3353:0:(client.c:2215:ptlrpc_check_set()) Completed RPC req@00000000e1965319 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_00:lustre-MDT0001-mdtlov_UUID:3353:1862456998017344:0@lo:6:osp-syn-0-1.0 00010000:00010000:0.0:1776178092.627002:0:20761:0:(ldlm_lock.c:820:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0000_UUID lock: 00000000499bab84/0xa40083ab9bf52e1e lrc: 3/0,1 mode: PW/PW res: [0x280000400:0x411:0x0].0x0 rrc: 2 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x50010080000000 nid: local remote: 0x0 expref: -99 pid: 20761 timeout: 0 lvb_type: 0 00010000:00010000:0.0:1776178092.627011:0:20761:0:(ldlm_lock.c:879:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 00000000499bab84/0xa40083ab9bf52e1e lrc: 2/0,0 mode: PW/PW res: [0x280000400:0x411:0x0].0x0 rrc: 2 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x50010480000000 nid: local remote: 0x0 expref: -99 pid: 20761 timeout: 0 lvb_type: 0 00010000:00010000:0.0:1776178092.627034:0:20761:0:(ldlm_lockd.c:2159:__ldlm_bl_to_thread()) added 1/1 locks to regular blp list, 1 blwis in pool 00000004:00080000:1.0:1776178092.627088:0:3354:0:(osp_sync.c:562:osp_sync_interpret()) reply req 000000002f729f84/2, rc 0, transno 2017 00000100:00100000:1.0:1776178092.627093:0:3354:0:(client.c:2215:ptlrpc_check_set()) Completed RPC req@000000002f729f84 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_01:lustre-MDT0001-mdtlov_UUID:3354:1862456998018624:0@lo:6:osp-syn-1-1.0 00000004:00080000:1.0:1776178092.627153:0:3355:0:(osp_sync.c:562:osp_sync_interpret()) reply req 00000000bc95ec51/2, rc 0, transno 2039 00000100:00100000:1.0:1776178092.627158:0:3355:0:(client.c:2215:ptlrpc_check_set()) Completed RPC req@00000000bc95ec51 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_02:lustre-MDT0001-mdtlov_UUID:3355:1862456998017792:0@lo:6:osp-syn-0-1.0 00000004:00080000:1.0:1776178092.627171:0:3355:0:(osp_sync.c:562:osp_sync_interpret()) reply req 00000000a2dca46e/2, rc 0, transno 2035 00000100:00100000:1.0:1776178092.627174:0:3355:0:(client.c:2215:ptlrpc_check_set()) Completed RPC req@00000000a2dca46e pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_02:lustre-MDT0001-mdtlov_UUID:3355:1862456998016384:0@lo:6:osp-syn-0-1.0 00010000:00010000:1.0:1776178092.627208:0:20769:0:(ldlm_request.c:199:ldlm_completion_tail()) ### client-side enqueue: granted ns: filter-lustre-OST0001_UUID lock: 00000000c9d7d9fc/0xa40083ab9bf52e10 lrc: 3/0,1 mode: PW/PW res: [0x2c0000400:0x2f0:0x0].0x0 rrc: 2 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x40010080000000 nid: local remote: 0x0 expref: -99 pid: 20769 timeout: 0 lvb_type: 0 00010000:00010000:1.0:1776178092.627219:0:20769:0:(ldlm_request.c:528:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0001_UUID lock: 00000000c9d7d9fc/0xa40083ab9bf52e10 lrc: 3/0,1 mode: PW/PW res: [0x2c0000400:0x2f0:0x0].0x0 rrc: 2 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x40010080000000 nid: local remote: 0x0 expref: -99 pid: 20769 timeout: 0 lvb_type: 0 00010000:00010000:1.0:1776178092.627226:0:20769:0:(ldlm_lock.c:820:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0001_UUID lock: 00000000c9d7d9fc/0xa40083ab9bf52e10 lrc: 3/0,1 mode: PW/PW res: [0x2c0000400:0x2f0:0x0].0x0 rrc: 2 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x50010080000000 nid: local remote: 0x0 expref: -99 pid: 20769 timeout: 0 lvb_type: 0 00010000:00010000:1.0:1776178092.627232:0:20769:0:(ldlm_lock.c:879:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0001_UUID lock: 00000000c9d7d9fc/0xa40083ab9bf52e10 lrc: 2/0,0 mode: PW/PW res: [0x2c0000400:0x2f0:0x0].0x0 rrc: 2 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x50010480000000 nid: local remote: 0x0 expref: -99 pid: 20769 timeout: 0 lvb_type: 0 00010000:00010000:1.0:1776178092.627246:0:20769:0:(ldlm_lockd.c:2159:__ldlm_bl_to_thread()) added 1/2 locks to regular blp list, 2 blwis in pool 00000100:00100000:1.0:1776178092.627378:0:20769:0:(service.c:2372:ptlrpc_server_handle_request()) Handled RPC req@0000000054e3866c pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_018:lustre-MDT0001-mdtlov_UUID+44:3355:x1862456998017536:12345-0@lo:6:osp-syn-1-1.0 Request processed in 19759us (19862us total) trans 4294969314 rc 0/0 00000100:00100000:1.0:1776178092.627384:0:20769:0:(nrs_fifo.c:241:nrs_fifo_req_stop()) NRS stop fifo request from 12345-0@lo, seq: 5222 00010000:00010000:1.0:1776178092.627401:0:6244:0:(ldlm_lockd.c:2789:ldlm_bl_get_work()) Got 1 locks of 1 total in blp. (1 blwis in pool) 00010000:00010000:1.0:1776178092.627405:0:6244:0:(ldlm_lockd.c:1879:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 00000000499bab84/0xa40083ab9bf52e1e lrc: 2/0,0 mode: PW/PW res: [0x280000400:0x411:0x0].0x0 rrc: 2 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x40010480000000 nid: local remote: 0x0 expref: -99 pid: 20761 timeout: 0 lvb_type: 0 00010000:00010000:1.0:1776178092.627413:0:6244:0:(ldlm_lockd.c:1893:ldlm_handle_bl_callback()) Lock 00000000499bab84 already unused, calling callback (0000000063c97258) 00010000:00010000:1.0:1776178092.627415:0:6244:0:(ldlm_request.c:364:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 00000000499bab84/0xa40083ab9bf52e1e lrc: 2/0,0 mode: PW/PW res: [0x280000400:0x411:0x0].0x0 rrc: 2 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x40010480000000 nid: local remote: 0x0 expref: -99 pid: 20761 timeout: 0 lvb_type: 0 00010000:00010000:1.0:1776178092.627427:0:6244:0:(ldlm_request.c:1257:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 00000000499bab84/0xa40083ab9bf52e1e lrc: 3/0,0 mode: PW/PW res: [0x280000400:0x411:0x0].0x0 rrc: 2 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x40018480000000 nid: local remote: 0x0 expref: -99 pid: 20761 timeout: 0 lvb_type: 0 00010000:00010000:1.0:1776178092.627442:0:6244:0:(ldlm_lockd.c:1903:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 00000000499bab84/0xa40083ab9bf52e1e lrc: 1/0,0 mode: --/PW res: [0x280000400:0x411:0x0].0x0 rrc: 2 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x44819480000000 nid: local remote: 0x0 expref: -99 pid: 20761 timeout: 0 lvb_type: 0 00010000:00010000:1.0:1776178092.627448:0:6244:0:(ldlm_lock.c:216:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 00000000499bab84/0xa40083ab9bf52e1e lrc: 0/0,0 mode: --/PW res: [0x280000400:0x411:0x0].0x0 rrc: 2 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x44819480000000 nid: local remote: 0x0 expref: -99 pid: 20761 timeout: 0 lvb_type: 0 00010000:00010000:1.0:1776178092.627467:0:6244:0:(ldlm_lockd.c:2789:ldlm_bl_get_work()) Got 1 locks of 0 total in blp. (0 blwis in pool) 00010000:00010000:1.0:1776178092.627470:0:6244:0:(ldlm_lockd.c:1879:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0001_UUID lock: 00000000c9d7d9fc/0xa40083ab9bf52e10 lrc: 2/0,0 mode: PW/PW res: [0x2c0000400:0x2f0:0x0].0x0 rrc: 2 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x40010480000000 nid: local remote: 0x0 expref: -99 pid: 20769 timeout: 0 lvb_type: 0 00010000:00010000:1.0:1776178092.627477:0:6244:0:(ldlm_lockd.c:1893:ldlm_handle_bl_callback()) Lock 00000000c9d7d9fc already unused, calling callback (0000000063c97258) 00010000:00010000:1.0:1776178092.627479:0:6244:0:(ldlm_request.c:364:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0001_UUID lock: 00000000c9d7d9fc/0xa40083ab9bf52e10 lrc: 2/0,0 mode: PW/PW res: [0x2c0000400:0x2f0:0x0].0x0 rrc: 2 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x40010480000000 nid: local remote: 0x0 expref: -99 pid: 20769 timeout: 0 lvb_type: 0 00010000:00010000:1.0:1776178092.627486:0:6244:0:(ldlm_request.c:1257:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0001_UUID lock: 00000000c9d7d9fc/0xa40083ab9bf52e10 lrc: 3/0,0 mode: PW/PW res: [0x2c0000400:0x2f0:0x0].0x0 rrc: 2 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x40018480000000 nid: local remote: 0x0 expref: -99 pid: 20769 timeout: 0 lvb_type: 0 00010000:00010000:1.0:1776178092.627498:0:6244:0:(ldlm_lockd.c:1903:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0001_UUID lock: 00000000c9d7d9fc/0xa40083ab9bf52e10 lrc: 1/0,0 mode: --/PW res: [0x2c0000400:0x2f0:0x0].0x0 rrc: 2 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x44819480000000 nid: local remote: 0x0 expref: -99 pid: 20769 timeout: 0 lvb_type: 0 00010000:00010000:1.0:1776178092.627504:0:6244:0:(ldlm_lock.c:216:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0001_UUID lock: 00000000c9d7d9fc/0xa40083ab9bf52e10 lrc: 0/0,0 mode: --/PW res: [0x2c0000400:0x2f0:0x0].0x0 rrc: 2 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x44819480000000 nid: local remote: 0x0 expref: -99 pid: 20769 timeout: 0 lvb_type: 0 00010000:00010000:1.0:1776178092.627534:0:6244:0:(ldlm_lockd.c:2792:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00010000:00010000:1.0:1776178092.627535:0:6244:0:(ldlm_lockd.c:2792:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00010000:00010000:1.0:1776178092.627537:0:6244:0:(ldlm_lockd.c:2792:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:0.0:1776178092.627539:0:20761:0:(service.c:2372:ptlrpc_server_handle_request()) Handled RPC req@00000000228025f2 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_012:lustre-MDT0001-mdtlov_UUID+51:3355:x1862456998018304:12345-0@lo:6:osp-syn-0-1.0 Request processed in 18441us (19967us total) trans 4294969337 rc 0/0 00000100:00100000:0.0:1776178092.627547:0:20761:0:(nrs_fifo.c:241:nrs_fifo_req_stop()) NRS stop fifo request from 12345-0@lo, seq: 5224 00000004:00080000:1.0:1776178092.627559:0:3355:0:(osp_sync.c:562:osp_sync_interpret()) reply req 00000000a1262b14/2, rc 0, transno 2018 00000100:00100000:1.0:1776178092.627565:0:3355:0:(client.c:2215:ptlrpc_check_set()) Completed RPC req@00000000a1262b14 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_02:lustre-MDT0001-mdtlov_UUID:3355:1862456998017536:0@lo:6:osp-syn-1-1.0 00000004:00080000:1.0:1776178092.627575:0:3355:0:(osp_sync.c:562:osp_sync_interpret()) reply req 000000009c58e3fe/2, rc 0, transno 2041 00000100:00100000:1.0:1776178092.627578:0:3355:0:(client.c:2215:ptlrpc_check_set()) Completed RPC req@000000009c58e3fe pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_02:lustre-MDT0001-mdtlov_UUID:3355:1862456998018304:0@lo:6:osp-syn-0-1.0 00000100:00100000:0.0:1776178092.635756:0:20762:0:(client.c:2215:ptlrpc_check_set()) Completed RPC req@00000000fb56871d pname:cluuid:pid:xid:nid:opc:job ll_ost00_013:lustre-OST0000_UUID:20762:1862456998019136:192.168.204.3@tcp:104: 00010000:00010000:0.0:1776178092.635794:0:20762:0:(ldlm_request.c:283:ldlm_completion_ast()) ### client-side enqueue returned a blocked locksleeping ns: filter-lustre-OST0000_UUID lock: 00000000f779de26/0xa40083ab9bf52e33 lrc: 3/0,1 mode: --/PW res: [0x280000400:0x315:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x40010080000000 nid: local remote: 0x0 expref: -99 pid: 20762 timeout: 0 lvb_type: 0 00000100:00100000:1.0:1776178092.639919:0:20754:0:(client.c:2215:ptlrpc_check_set()) Completed RPC req@000000004a9818ac pname:cluuid:pid:xid:nid:opc:job ll_ost00_005:lustre-OST0000_UUID:20754:1862456998019200:192.168.204.3@tcp:104: 00010000:00010000:1.0:1776178092.639939:0:20754:0:(ldlm_request.c:283:ldlm_completion_ast()) ### client-side enqueue returned a blocked locksleeping ns: filter-lustre-OST0000_UUID lock: 0000000025b13886/0xa40083ab9bf52e3a lrc: 3/0,1 mode: --/PW res: [0x280000400:0x410:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x40010080000000 nid: local remote: 0x0 expref: -99 pid: 20754 timeout: 0 lvb_type: 0 00000100:00100000:1.0:1776178092.640995:0:20770:0:(service.c:2152:ptlrpc_server_handle_req_in()) got req x1862456979075008 00010000:00010000:1.0:1776178092.641016:0:20770:0:(ldlm_lockd.c:2638:ldlm_cancel_hpreq_check()) ### hpreq cancel/convert lock ns: filter-lustre-OST0000_UUID lock: 0000000071c590b2/0xa40083ab9bf3721c lrc: 4/0,0 mode: PR/PR res: [0x280000400:0x315:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x60000400010020 nid: 192.168.204.3@tcp remote: 0x771f68aa1eb24078 expref: 10 pid: 8645 timeout: 899 lvb_type: 1 00000100:00100000:1.0:1776178092.641075:0:20770:0:(nrs_fifo.c:179:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.204.3@tcp, seq: 2070 00000100:00100000:1.0:1776178092.641084:0:20770:0:(service.c:2324:ptlrpc_server_handle_request()) Handling RPC req@00000000fb56871d pname:cluuid+ref:pid:xid:nid:opc:job ldlm_cn00_005:b50963bd-8eba-4432-9a54-8888d00e3fa1+10:2270:x1862456979074944:12345-192.168.204.3@tcp:103:ldlm_bl_01.0 00010000:00010000:1.0:1776178092.641105:0:20770:0:(ldlm_lockd.c:1720:ldlm_request_cancel()) ### server-side cancel handler START: 1 locks, starting at 0 00010000:00010000:1.0:1776178092.641110:0:20770:0:(ldlm_lockd.c:1771:ldlm_request_cancel()) ### server cancels blocked lock after 0s ns: filter-lustre-OST0000_UUID lock: 00000000d685ba19/0xa40083ab9bf3a258 lrc: 4/0,0 mode: PR/PR res: [0x280000400:0x377:0x0].0x0 rrc: 4 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x60000400010020 nid: 192.168.204.3@tcp remote: 0x771f68aa1eb2508c expref: 10 pid: 8645 timeout: 899 lvb_type: 1 00000100:00100000:0.0:1776178092.641526:0:6242:0:(nrs_fifo.c:179:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.204.3@tcp, seq: 2071 00000100:00100000:0.0:1776178092.641534:0:6242:0:(service.c:2324:ptlrpc_server_handle_request()) Handling RPC req@000000008245b0ee pname:cluuid+ref:pid:xid:nid:opc:job ldlm_cn00_001:b50963bd-8eba-4432-9a54-8888d00e3fa1+10:2270:x1862456979075008:12345-192.168.204.3@tcp:103:ldlm_bl_01.0 00010000:00010000:0.0:1776178092.641545:0:6242:0:(ldlm_lockd.c:1720:ldlm_request_cancel()) ### server-side cancel handler START: 1 locks, starting at 0 00010000:00010000:0.0:1776178092.641551:0:6242:0:(ldlm_lockd.c:1771:ldlm_request_cancel()) ### server cancels blocked lock after 0s ns: filter-lustre-OST0000_UUID lock: 0000000071c590b2/0xa40083ab9bf3721c lrc: 4/0,0 mode: PR/PR res: [0x280000400:0x315:0x0].0x0 rrc: 4 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x60000400010020 nid: 192.168.204.3@tcp remote: 0x771f68aa1eb24078 expref: 10 pid: 8645 timeout: 899 lvb_type: 1 00010000:00010000:0.0:1776178092.641564:0:6242:0:(ldlm_lockd.c:562:ldlm_del_waiting_lock()) ### removed ns: filter-lustre-OST0000_UUID lock: 0000000071c590b2/0xa40083ab9bf3721c lrc: 3/0,0 mode: PR/PR res: [0x280000400:0x315:0x0].0x0 rrc: 4 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x50000400010020 nid: 192.168.204.3@tcp remote: 0x771f68aa1eb24078 expref: 10 pid: 8645 timeout: 899 lvb_type: 1 00010000:00010000:0.0:1776178092.641599:0:6242:0:(ldlm_lock.c:216:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 0000000071c590b2/0xa40083ab9bf3721c lrc: 0/0,0 mode: --/PR res: [0x280000400:0x315:0x0].0x0 rrc: 4 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x44801400010020 nid: 192.168.204.3@tcp remote: 0x771f68aa1eb24078 expref: 10 pid: 8645 timeout: 899 lvb_type: 1 00010000:00010000:0.0:1776178092.641616:0:6242:0:(ldlm_extent.c:318:ldlm_check_contention()) contended locks = 0 00010000:00010000:0.0:1776178092.641619:0:6242:0:(ldlm_extent.c:318:ldlm_check_contention()) contended locks = 0 00010000:00010000:0.0:1776178092.641623:0:6242:0:(ldlm_lock.c:704:ldlm_add_cp_work_item()) ### lock granted; sending completion AST. ns: filter-lustre-OST0000_UUID lock: 00000000f779de26/0xa40083ab9bf52e33 lrc: 3/0,1 mode: PW/PW res: [0x280000400:0x315:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x40090080000000 nid: local remote: 0x0 expref: -99 pid: 20762 timeout: 0 lvb_type: 0 00010000:00010000:0.0:1776178092.641633:0:6242:0:(ldlm_resource.c:1650:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 00000000f779de26/0xa40083ab9bf52e33 lrc: 4/0,1 mode: PW/PW res: [0x280000400:0x315:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x40090080000000 nid: local remote: 0x0 expref: -99 pid: 20762 timeout: 0 lvb_type: 0 00010000:00010000:1.0:1776178092.641676:0:20770:0:(ldlm_lockd.c:562:ldlm_del_waiting_lock()) ### removed ns: filter-lustre-OST0000_UUID lock: 00000000d685ba19/0xa40083ab9bf3a258 lrc: 3/0,0 mode: PR/PR res: [0x280000400:0x377:0x0].0x0 rrc: 4 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x50000400010020 nid: 192.168.204.3@tcp remote: 0x771f68aa1eb2508c expref: 9 pid: 8645 timeout: 899 lvb_type: 1 00010000:00010000:0.0:1776178092.641679:0:6242:0:(ldlm_lockd.c:1782:ldlm_request_cancel()) ### server-side cancel handler END 00010000:00010000:1.0:1776178092.641703:0:20770:0:(ldlm_lock.c:216:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 00000000d685ba19/0xa40083ab9bf3a258 lrc: 0/0,0 mode: --/PR res: [0x280000400:0x377:0x0].0x0 rrc: 4 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x44801400010020 nid: 192.168.204.3@tcp remote: 0x771f68aa1eb2508c expref: 9 pid: 8645 timeout: 899 lvb_type: 1 00010000:00010000:1.0:1776178092.641713:0:20770:0:(ldlm_extent.c:318:ldlm_check_contention()) contended locks = 0 00010000:00010000:1.0:1776178092.641715:0:20770:0:(ldlm_extent.c:318:ldlm_check_contention()) contended locks = 0 00010000:00010000:1.0:1776178092.641718:0:20770:0:(ldlm_lock.c:704:ldlm_add_cp_work_item()) ### lock granted; sending completion AST. ns: filter-lustre-OST0000_UUID lock: 00000000c1e7ea3e/0xa40083ab9bf52e2c lrc: 3/0,1 mode: PW/PW res: [0x280000400:0x377:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x40090080000000 nid: local remote: 0x0 expref: -99 pid: 14440 timeout: 0 lvb_type: 0 00010000:00010000:1.0:1776178092.641725:0:20770:0:(ldlm_resource.c:1650:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 00000000c1e7ea3e/0xa40083ab9bf52e2c lrc: 4/0,1 mode: PW/PW res: [0x280000400:0x377:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x40090080000000 nid: local remote: 0x0 expref: -99 pid: 14440 timeout: 0 lvb_type: 0 00000100:00100000:0.0:1776178092.641736:0:6242:0:(service.c:2372:ptlrpc_server_handle_request()) Handled RPC req@000000008245b0ee pname:cluuid+ref:pid:xid:nid:opc:job ldlm_cn00_001:b50963bd-8eba-4432-9a54-8888d00e3fa1+8:2270:x1862456979075008:12345-192.168.204.3@tcp:103:ldlm_bl_01.0 Request processed in 200us (1054us total) trans 0 rc 0/0 00000100:00100000:0.0:1776178092.641746:0:6242:0:(nrs_fifo.c:241:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.204.3@tcp, seq: 2071 00010000:00010000:1.0:1776178092.641748:0:20770:0:(ldlm_lockd.c:1782:ldlm_request_cancel()) ### server-side cancel handler END 00000100:00100000:1.0:1776178092.641771:0:20770:0:(service.c:2372:ptlrpc_server_handle_request()) Handled RPC req@00000000fb56871d pname:cluuid+ref:pid:xid:nid:opc:job ldlm_cn00_005:b50963bd-8eba-4432-9a54-8888d00e3fa1+8:2270:x1862456979074944:12345-192.168.204.3@tcp:103:ldlm_bl_01.0 Request processed in 686us (3276us total) trans 0 rc 0/0 00000100:00100000:1.0:1776178092.641780:0:20770:0:(nrs_fifo.c:241:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.204.3@tcp, seq: 2070 00010000:00010000:1.0:1776178092.641807:0:14440:0:(ldlm_request.c:199:ldlm_completion_tail()) ### client-side enqueue: granted ns: filter-lustre-OST0000_UUID lock: 00000000c1e7ea3e/0xa40083ab9bf52e2c lrc: 3/0,1 mode: PW/PW res: [0x280000400:0x377:0x0].0x0 rrc: 2 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x40010080000000 nid: local remote: 0x0 expref: -99 pid: 14440 timeout: 0 lvb_type: 0 00010000:00010000:0.0:1776178092.642127:0:6245:0:(ldlm_lockd.c:2789:ldlm_bl_get_work()) Got 1 locks of 0 total in blp. (0 blwis in pool) 00010000:00010000:0.0:1776178092.642134:0:6245:0:(ldlm_lockd.c:1879:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 00000000f779de26/0xa40083ab9bf52e33 lrc: 2/0,0 mode: PW/PW res: [0x280000400:0x315:0x0].0x0 rrc: 2 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x40010480000000 nid: local remote: 0x0 expref: -99 pid: 20762 timeout: 0 lvb_type: 0 00010000:00010000:0.0:1776178092.642144:0:6245:0:(ldlm_lockd.c:1893:ldlm_handle_bl_callback()) Lock 00000000f779de26 already unused, calling callback (0000000063c97258) 00010000:00010000:0.0:1776178092.642149:0:6245:0:(ldlm_request.c:364:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 00000000f779de26/0xa40083ab9bf52e33 lrc: 2/0,0 mode: PW/PW res: [0x280000400:0x315:0x0].0x0 rrc: 2 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x40010480000000 nid: local remote: 0x0 expref: -99 pid: 20762 timeout: 0 lvb_type: 0 00010000:00010000:0.0:1776178092.642159:0:6245:0:(ldlm_request.c:1257:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 00000000f779de26/0xa40083ab9bf52e33 lrc: 3/0,0 mode: PW/PW res: [0x280000400:0x315:0x0].0x0 rrc: 2 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x40018480000000 nid: local remote: 0x0 expref: -99 pid: 20762 timeout: 0 lvb_type: 0 00010000:00010000:0.0:1776178092.642173:0:6245:0:(ldlm_lockd.c:1903:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 00000000f779de26/0xa40083ab9bf52e33 lrc: 1/0,0 mode: --/PW res: [0x280000400:0x315:0x0].0x0 rrc: 2 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x44819480000000 nid: local remote: 0x0 expref: -99 pid: 20762 timeout: 0 lvb_type: 0 00010000:00010000:0.0:1776178092.642179:0:6245:0:(ldlm_lock.c:216:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 00000000f779de26/0xa40083ab9bf52e33 lrc: 0/0,0 mode: --/PW res: [0x280000400:0x315:0x0].0x0 rrc: 2 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x44819480000000 nid: local remote: 0x0 expref: -99 pid: 20762 timeout: 0 lvb_type: 0 00010000:00010000:0.0:1776178092.642198:0:6245:0:(ldlm_lockd.c:2792:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00010000:00010000:0.0:1776178092.642199:0:6245:0:(ldlm_lockd.c:2792:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00010000:00010000:0.0:1776178092.642201:0:6245:0:(ldlm_lockd.c:2792:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00010000:00010000:1.0:1776178092.643767:0:14440:0:(ldlm_request.c:528:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 00000000c1e7ea3e/0xa40083ab9bf52e2c lrc: 3/0,1 mode: PW/PW res: [0x280000400:0x377:0x0].0x0 rrc: 2 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x40010080000000 nid: local remote: 0x0 expref: -99 pid: 14440 timeout: 0 lvb_type: 0 00010000:00010000:1.0:1776178092.643778:0:14440:0:(ldlm_lock.c:820:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0000_UUID lock: 00000000c1e7ea3e/0xa40083ab9bf52e2c lrc: 3/0,1 mode: PW/PW res: [0x280000400:0x377:0x0].0x0 rrc: 2 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x50010080000000 nid: local remote: 0x0 expref: -99 pid: 14440 timeout: 0 lvb_type: 0 00010000:00010000:1.0:1776178092.643784:0:14440:0:(ldlm_lock.c:879:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 00000000c1e7ea3e/0xa40083ab9bf52e2c lrc: 2/0,0 mode: PW/PW res: [0x280000400:0x377:0x0].0x0 rrc: 2 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x50010480000000 nid: local remote: 0x0 expref: -99 pid: 14440 timeout: 0 lvb_type: 0 00010000:00010000:1.0:1776178092.643915:0:14440:0:(ldlm_lockd.c:2159:__ldlm_bl_to_thread()) added 1/1 locks to regular blp list, 1 blwis in pool 00010000:00010000:0.0:1776178092.643931:0:20754:0:(ldlm_request.c:199:ldlm_completion_tail()) ### client-side enqueue: granted ns: filter-lustre-OST0000_UUID lock: 0000000025b13886/0xa40083ab9bf52e3a lrc: 3/0,1 mode: PW/PW res: [0x280000400:0x410:0x0].0x0 rrc: 2 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x40010080000000 nid: local remote: 0x0 expref: -99 pid: 20754 timeout: 0 lvb_type: 0 00010000:00010000:0.0:1776178092.643942:0:20754:0:(ldlm_request.c:528:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 0000000025b13886/0xa40083ab9bf52e3a lrc: 3/0,1 mode: PW/PW res: [0x280000400:0x410:0x0].0x0 rrc: 2 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x40010080000000 nid: local remote: 0x0 expref: -99 pid: 20754 timeout: 0 lvb_type: 0 00010000:00010000:0.0:1776178092.643949:0:20754:0:(ldlm_lock.c:820:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0000_UUID lock: 0000000025b13886/0xa40083ab9bf52e3a lrc: 3/0,1 mode: PW/PW res: [0x280000400:0x410:0x0].0x0 rrc: 2 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x50010080000000 nid: local remote: 0x0 expref: -99 pid: 20754 timeout: 0 lvb_type: 0 00010000:00010000:0.0:1776178092.643956:0:20754:0:(ldlm_lock.c:879:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 0000000025b13886/0xa40083ab9bf52e3a lrc: 2/0,0 mode: PW/PW res: [0x280000400:0x410:0x0].0x0 rrc: 2 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x50010480000000 nid: local remote: 0x0 expref: -99 pid: 20754 timeout: 0 lvb_type: 0 00010000:00010000:0.0:1776178092.643977:0:20754:0:(ldlm_lockd.c:2159:__ldlm_bl_to_thread()) added 1/2 locks to regular blp list, 2 blwis in pool 00000100:00100000:1.0:1776178092.644198:0:14440:0:(service.c:2372:ptlrpc_server_handle_request()) Handled RPC req@0000000031795589 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_004:lustre-MDT0001-mdtlov_UUID+52:3354:x1862456998018880:12345-0@lo:6:osp-syn-0-1.0 Request processed in 24577us (25898us total) trans 4294969339 rc 0/0 00000100:00100000:1.0:1776178092.644206:0:14440:0:(nrs_fifo.c:241:nrs_fifo_req_stop()) NRS stop fifo request from 12345-0@lo, seq: 5226 00000100:00100000:0.0:1776178092.644276:0:20754:0:(service.c:2372:ptlrpc_server_handle_request()) Handled RPC req@00000000de27a229 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_005:lustre-MDT0001-mdtlov_UUID+51:3355:x1862456998019008:12345-0@lo:6:osp-syn-0-1.0 Request processed in 23183us (23259us total) trans 4294969340 rc 0/0 00000004:00080000:1.0:1776178092.644277:0:3354:0:(osp_sync.c:562:osp_sync_interpret()) reply req 00000000e8f6b843/2, rc 0, transno 2043 00000100:00100000:0.0:1776178092.644288:0:20754:0:(nrs_fifo.c:241:nrs_fifo_req_stop()) NRS stop fifo request from 12345-0@lo, seq: 5228 00000100:00100000:1.0:1776178092.644290:0:3354:0:(client.c:2215:ptlrpc_check_set()) Completed RPC req@00000000e8f6b843 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_01:lustre-MDT0001-mdtlov_UUID:3354:1862456998018880:0@lo:6:osp-syn-0-1.0 00000004:00080000:1.0:1776178092.644504:0:3355:0:(osp_sync.c:562:osp_sync_interpret()) reply req 000000009cb972eb/2, rc 0, transno 2044 00000100:00100000:1.0:1776178092.644510:0:3355:0:(client.c:2215:ptlrpc_check_set()) Completed RPC req@000000009cb972eb pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_02:lustre-MDT0001-mdtlov_UUID:3355:1862456998019008:0@lo:6:osp-syn-0-1.0 00000004:00080000:1.0:1776178092.644520:0:3355:0:(osp_sync.c:562:osp_sync_interpret()) reply req 0000000084754621/2, rc 0, transno 2042 00000100:00100000:1.0:1776178092.644522:0:3355:0:(client.c:2215:ptlrpc_check_set()) Completed RPC req@0000000084754621 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_02:lustre-MDT0001-mdtlov_UUID:3355:1862456998018944:0@lo:6:osp-syn-0-1.0 00000100:00100000:1.0:1776178092.896956:0:3354:0:(ptlrpcd.c:415:ptlrpcd_check()) transfer 1 async RPCs [0->1] 00000100:00100000:1.0:1776178092.896990:0:3354:0:(client.c:1733:ptlrpc_send_new_req()) Sending RPC req@000000000154171f pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_01:lustre-MDT0000-mdtlov_UUID:3354:1862456998019264:0@lo:13:osp-pre-0-0.0 00000100:00100000:1.0:1776178092.897190:0:3354:0:(events.c:361:request_in_callback()) peer: 12345-0@lo (source: 12345-0@lo) 00000100:00100000:0.0:1776178092.897397:0:3353:0:(ptlrpcd.c:415:ptlrpcd_check()) transfer 1 async RPCs [1->0] 00000100:00100000:0.0:1776178092.897409:0:3353:0:(client.c:1733:ptlrpc_send_new_req()) Sending RPC req@000000001bbe82bd pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_00:lustre-MDT0000-mdtlov_UUID:3353:1862456998019328:0@lo:13:osp-pre-1-0.0 00000100:00100000:0.0:1776178092.897454:0:3355:0:(client.c:1733:ptlrpc_send_new_req()) Sending RPC req@000000003493d657 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_02:lustre-MDT0001-mdtlov_UUID:3355:1862456998019392:0@lo:41:osp-pre-0-1.0 00000100:00100000:0.0:1776178092.897485:0:3355:0:(events.c:361:request_in_callback()) peer: 12345-0@lo (source: 12345-0@lo) 00000100:00100000:0.0:1776178092.897527:0:3355:0:(ptlrpcd.c:415:ptlrpcd_check()) transfer 1 async RPCs [3->2] 00000100:00100000:0.0:1776178092.897534:0:3355:0:(client.c:1733:ptlrpc_send_new_req()) Sending RPC req@00000000fd0161d4 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_02:lustre-MDT0000-mdtlov_UUID:3355:1862456998019456:0@lo:41:osp-pre-1-0.0 00000100:00100000:0.0:1776178092.897556:0:3355:0:(events.c:361:request_in_callback()) peer: 12345-0@lo (source: 12345-0@lo) 00000100:00100000:0.0:1776178092.897601:0:6261:0:(service.c:2152:ptlrpc_server_handle_req_in()) got req x1862456998019392 00000100:00100000:0.0:1776178092.897657:0:6261:0:(service.c:2152:ptlrpc_server_handle_req_in()) got req x1862456998019456 00000100:00100000:0.0:1776178092.897681:0:6261:0:(nrs_fifo.c:179:nrs_fifo_req_get()) NRS start fifo request from 12345-0@lo, seq: 607 00000100:00100000:0.0:1776178092.897689:0:6261:0:(service.c:2324:ptlrpc_server_handle_request()) Handling RPC req@00000000c5909a76 pname:cluuid+ref:pid:xid:nid:opc:job mdt_out00_001:lustre-MDT0001-mdtlov_UUID+6:3355:x1862456998019392:12345-0@lo:41:osp-pre-0-1.0 00000004:00000024:0.0:1776178092.897737:0:6261:0:(mdt_handler.c:548:mdt_statfs()) blocks cached 0 granted 2097152 pending 0 free 1446125568 avail 1316663296 00000100:00100000:0.0:1776178092.897790:0:6261:0:(service.c:2372:ptlrpc_server_handle_request()) Handled RPC req@00000000c5909a76 pname:cluuid+ref:pid:xid:nid:opc:job mdt_out00_001:lustre-MDT0001-mdtlov_UUID+6:3355:x1862456998019392:12345-0@lo:41:osp-pre-0-1.0 Request processed in 101us (306us total) trans 0 rc 0/0 00000100:00100000:0.0:1776178092.897798:0:6261:0:(nrs_fifo.c:241:nrs_fifo_req_stop()) NRS stop fifo request from 12345-0@lo, seq: 607 00000100:00100000:0.0:1776178092.897827:0:6261:0:(nrs_fifo.c:179:nrs_fifo_req_get()) NRS start fifo request from 12345-0@lo, seq: 608 00000100:00100000:0.0:1776178092.897831:0:6261:0:(service.c:2324:ptlrpc_server_handle_request()) Handling RPC req@0000000088356013 pname:cluuid+ref:pid:xid:nid:opc:job mdt_out00_001:lustre-MDT0000-mdtlov_UUID+5:3355:x1862456998019456:12345-0@lo:41:osp-pre-1-0.0 00000004:00000024:0.0:1776178092.897849:0:6261:0:(mdt_handler.c:548:mdt_statfs()) blocks cached 0 granted 2097152 pending 0 free 1446039552 avail 1316577280 00000100:00100000:0.0:1776178092.897872:0:6261:0:(service.c:2372:ptlrpc_server_handle_request()) Handled RPC req@0000000088356013 pname:cluuid+ref:pid:xid:nid:opc:job mdt_out00_001:lustre-MDT0000-mdtlov_UUID+5:3355:x1862456998019456:12345-0@lo:41:osp-pre-1-0.0 Request processed in 41us (317us total) trans 0 rc 0/0 00000100:00100000:0.0:1776178092.897878:0:6261:0:(nrs_fifo.c:241:nrs_fifo_req_stop()) NRS stop fifo request from 12345-0@lo, seq: 608 00000100:00100000:0.0:1776178092.897923:0:3355:0:(client.c:2215:ptlrpc_check_set()) Completed RPC req@00000000fd0161d4 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_02:lustre-MDT0000-mdtlov_UUID:3355:1862456998019456:0@lo:41:osp-pre-1-0.0 00000100:00100000:0.0:1776178092.897933:0:3355:0:(client.c:2868:ptlrpc_free_committed()) lustre-MDT0000-osp-MDT0001: committing for last_committed 12884901895 gen 3 00000004:00080000:0.0:1776178092.897938:0:3355:0:(osp_trans.c:1044:osp_request_commit_cb()) trans no 12884901895 committed transno 12884901895 00000020:00080000:0.0:1776178092.897944:0:3355:0:(update_trans.c:80:top_multiple_thandle_dump()) lustre-MDT0001-osd tmt 000000001318f906 refcount 2 committed 0 result 0 batchid 12884901895 00000020:00080000:0.0:1776178092.897947:0:3355:0:(update_trans.c:89:top_multiple_thandle_dump()) st 00000000409a6a55 obd lustre-MDT0000-osp-MDT0001 committed 1 started 1 stopped 1 result 0 sub_th 00000000b8f976e8 00000020:00080000:0.0:1776178092.897950:0:3355:0:(update_trans.c:94:top_multiple_thandle_dump()) cookie [0x200000bd1:0x1:0x0].7 00000020:00080000:0.0:1776178092.897952:0:3355:0:(update_trans.c:89:top_multiple_thandle_dump()) st 00000000198fca5b obd lustre-MDT0001-osd committed 1 started 1 stopped 1 result 0 sub_th 000000003ad8a6c5 00000020:00080000:0.0:1776178092.897955:0:3355:0:(update_trans.c:94:top_multiple_thandle_dump()) cookie [0x240001b70:0x1:0x0].8 00000100:00100000:0.0:1776178092.897989:0:3355:0:(client.c:2215:ptlrpc_check_set()) Completed RPC req@000000003493d657 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_02:lustre-MDT0001-mdtlov_UUID:3355:1862456998019392:0@lo:41:osp-pre-0-1.0 00000020:00080000:1.0:1776178092.898522:0:18116:0:(update_trans.c:1589:distribute_txn_commit_thread()) lustre-MDT0001: batchid: 12884901895 committed batchid 12884901894 00000020:00080000:1.0:1776178092.898600:0:18116:0:(update_trans.c:1589:distribute_txn_commit_thread()) lustre-MDT0001: batchid: 0 committed batchid 12884901894 00000100:00100000:0.0:1776178092.898686:0:3353:0:(events.c:361:request_in_callback()) peer: 12345-0@lo (source: 12345-0@lo) 00000100:00100000:1.0:1776178092.901766:0:3354:0:(client.c:1733:ptlrpc_send_new_req()) Sending RPC req@0000000067671fc8 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_01:lustre-MDT0000-lwp-OST0000_UUID:3354:1862456998019584:0@lo:400:kworker/u8:2.0 00000100:00100000:1.0:1776178092.901853:0:3354:0:(events.c:361:request_in_callback()) peer: 12345-0@lo (source: 12345-0@lo) 00000100:00100000:1.0:1776178092.901885:0:3354:0:(client.c:1733:ptlrpc_send_new_req()) Sending RPC req@00000000a30eb964 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_01:lustre-MDT0001-mdtlov_UUID:3354:1862456998019840:0@lo:400:kworker/u8:2.0 00000100:00100000:1.0:1776178092.901913:0:3354:0:(events.c:361:request_in_callback()) peer: 12345-0@lo (source: 12345-0@lo) 00000100:00100000:1.0:1776178092.901940:0:3354:0:(client.c:1733:ptlrpc_send_new_req()) Sending RPC req@00000000f720f286 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_01:lustre-MDT0000-lwp-MDT0000_UUID:3354:1862456998020096:0@lo:400:kworker/u8:2.0 00000100:00100000:1.0:1776178092.901960:0:3354:0:(events.c:361:request_in_callback()) peer: 12345-0@lo (source: 12345-0@lo) 00000100:00100000:1.0:1776178092.901985:0:3354:0:(ptlrpcd.c:415:ptlrpcd_check()) transfer 3 async RPCs [0->1] 00000100:00100000:1.0:1776178092.901990:0:3354:0:(client.c:1733:ptlrpc_send_new_req()) Sending RPC req@0000000023210d1c pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_01:ac61f839-0e8e-4cac-9c59-08bcfbbad330:3354:1862456998019520:0@lo:400:kworker/u8:2.0 00000100:00100000:1.0:1776178092.902015:0:3354:0:(events.c:361:request_in_callback()) peer: 12345-0@lo (source: 12345-0@lo) 00000100:00100000:1.0:1776178092.902028:0:3354:0:(client.c:1733:ptlrpc_send_new_req()) Sending RPC req@00000000b766edf6 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_01:lustre-MDT0001-lwp-OST0001_UUID:3354:1862456998019776:0@lo:400:kworker/u8:2.0 00000100:00100000:1.0:1776178092.902044:0:3354:0:(events.c:361:request_in_callback()) peer: 12345-0@lo (source: 12345-0@lo) 00000100:00100000:1.0:1776178092.902053:0:3354:0:(client.c:1733:ptlrpc_send_new_req()) Sending RPC req@0000000017d54e92 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_01:lustre-MDT0000-mdtlov_UUID:3354:1862456998020032:0@lo:400:kworker/u8:2.0 00000100:00100000:1.0:1776178092.902089:0:3354:0:(events.c:361:request_in_callback()) peer: 12345-0@lo (source: 12345-0@lo) 00000100:00100000:1.0:1776178092.902117:0:6257:0:(service.c:2152:ptlrpc_server_handle_req_in()) got req x1862456998019584 00000100:00100000:1.0:1776178092.902154:0:6257:0:(service.c:2152:ptlrpc_server_handle_req_in()) got req x1862456998020096 00000100:00100000:1.0:1776178092.902165:0:6257:0:(service.c:2152:ptlrpc_server_handle_req_in()) got req x1862456998019776 00000100:00100000:1.0:1776178092.902176:0:6257:0:(nrs_fifo.c:179:nrs_fifo_req_get()) NRS start fifo request from 12345-0@lo, seq: 731 00000100:00100000:1.0:1776178092.902180:0:6257:0:(service.c:2324:ptlrpc_server_handle_request()) Handling RPC req@00000000a770fd7b pname:cluuid+ref:pid:xid:nid:opc:job mdt00_002:lustre-MDT0000-lwp-OST0000_UUID+5:3354:x1862456998019584:12345-0@lo:400:kworker/u8:2.0 00000100:00100000:1.0:1776178092.902483:0:6257:0:(service.c:2372:ptlrpc_server_handle_request()) Handled RPC req@00000000a770fd7b pname:cluuid+ref:pid:xid:nid:opc:job mdt00_002:lustre-MDT0000-lwp-OST0000_UUID+5:3354:x1862456998019584:12345-0@lo:400:kworker/u8:2.0 Request processed in 302us (631us total) trans 0 rc 0/0 00000100:00100000:1.0:1776178092.902489:0:6257:0:(nrs_fifo.c:241:nrs_fifo_req_stop()) NRS stop fifo request from 12345-0@lo, seq: 731 00000100:00100000:1.0:1776178092.902504:0:6257:0:(nrs_fifo.c:179:nrs_fifo_req_get()) NRS start fifo request from 12345-0@lo, seq: 732 00000100:00100000:1.0:1776178092.902506:0:6257:0:(service.c:2324:ptlrpc_server_handle_request()) Handling RPC req@0000000018b5e3c6 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_002:lustre-MDT0000-lwp-MDT0000_UUID+5:3354:x1862456998020096:12345-0@lo:400:kworker/u8:2.0 00000100:00100000:1.0:1776178092.902522:0:6257:0:(service.c:2372:ptlrpc_server_handle_request()) Handled RPC req@0000000018b5e3c6 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_002:lustre-MDT0000-lwp-MDT0000_UUID+5:3354:x1862456998020096:12345-0@lo:400:kworker/u8:2.0 Request processed in 16us (563us total) trans 0 rc 0/0 00000100:00100000:1.0:1776178092.902525:0:6257:0:(nrs_fifo.c:241:nrs_fifo_req_stop()) NRS stop fifo request from 12345-0@lo, seq: 732 00000100:00100000:1.0:1776178092.902531:0:6257:0:(nrs_fifo.c:179:nrs_fifo_req_get()) NRS start fifo request from 12345-0@lo, seq: 733 00000100:00100000:1.0:1776178092.902533:0:6257:0:(service.c:2324:ptlrpc_server_handle_request()) Handling RPC req@0000000043a010c5 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_002:lustre-MDT0001-lwp-OST0001_UUID+5:3354:x1862456998019776:12345-0@lo:400:kworker/u8:2.0 00000100:00100000:1.0:1776178092.902544:0:6257:0:(service.c:2372:ptlrpc_server_handle_request()) Handled RPC req@0000000043a010c5 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_002:lustre-MDT0001-lwp-OST0001_UUID+5:3354:x1862456998019776:12345-0@lo:400:kworker/u8:2.0 Request processed in 11us (501us total) trans 0 rc 0/0 00000100:00100000:1.0:1776178092.902547:0:6257:0:(nrs_fifo.c:241:nrs_fifo_req_stop()) NRS stop fifo request from 12345-0@lo, seq: 733 00000100:00100000:1.0:1776178092.902560:0:20137:0:(service.c:2152:ptlrpc_server_handle_req_in()) got req x1862456998019520 00000100:00100000:1.0:1776178092.902576:0:20137:0:(nrs_fifo.c:179:nrs_fifo_req_get()) NRS start fifo request from 12345-0@lo, seq: 111 00000100:00100000:1.0:1776178092.902579:0:20137:0:(service.c:2324:ptlrpc_server_handle_request()) Handling RPC req@00000000c201d06e pname:cluuid+ref:pid:xid:nid:opc:job ll_mgs_0000:ac61f839-0e8e-4cac-9c59-08bcfbbad330+23:3354:x1862456998019520:12345-0@lo:400:kworker/u8:2.0 00000100:00100000:1.0:1776178092.902595:0:20137:0:(service.c:2372:ptlrpc_server_handle_request()) Handled RPC req@00000000c201d06e pname:cluuid+ref:pid:xid:nid:opc:job ll_mgs_0000:ac61f839-0e8e-4cac-9c59-08bcfbbad330+23:3354:x1862456998019520:12345-0@lo:400:kworker/u8:2.0 Request processed in 15us (581us total) trans 0 rc 0/0 00000100:00100000:1.0:1776178092.902598:0:20137:0:(nrs_fifo.c:241:nrs_fifo_req_stop()) NRS stop fifo request from 12345-0@lo, seq: 111 00000100:00100000:1.0:1776178092.902626:0:14440:0:(service.c:2152:ptlrpc_server_handle_req_in()) got req x1862456998019840 00000100:00100000:1.0:1776178092.902640:0:14440:0:(service.c:2152:ptlrpc_server_handle_req_in()) got req x1862456998020032 00000100:00100000:1.0:1776178092.902650:0:14440:0:(nrs_fifo.c:179:nrs_fifo_req_get()) NRS start fifo request from 12345-0@lo, seq: 393 00000100:00100000:1.0:1776178092.902652:0:14440:0:(service.c:2324:ptlrpc_server_handle_request()) Handling RPC req@0000000053234520 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_004:lustre-MDT0001-mdtlov_UUID+51:3354:x1862456998019840:12345-0@lo:400:kworker/u8:2.0 00000100:00100000:1.0:1776178092.902669:0:14440:0:(service.c:2372:ptlrpc_server_handle_request()) Handled RPC req@0000000053234520 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_004:lustre-MDT0001-mdtlov_UUID+51:3354:x1862456998019840:12345-0@lo:400:kworker/u8:2.0 Request processed in 16us (757us total) trans 0 rc 0/0 00000100:00100000:1.0:1776178092.902673:0:14440:0:(nrs_fifo.c:241:nrs_fifo_req_stop()) NRS stop fifo request from 12345-0@lo, seq: 393 00000100:00100000:1.0:1776178092.902679:0:14440:0:(nrs_fifo.c:179:nrs_fifo_req_get()) NRS start fifo request from 12345-0@lo, seq: 394 00000100:00100000:1.0:1776178092.902682:0:14440:0:(service.c:2324:ptlrpc_server_handle_request()) Handling RPC req@000000002973935f pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_004:lustre-MDT0000-mdtlov_UUID+5:3354:x1862456998020032:12345-0@lo:400:kworker/u8:2.0 00000100:00100000:1.0:1776178092.902703:0:14440:0:(service.c:2372:ptlrpc_server_handle_request()) Handled RPC req@000000002973935f pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_004:lustre-MDT0000-mdtlov_UUID+5:3354:x1862456998020032:12345-0@lo:400:kworker/u8:2.0 Request processed in 19us (613us total) trans 0 rc 0/0 00000100:00100000:1.0:1776178092.902709:0:14440:0:(nrs_fifo.c:241:nrs_fifo_req_stop()) NRS stop fifo request from 12345-0@lo, seq: 394 00000100:00100000:1.0:1776178092.902741:0:3354:0:(client.c:2215:ptlrpc_check_set()) Completed RPC req@0000000023210d1c pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_01:ac61f839-0e8e-4cac-9c59-08bcfbbad330:3354:1862456998019520:0@lo:400:kworker/u8:2.0 00000100:00100000:1.0:1776178092.902754:0:3354:0:(client.c:2215:ptlrpc_check_set()) Completed RPC req@00000000b766edf6 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_01:lustre-MDT0001-lwp-OST0001_UUID:3354:1862456998019776:0@lo:400:kworker/u8:2.0 00000100:00100000:1.0:1776178092.902763:0:3354:0:(client.c:2215:ptlrpc_check_set()) Completed RPC req@0000000017d54e92 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_01:lustre-MDT0000-mdtlov_UUID:3354:1862456998020032:0@lo:400:kworker/u8:2.0 00000100:00100000:1.0:1776178092.902771:0:3354:0:(client.c:2215:ptlrpc_check_set()) Completed RPC req@0000000067671fc8 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_01:lustre-MDT0000-lwp-OST0000_UUID:3354:1862456998019584:0@lo:400:kworker/u8:2.0 00000100:00100000:1.0:1776178092.902779:0:3354:0:(client.c:2215:ptlrpc_check_set()) Completed RPC req@00000000a30eb964 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_01:lustre-MDT0001-mdtlov_UUID:3354:1862456998019840:0@lo:400:kworker/u8:2.0 00000100:00100000:1.0:1776178092.902787:0:3354:0:(client.c:2215:ptlrpc_check_set()) Completed RPC req@00000000f720f286 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_01:lustre-MDT0000-lwp-MDT0000_UUID:3354:1862456998020096:0@lo:400:kworker/u8:2.0 00000100:00100000:1.0:1776178092.902878:0:3355:0:(client.c:1733:ptlrpc_send_new_req()) Sending RPC req@00000000e64148ab pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_02:lustre-MDT0001-lwp-OST0000_UUID:3355:1862456998019648:0@lo:400:kworker/u8:2.0 00000100:00100000:1.0:1776178092.902903:0:3355:0:(events.c:361:request_in_callback()) peer: 12345-0@lo (source: 12345-0@lo) 00000100:00100000:1.0:1776178092.902921:0:8250:0:(service.c:2152:ptlrpc_server_handle_req_in()) got req x1862456998019648 00000100:00100000:1.0:1776178092.902934:0:8250:0:(nrs_fifo.c:179:nrs_fifo_req_get()) NRS start fifo request from 12345-0@lo, seq: 734 00000100:00100000:1.0:1776178092.902937:0:8250:0:(service.c:2324:ptlrpc_server_handle_request()) Handling RPC req@00000000f720f286 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_003:lustre-MDT0001-lwp-OST0000_UUID+5:3355:x1862456998019648:12345-0@lo:400:kworker/u8:2.0 00000100:00100000:1.0:1776178092.902954:0:8250:0:(service.c:2372:ptlrpc_server_handle_request()) Handled RPC req@00000000f720f286 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_003:lustre-MDT0001-lwp-OST0000_UUID+5:3355:x1862456998019648:12345-0@lo:400:kworker/u8:2.0 Request processed in 16us (51us total) trans 0 rc 0/0 00000100:00100000:1.0:1776178092.902957:0:8250:0:(nrs_fifo.c:241:nrs_fifo_req_stop()) NRS stop fifo request from 12345-0@lo, seq: 734 00000100:00100000:1.0:1776178092.902968:0:3355:0:(client.c:1733:ptlrpc_send_new_req()) Sending RPC req@00000000a59bfc89 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_02:lustre-MDT0001-mdtlov_UUID:3355:1862456998019904:0@lo:400:kworker/u8:2.0 00000100:00100000:1.0:1776178092.902980:0:3355:0:(events.c:361:request_in_callback()) peer: 12345-0@lo (source: 12345-0@lo) 00000100:00100000:1.0:1776178092.902994:0:20754:0:(service.c:2152:ptlrpc_server_handle_req_in()) got req x1862456998019904 00000100:00100000:1.0:1776178092.903007:0:20754:0:(nrs_fifo.c:179:nrs_fifo_req_get()) NRS start fifo request from 12345-0@lo, seq: 395 00000100:00100000:1.0:1776178092.903010:0:20754:0:(service.c:2324:ptlrpc_server_handle_request()) Handling RPC req@0000000017d54e92 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_005:lustre-MDT0001-mdtlov_UUID+44:3355:x1862456998019904:12345-0@lo:400:kworker/u8:2.0 00000100:00100000:1.0:1776178092.903034:0:20754:0:(service.c:2372:ptlrpc_server_handle_request()) Handled RPC req@0000000017d54e92 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_005:lustre-MDT0001-mdtlov_UUID+44:3355:x1862456998019904:12345-0@lo:400:kworker/u8:2.0 Request processed in 23us (53us total) trans 0 rc 0/0 00000100:00100000:1.0:1776178092.903040:0:20754:0:(nrs_fifo.c:241:nrs_fifo_req_stop()) NRS stop fifo request from 12345-0@lo, seq: 395 00000100:00100000:1.0:1776178092.903092:0:3355:0:(client.c:2215:ptlrpc_check_set()) Completed RPC req@00000000e64148ab pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_02:lustre-MDT0001-lwp-OST0000_UUID:3355:1862456998019648:0@lo:400:kworker/u8:2.0 00000100:00100000:1.0:1776178092.903105:0:3355:0:(client.c:2215:ptlrpc_check_set()) Completed RPC req@00000000a59bfc89 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_02:lustre-MDT0001-mdtlov_UUID:3355:1862456998019904:0@lo:400:kworker/u8:2.0 00000100:00100000:1.0:1776178092.903117:0:3355:0:(ptlrpcd.c:415:ptlrpcd_check()) transfer 2 async RPCs [3->2] 00000100:00100000:0.0:1776178092.903289:0:3355:0:(client.c:1733:ptlrpc_send_new_req()) Sending RPC req@000000009d5112f6 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_02:lustre-MDT0000-lwp-OST0001_UUID:3355:1862456998019712:0@lo:400:kworker/u8:2.0 00000100:00100000:0.0:1776178092.903908:0:3355:0:(events.c:361:request_in_callback()) peer: 12345-0@lo (source: 12345-0@lo) 00000100:00100000:0.0:1776178092.903932:0:3355:0:(client.c:1733:ptlrpc_send_new_req()) Sending RPC req@00000000ba273b2a pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_02:lustre-MDT0000-lwp-MDT0001_UUID:3355:1862456998019968:0@lo:400:kworker/u8:2.0 00000100:00100000:0.0:1776178092.903955:0:3355:0:(events.c:361:request_in_callback()) peer: 12345-0@lo (source: 12345-0@lo) 00000100:00100000:0.0:1776178092.903978:0:6255:0:(service.c:2152:ptlrpc_server_handle_req_in()) got req x1862456998019712 00000100:00100000:0.0:1776178092.904003:0:6255:0:(service.c:2152:ptlrpc_server_handle_req_in()) got req x1862456998019968 00000100:00100000:0.0:1776178092.904034:0:6255:0:(nrs_fifo.c:179:nrs_fifo_req_get()) NRS start fifo request from 12345-0@lo, seq: 735 00000100:00100000:0.0:1776178092.904038:0:6255:0:(service.c:2324:ptlrpc_server_handle_request()) Handling RPC req@00000000fd0161d4 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_000:lustre-MDT0000-lwp-OST0001_UUID+5:3355:x1862456998019712:12345-0@lo:400:kworker/u8:2.0 00000100:00100000:0.0:1776178092.904103:0:6255:0:(service.c:2372:ptlrpc_server_handle_request()) Handled RPC req@00000000fd0161d4 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_000:lustre-MDT0000-lwp-OST0001_UUID+5:3355:x1862456998019712:12345-0@lo:400:kworker/u8:2.0 Request processed in 64us (196us total) trans 0 rc 0/0 00000100:00100000:0.0:1776178092.904109:0:6255:0:(nrs_fifo.c:241:nrs_fifo_req_stop()) NRS stop fifo request from 12345-0@lo, seq: 735 00000100:00100000:0.0:1776178092.904132:0:6255:0:(nrs_fifo.c:179:nrs_fifo_req_get()) NRS start fifo request from 12345-0@lo, seq: 736 00000100:00100000:0.0:1776178092.904135:0:6255:0:(service.c:2324:ptlrpc_server_handle_request()) Handling RPC req@00000000a5677c3e pname:cluuid+ref:pid:xid:nid:opc:job mdt00_000:lustre-MDT0000-lwp-MDT0001_UUID+5:3355:x1862456998019968:12345-0@lo:400:kworker/u8:2.0 00000100:00100000:0.0:1776178092.904165:0:6255:0:(service.c:2372:ptlrpc_server_handle_request()) Handled RPC req@00000000a5677c3e pname:cluuid+ref:pid:xid:nid:opc:job mdt00_000:lustre-MDT0000-lwp-MDT0001_UUID+5:3355:x1862456998019968:12345-0@lo:400:kworker/u8:2.0 Request processed in 29us (210us total) trans 0 rc 0/0 00000100:00100000:0.0:1776178092.904170:0:6255:0:(nrs_fifo.c:241:nrs_fifo_req_stop()) NRS stop fifo request from 12345-0@lo, seq: 736 00000100:00100000:0.0:1776178092.904195:0:3355:0:(client.c:2215:ptlrpc_check_set()) Completed RPC req@000000009d5112f6 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_02:lustre-MDT0000-lwp-OST0001_UUID:3355:1862456998019712:0@lo:400:kworker/u8:2.0 00000100:00100000:0.0:1776178092.904204:0:3355:0:(client.c:2215:ptlrpc_check_set()) Completed RPC req@00000000ba273b2a pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_02:lustre-MDT0000-lwp-MDT0001_UUID:3355:1862456998019968:0@lo:400:kworker/u8:2.0 00000100:00100000:0.0:1776178092.904232:0:3353:0:(client.c:2215:ptlrpc_check_set()) Completed RPC req@000000001bbe82bd pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_00:lustre-MDT0000-mdtlov_UUID:3353:1862456998019328:0@lo:13:osp-pre-1-0.0 00000100:00100000:0.0:1776178093.860398:0:3347:0:(events.c:361:request_in_callback()) peer: 12345-192.168.204.3@tcp (source: 12345-192.168.204.3@tcp) 00000100:00100000:0.0:1776178093.860447:0:3347:0:(events.c:361:request_in_callback()) peer: 12345-192.168.204.3@tcp (source: 12345-192.168.204.3@tcp) 00000100:00100000:0.0:1776178093.860472:0:3347:0:(events.c:361:request_in_callback()) peer: 12345-192.168.204.3@tcp (source: 12345-192.168.204.3@tcp) 00000100:00100000:0.0:1776178093.860496:0:3347:0:(events.c:361:request_in_callback()) peer: 12345-192.168.204.3@tcp (source: 12345-192.168.204.3@tcp) 00000100:00100000:0.0:1776178093.860522:0:3347:0:(events.c:361:request_in_callback()) peer: 12345-192.168.204.3@tcp (source: 12345-192.168.204.3@tcp) 00000100:00100000:0.0:1776178093.860566:0:20140:0:(service.c:2152:ptlrpc_server_handle_req_in()) got req x1862456979075136 00000100:00100000:0.0:1776178093.860625:0:20140:0:(nrs_fifo.c:179:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.204.3@tcp, seq: 112 00000100:00100000:0.0:1776178093.860632:0:20140:0:(service.c:2324:ptlrpc_server_handle_request()) Handling RPC req@000000009c2c404e pname:cluuid+ref:pid:xid:nid:opc:job ll_mgs_0002:5b5c66b1-1a2f-4651-8548-0c75b9a38a94+9:2273:x1862456979075136:12345-192.168.204.3@tcp:400:kworker/u8:3.0 00000100:00100000:0.0:1776178093.860724:0:20140:0:(service.c:2372:ptlrpc_server_handle_request()) Handled RPC req@000000009c2c404e pname:cluuid+ref:pid:xid:nid:opc:job ll_mgs_0002:5b5c66b1-1a2f-4651-8548-0c75b9a38a94+9:2273:x1862456979075136:12345-192.168.204.3@tcp:400:kworker/u8:3.0 Request processed in 91us (338us total) trans 0 rc 0/0 00000100:00100000:0.0:1776178093.861467:0:20140:0:(nrs_fifo.c:241:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.204.3@tcp, seq: 112 00000100:00100000:0.0:1776178093.861528:0:11281:0:(service.c:2152:ptlrpc_server_handle_req_in()) got req x1862456979075264 00000100:00100000:0.0:1776178093.861563:0:11281:0:(service.c:2152:ptlrpc_server_handle_req_in()) got req x1862456979075200 00000100:00100000:0.0:1776178093.861579:0:11281:0:(nrs_fifo.c:179:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.204.3@tcp, seq: 737 00000100:00100000:0.0:1776178093.861583:0:11281:0:(service.c:2324:ptlrpc_server_handle_request()) Handling RPC req@000000001bbe82bd pname:cluuid+ref:pid:xid:nid:opc:job mdt00_004:b50963bd-8eba-4432-9a54-8888d00e3fa1+6:2273:x1862456979075264:12345-192.168.204.3@tcp:400:kworker/u8:3.0 00000100:00100000:0.0:1776178093.862019:0:11281:0:(service.c:2372:ptlrpc_server_handle_request()) Handled RPC req@000000001bbe82bd pname:cluuid+ref:pid:xid:nid:opc:job mdt00_004:b50963bd-8eba-4432-9a54-8888d00e3fa1+6:2273:x1862456979075264:12345-192.168.204.3@tcp:400:kworker/u8:3.0 Request processed in 76us (1214us total) trans 0 rc 0/0 00000100:00100000:0.0:1776178093.862030:0:11281:0:(nrs_fifo.c:241:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.204.3@tcp, seq: 737 00000100:00100000:0.0:1776178093.862048:0:11281:0:(nrs_fifo.c:179:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.204.3@tcp, seq: 738 00000100:00100000:0.0:1776178093.862053:0:11281:0:(service.c:2324:ptlrpc_server_handle_request()) Handling RPC req@000000006d22e848 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_004:b50963bd-8eba-4432-9a54-8888d00e3fa1+8:2273:x1862456979075200:12345-192.168.204.3@tcp:400:kworker/u8:3.0 00000100:00100000:0.0:1776178093.862099:0:11281:0:(service.c:2372:ptlrpc_server_handle_request()) Handled RPC req@000000006d22e848 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_004:b50963bd-8eba-4432-9a54-8888d00e3fa1+8:2273:x1862456979075200:12345-192.168.204.3@tcp:400:kworker/u8:3.0 Request processed in 46us (1603us total) trans 0 rc 0/0 00000100:00100000:0.0:1776178093.862110:0:11281:0:(nrs_fifo.c:241:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.204.3@tcp, seq: 738 00000100:00100000:0.0:1776178093.862134:0:20761:0:(service.c:2152:ptlrpc_server_handle_req_in()) got req x1862456979075392 00000100:00100000:0.0:1776178093.862155:0:20761:0:(service.c:2152:ptlrpc_server_handle_req_in()) got req x1862456979075328 00000100:00100000:1.0:1776178093.862167:0:20754:0:(nrs_fifo.c:179:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.204.3@tcp, seq: 396 00000100:00100000:1.0:1776178093.862175:0:20754:0:(service.c:2324:ptlrpc_server_handle_request()) Handling RPC req@00000000badbd310 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_005:b50963bd-8eba-4432-9a54-8888d00e3fa1+5:2273:x1862456979075392:12345-192.168.204.3@tcp:400:kworker/u8:3.0 00000100:00100000:0.0:1776178093.862178:0:20761:0:(nrs_fifo.c:179:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.204.3@tcp, seq: 397 00000100:00100000:0.0:1776178093.862186:0:20761:0:(service.c:2324:ptlrpc_server_handle_request()) Handling RPC req@000000006e48e9ca pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_012:b50963bd-8eba-4432-9a54-8888d00e3fa1+5:2273:x1862456979075328:12345-192.168.204.3@tcp:400:kworker/u8:3.0 00000100:00100000:1.0:1776178093.862252:0:20754:0:(service.c:2372:ptlrpc_server_handle_request()) Handled RPC req@00000000badbd310 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_005:b50963bd-8eba-4432-9a54-8888d00e3fa1+5:2273:x1862456979075392:12345-192.168.204.3@tcp:400:kworker/u8:3.0 Request processed in 77us (1779us total) trans 0 rc 0/0 00000100:00100000:1.0:1776178093.862260:0:20754:0:(nrs_fifo.c:241:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.204.3@tcp, seq: 396 00000100:00100000:0.0:1776178093.862263:0:20761:0:(service.c:2372:ptlrpc_server_handle_request()) Handled RPC req@000000006e48e9ca pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_012:b50963bd-8eba-4432-9a54-8888d00e3fa1+5:2273:x1862456979075328:12345-192.168.204.3@tcp:400:kworker/u8:3.0 Request processed in 76us (1740us total) trans 0 rc 0/0 00000100:00100000:0.0:1776178093.862272:0:20761:0:(nrs_fifo.c:241:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.204.3@tcp, seq: 397 00000100:00100000:1.0:1776178094.432521:0:8231:0:(service.c:2152:ptlrpc_server_handle_req_in()) got req x1862456998020160 00000100:00100000:1.0:1776178094.432587:0:8231:0:(nrs_fifo.c:179:nrs_fifo_req_get()) NRS start fifo request from 12345-0@lo, seq: 393 00000100:00100000:1.0:1776178094.432594:0:8231:0:(service.c:2324:ptlrpc_server_handle_request()) Handling RPC req@00000000f06d9dbd pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_create00:lustre-MDT0001-mdtlov_UUID+44:3356:x1862456998020160:12345-0@lo:13:osp-pre-1-1.0 00002000:00000024:1.0:1776178094.432627:0:8231:0:(ofd_obd.c:740:ofd_statfs()) blocks cached 229376 granted 491746368 pending 0 free 3923689472 avail 3692167168 00000100:00100000:1.0:1776178094.432682:0:8231:0:(service.c:2372:ptlrpc_server_handle_request()) Handled RPC req@00000000f06d9dbd pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_create00:lustre-MDT0001-mdtlov_UUID+44:3356:x1862456998020160:12345-0@lo:13:osp-pre-1-1.0 Request processed in 88us (273us total) trans 0 rc 0/0 00000100:00100000:1.0:1776178094.432688:0:8231:0:(nrs_fifo.c:241:nrs_fifo_req_stop()) NRS stop fifo request from 12345-0@lo, seq: 393 00000001:00080000:0.0:1776178094.439023:0:20134:0:(tgt_lastrcvd.c:913:tgt_cb_last_committed()) lustre-MDT0000: transno 12884901897 is committed 00010000:00010000:0.0:1776178094.496364:0:6245:0:(ldlm_lockd.c:2792:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:0.0:1776178094.944566:0:8232:0:(service.c:2152:ptlrpc_server_handle_req_in()) got req x1862456998020224 00000100:00100000:0.0:1776178094.944677:0:8232:0:(nrs_fifo.c:179:nrs_fifo_req_get()) NRS start fifo request from 12345-0@lo, seq: 394 00000100:00100000:0.0:1776178094.944686:0:8232:0:(service.c:2324:ptlrpc_server_handle_request()) Handling RPC req@00000000500c0ae6 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_create00:lustre-MDT0001-mdtlov_UUID+51:3355:x1862456998020224:12345-0@lo:13:osp-pre-0-1.0 00002000:00000024:0.0:1776178094.944758:0:8232:0:(ofd_obd.c:740:ofd_statfs()) blocks cached 315392 granted 484883776 pending 0 free 3923689472 avail 3692167168 00000100:00100000:0.0:1776178094.944843:0:8232:0:(service.c:2372:ptlrpc_server_handle_request()) Handled RPC req@00000000500c0ae6 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_create00:lustre-MDT0001-mdtlov_UUID+51:3355:x1862456998020224:12345-0@lo:13:osp-pre-0-1.0 Request processed in 154us (365us total) trans 0 rc 0/0 00000100:00100000:0.0:1776178094.944853:0:8232:0:(nrs_fifo.c:241:nrs_fifo_req_stop()) NRS stop fifo request from 12345-0@lo, seq: 394 00010000:00010000:1.0:1776178095.530192:0:9225:0:(ldlm_lockd.c:2792:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:0.0:1776178095.793428:0:3347:0:(events.c:361:request_in_callback()) peer: 12345-192.168.204.3@tcp (source: 12345-192.168.204.3@tcp) 00000100:00100000:1.0:1776178095.793612:0:6255:0:(service.c:2152:ptlrpc_server_handle_req_in()) got req x1862456979075520 00000100:00100000:1.0:1776178095.793717:0:6255:0:(nrs_fifo.c:179:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.204.3@tcp, seq: 6455 00000100:00100000:1.0:1776178095.793728:0:6255:0:(service.c:2324:ptlrpc_server_handle_request()) Handling RPC req@0000000045e2a220 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_000:b50963bd-8eba-4432-9a54-8888d00e3fa1+6:16976:x1862456979075520:12345-192.168.204.3@tcp:101:lfs.0 00010000:00010000:1.0:1776178095.793778:0:6255:0:(ldlm_lockd.c:1248:ldlm_handle_enqueue0()) ### server-side enqueue handler START 00010000:00010000:1.0:1776178095.793810:0:6255:0:(ldlm_lockd.c:1318:ldlm_handle_enqueue0()) ### server-side enqueue handler, new lock created ns: mdt-lustre-MDT0000_UUID lock: 00000000499bab84/0xa40083ab9bf52e4f lrc: 2/0,0 mode: --/CR res: [0x200000007:0x1:0x0].0x0 bits 0x0/0x0 rrc: 4 type: IBT gid 0 flags: 0x40000000000000 nid: local remote: 0x771f68aa1eb2794a expref: -99 pid: 6255 timeout: 0 lvb_type: 0 00010000:00010000:1.0:1776178095.793926:0:6255:0:(ldlm_lock.c:766:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(CR) ns: mdt-lustre-MDT0000_UUID lock: 0000000097975a3b/0xa40083ab9bf52e56 lrc: 3/1,0 mode: --/CR res: [0x200000007:0x1:0x0].0x0 bits 0x0/0x0 rrc: 5 type: IBT gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 6255 timeout: 0 lvb_type: 0 00010000:00010000:1.0:1776178095.793952:0:6255:0:(ldlm_lock.c:1064:ldlm_granted_list_add_lock()) ### About to add lock: ns: mdt-lustre-MDT0000_UUID lock: 0000000097975a3b/0xa40083ab9bf52e56 lrc: 3/1,0 mode: CR/CR res: [0x200000007:0x1:0x0].0x0 bits 0x2/0x0 rrc: 5 type: IBT gid 0 flags: 0x50210000000000 nid: local remote: 0x0 expref: -99 pid: 6255 timeout: 0 lvb_type: 0 00010000:00010000:1.0:1776178095.793970:0:6255:0:(ldlm_request.c:528:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: mdt-lustre-MDT0000_UUID lock: 0000000097975a3b/0xa40083ab9bf52e56 lrc: 3/1,0 mode: CR/CR res: [0x200000007:0x1:0x0].0x0 bits 0x2/0x0 rrc: 5 type: IBT gid 0 flags: 0x40210000000000 nid: local remote: 0x0 expref: -99 pid: 6255 timeout: 0 lvb_type: 0 00010000:00010000:1.0:1776178095.793994:0:6255:0:(ldlm_lock.c:766:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PR) ns: mdt-lustre-MDT0000_UUID lock: 00000000daa24072/0xa40083ab9bf52e5d lrc: 3/1,0 mode: --/PR res: [0x200000007:0x1:0x0].0xc2d93176 bits 0x0/0x0 rrc: 2 type: IBT gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 6255 timeout: 0 lvb_type: 0 00010000:00010000:1.0:1776178095.794002:0:6255:0:(ldlm_lock.c:1064:ldlm_granted_list_add_lock()) ### About to add lock: ns: mdt-lustre-MDT0000_UUID lock: 00000000daa24072/0xa40083ab9bf52e5d lrc: 3/1,0 mode: PR/PR res: [0x200000007:0x1:0x0].0xc2d93176 bits 0x2/0x0 rrc: 2 type: IBT gid 0 flags: 0x50210000000000 nid: local remote: 0x0 expref: -99 pid: 6255 timeout: 0 lvb_type: 0 00010000:00010000:1.0:1776178095.794016:0:6255:0:(ldlm_request.c:528:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: mdt-lustre-MDT0000_UUID lock: 00000000daa24072/0xa40083ab9bf52e5d lrc: 3/1,0 mode: PR/PR res: [0x200000007:0x1:0x0].0xc2d93176 bits 0x2/0x0 rrc: 2 type: IBT gid 0 flags: 0x40210000000000 nid: local remote: 0x0 expref: -99 pid: 6255 timeout: 0 lvb_type: 0 00010000:00010000:1.0:1776178095.794125:0:6255:0:(ldlm_lock.c:820:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(CR) ns: mdt-lustre-MDT0000_UUID lock: 0000000097975a3b/0xa40083ab9bf52e56 lrc: 3/1,0 mode: CR/CR res: [0x200000007:0x1:0x0].0x0 bits 0x2/0x0 rrc: 5 type: IBT gid 0 flags: 0x50210000000000 nid: local remote: 0x0 expref: -99 pid: 6255 timeout: 0 lvb_type: 0 00010000:00010000:1.0:1776178095.794133:0:6255:0:(ldlm_lock.c:879:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: mdt-lustre-MDT0000_UUID lock: 0000000097975a3b/0xa40083ab9bf52e56 lrc: 2/0,0 mode: CR/CR res: [0x200000007:0x1:0x0].0x0 bits 0x2/0x0 rrc: 5 type: IBT gid 0 flags: 0x50210400000000 nid: local remote: 0x0 expref: -99 pid: 6255 timeout: 0 lvb_type: 0 00010000:00010000:1.0:1776178095.794142:0:6255:0:(ldlm_lockd.c:1879:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: mdt-lustre-MDT0000_UUID lock: 0000000097975a3b/0xa40083ab9bf52e56 lrc: 3/0,0 mode: CR/CR res: [0x200000007:0x1:0x0].0x0 bits 0x2/0x0 rrc: 5 type: IBT gid 0 flags: 0x40210400000000 nid: local remote: 0x0 expref: -99 pid: 6255 timeout: 0 lvb_type: 0 00010000:00010000:1.0:1776178095.794150:0:6255:0:(ldlm_lockd.c:1893:ldlm_handle_bl_callback()) Lock 0000000097975a3b already unused, calling callback (0000000079e78348) 00010000:00010000:1.0:1776178095.794155:0:6255:0:(ldlm_request.c:364:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: mdt-lustre-MDT0000_UUID lock: 0000000097975a3b/0xa40083ab9bf52e56 lrc: 3/0,0 mode: CR/CR res: [0x200000007:0x1:0x0].0x0 bits 0x2/0x0 rrc: 5 type: IBT gid 0 flags: 0x40210400000000 nid: local remote: 0x0 expref: -99 pid: 6255 timeout: 0 lvb_type: 0 00010000:00010000:1.0:1776178095.794164:0:6255:0:(ldlm_request.c:1257:ldlm_cli_cancel_local()) ### server-side local cancel ns: mdt-lustre-MDT0000_UUID lock: 0000000097975a3b/0xa40083ab9bf52e56 lrc: 4/0,0 mode: CR/CR res: [0x200000007:0x1:0x0].0x0 bits 0x2/0x0 rrc: 5 type: IBT gid 0 flags: 0x40218400000000 nid: local remote: 0x0 expref: -99 pid: 6255 timeout: 0 lvb_type: 0 00010000:00010000:1.0:1776178095.794183:0:6255:0:(ldlm_inodebits.c:94:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x200000007:0x1:0x0].0x0 (00000000ad0d4094) 00010000:00010000:1.0:1776178095.794186:0:6255:0:(ldlm_inodebits.c:96:ldlm_reprocess_inodebits_queue()) Hint 2 00010000:00010000:1.0:1776178095.794193:0:6255:0:(ldlm_lockd.c:1903:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: mdt-lustre-MDT0000_UUID lock: 0000000097975a3b/0xa40083ab9bf52e56 lrc: 2/0,0 mode: --/CR res: [0x200000007:0x1:0x0].0x0 bits 0x2/0x0 rrc: 5 type: IBT gid 0 flags: 0x44a19400000000 nid: local remote: 0x0 expref: -99 pid: 6255 timeout: 0 lvb_type: 0 00010000:00010000:1.0:1776178095.794200:0:6255:0:(ldlm_lock.c:216:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: mdt-lustre-MDT0000_UUID lock: 0000000097975a3b/0xa40083ab9bf52e56 lrc: 0/0,0 mode: --/CR res: [0x200000007:0x1:0x0].0x0 bits 0x2/0x0 rrc: 5 type: IBT gid 0 flags: 0x44a19400000000 nid: local remote: 0x0 expref: -99 pid: 6255 timeout: 0 lvb_type: 0 00010000:00010000:1.0:1776178095.794216:0:6255:0:(ldlm_lock.c:820:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PR) ns: mdt-lustre-MDT0000_UUID lock: 00000000daa24072/0xa40083ab9bf52e5d lrc: 3/1,0 mode: PR/PR res: [0x200000007:0x1:0x0].0xc2d93176 bits 0x2/0x0 rrc: 2 type: IBT gid 0 flags: 0x50210000000000 nid: local remote: 0x0 expref: -99 pid: 6255 timeout: 0 lvb_type: 0 00010000:00010000:1.0:1776178095.794227:0:6255:0:(ldlm_lock.c:879:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: mdt-lustre-MDT0000_UUID lock: 00000000daa24072/0xa40083ab9bf52e5d lrc: 2/0,0 mode: PR/PR res: [0x200000007:0x1:0x0].0xc2d93176 bits 0x2/0x0 rrc: 2 type: IBT gid 0 flags: 0x50210400000000 nid: local remote: 0x0 expref: -99 pid: 6255 timeout: 0 lvb_type: 0 00010000:00010000:1.0:1776178095.794248:0:6255:0:(ldlm_lockd.c:1879:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: mdt-lustre-MDT0000_UUID lock: 00000000daa24072/0xa40083ab9bf52e5d lrc: 3/0,0 mode: PR/PR res: [0x200000007:0x1:0x0].0xc2d93176 bits 0x2/0x0 rrc: 2 type: IBT gid 0 flags: 0x40210400000000 nid: local remote: 0x0 expref: -99 pid: 6255 timeout: 0 lvb_type: 0 00010000:00010000:1.0:1776178095.794254:0:6255:0:(ldlm_lockd.c:1893:ldlm_handle_bl_callback()) Lock 00000000daa24072 already unused, calling callback (0000000079e78348) 00010000:00010000:1.0:1776178095.794257:0:6255:0:(ldlm_request.c:364:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: mdt-lustre-MDT0000_UUID lock: 00000000daa24072/0xa40083ab9bf52e5d lrc: 3/0,0 mode: PR/PR res: [0x200000007:0x1:0x0].0xc2d93176 bits 0x2/0x0 rrc: 2 type: IBT gid 0 flags: 0x40210400000000 nid: local remote: 0x0 expref: -99 pid: 6255 timeout: 0 lvb_type: 0 00010000:00010000:1.0:1776178095.794263:0:6255:0:(ldlm_request.c:1257:ldlm_cli_cancel_local()) ### server-side local cancel ns: mdt-lustre-MDT0000_UUID lock: 00000000daa24072/0xa40083ab9bf52e5d lrc: 4/0,0 mode: PR/PR res: [0x200000007:0x1:0x0].0xc2d93176 bits 0x2/0x0 rrc: 2 type: IBT gid 0 flags: 0x40218400000000 nid: local remote: 0x0 expref: -99 pid: 6255 timeout: 0 lvb_type: 0 00010000:00010000:1.0:1776178095.794271:0:6255:0:(ldlm_inodebits.c:94:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x200000007:0x1:0x0].0xc2d93176 (000000003b465a63) 00010000:00010000:1.0:1776178095.794274:0:6255:0:(ldlm_inodebits.c:96:ldlm_reprocess_inodebits_queue()) Hint 2 00010000:00010000:1.0:1776178095.794276:0:6255:0:(ldlm_lockd.c:1903:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: mdt-lustre-MDT0000_UUID lock: 00000000daa24072/0xa40083ab9bf52e5d lrc: 2/0,0 mode: --/PR res: [0x200000007:0x1:0x0].0xc2d93176 bits 0x2/0x0 rrc: 2 type: IBT gid 0 flags: 0x44a19400000000 nid: local remote: 0x0 expref: -99 pid: 6255 timeout: 0 lvb_type: 0 00010000:00010000:1.0:1776178095.794282:0:6255:0:(ldlm_lock.c:216:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: mdt-lustre-MDT0000_UUID lock: 00000000daa24072/0xa40083ab9bf52e5d lrc: 0/0,0 mode: --/PR res: [0x200000007:0x1:0x0].0xc2d93176 bits 0x2/0x0 rrc: 2 type: IBT gid 0 flags: 0x44a19400000000 nid: local remote: 0x0 expref: -99 pid: 6255 timeout: 0 lvb_type: 0 00010000:00010000:1.0:1776178095.794331:0:6255:0:(ldlm_lockd.c:1484:ldlm_handle_enqueue0()) ### server-side enqueue handler, sending reply (err=301, rc=0) ns: mdt-lustre-MDT0000_UUID lock: 00000000499bab84/0xa40083ab9bf52e4f lrc: 1/0,0 mode: --/CR res: [0x200000007:0x1:0x0].0x0 bits 0x2/0x0 rrc: 4 type: IBT gid 0 flags: 0x44000000000000 nid: 192.168.204.3@tcp remote: 0x771f68aa1eb2794a expref: 7 pid: 6255 timeout: 0 lvb_type: 0 00010000:00010000:1.0:1776178095.794341:0:6255:0:(ldlm_lock.c:216:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: mdt-lustre-MDT0000_UUID lock: 00000000499bab84/0xa40083ab9bf52e4f lrc: 0/0,0 mode: --/CR res: [0x200000007:0x1:0x0].0x0 bits 0x2/0x0 rrc: 4 type: IBT gid 0 flags: 0x44000000000000 nid: 192.168.204.3@tcp remote: 0x771f68aa1eb2794a expref: 7 pid: 6255 timeout: 0 lvb_type: 0 00010000:00010000:1.0:1776178095.794352:0:6255:0:(ldlm_lockd.c:1565:ldlm_handle_enqueue0()) ### server-side enqueue handler END (lock 00000000499bab84, rc 0) 00000100:00100000:1.0:1776178095.795213:0:6255:0:(service.c:2372:ptlrpc_server_handle_request()) Handled RPC req@0000000045e2a220 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_000:b50963bd-8eba-4432-9a54-8888d00e3fa1+6:16976:x1862456979075520:12345-192.168.204.3@tcp:101:lfs.0 Request processed in 1480us (1804us total) trans 0 rc 301/301 00000100:00100000:1.0:1776178095.795228:0:6255:0:(nrs_fifo.c:241:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.204.3@tcp, seq: 6455 00000100:00100000:1.0:1776178095.807780:0:3346:0:(events.c:361:request_in_callback()) peer: 12345-192.168.204.3@tcp (source: 12345-192.168.204.3@tcp) 00000100:00100000:1.0:1776178095.830418:0:6255:0:(service.c:2152:ptlrpc_server_handle_req_in()) got req x1862456979075648 00000100:00100000:1.0:1776178095.830528:0:6255:0:(nrs_fifo.c:179:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.204.3@tcp, seq: 6457 00000100:00100000:1.0:1776178095.830537:0:6255:0:(service.c:2324:ptlrpc_server_handle_request()) Handling RPC req@00000000d94d52a5 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_000:b50963bd-8eba-4432-9a54-8888d00e3fa1+6:16976:x1862456979075648:12345-192.168.204.3@tcp:101:lfs.0 00010000:00010000:1.0:1776178095.830573:0:6255:0:(ldlm_lockd.c:1248:ldlm_handle_enqueue0()) ### server-side enqueue handler START 00010000:00010000:1.0:1776178095.830605:0:6255:0:(ldlm_lockd.c:1318:ldlm_handle_enqueue0()) ### server-side enqueue handler, new lock created ns: mdt-lustre-MDT0000_UUID lock: 00000000499bab84/0xa40083ab9bf52e79 lrc: 2/0,0 mode: --/CR res: [0x200000007:0x1:0x0].0x0 bits 0x0/0x0 rrc: 4 type: IBT gid 0 flags: 0x40000000000000 nid: local remote: 0x771f68aa1eb27958 expref: -99 pid: 6255 timeout: 0 lvb_type: 0 00010000:00010000:1.0:1776178095.830717:0:6255:0:(ldlm_lock.c:766:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PR) ns: mdt-lustre-MDT0000_UUID lock: 00000000daa24072/0xa40083ab9bf52e80 lrc: 3/1,0 mode: --/PR res: [0x200000007:0x1:0x0].0x0 bits 0x0/0x0 rrc: 5 type: IBT gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 6255 timeout: 0 lvb_type: 0 00010000:00010000:1.0:1776178095.830740:0:6255:0:(ldlm_lock.c:1064:ldlm_granted_list_add_lock()) ### About to add lock: ns: mdt-lustre-MDT0000_UUID lock: 00000000daa24072/0xa40083ab9bf52e80 lrc: 3/1,0 mode: PR/PR res: [0x200000007:0x1:0x0].0x0 bits 0x13/0x0 rrc: 5 type: IBT gid 0 flags: 0x50210000000000 nid: local remote: 0x0 expref: -99 pid: 6255 timeout: 0 lvb_type: 0 00010000:00010000:1.0:1776178095.830757:0:6255:0:(ldlm_request.c:528:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: mdt-lustre-MDT0000_UUID lock: 00000000daa24072/0xa40083ab9bf52e80 lrc: 3/1,0 mode: PR/PR res: [0x200000007:0x1:0x0].0x0 bits 0x13/0x0 rrc: 5 type: IBT gid 0 flags: 0x40210000000000 nid: local remote: 0x0 expref: -99 pid: 6255 timeout: 0 lvb_type: 0 00000004:00200000:1.0:1776178095.830848:0:6255:0:(mdt_handler.c:912:mdt_pack_attr2body()) [0x200000007:0x1:0x0]: returning size 4096 00000004:00200000:1.0:1776178095.830853:0:6255:0:(mdt_handler.c:1523:mdt_getattr_internal()) dirent count 2 stripe count 1 MDT count 2 00000004:00010000:1.0:1776178095.830889:0:6255:0:(mdt_handler.c:2353:mdt_getattr_name_lock()) ### Returning lock to client ns: mdt-lustre-MDT0000_UUID lock: 00000000daa24072/0xa40083ab9bf52e80 lrc: 3/1,0 mode: PR/PR res: [0x200000007:0x1:0x0].0x0 bits 0x13/0x0 rrc: 5 type: IBT gid 0 flags: 0x40210000000000 nid: local remote: 0x0 expref: -99 pid: 6255 timeout: 0 lvb_type: 0 00010000:00010000:1.0:1776178095.830926:0:6255:0:(ldlm_lock.c:216:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: mdt-lustre-MDT0000_UUID lock: 00000000499bab84/0xa40083ab9bf52e79 lrc: 0/0,0 mode: --/CR res: [0x200000007:0x1:0x0].0x0 bits 0x2/0x0 rrc: 5 type: IBT gid 0 flags: 0x44000000000000 nid: 192.168.204.3@tcp remote: 0x771f68aa1eb27958 expref: 8 pid: 6255 timeout: 0 lvb_type: 0 00010000:00010000:1.0:1776178095.830951:0:6255:0:(ldlm_lockd.c:1484:ldlm_handle_enqueue0()) ### server-side enqueue handler, sending reply (err=0, rc=0) ns: mdt-lustre-MDT0000_UUID lock: 00000000daa24072/0xa40083ab9bf52e80 lrc: 3/0,0 mode: PR/PR res: [0x200000007:0x1:0x0].0x0 bits 0x13/0x0 rrc: 4 type: IBT gid 0 flags: 0x40200000000000 nid: 192.168.204.3@tcp remote: 0x771f68aa1eb27958 expref: 7 pid: 6255 timeout: 0 lvb_type: 0 00010000:00010000:1.0:1776178095.830962:0:6255:0:(ldlm_inodebits.c:94:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x200000007:0x1:0x0].0x0 (00000000ad0d4094) 00010000:00010000:1.0:1776178095.830966:0:6255:0:(ldlm_inodebits.c:96:ldlm_reprocess_inodebits_queue()) Hint 13 00010000:00010000:1.0:1776178095.830971:0:6255:0:(ldlm_lockd.c:1565:ldlm_handle_enqueue0()) ### server-side enqueue handler END (lock 00000000daa24072, rc 0) 00000100:00100000:1.0:1776178095.833348:0:6255:0:(service.c:2372:ptlrpc_server_handle_request()) Handled RPC req@00000000d94d52a5 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_000:b50963bd-8eba-4432-9a54-8888d00e3fa1+7:16976:x1862456979075648:12345-192.168.204.3@tcp:101:lfs.0 Request processed in 2795us (9686us total) trans 0 rc 0/0 00000100:00100000:1.0:1776178095.833369:0:6255:0:(nrs_fifo.c:241:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.204.3@tcp, seq: 6457 00000100:00100000:1.0:1776178095.838830:0:3347:0:(events.c:361:request_in_callback()) peer: 12345-192.168.204.3@tcp (source: 12345-192.168.204.3@tcp) 00000100:00100000:1.0:1776178095.838965:0:3347:0:(events.c:361:request_in_callback()) peer: 12345-192.168.204.3@tcp (source: 12345-192.168.204.3@tcp) 00000100:00100000:0.0:1776178095.839093:0:6255:0:(service.c:2152:ptlrpc_server_handle_req_in()) got req x1862456979075712 00000100:00100000:0.0:1776178095.839184:0:6255:0:(nrs_fifo.c:179:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.204.3@tcp, seq: 6458 00000100:00100000:0.0:1776178095.839193:0:6255:0:(service.c:2324:ptlrpc_server_handle_request()) Handling RPC req@000000008bb01090 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_000:b50963bd-8eba-4432-9a54-8888d00e3fa1+7:2271:x1862456979075712:12345-192.168.204.3@tcp:41:lfs.0 00000004:00000024:0.0:1776178095.839273:0:6255:0:(mdt_handler.c:548:mdt_statfs()) blocks cached 0 granted 2097152 pending 0 free 1446125568 avail 1316663296 00000100:00100000:0.0:1776178095.839372:0:6255:0:(service.c:2372:ptlrpc_server_handle_request()) Handled RPC req@000000008bb01090 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_000:b50963bd-8eba-4432-9a54-8888d00e3fa1+7:2271:x1862456979075712:12345-192.168.204.3@tcp:41:lfs.0 Request processed in 179us (407us total) trans 0 rc 0/0 00000100:00100000:0.0:1776178095.839380:0:6255:0:(nrs_fifo.c:241:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.204.3@tcp, seq: 6458 00000100:00100000:0.0:1776178095.846879:0:3346:0:(events.c:361:request_in_callback()) peer: 12345-192.168.204.3@tcp (source: 12345-192.168.204.3@tcp) 00010000:00010000:2.0:1776178095.850026:0:6256:0:(ldlm_lock.c:216:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: mdt-lustre-MDT0000_UUID lock: 000000000396420a/0xa40083ab9bf52e87 lrc: 0/0,0 mode: --/CW res: [0x200000007:0x1:0x0].0x0 bits 0x2/0x0 rrc: 4 type: IBT gid 0 flags: 0x44a19400000000 nid: local remote: 0x0 expref: -99 pid: 6256 timeout: 0 lvb_type: 0 00000004:00010000:2.0:1776178095.850045:0:6256:0:(mdt_handler.c:3992:mdt_save_lock()) ### save lock request 00000000cab68204 reply state 000000001c61ee10 transno 12884901898 ns: mdt-lustre-MDT0000_UUID lock: 00000000540a8f8b/0xa40083ab9bf52e8e lrc: 3/0,1 mode: PW/PW res: [0x200000007:0x1:0x0].0xc2d93176 bits 0x2/0x0 rrc: 2 type: IBT gid 0 flags: 0x40210000000000 nid: local remote: 0x0 expref: -99 pid: 6256 timeout: 0 lvb_type: 0 00000100:00100000:2.0:1776178095.850244:0:6256:0:(service.c:2372:ptlrpc_server_handle_request()) Handled RPC req@00000000cab68204 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_001:b50963bd-8eba-4432-9a54-8888d00e3fa1+8:16976:x1862456979075904:12345-192.168.204.3@tcp:36:lfs.0 Request processed in 2830us (3372us total) trans 12884901898 rc 0/0 00000100:00100000:2.0:1776178095.850255:0:6256:0:(nrs_fifo.c:241:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.204.3@tcp, seq: 6459 00010000:00010000:0.0:1776178095.852555:0:3340:0:(ldlm_lock.c:1064:ldlm_granted_list_add_lock()) ### About to add lock: ns: mdt-lustre-MDT0000_UUID lock: 00000000540a8f8b/0xa40083ab9bf52e8e lrc: 3/0,1 mode: COS/COS res: [0x200000007:0x1:0x0].0xc2d93176 bits 0x2/0x0 rrc: 2 type: IBT gid 0 flags: 0x50210000000000 nid: local remote: 0x0 expref: -99 pid: 6256 timeout: 0 lvb_type: 0 00010000:00010000:0.0:1776178095.852586:0:3340:0:(ldlm_inodebits.c:94:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x200000007:0x1:0x0].0xc2d93176 (00000000e8f73a70) 00010000:00010000:0.0:1776178095.852589:0:3340:0:(ldlm_inodebits.c:96:ldlm_reprocess_inodebits_queue()) Hint 2 00010000:00010000:0.0:1776178095.852595:0:3340:0:(ldlm_lock.c:1064:ldlm_granted_list_add_lock()) ### About to add lock: ns: mdt-lustre-MDT0000_UUID lock: 0000000096c1e297/0xa40083ab9bf52e95 lrc: 3/0,1 mode: COS/COS res: [0x2000013a1:0x1:0x0].0x0 bits 0x2/0x0 rrc: 2 type: IBT gid 0 flags: 0x50210000000000 nid: local remote: 0x0 expref: -99 pid: 6256 timeout: 0 lvb_type: 0 00010000:00010000:0.0:1776178095.852601:0:3340:0:(ldlm_inodebits.c:94:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x2000013a1:0x1:0x0].0x0 (00000000315facbe) 00010000:00010000:0.0:1776178095.852603:0:3340:0:(ldlm_inodebits.c:96:ldlm_reprocess_inodebits_queue()) Hint 2 00000100:00100000:1.0:1776178095.878293:0:6256:0:(service.c:2152:ptlrpc_server_handle_req_in()) got req x1862456979076032 00000100:00100000:1.0:1776178095.878376:0:6256:0:(nrs_fifo.c:179:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.204.3@tcp, seq: 6460 00000100:00100000:1.0:1776178095.878385:0:6256:0:(service.c:2324:ptlrpc_server_handle_request()) Handling RPC req@00000000803dbce0 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_001:b50963bd-8eba-4432-9a54-8888d00e3fa1+8:16984:x1862456979076032:12345-192.168.204.3@tcp:101:lfs.0 00010000:00010000:1.0:1776178095.878423:0:6256:0:(ldlm_lockd.c:1248:ldlm_handle_enqueue0()) ### server-side enqueue handler START 00010000:00010000:1.0:1776178095.878448:0:6256:0:(ldlm_lockd.c:1318:ldlm_handle_enqueue0()) ### server-side enqueue handler, new lock created ns: mdt-lustre-MDT0000_UUID lock: 00000000499bab84/0xa40083ab9bf52e9c lrc: 2/0,0 mode: --/CR res: [0x200000007:0x1:0x0].0x0 bits 0x0/0x0 rrc: 4 type: IBT gid 0 flags: 0x40000000000000 nid: local remote: 0x771f68aa1eb2795f expref: -99 pid: 6256 timeout: 0 lvb_type: 0 00010000:00010000:1.0:1776178095.878567:0:6256:0:(ldlm_lock.c:766:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(CR) ns: mdt-lustre-MDT0000_UUID lock: 0000000097975a3b/0xa40083ab9bf52ea3 lrc: 3/1,0 mode: --/CR res: [0x200000007:0x1:0x0].0x0 bits 0x0/0x0 rrc: 5 type: IBT gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 6256 timeout: 0 lvb_type: 0 00010000:00010000:1.0:1776178095.878590:0:6256:0:(ldlm_lock.c:1064:ldlm_granted_list_add_lock()) ### About to add lock: ns: mdt-lustre-MDT0000_UUID lock: 0000000097975a3b/0xa40083ab9bf52ea3 lrc: 3/1,0 mode: CR/CR res: [0x200000007:0x1:0x0].0x0 bits 0x2/0x0 rrc: 5 type: IBT gid 0 flags: 0x50210000000000 nid: local remote: 0x0 expref: -99 pid: 6256 timeout: 0 lvb_type: 0 00010000:00010000:1.0:1776178095.878610:0:6256:0:(ldlm_request.c:528:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: mdt-lustre-MDT0000_UUID lock: 0000000097975a3b/0xa40083ab9bf52ea3 lrc: 3/1,0 mode: CR/CR res: [0x200000007:0x1:0x0].0x0 bits 0x2/0x0 rrc: 5 type: IBT gid 0 flags: 0x40210000000000 nid: local remote: 0x0 expref: -99 pid: 6256 timeout: 0 lvb_type: 0 00010000:00010000:1.0:1776178095.878627:0:6256:0:(ldlm_lock.c:766:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PR) ns: mdt-lustre-MDT0000_UUID lock: 00000000b2e83690/0xa40083ab9bf52eaa lrc: 3/1,0 mode: --/PR res: [0x200000007:0x1:0x0].0xc2d93176 bits 0x0/0x0 rrc: 3 type: IBT gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 6256 timeout: 0 lvb_type: 0 00010000:00010000:1.0:1776178095.878645:0:6256:0:(ldlm_lock.c:1064:ldlm_granted_list_add_lock()) ### About to add lock: ns: mdt-lustre-MDT0000_UUID lock: 00000000b2e83690/0xa40083ab9bf52eaa lrc: 3/1,0 mode: PR/PR res: [0x200000007:0x1:0x0].0xc2d93176 bits 0x2/0x0 rrc: 3 type: IBT gid 0 flags: 0x50210000000000 nid: local remote: 0x0 expref: -99 pid: 6256 timeout: 0 lvb_type: 0 00010000:00010000:1.0:1776178095.878653:0:6256:0:(ldlm_request.c:528:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: mdt-lustre-MDT0000_UUID lock: 00000000b2e83690/0xa40083ab9bf52eaa lrc: 3/1,0 mode: PR/PR res: [0x200000007:0x1:0x0].0xc2d93176 bits 0x2/0x0 rrc: 3 type: IBT gid 0 flags: 0x40210000000000 nid: local remote: 0x0 expref: -99 pid: 6256 timeout: 0 lvb_type: 0 00000004:00200000:1.0:1776178095.878831:0:6256:0:(mdt_handler.c:912:mdt_pack_attr2body()) [0x2000013a1:0x1:0x0]: returning size 4096 00010000:00010000:1.0:1776178095.878970:0:6256:0:(ldlm_lock.c:820:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(CR) ns: mdt-lustre-MDT0000_UUID lock: 0000000097975a3b/0xa40083ab9bf52ea3 lrc: 3/1,0 mode: CR/CR res: [0x200000007:0x1:0x0].0x0 bits 0x2/0x0 rrc: 5 type: IBT gid 0 flags: 0x50210000000000 nid: local remote: 0x0 expref: -99 pid: 6256 timeout: 0 lvb_type: 0 00010000:00010000:1.0:1776178095.878980:0:6256:0:(ldlm_lock.c:879:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: mdt-lustre-MDT0000_UUID lock: 0000000097975a3b/0xa40083ab9bf52ea3 lrc: 2/0,0 mode: CR/CR res: [0x200000007:0x1:0x0].0x0 bits 0x2/0x0 rrc: 5 type: IBT gid 0 flags: 0x50210400000000 nid: local remote: 0x0 expref: -99 pid: 6256 timeout: 0 lvb_type: 0 00010000:00010000:1.0:1776178095.878989:0:6256:0:(ldlm_lockd.c:1879:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: mdt-lustre-MDT0000_UUID lock: 0000000097975a3b/0xa40083ab9bf52ea3 lrc: 3/0,0 mode: CR/CR res: [0x200000007:0x1:0x0].0x0 bits 0x2/0x0 rrc: 5 type: IBT gid 0 flags: 0x40210400000000 nid: local remote: 0x0 expref: -99 pid: 6256 timeout: 0 lvb_type: 0 00010000:00010000:1.0:1776178095.878996:0:6256:0:(ldlm_lockd.c:1893:ldlm_handle_bl_callback()) Lock 0000000097975a3b already unused, calling callback (0000000079e78348) 00010000:00010000:1.0:1776178095.879000:0:6256:0:(ldlm_request.c:364:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: mdt-lustre-MDT0000_UUID lock: 0000000097975a3b/0xa40083ab9bf52ea3 lrc: 3/0,0 mode: CR/CR res: [0x200000007:0x1:0x0].0x0 bits 0x2/0x0 rrc: 5 type: IBT gid 0 flags: 0x40210400000000 nid: local remote: 0x0 expref: -99 pid: 6256 timeout: 0 lvb_type: 0 00010000:00010000:1.0:1776178095.879011:0:6256:0:(ldlm_request.c:1257:ldlm_cli_cancel_local()) ### server-side local cancel ns: mdt-lustre-MDT0000_UUID lock: 0000000097975a3b/0xa40083ab9bf52ea3 lrc: 4/0,0 mode: CR/CR res: [0x200000007:0x1:0x0].0x0 bits 0x2/0x0 rrc: 5 type: IBT gid 0 flags: 0x40218400000000 nid: local remote: 0x0 expref: -99 pid: 6256 timeout: 0 lvb_type: 0 00010000:00010000:1.0:1776178095.879028:0:6256:0:(ldlm_inodebits.c:94:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x200000007:0x1:0x0].0x0 (00000000ad0d4094) 00010000:00010000:1.0:1776178095.879032:0:6256:0:(ldlm_inodebits.c:96:ldlm_reprocess_inodebits_queue()) Hint 2 00010000:00010000:1.0:1776178095.879040:0:6256:0:(ldlm_lockd.c:1903:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: mdt-lustre-MDT0000_UUID lock: 0000000097975a3b/0xa40083ab9bf52ea3 lrc: 2/0,0 mode: --/CR res: [0x200000007:0x1:0x0].0x0 bits 0x2/0x0 rrc: 5 type: IBT gid 0 flags: 0x44a19400000000 nid: local remote: 0x0 expref: -99 pid: 6256 timeout: 0 lvb_type: 0 00010000:00010000:1.0:1776178095.879048:0:6256:0:(ldlm_lock.c:216:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: mdt-lustre-MDT0000_UUID lock: 0000000097975a3b/0xa40083ab9bf52ea3 lrc: 0/0,0 mode: --/CR res: [0x200000007:0x1:0x0].0x0 bits 0x2/0x0 rrc: 5 type: IBT gid 0 flags: 0x44a19400000000 nid: local remote: 0x0 expref: -99 pid: 6256 timeout: 0 lvb_type: 0 00010000:00010000:1.0:1776178095.879104:0:6256:0:(ldlm_lock.c:820:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PR) ns: mdt-lustre-MDT0000_UUID lock: 00000000b2e83690/0xa40083ab9bf52eaa lrc: 3/1,0 mode: PR/PR res: [0x200000007:0x1:0x0].0xc2d93176 bits 0x2/0x0 rrc: 3 type: IBT gid 0 flags: 0x50210000000000 nid: local remote: 0x0 expref: -99 pid: 6256 timeout: 0 lvb_type: 0 00010000:00010000:1.0:1776178095.879112:0:6256:0:(ldlm_lock.c:879:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: mdt-lustre-MDT0000_UUID lock: 00000000b2e83690/0xa40083ab9bf52eaa lrc: 2/0,0 mode: PR/PR res: [0x200000007:0x1:0x0].0xc2d93176 bits 0x2/0x0 rrc: 3 type: IBT gid 0 flags: 0x50210400000000 nid: local remote: 0x0 expref: -99 pid: 6256 timeout: 0 lvb_type: 0 00010000:00010000:1.0:1776178095.879118:0:6256:0:(ldlm_lockd.c:1879:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: mdt-lustre-MDT0000_UUID lock: 00000000b2e83690/0xa40083ab9bf52eaa lrc: 3/0,0 mode: PR/PR res: [0x200000007:0x1:0x0].0xc2d93176 bits 0x2/0x0 rrc: 3 type: IBT gid 0 flags: 0x40210400000000 nid: local remote: 0x0 expref: -99 pid: 6256 timeout: 0 lvb_type: 0 00010000:00010000:1.0:1776178095.879124:0:6256:0:(ldlm_lockd.c:1893:ldlm_handle_bl_callback()) Lock 00000000b2e83690 already unused, calling callback (0000000079e78348) 00010000:00010000:1.0:1776178095.879126:0:6256:0:(ldlm_request.c:364:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: mdt-lustre-MDT0000_UUID lock: 00000000b2e83690/0xa40083ab9bf52eaa lrc: 3/0,0 mode: PR/PR res: [0x200000007:0x1:0x0].0xc2d93176 bits 0x2/0x0 rrc: 3 type: IBT gid 0 flags: 0x40210400000000 nid: local remote: 0x0 expref: -99 pid: 6256 timeout: 0 lvb_type: 0 00010000:00010000:1.0:1776178095.879131:0:6256:0:(ldlm_request.c:1257:ldlm_cli_cancel_local()) ### server-side local cancel ns: mdt-lustre-MDT0000_UUID lock: 00000000b2e83690/0xa40083ab9bf52eaa lrc: 4/0,0 mode: PR/PR res: [0x200000007:0x1:0x0].0xc2d93176 bits 0x2/0x0 rrc: 3 type: IBT gid 0 flags: 0x40218400000000 nid: local remote: 0x0 expref: -99 pid: 6256 timeout: 0 lvb_type: 0 00010000:00010000:1.0:1776178095.879139:0:6256:0:(ldlm_inodebits.c:94:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x200000007:0x1:0x0].0xc2d93176 (00000000e8f73a70) 00010000:00010000:1.0:1776178095.879141:0:6256:0:(ldlm_inodebits.c:96:ldlm_reprocess_inodebits_queue()) Hint 2 00010000:00010000:1.0:1776178095.879144:0:6256:0:(ldlm_lockd.c:1903:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: mdt-lustre-MDT0000_UUID lock: 00000000b2e83690/0xa40083ab9bf52eaa lrc: 2/0,0 mode: --/PR res: [0x200000007:0x1:0x0].0xc2d93176 bits 0x2/0x0 rrc: 3 type: IBT gid 0 flags: 0x44a19400000000 nid: local remote: 0x0 expref: -99 pid: 6256 timeout: 0 lvb_type: 0 00010000:00010000:1.0:1776178095.879150:0:6256:0:(ldlm_lock.c:216:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: mdt-lustre-MDT0000_UUID lock: 00000000b2e83690/0xa40083ab9bf52eaa lrc: 0/0,0 mode: --/PR res: [0x200000007:0x1:0x0].0xc2d93176 bits 0x2/0x0 rrc: 3 type: IBT gid 0 flags: 0x44a19400000000 nid: local remote: 0x0 expref: -99 pid: 6256 timeout: 0 lvb_type: 0 00010000:00010000:1.0:1776178095.879188:0:6256:0:(ldlm_lockd.c:1484:ldlm_handle_enqueue0()) ### server-side enqueue handler, sending reply (err=301, rc=0) ns: mdt-lustre-MDT0000_UUID lock: 00000000499bab84/0xa40083ab9bf52e9c lrc: 1/0,0 mode: --/CR res: [0x200000007:0x1:0x0].0x0 bits 0x1/0x0 rrc: 4 type: IBT gid 0 flags: 0x44000000000000 nid: 192.168.204.3@tcp remote: 0x771f68aa1eb2795f expref: 10 pid: 6256 timeout: 0 lvb_type: 0 00010000:00010000:1.0:1776178095.879197:0:6256:0:(ldlm_lock.c:216:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: mdt-lustre-MDT0000_UUID lock: 00000000499bab84/0xa40083ab9bf52e9c lrc: 0/0,0 mode: --/CR res: [0x200000007:0x1:0x0].0x0 bits 0x1/0x0 rrc: 4 type: IBT gid 0 flags: 0x44000000000000 nid: 192.168.204.3@tcp remote: 0x771f68aa1eb2795f expref: 10 pid: 6256 timeout: 0 lvb_type: 0 00010000:00010000:1.0:1776178095.879213:0:6256:0:(ldlm_lockd.c:1565:ldlm_handle_enqueue0()) ### server-side enqueue handler END (lock 00000000499bab84, rc 0) 00000100:00100000:1.0:1776178095.879308:0:6256:0:(service.c:2372:ptlrpc_server_handle_request()) Handled RPC req@00000000803dbce0 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_001:b50963bd-8eba-4432-9a54-8888d00e3fa1+9:16984:x1862456979076032:12345-192.168.204.3@tcp:101:lfs.0 Request processed in 923us (1135us total) trans 12884901899 rc 301/301 00000100:00100000:1.0:1776178095.879318:0:6256:0:(nrs_fifo.c:241:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.204.3@tcp, seq: 6460 00000100:00100000:2.0:1776178095.905784:0:3347:0:(events.c:361:request_in_callback()) peer: 12345-192.168.204.3@tcp (source: 12345-192.168.204.3@tcp) 00000100:00100000:2.0:1776178095.905873:0:6258:0:(service.c:2152:ptlrpc_server_handle_req_in()) got req x1862456979076288 00000100:00100000:2.0:1776178095.905961:0:6258:0:(nrs_fifo.c:179:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.204.3@tcp, seq: 2120 00000100:00100000:2.0:1776178095.905971:0:6258:0:(service.c:2324:ptlrpc_server_handle_request()) Handling RPC req@00000000a13f5de6 pname:cluuid+ref:pid:xid:nid:opc:job mdt_rdpg00_000:b50963bd-8eba-4432-9a54-8888d00e3fa1+10:16984:x1862456979076288:12345-192.168.204.3@tcp:35:lfs.0 00000100:00100000:2.0:1776178095.906429:0:6258:0:(service.c:2372:ptlrpc_server_handle_request()) Handled RPC req@00000000a13f5de6 pname:cluuid+ref:pid:xid:nid:opc:job mdt_rdpg00_000:b50963bd-8eba-4432-9a54-8888d00e3fa1+11:16984:x1862456979076288:12345-192.168.204.3@tcp:35:lfs.0 Request processed in 456us (670us total) trans 12884901901 rc 0/0 00000100:00100000:2.0:1776178095.906440:0:6258:0:(nrs_fifo.c:241:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.204.3@tcp, seq: 2120 00000100:00100000:2.0:1776178095.926108:0:3346:0:(events.c:361:request_in_callback()) peer: 12345-192.168.204.3@tcp (source: 12345-192.168.204.3@tcp) 00000100:00100000:2.0:1776178095.931983:0:6258:0:(service.c:2152:ptlrpc_server_handle_req_in()) got req x1862456979076480 00000100:00100000:2.0:1776178095.932096:0:6258:0:(nrs_fifo.c:179:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.204.3@tcp, seq: 2122 00000100:00100000:2.0:1776178095.932106:0:6258:0:(service.c:2324:ptlrpc_server_handle_request()) Handling RPC req@0000000008542173 pname:cluuid+ref:pid:xid:nid:opc:job mdt_rdpg00_000:b50963bd-8eba-4432-9a54-8888d00e3fa1+11:16984:x1862456979076480:12345-192.168.204.3@tcp:35:lfs.0 00000100:00100000:2.0:1776178095.932767:0:6258:0:(service.c:2372:ptlrpc_server_handle_request()) Handled RPC req@0000000008542173 pname:cluuid+ref:pid:xid:nid:opc:job mdt_rdpg00_000:b50963bd-8eba-4432-9a54-8888d00e3fa1+12:16984:x1862456979076480:12345-192.168.204.3@tcp:35:lfs.0 Request processed in 650us (899us total) trans 12884901902 rc 0/0 00000100:00100000:2.0:1776178095.932778:0:6258:0:(nrs_fifo.c:241:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.204.3@tcp, seq: 2122 00000100:00100000:2.0:1776178096.005042:0:3347:0:(events.c:361:request_in_callback()) peer: 12345-192.168.204.3@tcp (source: 12345-192.168.204.3@tcp) 00000100:00100000:1.0:1776178096.017707:0:6256:0:(service.c:2152:ptlrpc_server_handle_req_in()) got req x1862456979076672 00000100:00100000:1.0:1776178096.017812:0:6256:0:(nrs_fifo.c:179:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.204.3@tcp, seq: 6463 00000100:00100000:1.0:1776178096.017823:0:6256:0:(service.c:2324:ptlrpc_server_handle_request()) Handling RPC req@00000000ca033a09 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_001:b50963bd-8eba-4432-9a54-8888d00e3fa1+13:16988:x1862456979076672:12345-192.168.204.3@tcp:101:touch.0 00010000:00010000:1.0:1776178096.017866:0:6256:0:(ldlm_lockd.c:1248:ldlm_handle_enqueue0()) ### server-side enqueue handler START 00010000:00010000:1.0:1776178096.017905:0:6256:0:(ldlm_lockd.c:1318:ldlm_handle_enqueue0()) ### server-side enqueue handler, new lock created ns: mdt-lustre-MDT0000_UUID lock: 00000000499bab84/0xa40083ab9bf52ef0 lrc: 2/0,0 mode: --/CW res: [0x2000013a1:0x1:0x0].0x0 bits 0x0/0x0 rrc: 4 type: IBT gid 0 flags: 0x40000000000000 nid: local remote: 0x771f68aa1eb27974 expref: -99 pid: 6256 timeout: 0 lvb_type: 0 00010000:00010000:0.0:1776178096.018532:0:6256:0:(ldlm_lock.c:766:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(CW) ns: mdt-lustre-MDT0000_UUID lock: 00000000319537bc/0xa40083ab9bf52ef7 lrc: 3/0,1 mode: --/CW res: [0x2000013a1:0x1:0x0].0x0 bits 0x0/0x0 rrc: 5 type: IBT gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 6256 timeout: 0 lvb_type: 0 00010000:00010000:0.0:1776178096.018564:0:6256:0:(ldlm_lock.c:1064:ldlm_granted_list_add_lock()) ### About to add lock: ns: mdt-lustre-MDT0000_UUID lock: 00000000319537bc/0xa40083ab9bf52ef7 lrc: 3/0,1 mode: CW/CW res: [0x2000013a1:0x1:0x0].0x0 bits 0x2/0x0 rrc: 5 type: IBT gid 0 flags: 0x50210000000000 nid: local remote: 0x0 expref: -99 pid: 6256 timeout: 0 lvb_type: 0 00010000:00010000:0.0:1776178096.018584:0:6256:0:(ldlm_request.c:528:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: mdt-lustre-MDT0000_UUID lock: 00000000319537bc/0xa40083ab9bf52ef7 lrc: 3/0,1 mode: CW/CW res: [0x2000013a1:0x1:0x0].0x0 bits 0x2/0x0 rrc: 5 type: IBT gid 0 flags: 0x40210000000000 nid: local remote: 0x0 expref: -99 pid: 6256 timeout: 0 lvb_type: 0 00010000:00010000:0.0:1776178096.018606:0:6256:0:(ldlm_lock.c:766:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: mdt-lustre-MDT0000_UUID lock: 00000000e1acd7eb/0xa40083ab9bf52efe lrc: 3/0,1 mode: --/PW res: [0x2000013a1:0x1:0x0].0xcda302ea bits 0x0/0x0 rrc: 2 type: IBT gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 6256 timeout: 0 lvb_type: 0 00010000:00010000:0.0:1776178096.018615:0:6256:0:(ldlm_lock.c:1064:ldlm_granted_list_add_lock()) ### About to add lock: ns: mdt-lustre-MDT0000_UUID lock: 00000000e1acd7eb/0xa40083ab9bf52efe lrc: 3/0,1 mode: PW/PW res: [0x2000013a1:0x1:0x0].0xcda302ea bits 0x2/0x0 rrc: 2 type: IBT gid 0 flags: 0x50210000000000 nid: local remote: 0x0 expref: -99 pid: 6256 timeout: 0 lvb_type: 0 00010000:00010000:0.0:1776178096.018622:0:6256:0:(ldlm_request.c:528:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: mdt-lustre-MDT0000_UUID lock: 00000000e1acd7eb/0xa40083ab9bf52efe lrc: 3/0,1 mode: PW/PW res: [0x2000013a1:0x1:0x0].0xcda302ea bits 0x2/0x0 rrc: 2 type: IBT gid 0 flags: 0x40210000000000 nid: local remote: 0x0 expref: -99 pid: 6256 timeout: 0 lvb_type: 0 00000004:00080000:0.0:1776178096.019637:0:6256:0:(osp_internal.h:641:osp_update_last_fid()) Gap in objids: start=[0x100000000:0x2d:0x0], count =21 00000004:00080000:0.0:1776178096.019655:0:6256:0:(osp_object.c:1628:osp_create()) Writing gap [0x100000000:0x2d:0x0]+21 in llog 00000004:00080000:0.0:1776178096.019678:0:6256:0:(osp_object.c:1645:osp_create()) lustre-OST0000-osc-MDT0000: Wrote last used FID: [0x100000000:0x42:0x0], index 0: 0 00010000:00010000:0.0:1776178096.020627:0:6256:0:(ldlm_lock.c:766:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(CR) ns: mdt-lustre-MDT0000_UUID lock: 00000000fa092262/0xa40083ab9bf52f05 lrc: 3/1,0 mode: --/CR res: [0x2000013a1:0x2:0x0].0x0 bits 0x0/0x0 rrc: 2 type: IBT gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 6256 timeout: 0 lvb_type: 0 00010000:00010000:0.0:1776178096.020654:0:6256:0:(ldlm_lock.c:1064:ldlm_granted_list_add_lock()) ### About to add lock: ns: mdt-lustre-MDT0000_UUID lock: 00000000fa092262/0xa40083ab9bf52f05 lrc: 3/1,0 mode: CR/CR res: [0x2000013a1:0x2:0x0].0x0 bits 0x9/0x0 rrc: 2 type: IBT gid 0 flags: 0x50210000000000 nid: local remote: 0x0 expref: -99 pid: 6256 timeout: 0 lvb_type: 0 00010000:00010000:0.0:1776178096.020670:0:6256:0:(ldlm_request.c:528:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: mdt-lustre-MDT0000_UUID lock: 00000000fa092262/0xa40083ab9bf52f05 lrc: 3/1,0 mode: CR/CR res: [0x2000013a1:0x2:0x0].0x0 bits 0x9/0x0 rrc: 2 type: IBT gid 0 flags: 0x40210000000000 nid: local remote: 0x0 expref: -99 pid: 6256 timeout: 0 lvb_type: 0 00010000:00010000:0.0:1776178096.020828:0:6256:0:(ldlm_lock.c:820:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(CW) ns: mdt-lustre-MDT0000_UUID lock: 00000000319537bc/0xa40083ab9bf52ef7 lrc: 3/0,1 mode: CW/CW res: [0x2000013a1:0x1:0x0].0x0 bits 0x2/0x0 rrc: 5 type: IBT gid 0 flags: 0x50210000000000 nid: local remote: 0x0 expref: -99 pid: 6256 timeout: 0 lvb_type: 0 00010000:00010000:0.0:1776178096.020839:0:6256:0:(ldlm_lock.c:879:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: mdt-lustre-MDT0000_UUID lock: 00000000319537bc/0xa40083ab9bf52ef7 lrc: 2/0,0 mode: CW/CW res: [0x2000013a1:0x1:0x0].0x0 bits 0x2/0x0 rrc: 5 type: IBT gid 0 flags: 0x50210400000000 nid: local remote: 0x0 expref: -99 pid: 6256 timeout: 0 lvb_type: 0 00010000:00010000:0.0:1776178096.020847:0:6256:0:(ldlm_lockd.c:1879:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: mdt-lustre-MDT0000_UUID lock: 00000000319537bc/0xa40083ab9bf52ef7 lrc: 3/0,0 mode: CW/CW res: [0x2000013a1:0x1:0x0].0x0 bits 0x2/0x0 rrc: 5 type: IBT gid 0 flags: 0x40210400000000 nid: local remote: 0x0 expref: -99 pid: 6256 timeout: 0 lvb_type: 0 00010000:00010000:0.0:1776178096.020854:0:6256:0:(ldlm_lockd.c:1893:ldlm_handle_bl_callback()) Lock 00000000319537bc already unused, calling callback (0000000079e78348) 00010000:00010000:0.0:1776178096.020859:0:6256:0:(ldlm_request.c:364:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: mdt-lustre-MDT0000_UUID lock: 00000000319537bc/0xa40083ab9bf52ef7 lrc: 3/0,0 mode: CW/CW res: [0x2000013a1:0x1:0x0].0x0 bits 0x2/0x0 rrc: 5 type: IBT gid 0 flags: 0x40210400000000 nid: local remote: 0x0 expref: -99 pid: 6256 timeout: 0 lvb_type: 0 00010000:00010000:0.0:1776178096.020867:0:6256:0:(ldlm_request.c:1257:ldlm_cli_cancel_local()) ### server-side local cancel ns: mdt-lustre-MDT0000_UUID lock: 00000000319537bc/0xa40083ab9bf52ef7 lrc: 4/0,0 mode: CW/CW res: [0x2000013a1:0x1:0x0].0x0 bits 0x2/0x0 rrc: 5 type: IBT gid 0 flags: 0x40218400000000 nid: local remote: 0x0 expref: -99 pid: 6256 timeout: 0 lvb_type: 0 00010000:00010000:0.0:1776178096.020887:0:6256:0:(ldlm_inodebits.c:94:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x2000013a1:0x1:0x0].0x0 (00000000315facbe) 00010000:00010000:0.0:1776178096.020891:0:6256:0:(ldlm_inodebits.c:96:ldlm_reprocess_inodebits_queue()) Hint 2 00010000:00010000:0.0:1776178096.020898:0:6256:0:(ldlm_lockd.c:1903:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: mdt-lustre-MDT0000_UUID lock: 00000000319537bc/0xa40083ab9bf52ef7 lrc: 2/0,0 mode: --/CW res: [0x2000013a1:0x1:0x0].0x0 bits 0x2/0x0 rrc: 5 type: IBT gid 0 flags: 0x44a19400000000 nid: local remote: 0x0 expref: -99 pid: 6256 timeout: 0 lvb_type: 0 00010000:00010000:0.0:1776178096.020905:0:6256:0:(ldlm_lock.c:216:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: mdt-lustre-MDT0000_UUID lock: 00000000319537bc/0xa40083ab9bf52ef7 lrc: 0/0,0 mode: --/CW res: [0x2000013a1:0x1:0x0].0x0 bits 0x2/0x0 rrc: 5 type: IBT gid 0 flags: 0x44a19400000000 nid: local remote: 0x0 expref: -99 pid: 6256 timeout: 0 lvb_type: 0 00000004:00010000:0.0:1776178096.021360:0:6256:0:(mdt_handler.c:3992:mdt_save_lock()) ### save lock request 00000000ca033a09 reply state 000000004b721064 transno 12884901903 ns: mdt-lustre-MDT0000_UUID lock: 00000000e1acd7eb/0xa40083ab9bf52efe lrc: 3/0,1 mode: PW/PW res: [0x2000013a1:0x1:0x0].0xcda302ea bits 0x2/0x0 rrc: 2 type: IBT gid 0 flags: 0x40210000000000 nid: local remote: 0x0 expref: -99 pid: 6256 timeout: 0 lvb_type: 0 00010000:00010000:0.0:1776178096.021429:0:6256:0:(ldlm_lock.c:216:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: mdt-lustre-MDT0000_UUID lock: 00000000499bab84/0xa40083ab9bf52ef0 lrc: 0/0,0 mode: --/CW res: [0x2000013a1:0x1:0x0].0x0 bits 0x1/0x0 rrc: 4 type: IBT gid 0 flags: 0x44000000000000 nid: 192.168.204.3@tcp remote: 0x771f68aa1eb27974 expref: 16 pid: 6256 timeout: 0 lvb_type: 0 00010000:00010000:0.0:1776178096.021450:0:6256:0:(ldlm_lockd.c:1484:ldlm_handle_enqueue0()) ### server-side enqueue handler, sending reply (err=0, rc=0) ns: mdt-lustre-MDT0000_UUID lock: 00000000fa092262/0xa40083ab9bf52f05 lrc: 3/0,0 mode: CR/CR res: [0x2000013a1:0x2:0x0].0x0 bits 0x9/0x0 rrc: 2 type: IBT gid 0 flags: 0x40200000000000 nid: 192.168.204.3@tcp remote: 0x771f68aa1eb27974 expref: 15 pid: 6256 timeout: 0 lvb_type: 0 00010000:00010000:0.0:1776178096.021459:0:6256:0:(ldlm_inodebits.c:94:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x2000013a1:0x2:0x0].0x0 (000000005ad330c7) 00010000:00010000:0.0:1776178096.021461:0:6256:0:(ldlm_inodebits.c:96:ldlm_reprocess_inodebits_queue()) Hint 9 00010000:00010000:0.0:1776178096.021463:0:6256:0:(ldlm_lockd.c:1565:ldlm_handle_enqueue0()) ### server-side enqueue handler END (lock 00000000fa092262, rc 0) 00000100:00100000:0.0:1776178096.021687:0:6256:0:(service.c:2372:ptlrpc_server_handle_request()) Handled RPC req@00000000ca033a09 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_001:b50963bd-8eba-4432-9a54-8888d00e3fa1+16:16988:x1862456979076672:12345-192.168.204.3@tcp:101:touch.0 Request processed in 3861us (5705us total) trans 12884901903 rc 0/0 00000100:00100000:0.0:1776178096.021698:0:6256:0:(nrs_fifo.c:241:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.204.3@tcp, seq: 6463 00010000:00010000:2.0:1776178096.031483:0:3339:0:(ldlm_lock.c:1064:ldlm_granted_list_add_lock()) ### About to add lock: ns: mdt-lustre-MDT0000_UUID lock: 00000000e1acd7eb/0xa40083ab9bf52efe lrc: 3/0,1 mode: COS/COS res: [0x2000013a1:0x1:0x0].0xcda302ea bits 0x2/0x0 rrc: 2 type: IBT gid 0 flags: 0x50210000000000 nid: local remote: 0x0 expref: -99 pid: 6256 timeout: 0 lvb_type: 0 00010000:00010000:2.0:1776178096.031500:0:3339:0:(ldlm_inodebits.c:94:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x2000013a1:0x1:0x0].0xcda302ea (0000000008b13340) 00010000:00010000:2.0:1776178096.031507:0:3339:0:(ldlm_inodebits.c:96:ldlm_reprocess_inodebits_queue()) Hint 2 00010000:00010000:0.0:1776178096.035703:0:3340:0:(ldlm_lock.c:1064:ldlm_granted_list_add_lock()) ### About to add lock: ns: mdt-lustre-MDT0000_UUID lock: 00000000591efbb9/0xa40083ab9bf52f13 lrc: 3/0,1 mode: COS/COS res: [0x2000013a1:0x2:0x0].0x0 bits 0x2/0x0 rrc: 3 type: IBT gid 0 flags: 0x50210000000000 nid: local remote: 0x0 expref: -99 pid: 6255 timeout: 0 lvb_type: 0 00010000:00010000:0.0:1776178096.035750:0:3340:0:(ldlm_inodebits.c:94:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x2000013a1:0x2:0x0].0x0 (000000005ad330c7) 00010000:00010000:0.0:1776178096.035754:0:3340:0:(ldlm_inodebits.c:96:ldlm_reprocess_inodebits_queue()) Hint 2 00000100:00100000:0.0:1776178096.036807:0:14440:0:(service.c:2152:ptlrpc_server_handle_req_in()) got req x1862456979076864 00000100:00100000:0.0:1776178096.037119:0:14440:0:(nrs_fifo.c:179:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.204.3@tcp, seq: 5229 00000100:00100000:0.0:1776178096.037130:0:14440:0:(service.c:2324:ptlrpc_server_handle_request()) Handling RPC req@000000004dd93690 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_004:b50963bd-8eba-4432-9a54-8888d00e3fa1+5:2270:x1862456979076864:12345-192.168.204.3@tcp:2:touch.0 00000100:00100000:0.0:1776178096.037684:0:14440:0:(service.c:2372:ptlrpc_server_handle_request()) Handled RPC req@000000004dd93690 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_004:b50963bd-8eba-4432-9a54-8888d00e3fa1+6:2270:x1862456979076864:12345-192.168.204.3@tcp:2:touch.0 Request processed in 542us (1014us total) trans 4294969341 rc 0/0 00000100:00100000:0.0:1776178096.037772:0:14440:0:(nrs_fifo.c:241:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.204.3@tcp, seq: 5229 00000100:00100000:2.0:1776178096.041678:0:3347:0:(events.c:361:request_in_callback()) peer: 12345-192.168.204.3@tcp (source: 12345-192.168.204.3@tcp) 00000100:00100000:1.0:1776178096.041852:0:6258:0:(service.c:2152:ptlrpc_server_handle_req_in()) got req x1862456979076992 00000100:00100000:1.0:1776178096.041933:0:6258:0:(nrs_fifo.c:179:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.204.3@tcp, seq: 2123 00000100:00100000:1.0:1776178096.041946:0:6258:0:(service.c:2324:ptlrpc_server_handle_request()) Handling RPC req@00000000aba13576 pname:cluuid+ref:pid:xid:nid:opc:job mdt_rdpg00_000:b50963bd-8eba-4432-9a54-8888d00e3fa1+18:16988:x1862456979076992:12345-192.168.204.3@tcp:35:touch.0 00000100:00100000:1.0:1776178096.042919:0:6258:0:(service.c:2372:ptlrpc_server_handle_request()) Handled RPC req@00000000aba13576 pname:cluuid+ref:pid:xid:nid:opc:job mdt_rdpg00_000:b50963bd-8eba-4432-9a54-8888d00e3fa1+19:16988:x1862456979076992:12345-192.168.204.3@tcp:35:touch.0 Request processed in 969us (1254us total) trans 12884901905 rc 0/0 00000100:00100000:1.0:1776178096.042934:0:6258:0:(nrs_fifo.c:241:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.204.3@tcp, seq: 2123 00000100:00100000:0.0:1776178096.059355:0:6256:0:(service.c:2152:ptlrpc_server_handle_req_in()) got req x1862456979077120 00000100:00100000:0.0:1776178096.059448:0:6256:0:(nrs_fifo.c:179:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.204.3@tcp, seq: 6465 00000100:00100000:0.0:1776178096.059458:0:6256:0:(service.c:2324:ptlrpc_server_handle_request()) Handling RPC req@0000000017caeba0 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_001:b50963bd-8eba-4432-9a54-8888d00e3fa1+19:16989:x1862456979077120:12345-192.168.204.3@tcp:101:touch.0 00010000:00010000:0.0:1776178096.059505:0:6256:0:(ldlm_lockd.c:1248:ldlm_handle_enqueue0()) ### server-side enqueue handler START 00010000:00010000:0.0:1776178096.059543:0:6256:0:(ldlm_lockd.c:1318:ldlm_handle_enqueue0()) ### server-side enqueue handler, new lock created ns: mdt-lustre-MDT0000_UUID lock: 00000000319537bc/0xa40083ab9bf52f1a lrc: 2/0,0 mode: --/CW res: [0x2000013a1:0x1:0x0].0x0 bits 0x0/0x0 rrc: 4 type: IBT gid 0 flags: 0x40000000000000 nid: local remote: 0x771f68aa1eb2797b expref: -99 pid: 6256 timeout: 0 lvb_type: 0 00010000:00010000:0.0:1776178096.059745:0:6256:0:(ldlm_lock.c:766:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(CW) ns: mdt-lustre-MDT0000_UUID lock: 000000004bd1d7cb/0xa40083ab9bf52f21 lrc: 3/0,1 mode: --/CW res: [0x2000013a1:0x1:0x0].0x0 bits 0x0/0x0 rrc: 5 type: IBT gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 6256 timeout: 0 lvb_type: 0 00010000:00010000:0.0:1776178096.059776:0:6256:0:(ldlm_lock.c:1064:ldlm_granted_list_add_lock()) ### About to add lock: ns: mdt-lustre-MDT0000_UUID lock: 000000004bd1d7cb/0xa40083ab9bf52f21 lrc: 3/0,1 mode: CW/CW res: [0x2000013a1:0x1:0x0].0x0 bits 0x2/0x0 rrc: 5 type: IBT gid 0 flags: 0x50210000000000 nid: local remote: 0x0 expref: -99 pid: 6256 timeout: 0 lvb_type: 0 00010000:00010000:0.0:1776178096.059792:0:6256:0:(ldlm_request.c:528:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: mdt-lustre-MDT0000_UUID lock: 000000004bd1d7cb/0xa40083ab9bf52f21 lrc: 3/0,1 mode: CW/CW res: [0x2000013a1:0x1:0x0].0x0 bits 0x2/0x0 rrc: 5 type: IBT gid 0 flags: 0x40210000000000 nid: local remote: 0x0 expref: -99 pid: 6256 timeout: 0 lvb_type: 0 00010000:00010000:0.0:1776178096.059809:0:6256:0:(ldlm_lock.c:766:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: mdt-lustre-MDT0000_UUID lock: 00000000b82e73fc/0xa40083ab9bf52f28 lrc: 3/0,1 mode: --/PW res: [0x2000013a1:0x1:0x0].0x15fcbbea bits 0x0/0x0 rrc: 2 type: IBT gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 6256 timeout: 0 lvb_type: 0 00010000:00010000:0.0:1776178096.059816:0:6256:0:(ldlm_lock.c:1064:ldlm_granted_list_add_lock()) ### About to add lock: ns: mdt-lustre-MDT0000_UUID lock: 00000000b82e73fc/0xa40083ab9bf52f28 lrc: 3/0,1 mode: PW/PW res: [0x2000013a1:0x1:0x0].0x15fcbbea bits 0x2/0x0 rrc: 2 type: IBT gid 0 flags: 0x50210000000000 nid: local remote: 0x0 expref: -99 pid: 6256 timeout: 0 lvb_type: 0 00010000:00010000:0.0:1776178096.059823:0:6256:0:(ldlm_request.c:528:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: mdt-lustre-MDT0000_UUID lock: 00000000b82e73fc/0xa40083ab9bf52f28 lrc: 3/0,1 mode: PW/PW res: [0x2000013a1:0x1:0x0].0x15fcbbea bits 0x2/0x0 rrc: 2 type: IBT gid 0 flags: 0x40210000000000 nid: local remote: 0x0 expref: -99 pid: 6256 timeout: 0 lvb_type: 0 00000004:00080000:0.0:1776178096.060376:0:6256:0:(osp_internal.h:641:osp_update_last_fid()) Gap in objids: start=[0x100010000:0x2c:0x0], count =22 00000004:00080000:0.0:1776178096.060389:0:6256:0:(osp_object.c:1628:osp_create()) Writing gap [0x100010000:0x2c:0x0]+22 in llog 00000004:00080000:0.0:1776178096.060400:0:6256:0:(osp_object.c:1645:osp_create()) lustre-OST0001-osc-MDT0000: Wrote last used FID: [0x100010000:0x42:0x0], index 1: 0 00010000:00010000:0.0:1776178096.060632:0:6256:0:(ldlm_lock.c:766:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(CR) ns: mdt-lustre-MDT0000_UUID lock: 00000000157c1e3e/0xa40083ab9bf52f2f lrc: 3/1,0 mode: --/CR res: [0x2000013a1:0x3:0x0].0x0 bits 0x0/0x0 rrc: 2 type: IBT gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 6256 timeout: 0 lvb_type: 0 00010000:00010000:0.0:1776178096.060650:0:6256:0:(ldlm_lock.c:1064:ldlm_granted_list_add_lock()) ### About to add lock: ns: mdt-lustre-MDT0000_UUID lock: 00000000157c1e3e/0xa40083ab9bf52f2f lrc: 3/1,0 mode: CR/CR res: [0x2000013a1:0x3:0x0].0x0 bits 0x9/0x0 rrc: 2 type: IBT gid 0 flags: 0x50210000000000 nid: local remote: 0x0 expref: -99 pid: 6256 timeout: 0 lvb_type: 0 00010000:00010000:0.0:1776178096.060661:0:6256:0:(ldlm_request.c:528:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: mdt-lustre-MDT0000_UUID lock: 00000000157c1e3e/0xa40083ab9bf52f2f lrc: 3/1,0 mode: CR/CR res: [0x2000013a1:0x3:0x0].0x0 bits 0x9/0x0 rrc: 2 type: IBT gid 0 flags: 0x40210000000000 nid: local remote: 0x0 expref: -99 pid: 6256 timeout: 0 lvb_type: 0 00010000:00010000:0.0:1776178096.060715:0:6256:0:(ldlm_lock.c:820:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(CW) ns: mdt-lustre-MDT0000_UUID lock: 000000004bd1d7cb/0xa40083ab9bf52f21 lrc: 3/0,1 mode: CW/CW res: [0x2000013a1:0x1:0x0].0x0 bits 0x2/0x0 rrc: 5 type: IBT gid 0 flags: 0x50210000000000 nid: local remote: 0x0 expref: -99 pid: 6256 timeout: 0 lvb_type: 0 00010000:00010000:0.0:1776178096.060724:0:6256:0:(ldlm_lock.c:879:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: mdt-lustre-MDT0000_UUID lock: 000000004bd1d7cb/0xa40083ab9bf52f21 lrc: 2/0,0 mode: CW/CW res: [0x2000013a1:0x1:0x0].0x0 bits 0x2/0x0 rrc: 5 type: IBT gid 0 flags: 0x50210400000000 nid: local remote: 0x0 expref: -99 pid: 6256 timeout: 0 lvb_type: 0 00010000:00010000:0.0:1776178096.060731:0:6256:0:(ldlm_lockd.c:1879:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: mdt-lustre-MDT0000_UUID lock: 000000004bd1d7cb/0xa40083ab9bf52f21 lrc: 3/0,0 mode: CW/CW res: [0x2000013a1:0x1:0x0].0x0 bits 0x2/0x0 rrc: 5 type: IBT gid 0 flags: 0x40210400000000 nid: local remote: 0x0 expref: -99 pid: 6256 timeout: 0 lvb_type: 0 00010000:00010000:0.0:1776178096.060739:0:6256:0:(ldlm_lockd.c:1893:ldlm_handle_bl_callback()) Lock 000000004bd1d7cb already unused, calling callback (0000000079e78348) 00010000:00010000:0.0:1776178096.060743:0:6256:0:(ldlm_request.c:364:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: mdt-lustre-MDT0000_UUID lock: 000000004bd1d7cb/0xa40083ab9bf52f21 lrc: 3/0,0 mode: CW/CW res: [0x2000013a1:0x1:0x0].0x0 bits 0x2/0x0 rrc: 5 type: IBT gid 0 flags: 0x40210400000000 nid: local remote: 0x0 expref: -99 pid: 6256 timeout: 0 lvb_type: 0 00010000:00010000:0.0:1776178096.060751:0:6256:0:(ldlm_request.c:1257:ldlm_cli_cancel_local()) ### server-side local cancel ns: mdt-lustre-MDT0000_UUID lock: 000000004bd1d7cb/0xa40083ab9bf52f21 lrc: 4/0,0 mode: CW/CW res: [0x2000013a1:0x1:0x0].0x0 bits 0x2/0x0 rrc: 5 type: IBT gid 0 flags: 0x40218400000000 nid: local remote: 0x0 expref: -99 pid: 6256 timeout: 0 lvb_type: 0 00010000:00010000:0.0:1776178096.060777:0:6256:0:(ldlm_inodebits.c:94:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x2000013a1:0x1:0x0].0x0 (00000000315facbe) 00010000:00010000:0.0:1776178096.060781:0:6256:0:(ldlm_inodebits.c:96:ldlm_reprocess_inodebits_queue()) Hint 2 00010000:00010000:0.0:1776178096.060786:0:6256:0:(ldlm_lockd.c:1903:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: mdt-lustre-MDT0000_UUID lock: 000000004bd1d7cb/0xa40083ab9bf52f21 lrc: 2/0,0 mode: --/CW res: [0x2000013a1:0x1:0x0].0x0 bits 0x2/0x0 rrc: 5 type: IBT gid 0 flags: 0x44a19400000000 nid: local remote: 0x0 expref: -99 pid: 6256 timeout: 0 lvb_type: 0 00010000:00010000:0.0:1776178096.060792:0:6256:0:(ldlm_lock.c:216:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: mdt-lustre-MDT0000_UUID lock: 000000004bd1d7cb/0xa40083ab9bf52f21 lrc: 0/0,0 mode: --/CW res: [0x2000013a1:0x1:0x0].0x0 bits 0x2/0x0 rrc: 5 type: IBT gid 0 flags: 0x44a19400000000 nid: local remote: 0x0 expref: -99 pid: 6256 timeout: 0 lvb_type: 0 00000004:00010000:0.0:1776178096.060815:0:6256:0:(mdt_handler.c:3992:mdt_save_lock()) ### save lock request 0000000017caeba0 reply state 00000000a07778b3 transno 12884901906 ns: mdt-lustre-MDT0000_UUID lock: 00000000b82e73fc/0xa40083ab9bf52f28 lrc: 3/0,1 mode: PW/PW res: [0x2000013a1:0x1:0x0].0x15fcbbea bits 0x2/0x0 rrc: 2 type: IBT gid 0 flags: 0x40210000000000 nid: local remote: 0x0 expref: -99 pid: 6256 timeout: 0 lvb_type: 0 00010000:00010000:0.0:1776178096.060871:0:6256:0:(ldlm_lock.c:216:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: mdt-lustre-MDT0000_UUID lock: 00000000319537bc/0xa40083ab9bf52f1a lrc: 0/0,0 mode: --/CW res: [0x2000013a1:0x1:0x0].0x0 bits 0x1/0x0 rrc: 4 type: IBT gid 0 flags: 0x44000000000000 nid: 192.168.204.3@tcp remote: 0x771f68aa1eb2797b expref: 22 pid: 6256 timeout: 0 lvb_type: 0 00010000:00010000:0.0:1776178096.060885:0:6256:0:(ldlm_lockd.c:1484:ldlm_handle_enqueue0()) ### server-side enqueue handler, sending reply (err=0, rc=0) ns: mdt-lustre-MDT0000_UUID lock: 00000000157c1e3e/0xa40083ab9bf52f2f lrc: 3/0,0 mode: CR/CR res: [0x2000013a1:0x3:0x0].0x0 bits 0x9/0x0 rrc: 2 type: IBT gid 0 flags: 0x40200000000000 nid: 192.168.204.3@tcp remote: 0x771f68aa1eb2797b expref: 21 pid: 6256 timeout: 0 lvb_type: 0 00010000:00010000:0.0:1776178096.060893:0:6256:0:(ldlm_inodebits.c:94:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x2000013a1:0x3:0x0].0x0 (00000000d0a7c19f) 00010000:00010000:0.0:1776178096.060895:0:6256:0:(ldlm_inodebits.c:96:ldlm_reprocess_inodebits_queue()) Hint 9 00010000:00010000:0.0:1776178096.060897:0:6256:0:(ldlm_lockd.c:1565:ldlm_handle_enqueue0()) ### server-side enqueue handler END (lock 00000000157c1e3e, rc 0) 00000100:00100000:0.0:1776178096.061034:0:6256:0:(service.c:2372:ptlrpc_server_handle_request()) Handled RPC req@0000000017caeba0 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_001:b50963bd-8eba-4432-9a54-8888d00e3fa1+22:16989:x1862456979077120:12345-192.168.204.3@tcp:101:touch.0 Request processed in 1576us (1844us total) trans 12884901906 rc 0/0 00000100:00100000:0.0:1776178096.061046:0:6256:0:(nrs_fifo.c:241:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.204.3@tcp, seq: 6465 00000100:00100000:0.0:1776178096.065420:0:6255:0:(service.c:2152:ptlrpc_server_handle_req_in()) got req x1862456979077248 00000100:00100000:0.0:1776178096.065507:0:6255:0:(nrs_fifo.c:179:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.204.3@tcp, seq: 6466 00000100:00100000:0.0:1776178096.065517:0:6255:0:(service.c:2324:ptlrpc_server_handle_request()) Handling RPC req@00000000d6229cf9 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_000:b50963bd-8eba-4432-9a54-8888d00e3fa1+22:16989:x1862456979077248:12345-192.168.204.3@tcp:36:touch.0 00010000:00010000:0.0:1776178096.065683:0:6255:0:(ldlm_lock.c:766:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: mdt-lustre-MDT0000_UUID lock: 000000000584b645/0xa40083ab9bf52f3d lrc: 3/0,1 mode: --/PW res: [0x2000013a1:0x3:0x0].0x0 bits 0x0/0x0 rrc: 3 type: IBT gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 6255 timeout: 0 lvb_type: 0 00010000:00010000:0.0:1776178096.065705:0:6255:0:(ldlm_lock.c:1064:ldlm_granted_list_add_lock()) ### About to add lock: ns: mdt-lustre-MDT0000_UUID lock: 000000000584b645/0xa40083ab9bf52f3d lrc: 3/0,1 mode: PW/PW res: [0x2000013a1:0x3:0x0].0x0 bits 0x2/0x0 rrc: 3 type: IBT gid 0 flags: 0x50210000000000 nid: local remote: 0x0 expref: -99 pid: 6255 timeout: 0 lvb_type: 0 00010000:00010000:0.0:1776178096.065718:0:6255:0:(ldlm_request.c:528:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: mdt-lustre-MDT0000_UUID lock: 000000000584b645/0xa40083ab9bf52f3d lrc: 3/0,1 mode: PW/PW res: [0x2000013a1:0x3:0x0].0x0 bits 0x2/0x0 rrc: 3 type: IBT gid 0 flags: 0x40210000000000 nid: local remote: 0x0 expref: -99 pid: 6255 timeout: 0 lvb_type: 0 00000004:00010000:0.0:1776178096.065888:0:6255:0:(mdt_handler.c:3992:mdt_save_lock()) ### save lock request 00000000d6229cf9 reply state 0000000062cc3974 transno 12884901907 ns: mdt-lustre-MDT0000_UUID lock: 000000000584b645/0xa40083ab9bf52f3d lrc: 3/0,1 mode: PW/PW res: [0x2000013a1:0x3:0x0].0x0 bits 0x2/0x0 rrc: 3 type: IBT gid 0 flags: 0x40210000000000 nid: local remote: 0x0 expref: -99 pid: 6255 timeout: 0 lvb_type: 0 00000100:00100000:0.0:1776178096.066041:0:6255:0:(service.c:2372:ptlrpc_server_handle_request()) Handled RPC req@00000000d6229cf9 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_000:b50963bd-8eba-4432-9a54-8888d00e3fa1+24:16989:x1862456979077248:12345-192.168.204.3@tcp:36:touch.0 Request processed in 523us (902us total) trans 12884901907 rc 0/0 00000100:00100000:0.0:1776178096.066049:0:6255:0:(nrs_fifo.c:241:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.204.3@tcp, seq: 6466 00010000:00010000:2.0:1776178096.066577:0:3339:0:(ldlm_lock.c:1064:ldlm_granted_list_add_lock()) ### About to add lock: ns: mdt-lustre-MDT0000_UUID lock: 00000000b82e73fc/0xa40083ab9bf52f28 lrc: 3/0,1 mode: COS/COS res: [0x2000013a1:0x1:0x0].0x15fcbbea bits 0x2/0x0 rrc: 2 type: IBT gid 0 flags: 0x50210000000000 nid: local remote: 0x0 expref: -99 pid: 6256 timeout: 0 lvb_type: 0 00010000:00010000:2.0:1776178096.066594:0:3339:0:(ldlm_inodebits.c:94:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x2000013a1:0x1:0x0].0x15fcbbea (00000000a8172224) 00010000:00010000:2.0:1776178096.066597:0:3339:0:(ldlm_inodebits.c:96:ldlm_reprocess_inodebits_queue()) Hint 2 00000100:00100000:1.0:1776178096.069710:0:14440:0:(service.c:2152:ptlrpc_server_handle_req_in()) got req x1862456979077312 00000100:00100000:1.0:1776178096.069810:0:14440:0:(nrs_fifo.c:179:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.204.3@tcp, seq: 5230 00000100:00100000:1.0:1776178096.069821:0:14440:0:(service.c:2324:ptlrpc_server_handle_request()) Handling RPC req@00000000f4a76978 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_004:b50963bd-8eba-4432-9a54-8888d00e3fa1+5:2271:x1862456979077312:12345-192.168.204.3@tcp:2:touch.0 00010000:00010000:0.0:1776178096.070311:0:3340:0:(ldlm_lock.c:1064:ldlm_granted_list_add_lock()) ### About to add lock: ns: mdt-lustre-MDT0000_UUID lock: 000000000584b645/0xa40083ab9bf52f3d lrc: 3/0,1 mode: COS/COS res: [0x2000013a1:0x3:0x0].0x0 bits 0x2/0x0 rrc: 3 type: IBT gid 0 flags: 0x50210000000000 nid: local remote: 0x0 expref: -99 pid: 6255 timeout: 0 lvb_type: 0 00010000:00010000:0.0:1776178096.070336:0:3340:0:(ldlm_inodebits.c:94:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x2000013a1:0x3:0x0].0x0 (00000000d0a7c19f) 00010000:00010000:0.0:1776178096.070339:0:3340:0:(ldlm_inodebits.c:96:ldlm_reprocess_inodebits_queue()) Hint 2 00000100:00100000:1.0:1776178096.070547:0:14440:0:(service.c:2372:ptlrpc_server_handle_request()) Handled RPC req@00000000f4a76978 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_004:b50963bd-8eba-4432-9a54-8888d00e3fa1+6:2271:x1862456979077312:12345-192.168.204.3@tcp:2:touch.0 Request processed in 726us (1013us total) trans 4294969315 rc 0/0 00000100:00100000:1.0:1776178096.070558:0:14440:0:(nrs_fifo.c:241:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.204.3@tcp, seq: 5230 00000100:00100000:2.0:1776178096.074679:0:6258:0:(service.c:2152:ptlrpc_server_handle_req_in()) got req x1862456979077440 00000100:00100000:2.0:1776178096.074858:0:6258:0:(nrs_fifo.c:179:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.204.3@tcp, seq: 2124 00000100:00100000:2.0:1776178096.074871:0:6258:0:(service.c:2324:ptlrpc_server_handle_request()) Handling RPC req@000000005ec63ccb pname:cluuid+ref:pid:xid:nid:opc:job mdt_rdpg00_000:b50963bd-8eba-4432-9a54-8888d00e3fa1+24:16989:x1862456979077440:12345-192.168.204.3@tcp:35:touch.0 00000100:00100000:2.0:1776178096.076723:0:6258:0:(service.c:2372:ptlrpc_server_handle_request()) Handled RPC req@000000005ec63ccb pname:cluuid+ref:pid:xid:nid:opc:job mdt_rdpg00_000:b50963bd-8eba-4432-9a54-8888d00e3fa1+25:16989:x1862456979077440:12345-192.168.204.3@tcp:35:touch.0 Request processed in 1831us (2204us total) trans 12884901908 rc 0/0 00000100:00100000:2.0:1776178096.076748:0:6258:0:(nrs_fifo.c:241:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.204.3@tcp, seq: 2124 00000100:00100000:2.0:1776178096.105196:0:3346:0:(events.c:361:request_in_callback()) peer: 12345-192.168.204.3@tcp (source: 12345-192.168.204.3@tcp) 00000100:00100000:0.0:1776178096.105343:0:6255:0:(service.c:2152:ptlrpc_server_handle_req_in()) got req x1862456979077568 00000100:00100000:0.0:1776178096.105545:0:6255:0:(nrs_fifo.c:179:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.204.3@tcp, seq: 6467 00000100:00100000:0.0:1776178096.105555:0:6255:0:(service.c:2324:ptlrpc_server_handle_request()) Handling RPC req@000000001b4f0008 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_000:b50963bd-8eba-4432-9a54-8888d00e3fa1+25:16990:x1862456979077568:12345-192.168.204.3@tcp:101:touch.0 00010000:00010000:0.0:1776178096.105599:0:6255:0:(ldlm_lockd.c:1248:ldlm_handle_enqueue0()) ### server-side enqueue handler START 00010000:00010000:0.0:1776178096.105633:0:6255:0:(ldlm_lockd.c:1318:ldlm_handle_enqueue0()) ### server-side enqueue handler, new lock created ns: mdt-lustre-MDT0000_UUID lock: 00000000319537bc/0xa40083ab9bf52f44 lrc: 2/0,0 mode: --/CW res: [0x2000013a1:0x1:0x0].0x0 bits 0x0/0x0 rrc: 4 type: IBT gid 0 flags: 0x40000000000000 nid: local remote: 0x771f68aa1eb27982 expref: -99 pid: 6255 timeout: 0 lvb_type: 0 00010000:00010000:0.0:1776178096.105842:0:6255:0:(ldlm_lock.c:766:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(CW) ns: mdt-lustre-MDT0000_UUID lock: 000000004bd1d7cb/0xa40083ab9bf52f4b lrc: 3/0,1 mode: --/CW res: [0x2000013a1:0x1:0x0].0x0 bits 0x0/0x0 rrc: 5 type: IBT gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 6255 timeout: 0 lvb_type: 0 00010000:00010000:0.0:1776178096.105870:0:6255:0:(ldlm_lock.c:1064:ldlm_granted_list_add_lock()) ### About to add lock: ns: mdt-lustre-MDT0000_UUID lock: 000000004bd1d7cb/0xa40083ab9bf52f4b lrc: 3/0,1 mode: CW/CW res: [0x2000013a1:0x1:0x0].0x0 bits 0x2/0x0 rrc: 5 type: IBT gid 0 flags: 0x50210000000000 nid: local remote: 0x0 expref: -99 pid: 6255 timeout: 0 lvb_type: 0 00010000:00010000:0.0:1776178096.105890:0:6255:0:(ldlm_request.c:528:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: mdt-lustre-MDT0000_UUID lock: 000000004bd1d7cb/0xa40083ab9bf52f4b lrc: 3/0,1 mode: CW/CW res: [0x2000013a1:0x1:0x0].0x0 bits 0x2/0x0 rrc: 5 type: IBT gid 0 flags: 0x40210000000000 nid: local remote: 0x0 expref: -99 pid: 6255 timeout: 0 lvb_type: 0 00010000:00010000:0.0:1776178096.105913:0:6255:0:(ldlm_lock.c:766:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: mdt-lustre-MDT0000_UUID lock: 000000008cce7e8f/0xa40083ab9bf52f52 lrc: 3/0,1 mode: --/PW res: [0x2000013a1:0x1:0x0].0x5e5674ea bits 0x0/0x0 rrc: 2 type: IBT gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 6255 timeout: 0 lvb_type: 0 00010000:00010000:0.0:1776178096.105922:0:6255:0:(ldlm_lock.c:1064:ldlm_granted_list_add_lock()) ### About to add lock: ns: mdt-lustre-MDT0000_UUID lock: 000000008cce7e8f/0xa40083ab9bf52f52 lrc: 3/0,1 mode: PW/PW res: [0x2000013a1:0x1:0x0].0x5e5674ea bits 0x2/0x0 rrc: 2 type: IBT gid 0 flags: 0x50210000000000 nid: local remote: 0x0 expref: -99 pid: 6255 timeout: 0 lvb_type: 0 00010000:00010000:0.0:1776178096.105929:0:6255:0:(ldlm_request.c:528:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: mdt-lustre-MDT0000_UUID lock: 000000008cce7e8f/0xa40083ab9bf52f52 lrc: 3/0,1 mode: PW/PW res: [0x2000013a1:0x1:0x0].0x5e5674ea bits 0x2/0x0 rrc: 2 type: IBT gid 0 flags: 0x40210000000000 nid: local remote: 0x0 expref: -99 pid: 6255 timeout: 0 lvb_type: 0 00000004:00080000:0.0:1776178096.107641:0:6255:0:(osp_object.c:1645:osp_create()) lustre-OST0000-osc-MDT0000: Wrote last used FID: [0x100000000:0x43:0x0], index 0: 0 00010000:00010000:0.0:1776178096.109566:0:6255:0:(ldlm_lock.c:766:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(CR) ns: mdt-lustre-MDT0000_UUID lock: 00000000b4e1b221/0xa40083ab9bf52f59 lrc: 3/1,0 mode: --/CR res: [0x2000013a1:0x4:0x0].0x0 bits 0x0/0x0 rrc: 2 type: IBT gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 6255 timeout: 0 lvb_type: 0 00010000:00010000:0.0:1776178096.109606:0:6255:0:(ldlm_lock.c:1064:ldlm_granted_list_add_lock()) ### About to add lock: ns: mdt-lustre-MDT0000_UUID lock: 00000000b4e1b221/0xa40083ab9bf52f59 lrc: 3/1,0 mode: CR/CR res: [0x2000013a1:0x4:0x0].0x0 bits 0x9/0x0 rrc: 2 type: IBT gid 0 flags: 0x50210000000000 nid: local remote: 0x0 expref: -99 pid: 6255 timeout: 0 lvb_type: 0 00010000:00010000:0.0:1776178096.109624:0:6255:0:(ldlm_request.c:528:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: mdt-lustre-MDT0000_UUID lock: 00000000b4e1b221/0xa40083ab9bf52f59 lrc: 3/1,0 mode: CR/CR res: [0x2000013a1:0x4:0x0].0x0 bits 0x9/0x0 rrc: 2 type: IBT gid 0 flags: 0x40210000000000 nid: local remote: 0x0 expref: -99 pid: 6255 timeout: 0 lvb_type: 0 00010000:00010000:0.0:1776178096.109684:0:6255:0:(ldlm_lock.c:820:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(CW) ns: mdt-lustre-MDT0000_UUID lock: 000000004bd1d7cb/0xa40083ab9bf52f4b lrc: 3/0,1 mode: CW/CW res: [0x2000013a1:0x1:0x0].0x0 bits 0x2/0x0 rrc: 5 type: IBT gid 0 flags: 0x50210000000000 nid: local remote: 0x0 expref: -99 pid: 6255 timeout: 0 lvb_type: 0 00010000:00010000:0.0:1776178096.109692:0:6255:0:(ldlm_lock.c:879:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: mdt-lustre-MDT0000_UUID lock: 000000004bd1d7cb/0xa40083ab9bf52f4b lrc: 2/0,0 mode: CW/CW res: [0x2000013a1:0x1:0x0].0x0 bits 0x2/0x0 rrc: 5 type: IBT gid 0 flags: 0x50210400000000 nid: local remote: 0x0 expref: -99 pid: 6255 timeout: 0 lvb_type: 0 00010000:00010000:0.0:1776178096.109702:0:6255:0:(ldlm_lockd.c:1879:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: mdt-lustre-MDT0000_UUID lock: 000000004bd1d7cb/0xa40083ab9bf52f4b lrc: 3/0,0 mode: CW/CW res: [0x2000013a1:0x1:0x0].0x0 bits 0x2/0x0 rrc: 5 type: IBT gid 0 flags: 0x40210400000000 nid: local remote: 0x0 expref: -99 pid: 6255 timeout: 0 lvb_type: 0 00010000:00010000:0.0:1776178096.109709:0:6255:0:(ldlm_lockd.c:1893:ldlm_handle_bl_callback()) Lock 000000004bd1d7cb already unused, calling callback (0000000079e78348) 00010000:00010000:0.0:1776178096.109714:0:6255:0:(ldlm_request.c:364:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: mdt-lustre-MDT0000_UUID lock: 000000004bd1d7cb/0xa40083ab9bf52f4b lrc: 3/0,0 mode: CW/CW res: [0x2000013a1:0x1:0x0].0x0 bits 0x2/0x0 rrc: 5 type: IBT gid 0 flags: 0x40210400000000 nid: local remote: 0x0 expref: -99 pid: 6255 timeout: 0 lvb_type: 0 00010000:00010000:0.0:1776178096.109729:0:6255:0:(ldlm_request.c:1257:ldlm_cli_cancel_local()) ### server-side local cancel ns: mdt-lustre-MDT0000_UUID lock: 000000004bd1d7cb/0xa40083ab9bf52f4b lrc: 4/0,0 mode: CW/CW res: [0x2000013a1:0x1:0x0].0x0 bits 0x2/0x0 rrc: 5 type: IBT gid 0 flags: 0x40218400000000 nid: local remote: 0x0 expref: -99 pid: 6255 timeout: 0 lvb_type: 0 00010000:00010000:0.0:1776178096.109744:0:6255:0:(ldlm_inodebits.c:94:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x2000013a1:0x1:0x0].0x0 (00000000315facbe) 00010000:00010000:0.0:1776178096.109748:0:6255:0:(ldlm_inodebits.c:96:ldlm_reprocess_inodebits_queue()) Hint 2 00010000:00010000:0.0:1776178096.109754:0:6255:0:(ldlm_lockd.c:1903:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: mdt-lustre-MDT0000_UUID lock: 000000004bd1d7cb/0xa40083ab9bf52f4b lrc: 2/0,0 mode: --/CW res: [0x2000013a1:0x1:0x0].0x0 bits 0x2/0x0 rrc: 5 type: IBT gid 0 flags: 0x44a19400000000 nid: local remote: 0x0 expref: -99 pid: 6255 timeout: 0 lvb_type: 0 00010000:00010000:0.0:1776178096.109772:0:6255:0:(ldlm_lock.c:216:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: mdt-lustre-MDT0000_UUID lock: 000000004bd1d7cb/0xa40083ab9bf52f4b lrc: 0/0,0 mode: --/CW res: [0x2000013a1:0x1:0x0].0x0 bits 0x2/0x0 rrc: 5 type: IBT gid 0 flags: 0x44a19400000000 nid: local remote: 0x0 expref: -99 pid: 6255 timeout: 0 lvb_type: 0 00000004:00010000:0.0:1776178096.109789:0:6255:0:(mdt_handler.c:3992:mdt_save_lock()) ### save lock request 000000001b4f0008 reply state 000000003d86b673 transno 12884901909 ns: mdt-lustre-MDT0000_UUID lock: 000000008cce7e8f/0xa40083ab9bf52f52 lrc: 3/0,1 mode: PW/PW res: [0x2000013a1:0x1:0x0].0x5e5674ea bits 0x2/0x0 rrc: 2 type: IBT gid 0 flags: 0x40210000000000 nid: local remote: 0x0 expref: -99 pid: 6255 timeout: 0 lvb_type: 0 00010000:00010000:0.0:1776178096.109841:0:6255:0:(ldlm_lock.c:216:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: mdt-lustre-MDT0000_UUID lock: 00000000319537bc/0xa40083ab9bf52f44 lrc: 0/0,0 mode: --/CW res: [0x2000013a1:0x1:0x0].0x0 bits 0x1/0x0 rrc: 4 type: IBT gid 0 flags: 0x44000000000000 nid: 192.168.204.3@tcp remote: 0x771f68aa1eb27982 expref: 28 pid: 6255 timeout: 0 lvb_type: 0 00010000:00010000:0.0:1776178096.109855:0:6255:0:(ldlm_lockd.c:1484:ldlm_handle_enqueue0()) ### server-side enqueue handler, sending reply (err=0, rc=0) ns: mdt-lustre-MDT0000_UUID lock: 00000000b4e1b221/0xa40083ab9bf52f59 lrc: 3/0,0 mode: CR/CR res: [0x2000013a1:0x4:0x0].0x0 bits 0x9/0x0 rrc: 2 type: IBT gid 0 flags: 0x40200000000000 nid: 192.168.204.3@tcp remote: 0x771f68aa1eb27982 expref: 27 pid: 6255 timeout: 0 lvb_type: 0 00010000:00010000:0.0:1776178096.109862:0:6255:0:(ldlm_inodebits.c:94:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x2000013a1:0x4:0x0].0x0 (00000000ef6cecb2) 00010000:00010000:0.0:1776178096.109864:0:6255:0:(ldlm_inodebits.c:96:ldlm_reprocess_inodebits_queue()) Hint 9 00010000:00010000:0.0:1776178096.109866:0:6255:0:(ldlm_lockd.c:1565:ldlm_handle_enqueue0()) ### server-side enqueue handler END (lock 00000000b4e1b221, rc 0) 00000100:00100000:0.0:1776178096.110749:0:6255:0:(service.c:2372:ptlrpc_server_handle_request()) Handled RPC req@000000001b4f0008 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_000:b50963bd-8eba-4432-9a54-8888d00e3fa1+28:16990:x1862456979077568:12345-192.168.204.3@tcp:101:touch.0 Request processed in 5184us (5768us total) trans 12884901909 rc 0/0 00000100:00100000:0.0:1776178096.110775:0:6255:0:(nrs_fifo.c:241:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.204.3@tcp, seq: 6467 00000100:00100000:2.0:1776178096.123749:0:3348:0:(events.c:361:request_in_callback()) peer: 12345-192.168.204.3@tcp (source: 12345-192.168.204.3@tcp) 00010000:00010000:0.0:1776178096.123849:0:3339:0:(ldlm_lock.c:1064:ldlm_granted_list_add_lock()) ### About to add lock: ns: mdt-lustre-MDT0000_UUID lock: 000000008cce7e8f/0xa40083ab9bf52f52 lrc: 3/0,1 mode: COS/COS res: [0x2000013a1:0x1:0x0].0x5e5674ea bits 0x2/0x0 rrc: 2 type: IBT gid 0 flags: 0x50210000000000 nid: local remote: 0x0 expref: -99 pid: 6255 timeout: 0 lvb_type: 0 00010000:00010000:0.0:1776178096.123882:0:3339:0:(ldlm_inodebits.c:94:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x2000013a1:0x1:0x0].0x5e5674ea (000000008e570e08) 00010000:00010000:0.0:1776178096.123886:0:3339:0:(ldlm_inodebits.c:96:ldlm_reprocess_inodebits_queue()) Hint 2 00010000:00010000:0.0:1776178096.129923:0:3340:0:(ldlm_lock.c:1064:ldlm_granted_list_add_lock()) ### About to add lock: ns: mdt-lustre-MDT0000_UUID lock: 00000000dcea299c/0xa40083ab9bf52f67 lrc: 3/0,1 mode: COS/COS res: [0x2000013a1:0x4:0x0].0x0 bits 0x2/0x0 rrc: 3 type: IBT gid 0 flags: 0x50210000000000 nid: local remote: 0x0 expref: -99 pid: 6255 timeout: 0 lvb_type: 0 00010000:00010000:0.0:1776178096.129962:0:3340:0:(ldlm_inodebits.c:94:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x2000013a1:0x4:0x0].0x0 (00000000ef6cecb2) 00010000:00010000:0.0:1776178096.129965:0:3340:0:(ldlm_inodebits.c:96:ldlm_reprocess_inodebits_queue()) Hint 2 00000100:00100000:2.0:1776178096.130390:0:3348:0:(events.c:361:request_in_callback()) peer: 12345-192.168.204.3@tcp (source: 12345-192.168.204.3@tcp) 00000100:00100000:1.0:1776178096.130582:0:14440:0:(service.c:2152:ptlrpc_server_handle_req_in()) got req x1862456979077760 00000100:00100000:1.0:1776178096.130656:0:14440:0:(nrs_fifo.c:179:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.204.3@tcp, seq: 5231 00000100:00100000:1.0:1776178096.130665:0:14440:0:(service.c:2324:ptlrpc_server_handle_request()) Handling RPC req@000000001c8035b1 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_004:b50963bd-8eba-4432-9a54-8888d00e3fa1+6:2271:x1862456979077760:12345-192.168.204.3@tcp:2:touch.0 00000100:00100000:1.0:1776178096.131003:0:14440:0:(service.c:2372:ptlrpc_server_handle_request()) Handled RPC req@000000001c8035b1 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_004:b50963bd-8eba-4432-9a54-8888d00e3fa1+7:2271:x1862456979077760:12345-192.168.204.3@tcp:2:touch.0 Request processed in 338us (620us total) trans 4294969342 rc 0/0 00000100:00100000:1.0:1776178096.131012:0:14440:0:(nrs_fifo.c:241:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.204.3@tcp, seq: 5231 00000100:00100000:2.0:1776178096.137435:0:6258:0:(service.c:2152:ptlrpc_server_handle_req_in()) got req x1862456979077888 00000100:00100000:2.0:1776178096.137517:0:6258:0:(nrs_fifo.c:179:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.204.3@tcp, seq: 2125 00000100:00100000:2.0:1776178096.137527:0:6258:0:(service.c:2324:ptlrpc_server_handle_request()) Handling RPC req@0000000084030222 pname:cluuid+ref:pid:xid:nid:opc:job mdt_rdpg00_000:b50963bd-8eba-4432-9a54-8888d00e3fa1+30:16990:x1862456979077888:12345-192.168.204.3@tcp:35:touch.0 00000100:00100000:2.0:1776178096.138049:0:6258:0:(service.c:2372:ptlrpc_server_handle_request()) Handled RPC req@0000000084030222 pname:cluuid+ref:pid:xid:nid:opc:job mdt_rdpg00_000:b50963bd-8eba-4432-9a54-8888d00e3fa1+31:16990:x1862456979077888:12345-192.168.204.3@tcp:35:touch.0 Request processed in 520us (1024us total) trans 12884901911 rc 0/0 00000100:00100000:2.0:1776178096.138087:0:6258:0:(nrs_fifo.c:241:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.204.3@tcp, seq: 2125 00000100:00100000:0.0:1776178096.153256:0:6255:0:(service.c:2152:ptlrpc_server_handle_req_in()) got req x1862456979078016 00000100:00100000:1.0:1776178096.153441:0:6255:0:(nrs_fifo.c:179:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.204.3@tcp, seq: 6469 00000100:00100000:1.0:1776178096.153450:0:6255:0:(service.c:2324:ptlrpc_server_handle_request()) Handling RPC req@000000003cf21d73 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_000:b50963bd-8eba-4432-9a54-8888d00e3fa1+31:16991:x1862456979078016:12345-192.168.204.3@tcp:101:touch.0 00010000:00010000:1.0:1776178096.153494:0:6255:0:(ldlm_lockd.c:1248:ldlm_handle_enqueue0()) ### server-side enqueue handler START 00010000:00010000:1.0:1776178096.153529:0:6255:0:(ldlm_lockd.c:1318:ldlm_handle_enqueue0()) ### server-side enqueue handler, new lock created ns: mdt-lustre-MDT0000_UUID lock: 00000000b2e83690/0xa40083ab9bf52f6e lrc: 2/0,0 mode: --/CW res: [0x2000013a1:0x1:0x0].0x0 bits 0x0/0x0 rrc: 4 type: IBT gid 0 flags: 0x40000000000000 nid: local remote: 0x771f68aa1eb27989 expref: -99 pid: 6255 timeout: 0 lvb_type: 0 00010000:00010000:1.0:1776178096.153732:0:6255:0:(ldlm_lock.c:766:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(CW) ns: mdt-lustre-MDT0000_UUID lock: 0000000097975a3b/0xa40083ab9bf52f75 lrc: 3/0,1 mode: --/CW res: [0x2000013a1:0x1:0x0].0x0 bits 0x0/0x0 rrc: 5 type: IBT gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 6255 timeout: 0 lvb_type: 0 00010000:00010000:1.0:1776178096.153752:0:6255:0:(ldlm_lock.c:1064:ldlm_granted_list_add_lock()) ### About to add lock: ns: mdt-lustre-MDT0000_UUID lock: 0000000097975a3b/0xa40083ab9bf52f75 lrc: 3/0,1 mode: CW/CW res: [0x2000013a1:0x1:0x0].0x0 bits 0x2/0x0 rrc: 5 type: IBT gid 0 flags: 0x50210000000000 nid: local remote: 0x0 expref: -99 pid: 6255 timeout: 0 lvb_type: 0 00010000:00010000:1.0:1776178096.153781:0:6255:0:(ldlm_request.c:528:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: mdt-lustre-MDT0000_UUID lock: 0000000097975a3b/0xa40083ab9bf52f75 lrc: 3/0,1 mode: CW/CW res: [0x2000013a1:0x1:0x0].0x0 bits 0x2/0x0 rrc: 5 type: IBT gid 0 flags: 0x40210000000000 nid: local remote: 0x0 expref: -99 pid: 6255 timeout: 0 lvb_type: 0 00010000:00010000:1.0:1776178096.153799:0:6255:0:(ldlm_lock.c:766:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: mdt-lustre-MDT0000_UUID lock: 000000006b0afb52/0xa40083ab9bf52f7c lrc: 3/0,1 mode: --/PW res: [0x2000013a1:0x1:0x0].0xa6b02dea bits 0x0/0x0 rrc: 2 type: IBT gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 6255 timeout: 0 lvb_type: 0 00010000:00010000:1.0:1776178096.153806:0:6255:0:(ldlm_lock.c:1064:ldlm_granted_list_add_lock()) ### About to add lock: ns: mdt-lustre-MDT0000_UUID lock: 000000006b0afb52/0xa40083ab9bf52f7c lrc: 3/0,1 mode: PW/PW res: [0x2000013a1:0x1:0x0].0xa6b02dea bits 0x2/0x0 rrc: 2 type: IBT gid 0 flags: 0x50210000000000 nid: local remote: 0x0 expref: -99 pid: 6255 timeout: 0 lvb_type: 0 00010000:00010000:1.0:1776178096.153812:0:6255:0:(ldlm_request.c:528:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: mdt-lustre-MDT0000_UUID lock: 000000006b0afb52/0xa40083ab9bf52f7c lrc: 3/0,1 mode: PW/PW res: [0x2000013a1:0x1:0x0].0xa6b02dea bits 0x2/0x0 rrc: 2 type: IBT gid 0 flags: 0x40210000000000 nid: local remote: 0x0 expref: -99 pid: 6255 timeout: 0 lvb_type: 0 00000004:00080000:1.0:1776178096.154581:0:6255:0:(osp_object.c:1645:osp_create()) lustre-OST0001-osc-MDT0000: Wrote last used FID: [0x100010000:0x43:0x0], index 1: 0 00010000:00010000:1.0:1776178096.154826:0:6255:0:(ldlm_lock.c:766:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(CR) ns: mdt-lustre-MDT0000_UUID lock: 00000000336cd7d4/0xa40083ab9bf52f83 lrc: 3/1,0 mode: --/CR res: [0x2000013a1:0x5:0x0].0x0 bits 0x0/0x0 rrc: 2 type: IBT gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 6255 timeout: 0 lvb_type: 0 00010000:00010000:1.0:1776178096.154840:0:6255:0:(ldlm_lock.c:1064:ldlm_granted_list_add_lock()) ### About to add lock: ns: mdt-lustre-MDT0000_UUID lock: 00000000336cd7d4/0xa40083ab9bf52f83 lrc: 3/1,0 mode: CR/CR res: [0x2000013a1:0x5:0x0].0x0 bits 0x9/0x0 rrc: 2 type: IBT gid 0 flags: 0x50210000000000 nid: local remote: 0x0 expref: -99 pid: 6255 timeout: 0 lvb_type: 0 00010000:00010000:1.0:1776178096.154850:0:6255:0:(ldlm_request.c:528:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: mdt-lustre-MDT0000_UUID lock: 00000000336cd7d4/0xa40083ab9bf52f83 lrc: 3/1,0 mode: CR/CR res: [0x2000013a1:0x5:0x0].0x0 bits 0x9/0x0 rrc: 2 type: IBT gid 0 flags: 0x40210000000000 nid: local remote: 0x0 expref: -99 pid: 6255 timeout: 0 lvb_type: 0 00010000:00010000:1.0:1776178096.154908:0:6255:0:(ldlm_lock.c:820:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(CW) ns: mdt-lustre-MDT0000_UUID lock: 0000000097975a3b/0xa40083ab9bf52f75 lrc: 3/0,1 mode: CW/CW res: [0x2000013a1:0x1:0x0].0x0 bits 0x2/0x0 rrc: 5 type: IBT gid 0 flags: 0x50210000000000 nid: local remote: 0x0 expref: -99 pid: 6255 timeout: 0 lvb_type: 0 00010000:00010000:1.0:1776178096.154918:0:6255:0:(ldlm_lock.c:879:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: mdt-lustre-MDT0000_UUID lock: 0000000097975a3b/0xa40083ab9bf52f75 lrc: 2/0,0 mode: CW/CW res: [0x2000013a1:0x1:0x0].0x0 bits 0x2/0x0 rrc: 5 type: IBT gid 0 flags: 0x50210400000000 nid: local remote: 0x0 expref: -99 pid: 6255 timeout: 0 lvb_type: 0 00010000:00010000:1.0:1776178096.154925:0:6255:0:(ldlm_lockd.c:1879:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: mdt-lustre-MDT0000_UUID lock: 0000000097975a3b/0xa40083ab9bf52f75 lrc: 3/0,0 mode: CW/CW res: [0x2000013a1:0x1:0x0].0x0 bits 0x2/0x0 rrc: 5 type: IBT gid 0 flags: 0x40210400000000 nid: local remote: 0x0 expref: -99 pid: 6255 timeout: 0 lvb_type: 0 00010000:00010000:1.0:1776178096.154932:0:6255:0:(ldlm_lockd.c:1893:ldlm_handle_bl_callback()) Lock 0000000097975a3b already unused, calling callback (0000000079e78348) 00010000:00010000:1.0:1776178096.154937:0:6255:0:(ldlm_request.c:364:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: mdt-lustre-MDT0000_UUID lock: 0000000097975a3b/0xa40083ab9bf52f75 lrc: 3/0,0 mode: CW/CW res: [0x2000013a1:0x1:0x0].0x0 bits 0x2/0x0 rrc: 5 type: IBT gid 0 flags: 0x40210400000000 nid: local remote: 0x0 expref: -99 pid: 6255 timeout: 0 lvb_type: 0 00010000:00010000:1.0:1776178096.154946:0:6255:0:(ldlm_request.c:1257:ldlm_cli_cancel_local()) ### server-side local cancel ns: mdt-lustre-MDT0000_UUID lock: 0000000097975a3b/0xa40083ab9bf52f75 lrc: 4/0,0 mode: CW/CW res: [0x2000013a1:0x1:0x0].0x0 bits 0x2/0x0 rrc: 5 type: IBT gid 0 flags: 0x40218400000000 nid: local remote: 0x0 expref: -99 pid: 6255 timeout: 0 lvb_type: 0 00010000:00010000:1.0:1776178096.154962:0:6255:0:(ldlm_inodebits.c:94:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x2000013a1:0x1:0x0].0x0 (00000000315facbe) 00010000:00010000:1.0:1776178096.154966:0:6255:0:(ldlm_inodebits.c:96:ldlm_reprocess_inodebits_queue()) Hint 2 00010000:00010000:1.0:1776178096.154971:0:6255:0:(ldlm_lockd.c:1903:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: mdt-lustre-MDT0000_UUID lock: 0000000097975a3b/0xa40083ab9bf52f75 lrc: 2/0,0 mode: --/CW res: [0x2000013a1:0x1:0x0].0x0 bits 0x2/0x0 rrc: 5 type: IBT gid 0 flags: 0x44a19400000000 nid: local remote: 0x0 expref: -99 pid: 6255 timeout: 0 lvb_type: 0 00010000:00010000:1.0:1776178096.154979:0:6255:0:(ldlm_lock.c:216:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: mdt-lustre-MDT0000_UUID lock: 0000000097975a3b/0xa40083ab9bf52f75 lrc: 0/0,0 mode: --/CW res: [0x2000013a1:0x1:0x0].0x0 bits 0x2/0x0 rrc: 5 type: IBT gid 0 flags: 0x44a19400000000 nid: local remote: 0x0 expref: -99 pid: 6255 timeout: 0 lvb_type: 0 00000004:00010000:1.0:1776178096.154999:0:6255:0:(mdt_handler.c:3992:mdt_save_lock()) ### save lock request 000000003cf21d73 reply state 00000000b13de3e3 transno 12884901912 ns: mdt-lustre-MDT0000_UUID lock: 000000006b0afb52/0xa40083ab9bf52f7c lrc: 3/0,1 mode: PW/PW res: [0x2000013a1:0x1:0x0].0xa6b02dea bits 0x2/0x0 rrc: 2 type: IBT gid 0 flags: 0x40210000000000 nid: local remote: 0x0 expref: -99 pid: 6255 timeout: 0 lvb_type: 0 00010000:00010000:1.0:1776178096.155046:0:6255:0:(ldlm_lock.c:216:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: mdt-lustre-MDT0000_UUID lock: 00000000b2e83690/0xa40083ab9bf52f6e lrc: 0/0,0 mode: --/CW res: [0x2000013a1:0x1:0x0].0x0 bits 0x1/0x0 rrc: 4 type: IBT gid 0 flags: 0x44000000000000 nid: 192.168.204.3@tcp remote: 0x771f68aa1eb27989 expref: 34 pid: 6255 timeout: 0 lvb_type: 0 00010000:00010000:1.0:1776178096.155127:0:6255:0:(ldlm_lockd.c:1484:ldlm_handle_enqueue0()) ### server-side enqueue handler, sending reply (err=0, rc=0) ns: mdt-lustre-MDT0000_UUID lock: 00000000336cd7d4/0xa40083ab9bf52f83 lrc: 3/0,0 mode: CR/CR res: [0x2000013a1:0x5:0x0].0x0 bits 0x9/0x0 rrc: 2 type: IBT gid 0 flags: 0x40200000000000 nid: 192.168.204.3@tcp remote: 0x771f68aa1eb27989 expref: 33 pid: 6255 timeout: 0 lvb_type: 0 00010000:00010000:1.0:1776178096.155134:0:6255:0:(ldlm_inodebits.c:94:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x2000013a1:0x5:0x0].0x0 (000000003b465a63) 00010000:00010000:1.0:1776178096.155136:0:6255:0:(ldlm_inodebits.c:96:ldlm_reprocess_inodebits_queue()) Hint 9 00010000:00010000:1.0:1776178096.155138:0:6255:0:(ldlm_lockd.c:1565:ldlm_handle_enqueue0()) ### server-side enqueue handler END (lock 00000000336cd7d4, rc 0) 00000100:00100000:1.0:1776178096.155349:0:6255:0:(service.c:2372:ptlrpc_server_handle_request()) Handled RPC req@000000003cf21d73 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_000:b50963bd-8eba-4432-9a54-8888d00e3fa1+34:16991:x1862456979078016:12345-192.168.204.3@tcp:101:touch.0 Request processed in 1896us (2419us total) trans 12884901912 rc 0/0 00000100:00100000:1.0:1776178096.155360:0:6255:0:(nrs_fifo.c:241:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.204.3@tcp, seq: 6469 00010000:00010000:2.0:1776178096.157943:0:3339:0:(ldlm_lock.c:1064:ldlm_granted_list_add_lock()) ### About to add lock: ns: mdt-lustre-MDT0000_UUID lock: 000000006b0afb52/0xa40083ab9bf52f7c lrc: 3/0,1 mode: COS/COS res: [0x2000013a1:0x1:0x0].0xa6b02dea bits 0x2/0x0 rrc: 2 type: IBT gid 0 flags: 0x50210000000000 nid: local remote: 0x0 expref: -99 pid: 6255 timeout: 0 lvb_type: 0 00010000:00010000:2.0:1776178096.157980:0:3339:0:(ldlm_inodebits.c:94:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x2000013a1:0x1:0x0].0xa6b02dea (000000009cfa2d69) 00010000:00010000:2.0:1776178096.157984:0:3339:0:(ldlm_inodebits.c:96:ldlm_reprocess_inodebits_queue()) Hint 2 00000100:00100000:1.0:1776178096.162815:0:6255:0:(service.c:2152:ptlrpc_server_handle_req_in()) got req x1862456979078144 00000100:00100000:1.0:1776178096.162912:0:6255:0:(nrs_fifo.c:179:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.204.3@tcp, seq: 6470 00000100:00100000:1.0:1776178096.162921:0:6255:0:(service.c:2324:ptlrpc_server_handle_request()) Handling RPC req@0000000001be1ac2 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_000:b50963bd-8eba-4432-9a54-8888d00e3fa1+34:16991:x1862456979078144:12345-192.168.204.3@tcp:36:touch.0 00010000:00010000:1.0:1776178096.163678:0:6255:0:(ldlm_lock.c:766:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: mdt-lustre-MDT0000_UUID lock: 00000000b2e83690/0xa40083ab9bf52f91 lrc: 3/0,1 mode: --/PW res: [0x2000013a1:0x5:0x0].0x0 bits 0x0/0x0 rrc: 3 type: IBT gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 6255 timeout: 0 lvb_type: 0 00010000:00010000:1.0:1776178096.163712:0:6255:0:(ldlm_lock.c:1064:ldlm_granted_list_add_lock()) ### About to add lock: ns: mdt-lustre-MDT0000_UUID lock: 00000000b2e83690/0xa40083ab9bf52f91 lrc: 3/0,1 mode: PW/PW res: [0x2000013a1:0x5:0x0].0x0 bits 0x2/0x0 rrc: 3 type: IBT gid 0 flags: 0x50210000000000 nid: local remote: 0x0 expref: -99 pid: 6255 timeout: 0 lvb_type: 0 00010000:00010000:1.0:1776178096.163737:0:6255:0:(ldlm_request.c:528:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: mdt-lustre-MDT0000_UUID lock: 00000000b2e83690/0xa40083ab9bf52f91 lrc: 3/0,1 mode: PW/PW res: [0x2000013a1:0x5:0x0].0x0 bits 0x2/0x0 rrc: 3 type: IBT gid 0 flags: 0x40210000000000 nid: local remote: 0x0 expref: -99 pid: 6255 timeout: 0 lvb_type: 0 00000004:00010000:1.0:1776178096.164545:0:6255:0:(mdt_handler.c:3992:mdt_save_lock()) ### save lock request 0000000001be1ac2 reply state 0000000045b1aab3 transno 12884901913 ns: mdt-lustre-MDT0000_UUID lock: 00000000b2e83690/0xa40083ab9bf52f91 lrc: 3/0,1 mode: PW/PW res: [0x2000013a1:0x5:0x0].0x0 bits 0x2/0x0 rrc: 3 type: IBT gid 0 flags: 0x40210000000000 nid: local remote: 0x0 expref: -99 pid: 6255 timeout: 0 lvb_type: 0 00000100:00100000:1.0:1776178096.164773:0:6255:0:(service.c:2372:ptlrpc_server_handle_request()) Handled RPC req@0000000001be1ac2 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_000:b50963bd-8eba-4432-9a54-8888d00e3fa1+36:16991:x1862456979078144:12345-192.168.204.3@tcp:36:touch.0 Request processed in 1848us (2799us total) trans 12884901913 rc 0/0 00000100:00100000:1.0:1776178096.164784:0:6255:0:(nrs_fifo.c:241:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.204.3@tcp, seq: 6470 00010000:00010000:2.0:1776178096.175382:0:3340:0:(ldlm_lock.c:1064:ldlm_granted_list_add_lock()) ### About to add lock: ns: mdt-lustre-MDT0000_UUID lock: 00000000b2e83690/0xa40083ab9bf52f91 lrc: 3/0,1 mode: COS/COS res: [0x2000013a1:0x5:0x0].0x0 bits 0x2/0x0 rrc: 3 type: IBT gid 0 flags: 0x50210000000000 nid: local remote: 0x0 expref: -99 pid: 6255 timeout: 0 lvb_type: 0 00010000:00010000:2.0:1776178096.175412:0:3340:0:(ldlm_inodebits.c:94:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x2000013a1:0x5:0x0].0x0 (000000003b465a63) 00010000:00010000:2.0:1776178096.175415:0:3340:0:(ldlm_inodebits.c:96:ldlm_reprocess_inodebits_queue()) Hint 2 00000100:00100000:2.0:1776178096.176642:0:20762:0:(service.c:2152:ptlrpc_server_handle_req_in()) got req x1862456979078208 00000100:00100000:2.0:1776178096.177355:0:20762:0:(nrs_fifo.c:179:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.204.3@tcp, seq: 5232 00000100:00100000:2.0:1776178096.177366:0:20762:0:(service.c:2324:ptlrpc_server_handle_request()) Handling RPC req@00000000529daae4 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_013:b50963bd-8eba-4432-9a54-8888d00e3fa1+6:2273:x1862456979078208:12345-192.168.204.3@tcp:2:touch.0 00000100:00100000:2.0:1776178096.178896:0:20762:0:(service.c:2372:ptlrpc_server_handle_request()) Handled RPC req@00000000529daae4 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_013:b50963bd-8eba-4432-9a54-8888d00e3fa1+7:2273:x1862456979078208:12345-192.168.204.3@tcp:2:touch.0 Request processed in 1515us (2626us total) trans 4294969316 rc 0/0 00000100:00100000:2.0:1776178096.178920:0:20762:0:(nrs_fifo.c:241:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.204.3@tcp, seq: 5232 00000100:00100000:0.0:1776178096.190995:0:3348:0:(events.c:361:request_in_callback()) peer: 12345-192.168.204.3@tcp (source: 12345-192.168.204.3@tcp) 00000100:00100000:0.0:1776178096.210911:0:3347:0:(events.c:361:request_in_callback()) peer: 12345-192.168.204.3@tcp (source: 12345-192.168.204.3@tcp) 00000100:00100000:1.0:1776178096.211144:0:6255:0:(service.c:2152:ptlrpc_server_handle_req_in()) got req x1862456979078464 00000100:00100000:1.0:1776178096.211240:0:6255:0:(nrs_fifo.c:179:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.204.3@tcp, seq: 6471 00000100:00100000:1.0:1776178096.211265:0:6255:0:(service.c:2324:ptlrpc_server_handle_request()) Handling RPC req@00000000c8a61992 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_000:b50963bd-8eba-4432-9a54-8888d00e3fa1+37:16992:x1862456979078464:12345-192.168.204.3@tcp:101:touch.0 00010000:00010000:1.0:1776178096.211316:0:6255:0:(ldlm_lockd.c:1248:ldlm_handle_enqueue0()) ### server-side enqueue handler START 00010000:00010000:1.0:1776178096.211536:0:6255:0:(ldlm_lockd.c:1318:ldlm_handle_enqueue0()) ### server-side enqueue handler, new lock created ns: mdt-lustre-MDT0000_UUID lock: 0000000097975a3b/0xa40083ab9bf52f98 lrc: 2/0,0 mode: --/CW res: [0x2000013a1:0x1:0x0].0x0 bits 0x0/0x0 rrc: 4 type: IBT gid 0 flags: 0x40000000000000 nid: local remote: 0x771f68aa1eb27990 expref: -99 pid: 6255 timeout: 0 lvb_type: 0 00010000:00010000:1.0:1776178096.211742:0:6255:0:(ldlm_lock.c:766:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(CW) ns: mdt-lustre-MDT0000_UUID lock: 00000000a44d8583/0xa40083ab9bf52f9f lrc: 3/0,1 mode: --/CW res: [0x2000013a1:0x1:0x0].0x0 bits 0x0/0x0 rrc: 5 type: IBT gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 6255 timeout: 0 lvb_type: 0 00010000:00010000:1.0:1776178096.211780:0:6255:0:(ldlm_lock.c:1064:ldlm_granted_list_add_lock()) ### About to add lock: ns: mdt-lustre-MDT0000_UUID lock: 00000000a44d8583/0xa40083ab9bf52f9f lrc: 3/0,1 mode: CW/CW res: [0x2000013a1:0x1:0x0].0x0 bits 0x2/0x0 rrc: 5 type: IBT gid 0 flags: 0x50210000000000 nid: local remote: 0x0 expref: -99 pid: 6255 timeout: 0 lvb_type: 0 00010000:00010000:1.0:1776178096.211796:0:6255:0:(ldlm_request.c:528:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: mdt-lustre-MDT0000_UUID lock: 00000000a44d8583/0xa40083ab9bf52f9f lrc: 3/0,1 mode: CW/CW res: [0x2000013a1:0x1:0x0].0x0 bits 0x2/0x0 rrc: 5 type: IBT gid 0 flags: 0x40210000000000 nid: local remote: 0x0 expref: -99 pid: 6255 timeout: 0 lvb_type: 0 00010000:00010000:1.0:1776178096.211812:0:6255:0:(ldlm_lock.c:766:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: mdt-lustre-MDT0000_UUID lock: 0000000056462e0c/0xa40083ab9bf52fa6 lrc: 3/0,1 mode: --/PW res: [0x2000013a1:0x1:0x0].0xef09e6ea bits 0x0/0x0 rrc: 2 type: IBT gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 6255 timeout: 0 lvb_type: 0 00010000:00010000:1.0:1776178096.211824:0:6255:0:(ldlm_lock.c:1064:ldlm_granted_list_add_lock()) ### About to add lock: ns: mdt-lustre-MDT0000_UUID lock: 0000000056462e0c/0xa40083ab9bf52fa6 lrc: 3/0,1 mode: PW/PW res: [0x2000013a1:0x1:0x0].0xef09e6ea bits 0x2/0x0 rrc: 2 type: IBT gid 0 flags: 0x50210000000000 nid: local remote: 0x0 expref: -99 pid: 6255 timeout: 0 lvb_type: 0 00010000:00010000:1.0:1776178096.211830:0:6255:0:(ldlm_request.c:528:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: mdt-lustre-MDT0000_UUID lock: 0000000056462e0c/0xa40083ab9bf52fa6 lrc: 3/0,1 mode: PW/PW res: [0x2000013a1:0x1:0x0].0xef09e6ea bits 0x2/0x0 rrc: 2 type: IBT gid 0 flags: 0x40210000000000 nid: local remote: 0x0 expref: -99 pid: 6255 timeout: 0 lvb_type: 0 00000004:00080000:1.0:1776178096.212873:0:6255:0:(osp_object.c:1645:osp_create()) lustre-OST0000-osc-MDT0000: Wrote last used FID: [0x100000000:0x44:0x0], index 0: 0 00010000:00010000:1.0:1776178096.214824:0:6255:0:(ldlm_lock.c:766:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(CR) ns: mdt-lustre-MDT0000_UUID lock: 00000000d0f9e56c/0xa40083ab9bf52fad lrc: 3/1,0 mode: --/CR res: [0x2000013a1:0x6:0x0].0x0 bits 0x0/0x0 rrc: 2 type: IBT gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 6255 timeout: 0 lvb_type: 0 00010000:00010000:1.0:1776178096.214857:0:6255:0:(ldlm_lock.c:1064:ldlm_granted_list_add_lock()) ### About to add lock: ns: mdt-lustre-MDT0000_UUID lock: 00000000d0f9e56c/0xa40083ab9bf52fad lrc: 3/1,0 mode: CR/CR res: [0x2000013a1:0x6:0x0].0x0 bits 0x9/0x0 rrc: 2 type: IBT gid 0 flags: 0x50210000000000 nid: local remote: 0x0 expref: -99 pid: 6255 timeout: 0 lvb_type: 0 00010000:00010000:1.0:1776178096.214875:0:6255:0:(ldlm_request.c:528:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: mdt-lustre-MDT0000_UUID lock: 00000000d0f9e56c/0xa40083ab9bf52fad lrc: 3/1,0 mode: CR/CR res: [0x2000013a1:0x6:0x0].0x0 bits 0x9/0x0 rrc: 2 type: IBT gid 0 flags: 0x40210000000000 nid: local remote: 0x0 expref: -99 pid: 6255 timeout: 0 lvb_type: 0 00010000:00010000:1.0:1776178096.214945:0:6255:0:(ldlm_lock.c:820:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(CW) ns: mdt-lustre-MDT0000_UUID lock: 00000000a44d8583/0xa40083ab9bf52f9f lrc: 3/0,1 mode: CW/CW res: [0x2000013a1:0x1:0x0].0x0 bits 0x2/0x0 rrc: 5 type: IBT gid 0 flags: 0x50210000000000 nid: local remote: 0x0 expref: -99 pid: 6255 timeout: 0 lvb_type: 0 00010000:00010000:1.0:1776178096.214953:0:6255:0:(ldlm_lock.c:879:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: mdt-lustre-MDT0000_UUID lock: 00000000a44d8583/0xa40083ab9bf52f9f lrc: 2/0,0 mode: CW/CW res: [0x2000013a1:0x1:0x0].0x0 bits 0x2/0x0 rrc: 5 type: IBT gid 0 flags: 0x50210400000000 nid: local remote: 0x0 expref: -99 pid: 6255 timeout: 0 lvb_type: 0 00010000:00010000:1.0:1776178096.214961:0:6255:0:(ldlm_lockd.c:1879:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: mdt-lustre-MDT0000_UUID lock: 00000000a44d8583/0xa40083ab9bf52f9f lrc: 3/0,0 mode: CW/CW res: [0x2000013a1:0x1:0x0].0x0 bits 0x2/0x0 rrc: 5 type: IBT gid 0 flags: 0x40210400000000 nid: local remote: 0x0 expref: -99 pid: 6255 timeout: 0 lvb_type: 0 00010000:00010000:1.0:1776178096.214968:0:6255:0:(ldlm_lockd.c:1893:ldlm_handle_bl_callback()) Lock 00000000a44d8583 already unused, calling callback (0000000079e78348) 00010000:00010000:1.0:1776178096.214973:0:6255:0:(ldlm_request.c:364:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: mdt-lustre-MDT0000_UUID lock: 00000000a44d8583/0xa40083ab9bf52f9f lrc: 3/0,0 mode: CW/CW res: [0x2000013a1:0x1:0x0].0x0 bits 0x2/0x0 rrc: 5 type: IBT gid 0 flags: 0x40210400000000 nid: local remote: 0x0 expref: -99 pid: 6255 timeout: 0 lvb_type: 0 00010000:00010000:1.0:1776178096.214990:0:6255:0:(ldlm_request.c:1257:ldlm_cli_cancel_local()) ### server-side local cancel ns: mdt-lustre-MDT0000_UUID lock: 00000000a44d8583/0xa40083ab9bf52f9f lrc: 4/0,0 mode: CW/CW res: [0x2000013a1:0x1:0x0].0x0 bits 0x2/0x0 rrc: 5 type: IBT gid 0 flags: 0x40218400000000 nid: local remote: 0x0 expref: -99 pid: 6255 timeout: 0 lvb_type: 0 00010000:00010000:1.0:1776178096.215007:0:6255:0:(ldlm_inodebits.c:94:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x2000013a1:0x1:0x0].0x0 (00000000315facbe) 00010000:00010000:1.0:1776178096.215011:0:6255:0:(ldlm_inodebits.c:96:ldlm_reprocess_inodebits_queue()) Hint 2 00010000:00010000:1.0:1776178096.215017:0:6255:0:(ldlm_lockd.c:1903:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: mdt-lustre-MDT0000_UUID lock: 00000000a44d8583/0xa40083ab9bf52f9f lrc: 2/0,0 mode: --/CW res: [0x2000013a1:0x1:0x0].0x0 bits 0x2/0x0 rrc: 5 type: IBT gid 0 flags: 0x44a19400000000 nid: local remote: 0x0 expref: -99 pid: 6255 timeout: 0 lvb_type: 0 00010000:00010000:1.0:1776178096.215025:0:6255:0:(ldlm_lock.c:216:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: mdt-lustre-MDT0000_UUID lock: 00000000a44d8583/0xa40083ab9bf52f9f lrc: 0/0,0 mode: --/CW res: [0x2000013a1:0x1:0x0].0x0 bits 0x2/0x0 rrc: 5 type: IBT gid 0 flags: 0x44a19400000000 nid: local remote: 0x0 expref: -99 pid: 6255 timeout: 0 lvb_type: 0 00000004:00010000:1.0:1776178096.216572:0:6255:0:(mdt_handler.c:3992:mdt_save_lock()) ### save lock request 00000000c8a61992 reply state 00000000589c1956 transno 12884901915 ns: mdt-lustre-MDT0000_UUID lock: 0000000056462e0c/0xa40083ab9bf52fa6 lrc: 3/0,1 mode: PW/PW res: [0x2000013a1:0x1:0x0].0xef09e6ea bits 0x2/0x0 rrc: 2 type: IBT gid 0 flags: 0x40210000000000 nid: local remote: 0x0 expref: -99 pid: 6255 timeout: 0 lvb_type: 0 00010000:00010000:1.0:1776178096.216798:0:6255:0:(ldlm_lock.c:216:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: mdt-lustre-MDT0000_UUID lock: 0000000097975a3b/0xa40083ab9bf52f98 lrc: 0/0,0 mode: --/CW res: [0x2000013a1:0x1:0x0].0x0 bits 0x1/0x0 rrc: 4 type: IBT gid 0 flags: 0x44000000000000 nid: 192.168.204.3@tcp remote: 0x771f68aa1eb27990 expref: 40 pid: 6255 timeout: 0 lvb_type: 0 00010000:00010000:1.0:1776178096.216827:0:6255:0:(ldlm_lockd.c:1484:ldlm_handle_enqueue0()) ### server-side enqueue handler, sending reply (err=0, rc=0) ns: mdt-lustre-MDT0000_UUID lock: 00000000d0f9e56c/0xa40083ab9bf52fad lrc: 3/0,0 mode: CR/CR res: [0x2000013a1:0x6:0x0].0x0 bits 0x9/0x0 rrc: 2 type: IBT gid 0 flags: 0x40200000000000 nid: 192.168.204.3@tcp remote: 0x771f68aa1eb27990 expref: 39 pid: 6255 timeout: 0 lvb_type: 0 00010000:00010000:1.0:1776178096.216837:0:6255:0:(ldlm_inodebits.c:94:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x2000013a1:0x6:0x0].0x0 (00000000ef7185e3) 00010000:00010000:1.0:1776178096.216839:0:6255:0:(ldlm_inodebits.c:96:ldlm_reprocess_inodebits_queue()) Hint 9 00010000:00010000:1.0:1776178096.216843:0:6255:0:(ldlm_lockd.c:1565:ldlm_handle_enqueue0()) ### server-side enqueue handler END (lock 00000000d0f9e56c, rc 0) 00000100:00100000:1.0:1776178096.217011:0:6255:0:(service.c:2372:ptlrpc_server_handle_request()) Handled RPC req@00000000c8a61992 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_000:b50963bd-8eba-4432-9a54-8888d00e3fa1+40:16992:x1862456979078464:12345-192.168.204.3@tcp:101:touch.0 Request processed in 5757us (6114us total) trans 12884901915 rc 0/0 00000100:00100000:1.0:1776178096.217023:0:6255:0:(nrs_fifo.c:241:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.204.3@tcp, seq: 6471 00000100:00100000:2.0:1776178096.224661:0:3346:0:(events.c:361:request_in_callback()) peer: 12345-192.168.204.3@tcp (source: 12345-192.168.204.3@tcp) 00000100:00100000:1.0:1776178096.224836:0:6255:0:(service.c:2152:ptlrpc_server_handle_req_in()) got req x1862456979078592 00000100:00100000:1.0:1776178096.224924:0:6255:0:(nrs_fifo.c:179:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.204.3@tcp, seq: 6472 00000100:00100000:1.0:1776178096.224935:0:6255:0:(service.c:2324:ptlrpc_server_handle_request()) Handling RPC req@000000001c8035b1 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_000:b50963bd-8eba-4432-9a54-8888d00e3fa1+40:16992:x1862456979078592:12345-192.168.204.3@tcp:36:touch.0 00010000:00010000:1.0:1776178096.225131:0:6255:0:(ldlm_lock.c:766:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: mdt-lustre-MDT0000_UUID lock: 0000000097975a3b/0xa40083ab9bf52fbb lrc: 3/0,1 mode: --/PW res: [0x2000013a1:0x6:0x0].0x0 bits 0x0/0x0 rrc: 3 type: IBT gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 6255 timeout: 0 lvb_type: 0 00010000:00010000:1.0:1776178096.225155:0:6255:0:(ldlm_lock.c:1064:ldlm_granted_list_add_lock()) ### About to add lock: ns: mdt-lustre-MDT0000_UUID lock: 0000000097975a3b/0xa40083ab9bf52fbb lrc: 3/0,1 mode: PW/PW res: [0x2000013a1:0x6:0x0].0x0 bits 0x2/0x0 rrc: 3 type: IBT gid 0 flags: 0x50210000000000 nid: local remote: 0x0 expref: -99 pid: 6255 timeout: 0 lvb_type: 0 00010000:00010000:1.0:1776178096.225169:0:6255:0:(ldlm_request.c:528:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: mdt-lustre-MDT0000_UUID lock: 0000000097975a3b/0xa40083ab9bf52fbb lrc: 3/0,1 mode: PW/PW res: [0x2000013a1:0x6:0x0].0x0 bits 0x2/0x0 rrc: 3 type: IBT gid 0 flags: 0x40210000000000 nid: local remote: 0x0 expref: -99 pid: 6255 timeout: 0 lvb_type: 0 00000004:00010000:1.0:1776178096.225346:0:6255:0:(mdt_handler.c:3992:mdt_save_lock()) ### save lock request 000000001c8035b1 reply state 00000000415e91ab transno 12884901916 ns: mdt-lustre-MDT0000_UUID lock: 0000000097975a3b/0xa40083ab9bf52fbb lrc: 3/0,1 mode: PW/PW res: [0x2000013a1:0x6:0x0].0x0 bits 0x2/0x0 rrc: 3 type: IBT gid 0 flags: 0x40210000000000 nid: local remote: 0x0 expref: -99 pid: 6255 timeout: 0 lvb_type: 0 00000100:00100000:1.0:1776178096.225475:0:6255:0:(service.c:2372:ptlrpc_server_handle_request()) Handled RPC req@000000001c8035b1 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_000:b50963bd-8eba-4432-9a54-8888d00e3fa1+42:16992:x1862456979078592:12345-192.168.204.3@tcp:36:touch.0 Request processed in 539us (839us total) trans 12884901916 rc 0/0 00000100:00100000:1.0:1776178096.225486:0:6255:0:(nrs_fifo.c:241:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.204.3@tcp, seq: 6472 00010000:00010000:0.0:1776178096.225518:0:3339:0:(ldlm_lock.c:1064:ldlm_granted_list_add_lock()) ### About to add lock: ns: mdt-lustre-MDT0000_UUID lock: 0000000056462e0c/0xa40083ab9bf52fa6 lrc: 3/0,1 mode: COS/COS res: [0x2000013a1:0x1:0x0].0xef09e6ea bits 0x2/0x0 rrc: 2 type: IBT gid 0 flags: 0x50210000000000 nid: local remote: 0x0 expref: -99 pid: 6255 timeout: 0 lvb_type: 0 00010000:00010000:0.0:1776178096.225533:0:3339:0:(ldlm_inodebits.c:94:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x2000013a1:0x1:0x0].0xef09e6ea (0000000090c54b62) 00010000:00010000:0.0:1776178096.225536:0:3339:0:(ldlm_inodebits.c:96:ldlm_reprocess_inodebits_queue()) Hint 2 00000100:00100000:0.0:1776178096.230747:0:3348:0:(events.c:361:request_in_callback()) peer: 12345-192.168.204.3@tcp (source: 12345-192.168.204.3@tcp) 00000100:00100000:2.0:1776178096.231407:0:20762:0:(service.c:2152:ptlrpc_server_handle_req_in()) got req x1862456979078656 00000100:00100000:2.0:1776178096.231490:0:20762:0:(nrs_fifo.c:179:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.204.3@tcp, seq: 5233 00000100:00100000:2.0:1776178096.231501:0:20762:0:(service.c:2324:ptlrpc_server_handle_request()) Handling RPC req@0000000023c7374e pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_013:b50963bd-8eba-4432-9a54-8888d00e3fa1+7:2270:x1862456979078656:12345-192.168.204.3@tcp:2:touch.0 00000100:00100000:2.0:1776178096.231866:0:20762:0:(service.c:2372:ptlrpc_server_handle_request()) Handled RPC req@0000000023c7374e pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_013:b50963bd-8eba-4432-9a54-8888d00e3fa1+8:2270:x1862456979078656:12345-192.168.204.3@tcp:2:touch.0 Request processed in 364us (1144us total) trans 4294969343 rc 0/0 00000100:00100000:2.0:1776178096.231877:0:20762:0:(nrs_fifo.c:241:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.204.3@tcp, seq: 5233 00000100:00100000:2.0:1776178096.240475:0:6259:0:(service.c:2152:ptlrpc_server_handle_req_in()) got req x1862456979078784 00000100:00100000:2.0:1776178096.240562:0:6259:0:(nrs_fifo.c:179:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.204.3@tcp, seq: 2127 00000100:00100000:2.0:1776178096.240571:0:6259:0:(service.c:2324:ptlrpc_server_handle_request()) Handling RPC req@000000007fbc2285 pname:cluuid+ref:pid:xid:nid:opc:job mdt_rdpg00_001:b50963bd-8eba-4432-9a54-8888d00e3fa1+42:16992:x1862456979078784:12345-192.168.204.3@tcp:35:touch.0 00000100:00100000:2.0:1776178096.241095:0:6259:0:(service.c:2372:ptlrpc_server_handle_request()) Handled RPC req@000000007fbc2285 pname:cluuid+ref:pid:xid:nid:opc:job mdt_rdpg00_001:b50963bd-8eba-4432-9a54-8888d00e3fa1+43:16992:x1862456979078784:12345-192.168.204.3@tcp:35:touch.0 Request processed in 522us (1044us total) trans 12884901917 rc 0/0 00000100:00100000:2.0:1776178096.241106:0:6259:0:(nrs_fifo.c:241:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.204.3@tcp, seq: 2127 00000100:00100000:2.0:1776178096.271140:0:6256:0:(service.c:2152:ptlrpc_server_handle_req_in()) got req x1862456979078912 00000100:00100000:2.0:1776178096.271241:0:6256:0:(nrs_fifo.c:179:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.204.3@tcp, seq: 6473 00000100:00100000:2.0:1776178096.271261:0:6256:0:(service.c:2324:ptlrpc_server_handle_request()) Handling RPC req@0000000091430ae9 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_001:b50963bd-8eba-4432-9a54-8888d00e3fa1+43:16993:x1862456979078912:12345-192.168.204.3@tcp:101:touch.0 00010000:00010000:2.0:1776178096.271309:0:6256:0:(ldlm_lockd.c:1248:ldlm_handle_enqueue0()) ### server-side enqueue handler START 00010000:00010000:2.0:1776178096.271352:0:6256:0:(ldlm_lockd.c:1318:ldlm_handle_enqueue0()) ### server-side enqueue handler, new lock created ns: mdt-lustre-MDT0000_UUID lock: 000000000396420a/0xa40083ab9bf52fc2 lrc: 2/0,0 mode: --/CW res: [0x2000013a1:0x1:0x0].0x0 bits 0x0/0x0 rrc: 4 type: IBT gid 0 flags: 0x40000000000000 nid: local remote: 0x771f68aa1eb27997 expref: -99 pid: 6256 timeout: 0 lvb_type: 0 00010000:00010000:2.0:1776178096.271930:0:6256:0:(ldlm_lock.c:766:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(CW) ns: mdt-lustre-MDT0000_UUID lock: 00000000fcb32840/0xa40083ab9bf52fc9 lrc: 3/0,1 mode: --/CW res: [0x2000013a1:0x1:0x0].0x0 bits 0x0/0x0 rrc: 5 type: IBT gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 6256 timeout: 0 lvb_type: 0 00010000:00010000:2.0:1776178096.271953:0:6256:0:(ldlm_lock.c:1064:ldlm_granted_list_add_lock()) ### About to add lock: ns: mdt-lustre-MDT0000_UUID lock: 00000000fcb32840/0xa40083ab9bf52fc9 lrc: 3/0,1 mode: CW/CW res: [0x2000013a1:0x1:0x0].0x0 bits 0x2/0x0 rrc: 5 type: IBT gid 0 flags: 0x50210000000000 nid: local remote: 0x0 expref: -99 pid: 6256 timeout: 0 lvb_type: 0 00010000:00010000:2.0:1776178096.271968:0:6256:0:(ldlm_request.c:528:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: mdt-lustre-MDT0000_UUID lock: 00000000fcb32840/0xa40083ab9bf52fc9 lrc: 3/0,1 mode: CW/CW res: [0x2000013a1:0x1:0x0].0x0 bits 0x2/0x0 rrc: 5 type: IBT gid 0 flags: 0x40210000000000 nid: local remote: 0x0 expref: -99 pid: 6256 timeout: 0 lvb_type: 0 00010000:00010000:2.0:1776178096.271987:0:6256:0:(ldlm_lock.c:766:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: mdt-lustre-MDT0000_UUID lock: 000000000b5c0143/0xa40083ab9bf52fd0 lrc: 3/0,1 mode: --/PW res: [0x2000013a1:0x1:0x0].0x37639fea bits 0x0/0x0 rrc: 2 type: IBT gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 6256 timeout: 0 lvb_type: 0 00010000:00010000:2.0:1776178096.271995:0:6256:0:(ldlm_lock.c:1064:ldlm_granted_list_add_lock()) ### About to add lock: ns: mdt-lustre-MDT0000_UUID lock: 000000000b5c0143/0xa40083ab9bf52fd0 lrc: 3/0,1 mode: PW/PW res: [0x2000013a1:0x1:0x0].0x37639fea bits 0x2/0x0 rrc: 2 type: IBT gid 0 flags: 0x50210000000000 nid: local remote: 0x0 expref: -99 pid: 6256 timeout: 0 lvb_type: 0 00010000:00010000:2.0:1776178096.272002:0:6256:0:(ldlm_request.c:528:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: mdt-lustre-MDT0000_UUID lock: 000000000b5c0143/0xa40083ab9bf52fd0 lrc: 3/0,1 mode: PW/PW res: [0x2000013a1:0x1:0x0].0x37639fea bits 0x2/0x0 rrc: 2 type: IBT gid 0 flags: 0x40210000000000 nid: local remote: 0x0 expref: -99 pid: 6256 timeout: 0 lvb_type: 0 00000004:00080000:2.0:1776178096.275515:0:6256:0:(osp_object.c:1645:osp_create()) lustre-OST0001-osc-MDT0000: Wrote last used FID: [0x100010000:0x44:0x0], index 1: 0 00010000:00010000:2.0:1776178096.275803:0:6256:0:(ldlm_lock.c:766:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(CR) ns: mdt-lustre-MDT0000_UUID lock: 000000007e7ea26f/0xa40083ab9bf52fd7 lrc: 3/1,0 mode: --/CR res: [0x2000013a1:0x7:0x0].0x0 bits 0x0/0x0 rrc: 2 type: IBT gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 6256 timeout: 0 lvb_type: 0 00010000:00010000:2.0:1776178096.275834:0:6256:0:(ldlm_lock.c:1064:ldlm_granted_list_add_lock()) ### About to add lock: ns: mdt-lustre-MDT0000_UUID lock: 000000007e7ea26f/0xa40083ab9bf52fd7 lrc: 3/1,0 mode: CR/CR res: [0x2000013a1:0x7:0x0].0x0 bits 0x9/0x0 rrc: 2 type: IBT gid 0 flags: 0x50210000000000 nid: local remote: 0x0 expref: -99 pid: 6256 timeout: 0 lvb_type: 0 00010000:00010000:2.0:1776178096.275853:0:6256:0:(ldlm_request.c:528:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: mdt-lustre-MDT0000_UUID lock: 000000007e7ea26f/0xa40083ab9bf52fd7 lrc: 3/1,0 mode: CR/CR res: [0x2000013a1:0x7:0x0].0x0 bits 0x9/0x0 rrc: 2 type: IBT gid 0 flags: 0x40210000000000 nid: local remote: 0x0 expref: -99 pid: 6256 timeout: 0 lvb_type: 0 00010000:00010000:2.0:1776178096.275923:0:6256:0:(ldlm_lock.c:820:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(CW) ns: mdt-lustre-MDT0000_UUID lock: 00000000fcb32840/0xa40083ab9bf52fc9 lrc: 3/0,1 mode: CW/CW res: [0x2000013a1:0x1:0x0].0x0 bits 0x2/0x0 rrc: 5 type: IBT gid 0 flags: 0x50210000000000 nid: local remote: 0x0 expref: -99 pid: 6256 timeout: 0 lvb_type: 0 00010000:00010000:2.0:1776178096.275934:0:6256:0:(ldlm_lock.c:879:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: mdt-lustre-MDT0000_UUID lock: 00000000fcb32840/0xa40083ab9bf52fc9 lrc: 2/0,0 mode: CW/CW res: [0x2000013a1:0x1:0x0].0x0 bits 0x2/0x0 rrc: 5 type: IBT gid 0 flags: 0x50210400000000 nid: local remote: 0x0 expref: -99 pid: 6256 timeout: 0 lvb_type: 0 00010000:00010000:2.0:1776178096.275943:0:6256:0:(ldlm_lockd.c:1879:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: mdt-lustre-MDT0000_UUID lock: 00000000fcb32840/0xa40083ab9bf52fc9 lrc: 3/0,0 mode: CW/CW res: [0x2000013a1:0x1:0x0].0x0 bits 0x2/0x0 rrc: 5 type: IBT gid 0 flags: 0x40210400000000 nid: local remote: 0x0 expref: -99 pid: 6256 timeout: 0 lvb_type: 0 00010000:00010000:2.0:1776178096.275951:0:6256:0:(ldlm_lockd.c:1893:ldlm_handle_bl_callback()) Lock 00000000fcb32840 already unused, calling callback (0000000079e78348) 00010000:00010000:2.0:1776178096.275957:0:6256:0:(ldlm_request.c:364:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: mdt-lustre-MDT0000_UUID lock: 00000000fcb32840/0xa40083ab9bf52fc9 lrc: 3/0,0 mode: CW/CW res: [0x2000013a1:0x1:0x0].0x0 bits 0x2/0x0 rrc: 5 type: IBT gid 0 flags: 0x40210400000000 nid: local remote: 0x0 expref: -99 pid: 6256 timeout: 0 lvb_type: 0 00010000:00010000:2.0:1776178096.275966:0:6256:0:(ldlm_request.c:1257:ldlm_cli_cancel_local()) ### server-side local cancel ns: mdt-lustre-MDT0000_UUID lock: 00000000fcb32840/0xa40083ab9bf52fc9 lrc: 4/0,0 mode: CW/CW res: [0x2000013a1:0x1:0x0].0x0 bits 0x2/0x0 rrc: 5 type: IBT gid 0 flags: 0x40218400000000 nid: local remote: 0x0 expref: -99 pid: 6256 timeout: 0 lvb_type: 0 00010000:00010000:2.0:1776178096.275984:0:6256:0:(ldlm_inodebits.c:94:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x2000013a1:0x1:0x0].0x0 (00000000315facbe) 00010000:00010000:2.0:1776178096.275988:0:6256:0:(ldlm_inodebits.c:96:ldlm_reprocess_inodebits_queue()) Hint 2 00010000:00010000:2.0:1776178096.275994:0:6256:0:(ldlm_lockd.c:1903:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: mdt-lustre-MDT0000_UUID lock: 00000000fcb32840/0xa40083ab9bf52fc9 lrc: 2/0,0 mode: --/CW res: [0x2000013a1:0x1:0x0].0x0 bits 0x2/0x0 rrc: 5 type: IBT gid 0 flags: 0x44a19400000000 nid: local remote: 0x0 expref: -99 pid: 6256 timeout: 0 lvb_type: 0 00010000:00010000:2.0:1776178096.276001:0:6256:0:(ldlm_lock.c:216:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: mdt-lustre-MDT0000_UUID lock: 00000000fcb32840/0xa40083ab9bf52fc9 lrc: 0/0,0 mode: --/CW res: [0x2000013a1:0x1:0x0].0x0 bits 0x2/0x0 rrc: 5 type: IBT gid 0 flags: 0x44a19400000000 nid: local remote: 0x0 expref: -99 pid: 6256 timeout: 0 lvb_type: 0 00000004:00010000:2.0:1776178096.276019:0:6256:0:(mdt_handler.c:3992:mdt_save_lock()) ### save lock request 0000000091430ae9 reply state 00000000b441b66d transno 12884901918 ns: mdt-lustre-MDT0000_UUID lock: 000000000b5c0143/0xa40083ab9bf52fd0 lrc: 3/0,1 mode: PW/PW res: [0x2000013a1:0x1:0x0].0x37639fea bits 0x2/0x0 rrc: 2 type: IBT gid 0 flags: 0x40210000000000 nid: local remote: 0x0 expref: -99 pid: 6256 timeout: 0 lvb_type: 0 00010000:00010000:2.0:1776178096.276143:0:6256:0:(ldlm_lock.c:216:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: mdt-lustre-MDT0000_UUID lock: 000000000396420a/0xa40083ab9bf52fc2 lrc: 0/0,0 mode: --/CW res: [0x2000013a1:0x1:0x0].0x0 bits 0x1/0x0 rrc: 4 type: IBT gid 0 flags: 0x44000000000000 nid: 192.168.204.3@tcp remote: 0x771f68aa1eb27997 expref: 46 pid: 6256 timeout: 0 lvb_type: 0 00010000:00010000:2.0:1776178096.276166:0:6256:0:(ldlm_lockd.c:1484:ldlm_handle_enqueue0()) ### server-side enqueue handler, sending reply (err=0, rc=0) ns: mdt-lustre-MDT0000_UUID lock: 000000007e7ea26f/0xa40083ab9bf52fd7 lrc: 3/0,0 mode: CR/CR res: [0x2000013a1:0x7:0x0].0x0 bits 0x9/0x0 rrc: 2 type: IBT gid 0 flags: 0x40200000000000 nid: 192.168.204.3@tcp remote: 0x771f68aa1eb27997 expref: 45 pid: 6256 timeout: 0 lvb_type: 0 00010000:00010000:2.0:1776178096.276174:0:6256:0:(ldlm_inodebits.c:94:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x2000013a1:0x7:0x0].0x0 (00000000665a3017) 00010000:00010000:2.0:1776178096.276177:0:6256:0:(ldlm_inodebits.c:96:ldlm_reprocess_inodebits_queue()) Hint 9 00010000:00010000:2.0:1776178096.276179:0:6256:0:(ldlm_lockd.c:1565:ldlm_handle_enqueue0()) ### server-side enqueue handler END (lock 000000007e7ea26f, rc 0) 00000100:00100000:2.0:1776178096.276313:0:6256:0:(service.c:2372:ptlrpc_server_handle_request()) Handled RPC req@0000000091430ae9 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_001:b50963bd-8eba-4432-9a54-8888d00e3fa1+46:16993:x1862456979078912:12345-192.168.204.3@tcp:101:touch.0 Request processed in 5058us (5526us total) trans 12884901918 rc 0/0 00000100:00100000:2.0:1776178096.276327:0:6256:0:(nrs_fifo.c:241:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.204.3@tcp, seq: 6473 00010000:00010000:1.0:1776178096.285958:0:3339:0:(ldlm_lock.c:1064:ldlm_granted_list_add_lock()) ### About to add lock: ns: mdt-lustre-MDT0000_UUID lock: 000000000b5c0143/0xa40083ab9bf52fd0 lrc: 3/0,1 mode: COS/COS res: [0x2000013a1:0x1:0x0].0x37639fea bits 0x2/0x0 rrc: 2 type: IBT gid 0 flags: 0x50210000000000 nid: local remote: 0x0 expref: -99 pid: 6256 timeout: 0 lvb_type: 0 00010000:00010000:1.0:1776178096.285996:0:3339:0:(ldlm_inodebits.c:94:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x2000013a1:0x1:0x0].0x37639fea (00000000e4c83f6d) 00010000:00010000:1.0:1776178096.286000:0:3339:0:(ldlm_inodebits.c:96:ldlm_reprocess_inodebits_queue()) Hint 2 00000100:00100000:2.0:1776178096.292542:0:6256:0:(service.c:2152:ptlrpc_server_handle_req_in()) got req x1862456979079040 00000100:00100000:0.0:1776178096.292838:0:6256:0:(nrs_fifo.c:179:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.204.3@tcp, seq: 6474 00000100:00100000:0.0:1776178096.292848:0:6256:0:(service.c:2324:ptlrpc_server_handle_request()) Handling RPC req@0000000006fcb74e pname:cluuid+ref:pid:xid:nid:opc:job mdt00_001:b50963bd-8eba-4432-9a54-8888d00e3fa1+46:16993:x1862456979079040:12345-192.168.204.3@tcp:36:touch.0 00010000:00010000:0.0:1776178096.293047:0:6256:0:(ldlm_lock.c:766:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: mdt-lustre-MDT0000_UUID lock: 00000000319537bc/0xa40083ab9bf52fe5 lrc: 3/0,1 mode: --/PW res: [0x2000013a1:0x7:0x0].0x0 bits 0x0/0x0 rrc: 3 type: IBT gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 6256 timeout: 0 lvb_type: 0 00010000:00010000:0.0:1776178096.293123:0:6256:0:(ldlm_lock.c:1064:ldlm_granted_list_add_lock()) ### About to add lock: ns: mdt-lustre-MDT0000_UUID lock: 00000000319537bc/0xa40083ab9bf52fe5 lrc: 3/0,1 mode: PW/PW res: [0x2000013a1:0x7:0x0].0x0 bits 0x2/0x0 rrc: 3 type: IBT gid 0 flags: 0x50210000000000 nid: local remote: 0x0 expref: -99 pid: 6256 timeout: 0 lvb_type: 0 00010000:00010000:0.0:1776178096.293142:0:6256:0:(ldlm_request.c:528:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: mdt-lustre-MDT0000_UUID lock: 00000000319537bc/0xa40083ab9bf52fe5 lrc: 3/0,1 mode: PW/PW res: [0x2000013a1:0x7:0x0].0x0 bits 0x2/0x0 rrc: 3 type: IBT gid 0 flags: 0x40210000000000 nid: local remote: 0x0 expref: -99 pid: 6256 timeout: 0 lvb_type: 0 00000004:00010000:0.0:1776178096.293345:0:6256:0:(mdt_handler.c:3992:mdt_save_lock()) ### save lock request 0000000006fcb74e reply state 000000008b5d85d6 transno 12884901919 ns: mdt-lustre-MDT0000_UUID lock: 00000000319537bc/0xa40083ab9bf52fe5 lrc: 3/0,1 mode: PW/PW res: [0x2000013a1:0x7:0x0].0x0 bits 0x2/0x0 rrc: 3 type: IBT gid 0 flags: 0x40210000000000 nid: local remote: 0x0 expref: -99 pid: 6256 timeout: 0 lvb_type: 0 00000100:00100000:0.0:1776178096.293518:0:6256:0:(service.c:2372:ptlrpc_server_handle_request()) Handled RPC req@0000000006fcb74e pname:cluuid+ref:pid:xid:nid:opc:job mdt00_001:b50963bd-8eba-4432-9a54-8888d00e3fa1+48:16993:x1862456979079040:12345-192.168.204.3@tcp:36:touch.0 Request processed in 668us (4061us total) trans 12884901919 rc 0/0 00000100:00100000:0.0:1776178096.293529:0:6256:0:(nrs_fifo.c:241:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.204.3@tcp, seq: 6474 00000100:00100000:1.0:1776178096.306654:0:14440:0:(service.c:2152:ptlrpc_server_handle_req_in()) got req x1862456979079104 00000100:00100000:1.0:1776178096.306730:0:14440:0:(nrs_fifo.c:179:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.204.3@tcp, seq: 5234 00000100:00100000:1.0:1776178096.306737:0:14440:0:(service.c:2324:ptlrpc_server_handle_request()) Handling RPC req@00000000c32cfd7b pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_004:b50963bd-8eba-4432-9a54-8888d00e3fa1+7:2271:x1862456979079104:12345-192.168.204.3@tcp:2:touch.0 00000100:00100000:1.0:1776178096.307036:0:14440:0:(service.c:2372:ptlrpc_server_handle_request()) Handled RPC req@00000000c32cfd7b pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_004:b50963bd-8eba-4432-9a54-8888d00e3fa1+8:2271:x1862456979079104:12345-192.168.204.3@tcp:2:touch.0 Request processed in 298us (769us total) trans 4294969317 rc 0/0 00000100:00100000:1.0:1776178096.307046:0:14440:0:(nrs_fifo.c:241:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.204.3@tcp, seq: 5234 00000100:00100000:2.0:1776178096.310525:0:3347:0:(events.c:361:request_in_callback()) peer: 12345-192.168.204.3@tcp (source: 12345-192.168.204.3@tcp) 00000100:00100000:1.0:1776178096.310710:0:6258:0:(service.c:2152:ptlrpc_server_handle_req_in()) got req x1862456979079232 00000100:00100000:1.0:1776178096.310923:0:6258:0:(nrs_fifo.c:179:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.204.3@tcp, seq: 2128 00000100:00100000:1.0:1776178096.310932:0:6258:0:(service.c:2324:ptlrpc_server_handle_request()) Handling RPC req@000000004a209131 pname:cluuid+ref:pid:xid:nid:opc:job mdt_rdpg00_000:b50963bd-8eba-4432-9a54-8888d00e3fa1+48:16993:x1862456979079232:12345-192.168.204.3@tcp:35:touch.0 00000100:00100000:1.0:1776178096.311708:0:6258:0:(service.c:2372:ptlrpc_server_handle_request()) Handled RPC req@000000004a209131 pname:cluuid+ref:pid:xid:nid:opc:job mdt_rdpg00_000:b50963bd-8eba-4432-9a54-8888d00e3fa1+49:16993:x1862456979079232:12345-192.168.204.3@tcp:35:touch.0 Request processed in 773us (1193us total) trans 12884901920 rc 0/0 00000100:00100000:1.0:1776178096.311719:0:6258:0:(nrs_fifo.c:241:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.204.3@tcp, seq: 2128 00000100:00100000:0.0:1776178096.327352:0:6256:0:(service.c:2152:ptlrpc_server_handle_req_in()) got req x1862456979079360 00000100:00100000:0.0:1776178096.327435:0:6256:0:(nrs_fifo.c:179:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.204.3@tcp, seq: 6475 00000100:00100000:0.0:1776178096.327444:0:6256:0:(service.c:2324:ptlrpc_server_handle_request()) Handling RPC req@00000000ebe25083 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_001:b50963bd-8eba-4432-9a54-8888d00e3fa1+49:16994:x1862456979079360:12345-192.168.204.3@tcp:101:touch.0 00010000:00010000:0.0:1776178096.327481:0:6256:0:(ldlm_lockd.c:1248:ldlm_handle_enqueue0()) ### server-side enqueue handler START 00010000:00010000:0.0:1776178096.327511:0:6256:0:(ldlm_lockd.c:1318:ldlm_handle_enqueue0()) ### server-side enqueue handler, new lock created ns: mdt-lustre-MDT0000_UUID lock: 000000004bd1d7cb/0xa40083ab9bf52fec lrc: 2/0,0 mode: --/CW res: [0x2000013a1:0x1:0x0].0x0 bits 0x0/0x0 rrc: 4 type: IBT gid 0 flags: 0x40000000000000 nid: local remote: 0x771f68aa1eb2799e expref: -99 pid: 6256 timeout: 0 lvb_type: 0 00010000:00010000:0.0:1776178096.327672:0:6256:0:(ldlm_lock.c:766:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(CW) ns: mdt-lustre-MDT0000_UUID lock: 000000007797f5e2/0xa40083ab9bf52ff3 lrc: 3/0,1 mode: --/CW res: [0x2000013a1:0x1:0x0].0x0 bits 0x0/0x0 rrc: 5 type: IBT gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 6256 timeout: 0 lvb_type: 0 00010000:00010000:0.0:1776178096.327694:0:6256:0:(ldlm_lock.c:1064:ldlm_granted_list_add_lock()) ### About to add lock: ns: mdt-lustre-MDT0000_UUID lock: 000000007797f5e2/0xa40083ab9bf52ff3 lrc: 3/0,1 mode: CW/CW res: [0x2000013a1:0x1:0x0].0x0 bits 0x2/0x0 rrc: 5 type: IBT gid 0 flags: 0x50210000000000 nid: local remote: 0x0 expref: -99 pid: 6256 timeout: 0 lvb_type: 0 00010000:00010000:0.0:1776178096.327710:0:6256:0:(ldlm_request.c:528:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: mdt-lustre-MDT0000_UUID lock: 000000007797f5e2/0xa40083ab9bf52ff3 lrc: 3/0,1 mode: CW/CW res: [0x2000013a1:0x1:0x0].0x0 bits 0x2/0x0 rrc: 5 type: IBT gid 0 flags: 0x40210000000000 nid: local remote: 0x0 expref: -99 pid: 6256 timeout: 0 lvb_type: 0 00010000:00010000:0.0:1776178096.327727:0:6256:0:(ldlm_lock.c:766:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: mdt-lustre-MDT0000_UUID lock: 00000000000cfba0/0xa40083ab9bf52ffa lrc: 3/0,1 mode: --/PW res: [0x2000013a1:0x1:0x0].0x7fbd58ea bits 0x0/0x0 rrc: 2 type: IBT gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 6256 timeout: 0 lvb_type: 0 00010000:00010000:0.0:1776178096.327734:0:6256:0:(ldlm_lock.c:1064:ldlm_granted_list_add_lock()) ### About to add lock: ns: mdt-lustre-MDT0000_UUID lock: 00000000000cfba0/0xa40083ab9bf52ffa lrc: 3/0,1 mode: PW/PW res: [0x2000013a1:0x1:0x0].0x7fbd58ea bits 0x2/0x0 rrc: 2 type: IBT gid 0 flags: 0x50210000000000 nid: local remote: 0x0 expref: -99 pid: 6256 timeout: 0 lvb_type: 0 00010000:00010000:0.0:1776178096.327741:0:6256:0:(ldlm_request.c:528:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: mdt-lustre-MDT0000_UUID lock: 00000000000cfba0/0xa40083ab9bf52ffa lrc: 3/0,1 mode: PW/PW res: [0x2000013a1:0x1:0x0].0x7fbd58ea bits 0x2/0x0 rrc: 2 type: IBT gid 0 flags: 0x40210000000000 nid: local remote: 0x0 expref: -99 pid: 6256 timeout: 0 lvb_type: 0 00000004:00080000:0.0:1776178096.328709:0:6256:0:(osp_object.c:1645:osp_create()) lustre-OST0000-osc-MDT0000: Wrote last used FID: [0x100000000:0x45:0x0], index 0: 0 00010000:00010000:0.0:1776178096.328930:0:6256:0:(ldlm_lock.c:766:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(CR) ns: mdt-lustre-MDT0000_UUID lock: 00000000ae88aadd/0xa40083ab9bf53001 lrc: 3/1,0 mode: --/CR res: [0x2000013a1:0x8:0x0].0x0 bits 0x0/0x0 rrc: 2 type: IBT gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 6256 timeout: 0 lvb_type: 0 00010000:00010000:0.0:1776178096.328948:0:6256:0:(ldlm_lock.c:1064:ldlm_granted_list_add_lock()) ### About to add lock: ns: mdt-lustre-MDT0000_UUID lock: 00000000ae88aadd/0xa40083ab9bf53001 lrc: 3/1,0 mode: CR/CR res: [0x2000013a1:0x8:0x0].0x0 bits 0x9/0x0 rrc: 2 type: IBT gid 0 flags: 0x50210000000000 nid: local remote: 0x0 expref: -99 pid: 6256 timeout: 0 lvb_type: 0 00010000:00010000:0.0:1776178096.328960:0:6256:0:(ldlm_request.c:528:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: mdt-lustre-MDT0000_UUID lock: 00000000ae88aadd/0xa40083ab9bf53001 lrc: 3/1,0 mode: CR/CR res: [0x2000013a1:0x8:0x0].0x0 bits 0x9/0x0 rrc: 2 type: IBT gid 0 flags: 0x40210000000000 nid: local remote: 0x0 expref: -99 pid: 6256 timeout: 0 lvb_type: 0 00010000:00010000:0.0:1776178096.329671:0:6256:0:(ldlm_lock.c:820:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(CW) ns: mdt-lustre-MDT0000_UUID lock: 000000007797f5e2/0xa40083ab9bf52ff3 lrc: 3/0,1 mode: CW/CW res: [0x2000013a1:0x1:0x0].0x0 bits 0x2/0x0 rrc: 5 type: IBT gid 0 flags: 0x50210000000000 nid: local remote: 0x0 expref: -99 pid: 6256 timeout: 0 lvb_type: 0 00010000:00010000:0.0:1776178096.329687:0:6256:0:(ldlm_lock.c:879:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: mdt-lustre-MDT0000_UUID lock: 000000007797f5e2/0xa40083ab9bf52ff3 lrc: 2/0,0 mode: CW/CW res: [0x2000013a1:0x1:0x0].0x0 bits 0x2/0x0 rrc: 5 type: IBT gid 0 flags: 0x50210400000000 nid: local remote: 0x0 expref: -99 pid: 6256 timeout: 0 lvb_type: 0 00010000:00010000:0.0:1776178096.329696:0:6256:0:(ldlm_lockd.c:1879:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: mdt-lustre-MDT0000_UUID lock: 000000007797f5e2/0xa40083ab9bf52ff3 lrc: 3/0,0 mode: CW/CW res: [0x2000013a1:0x1:0x0].0x0 bits 0x2/0x0 rrc: 5 type: IBT gid 0 flags: 0x40210400000000 nid: local remote: 0x0 expref: -99 pid: 6256 timeout: 0 lvb_type: 0 00010000:00010000:0.0:1776178096.329704:0:6256:0:(ldlm_lockd.c:1893:ldlm_handle_bl_callback()) Lock 000000007797f5e2 already unused, calling callback (0000000079e78348) 00010000:00010000:0.0:1776178096.329710:0:6256:0:(ldlm_request.c:364:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: mdt-lustre-MDT0000_UUID lock: 000000007797f5e2/0xa40083ab9bf52ff3 lrc: 3/0,0 mode: CW/CW res: [0x2000013a1:0x1:0x0].0x0 bits 0x2/0x0 rrc: 5 type: IBT gid 0 flags: 0x40210400000000 nid: local remote: 0x0 expref: -99 pid: 6256 timeout: 0 lvb_type: 0 00010000:00010000:0.0:1776178096.329719:0:6256:0:(ldlm_request.c:1257:ldlm_cli_cancel_local()) ### server-side local cancel ns: mdt-lustre-MDT0000_UUID lock: 000000007797f5e2/0xa40083ab9bf52ff3 lrc: 4/0,0 mode: CW/CW res: [0x2000013a1:0x1:0x0].0x0 bits 0x2/0x0 rrc: 5 type: IBT gid 0 flags: 0x40218400000000 nid: local remote: 0x0 expref: -99 pid: 6256 timeout: 0 lvb_type: 0 00010000:00010000:0.0:1776178096.329736:0:6256:0:(ldlm_inodebits.c:94:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x2000013a1:0x1:0x0].0x0 (00000000315facbe) 00010000:00010000:0.0:1776178096.329739:0:6256:0:(ldlm_inodebits.c:96:ldlm_reprocess_inodebits_queue()) Hint 2 00010000:00010000:0.0:1776178096.329746:0:6256:0:(ldlm_lockd.c:1903:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: mdt-lustre-MDT0000_UUID lock: 000000007797f5e2/0xa40083ab9bf52ff3 lrc: 2/0,0 mode: --/CW res: [0x2000013a1:0x1:0x0].0x0 bits 0x2/0x0 rrc: 5 type: IBT gid 0 flags: 0x44a19400000000 nid: local remote: 0x0 expref: -99 pid: 6256 timeout: 0 lvb_type: 0 00010000:00010000:0.0:1776178096.329764:0:6256:0:(ldlm_lock.c:216:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: mdt-lustre-MDT0000_UUID lock: 000000007797f5e2/0xa40083ab9bf52ff3 lrc: 0/0,0 mode: --/CW res: [0x2000013a1:0x1:0x0].0x0 bits 0x2/0x0 rrc: 5 type: IBT gid 0 flags: 0x44a19400000000 nid: local remote: 0x0 expref: -99 pid: 6256 timeout: 0 lvb_type: 0 00000004:00010000:0.0:1776178096.329782:0:6256:0:(mdt_handler.c:3992:mdt_save_lock()) ### save lock request 00000000ebe25083 reply state 00000000f45b64bf transno 12884901921 ns: mdt-lustre-MDT0000_UUID lock: 00000000000cfba0/0xa40083ab9bf52ffa lrc: 3/0,1 mode: PW/PW res: [0x2000013a1:0x1:0x0].0x7fbd58ea bits 0x2/0x0 rrc: 2 type: IBT gid 0 flags: 0x40210000000000 nid: local remote: 0x0 expref: -99 pid: 6256 timeout: 0 lvb_type: 0 00010000:00010000:0.0:1776178096.329832:0:6256:0:(ldlm_lock.c:216:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: mdt-lustre-MDT0000_UUID lock: 000000004bd1d7cb/0xa40083ab9bf52fec lrc: 0/0,0 mode: --/CW res: [0x2000013a1:0x1:0x0].0x0 bits 0x1/0x0 rrc: 4 type: IBT gid 0 flags: 0x44000000000000 nid: 192.168.204.3@tcp remote: 0x771f68aa1eb2799e expref: 52 pid: 6256 timeout: 0 lvb_type: 0 00010000:00010000:0.0:1776178096.329846:0:6256:0:(ldlm_lockd.c:1484:ldlm_handle_enqueue0()) ### server-side enqueue handler, sending reply (err=0, rc=0) ns: mdt-lustre-MDT0000_UUID lock: 00000000ae88aadd/0xa40083ab9bf53001 lrc: 3/0,0 mode: CR/CR res: [0x2000013a1:0x8:0x0].0x0 bits 0x9/0x0 rrc: 2 type: IBT gid 0 flags: 0x40200000000000 nid: 192.168.204.3@tcp remote: 0x771f68aa1eb2799e expref: 51 pid: 6256 timeout: 0 lvb_type: 0 00010000:00010000:0.0:1776178096.329853:0:6256:0:(ldlm_inodebits.c:94:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x2000013a1:0x8:0x0].0x0 (00000000e9f0d179) 00010000:00010000:0.0:1776178096.329855:0:6256:0:(ldlm_inodebits.c:96:ldlm_reprocess_inodebits_queue()) Hint 9 00010000:00010000:0.0:1776178096.329857:0:6256:0:(ldlm_lockd.c:1565:ldlm_handle_enqueue0()) ### server-side enqueue handler END (lock 00000000ae88aadd, rc 0) 00000100:00100000:0.0:1776178096.329993:0:6256:0:(service.c:2372:ptlrpc_server_handle_request()) Handled RPC req@00000000ebe25083 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_001:b50963bd-8eba-4432-9a54-8888d00e3fa1+52:16994:x1862456979079360:12345-192.168.204.3@tcp:101:touch.0 Request processed in 2546us (3092us total) trans 12884901921 rc 0/0 00000100:00100000:0.0:1776178096.330016:0:6256:0:(nrs_fifo.c:241:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.204.3@tcp, seq: 6475 00010000:00010000:1.0:1776178096.341750:0:3339:0:(ldlm_lock.c:1064:ldlm_granted_list_add_lock()) ### About to add lock: ns: mdt-lustre-MDT0000_UUID lock: 00000000000cfba0/0xa40083ab9bf52ffa lrc: 3/0,1 mode: COS/COS res: [0x2000013a1:0x1:0x0].0x7fbd58ea bits 0x2/0x0 rrc: 2 type: IBT gid 0 flags: 0x50210000000000 nid: local remote: 0x0 expref: -99 pid: 6256 timeout: 0 lvb_type: 0 00010000:00010000:1.0:1776178096.341793:0:3339:0:(ldlm_inodebits.c:94:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x2000013a1:0x1:0x0].0x7fbd58ea (00000000fdbb109f) 00010000:00010000:1.0:1776178096.341797:0:3339:0:(ldlm_inodebits.c:96:ldlm_reprocess_inodebits_queue()) Hint 2 00000100:00100000:2.0:1776178096.341904:0:3346:0:(events.c:361:request_in_callback()) peer: 12345-192.168.204.3@tcp (source: 12345-192.168.204.3@tcp) 00000100:00100000:1.0:1776178096.342034:0:6256:0:(service.c:2152:ptlrpc_server_handle_req_in()) got req x1862456979079488 00000100:00100000:1.0:1776178096.342153:0:6256:0:(nrs_fifo.c:179:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.204.3@tcp, seq: 6476 00000100:00100000:1.0:1776178096.342163:0:6256:0:(service.c:2324:ptlrpc_server_handle_request()) Handling RPC req@00000000ee27102e pname:cluuid+ref:pid:xid:nid:opc:job mdt00_001:b50963bd-8eba-4432-9a54-8888d00e3fa1+52:16994:x1862456979079488:12345-192.168.204.3@tcp:36:touch.0 00010000:00010000:1.0:1776178096.342386:0:6256:0:(ldlm_lock.c:766:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: mdt-lustre-MDT0000_UUID lock: 00000000a44d8583/0xa40083ab9bf5300f lrc: 3/0,1 mode: --/PW res: [0x2000013a1:0x8:0x0].0x0 bits 0x0/0x0 rrc: 3 type: IBT gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 6256 timeout: 0 lvb_type: 0 00010000:00010000:1.0:1776178096.342410:0:6256:0:(ldlm_lock.c:1064:ldlm_granted_list_add_lock()) ### About to add lock: ns: mdt-lustre-MDT0000_UUID lock: 00000000a44d8583/0xa40083ab9bf5300f lrc: 3/0,1 mode: PW/PW res: [0x2000013a1:0x8:0x0].0x0 bits 0x2/0x0 rrc: 3 type: IBT gid 0 flags: 0x50210000000000 nid: local remote: 0x0 expref: -99 pid: 6256 timeout: 0 lvb_type: 0 00010000:00010000:1.0:1776178096.342424:0:6256:0:(ldlm_request.c:528:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: mdt-lustre-MDT0000_UUID lock: 00000000a44d8583/0xa40083ab9bf5300f lrc: 3/0,1 mode: PW/PW res: [0x2000013a1:0x8:0x0].0x0 bits 0x2/0x0 rrc: 3 type: IBT gid 0 flags: 0x40210000000000 nid: local remote: 0x0 expref: -99 pid: 6256 timeout: 0 lvb_type: 0 00000004:00010000:1.0:1776178096.342648:0:6256:0:(mdt_handler.c:3992:mdt_save_lock()) ### save lock request 00000000ee27102e reply state 000000005f292b68 transno 12884901922 ns: mdt-lustre-MDT0000_UUID lock: 00000000a44d8583/0xa40083ab9bf5300f lrc: 3/0,1 mode: PW/PW res: [0x2000013a1:0x8:0x0].0x0 bits 0x2/0x0 rrc: 3 type: IBT gid 0 flags: 0x40210000000000 nid: local remote: 0x0 expref: -99 pid: 6256 timeout: 0 lvb_type: 0 00000100:00100000:1.0:1776178096.342835:0:6256:0:(service.c:2372:ptlrpc_server_handle_request()) Handled RPC req@00000000ee27102e pname:cluuid+ref:pid:xid:nid:opc:job mdt00_001:b50963bd-8eba-4432-9a54-8888d00e3fa1+54:16994:x1862456979079488:12345-192.168.204.3@tcp:36:touch.0 Request processed in 671us (937us total) trans 12884901922 rc 0/0 00000100:00100000:1.0:1776178096.342845:0:6256:0:(nrs_fifo.c:241:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.204.3@tcp, seq: 6476 00010000:00010000:2.0:1776178096.346702:0:3340:0:(ldlm_lock.c:1064:ldlm_granted_list_add_lock()) ### About to add lock: ns: mdt-lustre-MDT0000_UUID lock: 00000000a44d8583/0xa40083ab9bf5300f lrc: 3/0,1 mode: COS/COS res: [0x2000013a1:0x8:0x0].0x0 bits 0x2/0x0 rrc: 3 type: IBT gid 0 flags: 0x50210000000000 nid: local remote: 0x0 expref: -99 pid: 6256 timeout: 0 lvb_type: 0 00010000:00010000:2.0:1776178096.346729:0:3340:0:(ldlm_inodebits.c:94:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x2000013a1:0x8:0x0].0x0 (00000000e9f0d179) 00010000:00010000:2.0:1776178096.346732:0:3340:0:(ldlm_inodebits.c:96:ldlm_reprocess_inodebits_queue()) Hint 2 00000100:00100000:1.0:1776178096.346811:0:14440:0:(service.c:2152:ptlrpc_server_handle_req_in()) got req x1862456979079552 00000100:00100000:1.0:1776178096.346891:0:14440:0:(nrs_fifo.c:179:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.204.3@tcp, seq: 5235 00000100:00100000:1.0:1776178096.346901:0:14440:0:(service.c:2324:ptlrpc_server_handle_request()) Handling RPC req@00000000d6cbf5e8 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_004:b50963bd-8eba-4432-9a54-8888d00e3fa1+8:2272:x1862456979079552:12345-192.168.204.3@tcp:2:touch.0 00000100:00100000:1.0:1776178096.347336:0:14440:0:(service.c:2372:ptlrpc_server_handle_request()) Handled RPC req@00000000d6cbf5e8 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_004:b50963bd-8eba-4432-9a54-8888d00e3fa1+9:2272:x1862456979079552:12345-192.168.204.3@tcp:2:touch.0 Request processed in 433us (625us total) trans 4294969344 rc 0/0 00000100:00100000:1.0:1776178096.347347:0:14440:0:(nrs_fifo.c:241:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.204.3@tcp, seq: 5235 00000100:00100000:2.0:1776178096.352838:0:3347:0:(events.c:361:request_in_callback()) peer: 12345-192.168.204.3@tcp (source: 12345-192.168.204.3@tcp) 00000100:00100000:1.0:1776178096.353266:0:6258:0:(service.c:2152:ptlrpc_server_handle_req_in()) got req x1862456979079680 00000100:00100000:1.0:1776178096.353382:0:6258:0:(nrs_fifo.c:179:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.204.3@tcp, seq: 2129 00000100:00100000:1.0:1776178096.353392:0:6258:0:(service.c:2324:ptlrpc_server_handle_request()) Handling RPC req@000000006e2391ef pname:cluuid+ref:pid:xid:nid:opc:job mdt_rdpg00_000:b50963bd-8eba-4432-9a54-8888d00e3fa1+54:16994:x1862456979079680:12345-192.168.204.3@tcp:35:touch.0 00000100:00100000:1.0:1776178096.353875:0:6258:0:(service.c:2372:ptlrpc_server_handle_request()) Handled RPC req@000000006e2391ef pname:cluuid+ref:pid:xid:nid:opc:job mdt_rdpg00_000:b50963bd-8eba-4432-9a54-8888d00e3fa1+55:16994:x1862456979079680:12345-192.168.204.3@tcp:35:touch.0 Request processed in 483us (1051us total) trans 12884901923 rc 0/0 00000100:00100000:1.0:1776178096.353885:0:6258:0:(nrs_fifo.c:241:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.204.3@tcp, seq: 2129 00010000:00010000:2.0:1776178096.546174:0:9224:0:(ldlm_lockd.c:2792:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:0.0:1776178097.443226:0:3346:0:(events.c:361:request_in_callback()) peer: 12345-192.168.204.3@tcp (source: 12345-192.168.204.3@tcp) 00000100:00100000:1.0:1776178097.450816:0:3348:0:(events.c:361:request_in_callback()) peer: 12345-192.168.204.3@tcp (source: 12345-192.168.204.3@tcp) 00000100:00100000:2.0:1776178097.450952:0:6256:0:(service.c:2152:ptlrpc_server_handle_req_in()) got req x1862456979079808 00000100:00100000:2.0:1776178097.451317:0:6256:0:(nrs_fifo.c:179:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.204.3@tcp, seq: 6478 00000100:00100000:2.0:1776178097.451327:0:6256:0:(service.c:2324:ptlrpc_server_handle_request()) Handling RPC req@00000000c1934814 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_001:b50963bd-8eba-4432-9a54-8888d00e3fa1+56:17005:x1862456979079808:12345-192.168.204.3@tcp:101:chmod.0 00010000:00010000:2.0:1776178097.451357:0:6256:0:(ldlm_lockd.c:1248:ldlm_handle_enqueue0()) ### server-side enqueue handler START 00010000:00010000:2.0:1776178097.451385:0:6256:0:(ldlm_lockd.c:1318:ldlm_handle_enqueue0()) ### server-side enqueue handler, new lock created ns: mdt-lustre-MDT0000_UUID lock: 000000000396420a/0xa40083ab9bf53024 lrc: 2/0,0 mode: --/CR res: [0x2000013a1:0x1:0x0].0x0 bits 0x0/0x0 rrc: 4 type: IBT gid 0 flags: 0x40000000000000 nid: local remote: 0x771f68aa1eb279ac expref: -99 pid: 6256 timeout: 0 lvb_type: 0 00010000:00010000:2.0:1776178097.451473:0:6256:0:(ldlm_lock.c:766:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PR) ns: mdt-lustre-MDT0000_UUID lock: 00000000fcb32840/0xa40083ab9bf5302b lrc: 3/1,0 mode: --/PR res: [0x2000013a1:0x2:0x0].0x0 bits 0x0/0x0 rrc: 4 type: IBT gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 6256 timeout: 0 lvb_type: 0 00010000:00010000:2.0:1776178097.451498:0:6256:0:(ldlm_lock.c:1064:ldlm_granted_list_add_lock()) ### About to add lock: ns: mdt-lustre-MDT0000_UUID lock: 00000000fcb32840/0xa40083ab9bf5302b lrc: 3/1,0 mode: PR/PR res: [0x2000013a1:0x2:0x0].0x0 bits 0x5b/0x0 rrc: 4 type: IBT gid 0 flags: 0x50210000000000 nid: local remote: 0x0 expref: -99 pid: 6256 timeout: 0 lvb_type: 0 00010000:00010000:2.0:1776178097.451515:0:6256:0:(ldlm_request.c:528:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: mdt-lustre-MDT0000_UUID lock: 00000000fcb32840/0xa40083ab9bf5302b lrc: 3/1,0 mode: PR/PR res: [0x2000013a1:0x2:0x0].0x0 bits 0x5b/0x0 rrc: 4 type: IBT gid 0 flags: 0x40210000000000 nid: local remote: 0x0 expref: -99 pid: 6256 timeout: 0 lvb_type: 0 00000004:00200000:2.0:1776178097.451601:0:6256:0:(mdt_handler.c:912:mdt_pack_attr2body()) [0x2000013a1:0x2:0x0]: returning size 0 00000004:00200000:2.0:1776178097.451606:0:6256:0:(mdt_handler.c:1523:mdt_getattr_internal()) dirent count 0 stripe count 1 MDT count 2 00000004:00010000:2.0:1776178097.451636:0:6256:0:(mdt_handler.c:2353:mdt_getattr_name_lock()) ### Returning lock to client ns: mdt-lustre-MDT0000_UUID lock: 00000000fcb32840/0xa40083ab9bf5302b lrc: 3/1,0 mode: PR/PR res: [0x2000013a1:0x2:0x0].0x0 bits 0x5b/0x0 rrc: 4 type: IBT gid 0 flags: 0x40210000000000 nid: local remote: 0x0 expref: -99 pid: 6256 timeout: 0 lvb_type: 0 00010000:00010000:2.0:1776178097.451654:0:6256:0:(ldlm_lock.c:1064:ldlm_granted_list_add_lock()) ### About to add lock: ns: mdt-lustre-MDT0000_UUID lock: 00000000fcb32840/0xa40083ab9bf5302b lrc: 3/1,0 mode: PR/PR res: [0x2000013a1:0x2:0x0].0x0 bits 0x1b/0x0 rrc: 4 type: IBT gid 0 flags: 0x50210000000000 nid: local remote: 0x0 expref: -99 pid: 6256 timeout: 0 lvb_type: 0 00010000:00010000:2.0:1776178097.451687:0:6256:0:(ldlm_lock.c:216:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: mdt-lustre-MDT0000_UUID lock: 000000000396420a/0xa40083ab9bf53024 lrc: 0/0,0 mode: --/CR res: [0x2000013a1:0x1:0x0].0x0 bits 0x2/0x0 rrc: 4 type: IBT gid 0 flags: 0x44000000000000 nid: 192.168.204.3@tcp remote: 0x771f68aa1eb279ac expref: 58 pid: 6256 timeout: 0 lvb_type: 0 00010000:00010000:2.0:1776178097.451705:0:6256:0:(ldlm_lockd.c:1484:ldlm_handle_enqueue0()) ### server-side enqueue handler, sending reply (err=0, rc=0) ns: mdt-lustre-MDT0000_UUID lock: 00000000fcb32840/0xa40083ab9bf5302b lrc: 3/0,0 mode: PR/PR res: [0x2000013a1:0x2:0x0].0x0 bits 0x1b/0x0 rrc: 4 type: IBT gid 0 flags: 0x40200000000000 nid: 192.168.204.3@tcp remote: 0x771f68aa1eb279ac expref: 57 pid: 6256 timeout: 0 lvb_type: 0 00010000:00010000:2.0:1776178097.451713:0:6256:0:(ldlm_inodebits.c:94:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x2000013a1:0x2:0x0].0x0 (000000005ad330c7) 00010000:00010000:2.0:1776178097.451716:0:6256:0:(ldlm_inodebits.c:96:ldlm_reprocess_inodebits_queue()) Hint 1b 00010000:00010000:2.0:1776178097.451720:0:6256:0:(ldlm_lockd.c:1565:ldlm_handle_enqueue0()) ### server-side enqueue handler END (lock 00000000fcb32840, rc 0) 00000100:00100000:2.0:1776178097.451807:0:6256:0:(service.c:2372:ptlrpc_server_handle_request()) Handled RPC req@00000000c1934814 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_001:b50963bd-8eba-4432-9a54-8888d00e3fa1+57:17005:x1862456979079808:12345-192.168.204.3@tcp:101:chmod.0 Request processed in 480us (1004us total) trans 0 rc 0/0 00000100:00100000:2.0:1776178097.451815:0:6256:0:(nrs_fifo.c:241:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.204.3@tcp, seq: 6478 00000100:00100000:2.0:1776178097.458818:0:3347:0:(events.c:361:request_in_callback()) peer: 12345-192.168.204.3@tcp (source: 12345-192.168.204.3@tcp) 00000100:00100000:0.0:1776178097.458914:0:20762:0:(service.c:2152:ptlrpc_server_handle_req_in()) got req x1862456979079872 00000100:00100000:0.0:1776178097.458981:0:20762:0:(nrs_fifo.c:179:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.204.3@tcp, seq: 5236 00000100:00100000:0.0:1776178097.458989:0:20762:0:(service.c:2324:ptlrpc_server_handle_request()) Handling RPC req@00000000865e4f65 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_013:b50963bd-8eba-4432-9a54-8888d00e3fa1+8:2272:x1862456979079872:12345-192.168.204.3@tcp:101:chmod.0 00010000:00010000:0.0:1776178097.459014:0:20762:0:(ldlm_lockd.c:1248:ldlm_handle_enqueue0()) ### server-side enqueue handler START 00010000:00010000:0.0:1776178097.459080:0:20762:0:(ldlm_lockd.c:1318:ldlm_handle_enqueue0()) ### server-side enqueue handler, new lock created ns: filter-lustre-OST0001_UUID lock: 000000004bd1d7cb/0xa40083ab9bf53032 lrc: 2/0,0 mode: --/PR res: [0x42:0x0:0x0].0x0 rrc: 2 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x771f68aa1eb279b3 expref: -99 pid: 20762 timeout: 0 lvb_type: 0 00002000:00010000:0.0:1776178097.459121:0:20762:0:(ofd_lvb.c:170:ofd_lvbo_init()) res: [0x100010000:0x42:0x0] initial LVB size: 0, mtime: 0x69de53af, atime: 0x69de53af, ctime: 0x69de53af, blocks: 0x0 00010000:00010000:0.0:1776178097.459157:0:20762:0:(ldlm_extent.c:318:ldlm_check_contention()) contended locks = 0 00010000:00010000:0.0:1776178097.459160:0:20762:0:(ldlm_extent.c:318:ldlm_check_contention()) contended locks = 0 00010000:00010000:0.0:1776178097.459309:0:20762:0:(ldlm_resource.c:1650:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0001_UUID lock: 000000004bd1d7cb/0xa40083ab9bf53032 lrc: 3/0,0 mode: PR/PR res: [0x42:0x0:0x0].0x0 rrc: 2 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x40000000000000 nid: 192.168.204.3@tcp remote: 0x771f68aa1eb279b3 expref: 9 pid: 20762 timeout: 0 lvb_type: 1 00010000:00010000:0.0:1776178097.459338:0:20762:0:(ldlm_lockd.c:1484:ldlm_handle_enqueue0()) ### server-side enqueue handler, sending reply (err=0, rc=0) ns: filter-lustre-OST0001_UUID lock: 000000004bd1d7cb/0xa40083ab9bf53032 lrc: 3/0,0 mode: PR/PR res: [0x42:0x0:0x0].0x0 rrc: 2 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x40000000000000 nid: 192.168.204.3@tcp remote: 0x771f68aa1eb279b3 expref: 9 pid: 20762 timeout: 0 lvb_type: 1 00010000:00010000:0.0:1776178097.459355:0:20762:0:(ldlm_lockd.c:1565:ldlm_handle_enqueue0()) ### server-side enqueue handler END (lock 000000004bd1d7cb, rc 0) 00000100:00100000:0.0:1776178097.459481:0:20762:0:(service.c:2372:ptlrpc_server_handle_request()) Handled RPC req@00000000865e4f65 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_013:b50963bd-8eba-4432-9a54-8888d00e3fa1+9:2272:x1862456979079872:12345-192.168.204.3@tcp:101:chmod.0 Request processed in 490us (672us total) trans 0 rc 0/0 00000100:00100000:0.0:1776178097.459490:0:20762:0:(nrs_fifo.c:241:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.204.3@tcp, seq: 5236 00000100:00100000:2.0:1776178097.475761:0:3346:0:(events.c:361:request_in_callback()) peer: 12345-192.168.204.3@tcp (source: 12345-192.168.204.3@tcp) 00000100:00100000:2.0:1776178097.475822:0:6255:0:(service.c:2152:ptlrpc_server_handle_req_in()) got req x1862456979079936 00000100:00100000:2.0:1776178097.475882:0:6255:0:(nrs_fifo.c:179:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.204.3@tcp, seq: 6479 00000100:00100000:2.0:1776178097.475890:0:6255:0:(service.c:2324:ptlrpc_server_handle_request()) Handling RPC req@0000000010d80ea5 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_000:b50963bd-8eba-4432-9a54-8888d00e3fa1+57:17009:x1862456979079936:12345-192.168.204.3@tcp:101:chmod.0 00010000:00010000:2.0:1776178097.475925:0:6255:0:(ldlm_lockd.c:1248:ldlm_handle_enqueue0()) ### server-side enqueue handler START 00010000:00010000:2.0:1776178097.475956:0:6255:0:(ldlm_lockd.c:1318:ldlm_handle_enqueue0()) ### server-side enqueue handler, new lock created ns: mdt-lustre-MDT0000_UUID lock: 000000000396420a/0xa40083ab9bf53039 lrc: 2/0,0 mode: --/CR res: [0x2000013a1:0x1:0x0].0x0 bits 0x0/0x0 rrc: 4 type: IBT gid 0 flags: 0x40000000000000 nid: local remote: 0x771f68aa1eb279ba expref: -99 pid: 6255 timeout: 0 lvb_type: 0 00010000:00010000:2.0:1776178097.476045:0:6255:0:(ldlm_lock.c:766:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PR) ns: mdt-lustre-MDT0000_UUID lock: 000000006cc9c357/0xa40083ab9bf53040 lrc: 3/1,0 mode: --/PR res: [0x2000013a1:0x6:0x0].0x0 bits 0x0/0x0 rrc: 4 type: IBT gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 6255 timeout: 0 lvb_type: 0 00010000:00010000:2.0:1776178097.476079:0:6255:0:(ldlm_lock.c:1064:ldlm_granted_list_add_lock()) ### About to add lock: ns: mdt-lustre-MDT0000_UUID lock: 000000006cc9c357/0xa40083ab9bf53040 lrc: 3/1,0 mode: PR/PR res: [0x2000013a1:0x6:0x0].0x0 bits 0x5b/0x0 rrc: 4 type: IBT gid 0 flags: 0x50210000000000 nid: local remote: 0x0 expref: -99 pid: 6255 timeout: 0 lvb_type: 0 00010000:00010000:2.0:1776178097.476095:0:6255:0:(ldlm_request.c:528:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: mdt-lustre-MDT0000_UUID lock: 000000006cc9c357/0xa40083ab9bf53040 lrc: 3/1,0 mode: PR/PR res: [0x2000013a1:0x6:0x0].0x0 bits 0x5b/0x0 rrc: 4 type: IBT gid 0 flags: 0x40210000000000 nid: local remote: 0x0 expref: -99 pid: 6255 timeout: 0 lvb_type: 0 00000004:00200000:2.0:1776178097.476155:0:6255:0:(mdt_handler.c:912:mdt_pack_attr2body()) [0x2000013a1:0x6:0x0]: returning size 0 00000004:00200000:2.0:1776178097.476160:0:6255:0:(mdt_handler.c:1523:mdt_getattr_internal()) dirent count 0 stripe count 1 MDT count 2 00000004:00010000:2.0:1776178097.476182:0:6255:0:(mdt_handler.c:2353:mdt_getattr_name_lock()) ### Returning lock to client ns: mdt-lustre-MDT0000_UUID lock: 000000006cc9c357/0xa40083ab9bf53040 lrc: 3/1,0 mode: PR/PR res: [0x2000013a1:0x6:0x0].0x0 bits 0x5b/0x0 rrc: 4 type: IBT gid 0 flags: 0x40210000000000 nid: local remote: 0x0 expref: -99 pid: 6255 timeout: 0 lvb_type: 0 00010000:00010000:2.0:1776178097.476194:0:6255:0:(ldlm_lock.c:1064:ldlm_granted_list_add_lock()) ### About to add lock: ns: mdt-lustre-MDT0000_UUID lock: 000000006cc9c357/0xa40083ab9bf53040 lrc: 3/1,0 mode: PR/PR res: [0x2000013a1:0x6:0x0].0x0 bits 0x1b/0x0 rrc: 4 type: IBT gid 0 flags: 0x50210000000000 nid: local remote: 0x0 expref: -99 pid: 6255 timeout: 0 lvb_type: 0 00010000:00010000:2.0:1776178097.476220:0:6255:0:(ldlm_lock.c:216:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: mdt-lustre-MDT0000_UUID lock: 000000000396420a/0xa40083ab9bf53039 lrc: 0/0,0 mode: --/CR res: [0x2000013a1:0x1:0x0].0x0 bits 0x2/0x0 rrc: 4 type: IBT gid 0 flags: 0x44000000000000 nid: 192.168.204.3@tcp remote: 0x771f68aa1eb279ba expref: 59 pid: 6255 timeout: 0 lvb_type: 0 00010000:00010000:2.0:1776178097.476237:0:6255:0:(ldlm_lockd.c:1484:ldlm_handle_enqueue0()) ### server-side enqueue handler, sending reply (err=0, rc=0) ns: mdt-lustre-MDT0000_UUID lock: 000000006cc9c357/0xa40083ab9bf53040 lrc: 3/0,0 mode: PR/PR res: [0x2000013a1:0x6:0x0].0x0 bits 0x1b/0x0 rrc: 4 type: IBT gid 0 flags: 0x40200000000000 nid: 192.168.204.3@tcp remote: 0x771f68aa1eb279ba expref: 58 pid: 6255 timeout: 0 lvb_type: 0 00010000:00010000:2.0:1776178097.476249:0:6255:0:(ldlm_inodebits.c:94:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x2000013a1:0x6:0x0].0x0 (00000000ef7185e3) 00010000:00010000:2.0:1776178097.476252:0:6255:0:(ldlm_inodebits.c:96:ldlm_reprocess_inodebits_queue()) Hint 1b 00010000:00010000:2.0:1776178097.476255:0:6255:0:(ldlm_lockd.c:1565:ldlm_handle_enqueue0()) ### server-side enqueue handler END (lock 000000006cc9c357, rc 0) 00000100:00100000:2.0:1776178097.476322:0:6255:0:(service.c:2372:ptlrpc_server_handle_request()) Handled RPC req@0000000010d80ea5 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_000:b50963bd-8eba-4432-9a54-8888d00e3fa1+58:17009:x1862456979079936:12345-192.168.204.3@tcp:101:chmod.0 Request processed in 431us (574us total) trans 0 rc 0/0 00000100:00100000:2.0:1776178097.476330:0:6255:0:(nrs_fifo.c:241:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.204.3@tcp, seq: 6479 00000100:00100000:1.0:1776178097.485334:0:3348:0:(events.c:361:request_in_callback()) peer: 12345-192.168.204.3@tcp (source: 12345-192.168.204.3@tcp) 00000100:00100000:1.0:1776178097.485373:0:3348:0:(events.c:361:request_in_callback()) peer: 12345-192.168.204.3@tcp (source: 12345-192.168.204.3@tcp) 00000100:00100000:1.0:1776178097.485398:0:3348:0:(events.c:361:request_in_callback()) peer: 12345-192.168.204.3@tcp (source: 12345-192.168.204.3@tcp) 00000100:00100000:1.0:1776178097.485423:0:3348:0:(events.c:361:request_in_callback()) peer: 12345-192.168.204.3@tcp (source: 12345-192.168.204.3@tcp) 00000100:00100000:1.0:1776178097.485466:0:6256:0:(service.c:2152:ptlrpc_server_handle_req_in()) got req x1862456979080064 00000100:00100000:1.0:1776178097.485517:0:6256:0:(service.c:2152:ptlrpc_server_handle_req_in()) got req x1862456979080192 00000100:00100000:1.0:1776178097.485532:0:6256:0:(service.c:2152:ptlrpc_server_handle_req_in()) got req x1862456979080256 00000100:00100000:1.0:1776178097.485546:0:6256:0:(nrs_fifo.c:179:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.204.3@tcp, seq: 6480 00000100:00100000:1.0:1776178097.485554:0:6256:0:(service.c:2324:ptlrpc_server_handle_request()) Handling RPC req@00000000dff3309c pname:cluuid+ref:pid:xid:nid:opc:job mdt00_001:b50963bd-8eba-4432-9a54-8888d00e3fa1+60:17008:x1862456979080064:12345-192.168.204.3@tcp:101:chmod.0 00010000:00010000:1.0:1776178097.485577:0:6256:0:(ldlm_lockd.c:1248:ldlm_handle_enqueue0()) ### server-side enqueue handler START 00010000:00010000:1.0:1776178097.485599:0:6256:0:(ldlm_lockd.c:1318:ldlm_handle_enqueue0()) ### server-side enqueue handler, new lock created ns: mdt-lustre-MDT0000_UUID lock: 000000003e1697a2/0xa40083ab9bf53047 lrc: 2/0,0 mode: --/CR res: [0x2000013a1:0x1:0x0].0x0 bits 0x0/0x0 rrc: 4 type: IBT gid 0 flags: 0x40000000000000 nid: local remote: 0x771f68aa1eb279c8 expref: -99 pid: 6256 timeout: 0 lvb_type: 0 00010000:00010000:1.0:1776178097.485674:0:6256:0:(ldlm_lock.c:766:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PR) ns: mdt-lustre-MDT0000_UUID lock: 0000000015189b1f/0xa40083ab9bf5304e lrc: 3/1,0 mode: --/PR res: [0x2000013a1:0x5:0x0].0x0 bits 0x0/0x0 rrc: 4 type: IBT gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 6256 timeout: 0 lvb_type: 0 00010000:00010000:1.0:1776178097.485693:0:6256:0:(ldlm_lock.c:1064:ldlm_granted_list_add_lock()) ### About to add lock: ns: mdt-lustre-MDT0000_UUID lock: 0000000015189b1f/0xa40083ab9bf5304e lrc: 3/1,0 mode: PR/PR res: [0x2000013a1:0x5:0x0].0x0 bits 0x5b/0x0 rrc: 4 type: IBT gid 0 flags: 0x50210000000000 nid: local remote: 0x0 expref: -99 pid: 6256 timeout: 0 lvb_type: 0 00010000:00010000:1.0:1776178097.485705:0:6256:0:(ldlm_request.c:528:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: mdt-lustre-MDT0000_UUID lock: 0000000015189b1f/0xa40083ab9bf5304e lrc: 3/1,0 mode: PR/PR res: [0x2000013a1:0x5:0x0].0x0 bits 0x5b/0x0 rrc: 4 type: IBT gid 0 flags: 0x40210000000000 nid: local remote: 0x0 expref: -99 pid: 6256 timeout: 0 lvb_type: 0 00000004:00200000:1.0:1776178097.485767:0:6256:0:(mdt_handler.c:912:mdt_pack_attr2body()) [0x2000013a1:0x5:0x0]: returning size 0 00000004:00200000:1.0:1776178097.485773:0:6256:0:(mdt_handler.c:1523:mdt_getattr_internal()) dirent count 0 stripe count 1 MDT count 2 00000004:00010000:1.0:1776178097.485800:0:6256:0:(mdt_handler.c:2353:mdt_getattr_name_lock()) ### Returning lock to client ns: mdt-lustre-MDT0000_UUID lock: 0000000015189b1f/0xa40083ab9bf5304e lrc: 3/1,0 mode: PR/PR res: [0x2000013a1:0x5:0x0].0x0 bits 0x5b/0x0 rrc: 4 type: IBT gid 0 flags: 0x40210000000000 nid: local remote: 0x0 expref: -99 pid: 6256 timeout: 0 lvb_type: 0 00010000:00010000:1.0:1776178097.485816:0:6256:0:(ldlm_lock.c:1064:ldlm_granted_list_add_lock()) ### About to add lock: ns: mdt-lustre-MDT0000_UUID lock: 0000000015189b1f/0xa40083ab9bf5304e lrc: 3/1,0 mode: PR/PR res: [0x2000013a1:0x5:0x0].0x0 bits 0x1b/0x0 rrc: 4 type: IBT gid 0 flags: 0x50210000000000 nid: local remote: 0x0 expref: -99 pid: 6256 timeout: 0 lvb_type: 0 00010000:00010000:1.0:1776178097.485847:0:6256:0:(ldlm_lock.c:216:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: mdt-lustre-MDT0000_UUID lock: 000000003e1697a2/0xa40083ab9bf53047 lrc: 0/0,0 mode: --/CR res: [0x2000013a1:0x1:0x0].0x0 bits 0x2/0x0 rrc: 4 type: IBT gid 0 flags: 0x44000000000000 nid: 192.168.204.3@tcp remote: 0x771f68aa1eb279c8 expref: 62 pid: 6256 timeout: 0 lvb_type: 0 00010000:00010000:1.0:1776178097.485878:0:6256:0:(ldlm_lockd.c:1484:ldlm_handle_enqueue0()) ### server-side enqueue handler, sending reply (err=0, rc=0) ns: mdt-lustre-MDT0000_UUID lock: 0000000015189b1f/0xa40083ab9bf5304e lrc: 3/0,0 mode: PR/PR res: [0x2000013a1:0x5:0x0].0x0 bits 0x1b/0x0 rrc: 4 type: IBT gid 0 flags: 0x40200000000000 nid: 192.168.204.3@tcp remote: 0x771f68aa1eb279c8 expref: 61 pid: 6256 timeout: 0 lvb_type: 0 00010000:00010000:1.0:1776178097.485886:0:6256:0:(ldlm_inodebits.c:94:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x2000013a1:0x5:0x0].0x0 (000000003b465a63) 00010000:00010000:1.0:1776178097.485889:0:6256:0:(ldlm_inodebits.c:96:ldlm_reprocess_inodebits_queue()) Hint 1b 00010000:00010000:1.0:1776178097.485894:0:6256:0:(ldlm_lockd.c:1565:ldlm_handle_enqueue0()) ### server-side enqueue handler END (lock 0000000015189b1f, rc 0) 00000100:00100000:1.0:1776178097.485963:0:6256:0:(service.c:2372:ptlrpc_server_handle_request()) Handled RPC req@00000000dff3309c pname:cluuid+ref:pid:xid:nid:opc:job mdt00_001:b50963bd-8eba-4432-9a54-8888d00e3fa1+61:17008:x1862456979080064:12345-192.168.204.3@tcp:101:chmod.0 Request processed in 409us (641us total) trans 0 rc 0/0 00000100:00100000:1.0:1776178097.485970:0:6256:0:(nrs_fifo.c:241:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.204.3@tcp, seq: 6480 00000100:00100000:1.0:1776178097.485992:0:6256:0:(nrs_fifo.c:179:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.204.3@tcp, seq: 6481 00000100:00100000:1.0:1776178097.485996:0:6256:0:(service.c:2324:ptlrpc_server_handle_request()) Handling RPC req@0000000053234520 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_001:b50963bd-8eba-4432-9a54-8888d00e3fa1+60:17006:x1862456979080192:12345-192.168.204.3@tcp:36:chmod.0 00000020:02000000:1.0:1776178097.486008:0:6256:0:(libcfs_fail.h:96:cfs_fail_check_set()) *** cfs_fail_loc=159, val=0*** 00000100:00100000:1.0:1776178097.487966:0:6256:0:(service.c:2372:ptlrpc_server_handle_request()) Handled RPC req@0000000053234520 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_001:b50963bd-8eba-4432-9a54-8888d00e3fa1+60:17006:x1862456979080192:12345-192.168.204.3@tcp:36:chmod.0 Request processed in 1958us (2557us total) trans 0 rc 0/-999 00000100:00100000:1.0:1776178097.487981:0:6256:0:(nrs_fifo.c:241:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.204.3@tcp, seq: 6481 00000100:00100000:1.0:1776178097.488027:0:6256:0:(nrs_fifo.c:179:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.204.3@tcp, seq: 6482 00000100:00100000:1.0:1776178097.488036:0:6256:0:(service.c:2324:ptlrpc_server_handle_request()) Handling RPC req@000000002973935f pname:cluuid+ref:pid:xid:nid:opc:job mdt00_001:b50963bd-8eba-4432-9a54-8888d00e3fa1+59:17010:x1862456979080256:12345-192.168.204.3@tcp:101:chmod.0 00010000:00010000:1.0:1776178097.488078:0:6256:0:(ldlm_lockd.c:1248:ldlm_handle_enqueue0()) ### server-side enqueue handler START 00000100:00100000:1.0:1776178097.488126:0:20762:0:(service.c:2152:ptlrpc_server_handle_req_in()) got req x1862456979080000 00000100:00100000:1.0:1776178097.488184:0:20762:0:(nrs_fifo.c:179:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.204.3@tcp, seq: 5237 00000100:00100000:1.0:1776178097.488189:0:20762:0:(service.c:2324:ptlrpc_server_handle_request()) Handling RPC req@00000000812a7ebe pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_013:b50963bd-8eba-4432-9a54-8888d00e3fa1+9:2270:x1862456979080000:12345-192.168.204.3@tcp:101:chmod.0 00010000:00010000:1.0:1776178097.488197:0:20762:0:(ldlm_lockd.c:1248:ldlm_handle_enqueue0()) ### server-side enqueue handler START 00010000:00010000:1.0:1776178097.488219:0:20762:0:(ldlm_lockd.c:1318:ldlm_handle_enqueue0()) ### server-side enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 000000002abc4399/0xa40083ab9bf53055 lrc: 2/0,0 mode: --/PR res: [0x42:0x0:0x0].0x0 rrc: 2 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x771f68aa1eb279c1 expref: -99 pid: 20762 timeout: 0 lvb_type: 0 00002000:00010000:1.0:1776178097.488246:0:20762:0:(ofd_lvb.c:170:ofd_lvbo_init()) res: [0x100000000:0x42:0x0] initial LVB size: 0, mtime: 0x69de53af, atime: 0x69de53af, ctime: 0x69de53af, blocks: 0x0 00010000:00010000:1.0:1776178097.488267:0:20762:0:(ldlm_extent.c:318:ldlm_check_contention()) contended locks = 0 00010000:00010000:1.0:1776178097.488270:0:20762:0:(ldlm_extent.c:318:ldlm_check_contention()) contended locks = 0 00010000:00010000:1.0:1776178097.488281:0:20762:0:(ldlm_resource.c:1650:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 000000002abc4399/0xa40083ab9bf53055 lrc: 3/0,0 mode: PR/PR res: [0x42:0x0:0x0].0x0 rrc: 2 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x40000000000000 nid: 192.168.204.3@tcp remote: 0x771f68aa1eb279c1 expref: 10 pid: 20762 timeout: 0 lvb_type: 1 00010000:00010000:1.0:1776178097.488305:0:20762:0:(ldlm_lockd.c:1484:ldlm_handle_enqueue0()) ### server-side enqueue handler, sending reply (err=0, rc=0) ns: filter-lustre-OST0000_UUID lock: 000000002abc4399/0xa40083ab9bf53055 lrc: 3/0,0 mode: PR/PR res: [0x42:0x0:0x0].0x0 rrc: 2 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x40000000000000 nid: 192.168.204.3@tcp remote: 0x771f68aa1eb279c1 expref: 10 pid: 20762 timeout: 0 lvb_type: 1 00010000:00010000:1.0:1776178097.488324:0:20762:0:(ldlm_lockd.c:1565:ldlm_handle_enqueue0()) ### server-side enqueue handler END (lock 000000002abc4399, rc 0) 00000100:00100000:1.0:1776178097.488386:0:20762:0:(service.c:2372:ptlrpc_server_handle_request()) Handled RPC req@00000000812a7ebe pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_013:b50963bd-8eba-4432-9a54-8888d00e3fa1+10:2270:x1862456979080000:12345-192.168.204.3@tcp:101:chmod.0 Request processed in 196us (3013us total) trans 0 rc 0/0 00000100:00100000:1.0:1776178097.488396:0:20762:0:(nrs_fifo.c:241:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.204.3@tcp, seq: 5237 00010000:00010000:1.0:1776178097.488570:0:6256:0:(ldlm_lockd.c:1318:ldlm_handle_enqueue0()) ### server-side enqueue handler, new lock created ns: mdt-lustre-MDT0000_UUID lock: 0000000050323271/0xa40083ab9bf5305c lrc: 2/0,0 mode: --/CR res: [0x2000013a1:0x1:0x0].0x0 bits 0x0/0x0 rrc: 4 type: IBT gid 0 flags: 0x40000000000000 nid: local remote: 0x771f68aa1eb279cf expref: -99 pid: 6256 timeout: 0 lvb_type: 0 00010000:00010000:1.0:1776178097.488629:0:6256:0:(ldlm_lock.c:766:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PR) ns: mdt-lustre-MDT0000_UUID lock: 00000000499bab84/0xa40083ab9bf53063 lrc: 3/1,0 mode: --/PR res: [0x2000013a1:0x7:0x0].0x0 bits 0x0/0x0 rrc: 4 type: IBT gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 6256 timeout: 0 lvb_type: 0 00010000:00010000:1.0:1776178097.488645:0:6256:0:(ldlm_lock.c:1064:ldlm_granted_list_add_lock()) ### About to add lock: ns: mdt-lustre-MDT0000_UUID lock: 00000000499bab84/0xa40083ab9bf53063 lrc: 3/1,0 mode: PR/PR res: [0x2000013a1:0x7:0x0].0x0 bits 0x5b/0x0 rrc: 4 type: IBT gid 0 flags: 0x50210000000000 nid: local remote: 0x0 expref: -99 pid: 6256 timeout: 0 lvb_type: 0 00010000:00010000:1.0:1776178097.488656:0:6256:0:(ldlm_request.c:528:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: mdt-lustre-MDT0000_UUID lock: 00000000499bab84/0xa40083ab9bf53063 lrc: 3/1,0 mode: PR/PR res: [0x2000013a1:0x7:0x0].0x0 bits 0x5b/0x0 rrc: 4 type: IBT gid 0 flags: 0x40210000000000 nid: local remote: 0x0 expref: -99 pid: 6256 timeout: 0 lvb_type: 0 00000004:00200000:1.0:1776178097.488721:0:6256:0:(mdt_handler.c:912:mdt_pack_attr2body()) [0x2000013a1:0x7:0x0]: returning size 0 00000004:00200000:1.0:1776178097.488727:0:6256:0:(mdt_handler.c:1523:mdt_getattr_internal()) dirent count 0 stripe count 1 MDT count 2 00000100:00100000:2.0:1776178097.493110:0:3347:0:(events.c:361:request_in_callback()) peer: 12345-192.168.204.3@tcp (source: 12345-192.168.204.3@tcp) 00000100:00100000:2.0:1776178097.493168:0:3347:0:(events.c:361:request_in_callback()) peer: 12345-192.168.204.3@tcp (source: 12345-192.168.204.3@tcp) 00000100:00100000:2.0:1776178097.493195:0:3347:0:(events.c:361:request_in_callback()) peer: 12345-192.168.204.3@tcp (source: 12345-192.168.204.3@tcp) 00000100:00100000:2.0:1776178097.493231:0:6257:0:(service.c:2152:ptlrpc_server_handle_req_in()) got req x1862456979080384 00000100:00100000:2.0:1776178097.493277:0:6257:0:(service.c:2152:ptlrpc_server_handle_req_in()) got req x1862456979080448 00000100:00100000:2.0:1776178097.493297:0:6257:0:(nrs_fifo.c:179:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.204.3@tcp, seq: 6483 00000100:00100000:2.0:1776178097.493306:0:6257:0:(service.c:2324:ptlrpc_server_handle_request()) Handling RPC req@0000000071333ecf pname:cluuid+ref:pid:xid:nid:opc:job mdt00_002:b50963bd-8eba-4432-9a54-8888d00e3fa1+62:17011:x1862456979080384:12345-192.168.204.3@tcp:101:chmod.0 00010000:00010000:2.0:1776178097.493336:0:6257:0:(ldlm_lockd.c:1248:ldlm_handle_enqueue0()) ### server-side enqueue handler START 00010000:00010000:2.0:1776178097.493363:0:6257:0:(ldlm_lockd.c:1318:ldlm_handle_enqueue0()) ### server-side enqueue handler, new lock created ns: mdt-lustre-MDT0000_UUID lock: 00000000e4217a5c/0xa40083ab9bf5306a lrc: 2/0,0 mode: --/CR res: [0x2000013a1:0x1:0x0].0x0 bits 0x0/0x0 rrc: 5 type: IBT gid 0 flags: 0x40000000000000 nid: local remote: 0x771f68aa1eb279dd expref: -99 pid: 6257 timeout: 0 lvb_type: 0 00010000:00010000:2.0:1776178097.493446:0:6257:0:(ldlm_lock.c:766:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PR) ns: mdt-lustre-MDT0000_UUID lock: 000000009a8f90ae/0xa40083ab9bf53071 lrc: 3/1,0 mode: --/PR res: [0x2000013a1:0x8:0x0].0x0 bits 0x0/0x0 rrc: 4 type: IBT gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 6257 timeout: 0 lvb_type: 0 00010000:00010000:2.0:1776178097.493465:0:6257:0:(ldlm_lock.c:1064:ldlm_granted_list_add_lock()) ### About to add lock: ns: mdt-lustre-MDT0000_UUID lock: 000000009a8f90ae/0xa40083ab9bf53071 lrc: 3/1,0 mode: PR/PR res: [0x2000013a1:0x8:0x0].0x0 bits 0x5b/0x0 rrc: 4 type: IBT gid 0 flags: 0x50210000000000 nid: local remote: 0x0 expref: -99 pid: 6257 timeout: 0 lvb_type: 0 00010000:00010000:2.0:1776178097.493477:0:6257:0:(ldlm_request.c:528:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: mdt-lustre-MDT0000_UUID lock: 000000009a8f90ae/0xa40083ab9bf53071 lrc: 3/1,0 mode: PR/PR res: [0x2000013a1:0x8:0x0].0x0 bits 0x5b/0x0 rrc: 4 type: IBT gid 0 flags: 0x40210000000000 nid: local remote: 0x0 expref: -99 pid: 6257 timeout: 0 lvb_type: 0 00000004:00200000:2.0:1776178097.493530:0:6257:0:(mdt_handler.c:912:mdt_pack_attr2body()) [0x2000013a1:0x8:0x0]: returning size 0 00000004:00200000:2.0:1776178097.493537:0:6257:0:(mdt_handler.c:1523:mdt_getattr_internal()) dirent count 0 stripe count 1 MDT count 2 00000004:00010000:2.0:1776178097.493559:0:6257:0:(mdt_handler.c:2353:mdt_getattr_name_lock()) ### Returning lock to client ns: mdt-lustre-MDT0000_UUID lock: 000000009a8f90ae/0xa40083ab9bf53071 lrc: 3/1,0 mode: PR/PR res: [0x2000013a1:0x8:0x0].0x0 bits 0x5b/0x0 rrc: 4 type: IBT gid 0 flags: 0x40210000000000 nid: local remote: 0x0 expref: -99 pid: 6257 timeout: 0 lvb_type: 0 00010000:00010000:2.0:1776178097.493570:0:6257:0:(ldlm_lock.c:1064:ldlm_granted_list_add_lock()) ### About to add lock: ns: mdt-lustre-MDT0000_UUID lock: 000000009a8f90ae/0xa40083ab9bf53071 lrc: 3/1,0 mode: PR/PR res: [0x2000013a1:0x8:0x0].0x0 bits 0x1b/0x0 rrc: 4 type: IBT gid 0 flags: 0x50210000000000 nid: local remote: 0x0 expref: -99 pid: 6257 timeout: 0 lvb_type: 0 00010000:00010000:2.0:1776178097.493592:0:6257:0:(ldlm_lock.c:216:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: mdt-lustre-MDT0000_UUID lock: 00000000e4217a5c/0xa40083ab9bf5306a lrc: 0/0,0 mode: --/CR res: [0x2000013a1:0x1:0x0].0x0 bits 0x2/0x0 rrc: 5 type: IBT gid 0 flags: 0x44000000000000 nid: 192.168.204.3@tcp remote: 0x771f68aa1eb279dd expref: 64 pid: 6257 timeout: 0 lvb_type: 0 00010000:00010000:2.0:1776178097.493608:0:6257:0:(ldlm_lockd.c:1484:ldlm_handle_enqueue0()) ### server-side enqueue handler, sending reply (err=0, rc=0) ns: mdt-lustre-MDT0000_UUID lock: 000000009a8f90ae/0xa40083ab9bf53071 lrc: 3/0,0 mode: PR/PR res: [0x2000013a1:0x8:0x0].0x0 bits 0x1b/0x0 rrc: 4 type: IBT gid 0 flags: 0x40200000000000 nid: 192.168.204.3@tcp remote: 0x771f68aa1eb279dd expref: 63 pid: 6257 timeout: 0 lvb_type: 0 00010000:00010000:2.0:1776178097.494515:0:6257:0:(ldlm_inodebits.c:94:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x2000013a1:0x8:0x0].0x0 (00000000e9f0d179) 00010000:00010000:2.0:1776178097.494520:0:6257:0:(ldlm_inodebits.c:96:ldlm_reprocess_inodebits_queue()) Hint 1b 00010000:00010000:2.0:1776178097.494526:0:6257:0:(ldlm_lockd.c:1565:ldlm_handle_enqueue0()) ### server-side enqueue handler END (lock 000000009a8f90ae, rc 0) 00000100:00100000:2.0:1776178097.494695:0:6257:0:(service.c:2372:ptlrpc_server_handle_request()) Handled RPC req@0000000071333ecf pname:cluuid+ref:pid:xid:nid:opc:job mdt00_002:b50963bd-8eba-4432-9a54-8888d00e3fa1+63:17011:x1862456979080384:12345-192.168.204.3@tcp:101:chmod.0 Request processed in 1386us (1525us total) trans 0 rc 0/0 00000100:00100000:2.0:1776178097.494705:0:6257:0:(nrs_fifo.c:241:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.204.3@tcp, seq: 6483 00000100:00100000:2.0:1776178097.494748:0:6257:0:(nrs_fifo.c:179:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.204.3@tcp, seq: 6484 00000100:00100000:2.0:1776178097.494754:0:6257:0:(service.c:2324:ptlrpc_server_handle_request()) Handling RPC req@000000009220b2b0 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_002:b50963bd-8eba-4432-9a54-8888d00e3fa1+62:17007:x1862456979080448:12345-192.168.204.3@tcp:101:chmod.0 00010000:00010000:2.0:1776178097.494774:0:6257:0:(ldlm_lockd.c:1248:ldlm_handle_enqueue0()) ### server-side enqueue handler START 00010000:00010000:2.0:1776178097.494791:0:6257:0:(ldlm_lockd.c:1318:ldlm_handle_enqueue0()) ### server-side enqueue handler, new lock created ns: mdt-lustre-MDT0000_UUID lock: 0000000073f2c7a9/0xa40083ab9bf53078 lrc: 2/0,0 mode: --/CR res: [0x2000013a1:0x1:0x0].0x0 bits 0x0/0x0 rrc: 5 type: IBT gid 0 flags: 0x40000000000000 nid: local remote: 0x771f68aa1eb279e4 expref: -99 pid: 6257 timeout: 0 lvb_type: 0 00010000:00010000:2.0:1776178097.494857:0:6257:0:(ldlm_lock.c:766:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PR) ns: mdt-lustre-MDT0000_UUID lock: 00000000a27c19b3/0xa40083ab9bf5307f lrc: 3/1,0 mode: --/PR res: [0x2000013a1:0x4:0x0].0x0 bits 0x0/0x0 rrc: 4 type: IBT gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 6257 timeout: 0 lvb_type: 0 00010000:00010000:2.0:1776178097.494872:0:6257:0:(ldlm_lock.c:1064:ldlm_granted_list_add_lock()) ### About to add lock: ns: mdt-lustre-MDT0000_UUID lock: 00000000a27c19b3/0xa40083ab9bf5307f lrc: 3/1,0 mode: PR/PR res: [0x2000013a1:0x4:0x0].0x0 bits 0x5b/0x0 rrc: 4 type: IBT gid 0 flags: 0x50210000000000 nid: local remote: 0x0 expref: -99 pid: 6257 timeout: 0 lvb_type: 0 00010000:00010000:2.0:1776178097.494885:0:6257:0:(ldlm_request.c:528:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: mdt-lustre-MDT0000_UUID lock: 00000000a27c19b3/0xa40083ab9bf5307f lrc: 3/1,0 mode: PR/PR res: [0x2000013a1:0x4:0x0].0x0 bits 0x5b/0x0 rrc: 4 type: IBT gid 0 flags: 0x40210000000000 nid: local remote: 0x0 expref: -99 pid: 6257 timeout: 0 lvb_type: 0 00000004:00200000:2.0:1776178097.494932:0:6257:0:(mdt_handler.c:912:mdt_pack_attr2body()) [0x2000013a1:0x4:0x0]: returning size 0 00000004:00200000:2.0:1776178097.494935:0:6257:0:(mdt_handler.c:1523:mdt_getattr_internal()) dirent count 0 stripe count 1 MDT count 2 00000004:00010000:2.0:1776178097.494951:0:6257:0:(mdt_handler.c:2353:mdt_getattr_name_lock()) ### Returning lock to client ns: mdt-lustre-MDT0000_UUID lock: 00000000a27c19b3/0xa40083ab9bf5307f lrc: 3/1,0 mode: PR/PR res: [0x2000013a1:0x4:0x0].0x0 bits 0x5b/0x0 rrc: 4 type: IBT gid 0 flags: 0x40210000000000 nid: local remote: 0x0 expref: -99 pid: 6257 timeout: 0 lvb_type: 0 00010000:00010000:2.0:1776178097.494961:0:6257:0:(ldlm_lock.c:1064:ldlm_granted_list_add_lock()) ### About to add lock: ns: mdt-lustre-MDT0000_UUID lock: 00000000a27c19b3/0xa40083ab9bf5307f lrc: 3/1,0 mode: PR/PR res: [0x2000013a1:0x4:0x0].0x0 bits 0x1b/0x0 rrc: 4 type: IBT gid 0 flags: 0x50210000000000 nid: local remote: 0x0 expref: -99 pid: 6257 timeout: 0 lvb_type: 0 00010000:00010000:2.0:1776178097.494983:0:6257:0:(ldlm_lock.c:216:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: mdt-lustre-MDT0000_UUID lock: 0000000073f2c7a9/0xa40083ab9bf53078 lrc: 0/0,0 mode: --/CR res: [0x2000013a1:0x1:0x0].0x0 bits 0x2/0x0 rrc: 5 type: IBT gid 0 flags: 0x44000000000000 nid: 192.168.204.3@tcp remote: 0x771f68aa1eb279e4 expref: 64 pid: 6257 timeout: 0 lvb_type: 0 00010000:00010000:2.0:1776178097.495000:0:6257:0:(ldlm_lockd.c:1484:ldlm_handle_enqueue0()) ### server-side enqueue handler, sending reply (err=0, rc=0) ns: mdt-lustre-MDT0000_UUID lock: 00000000a27c19b3/0xa40083ab9bf5307f lrc: 3/0,0 mode: PR/PR res: [0x2000013a1:0x4:0x0].0x0 bits 0x1b/0x0 rrc: 4 type: IBT gid 0 flags: 0x40200000000000 nid: 192.168.204.3@tcp remote: 0x771f68aa1eb279e4 expref: 63 pid: 6257 timeout: 0 lvb_type: 0 00010000:00010000:2.0:1776178097.495007:0:6257:0:(ldlm_inodebits.c:94:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x2000013a1:0x4:0x0].0x0 (00000000ef6cecb2) 00010000:00010000:2.0:1776178097.495010:0:6257:0:(ldlm_inodebits.c:96:ldlm_reprocess_inodebits_queue()) Hint 1b 00010000:00010000:2.0:1776178097.495013:0:6257:0:(ldlm_lockd.c:1565:ldlm_handle_enqueue0()) ### server-side enqueue handler END (lock 00000000a27c19b3, rc 0) 00000100:00100000:2.0:1776178097.495137:0:6257:0:(service.c:2372:ptlrpc_server_handle_request()) Handled RPC req@000000009220b2b0 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_002:b50963bd-8eba-4432-9a54-8888d00e3fa1+63:17007:x1862456979080448:12345-192.168.204.3@tcp:101:chmod.0 Request processed in 383us (1942us total) trans 0 rc 0/0 00000100:00100000:2.0:1776178097.495143:0:6257:0:(nrs_fifo.c:241:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.204.3@tcp, seq: 6484 00000100:00100000:2.0:1776178097.495264:0:14440:0:(service.c:2152:ptlrpc_server_handle_req_in()) got req x1862456979080320 00000100:00100000:2.0:1776178097.495316:0:14440:0:(nrs_fifo.c:179:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.204.3@tcp, seq: 5238 00000100:00100000:2.0:1776178097.495321:0:14440:0:(service.c:2324:ptlrpc_server_handle_request()) Handling RPC req@00000000597c3414 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_004:b50963bd-8eba-4432-9a54-8888d00e3fa1+10:2273:x1862456979080320:12345-192.168.204.3@tcp:101:chmod.0 00010000:00010000:2.0:1776178097.495330:0:14440:0:(ldlm_lockd.c:1248:ldlm_handle_enqueue0()) ### server-side enqueue handler START 00010000:00010000:2.0:1776178097.495348:0:14440:0:(ldlm_lockd.c:1318:ldlm_handle_enqueue0()) ### server-side enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 000000007f7ce364/0xa40083ab9bf53086 lrc: 2/0,0 mode: --/PR res: [0x44:0x0:0x0].0x0 rrc: 2 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x771f68aa1eb279d6 expref: -99 pid: 14440 timeout: 0 lvb_type: 0 00002000:00010000:2.0:1776178097.495373:0:14440:0:(ofd_lvb.c:170:ofd_lvbo_init()) res: [0x100000000:0x44:0x0] initial LVB size: 0, mtime: 0x69de53af, atime: 0x69de53af, ctime: 0x69de53af, blocks: 0x0 00010000:00010000:2.0:1776178097.495389:0:14440:0:(ldlm_extent.c:318:ldlm_check_contention()) contended locks = 0 00010000:00010000:2.0:1776178097.495391:0:14440:0:(ldlm_extent.c:318:ldlm_check_contention()) contended locks = 0 00010000:00010000:2.0:1776178097.495398:0:14440:0:(ldlm_resource.c:1650:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 000000007f7ce364/0xa40083ab9bf53086 lrc: 3/0,0 mode: PR/PR res: [0x44:0x0:0x0].0x0 rrc: 2 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x40000000000000 nid: 192.168.204.3@tcp remote: 0x771f68aa1eb279d6 expref: 11 pid: 14440 timeout: 0 lvb_type: 1 00010000:00010000:2.0:1776178097.495410:0:14440:0:(ldlm_lockd.c:1484:ldlm_handle_enqueue0()) ### server-side enqueue handler, sending reply (err=0, rc=0) ns: filter-lustre-OST0000_UUID lock: 000000007f7ce364/0xa40083ab9bf53086 lrc: 3/0,0 mode: PR/PR res: [0x44:0x0:0x0].0x0 rrc: 2 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x40000000000000 nid: 192.168.204.3@tcp remote: 0x771f68aa1eb279d6 expref: 11 pid: 14440 timeout: 0 lvb_type: 1 00010000:00010000:2.0:1776178097.495421:0:14440:0:(ldlm_lockd.c:1565:ldlm_handle_enqueue0()) ### server-side enqueue handler END (lock 000000007f7ce364, rc 0) 00000100:00100000:2.0:1776178097.495443:0:14440:0:(service.c:2372:ptlrpc_server_handle_request()) Handled RPC req@00000000597c3414 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_004:b50963bd-8eba-4432-9a54-8888d00e3fa1+11:2273:x1862456979080320:12345-192.168.204.3@tcp:101:chmod.0 Request processed in 122us (2346us total) trans 0 rc 0/0 00000100:00100000:2.0:1776178097.495449:0:14440:0:(nrs_fifo.c:241:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.204.3@tcp, seq: 5238 00000004:00010000:1.0:1776178097.499369:0:6256:0:(mdt_handler.c:2353:mdt_getattr_name_lock()) ### Returning lock to client ns: mdt-lustre-MDT0000_UUID lock: 00000000499bab84/0xa40083ab9bf53063 lrc: 3/1,0 mode: PR/PR res: [0x2000013a1:0x7:0x0].0x0 bits 0x5b/0x0 rrc: 4 type: IBT gid 0 flags: 0x40210000000000 nid: local remote: 0x0 expref: -99 pid: 6256 timeout: 0 lvb_type: 0 00010000:00010000:1.0:1776178097.499403:0:6256:0:(ldlm_lock.c:1064:ldlm_granted_list_add_lock()) ### About to add lock: ns: mdt-lustre-MDT0000_UUID lock: 00000000499bab84/0xa40083ab9bf53063 lrc: 3/1,0 mode: PR/PR res: [0x2000013a1:0x7:0x0].0x0 bits 0x1b/0x0 rrc: 4 type: IBT gid 0 flags: 0x50210000000000 nid: local remote: 0x0 expref: -99 pid: 6256 timeout: 0 lvb_type: 0 00010000:00010000:1.0:1776178097.499446:0:6256:0:(ldlm_lock.c:216:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: mdt-lustre-MDT0000_UUID lock: 0000000050323271/0xa40083ab9bf5305c lrc: 0/0,0 mode: --/CR res: [0x2000013a1:0x1:0x0].0x0 bits 0x2/0x0 rrc: 4 type: IBT gid 0 flags: 0x44000000000000 nid: 192.168.204.3@tcp remote: 0x771f68aa1eb279cf expref: 63 pid: 6256 timeout: 0 lvb_type: 0 00010000:00010000:1.0:1776178097.499465:0:6256:0:(ldlm_lockd.c:1484:ldlm_handle_enqueue0()) ### server-side enqueue handler, sending reply (err=0, rc=0) ns: mdt-lustre-MDT0000_UUID lock: 00000000499bab84/0xa40083ab9bf53063 lrc: 3/0,0 mode: PR/PR res: [0x2000013a1:0x7:0x0].0x0 bits 0x1b/0x0 rrc: 4 type: IBT gid 0 flags: 0x40200000000000 nid: 192.168.204.3@tcp remote: 0x771f68aa1eb279cf expref: 62 pid: 6256 timeout: 0 lvb_type: 0 00010000:00010000:1.0:1776178097.499474:0:6256:0:(ldlm_inodebits.c:94:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x2000013a1:0x7:0x0].0x0 (00000000665a3017) 00010000:00010000:1.0:1776178097.499477:0:6256:0:(ldlm_inodebits.c:96:ldlm_reprocess_inodebits_queue()) Hint 1b 00010000:00010000:1.0:1776178097.499482:0:6256:0:(ldlm_lockd.c:1565:ldlm_handle_enqueue0()) ### server-side enqueue handler END (lock 00000000499bab84, rc 0) 00000100:00100000:1.0:1776178097.499588:0:3346:0:(events.c:361:request_in_callback()) peer: 12345-192.168.204.3@tcp (source: 12345-192.168.204.3@tcp) 00000100:00100000:1.0:1776178097.499642:0:3346:0:(events.c:361:request_in_callback()) peer: 12345-192.168.204.3@tcp (source: 12345-192.168.204.3@tcp) 00000100:00100000:1.0:1776178097.499702:0:20762:0:(service.c:2152:ptlrpc_server_handle_req_in()) got req x1862456979080512 00000100:00100000:1.0:1776178097.499772:0:20762:0:(nrs_fifo.c:179:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.204.3@tcp, seq: 5239 00000100:00100000:1.0:1776178097.499779:0:20762:0:(service.c:2324:ptlrpc_server_handle_request()) Handling RPC req@0000000017d54e92 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_013:b50963bd-8eba-4432-9a54-8888d00e3fa1+9:2271:x1862456979080512:12345-192.168.204.3@tcp:101:chmod.0 00010000:00010000:1.0:1776178097.499796:0:20762:0:(ldlm_lockd.c:1248:ldlm_handle_enqueue0()) ### server-side enqueue handler START 00010000:00010000:1.0:1776178097.499820:0:20762:0:(ldlm_lockd.c:1318:ldlm_handle_enqueue0()) ### server-side enqueue handler, new lock created ns: filter-lustre-OST0001_UUID lock: 00000000daa24072/0xa40083ab9bf5308d lrc: 2/0,0 mode: --/PR res: [0x43:0x0:0x0].0x0 rrc: 2 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x771f68aa1eb279eb expref: -99 pid: 20762 timeout: 0 lvb_type: 0 00002000:00010000:1.0:1776178097.499848:0:20762:0:(ofd_lvb.c:170:ofd_lvbo_init()) res: [0x100010000:0x43:0x0] initial LVB size: 0, mtime: 0x69de53af, atime: 0x69de53af, ctime: 0x69de53af, blocks: 0x0 00010000:00010000:1.0:1776178097.499870:0:20762:0:(ldlm_extent.c:318:ldlm_check_contention()) contended locks = 0 00010000:00010000:1.0:1776178097.499874:0:20762:0:(ldlm_extent.c:318:ldlm_check_contention()) contended locks = 0 00010000:00010000:1.0:1776178097.499883:0:20762:0:(ldlm_resource.c:1650:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0001_UUID lock: 00000000daa24072/0xa40083ab9bf5308d lrc: 3/0,0 mode: PR/PR res: [0x43:0x0:0x0].0x0 rrc: 2 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x40000000000000 nid: 192.168.204.3@tcp remote: 0x771f68aa1eb279eb expref: 10 pid: 20762 timeout: 0 lvb_type: 1 00010000:00010000:1.0:1776178097.499899:0:20762:0:(ldlm_lockd.c:1484:ldlm_handle_enqueue0()) ### server-side enqueue handler, sending reply (err=0, rc=0) ns: filter-lustre-OST0001_UUID lock: 00000000daa24072/0xa40083ab9bf5308d lrc: 3/0,0 mode: PR/PR res: [0x43:0x0:0x0].0x0 rrc: 2 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x40000000000000 nid: 192.168.204.3@tcp remote: 0x771f68aa1eb279eb expref: 10 pid: 20762 timeout: 0 lvb_type: 1 00010000:00010000:1.0:1776178097.499911:0:20762:0:(ldlm_lockd.c:1565:ldlm_handle_enqueue0()) ### server-side enqueue handler END (lock 00000000daa24072, rc 0) 00000100:00100000:1.0:1776178097.499972:0:20762:0:(service.c:2372:ptlrpc_server_handle_request()) Handled RPC req@0000000017d54e92 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_013:b50963bd-8eba-4432-9a54-8888d00e3fa1+10:2271:x1862456979080512:12345-192.168.204.3@tcp:101:chmod.0 Request processed in 192us (386us total) trans 0 rc 0/0 00000100:00100000:1.0:1776178097.499979:0:20762:0:(nrs_fifo.c:241:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.204.3@tcp, seq: 5239 00000100:00100000:1.0:1776178097.503993:0:6255:0:(service.c:2152:ptlrpc_server_handle_req_in()) got req x1862456979080640 00000100:00100000:1.0:1776178097.504280:0:6255:0:(nrs_fifo.c:179:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.204.3@tcp, seq: 6485 00000100:00100000:1.0:1776178097.504291:0:6255:0:(service.c:2324:ptlrpc_server_handle_request()) Handling RPC req@000000003316d46e pname:cluuid+ref:pid:xid:nid:opc:job mdt00_000:b50963bd-8eba-4432-9a54-8888d00e3fa1+63:17005:x1862456979080640:12345-192.168.204.3@tcp:36:chmod.0 00000020:02000000:1.0:1776178097.504316:0:6255:0:(libcfs_fail.h:96:cfs_fail_check_set()) *** cfs_fail_loc=159, val=0*** 00000100:00100000:1.0:1776178097.504322:0:6255:0:(service.c:2372:ptlrpc_server_handle_request()) Handled RPC req@000000003316d46e pname:cluuid+ref:pid:xid:nid:opc:job mdt00_000:b50963bd-8eba-4432-9a54-8888d00e3fa1+63:17005:x1862456979080640:12345-192.168.204.3@tcp:36:chmod.0 Request processed in 33us (4680us total) trans 0 rc 0/-999 00000100:00100000:1.0:1776178097.504329:0:6255:0:(nrs_fifo.c:241:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.204.3@tcp, seq: 6485 00000100:00100000:1.0:1776178097.504407:0:6256:0:(service.c:2372:ptlrpc_server_handle_request()) Handled RPC req@000000002973935f pname:cluuid+ref:pid:xid:nid:opc:job mdt00_001:b50963bd-8eba-4432-9a54-8888d00e3fa1+62:17010:x1862456979080256:12345-192.168.204.3@tcp:101:chmod.0 Request processed in 16372us (18983us total) trans 0 rc 0/0 00000100:00100000:1.0:1776178097.504414:0:6256:0:(nrs_fifo.c:241:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.204.3@tcp, seq: 6482 00000100:00100000:1.0:1776178097.504513:0:3346:0:(events.c:361:request_in_callback()) peer: 12345-192.168.204.3@tcp (source: 12345-192.168.204.3@tcp) 00000100:00100000:1.0:1776178097.504596:0:3346:0:(events.c:361:request_in_callback()) peer: 12345-192.168.204.3@tcp (source: 12345-192.168.204.3@tcp) 00000100:00100000:1.0:1776178097.504646:0:3346:0:(events.c:361:request_in_callback()) peer: 12345-192.168.204.3@tcp (source: 12345-192.168.204.3@tcp) 00000100:00100000:1.0:1776178097.504678:0:20762:0:(service.c:2152:ptlrpc_server_handle_req_in()) got req x1862456979080832 00000100:00100000:1.0:1776178097.504729:0:20762:0:(service.c:2152:ptlrpc_server_handle_req_in()) got req x1862456979080896 00000100:00100000:1.0:1776178097.504764:0:20762:0:(nrs_fifo.c:179:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.204.3@tcp, seq: 5240 00000100:00100000:1.0:1776178097.504770:0:20762:0:(service.c:2324:ptlrpc_server_handle_request()) Handling RPC req@00000000b5702a7a pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_013:b50963bd-8eba-4432-9a54-8888d00e3fa1+12:2273:x1862456979080832:12345-192.168.204.3@tcp:101:chmod.0 00010000:00010000:1.0:1776178097.504786:0:20762:0:(ldlm_lockd.c:1248:ldlm_handle_enqueue0()) ### server-side enqueue handler START 00010000:00010000:1.0:1776178097.504820:0:20762:0:(ldlm_lockd.c:1318:ldlm_handle_enqueue0()) ### server-side enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 000000002cc4819c/0xa40083ab9bf53094 lrc: 2/0,0 mode: --/PR res: [0x43:0x0:0x0].0x0 rrc: 2 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x771f68aa1eb279f2 expref: -99 pid: 20762 timeout: 0 lvb_type: 0 00002000:00010000:1.0:1776178097.504850:0:20762:0:(ofd_lvb.c:170:ofd_lvbo_init()) res: [0x100000000:0x43:0x0] initial LVB size: 0, mtime: 0x69de53af, atime: 0x69de53af, ctime: 0x69de53af, blocks: 0x0 00010000:00010000:1.0:1776178097.504872:0:20762:0:(ldlm_extent.c:318:ldlm_check_contention()) contended locks = 0 00010000:00010000:1.0:1776178097.504875:0:20762:0:(ldlm_extent.c:318:ldlm_check_contention()) contended locks = 0 00010000:00010000:1.0:1776178097.504887:0:20762:0:(ldlm_resource.c:1650:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 000000002cc4819c/0xa40083ab9bf53094 lrc: 3/0,0 mode: PR/PR res: [0x43:0x0:0x0].0x0 rrc: 2 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x40000000000000 nid: 192.168.204.3@tcp remote: 0x771f68aa1eb279f2 expref: 13 pid: 20762 timeout: 0 lvb_type: 1 00010000:00010000:1.0:1776178097.504910:0:20762:0:(ldlm_lockd.c:1484:ldlm_handle_enqueue0()) ### server-side enqueue handler, sending reply (err=0, rc=0) ns: filter-lustre-OST0000_UUID lock: 000000002cc4819c/0xa40083ab9bf53094 lrc: 3/0,0 mode: PR/PR res: [0x43:0x0:0x0].0x0 rrc: 2 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x40000000000000 nid: 192.168.204.3@tcp remote: 0x771f68aa1eb279f2 expref: 13 pid: 20762 timeout: 0 lvb_type: 1 00010000:00010000:1.0:1776178097.504929:0:20762:0:(ldlm_lockd.c:1565:ldlm_handle_enqueue0()) ### server-side enqueue handler END (lock 000000002cc4819c, rc 0) 00000100:00100000:1.0:1776178097.504973:0:20762:0:(service.c:2372:ptlrpc_server_handle_request()) Handled RPC req@00000000b5702a7a pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_013:b50963bd-8eba-4432-9a54-8888d00e3fa1+13:2273:x1862456979080832:12345-192.168.204.3@tcp:101:chmod.0 Request processed in 202us (377us total) trans 0 rc 0/0 00000100:00100000:1.0:1776178097.504981:0:20762:0:(nrs_fifo.c:241:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.204.3@tcp, seq: 5240 00000100:00100000:1.0:1776178097.504998:0:20762:0:(nrs_fifo.c:179:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.204.3@tcp, seq: 5241 00000100:00100000:1.0:1776178097.505004:0:20762:0:(service.c:2324:ptlrpc_server_handle_request()) Handling RPC req@000000009d5112f6 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_013:b50963bd-8eba-4432-9a54-8888d00e3fa1+12:2271:x1862456979080896:12345-192.168.204.3@tcp:101:chmod.0 00010000:00010000:1.0:1776178097.505011:0:20762:0:(ldlm_lockd.c:1248:ldlm_handle_enqueue0()) ### server-side enqueue handler START 00010000:00010000:1.0:1776178097.505023:0:20762:0:(ldlm_lockd.c:1318:ldlm_handle_enqueue0()) ### server-side enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 00000000a14e680b/0xa40083ab9bf5309b lrc: 2/0,0 mode: --/PR res: [0x45:0x0:0x0].0x0 rrc: 2 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x771f68aa1eb279f9 expref: -99 pid: 20762 timeout: 0 lvb_type: 0 00002000:00010000:1.0:1776178097.505038:0:20762:0:(ofd_lvb.c:170:ofd_lvbo_init()) res: [0x100000000:0x45:0x0] initial LVB size: 0, mtime: 0x69de53af, atime: 0x69de53af, ctime: 0x69de53af, blocks: 0x0 00010000:00010000:1.0:1776178097.505050:0:20762:0:(ldlm_extent.c:318:ldlm_check_contention()) contended locks = 0 00010000:00010000:1.0:1776178097.505661:0:20762:0:(ldlm_extent.c:318:ldlm_check_contention()) contended locks = 0 00010000:00010000:1.0:1776178097.505668:0:20762:0:(ldlm_resource.c:1650:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 00000000a14e680b/0xa40083ab9bf5309b lrc: 3/0,0 mode: PR/PR res: [0x45:0x0:0x0].0x0 rrc: 2 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x40000000000000 nid: 192.168.204.3@tcp remote: 0x771f68aa1eb279f9 expref: 13 pid: 20762 timeout: 0 lvb_type: 1 00010000:00010000:1.0:1776178097.505747:0:20762:0:(ldlm_lockd.c:1484:ldlm_handle_enqueue0()) ### server-side enqueue handler, sending reply (err=0, rc=0) ns: filter-lustre-OST0000_UUID lock: 00000000a14e680b/0xa40083ab9bf5309b lrc: 3/0,0 mode: PR/PR res: [0x45:0x0:0x0].0x0 rrc: 2 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x40000000000000 nid: 192.168.204.3@tcp remote: 0x771f68aa1eb279f9 expref: 13 pid: 20762 timeout: 0 lvb_type: 1 00010000:00010000:1.0:1776178097.505759:0:20762:0:(ldlm_lockd.c:1565:ldlm_handle_enqueue0()) ### server-side enqueue handler END (lock 00000000a14e680b, rc 0) 00000100:00100000:1.0:1776178097.505785:0:20762:0:(service.c:2372:ptlrpc_server_handle_request()) Handled RPC req@000000009d5112f6 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_013:b50963bd-8eba-4432-9a54-8888d00e3fa1+13:2271:x1862456979080896:12345-192.168.204.3@tcp:101:chmod.0 Request processed in 782us (1140us total) trans 0 rc 0/0 00000100:00100000:1.0:1776178097.505792:0:20762:0:(nrs_fifo.c:241:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.204.3@tcp, seq: 5241 00000100:00100000:1.0:1776178097.505924:0:6255:0:(service.c:2152:ptlrpc_server_handle_req_in()) got req x1862456979080768 00000100:00100000:1.0:1776178097.505965:0:6255:0:(nrs_fifo.c:179:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.204.3@tcp, seq: 6486 00000100:00100000:1.0:1776178097.505974:0:6255:0:(service.c:2324:ptlrpc_server_handle_request()) Handling RPC req@00000000ba273b2a pname:cluuid+ref:pid:xid:nid:opc:job mdt00_000:b50963bd-8eba-4432-9a54-8888d00e3fa1+62:17009:x1862456979080768:12345-192.168.204.3@tcp:36:chmod.0 00000020:02000000:1.0:1776178097.505988:0:6255:0:(libcfs_fail.h:96:cfs_fail_check_set()) *** cfs_fail_loc=159, val=0*** 00000100:00100000:1.0:1776178097.505991:0:6255:0:(service.c:2372:ptlrpc_server_handle_request()) Handled RPC req@00000000ba273b2a pname:cluuid+ref:pid:xid:nid:opc:job mdt00_000:b50963bd-8eba-4432-9a54-8888d00e3fa1+62:17009:x1862456979080768:12345-192.168.204.3@tcp:36:chmod.0 Request processed in 18us (1482us total) trans 0 rc 0/-999 00000100:00100000:1.0:1776178097.505997:0:6255:0:(nrs_fifo.c:241:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.204.3@tcp, seq: 6486 00000100:00100000:1.0:1776178097.519043:0:6256:0:(service.c:2152:ptlrpc_server_handle_req_in()) got req x1862456979081152 00000100:00100000:1.0:1776178097.519264:0:6255:0:(service.c:2152:ptlrpc_server_handle_req_in()) got req x1862456979081344 00000100:00100000:1.0:1776178097.519284:0:6255:0:(service.c:2152:ptlrpc_server_handle_req_in()) got req x1862456979081088 00000100:00100000:1.0:1776178097.519300:0:6255:0:(nrs_fifo.c:179:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.204.3@tcp, seq: 6487 00000100:00100000:1.0:1776178097.519307:0:6255:0:(service.c:2324:ptlrpc_server_handle_request()) Handling RPC req@000000008f01aa95 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_000:b50963bd-8eba-4432-9a54-8888d00e3fa1+64:17007:x1862456979081152:12345-192.168.204.3@tcp:36:chmod.0 00000020:02000000:1.0:1776178097.519326:0:6255:0:(libcfs_fail.h:96:cfs_fail_check_set()) *** cfs_fail_loc=159, val=0*** 00000100:00100000:1.0:1776178097.519330:0:6255:0:(service.c:2372:ptlrpc_server_handle_request()) Handled RPC req@000000008f01aa95 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_000:b50963bd-8eba-4432-9a54-8888d00e3fa1+64:17007:x1862456979081152:12345-192.168.204.3@tcp:36:chmod.0 Request processed in 23us (837us total) trans 0 rc 0/-999 00000100:00100000:1.0:1776178097.519336:0:6255:0:(nrs_fifo.c:241:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.204.3@tcp, seq: 6487 00000100:00100000:1.0:1776178097.519355:0:6255:0:(nrs_fifo.c:179:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.204.3@tcp, seq: 6488 00000100:00100000:1.0:1776178097.519359:0:6255:0:(service.c:2324:ptlrpc_server_handle_request()) Handling RPC req@0000000001b11bc0 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_000:b50963bd-8eba-4432-9a54-8888d00e3fa1+63:17008:x1862456979081344:12345-192.168.204.3@tcp:36:chmod.0 00000020:02000000:1.0:1776178097.519365:0:6255:0:(libcfs_fail.h:96:cfs_fail_check_set()) *** cfs_fail_loc=159, val=0*** 00000100:00100000:1.0:1776178097.519367:0:6255:0:(service.c:2372:ptlrpc_server_handle_request()) Handled RPC req@0000000001b11bc0 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_000:b50963bd-8eba-4432-9a54-8888d00e3fa1+63:17008:x1862456979081344:12345-192.168.204.3@tcp:36:chmod.0 Request processed in 8us (785us total) trans 0 rc 0/-999 00000100:00100000:1.0:1776178097.519372:0:6255:0:(nrs_fifo.c:241:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.204.3@tcp, seq: 6488 00000100:00100000:1.0:1776178097.519381:0:6255:0:(nrs_fifo.c:179:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.204.3@tcp, seq: 6489 00000100:00100000:1.0:1776178097.519384:0:6255:0:(service.c:2324:ptlrpc_server_handle_request()) Handling RPC req@000000005ec63ccb pname:cluuid+ref:pid:xid:nid:opc:job mdt00_000:b50963bd-8eba-4432-9a54-8888d00e3fa1+62:17011:x1862456979081088:12345-192.168.204.3@tcp:36:chmod.0 00000020:02000000:1.0:1776178097.519389:0:6255:0:(libcfs_fail.h:96:cfs_fail_check_set()) *** cfs_fail_loc=159, val=0*** 00000100:00100000:1.0:1776178097.519391:0:6255:0:(service.c:2372:ptlrpc_server_handle_request()) Handled RPC req@000000005ec63ccb pname:cluuid+ref:pid:xid:nid:opc:job mdt00_000:b50963bd-8eba-4432-9a54-8888d00e3fa1+62:17011:x1862456979081088:12345-192.168.204.3@tcp:36:chmod.0 Request processed in 7us (780us total) trans 0 rc 0/-999 00000100:00100000:1.0:1776178097.519396:0:6255:0:(nrs_fifo.c:241:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.204.3@tcp, seq: 6489 00000100:00100000:0.0:1776178097.527574:0:14440:0:(service.c:2152:ptlrpc_server_handle_req_in()) got req x1862456979081216 00000100:00100000:0.0:1776178097.527659:0:14440:0:(nrs_fifo.c:179:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.204.3@tcp, seq: 5242 00000100:00100000:0.0:1776178097.527670:0:14440:0:(service.c:2324:ptlrpc_server_handle_request()) Handling RPC req@00000000e1d2e300 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_004:b50963bd-8eba-4432-9a54-8888d00e3fa1+10:2273:x1862456979081216:12345-192.168.204.3@tcp:101:chmod.0 00010000:00010000:0.0:1776178097.527697:0:14440:0:(ldlm_lockd.c:1248:ldlm_handle_enqueue0()) ### server-side enqueue handler START 00010000:00010000:0.0:1776178097.527741:0:14440:0:(ldlm_lockd.c:1318:ldlm_handle_enqueue0()) ### server-side enqueue handler, new lock created ns: filter-lustre-OST0001_UUID lock: 000000007797f5e2/0xa40083ab9bf530a2 lrc: 2/0,0 mode: --/PR res: [0x44:0x0:0x0].0x0 rrc: 2 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x771f68aa1eb27a00 expref: -99 pid: 14440 timeout: 0 lvb_type: 0 00002000:00010000:0.0:1776178097.527766:0:14440:0:(ofd_lvb.c:170:ofd_lvbo_init()) res: [0x100010000:0x44:0x0] initial LVB size: 0, mtime: 0x69de53af, atime: 0x69de53af, ctime: 0x69de53af, blocks: 0x0 00010000:00010000:0.0:1776178097.527789:0:14440:0:(ldlm_extent.c:318:ldlm_check_contention()) contended locks = 0 00010000:00010000:0.0:1776178097.527793:0:14440:0:(ldlm_extent.c:318:ldlm_check_contention()) contended locks = 0 00010000:00010000:0.0:1776178097.527804:0:14440:0:(ldlm_resource.c:1650:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0001_UUID lock: 000000007797f5e2/0xa40083ab9bf530a2 lrc: 3/0,0 mode: PR/PR res: [0x44:0x0:0x0].0x0 rrc: 2 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x40000000000000 nid: 192.168.204.3@tcp remote: 0x771f68aa1eb27a00 expref: 11 pid: 14440 timeout: 0 lvb_type: 1 00010000:00010000:0.0:1776178097.527822:0:14440:0:(ldlm_lockd.c:1484:ldlm_handle_enqueue0()) ### server-side enqueue handler, sending reply (err=0, rc=0) ns: filter-lustre-OST0001_UUID lock: 000000007797f5e2/0xa40083ab9bf530a2 lrc: 3/0,0 mode: PR/PR res: [0x44:0x0:0x0].0x0 rrc: 2 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x40000000000000 nid: 192.168.204.3@tcp remote: 0x771f68aa1eb27a00 expref: 11 pid: 14440 timeout: 0 lvb_type: 1 00010000:00010000:0.0:1776178097.527843:0:14440:0:(ldlm_lockd.c:1565:ldlm_handle_enqueue0()) ### server-side enqueue handler END (lock 000000007797f5e2, rc 0) 00000100:00100000:0.0:1776178097.527936:0:14440:0:(service.c:2372:ptlrpc_server_handle_request()) Handled RPC req@00000000e1d2e300 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_004:b50963bd-8eba-4432-9a54-8888d00e3fa1+11:2273:x1862456979081216:12345-192.168.204.3@tcp:101:chmod.0 Request processed in 267us (9385us total) trans 0 rc 0/0 00000100:00100000:0.0:1776178097.527945:0:14440:0:(nrs_fifo.c:241:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.204.3@tcp, seq: 5242 00000100:00100000:0.0:1776178097.539407:0:6257:0:(service.c:2152:ptlrpc_server_handle_req_in()) got req x1862456979081472 00000100:00100000:0.0:1776178097.539517:0:6257:0:(nrs_fifo.c:179:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.204.3@tcp, seq: 6490 00000100:00100000:0.0:1776178097.539525:0:6257:0:(service.c:2324:ptlrpc_server_handle_request()) Handling RPC req@0000000084030222 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_002:b50963bd-8eba-4432-9a54-8888d00e3fa1+62:17010:x1862456979081472:12345-192.168.204.3@tcp:36:chmod.0 00000020:02000000:0.0:1776178097.539543:0:6257:0:(libcfs_fail.h:96:cfs_fail_check_set()) *** cfs_fail_loc=159, val=0*** 00000100:00100000:0.0:1776178097.539547:0:6257:0:(service.c:2372:ptlrpc_server_handle_request()) Handled RPC req@0000000084030222 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_002:b50963bd-8eba-4432-9a54-8888d00e3fa1+62:17010:x1862456979081472:12345-192.168.204.3@tcp:36:chmod.0 Request processed in 23us (608us total) trans 0 rc 0/-999 00000100:00100000:0.0:1776178097.539555:0:6257:0:(nrs_fifo.c:241:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.204.3@tcp, seq: 6490 00000100:00100000:1.0:1776178098.017354:0:3353:0:(client.c:1733:ptlrpc_send_new_req()) Sending RPC req@000000007fbc2285 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_00:lustre-MDT0000-mdtlov_UUID:3353:1862456998020288:0@lo:13:osp-pre-0-0.0 00000100:00100000:1.0:1776178098.017441:0:3353:0:(events.c:361:request_in_callback()) peer: 12345-0@lo (source: 12345-0@lo) 00000100:00100000:2.0:1776178098.018257:0:8232:0:(service.c:2152:ptlrpc_server_handle_req_in()) got req x1862456998020288 00000100:00100000:2.0:1776178098.018322:0:8232:0:(nrs_fifo.c:179:nrs_fifo_req_get()) NRS start fifo request from 12345-0@lo, seq: 395 00000100:00100000:2.0:1776178098.018332:0:8232:0:(service.c:2324:ptlrpc_server_handle_request()) Handling RPC req@00000000a59bfc89 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_create00:lustre-MDT0000-mdtlov_UUID+5:3353:x1862456998020288:12345-0@lo:13:osp-pre-0-0.0 00002000:00000024:2.0:1776178098.018380:0:8232:0:(ofd_obd.c:740:ofd_statfs()) blocks cached 315392 granted 484883776 pending 0 free 3923689472 avail 3692167168 00000100:00100000:2.0:1776178098.018456:0:8232:0:(service.c:2372:ptlrpc_server_handle_request()) Handled RPC req@00000000a59bfc89 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_create00:lustre-MDT0000-mdtlov_UUID+5:3353:x1862456998020288:12345-0@lo:13:osp-pre-0-0.0 Request processed in 125us (1017us total) trans 0 rc 0/0 00000100:00100000:2.0:1776178098.018464:0:8232:0:(nrs_fifo.c:241:nrs_fifo_req_stop()) NRS stop fifo request from 12345-0@lo, seq: 395 00000100:00100000:2.0:1776178098.018498:0:3354:0:(client.c:1733:ptlrpc_send_new_req()) Sending RPC req@000000005c76b1b3 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_01:ac61f839-0e8e-4cac-9c59-08bcfbbad330:3354:1862456998020352:0@lo:400:kworker/u8:2.0 00000100:00100000:2.0:1776178098.018544:0:3354:0:(events.c:361:request_in_callback()) peer: 12345-0@lo (source: 12345-0@lo) 00000100:00100000:2.0:1776178098.018567:0:3354:0:(client.c:1733:ptlrpc_send_new_req()) Sending RPC req@0000000056dbc32b pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_01:lustre-MDT0001-lwp-OST0001_UUID:3354:1862456998020608:0@lo:400:kworker/u8:2.0 00000100:00100000:1.0:1776178098.018585:0:3353:0:(client.c:1733:ptlrpc_send_new_req()) Sending RPC req@000000006e48e9ca pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_00:lustre-MDT0000-lwp-OST0001_UUID:3353:1862456998020544:0@lo:400:kworker/u8:2.0 00000100:00100000:2.0:1776178098.018601:0:3354:0:(events.c:361:request_in_callback()) peer: 12345-0@lo (source: 12345-0@lo) 00000100:00100000:2.0:1776178098.018623:0:3354:0:(client.c:1733:ptlrpc_send_new_req()) Sending RPC req@0000000097ae2a31 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_01:lustre-MDT0000-lwp-MDT0001_UUID:3354:1862456998020864:0@lo:400:kworker/u8:2.0 00000100:00100000:1.0:1776178098.018625:0:3353:0:(events.c:361:request_in_callback()) peer: 12345-0@lo (source: 12345-0@lo) 00000100:00100000:1.0:1776178098.018651:0:3353:0:(client.c:1733:ptlrpc_send_new_req()) Sending RPC req@00000000956deaca pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_00:lustre-MDT0001-mdtlov_UUID:3353:1862456998020800:0@lo:400:kworker/u8:2.0 00000100:00100000:2.0:1776178098.018658:0:3354:0:(events.c:361:request_in_callback()) peer: 12345-0@lo (source: 12345-0@lo) 00000100:00100000:2.0:1776178098.018684:0:3354:0:(client.c:1733:ptlrpc_send_new_req()) Sending RPC req@00000000e64148ab pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_01:lustre-MDT0001-mdtlov_UUID:3354:1862456998021120:0@lo:41:osp-pre-0-1.0 00000100:00100000:1.0:1776178098.018697:0:3353:0:(events.c:361:request_in_callback()) peer: 12345-0@lo (source: 12345-0@lo) 00000100:00100000:2.0:1776178098.018716:0:3354:0:(events.c:361:request_in_callback()) peer: 12345-0@lo (source: 12345-0@lo) 00000100:00100000:1.0:1776178098.018730:0:3353:0:(client.c:1733:ptlrpc_send_new_req()) Sending RPC req@0000000045560455 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_00:lustre-MDT0000-lwp-MDT0000_UUID:3353:1862456998021056:0@lo:400:kworker/u8:2.0 00000100:00100000:1.0:1776178098.018759:0:3353:0:(events.c:361:request_in_callback()) peer: 12345-0@lo (source: 12345-0@lo) 00000100:00100000:0.0:1776178098.018841:0:3356:0:(client.c:1733:ptlrpc_send_new_req()) Sending RPC req@00000000badbd310 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_03:lustre-MDT0001-lwp-OST0000_UUID:3356:1862456998020480:0@lo:400:kworker/u8:2.0 00000100:00100000:0.0:1776178098.018870:0:3356:0:(events.c:361:request_in_callback()) peer: 12345-0@lo (source: 12345-0@lo) 00000100:00100000:0.0:1776178098.019142:0:3356:0:(client.c:1733:ptlrpc_send_new_req()) Sending RPC req@000000007fcab602 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_03:lustre-MDT0001-mdtlov_UUID:3356:1862456998020736:0@lo:400:kworker/u8:2.0 00000100:00100000:0.0:1776178098.019168:0:3356:0:(events.c:361:request_in_callback()) peer: 12345-0@lo (source: 12345-0@lo) 00000100:00100000:0.0:1776178098.019183:0:3356:0:(client.c:1733:ptlrpc_send_new_req()) Sending RPC req@000000007a321680 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_03:lustre-MDT0000-mdtlov_UUID:3356:1862456998020992:0@lo:400:kworker/u8:2.0 00000100:00100000:0.0:1776178098.019203:0:3356:0:(events.c:361:request_in_callback()) peer: 12345-0@lo (source: 12345-0@lo) 00000100:00100000:0.0:1776178098.019223:0:3356:0:(client.c:1733:ptlrpc_send_new_req()) Sending RPC req@000000007cbab86b pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_03:lustre-MDT0000-mdtlov_UUID:3356:1862456998021248:0@lo:41:osp-pre-1-0.0 00000100:00100000:0.0:1776178098.019242:0:3356:0:(events.c:361:request_in_callback()) peer: 12345-0@lo (source: 12345-0@lo) 00000100:00100000:0.0:1776178098.019274:0:20140:0:(service.c:2152:ptlrpc_server_handle_req_in()) got req x1862456998020352 00000100:00100000:0.0:1776178098.019305:0:20140:0:(nrs_fifo.c:179:nrs_fifo_req_get()) NRS start fifo request from 12345-0@lo, seq: 113 00000100:00100000:0.0:1776178098.019310:0:20140:0:(service.c:2324:ptlrpc_server_handle_request()) Handling RPC req@000000005f72129d pname:cluuid+ref:pid:xid:nid:opc:job ll_mgs_0002:ac61f839-0e8e-4cac-9c59-08bcfbbad330+23:3354:x1862456998020352:12345-0@lo:400:kworker/u8:2.0 00000100:00100000:0.0:1776178098.019349:0:20140:0:(service.c:2372:ptlrpc_server_handle_request()) Handled RPC req@000000005f72129d pname:cluuid+ref:pid:xid:nid:opc:job ll_mgs_0002:ac61f839-0e8e-4cac-9c59-08bcfbbad330+23:3354:x1862456998020352:12345-0@lo:400:kworker/u8:2.0 Request processed in 38us (806us total) trans 0 rc 0/0 00000100:00100000:0.0:1776178098.019355:0:20140:0:(nrs_fifo.c:241:nrs_fifo_req_stop()) NRS stop fifo request from 12345-0@lo, seq: 113 00000100:00100000:0.0:1776178098.019385:0:6260:0:(service.c:2152:ptlrpc_server_handle_req_in()) got req x1862456998021120 00000100:00100000:0.0:1776178098.019406:0:6260:0:(service.c:2152:ptlrpc_server_handle_req_in()) got req x1862456998021248 00000100:00100000:0.0:1776178098.019418:0:6260:0:(nrs_fifo.c:179:nrs_fifo_req_get()) NRS start fifo request from 12345-0@lo, seq: 609 00000100:00100000:0.0:1776178098.019422:0:6260:0:(service.c:2324:ptlrpc_server_handle_request()) Handling RPC req@00000000a33eecf4 pname:cluuid+ref:pid:xid:nid:opc:job mdt_out00_000:lustre-MDT0001-mdtlov_UUID+6:3354:x1862456998021120:12345-0@lo:41:osp-pre-0-1.0 00000004:00000024:0.0:1776178098.019566:0:6260:0:(mdt_handler.c:548:mdt_statfs()) blocks cached 0 granted 2097152 pending 0 free 1446121472 avail 1316659200 00000100:00100000:0.0:1776178098.019598:0:6260:0:(service.c:2372:ptlrpc_server_handle_request()) Handled RPC req@00000000a33eecf4 pname:cluuid+ref:pid:xid:nid:opc:job mdt_out00_000:lustre-MDT0001-mdtlov_UUID+6:3354:x1862456998021120:12345-0@lo:41:osp-pre-0-1.0 Request processed in 175us (884us total) trans 0 rc 0/0 00000100:00100000:0.0:1776178098.019604:0:6260:0:(nrs_fifo.c:241:nrs_fifo_req_stop()) NRS stop fifo request from 12345-0@lo, seq: 609 00000100:00100000:0.0:1776178098.019681:0:6260:0:(nrs_fifo.c:179:nrs_fifo_req_get()) NRS start fifo request from 12345-0@lo, seq: 610 00000100:00100000:0.0:1776178098.019686:0:6260:0:(service.c:2324:ptlrpc_server_handle_request()) Handling RPC req@0000000069ebab60 pname:cluuid+ref:pid:xid:nid:opc:job mdt_out00_000:lustre-MDT0000-mdtlov_UUID+5:3356:x1862456998021248:12345-0@lo:41:osp-pre-1-0.0 00000004:00000024:0.0:1776178098.019702:0:6260:0:(mdt_handler.c:548:mdt_statfs()) blocks cached 0 granted 2097152 pending 0 free 1446039552 avail 1316577280 00000100:00100000:0.0:1776178098.019744:0:6260:0:(service.c:2372:ptlrpc_server_handle_request()) Handled RPC req@0000000069ebab60 pname:cluuid+ref:pid:xid:nid:opc:job mdt_out00_000:lustre-MDT0000-mdtlov_UUID+5:3356:x1862456998021248:12345-0@lo:41:osp-pre-1-0.0 Request processed in 57us (501us total) trans 0 rc 0/0 00000100:00100000:0.0:1776178098.019750:0:6260:0:(nrs_fifo.c:241:nrs_fifo_req_stop()) NRS stop fifo request from 12345-0@lo, seq: 610 00000100:00100000:0.0:1776178098.019870:0:6255:0:(service.c:2152:ptlrpc_server_handle_req_in()) got req x1862456998020544 00000100:00100000:2.0:1776178098.019870:0:11281:0:(service.c:2152:ptlrpc_server_handle_req_in()) got req x1862456998020608 00000100:00100000:0.0:1776178098.019891:0:6255:0:(service.c:2152:ptlrpc_server_handle_req_in()) got req x1862456998020864 00000100:00100000:2.0:1776178098.019891:0:11281:0:(service.c:2152:ptlrpc_server_handle_req_in()) got req x1862456998021056 00000100:00100000:0.0:1776178098.019900:0:6255:0:(service.c:2152:ptlrpc_server_handle_req_in()) got req x1862456998020480 00000100:00100000:2.0:1776178098.019903:0:11281:0:(nrs_fifo.c:179:nrs_fifo_req_get()) NRS start fifo request from 12345-0@lo, seq: 739 00000100:00100000:2.0:1776178098.019907:0:11281:0:(service.c:2324:ptlrpc_server_handle_request()) Handling RPC req@00000000bacf691a pname:cluuid+ref:pid:xid:nid:opc:job mdt00_004:lustre-MDT0001-lwp-OST0001_UUID+5:3354:x1862456998020608:12345-0@lo:400:kworker/u8:2.0 00000100:00100000:0.0:1776178098.019909:0:6255:0:(nrs_fifo.c:179:nrs_fifo_req_get()) NRS start fifo request from 12345-0@lo, seq: 740 00000100:00100000:0.0:1776178098.019912:0:6255:0:(service.c:2324:ptlrpc_server_handle_request()) Handling RPC req@00000000a30eb964 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_000:lustre-MDT0000-lwp-OST0001_UUID+5:3353:x1862456998020544:12345-0@lo:400:kworker/u8:2.0 00000100:00100000:0.0:1776178098.019938:0:6255:0:(service.c:2372:ptlrpc_server_handle_request()) Handled RPC req@00000000a30eb964 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_000:lustre-MDT0000-lwp-OST0001_UUID+5:3353:x1862456998020544:12345-0@lo:400:kworker/u8:2.0 Request processed in 26us (1317us total) trans 0 rc 0/0 00000100:00100000:2.0:1776178098.019938:0:11281:0:(service.c:2372:ptlrpc_server_handle_request()) Handled RPC req@00000000bacf691a pname:cluuid+ref:pid:xid:nid:opc:job mdt00_004:lustre-MDT0001-lwp-OST0001_UUID+5:3354:x1862456998020608:12345-0@lo:400:kworker/u8:2.0 Request processed in 31us (1339us total) trans 0 rc 0/0 00000100:00100000:0.0:1776178098.019944:0:6255:0:(nrs_fifo.c:241:nrs_fifo_req_stop()) NRS stop fifo request from 12345-0@lo, seq: 740 00000100:00100000:1.0:1776178098.020016:0:3353:0:(client.c:2215:ptlrpc_check_set()) Completed RPC req@000000007fbc2285 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_00:lustre-MDT0000-mdtlov_UUID:3353:1862456998020288:0@lo:13:osp-pre-0-0.0 00000100:00100000:1.0:1776178098.020055:0:3353:0:(client.c:2215:ptlrpc_check_set()) Completed RPC req@000000006e48e9ca pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_00:lustre-MDT0000-lwp-OST0001_UUID:3353:1862456998020544:0@lo:400:kworker/u8:2.0 00000100:00100000:2.0:1776178098.023198:0:11281:0:(nrs_fifo.c:241:nrs_fifo_req_stop()) NRS stop fifo request from 12345-0@lo, seq: 739 00000100:00100000:2.0:1776178098.023250:0:11281:0:(nrs_fifo.c:179:nrs_fifo_req_get()) NRS start fifo request from 12345-0@lo, seq: 741 00000100:00100000:2.0:1776178098.023270:0:11281:0:(service.c:2324:ptlrpc_server_handle_request()) Handling RPC req@00000000e178f924 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_004:lustre-MDT0000-lwp-MDT0001_UUID+5:3354:x1862456998020864:12345-0@lo:400:kworker/u8:2.0 00000100:00100000:2.0:1776178098.023397:0:11281:0:(service.c:2372:ptlrpc_server_handle_request()) Handled RPC req@00000000e178f924 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_004:lustre-MDT0000-lwp-MDT0001_UUID+5:3354:x1862456998020864:12345-0@lo:400:kworker/u8:2.0 Request processed in 139us (4740us total) trans 0 rc 0/0 00000100:00100000:2.0:1776178098.023404:0:11281:0:(nrs_fifo.c:241:nrs_fifo_req_stop()) NRS stop fifo request from 12345-0@lo, seq: 741 00000100:00100000:2.0:1776178098.023419:0:11281:0:(nrs_fifo.c:179:nrs_fifo_req_get()) NRS start fifo request from 12345-0@lo, seq: 742 00000100:00100000:2.0:1776178098.023424:0:11281:0:(service.c:2324:ptlrpc_server_handle_request()) Handling RPC req@00000000b766edf6 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_004:lustre-MDT0000-lwp-MDT0000_UUID+5:3353:x1862456998021056:12345-0@lo:400:kworker/u8:2.0 00000100:00100000:2.0:1776178098.023485:0:11281:0:(service.c:2372:ptlrpc_server_handle_request()) Handled RPC req@00000000b766edf6 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_004:lustre-MDT0000-lwp-MDT0000_UUID+5:3353:x1862456998021056:12345-0@lo:400:kworker/u8:2.0 Request processed in 60us (4727us total) trans 0 rc 0/0 00000100:00100000:2.0:1776178098.023492:0:11281:0:(nrs_fifo.c:241:nrs_fifo_req_stop()) NRS stop fifo request from 12345-0@lo, seq: 742 00000100:00100000:2.0:1776178098.023504:0:11281:0:(nrs_fifo.c:179:nrs_fifo_req_get()) NRS start fifo request from 12345-0@lo, seq: 743 00000100:00100000:2.0:1776178098.023509:0:11281:0:(service.c:2324:ptlrpc_server_handle_request()) Handling RPC req@0000000097413742 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_004:lustre-MDT0001-lwp-OST0000_UUID+5:3356:x1862456998020480:12345-0@lo:400:kworker/u8:2.0 00000100:00100000:2.0:1776178098.023543:0:11281:0:(service.c:2372:ptlrpc_server_handle_request()) Handled RPC req@0000000097413742 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_004:lustre-MDT0001-lwp-OST0000_UUID+5:3356:x1862456998020480:12345-0@lo:400:kworker/u8:2.0 Request processed in 34us (4672us total) trans 0 rc 0/0 00000100:00100000:2.0:1776178098.023554:0:11281:0:(nrs_fifo.c:241:nrs_fifo_req_stop()) NRS stop fifo request from 12345-0@lo, seq: 743 00000100:00100000:2.0:1776178098.023653:0:20761:0:(service.c:2152:ptlrpc_server_handle_req_in()) got req x1862456998020800 00000100:00100000:2.0:1776178098.023696:0:20761:0:(service.c:2152:ptlrpc_server_handle_req_in()) got req x1862456998020736 00000100:00100000:2.0:1776178098.023712:0:20761:0:(service.c:2152:ptlrpc_server_handle_req_in()) got req x1862456998020992 00000100:00100000:2.0:1776178098.023743:0:20761:0:(nrs_fifo.c:179:nrs_fifo_req_get()) NRS start fifo request from 12345-0@lo, seq: 398 00000100:00100000:2.0:1776178098.023747:0:20761:0:(service.c:2324:ptlrpc_server_handle_request()) Handling RPC req@0000000067671fc8 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_012:lustre-MDT0001-mdtlov_UUID+44:3353:x1862456998020800:12345-0@lo:400:kworker/u8:2.0 00000100:00100000:2.0:1776178098.023783:0:20761:0:(service.c:2372:ptlrpc_server_handle_request()) Handled RPC req@0000000067671fc8 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_012:lustre-MDT0001-mdtlov_UUID+44:3353:x1862456998020800:12345-0@lo:400:kworker/u8:2.0 Request processed in 36us (5088us total) trans 0 rc 0/0 00000100:00100000:2.0:1776178098.023789:0:20761:0:(nrs_fifo.c:241:nrs_fifo_req_stop()) NRS stop fifo request from 12345-0@lo, seq: 398 00000100:00100000:2.0:1776178098.023799:0:20761:0:(nrs_fifo.c:179:nrs_fifo_req_get()) NRS start fifo request from 12345-0@lo, seq: 399 00000100:00100000:2.0:1776178098.023802:0:20761:0:(service.c:2324:ptlrpc_server_handle_request()) Handling RPC req@0000000056080284 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_012:lustre-MDT0001-mdtlov_UUID+51:3356:x1862456998020736:12345-0@lo:400:kworker/u8:2.0 00000100:00100000:2.0:1776178098.023825:0:20761:0:(service.c:2372:ptlrpc_server_handle_request()) Handled RPC req@0000000056080284 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_012:lustre-MDT0001-mdtlov_UUID+51:3356:x1862456998020736:12345-0@lo:400:kworker/u8:2.0 Request processed in 22us (4657us total) trans 0 rc 0/0 00000100:00100000:2.0:1776178098.023830:0:20761:0:(nrs_fifo.c:241:nrs_fifo_req_stop()) NRS stop fifo request from 12345-0@lo, seq: 399 00000100:00100000:2.0:1776178098.023839:0:20761:0:(nrs_fifo.c:179:nrs_fifo_req_get()) NRS start fifo request from 12345-0@lo, seq: 400 00000100:00100000:2.0:1776178098.023842:0:20761:0:(service.c:2324:ptlrpc_server_handle_request()) Handling RPC req@000000006be5e5b9 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_012:lustre-MDT0000-mdtlov_UUID+5:3356:x1862456998020992:12345-0@lo:400:kworker/u8:2.0 00000100:00100000:2.0:1776178098.023867:0:20761:0:(service.c:2372:ptlrpc_server_handle_request()) Handled RPC req@000000006be5e5b9 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_012:lustre-MDT0000-mdtlov_UUID+5:3356:x1862456998020992:12345-0@lo:400:kworker/u8:2.0 Request processed in 24us (4664us total) trans 0 rc 0/0 00000100:00100000:2.0:1776178098.023872:0:20761:0:(nrs_fifo.c:241:nrs_fifo_req_stop()) NRS stop fifo request from 12345-0@lo, seq: 400 00000100:00100000:2.0:1776178098.023903:0:3355:0:(events.c:361:request_in_callback()) peer: 12345-0@lo (source: 12345-0@lo) 00000100:00100000:2.0:1776178098.023940:0:3355:0:(client.c:1733:ptlrpc_send_new_req()) Sending RPC req@00000000f4d20636 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_02:lustre-MDT0001-mdtlov_UUID:3355:1862456998020672:0@lo:400:kworker/u8:2.0 00000100:00100000:2.0:1776178098.023982:0:3355:0:(events.c:361:request_in_callback()) peer: 12345-0@lo (source: 12345-0@lo) 00000100:00100000:2.0:1776178098.024000:0:3355:0:(client.c:1733:ptlrpc_send_new_req()) Sending RPC req@00000000efb13deb pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_02:lustre-MDT0000-mdtlov_UUID:3355:1862456998020928:0@lo:400:kworker/u8:2.0 00000100:00100000:2.0:1776178098.024041:0:3355:0:(events.c:361:request_in_callback()) peer: 12345-0@lo (source: 12345-0@lo) 00000100:00100000:2.0:1776178098.024091:0:3355:0:(client.c:1733:ptlrpc_send_new_req()) Sending RPC req@000000002cf093e4 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_02:lustre-MDT0000-mdtlov_UUID:3355:1862456998021184:0@lo:13:osp-pre-1-0.0 00000100:00100000:2.0:1776178098.024121:0:3355:0:(events.c:361:request_in_callback()) peer: 12345-0@lo (source: 12345-0@lo) 00000100:00100000:2.0:1776178098.024162:0:8231:0:(service.c:2152:ptlrpc_server_handle_req_in()) got req x1862456998021184 00000100:00100000:2.0:1776178098.024188:0:8231:0:(nrs_fifo.c:179:nrs_fifo_req_get()) NRS start fifo request from 12345-0@lo, seq: 396 00000100:00100000:2.0:1776178098.024193:0:8231:0:(service.c:2324:ptlrpc_server_handle_request()) Handling RPC req@000000000154171f pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_create00:lustre-MDT0000-mdtlov_UUID+5:3355:x1862456998021184:12345-0@lo:13:osp-pre-1-0.0 00002000:00000024:2.0:1776178098.024220:0:8231:0:(ofd_obd.c:740:ofd_statfs()) blocks cached 229376 granted 491746368 pending 0 free 3923689472 avail 3692167168 00000100:00100000:2.0:1776178098.024258:0:8231:0:(service.c:2372:ptlrpc_server_handle_request()) Handled RPC req@000000000154171f pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_create00:lustre-MDT0000-mdtlov_UUID+5:3355:x1862456998021184:12345-0@lo:13:osp-pre-1-0.0 Request processed in 64us (136us total) trans 0 rc 0/0 00000100:00100000:2.0:1776178098.024265:0:8231:0:(nrs_fifo.c:241:nrs_fifo_req_stop()) NRS stop fifo request from 12345-0@lo, seq: 396 00000100:00100000:2.0:1776178098.024285:0:15213:0:(service.c:2152:ptlrpc_server_handle_req_in()) got req x1862456998020672 00000100:00100000:2.0:1776178098.024303:0:15213:0:(service.c:2152:ptlrpc_server_handle_req_in()) got req x1862456998020928 00000100:00100000:2.0:1776178098.024313:0:15213:0:(nrs_fifo.c:179:nrs_fifo_req_get()) NRS start fifo request from 12345-0@lo, seq: 611 00000100:00100000:2.0:1776178098.024317:0:15213:0:(service.c:2324:ptlrpc_server_handle_request()) Handling RPC req@00000000d1e0f01c pname:cluuid+ref:pid:xid:nid:opc:job mdt_out00_003:lustre-MDT0001-mdtlov_UUID+6:3355:x1862456998020672:12345-0@lo:400:kworker/u8:2.0 00000100:00100000:2.0:1776178098.024344:0:15213:0:(service.c:2372:ptlrpc_server_handle_request()) Handled RPC req@00000000d1e0f01c pname:cluuid+ref:pid:xid:nid:opc:job mdt_out00_003:lustre-MDT0001-mdtlov_UUID+6:3355:x1862456998020672:12345-0@lo:400:kworker/u8:2.0 Request processed in 27us (362us total) trans 0 rc 0/0 00000100:00100000:2.0:1776178098.024349:0:15213:0:(nrs_fifo.c:241:nrs_fifo_req_stop()) NRS stop fifo request from 12345-0@lo, seq: 611 00000100:00100000:2.0:1776178098.024360:0:15213:0:(nrs_fifo.c:179:nrs_fifo_req_get()) NRS start fifo request from 12345-0@lo, seq: 612 00000100:00100000:2.0:1776178098.024363:0:15213:0:(service.c:2324:ptlrpc_server_handle_request()) Handling RPC req@00000000aba13576 pname:cluuid+ref:pid:xid:nid:opc:job mdt_out00_003:lustre-MDT0000-mdtlov_UUID+5:3355:x1862456998020928:12345-0@lo:400:kworker/u8:2.0 00000100:00100000:2.0:1776178098.024386:0:15213:0:(service.c:2372:ptlrpc_server_handle_request()) Handled RPC req@00000000aba13576 pname:cluuid+ref:pid:xid:nid:opc:job mdt_out00_003:lustre-MDT0000-mdtlov_UUID+5:3355:x1862456998020928:12345-0@lo:400:kworker/u8:2.0 Request processed in 21us (344us total) trans 0 rc 0/0 00000100:00100000:2.0:1776178098.024391:0:15213:0:(nrs_fifo.c:241:nrs_fifo_req_stop()) NRS stop fifo request from 12345-0@lo, seq: 612 00000100:00100000:2.0:1776178098.024410:0:6257:0:(service.c:2152:ptlrpc_server_handle_req_in()) got req x1862456998020416 00000100:00100000:2.0:1776178098.024429:0:6257:0:(nrs_fifo.c:179:nrs_fifo_req_get()) NRS start fifo request from 12345-0@lo, seq: 744 00000100:00100000:2.0:1776178098.024433:0:6257:0:(service.c:2324:ptlrpc_server_handle_request()) Handling RPC req@00000000a0f2a112 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_002:lustre-MDT0000-lwp-OST0000_UUID+5:3355:x1862456998020416:12345-0@lo:400:kworker/u8:2.0 00000100:00100000:2.0:1776178098.024460:0:6257:0:(service.c:2372:ptlrpc_server_handle_request()) Handled RPC req@00000000a0f2a112 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_002:lustre-MDT0000-lwp-OST0000_UUID+5:3355:x1862456998020416:12345-0@lo:400:kworker/u8:2.0 Request processed in 27us (559us total) trans 0 rc 0/0 00000100:00100000:2.0:1776178098.024466:0:6257:0:(nrs_fifo.c:241:nrs_fifo_req_stop()) NRS stop fifo request from 12345-0@lo, seq: 744 00000100:00100000:2.0:1776178098.024901:0:3355:0:(client.c:2215:ptlrpc_check_set()) Completed RPC req@00000000f8c48966 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_02:lustre-MDT0000-lwp-OST0000_UUID:3355:1862456998020416:0@lo:400:kworker/u8:2.0 00000100:00100000:2.0:1776178098.024920:0:3355:0:(client.c:2215:ptlrpc_check_set()) Completed RPC req@00000000f4d20636 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_02:lustre-MDT0001-mdtlov_UUID:3355:1862456998020672:0@lo:400:kworker/u8:2.0 00000100:00100000:2.0:1776178098.024928:0:3355:0:(client.c:2215:ptlrpc_check_set()) Completed RPC req@00000000efb13deb pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_02:lustre-MDT0000-mdtlov_UUID:3355:1862456998020928:0@lo:400:kworker/u8:2.0 00000100:00100000:2.0:1776178098.024943:0:3355:0:(client.c:2215:ptlrpc_check_set()) Completed RPC req@000000002cf093e4 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_02:lustre-MDT0000-mdtlov_UUID:3355:1862456998021184:0@lo:13:osp-pre-1-0.0 00000100:00100000:2.0:1776178098.025127:0:3354:0:(client.c:2215:ptlrpc_check_set()) Completed RPC req@00000000e64148ab pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_01:lustre-MDT0001-mdtlov_UUID:3354:1862456998021120:0@lo:41:osp-pre-0-1.0 00000100:00100000:2.0:1776178098.025140:0:3354:0:(client.c:2215:ptlrpc_check_set()) Completed RPC req@000000005c76b1b3 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_01:ac61f839-0e8e-4cac-9c59-08bcfbbad330:3354:1862456998020352:0@lo:400:kworker/u8:2.0 00000100:00100000:2.0:1776178098.025150:0:3354:0:(client.c:2215:ptlrpc_check_set()) Completed RPC req@0000000056dbc32b pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_01:lustre-MDT0001-lwp-OST0001_UUID:3354:1862456998020608:0@lo:400:kworker/u8:2.0 00000100:00100000:2.0:1776178098.025161:0:3354:0:(client.c:2215:ptlrpc_check_set()) Completed RPC req@0000000097ae2a31 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_01:lustre-MDT0000-lwp-MDT0001_UUID:3354:1862456998020864:0@lo:400:kworker/u8:2.0 00000100:00100000:2.0:1776178098.025201:0:3356:0:(client.c:2215:ptlrpc_check_set()) Completed RPC req@00000000badbd310 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_03:lustre-MDT0001-lwp-OST0000_UUID:3356:1862456998020480:0@lo:400:kworker/u8:2.0 00000100:00100000:2.0:1776178098.025214:0:3356:0:(client.c:2215:ptlrpc_check_set()) Completed RPC req@000000007fcab602 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_03:lustre-MDT0001-mdtlov_UUID:3356:1862456998020736:0@lo:400:kworker/u8:2.0 00000100:00100000:2.0:1776178098.025222:0:3356:0:(client.c:2215:ptlrpc_check_set()) Completed RPC req@000000007a321680 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_03:lustre-MDT0000-mdtlov_UUID:3356:1862456998020992:0@lo:400:kworker/u8:2.0 00000100:00100000:2.0:1776178098.025232:0:3356:0:(client.c:2215:ptlrpc_check_set()) Completed RPC req@000000007cbab86b pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_03:lustre-MDT0000-mdtlov_UUID:3356:1862456998021248:0@lo:41:osp-pre-1-0.0 00000100:00100000:2.0:1776178098.025296:0:3353:0:(client.c:2215:ptlrpc_check_set()) Completed RPC req@00000000956deaca pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_00:lustre-MDT0001-mdtlov_UUID:3353:1862456998020800:0@lo:400:kworker/u8:2.0 00000100:00100000:2.0:1776178098.025308:0:3353:0:(client.c:2215:ptlrpc_check_set()) Completed RPC req@0000000045560455 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_00:lustre-MDT0000-lwp-MDT0000_UUID:3353:1862456998021056:0@lo:400:kworker/u8:2.0 00000001:00080000:2.0:1776178098.541392:0:9217:0:(tgt_lastrcvd.c:913:tgt_cb_last_committed()) lustre-OST0001: transno 4294969317 is committed 00000001:00080000:2.0:1776178098.541408:0:9217:0:(tgt_lastrcvd.c:913:tgt_cb_last_committed()) lustre-OST0001: transno 4294969314 is committed 00000020:00080000:0.0:1776178098.546234:0:18097:0:(update_trans.c:1350:distribute_txn_batchid_cb()) lustre-MDT0001: 12884901895 batchid updated 00000020:00080000:1.0:1776178098.546322:0:18116:0:(update_trans.c:1589:distribute_txn_commit_thread()) lustre-MDT0001: batchid: 0 committed batchid 12884901895 00000040:00100000:1.0:1776178098.546337:0:18116:0:(llog.c:235:llog_cancel_arr_rec()) Canceling 1 records, first 7 in log [0x200000bd1:0x1:0x0] 00000004:00080000:1.0:1776178098.546411:0:18116:0:(osp_trans.c:1598:osp_trans_stop()) lustre-MDT0000-osp-MDT0001: add oth 0000000068ab4787 with version 19 00000020:00080000:1.0:1776178098.546439:0:18116:0:(update_trans.c:1312:distribute_txn_cancel_records()) lustre-MDT0000-osp-MDT0001: batchid 12884901895 cancel update log [0x200000bd1:0x1:0x0].7: rc = 0 00000040:00100000:1.0:1776178098.546447:0:18116:0:(llog.c:235:llog_cancel_arr_rec()) Canceling 1 records, first 8 in log [0x240001b70:0x1:0x0] 00000020:00080000:1.0:1776178098.548617:0:18116:0:(update_trans.c:1312:distribute_txn_cancel_records()) lustre-MDT0001-osd: batchid 12884901895 cancel update log [0x240001b70:0x1:0x0].8: rc = 0 00000020:00080000:1.0:1776178098.548648:0:18116:0:(update_trans.c:1589:distribute_txn_commit_thread()) lustre-MDT0001: batchid: 0 committed batchid 12884901895 00000100:00100000:2.0:1776178098.549274:0:6260:0:(service.c:2152:ptlrpc_server_handle_req_in()) got req x1862456998021312 00000100:00100000:2.0:1776178098.549357:0:6260:0:(nrs_fifo.c:179:nrs_fifo_req_get()) NRS start fifo request from 12345-0@lo, seq: 613 00000100:00100000:2.0:1776178098.549365:0:6260:0:(service.c:2324:ptlrpc_server_handle_request()) Handling RPC req@000000004dd93690 pname:cluuid+ref:pid:xid:nid:opc:job mdt_out00_000:lustre-MDT0001-mdtlov_UUID+6:18102:x1862456998021312:12345-0@lo:1000:osp_up0-1.0 00000100:00100000:2.0:1776178098.552358:0:6260:0:(service.c:2372:ptlrpc_server_handle_request()) Handled RPC req@000000004dd93690 pname:cluuid+ref:pid:xid:nid:opc:job mdt_out00_000:lustre-MDT0001-mdtlov_UUID+7:18102:x1862456998021312:12345-0@lo:1000:osp_up0-1.0 Request processed in 2982us (5039us total) trans 12884901924 rc 0/0 00000100:00100000:2.0:1776178098.552375:0:6260:0:(nrs_fifo.c:241:nrs_fifo_req_stop()) NRS stop fifo request from 12345-0@lo, seq: 613 00010000:00010000:0.0:1776178098.592462:0:6245:0:(ldlm_lockd.c:2792:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:1.0:1776178098.986835:0:20137:0:(service.c:2152:ptlrpc_server_handle_req_in()) got req x1862456979081536 00000100:00100000:1.0:1776178098.986939:0:20137:0:(nrs_fifo.c:179:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.204.3@tcp, seq: 114 00000100:00100000:1.0:1776178098.986948:0:20137:0:(service.c:2324:ptlrpc_server_handle_request()) Handling RPC req@00000000a6b47274 pname:cluuid+ref:pid:xid:nid:opc:job ll_mgs_0000:5b5c66b1-1a2f-4651-8548-0c75b9a38a94+9:2270:x1862456979081536:12345-192.168.204.3@tcp:400:kworker/u8:2.0 00000100:00100000:0.0:1776178098.987399:0:8250:0:(service.c:2152:ptlrpc_server_handle_req_in()) got req x1862456979081664 00000100:00100000:0.0:1776178098.987438:0:8250:0:(service.c:2152:ptlrpc_server_handle_req_in()) got req x1862456979081600 00000100:00100000:0.0:1776178098.987457:0:8250:0:(nrs_fifo.c:179:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.204.3@tcp, seq: 745 00000100:00100000:0.0:1776178098.987461:0:8250:0:(service.c:2324:ptlrpc_server_handle_request()) Handling RPC req@00000000f4a76978 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_003:b50963bd-8eba-4432-9a54-8888d00e3fa1+6:2270:x1862456979081664:12345-192.168.204.3@tcp:400:kworker/u8:2.0 00000100:00100000:1.0:1776178098.988089:0:20137:0:(service.c:2372:ptlrpc_server_handle_request()) Handled RPC req@00000000a6b47274 pname:cluuid+ref:pid:xid:nid:opc:job ll_mgs_0000:5b5c66b1-1a2f-4651-8548-0c75b9a38a94+9:2270:x1862456979081536:12345-192.168.204.3@tcp:400:kworker/u8:2.0 Request processed in 1139us (1793us total) trans 0 rc 0/0 00000100:00100000:1.0:1776178098.988099:0:20137:0:(nrs_fifo.c:241:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.204.3@tcp, seq: 114 00000100:00100000:1.0:1776178098.988150:0:6256:0:(nrs_fifo.c:179:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.204.3@tcp, seq: 746 00000100:00100000:1.0:1776178098.988155:0:6256:0:(service.c:2324:ptlrpc_server_handle_request()) Handling RPC req@000000007fbc2285 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_001:b50963bd-8eba-4432-9a54-8888d00e3fa1+62:2270:x1862456979081600:12345-192.168.204.3@tcp:400:kworker/u8:2.0 00000100:00100000:1.0:1776178098.988192:0:6256:0:(service.c:2372:ptlrpc_server_handle_request()) Handled RPC req@000000007fbc2285 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_001:b50963bd-8eba-4432-9a54-8888d00e3fa1+62:2270:x1862456979081600:12345-192.168.204.3@tcp:400:kworker/u8:2.0 Request processed in 36us (1447us total) trans 0 rc 0/0 00000100:00100000:1.0:1776178098.988198:0:6256:0:(nrs_fifo.c:241:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.204.3@tcp, seq: 746 00000100:00100000:0.0:1776178098.992023:0:8250:0:(service.c:2372:ptlrpc_server_handle_request()) Handled RPC req@00000000f4a76978 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_003:b50963bd-8eba-4432-9a54-8888d00e3fa1+6:2270:x1862456979081664:12345-192.168.204.3@tcp:400:kworker/u8:2.0 Request processed in 4544us (5471us total) trans 0 rc 0/0 00000100:00100000:0.0:1776178098.992048:0:8250:0:(nrs_fifo.c:241:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.204.3@tcp, seq: 745 00000100:00100000:0.0:1776178099.553588:0:8232:0:(service.c:2152:ptlrpc_server_handle_req_in()) got req x1862456998021376 00000100:00100000:0.0:1776178099.553643:0:8232:0:(nrs_fifo.c:179:nrs_fifo_req_get()) NRS start fifo request from 12345-0@lo, seq: 397 00000100:00100000:0.0:1776178099.553650:0:8232:0:(service.c:2324:ptlrpc_server_handle_request()) Handling RPC req@0000000050ea472d pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_create00:lustre-MDT0001-mdtlov_UUID+5:3354:x1862456998021376:12345-0@lo:13:osp-pre-1-1.0 00002000:00000024:0.0:1776178099.553689:0:8232:0:(ofd_obd.c:740:ofd_statfs()) blocks cached 229376 granted 491746368 pending 0 free 3923689472 avail 3692167168 00000100:00100000:0.0:1776178099.553763:0:8232:0:(service.c:2372:ptlrpc_server_handle_request()) Handled RPC req@0000000050ea472d pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_create00:lustre-MDT0001-mdtlov_UUID+5:3354:x1862456998021376:12345-0@lo:13:osp-pre-1-1.0 Request processed in 112us (399us total) trans 0 rc 0/0 00000100:00100000:0.0:1776178099.553771:0:8232:0:(nrs_fifo.c:241:nrs_fifo_req_stop()) NRS stop fifo request from 12345-0@lo, seq: 397 00000040:00100000:0.0:1776178099.554613:0:18106:0:(llog.c:235:llog_cancel_arr_rec()) Canceling 39 records, first 1001 in log [0x1:0x12:0x0] 00000004:00080000:0.0:1776178099.556662:0:18106:0:(osp_sync.c:1124:osp_sync_process_committed()) lustre-OST0001-osc-MDT0001: 0 changes, 0 in progress, 0 in flight, 39 done 00000100:00100000:2.0:1776178100.065239:0:8231:0:(service.c:2152:ptlrpc_server_handle_req_in()) got req x1862456998021440 00000100:00100000:2.0:1776178100.065298:0:8231:0:(nrs_fifo.c:179:nrs_fifo_req_get()) NRS start fifo request from 12345-0@lo, seq: 398 00000100:00100000:2.0:1776178100.065306:0:8231:0:(service.c:2324:ptlrpc_server_handle_request()) Handling RPC req@00000000ddc1dc02 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_create00:lustre-MDT0001-mdtlov_UUID+5:3354:x1862456998021440:12345-0@lo:13:osp-pre-0-1.0 00002000:00000024:2.0:1776178100.065349:0:8231:0:(ofd_obd.c:740:ofd_statfs()) blocks cached 315392 granted 484883776 pending 0 free 3923689472 avail 3692167168 00000100:00100000:2.0:1776178100.065393:0:8231:0:(service.c:2372:ptlrpc_server_handle_request()) Handled RPC req@00000000ddc1dc02 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_create00:lustre-MDT0001-mdtlov_UUID+5:3354:x1862456998021440:12345-0@lo:13:osp-pre-0-1.0 Request processed in 87us (298us total) trans 0 rc 0/0 00000100:00100000:2.0:1776178100.065401:0:8231:0:(nrs_fifo.c:241:nrs_fifo_req_stop()) NRS stop fifo request from 12345-0@lo, seq: 398 00000004:00080000:3.0:1776178100.065499:0:3354:0:(osp_sync.c:508:osp_sync_request_commit_cb()) commit req 000000002b0faed1, transno 4294969301 00000004:00080000:3.0:1776178100.065503:0:3354:0:(osp_sync.c:508:osp_sync_request_commit_cb()) commit req 00000000c63a98c1, transno 4294969302 00000004:00080000:3.0:1776178100.065506:0:3354:0:(osp_sync.c:508:osp_sync_request_commit_cb()) commit req 0000000008c9fb19, transno 4294969303 00000004:00080000:3.0:1776178100.065508:0:3354:0:(osp_sync.c:508:osp_sync_request_commit_cb()) commit req 0000000053e142bc, transno 4294969304 00000004:00080000:3.0:1776178100.065512:0:3354:0:(osp_sync.c:508:osp_sync_request_commit_cb()) commit req 000000009c55a259, transno 4294969305 00000004:00080000:3.0:1776178100.065514:0:3354:0:(osp_sync.c:508:osp_sync_request_commit_cb()) commit req 0000000095214b8e, transno 4294969306 00000004:00080000:3.0:1776178100.065516:0:3354:0:(osp_sync.c:508:osp_sync_request_commit_cb()) commit req 00000000f0ef812c, transno 4294969307 00000004:00080000:3.0:1776178100.065518:0:3354:0:(osp_sync.c:508:osp_sync_request_commit_cb()) commit req 000000004e8fbd9b, transno 4294969308 00000004:00080000:3.0:1776178100.065521:0:3354:0:(osp_sync.c:508:osp_sync_request_commit_cb()) commit req 00000000e837a7cd, transno 4294969309 00000004:00080000:3.0:1776178100.065524:0:3354:0:(osp_sync.c:508:osp_sync_request_commit_cb()) commit req 000000001554c6b4, transno 4294969310 00000004:00080000:3.0:1776178100.065526:0:3354:0:(osp_sync.c:508:osp_sync_request_commit_cb()) commit req 0000000007572592, transno 4294969311 00000004:00080000:3.0:1776178100.065530:0:3354:0:(osp_sync.c:508:osp_sync_request_commit_cb()) commit req 00000000d668bb9c, transno 4294969312 00000004:00080000:3.0:1776178100.065532:0:3354:0:(osp_sync.c:508:osp_sync_request_commit_cb()) commit req 0000000008b3a3ad, transno 4294969313 00000004:00080000:3.0:1776178100.065535:0:3354:0:(osp_sync.c:508:osp_sync_request_commit_cb()) commit req 000000008c593d89, transno 4294969314 00000004:00080000:3.0:1776178100.065537:0:3354:0:(osp_sync.c:508:osp_sync_request_commit_cb()) commit req 00000000e0d60cbc, transno 4294969315 00000004:00080000:3.0:1776178100.065539:0:3354:0:(osp_sync.c:508:osp_sync_request_commit_cb()) commit req 0000000092028508, transno 4294969316 00000004:00080000:3.0:1776178100.065541:0:3354:0:(osp_sync.c:508:osp_sync_request_commit_cb()) commit req 00000000b7aa289e, transno 4294969317 00000004:00080000:3.0:1776178100.065545:0:3354:0:(osp_sync.c:508:osp_sync_request_commit_cb()) commit req 000000005967fd64, transno 4294969318 00000004:00080000:3.0:1776178100.065547:0:3354:0:(osp_sync.c:508:osp_sync_request_commit_cb()) commit req 00000000cc80b69e, transno 4294969319 00000004:00080000:3.0:1776178100.065549:0:3354:0:(osp_sync.c:508:osp_sync_request_commit_cb()) commit req 000000006466ee18, transno 4294969320 00000004:00080000:3.0:1776178100.065552:0:3354:0:(osp_sync.c:508:osp_sync_request_commit_cb()) commit req 00000000cb1b73cb, transno 4294969321 00000004:00080000:3.0:1776178100.065554:0:3354:0:(osp_sync.c:508:osp_sync_request_commit_cb()) commit req 00000000a62daf9a, transno 4294969322 00000004:00080000:3.0:1776178100.065557:0:3354:0:(osp_sync.c:508:osp_sync_request_commit_cb()) commit req 00000000d2194868, transno 4294969323 00000004:00080000:3.0:1776178100.065559:0:3354:0:(osp_sync.c:508:osp_sync_request_commit_cb()) commit req 00000000fed3e037, transno 4294969324 00000004:00080000:3.0:1776178100.065561:0:3354:0:(osp_sync.c:508:osp_sync_request_commit_cb()) commit req 000000002a321d27, transno 4294969325 00000004:00080000:3.0:1776178100.065564:0:3354:0:(osp_sync.c:508:osp_sync_request_commit_cb()) commit req 000000006a209b80, transno 4294969326 00000004:00080000:3.0:1776178100.065567:0:3354:0:(osp_sync.c:508:osp_sync_request_commit_cb()) commit req 00000000478fd8d5, transno 4294969327 00000004:00080000:3.0:1776178100.065569:0:3354:0:(osp_sync.c:508:osp_sync_request_commit_cb()) commit req 000000009016e75e, transno 4294969328 00000004:00080000:3.0:1776178100.065572:0:3354:0:(osp_sync.c:508:osp_sync_request_commit_cb()) commit req 00000000f0f1167e, transno 4294969329 00000004:00080000:3.0:1776178100.065575:0:3354:0:(osp_sync.c:508:osp_sync_request_commit_cb()) commit req 0000000078c1d864, transno 4294969330 00000004:00080000:3.0:1776178100.065577:0:3354:0:(osp_sync.c:508:osp_sync_request_commit_cb()) commit req 00000000a2dca46e, transno 4294969331 00000004:00080000:3.0:1776178100.065582:0:3354:0:(osp_sync.c:508:osp_sync_request_commit_cb()) commit req 00000000ba5faff8, transno 4294969332 00000004:00080000:3.0:1776178100.065585:0:3354:0:(osp_sync.c:508:osp_sync_request_commit_cb()) commit req 000000007d60c0e9, transno 4294969333 00000004:00080000:3.0:1776178100.065587:0:3354:0:(osp_sync.c:508:osp_sync_request_commit_cb()) commit req 000000004cbcc47c, transno 4294969334 00000004:00080000:3.0:1776178100.065590:0:3354:0:(osp_sync.c:508:osp_sync_request_commit_cb()) commit req 00000000bc95ec51, transno 4294969335 00000004:00080000:3.0:1776178100.065593:0:3354:0:(osp_sync.c:508:osp_sync_request_commit_cb()) commit req 00000000e1965319, transno 4294969336 00000004:00080000:3.0:1776178100.065595:0:3354:0:(osp_sync.c:508:osp_sync_request_commit_cb()) commit req 000000009c58e3fe, transno 4294969337 00000004:00080000:3.0:1776178100.065597:0:3354:0:(osp_sync.c:508:osp_sync_request_commit_cb()) commit req 0000000084754621, transno 4294969338 00000004:00080000:3.0:1776178100.065600:0:3354:0:(osp_sync.c:508:osp_sync_request_commit_cb()) commit req 00000000e8f6b843, transno 4294969339 00000004:00080000:3.0:1776178100.065603:0:3354:0:(osp_sync.c:508:osp_sync_request_commit_cb()) commit req 000000009cb972eb, transno 4294969340 00000100:00100000:3.0:1776178100.065618:0:3354:0:(client.c:2215:ptlrpc_check_set()) Completed RPC req@00000000efb13deb pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_01:lustre-MDT0001-mdtlov_UUID:3354:1862456998021440:0@lo:13:osp-pre-0-1.0 00000040:00100000:2.0:1776178100.067991:0:18104:0:(llog.c:235:llog_cancel_arr_rec()) Canceling 46 records, first 1005 in log [0x1:0x10:0x0] 00000004:00080000:2.0:1776178100.068154:0:18104:0:(osp_sync.c:1124:osp_sync_process_committed()) lustre-OST0000-osc-MDT0001: 0 changes, 0 in progress, 0 in flight, 46 done 00010000:00010000:2.0:1776178100.643975:0:9224:0:(ldlm_lockd.c:2792:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000001:00080000:0.0:1776178101.621221:0:20134:0:(tgt_lastrcvd.c:913:tgt_cb_last_committed()) lustre-MDT0000: transno 12884901924 is committed 00000001:00080000:0.0:1776178101.621271:0:20134:0:(tgt_lastrcvd.c:913:tgt_cb_last_committed()) lustre-MDT0000: transno 12884901923 is committed 00010000:00010000:1.0:1776178101.621418:0:3339:0:(ldlm_lock.c:820:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(COS) ns: mdt-lustre-MDT0000_UUID lock: 00000000540a8f8b/0xa40083ab9bf52e8e lrc: 3/0,1 mode: COS/COS res: [0x200000007:0x1:0x0].0xc2d93176 bits 0x2/0x0 rrc: 2 type: IBT gid 0 flags: 0x50210000000000 nid: local remote: 0x0 expref: -99 pid: 6256 timeout: 0 lvb_type: 0 00010000:00010000:1.0:1776178101.621438:0:3339:0:(ldlm_lock.c:879:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: mdt-lustre-MDT0000_UUID lock: 00000000540a8f8b/0xa40083ab9bf52e8e lrc: 2/0,0 mode: COS/COS res: [0x200000007:0x1:0x0].0xc2d93176 bits 0x2/0x0 rrc: 2 type: IBT gid 0 flags: 0x50210400000000 nid: local remote: 0x0 expref: -99 pid: 6256 timeout: 0 lvb_type: 0 00010000:00010000:1.0:1776178101.621447:0:3339:0:(ldlm_lockd.c:1879:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: mdt-lustre-MDT0000_UUID lock: 00000000540a8f8b/0xa40083ab9bf52e8e lrc: 3/0,0 mode: COS/COS res: [0x200000007:0x1:0x0].0xc2d93176 bits 0x2/0x0 rrc: 2 type: IBT gid 0 flags: 0x40210400000000 nid: local remote: 0x0 expref: -99 pid: 6256 timeout: 0 lvb_type: 0 00010000:00010000:1.0:1776178101.621454:0:3339:0:(ldlm_lockd.c:1893:ldlm_handle_bl_callback()) Lock 00000000540a8f8b already unused, calling callback (0000000079e78348) 00010000:00010000:1.0:1776178101.621458:0:3339:0:(ldlm_request.c:364:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: mdt-lustre-MDT0000_UUID lock: 00000000540a8f8b/0xa40083ab9bf52e8e lrc: 3/0,0 mode: COS/COS res: [0x200000007:0x1:0x0].0xc2d93176 bits 0x2/0x0 rrc: 2 type: IBT gid 0 flags: 0x40210400000000 nid: local remote: 0x0 expref: -99 pid: 6256 timeout: 0 lvb_type: 0 00010000:00010000:1.0:1776178101.621466:0:3339:0:(ldlm_request.c:1257:ldlm_cli_cancel_local()) ### server-side local cancel ns: mdt-lustre-MDT0000_UUID lock: 00000000540a8f8b/0xa40083ab9bf52e8e lrc: 4/0,0 mode: COS/COS res: [0x200000007:0x1:0x0].0xc2d93176 bits 0x2/0x0 rrc: 2 type: IBT gid 0 flags: 0x40218400000000 nid: local remote: 0x0 expref: -99 pid: 6256 timeout: 0 lvb_type: 0 00010000:00010000:1.0:1776178101.621486:0:3339:0:(ldlm_inodebits.c:94:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x200000007:0x1:0x0].0xc2d93176 (00000000e8f73a70) 00010000:00010000:1.0:1776178101.621489:0:3339:0:(ldlm_inodebits.c:96:ldlm_reprocess_inodebits_queue()) Hint 2 00010000:00010000:1.0:1776178101.621493:0:3339:0:(ldlm_lockd.c:1903:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: mdt-lustre-MDT0000_UUID lock: 00000000540a8f8b/0xa40083ab9bf52e8e lrc: 2/0,0 mode: --/COS res: [0x200000007:0x1:0x0].0xc2d93176 bits 0x2/0x0 rrc: 2 type: IBT gid 0 flags: 0x44a19400000000 nid: local remote: 0x0 expref: -99 pid: 6256 timeout: 0 lvb_type: 0 00010000:00010000:1.0:1776178101.621499:0:3339:0:(ldlm_lock.c:216:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: mdt-lustre-MDT0000_UUID lock: 00000000540a8f8b/0xa40083ab9bf52e8e lrc: 0/0,0 mode: --/COS res: [0x200000007:0x1:0x0].0xc2d93176 bits 0x2/0x0 rrc: 2 type: IBT gid 0 flags: 0x44a19400000000 nid: local remote: 0x0 expref: -99 pid: 6256 timeout: 0 lvb_type: 0 00010000:00010000:1.0:1776178101.621521:0:3339:0:(ldlm_lock.c:820:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(COS) ns: mdt-lustre-MDT0000_UUID lock: 0000000096c1e297/0xa40083ab9bf52e95 lrc: 3/0,1 mode: COS/COS res: [0x2000013a1:0x1:0x0].0x0 bits 0x2/0x0 rrc: 3 type: IBT gid 0 flags: 0x50210000000000 nid: local remote: 0x0 expref: -99 pid: 6256 timeout: 0 lvb_type: 0 00010000:00010000:1.0:1776178101.621532:0:3339:0:(ldlm_lock.c:879:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: mdt-lustre-MDT0000_UUID lock: 0000000096c1e297/0xa40083ab9bf52e95 lrc: 2/0,0 mode: COS/COS res: [0x2000013a1:0x1:0x0].0x0 bits 0x2/0x0 rrc: 3 type: IBT gid 0 flags: 0x50210400000000 nid: local remote: 0x0 expref: -99 pid: 6256 timeout: 0 lvb_type: 0 00010000:00010000:1.0:1776178101.621538:0:3339:0:(ldlm_lockd.c:1879:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: mdt-lustre-MDT0000_UUID lock: 0000000096c1e297/0xa40083ab9bf52e95 lrc: 3/0,0 mode: COS/COS res: [0x2000013a1:0x1:0x0].0x0 bits 0x2/0x0 rrc: 3 type: IBT gid 0 flags: 0x40210400000000 nid: local remote: 0x0 expref: -99 pid: 6256 timeout: 0 lvb_type: 0 00010000:00010000:1.0:1776178101.621543:0:3339:0:(ldlm_lockd.c:1893:ldlm_handle_bl_callback()) Lock 0000000096c1e297 already unused, calling callback (0000000079e78348) 00010000:00010000:1.0:1776178101.621544:0:3339:0:(ldlm_request.c:364:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: mdt-lustre-MDT0000_UUID lock: 0000000096c1e297/0xa40083ab9bf52e95 lrc: 3/0,0 mode: COS/COS res: [0x2000013a1:0x1:0x0].0x0 bits 0x2/0x0 rrc: 3 type: IBT gid 0 flags: 0x40210400000000 nid: local remote: 0x0 expref: -99 pid: 6256 timeout: 0 lvb_type: 0 00010000:00010000:1.0:1776178101.621549:0:3339:0:(ldlm_request.c:1257:ldlm_cli_cancel_local()) ### server-side local cancel ns: mdt-lustre-MDT0000_UUID lock: 0000000096c1e297/0xa40083ab9bf52e95 lrc: 4/0,0 mode: COS/COS res: [0x2000013a1:0x1:0x0].0x0 bits 0x2/0x0 rrc: 3 type: IBT gid 0 flags: 0x40218400000000 nid: local remote: 0x0 expref: -99 pid: 6256 timeout: 0 lvb_type: 0 00010000:00010000:1.0:1776178101.621556:0:3339:0:(ldlm_inodebits.c:94:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x2000013a1:0x1:0x0].0x0 (00000000315facbe) 00010000:00010000:1.0:1776178101.621557:0:3339:0:(ldlm_inodebits.c:96:ldlm_reprocess_inodebits_queue()) Hint 2 00010000:00010000:1.0:1776178101.621559:0:3339:0:(ldlm_lockd.c:1903:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: mdt-lustre-MDT0000_UUID lock: 0000000096c1e297/0xa40083ab9bf52e95 lrc: 2/0,0 mode: --/COS res: [0x2000013a1:0x1:0x0].0x0 bits 0x2/0x0 rrc: 3 type: IBT gid 0 flags: 0x44a19400000000 nid: local remote: 0x0 expref: -99 pid: 6256 timeout: 0 lvb_type: 0 00010000:00010000:1.0:1776178101.621564:0:3339:0:(ldlm_lock.c:216:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: mdt-lustre-MDT0000_UUID lock: 0000000096c1e297/0xa40083ab9bf52e95 lrc: 0/0,0 mode: --/COS res: [0x2000013a1:0x1:0x0].0x0 bits 0x2/0x0 rrc: 3 type: IBT gid 0 flags: 0x44a19400000000 nid: local remote: 0x0 expref: -99 pid: 6256 timeout: 0 lvb_type: 0 00010000:00010000:1.0:1776178101.621579:0:3339:0:(ldlm_lock.c:820:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(COS) ns: mdt-lustre-MDT0000_UUID lock: 00000000e1acd7eb/0xa40083ab9bf52efe lrc: 3/0,1 mode: COS/COS res: [0x2000013a1:0x1:0x0].0xcda302ea bits 0x2/0x0 rrc: 2 type: IBT gid 0 flags: 0x50210000000000 nid: local remote: 0x0 expref: -99 pid: 6256 timeout: 0 lvb_type: 0 00010000:00010000:1.0:1776178101.621585:0:3339:0:(ldlm_lock.c:879:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: mdt-lustre-MDT0000_UUID lock: 00000000e1acd7eb/0xa40083ab9bf52efe lrc: 2/0,0 mode: COS/COS res: [0x2000013a1:0x1:0x0].0xcda302ea bits 0x2/0x0 rrc: 2 type: IBT gid 0 flags: 0x50210400000000 nid: local remote: 0x0 expref: -99 pid: 6256 timeout: 0 lvb_type: 0 00010000:00010000:1.0:1776178101.621590:0:3339:0:(ldlm_lockd.c:1879:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: mdt-lustre-MDT0000_UUID lock: 00000000e1acd7eb/0xa40083ab9bf52efe lrc: 3/0,0 mode: COS/COS res: [0x2000013a1:0x1:0x0].0xcda302ea bits 0x2/0x0 rrc: 2 type: IBT gid 0 flags: 0x40210400000000 nid: local remote: 0x0 expref: -99 pid: 6256 timeout: 0 lvb_type: 0 00010000:00010000:1.0:1776178101.621600:0:3339:0:(ldlm_lockd.c:1893:ldlm_handle_bl_callback()) Lock 00000000e1acd7eb already unused, calling callback (0000000079e78348) 00010000:00010000:1.0:1776178101.621602:0:3339:0:(ldlm_request.c:364:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: mdt-lustre-MDT0000_UUID lock: 00000000e1acd7eb/0xa40083ab9bf52efe lrc: 3/0,0 mode: COS/COS res: [0x2000013a1:0x1:0x0].0xcda302ea bits 0x2/0x0 rrc: 2 type: IBT gid 0 flags: 0x40210400000000 nid: local remote: 0x0 expref: -99 pid: 6256 timeout: 0 lvb_type: 0 00010000:00010000:1.0:1776178101.621607:0:3339:0:(ldlm_request.c:1257:ldlm_cli_cancel_local()) ### server-side local cancel ns: mdt-lustre-MDT0000_UUID lock: 00000000e1acd7eb/0xa40083ab9bf52efe lrc: 4/0,0 mode: COS/COS res: [0x2000013a1:0x1:0x0].0xcda302ea bits 0x2/0x0 rrc: 2 type: IBT gid 0 flags: 0x40218400000000 nid: local remote: 0x0 expref: -99 pid: 6256 timeout: 0 lvb_type: 0 00010000:00010000:1.0:1776178101.621613:0:3339:0:(ldlm_inodebits.c:94:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x2000013a1:0x1:0x0].0xcda302ea (0000000008b13340) 00010000:00010000:1.0:1776178101.621615:0:3339:0:(ldlm_inodebits.c:96:ldlm_reprocess_inodebits_queue()) Hint 2 00010000:00010000:1.0:1776178101.621617:0:3339:0:(ldlm_lockd.c:1903:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: mdt-lustre-MDT0000_UUID lock: 00000000e1acd7eb/0xa40083ab9bf52efe lrc: 2/0,0 mode: --/COS res: [0x2000013a1:0x1:0x0].0xcda302ea bits 0x2/0x0 rrc: 2 type: IBT gid 0 flags: 0x44a19400000000 nid: local remote: 0x0 expref: -99 pid: 6256 timeout: 0 lvb_type: 0 00010000:00010000:1.0:1776178101.621622:0:3339:0:(ldlm_lock.c:216:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: mdt-lustre-MDT0000_UUID lock: 00000000e1acd7eb/0xa40083ab9bf52efe lrc: 0/0,0 mode: --/COS res: [0x2000013a1:0x1:0x0].0xcda302ea bits 0x2/0x0 rrc: 2 type: IBT gid 0 flags: 0x44a19400000000 nid: local remote: 0x0 expref: -99 pid: 6256 timeout: 0 lvb_type: 0 00010000:00010000:1.0:1776178101.621634:0:3339:0:(ldlm_lock.c:820:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(COS) ns: mdt-lustre-MDT0000_UUID lock: 00000000591efbb9/0xa40083ab9bf52f13 lrc: 3/0,1 mode: COS/COS res: [0x2000013a1:0x2:0x0].0x0 bits 0x2/0x0 rrc: 4 type: IBT gid 0 flags: 0x50210000000000 nid: local remote: 0x0 expref: -99 pid: 6255 timeout: 0 lvb_type: 0 00010000:00010000:1.0:1776178101.621639:0:3339:0:(ldlm_lock.c:879:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: mdt-lustre-MDT0000_UUID lock: 00000000591efbb9/0xa40083ab9bf52f13 lrc: 2/0,0 mode: COS/COS res: [0x2000013a1:0x2:0x0].0x0 bits 0x2/0x0 rrc: 4 type: IBT gid 0 flags: 0x50210400000000 nid: local remote: 0x0 expref: -99 pid: 6255 timeout: 0 lvb_type: 0 00010000:00010000:1.0:1776178101.621644:0:3339:0:(ldlm_lockd.c:1879:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: mdt-lustre-MDT0000_UUID lock: 00000000591efbb9/0xa40083ab9bf52f13 lrc: 3/0,0 mode: COS/COS res: [0x2000013a1:0x2:0x0].0x0 bits 0x2/0x0 rrc: 4 type: IBT gid 0 flags: 0x40210400000000 nid: local remote: 0x0 expref: -99 pid: 6255 timeout: 0 lvb_type: 0 00010000:00010000:1.0:1776178101.621648:0:3339:0:(ldlm_lockd.c:1893:ldlm_handle_bl_callback()) Lock 00000000591efbb9 already unused, calling callback (0000000079e78348) 00010000:00010000:1.0:1776178101.621650:0:3339:0:(ldlm_request.c:364:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: mdt-lustre-MDT0000_UUID lock: 00000000591efbb9/0xa40083ab9bf52f13 lrc: 3/0,0 mode: COS/COS res: [0x2000013a1:0x2:0x0].0x0 bits 0x2/0x0 rrc: 4 type: IBT gid 0 flags: 0x40210400000000 nid: local remote: 0x0 expref: -99 pid: 6255 timeout: 0 lvb_type: 0 00010000:00010000:1.0:1776178101.621665:0:3339:0:(ldlm_request.c:1257:ldlm_cli_cancel_local()) ### server-side local cancel ns: mdt-lustre-MDT0000_UUID lock: 00000000591efbb9/0xa40083ab9bf52f13 lrc: 4/0,0 mode: COS/COS res: [0x2000013a1:0x2:0x0].0x0 bits 0x2/0x0 rrc: 4 type: IBT gid 0 flags: 0x40218400000000 nid: local remote: 0x0 expref: -99 pid: 6255 timeout: 0 lvb_type: 0 00010000:00010000:1.0:1776178101.621673:0:3339:0:(ldlm_inodebits.c:94:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x2000013a1:0x2:0x0].0x0 (000000005ad330c7) 00010000:00010000:1.0:1776178101.621675:0:3339:0:(ldlm_inodebits.c:96:ldlm_reprocess_inodebits_queue()) Hint 2 00010000:00010000:1.0:1776178101.621678:0:3339:0:(ldlm_lockd.c:1903:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: mdt-lustre-MDT0000_UUID lock: 00000000591efbb9/0xa40083ab9bf52f13 lrc: 2/0,0 mode: --/COS res: [0x2000013a1:0x2:0x0].0x0 bits 0x2/0x0 rrc: 4 type: IBT gid 0 flags: 0x44a19400000000 nid: local remote: 0x0 expref: -99 pid: 6255 timeout: 0 lvb_type: 0 00010000:00010000:1.0:1776178101.621682:0:3339:0:(ldlm_lock.c:216:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: mdt-lustre-MDT0000_UUID lock: 00000000591efbb9/0xa40083ab9bf52f13 lrc: 0/0,0 mode: --/COS res: [0x2000013a1:0x2:0x0].0x0 bits 0x2/0x0 rrc: 4 type: IBT gid 0 flags: 0x44a19400000000 nid: local remote: 0x0 expref: -99 pid: 6255 timeout: 0 lvb_type: 0 00010000:00010000:1.0:1776178101.621692:0:3339:0:(ldlm_lock.c:820:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(COS) ns: mdt-lustre-MDT0000_UUID lock: 00000000b82e73fc/0xa40083ab9bf52f28 lrc: 3/0,1 mode: COS/COS res: [0x2000013a1:0x1:0x0].0x15fcbbea bits 0x2/0x0 rrc: 2 type: IBT gid 0 flags: 0x50210000000000 nid: local remote: 0x0 expref: -99 pid: 6256 timeout: 0 lvb_type: 0 00010000:00010000:1.0:1776178101.621698:0:3339:0:(ldlm_lock.c:879:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: mdt-lustre-MDT0000_UUID lock: 00000000b82e73fc/0xa40083ab9bf52f28 lrc: 2/0,0 mode: COS/COS res: [0x2000013a1:0x1:0x0].0x15fcbbea bits 0x2/0x0 rrc: 2 type: IBT gid 0 flags: 0x50210400000000 nid: local remote: 0x0 expref: -99 pid: 6256 timeout: 0 lvb_type: 0 00010000:00010000:1.0:1776178101.621702:0:3339:0:(ldlm_lockd.c:1879:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: mdt-lustre-MDT0000_UUID lock: 00000000b82e73fc/0xa40083ab9bf52f28 lrc: 3/0,0 mode: COS/COS res: [0x2000013a1:0x1:0x0].0x15fcbbea bits 0x2/0x0 rrc: 2 type: IBT gid 0 flags: 0x40210400000000 nid: local remote: 0x0 expref: -99 pid: 6256 timeout: 0 lvb_type: 0 00010000:00010000:1.0:1776178101.621707:0:3339:0:(ldlm_lockd.c:1893:ldlm_handle_bl_callback()) Lock 00000000b82e73fc already unused, calling callback (0000000079e78348) 00010000:00010000:1.0:1776178101.621709:0:3339:0:(ldlm_request.c:364:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: mdt-lustre-MDT0000_UUID lock: 00000000b82e73fc/0xa40083ab9bf52f28 lrc: 3/0,0 mode: COS/COS res: [0x2000013a1:0x1:0x0].0x15fcbbea bits 0x2/0x0 rrc: 2 type: IBT gid 0 flags: 0x40210400000000 nid: local remote: 0x0 expref: -99 pid: 6256 timeout: 0 lvb_type: 0 00010000:00010000:1.0:1776178101.621724:0:3339:0:(ldlm_request.c:1257:ldlm_cli_cancel_local()) ### server-side local cancel ns: mdt-lustre-MDT0000_UUID lock: 00000000b82e73fc/0xa40083ab9bf52f28 lrc: 4/0,0 mode: COS/COS res: [0x2000013a1:0x1:0x0].0x15fcbbea bits 0x2/0x0 rrc: 2 type: IBT gid 0 flags: 0x40218400000000 nid: local remote: 0x0 expref: -99 pid: 6256 timeout: 0 lvb_type: 0 00010000:00010000:1.0:1776178101.621731:0:3339:0:(ldlm_inodebits.c:94:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x2000013a1:0x1:0x0].0x15fcbbea (00000000a8172224) 00010000:00010000:1.0:1776178101.621733:0:3339:0:(ldlm_inodebits.c:96:ldlm_reprocess_inodebits_queue()) Hint 2 00010000:00010000:1.0:1776178101.621735:0:3339:0:(ldlm_lockd.c:1903:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: mdt-lustre-MDT0000_UUID lock: 00000000b82e73fc/0xa40083ab9bf52f28 lrc: 2/0,0 mode: --/COS res: [0x2000013a1:0x1:0x0].0x15fcbbea bits 0x2/0x0 rrc: 2 type: IBT gid 0 flags: 0x44a19400000000 nid: local remote: 0x0 expref: -99 pid: 6256 timeout: 0 lvb_type: 0 00010000:00010000:1.0:1776178101.621744:0:3339:0:(ldlm_lock.c:216:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: mdt-lustre-MDT0000_UUID lock: 00000000b82e73fc/0xa40083ab9bf52f28 lrc: 0/0,0 mode: --/COS res: [0x2000013a1:0x1:0x0].0x15fcbbea bits 0x2/0x0 rrc: 2 type: IBT gid 0 flags: 0x44a19400000000 nid: local remote: 0x0 expref: -99 pid: 6256 timeout: 0 lvb_type: 0 00010000:00010000:1.0:1776178101.621758:0:3339:0:(ldlm_lock.c:820:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(COS) ns: mdt-lustre-MDT0000_UUID lock: 000000000584b645/0xa40083ab9bf52f3d lrc: 3/0,1 mode: COS/COS res: [0x2000013a1:0x3:0x0].0x0 bits 0x2/0x0 rrc: 4 type: IBT gid 0 flags: 0x50210000000000 nid: local remote: 0x0 expref: -99 pid: 6255 timeout: 0 lvb_type: 0 00010000:00010000:1.0:1776178101.621762:0:3339:0:(ldlm_lock.c:879:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: mdt-lustre-MDT0000_UUID lock: 000000000584b645/0xa40083ab9bf52f3d lrc: 2/0,0 mode: COS/COS res: [0x2000013a1:0x3:0x0].0x0 bits 0x2/0x0 rrc: 4 type: IBT gid 0 flags: 0x50210400000000 nid: local remote: 0x0 expref: -99 pid: 6255 timeout: 0 lvb_type: 0 00010000:00010000:1.0:1776178101.621767:0:3339:0:(ldlm_lockd.c:1879:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: mdt-lustre-MDT0000_UUID lock: 000000000584b645/0xa40083ab9bf52f3d lrc: 3/0,0 mode: COS/COS res: [0x2000013a1:0x3:0x0].0x0 bits 0x2/0x0 rrc: 4 type: IBT gid 0 flags: 0x40210400000000 nid: local remote: 0x0 expref: -99 pid: 6255 timeout: 0 lvb_type: 0 00010000:00010000:1.0:1776178101.621771:0:3339:0:(ldlm_lockd.c:1893:ldlm_handle_bl_callback()) Lock 000000000584b645 already unused, calling callback (0000000079e78348) 00010000:00010000:1.0:1776178101.621773:0:3339:0:(ldlm_request.c:364:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: mdt-lustre-MDT0000_UUID lock: 000000000584b645/0xa40083ab9bf52f3d lrc: 3/0,0 mode: COS/COS res: [0x2000013a1:0x3:0x0].0x0 bits 0x2/0x0 rrc: 4 type: IBT gid 0 flags: 0x40210400000000 nid: local remote: 0x0 expref: -99 pid: 6255 timeout: 0 lvb_type: 0 00010000:00010000:1.0:1776178101.621787:0:3339:0:(ldlm_request.c:1257:ldlm_cli_cancel_local()) ### server-side local cancel ns: mdt-lustre-MDT0000_UUID lock: 000000000584b645/0xa40083ab9bf52f3d lrc: 4/0,0 mode: COS/COS res: [0x2000013a1:0x3:0x0].0x0 bits 0x2/0x0 rrc: 4 type: IBT gid 0 flags: 0x40218400000000 nid: local remote: 0x0 expref: -99 pid: 6255 timeout: 0 lvb_type: 0 00010000:00010000:1.0:1776178101.621794:0:3339:0:(ldlm_inodebits.c:94:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x2000013a1:0x3:0x0].0x0 (00000000d0a7c19f) 00010000:00010000:1.0:1776178101.621796:0:3339:0:(ldlm_inodebits.c:96:ldlm_reprocess_inodebits_queue()) Hint 2 00010000:00010000:1.0:1776178101.621797:0:3339:0:(ldlm_lockd.c:1903:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: mdt-lustre-MDT0000_UUID lock: 000000000584b645/0xa40083ab9bf52f3d lrc: 2/0,0 mode: --/COS res: [0x2000013a1:0x3:0x0].0x0 bits 0x2/0x0 rrc: 4 type: IBT gid 0 flags: 0x44a19400000000 nid: local remote: 0x0 expref: -99 pid: 6255 timeout: 0 lvb_type: 0 00010000:00010000:1.0:1776178101.621802:0:3339:0:(ldlm_lock.c:216:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: mdt-lustre-MDT0000_UUID lock: 000000000584b645/0xa40083ab9bf52f3d lrc: 0/0,0 mode: --/COS res: [0x2000013a1:0x3:0x0].0x0 bits 0x2/0x0 rrc: 4 type: IBT gid 0 flags: 0x44a19400000000 nid: local remote: 0x0 expref: -99 pid: 6255 timeout: 0 lvb_type: 0 00010000:00010000:1.0:1776178101.621810:0:3339:0:(ldlm_lock.c:820:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(COS) ns: mdt-lustre-MDT0000_UUID lock: 000000008cce7e8f/0xa40083ab9bf52f52 lrc: 3/0,1 mode: COS/COS res: [0x2000013a1:0x1:0x0].0x5e5674ea bits 0x2/0x0 rrc: 2 type: IBT gid 0 flags: 0x50210000000000 nid: local remote: 0x0 expref: -99 pid: 6255 timeout: 0 lvb_type: 0 00010000:00010000:1.0:1776178101.621819:0:3339:0:(ldlm_lock.c:879:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: mdt-lustre-MDT0000_UUID lock: 000000008cce7e8f/0xa40083ab9bf52f52 lrc: 2/0,0 mode: COS/COS res: [0x2000013a1:0x1:0x0].0x5e5674ea bits 0x2/0x0 rrc: 2 type: IBT gid 0 flags: 0x50210400000000 nid: local remote: 0x0 expref: -99 pid: 6255 timeout: 0 lvb_type: 0 00010000:00010000:1.0:1776178101.621824:0:3339:0:(ldlm_lockd.c:1879:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: mdt-lustre-MDT0000_UUID lock: 000000008cce7e8f/0xa40083ab9bf52f52 lrc: 3/0,0 mode: COS/COS res: [0x2000013a1:0x1:0x0].0x5e5674ea bits 0x2/0x0 rrc: 2 type: IBT gid 0 flags: 0x40210400000000 nid: local remote: 0x0 expref: -99 pid: 6255 timeout: 0 lvb_type: 0 00010000:00010000:1.0:1776178101.621828:0:3339:0:(ldlm_lockd.c:1893:ldlm_handle_bl_callback()) Lock 000000008cce7e8f already unused, calling callback (0000000079e78348) 00010000:00010000:1.0:1776178101.621830:0:3339:0:(ldlm_request.c:364:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: mdt-lustre-MDT0000_UUID lock: 000000008cce7e8f/0xa40083ab9bf52f52 lrc: 3/0,0 mode: COS/COS res: [0x2000013a1:0x1:0x0].0x5e5674ea bits 0x2/0x0 rrc: 2 type: IBT gid 0 flags: 0x40210400000000 nid: local remote: 0x0 expref: -99 pid: 6255 timeout: 0 lvb_type: 0 00010000:00010000:1.0:1776178101.621834:0:3339:0:(ldlm_request.c:1257:ldlm_cli_cancel_local()) ### server-side local cancel ns: mdt-lustre-MDT0000_UUID lock: 000000008cce7e8f/0xa40083ab9bf52f52 lrc: 4/0,0 mode: COS/COS res: [0x2000013a1:0x1:0x0].0x5e5674ea bits 0x2/0x0 rrc: 2 type: IBT gid 0 flags: 0x40218400000000 nid: local remote: 0x0 expref: -99 pid: 6255 timeout: 0 lvb_type: 0 00010000:00010000:1.0:1776178101.621839:0:3339:0:(ldlm_inodebits.c:94:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x2000013a1:0x1:0x0].0x5e5674ea (000000008e570e08) 00010000:00010000:1.0:1776178101.621841:0:3339:0:(ldlm_inodebits.c:96:ldlm_reprocess_inodebits_queue()) Hint 2 00010000:00010000:1.0:1776178101.621843:0:3339:0:(ldlm_lockd.c:1903:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: mdt-lustre-MDT0000_UUID lock: 000000008cce7e8f/0xa40083ab9bf52f52 lrc: 2/0,0 mode: --/COS res: [0x2000013a1:0x1:0x0].0x5e5674ea bits 0x2/0x0 rrc: 2 type: IBT gid 0 flags: 0x44a19400000000 nid: local remote: 0x0 expref: -99 pid: 6255 timeout: 0 lvb_type: 0 00010000:00010000:1.0:1776178101.621848:0:3339:0:(ldlm_lock.c:216:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: mdt-lustre-MDT0000_UUID lock: 000000008cce7e8f/0xa40083ab9bf52f52 lrc: 0/0,0 mode: --/COS res: [0x2000013a1:0x1:0x0].0x5e5674ea bits 0x2/0x0 rrc: 2 type: IBT gid 0 flags: 0x44a19400000000 nid: local remote: 0x0 expref: -99 pid: 6255 timeout: 0 lvb_type: 0 00010000:00010000:1.0:1776178101.621858:0:3339:0:(ldlm_lock.c:820:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(COS) ns: mdt-lustre-MDT0000_UUID lock: 00000000dcea299c/0xa40083ab9bf52f67 lrc: 3/0,1 mode: COS/COS res: [0x2000013a1:0x4:0x0].0x0 bits 0x2/0x0 rrc: 4 type: IBT gid 0 flags: 0x50210000000000 nid: local remote: 0x0 expref: -99 pid: 6255 timeout: 0 lvb_type: 0 00010000:00010000:1.0:1776178101.621863:0:3339:0:(ldlm_lock.c:879:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: mdt-lustre-MDT0000_UUID lock: 00000000dcea299c/0xa40083ab9bf52f67 lrc: 2/0,0 mode: COS/COS res: [0x2000013a1:0x4:0x0].0x0 bits 0x2/0x0 rrc: 4 type: IBT gid 0 flags: 0x50210400000000 nid: local remote: 0x0 expref: -99 pid: 6255 timeout: 0 lvb_type: 0 00010000:00010000:1.0:1776178101.621868:0:3339:0:(ldlm_lockd.c:1879:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: mdt-lustre-MDT0000_UUID lock: 00000000dcea299c/0xa40083ab9bf52f67 lrc: 3/0,0 mode: COS/COS res: [0x2000013a1:0x4:0x0].0x0 bits 0x2/0x0 rrc: 4 type: IBT gid 0 flags: 0x40210400000000 nid: local remote: 0x0 expref: -99 pid: 6255 timeout: 0 lvb_type: 0 00010000:00010000:1.0:1776178101.621878:0:3339:0:(ldlm_lockd.c:1893:ldlm_handle_bl_callback()) Lock 00000000dcea299c already unused, calling callback (0000000079e78348) 00010000:00010000:1.0:1776178101.621880:0:3339:0:(ldlm_request.c:364:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: mdt-lustre-MDT0000_UUID lock: 00000000dcea299c/0xa40083ab9bf52f67 lrc: 3/0,0 mode: COS/COS res: [0x2000013a1:0x4:0x0].0x0 bits 0x2/0x0 rrc: 4 type: IBT gid 0 flags: 0x40210400000000 nid: local remote: 0x0 expref: -99 pid: 6255 timeout: 0 lvb_type: 0 00010000:00010000:1.0:1776178101.621885:0:3339:0:(ldlm_request.c:1257:ldlm_cli_cancel_local()) ### server-side local cancel ns: mdt-lustre-MDT0000_UUID lock: 00000000dcea299c/0xa40083ab9bf52f67 lrc: 4/0,0 mode: COS/COS res: [0x2000013a1:0x4:0x0].0x0 bits 0x2/0x0 rrc: 4 type: IBT gid 0 flags: 0x40218400000000 nid: local remote: 0x0 expref: -99 pid: 6255 timeout: 0 lvb_type: 0 00010000:00010000:1.0:1776178101.621891:0:3339:0:(ldlm_inodebits.c:94:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x2000013a1:0x4:0x0].0x0 (00000000ef6cecb2) 00010000:00010000:1.0:1776178101.621892:0:3339:0:(ldlm_inodebits.c:96:ldlm_reprocess_inodebits_queue()) Hint 2 00010000:00010000:1.0:1776178101.621893:0:3339:0:(ldlm_lockd.c:1903:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: mdt-lustre-MDT0000_UUID lock: 00000000dcea299c/0xa40083ab9bf52f67 lrc: 2/0,0 mode: --/COS res: [0x2000013a1:0x4:0x0].0x0 bits 0x2/0x0 rrc: 4 type: IBT gid 0 flags: 0x44a19400000000 nid: local remote: 0x0 expref: -99 pid: 6255 timeout: 0 lvb_type: 0 00010000:00010000:1.0:1776178101.621897:0:3339:0:(ldlm_lock.c:216:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: mdt-lustre-MDT0000_UUID lock: 00000000dcea299c/0xa40083ab9bf52f67 lrc: 0/0,0 mode: --/COS res: [0x2000013a1:0x4:0x0].0x0 bits 0x2/0x0 rrc: 4 type: IBT gid 0 flags: 0x44a19400000000 nid: local remote: 0x0 expref: -99 pid: 6255 timeout: 0 lvb_type: 0 00010000:00010000:1.0:1776178101.621905:0:3339:0:(ldlm_lock.c:820:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(COS) ns: mdt-lustre-MDT0000_UUID lock: 000000006b0afb52/0xa40083ab9bf52f7c lrc: 3/0,1 mode: COS/COS res: [0x2000013a1:0x1:0x0].0xa6b02dea bits 0x2/0x0 rrc: 2 type: IBT gid 0 flags: 0x50210000000000 nid: local remote: 0x0 expref: -99 pid: 6255 timeout: 0 lvb_type: 0 00010000:00010000:1.0:1776178101.621911:0:3339:0:(ldlm_lock.c:879:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: mdt-lustre-MDT0000_UUID lock: 000000006b0afb52/0xa40083ab9bf52f7c lrc: 2/0,0 mode: COS/COS res: [0x2000013a1:0x1:0x0].0xa6b02dea bits 0x2/0x0 rrc: 2 type: IBT gid 0 flags: 0x50210400000000 nid: local remote: 0x0 expref: -99 pid: 6255 timeout: 0 lvb_type: 0 00010000:00010000:1.0:1776178101.621916:0:3339:0:(ldlm_lockd.c:1879:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: mdt-lustre-MDT0000_UUID lock: 000000006b0afb52/0xa40083ab9bf52f7c lrc: 3/0,0 mode: COS/COS res: [0x2000013a1:0x1:0x0].0xa6b02dea bits 0x2/0x0 rrc: 2 type: IBT gid 0 flags: 0x40210400000000 nid: local remote: 0x0 expref: -99 pid: 6255 timeout: 0 lvb_type: 0 00010000:00010000:1.0:1776178101.621921:0:3339:0:(ldlm_lockd.c:1893:ldlm_handle_bl_callback()) Lock 000000006b0afb52 already unused, calling callback (0000000079e78348) 00010000:00010000:1.0:1776178101.621922:0:3339:0:(ldlm_request.c:364:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: mdt-lustre-MDT0000_UUID lock: 000000006b0afb52/0xa40083ab9bf52f7c lrc: 3/0,0 mode: COS/COS res: [0x2000013a1:0x1:0x0].0xa6b02dea bits 0x2/0x0 rrc: 2 type: IBT gid 0 flags: 0x40210400000000 nid: local remote: 0x0 expref: -99 pid: 6255 timeout: 0 lvb_type: 0 00010000:00010000:1.0:1776178101.621927:0:3339:0:(ldlm_request.c:1257:ldlm_cli_cancel_local()) ### server-side local cancel ns: mdt-lustre-MDT0000_UUID lock: 000000006b0afb52/0xa40083ab9bf52f7c lrc: 4/0,0 mode: COS/COS res: [0x2000013a1:0x1:0x0].0xa6b02dea bits 0x2/0x0 rrc: 2 type: IBT gid 0 flags: 0x40218400000000 nid: local remote: 0x0 expref: -99 pid: 6255 timeout: 0 lvb_type: 0 00010000:00010000:1.0:1776178101.621934:0:3339:0:(ldlm_inodebits.c:94:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x2000013a1:0x1:0x0].0xa6b02dea (000000009cfa2d69) 00010000:00010000:1.0:1776178101.621936:0:3339:0:(ldlm_inodebits.c:96:ldlm_reprocess_inodebits_queue()) Hint 2 00010000:00010000:1.0:1776178101.621938:0:3339:0:(ldlm_lockd.c:1903:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: mdt-lustre-MDT0000_UUID lock: 000000006b0afb52/0xa40083ab9bf52f7c lrc: 2/0,0 mode: --/COS res: [0x2000013a1:0x1:0x0].0xa6b02dea bits 0x2/0x0 rrc: 2 type: IBT gid 0 flags: 0x44a19400000000 nid: local remote: 0x0 expref: -99 pid: 6255 timeout: 0 lvb_type: 0 00010000:00010000:1.0:1776178101.621943:0:3339:0:(ldlm_lock.c:216:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: mdt-lustre-MDT0000_UUID lock: 000000006b0afb52/0xa40083ab9bf52f7c lrc: 0/0,0 mode: --/COS res: [0x2000013a1:0x1:0x0].0xa6b02dea bits 0x2/0x0 rrc: 2 type: IBT gid 0 flags: 0x44a19400000000 nid: local remote: 0x0 expref: -99 pid: 6255 timeout: 0 lvb_type: 0 00010000:00010000:1.0:1776178101.621967:0:3339:0:(ldlm_lock.c:820:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(COS) ns: mdt-lustre-MDT0000_UUID lock: 00000000b2e83690/0xa40083ab9bf52f91 lrc: 3/0,1 mode: COS/COS res: [0x2000013a1:0x5:0x0].0x0 bits 0x2/0x0 rrc: 4 type: IBT gid 0 flags: 0x50210000000000 nid: local remote: 0x0 expref: -99 pid: 6255 timeout: 0 lvb_type: 0 00010000:00010000:1.0:1776178101.621974:0:3339:0:(ldlm_lock.c:879:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: mdt-lustre-MDT0000_UUID lock: 00000000b2e83690/0xa40083ab9bf52f91 lrc: 2/0,0 mode: COS/COS res: [0x2000013a1:0x5:0x0].0x0 bits 0x2/0x0 rrc: 4 type: IBT gid 0 flags: 0x50210400000000 nid: local remote: 0x0 expref: -99 pid: 6255 timeout: 0 lvb_type: 0 00010000:00010000:1.0:1776178101.621979:0:3339:0:(ldlm_lockd.c:1879:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: mdt-lustre-MDT0000_UUID lock: 00000000b2e83690/0xa40083ab9bf52f91 lrc: 3/0,0 mode: COS/COS res: [0x2000013a1:0x5:0x0].0x0 bits 0x2/0x0 rrc: 4 type: IBT gid 0 flags: 0x40210400000000 nid: local remote: 0x0 expref: -99 pid: 6255 timeout: 0 lvb_type: 0 00010000:00010000:1.0:1776178101.621984:0:3339:0:(ldlm_lockd.c:1893:ldlm_handle_bl_callback()) Lock 00000000b2e83690 already unused, calling callback (0000000079e78348) 00010000:00010000:1.0:1776178101.621985:0:3339:0:(ldlm_request.c:364:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: mdt-lustre-MDT0000_UUID lock: 00000000b2e83690/0xa40083ab9bf52f91 lrc: 3/0,0 mode: COS/COS res: [0x2000013a1:0x5:0x0].0x0 bits 0x2/0x0 rrc: 4 type: IBT gid 0 flags: 0x40210400000000 nid: local remote: 0x0 expref: -99 pid: 6255 timeout: 0 lvb_type: 0 00010000:00010000:1.0:1776178101.621990:0:3339:0:(ldlm_request.c:1257:ldlm_cli_cancel_local()) ### server-side local cancel ns: mdt-lustre-MDT0000_UUID lock: 00000000b2e83690/0xa40083ab9bf52f91 lrc: 4/0,0 mode: COS/COS res: [0x2000013a1:0x5:0x0].0x0 bits 0x2/0x0 rrc: 4 type: IBT gid 0 flags: 0x40218400000000 nid: local remote: 0x0 expref: -99 pid: 6255 timeout: 0 lvb_type: 0 00010000:00010000:1.0:1776178101.621998:0:3339:0:(ldlm_inodebits.c:94:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x2000013a1:0x5:0x0].0x0 (000000003b465a63) 00010000:00010000:1.0:1776178101.622000:0:3339:0:(ldlm_inodebits.c:96:ldlm_reprocess_inodebits_queue()) Hint 2 00010000:00010000:1.0:1776178101.622001:0:3339:0:(ldlm_lockd.c:1903:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: mdt-lustre-MDT0000_UUID lock: 00000000b2e83690/0xa40083ab9bf52f91 lrc: 2/0,0 mode: --/COS res: [0x2000013a1:0x5:0x0].0x0 bits 0x2/0x0 rrc: 4 type: IBT gid 0 flags: 0x44a19400000000 nid: local remote: 0x0 expref: -99 pid: 6255 timeout: 0 lvb_type: 0 00010000:00010000:1.0:1776178101.622012:0:3339:0:(ldlm_lock.c:216:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: mdt-lustre-MDT0000_UUID lock: 00000000b2e83690/0xa40083ab9bf52f91 lrc: 0/0,0 mode: --/COS res: [0x2000013a1:0x5:0x0].0x0 bits 0x2/0x0 rrc: 4 type: IBT gid 0 flags: 0x44a19400000000 nid: local remote: 0x0 expref: -99 pid: 6255 timeout: 0 lvb_type: 0 00010000:00010000:1.0:1776178101.622021:0:3339:0:(ldlm_lock.c:820:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(COS) ns: mdt-lustre-MDT0000_UUID lock: 0000000056462e0c/0xa40083ab9bf52fa6 lrc: 3/0,1 mode: COS/COS res: [0x2000013a1:0x1:0x0].0xef09e6ea bits 0x2/0x0 rrc: 2 type: IBT gid 0 flags: 0x50210000000000 nid: local remote: 0x0 expref: -99 pid: 6255 timeout: 0 lvb_type: 0 00010000:00010000:1.0:1776178101.622026:0:3339:0:(ldlm_lock.c:879:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: mdt-lustre-MDT0000_UUID lock: 0000000056462e0c/0xa40083ab9bf52fa6 lrc: 2/0,0 mode: COS/COS res: [0x2000013a1:0x1:0x0].0xef09e6ea bits 0x2/0x0 rrc: 2 type: IBT gid 0 flags: 0x50210400000000 nid: local remote: 0x0 expref: -99 pid: 6255 timeout: 0 lvb_type: 0 00010000:00010000:1.0:1776178101.622032:0:3339:0:(ldlm_lockd.c:1879:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: mdt-lustre-MDT0000_UUID lock: 0000000056462e0c/0xa40083ab9bf52fa6 lrc: 3/0,0 mode: COS/COS res: [0x2000013a1:0x1:0x0].0xef09e6ea bits 0x2/0x0 rrc: 2 type: IBT gid 0 flags: 0x40210400000000 nid: local remote: 0x0 expref: -99 pid: 6255 timeout: 0 lvb_type: 0 00010000:00010000:1.0:1776178101.622037:0:3339:0:(ldlm_lockd.c:1893:ldlm_handle_bl_callback()) Lock 0000000056462e0c already unused, calling callback (0000000079e78348) 00010000:00010000:1.0:1776178101.622038:0:3339:0:(ldlm_request.c:364:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: mdt-lustre-MDT0000_UUID lock: 0000000056462e0c/0xa40083ab9bf52fa6 lrc: 3/0,0 mode: COS/COS res: [0x2000013a1:0x1:0x0].0xef09e6ea bits 0x2/0x0 rrc: 2 type: IBT gid 0 flags: 0x40210400000000 nid: local remote: 0x0 expref: -99 pid: 6255 timeout: 0 lvb_type: 0 00010000:00010000:1.0:1776178101.622044:0:3339:0:(ldlm_request.c:1257:ldlm_cli_cancel_local()) ### server-side local cancel ns: mdt-lustre-MDT0000_UUID lock: 0000000056462e0c/0xa40083ab9bf52fa6 lrc: 4/0,0 mode: COS/COS res: [0x2000013a1:0x1:0x0].0xef09e6ea bits 0x2/0x0 rrc: 2 type: IBT gid 0 flags: 0x40218400000000 nid: local remote: 0x0 expref: -99 pid: 6255 timeout: 0 lvb_type: 0 00010000:00010000:1.0:1776178101.622050:0:3339:0:(ldlm_inodebits.c:94:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x2000013a1:0x1:0x0].0xef09e6ea (0000000090c54b62) 00010000:00010000:1.0:1776178101.622052:0:3339:0:(ldlm_inodebits.c:96:ldlm_reprocess_inodebits_queue()) Hint 2 00010000:00010000:1.0:1776178101.622054:0:3339:0:(ldlm_lockd.c:1903:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: mdt-lustre-MDT0000_UUID lock: 0000000056462e0c/0xa40083ab9bf52fa6 lrc: 2/0,0 mode: --/COS res: [0x2000013a1:0x1:0x0].0xef09e6ea bits 0x2/0x0 rrc: 2 type: IBT gid 0 flags: 0x44a19400000000 nid: local remote: 0x0 expref: -99 pid: 6255 timeout: 0 lvb_type: 0 00010000:00010000:1.0:1776178101.622115:0:3339:0:(ldlm_lock.c:216:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: mdt-lustre-MDT0000_UUID lock: 0000000056462e0c/0xa40083ab9bf52fa6 lrc: 0/0,0 mode: --/COS res: [0x2000013a1:0x1:0x0].0xef09e6ea bits 0x2/0x0 rrc: 2 type: IBT gid 0 flags: 0x44a19400000000 nid: local remote: 0x0 expref: -99 pid: 6255 timeout: 0 lvb_type: 0 00010000:00010000:1.0:1776178101.622141:0:3339:0:(ldlm_lock.c:820:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(COS) ns: mdt-lustre-MDT0000_UUID lock: 0000000097975a3b/0xa40083ab9bf52fbb lrc: 3/0,1 mode: COS/COS res: [0x2000013a1:0x6:0x0].0x0 bits 0x2/0x0 rrc: 4 type: IBT gid 0 flags: 0x50210000000000 nid: local remote: 0x0 expref: -99 pid: 6255 timeout: 0 lvb_type: 0 00010000:00010000:1.0:1776178101.622151:0:3339:0:(ldlm_lock.c:879:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: mdt-lustre-MDT0000_UUID lock: 0000000097975a3b/0xa40083ab9bf52fbb lrc: 2/0,0 mode: COS/COS res: [0x2000013a1:0x6:0x0].0x0 bits 0x2/0x0 rrc: 4 type: IBT gid 0 flags: 0x50210400000000 nid: local remote: 0x0 expref: -99 pid: 6255 timeout: 0 lvb_type: 0 00010000:00010000:1.0:1776178101.622157:0:3339:0:(ldlm_lockd.c:1879:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: mdt-lustre-MDT0000_UUID lock: 0000000097975a3b/0xa40083ab9bf52fbb lrc: 3/0,0 mode: COS/COS res: [0x2000013a1:0x6:0x0].0x0 bits 0x2/0x0 rrc: 4 type: IBT gid 0 flags: 0x40210400000000 nid: local remote: 0x0 expref: -99 pid: 6255 timeout: 0 lvb_type: 0 00010000:00010000:1.0:1776178101.622161:0:3339:0:(ldlm_lockd.c:1893:ldlm_handle_bl_callback()) Lock 0000000097975a3b already unused, calling callback (0000000079e78348) 00010000:00010000:1.0:1776178101.622163:0:3339:0:(ldlm_request.c:364:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: mdt-lustre-MDT0000_UUID lock: 0000000097975a3b/0xa40083ab9bf52fbb lrc: 3/0,0 mode: COS/COS res: [0x2000013a1:0x6:0x0].0x0 bits 0x2/0x0 rrc: 4 type: IBT gid 0 flags: 0x40210400000000 nid: local remote: 0x0 expref: -99 pid: 6255 timeout: 0 lvb_type: 0 00010000:00010000:1.0:1776178101.622168:0:3339:0:(ldlm_request.c:1257:ldlm_cli_cancel_local()) ### server-side local cancel ns: mdt-lustre-MDT0000_UUID lock: 0000000097975a3b/0xa40083ab9bf52fbb lrc: 4/0,0 mode: COS/COS res: [0x2000013a1:0x6:0x0].0x0 bits 0x2/0x0 rrc: 4 type: IBT gid 0 flags: 0x40218400000000 nid: local remote: 0x0 expref: -99 pid: 6255 timeout: 0 lvb_type: 0 00010000:00010000:1.0:1776178101.622174:0:3339:0:(ldlm_inodebits.c:94:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x2000013a1:0x6:0x0].0x0 (00000000ef7185e3) 00010000:00010000:1.0:1776178101.622176:0:3339:0:(ldlm_inodebits.c:96:ldlm_reprocess_inodebits_queue()) Hint 2 00010000:00010000:1.0:1776178101.622178:0:3339:0:(ldlm_lockd.c:1903:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: mdt-lustre-MDT0000_UUID lock: 0000000097975a3b/0xa40083ab9bf52fbb lrc: 2/0,0 mode: --/COS res: [0x2000013a1:0x6:0x0].0x0 bits 0x2/0x0 rrc: 4 type: IBT gid 0 flags: 0x44a19400000000 nid: local remote: 0x0 expref: -99 pid: 6255 timeout: 0 lvb_type: 0 00010000:00010000:1.0:1776178101.622183:0:3339:0:(ldlm_lock.c:216:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: mdt-lustre-MDT0000_UUID lock: 0000000097975a3b/0xa40083ab9bf52fbb lrc: 0/0,0 mode: --/COS res: [0x2000013a1:0x6:0x0].0x0 bits 0x2/0x0 rrc: 4 type: IBT gid 0 flags: 0x44a19400000000 nid: local remote: 0x0 expref: -99 pid: 6255 timeout: 0 lvb_type: 0 00010000:00010000:1.0:1776178101.622191:0:3339:0:(ldlm_lock.c:820:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(COS) ns: mdt-lustre-MDT0000_UUID lock: 000000000b5c0143/0xa40083ab9bf52fd0 lrc: 3/0,1 mode: COS/COS res: [0x2000013a1:0x1:0x0].0x37639fea bits 0x2/0x0 rrc: 2 type: IBT gid 0 flags: 0x50210000000000 nid: local remote: 0x0 expref: -99 pid: 6256 timeout: 0 lvb_type: 0 00010000:00010000:1.0:1776178101.622197:0:3339:0:(ldlm_lock.c:879:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: mdt-lustre-MDT0000_UUID lock: 000000000b5c0143/0xa40083ab9bf52fd0 lrc: 2/0,0 mode: COS/COS res: [0x2000013a1:0x1:0x0].0x37639fea bits 0x2/0x0 rrc: 2 type: IBT gid 0 flags: 0x50210400000000 nid: local remote: 0x0 expref: -99 pid: 6256 timeout: 0 lvb_type: 0 00010000:00010000:1.0:1776178101.622202:0:3339:0:(ldlm_lockd.c:1879:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: mdt-lustre-MDT0000_UUID lock: 000000000b5c0143/0xa40083ab9bf52fd0 lrc: 3/0,0 mode: COS/COS res: [0x2000013a1:0x1:0x0].0x37639fea bits 0x2/0x0 rrc: 2 type: IBT gid 0 flags: 0x40210400000000 nid: local remote: 0x0 expref: -99 pid: 6256 timeout: 0 lvb_type: 0 00010000:00010000:1.0:1776178101.622212:0:3339:0:(ldlm_lockd.c:1893:ldlm_handle_bl_callback()) Lock 000000000b5c0143 already unused, calling callback (0000000079e78348) 00010000:00010000:1.0:1776178101.622214:0:3339:0:(ldlm_request.c:364:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: mdt-lustre-MDT0000_UUID lock: 000000000b5c0143/0xa40083ab9bf52fd0 lrc: 3/0,0 mode: COS/COS res: [0x2000013a1:0x1:0x0].0x37639fea bits 0x2/0x0 rrc: 2 type: IBT gid 0 flags: 0x40210400000000 nid: local remote: 0x0 expref: -99 pid: 6256 timeout: 0 lvb_type: 0 00010000:00010000:1.0:1776178101.622219:0:3339:0:(ldlm_request.c:1257:ldlm_cli_cancel_local()) ### server-side local cancel ns: mdt-lustre-MDT0000_UUID lock: 000000000b5c0143/0xa40083ab9bf52fd0 lrc: 4/0,0 mode: COS/COS res: [0x2000013a1:0x1:0x0].0x37639fea bits 0x2/0x0 rrc: 2 type: IBT gid 0 flags: 0x40218400000000 nid: local remote: 0x0 expref: -99 pid: 6256 timeout: 0 lvb_type: 0 00010000:00010000:1.0:1776178101.622225:0:3339:0:(ldlm_inodebits.c:94:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x2000013a1:0x1:0x0].0x37639fea (00000000e4c83f6d) 00010000:00010000:1.0:1776178101.622227:0:3339:0:(ldlm_inodebits.c:96:ldlm_reprocess_inodebits_queue()) Hint 2 00010000:00010000:1.0:1776178101.622228:0:3339:0:(ldlm_lockd.c:1903:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: mdt-lustre-MDT0000_UUID lock: 000000000b5c0143/0xa40083ab9bf52fd0 lrc: 2/0,0 mode: --/COS res: [0x2000013a1:0x1:0x0].0x37639fea bits 0x2/0x0 rrc: 2 type: IBT gid 0 flags: 0x44a19400000000 nid: local remote: 0x0 expref: -99 pid: 6256 timeout: 0 lvb_type: 0 00010000:00010000:1.0:1776178101.622233:0:3339:0:(ldlm_lock.c:216:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: mdt-lustre-MDT0000_UUID lock: 000000000b5c0143/0xa40083ab9bf52fd0 lrc: 0/0,0 mode: --/COS res: [0x2000013a1:0x1:0x0].0x37639fea bits 0x2/0x0 rrc: 2 type: IBT gid 0 flags: 0x44a19400000000 nid: local remote: 0x0 expref: -99 pid: 6256 timeout: 0 lvb_type: 0 00010000:00010000:1.0:1776178101.622242:0:3339:0:(ldlm_lock.c:820:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(COS) ns: mdt-lustre-MDT0000_UUID lock: 00000000319537bc/0xa40083ab9bf52fe5 lrc: 3/0,1 mode: COS/COS res: [0x2000013a1:0x7:0x0].0x0 bits 0x2/0x0 rrc: 4 type: IBT gid 0 flags: 0x50210000000000 nid: local remote: 0x0 expref: -99 pid: 6256 timeout: 0 lvb_type: 0 00010000:00010000:1.0:1776178101.622248:0:3339:0:(ldlm_lock.c:879:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: mdt-lustre-MDT0000_UUID lock: 00000000319537bc/0xa40083ab9bf52fe5 lrc: 2/0,0 mode: COS/COS res: [0x2000013a1:0x7:0x0].0x0 bits 0x2/0x0 rrc: 4 type: IBT gid 0 flags: 0x50210400000000 nid: local remote: 0x0 expref: -99 pid: 6256 timeout: 0 lvb_type: 0 00010000:00010000:1.0:1776178101.622253:0:3339:0:(ldlm_lockd.c:1879:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: mdt-lustre-MDT0000_UUID lock: 00000000319537bc/0xa40083ab9bf52fe5 lrc: 3/0,0 mode: COS/COS res: [0x2000013a1:0x7:0x0].0x0 bits 0x2/0x0 rrc: 4 type: IBT gid 0 flags: 0x40210400000000 nid: local remote: 0x0 expref: -99 pid: 6256 timeout: 0 lvb_type: 0 00010000:00010000:1.0:1776178101.622257:0:3339:0:(ldlm_lockd.c:1893:ldlm_handle_bl_callback()) Lock 00000000319537bc already unused, calling callback (0000000079e78348) 00010000:00010000:1.0:1776178101.622259:0:3339:0:(ldlm_request.c:364:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: mdt-lustre-MDT0000_UUID lock: 00000000319537bc/0xa40083ab9bf52fe5 lrc: 3/0,0 mode: COS/COS res: [0x2000013a1:0x7:0x0].0x0 bits 0x2/0x0 rrc: 4 type: IBT gid 0 flags: 0x40210400000000 nid: local remote: 0x0 expref: -99 pid: 6256 timeout: 0 lvb_type: 0 00010000:00010000:1.0:1776178101.622264:0:3339:0:(ldlm_request.c:1257:ldlm_cli_cancel_local()) ### server-side local cancel ns: mdt-lustre-MDT0000_UUID lock: 00000000319537bc/0xa40083ab9bf52fe5 lrc: 4/0,0 mode: COS/COS res: [0x2000013a1:0x7:0x0].0x0 bits 0x2/0x0 rrc: 4 type: IBT gid 0 flags: 0x40218400000000 nid: local remote: 0x0 expref: -99 pid: 6256 timeout: 0 lvb_type: 0 00010000:00010000:1.0:1776178101.622271:0:3339:0:(ldlm_inodebits.c:94:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x2000013a1:0x7:0x0].0x0 (00000000665a3017) 00010000:00010000:1.0:1776178101.622274:0:3339:0:(ldlm_inodebits.c:96:ldlm_reprocess_inodebits_queue()) Hint 2 00010000:00010000:1.0:1776178101.622275:0:3339:0:(ldlm_lockd.c:1903:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: mdt-lustre-MDT0000_UUID lock: 00000000319537bc/0xa40083ab9bf52fe5 lrc: 2/0,0 mode: --/COS res: [0x2000013a1:0x7:0x0].0x0 bits 0x2/0x0 rrc: 4 type: IBT gid 0 flags: 0x44a19400000000 nid: local remote: 0x0 expref: -99 pid: 6256 timeout: 0 lvb_type: 0 00010000:00010000:1.0:1776178101.622280:0:3339:0:(ldlm_lock.c:216:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: mdt-lustre-MDT0000_UUID lock: 00000000319537bc/0xa40083ab9bf52fe5 lrc: 0/0,0 mode: --/COS res: [0x2000013a1:0x7:0x0].0x0 bits 0x2/0x0 rrc: 4 type: IBT gid 0 flags: 0x44a19400000000 nid: local remote: 0x0 expref: -99 pid: 6256 timeout: 0 lvb_type: 0 00010000:00010000:1.0:1776178101.622288:0:3339:0:(ldlm_lock.c:820:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(COS) ns: mdt-lustre-MDT0000_UUID lock: 00000000000cfba0/0xa40083ab9bf52ffa lrc: 3/0,1 mode: COS/COS res: [0x2000013a1:0x1:0x0].0x7fbd58ea bits 0x2/0x0 rrc: 2 type: IBT gid 0 flags: 0x50210000000000 nid: local remote: 0x0 expref: -99 pid: 6256 timeout: 0 lvb_type: 0 00010000:00010000:1.0:1776178101.622293:0:3339:0:(ldlm_lock.c:879:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: mdt-lustre-MDT0000_UUID lock: 00000000000cfba0/0xa40083ab9bf52ffa lrc: 2/0,0 mode: COS/COS res: [0x2000013a1:0x1:0x0].0x7fbd58ea bits 0x2/0x0 rrc: 2 type: IBT gid 0 flags: 0x50210400000000 nid: local remote: 0x0 expref: -99 pid: 6256 timeout: 0 lvb_type: 0 00010000:00010000:1.0:1776178101.622298:0:3339:0:(ldlm_lockd.c:1879:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: mdt-lustre-MDT0000_UUID lock: 00000000000cfba0/0xa40083ab9bf52ffa lrc: 3/0,0 mode: COS/COS res: [0x2000013a1:0x1:0x0].0x7fbd58ea bits 0x2/0x0 rrc: 2 type: IBT gid 0 flags: 0x40210400000000 nid: local remote: 0x0 expref: -99 pid: 6256 timeout: 0 lvb_type: 0 00010000:00010000:1.0:1776178101.622303:0:3339:0:(ldlm_lockd.c:1893:ldlm_handle_bl_callback()) Lock 00000000000cfba0 already unused, calling callback (0000000079e78348) 00010000:00010000:1.0:1776178101.622304:0:3339:0:(ldlm_request.c:364:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: mdt-lustre-MDT0000_UUID lock: 00000000000cfba0/0xa40083ab9bf52ffa lrc: 3/0,0 mode: COS/COS res: [0x2000013a1:0x1:0x0].0x7fbd58ea bits 0x2/0x0 rrc: 2 type: IBT gid 0 flags: 0x40210400000000 nid: local remote: 0x0 expref: -99 pid: 6256 timeout: 0 lvb_type: 0 00010000:00010000:1.0:1776178101.622309:0:3339:0:(ldlm_request.c:1257:ldlm_cli_cancel_local()) ### server-side local cancel ns: mdt-lustre-MDT0000_UUID lock: 00000000000cfba0/0xa40083ab9bf52ffa lrc: 4/0,0 mode: COS/COS res: [0x2000013a1:0x1:0x0].0x7fbd58ea bits 0x2/0x0 rrc: 2 type: IBT gid 0 flags: 0x40218400000000 nid: local remote: 0x0 expref: -99 pid: 6256 timeout: 0 lvb_type: 0 00010000:00010000:1.0:1776178101.622315:0:3339:0:(ldlm_inodebits.c:94:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x2000013a1:0x1:0x0].0x7fbd58ea (00000000fdbb109f) 00010000:00010000:1.0:1776178101.622317:0:3339:0:(ldlm_inodebits.c:96:ldlm_reprocess_inodebits_queue()) Hint 2 00010000:00010000:1.0:1776178101.622319:0:3339:0:(ldlm_lockd.c:1903:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: mdt-lustre-MDT0000_UUID lock: 00000000000cfba0/0xa40083ab9bf52ffa lrc: 2/0,0 mode: --/COS res: [0x2000013a1:0x1:0x0].0x7fbd58ea bits 0x2/0x0 rrc: 2 type: IBT gid 0 flags: 0x44a19400000000 nid: local remote: 0x0 expref: -99 pid: 6256 timeout: 0 lvb_type: 0 00010000:00010000:1.0:1776178101.622330:0:3339:0:(ldlm_lock.c:216:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: mdt-lustre-MDT0000_UUID lock: 00000000000cfba0/0xa40083ab9bf52ffa lrc: 0/0,0 mode: --/COS res: [0x2000013a1:0x1:0x0].0x7fbd58ea bits 0x2/0x0 rrc: 2 type: IBT gid 0 flags: 0x44a19400000000 nid: local remote: 0x0 expref: -99 pid: 6256 timeout: 0 lvb_type: 0 00010000:00010000:1.0:1776178101.622340:0:3339:0:(ldlm_lock.c:820:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(COS) ns: mdt-lustre-MDT0000_UUID lock: 00000000a44d8583/0xa40083ab9bf5300f lrc: 3/0,1 mode: COS/COS res: [0x2000013a1:0x8:0x0].0x0 bits 0x2/0x0 rrc: 4 type: IBT gid 0 flags: 0x50210000000000 nid: local remote: 0x0 expref: -99 pid: 6256 timeout: 0 lvb_type: 0 00010000:00010000:1.0:1776178101.622345:0:3339:0:(ldlm_lock.c:879:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: mdt-lustre-MDT0000_UUID lock: 00000000a44d8583/0xa40083ab9bf5300f lrc: 2/0,0 mode: COS/COS res: [0x2000013a1:0x8:0x0].0x0 bits 0x2/0x0 rrc: 4 type: IBT gid 0 flags: 0x50210400000000 nid: local remote: 0x0 expref: -99 pid: 6256 timeout: 0 lvb_type: 0 00010000:00010000:1.0:1776178101.622351:0:3339:0:(ldlm_lockd.c:1879:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: mdt-lustre-MDT0000_UUID lock: 00000000a44d8583/0xa40083ab9bf5300f lrc: 3/0,0 mode: COS/COS res: [0x2000013a1:0x8:0x0].0x0 bits 0x2/0x0 rrc: 4 type: IBT gid 0 flags: 0x40210400000000 nid: local remote: 0x0 expref: -99 pid: 6256 timeout: 0 lvb_type: 0 00010000:00010000:1.0:1776178101.622356:0:3339:0:(ldlm_lockd.c:1893:ldlm_handle_bl_callback()) Lock 00000000a44d8583 already unused, calling callback (0000000079e78348) 00010000:00010000:1.0:1776178101.622358:0:3339:0:(ldlm_request.c:364:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: mdt-lustre-MDT0000_UUID lock: 00000000a44d8583/0xa40083ab9bf5300f lrc: 3/0,0 mode: COS/COS res: [0x2000013a1:0x8:0x0].0x0 bits 0x2/0x0 rrc: 4 type: IBT gid 0 flags: 0x40210400000000 nid: local remote: 0x0 expref: -99 pid: 6256 timeout: 0 lvb_type: 0 00010000:00010000:1.0:1776178101.622363:0:3339:0:(ldlm_request.c:1257:ldlm_cli_cancel_local()) ### server-side local cancel ns: mdt-lustre-MDT0000_UUID lock: 00000000a44d8583/0xa40083ab9bf5300f lrc: 4/0,0 mode: COS/COS res: [0x2000013a1:0x8:0x0].0x0 bits 0x2/0x0 rrc: 4 type: IBT gid 0 flags: 0x40218400000000 nid: local remote: 0x0 expref: -99 pid: 6256 timeout: 0 lvb_type: 0 00010000:00010000:1.0:1776178101.622369:0:3339:0:(ldlm_inodebits.c:94:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x2000013a1:0x8:0x0].0x0 (00000000e9f0d179) 00010000:00010000:1.0:1776178101.622371:0:3339:0:(ldlm_inodebits.c:96:ldlm_reprocess_inodebits_queue()) Hint 2 00010000:00010000:1.0:1776178101.622372:0:3339:0:(ldlm_lockd.c:1903:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: mdt-lustre-MDT0000_UUID lock: 00000000a44d8583/0xa40083ab9bf5300f lrc: 2/0,0 mode: --/COS res: [0x2000013a1:0x8:0x0].0x0 bits 0x2/0x0 rrc: 4 type: IBT gid 0 flags: 0x44a19400000000 nid: local remote: 0x0 expref: -99 pid: 6256 timeout: 0 lvb_type: 0 00010000:00010000:1.0:1776178101.622375:0:3339:0:(ldlm_lock.c:216:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: mdt-lustre-MDT0000_UUID lock: 00000000a44d8583/0xa40083ab9bf5300f lrc: 0/0,0 mode: --/COS res: [0x2000013a1:0x8:0x0].0x0 bits 0x2/0x0 rrc: 4 type: IBT gid 0 flags: 0x44a19400000000 nid: local remote: 0x0 expref: -99 pid: 6256 timeout: 0 lvb_type: 0 00010000:00010000:3.0:1776178101.664380:0:6244:0:(ldlm_lockd.c:2792:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00010000:00010000:0.0:1776178102.688493:0:6245:0:(ldlm_lockd.c:2792:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:0.0:1776178103.136565:0:3356:0:(ptlrpcd.c:415:ptlrpcd_check()) transfer 1 async RPCs [2->3] 00000100:00100000:0.0:1776178103.136607:0:3356:0:(client.c:1733:ptlrpc_send_new_req()) Sending RPC req@000000008c469ec0 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_03:lustre-MDT0000-mdtlov_UUID:3356:1862456998021504:0@lo:13:osp-pre-0-0.0 00000100:00100000:0.0:1776178103.136709:0:3356:0:(events.c:361:request_in_callback()) peer: 12345-0@lo (source: 12345-0@lo) 00000100:00100000:0.0:1776178103.136756:0:3356:0:(client.c:1733:ptlrpc_send_new_req()) Sending RPC req@0000000023210d1c pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_03:ac61f839-0e8e-4cac-9c59-08bcfbbad330:3356:1862456998021568:0@lo:400:kworker/u8:0.0 00000100:00100000:0.0:1776178103.136795:0:3356:0:(events.c:361:request_in_callback()) peer: 12345-0@lo (source: 12345-0@lo) 00000100:00100000:0.0:1776178103.136813:0:3356:0:(client.c:1733:ptlrpc_send_new_req()) Sending RPC req@00000000ecb64602 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_03:lustre-MDT0001-lwp-OST0001_UUID:3356:1862456998021824:0@lo:400:kworker/u8:0.0 00000100:00100000:0.0:1776178103.136841:0:3356:0:(events.c:361:request_in_callback()) peer: 12345-0@lo (source: 12345-0@lo) 00000100:00100000:0.0:1776178103.136858:0:3356:0:(client.c:1733:ptlrpc_send_new_req()) Sending RPC req@000000005643ab1b pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_03:lustre-MDT0000-lwp-MDT0001_UUID:3356:1862456998022080:0@lo:400:kworker/u8:0.0 00000100:00100000:0.0:1776178103.136879:0:3356:0:(events.c:361:request_in_callback()) peer: 12345-0@lo (source: 12345-0@lo) 00000100:00100000:0.0:1776178103.136893:0:3356:0:(ptlrpcd.c:415:ptlrpcd_check()) transfer 3 async RPCs [2->3] 00000100:00100000:0.0:1776178103.136898:0:3356:0:(client.c:1733:ptlrpc_send_new_req()) Sending RPC req@000000005927b2c3 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_03:lustre-MDT0000-lwp-OST0001_UUID:3356:1862456998021760:0@lo:400:kworker/u8:0.0 00000100:00100000:0.0:1776178103.136916:0:3356:0:(events.c:361:request_in_callback()) peer: 12345-0@lo (source: 12345-0@lo) 00000100:00100000:0.0:1776178103.136929:0:3356:0:(client.c:1733:ptlrpc_send_new_req()) Sending RPC req@0000000047458d9f pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_03:lustre-MDT0001-mdtlov_UUID:3356:1862456998022016:0@lo:400:kworker/u8:0.0 00000100:00100000:0.0:1776178103.136950:0:3356:0:(events.c:361:request_in_callback()) peer: 12345-0@lo (source: 12345-0@lo) 00000100:00100000:0.0:1776178103.136963:0:3356:0:(client.c:1733:ptlrpc_send_new_req()) Sending RPC req@000000007fcab602 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_03:lustre-MDT0000-lwp-MDT0000_UUID:3356:1862456998022272:0@lo:400:kworker/u8:0.0 00000100:00100000:0.0:1776178103.136992:0:3356:0:(events.c:361:request_in_callback()) peer: 12345-0@lo (source: 12345-0@lo) 00000100:00100000:0.0:1776178103.137028:0:8232:0:(service.c:2152:ptlrpc_server_handle_req_in()) got req x1862456998021504 00000100:00100000:0.0:1776178103.137097:0:8232:0:(nrs_fifo.c:179:nrs_fifo_req_get()) NRS start fifo request from 12345-0@lo, seq: 399 00000100:00100000:0.0:1776178103.137103:0:8232:0:(service.c:2324:ptlrpc_server_handle_request()) Handling RPC req@0000000022c7a99e pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_create00:lustre-MDT0000-mdtlov_UUID+5:3356:x1862456998021504:12345-0@lo:13:osp-pre-0-0.0 00000100:00100000:1.0:1776178103.137105:0:11281:0:(service.c:2152:ptlrpc_server_handle_req_in()) got req x1862456998021824 00002000:00000024:0.0:1776178103.137140:0:8232:0:(ofd_obd.c:740:ofd_statfs()) blocks cached 315392 granted 484883776 pending 0 free 3923689472 avail 3692167168 00000100:00100000:1.0:1776178103.137143:0:11281:0:(service.c:2152:ptlrpc_server_handle_req_in()) got req x1862456998022080 00000100:00100000:1.0:1776178103.137158:0:11281:0:(service.c:2152:ptlrpc_server_handle_req_in()) got req x1862456998021760 00000100:00100000:1.0:1776178103.137171:0:11281:0:(service.c:2152:ptlrpc_server_handle_req_in()) got req x1862456998022272 00000100:00100000:1.0:1776178103.137190:0:11281:0:(nrs_fifo.c:179:nrs_fifo_req_get()) NRS start fifo request from 12345-0@lo, seq: 747 00000100:00100000:0.0:1776178103.137194:0:8232:0:(service.c:2372:ptlrpc_server_handle_request()) Handled RPC req@0000000022c7a99e pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_create00:lustre-MDT0000-mdtlov_UUID+5:3356:x1862456998021504:12345-0@lo:13:osp-pre-0-0.0 Request processed in 88us (484us total) trans 0 rc 0/0 00000100:00100000:1.0:1776178103.137200:0:11281:0:(service.c:2324:ptlrpc_server_handle_request()) Handling RPC req@00000000bd43c651 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_004:lustre-MDT0001-lwp-OST0001_UUID+5:3356:x1862456998021824:12345-0@lo:400:kworker/u8:0.0 00000100:00100000:0.0:1776178103.137206:0:8232:0:(nrs_fifo.c:241:nrs_fifo_req_stop()) NRS stop fifo request from 12345-0@lo, seq: 399 00000100:00100000:0.0:1776178103.137236:0:20140:0:(service.c:2152:ptlrpc_server_handle_req_in()) got req x1862456998021568 00000100:00100000:1.0:1776178103.137246:0:11281:0:(service.c:2372:ptlrpc_server_handle_request()) Handled RPC req@00000000bd43c651 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_004:lustre-MDT0001-lwp-OST0001_UUID+5:3356:x1862456998021824:12345-0@lo:400:kworker/u8:0.0 Request processed in 48us (404us total) trans 0 rc 0/0 00000100:00100000:1.0:1776178103.137254:0:11281:0:(nrs_fifo.c:241:nrs_fifo_req_stop()) NRS stop fifo request from 12345-0@lo, seq: 747 00000100:00100000:0.0:1776178103.137264:0:20140:0:(nrs_fifo.c:179:nrs_fifo_req_get()) NRS start fifo request from 12345-0@lo, seq: 115 00000100:00100000:0.0:1776178103.137269:0:20140:0:(service.c:2324:ptlrpc_server_handle_request()) Handling RPC req@0000000080051315 pname:cluuid+ref:pid:xid:nid:opc:job ll_mgs_0002:ac61f839-0e8e-4cac-9c59-08bcfbbad330+23:3356:x1862456998021568:12345-0@lo:400:kworker/u8:0.0 00000100:00100000:1.0:1776178103.137275:0:11281:0:(nrs_fifo.c:179:nrs_fifo_req_get()) NRS start fifo request from 12345-0@lo, seq: 748 00000100:00100000:1.0:1776178103.137279:0:11281:0:(service.c:2324:ptlrpc_server_handle_request()) Handling RPC req@000000001879c2bc pname:cluuid+ref:pid:xid:nid:opc:job mdt00_004:lustre-MDT0000-lwp-MDT0001_UUID+5:3356:x1862456998022080:12345-0@lo:400:kworker/u8:0.0 00000100:00100000:3.0:1776178103.137472:0:3353:0:(client.c:1733:ptlrpc_send_new_req()) Sending RPC req@00000000f0ef812c pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_00:lustre-MDT0000-lwp-OST0000_UUID:3353:1862456998021632:0@lo:400:kworker/u8:0.0 00000100:00100000:2.0:1776178103.137502:0:6257:0:(nrs_fifo.c:179:nrs_fifo_req_get()) NRS start fifo request from 12345-0@lo, seq: 749 00000100:00100000:2.0:1776178103.137511:0:6257:0:(service.c:2324:ptlrpc_server_handle_request()) Handling RPC req@000000009383c871 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_002:lustre-MDT0000-lwp-OST0001_UUID+5:3356:x1862456998021760:12345-0@lo:400:kworker/u8:0.0 00000100:00100000:3.0:1776178103.137514:0:3353:0:(events.c:361:request_in_callback()) peer: 12345-0@lo (source: 12345-0@lo) 00000100:00100000:3.0:1776178103.137530:0:3353:0:(client.c:1733:ptlrpc_send_new_req()) Sending RPC req@00000000f95e9da5 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_00:lustre-MDT0001-mdtlov_UUID:3353:1862456998021888:0@lo:400:kworker/u8:0.0 00000100:00100000:2.0:1776178103.137560:0:6257:0:(service.c:2372:ptlrpc_server_handle_request()) Handled RPC req@000000009383c871 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_002:lustre-MDT0000-lwp-OST0001_UUID+5:3356:x1862456998021760:12345-0@lo:400:kworker/u8:0.0 Request processed in 51us (643us total) trans 0 rc 0/0 00000100:00100000:2.0:1776178103.137567:0:6257:0:(nrs_fifo.c:241:nrs_fifo_req_stop()) NRS stop fifo request from 12345-0@lo, seq: 749 00000100:00100000:2.0:1776178103.137581:0:6257:0:(service.c:2152:ptlrpc_server_handle_req_in()) got req x1862456998021632 00000100:00100000:3.0:1776178103.137604:0:3353:0:(events.c:361:request_in_callback()) peer: 12345-0@lo (source: 12345-0@lo) 00000100:00100000:2.0:1776178103.137610:0:6257:0:(nrs_fifo.c:179:nrs_fifo_req_get()) NRS start fifo request from 12345-0@lo, seq: 750 00000100:00100000:2.0:1776178103.137615:0:6257:0:(service.c:2324:ptlrpc_server_handle_request()) Handling RPC req@000000002b0faed1 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_002:lustre-MDT0000-lwp-MDT0000_UUID+5:3356:x1862456998022272:12345-0@lo:400:kworker/u8:0.0 00000100:00100000:3.0:1776178103.137637:0:3353:0:(client.c:1733:ptlrpc_send_new_req()) Sending RPC req@00000000e64148ab pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_00:lustre-MDT0000-mdtlov_UUID:3353:1862456998022144:0@lo:400:kworker/u8:0.0 00000100:00100000:2.0:1776178103.137662:0:6257:0:(service.c:2372:ptlrpc_server_handle_request()) Handled RPC req@000000002b0faed1 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_002:lustre-MDT0000-lwp-MDT0000_UUID+5:3356:x1862456998022272:12345-0@lo:400:kworker/u8:0.0 Request processed in 45us (669us total) trans 0 rc 0/0 00000100:00100000:3.0:1776178103.137670:0:3353:0:(events.c:361:request_in_callback()) peer: 12345-0@lo (source: 12345-0@lo) 00000100:00100000:2.0:1776178103.137673:0:6257:0:(nrs_fifo.c:241:nrs_fifo_req_stop()) NRS stop fifo request from 12345-0@lo, seq: 750 00000100:00100000:2.0:1776178103.137687:0:6257:0:(nrs_fifo.c:179:nrs_fifo_req_get()) NRS start fifo request from 12345-0@lo, seq: 751 00000100:00100000:2.0:1776178103.137691:0:6257:0:(service.c:2324:ptlrpc_server_handle_request()) Handling RPC req@000000004451d94d pname:cluuid+ref:pid:xid:nid:opc:job mdt00_002:lustre-MDT0000-lwp-OST0000_UUID+5:3353:x1862456998021632:12345-0@lo:400:kworker/u8:0.0 00000100:00100000:3.0:1776178103.137697:0:3353:0:(ptlrpcd.c:415:ptlrpcd_check()) transfer 3 async RPCs [1->0] 00000100:00100000:2.0:1776178103.137723:0:6257:0:(service.c:2372:ptlrpc_server_handle_request()) Handled RPC req@000000004451d94d pname:cluuid+ref:pid:xid:nid:opc:job mdt00_002:lustre-MDT0000-lwp-OST0000_UUID+5:3353:x1862456998021632:12345-0@lo:400:kworker/u8:0.0 Request processed in 31us (213us total) trans 0 rc 0/0 00000100:00100000:2.0:1776178103.137729:0:6257:0:(nrs_fifo.c:241:nrs_fifo_req_stop()) NRS stop fifo request from 12345-0@lo, seq: 751 00000100:00100000:2.0:1776178103.137774:0:6260:0:(service.c:2152:ptlrpc_server_handle_req_in()) got req x1862456998021888 00000100:00100000:2.0:1776178103.137796:0:6260:0:(service.c:2152:ptlrpc_server_handle_req_in()) got req x1862456998022144 00000100:00100000:1.0:1776178103.137800:0:11281:0:(service.c:2372:ptlrpc_server_handle_request()) Handled RPC req@000000001879c2bc pname:cluuid+ref:pid:xid:nid:opc:job mdt00_004:lustre-MDT0000-lwp-MDT0001_UUID+5:3356:x1862456998022080:12345-0@lo:400:kworker/u8:0.0 Request processed in 518us (918us total) trans 0 rc 0/0 00000100:00100000:2.0:1776178103.137813:0:6260:0:(nrs_fifo.c:179:nrs_fifo_req_get()) NRS start fifo request from 12345-0@lo, seq: 614 00000100:00100000:2.0:1776178103.137817:0:6260:0:(service.c:2324:ptlrpc_server_handle_request()) Handling RPC req@000000009c55a259 pname:cluuid+ref:pid:xid:nid:opc:job mdt_out00_000:lustre-MDT0001-mdtlov_UUID+6:3353:x1862456998021888:12345-0@lo:400:kworker/u8:0.0 00000100:00100000:2.0:1776178103.137846:0:6260:0:(service.c:2372:ptlrpc_server_handle_request()) Handled RPC req@000000009c55a259 pname:cluuid+ref:pid:xid:nid:opc:job mdt_out00_000:lustre-MDT0001-mdtlov_UUID+6:3353:x1862456998021888:12345-0@lo:400:kworker/u8:0.0 Request processed in 28us (244us total) trans 0 rc 0/0 00000100:00100000:2.0:1776178103.137851:0:6260:0:(nrs_fifo.c:241:nrs_fifo_req_stop()) NRS stop fifo request from 12345-0@lo, seq: 614 00000100:00100000:2.0:1776178103.137861:0:6260:0:(nrs_fifo.c:179:nrs_fifo_req_get()) NRS start fifo request from 12345-0@lo, seq: 615 00000100:00100000:2.0:1776178103.137864:0:6260:0:(service.c:2324:ptlrpc_server_handle_request()) Handling RPC req@0000000095214b8e pname:cluuid+ref:pid:xid:nid:opc:job mdt_out00_000:lustre-MDT0000-mdtlov_UUID+5:3353:x1862456998022144:12345-0@lo:400:kworker/u8:0.0 00000100:00100000:2.0:1776178103.137884:0:6260:0:(service.c:2372:ptlrpc_server_handle_request()) Handled RPC req@0000000095214b8e pname:cluuid+ref:pid:xid:nid:opc:job mdt_out00_000:lustre-MDT0000-mdtlov_UUID+5:3353:x1862456998022144:12345-0@lo:400:kworker/u8:0.0 Request processed in 20us (217us total) trans 0 rc 0/0 00000100:00100000:2.0:1776178103.137890:0:6260:0:(nrs_fifo.c:241:nrs_fifo_req_stop()) NRS stop fifo request from 12345-0@lo, seq: 615 00000100:00100000:2.0:1776178103.137902:0:3355:0:(ptlrpcd.c:415:ptlrpcd_check()) transfer 1 async RPCs [3->2] 00000100:00100000:2.0:1776178103.137909:0:3355:0:(client.c:1733:ptlrpc_send_new_req()) Sending RPC req@00000000efb13deb pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_02:lustre-MDT0001-mdtlov_UUID:3355:1862456998022336:0@lo:41:osp-pre-0-1.0 00000100:00100000:2.0:1776178103.137932:0:3355:0:(events.c:361:request_in_callback()) peer: 12345-0@lo (source: 12345-0@lo) 00000100:00100000:2.0:1776178103.137952:0:6261:0:(service.c:2152:ptlrpc_server_handle_req_in()) got req x1862456998022336 00000100:00100000:2.0:1776178103.137967:0:6261:0:(nrs_fifo.c:179:nrs_fifo_req_get()) NRS start fifo request from 12345-0@lo, seq: 616 00000100:00100000:2.0:1776178103.137971:0:6261:0:(service.c:2324:ptlrpc_server_handle_request()) Handling RPC req@00000000e1965319 pname:cluuid+ref:pid:xid:nid:opc:job mdt_out00_001:lustre-MDT0001-mdtlov_UUID+6:3355:x1862456998022336:12345-0@lo:41:osp-pre-0-1.0 00000004:00000024:2.0:1776178103.137990:0:6261:0:(mdt_handler.c:548:mdt_statfs()) blocks cached 0 granted 2097152 pending 0 free 1446121472 avail 1316659200 00000100:00100000:2.0:1776178103.138017:0:6261:0:(service.c:2372:ptlrpc_server_handle_request()) Handled RPC req@00000000e1965319 pname:cluuid+ref:pid:xid:nid:opc:job mdt_out00_001:lustre-MDT0001-mdtlov_UUID+6:3355:x1862456998022336:12345-0@lo:41:osp-pre-0-1.0 Request processed in 45us (84us total) trans 0 rc 0/0 00000100:00100000:2.0:1776178103.138023:0:6261:0:(nrs_fifo.c:241:nrs_fifo_req_stop()) NRS stop fifo request from 12345-0@lo, seq: 616 00000100:00100000:1.0:1776178103.138029:0:11281:0:(nrs_fifo.c:241:nrs_fifo_req_stop()) NRS stop fifo request from 12345-0@lo, seq: 748 00000100:00100000:2.0:1776178103.138057:0:3355:0:(client.c:2868:ptlrpc_free_committed()) lustre-MDT0000-osp-MDT0001: committing for last_committed 12884901924 gen 3 00000004:00080000:2.0:1776178103.138082:0:3355:0:(osp_trans.c:1044:osp_request_commit_cb()) trans no 12884901924 committed transno 12884901924 00000100:00100000:2.0:1776178103.138114:0:3355:0:(client.c:2215:ptlrpc_check_set()) Completed RPC req@00000000efb13deb pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_02:lustre-MDT0001-mdtlov_UUID:3355:1862456998022336:0@lo:41:osp-pre-0-1.0 00000100:00100000:1.0:1776178103.138146:0:3356:0:(client.c:2215:ptlrpc_check_set()) Completed RPC req@000000005927b2c3 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_03:lustre-MDT0000-lwp-OST0001_UUID:3356:1862456998021760:0@lo:400:kworker/u8:0.0 00000100:00100000:1.0:1776178103.138162:0:3356:0:(client.c:2215:ptlrpc_check_set()) Completed RPC req@000000007fcab602 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_03:lustre-MDT0000-lwp-MDT0000_UUID:3356:1862456998022272:0@lo:400:kworker/u8:0.0 00000100:00100000:2.0:1776178103.138165:0:14440:0:(service.c:2152:ptlrpc_server_handle_req_in()) got req x1862456998022016 00000100:00100000:1.0:1776178103.138174:0:3356:0:(client.c:2215:ptlrpc_check_set()) Completed RPC req@000000005643ab1b pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_03:lustre-MDT0000-lwp-MDT0001_UUID:3356:1862456998022080:0@lo:400:kworker/u8:0.0 00000100:00100000:1.0:1776178103.138188:0:3356:0:(client.c:2215:ptlrpc_check_set()) Completed RPC req@00000000ecb64602 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_03:lustre-MDT0001-lwp-OST0001_UUID:3356:1862456998021824:0@lo:400:kworker/u8:0.0 00000100:00100000:2.0:1776178103.138195:0:14440:0:(nrs_fifo.c:179:nrs_fifo_req_get()) NRS start fifo request from 12345-0@lo, seq: 403 00000100:00100000:2.0:1776178103.138201:0:14440:0:(service.c:2324:ptlrpc_server_handle_request()) Handling RPC req@00000000a1262b14 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_004:lustre-MDT0001-mdtlov_UUID+5:3356:x1862456998022016:12345-0@lo:400:kworker/u8:0.0 00000100:00100000:1.0:1776178103.138208:0:3356:0:(client.c:2215:ptlrpc_check_set()) Completed RPC req@000000008c469ec0 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_03:lustre-MDT0000-mdtlov_UUID:3356:1862456998021504:0@lo:13:osp-pre-0-0.0 00000100:00100000:2.0:1776178103.138239:0:14440:0:(service.c:2372:ptlrpc_server_handle_request()) Handled RPC req@00000000a1262b14 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_004:lustre-MDT0001-mdtlov_UUID+5:3356:x1862456998022016:12345-0@lo:400:kworker/u8:0.0 Request processed in 37us (1287us total) trans 0 rc 0/0 00000100:00100000:1.0:1776178103.138248:0:3356:0:(client.c:2215:ptlrpc_check_set()) Completed RPC req@0000000047458d9f pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_03:lustre-MDT0001-mdtlov_UUID:3356:1862456998022016:0@lo:400:kworker/u8:0.0 00000100:00100000:2.0:1776178103.138248:0:14440:0:(nrs_fifo.c:241:nrs_fifo_req_stop()) NRS stop fifo request from 12345-0@lo, seq: 403 00000100:00100000:3.0:1776178103.138428:0:3353:0:(client.c:1733:ptlrpc_send_new_req()) Sending RPC req@000000007fcab602 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_00:lustre-MDT0000-mdtlov_UUID:3353:1862456998022400:0@lo:41:osp-pre-1-0.0 00000100:00100000:3.0:1776178103.138458:0:3353:0:(events.c:361:request_in_callback()) peer: 12345-0@lo (source: 12345-0@lo) 00000100:00100000:3.0:1776178103.138480:0:3353:0:(client.c:1733:ptlrpc_send_new_req()) Sending RPC req@00000000478fd8d5 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_00:lustre-MDT0001-lwp-OST0000_UUID:3353:1862456998021696:0@lo:400:kworker/u8:0.0 00000100:00100000:0.0:1776178103.138495:0:20140:0:(service.c:2372:ptlrpc_server_handle_request()) Handled RPC req@0000000080051315 pname:cluuid+ref:pid:xid:nid:opc:job ll_mgs_0002:ac61f839-0e8e-4cac-9c59-08bcfbbad330+23:3356:x1862456998021568:12345-0@lo:400:kworker/u8:0.0 Request processed in 1223us (1699us total) trans 0 rc 0/0 00000100:00100000:0.0:1776178103.138505:0:20140:0:(nrs_fifo.c:241:nrs_fifo_req_stop()) NRS stop fifo request from 12345-0@lo, seq: 115 00000100:00100000:3.0:1776178103.138509:0:3353:0:(events.c:361:request_in_callback()) peer: 12345-0@lo (source: 12345-0@lo) 00000100:00100000:3.0:1776178103.138531:0:3353:0:(client.c:1733:ptlrpc_send_new_req()) Sending RPC req@000000001d51eec7 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_00:lustre-MDT0001-mdtlov_UUID:3353:1862456998021952:0@lo:400:kworker/u8:0.0 00000100:00100000:0.0:1776178103.138549:0:14128:0:(service.c:2152:ptlrpc_server_handle_req_in()) got req x1862456998022400 00000100:00100000:3.0:1776178103.138557:0:3353:0:(events.c:361:request_in_callback()) peer: 12345-0@lo (source: 12345-0@lo) 00000100:00100000:3.0:1776178103.138573:0:3353:0:(client.c:1733:ptlrpc_send_new_req()) Sending RPC req@00000000956deaca pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_00:lustre-MDT0000-mdtlov_UUID:3353:1862456998022208:0@lo:400:kworker/u8:0.0 00000100:00100000:0.0:1776178103.138578:0:14128:0:(nrs_fifo.c:179:nrs_fifo_req_get()) NRS start fifo request from 12345-0@lo, seq: 617 00000100:00100000:3.0:1776178103.138601:0:3353:0:(events.c:361:request_in_callback()) peer: 12345-0@lo (source: 12345-0@lo) 00000100:00100000:3.0:1776178103.138636:0:3353:0:(client.c:2215:ptlrpc_check_set()) Completed RPC req@00000000f0ef812c pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_00:lustre-MDT0000-lwp-OST0000_UUID:3353:1862456998021632:0@lo:400:kworker/u8:0.0 00000100:00100000:3.0:1776178103.138647:0:3353:0:(client.c:2215:ptlrpc_check_set()) Completed RPC req@00000000f95e9da5 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_00:lustre-MDT0001-mdtlov_UUID:3353:1862456998021888:0@lo:400:kworker/u8:0.0 00000100:00100000:3.0:1776178103.138655:0:3353:0:(client.c:2215:ptlrpc_check_set()) Completed RPC req@00000000e64148ab pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_00:lustre-MDT0000-mdtlov_UUID:3353:1862456998022144:0@lo:400:kworker/u8:0.0 00000100:00100000:3.0:1776178103.138721:0:20769:0:(service.c:2152:ptlrpc_server_handle_req_in()) got req x1862456998021952 00000100:00100000:3.0:1776178103.138744:0:20769:0:(service.c:2152:ptlrpc_server_handle_req_in()) got req x1862456998022208 00000100:00100000:3.0:1776178103.138757:0:20769:0:(nrs_fifo.c:179:nrs_fifo_req_get()) NRS start fifo request from 12345-0@lo, seq: 404 00000100:00100000:3.0:1776178103.138760:0:20769:0:(service.c:2324:ptlrpc_server_handle_request()) Handling RPC req@0000000008b3a3ad pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_018:lustre-MDT0001-mdtlov_UUID+5:3353:x1862456998021952:12345-0@lo:400:kworker/u8:0.0 00000100:00100000:3.0:1776178103.138788:0:20769:0:(service.c:2372:ptlrpc_server_handle_request()) Handled RPC req@0000000008b3a3ad pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_018:lustre-MDT0001-mdtlov_UUID+5:3353:x1862456998021952:12345-0@lo:400:kworker/u8:0.0 Request processed in 27us (232us total) trans 0 rc 0/0 00000100:00100000:3.0:1776178103.138794:0:20769:0:(nrs_fifo.c:241:nrs_fifo_req_stop()) NRS stop fifo request from 12345-0@lo, seq: 404 00000100:00100000:2.0:1776178103.138814:0:20762:0:(nrs_fifo.c:179:nrs_fifo_req_get()) NRS start fifo request from 12345-0@lo, seq: 405 00000100:00100000:2.0:1776178103.138819:0:20762:0:(service.c:2324:ptlrpc_server_handle_request()) Handling RPC req@0000000092028508 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_013:lustre-MDT0000-mdtlov_UUID+5:3353:x1862456998022208:12345-0@lo:400:kworker/u8:0.0 00000100:00100000:2.0:1776178103.138851:0:20762:0:(service.c:2372:ptlrpc_server_handle_request()) Handled RPC req@0000000092028508 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_013:lustre-MDT0000-mdtlov_UUID+5:3353:x1862456998022208:12345-0@lo:400:kworker/u8:0.0 Request processed in 31us (251us total) trans 0 rc 0/0 00000100:00100000:2.0:1776178103.138857:0:20762:0:(nrs_fifo.c:241:nrs_fifo_req_stop()) NRS stop fifo request from 12345-0@lo, seq: 405 00000100:00100000:2.0:1776178103.138877:0:8250:0:(service.c:2152:ptlrpc_server_handle_req_in()) got req x1862456998021696 00000100:00100000:2.0:1776178103.138901:0:8250:0:(nrs_fifo.c:179:nrs_fifo_req_get()) NRS start fifo request from 12345-0@lo, seq: 752 00000100:00100000:2.0:1776178103.138905:0:8250:0:(service.c:2324:ptlrpc_server_handle_request()) Handling RPC req@00000000e837a7cd pname:cluuid+ref:pid:xid:nid:opc:job mdt00_003:lustre-MDT0001-lwp-OST0000_UUID+5:3353:x1862456998021696:12345-0@lo:400:kworker/u8:0.0 00000100:00100000:2.0:1776178103.138930:0:8250:0:(service.c:2372:ptlrpc_server_handle_request()) Handled RPC req@00000000e837a7cd pname:cluuid+ref:pid:xid:nid:opc:job mdt00_003:lustre-MDT0001-lwp-OST0000_UUID+5:3353:x1862456998021696:12345-0@lo:400:kworker/u8:0.0 Request processed in 24us (422us total) trans 0 rc 0/0 00000100:00100000:2.0:1776178103.138935:0:8250:0:(nrs_fifo.c:241:nrs_fifo_req_stop()) NRS stop fifo request from 12345-0@lo, seq: 752 00000100:00100000:2.0:1776178103.138962:0:3354:0:(client.c:1733:ptlrpc_send_new_req()) Sending RPC req@000000005967fd64 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_01:lustre-MDT0000-mdtlov_UUID:3354:1862456998022464:0@lo:13:osp-pre-1-0.0 00000100:00100000:2.0:1776178103.138988:0:3354:0:(events.c:361:request_in_callback()) peer: 12345-0@lo (source: 12345-0@lo) 00000100:00100000:2.0:1776178103.139013:0:8231:0:(service.c:2152:ptlrpc_server_handle_req_in()) got req x1862456998022464 00000100:00100000:2.0:1776178103.139033:0:8231:0:(nrs_fifo.c:179:nrs_fifo_req_get()) NRS start fifo request from 12345-0@lo, seq: 400 00000100:00100000:2.0:1776178103.139036:0:8231:0:(service.c:2324:ptlrpc_server_handle_request()) Handling RPC req@0000000003cebab2 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_create00:lustre-MDT0000-mdtlov_UUID+5:3354:x1862456998022464:12345-0@lo:13:osp-pre-1-0.0 00002000:00000024:2.0:1776178103.139056:0:8231:0:(ofd_obd.c:740:ofd_statfs()) blocks cached 229376 granted 491746368 pending 0 free 3923689472 avail 3692167168 00000100:00100000:2.0:1776178103.139114:0:8231:0:(service.c:2372:ptlrpc_server_handle_request()) Handled RPC req@0000000003cebab2 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_create00:lustre-MDT0000-mdtlov_UUID+5:3354:x1862456998022464:12345-0@lo:13:osp-pre-1-0.0 Request processed in 77us (126us total) trans 0 rc 0/0 00000100:00100000:2.0:1776178103.139120:0:8231:0:(nrs_fifo.c:241:nrs_fifo_req_stop()) NRS stop fifo request from 12345-0@lo, seq: 400 00000100:00100000:2.0:1776178103.139147:0:3354:0:(client.c:2215:ptlrpc_check_set()) Completed RPC req@000000005967fd64 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_01:lustre-MDT0000-mdtlov_UUID:3354:1862456998022464:0@lo:13:osp-pre-1-0.0 00000100:00100000:2.0:1776178103.139175:0:3353:0:(client.c:2215:ptlrpc_check_set()) Completed RPC req@00000000478fd8d5 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_00:lustre-MDT0001-lwp-OST0000_UUID:3353:1862456998021696:0@lo:400:kworker/u8:0.0 00000100:00100000:2.0:1776178103.139184:0:3353:0:(client.c:2215:ptlrpc_check_set()) Completed RPC req@000000001d51eec7 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_00:lustre-MDT0001-mdtlov_UUID:3353:1862456998021952:0@lo:400:kworker/u8:0.0 00000100:00100000:2.0:1776178103.139192:0:3353:0:(client.c:2215:ptlrpc_check_set()) Completed RPC req@00000000956deaca pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_00:lustre-MDT0000-mdtlov_UUID:3353:1862456998022208:0@lo:400:kworker/u8:0.0 00000100:00100000:2.0:1776178103.139242:0:3356:0:(client.c:2215:ptlrpc_check_set()) Completed RPC req@0000000023210d1c pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_03:ac61f839-0e8e-4cac-9c59-08bcfbbad330:3356:1862456998021568:0@lo:400:kworker/u8:0.0 00000100:00100000:0.0:1776178103.139466:0:14128:0:(service.c:2324:ptlrpc_server_handle_request()) Handling RPC req@000000004e8fbd9b pname:cluuid+ref:pid:xid:nid:opc:job mdt_out00_002:lustre-MDT0000-mdtlov_UUID+5:3353:x1862456998022400:12345-0@lo:41:osp-pre-1-0.0 00000004:00000024:0.0:1776178103.139488:0:14128:0:(mdt_handler.c:548:mdt_statfs()) blocks cached 0 granted 2097152 pending 0 free 1446039552 avail 1316577280 00000100:00100000:0.0:1776178103.139519:0:14128:0:(service.c:2372:ptlrpc_server_handle_request()) Handled RPC req@000000004e8fbd9b pname:cluuid+ref:pid:xid:nid:opc:job mdt_out00_002:lustre-MDT0000-mdtlov_UUID+5:3353:x1862456998022400:12345-0@lo:41:osp-pre-1-0.0 Request processed in 934us (1062us total) trans 0 rc 0/0 00000100:00100000:0.0:1776178103.139524:0:14128:0:(nrs_fifo.c:241:nrs_fifo_req_stop()) NRS stop fifo request from 12345-0@lo, seq: 617 00000100:00100000:1.0:1776178103.140005:0:3353:0:(client.c:2215:ptlrpc_check_set()) Completed RPC req@000000007fcab602 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_00:lustre-MDT0000-mdtlov_UUID:3353:1862456998022400:0@lo:41:osp-pre-1-0.0 00010000:00010000:3.0:1776178103.721871:0:9225:0:(ldlm_lockd.c:2792:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:3.0:1776178104.103018:0:3346:0:(events.c:361:request_in_callback()) peer: 12345-192.168.204.3@tcp (source: 12345-192.168.204.3@tcp) 00000100:00100000:3.0:1776178104.103120:0:3346:0:(events.c:361:request_in_callback()) peer: 12345-192.168.204.3@tcp (source: 12345-192.168.204.3@tcp) 00000100:00100000:3.0:1776178104.103152:0:3346:0:(events.c:361:request_in_callback()) peer: 12345-192.168.204.3@tcp (source: 12345-192.168.204.3@tcp) 00000100:00100000:3.0:1776178104.103188:0:3346:0:(events.c:361:request_in_callback()) peer: 12345-192.168.204.3@tcp (source: 12345-192.168.204.3@tcp) 00000100:00100000:3.0:1776178104.103212:0:3346:0:(events.c:361:request_in_callback()) peer: 12345-192.168.204.3@tcp (source: 12345-192.168.204.3@tcp) 00000100:00100000:1.0:1776178104.103326:0:20137:0:(service.c:2152:ptlrpc_server_handle_req_in()) got req x1862456979081856 00000100:00100000:2.0:1776178104.103326:0:6256:0:(service.c:2152:ptlrpc_server_handle_req_in()) got req x1862456979081920 00000100:00100000:2.0:1776178104.103400:0:6256:0:(service.c:2152:ptlrpc_server_handle_req_in()) got req x1862456979081984 00000100:00100000:1.0:1776178104.103406:0:20137:0:(nrs_fifo.c:179:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.204.3@tcp, seq: 116 00000100:00100000:1.0:1776178104.103417:0:20137:0:(service.c:2324:ptlrpc_server_handle_request()) Handling RPC req@000000008f0263f6 pname:cluuid+ref:pid:xid:nid:opc:job ll_mgs_0000:5b5c66b1-1a2f-4651-8548-0c75b9a38a94+9:2270:x1862456979081856:12345-192.168.204.3@tcp:400:kworker/u8:2.0 00000100:00100000:2.0:1776178104.103426:0:6256:0:(nrs_fifo.c:179:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.204.3@tcp, seq: 753 00000100:00100000:2.0:1776178104.103433:0:6256:0:(service.c:2324:ptlrpc_server_handle_request()) Handling RPC req@00000000a2dca46e pname:cluuid+ref:pid:xid:nid:opc:job mdt00_001:b50963bd-8eba-4432-9a54-8888d00e3fa1+21:2270:x1862456979081920:12345-192.168.204.3@tcp:400:kworker/u8:2.0 00000100:00100000:3.0:1776178104.103551:0:20764:0:(service.c:2152:ptlrpc_server_handle_req_in()) got req x1862456979082048 00000100:00100000:3.0:1776178104.103580:0:20764:0:(service.c:2152:ptlrpc_server_handle_req_in()) got req x1862456979082112 00000100:00100000:3.0:1776178104.103599:0:20764:0:(nrs_fifo.c:179:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.204.3@tcp, seq: 406 00000100:00100000:3.0:1776178104.103603:0:20764:0:(service.c:2324:ptlrpc_server_handle_request()) Handling RPC req@0000000095c019b3 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_015:b50963bd-8eba-4432-9a54-8888d00e3fa1+9:2270:x1862456979082048:12345-192.168.204.3@tcp:400:kworker/u8:2.0 00000100:00100000:1.0:1776178104.103684:0:20137:0:(service.c:2372:ptlrpc_server_handle_request()) Handled RPC req@000000008f0263f6 pname:cluuid+ref:pid:xid:nid:opc:job ll_mgs_0000:5b5c66b1-1a2f-4651-8548-0c75b9a38a94+9:2270:x1862456979081856:12345-192.168.204.3@tcp:400:kworker/u8:2.0 Request processed in 268us (530us total) trans 0 rc 0/0 00000100:00100000:1.0:1776178104.103692:0:20137:0:(nrs_fifo.c:241:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.204.3@tcp, seq: 116 00000100:00100000:0.0:1776178104.103894:0:20769:0:(nrs_fifo.c:179:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.204.3@tcp, seq: 407 00000100:00100000:0.0:1776178104.103900:0:20769:0:(service.c:2324:ptlrpc_server_handle_request()) Handling RPC req@000000008c469ec0 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_018:b50963bd-8eba-4432-9a54-8888d00e3fa1+8:2270:x1862456979082112:12345-192.168.204.3@tcp:400:kworker/u8:2.0 00000100:00100000:2.0:1776178104.105757:0:6256:0:(service.c:2372:ptlrpc_server_handle_request()) Handled RPC req@00000000a2dca46e pname:cluuid+ref:pid:xid:nid:opc:job mdt00_001:b50963bd-8eba-4432-9a54-8888d00e3fa1+21:2270:x1862456979081920:12345-192.168.204.3@tcp:400:kworker/u8:2.0 Request processed in 2309us (2732us total) trans 0 rc 0/0 00000100:00100000:2.0:1776178104.106036:0:6256:0:(nrs_fifo.c:241:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.204.3@tcp, seq: 753 00000100:00100000:2.0:1776178104.106097:0:6256:0:(nrs_fifo.c:179:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.204.3@tcp, seq: 754 00000100:00100000:2.0:1776178104.106105:0:6256:0:(service.c:2324:ptlrpc_server_handle_request()) Handling RPC req@000000005967fd64 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_001:b50963bd-8eba-4432-9a54-8888d00e3fa1+6:2270:x1862456979081984:12345-192.168.204.3@tcp:400:kworker/u8:2.0 00000100:00100000:2.0:1776178104.106159:0:6256:0:(service.c:2372:ptlrpc_server_handle_request()) Handled RPC req@000000005967fd64 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_001:b50963bd-8eba-4432-9a54-8888d00e3fa1+6:2270:x1862456979081984:12345-192.168.204.3@tcp:400:kworker/u8:2.0 Request processed in 55us (2970us total) trans 0 rc 0/0 00000100:00100000:2.0:1776178104.106165:0:6256:0:(nrs_fifo.c:241:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.204.3@tcp, seq: 754 00000100:00100000:3.0:1776178104.106459:0:20764:0:(service.c:2372:ptlrpc_server_handle_request()) Handled RPC req@0000000095c019b3 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_015:b50963bd-8eba-4432-9a54-8888d00e3fa1+9:2270:x1862456979082048:12345-192.168.204.3@tcp:400:kworker/u8:2.0 Request processed in 2851us (3336us total) trans 0 rc 0/0 00000100:00100000:3.0:1776178104.106469:0:20764:0:(nrs_fifo.c:241:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.204.3@tcp, seq: 406 00000100:00100000:0.0:1776178104.108738:0:20769:0:(service.c:2372:ptlrpc_server_handle_request()) Handled RPC req@000000008c469ec0 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_018:b50963bd-8eba-4432-9a54-8888d00e3fa1+8:2270:x1862456979082112:12345-192.168.204.3@tcp:400:kworker/u8:2.0 Request processed in 4826us (5513us total) trans 0 rc 0/0 00000100:00100000:0.0:1776178104.108754:0:20769:0:(nrs_fifo.c:241:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.204.3@tcp, seq: 407 00000100:00100000:2.0:1776178104.673923:0:3355:0:(client.c:1733:ptlrpc_send_new_req()) Sending RPC req@0000000053e142bc pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_02:lustre-MDT0001-mdtlov_UUID:3355:1862456998022528:0@lo:13:osp-pre-1-1.0 00000100:00100000:2.0:1776178104.674018:0:3355:0:(events.c:361:request_in_callback()) peer: 12345-0@lo (source: 12345-0@lo) 00000100:00100000:3.0:1776178104.675959:0:8232:0:(service.c:2152:ptlrpc_server_handle_req_in()) got req x1862456998022528 00000100:00100000:3.0:1776178104.676046:0:8232:0:(nrs_fifo.c:179:nrs_fifo_req_get()) NRS start fifo request from 12345-0@lo, seq: 401 00000100:00100000:3.0:1776178104.676082:0:8232:0:(service.c:2324:ptlrpc_server_handle_request()) Handling RPC req@00000000efb13deb pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_create00:lustre-MDT0001-mdtlov_UUID+5:3355:x1862456998022528:12345-0@lo:13:osp-pre-1-1.0 00002000:00000024:3.0:1776178104.676138:0:8232:0:(ofd_obd.c:740:ofd_statfs()) blocks cached 229376 granted 491746368 pending 0 free 3923689472 avail 3692167168 00000100:00100000:3.0:1776178104.676266:0:8232:0:(service.c:2372:ptlrpc_server_handle_request()) Handled RPC req@00000000efb13deb pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_create00:lustre-MDT0001-mdtlov_UUID+5:3355:x1862456998022528:12345-0@lo:13:osp-pre-1-1.0 Request processed in 211us (2249us total) trans 0 rc 0/0 00000100:00100000:3.0:1776178104.676276:0:8232:0:(nrs_fifo.c:241:nrs_fifo_req_stop()) NRS stop fifo request from 12345-0@lo, seq: 401 00000100:00100000:2.0:1776178104.676313:0:3355:0:(client.c:2215:ptlrpc_check_set()) Completed RPC req@0000000053e142bc pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_02:lustre-MDT0001-mdtlov_UUID:3355:1862456998022528:0@lo:13:osp-pre-1-1.0 00010000:00010000:2.0:1776178104.736902:0:9224:0:(ldlm_lockd.c:2792:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:0.0:1776178105.184693:0:3355:0:(ptlrpcd.c:415:ptlrpcd_check()) transfer 1 async RPCs [3->2] 00000100:00100000:0.0:1776178105.184718:0:3355:0:(client.c:1733:ptlrpc_send_new_req()) Sending RPC req@00000000bc95ec51 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_02:lustre-MDT0001-mdtlov_UUID:3355:1862456998022592:0@lo:13:osp-pre-0-1.0 00000100:00100000:0.0:1776178105.184787:0:3355:0:(events.c:361:request_in_callback()) peer: 12345-0@lo (source: 12345-0@lo) 00000100:00100000:3.0:1776178105.184985:0:8231:0:(service.c:2152:ptlrpc_server_handle_req_in()) got req x1862456998022592 00000100:00100000:3.0:1776178105.185187:0:8231:0:(nrs_fifo.c:179:nrs_fifo_req_get()) NRS start fifo request from 12345-0@lo, seq: 402 00000100:00100000:3.0:1776178105.185193:0:8231:0:(service.c:2324:ptlrpc_server_handle_request()) Handling RPC req@0000000007572592 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_create00:lustre-MDT0001-mdtlov_UUID+5:3355:x1862456998022592:12345-0@lo:13:osp-pre-0-1.0 00002000:00000024:3.0:1776178105.185498:0:8231:0:(ofd_obd.c:740:ofd_statfs()) blocks cached 315392 granted 484883776 pending 0 free 3923689472 avail 3692167168 00000100:00100000:3.0:1776178105.185723:0:8231:0:(service.c:2372:ptlrpc_server_handle_request()) Handled RPC req@0000000007572592 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_create00:lustre-MDT0001-mdtlov_UUID+5:3355:x1862456998022592:12345-0@lo:13:osp-pre-0-1.0 Request processed in 528us (936us total) trans 0 rc 0/0 00000100:00100000:3.0:1776178105.185732:0:8231:0:(nrs_fifo.c:241:nrs_fifo_req_stop()) NRS stop fifo request from 12345-0@lo, seq: 402 00000100:00100000:0.0:1776178105.191307:0:3355:0:(client.c:2215:ptlrpc_check_set()) Completed RPC req@00000000bc95ec51 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_02:lustre-MDT0001-mdtlov_UUID:3355:1862456998022592:0@lo:13:osp-pre-0-1.0 00010000:00010000:3.0:1776178105.760405:0:6244:0:(ldlm_lockd.c:2792:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00010000:00010000:0.0:1776178106.784416:0:6245:0:(ldlm_lockd.c:2792:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00010000:00010000:3.0:1776178107.813469:0:9225:0:(ldlm_lockd.c:2792:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:0.0:1776178108.256784:0:3354:0:(ptlrpcd.c:415:ptlrpcd_check()) transfer 1 async RPCs [0->1] 00000100:00100000:0.0:1776178108.256811:0:3354:0:(client.c:1733:ptlrpc_send_new_req()) Sending RPC req@000000002b69594e pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_01:lustre-MDT0001-mdtlov_UUID:3354:1862456998022656:0@lo:41:osp-pre-0-1.0 00000100:00100000:0.0:1776178108.256908:0:3354:0:(events.c:361:request_in_callback()) peer: 12345-0@lo (source: 12345-0@lo) 00000100:00100000:2.0:1776178108.257177:0:14128:0:(service.c:2152:ptlrpc_server_handle_req_in()) got req x1862456998022656 00000100:00100000:2.0:1776178108.257239:0:14128:0:(nrs_fifo.c:179:nrs_fifo_req_get()) NRS start fifo request from 12345-0@lo, seq: 618 00000100:00100000:2.0:1776178108.257245:0:14128:0:(service.c:2324:ptlrpc_server_handle_request()) Handling RPC req@00000000d668bb9c pname:cluuid+ref:pid:xid:nid:opc:job mdt_out00_002:lustre-MDT0001-mdtlov_UUID+6:3354:x1862456998022656:12345-0@lo:41:osp-pre-0-1.0 00000004:00000024:2.0:1776178108.257286:0:14128:0:(mdt_handler.c:548:mdt_statfs()) blocks cached 0 granted 2097152 pending 0 free 1446121472 avail 1316659200 00000100:00100000:2.0:1776178108.257338:0:14128:0:(service.c:2372:ptlrpc_server_handle_request()) Handled RPC req@00000000d668bb9c pname:cluuid+ref:pid:xid:nid:opc:job mdt_out00_002:lustre-MDT0001-mdtlov_UUID+6:3354:x1862456998022656:12345-0@lo:41:osp-pre-0-1.0 Request processed in 92us (433us total) trans 0 rc 0/0 00000100:00100000:2.0:1776178108.257345:0:14128:0:(nrs_fifo.c:241:nrs_fifo_req_stop()) NRS stop fifo request from 12345-0@lo, seq: 618 00000100:00100000:0.0:1776178108.257686:0:3354:0:(client.c:2215:ptlrpc_check_set()) Completed RPC req@000000002b69594e pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_01:lustre-MDT0001-mdtlov_UUID:3354:1862456998022656:0@lo:41:osp-pre-0-1.0 00000100:00100000:0.0:1776178108.257895:0:3354:0:(client.c:1733:ptlrpc_send_new_req()) Sending RPC req@000000001554c6b4 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_01:lustre-MDT0000-mdtlov_UUID:3354:1862456998022720:0@lo:13:osp-pre-1-0.0 00000100:00100000:0.0:1776178108.257930:0:3354:0:(events.c:361:request_in_callback()) peer: 12345-0@lo (source: 12345-0@lo) 00000100:00100000:2.0:1776178108.258030:0:8231:0:(service.c:2152:ptlrpc_server_handle_req_in()) got req x1862456998022720 00000100:00100000:2.0:1776178108.258086:0:8231:0:(nrs_fifo.c:179:nrs_fifo_req_get()) NRS start fifo request from 12345-0@lo, seq: 403 00000100:00100000:2.0:1776178108.258090:0:8231:0:(service.c:2324:ptlrpc_server_handle_request()) Handling RPC req@000000008c593d89 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_create00:lustre-MDT0000-mdtlov_UUID+5:3354:x1862456998022720:12345-0@lo:13:osp-pre-1-0.0 00002000:00000024:2.0:1776178108.258114:0:8231:0:(ofd_obd.c:740:ofd_statfs()) blocks cached 229376 granted 491746368 pending 0 free 3923689472 avail 3692167168 00000100:00100000:2.0:1776178108.258588:0:8231:0:(service.c:2372:ptlrpc_server_handle_request()) Handled RPC req@000000008c593d89 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_create00:lustre-MDT0000-mdtlov_UUID+5:3354:x1862456998022720:12345-0@lo:13:osp-pre-1-0.0 Request processed in 495us (655us total) trans 0 rc 0/0 00000100:00100000:2.0:1776178108.258596:0:8231:0:(nrs_fifo.c:241:nrs_fifo_req_stop()) NRS stop fifo request from 12345-0@lo, seq: 403 00000100:00100000:0.0:1776178108.258662:0:3354:0:(client.c:2215:ptlrpc_check_set()) Completed RPC req@000000001554c6b4 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_01:lustre-MDT0000-mdtlov_UUID:3354:1862456998022720:0@lo:13:osp-pre-1-0.0 00000100:00100000:3.0:1776178108.261076:0:3355:0:(client.c:1733:ptlrpc_send_new_req()) Sending RPC req@0000000078c1d864 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_02:lustre-MDT0000-mdtlov_UUID:3355:1862456998022784:0@lo:13:osp-pre-0-0.0 00000100:00100000:3.0:1776178108.261426:0:3355:0:(events.c:361:request_in_callback()) peer: 12345-0@lo (source: 12345-0@lo) 00000100:00100000:3.0:1776178108.261509:0:8231:0:(service.c:2152:ptlrpc_server_handle_req_in()) got req x1862456998022784 00000100:00100000:3.0:1776178108.261571:0:8231:0:(nrs_fifo.c:179:nrs_fifo_req_get()) NRS start fifo request from 12345-0@lo, seq: 404 00000100:00100000:3.0:1776178108.261578:0:8231:0:(service.c:2324:ptlrpc_server_handle_request()) Handling RPC req@000000005accf721 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_create00:lustre-MDT0000-mdtlov_UUID+5:3355:x1862456998022784:12345-0@lo:13:osp-pre-0-0.0 00002000:00000024:3.0:1776178108.261618:0:8231:0:(ofd_obd.c:740:ofd_statfs()) blocks cached 315392 granted 484883776 pending 0 free 3923689472 avail 3692167168 00000100:00100000:3.0:1776178108.261730:0:8231:0:(service.c:2372:ptlrpc_server_handle_request()) Handled RPC req@000000005accf721 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_create00:lustre-MDT0000-mdtlov_UUID+5:3355:x1862456998022784:12345-0@lo:13:osp-pre-0-0.0 Request processed in 151us (306us total) trans 0 rc 0/0 00000100:00100000:3.0:1776178108.261738:0:8231:0:(nrs_fifo.c:241:nrs_fifo_req_stop()) NRS stop fifo request from 12345-0@lo, seq: 404 00000100:00100000:3.0:1776178108.261801:0:3355:0:(client.c:2215:ptlrpc_check_set()) Completed RPC req@0000000078c1d864 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_02:lustre-MDT0000-mdtlov_UUID:3355:1862456998022784:0@lo:13:osp-pre-0-0.0 00000100:00100000:3.0:1776178108.262642:0:3356:0:(client.c:1733:ptlrpc_send_new_req()) Sending RPC req@00000000e0d60cbc pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_03:lustre-MDT0000-mdtlov_UUID:3356:1862456998022848:0@lo:41:osp-pre-1-0.0 00000100:00100000:3.0:1776178108.263845:0:3356:0:(events.c:361:request_in_callback()) peer: 12345-0@lo (source: 12345-0@lo) 00000100:00100000:0.0:1776178108.263977:0:15213:0:(service.c:2152:ptlrpc_server_handle_req_in()) got req x1862456998022848 00000100:00100000:0.0:1776178108.265331:0:15213:0:(nrs_fifo.c:179:nrs_fifo_req_get()) NRS start fifo request from 12345-0@lo, seq: 619 00000100:00100000:0.0:1776178108.265345:0:15213:0:(service.c:2324:ptlrpc_server_handle_request()) Handling RPC req@0000000078c1d864 pname:cluuid+ref:pid:xid:nid:opc:job mdt_out00_003:lustre-MDT0000-mdtlov_UUID+5:3356:x1862456998022848:12345-0@lo:41:osp-pre-1-0.0 00000004:00000024:0.0:1776178108.265401:0:15213:0:(mdt_handler.c:548:mdt_statfs()) blocks cached 0 granted 2097152 pending 0 free 1446039552 avail 1316577280 00000100:00100000:0.0:1776178108.265507:0:15213:0:(service.c:2372:ptlrpc_server_handle_request()) Handled RPC req@0000000078c1d864 pname:cluuid+ref:pid:xid:nid:opc:job mdt_out00_003:lustre-MDT0000-mdtlov_UUID+5:3356:x1862456998022848:12345-0@lo:41:osp-pre-1-0.0 Request processed in 163us (1667us total) trans 0 rc 0/0 00000100:00100000:0.0:1776178108.265514:0:15213:0:(nrs_fifo.c:241:nrs_fifo_req_stop()) NRS stop fifo request from 12345-0@lo, seq: 619 00000100:00100000:3.0:1776178108.265666:0:3356:0:(client.c:2215:ptlrpc_check_set()) Completed RPC req@00000000e0d60cbc pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_03:lustre-MDT0000-mdtlov_UUID:3356:1862456998022848:0@lo:41:osp-pre-1-0.0 00000100:00100000:2.0:1776178108.266867:0:3353:0:(client.c:1733:ptlrpc_send_new_req()) Sending RPC req@000000007886b426 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_00:ac61f839-0e8e-4cac-9c59-08bcfbbad330:3353:1862456998022912:0@lo:400:kworker/u8:0.0 00000100:00100000:2.0:1776178108.266942:0:3353:0:(events.c:361:request_in_callback()) peer: 12345-0@lo (source: 12345-0@lo) 00000100:00100000:2.0:1776178108.266977:0:3353:0:(client.c:1733:ptlrpc_send_new_req()) Sending RPC req@00000000fa8483d9 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_00:lustre-MDT0001-lwp-OST0001_UUID:3353:1862456998023168:0@lo:400:kworker/u8:0.0 00000100:00100000:2.0:1776178108.267008:0:3353:0:(events.c:361:request_in_callback()) peer: 12345-0@lo (source: 12345-0@lo) 00000100:00100000:2.0:1776178108.267033:0:3353:0:(client.c:1733:ptlrpc_send_new_req()) Sending RPC req@000000009c58e3fe pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_00:lustre-MDT0000-lwp-MDT0000_UUID:3353:1862456998023424:0@lo:400:kworker/u8:0.0 00000100:00100000:2.0:1776178108.267078:0:3353:0:(events.c:361:request_in_callback()) peer: 12345-0@lo (source: 12345-0@lo) 00000100:00100000:2.0:1776178108.267111:0:3353:0:(ptlrpcd.c:415:ptlrpcd_check()) transfer 2 async RPCs [1->0] 00000100:00100000:2.0:1776178108.267120:0:3353:0:(client.c:1733:ptlrpc_send_new_req()) Sending RPC req@000000002b69594e pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_00:lustre-MDT0000-lwp-OST0000_UUID:3353:1862456998022976:0@lo:400:kworker/u8:0.0 00000100:00100000:2.0:1776178108.267150:0:3353:0:(events.c:361:request_in_callback()) peer: 12345-0@lo (source: 12345-0@lo) 00000100:00100000:2.0:1776178108.267173:0:3353:0:(client.c:1733:ptlrpc_send_new_req()) Sending RPC req@00000000232b3a5e pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_00:lustre-MDT0001-mdtlov_UUID:3353:1862456998023232:0@lo:400:kworker/u8:0.0 00000100:00100000:2.0:1776178108.267204:0:3353:0:(events.c:361:request_in_callback()) peer: 12345-0@lo (source: 12345-0@lo) 00000100:00100000:2.0:1776178108.267256:0:8250:0:(service.c:2152:ptlrpc_server_handle_req_in()) got req x1862456998023168 00000100:00100000:2.0:1776178108.267318:0:8250:0:(service.c:2152:ptlrpc_server_handle_req_in()) got req x1862456998023424 00000100:00100000:2.0:1776178108.267345:0:8250:0:(service.c:2152:ptlrpc_server_handle_req_in()) got req x1862456998022976 00000100:00100000:2.0:1776178108.267372:0:8250:0:(nrs_fifo.c:179:nrs_fifo_req_get()) NRS start fifo request from 12345-0@lo, seq: 755 00000100:00100000:2.0:1776178108.267380:0:8250:0:(service.c:2324:ptlrpc_server_handle_request()) Handling RPC req@0000000084754621 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_003:lustre-MDT0001-lwp-OST0001_UUID+5:3353:x1862456998023168:12345-0@lo:400:kworker/u8:0.0 00000100:00100000:2.0:1776178108.267480:0:8250:0:(service.c:2372:ptlrpc_server_handle_request()) Handled RPC req@0000000084754621 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_003:lustre-MDT0001-lwp-OST0001_UUID+5:3353:x1862456998023168:12345-0@lo:400:kworker/u8:0.0 Request processed in 99us (472us total) trans 0 rc 0/0 00000100:00100000:2.0:1776178108.267490:0:8250:0:(nrs_fifo.c:241:nrs_fifo_req_stop()) NRS stop fifo request from 12345-0@lo, seq: 755 00000100:00100000:2.0:1776178108.267516:0:8250:0:(nrs_fifo.c:179:nrs_fifo_req_get()) NRS start fifo request from 12345-0@lo, seq: 756 00000100:00100000:2.0:1776178108.267522:0:8250:0:(service.c:2324:ptlrpc_server_handle_request()) Handling RPC req@00000000e8f6b843 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_003:lustre-MDT0000-lwp-MDT0000_UUID+5:3353:x1862456998023424:12345-0@lo:400:kworker/u8:0.0 00000100:00100000:2.0:1776178108.267568:0:8250:0:(service.c:2372:ptlrpc_server_handle_request()) Handled RPC req@00000000e8f6b843 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_003:lustre-MDT0000-lwp-MDT0000_UUID+5:3353:x1862456998023424:12345-0@lo:400:kworker/u8:0.0 Request processed in 46us (489us total) trans 0 rc 0/0 00000100:00100000:2.0:1776178108.267576:0:8250:0:(nrs_fifo.c:241:nrs_fifo_req_stop()) NRS stop fifo request from 12345-0@lo, seq: 756 00000100:00100000:2.0:1776178108.267597:0:8250:0:(nrs_fifo.c:179:nrs_fifo_req_get()) NRS start fifo request from 12345-0@lo, seq: 757 00000100:00100000:2.0:1776178108.267602:0:8250:0:(service.c:2324:ptlrpc_server_handle_request()) Handling RPC req@000000009cb972eb pname:cluuid+ref:pid:xid:nid:opc:job mdt00_003:lustre-MDT0000-lwp-OST0000_UUID+5:3353:x1862456998022976:12345-0@lo:400:kworker/u8:0.0 00000100:00100000:2.0:1776178108.267637:0:8250:0:(service.c:2372:ptlrpc_server_handle_request()) Handled RPC req@000000009cb972eb pname:cluuid+ref:pid:xid:nid:opc:job mdt00_003:lustre-MDT0000-lwp-OST0000_UUID+5:3353:x1862456998022976:12345-0@lo:400:kworker/u8:0.0 Request processed in 34us (488us total) trans 0 rc 0/0 00000100:00100000:2.0:1776178108.267644:0:8250:0:(nrs_fifo.c:241:nrs_fifo_req_stop()) NRS stop fifo request from 12345-0@lo, seq: 757 00000100:00100000:2.0:1776178108.267678:0:20769:0:(service.c:2152:ptlrpc_server_handle_req_in()) got req x1862456998023232 00000100:00100000:2.0:1776178108.267713:0:20769:0:(nrs_fifo.c:179:nrs_fifo_req_get()) NRS start fifo request from 12345-0@lo, seq: 408 00000100:00100000:2.0:1776178108.267716:0:20769:0:(service.c:2324:ptlrpc_server_handle_request()) Handling RPC req@00000000a2512e82 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_018:lustre-MDT0001-mdtlov_UUID+5:3353:x1862456998023232:12345-0@lo:400:kworker/u8:0.0 00000100:00100000:2.0:1776178108.267748:0:20769:0:(service.c:2372:ptlrpc_server_handle_request()) Handled RPC req@00000000a2512e82 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_018:lustre-MDT0001-mdtlov_UUID+5:3353:x1862456998023232:12345-0@lo:400:kworker/u8:0.0 Request processed in 31us (547us total) trans 0 rc 0/0 00000100:00100000:2.0:1776178108.267754:0:20769:0:(nrs_fifo.c:241:nrs_fifo_req_stop()) NRS stop fifo request from 12345-0@lo, seq: 408 00000100:00100000:2.0:1776178108.267802:0:3353:0:(client.c:2215:ptlrpc_check_set()) Completed RPC req@000000002b69594e pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_00:lustre-MDT0000-lwp-OST0000_UUID:3353:1862456998022976:0@lo:400:kworker/u8:0.0 00000100:00100000:2.0:1776178108.267812:0:3353:0:(client.c:2215:ptlrpc_check_set()) Completed RPC req@00000000232b3a5e pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_00:lustre-MDT0001-mdtlov_UUID:3353:1862456998023232:0@lo:400:kworker/u8:0.0 00000100:00100000:2.0:1776178108.267820:0:3353:0:(client.c:2215:ptlrpc_check_set()) Completed RPC req@00000000fa8483d9 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_00:lustre-MDT0001-lwp-OST0001_UUID:3353:1862456998023168:0@lo:400:kworker/u8:0.0 00000100:00100000:2.0:1776178108.267828:0:3353:0:(client.c:2215:ptlrpc_check_set()) Completed RPC req@000000009c58e3fe pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_00:lustre-MDT0000-lwp-MDT0000_UUID:3353:1862456998023424:0@lo:400:kworker/u8:0.0 00000100:00100000:2.0:1776178108.267862:0:3356:0:(client.c:1733:ptlrpc_send_new_req()) Sending RPC req@000000005d95861f pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_03:lustre-MDT0000-lwp-OST0001_UUID:3356:1862456998023104:0@lo:400:kworker/u8:0.0 00000100:00100000:2.0:1776178108.267887:0:3356:0:(events.c:361:request_in_callback()) peer: 12345-0@lo (source: 12345-0@lo) 00000100:00100000:2.0:1776178108.267901:0:3356:0:(client.c:1733:ptlrpc_send_new_req()) Sending RPC req@000000004cbcc47c pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_03:lustre-MDT0000-lwp-MDT0001_UUID:3356:1862456998023360:0@lo:400:kworker/u8:0.0 00000100:00100000:2.0:1776178108.267919:0:3356:0:(events.c:361:request_in_callback()) peer: 12345-0@lo (source: 12345-0@lo) 00000100:00100000:2.0:1776178108.267932:0:3356:0:(ptlrpcd.c:415:ptlrpcd_check()) transfer 2 async RPCs [2->3] 00000100:00100000:2.0:1776178108.267938:0:3356:0:(client.c:1733:ptlrpc_send_new_req()) Sending RPC req@00000000bc95ec51 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_03:lustre-MDT0001-lwp-OST0000_UUID:3356:1862456998023040:0@lo:400:kworker/u8:0.0 00000100:00100000:2.0:1776178108.267956:0:3356:0:(events.c:361:request_in_callback()) peer: 12345-0@lo (source: 12345-0@lo) 00000100:00100000:2.0:1776178108.267968:0:3356:0:(client.c:1733:ptlrpc_send_new_req()) Sending RPC req@00000000ba5faff8 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_03:lustre-MDT0001-mdtlov_UUID:3356:1862456998023296:0@lo:400:kworker/u8:0.0 00000100:00100000:2.0:1776178108.267985:0:3356:0:(events.c:361:request_in_callback()) peer: 12345-0@lo (source: 12345-0@lo) 00000100:00100000:2.0:1776178108.268010:0:6257:0:(service.c:2152:ptlrpc_server_handle_req_in()) got req x1862456998023104 00000100:00100000:1.0:1776178108.268032:0:20137:0:(service.c:2152:ptlrpc_server_handle_req_in()) got req x1862456998022912 00000100:00100000:2.0:1776178108.268037:0:6257:0:(service.c:2152:ptlrpc_server_handle_req_in()) got req x1862456998023360 00000100:00100000:2.0:1776178108.268081:0:6257:0:(service.c:2152:ptlrpc_server_handle_req_in()) got req x1862456998023040 00000100:00100000:1.0:1776178108.268090:0:20137:0:(nrs_fifo.c:179:nrs_fifo_req_get()) NRS start fifo request from 12345-0@lo, seq: 117 00000100:00100000:1.0:1776178108.268096:0:20137:0:(service.c:2324:ptlrpc_server_handle_request()) Handling RPC req@000000000084a3e4 pname:cluuid+ref:pid:xid:nid:opc:job ll_mgs_0000:ac61f839-0e8e-4cac-9c59-08bcfbbad330+23:3353:x1862456998022912:12345-0@lo:400:kworker/u8:0.0 00000100:00100000:2.0:1776178108.268102:0:6257:0:(nrs_fifo.c:179:nrs_fifo_req_get()) NRS start fifo request from 12345-0@lo, seq: 758 00000100:00100000:2.0:1776178108.268107:0:6257:0:(service.c:2324:ptlrpc_server_handle_request()) Handling RPC req@000000009c58e3fe pname:cluuid+ref:pid:xid:nid:opc:job mdt00_002:lustre-MDT0000-lwp-OST0001_UUID+5:3356:x1862456998023104:12345-0@lo:400:kworker/u8:0.0 00000100:00100000:1.0:1776178108.268146:0:20137:0:(service.c:2372:ptlrpc_server_handle_request()) Handled RPC req@000000000084a3e4 pname:cluuid+ref:pid:xid:nid:opc:job ll_mgs_0000:ac61f839-0e8e-4cac-9c59-08bcfbbad330+23:3353:x1862456998022912:12345-0@lo:400:kworker/u8:0.0 Request processed in 49us (1207us total) trans 0 rc 0/0 00000100:00100000:1.0:1776178108.268154:0:20137:0:(nrs_fifo.c:241:nrs_fifo_req_stop()) NRS stop fifo request from 12345-0@lo, seq: 117 00000100:00100000:2.0:1776178108.268161:0:6257:0:(service.c:2372:ptlrpc_server_handle_request()) Handled RPC req@000000009c58e3fe pname:cluuid+ref:pid:xid:nid:opc:job mdt00_002:lustre-MDT0000-lwp-OST0001_UUID+5:3356:x1862456998023104:12345-0@lo:400:kworker/u8:0.0 Request processed in 51us (272us total) trans 0 rc 0/0 00000100:00100000:2.0:1776178108.268168:0:6257:0:(nrs_fifo.c:241:nrs_fifo_req_stop()) NRS stop fifo request from 12345-0@lo, seq: 758 00000100:00100000:2.0:1776178108.268186:0:6257:0:(nrs_fifo.c:179:nrs_fifo_req_get()) NRS start fifo request from 12345-0@lo, seq: 759 00000100:00100000:2.0:1776178108.268191:0:6257:0:(service.c:2324:ptlrpc_server_handle_request()) Handling RPC req@00000000fa8483d9 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_002:lustre-MDT0000-lwp-MDT0001_UUID+5:3356:x1862456998023360:12345-0@lo:400:kworker/u8:0.0 00000100:00100000:1.0:1776178108.268202:0:3353:0:(client.c:2215:ptlrpc_check_set()) Completed RPC req@000000007886b426 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_00:ac61f839-0e8e-4cac-9c59-08bcfbbad330:3353:1862456998022912:0@lo:400:kworker/u8:0.0 00000100:00100000:2.0:1776178108.268223:0:6257:0:(service.c:2372:ptlrpc_server_handle_request()) Handled RPC req@00000000fa8483d9 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_002:lustre-MDT0000-lwp-MDT0001_UUID+5:3356:x1862456998023360:12345-0@lo:400:kworker/u8:0.0 Request processed in 31us (303us total) trans 0 rc 0/0 00000100:00100000:2.0:1776178108.268230:0:6257:0:(nrs_fifo.c:241:nrs_fifo_req_stop()) NRS stop fifo request from 12345-0@lo, seq: 759 00000100:00100000:2.0:1776178108.268243:0:6257:0:(nrs_fifo.c:179:nrs_fifo_req_get()) NRS start fifo request from 12345-0@lo, seq: 760 00000100:00100000:2.0:1776178108.268246:0:6257:0:(service.c:2324:ptlrpc_server_handle_request()) Handling RPC req@00000000232b3a5e pname:cluuid+ref:pid:xid:nid:opc:job mdt00_002:lustre-MDT0001-lwp-OST0000_UUID+5:3356:x1862456998023040:12345-0@lo:400:kworker/u8:0.0 00000100:00100000:2.0:1776178108.268273:0:6257:0:(service.c:2372:ptlrpc_server_handle_request()) Handled RPC req@00000000232b3a5e pname:cluuid+ref:pid:xid:nid:opc:job mdt00_002:lustre-MDT0001-lwp-OST0000_UUID+5:3356:x1862456998023040:12345-0@lo:400:kworker/u8:0.0 Request processed in 26us (316us total) trans 0 rc 0/0 00000100:00100000:2.0:1776178108.268278:0:6257:0:(nrs_fifo.c:241:nrs_fifo_req_stop()) NRS stop fifo request from 12345-0@lo, seq: 760 00000100:00100000:2.0:1776178108.268299:0:20769:0:(service.c:2152:ptlrpc_server_handle_req_in()) got req x1862456998023296 00000100:00100000:2.0:1776178108.268322:0:20769:0:(nrs_fifo.c:179:nrs_fifo_req_get()) NRS start fifo request from 12345-0@lo, seq: 409 00000100:00100000:2.0:1776178108.268325:0:20769:0:(service.c:2324:ptlrpc_server_handle_request()) Handling RPC req@000000002f729f84 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_018:lustre-MDT0001-mdtlov_UUID+5:3356:x1862456998023296:12345-0@lo:400:kworker/u8:0.0 00000100:00100000:2.0:1776178108.268353:0:20769:0:(service.c:2372:ptlrpc_server_handle_request()) Handled RPC req@000000002f729f84 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_018:lustre-MDT0001-mdtlov_UUID+5:3356:x1862456998023296:12345-0@lo:400:kworker/u8:0.0 Request processed in 26us (367us total) trans 0 rc 0/0 00000100:00100000:2.0:1776178108.268358:0:20769:0:(nrs_fifo.c:241:nrs_fifo_req_stop()) NRS stop fifo request from 12345-0@lo, seq: 409 00000100:00100000:2.0:1776178108.268383:0:3356:0:(client.c:2215:ptlrpc_check_set()) Completed RPC req@00000000bc95ec51 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_03:lustre-MDT0001-lwp-OST0000_UUID:3356:1862456998023040:0@lo:400:kworker/u8:0.0 00000100:00100000:2.0:1776178108.268394:0:3356:0:(client.c:2215:ptlrpc_check_set()) Completed RPC req@00000000ba5faff8 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_03:lustre-MDT0001-mdtlov_UUID:3356:1862456998023296:0@lo:400:kworker/u8:0.0 00000100:00100000:2.0:1776178108.268402:0:3356:0:(client.c:2215:ptlrpc_check_set()) Completed RPC req@000000005d95861f pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_03:lustre-MDT0000-lwp-OST0001_UUID:3356:1862456998023104:0@lo:400:kworker/u8:0.0 00000100:00100000:2.0:1776178108.268409:0:3356:0:(client.c:2215:ptlrpc_check_set()) Completed RPC req@000000004cbcc47c pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_03:lustre-MDT0000-lwp-MDT0001_UUID:3356:1862456998023360:0@lo:400:kworker/u8:0.0 00010000:00010000:2.0:1776178108.837798:0:9224:0:(ldlm_lockd.c:2792:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:3.0:1776178109.219608:0:3346:0:(events.c:361:request_in_callback()) peer: 12345-192.168.204.3@tcp (source: 12345-192.168.204.3@tcp) 00000100:00100000:3.0:1776178109.219713:0:3346:0:(events.c:361:request_in_callback()) peer: 12345-192.168.204.3@tcp (source: 12345-192.168.204.3@tcp) 00000100:00100000:3.0:1776178109.219751:0:3346:0:(events.c:361:request_in_callback()) peer: 12345-192.168.204.3@tcp (source: 12345-192.168.204.3@tcp) 00000100:00100000:3.0:1776178109.219796:0:3346:0:(events.c:361:request_in_callback()) peer: 12345-192.168.204.3@tcp (source: 12345-192.168.204.3@tcp) 00000100:00100000:0.0:1776178109.219884:0:20754:0:(service.c:2152:ptlrpc_server_handle_req_in()) got req x1862456979082368 00000100:00100000:0.0:1776178109.219962:0:20754:0:(nrs_fifo.c:179:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.204.3@tcp, seq: 410 00000100:00100000:0.0:1776178109.219971:0:20754:0:(service.c:2324:ptlrpc_server_handle_request()) Handling RPC req@0000000068adf51c pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_005:b50963bd-8eba-4432-9a54-8888d00e3fa1+9:2271:x1862456979082368:12345-192.168.204.3@tcp:400:kworker/u8:3.0 00000100:00100000:0.0:1776178109.220114:0:20754:0:(service.c:2372:ptlrpc_server_handle_request()) Handled RPC req@0000000068adf51c pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_005:b50963bd-8eba-4432-9a54-8888d00e3fa1+9:2271:x1862456979082368:12345-192.168.204.3@tcp:400:kworker/u8:3.0 Request processed in 143us (362us total) trans 0 rc 0/0 00000100:00100000:0.0:1776178109.220123:0:20754:0:(nrs_fifo.c:241:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.204.3@tcp, seq: 410 00000100:00100000:1.0:1776178109.220531:0:20140:0:(service.c:2152:ptlrpc_server_handle_req_in()) got req x1862456979082176 00000100:00100000:1.0:1776178109.220576:0:20140:0:(nrs_fifo.c:179:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.204.3@tcp, seq: 118 00000100:00100000:1.0:1776178109.220581:0:20140:0:(service.c:2324:ptlrpc_server_handle_request()) Handling RPC req@00000000ff2b550f pname:cluuid+ref:pid:xid:nid:opc:job ll_mgs_0002:5b5c66b1-1a2f-4651-8548-0c75b9a38a94+9:2270:x1862456979082176:12345-192.168.204.3@tcp:400:kworker/u8:3.0 00000100:00100000:1.0:1776178109.220613:0:20140:0:(service.c:2372:ptlrpc_server_handle_request()) Handled RPC req@00000000ff2b550f pname:cluuid+ref:pid:xid:nid:opc:job ll_mgs_0002:5b5c66b1-1a2f-4651-8548-0c75b9a38a94+9:2270:x1862456979082176:12345-192.168.204.3@tcp:400:kworker/u8:3.0 Request processed in 32us (902us total) trans 0 rc 0/0 00000100:00100000:1.0:1776178109.220620:0:20140:0:(nrs_fifo.c:241:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.204.3@tcp, seq: 118 00000100:00100000:3.0:1776178109.223544:0:3346:0:(events.c:361:request_in_callback()) peer: 12345-192.168.204.3@tcp (source: 12345-192.168.204.3@tcp) 00000100:00100000:1.0:1776178109.223817:0:20769:0:(service.c:2152:ptlrpc_server_handle_req_in()) got req x1862456979082432 00000100:00100000:1.0:1776178109.223888:0:20769:0:(nrs_fifo.c:179:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.204.3@tcp, seq: 411 00000100:00100000:1.0:1776178109.223896:0:20769:0:(service.c:2324:ptlrpc_server_handle_request()) Handling RPC req@00000000ae838296 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_018:b50963bd-8eba-4432-9a54-8888d00e3fa1+8:2270:x1862456979082432:12345-192.168.204.3@tcp:400:kworker/u8:3.0 00000100:00100000:1.0:1776178109.223991:0:20769:0:(service.c:2372:ptlrpc_server_handle_request()) Handled RPC req@00000000ae838296 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_018:b50963bd-8eba-4432-9a54-8888d00e3fa1+8:2270:x1862456979082432:12345-192.168.204.3@tcp:400:kworker/u8:3.0 Request processed in 95us (467us total) trans 0 rc 0/0 00000100:00100000:1.0:1776178109.223999:0:20769:0:(nrs_fifo.c:241:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.204.3@tcp, seq: 411 00000100:00100000:2.0:1776178109.224407:0:6256:0:(service.c:2152:ptlrpc_server_handle_req_in()) got req x1862456979082240 00000100:00100000:2.0:1776178109.224459:0:6256:0:(service.c:2152:ptlrpc_server_handle_req_in()) got req x1862456979082304 00000100:00100000:2.0:1776178109.224485:0:6256:0:(nrs_fifo.c:179:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.204.3@tcp, seq: 761 00000100:00100000:2.0:1776178109.224490:0:6256:0:(service.c:2324:ptlrpc_server_handle_request()) Handling RPC req@000000006247494d pname:cluuid+ref:pid:xid:nid:opc:job mdt00_001:b50963bd-8eba-4432-9a54-8888d00e3fa1+21:2271:x1862456979082240:12345-192.168.204.3@tcp:400:kworker/u8:3.0 00000100:00100000:2.0:1776178109.224549:0:6256:0:(service.c:2372:ptlrpc_server_handle_request()) Handled RPC req@000000006247494d pname:cluuid+ref:pid:xid:nid:opc:job mdt00_001:b50963bd-8eba-4432-9a54-8888d00e3fa1+21:2271:x1862456979082240:12345-192.168.204.3@tcp:400:kworker/u8:3.0 Request processed in 57us (4961us total) trans 0 rc 0/0 00000100:00100000:2.0:1776178109.224562:0:6256:0:(nrs_fifo.c:241:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.204.3@tcp, seq: 761 00000100:00100000:2.0:1776178109.224578:0:6256:0:(nrs_fifo.c:179:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.204.3@tcp, seq: 762 00000100:00100000:2.0:1776178109.224581:0:6256:0:(service.c:2324:ptlrpc_server_handle_request()) Handling RPC req@0000000015909b2d pname:cluuid+ref:pid:xid:nid:opc:job mdt00_001:b50963bd-8eba-4432-9a54-8888d00e3fa1+6:2270:x1862456979082304:12345-192.168.204.3@tcp:400:kworker/u8:3.0 00000100:00100000:2.0:1776178109.224622:0:6256:0:(service.c:2372:ptlrpc_server_handle_request()) Handled RPC req@0000000015909b2d pname:cluuid+ref:pid:xid:nid:opc:job mdt00_001:b50963bd-8eba-4432-9a54-8888d00e3fa1+6:2270:x1862456979082304:12345-192.168.204.3@tcp:400:kworker/u8:3.0 Request processed in 39us (4826us total) trans 0 rc 0/0 00000100:00100000:2.0:1776178109.224629:0:6256:0:(nrs_fifo.c:241:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.204.3@tcp, seq: 762 00000100:00100000:2.0:1776178109.792364:0:3353:0:(ptlrpcd.c:415:ptlrpcd_check()) transfer 1 async RPCs [1->0] 00000100:00100000:2.0:1776178109.792393:0:3353:0:(client.c:1733:ptlrpc_send_new_req()) Sending RPC req@00000000124cea36 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_00:lustre-MDT0001-mdtlov_UUID:3353:1862456998023488:0@lo:13:osp-pre-1-1.0 00000100:00100000:2.0:1776178109.792478:0:3353:0:(events.c:361:request_in_callback()) peer: 12345-0@lo (source: 12345-0@lo) 00000100:00100000:3.0:1776178109.792576:0:8232:0:(service.c:2152:ptlrpc_server_handle_req_in()) got req x1862456998023488 00000100:00100000:3.0:1776178109.792670:0:8232:0:(nrs_fifo.c:179:nrs_fifo_req_get()) NRS start fifo request from 12345-0@lo, seq: 405 00000100:00100000:3.0:1776178109.792678:0:8232:0:(service.c:2324:ptlrpc_server_handle_request()) Handling RPC req@000000004cbcc47c pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_create00:lustre-MDT0001-mdtlov_UUID+5:3353:x1862456998023488:12345-0@lo:13:osp-pre-1-1.0 00002000:00000024:3.0:1776178109.792716:0:8232:0:(ofd_obd.c:740:ofd_statfs()) blocks cached 229376 granted 491746368 pending 0 free 3923689472 avail 3692167168 00000100:00100000:3.0:1776178109.792776:0:8232:0:(service.c:2372:ptlrpc_server_handle_request()) Handled RPC req@000000004cbcc47c pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_create00:lustre-MDT0001-mdtlov_UUID+5:3353:x1862456998023488:12345-0@lo:13:osp-pre-1-1.0 Request processed in 96us (298us total) trans 0 rc 0/0 00000100:00100000:3.0:1776178109.792783:0:8232:0:(nrs_fifo.c:241:nrs_fifo_req_stop()) NRS stop fifo request from 12345-0@lo, seq: 405 00000100:00100000:2.0:1776178109.792877:0:3353:0:(client.c:2215:ptlrpc_check_set()) Completed RPC req@00000000124cea36 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_00:lustre-MDT0001-mdtlov_UUID:3353:1862456998023488:0@lo:13:osp-pre-1-1.0 00010000:00010000:1.0:1776178109.869763:0:6244:0:(ldlm_lockd.c:2792:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:1.0:1776178110.304461:0:3355:0:(client.c:1733:ptlrpc_send_new_req()) Sending RPC req@000000005d95861f pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_02:lustre-MDT0001-mdtlov_UUID:3355:1862456998023552:0@lo:13:osp-pre-0-1.0 00000100:00100000:1.0:1776178110.304568:0:3355:0:(events.c:361:request_in_callback()) peer: 12345-0@lo (source: 12345-0@lo) 00000100:00100000:2.0:1776178110.308359:0:8232:0:(service.c:2152:ptlrpc_server_handle_req_in()) got req x1862456998023552 00000100:00100000:2.0:1776178110.308467:0:8232:0:(nrs_fifo.c:179:nrs_fifo_req_get()) NRS start fifo request from 12345-0@lo, seq: 406 00000100:00100000:2.0:1776178110.308474:0:8232:0:(service.c:2324:ptlrpc_server_handle_request()) Handling RPC req@000000007fcab602 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_create00:lustre-MDT0001-mdtlov_UUID+5:3355:x1862456998023552:12345-0@lo:13:osp-pre-0-1.0 00002000:00000024:2.0:1776178110.308637:0:8232:0:(ofd_obd.c:740:ofd_statfs()) blocks cached 315392 granted 484883776 pending 0 free 3923689472 avail 3692167168 00000100:00100000:2.0:1776178110.308865:0:8232:0:(service.c:2372:ptlrpc_server_handle_request()) Handled RPC req@000000007fcab602 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_create00:lustre-MDT0001-mdtlov_UUID+5:3355:x1862456998023552:12345-0@lo:13:osp-pre-0-1.0 Request processed in 390us (4299us total) trans 0 rc 0/0 00000100:00100000:2.0:1776178110.308875:0:8232:0:(nrs_fifo.c:241:nrs_fifo_req_stop()) NRS stop fifo request from 12345-0@lo, seq: 406 00000100:00100000:1.0:1776178110.309740:0:3355:0:(client.c:2215:ptlrpc_check_set()) Completed RPC req@000000005d95861f pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_02:lustre-MDT0001-mdtlov_UUID:3355:1862456998023552:0@lo:13:osp-pre-0-1.0 00010000:00010000:0.0:1776178110.880322:0:6245:0:(ldlm_lockd.c:2792:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00010000:00010000:3.0:1776178111.904803:0:9225:0:(ldlm_lockd.c:2792:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00010000:00010000:2.0:1776178112.932795:0:9224:0:(ldlm_lockd.c:2792:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:2.0:1776178113.376554:0:3353:0:(client.c:1733:ptlrpc_send_new_req()) Sending RPC req@00000000b7aa289e pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_00:ac61f839-0e8e-4cac-9c59-08bcfbbad330:3353:1862456998023680:0@lo:400:kworker/u8:0.0 00000100:00100000:2.0:1776178113.376660:0:3353:0:(events.c:361:request_in_callback()) peer: 12345-0@lo (source: 12345-0@lo) 00000100:00100000:2.0:1776178113.376721:0:3353:0:(client.c:1733:ptlrpc_send_new_req()) Sending RPC req@000000007a321680 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_00:lustre-MDT0001-lwp-OST0001_UUID:3353:1862456998023936:0@lo:400:kworker/u8:0.0 00000100:00100000:2.0:1776178113.376774:0:3353:0:(events.c:361:request_in_callback()) peer: 12345-0@lo (source: 12345-0@lo) 00000100:00100000:0.0:1776178113.376811:0:3354:0:(client.c:1733:ptlrpc_send_new_req()) Sending RPC req@000000002a321d27 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_01:lustre-MDT0000-lwp-OST0000_UUID:3354:1862456998023744:0@lo:400:kworker/u8:0.0 00000100:00100000:2.0:1776178113.376812:0:3353:0:(client.c:1733:ptlrpc_send_new_req()) Sending RPC req@000000006466ee18 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_00:lustre-MDT0000-lwp-MDT0001_UUID:3353:1862456998024192:0@lo:400:kworker/u8:0.0 00000100:00100000:1.0:1776178113.376941:0:3356:0:(client.c:1733:ptlrpc_send_new_req()) Sending RPC req@00000000ba5faff8 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_03:lustre-MDT0001-mdtlov_UUID:3356:1862456998023616:0@lo:41:osp-pre-0-1.0 00000100:00100000:3.0:1776178113.377076:0:20137:0:(service.c:2152:ptlrpc_server_handle_req_in()) got req x1862456998023680 00000100:00100000:3.0:1776178113.377452:0:20137:0:(nrs_fifo.c:179:nrs_fifo_req_get()) NRS start fifo request from 12345-0@lo, seq: 119 00000100:00100000:3.0:1776178113.377459:0:20137:0:(service.c:2324:ptlrpc_server_handle_request()) Handling RPC req@00000000fb48a5c8 pname:cluuid+ref:pid:xid:nid:opc:job ll_mgs_0000:ac61f839-0e8e-4cac-9c59-08bcfbbad330+23:3353:x1862456998023680:12345-0@lo:400:kworker/u8:0.0 00000100:00100000:2.0:1776178113.377477:0:3353:0:(events.c:361:request_in_callback()) peer: 12345-0@lo (source: 12345-0@lo) 00000100:00100000:2.0:1776178113.377500:0:3353:0:(client.c:1733:ptlrpc_send_new_req()) Sending RPC req@00000000fed3e037 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_00:lustre-MDT0000-lwp-MDT0000_UUID:3353:1862456998024448:0@lo:400:kworker/u8:0.0 00000100:00100000:3.0:1776178113.377514:0:20137:0:(service.c:2372:ptlrpc_server_handle_request()) Handled RPC req@00000000fb48a5c8 pname:cluuid+ref:pid:xid:nid:opc:job ll_mgs_0000:ac61f839-0e8e-4cac-9c59-08bcfbbad330+23:3353:x1862456998023680:12345-0@lo:400:kworker/u8:0.0 Request processed in 55us (856us total) trans 0 rc 0/0 00000100:00100000:3.0:1776178113.377523:0:20137:0:(nrs_fifo.c:241:nrs_fifo_req_stop()) NRS stop fifo request from 12345-0@lo, seq: 119 00000100:00100000:2.0:1776178113.377525:0:3353:0:(events.c:361:request_in_callback()) peer: 12345-0@lo (source: 12345-0@lo) 00000100:00100000:2.0:1776178113.377557:0:3353:0:(client.c:2215:ptlrpc_check_set()) Completed RPC req@00000000b7aa289e pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_00:ac61f839-0e8e-4cac-9c59-08bcfbbad330:3353:1862456998023680:0@lo:400:kworker/u8:0.0 00000100:00100000:3.0:1776178113.377571:0:11281:0:(service.c:2152:ptlrpc_server_handle_req_in()) got req x1862456998023936 00000100:00100000:2.0:1776178113.377581:0:3353:0:(ptlrpcd.c:415:ptlrpcd_check()) transfer 1 async RPCs [1->0] 00000100:00100000:2.0:1776178113.377587:0:3353:0:(client.c:1733:ptlrpc_send_new_req()) Sending RPC req@00000000a2f460e7 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_00:lustre-MDT0000-mdtlov_UUID:3353:1862456998024512:0@lo:13:osp-pre-1-0.0 00000100:00100000:3.0:1776178113.377602:0:11281:0:(service.c:2152:ptlrpc_server_handle_req_in()) got req x1862456998024192 00000100:00100000:2.0:1776178113.377614:0:3353:0:(events.c:361:request_in_callback()) peer: 12345-0@lo (source: 12345-0@lo) 00000100:00100000:3.0:1776178113.377618:0:11281:0:(service.c:2152:ptlrpc_server_handle_req_in()) got req x1862456998024448 00000100:00100000:3.0:1776178113.377629:0:11281:0:(nrs_fifo.c:179:nrs_fifo_req_get()) NRS start fifo request from 12345-0@lo, seq: 763 00000100:00100000:3.0:1776178113.377633:0:11281:0:(service.c:2324:ptlrpc_server_handle_request()) Handling RPC req@000000003db79e8d pname:cluuid+ref:pid:xid:nid:opc:job mdt00_004:lustre-MDT0001-lwp-OST0001_UUID+5:3353:x1862456998023936:12345-0@lo:400:kworker/u8:0.0 00000100:00100000:2.0:1776178113.377645:0:8250:0:(nrs_fifo.c:179:nrs_fifo_req_get()) NRS start fifo request from 12345-0@lo, seq: 764 00000100:00100000:2.0:1776178113.377650:0:8250:0:(service.c:2324:ptlrpc_server_handle_request()) Handling RPC req@00000000ef2381ea pname:cluuid+ref:pid:xid:nid:opc:job mdt00_003:lustre-MDT0000-lwp-MDT0001_UUID+5:3353:x1862456998024192:12345-0@lo:400:kworker/u8:0.0 00000100:00100000:0.0:1776178113.377661:0:3354:0:(events.c:361:request_in_callback()) peer: 12345-0@lo (source: 12345-0@lo) 00000100:00100000:3.0:1776178113.377693:0:11281:0:(service.c:2372:ptlrpc_server_handle_request()) Handled RPC req@000000003db79e8d pname:cluuid+ref:pid:xid:nid:opc:job mdt00_004:lustre-MDT0001-lwp-OST0001_UUID+5:3353:x1862456998023936:12345-0@lo:400:kworker/u8:0.0 Request processed in 57us (919us total) trans 0 rc 0/0 00000100:00100000:3.0:1776178113.377702:0:11281:0:(nrs_fifo.c:241:nrs_fifo_req_stop()) NRS stop fifo request from 12345-0@lo, seq: 763 00000100:00100000:0.0:1776178113.377709:0:3354:0:(client.c:1733:ptlrpc_send_new_req()) Sending RPC req@00000000e0d60cbc pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_01:lustre-MDT0001-mdtlov_UUID:3354:1862456998024000:0@lo:400:kworker/u8:0.0 00000100:00100000:1.0:1776178113.377720:0:3356:0:(events.c:361:request_in_callback()) peer: 12345-0@lo (source: 12345-0@lo) 00000100:00100000:3.0:1776178113.377720:0:11281:0:(service.c:2152:ptlrpc_server_handle_req_in()) got req x1862456998023744 00000100:00100000:3.0:1776178113.377742:0:11281:0:(nrs_fifo.c:179:nrs_fifo_req_get()) NRS start fifo request from 12345-0@lo, seq: 765 00000100:00100000:3.0:1776178113.377746:0:11281:0:(service.c:2324:ptlrpc_server_handle_request()) Handling RPC req@0000000015f7661d pname:cluuid+ref:pid:xid:nid:opc:job mdt00_004:lustre-MDT0000-lwp-MDT0000_UUID+5:3353:x1862456998024448:12345-0@lo:400:kworker/u8:0.0 00000100:00100000:1.0:1776178113.377756:0:3356:0:(client.c:1733:ptlrpc_send_new_req()) Sending RPC req@000000007cbab86b pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_03:lustre-MDT0000-lwp-OST0001_UUID:3356:1862456998023872:0@lo:400:kworker/u8:0.0 00000100:00100000:0.0:1776178113.377764:0:3354:0:(events.c:361:request_in_callback()) peer: 12345-0@lo (source: 12345-0@lo) 00000100:00100000:1.0:1776178113.377787:0:3356:0:(events.c:361:request_in_callback()) peer: 12345-0@lo (source: 12345-0@lo) 00000100:00100000:3.0:1776178113.377790:0:11281:0:(service.c:2372:ptlrpc_server_handle_request()) Handled RPC req@0000000015f7661d pname:cluuid+ref:pid:xid:nid:opc:job mdt00_004:lustre-MDT0000-lwp-MDT0000_UUID+5:3353:x1862456998024448:12345-0@lo:400:kworker/u8:0.0 Request processed in 43us (266us total) trans 0 rc 0/0 00000100:00100000:3.0:1776178113.377797:0:11281:0:(nrs_fifo.c:241:nrs_fifo_req_stop()) NRS stop fifo request from 12345-0@lo, seq: 765 00000100:00100000:0.0:1776178113.377809:0:3354:0:(client.c:1733:ptlrpc_send_new_req()) Sending RPC req@00000000cb1b73cb pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_01:lustre-MDT0000-mdtlov_UUID:3354:1862456998024256:0@lo:400:kworker/u8:0.0 00000100:00100000:1.0:1776178113.377814:0:3356:0:(client.c:1733:ptlrpc_send_new_req()) Sending RPC req@00000000cc80b69e pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_03:lustre-MDT0001-mdtlov_UUID:3356:1862456998024128:0@lo:400:kworker/u8:0.0 00000100:00100000:3.0:1776178113.377814:0:11281:0:(service.c:2152:ptlrpc_server_handle_req_in()) got req x1862456998023872 00000100:00100000:3.0:1776178113.377828:0:11281:0:(nrs_fifo.c:179:nrs_fifo_req_get()) NRS start fifo request from 12345-0@lo, seq: 766 00000100:00100000:3.0:1776178113.377834:0:11281:0:(service.c:2324:ptlrpc_server_handle_request()) Handling RPC req@000000006a209b80 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_004:lustre-MDT0000-lwp-OST0000_UUID+5:3354:x1862456998023744:12345-0@lo:400:kworker/u8:0.0 00000100:00100000:0.0:1776178113.377859:0:3354:0:(events.c:361:request_in_callback()) peer: 12345-0@lo (source: 12345-0@lo) 00000100:00100000:3.0:1776178113.379480:0:11281:0:(service.c:2372:ptlrpc_server_handle_request()) Handled RPC req@000000006a209b80 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_004:lustre-MDT0000-lwp-OST0000_UUID+5:3354:x1862456998023744:12345-0@lo:400:kworker/u8:0.0 Request processed in 1638us (1819us total) trans 0 rc 0/0 00000100:00100000:3.0:1776178113.379495:0:11281:0:(nrs_fifo.c:241:nrs_fifo_req_stop()) NRS stop fifo request from 12345-0@lo, seq: 766 00000100:00100000:3.0:1776178113.379528:0:11281:0:(nrs_fifo.c:179:nrs_fifo_req_get()) NRS start fifo request from 12345-0@lo, seq: 767 00000100:00100000:3.0:1776178113.379534:0:11281:0:(service.c:2324:ptlrpc_server_handle_request()) Handling RPC req@0000000097ae2a31 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_004:lustre-MDT0000-lwp-OST0001_UUID+5:3356:x1862456998023872:12345-0@lo:400:kworker/u8:0.0 00000100:00100000:3.0:1776178113.379593:0:11281:0:(service.c:2372:ptlrpc_server_handle_request()) Handled RPC req@0000000097ae2a31 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_004:lustre-MDT0000-lwp-OST0001_UUID+5:3356:x1862456998023872:12345-0@lo:400:kworker/u8:0.0 Request processed in 59us (1808us total) trans 0 rc 0/0 00000100:00100000:3.0:1776178113.379599:0:11281:0:(nrs_fifo.c:241:nrs_fifo_req_stop()) NRS stop fifo request from 12345-0@lo, seq: 767 00000100:00100000:3.0:1776178113.379670:0:8232:0:(service.c:2152:ptlrpc_server_handle_req_in()) got req x1862456998024512 00000100:00100000:2.0:1776178113.381554:0:8250:0:(service.c:2372:ptlrpc_server_handle_request()) Handled RPC req@00000000ef2381ea pname:cluuid+ref:pid:xid:nid:opc:job mdt00_003:lustre-MDT0000-lwp-MDT0001_UUID+5:3353:x1862456998024192:12345-0@lo:400:kworker/u8:0.0 Request processed in 3804us (3977us total) trans 0 rc 0/0 00000100:00100000:2.0:1776178113.381573:0:8250:0:(nrs_fifo.c:241:nrs_fifo_req_stop()) NRS stop fifo request from 12345-0@lo, seq: 764 00000100:00100000:2.0:1776178113.381671:0:3353:0:(client.c:2215:ptlrpc_check_set()) Completed RPC req@000000006466ee18 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_00:lustre-MDT0000-lwp-MDT0001_UUID:3353:1862456998024192:0@lo:400:kworker/u8:0.0 00000100:00100000:2.0:1776178113.381688:0:3353:0:(client.c:2215:ptlrpc_check_set()) Completed RPC req@00000000fed3e037 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_00:lustre-MDT0000-lwp-MDT0000_UUID:3353:1862456998024448:0@lo:400:kworker/u8:0.0 00000100:00100000:2.0:1776178113.381702:0:3353:0:(client.c:2215:ptlrpc_check_set()) Completed RPC req@000000007a321680 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_00:lustre-MDT0001-lwp-OST0001_UUID:3353:1862456998023936:0@lo:400:kworker/u8:0.0 00000100:00100000:2.0:1776178113.381802:0:6260:0:(service.c:2152:ptlrpc_server_handle_req_in()) got req x1862456998023616 00000100:00100000:3.0:1776178113.382503:0:8232:0:(nrs_fifo.c:179:nrs_fifo_req_get()) NRS start fifo request from 12345-0@lo, seq: 407 00000100:00100000:3.0:1776178113.382513:0:8232:0:(service.c:2324:ptlrpc_server_handle_request()) Handling RPC req@000000005d3a2b34 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_create00:lustre-MDT0000-mdtlov_UUID+5:3353:x1862456998024512:12345-0@lo:13:osp-pre-1-0.0 00002000:00000024:3.0:1776178113.382563:0:8232:0:(ofd_obd.c:740:ofd_statfs()) blocks cached 229376 granted 491746368 pending 0 free 3923689472 avail 3692167168 00000100:00100000:1.0:1776178113.382614:0:3356:0:(events.c:361:request_in_callback()) peer: 12345-0@lo (source: 12345-0@lo) 00000100:00100000:3.0:1776178113.382631:0:8232:0:(service.c:2372:ptlrpc_server_handle_request()) Handled RPC req@000000005d3a2b34 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_create00:lustre-MDT0000-mdtlov_UUID+5:3353:x1862456998024512:12345-0@lo:13:osp-pre-1-0.0 Request processed in 120us (5017us total) trans 0 rc 0/0 00000100:00100000:3.0:1776178113.382640:0:8232:0:(nrs_fifo.c:241:nrs_fifo_req_stop()) NRS stop fifo request from 12345-0@lo, seq: 407 00000100:00100000:1.0:1776178113.382652:0:3356:0:(client.c:1733:ptlrpc_send_new_req()) Sending RPC req@00000000d2194868 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_03:lustre-MDT0000-mdtlov_UUID:3356:1862456998024384:0@lo:400:kworker/u8:0.0 00000100:00100000:3.0:1776178113.382697:0:3353:0:(client.c:2215:ptlrpc_check_set()) Completed RPC req@00000000a2f460e7 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_00:lustre-MDT0000-mdtlov_UUID:3353:1862456998024512:0@lo:13:osp-pre-1-0.0 00000100:00100000:2.0:1776178113.382731:0:6260:0:(service.c:2152:ptlrpc_server_handle_req_in()) got req x1862456998024000 00000100:00100000:3.0:1776178113.382738:0:15213:0:(service.c:2152:ptlrpc_server_handle_req_in()) got req x1862456998024256 00000100:00100000:2.0:1776178113.382744:0:6260:0:(nrs_fifo.c:179:nrs_fifo_req_get()) NRS start fifo request from 12345-0@lo, seq: 620 00000100:00100000:2.0:1776178113.382748:0:6260:0:(service.c:2324:ptlrpc_server_handle_request()) Handling RPC req@00000000badbd310 pname:cluuid+ref:pid:xid:nid:opc:job mdt_out00_000:lustre-MDT0001-mdtlov_UUID+7:3356:x1862456998023616:12345-0@lo:41:osp-pre-0-1.0 00000100:00100000:3.0:1776178113.382753:0:15213:0:(nrs_fifo.c:179:nrs_fifo_req_get()) NRS start fifo request from 12345-0@lo, seq: 621 00000100:00100000:3.0:1776178113.382757:0:15213:0:(service.c:2324:ptlrpc_server_handle_request()) Handling RPC req@000000009016e75e pname:cluuid+ref:pid:xid:nid:opc:job mdt_out00_003:lustre-MDT0001-mdtlov_UUID+7:3354:x1862456998024000:12345-0@lo:400:kworker/u8:0.0 00000004:00000024:2.0:1776178113.382776:0:6260:0:(mdt_handler.c:548:mdt_statfs()) blocks cached 0 granted 2097152 pending 0 free 1446121472 avail 1316659200 00000100:00100000:3.0:1776178113.382792:0:15213:0:(service.c:2372:ptlrpc_server_handle_request()) Handled RPC req@000000009016e75e pname:cluuid+ref:pid:xid:nid:opc:job mdt_out00_003:lustre-MDT0001-mdtlov_UUID+7:3354:x1862456998024000:12345-0@lo:400:kworker/u8:0.0 Request processed in 34us (5031us total) trans 0 rc 0/0 00000100:00100000:3.0:1776178113.382798:0:15213:0:(nrs_fifo.c:241:nrs_fifo_req_stop()) NRS stop fifo request from 12345-0@lo, seq: 621 00000100:00100000:2.0:1776178113.382808:0:6260:0:(service.c:2372:ptlrpc_server_handle_request()) Handled RPC req@00000000badbd310 pname:cluuid+ref:pid:xid:nid:opc:job mdt_out00_000:lustre-MDT0001-mdtlov_UUID+6:3356:x1862456998023616:12345-0@lo:41:osp-pre-0-1.0 Request processed in 60us (5091us total) trans 0 rc 0/0 00000100:00100000:3.0:1776178113.382811:0:15213:0:(nrs_fifo.c:179:nrs_fifo_req_get()) NRS start fifo request from 12345-0@lo, seq: 622 00000100:00100000:2.0:1776178113.382814:0:6260:0:(nrs_fifo.c:241:nrs_fifo_req_stop()) NRS stop fifo request from 12345-0@lo, seq: 620 00000100:00100000:3.0:1776178113.382814:0:15213:0:(service.c:2324:ptlrpc_server_handle_request()) Handling RPC req@00000000f9929c4b pname:cluuid+ref:pid:xid:nid:opc:job mdt_out00_003:lustre-MDT0000-mdtlov_UUID+5:3354:x1862456998024256:12345-0@lo:400:kworker/u8:0.0 00000100:00100000:3.0:1776178113.382841:0:15213:0:(service.c:2372:ptlrpc_server_handle_request()) Handled RPC req@00000000f9929c4b pname:cluuid+ref:pid:xid:nid:opc:job mdt_out00_003:lustre-MDT0000-mdtlov_UUID+5:3354:x1862456998024256:12345-0@lo:400:kworker/u8:0.0 Request processed in 26us (4985us total) trans 0 rc 0/0 00000100:00100000:3.0:1776178113.382847:0:15213:0:(nrs_fifo.c:241:nrs_fifo_req_stop()) NRS stop fifo request from 12345-0@lo, seq: 622 00000100:00100000:3.0:1776178113.385077:0:3355:0:(client.c:1733:ptlrpc_send_new_req()) Sending RPC req@0000000045560455 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_02:lustre-MDT0001-lwp-OST0000_UUID:3355:1862456998023808:0@lo:400:kworker/u8:0.0 00000100:00100000:2.0:1776178113.386781:0:20754:0:(service.c:2152:ptlrpc_server_handle_req_in()) got req x1862456998024128 00000100:00100000:2.0:1776178113.386876:0:20754:0:(nrs_fifo.c:179:nrs_fifo_req_get()) NRS start fifo request from 12345-0@lo, seq: 412 00000100:00100000:2.0:1776178113.386883:0:20754:0:(service.c:2324:ptlrpc_server_handle_request()) Handling RPC req@0000000056dbc32b pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_005:lustre-MDT0001-mdtlov_UUID+5:3356:x1862456998024128:12345-0@lo:400:kworker/u8:0.0 00000100:00100000:0.0:1776178113.390922:0:3354:0:(client.c:2215:ptlrpc_check_set()) Completed RPC req@00000000cb1b73cb pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_01:lustre-MDT0000-mdtlov_UUID:3354:1862456998024256:0@lo:400:kworker/u8:0.0 00000100:00100000:0.0:1776178113.391011:0:3354:0:(client.c:2215:ptlrpc_check_set()) Completed RPC req@000000002a321d27 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_01:lustre-MDT0000-lwp-OST0000_UUID:3354:1862456998023744:0@lo:400:kworker/u8:0.0 00000100:00100000:0.0:1776178113.391033:0:3354:0:(client.c:2215:ptlrpc_check_set()) Completed RPC req@00000000e0d60cbc pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_01:lustre-MDT0001-mdtlov_UUID:3354:1862456998024000:0@lo:400:kworker/u8:0.0 00000100:00100000:3.0:1776178113.394766:0:3355:0:(events.c:361:request_in_callback()) peer: 12345-0@lo (source: 12345-0@lo) 00000100:00100000:0.0:1776178113.395464:0:6255:0:(service.c:2152:ptlrpc_server_handle_req_in()) got req x1862456998023808 00000100:00100000:0.0:1776178113.395543:0:6255:0:(nrs_fifo.c:179:nrs_fifo_req_get()) NRS start fifo request from 12345-0@lo, seq: 768 00000100:00100000:0.0:1776178113.395550:0:6255:0:(service.c:2324:ptlrpc_server_handle_request()) Handling RPC req@00000000a2f460e7 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_000:lustre-MDT0001-lwp-OST0000_UUID+5:3355:x1862456998023808:12345-0@lo:400:kworker/u8:0.0 00000100:00100000:0.0:1776178113.395638:0:6255:0:(service.c:2372:ptlrpc_server_handle_request()) Handled RPC req@00000000a2f460e7 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_000:lustre-MDT0001-lwp-OST0000_UUID+5:3355:x1862456998023808:12345-0@lo:400:kworker/u8:0.0 Request processed in 87us (889us total) trans 0 rc 0/0 00000100:00100000:0.0:1776178113.395647:0:6255:0:(nrs_fifo.c:241:nrs_fifo_req_stop()) NRS stop fifo request from 12345-0@lo, seq: 768 00000100:00100000:3.0:1776178113.395997:0:3355:0:(client.c:1733:ptlrpc_send_new_req()) Sending RPC req@00000000a7e4e4e8 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_02:lustre-MDT0001-mdtlov_UUID:3355:1862456998024064:0@lo:400:kworker/u8:0.0 00000100:00100000:3.0:1776178113.396054:0:3355:0:(events.c:361:request_in_callback()) peer: 12345-0@lo (source: 12345-0@lo) 00000100:00100000:3.0:1776178113.396103:0:3355:0:(client.c:1733:ptlrpc_send_new_req()) Sending RPC req@00000000a62daf9a pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_02:lustre-MDT0000-mdtlov_UUID:3355:1862456998024320:0@lo:400:kworker/u8:0.0 00000100:00100000:2.0:1776178113.396276:0:20754:0:(service.c:2372:ptlrpc_server_handle_request()) Handled RPC req@0000000056dbc32b pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_005:lustre-MDT0001-mdtlov_UUID+5:3356:x1862456998024128:12345-0@lo:400:kworker/u8:0.0 Request processed in 9390us (13661us total) trans 0 rc 0/0 00000100:00100000:2.0:1776178113.396288:0:20754:0:(nrs_fifo.c:241:nrs_fifo_req_stop()) NRS stop fifo request from 12345-0@lo, seq: 412 00000100:00100000:2.0:1776178113.396317:0:20754:0:(service.c:2152:ptlrpc_server_handle_req_in()) got req x1862456998024064 00000100:00100000:2.0:1776178113.396346:0:20754:0:(nrs_fifo.c:179:nrs_fifo_req_get()) NRS start fifo request from 12345-0@lo, seq: 413 00000100:00100000:2.0:1776178113.396351:0:20754:0:(service.c:2324:ptlrpc_server_handle_request()) Handling RPC req@00000000b9a3856b pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_005:lustre-MDT0001-mdtlov_UUID+5:3355:x1862456998024064:12345-0@lo:400:kworker/u8:0.0 00000100:00100000:2.0:1776178113.396391:0:20754:0:(service.c:2372:ptlrpc_server_handle_request()) Handled RPC req@00000000b9a3856b pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_005:lustre-MDT0001-mdtlov_UUID+5:3355:x1862456998024064:12345-0@lo:400:kworker/u8:0.0 Request processed in 39us (337us total) trans 0 rc 0/0 00000100:00100000:2.0:1776178113.396396:0:20754:0:(nrs_fifo.c:241:nrs_fifo_req_stop()) NRS stop fifo request from 12345-0@lo, seq: 413 00000100:00100000:1.0:1776178113.396509:0:3356:0:(events.c:361:request_in_callback()) peer: 12345-0@lo (source: 12345-0@lo) 00000100:00100000:1.0:1776178113.396554:0:3356:0:(client.c:1733:ptlrpc_send_new_req()) Sending RPC req@000000008c4bfd71 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_03:lustre-MDT0000-mdtlov_UUID:3356:1862456998024640:0@lo:13:osp-pre-0-0.0 00000100:00100000:1.0:1776178113.396588:0:3356:0:(events.c:361:request_in_callback()) peer: 12345-0@lo (source: 12345-0@lo) 00000100:00100000:1.0:1776178113.396730:0:3356:0:(client.c:2215:ptlrpc_check_set()) Completed RPC req@00000000ba5faff8 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_03:lustre-MDT0001-mdtlov_UUID:3356:1862456998023616:0@lo:41:osp-pre-0-1.0 00000100:00100000:1.0:1776178113.396751:0:3356:0:(client.c:2215:ptlrpc_check_set()) Completed RPC req@000000007cbab86b pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_03:lustre-MDT0000-lwp-OST0001_UUID:3356:1862456998023872:0@lo:400:kworker/u8:0.0 00000100:00100000:1.0:1776178113.396768:0:3356:0:(client.c:2215:ptlrpc_check_set()) Completed RPC req@00000000cc80b69e pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_03:lustre-MDT0001-mdtlov_UUID:3356:1862456998024128:0@lo:400:kworker/u8:0.0 00000100:00100000:1.0:1776178113.396818:0:8231:0:(service.c:2152:ptlrpc_server_handle_req_in()) got req x1862456998024640 00000100:00100000:1.0:1776178113.396849:0:8231:0:(nrs_fifo.c:179:nrs_fifo_req_get()) NRS start fifo request from 12345-0@lo, seq: 408 00000100:00100000:1.0:1776178113.396855:0:8231:0:(service.c:2324:ptlrpc_server_handle_request()) Handling RPC req@00000000f4d20636 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_create00:lustre-MDT0000-mdtlov_UUID+5:3356:x1862456998024640:12345-0@lo:13:osp-pre-0-0.0 00000100:00100000:3.0:1776178113.397007:0:3355:0:(events.c:361:request_in_callback()) peer: 12345-0@lo (source: 12345-0@lo) 00000100:00100000:3.0:1776178113.397024:0:3355:0:(client.c:1733:ptlrpc_send_new_req()) Sending RPC req@000000009126d8c6 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_02:lustre-MDT0000-mdtlov_UUID:3355:1862456998024576:0@lo:41:osp-pre-1-0.0 00000100:00100000:3.0:1776178113.397050:0:3355:0:(events.c:361:request_in_callback()) peer: 12345-0@lo (source: 12345-0@lo) 00000100:00100000:2.0:1776178113.397089:0:20764:0:(service.c:2152:ptlrpc_server_handle_req_in()) got req x1862456998024384 00000100:00100000:2.0:1776178113.397117:0:20764:0:(service.c:2152:ptlrpc_server_handle_req_in()) got req x1862456998024320 00000100:00100000:2.0:1776178113.397131:0:20764:0:(nrs_fifo.c:179:nrs_fifo_req_get()) NRS start fifo request from 12345-0@lo, seq: 414 00000100:00100000:2.0:1776178113.397135:0:20764:0:(service.c:2324:ptlrpc_server_handle_request()) Handling RPC req@000000005c76b1b3 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_015:lustre-MDT0000-mdtlov_UUID+5:3356:x1862456998024384:12345-0@lo:400:kworker/u8:0.0 00000100:00100000:2.0:1776178113.397179:0:20764:0:(service.c:2372:ptlrpc_server_handle_request()) Handled RPC req@000000005c76b1b3 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_015:lustre-MDT0000-mdtlov_UUID+5:3356:x1862456998024384:12345-0@lo:400:kworker/u8:0.0 Request processed in 43us (671us total) trans 0 rc 0/0 00000100:00100000:2.0:1776178113.397185:0:20764:0:(nrs_fifo.c:241:nrs_fifo_req_stop()) NRS stop fifo request from 12345-0@lo, seq: 414 00000100:00100000:2.0:1776178113.397198:0:20764:0:(nrs_fifo.c:179:nrs_fifo_req_get()) NRS start fifo request from 12345-0@lo, seq: 415 00000100:00100000:2.0:1776178113.397202:0:20764:0:(service.c:2324:ptlrpc_server_handle_request()) Handling RPC req@00000000599b0cf2 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_015:lustre-MDT0000-mdtlov_UUID+6:3355:x1862456998024320:12345-0@lo:400:kworker/u8:0.0 00000100:00100000:2.0:1776178113.397226:0:20764:0:(service.c:2372:ptlrpc_server_handle_request()) Handled RPC req@00000000599b0cf2 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_015:lustre-MDT0000-mdtlov_UUID+6:3355:x1862456998024320:12345-0@lo:400:kworker/u8:0.0 Request processed in 23us (220us total) trans 0 rc 0/0 00000100:00100000:2.0:1776178113.397232:0:20764:0:(nrs_fifo.c:241:nrs_fifo_req_stop()) NRS stop fifo request from 12345-0@lo, seq: 415 00000100:00100000:2.0:1776178113.397298:0:6261:0:(service.c:2152:ptlrpc_server_handle_req_in()) got req x1862456998024576 00000100:00100000:0.0:1776178113.397310:0:3356:0:(client.c:2215:ptlrpc_check_set()) Completed RPC req@00000000d2194868 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_03:lustre-MDT0000-mdtlov_UUID:3356:1862456998024384:0@lo:400:kworker/u8:0.0 00000100:00100000:2.0:1776178113.397341:0:6261:0:(nrs_fifo.c:179:nrs_fifo_req_get()) NRS start fifo request from 12345-0@lo, seq: 623 00000100:00100000:2.0:1776178113.397348:0:6261:0:(service.c:2324:ptlrpc_server_handle_request()) Handling RPC req@000000005fdea934 pname:cluuid+ref:pid:xid:nid:opc:job mdt_out00_001:lustre-MDT0000-mdtlov_UUID+5:3355:x1862456998024576:12345-0@lo:41:osp-pre-1-0.0 00000004:00000024:2.0:1776178113.397386:0:6261:0:(mdt_handler.c:548:mdt_statfs()) blocks cached 0 granted 2097152 pending 0 free 1446039552 avail 1316577280 00000100:00100000:2.0:1776178113.397420:0:6261:0:(service.c:2372:ptlrpc_server_handle_request()) Handled RPC req@000000005fdea934 pname:cluuid+ref:pid:xid:nid:opc:job mdt_out00_001:lustre-MDT0000-mdtlov_UUID+5:3355:x1862456998024576:12345-0@lo:41:osp-pre-1-0.0 Request processed in 73us (371us total) trans 0 rc 0/0 00000100:00100000:2.0:1776178113.397438:0:6261:0:(nrs_fifo.c:241:nrs_fifo_req_stop()) NRS stop fifo request from 12345-0@lo, seq: 623 00002000:00000024:1.0:1776178113.397511:0:8231:0:(ofd_obd.c:740:ofd_statfs()) blocks cached 315392 granted 484883776 pending 0 free 3923689472 avail 3692167168 00000100:00100000:3.0:1776178113.397521:0:3355:0:(client.c:2215:ptlrpc_check_set()) Completed RPC req@0000000045560455 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_02:lustre-MDT0001-lwp-OST0000_UUID:3355:1862456998023808:0@lo:400:kworker/u8:0.0 00000100:00100000:3.0:1776178113.397539:0:3355:0:(client.c:2215:ptlrpc_check_set()) Completed RPC req@00000000a7e4e4e8 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_02:lustre-MDT0001-mdtlov_UUID:3355:1862456998024064:0@lo:400:kworker/u8:0.0 00000100:00100000:3.0:1776178113.397551:0:3355:0:(client.c:2215:ptlrpc_check_set()) Completed RPC req@00000000a62daf9a pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_02:lustre-MDT0000-mdtlov_UUID:3355:1862456998024320:0@lo:400:kworker/u8:0.0 00000100:00100000:3.0:1776178113.397565:0:3355:0:(client.c:2215:ptlrpc_check_set()) Completed RPC req@000000009126d8c6 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_02:lustre-MDT0000-mdtlov_UUID:3355:1862456998024576:0@lo:41:osp-pre-1-0.0 00000100:00100000:1.0:1776178113.397572:0:8231:0:(service.c:2372:ptlrpc_server_handle_request()) Handled RPC req@00000000f4d20636 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_create00:lustre-MDT0000-mdtlov_UUID+5:3356:x1862456998024640:12345-0@lo:13:osp-pre-0-0.0 Request processed in 717us (982us total) trans 0 rc 0/0 00000100:00100000:1.0:1776178113.397583:0:8231:0:(nrs_fifo.c:241:nrs_fifo_req_stop()) NRS stop fifo request from 12345-0@lo, seq: 408 00000100:00100000:0.0:1776178113.397653:0:3356:0:(client.c:2215:ptlrpc_check_set()) Completed RPC req@000000008c4bfd71 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_03:lustre-MDT0000-mdtlov_UUID:3356:1862456998024640:0@lo:13:osp-pre-0-0.0 00010000:00010000:1.0:1776178113.953100:0:6244:0:(ldlm_lockd.c:2792:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:3.0:1776178114.338292:0:3346:0:(events.c:361:request_in_callback()) peer: 12345-192.168.204.3@tcp (source: 12345-192.168.204.3@tcp) 00000100:00100000:3.0:1776178114.338517:0:3346:0:(events.c:361:request_in_callback()) peer: 12345-192.168.204.3@tcp (source: 12345-192.168.204.3@tcp) 00000100:00100000:3.0:1776178114.338567:0:3346:0:(events.c:361:request_in_callback()) peer: 12345-192.168.204.3@tcp (source: 12345-192.168.204.3@tcp) 00000100:00100000:3.0:1776178114.338594:0:3346:0:(events.c:361:request_in_callback()) peer: 12345-192.168.204.3@tcp (source: 12345-192.168.204.3@tcp) 00000100:00100000:3.0:1776178114.338621:0:3346:0:(events.c:361:request_in_callback()) peer: 12345-192.168.204.3@tcp (source: 12345-192.168.204.3@tcp) 00000100:00100000:3.0:1776178114.338661:0:20140:0:(service.c:2152:ptlrpc_server_handle_req_in()) got req x1862456979082496 00000100:00100000:1.0:1776178114.338702:0:20762:0:(service.c:2152:ptlrpc_server_handle_req_in()) got req x1862456979082688 00000100:00100000:3.0:1776178114.338725:0:20140:0:(nrs_fifo.c:179:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.204.3@tcp, seq: 120 00000100:00100000:3.0:1776178114.338734:0:20140:0:(service.c:2324:ptlrpc_server_handle_request()) Handling RPC req@0000000060db901b pname:cluuid+ref:pid:xid:nid:opc:job ll_mgs_0002:5b5c66b1-1a2f-4651-8548-0c75b9a38a94+9:2270:x1862456979082496:12345-192.168.204.3@tcp:400:kworker/u8:4.0 00000100:00100000:1.0:1776178114.338735:0:20762:0:(service.c:2152:ptlrpc_server_handle_req_in()) got req x1862456979082752 00000100:00100000:1.0:1776178114.338752:0:20762:0:(nrs_fifo.c:179:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.204.3@tcp, seq: 416 00000100:00100000:1.0:1776178114.338758:0:20762:0:(service.c:2324:ptlrpc_server_handle_request()) Handling RPC req@00000000f141ac55 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_013:b50963bd-8eba-4432-9a54-8888d00e3fa1+9:2270:x1862456979082688:12345-192.168.204.3@tcp:400:kworker/u8:4.0 00000100:00100000:2.0:1776178114.338846:0:6257:0:(service.c:2152:ptlrpc_server_handle_req_in()) got req x1862456979082560 00000100:00100000:3.0:1776178114.339234:0:20140:0:(service.c:2372:ptlrpc_server_handle_request()) Handled RPC req@0000000060db901b pname:cluuid+ref:pid:xid:nid:opc:job ll_mgs_0002:5b5c66b1-1a2f-4651-8548-0c75b9a38a94+9:2270:x1862456979082496:12345-192.168.204.3@tcp:400:kworker/u8:4.0 Request processed in 500us (666us total) trans 0 rc 0/0 00000100:00100000:3.0:1776178114.339243:0:20140:0:(nrs_fifo.c:241:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.204.3@tcp, seq: 120 00000100:00100000:2.0:1776178114.341104:0:6257:0:(service.c:2152:ptlrpc_server_handle_req_in()) got req x1862456979082624 00000100:00100000:2.0:1776178114.341132:0:6257:0:(nrs_fifo.c:179:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.204.3@tcp, seq: 769 00000100:00100000:2.0:1776178114.341139:0:6257:0:(service.c:2324:ptlrpc_server_handle_request()) Handling RPC req@000000009126d8c6 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_002:b50963bd-8eba-4432-9a54-8888d00e3fa1+21:2270:x1862456979082560:12345-192.168.204.3@tcp:400:kworker/u8:4.0 00000100:00100000:0.0:1776178114.341296:0:20764:0:(nrs_fifo.c:179:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.204.3@tcp, seq: 417 00000100:00100000:0.0:1776178114.341316:0:20764:0:(service.c:2324:ptlrpc_server_handle_request()) Handling RPC req@000000001f0b1867 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_015:b50963bd-8eba-4432-9a54-8888d00e3fa1+8:2270:x1862456979082752:12345-192.168.204.3@tcp:400:kworker/u8:4.0 00000100:00100000:1.0:1776178114.344822:0:20762:0:(service.c:2372:ptlrpc_server_handle_request()) Handled RPC req@00000000f141ac55 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_013:b50963bd-8eba-4432-9a54-8888d00e3fa1+9:2270:x1862456979082688:12345-192.168.204.3@tcp:400:kworker/u8:4.0 Request processed in 6055us (6299us total) trans 0 rc 0/0 00000100:00100000:1.0:1776178114.344837:0:20762:0:(nrs_fifo.c:241:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.204.3@tcp, seq: 416 00000100:00100000:1.0:1776178114.344939:0:8250:0:(nrs_fifo.c:179:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.204.3@tcp, seq: 770 00000100:00100000:1.0:1776178114.344947:0:8250:0:(service.c:2324:ptlrpc_server_handle_request()) Handling RPC req@00000000f6af7e5b pname:cluuid+ref:pid:xid:nid:opc:job mdt00_003:b50963bd-8eba-4432-9a54-8888d00e3fa1+6:2270:x1862456979082624:12345-192.168.204.3@tcp:400:kworker/u8:4.0 00000100:00100000:1.0:1776178114.344994:0:8250:0:(service.c:2372:ptlrpc_server_handle_request()) Handled RPC req@00000000f6af7e5b pname:cluuid+ref:pid:xid:nid:opc:job mdt00_003:b50963bd-8eba-4432-9a54-8888d00e3fa1+6:2270:x1862456979082624:12345-192.168.204.3@tcp:400:kworker/u8:4.0 Request processed in 48us (6400us total) trans 0 rc 0/0 00000100:00100000:1.0:1776178114.345000:0:8250:0:(nrs_fifo.c:241:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.204.3@tcp, seq: 770 00000100:00100000:2.0:1776178114.350842:0:6257:0:(service.c:2372:ptlrpc_server_handle_request()) Handled RPC req@000000009126d8c6 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_002:b50963bd-8eba-4432-9a54-8888d00e3fa1+21:2270:x1862456979082560:12345-192.168.204.3@tcp:400:kworker/u8:4.0 Request processed in 9690us (12547us total) trans 0 rc 0/0 00000100:00100000:2.0:1776178114.350868:0:6257:0:(nrs_fifo.c:241:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.204.3@tcp, seq: 769 00000100:00100000:0.0:1776178114.357261:0:20764:0:(service.c:2372:ptlrpc_server_handle_request()) Handled RPC req@000000001f0b1867 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_015:b50963bd-8eba-4432-9a54-8888d00e3fa1+8:2270:x1862456979082752:12345-192.168.204.3@tcp:400:kworker/u8:4.0 Request processed in 15944us (18626us total) trans 0 rc 0/0 00000100:00100000:0.0:1776178114.357284:0:20764:0:(nrs_fifo.c:241:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.204.3@tcp, seq: 417 00000100:00100000:2.0:1776178114.918351:0:3353:0:(client.c:1733:ptlrpc_send_new_req()) Sending RPC req@00000000a9cfe89c pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_00:lustre-MDT0001-mdtlov_UUID:3353:1862456998024704:0@lo:13:osp-pre-1-1.0 00000100:00100000:2.0:1776178114.918903:0:3353:0:(events.c:361:request_in_callback()) peer: 12345-0@lo (source: 12345-0@lo) 00000100:00100000:1.0:1776178114.919200:0:8231:0:(service.c:2152:ptlrpc_server_handle_req_in()) got req x1862456998024704 00000100:00100000:1.0:1776178114.919257:0:8231:0:(nrs_fifo.c:179:nrs_fifo_req_get()) NRS start fifo request from 12345-0@lo, seq: 409 00000100:00100000:1.0:1776178114.919264:0:8231:0:(service.c:2324:ptlrpc_server_handle_request()) Handling RPC req@000000002ab0a797 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_create00:lustre-MDT0001-mdtlov_UUID+5:3353:x1862456998024704:12345-0@lo:13:osp-pre-1-1.0 00002000:00000024:1.0:1776178114.919292:0:8231:0:(ofd_obd.c:740:ofd_statfs()) blocks cached 229376 granted 491746368 pending 0 free 3923689472 avail 3692167168 00000100:00100000:1.0:1776178114.919354:0:8231:0:(service.c:2372:ptlrpc_server_handle_request()) Handled RPC req@000000002ab0a797 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_create00:lustre-MDT0001-mdtlov_UUID+5:3353:x1862456998024704:12345-0@lo:13:osp-pre-1-1.0 Request processed in 90us (453us total) trans 0 rc 0/0 00000100:00100000:1.0:1776178114.919361:0:8231:0:(nrs_fifo.c:241:nrs_fifo_req_stop()) NRS stop fifo request from 12345-0@lo, seq: 409 00000100:00100000:2.0:1776178114.919595:0:3353:0:(client.c:2215:ptlrpc_check_set()) Completed RPC req@00000000a9cfe89c pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_00:lustre-MDT0001-mdtlov_UUID:3353:1862456998024704:0@lo:13:osp-pre-1-1.0 00010000:00010000:0.0:1776178114.977024:0:6245:0:(ldlm_lockd.c:2792:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:3.0:1776178115.424863:0:3353:0:(ptlrpcd.c:415:ptlrpcd_check()) transfer 1 async RPCs [1->0] 00000100:00100000:3.0:1776178115.424899:0:3353:0:(client.c:1733:ptlrpc_send_new_req()) Sending RPC req@000000005112bc3b pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_00:lustre-MDT0001-mdtlov_UUID:3353:1862456998024768:0@lo:13:osp-pre-0-1.0 00000100:00100000:3.0:1776178115.424980:0:3353:0:(events.c:361:request_in_callback()) peer: 12345-0@lo (source: 12345-0@lo) 00000100:00100000:1.0:1776178115.425166:0:8232:0:(service.c:2152:ptlrpc_server_handle_req_in()) got req x1862456998024768 00000100:00100000:1.0:1776178115.425238:0:8232:0:(nrs_fifo.c:179:nrs_fifo_req_get()) NRS start fifo request from 12345-0@lo, seq: 410 00000100:00100000:1.0:1776178115.425244:0:8232:0:(service.c:2324:ptlrpc_server_handle_request()) Handling RPC req@00000000ee45921e pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_create00:lustre-MDT0001-mdtlov_UUID+5:3353:x1862456998024768:12345-0@lo:13:osp-pre-0-1.0 00002000:00000024:1.0:1776178115.425284:0:8232:0:(ofd_obd.c:740:ofd_statfs()) blocks cached 315392 granted 484883776 pending 0 free 3923689472 avail 3692167168 00000100:00100000:1.0:1776178115.425334:0:8232:0:(service.c:2372:ptlrpc_server_handle_request()) Handled RPC req@00000000ee45921e pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_create00:lustre-MDT0001-mdtlov_UUID+5:3353:x1862456998024768:12345-0@lo:13:osp-pre-0-1.0 Request processed in 89us (356us total) trans 0 rc 0/0 00000100:00100000:1.0:1776178115.425344:0:8232:0:(nrs_fifo.c:241:nrs_fifo_req_stop()) NRS stop fifo request from 12345-0@lo, seq: 410 00000100:00100000:2.0:1776178115.425755:0:3353:0:(client.c:2215:ptlrpc_check_set()) Completed RPC req@000000005112bc3b pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_00:lustre-MDT0001-mdtlov_UUID:3353:1862456998024768:0@lo:13:osp-pre-0-1.0 00010000:00010000:3.0:1776178116.003235:0:9225:0:(ldlm_lockd.c:2792:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00010000:00010000:2.0:1776178117.025189:0:9224:0:(ldlm_lockd.c:2792:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00010000:00010000:1.0:1776178118.052189:0:6244:0:(ldlm_lockd.c:2792:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:1.0:1776178118.496495:0:3355:0:(client.c:1733:ptlrpc_send_new_req()) Sending RPC req@00000000a9cfe89c pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_02:lustre-MDT0000-mdtlov_UUID:3355:1862456998024832:0@lo:41:osp-pre-1-0.0 00000100:00100000:1.0:1776178118.496606:0:3355:0:(events.c:361:request_in_callback()) peer: 12345-0@lo (source: 12345-0@lo) 00000100:00100000:1.0:1776178118.496658:0:3355:0:(client.c:1733:ptlrpc_send_new_req()) Sending RPC req@00000000a62daf9a pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_02:lustre-MDT0000-lwp-OST0001_UUID:3355:1862456998025088:0@lo:400:kworker/u8:0.0 00000100:00100000:1.0:1776178118.496687:0:3355:0:(events.c:361:request_in_callback()) peer: 12345-0@lo (source: 12345-0@lo) 00000100:00100000:0.0:1776178118.496703:0:3353:0:(client.c:1733:ptlrpc_send_new_req()) Sending RPC req@00000000cb1b73cb pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_00:lustre-MDT0000-lwp-OST0000_UUID:3353:1862456998024960:0@lo:400:kworker/u8:0.0 00000100:00100000:1.0:1776178118.496706:0:3355:0:(client.c:1733:ptlrpc_send_new_req()) Sending RPC req@000000009de51106 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_02:lustre-MDT0001-mdtlov_UUID:3355:1862456998025344:0@lo:400:kworker/u8:0.0 00000100:00100000:1.0:1776178118.496750:0:3355:0:(events.c:361:request_in_callback()) peer: 12345-0@lo (source: 12345-0@lo) 00000100:00100000:0.0:1776178118.496755:0:3353:0:(events.c:361:request_in_callback()) peer: 12345-0@lo (source: 12345-0@lo) 00000100:00100000:1.0:1776178118.496772:0:3355:0:(client.c:1733:ptlrpc_send_new_req()) Sending RPC req@000000007d60c0e9 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_02:lustre-MDT0000-mdtlov_UUID:3355:1862456998025600:0@lo:400:kworker/u8:0.0 00000100:00100000:0.0:1776178118.496786:0:3353:0:(client.c:1733:ptlrpc_send_new_req()) Sending RPC req@00000000fed3e037 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_00:lustre-MDT0001-mdtlov_UUID:3353:1862456998025216:0@lo:400:kworker/u8:0.0 00000100:00100000:1.0:1776178118.496799:0:3355:0:(events.c:361:request_in_callback()) peer: 12345-0@lo (source: 12345-0@lo) 00000100:00100000:0.0:1776178118.496819:0:3353:0:(events.c:361:request_in_callback()) peer: 12345-0@lo (source: 12345-0@lo) 00000100:00100000:1.0:1776178118.496823:0:3355:0:(ptlrpcd.c:415:ptlrpcd_check()) transfer 4 async RPCs [3->2] 00000100:00100000:1.0:1776178118.496831:0:3355:0:(client.c:1733:ptlrpc_send_new_req()) Sending RPC req@00000000d2194868 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_02:ac61f839-0e8e-4cac-9c59-08bcfbbad330:3355:1862456998024896:0@lo:400:kworker/u8:0.0 00000100:00100000:0.0:1776178118.496841:0:3353:0:(client.c:1733:ptlrpc_send_new_req()) Sending RPC req@0000000015095c30 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_00:lustre-MDT0000-mdtlov_UUID:3353:1862456998025472:0@lo:400:kworker/u8:0.0 00000100:00100000:2.0:1776178118.496938:0:6257:0:(service.c:2152:ptlrpc_server_handle_req_in()) got req x1862456998025088 00000100:00100000:2.0:1776178118.496999:0:6257:0:(service.c:2152:ptlrpc_server_handle_req_in()) got req x1862456998024960 00000100:00100000:2.0:1776178118.497022:0:6257:0:(nrs_fifo.c:179:nrs_fifo_req_get()) NRS start fifo request from 12345-0@lo, seq: 771 00000100:00100000:2.0:1776178118.497029:0:6257:0:(service.c:2324:ptlrpc_server_handle_request()) Handling RPC req@000000006e48e9ca pname:cluuid+ref:pid:xid:nid:opc:job mdt00_002:lustre-MDT0000-lwp-OST0001_UUID+5:3355:x1862456998025088:12345-0@lo:400:kworker/u8:0.0 00000100:00100000:3.0:1776178118.498830:0:6255:0:(nrs_fifo.c:179:nrs_fifo_req_get()) NRS start fifo request from 12345-0@lo, seq: 772 00000100:00100000:3.0:1776178118.498845:0:6255:0:(service.c:2324:ptlrpc_server_handle_request()) Handling RPC req@0000000058cd86cb pname:cluuid+ref:pid:xid:nid:opc:job mdt00_000:lustre-MDT0000-lwp-OST0000_UUID+5:3353:x1862456998024960:12345-0@lo:400:kworker/u8:0.0 00000100:00100000:1.0:1776178118.500197:0:3355:0:(events.c:361:request_in_callback()) peer: 12345-0@lo (source: 12345-0@lo) 00000100:00100000:1.0:1776178118.500423:0:20137:0:(service.c:2152:ptlrpc_server_handle_req_in()) got req x1862456998024896 00000100:00100000:1.0:1776178118.500494:0:20137:0:(nrs_fifo.c:179:nrs_fifo_req_get()) NRS start fifo request from 12345-0@lo, seq: 121 00000100:00100000:1.0:1776178118.500501:0:20137:0:(service.c:2324:ptlrpc_server_handle_request()) Handling RPC req@000000003e5a23bc pname:cluuid+ref:pid:xid:nid:opc:job ll_mgs_0000:ac61f839-0e8e-4cac-9c59-08bcfbbad330+23:3355:x1862456998024896:12345-0@lo:400:kworker/u8:0.0 00000100:00100000:1.0:1776178118.500570:0:20137:0:(service.c:2372:ptlrpc_server_handle_request()) Handled RPC req@000000003e5a23bc pname:cluuid+ref:pid:xid:nid:opc:job ll_mgs_0000:ac61f839-0e8e-4cac-9c59-08bcfbbad330+23:3355:x1862456998024896:12345-0@lo:400:kworker/u8:0.0 Request processed in 68us (380us total) trans 0 rc 0/0 00000100:00100000:1.0:1776178118.500579:0:20137:0:(nrs_fifo.c:241:nrs_fifo_req_stop()) NRS stop fifo request from 12345-0@lo, seq: 121 00000100:00100000:1.0:1776178118.500768:0:3355:0:(client.c:1733:ptlrpc_send_new_req()) Sending RPC req@00000000cc80b69e pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_02:lustre-MDT0001-lwp-OST0001_UUID:3355:1862456998025152:0@lo:400:kworker/u8:0.0 00000100:00100000:1.0:1776178118.500806:0:3355:0:(events.c:361:request_in_callback()) peer: 12345-0@lo (source: 12345-0@lo) 00000100:00100000:1.0:1776178118.500827:0:8250:0:(service.c:2152:ptlrpc_server_handle_req_in()) got req x1862456998025152 00000100:00100000:1.0:1776178118.500855:0:8250:0:(nrs_fifo.c:179:nrs_fifo_req_get()) NRS start fifo request from 12345-0@lo, seq: 773 00000100:00100000:1.0:1776178118.500859:0:8250:0:(service.c:2324:ptlrpc_server_handle_request()) Handling RPC req@000000001d51eec7 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_003:lustre-MDT0001-lwp-OST0001_UUID+5:3355:x1862456998025152:12345-0@lo:400:kworker/u8:0.0 00000100:00100000:1.0:1776178118.500889:0:8250:0:(service.c:2372:ptlrpc_server_handle_request()) Handled RPC req@000000001d51eec7 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_003:lustre-MDT0001-lwp-OST0001_UUID+5:3355:x1862456998025152:12345-0@lo:400:kworker/u8:0.0 Request processed in 30us (84us total) trans 0 rc 0/0 00000100:00100000:1.0:1776178118.500895:0:8250:0:(nrs_fifo.c:241:nrs_fifo_req_stop()) NRS stop fifo request from 12345-0@lo, seq: 773 00000100:00100000:0.0:1776178118.500913:0:3353:0:(events.c:361:request_in_callback()) peer: 12345-0@lo (source: 12345-0@lo) 00000100:00100000:1.0:1776178118.500914:0:3355:0:(client.c:1733:ptlrpc_send_new_req()) Sending RPC req@00000000c63a98c1 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_02:lustre-MDT0000-lwp-MDT0001_UUID:3355:1862456998025408:0@lo:400:kworker/u8:0.0 00000100:00100000:1.0:1776178118.500944:0:3355:0:(events.c:361:request_in_callback()) peer: 12345-0@lo (source: 12345-0@lo) 00000100:00100000:0.0:1776178118.500956:0:3353:0:(client.c:1733:ptlrpc_send_new_req()) Sending RPC req@000000008c4bfd71 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_00:lustre-MDT0001-mdtlov_UUID:3353:1862456998025728:0@lo:41:osp-pre-0-1.0 00000100:00100000:1.0:1776178118.500964:0:3355:0:(client.c:1733:ptlrpc_send_new_req()) Sending RPC req@0000000049b1b339 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_02:lustre-MDT0000-lwp-MDT0000_UUID:3355:1862456998025664:0@lo:400:kworker/u8:0.0 00000100:00100000:0.0:1776178118.500999:0:3353:0:(events.c:361:request_in_callback()) peer: 12345-0@lo (source: 12345-0@lo) 00000100:00100000:1.0:1776178118.501006:0:3355:0:(events.c:361:request_in_callback()) peer: 12345-0@lo (source: 12345-0@lo) 00000100:00100000:0.0:1776178118.501034:0:3353:0:(ptlrpcd.c:415:ptlrpcd_check()) transfer 4 async RPCs [1->0] 00000100:00100000:1.0:1776178118.501036:0:8250:0:(service.c:2152:ptlrpc_server_handle_req_in()) got req x1862456998025408 00000100:00100000:0.0:1776178118.501044:0:3353:0:(client.c:1733:ptlrpc_send_new_req()) Sending RPC req@000000007d5bffd0 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_00:lustre-MDT0001-lwp-OST0000_UUID:3353:1862456998025024:0@lo:400:kworker/u8:0.0 00000100:00100000:1.0:1776178118.501053:0:8250:0:(service.c:2152:ptlrpc_server_handle_req_in()) got req x1862456998025664 00000100:00100000:0.0:1776178118.501098:0:3353:0:(events.c:361:request_in_callback()) peer: 12345-0@lo (source: 12345-0@lo) 00000100:00100000:0.0:1776178118.501110:0:3353:0:(client.c:1733:ptlrpc_send_new_req()) Sending RPC req@000000006466ee18 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_00:lustre-MDT0001-mdtlov_UUID:3353:1862456998025280:0@lo:400:kworker/u8:0.0 00000100:00100000:0.0:1776178118.501134:0:3353:0:(events.c:361:request_in_callback()) peer: 12345-0@lo (source: 12345-0@lo) 00000100:00100000:0.0:1776178118.501152:0:3353:0:(client.c:1733:ptlrpc_send_new_req()) Sending RPC req@00000000f0f1167e pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_00:lustre-MDT0000-mdtlov_UUID:3353:1862456998025536:0@lo:400:kworker/u8:0.0 00000100:00100000:0.0:1776178118.501171:0:3353:0:(events.c:361:request_in_callback()) peer: 12345-0@lo (source: 12345-0@lo) 00000100:00100000:0.0:1776178118.501188:0:3353:0:(client.c:1733:ptlrpc_send_new_req()) Sending RPC req@00000000124cea36 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_00:lustre-MDT0000-mdtlov_UUID:3353:1862456998025792:0@lo:13:osp-pre-0-0.0 00000100:00100000:1.0:1776178118.501198:0:8250:0:(service.c:2152:ptlrpc_server_handle_req_in()) got req x1862456998025024 00000100:00100000:1.0:1776178118.501215:0:8250:0:(nrs_fifo.c:179:nrs_fifo_req_get()) NRS start fifo request from 12345-0@lo, seq: 774 00000100:00100000:1.0:1776178118.501218:0:8250:0:(service.c:2324:ptlrpc_server_handle_request()) Handling RPC req@00000000478fd8d5 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_003:lustre-MDT0000-lwp-MDT0001_UUID+5:3355:x1862456998025408:12345-0@lo:400:kworker/u8:0.0 00000100:00100000:0.0:1776178118.501399:0:3353:0:(events.c:361:request_in_callback()) peer: 12345-0@lo (source: 12345-0@lo) 00000100:00100000:1.0:1776178118.501446:0:8250:0:(service.c:2372:ptlrpc_server_handle_request()) Handled RPC req@00000000478fd8d5 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_003:lustre-MDT0000-lwp-MDT0001_UUID+5:3355:x1862456998025408:12345-0@lo:400:kworker/u8:0.0 Request processed in 225us (502us total) trans 0 rc 0/0 00000100:00100000:1.0:1776178118.501454:0:8250:0:(nrs_fifo.c:241:nrs_fifo_req_stop()) NRS stop fifo request from 12345-0@lo, seq: 774 00000100:00100000:1.0:1776178118.501471:0:8250:0:(nrs_fifo.c:179:nrs_fifo_req_get()) NRS start fifo request from 12345-0@lo, seq: 775 00000100:00100000:1.0:1776178118.501476:0:8250:0:(service.c:2324:ptlrpc_server_handle_request()) Handling RPC req@00000000e64148ab pname:cluuid+ref:pid:xid:nid:opc:job mdt00_003:lustre-MDT0000-lwp-MDT0000_UUID+5:3355:x1862456998025664:12345-0@lo:400:kworker/u8:0.0 00000100:00100000:2.0:1776178118.501489:0:6257:0:(service.c:2372:ptlrpc_server_handle_request()) Handled RPC req@000000006e48e9ca pname:cluuid+ref:pid:xid:nid:opc:job mdt00_002:lustre-MDT0000-lwp-OST0001_UUID+5:3355:x1862456998025088:12345-0@lo:400:kworker/u8:0.0 Request processed in 4457us (4798us total) trans 0 rc 0/0 00000100:00100000:2.0:1776178118.501498:0:6257:0:(nrs_fifo.c:241:nrs_fifo_req_stop()) NRS stop fifo request from 12345-0@lo, seq: 771 00000100:00100000:1.0:1776178118.501511:0:8250:0:(service.c:2372:ptlrpc_server_handle_request()) Handled RPC req@00000000e64148ab pname:cluuid+ref:pid:xid:nid:opc:job mdt00_003:lustre-MDT0000-lwp-MDT0000_UUID+5:3355:x1862456998025664:12345-0@lo:400:kworker/u8:0.0 Request processed in 34us (505us total) trans 0 rc 0/0 00000100:00100000:1.0:1776178118.501519:0:8250:0:(nrs_fifo.c:241:nrs_fifo_req_stop()) NRS stop fifo request from 12345-0@lo, seq: 775 00000100:00100000:2.0:1776178118.501525:0:6257:0:(nrs_fifo.c:179:nrs_fifo_req_get()) NRS start fifo request from 12345-0@lo, seq: 776 00000100:00100000:2.0:1776178118.501531:0:6257:0:(service.c:2324:ptlrpc_server_handle_request()) Handling RPC req@0000000026195e8c pname:cluuid+ref:pid:xid:nid:opc:job mdt00_002:lustre-MDT0001-lwp-OST0000_UUID+5:3353:x1862456998025024:12345-0@lo:400:kworker/u8:0.0 00000100:00100000:1.0:1776178118.501546:0:20754:0:(service.c:2152:ptlrpc_server_handle_req_in()) got req x1862456998025344 00000100:00100000:2.0:1776178118.501569:0:6257:0:(service.c:2372:ptlrpc_server_handle_request()) Handled RPC req@0000000026195e8c pname:cluuid+ref:pid:xid:nid:opc:job mdt00_002:lustre-MDT0001-lwp-OST0000_UUID+5:3353:x1862456998025024:12345-0@lo:400:kworker/u8:0.0 Request processed in 37us (470us total) trans 0 rc 0/0 00000100:00100000:2.0:1776178118.501575:0:6257:0:(nrs_fifo.c:241:nrs_fifo_req_stop()) NRS stop fifo request from 12345-0@lo, seq: 776 00000100:00100000:1.0:1776178118.501581:0:20754:0:(service.c:2152:ptlrpc_server_handle_req_in()) got req x1862456998025600 00000100:00100000:2.0:1776178118.501604:0:6260:0:(service.c:2152:ptlrpc_server_handle_req_in()) got req x1862456998024832 00000100:00100000:1.0:1776178118.501607:0:20754:0:(service.c:2152:ptlrpc_server_handle_req_in()) got req x1862456998025280 00000100:00100000:2.0:1776178118.501629:0:6260:0:(service.c:2152:ptlrpc_server_handle_req_in()) got req x1862456998025216 00000100:00100000:1.0:1776178118.501645:0:20754:0:(service.c:2152:ptlrpc_server_handle_req_in()) got req x1862456998025536 00000100:00100000:2.0:1776178118.501651:0:6260:0:(service.c:2152:ptlrpc_server_handle_req_in()) got req x1862456998025472 00000100:00100000:1.0:1776178118.501672:0:20754:0:(nrs_fifo.c:179:nrs_fifo_req_get()) NRS start fifo request from 12345-0@lo, seq: 418 00000100:00100000:1.0:1776178118.501676:0:20754:0:(service.c:2324:ptlrpc_server_handle_request()) Handling RPC req@00000000956deaca pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_005:lustre-MDT0001-mdtlov_UUID+5:3355:x1862456998025344:12345-0@lo:400:kworker/u8:0.0 00000100:00100000:2.0:1776178118.501690:0:6260:0:(service.c:2152:ptlrpc_server_handle_req_in()) got req x1862456998025728 00000100:00100000:2.0:1776178118.501704:0:6260:0:(nrs_fifo.c:179:nrs_fifo_req_get()) NRS start fifo request from 12345-0@lo, seq: 624 00000100:00100000:2.0:1776178118.501710:0:6260:0:(service.c:2324:ptlrpc_server_handle_request()) Handling RPC req@00000000f8c48966 pname:cluuid+ref:pid:xid:nid:opc:job mdt_out00_000:lustre-MDT0000-mdtlov_UUID+6:3355:x1862456998024832:12345-0@lo:41:osp-pre-1-0.0 00000100:00100000:1.0:1776178118.501715:0:20754:0:(service.c:2372:ptlrpc_server_handle_request()) Handled RPC req@00000000956deaca pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_005:lustre-MDT0001-mdtlov_UUID+5:3355:x1862456998025344:12345-0@lo:400:kworker/u8:0.0 Request processed in 37us (4964us total) trans 0 rc 0/0 00000100:00100000:1.0:1776178118.501722:0:20754:0:(nrs_fifo.c:241:nrs_fifo_req_stop()) NRS stop fifo request from 12345-0@lo, seq: 418 00000100:00100000:1.0:1776178118.501734:0:20754:0:(nrs_fifo.c:179:nrs_fifo_req_get()) NRS start fifo request from 12345-0@lo, seq: 419 00000100:00100000:1.0:1776178118.501737:0:20754:0:(service.c:2324:ptlrpc_server_handle_request()) Handling RPC req@0000000023210d1c pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_005:lustre-MDT0000-mdtlov_UUID+5:3355:x1862456998025600:12345-0@lo:400:kworker/u8:0.0 00000004:00000024:2.0:1776178118.501743:0:6260:0:(mdt_handler.c:548:mdt_statfs()) blocks cached 0 granted 2097152 pending 0 free 1446039552 avail 1316577280 00000100:00100000:1.0:1776178118.501768:0:20754:0:(service.c:2372:ptlrpc_server_handle_request()) Handled RPC req@0000000023210d1c pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_005:lustre-MDT0000-mdtlov_UUID+5:3355:x1862456998025600:12345-0@lo:400:kworker/u8:0.0 Request processed in 30us (4969us total) trans 0 rc 0/0 00000100:00100000:1.0:1776178118.501775:0:20754:0:(nrs_fifo.c:241:nrs_fifo_req_stop()) NRS stop fifo request from 12345-0@lo, seq: 419 00000100:00100000:2.0:1776178118.501791:0:6260:0:(service.c:2372:ptlrpc_server_handle_request()) Handled RPC req@00000000f8c48966 pname:cluuid+ref:pid:xid:nid:opc:job mdt_out00_000:lustre-MDT0000-mdtlov_UUID+6:3355:x1862456998024832:12345-0@lo:41:osp-pre-1-0.0 Request processed in 78us (5184us total) trans 0 rc 0/0 00000100:00100000:1.0:1776178118.501792:0:20754:0:(nrs_fifo.c:179:nrs_fifo_req_get()) NRS start fifo request from 12345-0@lo, seq: 420 00000100:00100000:1.0:1776178118.501798:0:20754:0:(service.c:2324:ptlrpc_server_handle_request()) Handling RPC req@0000000021d5d4e8 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_005:lustre-MDT0001-mdtlov_UUID+5:3353:x1862456998025280:12345-0@lo:400:kworker/u8:0.0 00000100:00100000:2.0:1776178118.501803:0:6260:0:(nrs_fifo.c:241:nrs_fifo_req_stop()) NRS stop fifo request from 12345-0@lo, seq: 624 00000100:00100000:2.0:1776178118.501818:0:6260:0:(nrs_fifo.c:179:nrs_fifo_req_get()) NRS start fifo request from 12345-0@lo, seq: 625 00000100:00100000:2.0:1776178118.501823:0:6260:0:(service.c:2324:ptlrpc_server_handle_request()) Handling RPC req@000000004420855e pname:cluuid+ref:pid:xid:nid:opc:job mdt_out00_000:lustre-MDT0001-mdtlov_UUID+7:3353:x1862456998025216:12345-0@lo:400:kworker/u8:0.0 00000100:00100000:1.0:1776178118.501831:0:20754:0:(service.c:2372:ptlrpc_server_handle_request()) Handled RPC req@0000000021d5d4e8 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_005:lustre-MDT0001-mdtlov_UUID+5:3353:x1862456998025280:12345-0@lo:400:kworker/u8:0.0 Request processed in 32us (695us total) trans 0 rc 0/0 00000100:00100000:1.0:1776178118.501838:0:20754:0:(nrs_fifo.c:241:nrs_fifo_req_stop()) NRS stop fifo request from 12345-0@lo, seq: 420 00000100:00100000:1.0:1776178118.501850:0:20754:0:(nrs_fifo.c:179:nrs_fifo_req_get()) NRS start fifo request from 12345-0@lo, seq: 421 00000100:00100000:1.0:1776178118.501855:0:20754:0:(service.c:2324:ptlrpc_server_handle_request()) Handling RPC req@0000000078ae567e pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_005:lustre-MDT0000-mdtlov_UUID+5:3353:x1862456998025536:12345-0@lo:400:kworker/u8:0.0 00000100:00100000:2.0:1776178118.501862:0:6260:0:(service.c:2372:ptlrpc_server_handle_request()) Handled RPC req@000000004420855e pname:cluuid+ref:pid:xid:nid:opc:job mdt_out00_000:lustre-MDT0001-mdtlov_UUID+7:3353:x1862456998025216:12345-0@lo:400:kworker/u8:0.0 Request processed in 37us (5043us total) trans 0 rc 0/0 00000100:00100000:2.0:1776178118.501869:0:6260:0:(nrs_fifo.c:241:nrs_fifo_req_stop()) NRS stop fifo request from 12345-0@lo, seq: 625 00000100:00100000:2.0:1776178118.501886:0:6260:0:(nrs_fifo.c:179:nrs_fifo_req_get()) NRS start fifo request from 12345-0@lo, seq: 626 00000100:00100000:1.0:1776178118.501889:0:20754:0:(service.c:2372:ptlrpc_server_handle_request()) Handled RPC req@0000000078ae567e pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_005:lustre-MDT0000-mdtlov_UUID+5:3353:x1862456998025536:12345-0@lo:400:kworker/u8:0.0 Request processed in 33us (716us total) trans 0 rc 0/0 00000100:00100000:2.0:1776178118.501893:0:6260:0:(service.c:2324:ptlrpc_server_handle_request()) Handling RPC req@00000000e2b18a6d pname:cluuid+ref:pid:xid:nid:opc:job mdt_out00_000:lustre-MDT0000-mdtlov_UUID+5:3353:x1862456998025472:12345-0@lo:400:kworker/u8:0.0 00000100:00100000:1.0:1776178118.501903:0:20754:0:(nrs_fifo.c:241:nrs_fifo_req_stop()) NRS stop fifo request from 12345-0@lo, seq: 421 00000100:00100000:2.0:1776178118.501930:0:6260:0:(service.c:2372:ptlrpc_server_handle_request()) Handled RPC req@00000000e2b18a6d pname:cluuid+ref:pid:xid:nid:opc:job mdt_out00_000:lustre-MDT0000-mdtlov_UUID+5:3353:x1862456998025472:12345-0@lo:400:kworker/u8:0.0 Request processed in 38us (1018us total) trans 0 rc 0/0 00000100:00100000:2.0:1776178118.501937:0:6260:0:(nrs_fifo.c:241:nrs_fifo_req_stop()) NRS stop fifo request from 12345-0@lo, seq: 626 00000100:00100000:1.0:1776178118.501950:0:3355:0:(client.c:2215:ptlrpc_check_set()) Completed RPC req@00000000a62daf9a pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_02:lustre-MDT0000-lwp-OST0001_UUID:3355:1862456998025088:0@lo:400:kworker/u8:0.0 00000100:00100000:2.0:1776178118.501951:0:6260:0:(nrs_fifo.c:179:nrs_fifo_req_get()) NRS start fifo request from 12345-0@lo, seq: 627 00000100:00100000:2.0:1776178118.501958:0:6260:0:(service.c:2324:ptlrpc_server_handle_request()) Handling RPC req@00000000456acdaf pname:cluuid+ref:pid:xid:nid:opc:job mdt_out00_000:lustre-MDT0001-mdtlov_UUID+6:3353:x1862456998025728:12345-0@lo:41:osp-pre-0-1.0 00000100:00100000:1.0:1776178118.501970:0:3355:0:(client.c:2215:ptlrpc_check_set()) Completed RPC req@000000009de51106 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_02:lustre-MDT0001-mdtlov_UUID:3355:1862456998025344:0@lo:400:kworker/u8:0.0 00000004:00000024:2.0:1776178118.501979:0:6260:0:(mdt_handler.c:548:mdt_statfs()) blocks cached 0 granted 2097152 pending 0 free 1446121472 avail 1316659200 00000100:00100000:1.0:1776178118.501987:0:3355:0:(client.c:2215:ptlrpc_check_set()) Completed RPC req@000000007d60c0e9 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_02:lustre-MDT0000-mdtlov_UUID:3355:1862456998025600:0@lo:400:kworker/u8:0.0 00000100:00100000:1.0:1776178118.502007:0:3355:0:(client.c:2215:ptlrpc_check_set()) Completed RPC req@00000000a9cfe89c pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_02:lustre-MDT0000-mdtlov_UUID:3355:1862456998024832:0@lo:41:osp-pre-1-0.0 00000100:00100000:2.0:1776178118.502012:0:6260:0:(service.c:2372:ptlrpc_server_handle_request()) Handled RPC req@00000000456acdaf pname:cluuid+ref:pid:xid:nid:opc:job mdt_out00_000:lustre-MDT0001-mdtlov_UUID+6:3353:x1862456998025728:12345-0@lo:41:osp-pre-0-1.0 Request processed in 53us (1012us total) trans 0 rc 0/0 00000100:00100000:2.0:1776178118.502018:0:6260:0:(nrs_fifo.c:241:nrs_fifo_req_stop()) NRS stop fifo request from 12345-0@lo, seq: 627 00000100:00100000:1.0:1776178118.502050:0:3355:0:(client.c:2215:ptlrpc_check_set()) Completed RPC req@00000000d2194868 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_02:ac61f839-0e8e-4cac-9c59-08bcfbbad330:3355:1862456998024896:0@lo:400:kworker/u8:0.0 00000100:00100000:2.0:1776178118.502203:0:3356:0:(ptlrpcd.c:415:ptlrpcd_check()) transfer 1 async RPCs [2->3] 00000100:00100000:0.0:1776178118.502526:0:3353:0:(client.c:2215:ptlrpc_check_set()) Completed RPC req@000000008c4bfd71 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_00:lustre-MDT0001-mdtlov_UUID:3353:1862456998025728:0@lo:41:osp-pre-0-1.0 00000100:00100000:0.0:1776178118.502538:0:3353:0:(client.c:2215:ptlrpc_check_set()) Completed RPC req@00000000fed3e037 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_00:lustre-MDT0001-mdtlov_UUID:3353:1862456998025216:0@lo:400:kworker/u8:0.0 00000100:00100000:0.0:1776178118.502546:0:3353:0:(client.c:2215:ptlrpc_check_set()) Completed RPC req@0000000015095c30 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_00:lustre-MDT0000-mdtlov_UUID:3353:1862456998025472:0@lo:400:kworker/u8:0.0 00000100:00100000:0.0:1776178118.502573:0:3353:0:(client.c:2215:ptlrpc_check_set()) Completed RPC req@000000007d5bffd0 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_00:lustre-MDT0001-lwp-OST0000_UUID:3353:1862456998025024:0@lo:400:kworker/u8:0.0 00000100:00100000:0.0:1776178118.502585:0:3353:0:(client.c:2215:ptlrpc_check_set()) Completed RPC req@000000006466ee18 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_00:lustre-MDT0001-mdtlov_UUID:3353:1862456998025280:0@lo:400:kworker/u8:0.0 00000100:00100000:0.0:1776178118.502595:0:3353:0:(client.c:2215:ptlrpc_check_set()) Completed RPC req@00000000f0f1167e pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_00:lustre-MDT0000-mdtlov_UUID:3353:1862456998025536:0@lo:400:kworker/u8:0.0 00000100:00100000:1.0:1776178118.502626:0:3355:0:(client.c:2215:ptlrpc_check_set()) Completed RPC req@00000000cc80b69e pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_02:lustre-MDT0001-lwp-OST0001_UUID:3355:1862456998025152:0@lo:400:kworker/u8:0.0 00000100:00100000:1.0:1776178118.502648:0:3355:0:(client.c:2215:ptlrpc_check_set()) Completed RPC req@00000000c63a98c1 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_02:lustre-MDT0000-lwp-MDT0001_UUID:3355:1862456998025408:0@lo:400:kworker/u8:0.0 00000100:00100000:0.0:1776178118.502654:0:8231:0:(service.c:2152:ptlrpc_server_handle_req_in()) got req x1862456998025792 00000100:00100000:1.0:1776178118.502666:0:3355:0:(client.c:2215:ptlrpc_check_set()) Completed RPC req@0000000049b1b339 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_02:lustre-MDT0000-lwp-MDT0000_UUID:3355:1862456998025664:0@lo:400:kworker/u8:0.0 00000100:00100000:0.0:1776178118.502709:0:8231:0:(nrs_fifo.c:179:nrs_fifo_req_get()) NRS start fifo request from 12345-0@lo, seq: 411 00000100:00100000:0.0:1776178118.502714:0:8231:0:(service.c:2324:ptlrpc_server_handle_request()) Handling RPC req@00000000a6227918 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_create00:lustre-MDT0000-mdtlov_UUID+5:3353:x1862456998025792:12345-0@lo:13:osp-pre-0-0.0 00002000:00000024:0.0:1776178118.502739:0:8231:0:(ofd_obd.c:740:ofd_statfs()) blocks cached 315392 granted 484883776 pending 0 free 3923689472 avail 3692167168 00000100:00100000:0.0:1776178118.502781:0:8231:0:(service.c:2372:ptlrpc_server_handle_request()) Handled RPC req@00000000a6227918 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_create00:lustre-MDT0000-mdtlov_UUID+5:3353:x1862456998025792:12345-0@lo:13:osp-pre-0-0.0 Request processed in 66us (1383us total) trans 0 rc 0/0 00000100:00100000:0.0:1776178118.502787:0:8231:0:(nrs_fifo.c:241:nrs_fifo_req_stop()) NRS stop fifo request from 12345-0@lo, seq: 411 00000100:00100000:0.0:1776178118.502821:0:3353:0:(client.c:2215:ptlrpc_check_set()) Completed RPC req@00000000124cea36 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_00:lustre-MDT0000-mdtlov_UUID:3353:1862456998025792:0@lo:13:osp-pre-0-0.0 00000100:00100000:2.0:1776178118.502925:0:3356:0:(client.c:1733:ptlrpc_send_new_req()) Sending RPC req@000000005112bc3b pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_03:lustre-MDT0000-mdtlov_UUID:3356:1862456998025856:0@lo:13:osp-pre-1-0.0 00000100:00100000:2.0:1776178118.502966:0:3356:0:(events.c:361:request_in_callback()) peer: 12345-0@lo (source: 12345-0@lo) 00000100:00100000:3.0:1776178118.503020:0:6255:0:(service.c:2372:ptlrpc_server_handle_request()) Handled RPC req@0000000058cd86cb pname:cluuid+ref:pid:xid:nid:opc:job mdt00_000:lustre-MDT0000-lwp-OST0000_UUID+5:3353:x1862456998024960:12345-0@lo:400:kworker/u8:0.0 Request processed in 4175us (6267us total) trans 0 rc 0/0 00000100:00100000:3.0:1776178118.503031:0:6255:0:(nrs_fifo.c:241:nrs_fifo_req_stop()) NRS stop fifo request from 12345-0@lo, seq: 772 00000100:00100000:0.0:1776178118.503170:0:8231:0:(service.c:2152:ptlrpc_server_handle_req_in()) got req x1862456998025856 00000100:00100000:2.0:1776178118.503385:0:8232:0:(nrs_fifo.c:179:nrs_fifo_req_get()) NRS start fifo request from 12345-0@lo, seq: 412 00000100:00100000:2.0:1776178118.503390:0:8232:0:(service.c:2324:ptlrpc_server_handle_request()) Handling RPC req@0000000080e4108d pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_create00:lustre-MDT0000-mdtlov_UUID+5:3356:x1862456998025856:12345-0@lo:13:osp-pre-1-0.0 00002000:00000024:2.0:1776178118.503416:0:8232:0:(ofd_obd.c:740:ofd_statfs()) blocks cached 229376 granted 491746368 pending 0 free 3923689472 avail 3692167168 00000100:00100000:2.0:1776178118.503463:0:8232:0:(service.c:2372:ptlrpc_server_handle_request()) Handled RPC req@0000000080e4108d pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_create00:lustre-MDT0000-mdtlov_UUID+5:3356:x1862456998025856:12345-0@lo:13:osp-pre-1-0.0 Request processed in 73us (498us total) trans 0 rc 0/0 00000100:00100000:2.0:1776178118.503469:0:8232:0:(nrs_fifo.c:241:nrs_fifo_req_stop()) NRS stop fifo request from 12345-0@lo, seq: 412 00000100:00100000:1.0:1776178118.504034:0:3353:0:(client.c:2215:ptlrpc_check_set()) Completed RPC req@00000000cb1b73cb pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_00:lustre-MDT0000-lwp-OST0000_UUID:3353:1862456998024960:0@lo:400:kworker/u8:0.0 00000100:00100000:3.0:1776178118.504482:0:3356:0:(client.c:2215:ptlrpc_check_set()) Completed RPC req@000000005112bc3b pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_03:lustre-MDT0000-mdtlov_UUID:3356:1862456998025856:0@lo:13:osp-pre-1-0.0 00010000:00010000:0.0:1776178119.072251:0:6245:0:(ldlm_lockd.c:2792:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:3.0:1776178119.457822:0:3348:0:(events.c:361:request_in_callback()) peer: 12345-192.168.204.3@tcp (source: 12345-192.168.204.3@tcp) 00000100:00100000:1.0:1776178119.458170:0:20140:0:(service.c:2152:ptlrpc_server_handle_req_in()) got req x1862456979082816 00000100:00100000:1.0:1776178119.458244:0:20140:0:(nrs_fifo.c:179:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.204.3@tcp, seq: 122 00000100:00100000:1.0:1776178119.458262:0:20140:0:(service.c:2324:ptlrpc_server_handle_request()) Handling RPC req@00000000fc8eef33 pname:cluuid+ref:pid:xid:nid:opc:job ll_mgs_0002:5b5c66b1-1a2f-4651-8548-0c75b9a38a94+9:2270:x1862456979082816:12345-192.168.204.3@tcp:400:kworker/u8:4.0 00000100:00100000:1.0:1776178119.458355:0:20140:0:(service.c:2372:ptlrpc_server_handle_request()) Handled RPC req@00000000fc8eef33 pname:cluuid+ref:pid:xid:nid:opc:job ll_mgs_0002:5b5c66b1-1a2f-4651-8548-0c75b9a38a94+9:2270:x1862456979082816:12345-192.168.204.3@tcp:400:kworker/u8:4.0 Request processed in 103us (542us total) trans 0 rc 0/0 00000100:00100000:1.0:1776178119.458363:0:20140:0:(nrs_fifo.c:241:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.204.3@tcp, seq: 122 00000100:00100000:3.0:1776178119.469011:0:3346:0:(events.c:361:request_in_callback()) peer: 12345-192.168.204.3@tcp (source: 12345-192.168.204.3@tcp) 00000100:00100000:3.0:1776178119.469123:0:3346:0:(events.c:361:request_in_callback()) peer: 12345-192.168.204.3@tcp (source: 12345-192.168.204.3@tcp) 00000100:00100000:3.0:1776178119.469154:0:3346:0:(events.c:361:request_in_callback()) peer: 12345-192.168.204.3@tcp (source: 12345-192.168.204.3@tcp) 00000100:00100000:3.0:1776178119.469185:0:3346:0:(events.c:361:request_in_callback()) peer: 12345-192.168.204.3@tcp (source: 12345-192.168.204.3@tcp) 00000100:00100000:0.0:1776178119.469388:0:20759:0:(service.c:2152:ptlrpc_server_handle_req_in()) got req x1862456979083008 00000100:00100000:0.0:1776178119.469444:0:20759:0:(service.c:2152:ptlrpc_server_handle_req_in()) got req x1862456979083072 00000100:00100000:0.0:1776178119.469463:0:20759:0:(nrs_fifo.c:179:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.204.3@tcp, seq: 422 00000100:00100000:0.0:1776178119.469472:0:20759:0:(service.c:2324:ptlrpc_server_handle_request()) Handling RPC req@0000000053e142bc pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_010:b50963bd-8eba-4432-9a54-8888d00e3fa1+9:2271:x1862456979083008:12345-192.168.204.3@tcp:400:kworker/u8:4.0 00000100:00100000:0.0:1776178119.469543:0:20759:0:(service.c:2372:ptlrpc_server_handle_request()) Handled RPC req@0000000053e142bc pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_010:b50963bd-8eba-4432-9a54-8888d00e3fa1+9:2271:x1862456979083008:12345-192.168.204.3@tcp:400:kworker/u8:4.0 Request processed in 71us (421us total) trans 0 rc 0/0 00000100:00100000:0.0:1776178119.469550:0:20759:0:(nrs_fifo.c:241:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.204.3@tcp, seq: 422 00000100:00100000:0.0:1776178119.469565:0:20759:0:(nrs_fifo.c:179:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.204.3@tcp, seq: 423 00000100:00100000:0.0:1776178119.469569:0:20759:0:(service.c:2324:ptlrpc_server_handle_request()) Handling RPC req@000000008c4bfd71 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_010:b50963bd-8eba-4432-9a54-8888d00e3fa1+8:2270:x1862456979083072:12345-192.168.204.3@tcp:400:kworker/u8:4.0 00000100:00100000:0.0:1776178119.469586:0:20759:0:(service.c:2372:ptlrpc_server_handle_request()) Handled RPC req@000000008c4bfd71 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_010:b50963bd-8eba-4432-9a54-8888d00e3fa1+8:2270:x1862456979083072:12345-192.168.204.3@tcp:400:kworker/u8:4.0 Request processed in 17us (401us total) trans 0 rc 0/0 00000100:00100000:0.0:1776178119.469592:0:20759:0:(nrs_fifo.c:241:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.204.3@tcp, seq: 423 00000100:00100000:2.0:1776178119.469772:0:6256:0:(service.c:2152:ptlrpc_server_handle_req_in()) got req x1862456979082880 00000100:00100000:2.0:1776178119.469804:0:6256:0:(service.c:2152:ptlrpc_server_handle_req_in()) got req x1862456979082944 00000100:00100000:2.0:1776178119.469821:0:6256:0:(nrs_fifo.c:179:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.204.3@tcp, seq: 777 00000100:00100000:2.0:1776178119.469825:0:6256:0:(service.c:2324:ptlrpc_server_handle_request()) Handling RPC req@000000001554c6b4 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_001:b50963bd-8eba-4432-9a54-8888d00e3fa1+21:2271:x1862456979082880:12345-192.168.204.3@tcp:400:kworker/u8:4.0 00000100:00100000:2.0:1776178119.469855:0:6256:0:(service.c:2372:ptlrpc_server_handle_request()) Handled RPC req@000000001554c6b4 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_001:b50963bd-8eba-4432-9a54-8888d00e3fa1+21:2271:x1862456979082880:12345-192.168.204.3@tcp:400:kworker/u8:4.0 Request processed in 30us (858us total) trans 0 rc 0/0 00000100:00100000:2.0:1776178119.469861:0:6256:0:(nrs_fifo.c:241:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.204.3@tcp, seq: 777 00000100:00100000:2.0:1776178119.469871:0:6256:0:(nrs_fifo.c:179:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.204.3@tcp, seq: 778 00000100:00100000:2.0:1776178119.469874:0:6256:0:(service.c:2324:ptlrpc_server_handle_request()) Handling RPC req@00000000124cea36 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_001:b50963bd-8eba-4432-9a54-8888d00e3fa1+6:2270:x1862456979082944:12345-192.168.204.3@tcp:400:kworker/u8:4.0 00000100:00100000:2.0:1776178119.469890:0:6256:0:(service.c:2372:ptlrpc_server_handle_request()) Handled RPC req@00000000124cea36 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_001:b50963bd-8eba-4432-9a54-8888d00e3fa1+6:2270:x1862456979082944:12345-192.168.204.3@tcp:400:kworker/u8:4.0 Request processed in 16us (737us total) trans 0 rc 0/0 00000100:00100000:2.0:1776178119.469895:0:6256:0:(nrs_fifo.c:241:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.204.3@tcp, seq: 778 00000100:00100000:2.0:1776178120.034587:0:3356:0:(client.c:1733:ptlrpc_send_new_req()) Sending RPC req@00000000a9cfe89c pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_03:lustre-MDT0001-mdtlov_UUID:3356:1862456998025920:0@lo:13:osp-pre-1-1.0 00000100:00100000:2.0:1776178120.034683:0:3356:0:(events.c:361:request_in_callback()) peer: 12345-0@lo (source: 12345-0@lo) 00000100:00100000:0.0:1776178120.034793:0:8231:0:(service.c:2152:ptlrpc_server_handle_req_in()) got req x1862456998025920 00000100:00100000:0.0:1776178120.034850:0:8231:0:(nrs_fifo.c:179:nrs_fifo_req_get()) NRS start fifo request from 12345-0@lo, seq: 413 00000100:00100000:0.0:1776178120.034856:0:8231:0:(service.c:2324:ptlrpc_server_handle_request()) Handling RPC req@000000009e476239 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_create00:lustre-MDT0001-mdtlov_UUID+5:3356:x1862456998025920:12345-0@lo:13:osp-pre-1-1.0 00002000:00000024:0.0:1776178120.034888:0:8231:0:(ofd_obd.c:740:ofd_statfs()) blocks cached 229376 granted 491746368 pending 0 free 3923689472 avail 3692167168 00000100:00100000:0.0:1776178120.034946:0:8231:0:(service.c:2372:ptlrpc_server_handle_request()) Handled RPC req@000000009e476239 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_create00:lustre-MDT0001-mdtlov_UUID+5:3356:x1862456998025920:12345-0@lo:13:osp-pre-1-1.0 Request processed in 89us (264us total) trans 0 rc 0/0 00000100:00100000:0.0:1776178120.034953:0:8231:0:(nrs_fifo.c:241:nrs_fifo_req_stop()) NRS stop fifo request from 12345-0@lo, seq: 413 00000100:00100000:3.0:1776178120.035769:0:3356:0:(client.c:2215:ptlrpc_check_set()) Completed RPC req@00000000a9cfe89c pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_03:lustre-MDT0001-mdtlov_UUID:3356:1862456998025920:0@lo:13:osp-pre-1-1.0 00010000:00010000:3.0:1776178120.096335:0:9225:0:(ldlm_lockd.c:2792:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:1.0:1776178120.544482:0:3353:0:(client.c:1733:ptlrpc_send_new_req()) Sending RPC req@0000000054e3866c pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_00:lustre-MDT0001-mdtlov_UUID:3353:1862456998025984:0@lo:13:osp-pre-0-1.0 00000100:00100000:1.0:1776178120.544556:0:3353:0:(events.c:361:request_in_callback()) peer: 12345-0@lo (source: 12345-0@lo) 00000100:00100000:2.0:1776178120.544764:0:8232:0:(service.c:2152:ptlrpc_server_handle_req_in()) got req x1862456998025984 00000100:00100000:2.0:1776178120.544834:0:8232:0:(nrs_fifo.c:179:nrs_fifo_req_get()) NRS start fifo request from 12345-0@lo, seq: 414 00000100:00100000:2.0:1776178120.544840:0:8232:0:(service.c:2324:ptlrpc_server_handle_request()) Handling RPC req@00000000f95e9da5 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_create00:lustre-MDT0001-mdtlov_UUID+5:3353:x1862456998025984:12345-0@lo:13:osp-pre-0-1.0 00002000:00000024:2.0:1776178120.544871:0:8232:0:(ofd_obd.c:740:ofd_statfs()) blocks cached 315392 granted 484883776 pending 0 free 3923689472 avail 3692167168 00000100:00100000:2.0:1776178120.544923:0:8232:0:(service.c:2372:ptlrpc_server_handle_request()) Handled RPC req@00000000f95e9da5 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_create00:lustre-MDT0001-mdtlov_UUID+5:3353:x1862456998025984:12345-0@lo:13:osp-pre-0-1.0 Request processed in 82us (368us total) trans 0 rc 0/0 00000100:00100000:2.0:1776178120.544931:0:8232:0:(nrs_fifo.c:241:nrs_fifo_req_stop()) NRS stop fifo request from 12345-0@lo, seq: 414 00000100:00100000:1.0:1776178120.545134:0:3353:0:(client.c:2215:ptlrpc_check_set()) Completed RPC req@0000000054e3866c pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_00:lustre-MDT0001-mdtlov_UUID:3353:1862456998025984:0@lo:13:osp-pre-0-1.0 00010000:00010000:2.0:1776178121.122725:0:9224:0:(ldlm_lockd.c:2792:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00010000:00010000:1.0:1776178122.144373:0:6244:0:(ldlm_lockd.c:2792:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00010000:00010000:1.0:1776178123.171845:0:6245:0:(ldlm_lockd.c:2792:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:1.0:1776178123.616380:0:3353:0:(ptlrpcd.c:415:ptlrpcd_check()) transfer 1 async RPCs [1->0] 00000100:00100000:1.0:1776178123.616405:0:3353:0:(client.c:1733:ptlrpc_send_new_req()) Sending RPC req@00000000228025f2 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_00:lustre-MDT0001-mdtlov_UUID:3353:1862456998026048:0@lo:41:osp-pre-0-1.0 00000100:00100000:1.0:1776178123.616470:0:3353:0:(events.c:361:request_in_callback()) peer: 12345-0@lo (source: 12345-0@lo) 00000100:00100000:1.0:1776178123.616499:0:3353:0:(client.c:1733:ptlrpc_send_new_req()) Sending RPC req@00000000a6227918 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_00:lustre-MDT0001-lwp-OST0000_UUID:3353:1862456998026240:0@lo:400:kworker/u8:2.0 00000100:00100000:0.0:1776178123.616517:0:3354:0:(client.c:1733:ptlrpc_send_new_req()) Sending RPC req@00000000f0f1167e pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_01:lustre-MDT0000-lwp-OST0001_UUID:3354:1862456998026304:0@lo:400:kworker/u8:2.0 00000100:00100000:3.0:1776178123.616620:0:3356:0:(client.c:1733:ptlrpc_send_new_req()) Sending RPC req@0000000022d4dc29 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_03:lustre-MDT0000-lwp-OST0000_UUID:3356:1862456998026176:0@lo:400:kworker/u8:2.0 00000100:00100000:2.0:1776178123.616739:0:3355:0:(client.c:1733:ptlrpc_send_new_req()) Sending RPC req@00000000699e66b5 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_02:ac61f839-0e8e-4cac-9c59-08bcfbbad330:3355:1862456998026112:0@lo:400:kworker/u8:2.0 00000100:00100000:1.0:1776178123.621947:0:3353:0:(events.c:361:request_in_callback()) peer: 12345-0@lo (source: 12345-0@lo) 00000100:00100000:1.0:1776178123.622003:0:3353:0:(client.c:1733:ptlrpc_send_new_req()) Sending RPC req@0000000024989815 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_00:lustre-MDT0001-mdtlov_UUID:3353:1862456998026496:0@lo:400:kworker/u8:2.0 00000100:00100000:1.0:1776178123.622054:0:3353:0:(events.c:361:request_in_callback()) peer: 12345-0@lo (source: 12345-0@lo) 00000100:00100000:1.0:1776178123.625972:0:3353:0:(client.c:1733:ptlrpc_send_new_req()) Sending RPC req@000000007d60c0e9 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_00:lustre-MDT0000-mdtlov_UUID:3353:1862456998026752:0@lo:400:kworker/u8:2.0 00000100:00100000:1.0:1776178123.626040:0:3353:0:(events.c:361:request_in_callback()) peer: 12345-0@lo (source: 12345-0@lo) 00000100:00100000:1.0:1776178123.626115:0:3353:0:(client.c:1733:ptlrpc_send_new_req()) Sending RPC req@000000008b858f58 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_00:lustre-MDT0000-mdtlov_UUID:3353:1862456998027008:0@lo:41:osp-pre-1-0.0 00000100:00100000:1.0:1776178123.626949:0:3353:0:(events.c:361:request_in_callback()) peer: 12345-0@lo (source: 12345-0@lo) 00000100:00100000:1.0:1776178123.626985:0:3353:0:(ptlrpcd.c:415:ptlrpcd_check()) transfer 1 async RPCs [1->0] 00000100:00100000:1.0:1776178123.626993:0:3353:0:(client.c:1733:ptlrpc_send_new_req()) Sending RPC req@0000000031795589 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_00:lustre-MDT0000-mdtlov_UUID:3353:1862456998027072:0@lo:13:osp-pre-1-0.0 00000100:00100000:1.0:1776178123.627021:0:3353:0:(events.c:361:request_in_callback()) peer: 12345-0@lo (source: 12345-0@lo) 00000100:00100000:1.0:1776178123.627110:0:15213:0:(service.c:2152:ptlrpc_server_handle_req_in()) got req x1862456998026048 00000100:00100000:1.0:1776178123.627336:0:15213:0:(service.c:2152:ptlrpc_server_handle_req_in()) got req x1862456998027008 00000100:00100000:1.0:1776178123.627357:0:15213:0:(nrs_fifo.c:179:nrs_fifo_req_get()) NRS start fifo request from 12345-0@lo, seq: 628 00000100:00100000:1.0:1776178123.627362:0:15213:0:(service.c:2324:ptlrpc_server_handle_request()) Handling RPC req@00000000f0ef812c pname:cluuid+ref:pid:xid:nid:opc:job mdt_out00_003:lustre-MDT0001-mdtlov_UUID+6:3353:x1862456998026048:12345-0@lo:41:osp-pre-0-1.0 00000004:00000024:1.0:1776178123.627396:0:15213:0:(mdt_handler.c:548:mdt_statfs()) blocks cached 0 granted 2097152 pending 0 free 1446121472 avail 1316659200 00000100:00100000:1.0:1776178123.627439:0:15213:0:(service.c:2372:ptlrpc_server_handle_request()) Handled RPC req@00000000f0ef812c pname:cluuid+ref:pid:xid:nid:opc:job mdt_out00_003:lustre-MDT0001-mdtlov_UUID+6:3353:x1862456998026048:12345-0@lo:41:osp-pre-0-1.0 Request processed in 75us (10969us total) trans 0 rc 0/0 00000100:00100000:1.0:1776178123.627446:0:15213:0:(nrs_fifo.c:241:nrs_fifo_req_stop()) NRS stop fifo request from 12345-0@lo, seq: 628 00000100:00100000:1.0:1776178123.627463:0:15213:0:(nrs_fifo.c:179:nrs_fifo_req_get()) NRS start fifo request from 12345-0@lo, seq: 629 00000100:00100000:1.0:1776178123.627466:0:15213:0:(service.c:2324:ptlrpc_server_handle_request()) Handling RPC req@000000005927b2c3 pname:cluuid+ref:pid:xid:nid:opc:job mdt_out00_003:lustre-MDT0000-mdtlov_UUID+5:3353:x1862456998027008:12345-0@lo:41:osp-pre-1-0.0 00000004:00000024:1.0:1776178123.627482:0:15213:0:(mdt_handler.c:548:mdt_statfs()) blocks cached 0 granted 2097152 pending 0 free 1446039552 avail 1316577280 00000100:00100000:1.0:1776178123.628254:0:15213:0:(service.c:2372:ptlrpc_server_handle_request()) Handled RPC req@000000005927b2c3 pname:cluuid+ref:pid:xid:nid:opc:job mdt_out00_003:lustre-MDT0000-mdtlov_UUID+5:3353:x1862456998027008:12345-0@lo:41:osp-pre-1-0.0 Request processed in 784us (1305us total) trans 0 rc 0/0 00000100:00100000:1.0:1776178123.628263:0:15213:0:(nrs_fifo.c:241:nrs_fifo_req_stop()) NRS stop fifo request from 12345-0@lo, seq: 629 00000100:00100000:1.0:1776178123.628321:0:3353:0:(client.c:2215:ptlrpc_check_set()) Completed RPC req@000000008b858f58 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_00:lustre-MDT0000-mdtlov_UUID:3353:1862456998027008:0@lo:41:osp-pre-1-0.0 00000100:00100000:1.0:1776178123.628336:0:3353:0:(client.c:2215:ptlrpc_check_set()) Completed RPC req@00000000228025f2 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_00:lustre-MDT0001-mdtlov_UUID:3353:1862456998026048:0@lo:41:osp-pre-0-1.0 00000100:00100000:1.0:1776178123.628385:0:6255:0:(service.c:2152:ptlrpc_server_handle_req_in()) got req x1862456998026240 00000100:00100000:1.0:1776178123.628418:0:6255:0:(nrs_fifo.c:179:nrs_fifo_req_get()) NRS start fifo request from 12345-0@lo, seq: 779 00000100:00100000:1.0:1776178123.628423:0:6255:0:(service.c:2324:ptlrpc_server_handle_request()) Handling RPC req@0000000047458d9f pname:cluuid+ref:pid:xid:nid:opc:job mdt00_000:lustre-MDT0001-lwp-OST0000_UUID+5:3353:x1862456998026240:12345-0@lo:400:kworker/u8:2.0 00000100:00100000:1.0:1776178123.628462:0:6255:0:(service.c:2372:ptlrpc_server_handle_request()) Handled RPC req@0000000047458d9f pname:cluuid+ref:pid:xid:nid:opc:job mdt00_000:lustre-MDT0001-lwp-OST0000_UUID+5:3353:x1862456998026240:12345-0@lo:400:kworker/u8:2.0 Request processed in 38us (6522us total) trans 0 rc 0/0 00000100:00100000:1.0:1776178123.628468:0:6255:0:(nrs_fifo.c:241:nrs_fifo_req_stop()) NRS stop fifo request from 12345-0@lo, seq: 779 00000100:00100000:1.0:1776178123.628488:0:8227:0:(service.c:2152:ptlrpc_server_handle_req_in()) got req x1862456998026496 00000100:00100000:1.0:1776178123.628945:0:8227:0:(service.c:2152:ptlrpc_server_handle_req_in()) got req x1862456998026752 00000100:00100000:1.0:1776178123.628970:0:8227:0:(nrs_fifo.c:179:nrs_fifo_req_get()) NRS start fifo request from 12345-0@lo, seq: 424 00000100:00100000:1.0:1776178123.628973:0:8227:0:(service.c:2324:ptlrpc_server_handle_request()) Handling RPC req@00000000ecb64602 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_000:lustre-MDT0001-mdtlov_UUID+5:3353:x1862456998026496:12345-0@lo:400:kworker/u8:2.0 00000100:00100000:1.0:1776178123.629006:0:8227:0:(service.c:2372:ptlrpc_server_handle_request()) Handled RPC req@00000000ecb64602 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_000:lustre-MDT0001-mdtlov_UUID+5:3353:x1862456998026496:12345-0@lo:400:kworker/u8:2.0 Request processed in 32us (6951us total) trans 0 rc 0/0 00000100:00100000:1.0:1776178123.629011:0:8227:0:(nrs_fifo.c:241:nrs_fifo_req_stop()) NRS stop fifo request from 12345-0@lo, seq: 424 00000100:00100000:1.0:1776178123.629023:0:8227:0:(nrs_fifo.c:179:nrs_fifo_req_get()) NRS start fifo request from 12345-0@lo, seq: 425 00000100:00100000:1.0:1776178123.629026:0:8227:0:(service.c:2324:ptlrpc_server_handle_request()) Handling RPC req@000000005643ab1b pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_000:lustre-MDT0000-mdtlov_UUID+5:3353:x1862456998026752:12345-0@lo:400:kworker/u8:2.0 00000100:00100000:1.0:1776178123.629050:0:8227:0:(service.c:2372:ptlrpc_server_handle_request()) Handled RPC req@000000005643ab1b pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_000:lustre-MDT0000-mdtlov_UUID+5:3353:x1862456998026752:12345-0@lo:400:kworker/u8:2.0 Request processed in 23us (3012us total) trans 0 rc 0/0 00000100:00100000:1.0:1776178123.629056:0:8227:0:(nrs_fifo.c:241:nrs_fifo_req_stop()) NRS stop fifo request from 12345-0@lo, seq: 425 00000100:00100000:1.0:1776178123.629109:0:3353:0:(client.c:2215:ptlrpc_check_set()) Completed RPC req@00000000a6227918 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_00:lustre-MDT0001-lwp-OST0000_UUID:3353:1862456998026240:0@lo:400:kworker/u8:2.0 00000100:00100000:1.0:1776178123.629120:0:3353:0:(client.c:2215:ptlrpc_check_set()) Completed RPC req@0000000024989815 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_00:lustre-MDT0001-mdtlov_UUID:3353:1862456998026496:0@lo:400:kworker/u8:2.0 00000100:00100000:1.0:1776178123.629140:0:3353:0:(client.c:2215:ptlrpc_check_set()) Completed RPC req@000000007d60c0e9 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_00:lustre-MDT0000-mdtlov_UUID:3353:1862456998026752:0@lo:400:kworker/u8:2.0 00000100:00100000:3.0:1776178123.629361:0:8231:0:(service.c:2152:ptlrpc_server_handle_req_in()) got req x1862456998027072 00000100:00100000:3.0:1776178123.629393:0:8231:0:(nrs_fifo.c:179:nrs_fifo_req_get()) NRS start fifo request from 12345-0@lo, seq: 415 00000100:00100000:3.0:1776178123.629397:0:8231:0:(service.c:2324:ptlrpc_server_handle_request()) Handling RPC req@000000009e7a8bb5 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_create00:lustre-MDT0000-mdtlov_UUID+5:3353:x1862456998027072:12345-0@lo:13:osp-pre-1-0.0 00002000:00000024:3.0:1776178123.629418:0:8231:0:(ofd_obd.c:740:ofd_statfs()) blocks cached 229376 granted 491746368 pending 0 free 3923689472 avail 3692167168 00000100:00100000:3.0:1776178123.629452:0:8231:0:(service.c:2372:ptlrpc_server_handle_request()) Handled RPC req@000000009e7a8bb5 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_create00:lustre-MDT0000-mdtlov_UUID+5:3353:x1862456998027072:12345-0@lo:13:osp-pre-1-0.0 Request processed in 54us (2431us total) trans 0 rc 0/0 00000100:00100000:3.0:1776178123.629458:0:8231:0:(nrs_fifo.c:241:nrs_fifo_req_stop()) NRS stop fifo request from 12345-0@lo, seq: 415 00000100:00100000:3.0:1776178123.629497:0:3356:0:(events.c:361:request_in_callback()) peer: 12345-0@lo (source: 12345-0@lo) 00000100:00100000:3.0:1776178123.629540:0:3356:0:(client.c:1733:ptlrpc_send_new_req()) Sending RPC req@00000000ed90ff75 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_03:lustre-MDT0001-mdtlov_UUID:3356:1862456998026432:0@lo:400:kworker/u8:2.0 00000100:00100000:3.0:1776178123.629566:0:3356:0:(events.c:361:request_in_callback()) peer: 12345-0@lo (source: 12345-0@lo) 00000100:00100000:3.0:1776178123.629579:0:3356:0:(client.c:1733:ptlrpc_send_new_req()) Sending RPC req@0000000049b1b339 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_03:lustre-MDT0000-mdtlov_UUID:3356:1862456998026688:0@lo:400:kworker/u8:2.0 00000100:00100000:3.0:1776178123.629597:0:3356:0:(events.c:361:request_in_callback()) peer: 12345-0@lo (source: 12345-0@lo) 00000100:00100000:0.0:1776178123.630105:0:3354:0:(events.c:361:request_in_callback()) peer: 12345-0@lo (source: 12345-0@lo) 00000100:00100000:0.0:1776178123.630127:0:3354:0:(client.c:1733:ptlrpc_send_new_req()) Sending RPC req@0000000039ff8335 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_01:lustre-MDT0001-mdtlov_UUID:3354:1862456998026560:0@lo:400:kworker/u8:2.0 00000100:00100000:0.0:1776178123.630151:0:3354:0:(events.c:361:request_in_callback()) peer: 12345-0@lo (source: 12345-0@lo) 00000100:00100000:0.0:1776178123.630173:0:3354:0:(client.c:1733:ptlrpc_send_new_req()) Sending RPC req@00000000500c0ae6 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_01:lustre-MDT0000-mdtlov_UUID:3354:1862456998026816:0@lo:400:kworker/u8:2.0 00000100:00100000:0.0:1776178123.630193:0:3354:0:(events.c:361:request_in_callback()) peer: 12345-0@lo (source: 12345-0@lo) 00000100:00100000:0.0:1776178123.630228:0:11281:0:(service.c:2152:ptlrpc_server_handle_req_in()) got req x1862456998026176 00000100:00100000:0.0:1776178123.630264:0:11281:0:(service.c:2152:ptlrpc_server_handle_req_in()) got req x1862456998026304 00000100:00100000:0.0:1776178123.630283:0:11281:0:(nrs_fifo.c:179:nrs_fifo_req_get()) NRS start fifo request from 12345-0@lo, seq: 780 00000100:00100000:0.0:1776178123.630286:0:11281:0:(service.c:2324:ptlrpc_server_handle_request()) Handling RPC req@000000008d6accd5 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_004:lustre-MDT0000-lwp-OST0000_UUID+5:3356:x1862456998026176:12345-0@lo:400:kworker/u8:2.0 00000100:00100000:0.0:1776178123.630322:0:11281:0:(service.c:2372:ptlrpc_server_handle_request()) Handled RPC req@000000008d6accd5 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_004:lustre-MDT0000-lwp-OST0000_UUID+5:3356:x1862456998026176:12345-0@lo:400:kworker/u8:2.0 Request processed in 35us (825us total) trans 0 rc 0/0 00000100:00100000:0.0:1776178123.630328:0:11281:0:(nrs_fifo.c:241:nrs_fifo_req_stop()) NRS stop fifo request from 12345-0@lo, seq: 780 00000100:00100000:0.0:1776178123.630341:0:11281:0:(nrs_fifo.c:179:nrs_fifo_req_get()) NRS start fifo request from 12345-0@lo, seq: 781 00000100:00100000:0.0:1776178123.630344:0:11281:0:(service.c:2324:ptlrpc_server_handle_request()) Handling RPC req@000000000541929f pname:cluuid+ref:pid:xid:nid:opc:job mdt00_004:lustre-MDT0000-lwp-OST0001_UUID+5:3354:x1862456998026304:12345-0@lo:400:kworker/u8:2.0 00000100:00100000:0.0:1776178123.630373:0:11281:0:(service.c:2372:ptlrpc_server_handle_request()) Handled RPC req@000000000541929f pname:cluuid+ref:pid:xid:nid:opc:job mdt00_004:lustre-MDT0000-lwp-OST0001_UUID+5:3354:x1862456998026304:12345-0@lo:400:kworker/u8:2.0 Request processed in 28us (269us total) trans 0 rc 0/0 00000100:00100000:0.0:1776178123.630378:0:11281:0:(nrs_fifo.c:241:nrs_fifo_req_stop()) NRS stop fifo request from 12345-0@lo, seq: 781 00000100:00100000:0.0:1776178123.630397:0:20769:0:(service.c:2152:ptlrpc_server_handle_req_in()) got req x1862456998026560 00000100:00100000:0.0:1776178123.630415:0:20769:0:(service.c:2152:ptlrpc_server_handle_req_in()) got req x1862456998026816 00000100:00100000:0.0:1776178123.630433:0:20769:0:(nrs_fifo.c:179:nrs_fifo_req_get()) NRS start fifo request from 12345-0@lo, seq: 426 00000100:00100000:0.0:1776178123.630436:0:20769:0:(service.c:2324:ptlrpc_server_handle_request()) Handling RPC req@00000000002be227 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_018:lustre-MDT0001-mdtlov_UUID+5:3354:x1862456998026560:12345-0@lo:400:kworker/u8:2.0 00000100:00100000:0.0:1776178123.630462:0:20769:0:(service.c:2372:ptlrpc_server_handle_request()) Handled RPC req@00000000002be227 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_018:lustre-MDT0001-mdtlov_UUID+5:3354:x1862456998026560:12345-0@lo:400:kworker/u8:2.0 Request processed in 25us (310us total) trans 0 rc 0/0 00000100:00100000:0.0:1776178123.630466:0:20769:0:(nrs_fifo.c:241:nrs_fifo_req_stop()) NRS stop fifo request from 12345-0@lo, seq: 426 00000100:00100000:0.0:1776178123.630476:0:20769:0:(nrs_fifo.c:179:nrs_fifo_req_get()) NRS start fifo request from 12345-0@lo, seq: 427 00000100:00100000:0.0:1776178123.630478:0:20769:0:(service.c:2324:ptlrpc_server_handle_request()) Handling RPC req@00000000a8c8cb2d pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_018:lustre-MDT0000-mdtlov_UUID+5:3354:x1862456998026816:12345-0@lo:400:kworker/u8:2.0 00000100:00100000:0.0:1776178123.630497:0:20769:0:(service.c:2372:ptlrpc_server_handle_request()) Handled RPC req@00000000a8c8cb2d pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_018:lustre-MDT0000-mdtlov_UUID+5:3354:x1862456998026816:12345-0@lo:400:kworker/u8:2.0 Request processed in 19us (305us total) trans 0 rc 0/0 00000100:00100000:0.0:1776178123.630502:0:20769:0:(nrs_fifo.c:241:nrs_fifo_req_stop()) NRS stop fifo request from 12345-0@lo, seq: 427 00000100:00100000:0.0:1776178123.630559:0:3354:0:(client.c:2215:ptlrpc_check_set()) Completed RPC req@00000000f0f1167e pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_01:lustre-MDT0000-lwp-OST0001_UUID:3354:1862456998026304:0@lo:400:kworker/u8:2.0 00000100:00100000:0.0:1776178123.630577:0:3354:0:(client.c:2215:ptlrpc_check_set()) Completed RPC req@0000000039ff8335 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_01:lustre-MDT0001-mdtlov_UUID:3354:1862456998026560:0@lo:400:kworker/u8:2.0 00000100:00100000:0.0:1776178123.630584:0:3354:0:(client.c:2215:ptlrpc_check_set()) Completed RPC req@00000000500c0ae6 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_01:lustre-MDT0000-mdtlov_UUID:3354:1862456998026816:0@lo:400:kworker/u8:2.0 00000100:00100000:0.0:1776178123.630625:0:6261:0:(service.c:2152:ptlrpc_server_handle_req_in()) got req x1862456998026432 00000100:00100000:0.0:1776178123.630643:0:6261:0:(service.c:2152:ptlrpc_server_handle_req_in()) got req x1862456998026688 00000100:00100000:0.0:1776178123.630655:0:6261:0:(nrs_fifo.c:179:nrs_fifo_req_get()) NRS start fifo request from 12345-0@lo, seq: 630 00000100:00100000:0.0:1776178123.630659:0:6261:0:(service.c:2324:ptlrpc_server_handle_request()) Handling RPC req@000000005b125149 pname:cluuid+ref:pid:xid:nid:opc:job mdt_out00_001:lustre-MDT0001-mdtlov_UUID+6:3356:x1862456998026432:12345-0@lo:400:kworker/u8:2.0 00000100:00100000:0.0:1776178123.630685:0:6261:0:(service.c:2372:ptlrpc_server_handle_request()) Handled RPC req@000000005b125149 pname:cluuid+ref:pid:xid:nid:opc:job mdt_out00_001:lustre-MDT0001-mdtlov_UUID+6:3356:x1862456998026432:12345-0@lo:400:kworker/u8:2.0 Request processed in 26us (1120us total) trans 0 rc 0/0 00000100:00100000:0.0:1776178123.630690:0:6261:0:(nrs_fifo.c:241:nrs_fifo_req_stop()) NRS stop fifo request from 12345-0@lo, seq: 630 00000100:00100000:0.0:1776178123.630713:0:6261:0:(nrs_fifo.c:179:nrs_fifo_req_get()) NRS start fifo request from 12345-0@lo, seq: 631 00000100:00100000:0.0:1776178123.630716:0:6261:0:(service.c:2324:ptlrpc_server_handle_request()) Handling RPC req@000000005accf721 pname:cluuid+ref:pid:xid:nid:opc:job mdt_out00_001:lustre-MDT0000-mdtlov_UUID+5:3356:x1862456998026688:12345-0@lo:400:kworker/u8:2.0 00000100:00100000:0.0:1776178123.630739:0:6261:0:(service.c:2372:ptlrpc_server_handle_request()) Handled RPC req@000000005accf721 pname:cluuid+ref:pid:xid:nid:opc:job mdt_out00_001:lustre-MDT0000-mdtlov_UUID+5:3356:x1862456998026688:12345-0@lo:400:kworker/u8:2.0 Request processed in 22us (1141us total) trans 0 rc 0/0 00000100:00100000:0.0:1776178123.630744:0:6261:0:(nrs_fifo.c:241:nrs_fifo_req_stop()) NRS stop fifo request from 12345-0@lo, seq: 631 00000100:00100000:3.0:1776178123.631747:0:3356:0:(client.c:1733:ptlrpc_send_new_req()) Sending RPC req@00000000a9cfe89c pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_03:lustre-MDT0000-mdtlov_UUID:3356:1862456998026944:0@lo:13:osp-pre-0-0.0 00000100:00100000:3.0:1776178123.631799:0:3356:0:(events.c:361:request_in_callback()) peer: 12345-0@lo (source: 12345-0@lo) 00000100:00100000:3.0:1776178123.631837:0:3356:0:(client.c:2215:ptlrpc_check_set()) Completed RPC req@0000000022d4dc29 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_03:lustre-MDT0000-lwp-OST0000_UUID:3356:1862456998026176:0@lo:400:kworker/u8:2.0 00000100:00100000:3.0:1776178123.631846:0:3356:0:(client.c:2215:ptlrpc_check_set()) Completed RPC req@00000000ed90ff75 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_03:lustre-MDT0001-mdtlov_UUID:3356:1862456998026432:0@lo:400:kworker/u8:2.0 00000100:00100000:3.0:1776178123.631853:0:3356:0:(client.c:2215:ptlrpc_check_set()) Completed RPC req@0000000049b1b339 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_03:lustre-MDT0000-mdtlov_UUID:3356:1862456998026688:0@lo:400:kworker/u8:2.0 00000100:00100000:0.0:1776178123.632290:0:8232:0:(service.c:2152:ptlrpc_server_handle_req_in()) got req x1862456998026944 00000100:00100000:0.0:1776178123.632330:0:8232:0:(nrs_fifo.c:179:nrs_fifo_req_get()) NRS start fifo request from 12345-0@lo, seq: 416 00000100:00100000:0.0:1776178123.632334:0:8232:0:(service.c:2324:ptlrpc_server_handle_request()) Handling RPC req@00000000ee45921e pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_create00:lustre-MDT0000-mdtlov_UUID+5:3356:x1862456998026944:12345-0@lo:13:osp-pre-0-0.0 00002000:00000024:0.0:1776178123.632359:0:8232:0:(ofd_obd.c:740:ofd_statfs()) blocks cached 315392 granted 484883776 pending 0 free 3923689472 avail 3692167168 00000100:00100000:0.0:1776178123.632398:0:8232:0:(service.c:2372:ptlrpc_server_handle_request()) Handled RPC req@00000000ee45921e pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_create00:lustre-MDT0000-mdtlov_UUID+5:3356:x1862456998026944:12345-0@lo:13:osp-pre-0-0.0 Request processed in 63us (598us total) trans 0 rc 0/0 00000100:00100000:0.0:1776178123.632404:0:8232:0:(nrs_fifo.c:241:nrs_fifo_req_stop()) NRS stop fifo request from 12345-0@lo, seq: 416 00000100:00100000:3.0:1776178123.632484:0:3356:0:(client.c:2215:ptlrpc_check_set()) Completed RPC req@00000000a9cfe89c pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_03:lustre-MDT0000-mdtlov_UUID:3356:1862456998026944:0@lo:13:osp-pre-0-0.0 00000100:00100000:1.0:1776178123.637210:0:3353:0:(client.c:2215:ptlrpc_check_set()) Completed RPC req@0000000031795589 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_00:lustre-MDT0000-mdtlov_UUID:3353:1862456998027072:0@lo:13:osp-pre-1-0.0 00000100:00100000:2.0:1776178123.638183:0:3355:0:(events.c:361:request_in_callback()) peer: 12345-0@lo (source: 12345-0@lo) 00000100:00100000:2.0:1776178123.638211:0:3355:0:(client.c:1733:ptlrpc_send_new_req()) Sending RPC req@00000000eb400454 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_02:lustre-MDT0001-lwp-OST0001_UUID:3355:1862456998026368:0@lo:400:kworker/u8:2.0 00000100:00100000:2.0:1776178123.638252:0:3355:0:(events.c:361:request_in_callback()) peer: 12345-0@lo (source: 12345-0@lo) 00000100:00100000:2.0:1776178123.638284:0:3355:0:(client.c:1733:ptlrpc_send_new_req()) Sending RPC req@0000000056ddcb09 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_02:lustre-MDT0000-lwp-MDT0001_UUID:3355:1862456998026624:0@lo:400:kworker/u8:2.0 00000100:00100000:2.0:1776178123.638305:0:3355:0:(events.c:361:request_in_callback()) peer: 12345-0@lo (source: 12345-0@lo) 00000100:00100000:2.0:1776178123.638317:0:3355:0:(client.c:1733:ptlrpc_send_new_req()) Sending RPC req@000000000154171f pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_02:lustre-MDT0000-lwp-MDT0000_UUID:3355:1862456998026880:0@lo:400:kworker/u8:2.0 00000100:00100000:2.0:1776178123.638335:0:3355:0:(events.c:361:request_in_callback()) peer: 12345-0@lo (source: 12345-0@lo) 00000100:00100000:3.0:1776178123.639287:0:8250:0:(service.c:2152:ptlrpc_server_handle_req_in()) got req x1862456998026368 00000100:00100000:0.0:1776178123.639307:0:6256:0:(service.c:2152:ptlrpc_server_handle_req_in()) got req x1862456998026624 00000100:00100000:1.0:1776178123.639378:0:20137:0:(service.c:2152:ptlrpc_server_handle_req_in()) got req x1862456998026112 00000100:00100000:1.0:1776178123.639412:0:20137:0:(nrs_fifo.c:179:nrs_fifo_req_get()) NRS start fifo request from 12345-0@lo, seq: 123 00000100:00100000:1.0:1776178123.639418:0:20137:0:(service.c:2324:ptlrpc_server_handle_request()) Handling RPC req@0000000044c7032a pname:cluuid+ref:pid:xid:nid:opc:job ll_mgs_0000:ac61f839-0e8e-4cac-9c59-08bcfbbad330+23:3355:x1862456998026112:12345-0@lo:400:kworker/u8:2.0 00000100:00100000:1.0:1776178123.639475:0:20137:0:(service.c:2372:ptlrpc_server_handle_request()) Handled RPC req@0000000044c7032a pname:cluuid+ref:pid:xid:nid:opc:job ll_mgs_0000:ac61f839-0e8e-4cac-9c59-08bcfbbad330+23:3355:x1862456998026112:12345-0@lo:400:kworker/u8:2.0 Request processed in 56us (1295us total) trans 0 rc 0/0 00000100:00100000:1.0:1776178123.639482:0:20137:0:(nrs_fifo.c:241:nrs_fifo_req_stop()) NRS stop fifo request from 12345-0@lo, seq: 123 00000100:00100000:0.0:1776178123.639684:0:6256:0:(service.c:2152:ptlrpc_server_handle_req_in()) got req x1862456998026880 00000100:00100000:1.0:1776178123.645434:0:6255:0:(nrs_fifo.c:179:nrs_fifo_req_get()) NRS start fifo request from 12345-0@lo, seq: 782 00000100:00100000:1.0:1776178123.645449:0:6255:0:(service.c:2324:ptlrpc_server_handle_request()) Handling RPC req@0000000031795589 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_000:lustre-MDT0000-lwp-MDT0001_UUID+5:3355:x1862456998026624:12345-0@lo:400:kworker/u8:2.0 00000100:00100000:1.0:1776178123.645572:0:6255:0:(service.c:2372:ptlrpc_server_handle_request()) Handled RPC req@0000000031795589 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_000:lustre-MDT0000-lwp-MDT0001_UUID+5:3355:x1862456998026624:12345-0@lo:400:kworker/u8:2.0 Request processed in 125us (7266us total) trans 0 rc 0/0 00000100:00100000:1.0:1776178123.645580:0:6255:0:(nrs_fifo.c:241:nrs_fifo_req_stop()) NRS stop fifo request from 12345-0@lo, seq: 782 00000100:00100000:1.0:1776178123.645607:0:6255:0:(nrs_fifo.c:179:nrs_fifo_req_get()) NRS start fifo request from 12345-0@lo, seq: 783 00000100:00100000:1.0:1776178123.645611:0:6255:0:(service.c:2324:ptlrpc_server_handle_request()) Handling RPC req@00000000de27a229 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_000:lustre-MDT0001-lwp-OST0001_UUID+5:3355:x1862456998026368:12345-0@lo:400:kworker/u8:2.0 00000100:00100000:1.0:1776178123.645640:0:6255:0:(service.c:2372:ptlrpc_server_handle_request()) Handled RPC req@00000000de27a229 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_000:lustre-MDT0001-lwp-OST0001_UUID+5:3355:x1862456998026368:12345-0@lo:400:kworker/u8:2.0 Request processed in 28us (7388us total) trans 0 rc 0/0 00000100:00100000:1.0:1776178123.645645:0:6255:0:(nrs_fifo.c:241:nrs_fifo_req_stop()) NRS stop fifo request from 12345-0@lo, seq: 783 00000100:00100000:1.0:1776178123.645658:0:6255:0:(nrs_fifo.c:179:nrs_fifo_req_get()) NRS start fifo request from 12345-0@lo, seq: 784 00000100:00100000:1.0:1776178123.645661:0:6255:0:(service.c:2324:ptlrpc_server_handle_request()) Handling RPC req@00000000228025f2 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_000:lustre-MDT0000-lwp-MDT0000_UUID+5:3355:x1862456998026880:12345-0@lo:400:kworker/u8:2.0 00000100:00100000:1.0:1776178123.645687:0:6255:0:(service.c:2372:ptlrpc_server_handle_request()) Handled RPC req@00000000228025f2 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_000:lustre-MDT0000-lwp-MDT0000_UUID+5:3355:x1862456998026880:12345-0@lo:400:kworker/u8:2.0 Request processed in 24us (7351us total) trans 0 rc 0/0 00000100:00100000:1.0:1776178123.645692:0:6255:0:(nrs_fifo.c:241:nrs_fifo_req_stop()) NRS stop fifo request from 12345-0@lo, seq: 784 00000100:00100000:3.0:1776178123.646030:0:3355:0:(client.c:2215:ptlrpc_check_set()) Completed RPC req@00000000699e66b5 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_02:ac61f839-0e8e-4cac-9c59-08bcfbbad330:3355:1862456998026112:0@lo:400:kworker/u8:2.0 00000100:00100000:3.0:1776178123.646055:0:3355:0:(client.c:2215:ptlrpc_check_set()) Completed RPC req@00000000eb400454 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_02:lustre-MDT0001-lwp-OST0001_UUID:3355:1862456998026368:0@lo:400:kworker/u8:2.0 00000100:00100000:3.0:1776178123.646084:0:3355:0:(client.c:2215:ptlrpc_check_set()) Completed RPC req@0000000056ddcb09 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_02:lustre-MDT0000-lwp-MDT0001_UUID:3355:1862456998026624:0@lo:400:kworker/u8:2.0 00000100:00100000:3.0:1776178123.646095:0:3355:0:(client.c:2215:ptlrpc_check_set()) Completed RPC req@000000000154171f pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_02:lustre-MDT0000-lwp-MDT0000_UUID:3355:1862456998026880:0@lo:400:kworker/u8:2.0 00010000:00010000:3.0:1776178124.192325:0:9225:0:(ldlm_lockd.c:2792:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:3.0:1776178124.584639:0:3346:0:(events.c:361:request_in_callback()) peer: 12345-192.168.204.3@tcp (source: 12345-192.168.204.3@tcp) 00000100:00100000:3.0:1776178124.584689:0:3346:0:(events.c:361:request_in_callback()) peer: 12345-192.168.204.3@tcp (source: 12345-192.168.204.3@tcp) 00000100:00100000:3.0:1776178124.584737:0:3346:0:(events.c:361:request_in_callback()) peer: 12345-192.168.204.3@tcp (source: 12345-192.168.204.3@tcp) 00000100:00100000:3.0:1776178124.584769:0:3346:0:(events.c:361:request_in_callback()) peer: 12345-192.168.204.3@tcp (source: 12345-192.168.204.3@tcp) 00000100:00100000:3.0:1776178124.584795:0:3346:0:(events.c:361:request_in_callback()) peer: 12345-192.168.204.3@tcp (source: 12345-192.168.204.3@tcp) 00000100:00100000:3.0:1776178124.584836:0:20759:0:(service.c:2152:ptlrpc_server_handle_req_in()) got req x1862456979083328 00000100:00100000:3.0:1776178124.584885:0:20759:0:(service.c:2152:ptlrpc_server_handle_req_in()) got req x1862456979083392 00000100:00100000:3.0:1776178124.584907:0:20759:0:(nrs_fifo.c:179:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.204.3@tcp, seq: 428 00000100:00100000:3.0:1776178124.584914:0:20759:0:(service.c:2324:ptlrpc_server_handle_request()) Handling RPC req@00000000a7e4e4e8 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_010:b50963bd-8eba-4432-9a54-8888d00e3fa1+9:2270:x1862456979083328:12345-192.168.204.3@tcp:400:kworker/u8:3.0 00000100:00100000:3.0:1776178124.584982:0:20759:0:(service.c:2372:ptlrpc_server_handle_request()) Handled RPC req@00000000a7e4e4e8 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_010:b50963bd-8eba-4432-9a54-8888d00e3fa1+9:2270:x1862456979083328:12345-192.168.204.3@tcp:400:kworker/u8:3.0 Request processed in 68us (293us total) trans 0 rc 0/0 00000100:00100000:3.0:1776178124.584989:0:20759:0:(nrs_fifo.c:241:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.204.3@tcp, seq: 428 00000100:00100000:3.0:1776178124.585005:0:20759:0:(nrs_fifo.c:179:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.204.3@tcp, seq: 429 00000100:00100000:3.0:1776178124.585009:0:20759:0:(service.c:2324:ptlrpc_server_handle_request()) Handling RPC req@00000000d0706d7c pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_010:b50963bd-8eba-4432-9a54-8888d00e3fa1+8:2270:x1862456979083392:12345-192.168.204.3@tcp:400:kworker/u8:3.0 00000100:00100000:3.0:1776178124.585033:0:20759:0:(service.c:2372:ptlrpc_server_handle_request()) Handled RPC req@00000000d0706d7c pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_010:b50963bd-8eba-4432-9a54-8888d00e3fa1+8:2270:x1862456979083392:12345-192.168.204.3@tcp:400:kworker/u8:3.0 Request processed in 23us (239us total) trans 0 rc 0/0 00000100:00100000:3.0:1776178124.585043:0:20759:0:(nrs_fifo.c:241:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.204.3@tcp, seq: 429 00000100:00100000:2.0:1776178124.585086:0:6257:0:(service.c:2152:ptlrpc_server_handle_req_in()) got req x1862456979083200 00000100:00100000:2.0:1776178124.585115:0:6257:0:(service.c:2152:ptlrpc_server_handle_req_in()) got req x1862456979083264 00000100:00100000:2.0:1776178124.585132:0:6257:0:(nrs_fifo.c:179:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.204.3@tcp, seq: 785 00000100:00100000:2.0:1776178124.585136:0:6257:0:(service.c:2324:ptlrpc_server_handle_request()) Handling RPC req@00000000a9cfe89c pname:cluuid+ref:pid:xid:nid:opc:job mdt00_002:b50963bd-8eba-4432-9a54-8888d00e3fa1+21:2270:x1862456979083200:12345-192.168.204.3@tcp:400:kworker/u8:3.0 00000100:00100000:2.0:1776178124.585164:0:6257:0:(service.c:2372:ptlrpc_server_handle_request()) Handled RPC req@00000000a9cfe89c pname:cluuid+ref:pid:xid:nid:opc:job mdt00_002:b50963bd-8eba-4432-9a54-8888d00e3fa1+21:2270:x1862456979083200:12345-192.168.204.3@tcp:400:kworker/u8:3.0 Request processed in 26us (534us total) trans 0 rc 0/0 00000100:00100000:2.0:1776178124.585170:0:6257:0:(nrs_fifo.c:241:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.204.3@tcp, seq: 785 00000100:00100000:2.0:1776178124.585182:0:6257:0:(nrs_fifo.c:179:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.204.3@tcp, seq: 786 00000100:00100000:2.0:1776178124.585185:0:6257:0:(service.c:2324:ptlrpc_server_handle_request()) Handling RPC req@0000000018082495 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_002:b50963bd-8eba-4432-9a54-8888d00e3fa1+6:2270:x1862456979083264:12345-192.168.204.3@tcp:400:kworker/u8:3.0 00000100:00100000:2.0:1776178124.585225:0:6257:0:(service.c:2372:ptlrpc_server_handle_request()) Handled RPC req@0000000018082495 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_002:b50963bd-8eba-4432-9a54-8888d00e3fa1+6:2270:x1862456979083264:12345-192.168.204.3@tcp:400:kworker/u8:3.0 Request processed in 27us (454us total) trans 0 rc 0/0 00000100:00100000:2.0:1776178124.585231:0:6257:0:(nrs_fifo.c:241:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.204.3@tcp, seq: 786 00000100:00100000:1.0:1776178124.585479:0:20140:0:(service.c:2152:ptlrpc_server_handle_req_in()) got req x1862456979083136 00000100:00100000:1.0:1776178124.585517:0:20140:0:(nrs_fifo.c:179:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.204.3@tcp, seq: 124 00000100:00100000:1.0:1776178124.585521:0:20140:0:(service.c:2324:ptlrpc_server_handle_request()) Handling RPC req@00000000c6a76119 pname:cluuid+ref:pid:xid:nid:opc:job ll_mgs_0002:5b5c66b1-1a2f-4651-8548-0c75b9a38a94+9:2270:x1862456979083136:12345-192.168.204.3@tcp:400:kworker/u8:3.0 00000100:00100000:1.0:1776178124.585553:0:20140:0:(service.c:2372:ptlrpc_server_handle_request()) Handled RPC req@00000000c6a76119 pname:cluuid+ref:pid:xid:nid:opc:job ll_mgs_0002:5b5c66b1-1a2f-4651-8548-0c75b9a38a94+9:2270:x1862456979083136:12345-192.168.204.3@tcp:400:kworker/u8:3.0 Request processed in 31us (817us total) trans 0 rc 0/0 00000100:00100000:1.0:1776178124.585559:0:20140:0:(nrs_fifo.c:241:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.204.3@tcp, seq: 124 00000100:00100000:0.0:1776178125.152279:0:3356:0:(ptlrpcd.c:415:ptlrpcd_check()) transfer 1 async RPCs [2->3] 00000100:00100000:0.0:1776178125.152308:0:3356:0:(client.c:1733:ptlrpc_send_new_req()) Sending RPC req@00000000bcefda18 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_03:lustre-MDT0001-mdtlov_UUID:3356:1862456998027136:0@lo:13:osp-pre-1-1.0 00000100:00100000:0.0:1776178125.152619:0:3356:0:(events.c:361:request_in_callback()) peer: 12345-0@lo (source: 12345-0@lo) 00000100:00100000:3.0:1776178125.152676:0:8231:0:(service.c:2152:ptlrpc_server_handle_req_in()) got req x1862456998027136 00000100:00100000:3.0:1776178125.152737:0:8231:0:(nrs_fifo.c:179:nrs_fifo_req_get()) NRS start fifo request from 12345-0@lo, seq: 417 00000100:00100000:3.0:1776178125.152743:0:8231:0:(service.c:2324:ptlrpc_server_handle_request()) Handling RPC req@0000000050ea472d pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_create00:lustre-MDT0001-mdtlov_UUID+5:3356:x1862456998027136:12345-0@lo:13:osp-pre-1-1.0 00002000:00000024:3.0:1776178125.152776:0:8231:0:(ofd_obd.c:740:ofd_statfs()) blocks cached 229376 granted 491746368 pending 0 free 3923689472 avail 3692167168 00000100:00100000:3.0:1776178125.152836:0:8231:0:(service.c:2372:ptlrpc_server_handle_request()) Handled RPC req@0000000050ea472d pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_create00:lustre-MDT0001-mdtlov_UUID+5:3356:x1862456998027136:12345-0@lo:13:osp-pre-1-1.0 Request processed in 92us (219us total) trans 0 rc 0/0 00000100:00100000:3.0:1776178125.152843:0:8231:0:(nrs_fifo.c:241:nrs_fifo_req_stop()) NRS stop fifo request from 12345-0@lo, seq: 417 00000100:00100000:1.0:1776178125.152951:0:3356:0:(client.c:2215:ptlrpc_check_set()) Completed RPC req@00000000bcefda18 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_03:lustre-MDT0001-mdtlov_UUID:3356:1862456998027136:0@lo:13:osp-pre-1-1.0 00010000:00010000:2.0:1776178125.216667:0:9224:0:(ldlm_lockd.c:2792:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:1.0:1776178125.664488:0:3356:0:(client.c:1733:ptlrpc_send_new_req()) Sending RPC req@000000008b858f58 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_03:lustre-MDT0001-mdtlov_UUID:3356:1862456998027200:0@lo:13:osp-pre-0-1.0 00000100:00100000:1.0:1776178125.664561:0:3356:0:(events.c:361:request_in_callback()) peer: 12345-0@lo (source: 12345-0@lo) 00000100:00100000:0.0:1776178125.666402:0:8232:0:(service.c:2152:ptlrpc_server_handle_req_in()) got req x1862456998027200 00000100:00100000:0.0:1776178125.666489:0:8232:0:(nrs_fifo.c:179:nrs_fifo_req_get()) NRS start fifo request from 12345-0@lo, seq: 418 00000100:00100000:0.0:1776178125.666496:0:8232:0:(service.c:2324:ptlrpc_server_handle_request()) Handling RPC req@0000000072865dff pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_create00:lustre-MDT0001-mdtlov_UUID+5:3356:x1862456998027200:12345-0@lo:13:osp-pre-0-1.0 00002000:00000024:0.0:1776178125.666533:0:8232:0:(ofd_obd.c:740:ofd_statfs()) blocks cached 315392 granted 484883776 pending 0 free 3923689472 avail 3692167168 00000100:00100000:0.0:1776178125.666615:0:8232:0:(service.c:2372:ptlrpc_server_handle_request()) Handled RPC req@0000000072865dff pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_create00:lustre-MDT0001-mdtlov_UUID+5:3356:x1862456998027200:12345-0@lo:13:osp-pre-0-1.0 Request processed in 119us (2055us total) trans 0 rc 0/0 00000100:00100000:0.0:1776178125.666623:0:8232:0:(nrs_fifo.c:241:nrs_fifo_req_stop()) NRS stop fifo request from 12345-0@lo, seq: 418 00000100:00100000:1.0:1776178125.667773:0:3356:0:(client.c:2215:ptlrpc_check_set()) Completed RPC req@000000008b858f58 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_03:lustre-MDT0001-mdtlov_UUID:3356:1862456998027200:0@lo:13:osp-pre-0-1.0 00010000:00010000:1.0:1776178126.243596:0:6244:0:(ldlm_lockd.c:2792:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00010000:00010000:1.0:1776178127.266834:0:6245:0:(ldlm_lockd.c:2792:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00010000:00010000:1.0:1776178128.289493:0:9225:0:(ldlm_lockd.c:2792:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:0.0:1776178128.736484:0:3354:0:(ptlrpcd.c:415:ptlrpcd_check()) transfer 1 async RPCs [0->1] 00000100:00100000:0.0:1776178128.736511:0:3354:0:(client.c:1733:ptlrpc_send_new_req()) Sending RPC req@00000000ddc1dc02 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_01:lustre-MDT0000-mdtlov_UUID:3354:1862456998027264:0@lo:41:osp-pre-1-0.0 00000100:00100000:0.0:1776178128.736602:0:3354:0:(events.c:361:request_in_callback()) peer: 12345-0@lo (source: 12345-0@lo) 00000100:00100000:1.0:1776178128.736978:0:3356:0:(client.c:1733:ptlrpc_send_new_req()) Sending RPC req@00000000a59bfc89 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_03:lustre-MDT0001-lwp-OST0000_UUID:3356:1862456998027456:0@lo:400:kworker/u8:0.0 00000100:00100000:1.0:1776178128.737011:0:3356:0:(events.c:361:request_in_callback()) peer: 12345-0@lo (source: 12345-0@lo) 00000100:00100000:1.0:1776178128.737032:0:3356:0:(client.c:1733:ptlrpc_send_new_req()) Sending RPC req@000000007d60c0e9 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_03:lustre-MDT0001-mdtlov_UUID:3356:1862456998027712:0@lo:400:kworker/u8:0.0 00000100:00100000:1.0:1776178128.737054:0:3356:0:(events.c:361:request_in_callback()) peer: 12345-0@lo (source: 12345-0@lo) 00000100:00100000:1.0:1776178128.737086:0:3356:0:(client.c:1733:ptlrpc_send_new_req()) Sending RPC req@0000000039ff8335 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_03:lustre-MDT0000-mdtlov_UUID:3356:1862456998027968:0@lo:400:kworker/u8:0.0 00000100:00100000:1.0:1776178128.737109:0:3356:0:(events.c:361:request_in_callback()) peer: 12345-0@lo (source: 12345-0@lo) 00000100:00100000:1.0:1776178128.737126:0:3356:0:(ptlrpcd.c:415:ptlrpcd_check()) transfer 3 async RPCs [2->3] 00000100:00100000:1.0:1776178128.737133:0:3356:0:(client.c:1733:ptlrpc_send_new_req()) Sending RPC req@00000000df370bee pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_03:lustre-MDT0000-lwp-OST0000_UUID:3356:1862456998027392:0@lo:400:kworker/u8:0.0 00000100:00100000:1.0:1776178128.737168:0:3356:0:(events.c:361:request_in_callback()) peer: 12345-0@lo (source: 12345-0@lo) 00000100:00100000:1.0:1776178128.737184:0:3356:0:(client.c:1733:ptlrpc_send_new_req()) Sending RPC req@00000000699e66b5 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_03:lustre-MDT0001-mdtlov_UUID:3356:1862456998027648:0@lo:400:kworker/u8:0.0 00000100:00100000:1.0:1776178128.737253:0:3356:0:(events.c:361:request_in_callback()) peer: 12345-0@lo (source: 12345-0@lo) 00000100:00100000:1.0:1776178128.737270:0:3356:0:(client.c:1733:ptlrpc_send_new_req()) Sending RPC req@0000000022d4dc29 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_03:lustre-MDT0000-mdtlov_UUID:3356:1862456998027904:0@lo:400:kworker/u8:0.0 00000100:00100000:1.0:1776178128.737291:0:3356:0:(events.c:361:request_in_callback()) peer: 12345-0@lo (source: 12345-0@lo) 00000100:00100000:1.0:1776178128.737323:0:11281:0:(service.c:2152:ptlrpc_server_handle_req_in()) got req x1862456998027456 00000100:00100000:1.0:1776178128.737384:0:11281:0:(service.c:2152:ptlrpc_server_handle_req_in()) got req x1862456998027392 00000100:00100000:1.0:1776178128.737402:0:11281:0:(nrs_fifo.c:179:nrs_fifo_req_get()) NRS start fifo request from 12345-0@lo, seq: 787 00000100:00100000:1.0:1776178128.737407:0:11281:0:(service.c:2324:ptlrpc_server_handle_request()) Handling RPC req@0000000024989815 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_004:lustre-MDT0001-lwp-OST0000_UUID+5:3356:x1862456998027456:12345-0@lo:400:kworker/u8:0.0 00000100:00100000:1.0:1776178128.737468:0:11281:0:(service.c:2372:ptlrpc_server_handle_request()) Handled RPC req@0000000024989815 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_004:lustre-MDT0001-lwp-OST0000_UUID+5:3356:x1862456998027456:12345-0@lo:400:kworker/u8:0.0 Request processed in 60us (457us total) trans 0 rc 0/0 00000100:00100000:1.0:1776178128.737476:0:11281:0:(nrs_fifo.c:241:nrs_fifo_req_stop()) NRS stop fifo request from 12345-0@lo, seq: 787 00000100:00100000:1.0:1776178128.737495:0:11281:0:(nrs_fifo.c:179:nrs_fifo_req_get()) NRS start fifo request from 12345-0@lo, seq: 788 00000100:00100000:1.0:1776178128.737498:0:11281:0:(service.c:2324:ptlrpc_server_handle_request()) Handling RPC req@00000000af9e3283 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_004:lustre-MDT0000-lwp-OST0000_UUID+5:3356:x1862456998027392:12345-0@lo:400:kworker/u8:0.0 00000100:00100000:1.0:1776178128.737522:0:11281:0:(service.c:2372:ptlrpc_server_handle_request()) Handled RPC req@00000000af9e3283 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_004:lustre-MDT0000-lwp-OST0000_UUID+5:3356:x1862456998027392:12345-0@lo:400:kworker/u8:0.0 Request processed in 23us (355us total) trans 0 rc 0/0 00000100:00100000:1.0:1776178128.737528:0:11281:0:(nrs_fifo.c:241:nrs_fifo_req_stop()) NRS stop fifo request from 12345-0@lo, seq: 788 00000100:00100000:1.0:1776178128.737548:0:20762:0:(service.c:2152:ptlrpc_server_handle_req_in()) got req x1862456998027712 00000100:00100000:1.0:1776178128.737568:0:20762:0:(service.c:2152:ptlrpc_server_handle_req_in()) got req x1862456998027968 00000100:00100000:1.0:1776178128.737584:0:20762:0:(service.c:2152:ptlrpc_server_handle_req_in()) got req x1862456998027904 00000100:00100000:3.0:1776178128.737945:0:14440:0:(nrs_fifo.c:179:nrs_fifo_req_get()) NRS start fifo request from 12345-0@lo, seq: 430 00000100:00100000:3.0:1776178128.737950:0:14440:0:(service.c:2324:ptlrpc_server_handle_request()) Handling RPC req@00000000a6227918 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_004:lustre-MDT0001-mdtlov_UUID+5:3356:x1862456998027712:12345-0@lo:400:kworker/u8:0.0 00000100:00100000:3.0:1776178128.737984:0:14440:0:(service.c:2372:ptlrpc_server_handle_request()) Handled RPC req@00000000a6227918 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_004:lustre-MDT0001-mdtlov_UUID+5:3356:x1862456998027712:12345-0@lo:400:kworker/u8:0.0 Request processed in 33us (929us total) trans 0 rc 0/0 00000100:00100000:3.0:1776178128.737990:0:14440:0:(nrs_fifo.c:241:nrs_fifo_req_stop()) NRS stop fifo request from 12345-0@lo, seq: 430 00000100:00100000:3.0:1776178128.738003:0:14440:0:(nrs_fifo.c:179:nrs_fifo_req_get()) NRS start fifo request from 12345-0@lo, seq: 431 00000100:00100000:3.0:1776178128.738006:0:14440:0:(service.c:2324:ptlrpc_server_handle_request()) Handling RPC req@000000003334dea7 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_004:lustre-MDT0000-mdtlov_UUID+5:3356:x1862456998027968:12345-0@lo:400:kworker/u8:0.0 00000100:00100000:3.0:1776178128.738031:0:14440:0:(service.c:2372:ptlrpc_server_handle_request()) Handled RPC req@000000003334dea7 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_004:lustre-MDT0000-mdtlov_UUID+5:3356:x1862456998027968:12345-0@lo:400:kworker/u8:0.0 Request processed in 24us (922us total) trans 0 rc 0/0 00000100:00100000:3.0:1776178128.738036:0:14440:0:(nrs_fifo.c:241:nrs_fifo_req_stop()) NRS stop fifo request from 12345-0@lo, seq: 431 00000100:00100000:3.0:1776178128.738102:0:3355:0:(client.c:1733:ptlrpc_send_new_req()) Sending RPC req@00000000032739c9 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_02:lustre-MDT0000-mdtlov_UUID:3355:1862456998028160:0@lo:13:osp-pre-0-0.0 00000100:00100000:3.0:1776178128.738132:0:3355:0:(events.c:361:request_in_callback()) peer: 12345-0@lo (source: 12345-0@lo) 00000100:00100000:3.0:1776178128.738170:0:8231:0:(service.c:2152:ptlrpc_server_handle_req_in()) got req x1862456998028160 00000100:00100000:3.0:1776178128.738195:0:8231:0:(nrs_fifo.c:179:nrs_fifo_req_get()) NRS start fifo request from 12345-0@lo, seq: 419 00000100:00100000:3.0:1776178128.738199:0:8231:0:(service.c:2324:ptlrpc_server_handle_request()) Handling RPC req@00000000544c54c4 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_create00:lustre-MDT0000-mdtlov_UUID+6:3355:x1862456998028160:12345-0@lo:13:osp-pre-0-0.0 00002000:00000024:3.0:1776178128.738218:0:8231:0:(ofd_obd.c:740:ofd_statfs()) blocks cached 315392 granted 484883776 pending 0 free 3923689472 avail 3692167168 00000100:00100000:3.0:1776178128.738246:0:8231:0:(service.c:2372:ptlrpc_server_handle_request()) Handled RPC req@00000000544c54c4 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_create00:lustre-MDT0000-mdtlov_UUID+6:3355:x1862456998028160:12345-0@lo:13:osp-pre-0-0.0 Request processed in 46us (114us total) trans 0 rc 0/0 00000100:00100000:3.0:1776178128.738252:0:8231:0:(nrs_fifo.c:241:nrs_fifo_req_stop()) NRS stop fifo request from 12345-0@lo, seq: 419 00000100:00100000:3.0:1776178128.738293:0:3355:0:(client.c:2215:ptlrpc_check_set()) Completed RPC req@00000000032739c9 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_02:lustre-MDT0000-mdtlov_UUID:3355:1862456998028160:0@lo:13:osp-pre-0-0.0 00000100:00100000:3.0:1776178128.738574:0:14128:0:(service.c:2152:ptlrpc_server_handle_req_in()) got req x1862456998027264 00000100:00100000:3.0:1776178128.738602:0:14128:0:(service.c:2152:ptlrpc_server_handle_req_in()) got req x1862456998027648 00000100:00100000:3.0:1776178128.738619:0:14128:0:(nrs_fifo.c:179:nrs_fifo_req_get()) NRS start fifo request from 12345-0@lo, seq: 632 00000100:00100000:3.0:1776178128.738623:0:14128:0:(service.c:2324:ptlrpc_server_handle_request()) Handling RPC req@0000000057fa7e26 pname:cluuid+ref:pid:xid:nid:opc:job mdt_out00_002:lustre-MDT0000-mdtlov_UUID+5:3354:x1862456998027264:12345-0@lo:41:osp-pre-1-0.0 00000004:00000024:3.0:1776178128.738657:0:14128:0:(mdt_handler.c:548:mdt_statfs()) blocks cached 0 granted 2097152 pending 0 free 1446039552 avail 1316577280 00000100:00100000:3.0:1776178128.738690:0:14128:0:(service.c:2372:ptlrpc_server_handle_request()) Handled RPC req@0000000057fa7e26 pname:cluuid+ref:pid:xid:nid:opc:job mdt_out00_002:lustre-MDT0000-mdtlov_UUID+5:3354:x1862456998027264:12345-0@lo:41:osp-pre-1-0.0 Request processed in 66us (2089us total) trans 0 rc 0/0 00000100:00100000:3.0:1776178128.738696:0:14128:0:(nrs_fifo.c:241:nrs_fifo_req_stop()) NRS stop fifo request from 12345-0@lo, seq: 632 00000100:00100000:3.0:1776178128.739110:0:14128:0:(nrs_fifo.c:179:nrs_fifo_req_get()) NRS start fifo request from 12345-0@lo, seq: 633 00000100:00100000:3.0:1776178128.739114:0:14128:0:(service.c:2324:ptlrpc_server_handle_request()) Handling RPC req@00000000c63a98c1 pname:cluuid+ref:pid:xid:nid:opc:job mdt_out00_002:lustre-MDT0001-mdtlov_UUID+6:3356:x1862456998027648:12345-0@lo:400:kworker/u8:0.0 00000100:00100000:3.0:1776178128.739160:0:14128:0:(service.c:2372:ptlrpc_server_handle_request()) Handled RPC req@00000000c63a98c1 pname:cluuid+ref:pid:xid:nid:opc:job mdt_out00_002:lustre-MDT0001-mdtlov_UUID+6:3356:x1862456998027648:12345-0@lo:400:kworker/u8:0.0 Request processed in 44us (1907us total) trans 0 rc 0/0 00000100:00100000:3.0:1776178128.739167:0:14128:0:(nrs_fifo.c:241:nrs_fifo_req_stop()) NRS stop fifo request from 12345-0@lo, seq: 633 00000100:00100000:3.0:1776178128.739195:0:3353:0:(client.c:1733:ptlrpc_send_new_req()) Sending RPC req@0000000056ddcb09 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_00:lustre-MDT0000-lwp-OST0001_UUID:3353:1862456998027520:0@lo:400:kworker/u8:0.0 00000100:00100000:3.0:1776178128.739223:0:3353:0:(events.c:361:request_in_callback()) peer: 12345-0@lo (source: 12345-0@lo) 00000100:00100000:3.0:1776178128.739243:0:3353:0:(client.c:1733:ptlrpc_send_new_req()) Sending RPC req@0000000049b1b339 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_00:lustre-MDT0001-mdtlov_UUID:3353:1862456998027776:0@lo:400:kworker/u8:0.0 00000100:00100000:3.0:1776178128.739264:0:3353:0:(events.c:361:request_in_callback()) peer: 12345-0@lo (source: 12345-0@lo) 00000100:00100000:3.0:1776178128.739278:0:3353:0:(client.c:1733:ptlrpc_send_new_req()) Sending RPC req@00000000f0f1167e pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_00:lustre-MDT0000-lwp-MDT0000_UUID:3353:1862456998028032:0@lo:400:kworker/u8:0.0 00000100:00100000:3.0:1776178128.739297:0:3353:0:(events.c:361:request_in_callback()) peer: 12345-0@lo (source: 12345-0@lo) 00000100:00100000:3.0:1776178128.739313:0:3353:0:(ptlrpcd.c:415:ptlrpcd_check()) transfer 4 async RPCs [1->0] 00000100:00100000:3.0:1776178128.739320:0:3353:0:(client.c:1733:ptlrpc_send_new_req()) Sending RPC req@00000000575d0558 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_00:ac61f839-0e8e-4cac-9c59-08bcfbbad330:3353:1862456998027328:0@lo:400:kworker/u8:0.0 00000100:00100000:3.0:1776178128.739347:0:3353:0:(events.c:361:request_in_callback()) peer: 12345-0@lo (source: 12345-0@lo) 00000100:00100000:3.0:1776178128.739367:0:3353:0:(client.c:1733:ptlrpc_send_new_req()) Sending RPC req@00000000eb400454 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_00:lustre-MDT0001-lwp-OST0001_UUID:3353:1862456998027584:0@lo:400:kworker/u8:0.0 00000100:00100000:3.0:1776178128.739387:0:3353:0:(events.c:361:request_in_callback()) peer: 12345-0@lo (source: 12345-0@lo) 00000100:00100000:0.0:1776178128.739462:0:3354:0:(client.c:2215:ptlrpc_check_set()) Completed RPC req@00000000ddc1dc02 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_01:lustre-MDT0000-mdtlov_UUID:3354:1862456998027264:0@lo:41:osp-pre-1-0.0 00000100:00100000:1.0:1776178128.739865:0:20762:0:(service.c:2152:ptlrpc_server_handle_req_in()) got req x1862456998027776 00000100:00100000:1.0:1776178128.739889:0:20762:0:(nrs_fifo.c:179:nrs_fifo_req_get()) NRS start fifo request from 12345-0@lo, seq: 432 00000100:00100000:1.0:1776178128.739894:0:20762:0:(service.c:2324:ptlrpc_server_handle_request()) Handling RPC req@0000000015095c30 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_013:lustre-MDT0000-mdtlov_UUID+5:3356:x1862456998027904:12345-0@lo:400:kworker/u8:0.0 00000100:00100000:1.0:1776178128.739934:0:20762:0:(service.c:2372:ptlrpc_server_handle_request()) Handled RPC req@0000000015095c30 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_013:lustre-MDT0000-mdtlov_UUID+5:3356:x1862456998027904:12345-0@lo:400:kworker/u8:0.0 Request processed in 40us (2643us total) trans 0 rc 0/0 00000100:00100000:1.0:1776178128.739940:0:20762:0:(nrs_fifo.c:241:nrs_fifo_req_stop()) NRS stop fifo request from 12345-0@lo, seq: 432 00000100:00100000:1.0:1776178128.739952:0:20762:0:(nrs_fifo.c:179:nrs_fifo_req_get()) NRS start fifo request from 12345-0@lo, seq: 433 00000100:00100000:1.0:1776178128.739955:0:20762:0:(service.c:2324:ptlrpc_server_handle_request()) Handling RPC req@000000006c7327ca pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_013:lustre-MDT0001-mdtlov_UUID+5:3353:x1862456998027776:12345-0@lo:400:kworker/u8:0.0 00000100:00100000:1.0:1776178128.739980:0:20762:0:(service.c:2372:ptlrpc_server_handle_request()) Handled RPC req@000000006c7327ca pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_013:lustre-MDT0001-mdtlov_UUID+5:3353:x1862456998027776:12345-0@lo:400:kworker/u8:0.0 Request processed in 23us (715us total) trans 0 rc 0/0 00000100:00100000:1.0:1776178128.739985:0:20762:0:(nrs_fifo.c:241:nrs_fifo_req_stop()) NRS stop fifo request from 12345-0@lo, seq: 433 00000100:00100000:1.0:1776178128.740048:0:3356:0:(client.c:1733:ptlrpc_send_new_req()) Sending RPC req@0000000054e3866c pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_03:lustre-MDT0000-mdtlov_UUID:3356:1862456998028224:0@lo:13:osp-pre-1-0.0 00000100:00100000:1.0:1776178128.740756:0:3356:0:(events.c:361:request_in_callback()) peer: 12345-0@lo (source: 12345-0@lo) 00000100:00100000:1.0:1776178128.740807:0:3356:0:(client.c:2215:ptlrpc_check_set()) Completed RPC req@00000000df370bee pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_03:lustre-MDT0000-lwp-OST0000_UUID:3356:1862456998027392:0@lo:400:kworker/u8:0.0 00000100:00100000:1.0:1776178128.740821:0:3356:0:(client.c:2215:ptlrpc_check_set()) Completed RPC req@00000000699e66b5 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_03:lustre-MDT0001-mdtlov_UUID:3356:1862456998027648:0@lo:400:kworker/u8:0.0 00000100:00100000:1.0:1776178128.740836:0:3356:0:(client.c:2215:ptlrpc_check_set()) Completed RPC req@0000000022d4dc29 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_03:lustre-MDT0000-mdtlov_UUID:3356:1862456998027904:0@lo:400:kworker/u8:0.0 00000100:00100000:1.0:1776178128.740849:0:3356:0:(client.c:2215:ptlrpc_check_set()) Completed RPC req@00000000a59bfc89 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_03:lustre-MDT0001-lwp-OST0000_UUID:3356:1862456998027456:0@lo:400:kworker/u8:0.0 00000100:00100000:1.0:1776178128.740859:0:3356:0:(client.c:2215:ptlrpc_check_set()) Completed RPC req@000000007d60c0e9 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_03:lustre-MDT0001-mdtlov_UUID:3356:1862456998027712:0@lo:400:kworker/u8:0.0 00000100:00100000:1.0:1776178128.740870:0:3356:0:(client.c:2215:ptlrpc_check_set()) Completed RPC req@0000000039ff8335 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_03:lustre-MDT0000-mdtlov_UUID:3356:1862456998027968:0@lo:400:kworker/u8:0.0 00000100:00100000:2.0:1776178128.741402:0:20137:0:(service.c:2152:ptlrpc_server_handle_req_in()) got req x1862456998027328 00000100:00100000:2.0:1776178128.741462:0:20137:0:(nrs_fifo.c:179:nrs_fifo_req_get()) NRS start fifo request from 12345-0@lo, seq: 125 00000100:00100000:2.0:1776178128.741467:0:20137:0:(service.c:2324:ptlrpc_server_handle_request()) Handling RPC req@00000000a2768ffd pname:cluuid+ref:pid:xid:nid:opc:job ll_mgs_0000:ac61f839-0e8e-4cac-9c59-08bcfbbad330+23:3353:x1862456998027328:12345-0@lo:400:kworker/u8:0.0 00000100:00100000:2.0:1776178128.741522:0:20137:0:(service.c:2372:ptlrpc_server_handle_request()) Handled RPC req@00000000a2768ffd pname:cluuid+ref:pid:xid:nid:opc:job ll_mgs_0000:ac61f839-0e8e-4cac-9c59-08bcfbbad330+23:3353:x1862456998027328:12345-0@lo:400:kworker/u8:0.0 Request processed in 55us (2174us total) trans 0 rc 0/0 00000100:00100000:2.0:1776178128.741530:0:20137:0:(nrs_fifo.c:241:nrs_fifo_req_stop()) NRS stop fifo request from 12345-0@lo, seq: 125 00000100:00100000:3.0:1776178128.742218:0:8232:0:(service.c:2152:ptlrpc_server_handle_req_in()) got req x1862456998028224 00000100:00100000:3.0:1776178128.742254:0:8232:0:(nrs_fifo.c:179:nrs_fifo_req_get()) NRS start fifo request from 12345-0@lo, seq: 420 00000100:00100000:3.0:1776178128.742259:0:8232:0:(service.c:2324:ptlrpc_server_handle_request()) Handling RPC req@000000009de51106 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_create00:lustre-MDT0000-mdtlov_UUID+5:3356:x1862456998028224:12345-0@lo:13:osp-pre-1-0.0 00002000:00000024:3.0:1776178128.742285:0:8232:0:(ofd_obd.c:740:ofd_statfs()) blocks cached 229376 granted 491746368 pending 0 free 3923689472 avail 3692167168 00000100:00100000:3.0:1776178128.742325:0:8232:0:(service.c:2372:ptlrpc_server_handle_request()) Handled RPC req@000000009de51106 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_create00:lustre-MDT0000-mdtlov_UUID+5:3356:x1862456998028224:12345-0@lo:13:osp-pre-1-0.0 Request processed in 66us (1574us total) trans 0 rc 0/0 00000100:00100000:3.0:1776178128.742331:0:8232:0:(nrs_fifo.c:241:nrs_fifo_req_stop()) NRS stop fifo request from 12345-0@lo, seq: 420 00000100:00100000:3.0:1776178128.742354:0:3353:0:(client.c:1733:ptlrpc_send_new_req()) Sending RPC req@00000000ed90ff75 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_00:lustre-MDT0000-lwp-MDT0001_UUID:3353:1862456998027840:0@lo:400:kworker/u8:0.0 00000100:00100000:3.0:1776178128.742384:0:3353:0:(events.c:361:request_in_callback()) peer: 12345-0@lo (source: 12345-0@lo) 00000100:00100000:3.0:1776178128.743215:0:3353:0:(client.c:1733:ptlrpc_send_new_req()) Sending RPC req@00000000d6e5d9ba pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_00:lustre-MDT0001-mdtlov_UUID:3353:1862456998028096:0@lo:41:osp-pre-0-1.0 00000100:00100000:3.0:1776178128.743248:0:3353:0:(events.c:361:request_in_callback()) peer: 12345-0@lo (source: 12345-0@lo) 00000100:00100000:3.0:1776178128.743282:0:3353:0:(client.c:2215:ptlrpc_check_set()) Completed RPC req@0000000049b1b339 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_00:lustre-MDT0001-mdtlov_UUID:3353:1862456998027776:0@lo:400:kworker/u8:0.0 00000100:00100000:3.0:1776178128.743317:0:3353:0:(client.c:2215:ptlrpc_check_set()) Completed RPC req@00000000575d0558 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_00:ac61f839-0e8e-4cac-9c59-08bcfbbad330:3353:1862456998027328:0@lo:400:kworker/u8:0.0 00000100:00100000:0.0:1776178128.743457:0:6256:0:(service.c:2152:ptlrpc_server_handle_req_in()) got req x1862456998027520 00000100:00100000:0.0:1776178128.743493:0:6256:0:(service.c:2152:ptlrpc_server_handle_req_in()) got req x1862456998028032 00000100:00100000:0.0:1776178128.743507:0:6256:0:(service.c:2152:ptlrpc_server_handle_req_in()) got req x1862456998027584 00000100:00100000:0.0:1776178128.743518:0:6256:0:(service.c:2152:ptlrpc_server_handle_req_in()) got req x1862456998027840 00000100:00100000:0.0:1776178128.743528:0:6256:0:(nrs_fifo.c:179:nrs_fifo_req_get()) NRS start fifo request from 12345-0@lo, seq: 789 00000100:00100000:0.0:1776178128.743532:0:6256:0:(service.c:2324:ptlrpc_server_handle_request()) Handling RPC req@00000000044bd507 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_001:lustre-MDT0000-lwp-OST0001_UUID+5:3353:x1862456998027520:12345-0@lo:400:kworker/u8:0.0 00000100:00100000:0.0:1776178128.743588:0:6256:0:(service.c:2372:ptlrpc_server_handle_request()) Handled RPC req@00000000044bd507 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_001:lustre-MDT0000-lwp-OST0001_UUID+5:3353:x1862456998027520:12345-0@lo:400:kworker/u8:0.0 Request processed in 54us (4364us total) trans 0 rc 0/0 00000100:00100000:0.0:1776178128.743594:0:6256:0:(nrs_fifo.c:241:nrs_fifo_req_stop()) NRS stop fifo request from 12345-0@lo, seq: 789 00000100:00100000:0.0:1776178128.743608:0:6256:0:(nrs_fifo.c:179:nrs_fifo_req_get()) NRS start fifo request from 12345-0@lo, seq: 790 00000100:00100000:0.0:1776178128.743622:0:6256:0:(service.c:2324:ptlrpc_server_handle_request()) Handling RPC req@000000008569a159 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_001:lustre-MDT0000-lwp-MDT0000_UUID+5:3353:x1862456998028032:12345-0@lo:400:kworker/u8:0.0 00000100:00100000:0.0:1776178128.743655:0:6256:0:(service.c:2372:ptlrpc_server_handle_request()) Handled RPC req@000000008569a159 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_001:lustre-MDT0000-lwp-MDT0000_UUID+5:3353:x1862456998028032:12345-0@lo:400:kworker/u8:0.0 Request processed in 43us (4357us total) trans 0 rc 0/0 00000100:00100000:0.0:1776178128.743660:0:6256:0:(nrs_fifo.c:241:nrs_fifo_req_stop()) NRS stop fifo request from 12345-0@lo, seq: 790 00000100:00100000:0.0:1776178128.743670:0:6256:0:(nrs_fifo.c:179:nrs_fifo_req_get()) NRS start fifo request from 12345-0@lo, seq: 791 00000100:00100000:0.0:1776178128.743673:0:6256:0:(service.c:2324:ptlrpc_server_handle_request()) Handling RPC req@00000000032739c9 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_001:lustre-MDT0001-lwp-OST0001_UUID+5:3353:x1862456998027584:12345-0@lo:400:kworker/u8:0.0 00000100:00100000:0.0:1776178128.743694:0:6256:0:(service.c:2372:ptlrpc_server_handle_request()) Handled RPC req@00000000032739c9 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_001:lustre-MDT0001-lwp-OST0001_UUID+5:3353:x1862456998027584:12345-0@lo:400:kworker/u8:0.0 Request processed in 20us (4307us total) trans 0 rc 0/0 00000100:00100000:0.0:1776178128.743700:0:6256:0:(nrs_fifo.c:241:nrs_fifo_req_stop()) NRS stop fifo request from 12345-0@lo, seq: 791 00000100:00100000:0.0:1776178128.743708:0:6256:0:(nrs_fifo.c:179:nrs_fifo_req_get()) NRS start fifo request from 12345-0@lo, seq: 792 00000100:00100000:0.0:1776178128.743711:0:6256:0:(service.c:2324:ptlrpc_server_handle_request()) Handling RPC req@00000000b8464782 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_001:lustre-MDT0000-lwp-MDT0001_UUID+5:3353:x1862456998027840:12345-0@lo:400:kworker/u8:0.0 00000100:00100000:0.0:1776178128.743732:0:6256:0:(service.c:2372:ptlrpc_server_handle_request()) Handled RPC req@00000000b8464782 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_001:lustre-MDT0000-lwp-MDT0001_UUID+5:3353:x1862456998027840:12345-0@lo:400:kworker/u8:0.0 Request processed in 20us (1348us total) trans 0 rc 0/0 00000100:00100000:0.0:1776178128.743737:0:6256:0:(nrs_fifo.c:241:nrs_fifo_req_stop()) NRS stop fifo request from 12345-0@lo, seq: 792 00000100:00100000:0.0:1776178128.743847:0:3356:0:(client.c:2215:ptlrpc_check_set()) Completed RPC req@0000000054e3866c pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_03:lustre-MDT0000-mdtlov_UUID:3356:1862456998028224:0@lo:13:osp-pre-1-0.0 00000100:00100000:0.0:1776178128.743961:0:6260:0:(service.c:2152:ptlrpc_server_handle_req_in()) got req x1862456998028096 00000100:00100000:0.0:1776178128.743985:0:6260:0:(nrs_fifo.c:179:nrs_fifo_req_get()) NRS start fifo request from 12345-0@lo, seq: 634 00000100:00100000:0.0:1776178128.743989:0:6260:0:(service.c:2324:ptlrpc_server_handle_request()) Handling RPC req@00000000638772fd pname:cluuid+ref:pid:xid:nid:opc:job mdt_out00_000:lustre-MDT0001-mdtlov_UUID+6:3353:x1862456998028096:12345-0@lo:41:osp-pre-0-1.0 00000004:00000024:0.0:1776178128.744011:0:6260:0:(mdt_handler.c:548:mdt_statfs()) blocks cached 0 granted 2097152 pending 0 free 1446121472 avail 1316659200 00000100:00100000:0.0:1776178128.744041:0:6260:0:(service.c:2372:ptlrpc_server_handle_request()) Handled RPC req@00000000638772fd pname:cluuid+ref:pid:xid:nid:opc:job mdt_out00_000:lustre-MDT0001-mdtlov_UUID+6:3353:x1862456998028096:12345-0@lo:41:osp-pre-0-1.0 Request processed in 51us (793us total) trans 0 rc 0/0 00000100:00100000:0.0:1776178128.744047:0:6260:0:(nrs_fifo.c:241:nrs_fifo_req_stop()) NRS stop fifo request from 12345-0@lo, seq: 634 00000100:00100000:2.0:1776178128.744843:0:3353:0:(client.c:2215:ptlrpc_check_set()) Completed RPC req@00000000eb400454 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_00:lustre-MDT0001-lwp-OST0001_UUID:3353:1862456998027584:0@lo:400:kworker/u8:0.0 00000100:00100000:2.0:1776178128.744858:0:3353:0:(client.c:2215:ptlrpc_check_set()) Completed RPC req@00000000ed90ff75 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_00:lustre-MDT0000-lwp-MDT0001_UUID:3353:1862456998027840:0@lo:400:kworker/u8:0.0 00000100:00100000:2.0:1776178128.744873:0:3353:0:(client.c:2215:ptlrpc_check_set()) Completed RPC req@00000000d6e5d9ba pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_00:lustre-MDT0001-mdtlov_UUID:3353:1862456998028096:0@lo:41:osp-pre-0-1.0 00000100:00100000:2.0:1776178128.744885:0:3353:0:(client.c:2215:ptlrpc_check_set()) Completed RPC req@0000000056ddcb09 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_00:lustre-MDT0000-lwp-OST0001_UUID:3353:1862456998027520:0@lo:400:kworker/u8:0.0 00000100:00100000:2.0:1776178128.744894:0:3353:0:(client.c:2215:ptlrpc_check_set()) Completed RPC req@00000000f0f1167e pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_00:lustre-MDT0000-lwp-MDT0000_UUID:3353:1862456998028032:0@lo:400:kworker/u8:0.0 00010000:00010000:2.0:1776178129.316840:0:9224:0:(ldlm_lockd.c:2792:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:3.0:1776178129.697676:0:3348:0:(events.c:361:request_in_callback()) peer: 12345-192.168.204.3@tcp (source: 12345-192.168.204.3@tcp) 00000100:00100000:3.0:1776178129.697725:0:3348:0:(events.c:361:request_in_callback()) peer: 12345-192.168.204.3@tcp (source: 12345-192.168.204.3@tcp) 00000100:00100000:3.0:1776178129.697764:0:3348:0:(events.c:361:request_in_callback()) peer: 12345-192.168.204.3@tcp (source: 12345-192.168.204.3@tcp) 00000100:00100000:3.0:1776178129.697785:0:3348:0:(events.c:361:request_in_callback()) peer: 12345-192.168.204.3@tcp (source: 12345-192.168.204.3@tcp) 00000100:00100000:3.0:1776178129.697807:0:3348:0:(events.c:361:request_in_callback()) peer: 12345-192.168.204.3@tcp (source: 12345-192.168.204.3@tcp) 00000100:00100000:2.0:1776178129.698953:0:20140:0:(service.c:2152:ptlrpc_server_handle_req_in()) got req x1862456979083456 00000100:00100000:2.0:1776178129.699020:0:20140:0:(nrs_fifo.c:179:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.204.3@tcp, seq: 126 00000100:00100000:2.0:1776178129.699027:0:20140:0:(service.c:2324:ptlrpc_server_handle_request()) Handling RPC req@00000000887443f4 pname:cluuid+ref:pid:xid:nid:opc:job ll_mgs_0002:5b5c66b1-1a2f-4651-8548-0c75b9a38a94+9:2270:x1862456979083456:12345-192.168.204.3@tcp:400:kworker/u8:3.0 00000100:00100000:2.0:1776178129.699165:0:20140:0:(service.c:2372:ptlrpc_server_handle_request()) Handled RPC req@00000000887443f4 pname:cluuid+ref:pid:xid:nid:opc:job ll_mgs_0002:5b5c66b1-1a2f-4651-8548-0c75b9a38a94+9:2270:x1862456979083456:12345-192.168.204.3@tcp:400:kworker/u8:3.0 Request processed in 137us (1498us total) trans 0 rc 0/0 00000100:00100000:2.0:1776178129.699177:0:20140:0:(nrs_fifo.c:241:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.204.3@tcp, seq: 126 00000100:00100000:2.0:1776178129.699224:0:20769:0:(service.c:2152:ptlrpc_server_handle_req_in()) got req x1862456979083712 00000100:00100000:2.0:1776178129.699248:0:20769:0:(service.c:2152:ptlrpc_server_handle_req_in()) got req x1862456979083648 00000100:00100000:2.0:1776178129.699265:0:20769:0:(nrs_fifo.c:179:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.204.3@tcp, seq: 434 00000100:00100000:2.0:1776178129.699269:0:20769:0:(service.c:2324:ptlrpc_server_handle_request()) Handling RPC req@00000000f586aea3 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_018:b50963bd-8eba-4432-9a54-8888d00e3fa1+8:2270:x1862456979083712:12345-192.168.204.3@tcp:400:kworker/u8:3.0 00000100:00100000:2.0:1776178129.699300:0:20769:0:(service.c:2372:ptlrpc_server_handle_request()) Handled RPC req@00000000f586aea3 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_018:b50963bd-8eba-4432-9a54-8888d00e3fa1+8:2270:x1862456979083712:12345-192.168.204.3@tcp:400:kworker/u8:3.0 Request processed in 30us (1536us total) trans 0 rc 0/0 00000100:00100000:2.0:1776178129.699307:0:20769:0:(nrs_fifo.c:241:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.204.3@tcp, seq: 434 00000100:00100000:2.0:1776178129.699318:0:20769:0:(nrs_fifo.c:179:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.204.3@tcp, seq: 435 00000100:00100000:2.0:1776178129.699321:0:20769:0:(service.c:2324:ptlrpc_server_handle_request()) Handling RPC req@000000006f8bc860 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_018:b50963bd-8eba-4432-9a54-8888d00e3fa1+9:2270:x1862456979083648:12345-192.168.204.3@tcp:400:kworker/u8:3.0 00000100:00100000:2.0:1776178129.699346:0:20769:0:(service.c:2372:ptlrpc_server_handle_request()) Handled RPC req@000000006f8bc860 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_018:b50963bd-8eba-4432-9a54-8888d00e3fa1+9:2270:x1862456979083648:12345-192.168.204.3@tcp:400:kworker/u8:3.0 Request processed in 24us (1539us total) trans 0 rc 0/0 00000100:00100000:2.0:1776178129.699352:0:20769:0:(nrs_fifo.c:241:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.204.3@tcp, seq: 435 00000100:00100000:2.0:1776178129.699372:0:8250:0:(service.c:2152:ptlrpc_server_handle_req_in()) got req x1862456979083584 00000100:00100000:2.0:1776178129.699394:0:8250:0:(service.c:2152:ptlrpc_server_handle_req_in()) got req x1862456979083520 00000100:00100000:2.0:1776178129.699410:0:8250:0:(nrs_fifo.c:179:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.204.3@tcp, seq: 793 00000100:00100000:2.0:1776178129.699414:0:8250:0:(service.c:2324:ptlrpc_server_handle_request()) Handling RPC req@0000000035fd3386 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_003:b50963bd-8eba-4432-9a54-8888d00e3fa1+6:2270:x1862456979083584:12345-192.168.204.3@tcp:400:kworker/u8:3.0 00000100:00100000:2.0:1776178129.699441:0:8250:0:(service.c:2372:ptlrpc_server_handle_request()) Handled RPC req@0000000035fd3386 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_003:b50963bd-8eba-4432-9a54-8888d00e3fa1+6:2270:x1862456979083584:12345-192.168.204.3@tcp:400:kworker/u8:3.0 Request processed in 27us (1717us total) trans 0 rc 0/0 00000100:00100000:2.0:1776178129.699447:0:8250:0:(nrs_fifo.c:241:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.204.3@tcp, seq: 793 00000100:00100000:2.0:1776178129.699458:0:8250:0:(nrs_fifo.c:179:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.204.3@tcp, seq: 794 00000100:00100000:2.0:1776178129.699461:0:8250:0:(service.c:2324:ptlrpc_server_handle_request()) Handling RPC req@0000000078361ddf pname:cluuid+ref:pid:xid:nid:opc:job mdt00_003:b50963bd-8eba-4432-9a54-8888d00e3fa1+21:2270:x1862456979083520:12345-192.168.204.3@tcp:400:kworker/u8:3.0 00000100:00100000:2.0:1776178129.699487:0:8250:0:(service.c:2372:ptlrpc_server_handle_request()) Handled RPC req@0000000078361ddf pname:cluuid+ref:pid:xid:nid:opc:job mdt00_003:b50963bd-8eba-4432-9a54-8888d00e3fa1+21:2270:x1862456979083520:12345-192.168.204.3@tcp:400:kworker/u8:3.0 Request processed in 25us (1702us total) trans 0 rc 0/0 00000100:00100000:2.0:1776178129.699494:0:8250:0:(nrs_fifo.c:241:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.204.3@tcp, seq: 794 00000100:00100000:0.0:1776178130.272901:0:3354:0:(ptlrpcd.c:415:ptlrpcd_check()) transfer 1 async RPCs [0->1] 00000100:00100000:0.0:1776178130.272931:0:3354:0:(client.c:1733:ptlrpc_send_new_req()) Sending RPC req@0000000020afd0b3 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_01:lustre-MDT0001-mdtlov_UUID:3354:1862456998028288:0@lo:13:osp-pre-1-1.0 00000100:00100000:0.0:1776178130.273012:0:3354:0:(events.c:361:request_in_callback()) peer: 12345-0@lo (source: 12345-0@lo) 00000100:00100000:3.0:1776178130.273098:0:8231:0:(service.c:2152:ptlrpc_server_handle_req_in()) got req x1862456998028288 00000100:00100000:3.0:1776178130.273172:0:8231:0:(nrs_fifo.c:179:nrs_fifo_req_get()) NRS start fifo request from 12345-0@lo, seq: 421 00000100:00100000:3.0:1776178130.273178:0:8231:0:(service.c:2324:ptlrpc_server_handle_request()) Handling RPC req@00000000ddc1dc02 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_create00:lustre-MDT0001-mdtlov_UUID+5:3354:x1862456998028288:12345-0@lo:13:osp-pre-1-1.0 00002000:00000024:3.0:1776178130.273204:0:8231:0:(ofd_obd.c:740:ofd_statfs()) blocks cached 229376 granted 491746368 pending 0 free 3923689472 avail 3692167168 00000100:00100000:3.0:1776178130.273301:0:8231:0:(service.c:2372:ptlrpc_server_handle_request()) Handled RPC req@00000000ddc1dc02 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_create00:lustre-MDT0001-mdtlov_UUID+5:3354:x1862456998028288:12345-0@lo:13:osp-pre-1-1.0 Request processed in 122us (290us total) trans 0 rc 0/0 00000100:00100000:3.0:1776178130.273309:0:8231:0:(nrs_fifo.c:241:nrs_fifo_req_stop()) NRS stop fifo request from 12345-0@lo, seq: 421 00000100:00100000:1.0:1776178130.273396:0:3354:0:(client.c:2215:ptlrpc_check_set()) Completed RPC req@0000000020afd0b3 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_01:lustre-MDT0001-mdtlov_UUID:3354:1862456998028288:0@lo:13:osp-pre-1-1.0 00010000:00010000:1.0:1776178130.340621:0:6244:0:(ldlm_lockd.c:2792:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:1.0:1776178130.785114:0:3354:0:(client.c:1733:ptlrpc_send_new_req()) Sending RPC req@000000004cbcc47c pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_01:lustre-MDT0001-mdtlov_UUID:3354:1862456998028352:0@lo:13:osp-pre-0-1.0 00000100:00100000:1.0:1776178130.785210:0:3354:0:(events.c:361:request_in_callback()) peer: 12345-0@lo (source: 12345-0@lo) 00000100:00100000:0.0:1776178130.785312:0:8232:0:(service.c:2152:ptlrpc_server_handle_req_in()) got req x1862456998028352 00000100:00100000:0.0:1776178130.785384:0:8232:0:(nrs_fifo.c:179:nrs_fifo_req_get()) NRS start fifo request from 12345-0@lo, seq: 422 00000100:00100000:0.0:1776178130.785392:0:8232:0:(service.c:2324:ptlrpc_server_handle_request()) Handling RPC req@000000009016e75e pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_create00:lustre-MDT0001-mdtlov_UUID+5:3354:x1862456998028352:12345-0@lo:13:osp-pre-0-1.0 00002000:00000024:0.0:1776178130.785432:0:8232:0:(ofd_obd.c:740:ofd_statfs()) blocks cached 315392 granted 484883776 pending 0 free 3923689472 avail 3692167168 00000100:00100000:0.0:1776178130.786115:0:8232:0:(service.c:2372:ptlrpc_server_handle_request()) Handled RPC req@000000009016e75e pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_create00:lustre-MDT0001-mdtlov_UUID+5:3354:x1862456998028352:12345-0@lo:13:osp-pre-0-1.0 Request processed in 720us (903us total) trans 0 rc 0/0 00000100:00100000:0.0:1776178130.786125:0:8232:0:(nrs_fifo.c:241:nrs_fifo_req_stop()) NRS stop fifo request from 12345-0@lo, seq: 422 00000100:00100000:1.0:1776178130.786358:0:3354:0:(client.c:2215:ptlrpc_check_set()) Completed RPC req@000000004cbcc47c pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_01:lustre-MDT0001-mdtlov_UUID:3354:1862456998028352:0@lo:13:osp-pre-0-1.0 00010000:00010000:1.0:1776178131.366176:0:6245:0:(ldlm_lockd.c:2792:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00010000:00010000:1.0:1776178132.387886:0:9225:0:(ldlm_lockd.c:2792:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00010000:00010000:2.0:1776178133.408951:0:9224:0:(ldlm_lockd.c:2792:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:0.0:1776178133.856354:0:3356:0:(ptlrpcd.c:415:ptlrpcd_check()) transfer 1 async RPCs [2->3] 00000100:00100000:0.0:1776178133.856380:0:3356:0:(client.c:1733:ptlrpc_send_new_req()) Sending RPC req@000000006d540857 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_03:lustre-MDT0000-mdtlov_UUID:3356:1862456998028416:0@lo:41:osp-pre-1-0.0 00000100:00100000:0.0:1776178133.856451:0:3356:0:(events.c:361:request_in_callback()) peer: 12345-0@lo (source: 12345-0@lo) 00000100:00100000:0.0:1776178133.856525:0:15213:0:(service.c:2152:ptlrpc_server_handle_req_in()) got req x1862456998028416 00000100:00100000:0.0:1776178133.856582:0:15213:0:(nrs_fifo.c:179:nrs_fifo_req_get()) NRS start fifo request from 12345-0@lo, seq: 635 00000100:00100000:0.0:1776178133.856589:0:15213:0:(service.c:2324:ptlrpc_server_handle_request()) Handling RPC req@000000003095df64 pname:cluuid+ref:pid:xid:nid:opc:job mdt_out00_003:lustre-MDT0000-mdtlov_UUID+5:3356:x1862456998028416:12345-0@lo:41:osp-pre-1-0.0 00000004:00000024:0.0:1776178133.856621:0:15213:0:(mdt_handler.c:548:mdt_statfs()) blocks cached 0 granted 2097152 pending 0 free 1446039552 avail 1316577280 00000100:00100000:0.0:1776178133.856659:0:15213:0:(service.c:2372:ptlrpc_server_handle_request()) Handled RPC req@000000003095df64 pname:cluuid+ref:pid:xid:nid:opc:job mdt_out00_003:lustre-MDT0000-mdtlov_UUID+5:3356:x1862456998028416:12345-0@lo:41:osp-pre-1-0.0 Request processed in 71us (209us total) trans 0 rc 0/0 00000100:00100000:0.0:1776178133.856665:0:15213:0:(nrs_fifo.c:241:nrs_fifo_req_stop()) NRS stop fifo request from 12345-0@lo, seq: 635 00000100:00100000:0.0:1776178133.856709:0:3356:0:(client.c:2215:ptlrpc_check_set()) Completed RPC req@000000006d540857 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_03:lustre-MDT0000-mdtlov_UUID:3356:1862456998028416:0@lo:41:osp-pre-1-0.0 00000100:00100000:3.0:1776178133.857291:0:3353:0:(client.c:1733:ptlrpc_send_new_req()) Sending RPC req@00000000339a9a57 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_00:lustre-MDT0000-lwp-OST0000_UUID:3353:1862456998028544:0@lo:400:kworker/u8:0.0 00000100:00100000:3.0:1776178133.857349:0:3353:0:(events.c:361:request_in_callback()) peer: 12345-0@lo (source: 12345-0@lo) 00000100:00100000:3.0:1776178133.857367:0:3353:0:(client.c:1733:ptlrpc_send_new_req()) Sending RPC req@00000000b13c16eb pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_00:lustre-MDT0001-mdtlov_UUID:3353:1862456998028800:0@lo:400:kworker/u8:0.0 00000100:00100000:3.0:1776178133.857388:0:3353:0:(events.c:361:request_in_callback()) peer: 12345-0@lo (source: 12345-0@lo) 00000100:00100000:3.0:1776178133.857402:0:3353:0:(client.c:1733:ptlrpc_send_new_req()) Sending RPC req@00000000f0f1167e pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_00:lustre-MDT0000-mdtlov_UUID:3353:1862456998029056:0@lo:400:kworker/u8:0.0 00000100:00100000:3.0:1776178133.857622:0:3353:0:(events.c:361:request_in_callback()) peer: 12345-0@lo (source: 12345-0@lo) 00000100:00100000:3.0:1776178133.857642:0:3353:0:(ptlrpcd.c:415:ptlrpcd_check()) transfer 3 async RPCs [1->0] 00000100:00100000:3.0:1776178133.857647:0:3353:0:(client.c:1733:ptlrpc_send_new_req()) Sending RPC req@0000000038c09127 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_00:lustre-MDT0001-lwp-OST0000_UUID:3353:1862456998028608:0@lo:400:kworker/u8:0.0 00000100:00100000:3.0:1776178133.857669:0:3353:0:(events.c:361:request_in_callback()) peer: 12345-0@lo (source: 12345-0@lo) 00000100:00100000:3.0:1776178133.857681:0:3353:0:(client.c:1733:ptlrpc_send_new_req()) Sending RPC req@0000000020afd0b3 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_00:lustre-MDT0001-mdtlov_UUID:3353:1862456998028864:0@lo:400:kworker/u8:0.0 00000100:00100000:3.0:1776178133.857698:0:3353:0:(events.c:361:request_in_callback()) peer: 12345-0@lo (source: 12345-0@lo) 00000100:00100000:3.0:1776178133.857709:0:3353:0:(client.c:1733:ptlrpc_send_new_req()) Sending RPC req@0000000056ddcb09 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_00:lustre-MDT0000-mdtlov_UUID:3353:1862456998029120:0@lo:400:kworker/u8:0.0 00000100:00100000:3.0:1776178133.857726:0:3353:0:(events.c:361:request_in_callback()) peer: 12345-0@lo (source: 12345-0@lo) 00000100:00100000:3.0:1776178133.857751:0:6260:0:(service.c:2152:ptlrpc_server_handle_req_in()) got req x1862456998028800 00000100:00100000:3.0:1776178133.857784:0:6260:0:(nrs_fifo.c:179:nrs_fifo_req_get()) NRS start fifo request from 12345-0@lo, seq: 636 00000100:00100000:3.0:1776178133.857788:0:6260:0:(service.c:2324:ptlrpc_server_handle_request()) Handling RPC req@0000000049b1b339 pname:cluuid+ref:pid:xid:nid:opc:job mdt_out00_000:lustre-MDT0001-mdtlov_UUID+6:3353:x1862456998028800:12345-0@lo:400:kworker/u8:0.0 00000100:00100000:3.0:1776178133.857825:0:6260:0:(service.c:2372:ptlrpc_server_handle_request()) Handled RPC req@0000000049b1b339 pname:cluuid+ref:pid:xid:nid:opc:job mdt_out00_000:lustre-MDT0001-mdtlov_UUID+6:3353:x1862456998028800:12345-0@lo:400:kworker/u8:0.0 Request processed in 37us (437us total) trans 0 rc 0/0 00000100:00100000:3.0:1776178133.857831:0:6260:0:(nrs_fifo.c:241:nrs_fifo_req_stop()) NRS stop fifo request from 12345-0@lo, seq: 636 00000100:00100000:3.0:1776178133.857863:0:3353:0:(client.c:2215:ptlrpc_check_set()) Completed RPC req@00000000b13c16eb pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_00:lustre-MDT0001-mdtlov_UUID:3353:1862456998028800:0@lo:400:kworker/u8:0.0 00000100:00100000:1.0:1776178133.857961:0:6255:0:(service.c:2152:ptlrpc_server_handle_req_in()) got req x1862456998028544 00000100:00100000:1.0:1776178133.857998:0:6255:0:(service.c:2152:ptlrpc_server_handle_req_in()) got req x1862456998028608 00000100:00100000:1.0:1776178133.858020:0:6255:0:(nrs_fifo.c:179:nrs_fifo_req_get()) NRS start fifo request from 12345-0@lo, seq: 795 00000100:00100000:1.0:1776178133.858024:0:6255:0:(service.c:2324:ptlrpc_server_handle_request()) Handling RPC req@00000000eb400454 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_000:lustre-MDT0000-lwp-OST0000_UUID+5:3353:x1862456998028544:12345-0@lo:400:kworker/u8:0.0 00000100:00100000:1.0:1776178133.858091:0:6255:0:(service.c:2372:ptlrpc_server_handle_request()) Handled RPC req@00000000eb400454 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_000:lustre-MDT0000-lwp-OST0000_UUID+5:3353:x1862456998028544:12345-0@lo:400:kworker/u8:0.0 Request processed in 66us (749us total) trans 0 rc 0/0 00000100:00100000:1.0:1776178133.858098:0:6255:0:(nrs_fifo.c:241:nrs_fifo_req_stop()) NRS stop fifo request from 12345-0@lo, seq: 795 00000100:00100000:1.0:1776178133.858109:0:6255:0:(nrs_fifo.c:179:nrs_fifo_req_get()) NRS start fifo request from 12345-0@lo, seq: 796 00000100:00100000:1.0:1776178133.858113:0:6255:0:(service.c:2324:ptlrpc_server_handle_request()) Handling RPC req@000000007d60c0e9 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_000:lustre-MDT0001-lwp-OST0000_UUID+5:3353:x1862456998028608:12345-0@lo:400:kworker/u8:0.0 00000100:00100000:1.0:1776178133.858136:0:6255:0:(service.c:2372:ptlrpc_server_handle_request()) Handled RPC req@000000007d60c0e9 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_000:lustre-MDT0001-lwp-OST0000_UUID+5:3353:x1862456998028608:12345-0@lo:400:kworker/u8:0.0 Request processed in 23us (468us total) trans 0 rc 0/0 00000100:00100000:1.0:1776178133.858141:0:6255:0:(nrs_fifo.c:241:nrs_fifo_req_stop()) NRS stop fifo request from 12345-0@lo, seq: 796 00000100:00100000:1.0:1776178133.858160:0:20762:0:(service.c:2152:ptlrpc_server_handle_req_in()) got req x1862456998029056 00000100:00100000:1.0:1776178133.858180:0:20762:0:(service.c:2152:ptlrpc_server_handle_req_in()) got req x1862456998028864 00000100:00100000:1.0:1776178133.858191:0:20762:0:(service.c:2152:ptlrpc_server_handle_req_in()) got req x1862456998029120 00000100:00100000:1.0:1776178133.858204:0:20762:0:(nrs_fifo.c:179:nrs_fifo_req_get()) NRS start fifo request from 12345-0@lo, seq: 436 00000100:00100000:1.0:1776178133.858208:0:20762:0:(service.c:2324:ptlrpc_server_handle_request()) Handling RPC req@0000000039ff8335 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_013:lustre-MDT0000-mdtlov_UUID+5:3353:x1862456998029056:12345-0@lo:400:kworker/u8:0.0 00000100:00100000:1.0:1776178133.858235:0:20762:0:(service.c:2372:ptlrpc_server_handle_request()) Handled RPC req@0000000039ff8335 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_013:lustre-MDT0000-mdtlov_UUID+5:3353:x1862456998029056:12345-0@lo:400:kworker/u8:0.0 Request processed in 26us (614us total) trans 0 rc 0/0 00000100:00100000:1.0:1776178133.858241:0:20762:0:(nrs_fifo.c:241:nrs_fifo_req_stop()) NRS stop fifo request from 12345-0@lo, seq: 436 00000100:00100000:1.0:1776178133.858250:0:20762:0:(nrs_fifo.c:179:nrs_fifo_req_get()) NRS start fifo request from 12345-0@lo, seq: 437 00000100:00100000:1.0:1776178133.858253:0:20762:0:(service.c:2324:ptlrpc_server_handle_request()) Handling RPC req@0000000022d4dc29 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_013:lustre-MDT0001-mdtlov_UUID+5:3353:x1862456998028864:12345-0@lo:400:kworker/u8:0.0 00000100:00100000:1.0:1776178133.858271:0:20762:0:(service.c:2372:ptlrpc_server_handle_request()) Handled RPC req@0000000022d4dc29 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_013:lustre-MDT0001-mdtlov_UUID+5:3353:x1862456998028864:12345-0@lo:400:kworker/u8:0.0 Request processed in 18us (573us total) trans 0 rc 0/0 00000100:00100000:1.0:1776178133.858276:0:20762:0:(nrs_fifo.c:241:nrs_fifo_req_stop()) NRS stop fifo request from 12345-0@lo, seq: 437 00000100:00100000:1.0:1776178133.858284:0:20762:0:(nrs_fifo.c:179:nrs_fifo_req_get()) NRS start fifo request from 12345-0@lo, seq: 438 00000100:00100000:1.0:1776178133.858286:0:20762:0:(service.c:2324:ptlrpc_server_handle_request()) Handling RPC req@00000000699e66b5 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_013:lustre-MDT0000-mdtlov_UUID+5:3353:x1862456998029120:12345-0@lo:400:kworker/u8:0.0 00000100:00100000:1.0:1776178133.858318:0:20762:0:(service.c:2372:ptlrpc_server_handle_request()) Handled RPC req@00000000699e66b5 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_013:lustre-MDT0000-mdtlov_UUID+5:3353:x1862456998029120:12345-0@lo:400:kworker/u8:0.0 Request processed in 17us (577us total) trans 0 rc 0/0 00000100:00100000:1.0:1776178133.858323:0:20762:0:(nrs_fifo.c:241:nrs_fifo_req_stop()) NRS stop fifo request from 12345-0@lo, seq: 438 00000100:00100000:1.0:1776178133.858522:0:3355:0:(client.c:1733:ptlrpc_send_new_req()) Sending RPC req@0000000021b0d516 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_02:lustre-MDT0000-lwp-OST0001_UUID:3355:1862456998028672:0@lo:400:kworker/u8:0.0 00000100:00100000:1.0:1776178133.858554:0:3355:0:(events.c:361:request_in_callback()) peer: 12345-0@lo (source: 12345-0@lo) 00000100:00100000:1.0:1776178133.858572:0:3355:0:(client.c:1733:ptlrpc_send_new_req()) Sending RPC req@00000000d6e5d9ba pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_02:lustre-MDT0001-mdtlov_UUID:3355:1862456998028928:0@lo:400:kworker/u8:0.0 00000100:00100000:1.0:1776178133.858591:0:3355:0:(events.c:361:request_in_callback()) peer: 12345-0@lo (source: 12345-0@lo) 00000100:00100000:1.0:1776178133.858603:0:3355:0:(client.c:1733:ptlrpc_send_new_req()) Sending RPC req@00000000ed90ff75 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_02:lustre-MDT0000-lwp-MDT0000_UUID:3355:1862456998029184:0@lo:400:kworker/u8:0.0 00000100:00100000:1.0:1776178133.858620:0:3355:0:(events.c:361:request_in_callback()) peer: 12345-0@lo (source: 12345-0@lo) 00000100:00100000:1.0:1776178133.858635:0:3355:0:(ptlrpcd.c:415:ptlrpcd_check()) transfer 3 async RPCs [3->2] 00000100:00100000:1.0:1776178133.858640:0:3355:0:(client.c:1733:ptlrpc_send_new_req()) Sending RPC req@0000000023562990 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_02:ac61f839-0e8e-4cac-9c59-08bcfbbad330:3355:1862456998028480:0@lo:400:kworker/u8:0.0 00000100:00100000:1.0:1776178133.858665:0:3355:0:(events.c:361:request_in_callback()) peer: 12345-0@lo (source: 12345-0@lo) 00000100:00100000:1.0:1776178133.858677:0:3355:0:(client.c:1733:ptlrpc_send_new_req()) Sending RPC req@00000000bdad24ac pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_02:lustre-MDT0001-lwp-OST0001_UUID:3355:1862456998028736:0@lo:400:kworker/u8:0.0 00000100:00100000:1.0:1776178133.858696:0:3355:0:(events.c:361:request_in_callback()) peer: 12345-0@lo (source: 12345-0@lo) 00000100:00100000:1.0:1776178133.858708:0:3355:0:(client.c:1733:ptlrpc_send_new_req()) Sending RPC req@00000000bcefda18 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_02:lustre-MDT0000-lwp-MDT0001_UUID:3355:1862456998028992:0@lo:400:kworker/u8:0.0 00000100:00100000:1.0:1776178133.858727:0:3355:0:(events.c:361:request_in_callback()) peer: 12345-0@lo (source: 12345-0@lo) 00000100:00100000:1.0:1776178133.858746:0:6257:0:(service.c:2152:ptlrpc_server_handle_req_in()) got req x1862456998028672 00000100:00100000:1.0:1776178133.858764:0:6257:0:(service.c:2152:ptlrpc_server_handle_req_in()) got req x1862456998029184 00000100:00100000:1.0:1776178133.858773:0:6257:0:(service.c:2152:ptlrpc_server_handle_req_in()) got req x1862456998028736 00000100:00100000:1.0:1776178133.858786:0:6257:0:(service.c:2152:ptlrpc_server_handle_req_in()) got req x1862456998028992 00000100:00100000:1.0:1776178133.858796:0:6257:0:(nrs_fifo.c:179:nrs_fifo_req_get()) NRS start fifo request from 12345-0@lo, seq: 797 00000100:00100000:1.0:1776178133.858799:0:6257:0:(service.c:2324:ptlrpc_server_handle_request()) Handling RPC req@00000000f9929c4b pname:cluuid+ref:pid:xid:nid:opc:job mdt00_002:lustre-MDT0000-lwp-OST0001_UUID+5:3355:x1862456998028672:12345-0@lo:400:kworker/u8:0.0 00000100:00100000:1.0:1776178133.858832:0:6257:0:(service.c:2372:ptlrpc_server_handle_request()) Handled RPC req@00000000f9929c4b pname:cluuid+ref:pid:xid:nid:opc:job mdt00_002:lustre-MDT0000-lwp-OST0001_UUID+5:3355:x1862456998028672:12345-0@lo:400:kworker/u8:0.0 Request processed in 32us (278us total) trans 0 rc 0/0 00000100:00100000:1.0:1776178133.858838:0:6257:0:(nrs_fifo.c:241:nrs_fifo_req_stop()) NRS stop fifo request from 12345-0@lo, seq: 797 00000100:00100000:1.0:1776178133.858849:0:6257:0:(nrs_fifo.c:179:nrs_fifo_req_get()) NRS start fifo request from 12345-0@lo, seq: 798 00000100:00100000:1.0:1776178133.858852:0:6257:0:(service.c:2324:ptlrpc_server_handle_request()) Handling RPC req@00000000cb1b73cb pname:cluuid+ref:pid:xid:nid:opc:job mdt00_002:lustre-MDT0000-lwp-MDT0000_UUID+5:3355:x1862456998029184:12345-0@lo:400:kworker/u8:0.0 00000100:00100000:1.0:1776178133.858875:0:6257:0:(service.c:2372:ptlrpc_server_handle_request()) Handled RPC req@00000000cb1b73cb pname:cluuid+ref:pid:xid:nid:opc:job mdt00_002:lustre-MDT0000-lwp-MDT0000_UUID+5:3355:x1862456998029184:12345-0@lo:400:kworker/u8:0.0 Request processed in 22us (255us total) trans 0 rc 0/0 00000100:00100000:1.0:1776178133.858880:0:6257:0:(nrs_fifo.c:241:nrs_fifo_req_stop()) NRS stop fifo request from 12345-0@lo, seq: 798 00000100:00100000:1.0:1776178133.858890:0:6257:0:(nrs_fifo.c:179:nrs_fifo_req_get()) NRS start fifo request from 12345-0@lo, seq: 799 00000100:00100000:1.0:1776178133.858892:0:6257:0:(service.c:2324:ptlrpc_server_handle_request()) Handling RPC req@00000000cc80b69e pname:cluuid+ref:pid:xid:nid:opc:job mdt00_002:lustre-MDT0001-lwp-OST0001_UUID+5:3355:x1862456998028736:12345-0@lo:400:kworker/u8:0.0 00000100:00100000:1.0:1776178133.858914:0:6257:0:(service.c:2372:ptlrpc_server_handle_request()) Handled RPC req@00000000cc80b69e pname:cluuid+ref:pid:xid:nid:opc:job mdt00_002:lustre-MDT0001-lwp-OST0001_UUID+5:3355:x1862456998028736:12345-0@lo:400:kworker/u8:0.0 Request processed in 20us (217us total) trans 0 rc 0/0 00000100:00100000:1.0:1776178133.858919:0:6257:0:(nrs_fifo.c:241:nrs_fifo_req_stop()) NRS stop fifo request from 12345-0@lo, seq: 799 00000100:00100000:1.0:1776178133.858928:0:6257:0:(nrs_fifo.c:179:nrs_fifo_req_get()) NRS start fifo request from 12345-0@lo, seq: 800 00000100:00100000:1.0:1776178133.858931:0:6257:0:(service.c:2324:ptlrpc_server_handle_request()) Handling RPC req@00000000d2194868 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_002:lustre-MDT0000-lwp-MDT0001_UUID+5:3355:x1862456998028992:12345-0@lo:400:kworker/u8:0.0 00000100:00100000:1.0:1776178133.858951:0:6257:0:(service.c:2372:ptlrpc_server_handle_request()) Handled RPC req@00000000d2194868 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_002:lustre-MDT0000-lwp-MDT0001_UUID+5:3355:x1862456998028992:12345-0@lo:400:kworker/u8:0.0 Request processed in 19us (224us total) trans 0 rc 0/0 00000100:00100000:1.0:1776178133.858956:0:6257:0:(nrs_fifo.c:241:nrs_fifo_req_stop()) NRS stop fifo request from 12345-0@lo, seq: 800 00000100:00100000:1.0:1776178133.858971:0:20759:0:(service.c:2152:ptlrpc_server_handle_req_in()) got req x1862456998028928 00000100:00100000:1.0:1776178133.858988:0:20759:0:(nrs_fifo.c:179:nrs_fifo_req_get()) NRS start fifo request from 12345-0@lo, seq: 439 00000100:00100000:1.0:1776178133.858991:0:20759:0:(service.c:2324:ptlrpc_server_handle_request()) Handling RPC req@000000007b15d0e4 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_010:lustre-MDT0001-mdtlov_UUID+5:3355:x1862456998028928:12345-0@lo:400:kworker/u8:0.0 00000100:00100000:1.0:1776178133.859016:0:20759:0:(service.c:2372:ptlrpc_server_handle_request()) Handled RPC req@000000007b15d0e4 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_010:lustre-MDT0001-mdtlov_UUID+5:3355:x1862456998028928:12345-0@lo:400:kworker/u8:0.0 Request processed in 23us (425us total) trans 0 rc 0/0 00000100:00100000:1.0:1776178133.859021:0:20759:0:(nrs_fifo.c:241:nrs_fifo_req_stop()) NRS stop fifo request from 12345-0@lo, seq: 439 00000100:00100000:1.0:1776178133.859093:0:3355:0:(client.c:2215:ptlrpc_check_set()) Completed RPC req@00000000bdad24ac pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_02:lustre-MDT0001-lwp-OST0001_UUID:3355:1862456998028736:0@lo:400:kworker/u8:0.0 00000100:00100000:1.0:1776178133.859102:0:3355:0:(client.c:2215:ptlrpc_check_set()) Completed RPC req@00000000bcefda18 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_02:lustre-MDT0000-lwp-MDT0001_UUID:3355:1862456998028992:0@lo:400:kworker/u8:0.0 00000100:00100000:1.0:1776178133.859110:0:3355:0:(client.c:2215:ptlrpc_check_set()) Completed RPC req@0000000021b0d516 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_02:lustre-MDT0000-lwp-OST0001_UUID:3355:1862456998028672:0@lo:400:kworker/u8:0.0 00000100:00100000:1.0:1776178133.859117:0:3355:0:(client.c:2215:ptlrpc_check_set()) Completed RPC req@00000000d6e5d9ba pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_02:lustre-MDT0001-mdtlov_UUID:3355:1862456998028928:0@lo:400:kworker/u8:0.0 00000100:00100000:1.0:1776178133.859124:0:3355:0:(client.c:2215:ptlrpc_check_set()) Completed RPC req@00000000ed90ff75 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_02:lustre-MDT0000-lwp-MDT0000_UUID:3355:1862456998029184:0@lo:400:kworker/u8:0.0 00000100:00100000:2.0:1776178133.860694:0:3356:0:(client.c:1733:ptlrpc_send_new_req()) Sending RPC req@000000005d3a2b34 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_03:lustre-MDT0000-mdtlov_UUID:3356:1862456998029248:0@lo:13:osp-pre-1-0.0 00000100:00100000:2.0:1776178133.860747:0:3356:0:(events.c:361:request_in_callback()) peer: 12345-0@lo (source: 12345-0@lo) 00000100:00100000:3.0:1776178133.861561:0:20137:0:(service.c:2152:ptlrpc_server_handle_req_in()) got req x1862456998028480 00000100:00100000:3.0:1776178133.861612:0:20137:0:(nrs_fifo.c:179:nrs_fifo_req_get()) NRS start fifo request from 12345-0@lo, seq: 127 00000100:00100000:3.0:1776178133.861617:0:20137:0:(service.c:2324:ptlrpc_server_handle_request()) Handling RPC req@0000000076ad720a pname:cluuid+ref:pid:xid:nid:opc:job ll_mgs_0000:ac61f839-0e8e-4cac-9c59-08bcfbbad330+23:3355:x1862456998028480:12345-0@lo:400:kworker/u8:0.0 00000100:00100000:3.0:1776178133.861674:0:20137:0:(service.c:2372:ptlrpc_server_handle_request()) Handled RPC req@0000000076ad720a pname:cluuid+ref:pid:xid:nid:opc:job ll_mgs_0000:ac61f839-0e8e-4cac-9c59-08bcfbbad330+23:3355:x1862456998028480:12345-0@lo:400:kworker/u8:0.0 Request processed in 57us (3009us total) trans 0 rc 0/0 00000100:00100000:3.0:1776178133.861681:0:20137:0:(nrs_fifo.c:241:nrs_fifo_req_stop()) NRS stop fifo request from 12345-0@lo, seq: 127 00000100:00100000:3.0:1776178133.861728:0:3355:0:(client.c:2215:ptlrpc_check_set()) Completed RPC req@0000000023562990 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_02:ac61f839-0e8e-4cac-9c59-08bcfbbad330:3355:1862456998028480:0@lo:400:kworker/u8:0.0 00000100:00100000:3.0:1776178133.861879:0:3353:0:(client.c:2215:ptlrpc_check_set()) Completed RPC req@0000000038c09127 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_00:lustre-MDT0001-lwp-OST0000_UUID:3353:1862456998028608:0@lo:400:kworker/u8:0.0 00000100:00100000:3.0:1776178133.861891:0:3353:0:(client.c:2215:ptlrpc_check_set()) Completed RPC req@0000000020afd0b3 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_00:lustre-MDT0001-mdtlov_UUID:3353:1862456998028864:0@lo:400:kworker/u8:0.0 00000100:00100000:3.0:1776178133.861902:0:3353:0:(client.c:2215:ptlrpc_check_set()) Completed RPC req@0000000056ddcb09 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_00:lustre-MDT0000-mdtlov_UUID:3353:1862456998029120:0@lo:400:kworker/u8:0.0 00000100:00100000:3.0:1776178133.861912:0:3353:0:(client.c:2215:ptlrpc_check_set()) Completed RPC req@00000000339a9a57 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_00:lustre-MDT0000-lwp-OST0000_UUID:3353:1862456998028544:0@lo:400:kworker/u8:0.0 00000100:00100000:3.0:1776178133.861922:0:3353:0:(client.c:2215:ptlrpc_check_set()) Completed RPC req@00000000f0f1167e pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_00:lustre-MDT0000-mdtlov_UUID:3353:1862456998029056:0@lo:400:kworker/u8:0.0 00000100:00100000:0.0:1776178133.862877:0:8232:0:(service.c:2152:ptlrpc_server_handle_req_in()) got req x1862456998029248 00000100:00100000:0.0:1776178133.862915:0:8232:0:(nrs_fifo.c:179:nrs_fifo_req_get()) NRS start fifo request from 12345-0@lo, seq: 423 00000100:00100000:0.0:1776178133.862919:0:8232:0:(service.c:2324:ptlrpc_server_handle_request()) Handling RPC req@000000002ab0a797 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_create00:lustre-MDT0000-mdtlov_UUID+5:3356:x1862456998029248:12345-0@lo:13:osp-pre-1-0.0 00002000:00000024:0.0:1776178133.862945:0:8232:0:(ofd_obd.c:740:ofd_statfs()) blocks cached 229376 granted 491746368 pending 0 free 3923689472 avail 3692167168 00000100:00100000:0.0:1776178133.863000:0:8232:0:(service.c:2372:ptlrpc_server_handle_request()) Handled RPC req@000000002ab0a797 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_create00:lustre-MDT0000-mdtlov_UUID+5:3356:x1862456998029248:12345-0@lo:13:osp-pre-1-0.0 Request processed in 80us (2254us total) trans 0 rc 0/0 00000100:00100000:0.0:1776178133.863007:0:8232:0:(nrs_fifo.c:241:nrs_fifo_req_stop()) NRS stop fifo request from 12345-0@lo, seq: 423 00000100:00100000:0.0:1776178133.863058:0:3356:0:(client.c:2215:ptlrpc_check_set()) Completed RPC req@000000005d3a2b34 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_03:lustre-MDT0000-mdtlov_UUID:3356:1862456998029248:0@lo:13:osp-pre-1-0.0 00000100:00100000:0.0:1776178133.863104:0:3354:0:(ptlrpcd.c:415:ptlrpcd_check()) transfer 1 async RPCs [0->1] 00000100:00100000:0.0:1776178133.863114:0:3354:0:(client.c:1733:ptlrpc_send_new_req()) Sending RPC req@000000006d540857 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_01:lustre-MDT0000-mdtlov_UUID:3354:1862456998029312:0@lo:13:osp-pre-0-0.0 00000100:00100000:0.0:1776178133.863144:0:3354:0:(events.c:361:request_in_callback()) peer: 12345-0@lo (source: 12345-0@lo) 00000100:00100000:0.0:1776178133.863170:0:8231:0:(service.c:2152:ptlrpc_server_handle_req_in()) got req x1862456998029312 00000100:00100000:0.0:1776178133.863191:0:8231:0:(nrs_fifo.c:179:nrs_fifo_req_get()) NRS start fifo request from 12345-0@lo, seq: 424 00000100:00100000:0.0:1776178133.863194:0:8231:0:(service.c:2324:ptlrpc_server_handle_request()) Handling RPC req@000000009c55a259 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_create00:lustre-MDT0000-mdtlov_UUID+5:3354:x1862456998029312:12345-0@lo:13:osp-pre-0-0.0 00002000:00000024:0.0:1776178133.863269:0:8231:0:(ofd_obd.c:740:ofd_statfs()) blocks cached 315392 granted 484883776 pending 0 free 3923689472 avail 3692167168 00000100:00100000:0.0:1776178133.863300:0:8231:0:(service.c:2372:ptlrpc_server_handle_request()) Handled RPC req@000000009c55a259 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_create00:lustre-MDT0000-mdtlov_UUID+5:3354:x1862456998029312:12345-0@lo:13:osp-pre-0-0.0 Request processed in 105us (156us total) trans 0 rc 0/0 00000100:00100000:2.0:1776178133.863972:0:3354:0:(client.c:2215:ptlrpc_check_set()) Completed RPC req@000000006d540857 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_01:lustre-MDT0000-mdtlov_UUID:3354:1862456998029312:0@lo:13:osp-pre-0-0.0 00000100:00100000:0.0:1776178133.868304:0:8231:0:(nrs_fifo.c:241:nrs_fifo_req_stop()) NRS stop fifo request from 12345-0@lo, seq: 424 00000100:00100000:2.0:1776178133.869292:0:3354:0:(client.c:1733:ptlrpc_send_new_req()) Sending RPC req@000000006466ee18 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_01:lustre-MDT0001-mdtlov_UUID:3354:1862456998029376:0@lo:41:osp-pre-0-1.0 00000100:00100000:2.0:1776178133.869376:0:3354:0:(events.c:361:request_in_callback()) peer: 12345-0@lo (source: 12345-0@lo) 00000100:00100000:3.0:1776178133.871641:0:15213:0:(service.c:2152:ptlrpc_server_handle_req_in()) got req x1862456998029376 00000100:00100000:3.0:1776178133.871716:0:15213:0:(nrs_fifo.c:179:nrs_fifo_req_get()) NRS start fifo request from 12345-0@lo, seq: 637 00000100:00100000:3.0:1776178133.871723:0:15213:0:(service.c:2324:ptlrpc_server_handle_request()) Handling RPC req@0000000080e4108d pname:cluuid+ref:pid:xid:nid:opc:job mdt_out00_003:lustre-MDT0001-mdtlov_UUID+6:3354:x1862456998029376:12345-0@lo:41:osp-pre-0-1.0 00000004:00000024:3.0:1776178133.871760:0:15213:0:(mdt_handler.c:548:mdt_statfs()) blocks cached 0 granted 2097152 pending 0 free 1446121472 avail 1316659200 00000100:00100000:3.0:1776178133.871845:0:15213:0:(service.c:2372:ptlrpc_server_handle_request()) Handled RPC req@0000000080e4108d pname:cluuid+ref:pid:xid:nid:opc:job mdt_out00_003:lustre-MDT0001-mdtlov_UUID+6:3354:x1862456998029376:12345-0@lo:41:osp-pre-0-1.0 Request processed in 121us (2468us total) trans 0 rc 0/0 00000100:00100000:3.0:1776178133.871851:0:15213:0:(nrs_fifo.c:241:nrs_fifo_req_stop()) NRS stop fifo request from 12345-0@lo, seq: 637 00000100:00100000:1.0:1776178133.871948:0:3354:0:(client.c:2215:ptlrpc_check_set()) Completed RPC req@000000006466ee18 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_01:lustre-MDT0001-mdtlov_UUID:3354:1862456998029376:0@lo:41:osp-pre-0-1.0 00010000:00010000:1.0:1776178134.433087:0:6244:0:(ldlm_lockd.c:2792:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:3.0:1776178134.817564:0:3346:0:(events.c:361:request_in_callback()) peer: 12345-192.168.204.3@tcp (source: 12345-192.168.204.3@tcp) 00000100:00100000:3.0:1776178134.817610:0:3346:0:(events.c:361:request_in_callback()) peer: 12345-192.168.204.3@tcp (source: 12345-192.168.204.3@tcp) 00000100:00100000:3.0:1776178134.817653:0:3346:0:(events.c:361:request_in_callback()) peer: 12345-192.168.204.3@tcp (source: 12345-192.168.204.3@tcp) 00000100:00100000:3.0:1776178134.817682:0:3346:0:(events.c:361:request_in_callback()) peer: 12345-192.168.204.3@tcp (source: 12345-192.168.204.3@tcp) 00000100:00100000:3.0:1776178134.817713:0:3346:0:(events.c:361:request_in_callback()) peer: 12345-192.168.204.3@tcp (source: 12345-192.168.204.3@tcp) 00000100:00100000:3.0:1776178134.817753:0:11281:0:(service.c:2152:ptlrpc_server_handle_req_in()) got req x1862456979083840 00000100:00100000:3.0:1776178134.817802:0:11281:0:(service.c:2152:ptlrpc_server_handle_req_in()) got req x1862456979083904 00000100:00100000:3.0:1776178134.817821:0:11281:0:(nrs_fifo.c:179:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.204.3@tcp, seq: 801 00000100:00100000:3.0:1776178134.817828:0:11281:0:(service.c:2324:ptlrpc_server_handle_request()) Handling RPC req@00000000f0f1167e pname:cluuid+ref:pid:xid:nid:opc:job mdt00_004:b50963bd-8eba-4432-9a54-8888d00e3fa1+21:2270:x1862456979083840:12345-192.168.204.3@tcp:400:kworker/u8:3.0 00000100:00100000:3.0:1776178134.817899:0:11281:0:(service.c:2372:ptlrpc_server_handle_request()) Handled RPC req@00000000f0f1167e pname:cluuid+ref:pid:xid:nid:opc:job mdt00_004:b50963bd-8eba-4432-9a54-8888d00e3fa1+21:2270:x1862456979083840:12345-192.168.204.3@tcp:400:kworker/u8:3.0 Request processed in 70us (353us total) trans 0 rc 0/0 00000100:00100000:3.0:1776178134.817907:0:11281:0:(nrs_fifo.c:241:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.204.3@tcp, seq: 801 00000100:00100000:3.0:1776178134.817924:0:11281:0:(nrs_fifo.c:179:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.204.3@tcp, seq: 802 00000100:00100000:3.0:1776178134.817927:0:11281:0:(service.c:2324:ptlrpc_server_handle_request()) Handling RPC req@000000001982f085 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_004:b50963bd-8eba-4432-9a54-8888d00e3fa1+6:2270:x1862456979083904:12345-192.168.204.3@tcp:400:kworker/u8:3.0 00000100:00100000:0.0:1776178134.818106:0:20764:0:(service.c:2152:ptlrpc_server_handle_req_in()) got req x1862456979083968 00000100:00100000:0.0:1776178134.818134:0:20764:0:(service.c:2152:ptlrpc_server_handle_req_in()) got req x1862456979084032 00000100:00100000:0.0:1776178134.818151:0:20764:0:(nrs_fifo.c:179:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.204.3@tcp, seq: 440 00000100:00100000:0.0:1776178134.818154:0:20764:0:(service.c:2324:ptlrpc_server_handle_request()) Handling RPC req@0000000056ddcb09 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_015:b50963bd-8eba-4432-9a54-8888d00e3fa1+9:2270:x1862456979083968:12345-192.168.204.3@tcp:400:kworker/u8:3.0 00000100:00100000:0.0:1776178134.818184:0:20764:0:(service.c:2372:ptlrpc_server_handle_request()) Handled RPC req@0000000056ddcb09 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_015:b50963bd-8eba-4432-9a54-8888d00e3fa1+9:2270:x1862456979083968:12345-192.168.204.3@tcp:400:kworker/u8:3.0 Request processed in 28us (574us total) trans 0 rc 0/0 00000100:00100000:0.0:1776178134.818190:0:20764:0:(nrs_fifo.c:241:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.204.3@tcp, seq: 440 00000100:00100000:0.0:1776178134.818202:0:20764:0:(nrs_fifo.c:179:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.204.3@tcp, seq: 441 00000100:00100000:0.0:1776178134.818205:0:20764:0:(service.c:2324:ptlrpc_server_handle_request()) Handling RPC req@000000004420855e pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_015:b50963bd-8eba-4432-9a54-8888d00e3fa1+8:2270:x1862456979084032:12345-192.168.204.3@tcp:400:kworker/u8:3.0 00000100:00100000:0.0:1776178134.818225:0:20764:0:(service.c:2372:ptlrpc_server_handle_request()) Handled RPC req@000000004420855e pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_015:b50963bd-8eba-4432-9a54-8888d00e3fa1+8:2270:x1862456979084032:12345-192.168.204.3@tcp:400:kworker/u8:3.0 Request processed in 20us (513us total) trans 0 rc 0/0 00000100:00100000:0.0:1776178134.818230:0:20764:0:(nrs_fifo.c:241:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.204.3@tcp, seq: 441 00000100:00100000:3.0:1776178134.818737:0:11281:0:(service.c:2372:ptlrpc_server_handle_request()) Handled RPC req@000000001982f085 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_004:b50963bd-8eba-4432-9a54-8888d00e3fa1+6:2270:x1862456979083904:12345-192.168.204.3@tcp:400:kworker/u8:3.0 Request processed in 807us (1054us total) trans 0 rc 0/0 00000100:00100000:3.0:1776178134.818744:0:11281:0:(nrs_fifo.c:241:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.204.3@tcp, seq: 802 00000100:00100000:2.0:1776178134.818819:0:20140:0:(service.c:2152:ptlrpc_server_handle_req_in()) got req x1862456979083776 00000100:00100000:2.0:1776178134.818852:0:20140:0:(nrs_fifo.c:179:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.204.3@tcp, seq: 128 00000100:00100000:2.0:1776178134.818857:0:20140:0:(service.c:2324:ptlrpc_server_handle_request()) Handling RPC req@00000000d4927be7 pname:cluuid+ref:pid:xid:nid:opc:job ll_mgs_0002:5b5c66b1-1a2f-4651-8548-0c75b9a38a94+9:2270:x1862456979083776:12345-192.168.204.3@tcp:400:kworker/u8:3.0 00000100:00100000:2.0:1776178134.818883:0:20140:0:(service.c:2372:ptlrpc_server_handle_request()) Handled RPC req@00000000d4927be7 pname:cluuid+ref:pid:xid:nid:opc:job ll_mgs_0002:5b5c66b1-1a2f-4651-8548-0c75b9a38a94+9:2270:x1862456979083776:12345-192.168.204.3@tcp:400:kworker/u8:3.0 Request processed in 25us (1230us total) trans 0 rc 0/0 00000100:00100000:2.0:1776178134.818889:0:20140:0:(nrs_fifo.c:241:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.204.3@tcp, seq: 128 00000100:00100000:0.0:1776178135.392842:0:3356:0:(ptlrpcd.c:415:ptlrpcd_check()) transfer 1 async RPCs [2->3] 00000100:00100000:0.0:1776178135.392866:0:3356:0:(client.c:1733:ptlrpc_send_new_req()) Sending RPC req@00000000e2b18a6d pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_03:lustre-MDT0001-mdtlov_UUID:3356:1862456998029440:0@lo:13:osp-pre-1-1.0 00000100:00100000:0.0:1776178135.392942:0:3356:0:(events.c:361:request_in_callback()) peer: 12345-0@lo (source: 12345-0@lo) 00000100:00100000:1.0:1776178135.394506:0:8231:0:(service.c:2152:ptlrpc_server_handle_req_in()) got req x1862456998029440 00000100:00100000:1.0:1776178135.394580:0:8231:0:(nrs_fifo.c:179:nrs_fifo_req_get()) NRS start fifo request from 12345-0@lo, seq: 425 00000100:00100000:1.0:1776178135.394587:0:8231:0:(service.c:2324:ptlrpc_server_handle_request()) Handling RPC req@0000000095214b8e pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_create00:lustre-MDT0001-mdtlov_UUID+5:3356:x1862456998029440:12345-0@lo:13:osp-pre-1-1.0 00002000:00000024:1.0:1776178135.394612:0:8231:0:(ofd_obd.c:740:ofd_statfs()) blocks cached 229376 granted 491746368 pending 0 free 3923689472 avail 3692167168 00000100:00100000:1.0:1776178135.394683:0:8231:0:(service.c:2372:ptlrpc_server_handle_request()) Handled RPC req@0000000095214b8e pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_create00:lustre-MDT0001-mdtlov_UUID+5:3356:x1862456998029440:12345-0@lo:13:osp-pre-1-1.0 Request processed in 95us (1752us total) trans 0 rc 0/0 00000100:00100000:1.0:1776178135.394693:0:8231:0:(nrs_fifo.c:241:nrs_fifo_req_stop()) NRS stop fifo request from 12345-0@lo, seq: 425 00000100:00100000:0.0:1776178135.395311:0:3356:0:(client.c:2215:ptlrpc_check_set()) Completed RPC req@00000000e2b18a6d pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_03:lustre-MDT0001-mdtlov_UUID:3356:1862456998029440:0@lo:13:osp-pre-1-1.0 00010000:00010000:1.0:1776178135.459888:0:6245:0:(ldlm_lockd.c:2792:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:0.0:1776178135.905223:0:3356:0:(client.c:1733:ptlrpc_send_new_req()) Sending RPC req@000000005112bc3b pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_03:lustre-MDT0001-mdtlov_UUID:3356:1862456998029504:0@lo:13:osp-pre-0-1.0 00000100:00100000:0.0:1776178135.905305:0:3356:0:(events.c:361:request_in_callback()) peer: 12345-0@lo (source: 12345-0@lo) 00000100:00100000:1.0:1776178135.907922:0:8232:0:(service.c:2152:ptlrpc_server_handle_req_in()) got req x1862456998029504 00000100:00100000:1.0:1776178135.908017:0:8232:0:(nrs_fifo.c:179:nrs_fifo_req_get()) NRS start fifo request from 12345-0@lo, seq: 426 00000100:00100000:1.0:1776178135.908024:0:8232:0:(service.c:2324:ptlrpc_server_handle_request()) Handling RPC req@000000004e8fbd9b pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_create00:lustre-MDT0001-mdtlov_UUID+5:3356:x1862456998029504:12345-0@lo:13:osp-pre-0-1.0 00002000:00000024:1.0:1776178135.908096:0:8232:0:(ofd_obd.c:740:ofd_statfs()) blocks cached 315392 granted 484883776 pending 0 free 3923689472 avail 3692167168 00000100:00100000:1.0:1776178135.908188:0:8232:0:(service.c:2372:ptlrpc_server_handle_request()) Handled RPC req@000000004e8fbd9b pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_create00:lustre-MDT0001-mdtlov_UUID+5:3356:x1862456998029504:12345-0@lo:13:osp-pre-0-1.0 Request processed in 164us (2884us total) trans 0 rc 0/0 00000100:00100000:1.0:1776178135.908195:0:8232:0:(nrs_fifo.c:241:nrs_fifo_req_stop()) NRS stop fifo request from 12345-0@lo, seq: 426 00000100:00100000:2.0:1776178135.910950:0:3356:0:(client.c:2215:ptlrpc_check_set()) Completed RPC req@000000005112bc3b pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_03:lustre-MDT0001-mdtlov_UUID:3356:1862456998029504:0@lo:13:osp-pre-0-1.0 00010000:00010000:1.0:1776178136.480392:0:9225:0:(ldlm_lockd.c:2792:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00010000:00010000:2.0:1776178137.504447:0:9224:0:(ldlm_lockd.c:2792:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00010000:00010000:1.0:1776178138.528805:0:6244:0:(ldlm_lockd.c:2792:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:3.0:1776178138.977275:0:3354:0:(ptlrpcd.c:415:ptlrpcd_check()) transfer 1 async RPCs [0->1] 00000100:00100000:3.0:1776178138.977299:0:3354:0:(client.c:1733:ptlrpc_send_new_req()) Sending RPC req@000000006466ee18 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_01:lustre-MDT0001-mdtlov_UUID:3354:1862456998029568:0@lo:41:osp-pre-0-1.0 00000100:00100000:3.0:1776178138.977368:0:3354:0:(events.c:361:request_in_callback()) peer: 12345-0@lo (source: 12345-0@lo) 00000100:00100000:0.0:1776178138.977847:0:6260:0:(service.c:2152:ptlrpc_server_handle_req_in()) got req x1862456998029568 00000100:00100000:0.0:1776178138.977894:0:6260:0:(nrs_fifo.c:179:nrs_fifo_req_get()) NRS start fifo request from 12345-0@lo, seq: 638 00000100:00100000:0.0:1776178138.977902:0:6260:0:(service.c:2324:ptlrpc_server_handle_request()) Handling RPC req@00000000456acdaf pname:cluuid+ref:pid:xid:nid:opc:job mdt_out00_000:lustre-MDT0001-mdtlov_UUID+6:3354:x1862456998029568:12345-0@lo:41:osp-pre-0-1.0 00000100:00100000:1.0:1776178138.977923:0:3355:0:(client.c:1733:ptlrpc_send_new_req()) Sending RPC req@00000000fed3e037 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_02:lustre-MDT0000-lwp-OST0000_UUID:3355:1862456998029696:0@lo:400:kworker/u8:2.0 00000004:00000024:0.0:1776178138.977935:0:6260:0:(mdt_handler.c:548:mdt_statfs()) blocks cached 0 granted 2097152 pending 0 free 1446121472 avail 1316659200 00000100:00100000:0.0:1776178138.977992:0:6260:0:(service.c:2372:ptlrpc_server_handle_request()) Handled RPC req@00000000456acdaf pname:cluuid+ref:pid:xid:nid:opc:job mdt_out00_000:lustre-MDT0001-mdtlov_UUID+6:3354:x1862456998029568:12345-0@lo:41:osp-pre-0-1.0 Request processed in 91us (626us total) trans 0 rc 0/0 00000100:00100000:0.0:1776178138.978001:0:6260:0:(nrs_fifo.c:241:nrs_fifo_req_stop()) NRS stop fifo request from 12345-0@lo, seq: 638 00000100:00100000:3.0:1776178138.978020:0:3354:0:(client.c:1733:ptlrpc_send_new_req()) Sending RPC req@000000007d5bffd0 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_01:ac61f839-0e8e-4cac-9c59-08bcfbbad330:3354:1862456998029632:0@lo:400:kworker/u8:2.0 00000100:00100000:3.0:1776178138.978080:0:3354:0:(events.c:361:request_in_callback()) peer: 12345-0@lo (source: 12345-0@lo) 00000100:00100000:3.0:1776178138.978102:0:3354:0:(client.c:1733:ptlrpc_send_new_req()) Sending RPC req@00000000e3063d85 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_01:lustre-MDT0001-lwp-OST0001_UUID:3354:1862456998029888:0@lo:400:kworker/u8:2.0 00000100:00100000:0.0:1776178138.978112:0:20137:0:(service.c:2152:ptlrpc_server_handle_req_in()) got req x1862456998029632 00000100:00100000:0.0:1776178138.978137:0:20137:0:(nrs_fifo.c:179:nrs_fifo_req_get()) NRS start fifo request from 12345-0@lo, seq: 129 00000100:00100000:0.0:1776178138.978140:0:20137:0:(service.c:2324:ptlrpc_server_handle_request()) Handling RPC req@00000000286b1e8d pname:cluuid+ref:pid:xid:nid:opc:job ll_mgs_0000:ac61f839-0e8e-4cac-9c59-08bcfbbad330+23:3354:x1862456998029632:12345-0@lo:400:kworker/u8:2.0 00000100:00100000:3.0:1776178138.978153:0:3354:0:(events.c:361:request_in_callback()) peer: 12345-0@lo (source: 12345-0@lo) 00000100:00100000:3.0:1776178138.978179:0:3354:0:(client.c:1733:ptlrpc_send_new_req()) Sending RPC req@000000006b0ae3d8 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_01:lustre-MDT0000-mdtlov_UUID:3354:1862456998030144:0@lo:400:kworker/u8:2.0 00000100:00100000:0.0:1776178138.978183:0:20137:0:(service.c:2372:ptlrpc_server_handle_request()) Handled RPC req@00000000286b1e8d pname:cluuid+ref:pid:xid:nid:opc:job ll_mgs_0000:ac61f839-0e8e-4cac-9c59-08bcfbbad330+23:3354:x1862456998029632:12345-0@lo:400:kworker/u8:2.0 Request processed in 39us (121us total) trans 0 rc 0/0 00000100:00100000:0.0:1776178138.978191:0:20137:0:(nrs_fifo.c:241:nrs_fifo_req_stop()) NRS stop fifo request from 12345-0@lo, seq: 129 00000100:00100000:3.0:1776178138.978212:0:3354:0:(events.c:361:request_in_callback()) peer: 12345-0@lo (source: 12345-0@lo) 00000100:00100000:0.0:1776178138.978255:0:6255:0:(service.c:2152:ptlrpc_server_handle_req_in()) got req x1862456998029888 00000100:00100000:3.0:1776178138.978256:0:3354:0:(client.c:2215:ptlrpc_check_set()) Completed RPC req@000000006466ee18 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_01:lustre-MDT0001-mdtlov_UUID:3354:1862456998029568:0@lo:41:osp-pre-0-1.0 00000100:00100000:3.0:1776178138.978282:0:3354:0:(client.c:1733:ptlrpc_send_new_req()) Sending RPC req@000000006d540857 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_01:lustre-MDT0000-mdtlov_UUID:3354:1862456998030400:0@lo:13:osp-pre-0-0.0 00000100:00100000:0.0:1776178138.978286:0:6255:0:(nrs_fifo.c:179:nrs_fifo_req_get()) NRS start fifo request from 12345-0@lo, seq: 803 00000100:00100000:0.0:1776178138.978292:0:6255:0:(service.c:2324:ptlrpc_server_handle_request()) Handling RPC req@00000000e2b18a6d pname:cluuid+ref:pid:xid:nid:opc:job mdt00_000:lustre-MDT0001-lwp-OST0001_UUID+5:3354:x1862456998029888:12345-0@lo:400:kworker/u8:2.0 00000100:00100000:3.0:1776178138.978313:0:3354:0:(events.c:361:request_in_callback()) peer: 12345-0@lo (source: 12345-0@lo) 00000100:00100000:0.0:1776178138.978331:0:6255:0:(service.c:2372:ptlrpc_server_handle_request()) Handled RPC req@00000000e2b18a6d pname:cluuid+ref:pid:xid:nid:opc:job mdt00_000:lustre-MDT0001-lwp-OST0001_UUID+5:3354:x1862456998029888:12345-0@lo:400:kworker/u8:2.0 Request processed in 37us (179us total) trans 0 rc 0/0 00000100:00100000:0.0:1776178138.978339:0:6255:0:(nrs_fifo.c:241:nrs_fifo_req_stop()) NRS stop fifo request from 12345-0@lo, seq: 803 00000100:00100000:3.0:1776178138.978344:0:3354:0:(client.c:2215:ptlrpc_check_set()) Completed RPC req@000000007d5bffd0 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_01:ac61f839-0e8e-4cac-9c59-08bcfbbad330:3354:1862456998029632:0@lo:400:kworker/u8:2.0 00000100:00100000:3.0:1776178138.978358:0:3354:0:(client.c:2215:ptlrpc_check_set()) Completed RPC req@00000000e3063d85 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_01:lustre-MDT0001-lwp-OST0001_UUID:3354:1862456998029888:0@lo:400:kworker/u8:2.0 00000100:00100000:0.0:1776178138.978361:0:6260:0:(service.c:2152:ptlrpc_server_handle_req_in()) got req x1862456998030144 00000100:00100000:0.0:1776178138.978385:0:6260:0:(nrs_fifo.c:179:nrs_fifo_req_get()) NRS start fifo request from 12345-0@lo, seq: 639 00000100:00100000:0.0:1776178138.978390:0:6260:0:(service.c:2324:ptlrpc_server_handle_request()) Handling RPC req@00000000ed90ff75 pname:cluuid+ref:pid:xid:nid:opc:job mdt_out00_000:lustre-MDT0000-mdtlov_UUID+5:3354:x1862456998030144:12345-0@lo:400:kworker/u8:2.0 00000100:00100000:3.0:1776178138.978391:0:3354:0:(ptlrpcd.c:415:ptlrpcd_check()) transfer 3 async RPCs [0->1] 00000100:00100000:3.0:1776178138.978400:0:3354:0:(client.c:1733:ptlrpc_send_new_req()) Sending RPC req@0000000013603266 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_01:lustre-MDT0000-lwp-OST0001_UUID:3354:1862456998029824:0@lo:400:kworker/u8:2.0 00000100:00100000:0.0:1776178138.978430:0:6260:0:(service.c:2372:ptlrpc_server_handle_request()) Handled RPC req@00000000ed90ff75 pname:cluuid+ref:pid:xid:nid:opc:job mdt_out00_000:lustre-MDT0000-mdtlov_UUID+5:3354:x1862456998030144:12345-0@lo:400:kworker/u8:2.0 Request processed in 40us (221us total) trans 0 rc 0/0 00000100:00100000:3.0:1776178138.978438:0:3354:0:(events.c:361:request_in_callback()) peer: 12345-0@lo (source: 12345-0@lo) 00000100:00100000:0.0:1776178138.978442:0:6260:0:(nrs_fifo.c:241:nrs_fifo_req_stop()) NRS stop fifo request from 12345-0@lo, seq: 639 00000100:00100000:3.0:1776178138.978460:0:3354:0:(client.c:1733:ptlrpc_send_new_req()) Sending RPC req@0000000052a7c870 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_01:lustre-MDT0000-lwp-MDT0001_UUID:3354:1862456998030080:0@lo:400:kworker/u8:2.0 00000100:00100000:0.0:1776178138.978475:0:11281:0:(service.c:2152:ptlrpc_server_handle_req_in()) got req x1862456998029824 00000100:00100000:3.0:1776178138.978496:0:3354:0:(events.c:361:request_in_callback()) peer: 12345-0@lo (source: 12345-0@lo) 00000100:00100000:0.0:1776178138.978500:0:11281:0:(nrs_fifo.c:179:nrs_fifo_req_get()) NRS start fifo request from 12345-0@lo, seq: 804 00000100:00100000:0.0:1776178138.978505:0:11281:0:(service.c:2324:ptlrpc_server_handle_request()) Handling RPC req@000000007d5bffd0 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_004:lustre-MDT0000-lwp-OST0001_UUID+5:3354:x1862456998029824:12345-0@lo:400:kworker/u8:2.0 00000100:00100000:3.0:1776178138.978516:0:3354:0:(client.c:1733:ptlrpc_send_new_req()) Sending RPC req@000000001f170bf7 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_01:lustre-MDT0000-lwp-MDT0000_UUID:3354:1862456998030336:0@lo:400:kworker/u8:2.0 00000100:00100000:0.0:1776178138.978540:0:11281:0:(service.c:2372:ptlrpc_server_handle_request()) Handled RPC req@000000007d5bffd0 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_004:lustre-MDT0000-lwp-OST0001_UUID+5:3354:x1862456998029824:12345-0@lo:400:kworker/u8:2.0 Request processed in 34us (106us total) trans 0 rc 0/0 00000100:00100000:0.0:1776178138.978551:0:11281:0:(nrs_fifo.c:241:nrs_fifo_req_stop()) NRS stop fifo request from 12345-0@lo, seq: 804 00000100:00100000:3.0:1776178138.978551:0:3354:0:(events.c:361:request_in_callback()) peer: 12345-0@lo (source: 12345-0@lo) 00000100:00100000:0.0:1776178138.978568:0:11281:0:(service.c:2152:ptlrpc_server_handle_req_in()) got req x1862456998030080 00000100:00100000:3.0:1776178138.978578:0:3354:0:(client.c:2215:ptlrpc_check_set()) Completed RPC req@000000006b0ae3d8 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_01:lustre-MDT0000-mdtlov_UUID:3354:1862456998030144:0@lo:400:kworker/u8:2.0 00000100:00100000:0.0:1776178138.978582:0:11281:0:(service.c:2152:ptlrpc_server_handle_req_in()) got req x1862456998030336 00000100:00100000:3.0:1776178138.978594:0:3354:0:(client.c:2215:ptlrpc_check_set()) Completed RPC req@0000000013603266 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_01:lustre-MDT0000-lwp-OST0001_UUID:3354:1862456998029824:0@lo:400:kworker/u8:2.0 00000100:00100000:0.0:1776178138.978614:0:11281:0:(nrs_fifo.c:179:nrs_fifo_req_get()) NRS start fifo request from 12345-0@lo, seq: 805 00000100:00100000:0.0:1776178138.978619:0:11281:0:(service.c:2324:ptlrpc_server_handle_request()) Handling RPC req@00000000e3063d85 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_004:lustre-MDT0000-lwp-MDT0001_UUID+5:3354:x1862456998030080:12345-0@lo:400:kworker/u8:2.0 00000100:00100000:3.0:1776178138.978625:0:6256:0:(nrs_fifo.c:179:nrs_fifo_req_get()) NRS start fifo request from 12345-0@lo, seq: 806 00000100:00100000:3.0:1776178138.978630:0:6256:0:(service.c:2324:ptlrpc_server_handle_request()) Handling RPC req@00000000339a9a57 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_001:lustre-MDT0000-lwp-MDT0000_UUID+5:3354:x1862456998030336:12345-0@lo:400:kworker/u8:2.0 00000100:00100000:0.0:1776178138.978649:0:11281:0:(service.c:2372:ptlrpc_server_handle_request()) Handled RPC req@00000000e3063d85 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_004:lustre-MDT0000-lwp-MDT0001_UUID+5:3354:x1862456998030080:12345-0@lo:400:kworker/u8:2.0 Request processed in 29us (166us total) trans 0 rc 0/0 00000100:00100000:0.0:1776178138.978656:0:11281:0:(nrs_fifo.c:241:nrs_fifo_req_stop()) NRS stop fifo request from 12345-0@lo, seq: 805 00000100:00100000:3.0:1776178138.978673:0:6256:0:(service.c:2372:ptlrpc_server_handle_request()) Handled RPC req@00000000339a9a57 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_001:lustre-MDT0000-lwp-MDT0000_UUID+5:3354:x1862456998030336:12345-0@lo:400:kworker/u8:2.0 Request processed in 40us (125us total) trans 0 rc 0/0 00000100:00100000:3.0:1776178138.978681:0:6256:0:(nrs_fifo.c:241:nrs_fifo_req_stop()) NRS stop fifo request from 12345-0@lo, seq: 806 00000100:00100000:0.0:1776178138.978683:0:8231:0:(service.c:2152:ptlrpc_server_handle_req_in()) got req x1862456998030400 00000100:00100000:0.0:1776178138.978715:0:8231:0:(nrs_fifo.c:179:nrs_fifo_req_get()) NRS start fifo request from 12345-0@lo, seq: 427 00000100:00100000:0.0:1776178138.978722:0:8231:0:(service.c:2324:ptlrpc_server_handle_request()) Handling RPC req@000000006466ee18 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_create00:lustre-MDT0000-mdtlov_UUID+5:3354:x1862456998030400:12345-0@lo:13:osp-pre-0-0.0 00000100:00100000:3.0:1776178138.978722:0:3354:0:(client.c:2215:ptlrpc_check_set()) Completed RPC req@0000000052a7c870 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_01:lustre-MDT0000-lwp-MDT0001_UUID:3354:1862456998030080:0@lo:400:kworker/u8:2.0 00000100:00100000:3.0:1776178138.978737:0:3354:0:(client.c:2215:ptlrpc_check_set()) Completed RPC req@000000001f170bf7 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_01:lustre-MDT0000-lwp-MDT0000_UUID:3354:1862456998030336:0@lo:400:kworker/u8:2.0 00002000:00000024:0.0:1776178138.978746:0:8231:0:(ofd_obd.c:740:ofd_statfs()) blocks cached 315392 granted 484883776 pending 0 free 3923689472 avail 3692167168 00000100:00100000:0.0:1776178138.978781:0:8231:0:(service.c:2372:ptlrpc_server_handle_request()) Handled RPC req@000000006466ee18 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_create00:lustre-MDT0000-mdtlov_UUID+5:3354:x1862456998030400:12345-0@lo:13:osp-pre-0-0.0 Request processed in 60us (469us total) trans 0 rc 0/0 00000100:00100000:0.0:1776178138.978787:0:8231:0:(nrs_fifo.c:241:nrs_fifo_req_stop()) NRS stop fifo request from 12345-0@lo, seq: 427 00000100:00100000:3.0:1776178138.978794:0:3354:0:(client.c:2215:ptlrpc_check_set()) Completed RPC req@000000006d540857 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_01:lustre-MDT0000-mdtlov_UUID:3354:1862456998030400:0@lo:13:osp-pre-0-0.0 00000100:00100000:2.0:1776178138.979101:0:3356:0:(client.c:1733:ptlrpc_send_new_req()) Sending RPC req@00000000a62daf9a pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_03:lustre-MDT0001-lwp-OST0000_UUID:3356:1862456998029760:0@lo:400:kworker/u8:2.0 00000100:00100000:2.0:1776178138.979127:0:3356:0:(events.c:361:request_in_callback()) peer: 12345-0@lo (source: 12345-0@lo) 00000100:00100000:2.0:1776178138.979139:0:3356:0:(client.c:1733:ptlrpc_send_new_req()) Sending RPC req@0000000037059d8f pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_03:lustre-MDT0001-mdtlov_UUID:3356:1862456998030016:0@lo:400:kworker/u8:2.0 00000100:00100000:2.0:1776178138.979158:0:3356:0:(events.c:361:request_in_callback()) peer: 12345-0@lo (source: 12345-0@lo) 00000100:00100000:2.0:1776178138.979184:0:3356:0:(client.c:1733:ptlrpc_send_new_req()) Sending RPC req@0000000090a7aa50 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_03:lustre-MDT0000-mdtlov_UUID:3356:1862456998030272:0@lo:400:kworker/u8:2.0 00000100:00100000:2.0:1776178138.979202:0:3356:0:(events.c:361:request_in_callback()) peer: 12345-0@lo (source: 12345-0@lo) 00000100:00100000:2.0:1776178138.979215:0:3356:0:(client.c:1733:ptlrpc_send_new_req()) Sending RPC req@000000005112bc3b pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_03:lustre-MDT0000-mdtlov_UUID:3356:1862456998030528:0@lo:13:osp-pre-1-0.0 00000100:00100000:2.0:1776178138.979242:0:3356:0:(events.c:361:request_in_callback()) peer: 12345-0@lo (source: 12345-0@lo) 00000100:00100000:2.0:1776178138.979265:0:3356:0:(ptlrpcd.c:415:ptlrpcd_check()) transfer 1 async RPCs [2->3] 00000100:00100000:2.0:1776178138.979272:0:3356:0:(client.c:1733:ptlrpc_send_new_req()) Sending RPC req@00000000d2922afa pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_03:lustre-MDT0000-mdtlov_UUID:3356:1862456998030464:0@lo:41:osp-pre-1-0.0 00000100:00100000:1.0:1776178138.979275:0:3355:0:(events.c:361:request_in_callback()) peer: 12345-0@lo (source: 12345-0@lo) 00000100:00100000:1.0:1776178138.979302:0:3355:0:(client.c:1733:ptlrpc_send_new_req()) Sending RPC req@00000000bd795500 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_02:lustre-MDT0001-mdtlov_UUID:3355:1862456998029952:0@lo:400:kworker/u8:2.0 00000100:00100000:2.0:1776178138.979304:0:3356:0:(events.c:361:request_in_callback()) peer: 12345-0@lo (source: 12345-0@lo) 00000100:00100000:1.0:1776178138.979334:0:3355:0:(events.c:361:request_in_callback()) peer: 12345-0@lo (source: 12345-0@lo) 00000100:00100000:2.0:1776178138.979346:0:14440:0:(service.c:2152:ptlrpc_server_handle_req_in()) got req x1862456998030016 00000100:00100000:1.0:1776178138.979354:0:3355:0:(client.c:1733:ptlrpc_send_new_req()) Sending RPC req@000000000e0e1967 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_02:lustre-MDT0000-mdtlov_UUID:3355:1862456998030208:0@lo:400:kworker/u8:2.0 00000100:00100000:2.0:1776178138.979374:0:14440:0:(service.c:2152:ptlrpc_server_handle_req_in()) got req x1862456998030272 00000100:00100000:1.0:1776178138.979390:0:3355:0:(events.c:361:request_in_callback()) peer: 12345-0@lo (source: 12345-0@lo) 00000100:00100000:0.0:1776178138.979481:0:6257:0:(service.c:2152:ptlrpc_server_handle_req_in()) got req x1862456998029760 00000100:00100000:0.0:1776178138.979502:0:6257:0:(service.c:2152:ptlrpc_server_handle_req_in()) got req x1862456998029696 00000100:00100000:0.0:1776178138.979515:0:6257:0:(nrs_fifo.c:179:nrs_fifo_req_get()) NRS start fifo request from 12345-0@lo, seq: 807 00000100:00100000:0.0:1776178138.979518:0:6257:0:(service.c:2324:ptlrpc_server_handle_request()) Handling RPC req@00000000ba5faff8 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_002:lustre-MDT0001-lwp-OST0000_UUID+5:3356:x1862456998029760:12345-0@lo:400:kworker/u8:2.0 00000100:00100000:0.0:1776178138.979555:0:6257:0:(service.c:2372:ptlrpc_server_handle_request()) Handled RPC req@00000000ba5faff8 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_002:lustre-MDT0001-lwp-OST0000_UUID+5:3356:x1862456998029760:12345-0@lo:400:kworker/u8:2.0 Request processed in 35us (428us total) trans 0 rc 0/0 00000100:00100000:0.0:1776178138.979560:0:6257:0:(nrs_fifo.c:241:nrs_fifo_req_stop()) NRS stop fifo request from 12345-0@lo, seq: 807 00000100:00100000:0.0:1776178138.979571:0:6257:0:(nrs_fifo.c:179:nrs_fifo_req_get()) NRS start fifo request from 12345-0@lo, seq: 808 00000100:00100000:0.0:1776178138.979574:0:6257:0:(service.c:2324:ptlrpc_server_handle_request()) Handling RPC req@000000007d5cfd93 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_002:lustre-MDT0000-lwp-OST0000_UUID+5:3355:x1862456998029696:12345-0@lo:400:kworker/u8:2.0 00000100:00100000:0.0:1776178138.979614:0:6257:0:(service.c:2372:ptlrpc_server_handle_request()) Handled RPC req@000000007d5cfd93 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_002:lustre-MDT0000-lwp-OST0000_UUID+5:3355:x1862456998029696:12345-0@lo:400:kworker/u8:2.0 Request processed in 38us (345us total) trans 0 rc 0/0 00000100:00100000:0.0:1776178138.979619:0:6257:0:(nrs_fifo.c:241:nrs_fifo_req_stop()) NRS stop fifo request from 12345-0@lo, seq: 808 00000100:00100000:2.0:1776178138.982833:0:14440:0:(service.c:2152:ptlrpc_server_handle_req_in()) got req x1862456998029952 00000100:00100000:2.0:1776178138.982902:0:14440:0:(service.c:2152:ptlrpc_server_handle_req_in()) got req x1862456998030208 00000100:00100000:2.0:1776178138.982925:0:14440:0:(nrs_fifo.c:179:nrs_fifo_req_get()) NRS start fifo request from 12345-0@lo, seq: 442 00000100:00100000:2.0:1776178138.982931:0:14440:0:(service.c:2324:ptlrpc_server_handle_request()) Handling RPC req@000000005d95861f pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_004:lustre-MDT0001-mdtlov_UUID+5:3356:x1862456998030016:12345-0@lo:400:kworker/u8:2.0 00000100:00100000:1.0:1776178138.986690:0:20759:0:(nrs_fifo.c:179:nrs_fifo_req_get()) NRS start fifo request from 12345-0@lo, seq: 443 00000100:00100000:1.0:1776178138.986705:0:20759:0:(service.c:2324:ptlrpc_server_handle_request()) Handling RPC req@000000005d3a2b34 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_010:lustre-MDT0000-mdtlov_UUID+5:3356:x1862456998030272:12345-0@lo:400:kworker/u8:2.0 00000100:00100000:1.0:1776178138.986814:0:20759:0:(service.c:2372:ptlrpc_server_handle_request()) Handled RPC req@000000005d3a2b34 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_010:lustre-MDT0000-mdtlov_UUID+5:3356:x1862456998030272:12345-0@lo:400:kworker/u8:2.0 Request processed in 111us (7611us total) trans 0 rc 0/0 00000100:00100000:1.0:1776178138.986823:0:20759:0:(nrs_fifo.c:241:nrs_fifo_req_stop()) NRS stop fifo request from 12345-0@lo, seq: 443 00000100:00100000:1.0:1776178138.986848:0:20759:0:(nrs_fifo.c:179:nrs_fifo_req_get()) NRS start fifo request from 12345-0@lo, seq: 444 00000100:00100000:1.0:1776178138.986853:0:20759:0:(service.c:2324:ptlrpc_server_handle_request()) Handling RPC req@0000000098e424d0 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_010:lustre-MDT0001-mdtlov_UUID+5:3355:x1862456998029952:12345-0@lo:400:kworker/u8:2.0 00000100:00100000:1.0:1776178138.986889:0:20759:0:(service.c:2372:ptlrpc_server_handle_request()) Handled RPC req@0000000098e424d0 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_010:lustre-MDT0001-mdtlov_UUID+5:3355:x1862456998029952:12345-0@lo:400:kworker/u8:2.0 Request processed in 36us (7556us total) trans 0 rc 0/0 00000100:00100000:1.0:1776178138.986895:0:20759:0:(nrs_fifo.c:241:nrs_fifo_req_stop()) NRS stop fifo request from 12345-0@lo, seq: 444 00000100:00100000:1.0:1776178138.986907:0:20759:0:(nrs_fifo.c:179:nrs_fifo_req_get()) NRS start fifo request from 12345-0@lo, seq: 445 00000100:00100000:1.0:1776178138.986911:0:20759:0:(service.c:2324:ptlrpc_server_handle_request()) Handling RPC req@000000001f170bf7 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_010:lustre-MDT0000-mdtlov_UUID+5:3355:x1862456998030208:12345-0@lo:400:kworker/u8:2.0 00000100:00100000:1.0:1776178138.986938:0:20759:0:(service.c:2372:ptlrpc_server_handle_request()) Handled RPC req@000000001f170bf7 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_010:lustre-MDT0000-mdtlov_UUID+5:3355:x1862456998030208:12345-0@lo:400:kworker/u8:2.0 Request processed in 26us (7549us total) trans 0 rc 0/0 00000100:00100000:1.0:1776178138.989579:0:20759:0:(nrs_fifo.c:241:nrs_fifo_req_stop()) NRS stop fifo request from 12345-0@lo, seq: 445 00000100:00100000:1.0:1776178138.989682:0:3355:0:(client.c:2215:ptlrpc_check_set()) Completed RPC req@00000000fed3e037 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_02:lustre-MDT0000-lwp-OST0000_UUID:3355:1862456998029696:0@lo:400:kworker/u8:2.0 00000100:00100000:1.0:1776178138.989696:0:3355:0:(client.c:2215:ptlrpc_check_set()) Completed RPC req@00000000bd795500 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_02:lustre-MDT0001-mdtlov_UUID:3355:1862456998029952:0@lo:400:kworker/u8:2.0 00000100:00100000:1.0:1776178138.989707:0:3355:0:(client.c:2215:ptlrpc_check_set()) Completed RPC req@000000000e0e1967 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_02:lustre-MDT0000-mdtlov_UUID:3355:1862456998030208:0@lo:400:kworker/u8:2.0 00000100:00100000:1.0:1776178138.989805:0:15213:0:(service.c:2152:ptlrpc_server_handle_req_in()) got req x1862456998030464 00000100:00100000:1.0:1776178138.989872:0:15213:0:(nrs_fifo.c:179:nrs_fifo_req_get()) NRS start fifo request from 12345-0@lo, seq: 640 00000100:00100000:1.0:1776178138.989877:0:15213:0:(service.c:2324:ptlrpc_server_handle_request()) Handling RPC req@0000000054e3866c pname:cluuid+ref:pid:xid:nid:opc:job mdt_out00_003:lustre-MDT0000-mdtlov_UUID+5:3356:x1862456998030464:12345-0@lo:41:osp-pre-1-0.0 00000004:00000024:1.0:1776178138.989916:0:15213:0:(mdt_handler.c:548:mdt_statfs()) blocks cached 0 granted 2097152 pending 0 free 1446039552 avail 1316577280 00000100:00100000:1.0:1776178138.989988:0:15213:0:(service.c:2372:ptlrpc_server_handle_request()) Handled RPC req@0000000054e3866c pname:cluuid+ref:pid:xid:nid:opc:job mdt_out00_003:lustre-MDT0000-mdtlov_UUID+5:3356:x1862456998030464:12345-0@lo:41:osp-pre-1-0.0 Request processed in 109us (10685us total) trans 0 rc 0/0 00000100:00100000:1.0:1776178138.989995:0:15213:0:(nrs_fifo.c:241:nrs_fifo_req_stop()) NRS stop fifo request from 12345-0@lo, seq: 640 00000100:00100000:1.0:1776178138.990027:0:3356:0:(client.c:2215:ptlrpc_check_set()) Completed RPC req@00000000d2922afa pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_03:lustre-MDT0000-mdtlov_UUID:3356:1862456998030464:0@lo:41:osp-pre-1-0.0 00000100:00100000:1.0:1776178138.990039:0:3356:0:(client.c:2215:ptlrpc_check_set()) Completed RPC req@00000000a62daf9a pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_03:lustre-MDT0001-lwp-OST0000_UUID:3356:1862456998029760:0@lo:400:kworker/u8:2.0 00000100:00100000:1.0:1776178138.990050:0:3356:0:(client.c:2215:ptlrpc_check_set()) Completed RPC req@0000000090a7aa50 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_03:lustre-MDT0000-mdtlov_UUID:3356:1862456998030272:0@lo:400:kworker/u8:2.0 00000100:00100000:1.0:1776178138.990100:0:8231:0:(service.c:2152:ptlrpc_server_handle_req_in()) got req x1862456998030528 00000100:00100000:2.0:1776178138.991169:0:14440:0:(service.c:2372:ptlrpc_server_handle_request()) Handled RPC req@000000005d95861f pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_004:lustre-MDT0001-mdtlov_UUID+5:3356:x1862456998030016:12345-0@lo:400:kworker/u8:2.0 Request processed in 8235us (12007us total) trans 0 rc 0/0 00000100:00100000:2.0:1776178138.991179:0:14440:0:(nrs_fifo.c:241:nrs_fifo_req_stop()) NRS stop fifo request from 12345-0@lo, seq: 442 00000100:00100000:2.0:1776178138.991218:0:3356:0:(client.c:2215:ptlrpc_check_set()) Completed RPC req@0000000037059d8f pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_03:lustre-MDT0001-mdtlov_UUID:3356:1862456998030016:0@lo:400:kworker/u8:2.0 00000100:00100000:1.0:1776178138.993346:0:8231:0:(nrs_fifo.c:179:nrs_fifo_req_get()) NRS start fifo request from 12345-0@lo, seq: 428 00000100:00100000:1.0:1776178138.993357:0:8231:0:(service.c:2324:ptlrpc_server_handle_request()) Handling RPC req@000000004cbcc47c pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_create00:lustre-MDT0000-mdtlov_UUID+5:3356:x1862456998030528:12345-0@lo:13:osp-pre-1-0.0 00002000:00000024:1.0:1776178138.993581:0:8231:0:(ofd_obd.c:740:ofd_statfs()) blocks cached 229376 granted 491746368 pending 0 free 3923689472 avail 3692167168 00000100:00100000:1.0:1776178138.993655:0:8231:0:(service.c:2372:ptlrpc_server_handle_request()) Handled RPC req@000000004cbcc47c pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_create00:lustre-MDT0000-mdtlov_UUID+5:3356:x1862456998030528:12345-0@lo:13:osp-pre-1-0.0 Request processed in 299us (14421us total) trans 0 rc 0/0 00000100:00100000:1.0:1776178138.993662:0:8231:0:(nrs_fifo.c:241:nrs_fifo_req_stop()) NRS stop fifo request from 12345-0@lo, seq: 428 00000100:00100000:1.0:1776178138.993721:0:3356:0:(client.c:2215:ptlrpc_check_set()) Completed RPC req@000000005112bc3b pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_03:lustre-MDT0000-mdtlov_UUID:3356:1862456998030528:0@lo:13:osp-pre-1-0.0 00010000:00010000:1.0:1776178139.555554:0:6245:0:(ldlm_lockd.c:2792:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:3.0:1776178139.938040:0:3346:0:(events.c:361:request_in_callback()) peer: 12345-192.168.204.3@tcp (source: 12345-192.168.204.3@tcp) 00000100:00100000:2.0:1776178139.938208:0:6256:0:(service.c:2152:ptlrpc_server_handle_req_in()) got req x1862456979084160 00000100:00100000:2.0:1776178139.938286:0:6256:0:(nrs_fifo.c:179:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.204.3@tcp, seq: 809 00000100:00100000:2.0:1776178139.938294:0:6256:0:(service.c:2324:ptlrpc_server_handle_request()) Handling RPC req@0000000020afd0b3 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_001:b50963bd-8eba-4432-9a54-8888d00e3fa1+21:2271:x1862456979084160:12345-192.168.204.3@tcp:400:kworker/u8:4.0 00000100:00100000:2.0:1776178139.938395:0:6256:0:(service.c:2372:ptlrpc_server_handle_request()) Handled RPC req@0000000020afd0b3 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_001:b50963bd-8eba-4432-9a54-8888d00e3fa1+21:2271:x1862456979084160:12345-192.168.204.3@tcp:400:kworker/u8:4.0 Request processed in 101us (363us total) trans 0 rc 0/0 00000100:00100000:2.0:1776178139.938402:0:6256:0:(nrs_fifo.c:241:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.204.3@tcp, seq: 809 00000100:00100000:0.0:1776178139.944256:0:3347:0:(events.c:361:request_in_callback()) peer: 12345-192.168.204.3@tcp (source: 12345-192.168.204.3@tcp) 00000100:00100000:0.0:1776178139.944306:0:3347:0:(events.c:361:request_in_callback()) peer: 12345-192.168.204.3@tcp (source: 12345-192.168.204.3@tcp) 00000100:00100000:0.0:1776178139.944338:0:3347:0:(events.c:361:request_in_callback()) peer: 12345-192.168.204.3@tcp (source: 12345-192.168.204.3@tcp) 00000100:00100000:1.0:1776178139.944446:0:6255:0:(service.c:2152:ptlrpc_server_handle_req_in()) got req x1862456979084224 00000100:00100000:1.0:1776178139.944514:0:6255:0:(nrs_fifo.c:179:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.204.3@tcp, seq: 810 00000100:00100000:1.0:1776178139.944523:0:6255:0:(service.c:2324:ptlrpc_server_handle_request()) Handling RPC req@00000000a66f2317 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_000:b50963bd-8eba-4432-9a54-8888d00e3fa1+6:2270:x1862456979084224:12345-192.168.204.3@tcp:400:kworker/u8:4.0 00000100:00100000:1.0:1776178139.944618:0:6255:0:(service.c:2372:ptlrpc_server_handle_request()) Handled RPC req@00000000a66f2317 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_000:b50963bd-8eba-4432-9a54-8888d00e3fa1+6:2270:x1862456979084224:12345-192.168.204.3@tcp:400:kworker/u8:4.0 Request processed in 94us (310us total) trans 0 rc 0/0 00000100:00100000:1.0:1776178139.944627:0:6255:0:(nrs_fifo.c:241:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.204.3@tcp, seq: 810 00000100:00100000:3.0:1776178139.944633:0:20764:0:(service.c:2152:ptlrpc_server_handle_req_in()) got req x1862456979084352 00000100:00100000:3.0:1776178139.944666:0:20764:0:(nrs_fifo.c:179:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.204.3@tcp, seq: 446 00000100:00100000:3.0:1776178139.944671:0:20764:0:(service.c:2324:ptlrpc_server_handle_request()) Handling RPC req@00000000a33eecf4 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_015:b50963bd-8eba-4432-9a54-8888d00e3fa1+8:2270:x1862456979084352:12345-192.168.204.3@tcp:400:kworker/u8:4.0 00000100:00100000:3.0:1776178139.944702:0:20764:0:(service.c:2372:ptlrpc_server_handle_request()) Handled RPC req@00000000a33eecf4 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_015:b50963bd-8eba-4432-9a54-8888d00e3fa1+8:2270:x1862456979084352:12345-192.168.204.3@tcp:400:kworker/u8:4.0 Request processed in 30us (364us total) trans 0 rc 0/0 00000100:00100000:3.0:1776178139.944709:0:20764:0:(nrs_fifo.c:241:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.204.3@tcp, seq: 446 00000100:00100000:2.0:1776178139.944895:0:20140:0:(service.c:2152:ptlrpc_server_handle_req_in()) got req x1862456979084096 00000100:00100000:2.0:1776178139.944943:0:20140:0:(nrs_fifo.c:179:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.204.3@tcp, seq: 130 00000100:00100000:2.0:1776178139.944947:0:20140:0:(service.c:2324:ptlrpc_server_handle_request()) Handling RPC req@00000000fd0a07ee pname:cluuid+ref:pid:xid:nid:opc:job ll_mgs_0002:5b5c66b1-1a2f-4651-8548-0c75b9a38a94+9:2270:x1862456979084096:12345-192.168.204.3@tcp:400:kworker/u8:4.0 00000100:00100000:2.0:1776178139.944991:0:20140:0:(service.c:2372:ptlrpc_server_handle_request()) Handled RPC req@00000000fd0a07ee pname:cluuid+ref:pid:xid:nid:opc:job ll_mgs_0002:5b5c66b1-1a2f-4651-8548-0c75b9a38a94+9:2270:x1862456979084096:12345-192.168.204.3@tcp:400:kworker/u8:4.0 Request processed in 43us (745us total) trans 0 rc 0/0 00000100:00100000:2.0:1776178139.945002:0:20140:0:(nrs_fifo.c:241:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.204.3@tcp, seq: 130 00000100:00100000:0.0:1776178139.953781:0:3347:0:(events.c:361:request_in_callback()) peer: 12345-192.168.204.3@tcp (source: 12345-192.168.204.3@tcp) 00000100:00100000:3.0:1776178139.953855:0:8227:0:(service.c:2152:ptlrpc_server_handle_req_in()) got req x1862456979084288 00000100:00100000:3.0:1776178139.953941:0:8227:0:(nrs_fifo.c:179:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.204.3@tcp, seq: 447 00000100:00100000:3.0:1776178139.953949:0:8227:0:(service.c:2324:ptlrpc_server_handle_request()) Handling RPC req@00000000d1e0f01c pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_000:b50963bd-8eba-4432-9a54-8888d00e3fa1+9:2271:x1862456979084288:12345-192.168.204.3@tcp:400:kworker/u8:4.0 00000100:00100000:3.0:1776178139.954092:0:8227:0:(service.c:2372:ptlrpc_server_handle_request()) Handled RPC req@00000000d1e0f01c pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_000:b50963bd-8eba-4432-9a54-8888d00e3fa1+9:2271:x1862456979084288:12345-192.168.204.3@tcp:400:kworker/u8:4.0 Request processed in 143us (321us total) trans 0 rc 0/0 00000100:00100000:3.0:1776178139.954099:0:8227:0:(nrs_fifo.c:241:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.204.3@tcp, seq: 447 00000100:00100000:0.0:1776178140.515540:0:3354:0:(ptlrpcd.c:415:ptlrpcd_check()) transfer 1 async RPCs [0->1] 00000100:00100000:0.0:1776178140.515568:0:3354:0:(client.c:1733:ptlrpc_send_new_req()) Sending RPC req@0000000038c09127 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_01:lustre-MDT0001-mdtlov_UUID:3354:1862456998030592:0@lo:13:osp-pre-1-1.0 00000100:00100000:0.0:1776178140.515647:0:3354:0:(events.c:361:request_in_callback()) peer: 12345-0@lo (source: 12345-0@lo) 00000100:00100000:3.0:1776178140.515902:0:8232:0:(service.c:2152:ptlrpc_server_handle_req_in()) got req x1862456998030592 00000100:00100000:3.0:1776178140.515970:0:8232:0:(nrs_fifo.c:179:nrs_fifo_req_get()) NRS start fifo request from 12345-0@lo, seq: 429 00000100:00100000:3.0:1776178140.515977:0:8232:0:(service.c:2324:ptlrpc_server_handle_request()) Handling RPC req@00000000aba13576 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_create00:lustre-MDT0001-mdtlov_UUID+5:3354:x1862456998030592:12345-0@lo:13:osp-pre-1-1.0 00002000:00000024:3.0:1776178140.516002:0:8232:0:(ofd_obd.c:740:ofd_statfs()) blocks cached 229376 granted 491746368 pending 0 free 3923689472 avail 3692167168 00000100:00100000:0.0:1776178140.516219:0:3354:0:(client.c:2215:ptlrpc_check_set()) Completed RPC req@0000000038c09127 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_01:lustre-MDT0001-mdtlov_UUID:3354:1862456998030592:0@lo:13:osp-pre-1-1.0 00000100:00100000:3.0:1776178140.516509:0:8232:0:(service.c:2372:ptlrpc_server_handle_request()) Handled RPC req@00000000aba13576 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_create00:lustre-MDT0001-mdtlov_UUID+5:3354:x1862456998030592:12345-0@lo:13:osp-pre-1-1.0 Request processed in 112us (443us total) trans 0 rc 0/0 00000100:00100000:3.0:1776178140.516519:0:8232:0:(nrs_fifo.c:241:nrs_fifo_req_stop()) NRS stop fifo request from 12345-0@lo, seq: 429 00010000:00010000:1.0:1776178140.578046:0:9225:0:(ldlm_lockd.c:2792:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:3.0:1776178141.025285:0:3353:0:(ptlrpcd.c:415:ptlrpcd_check()) transfer 1 async RPCs [1->0] 00000100:00100000:3.0:1776178141.025312:0:3353:0:(client.c:1733:ptlrpc_send_new_req()) Sending RPC req@000000008b858f58 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_00:lustre-MDT0001-mdtlov_UUID:3353:1862456998030656:0@lo:13:osp-pre-0-1.0 00000100:00100000:3.0:1776178141.025401:0:3353:0:(events.c:361:request_in_callback()) peer: 12345-0@lo (source: 12345-0@lo) 00000100:00100000:1.0:1776178141.025930:0:8231:0:(service.c:2152:ptlrpc_server_handle_req_in()) got req x1862456998030656 00000100:00100000:1.0:1776178141.025995:0:8231:0:(nrs_fifo.c:179:nrs_fifo_req_get()) NRS start fifo request from 12345-0@lo, seq: 430 00000100:00100000:1.0:1776178141.026002:0:8231:0:(service.c:2324:ptlrpc_server_handle_request()) Handling RPC req@0000000023562990 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_create00:lustre-MDT0001-mdtlov_UUID+5:3353:x1862456998030656:12345-0@lo:13:osp-pre-0-1.0 00002000:00000024:1.0:1776178141.026037:0:8231:0:(ofd_obd.c:740:ofd_statfs()) blocks cached 315392 granted 484883776 pending 0 free 3923689472 avail 3692167168 00000100:00100000:1.0:1776178141.026138:0:8231:0:(service.c:2372:ptlrpc_server_handle_request()) Handled RPC req@0000000023562990 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_create00:lustre-MDT0001-mdtlov_UUID+5:3353:x1862456998030656:12345-0@lo:13:osp-pre-0-1.0 Request processed in 135us (737us total) trans 0 rc 0/0 00000100:00100000:1.0:1776178141.026146:0:8231:0:(nrs_fifo.c:241:nrs_fifo_req_stop()) NRS stop fifo request from 12345-0@lo, seq: 430 00000100:00100000:0.0:1776178141.026678:0:3353:0:(client.c:2215:ptlrpc_check_set()) Completed RPC req@000000008b858f58 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_00:lustre-MDT0001-mdtlov_UUID:3353:1862456998030656:0@lo:13:osp-pre-0-1.0 00010000:00010000:2.0:1776178141.606248:0:9224:0:(ldlm_lockd.c:2792:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00010000:00010000:1.0:1776178142.624430:0:6244:0:(ldlm_lockd.c:2792:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00010000:00010000:1.0:1776178143.651751:0:6245:0:(ldlm_lockd.c:2792:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:2.0:1776178144.096503:0:3356:0:(client.c:1733:ptlrpc_send_new_req()) Sending RPC req@000000000e0e1967 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_03:lustre-MDT0000-lwp-OST0000_UUID:3356:1862456998030784:0@lo:400:kworker/u8:2.0 00000100:00100000:2.0:1776178144.096584:0:3356:0:(events.c:361:request_in_callback()) peer: 12345-0@lo (source: 12345-0@lo) 00000100:00100000:2.0:1776178144.096625:0:3356:0:(client.c:1733:ptlrpc_send_new_req()) Sending RPC req@00000000741c4ec9 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_03:lustre-MDT0001-mdtlov_UUID:3356:1862456998031040:0@lo:400:kworker/u8:2.0 00000100:00100000:2.0:1776178144.096652:0:3356:0:(events.c:361:request_in_callback()) peer: 12345-0@lo (source: 12345-0@lo) 00000100:00100000:1.0:1776178144.096658:0:6256:0:(service.c:2152:ptlrpc_server_handle_req_in()) got req x1862456998030784 00000100:00100000:2.0:1776178144.096668:0:3356:0:(client.c:1733:ptlrpc_send_new_req()) Sending RPC req@0000000090cc55f6 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_03:lustre-MDT0000-lwp-MDT0001_UUID:3356:1862456998031296:0@lo:400:kworker/u8:2.0 00000100:00100000:2.0:1776178144.096690:0:3356:0:(events.c:361:request_in_callback()) peer: 12345-0@lo (source: 12345-0@lo) 00000100:00100000:2.0:1776178144.096708:0:3356:0:(client.c:1733:ptlrpc_send_new_req()) Sending RPC req@0000000055392450 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_03:lustre-MDT0000-lwp-MDT0000_UUID:3356:1862456998031552:0@lo:400:kworker/u8:2.0 00000100:00100000:1.0:1776178144.096726:0:6256:0:(service.c:2152:ptlrpc_server_handle_req_in()) got req x1862456998031296 00000100:00100000:2.0:1776178144.096727:0:3356:0:(events.c:361:request_in_callback()) peer: 12345-0@lo (source: 12345-0@lo) 00000100:00100000:2.0:1776178144.096740:0:3356:0:(ptlrpcd.c:415:ptlrpcd_check()) transfer 4 async RPCs [2->3] 00000100:00100000:1.0:1776178144.096741:0:6256:0:(service.c:2152:ptlrpc_server_handle_req_in()) got req x1862456998031552 00000100:00100000:2.0:1776178144.096746:0:3356:0:(client.c:1733:ptlrpc_send_new_req()) Sending RPC req@0000000090a7aa50 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_03:ac61f839-0e8e-4cac-9c59-08bcfbbad330:3356:1862456998030720:0@lo:400:kworker/u8:2.0 00000100:00100000:1.0:1776178144.096754:0:6256:0:(nrs_fifo.c:179:nrs_fifo_req_get()) NRS start fifo request from 12345-0@lo, seq: 811 00000100:00100000:1.0:1776178144.096760:0:6256:0:(service.c:2324:ptlrpc_server_handle_request()) Handling RPC req@000000005112bc3b pname:cluuid+ref:pid:xid:nid:opc:job mdt00_001:lustre-MDT0000-lwp-OST0000_UUID+5:3356:x1862456998030784:12345-0@lo:400:kworker/u8:2.0 00000100:00100000:2.0:1776178144.096773:0:3356:0:(events.c:361:request_in_callback()) peer: 12345-0@lo (source: 12345-0@lo) 00000100:00100000:2.0:1776178144.096790:0:3356:0:(client.c:1733:ptlrpc_send_new_req()) Sending RPC req@00000000fed3e037 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_03:lustre-MDT0001-lwp-OST0001_UUID:3356:1862456998030976:0@lo:400:kworker/u8:2.0 00000100:00100000:2.0:1776178144.096806:0:3356:0:(events.c:361:request_in_callback()) peer: 12345-0@lo (source: 12345-0@lo) 00000100:00100000:1.0:1776178144.096813:0:6256:0:(service.c:2372:ptlrpc_server_handle_request()) Handled RPC req@000000005112bc3b pname:cluuid+ref:pid:xid:nid:opc:job mdt00_001:lustre-MDT0000-lwp-OST0000_UUID+5:3356:x1862456998030784:12345-0@lo:400:kworker/u8:2.0 Request processed in 52us (231us total) trans 0 rc 0/0 00000100:00100000:1.0:1776178144.096820:0:6256:0:(nrs_fifo.c:241:nrs_fifo_req_stop()) NRS stop fifo request from 12345-0@lo, seq: 811 00000100:00100000:2.0:1776178144.096827:0:3356:0:(client.c:1733:ptlrpc_send_new_req()) Sending RPC req@000000008b858f58 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_03:lustre-MDT0000-mdtlov_UUID:3356:1862456998031232:0@lo:13:osp-pre-1-0.0 00000100:00100000:1.0:1776178144.096836:0:6256:0:(service.c:2152:ptlrpc_server_handle_req_in()) got req x1862456998030976 00000100:00100000:2.0:1776178144.096859:0:3356:0:(events.c:361:request_in_callback()) peer: 12345-0@lo (source: 12345-0@lo) 00000100:00100000:1.0:1776178144.096867:0:6256:0:(nrs_fifo.c:179:nrs_fifo_req_get()) NRS start fifo request from 12345-0@lo, seq: 812 00000100:00100000:1.0:1776178144.096870:0:6256:0:(service.c:2324:ptlrpc_server_handle_request()) Handling RPC req@00000000e36da5a6 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_001:lustre-MDT0000-lwp-MDT0001_UUID+5:3356:x1862456998031296:12345-0@lo:400:kworker/u8:2.0 00000100:00100000:2.0:1776178144.096874:0:3356:0:(client.c:1733:ptlrpc_send_new_req()) Sending RPC req@0000000043589f56 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_03:lustre-MDT0000-mdtlov_UUID:3356:1862456998031488:0@lo:400:kworker/u8:2.0 00000100:00100000:2.0:1776178144.096895:0:3356:0:(events.c:361:request_in_callback()) peer: 12345-0@lo (source: 12345-0@lo) 00000100:00100000:1.0:1776178144.096896:0:6256:0:(service.c:2372:ptlrpc_server_handle_request()) Handled RPC req@00000000e36da5a6 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_001:lustre-MDT0000-lwp-MDT0001_UUID+5:3356:x1862456998031296:12345-0@lo:400:kworker/u8:2.0 Request processed in 26us (206us total) trans 0 rc 0/0 00000100:00100000:1.0:1776178144.096901:0:6256:0:(nrs_fifo.c:241:nrs_fifo_req_stop()) NRS stop fifo request from 12345-0@lo, seq: 812 00000100:00100000:1.0:1776178144.096912:0:6256:0:(nrs_fifo.c:179:nrs_fifo_req_get()) NRS start fifo request from 12345-0@lo, seq: 813 00000100:00100000:1.0:1776178144.096915:0:6256:0:(service.c:2324:ptlrpc_server_handle_request()) Handling RPC req@0000000046115d91 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_001:lustre-MDT0000-lwp-MDT0000_UUID+5:3356:x1862456998031552:12345-0@lo:400:kworker/u8:2.0 00000100:00100000:2.0:1776178144.096921:0:3356:0:(client.c:2215:ptlrpc_check_set()) Completed RPC req@000000000e0e1967 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_03:lustre-MDT0000-lwp-OST0000_UUID:3356:1862456998030784:0@lo:400:kworker/u8:2.0 00000100:00100000:2.0:1776178144.096931:0:3356:0:(client.c:2215:ptlrpc_check_set()) Completed RPC req@0000000090cc55f6 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_03:lustre-MDT0000-lwp-MDT0001_UUID:3356:1862456998031296:0@lo:400:kworker/u8:2.0 00000100:00100000:1.0:1776178144.096936:0:6256:0:(service.c:2372:ptlrpc_server_handle_request()) Handled RPC req@0000000046115d91 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_001:lustre-MDT0000-lwp-MDT0000_UUID+5:3356:x1862456998031552:12345-0@lo:400:kworker/u8:2.0 Request processed in 21us (209us total) trans 0 rc 0/0 00000100:00100000:1.0:1776178144.096940:0:6256:0:(nrs_fifo.c:241:nrs_fifo_req_stop()) NRS stop fifo request from 12345-0@lo, seq: 813 00000100:00100000:1.0:1776178144.096949:0:6256:0:(nrs_fifo.c:179:nrs_fifo_req_get()) NRS start fifo request from 12345-0@lo, seq: 814 00000100:00100000:2.0:1776178144.096952:0:3356:0:(client.c:2215:ptlrpc_check_set()) Completed RPC req@0000000055392450 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_03:lustre-MDT0000-lwp-MDT0000_UUID:3356:1862456998031552:0@lo:400:kworker/u8:2.0 00000100:00100000:1.0:1776178144.096953:0:6256:0:(service.c:2324:ptlrpc_server_handle_request()) Handling RPC req@00000000305b2aed pname:cluuid+ref:pid:xid:nid:opc:job mdt00_001:lustre-MDT0001-lwp-OST0001_UUID+5:3356:x1862456998030976:12345-0@lo:400:kworker/u8:2.0 00000100:00100000:2.0:1776178144.096975:0:20137:0:(service.c:2152:ptlrpc_server_handle_req_in()) got req x1862456998030720 00000100:00100000:1.0:1776178144.096979:0:6256:0:(service.c:2372:ptlrpc_server_handle_request()) Handled RPC req@00000000305b2aed pname:cluuid+ref:pid:xid:nid:opc:job mdt00_001:lustre-MDT0001-lwp-OST0001_UUID+5:3356:x1862456998030976:12345-0@lo:400:kworker/u8:2.0 Request processed in 26us (173us total) trans 0 rc 0/0 00000100:00100000:1.0:1776178144.096985:0:6256:0:(nrs_fifo.c:241:nrs_fifo_req_stop()) NRS stop fifo request from 12345-0@lo, seq: 814 00000100:00100000:2.0:1776178144.097000:0:20137:0:(nrs_fifo.c:179:nrs_fifo_req_get()) NRS start fifo request from 12345-0@lo, seq: 131 00000100:00100000:1.0:1776178144.097001:0:6260:0:(service.c:2152:ptlrpc_server_handle_req_in()) got req x1862456998031040 00000100:00100000:2.0:1776178144.097004:0:20137:0:(service.c:2324:ptlrpc_server_handle_request()) Handling RPC req@00000000f240c829 pname:cluuid+ref:pid:xid:nid:opc:job ll_mgs_0000:ac61f839-0e8e-4cac-9c59-08bcfbbad330+23:3356:x1862456998030720:12345-0@lo:400:kworker/u8:2.0 00000100:00100000:1.0:1776178144.097021:0:6260:0:(nrs_fifo.c:179:nrs_fifo_req_get()) NRS start fifo request from 12345-0@lo, seq: 641 00000100:00100000:1.0:1776178144.097025:0:6260:0:(service.c:2324:ptlrpc_server_handle_request()) Handling RPC req@00000000490667a2 pname:cluuid+ref:pid:xid:nid:opc:job mdt_out00_000:lustre-MDT0001-mdtlov_UUID+6:3356:x1862456998031040:12345-0@lo:400:kworker/u8:2.0 00000100:00100000:2.0:1776178144.097029:0:20137:0:(service.c:2372:ptlrpc_server_handle_request()) Handled RPC req@00000000f240c829 pname:cluuid+ref:pid:xid:nid:opc:job ll_mgs_0000:ac61f839-0e8e-4cac-9c59-08bcfbbad330+23:3356:x1862456998030720:12345-0@lo:400:kworker/u8:2.0 Request processed in 25us (256us total) trans 0 rc 0/0 00000100:00100000:2.0:1776178144.097034:0:20137:0:(nrs_fifo.c:241:nrs_fifo_req_stop()) NRS stop fifo request from 12345-0@lo, seq: 131 00000100:00100000:1.0:1776178144.097046:0:6260:0:(service.c:2372:ptlrpc_server_handle_request()) Handled RPC req@00000000490667a2 pname:cluuid+ref:pid:xid:nid:opc:job mdt_out00_000:lustre-MDT0001-mdtlov_UUID+6:3356:x1862456998031040:12345-0@lo:400:kworker/u8:2.0 Request processed in 21us (395us total) trans 0 rc 0/0 00000100:00100000:1.0:1776178144.097052:0:6260:0:(nrs_fifo.c:241:nrs_fifo_req_stop()) NRS stop fifo request from 12345-0@lo, seq: 641 00000100:00100000:2.0:1776178144.097081:0:8232:0:(service.c:2152:ptlrpc_server_handle_req_in()) got req x1862456998031232 00000100:00100000:1.0:1776178144.097100:0:3356:0:(client.c:2215:ptlrpc_check_set()) Completed RPC req@0000000090a7aa50 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_03:ac61f839-0e8e-4cac-9c59-08bcfbbad330:3356:1862456998030720:0@lo:400:kworker/u8:2.0 00000100:00100000:2.0:1776178144.097104:0:8232:0:(nrs_fifo.c:179:nrs_fifo_req_get()) NRS start fifo request from 12345-0@lo, seq: 431 00000100:00100000:2.0:1776178144.097107:0:8232:0:(service.c:2324:ptlrpc_server_handle_request()) Handling RPC req@0000000069ebab60 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_create00:lustre-MDT0000-mdtlov_UUID+5:3356:x1862456998031232:12345-0@lo:13:osp-pre-1-0.0 00000100:00100000:1.0:1776178144.097109:0:3356:0:(client.c:2215:ptlrpc_check_set()) Completed RPC req@00000000fed3e037 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_03:lustre-MDT0001-lwp-OST0001_UUID:3356:1862456998030976:0@lo:400:kworker/u8:2.0 00000100:00100000:1.0:1776178144.097117:0:3356:0:(client.c:2215:ptlrpc_check_set()) Completed RPC req@00000000741c4ec9 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_03:lustre-MDT0001-mdtlov_UUID:3356:1862456998031040:0@lo:400:kworker/u8:2.0 00002000:00000024:2.0:1776178144.097127:0:8232:0:(ofd_obd.c:740:ofd_statfs()) blocks cached 229376 granted 491746368 pending 0 free 3923689472 avail 3692167168 00000100:00100000:2.0:1776178144.097158:0:8232:0:(service.c:2372:ptlrpc_server_handle_request()) Handled RPC req@0000000069ebab60 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_create00:lustre-MDT0000-mdtlov_UUID+5:3356:x1862456998031232:12345-0@lo:13:osp-pre-1-0.0 Request processed in 50us (299us total) trans 0 rc 0/0 00000100:00100000:2.0:1776178144.097164:0:8232:0:(nrs_fifo.c:241:nrs_fifo_req_stop()) NRS stop fifo request from 12345-0@lo, seq: 431 00000100:00100000:1.0:1776178144.097166:0:3356:0:(client.c:2215:ptlrpc_check_set()) Completed RPC req@000000008b858f58 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_03:lustre-MDT0000-mdtlov_UUID:3356:1862456998031232:0@lo:13:osp-pre-1-0.0 00000100:00100000:0.0:1776178144.097400:0:3354:0:(client.c:1733:ptlrpc_send_new_req()) Sending RPC req@00000000a62daf9a pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_01:lustre-MDT0000-lwp-OST0001_UUID:3354:1862456998030912:0@lo:400:kworker/u8:2.0 00000100:00100000:0.0:1776178144.097433:0:3354:0:(events.c:361:request_in_callback()) peer: 12345-0@lo (source: 12345-0@lo) 00000100:00100000:0.0:1776178144.097459:0:3354:0:(client.c:1733:ptlrpc_send_new_req()) Sending RPC req@00000000736589c7 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_01:lustre-MDT0001-mdtlov_UUID:3354:1862456998031168:0@lo:400:kworker/u8:2.0 00000100:00100000:0.0:1776178144.097482:0:3354:0:(events.c:361:request_in_callback()) peer: 12345-0@lo (source: 12345-0@lo) 00000100:00100000:0.0:1776178144.097512:0:3354:0:(client.c:1733:ptlrpc_send_new_req()) Sending RPC req@00000000f114dadf pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_01:lustre-MDT0000-mdtlov_UUID:3354:1862456998031424:0@lo:400:kworker/u8:2.0 00000100:00100000:0.0:1776178144.097536:0:3354:0:(events.c:361:request_in_callback()) peer: 12345-0@lo (source: 12345-0@lo) 00000100:00100000:0.0:1776178144.097554:0:3354:0:(ptlrpcd.c:415:ptlrpcd_check()) transfer 4 async RPCs [0->1] 00000100:00100000:0.0:1776178144.097560:0:3354:0:(client.c:1733:ptlrpc_send_new_req()) Sending RPC req@000000006b0ae3d8 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_01:lustre-MDT0001-lwp-OST0000_UUID:3354:1862456998030848:0@lo:400:kworker/u8:2.0 00000100:00100000:0.0:1776178144.097583:0:3354:0:(events.c:361:request_in_callback()) peer: 12345-0@lo (source: 12345-0@lo) 00000100:00100000:0.0:1776178144.097596:0:3354:0:(client.c:1733:ptlrpc_send_new_req()) Sending RPC req@000000004a3d9287 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_01:lustre-MDT0001-mdtlov_UUID:3354:1862456998031104:0@lo:400:kworker/u8:2.0 00000100:00100000:0.0:1776178144.097616:0:3354:0:(events.c:361:request_in_callback()) peer: 12345-0@lo (source: 12345-0@lo) 00000100:00100000:0.0:1776178144.097628:0:3354:0:(client.c:1733:ptlrpc_send_new_req()) Sending RPC req@00000000456ce710 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_01:lustre-MDT0000-mdtlov_UUID:3354:1862456998031360:0@lo:400:kworker/u8:2.0 00000100:00100000:0.0:1776178144.097650:0:3354:0:(events.c:361:request_in_callback()) peer: 12345-0@lo (source: 12345-0@lo) 00000100:00100000:1.0:1776178144.097652:0:11281:0:(service.c:2152:ptlrpc_server_handle_req_in()) got req x1862456998030912 00000100:00100000:0.0:1776178144.097663:0:3354:0:(client.c:1733:ptlrpc_send_new_req()) Sending RPC req@000000000154171f pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_01:lustre-MDT0000-mdtlov_UUID:3354:1862456998031616:0@lo:41:osp-pre-1-0.0 00000100:00100000:0.0:1776178144.097684:0:3354:0:(events.c:361:request_in_callback()) peer: 12345-0@lo (source: 12345-0@lo) 00000100:00100000:1.0:1776178144.097687:0:11281:0:(service.c:2152:ptlrpc_server_handle_req_in()) got req x1862456998030848 00000100:00100000:1.0:1776178144.097703:0:11281:0:(nrs_fifo.c:179:nrs_fifo_req_get()) NRS start fifo request from 12345-0@lo, seq: 815 00000100:00100000:0.0:1776178144.097708:0:15213:0:(service.c:2152:ptlrpc_server_handle_req_in()) got req x1862456998031360 00000100:00100000:0.0:1776178144.097727:0:15213:0:(service.c:2152:ptlrpc_server_handle_req_in()) got req x1862456998031616 00000100:00100000:0.0:1776178144.097739:0:15213:0:(nrs_fifo.c:179:nrs_fifo_req_get()) NRS start fifo request from 12345-0@lo, seq: 642 00000100:00100000:0.0:1776178144.097743:0:15213:0:(service.c:2324:ptlrpc_server_handle_request()) Handling RPC req@00000000e59ab59a pname:cluuid+ref:pid:xid:nid:opc:job mdt_out00_003:lustre-MDT0000-mdtlov_UUID+6:3354:x1862456998031360:12345-0@lo:400:kworker/u8:2.0 00000100:00100000:0.0:1776178144.097777:0:15213:0:(service.c:2372:ptlrpc_server_handle_request()) Handled RPC req@00000000e59ab59a pname:cluuid+ref:pid:xid:nid:opc:job mdt_out00_003:lustre-MDT0000-mdtlov_UUID+6:3354:x1862456998031360:12345-0@lo:400:kworker/u8:2.0 Request processed in 33us (126us total) trans 0 rc 0/0 00000100:00100000:0.0:1776178144.097783:0:15213:0:(nrs_fifo.c:241:nrs_fifo_req_stop()) NRS stop fifo request from 12345-0@lo, seq: 642 00000100:00100000:0.0:1776178144.097794:0:15213:0:(nrs_fifo.c:179:nrs_fifo_req_get()) NRS start fifo request from 12345-0@lo, seq: 643 00000100:00100000:0.0:1776178144.097797:0:15213:0:(service.c:2324:ptlrpc_server_handle_request()) Handling RPC req@00000000f06d9dbd pname:cluuid+ref:pid:xid:nid:opc:job mdt_out00_003:lustre-MDT0000-mdtlov_UUID+5:3354:x1862456998031616:12345-0@lo:41:osp-pre-1-0.0 00000004:00000024:0.0:1776178144.097830:0:15213:0:(mdt_handler.c:548:mdt_statfs()) blocks cached 0 granted 2097152 pending 0 free 1446039552 avail 1316577280 00000100:00100000:0.0:1776178144.097866:0:15213:0:(service.c:2372:ptlrpc_server_handle_request()) Handled RPC req@00000000f06d9dbd pname:cluuid+ref:pid:xid:nid:opc:job mdt_out00_003:lustre-MDT0000-mdtlov_UUID+5:3354:x1862456998031616:12345-0@lo:41:osp-pre-1-0.0 Request processed in 67us (181us total) trans 0 rc 0/0 00000100:00100000:0.0:1776178144.097871:0:15213:0:(nrs_fifo.c:241:nrs_fifo_req_stop()) NRS stop fifo request from 12345-0@lo, seq: 643 00000100:00100000:0.0:1776178144.097888:0:6257:0:(nrs_fifo.c:179:nrs_fifo_req_get()) NRS start fifo request from 12345-0@lo, seq: 816 00000100:00100000:0.0:1776178144.097891:0:6257:0:(service.c:2324:ptlrpc_server_handle_request()) Handling RPC req@000000004dd93690 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_002:lustre-MDT0001-lwp-OST0000_UUID+5:3354:x1862456998030848:12345-0@lo:400:kworker/u8:2.0 00000100:00100000:0.0:1776178144.097917:0:6257:0:(service.c:2372:ptlrpc_server_handle_request()) Handled RPC req@000000004dd93690 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_002:lustre-MDT0001-lwp-OST0000_UUID+5:3354:x1862456998030848:12345-0@lo:400:kworker/u8:2.0 Request processed in 25us (334us total) trans 0 rc 0/0 00000100:00100000:0.0:1776178144.097923:0:6257:0:(nrs_fifo.c:241:nrs_fifo_req_stop()) NRS stop fifo request from 12345-0@lo, seq: 816 00000100:00100000:0.0:1776178144.097959:0:3354:0:(client.c:2215:ptlrpc_check_set()) Completed RPC req@000000006b0ae3d8 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_01:lustre-MDT0001-lwp-OST0000_UUID:3354:1862456998030848:0@lo:400:kworker/u8:2.0 00000100:00100000:0.0:1776178144.097968:0:3354:0:(client.c:2215:ptlrpc_check_set()) Completed RPC req@00000000456ce710 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_01:lustre-MDT0000-mdtlov_UUID:3354:1862456998031360:0@lo:400:kworker/u8:2.0 00000100:00100000:0.0:1776178144.097977:0:3354:0:(client.c:2215:ptlrpc_check_set()) Completed RPC req@000000000154171f pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_01:lustre-MDT0000-mdtlov_UUID:3354:1862456998031616:0@lo:41:osp-pre-1-0.0 00000100:00100000:0.0:1776178144.098002:0:20764:0:(service.c:2152:ptlrpc_server_handle_req_in()) got req x1862456998031488 00000100:00100000:0.0:1776178144.098023:0:20764:0:(service.c:2152:ptlrpc_server_handle_req_in()) got req x1862456998031168 00000100:00100000:0.0:1776178144.098037:0:20764:0:(service.c:2152:ptlrpc_server_handle_req_in()) got req x1862456998031424 00000100:00100000:0.0:1776178144.098050:0:20764:0:(service.c:2152:ptlrpc_server_handle_req_in()) got req x1862456998031104 00000100:00100000:1.0:1776178144.098085:0:11281:0:(service.c:2324:ptlrpc_server_handle_request()) Handling RPC req@00000000500c0ae6 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_004:lustre-MDT0000-lwp-OST0001_UUID+5:3354:x1862456998030912:12345-0@lo:400:kworker/u8:2.0 00000100:00100000:0.0:1776178144.098098:0:20764:0:(nrs_fifo.c:179:nrs_fifo_req_get()) NRS start fifo request from 12345-0@lo, seq: 448 00000100:00100000:0.0:1776178144.098101:0:20764:0:(service.c:2324:ptlrpc_server_handle_request()) Handling RPC req@00000000d668bb9c pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_015:lustre-MDT0000-mdtlov_UUID+5:3356:x1862456998031488:12345-0@lo:400:kworker/u8:2.0 00000100:00100000:1.0:1776178144.098122:0:11281:0:(service.c:2372:ptlrpc_server_handle_request()) Handled RPC req@00000000500c0ae6 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_004:lustre-MDT0000-lwp-OST0001_UUID+5:3354:x1862456998030912:12345-0@lo:400:kworker/u8:2.0 Request processed in 414us (688us total) trans 0 rc 0/0 00000100:00100000:0.0:1776178144.098129:0:20764:0:(service.c:2372:ptlrpc_server_handle_request()) Handled RPC req@00000000d668bb9c pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_015:lustre-MDT0000-mdtlov_UUID+5:3356:x1862456998031488:12345-0@lo:400:kworker/u8:2.0 Request processed in 27us (1234us total) trans 0 rc 0/0 00000100:00100000:1.0:1776178144.098129:0:11281:0:(nrs_fifo.c:241:nrs_fifo_req_stop()) NRS stop fifo request from 12345-0@lo, seq: 815 00000100:00100000:0.0:1776178144.098134:0:20764:0:(nrs_fifo.c:241:nrs_fifo_req_stop()) NRS stop fifo request from 12345-0@lo, seq: 448 00000100:00100000:0.0:1776178144.098143:0:20764:0:(nrs_fifo.c:179:nrs_fifo_req_get()) NRS start fifo request from 12345-0@lo, seq: 449 00000100:00100000:0.0:1776178144.098146:0:20764:0:(service.c:2324:ptlrpc_server_handle_request()) Handling RPC req@00000000d2922afa pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_015:lustre-MDT0001-mdtlov_UUID+5:3354:x1862456998031168:12345-0@lo:400:kworker/u8:2.0 00000100:00100000:1.0:1776178144.098147:0:20769:0:(nrs_fifo.c:179:nrs_fifo_req_get()) NRS start fifo request from 12345-0@lo, seq: 450 00000100:00100000:1.0:1776178144.098150:0:20769:0:(service.c:2324:ptlrpc_server_handle_request()) Handling RPC req@000000000f59f1a8 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_018:lustre-MDT0000-mdtlov_UUID+5:3354:x1862456998031424:12345-0@lo:400:kworker/u8:2.0 00000100:00100000:0.0:1776178144.098170:0:20764:0:(service.c:2372:ptlrpc_server_handle_request()) Handled RPC req@00000000d2922afa pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_015:lustre-MDT0001-mdtlov_UUID+5:3354:x1862456998031168:12345-0@lo:400:kworker/u8:2.0 Request processed in 23us (688us total) trans 0 rc 0/0 00000100:00100000:1.0:1776178144.098170:0:20769:0:(service.c:2372:ptlrpc_server_handle_request()) Handled RPC req@000000000f59f1a8 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_018:lustre-MDT0000-mdtlov_UUID+5:3354:x1862456998031424:12345-0@lo:400:kworker/u8:2.0 Request processed in 19us (633us total) trans 0 rc 0/0 00000100:00100000:0.0:1776178144.098175:0:20764:0:(nrs_fifo.c:241:nrs_fifo_req_stop()) NRS stop fifo request from 12345-0@lo, seq: 449 00000100:00100000:1.0:1776178144.098313:0:20769:0:(nrs_fifo.c:241:nrs_fifo_req_stop()) NRS stop fifo request from 12345-0@lo, seq: 450 00000100:00100000:2.0:1776178144.098364:0:14440:0:(nrs_fifo.c:179:nrs_fifo_req_get()) NRS start fifo request from 12345-0@lo, seq: 451 00000100:00100000:2.0:1776178144.098368:0:14440:0:(service.c:2324:ptlrpc_server_handle_request()) Handling RPC req@00000000e1965319 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_004:lustre-MDT0001-mdtlov_UUID+5:3354:x1862456998031104:12345-0@lo:400:kworker/u8:2.0 00000100:00100000:2.0:1776178144.098399:0:14440:0:(service.c:2372:ptlrpc_server_handle_request()) Handled RPC req@00000000e1965319 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_004:lustre-MDT0001-mdtlov_UUID+5:3354:x1862456998031104:12345-0@lo:400:kworker/u8:2.0 Request processed in 30us (783us total) trans 0 rc 0/0 00000100:00100000:2.0:1776178144.098403:0:14440:0:(nrs_fifo.c:241:nrs_fifo_req_stop()) NRS stop fifo request from 12345-0@lo, seq: 451 00000100:00100000:2.0:1776178144.098469:0:3354:0:(client.c:2215:ptlrpc_check_set()) Completed RPC req@000000004a3d9287 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_01:lustre-MDT0001-mdtlov_UUID:3354:1862456998031104:0@lo:400:kworker/u8:2.0 00000100:00100000:2.0:1776178144.098479:0:3354:0:(client.c:2215:ptlrpc_check_set()) Completed RPC req@00000000a62daf9a pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_01:lustre-MDT0000-lwp-OST0001_UUID:3354:1862456998030912:0@lo:400:kworker/u8:2.0 00000100:00100000:2.0:1776178144.098488:0:3354:0:(client.c:2215:ptlrpc_check_set()) Completed RPC req@00000000736589c7 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_01:lustre-MDT0001-mdtlov_UUID:3354:1862456998031168:0@lo:400:kworker/u8:2.0 00000100:00100000:2.0:1776178144.098497:0:3354:0:(client.c:2215:ptlrpc_check_set()) Completed RPC req@00000000f114dadf pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_01:lustre-MDT0000-mdtlov_UUID:3354:1862456998031424:0@lo:400:kworker/u8:2.0 00000100:00100000:3.0:1776178144.098716:0:3353:0:(ptlrpcd.c:415:ptlrpcd_check()) transfer 1 async RPCs [1->0] 00000100:00100000:3.0:1776178144.098728:0:3353:0:(client.c:1733:ptlrpc_send_new_req()) Sending RPC req@00000000d6e5d9ba pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_00:lustre-MDT0001-mdtlov_UUID:3353:1862456998031680:0@lo:41:osp-pre-0-1.0 00000100:00100000:3.0:1776178144.098759:0:3353:0:(events.c:361:request_in_callback()) peer: 12345-0@lo (source: 12345-0@lo) 00000100:00100000:3.0:1776178144.098785:0:14128:0:(service.c:2152:ptlrpc_server_handle_req_in()) got req x1862456998031680 00000100:00100000:3.0:1776178144.098809:0:14128:0:(nrs_fifo.c:179:nrs_fifo_req_get()) NRS start fifo request from 12345-0@lo, seq: 644 00000100:00100000:3.0:1776178144.098824:0:14128:0:(service.c:2324:ptlrpc_server_handle_request()) Handling RPC req@0000000021b0d516 pname:cluuid+ref:pid:xid:nid:opc:job mdt_out00_002:lustre-MDT0001-mdtlov_UUID+6:3353:x1862456998031680:12345-0@lo:41:osp-pre-0-1.0 00000004:00000024:3.0:1776178144.098852:0:14128:0:(mdt_handler.c:548:mdt_statfs()) blocks cached 0 granted 2097152 pending 0 free 1446121472 avail 1316659200 00000100:00100000:3.0:1776178144.098883:0:14128:0:(service.c:2372:ptlrpc_server_handle_request()) Handled RPC req@0000000021b0d516 pname:cluuid+ref:pid:xid:nid:opc:job mdt_out00_002:lustre-MDT0001-mdtlov_UUID+6:3353:x1862456998031680:12345-0@lo:41:osp-pre-0-1.0 Request processed in 58us (124us total) trans 0 rc 0/0 00000100:00100000:3.0:1776178144.098889:0:14128:0:(nrs_fifo.c:241:nrs_fifo_req_stop()) NRS stop fifo request from 12345-0@lo, seq: 644 00000100:00100000:3.0:1776178144.098917:0:3353:0:(client.c:2215:ptlrpc_check_set()) Completed RPC req@00000000d6e5d9ba pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_00:lustre-MDT0001-mdtlov_UUID:3353:1862456998031680:0@lo:41:osp-pre-0-1.0 00000100:00100000:3.0:1776178144.098950:0:3356:0:(client.c:2215:ptlrpc_check_set()) Completed RPC req@0000000043589f56 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_03:lustre-MDT0000-mdtlov_UUID:3356:1862456998031488:0@lo:400:kworker/u8:2.0 00000100:00100000:3.0:1776178144.100834:0:3355:0:(client.c:1733:ptlrpc_send_new_req()) Sending RPC req@0000000003cebab2 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_02:lustre-MDT0000-mdtlov_UUID:3355:1862456998031744:0@lo:13:osp-pre-0-0.0 00000100:00100000:3.0:1776178144.100907:0:3355:0:(events.c:361:request_in_callback()) peer: 12345-0@lo (source: 12345-0@lo) 00000100:00100000:2.0:1776178144.101085:0:8232:0:(service.c:2152:ptlrpc_server_handle_req_in()) got req x1862456998031744 00000100:00100000:2.0:1776178144.101143:0:8232:0:(nrs_fifo.c:179:nrs_fifo_req_get()) NRS start fifo request from 12345-0@lo, seq: 432 00000100:00100000:2.0:1776178144.101149:0:8232:0:(service.c:2324:ptlrpc_server_handle_request()) Handling RPC req@00000000d6e5d9ba pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_create00:lustre-MDT0000-mdtlov_UUID+5:3355:x1862456998031744:12345-0@lo:13:osp-pre-0-0.0 00002000:00000024:2.0:1776178144.101178:0:8232:0:(ofd_obd.c:740:ofd_statfs()) blocks cached 315392 granted 484883776 pending 0 free 3923689472 avail 3692167168 00000100:00100000:2.0:1776178144.101998:0:8232:0:(service.c:2372:ptlrpc_server_handle_request()) Handled RPC req@00000000d6e5d9ba pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_create00:lustre-MDT0000-mdtlov_UUID+5:3355:x1862456998031744:12345-0@lo:13:osp-pre-0-0.0 Request processed in 845us (1088us total) trans 0 rc 0/0 00000100:00100000:2.0:1776178144.102010:0:8232:0:(nrs_fifo.c:241:nrs_fifo_req_stop()) NRS stop fifo request from 12345-0@lo, seq: 432 00000100:00100000:0.0:1776178144.102142:0:3355:0:(client.c:2215:ptlrpc_check_set()) Completed RPC req@0000000003cebab2 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_02:lustre-MDT0000-mdtlov_UUID:3355:1862456998031744:0@lo:13:osp-pre-0-0.0 00010000:00010000:1.0:1776178144.672951:0:9225:0:(ldlm_lockd.c:2792:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:1.0:1776178145.059575:0:3348:0:(events.c:361:request_in_callback()) peer: 12345-192.168.204.3@tcp (source: 12345-192.168.204.3@tcp) 00000100:00100000:1.0:1776178145.059629:0:3348:0:(events.c:361:request_in_callback()) peer: 12345-192.168.204.3@tcp (source: 12345-192.168.204.3@tcp) 00000100:00100000:1.0:1776178145.059677:0:3348:0:(events.c:361:request_in_callback()) peer: 12345-192.168.204.3@tcp (source: 12345-192.168.204.3@tcp) 00000100:00100000:1.0:1776178145.059710:0:3348:0:(events.c:361:request_in_callback()) peer: 12345-192.168.204.3@tcp (source: 12345-192.168.204.3@tcp) 00000100:00100000:1.0:1776178145.059742:0:3348:0:(events.c:361:request_in_callback()) peer: 12345-192.168.204.3@tcp (source: 12345-192.168.204.3@tcp) 00000100:00100000:1.0:1776178145.059782:0:11281:0:(service.c:2152:ptlrpc_server_handle_req_in()) got req x1862456979084480 00000100:00100000:1.0:1776178145.059849:0:11281:0:(service.c:2152:ptlrpc_server_handle_req_in()) got req x1862456979084544 00000100:00100000:1.0:1776178145.059871:0:11281:0:(nrs_fifo.c:179:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.204.3@tcp, seq: 817 00000100:00100000:1.0:1776178145.059879:0:11281:0:(service.c:2324:ptlrpc_server_handle_request()) Handling RPC req@00000000741c4ec9 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_004:b50963bd-8eba-4432-9a54-8888d00e3fa1+21:2270:x1862456979084480:12345-192.168.204.3@tcp:400:kworker/u8:3.0 00000100:00100000:1.0:1776178145.059967:0:11281:0:(service.c:2372:ptlrpc_server_handle_request()) Handled RPC req@00000000741c4ec9 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_004:b50963bd-8eba-4432-9a54-8888d00e3fa1+21:2270:x1862456979084480:12345-192.168.204.3@tcp:400:kworker/u8:3.0 Request processed in 89us (405us total) trans 0 rc 0/0 00000100:00100000:1.0:1776178145.059976:0:11281:0:(nrs_fifo.c:241:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.204.3@tcp, seq: 817 00000100:00100000:1.0:1776178145.059995:0:11281:0:(nrs_fifo.c:179:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.204.3@tcp, seq: 818 00000100:00100000:1.0:1776178145.059999:0:11281:0:(service.c:2324:ptlrpc_server_handle_request()) Handling RPC req@000000005b5b62e4 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_004:b50963bd-8eba-4432-9a54-8888d00e3fa1+6:2270:x1862456979084544:12345-192.168.204.3@tcp:400:kworker/u8:3.0 00000100:00100000:1.0:1776178145.060037:0:11281:0:(service.c:2372:ptlrpc_server_handle_request()) Handled RPC req@000000005b5b62e4 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_004:b50963bd-8eba-4432-9a54-8888d00e3fa1+6:2270:x1862456979084544:12345-192.168.204.3@tcp:400:kworker/u8:3.0 Request processed in 37us (327us total) trans 0 rc 0/0 00000100:00100000:1.0:1776178145.060043:0:11281:0:(nrs_fifo.c:241:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.204.3@tcp, seq: 818 00000100:00100000:0.0:1776178145.060187:0:20140:0:(service.c:2152:ptlrpc_server_handle_req_in()) got req x1862456979084416 00000100:00100000:0.0:1776178145.060226:0:20140:0:(nrs_fifo.c:179:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.204.3@tcp, seq: 132 00000100:00100000:0.0:1776178145.060231:0:20140:0:(service.c:2324:ptlrpc_server_handle_request()) Handling RPC req@0000000091f9a08d pname:cluuid+ref:pid:xid:nid:opc:job ll_mgs_0002:5b5c66b1-1a2f-4651-8548-0c75b9a38a94+9:2270:x1862456979084416:12345-192.168.204.3@tcp:400:kworker/u8:3.0 00000100:00100000:0.0:1776178145.060260:0:20140:0:(service.c:2372:ptlrpc_server_handle_request()) Handled RPC req@0000000091f9a08d pname:cluuid+ref:pid:xid:nid:opc:job ll_mgs_0002:5b5c66b1-1a2f-4651-8548-0c75b9a38a94+9:2270:x1862456979084416:12345-192.168.204.3@tcp:400:kworker/u8:3.0 Request processed in 28us (583us total) trans 0 rc 0/0 00000100:00100000:0.0:1776178145.060265:0:20140:0:(nrs_fifo.c:241:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.204.3@tcp, seq: 132 00000100:00100000:0.0:1776178145.060406:0:20764:0:(service.c:2152:ptlrpc_server_handle_req_in()) got req x1862456979084608 00000100:00100000:0.0:1776178145.060428:0:20764:0:(service.c:2152:ptlrpc_server_handle_req_in()) got req x1862456979084672 00000100:00100000:0.0:1776178145.060444:0:20764:0:(nrs_fifo.c:179:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.204.3@tcp, seq: 452 00000100:00100000:0.0:1776178145.060448:0:20764:0:(service.c:2324:ptlrpc_server_handle_request()) Handling RPC req@000000001ea4218d pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_015:b50963bd-8eba-4432-9a54-8888d00e3fa1+9:2270:x1862456979084608:12345-192.168.204.3@tcp:400:kworker/u8:3.0 00000100:00100000:0.0:1776178145.060488:0:20764:0:(service.c:2372:ptlrpc_server_handle_request()) Handled RPC req@000000001ea4218d pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_015:b50963bd-8eba-4432-9a54-8888d00e3fa1+9:2270:x1862456979084608:12345-192.168.204.3@tcp:400:kworker/u8:3.0 Request processed in 39us (859us total) trans 0 rc 0/0 00000100:00100000:0.0:1776178145.060495:0:20764:0:(nrs_fifo.c:241:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.204.3@tcp, seq: 452 00000100:00100000:0.0:1776178145.060529:0:20764:0:(nrs_fifo.c:179:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.204.3@tcp, seq: 453 00000100:00100000:0.0:1776178145.060533:0:20764:0:(service.c:2324:ptlrpc_server_handle_request()) Handling RPC req@0000000022e48f50 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_015:b50963bd-8eba-4432-9a54-8888d00e3fa1+8:2270:x1862456979084672:12345-192.168.204.3@tcp:400:kworker/u8:3.0 00000100:00100000:0.0:1776178145.060559:0:20764:0:(service.c:2372:ptlrpc_server_handle_request()) Handled RPC req@0000000022e48f50 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_015:b50963bd-8eba-4432-9a54-8888d00e3fa1+8:2270:x1862456979084672:12345-192.168.204.3@tcp:400:kworker/u8:3.0 Request processed in 25us (817us total) trans 0 rc 0/0 00000100:00100000:0.0:1776178145.060565:0:20764:0:(nrs_fifo.c:241:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.204.3@tcp, seq: 453 00000100:00100000:0.0:1776178145.633859:0:3355:0:(ptlrpcd.c:415:ptlrpcd_check()) transfer 1 async RPCs [3->2] 00000100:00100000:0.0:1776178145.633894:0:3355:0:(client.c:1733:ptlrpc_send_new_req()) Sending RPC req@00000000bcefda18 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_02:lustre-MDT0001-mdtlov_UUID:3355:1862456998031808:0@lo:13:osp-pre-1-1.0 00000100:00100000:0.0:1776178145.634002:0:3355:0:(events.c:361:request_in_callback()) peer: 12345-0@lo (source: 12345-0@lo) 00000100:00100000:3.0:1776178145.634110:0:8231:0:(service.c:2152:ptlrpc_server_handle_req_in()) got req x1862456998031808 00000100:00100000:3.0:1776178145.634195:0:8231:0:(nrs_fifo.c:179:nrs_fifo_req_get()) NRS start fifo request from 12345-0@lo, seq: 433 00000100:00100000:3.0:1776178145.634203:0:8231:0:(service.c:2324:ptlrpc_server_handle_request()) Handling RPC req@0000000003cebab2 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_create00:lustre-MDT0001-mdtlov_UUID+5:3355:x1862456998031808:12345-0@lo:13:osp-pre-1-1.0 00002000:00000024:3.0:1776178145.634232:0:8231:0:(ofd_obd.c:740:ofd_statfs()) blocks cached 229376 granted 491746368 pending 0 free 3923689472 avail 3692167168 00000100:00100000:3.0:1776178145.634288:0:8231:0:(service.c:2372:ptlrpc_server_handle_request()) Handled RPC req@0000000003cebab2 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_create00:lustre-MDT0001-mdtlov_UUID+5:3355:x1862456998031808:12345-0@lo:13:osp-pre-1-1.0 Request processed in 84us (288us total) trans 0 rc 0/0 00000100:00100000:3.0:1776178145.634295:0:8231:0:(nrs_fifo.c:241:nrs_fifo_req_stop()) NRS stop fifo request from 12345-0@lo, seq: 433 00000100:00100000:0.0:1776178145.634619:0:3355:0:(client.c:2215:ptlrpc_check_set()) Completed RPC req@00000000bcefda18 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_02:lustre-MDT0001-mdtlov_UUID:3355:1862456998031808:0@lo:13:osp-pre-1-1.0 00000100:00100000:0.0:1776178145.664289:0:3347:0:(events.c:361:request_in_callback()) peer: 12345-192.168.204.3@tcp (source: 12345-192.168.204.3@tcp) 00000100:00100000:0.0:1776178145.664348:0:3347:0:(events.c:361:request_in_callback()) peer: 12345-192.168.204.3@tcp (source: 12345-192.168.204.3@tcp) 00000100:00100000:0.0:1776178145.664389:0:3347:0:(events.c:361:request_in_callback()) peer: 12345-192.168.204.3@tcp (source: 12345-192.168.204.3@tcp) 00000100:00100000:0.0:1776178145.664436:0:3347:0:(events.c:361:request_in_callback()) peer: 12345-192.168.204.3@tcp (source: 12345-192.168.204.3@tcp) 00000100:00100000:3.0:1776178145.664565:0:20759:0:(service.c:2152:ptlrpc_server_handle_req_in()) got req x1862456979084992 00000100:00100000:3.0:1776178145.664626:0:20759:0:(nrs_fifo.c:179:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.204.3@tcp, seq: 454 00000100:00100000:3.0:1776178145.664635:0:20759:0:(service.c:2324:ptlrpc_server_handle_request()) Handling RPC req@00000000e1d2e300 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_010:b50963bd-8eba-4432-9a54-8888d00e3fa1+8:2273:x1862456979084992:12345-192.168.204.3@tcp:400:kworker/u8:3.0 00000100:00100000:3.0:1776178145.664722:0:20759:0:(service.c:2372:ptlrpc_server_handle_request()) Handled RPC req@00000000e1d2e300 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_010:b50963bd-8eba-4432-9a54-8888d00e3fa1+8:2273:x1862456979084992:12345-192.168.204.3@tcp:400:kworker/u8:3.0 Request processed in 87us (333us total) trans 0 rc 0/0 00000100:00100000:3.0:1776178145.664730:0:20759:0:(nrs_fifo.c:241:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.204.3@tcp, seq: 454 00000100:00100000:1.0:1776178145.664955:0:6257:0:(service.c:2152:ptlrpc_server_handle_req_in()) got req x1862456979084800 00000100:00100000:1.0:1776178145.664993:0:6257:0:(service.c:2152:ptlrpc_server_handle_req_in()) got req x1862456979084864 00000100:00100000:1.0:1776178145.665016:0:6257:0:(nrs_fifo.c:179:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.204.3@tcp, seq: 819 00000100:00100000:1.0:1776178145.665021:0:6257:0:(service.c:2324:ptlrpc_server_handle_request()) Handling RPC req@00000000c32cfd7b pname:cluuid+ref:pid:xid:nid:opc:job mdt00_002:b50963bd-8eba-4432-9a54-8888d00e3fa1+21:2269:x1862456979084800:12345-192.168.204.3@tcp:38:kworker/u8:3.0 00010000:02000400:1.0:1776178145.665049:0:6257:0:(ldlm_lib.c:914:target_handle_reconnect()) lustre-MDT0000: Client b50963bd-8eba-4432-9a54-8888d00e3fa1 (at 192.168.204.3@tcp) reconnecting 00000100:00100000:3.0:1776178145.665586:0:8250:0:(nrs_fifo.c:179:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.204.3@tcp, seq: 820 00000100:00100000:3.0:1776178145.665594:0:8250:0:(service.c:2324:ptlrpc_server_handle_request()) Handling RPC req@00000000d6cbf5e8 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_003:b50963bd-8eba-4432-9a54-8888d00e3fa1+6:2273:x1862456979084864:12345-192.168.204.3@tcp:400:kworker/u8:3.0 00000100:00100000:3.0:1776178145.665651:0:8250:0:(service.c:2372:ptlrpc_server_handle_request()) Handled RPC req@00000000d6cbf5e8 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_003:b50963bd-8eba-4432-9a54-8888d00e3fa1+6:2273:x1862456979084864:12345-192.168.204.3@tcp:400:kworker/u8:3.0 Request processed in 56us (1303us total) trans 0 rc 0/0 00000100:00100000:3.0:1776178145.665658:0:8250:0:(nrs_fifo.c:241:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.204.3@tcp, seq: 820 00000100:00100000:3.0:1776178145.665709:0:20137:0:(service.c:2152:ptlrpc_server_handle_req_in()) got req x1862456979084736 00000100:00100000:3.0:1776178145.665740:0:20137:0:(nrs_fifo.c:179:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.204.3@tcp, seq: 133 00000100:00100000:3.0:1776178145.665744:0:20137:0:(service.c:2324:ptlrpc_server_handle_request()) Handling RPC req@00000000c0367ec3 pname:cluuid+ref:pid:xid:nid:opc:job ll_mgs_0000:5b5c66b1-1a2f-4651-8548-0c75b9a38a94+9:2273:x1862456979084736:12345-192.168.204.3@tcp:400:kworker/u8:3.0 00000100:00100000:3.0:1776178145.665772:0:20137:0:(service.c:2372:ptlrpc_server_handle_request()) Handled RPC req@00000000c0367ec3 pname:cluuid+ref:pid:xid:nid:opc:job ll_mgs_0000:5b5c66b1-1a2f-4651-8548-0c75b9a38a94+9:2273:x1862456979084736:12345-192.168.204.3@tcp:400:kworker/u8:3.0 Request processed in 27us (1337us total) trans 0 rc 0/0 00000100:00100000:3.0:1776178145.665779:0:20137:0:(nrs_fifo.c:241:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.204.3@tcp, seq: 133 00000100:00100000:0.0:1776178145.667041:0:3347:0:(events.c:361:request_in_callback()) peer: 12345-192.168.204.3@tcp (source: 12345-192.168.204.3@tcp) 00000100:00100000:2.0:1776178145.667143:0:20762:0:(service.c:2152:ptlrpc_server_handle_req_in()) got req x1862456979084928 00000100:00100000:2.0:1776178145.667186:0:20762:0:(nrs_fifo.c:179:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.204.3@tcp, seq: 455 00000100:00100000:2.0:1776178145.667191:0:20762:0:(service.c:2324:ptlrpc_server_handle_request()) Handling RPC req@000000002cf093e4 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_013:b50963bd-8eba-4432-9a54-8888d00e3fa1+9:2273:x1862456979084928:12345-192.168.204.3@tcp:400:kworker/u8:3.0 00000100:00100000:2.0:1776178145.667261:0:20762:0:(service.c:2372:ptlrpc_server_handle_request()) Handled RPC req@000000002cf093e4 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_013:b50963bd-8eba-4432-9a54-8888d00e3fa1+9:2273:x1862456979084928:12345-192.168.204.3@tcp:400:kworker/u8:3.0 Request processed in 69us (225us total) trans 0 rc 0/0 00000100:00100000:2.0:1776178145.667268:0:20762:0:(nrs_fifo.c:241:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.204.3@tcp, seq: 455 00010000:00080000:1.0:1776178145.674193:0:6257:0:(ldlm_lib.c:1370:target_handle_connect()) lustre-MDT0000: connection from b50963bd-8eba-4432-9a54-8888d00e3fa1@192.168.204.3@tcp t18446615405130655872 exp 00000000117deaf8 cur 853 last 1776178145 00000004:00100000:1.0:1776178145.674237:0:6257:0:(mdt_handler.c:6641:mdt_connect_internal()) lustre-MDT0000: cli 192.168.204.3@tcp supports cksum type f7, return f7 00000100:00080000:1.0:1776178145.674270:0:6257:0:(import.c:86:import_set_state_nolock()) 00000000095a2387 : changing import state from FULL to RECOVER 00000100:00080000:1.0:1776178145.674282:0:6257:0:(import.c:86:import_set_state_nolock()) 00000000095a2387 : changing import state from RECOVER to FULL 00000100:00100000:1.0:1776178145.674382:0:6257:0:(service.c:2372:ptlrpc_server_handle_request()) Handled RPC req@00000000c32cfd7b pname:cluuid+ref:pid:xid:nid:opc:job mdt00_002:b50963bd-8eba-4432-9a54-8888d00e3fa1+21:2269:x1862456979084800:12345-192.168.204.3@tcp:38:kworker/u8:3.0 Request processed in 9359us (10102us total) trans 0 rc 0/0 00000100:00100000:1.0:1776178145.674394:0:6257:0:(nrs_fifo.c:241:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.204.3@tcp, seq: 819 00010000:00010000:2.0:1776178145.696445:0:9224:0:(ldlm_lockd.c:2792:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:0.0:1776178145.701262:0:3347:0:(events.c:361:request_in_callback()) peer: 12345-192.168.204.3@tcp (source: 12345-192.168.204.3@tcp) 00000100:00100000:0.0:1776178145.701313:0:3347:0:(events.c:361:request_in_callback()) peer: 12345-192.168.204.3@tcp (source: 12345-192.168.204.3@tcp) 00000100:00100000:0.0:1776178145.701449:0:3347:0:(events.c:361:request_in_callback()) peer: 12345-192.168.204.3@tcp (source: 12345-192.168.204.3@tcp) 00000100:00100000:0.0:1776178145.701546:0:3347:0:(events.c:361:request_in_callback()) peer: 12345-192.168.204.3@tcp (source: 12345-192.168.204.3@tcp) 00000100:00100000:2.0:1776178145.701609:0:6255:0:(service.c:2152:ptlrpc_server_handle_req_in()) got req x1862456979080640 00000100:00100000:2.0:1776178145.701673:0:6255:0:(service.c:2152:ptlrpc_server_handle_req_in()) got req x1862456979081472 00000100:00100000:2.0:1776178145.701693:0:6255:0:(service.c:2152:ptlrpc_server_handle_req_in()) got req x1862456979081088 00000100:00100000:2.0:1776178145.701714:0:6255:0:(nrs_fifo.c:179:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.204.3@tcp, seq: 6491 00000100:00100000:2.0:1776178145.701722:0:6255:0:(service.c:2324:ptlrpc_server_handle_request()) Handling RPC req@00000000865e4f65 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_000:b50963bd-8eba-4432-9a54-8888d00e3fa1+23:17005:x1862456979080640:12345-192.168.204.3@tcp:36:chmod.0 00000004:00080000:2.0:1776178145.701786:0:6255:0:(mdt_internal.h:1195:mdt_check_resent()) @@@ no reply data found for RESENT req, rc = 0 req@00000000865e4f65 x1862456979080640/t0(0) o36->b50963bd-8eba-4432-9a54-8888d00e3fa1@192.168.204.3@tcp:677/0 lens 488/3152 e 0 to 0 dl 1776178192 ref 1 fl Interpret:/2/0 rc 0/0 job:'chmod.0' 00010000:00010000:2.0:1776178145.701799:0:6255:0:(ldlm_lockd.c:1720:ldlm_request_cancel()) ### server-side cancel handler START: 2 locks, starting at 0 00010000:00010000:2.0:1776178145.701841:0:6255:0:(ldlm_lock.c:216:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: mdt-lustre-MDT0000_UUID lock: 00000000fcb32840/0xa40083ab9bf5302b lrc: 0/0,0 mode: --/PR res: [0x2000013a1:0x2:0x0].0x0 bits 0x1b/0x0 rrc: 4 type: IBT gid 0 flags: 0x44a01000000000 nid: 192.168.204.3@tcp remote: 0x771f68aa1eb279ac expref: 23 pid: 6256 timeout: 0 lvb_type: 0 00010000:00010000:2.0:1776178145.701866:0:6255:0:(ldlm_lock.c:216:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: mdt-lustre-MDT0000_UUID lock: 00000000fa092262/0xa40083ab9bf52f05 lrc: 0/0,0 mode: --/CR res: [0x2000013a1:0x2:0x0].0x0 bits 0x9/0x0 rrc: 3 type: IBT gid 0 flags: 0x44a01000000000 nid: 192.168.204.3@tcp remote: 0x771f68aa1eb27974 expref: 22 pid: 6256 timeout: 0 lvb_type: 0 00010000:00010000:2.0:1776178145.701882:0:6255:0:(ldlm_inodebits.c:94:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x2000013a1:0x2:0x0].0x0 (000000005ad330c7) 00010000:00010000:2.0:1776178145.701894:0:6255:0:(ldlm_lockd.c:1782:ldlm_request_cancel()) ### server-side cancel handler END 00010000:00010000:2.0:1776178145.701928:0:6255:0:(ldlm_lock.c:766:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: mdt-lustre-MDT0000_UUID lock: 0000000073f2c7a9/0xa40083ab9bf530a9 lrc: 3/0,1 mode: --/PW res: [0x2000013a1:0x2:0x0].0x0 bits 0x0/0x0 rrc: 2 type: IBT gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 6255 timeout: 0 lvb_type: 0 00010000:00010000:2.0:1776178145.701941:0:6255:0:(ldlm_lock.c:1064:ldlm_granted_list_add_lock()) ### About to add lock: ns: mdt-lustre-MDT0000_UUID lock: 0000000073f2c7a9/0xa40083ab9bf530a9 lrc: 3/0,1 mode: PW/PW res: [0x2000013a1:0x2:0x0].0x0 bits 0x13/0x0 rrc: 2 type: IBT gid 0 flags: 0x50210000000000 nid: local remote: 0x0 expref: -99 pid: 6255 timeout: 0 lvb_type: 0 00010000:00010000:2.0:1776178145.701951:0:6255:0:(ldlm_request.c:528:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: mdt-lustre-MDT0000_UUID lock: 0000000073f2c7a9/0xa40083ab9bf530a9 lrc: 3/0,1 mode: PW/PW res: [0x2000013a1:0x2:0x0].0x0 bits 0x13/0x0 rrc: 2 type: IBT gid 0 flags: 0x40210000000000 nid: local remote: 0x0 expref: -99 pid: 6255 timeout: 0 lvb_type: 0 00000004:00010000:2.0:1776178145.702216:0:6255:0:(mdt_handler.c:3992:mdt_save_lock()) ### save lock request 00000000865e4f65 reply state 00000000a48f161d transno 12884901925 ns: mdt-lustre-MDT0000_UUID lock: 0000000073f2c7a9/0xa40083ab9bf530a9 lrc: 3/0,1 mode: PW/PW res: [0x2000013a1:0x2:0x0].0x0 bits 0x13/0x0 rrc: 2 type: IBT gid 0 flags: 0x40210000000000 nid: local remote: 0x0 expref: -99 pid: 6255 timeout: 0 lvb_type: 0 00000100:00100000:2.0:1776178145.702325:0:6255:0:(service.c:2372:ptlrpc_server_handle_request()) Handled RPC req@00000000865e4f65 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_000:b50963bd-8eba-4432-9a54-8888d00e3fa1+23:17005:x1862456979080640:12345-192.168.204.3@tcp:36:chmod.0 Request processed in 603us (1079us total) trans 12884901925 rc 0/0 00000100:00100000:2.0:1776178145.702332:0:6255:0:(nrs_fifo.c:241:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.204.3@tcp, seq: 6491 00000100:00100000:2.0:1776178145.702353:0:6255:0:(nrs_fifo.c:179:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.204.3@tcp, seq: 6492 00000100:00100000:2.0:1776178145.702357:0:6255:0:(service.c:2324:ptlrpc_server_handle_request()) Handling RPC req@00000000597c3414 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_000:b50963bd-8eba-4432-9a54-8888d00e3fa1+22:17010:x1862456979081472:12345-192.168.204.3@tcp:36:chmod.0 00000004:00080000:2.0:1776178145.702379:0:6255:0:(mdt_internal.h:1195:mdt_check_resent()) @@@ no reply data found for RESENT req, rc = 0 req@00000000597c3414 x1862456979081472/t0(0) o36->b50963bd-8eba-4432-9a54-8888d00e3fa1@192.168.204.3@tcp:677/0 lens 488/3152 e 0 to 0 dl 1776178192 ref 1 fl Interpret:/2/0 rc 0/0 job:'chmod.0' 00010000:00010000:2.0:1776178145.702387:0:6255:0:(ldlm_lockd.c:1720:ldlm_request_cancel()) ### server-side cancel handler START: 2 locks, starting at 0 00010000:00010000:2.0:1776178145.702402:0:6255:0:(ldlm_lock.c:216:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: mdt-lustre-MDT0000_UUID lock: 00000000499bab84/0xa40083ab9bf53063 lrc: 0/0,0 mode: --/PR res: [0x2000013a1:0x7:0x0].0x0 bits 0x1b/0x0 rrc: 4 type: IBT gid 0 flags: 0x44a01000000000 nid: 192.168.204.3@tcp remote: 0x771f68aa1eb279cf expref: 22 pid: 6256 timeout: 0 lvb_type: 0 00010000:00010000:2.0:1776178145.702418:0:6255:0:(ldlm_lock.c:216:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: mdt-lustre-MDT0000_UUID lock: 000000007e7ea26f/0xa40083ab9bf52fd7 lrc: 0/0,0 mode: --/CR res: [0x2000013a1:0x7:0x0].0x0 bits 0x9/0x0 rrc: 3 type: IBT gid 0 flags: 0x44a01000000000 nid: 192.168.204.3@tcp remote: 0x771f68aa1eb27997 expref: 21 pid: 6256 timeout: 0 lvb_type: 0 00010000:00010000:2.0:1776178145.702427:0:6255:0:(ldlm_inodebits.c:94:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x2000013a1:0x7:0x0].0x0 (00000000665a3017) 00010000:00010000:2.0:1776178145.702434:0:6255:0:(ldlm_lockd.c:1782:ldlm_request_cancel()) ### server-side cancel handler END 00010000:00010000:2.0:1776178145.702447:0:6255:0:(ldlm_lock.c:766:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: mdt-lustre-MDT0000_UUID lock: 00000000e4217a5c/0xa40083ab9bf530b0 lrc: 3/0,1 mode: --/PW res: [0x2000013a1:0x7:0x0].0x0 bits 0x0/0x0 rrc: 2 type: IBT gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 6255 timeout: 0 lvb_type: 0 00010000:00010000:2.0:1776178145.702454:0:6255:0:(ldlm_lock.c:1064:ldlm_granted_list_add_lock()) ### About to add lock: ns: mdt-lustre-MDT0000_UUID lock: 00000000e4217a5c/0xa40083ab9bf530b0 lrc: 3/0,1 mode: PW/PW res: [0x2000013a1:0x7:0x0].0x0 bits 0x13/0x0 rrc: 2 type: IBT gid 0 flags: 0x50210000000000 nid: local remote: 0x0 expref: -99 pid: 6255 timeout: 0 lvb_type: 0 00010000:00010000:2.0:1776178145.702462:0:6255:0:(ldlm_request.c:528:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: mdt-lustre-MDT0000_UUID lock: 00000000e4217a5c/0xa40083ab9bf530b0 lrc: 3/0,1 mode: PW/PW res: [0x2000013a1:0x7:0x0].0x0 bits 0x13/0x0 rrc: 2 type: IBT gid 0 flags: 0x40210000000000 nid: local remote: 0x0 expref: -99 pid: 6255 timeout: 0 lvb_type: 0 00000004:00010000:2.0:1776178145.702524:0:6255:0:(mdt_handler.c:3992:mdt_save_lock()) ### save lock request 00000000597c3414 reply state 00000000d5c5ede9 transno 12884901926 ns: mdt-lustre-MDT0000_UUID lock: 00000000e4217a5c/0xa40083ab9bf530b0 lrc: 3/0,1 mode: PW/PW res: [0x2000013a1:0x7:0x0].0x0 bits 0x13/0x0 rrc: 2 type: IBT gid 0 flags: 0x40210000000000 nid: local remote: 0x0 expref: -99 pid: 6255 timeout: 0 lvb_type: 0 00000100:00100000:2.0:1776178145.702565:0:6255:0:(service.c:2372:ptlrpc_server_handle_request()) Handled RPC req@00000000597c3414 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_000:b50963bd-8eba-4432-9a54-8888d00e3fa1+22:17010:x1862456979081472:12345-192.168.204.3@tcp:36:chmod.0 Request processed in 207us (1252us total) trans 12884901926 rc 0/0 00000100:00100000:2.0:1776178145.702571:0:6255:0:(nrs_fifo.c:241:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.204.3@tcp, seq: 6492 00000100:00100000:2.0:1776178145.702586:0:6255:0:(nrs_fifo.c:179:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.204.3@tcp, seq: 6493 00000100:00100000:2.0:1776178145.702590:0:6255:0:(service.c:2324:ptlrpc_server_handle_request()) Handling RPC req@00000000a33eecf4 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_000:b50963bd-8eba-4432-9a54-8888d00e3fa1+21:17011:x1862456979081088:12345-192.168.204.3@tcp:36:chmod.0 00000004:00080000:2.0:1776178145.702608:0:6255:0:(mdt_internal.h:1195:mdt_check_resent()) @@@ no reply data found for RESENT req, rc = 0 req@00000000a33eecf4 x1862456979081088/t0(0) o36->b50963bd-8eba-4432-9a54-8888d00e3fa1@192.168.204.3@tcp:677/0 lens 488/3152 e 0 to 0 dl 1776178192 ref 1 fl Interpret:/2/0 rc 0/0 job:'chmod.0' 00010000:00010000:2.0:1776178145.702614:0:6255:0:(ldlm_lockd.c:1720:ldlm_request_cancel()) ### server-side cancel handler START: 2 locks, starting at 0 00010000:00010000:2.0:1776178145.702624:0:6255:0:(ldlm_lock.c:216:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: mdt-lustre-MDT0000_UUID lock: 000000009a8f90ae/0xa40083ab9bf53071 lrc: 0/0,0 mode: --/PR res: [0x2000013a1:0x8:0x0].0x0 bits 0x1b/0x0 rrc: 4 type: IBT gid 0 flags: 0x44a01000000000 nid: 192.168.204.3@tcp remote: 0x771f68aa1eb279dd expref: 21 pid: 6257 timeout: 0 lvb_type: 0 00010000:00010000:2.0:1776178145.702637:0:6255:0:(ldlm_lock.c:216:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: mdt-lustre-MDT0000_UUID lock: 00000000ae88aadd/0xa40083ab9bf53001 lrc: 0/0,0 mode: --/CR res: [0x2000013a1:0x8:0x0].0x0 bits 0x9/0x0 rrc: 3 type: IBT gid 0 flags: 0x44a01000000000 nid: 192.168.204.3@tcp remote: 0x771f68aa1eb2799e expref: 20 pid: 6256 timeout: 0 lvb_type: 0 00010000:00010000:2.0:1776178145.702644:0:6255:0:(ldlm_inodebits.c:94:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x2000013a1:0x8:0x0].0x0 (00000000e9f0d179) 00010000:00010000:2.0:1776178145.702650:0:6255:0:(ldlm_lockd.c:1782:ldlm_request_cancel()) ### server-side cancel handler END 00010000:00010000:2.0:1776178145.702661:0:6255:0:(ldlm_lock.c:766:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: mdt-lustre-MDT0000_UUID lock: 000000000396420a/0xa40083ab9bf530b7 lrc: 3/0,1 mode: --/PW res: [0x2000013a1:0x8:0x0].0x0 bits 0x0/0x0 rrc: 2 type: IBT gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 6255 timeout: 0 lvb_type: 0 00010000:00010000:2.0:1776178145.702669:0:6255:0:(ldlm_lock.c:1064:ldlm_granted_list_add_lock()) ### About to add lock: ns: mdt-lustre-MDT0000_UUID lock: 000000000396420a/0xa40083ab9bf530b7 lrc: 3/0,1 mode: PW/PW res: [0x2000013a1:0x8:0x0].0x0 bits 0x13/0x0 rrc: 2 type: IBT gid 0 flags: 0x50210000000000 nid: local remote: 0x0 expref: -99 pid: 6255 timeout: 0 lvb_type: 0 00010000:00010000:2.0:1776178145.702675:0:6255:0:(ldlm_request.c:528:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: mdt-lustre-MDT0000_UUID lock: 000000000396420a/0xa40083ab9bf530b7 lrc: 3/0,1 mode: PW/PW res: [0x2000013a1:0x8:0x0].0x0 bits 0x13/0x0 rrc: 2 type: IBT gid 0 flags: 0x40210000000000 nid: local remote: 0x0 expref: -99 pid: 6255 timeout: 0 lvb_type: 0 00000004:00010000:2.0:1776178145.702725:0:6255:0:(mdt_handler.c:3992:mdt_save_lock()) ### save lock request 00000000a33eecf4 reply state 0000000032ab8286 transno 12884901927 ns: mdt-lustre-MDT0000_UUID lock: 000000000396420a/0xa40083ab9bf530b7 lrc: 3/0,1 mode: PW/PW res: [0x2000013a1:0x8:0x0].0x0 bits 0x13/0x0 rrc: 2 type: IBT gid 0 flags: 0x40210000000000 nid: local remote: 0x0 expref: -99 pid: 6255 timeout: 0 lvb_type: 0 00000100:00100000:2.0:1776178145.702778:0:6255:0:(service.c:2372:ptlrpc_server_handle_request()) Handled RPC req@00000000a33eecf4 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_000:b50963bd-8eba-4432-9a54-8888d00e3fa1+21:17011:x1862456979081088:12345-192.168.204.3@tcp:36:chmod.0 Request processed in 186us (1329us total) trans 12884901927 rc 0/0 00000100:00100000:2.0:1776178145.702784:0:6255:0:(nrs_fifo.c:241:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.204.3@tcp, seq: 6493 00000100:00100000:0.0:1776178145.704005:0:3347:0:(events.c:361:request_in_callback()) peer: 12345-192.168.204.3@tcp (source: 12345-192.168.204.3@tcp) 00000100:00100000:0.0:1776178145.704103:0:3347:0:(events.c:361:request_in_callback()) peer: 12345-192.168.204.3@tcp (source: 12345-192.168.204.3@tcp) 00000100:00100000:3.0:1776178145.704288:0:11281:0:(service.c:2152:ptlrpc_server_handle_req_in()) got req x1862456979081344 00000100:00100000:3.0:1776178145.704338:0:11281:0:(service.c:2152:ptlrpc_server_handle_req_in()) got req x1862456979080768 00000100:00100000:3.0:1776178145.704358:0:11281:0:(nrs_fifo.c:179:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.204.3@tcp, seq: 6494 00000100:00100000:3.0:1776178145.704364:0:11281:0:(service.c:2324:ptlrpc_server_handle_request()) Handling RPC req@00000000d1e0f01c pname:cluuid+ref:pid:xid:nid:opc:job mdt00_004:b50963bd-8eba-4432-9a54-8888d00e3fa1+22:17008:x1862456979081344:12345-192.168.204.3@tcp:36:chmod.0 00000004:00080000:3.0:1776178145.704417:0:11281:0:(mdt_internal.h:1195:mdt_check_resent()) @@@ no reply data found for RESENT req, rc = 0 req@00000000d1e0f01c x1862456979081344/t0(0) o36->b50963bd-8eba-4432-9a54-8888d00e3fa1@192.168.204.3@tcp:677/0 lens 488/3152 e 0 to 0 dl 1776178192 ref 1 fl Interpret:/2/0 rc 0/0 job:'chmod.0' 00010000:00010000:3.0:1776178145.704430:0:11281:0:(ldlm_lockd.c:1720:ldlm_request_cancel()) ### server-side cancel handler START: 2 locks, starting at 0 00010000:00010000:3.0:1776178145.704452:0:11281:0:(ldlm_lock.c:216:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: mdt-lustre-MDT0000_UUID lock: 0000000015189b1f/0xa40083ab9bf5304e lrc: 0/0,0 mode: --/PR res: [0x2000013a1:0x5:0x0].0x0 bits 0x1b/0x0 rrc: 4 type: IBT gid 0 flags: 0x44a01000000000 nid: 192.168.204.3@tcp remote: 0x771f68aa1eb279c8 expref: 22 pid: 6256 timeout: 0 lvb_type: 0 00010000:00010000:3.0:1776178145.704485:0:11281:0:(ldlm_lock.c:216:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: mdt-lustre-MDT0000_UUID lock: 00000000336cd7d4/0xa40083ab9bf52f83 lrc: 0/0,0 mode: --/CR res: [0x2000013a1:0x5:0x0].0x0 bits 0x9/0x0 rrc: 3 type: IBT gid 0 flags: 0x44a01000000000 nid: 192.168.204.3@tcp remote: 0x771f68aa1eb27989 expref: 21 pid: 6255 timeout: 0 lvb_type: 0 00010000:00010000:3.0:1776178145.704494:0:11281:0:(ldlm_inodebits.c:94:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x2000013a1:0x5:0x0].0x0 (000000003b465a63) 00010000:00010000:3.0:1776178145.704503:0:11281:0:(ldlm_lockd.c:1782:ldlm_request_cancel()) ### server-side cancel handler END 00010000:00010000:3.0:1776178145.704536:0:11281:0:(ldlm_lock.c:766:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: mdt-lustre-MDT0000_UUID lock: 00000000575572c2/0xa40083ab9bf530be lrc: 3/0,1 mode: --/PW res: [0x2000013a1:0x5:0x0].0x0 bits 0x0/0x0 rrc: 2 type: IBT gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 11281 timeout: 0 lvb_type: 0 00010000:00010000:3.0:1776178145.704548:0:11281:0:(ldlm_lock.c:1064:ldlm_granted_list_add_lock()) ### About to add lock: ns: mdt-lustre-MDT0000_UUID lock: 00000000575572c2/0xa40083ab9bf530be lrc: 3/0,1 mode: PW/PW res: [0x2000013a1:0x5:0x0].0x0 bits 0x13/0x0 rrc: 2 type: IBT gid 0 flags: 0x50210000000000 nid: local remote: 0x0 expref: -99 pid: 11281 timeout: 0 lvb_type: 0 00010000:00010000:3.0:1776178145.704558:0:11281:0:(ldlm_request.c:528:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: mdt-lustre-MDT0000_UUID lock: 00000000575572c2/0xa40083ab9bf530be lrc: 3/0,1 mode: PW/PW res: [0x2000013a1:0x5:0x0].0x0 bits 0x13/0x0 rrc: 2 type: IBT gid 0 flags: 0x40210000000000 nid: local remote: 0x0 expref: -99 pid: 11281 timeout: 0 lvb_type: 0 00000004:00010000:3.0:1776178145.704753:0:11281:0:(mdt_handler.c:3992:mdt_save_lock()) ### save lock request 00000000d1e0f01c reply state 00000000acaaf2e3 transno 12884901928 ns: mdt-lustre-MDT0000_UUID lock: 00000000575572c2/0xa40083ab9bf530be lrc: 3/0,1 mode: PW/PW res: [0x2000013a1:0x5:0x0].0x0 bits 0x13/0x0 rrc: 2 type: IBT gid 0 flags: 0x40210000000000 nid: local remote: 0x0 expref: -99 pid: 11281 timeout: 0 lvb_type: 0 00000100:00100000:3.0:1776178145.704860:0:11281:0:(service.c:2372:ptlrpc_server_handle_request()) Handled RPC req@00000000d1e0f01c pname:cluuid+ref:pid:xid:nid:opc:job mdt00_004:b50963bd-8eba-4432-9a54-8888d00e3fa1+22:17008:x1862456979081344:12345-192.168.204.3@tcp:36:chmod.0 Request processed in 495us (3314us total) trans 12884901928 rc 0/0 00000100:00100000:3.0:1776178145.704867:0:11281:0:(nrs_fifo.c:241:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.204.3@tcp, seq: 6494 00000100:00100000:3.0:1776178145.704885:0:11281:0:(nrs_fifo.c:179:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.204.3@tcp, seq: 6495 00000100:00100000:3.0:1776178145.704888:0:11281:0:(service.c:2324:ptlrpc_server_handle_request()) Handling RPC req@00000000d2922afa pname:cluuid+ref:pid:xid:nid:opc:job mdt00_004:b50963bd-8eba-4432-9a54-8888d00e3fa1+21:17009:x1862456979080768:12345-192.168.204.3@tcp:36:chmod.0 00000004:00080000:3.0:1776178145.704908:0:11281:0:(mdt_internal.h:1195:mdt_check_resent()) @@@ no reply data found for RESENT req, rc = 0 req@00000000d2922afa x1862456979080768/t0(0) o36->b50963bd-8eba-4432-9a54-8888d00e3fa1@192.168.204.3@tcp:677/0 lens 488/3152 e 0 to 0 dl 1776178192 ref 1 fl Interpret:/2/0 rc 0/0 job:'chmod.0' 00010000:00010000:3.0:1776178145.704916:0:11281:0:(ldlm_lockd.c:1720:ldlm_request_cancel()) ### server-side cancel handler START: 2 locks, starting at 0 00010000:00010000:3.0:1776178145.704926:0:11281:0:(ldlm_lock.c:216:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: mdt-lustre-MDT0000_UUID lock: 000000006cc9c357/0xa40083ab9bf53040 lrc: 0/0,0 mode: --/PR res: [0x2000013a1:0x6:0x0].0x0 bits 0x1b/0x0 rrc: 4 type: IBT gid 0 flags: 0x44a01000000000 nid: 192.168.204.3@tcp remote: 0x771f68aa1eb279ba expref: 21 pid: 6255 timeout: 0 lvb_type: 0 00010000:00010000:3.0:1776178145.704939:0:11281:0:(ldlm_lock.c:216:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: mdt-lustre-MDT0000_UUID lock: 00000000d0f9e56c/0xa40083ab9bf52fad lrc: 0/0,0 mode: --/CR res: [0x2000013a1:0x6:0x0].0x0 bits 0x9/0x0 rrc: 3 type: IBT gid 0 flags: 0x44a01000000000 nid: 192.168.204.3@tcp remote: 0x771f68aa1eb27990 expref: 20 pid: 6255 timeout: 0 lvb_type: 0 00010000:00010000:3.0:1776178145.704949:0:11281:0:(ldlm_inodebits.c:94:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x2000013a1:0x6:0x0].0x0 (00000000ef7185e3) 00010000:00010000:3.0:1776178145.704955:0:11281:0:(ldlm_lockd.c:1782:ldlm_request_cancel()) ### server-side cancel handler END 00010000:00010000:3.0:1776178145.704967:0:11281:0:(ldlm_lock.c:766:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: mdt-lustre-MDT0000_UUID lock: 00000000b335466f/0xa40083ab9bf530c5 lrc: 3/0,1 mode: --/PW res: [0x2000013a1:0x6:0x0].0x0 bits 0x0/0x0 rrc: 2 type: IBT gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 11281 timeout: 0 lvb_type: 0 00010000:00010000:3.0:1776178145.704974:0:11281:0:(ldlm_lock.c:1064:ldlm_granted_list_add_lock()) ### About to add lock: ns: mdt-lustre-MDT0000_UUID lock: 00000000b335466f/0xa40083ab9bf530c5 lrc: 3/0,1 mode: PW/PW res: [0x2000013a1:0x6:0x0].0x0 bits 0x13/0x0 rrc: 2 type: IBT gid 0 flags: 0x50210000000000 nid: local remote: 0x0 expref: -99 pid: 11281 timeout: 0 lvb_type: 0 00010000:00010000:3.0:1776178145.704980:0:11281:0:(ldlm_request.c:528:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: mdt-lustre-MDT0000_UUID lock: 00000000b335466f/0xa40083ab9bf530c5 lrc: 3/0,1 mode: PW/PW res: [0x2000013a1:0x6:0x0].0x0 bits 0x13/0x0 rrc: 2 type: IBT gid 0 flags: 0x40210000000000 nid: local remote: 0x0 expref: -99 pid: 11281 timeout: 0 lvb_type: 0 00000004:00010000:3.0:1776178145.705032:0:11281:0:(mdt_handler.c:3992:mdt_save_lock()) ### save lock request 00000000d2922afa reply state 00000000da6ffb81 transno 12884901929 ns: mdt-lustre-MDT0000_UUID lock: 00000000b335466f/0xa40083ab9bf530c5 lrc: 3/0,1 mode: PW/PW res: [0x2000013a1:0x6:0x0].0x0 bits 0x13/0x0 rrc: 2 type: IBT gid 0 flags: 0x40210000000000 nid: local remote: 0x0 expref: -99 pid: 11281 timeout: 0 lvb_type: 0 00000100:00100000:3.0:1776178145.705090:0:11281:0:(service.c:2372:ptlrpc_server_handle_request()) Handled RPC req@00000000d2922afa pname:cluuid+ref:pid:xid:nid:opc:job mdt00_004:b50963bd-8eba-4432-9a54-8888d00e3fa1+21:17009:x1862456979080768:12345-192.168.204.3@tcp:36:chmod.0 Request processed in 200us (1094us total) trans 12884901929 rc 0/0 00000100:00100000:3.0:1776178145.705096:0:11281:0:(nrs_fifo.c:241:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.204.3@tcp, seq: 6495 00000100:00100000:2.0:1776178145.706389:0:8250:0:(service.c:2152:ptlrpc_server_handle_req_in()) got req x1862456979080192 00000100:00100000:0.0:1776178145.706502:0:3346:0:(events.c:361:request_in_callback()) peer: 12345-192.168.204.3@tcp (source: 12345-192.168.204.3@tcp) 00000100:00100000:1.0:1776178145.706780:0:6257:0:(service.c:2152:ptlrpc_server_handle_req_in()) got req x1862456979081152 00010000:00010000:1.0:1776178145.706805:0:3339:0:(ldlm_lock.c:1064:ldlm_granted_list_add_lock()) ### About to add lock: ns: mdt-lustre-MDT0000_UUID lock: 00000000e4217a5c/0xa40083ab9bf530b0 lrc: 3/0,1 mode: COS/COS res: [0x2000013a1:0x7:0x0].0x0 bits 0x13/0x0 rrc: 2 type: IBT gid 0 flags: 0x50210000000000 nid: local remote: 0x0 expref: -99 pid: 6255 timeout: 0 lvb_type: 0 00010000:00010000:1.0:1776178145.706833:0:3339:0:(ldlm_inodebits.c:94:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x2000013a1:0x7:0x0].0x0 (00000000aeb25ebf) 00010000:00010000:1.0:1776178145.706836:0:3339:0:(ldlm_inodebits.c:96:ldlm_reprocess_inodebits_queue()) Hint 13 00000100:00100000:1.0:1776178145.706876:0:11281:0:(nrs_fifo.c:179:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.204.3@tcp, seq: 6496 00000100:00100000:1.0:1776178145.706883:0:11281:0:(service.c:2324:ptlrpc_server_handle_request()) Handling RPC req@000000000154171f pname:cluuid+ref:pid:xid:nid:opc:job mdt00_004:b50963bd-8eba-4432-9a54-8888d00e3fa1+22:17006:x1862456979080192:12345-192.168.204.3@tcp:36:chmod.0 00000004:00080000:1.0:1776178145.706944:0:11281:0:(mdt_internal.h:1195:mdt_check_resent()) @@@ no reply data found for RESENT req, rc = 0 req@000000000154171f x1862456979080192/t0(0) o36->b50963bd-8eba-4432-9a54-8888d00e3fa1@192.168.204.3@tcp:677/0 lens 488/3152 e 0 to 0 dl 1776178192 ref 1 fl Interpret:/2/0 rc 0/0 job:'chmod.0' 00010000:00010000:1.0:1776178145.706957:0:11281:0:(ldlm_lockd.c:1720:ldlm_request_cancel()) ### server-side cancel handler START: 2 locks, starting at 0 00010000:00010000:1.0:1776178145.706980:0:11281:0:(ldlm_lock.c:216:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: mdt-lustre-MDT0000_UUID lock: 000000006a188f9a/0xa40083ab9bf5301d lrc: 0/0,0 mode: --/PR res: [0x2000013a1:0x3:0x0].0x0 bits 0x1b/0x0 rrc: 4 type: IBT gid 0 flags: 0x44a01000000000 nid: 192.168.204.3@tcp remote: 0x771f68aa1eb279a5 expref: 22 pid: 6255 timeout: 0 lvb_type: 0 00010000:00010000:1.0:1776178145.706999:0:11281:0:(ldlm_lock.c:216:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: mdt-lustre-MDT0000_UUID lock: 00000000157c1e3e/0xa40083ab9bf52f2f lrc: 0/0,0 mode: --/CR res: [0x2000013a1:0x3:0x0].0x0 bits 0x9/0x0 rrc: 3 type: IBT gid 0 flags: 0x44a01000000000 nid: 192.168.204.3@tcp remote: 0x771f68aa1eb2797b expref: 21 pid: 6256 timeout: 0 lvb_type: 0 00010000:00010000:1.0:1776178145.707007:0:11281:0:(ldlm_inodebits.c:94:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x2000013a1:0x3:0x0].0x0 (00000000d0a7c19f) 00010000:00010000:1.0:1776178145.707016:0:11281:0:(ldlm_lockd.c:1782:ldlm_request_cancel()) ### server-side cancel handler END 00010000:00010000:1.0:1776178145.707044:0:11281:0:(ldlm_lock.c:766:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: mdt-lustre-MDT0000_UUID lock: 00000000a44d8583/0xa40083ab9bf530cc lrc: 3/0,1 mode: --/PW res: [0x2000013a1:0x3:0x0].0x0 bits 0x0/0x0 rrc: 2 type: IBT gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 11281 timeout: 0 lvb_type: 0 00010000:00010000:1.0:1776178145.707054:0:11281:0:(ldlm_lock.c:1064:ldlm_granted_list_add_lock()) ### About to add lock: ns: mdt-lustre-MDT0000_UUID lock: 00000000a44d8583/0xa40083ab9bf530cc lrc: 3/0,1 mode: PW/PW res: [0x2000013a1:0x3:0x0].0x0 bits 0x13/0x0 rrc: 2 type: IBT gid 0 flags: 0x50210000000000 nid: local remote: 0x0 expref: -99 pid: 11281 timeout: 0 lvb_type: 0 00010000:00010000:1.0:1776178145.707079:0:11281:0:(ldlm_request.c:528:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: mdt-lustre-MDT0000_UUID lock: 00000000a44d8583/0xa40083ab9bf530cc lrc: 3/0,1 mode: PW/PW res: [0x2000013a1:0x3:0x0].0x0 bits 0x13/0x0 rrc: 2 type: IBT gid 0 flags: 0x40210000000000 nid: local remote: 0x0 expref: -99 pid: 11281 timeout: 0 lvb_type: 0 00000004:00010000:1.0:1776178145.707378:0:11281:0:(mdt_handler.c:3992:mdt_save_lock()) ### save lock request 000000000154171f reply state 000000002800e6ec transno 12884901930 ns: mdt-lustre-MDT0000_UUID lock: 00000000a44d8583/0xa40083ab9bf530cc lrc: 3/0,1 mode: PW/PW res: [0x2000013a1:0x3:0x0].0x0 bits 0x13/0x0 rrc: 2 type: IBT gid 0 flags: 0x40210000000000 nid: local remote: 0x0 expref: -99 pid: 11281 timeout: 0 lvb_type: 0 00000100:00100000:1.0:1776178145.707478:0:11281:0:(service.c:2372:ptlrpc_server_handle_request()) Handled RPC req@000000000154171f pname:cluuid+ref:pid:xid:nid:opc:job mdt00_004:b50963bd-8eba-4432-9a54-8888d00e3fa1+22:17006:x1862456979080192:12345-192.168.204.3@tcp:36:chmod.0 Request processed in 594us (3374us total) trans 12884901930 rc 0/0 00000100:00100000:1.0:1776178145.707485:0:11281:0:(nrs_fifo.c:241:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.204.3@tcp, seq: 6496 00000100:00100000:1.0:1776178145.707505:0:11281:0:(nrs_fifo.c:179:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.204.3@tcp, seq: 6497 00000100:00100000:1.0:1776178145.707509:0:11281:0:(service.c:2324:ptlrpc_server_handle_request()) Handling RPC req@0000000027e376cf pname:cluuid+ref:pid:xid:nid:opc:job mdt00_004:b50963bd-8eba-4432-9a54-8888d00e3fa1+21:17007:x1862456979081152:12345-192.168.204.3@tcp:36:chmod.0 00000004:00080000:1.0:1776178145.707529:0:11281:0:(mdt_internal.h:1195:mdt_check_resent()) @@@ no reply data found for RESENT req, rc = 0 req@0000000027e376cf x1862456979081152/t0(0) o36->b50963bd-8eba-4432-9a54-8888d00e3fa1@192.168.204.3@tcp:677/0 lens 488/3152 e 0 to 0 dl 1776178192 ref 1 fl Interpret:/2/0 rc 0/0 job:'chmod.0' 00010000:00010000:1.0:1776178145.707536:0:11281:0:(ldlm_lockd.c:1720:ldlm_request_cancel()) ### server-side cancel handler START: 2 locks, starting at 0 00010000:00010000:1.0:1776178145.707545:0:11281:0:(ldlm_lock.c:216:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: mdt-lustre-MDT0000_UUID lock: 00000000a27c19b3/0xa40083ab9bf5307f lrc: 0/0,0 mode: --/PR res: [0x2000013a1:0x4:0x0].0x0 bits 0x1b/0x0 rrc: 4 type: IBT gid 0 flags: 0x44a01000000000 nid: 192.168.204.3@tcp remote: 0x771f68aa1eb279e4 expref: 21 pid: 6257 timeout: 0 lvb_type: 0 00010000:00010000:1.0:1776178145.707558:0:11281:0:(ldlm_lock.c:216:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: mdt-lustre-MDT0000_UUID lock: 00000000b4e1b221/0xa40083ab9bf52f59 lrc: 0/0,0 mode: --/CR res: [0x2000013a1:0x4:0x0].0x0 bits 0x9/0x0 rrc: 3 type: IBT gid 0 flags: 0x44a01000000000 nid: 192.168.204.3@tcp remote: 0x771f68aa1eb27982 expref: 20 pid: 6255 timeout: 0 lvb_type: 0 00010000:00010000:1.0:1776178145.707565:0:11281:0:(ldlm_inodebits.c:94:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x2000013a1:0x4:0x0].0x0 (00000000ef6cecb2) 00010000:00010000:1.0:1776178145.707571:0:11281:0:(ldlm_lockd.c:1782:ldlm_request_cancel()) ### server-side cancel handler END 00010000:00010000:1.0:1776178145.707583:0:11281:0:(ldlm_lock.c:766:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: mdt-lustre-MDT0000_UUID lock: 0000000097975a3b/0xa40083ab9bf530d3 lrc: 3/0,1 mode: --/PW res: [0x2000013a1:0x4:0x0].0x0 bits 0x0/0x0 rrc: 2 type: IBT gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 11281 timeout: 0 lvb_type: 0 00010000:00010000:1.0:1776178145.707590:0:11281:0:(ldlm_lock.c:1064:ldlm_granted_list_add_lock()) ### About to add lock: ns: mdt-lustre-MDT0000_UUID lock: 0000000097975a3b/0xa40083ab9bf530d3 lrc: 3/0,1 mode: PW/PW res: [0x2000013a1:0x4:0x0].0x0 bits 0x13/0x0 rrc: 2 type: IBT gid 0 flags: 0x50210000000000 nid: local remote: 0x0 expref: -99 pid: 11281 timeout: 0 lvb_type: 0 00010000:00010000:1.0:1776178145.707596:0:11281:0:(ldlm_request.c:528:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: mdt-lustre-MDT0000_UUID lock: 0000000097975a3b/0xa40083ab9bf530d3 lrc: 3/0,1 mode: PW/PW res: [0x2000013a1:0x4:0x0].0x0 bits 0x13/0x0 rrc: 2 type: IBT gid 0 flags: 0x40210000000000 nid: local remote: 0x0 expref: -99 pid: 11281 timeout: 0 lvb_type: 0 00000004:00010000:1.0:1776178145.707646:0:11281:0:(mdt_handler.c:3992:mdt_save_lock()) ### save lock request 0000000027e376cf reply state 00000000393ecafc transno 12884901931 ns: mdt-lustre-MDT0000_UUID lock: 0000000097975a3b/0xa40083ab9bf530d3 lrc: 3/0,1 mode: PW/PW res: [0x2000013a1:0x4:0x0].0x0 bits 0x13/0x0 rrc: 2 type: IBT gid 0 flags: 0x40210000000000 nid: local remote: 0x0 expref: -99 pid: 11281 timeout: 0 lvb_type: 0 00000100:00100000:1.0:1776178145.707687:0:11281:0:(service.c:2372:ptlrpc_server_handle_request()) Handled RPC req@0000000027e376cf pname:cluuid+ref:pid:xid:nid:opc:job mdt00_004:b50963bd-8eba-4432-9a54-8888d00e3fa1+21:17007:x1862456979081152:12345-192.168.204.3@tcp:36:chmod.0 Request processed in 177us (1190us total) trans 12884901931 rc 0/0 00000100:00100000:1.0:1776178145.707693:0:11281:0:(nrs_fifo.c:241:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.204.3@tcp, seq: 6497 00010000:00010000:2.0:1776178145.708384:0:3340:0:(ldlm_lock.c:1064:ldlm_granted_list_add_lock()) ### About to add lock: ns: mdt-lustre-MDT0000_UUID lock: 000000000396420a/0xa40083ab9bf530b7 lrc: 3/0,1 mode: COS/COS res: [0x2000013a1:0x8:0x0].0x0 bits 0x13/0x0 rrc: 2 type: IBT gid 0 flags: 0x50210000000000 nid: local remote: 0x0 expref: -99 pid: 6255 timeout: 0 lvb_type: 0 00010000:00010000:2.0:1776178145.708394:0:3340:0:(ldlm_inodebits.c:94:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x2000013a1:0x8:0x0].0x0 (000000009a65918f) 00010000:00010000:2.0:1776178145.708396:0:3340:0:(ldlm_inodebits.c:96:ldlm_reprocess_inodebits_queue()) Hint 13 00010000:00010000:2.0:1776178145.708402:0:3340:0:(ldlm_lock.c:1064:ldlm_granted_list_add_lock()) ### About to add lock: ns: mdt-lustre-MDT0000_UUID lock: 0000000073f2c7a9/0xa40083ab9bf530a9 lrc: 3/0,1 mode: COS/COS res: [0x2000013a1:0x2:0x0].0x0 bits 0x13/0x0 rrc: 2 type: IBT gid 0 flags: 0x50210000000000 nid: local remote: 0x0 expref: -99 pid: 6255 timeout: 0 lvb_type: 0 00010000:00010000:2.0:1776178145.708407:0:3340:0:(ldlm_inodebits.c:94:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x2000013a1:0x2:0x0].0x0 (00000000e4751492) 00010000:00010000:2.0:1776178145.708409:0:3340:0:(ldlm_inodebits.c:96:ldlm_reprocess_inodebits_queue()) Hint 13 00000100:00100000:0.0:1776178145.711243:0:3347:0:(events.c:361:request_in_callback()) peer: 12345-192.168.204.3@tcp (source: 12345-192.168.204.3@tcp) 00000100:00100000:0.0:1776178145.711333:0:3347:0:(events.c:361:request_in_callback()) peer: 12345-192.168.204.3@tcp (source: 12345-192.168.204.3@tcp) 00010000:00010000:2.0:1776178145.711552:0:3340:0:(ldlm_lock.c:1064:ldlm_granted_list_add_lock()) ### About to add lock: ns: mdt-lustre-MDT0000_UUID lock: 0000000097975a3b/0xa40083ab9bf530d3 lrc: 3/0,1 mode: COS/COS res: [0x2000013a1:0x4:0x0].0x0 bits 0x13/0x0 rrc: 2 type: IBT gid 0 flags: 0x50210000000000 nid: local remote: 0x0 expref: -99 pid: 11281 timeout: 0 lvb_type: 0 00010000:00010000:2.0:1776178145.711572:0:3340:0:(ldlm_inodebits.c:94:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x2000013a1:0x4:0x0].0x0 (000000009cfa2d69) 00010000:00010000:2.0:1776178145.711575:0:3340:0:(ldlm_inodebits.c:96:ldlm_reprocess_inodebits_queue()) Hint 13 00010000:00010000:2.0:1776178145.711588:0:3340:0:(ldlm_lock.c:1064:ldlm_granted_list_add_lock()) ### About to add lock: ns: mdt-lustre-MDT0000_UUID lock: 00000000b335466f/0xa40083ab9bf530c5 lrc: 3/0,1 mode: COS/COS res: [0x2000013a1:0x6:0x0].0x0 bits 0x13/0x0 rrc: 2 type: IBT gid 0 flags: 0x50210000000000 nid: local remote: 0x0 expref: -99 pid: 11281 timeout: 0 lvb_type: 0 00010000:00010000:2.0:1776178145.711595:0:3340:0:(ldlm_inodebits.c:94:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x2000013a1:0x6:0x0].0x0 (0000000019a0327e) 00010000:00010000:2.0:1776178145.711597:0:3340:0:(ldlm_inodebits.c:96:ldlm_reprocess_inodebits_queue()) Hint 13 00010000:00010000:1.0:1776178145.711657:0:3339:0:(ldlm_lock.c:1064:ldlm_granted_list_add_lock()) ### About to add lock: ns: mdt-lustre-MDT0000_UUID lock: 00000000a44d8583/0xa40083ab9bf530cc lrc: 3/0,1 mode: COS/COS res: [0x2000013a1:0x3:0x0].0x0 bits 0x13/0x0 rrc: 2 type: IBT gid 0 flags: 0x50210000000000 nid: local remote: 0x0 expref: -99 pid: 11281 timeout: 0 lvb_type: 0 00010000:00010000:1.0:1776178145.711667:0:3339:0:(ldlm_inodebits.c:94:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x2000013a1:0x3:0x0].0x0 (0000000090c54b62) 00010000:00010000:1.0:1776178145.711669:0:3339:0:(ldlm_inodebits.c:96:ldlm_reprocess_inodebits_queue()) Hint 13 00010000:00010000:1.0:1776178145.711676:0:3339:0:(ldlm_lock.c:1064:ldlm_granted_list_add_lock()) ### About to add lock: ns: mdt-lustre-MDT0000_UUID lock: 00000000575572c2/0xa40083ab9bf530be lrc: 3/0,1 mode: COS/COS res: [0x2000013a1:0x5:0x0].0x0 bits 0x13/0x0 rrc: 2 type: IBT gid 0 flags: 0x50210000000000 nid: local remote: 0x0 expref: -99 pid: 11281 timeout: 0 lvb_type: 0 00010000:00010000:1.0:1776178145.711682:0:3339:0:(ldlm_inodebits.c:94:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x2000013a1:0x5:0x0].0x0 (000000006b0fcbcb) 00010000:00010000:1.0:1776178145.711684:0:3339:0:(ldlm_inodebits.c:96:ldlm_reprocess_inodebits_queue()) Hint 13 00000100:00100000:3.0:1776178145.712355:0:6256:0:(service.c:2152:ptlrpc_server_handle_req_in()) got req x1862456979085568 00000100:00100000:3.0:1776178145.712405:0:6256:0:(service.c:2152:ptlrpc_server_handle_req_in()) got req x1862456979085504 00000100:00100000:3.0:1776178145.712433:0:6256:0:(nrs_fifo.c:179:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.204.3@tcp, seq: 6498 00000100:00100000:3.0:1776178145.712440:0:6256:0:(service.c:2324:ptlrpc_server_handle_request()) Handling RPC req@00000000efb13deb pname:cluuid+ref:pid:xid:nid:opc:job mdt00_001:b50963bd-8eba-4432-9a54-8888d00e3fa1+22:17011:x1862456979085568:12345-192.168.204.3@tcp:101:chmod.0 00010000:00010000:3.0:1776178145.712465:0:6256:0:(ldlm_lockd.c:1248:ldlm_handle_enqueue0()) ### server-side enqueue handler START 00010000:00010000:3.0:1776178145.712485:0:6256:0:(ldlm_lockd.c:1318:ldlm_handle_enqueue0()) ### server-side enqueue handler, new lock created ns: mdt-lustre-MDT0000_UUID lock: 00000000c3ab77cd/0xa40083ab9bf530da lrc: 2/0,0 mode: --/CR res: [0x2000013a1:0x8:0x0].0x0 bits 0x0/0x0 rrc: 3 type: IBT gid 0 flags: 0x40000000000000 nid: local remote: 0x771f68aa1eb27a0e expref: -99 pid: 6256 timeout: 0 lvb_type: 0 00010000:00010000:3.0:1776178145.712546:0:6256:0:(ldlm_lock.c:1064:ldlm_granted_list_add_lock()) ### About to add lock: ns: mdt-lustre-MDT0000_UUID lock: 00000000c3ab77cd/0xa40083ab9bf530da lrc: 3/0,0 mode: CR/CR res: [0x2000013a1:0x8:0x0].0x0 bits 0x8/0x0 rrc: 3 type: IBT gid 0 flags: 0x50000000000000 nid: 192.168.204.3@tcp remote: 0x771f68aa1eb27a0e expref: 23 pid: 6256 timeout: 0 lvb_type: 0 00010000:00010000:3.0:1776178145.712557:0:6256:0:(ldlm_lockd.c:1484:ldlm_handle_enqueue0()) ### server-side enqueue handler, sending reply (err=0, rc=0) ns: mdt-lustre-MDT0000_UUID lock: 00000000c3ab77cd/0xa40083ab9bf530da lrc: 3/0,0 mode: CR/CR res: [0x2000013a1:0x8:0x0].0x0 bits 0x8/0x0 rrc: 3 type: IBT gid 0 flags: 0x40000000000000 nid: 192.168.204.3@tcp remote: 0x771f68aa1eb27a0e expref: 23 pid: 6256 timeout: 0 lvb_type: 0 00010000:00010000:3.0:1776178145.712580:0:6256:0:(ldlm_inodebits.c:94:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x2000013a1:0x8:0x0].0x0 (000000009a65918f) 00010000:00010000:3.0:1776178145.712582:0:6256:0:(ldlm_inodebits.c:96:ldlm_reprocess_inodebits_queue()) Hint 8 00010000:00010000:3.0:1776178145.712585:0:6256:0:(ldlm_lockd.c:1565:ldlm_handle_enqueue0()) ### server-side enqueue handler END (lock 00000000c3ab77cd, rc 0) 00000100:00100000:3.0:1776178145.712645:0:6256:0:(service.c:2372:ptlrpc_server_handle_request()) Handled RPC req@00000000efb13deb pname:cluuid+ref:pid:xid:nid:opc:job mdt00_001:b50963bd-8eba-4432-9a54-8888d00e3fa1+23:17011:x1862456979085568:12345-192.168.204.3@tcp:101:chmod.0 Request processed in 203us (1411us total) trans 0 rc 0/0 00000100:00100000:3.0:1776178145.712652:0:6256:0:(nrs_fifo.c:241:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.204.3@tcp, seq: 6498 00000100:00100000:3.0:1776178145.712674:0:6256:0:(nrs_fifo.c:179:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.204.3@tcp, seq: 6499 00000100:00100000:3.0:1776178145.712678:0:6256:0:(service.c:2324:ptlrpc_server_handle_request()) Handling RPC req@0000000023c7374e pname:cluuid+ref:pid:xid:nid:opc:job mdt00_001:b50963bd-8eba-4432-9a54-8888d00e3fa1+22:17010:x1862456979085504:12345-192.168.204.3@tcp:101:chmod.0 00010000:00010000:3.0:1776178145.712685:0:6256:0:(ldlm_lockd.c:1248:ldlm_handle_enqueue0()) ### server-side enqueue handler START 00010000:00010000:3.0:1776178145.712694:0:6256:0:(ldlm_lockd.c:1318:ldlm_handle_enqueue0()) ### server-side enqueue handler, new lock created ns: mdt-lustre-MDT0000_UUID lock: 0000000055d6a00d/0xa40083ab9bf530e1 lrc: 2/0,0 mode: --/CR res: [0x2000013a1:0x7:0x0].0x0 bits 0x0/0x0 rrc: 3 type: IBT gid 0 flags: 0x40000000000000 nid: local remote: 0x771f68aa1eb27a07 expref: -99 pid: 6256 timeout: 0 lvb_type: 0 00010000:00010000:3.0:1776178145.712718:0:6256:0:(ldlm_lock.c:1064:ldlm_granted_list_add_lock()) ### About to add lock: ns: mdt-lustre-MDT0000_UUID lock: 0000000055d6a00d/0xa40083ab9bf530e1 lrc: 3/0,0 mode: CR/CR res: [0x2000013a1:0x7:0x0].0x0 bits 0x8/0x0 rrc: 3 type: IBT gid 0 flags: 0x50000000000000 nid: 192.168.204.3@tcp remote: 0x771f68aa1eb27a07 expref: 23 pid: 6256 timeout: 0 lvb_type: 0 00010000:00010000:3.0:1776178145.712726:0:6256:0:(ldlm_lockd.c:1484:ldlm_handle_enqueue0()) ### server-side enqueue handler, sending reply (err=0, rc=0) ns: mdt-lustre-MDT0000_UUID lock: 0000000055d6a00d/0xa40083ab9bf530e1 lrc: 3/0,0 mode: CR/CR res: [0x2000013a1:0x7:0x0].0x0 bits 0x8/0x0 rrc: 3 type: IBT gid 0 flags: 0x40000000000000 nid: 192.168.204.3@tcp remote: 0x771f68aa1eb27a07 expref: 23 pid: 6256 timeout: 0 lvb_type: 0 00010000:00010000:3.0:1776178145.712739:0:6256:0:(ldlm_inodebits.c:94:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x2000013a1:0x7:0x0].0x0 (00000000aeb25ebf) 00010000:00010000:3.0:1776178145.712741:0:6256:0:(ldlm_inodebits.c:96:ldlm_reprocess_inodebits_queue()) Hint 8 00010000:00010000:3.0:1776178145.712742:0:6256:0:(ldlm_lockd.c:1565:ldlm_handle_enqueue0()) ### server-side enqueue handler END (lock 0000000055d6a00d, rc 0) 00000100:00100000:3.0:1776178145.712760:0:6256:0:(service.c:2372:ptlrpc_server_handle_request()) Handled RPC req@0000000023c7374e pname:cluuid+ref:pid:xid:nid:opc:job mdt00_001:b50963bd-8eba-4432-9a54-8888d00e3fa1+23:17010:x1862456979085504:12345-192.168.204.3@tcp:101:chmod.0 Request processed in 81us (1428us total) trans 0 rc 0/0 00000100:00100000:3.0:1776178145.712766:0:6256:0:(nrs_fifo.c:241:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.204.3@tcp, seq: 6499 00000100:00100000:1.0:1776178145.716291:0:3348:0:(events.c:361:request_in_callback()) peer: 12345-192.168.204.3@tcp (source: 12345-192.168.204.3@tcp) 00000100:00100000:1.0:1776178145.716388:0:3348:0:(events.c:361:request_in_callback()) peer: 12345-192.168.204.3@tcp (source: 12345-192.168.204.3@tcp) 00000100:00100000:1.0:1776178145.716696:0:3348:0:(events.c:361:request_in_callback()) peer: 12345-192.168.204.3@tcp (source: 12345-192.168.204.3@tcp) 00000100:00100000:2.0:1776178145.716727:0:6255:0:(service.c:2152:ptlrpc_server_handle_req_in()) got req x1862456979085696 00000100:00100000:0.0:1776178145.717226:0:8250:0:(service.c:2152:ptlrpc_server_handle_req_in()) got req x1862456979085760 00000100:00100000:0.0:1776178145.717273:0:8250:0:(nrs_fifo.c:179:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.204.3@tcp, seq: 6500 00000100:00100000:0.0:1776178145.717284:0:8250:0:(service.c:2324:ptlrpc_server_handle_request()) Handling RPC req@0000000006b93299 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_003:b50963bd-8eba-4432-9a54-8888d00e3fa1+24:17009:x1862456979085696:12345-192.168.204.3@tcp:101:chmod.0 00010000:00010000:0.0:1776178145.717322:0:8250:0:(ldlm_lockd.c:1248:ldlm_handle_enqueue0()) ### server-side enqueue handler START 00010000:00010000:0.0:1776178145.717354:0:8250:0:(ldlm_lockd.c:1318:ldlm_handle_enqueue0()) ### server-side enqueue handler, new lock created ns: mdt-lustre-MDT0000_UUID lock: 00000000f190b39a/0xa40083ab9bf530e8 lrc: 2/0,0 mode: --/CR res: [0x2000013a1:0x6:0x0].0x0 bits 0x0/0x0 rrc: 3 type: IBT gid 0 flags: 0x40000000000000 nid: local remote: 0x771f68aa1eb27a1c expref: -99 pid: 8250 timeout: 0 lvb_type: 0 00010000:00010000:0.0:1776178145.717446:0:8250:0:(ldlm_lock.c:1064:ldlm_granted_list_add_lock()) ### About to add lock: ns: mdt-lustre-MDT0000_UUID lock: 00000000f190b39a/0xa40083ab9bf530e8 lrc: 3/0,0 mode: CR/CR res: [0x2000013a1:0x6:0x0].0x0 bits 0x8/0x0 rrc: 3 type: IBT gid 0 flags: 0x50000000000000 nid: 192.168.204.3@tcp remote: 0x771f68aa1eb27a1c expref: 25 pid: 8250 timeout: 0 lvb_type: 0 00010000:00010000:0.0:1776178145.717463:0:8250:0:(ldlm_lockd.c:1484:ldlm_handle_enqueue0()) ### server-side enqueue handler, sending reply (err=0, rc=0) ns: mdt-lustre-MDT0000_UUID lock: 00000000f190b39a/0xa40083ab9bf530e8 lrc: 3/0,0 mode: CR/CR res: [0x2000013a1:0x6:0x0].0x0 bits 0x8/0x0 rrc: 3 type: IBT gid 0 flags: 0x40000000000000 nid: 192.168.204.3@tcp remote: 0x771f68aa1eb27a1c expref: 25 pid: 8250 timeout: 0 lvb_type: 0 00010000:00010000:0.0:1776178145.717488:0:8250:0:(ldlm_inodebits.c:94:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x2000013a1:0x6:0x0].0x0 (0000000019a0327e) 00010000:00010000:0.0:1776178145.717491:0:8250:0:(ldlm_inodebits.c:96:ldlm_reprocess_inodebits_queue()) Hint 8 00010000:00010000:0.0:1776178145.717495:0:8250:0:(ldlm_lockd.c:1565:ldlm_handle_enqueue0()) ### server-side enqueue handler END (lock 00000000f190b39a, rc 0) 00000100:00100000:0.0:1776178145.717576:0:8250:0:(service.c:2372:ptlrpc_server_handle_request()) Handled RPC req@0000000006b93299 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_003:b50963bd-8eba-4432-9a54-8888d00e3fa1+25:17009:x1862456979085696:12345-192.168.204.3@tcp:101:chmod.0 Request processed in 293us (1297us total) trans 0 rc 0/0 00000100:00100000:0.0:1776178145.717584:0:8250:0:(nrs_fifo.c:241:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.204.3@tcp, seq: 6500 00000100:00100000:0.0:1776178145.717607:0:8250:0:(nrs_fifo.c:179:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.204.3@tcp, seq: 6501 00000100:00100000:0.0:1776178145.717611:0:8250:0:(service.c:2324:ptlrpc_server_handle_request()) Handling RPC req@0000000071fcbd32 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_003:b50963bd-8eba-4432-9a54-8888d00e3fa1+24:17007:x1862456979085760:12345-192.168.204.3@tcp:101:chmod.0 00010000:00010000:0.0:1776178145.717618:0:8250:0:(ldlm_lockd.c:1248:ldlm_handle_enqueue0()) ### server-side enqueue handler START 00010000:00010000:0.0:1776178145.717626:0:8250:0:(ldlm_lockd.c:1318:ldlm_handle_enqueue0()) ### server-side enqueue handler, new lock created ns: mdt-lustre-MDT0000_UUID lock: 0000000015ae4ee6/0xa40083ab9bf530ef lrc: 2/0,0 mode: --/CR res: [0x2000013a1:0x4:0x0].0x0 bits 0x0/0x0 rrc: 3 type: IBT gid 0 flags: 0x40000000000000 nid: local remote: 0x771f68aa1eb27a23 expref: -99 pid: 8250 timeout: 0 lvb_type: 0 00010000:00010000:0.0:1776178145.717762:0:8250:0:(ldlm_lock.c:1064:ldlm_granted_list_add_lock()) ### About to add lock: ns: mdt-lustre-MDT0000_UUID lock: 0000000015ae4ee6/0xa40083ab9bf530ef lrc: 3/0,0 mode: CR/CR res: [0x2000013a1:0x4:0x0].0x0 bits 0x8/0x0 rrc: 3 type: IBT gid 0 flags: 0x50000000000000 nid: 192.168.204.3@tcp remote: 0x771f68aa1eb27a23 expref: 25 pid: 8250 timeout: 0 lvb_type: 0 00010000:00010000:0.0:1776178145.717773:0:8250:0:(ldlm_lockd.c:1484:ldlm_handle_enqueue0()) ### server-side enqueue handler, sending reply (err=0, rc=0) ns: mdt-lustre-MDT0000_UUID lock: 0000000015ae4ee6/0xa40083ab9bf530ef lrc: 3/0,0 mode: CR/CR res: [0x2000013a1:0x4:0x0].0x0 bits 0x8/0x0 rrc: 3 type: IBT gid 0 flags: 0x40000000000000 nid: 192.168.204.3@tcp remote: 0x771f68aa1eb27a23 expref: 25 pid: 8250 timeout: 0 lvb_type: 0 00010000:00010000:0.0:1776178145.717785:0:8250:0:(ldlm_inodebits.c:94:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x2000013a1:0x4:0x0].0x0 (000000009cfa2d69) 00010000:00010000:0.0:1776178145.717788:0:8250:0:(ldlm_inodebits.c:96:ldlm_reprocess_inodebits_queue()) Hint 8 00010000:00010000:0.0:1776178145.717790:0:8250:0:(ldlm_lockd.c:1565:ldlm_handle_enqueue0()) ### server-side enqueue handler END (lock 0000000015ae4ee6, rc 0) 00000100:00100000:0.0:1776178145.718375:0:8250:0:(service.c:2372:ptlrpc_server_handle_request()) Handled RPC req@0000000071fcbd32 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_003:b50963bd-8eba-4432-9a54-8888d00e3fa1+25:17007:x1862456979085760:12345-192.168.204.3@tcp:101:chmod.0 Request processed in 200us (1425us total) trans 0 rc 0/0 00000100:00100000:0.0:1776178145.718384:0:8250:0:(nrs_fifo.c:241:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.204.3@tcp, seq: 6501 00000100:00100000:1.0:1776178145.718885:0:3348:0:(events.c:361:request_in_callback()) peer: 12345-192.168.204.3@tcp (source: 12345-192.168.204.3@tcp) 00000100:00100000:2.0:1776178145.719076:0:6255:0:(service.c:2152:ptlrpc_server_handle_req_in()) got req x1862456979085824 00000100:00100000:2.0:1776178145.719113:0:6255:0:(nrs_fifo.c:179:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.204.3@tcp, seq: 6502 00000100:00100000:2.0:1776178145.719117:0:6255:0:(service.c:2324:ptlrpc_server_handle_request()) Handling RPC req@000000006d569c58 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_000:b50963bd-8eba-4432-9a54-8888d00e3fa1+25:17008:x1862456979085824:12345-192.168.204.3@tcp:101:chmod.0 00010000:00010000:2.0:1776178145.719144:0:6255:0:(ldlm_lockd.c:1248:ldlm_handle_enqueue0()) ### server-side enqueue handler START 00010000:00010000:2.0:1776178145.719161:0:6255:0:(ldlm_lockd.c:1318:ldlm_handle_enqueue0()) ### server-side enqueue handler, new lock created ns: mdt-lustre-MDT0000_UUID lock: 000000009a8f90ae/0xa40083ab9bf530f6 lrc: 2/0,0 mode: --/CR res: [0x2000013a1:0x5:0x0].0x0 bits 0x0/0x0 rrc: 3 type: IBT gid 0 flags: 0x40000000000000 nid: local remote: 0x771f68aa1eb27a2a expref: -99 pid: 6255 timeout: 0 lvb_type: 0 00010000:00010000:2.0:1776178145.719211:0:6255:0:(ldlm_lock.c:1064:ldlm_granted_list_add_lock()) ### About to add lock: ns: mdt-lustre-MDT0000_UUID lock: 000000009a8f90ae/0xa40083ab9bf530f6 lrc: 3/0,0 mode: CR/CR res: [0x2000013a1:0x5:0x0].0x0 bits 0x8/0x0 rrc: 3 type: IBT gid 0 flags: 0x50000000000000 nid: 192.168.204.3@tcp remote: 0x771f68aa1eb27a2a expref: 26 pid: 6255 timeout: 0 lvb_type: 0 00010000:00010000:2.0:1776178145.719223:0:6255:0:(ldlm_lockd.c:1484:ldlm_handle_enqueue0()) ### server-side enqueue handler, sending reply (err=0, rc=0) ns: mdt-lustre-MDT0000_UUID lock: 000000009a8f90ae/0xa40083ab9bf530f6 lrc: 3/0,0 mode: CR/CR res: [0x2000013a1:0x5:0x0].0x0 bits 0x8/0x0 rrc: 3 type: IBT gid 0 flags: 0x40000000000000 nid: 192.168.204.3@tcp remote: 0x771f68aa1eb27a2a expref: 26 pid: 6255 timeout: 0 lvb_type: 0 00010000:00010000:2.0:1776178145.719238:0:6255:0:(ldlm_inodebits.c:94:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x2000013a1:0x5:0x0].0x0 (000000006b0fcbcb) 00010000:00010000:2.0:1776178145.719241:0:6255:0:(ldlm_inodebits.c:96:ldlm_reprocess_inodebits_queue()) Hint 8 00010000:00010000:2.0:1776178145.719245:0:6255:0:(ldlm_lockd.c:1565:ldlm_handle_enqueue0()) ### server-side enqueue handler END (lock 000000009a8f90ae, rc 0) 00000100:00100000:2.0:1776178145.719283:0:6255:0:(service.c:2372:ptlrpc_server_handle_request()) Handled RPC req@000000006d569c58 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_000:b50963bd-8eba-4432-9a54-8888d00e3fa1+26:17008:x1862456979085824:12345-192.168.204.3@tcp:101:chmod.0 Request processed in 166us (2589us total) trans 0 rc 0/0 00000100:00100000:2.0:1776178145.719290:0:6255:0:(nrs_fifo.c:241:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.204.3@tcp, seq: 6502 00000100:00100000:0.0:1776178145.719714:0:8250:0:(service.c:2152:ptlrpc_server_handle_req_in()) got req x1862456979085888 00000100:00100000:0.0:1776178145.719746:0:8250:0:(nrs_fifo.c:179:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.204.3@tcp, seq: 6503 00000100:00100000:0.0:1776178145.719751:0:8250:0:(service.c:2324:ptlrpc_server_handle_request()) Handling RPC req@00000000aa6565e1 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_003:b50963bd-8eba-4432-9a54-8888d00e3fa1+26:17006:x1862456979085888:12345-192.168.204.3@tcp:101:chmod.0 00010000:00010000:0.0:1776178145.719762:0:8250:0:(ldlm_lockd.c:1248:ldlm_handle_enqueue0()) ### server-side enqueue handler START 00010000:00010000:0.0:1776178145.719773:0:8250:0:(ldlm_lockd.c:1318:ldlm_handle_enqueue0()) ### server-side enqueue handler, new lock created ns: mdt-lustre-MDT0000_UUID lock: 00000000f51fd0d7/0xa40083ab9bf530fd lrc: 2/0,0 mode: --/CR res: [0x2000013a1:0x3:0x0].0x0 bits 0x0/0x0 rrc: 3 type: IBT gid 0 flags: 0x40000000000000 nid: local remote: 0x771f68aa1eb27a31 expref: -99 pid: 8250 timeout: 0 lvb_type: 0 00010000:00010000:0.0:1776178145.719807:0:8250:0:(ldlm_lock.c:1064:ldlm_granted_list_add_lock()) ### About to add lock: ns: mdt-lustre-MDT0000_UUID lock: 00000000f51fd0d7/0xa40083ab9bf530fd lrc: 3/0,0 mode: CR/CR res: [0x2000013a1:0x3:0x0].0x0 bits 0x8/0x0 rrc: 3 type: IBT gid 0 flags: 0x50000000000000 nid: 192.168.204.3@tcp remote: 0x771f68aa1eb27a31 expref: 27 pid: 8250 timeout: 0 lvb_type: 0 00010000:00010000:0.0:1776178145.719833:0:8250:0:(ldlm_lockd.c:1484:ldlm_handle_enqueue0()) ### server-side enqueue handler, sending reply (err=0, rc=0) ns: mdt-lustre-MDT0000_UUID lock: 00000000f51fd0d7/0xa40083ab9bf530fd lrc: 3/0,0 mode: CR/CR res: [0x2000013a1:0x3:0x0].0x0 bits 0x8/0x0 rrc: 3 type: IBT gid 0 flags: 0x40000000000000 nid: 192.168.204.3@tcp remote: 0x771f68aa1eb27a31 expref: 27 pid: 8250 timeout: 0 lvb_type: 0 00010000:00010000:0.0:1776178145.719850:0:8250:0:(ldlm_inodebits.c:94:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x2000013a1:0x3:0x0].0x0 (0000000090c54b62) 00010000:00010000:0.0:1776178145.719852:0:8250:0:(ldlm_inodebits.c:96:ldlm_reprocess_inodebits_queue()) Hint 8 00010000:00010000:0.0:1776178145.719854:0:8250:0:(ldlm_lockd.c:1565:ldlm_handle_enqueue0()) ### server-side enqueue handler END (lock 00000000f51fd0d7, rc 0) 00000100:00100000:0.0:1776178145.719886:0:8250:0:(service.c:2372:ptlrpc_server_handle_request()) Handled RPC req@00000000aa6565e1 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_003:b50963bd-8eba-4432-9a54-8888d00e3fa1+27:17006:x1862456979085888:12345-192.168.204.3@tcp:101:chmod.0 Request processed in 134us (1002us total) trans 0 rc 0/0 00000100:00100000:0.0:1776178145.719893:0:8250:0:(nrs_fifo.c:241:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.204.3@tcp, seq: 6503 00000100:00100000:1.0:1776178145.720106:0:3348:0:(events.c:361:request_in_callback()) peer: 12345-192.168.204.3@tcp (source: 12345-192.168.204.3@tcp) 00000100:00100000:1.0:1776178145.720140:0:3348:0:(events.c:361:request_in_callback()) peer: 12345-192.168.204.3@tcp (source: 12345-192.168.204.3@tcp) 00000100:00100000:1.0:1776178145.720166:0:3348:0:(events.c:361:request_in_callback()) peer: 12345-192.168.204.3@tcp (source: 12345-192.168.204.3@tcp) 00000100:00100000:2.0:1776178145.720228:0:20762:0:(service.c:2152:ptlrpc_server_handle_req_in()) got req x1862456979086016 00000100:00100000:2.0:1776178145.720260:0:20762:0:(service.c:2152:ptlrpc_server_handle_req_in()) got req x1862456979085952 00000100:00100000:2.0:1776178145.720279:0:20762:0:(nrs_fifo.c:179:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.204.3@tcp, seq: 5243 00000100:00100000:2.0:1776178145.720283:0:20762:0:(service.c:2324:ptlrpc_server_handle_request()) Handling RPC req@00000000bd5b5f6f pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_013:b50963bd-8eba-4432-9a54-8888d00e3fa1+9:2270:x1862456979086016:12345-192.168.204.3@tcp:2:chmod.0 00002000:00010000:2.0:1776178145.720463:0:20762:0:(ofd_lvb.c:324:ofd_lvbo_update()) res: [0x100000000:0x45:0x0] updating lvb size from disk: 0 -> 0 00002000:00010000:2.0:1776178145.720466:0:20762:0:(ofd_lvb.c:331:ofd_lvbo_update()) res: [0x100000000:0x45:0x0] updating lvb mtime from disk: 1776178095 -> 1776178095 00002000:00010000:2.0:1776178145.720468:0:20762:0:(ofd_lvb.c:337:ofd_lvbo_update()) res: [0x100000000:0x45:0x0] updating lvb atime from disk: 1776178095 -> 1776178095 00002000:00010000:2.0:1776178145.720470:0:20762:0:(ofd_lvb.c:343:ofd_lvbo_update()) res: [0x100000000:0x45:0x0] updating lvb ctime from disk: 1776178095 -> 1776178097 00002000:00010000:2.0:1776178145.720472:0:20762:0:(ofd_lvb.c:349:ofd_lvbo_update()) res: [0x100000000:0x45:0x0] updating lvb blocks from disk: 0 -> 0 00000100:00100000:2.0:1776178145.720499:0:20762:0:(service.c:2372:ptlrpc_server_handle_request()) Handled RPC req@00000000bd5b5f6f pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_013:b50963bd-8eba-4432-9a54-8888d00e3fa1+10:2270:x1862456979086016:12345-192.168.204.3@tcp:2:chmod.0 Request processed in 215us (394us total) trans 4294969345 rc 0/0 00000100:00100000:2.0:1776178145.720506:0:20762:0:(nrs_fifo.c:241:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.204.3@tcp, seq: 5243 00000100:00100000:2.0:1776178145.720519:0:20762:0:(nrs_fifo.c:179:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.204.3@tcp, seq: 5244 00000100:00100000:2.0:1776178145.720523:0:20762:0:(service.c:2324:ptlrpc_server_handle_request()) Handling RPC req@000000001111b219 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_013:b50963bd-8eba-4432-9a54-8888d00e3fa1+8:2272:x1862456979085952:12345-192.168.204.3@tcp:2:chmod.0 00002000:00010000:2.0:1776178145.720597:0:20762:0:(ofd_lvb.c:324:ofd_lvbo_update()) res: [0x100010000:0x44:0x0] updating lvb size from disk: 0 -> 0 00002000:00010000:2.0:1776178145.720599:0:20762:0:(ofd_lvb.c:331:ofd_lvbo_update()) res: [0x100010000:0x44:0x0] updating lvb mtime from disk: 1776178095 -> 1776178095 00002000:00010000:2.0:1776178145.720601:0:20762:0:(ofd_lvb.c:337:ofd_lvbo_update()) res: [0x100010000:0x44:0x0] updating lvb atime from disk: 1776178095 -> 1776178095 00002000:00010000:2.0:1776178145.720603:0:20762:0:(ofd_lvb.c:343:ofd_lvbo_update()) res: [0x100010000:0x44:0x0] updating lvb ctime from disk: 1776178095 -> 1776178097 00002000:00010000:2.0:1776178145.720604:0:20762:0:(ofd_lvb.c:349:ofd_lvbo_update()) res: [0x100010000:0x44:0x0] updating lvb blocks from disk: 0 -> 0 00000100:00100000:2.0:1776178145.720623:0:20762:0:(service.c:2372:ptlrpc_server_handle_request()) Handled RPC req@000000001111b219 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_013:b50963bd-8eba-4432-9a54-8888d00e3fa1+9:2272:x1862456979085952:12345-192.168.204.3@tcp:2:chmod.0 Request processed in 99us (484us total) trans 4294969318 rc 0/0 00000100:00100000:2.0:1776178145.720630:0:20762:0:(nrs_fifo.c:241:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.204.3@tcp, seq: 5244 00000100:00100000:2.0:1776178145.720650:0:6256:0:(service.c:2152:ptlrpc_server_handle_req_in()) got req x1862456979085632 00000100:00100000:2.0:1776178145.720671:0:6256:0:(nrs_fifo.c:179:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.204.3@tcp, seq: 6504 00000100:00100000:2.0:1776178145.720675:0:6256:0:(service.c:2324:ptlrpc_server_handle_request()) Handling RPC req@00000000ee1f9054 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_001:b50963bd-8eba-4432-9a54-8888d00e3fa1+27:17005:x1862456979085632:12345-192.168.204.3@tcp:101:chmod.0 00010000:00010000:2.0:1776178145.720687:0:6256:0:(ldlm_lockd.c:1248:ldlm_handle_enqueue0()) ### server-side enqueue handler START 00010000:00010000:2.0:1776178145.720699:0:6256:0:(ldlm_lockd.c:1318:ldlm_handle_enqueue0()) ### server-side enqueue handler, new lock created ns: mdt-lustre-MDT0000_UUID lock: 000000007e7ea26f/0xa40083ab9bf53104 lrc: 2/0,0 mode: --/CR res: [0x2000013a1:0x2:0x0].0x0 bits 0x0/0x0 rrc: 3 type: IBT gid 0 flags: 0x40000000000000 nid: local remote: 0x771f68aa1eb27a15 expref: -99 pid: 6256 timeout: 0 lvb_type: 0 00010000:00010000:2.0:1776178145.720740:0:6256:0:(ldlm_lock.c:1064:ldlm_granted_list_add_lock()) ### About to add lock: ns: mdt-lustre-MDT0000_UUID lock: 000000007e7ea26f/0xa40083ab9bf53104 lrc: 3/0,0 mode: CR/CR res: [0x2000013a1:0x2:0x0].0x0 bits 0x8/0x0 rrc: 3 type: IBT gid 0 flags: 0x50000000000000 nid: 192.168.204.3@tcp remote: 0x771f68aa1eb27a15 expref: 28 pid: 6256 timeout: 0 lvb_type: 0 00010000:00010000:2.0:1776178145.720752:0:6256:0:(ldlm_lockd.c:1484:ldlm_handle_enqueue0()) ### server-side enqueue handler, sending reply (err=0, rc=0) ns: mdt-lustre-MDT0000_UUID lock: 000000007e7ea26f/0xa40083ab9bf53104 lrc: 3/0,0 mode: CR/CR res: [0x2000013a1:0x2:0x0].0x0 bits 0x8/0x0 rrc: 3 type: IBT gid 0 flags: 0x40000000000000 nid: 192.168.204.3@tcp remote: 0x771f68aa1eb27a15 expref: 28 pid: 6256 timeout: 0 lvb_type: 0 00010000:00010000:2.0:1776178145.720767:0:6256:0:(ldlm_inodebits.c:94:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x2000013a1:0x2:0x0].0x0 (00000000e4751492) 00010000:00010000:2.0:1776178145.720770:0:6256:0:(ldlm_inodebits.c:96:ldlm_reprocess_inodebits_queue()) Hint 8 00010000:00010000:2.0:1776178145.720772:0:6256:0:(ldlm_lockd.c:1565:ldlm_handle_enqueue0()) ### server-side enqueue handler END (lock 000000007e7ea26f, rc 0) 00000100:00100000:2.0:1776178145.720792:0:6256:0:(service.c:2372:ptlrpc_server_handle_request()) Handled RPC req@00000000ee1f9054 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_001:b50963bd-8eba-4432-9a54-8888d00e3fa1+28:17005:x1862456979085632:12345-192.168.204.3@tcp:101:chmod.0 Request processed in 116us (625us total) trans 0 rc 0/0 00000100:00100000:2.0:1776178145.720797:0:6256:0:(nrs_fifo.c:241:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.204.3@tcp, seq: 6504 00000100:00100000:0.0:1776178145.736784:0:3347:0:(events.c:361:request_in_callback()) peer: 12345-192.168.204.3@tcp (source: 12345-192.168.204.3@tcp) 00000100:00100000:0.0:1776178145.736856:0:3347:0:(events.c:361:request_in_callback()) peer: 12345-192.168.204.3@tcp (source: 12345-192.168.204.3@tcp) 00000100:00100000:3.0:1776178145.737262:0:20759:0:(service.c:2152:ptlrpc_server_handle_req_in()) got req x1862456979086080 00000100:00100000:3.0:1776178145.737324:0:20759:0:(service.c:2152:ptlrpc_server_handle_req_in()) got req x1862456979086144 00000100:00100000:3.0:1776178145.737351:0:20759:0:(nrs_fifo.c:179:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.204.3@tcp, seq: 5245 00000100:00100000:3.0:1776178145.737358:0:20759:0:(service.c:2324:ptlrpc_server_handle_request()) Handling RPC req@0000000056dbc32b pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_010:b50963bd-8eba-4432-9a54-8888d00e3fa1+11:2270:x1862456979086080:12345-192.168.204.3@tcp:2:chmod.0 00002000:00010000:3.0:1776178145.737554:0:20759:0:(ofd_lvb.c:324:ofd_lvbo_update()) res: [0x100000000:0x43:0x0] updating lvb size from disk: 0 -> 0 00002000:00010000:3.0:1776178145.737557:0:20759:0:(ofd_lvb.c:331:ofd_lvbo_update()) res: [0x100000000:0x43:0x0] updating lvb mtime from disk: 1776178095 -> 1776178095 00002000:00010000:3.0:1776178145.737559:0:20759:0:(ofd_lvb.c:337:ofd_lvbo_update()) res: [0x100000000:0x43:0x0] updating lvb atime from disk: 1776178095 -> 1776178095 00002000:00010000:3.0:1776178145.737562:0:20759:0:(ofd_lvb.c:343:ofd_lvbo_update()) res: [0x100000000:0x43:0x0] updating lvb ctime from disk: 1776178095 -> 1776178097 00002000:00010000:3.0:1776178145.737563:0:20759:0:(ofd_lvb.c:349:ofd_lvbo_update()) res: [0x100000000:0x43:0x0] updating lvb blocks from disk: 0 -> 0 00000100:00100000:0.0:1776178145.737745:0:20769:0:(nrs_fifo.c:179:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.204.3@tcp, seq: 5246 00000100:00100000:0.0:1776178145.737750:0:20769:0:(service.c:2324:ptlrpc_server_handle_request()) Handling RPC req@000000005c76b1b3 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_018:b50963bd-8eba-4432-9a54-8888d00e3fa1+12:2272:x1862456979086144:12345-192.168.204.3@tcp:2:chmod.0 00002000:00010000:0.0:1776178145.737835:0:20769:0:(ofd_lvb.c:324:ofd_lvbo_update()) res: [0x100000000:0x44:0x0] updating lvb size from disk: 0 -> 0 00002000:00010000:0.0:1776178145.737838:0:20769:0:(ofd_lvb.c:331:ofd_lvbo_update()) res: [0x100000000:0x44:0x0] updating lvb mtime from disk: 1776178095 -> 1776178095 00002000:00010000:0.0:1776178145.737840:0:20769:0:(ofd_lvb.c:337:ofd_lvbo_update()) res: [0x100000000:0x44:0x0] updating lvb atime from disk: 1776178095 -> 1776178095 00002000:00010000:0.0:1776178145.737841:0:20769:0:(ofd_lvb.c:343:ofd_lvbo_update()) res: [0x100000000:0x44:0x0] updating lvb ctime from disk: 1776178095 -> 1776178097 00002000:00010000:0.0:1776178145.737843:0:20769:0:(ofd_lvb.c:349:ofd_lvbo_update()) res: [0x100000000:0x44:0x0] updating lvb blocks from disk: 0 -> 0 00000100:00100000:3.0:1776178145.737962:0:20759:0:(service.c:2372:ptlrpc_server_handle_request()) Handled RPC req@0000000056dbc32b pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_010:b50963bd-8eba-4432-9a54-8888d00e3fa1+13:2270:x1862456979086080:12345-192.168.204.3@tcp:2:chmod.0 Request processed in 602us (1192us total) trans 4294969346 rc 0/0 00000100:00100000:3.0:1776178145.737971:0:20759:0:(nrs_fifo.c:241:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.204.3@tcp, seq: 5245 00000100:00100000:0.0:1776178145.738355:0:20769:0:(service.c:2372:ptlrpc_server_handle_request()) Handled RPC req@000000005c76b1b3 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_018:b50963bd-8eba-4432-9a54-8888d00e3fa1+12:2272:x1862456979086144:12345-192.168.204.3@tcp:2:chmod.0 Request processed in 603us (1499us total) trans 4294969347 rc 0/0 00000100:00100000:0.0:1776178145.738364:0:20769:0:(nrs_fifo.c:241:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.204.3@tcp, seq: 5246 00000100:00100000:2.0:1776178145.740078:0:3346:0:(events.c:361:request_in_callback()) peer: 12345-192.168.204.3@tcp (source: 12345-192.168.204.3@tcp) 00000100:00100000:2.0:1776178145.740201:0:3346:0:(events.c:361:request_in_callback()) peer: 12345-192.168.204.3@tcp (source: 12345-192.168.204.3@tcp) 00000100:00100000:1.0:1776178145.740418:0:20764:0:(service.c:2152:ptlrpc_server_handle_req_in()) got req x1862456979086208 00000100:00100000:1.0:1776178145.740475:0:20764:0:(service.c:2152:ptlrpc_server_handle_req_in()) got req x1862456979086272 00000100:00100000:1.0:1776178145.740511:0:20764:0:(nrs_fifo.c:179:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.204.3@tcp, seq: 5247 00000100:00100000:1.0:1776178145.740517:0:20764:0:(service.c:2324:ptlrpc_server_handle_request()) Handling RPC req@0000000003506921 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_015:b50963bd-8eba-4432-9a54-8888d00e3fa1+10:2271:x1862456979086208:12345-192.168.204.3@tcp:2:chmod.0 00002000:00010000:1.0:1776178145.740661:0:20764:0:(ofd_lvb.c:324:ofd_lvbo_update()) res: [0x100010000:0x43:0x0] updating lvb size from disk: 0 -> 0 00002000:00010000:1.0:1776178145.740665:0:20764:0:(ofd_lvb.c:331:ofd_lvbo_update()) res: [0x100010000:0x43:0x0] updating lvb mtime from disk: 1776178095 -> 1776178095 00002000:00010000:1.0:1776178145.740667:0:20764:0:(ofd_lvb.c:337:ofd_lvbo_update()) res: [0x100010000:0x43:0x0] updating lvb atime from disk: 1776178095 -> 1776178095 00002000:00010000:1.0:1776178145.740670:0:20764:0:(ofd_lvb.c:343:ofd_lvbo_update()) res: [0x100010000:0x43:0x0] updating lvb ctime from disk: 1776178095 -> 1776178097 00002000:00010000:1.0:1776178145.740671:0:20764:0:(ofd_lvb.c:349:ofd_lvbo_update()) res: [0x100010000:0x43:0x0] updating lvb blocks from disk: 0 -> 0 00000100:00100000:1.0:1776178145.740722:0:20764:0:(service.c:2372:ptlrpc_server_handle_request()) Handled RPC req@0000000003506921 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_015:b50963bd-8eba-4432-9a54-8888d00e3fa1+11:2271:x1862456979086208:12345-192.168.204.3@tcp:2:chmod.0 Request processed in 204us (672us total) trans 4294969319 rc 0/0 00000100:00100000:1.0:1776178145.740730:0:20764:0:(nrs_fifo.c:241:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.204.3@tcp, seq: 5247 00000100:00100000:1.0:1776178145.740843:0:20764:0:(nrs_fifo.c:179:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.204.3@tcp, seq: 5248 00000100:00100000:1.0:1776178145.740848:0:20764:0:(service.c:2324:ptlrpc_server_handle_request()) Handling RPC req@0000000022c7a99e pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_015:b50963bd-8eba-4432-9a54-8888d00e3fa1+10:2270:x1862456979086272:12345-192.168.204.3@tcp:2:chmod.0 00000100:00100000:2.0:1776178145.744010:0:3348:0:(events.c:361:request_in_callback()) peer: 12345-192.168.204.3@tcp (source: 12345-192.168.204.3@tcp) 00002000:00010000:1.0:1776178145.744186:0:20764:0:(ofd_lvb.c:324:ofd_lvbo_update()) res: [0x100010000:0x42:0x0] updating lvb size from disk: 0 -> 0 00002000:00010000:1.0:1776178145.744190:0:20764:0:(ofd_lvb.c:331:ofd_lvbo_update()) res: [0x100010000:0x42:0x0] updating lvb mtime from disk: 1776178095 -> 1776178095 00002000:00010000:1.0:1776178145.744193:0:20764:0:(ofd_lvb.c:337:ofd_lvbo_update()) res: [0x100010000:0x42:0x0] updating lvb atime from disk: 1776178095 -> 1776178095 00002000:00010000:1.0:1776178145.744194:0:20764:0:(ofd_lvb.c:343:ofd_lvbo_update()) res: [0x100010000:0x42:0x0] updating lvb ctime from disk: 1776178095 -> 1776178097 00002000:00010000:1.0:1776178145.744196:0:20764:0:(ofd_lvb.c:349:ofd_lvbo_update()) res: [0x100010000:0x42:0x0] updating lvb blocks from disk: 0 -> 0 00000100:00100000:3.0:1776178145.744489:0:20759:0:(service.c:2152:ptlrpc_server_handle_req_in()) got req x1862456979086336 00000100:00100000:3.0:1776178145.744543:0:20759:0:(nrs_fifo.c:179:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.204.3@tcp, seq: 5249 00000100:00100000:3.0:1776178145.744550:0:20759:0:(service.c:2324:ptlrpc_server_handle_request()) Handling RPC req@0000000007572592 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_010:b50963bd-8eba-4432-9a54-8888d00e3fa1+12:2271:x1862456979086336:12345-192.168.204.3@tcp:2:chmod.0 00002000:00010000:3.0:1776178145.744699:0:20759:0:(ofd_lvb.c:324:ofd_lvbo_update()) res: [0x100000000:0x42:0x0] updating lvb size from disk: 0 -> 0 00002000:00010000:3.0:1776178145.744702:0:20759:0:(ofd_lvb.c:331:ofd_lvbo_update()) res: [0x100000000:0x42:0x0] updating lvb mtime from disk: 1776178095 -> 1776178095 00002000:00010000:3.0:1776178145.744703:0:20759:0:(ofd_lvb.c:337:ofd_lvbo_update()) res: [0x100000000:0x42:0x0] updating lvb atime from disk: 1776178095 -> 1776178095 00002000:00010000:3.0:1776178145.744705:0:20759:0:(ofd_lvb.c:343:ofd_lvbo_update()) res: [0x100000000:0x42:0x0] updating lvb ctime from disk: 1776178095 -> 1776178097 00002000:00010000:3.0:1776178145.744707:0:20759:0:(ofd_lvb.c:349:ofd_lvbo_update()) res: [0x100000000:0x42:0x0] updating lvb blocks from disk: 0 -> 0 00000100:00100000:1.0:1776178145.744750:0:20764:0:(service.c:2372:ptlrpc_server_handle_request()) Handled RPC req@0000000022c7a99e pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_015:b50963bd-8eba-4432-9a54-8888d00e3fa1+11:2270:x1862456979086272:12345-192.168.204.3@tcp:2:chmod.0 Request processed in 3900us (4549us total) trans 4294969320 rc 0/0 00000100:00100000:1.0:1776178145.744760:0:20764:0:(nrs_fifo.c:241:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.204.3@tcp, seq: 5248 00000100:00100000:3.0:1776178145.744911:0:20759:0:(service.c:2372:ptlrpc_server_handle_request()) Handled RPC req@0000000007572592 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_010:b50963bd-8eba-4432-9a54-8888d00e3fa1+13:2271:x1862456979086336:12345-192.168.204.3@tcp:2:chmod.0 Request processed in 359us (912us total) trans 4294969348 rc 0/0 00000100:00100000:3.0:1776178145.744918:0:20759:0:(nrs_fifo.c:241:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.204.3@tcp, seq: 5249 00000100:00100000:1.0:1776178145.803497:0:3347:0:(events.c:361:request_in_callback()) peer: 12345-192.168.204.3@tcp (source: 12345-192.168.204.3@tcp) 00000100:00100000:3.0:1776178145.803607:0:6257:0:(service.c:2152:ptlrpc_server_handle_req_in()) got req x1862456979086400 00000100:00100000:3.0:1776178145.803676:0:6257:0:(nrs_fifo.c:179:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.204.3@tcp, seq: 6505 00000100:00100000:3.0:1776178145.803684:0:6257:0:(service.c:2324:ptlrpc_server_handle_request()) Handling RPC req@00000000246f8bc6 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_002:b50963bd-8eba-4432-9a54-8888d00e3fa1+28:17030:x1862456979086400:12345-192.168.204.3@tcp:101:checkstat.0 00010000:00010000:3.0:1776178145.803710:0:6257:0:(ldlm_lockd.c:1248:ldlm_handle_enqueue0()) ### server-side enqueue handler START 00010000:00010000:3.0:1776178145.803735:0:6257:0:(ldlm_lockd.c:1318:ldlm_handle_enqueue0()) ### server-side enqueue handler, new lock created ns: mdt-lustre-MDT0000_UUID lock: 000000009f2de1df/0xa40083ab9bf5310b lrc: 2/0,0 mode: --/CR res: [0x2000013a1:0x1:0x0].0x0 bits 0x0/0x0 rrc: 3 type: IBT gid 0 flags: 0x40000000000000 nid: local remote: 0x771f68aa1eb27a38 expref: -99 pid: 6257 timeout: 0 lvb_type: 0 00010000:00010000:3.0:1776178145.803827:0:6257:0:(ldlm_lock.c:766:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(CR) ns: mdt-lustre-MDT0000_UUID lock: 00000000712476de/0xa40083ab9bf53112 lrc: 3/1,0 mode: --/CR res: [0x2000013a1:0x1:0x0].0x0 bits 0x0/0x0 rrc: 4 type: IBT gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 6257 timeout: 0 lvb_type: 0 00010000:00010000:3.0:1776178145.803844:0:6257:0:(ldlm_lock.c:1064:ldlm_granted_list_add_lock()) ### About to add lock: ns: mdt-lustre-MDT0000_UUID lock: 00000000712476de/0xa40083ab9bf53112 lrc: 3/1,0 mode: CR/CR res: [0x2000013a1:0x1:0x0].0x0 bits 0x2/0x0 rrc: 4 type: IBT gid 0 flags: 0x50210000000000 nid: local remote: 0x0 expref: -99 pid: 6257 timeout: 0 lvb_type: 0 00010000:00010000:3.0:1776178145.803857:0:6257:0:(ldlm_request.c:528:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: mdt-lustre-MDT0000_UUID lock: 00000000712476de/0xa40083ab9bf53112 lrc: 3/1,0 mode: CR/CR res: [0x2000013a1:0x1:0x0].0x0 bits 0x2/0x0 rrc: 4 type: IBT gid 0 flags: 0x40210000000000 nid: local remote: 0x0 expref: -99 pid: 6257 timeout: 0 lvb_type: 0 00010000:00010000:3.0:1776178145.803873:0:6257:0:(ldlm_lock.c:766:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PR) ns: mdt-lustre-MDT0000_UUID lock: 00000000bef7c62f/0xa40083ab9bf53119 lrc: 3/1,0 mode: --/PR res: [0x2000013a1:0x1:0x0].0xcda302ea bits 0x0/0x0 rrc: 2 type: IBT gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 6257 timeout: 0 lvb_type: 0 00010000:00010000:3.0:1776178145.803879:0:6257:0:(ldlm_lock.c:1064:ldlm_granted_list_add_lock()) ### About to add lock: ns: mdt-lustre-MDT0000_UUID lock: 00000000bef7c62f/0xa40083ab9bf53119 lrc: 3/1,0 mode: PR/PR res: [0x2000013a1:0x1:0x0].0xcda302ea bits 0x2/0x0 rrc: 2 type: IBT gid 0 flags: 0x50210000000000 nid: local remote: 0x0 expref: -99 pid: 6257 timeout: 0 lvb_type: 0 00010000:00010000:3.0:1776178145.803885:0:6257:0:(ldlm_request.c:528:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: mdt-lustre-MDT0000_UUID lock: 00000000bef7c62f/0xa40083ab9bf53119 lrc: 3/1,0 mode: PR/PR res: [0x2000013a1:0x1:0x0].0xcda302ea bits 0x2/0x0 rrc: 2 type: IBT gid 0 flags: 0x40210000000000 nid: local remote: 0x0 expref: -99 pid: 6257 timeout: 0 lvb_type: 0 00010000:00010000:3.0:1776178145.803973:0:6257:0:(ldlm_lock.c:766:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PR) ns: mdt-lustre-MDT0000_UUID lock: 000000007353d00c/0xa40083ab9bf53120 lrc: 3/1,0 mode: --/PR res: [0x2000013a1:0x2:0x0].0x0 bits 0x0/0x0 rrc: 4 type: IBT gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 6257 timeout: 0 lvb_type: 0 00010000:00010000:3.0:1776178145.803982:0:6257:0:(ldlm_lock.c:1064:ldlm_granted_list_add_lock()) ### About to add lock: ns: mdt-lustre-MDT0000_UUID lock: 000000007353d00c/0xa40083ab9bf53120 lrc: 3/1,0 mode: PR/PR res: [0x2000013a1:0x2:0x0].0x0 bits 0x5b/0x0 rrc: 4 type: IBT gid 0 flags: 0x50210000000000 nid: local remote: 0x0 expref: -99 pid: 6257 timeout: 0 lvb_type: 0 00010000:00010000:3.0:1776178145.803989:0:6257:0:(ldlm_request.c:528:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: mdt-lustre-MDT0000_UUID lock: 000000007353d00c/0xa40083ab9bf53120 lrc: 3/1,0 mode: PR/PR res: [0x2000013a1:0x2:0x0].0x0 bits 0x5b/0x0 rrc: 4 type: IBT gid 0 flags: 0x40210000000000 nid: local remote: 0x0 expref: -99 pid: 6257 timeout: 0 lvb_type: 0 00000004:00200000:3.0:1776178145.804029:0:6257:0:(mdt_handler.c:912:mdt_pack_attr2body()) [0x2000013a1:0x2:0x0]: returning size 0 00000004:00200000:3.0:1776178145.804034:0:6257:0:(mdt_handler.c:1523:mdt_getattr_internal()) dirent count 0 stripe count 1 MDT count 2 00000004:00010000:3.0:1776178145.804125:0:6257:0:(mdt_handler.c:2353:mdt_getattr_name_lock()) ### Returning lock to client ns: mdt-lustre-MDT0000_UUID lock: 000000007353d00c/0xa40083ab9bf53120 lrc: 3/1,0 mode: PR/PR res: [0x2000013a1:0x2:0x0].0x0 bits 0x5b/0x0 rrc: 4 type: IBT gid 0 flags: 0x40210000000000 nid: local remote: 0x0 expref: -99 pid: 6257 timeout: 0 lvb_type: 0 00010000:00010000:3.0:1776178145.804138:0:6257:0:(ldlm_lock.c:1064:ldlm_granted_list_add_lock()) ### About to add lock: ns: mdt-lustre-MDT0000_UUID lock: 000000007353d00c/0xa40083ab9bf53120 lrc: 3/1,0 mode: PR/PR res: [0x2000013a1:0x2:0x0].0x0 bits 0x1b/0x0 rrc: 4 type: IBT gid 0 flags: 0x50210000000000 nid: local remote: 0x0 expref: -99 pid: 6257 timeout: 0 lvb_type: 0 00010000:00010000:3.0:1776178145.804146:0:6257:0:(ldlm_lock.c:820:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(CR) ns: mdt-lustre-MDT0000_UUID lock: 00000000712476de/0xa40083ab9bf53112 lrc: 3/1,0 mode: CR/CR res: [0x2000013a1:0x1:0x0].0x0 bits 0x2/0x0 rrc: 4 type: IBT gid 0 flags: 0x50210000000000 nid: local remote: 0x0 expref: -99 pid: 6257 timeout: 0 lvb_type: 0 00010000:00010000:3.0:1776178145.804151:0:6257:0:(ldlm_lock.c:879:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: mdt-lustre-MDT0000_UUID lock: 00000000712476de/0xa40083ab9bf53112 lrc: 2/0,0 mode: CR/CR res: [0x2000013a1:0x1:0x0].0x0 bits 0x2/0x0 rrc: 4 type: IBT gid 0 flags: 0x50210400000000 nid: local remote: 0x0 expref: -99 pid: 6257 timeout: 0 lvb_type: 0 00010000:00010000:3.0:1776178145.804158:0:6257:0:(ldlm_lockd.c:1879:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: mdt-lustre-MDT0000_UUID lock: 00000000712476de/0xa40083ab9bf53112 lrc: 3/0,0 mode: CR/CR res: [0x2000013a1:0x1:0x0].0x0 bits 0x2/0x0 rrc: 4 type: IBT gid 0 flags: 0x40210400000000 nid: local remote: 0x0 expref: -99 pid: 6257 timeout: 0 lvb_type: 0 00010000:00010000:3.0:1776178145.804163:0:6257:0:(ldlm_lockd.c:1893:ldlm_handle_bl_callback()) Lock 00000000712476de already unused, calling callback (0000000079e78348) 00010000:00010000:3.0:1776178145.804167:0:6257:0:(ldlm_request.c:364:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: mdt-lustre-MDT0000_UUID lock: 00000000712476de/0xa40083ab9bf53112 lrc: 3/0,0 mode: CR/CR res: [0x2000013a1:0x1:0x0].0x0 bits 0x2/0x0 rrc: 4 type: IBT gid 0 flags: 0x40210400000000 nid: local remote: 0x0 expref: -99 pid: 6257 timeout: 0 lvb_type: 0 00010000:00010000:3.0:1776178145.804174:0:6257:0:(ldlm_request.c:1257:ldlm_cli_cancel_local()) ### server-side local cancel ns: mdt-lustre-MDT0000_UUID lock: 00000000712476de/0xa40083ab9bf53112 lrc: 4/0,0 mode: CR/CR res: [0x2000013a1:0x1:0x0].0x0 bits 0x2/0x0 rrc: 4 type: IBT gid 0 flags: 0x40218400000000 nid: local remote: 0x0 expref: -99 pid: 6257 timeout: 0 lvb_type: 0 00010000:00010000:3.0:1776178145.804186:0:6257:0:(ldlm_inodebits.c:94:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x2000013a1:0x1:0x0].0x0 (00000000315facbe) 00010000:00010000:3.0:1776178145.804188:0:6257:0:(ldlm_inodebits.c:96:ldlm_reprocess_inodebits_queue()) Hint 2 00010000:00010000:3.0:1776178145.804193:0:6257:0:(ldlm_lockd.c:1903:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: mdt-lustre-MDT0000_UUID lock: 00000000712476de/0xa40083ab9bf53112 lrc: 2/0,0 mode: --/CR res: [0x2000013a1:0x1:0x0].0x0 bits 0x2/0x0 rrc: 4 type: IBT gid 0 flags: 0x44a19400000000 nid: local remote: 0x0 expref: -99 pid: 6257 timeout: 0 lvb_type: 0 00010000:00010000:3.0:1776178145.804201:0:6257:0:(ldlm_lock.c:216:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: mdt-lustre-MDT0000_UUID lock: 00000000712476de/0xa40083ab9bf53112 lrc: 0/0,0 mode: --/CR res: [0x2000013a1:0x1:0x0].0x0 bits 0x2/0x0 rrc: 4 type: IBT gid 0 flags: 0x44a19400000000 nid: local remote: 0x0 expref: -99 pid: 6257 timeout: 0 lvb_type: 0 00010000:00010000:3.0:1776178145.804214:0:6257:0:(ldlm_lock.c:820:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PR) ns: mdt-lustre-MDT0000_UUID lock: 00000000bef7c62f/0xa40083ab9bf53119 lrc: 3/1,0 mode: PR/PR res: [0x2000013a1:0x1:0x0].0xcda302ea bits 0x2/0x0 rrc: 2 type: IBT gid 0 flags: 0x50210000000000 nid: local remote: 0x0 expref: -99 pid: 6257 timeout: 0 lvb_type: 0 00010000:00010000:3.0:1776178145.804224:0:6257:0:(ldlm_lock.c:879:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: mdt-lustre-MDT0000_UUID lock: 00000000bef7c62f/0xa40083ab9bf53119 lrc: 2/0,0 mode: PR/PR res: [0x2000013a1:0x1:0x0].0xcda302ea bits 0x2/0x0 rrc: 2 type: IBT gid 0 flags: 0x50210400000000 nid: local remote: 0x0 expref: -99 pid: 6257 timeout: 0 lvb_type: 0 00010000:00010000:3.0:1776178145.804230:0:6257:0:(ldlm_lockd.c:1879:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: mdt-lustre-MDT0000_UUID lock: 00000000bef7c62f/0xa40083ab9bf53119 lrc: 3/0,0 mode: PR/PR res: [0x2000013a1:0x1:0x0].0xcda302ea bits 0x2/0x0 rrc: 2 type: IBT gid 0 flags: 0x40210400000000 nid: local remote: 0x0 expref: -99 pid: 6257 timeout: 0 lvb_type: 0 00010000:00010000:3.0:1776178145.804235:0:6257:0:(ldlm_lockd.c:1893:ldlm_handle_bl_callback()) Lock 00000000bef7c62f already unused, calling callback (0000000079e78348) 00010000:00010000:3.0:1776178145.804236:0:6257:0:(ldlm_request.c:364:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: mdt-lustre-MDT0000_UUID lock: 00000000bef7c62f/0xa40083ab9bf53119 lrc: 3/0,0 mode: PR/PR res: [0x2000013a1:0x1:0x0].0xcda302ea bits 0x2/0x0 rrc: 2 type: IBT gid 0 flags: 0x40210400000000 nid: local remote: 0x0 expref: -99 pid: 6257 timeout: 0 lvb_type: 0 00010000:00010000:3.0:1776178145.804242:0:6257:0:(ldlm_request.c:1257:ldlm_cli_cancel_local()) ### server-side local cancel ns: mdt-lustre-MDT0000_UUID lock: 00000000bef7c62f/0xa40083ab9bf53119 lrc: 4/0,0 mode: PR/PR res: [0x2000013a1:0x1:0x0].0xcda302ea bits 0x2/0x0 rrc: 2 type: IBT gid 0 flags: 0x40218400000000 nid: local remote: 0x0 expref: -99 pid: 6257 timeout: 0 lvb_type: 0 00010000:00010000:3.0:1776178145.804248:0:6257:0:(ldlm_inodebits.c:94:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x2000013a1:0x1:0x0].0xcda302ea (00000000b86d0927) 00010000:00010000:3.0:1776178145.804250:0:6257:0:(ldlm_inodebits.c:96:ldlm_reprocess_inodebits_queue()) Hint 2 00010000:00010000:3.0:1776178145.804252:0:6257:0:(ldlm_lockd.c:1903:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: mdt-lustre-MDT0000_UUID lock: 00000000bef7c62f/0xa40083ab9bf53119 lrc: 2/0,0 mode: --/PR res: [0x2000013a1:0x1:0x0].0xcda302ea bits 0x2/0x0 rrc: 2 type: IBT gid 0 flags: 0x44a19400000000 nid: local remote: 0x0 expref: -99 pid: 6257 timeout: 0 lvb_type: 0 00010000:00010000:3.0:1776178145.804256:0:6257:0:(ldlm_lock.c:216:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: mdt-lustre-MDT0000_UUID lock: 00000000bef7c62f/0xa40083ab9bf53119 lrc: 0/0,0 mode: --/PR res: [0x2000013a1:0x1:0x0].0xcda302ea bits 0x2/0x0 rrc: 2 type: IBT gid 0 flags: 0x44a19400000000 nid: local remote: 0x0 expref: -99 pid: 6257 timeout: 0 lvb_type: 0 00010000:00010000:3.0:1776178145.804288:0:6257:0:(ldlm_lock.c:216:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: mdt-lustre-MDT0000_UUID lock: 000000009f2de1df/0xa40083ab9bf5310b lrc: 0/0,0 mode: --/CR res: [0x2000013a1:0x1:0x0].0x0 bits 0x2/0x0 rrc: 3 type: IBT gid 0 flags: 0x44000000000000 nid: 192.168.204.3@tcp remote: 0x771f68aa1eb27a38 expref: 30 pid: 6257 timeout: 0 lvb_type: 0 00010000:00010000:3.0:1776178145.804300:0:6257:0:(ldlm_lockd.c:1484:ldlm_handle_enqueue0()) ### server-side enqueue handler, sending reply (err=0, rc=0) ns: mdt-lustre-MDT0000_UUID lock: 000000007353d00c/0xa40083ab9bf53120 lrc: 3/0,0 mode: PR/PR res: [0x2000013a1:0x2:0x0].0x0 bits 0x1b/0x0 rrc: 4 type: IBT gid 0 flags: 0x40200000000000 nid: 192.168.204.3@tcp remote: 0x771f68aa1eb27a38 expref: 29 pid: 6257 timeout: 0 lvb_type: 0 00010000:00010000:3.0:1776178145.804306:0:6257:0:(ldlm_inodebits.c:94:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x2000013a1:0x2:0x0].0x0 (00000000e4751492) 00010000:00010000:3.0:1776178145.804308:0:6257:0:(ldlm_inodebits.c:96:ldlm_reprocess_inodebits_queue()) Hint 1b 00010000:00010000:3.0:1776178145.804310:0:6257:0:(ldlm_lockd.c:1565:ldlm_handle_enqueue0()) ### server-side enqueue handler END (lock 000000007353d00c, rc 0) 00000100:00100000:3.0:1776178145.804383:0:6257:0:(service.c:2372:ptlrpc_server_handle_request()) Handled RPC req@00000000246f8bc6 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_002:b50963bd-8eba-4432-9a54-8888d00e3fa1+29:17030:x1862456979086400:12345-192.168.204.3@tcp:101:checkstat.0 Request processed in 698us (897us total) trans 0 rc 0/0 00000100:00100000:3.0:1776178145.804390:0:6257:0:(nrs_fifo.c:241:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.204.3@tcp, seq: 6505 00000100:00100000:2.0:1776178145.819681:0:3346:0:(events.c:361:request_in_callback()) peer: 12345-192.168.204.3@tcp (source: 12345-192.168.204.3@tcp) 00000100:00100000:0.0:1776178145.819821:0:8250:0:(service.c:2152:ptlrpc_server_handle_req_in()) got req x1862456979086464 00000100:00100000:0.0:1776178145.819889:0:8250:0:(nrs_fifo.c:179:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.204.3@tcp, seq: 6506 00000100:00100000:0.0:1776178145.819896:0:8250:0:(service.c:2324:ptlrpc_server_handle_request()) Handling RPC req@000000008c593d89 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_003:b50963bd-8eba-4432-9a54-8888d00e3fa1+29:17031:x1862456979086464:12345-192.168.204.3@tcp:101:checkstat.0 00010000:00010000:0.0:1776178145.819922:0:8250:0:(ldlm_lockd.c:1248:ldlm_handle_enqueue0()) ### server-side enqueue handler START 00010000:00010000:0.0:1776178145.819944:0:8250:0:(ldlm_lockd.c:1318:ldlm_handle_enqueue0()) ### server-side enqueue handler, new lock created ns: mdt-lustre-MDT0000_UUID lock: 000000000b903578/0xa40083ab9bf53127 lrc: 2/0,0 mode: --/CR res: [0x2000013a1:0x1:0x0].0x0 bits 0x0/0x0 rrc: 3 type: IBT gid 0 flags: 0x40000000000000 nid: local remote: 0x771f68aa1eb27a3f expref: -99 pid: 8250 timeout: 0 lvb_type: 0 00010000:00010000:0.0:1776178145.820008:0:8250:0:(ldlm_lock.c:766:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(CR) ns: mdt-lustre-MDT0000_UUID lock: 0000000095e53696/0xa40083ab9bf5312e lrc: 3/1,0 mode: --/CR res: [0x2000013a1:0x1:0x0].0x0 bits 0x0/0x0 rrc: 4 type: IBT gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 8250 timeout: 0 lvb_type: 0 00010000:00010000:0.0:1776178145.820024:0:8250:0:(ldlm_lock.c:1064:ldlm_granted_list_add_lock()) ### About to add lock: ns: mdt-lustre-MDT0000_UUID lock: 0000000095e53696/0xa40083ab9bf5312e lrc: 3/1,0 mode: CR/CR res: [0x2000013a1:0x1:0x0].0x0 bits 0x2/0x0 rrc: 4 type: IBT gid 0 flags: 0x50210000000000 nid: local remote: 0x0 expref: -99 pid: 8250 timeout: 0 lvb_type: 0 00010000:00010000:0.0:1776178145.820041:0:8250:0:(ldlm_request.c:528:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: mdt-lustre-MDT0000_UUID lock: 0000000095e53696/0xa40083ab9bf5312e lrc: 3/1,0 mode: CR/CR res: [0x2000013a1:0x1:0x0].0x0 bits 0x2/0x0 rrc: 4 type: IBT gid 0 flags: 0x40210000000000 nid: local remote: 0x0 expref: -99 pid: 8250 timeout: 0 lvb_type: 0 00010000:00010000:0.0:1776178145.820056:0:8250:0:(ldlm_lock.c:766:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PR) ns: mdt-lustre-MDT0000_UUID lock: 00000000f90c1e39/0xa40083ab9bf53135 lrc: 3/1,0 mode: --/PR res: [0x2000013a1:0x1:0x0].0x15fcbbea bits 0x0/0x0 rrc: 2 type: IBT gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 8250 timeout: 0 lvb_type: 0 00010000:00010000:0.0:1776178145.820119:0:8250:0:(ldlm_lock.c:1064:ldlm_granted_list_add_lock()) ### About to add lock: ns: mdt-lustre-MDT0000_UUID lock: 00000000f90c1e39/0xa40083ab9bf53135 lrc: 3/1,0 mode: PR/PR res: [0x2000013a1:0x1:0x0].0x15fcbbea bits 0x2/0x0 rrc: 2 type: IBT gid 0 flags: 0x50210000000000 nid: local remote: 0x0 expref: -99 pid: 8250 timeout: 0 lvb_type: 0 00010000:00010000:0.0:1776178145.820126:0:8250:0:(ldlm_request.c:528:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: mdt-lustre-MDT0000_UUID lock: 00000000f90c1e39/0xa40083ab9bf53135 lrc: 3/1,0 mode: PR/PR res: [0x2000013a1:0x1:0x0].0x15fcbbea bits 0x2/0x0 rrc: 2 type: IBT gid 0 flags: 0x40210000000000 nid: local remote: 0x0 expref: -99 pid: 8250 timeout: 0 lvb_type: 0 00010000:00010000:0.0:1776178145.820205:0:8250:0:(ldlm_lock.c:766:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PR) ns: mdt-lustre-MDT0000_UUID lock: 0000000030bf6b1e/0xa40083ab9bf5313c lrc: 3/1,0 mode: --/PR res: [0x2000013a1:0x3:0x0].0x0 bits 0x0/0x0 rrc: 4 type: IBT gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 8250 timeout: 0 lvb_type: 0 00010000:00010000:0.0:1776178145.820214:0:8250:0:(ldlm_lock.c:1064:ldlm_granted_list_add_lock()) ### About to add lock: ns: mdt-lustre-MDT0000_UUID lock: 0000000030bf6b1e/0xa40083ab9bf5313c lrc: 3/1,0 mode: PR/PR res: [0x2000013a1:0x3:0x0].0x0 bits 0x5b/0x0 rrc: 4 type: IBT gid 0 flags: 0x50210000000000 nid: local remote: 0x0 expref: -99 pid: 8250 timeout: 0 lvb_type: 0 00010000:00010000:0.0:1776178145.820220:0:8250:0:(ldlm_request.c:528:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: mdt-lustre-MDT0000_UUID lock: 0000000030bf6b1e/0xa40083ab9bf5313c lrc: 3/1,0 mode: PR/PR res: [0x2000013a1:0x3:0x0].0x0 bits 0x5b/0x0 rrc: 4 type: IBT gid 0 flags: 0x40210000000000 nid: local remote: 0x0 expref: -99 pid: 8250 timeout: 0 lvb_type: 0 00000004:00200000:0.0:1776178145.820473:0:8250:0:(mdt_handler.c:912:mdt_pack_attr2body()) [0x2000013a1:0x3:0x0]: returning size 0 00000004:00200000:0.0:1776178145.820478:0:8250:0:(mdt_handler.c:1523:mdt_getattr_internal()) dirent count 0 stripe count 1 MDT count 2 00000004:00010000:0.0:1776178145.820504:0:8250:0:(mdt_handler.c:2353:mdt_getattr_name_lock()) ### Returning lock to client ns: mdt-lustre-MDT0000_UUID lock: 0000000030bf6b1e/0xa40083ab9bf5313c lrc: 3/1,0 mode: PR/PR res: [0x2000013a1:0x3:0x0].0x0 bits 0x5b/0x0 rrc: 4 type: IBT gid 0 flags: 0x40210000000000 nid: local remote: 0x0 expref: -99 pid: 8250 timeout: 0 lvb_type: 0 00010000:00010000:0.0:1776178145.820515:0:8250:0:(ldlm_lock.c:1064:ldlm_granted_list_add_lock()) ### About to add lock: ns: mdt-lustre-MDT0000_UUID lock: 0000000030bf6b1e/0xa40083ab9bf5313c lrc: 3/1,0 mode: PR/PR res: [0x2000013a1:0x3:0x0].0x0 bits 0x1b/0x0 rrc: 4 type: IBT gid 0 flags: 0x50210000000000 nid: local remote: 0x0 expref: -99 pid: 8250 timeout: 0 lvb_type: 0 00010000:00010000:0.0:1776178145.820523:0:8250:0:(ldlm_lock.c:820:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(CR) ns: mdt-lustre-MDT0000_UUID lock: 0000000095e53696/0xa40083ab9bf5312e lrc: 3/1,0 mode: CR/CR res: [0x2000013a1:0x1:0x0].0x0 bits 0x2/0x0 rrc: 4 type: IBT gid 0 flags: 0x50210000000000 nid: local remote: 0x0 expref: -99 pid: 8250 timeout: 0 lvb_type: 0 00010000:00010000:0.0:1776178145.820530:0:8250:0:(ldlm_lock.c:879:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: mdt-lustre-MDT0000_UUID lock: 0000000095e53696/0xa40083ab9bf5312e lrc: 2/0,0 mode: CR/CR res: [0x2000013a1:0x1:0x0].0x0 bits 0x2/0x0 rrc: 4 type: IBT gid 0 flags: 0x50210400000000 nid: local remote: 0x0 expref: -99 pid: 8250 timeout: 0 lvb_type: 0 00010000:00010000:0.0:1776178145.820536:0:8250:0:(ldlm_lockd.c:1879:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: mdt-lustre-MDT0000_UUID lock: 0000000095e53696/0xa40083ab9bf5312e lrc: 3/0,0 mode: CR/CR res: [0x2000013a1:0x1:0x0].0x0 bits 0x2/0x0 rrc: 4 type: IBT gid 0 flags: 0x40210400000000 nid: local remote: 0x0 expref: -99 pid: 8250 timeout: 0 lvb_type: 0 00010000:00010000:0.0:1776178145.820542:0:8250:0:(ldlm_lockd.c:1893:ldlm_handle_bl_callback()) Lock 0000000095e53696 already unused, calling callback (0000000079e78348) 00010000:00010000:0.0:1776178145.820546:0:8250:0:(ldlm_request.c:364:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: mdt-lustre-MDT0000_UUID lock: 0000000095e53696/0xa40083ab9bf5312e lrc: 3/0,0 mode: CR/CR res: [0x2000013a1:0x1:0x0].0x0 bits 0x2/0x0 rrc: 4 type: IBT gid 0 flags: 0x40210400000000 nid: local remote: 0x0 expref: -99 pid: 8250 timeout: 0 lvb_type: 0 00010000:00010000:0.0:1776178145.820554:0:8250:0:(ldlm_request.c:1257:ldlm_cli_cancel_local()) ### server-side local cancel ns: mdt-lustre-MDT0000_UUID lock: 0000000095e53696/0xa40083ab9bf5312e lrc: 4/0,0 mode: CR/CR res: [0x2000013a1:0x1:0x0].0x0 bits 0x2/0x0 rrc: 4 type: IBT gid 0 flags: 0x40218400000000 nid: local remote: 0x0 expref: -99 pid: 8250 timeout: 0 lvb_type: 0 00010000:00010000:0.0:1776178145.820566:0:8250:0:(ldlm_inodebits.c:94:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x2000013a1:0x1:0x0].0x0 (00000000315facbe) 00010000:00010000:0.0:1776178145.820570:0:8250:0:(ldlm_inodebits.c:96:ldlm_reprocess_inodebits_queue()) Hint 2 00010000:00010000:0.0:1776178145.820574:0:8250:0:(ldlm_lockd.c:1903:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: mdt-lustre-MDT0000_UUID lock: 0000000095e53696/0xa40083ab9bf5312e lrc: 2/0,0 mode: --/CR res: [0x2000013a1:0x1:0x0].0x0 bits 0x2/0x0 rrc: 4 type: IBT gid 0 flags: 0x44a19400000000 nid: local remote: 0x0 expref: -99 pid: 8250 timeout: 0 lvb_type: 0 00010000:00010000:0.0:1776178145.820579:0:8250:0:(ldlm_lock.c:216:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: mdt-lustre-MDT0000_UUID lock: 0000000095e53696/0xa40083ab9bf5312e lrc: 0/0,0 mode: --/CR res: [0x2000013a1:0x1:0x0].0x0 bits 0x2/0x0 rrc: 4 type: IBT gid 0 flags: 0x44a19400000000 nid: local remote: 0x0 expref: -99 pid: 8250 timeout: 0 lvb_type: 0 00010000:00010000:0.0:1776178145.820592:0:8250:0:(ldlm_lock.c:820:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PR) ns: mdt-lustre-MDT0000_UUID lock: 00000000f90c1e39/0xa40083ab9bf53135 lrc: 3/1,0 mode: PR/PR res: [0x2000013a1:0x1:0x0].0x15fcbbea bits 0x2/0x0 rrc: 2 type: IBT gid 0 flags: 0x50210000000000 nid: local remote: 0x0 expref: -99 pid: 8250 timeout: 0 lvb_type: 0 00010000:00010000:0.0:1776178145.820598:0:8250:0:(ldlm_lock.c:879:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: mdt-lustre-MDT0000_UUID lock: 00000000f90c1e39/0xa40083ab9bf53135 lrc: 2/0,0 mode: PR/PR res: [0x2000013a1:0x1:0x0].0x15fcbbea bits 0x2/0x0 rrc: 2 type: IBT gid 0 flags: 0x50210400000000 nid: local remote: 0x0 expref: -99 pid: 8250 timeout: 0 lvb_type: 0 00010000:00010000:0.0:1776178145.820603:0:8250:0:(ldlm_lockd.c:1879:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: mdt-lustre-MDT0000_UUID lock: 00000000f90c1e39/0xa40083ab9bf53135 lrc: 3/0,0 mode: PR/PR res: [0x2000013a1:0x1:0x0].0x15fcbbea bits 0x2/0x0 rrc: 2 type: IBT gid 0 flags: 0x40210400000000 nid: local remote: 0x0 expref: -99 pid: 8250 timeout: 0 lvb_type: 0 00010000:00010000:0.0:1776178145.820614:0:8250:0:(ldlm_lockd.c:1893:ldlm_handle_bl_callback()) Lock 00000000f90c1e39 already unused, calling callback (0000000079e78348) 00010000:00010000:0.0:1776178145.820616:0:8250:0:(ldlm_request.c:364:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: mdt-lustre-MDT0000_UUID lock: 00000000f90c1e39/0xa40083ab9bf53135 lrc: 3/0,0 mode: PR/PR res: [0x2000013a1:0x1:0x0].0x15fcbbea bits 0x2/0x0 rrc: 2 type: IBT gid 0 flags: 0x40210400000000 nid: local remote: 0x0 expref: -99 pid: 8250 timeout: 0 lvb_type: 0 00010000:00010000:0.0:1776178145.820621:0:8250:0:(ldlm_request.c:1257:ldlm_cli_cancel_local()) ### server-side local cancel ns: mdt-lustre-MDT0000_UUID lock: 00000000f90c1e39/0xa40083ab9bf53135 lrc: 4/0,0 mode: PR/PR res: [0x2000013a1:0x1:0x0].0x15fcbbea bits 0x2/0x0 rrc: 2 type: IBT gid 0 flags: 0x40218400000000 nid: local remote: 0x0 expref: -99 pid: 8250 timeout: 0 lvb_type: 0 00010000:00010000:0.0:1776178145.820628:0:8250:0:(ldlm_inodebits.c:94:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x2000013a1:0x1:0x0].0x15fcbbea (00000000431b0f4e) 00010000:00010000:0.0:1776178145.820630:0:8250:0:(ldlm_inodebits.c:96:ldlm_reprocess_inodebits_queue()) Hint 2 00010000:00010000:0.0:1776178145.820631:0:8250:0:(ldlm_lockd.c:1903:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: mdt-lustre-MDT0000_UUID lock: 00000000f90c1e39/0xa40083ab9bf53135 lrc: 2/0,0 mode: --/PR res: [0x2000013a1:0x1:0x0].0x15fcbbea bits 0x2/0x0 rrc: 2 type: IBT gid 0 flags: 0x44a19400000000 nid: local remote: 0x0 expref: -99 pid: 8250 timeout: 0 lvb_type: 0 00010000:00010000:0.0:1776178145.820636:0:8250:0:(ldlm_lock.c:216:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: mdt-lustre-MDT0000_UUID lock: 00000000f90c1e39/0xa40083ab9bf53135 lrc: 0/0,0 mode: --/PR res: [0x2000013a1:0x1:0x0].0x15fcbbea bits 0x2/0x0 rrc: 2 type: IBT gid 0 flags: 0x44a19400000000 nid: local remote: 0x0 expref: -99 pid: 8250 timeout: 0 lvb_type: 0 00010000:00010000:0.0:1776178145.820667:0:8250:0:(ldlm_lock.c:216:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: mdt-lustre-MDT0000_UUID lock: 000000000b903578/0xa40083ab9bf53127 lrc: 0/0,0 mode: --/CR res: [0x2000013a1:0x1:0x0].0x0 bits 0x2/0x0 rrc: 3 type: IBT gid 0 flags: 0x44000000000000 nid: 192.168.204.3@tcp remote: 0x771f68aa1eb27a3f expref: 31 pid: 8250 timeout: 0 lvb_type: 0 00010000:00010000:0.0:1776178145.820680:0:8250:0:(ldlm_lockd.c:1484:ldlm_handle_enqueue0()) ### server-side enqueue handler, sending reply (err=0, rc=0) ns: mdt-lustre-MDT0000_UUID lock: 0000000030bf6b1e/0xa40083ab9bf5313c lrc: 3/0,0 mode: PR/PR res: [0x2000013a1:0x3:0x0].0x0 bits 0x1b/0x0 rrc: 4 type: IBT gid 0 flags: 0x40200000000000 nid: 192.168.204.3@tcp remote: 0x771f68aa1eb27a3f expref: 30 pid: 8250 timeout: 0 lvb_type: 0 00010000:00010000:0.0:1776178145.820686:0:8250:0:(ldlm_inodebits.c:94:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x2000013a1:0x3:0x0].0x0 (0000000090c54b62) 00010000:00010000:0.0:1776178145.820688:0:8250:0:(ldlm_inodebits.c:96:ldlm_reprocess_inodebits_queue()) Hint 1b 00010000:00010000:0.0:1776178145.820690:0:8250:0:(ldlm_lockd.c:1565:ldlm_handle_enqueue0()) ### server-side enqueue handler END (lock 0000000030bf6b1e, rc 0) 00000100:00100000:0.0:1776178145.820861:0:8250:0:(service.c:2372:ptlrpc_server_handle_request()) Handled RPC req@000000008c593d89 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_003:b50963bd-8eba-4432-9a54-8888d00e3fa1+30:17031:x1862456979086464:12345-192.168.204.3@tcp:101:checkstat.0 Request processed in 964us (1188us total) trans 0 rc 0/0 00000100:00100000:0.0:1776178145.820872:0:8250:0:(nrs_fifo.c:241:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.204.3@tcp, seq: 6506 00000100:00100000:3.0:1776178145.845973:0:3348:0:(events.c:361:request_in_callback()) peer: 12345-192.168.204.3@tcp (source: 12345-192.168.204.3@tcp) 00000100:00100000:0.0:1776178145.846142:0:8250:0:(service.c:2152:ptlrpc_server_handle_req_in()) got req x1862456979086528 00000100:00100000:0.0:1776178145.846204:0:8250:0:(nrs_fifo.c:179:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.204.3@tcp, seq: 6507 00000100:00100000:0.0:1776178145.846212:0:8250:0:(service.c:2324:ptlrpc_server_handle_request()) Handling RPC req@00000000bdad24ac pname:cluuid+ref:pid:xid:nid:opc:job mdt00_003:b50963bd-8eba-4432-9a54-8888d00e3fa1+30:17032:x1862456979086528:12345-192.168.204.3@tcp:101:checkstat.0 00010000:00010000:0.0:1776178145.846237:0:8250:0:(ldlm_lockd.c:1248:ldlm_handle_enqueue0()) ### server-side enqueue handler START 00010000:00010000:0.0:1776178145.846271:0:8250:0:(ldlm_lockd.c:1318:ldlm_handle_enqueue0()) ### server-side enqueue handler, new lock created ns: mdt-lustre-MDT0000_UUID lock: 0000000068850c85/0xa40083ab9bf53143 lrc: 2/0,0 mode: --/CR res: [0x2000013a1:0x1:0x0].0x0 bits 0x0/0x0 rrc: 3 type: IBT gid 0 flags: 0x40000000000000 nid: local remote: 0x771f68aa1eb27a46 expref: -99 pid: 8250 timeout: 0 lvb_type: 0 00010000:00010000:0.0:1776178145.846335:0:8250:0:(ldlm_lock.c:766:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(CR) ns: mdt-lustre-MDT0000_UUID lock: 000000000e7a37e0/0xa40083ab9bf5314a lrc: 3/1,0 mode: --/CR res: [0x2000013a1:0x1:0x0].0x0 bits 0x0/0x0 rrc: 4 type: IBT gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 8250 timeout: 0 lvb_type: 0 00010000:00010000:0.0:1776178145.846350:0:8250:0:(ldlm_lock.c:1064:ldlm_granted_list_add_lock()) ### About to add lock: ns: mdt-lustre-MDT0000_UUID lock: 000000000e7a37e0/0xa40083ab9bf5314a lrc: 3/1,0 mode: CR/CR res: [0x2000013a1:0x1:0x0].0x0 bits 0x2/0x0 rrc: 4 type: IBT gid 0 flags: 0x50210000000000 nid: local remote: 0x0 expref: -99 pid: 8250 timeout: 0 lvb_type: 0 00010000:00010000:0.0:1776178145.846362:0:8250:0:(ldlm_request.c:528:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: mdt-lustre-MDT0000_UUID lock: 000000000e7a37e0/0xa40083ab9bf5314a lrc: 3/1,0 mode: CR/CR res: [0x2000013a1:0x1:0x0].0x0 bits 0x2/0x0 rrc: 4 type: IBT gid 0 flags: 0x40210000000000 nid: local remote: 0x0 expref: -99 pid: 8250 timeout: 0 lvb_type: 0 00010000:00010000:0.0:1776178145.846378:0:8250:0:(ldlm_lock.c:766:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PR) ns: mdt-lustre-MDT0000_UUID lock: 00000000782e253e/0xa40083ab9bf53151 lrc: 3/1,0 mode: --/PR res: [0x2000013a1:0x1:0x0].0x5e5674ea bits 0x0/0x0 rrc: 2 type: IBT gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 8250 timeout: 0 lvb_type: 0 00010000:00010000:0.0:1776178145.846385:0:8250:0:(ldlm_lock.c:1064:ldlm_granted_list_add_lock()) ### About to add lock: ns: mdt-lustre-MDT0000_UUID lock: 00000000782e253e/0xa40083ab9bf53151 lrc: 3/1,0 mode: PR/PR res: [0x2000013a1:0x1:0x0].0x5e5674ea bits 0x2/0x0 rrc: 2 type: IBT gid 0 flags: 0x50210000000000 nid: local remote: 0x0 expref: -99 pid: 8250 timeout: 0 lvb_type: 0 00010000:00010000:0.0:1776178145.846391:0:8250:0:(ldlm_request.c:528:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: mdt-lustre-MDT0000_UUID lock: 00000000782e253e/0xa40083ab9bf53151 lrc: 3/1,0 mode: PR/PR res: [0x2000013a1:0x1:0x0].0x5e5674ea bits 0x2/0x0 rrc: 2 type: IBT gid 0 flags: 0x40210000000000 nid: local remote: 0x0 expref: -99 pid: 8250 timeout: 0 lvb_type: 0 00010000:00010000:0.0:1776178145.846479:0:8250:0:(ldlm_lock.c:766:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PR) ns: mdt-lustre-MDT0000_UUID lock: 000000001735c8e5/0xa40083ab9bf53158 lrc: 3/1,0 mode: --/PR res: [0x2000013a1:0x4:0x0].0x0 bits 0x0/0x0 rrc: 4 type: IBT gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 8250 timeout: 0 lvb_type: 0 00010000:00010000:0.0:1776178145.846489:0:8250:0:(ldlm_lock.c:1064:ldlm_granted_list_add_lock()) ### About to add lock: ns: mdt-lustre-MDT0000_UUID lock: 000000001735c8e5/0xa40083ab9bf53158 lrc: 3/1,0 mode: PR/PR res: [0x2000013a1:0x4:0x0].0x0 bits 0x5b/0x0 rrc: 4 type: IBT gid 0 flags: 0x50210000000000 nid: local remote: 0x0 expref: -99 pid: 8250 timeout: 0 lvb_type: 0 00010000:00010000:0.0:1776178145.846495:0:8250:0:(ldlm_request.c:528:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: mdt-lustre-MDT0000_UUID lock: 000000001735c8e5/0xa40083ab9bf53158 lrc: 3/1,0 mode: PR/PR res: [0x2000013a1:0x4:0x0].0x0 bits 0x5b/0x0 rrc: 4 type: IBT gid 0 flags: 0x40210000000000 nid: local remote: 0x0 expref: -99 pid: 8250 timeout: 0 lvb_type: 0 00000004:00200000:0.0:1776178145.846540:0:8250:0:(mdt_handler.c:912:mdt_pack_attr2body()) [0x2000013a1:0x4:0x0]: returning size 0 00000004:00200000:0.0:1776178145.846544:0:8250:0:(mdt_handler.c:1523:mdt_getattr_internal()) dirent count 0 stripe count 1 MDT count 2 00000004:00010000:0.0:1776178145.846565:0:8250:0:(mdt_handler.c:2353:mdt_getattr_name_lock()) ### Returning lock to client ns: mdt-lustre-MDT0000_UUID lock: 000000001735c8e5/0xa40083ab9bf53158 lrc: 3/1,0 mode: PR/PR res: [0x2000013a1:0x4:0x0].0x0 bits 0x5b/0x0 rrc: 4 type: IBT gid 0 flags: 0x40210000000000 nid: local remote: 0x0 expref: -99 pid: 8250 timeout: 0 lvb_type: 0 00010000:00010000:0.0:1776178145.846578:0:8250:0:(ldlm_lock.c:1064:ldlm_granted_list_add_lock()) ### About to add lock: ns: mdt-lustre-MDT0000_UUID lock: 000000001735c8e5/0xa40083ab9bf53158 lrc: 3/1,0 mode: PR/PR res: [0x2000013a1:0x4:0x0].0x0 bits 0x1b/0x0 rrc: 4 type: IBT gid 0 flags: 0x50210000000000 nid: local remote: 0x0 expref: -99 pid: 8250 timeout: 0 lvb_type: 0 00010000:00010000:0.0:1776178145.846585:0:8250:0:(ldlm_lock.c:820:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(CR) ns: mdt-lustre-MDT0000_UUID lock: 000000000e7a37e0/0xa40083ab9bf5314a lrc: 3/1,0 mode: CR/CR res: [0x2000013a1:0x1:0x0].0x0 bits 0x2/0x0 rrc: 4 type: IBT gid 0 flags: 0x50210000000000 nid: local remote: 0x0 expref: -99 pid: 8250 timeout: 0 lvb_type: 0 00010000:00010000:0.0:1776178145.846590:0:8250:0:(ldlm_lock.c:879:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: mdt-lustre-MDT0000_UUID lock: 000000000e7a37e0/0xa40083ab9bf5314a lrc: 2/0,0 mode: CR/CR res: [0x2000013a1:0x1:0x0].0x0 bits 0x2/0x0 rrc: 4 type: IBT gid 0 flags: 0x50210400000000 nid: local remote: 0x0 expref: -99 pid: 8250 timeout: 0 lvb_type: 0 00010000:00010000:0.0:1776178145.846597:0:8250:0:(ldlm_lockd.c:1879:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: mdt-lustre-MDT0000_UUID lock: 000000000e7a37e0/0xa40083ab9bf5314a lrc: 3/0,0 mode: CR/CR res: [0x2000013a1:0x1:0x0].0x0 bits 0x2/0x0 rrc: 4 type: IBT gid 0 flags: 0x40210400000000 nid: local remote: 0x0 expref: -99 pid: 8250 timeout: 0 lvb_type: 0 00010000:00010000:0.0:1776178145.846603:0:8250:0:(ldlm_lockd.c:1893:ldlm_handle_bl_callback()) Lock 000000000e7a37e0 already unused, calling callback (0000000079e78348) 00010000:00010000:0.0:1776178145.846606:0:8250:0:(ldlm_request.c:364:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: mdt-lustre-MDT0000_UUID lock: 000000000e7a37e0/0xa40083ab9bf5314a lrc: 3/0,0 mode: CR/CR res: [0x2000013a1:0x1:0x0].0x0 bits 0x2/0x0 rrc: 4 type: IBT gid 0 flags: 0x40210400000000 nid: local remote: 0x0 expref: -99 pid: 8250 timeout: 0 lvb_type: 0 00010000:00010000:0.0:1776178145.846614:0:8250:0:(ldlm_request.c:1257:ldlm_cli_cancel_local()) ### server-side local cancel ns: mdt-lustre-MDT0000_UUID lock: 000000000e7a37e0/0xa40083ab9bf5314a lrc: 4/0,0 mode: CR/CR res: [0x2000013a1:0x1:0x0].0x0 bits 0x2/0x0 rrc: 4 type: IBT gid 0 flags: 0x40218400000000 nid: local remote: 0x0 expref: -99 pid: 8250 timeout: 0 lvb_type: 0 00010000:00010000:0.0:1776178145.846626:0:8250:0:(ldlm_inodebits.c:94:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x2000013a1:0x1:0x0].0x0 (00000000315facbe) 00010000:00010000:0.0:1776178145.846629:0:8250:0:(ldlm_inodebits.c:96:ldlm_reprocess_inodebits_queue()) Hint 2 00010000:00010000:0.0:1776178145.846633:0:8250:0:(ldlm_lockd.c:1903:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: mdt-lustre-MDT0000_UUID lock: 000000000e7a37e0/0xa40083ab9bf5314a lrc: 2/0,0 mode: --/CR res: [0x2000013a1:0x1:0x0].0x0 bits 0x2/0x0 rrc: 4 type: IBT gid 0 flags: 0x44a19400000000 nid: local remote: 0x0 expref: -99 pid: 8250 timeout: 0 lvb_type: 0 00010000:00010000:0.0:1776178145.846638:0:8250:0:(ldlm_lock.c:216:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: mdt-lustre-MDT0000_UUID lock: 000000000e7a37e0/0xa40083ab9bf5314a lrc: 0/0,0 mode: --/CR res: [0x2000013a1:0x1:0x0].0x0 bits 0x2/0x0 rrc: 4 type: IBT gid 0 flags: 0x44a19400000000 nid: local remote: 0x0 expref: -99 pid: 8250 timeout: 0 lvb_type: 0 00010000:00010000:0.0:1776178145.846653:0:8250:0:(ldlm_lock.c:820:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PR) ns: mdt-lustre-MDT0000_UUID lock: 00000000782e253e/0xa40083ab9bf53151 lrc: 3/1,0 mode: PR/PR res: [0x2000013a1:0x1:0x0].0x5e5674ea bits 0x2/0x0 rrc: 2 type: IBT gid 0 flags: 0x50210000000000 nid: local remote: 0x0 expref: -99 pid: 8250 timeout: 0 lvb_type: 0 00010000:00010000:0.0:1776178145.846658:0:8250:0:(ldlm_lock.c:879:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: mdt-lustre-MDT0000_UUID lock: 00000000782e253e/0xa40083ab9bf53151 lrc: 2/0,0 mode: PR/PR res: [0x2000013a1:0x1:0x0].0x5e5674ea bits 0x2/0x0 rrc: 2 type: IBT gid 0 flags: 0x50210400000000 nid: local remote: 0x0 expref: -99 pid: 8250 timeout: 0 lvb_type: 0 00010000:00010000:0.0:1776178145.846662:0:8250:0:(ldlm_lockd.c:1879:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: mdt-lustre-MDT0000_UUID lock: 00000000782e253e/0xa40083ab9bf53151 lrc: 3/0,0 mode: PR/PR res: [0x2000013a1:0x1:0x0].0x5e5674ea bits 0x2/0x0 rrc: 2 type: IBT gid 0 flags: 0x40210400000000 nid: local remote: 0x0 expref: -99 pid: 8250 timeout: 0 lvb_type: 0 00010000:00010000:0.0:1776178145.846667:0:8250:0:(ldlm_lockd.c:1893:ldlm_handle_bl_callback()) Lock 00000000782e253e already unused, calling callback (0000000079e78348) 00010000:00010000:0.0:1776178145.846669:0:8250:0:(ldlm_request.c:364:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: mdt-lustre-MDT0000_UUID lock: 00000000782e253e/0xa40083ab9bf53151 lrc: 3/0,0 mode: PR/PR res: [0x2000013a1:0x1:0x0].0x5e5674ea bits 0x2/0x0 rrc: 2 type: IBT gid 0 flags: 0x40210400000000 nid: local remote: 0x0 expref: -99 pid: 8250 timeout: 0 lvb_type: 0 00010000:00010000:0.0:1776178145.846674:0:8250:0:(ldlm_request.c:1257:ldlm_cli_cancel_local()) ### server-side local cancel ns: mdt-lustre-MDT0000_UUID lock: 00000000782e253e/0xa40083ab9bf53151 lrc: 4/0,0 mode: PR/PR res: [0x2000013a1:0x1:0x0].0x5e5674ea bits 0x2/0x0 rrc: 2 type: IBT gid 0 flags: 0x40218400000000 nid: local remote: 0x0 expref: -99 pid: 8250 timeout: 0 lvb_type: 0 00010000:00010000:0.0:1776178145.846681:0:8250:0:(ldlm_inodebits.c:94:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x2000013a1:0x1:0x0].0x5e5674ea (00000000eb684e0b) 00010000:00010000:0.0:1776178145.846682:0:8250:0:(ldlm_inodebits.c:96:ldlm_reprocess_inodebits_queue()) Hint 2 00010000:00010000:0.0:1776178145.846684:0:8250:0:(ldlm_lockd.c:1903:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: mdt-lustre-MDT0000_UUID lock: 00000000782e253e/0xa40083ab9bf53151 lrc: 2/0,0 mode: --/PR res: [0x2000013a1:0x1:0x0].0x5e5674ea bits 0x2/0x0 rrc: 2 type: IBT gid 0 flags: 0x44a19400000000 nid: local remote: 0x0 expref: -99 pid: 8250 timeout: 0 lvb_type: 0 00010000:00010000:0.0:1776178145.846689:0:8250:0:(ldlm_lock.c:216:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: mdt-lustre-MDT0000_UUID lock: 00000000782e253e/0xa40083ab9bf53151 lrc: 0/0,0 mode: --/PR res: [0x2000013a1:0x1:0x0].0x5e5674ea bits 0x2/0x0 rrc: 2 type: IBT gid 0 flags: 0x44a19400000000 nid: local remote: 0x0 expref: -99 pid: 8250 timeout: 0 lvb_type: 0 00010000:00010000:0.0:1776178145.846720:0:8250:0:(ldlm_lock.c:216:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: mdt-lustre-MDT0000_UUID lock: 0000000068850c85/0xa40083ab9bf53143 lrc: 0/0,0 mode: --/CR res: [0x2000013a1:0x1:0x0].0x0 bits 0x2/0x0 rrc: 3 type: IBT gid 0 flags: 0x44000000000000 nid: 192.168.204.3@tcp remote: 0x771f68aa1eb27a46 expref: 32 pid: 8250 timeout: 0 lvb_type: 0 00010000:00010000:0.0:1776178145.846733:0:8250:0:(ldlm_lockd.c:1484:ldlm_handle_enqueue0()) ### server-side enqueue handler, sending reply (err=0, rc=0) ns: mdt-lustre-MDT0000_UUID lock: 000000001735c8e5/0xa40083ab9bf53158 lrc: 3/0,0 mode: PR/PR res: [0x2000013a1:0x4:0x0].0x0 bits 0x1b/0x0 rrc: 4 type: IBT gid 0 flags: 0x40200000000000 nid: 192.168.204.3@tcp remote: 0x771f68aa1eb27a46 expref: 31 pid: 8250 timeout: 0 lvb_type: 0 00010000:00010000:0.0:1776178145.846745:0:8250:0:(ldlm_inodebits.c:94:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x2000013a1:0x4:0x0].0x0 (000000009cfa2d69) 00010000:00010000:0.0:1776178145.846747:0:8250:0:(ldlm_inodebits.c:96:ldlm_reprocess_inodebits_queue()) Hint 1b 00010000:00010000:0.0:1776178145.846750:0:8250:0:(ldlm_lockd.c:1565:ldlm_handle_enqueue0()) ### server-side enqueue handler END (lock 000000001735c8e5, rc 0) 00000100:00100000:0.0:1776178145.846832:0:8250:0:(service.c:2372:ptlrpc_server_handle_request()) Handled RPC req@00000000bdad24ac pname:cluuid+ref:pid:xid:nid:opc:job mdt00_003:b50963bd-8eba-4432-9a54-8888d00e3fa1+31:17032:x1862456979086528:12345-192.168.204.3@tcp:101:checkstat.0 Request processed in 619us (870us total) trans 0 rc 0/0 00000100:00100000:0.0:1776178145.846839:0:8250:0:(nrs_fifo.c:241:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.204.3@tcp, seq: 6507 00000100:00100000:1.0:1776178145.867781:0:3347:0:(events.c:361:request_in_callback()) peer: 12345-192.168.204.3@tcp (source: 12345-192.168.204.3@tcp) 00000100:00100000:3.0:1776178145.867961:0:6257:0:(service.c:2152:ptlrpc_server_handle_req_in()) got req x1862456979086592 00000100:00100000:3.0:1776178145.868127:0:6257:0:(nrs_fifo.c:179:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.204.3@tcp, seq: 6508 00000100:00100000:3.0:1776178145.868135:0:6257:0:(service.c:2324:ptlrpc_server_handle_request()) Handling RPC req@00000000977896b4 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_002:b50963bd-8eba-4432-9a54-8888d00e3fa1+31:17033:x1862456979086592:12345-192.168.204.3@tcp:101:checkstat.0 00010000:00010000:3.0:1776178145.868160:0:6257:0:(ldlm_lockd.c:1248:ldlm_handle_enqueue0()) ### server-side enqueue handler START 00010000:00010000:3.0:1776178145.868182:0:6257:0:(ldlm_lockd.c:1318:ldlm_handle_enqueue0()) ### server-side enqueue handler, new lock created ns: mdt-lustre-MDT0000_UUID lock: 000000009f2de1df/0xa40083ab9bf5315f lrc: 2/0,0 mode: --/CR res: [0x2000013a1:0x1:0x0].0x0 bits 0x0/0x0 rrc: 3 type: IBT gid 0 flags: 0x40000000000000 nid: local remote: 0x771f68aa1eb27a4d expref: -99 pid: 6257 timeout: 0 lvb_type: 0 00010000:00010000:3.0:1776178145.868255:0:6257:0:(ldlm_lock.c:766:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(CR) ns: mdt-lustre-MDT0000_UUID lock: 00000000bef7c62f/0xa40083ab9bf53166 lrc: 3/1,0 mode: --/CR res: [0x2000013a1:0x1:0x0].0x0 bits 0x0/0x0 rrc: 4 type: IBT gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 6257 timeout: 0 lvb_type: 0 00010000:00010000:3.0:1776178145.868271:0:6257:0:(ldlm_lock.c:1064:ldlm_granted_list_add_lock()) ### About to add lock: ns: mdt-lustre-MDT0000_UUID lock: 00000000bef7c62f/0xa40083ab9bf53166 lrc: 3/1,0 mode: CR/CR res: [0x2000013a1:0x1:0x0].0x0 bits 0x2/0x0 rrc: 4 type: IBT gid 0 flags: 0x50210000000000 nid: local remote: 0x0 expref: -99 pid: 6257 timeout: 0 lvb_type: 0 00010000:00010000:3.0:1776178145.868283:0:6257:0:(ldlm_request.c:528:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: mdt-lustre-MDT0000_UUID lock: 00000000bef7c62f/0xa40083ab9bf53166 lrc: 3/1,0 mode: CR/CR res: [0x2000013a1:0x1:0x0].0x0 bits 0x2/0x0 rrc: 4 type: IBT gid 0 flags: 0x40210000000000 nid: local remote: 0x0 expref: -99 pid: 6257 timeout: 0 lvb_type: 0 00010000:00010000:3.0:1776178145.868299:0:6257:0:(ldlm_lock.c:766:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PR) ns: mdt-lustre-MDT0000_UUID lock: 00000000712476de/0xa40083ab9bf5316d lrc: 3/1,0 mode: --/PR res: [0x2000013a1:0x1:0x0].0xa6b02dea bits 0x0/0x0 rrc: 2 type: IBT gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 6257 timeout: 0 lvb_type: 0 00010000:00010000:3.0:1776178145.868306:0:6257:0:(ldlm_lock.c:1064:ldlm_granted_list_add_lock()) ### About to add lock: ns: mdt-lustre-MDT0000_UUID lock: 00000000712476de/0xa40083ab9bf5316d lrc: 3/1,0 mode: PR/PR res: [0x2000013a1:0x1:0x0].0xa6b02dea bits 0x2/0x0 rrc: 2 type: IBT gid 0 flags: 0x50210000000000 nid: local remote: 0x0 expref: -99 pid: 6257 timeout: 0 lvb_type: 0 00010000:00010000:3.0:1776178145.868312:0:6257:0:(ldlm_request.c:528:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: mdt-lustre-MDT0000_UUID lock: 00000000712476de/0xa40083ab9bf5316d lrc: 3/1,0 mode: PR/PR res: [0x2000013a1:0x1:0x0].0xa6b02dea bits 0x2/0x0 rrc: 2 type: IBT gid 0 flags: 0x40210000000000 nid: local remote: 0x0 expref: -99 pid: 6257 timeout: 0 lvb_type: 0 00010000:00010000:3.0:1776178145.868390:0:6257:0:(ldlm_lock.c:766:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PR) ns: mdt-lustre-MDT0000_UUID lock: 0000000068a2dd4a/0xa40083ab9bf53174 lrc: 3/1,0 mode: --/PR res: [0x2000013a1:0x5:0x0].0x0 bits 0x0/0x0 rrc: 4 type: IBT gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 6257 timeout: 0 lvb_type: 0 00010000:00010000:3.0:1776178145.868399:0:6257:0:(ldlm_lock.c:1064:ldlm_granted_list_add_lock()) ### About to add lock: ns: mdt-lustre-MDT0000_UUID lock: 0000000068a2dd4a/0xa40083ab9bf53174 lrc: 3/1,0 mode: PR/PR res: [0x2000013a1:0x5:0x0].0x0 bits 0x5b/0x0 rrc: 4 type: IBT gid 0 flags: 0x50210000000000 nid: local remote: 0x0 expref: -99 pid: 6257 timeout: 0 lvb_type: 0 00010000:00010000:3.0:1776178145.868406:0:6257:0:(ldlm_request.c:528:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: mdt-lustre-MDT0000_UUID lock: 0000000068a2dd4a/0xa40083ab9bf53174 lrc: 3/1,0 mode: PR/PR res: [0x2000013a1:0x5:0x0].0x0 bits 0x5b/0x0 rrc: 4 type: IBT gid 0 flags: 0x40210000000000 nid: local remote: 0x0 expref: -99 pid: 6257 timeout: 0 lvb_type: 0 00000004:00200000:3.0:1776178145.868445:0:6257:0:(mdt_handler.c:912:mdt_pack_attr2body()) [0x2000013a1:0x5:0x0]: returning size 0 00000004:00200000:3.0:1776178145.868449:0:6257:0:(mdt_handler.c:1523:mdt_getattr_internal()) dirent count 0 stripe count 1 MDT count 2 00000004:00010000:3.0:1776178145.868472:0:6257:0:(mdt_handler.c:2353:mdt_getattr_name_lock()) ### Returning lock to client ns: mdt-lustre-MDT0000_UUID lock: 0000000068a2dd4a/0xa40083ab9bf53174 lrc: 3/1,0 mode: PR/PR res: [0x2000013a1:0x5:0x0].0x0 bits 0x5b/0x0 rrc: 4 type: IBT gid 0 flags: 0x40210000000000 nid: local remote: 0x0 expref: -99 pid: 6257 timeout: 0 lvb_type: 0 00010000:00010000:3.0:1776178145.868484:0:6257:0:(ldlm_lock.c:1064:ldlm_granted_list_add_lock()) ### About to add lock: ns: mdt-lustre-MDT0000_UUID lock: 0000000068a2dd4a/0xa40083ab9bf53174 lrc: 3/1,0 mode: PR/PR res: [0x2000013a1:0x5:0x0].0x0 bits 0x1b/0x0 rrc: 4 type: IBT gid 0 flags: 0x50210000000000 nid: local remote: 0x0 expref: -99 pid: 6257 timeout: 0 lvb_type: 0 00010000:00010000:3.0:1776178145.868493:0:6257:0:(ldlm_lock.c:820:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(CR) ns: mdt-lustre-MDT0000_UUID lock: 00000000bef7c62f/0xa40083ab9bf53166 lrc: 3/1,0 mode: CR/CR res: [0x2000013a1:0x1:0x0].0x0 bits 0x2/0x0 rrc: 4 type: IBT gid 0 flags: 0x50210000000000 nid: local remote: 0x0 expref: -99 pid: 6257 timeout: 0 lvb_type: 0 00010000:00010000:3.0:1776178145.868499:0:6257:0:(ldlm_lock.c:879:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: mdt-lustre-MDT0000_UUID lock: 00000000bef7c62f/0xa40083ab9bf53166 lrc: 2/0,0 mode: CR/CR res: [0x2000013a1:0x1:0x0].0x0 bits 0x2/0x0 rrc: 4 type: IBT gid 0 flags: 0x50210400000000 nid: local remote: 0x0 expref: -99 pid: 6257 timeout: 0 lvb_type: 0 00010000:00010000:3.0:1776178145.868506:0:6257:0:(ldlm_lockd.c:1879:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: mdt-lustre-MDT0000_UUID lock: 00000000bef7c62f/0xa40083ab9bf53166 lrc: 3/0,0 mode: CR/CR res: [0x2000013a1:0x1:0x0].0x0 bits 0x2/0x0 rrc: 4 type: IBT gid 0 flags: 0x40210400000000 nid: local remote: 0x0 expref: -99 pid: 6257 timeout: 0 lvb_type: 0 00010000:00010000:3.0:1776178145.868512:0:6257:0:(ldlm_lockd.c:1893:ldlm_handle_bl_callback()) Lock 00000000bef7c62f already unused, calling callback (0000000079e78348) 00010000:00010000:3.0:1776178145.868517:0:6257:0:(ldlm_request.c:364:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: mdt-lustre-MDT0000_UUID lock: 00000000bef7c62f/0xa40083ab9bf53166 lrc: 3/0,0 mode: CR/CR res: [0x2000013a1:0x1:0x0].0x0 bits 0x2/0x0 rrc: 4 type: IBT gid 0 flags: 0x40210400000000 nid: local remote: 0x0 expref: -99 pid: 6257 timeout: 0 lvb_type: 0 00010000:00010000:3.0:1776178145.868528:0:6257:0:(ldlm_request.c:1257:ldlm_cli_cancel_local()) ### server-side local cancel ns: mdt-lustre-MDT0000_UUID lock: 00000000bef7c62f/0xa40083ab9bf53166 lrc: 4/0,0 mode: CR/CR res: [0x2000013a1:0x1:0x0].0x0 bits 0x2/0x0 rrc: 4 type: IBT gid 0 flags: 0x40218400000000 nid: local remote: 0x0 expref: -99 pid: 6257 timeout: 0 lvb_type: 0 00010000:00010000:3.0:1776178145.868541:0:6257:0:(ldlm_inodebits.c:94:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x2000013a1:0x1:0x0].0x0 (00000000315facbe) 00010000:00010000:3.0:1776178145.868544:0:6257:0:(ldlm_inodebits.c:96:ldlm_reprocess_inodebits_queue()) Hint 2 00010000:00010000:3.0:1776178145.868549:0:6257:0:(ldlm_lockd.c:1903:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: mdt-lustre-MDT0000_UUID lock: 00000000bef7c62f/0xa40083ab9bf53166 lrc: 2/0,0 mode: --/CR res: [0x2000013a1:0x1:0x0].0x0 bits 0x2/0x0 rrc: 4 type: IBT gid 0 flags: 0x44a19400000000 nid: local remote: 0x0 expref: -99 pid: 6257 timeout: 0 lvb_type: 0 00010000:00010000:3.0:1776178145.868556:0:6257:0:(ldlm_lock.c:216:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: mdt-lustre-MDT0000_UUID lock: 00000000bef7c62f/0xa40083ab9bf53166 lrc: 0/0,0 mode: --/CR res: [0x2000013a1:0x1:0x0].0x0 bits 0x2/0x0 rrc: 4 type: IBT gid 0 flags: 0x44a19400000000 nid: local remote: 0x0 expref: -99 pid: 6257 timeout: 0 lvb_type: 0 00010000:00010000:3.0:1776178145.868570:0:6257:0:(ldlm_lock.c:820:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PR) ns: mdt-lustre-MDT0000_UUID lock: 00000000712476de/0xa40083ab9bf5316d lrc: 3/1,0 mode: PR/PR res: [0x2000013a1:0x1:0x0].0xa6b02dea bits 0x2/0x0 rrc: 2 type: IBT gid 0 flags: 0x50210000000000 nid: local remote: 0x0 expref: -99 pid: 6257 timeout: 0 lvb_type: 0 00010000:00010000:3.0:1776178145.868576:0:6257:0:(ldlm_lock.c:879:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: mdt-lustre-MDT0000_UUID lock: 00000000712476de/0xa40083ab9bf5316d lrc: 2/0,0 mode: PR/PR res: [0x2000013a1:0x1:0x0].0xa6b02dea bits 0x2/0x0 rrc: 2 type: IBT gid 0 flags: 0x50210400000000 nid: local remote: 0x0 expref: -99 pid: 6257 timeout: 0 lvb_type: 0 00010000:00010000:3.0:1776178145.868581:0:6257:0:(ldlm_lockd.c:1879:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: mdt-lustre-MDT0000_UUID lock: 00000000712476de/0xa40083ab9bf5316d lrc: 3/0,0 mode: PR/PR res: [0x2000013a1:0x1:0x0].0xa6b02dea bits 0x2/0x0 rrc: 2 type: IBT gid 0 flags: 0x40210400000000 nid: local remote: 0x0 expref: -99 pid: 6257 timeout: 0 lvb_type: 0 00010000:00010000:3.0:1776178145.868586:0:6257:0:(ldlm_lockd.c:1893:ldlm_handle_bl_callback()) Lock 00000000712476de already unused, calling callback (0000000079e78348) 00010000:00010000:3.0:1776178145.868588:0:6257:0:(ldlm_request.c:364:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: mdt-lustre-MDT0000_UUID lock: 00000000712476de/0xa40083ab9bf5316d lrc: 3/0,0 mode: PR/PR res: [0x2000013a1:0x1:0x0].0xa6b02dea bits 0x2/0x0 rrc: 2 type: IBT gid 0 flags: 0x40210400000000 nid: local remote: 0x0 expref: -99 pid: 6257 timeout: 0 lvb_type: 0 00010000:00010000:3.0:1776178145.868594:0:6257:0:(ldlm_request.c:1257:ldlm_cli_cancel_local()) ### server-side local cancel ns: mdt-lustre-MDT0000_UUID lock: 00000000712476de/0xa40083ab9bf5316d lrc: 4/0,0 mode: PR/PR res: [0x2000013a1:0x1:0x0].0xa6b02dea bits 0x2/0x0 rrc: 2 type: IBT gid 0 flags: 0x40218400000000 nid: local remote: 0x0 expref: -99 pid: 6257 timeout: 0 lvb_type: 0 00010000:00010000:3.0:1776178145.868600:0:6257:0:(ldlm_inodebits.c:94:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x2000013a1:0x1:0x0].0xa6b02dea (00000000b86d0927) 00010000:00010000:3.0:1776178145.868602:0:6257:0:(ldlm_inodebits.c:96:ldlm_reprocess_inodebits_queue()) Hint 2 00010000:00010000:3.0:1776178145.868605:0:6257:0:(ldlm_lockd.c:1903:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: mdt-lustre-MDT0000_UUID lock: 00000000712476de/0xa40083ab9bf5316d lrc: 2/0,0 mode: --/PR res: [0x2000013a1:0x1:0x0].0xa6b02dea bits 0x2/0x0 rrc: 2 type: IBT gid 0 flags: 0x44a19400000000 nid: local remote: 0x0 expref: -99 pid: 6257 timeout: 0 lvb_type: 0 00010000:00010000:3.0:1776178145.868610:0:6257:0:(ldlm_lock.c:216:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: mdt-lustre-MDT0000_UUID lock: 00000000712476de/0xa40083ab9bf5316d lrc: 0/0,0 mode: --/PR res: [0x2000013a1:0x1:0x0].0xa6b02dea bits 0x2/0x0 rrc: 2 type: IBT gid 0 flags: 0x44a19400000000 nid: local remote: 0x0 expref: -99 pid: 6257 timeout: 0 lvb_type: 0 00010000:00010000:3.0:1776178145.868643:0:6257:0:(ldlm_lock.c:216:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: mdt-lustre-MDT0000_UUID lock: 000000009f2de1df/0xa40083ab9bf5315f lrc: 0/0,0 mode: --/CR res: [0x2000013a1:0x1:0x0].0x0 bits 0x2/0x0 rrc: 3 type: IBT gid 0 flags: 0x44000000000000 nid: 192.168.204.3@tcp remote: 0x771f68aa1eb27a4d expref: 33 pid: 6257 timeout: 0 lvb_type: 0 00010000:00010000:3.0:1776178145.868655:0:6257:0:(ldlm_lockd.c:1484:ldlm_handle_enqueue0()) ### server-side enqueue handler, sending reply (err=0, rc=0) ns: mdt-lustre-MDT0000_UUID lock: 0000000068a2dd4a/0xa40083ab9bf53174 lrc: 3/0,0 mode: PR/PR res: [0x2000013a1:0x5:0x0].0x0 bits 0x1b/0x0 rrc: 4 type: IBT gid 0 flags: 0x40200000000000 nid: 192.168.204.3@tcp remote: 0x771f68aa1eb27a4d expref: 32 pid: 6257 timeout: 0 lvb_type: 0 00010000:00010000:3.0:1776178145.868661:0:6257:0:(ldlm_inodebits.c:94:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x2000013a1:0x5:0x0].0x0 (000000006b0fcbcb) 00010000:00010000:3.0:1776178145.868663:0:6257:0:(ldlm_inodebits.c:96:ldlm_reprocess_inodebits_queue()) Hint 1b 00010000:00010000:3.0:1776178145.868665:0:6257:0:(ldlm_lockd.c:1565:ldlm_handle_enqueue0()) ### server-side enqueue handler END (lock 0000000068a2dd4a, rc 0) 00000100:00100000:3.0:1776178145.868740:0:6257:0:(service.c:2372:ptlrpc_server_handle_request()) Handled RPC req@00000000977896b4 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_002:b50963bd-8eba-4432-9a54-8888d00e3fa1+32:17033:x1862456979086592:12345-192.168.204.3@tcp:101:checkstat.0 Request processed in 604us (973us total) trans 0 rc 0/0 00000100:00100000:3.0:1776178145.868748:0:6257:0:(nrs_fifo.c:241:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.204.3@tcp, seq: 6508 00000100:00100000:1.0:1776178145.902257:0:3346:0:(events.c:361:request_in_callback()) peer: 12345-192.168.204.3@tcp (source: 12345-192.168.204.3@tcp) 00000100:00100000:3.0:1776178145.902623:0:6257:0:(service.c:2152:ptlrpc_server_handle_req_in()) got req x1862456979086656 00000100:00100000:3.0:1776178145.902691:0:6257:0:(nrs_fifo.c:179:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.204.3@tcp, seq: 6509 00000100:00100000:3.0:1776178145.902698:0:6257:0:(service.c:2324:ptlrpc_server_handle_request()) Handling RPC req@00000000b8039ed3 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_002:b50963bd-8eba-4432-9a54-8888d00e3fa1+32:17034:x1862456979086656:12345-192.168.204.3@tcp:101:checkstat.0 00010000:00010000:3.0:1776178145.902723:0:6257:0:(ldlm_lockd.c:1248:ldlm_handle_enqueue0()) ### server-side enqueue handler START 00010000:00010000:3.0:1776178145.902750:0:6257:0:(ldlm_lockd.c:1318:ldlm_handle_enqueue0()) ### server-side enqueue handler, new lock created ns: mdt-lustre-MDT0000_UUID lock: 000000009f2de1df/0xa40083ab9bf5317b lrc: 2/0,0 mode: --/CR res: [0x2000013a1:0x1:0x0].0x0 bits 0x0/0x0 rrc: 3 type: IBT gid 0 flags: 0x40000000000000 nid: local remote: 0x771f68aa1eb27a54 expref: -99 pid: 6257 timeout: 0 lvb_type: 0 00010000:00010000:3.0:1776178145.902917:0:6257:0:(ldlm_lock.c:766:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(CR) ns: mdt-lustre-MDT0000_UUID lock: 00000000712476de/0xa40083ab9bf53182 lrc: 3/1,0 mode: --/CR res: [0x2000013a1:0x1:0x0].0x0 bits 0x0/0x0 rrc: 4 type: IBT gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 6257 timeout: 0 lvb_type: 0 00010000:00010000:3.0:1776178145.902934:0:6257:0:(ldlm_lock.c:1064:ldlm_granted_list_add_lock()) ### About to add lock: ns: mdt-lustre-MDT0000_UUID lock: 00000000712476de/0xa40083ab9bf53182 lrc: 3/1,0 mode: CR/CR res: [0x2000013a1:0x1:0x0].0x0 bits 0x2/0x0 rrc: 4 type: IBT gid 0 flags: 0x50210000000000 nid: local remote: 0x0 expref: -99 pid: 6257 timeout: 0 lvb_type: 0 00010000:00010000:3.0:1776178145.902949:0:6257:0:(ldlm_request.c:528:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: mdt-lustre-MDT0000_UUID lock: 00000000712476de/0xa40083ab9bf53182 lrc: 3/1,0 mode: CR/CR res: [0x2000013a1:0x1:0x0].0x0 bits 0x2/0x0 rrc: 4 type: IBT gid 0 flags: 0x40210000000000 nid: local remote: 0x0 expref: -99 pid: 6257 timeout: 0 lvb_type: 0 00010000:00010000:3.0:1776178145.902965:0:6257:0:(ldlm_lock.c:766:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PR) ns: mdt-lustre-MDT0000_UUID lock: 00000000bef7c62f/0xa40083ab9bf53189 lrc: 3/1,0 mode: --/PR res: [0x2000013a1:0x1:0x0].0xef09e6ea bits 0x0/0x0 rrc: 2 type: IBT gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 6257 timeout: 0 lvb_type: 0 00010000:00010000:3.0:1776178145.902971:0:6257:0:(ldlm_lock.c:1064:ldlm_granted_list_add_lock()) ### About to add lock: ns: mdt-lustre-MDT0000_UUID lock: 00000000bef7c62f/0xa40083ab9bf53189 lrc: 3/1,0 mode: PR/PR res: [0x2000013a1:0x1:0x0].0xef09e6ea bits 0x2/0x0 rrc: 2 type: IBT gid 0 flags: 0x50210000000000 nid: local remote: 0x0 expref: -99 pid: 6257 timeout: 0 lvb_type: 0 00010000:00010000:3.0:1776178145.902978:0:6257:0:(ldlm_request.c:528:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: mdt-lustre-MDT0000_UUID lock: 00000000bef7c62f/0xa40083ab9bf53189 lrc: 3/1,0 mode: PR/PR res: [0x2000013a1:0x1:0x0].0xef09e6ea bits 0x2/0x0 rrc: 2 type: IBT gid 0 flags: 0x40210000000000 nid: local remote: 0x0 expref: -99 pid: 6257 timeout: 0 lvb_type: 0 00010000:00010000:3.0:1776178145.903055:0:6257:0:(ldlm_lock.c:766:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PR) ns: mdt-lustre-MDT0000_UUID lock: 000000008cbcbaf5/0xa40083ab9bf53190 lrc: 3/1,0 mode: --/PR res: [0x2000013a1:0x6:0x0].0x0 bits 0x0/0x0 rrc: 4 type: IBT gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 6257 timeout: 0 lvb_type: 0 00010000:00010000:3.0:1776178145.903110:0:6257:0:(ldlm_lock.c:1064:ldlm_granted_list_add_lock()) ### About to add lock: ns: mdt-lustre-MDT0000_UUID lock: 000000008cbcbaf5/0xa40083ab9bf53190 lrc: 3/1,0 mode: PR/PR res: [0x2000013a1:0x6:0x0].0x0 bits 0x5b/0x0 rrc: 4 type: IBT gid 0 flags: 0x50210000000000 nid: local remote: 0x0 expref: -99 pid: 6257 timeout: 0 lvb_type: 0 00010000:00010000:3.0:1776178145.903117:0:6257:0:(ldlm_request.c:528:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: mdt-lustre-MDT0000_UUID lock: 000000008cbcbaf5/0xa40083ab9bf53190 lrc: 3/1,0 mode: PR/PR res: [0x2000013a1:0x6:0x0].0x0 bits 0x5b/0x0 rrc: 4 type: IBT gid 0 flags: 0x40210000000000 nid: local remote: 0x0 expref: -99 pid: 6257 timeout: 0 lvb_type: 0 00000004:00200000:3.0:1776178145.903173:0:6257:0:(mdt_handler.c:912:mdt_pack_attr2body()) [0x2000013a1:0x6:0x0]: returning size 0 00000004:00200000:3.0:1776178145.903177:0:6257:0:(mdt_handler.c:1523:mdt_getattr_internal()) dirent count 0 stripe count 1 MDT count 2 00000004:00010000:3.0:1776178145.903198:0:6257:0:(mdt_handler.c:2353:mdt_getattr_name_lock()) ### Returning lock to client ns: mdt-lustre-MDT0000_UUID lock: 000000008cbcbaf5/0xa40083ab9bf53190 lrc: 3/1,0 mode: PR/PR res: [0x2000013a1:0x6:0x0].0x0 bits 0x5b/0x0 rrc: 4 type: IBT gid 0 flags: 0x40210000000000 nid: local remote: 0x0 expref: -99 pid: 6257 timeout: 0 lvb_type: 0 00010000:00010000:3.0:1776178145.903210:0:6257:0:(ldlm_lock.c:1064:ldlm_granted_list_add_lock()) ### About to add lock: ns: mdt-lustre-MDT0000_UUID lock: 000000008cbcbaf5/0xa40083ab9bf53190 lrc: 3/1,0 mode: PR/PR res: [0x2000013a1:0x6:0x0].0x0 bits 0x1b/0x0 rrc: 4 type: IBT gid 0 flags: 0x50210000000000 nid: local remote: 0x0 expref: -99 pid: 6257 timeout: 0 lvb_type: 0 00010000:00010000:3.0:1776178145.903219:0:6257:0:(ldlm_lock.c:820:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(CR) ns: mdt-lustre-MDT0000_UUID lock: 00000000712476de/0xa40083ab9bf53182 lrc: 3/1,0 mode: CR/CR res: [0x2000013a1:0x1:0x0].0x0 bits 0x2/0x0 rrc: 4 type: IBT gid 0 flags: 0x50210000000000 nid: local remote: 0x0 expref: -99 pid: 6257 timeout: 0 lvb_type: 0 00010000:00010000:3.0:1776178145.903226:0:6257:0:(ldlm_lock.c:879:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: mdt-lustre-MDT0000_UUID lock: 00000000712476de/0xa40083ab9bf53182 lrc: 2/0,0 mode: CR/CR res: [0x2000013a1:0x1:0x0].0x0 bits 0x2/0x0 rrc: 4 type: IBT gid 0 flags: 0x50210400000000 nid: local remote: 0x0 expref: -99 pid: 6257 timeout: 0 lvb_type: 0 00010000:00010000:3.0:1776178145.903234:0:6257:0:(ldlm_lockd.c:1879:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: mdt-lustre-MDT0000_UUID lock: 00000000712476de/0xa40083ab9bf53182 lrc: 3/0,0 mode: CR/CR res: [0x2000013a1:0x1:0x0].0x0 bits 0x2/0x0 rrc: 4 type: IBT gid 0 flags: 0x40210400000000 nid: local remote: 0x0 expref: -99 pid: 6257 timeout: 0 lvb_type: 0 00010000:00010000:3.0:1776178145.903241:0:6257:0:(ldlm_lockd.c:1893:ldlm_handle_bl_callback()) Lock 00000000712476de already unused, calling callback (0000000079e78348) 00010000:00010000:3.0:1776178145.903245:0:6257:0:(ldlm_request.c:364:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: mdt-lustre-MDT0000_UUID lock: 00000000712476de/0xa40083ab9bf53182 lrc: 3/0,0 mode: CR/CR res: [0x2000013a1:0x1:0x0].0x0 bits 0x2/0x0 rrc: 4 type: IBT gid 0 flags: 0x40210400000000 nid: local remote: 0x0 expref: -99 pid: 6257 timeout: 0 lvb_type: 0 00010000:00010000:3.0:1776178145.903253:0:6257:0:(ldlm_request.c:1257:ldlm_cli_cancel_local()) ### server-side local cancel ns: mdt-lustre-MDT0000_UUID lock: 00000000712476de/0xa40083ab9bf53182 lrc: 4/0,0 mode: CR/CR res: [0x2000013a1:0x1:0x0].0x0 bits 0x2/0x0 rrc: 4 type: IBT gid 0 flags: 0x40218400000000 nid: local remote: 0x0 expref: -99 pid: 6257 timeout: 0 lvb_type: 0 00010000:00010000:3.0:1776178145.903266:0:6257:0:(ldlm_inodebits.c:94:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x2000013a1:0x1:0x0].0x0 (00000000315facbe) 00010000:00010000:3.0:1776178145.903269:0:6257:0:(ldlm_inodebits.c:96:ldlm_reprocess_inodebits_queue()) Hint 2 00010000:00010000:3.0:1776178145.903275:0:6257:0:(ldlm_lockd.c:1903:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: mdt-lustre-MDT0000_UUID lock: 00000000712476de/0xa40083ab9bf53182 lrc: 2/0,0 mode: --/CR res: [0x2000013a1:0x1:0x0].0x0 bits 0x2/0x0 rrc: 4 type: IBT gid 0 flags: 0x44a19400000000 nid: local remote: 0x0 expref: -99 pid: 6257 timeout: 0 lvb_type: 0 00010000:00010000:3.0:1776178145.903281:0:6257:0:(ldlm_lock.c:216:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: mdt-lustre-MDT0000_UUID lock: 00000000712476de/0xa40083ab9bf53182 lrc: 0/0,0 mode: --/CR res: [0x2000013a1:0x1:0x0].0x0 bits 0x2/0x0 rrc: 4 type: IBT gid 0 flags: 0x44a19400000000 nid: local remote: 0x0 expref: -99 pid: 6257 timeout: 0 lvb_type: 0 00010000:00010000:3.0:1776178145.903295:0:6257:0:(ldlm_lock.c:820:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PR) ns: mdt-lustre-MDT0000_UUID lock: 00000000bef7c62f/0xa40083ab9bf53189 lrc: 3/1,0 mode: PR/PR res: [0x2000013a1:0x1:0x0].0xef09e6ea bits 0x2/0x0 rrc: 2 type: IBT gid 0 flags: 0x50210000000000 nid: local remote: 0x0 expref: -99 pid: 6257 timeout: 0 lvb_type: 0 00010000:00010000:3.0:1776178145.903302:0:6257:0:(ldlm_lock.c:879:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: mdt-lustre-MDT0000_UUID lock: 00000000bef7c62f/0xa40083ab9bf53189 lrc: 2/0,0 mode: PR/PR res: [0x2000013a1:0x1:0x0].0xef09e6ea bits 0x2/0x0 rrc: 2 type: IBT gid 0 flags: 0x50210400000000 nid: local remote: 0x0 expref: -99 pid: 6257 timeout: 0 lvb_type: 0 00010000:00010000:3.0:1776178145.903307:0:6257:0:(ldlm_lockd.c:1879:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: mdt-lustre-MDT0000_UUID lock: 00000000bef7c62f/0xa40083ab9bf53189 lrc: 3/0,0 mode: PR/PR res: [0x2000013a1:0x1:0x0].0xef09e6ea bits 0x2/0x0 rrc: 2 type: IBT gid 0 flags: 0x40210400000000 nid: local remote: 0x0 expref: -99 pid: 6257 timeout: 0 lvb_type: 0 00010000:00010000:3.0:1776178145.903317:0:6257:0:(ldlm_lockd.c:1893:ldlm_handle_bl_callback()) Lock 00000000bef7c62f already unused, calling callback (0000000079e78348) 00010000:00010000:3.0:1776178145.903319:0:6257:0:(ldlm_request.c:364:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: mdt-lustre-MDT0000_UUID lock: 00000000bef7c62f/0xa40083ab9bf53189 lrc: 3/0,0 mode: PR/PR res: [0x2000013a1:0x1:0x0].0xef09e6ea bits 0x2/0x0 rrc: 2 type: IBT gid 0 flags: 0x40210400000000 nid: local remote: 0x0 expref: -99 pid: 6257 timeout: 0 lvb_type: 0 00010000:00010000:3.0:1776178145.903324:0:6257:0:(ldlm_request.c:1257:ldlm_cli_cancel_local()) ### server-side local cancel ns: mdt-lustre-MDT0000_UUID lock: 00000000bef7c62f/0xa40083ab9bf53189 lrc: 4/0,0 mode: PR/PR res: [0x2000013a1:0x1:0x0].0xef09e6ea bits 0x2/0x0 rrc: 2 type: IBT gid 0 flags: 0x40218400000000 nid: local remote: 0x0 expref: -99 pid: 6257 timeout: 0 lvb_type: 0 00010000:00010000:3.0:1776178145.903330:0:6257:0:(ldlm_inodebits.c:94:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x2000013a1:0x1:0x0].0xef09e6ea (00000000b86d0927) 00010000:00010000:3.0:1776178145.903332:0:6257:0:(ldlm_inodebits.c:96:ldlm_reprocess_inodebits_queue()) Hint 2 00010000:00010000:3.0:1776178145.903334:0:6257:0:(ldlm_lockd.c:1903:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: mdt-lustre-MDT0000_UUID lock: 00000000bef7c62f/0xa40083ab9bf53189 lrc: 2/0,0 mode: --/PR res: [0x2000013a1:0x1:0x0].0xef09e6ea bits 0x2/0x0 rrc: 2 type: IBT gid 0 flags: 0x44a19400000000 nid: local remote: 0x0 expref: -99 pid: 6257 timeout: 0 lvb_type: 0 00010000:00010000:3.0:1776178145.903339:0:6257:0:(ldlm_lock.c:216:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: mdt-lustre-MDT0000_UUID lock: 00000000bef7c62f/0xa40083ab9bf53189 lrc: 0/0,0 mode: --/PR res: [0x2000013a1:0x1:0x0].0xef09e6ea bits 0x2/0x0 rrc: 2 type: IBT gid 0 flags: 0x44a19400000000 nid: local remote: 0x0 expref: -99 pid: 6257 timeout: 0 lvb_type: 0 00010000:00010000:3.0:1776178145.903374:0:6257:0:(ldlm_lock.c:216:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: mdt-lustre-MDT0000_UUID lock: 000000009f2de1df/0xa40083ab9bf5317b lrc: 0/0,0 mode: --/CR res: [0x2000013a1:0x1:0x0].0x0 bits 0x2/0x0 rrc: 3 type: IBT gid 0 flags: 0x44000000000000 nid: 192.168.204.3@tcp remote: 0x771f68aa1eb27a54 expref: 34 pid: 6257 timeout: 0 lvb_type: 0 00010000:00010000:3.0:1776178145.903387:0:6257:0:(ldlm_lockd.c:1484:ldlm_handle_enqueue0()) ### server-side enqueue handler, sending reply (err=0, rc=0) ns: mdt-lustre-MDT0000_UUID lock: 000000008cbcbaf5/0xa40083ab9bf53190 lrc: 3/0,0 mode: PR/PR res: [0x2000013a1:0x6:0x0].0x0 bits 0x1b/0x0 rrc: 4 type: IBT gid 0 flags: 0x40200000000000 nid: 192.168.204.3@tcp remote: 0x771f68aa1eb27a54 expref: 33 pid: 6257 timeout: 0 lvb_type: 0 00010000:00010000:3.0:1776178145.903393:0:6257:0:(ldlm_inodebits.c:94:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x2000013a1:0x6:0x0].0x0 (0000000019a0327e) 00010000:00010000:3.0:1776178145.903395:0:6257:0:(ldlm_inodebits.c:96:ldlm_reprocess_inodebits_queue()) Hint 1b 00010000:00010000:3.0:1776178145.903397:0:6257:0:(ldlm_lockd.c:1565:ldlm_handle_enqueue0()) ### server-side enqueue handler END (lock 000000008cbcbaf5, rc 0) 00000100:00100000:3.0:1776178145.903470:0:6257:0:(service.c:2372:ptlrpc_server_handle_request()) Handled RPC req@00000000b8039ed3 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_002:b50963bd-8eba-4432-9a54-8888d00e3fa1+33:17034:x1862456979086656:12345-192.168.204.3@tcp:101:checkstat.0 Request processed in 771us (1223us total) trans 0 rc 0/0 00000100:00100000:3.0:1776178145.903479:0:6257:0:(nrs_fifo.c:241:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.204.3@tcp, seq: 6509 00000100:00100000:1.0:1776178145.943752:0:3348:0:(events.c:361:request_in_callback()) peer: 12345-192.168.204.3@tcp (source: 12345-192.168.204.3@tcp) 00000100:00100000:0.0:1776178145.945174:0:8250:0:(service.c:2152:ptlrpc_server_handle_req_in()) got req x1862456979086720 00000100:00100000:0.0:1776178145.945271:0:8250:0:(nrs_fifo.c:179:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.204.3@tcp, seq: 6510 00000100:00100000:0.0:1776178145.945279:0:8250:0:(service.c:2324:ptlrpc_server_handle_request()) Handling RPC req@00000000c0414cc7 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_003:b50963bd-8eba-4432-9a54-8888d00e3fa1+33:17035:x1862456979086720:12345-192.168.204.3@tcp:101:checkstat.0 00010000:00010000:0.0:1776178145.945304:0:8250:0:(ldlm_lockd.c:1248:ldlm_handle_enqueue0()) ### server-side enqueue handler START 00010000:00010000:0.0:1776178145.945331:0:8250:0:(ldlm_lockd.c:1318:ldlm_handle_enqueue0()) ### server-side enqueue handler, new lock created ns: mdt-lustre-MDT0000_UUID lock: 0000000068850c85/0xa40083ab9bf53197 lrc: 2/0,0 mode: --/CR res: [0x2000013a1:0x1:0x0].0x0 bits 0x0/0x0 rrc: 3 type: IBT gid 0 flags: 0x40000000000000 nid: local remote: 0x771f68aa1eb27a5b expref: -99 pid: 8250 timeout: 0 lvb_type: 0 00010000:00010000:0.0:1776178145.945413:0:8250:0:(ldlm_lock.c:766:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(CR) ns: mdt-lustre-MDT0000_UUID lock: 00000000782e253e/0xa40083ab9bf5319e lrc: 3/1,0 mode: --/CR res: [0x2000013a1:0x1:0x0].0x0 bits 0x0/0x0 rrc: 4 type: IBT gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 8250 timeout: 0 lvb_type: 0 00010000:00010000:0.0:1776178145.945430:0:8250:0:(ldlm_lock.c:1064:ldlm_granted_list_add_lock()) ### About to add lock: ns: mdt-lustre-MDT0000_UUID lock: 00000000782e253e/0xa40083ab9bf5319e lrc: 3/1,0 mode: CR/CR res: [0x2000013a1:0x1:0x0].0x0 bits 0x2/0x0 rrc: 4 type: IBT gid 0 flags: 0x50210000000000 nid: local remote: 0x0 expref: -99 pid: 8250 timeout: 0 lvb_type: 0 00010000:00010000:0.0:1776178145.945444:0:8250:0:(ldlm_request.c:528:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: mdt-lustre-MDT0000_UUID lock: 00000000782e253e/0xa40083ab9bf5319e lrc: 3/1,0 mode: CR/CR res: [0x2000013a1:0x1:0x0].0x0 bits 0x2/0x0 rrc: 4 type: IBT gid 0 flags: 0x40210000000000 nid: local remote: 0x0 expref: -99 pid: 8250 timeout: 0 lvb_type: 0 00010000:00010000:0.0:1776178145.945459:0:8250:0:(ldlm_lock.c:766:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PR) ns: mdt-lustre-MDT0000_UUID lock: 000000000e7a37e0/0xa40083ab9bf531a5 lrc: 3/1,0 mode: --/PR res: [0x2000013a1:0x1:0x0].0x37639fea bits 0x0/0x0 rrc: 2 type: IBT gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 8250 timeout: 0 lvb_type: 0 00010000:00010000:0.0:1776178145.945466:0:8250:0:(ldlm_lock.c:1064:ldlm_granted_list_add_lock()) ### About to add lock: ns: mdt-lustre-MDT0000_UUID lock: 000000000e7a37e0/0xa40083ab9bf531a5 lrc: 3/1,0 mode: PR/PR res: [0x2000013a1:0x1:0x0].0x37639fea bits 0x2/0x0 rrc: 2 type: IBT gid 0 flags: 0x50210000000000 nid: local remote: 0x0 expref: -99 pid: 8250 timeout: 0 lvb_type: 0 00010000:00010000:0.0:1776178145.945477:0:8250:0:(ldlm_request.c:528:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: mdt-lustre-MDT0000_UUID lock: 000000000e7a37e0/0xa40083ab9bf531a5 lrc: 3/1,0 mode: PR/PR res: [0x2000013a1:0x1:0x0].0x37639fea bits 0x2/0x0 rrc: 2 type: IBT gid 0 flags: 0x40210000000000 nid: local remote: 0x0 expref: -99 pid: 8250 timeout: 0 lvb_type: 0 00010000:00010000:0.0:1776178145.945570:0:8250:0:(ldlm_lock.c:766:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PR) ns: mdt-lustre-MDT0000_UUID lock: 000000000b903578/0xa40083ab9bf531ac lrc: 3/1,0 mode: --/PR res: [0x2000013a1:0x7:0x0].0x0 bits 0x0/0x0 rrc: 4 type: IBT gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 8250 timeout: 0 lvb_type: 0 00010000:00010000:0.0:1776178145.945579:0:8250:0:(ldlm_lock.c:1064:ldlm_granted_list_add_lock()) ### About to add lock: ns: mdt-lustre-MDT0000_UUID lock: 000000000b903578/0xa40083ab9bf531ac lrc: 3/1,0 mode: PR/PR res: [0x2000013a1:0x7:0x0].0x0 bits 0x5b/0x0 rrc: 4 type: IBT gid 0 flags: 0x50210000000000 nid: local remote: 0x0 expref: -99 pid: 8250 timeout: 0 lvb_type: 0 00010000:00010000:0.0:1776178145.945585:0:8250:0:(ldlm_request.c:528:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: mdt-lustre-MDT0000_UUID lock: 000000000b903578/0xa40083ab9bf531ac lrc: 3/1,0 mode: PR/PR res: [0x2000013a1:0x7:0x0].0x0 bits 0x5b/0x0 rrc: 4 type: IBT gid 0 flags: 0x40210000000000 nid: local remote: 0x0 expref: -99 pid: 8250 timeout: 0 lvb_type: 0 00000004:00200000:0.0:1776178145.945651:0:8250:0:(mdt_handler.c:912:mdt_pack_attr2body()) [0x2000013a1:0x7:0x0]: returning size 0 00000004:00200000:0.0:1776178145.945656:0:8250:0:(mdt_handler.c:1523:mdt_getattr_internal()) dirent count 0 stripe count 1 MDT count 2 00000004:00010000:0.0:1776178145.945681:0:8250:0:(mdt_handler.c:2353:mdt_getattr_name_lock()) ### Returning lock to client ns: mdt-lustre-MDT0000_UUID lock: 000000000b903578/0xa40083ab9bf531ac lrc: 3/1,0 mode: PR/PR res: [0x2000013a1:0x7:0x0].0x0 bits 0x5b/0x0 rrc: 4 type: IBT gid 0 flags: 0x40210000000000 nid: local remote: 0x0 expref: -99 pid: 8250 timeout: 0 lvb_type: 0 00010000:00010000:0.0:1776178145.945693:0:8250:0:(ldlm_lock.c:1064:ldlm_granted_list_add_lock()) ### About to add lock: ns: mdt-lustre-MDT0000_UUID lock: 000000000b903578/0xa40083ab9bf531ac lrc: 3/1,0 mode: PR/PR res: [0x2000013a1:0x7:0x0].0x0 bits 0x1b/0x0 rrc: 4 type: IBT gid 0 flags: 0x50210000000000 nid: local remote: 0x0 expref: -99 pid: 8250 timeout: 0 lvb_type: 0 00010000:00010000:0.0:1776178145.945702:0:8250:0:(ldlm_lock.c:820:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(CR) ns: mdt-lustre-MDT0000_UUID lock: 00000000782e253e/0xa40083ab9bf5319e lrc: 3/1,0 mode: CR/CR res: [0x2000013a1:0x1:0x0].0x0 bits 0x2/0x0 rrc: 4 type: IBT gid 0 flags: 0x50210000000000 nid: local remote: 0x0 expref: -99 pid: 8250 timeout: 0 lvb_type: 0 00010000:00010000:0.0:1776178145.945708:0:8250:0:(ldlm_lock.c:879:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: mdt-lustre-MDT0000_UUID lock: 00000000782e253e/0xa40083ab9bf5319e lrc: 2/0,0 mode: CR/CR res: [0x2000013a1:0x1:0x0].0x0 bits 0x2/0x0 rrc: 4 type: IBT gid 0 flags: 0x50210400000000 nid: local remote: 0x0 expref: -99 pid: 8250 timeout: 0 lvb_type: 0 00010000:00010000:0.0:1776178145.945714:0:8250:0:(ldlm_lockd.c:1879:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: mdt-lustre-MDT0000_UUID lock: 00000000782e253e/0xa40083ab9bf5319e lrc: 3/0,0 mode: CR/CR res: [0x2000013a1:0x1:0x0].0x0 bits 0x2/0x0 rrc: 4 type: IBT gid 0 flags: 0x40210400000000 nid: local remote: 0x0 expref: -99 pid: 8250 timeout: 0 lvb_type: 0 00010000:00010000:0.0:1776178145.945721:0:8250:0:(ldlm_lockd.c:1893:ldlm_handle_bl_callback()) Lock 00000000782e253e already unused, calling callback (0000000079e78348) 00010000:00010000:0.0:1776178145.945725:0:8250:0:(ldlm_request.c:364:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: mdt-lustre-MDT0000_UUID lock: 00000000782e253e/0xa40083ab9bf5319e lrc: 3/0,0 mode: CR/CR res: [0x2000013a1:0x1:0x0].0x0 bits 0x2/0x0 rrc: 4 type: IBT gid 0 flags: 0x40210400000000 nid: local remote: 0x0 expref: -99 pid: 8250 timeout: 0 lvb_type: 0 00010000:00010000:0.0:1776178145.945733:0:8250:0:(ldlm_request.c:1257:ldlm_cli_cancel_local()) ### server-side local cancel ns: mdt-lustre-MDT0000_UUID lock: 00000000782e253e/0xa40083ab9bf5319e lrc: 4/0,0 mode: CR/CR res: [0x2000013a1:0x1:0x0].0x0 bits 0x2/0x0 rrc: 4 type: IBT gid 0 flags: 0x40218400000000 nid: local remote: 0x0 expref: -99 pid: 8250 timeout: 0 lvb_type: 0 00010000:00010000:0.0:1776178145.945745:0:8250:0:(ldlm_inodebits.c:94:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x2000013a1:0x1:0x0].0x0 (00000000315facbe) 00010000:00010000:0.0:1776178145.945748:0:8250:0:(ldlm_inodebits.c:96:ldlm_reprocess_inodebits_queue()) Hint 2 00010000:00010000:0.0:1776178145.945752:0:8250:0:(ldlm_lockd.c:1903:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: mdt-lustre-MDT0000_UUID lock: 00000000782e253e/0xa40083ab9bf5319e lrc: 2/0,0 mode: --/CR res: [0x2000013a1:0x1:0x0].0x0 bits 0x2/0x0 rrc: 4 type: IBT gid 0 flags: 0x44a19400000000 nid: local remote: 0x0 expref: -99 pid: 8250 timeout: 0 lvb_type: 0 00010000:00010000:0.0:1776178145.945757:0:8250:0:(ldlm_lock.c:216:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: mdt-lustre-MDT0000_UUID lock: 00000000782e253e/0xa40083ab9bf5319e lrc: 0/0,0 mode: --/CR res: [0x2000013a1:0x1:0x0].0x0 bits 0x2/0x0 rrc: 4 type: IBT gid 0 flags: 0x44a19400000000 nid: local remote: 0x0 expref: -99 pid: 8250 timeout: 0 lvb_type: 0 00010000:00010000:0.0:1776178145.945772:0:8250:0:(ldlm_lock.c:820:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PR) ns: mdt-lustre-MDT0000_UUID lock: 000000000e7a37e0/0xa40083ab9bf531a5 lrc: 3/1,0 mode: PR/PR res: [0x2000013a1:0x1:0x0].0x37639fea bits 0x2/0x0 rrc: 2 type: IBT gid 0 flags: 0x50210000000000 nid: local remote: 0x0 expref: -99 pid: 8250 timeout: 0 lvb_type: 0 00010000:00010000:0.0:1776178145.945778:0:8250:0:(ldlm_lock.c:879:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: mdt-lustre-MDT0000_UUID lock: 000000000e7a37e0/0xa40083ab9bf531a5 lrc: 2/0,0 mode: PR/PR res: [0x2000013a1:0x1:0x0].0x37639fea bits 0x2/0x0 rrc: 2 type: IBT gid 0 flags: 0x50210400000000 nid: local remote: 0x0 expref: -99 pid: 8250 timeout: 0 lvb_type: 0 00010000:00010000:0.0:1776178145.945783:0:8250:0:(ldlm_lockd.c:1879:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: mdt-lustre-MDT0000_UUID lock: 000000000e7a37e0/0xa40083ab9bf531a5 lrc: 3/0,0 mode: PR/PR res: [0x2000013a1:0x1:0x0].0x37639fea bits 0x2/0x0 rrc: 2 type: IBT gid 0 flags: 0x40210400000000 nid: local remote: 0x0 expref: -99 pid: 8250 timeout: 0 lvb_type: 0 00010000:00010000:0.0:1776178145.945788:0:8250:0:(ldlm_lockd.c:1893:ldlm_handle_bl_callback()) Lock 000000000e7a37e0 already unused, calling callback (0000000079e78348) 00010000:00010000:0.0:1776178145.945790:0:8250:0:(ldlm_request.c:364:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: mdt-lustre-MDT0000_UUID lock: 000000000e7a37e0/0xa40083ab9bf531a5 lrc: 3/0,0 mode: PR/PR res: [0x2000013a1:0x1:0x0].0x37639fea bits 0x2/0x0 rrc: 2 type: IBT gid 0 flags: 0x40210400000000 nid: local remote: 0x0 expref: -99 pid: 8250 timeout: 0 lvb_type: 0 00010000:00010000:0.0:1776178145.945796:0:8250:0:(ldlm_request.c:1257:ldlm_cli_cancel_local()) ### server-side local cancel ns: mdt-lustre-MDT0000_UUID lock: 000000000e7a37e0/0xa40083ab9bf531a5 lrc: 4/0,0 mode: PR/PR res: [0x2000013a1:0x1:0x0].0x37639fea bits 0x2/0x0 rrc: 2 type: IBT gid 0 flags: 0x40218400000000 nid: local remote: 0x0 expref: -99 pid: 8250 timeout: 0 lvb_type: 0 00010000:00010000:0.0:1776178145.945802:0:8250:0:(ldlm_inodebits.c:94:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x2000013a1:0x1:0x0].0x37639fea (00000000eb684e0b) 00010000:00010000:0.0:1776178145.945805:0:8250:0:(ldlm_inodebits.c:96:ldlm_reprocess_inodebits_queue()) Hint 2 00010000:00010000:0.0:1776178145.945816:0:8250:0:(ldlm_lockd.c:1903:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: mdt-lustre-MDT0000_UUID lock: 000000000e7a37e0/0xa40083ab9bf531a5 lrc: 2/0,0 mode: --/PR res: [0x2000013a1:0x1:0x0].0x37639fea bits 0x2/0x0 rrc: 2 type: IBT gid 0 flags: 0x44a19400000000 nid: local remote: 0x0 expref: -99 pid: 8250 timeout: 0 lvb_type: 0 00010000:00010000:0.0:1776178145.945822:0:8250:0:(ldlm_lock.c:216:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: mdt-lustre-MDT0000_UUID lock: 000000000e7a37e0/0xa40083ab9bf531a5 lrc: 0/0,0 mode: --/PR res: [0x2000013a1:0x1:0x0].0x37639fea bits 0x2/0x0 rrc: 2 type: IBT gid 0 flags: 0x44a19400000000 nid: local remote: 0x0 expref: -99 pid: 8250 timeout: 0 lvb_type: 0 00010000:00010000:0.0:1776178145.945851:0:8250:0:(ldlm_lock.c:216:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: mdt-lustre-MDT0000_UUID lock: 0000000068850c85/0xa40083ab9bf53197 lrc: 0/0,0 mode: --/CR res: [0x2000013a1:0x1:0x0].0x0 bits 0x2/0x0 rrc: 3 type: IBT gid 0 flags: 0x44000000000000 nid: 192.168.204.3@tcp remote: 0x771f68aa1eb27a5b expref: 35 pid: 8250 timeout: 0 lvb_type: 0 00010000:00010000:0.0:1776178145.945863:0:8250:0:(ldlm_lockd.c:1484:ldlm_handle_enqueue0()) ### server-side enqueue handler, sending reply (err=0, rc=0) ns: mdt-lustre-MDT0000_UUID lock: 000000000b903578/0xa40083ab9bf531ac lrc: 3/0,0 mode: PR/PR res: [0x2000013a1:0x7:0x0].0x0 bits 0x1b/0x0 rrc: 4 type: IBT gid 0 flags: 0x40200000000000 nid: 192.168.204.3@tcp remote: 0x771f68aa1eb27a5b expref: 34 pid: 8250 timeout: 0 lvb_type: 0 00010000:00010000:0.0:1776178145.945869:0:8250:0:(ldlm_inodebits.c:94:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x2000013a1:0x7:0x0].0x0 (00000000aeb25ebf) 00010000:00010000:0.0:1776178145.945871:0:8250:0:(ldlm_inodebits.c:96:ldlm_reprocess_inodebits_queue()) Hint 1b 00010000:00010000:0.0:1776178145.945873:0:8250:0:(ldlm_lockd.c:1565:ldlm_handle_enqueue0()) ### server-side enqueue handler END (lock 000000000b903578, rc 0) 00000100:00100000:0.0:1776178145.945944:0:8250:0:(service.c:2372:ptlrpc_server_handle_request()) Handled RPC req@00000000c0414cc7 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_003:b50963bd-8eba-4432-9a54-8888d00e3fa1+34:17035:x1862456979086720:12345-192.168.204.3@tcp:101:checkstat.0 Request processed in 664us (2208us total) trans 0 rc 0/0 00000100:00100000:0.0:1776178145.945952:0:8250:0:(nrs_fifo.c:241:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.204.3@tcp, seq: 6510 00000100:00100000:1.0:1776178145.963822:0:3347:0:(events.c:361:request_in_callback()) peer: 12345-192.168.204.3@tcp (source: 12345-192.168.204.3@tcp) 00000100:00100000:3.0:1776178145.967178:0:6257:0:(service.c:2152:ptlrpc_server_handle_req_in()) got req x1862456979086784 00000100:00100000:3.0:1776178145.968309:0:6257:0:(nrs_fifo.c:179:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.204.3@tcp, seq: 6511 00000100:00100000:3.0:1776178145.968320:0:6257:0:(service.c:2324:ptlrpc_server_handle_request()) Handling RPC req@00000000bfb62f7a pname:cluuid+ref:pid:xid:nid:opc:job mdt00_002:b50963bd-8eba-4432-9a54-8888d00e3fa1+34:17036:x1862456979086784:12345-192.168.204.3@tcp:101:checkstat.0 00010000:00010000:3.0:1776178145.968350:0:6257:0:(ldlm_lockd.c:1248:ldlm_handle_enqueue0()) ### server-side enqueue handler START 00010000:00010000:3.0:1776178145.968375:0:6257:0:(ldlm_lockd.c:1318:ldlm_handle_enqueue0()) ### server-side enqueue handler, new lock created ns: mdt-lustre-MDT0000_UUID lock: 000000009f2de1df/0xa40083ab9bf531b3 lrc: 2/0,0 mode: --/CR res: [0x2000013a1:0x1:0x0].0x0 bits 0x0/0x0 rrc: 3 type: IBT gid 0 flags: 0x40000000000000 nid: local remote: 0x771f68aa1eb27a62 expref: -99 pid: 6257 timeout: 0 lvb_type: 0 00010000:00010000:3.0:1776178145.968457:0:6257:0:(ldlm_lock.c:766:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(CR) ns: mdt-lustre-MDT0000_UUID lock: 00000000bef7c62f/0xa40083ab9bf531ba lrc: 3/1,0 mode: --/CR res: [0x2000013a1:0x1:0x0].0x0 bits 0x0/0x0 rrc: 4 type: IBT gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 6257 timeout: 0 lvb_type: 0 00010000:00010000:3.0:1776178145.968475:0:6257:0:(ldlm_lock.c:1064:ldlm_granted_list_add_lock()) ### About to add lock: ns: mdt-lustre-MDT0000_UUID lock: 00000000bef7c62f/0xa40083ab9bf531ba lrc: 3/1,0 mode: CR/CR res: [0x2000013a1:0x1:0x0].0x0 bits 0x2/0x0 rrc: 4 type: IBT gid 0 flags: 0x50210000000000 nid: local remote: 0x0 expref: -99 pid: 6257 timeout: 0 lvb_type: 0 00010000:00010000:3.0:1776178145.968488:0:6257:0:(ldlm_request.c:528:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: mdt-lustre-MDT0000_UUID lock: 00000000bef7c62f/0xa40083ab9bf531ba lrc: 3/1,0 mode: CR/CR res: [0x2000013a1:0x1:0x0].0x0 bits 0x2/0x0 rrc: 4 type: IBT gid 0 flags: 0x40210000000000 nid: local remote: 0x0 expref: -99 pid: 6257 timeout: 0 lvb_type: 0 00010000:00010000:3.0:1776178145.968505:0:6257:0:(ldlm_lock.c:766:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PR) ns: mdt-lustre-MDT0000_UUID lock: 00000000712476de/0xa40083ab9bf531c1 lrc: 3/1,0 mode: --/PR res: [0x2000013a1:0x1:0x0].0x7fbd58ea bits 0x0/0x0 rrc: 2 type: IBT gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 6257 timeout: 0 lvb_type: 0 00010000:00010000:3.0:1776178145.968513:0:6257:0:(ldlm_lock.c:1064:ldlm_granted_list_add_lock()) ### About to add lock: ns: mdt-lustre-MDT0000_UUID lock: 00000000712476de/0xa40083ab9bf531c1 lrc: 3/1,0 mode: PR/PR res: [0x2000013a1:0x1:0x0].0x7fbd58ea bits 0x2/0x0 rrc: 2 type: IBT gid 0 flags: 0x50210000000000 nid: local remote: 0x0 expref: -99 pid: 6257 timeout: 0 lvb_type: 0 00010000:00010000:3.0:1776178145.968519:0:6257:0:(ldlm_request.c:528:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: mdt-lustre-MDT0000_UUID lock: 00000000712476de/0xa40083ab9bf531c1 lrc: 3/1,0 mode: PR/PR res: [0x2000013a1:0x1:0x0].0x7fbd58ea bits 0x2/0x0 rrc: 2 type: IBT gid 0 flags: 0x40210000000000 nid: local remote: 0x0 expref: -99 pid: 6257 timeout: 0 lvb_type: 0 00010000:00010000:3.0:1776178145.968614:0:6257:0:(ldlm_lock.c:766:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PR) ns: mdt-lustre-MDT0000_UUID lock: 000000005d93f094/0xa40083ab9bf531c8 lrc: 3/1,0 mode: --/PR res: [0x2000013a1:0x8:0x0].0x0 bits 0x0/0x0 rrc: 4 type: IBT gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 6257 timeout: 0 lvb_type: 0 00010000:00010000:3.0:1776178145.968637:0:6257:0:(ldlm_lock.c:1064:ldlm_granted_list_add_lock()) ### About to add lock: ns: mdt-lustre-MDT0000_UUID lock: 000000005d93f094/0xa40083ab9bf531c8 lrc: 3/1,0 mode: PR/PR res: [0x2000013a1:0x8:0x0].0x0 bits 0x5b/0x0 rrc: 4 type: IBT gid 0 flags: 0x50210000000000 nid: local remote: 0x0 expref: -99 pid: 6257 timeout: 0 lvb_type: 0 00010000:00010000:3.0:1776178145.968644:0:6257:0:(ldlm_request.c:528:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: mdt-lustre-MDT0000_UUID lock: 000000005d93f094/0xa40083ab9bf531c8 lrc: 3/1,0 mode: PR/PR res: [0x2000013a1:0x8:0x0].0x0 bits 0x5b/0x0 rrc: 4 type: IBT gid 0 flags: 0x40210000000000 nid: local remote: 0x0 expref: -99 pid: 6257 timeout: 0 lvb_type: 0 00000004:00200000:3.0:1776178145.968701:0:6257:0:(mdt_handler.c:912:mdt_pack_attr2body()) [0x2000013a1:0x8:0x0]: returning size 0 00000004:00200000:3.0:1776178145.968706:0:6257:0:(mdt_handler.c:1523:mdt_getattr_internal()) dirent count 0 stripe count 1 MDT count 2 00000004:00010000:3.0:1776178145.968731:0:6257:0:(mdt_handler.c:2353:mdt_getattr_name_lock()) ### Returning lock to client ns: mdt-lustre-MDT0000_UUID lock: 000000005d93f094/0xa40083ab9bf531c8 lrc: 3/1,0 mode: PR/PR res: [0x2000013a1:0x8:0x0].0x0 bits 0x5b/0x0 rrc: 4 type: IBT gid 0 flags: 0x40210000000000 nid: local remote: 0x0 expref: -99 pid: 6257 timeout: 0 lvb_type: 0 00010000:00010000:3.0:1776178145.968743:0:6257:0:(ldlm_lock.c:1064:ldlm_granted_list_add_lock()) ### About to add lock: ns: mdt-lustre-MDT0000_UUID lock: 000000005d93f094/0xa40083ab9bf531c8 lrc: 3/1,0 mode: PR/PR res: [0x2000013a1:0x8:0x0].0x0 bits 0x1b/0x0 rrc: 4 type: IBT gid 0 flags: 0x50210000000000 nid: local remote: 0x0 expref: -99 pid: 6257 timeout: 0 lvb_type: 0 00010000:00010000:3.0:1776178145.968750:0:6257:0:(ldlm_lock.c:820:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(CR) ns: mdt-lustre-MDT0000_UUID lock: 00000000bef7c62f/0xa40083ab9bf531ba lrc: 3/1,0 mode: CR/CR res: [0x2000013a1:0x1:0x0].0x0 bits 0x2/0x0 rrc: 4 type: IBT gid 0 flags: 0x50210000000000 nid: local remote: 0x0 expref: -99 pid: 6257 timeout: 0 lvb_type: 0 00010000:00010000:3.0:1776178145.968756:0:6257:0:(ldlm_lock.c:879:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: mdt-lustre-MDT0000_UUID lock: 00000000bef7c62f/0xa40083ab9bf531ba lrc: 2/0,0 mode: CR/CR res: [0x2000013a1:0x1:0x0].0x0 bits 0x2/0x0 rrc: 4 type: IBT gid 0 flags: 0x50210400000000 nid: local remote: 0x0 expref: -99 pid: 6257 timeout: 0 lvb_type: 0 00010000:00010000:3.0:1776178145.968763:0:6257:0:(ldlm_lockd.c:1879:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: mdt-lustre-MDT0000_UUID lock: 00000000bef7c62f/0xa40083ab9bf531ba lrc: 3/0,0 mode: CR/CR res: [0x2000013a1:0x1:0x0].0x0 bits 0x2/0x0 rrc: 4 type: IBT gid 0 flags: 0x40210400000000 nid: local remote: 0x0 expref: -99 pid: 6257 timeout: 0 lvb_type: 0 00010000:00010000:3.0:1776178145.968769:0:6257:0:(ldlm_lockd.c:1893:ldlm_handle_bl_callback()) Lock 00000000bef7c62f already unused, calling callback (0000000079e78348) 00010000:00010000:3.0:1776178145.968772:0:6257:0:(ldlm_request.c:364:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: mdt-lustre-MDT0000_UUID lock: 00000000bef7c62f/0xa40083ab9bf531ba lrc: 3/0,0 mode: CR/CR res: [0x2000013a1:0x1:0x0].0x0 bits 0x2/0x0 rrc: 4 type: IBT gid 0 flags: 0x40210400000000 nid: local remote: 0x0 expref: -99 pid: 6257 timeout: 0 lvb_type: 0 00010000:00010000:3.0:1776178145.968779:0:6257:0:(ldlm_request.c:1257:ldlm_cli_cancel_local()) ### server-side local cancel ns: mdt-lustre-MDT0000_UUID lock: 00000000bef7c62f/0xa40083ab9bf531ba lrc: 4/0,0 mode: CR/CR res: [0x2000013a1:0x1:0x0].0x0 bits 0x2/0x0 rrc: 4 type: IBT gid 0 flags: 0x40218400000000 nid: local remote: 0x0 expref: -99 pid: 6257 timeout: 0 lvb_type: 0 00010000:00010000:3.0:1776178145.968790:0:6257:0:(ldlm_inodebits.c:94:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x2000013a1:0x1:0x0].0x0 (00000000315facbe) 00010000:00010000:3.0:1776178145.968792:0:6257:0:(ldlm_inodebits.c:96:ldlm_reprocess_inodebits_queue()) Hint 2 00010000:00010000:3.0:1776178145.968796:0:6257:0:(ldlm_lockd.c:1903:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: mdt-lustre-MDT0000_UUID lock: 00000000bef7c62f/0xa40083ab9bf531ba lrc: 2/0,0 mode: --/CR res: [0x2000013a1:0x1:0x0].0x0 bits 0x2/0x0 rrc: 4 type: IBT gid 0 flags: 0x44a19400000000 nid: local remote: 0x0 expref: -99 pid: 6257 timeout: 0 lvb_type: 0 00010000:00010000:3.0:1776178145.968802:0:6257:0:(ldlm_lock.c:216:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: mdt-lustre-MDT0000_UUID lock: 00000000bef7c62f/0xa40083ab9bf531ba lrc: 0/0,0 mode: --/CR res: [0x2000013a1:0x1:0x0].0x0 bits 0x2/0x0 rrc: 4 type: IBT gid 0 flags: 0x44a19400000000 nid: local remote: 0x0 expref: -99 pid: 6257 timeout: 0 lvb_type: 0 00010000:00010000:3.0:1776178145.968824:0:6257:0:(ldlm_lock.c:820:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PR) ns: mdt-lustre-MDT0000_UUID lock: 00000000712476de/0xa40083ab9bf531c1 lrc: 3/1,0 mode: PR/PR res: [0x2000013a1:0x1:0x0].0x7fbd58ea bits 0x2/0x0 rrc: 2 type: IBT gid 0 flags: 0x50210000000000 nid: local remote: 0x0 expref: -99 pid: 6257 timeout: 0 lvb_type: 0 00010000:00010000:3.0:1776178145.968830:0:6257:0:(ldlm_lock.c:879:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: mdt-lustre-MDT0000_UUID lock: 00000000712476de/0xa40083ab9bf531c1 lrc: 2/0,0 mode: PR/PR res: [0x2000013a1:0x1:0x0].0x7fbd58ea bits 0x2/0x0 rrc: 2 type: IBT gid 0 flags: 0x50210400000000 nid: local remote: 0x0 expref: -99 pid: 6257 timeout: 0 lvb_type: 0 00010000:00010000:3.0:1776178145.968836:0:6257:0:(ldlm_lockd.c:1879:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: mdt-lustre-MDT0000_UUID lock: 00000000712476de/0xa40083ab9bf531c1 lrc: 3/0,0 mode: PR/PR res: [0x2000013a1:0x1:0x0].0x7fbd58ea bits 0x2/0x0 rrc: 2 type: IBT gid 0 flags: 0x40210400000000 nid: local remote: 0x0 expref: -99 pid: 6257 timeout: 0 lvb_type: 0 00010000:00010000:3.0:1776178145.968841:0:6257:0:(ldlm_lockd.c:1893:ldlm_handle_bl_callback()) Lock 00000000712476de already unused, calling callback (0000000079e78348) 00010000:00010000:3.0:1776178145.968843:0:6257:0:(ldlm_request.c:364:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: mdt-lustre-MDT0000_UUID lock: 00000000712476de/0xa40083ab9bf531c1 lrc: 3/0,0 mode: PR/PR res: [0x2000013a1:0x1:0x0].0x7fbd58ea bits 0x2/0x0 rrc: 2 type: IBT gid 0 flags: 0x40210400000000 nid: local remote: 0x0 expref: -99 pid: 6257 timeout: 0 lvb_type: 0 00010000:00010000:3.0:1776178145.968848:0:6257:0:(ldlm_request.c:1257:ldlm_cli_cancel_local()) ### server-side local cancel ns: mdt-lustre-MDT0000_UUID lock: 00000000712476de/0xa40083ab9bf531c1 lrc: 4/0,0 mode: PR/PR res: [0x2000013a1:0x1:0x0].0x7fbd58ea bits 0x2/0x0 rrc: 2 type: IBT gid 0 flags: 0x40218400000000 nid: local remote: 0x0 expref: -99 pid: 6257 timeout: 0 lvb_type: 0 00010000:00010000:3.0:1776178145.968855:0:6257:0:(ldlm_inodebits.c:94:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x2000013a1:0x1:0x0].0x7fbd58ea (00000000b86d0927) 00010000:00010000:3.0:1776178145.968857:0:6257:0:(ldlm_inodebits.c:96:ldlm_reprocess_inodebits_queue()) Hint 2 00010000:00010000:3.0:1776178145.968859:0:6257:0:(ldlm_lockd.c:1903:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: mdt-lustre-MDT0000_UUID lock: 00000000712476de/0xa40083ab9bf531c1 lrc: 2/0,0 mode: --/PR res: [0x2000013a1:0x1:0x0].0x7fbd58ea bits 0x2/0x0 rrc: 2 type: IBT gid 0 flags: 0x44a19400000000 nid: local remote: 0x0 expref: -99 pid: 6257 timeout: 0 lvb_type: 0 00010000:00010000:3.0:1776178145.968863:0:6257:0:(ldlm_lock.c:216:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: mdt-lustre-MDT0000_UUID lock: 00000000712476de/0xa40083ab9bf531c1 lrc: 0/0,0 mode: --/PR res: [0x2000013a1:0x1:0x0].0x7fbd58ea bits 0x2/0x0 rrc: 2 type: IBT gid 0 flags: 0x44a19400000000 nid: local remote: 0x0 expref: -99 pid: 6257 timeout: 0 lvb_type: 0 00010000:00010000:3.0:1776178145.968894:0:6257:0:(ldlm_lock.c:216:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: mdt-lustre-MDT0000_UUID lock: 000000009f2de1df/0xa40083ab9bf531b3 lrc: 0/0,0 mode: --/CR res: [0x2000013a1:0x1:0x0].0x0 bits 0x2/0x0 rrc: 3 type: IBT gid 0 flags: 0x44000000000000 nid: 192.168.204.3@tcp remote: 0x771f68aa1eb27a62 expref: 36 pid: 6257 timeout: 0 lvb_type: 0 00010000:00010000:3.0:1776178145.968907:0:6257:0:(ldlm_lockd.c:1484:ldlm_handle_enqueue0()) ### server-side enqueue handler, sending reply (err=0, rc=0) ns: mdt-lustre-MDT0000_UUID lock: 000000005d93f094/0xa40083ab9bf531c8 lrc: 3/0,0 mode: PR/PR res: [0x2000013a1:0x8:0x0].0x0 bits 0x1b/0x0 rrc: 4 type: IBT gid 0 flags: 0x40200000000000 nid: 192.168.204.3@tcp remote: 0x771f68aa1eb27a62 expref: 35 pid: 6257 timeout: 0 lvb_type: 0 00010000:00010000:3.0:1776178145.968919:0:6257:0:(ldlm_inodebits.c:94:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x2000013a1:0x8:0x0].0x0 (000000009a65918f) 00010000:00010000:3.0:1776178145.968921:0:6257:0:(ldlm_inodebits.c:96:ldlm_reprocess_inodebits_queue()) Hint 1b 00010000:00010000:3.0:1776178145.968923:0:6257:0:(ldlm_lockd.c:1565:ldlm_handle_enqueue0()) ### server-side enqueue handler END (lock 000000005d93f094, rc 0) 00000100:00100000:3.0:1776178145.969017:0:6257:0:(service.c:2372:ptlrpc_server_handle_request()) Handled RPC req@00000000bfb62f7a pname:cluuid+ref:pid:xid:nid:opc:job mdt00_002:b50963bd-8eba-4432-9a54-8888d00e3fa1+35:17036:x1862456979086784:12345-192.168.204.3@tcp:101:checkstat.0 Request processed in 697us (5215us total) trans 0 rc 0/0 00000100:00100000:3.0:1776178145.969026:0:6257:0:(nrs_fifo.c:241:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.204.3@tcp, seq: 6511 00000100:00100000:1.0:1776178145.991029:0:3346:0:(events.c:361:request_in_callback()) peer: 12345-192.168.204.3@tcp (source: 12345-192.168.204.3@tcp) 00000100:00100000:3.0:1776178145.991202:0:6257:0:(service.c:2152:ptlrpc_server_handle_req_in()) got req x1862456979086848 00000100:00100000:3.0:1776178145.991273:0:6257:0:(nrs_fifo.c:179:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.204.3@tcp, seq: 6512 00000100:00100000:3.0:1776178145.991279:0:6257:0:(service.c:2324:ptlrpc_server_handle_request()) Handling RPC req@00000000f114dadf pname:cluuid+ref:pid:xid:nid:opc:job mdt00_002:b50963bd-8eba-4432-9a54-8888d00e3fa1+35:17037:x1862456979086848:12345-192.168.204.3@tcp:101:rm.0 00010000:00010000:3.0:1776178145.991306:0:6257:0:(ldlm_lockd.c:1248:ldlm_handle_enqueue0()) ### server-side enqueue handler START 00010000:00010000:3.0:1776178145.991328:0:6257:0:(ldlm_lockd.c:1318:ldlm_handle_enqueue0()) ### server-side enqueue handler, new lock created ns: mdt-lustre-MDT0000_UUID lock: 000000009f2de1df/0xa40083ab9bf531cf lrc: 2/0,0 mode: --/CR res: [0x200000007:0x1:0x0].0x0 bits 0x0/0x0 rrc: 4 type: IBT gid 0 flags: 0x40000000000000 nid: local remote: 0x771f68aa1eb27a69 expref: -99 pid: 6257 timeout: 0 lvb_type: 0 00010000:00010000:3.0:1776178145.991401:0:6257:0:(ldlm_lock.c:766:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PR) ns: mdt-lustre-MDT0000_UUID lock: 00000000712476de/0xa40083ab9bf531d6 lrc: 3/1,0 mode: --/PR res: [0x2000013a1:0x1:0x0].0x0 bits 0x0/0x0 rrc: 3 type: IBT gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 6257 timeout: 0 lvb_type: 0 00010000:00010000:3.0:1776178145.991417:0:6257:0:(ldlm_lock.c:1064:ldlm_granted_list_add_lock()) ### About to add lock: ns: mdt-lustre-MDT0000_UUID lock: 00000000712476de/0xa40083ab9bf531d6 lrc: 3/1,0 mode: PR/PR res: [0x2000013a1:0x1:0x0].0x0 bits 0x13/0x0 rrc: 3 type: IBT gid 0 flags: 0x50210000000000 nid: local remote: 0x0 expref: -99 pid: 6257 timeout: 0 lvb_type: 0 00010000:00010000:3.0:1776178145.991431:0:6257:0:(ldlm_request.c:528:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: mdt-lustre-MDT0000_UUID lock: 00000000712476de/0xa40083ab9bf531d6 lrc: 3/1,0 mode: PR/PR res: [0x2000013a1:0x1:0x0].0x0 bits 0x13/0x0 rrc: 3 type: IBT gid 0 flags: 0x40210000000000 nid: local remote: 0x0 expref: -99 pid: 6257 timeout: 0 lvb_type: 0 00000004:00200000:3.0:1776178145.991484:0:6257:0:(mdt_handler.c:912:mdt_pack_attr2body()) [0x2000013a1:0x1:0x0]: returning size 4096 00000004:00200000:3.0:1776178145.991489:0:6257:0:(mdt_handler.c:1523:mdt_getattr_internal()) dirent count 9 stripe count 1 MDT count 2 00000004:00010000:3.0:1776178145.991508:0:6257:0:(mdt_handler.c:2353:mdt_getattr_name_lock()) ### Returning lock to client ns: mdt-lustre-MDT0000_UUID lock: 00000000712476de/0xa40083ab9bf531d6 lrc: 3/1,0 mode: PR/PR res: [0x2000013a1:0x1:0x0].0x0 bits 0x13/0x0 rrc: 3 type: IBT gid 0 flags: 0x40210000000000 nid: local remote: 0x0 expref: -99 pid: 6257 timeout: 0 lvb_type: 0 00010000:00010000:3.0:1776178145.991537:0:6257:0:(ldlm_lock.c:216:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: mdt-lustre-MDT0000_UUID lock: 000000009f2de1df/0xa40083ab9bf531cf lrc: 0/0,0 mode: --/CR res: [0x200000007:0x1:0x0].0x0 bits 0x2/0x0 rrc: 4 type: IBT gid 0 flags: 0x44000000000000 nid: 192.168.204.3@tcp remote: 0x771f68aa1eb27a69 expref: 37 pid: 6257 timeout: 0 lvb_type: 0 00010000:00010000:3.0:1776178145.991556:0:6257:0:(ldlm_lockd.c:1484:ldlm_handle_enqueue0()) ### server-side enqueue handler, sending reply (err=0, rc=0) ns: mdt-lustre-MDT0000_UUID lock: 00000000712476de/0xa40083ab9bf531d6 lrc: 3/0,0 mode: PR/PR res: [0x2000013a1:0x1:0x0].0x0 bits 0x13/0x0 rrc: 3 type: IBT gid 0 flags: 0x40200000000000 nid: 192.168.204.3@tcp remote: 0x771f68aa1eb27a69 expref: 36 pid: 6257 timeout: 0 lvb_type: 0 00010000:00010000:3.0:1776178145.991564:0:6257:0:(ldlm_inodebits.c:94:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x2000013a1:0x1:0x0].0x0 (00000000315facbe) 00010000:00010000:3.0:1776178145.991567:0:6257:0:(ldlm_inodebits.c:96:ldlm_reprocess_inodebits_queue()) Hint 13 00010000:00010000:3.0:1776178145.991570:0:6257:0:(ldlm_lockd.c:1565:ldlm_handle_enqueue0()) ### server-side enqueue handler END (lock 00000000712476de, rc 0) 00000100:00100000:3.0:1776178145.991632:0:6257:0:(service.c:2372:ptlrpc_server_handle_request()) Handled RPC req@00000000f114dadf pname:cluuid+ref:pid:xid:nid:opc:job mdt00_002:b50963bd-8eba-4432-9a54-8888d00e3fa1+36:17037:x1862456979086848:12345-192.168.204.3@tcp:101:rm.0 Request processed in 352us (623us total) trans 0 rc 0/0 00000100:00100000:3.0:1776178145.991639:0:6257:0:(nrs_fifo.c:241:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.204.3@tcp, seq: 6512 00000100:00100000:1.0:1776178146.006434:0:3348:0:(events.c:361:request_in_callback()) peer: 12345-192.168.204.3@tcp (source: 12345-192.168.204.3@tcp) 00000100:00100000:3.0:1776178146.009491:0:6257:0:(service.c:2152:ptlrpc_server_handle_req_in()) got req x1862456979086976 00000100:00100000:3.0:1776178146.009611:0:6257:0:(nrs_fifo.c:179:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.204.3@tcp, seq: 6513 00000100:00100000:3.0:1776178146.009620:0:6257:0:(service.c:2324:ptlrpc_server_handle_request()) Handling RPC req@00000000736589c7 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_002:b50963bd-8eba-4432-9a54-8888d00e3fa1+36:17037:x1862456979086976:12345-192.168.204.3@tcp:101:rm.0 00010000:00010000:3.0:1776178146.009656:0:6257:0:(ldlm_lockd.c:1248:ldlm_handle_enqueue0()) ### server-side enqueue handler START 00010000:00010000:3.0:1776178146.009691:0:6257:0:(ldlm_lockd.c:1318:ldlm_handle_enqueue0()) ### server-side enqueue handler, new lock created ns: mdt-lustre-MDT0000_UUID lock: 000000009f2de1df/0xa40083ab9bf531dd lrc: 2/0,0 mode: --/CR res: [0x200000007:0x1:0x0].0x0 bits 0x0/0x0 rrc: 4 type: IBT gid 0 flags: 0x40000000000000 nid: local remote: 0x771f68aa1eb27a70 expref: -99 pid: 6257 timeout: 0 lvb_type: 0 00000004:00200000:3.0:1776178146.009850:0:6257:0:(mdt_handler.c:912:mdt_pack_attr2body()) [0x2000013a1:0x1:0x0]: returning size 4096 00010000:00010000:3.0:1776178146.009989:0:6257:0:(ldlm_lockd.c:1484:ldlm_handle_enqueue0()) ### server-side enqueue handler, sending reply (err=301, rc=0) ns: mdt-lustre-MDT0000_UUID lock: 000000009f2de1df/0xa40083ab9bf531dd lrc: 1/0,0 mode: --/CR res: [0x200000007:0x1:0x0].0x0 bits 0x1/0x0 rrc: 4 type: IBT gid 0 flags: 0x44000000000000 nid: 192.168.204.3@tcp remote: 0x771f68aa1eb27a70 expref: 38 pid: 6257 timeout: 0 lvb_type: 0 00010000:00010000:3.0:1776178146.010006:0:6257:0:(ldlm_lock.c:216:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: mdt-lustre-MDT0000_UUID lock: 000000009f2de1df/0xa40083ab9bf531dd lrc: 0/0,0 mode: --/CR res: [0x200000007:0x1:0x0].0x0 bits 0x1/0x0 rrc: 4 type: IBT gid 0 flags: 0x44000000000000 nid: 192.168.204.3@tcp remote: 0x771f68aa1eb27a70 expref: 38 pid: 6257 timeout: 0 lvb_type: 0 00010000:00010000:3.0:1776178146.010023:0:6257:0:(ldlm_lockd.c:1565:ldlm_handle_enqueue0()) ### server-side enqueue handler END (lock 000000009f2de1df, rc 0) 00000100:00100000:3.0:1776178146.010196:0:6257:0:(service.c:2372:ptlrpc_server_handle_request()) Handled RPC req@00000000736589c7 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_002:b50963bd-8eba-4432-9a54-8888d00e3fa1+37:17037:x1862456979086976:12345-192.168.204.3@tcp:101:rm.0 Request processed in 576us (3777us total) trans 12884901932 rc 301/301 00000100:00100000:3.0:1776178146.010209:0:6257:0:(nrs_fifo.c:241:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.204.3@tcp, seq: 6513 00000100:00100000:1.0:1776178146.017350:0:3347:0:(events.c:361:request_in_callback()) peer: 12345-192.168.204.3@tcp (source: 12345-192.168.204.3@tcp) 00000100:00100000:2.0:1776178146.018303:0:6258:0:(service.c:2152:ptlrpc_server_handle_req_in()) got req x1862456979087040 00000100:00100000:2.0:1776178146.018434:0:6258:0:(nrs_fifo.c:179:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.204.3@tcp, seq: 2130 00000100:00100000:2.0:1776178146.018442:0:6258:0:(service.c:2324:ptlrpc_server_handle_request()) Handling RPC req@000000004a3d9287 pname:cluuid+ref:pid:xid:nid:opc:job mdt_rdpg00_000:b50963bd-8eba-4432-9a54-8888d00e3fa1+37:17037:x1862456979087040:12345-192.168.204.3@tcp:37:rm.0 00000100:00100000:2.0:1776178146.037495:0:6258:0:(service.c:2372:ptlrpc_server_handle_request()) Handled RPC req@000000004a3d9287 pname:cluuid+ref:pid:xid:nid:opc:job mdt_rdpg00_000:b50963bd-8eba-4432-9a54-8888d00e3fa1+37:17037:x1862456979087040:12345-192.168.204.3@tcp:37:rm.0 Request processed in 19041us (20145us total) trans 0 rc 0/0 00000100:00100000:2.0:1776178146.037516:0:6258:0:(nrs_fifo.c:241:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.204.3@tcp, seq: 2130 00000100:00100000:1.0:1776178146.048686:0:3348:0:(events.c:361:request_in_callback()) peer: 12345-192.168.204.3@tcp (source: 12345-192.168.204.3@tcp) 00000100:00100000:2.0:1776178146.050317:0:6258:0:(service.c:2152:ptlrpc_server_handle_req_in()) got req x1862456979087168 00000100:00100000:2.0:1776178146.050396:0:6258:0:(nrs_fifo.c:179:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.204.3@tcp, seq: 2131 00000100:00100000:2.0:1776178146.050403:0:6258:0:(service.c:2324:ptlrpc_server_handle_request()) Handling RPC req@0000000043589f56 pname:cluuid+ref:pid:xid:nid:opc:job mdt_rdpg00_000:b50963bd-8eba-4432-9a54-8888d00e3fa1+37:17037:x1862456979087168:12345-192.168.204.3@tcp:35:rm.0 00000100:00100000:2.0:1776178146.050695:0:6258:0:(service.c:2372:ptlrpc_server_handle_request()) Handled RPC req@0000000043589f56 pname:cluuid+ref:pid:xid:nid:opc:job mdt_rdpg00_000:b50963bd-8eba-4432-9a54-8888d00e3fa1+38:17037:x1862456979087168:12345-192.168.204.3@tcp:35:rm.0 Request processed in 291us (2023us total) trans 12884901933 rc 0/0 00000100:00100000:2.0:1776178146.050704:0:6258:0:(nrs_fifo.c:241:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.204.3@tcp, seq: 2131 00000100:00100000:0.0:1776178146.053399:0:3347:0:(events.c:361:request_in_callback()) peer: 12345-192.168.204.3@tcp (source: 12345-192.168.204.3@tcp) 00000100:00100000:2.0:1776178146.053517:0:8250:0:(service.c:2152:ptlrpc_server_handle_req_in()) got req x1862456979087296 00000100:00100000:2.0:1776178146.053579:0:8250:0:(nrs_fifo.c:179:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.204.3@tcp, seq: 6514 00000100:00100000:2.0:1776178146.053587:0:8250:0:(service.c:2324:ptlrpc_server_handle_request()) Handling RPC req@00000000956deaca pname:cluuid+ref:pid:xid:nid:opc:job mdt00_003:b50963bd-8eba-4432-9a54-8888d00e3fa1+38:17037:x1862456979087296:12345-192.168.204.3@tcp:101:rm.0 00010000:00010000:2.0:1776178146.053615:0:8250:0:(ldlm_lockd.c:1248:ldlm_handle_enqueue0()) ### server-side enqueue handler START 00010000:00010000:2.0:1776178146.053638:0:8250:0:(ldlm_lockd.c:1318:ldlm_handle_enqueue0()) ### server-side enqueue handler, new lock created ns: mdt-lustre-MDT0000_UUID lock: 00000000fcb32840/0xa40083ab9bf531eb lrc: 2/0,0 mode: --/CR res: [0x200000007:0x1:0x0].0x0 bits 0x0/0x0 rrc: 4 type: IBT gid 0 flags: 0x40000000000000 nid: local remote: 0x771f68aa1eb27a77 expref: -99 pid: 8250 timeout: 0 lvb_type: 0 00010000:00010000:2.0:1776178146.053914:0:8250:0:(ldlm_lock.c:766:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(CR) ns: mdt-lustre-MDT0000_UUID lock: 00000000a27c19b3/0xa40083ab9bf531f2 lrc: 3/1,0 mode: --/CR res: [0x2000013a1:0x1:0x0].0x0 bits 0x0/0x0 rrc: 4 type: IBT gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 8250 timeout: 0 lvb_type: 0 00010000:00010000:2.0:1776178146.053931:0:8250:0:(ldlm_lock.c:1064:ldlm_granted_list_add_lock()) ### About to add lock: ns: mdt-lustre-MDT0000_UUID lock: 00000000a27c19b3/0xa40083ab9bf531f2 lrc: 3/1,0 mode: CR/CR res: [0x2000013a1:0x1:0x0].0x0 bits 0x5/0x0 rrc: 4 type: IBT gid 0 flags: 0x50210000000000 nid: local remote: 0x0 expref: -99 pid: 8250 timeout: 0 lvb_type: 0 00010000:00010000:2.0:1776178146.053942:0:8250:0:(ldlm_request.c:528:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: mdt-lustre-MDT0000_UUID lock: 00000000a27c19b3/0xa40083ab9bf531f2 lrc: 3/1,0 mode: CR/CR res: [0x2000013a1:0x1:0x0].0x0 bits 0x5/0x0 rrc: 4 type: IBT gid 0 flags: 0x40210000000000 nid: local remote: 0x0 expref: -99 pid: 8250 timeout: 0 lvb_type: 0 00000004:00200000:2.0:1776178146.053955:0:8250:0:(mdt_handler.c:912:mdt_pack_attr2body()) [0x2000013a1:0x1:0x0]: returning size 4096 00010000:00010000:2.0:1776178146.054302:0:8250:0:(ldlm_lock.c:216:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: mdt-lustre-MDT0000_UUID lock: 00000000fcb32840/0xa40083ab9bf531eb lrc: 0/0,0 mode: --/CR res: [0x200000007:0x1:0x0].0x0 bits 0x1/0x0 rrc: 4 type: IBT gid 0 flags: 0x44000000000000 nid: 192.168.204.3@tcp remote: 0x771f68aa1eb27a77 expref: 41 pid: 8250 timeout: 0 lvb_type: 0 00010000:00010000:2.0:1776178146.054320:0:8250:0:(ldlm_lockd.c:1484:ldlm_handle_enqueue0()) ### server-side enqueue handler, sending reply (err=0, rc=0) ns: mdt-lustre-MDT0000_UUID lock: 00000000a27c19b3/0xa40083ab9bf531f2 lrc: 3/0,0 mode: CR/CR res: [0x2000013a1:0x1:0x0].0x0 bits 0x5/0x0 rrc: 4 type: IBT gid 0 flags: 0x40200000000000 nid: 192.168.204.3@tcp remote: 0x771f68aa1eb27a77 expref: 40 pid: 8250 timeout: 0 lvb_type: 0 00010000:00010000:2.0:1776178146.054333:0:8250:0:(ldlm_inodebits.c:94:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x2000013a1:0x1:0x0].0x0 (00000000315facbe) 00010000:00010000:2.0:1776178146.054336:0:8250:0:(ldlm_inodebits.c:96:ldlm_reprocess_inodebits_queue()) Hint 5 00010000:00010000:2.0:1776178146.054339:0:8250:0:(ldlm_lockd.c:1565:ldlm_handle_enqueue0()) ### server-side enqueue handler END (lock 00000000a27c19b3, rc 0) 00000100:00100000:2.0:1776178146.054451:0:8250:0:(service.c:2372:ptlrpc_server_handle_request()) Handled RPC req@00000000956deaca pname:cluuid+ref:pid:xid:nid:opc:job mdt00_003:b50963bd-8eba-4432-9a54-8888d00e3fa1+40:17037:x1862456979087296:12345-192.168.204.3@tcp:101:rm.0 Request processed in 863us (1061us total) trans 12884901934 rc 0/0 00000100:00100000:2.0:1776178146.054460:0:8250:0:(nrs_fifo.c:241:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.204.3@tcp, seq: 6514 00000100:00100000:1.0:1776178146.071439:0:3346:0:(events.c:361:request_in_callback()) peer: 12345-192.168.204.3@tcp (source: 12345-192.168.204.3@tcp) 00000100:00100000:2.0:1776178146.071664:0:8250:0:(service.c:2152:ptlrpc_server_handle_req_in()) got req x1862456979087360 00000100:00100000:2.0:1776178146.071739:0:8250:0:(nrs_fifo.c:179:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.204.3@tcp, seq: 6515 00000100:00100000:2.0:1776178146.071754:0:8250:0:(service.c:2324:ptlrpc_server_handle_request()) Handling RPC req@00000000456ce710 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_003:b50963bd-8eba-4432-9a54-8888d00e3fa1+40:17037:x1862456979087360:12345-192.168.204.3@tcp:41:rm.0 00000004:00000024:2.0:1776178146.071830:0:8250:0:(mdt_handler.c:548:mdt_statfs()) blocks cached 0 granted 2097152 pending 0 free 7847378944 avail 7383789568 00000100:00100000:2.0:1776178146.071905:0:8250:0:(service.c:2372:ptlrpc_server_handle_request()) Handled RPC req@00000000456ce710 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_003:b50963bd-8eba-4432-9a54-8888d00e3fa1+40:17037:x1862456979087360:12345-192.168.204.3@tcp:41:rm.0 Request processed in 151us (480us total) trans 0 rc 0/0 00000100:00100000:2.0:1776178146.071913:0:8250:0:(nrs_fifo.c:241:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.204.3@tcp, seq: 6515 00000100:00100000:1.0:1776178146.075931:0:3348:0:(events.c:361:request_in_callback()) peer: 12345-192.168.204.3@tcp (source: 12345-192.168.204.3@tcp) 00000100:00100000:2.0:1776178146.077952:0:8250:0:(service.c:2152:ptlrpc_server_handle_req_in()) got req x1862456979087488 00000100:00100000:3.0:1776178146.078370:0:6257:0:(nrs_fifo.c:179:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.204.3@tcp, seq: 6516 00000100:00100000:3.0:1776178146.078379:0:6257:0:(service.c:2324:ptlrpc_server_handle_request()) Handling RPC req@0000000055392450 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_002:b50963bd-8eba-4432-9a54-8888d00e3fa1+40:17037:x1862456979087488:12345-192.168.204.3@tcp:36:rm.0 00010000:00010000:3.0:1776178146.078462:0:6257:0:(ldlm_lockd.c:1720:ldlm_request_cancel()) ### server-side cancel handler START: 3 locks, starting at 0 00010000:00010000:3.0:1776178146.078491:0:6257:0:(ldlm_lock.c:216:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: mdt-lustre-MDT0000_UUID lock: 000000001735c8e5/0xa40083ab9bf53158 lrc: 0/0,0 mode: --/PR res: [0x2000013a1:0x4:0x0].0x0 bits 0x1b/0x0 rrc: 5 type: IBT gid 0 flags: 0x44a01000000000 nid: 192.168.204.3@tcp remote: 0x771f68aa1eb27a46 expref: 40 pid: 8250 timeout: 0 lvb_type: 0 00010000:00010000:3.0:1776178146.078515:0:6257:0:(ldlm_lock.c:216:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: mdt-lustre-MDT0000_UUID lock: 0000000015ae4ee6/0xa40083ab9bf530ef lrc: 0/0,0 mode: --/CR res: [0x2000013a1:0x4:0x0].0x0 bits 0x8/0x0 rrc: 4 type: IBT gid 0 flags: 0x44801000000000 nid: 192.168.204.3@tcp remote: 0x771f68aa1eb27a23 expref: 39 pid: 8250 timeout: 0 lvb_type: 0 00010000:00010000:3.0:1776178146.078524:0:6257:0:(ldlm_inodebits.c:94:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x2000013a1:0x4:0x0].0x0 (000000009cfa2d69) 00010000:00010000:3.0:1776178146.078534:0:6257:0:(ldlm_lock.c:216:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: mdt-lustre-MDT0000_UUID lock: 00000000712476de/0xa40083ab9bf531d6 lrc: 0/0,0 mode: --/PR res: [0x2000013a1:0x1:0x0].0x0 bits 0x13/0x0 rrc: 5 type: IBT gid 0 flags: 0x44a01000000000 nid: 192.168.204.3@tcp remote: 0x771f68aa1eb27a69 expref: 38 pid: 6257 timeout: 0 lvb_type: 0 00010000:00010000:3.0:1776178146.078540:0:6257:0:(ldlm_inodebits.c:94:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x2000013a1:0x1:0x0].0x0 (00000000315facbe) 00010000:00010000:3.0:1776178146.078542:0:6257:0:(ldlm_lockd.c:1782:ldlm_request_cancel()) ### server-side cancel handler END 00010000:00010000:3.0:1776178146.078573:0:6257:0:(ldlm_lock.c:766:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(CW) ns: mdt-lustre-MDT0000_UUID lock: 000000009f2de1df/0xa40083ab9bf53200 lrc: 3/0,1 mode: --/CW res: [0x2000013a1:0x1:0x0].0x0 bits 0x0/0x0 rrc: 4 type: IBT gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 6257 timeout: 0 lvb_type: 0 00010000:00010000:3.0:1776178146.078589:0:6257:0:(ldlm_lock.c:1064:ldlm_granted_list_add_lock()) ### About to add lock: ns: mdt-lustre-MDT0000_UUID lock: 000000009f2de1df/0xa40083ab9bf53200 lrc: 3/0,1 mode: CW/CW res: [0x2000013a1:0x1:0x0].0x0 bits 0x2/0x0 rrc: 4 type: IBT gid 0 flags: 0x50210000000000 nid: local remote: 0x0 expref: -99 pid: 6257 timeout: 0 lvb_type: 0 00010000:00010000:3.0:1776178146.078599:0:6257:0:(ldlm_request.c:528:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: mdt-lustre-MDT0000_UUID lock: 000000009f2de1df/0xa40083ab9bf53200 lrc: 3/0,1 mode: CW/CW res: [0x2000013a1:0x1:0x0].0x0 bits 0x2/0x0 rrc: 4 type: IBT gid 0 flags: 0x40210000000000 nid: local remote: 0x0 expref: -99 pid: 6257 timeout: 0 lvb_type: 0 00010000:00010000:3.0:1776178146.078613:0:6257:0:(ldlm_lock.c:766:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: mdt-lustre-MDT0000_UUID lock: 00000000bef7c62f/0xa40083ab9bf53207 lrc: 3/0,1 mode: --/PW res: [0x2000013a1:0x1:0x0].0x5e5674ea bits 0x0/0x0 rrc: 2 type: IBT gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 6257 timeout: 0 lvb_type: 0 00010000:00010000:3.0:1776178146.078625:0:6257:0:(ldlm_lock.c:1064:ldlm_granted_list_add_lock()) ### About to add lock: ns: mdt-lustre-MDT0000_UUID lock: 00000000bef7c62f/0xa40083ab9bf53207 lrc: 3/0,1 mode: PW/PW res: [0x2000013a1:0x1:0x0].0x5e5674ea bits 0x2/0x0 rrc: 2 type: IBT gid 0 flags: 0x50210000000000 nid: local remote: 0x0 expref: -99 pid: 6257 timeout: 0 lvb_type: 0 00010000:00010000:3.0:1776178146.078630:0:6257:0:(ldlm_request.c:528:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: mdt-lustre-MDT0000_UUID lock: 00000000bef7c62f/0xa40083ab9bf53207 lrc: 3/0,1 mode: PW/PW res: [0x2000013a1:0x1:0x0].0x5e5674ea bits 0x2/0x0 rrc: 2 type: IBT gid 0 flags: 0x40210000000000 nid: local remote: 0x0 expref: -99 pid: 6257 timeout: 0 lvb_type: 0 00010000:00010000:3.0:1776178146.078709:0:6257:0:(ldlm_lock.c:766:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(EX) ns: mdt-lustre-MDT0000_UUID lock: 000000006cb7334a/0xa40083ab9bf5320e lrc: 3/0,1 mode: --/EX res: [0x2000013a1:0x4:0x0].0x0 bits 0x0/0x0 rrc: 3 type: IBT gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 6257 timeout: 0 lvb_type: 0 00010000:00010000:3.0:1776178146.078717:0:6257:0:(ldlm_lock.c:1064:ldlm_granted_list_add_lock()) ### About to add lock: ns: mdt-lustre-MDT0000_UUID lock: 000000006cb7334a/0xa40083ab9bf5320e lrc: 3/0,1 mode: EX/EX res: [0x2000013a1:0x4:0x0].0x0 bits 0x3/0x0 rrc: 3 type: IBT gid 0 flags: 0x50210000000000 nid: local remote: 0x0 expref: -99 pid: 6257 timeout: 0 lvb_type: 0 00010000:00010000:3.0:1776178146.078723:0:6257:0:(ldlm_request.c:528:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: mdt-lustre-MDT0000_UUID lock: 000000006cb7334a/0xa40083ab9bf5320e lrc: 3/0,1 mode: EX/EX res: [0x2000013a1:0x4:0x0].0x0 bits 0x3/0x0 rrc: 3 type: IBT gid 0 flags: 0x40210000000000 nid: local remote: 0x0 expref: -99 pid: 6257 timeout: 0 lvb_type: 0 00000040:00080000:3.0:1776178146.078991:0:6257:0:(llog_osd.c:738:llog_osd_write_rec()) added record [0x1:0xc:0x0].3, 64 off8384 00000004:00080000:3.0:1776178146.079008:0:6257:0:(osp_sync.c:1658:osp_sync_add_commit_cb()) lustre-OST0000-osc-MDT0000: add commit cb at 853405947011ns, next at 854405944388ns, rc = 0 00010000:00010000:3.0:1776178146.079089:0:6257:0:(ldlm_lock.c:766:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: mdt-lustre-MDT0000_UUID lock: 0000000026fd08a4/0xa40083ab9bf53215 lrc: 3/0,1 mode: --/PW res: [0x2000013a1:0x4:0x0].0x0 bits 0x0/0x0 rrc: 4 type: IBT gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 6257 timeout: 0 lvb_type: 0 00010000:00010000:3.0:1776178146.079099:0:6257:0:(ldlm_lock.c:1064:ldlm_granted_list_add_lock()) ### About to add lock: ns: mdt-lustre-MDT0000_UUID lock: 0000000026fd08a4/0xa40083ab9bf53215 lrc: 3/0,1 mode: PW/PW res: [0x2000013a1:0x4:0x0].0x0 bits 0x40/0x0 rrc: 4 type: IBT gid 18446635902900449728 flags: 0x50010080000000 nid: local remote: 0x0 expref: -99 pid: 6257 timeout: 0 lvb_type: 0 00010000:00010000:3.0:1776178146.079106:0:6257:0:(ldlm_request.c:528:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: mdt-lustre-MDT0000_UUID lock: 0000000026fd08a4/0xa40083ab9bf53215 lrc: 3/0,1 mode: PW/PW res: [0x2000013a1:0x4:0x0].0x0 bits 0x40/0x0 rrc: 4 type: IBT gid 18446635902900449728 flags: 0x40010080000000 nid: local remote: 0x0 expref: -99 pid: 6257 timeout: 0 lvb_type: 0 00010000:00010000:3.0:1776178146.079113:0:6257:0:(ldlm_lock.c:942:ldlm_lock_decref_and_cancel()) ### ldlm_lock_decref(PW) ns: mdt-lustre-MDT0000_UUID lock: 0000000026fd08a4/0xa40083ab9bf53215 lrc: 4/0,1 mode: PW/PW res: [0x2000013a1:0x4:0x0].0x0 bits 0x40/0x0 rrc: 4 type: IBT gid 18446635902900449728 flags: 0x40010080000000 nid: local remote: 0x0 expref: -99 pid: 6257 timeout: 0 lvb_type: 0 00010000:00010000:3.0:1776178146.079123:0:6257:0:(ldlm_lock.c:820:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: mdt-lustre-MDT0000_UUID lock: 0000000026fd08a4/0xa40083ab9bf53215 lrc: 4/0,1 mode: PW/PW res: [0x2000013a1:0x4:0x0].0x0 bits 0x40/0x0 rrc: 4 type: IBT gid 18446635902900449728 flags: 0x50010480000000 nid: local remote: 0x0 expref: -99 pid: 6257 timeout: 0 lvb_type: 0 00010000:00010000:3.0:1776178146.079130:0:6257:0:(ldlm_lock.c:879:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: mdt-lustre-MDT0000_UUID lock: 0000000026fd08a4/0xa40083ab9bf53215 lrc: 3/0,0 mode: PW/PW res: [0x2000013a1:0x4:0x0].0x0 bits 0x40/0x0 rrc: 4 type: IBT gid 18446635902900449728 flags: 0x50010480000000 nid: local remote: 0x0 expref: -99 pid: 6257 timeout: 0 lvb_type: 0 00010000:00010000:3.0:1776178146.079158:0:6257:0:(ldlm_lockd.c:2159:__ldlm_bl_to_thread()) added 1/1 locks to regular blp list, 1 blwis in pool 00000004:00010000:3.0:1776178146.079179:0:6257:0:(mdt_handler.c:3992:mdt_save_lock()) ### save lock request 0000000055392450 reply state 000000001dc19eda transno 12884901935 ns: mdt-lustre-MDT0000_UUID lock: 000000006cb7334a/0xa40083ab9bf5320e lrc: 3/0,1 mode: EX/EX res: [0x2000013a1:0x4:0x0].0x0 bits 0x3/0x0 rrc: 4 type: IBT gid 0 flags: 0x40210000000000 nid: local remote: 0x0 expref: -99 pid: 6257 timeout: 0 lvb_type: 0 00010000:00010000:1.0:1776178146.079543:0:6244:0:(ldlm_lockd.c:2789:ldlm_bl_get_work()) Got 1 locks of 0 total in blp. (0 blwis in pool) 00010000:00010000:1.0:1776178146.079549:0:6244:0:(ldlm_lockd.c:1879:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: mdt-lustre-MDT0000_UUID lock: 0000000026fd08a4/0xa40083ab9bf53215 lrc: 2/0,0 mode: PW/PW res: [0x2000013a1:0x4:0x0].0x0 bits 0x40/0x0 rrc: 4 type: IBT gid 18446635902900449728 flags: 0x40010480000000 nid: local remote: 0x0 expref: -99 pid: 6257 timeout: 0 lvb_type: 0 00010000:00010000:1.0:1776178146.079557:0:6244:0:(ldlm_lockd.c:1893:ldlm_handle_bl_callback()) Lock 0000000026fd08a4 already unused, calling callback (0000000063c97258) 00010000:00010000:1.0:1776178146.079561:0:6244:0:(ldlm_request.c:364:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: mdt-lustre-MDT0000_UUID lock: 0000000026fd08a4/0xa40083ab9bf53215 lrc: 2/0,0 mode: PW/PW res: [0x2000013a1:0x4:0x0].0x0 bits 0x40/0x0 rrc: 4 type: IBT gid 18446635902900449728 flags: 0x40010480000000 nid: local remote: 0x0 expref: -99 pid: 6257 timeout: 0 lvb_type: 0 00010000:00010000:1.0:1776178146.079569:0:6244:0:(ldlm_request.c:1257:ldlm_cli_cancel_local()) ### server-side local cancel ns: mdt-lustre-MDT0000_UUID lock: 0000000026fd08a4/0xa40083ab9bf53215 lrc: 3/0,0 mode: PW/PW res: [0x2000013a1:0x4:0x0].0x0 bits 0x40/0x0 rrc: 4 type: IBT gid 18446635902900449728 flags: 0x40018480000000 nid: local remote: 0x0 expref: -99 pid: 6257 timeout: 0 lvb_type: 0 00010000:00010000:1.0:1776178146.079582:0:6244:0:(ldlm_inodebits.c:94:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x2000013a1:0x4:0x0].0x0 (000000009cfa2d69) 00010000:00010000:1.0:1776178146.079584:0:6244:0:(ldlm_inodebits.c:96:ldlm_reprocess_inodebits_queue()) Hint 40 00010000:00010000:1.0:1776178146.079588:0:6244:0:(ldlm_lockd.c:1903:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: mdt-lustre-MDT0000_UUID lock: 0000000026fd08a4/0xa40083ab9bf53215 lrc: 1/0,0 mode: --/PW res: [0x2000013a1:0x4:0x0].0x0 bits 0x40/0x0 rrc: 4 type: IBT gid 18446635902900449728 flags: 0x44819480000000 nid: local remote: 0x0 expref: -99 pid: 6257 timeout: 0 lvb_type: 0 00010000:00010000:1.0:1776178146.079593:0:6244:0:(ldlm_lock.c:216:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: mdt-lustre-MDT0000_UUID lock: 0000000026fd08a4/0xa40083ab9bf53215 lrc: 0/0,0 mode: --/PW res: [0x2000013a1:0x4:0x0].0x0 bits 0x40/0x0 rrc: 4 type: IBT gid 18446635902900449728 flags: 0x44819480000000 nid: local remote: 0x0 expref: -99 pid: 6257 timeout: 0 lvb_type: 0 00010000:00010000:1.0:1776178146.079606:0:6244:0:(ldlm_lockd.c:2792:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00010000:00010000:1.0:1776178146.079607:0:6244:0:(ldlm_lockd.c:2792:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00010000:00010000:1.0:1776178146.079609:0:6244:0:(ldlm_lockd.c:2792:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00010000:00010000:3.0:1776178146.079752:0:6257:0:(ldlm_lock.c:820:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(CW) ns: mdt-lustre-MDT0000_UUID lock: 000000009f2de1df/0xa40083ab9bf53200 lrc: 3/0,1 mode: CW/CW res: [0x2000013a1:0x1:0x0].0x0 bits 0x2/0x0 rrc: 4 type: IBT gid 0 flags: 0x50210000000000 nid: local remote: 0x0 expref: -99 pid: 6257 timeout: 0 lvb_type: 0 00010000:00010000:3.0:1776178146.079760:0:6257:0:(ldlm_lock.c:879:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: mdt-lustre-MDT0000_UUID lock: 000000009f2de1df/0xa40083ab9bf53200 lrc: 2/0,0 mode: CW/CW res: [0x2000013a1:0x1:0x0].0x0 bits 0x2/0x0 rrc: 4 type: IBT gid 0 flags: 0x50210400000000 nid: local remote: 0x0 expref: -99 pid: 6257 timeout: 0 lvb_type: 0 00010000:00010000:3.0:1776178146.079765:0:6257:0:(ldlm_lockd.c:1879:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: mdt-lustre-MDT0000_UUID lock: 000000009f2de1df/0xa40083ab9bf53200 lrc: 3/0,0 mode: CW/CW res: [0x2000013a1:0x1:0x0].0x0 bits 0x2/0x0 rrc: 4 type: IBT gid 0 flags: 0x40210400000000 nid: local remote: 0x0 expref: -99 pid: 6257 timeout: 0 lvb_type: 0 00010000:00010000:3.0:1776178146.079770:0:6257:0:(ldlm_lockd.c:1893:ldlm_handle_bl_callback()) Lock 000000009f2de1df already unused, calling callback (0000000079e78348) 00010000:00010000:3.0:1776178146.079773:0:6257:0:(ldlm_request.c:364:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: mdt-lustre-MDT0000_UUID lock: 000000009f2de1df/0xa40083ab9bf53200 lrc: 3/0,0 mode: CW/CW res: [0x2000013a1:0x1:0x0].0x0 bits 0x2/0x0 rrc: 4 type: IBT gid 0 flags: 0x40210400000000 nid: local remote: 0x0 expref: -99 pid: 6257 timeout: 0 lvb_type: 0 00010000:00010000:3.0:1776178146.079779:0:6257:0:(ldlm_request.c:1257:ldlm_cli_cancel_local()) ### server-side local cancel ns: mdt-lustre-MDT0000_UUID lock: 000000009f2de1df/0xa40083ab9bf53200 lrc: 4/0,0 mode: CW/CW res: [0x2000013a1:0x1:0x0].0x0 bits 0x2/0x0 rrc: 4 type: IBT gid 0 flags: 0x40218400000000 nid: local remote: 0x0 expref: -99 pid: 6257 timeout: 0 lvb_type: 0 00010000:00010000:3.0:1776178146.079787:0:6257:0:(ldlm_inodebits.c:94:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x2000013a1:0x1:0x0].0x0 (00000000315facbe) 00010000:00010000:3.0:1776178146.079790:0:6257:0:(ldlm_inodebits.c:96:ldlm_reprocess_inodebits_queue()) Hint 2 00010000:00010000:3.0:1776178146.079792:0:6257:0:(ldlm_lockd.c:1903:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: mdt-lustre-MDT0000_UUID lock: 000000009f2de1df/0xa40083ab9bf53200 lrc: 2/0,0 mode: --/CW res: [0x2000013a1:0x1:0x0].0x0 bits 0x2/0x0 rrc: 4 type: IBT gid 0 flags: 0x44a19400000000 nid: local remote: 0x0 expref: -99 pid: 6257 timeout: 0 lvb_type: 0 00010000:00010000:3.0:1776178146.079798:0:6257:0:(ldlm_lock.c:216:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: mdt-lustre-MDT0000_UUID lock: 000000009f2de1df/0xa40083ab9bf53200 lrc: 0/0,0 mode: --/CW res: [0x2000013a1:0x1:0x0].0x0 bits 0x2/0x0 rrc: 4 type: IBT gid 0 flags: 0x44a19400000000 nid: local remote: 0x0 expref: -99 pid: 6257 timeout: 0 lvb_type: 0 00000004:00010000:3.0:1776178146.079816:0:6257:0:(mdt_handler.c:3992:mdt_save_lock()) ### save lock request 0000000055392450 reply state 000000001dc19eda transno 12884901935 ns: mdt-lustre-MDT0000_UUID lock: 00000000bef7c62f/0xa40083ab9bf53207 lrc: 3/0,1 mode: PW/PW res: [0x2000013a1:0x1:0x0].0x5e5674ea bits 0x2/0x0 rrc: 2 type: IBT gid 0 flags: 0x40210000000000 nid: local remote: 0x0 expref: -99 pid: 6257 timeout: 0 lvb_type: 0 00000100:00100000:3.0:1776178146.079915:0:6257:0:(service.c:2372:ptlrpc_server_handle_request()) Handled RPC req@0000000055392450 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_002:b50963bd-8eba-4432-9a54-8888d00e3fa1+39:17037:x1862456979087488:12345-192.168.204.3@tcp:36:rm.0 Request processed in 1533us (3990us total) trans 12884901935 rc 0/0 00000100:00100000:3.0:1776178146.079924:0:6257:0:(nrs_fifo.c:241:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.204.3@tcp, seq: 6516 00000100:00100000:0.0:1776178146.087613:0:3347:0:(events.c:361:request_in_callback()) peer: 12345-192.168.204.3@tcp (source: 12345-192.168.204.3@tcp) 00010000:00010000:1.0:1776178146.087887:0:3339:0:(ldlm_lock.c:1064:ldlm_granted_list_add_lock()) ### About to add lock: ns: mdt-lustre-MDT0000_UUID lock: 00000000bef7c62f/0xa40083ab9bf53207 lrc: 3/0,1 mode: COS/COS res: [0x2000013a1:0x1:0x0].0x5e5674ea bits 0x2/0x0 rrc: 2 type: IBT gid 0 flags: 0x50210000000000 nid: local remote: 0x0 expref: -99 pid: 6257 timeout: 0 lvb_type: 0 00010000:00010000:1.0:1776178146.087909:0:3339:0:(ldlm_inodebits.c:94:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x2000013a1:0x1:0x0].0x5e5674ea (00000000b86d0927) 00010000:00010000:1.0:1776178146.087913:0:3339:0:(ldlm_inodebits.c:96:ldlm_reprocess_inodebits_queue()) Hint 2 00010000:00010000:1.0:1776178146.087920:0:3339:0:(ldlm_lock.c:1064:ldlm_granted_list_add_lock()) ### About to add lock: ns: mdt-lustre-MDT0000_UUID lock: 000000006cb7334a/0xa40083ab9bf5320e lrc: 3/0,1 mode: COS/COS res: [0x2000013a1:0x4:0x0].0x0 bits 0x3/0x0 rrc: 3 type: IBT gid 0 flags: 0x50210000000000 nid: local remote: 0x0 expref: -99 pid: 6257 timeout: 0 lvb_type: 0 00010000:00010000:1.0:1776178146.087926:0:3339:0:(ldlm_inodebits.c:94:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x2000013a1:0x4:0x0].0x0 (000000009cfa2d69) 00010000:00010000:1.0:1776178146.087929:0:3339:0:(ldlm_inodebits.c:96:ldlm_reprocess_inodebits_queue()) Hint 3 00000100:00100000:2.0:1776178146.088256:0:8250:0:(service.c:2152:ptlrpc_server_handle_req_in()) got req x1862456979087616 00000100:00100000:2.0:1776178146.088324:0:8250:0:(nrs_fifo.c:179:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.204.3@tcp, seq: 6517 00000100:00100000:2.0:1776178146.088332:0:8250:0:(service.c:2324:ptlrpc_server_handle_request()) Handling RPC req@0000000077f0e23c pname:cluuid+ref:pid:xid:nid:opc:job mdt00_003:b50963bd-8eba-4432-9a54-8888d00e3fa1+39:17037:x1862456979087616:12345-192.168.204.3@tcp:36:rm.0 00010000:00010000:2.0:1776178146.088399:0:8250:0:(ldlm_lockd.c:1720:ldlm_request_cancel()) ### server-side cancel handler START: 2 locks, starting at 0 00010000:00010000:2.0:1776178146.088425:0:8250:0:(ldlm_lock.c:216:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: mdt-lustre-MDT0000_UUID lock: 000000007353d00c/0xa40083ab9bf53120 lrc: 0/0,0 mode: --/PR res: [0x2000013a1:0x2:0x0].0x0 bits 0x1b/0x0 rrc: 5 type: IBT gid 0 flags: 0x44a01000000000 nid: 192.168.204.3@tcp remote: 0x771f68aa1eb27a38 expref: 39 pid: 6257 timeout: 0 lvb_type: 0 00010000:00010000:2.0:1776178146.088446:0:8250:0:(ldlm_lock.c:216:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: mdt-lustre-MDT0000_UUID lock: 000000007e7ea26f/0xa40083ab9bf53104 lrc: 0/0,0 mode: --/CR res: [0x2000013a1:0x2:0x0].0x0 bits 0x8/0x0 rrc: 4 type: IBT gid 0 flags: 0x44801000000000 nid: 192.168.204.3@tcp remote: 0x771f68aa1eb27a15 expref: 38 pid: 6256 timeout: 0 lvb_type: 0 00010000:00010000:2.0:1776178146.088455:0:8250:0:(ldlm_inodebits.c:94:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x2000013a1:0x2:0x0].0x0 (00000000e4751492) 00010000:00010000:2.0:1776178146.088459:0:8250:0:(ldlm_lockd.c:1782:ldlm_request_cancel()) ### server-side cancel handler END 00010000:00010000:2.0:1776178146.088492:0:8250:0:(ldlm_lock.c:766:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(CW) ns: mdt-lustre-MDT0000_UUID lock: 00000000fcb32840/0xa40083ab9bf5321c lrc: 3/0,1 mode: --/CW res: [0x2000013a1:0x1:0x0].0x0 bits 0x0/0x0 rrc: 4 type: IBT gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 8250 timeout: 0 lvb_type: 0 00010000:00010000:2.0:1776178146.088506:0:8250:0:(ldlm_lock.c:1064:ldlm_granted_list_add_lock()) ### About to add lock: ns: mdt-lustre-MDT0000_UUID lock: 00000000fcb32840/0xa40083ab9bf5321c lrc: 3/0,1 mode: CW/CW res: [0x2000013a1:0x1:0x0].0x0 bits 0x2/0x0 rrc: 4 type: IBT gid 0 flags: 0x50210000000000 nid: local remote: 0x0 expref: -99 pid: 8250 timeout: 0 lvb_type: 0 00010000:00010000:2.0:1776178146.088514:0:8250:0:(ldlm_request.c:528:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: mdt-lustre-MDT0000_UUID lock: 00000000fcb32840/0xa40083ab9bf5321c lrc: 3/0,1 mode: CW/CW res: [0x2000013a1:0x1:0x0].0x0 bits 0x2/0x0 rrc: 4 type: IBT gid 0 flags: 0x40210000000000 nid: local remote: 0x0 expref: -99 pid: 8250 timeout: 0 lvb_type: 0 00010000:00010000:2.0:1776178146.088529:0:8250:0:(ldlm_lock.c:766:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: mdt-lustre-MDT0000_UUID lock: 000000006cc9c357/0xa40083ab9bf53223 lrc: 3/0,1 mode: --/PW res: [0x2000013a1:0x1:0x0].0xcda302ea bits 0x0/0x0 rrc: 2 type: IBT gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 8250 timeout: 0 lvb_type: 0 00010000:00010000:2.0:1776178146.088535:0:8250:0:(ldlm_lock.c:1064:ldlm_granted_list_add_lock()) ### About to add lock: ns: mdt-lustre-MDT0000_UUID lock: 000000006cc9c357/0xa40083ab9bf53223 lrc: 3/0,1 mode: PW/PW res: [0x2000013a1:0x1:0x0].0xcda302ea bits 0x2/0x0 rrc: 2 type: IBT gid 0 flags: 0x50210000000000 nid: local remote: 0x0 expref: -99 pid: 8250 timeout: 0 lvb_type: 0 00010000:00010000:2.0:1776178146.088540:0:8250:0:(ldlm_request.c:528:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: mdt-lustre-MDT0000_UUID lock: 000000006cc9c357/0xa40083ab9bf53223 lrc: 3/0,1 mode: PW/PW res: [0x2000013a1:0x1:0x0].0xcda302ea bits 0x2/0x0 rrc: 2 type: IBT gid 0 flags: 0x40210000000000 nid: local remote: 0x0 expref: -99 pid: 8250 timeout: 0 lvb_type: 0 00010000:00010000:2.0:1776178146.088635:0:8250:0:(ldlm_lock.c:766:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(EX) ns: mdt-lustre-MDT0000_UUID lock: 000000000b5c0143/0xa40083ab9bf5322a lrc: 3/0,1 mode: --/EX res: [0x2000013a1:0x2:0x0].0x0 bits 0x0/0x0 rrc: 3 type: IBT gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 8250 timeout: 0 lvb_type: 0 00010000:00010000:2.0:1776178146.088644:0:8250:0:(ldlm_lock.c:1064:ldlm_granted_list_add_lock()) ### About to add lock: ns: mdt-lustre-MDT0000_UUID lock: 000000000b5c0143/0xa40083ab9bf5322a lrc: 3/0,1 mode: EX/EX res: [0x2000013a1:0x2:0x0].0x0 bits 0x3/0x0 rrc: 3 type: IBT gid 0 flags: 0x50210000000000 nid: local remote: 0x0 expref: -99 pid: 8250 timeout: 0 lvb_type: 0 00010000:00010000:2.0:1776178146.088651:0:8250:0:(ldlm_request.c:528:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: mdt-lustre-MDT0000_UUID lock: 000000000b5c0143/0xa40083ab9bf5322a lrc: 3/0,1 mode: EX/EX res: [0x2000013a1:0x2:0x0].0x0 bits 0x3/0x0 rrc: 3 type: IBT gid 0 flags: 0x40210000000000 nid: local remote: 0x0 expref: -99 pid: 8250 timeout: 0 lvb_type: 0 00000040:00080000:2.0:1776178146.088882:0:8250:0:(llog_osd.c:738:llog_osd_write_rec()) added record [0x1:0xc:0x0].4, 64 off8448 00010000:00010000:2.0:1776178146.088929:0:8250:0:(ldlm_lock.c:766:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: mdt-lustre-MDT0000_UUID lock: 0000000096c1e297/0xa40083ab9bf53231 lrc: 3/0,1 mode: --/PW res: [0x2000013a1:0x2:0x0].0x0 bits 0x0/0x0 rrc: 4 type: IBT gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 8250 timeout: 0 lvb_type: 0 00010000:00010000:2.0:1776178146.088943:0:8250:0:(ldlm_lock.c:1064:ldlm_granted_list_add_lock()) ### About to add lock: ns: mdt-lustre-MDT0000_UUID lock: 0000000096c1e297/0xa40083ab9bf53231 lrc: 3/0,1 mode: PW/PW res: [0x2000013a1:0x2:0x0].0x0 bits 0x40/0x0 rrc: 4 type: IBT gid 18446635898608079296 flags: 0x50010080000000 nid: local remote: 0x0 expref: -99 pid: 8250 timeout: 0 lvb_type: 0 00010000:00010000:2.0:1776178146.088950:0:8250:0:(ldlm_request.c:528:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: mdt-lustre-MDT0000_UUID lock: 0000000096c1e297/0xa40083ab9bf53231 lrc: 3/0,1 mode: PW/PW res: [0x2000013a1:0x2:0x0].0x0 bits 0x40/0x0 rrc: 4 type: IBT gid 18446635898608079296 flags: 0x40010080000000 nid: local remote: 0x0 expref: -99 pid: 8250 timeout: 0 lvb_type: 0 00010000:00010000:2.0:1776178146.088956:0:8250:0:(ldlm_lock.c:942:ldlm_lock_decref_and_cancel()) ### ldlm_lock_decref(PW) ns: mdt-lustre-MDT0000_UUID lock: 0000000096c1e297/0xa40083ab9bf53231 lrc: 4/0,1 mode: PW/PW res: [0x2000013a1:0x2:0x0].0x0 bits 0x40/0x0 rrc: 4 type: IBT gid 18446635898608079296 flags: 0x40010080000000 nid: local remote: 0x0 expref: -99 pid: 8250 timeout: 0 lvb_type: 0 00010000:00010000:2.0:1776178146.088962:0:8250:0:(ldlm_lock.c:820:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: mdt-lustre-MDT0000_UUID lock: 0000000096c1e297/0xa40083ab9bf53231 lrc: 4/0,1 mode: PW/PW res: [0x2000013a1:0x2:0x0].0x0 bits 0x40/0x0 rrc: 4 type: IBT gid 18446635898608079296 flags: 0x50010480000000 nid: local remote: 0x0 expref: -99 pid: 8250 timeout: 0 lvb_type: 0 00010000:00010000:2.0:1776178146.088968:0:8250:0:(ldlm_lock.c:879:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: mdt-lustre-MDT0000_UUID lock: 0000000096c1e297/0xa40083ab9bf53231 lrc: 3/0,0 mode: PW/PW res: [0x2000013a1:0x2:0x0].0x0 bits 0x40/0x0 rrc: 4 type: IBT gid 18446635898608079296 flags: 0x50010480000000 nid: local remote: 0x0 expref: -99 pid: 8250 timeout: 0 lvb_type: 0 00010000:00010000:2.0:1776178146.088996:0:8250:0:(ldlm_lockd.c:2159:__ldlm_bl_to_thread()) added 1/1 locks to regular blp list, 1 blwis in pool 00000004:00010000:2.0:1776178146.089028:0:8250:0:(mdt_handler.c:3992:mdt_save_lock()) ### save lock request 0000000077f0e23c reply state 00000000b99f431e transno 12884901936 ns: mdt-lustre-MDT0000_UUID lock: 000000000b5c0143/0xa40083ab9bf5322a lrc: 3/0,1 mode: EX/EX res: [0x2000013a1:0x2:0x0].0x0 bits 0x3/0x0 rrc: 4 type: IBT gid 0 flags: 0x40210000000000 nid: local remote: 0x0 expref: -99 pid: 8250 timeout: 0 lvb_type: 0 00010000:00010000:1.0:1776178146.089416:0:6245:0:(ldlm_lockd.c:2789:ldlm_bl_get_work()) Got 1 locks of 0 total in blp. (0 blwis in pool) 00010000:00010000:1.0:1776178146.089423:0:6245:0:(ldlm_lockd.c:1879:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: mdt-lustre-MDT0000_UUID lock: 0000000096c1e297/0xa40083ab9bf53231 lrc: 2/0,0 mode: PW/PW res: [0x2000013a1:0x2:0x0].0x0 bits 0x40/0x0 rrc: 4 type: IBT gid 18446635898608079296 flags: 0x40010480000000 nid: local remote: 0x0 expref: -99 pid: 8250 timeout: 0 lvb_type: 0 00010000:00010000:1.0:1776178146.089432:0:6245:0:(ldlm_lockd.c:1893:ldlm_handle_bl_callback()) Lock 0000000096c1e297 already unused, calling callback (0000000063c97258) 00010000:00010000:1.0:1776178146.089435:0:6245:0:(ldlm_request.c:364:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: mdt-lustre-MDT0000_UUID lock: 0000000096c1e297/0xa40083ab9bf53231 lrc: 2/0,0 mode: PW/PW res: [0x2000013a1:0x2:0x0].0x0 bits 0x40/0x0 rrc: 4 type: IBT gid 18446635898608079296 flags: 0x40010480000000 nid: local remote: 0x0 expref: -99 pid: 8250 timeout: 0 lvb_type: 0 00010000:00010000:1.0:1776178146.089442:0:6245:0:(ldlm_request.c:1257:ldlm_cli_cancel_local()) ### server-side local cancel ns: mdt-lustre-MDT0000_UUID lock: 0000000096c1e297/0xa40083ab9bf53231 lrc: 3/0,0 mode: PW/PW res: [0x2000013a1:0x2:0x0].0x0 bits 0x40/0x0 rrc: 4 type: IBT gid 18446635898608079296 flags: 0x40018480000000 nid: local remote: 0x0 expref: -99 pid: 8250 timeout: 0 lvb_type: 0 00010000:00010000:1.0:1776178146.089454:0:6245:0:(ldlm_inodebits.c:94:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x2000013a1:0x2:0x0].0x0 (00000000e4751492) 00010000:00010000:1.0:1776178146.089456:0:6245:0:(ldlm_inodebits.c:96:ldlm_reprocess_inodebits_queue()) Hint 40 00010000:00010000:1.0:1776178146.089460:0:6245:0:(ldlm_lockd.c:1903:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: mdt-lustre-MDT0000_UUID lock: 0000000096c1e297/0xa40083ab9bf53231 lrc: 1/0,0 mode: --/PW res: [0x2000013a1:0x2:0x0].0x0 bits 0x40/0x0 rrc: 4 type: IBT gid 18446635898608079296 flags: 0x44819480000000 nid: local remote: 0x0 expref: -99 pid: 8250 timeout: 0 lvb_type: 0 00010000:00010000:1.0:1776178146.089465:0:6245:0:(ldlm_lock.c:216:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: mdt-lustre-MDT0000_UUID lock: 0000000096c1e297/0xa40083ab9bf53231 lrc: 0/0,0 mode: --/PW res: [0x2000013a1:0x2:0x0].0x0 bits 0x40/0x0 rrc: 4 type: IBT gid 18446635898608079296 flags: 0x44819480000000 nid: local remote: 0x0 expref: -99 pid: 8250 timeout: 0 lvb_type: 0 00010000:00010000:1.0:1776178146.089478:0:6245:0:(ldlm_lockd.c:2792:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00010000:00010000:1.0:1776178146.089480:0:6245:0:(ldlm_lockd.c:2792:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00010000:00010000:1.0:1776178146.089481:0:6245:0:(ldlm_lockd.c:2792:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00010000:00010000:2.0:1776178146.092514:0:8250:0:(ldlm_lock.c:820:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(CW) ns: mdt-lustre-MDT0000_UUID lock: 00000000fcb32840/0xa40083ab9bf5321c lrc: 3/0,1 mode: CW/CW res: [0x2000013a1:0x1:0x0].0x0 bits 0x2/0x0 rrc: 4 type: IBT gid 0 flags: 0x50210000000000 nid: local remote: 0x0 expref: -99 pid: 8250 timeout: 0 lvb_type: 0 00010000:00010000:2.0:1776178146.092536:0:8250:0:(ldlm_lock.c:879:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: mdt-lustre-MDT0000_UUID lock: 00000000fcb32840/0xa40083ab9bf5321c lrc: 2/0,0 mode: CW/CW res: [0x2000013a1:0x1:0x0].0x0 bits 0x2/0x0 rrc: 4 type: IBT gid 0 flags: 0x50210400000000 nid: local remote: 0x0 expref: -99 pid: 8250 timeout: 0 lvb_type: 0 00010000:00010000:2.0:1776178146.092545:0:8250:0:(ldlm_lockd.c:1879:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: mdt-lustre-MDT0000_UUID lock: 00000000fcb32840/0xa40083ab9bf5321c lrc: 3/0,0 mode: CW/CW res: [0x2000013a1:0x1:0x0].0x0 bits 0x2/0x0 rrc: 4 type: IBT gid 0 flags: 0x40210400000000 nid: local remote: 0x0 expref: -99 pid: 8250 timeout: 0 lvb_type: 0 00010000:00010000:2.0:1776178146.092551:0:8250:0:(ldlm_lockd.c:1893:ldlm_handle_bl_callback()) Lock 00000000fcb32840 already unused, calling callback (0000000079e78348) 00010000:00010000:2.0:1776178146.092556:0:8250:0:(ldlm_request.c:364:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: mdt-lustre-MDT0000_UUID lock: 00000000fcb32840/0xa40083ab9bf5321c lrc: 3/0,0 mode: CW/CW res: [0x2000013a1:0x1:0x0].0x0 bits 0x2/0x0 rrc: 4 type: IBT gid 0 flags: 0x40210400000000 nid: local remote: 0x0 expref: -99 pid: 8250 timeout: 0 lvb_type: 0 00010000:00010000:2.0:1776178146.092563:0:8250:0:(ldlm_request.c:1257:ldlm_cli_cancel_local()) ### server-side local cancel ns: mdt-lustre-MDT0000_UUID lock: 00000000fcb32840/0xa40083ab9bf5321c lrc: 4/0,0 mode: CW/CW res: [0x2000013a1:0x1:0x0].0x0 bits 0x2/0x0 rrc: 4 type: IBT gid 0 flags: 0x40218400000000 nid: local remote: 0x0 expref: -99 pid: 8250 timeout: 0 lvb_type: 0 00010000:00010000:2.0:1776178146.092581:0:8250:0:(ldlm_inodebits.c:94:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x2000013a1:0x1:0x0].0x0 (00000000315facbe) 00010000:00010000:2.0:1776178146.092584:0:8250:0:(ldlm_inodebits.c:96:ldlm_reprocess_inodebits_queue()) Hint 2 00010000:00010000:2.0:1776178146.092588:0:8250:0:(ldlm_lockd.c:1903:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: mdt-lustre-MDT0000_UUID lock: 00000000fcb32840/0xa40083ab9bf5321c lrc: 2/0,0 mode: --/CW res: [0x2000013a1:0x1:0x0].0x0 bits 0x2/0x0 rrc: 4 type: IBT gid 0 flags: 0x44a19400000000 nid: local remote: 0x0 expref: -99 pid: 8250 timeout: 0 lvb_type: 0 00010000:00010000:2.0:1776178146.092593:0:8250:0:(ldlm_lock.c:216:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: mdt-lustre-MDT0000_UUID lock: 00000000fcb32840/0xa40083ab9bf5321c lrc: 0/0,0 mode: --/CW res: [0x2000013a1:0x1:0x0].0x0 bits 0x2/0x0 rrc: 4 type: IBT gid 0 flags: 0x44a19400000000 nid: local remote: 0x0 expref: -99 pid: 8250 timeout: 0 lvb_type: 0 00000004:00010000:2.0:1776178146.092604:0:8250:0:(mdt_handler.c:3992:mdt_save_lock()) ### save lock request 0000000077f0e23c reply state 00000000b99f431e transno 12884901936 ns: mdt-lustre-MDT0000_UUID lock: 000000006cc9c357/0xa40083ab9bf53223 lrc: 3/0,1 mode: PW/PW res: [0x2000013a1:0x1:0x0].0xcda302ea bits 0x2/0x0 rrc: 2 type: IBT gid 0 flags: 0x40210000000000 nid: local remote: 0x0 expref: -99 pid: 8250 timeout: 0 lvb_type: 0 00000100:00100000:2.0:1776178146.092720:0:8250:0:(service.c:2372:ptlrpc_server_handle_request()) Handled RPC req@0000000077f0e23c pname:cluuid+ref:pid:xid:nid:opc:job mdt00_003:b50963bd-8eba-4432-9a54-8888d00e3fa1+39:17037:x1862456979087616:12345-192.168.204.3@tcp:36:rm.0 Request processed in 4384us (5115us total) trans 12884901936 rc 0/0 00000100:00100000:2.0:1776178146.092729:0:8250:0:(nrs_fifo.c:241:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.204.3@tcp, seq: 6517 00010000:00010000:2.0:1776178146.095624:0:3340:0:(ldlm_lock.c:1064:ldlm_granted_list_add_lock()) ### About to add lock: ns: mdt-lustre-MDT0000_UUID lock: 000000006cc9c357/0xa40083ab9bf53223 lrc: 3/0,1 mode: COS/COS res: [0x2000013a1:0x1:0x0].0xcda302ea bits 0x2/0x0 rrc: 2 type: IBT gid 0 flags: 0x50210000000000 nid: local remote: 0x0 expref: -99 pid: 8250 timeout: 0 lvb_type: 0 00010000:00010000:2.0:1776178146.095649:0:3340:0:(ldlm_inodebits.c:94:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x2000013a1:0x1:0x0].0xcda302ea (0000000043c2d1ef) 00010000:00010000:2.0:1776178146.095652:0:3340:0:(ldlm_inodebits.c:96:ldlm_reprocess_inodebits_queue()) Hint 2 00010000:00010000:2.0:1776178146.095657:0:3340:0:(ldlm_lock.c:1064:ldlm_granted_list_add_lock()) ### About to add lock: ns: mdt-lustre-MDT0000_UUID lock: 000000000b5c0143/0xa40083ab9bf5322a lrc: 3/0,1 mode: COS/COS res: [0x2000013a1:0x2:0x0].0x0 bits 0x3/0x0 rrc: 3 type: IBT gid 0 flags: 0x50210000000000 nid: local remote: 0x0 expref: -99 pid: 8250 timeout: 0 lvb_type: 0 00010000:00010000:2.0:1776178146.095664:0:3340:0:(ldlm_inodebits.c:94:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x2000013a1:0x2:0x0].0x0 (00000000e4751492) 00010000:00010000:2.0:1776178146.095666:0:3340:0:(ldlm_inodebits.c:96:ldlm_reprocess_inodebits_queue()) Hint 3 00000100:00100000:0.0:1776178146.096857:0:3347:0:(events.c:361:request_in_callback()) peer: 12345-192.168.204.3@tcp (source: 12345-192.168.204.3@tcp) 00000100:00100000:2.0:1776178146.096948:0:8250:0:(service.c:2152:ptlrpc_server_handle_req_in()) got req x1862456979087744 00000100:00100000:2.0:1776178146.097020:0:8250:0:(nrs_fifo.c:179:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.204.3@tcp, seq: 6518 00000100:00100000:2.0:1776178146.097027:0:8250:0:(service.c:2324:ptlrpc_server_handle_request()) Handling RPC req@00000000a782bcc4 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_003:b50963bd-8eba-4432-9a54-8888d00e3fa1+39:17037:x1862456979087744:12345-192.168.204.3@tcp:36:rm.0 00010000:00010000:2.0:1776178146.097448:0:8250:0:(ldlm_lockd.c:1720:ldlm_request_cancel()) ### server-side cancel handler START: 2 locks, starting at 0 00010000:00010000:2.0:1776178146.097478:0:8250:0:(ldlm_lock.c:216:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: mdt-lustre-MDT0000_UUID lock: 000000008cbcbaf5/0xa40083ab9bf53190 lrc: 0/0,0 mode: --/PR res: [0x2000013a1:0x6:0x0].0x0 bits 0x1b/0x0 rrc: 5 type: IBT gid 0 flags: 0x44a01000000000 nid: 192.168.204.3@tcp remote: 0x771f68aa1eb27a54 expref: 39 pid: 6257 timeout: 0 lvb_type: 0 00010000:00010000:2.0:1776178146.097503:0:8250:0:(ldlm_lock.c:216:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: mdt-lustre-MDT0000_UUID lock: 00000000f190b39a/0xa40083ab9bf530e8 lrc: 0/0,0 mode: --/CR res: [0x2000013a1:0x6:0x0].0x0 bits 0x8/0x0 rrc: 4 type: IBT gid 0 flags: 0x44801000000000 nid: 192.168.204.3@tcp remote: 0x771f68aa1eb27a1c expref: 38 pid: 8250 timeout: 0 lvb_type: 0 00010000:00010000:2.0:1776178146.097513:0:8250:0:(ldlm_inodebits.c:94:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x2000013a1:0x6:0x0].0x0 (0000000019a0327e) 00010000:00010000:2.0:1776178146.097517:0:8250:0:(ldlm_lockd.c:1782:ldlm_request_cancel()) ### server-side cancel handler END 00010000:00010000:2.0:1776178146.097550:0:8250:0:(ldlm_lock.c:766:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(CW) ns: mdt-lustre-MDT0000_UUID lock: 00000000540a8f8b/0xa40083ab9bf53238 lrc: 3/0,1 mode: --/CW res: [0x2000013a1:0x1:0x0].0x0 bits 0x0/0x0 rrc: 4 type: IBT gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 8250 timeout: 0 lvb_type: 0 00010000:00010000:2.0:1776178146.097564:0:8250:0:(ldlm_lock.c:1064:ldlm_granted_list_add_lock()) ### About to add lock: ns: mdt-lustre-MDT0000_UUID lock: 00000000540a8f8b/0xa40083ab9bf53238 lrc: 3/0,1 mode: CW/CW res: [0x2000013a1:0x1:0x0].0x0 bits 0x2/0x0 rrc: 4 type: IBT gid 0 flags: 0x50210000000000 nid: local remote: 0x0 expref: -99 pid: 8250 timeout: 0 lvb_type: 0 00010000:00010000:2.0:1776178146.097574:0:8250:0:(ldlm_request.c:528:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: mdt-lustre-MDT0000_UUID lock: 00000000540a8f8b/0xa40083ab9bf53238 lrc: 3/0,1 mode: CW/CW res: [0x2000013a1:0x1:0x0].0x0 bits 0x2/0x0 rrc: 4 type: IBT gid 0 flags: 0x40210000000000 nid: local remote: 0x0 expref: -99 pid: 8250 timeout: 0 lvb_type: 0 00010000:00010000:2.0:1776178146.097589:0:8250:0:(ldlm_lock.c:766:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: mdt-lustre-MDT0000_UUID lock: 0000000025b13886/0xa40083ab9bf5323f lrc: 3/0,1 mode: --/PW res: [0x2000013a1:0x1:0x0].0xef09e6ea bits 0x0/0x0 rrc: 2 type: IBT gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 8250 timeout: 0 lvb_type: 0 00010000:00010000:2.0:1776178146.097596:0:8250:0:(ldlm_lock.c:1064:ldlm_granted_list_add_lock()) ### About to add lock: ns: mdt-lustre-MDT0000_UUID lock: 0000000025b13886/0xa40083ab9bf5323f lrc: 3/0,1 mode: PW/PW res: [0x2000013a1:0x1:0x0].0xef09e6ea bits 0x2/0x0 rrc: 2 type: IBT gid 0 flags: 0x50210000000000 nid: local remote: 0x0 expref: -99 pid: 8250 timeout: 0 lvb_type: 0 00010000:00010000:2.0:1776178146.097601:0:8250:0:(ldlm_request.c:528:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: mdt-lustre-MDT0000_UUID lock: 0000000025b13886/0xa40083ab9bf5323f lrc: 3/0,1 mode: PW/PW res: [0x2000013a1:0x1:0x0].0xef09e6ea bits 0x2/0x0 rrc: 2 type: IBT gid 0 flags: 0x40210000000000 nid: local remote: 0x0 expref: -99 pid: 8250 timeout: 0 lvb_type: 0 00010000:00010000:2.0:1776178146.097678:0:8250:0:(ldlm_lock.c:766:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(EX) ns: mdt-lustre-MDT0000_UUID lock: 00000000f779de26/0xa40083ab9bf53246 lrc: 3/0,1 mode: --/EX res: [0x2000013a1:0x6:0x0].0x0 bits 0x0/0x0 rrc: 3 type: IBT gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 8250 timeout: 0 lvb_type: 0 00010000:00010000:2.0:1776178146.097687:0:8250:0:(ldlm_lock.c:1064:ldlm_granted_list_add_lock()) ### About to add lock: ns: mdt-lustre-MDT0000_UUID lock: 00000000f779de26/0xa40083ab9bf53246 lrc: 3/0,1 mode: EX/EX res: [0x2000013a1:0x6:0x0].0x0 bits 0x3/0x0 rrc: 3 type: IBT gid 0 flags: 0x50210000000000 nid: local remote: 0x0 expref: -99 pid: 8250 timeout: 0 lvb_type: 0 00010000:00010000:2.0:1776178146.097694:0:8250:0:(ldlm_request.c:528:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: mdt-lustre-MDT0000_UUID lock: 00000000f779de26/0xa40083ab9bf53246 lrc: 3/0,1 mode: EX/EX res: [0x2000013a1:0x6:0x0].0x0 bits 0x3/0x0 rrc: 3 type: IBT gid 0 flags: 0x40210000000000 nid: local remote: 0x0 expref: -99 pid: 8250 timeout: 0 lvb_type: 0 00000040:00080000:2.0:1776178146.097913:0:8250:0:(llog_osd.c:738:llog_osd_write_rec()) added record [0x1:0xc:0x0].5, 64 off8512 00010000:00010000:2.0:1776178146.097964:0:8250:0:(ldlm_lock.c:766:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: mdt-lustre-MDT0000_UUID lock: 00000000c9d7d9fc/0xa40083ab9bf5324d lrc: 3/0,1 mode: --/PW res: [0x2000013a1:0x6:0x0].0x0 bits 0x0/0x0 rrc: 4 type: IBT gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 8250 timeout: 0 lvb_type: 0 00010000:00010000:2.0:1776178146.097973:0:8250:0:(ldlm_lock.c:1064:ldlm_granted_list_add_lock()) ### About to add lock: ns: mdt-lustre-MDT0000_UUID lock: 00000000c9d7d9fc/0xa40083ab9bf5324d lrc: 3/0,1 mode: PW/PW res: [0x2000013a1:0x6:0x0].0x0 bits 0x40/0x0 rrc: 4 type: IBT gid 18446635898608079296 flags: 0x50010080000000 nid: local remote: 0x0 expref: -99 pid: 8250 timeout: 0 lvb_type: 0 00010000:00010000:2.0:1776178146.097981:0:8250:0:(ldlm_request.c:528:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: mdt-lustre-MDT0000_UUID lock: 00000000c9d7d9fc/0xa40083ab9bf5324d lrc: 3/0,1 mode: PW/PW res: [0x2000013a1:0x6:0x0].0x0 bits 0x40/0x0 rrc: 4 type: IBT gid 18446635898608079296 flags: 0x40010080000000 nid: local remote: 0x0 expref: -99 pid: 8250 timeout: 0 lvb_type: 0 00010000:00010000:2.0:1776178146.097989:0:8250:0:(ldlm_lock.c:942:ldlm_lock_decref_and_cancel()) ### ldlm_lock_decref(PW) ns: mdt-lustre-MDT0000_UUID lock: 00000000c9d7d9fc/0xa40083ab9bf5324d lrc: 4/0,1 mode: PW/PW res: [0x2000013a1:0x6:0x0].0x0 bits 0x40/0x0 rrc: 4 type: IBT gid 18446635898608079296 flags: 0x40010080000000 nid: local remote: 0x0 expref: -99 pid: 8250 timeout: 0 lvb_type: 0 00010000:00010000:2.0:1776178146.098008:0:8250:0:(ldlm_lock.c:820:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: mdt-lustre-MDT0000_UUID lock: 00000000c9d7d9fc/0xa40083ab9bf5324d lrc: 4/0,1 mode: PW/PW res: [0x2000013a1:0x6:0x0].0x0 bits 0x40/0x0 rrc: 4 type: IBT gid 18446635898608079296 flags: 0x50010480000000 nid: local remote: 0x0 expref: -99 pid: 8250 timeout: 0 lvb_type: 0 00010000:00010000:2.0:1776178146.098014:0:8250:0:(ldlm_lock.c:879:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: mdt-lustre-MDT0000_UUID lock: 00000000c9d7d9fc/0xa40083ab9bf5324d lrc: 3/0,0 mode: PW/PW res: [0x2000013a1:0x6:0x0].0x0 bits 0x40/0x0 rrc: 4 type: IBT gid 18446635898608079296 flags: 0x50010480000000 nid: local remote: 0x0 expref: -99 pid: 8250 timeout: 0 lvb_type: 0 00010000:00010000:2.0:1776178146.098109:0:8250:0:(ldlm_lockd.c:2159:__ldlm_bl_to_thread()) added 1/1 locks to regular blp list, 1 blwis in pool 00000004:00010000:2.0:1776178146.098127:0:8250:0:(mdt_handler.c:3992:mdt_save_lock()) ### save lock request 00000000a782bcc4 reply state 000000007f2836aa transno 12884901937 ns: mdt-lustre-MDT0000_UUID lock: 00000000f779de26/0xa40083ab9bf53246 lrc: 3/0,1 mode: EX/EX res: [0x2000013a1:0x6:0x0].0x0 bits 0x3/0x0 rrc: 4 type: IBT gid 0 flags: 0x40210000000000 nid: local remote: 0x0 expref: -99 pid: 8250 timeout: 0 lvb_type: 0 00010000:00010000:0.0:1776178146.100788:0:8250:0:(ldlm_lock.c:820:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(CW) ns: mdt-lustre-MDT0000_UUID lock: 00000000540a8f8b/0xa40083ab9bf53238 lrc: 3/0,1 mode: CW/CW res: [0x2000013a1:0x1:0x0].0x0 bits 0x2/0x0 rrc: 4 type: IBT gid 0 flags: 0x50210000000000 nid: local remote: 0x0 expref: -99 pid: 8250 timeout: 0 lvb_type: 0 00010000:00010000:0.0:1776178146.100819:0:8250:0:(ldlm_lock.c:879:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: mdt-lustre-MDT0000_UUID lock: 00000000540a8f8b/0xa40083ab9bf53238 lrc: 2/0,0 mode: CW/CW res: [0x2000013a1:0x1:0x0].0x0 bits 0x2/0x0 rrc: 4 type: IBT gid 0 flags: 0x50210400000000 nid: local remote: 0x0 expref: -99 pid: 8250 timeout: 0 lvb_type: 0 00010000:00010000:0.0:1776178146.100827:0:8250:0:(ldlm_lockd.c:1879:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: mdt-lustre-MDT0000_UUID lock: 00000000540a8f8b/0xa40083ab9bf53238 lrc: 3/0,0 mode: CW/CW res: [0x2000013a1:0x1:0x0].0x0 bits 0x2/0x0 rrc: 4 type: IBT gid 0 flags: 0x40210400000000 nid: local remote: 0x0 expref: -99 pid: 8250 timeout: 0 lvb_type: 0 00010000:00010000:0.0:1776178146.100838:0:8250:0:(ldlm_lockd.c:1893:ldlm_handle_bl_callback()) Lock 00000000540a8f8b already unused, calling callback (0000000079e78348) 00010000:00010000:0.0:1776178146.100842:0:8250:0:(ldlm_request.c:364:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: mdt-lustre-MDT0000_UUID lock: 00000000540a8f8b/0xa40083ab9bf53238 lrc: 3/0,0 mode: CW/CW res: [0x2000013a1:0x1:0x0].0x0 bits 0x2/0x0 rrc: 4 type: IBT gid 0 flags: 0x40210400000000 nid: local remote: 0x0 expref: -99 pid: 8250 timeout: 0 lvb_type: 0 00010000:00010000:0.0:1776178146.100849:0:8250:0:(ldlm_request.c:1257:ldlm_cli_cancel_local()) ### server-side local cancel ns: mdt-lustre-MDT0000_UUID lock: 00000000540a8f8b/0xa40083ab9bf53238 lrc: 4/0,0 mode: CW/CW res: [0x2000013a1:0x1:0x0].0x0 bits 0x2/0x0 rrc: 4 type: IBT gid 0 flags: 0x40218400000000 nid: local remote: 0x0 expref: -99 pid: 8250 timeout: 0 lvb_type: 0 00010000:00010000:0.0:1776178146.100867:0:8250:0:(ldlm_inodebits.c:94:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x2000013a1:0x1:0x0].0x0 (00000000315facbe) 00010000:00010000:0.0:1776178146.100869:0:8250:0:(ldlm_inodebits.c:96:ldlm_reprocess_inodebits_queue()) Hint 2 00010000:00010000:0.0:1776178146.100873:0:8250:0:(ldlm_lockd.c:1903:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: mdt-lustre-MDT0000_UUID lock: 00000000540a8f8b/0xa40083ab9bf53238 lrc: 2/0,0 mode: --/CW res: [0x2000013a1:0x1:0x0].0x0 bits 0x2/0x0 rrc: 4 type: IBT gid 0 flags: 0x44a19400000000 nid: local remote: 0x0 expref: -99 pid: 8250 timeout: 0 lvb_type: 0 00010000:00010000:0.0:1776178146.100878:0:8250:0:(ldlm_lock.c:216:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: mdt-lustre-MDT0000_UUID lock: 00000000540a8f8b/0xa40083ab9bf53238 lrc: 0/0,0 mode: --/CW res: [0x2000013a1:0x1:0x0].0x0 bits 0x2/0x0 rrc: 4 type: IBT gid 0 flags: 0x44a19400000000 nid: local remote: 0x0 expref: -99 pid: 8250 timeout: 0 lvb_type: 0 00000004:00010000:0.0:1776178146.100888:0:8250:0:(mdt_handler.c:3992:mdt_save_lock()) ### save lock request 00000000a782bcc4 reply state 000000007f2836aa transno 12884901937 ns: mdt-lustre-MDT0000_UUID lock: 0000000025b13886/0xa40083ab9bf5323f lrc: 3/0,1 mode: PW/PW res: [0x2000013a1:0x1:0x0].0xef09e6ea bits 0x2/0x0 rrc: 2 type: IBT gid 0 flags: 0x40210000000000 nid: local remote: 0x0 expref: -99 pid: 8250 timeout: 0 lvb_type: 0 00000100:00100000:0.0:1776178146.101027:0:8250:0:(service.c:2372:ptlrpc_server_handle_request()) Handled RPC req@00000000a782bcc4 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_003:b50963bd-8eba-4432-9a54-8888d00e3fa1+39:17037:x1862456979087744:12345-192.168.204.3@tcp:36:rm.0 Request processed in 3996us (4173us total) trans 12884901937 rc 0/0 00000100:00100000:0.0:1776178146.101037:0:8250:0:(nrs_fifo.c:241:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.204.3@tcp, seq: 6518 00010000:00010000:1.0:1776178146.101497:0:9225:0:(ldlm_lockd.c:2789:ldlm_bl_get_work()) Got 1 locks of 0 total in blp. (0 blwis in pool) 00010000:00010000:1.0:1776178146.101504:0:9225:0:(ldlm_lockd.c:1879:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: mdt-lustre-MDT0000_UUID lock: 00000000c9d7d9fc/0xa40083ab9bf5324d lrc: 2/0,0 mode: PW/PW res: [0x2000013a1:0x6:0x0].0x0 bits 0x40/0x0 rrc: 4 type: IBT gid 18446635898608079296 flags: 0x40010480000000 nid: local remote: 0x0 expref: -99 pid: 8250 timeout: 0 lvb_type: 0 00010000:00010000:1.0:1776178146.101514:0:9225:0:(ldlm_lockd.c:1893:ldlm_handle_bl_callback()) Lock 00000000c9d7d9fc already unused, calling callback (0000000063c97258) 00010000:00010000:1.0:1776178146.101518:0:9225:0:(ldlm_request.c:364:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: mdt-lustre-MDT0000_UUID lock: 00000000c9d7d9fc/0xa40083ab9bf5324d lrc: 2/0,0 mode: PW/PW res: [0x2000013a1:0x6:0x0].0x0 bits 0x40/0x0 rrc: 4 type: IBT gid 18446635898608079296 flags: 0x40010480000000 nid: local remote: 0x0 expref: -99 pid: 8250 timeout: 0 lvb_type: 0 00010000:00010000:1.0:1776178146.101530:0:9225:0:(ldlm_request.c:1257:ldlm_cli_cancel_local()) ### server-side local cancel ns: mdt-lustre-MDT0000_UUID lock: 00000000c9d7d9fc/0xa40083ab9bf5324d lrc: 3/0,0 mode: PW/PW res: [0x2000013a1:0x6:0x0].0x0 bits 0x40/0x0 rrc: 4 type: IBT gid 18446635898608079296 flags: 0x40018480000000 nid: local remote: 0x0 expref: -99 pid: 8250 timeout: 0 lvb_type: 0 00010000:00010000:1.0:1776178146.101541:0:9225:0:(ldlm_inodebits.c:94:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x2000013a1:0x6:0x0].0x0 (0000000019a0327e) 00010000:00010000:1.0:1776178146.101543:0:9225:0:(ldlm_inodebits.c:96:ldlm_reprocess_inodebits_queue()) Hint 40 00010000:00010000:1.0:1776178146.101546:0:9225:0:(ldlm_lockd.c:1903:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: mdt-lustre-MDT0000_UUID lock: 00000000c9d7d9fc/0xa40083ab9bf5324d lrc: 1/0,0 mode: --/PW res: [0x2000013a1:0x6:0x0].0x0 bits 0x40/0x0 rrc: 4 type: IBT gid 18446635898608079296 flags: 0x44819480000000 nid: local remote: 0x0 expref: -99 pid: 8250 timeout: 0 lvb_type: 0 00010000:00010000:1.0:1776178146.101552:0:9225:0:(ldlm_lock.c:216:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: mdt-lustre-MDT0000_UUID lock: 00000000c9d7d9fc/0xa40083ab9bf5324d lrc: 0/0,0 mode: --/PW res: [0x2000013a1:0x6:0x0].0x0 bits 0x40/0x0 rrc: 4 type: IBT gid 18446635898608079296 flags: 0x44819480000000 nid: local remote: 0x0 expref: -99 pid: 8250 timeout: 0 lvb_type: 0 00010000:00010000:0.0:1776178146.101611:0:9225:0:(ldlm_lockd.c:2792:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00010000:00010000:0.0:1776178146.101613:0:9225:0:(ldlm_lockd.c:2792:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00010000:00010000:0.0:1776178146.101615:0:9225:0:(ldlm_lockd.c:2792:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00010000:00010000:2.0:1776178146.105256:0:3339:0:(ldlm_lock.c:1064:ldlm_granted_list_add_lock()) ### About to add lock: ns: mdt-lustre-MDT0000_UUID lock: 0000000025b13886/0xa40083ab9bf5323f lrc: 3/0,1 mode: COS/COS res: [0x2000013a1:0x1:0x0].0xef09e6ea bits 0x2/0x0 rrc: 2 type: IBT gid 0 flags: 0x50210000000000 nid: local remote: 0x0 expref: -99 pid: 8250 timeout: 0 lvb_type: 0 00010000:00010000:2.0:1776178146.105282:0:3339:0:(ldlm_inodebits.c:94:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x2000013a1:0x1:0x0].0xef09e6ea (00000000c31449d9) 00010000:00010000:2.0:1776178146.105284:0:3339:0:(ldlm_inodebits.c:96:ldlm_reprocess_inodebits_queue()) Hint 2 00010000:00010000:2.0:1776178146.105292:0:3339:0:(ldlm_lock.c:1064:ldlm_granted_list_add_lock()) ### About to add lock: ns: mdt-lustre-MDT0000_UUID lock: 00000000f779de26/0xa40083ab9bf53246 lrc: 3/0,1 mode: COS/COS res: [0x2000013a1:0x6:0x0].0x0 bits 0x3/0x0 rrc: 3 type: IBT gid 0 flags: 0x50210000000000 nid: local remote: 0x0 expref: -99 pid: 8250 timeout: 0 lvb_type: 0 00010000:00010000:2.0:1776178146.105451:0:3339:0:(ldlm_inodebits.c:94:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x2000013a1:0x6:0x0].0x0 (0000000019a0327e) 00010000:00010000:2.0:1776178146.105454:0:3339:0:(ldlm_inodebits.c:96:ldlm_reprocess_inodebits_queue()) Hint 3 00000100:00100000:0.0:1776178146.106862:0:3347:0:(events.c:361:request_in_callback()) peer: 12345-192.168.204.3@tcp (source: 12345-192.168.204.3@tcp) 00000100:00100000:1.0:1776178146.109551:0:8250:0:(service.c:2152:ptlrpc_server_handle_req_in()) got req x1862456979087872 00000100:00100000:0.0:1776178146.112782:0:8250:0:(nrs_fifo.c:179:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.204.3@tcp, seq: 6519 00000100:00100000:0.0:1776178146.112795:0:8250:0:(service.c:2324:ptlrpc_server_handle_request()) Handling RPC req@00000000d5cc7d3c pname:cluuid+ref:pid:xid:nid:opc:job mdt00_003:b50963bd-8eba-4432-9a54-8888d00e3fa1+39:17037:x1862456979087872:12345-192.168.204.3@tcp:36:rm.0 00010000:00010000:0.0:1776178146.112870:0:8250:0:(ldlm_lockd.c:1720:ldlm_request_cancel()) ### server-side cancel handler START: 2 locks, starting at 0 00010000:00010000:0.0:1776178146.112904:0:8250:0:(ldlm_lock.c:216:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: mdt-lustre-MDT0000_UUID lock: 0000000030bf6b1e/0xa40083ab9bf5313c lrc: 0/0,0 mode: --/PR res: [0x2000013a1:0x3:0x0].0x0 bits 0x1b/0x0 rrc: 5 type: IBT gid 0 flags: 0x44a01000000000 nid: 192.168.204.3@tcp remote: 0x771f68aa1eb27a3f expref: 39 pid: 8250 timeout: 0 lvb_type: 0 00010000:00010000:0.0:1776178146.112926:0:8250:0:(ldlm_lock.c:216:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: mdt-lustre-MDT0000_UUID lock: 00000000f51fd0d7/0xa40083ab9bf530fd lrc: 0/0,0 mode: --/CR res: [0x2000013a1:0x3:0x0].0x0 bits 0x8/0x0 rrc: 4 type: IBT gid 0 flags: 0x44801000000000 nid: 192.168.204.3@tcp remote: 0x771f68aa1eb27a31 expref: 38 pid: 8250 timeout: 0 lvb_type: 0 00010000:00010000:0.0:1776178146.112936:0:8250:0:(ldlm_inodebits.c:94:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x2000013a1:0x3:0x0].0x0 (0000000090c54b62) 00010000:00010000:0.0:1776178146.112941:0:8250:0:(ldlm_lockd.c:1782:ldlm_request_cancel()) ### server-side cancel handler END 00010000:00010000:0.0:1776178146.112967:0:8250:0:(ldlm_lock.c:766:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(CW) ns: mdt-lustre-MDT0000_UUID lock: 0000000068850c85/0xa40083ab9bf53254 lrc: 3/0,1 mode: --/CW res: [0x2000013a1:0x1:0x0].0x0 bits 0x0/0x0 rrc: 4 type: IBT gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 8250 timeout: 0 lvb_type: 0 00010000:00010000:0.0:1776178146.112980:0:8250:0:(ldlm_lock.c:1064:ldlm_granted_list_add_lock()) ### About to add lock: ns: mdt-lustre-MDT0000_UUID lock: 0000000068850c85/0xa40083ab9bf53254 lrc: 3/0,1 mode: CW/CW res: [0x2000013a1:0x1:0x0].0x0 bits 0x2/0x0 rrc: 4 type: IBT gid 0 flags: 0x50210000000000 nid: local remote: 0x0 expref: -99 pid: 8250 timeout: 0 lvb_type: 0 00010000:00010000:0.0:1776178146.112990:0:8250:0:(ldlm_request.c:528:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: mdt-lustre-MDT0000_UUID lock: 0000000068850c85/0xa40083ab9bf53254 lrc: 3/0,1 mode: CW/CW res: [0x2000013a1:0x1:0x0].0x0 bits 0x2/0x0 rrc: 4 type: IBT gid 0 flags: 0x40210000000000 nid: local remote: 0x0 expref: -99 pid: 8250 timeout: 0 lvb_type: 0 00010000:00010000:0.0:1776178146.113017:0:8250:0:(ldlm_lock.c:766:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: mdt-lustre-MDT0000_UUID lock: 000000000e7a37e0/0xa40083ab9bf5325b lrc: 3/0,1 mode: --/PW res: [0x2000013a1:0x1:0x0].0x15fcbbea bits 0x0/0x0 rrc: 2 type: IBT gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 8250 timeout: 0 lvb_type: 0 00010000:00010000:0.0:1776178146.113024:0:8250:0:(ldlm_lock.c:1064:ldlm_granted_list_add_lock()) ### About to add lock: ns: mdt-lustre-MDT0000_UUID lock: 000000000e7a37e0/0xa40083ab9bf5325b lrc: 3/0,1 mode: PW/PW res: [0x2000013a1:0x1:0x0].0x15fcbbea bits 0x2/0x0 rrc: 2 type: IBT gid 0 flags: 0x50210000000000 nid: local remote: 0x0 expref: -99 pid: 8250 timeout: 0 lvb_type: 0 00010000:00010000:0.0:1776178146.113030:0:8250:0:(ldlm_request.c:528:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: mdt-lustre-MDT0000_UUID lock: 000000000e7a37e0/0xa40083ab9bf5325b lrc: 3/0,1 mode: PW/PW res: [0x2000013a1:0x1:0x0].0x15fcbbea bits 0x2/0x0 rrc: 2 type: IBT gid 0 flags: 0x40210000000000 nid: local remote: 0x0 expref: -99 pid: 8250 timeout: 0 lvb_type: 0 00010000:00010000:0.0:1776178146.113446:0:8250:0:(ldlm_lock.c:766:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(EX) ns: mdt-lustre-MDT0000_UUID lock: 00000000782e253e/0xa40083ab9bf53262 lrc: 3/0,1 mode: --/EX res: [0x2000013a1:0x3:0x0].0x0 bits 0x0/0x0 rrc: 3 type: IBT gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 8250 timeout: 0 lvb_type: 0 00010000:00010000:0.0:1776178146.113461:0:8250:0:(ldlm_lock.c:1064:ldlm_granted_list_add_lock()) ### About to add lock: ns: mdt-lustre-MDT0000_UUID lock: 00000000782e253e/0xa40083ab9bf53262 lrc: 3/0,1 mode: EX/EX res: [0x2000013a1:0x3:0x0].0x0 bits 0x3/0x0 rrc: 3 type: IBT gid 0 flags: 0x50210000000000 nid: local remote: 0x0 expref: -99 pid: 8250 timeout: 0 lvb_type: 0 00010000:00010000:0.0:1776178146.113469:0:8250:0:(ldlm_request.c:528:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: mdt-lustre-MDT0000_UUID lock: 00000000782e253e/0xa40083ab9bf53262 lrc: 3/0,1 mode: EX/EX res: [0x2000013a1:0x3:0x0].0x0 bits 0x3/0x0 rrc: 3 type: IBT gid 0 flags: 0x40210000000000 nid: local remote: 0x0 expref: -99 pid: 8250 timeout: 0 lvb_type: 0 00000040:00080000:0.0:1776178146.113685:0:8250:0:(llog_osd.c:738:llog_osd_write_rec()) added record [0x1:0xe:0x0].2, 64 off8320 00000004:00080000:0.0:1776178146.113692:0:8250:0:(osp_sync.c:1658:osp_sync_add_commit_cb()) lustre-OST0001-osc-MDT0000: add commit cb at 853440631363ns, next at 854440629236ns, rc = 0 00010000:00010000:0.0:1776178146.113738:0:8250:0:(ldlm_lock.c:766:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: mdt-lustre-MDT0000_UUID lock: 00000000f90c1e39/0xa40083ab9bf53269 lrc: 3/0,1 mode: --/PW res: [0x2000013a1:0x3:0x0].0x0 bits 0x0/0x0 rrc: 4 type: IBT gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 8250 timeout: 0 lvb_type: 0 00010000:00010000:0.0:1776178146.113746:0:8250:0:(ldlm_lock.c:1064:ldlm_granted_list_add_lock()) ### About to add lock: ns: mdt-lustre-MDT0000_UUID lock: 00000000f90c1e39/0xa40083ab9bf53269 lrc: 3/0,1 mode: PW/PW res: [0x2000013a1:0x3:0x0].0x0 bits 0x40/0x0 rrc: 4 type: IBT gid 18446635898608079296 flags: 0x50010080000000 nid: local remote: 0x0 expref: -99 pid: 8250 timeout: 0 lvb_type: 0 00010000:00010000:0.0:1776178146.113754:0:8250:0:(ldlm_request.c:528:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: mdt-lustre-MDT0000_UUID lock: 00000000f90c1e39/0xa40083ab9bf53269 lrc: 3/0,1 mode: PW/PW res: [0x2000013a1:0x3:0x0].0x0 bits 0x40/0x0 rrc: 4 type: IBT gid 18446635898608079296 flags: 0x40010080000000 nid: local remote: 0x0 expref: -99 pid: 8250 timeout: 0 lvb_type: 0 00010000:00010000:0.0:1776178146.113761:0:8250:0:(ldlm_lock.c:942:ldlm_lock_decref_and_cancel()) ### ldlm_lock_decref(PW) ns: mdt-lustre-MDT0000_UUID lock: 00000000f90c1e39/0xa40083ab9bf53269 lrc: 4/0,1 mode: PW/PW res: [0x2000013a1:0x3:0x0].0x0 bits 0x40/0x0 rrc: 4 type: IBT gid 18446635898608079296 flags: 0x40010080000000 nid: local remote: 0x0 expref: -99 pid: 8250 timeout: 0 lvb_type: 0 00010000:00010000:0.0:1776178146.113767:0:8250:0:(ldlm_lock.c:820:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: mdt-lustre-MDT0000_UUID lock: 00000000f90c1e39/0xa40083ab9bf53269 lrc: 4/0,1 mode: PW/PW res: [0x2000013a1:0x3:0x0].0x0 bits 0x40/0x0 rrc: 4 type: IBT gid 18446635898608079296 flags: 0x50010480000000 nid: local remote: 0x0 expref: -99 pid: 8250 timeout: 0 lvb_type: 0 00010000:00010000:0.0:1776178146.113772:0:8250:0:(ldlm_lock.c:879:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: mdt-lustre-MDT0000_UUID lock: 00000000f90c1e39/0xa40083ab9bf53269 lrc: 3/0,0 mode: PW/PW res: [0x2000013a1:0x3:0x0].0x0 bits 0x40/0x0 rrc: 4 type: IBT gid 18446635898608079296 flags: 0x50010480000000 nid: local remote: 0x0 expref: -99 pid: 8250 timeout: 0 lvb_type: 0 00010000:00010000:0.0:1776178146.113869:0:8250:0:(ldlm_lockd.c:2159:__ldlm_bl_to_thread()) added 1/1 locks to regular blp list, 1 blwis in pool 00000004:00010000:0.0:1776178146.113889:0:8250:0:(mdt_handler.c:3992:mdt_save_lock()) ### save lock request 00000000d5cc7d3c reply state 00000000639f8179 transno 12884901938 ns: mdt-lustre-MDT0000_UUID lock: 00000000782e253e/0xa40083ab9bf53262 lrc: 3/0,1 mode: EX/EX res: [0x2000013a1:0x3:0x0].0x0 bits 0x3/0x0 rrc: 4 type: IBT gid 0 flags: 0x40210000000000 nid: local remote: 0x0 expref: -99 pid: 8250 timeout: 0 lvb_type: 0 00010000:00010000:0.0:1776178146.114012:0:8250:0:(ldlm_lock.c:820:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(CW) ns: mdt-lustre-MDT0000_UUID lock: 0000000068850c85/0xa40083ab9bf53254 lrc: 3/0,1 mode: CW/CW res: [0x2000013a1:0x1:0x0].0x0 bits 0x2/0x0 rrc: 4 type: IBT gid 0 flags: 0x50210000000000 nid: local remote: 0x0 expref: -99 pid: 8250 timeout: 0 lvb_type: 0 00010000:00010000:0.0:1776178146.114019:0:8250:0:(ldlm_lock.c:879:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: mdt-lustre-MDT0000_UUID lock: 0000000068850c85/0xa40083ab9bf53254 lrc: 2/0,0 mode: CW/CW res: [0x2000013a1:0x1:0x0].0x0 bits 0x2/0x0 rrc: 4 type: IBT gid 0 flags: 0x50210400000000 nid: local remote: 0x0 expref: -99 pid: 8250 timeout: 0 lvb_type: 0 00010000:00010000:0.0:1776178146.114025:0:8250:0:(ldlm_lockd.c:1879:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: mdt-lustre-MDT0000_UUID lock: 0000000068850c85/0xa40083ab9bf53254 lrc: 3/0,0 mode: CW/CW res: [0x2000013a1:0x1:0x0].0x0 bits 0x2/0x0 rrc: 4 type: IBT gid 0 flags: 0x40210400000000 nid: local remote: 0x0 expref: -99 pid: 8250 timeout: 0 lvb_type: 0 00010000:00010000:0.0:1776178146.114031:0:8250:0:(ldlm_lockd.c:1893:ldlm_handle_bl_callback()) Lock 0000000068850c85 already unused, calling callback (0000000079e78348) 00010000:00010000:0.0:1776178146.114034:0:8250:0:(ldlm_request.c:364:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: mdt-lustre-MDT0000_UUID lock: 0000000068850c85/0xa40083ab9bf53254 lrc: 3/0,0 mode: CW/CW res: [0x2000013a1:0x1:0x0].0x0 bits 0x2/0x0 rrc: 4 type: IBT gid 0 flags: 0x40210400000000 nid: local remote: 0x0 expref: -99 pid: 8250 timeout: 0 lvb_type: 0 00010000:00010000:0.0:1776178146.114041:0:8250:0:(ldlm_request.c:1257:ldlm_cli_cancel_local()) ### server-side local cancel ns: mdt-lustre-MDT0000_UUID lock: 0000000068850c85/0xa40083ab9bf53254 lrc: 4/0,0 mode: CW/CW res: [0x2000013a1:0x1:0x0].0x0 bits 0x2/0x0 rrc: 4 type: IBT gid 0 flags: 0x40218400000000 nid: local remote: 0x0 expref: -99 pid: 8250 timeout: 0 lvb_type: 0 00010000:00010000:0.0:1776178146.114052:0:8250:0:(ldlm_inodebits.c:94:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x2000013a1:0x1:0x0].0x0 (00000000315facbe) 00010000:00010000:0.0:1776178146.114054:0:8250:0:(ldlm_inodebits.c:96:ldlm_reprocess_inodebits_queue()) Hint 2 00010000:00010000:0.0:1776178146.114096:0:8250:0:(ldlm_lockd.c:1903:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: mdt-lustre-MDT0000_UUID lock: 0000000068850c85/0xa40083ab9bf53254 lrc: 2/0,0 mode: --/CW res: [0x2000013a1:0x1:0x0].0x0 bits 0x2/0x0 rrc: 4 type: IBT gid 0 flags: 0x44a19400000000 nid: local remote: 0x0 expref: -99 pid: 8250 timeout: 0 lvb_type: 0 00010000:00010000:0.0:1776178146.114105:0:8250:0:(ldlm_lock.c:216:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: mdt-lustre-MDT0000_UUID lock: 0000000068850c85/0xa40083ab9bf53254 lrc: 0/0,0 mode: --/CW res: [0x2000013a1:0x1:0x0].0x0 bits 0x2/0x0 rrc: 4 type: IBT gid 0 flags: 0x44a19400000000 nid: local remote: 0x0 expref: -99 pid: 8250 timeout: 0 lvb_type: 0 00000004:00010000:0.0:1776178146.114120:0:8250:0:(mdt_handler.c:3992:mdt_save_lock()) ### save lock request 00000000d5cc7d3c reply state 00000000639f8179 transno 12884901938 ns: mdt-lustre-MDT0000_UUID lock: 000000000e7a37e0/0xa40083ab9bf5325b lrc: 3/0,1 mode: PW/PW res: [0x2000013a1:0x1:0x0].0x15fcbbea bits 0x2/0x0 rrc: 2 type: IBT gid 0 flags: 0x40210000000000 nid: local remote: 0x0 expref: -99 pid: 8250 timeout: 0 lvb_type: 0 00000100:00100000:0.0:1776178146.114205:0:8250:0:(service.c:2372:ptlrpc_server_handle_request()) Handled RPC req@00000000d5cc7d3c pname:cluuid+ref:pid:xid:nid:opc:job mdt00_003:b50963bd-8eba-4432-9a54-8888d00e3fa1+39:17037:x1862456979087872:12345-192.168.204.3@tcp:36:rm.0 Request processed in 1410us (7349us total) trans 12884901938 rc 0/0 00000100:00100000:0.0:1776178146.114217:0:8250:0:(nrs_fifo.c:241:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.204.3@tcp, seq: 6519 00010000:00010000:2.0:1776178146.115903:0:9224:0:(ldlm_lockd.c:2789:ldlm_bl_get_work()) Got 1 locks of 0 total in blp. (0 blwis in pool) 00010000:00010000:2.0:1776178146.115916:0:9224:0:(ldlm_lockd.c:1879:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: mdt-lustre-MDT0000_UUID lock: 00000000f90c1e39/0xa40083ab9bf53269 lrc: 2/0,0 mode: PW/PW res: [0x2000013a1:0x3:0x0].0x0 bits 0x40/0x0 rrc: 4 type: IBT gid 18446635898608079296 flags: 0x40010480000000 nid: local remote: 0x0 expref: -99 pid: 8250 timeout: 0 lvb_type: 0 00010000:00010000:2.0:1776178146.115930:0:9224:0:(ldlm_lockd.c:1893:ldlm_handle_bl_callback()) Lock 00000000f90c1e39 already unused, calling callback (0000000063c97258) 00010000:00010000:2.0:1776178146.115933:0:9224:0:(ldlm_request.c:364:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: mdt-lustre-MDT0000_UUID lock: 00000000f90c1e39/0xa40083ab9bf53269 lrc: 2/0,0 mode: PW/PW res: [0x2000013a1:0x3:0x0].0x0 bits 0x40/0x0 rrc: 4 type: IBT gid 18446635898608079296 flags: 0x40010480000000 nid: local remote: 0x0 expref: -99 pid: 8250 timeout: 0 lvb_type: 0 00010000:00010000:2.0:1776178146.115941:0:9224:0:(ldlm_request.c:1257:ldlm_cli_cancel_local()) ### server-side local cancel ns: mdt-lustre-MDT0000_UUID lock: 00000000f90c1e39/0xa40083ab9bf53269 lrc: 3/0,0 mode: PW/PW res: [0x2000013a1:0x3:0x0].0x0 bits 0x40/0x0 rrc: 4 type: IBT gid 18446635898608079296 flags: 0x40018480000000 nid: local remote: 0x0 expref: -99 pid: 8250 timeout: 0 lvb_type: 0 00010000:00010000:2.0:1776178146.115957:0:9224:0:(ldlm_inodebits.c:94:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x2000013a1:0x3:0x0].0x0 (0000000090c54b62) 00010000:00010000:2.0:1776178146.115960:0:9224:0:(ldlm_inodebits.c:96:ldlm_reprocess_inodebits_queue()) Hint 40 00010000:00010000:2.0:1776178146.115963:0:9224:0:(ldlm_lockd.c:1903:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: mdt-lustre-MDT0000_UUID lock: 00000000f90c1e39/0xa40083ab9bf53269 lrc: 1/0,0 mode: --/PW res: [0x2000013a1:0x3:0x0].0x0 bits 0x40/0x0 rrc: 4 type: IBT gid 18446635898608079296 flags: 0x44819480000000 nid: local remote: 0x0 expref: -99 pid: 8250 timeout: 0 lvb_type: 0 00010000:00010000:2.0:1776178146.115969:0:9224:0:(ldlm_lock.c:216:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: mdt-lustre-MDT0000_UUID lock: 00000000f90c1e39/0xa40083ab9bf53269 lrc: 0/0,0 mode: --/PW res: [0x2000013a1:0x3:0x0].0x0 bits 0x40/0x0 rrc: 4 type: IBT gid 18446635898608079296 flags: 0x44819480000000 nid: local remote: 0x0 expref: -99 pid: 8250 timeout: 0 lvb_type: 0 00010000:00010000:2.0:1776178146.115986:0:9224:0:(ldlm_lockd.c:2792:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00010000:00010000:2.0:1776178146.115988:0:9224:0:(ldlm_lockd.c:2792:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00010000:00010000:2.0:1776178146.115989:0:9224:0:(ldlm_lockd.c:2792:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00010000:00010000:2.0:1776178146.116914:0:3340:0:(ldlm_lock.c:1064:ldlm_granted_list_add_lock()) ### About to add lock: ns: mdt-lustre-MDT0000_UUID lock: 000000000e7a37e0/0xa40083ab9bf5325b lrc: 3/0,1 mode: COS/COS res: [0x2000013a1:0x1:0x0].0x15fcbbea bits 0x2/0x0 rrc: 2 type: IBT gid 0 flags: 0x50210000000000 nid: local remote: 0x0 expref: -99 pid: 8250 timeout: 0 lvb_type: 0 00010000:00010000:2.0:1776178146.116929:0:3340:0:(ldlm_inodebits.c:94:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x2000013a1:0x1:0x0].0x15fcbbea (00000000eb684e0b) 00010000:00010000:2.0:1776178146.116932:0:3340:0:(ldlm_inodebits.c:96:ldlm_reprocess_inodebits_queue()) Hint 2 00010000:00010000:2.0:1776178146.116936:0:3340:0:(ldlm_lock.c:1064:ldlm_granted_list_add_lock()) ### About to add lock: ns: mdt-lustre-MDT0000_UUID lock: 00000000782e253e/0xa40083ab9bf53262 lrc: 3/0,1 mode: COS/COS res: [0x2000013a1:0x3:0x0].0x0 bits 0x3/0x0 rrc: 3 type: IBT gid 0 flags: 0x50210000000000 nid: local remote: 0x0 expref: -99 pid: 8250 timeout: 0 lvb_type: 0 00010000:00010000:2.0:1776178146.116942:0:3340:0:(ldlm_inodebits.c:94:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x2000013a1:0x3:0x0].0x0 (0000000090c54b62) 00010000:00010000:2.0:1776178146.116943:0:3340:0:(ldlm_inodebits.c:96:ldlm_reprocess_inodebits_queue()) Hint 3 00000100:00100000:0.0:1776178146.118494:0:3347:0:(events.c:361:request_in_callback()) peer: 12345-192.168.204.3@tcp (source: 12345-192.168.204.3@tcp) 00000100:00100000:2.0:1776178146.118643:0:8250:0:(service.c:2152:ptlrpc_server_handle_req_in()) got req x1862456979088000 00000100:00100000:3.0:1776178146.118766:0:6257:0:(nrs_fifo.c:179:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.204.3@tcp, seq: 6520 00000100:00100000:3.0:1776178146.118776:0:6257:0:(service.c:2324:ptlrpc_server_handle_request()) Handling RPC req@00000000c5909a76 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_002:b50963bd-8eba-4432-9a54-8888d00e3fa1+39:17037:x1862456979088000:12345-192.168.204.3@tcp:36:rm.0 00010000:00010000:3.0:1776178146.118857:0:6257:0:(ldlm_lockd.c:1720:ldlm_request_cancel()) ### server-side cancel handler START: 2 locks, starting at 0 00010000:00010000:3.0:1776178146.118891:0:6257:0:(ldlm_lock.c:216:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: mdt-lustre-MDT0000_UUID lock: 000000000b903578/0xa40083ab9bf531ac lrc: 0/0,0 mode: --/PR res: [0x2000013a1:0x7:0x0].0x0 bits 0x1b/0x0 rrc: 5 type: IBT gid 0 flags: 0x44a01000000000 nid: 192.168.204.3@tcp remote: 0x771f68aa1eb27a5b expref: 39 pid: 8250 timeout: 0 lvb_type: 0 00010000:00010000:3.0:1776178146.118917:0:6257:0:(ldlm_lock.c:216:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: mdt-lustre-MDT0000_UUID lock: 0000000055d6a00d/0xa40083ab9bf530e1 lrc: 0/0,0 mode: --/CR res: [0x2000013a1:0x7:0x0].0x0 bits 0x8/0x0 rrc: 4 type: IBT gid 0 flags: 0x44801000000000 nid: 192.168.204.3@tcp remote: 0x771f68aa1eb27a07 expref: 38 pid: 6256 timeout: 0 lvb_type: 0 00010000:00010000:3.0:1776178146.118926:0:6257:0:(ldlm_inodebits.c:94:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x2000013a1:0x7:0x0].0x0 (00000000aeb25ebf) 00010000:00010000:3.0:1776178146.118931:0:6257:0:(ldlm_lockd.c:1782:ldlm_request_cancel()) ### server-side cancel handler END 00010000:00010000:3.0:1776178146.118962:0:6257:0:(ldlm_lock.c:766:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(CW) ns: mdt-lustre-MDT0000_UUID lock: 000000009f2de1df/0xa40083ab9bf53270 lrc: 3/0,1 mode: --/CW res: [0x2000013a1:0x1:0x0].0x0 bits 0x0/0x0 rrc: 4 type: IBT gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 6257 timeout: 0 lvb_type: 0 00010000:00010000:3.0:1776178146.118977:0:6257:0:(ldlm_lock.c:1064:ldlm_granted_list_add_lock()) ### About to add lock: ns: mdt-lustre-MDT0000_UUID lock: 000000009f2de1df/0xa40083ab9bf53270 lrc: 3/0,1 mode: CW/CW res: [0x2000013a1:0x1:0x0].0x0 bits 0x2/0x0 rrc: 4 type: IBT gid 0 flags: 0x50210000000000 nid: local remote: 0x0 expref: -99 pid: 6257 timeout: 0 lvb_type: 0 00010000:00010000:3.0:1776178146.118992:0:6257:0:(ldlm_request.c:528:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: mdt-lustre-MDT0000_UUID lock: 000000009f2de1df/0xa40083ab9bf53270 lrc: 3/0,1 mode: CW/CW res: [0x2000013a1:0x1:0x0].0x0 bits 0x2/0x0 rrc: 4 type: IBT gid 0 flags: 0x40210000000000 nid: local remote: 0x0 expref: -99 pid: 6257 timeout: 0 lvb_type: 0 00010000:00010000:3.0:1776178146.119359:0:6257:0:(ldlm_lock.c:766:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: mdt-lustre-MDT0000_UUID lock: 00000000712476de/0xa40083ab9bf53277 lrc: 3/0,1 mode: --/PW res: [0x2000013a1:0x1:0x0].0x37639fea bits 0x0/0x0 rrc: 2 type: IBT gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 6257 timeout: 0 lvb_type: 0 00010000:00010000:3.0:1776178146.119432:0:6257:0:(ldlm_lock.c:1064:ldlm_granted_list_add_lock()) ### About to add lock: ns: mdt-lustre-MDT0000_UUID lock: 00000000712476de/0xa40083ab9bf53277 lrc: 3/0,1 mode: PW/PW res: [0x2000013a1:0x1:0x0].0x37639fea bits 0x2/0x0 rrc: 2 type: IBT gid 0 flags: 0x50210000000000 nid: local remote: 0x0 expref: -99 pid: 6257 timeout: 0 lvb_type: 0 00010000:00010000:3.0:1776178146.119441:0:6257:0:(ldlm_request.c:528:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: mdt-lustre-MDT0000_UUID lock: 00000000712476de/0xa40083ab9bf53277 lrc: 3/0,1 mode: PW/PW res: [0x2000013a1:0x1:0x0].0x37639fea bits 0x2/0x0 rrc: 2 type: IBT gid 0 flags: 0x40210000000000 nid: local remote: 0x0 expref: -99 pid: 6257 timeout: 0 lvb_type: 0 00010000:00010000:3.0:1776178146.119533:0:6257:0:(ldlm_lock.c:766:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(EX) ns: mdt-lustre-MDT0000_UUID lock: 000000008cbcbaf5/0xa40083ab9bf5327e lrc: 3/0,1 mode: --/EX res: [0x2000013a1:0x7:0x0].0x0 bits 0x0/0x0 rrc: 3 type: IBT gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 6257 timeout: 0 lvb_type: 0 00010000:00010000:3.0:1776178146.119541:0:6257:0:(ldlm_lock.c:1064:ldlm_granted_list_add_lock()) ### About to add lock: ns: mdt-lustre-MDT0000_UUID lock: 000000008cbcbaf5/0xa40083ab9bf5327e lrc: 3/0,1 mode: EX/EX res: [0x2000013a1:0x7:0x0].0x0 bits 0x3/0x0 rrc: 3 type: IBT gid 0 flags: 0x50210000000000 nid: local remote: 0x0 expref: -99 pid: 6257 timeout: 0 lvb_type: 0 00010000:00010000:3.0:1776178146.119549:0:6257:0:(ldlm_request.c:528:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: mdt-lustre-MDT0000_UUID lock: 000000008cbcbaf5/0xa40083ab9bf5327e lrc: 3/0,1 mode: EX/EX res: [0x2000013a1:0x7:0x0].0x0 bits 0x3/0x0 rrc: 3 type: IBT gid 0 flags: 0x40210000000000 nid: local remote: 0x0 expref: -99 pid: 6257 timeout: 0 lvb_type: 0 00000040:00080000:3.0:1776178146.119771:0:6257:0:(llog_osd.c:738:llog_osd_write_rec()) added record [0x1:0xe:0x0].3, 64 off8384 00010000:00010000:3.0:1776178146.119832:0:6257:0:(ldlm_lock.c:766:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: mdt-lustre-MDT0000_UUID lock: 000000007353d00c/0xa40083ab9bf53285 lrc: 3/0,1 mode: --/PW res: [0x2000013a1:0x7:0x0].0x0 bits 0x0/0x0 rrc: 4 type: IBT gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 6257 timeout: 0 lvb_type: 0 00010000:00010000:3.0:1776178146.119841:0:6257:0:(ldlm_lock.c:1064:ldlm_granted_list_add_lock()) ### About to add lock: ns: mdt-lustre-MDT0000_UUID lock: 000000007353d00c/0xa40083ab9bf53285 lrc: 3/0,1 mode: PW/PW res: [0x2000013a1:0x7:0x0].0x0 bits 0x40/0x0 rrc: 4 type: IBT gid 18446635902900449728 flags: 0x50010080000000 nid: local remote: 0x0 expref: -99 pid: 6257 timeout: 0 lvb_type: 0 00010000:00010000:3.0:1776178146.119849:0:6257:0:(ldlm_request.c:528:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: mdt-lustre-MDT0000_UUID lock: 000000007353d00c/0xa40083ab9bf53285 lrc: 3/0,1 mode: PW/PW res: [0x2000013a1:0x7:0x0].0x0 bits 0x40/0x0 rrc: 4 type: IBT gid 18446635902900449728 flags: 0x40010080000000 nid: local remote: 0x0 expref: -99 pid: 6257 timeout: 0 lvb_type: 0 00010000:00010000:3.0:1776178146.119861:0:6257:0:(ldlm_lock.c:942:ldlm_lock_decref_and_cancel()) ### ldlm_lock_decref(PW) ns: mdt-lustre-MDT0000_UUID lock: 000000007353d00c/0xa40083ab9bf53285 lrc: 4/0,1 mode: PW/PW res: [0x2000013a1:0x7:0x0].0x0 bits 0x40/0x0 rrc: 4 type: IBT gid 18446635902900449728 flags: 0x40010080000000 nid: local remote: 0x0 expref: -99 pid: 6257 timeout: 0 lvb_type: 0 00010000:00010000:3.0:1776178146.119866:0:6257:0:(ldlm_lock.c:820:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: mdt-lustre-MDT0000_UUID lock: 000000007353d00c/0xa40083ab9bf53285 lrc: 4/0,1 mode: PW/PW res: [0x2000013a1:0x7:0x0].0x0 bits 0x40/0x0 rrc: 4 type: IBT gid 18446635902900449728 flags: 0x50010480000000 nid: local remote: 0x0 expref: -99 pid: 6257 timeout: 0 lvb_type: 0 00010000:00010000:3.0:1776178146.119872:0:6257:0:(ldlm_lock.c:879:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: mdt-lustre-MDT0000_UUID lock: 000000007353d00c/0xa40083ab9bf53285 lrc: 3/0,0 mode: PW/PW res: [0x2000013a1:0x7:0x0].0x0 bits 0x40/0x0 rrc: 4 type: IBT gid 18446635902900449728 flags: 0x50010480000000 nid: local remote: 0x0 expref: -99 pid: 6257 timeout: 0 lvb_type: 0 00010000:00010000:3.0:1776178146.119899:0:6257:0:(ldlm_lockd.c:2159:__ldlm_bl_to_thread()) added 1/1 locks to regular blp list, 1 blwis in pool 00000004:00010000:3.0:1776178146.119917:0:6257:0:(mdt_handler.c:3992:mdt_save_lock()) ### save lock request 00000000c5909a76 reply state 00000000b5e21ec2 transno 12884901939 ns: mdt-lustre-MDT0000_UUID lock: 000000008cbcbaf5/0xa40083ab9bf5327e lrc: 3/0,1 mode: EX/EX res: [0x2000013a1:0x7:0x0].0x0 bits 0x3/0x0 rrc: 4 type: IBT gid 0 flags: 0x40210000000000 nid: local remote: 0x0 expref: -99 pid: 6257 timeout: 0 lvb_type: 0 00010000:00010000:1.0:1776178146.120227:0:6244:0:(ldlm_lockd.c:2789:ldlm_bl_get_work()) Got 1 locks of 0 total in blp. (0 blwis in pool) 00010000:00010000:1.0:1776178146.120233:0:6244:0:(ldlm_lockd.c:1879:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: mdt-lustre-MDT0000_UUID lock: 000000007353d00c/0xa40083ab9bf53285 lrc: 2/0,0 mode: PW/PW res: [0x2000013a1:0x7:0x0].0x0 bits 0x40/0x0 rrc: 4 type: IBT gid 18446635902900449728 flags: 0x40010480000000 nid: local remote: 0x0 expref: -99 pid: 6257 timeout: 0 lvb_type: 0 00010000:00010000:1.0:1776178146.120251:0:6244:0:(ldlm_lockd.c:1893:ldlm_handle_bl_callback()) Lock 000000007353d00c already unused, calling callback (0000000063c97258) 00010000:00010000:1.0:1776178146.120255:0:6244:0:(ldlm_request.c:364:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: mdt-lustre-MDT0000_UUID lock: 000000007353d00c/0xa40083ab9bf53285 lrc: 2/0,0 mode: PW/PW res: [0x2000013a1:0x7:0x0].0x0 bits 0x40/0x0 rrc: 4 type: IBT gid 18446635902900449728 flags: 0x40010480000000 nid: local remote: 0x0 expref: -99 pid: 6257 timeout: 0 lvb_type: 0 00010000:00010000:1.0:1776178146.120262:0:6244:0:(ldlm_request.c:1257:ldlm_cli_cancel_local()) ### server-side local cancel ns: mdt-lustre-MDT0000_UUID lock: 000000007353d00c/0xa40083ab9bf53285 lrc: 3/0,0 mode: PW/PW res: [0x2000013a1:0x7:0x0].0x0 bits 0x40/0x0 rrc: 4 type: IBT gid 18446635902900449728 flags: 0x40018480000000 nid: local remote: 0x0 expref: -99 pid: 6257 timeout: 0 lvb_type: 0 00010000:00010000:1.0:1776178146.120274:0:6244:0:(ldlm_inodebits.c:94:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x2000013a1:0x7:0x0].0x0 (00000000aeb25ebf) 00010000:00010000:1.0:1776178146.120276:0:6244:0:(ldlm_inodebits.c:96:ldlm_reprocess_inodebits_queue()) Hint 40 00010000:00010000:1.0:1776178146.120279:0:6244:0:(ldlm_lockd.c:1903:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: mdt-lustre-MDT0000_UUID lock: 000000007353d00c/0xa40083ab9bf53285 lrc: 1/0,0 mode: --/PW res: [0x2000013a1:0x7:0x0].0x0 bits 0x40/0x0 rrc: 4 type: IBT gid 18446635902900449728 flags: 0x44819480000000 nid: local remote: 0x0 expref: -99 pid: 6257 timeout: 0 lvb_type: 0 00010000:00010000:1.0:1776178146.120289:0:6244:0:(ldlm_lock.c:216:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: mdt-lustre-MDT0000_UUID lock: 000000007353d00c/0xa40083ab9bf53285 lrc: 0/0,0 mode: --/PW res: [0x2000013a1:0x7:0x0].0x0 bits 0x40/0x0 rrc: 4 type: IBT gid 18446635902900449728 flags: 0x44819480000000 nid: local remote: 0x0 expref: -99 pid: 6257 timeout: 0 lvb_type: 0 00010000:00010000:1.0:1776178146.120300:0:6244:0:(ldlm_lockd.c:2792:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00010000:00010000:1.0:1776178146.120301:0:6244:0:(ldlm_lockd.c:2792:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00010000:00010000:1.0:1776178146.120302:0:6244:0:(ldlm_lockd.c:2792:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00010000:00010000:3.0:1776178146.120372:0:6257:0:(ldlm_lock.c:820:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(CW) ns: mdt-lustre-MDT0000_UUID lock: 000000009f2de1df/0xa40083ab9bf53270 lrc: 3/0,1 mode: CW/CW res: [0x2000013a1:0x1:0x0].0x0 bits 0x2/0x0 rrc: 4 type: IBT gid 0 flags: 0x50210000000000 nid: local remote: 0x0 expref: -99 pid: 6257 timeout: 0 lvb_type: 0 00010000:00010000:3.0:1776178146.120380:0:6257:0:(ldlm_lock.c:879:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: mdt-lustre-MDT0000_UUID lock: 000000009f2de1df/0xa40083ab9bf53270 lrc: 2/0,0 mode: CW/CW res: [0x2000013a1:0x1:0x0].0x0 bits 0x2/0x0 rrc: 4 type: IBT gid 0 flags: 0x50210400000000 nid: local remote: 0x0 expref: -99 pid: 6257 timeout: 0 lvb_type: 0 00010000:00010000:3.0:1776178146.120387:0:6257:0:(ldlm_lockd.c:1879:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: mdt-lustre-MDT0000_UUID lock: 000000009f2de1df/0xa40083ab9bf53270 lrc: 3/0,0 mode: CW/CW res: [0x2000013a1:0x1:0x0].0x0 bits 0x2/0x0 rrc: 4 type: IBT gid 0 flags: 0x40210400000000 nid: local remote: 0x0 expref: -99 pid: 6257 timeout: 0 lvb_type: 0 00010000:00010000:3.0:1776178146.120392:0:6257:0:(ldlm_lockd.c:1893:ldlm_handle_bl_callback()) Lock 000000009f2de1df already unused, calling callback (0000000079e78348) 00010000:00010000:3.0:1776178146.120395:0:6257:0:(ldlm_request.c:364:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: mdt-lustre-MDT0000_UUID lock: 000000009f2de1df/0xa40083ab9bf53270 lrc: 3/0,0 mode: CW/CW res: [0x2000013a1:0x1:0x0].0x0 bits 0x2/0x0 rrc: 4 type: IBT gid 0 flags: 0x40210400000000 nid: local remote: 0x0 expref: -99 pid: 6257 timeout: 0 lvb_type: 0 00010000:00010000:3.0:1776178146.120401:0:6257:0:(ldlm_request.c:1257:ldlm_cli_cancel_local()) ### server-side local cancel ns: mdt-lustre-MDT0000_UUID lock: 000000009f2de1df/0xa40083ab9bf53270 lrc: 4/0,0 mode: CW/CW res: [0x2000013a1:0x1:0x0].0x0 bits 0x2/0x0 rrc: 4 type: IBT gid 0 flags: 0x40218400000000 nid: local remote: 0x0 expref: -99 pid: 6257 timeout: 0 lvb_type: 0 00010000:00010000:3.0:1776178146.120417:0:6257:0:(ldlm_inodebits.c:94:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x2000013a1:0x1:0x0].0x0 (00000000315facbe) 00010000:00010000:3.0:1776178146.120419:0:6257:0:(ldlm_inodebits.c:96:ldlm_reprocess_inodebits_queue()) Hint 2 00010000:00010000:3.0:1776178146.120421:0:6257:0:(ldlm_lockd.c:1903:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: mdt-lustre-MDT0000_UUID lock: 000000009f2de1df/0xa40083ab9bf53270 lrc: 2/0,0 mode: --/CW res: [0x2000013a1:0x1:0x0].0x0 bits 0x2/0x0 rrc: 4 type: IBT gid 0 flags: 0x44a19400000000 nid: local remote: 0x0 expref: -99 pid: 6257 timeout: 0 lvb_type: 0 00010000:00010000:3.0:1776178146.120427:0:6257:0:(ldlm_lock.c:216:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: mdt-lustre-MDT0000_UUID lock: 000000009f2de1df/0xa40083ab9bf53270 lrc: 0/0,0 mode: --/CW res: [0x2000013a1:0x1:0x0].0x0 bits 0x2/0x0 rrc: 4 type: IBT gid 0 flags: 0x44a19400000000 nid: local remote: 0x0 expref: -99 pid: 6257 timeout: 0 lvb_type: 0 00000004:00010000:3.0:1776178146.120435:0:6257:0:(mdt_handler.c:3992:mdt_save_lock()) ### save lock request 00000000c5909a76 reply state 00000000b5e21ec2 transno 12884901939 ns: mdt-lustre-MDT0000_UUID lock: 00000000712476de/0xa40083ab9bf53277 lrc: 3/0,1 mode: PW/PW res: [0x2000013a1:0x1:0x0].0x37639fea bits 0x2/0x0 rrc: 2 type: IBT gid 0 flags: 0x40210000000000 nid: local remote: 0x0 expref: -99 pid: 6257 timeout: 0 lvb_type: 0 00000100:00100000:3.0:1776178146.120576:0:6257:0:(service.c:2372:ptlrpc_server_handle_request()) Handled RPC req@00000000c5909a76 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_002:b50963bd-8eba-4432-9a54-8888d00e3fa1+39:17037:x1862456979088000:12345-192.168.204.3@tcp:36:rm.0 Request processed in 1797us (2088us total) trans 12884901939 rc 0/0 00000100:00100000:3.0:1776178146.120585:0:6257:0:(nrs_fifo.c:241:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.204.3@tcp, seq: 6520 00010000:00010000:3.0:1776178146.122883:0:3339:0:(ldlm_lock.c:1064:ldlm_granted_list_add_lock()) ### About to add lock: ns: mdt-lustre-MDT0000_UUID lock: 00000000712476de/0xa40083ab9bf53277 lrc: 3/0,1 mode: COS/COS res: [0x2000013a1:0x1:0x0].0x37639fea bits 0x2/0x0 rrc: 2 type: IBT gid 0 flags: 0x50210000000000 nid: local remote: 0x0 expref: -99 pid: 6257 timeout: 0 lvb_type: 0 00010000:00010000:3.0:1776178146.122904:0:3339:0:(ldlm_inodebits.c:94:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x2000013a1:0x1:0x0].0x37639fea (000000002853dc93) 00010000:00010000:3.0:1776178146.122906:0:3339:0:(ldlm_inodebits.c:96:ldlm_reprocess_inodebits_queue()) Hint 2 00010000:00010000:3.0:1776178146.122911:0:3339:0:(ldlm_lock.c:1064:ldlm_granted_list_add_lock()) ### About to add lock: ns: mdt-lustre-MDT0000_UUID lock: 000000008cbcbaf5/0xa40083ab9bf5327e lrc: 3/0,1 mode: COS/COS res: [0x2000013a1:0x7:0x0].0x0 bits 0x3/0x0 rrc: 3 type: IBT gid 0 flags: 0x50210000000000 nid: local remote: 0x0 expref: -99 pid: 6257 timeout: 0 lvb_type: 0 00010000:00010000:3.0:1776178146.122917:0:3339:0:(ldlm_inodebits.c:94:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x2000013a1:0x7:0x0].0x0 (00000000aeb25ebf) 00010000:00010000:3.0:1776178146.122918:0:3339:0:(ldlm_inodebits.c:96:ldlm_reprocess_inodebits_queue()) Hint 3 00000100:00100000:0.0:1776178146.124684:0:3347:0:(events.c:361:request_in_callback()) peer: 12345-192.168.204.3@tcp (source: 12345-192.168.204.3@tcp) 00000100:00100000:3.0:1776178146.125658:0:6257:0:(service.c:2152:ptlrpc_server_handle_req_in()) got req x1862456979088128 00000100:00100000:3.0:1776178146.125797:0:6257:0:(nrs_fifo.c:179:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.204.3@tcp, seq: 6521 00000100:00100000:3.0:1776178146.125815:0:6257:0:(service.c:2324:ptlrpc_server_handle_request()) Handling RPC req@0000000088356013 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_002:b50963bd-8eba-4432-9a54-8888d00e3fa1+39:17037:x1862456979088128:12345-192.168.204.3@tcp:36:rm.0 00010000:00010000:3.0:1776178146.125874:0:6257:0:(ldlm_lockd.c:1720:ldlm_request_cancel()) ### server-side cancel handler START: 2 locks, starting at 0 00010000:00010000:3.0:1776178146.125905:0:6257:0:(ldlm_lock.c:216:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: mdt-lustre-MDT0000_UUID lock: 000000005d93f094/0xa40083ab9bf531c8 lrc: 0/0,0 mode: --/PR res: [0x2000013a1:0x8:0x0].0x0 bits 0x1b/0x0 rrc: 5 type: IBT gid 0 flags: 0x44a01000000000 nid: 192.168.204.3@tcp remote: 0x771f68aa1eb27a62 expref: 39 pid: 6257 timeout: 0 lvb_type: 0 00010000:00010000:3.0:1776178146.125927:0:6257:0:(ldlm_lock.c:216:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: mdt-lustre-MDT0000_UUID lock: 00000000c3ab77cd/0xa40083ab9bf530da lrc: 0/0,0 mode: --/CR res: [0x2000013a1:0x8:0x0].0x0 bits 0x8/0x0 rrc: 4 type: IBT gid 0 flags: 0x44801000000000 nid: 192.168.204.3@tcp remote: 0x771f68aa1eb27a0e expref: 38 pid: 6256 timeout: 0 lvb_type: 0 00010000:00010000:3.0:1776178146.125937:0:6257:0:(ldlm_inodebits.c:94:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x2000013a1:0x8:0x0].0x0 (000000009a65918f) 00010000:00010000:3.0:1776178146.125942:0:6257:0:(ldlm_lockd.c:1782:ldlm_request_cancel()) ### server-side cancel handler END 00010000:00010000:3.0:1776178146.125969:0:6257:0:(ldlm_lock.c:766:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(CW) ns: mdt-lustre-MDT0000_UUID lock: 0000000026fd08a4/0xa40083ab9bf5328c lrc: 3/0,1 mode: --/CW res: [0x2000013a1:0x1:0x0].0x0 bits 0x0/0x0 rrc: 4 type: IBT gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 6257 timeout: 0 lvb_type: 0 00010000:00010000:3.0:1776178146.125983:0:6257:0:(ldlm_lock.c:1064:ldlm_granted_list_add_lock()) ### About to add lock: ns: mdt-lustre-MDT0000_UUID lock: 0000000026fd08a4/0xa40083ab9bf5328c lrc: 3/0,1 mode: CW/CW res: [0x2000013a1:0x1:0x0].0x0 bits 0x2/0x0 rrc: 4 type: IBT gid 0 flags: 0x50210000000000 nid: local remote: 0x0 expref: -99 pid: 6257 timeout: 0 lvb_type: 0 00010000:00010000:3.0:1776178146.125994:0:6257:0:(ldlm_request.c:528:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: mdt-lustre-MDT0000_UUID lock: 0000000026fd08a4/0xa40083ab9bf5328c lrc: 3/0,1 mode: CW/CW res: [0x2000013a1:0x1:0x0].0x0 bits 0x2/0x0 rrc: 4 type: IBT gid 0 flags: 0x40210000000000 nid: local remote: 0x0 expref: -99 pid: 6257 timeout: 0 lvb_type: 0 00010000:00010000:3.0:1776178146.126022:0:6257:0:(ldlm_lock.c:766:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: mdt-lustre-MDT0000_UUID lock: 000000006a188f9a/0xa40083ab9bf53293 lrc: 3/0,1 mode: --/PW res: [0x2000013a1:0x1:0x0].0x7fbd58ea bits 0x0/0x0 rrc: 2 type: IBT gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 6257 timeout: 0 lvb_type: 0 00010000:00010000:3.0:1776178146.126030:0:6257:0:(ldlm_lock.c:1064:ldlm_granted_list_add_lock()) ### About to add lock: ns: mdt-lustre-MDT0000_UUID lock: 000000006a188f9a/0xa40083ab9bf53293 lrc: 3/0,1 mode: PW/PW res: [0x2000013a1:0x1:0x0].0x7fbd58ea bits 0x2/0x0 rrc: 2 type: IBT gid 0 flags: 0x50210000000000 nid: local remote: 0x0 expref: -99 pid: 6257 timeout: 0 lvb_type: 0 00010000:00010000:3.0:1776178146.126037:0:6257:0:(ldlm_request.c:528:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: mdt-lustre-MDT0000_UUID lock: 000000006a188f9a/0xa40083ab9bf53293 lrc: 3/0,1 mode: PW/PW res: [0x2000013a1:0x1:0x0].0x7fbd58ea bits 0x2/0x0 rrc: 2 type: IBT gid 0 flags: 0x40210000000000 nid: local remote: 0x0 expref: -99 pid: 6257 timeout: 0 lvb_type: 0 00010000:00010000:3.0:1776178146.126160:0:6257:0:(ldlm_lock.c:766:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(EX) ns: mdt-lustre-MDT0000_UUID lock: 00000000dcea299c/0xa40083ab9bf5329a lrc: 3/0,1 mode: --/EX res: [0x2000013a1:0x8:0x0].0x0 bits 0x0/0x0 rrc: 3 type: IBT gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 6257 timeout: 0 lvb_type: 0 00010000:00010000:3.0:1776178146.126169:0:6257:0:(ldlm_lock.c:1064:ldlm_granted_list_add_lock()) ### About to add lock: ns: mdt-lustre-MDT0000_UUID lock: 00000000dcea299c/0xa40083ab9bf5329a lrc: 3/0,1 mode: EX/EX res: [0x2000013a1:0x8:0x0].0x0 bits 0x3/0x0 rrc: 3 type: IBT gid 0 flags: 0x50210000000000 nid: local remote: 0x0 expref: -99 pid: 6257 timeout: 0 lvb_type: 0 00010000:00010000:3.0:1776178146.126179:0:6257:0:(ldlm_request.c:528:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: mdt-lustre-MDT0000_UUID lock: 00000000dcea299c/0xa40083ab9bf5329a lrc: 3/0,1 mode: EX/EX res: [0x2000013a1:0x8:0x0].0x0 bits 0x3/0x0 rrc: 3 type: IBT gid 0 flags: 0x40210000000000 nid: local remote: 0x0 expref: -99 pid: 6257 timeout: 0 lvb_type: 0 00000040:00080000:3.0:1776178146.126394:0:6257:0:(llog_osd.c:738:llog_osd_write_rec()) added record [0x1:0xc:0x0].6, 64 off8576 00010000:00010000:3.0:1776178146.126441:0:6257:0:(ldlm_lock.c:766:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: mdt-lustre-MDT0000_UUID lock: 00000000591efbb9/0xa40083ab9bf532a1 lrc: 3/0,1 mode: --/PW res: [0x2000013a1:0x8:0x0].0x0 bits 0x0/0x0 rrc: 4 type: IBT gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 6257 timeout: 0 lvb_type: 0 00010000:00010000:3.0:1776178146.126450:0:6257:0:(ldlm_lock.c:1064:ldlm_granted_list_add_lock()) ### About to add lock: ns: mdt-lustre-MDT0000_UUID lock: 00000000591efbb9/0xa40083ab9bf532a1 lrc: 3/0,1 mode: PW/PW res: [0x2000013a1:0x8:0x0].0x0 bits 0x40/0x0 rrc: 4 type: IBT gid 18446635902900449728 flags: 0x50010080000000 nid: local remote: 0x0 expref: -99 pid: 6257 timeout: 0 lvb_type: 0 00010000:00010000:3.0:1776178146.126458:0:6257:0:(ldlm_request.c:528:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: mdt-lustre-MDT0000_UUID lock: 00000000591efbb9/0xa40083ab9bf532a1 lrc: 3/0,1 mode: PW/PW res: [0x2000013a1:0x8:0x0].0x0 bits 0x40/0x0 rrc: 4 type: IBT gid 18446635902900449728 flags: 0x40010080000000 nid: local remote: 0x0 expref: -99 pid: 6257 timeout: 0 lvb_type: 0 00010000:00010000:3.0:1776178146.126466:0:6257:0:(ldlm_lock.c:942:ldlm_lock_decref_and_cancel()) ### ldlm_lock_decref(PW) ns: mdt-lustre-MDT0000_UUID lock: 00000000591efbb9/0xa40083ab9bf532a1 lrc: 4/0,1 mode: PW/PW res: [0x2000013a1:0x8:0x0].0x0 bits 0x40/0x0 rrc: 4 type: IBT gid 18446635902900449728 flags: 0x40010080000000 nid: local remote: 0x0 expref: -99 pid: 6257 timeout: 0 lvb_type: 0 00010000:00010000:3.0:1776178146.126472:0:6257:0:(ldlm_lock.c:820:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: mdt-lustre-MDT0000_UUID lock: 00000000591efbb9/0xa40083ab9bf532a1 lrc: 4/0,1 mode: PW/PW res: [0x2000013a1:0x8:0x0].0x0 bits 0x40/0x0 rrc: 4 type: IBT gid 18446635902900449728 flags: 0x50010480000000 nid: local remote: 0x0 expref: -99 pid: 6257 timeout: 0 lvb_type: 0 00010000:00010000:3.0:1776178146.126478:0:6257:0:(ldlm_lock.c:879:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: mdt-lustre-MDT0000_UUID lock: 00000000591efbb9/0xa40083ab9bf532a1 lrc: 3/0,0 mode: PW/PW res: [0x2000013a1:0x8:0x0].0x0 bits 0x40/0x0 rrc: 4 type: IBT gid 18446635902900449728 flags: 0x50010480000000 nid: local remote: 0x0 expref: -99 pid: 6257 timeout: 0 lvb_type: 0 00010000:00010000:3.0:1776178146.126507:0:6257:0:(ldlm_lockd.c:2159:__ldlm_bl_to_thread()) added 1/1 locks to regular blp list, 1 blwis in pool 00000004:00010000:3.0:1776178146.126526:0:6257:0:(mdt_handler.c:3992:mdt_save_lock()) ### save lock request 0000000088356013 reply state 000000006f277865 transno 12884901940 ns: mdt-lustre-MDT0000_UUID lock: 00000000dcea299c/0xa40083ab9bf5329a lrc: 3/0,1 mode: EX/EX res: [0x2000013a1:0x8:0x0].0x0 bits 0x3/0x0 rrc: 4 type: IBT gid 0 flags: 0x40210000000000 nid: local remote: 0x0 expref: -99 pid: 6257 timeout: 0 lvb_type: 0 00010000:00010000:3.0:1776178146.126629:0:6257:0:(ldlm_lock.c:820:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(CW) ns: mdt-lustre-MDT0000_UUID lock: 0000000026fd08a4/0xa40083ab9bf5328c lrc: 3/0,1 mode: CW/CW res: [0x2000013a1:0x1:0x0].0x0 bits 0x2/0x0 rrc: 4 type: IBT gid 0 flags: 0x50210000000000 nid: local remote: 0x0 expref: -99 pid: 6257 timeout: 0 lvb_type: 0 00010000:00010000:3.0:1776178146.126641:0:6257:0:(ldlm_lock.c:879:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: mdt-lustre-MDT0000_UUID lock: 0000000026fd08a4/0xa40083ab9bf5328c lrc: 2/0,0 mode: CW/CW res: [0x2000013a1:0x1:0x0].0x0 bits 0x2/0x0 rrc: 4 type: IBT gid 0 flags: 0x50210400000000 nid: local remote: 0x0 expref: -99 pid: 6257 timeout: 0 lvb_type: 0 00010000:00010000:3.0:1776178146.126647:0:6257:0:(ldlm_lockd.c:1879:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: mdt-lustre-MDT0000_UUID lock: 0000000026fd08a4/0xa40083ab9bf5328c lrc: 3/0,0 mode: CW/CW res: [0x2000013a1:0x1:0x0].0x0 bits 0x2/0x0 rrc: 4 type: IBT gid 0 flags: 0x40210400000000 nid: local remote: 0x0 expref: -99 pid: 6257 timeout: 0 lvb_type: 0 00010000:00010000:3.0:1776178146.126654:0:6257:0:(ldlm_lockd.c:1893:ldlm_handle_bl_callback()) Lock 0000000026fd08a4 already unused, calling callback (0000000079e78348) 00010000:00010000:3.0:1776178146.126657:0:6257:0:(ldlm_request.c:364:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: mdt-lustre-MDT0000_UUID lock: 0000000026fd08a4/0xa40083ab9bf5328c lrc: 3/0,0 mode: CW/CW res: [0x2000013a1:0x1:0x0].0x0 bits 0x2/0x0 rrc: 4 type: IBT gid 0 flags: 0x40210400000000 nid: local remote: 0x0 expref: -99 pid: 6257 timeout: 0 lvb_type: 0 00010000:00010000:3.0:1776178146.126663:0:6257:0:(ldlm_request.c:1257:ldlm_cli_cancel_local()) ### server-side local cancel ns: mdt-lustre-MDT0000_UUID lock: 0000000026fd08a4/0xa40083ab9bf5328c lrc: 4/0,0 mode: CW/CW res: [0x2000013a1:0x1:0x0].0x0 bits 0x2/0x0 rrc: 4 type: IBT gid 0 flags: 0x40218400000000 nid: local remote: 0x0 expref: -99 pid: 6257 timeout: 0 lvb_type: 0 00010000:00010000:3.0:1776178146.126673:0:6257:0:(ldlm_inodebits.c:94:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x2000013a1:0x1:0x0].0x0 (00000000315facbe) 00010000:00010000:3.0:1776178146.126676:0:6257:0:(ldlm_inodebits.c:96:ldlm_reprocess_inodebits_queue()) Hint 2 00010000:00010000:3.0:1776178146.126679:0:6257:0:(ldlm_lockd.c:1903:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: mdt-lustre-MDT0000_UUID lock: 0000000026fd08a4/0xa40083ab9bf5328c lrc: 2/0,0 mode: --/CW res: [0x2000013a1:0x1:0x0].0x0 bits 0x2/0x0 rrc: 4 type: IBT gid 0 flags: 0x44a19400000000 nid: local remote: 0x0 expref: -99 pid: 6257 timeout: 0 lvb_type: 0 00010000:00010000:3.0:1776178146.126684:0:6257:0:(ldlm_lock.c:216:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: mdt-lustre-MDT0000_UUID lock: 0000000026fd08a4/0xa40083ab9bf5328c lrc: 0/0,0 mode: --/CW res: [0x2000013a1:0x1:0x0].0x0 bits 0x2/0x0 rrc: 4 type: IBT gid 0 flags: 0x44a19400000000 nid: local remote: 0x0 expref: -99 pid: 6257 timeout: 0 lvb_type: 0 00000004:00010000:3.0:1776178146.126692:0:6257:0:(mdt_handler.c:3992:mdt_save_lock()) ### save lock request 0000000088356013 reply state 000000006f277865 transno 12884901940 ns: mdt-lustre-MDT0000_UUID lock: 000000006a188f9a/0xa40083ab9bf53293 lrc: 3/0,1 mode: PW/PW res: [0x2000013a1:0x1:0x0].0x7fbd58ea bits 0x2/0x0 rrc: 2 type: IBT gid 0 flags: 0x40210000000000 nid: local remote: 0x0 expref: -99 pid: 6257 timeout: 0 lvb_type: 0 00000100:00100000:3.0:1776178146.126779:0:6257:0:(service.c:2372:ptlrpc_server_handle_request()) Handled RPC req@0000000088356013 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_002:b50963bd-8eba-4432-9a54-8888d00e3fa1+39:17037:x1862456979088128:12345-192.168.204.3@tcp:36:rm.0 Request processed in 972us (2106us total) trans 12884901940 rc 0/0 00000100:00100000:3.0:1776178146.126787:0:6257:0:(nrs_fifo.c:241:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.204.3@tcp, seq: 6521 00010000:00010000:0.0:1776178146.128923:0:6245:0:(ldlm_lockd.c:2789:ldlm_bl_get_work()) Got 1 locks of 0 total in blp. (0 blwis in pool) 00010000:00010000:0.0:1776178146.128938:0:6245:0:(ldlm_lockd.c:1879:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: mdt-lustre-MDT0000_UUID lock: 00000000591efbb9/0xa40083ab9bf532a1 lrc: 2/0,0 mode: PW/PW res: [0x2000013a1:0x8:0x0].0x0 bits 0x40/0x0 rrc: 4 type: IBT gid 18446635902900449728 flags: 0x40010480000000 nid: local remote: 0x0 expref: -99 pid: 6257 timeout: 0 lvb_type: 0 00010000:00010000:0.0:1776178146.128955:0:6245:0:(ldlm_lockd.c:1893:ldlm_handle_bl_callback()) Lock 00000000591efbb9 already unused, calling callback (0000000063c97258) 00010000:00010000:0.0:1776178146.128958:0:6245:0:(ldlm_request.c:364:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: mdt-lustre-MDT0000_UUID lock: 00000000591efbb9/0xa40083ab9bf532a1 lrc: 2/0,0 mode: PW/PW res: [0x2000013a1:0x8:0x0].0x0 bits 0x40/0x0 rrc: 4 type: IBT gid 18446635902900449728 flags: 0x40010480000000 nid: local remote: 0x0 expref: -99 pid: 6257 timeout: 0 lvb_type: 0 00010000:00010000:0.0:1776178146.128970:0:6245:0:(ldlm_request.c:1257:ldlm_cli_cancel_local()) ### server-side local cancel ns: mdt-lustre-MDT0000_UUID lock: 00000000591efbb9/0xa40083ab9bf532a1 lrc: 3/0,0 mode: PW/PW res: [0x2000013a1:0x8:0x0].0x0 bits 0x40/0x0 rrc: 4 type: IBT gid 18446635902900449728 flags: 0x40018480000000 nid: local remote: 0x0 expref: -99 pid: 6257 timeout: 0 lvb_type: 0 00010000:00010000:0.0:1776178146.128990:0:6245:0:(ldlm_inodebits.c:94:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x2000013a1:0x8:0x0].0x0 (000000009a65918f) 00010000:00010000:0.0:1776178146.128993:0:6245:0:(ldlm_inodebits.c:96:ldlm_reprocess_inodebits_queue()) Hint 40 00010000:00010000:0.0:1776178146.128997:0:6245:0:(ldlm_lockd.c:1903:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: mdt-lustre-MDT0000_UUID lock: 00000000591efbb9/0xa40083ab9bf532a1 lrc: 1/0,0 mode: --/PW res: [0x2000013a1:0x8:0x0].0x0 bits 0x40/0x0 rrc: 4 type: IBT gid 18446635902900449728 flags: 0x44819480000000 nid: local remote: 0x0 expref: -99 pid: 6257 timeout: 0 lvb_type: 0 00010000:00010000:0.0:1776178146.129002:0:6245:0:(ldlm_lock.c:216:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: mdt-lustre-MDT0000_UUID lock: 00000000591efbb9/0xa40083ab9bf532a1 lrc: 0/0,0 mode: --/PW res: [0x2000013a1:0x8:0x0].0x0 bits 0x40/0x0 rrc: 4 type: IBT gid 18446635902900449728 flags: 0x44819480000000 nid: local remote: 0x0 expref: -99 pid: 6257 timeout: 0 lvb_type: 0 00010000:00010000:0.0:1776178146.129035:0:6245:0:(ldlm_lockd.c:2792:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00010000:00010000:0.0:1776178146.129036:0:6245:0:(ldlm_lockd.c:2792:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00010000:00010000:0.0:1776178146.129037:0:6245:0:(ldlm_lockd.c:2792:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00010000:00010000:2.0:1776178146.129978:0:3340:0:(ldlm_lock.c:1064:ldlm_granted_list_add_lock()) ### About to add lock: ns: mdt-lustre-MDT0000_UUID lock: 000000006a188f9a/0xa40083ab9bf53293 lrc: 3/0,1 mode: COS/COS res: [0x2000013a1:0x1:0x0].0x7fbd58ea bits 0x2/0x0 rrc: 2 type: IBT gid 0 flags: 0x50210000000000 nid: local remote: 0x0 expref: -99 pid: 6257 timeout: 0 lvb_type: 0 00010000:00010000:2.0:1776178146.130002:0:3340:0:(ldlm_inodebits.c:94:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x2000013a1:0x1:0x0].0x7fbd58ea (00000000e9c8e2b7) 00010000:00010000:2.0:1776178146.130004:0:3340:0:(ldlm_inodebits.c:96:ldlm_reprocess_inodebits_queue()) Hint 2 00010000:00010000:2.0:1776178146.130014:0:3340:0:(ldlm_lock.c:1064:ldlm_granted_list_add_lock()) ### About to add lock: ns: mdt-lustre-MDT0000_UUID lock: 00000000dcea299c/0xa40083ab9bf5329a lrc: 3/0,1 mode: COS/COS res: [0x2000013a1:0x8:0x0].0x0 bits 0x3/0x0 rrc: 3 type: IBT gid 0 flags: 0x50210000000000 nid: local remote: 0x0 expref: -99 pid: 6257 timeout: 0 lvb_type: 0 00010000:00010000:2.0:1776178146.130021:0:3340:0:(ldlm_inodebits.c:94:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x2000013a1:0x8:0x0].0x0 (000000009a65918f) 00010000:00010000:2.0:1776178146.130023:0:3340:0:(ldlm_inodebits.c:96:ldlm_reprocess_inodebits_queue()) Hint 3 00000100:00100000:0.0:1776178146.131022:0:3347:0:(events.c:361:request_in_callback()) peer: 12345-192.168.204.3@tcp (source: 12345-192.168.204.3@tcp) 00000100:00100000:3.0:1776178146.131116:0:6257:0:(service.c:2152:ptlrpc_server_handle_req_in()) got req x1862456979088256 00000100:00100000:3.0:1776178146.131172:0:6257:0:(nrs_fifo.c:179:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.204.3@tcp, seq: 6522 00000100:00100000:3.0:1776178146.131178:0:6257:0:(service.c:2324:ptlrpc_server_handle_request()) Handling RPC req@00000000badbd310 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_002:b50963bd-8eba-4432-9a54-8888d00e3fa1+39:17037:x1862456979088256:12345-192.168.204.3@tcp:36:rm.0 00010000:00010000:3.0:1776178146.131232:0:6257:0:(ldlm_lockd.c:1720:ldlm_request_cancel()) ### server-side cancel handler START: 2 locks, starting at 0 00010000:00010000:3.0:1776178146.131267:0:6257:0:(ldlm_lock.c:216:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: mdt-lustre-MDT0000_UUID lock: 0000000068a2dd4a/0xa40083ab9bf53174 lrc: 0/0,0 mode: --/PR res: [0x2000013a1:0x5:0x0].0x0 bits 0x1b/0x0 rrc: 5 type: IBT gid 0 flags: 0x44a01000000000 nid: 192.168.204.3@tcp remote: 0x771f68aa1eb27a4d expref: 39 pid: 6257 timeout: 0 lvb_type: 0 00010000:00010000:3.0:1776178146.131284:0:6257:0:(ldlm_lock.c:216:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: mdt-lustre-MDT0000_UUID lock: 000000009a8f90ae/0xa40083ab9bf530f6 lrc: 0/0,0 mode: --/CR res: [0x2000013a1:0x5:0x0].0x0 bits 0x8/0x0 rrc: 4 type: IBT gid 0 flags: 0x44801000000000 nid: 192.168.204.3@tcp remote: 0x771f68aa1eb27a2a expref: 38 pid: 6255 timeout: 0 lvb_type: 0 00010000:00010000:3.0:1776178146.131293:0:6257:0:(ldlm_inodebits.c:94:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x2000013a1:0x5:0x0].0x0 (000000006b0fcbcb) 00010000:00010000:3.0:1776178146.131297:0:6257:0:(ldlm_lockd.c:1782:ldlm_request_cancel()) ### server-side cancel handler END 00010000:00010000:3.0:1776178146.131322:0:6257:0:(ldlm_lock.c:766:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(CW) ns: mdt-lustre-MDT0000_UUID lock: 000000009f2de1df/0xa40083ab9bf532a8 lrc: 3/0,1 mode: --/CW res: [0x2000013a1:0x1:0x0].0x0 bits 0x0/0x0 rrc: 4 type: IBT gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 6257 timeout: 0 lvb_type: 0 00010000:00010000:3.0:1776178146.131333:0:6257:0:(ldlm_lock.c:1064:ldlm_granted_list_add_lock()) ### About to add lock: ns: mdt-lustre-MDT0000_UUID lock: 000000009f2de1df/0xa40083ab9bf532a8 lrc: 3/0,1 mode: CW/CW res: [0x2000013a1:0x1:0x0].0x0 bits 0x2/0x0 rrc: 4 type: IBT gid 0 flags: 0x50210000000000 nid: local remote: 0x0 expref: -99 pid: 6257 timeout: 0 lvb_type: 0 00010000:00010000:3.0:1776178146.131342:0:6257:0:(ldlm_request.c:528:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: mdt-lustre-MDT0000_UUID lock: 000000009f2de1df/0xa40083ab9bf532a8 lrc: 3/0,1 mode: CW/CW res: [0x2000013a1:0x1:0x0].0x0 bits 0x2/0x0 rrc: 4 type: IBT gid 0 flags: 0x40210000000000 nid: local remote: 0x0 expref: -99 pid: 6257 timeout: 0 lvb_type: 0 00010000:00010000:3.0:1776178146.131354:0:6257:0:(ldlm_lock.c:766:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: mdt-lustre-MDT0000_UUID lock: 0000000055d6a00d/0xa40083ab9bf532af lrc: 3/0,1 mode: --/PW res: [0x2000013a1:0x1:0x0].0xa6b02dea bits 0x0/0x0 rrc: 2 type: IBT gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 6257 timeout: 0 lvb_type: 0 00010000:00010000:3.0:1776178146.131360:0:6257:0:(ldlm_lock.c:1064:ldlm_granted_list_add_lock()) ### About to add lock: ns: mdt-lustre-MDT0000_UUID lock: 0000000055d6a00d/0xa40083ab9bf532af lrc: 3/0,1 mode: PW/PW res: [0x2000013a1:0x1:0x0].0xa6b02dea bits 0x2/0x0 rrc: 2 type: IBT gid 0 flags: 0x50210000000000 nid: local remote: 0x0 expref: -99 pid: 6257 timeout: 0 lvb_type: 0 00010000:00010000:3.0:1776178146.131366:0:6257:0:(ldlm_request.c:528:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: mdt-lustre-MDT0000_UUID lock: 0000000055d6a00d/0xa40083ab9bf532af lrc: 3/0,1 mode: PW/PW res: [0x2000013a1:0x1:0x0].0xa6b02dea bits 0x2/0x0 rrc: 2 type: IBT gid 0 flags: 0x40210000000000 nid: local remote: 0x0 expref: -99 pid: 6257 timeout: 0 lvb_type: 0 00010000:00010000:3.0:1776178146.131427:0:6257:0:(ldlm_lock.c:766:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(EX) ns: mdt-lustre-MDT0000_UUID lock: 00000000c1e7ea3e/0xa40083ab9bf532b6 lrc: 3/0,1 mode: --/EX res: [0x2000013a1:0x5:0x0].0x0 bits 0x0/0x0 rrc: 3 type: IBT gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 6257 timeout: 0 lvb_type: 0 00010000:00010000:3.0:1776178146.131434:0:6257:0:(ldlm_lock.c:1064:ldlm_granted_list_add_lock()) ### About to add lock: ns: mdt-lustre-MDT0000_UUID lock: 00000000c1e7ea3e/0xa40083ab9bf532b6 lrc: 3/0,1 mode: EX/EX res: [0x2000013a1:0x5:0x0].0x0 bits 0x3/0x0 rrc: 3 type: IBT gid 0 flags: 0x50210000000000 nid: local remote: 0x0 expref: -99 pid: 6257 timeout: 0 lvb_type: 0 00010000:00010000:3.0:1776178146.131440:0:6257:0:(ldlm_request.c:528:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: mdt-lustre-MDT0000_UUID lock: 00000000c1e7ea3e/0xa40083ab9bf532b6 lrc: 3/0,1 mode: EX/EX res: [0x2000013a1:0x5:0x0].0x0 bits 0x3/0x0 rrc: 3 type: IBT gid 0 flags: 0x40210000000000 nid: local remote: 0x0 expref: -99 pid: 6257 timeout: 0 lvb_type: 0 00000040:00080000:3.0:1776178146.131639:0:6257:0:(llog_osd.c:738:llog_osd_write_rec()) added record [0x1:0xe:0x0].4, 64 off8448 00010000:00010000:3.0:1776178146.131683:0:6257:0:(ldlm_lock.c:766:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: mdt-lustre-MDT0000_UUID lock: 00000000bbda5115/0xa40083ab9bf532bd lrc: 3/0,1 mode: --/PW res: [0x2000013a1:0x5:0x0].0x0 bits 0x0/0x0 rrc: 4 type: IBT gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 6257 timeout: 0 lvb_type: 0 00010000:00010000:3.0:1776178146.131691:0:6257:0:(ldlm_lock.c:1064:ldlm_granted_list_add_lock()) ### About to add lock: ns: mdt-lustre-MDT0000_UUID lock: 00000000bbda5115/0xa40083ab9bf532bd lrc: 3/0,1 mode: PW/PW res: [0x2000013a1:0x5:0x0].0x0 bits 0x40/0x0 rrc: 4 type: IBT gid 18446635902900449728 flags: 0x50010080000000 nid: local remote: 0x0 expref: -99 pid: 6257 timeout: 0 lvb_type: 0 00010000:00010000:3.0:1776178146.131698:0:6257:0:(ldlm_request.c:528:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: mdt-lustre-MDT0000_UUID lock: 00000000bbda5115/0xa40083ab9bf532bd lrc: 3/0,1 mode: PW/PW res: [0x2000013a1:0x5:0x0].0x0 bits 0x40/0x0 rrc: 4 type: IBT gid 18446635902900449728 flags: 0x40010080000000 nid: local remote: 0x0 expref: -99 pid: 6257 timeout: 0 lvb_type: 0 00010000:00010000:3.0:1776178146.131706:0:6257:0:(ldlm_lock.c:942:ldlm_lock_decref_and_cancel()) ### ldlm_lock_decref(PW) ns: mdt-lustre-MDT0000_UUID lock: 00000000bbda5115/0xa40083ab9bf532bd lrc: 4/0,1 mode: PW/PW res: [0x2000013a1:0x5:0x0].0x0 bits 0x40/0x0 rrc: 4 type: IBT gid 18446635902900449728 flags: 0x40010080000000 nid: local remote: 0x0 expref: -99 pid: 6257 timeout: 0 lvb_type: 0 00010000:00010000:3.0:1776178146.131713:0:6257:0:(ldlm_lock.c:820:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: mdt-lustre-MDT0000_UUID lock: 00000000bbda5115/0xa40083ab9bf532bd lrc: 4/0,1 mode: PW/PW res: [0x2000013a1:0x5:0x0].0x0 bits 0x40/0x0 rrc: 4 type: IBT gid 18446635902900449728 flags: 0x50010480000000 nid: local remote: 0x0 expref: -99 pid: 6257 timeout: 0 lvb_type: 0 00010000:00010000:3.0:1776178146.131718:0:6257:0:(ldlm_lock.c:879:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: mdt-lustre-MDT0000_UUID lock: 00000000bbda5115/0xa40083ab9bf532bd lrc: 3/0,0 mode: PW/PW res: [0x2000013a1:0x5:0x0].0x0 bits 0x40/0x0 rrc: 4 type: IBT gid 18446635902900449728 flags: 0x50010480000000 nid: local remote: 0x0 expref: -99 pid: 6257 timeout: 0 lvb_type: 0 00010000:00010000:3.0:1776178146.131731:0:6257:0:(ldlm_lockd.c:2159:__ldlm_bl_to_thread()) added 1/1 locks to regular blp list, 1 blwis in pool 00000004:00010000:3.0:1776178146.131747:0:6257:0:(mdt_handler.c:3992:mdt_save_lock()) ### save lock request 00000000badbd310 reply state 000000009edbdc7d transno 12884901941 ns: mdt-lustre-MDT0000_UUID lock: 00000000c1e7ea3e/0xa40083ab9bf532b6 lrc: 3/0,1 mode: EX/EX res: [0x2000013a1:0x5:0x0].0x0 bits 0x3/0x0 rrc: 4 type: IBT gid 0 flags: 0x40210000000000 nid: local remote: 0x0 expref: -99 pid: 6257 timeout: 0 lvb_type: 0 00010000:00010000:3.0:1776178146.131856:0:6257:0:(ldlm_lock.c:820:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(CW) ns: mdt-lustre-MDT0000_UUID lock: 000000009f2de1df/0xa40083ab9bf532a8 lrc: 3/0,1 mode: CW/CW res: [0x2000013a1:0x1:0x0].0x0 bits 0x2/0x0 rrc: 4 type: IBT gid 0 flags: 0x50210000000000 nid: local remote: 0x0 expref: -99 pid: 6257 timeout: 0 lvb_type: 0 00010000:00010000:3.0:1776178146.131863:0:6257:0:(ldlm_lock.c:879:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: mdt-lustre-MDT0000_UUID lock: 000000009f2de1df/0xa40083ab9bf532a8 lrc: 2/0,0 mode: CW/CW res: [0x2000013a1:0x1:0x0].0x0 bits 0x2/0x0 rrc: 4 type: IBT gid 0 flags: 0x50210400000000 nid: local remote: 0x0 expref: -99 pid: 6257 timeout: 0 lvb_type: 0 00010000:00010000:3.0:1776178146.131869:0:6257:0:(ldlm_lockd.c:1879:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: mdt-lustre-MDT0000_UUID lock: 000000009f2de1df/0xa40083ab9bf532a8 lrc: 3/0,0 mode: CW/CW res: [0x2000013a1:0x1:0x0].0x0 bits 0x2/0x0 rrc: 4 type: IBT gid 0 flags: 0x40210400000000 nid: local remote: 0x0 expref: -99 pid: 6257 timeout: 0 lvb_type: 0 00010000:00010000:3.0:1776178146.131874:0:6257:0:(ldlm_lockd.c:1893:ldlm_handle_bl_callback()) Lock 000000009f2de1df already unused, calling callback (0000000079e78348) 00010000:00010000:3.0:1776178146.131877:0:6257:0:(ldlm_request.c:364:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: mdt-lustre-MDT0000_UUID lock: 000000009f2de1df/0xa40083ab9bf532a8 lrc: 3/0,0 mode: CW/CW res: [0x2000013a1:0x1:0x0].0x0 bits 0x2/0x0 rrc: 4 type: IBT gid 0 flags: 0x40210400000000 nid: local remote: 0x0 expref: -99 pid: 6257 timeout: 0 lvb_type: 0 00010000:00010000:3.0:1776178146.131884:0:6257:0:(ldlm_request.c:1257:ldlm_cli_cancel_local()) ### server-side local cancel ns: mdt-lustre-MDT0000_UUID lock: 000000009f2de1df/0xa40083ab9bf532a8 lrc: 4/0,0 mode: CW/CW res: [0x2000013a1:0x1:0x0].0x0 bits 0x2/0x0 rrc: 4 type: IBT gid 0 flags: 0x40218400000000 nid: local remote: 0x0 expref: -99 pid: 6257 timeout: 0 lvb_type: 0 00010000:00010000:3.0:1776178146.131891:0:6257:0:(ldlm_inodebits.c:94:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x2000013a1:0x1:0x0].0x0 (00000000315facbe) 00010000:00010000:3.0:1776178146.131894:0:6257:0:(ldlm_inodebits.c:96:ldlm_reprocess_inodebits_queue()) Hint 2 00010000:00010000:3.0:1776178146.131896:0:6257:0:(ldlm_lockd.c:1903:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: mdt-lustre-MDT0000_UUID lock: 000000009f2de1df/0xa40083ab9bf532a8 lrc: 2/0,0 mode: --/CW res: [0x2000013a1:0x1:0x0].0x0 bits 0x2/0x0 rrc: 4 type: IBT gid 0 flags: 0x44a19400000000 nid: local remote: 0x0 expref: -99 pid: 6257 timeout: 0 lvb_type: 0 00010000:00010000:3.0:1776178146.131901:0:6257:0:(ldlm_lock.c:216:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: mdt-lustre-MDT0000_UUID lock: 000000009f2de1df/0xa40083ab9bf532a8 lrc: 0/0,0 mode: --/CW res: [0x2000013a1:0x1:0x0].0x0 bits 0x2/0x0 rrc: 4 type: IBT gid 0 flags: 0x44a19400000000 nid: local remote: 0x0 expref: -99 pid: 6257 timeout: 0 lvb_type: 0 00000004:00010000:3.0:1776178146.131908:0:6257:0:(mdt_handler.c:3992:mdt_save_lock()) ### save lock request 00000000badbd310 reply state 000000009edbdc7d transno 12884901941 ns: mdt-lustre-MDT0000_UUID lock: 0000000055d6a00d/0xa40083ab9bf532af lrc: 3/0,1 mode: PW/PW res: [0x2000013a1:0x1:0x0].0xa6b02dea bits 0x2/0x0 rrc: 2 type: IBT gid 0 flags: 0x40210000000000 nid: local remote: 0x0 expref: -99 pid: 6257 timeout: 0 lvb_type: 0 00000100:00100000:3.0:1776178146.132211:0:6257:0:(service.c:2372:ptlrpc_server_handle_request()) Handled RPC req@00000000badbd310 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_002:b50963bd-8eba-4432-9a54-8888d00e3fa1+39:17037:x1862456979088256:12345-192.168.204.3@tcp:36:rm.0 Request processed in 1031us (1194us total) trans 12884901941 rc 0/0 00000100:00100000:3.0:1776178146.132223:0:6257:0:(nrs_fifo.c:241:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.204.3@tcp, seq: 6522 00010000:00010000:0.0:1776178146.132500:0:9225:0:(ldlm_lockd.c:2789:ldlm_bl_get_work()) Got 1 locks of 0 total in blp. (0 blwis in pool) 00010000:00010000:0.0:1776178146.132505:0:9225:0:(ldlm_lockd.c:1879:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: mdt-lustre-MDT0000_UUID lock: 00000000bbda5115/0xa40083ab9bf532bd lrc: 2/0,0 mode: PW/PW res: [0x2000013a1:0x5:0x0].0x0 bits 0x40/0x0 rrc: 4 type: IBT gid 18446635902900449728 flags: 0x40010480000000 nid: local remote: 0x0 expref: -99 pid: 6257 timeout: 0 lvb_type: 0 00010000:00010000:0.0:1776178146.132515:0:9225:0:(ldlm_lockd.c:1893:ldlm_handle_bl_callback()) Lock 00000000bbda5115 already unused, calling callback (0000000063c97258) 00010000:00010000:0.0:1776178146.132517:0:9225:0:(ldlm_request.c:364:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: mdt-lustre-MDT0000_UUID lock: 00000000bbda5115/0xa40083ab9bf532bd lrc: 2/0,0 mode: PW/PW res: [0x2000013a1:0x5:0x0].0x0 bits 0x40/0x0 rrc: 4 type: IBT gid 18446635902900449728 flags: 0x40010480000000 nid: local remote: 0x0 expref: -99 pid: 6257 timeout: 0 lvb_type: 0 00010000:00010000:0.0:1776178146.132523:0:9225:0:(ldlm_request.c:1257:ldlm_cli_cancel_local()) ### server-side local cancel ns: mdt-lustre-MDT0000_UUID lock: 00000000bbda5115/0xa40083ab9bf532bd lrc: 3/0,0 mode: PW/PW res: [0x2000013a1:0x5:0x0].0x0 bits 0x40/0x0 rrc: 4 type: IBT gid 18446635902900449728 flags: 0x40018480000000 nid: local remote: 0x0 expref: -99 pid: 6257 timeout: 0 lvb_type: 0 00010000:00010000:0.0:1776178146.132532:0:9225:0:(ldlm_inodebits.c:94:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x2000013a1:0x5:0x0].0x0 (000000006b0fcbcb) 00010000:00010000:0.0:1776178146.132534:0:9225:0:(ldlm_inodebits.c:96:ldlm_reprocess_inodebits_queue()) Hint 40 00010000:00010000:0.0:1776178146.132537:0:9225:0:(ldlm_lockd.c:1903:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: mdt-lustre-MDT0000_UUID lock: 00000000bbda5115/0xa40083ab9bf532bd lrc: 1/0,0 mode: --/PW res: [0x2000013a1:0x5:0x0].0x0 bits 0x40/0x0 rrc: 4 type: IBT gid 18446635902900449728 flags: 0x44819480000000 nid: local remote: 0x0 expref: -99 pid: 6257 timeout: 0 lvb_type: 0 00010000:00010000:0.0:1776178146.132541:0:9225:0:(ldlm_lock.c:216:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: mdt-lustre-MDT0000_UUID lock: 00000000bbda5115/0xa40083ab9bf532bd lrc: 0/0,0 mode: --/PW res: [0x2000013a1:0x5:0x0].0x0 bits 0x40/0x0 rrc: 4 type: IBT gid 18446635902900449728 flags: 0x44819480000000 nid: local remote: 0x0 expref: -99 pid: 6257 timeout: 0 lvb_type: 0 00010000:00010000:0.0:1776178146.132552:0:9225:0:(ldlm_lockd.c:2792:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00010000:00010000:0.0:1776178146.132554:0:9225:0:(ldlm_lockd.c:2792:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00010000:00010000:0.0:1776178146.132555:0:9225:0:(ldlm_lockd.c:2792:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00010000:00010000:3.0:1776178146.135692:0:3339:0:(ldlm_lock.c:1064:ldlm_granted_list_add_lock()) ### About to add lock: ns: mdt-lustre-MDT0000_UUID lock: 0000000055d6a00d/0xa40083ab9bf532af lrc: 3/0,1 mode: COS/COS res: [0x2000013a1:0x1:0x0].0xa6b02dea bits 0x2/0x0 rrc: 2 type: IBT gid 0 flags: 0x50210000000000 nid: local remote: 0x0 expref: -99 pid: 6257 timeout: 0 lvb_type: 0 00010000:00010000:3.0:1776178146.135717:0:3339:0:(ldlm_inodebits.c:94:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x2000013a1:0x1:0x0].0xa6b02dea (000000007bfb9649) 00010000:00010000:3.0:1776178146.135719:0:3339:0:(ldlm_inodebits.c:96:ldlm_reprocess_inodebits_queue()) Hint 2 00010000:00010000:3.0:1776178146.135727:0:3339:0:(ldlm_lock.c:1064:ldlm_granted_list_add_lock()) ### About to add lock: ns: mdt-lustre-MDT0000_UUID lock: 00000000c1e7ea3e/0xa40083ab9bf532b6 lrc: 3/0,1 mode: COS/COS res: [0x2000013a1:0x5:0x0].0x0 bits 0x3/0x0 rrc: 3 type: IBT gid 0 flags: 0x50210000000000 nid: local remote: 0x0 expref: -99 pid: 6257 timeout: 0 lvb_type: 0 00010000:00010000:3.0:1776178146.135733:0:3339:0:(ldlm_inodebits.c:94:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x2000013a1:0x5:0x0].0x0 (000000006b0fcbcb) 00010000:00010000:3.0:1776178146.135734:0:3339:0:(ldlm_inodebits.c:96:ldlm_reprocess_inodebits_queue()) Hint 3 00000100:00100000:0.0:1776178146.136597:0:3347:0:(events.c:361:request_in_callback()) peer: 12345-192.168.204.3@tcp (source: 12345-192.168.204.3@tcp) 00000100:00100000:3.0:1776178146.138439:0:6259:0:(service.c:2152:ptlrpc_server_handle_req_in()) got req x1862456979088384 00000100:00100000:3.0:1776178146.138649:0:6259:0:(nrs_fifo.c:179:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.204.3@tcp, seq: 2132 00000100:00100000:3.0:1776178146.138657:0:6259:0:(service.c:2324:ptlrpc_server_handle_request()) Handling RPC req@00000000f8c48966 pname:cluuid+ref:pid:xid:nid:opc:job mdt_rdpg00_001:b50963bd-8eba-4432-9a54-8888d00e3fa1+39:17037:x1862456979088384:12345-192.168.204.3@tcp:35:rm.0 00000100:00100000:3.0:1776178146.138904:0:6259:0:(service.c:2372:ptlrpc_server_handle_request()) Handled RPC req@00000000f8c48966 pname:cluuid+ref:pid:xid:nid:opc:job mdt_rdpg00_001:b50963bd-8eba-4432-9a54-8888d00e3fa1+40:17037:x1862456979088384:12345-192.168.204.3@tcp:35:rm.0 Request processed in 247us (2312us total) trans 12884901942 rc 0/0 00000100:00100000:3.0:1776178146.138911:0:6259:0:(nrs_fifo.c:241:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.204.3@tcp, seq: 2132 00000100:00100000:0.0:1776178146.141836:0:3348:0:(events.c:361:request_in_callback()) peer: 12345-192.168.204.3@tcp (source: 12345-192.168.204.3@tcp) 00000100:00100000:3.0:1776178146.141912:0:6257:0:(service.c:2152:ptlrpc_server_handle_req_in()) got req x1862456979088512 00000100:00100000:3.0:1776178146.142004:0:6257:0:(nrs_fifo.c:179:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.204.3@tcp, seq: 6523 00000100:00100000:3.0:1776178146.142019:0:6257:0:(service.c:2324:ptlrpc_server_handle_request()) Handling RPC req@000000003493d657 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_002:b50963bd-8eba-4432-9a54-8888d00e3fa1+40:17037:x1862456979088512:12345-192.168.204.3@tcp:36:rm.0 00010000:00010000:3.0:1776178146.142094:0:6257:0:(ldlm_lockd.c:1720:ldlm_request_cancel()) ### server-side cancel handler START: 2 locks, starting at 0 00010000:00010000:3.0:1776178146.142128:0:6257:0:(ldlm_lock.c:216:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: mdt-lustre-MDT0000_UUID lock: 00000000a27c19b3/0xa40083ab9bf531f2 lrc: 0/0,0 mode: --/CR res: [0x2000013a1:0x1:0x0].0x0 bits 0x5/0x0 rrc: 4 type: IBT gid 0 flags: 0x44a01000000000 nid: 192.168.204.3@tcp remote: 0x771f68aa1eb27a77 expref: 40 pid: 8250 timeout: 0 lvb_type: 0 00010000:00010000:3.0:1776178146.142150:0:6257:0:(ldlm_lock.c:216:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: mdt-lustre-MDT0000_UUID lock: 000000001b3f1324/0xa40083ab9bf52ee9 lrc: 0/0,0 mode: --/PR res: [0x2000013a1:0x1:0x0].0x0 bits 0x11/0x0 rrc: 3 type: IBT gid 0 flags: 0x44a01000000000 nid: 192.168.204.3@tcp remote: 0x771f68aa1eb2796d expref: 39 pid: 6256 timeout: 0 lvb_type: 0 00010000:00010000:3.0:1776178146.142166:0:6257:0:(ldlm_inodebits.c:94:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x2000013a1:0x1:0x0].0x0 (00000000315facbe) 00010000:00010000:3.0:1776178146.142177:0:6257:0:(ldlm_lockd.c:1782:ldlm_request_cancel()) ### server-side cancel handler END 00010000:00010000:3.0:1776178146.142205:0:6257:0:(ldlm_lock.c:766:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(CW) ns: mdt-lustre-MDT0000_UUID lock: 000000006c464689/0xa40083ab9bf532c4 lrc: 3/0,1 mode: --/CW res: [0x200000007:0x1:0x0].0x0 bits 0x0/0x0 rrc: 4 type: IBT gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 6257 timeout: 0 lvb_type: 0 00010000:00010000:3.0:1776178146.142220:0:6257:0:(ldlm_lock.c:1064:ldlm_granted_list_add_lock()) ### About to add lock: ns: mdt-lustre-MDT0000_UUID lock: 000000006c464689/0xa40083ab9bf532c4 lrc: 3/0,1 mode: CW/CW res: [0x200000007:0x1:0x0].0x0 bits 0x2/0x0 rrc: 4 type: IBT gid 0 flags: 0x50210000000000 nid: local remote: 0x0 expref: -99 pid: 6257 timeout: 0 lvb_type: 0 00010000:00010000:3.0:1776178146.142230:0:6257:0:(ldlm_request.c:528:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: mdt-lustre-MDT0000_UUID lock: 000000006c464689/0xa40083ab9bf532c4 lrc: 3/0,1 mode: CW/CW res: [0x200000007:0x1:0x0].0x0 bits 0x2/0x0 rrc: 4 type: IBT gid 0 flags: 0x40210000000000 nid: local remote: 0x0 expref: -99 pid: 6257 timeout: 0 lvb_type: 0 00010000:00010000:3.0:1776178146.142255:0:6257:0:(ldlm_lock.c:766:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: mdt-lustre-MDT0000_UUID lock: 000000002a990613/0xa40083ab9bf532cb lrc: 3/0,1 mode: --/PW res: [0x200000007:0x1:0x0].0xc2d93176 bits 0x0/0x0 rrc: 2 type: IBT gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 6257 timeout: 0 lvb_type: 0 00010000:00010000:3.0:1776178146.142263:0:6257:0:(ldlm_lock.c:1064:ldlm_granted_list_add_lock()) ### About to add lock: ns: mdt-lustre-MDT0000_UUID lock: 000000002a990613/0xa40083ab9bf532cb lrc: 3/0,1 mode: PW/PW res: [0x200000007:0x1:0x0].0xc2d93176 bits 0x2/0x0 rrc: 2 type: IBT gid 0 flags: 0x50210000000000 nid: local remote: 0x0 expref: -99 pid: 6257 timeout: 0 lvb_type: 0 00010000:00010000:3.0:1776178146.142269:0:6257:0:(ldlm_request.c:528:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: mdt-lustre-MDT0000_UUID lock: 000000002a990613/0xa40083ab9bf532cb lrc: 3/0,1 mode: PW/PW res: [0x200000007:0x1:0x0].0xc2d93176 bits 0x2/0x0 rrc: 2 type: IBT gid 0 flags: 0x40210000000000 nid: local remote: 0x0 expref: -99 pid: 6257 timeout: 0 lvb_type: 0 00010000:00010000:3.0:1776178146.142363:0:6257:0:(ldlm_lock.c:766:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(EX) ns: mdt-lustre-MDT0000_UUID lock: 00000000014a8a18/0xa40083ab9bf532d2 lrc: 3/0,1 mode: --/EX res: [0x2000013a1:0x1:0x0].0x0 bits 0x0/0x0 rrc: 2 type: IBT gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 6257 timeout: 0 lvb_type: 0 00010000:00010000:3.0:1776178146.142371:0:6257:0:(ldlm_lock.c:1064:ldlm_granted_list_add_lock()) ### About to add lock: ns: mdt-lustre-MDT0000_UUID lock: 00000000014a8a18/0xa40083ab9bf532d2 lrc: 3/0,1 mode: EX/EX res: [0x2000013a1:0x1:0x0].0x0 bits 0x3/0x0 rrc: 2 type: IBT gid 0 flags: 0x50210000000000 nid: local remote: 0x0 expref: -99 pid: 6257 timeout: 0 lvb_type: 0 00010000:00010000:3.0:1776178146.142378:0:6257:0:(ldlm_request.c:528:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: mdt-lustre-MDT0000_UUID lock: 00000000014a8a18/0xa40083ab9bf532d2 lrc: 3/0,1 mode: EX/EX res: [0x2000013a1:0x1:0x0].0x0 bits 0x3/0x0 rrc: 2 type: IBT gid 0 flags: 0x40210000000000 nid: local remote: 0x0 expref: -99 pid: 6257 timeout: 0 lvb_type: 0 00000004:00200000:3.0:1776178146.142619:0:6257:0:(mdt_handler.c:912:mdt_pack_attr2body()) [0x2000013a1:0x1:0x0]: returning size 4096 00000004:00010000:3.0:1776178146.142638:0:6257:0:(mdt_handler.c:3992:mdt_save_lock()) ### save lock request 000000003493d657 reply state 00000000d88f680f transno 12884901943 ns: mdt-lustre-MDT0000_UUID lock: 00000000014a8a18/0xa40083ab9bf532d2 lrc: 3/0,1 mode: EX/EX res: [0x2000013a1:0x1:0x0].0x0 bits 0x3/0x0 rrc: 2 type: IBT gid 0 flags: 0x40210000000000 nid: local remote: 0x0 expref: -99 pid: 6257 timeout: 0 lvb_type: 0 00010000:00010000:3.0:1776178146.142674:0:6257:0:(ldlm_lock.c:820:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(CW) ns: mdt-lustre-MDT0000_UUID lock: 000000006c464689/0xa40083ab9bf532c4 lrc: 3/0,1 mode: CW/CW res: [0x200000007:0x1:0x0].0x0 bits 0x2/0x0 rrc: 4 type: IBT gid 0 flags: 0x50210000000000 nid: local remote: 0x0 expref: -99 pid: 6257 timeout: 0 lvb_type: 0 00010000:00010000:3.0:1776178146.142682:0:6257:0:(ldlm_lock.c:879:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: mdt-lustre-MDT0000_UUID lock: 000000006c464689/0xa40083ab9bf532c4 lrc: 2/0,0 mode: CW/CW res: [0x200000007:0x1:0x0].0x0 bits 0x2/0x0 rrc: 4 type: IBT gid 0 flags: 0x50210400000000 nid: local remote: 0x0 expref: -99 pid: 6257 timeout: 0 lvb_type: 0 00010000:00010000:3.0:1776178146.142688:0:6257:0:(ldlm_lockd.c:1879:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: mdt-lustre-MDT0000_UUID lock: 000000006c464689/0xa40083ab9bf532c4 lrc: 3/0,0 mode: CW/CW res: [0x200000007:0x1:0x0].0x0 bits 0x2/0x0 rrc: 4 type: IBT gid 0 flags: 0x40210400000000 nid: local remote: 0x0 expref: -99 pid: 6257 timeout: 0 lvb_type: 0 00010000:00010000:3.0:1776178146.142694:0:6257:0:(ldlm_lockd.c:1893:ldlm_handle_bl_callback()) Lock 000000006c464689 already unused, calling callback (0000000079e78348) 00010000:00010000:3.0:1776178146.142698:0:6257:0:(ldlm_request.c:364:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: mdt-lustre-MDT0000_UUID lock: 000000006c464689/0xa40083ab9bf532c4 lrc: 3/0,0 mode: CW/CW res: [0x200000007:0x1:0x0].0x0 bits 0x2/0x0 rrc: 4 type: IBT gid 0 flags: 0x40210400000000 nid: local remote: 0x0 expref: -99 pid: 6257 timeout: 0 lvb_type: 0 00010000:00010000:3.0:1776178146.142704:0:6257:0:(ldlm_request.c:1257:ldlm_cli_cancel_local()) ### server-side local cancel ns: mdt-lustre-MDT0000_UUID lock: 000000006c464689/0xa40083ab9bf532c4 lrc: 4/0,0 mode: CW/CW res: [0x200000007:0x1:0x0].0x0 bits 0x2/0x0 rrc: 4 type: IBT gid 0 flags: 0x40218400000000 nid: local remote: 0x0 expref: -99 pid: 6257 timeout: 0 lvb_type: 0 00010000:00010000:3.0:1776178146.142712:0:6257:0:(ldlm_inodebits.c:94:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x200000007:0x1:0x0].0x0 (00000000ad0d4094) 00010000:00010000:3.0:1776178146.142714:0:6257:0:(ldlm_inodebits.c:96:ldlm_reprocess_inodebits_queue()) Hint 2 00010000:00010000:3.0:1776178146.142717:0:6257:0:(ldlm_lockd.c:1903:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: mdt-lustre-MDT0000_UUID lock: 000000006c464689/0xa40083ab9bf532c4 lrc: 2/0,0 mode: --/CW res: [0x200000007:0x1:0x0].0x0 bits 0x2/0x0 rrc: 4 type: IBT gid 0 flags: 0x44a19400000000 nid: local remote: 0x0 expref: -99 pid: 6257 timeout: 0 lvb_type: 0 00010000:00010000:3.0:1776178146.142722:0:6257:0:(ldlm_lock.c:216:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: mdt-lustre-MDT0000_UUID lock: 000000006c464689/0xa40083ab9bf532c4 lrc: 0/0,0 mode: --/CW res: [0x200000007:0x1:0x0].0x0 bits 0x2/0x0 rrc: 4 type: IBT gid 0 flags: 0x44a19400000000 nid: local remote: 0x0 expref: -99 pid: 6257 timeout: 0 lvb_type: 0 00000004:00010000:3.0:1776178146.142730:0:6257:0:(mdt_handler.c:3992:mdt_save_lock()) ### save lock request 000000003493d657 reply state 00000000d88f680f transno 12884901943 ns: mdt-lustre-MDT0000_UUID lock: 000000002a990613/0xa40083ab9bf532cb lrc: 3/0,1 mode: PW/PW res: [0x200000007:0x1:0x0].0xc2d93176 bits 0x2/0x0 rrc: 2 type: IBT gid 0 flags: 0x40210000000000 nid: local remote: 0x0 expref: -99 pid: 6257 timeout: 0 lvb_type: 0 00000100:00100000:3.0:1776178146.142820:0:6257:0:(service.c:2372:ptlrpc_server_handle_request()) Handled RPC req@000000003493d657 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_002:b50963bd-8eba-4432-9a54-8888d00e3fa1+40:17037:x1862456979088512:12345-192.168.204.3@tcp:36:rm.0 Request processed in 801us (991us total) trans 12884901943 rc 0/0 00000100:00100000:3.0:1776178146.142829:0:6257:0:(nrs_fifo.c:241:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.204.3@tcp, seq: 6523 00010000:00010000:2.0:1776178146.154683:0:3340:0:(ldlm_lock.c:1064:ldlm_granted_list_add_lock()) ### About to add lock: ns: mdt-lustre-MDT0000_UUID lock: 000000002a990613/0xa40083ab9bf532cb lrc: 3/0,1 mode: COS/COS res: [0x200000007:0x1:0x0].0xc2d93176 bits 0x2/0x0 rrc: 2 type: IBT gid 0 flags: 0x50210000000000 nid: local remote: 0x0 expref: -99 pid: 6257 timeout: 0 lvb_type: 0 00010000:00010000:2.0:1776178146.154707:0:3340:0:(ldlm_inodebits.c:94:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x200000007:0x1:0x0].0xc2d93176 (000000007b7d95c1) 00010000:00010000:2.0:1776178146.154710:0:3340:0:(ldlm_inodebits.c:96:ldlm_reprocess_inodebits_queue()) Hint 2 00010000:00010000:2.0:1776178146.154716:0:3340:0:(ldlm_lock.c:1064:ldlm_granted_list_add_lock()) ### About to add lock: ns: mdt-lustre-MDT0000_UUID lock: 00000000014a8a18/0xa40083ab9bf532d2 lrc: 3/0,1 mode: COS/COS res: [0x2000013a1:0x1:0x0].0x0 bits 0x3/0x0 rrc: 2 type: IBT gid 0 flags: 0x50210000000000 nid: local remote: 0x0 expref: -99 pid: 6257 timeout: 0 lvb_type: 0 00010000:00010000:2.0:1776178146.154721:0:3340:0:(ldlm_inodebits.c:94:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x2000013a1:0x1:0x0].0x0 (00000000099618da) 00010000:00010000:2.0:1776178146.154723:0:3340:0:(ldlm_inodebits.c:96:ldlm_reprocess_inodebits_queue()) Hint 3 00000100:00100000:3.0:1776178146.169774:0:3353:0:(client.c:1733:ptlrpc_send_new_req()) Sending RPC req@0000000090cc55f6 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_00:lustre-MDT0001-mdtlov_UUID:3353:1862456998031872:0@lo:13:osp-pre-0-1.0 00000100:00100000:3.0:1776178146.169866:0:3353:0:(events.c:361:request_in_callback()) peer: 12345-0@lo (source: 12345-0@lo) 00000100:00100000:0.0:1776178146.172655:0:8231:0:(service.c:2152:ptlrpc_server_handle_req_in()) got req x1862456998031872 00000100:00100000:1.0:1776178146.172882:0:8231:0:(nrs_fifo.c:179:nrs_fifo_req_get()) NRS start fifo request from 12345-0@lo, seq: 434 00000100:00100000:1.0:1776178146.172889:0:8231:0:(service.c:2324:ptlrpc_server_handle_request()) Handling RPC req@00000000b13c16eb pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_create00:lustre-MDT0001-mdtlov_UUID+5:3353:x1862456998031872:12345-0@lo:13:osp-pre-0-1.0 00002000:00000024:1.0:1776178146.172927:0:8231:0:(ofd_obd.c:740:ofd_statfs()) blocks cached 315392 granted 484883776 pending 0 free 3923689472 avail 3692167168 00000100:00100000:1.0:1776178146.173025:0:8231:0:(service.c:2372:ptlrpc_server_handle_request()) Handled RPC req@00000000b13c16eb pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_create00:lustre-MDT0001-mdtlov_UUID+5:3353:x1862456998031872:12345-0@lo:13:osp-pre-0-1.0 Request processed in 135us (3159us total) trans 0 rc 0/0 00000100:00100000:1.0:1776178146.173033:0:8231:0:(nrs_fifo.c:241:nrs_fifo_req_stop()) NRS stop fifo request from 12345-0@lo, seq: 434 00000100:00100000:3.0:1776178146.173184:0:3353:0:(client.c:2215:ptlrpc_check_set()) Completed RPC req@0000000090cc55f6 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_00:lustre-MDT0001-mdtlov_UUID:3353:1862456998031872:0@lo:13:osp-pre-0-1.0 00010000:00010000:2.0:1776178146.720294:0:9224:0:(ldlm_lockd.c:2792:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00010000:00010000:1.0:1776178147.744289:0:6244:0:(ldlm_lockd.c:2792:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00010000:00010000:0.0:1776178148.771734:0:6245:0:(ldlm_lockd.c:2792:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000004:00080000:0.0:1776178149.217271:0:20151:0:(osp_sync.c:1658:osp_sync_add_commit_cb()) lustre-OST0000-osc-MDT0000: add commit cb at 856544208955ns, next at 857544205990ns, rc = 0 00000100:00100000:0.0:1776178149.217308:0:3354:0:(client.c:1733:ptlrpc_send_new_req()) Sending RPC req@000000007fcab602 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_01:lustre-MDT0000-mdtlov_UUID:3354:1862456998031936:0@lo:13:osp-pre-0-0.0 00000100:00100000:0.0:1776178149.217376:0:3354:0:(events.c:361:request_in_callback()) peer: 12345-0@lo (source: 12345-0@lo) 00000004:00080000:0.0:1776178149.217485:0:20153:0:(osp_sync.c:1658:osp_sync_add_commit_cb()) lustre-OST0001-osc-MDT0000: add commit cb at 856544424670ns, next at 857544423568ns, rc = 0 00000100:00100000:1.0:1776178149.217500:0:3355:0:(client.c:1733:ptlrpc_send_new_req()) Sending RPC req@000000008d82b734 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_02:lustre-MDT0000-mdtlov_UUID:3355:1862456998032000:0@lo:13:osp-pre-1-0.0 00000100:00100000:1.0:1776178149.217533:0:3355:0:(events.c:361:request_in_callback()) peer: 12345-0@lo (source: 12345-0@lo) 00000100:00100000:1.0:1776178149.217574:0:8231:0:(service.c:2152:ptlrpc_server_handle_req_in()) got req x1862456998031936 00000100:00100000:3.0:1776178149.217656:0:3356:0:(client.c:1733:ptlrpc_send_new_req()) Sending RPC req@00000000bcefda18 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_03:lustre-MDT0001-mdtlov_UUID:3356:1862456998032064:0@lo:41:osp-pre-0-1.0 00000100:00100000:3.0:1776178149.217690:0:3356:0:(events.c:361:request_in_callback()) peer: 12345-0@lo (source: 12345-0@lo) 00000100:00100000:3.0:1776178149.217733:0:6261:0:(service.c:2152:ptlrpc_server_handle_req_in()) got req x1862456998032064 00000100:00100000:3.0:1776178149.217783:0:6261:0:(nrs_fifo.c:179:nrs_fifo_req_get()) NRS start fifo request from 12345-0@lo, seq: 645 00000100:00100000:3.0:1776178149.217788:0:6261:0:(service.c:2324:ptlrpc_server_handle_request()) Handling RPC req@00000000d0706d7c pname:cluuid+ref:pid:xid:nid:opc:job mdt_out00_001:lustre-MDT0001-mdtlov_UUID+6:3356:x1862456998032064:12345-0@lo:41:osp-pre-0-1.0 00000004:00000024:3.0:1776178149.217823:0:6261:0:(mdt_handler.c:548:mdt_statfs()) blocks cached 0 granted 2097152 pending 0 free 1446125568 avail 1316663296 00000100:00100000:3.0:1776178149.217867:0:6261:0:(service.c:2372:ptlrpc_server_handle_request()) Handled RPC req@00000000d0706d7c pname:cluuid+ref:pid:xid:nid:opc:job mdt_out00_001:lustre-MDT0001-mdtlov_UUID+6:3356:x1862456998032064:12345-0@lo:41:osp-pre-0-1.0 Request processed in 78us (177us total) trans 0 rc 0/0 00000100:00100000:3.0:1776178149.217874:0:6261:0:(nrs_fifo.c:241:nrs_fifo_req_stop()) NRS stop fifo request from 12345-0@lo, seq: 645 00000100:00100000:3.0:1776178149.217923:0:3356:0:(client.c:2215:ptlrpc_check_set()) Completed RPC req@00000000bcefda18 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_03:lustre-MDT0001-mdtlov_UUID:3356:1862456998032064:0@lo:41:osp-pre-0-1.0 00000100:00100000:1.0:1776178149.218045:0:8231:0:(service.c:2152:ptlrpc_server_handle_req_in()) got req x1862456998032000 00000100:00100000:1.0:1776178149.218133:0:8231:0:(nrs_fifo.c:179:nrs_fifo_req_get()) NRS start fifo request from 12345-0@lo, seq: 435 00000100:00100000:1.0:1776178149.218137:0:8231:0:(service.c:2324:ptlrpc_server_handle_request()) Handling RPC req@00000000f4d20636 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_create00:lustre-MDT0000-mdtlov_UUID+5:3354:x1862456998031936:12345-0@lo:13:osp-pre-0-0.0 00000100:00100000:3.0:1776178149.218150:0:3353:0:(client.c:1733:ptlrpc_send_new_req()) Sending RPC req@00000000bcefda18 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_00:ac61f839-0e8e-4cac-9c59-08bcfbbad330:3353:1862456998032128:0@lo:400:kworker/u8:2.0 00002000:00000024:1.0:1776178149.218160:0:8231:0:(ofd_obd.c:740:ofd_statfs()) blocks cached 315392 granted 484883776 pending 0 free 3923689472 avail 3692167168 00000100:00100000:3.0:1776178149.218188:0:3353:0:(events.c:361:request_in_callback()) peer: 12345-0@lo (source: 12345-0@lo) 00000100:00100000:0.0:1776178149.218195:0:9554:0:(nrs_fifo.c:179:nrs_fifo_req_get()) NRS start fifo request from 12345-0@lo, seq: 436 00000100:00100000:1.0:1776178149.218200:0:8231:0:(service.c:2372:ptlrpc_server_handle_request()) Handled RPC req@00000000f4d20636 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_create00:lustre-MDT0000-mdtlov_UUID+5:3354:x1862456998031936:12345-0@lo:13:osp-pre-0-0.0 Request processed in 61us (825us total) trans 0 rc 0/0 00000100:00100000:0.0:1776178149.218204:0:9554:0:(service.c:2324:ptlrpc_server_handle_request()) Handling RPC req@0000000090cc55f6 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_create00:lustre-MDT0000-mdtlov_UUID+5:3355:x1862456998032000:12345-0@lo:13:osp-pre-1-0.0 00000100:00100000:3.0:1776178149.218209:0:3353:0:(client.c:1733:ptlrpc_send_new_req()) Sending RPC req@0000000038c09127 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_00:lustre-MDT0001-lwp-OST0001_UUID:3353:1862456998032384:0@lo:400:kworker/u8:2.0 00000100:00100000:1.0:1776178149.218213:0:8231:0:(nrs_fifo.c:241:nrs_fifo_req_stop()) NRS stop fifo request from 12345-0@lo, seq: 435 00002000:00000024:0.0:1776178149.218227:0:9554:0:(ofd_obd.c:740:ofd_statfs()) blocks cached 229376 granted 491746368 pending 0 free 3923689472 avail 3692167168 00000100:00100000:3.0:1776178149.218241:0:3353:0:(events.c:361:request_in_callback()) peer: 12345-0@lo (source: 12345-0@lo) 00000100:00100000:3.0:1776178149.218261:0:3353:0:(client.c:1733:ptlrpc_send_new_req()) Sending RPC req@0000000095c019b3 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_00:lustre-MDT0000-mdtlov_UUID:3353:1862456998032640:0@lo:400:kworker/u8:2.0 00000100:00100000:0.0:1776178149.218268:0:9554:0:(service.c:2372:ptlrpc_server_handle_request()) Handled RPC req@0000000090cc55f6 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_create00:lustre-MDT0000-mdtlov_UUID+5:3355:x1862456998032000:12345-0@lo:13:osp-pre-1-0.0 Request processed in 65us (734us total) trans 0 rc 0/0 00000100:00100000:0.0:1776178149.218276:0:9554:0:(nrs_fifo.c:241:nrs_fifo_req_stop()) NRS stop fifo request from 12345-0@lo, seq: 436 00000100:00100000:3.0:1776178149.218287:0:3353:0:(events.c:361:request_in_callback()) peer: 12345-0@lo (source: 12345-0@lo) 00000100:00100000:3.0:1776178149.218307:0:3353:0:(ptlrpcd.c:415:ptlrpcd_check()) transfer 3 async RPCs [1->0] 00000100:00100000:3.0:1776178149.218313:0:3353:0:(client.c:1733:ptlrpc_send_new_req()) Sending RPC req@000000006c7327ca pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_00:lustre-MDT0000-lwp-OST0000_UUID:3353:1862456998032192:0@lo:400:kworker/u8:2.0 00000100:00100000:0.0:1776178149.218334:0:3354:0:(client.c:2215:ptlrpc_check_set()) Completed RPC req@000000007fcab602 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_01:lustre-MDT0000-mdtlov_UUID:3354:1862456998031936:0@lo:13:osp-pre-0-0.0 00000100:00100000:1.0:1776178149.218418:0:3355:0:(client.c:1733:ptlrpc_send_new_req()) Sending RPC req@00000000f586aea3 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_02:lustre-MDT0001-lwp-OST0000_UUID:3355:1862456998032256:0@lo:400:kworker/u8:2.0 00000100:00100000:1.0:1776178149.218533:0:6257:0:(service.c:2152:ptlrpc_server_handle_req_in()) got req x1862456998032384 00000100:00100000:1.0:1776178149.218562:0:6257:0:(nrs_fifo.c:179:nrs_fifo_req_get()) NRS start fifo request from 12345-0@lo, seq: 821 00000100:00100000:1.0:1776178149.218567:0:6257:0:(service.c:2324:ptlrpc_server_handle_request()) Handling RPC req@00000000002be227 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_002:lustre-MDT0001-lwp-OST0001_UUID+5:3353:x1862456998032384:12345-0@lo:400:kworker/u8:2.0 00000100:00100000:1.0:1776178149.218601:0:6257:0:(service.c:2372:ptlrpc_server_handle_request()) Handled RPC req@00000000002be227 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_002:lustre-MDT0001-lwp-OST0001_UUID+5:3353:x1862456998032384:12345-0@lo:400:kworker/u8:2.0 Request processed in 32us (360us total) trans 0 rc 0/0 00000100:00100000:1.0:1776178149.218607:0:6257:0:(nrs_fifo.c:241:nrs_fifo_req_stop()) NRS stop fifo request from 12345-0@lo, seq: 821 00000100:00100000:1.0:1776178149.218931:0:20140:0:(service.c:2152:ptlrpc_server_handle_req_in()) got req x1862456998032128 00000100:00100000:3.0:1776178149.218945:0:3353:0:(events.c:361:request_in_callback()) peer: 12345-0@lo (source: 12345-0@lo) 00000100:00100000:1.0:1776178149.218958:0:20140:0:(nrs_fifo.c:179:nrs_fifo_req_get()) NRS start fifo request from 12345-0@lo, seq: 134 00000100:00100000:1.0:1776178149.218965:0:20140:0:(service.c:2324:ptlrpc_server_handle_request()) Handling RPC req@00000000ecf3e0a2 pname:cluuid+ref:pid:xid:nid:opc:job ll_mgs_0002:ac61f839-0e8e-4cac-9c59-08bcfbbad330+23:3353:x1862456998032128:12345-0@lo:400:kworker/u8:2.0 00000100:00100000:3.0:1776178149.218979:0:3353:0:(client.c:1733:ptlrpc_send_new_req()) Sending RPC req@000000009b5eeec2 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_00:lustre-MDT0001-mdtlov_UUID:3353:1862456998032448:0@lo:400:kworker/u8:2.0 00000100:00100000:2.0:1776178149.219002:0:14128:0:(service.c:2152:ptlrpc_server_handle_req_in()) got req x1862456998032640 00000100:00100000:1.0:1776178149.219008:0:20140:0:(service.c:2372:ptlrpc_server_handle_request()) Handled RPC req@00000000ecf3e0a2 pname:cluuid+ref:pid:xid:nid:opc:job ll_mgs_0002:ac61f839-0e8e-4cac-9c59-08bcfbbad330+23:3353:x1862456998032128:12345-0@lo:400:kworker/u8:2.0 Request processed in 43us (821us total) trans 0 rc 0/0 00000100:00100000:1.0:1776178149.219016:0:20140:0:(nrs_fifo.c:241:nrs_fifo_req_stop()) NRS stop fifo request from 12345-0@lo, seq: 134 00000100:00100000:3.0:1776178149.219020:0:3353:0:(events.c:361:request_in_callback()) peer: 12345-0@lo (source: 12345-0@lo) 00000100:00100000:1.0:1776178149.219050:0:8250:0:(service.c:2152:ptlrpc_server_handle_req_in()) got req x1862456998032192 00000100:00100000:3.0:1776178149.219054:0:3353:0:(client.c:1733:ptlrpc_send_new_req()) Sending RPC req@000000008c469ec0 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_00:lustre-MDT0000-lwp-MDT0000_UUID:3353:1862456998032704:0@lo:400:kworker/u8:2.0 00000100:00100000:3.0:1776178149.219105:0:3353:0:(events.c:361:request_in_callback()) peer: 12345-0@lo (source: 12345-0@lo) 00000100:00100000:1.0:1776178149.219339:0:8250:0:(service.c:2152:ptlrpc_server_handle_req_in()) got req x1862456998032704 00000100:00100000:3.0:1776178149.219357:0:3353:0:(client.c:2215:ptlrpc_check_set()) Completed RPC req@00000000bcefda18 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_00:ac61f839-0e8e-4cac-9c59-08bcfbbad330:3353:1862456998032128:0@lo:400:kworker/u8:2.0 00000100:00100000:1.0:1776178149.219366:0:8250:0:(nrs_fifo.c:179:nrs_fifo_req_get()) NRS start fifo request from 12345-0@lo, seq: 822 00000100:00100000:1.0:1776178149.219371:0:8250:0:(service.c:2324:ptlrpc_server_handle_request()) Handling RPC req@000000006d540857 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_003:lustre-MDT0000-lwp-OST0000_UUID+5:3353:x1862456998032192:12345-0@lo:400:kworker/u8:2.0 00000100:00100000:3.0:1776178149.219373:0:3353:0:(client.c:2215:ptlrpc_check_set()) Completed RPC req@0000000038c09127 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_00:lustre-MDT0001-lwp-OST0001_UUID:3353:1862456998032384:0@lo:400:kworker/u8:2.0 00000100:00100000:3.0:1776178149.219409:0:6257:0:(nrs_fifo.c:179:nrs_fifo_req_get()) NRS start fifo request from 12345-0@lo, seq: 823 00000100:00100000:3.0:1776178149.219415:0:6257:0:(service.c:2324:ptlrpc_server_handle_request()) Handling RPC req@00000000bd795500 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_002:lustre-MDT0000-lwp-MDT0000_UUID+5:3353:x1862456998032704:12345-0@lo:400:kworker/u8:2.0 00000100:00100000:1.0:1776178149.219419:0:8250:0:(service.c:2372:ptlrpc_server_handle_request()) Handled RPC req@000000006d540857 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_003:lustre-MDT0000-lwp-OST0000_UUID+5:3353:x1862456998032192:12345-0@lo:400:kworker/u8:2.0 Request processed in 46us (937us total) trans 0 rc 0/0 00000100:00100000:1.0:1776178149.219428:0:8250:0:(nrs_fifo.c:241:nrs_fifo_req_stop()) NRS stop fifo request from 12345-0@lo, seq: 822 00000100:00100000:3.0:1776178149.219456:0:6257:0:(service.c:2372:ptlrpc_server_handle_request()) Handled RPC req@00000000bd795500 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_002:lustre-MDT0000-lwp-MDT0000_UUID+5:3353:x1862456998032704:12345-0@lo:400:kworker/u8:2.0 Request processed in 40us (350us total) trans 0 rc 0/0 00000100:00100000:3.0:1776178149.219464:0:6257:0:(nrs_fifo.c:241:nrs_fifo_req_stop()) NRS stop fifo request from 12345-0@lo, seq: 823 00000100:00100000:3.0:1776178149.219503:0:3355:0:(events.c:361:request_in_callback()) peer: 12345-0@lo (source: 12345-0@lo) 00000100:00100000:3.0:1776178149.219520:0:3355:0:(client.c:1733:ptlrpc_send_new_req()) Sending RPC req@0000000008b3a3ad pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_02:lustre-MDT0001-mdtlov_UUID:3355:1862456998032512:0@lo:400:kworker/u8:2.0 00000100:00100000:3.0:1776178149.219656:0:3355:0:(events.c:361:request_in_callback()) peer: 12345-0@lo (source: 12345-0@lo) 00000100:00100000:3.0:1776178149.219695:0:3355:0:(client.c:2215:ptlrpc_check_set()) Completed RPC req@000000008d82b734 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_02:lustre-MDT0000-mdtlov_UUID:3355:1862456998032000:0@lo:13:osp-pre-1-0.0 00000100:00100000:3.0:1776178149.219719:0:3355:0:(client.c:1733:ptlrpc_send_new_req()) Sending RPC req@0000000045560455 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_02:lustre-MDT0000-mdtlov_UUID:3355:1862456998032768:0@lo:41:osp-pre-1-0.0 00000100:00100000:3.0:1776178149.219754:0:3355:0:(events.c:361:request_in_callback()) peer: 12345-0@lo (source: 12345-0@lo) 00000100:00100000:3.0:1776178149.219772:0:3355:0:(ptlrpcd.c:415:ptlrpcd_check()) transfer 2 async RPCs [3->2] 00000100:00100000:3.0:1776178149.219778:0:3355:0:(client.c:1733:ptlrpc_send_new_req()) Sending RPC req@000000006f8bc860 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_02:lustre-MDT0000-lwp-OST0001_UUID:3355:1862456998032320:0@lo:400:kworker/u8:2.0 00000100:00100000:3.0:1776178149.219801:0:3355:0:(events.c:361:request_in_callback()) peer: 12345-0@lo (source: 12345-0@lo) 00000100:00100000:3.0:1776178149.219814:0:3355:0:(client.c:1733:ptlrpc_send_new_req()) Sending RPC req@0000000092028508 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_02:lustre-MDT0000-lwp-MDT0001_UUID:3355:1862456998032576:0@lo:400:kworker/u8:2.0 00000100:00100000:3.0:1776178149.219835:0:3355:0:(events.c:361:request_in_callback()) peer: 12345-0@lo (source: 12345-0@lo) 00000100:00100000:3.0:1776178149.219860:0:20762:0:(service.c:2152:ptlrpc_server_handle_req_in()) got req x1862456998032448 00000100:00100000:3.0:1776178149.219900:0:20762:0:(service.c:2152:ptlrpc_server_handle_req_in()) got req x1862456998032512 00000100:00100000:0.0:1776178149.219914:0:20769:0:(nrs_fifo.c:179:nrs_fifo_req_get()) NRS start fifo request from 12345-0@lo, seq: 456 00000100:00100000:0.0:1776178149.219921:0:20769:0:(service.c:2324:ptlrpc_server_handle_request()) Handling RPC req@0000000037059d8f pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_018:lustre-MDT0001-mdtlov_UUID+5:3353:x1862456998032448:12345-0@lo:400:kworker/u8:2.0 00000100:00100000:3.0:1776178149.219933:0:20762:0:(nrs_fifo.c:179:nrs_fifo_req_get()) NRS start fifo request from 12345-0@lo, seq: 457 00000100:00100000:3.0:1776178149.219938:0:20762:0:(service.c:2324:ptlrpc_server_handle_request()) Handling RPC req@0000000038c09127 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_013:lustre-MDT0001-mdtlov_UUID+5:3355:x1862456998032512:12345-0@lo:400:kworker/u8:2.0 00000100:00100000:0.0:1776178149.219966:0:20769:0:(service.c:2372:ptlrpc_server_handle_request()) Handled RPC req@0000000037059d8f pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_018:lustre-MDT0001-mdtlov_UUID+5:3353:x1862456998032448:12345-0@lo:400:kworker/u8:2.0 Request processed in 45us (948us total) trans 0 rc 0/0 00000100:00100000:0.0:1776178149.219974:0:20769:0:(nrs_fifo.c:241:nrs_fifo_req_stop()) NRS stop fifo request from 12345-0@lo, seq: 456 00000100:00100000:3.0:1776178149.219985:0:20762:0:(service.c:2372:ptlrpc_server_handle_request()) Handled RPC req@0000000038c09127 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_013:lustre-MDT0001-mdtlov_UUID+5:3355:x1862456998032512:12345-0@lo:400:kworker/u8:2.0 Request processed in 44us (327us total) trans 0 rc 0/0 00000100:00100000:3.0:1776178149.219992:0:20762:0:(nrs_fifo.c:241:nrs_fifo_req_stop()) NRS stop fifo request from 12345-0@lo, seq: 457 00000100:00100000:3.0:1776178149.220019:0:6260:0:(service.c:2152:ptlrpc_server_handle_req_in()) got req x1862456998032768 00000100:00100000:3.0:1776178149.220053:0:6260:0:(nrs_fifo.c:179:nrs_fifo_req_get()) NRS start fifo request from 12345-0@lo, seq: 646 00000100:00100000:3.0:1776178149.220091:0:6260:0:(service.c:2324:ptlrpc_server_handle_request()) Handling RPC req@00000000a8c8cb2d pname:cluuid+ref:pid:xid:nid:opc:job mdt_out00_000:lustre-MDT0000-mdtlov_UUID+6:3353:x1862456998032640:12345-0@lo:400:kworker/u8:2.0 00000100:00100000:0.0:1776178149.220118:0:8250:0:(service.c:2152:ptlrpc_server_handle_req_in()) got req x1862456998032256 00000100:00100000:3.0:1776178149.220126:0:6260:0:(service.c:2372:ptlrpc_server_handle_request()) Handled RPC req@00000000a8c8cb2d pname:cluuid+ref:pid:xid:nid:opc:job mdt_out00_000:lustre-MDT0000-mdtlov_UUID+6:3353:x1862456998032640:12345-0@lo:400:kworker/u8:2.0 Request processed in 66us (1839us total) trans 0 rc 0/0 00000100:00100000:3.0:1776178149.220132:0:6260:0:(nrs_fifo.c:241:nrs_fifo_req_stop()) NRS stop fifo request from 12345-0@lo, seq: 646 00000100:00100000:3.0:1776178149.220145:0:6260:0:(nrs_fifo.c:179:nrs_fifo_req_get()) NRS start fifo request from 12345-0@lo, seq: 647 00000100:00100000:3.0:1776178149.220148:0:6260:0:(service.c:2324:ptlrpc_server_handle_request()) Handling RPC req@00000000bcefda18 pname:cluuid+ref:pid:xid:nid:opc:job mdt_out00_000:lustre-MDT0000-mdtlov_UUID+5:3355:x1862456998032768:12345-0@lo:41:osp-pre-1-0.0 00000004:00000024:3.0:1776178149.220170:0:6260:0:(mdt_handler.c:548:mdt_statfs()) blocks cached 0 granted 2097152 pending 0 free 1446039552 avail 1316577280 00000100:00100000:3.0:1776178149.220197:0:6260:0:(service.c:2372:ptlrpc_server_handle_request()) Handled RPC req@00000000bcefda18 pname:cluuid+ref:pid:xid:nid:opc:job mdt_out00_000:lustre-MDT0000-mdtlov_UUID+5:3355:x1862456998032768:12345-0@lo:41:osp-pre-1-0.0 Request processed in 47us (452us total) trans 0 rc 0/0 00000100:00100000:3.0:1776178149.220203:0:6260:0:(nrs_fifo.c:241:nrs_fifo_req_stop()) NRS stop fifo request from 12345-0@lo, seq: 647 00000100:00100000:3.0:1776178149.220223:0:6257:0:(service.c:2152:ptlrpc_server_handle_req_in()) got req x1862456998032320 00000100:00100000:3.0:1776178149.220238:0:6257:0:(service.c:2152:ptlrpc_server_handle_req_in()) got req x1862456998032576 00000100:00100000:3.0:1776178149.220250:0:6257:0:(nrs_fifo.c:179:nrs_fifo_req_get()) NRS start fifo request from 12345-0@lo, seq: 824 00000100:00100000:3.0:1776178149.220253:0:6257:0:(service.c:2324:ptlrpc_server_handle_request()) Handling RPC req@0000000013603266 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_002:lustre-MDT0000-lwp-OST0001_UUID+5:3355:x1862456998032320:12345-0@lo:400:kworker/u8:2.0 00000100:00100000:3.0:1776178149.220291:0:6257:0:(service.c:2372:ptlrpc_server_handle_request()) Handled RPC req@0000000013603266 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_002:lustre-MDT0000-lwp-OST0001_UUID+5:3355:x1862456998032320:12345-0@lo:400:kworker/u8:2.0 Request processed in 37us (491us total) trans 0 rc 0/0 00000100:00100000:3.0:1776178149.220297:0:6257:0:(nrs_fifo.c:241:nrs_fifo_req_stop()) NRS stop fifo request from 12345-0@lo, seq: 824 00000100:00100000:3.0:1776178149.220307:0:6257:0:(nrs_fifo.c:179:nrs_fifo_req_get()) NRS start fifo request from 12345-0@lo, seq: 825 00000100:00100000:3.0:1776178149.220311:0:6257:0:(service.c:2324:ptlrpc_server_handle_request()) Handling RPC req@00000000b5702a7a pname:cluuid+ref:pid:xid:nid:opc:job mdt00_002:lustre-MDT0000-lwp-MDT0001_UUID+5:3355:x1862456998032576:12345-0@lo:400:kworker/u8:2.0 00000100:00100000:3.0:1776178149.220336:0:6257:0:(service.c:2372:ptlrpc_server_handle_request()) Handled RPC req@00000000b5702a7a pname:cluuid+ref:pid:xid:nid:opc:job mdt00_002:lustre-MDT0000-lwp-MDT0001_UUID+5:3355:x1862456998032576:12345-0@lo:400:kworker/u8:2.0 Request processed in 24us (501us total) trans 0 rc 0/0 00000100:00100000:3.0:1776178149.220341:0:6257:0:(nrs_fifo.c:241:nrs_fifo_req_stop()) NRS stop fifo request from 12345-0@lo, seq: 825 00000100:00100000:3.0:1776178149.220367:0:3355:0:(client.c:2215:ptlrpc_check_set()) Completed RPC req@000000006f8bc860 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_02:lustre-MDT0000-lwp-OST0001_UUID:3355:1862456998032320:0@lo:400:kworker/u8:2.0 00000100:00100000:3.0:1776178149.220377:0:3355:0:(client.c:2215:ptlrpc_check_set()) Completed RPC req@0000000092028508 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_02:lustre-MDT0000-lwp-MDT0001_UUID:3355:1862456998032576:0@lo:400:kworker/u8:2.0 00000100:00100000:3.0:1776178149.220389:0:3355:0:(client.c:2215:ptlrpc_check_set()) Completed RPC req@0000000045560455 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_02:lustre-MDT0000-mdtlov_UUID:3355:1862456998032768:0@lo:41:osp-pre-1-0.0 00000100:00100000:3.0:1776178149.220397:0:3355:0:(client.c:2215:ptlrpc_check_set()) Completed RPC req@0000000008b3a3ad pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_02:lustre-MDT0001-mdtlov_UUID:3355:1862456998032512:0@lo:400:kworker/u8:2.0 00000100:00100000:1.0:1776178149.220560:0:3353:0:(client.c:2215:ptlrpc_check_set()) Completed RPC req@000000006c7327ca pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_00:lustre-MDT0000-lwp-OST0000_UUID:3353:1862456998032192:0@lo:400:kworker/u8:2.0 00000100:00100000:1.0:1776178149.220573:0:3353:0:(client.c:2215:ptlrpc_check_set()) Completed RPC req@000000009b5eeec2 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_00:lustre-MDT0001-mdtlov_UUID:3353:1862456998032448:0@lo:400:kworker/u8:2.0 00000100:00100000:1.0:1776178149.220582:0:3353:0:(client.c:2215:ptlrpc_check_set()) Completed RPC req@000000008c469ec0 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_00:lustre-MDT0000-lwp-MDT0000_UUID:3353:1862456998032704:0@lo:400:kworker/u8:2.0 00000100:00100000:1.0:1776178149.220591:0:3353:0:(client.c:2215:ptlrpc_check_set()) Completed RPC req@0000000095c019b3 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_00:lustre-MDT0000-mdtlov_UUID:3353:1862456998032640:0@lo:400:kworker/u8:2.0 00000100:00100000:0.0:1776178149.221458:0:8250:0:(nrs_fifo.c:179:nrs_fifo_req_get()) NRS start fifo request from 12345-0@lo, seq: 826 00000100:00100000:0.0:1776178149.221465:0:8250:0:(service.c:2324:ptlrpc_server_handle_request()) Handling RPC req@0000000052a7c870 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_003:lustre-MDT0001-lwp-OST0000_UUID+5:3355:x1862456998032256:12345-0@lo:400:kworker/u8:2.0 00000100:00100000:0.0:1776178149.221525:0:8250:0:(service.c:2372:ptlrpc_server_handle_request()) Handled RPC req@0000000052a7c870 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_003:lustre-MDT0001-lwp-OST0000_UUID+5:3355:x1862456998032256:12345-0@lo:400:kworker/u8:2.0 Request processed in 60us (2021us total) trans 0 rc 0/0 00000100:00100000:0.0:1776178149.221532:0:8250:0:(nrs_fifo.c:241:nrs_fifo_req_stop()) NRS stop fifo request from 12345-0@lo, seq: 826 00000100:00100000:0.0:1776178149.221585:0:3355:0:(client.c:2215:ptlrpc_check_set()) Completed RPC req@00000000f586aea3 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_02:lustre-MDT0001-lwp-OST0000_UUID:3355:1862456998032256:0@lo:400:kworker/u8:2.0 00010000:00010000:0.0:1776178149.792231:0:9225:0:(ldlm_lockd.c:2792:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:0.0:1776178150.691756:0:3348:0:(events.c:361:request_in_callback()) peer: 12345-192.168.204.3@tcp (source: 12345-192.168.204.3@tcp) 00000100:00100000:0.0:1776178150.691814:0:3348:0:(events.c:361:request_in_callback()) peer: 12345-192.168.204.3@tcp (source: 12345-192.168.204.3@tcp) 00000100:00100000:0.0:1776178150.691917:0:3348:0:(events.c:361:request_in_callback()) peer: 12345-192.168.204.3@tcp (source: 12345-192.168.204.3@tcp) 00000100:00100000:0.0:1776178150.691950:0:3348:0:(events.c:361:request_in_callback()) peer: 12345-192.168.204.3@tcp (source: 12345-192.168.204.3@tcp) 00000100:00100000:3.0:1776178150.692029:0:20764:0:(service.c:2152:ptlrpc_server_handle_req_in()) got req x1862456979088832 00000100:00100000:3.0:1776178150.692131:0:20764:0:(nrs_fifo.c:179:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.204.3@tcp, seq: 458 00000100:00100000:3.0:1776178150.692139:0:20764:0:(service.c:2324:ptlrpc_server_handle_request()) Handling RPC req@000000001ff33da0 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_015:b50963bd-8eba-4432-9a54-8888d00e3fa1+11:2272:x1862456979088832:12345-192.168.204.3@tcp:400:kworker/u8:3.0 00000100:00100000:3.0:1776178150.692236:0:20764:0:(service.c:2372:ptlrpc_server_handle_request()) Handled RPC req@000000001ff33da0 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_015:b50963bd-8eba-4432-9a54-8888d00e3fa1+11:2272:x1862456979088832:12345-192.168.204.3@tcp:400:kworker/u8:3.0 Request processed in 97us (319us total) trans 0 rc 0/0 00000100:00100000:3.0:1776178150.692244:0:20764:0:(nrs_fifo.c:241:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.204.3@tcp, seq: 458 00000100:00100000:1.0:1776178150.692937:0:20140:0:(service.c:2152:ptlrpc_server_handle_req_in()) got req x1862456979088576 00000100:00100000:1.0:1776178150.692983:0:20140:0:(nrs_fifo.c:179:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.204.3@tcp, seq: 135 00000100:00100000:1.0:1776178150.692990:0:20140:0:(service.c:2324:ptlrpc_server_handle_request()) Handling RPC req@000000003960fbcb pname:cluuid+ref:pid:xid:nid:opc:job ll_mgs_0002:5b5c66b1-1a2f-4651-8548-0c75b9a38a94+9:2272:x1862456979088576:12345-192.168.204.3@tcp:400:kworker/u8:3.0 00000100:00100000:1.0:1776178150.693031:0:20140:0:(service.c:2372:ptlrpc_server_handle_request()) Handled RPC req@000000003960fbcb pname:cluuid+ref:pid:xid:nid:opc:job ll_mgs_0002:5b5c66b1-1a2f-4651-8548-0c75b9a38a94+9:2272:x1862456979088576:12345-192.168.204.3@tcp:400:kworker/u8:3.0 Request processed in 41us (1298us total) trans 0 rc 0/0 00000100:00100000:1.0:1776178150.693037:0:20140:0:(nrs_fifo.c:241:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.204.3@tcp, seq: 135 00000100:00100000:0.0:1776178150.703001:0:3348:0:(events.c:361:request_in_callback()) peer: 12345-192.168.204.3@tcp (source: 12345-192.168.204.3@tcp) 00000100:00100000:3.0:1776178150.703186:0:20759:0:(service.c:2152:ptlrpc_server_handle_req_in()) got req x1862456979088768 00000100:00100000:3.0:1776178150.703250:0:20759:0:(nrs_fifo.c:179:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.204.3@tcp, seq: 459 00000100:00100000:3.0:1776178150.703258:0:20759:0:(service.c:2324:ptlrpc_server_handle_request()) Handling RPC req@00000000fb58fc15 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_010:b50963bd-8eba-4432-9a54-8888d00e3fa1+13:2272:x1862456979088768:12345-192.168.204.3@tcp:400:kworker/u8:3.0 00000100:00100000:3.0:1776178150.703347:0:20759:0:(service.c:2372:ptlrpc_server_handle_request()) Handled RPC req@00000000fb58fc15 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_010:b50963bd-8eba-4432-9a54-8888d00e3fa1+13:2272:x1862456979088768:12345-192.168.204.3@tcp:400:kworker/u8:3.0 Request processed in 88us (356us total) trans 0 rc 0/0 00000100:00100000:3.0:1776178150.703355:0:20759:0:(nrs_fifo.c:241:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.204.3@tcp, seq: 459 00000100:00100000:2.0:1776178150.711810:0:11281:0:(service.c:2152:ptlrpc_server_handle_req_in()) got req x1862456979088704 00000100:00100000:2.0:1776178150.711908:0:11281:0:(service.c:2152:ptlrpc_server_handle_req_in()) got req x1862456979088640 00000100:00100000:2.0:1776178150.711937:0:11281:0:(nrs_fifo.c:179:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.204.3@tcp, seq: 827 00000100:00100000:2.0:1776178150.711944:0:11281:0:(service.c:2324:ptlrpc_server_handle_request()) Handling RPC req@000000009e03ef2c pname:cluuid+ref:pid:xid:nid:opc:job mdt00_004:b50963bd-8eba-4432-9a54-8888d00e3fa1+6:2272:x1862456979088704:12345-192.168.204.3@tcp:400:kworker/u8:3.0 00000100:00100000:2.0:1776178150.712082:0:11281:0:(service.c:2372:ptlrpc_server_handle_request()) Handled RPC req@000000009e03ef2c pname:cluuid+ref:pid:xid:nid:opc:job mdt00_004:b50963bd-8eba-4432-9a54-8888d00e3fa1+6:2272:x1862456979088704:12345-192.168.204.3@tcp:400:kworker/u8:3.0 Request processed in 138us (20268us total) trans 0 rc 0/0 00000100:00100000:2.0:1776178150.712090:0:11281:0:(nrs_fifo.c:241:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.204.3@tcp, seq: 827 00000100:00100000:2.0:1776178150.712113:0:11281:0:(nrs_fifo.c:179:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.204.3@tcp, seq: 828 00000100:00100000:2.0:1776178150.712118:0:11281:0:(service.c:2324:ptlrpc_server_handle_request()) Handling RPC req@0000000056aa3aa8 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_004:b50963bd-8eba-4432-9a54-8888d00e3fa1+40:2272:x1862456979088640:12345-192.168.204.3@tcp:400:kworker/u8:3.0 00000100:00100000:2.0:1776178150.712151:0:11281:0:(service.c:2372:ptlrpc_server_handle_request()) Handled RPC req@0000000056aa3aa8 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_004:b50963bd-8eba-4432-9a54-8888d00e3fa1+40:2272:x1862456979088640:12345-192.168.204.3@tcp:400:kworker/u8:3.0 Request processed in 32us (20201us total) trans 0 rc 0/0 00000100:00100000:2.0:1776178150.712157:0:11281:0:(nrs_fifo.c:241:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.204.3@tcp, seq: 828 00000100:00100000:0.0:1776178150.752797:0:3356:0:(client.c:1733:ptlrpc_send_new_req()) Sending RPC req@000000009d5112f6 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_03:lustre-MDT0001-mdtlov_UUID:3356:1862456998032832:0@lo:13:osp-pre-1-1.0 00000100:00100000:0.0:1776178150.752897:0:3356:0:(events.c:361:request_in_callback()) peer: 12345-0@lo (source: 12345-0@lo) 00000100:00100000:2.0:1776178150.753197:0:8232:0:(service.c:2152:ptlrpc_server_handle_req_in()) got req x1862456998032832 00000100:00100000:2.0:1776178150.753264:0:8232:0:(nrs_fifo.c:179:nrs_fifo_req_get()) NRS start fifo request from 12345-0@lo, seq: 437 00000100:00100000:2.0:1776178150.753271:0:8232:0:(service.c:2324:ptlrpc_server_handle_request()) Handling RPC req@0000000004c8f27c pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_create00:lustre-MDT0001-mdtlov_UUID+5:3356:x1862456998032832:12345-0@lo:13:osp-pre-1-1.0 00002000:00000024:2.0:1776178150.753321:0:8232:0:(ofd_obd.c:740:ofd_statfs()) blocks cached 229376 granted 491746368 pending 0 free 3923689472 avail 3692167168 00000100:00100000:2.0:1776178150.753384:0:8232:0:(service.c:2372:ptlrpc_server_handle_request()) Handled RPC req@0000000004c8f27c pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_create00:lustre-MDT0001-mdtlov_UUID+5:3356:x1862456998032832:12345-0@lo:13:osp-pre-1-1.0 Request processed in 111us (488us total) trans 0 rc 0/0 00000100:00100000:2.0:1776178150.753391:0:8232:0:(nrs_fifo.c:241:nrs_fifo_req_stop()) NRS stop fifo request from 12345-0@lo, seq: 437 00000100:00100000:3.0:1776178150.753639:0:3356:0:(client.c:2215:ptlrpc_check_set()) Completed RPC req@000000009d5112f6 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_03:lustre-MDT0001-mdtlov_UUID:3356:1862456998032832:0@lo:13:osp-pre-1-1.0 00010000:00010000:2.0:1776178150.821298:0:9224:0:(ldlm_lockd.c:2792:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:2.0:1776178151.264797:0:3354:0:(ptlrpcd.c:415:ptlrpcd_check()) transfer 1 async RPCs [0->1] 00000100:00100000:2.0:1776178151.266566:0:3354:0:(client.c:1733:ptlrpc_send_new_req()) Sending RPC req@00000000e5c18d8a pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_01:lustre-MDT0001-mdtlov_UUID:3354:1862456998032896:0@lo:13:osp-pre-0-1.0 00000100:00100000:2.0:1776178151.266663:0:3354:0:(events.c:361:request_in_callback()) peer: 12345-0@lo (source: 12345-0@lo) 00000100:00100000:3.0:1776178151.266793:0:9554:0:(service.c:2152:ptlrpc_server_handle_req_in()) got req x1862456998032896 00000100:00100000:3.0:1776178151.266857:0:9554:0:(nrs_fifo.c:179:nrs_fifo_req_get()) NRS start fifo request from 12345-0@lo, seq: 438 00000100:00100000:3.0:1776178151.266865:0:9554:0:(service.c:2324:ptlrpc_server_handle_request()) Handling RPC req@000000007cbab86b pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_create00:lustre-MDT0001-mdtlov_UUID+5:3354:x1862456998032896:12345-0@lo:13:osp-pre-0-1.0 00002000:00000024:3.0:1776178151.266907:0:9554:0:(ofd_obd.c:740:ofd_statfs()) blocks cached 315392 granted 484883776 pending 0 free 3923689472 avail 3692167168 00000100:00100000:3.0:1776178151.266961:0:9554:0:(service.c:2372:ptlrpc_server_handle_request()) Handled RPC req@000000007cbab86b pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_create00:lustre-MDT0001-mdtlov_UUID+5:3354:x1862456998032896:12345-0@lo:13:osp-pre-0-1.0 Request processed in 96us (300us total) trans 0 rc 0/0 00000100:00100000:3.0:1776178151.266970:0:9554:0:(nrs_fifo.c:241:nrs_fifo_req_stop()) NRS stop fifo request from 12345-0@lo, seq: 438 00000100:00100000:2.0:1776178151.267104:0:3354:0:(client.c:2215:ptlrpc_check_set()) Completed RPC req@00000000e5c18d8a pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_01:lustre-MDT0001-mdtlov_UUID:3354:1862456998032896:0@lo:13:osp-pre-0-1.0 00000004:00080000:1.0:1776178151.783430:0:20134:0:(osp_sync.c:1623:osp_sync_local_commit_cb()) lustre-OST0001-osc-MDT0000: 3 committed 00000004:00080000:1.0:1776178151.783463:0:20134:0:(osp_sync.c:1623:osp_sync_local_commit_cb()) lustre-OST0000-osc-MDT0000: 5 committed 00000001:00080000:1.0:1776178151.783486:0:20134:0:(tgt_lastrcvd.c:913:tgt_cb_last_committed()) lustre-MDT0000: transno 12884901943 is committed 00000004:00080000:1.0:1776178151.783500:0:20134:0:(osp_sync.c:1623:osp_sync_local_commit_cb()) lustre-OST0001-osc-MDT0000: 2 committed 00000004:00080000:1.0:1776178151.783507:0:20134:0:(osp_sync.c:1623:osp_sync_local_commit_cb()) lustre-OST0000-osc-MDT0000: 4 committed 00010000:00010000:3.0:1776178151.783821:0:3339:0:(ldlm_lock.c:820:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(COS) ns: mdt-lustre-MDT0000_UUID lock: 0000000073f2c7a9/0xa40083ab9bf530a9 lrc: 3/0,1 mode: COS/COS res: [0x2000013a1:0x2:0x0].0x0 bits 0x13/0x0 rrc: 3 type: IBT gid 0 flags: 0x50210000000000 nid: local remote: 0x0 expref: -99 pid: 6255 timeout: 0 lvb_type: 0 00010000:00010000:3.0:1776178151.783848:0:3339:0:(ldlm_lock.c:879:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: mdt-lustre-MDT0000_UUID lock: 0000000073f2c7a9/0xa40083ab9bf530a9 lrc: 2/0,0 mode: COS/COS res: [0x2000013a1:0x2:0x0].0x0 bits 0x13/0x0 rrc: 3 type: IBT gid 0 flags: 0x50210400000000 nid: local remote: 0x0 expref: -99 pid: 6255 timeout: 0 lvb_type: 0 00010000:00010000:3.0:1776178151.783855:0:3339:0:(ldlm_lockd.c:1879:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: mdt-lustre-MDT0000_UUID lock: 0000000073f2c7a9/0xa40083ab9bf530a9 lrc: 3/0,0 mode: COS/COS res: [0x2000013a1:0x2:0x0].0x0 bits 0x13/0x0 rrc: 3 type: IBT gid 0 flags: 0x40210400000000 nid: local remote: 0x0 expref: -99 pid: 6255 timeout: 0 lvb_type: 0 00010000:00010000:3.0:1776178151.783861:0:3339:0:(ldlm_lockd.c:1893:ldlm_handle_bl_callback()) Lock 0000000073f2c7a9 already unused, calling callback (0000000079e78348) 00010000:00010000:3.0:1776178151.783866:0:3339:0:(ldlm_request.c:364:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: mdt-lustre-MDT0000_UUID lock: 0000000073f2c7a9/0xa40083ab9bf530a9 lrc: 3/0,0 mode: COS/COS res: [0x2000013a1:0x2:0x0].0x0 bits 0x13/0x0 rrc: 3 type: IBT gid 0 flags: 0x40210400000000 nid: local remote: 0x0 expref: -99 pid: 6255 timeout: 0 lvb_type: 0 00010000:00010000:3.0:1776178151.783872:0:3339:0:(ldlm_request.c:1257:ldlm_cli_cancel_local()) ### server-side local cancel ns: mdt-lustre-MDT0000_UUID lock: 0000000073f2c7a9/0xa40083ab9bf530a9 lrc: 4/0,0 mode: COS/COS res: [0x2000013a1:0x2:0x0].0x0 bits 0x13/0x0 rrc: 3 type: IBT gid 0 flags: 0x40218400000000 nid: local remote: 0x0 expref: -99 pid: 6255 timeout: 0 lvb_type: 0 00010000:00010000:3.0:1776178151.783889:0:3339:0:(ldlm_inodebits.c:94:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x2000013a1:0x2:0x0].0x0 (00000000e4751492) 00010000:00010000:3.0:1776178151.783892:0:3339:0:(ldlm_inodebits.c:96:ldlm_reprocess_inodebits_queue()) Hint 13 00010000:00010000:3.0:1776178151.783896:0:3339:0:(ldlm_lockd.c:1903:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: mdt-lustre-MDT0000_UUID lock: 0000000073f2c7a9/0xa40083ab9bf530a9 lrc: 2/0,0 mode: --/COS res: [0x2000013a1:0x2:0x0].0x0 bits 0x13/0x0 rrc: 3 type: IBT gid 0 flags: 0x44a19400000000 nid: local remote: 0x0 expref: -99 pid: 6255 timeout: 0 lvb_type: 0 00010000:00010000:3.0:1776178151.783902:0:3339:0:(ldlm_lock.c:216:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: mdt-lustre-MDT0000_UUID lock: 0000000073f2c7a9/0xa40083ab9bf530a9 lrc: 0/0,0 mode: --/COS res: [0x2000013a1:0x2:0x0].0x0 bits 0x13/0x0 rrc: 3 type: IBT gid 0 flags: 0x44a19400000000 nid: local remote: 0x0 expref: -99 pid: 6255 timeout: 0 lvb_type: 0 00010000:00010000:3.0:1776178151.783927:0:3339:0:(ldlm_lock.c:820:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(COS) ns: mdt-lustre-MDT0000_UUID lock: 00000000e4217a5c/0xa40083ab9bf530b0 lrc: 3/0,1 mode: COS/COS res: [0x2000013a1:0x7:0x0].0x0 bits 0x13/0x0 rrc: 3 type: IBT gid 0 flags: 0x50210000000000 nid: local remote: 0x0 expref: -99 pid: 6255 timeout: 0 lvb_type: 0 00010000:00010000:3.0:1776178151.783933:0:3339:0:(ldlm_lock.c:879:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: mdt-lustre-MDT0000_UUID lock: 00000000e4217a5c/0xa40083ab9bf530b0 lrc: 2/0,0 mode: COS/COS res: [0x2000013a1:0x7:0x0].0x0 bits 0x13/0x0 rrc: 3 type: IBT gid 0 flags: 0x50210400000000 nid: local remote: 0x0 expref: -99 pid: 6255 timeout: 0 lvb_type: 0 00010000:00010000:3.0:1776178151.783939:0:3339:0:(ldlm_lockd.c:1879:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: mdt-lustre-MDT0000_UUID lock: 00000000e4217a5c/0xa40083ab9bf530b0 lrc: 3/0,0 mode: COS/COS res: [0x2000013a1:0x7:0x0].0x0 bits 0x13/0x0 rrc: 3 type: IBT gid 0 flags: 0x40210400000000 nid: local remote: 0x0 expref: -99 pid: 6255 timeout: 0 lvb_type: 0 00010000:00010000:3.0:1776178151.783943:0:3339:0:(ldlm_lockd.c:1893:ldlm_handle_bl_callback()) Lock 00000000e4217a5c already unused, calling callback (0000000079e78348) 00010000:00010000:3.0:1776178151.783945:0:3339:0:(ldlm_request.c:364:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: mdt-lustre-MDT0000_UUID lock: 00000000e4217a5c/0xa40083ab9bf530b0 lrc: 3/0,0 mode: COS/COS res: [0x2000013a1:0x7:0x0].0x0 bits 0x13/0x0 rrc: 3 type: IBT gid 0 flags: 0x40210400000000 nid: local remote: 0x0 expref: -99 pid: 6255 timeout: 0 lvb_type: 0 00010000:00010000:3.0:1776178151.783950:0:3339:0:(ldlm_request.c:1257:ldlm_cli_cancel_local()) ### server-side local cancel ns: mdt-lustre-MDT0000_UUID lock: 00000000e4217a5c/0xa40083ab9bf530b0 lrc: 4/0,0 mode: COS/COS res: [0x2000013a1:0x7:0x0].0x0 bits 0x13/0x0 rrc: 3 type: IBT gid 0 flags: 0x40218400000000 nid: local remote: 0x0 expref: -99 pid: 6255 timeout: 0 lvb_type: 0 00010000:00010000:3.0:1776178151.783957:0:3339:0:(ldlm_inodebits.c:94:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x2000013a1:0x7:0x0].0x0 (00000000aeb25ebf) 00010000:00010000:3.0:1776178151.783959:0:3339:0:(ldlm_inodebits.c:96:ldlm_reprocess_inodebits_queue()) Hint 13 00010000:00010000:3.0:1776178151.783961:0:3339:0:(ldlm_lockd.c:1903:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: mdt-lustre-MDT0000_UUID lock: 00000000e4217a5c/0xa40083ab9bf530b0 lrc: 2/0,0 mode: --/COS res: [0x2000013a1:0x7:0x0].0x0 bits 0x13/0x0 rrc: 3 type: IBT gid 0 flags: 0x44a19400000000 nid: local remote: 0x0 expref: -99 pid: 6255 timeout: 0 lvb_type: 0 00010000:00010000:3.0:1776178151.783966:0:3339:0:(ldlm_lock.c:216:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: mdt-lustre-MDT0000_UUID lock: 00000000e4217a5c/0xa40083ab9bf530b0 lrc: 0/0,0 mode: --/COS res: [0x2000013a1:0x7:0x0].0x0 bits 0x13/0x0 rrc: 3 type: IBT gid 0 flags: 0x44a19400000000 nid: local remote: 0x0 expref: -99 pid: 6255 timeout: 0 lvb_type: 0 00010000:00010000:3.0:1776178151.783975:0:3339:0:(ldlm_lock.c:820:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(COS) ns: mdt-lustre-MDT0000_UUID lock: 000000000396420a/0xa40083ab9bf530b7 lrc: 3/0,1 mode: COS/COS res: [0x2000013a1:0x8:0x0].0x0 bits 0x13/0x0 rrc: 3 type: IBT gid 0 flags: 0x50210000000000 nid: local remote: 0x0 expref: -99 pid: 6255 timeout: 0 lvb_type: 0 00010000:00010000:3.0:1776178151.783981:0:3339:0:(ldlm_lock.c:879:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: mdt-lustre-MDT0000_UUID lock: 000000000396420a/0xa40083ab9bf530b7 lrc: 2/0,0 mode: COS/COS res: [0x2000013a1:0x8:0x0].0x0 bits 0x13/0x0 rrc: 3 type: IBT gid 0 flags: 0x50210400000000 nid: local remote: 0x0 expref: -99 pid: 6255 timeout: 0 lvb_type: 0 00010000:00010000:3.0:1776178151.783990:0:3339:0:(ldlm_lockd.c:1879:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: mdt-lustre-MDT0000_UUID lock: 000000000396420a/0xa40083ab9bf530b7 lrc: 3/0,0 mode: COS/COS res: [0x2000013a1:0x8:0x0].0x0 bits 0x13/0x0 rrc: 3 type: IBT gid 0 flags: 0x40210400000000 nid: local remote: 0x0 expref: -99 pid: 6255 timeout: 0 lvb_type: 0 00010000:00010000:3.0:1776178151.783995:0:3339:0:(ldlm_lockd.c:1893:ldlm_handle_bl_callback()) Lock 000000000396420a already unused, calling callback (0000000079e78348) 00010000:00010000:3.0:1776178151.783997:0:3339:0:(ldlm_request.c:364:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: mdt-lustre-MDT0000_UUID lock: 000000000396420a/0xa40083ab9bf530b7 lrc: 3/0,0 mode: COS/COS res: [0x2000013a1:0x8:0x0].0x0 bits 0x13/0x0 rrc: 3 type: IBT gid 0 flags: 0x40210400000000 nid: local remote: 0x0 expref: -99 pid: 6255 timeout: 0 lvb_type: 0 00010000:00010000:3.0:1776178151.784001:0:3339:0:(ldlm_request.c:1257:ldlm_cli_cancel_local()) ### server-side local cancel ns: mdt-lustre-MDT0000_UUID lock: 000000000396420a/0xa40083ab9bf530b7 lrc: 4/0,0 mode: COS/COS res: [0x2000013a1:0x8:0x0].0x0 bits 0x13/0x0 rrc: 3 type: IBT gid 0 flags: 0x40218400000000 nid: local remote: 0x0 expref: -99 pid: 6255 timeout: 0 lvb_type: 0 00010000:00010000:3.0:1776178151.784008:0:3339:0:(ldlm_inodebits.c:94:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x2000013a1:0x8:0x0].0x0 (000000009a65918f) 00010000:00010000:3.0:1776178151.784009:0:3339:0:(ldlm_inodebits.c:96:ldlm_reprocess_inodebits_queue()) Hint 13 00010000:00010000:3.0:1776178151.784011:0:3339:0:(ldlm_lockd.c:1903:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: mdt-lustre-MDT0000_UUID lock: 000000000396420a/0xa40083ab9bf530b7 lrc: 2/0,0 mode: --/COS res: [0x2000013a1:0x8:0x0].0x0 bits 0x13/0x0 rrc: 3 type: IBT gid 0 flags: 0x44a19400000000 nid: local remote: 0x0 expref: -99 pid: 6255 timeout: 0 lvb_type: 0 00010000:00010000:3.0:1776178151.784016:0:3339:0:(ldlm_lock.c:216:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: mdt-lustre-MDT0000_UUID lock: 000000000396420a/0xa40083ab9bf530b7 lrc: 0/0,0 mode: --/COS res: [0x2000013a1:0x8:0x0].0x0 bits 0x13/0x0 rrc: 3 type: IBT gid 0 flags: 0x44a19400000000 nid: local remote: 0x0 expref: -99 pid: 6255 timeout: 0 lvb_type: 0 00010000:00010000:3.0:1776178151.784026:0:3339:0:(ldlm_lock.c:820:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(COS) ns: mdt-lustre-MDT0000_UUID lock: 00000000575572c2/0xa40083ab9bf530be lrc: 3/0,1 mode: COS/COS res: [0x2000013a1:0x5:0x0].0x0 bits 0x13/0x0 rrc: 3 type: IBT gid 0 flags: 0x50210000000000 nid: local remote: 0x0 expref: -99 pid: 11281 timeout: 0 lvb_type: 0 00010000:00010000:3.0:1776178151.784031:0:3339:0:(ldlm_lock.c:879:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: mdt-lustre-MDT0000_UUID lock: 00000000575572c2/0xa40083ab9bf530be lrc: 2/0,0 mode: COS/COS res: [0x2000013a1:0x5:0x0].0x0 bits 0x13/0x0 rrc: 3 type: IBT gid 0 flags: 0x50210400000000 nid: local remote: 0x0 expref: -99 pid: 11281 timeout: 0 lvb_type: 0 00010000:00010000:3.0:1776178151.784036:0:3339:0:(ldlm_lockd.c:1879:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: mdt-lustre-MDT0000_UUID lock: 00000000575572c2/0xa40083ab9bf530be lrc: 3/0,0 mode: COS/COS res: [0x2000013a1:0x5:0x0].0x0 bits 0x13/0x0 rrc: 3 type: IBT gid 0 flags: 0x40210400000000 nid: local remote: 0x0 expref: -99 pid: 11281 timeout: 0 lvb_type: 0 00010000:00010000:3.0:1776178151.784041:0:3339:0:(ldlm_lockd.c:1893:ldlm_handle_bl_callback()) Lock 00000000575572c2 already unused, calling callback (0000000079e78348) 00010000:00010000:3.0:1776178151.784043:0:3339:0:(ldlm_request.c:364:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: mdt-lustre-MDT0000_UUID lock: 00000000575572c2/0xa40083ab9bf530be lrc: 3/0,0 mode: COS/COS res: [0x2000013a1:0x5:0x0].0x0 bits 0x13/0x0 rrc: 3 type: IBT gid 0 flags: 0x40210400000000 nid: local remote: 0x0 expref: -99 pid: 11281 timeout: 0 lvb_type: 0 00010000:00010000:3.0:1776178151.784048:0:3339:0:(ldlm_request.c:1257:ldlm_cli_cancel_local()) ### server-side local cancel ns: mdt-lustre-MDT0000_UUID lock: 00000000575572c2/0xa40083ab9bf530be lrc: 4/0,0 mode: COS/COS res: [0x2000013a1:0x5:0x0].0x0 bits 0x13/0x0 rrc: 3 type: IBT gid 0 flags: 0x40218400000000 nid: local remote: 0x0 expref: -99 pid: 11281 timeout: 0 lvb_type: 0 00010000:00010000:3.0:1776178151.784055:0:3339:0:(ldlm_inodebits.c:94:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x2000013a1:0x5:0x0].0x0 (000000006b0fcbcb) 00010000:00010000:3.0:1776178151.784056:0:3339:0:(ldlm_inodebits.c:96:ldlm_reprocess_inodebits_queue()) Hint 13 00010000:00010000:3.0:1776178151.784088:0:3339:0:(ldlm_lockd.c:1903:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: mdt-lustre-MDT0000_UUID lock: 00000000575572c2/0xa40083ab9bf530be lrc: 2/0,0 mode: --/COS res: [0x2000013a1:0x5:0x0].0x0 bits 0x13/0x0 rrc: 3 type: IBT gid 0 flags: 0x44a19400000000 nid: local remote: 0x0 expref: -99 pid: 11281 timeout: 0 lvb_type: 0 00010000:00010000:3.0:1776178151.784094:0:3339:0:(ldlm_lock.c:216:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: mdt-lustre-MDT0000_UUID lock: 00000000575572c2/0xa40083ab9bf530be lrc: 0/0,0 mode: --/COS res: [0x2000013a1:0x5:0x0].0x0 bits 0x13/0x0 rrc: 3 type: IBT gid 0 flags: 0x44a19400000000 nid: local remote: 0x0 expref: -99 pid: 11281 timeout: 0 lvb_type: 0 00010000:00010000:3.0:1776178151.784106:0:3339:0:(ldlm_lock.c:820:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(COS) ns: mdt-lustre-MDT0000_UUID lock: 00000000b335466f/0xa40083ab9bf530c5 lrc: 3/0,1 mode: COS/COS res: [0x2000013a1:0x6:0x0].0x0 bits 0x13/0x0 rrc: 3 type: IBT gid 0 flags: 0x50210000000000 nid: local remote: 0x0 expref: -99 pid: 11281 timeout: 0 lvb_type: 0 00010000:00010000:3.0:1776178151.784111:0:3339:0:(ldlm_lock.c:879:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: mdt-lustre-MDT0000_UUID lock: 00000000b335466f/0xa40083ab9bf530c5 lrc: 2/0,0 mode: COS/COS res: [0x2000013a1:0x6:0x0].0x0 bits 0x13/0x0 rrc: 3 type: IBT gid 0 flags: 0x50210400000000 nid: local remote: 0x0 expref: -99 pid: 11281 timeout: 0 lvb_type: 0 00010000:00010000:3.0:1776178151.784116:0:3339:0:(ldlm_lockd.c:1879:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: mdt-lustre-MDT0000_UUID lock: 00000000b335466f/0xa40083ab9bf530c5 lrc: 3/0,0 mode: COS/COS res: [0x2000013a1:0x6:0x0].0x0 bits 0x13/0x0 rrc: 3 type: IBT gid 0 flags: 0x40210400000000 nid: local remote: 0x0 expref: -99 pid: 11281 timeout: 0 lvb_type: 0 00010000:00010000:3.0:1776178151.784120:0:3339:0:(ldlm_lockd.c:1893:ldlm_handle_bl_callback()) Lock 00000000b335466f already unused, calling callback (0000000079e78348) 00010000:00010000:3.0:1776178151.784122:0:3339:0:(ldlm_request.c:364:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: mdt-lustre-MDT0000_UUID lock: 00000000b335466f/0xa40083ab9bf530c5 lrc: 3/0,0 mode: COS/COS res: [0x2000013a1:0x6:0x0].0x0 bits 0x13/0x0 rrc: 3 type: IBT gid 0 flags: 0x40210400000000 nid: local remote: 0x0 expref: -99 pid: 11281 timeout: 0 lvb_type: 0 00010000:00010000:3.0:1776178151.784127:0:3339:0:(ldlm_request.c:1257:ldlm_cli_cancel_local()) ### server-side local cancel ns: mdt-lustre-MDT0000_UUID lock: 00000000b335466f/0xa40083ab9bf530c5 lrc: 4/0,0 mode: COS/COS res: [0x2000013a1:0x6:0x0].0x0 bits 0x13/0x0 rrc: 3 type: IBT gid 0 flags: 0x40218400000000 nid: local remote: 0x0 expref: -99 pid: 11281 timeout: 0 lvb_type: 0 00010000:00010000:3.0:1776178151.784134:0:3339:0:(ldlm_inodebits.c:94:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x2000013a1:0x6:0x0].0x0 (0000000019a0327e) 00010000:00010000:3.0:1776178151.784135:0:3339:0:(ldlm_inodebits.c:96:ldlm_reprocess_inodebits_queue()) Hint 13 00010000:00010000:3.0:1776178151.784138:0:3339:0:(ldlm_lockd.c:1903:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: mdt-lustre-MDT0000_UUID lock: 00000000b335466f/0xa40083ab9bf530c5 lrc: 2/0,0 mode: --/COS res: [0x2000013a1:0x6:0x0].0x0 bits 0x13/0x0 rrc: 3 type: IBT gid 0 flags: 0x44a19400000000 nid: local remote: 0x0 expref: -99 pid: 11281 timeout: 0 lvb_type: 0 00010000:00010000:3.0:1776178151.784143:0:3339:0:(ldlm_lock.c:216:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: mdt-lustre-MDT0000_UUID lock: 00000000b335466f/0xa40083ab9bf530c5 lrc: 0/0,0 mode: --/COS res: [0x2000013a1:0x6:0x0].0x0 bits 0x13/0x0 rrc: 3 type: IBT gid 0 flags: 0x44a19400000000 nid: local remote: 0x0 expref: -99 pid: 11281 timeout: 0 lvb_type: 0 00010000:00010000:3.0:1776178151.784152:0:3339:0:(ldlm_lock.c:820:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(COS) ns: mdt-lustre-MDT0000_UUID lock: 00000000a44d8583/0xa40083ab9bf530cc lrc: 3/0,1 mode: COS/COS res: [0x2000013a1:0x3:0x0].0x0 bits 0x13/0x0 rrc: 3 type: IBT gid 0 flags: 0x50210000000000 nid: local remote: 0x0 expref: -99 pid: 11281 timeout: 0 lvb_type: 0 00010000:00010000:3.0:1776178151.784157:0:3339:0:(ldlm_lock.c:879:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: mdt-lustre-MDT0000_UUID lock: 00000000a44d8583/0xa40083ab9bf530cc lrc: 2/0,0 mode: COS/COS res: [0x2000013a1:0x3:0x0].0x0 bits 0x13/0x0 rrc: 3 type: IBT gid 0 flags: 0x50210400000000 nid: local remote: 0x0 expref: -99 pid: 11281 timeout: 0 lvb_type: 0 00010000:00010000:3.0:1776178151.784162:0:3339:0:(ldlm_lockd.c:1879:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: mdt-lustre-MDT0000_UUID lock: 00000000a44d8583/0xa40083ab9bf530cc lrc: 3/0,0 mode: COS/COS res: [0x2000013a1:0x3:0x0].0x0 bits 0x13/0x0 rrc: 3 type: IBT gid 0 flags: 0x40210400000000 nid: local remote: 0x0 expref: -99 pid: 11281 timeout: 0 lvb_type: 0 00010000:00010000:3.0:1776178151.784166:0:3339:0:(ldlm_lockd.c:1893:ldlm_handle_bl_callback()) Lock 00000000a44d8583 already unused, calling callback (0000000079e78348) 00010000:00010000:3.0:1776178151.784168:0:3339:0:(ldlm_request.c:364:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: mdt-lustre-MDT0000_UUID lock: 00000000a44d8583/0xa40083ab9bf530cc lrc: 3/0,0 mode: COS/COS res: [0x2000013a1:0x3:0x0].0x0 bits 0x13/0x0 rrc: 3 type: IBT gid 0 flags: 0x40210400000000 nid: local remote: 0x0 expref: -99 pid: 11281 timeout: 0 lvb_type: 0 00010000:00010000:3.0:1776178151.784173:0:3339:0:(ldlm_request.c:1257:ldlm_cli_cancel_local()) ### server-side local cancel ns: mdt-lustre-MDT0000_UUID lock: 00000000a44d8583/0xa40083ab9bf530cc lrc: 4/0,0 mode: COS/COS res: [0x2000013a1:0x3:0x0].0x0 bits 0x13/0x0 rrc: 3 type: IBT gid 0 flags: 0x40218400000000 nid: local remote: 0x0 expref: -99 pid: 11281 timeout: 0 lvb_type: 0 00010000:00010000:3.0:1776178151.784179:0:3339:0:(ldlm_inodebits.c:94:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x2000013a1:0x3:0x0].0x0 (0000000090c54b62) 00010000:00010000:3.0:1776178151.784181:0:3339:0:(ldlm_inodebits.c:96:ldlm_reprocess_inodebits_queue()) Hint 13 00010000:00010000:3.0:1776178151.784183:0:3339:0:(ldlm_lockd.c:1903:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: mdt-lustre-MDT0000_UUID lock: 00000000a44d8583/0xa40083ab9bf530cc lrc: 2/0,0 mode: --/COS res: [0x2000013a1:0x3:0x0].0x0 bits 0x13/0x0 rrc: 3 type: IBT gid 0 flags: 0x44a19400000000 nid: local remote: 0x0 expref: -99 pid: 11281 timeout: 0 lvb_type: 0 00010000:00010000:3.0:1776178151.784188:0:3339:0:(ldlm_lock.c:216:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: mdt-lustre-MDT0000_UUID lock: 00000000a44d8583/0xa40083ab9bf530cc lrc: 0/0,0 mode: --/COS res: [0x2000013a1:0x3:0x0].0x0 bits 0x13/0x0 rrc: 3 type: IBT gid 0 flags: 0x44a19400000000 nid: local remote: 0x0 expref: -99 pid: 11281 timeout: 0 lvb_type: 0 00010000:00010000:3.0:1776178151.784197:0:3339:0:(ldlm_lock.c:820:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(COS) ns: mdt-lustre-MDT0000_UUID lock: 0000000097975a3b/0xa40083ab9bf530d3 lrc: 3/0,1 mode: COS/COS res: [0x2000013a1:0x4:0x0].0x0 bits 0x13/0x0 rrc: 3 type: IBT gid 0 flags: 0x50210000000000 nid: local remote: 0x0 expref: -99 pid: 11281 timeout: 0 lvb_type: 0 00010000:00010000:3.0:1776178151.784203:0:3339:0:(ldlm_lock.c:879:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: mdt-lustre-MDT0000_UUID lock: 0000000097975a3b/0xa40083ab9bf530d3 lrc: 2/0,0 mode: COS/COS res: [0x2000013a1:0x4:0x0].0x0 bits 0x13/0x0 rrc: 3 type: IBT gid 0 flags: 0x50210400000000 nid: local remote: 0x0 expref: -99 pid: 11281 timeout: 0 lvb_type: 0 00010000:00010000:3.0:1776178151.784208:0:3339:0:(ldlm_lockd.c:1879:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: mdt-lustre-MDT0000_UUID lock: 0000000097975a3b/0xa40083ab9bf530d3 lrc: 3/0,0 mode: COS/COS res: [0x2000013a1:0x4:0x0].0x0 bits 0x13/0x0 rrc: 3 type: IBT gid 0 flags: 0x40210400000000 nid: local remote: 0x0 expref: -99 pid: 11281 timeout: 0 lvb_type: 0 00010000:00010000:3.0:1776178151.784214:0:3339:0:(ldlm_lockd.c:1893:ldlm_handle_bl_callback()) Lock 0000000097975a3b already unused, calling callback (0000000079e78348) 00010000:00010000:3.0:1776178151.784216:0:3339:0:(ldlm_request.c:364:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: mdt-lustre-MDT0000_UUID lock: 0000000097975a3b/0xa40083ab9bf530d3 lrc: 3/0,0 mode: COS/COS res: [0x2000013a1:0x4:0x0].0x0 bits 0x13/0x0 rrc: 3 type: IBT gid 0 flags: 0x40210400000000 nid: local remote: 0x0 expref: -99 pid: 11281 timeout: 0 lvb_type: 0 00010000:00010000:3.0:1776178151.784221:0:3339:0:(ldlm_request.c:1257:ldlm_cli_cancel_local()) ### server-side local cancel ns: mdt-lustre-MDT0000_UUID lock: 0000000097975a3b/0xa40083ab9bf530d3 lrc: 4/0,0 mode: COS/COS res: [0x2000013a1:0x4:0x0].0x0 bits 0x13/0x0 rrc: 3 type: IBT gid 0 flags: 0x40218400000000 nid: local remote: 0x0 expref: -99 pid: 11281 timeout: 0 lvb_type: 0 00010000:00010000:3.0:1776178151.784228:0:3339:0:(ldlm_inodebits.c:94:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x2000013a1:0x4:0x0].0x0 (000000009cfa2d69) 00010000:00010000:3.0:1776178151.784230:0:3339:0:(ldlm_inodebits.c:96:ldlm_reprocess_inodebits_queue()) Hint 13 00010000:00010000:3.0:1776178151.784232:0:3339:0:(ldlm_lockd.c:1903:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: mdt-lustre-MDT0000_UUID lock: 0000000097975a3b/0xa40083ab9bf530d3 lrc: 2/0,0 mode: --/COS res: [0x2000013a1:0x4:0x0].0x0 bits 0x13/0x0 rrc: 3 type: IBT gid 0 flags: 0x44a19400000000 nid: local remote: 0x0 expref: -99 pid: 11281 timeout: 0 lvb_type: 0 00010000:00010000:3.0:1776178151.784237:0:3339:0:(ldlm_lock.c:216:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: mdt-lustre-MDT0000_UUID lock: 0000000097975a3b/0xa40083ab9bf530d3 lrc: 0/0,0 mode: --/COS res: [0x2000013a1:0x4:0x0].0x0 bits 0x13/0x0 rrc: 3 type: IBT gid 0 flags: 0x44a19400000000 nid: local remote: 0x0 expref: -99 pid: 11281 timeout: 0 lvb_type: 0 00010000:00010000:3.0:1776178151.784246:0:3339:0:(ldlm_lock.c:820:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(COS) ns: mdt-lustre-MDT0000_UUID lock: 00000000bef7c62f/0xa40083ab9bf53207 lrc: 3/0,1 mode: COS/COS res: [0x2000013a1:0x1:0x0].0x5e5674ea bits 0x2/0x0 rrc: 2 type: IBT gid 0 flags: 0x50210000000000 nid: local remote: 0x0 expref: -99 pid: 6257 timeout: 0 lvb_type: 0 00010000:00010000:3.0:1776178151.784252:0:3339:0:(ldlm_lock.c:879:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: mdt-lustre-MDT0000_UUID lock: 00000000bef7c62f/0xa40083ab9bf53207 lrc: 2/0,0 mode: COS/COS res: [0x2000013a1:0x1:0x0].0x5e5674ea bits 0x2/0x0 rrc: 2 type: IBT gid 0 flags: 0x50210400000000 nid: local remote: 0x0 expref: -99 pid: 6257 timeout: 0 lvb_type: 0 00010000:00010000:3.0:1776178151.784257:0:3339:0:(ldlm_lockd.c:1879:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: mdt-lustre-MDT0000_UUID lock: 00000000bef7c62f/0xa40083ab9bf53207 lrc: 3/0,0 mode: COS/COS res: [0x2000013a1:0x1:0x0].0x5e5674ea bits 0x2/0x0 rrc: 2 type: IBT gid 0 flags: 0x40210400000000 nid: local remote: 0x0 expref: -99 pid: 6257 timeout: 0 lvb_type: 0 00010000:00010000:3.0:1776178151.784266:0:3339:0:(ldlm_lockd.c:1893:ldlm_handle_bl_callback()) Lock 00000000bef7c62f already unused, calling callback (0000000079e78348) 00010000:00010000:3.0:1776178151.784268:0:3339:0:(ldlm_request.c:364:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: mdt-lustre-MDT0000_UUID lock: 00000000bef7c62f/0xa40083ab9bf53207 lrc: 3/0,0 mode: COS/COS res: [0x2000013a1:0x1:0x0].0x5e5674ea bits 0x2/0x0 rrc: 2 type: IBT gid 0 flags: 0x40210400000000 nid: local remote: 0x0 expref: -99 pid: 6257 timeout: 0 lvb_type: 0 00010000:00010000:3.0:1776178151.784274:0:3339:0:(ldlm_request.c:1257:ldlm_cli_cancel_local()) ### server-side local cancel ns: mdt-lustre-MDT0000_UUID lock: 00000000bef7c62f/0xa40083ab9bf53207 lrc: 4/0,0 mode: COS/COS res: [0x2000013a1:0x1:0x0].0x5e5674ea bits 0x2/0x0 rrc: 2 type: IBT gid 0 flags: 0x40218400000000 nid: local remote: 0x0 expref: -99 pid: 6257 timeout: 0 lvb_type: 0 00010000:00010000:3.0:1776178151.784281:0:3339:0:(ldlm_inodebits.c:94:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x2000013a1:0x1:0x0].0x5e5674ea (00000000b86d0927) 00010000:00010000:3.0:1776178151.784282:0:3339:0:(ldlm_inodebits.c:96:ldlm_reprocess_inodebits_queue()) Hint 2 00010000:00010000:3.0:1776178151.784284:0:3339:0:(ldlm_lockd.c:1903:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: mdt-lustre-MDT0000_UUID lock: 00000000bef7c62f/0xa40083ab9bf53207 lrc: 2/0,0 mode: --/COS res: [0x2000013a1:0x1:0x0].0x5e5674ea bits 0x2/0x0 rrc: 2 type: IBT gid 0 flags: 0x44a19400000000 nid: local remote: 0x0 expref: -99 pid: 6257 timeout: 0 lvb_type: 0 00010000:00010000:3.0:1776178151.784289:0:3339:0:(ldlm_lock.c:216:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: mdt-lustre-MDT0000_UUID lock: 00000000bef7c62f/0xa40083ab9bf53207 lrc: 0/0,0 mode: --/COS res: [0x2000013a1:0x1:0x0].0x5e5674ea bits 0x2/0x0 rrc: 2 type: IBT gid 0 flags: 0x44a19400000000 nid: local remote: 0x0 expref: -99 pid: 6257 timeout: 0 lvb_type: 0 00010000:00010000:3.0:1776178151.784303:0:3339:0:(ldlm_lock.c:820:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(COS) ns: mdt-lustre-MDT0000_UUID lock: 000000006cb7334a/0xa40083ab9bf5320e lrc: 3/0,1 mode: COS/COS res: [0x2000013a1:0x4:0x0].0x0 bits 0x3/0x0 rrc: 2 type: IBT gid 0 flags: 0x50210000000000 nid: local remote: 0x0 expref: -99 pid: 6257 timeout: 0 lvb_type: 0 00010000:00010000:3.0:1776178151.784308:0:3339:0:(ldlm_lock.c:879:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: mdt-lustre-MDT0000_UUID lock: 000000006cb7334a/0xa40083ab9bf5320e lrc: 2/0,0 mode: COS/COS res: [0x2000013a1:0x4:0x0].0x0 bits 0x3/0x0 rrc: 2 type: IBT gid 0 flags: 0x50210400000000 nid: local remote: 0x0 expref: -99 pid: 6257 timeout: 0 lvb_type: 0 00010000:00010000:3.0:1776178151.784313:0:3339:0:(ldlm_lockd.c:1879:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: mdt-lustre-MDT0000_UUID lock: 000000006cb7334a/0xa40083ab9bf5320e lrc: 3/0,0 mode: COS/COS res: [0x2000013a1:0x4:0x0].0x0 bits 0x3/0x0 rrc: 2 type: IBT gid 0 flags: 0x40210400000000 nid: local remote: 0x0 expref: -99 pid: 6257 timeout: 0 lvb_type: 0 00010000:00010000:3.0:1776178151.784318:0:3339:0:(ldlm_lockd.c:1893:ldlm_handle_bl_callback()) Lock 000000006cb7334a already unused, calling callback (0000000079e78348) 00010000:00010000:3.0:1776178151.784319:0:3339:0:(ldlm_request.c:364:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: mdt-lustre-MDT0000_UUID lock: 000000006cb7334a/0xa40083ab9bf5320e lrc: 3/0,0 mode: COS/COS res: [0x2000013a1:0x4:0x0].0x0 bits 0x3/0x0 rrc: 2 type: IBT gid 0 flags: 0x40210400000000 nid: local remote: 0x0 expref: -99 pid: 6257 timeout: 0 lvb_type: 0 00010000:00010000:3.0:1776178151.784325:0:3339:0:(ldlm_request.c:1257:ldlm_cli_cancel_local()) ### server-side local cancel ns: mdt-lustre-MDT0000_UUID lock: 000000006cb7334a/0xa40083ab9bf5320e lrc: 4/0,0 mode: COS/COS res: [0x2000013a1:0x4:0x0].0x0 bits 0x3/0x0 rrc: 2 type: IBT gid 0 flags: 0x40218400000000 nid: local remote: 0x0 expref: -99 pid: 6257 timeout: 0 lvb_type: 0 00010000:00010000:3.0:1776178151.784332:0:3339:0:(ldlm_inodebits.c:94:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x2000013a1:0x4:0x0].0x0 (000000009cfa2d69) 00010000:00010000:3.0:1776178151.784333:0:3339:0:(ldlm_inodebits.c:96:ldlm_reprocess_inodebits_queue()) Hint 3 00010000:00010000:3.0:1776178151.784335:0:3339:0:(ldlm_lockd.c:1903:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: mdt-lustre-MDT0000_UUID lock: 000000006cb7334a/0xa40083ab9bf5320e lrc: 2/0,0 mode: --/COS res: [0x2000013a1:0x4:0x0].0x0 bits 0x3/0x0 rrc: 2 type: IBT gid 0 flags: 0x44a19400000000 nid: local remote: 0x0 expref: -99 pid: 6257 timeout: 0 lvb_type: 0 00010000:00010000:3.0:1776178151.784340:0:3339:0:(ldlm_lock.c:216:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: mdt-lustre-MDT0000_UUID lock: 000000006cb7334a/0xa40083ab9bf5320e lrc: 0/0,0 mode: --/COS res: [0x2000013a1:0x4:0x0].0x0 bits 0x3/0x0 rrc: 2 type: IBT gid 0 flags: 0x44a19400000000 nid: local remote: 0x0 expref: -99 pid: 6257 timeout: 0 lvb_type: 0 00010000:00010000:3.0:1776178151.784352:0:3339:0:(ldlm_lock.c:820:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(COS) ns: mdt-lustre-MDT0000_UUID lock: 000000006cc9c357/0xa40083ab9bf53223 lrc: 3/0,1 mode: COS/COS res: [0x2000013a1:0x1:0x0].0xcda302ea bits 0x2/0x0 rrc: 2 type: IBT gid 0 flags: 0x50210000000000 nid: local remote: 0x0 expref: -99 pid: 8250 timeout: 0 lvb_type: 0 00010000:00010000:3.0:1776178151.784357:0:3339:0:(ldlm_lock.c:879:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: mdt-lustre-MDT0000_UUID lock: 000000006cc9c357/0xa40083ab9bf53223 lrc: 2/0,0 mode: COS/COS res: [0x2000013a1:0x1:0x0].0xcda302ea bits 0x2/0x0 rrc: 2 type: IBT gid 0 flags: 0x50210400000000 nid: local remote: 0x0 expref: -99 pid: 8250 timeout: 0 lvb_type: 0 00010000:00010000:3.0:1776178151.784363:0:3339:0:(ldlm_lockd.c:1879:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: mdt-lustre-MDT0000_UUID lock: 000000006cc9c357/0xa40083ab9bf53223 lrc: 3/0,0 mode: COS/COS res: [0x2000013a1:0x1:0x0].0xcda302ea bits 0x2/0x0 rrc: 2 type: IBT gid 0 flags: 0x40210400000000 nid: local remote: 0x0 expref: -99 pid: 8250 timeout: 0 lvb_type: 0 00010000:00010000:3.0:1776178151.784368:0:3339:0:(ldlm_lockd.c:1893:ldlm_handle_bl_callback()) Lock 000000006cc9c357 already unused, calling callback (0000000079e78348) 00010000:00010000:3.0:1776178151.784369:0:3339:0:(ldlm_request.c:364:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: mdt-lustre-MDT0000_UUID lock: 000000006cc9c357/0xa40083ab9bf53223 lrc: 3/0,0 mode: COS/COS res: [0x2000013a1:0x1:0x0].0xcda302ea bits 0x2/0x0 rrc: 2 type: IBT gid 0 flags: 0x40210400000000 nid: local remote: 0x0 expref: -99 pid: 8250 timeout: 0 lvb_type: 0 00010000:00010000:3.0:1776178151.784375:0:3339:0:(ldlm_request.c:1257:ldlm_cli_cancel_local()) ### server-side local cancel ns: mdt-lustre-MDT0000_UUID lock: 000000006cc9c357/0xa40083ab9bf53223 lrc: 4/0,0 mode: COS/COS res: [0x2000013a1:0x1:0x0].0xcda302ea bits 0x2/0x0 rrc: 2 type: IBT gid 0 flags: 0x40218400000000 nid: local remote: 0x0 expref: -99 pid: 8250 timeout: 0 lvb_type: 0 00010000:00010000:3.0:1776178151.784382:0:3339:0:(ldlm_inodebits.c:94:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x2000013a1:0x1:0x0].0xcda302ea (0000000043c2d1ef) 00010000:00010000:3.0:1776178151.784384:0:3339:0:(ldlm_inodebits.c:96:ldlm_reprocess_inodebits_queue()) Hint 2 00010000:00010000:3.0:1776178151.784386:0:3339:0:(ldlm_lockd.c:1903:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: mdt-lustre-MDT0000_UUID lock: 000000006cc9c357/0xa40083ab9bf53223 lrc: 2/0,0 mode: --/COS res: [0x2000013a1:0x1:0x0].0xcda302ea bits 0x2/0x0 rrc: 2 type: IBT gid 0 flags: 0x44a19400000000 nid: local remote: 0x0 expref: -99 pid: 8250 timeout: 0 lvb_type: 0 00010000:00010000:3.0:1776178151.784396:0:3339:0:(ldlm_lock.c:216:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: mdt-lustre-MDT0000_UUID lock: 000000006cc9c357/0xa40083ab9bf53223 lrc: 0/0,0 mode: --/COS res: [0x2000013a1:0x1:0x0].0xcda302ea bits 0x2/0x0 rrc: 2 type: IBT gid 0 flags: 0x44a19400000000 nid: local remote: 0x0 expref: -99 pid: 8250 timeout: 0 lvb_type: 0 00010000:00010000:3.0:1776178151.784406:0:3339:0:(ldlm_lock.c:820:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(COS) ns: mdt-lustre-MDT0000_UUID lock: 000000000b5c0143/0xa40083ab9bf5322a lrc: 3/0,1 mode: COS/COS res: [0x2000013a1:0x2:0x0].0x0 bits 0x3/0x0 rrc: 2 type: IBT gid 0 flags: 0x50210000000000 nid: local remote: 0x0 expref: -99 pid: 8250 timeout: 0 lvb_type: 0 00010000:00010000:3.0:1776178151.784411:0:3339:0:(ldlm_lock.c:879:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: mdt-lustre-MDT0000_UUID lock: 000000000b5c0143/0xa40083ab9bf5322a lrc: 2/0,0 mode: COS/COS res: [0x2000013a1:0x2:0x0].0x0 bits 0x3/0x0 rrc: 2 type: IBT gid 0 flags: 0x50210400000000 nid: local remote: 0x0 expref: -99 pid: 8250 timeout: 0 lvb_type: 0 00010000:00010000:3.0:1776178151.784416:0:3339:0:(ldlm_lockd.c:1879:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: mdt-lustre-MDT0000_UUID lock: 000000000b5c0143/0xa40083ab9bf5322a lrc: 3/0,0 mode: COS/COS res: [0x2000013a1:0x2:0x0].0x0 bits 0x3/0x0 rrc: 2 type: IBT gid 0 flags: 0x40210400000000 nid: local remote: 0x0 expref: -99 pid: 8250 timeout: 0 lvb_type: 0 00010000:00010000:3.0:1776178151.784421:0:3339:0:(ldlm_lockd.c:1893:ldlm_handle_bl_callback()) Lock 000000000b5c0143 already unused, calling callback (0000000079e78348) 00010000:00010000:3.0:1776178151.784422:0:3339:0:(ldlm_request.c:364:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: mdt-lustre-MDT0000_UUID lock: 000000000b5c0143/0xa40083ab9bf5322a lrc: 3/0,0 mode: COS/COS res: [0x2000013a1:0x2:0x0].0x0 bits 0x3/0x0 rrc: 2 type: IBT gid 0 flags: 0x40210400000000 nid: local remote: 0x0 expref: -99 pid: 8250 timeout: 0 lvb_type: 0 00010000:00010000:3.0:1776178151.784428:0:3339:0:(ldlm_request.c:1257:ldlm_cli_cancel_local()) ### server-side local cancel ns: mdt-lustre-MDT0000_UUID lock: 000000000b5c0143/0xa40083ab9bf5322a lrc: 4/0,0 mode: COS/COS res: [0x2000013a1:0x2:0x0].0x0 bits 0x3/0x0 rrc: 2 type: IBT gid 0 flags: 0x40218400000000 nid: local remote: 0x0 expref: -99 pid: 8250 timeout: 0 lvb_type: 0 00010000:00010000:3.0:1776178151.784435:0:3339:0:(ldlm_inodebits.c:94:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x2000013a1:0x2:0x0].0x0 (00000000e4751492) 00010000:00010000:3.0:1776178151.784436:0:3339:0:(ldlm_inodebits.c:96:ldlm_reprocess_inodebits_queue()) Hint 3 00010000:00010000:3.0:1776178151.784438:0:3339:0:(ldlm_lockd.c:1903:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: mdt-lustre-MDT0000_UUID lock: 000000000b5c0143/0xa40083ab9bf5322a lrc: 2/0,0 mode: --/COS res: [0x2000013a1:0x2:0x0].0x0 bits 0x3/0x0 rrc: 2 type: IBT gid 0 flags: 0x44a19400000000 nid: local remote: 0x0 expref: -99 pid: 8250 timeout: 0 lvb_type: 0 00010000:00010000:3.0:1776178151.784443:0:3339:0:(ldlm_lock.c:216:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: mdt-lustre-MDT0000_UUID lock: 000000000b5c0143/0xa40083ab9bf5322a lrc: 0/0,0 mode: --/COS res: [0x2000013a1:0x2:0x0].0x0 bits 0x3/0x0 rrc: 2 type: IBT gid 0 flags: 0x44a19400000000 nid: local remote: 0x0 expref: -99 pid: 8250 timeout: 0 lvb_type: 0 00010000:00010000:3.0:1776178151.784455:0:3339:0:(ldlm_lock.c:820:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(COS) ns: mdt-lustre-MDT0000_UUID lock: 0000000025b13886/0xa40083ab9bf5323f lrc: 3/0,1 mode: COS/COS res: [0x2000013a1:0x1:0x0].0xef09e6ea bits 0x2/0x0 rrc: 2 type: IBT gid 0 flags: 0x50210000000000 nid: local remote: 0x0 expref: -99 pid: 8250 timeout: 0 lvb_type: 0 00010000:00010000:3.0:1776178151.784466:0:3339:0:(ldlm_lock.c:879:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: mdt-lustre-MDT0000_UUID lock: 0000000025b13886/0xa40083ab9bf5323f lrc: 2/0,0 mode: COS/COS res: [0x2000013a1:0x1:0x0].0xef09e6ea bits 0x2/0x0 rrc: 2 type: IBT gid 0 flags: 0x50210400000000 nid: local remote: 0x0 expref: -99 pid: 8250 timeout: 0 lvb_type: 0 00010000:00010000:3.0:1776178151.784471:0:3339:0:(ldlm_lockd.c:1879:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: mdt-lustre-MDT0000_UUID lock: 0000000025b13886/0xa40083ab9bf5323f lrc: 3/0,0 mode: COS/COS res: [0x2000013a1:0x1:0x0].0xef09e6ea bits 0x2/0x0 rrc: 2 type: IBT gid 0 flags: 0x40210400000000 nid: local remote: 0x0 expref: -99 pid: 8250 timeout: 0 lvb_type: 0 00010000:00010000:3.0:1776178151.784477:0:3339:0:(ldlm_lockd.c:1893:ldlm_handle_bl_callback()) Lock 0000000025b13886 already unused, calling callback (0000000079e78348) 00010000:00010000:3.0:1776178151.784479:0:3339:0:(ldlm_request.c:364:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: mdt-lustre-MDT0000_UUID lock: 0000000025b13886/0xa40083ab9bf5323f lrc: 3/0,0 mode: COS/COS res: [0x2000013a1:0x1:0x0].0xef09e6ea bits 0x2/0x0 rrc: 2 type: IBT gid 0 flags: 0x40210400000000 nid: local remote: 0x0 expref: -99 pid: 8250 timeout: 0 lvb_type: 0 00000100:00100000:1.0:1776178151.784806:0:3354:0:(client.c:1733:ptlrpc_send_new_req()) Sending RPC req@000000009d5112f6 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_01:lustre-MDT0000-mdtlov_UUID:3354:1862456998032960:0@lo:6:osp-syn-1-0.0 00000100:00100000:1.0:1776178151.784890:0:3354:0:(events.c:361:request_in_callback()) peer: 12345-0@lo (source: 12345-0@lo) 00000100:00100000:1.0:1776178151.784924:0:3354:0:(client.c:1733:ptlrpc_send_new_req()) Sending RPC req@000000005643ab1b pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_01:lustre-MDT0000-mdtlov_UUID:3354:1862456998033216:0@lo:6:osp-syn-0-0.0 00000100:00100000:1.0:1776178151.784946:0:3354:0:(events.c:361:request_in_callback()) peer: 12345-0@lo (source: 12345-0@lo) 00000100:00100000:1.0:1776178151.784968:0:3354:0:(ptlrpcd.c:415:ptlrpcd_check()) transfer 1 async RPCs [0->1] 00000100:00100000:1.0:1776178151.784973:0:3354:0:(client.c:1733:ptlrpc_send_new_req()) Sending RPC req@00000000ecb64602 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_01:lustre-MDT0000-mdtlov_UUID:3354:1862456998033152:0@lo:6:osp-syn-0-0.0 00000100:00100000:1.0:1776178151.784992:0:3354:0:(events.c:361:request_in_callback()) peer: 12345-0@lo (source: 12345-0@lo) 00000100:00100000:1.0:1776178151.785033:0:20759:0:(service.c:2152:ptlrpc_server_handle_req_in()) got req x1862456998032960 00000100:00100000:1.0:1776178151.785088:0:20759:0:(nrs_fifo.c:179:nrs_fifo_req_get()) NRS start fifo request from 12345-0@lo, seq: 5250 00000100:00100000:1.0:1776178151.785095:0:20759:0:(service.c:2324:ptlrpc_server_handle_request()) Handling RPC req@00000000a7e4e4e8 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_010:lustre-MDT0000-mdtlov_UUID+5:3354:x1862456998032960:12345-0@lo:6:osp-syn-1-0.0 00002000:00080000:1.0:1776178151.785127:0:20759:0:(ofd_dev.c:1822:ofd_destroy_hdl()) lustre-OST0001: Destroy object 0x0:66 count 1 00010000:00010000:1.0:1776178151.785151:0:20759:0:(ldlm_lock.c:766:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0001_UUID lock: 0000000056462e0c/0xa40083ab9bf532d9 lrc: 3/0,1 mode: --/PW res: [0x42:0x0:0x0].0x0 rrc: 3 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 20759 timeout: 0 lvb_type: 0 00010000:00010000:1.0:1776178151.785169:0:20759:0:(ldlm_lock.c:675:ldlm_add_bl_work_item()) ### lock incompatible; sending blocking AST. ns: filter-lustre-OST0001_UUID lock: 000000004bd1d7cb/0xa40083ab9bf53032 lrc: 2/0,0 mode: PR/PR res: [0x42:0x0:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x40000000000000 nid: 192.168.204.3@tcp remote: 0x771f68aa1eb279b3 expref: 10 pid: 20762 timeout: 0 lvb_type: 1 00010000:00010000:1.0:1776178151.785178:0:20759:0:(ldlm_extent.c:318:ldlm_check_contention()) contended locks = 0 00010000:00010000:1.0:1776178151.785181:0:20759:0:(ldlm_extent.c:318:ldlm_check_contention()) contended locks = 0 00010000:00010000:1.0:1776178151.785183:0:20759:0:(ldlm_resource.c:1650:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0001_UUID lock: 0000000056462e0c/0xa40083ab9bf532d9 lrc: 4/0,1 mode: --/PW res: [0x42:0x0:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x50010080000000 nid: local remote: 0x0 expref: -99 pid: 20759 timeout: 0 lvb_type: 0 00010000:00010000:1.0:1776178151.785214:0:20759:0:(ldlm_lockd.c:953:ldlm_server_blocking_ast()) ### server preparing blocking AST ns: filter-lustre-OST0001_UUID lock: 000000004bd1d7cb/0xa40083ab9bf53032 lrc: 3/0,0 mode: PR/PR res: [0x42:0x0:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x50000000010020 nid: 192.168.204.3@tcp remote: 0x771f68aa1eb279b3 expref: 10 pid: 20762 timeout: 0 lvb_type: 1 00010000:00010000:1.0:1776178151.785227:0:20759:0:(ldlm_lockd.c:489:ldlm_add_waiting_lock()) ### adding to wait list(timeout: 100, AT: on) ns: filter-lustre-OST0001_UUID lock: 000000004bd1d7cb/0xa40083ab9bf53032 lrc: 4/0,0 mode: PR/PR res: [0x42:0x0:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x70000400010020 nid: 192.168.204.3@tcp remote: 0x771f68aa1eb279b3 expref: 10 pid: 20762 timeout: 959 lvb_type: 1 00000100:00100000:1.0:1776178151.785238:0:20759:0:(client.c:1733:ptlrpc_send_new_req()) Sending RPC req@0000000021d5d4e8 pname:cluuid:pid:xid:nid:opc:job ll_ost00_010:lustre-OST0001_UUID:20759:1862456998033408:192.168.204.3@tcp:104: 00000100:00100000:1.0:1776178151.785287:0:20759:0:(client.c:2500:ptlrpc_set_wait()) set 000000008ec407df going to sleep for 6 seconds 00000100:00100000:1.0:1776178151.785304:0:20762:0:(service.c:2152:ptlrpc_server_handle_req_in()) got req x1862456998033216 00000100:00100000:1.0:1776178151.785329:0:20762:0:(service.c:2152:ptlrpc_server_handle_req_in()) got req x1862456998033152 00000100:00100000:1.0:1776178151.785342:0:20762:0:(nrs_fifo.c:179:nrs_fifo_req_get()) NRS start fifo request from 12345-0@lo, seq: 5251 00000100:00100000:1.0:1776178151.785346:0:20762:0:(service.c:2324:ptlrpc_server_handle_request()) Handling RPC req@000000003334dea7 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_013:lustre-MDT0000-mdtlov_UUID+6:3354:x1862456998033216:12345-0@lo:6:osp-syn-0-0.0 00002000:00080000:1.0:1776178151.785358:0:20762:0:(ofd_dev.c:1822:ofd_destroy_hdl()) lustre-OST0000: Destroy object 0x0:66 count 1 00010000:00010000:1.0:1776178151.785373:0:20762:0:(ldlm_lock.c:766:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0000_UUID lock: 00000000b2e83690/0xa40083ab9bf532e0 lrc: 3/0,1 mode: --/PW res: [0x42:0x0:0x0].0x0 rrc: 3 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 20762 timeout: 0 lvb_type: 0 00010000:00010000:1.0:1776178151.785384:0:20762:0:(ldlm_lock.c:675:ldlm_add_bl_work_item()) ### lock incompatible; sending blocking AST. ns: filter-lustre-OST0000_UUID lock: 000000002abc4399/0xa40083ab9bf53055 lrc: 2/0,0 mode: PR/PR res: [0x42:0x0:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x40000000000000 nid: 192.168.204.3@tcp remote: 0x771f68aa1eb279c1 expref: 12 pid: 20762 timeout: 0 lvb_type: 1 00010000:00010000:1.0:1776178151.785390:0:20762:0:(ldlm_extent.c:318:ldlm_check_contention()) contended locks = 0 00010000:00010000:1.0:1776178151.785392:0:20762:0:(ldlm_extent.c:318:ldlm_check_contention()) contended locks = 0 00010000:00010000:1.0:1776178151.785393:0:20762:0:(ldlm_resource.c:1650:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 00000000b2e83690/0xa40083ab9bf532e0 lrc: 4/0,1 mode: --/PW res: [0x42:0x0:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x50010080000000 nid: local remote: 0x0 expref: -99 pid: 20762 timeout: 0 lvb_type: 0 00010000:00010000:1.0:1776178151.785411:0:20762:0:(ldlm_lockd.c:953:ldlm_server_blocking_ast()) ### server preparing blocking AST ns: filter-lustre-OST0000_UUID lock: 000000002abc4399/0xa40083ab9bf53055 lrc: 3/0,0 mode: PR/PR res: [0x42:0x0:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x50000000010020 nid: 192.168.204.3@tcp remote: 0x771f68aa1eb279c1 expref: 12 pid: 20762 timeout: 0 lvb_type: 1 00010000:00010000:1.0:1776178151.785420:0:20762:0:(ldlm_lockd.c:489:ldlm_add_waiting_lock()) ### adding to wait list(timeout: 100, AT: on) ns: filter-lustre-OST0000_UUID lock: 000000002abc4399/0xa40083ab9bf53055 lrc: 4/0,0 mode: PR/PR res: [0x42:0x0:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x70000400010020 nid: 192.168.204.3@tcp remote: 0x771f68aa1eb279c1 expref: 12 pid: 20762 timeout: 959 lvb_type: 1 00000100:00100000:1.0:1776178151.785430:0:20762:0:(client.c:1733:ptlrpc_send_new_req()) Sending RPC req@0000000078ae567e pname:cluuid:pid:xid:nid:opc:job ll_ost00_013:lustre-OST0000_UUID:20762:1862456998033472:192.168.204.3@tcp:104: 00000100:00100000:1.0:1776178151.785452:0:20762:0:(client.c:2500:ptlrpc_set_wait()) set 00000000856ca387 going to sleep for 6 seconds 00000100:00100000:1.0:1776178151.785468:0:8227:0:(nrs_fifo.c:179:nrs_fifo_req_get()) NRS start fifo request from 12345-0@lo, seq: 5252 00000100:00100000:1.0:1776178151.785472:0:8227:0:(service.c:2324:ptlrpc_server_handle_request()) Handling RPC req@0000000015095c30 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_000:lustre-MDT0000-mdtlov_UUID+6:3354:x1862456998033152:12345-0@lo:6:osp-syn-0-0.0 00002000:00080000:1.0:1776178151.785484:0:8227:0:(ofd_dev.c:1822:ofd_destroy_hdl()) lustre-OST0000: Destroy object 0x0:67 count 1 00010000:00010000:3.0:1776178151.785688:0:3339:0:(ldlm_request.c:1257:ldlm_cli_cancel_local()) ### server-side local cancel ns: mdt-lustre-MDT0000_UUID lock: 0000000025b13886/0xa40083ab9bf5323f lrc: 4/0,0 mode: COS/COS res: [0x2000013a1:0x1:0x0].0xef09e6ea bits 0x2/0x0 rrc: 2 type: IBT gid 0 flags: 0x40218400000000 nid: local remote: 0x0 expref: -99 pid: 8250 timeout: 0 lvb_type: 0 00010000:00010000:3.0:1776178151.785720:0:3339:0:(ldlm_inodebits.c:94:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x2000013a1:0x1:0x0].0xef09e6ea (00000000c31449d9) 00010000:00010000:3.0:1776178151.785723:0:3339:0:(ldlm_inodebits.c:96:ldlm_reprocess_inodebits_queue()) Hint 2 00010000:00010000:3.0:1776178151.785728:0:3339:0:(ldlm_lockd.c:1903:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: mdt-lustre-MDT0000_UUID lock: 0000000025b13886/0xa40083ab9bf5323f lrc: 2/0,0 mode: --/COS res: [0x2000013a1:0x1:0x0].0xef09e6ea bits 0x2/0x0 rrc: 2 type: IBT gid 0 flags: 0x44a19400000000 nid: local remote: 0x0 expref: -99 pid: 8250 timeout: 0 lvb_type: 0 00010000:00010000:3.0:1776178151.785734:0:3339:0:(ldlm_lock.c:216:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: mdt-lustre-MDT0000_UUID lock: 0000000025b13886/0xa40083ab9bf5323f lrc: 0/0,0 mode: --/COS res: [0x2000013a1:0x1:0x0].0xef09e6ea bits 0x2/0x0 rrc: 2 type: IBT gid 0 flags: 0x44a19400000000 nid: local remote: 0x0 expref: -99 pid: 8250 timeout: 0 lvb_type: 0 00010000:00010000:3.0:1776178151.785756:0:3339:0:(ldlm_lock.c:820:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(COS) ns: mdt-lustre-MDT0000_UUID lock: 00000000f779de26/0xa40083ab9bf53246 lrc: 3/0,1 mode: COS/COS res: [0x2000013a1:0x6:0x0].0x0 bits 0x3/0x0 rrc: 2 type: IBT gid 0 flags: 0x50210000000000 nid: local remote: 0x0 expref: -99 pid: 8250 timeout: 0 lvb_type: 0 00010000:00010000:3.0:1776178151.785762:0:3339:0:(ldlm_lock.c:879:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: mdt-lustre-MDT0000_UUID lock: 00000000f779de26/0xa40083ab9bf53246 lrc: 2/0,0 mode: COS/COS res: [0x2000013a1:0x6:0x0].0x0 bits 0x3/0x0 rrc: 2 type: IBT gid 0 flags: 0x50210400000000 nid: local remote: 0x0 expref: -99 pid: 8250 timeout: 0 lvb_type: 0 00010000:00010000:3.0:1776178151.785768:0:3339:0:(ldlm_lockd.c:1879:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: mdt-lustre-MDT0000_UUID lock: 00000000f779de26/0xa40083ab9bf53246 lrc: 3/0,0 mode: COS/COS res: [0x2000013a1:0x6:0x0].0x0 bits 0x3/0x0 rrc: 2 type: IBT gid 0 flags: 0x40210400000000 nid: local remote: 0x0 expref: -99 pid: 8250 timeout: 0 lvb_type: 0 00010000:00010000:3.0:1776178151.785779:0:3339:0:(ldlm_lockd.c:1893:ldlm_handle_bl_callback()) Lock 00000000f779de26 already unused, calling callback (0000000079e78348) 00010000:00010000:3.0:1776178151.785782:0:3339:0:(ldlm_request.c:364:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: mdt-lustre-MDT0000_UUID lock: 00000000f779de26/0xa40083ab9bf53246 lrc: 3/0,0 mode: COS/COS res: [0x2000013a1:0x6:0x0].0x0 bits 0x3/0x0 rrc: 2 type: IBT gid 0 flags: 0x40210400000000 nid: local remote: 0x0 expref: -99 pid: 8250 timeout: 0 lvb_type: 0 00010000:00010000:3.0:1776178151.785788:0:3339:0:(ldlm_request.c:1257:ldlm_cli_cancel_local()) ### server-side local cancel ns: mdt-lustre-MDT0000_UUID lock: 00000000f779de26/0xa40083ab9bf53246 lrc: 4/0,0 mode: COS/COS res: [0x2000013a1:0x6:0x0].0x0 bits 0x3/0x0 rrc: 2 type: IBT gid 0 flags: 0x40218400000000 nid: local remote: 0x0 expref: -99 pid: 8250 timeout: 0 lvb_type: 0 00010000:00010000:3.0:1776178151.785795:0:3339:0:(ldlm_inodebits.c:94:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x2000013a1:0x6:0x0].0x0 (0000000019a0327e) 00010000:00010000:3.0:1776178151.785797:0:3339:0:(ldlm_inodebits.c:96:ldlm_reprocess_inodebits_queue()) Hint 3 00010000:00010000:3.0:1776178151.785798:0:3339:0:(ldlm_lockd.c:1903:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: mdt-lustre-MDT0000_UUID lock: 00000000f779de26/0xa40083ab9bf53246 lrc: 2/0,0 mode: --/COS res: [0x2000013a1:0x6:0x0].0x0 bits 0x3/0x0 rrc: 2 type: IBT gid 0 flags: 0x44a19400000000 nid: local remote: 0x0 expref: -99 pid: 8250 timeout: 0 lvb_type: 0 00010000:00010000:3.0:1776178151.785803:0:3339:0:(ldlm_lock.c:216:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: mdt-lustre-MDT0000_UUID lock: 00000000f779de26/0xa40083ab9bf53246 lrc: 0/0,0 mode: --/COS res: [0x2000013a1:0x6:0x0].0x0 bits 0x3/0x0 rrc: 2 type: IBT gid 0 flags: 0x44a19400000000 nid: local remote: 0x0 expref: -99 pid: 8250 timeout: 0 lvb_type: 0 00010000:00010000:3.0:1776178151.785820:0:3339:0:(ldlm_lock.c:820:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(COS) ns: mdt-lustre-MDT0000_UUID lock: 000000000e7a37e0/0xa40083ab9bf5325b lrc: 3/0,1 mode: COS/COS res: [0x2000013a1:0x1:0x0].0x15fcbbea bits 0x2/0x0 rrc: 2 type: IBT gid 0 flags: 0x50210000000000 nid: local remote: 0x0 expref: -99 pid: 8250 timeout: 0 lvb_type: 0 00010000:00010000:3.0:1776178151.785825:0:3339:0:(ldlm_lock.c:879:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: mdt-lustre-MDT0000_UUID lock: 000000000e7a37e0/0xa40083ab9bf5325b lrc: 2/0,0 mode: COS/COS res: [0x2000013a1:0x1:0x0].0x15fcbbea bits 0x2/0x0 rrc: 2 type: IBT gid 0 flags: 0x50210400000000 nid: local remote: 0x0 expref: -99 pid: 8250 timeout: 0 lvb_type: 0 00010000:00010000:3.0:1776178151.785830:0:3339:0:(ldlm_lockd.c:1879:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: mdt-lustre-MDT0000_UUID lock: 000000000e7a37e0/0xa40083ab9bf5325b lrc: 3/0,0 mode: COS/COS res: [0x2000013a1:0x1:0x0].0x15fcbbea bits 0x2/0x0 rrc: 2 type: IBT gid 0 flags: 0x40210400000000 nid: local remote: 0x0 expref: -99 pid: 8250 timeout: 0 lvb_type: 0 00010000:00010000:3.0:1776178151.785835:0:3339:0:(ldlm_lockd.c:1893:ldlm_handle_bl_callback()) Lock 000000000e7a37e0 already unused, calling callback (0000000079e78348) 00010000:00010000:3.0:1776178151.785837:0:3339:0:(ldlm_request.c:364:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: mdt-lustre-MDT0000_UUID lock: 000000000e7a37e0/0xa40083ab9bf5325b lrc: 3/0,0 mode: COS/COS res: [0x2000013a1:0x1:0x0].0x15fcbbea bits 0x2/0x0 rrc: 2 type: IBT gid 0 flags: 0x40210400000000 nid: local remote: 0x0 expref: -99 pid: 8250 timeout: 0 lvb_type: 0 00010000:00010000:3.0:1776178151.785842:0:3339:0:(ldlm_request.c:1257:ldlm_cli_cancel_local()) ### server-side local cancel ns: mdt-lustre-MDT0000_UUID lock: 000000000e7a37e0/0xa40083ab9bf5325b lrc: 4/0,0 mode: COS/COS res: [0x2000013a1:0x1:0x0].0x15fcbbea bits 0x2/0x0 rrc: 2 type: IBT gid 0 flags: 0x40218400000000 nid: local remote: 0x0 expref: -99 pid: 8250 timeout: 0 lvb_type: 0 00010000:00010000:3.0:1776178151.785850:0:3339:0:(ldlm_inodebits.c:94:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x2000013a1:0x1:0x0].0x15fcbbea (00000000eb684e0b) 00010000:00010000:3.0:1776178151.785852:0:3339:0:(ldlm_inodebits.c:96:ldlm_reprocess_inodebits_queue()) Hint 2 00010000:00010000:3.0:1776178151.785855:0:3339:0:(ldlm_lockd.c:1903:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: mdt-lustre-MDT0000_UUID lock: 000000000e7a37e0/0xa40083ab9bf5325b lrc: 2/0,0 mode: --/COS res: [0x2000013a1:0x1:0x0].0x15fcbbea bits 0x2/0x0 rrc: 2 type: IBT gid 0 flags: 0x44a19400000000 nid: local remote: 0x0 expref: -99 pid: 8250 timeout: 0 lvb_type: 0 00010000:00010000:3.0:1776178151.785860:0:3339:0:(ldlm_lock.c:216:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: mdt-lustre-MDT0000_UUID lock: 000000000e7a37e0/0xa40083ab9bf5325b lrc: 0/0,0 mode: --/COS res: [0x2000013a1:0x1:0x0].0x15fcbbea bits 0x2/0x0 rrc: 2 type: IBT gid 0 flags: 0x44a19400000000 nid: local remote: 0x0 expref: -99 pid: 8250 timeout: 0 lvb_type: 0 00010000:00010000:3.0:1776178151.785871:0:3339:0:(ldlm_lock.c:820:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(COS) ns: mdt-lustre-MDT0000_UUID lock: 00000000782e253e/0xa40083ab9bf53262 lrc: 3/0,1 mode: COS/COS res: [0x2000013a1:0x3:0x0].0x0 bits 0x3/0x0 rrc: 2 type: IBT gid 0 flags: 0x50210000000000 nid: local remote: 0x0 expref: -99 pid: 8250 timeout: 0 lvb_type: 0 00010000:00010000:3.0:1776178151.785877:0:3339:0:(ldlm_lock.c:879:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: mdt-lustre-MDT0000_UUID lock: 00000000782e253e/0xa40083ab9bf53262 lrc: 2/0,0 mode: COS/COS res: [0x2000013a1:0x3:0x0].0x0 bits 0x3/0x0 rrc: 2 type: IBT gid 0 flags: 0x50210400000000 nid: local remote: 0x0 expref: -99 pid: 8250 timeout: 0 lvb_type: 0 00010000:00010000:3.0:1776178151.785882:0:3339:0:(ldlm_lockd.c:1879:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: mdt-lustre-MDT0000_UUID lock: 00000000782e253e/0xa40083ab9bf53262 lrc: 3/0,0 mode: COS/COS res: [0x2000013a1:0x3:0x0].0x0 bits 0x3/0x0 rrc: 2 type: IBT gid 0 flags: 0x40210400000000 nid: local remote: 0x0 expref: -99 pid: 8250 timeout: 0 lvb_type: 0 00010000:00010000:3.0:1776178151.785886:0:3339:0:(ldlm_lockd.c:1893:ldlm_handle_bl_callback()) Lock 00000000782e253e already unused, calling callback (0000000079e78348) 00010000:00010000:3.0:1776178151.785888:0:3339:0:(ldlm_request.c:364:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: mdt-lustre-MDT0000_UUID lock: 00000000782e253e/0xa40083ab9bf53262 lrc: 3/0,0 mode: COS/COS res: [0x2000013a1:0x3:0x0].0x0 bits 0x3/0x0 rrc: 2 type: IBT gid 0 flags: 0x40210400000000 nid: local remote: 0x0 expref: -99 pid: 8250 timeout: 0 lvb_type: 0 00010000:00010000:3.0:1776178151.785893:0:3339:0:(ldlm_request.c:1257:ldlm_cli_cancel_local()) ### server-side local cancel ns: mdt-lustre-MDT0000_UUID lock: 00000000782e253e/0xa40083ab9bf53262 lrc: 4/0,0 mode: COS/COS res: [0x2000013a1:0x3:0x0].0x0 bits 0x3/0x0 rrc: 2 type: IBT gid 0 flags: 0x40218400000000 nid: local remote: 0x0 expref: -99 pid: 8250 timeout: 0 lvb_type: 0 00010000:00010000:3.0:1776178151.785900:0:3339:0:(ldlm_inodebits.c:94:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x2000013a1:0x3:0x0].0x0 (0000000090c54b62) 00010000:00010000:3.0:1776178151.785902:0:3339:0:(ldlm_inodebits.c:96:ldlm_reprocess_inodebits_queue()) Hint 3 00010000:00010000:3.0:1776178151.785904:0:3339:0:(ldlm_lockd.c:1903:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: mdt-lustre-MDT0000_UUID lock: 00000000782e253e/0xa40083ab9bf53262 lrc: 2/0,0 mode: --/COS res: [0x2000013a1:0x3:0x0].0x0 bits 0x3/0x0 rrc: 2 type: IBT gid 0 flags: 0x44a19400000000 nid: local remote: 0x0 expref: -99 pid: 8250 timeout: 0 lvb_type: 0 00010000:00010000:3.0:1776178151.785915:0:3339:0:(ldlm_lock.c:216:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: mdt-lustre-MDT0000_UUID lock: 00000000782e253e/0xa40083ab9bf53262 lrc: 0/0,0 mode: --/COS res: [0x2000013a1:0x3:0x0].0x0 bits 0x3/0x0 rrc: 2 type: IBT gid 0 flags: 0x44a19400000000 nid: local remote: 0x0 expref: -99 pid: 8250 timeout: 0 lvb_type: 0 00010000:00010000:3.0:1776178151.785929:0:3339:0:(ldlm_lock.c:820:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(COS) ns: mdt-lustre-MDT0000_UUID lock: 00000000712476de/0xa40083ab9bf53277 lrc: 3/0,1 mode: COS/COS res: [0x2000013a1:0x1:0x0].0x37639fea bits 0x2/0x0 rrc: 2 type: IBT gid 0 flags: 0x50210000000000 nid: local remote: 0x0 expref: -99 pid: 6257 timeout: 0 lvb_type: 0 00010000:00010000:3.0:1776178151.785935:0:3339:0:(ldlm_lock.c:879:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: mdt-lustre-MDT0000_UUID lock: 00000000712476de/0xa40083ab9bf53277 lrc: 2/0,0 mode: COS/COS res: [0x2000013a1:0x1:0x0].0x37639fea bits 0x2/0x0 rrc: 2 type: IBT gid 0 flags: 0x50210400000000 nid: local remote: 0x0 expref: -99 pid: 6257 timeout: 0 lvb_type: 0 00010000:00010000:3.0:1776178151.785941:0:3339:0:(ldlm_lockd.c:1879:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: mdt-lustre-MDT0000_UUID lock: 00000000712476de/0xa40083ab9bf53277 lrc: 3/0,0 mode: COS/COS res: [0x2000013a1:0x1:0x0].0x37639fea bits 0x2/0x0 rrc: 2 type: IBT gid 0 flags: 0x40210400000000 nid: local remote: 0x0 expref: -99 pid: 6257 timeout: 0 lvb_type: 0 00010000:00010000:3.0:1776178151.785946:0:3339:0:(ldlm_lockd.c:1893:ldlm_handle_bl_callback()) Lock 00000000712476de already unused, calling callback (0000000079e78348) 00010000:00010000:3.0:1776178151.785948:0:3339:0:(ldlm_request.c:364:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: mdt-lustre-MDT0000_UUID lock: 00000000712476de/0xa40083ab9bf53277 lrc: 3/0,0 mode: COS/COS res: [0x2000013a1:0x1:0x0].0x37639fea bits 0x2/0x0 rrc: 2 type: IBT gid 0 flags: 0x40210400000000 nid: local remote: 0x0 expref: -99 pid: 6257 timeout: 0 lvb_type: 0 00010000:00010000:3.0:1776178151.785953:0:3339:0:(ldlm_request.c:1257:ldlm_cli_cancel_local()) ### server-side local cancel ns: mdt-lustre-MDT0000_UUID lock: 00000000712476de/0xa40083ab9bf53277 lrc: 4/0,0 mode: COS/COS res: [0x2000013a1:0x1:0x0].0x37639fea bits 0x2/0x0 rrc: 2 type: IBT gid 0 flags: 0x40218400000000 nid: local remote: 0x0 expref: -99 pid: 6257 timeout: 0 lvb_type: 0 00010000:00010000:3.0:1776178151.785961:0:3339:0:(ldlm_inodebits.c:94:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x2000013a1:0x1:0x0].0x37639fea (000000002853dc93) 00010000:00010000:3.0:1776178151.785963:0:3339:0:(ldlm_inodebits.c:96:ldlm_reprocess_inodebits_queue()) Hint 2 00010000:00010000:3.0:1776178151.785965:0:3339:0:(ldlm_lockd.c:1903:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: mdt-lustre-MDT0000_UUID lock: 00000000712476de/0xa40083ab9bf53277 lrc: 2/0,0 mode: --/COS res: [0x2000013a1:0x1:0x0].0x37639fea bits 0x2/0x0 rrc: 2 type: IBT gid 0 flags: 0x44a19400000000 nid: local remote: 0x0 expref: -99 pid: 6257 timeout: 0 lvb_type: 0 00010000:00010000:3.0:1776178151.785971:0:3339:0:(ldlm_lock.c:216:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: mdt-lustre-MDT0000_UUID lock: 00000000712476de/0xa40083ab9bf53277 lrc: 0/0,0 mode: --/COS res: [0x2000013a1:0x1:0x0].0x37639fea bits 0x2/0x0 rrc: 2 type: IBT gid 0 flags: 0x44a19400000000 nid: local remote: 0x0 expref: -99 pid: 6257 timeout: 0 lvb_type: 0 00010000:00010000:3.0:1776178151.785983:0:3339:0:(ldlm_lock.c:820:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(COS) ns: mdt-lustre-MDT0000_UUID lock: 000000008cbcbaf5/0xa40083ab9bf5327e lrc: 3/0,1 mode: COS/COS res: [0x2000013a1:0x7:0x0].0x0 bits 0x3/0x0 rrc: 2 type: IBT gid 0 flags: 0x50210000000000 nid: local remote: 0x0 expref: -99 pid: 6257 timeout: 0 lvb_type: 0 00010000:00010000:3.0:1776178151.785993:0:3339:0:(ldlm_lock.c:879:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: mdt-lustre-MDT0000_UUID lock: 000000008cbcbaf5/0xa40083ab9bf5327e lrc: 2/0,0 mode: COS/COS res: [0x2000013a1:0x7:0x0].0x0 bits 0x3/0x0 rrc: 2 type: IBT gid 0 flags: 0x50210400000000 nid: local remote: 0x0 expref: -99 pid: 6257 timeout: 0 lvb_type: 0 00010000:00010000:3.0:1776178151.785999:0:3339:0:(ldlm_lockd.c:1879:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: mdt-lustre-MDT0000_UUID lock: 000000008cbcbaf5/0xa40083ab9bf5327e lrc: 3/0,0 mode: COS/COS res: [0x2000013a1:0x7:0x0].0x0 bits 0x3/0x0 rrc: 2 type: IBT gid 0 flags: 0x40210400000000 nid: local remote: 0x0 expref: -99 pid: 6257 timeout: 0 lvb_type: 0 00010000:00010000:3.0:1776178151.786005:0:3339:0:(ldlm_lockd.c:1893:ldlm_handle_bl_callback()) Lock 000000008cbcbaf5 already unused, calling callback (0000000079e78348) 00010000:00010000:3.0:1776178151.786007:0:3339:0:(ldlm_request.c:364:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: mdt-lustre-MDT0000_UUID lock: 000000008cbcbaf5/0xa40083ab9bf5327e lrc: 3/0,0 mode: COS/COS res: [0x2000013a1:0x7:0x0].0x0 bits 0x3/0x0 rrc: 2 type: IBT gid 0 flags: 0x40210400000000 nid: local remote: 0x0 expref: -99 pid: 6257 timeout: 0 lvb_type: 0 00010000:00010000:3.0:1776178151.786012:0:3339:0:(ldlm_request.c:1257:ldlm_cli_cancel_local()) ### server-side local cancel ns: mdt-lustre-MDT0000_UUID lock: 000000008cbcbaf5/0xa40083ab9bf5327e lrc: 4/0,0 mode: COS/COS res: [0x2000013a1:0x7:0x0].0x0 bits 0x3/0x0 rrc: 2 type: IBT gid 0 flags: 0x40218400000000 nid: local remote: 0x0 expref: -99 pid: 6257 timeout: 0 lvb_type: 0 00010000:00010000:3.0:1776178151.786018:0:3339:0:(ldlm_inodebits.c:94:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x2000013a1:0x7:0x0].0x0 (00000000aeb25ebf) 00010000:00010000:3.0:1776178151.786020:0:3339:0:(ldlm_inodebits.c:96:ldlm_reprocess_inodebits_queue()) Hint 3 00010000:00010000:3.0:1776178151.786022:0:3339:0:(ldlm_lockd.c:1903:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: mdt-lustre-MDT0000_UUID lock: 000000008cbcbaf5/0xa40083ab9bf5327e lrc: 2/0,0 mode: --/COS res: [0x2000013a1:0x7:0x0].0x0 bits 0x3/0x0 rrc: 2 type: IBT gid 0 flags: 0x44a19400000000 nid: local remote: 0x0 expref: -99 pid: 6257 timeout: 0 lvb_type: 0 00010000:00010000:3.0:1776178151.786027:0:3339:0:(ldlm_lock.c:216:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: mdt-lustre-MDT0000_UUID lock: 000000008cbcbaf5/0xa40083ab9bf5327e lrc: 0/0,0 mode: --/COS res: [0x2000013a1:0x7:0x0].0x0 bits 0x3/0x0 rrc: 2 type: IBT gid 0 flags: 0x44a19400000000 nid: local remote: 0x0 expref: -99 pid: 6257 timeout: 0 lvb_type: 0 00010000:00010000:3.0:1776178151.786040:0:3339:0:(ldlm_lock.c:820:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(COS) ns: mdt-lustre-MDT0000_UUID lock: 000000006a188f9a/0xa40083ab9bf53293 lrc: 3/0,1 mode: COS/COS res: [0x2000013a1:0x1:0x0].0x7fbd58ea bits 0x2/0x0 rrc: 2 type: IBT gid 0 flags: 0x50210000000000 nid: local remote: 0x0 expref: -99 pid: 6257 timeout: 0 lvb_type: 0 00010000:00010000:3.0:1776178151.786045:0:3339:0:(ldlm_lock.c:879:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: mdt-lustre-MDT0000_UUID lock: 000000006a188f9a/0xa40083ab9bf53293 lrc: 2/0,0 mode: COS/COS res: [0x2000013a1:0x1:0x0].0x7fbd58ea bits 0x2/0x0 rrc: 2 type: IBT gid 0 flags: 0x50210400000000 nid: local remote: 0x0 expref: -99 pid: 6257 timeout: 0 lvb_type: 0 00010000:00010000:3.0:1776178151.786049:0:3339:0:(ldlm_lockd.c:1879:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: mdt-lustre-MDT0000_UUID lock: 000000006a188f9a/0xa40083ab9bf53293 lrc: 3/0,0 mode: COS/COS res: [0x2000013a1:0x1:0x0].0x7fbd58ea bits 0x2/0x0 rrc: 2 type: IBT gid 0 flags: 0x40210400000000 nid: local remote: 0x0 expref: -99 pid: 6257 timeout: 0 lvb_type: 0 00010000:00010000:3.0:1776178151.786078:0:3339:0:(ldlm_lockd.c:1893:ldlm_handle_bl_callback()) Lock 000000006a188f9a already unused, calling callback (0000000079e78348) 00010000:00010000:3.0:1776178151.786080:0:3339:0:(ldlm_request.c:364:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: mdt-lustre-MDT0000_UUID lock: 000000006a188f9a/0xa40083ab9bf53293 lrc: 3/0,0 mode: COS/COS res: [0x2000013a1:0x1:0x0].0x7fbd58ea bits 0x2/0x0 rrc: 2 type: IBT gid 0 flags: 0x40210400000000 nid: local remote: 0x0 expref: -99 pid: 6257 timeout: 0 lvb_type: 0 00010000:00010000:3.0:1776178151.786086:0:3339:0:(ldlm_request.c:1257:ldlm_cli_cancel_local()) ### server-side local cancel ns: mdt-lustre-MDT0000_UUID lock: 000000006a188f9a/0xa40083ab9bf53293 lrc: 4/0,0 mode: COS/COS res: [0x2000013a1:0x1:0x0].0x7fbd58ea bits 0x2/0x0 rrc: 2 type: IBT gid 0 flags: 0x40218400000000 nid: local remote: 0x0 expref: -99 pid: 6257 timeout: 0 lvb_type: 0 00010000:00010000:3.0:1776178151.786094:0:3339:0:(ldlm_inodebits.c:94:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x2000013a1:0x1:0x0].0x7fbd58ea (00000000e9c8e2b7) 00010000:00010000:3.0:1776178151.786096:0:3339:0:(ldlm_inodebits.c:96:ldlm_reprocess_inodebits_queue()) Hint 2 00010000:00010000:3.0:1776178151.786098:0:3339:0:(ldlm_lockd.c:1903:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: mdt-lustre-MDT0000_UUID lock: 000000006a188f9a/0xa40083ab9bf53293 lrc: 2/0,0 mode: --/COS res: [0x2000013a1:0x1:0x0].0x7fbd58ea bits 0x2/0x0 rrc: 2 type: IBT gid 0 flags: 0x44a19400000000 nid: local remote: 0x0 expref: -99 pid: 6257 timeout: 0 lvb_type: 0 00010000:00010000:3.0:1776178151.786102:0:3339:0:(ldlm_lock.c:216:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: mdt-lustre-MDT0000_UUID lock: 000000006a188f9a/0xa40083ab9bf53293 lrc: 0/0,0 mode: --/COS res: [0x2000013a1:0x1:0x0].0x7fbd58ea bits 0x2/0x0 rrc: 2 type: IBT gid 0 flags: 0x44a19400000000 nid: local remote: 0x0 expref: -99 pid: 6257 timeout: 0 lvb_type: 0 00010000:00010000:3.0:1776178151.786114:0:3339:0:(ldlm_lock.c:820:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(COS) ns: mdt-lustre-MDT0000_UUID lock: 00000000dcea299c/0xa40083ab9bf5329a lrc: 3/0,1 mode: COS/COS res: [0x2000013a1:0x8:0x0].0x0 bits 0x3/0x0 rrc: 2 type: IBT gid 0 flags: 0x50210000000000 nid: local remote: 0x0 expref: -99 pid: 6257 timeout: 0 lvb_type: 0 00010000:00010000:3.0:1776178151.786120:0:3339:0:(ldlm_lock.c:879:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: mdt-lustre-MDT0000_UUID lock: 00000000dcea299c/0xa40083ab9bf5329a lrc: 2/0,0 mode: COS/COS res: [0x2000013a1:0x8:0x0].0x0 bits 0x3/0x0 rrc: 2 type: IBT gid 0 flags: 0x50210400000000 nid: local remote: 0x0 expref: -99 pid: 6257 timeout: 0 lvb_type: 0 00010000:00010000:3.0:1776178151.786126:0:3339:0:(ldlm_lockd.c:1879:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: mdt-lustre-MDT0000_UUID lock: 00000000dcea299c/0xa40083ab9bf5329a lrc: 3/0,0 mode: COS/COS res: [0x2000013a1:0x8:0x0].0x0 bits 0x3/0x0 rrc: 2 type: IBT gid 0 flags: 0x40210400000000 nid: local remote: 0x0 expref: -99 pid: 6257 timeout: 0 lvb_type: 0 00010000:00010000:3.0:1776178151.786132:0:3339:0:(ldlm_lockd.c:1893:ldlm_handle_bl_callback()) Lock 00000000dcea299c already unused, calling callback (0000000079e78348) 00010000:00010000:3.0:1776178151.786134:0:3339:0:(ldlm_request.c:364:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: mdt-lustre-MDT0000_UUID lock: 00000000dcea299c/0xa40083ab9bf5329a lrc: 3/0,0 mode: COS/COS res: [0x2000013a1:0x8:0x0].0x0 bits 0x3/0x0 rrc: 2 type: IBT gid 0 flags: 0x40210400000000 nid: local remote: 0x0 expref: -99 pid: 6257 timeout: 0 lvb_type: 0 00010000:00010000:3.0:1776178151.786139:0:3339:0:(ldlm_request.c:1257:ldlm_cli_cancel_local()) ### server-side local cancel ns: mdt-lustre-MDT0000_UUID lock: 00000000dcea299c/0xa40083ab9bf5329a lrc: 4/0,0 mode: COS/COS res: [0x2000013a1:0x8:0x0].0x0 bits 0x3/0x0 rrc: 2 type: IBT gid 0 flags: 0x40218400000000 nid: local remote: 0x0 expref: -99 pid: 6257 timeout: 0 lvb_type: 0 00010000:00010000:3.0:1776178151.786148:0:3339:0:(ldlm_inodebits.c:94:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x2000013a1:0x8:0x0].0x0 (000000009a65918f) 00010000:00010000:3.0:1776178151.786150:0:3339:0:(ldlm_inodebits.c:96:ldlm_reprocess_inodebits_queue()) Hint 3 00010000:00010000:3.0:1776178151.786152:0:3339:0:(ldlm_lockd.c:1903:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: mdt-lustre-MDT0000_UUID lock: 00000000dcea299c/0xa40083ab9bf5329a lrc: 2/0,0 mode: --/COS res: [0x2000013a1:0x8:0x0].0x0 bits 0x3/0x0 rrc: 2 type: IBT gid 0 flags: 0x44a19400000000 nid: local remote: 0x0 expref: -99 pid: 6257 timeout: 0 lvb_type: 0 00010000:00010000:3.0:1776178151.786158:0:3339:0:(ldlm_lock.c:216:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: mdt-lustre-MDT0000_UUID lock: 00000000dcea299c/0xa40083ab9bf5329a lrc: 0/0,0 mode: --/COS res: [0x2000013a1:0x8:0x0].0x0 bits 0x3/0x0 rrc: 2 type: IBT gid 0 flags: 0x44a19400000000 nid: local remote: 0x0 expref: -99 pid: 6257 timeout: 0 lvb_type: 0 00010000:00010000:3.0:1776178151.786172:0:3339:0:(ldlm_lock.c:820:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(COS) ns: mdt-lustre-MDT0000_UUID lock: 0000000055d6a00d/0xa40083ab9bf532af lrc: 3/0,1 mode: COS/COS res: [0x2000013a1:0x1:0x0].0xa6b02dea bits 0x2/0x0 rrc: 2 type: IBT gid 0 flags: 0x50210000000000 nid: local remote: 0x0 expref: -99 pid: 6257 timeout: 0 lvb_type: 0 00010000:00010000:3.0:1776178151.786177:0:3339:0:(ldlm_lock.c:879:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: mdt-lustre-MDT0000_UUID lock: 0000000055d6a00d/0xa40083ab9bf532af lrc: 2/0,0 mode: COS/COS res: [0x2000013a1:0x1:0x0].0xa6b02dea bits 0x2/0x0 rrc: 2 type: IBT gid 0 flags: 0x50210400000000 nid: local remote: 0x0 expref: -99 pid: 6257 timeout: 0 lvb_type: 0 00010000:00010000:3.0:1776178151.786182:0:3339:0:(ldlm_lockd.c:1879:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: mdt-lustre-MDT0000_UUID lock: 0000000055d6a00d/0xa40083ab9bf532af lrc: 3/0,0 mode: COS/COS res: [0x2000013a1:0x1:0x0].0xa6b02dea bits 0x2/0x0 rrc: 2 type: IBT gid 0 flags: 0x40210400000000 nid: local remote: 0x0 expref: -99 pid: 6257 timeout: 0 lvb_type: 0 00010000:00010000:3.0:1776178151.786187:0:3339:0:(ldlm_lockd.c:1893:ldlm_handle_bl_callback()) Lock 0000000055d6a00d already unused, calling callback (0000000079e78348) 00010000:00010000:3.0:1776178151.786189:0:3339:0:(ldlm_request.c:364:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: mdt-lustre-MDT0000_UUID lock: 0000000055d6a00d/0xa40083ab9bf532af lrc: 3/0,0 mode: COS/COS res: [0x2000013a1:0x1:0x0].0xa6b02dea bits 0x2/0x0 rrc: 2 type: IBT gid 0 flags: 0x40210400000000 nid: local remote: 0x0 expref: -99 pid: 6257 timeout: 0 lvb_type: 0 00010000:00010000:3.0:1776178151.786194:0:3339:0:(ldlm_request.c:1257:ldlm_cli_cancel_local()) ### server-side local cancel ns: mdt-lustre-MDT0000_UUID lock: 0000000055d6a00d/0xa40083ab9bf532af lrc: 4/0,0 mode: COS/COS res: [0x2000013a1:0x1:0x0].0xa6b02dea bits 0x2/0x0 rrc: 2 type: IBT gid 0 flags: 0x40218400000000 nid: local remote: 0x0 expref: -99 pid: 6257 timeout: 0 lvb_type: 0 00010000:00010000:3.0:1776178151.786201:0:3339:0:(ldlm_inodebits.c:94:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x2000013a1:0x1:0x0].0xa6b02dea (000000007bfb9649) 00010000:00010000:3.0:1776178151.786202:0:3339:0:(ldlm_inodebits.c:96:ldlm_reprocess_inodebits_queue()) Hint 2 00010000:00010000:3.0:1776178151.786205:0:3339:0:(ldlm_lockd.c:1903:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: mdt-lustre-MDT0000_UUID lock: 0000000055d6a00d/0xa40083ab9bf532af lrc: 2/0,0 mode: --/COS res: [0x2000013a1:0x1:0x0].0xa6b02dea bits 0x2/0x0 rrc: 2 type: IBT gid 0 flags: 0x44a19400000000 nid: local remote: 0x0 expref: -99 pid: 6257 timeout: 0 lvb_type: 0 00010000:00010000:3.0:1776178151.786214:0:3339:0:(ldlm_lock.c:216:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: mdt-lustre-MDT0000_UUID lock: 0000000055d6a00d/0xa40083ab9bf532af lrc: 0/0,0 mode: --/COS res: [0x2000013a1:0x1:0x0].0xa6b02dea bits 0x2/0x0 rrc: 2 type: IBT gid 0 flags: 0x44a19400000000 nid: local remote: 0x0 expref: -99 pid: 6257 timeout: 0 lvb_type: 0 00010000:00010000:3.0:1776178151.786227:0:3339:0:(ldlm_lock.c:820:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(COS) ns: mdt-lustre-MDT0000_UUID lock: 00000000c1e7ea3e/0xa40083ab9bf532b6 lrc: 3/0,1 mode: COS/COS res: [0x2000013a1:0x5:0x0].0x0 bits 0x3/0x0 rrc: 2 type: IBT gid 0 flags: 0x50210000000000 nid: local remote: 0x0 expref: -99 pid: 6257 timeout: 0 lvb_type: 0 00010000:00010000:3.0:1776178151.786233:0:3339:0:(ldlm_lock.c:879:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: mdt-lustre-MDT0000_UUID lock: 00000000c1e7ea3e/0xa40083ab9bf532b6 lrc: 2/0,0 mode: COS/COS res: [0x2000013a1:0x5:0x0].0x0 bits 0x3/0x0 rrc: 2 type: IBT gid 0 flags: 0x50210400000000 nid: local remote: 0x0 expref: -99 pid: 6257 timeout: 0 lvb_type: 0 00010000:00010000:3.0:1776178151.786239:0:3339:0:(ldlm_lockd.c:1879:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: mdt-lustre-MDT0000_UUID lock: 00000000c1e7ea3e/0xa40083ab9bf532b6 lrc: 3/0,0 mode: COS/COS res: [0x2000013a1:0x5:0x0].0x0 bits 0x3/0x0 rrc: 2 type: IBT gid 0 flags: 0x40210400000000 nid: local remote: 0x0 expref: -99 pid: 6257 timeout: 0 lvb_type: 0 00010000:00010000:3.0:1776178151.786245:0:3339:0:(ldlm_lockd.c:1893:ldlm_handle_bl_callback()) Lock 00000000c1e7ea3e already unused, calling callback (0000000079e78348) 00010000:00010000:3.0:1776178151.786248:0:3339:0:(ldlm_request.c:364:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: mdt-lustre-MDT0000_UUID lock: 00000000c1e7ea3e/0xa40083ab9bf532b6 lrc: 3/0,0 mode: COS/COS res: [0x2000013a1:0x5:0x0].0x0 bits 0x3/0x0 rrc: 2 type: IBT gid 0 flags: 0x40210400000000 nid: local remote: 0x0 expref: -99 pid: 6257 timeout: 0 lvb_type: 0 00010000:00010000:3.0:1776178151.786254:0:3339:0:(ldlm_request.c:1257:ldlm_cli_cancel_local()) ### server-side local cancel ns: mdt-lustre-MDT0000_UUID lock: 00000000c1e7ea3e/0xa40083ab9bf532b6 lrc: 4/0,0 mode: COS/COS res: [0x2000013a1:0x5:0x0].0x0 bits 0x3/0x0 rrc: 2 type: IBT gid 0 flags: 0x40218400000000 nid: local remote: 0x0 expref: -99 pid: 6257 timeout: 0 lvb_type: 0 00010000:00010000:3.0:1776178151.786262:0:3339:0:(ldlm_inodebits.c:94:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x2000013a1:0x5:0x0].0x0 (000000006b0fcbcb) 00010000:00010000:3.0:1776178151.786265:0:3339:0:(ldlm_inodebits.c:96:ldlm_reprocess_inodebits_queue()) Hint 3 00010000:00010000:3.0:1776178151.786267:0:3339:0:(ldlm_lockd.c:1903:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: mdt-lustre-MDT0000_UUID lock: 00000000c1e7ea3e/0xa40083ab9bf532b6 lrc: 2/0,0 mode: --/COS res: [0x2000013a1:0x5:0x0].0x0 bits 0x3/0x0 rrc: 2 type: IBT gid 0 flags: 0x44a19400000000 nid: local remote: 0x0 expref: -99 pid: 6257 timeout: 0 lvb_type: 0 00010000:00010000:3.0:1776178151.786273:0:3339:0:(ldlm_lock.c:216:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: mdt-lustre-MDT0000_UUID lock: 00000000c1e7ea3e/0xa40083ab9bf532b6 lrc: 0/0,0 mode: --/COS res: [0x2000013a1:0x5:0x0].0x0 bits 0x3/0x0 rrc: 2 type: IBT gid 0 flags: 0x44a19400000000 nid: local remote: 0x0 expref: -99 pid: 6257 timeout: 0 lvb_type: 0 00010000:00010000:3.0:1776178151.786286:0:3339:0:(ldlm_lock.c:820:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(COS) ns: mdt-lustre-MDT0000_UUID lock: 000000002a990613/0xa40083ab9bf532cb lrc: 3/0,1 mode: COS/COS res: [0x200000007:0x1:0x0].0xc2d93176 bits 0x2/0x0 rrc: 2 type: IBT gid 0 flags: 0x50210000000000 nid: local remote: 0x0 expref: -99 pid: 6257 timeout: 0 lvb_type: 0 00010000:00010000:3.0:1776178151.786297:0:3339:0:(ldlm_lock.c:879:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: mdt-lustre-MDT0000_UUID lock: 000000002a990613/0xa40083ab9bf532cb lrc: 2/0,0 mode: COS/COS res: [0x200000007:0x1:0x0].0xc2d93176 bits 0x2/0x0 rrc: 2 type: IBT gid 0 flags: 0x50210400000000 nid: local remote: 0x0 expref: -99 pid: 6257 timeout: 0 lvb_type: 0 00010000:00010000:3.0:1776178151.786303:0:3339:0:(ldlm_lockd.c:1879:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: mdt-lustre-MDT0000_UUID lock: 000000002a990613/0xa40083ab9bf532cb lrc: 3/0,0 mode: COS/COS res: [0x200000007:0x1:0x0].0xc2d93176 bits 0x2/0x0 rrc: 2 type: IBT gid 0 flags: 0x40210400000000 nid: local remote: 0x0 expref: -99 pid: 6257 timeout: 0 lvb_type: 0 00010000:00010000:3.0:1776178151.786309:0:3339:0:(ldlm_lockd.c:1893:ldlm_handle_bl_callback()) Lock 000000002a990613 already unused, calling callback (0000000079e78348) 00010000:00010000:3.0:1776178151.786311:0:3339:0:(ldlm_request.c:364:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: mdt-lustre-MDT0000_UUID lock: 000000002a990613/0xa40083ab9bf532cb lrc: 3/0,0 mode: COS/COS res: [0x200000007:0x1:0x0].0xc2d93176 bits 0x2/0x0 rrc: 2 type: IBT gid 0 flags: 0x40210400000000 nid: local remote: 0x0 expref: -99 pid: 6257 timeout: 0 lvb_type: 0 00010000:00010000:3.0:1776178151.786316:0:3339:0:(ldlm_request.c:1257:ldlm_cli_cancel_local()) ### server-side local cancel ns: mdt-lustre-MDT0000_UUID lock: 000000002a990613/0xa40083ab9bf532cb lrc: 4/0,0 mode: COS/COS res: [0x200000007:0x1:0x0].0xc2d93176 bits 0x2/0x0 rrc: 2 type: IBT gid 0 flags: 0x40218400000000 nid: local remote: 0x0 expref: -99 pid: 6257 timeout: 0 lvb_type: 0 00010000:00010000:3.0:1776178151.786324:0:3339:0:(ldlm_inodebits.c:94:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x200000007:0x1:0x0].0xc2d93176 (000000007b7d95c1) 00010000:00010000:3.0:1776178151.786326:0:3339:0:(ldlm_inodebits.c:96:ldlm_reprocess_inodebits_queue()) Hint 2 00010000:00010000:3.0:1776178151.786328:0:3339:0:(ldlm_lockd.c:1903:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: mdt-lustre-MDT0000_UUID lock: 000000002a990613/0xa40083ab9bf532cb lrc: 2/0,0 mode: --/COS res: [0x200000007:0x1:0x0].0xc2d93176 bits 0x2/0x0 rrc: 2 type: IBT gid 0 flags: 0x44a19400000000 nid: local remote: 0x0 expref: -99 pid: 6257 timeout: 0 lvb_type: 0 00010000:00010000:3.0:1776178151.786334:0:3339:0:(ldlm_lock.c:216:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: mdt-lustre-MDT0000_UUID lock: 000000002a990613/0xa40083ab9bf532cb lrc: 0/0,0 mode: --/COS res: [0x200000007:0x1:0x0].0xc2d93176 bits 0x2/0x0 rrc: 2 type: IBT gid 0 flags: 0x44a19400000000 nid: local remote: 0x0 expref: -99 pid: 6257 timeout: 0 lvb_type: 0 00010000:00010000:3.0:1776178151.786346:0:3339:0:(ldlm_lock.c:820:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(COS) ns: mdt-lustre-MDT0000_UUID lock: 00000000014a8a18/0xa40083ab9bf532d2 lrc: 3/0,1 mode: COS/COS res: [0x2000013a1:0x1:0x0].0x0 bits 0x3/0x0 rrc: 2 type: IBT gid 0 flags: 0x50210000000000 nid: local remote: 0x0 expref: -99 pid: 6257 timeout: 0 lvb_type: 0 00010000:00010000:3.0:1776178151.786353:0:3339:0:(ldlm_lock.c:879:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: mdt-lustre-MDT0000_UUID lock: 00000000014a8a18/0xa40083ab9bf532d2 lrc: 2/0,0 mode: COS/COS res: [0x2000013a1:0x1:0x0].0x0 bits 0x3/0x0 rrc: 2 type: IBT gid 0 flags: 0x50210400000000 nid: local remote: 0x0 expref: -99 pid: 6257 timeout: 0 lvb_type: 0 00010000:00010000:3.0:1776178151.786358:0:3339:0:(ldlm_lockd.c:1879:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: mdt-lustre-MDT0000_UUID lock: 00000000014a8a18/0xa40083ab9bf532d2 lrc: 3/0,0 mode: COS/COS res: [0x2000013a1:0x1:0x0].0x0 bits 0x3/0x0 rrc: 2 type: IBT gid 0 flags: 0x40210400000000 nid: local remote: 0x0 expref: -99 pid: 6257 timeout: 0 lvb_type: 0 00010000:00010000:3.0:1776178151.786366:0:3339:0:(ldlm_lockd.c:1893:ldlm_handle_bl_callback()) Lock 00000000014a8a18 already unused, calling callback (0000000079e78348) 00010000:00010000:3.0:1776178151.786368:0:3339:0:(ldlm_request.c:364:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: mdt-lustre-MDT0000_UUID lock: 00000000014a8a18/0xa40083ab9bf532d2 lrc: 3/0,0 mode: COS/COS res: [0x2000013a1:0x1:0x0].0x0 bits 0x3/0x0 rrc: 2 type: IBT gid 0 flags: 0x40210400000000 nid: local remote: 0x0 expref: -99 pid: 6257 timeout: 0 lvb_type: 0 00010000:00010000:3.0:1776178151.786373:0:3339:0:(ldlm_request.c:1257:ldlm_cli_cancel_local()) ### server-side local cancel ns: mdt-lustre-MDT0000_UUID lock: 00000000014a8a18/0xa40083ab9bf532d2 lrc: 4/0,0 mode: COS/COS res: [0x2000013a1:0x1:0x0].0x0 bits 0x3/0x0 rrc: 2 type: IBT gid 0 flags: 0x40218400000000 nid: local remote: 0x0 expref: -99 pid: 6257 timeout: 0 lvb_type: 0 00010000:00010000:3.0:1776178151.786380:0:3339:0:(ldlm_inodebits.c:94:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x2000013a1:0x1:0x0].0x0 (00000000099618da) 00010000:00010000:3.0:1776178151.786382:0:3339:0:(ldlm_inodebits.c:96:ldlm_reprocess_inodebits_queue()) Hint 3 00010000:00010000:3.0:1776178151.786383:0:3339:0:(ldlm_lockd.c:1903:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: mdt-lustre-MDT0000_UUID lock: 00000000014a8a18/0xa40083ab9bf532d2 lrc: 2/0,0 mode: --/COS res: [0x2000013a1:0x1:0x0].0x0 bits 0x3/0x0 rrc: 2 type: IBT gid 0 flags: 0x44a19400000000 nid: local remote: 0x0 expref: -99 pid: 6257 timeout: 0 lvb_type: 0 00010000:00010000:3.0:1776178151.786388:0:3339:0:(ldlm_lock.c:216:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: mdt-lustre-MDT0000_UUID lock: 00000000014a8a18/0xa40083ab9bf532d2 lrc: 0/0,0 mode: --/COS res: [0x2000013a1:0x1:0x0].0x0 bits 0x3/0x0 rrc: 2 type: IBT gid 0 flags: 0x44a19400000000 nid: local remote: 0x0 expref: -99 pid: 6257 timeout: 0 lvb_type: 0 00010000:00010000:1.0:1776178151.786656:0:8227:0:(ldlm_lock.c:766:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0000_UUID lock: 000000006b0afb52/0xa40083ab9bf532e7 lrc: 3/0,1 mode: --/PW res: [0x43:0x0:0x0].0x0 rrc: 3 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 8227 timeout: 0 lvb_type: 0 00010000:00010000:1.0:1776178151.786674:0:8227:0:(ldlm_lock.c:675:ldlm_add_bl_work_item()) ### lock incompatible; sending blocking AST. ns: filter-lustre-OST0000_UUID lock: 000000002cc4819c/0xa40083ab9bf53094 lrc: 2/0,0 mode: PR/PR res: [0x43:0x0:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x40000000000000 nid: 192.168.204.3@tcp remote: 0x771f68aa1eb279f2 expref: 12 pid: 20762 timeout: 0 lvb_type: 1 00010000:00010000:1.0:1776178151.786681:0:8227:0:(ldlm_extent.c:318:ldlm_check_contention()) contended locks = 0 00010000:00010000:1.0:1776178151.786683:0:8227:0:(ldlm_extent.c:318:ldlm_check_contention()) contended locks = 0 00010000:00010000:1.0:1776178151.786685:0:8227:0:(ldlm_resource.c:1650:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 000000006b0afb52/0xa40083ab9bf532e7 lrc: 4/0,1 mode: --/PW res: [0x43:0x0:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x50010080000000 nid: local remote: 0x0 expref: -99 pid: 8227 timeout: 0 lvb_type: 0 00010000:00010000:1.0:1776178151.786734:0:8227:0:(ldlm_lockd.c:953:ldlm_server_blocking_ast()) ### server preparing blocking AST ns: filter-lustre-OST0000_UUID lock: 000000002cc4819c/0xa40083ab9bf53094 lrc: 3/0,0 mode: PR/PR res: [0x43:0x0:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x50000000010020 nid: 192.168.204.3@tcp remote: 0x771f68aa1eb279f2 expref: 12 pid: 20762 timeout: 0 lvb_type: 1 00010000:00010000:1.0:1776178151.786745:0:8227:0:(ldlm_lockd.c:489:ldlm_add_waiting_lock()) ### adding to wait list(timeout: 100, AT: on) ns: filter-lustre-OST0000_UUID lock: 000000002cc4819c/0xa40083ab9bf53094 lrc: 4/0,0 mode: PR/PR res: [0x43:0x0:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x70000400010020 nid: 192.168.204.3@tcp remote: 0x771f68aa1eb279f2 expref: 12 pid: 20762 timeout: 959 lvb_type: 1 00000100:00100000:1.0:1776178151.786759:0:8227:0:(client.c:1733:ptlrpc_send_new_req()) Sending RPC req@00000000a6227918 pname:cluuid:pid:xid:nid:opc:job ll_ost00_000:lustre-OST0000_UUID:8227:1862456998033536:192.168.204.3@tcp:104: 00000100:00100000:1.0:1776178151.786814:0:8227:0:(client.c:2500:ptlrpc_set_wait()) set 00000000f8af0a3b going to sleep for 6 seconds 00000100:00100000:1.0:1776178151.789273:0:3356:0:(client.c:1733:ptlrpc_send_new_req()) Sending RPC req@0000000023210d1c pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_03:lustre-MDT0000-mdtlov_UUID:3356:1862456998033088:0@lo:6:osp-syn-1-0.0 00000100:00100000:1.0:1776178151.789370:0:3356:0:(events.c:361:request_in_callback()) peer: 12345-0@lo (source: 12345-0@lo) 00000100:00100000:1.0:1776178151.789419:0:3356:0:(client.c:1733:ptlrpc_send_new_req()) Sending RPC req@00000000a59bfc89 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_03:lustre-MDT0000-mdtlov_UUID:3356:1862456998033344:0@lo:6:osp-syn-0-0.0 00000100:00100000:1.0:1776178151.789443:0:3356:0:(events.c:361:request_in_callback()) peer: 12345-0@lo (source: 12345-0@lo) 00000100:00100000:1.0:1776178151.789470:0:3356:0:(ptlrpcd.c:415:ptlrpcd_check()) transfer 2 async RPCs [2->3] 00000100:00100000:1.0:1776178151.789477:0:3356:0:(client.c:1733:ptlrpc_send_new_req()) Sending RPC req@0000000067671fc8 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_03:lustre-MDT0000-mdtlov_UUID:3356:1862456998033024:0@lo:6:osp-syn-1-0.0 00000100:00100000:1.0:1776178151.789508:0:3356:0:(events.c:361:request_in_callback()) peer: 12345-0@lo (source: 12345-0@lo) 00000100:00100000:1.0:1776178151.789525:0:3356:0:(client.c:1733:ptlrpc_send_new_req()) Sending RPC req@00000000575d0558 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_03:lustre-MDT0000-mdtlov_UUID:3356:1862456998033280:0@lo:6:osp-syn-0-0.0 00000100:00100000:1.0:1776178151.789543:0:3356:0:(events.c:361:request_in_callback()) peer: 12345-0@lo (source: 12345-0@lo) 00000100:00100000:1.0:1776178151.789591:0:20764:0:(service.c:2152:ptlrpc_server_handle_req_in()) got req x1862456998033088 00000100:00100000:1.0:1776178151.789638:0:20764:0:(service.c:2152:ptlrpc_server_handle_req_in()) got req x1862456998033344 00000100:00100000:1.0:1776178151.789653:0:20764:0:(service.c:2152:ptlrpc_server_handle_req_in()) got req x1862456998033024 00000001:00080000:3.0:1776178151.789841:0:8225:0:(tgt_lastrcvd.c:913:tgt_cb_last_committed()) lustre-OST0000: transno 4294969348 is committed 00000001:00080000:3.0:1776178151.789885:0:9217:0:(tgt_lastrcvd.c:913:tgt_cb_last_committed()) lustre-OST0001: transno 4294969320 is committed 00000100:00100000:1.0:1776178151.790295:0:20764:0:(service.c:2152:ptlrpc_server_handle_req_in()) got req x1862456998033280 00000100:00100000:3.0:1776178151.790592:0:8228:0:(nrs_fifo.c:179:nrs_fifo_req_get()) NRS start fifo request from 12345-0@lo, seq: 5253 00000100:00100000:3.0:1776178151.790599:0:8228:0:(service.c:2324:ptlrpc_server_handle_request()) Handling RPC req@0000000039ff8335 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_001:lustre-MDT0000-mdtlov_UUID+7:3356:x1862456998033088:12345-0@lo:6:osp-syn-1-0.0 00002000:00080000:3.0:1776178151.790635:0:8228:0:(ofd_dev.c:1822:ofd_destroy_hdl()) lustre-OST0001: Destroy object 0x0:67 count 1 00010000:00010000:3.0:1776178151.790664:0:8228:0:(ldlm_lock.c:766:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0001_UUID lock: 000000006c464689/0xa40083ab9bf532ee lrc: 3/0,1 mode: --/PW res: [0x43:0x0:0x0].0x0 rrc: 3 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 8228 timeout: 0 lvb_type: 0 00010000:00010000:3.0:1776178151.790686:0:8228:0:(ldlm_lock.c:675:ldlm_add_bl_work_item()) ### lock incompatible; sending blocking AST. ns: filter-lustre-OST0001_UUID lock: 00000000daa24072/0xa40083ab9bf5308d lrc: 2/0,0 mode: PR/PR res: [0x43:0x0:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x40000000000000 nid: 192.168.204.3@tcp remote: 0x771f68aa1eb279eb expref: 7 pid: 20762 timeout: 0 lvb_type: 1 00010000:00010000:3.0:1776178151.790707:0:8228:0:(ldlm_extent.c:318:ldlm_check_contention()) contended locks = 0 00010000:00010000:3.0:1776178151.790710:0:8228:0:(ldlm_extent.c:318:ldlm_check_contention()) contended locks = 0 00010000:00010000:3.0:1776178151.790714:0:8228:0:(ldlm_resource.c:1650:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0001_UUID lock: 000000006c464689/0xa40083ab9bf532ee lrc: 4/0,1 mode: --/PW res: [0x43:0x0:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x50010080000000 nid: local remote: 0x0 expref: -99 pid: 8228 timeout: 0 lvb_type: 0 00000100:00100000:2.0:1776178151.790728:0:20755:0:(nrs_fifo.c:179:nrs_fifo_req_get()) NRS start fifo request from 12345-0@lo, seq: 5254 00000100:00100000:2.0:1776178151.790736:0:20755:0:(service.c:2324:ptlrpc_server_handle_request()) Handling RPC req@0000000022d4dc29 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_006:lustre-MDT0000-mdtlov_UUID+8:3356:x1862456998033344:12345-0@lo:6:osp-syn-0-0.0 00010000:00010000:3.0:1776178151.790755:0:8228:0:(ldlm_lockd.c:953:ldlm_server_blocking_ast()) ### server preparing blocking AST ns: filter-lustre-OST0001_UUID lock: 00000000daa24072/0xa40083ab9bf5308d lrc: 3/0,0 mode: PR/PR res: [0x43:0x0:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x50000000010020 nid: 192.168.204.3@tcp remote: 0x771f68aa1eb279eb expref: 7 pid: 20762 timeout: 0 lvb_type: 1 00002000:00080000:2.0:1776178151.790765:0:20755:0:(ofd_dev.c:1822:ofd_destroy_hdl()) lustre-OST0000: Destroy object 0x0:69 count 1 00010000:00010000:3.0:1776178151.790769:0:8228:0:(ldlm_lockd.c:489:ldlm_add_waiting_lock()) ### adding to wait list(timeout: 100, AT: on) ns: filter-lustre-OST0001_UUID lock: 00000000daa24072/0xa40083ab9bf5308d lrc: 4/0,0 mode: PR/PR res: [0x43:0x0:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x70000400010020 nid: 192.168.204.3@tcp remote: 0x771f68aa1eb279eb expref: 7 pid: 20762 timeout: 959 lvb_type: 1 00010000:00010000:2.0:1776178151.790782:0:20755:0:(ldlm_lock.c:766:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0000_UUID lock: 00000000fcb32840/0xa40083ab9bf532f5 lrc: 3/0,1 mode: --/PW res: [0x45:0x0:0x0].0x0 rrc: 3 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 20755 timeout: 0 lvb_type: 0 00000100:00100000:3.0:1776178151.790785:0:8228:0:(client.c:1733:ptlrpc_send_new_req()) Sending RPC req@00000000df370bee pname:cluuid:pid:xid:nid:opc:job ll_ost00_001:lustre-OST0001_UUID:8228:1862456998033600:192.168.204.3@tcp:104: 00010000:00010000:2.0:1776178151.790801:0:20755:0:(ldlm_lock.c:675:ldlm_add_bl_work_item()) ### lock incompatible; sending blocking AST. ns: filter-lustre-OST0000_UUID lock: 00000000a14e680b/0xa40083ab9bf5309b lrc: 2/0,0 mode: PR/PR res: [0x45:0x0:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x40000000000000 nid: 192.168.204.3@tcp remote: 0x771f68aa1eb279f9 expref: 8 pid: 20762 timeout: 0 lvb_type: 1 00010000:00010000:2.0:1776178151.790810:0:20755:0:(ldlm_extent.c:318:ldlm_check_contention()) contended locks = 0 00010000:00010000:2.0:1776178151.790812:0:20755:0:(ldlm_extent.c:318:ldlm_check_contention()) contended locks = 0 00010000:00010000:2.0:1776178151.790814:0:20755:0:(ldlm_resource.c:1650:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 00000000fcb32840/0xa40083ab9bf532f5 lrc: 4/0,1 mode: --/PW res: [0x45:0x0:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x50010080000000 nid: local remote: 0x0 expref: -99 pid: 20755 timeout: 0 lvb_type: 0 00010000:00010000:2.0:1776178151.790836:0:20755:0:(ldlm_lockd.c:953:ldlm_server_blocking_ast()) ### server preparing blocking AST ns: filter-lustre-OST0000_UUID lock: 00000000a14e680b/0xa40083ab9bf5309b lrc: 3/0,0 mode: PR/PR res: [0x45:0x0:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x50000000010020 nid: 192.168.204.3@tcp remote: 0x771f68aa1eb279f9 expref: 8 pid: 20762 timeout: 0 lvb_type: 1 00010000:00010000:2.0:1776178151.790845:0:20755:0:(ldlm_lockd.c:489:ldlm_add_waiting_lock()) ### adding to wait list(timeout: 100, AT: on) ns: filter-lustre-OST0000_UUID lock: 00000000a14e680b/0xa40083ab9bf5309b lrc: 4/0,0 mode: PR/PR res: [0x45:0x0:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x70000400010020 nid: 192.168.204.3@tcp remote: 0x771f68aa1eb279f9 expref: 8 pid: 20762 timeout: 959 lvb_type: 1 00000100:00100000:2.0:1776178151.790857:0:20755:0:(client.c:1733:ptlrpc_send_new_req()) Sending RPC req@00000000e0d60cbc pname:cluuid:pid:xid:nid:opc:job ll_ost00_006:lustre-OST0000_UUID:20755:1862456998033664:192.168.204.3@tcp:104: 00000100:00100000:3.0:1776178151.790860:0:8228:0:(client.c:2500:ptlrpc_set_wait()) set 0000000070d31bf6 going to sleep for 6 seconds 00000100:00100000:3.0:1776178151.790885:0:20765:0:(nrs_fifo.c:179:nrs_fifo_req_get()) NRS start fifo request from 12345-0@lo, seq: 5255 00000100:00100000:2.0:1776178151.790887:0:20755:0:(client.c:2500:ptlrpc_set_wait()) set 00000000599985c1 going to sleep for 6 seconds 00000100:00100000:3.0:1776178151.790891:0:20765:0:(service.c:2324:ptlrpc_server_handle_request()) Handling RPC req@00000000699e66b5 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_016:lustre-MDT0000-mdtlov_UUID+7:3356:x1862456998033024:12345-0@lo:6:osp-syn-1-0.0 00002000:00080000:3.0:1776178151.790905:0:20765:0:(ofd_dev.c:1822:ofd_destroy_hdl()) lustre-OST0001: Destroy object 0x0:68 count 1 00010000:00010000:3.0:1776178151.790914:0:20765:0:(ldlm_lock.c:766:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0001_UUID lock: 000000001b3f1324/0xa40083ab9bf532fc lrc: 3/0,1 mode: --/PW res: [0x44:0x0:0x0].0x0 rrc: 3 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 20765 timeout: 0 lvb_type: 0 00010000:00010000:3.0:1776178151.790924:0:20765:0:(ldlm_lock.c:675:ldlm_add_bl_work_item()) ### lock incompatible; sending blocking AST. ns: filter-lustre-OST0001_UUID lock: 000000007797f5e2/0xa40083ab9bf530a2 lrc: 2/0,0 mode: PR/PR res: [0x44:0x0:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x40000000000000 nid: 192.168.204.3@tcp remote: 0x771f68aa1eb27a00 expref: 7 pid: 14440 timeout: 0 lvb_type: 1 00010000:00010000:3.0:1776178151.790931:0:20765:0:(ldlm_extent.c:318:ldlm_check_contention()) contended locks = 0 00010000:00010000:3.0:1776178151.790932:0:20765:0:(ldlm_extent.c:318:ldlm_check_contention()) contended locks = 0 00010000:00010000:3.0:1776178151.790933:0:20765:0:(ldlm_resource.c:1650:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0001_UUID lock: 000000001b3f1324/0xa40083ab9bf532fc lrc: 4/0,1 mode: --/PW res: [0x44:0x0:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x50010080000000 nid: local remote: 0x0 expref: -99 pid: 20765 timeout: 0 lvb_type: 0 00010000:00010000:3.0:1776178151.790948:0:20765:0:(ldlm_lockd.c:953:ldlm_server_blocking_ast()) ### server preparing blocking AST ns: filter-lustre-OST0001_UUID lock: 000000007797f5e2/0xa40083ab9bf530a2 lrc: 3/0,0 mode: PR/PR res: [0x44:0x0:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x50000000010020 nid: 192.168.204.3@tcp remote: 0x771f68aa1eb27a00 expref: 7 pid: 14440 timeout: 0 lvb_type: 1 00010000:00010000:3.0:1776178151.790961:0:20765:0:(ldlm_lockd.c:489:ldlm_add_waiting_lock()) ### adding to wait list(timeout: 100, AT: on) ns: filter-lustre-OST0001_UUID lock: 000000007797f5e2/0xa40083ab9bf530a2 lrc: 4/0,0 mode: PR/PR res: [0x44:0x0:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x70000400010020 nid: 192.168.204.3@tcp remote: 0x771f68aa1eb27a00 expref: 7 pid: 14440 timeout: 959 lvb_type: 1 00000100:00100000:3.0:1776178151.790970:0:20765:0:(client.c:1733:ptlrpc_send_new_req()) Sending RPC req@0000000089632c95 pname:cluuid:pid:xid:nid:opc:job ll_ost00_016:lustre-OST0001_UUID:20765:1862456998033728:192.168.204.3@tcp:104: 00000100:00100000:3.0:1776178151.790993:0:20765:0:(client.c:2500:ptlrpc_set_wait()) set 00000000b8f976e8 going to sleep for 6 seconds 00000100:00100000:3.0:1776178151.791182:0:20769:0:(nrs_fifo.c:179:nrs_fifo_req_get()) NRS start fifo request from 12345-0@lo, seq: 5256 00000100:00100000:3.0:1776178151.791187:0:20769:0:(service.c:2324:ptlrpc_server_handle_request()) Handling RPC req@0000000056ddcb09 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_018:lustre-MDT0000-mdtlov_UUID+8:3356:x1862456998033280:12345-0@lo:6:osp-syn-0-0.0 00002000:00080000:3.0:1776178151.791201:0:20769:0:(ofd_dev.c:1822:ofd_destroy_hdl()) lustre-OST0000: Destroy object 0x0:68 count 1 00010000:00010000:3.0:1776178151.791213:0:20769:0:(ldlm_lock.c:766:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0000_UUID lock: 000000009f2de1df/0xa40083ab9bf53303 lrc: 3/0,1 mode: --/PW res: [0x44:0x0:0x0].0x0 rrc: 3 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 20769 timeout: 0 lvb_type: 0 00010000:00010000:3.0:1776178151.791223:0:20769:0:(ldlm_lock.c:675:ldlm_add_bl_work_item()) ### lock incompatible; sending blocking AST. ns: filter-lustre-OST0000_UUID lock: 000000007f7ce364/0xa40083ab9bf53086 lrc: 2/0,0 mode: PR/PR res: [0x44:0x0:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x40000000000000 nid: 192.168.204.3@tcp remote: 0x771f68aa1eb279d6 expref: 8 pid: 14440 timeout: 0 lvb_type: 1 00010000:00010000:3.0:1776178151.791229:0:20769:0:(ldlm_extent.c:318:ldlm_check_contention()) contended locks = 0 00010000:00010000:3.0:1776178151.791231:0:20769:0:(ldlm_extent.c:318:ldlm_check_contention()) contended locks = 0 00010000:00010000:3.0:1776178151.791232:0:20769:0:(ldlm_resource.c:1650:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 000000009f2de1df/0xa40083ab9bf53303 lrc: 4/0,1 mode: --/PW res: [0x44:0x0:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x50010080000000 nid: local remote: 0x0 expref: -99 pid: 20769 timeout: 0 lvb_type: 0 00010000:00010000:3.0:1776178151.791250:0:20769:0:(ldlm_lockd.c:953:ldlm_server_blocking_ast()) ### server preparing blocking AST ns: filter-lustre-OST0000_UUID lock: 000000007f7ce364/0xa40083ab9bf53086 lrc: 3/0,0 mode: PR/PR res: [0x44:0x0:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x50000000010020 nid: 192.168.204.3@tcp remote: 0x771f68aa1eb279d6 expref: 8 pid: 14440 timeout: 0 lvb_type: 1 00010000:00010000:3.0:1776178151.791257:0:20769:0:(ldlm_lockd.c:489:ldlm_add_waiting_lock()) ### adding to wait list(timeout: 100, AT: on) ns: filter-lustre-OST0000_UUID lock: 000000007f7ce364/0xa40083ab9bf53086 lrc: 4/0,0 mode: PR/PR res: [0x44:0x0:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x70000400010020 nid: 192.168.204.3@tcp remote: 0x771f68aa1eb279d6 expref: 8 pid: 14440 timeout: 959 lvb_type: 1 00000100:00100000:3.0:1776178151.791265:0:20769:0:(client.c:1733:ptlrpc_send_new_req()) Sending RPC req@00000000f0ef812c pname:cluuid:pid:xid:nid:opc:job ll_ost00_018:lustre-OST0000_UUID:20769:1862456998033792:192.168.204.3@tcp:104: 00000100:00100000:3.0:1776178151.791360:0:20769:0:(client.c:2500:ptlrpc_set_wait()) set 00000000707b2f30 going to sleep for 6 seconds 00000100:00100000:3.0:1776178151.806539:0:20762:0:(client.c:2215:ptlrpc_check_set()) Completed RPC req@0000000078ae567e pname:cluuid:pid:xid:nid:opc:job ll_ost00_013:lustre-OST0000_UUID:20762:1862456998033472:192.168.204.3@tcp:104: 00010000:00010000:3.0:1776178151.806583:0:20762:0:(ldlm_request.c:283:ldlm_completion_ast()) ### client-side enqueue returned a blocked locksleeping ns: filter-lustre-OST0000_UUID lock: 00000000b2e83690/0xa40083ab9bf532e0 lrc: 3/0,1 mode: --/PW res: [0x42:0x0:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x40010080000000 nid: local remote: 0x0 expref: -99 pid: 20762 timeout: 0 lvb_type: 0 00000100:00100000:2.0:1776178151.811720:0:8228:0:(client.c:2215:ptlrpc_check_set()) Completed RPC req@00000000df370bee pname:cluuid:pid:xid:nid:opc:job ll_ost00_001:lustre-OST0001_UUID:8228:1862456998033600:192.168.204.3@tcp:104: 00010000:00010000:2.0:1776178151.811763:0:8228:0:(ldlm_request.c:283:ldlm_completion_ast()) ### client-side enqueue returned a blocked locksleeping ns: filter-lustre-OST0001_UUID lock: 000000006c464689/0xa40083ab9bf532ee lrc: 3/0,1 mode: --/PW res: [0x43:0x0:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x40010080000000 nid: local remote: 0x0 expref: -99 pid: 8228 timeout: 0 lvb_type: 0 00000100:00100000:2.0:1776178151.811800:0:20755:0:(client.c:2215:ptlrpc_check_set()) Completed RPC req@00000000e0d60cbc pname:cluuid:pid:xid:nid:opc:job ll_ost00_006:lustre-OST0000_UUID:20755:1862456998033664:192.168.204.3@tcp:104: 00010000:00010000:2.0:1776178151.811814:0:20755:0:(ldlm_request.c:283:ldlm_completion_ast()) ### client-side enqueue returned a blocked locksleeping ns: filter-lustre-OST0000_UUID lock: 00000000fcb32840/0xa40083ab9bf532f5 lrc: 3/0,1 mode: --/PW res: [0x45:0x0:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x40010080000000 nid: local remote: 0x0 expref: -99 pid: 20755 timeout: 0 lvb_type: 0 00000100:00100000:2.0:1776178151.811844:0:20765:0:(client.c:2215:ptlrpc_check_set()) Completed RPC req@0000000089632c95 pname:cluuid:pid:xid:nid:opc:job ll_ost00_016:lustre-OST0001_UUID:20765:1862456998033728:192.168.204.3@tcp:104: 00010000:00010000:2.0:1776178151.811857:0:20765:0:(ldlm_request.c:283:ldlm_completion_ast()) ### client-side enqueue returned a blocked locksleeping ns: filter-lustre-OST0001_UUID lock: 000000001b3f1324/0xa40083ab9bf532fc lrc: 3/0,1 mode: --/PW res: [0x44:0x0:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x40010080000000 nid: local remote: 0x0 expref: -99 pid: 20765 timeout: 0 lvb_type: 0 00000100:00100000:0.0:1776178151.812107:0:8227:0:(client.c:2215:ptlrpc_check_set()) Completed RPC req@00000000a6227918 pname:cluuid:pid:xid:nid:opc:job ll_ost00_000:lustre-OST0000_UUID:8227:1862456998033536:192.168.204.3@tcp:104: 00010000:00010000:0.0:1776178151.812126:0:8227:0:(ldlm_request.c:283:ldlm_completion_ast()) ### client-side enqueue returned a blocked locksleeping ns: filter-lustre-OST0000_UUID lock: 000000006b0afb52/0xa40083ab9bf532e7 lrc: 3/0,1 mode: --/PW res: [0x43:0x0:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x40010080000000 nid: local remote: 0x0 expref: -99 pid: 8227 timeout: 0 lvb_type: 0 00000100:00100000:3.0:1776178151.814020:0:3348:0:(events.c:361:request_in_callback()) peer: 12345-192.168.204.3@tcp (source: 12345-192.168.204.3@tcp) 00000100:00100000:2.0:1776178151.814132:0:20769:0:(client.c:2215:ptlrpc_check_set()) Completed RPC req@00000000f0ef812c pname:cluuid:pid:xid:nid:opc:job ll_ost00_018:lustre-OST0000_UUID:20769:1862456998033792:192.168.204.3@tcp:104: 00010000:00010000:2.0:1776178151.814158:0:20769:0:(ldlm_request.c:283:ldlm_completion_ast()) ### client-side enqueue returned a blocked locksleeping ns: filter-lustre-OST0000_UUID lock: 000000009f2de1df/0xa40083ab9bf53303 lrc: 3/0,1 mode: --/PW res: [0x44:0x0:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x40010080000000 nid: local remote: 0x0 expref: -99 pid: 20769 timeout: 0 lvb_type: 0 00000100:00100000:0.0:1776178151.814165:0:20767:0:(service.c:2152:ptlrpc_server_handle_req_in()) got req x1862456979088960 00010000:00010000:0.0:1776178151.814198:0:20767:0:(ldlm_lockd.c:2638:ldlm_cancel_hpreq_check()) ### hpreq cancel/convert lock ns: filter-lustre-OST0000_UUID lock: 000000002abc4399/0xa40083ab9bf53055 lrc: 4/0,0 mode: PR/PR res: [0x42:0x0:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x60000400010020 nid: 192.168.204.3@tcp remote: 0x771f68aa1eb279c1 expref: 9 pid: 20762 timeout: 959 lvb_type: 1 00000100:00100000:0.0:1776178151.814239:0:20767:0:(nrs_fifo.c:179:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.204.3@tcp, seq: 2073 00000100:00100000:0.0:1776178151.814245:0:20767:0:(service.c:2324:ptlrpc_server_handle_request()) Handling RPC req@000000005927b2c3 pname:cluuid+ref:pid:xid:nid:opc:job ldlm_cn00_004:b50963bd-8eba-4432-9a54-8888d00e3fa1+9:2271:x1862456979088960:12345-192.168.204.3@tcp:103:ldlm_bl_02.0 00010000:00010000:0.0:1776178151.814259:0:20767:0:(ldlm_lockd.c:1720:ldlm_request_cancel()) ### server-side cancel handler START: 1 locks, starting at 0 00010000:00010000:0.0:1776178151.814264:0:20767:0:(ldlm_lockd.c:1771:ldlm_request_cancel()) ### server cancels blocked lock after 0s ns: filter-lustre-OST0000_UUID lock: 000000002abc4399/0xa40083ab9bf53055 lrc: 4/0,0 mode: PR/PR res: [0x42:0x0:0x0].0x0 rrc: 4 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x60000400010020 nid: 192.168.204.3@tcp remote: 0x771f68aa1eb279c1 expref: 9 pid: 20762 timeout: 959 lvb_type: 1 00010000:00010000:0.0:1776178151.814274:0:20767:0:(ldlm_lockd.c:562:ldlm_del_waiting_lock()) ### removed ns: filter-lustre-OST0000_UUID lock: 000000002abc4399/0xa40083ab9bf53055 lrc: 3/0,0 mode: PR/PR res: [0x42:0x0:0x0].0x0 rrc: 4 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x50000400010020 nid: 192.168.204.3@tcp remote: 0x771f68aa1eb279c1 expref: 9 pid: 20762 timeout: 959 lvb_type: 1 00010000:00010000:0.0:1776178151.814301:0:20767:0:(ldlm_lock.c:216:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 000000002abc4399/0xa40083ab9bf53055 lrc: 0/0,0 mode: --/PR res: [0x42:0x0:0x0].0x0 rrc: 4 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x44801400010020 nid: 192.168.204.3@tcp remote: 0x771f68aa1eb279c1 expref: 9 pid: 20762 timeout: 959 lvb_type: 1 00010000:00010000:0.0:1776178151.814318:0:20767:0:(ldlm_extent.c:318:ldlm_check_contention()) contended locks = 0 00010000:00010000:0.0:1776178151.814320:0:20767:0:(ldlm_extent.c:318:ldlm_check_contention()) contended locks = 0 00010000:00010000:0.0:1776178151.814324:0:20767:0:(ldlm_lock.c:704:ldlm_add_cp_work_item()) ### lock granted; sending completion AST. ns: filter-lustre-OST0000_UUID lock: 00000000b2e83690/0xa40083ab9bf532e0 lrc: 3/0,1 mode: PW/PW res: [0x42:0x0:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x40090080000000 nid: local remote: 0x0 expref: -99 pid: 20762 timeout: 0 lvb_type: 0 00010000:00010000:0.0:1776178151.814333:0:20767:0:(ldlm_resource.c:1650:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 00000000b2e83690/0xa40083ab9bf532e0 lrc: 4/0,1 mode: PW/PW res: [0x42:0x0:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x40090080000000 nid: local remote: 0x0 expref: -99 pid: 20762 timeout: 0 lvb_type: 0 00010000:00010000:0.0:1776178151.814368:0:20767:0:(ldlm_lockd.c:1782:ldlm_request_cancel()) ### server-side cancel handler END 00010000:00010000:2.0:1776178151.814444:0:20762:0:(ldlm_request.c:199:ldlm_completion_tail()) ### client-side enqueue: granted ns: filter-lustre-OST0000_UUID lock: 00000000b2e83690/0xa40083ab9bf532e0 lrc: 3/0,1 mode: PW/PW res: [0x42:0x0:0x0].0x0 rrc: 2 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x40010080000000 nid: local remote: 0x0 expref: -99 pid: 20762 timeout: 0 lvb_type: 0 00010000:00010000:2.0:1776178151.814457:0:20762:0:(ldlm_request.c:528:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 00000000b2e83690/0xa40083ab9bf532e0 lrc: 3/0,1 mode: PW/PW res: [0x42:0x0:0x0].0x0 rrc: 2 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x40010080000000 nid: local remote: 0x0 expref: -99 pid: 20762 timeout: 0 lvb_type: 0 00010000:00010000:2.0:1776178151.814467:0:20762:0:(ldlm_lock.c:820:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0000_UUID lock: 00000000b2e83690/0xa40083ab9bf532e0 lrc: 3/0,1 mode: PW/PW res: [0x42:0x0:0x0].0x0 rrc: 2 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x50010080000000 nid: local remote: 0x0 expref: -99 pid: 20762 timeout: 0 lvb_type: 0 00010000:00010000:2.0:1776178151.814474:0:20762:0:(ldlm_lock.c:879:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 00000000b2e83690/0xa40083ab9bf532e0 lrc: 2/0,0 mode: PW/PW res: [0x42:0x0:0x0].0x0 rrc: 2 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x50010480000000 nid: local remote: 0x0 expref: -99 pid: 20762 timeout: 0 lvb_type: 0 00010000:00010000:2.0:1776178151.814498:0:20762:0:(ldlm_lockd.c:2159:__ldlm_bl_to_thread()) added 1/1 locks to regular blp list, 1 blwis in pool 00000100:00100000:3.0:1776178151.816953:0:3348:0:(events.c:361:request_in_callback()) peer: 12345-192.168.204.3@tcp (source: 12345-192.168.204.3@tcp) 00000100:00100000:3.0:1776178151.817206:0:3348:0:(events.c:361:request_in_callback()) peer: 12345-192.168.204.3@tcp (source: 12345-192.168.204.3@tcp) 00000100:00100000:2.0:1776178151.817227:0:20762:0:(service.c:2372:ptlrpc_server_handle_request()) Handled RPC req@000000003334dea7 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_013:lustre-MDT0000-mdtlov_UUID+9:3354:x1862456998033216:12345-0@lo:6:osp-syn-0-0.0 Request processed in 31876us (32277us total) trans 4294969349 rc 0/0 00000100:00100000:3.0:1776178151.817239:0:3348:0:(events.c:361:request_in_callback()) peer: 12345-192.168.204.3@tcp (source: 12345-192.168.204.3@tcp) 00000100:00100000:2.0:1776178151.817245:0:20762:0:(nrs_fifo.c:241:nrs_fifo_req_stop()) NRS stop fifo request from 12345-0@lo, seq: 5251 00000100:00100000:3.0:1776178151.817265:0:3348:0:(events.c:361:request_in_callback()) peer: 12345-192.168.204.3@tcp (source: 12345-192.168.204.3@tcp) 00010000:00010000:2.0:1776178151.817290:0:6244:0:(ldlm_lockd.c:2789:ldlm_bl_get_work()) Got 1 locks of 0 total in blp. (0 blwis in pool) 00000100:00100000:3.0:1776178151.817290:0:3348:0:(events.c:361:request_in_callback()) peer: 12345-192.168.204.3@tcp (source: 12345-192.168.204.3@tcp) 00010000:00010000:2.0:1776178151.817385:0:6244:0:(ldlm_lockd.c:1879:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 00000000b2e83690/0xa40083ab9bf532e0 lrc: 2/0,0 mode: PW/PW res: [0x42:0x0:0x0].0x0 rrc: 2 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x40010480000000 nid: local remote: 0x0 expref: -99 pid: 20762 timeout: 0 lvb_type: 0 00000100:00100000:3.0:1776178151.817387:0:3348:0:(events.c:361:request_in_callback()) peer: 12345-192.168.204.3@tcp (source: 12345-192.168.204.3@tcp) 00010000:00010000:2.0:1776178151.817401:0:6244:0:(ldlm_lockd.c:1893:ldlm_handle_bl_callback()) Lock 00000000b2e83690 already unused, calling callback (0000000063c97258) 00010000:00010000:2.0:1776178151.817405:0:6244:0:(ldlm_request.c:364:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 00000000b2e83690/0xa40083ab9bf532e0 lrc: 2/0,0 mode: PW/PW res: [0x42:0x0:0x0].0x0 rrc: 2 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x40010480000000 nid: local remote: 0x0 expref: -99 pid: 20762 timeout: 0 lvb_type: 0 00010000:00010000:2.0:1776178151.817417:0:6244:0:(ldlm_request.c:1257:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 00000000b2e83690/0xa40083ab9bf532e0 lrc: 3/0,0 mode: PW/PW res: [0x42:0x0:0x0].0x0 rrc: 2 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x40018480000000 nid: local remote: 0x0 expref: -99 pid: 20762 timeout: 0 lvb_type: 0 00000100:00100000:3.0:1776178151.817422:0:6241:0:(service.c:2152:ptlrpc_server_handle_req_in()) got req x1862456979089088 00010000:00010000:2.0:1776178151.817443:0:6244:0:(ldlm_lockd.c:1903:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 00000000b2e83690/0xa40083ab9bf532e0 lrc: 1/0,0 mode: --/PW res: [0x42:0x0:0x0].0x0 rrc: 2 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x44819480000000 nid: local remote: 0x0 expref: -99 pid: 20762 timeout: 0 lvb_type: 0 00010000:00010000:2.0:1776178151.817451:0:6244:0:(ldlm_lock.c:216:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 00000000b2e83690/0xa40083ab9bf532e0 lrc: 0/0,0 mode: --/PW res: [0x42:0x0:0x0].0x0 rrc: 2 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x44819480000000 nid: local remote: 0x0 expref: -99 pid: 20762 timeout: 0 lvb_type: 0 00010000:00010000:3.0:1776178151.817460:0:6241:0:(ldlm_lockd.c:2638:ldlm_cancel_hpreq_check()) ### hpreq cancel/convert lock ns: filter-lustre-OST0000_UUID lock: 00000000a14e680b/0xa40083ab9bf5309b lrc: 4/0,0 mode: PR/PR res: [0x45:0x0:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x60000400010020 nid: 192.168.204.3@tcp remote: 0x771f68aa1eb279f9 expref: 9 pid: 20762 timeout: 959 lvb_type: 1 00010000:00010000:2.0:1776178151.817484:0:6244:0:(ldlm_lockd.c:2792:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:3.0:1776178151.817486:0:6241:0:(service.c:2152:ptlrpc_server_handle_req_in()) got req x1862456979089216 00010000:00010000:2.0:1776178151.817487:0:6244:0:(ldlm_lockd.c:2792:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00010000:00010000:2.0:1776178151.817489:0:6244:0:(ldlm_lockd.c:2792:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00010000:00010000:3.0:1776178151.817499:0:6241:0:(ldlm_lockd.c:2638:ldlm_cancel_hpreq_check()) ### hpreq cancel/convert lock ns: filter-lustre-OST0000_UUID lock: 000000007f7ce364/0xa40083ab9bf53086 lrc: 4/0,0 mode: PR/PR res: [0x44:0x0:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x60000400010020 nid: 192.168.204.3@tcp remote: 0x771f68aa1eb279d6 expref: 10 pid: 14440 timeout: 959 lvb_type: 1 00000100:00100000:3.0:1776178151.817515:0:6241:0:(service.c:2152:ptlrpc_server_handle_req_in()) got req x1862456979088896 00010000:00010000:3.0:1776178151.817541:0:6241:0:(ldlm_lockd.c:2638:ldlm_cancel_hpreq_check()) ### hpreq cancel/convert lock ns: filter-lustre-OST0000_UUID lock: 000000002cc4819c/0xa40083ab9bf53094 lrc: 4/0,0 mode: PR/PR res: [0x43:0x0:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x60000400010020 nid: 192.168.204.3@tcp remote: 0x771f68aa1eb279f2 expref: 11 pid: 20762 timeout: 959 lvb_type: 1 00000100:00100000:3.0:1776178151.817554:0:6241:0:(service.c:2152:ptlrpc_server_handle_req_in()) got req x1862456979089152 00010000:00010000:3.0:1776178151.817563:0:6241:0:(ldlm_lockd.c:2638:ldlm_cancel_hpreq_check()) ### hpreq cancel/convert lock ns: filter-lustre-OST0001_UUID lock: 000000007797f5e2/0xa40083ab9bf530a2 lrc: 4/0,0 mode: PR/PR res: [0x44:0x0:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x60000400010020 nid: 192.168.204.3@tcp remote: 0x771f68aa1eb27a00 expref: 8 pid: 14440 timeout: 959 lvb_type: 1 00000100:00100000:3.0:1776178151.817579:0:6241:0:(service.c:2152:ptlrpc_server_handle_req_in()) got req x1862456979089024 00010000:00010000:3.0:1776178151.817586:0:6241:0:(ldlm_lockd.c:2638:ldlm_cancel_hpreq_check()) ### hpreq cancel/convert lock ns: filter-lustre-OST0001_UUID lock: 00000000daa24072/0xa40083ab9bf5308d lrc: 4/0,0 mode: PR/PR res: [0x43:0x0:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x60000400010020 nid: 192.168.204.3@tcp remote: 0x771f68aa1eb279eb expref: 9 pid: 20762 timeout: 959 lvb_type: 1 00000100:00100000:3.0:1776178151.817596:0:6241:0:(service.c:2152:ptlrpc_server_handle_req_in()) got req x1862456979089280 00010000:00010000:3.0:1776178151.817603:0:6241:0:(ldlm_lockd.c:2638:ldlm_cancel_hpreq_check()) ### hpreq cancel/convert lock ns: filter-lustre-OST0001_UUID lock: 000000004bd1d7cb/0xa40083ab9bf53032 lrc: 5/0,0 mode: PR/PR res: [0x42:0x0:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x60000400010020 nid: 192.168.204.3@tcp remote: 0x771f68aa1eb279b3 expref: 10 pid: 20762 timeout: 959 lvb_type: 1 00000100:00100000:3.0:1776178151.817616:0:6241:0:(nrs_fifo.c:179:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.204.3@tcp, seq: 2074 00000100:00100000:3.0:1776178151.817622:0:6241:0:(service.c:2324:ptlrpc_server_handle_request()) Handling RPC req@00000000f0ef812c pname:cluuid+ref:pid:xid:nid:opc:job ldlm_cn00_000:b50963bd-8eba-4432-9a54-8888d00e3fa1+11:2271:x1862456979089088:12345-192.168.204.3@tcp:103:ldlm_bl_02.0 00010000:00010000:3.0:1776178151.817714:0:6241:0:(ldlm_lockd.c:1720:ldlm_request_cancel()) ### server-side cancel handler START: 1 locks, starting at 0 00010000:00010000:3.0:1776178151.817720:0:6241:0:(ldlm_lockd.c:1771:ldlm_request_cancel()) ### server cancels blocked lock after 0s ns: filter-lustre-OST0000_UUID lock: 00000000a14e680b/0xa40083ab9bf5309b lrc: 4/0,0 mode: PR/PR res: [0x45:0x0:0x0].0x0 rrc: 4 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x60000400010020 nid: 192.168.204.3@tcp remote: 0x771f68aa1eb279f9 expref: 11 pid: 20762 timeout: 959 lvb_type: 1 00010000:00010000:3.0:1776178151.817732:0:6241:0:(ldlm_lockd.c:562:ldlm_del_waiting_lock()) ### removed ns: filter-lustre-OST0000_UUID lock: 00000000a14e680b/0xa40083ab9bf5309b lrc: 3/0,0 mode: PR/PR res: [0x45:0x0:0x0].0x0 rrc: 4 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x50000400010020 nid: 192.168.204.3@tcp remote: 0x771f68aa1eb279f9 expref: 11 pid: 20762 timeout: 959 lvb_type: 1 00010000:00010000:3.0:1776178151.817754:0:6241:0:(ldlm_lock.c:216:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 00000000a14e680b/0xa40083ab9bf5309b lrc: 0/0,0 mode: --/PR res: [0x45:0x0:0x0].0x0 rrc: 4 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x44801400010020 nid: 192.168.204.3@tcp remote: 0x771f68aa1eb279f9 expref: 11 pid: 20762 timeout: 959 lvb_type: 1 00010000:00010000:3.0:1776178151.817769:0:6241:0:(ldlm_extent.c:318:ldlm_check_contention()) contended locks = 0 00010000:00010000:3.0:1776178151.817772:0:6241:0:(ldlm_extent.c:318:ldlm_check_contention()) contended locks = 0 00010000:00010000:3.0:1776178151.817776:0:6241:0:(ldlm_lock.c:704:ldlm_add_cp_work_item()) ### lock granted; sending completion AST. ns: filter-lustre-OST0000_UUID lock: 00000000fcb32840/0xa40083ab9bf532f5 lrc: 3/0,1 mode: PW/PW res: [0x45:0x0:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x40090080000000 nid: local remote: 0x0 expref: -99 pid: 20755 timeout: 0 lvb_type: 0 00010000:00010000:3.0:1776178151.817785:0:6241:0:(ldlm_resource.c:1650:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 00000000fcb32840/0xa40083ab9bf532f5 lrc: 4/0,1 mode: PW/PW res: [0x45:0x0:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x40090080000000 nid: local remote: 0x0 expref: -99 pid: 20755 timeout: 0 lvb_type: 0 00010000:00010000:3.0:1776178151.817815:0:6241:0:(ldlm_lockd.c:1782:ldlm_request_cancel()) ### server-side cancel handler END 00000100:00100000:1.0:1776178151.817820:0:20759:0:(client.c:2215:ptlrpc_check_set()) Completed RPC req@0000000021d5d4e8 pname:cluuid:pid:xid:nid:opc:job ll_ost00_010:lustre-OST0001_UUID:20759:1862456998033408:192.168.204.3@tcp:104: 00010000:00010000:1.0:1776178151.817846:0:20759:0:(ldlm_request.c:283:ldlm_completion_ast()) ### client-side enqueue returned a blocked locksleeping ns: filter-lustre-OST0001_UUID lock: 0000000056462e0c/0xa40083ab9bf532d9 lrc: 3/0,1 mode: --/PW res: [0x42:0x0:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x40010080000000 nid: local remote: 0x0 expref: -99 pid: 20759 timeout: 0 lvb_type: 0 00000100:00100000:3.0:1776178151.817867:0:6241:0:(service.c:2372:ptlrpc_server_handle_request()) Handled RPC req@00000000f0ef812c pname:cluuid+ref:pid:xid:nid:opc:job ldlm_cn00_000:b50963bd-8eba-4432-9a54-8888d00e3fa1+10:2271:x1862456979089088:12345-192.168.204.3@tcp:103:ldlm_bl_02.0 Request processed in 243us (922us total) trans 0 rc 0/0 00000100:00100000:3.0:1776178151.817877:0:6241:0:(nrs_fifo.c:241:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.204.3@tcp, seq: 2074 00000100:00100000:3.0:1776178151.817901:0:6241:0:(nrs_fifo.c:179:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.204.3@tcp, seq: 2075 00000100:00100000:3.0:1776178151.817906:0:6241:0:(service.c:2324:ptlrpc_server_handle_request()) Handling RPC req@000000006da54a8b pname:cluuid+ref:pid:xid:nid:opc:job ldlm_cn00_000:b50963bd-8eba-4432-9a54-8888d00e3fa1+9:2270:x1862456979089216:12345-192.168.204.3@tcp:103:ldlm_bl_02.0 00010000:00010000:3.0:1776178151.817915:0:6241:0:(ldlm_lockd.c:1720:ldlm_request_cancel()) ### server-side cancel handler START: 1 locks, starting at 0 00010000:00010000:3.0:1776178151.817919:0:6241:0:(ldlm_lockd.c:1771:ldlm_request_cancel()) ### server cancels blocked lock after 0s ns: filter-lustre-OST0000_UUID lock: 000000007f7ce364/0xa40083ab9bf53086 lrc: 4/0,0 mode: PR/PR res: [0x44:0x0:0x0].0x0 rrc: 4 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x60000400010020 nid: 192.168.204.3@tcp remote: 0x771f68aa1eb279d6 expref: 9 pid: 14440 timeout: 959 lvb_type: 1 00010000:00010000:3.0:1776178151.817927:0:6241:0:(ldlm_lockd.c:562:ldlm_del_waiting_lock()) ### removed ns: filter-lustre-OST0000_UUID lock: 000000007f7ce364/0xa40083ab9bf53086 lrc: 3/0,0 mode: PR/PR res: [0x44:0x0:0x0].0x0 rrc: 4 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x50000400010020 nid: 192.168.204.3@tcp remote: 0x771f68aa1eb279d6 expref: 9 pid: 14440 timeout: 959 lvb_type: 1 00010000:00010000:3.0:1776178151.817941:0:6241:0:(ldlm_lock.c:216:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 000000007f7ce364/0xa40083ab9bf53086 lrc: 0/0,0 mode: --/PR res: [0x44:0x0:0x0].0x0 rrc: 4 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x44801400010020 nid: 192.168.204.3@tcp remote: 0x771f68aa1eb279d6 expref: 9 pid: 14440 timeout: 959 lvb_type: 1 00010000:00010000:3.0:1776178151.817950:0:6241:0:(ldlm_extent.c:318:ldlm_check_contention()) contended locks = 0 00010000:00010000:3.0:1776178151.817951:0:6241:0:(ldlm_extent.c:318:ldlm_check_contention()) contended locks = 0 00000004:00080000:2.0:1776178151.817953:0:3354:0:(osp_sync.c:562:osp_sync_interpret()) reply req 000000005643ab1b/2, rc 0, transno 2053 00010000:00010000:3.0:1776178151.817954:0:6241:0:(ldlm_lock.c:704:ldlm_add_cp_work_item()) ### lock granted; sending completion AST. ns: filter-lustre-OST0000_UUID lock: 000000009f2de1df/0xa40083ab9bf53303 lrc: 3/0,1 mode: PW/PW res: [0x44:0x0:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x40090080000000 nid: local remote: 0x0 expref: -99 pid: 20769 timeout: 0 lvb_type: 0 00000100:00100000:2.0:1776178151.817964:0:3354:0:(client.c:2215:ptlrpc_check_set()) Completed RPC req@000000005643ab1b pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_01:lustre-MDT0000-mdtlov_UUID:3354:1862456998033216:0@lo:6:osp-syn-0-0.0 00010000:00010000:3.0:1776178151.817964:0:6241:0:(ldlm_resource.c:1650:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 000000009f2de1df/0xa40083ab9bf53303 lrc: 4/0,1 mode: PW/PW res: [0x44:0x0:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x40090080000000 nid: local remote: 0x0 expref: -99 pid: 20769 timeout: 0 lvb_type: 0 00010000:00010000:3.0:1776178151.817996:0:6241:0:(ldlm_lockd.c:1782:ldlm_request_cancel()) ### server-side cancel handler END 00000100:00100000:2.0:1776178151.818000:0:6243:0:(nrs_fifo.c:179:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.204.3@tcp, seq: 2076 00000100:00100000:2.0:1776178151.818008:0:6243:0:(service.c:2324:ptlrpc_server_handle_request()) Handling RPC req@0000000089632c95 pname:cluuid+ref:pid:xid:nid:opc:job ldlm_cn00_002:b50963bd-8eba-4432-9a54-8888d00e3fa1+8:2273:x1862456979088896:12345-192.168.204.3@tcp:103:ldlm_bl_03.0 00000100:00100000:3.0:1776178151.818018:0:6241:0:(service.c:2372:ptlrpc_server_handle_request()) Handled RPC req@000000006da54a8b pname:cluuid+ref:pid:xid:nid:opc:job ldlm_cn00_000:b50963bd-8eba-4432-9a54-8888d00e3fa1+8:2270:x1862456979089216:12345-192.168.204.3@tcp:103:ldlm_bl_02.0 Request processed in 110us (812us total) trans 0 rc 0/0 00010000:00010000:2.0:1776178151.818020:0:6243:0:(ldlm_lockd.c:1720:ldlm_request_cancel()) ### server-side cancel handler START: 1 locks, starting at 0 00010000:00010000:2.0:1776178151.818028:0:6243:0:(ldlm_lockd.c:1771:ldlm_request_cancel()) ### server cancels blocked lock after 0s ns: filter-lustre-OST0000_UUID lock: 000000002cc4819c/0xa40083ab9bf53094 lrc: 4/0,0 mode: PR/PR res: [0x43:0x0:0x0].0x0 rrc: 4 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x60000400010020 nid: 192.168.204.3@tcp remote: 0x771f68aa1eb279f2 expref: 8 pid: 20762 timeout: 959 lvb_type: 1 00000100:00100000:3.0:1776178151.818031:0:6241:0:(nrs_fifo.c:241:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.204.3@tcp, seq: 2075 00010000:00010000:2.0:1776178151.818043:0:6243:0:(ldlm_lockd.c:562:ldlm_del_waiting_lock()) ### removed ns: filter-lustre-OST0000_UUID lock: 000000002cc4819c/0xa40083ab9bf53094 lrc: 3/0,0 mode: PR/PR res: [0x43:0x0:0x0].0x0 rrc: 4 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x50000400010020 nid: 192.168.204.3@tcp remote: 0x771f68aa1eb279f2 expref: 7 pid: 20762 timeout: 959 lvb_type: 1 00000100:00100000:3.0:1776178151.818051:0:6241:0:(nrs_fifo.c:179:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.204.3@tcp, seq: 2077 00000100:00100000:3.0:1776178151.818085:0:6241:0:(service.c:2324:ptlrpc_server_handle_request()) Handling RPC req@00000000df370bee pname:cluuid+ref:pid:xid:nid:opc:job ldlm_cn00_000:b50963bd-8eba-4432-9a54-8888d00e3fa1+10:2273:x1862456979089152:12345-192.168.204.3@tcp:103:ldlm_bl_03.0 00010000:00010000:2.0:1776178151.818090:0:6243:0:(ldlm_lock.c:216:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 000000002cc4819c/0xa40083ab9bf53094 lrc: 0/0,0 mode: --/PR res: [0x43:0x0:0x0].0x0 rrc: 4 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x44801400010020 nid: 192.168.204.3@tcp remote: 0x771f68aa1eb279f2 expref: 7 pid: 20762 timeout: 959 lvb_type: 1 00010000:00010000:3.0:1776178151.818098:0:6241:0:(ldlm_lockd.c:1720:ldlm_request_cancel()) ### server-side cancel handler START: 1 locks, starting at 0 00010000:00010000:2.0:1776178151.818104:0:6243:0:(ldlm_extent.c:318:ldlm_check_contention()) contended locks = 0 00010000:00010000:3.0:1776178151.818105:0:6241:0:(ldlm_lockd.c:1771:ldlm_request_cancel()) ### server cancels blocked lock after 0s ns: filter-lustre-OST0001_UUID lock: 000000007797f5e2/0xa40083ab9bf530a2 lrc: 4/0,0 mode: PR/PR res: [0x44:0x0:0x0].0x0 rrc: 4 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x60000400010020 nid: 192.168.204.3@tcp remote: 0x771f68aa1eb27a00 expref: 10 pid: 14440 timeout: 959 lvb_type: 1 00010000:00010000:2.0:1776178151.818107:0:6243:0:(ldlm_extent.c:318:ldlm_check_contention()) contended locks = 0 00010000:00010000:2.0:1776178151.818110:0:6243:0:(ldlm_lock.c:704:ldlm_add_cp_work_item()) ### lock granted; sending completion AST. ns: filter-lustre-OST0000_UUID lock: 000000006b0afb52/0xa40083ab9bf532e7 lrc: 3/0,1 mode: PW/PW res: [0x43:0x0:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x40090080000000 nid: local remote: 0x0 expref: -99 pid: 8227 timeout: 0 lvb_type: 0 00010000:00010000:2.0:1776178151.818122:0:6243:0:(ldlm_resource.c:1650:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 000000006b0afb52/0xa40083ab9bf532e7 lrc: 4/0,1 mode: PW/PW res: [0x43:0x0:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x40090080000000 nid: local remote: 0x0 expref: -99 pid: 8227 timeout: 0 lvb_type: 0 00010000:00010000:3.0:1776178151.818123:0:6241:0:(ldlm_lockd.c:562:ldlm_del_waiting_lock()) ### removed ns: filter-lustre-OST0001_UUID lock: 000000007797f5e2/0xa40083ab9bf530a2 lrc: 3/0,0 mode: PR/PR res: [0x44:0x0:0x0].0x0 rrc: 4 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x50000400010020 nid: 192.168.204.3@tcp remote: 0x771f68aa1eb27a00 expref: 10 pid: 14440 timeout: 959 lvb_type: 1 00010000:00010000:3.0:1776178151.818156:0:6241:0:(ldlm_lock.c:216:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0001_UUID lock: 000000007797f5e2/0xa40083ab9bf530a2 lrc: 0/0,0 mode: --/PR res: [0x44:0x0:0x0].0x0 rrc: 4 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x44801400010020 nid: 192.168.204.3@tcp remote: 0x771f68aa1eb27a00 expref: 10 pid: 14440 timeout: 959 lvb_type: 1 00010000:00010000:2.0:1776178151.818157:0:6243:0:(ldlm_lockd.c:1782:ldlm_request_cancel()) ### server-side cancel handler END 00010000:00010000:3.0:1776178151.818169:0:6241:0:(ldlm_extent.c:318:ldlm_check_contention()) contended locks = 0 00010000:00010000:3.0:1776178151.818171:0:6241:0:(ldlm_extent.c:318:ldlm_check_contention()) contended locks = 0 00010000:00010000:3.0:1776178151.818174:0:6241:0:(ldlm_lock.c:704:ldlm_add_cp_work_item()) ### lock granted; sending completion AST. ns: filter-lustre-OST0001_UUID lock: 000000001b3f1324/0xa40083ab9bf532fc lrc: 3/0,1 mode: PW/PW res: [0x44:0x0:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x40090080000000 nid: local remote: 0x0 expref: -99 pid: 20765 timeout: 0 lvb_type: 0 00010000:00010000:3.0:1776178151.818181:0:6241:0:(ldlm_resource.c:1650:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0001_UUID lock: 000000001b3f1324/0xa40083ab9bf532fc lrc: 4/0,1 mode: PW/PW res: [0x44:0x0:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x40090080000000 nid: local remote: 0x0 expref: -99 pid: 20765 timeout: 0 lvb_type: 0 00000100:00100000:2.0:1776178151.818184:0:6243:0:(service.c:2372:ptlrpc_server_handle_request()) Handled RPC req@0000000089632c95 pname:cluuid+ref:pid:xid:nid:opc:job ldlm_cn00_002:b50963bd-8eba-4432-9a54-8888d00e3fa1+6:2273:x1862456979088896:12345-192.168.204.3@tcp:103:ldlm_bl_03.0 Request processed in 174us (944us total) trans 0 rc 0/0 00000100:00100000:2.0:1776178151.818199:0:6243:0:(nrs_fifo.c:241:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.204.3@tcp, seq: 2076 00010000:00010000:3.0:1776178151.818212:0:6241:0:(ldlm_lockd.c:1782:ldlm_request_cancel()) ### server-side cancel handler END 00000100:00100000:2.0:1776178151.818218:0:6243:0:(nrs_fifo.c:179:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.204.3@tcp, seq: 2078 00000100:00100000:2.0:1776178151.818223:0:6243:0:(service.c:2324:ptlrpc_server_handle_request()) Handling RPC req@0000000095c019b3 pname:cluuid+ref:pid:xid:nid:opc:job ldlm_cn00_002:b50963bd-8eba-4432-9a54-8888d00e3fa1+9:2273:x1862456979089024:12345-192.168.204.3@tcp:103:ldlm_bl_03.0 00010000:00010000:2.0:1776178151.818232:0:6243:0:(ldlm_lockd.c:1720:ldlm_request_cancel()) ### server-side cancel handler START: 1 locks, starting at 0 00010000:00010000:2.0:1776178151.818237:0:6243:0:(ldlm_lockd.c:1771:ldlm_request_cancel()) ### server cancels blocked lock after 0s ns: filter-lustre-OST0001_UUID lock: 00000000daa24072/0xa40083ab9bf5308d lrc: 4/0,0 mode: PR/PR res: [0x43:0x0:0x0].0x0 rrc: 4 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x60000400010020 nid: 192.168.204.3@tcp remote: 0x771f68aa1eb279eb expref: 9 pid: 20762 timeout: 959 lvb_type: 1 00000100:00100000:3.0:1776178151.818239:0:6241:0:(service.c:2372:ptlrpc_server_handle_request()) Handled RPC req@00000000df370bee pname:cluuid+ref:pid:xid:nid:opc:job ldlm_cn00_000:b50963bd-8eba-4432-9a54-8888d00e3fa1+9:2273:x1862456979089152:12345-192.168.204.3@tcp:103:ldlm_bl_03.0 Request processed in 178us (973us total) trans 0 rc 0/0 00000100:00100000:3.0:1776178151.818255:0:6241:0:(nrs_fifo.c:241:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.204.3@tcp, seq: 2077 00010000:00010000:2.0:1776178151.818257:0:6243:0:(ldlm_lockd.c:562:ldlm_del_waiting_lock()) ### removed ns: filter-lustre-OST0001_UUID lock: 00000000daa24072/0xa40083ab9bf5308d lrc: 3/0,0 mode: PR/PR res: [0x43:0x0:0x0].0x0 rrc: 4 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x50000400010020 nid: 192.168.204.3@tcp remote: 0x771f68aa1eb279eb expref: 9 pid: 20762 timeout: 959 lvb_type: 1 00000100:00100000:3.0:1776178151.818276:0:6241:0:(nrs_fifo.c:179:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.204.3@tcp, seq: 2079 00010000:00010000:2.0:1776178151.818278:0:6243:0:(ldlm_lock.c:216:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0001_UUID lock: 00000000daa24072/0xa40083ab9bf5308d lrc: 0/0,0 mode: --/PR res: [0x43:0x0:0x0].0x0 rrc: 4 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x44801400010020 nid: 192.168.204.3@tcp remote: 0x771f68aa1eb279eb expref: 8 pid: 20762 timeout: 959 lvb_type: 1 00000100:00100000:3.0:1776178151.818287:0:6241:0:(service.c:2324:ptlrpc_server_handle_request()) Handling RPC req@000000008c469ec0 pname:cluuid+ref:pid:xid:nid:opc:job ldlm_cn00_000:b50963bd-8eba-4432-9a54-8888d00e3fa1+8:2270:x1862456979089280:12345-192.168.204.3@tcp:103:ldlm_bl_03.0 00010000:00010000:2.0:1776178151.818293:0:6243:0:(ldlm_extent.c:318:ldlm_check_contention()) contended locks = 0 00010000:00010000:3.0:1776178151.818297:0:6241:0:(ldlm_lockd.c:1720:ldlm_request_cancel()) ### server-side cancel handler START: 1 locks, starting at 0 00010000:00010000:3.0:1776178151.818302:0:6241:0:(ldlm_lockd.c:1771:ldlm_request_cancel()) ### server cancels blocked lock after 0s ns: filter-lustre-OST0001_UUID lock: 000000004bd1d7cb/0xa40083ab9bf53032 lrc: 4/0,0 mode: PR/PR res: [0x42:0x0:0x0].0x0 rrc: 4 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x60000400010020 nid: 192.168.204.3@tcp remote: 0x771f68aa1eb279b3 expref: 7 pid: 20762 timeout: 959 lvb_type: 1 00010000:00010000:3.0:1776178151.818312:0:6241:0:(ldlm_lockd.c:562:ldlm_del_waiting_lock()) ### removed ns: filter-lustre-OST0001_UUID lock: 000000004bd1d7cb/0xa40083ab9bf53032 lrc: 3/0,0 mode: PR/PR res: [0x42:0x0:0x0].0x0 rrc: 4 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x50000400010020 nid: 192.168.204.3@tcp remote: 0x771f68aa1eb279b3 expref: 7 pid: 20762 timeout: 959 lvb_type: 1 00010000:00010000:3.0:1776178151.818326:0:6241:0:(ldlm_lock.c:216:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0001_UUID lock: 000000004bd1d7cb/0xa40083ab9bf53032 lrc: 0/0,0 mode: --/PR res: [0x42:0x0:0x0].0x0 rrc: 4 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x44801400010020 nid: 192.168.204.3@tcp remote: 0x771f68aa1eb279b3 expref: 7 pid: 20762 timeout: 959 lvb_type: 1 00010000:00010000:3.0:1776178151.818336:0:6241:0:(ldlm_extent.c:318:ldlm_check_contention()) contended locks = 0 00010000:00010000:3.0:1776178151.818337:0:6241:0:(ldlm_extent.c:318:ldlm_check_contention()) contended locks = 0 00010000:00010000:3.0:1776178151.818339:0:6241:0:(ldlm_lock.c:704:ldlm_add_cp_work_item()) ### lock granted; sending completion AST. ns: filter-lustre-OST0001_UUID lock: 0000000056462e0c/0xa40083ab9bf532d9 lrc: 3/0,1 mode: PW/PW res: [0x42:0x0:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x40090080000000 nid: local remote: 0x0 expref: -99 pid: 20759 timeout: 0 lvb_type: 0 00010000:00010000:3.0:1776178151.818345:0:6241:0:(ldlm_resource.c:1650:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0001_UUID lock: 0000000056462e0c/0xa40083ab9bf532d9 lrc: 4/0,1 mode: PW/PW res: [0x42:0x0:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x40090080000000 nid: local remote: 0x0 expref: -99 pid: 20759 timeout: 0 lvb_type: 0 00010000:00010000:3.0:1776178151.818368:0:6241:0:(ldlm_lockd.c:1782:ldlm_request_cancel()) ### server-side cancel handler END 00000100:00100000:3.0:1776178151.818388:0:6241:0:(service.c:2372:ptlrpc_server_handle_request()) Handled RPC req@000000008c469ec0 pname:cluuid+ref:pid:xid:nid:opc:job ldlm_cn00_000:b50963bd-8eba-4432-9a54-8888d00e3fa1+6:2270:x1862456979089280:12345-192.168.204.3@tcp:103:ldlm_bl_03.0 Request processed in 105us (1003us total) trans 0 rc 0/0 00000100:00100000:3.0:1776178151.818395:0:6241:0:(nrs_fifo.c:241:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.204.3@tcp, seq: 2079 00010000:00010000:2.0:1776178151.818450:0:6243:0:(ldlm_extent.c:318:ldlm_check_contention()) contended locks = 0 00010000:00010000:2.0:1776178151.818453:0:6243:0:(ldlm_lock.c:704:ldlm_add_cp_work_item()) ### lock granted; sending completion AST. ns: filter-lustre-OST0001_UUID lock: 000000006c464689/0xa40083ab9bf532ee lrc: 3/0,1 mode: PW/PW res: [0x43:0x0:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x40090080000000 nid: local remote: 0x0 expref: -99 pid: 8228 timeout: 0 lvb_type: 0 00010000:00010000:2.0:1776178151.818460:0:6243:0:(ldlm_resource.c:1650:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0001_UUID lock: 000000006c464689/0xa40083ab9bf532ee lrc: 4/0,1 mode: PW/PW res: [0x43:0x0:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x40090080000000 nid: local remote: 0x0 expref: -99 pid: 8228 timeout: 0 lvb_type: 0 00010000:00010000:1.0:1776178151.818470:0:20759:0:(ldlm_request.c:199:ldlm_completion_tail()) ### client-side enqueue: granted ns: filter-lustre-OST0001_UUID lock: 0000000056462e0c/0xa40083ab9bf532d9 lrc: 3/0,1 mode: PW/PW res: [0x42:0x0:0x0].0x0 rrc: 2 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x40010080000000 nid: local remote: 0x0 expref: -99 pid: 20759 timeout: 0 lvb_type: 0 00010000:00010000:1.0:1776178151.818482:0:20759:0:(ldlm_request.c:528:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0001_UUID lock: 0000000056462e0c/0xa40083ab9bf532d9 lrc: 3/0,1 mode: PW/PW res: [0x42:0x0:0x0].0x0 rrc: 2 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x40010080000000 nid: local remote: 0x0 expref: -99 pid: 20759 timeout: 0 lvb_type: 0 00010000:00010000:2.0:1776178151.818486:0:6243:0:(ldlm_lockd.c:1782:ldlm_request_cancel()) ### server-side cancel handler END 00010000:00010000:1.0:1776178151.818495:0:20759:0:(ldlm_lock.c:820:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0001_UUID lock: 0000000056462e0c/0xa40083ab9bf532d9 lrc: 3/0,1 mode: PW/PW res: [0x42:0x0:0x0].0x0 rrc: 2 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x50010080000000 nid: local remote: 0x0 expref: -99 pid: 20759 timeout: 0 lvb_type: 0 00000100:00100000:2.0:1776178151.818509:0:6243:0:(service.c:2372:ptlrpc_server_handle_request()) Handled RPC req@0000000095c019b3 pname:cluuid+ref:pid:xid:nid:opc:job ldlm_cn00_002:b50963bd-8eba-4432-9a54-8888d00e3fa1+5:2273:x1862456979089024:12345-192.168.204.3@tcp:103:ldlm_bl_03.0 Request processed in 281us (1216us total) trans 0 rc 0/0 00010000:00010000:1.0:1776178151.818512:0:20759:0:(ldlm_lock.c:879:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0001_UUID lock: 0000000056462e0c/0xa40083ab9bf532d9 lrc: 2/0,0 mode: PW/PW res: [0x42:0x0:0x0].0x0 rrc: 2 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x50010480000000 nid: local remote: 0x0 expref: -99 pid: 20759 timeout: 0 lvb_type: 0 00000100:00100000:2.0:1776178151.818523:0:6243:0:(nrs_fifo.c:241:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.204.3@tcp, seq: 2078 00010000:00010000:1.0:1776178151.818538:0:20759:0:(ldlm_lockd.c:2159:__ldlm_bl_to_thread()) added 1/1 locks to regular blp list, 1 blwis in pool 00010000:00010000:2.0:1776178151.818557:0:20765:0:(ldlm_request.c:199:ldlm_completion_tail()) ### client-side enqueue: granted ns: filter-lustre-OST0001_UUID lock: 000000001b3f1324/0xa40083ab9bf532fc lrc: 3/0,1 mode: PW/PW res: [0x44:0x0:0x0].0x0 rrc: 2 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x40010080000000 nid: local remote: 0x0 expref: -99 pid: 20765 timeout: 0 lvb_type: 0 00010000:00010000:2.0:1776178151.818889:0:20765:0:(ldlm_request.c:528:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0001_UUID lock: 000000001b3f1324/0xa40083ab9bf532fc lrc: 3/0,1 mode: PW/PW res: [0x44:0x0:0x0].0x0 rrc: 2 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x40010080000000 nid: local remote: 0x0 expref: -99 pid: 20765 timeout: 0 lvb_type: 0 00010000:00010000:2.0:1776178151.818897:0:20765:0:(ldlm_lock.c:820:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0001_UUID lock: 000000001b3f1324/0xa40083ab9bf532fc lrc: 3/0,1 mode: PW/PW res: [0x44:0x0:0x0].0x0 rrc: 2 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x50010080000000 nid: local remote: 0x0 expref: -99 pid: 20765 timeout: 0 lvb_type: 0 00010000:00010000:2.0:1776178151.818910:0:20765:0:(ldlm_lock.c:879:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0001_UUID lock: 000000001b3f1324/0xa40083ab9bf532fc lrc: 2/0,0 mode: PW/PW res: [0x44:0x0:0x0].0x0 rrc: 2 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x50010480000000 nid: local remote: 0x0 expref: -99 pid: 20765 timeout: 0 lvb_type: 0 00000100:00100000:1.0:1776178151.818920:0:20759:0:(service.c:2372:ptlrpc_server_handle_request()) Handled RPC req@00000000a7e4e4e8 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_010:lustre-MDT0000-mdtlov_UUID+8:3354:x1862456998032960:12345-0@lo:6:osp-syn-1-0.0 Request processed in 33823us (34029us total) trans 4294969321 rc 0/0 00000100:00100000:1.0:1776178151.818931:0:20759:0:(nrs_fifo.c:241:nrs_fifo_req_stop()) NRS stop fifo request from 12345-0@lo, seq: 5250 00010000:00010000:2.0:1776178151.818931:0:20765:0:(ldlm_lockd.c:2159:__ldlm_bl_to_thread()) added 1/2 locks to regular blp list, 2 blwis in pool 00000100:00100000:2.0:1776178151.819094:0:20765:0:(service.c:2372:ptlrpc_server_handle_request()) Handled RPC req@00000000699e66b5 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_016:lustre-MDT0000-mdtlov_UUID+8:3356:x1862456998033024:12345-0@lo:6:osp-syn-1-0.0 Request processed in 28202us (29584us total) trans 4294969322 rc 0/0 00000100:00100000:2.0:1776178151.819102:0:20765:0:(nrs_fifo.c:241:nrs_fifo_req_stop()) NRS stop fifo request from 12345-0@lo, seq: 5255 00000004:00080000:2.0:1776178151.819159:0:3354:0:(osp_sync.c:562:osp_sync_interpret()) reply req 000000009d5112f6/2, rc 0, transno 2025 00000100:00100000:2.0:1776178151.819165:0:3354:0:(client.c:2215:ptlrpc_check_set()) Completed RPC req@000000009d5112f6 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_01:lustre-MDT0000-mdtlov_UUID:3354:1862456998032960:0@lo:6:osp-syn-1-0.0 00010000:00010000:2.0:1776178151.819190:0:9225:0:(ldlm_lockd.c:2789:ldlm_bl_get_work()) Got 1 locks of 1 total in blp. (1 blwis in pool) 00010000:00010000:2.0:1776178151.819196:0:9225:0:(ldlm_lockd.c:1879:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0001_UUID lock: 0000000056462e0c/0xa40083ab9bf532d9 lrc: 2/0,0 mode: PW/PW res: [0x42:0x0:0x0].0x0 rrc: 2 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x40010480000000 nid: local remote: 0x0 expref: -99 pid: 20759 timeout: 0 lvb_type: 0 00010000:00010000:2.0:1776178151.819205:0:9225:0:(ldlm_lockd.c:1893:ldlm_handle_bl_callback()) Lock 0000000056462e0c already unused, calling callback (0000000063c97258) 00010000:00010000:2.0:1776178151.819210:0:9225:0:(ldlm_request.c:364:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0001_UUID lock: 0000000056462e0c/0xa40083ab9bf532d9 lrc: 2/0,0 mode: PW/PW res: [0x42:0x0:0x0].0x0 rrc: 2 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x40010480000000 nid: local remote: 0x0 expref: -99 pid: 20759 timeout: 0 lvb_type: 0 00010000:00010000:2.0:1776178151.819220:0:9225:0:(ldlm_request.c:1257:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0001_UUID lock: 0000000056462e0c/0xa40083ab9bf532d9 lrc: 3/0,0 mode: PW/PW res: [0x42:0x0:0x0].0x0 rrc: 2 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x40018480000000 nid: local remote: 0x0 expref: -99 pid: 20759 timeout: 0 lvb_type: 0 00010000:00010000:2.0:1776178151.819238:0:9225:0:(ldlm_lockd.c:1903:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0001_UUID lock: 0000000056462e0c/0xa40083ab9bf532d9 lrc: 1/0,0 mode: --/PW res: [0x42:0x0:0x0].0x0 rrc: 2 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x44819480000000 nid: local remote: 0x0 expref: -99 pid: 20759 timeout: 0 lvb_type: 0 00010000:00010000:2.0:1776178151.819245:0:9225:0:(ldlm_lock.c:216:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0001_UUID lock: 0000000056462e0c/0xa40083ab9bf532d9 lrc: 0/0,0 mode: --/PW res: [0x42:0x0:0x0].0x0 rrc: 2 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x44819480000000 nid: local remote: 0x0 expref: -99 pid: 20759 timeout: 0 lvb_type: 0 00000100:00100000:0.0:1776178151.819255:0:20767:0:(service.c:2372:ptlrpc_server_handle_request()) Handled RPC req@000000005927b2c3 pname:cluuid+ref:pid:xid:nid:opc:job ldlm_cn00_004:b50963bd-8eba-4432-9a54-8888d00e3fa1+5:2271:x1862456979088960:12345-192.168.204.3@tcp:103:ldlm_bl_02.0 Request processed in 5004us (5236us total) trans 0 rc 0/0 00000100:00100000:0.0:1776178151.819270:0:20767:0:(nrs_fifo.c:241:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.204.3@tcp, seq: 2073 00010000:00010000:2.0:1776178151.819281:0:9225:0:(ldlm_lockd.c:2789:ldlm_bl_get_work()) Got 1 locks of 0 total in blp. (0 blwis in pool) 00010000:00010000:2.0:1776178151.819285:0:9225:0:(ldlm_lockd.c:1879:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0001_UUID lock: 000000001b3f1324/0xa40083ab9bf532fc lrc: 2/0,0 mode: PW/PW res: [0x44:0x0:0x0].0x0 rrc: 2 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x40010480000000 nid: local remote: 0x0 expref: -99 pid: 20765 timeout: 0 lvb_type: 0 00010000:00010000:2.0:1776178151.819292:0:9225:0:(ldlm_lockd.c:1893:ldlm_handle_bl_callback()) Lock 000000001b3f1324 already unused, calling callback (0000000063c97258) 00010000:00010000:2.0:1776178151.819510:0:9225:0:(ldlm_request.c:364:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0001_UUID lock: 000000001b3f1324/0xa40083ab9bf532fc lrc: 2/0,0 mode: PW/PW res: [0x44:0x0:0x0].0x0 rrc: 2 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x40010480000000 nid: local remote: 0x0 expref: -99 pid: 20765 timeout: 0 lvb_type: 0 00010000:00010000:2.0:1776178151.819521:0:9225:0:(ldlm_request.c:1257:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0001_UUID lock: 000000001b3f1324/0xa40083ab9bf532fc lrc: 3/0,0 mode: PW/PW res: [0x44:0x0:0x0].0x0 rrc: 2 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x40018480000000 nid: local remote: 0x0 expref: -99 pid: 20765 timeout: 0 lvb_type: 0 00010000:00010000:2.0:1776178151.819532:0:9225:0:(ldlm_lockd.c:1903:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0001_UUID lock: 000000001b3f1324/0xa40083ab9bf532fc lrc: 1/0,0 mode: --/PW res: [0x44:0x0:0x0].0x0 rrc: 2 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x44819480000000 nid: local remote: 0x0 expref: -99 pid: 20765 timeout: 0 lvb_type: 0 00010000:00010000:2.0:1776178151.819538:0:9225:0:(ldlm_lock.c:216:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0001_UUID lock: 000000001b3f1324/0xa40083ab9bf532fc lrc: 0/0,0 mode: --/PW res: [0x44:0x0:0x0].0x0 rrc: 2 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x44819480000000 nid: local remote: 0x0 expref: -99 pid: 20765 timeout: 0 lvb_type: 0 00010000:00010000:2.0:1776178151.819755:0:9225:0:(ldlm_lockd.c:2792:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00010000:00010000:2.0:1776178151.819757:0:9225:0:(ldlm_lockd.c:2792:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00010000:00010000:2.0:1776178151.819758:0:9225:0:(ldlm_lockd.c:2792:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000004:00080000:2.0:1776178151.819787:0:3356:0:(osp_sync.c:562:osp_sync_interpret()) reply req 0000000067671fc8/2, rc 0, transno 2026 00000100:00100000:2.0:1776178151.819791:0:3356:0:(client.c:2215:ptlrpc_check_set()) Completed RPC req@0000000067671fc8 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_03:lustre-MDT0000-mdtlov_UUID:3356:1862456998033024:0@lo:6:osp-syn-1-0.0 00010000:00010000:2.0:1776178151.819810:0:20755:0:(ldlm_request.c:199:ldlm_completion_tail()) ### client-side enqueue: granted ns: filter-lustre-OST0000_UUID lock: 00000000fcb32840/0xa40083ab9bf532f5 lrc: 3/0,1 mode: PW/PW res: [0x45:0x0:0x0].0x0 rrc: 2 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x40010080000000 nid: local remote: 0x0 expref: -99 pid: 20755 timeout: 0 lvb_type: 0 00010000:00010000:2.0:1776178151.819818:0:20755:0:(ldlm_request.c:528:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 00000000fcb32840/0xa40083ab9bf532f5 lrc: 3/0,1 mode: PW/PW res: [0x45:0x0:0x0].0x0 rrc: 2 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x40010080000000 nid: local remote: 0x0 expref: -99 pid: 20755 timeout: 0 lvb_type: 0 00010000:00010000:2.0:1776178151.819831:0:20755:0:(ldlm_lock.c:820:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0000_UUID lock: 00000000fcb32840/0xa40083ab9bf532f5 lrc: 3/0,1 mode: PW/PW res: [0x45:0x0:0x0].0x0 rrc: 2 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x50010080000000 nid: local remote: 0x0 expref: -99 pid: 20755 timeout: 0 lvb_type: 0 00010000:00010000:2.0:1776178151.819838:0:20755:0:(ldlm_lock.c:879:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 00000000fcb32840/0xa40083ab9bf532f5 lrc: 2/0,0 mode: PW/PW res: [0x45:0x0:0x0].0x0 rrc: 2 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x50010480000000 nid: local remote: 0x0 expref: -99 pid: 20755 timeout: 0 lvb_type: 0 00010000:00010000:2.0:1776178151.819857:0:20755:0:(ldlm_lockd.c:2159:__ldlm_bl_to_thread()) added 1/1 locks to regular blp list, 1 blwis in pool 00000100:00100000:2.0:1776178151.820013:0:20755:0:(service.c:2372:ptlrpc_server_handle_request()) Handled RPC req@0000000022d4dc29 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_006:lustre-MDT0000-mdtlov_UUID+9:3356:x1862456998033344:12345-0@lo:6:osp-syn-0-0.0 Request processed in 29278us (30570us total) trans 4294969350 rc 0/0 00000100:00100000:2.0:1776178151.820021:0:20755:0:(nrs_fifo.c:241:nrs_fifo_req_stop()) NRS stop fifo request from 12345-0@lo, seq: 5254 00010000:00010000:2.0:1776178151.820040:0:9224:0:(ldlm_lockd.c:2789:ldlm_bl_get_work()) Got 1 locks of 0 total in blp. (0 blwis in pool) 00010000:00010000:2.0:1776178151.820043:0:9224:0:(ldlm_lockd.c:1879:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 00000000fcb32840/0xa40083ab9bf532f5 lrc: 2/0,0 mode: PW/PW res: [0x45:0x0:0x0].0x0 rrc: 2 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x40010480000000 nid: local remote: 0x0 expref: -99 pid: 20755 timeout: 0 lvb_type: 0 00010000:00010000:2.0:1776178151.820050:0:9224:0:(ldlm_lockd.c:1893:ldlm_handle_bl_callback()) Lock 00000000fcb32840 already unused, calling callback (0000000063c97258) 00010000:00010000:2.0:1776178151.820052:0:9224:0:(ldlm_request.c:364:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 00000000fcb32840/0xa40083ab9bf532f5 lrc: 2/0,0 mode: PW/PW res: [0x45:0x0:0x0].0x0 rrc: 2 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x40010480000000 nid: local remote: 0x0 expref: -99 pid: 20755 timeout: 0 lvb_type: 0 00010000:00010000:2.0:1776178151.820122:0:9224:0:(ldlm_request.c:1257:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 00000000fcb32840/0xa40083ab9bf532f5 lrc: 3/0,0 mode: PW/PW res: [0x45:0x0:0x0].0x0 rrc: 2 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x40018480000000 nid: local remote: 0x0 expref: -99 pid: 20755 timeout: 0 lvb_type: 0 00010000:00010000:2.0:1776178151.820135:0:9224:0:(ldlm_lockd.c:1903:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 00000000fcb32840/0xa40083ab9bf532f5 lrc: 1/0,0 mode: --/PW res: [0x45:0x0:0x0].0x0 rrc: 2 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x44819480000000 nid: local remote: 0x0 expref: -99 pid: 20755 timeout: 0 lvb_type: 0 00010000:00010000:2.0:1776178151.820141:0:9224:0:(ldlm_lock.c:216:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 00000000fcb32840/0xa40083ab9bf532f5 lrc: 0/0,0 mode: --/PW res: [0x45:0x0:0x0].0x0 rrc: 2 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x44819480000000 nid: local remote: 0x0 expref: -99 pid: 20755 timeout: 0 lvb_type: 0 00010000:00010000:2.0:1776178151.820159:0:9224:0:(ldlm_lockd.c:2792:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00010000:00010000:2.0:1776178151.820160:0:9224:0:(ldlm_lockd.c:2792:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00010000:00010000:2.0:1776178151.820162:0:9224:0:(ldlm_lockd.c:2792:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000004:00080000:2.0:1776178151.820184:0:3356:0:(osp_sync.c:562:osp_sync_interpret()) reply req 00000000a59bfc89/2, rc 0, transno 2054 00000100:00100000:2.0:1776178151.820189:0:3356:0:(client.c:2215:ptlrpc_check_set()) Completed RPC req@00000000a59bfc89 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_03:lustre-MDT0000-mdtlov_UUID:3356:1862456998033344:0@lo:6:osp-syn-0-0.0 00010000:00010000:2.0:1776178151.820202:0:20769:0:(ldlm_request.c:199:ldlm_completion_tail()) ### client-side enqueue: granted ns: filter-lustre-OST0000_UUID lock: 000000009f2de1df/0xa40083ab9bf53303 lrc: 3/0,1 mode: PW/PW res: [0x44:0x0:0x0].0x0 rrc: 2 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x40010080000000 nid: local remote: 0x0 expref: -99 pid: 20769 timeout: 0 lvb_type: 0 00010000:00010000:2.0:1776178151.820209:0:20769:0:(ldlm_request.c:528:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 000000009f2de1df/0xa40083ab9bf53303 lrc: 3/0,1 mode: PW/PW res: [0x44:0x0:0x0].0x0 rrc: 2 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x40010080000000 nid: local remote: 0x0 expref: -99 pid: 20769 timeout: 0 lvb_type: 0 00010000:00010000:2.0:1776178151.820480:0:20769:0:(ldlm_lock.c:820:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0000_UUID lock: 000000009f2de1df/0xa40083ab9bf53303 lrc: 3/0,1 mode: PW/PW res: [0x44:0x0:0x0].0x0 rrc: 2 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x50010080000000 nid: local remote: 0x0 expref: -99 pid: 20769 timeout: 0 lvb_type: 0 00010000:00010000:2.0:1776178151.820487:0:20769:0:(ldlm_lock.c:879:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 000000009f2de1df/0xa40083ab9bf53303 lrc: 2/0,0 mode: PW/PW res: [0x44:0x0:0x0].0x0 rrc: 2 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x50010480000000 nid: local remote: 0x0 expref: -99 pid: 20769 timeout: 0 lvb_type: 0 00010000:00010000:2.0:1776178151.820501:0:20769:0:(ldlm_lockd.c:2159:__ldlm_bl_to_thread()) added 1/1 locks to regular blp list, 1 blwis in pool 00000100:00100000:2.0:1776178151.820765:0:20769:0:(service.c:2372:ptlrpc_server_handle_request()) Handled RPC req@0000000056ddcb09 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_018:lustre-MDT0000-mdtlov_UUID+9:3356:x1862456998033280:12345-0@lo:6:osp-syn-0-0.0 Request processed in 29573us (31217us total) trans 4294969351 rc 0/0 00000100:00100000:2.0:1776178151.820773:0:20769:0:(nrs_fifo.c:241:nrs_fifo_req_stop()) NRS stop fifo request from 12345-0@lo, seq: 5256 00010000:00010000:2.0:1776178151.820793:0:6244:0:(ldlm_lockd.c:2789:ldlm_bl_get_work()) Got 1 locks of 0 total in blp. (0 blwis in pool) 00010000:00010000:2.0:1776178151.820796:0:6244:0:(ldlm_lockd.c:1879:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 000000009f2de1df/0xa40083ab9bf53303 lrc: 2/0,0 mode: PW/PW res: [0x44:0x0:0x0].0x0 rrc: 2 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x40010480000000 nid: local remote: 0x0 expref: -99 pid: 20769 timeout: 0 lvb_type: 0 00010000:00010000:2.0:1776178151.820804:0:6244:0:(ldlm_lockd.c:1893:ldlm_handle_bl_callback()) Lock 000000009f2de1df already unused, calling callback (0000000063c97258) 00010000:00010000:2.0:1776178151.820806:0:6244:0:(ldlm_request.c:364:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 000000009f2de1df/0xa40083ab9bf53303 lrc: 2/0,0 mode: PW/PW res: [0x44:0x0:0x0].0x0 rrc: 2 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x40010480000000 nid: local remote: 0x0 expref: -99 pid: 20769 timeout: 0 lvb_type: 0 00010000:00010000:2.0:1776178151.820812:0:6244:0:(ldlm_request.c:1257:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 000000009f2de1df/0xa40083ab9bf53303 lrc: 3/0,0 mode: PW/PW res: [0x44:0x0:0x0].0x0 rrc: 2 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x40018480000000 nid: local remote: 0x0 expref: -99 pid: 20769 timeout: 0 lvb_type: 0 00010000:00010000:2.0:1776178151.820824:0:6244:0:(ldlm_lockd.c:1903:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 000000009f2de1df/0xa40083ab9bf53303 lrc: 1/0,0 mode: --/PW res: [0x44:0x0:0x0].0x0 rrc: 2 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x44819480000000 nid: local remote: 0x0 expref: -99 pid: 20769 timeout: 0 lvb_type: 0 00010000:00010000:2.0:1776178151.820830:0:6244:0:(ldlm_lock.c:216:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 000000009f2de1df/0xa40083ab9bf53303 lrc: 0/0,0 mode: --/PW res: [0x44:0x0:0x0].0x0 rrc: 2 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x44819480000000 nid: local remote: 0x0 expref: -99 pid: 20769 timeout: 0 lvb_type: 0 00010000:00010000:2.0:1776178151.820844:0:6244:0:(ldlm_lockd.c:2792:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00010000:00010000:2.0:1776178151.820846:0:6244:0:(ldlm_lockd.c:2792:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00010000:00010000:2.0:1776178151.820847:0:6244:0:(ldlm_lockd.c:2792:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000004:00080000:2.0:1776178151.820865:0:3356:0:(osp_sync.c:562:osp_sync_interpret()) reply req 00000000575d0558/2, rc 0, transno 2055 00000100:00100000:2.0:1776178151.820869:0:3356:0:(client.c:2215:ptlrpc_check_set()) Completed RPC req@00000000575d0558 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_03:lustre-MDT0000-mdtlov_UUID:3356:1862456998033280:0@lo:6:osp-syn-0-0.0 00010000:00010000:2.0:1776178151.820883:0:8227:0:(ldlm_request.c:199:ldlm_completion_tail()) ### client-side enqueue: granted ns: filter-lustre-OST0000_UUID lock: 000000006b0afb52/0xa40083ab9bf532e7 lrc: 3/0,1 mode: PW/PW res: [0x43:0x0:0x0].0x0 rrc: 2 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x40010080000000 nid: local remote: 0x0 expref: -99 pid: 8227 timeout: 0 lvb_type: 0 00010000:00010000:2.0:1776178151.820891:0:8227:0:(ldlm_request.c:528:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 000000006b0afb52/0xa40083ab9bf532e7 lrc: 3/0,1 mode: PW/PW res: [0x43:0x0:0x0].0x0 rrc: 2 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x40010080000000 nid: local remote: 0x0 expref: -99 pid: 8227 timeout: 0 lvb_type: 0 00010000:00010000:2.0:1776178151.820898:0:8227:0:(ldlm_lock.c:820:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0000_UUID lock: 000000006b0afb52/0xa40083ab9bf532e7 lrc: 3/0,1 mode: PW/PW res: [0x43:0x0:0x0].0x0 rrc: 2 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x50010080000000 nid: local remote: 0x0 expref: -99 pid: 8227 timeout: 0 lvb_type: 0 00010000:00010000:2.0:1776178151.820904:0:8227:0:(ldlm_lock.c:879:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 000000006b0afb52/0xa40083ab9bf532e7 lrc: 2/0,0 mode: PW/PW res: [0x43:0x0:0x0].0x0 rrc: 2 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x50010480000000 nid: local remote: 0x0 expref: -99 pid: 8227 timeout: 0 lvb_type: 0 00010000:00010000:2.0:1776178151.820923:0:8227:0:(ldlm_lockd.c:2159:__ldlm_bl_to_thread()) added 1/1 locks to regular blp list, 1 blwis in pool 00010000:00010000:2.0:1776178151.820929:0:9225:0:(ldlm_lockd.c:2789:ldlm_bl_get_work()) Got 1 locks of 0 total in blp. (0 blwis in pool) 00010000:00010000:2.0:1776178151.821211:0:9225:0:(ldlm_lockd.c:1879:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 000000006b0afb52/0xa40083ab9bf532e7 lrc: 2/0,0 mode: PW/PW res: [0x43:0x0:0x0].0x0 rrc: 2 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x40010480000000 nid: local remote: 0x0 expref: -99 pid: 8227 timeout: 0 lvb_type: 0 00010000:00010000:2.0:1776178151.821219:0:9225:0:(ldlm_lockd.c:1893:ldlm_handle_bl_callback()) Lock 000000006b0afb52 already unused, calling callback (0000000063c97258) 00010000:00010000:2.0:1776178151.821221:0:9225:0:(ldlm_request.c:364:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 000000006b0afb52/0xa40083ab9bf532e7 lrc: 2/0,0 mode: PW/PW res: [0x43:0x0:0x0].0x0 rrc: 2 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x40010480000000 nid: local remote: 0x0 expref: -99 pid: 8227 timeout: 0 lvb_type: 0 00010000:00010000:2.0:1776178151.821227:0:9225:0:(ldlm_request.c:1257:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 000000006b0afb52/0xa40083ab9bf532e7 lrc: 3/0,0 mode: PW/PW res: [0x43:0x0:0x0].0x0 rrc: 2 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x40018480000000 nid: local remote: 0x0 expref: -99 pid: 8227 timeout: 0 lvb_type: 0 00010000:00010000:2.0:1776178151.821238:0:9225:0:(ldlm_lockd.c:1903:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 000000006b0afb52/0xa40083ab9bf532e7 lrc: 1/0,0 mode: --/PW res: [0x43:0x0:0x0].0x0 rrc: 2 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x44819480000000 nid: local remote: 0x0 expref: -99 pid: 8227 timeout: 0 lvb_type: 0 00010000:00010000:2.0:1776178151.821243:0:9225:0:(ldlm_lock.c:216:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 000000006b0afb52/0xa40083ab9bf532e7 lrc: 0/0,0 mode: --/PW res: [0x43:0x0:0x0].0x0 rrc: 2 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x44819480000000 nid: local remote: 0x0 expref: -99 pid: 8227 timeout: 0 lvb_type: 0 00010000:00010000:2.0:1776178151.821258:0:9225:0:(ldlm_lockd.c:2792:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00010000:00010000:2.0:1776178151.821259:0:9225:0:(ldlm_lockd.c:2792:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00010000:00010000:2.0:1776178151.821260:0:9225:0:(ldlm_lockd.c:2792:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:2.0:1776178151.821457:0:8227:0:(service.c:2372:ptlrpc_server_handle_request()) Handled RPC req@0000000015095c30 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_000:lustre-MDT0000-mdtlov_UUID+9:3354:x1862456998033152:12345-0@lo:6:osp-syn-0-0.0 Request processed in 35983us (36463us total) trans 4294969352 rc 0/0 00000100:00100000:2.0:1776178151.821464:0:8227:0:(nrs_fifo.c:241:nrs_fifo_req_stop()) NRS stop fifo request from 12345-0@lo, seq: 5252 00000004:00080000:2.0:1776178151.821492:0:3354:0:(osp_sync.c:562:osp_sync_interpret()) reply req 00000000ecb64602/2, rc 0, transno 2056 00000100:00100000:2.0:1776178151.821497:0:3354:0:(client.c:2215:ptlrpc_check_set()) Completed RPC req@00000000ecb64602 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_01:lustre-MDT0000-mdtlov_UUID:3354:1862456998033152:0@lo:6:osp-syn-0-0.0 00010000:00010000:2.0:1776178151.821538:0:8228:0:(ldlm_request.c:199:ldlm_completion_tail()) ### client-side enqueue: granted ns: filter-lustre-OST0001_UUID lock: 000000006c464689/0xa40083ab9bf532ee lrc: 3/0,1 mode: PW/PW res: [0x43:0x0:0x0].0x0 rrc: 2 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x40010080000000 nid: local remote: 0x0 expref: -99 pid: 8228 timeout: 0 lvb_type: 0 00010000:00010000:2.0:1776178151.821688:0:8228:0:(ldlm_request.c:528:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0001_UUID lock: 000000006c464689/0xa40083ab9bf532ee lrc: 3/0,1 mode: PW/PW res: [0x43:0x0:0x0].0x0 rrc: 2 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x40010080000000 nid: local remote: 0x0 expref: -99 pid: 8228 timeout: 0 lvb_type: 0 00010000:00010000:2.0:1776178151.821706:0:8228:0:(ldlm_lock.c:820:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0001_UUID lock: 000000006c464689/0xa40083ab9bf532ee lrc: 3/0,1 mode: PW/PW res: [0x43:0x0:0x0].0x0 rrc: 2 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x50010080000000 nid: local remote: 0x0 expref: -99 pid: 8228 timeout: 0 lvb_type: 0 00010000:00010000:2.0:1776178151.821712:0:8228:0:(ldlm_lock.c:879:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0001_UUID lock: 000000006c464689/0xa40083ab9bf532ee lrc: 2/0,0 mode: PW/PW res: [0x43:0x0:0x0].0x0 rrc: 2 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x50010480000000 nid: local remote: 0x0 expref: -99 pid: 8228 timeout: 0 lvb_type: 0 00010000:00010000:2.0:1776178151.821726:0:8228:0:(ldlm_lockd.c:2159:__ldlm_bl_to_thread()) added 1/1 locks to regular blp list, 1 blwis in pool 00000100:00100000:2.0:1776178151.821859:0:8228:0:(service.c:2372:ptlrpc_server_handle_request()) Handled RPC req@0000000039ff8335 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_001:lustre-MDT0000-mdtlov_UUID+8:3356:x1862456998033088:12345-0@lo:6:osp-syn-1-0.0 Request processed in 31258us (32490us total) trans 4294969323 rc 0/0 00000100:00100000:2.0:1776178151.821866:0:8228:0:(nrs_fifo.c:241:nrs_fifo_req_stop()) NRS stop fifo request from 12345-0@lo, seq: 5253 00010000:00010000:2.0:1776178151.821884:0:9224:0:(ldlm_lockd.c:2789:ldlm_bl_get_work()) Got 1 locks of 0 total in blp. (0 blwis in pool) 00010000:00010000:2.0:1776178151.821887:0:9224:0:(ldlm_lockd.c:1879:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0001_UUID lock: 000000006c464689/0xa40083ab9bf532ee lrc: 2/0,0 mode: PW/PW res: [0x43:0x0:0x0].0x0 rrc: 2 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x40010480000000 nid: local remote: 0x0 expref: -99 pid: 8228 timeout: 0 lvb_type: 0 00010000:00010000:2.0:1776178151.821894:0:9224:0:(ldlm_lockd.c:1893:ldlm_handle_bl_callback()) Lock 000000006c464689 already unused, calling callback (0000000063c97258) 00010000:00010000:2.0:1776178151.821896:0:9224:0:(ldlm_request.c:364:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0001_UUID lock: 000000006c464689/0xa40083ab9bf532ee lrc: 2/0,0 mode: PW/PW res: [0x43:0x0:0x0].0x0 rrc: 2 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x40010480000000 nid: local remote: 0x0 expref: -99 pid: 8228 timeout: 0 lvb_type: 0 00010000:00010000:2.0:1776178151.821902:0:9224:0:(ldlm_request.c:1257:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0001_UUID lock: 000000006c464689/0xa40083ab9bf532ee lrc: 3/0,0 mode: PW/PW res: [0x43:0x0:0x0].0x0 rrc: 2 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x40018480000000 nid: local remote: 0x0 expref: -99 pid: 8228 timeout: 0 lvb_type: 0 00010000:00010000:2.0:1776178151.821915:0:9224:0:(ldlm_lockd.c:1903:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0001_UUID lock: 000000006c464689/0xa40083ab9bf532ee lrc: 1/0,0 mode: --/PW res: [0x43:0x0:0x0].0x0 rrc: 2 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x44819480000000 nid: local remote: 0x0 expref: -99 pid: 8228 timeout: 0 lvb_type: 0 00010000:00010000:2.0:1776178151.821927:0:9224:0:(ldlm_lock.c:216:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0001_UUID lock: 000000006c464689/0xa40083ab9bf532ee lrc: 0/0,0 mode: --/PW res: [0x43:0x0:0x0].0x0 rrc: 2 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x44819480000000 nid: local remote: 0x0 expref: -99 pid: 8228 timeout: 0 lvb_type: 0 00010000:00010000:2.0:1776178151.821941:0:9224:0:(ldlm_lockd.c:2792:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00010000:00010000:2.0:1776178151.821943:0:9224:0:(ldlm_lockd.c:2792:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00010000:00010000:2.0:1776178151.821944:0:9224:0:(ldlm_lockd.c:2792:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000004:00080000:2.0:1776178151.821963:0:3356:0:(osp_sync.c:562:osp_sync_interpret()) reply req 0000000023210d1c/2, rc 0, transno 2027 00000100:00100000:2.0:1776178151.821968:0:3356:0:(client.c:2215:ptlrpc_check_set()) Completed RPC req@0000000023210d1c pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_03:lustre-MDT0000-mdtlov_UUID:3356:1862456998033088:0@lo:6:osp-syn-1-0.0 00010000:00010000:2.0:1776178151.821991:0:6245:0:(ldlm_lockd.c:2792:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00010000:00010000:3.0:1776178151.843944:0:6244:0:(ldlm_lockd.c:2792:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00010000:00010000:3.0:1776178152.864403:0:9225:0:(ldlm_lockd.c:2792:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000400:00000080:2.0:1776178153.705114:0:21290:0:(module.c:221:libcfs_ioctl()) libcfs ioctl cmd 3221775648 00000001:02000400:2.0:1776178153.705120:0:21290:0:(debug.c:702:libcfs_debug_mark_buffer()) DEBUG MARKER: == replay-single test 102b: check resend (reply lost) with multiple modify RPCs in flight ========================================================== 10:49:11 (1776178151) 00000100:00100000:2.0:1776178153.821383:0:3347:0:(events.c:361:request_in_callback()) peer: 12345-192.168.204.3@tcp (source: 12345-192.168.204.3@tcp) 00000100:00100000:3.0:1776178153.821560:0:8250:0:(service.c:2152:ptlrpc_server_handle_req_in()) got req x1862456979089344 00000100:00100000:3.0:1776178153.821629:0:8250:0:(nrs_fifo.c:179:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.204.3@tcp, seq: 6524 00000100:00100000:3.0:1776178153.821637:0:8250:0:(service.c:2324:ptlrpc_server_handle_request()) Handling RPC req@00000000e0d60cbc pname:cluuid+ref:pid:xid:nid:opc:job mdt00_003:b50963bd-8eba-4432-9a54-8888d00e3fa1+6:17614:x1862456979089344:12345-192.168.204.3@tcp:101:lfs.0 00010000:00010000:3.0:1776178153.821672:0:8250:0:(ldlm_lockd.c:1248:ldlm_handle_enqueue0()) ### server-side enqueue handler START 00010000:00010000:3.0:1776178153.821693:0:8250:0:(ldlm_lockd.c:1318:ldlm_handle_enqueue0()) ### server-side enqueue handler, new lock created ns: mdt-lustre-MDT0000_UUID lock: 00000000014a8a18/0xa40083ab9bf5330a lrc: 2/0,0 mode: --/CR res: [0x200000007:0x1:0x0].0x0 bits 0x0/0x0 rrc: 4 type: IBT gid 0 flags: 0x40000000000000 nid: local remote: 0x771f68aa1eb27a7e expref: -99 pid: 8250 timeout: 0 lvb_type: 0 00010000:00010000:3.0:1776178153.821756:0:8250:0:(ldlm_lock.c:766:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(CR) ns: mdt-lustre-MDT0000_UUID lock: 000000002a990613/0xa40083ab9bf53311 lrc: 3/1,0 mode: --/CR res: [0x200000007:0x1:0x0].0x0 bits 0x0/0x0 rrc: 5 type: IBT gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 8250 timeout: 0 lvb_type: 0 00010000:00010000:3.0:1776178153.821772:0:8250:0:(ldlm_lock.c:1064:ldlm_granted_list_add_lock()) ### About to add lock: ns: mdt-lustre-MDT0000_UUID lock: 000000002a990613/0xa40083ab9bf53311 lrc: 3/1,0 mode: CR/CR res: [0x200000007:0x1:0x0].0x0 bits 0x2/0x0 rrc: 5 type: IBT gid 0 flags: 0x50210000000000 nid: local remote: 0x0 expref: -99 pid: 8250 timeout: 0 lvb_type: 0 00010000:00010000:3.0:1776178153.821788:0:8250:0:(ldlm_request.c:528:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: mdt-lustre-MDT0000_UUID lock: 000000002a990613/0xa40083ab9bf53311 lrc: 3/1,0 mode: CR/CR res: [0x200000007:0x1:0x0].0x0 bits 0x2/0x0 rrc: 5 type: IBT gid 0 flags: 0x40210000000000 nid: local remote: 0x0 expref: -99 pid: 8250 timeout: 0 lvb_type: 0 00010000:00010000:3.0:1776178153.821803:0:8250:0:(ldlm_lock.c:766:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PR) ns: mdt-lustre-MDT0000_UUID lock: 00000000c1e7ea3e/0xa40083ab9bf53318 lrc: 3/1,0 mode: --/PR res: [0x200000007:0x1:0x0].0x58a1b1aa bits 0x0/0x0 rrc: 2 type: IBT gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 8250 timeout: 0 lvb_type: 0 00010000:00010000:3.0:1776178153.821810:0:8250:0:(ldlm_lock.c:1064:ldlm_granted_list_add_lock()) ### About to add lock: ns: mdt-lustre-MDT0000_UUID lock: 00000000c1e7ea3e/0xa40083ab9bf53318 lrc: 3/1,0 mode: PR/PR res: [0x200000007:0x1:0x0].0x58a1b1aa bits 0x2/0x0 rrc: 2 type: IBT gid 0 flags: 0x50210000000000 nid: local remote: 0x0 expref: -99 pid: 8250 timeout: 0 lvb_type: 0 00010000:00010000:3.0:1776178153.821816:0:8250:0:(ldlm_request.c:528:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: mdt-lustre-MDT0000_UUID lock: 00000000c1e7ea3e/0xa40083ab9bf53318 lrc: 3/1,0 mode: PR/PR res: [0x200000007:0x1:0x0].0x58a1b1aa bits 0x2/0x0 rrc: 2 type: IBT gid 0 flags: 0x40210000000000 nid: local remote: 0x0 expref: -99 pid: 8250 timeout: 0 lvb_type: 0 00010000:00010000:3.0:1776178153.821876:0:8250:0:(ldlm_lock.c:820:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(CR) ns: mdt-lustre-MDT0000_UUID lock: 000000002a990613/0xa40083ab9bf53311 lrc: 3/1,0 mode: CR/CR res: [0x200000007:0x1:0x0].0x0 bits 0x2/0x0 rrc: 5 type: IBT gid 0 flags: 0x50210000000000 nid: local remote: 0x0 expref: -99 pid: 8250 timeout: 0 lvb_type: 0 00010000:00010000:3.0:1776178153.821884:0:8250:0:(ldlm_lock.c:879:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: mdt-lustre-MDT0000_UUID lock: 000000002a990613/0xa40083ab9bf53311 lrc: 2/0,0 mode: CR/CR res: [0x200000007:0x1:0x0].0x0 bits 0x2/0x0 rrc: 5 type: IBT gid 0 flags: 0x50210400000000 nid: local remote: 0x0 expref: -99 pid: 8250 timeout: 0 lvb_type: 0 00010000:00010000:3.0:1776178153.821891:0:8250:0:(ldlm_lockd.c:1879:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: mdt-lustre-MDT0000_UUID lock: 000000002a990613/0xa40083ab9bf53311 lrc: 3/0,0 mode: CR/CR res: [0x200000007:0x1:0x0].0x0 bits 0x2/0x0 rrc: 5 type: IBT gid 0 flags: 0x40210400000000 nid: local remote: 0x0 expref: -99 pid: 8250 timeout: 0 lvb_type: 0 00010000:00010000:3.0:1776178153.821903:0:8250:0:(ldlm_lockd.c:1893:ldlm_handle_bl_callback()) Lock 000000002a990613 already unused, calling callback (0000000079e78348) 00010000:00010000:3.0:1776178153.821906:0:8250:0:(ldlm_request.c:364:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: mdt-lustre-MDT0000_UUID lock: 000000002a990613/0xa40083ab9bf53311 lrc: 3/0,0 mode: CR/CR res: [0x200000007:0x1:0x0].0x0 bits 0x2/0x0 rrc: 5 type: IBT gid 0 flags: 0x40210400000000 nid: local remote: 0x0 expref: -99 pid: 8250 timeout: 0 lvb_type: 0 00010000:00010000:3.0:1776178153.821914:0:8250:0:(ldlm_request.c:1257:ldlm_cli_cancel_local()) ### server-side local cancel ns: mdt-lustre-MDT0000_UUID lock: 000000002a990613/0xa40083ab9bf53311 lrc: 4/0,0 mode: CR/CR res: [0x200000007:0x1:0x0].0x0 bits 0x2/0x0 rrc: 5 type: IBT gid 0 flags: 0x40218400000000 nid: local remote: 0x0 expref: -99 pid: 8250 timeout: 0 lvb_type: 0 00010000:00010000:3.0:1776178153.821927:0:8250:0:(ldlm_inodebits.c:94:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x200000007:0x1:0x0].0x0 (00000000ad0d4094) 00010000:00010000:3.0:1776178153.821930:0:8250:0:(ldlm_inodebits.c:96:ldlm_reprocess_inodebits_queue()) Hint 2 00010000:00010000:3.0:1776178153.821935:0:8250:0:(ldlm_lockd.c:1903:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: mdt-lustre-MDT0000_UUID lock: 000000002a990613/0xa40083ab9bf53311 lrc: 2/0,0 mode: --/CR res: [0x200000007:0x1:0x0].0x0 bits 0x2/0x0 rrc: 5 type: IBT gid 0 flags: 0x44a19400000000 nid: local remote: 0x0 expref: -99 pid: 8250 timeout: 0 lvb_type: 0 00010000:00010000:3.0:1776178153.821941:0:8250:0:(ldlm_lock.c:216:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: mdt-lustre-MDT0000_UUID lock: 000000002a990613/0xa40083ab9bf53311 lrc: 0/0,0 mode: --/CR res: [0x200000007:0x1:0x0].0x0 bits 0x2/0x0 rrc: 5 type: IBT gid 0 flags: 0x44a19400000000 nid: local remote: 0x0 expref: -99 pid: 8250 timeout: 0 lvb_type: 0 00010000:00010000:3.0:1776178153.821961:0:8250:0:(ldlm_lock.c:820:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PR) ns: mdt-lustre-MDT0000_UUID lock: 00000000c1e7ea3e/0xa40083ab9bf53318 lrc: 3/1,0 mode: PR/PR res: [0x200000007:0x1:0x0].0x58a1b1aa bits 0x2/0x0 rrc: 2 type: IBT gid 0 flags: 0x50210000000000 nid: local remote: 0x0 expref: -99 pid: 8250 timeout: 0 lvb_type: 0 00010000:00010000:3.0:1776178153.821968:0:8250:0:(ldlm_lock.c:879:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: mdt-lustre-MDT0000_UUID lock: 00000000c1e7ea3e/0xa40083ab9bf53318 lrc: 2/0,0 mode: PR/PR res: [0x200000007:0x1:0x0].0x58a1b1aa bits 0x2/0x0 rrc: 2 type: IBT gid 0 flags: 0x50210400000000 nid: local remote: 0x0 expref: -99 pid: 8250 timeout: 0 lvb_type: 0 00010000:00010000:3.0:1776178153.821973:0:8250:0:(ldlm_lockd.c:1879:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: mdt-lustre-MDT0000_UUID lock: 00000000c1e7ea3e/0xa40083ab9bf53318 lrc: 3/0,0 mode: PR/PR res: [0x200000007:0x1:0x0].0x58a1b1aa bits 0x2/0x0 rrc: 2 type: IBT gid 0 flags: 0x40210400000000 nid: local remote: 0x0 expref: -99 pid: 8250 timeout: 0 lvb_type: 0 00010000:00010000:3.0:1776178153.821979:0:8250:0:(ldlm_lockd.c:1893:ldlm_handle_bl_callback()) Lock 00000000c1e7ea3e already unused, calling callback (0000000079e78348) 00010000:00010000:3.0:1776178153.821981:0:8250:0:(ldlm_request.c:364:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: mdt-lustre-MDT0000_UUID lock: 00000000c1e7ea3e/0xa40083ab9bf53318 lrc: 3/0,0 mode: PR/PR res: [0x200000007:0x1:0x0].0x58a1b1aa bits 0x2/0x0 rrc: 2 type: IBT gid 0 flags: 0x40210400000000 nid: local remote: 0x0 expref: -99 pid: 8250 timeout: 0 lvb_type: 0 00010000:00010000:3.0:1776178153.821987:0:8250:0:(ldlm_request.c:1257:ldlm_cli_cancel_local()) ### server-side local cancel ns: mdt-lustre-MDT0000_UUID lock: 00000000c1e7ea3e/0xa40083ab9bf53318 lrc: 4/0,0 mode: PR/PR res: [0x200000007:0x1:0x0].0x58a1b1aa bits 0x2/0x0 rrc: 2 type: IBT gid 0 flags: 0x40218400000000 nid: local remote: 0x0 expref: -99 pid: 8250 timeout: 0 lvb_type: 0 00010000:00010000:3.0:1776178153.821994:0:8250:0:(ldlm_inodebits.c:94:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x200000007:0x1:0x0].0x58a1b1aa (00000000099618da) 00010000:00010000:3.0:1776178153.821997:0:8250:0:(ldlm_inodebits.c:96:ldlm_reprocess_inodebits_queue()) Hint 2 00010000:00010000:3.0:1776178153.821999:0:8250:0:(ldlm_lockd.c:1903:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: mdt-lustre-MDT0000_UUID lock: 00000000c1e7ea3e/0xa40083ab9bf53318 lrc: 2/0,0 mode: --/PR res: [0x200000007:0x1:0x0].0x58a1b1aa bits 0x2/0x0 rrc: 2 type: IBT gid 0 flags: 0x44a19400000000 nid: local remote: 0x0 expref: -99 pid: 8250 timeout: 0 lvb_type: 0 00010000:00010000:3.0:1776178153.822004:0:8250:0:(ldlm_lock.c:216:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: mdt-lustre-MDT0000_UUID lock: 00000000c1e7ea3e/0xa40083ab9bf53318 lrc: 0/0,0 mode: --/PR res: [0x200000007:0x1:0x0].0x58a1b1aa bits 0x2/0x0 rrc: 2 type: IBT gid 0 flags: 0x44a19400000000 nid: local remote: 0x0 expref: -99 pid: 8250 timeout: 0 lvb_type: 0 00010000:00010000:3.0:1776178153.822040:0:8250:0:(ldlm_lockd.c:1484:ldlm_handle_enqueue0()) ### server-side enqueue handler, sending reply (err=301, rc=0) ns: mdt-lustre-MDT0000_UUID lock: 00000000014a8a18/0xa40083ab9bf5330a lrc: 1/0,0 mode: --/CR res: [0x200000007:0x1:0x0].0x0 bits 0x2/0x0 rrc: 4 type: IBT gid 0 flags: 0x44000000000000 nid: 192.168.204.3@tcp remote: 0x771f68aa1eb27a7e expref: 7 pid: 8250 timeout: 0 lvb_type: 0 00010000:00010000:3.0:1776178153.822048:0:8250:0:(ldlm_lock.c:216:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: mdt-lustre-MDT0000_UUID lock: 00000000014a8a18/0xa40083ab9bf5330a lrc: 0/0,0 mode: --/CR res: [0x200000007:0x1:0x0].0x0 bits 0x2/0x0 rrc: 4 type: IBT gid 0 flags: 0x44000000000000 nid: 192.168.204.3@tcp remote: 0x771f68aa1eb27a7e expref: 7 pid: 8250 timeout: 0 lvb_type: 0 00010000:00010000:3.0:1776178153.822056:0:8250:0:(ldlm_lockd.c:1565:ldlm_handle_enqueue0()) ### server-side enqueue handler END (lock 00000000014a8a18, rc 0) 00000100:00100000:3.0:1776178153.823683:0:8250:0:(service.c:2372:ptlrpc_server_handle_request()) Handled RPC req@00000000e0d60cbc pname:cluuid+ref:pid:xid:nid:opc:job mdt00_003:b50963bd-8eba-4432-9a54-8888d00e3fa1+6:17614:x1862456979089344:12345-192.168.204.3@tcp:101:lfs.0 Request processed in 2040us (2304us total) trans 0 rc 301/301 00000100:00100000:3.0:1776178153.823696:0:8250:0:(nrs_fifo.c:241:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.204.3@tcp, seq: 6524 00000100:00100000:0.0:1776178153.834296:0:3346:0:(events.c:361:request_in_callback()) peer: 12345-192.168.204.3@tcp (source: 12345-192.168.204.3@tcp) 00000100:00100000:3.0:1776178153.834416:0:8250:0:(service.c:2152:ptlrpc_server_handle_req_in()) got req x1862456979089408 00000100:00100000:3.0:1776178153.834518:0:8250:0:(nrs_fifo.c:179:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.204.3@tcp, seq: 6525 00000100:00100000:3.0:1776178153.834527:0:8250:0:(service.c:2324:ptlrpc_server_handle_request()) Handling RPC req@00000000ed0a8ae7 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_003:b50963bd-8eba-4432-9a54-8888d00e3fa1+6:17614:x1862456979089408:12345-192.168.204.3@tcp:101:lfs.0 00010000:00010000:3.0:1776178153.834552:0:8250:0:(ldlm_lockd.c:1248:ldlm_handle_enqueue0()) ### server-side enqueue handler START 00010000:00010000:3.0:1776178153.834577:0:8250:0:(ldlm_lockd.c:1318:ldlm_handle_enqueue0()) ### server-side enqueue handler, new lock created ns: mdt-lustre-MDT0000_UUID lock: 00000000014a8a18/0xa40083ab9bf5331f lrc: 2/0,0 mode: --/CR res: [0x200000007:0x1:0x0].0x0 bits 0x0/0x0 rrc: 4 type: IBT gid 0 flags: 0x40000000000000 nid: local remote: 0x771f68aa1eb27a85 expref: -99 pid: 8250 timeout: 0 lvb_type: 0 00010000:00010000:3.0:1776178153.834651:0:8250:0:(ldlm_lock.c:766:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(CR) ns: mdt-lustre-MDT0000_UUID lock: 00000000c1e7ea3e/0xa40083ab9bf53326 lrc: 3/1,0 mode: --/CR res: [0x200000007:0x1:0x0].0x0 bits 0x0/0x0 rrc: 5 type: IBT gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 8250 timeout: 0 lvb_type: 0 00010000:00010000:3.0:1776178153.834718:0:8250:0:(ldlm_lock.c:1064:ldlm_granted_list_add_lock()) ### About to add lock: ns: mdt-lustre-MDT0000_UUID lock: 00000000c1e7ea3e/0xa40083ab9bf53326 lrc: 3/1,0 mode: CR/CR res: [0x200000007:0x1:0x0].0x0 bits 0x2/0x0 rrc: 5 type: IBT gid 0 flags: 0x50210000000000 nid: local remote: 0x0 expref: -99 pid: 8250 timeout: 0 lvb_type: 0 00010000:00010000:3.0:1776178153.834731:0:8250:0:(ldlm_request.c:528:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: mdt-lustre-MDT0000_UUID lock: 00000000c1e7ea3e/0xa40083ab9bf53326 lrc: 3/1,0 mode: CR/CR res: [0x200000007:0x1:0x0].0x0 bits 0x2/0x0 rrc: 5 type: IBT gid 0 flags: 0x40210000000000 nid: local remote: 0x0 expref: -99 pid: 8250 timeout: 0 lvb_type: 0 00010000:00010000:3.0:1776178153.834748:0:8250:0:(ldlm_lock.c:766:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PR) ns: mdt-lustre-MDT0000_UUID lock: 000000002a990613/0xa40083ab9bf5332d lrc: 3/1,0 mode: --/PR res: [0x200000007:0x1:0x0].0x58a1b1aa bits 0x0/0x0 rrc: 2 type: IBT gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 8250 timeout: 0 lvb_type: 0 00010000:00010000:3.0:1776178153.834755:0:8250:0:(ldlm_lock.c:1064:ldlm_granted_list_add_lock()) ### About to add lock: ns: mdt-lustre-MDT0000_UUID lock: 000000002a990613/0xa40083ab9bf5332d lrc: 3/1,0 mode: PR/PR res: [0x200000007:0x1:0x0].0x58a1b1aa bits 0x2/0x0 rrc: 2 type: IBT gid 0 flags: 0x50210000000000 nid: local remote: 0x0 expref: -99 pid: 8250 timeout: 0 lvb_type: 0 00010000:00010000:3.0:1776178153.834762:0:8250:0:(ldlm_request.c:528:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: mdt-lustre-MDT0000_UUID lock: 000000002a990613/0xa40083ab9bf5332d lrc: 3/1,0 mode: PR/PR res: [0x200000007:0x1:0x0].0x58a1b1aa bits 0x2/0x0 rrc: 2 type: IBT gid 0 flags: 0x40210000000000 nid: local remote: 0x0 expref: -99 pid: 8250 timeout: 0 lvb_type: 0 00010000:00010000:3.0:1776178153.834811:0:8250:0:(ldlm_lock.c:820:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(CR) ns: mdt-lustre-MDT0000_UUID lock: 00000000c1e7ea3e/0xa40083ab9bf53326 lrc: 3/1,0 mode: CR/CR res: [0x200000007:0x1:0x0].0x0 bits 0x2/0x0 rrc: 5 type: IBT gid 0 flags: 0x50210000000000 nid: local remote: 0x0 expref: -99 pid: 8250 timeout: 0 lvb_type: 0 00010000:00010000:3.0:1776178153.834819:0:8250:0:(ldlm_lock.c:879:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: mdt-lustre-MDT0000_UUID lock: 00000000c1e7ea3e/0xa40083ab9bf53326 lrc: 2/0,0 mode: CR/CR res: [0x200000007:0x1:0x0].0x0 bits 0x2/0x0 rrc: 5 type: IBT gid 0 flags: 0x50210400000000 nid: local remote: 0x0 expref: -99 pid: 8250 timeout: 0 lvb_type: 0 00010000:00010000:3.0:1776178153.834826:0:8250:0:(ldlm_lockd.c:1879:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: mdt-lustre-MDT0000_UUID lock: 00000000c1e7ea3e/0xa40083ab9bf53326 lrc: 3/0,0 mode: CR/CR res: [0x200000007:0x1:0x0].0x0 bits 0x2/0x0 rrc: 5 type: IBT gid 0 flags: 0x40210400000000 nid: local remote: 0x0 expref: -99 pid: 8250 timeout: 0 lvb_type: 0 00010000:00010000:3.0:1776178153.834833:0:8250:0:(ldlm_lockd.c:1893:ldlm_handle_bl_callback()) Lock 00000000c1e7ea3e already unused, calling callback (0000000079e78348) 00010000:00010000:3.0:1776178153.834837:0:8250:0:(ldlm_request.c:364:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: mdt-lustre-MDT0000_UUID lock: 00000000c1e7ea3e/0xa40083ab9bf53326 lrc: 3/0,0 mode: CR/CR res: [0x200000007:0x1:0x0].0x0 bits 0x2/0x0 rrc: 5 type: IBT gid 0 flags: 0x40210400000000 nid: local remote: 0x0 expref: -99 pid: 8250 timeout: 0 lvb_type: 0 00010000:00010000:3.0:1776178153.834845:0:8250:0:(ldlm_request.c:1257:ldlm_cli_cancel_local()) ### server-side local cancel ns: mdt-lustre-MDT0000_UUID lock: 00000000c1e7ea3e/0xa40083ab9bf53326 lrc: 4/0,0 mode: CR/CR res: [0x200000007:0x1:0x0].0x0 bits 0x2/0x0 rrc: 5 type: IBT gid 0 flags: 0x40218400000000 nid: local remote: 0x0 expref: -99 pid: 8250 timeout: 0 lvb_type: 0 00010000:00010000:3.0:1776178153.834860:0:8250:0:(ldlm_inodebits.c:94:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x200000007:0x1:0x0].0x0 (00000000ad0d4094) 00010000:00010000:3.0:1776178153.834864:0:8250:0:(ldlm_inodebits.c:96:ldlm_reprocess_inodebits_queue()) Hint 2 00010000:00010000:3.0:1776178153.834869:0:8250:0:(ldlm_lockd.c:1903:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: mdt-lustre-MDT0000_UUID lock: 00000000c1e7ea3e/0xa40083ab9bf53326 lrc: 2/0,0 mode: --/CR res: [0x200000007:0x1:0x0].0x0 bits 0x2/0x0 rrc: 5 type: IBT gid 0 flags: 0x44a19400000000 nid: local remote: 0x0 expref: -99 pid: 8250 timeout: 0 lvb_type: 0 00010000:00010000:3.0:1776178153.834876:0:8250:0:(ldlm_lock.c:216:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: mdt-lustre-MDT0000_UUID lock: 00000000c1e7ea3e/0xa40083ab9bf53326 lrc: 0/0,0 mode: --/CR res: [0x200000007:0x1:0x0].0x0 bits 0x2/0x0 rrc: 5 type: IBT gid 0 flags: 0x44a19400000000 nid: local remote: 0x0 expref: -99 pid: 8250 timeout: 0 lvb_type: 0 00010000:00010000:3.0:1776178153.835746:0:8250:0:(ldlm_lock.c:820:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PR) ns: mdt-lustre-MDT0000_UUID lock: 000000002a990613/0xa40083ab9bf5332d lrc: 3/1,0 mode: PR/PR res: [0x200000007:0x1:0x0].0x58a1b1aa bits 0x2/0x0 rrc: 2 type: IBT gid 0 flags: 0x50210000000000 nid: local remote: 0x0 expref: -99 pid: 8250 timeout: 0 lvb_type: 0 00010000:00010000:3.0:1776178153.835759:0:8250:0:(ldlm_lock.c:879:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: mdt-lustre-MDT0000_UUID lock: 000000002a990613/0xa40083ab9bf5332d lrc: 2/0,0 mode: PR/PR res: [0x200000007:0x1:0x0].0x58a1b1aa bits 0x2/0x0 rrc: 2 type: IBT gid 0 flags: 0x50210400000000 nid: local remote: 0x0 expref: -99 pid: 8250 timeout: 0 lvb_type: 0 00010000:00010000:3.0:1776178153.835766:0:8250:0:(ldlm_lockd.c:1879:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: mdt-lustre-MDT0000_UUID lock: 000000002a990613/0xa40083ab9bf5332d lrc: 3/0,0 mode: PR/PR res: [0x200000007:0x1:0x0].0x58a1b1aa bits 0x2/0x0 rrc: 2 type: IBT gid 0 flags: 0x40210400000000 nid: local remote: 0x0 expref: -99 pid: 8250 timeout: 0 lvb_type: 0 00010000:00010000:3.0:1776178153.835772:0:8250:0:(ldlm_lockd.c:1893:ldlm_handle_bl_callback()) Lock 000000002a990613 already unused, calling callback (0000000079e78348) 00010000:00010000:3.0:1776178153.835775:0:8250:0:(ldlm_request.c:364:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: mdt-lustre-MDT0000_UUID lock: 000000002a990613/0xa40083ab9bf5332d lrc: 3/0,0 mode: PR/PR res: [0x200000007:0x1:0x0].0x58a1b1aa bits 0x2/0x0 rrc: 2 type: IBT gid 0 flags: 0x40210400000000 nid: local remote: 0x0 expref: -99 pid: 8250 timeout: 0 lvb_type: 0 00010000:00010000:3.0:1776178153.835780:0:8250:0:(ldlm_request.c:1257:ldlm_cli_cancel_local()) ### server-side local cancel ns: mdt-lustre-MDT0000_UUID lock: 000000002a990613/0xa40083ab9bf5332d lrc: 4/0,0 mode: PR/PR res: [0x200000007:0x1:0x0].0x58a1b1aa bits 0x2/0x0 rrc: 2 type: IBT gid 0 flags: 0x40218400000000 nid: local remote: 0x0 expref: -99 pid: 8250 timeout: 0 lvb_type: 0 00010000:00010000:3.0:1776178153.835792:0:8250:0:(ldlm_inodebits.c:94:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x200000007:0x1:0x0].0x58a1b1aa (00000000099618da) 00010000:00010000:3.0:1776178153.835795:0:8250:0:(ldlm_inodebits.c:96:ldlm_reprocess_inodebits_queue()) Hint 2 00010000:00010000:3.0:1776178153.835797:0:8250:0:(ldlm_lockd.c:1903:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: mdt-lustre-MDT0000_UUID lock: 000000002a990613/0xa40083ab9bf5332d lrc: 2/0,0 mode: --/PR res: [0x200000007:0x1:0x0].0x58a1b1aa bits 0x2/0x0 rrc: 2 type: IBT gid 0 flags: 0x44a19400000000 nid: local remote: 0x0 expref: -99 pid: 8250 timeout: 0 lvb_type: 0 00010000:00010000:3.0:1776178153.835808:0:8250:0:(ldlm_lock.c:216:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: mdt-lustre-MDT0000_UUID lock: 000000002a990613/0xa40083ab9bf5332d lrc: 0/0,0 mode: --/PR res: [0x200000007:0x1:0x0].0x58a1b1aa bits 0x2/0x0 rrc: 2 type: IBT gid 0 flags: 0x44a19400000000 nid: local remote: 0x0 expref: -99 pid: 8250 timeout: 0 lvb_type: 0 00010000:00010000:3.0:1776178153.835850:0:8250:0:(ldlm_lockd.c:1484:ldlm_handle_enqueue0()) ### server-side enqueue handler, sending reply (err=301, rc=0) ns: mdt-lustre-MDT0000_UUID lock: 00000000014a8a18/0xa40083ab9bf5331f lrc: 1/0,0 mode: --/CR res: [0x200000007:0x1:0x0].0x0 bits 0x2/0x0 rrc: 4 type: IBT gid 0 flags: 0x44000000000000 nid: 192.168.204.3@tcp remote: 0x771f68aa1eb27a85 expref: 7 pid: 8250 timeout: 0 lvb_type: 0 00010000:00010000:3.0:1776178153.835858:0:8250:0:(ldlm_lock.c:216:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: mdt-lustre-MDT0000_UUID lock: 00000000014a8a18/0xa40083ab9bf5331f lrc: 0/0,0 mode: --/CR res: [0x200000007:0x1:0x0].0x0 bits 0x2/0x0 rrc: 4 type: IBT gid 0 flags: 0x44000000000000 nid: 192.168.204.3@tcp remote: 0x771f68aa1eb27a85 expref: 7 pid: 8250 timeout: 0 lvb_type: 0 00010000:00010000:3.0:1776178153.835868:0:8250:0:(ldlm_lockd.c:1565:ldlm_handle_enqueue0()) ### server-side enqueue handler END (lock 00000000014a8a18, rc 0) 00000100:00100000:3.0:1776178153.836049:0:8250:0:(service.c:2372:ptlrpc_server_handle_request()) Handled RPC req@00000000ed0a8ae7 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_003:b50963bd-8eba-4432-9a54-8888d00e3fa1+6:17614:x1862456979089408:12345-192.168.204.3@tcp:101:lfs.0 Request processed in 1521us (1764us total) trans 0 rc 301/301 00000100:00100000:3.0:1776178153.836075:0:8250:0:(nrs_fifo.c:241:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.204.3@tcp, seq: 6525 00000100:00100000:2.0:1776178153.841417:0:3348:0:(events.c:361:request_in_callback()) peer: 12345-192.168.204.3@tcp (source: 12345-192.168.204.3@tcp) 00000100:00100000:3.0:1776178153.841644:0:8250:0:(service.c:2152:ptlrpc_server_handle_req_in()) got req x1862456979089472 00000100:00100000:3.0:1776178153.841765:0:8250:0:(nrs_fifo.c:179:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.204.3@tcp, seq: 6526 00000100:00100000:3.0:1776178153.841773:0:8250:0:(service.c:2324:ptlrpc_server_handle_request()) Handling RPC req@000000002a321d27 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_003:b50963bd-8eba-4432-9a54-8888d00e3fa1+6:17614:x1862456979089472:12345-192.168.204.3@tcp:101:lfs.0 00010000:00010000:3.0:1776178153.841796:0:8250:0:(ldlm_lockd.c:1248:ldlm_handle_enqueue0()) ### server-side enqueue handler START 00010000:00010000:3.0:1776178153.841821:0:8250:0:(ldlm_lockd.c:1318:ldlm_handle_enqueue0()) ### server-side enqueue handler, new lock created ns: mdt-lustre-MDT0000_UUID lock: 0000000055d6a00d/0xa40083ab9bf53334 lrc: 2/0,0 mode: --/CR res: [0x200000007:0x1:0x0].0x0 bits 0x0/0x0 rrc: 4 type: IBT gid 0 flags: 0x40000000000000 nid: local remote: 0x771f68aa1eb27a8c expref: -99 pid: 8250 timeout: 0 lvb_type: 0 00010000:00010000:3.0:1776178153.841896:0:8250:0:(ldlm_lock.c:766:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PR) ns: mdt-lustre-MDT0000_UUID lock: 00000000dcea299c/0xa40083ab9bf5333b lrc: 3/1,0 mode: --/PR res: [0x200000007:0x1:0x0].0x0 bits 0x0/0x0 rrc: 5 type: IBT gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 8250 timeout: 0 lvb_type: 0 00010000:00010000:3.0:1776178153.841913:0:8250:0:(ldlm_lock.c:1064:ldlm_granted_list_add_lock()) ### About to add lock: ns: mdt-lustre-MDT0000_UUID lock: 00000000dcea299c/0xa40083ab9bf5333b lrc: 3/1,0 mode: PR/PR res: [0x200000007:0x1:0x0].0x0 bits 0x13/0x0 rrc: 5 type: IBT gid 0 flags: 0x50210000000000 nid: local remote: 0x0 expref: -99 pid: 8250 timeout: 0 lvb_type: 0 00010000:00010000:3.0:1776178153.841926:0:8250:0:(ldlm_request.c:528:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: mdt-lustre-MDT0000_UUID lock: 00000000dcea299c/0xa40083ab9bf5333b lrc: 3/1,0 mode: PR/PR res: [0x200000007:0x1:0x0].0x0 bits 0x13/0x0 rrc: 5 type: IBT gid 0 flags: 0x40210000000000 nid: local remote: 0x0 expref: -99 pid: 8250 timeout: 0 lvb_type: 0 00000004:00200000:3.0:1776178153.841981:0:8250:0:(mdt_handler.c:912:mdt_pack_attr2body()) [0x200000007:0x1:0x0]: returning size 4096 00000004:00200000:3.0:1776178153.841985:0:8250:0:(mdt_handler.c:1523:mdt_getattr_internal()) dirent count 2 stripe count 1 MDT count 2 00000004:00010000:3.0:1776178153.842004:0:8250:0:(mdt_handler.c:2353:mdt_getattr_name_lock()) ### Returning lock to client ns: mdt-lustre-MDT0000_UUID lock: 00000000dcea299c/0xa40083ab9bf5333b lrc: 3/1,0 mode: PR/PR res: [0x200000007:0x1:0x0].0x0 bits 0x13/0x0 rrc: 5 type: IBT gid 0 flags: 0x40210000000000 nid: local remote: 0x0 expref: -99 pid: 8250 timeout: 0 lvb_type: 0 00010000:00010000:3.0:1776178153.842031:0:8250:0:(ldlm_lock.c:216:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: mdt-lustre-MDT0000_UUID lock: 0000000055d6a00d/0xa40083ab9bf53334 lrc: 0/0,0 mode: --/CR res: [0x200000007:0x1:0x0].0x0 bits 0x2/0x0 rrc: 5 type: IBT gid 0 flags: 0x44000000000000 nid: 192.168.204.3@tcp remote: 0x771f68aa1eb27a8c expref: 8 pid: 8250 timeout: 0 lvb_type: 0 00010000:00010000:3.0:1776178153.842048:0:8250:0:(ldlm_lockd.c:1484:ldlm_handle_enqueue0()) ### server-side enqueue handler, sending reply (err=0, rc=0) ns: mdt-lustre-MDT0000_UUID lock: 00000000dcea299c/0xa40083ab9bf5333b lrc: 3/0,0 mode: PR/PR res: [0x200000007:0x1:0x0].0x0 bits 0x13/0x0 rrc: 4 type: IBT gid 0 flags: 0x40200000000000 nid: 192.168.204.3@tcp remote: 0x771f68aa1eb27a8c expref: 7 pid: 8250 timeout: 0 lvb_type: 0 00010000:00010000:3.0:1776178153.842083:0:8250:0:(ldlm_inodebits.c:94:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x200000007:0x1:0x0].0x0 (00000000ad0d4094) 00010000:00010000:3.0:1776178153.842087:0:8250:0:(ldlm_inodebits.c:96:ldlm_reprocess_inodebits_queue()) Hint 13 00010000:00010000:3.0:1776178153.842094:0:8250:0:(ldlm_lockd.c:1565:ldlm_handle_enqueue0()) ### server-side enqueue handler END (lock 00000000dcea299c, rc 0) 00000100:00100000:3.0:1776178153.842157:0:8250:0:(service.c:2372:ptlrpc_server_handle_request()) Handled RPC req@000000002a321d27 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_003:b50963bd-8eba-4432-9a54-8888d00e3fa1+7:17614:x1862456979089472:12345-192.168.204.3@tcp:101:lfs.0 Request processed in 384us (747us total) trans 0 rc 0/0 00000100:00100000:3.0:1776178153.842165:0:8250:0:(nrs_fifo.c:241:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.204.3@tcp, seq: 6526 00000100:00100000:3.0:1776178153.852873:0:3347:0:(events.c:361:request_in_callback()) peer: 12345-192.168.204.3@tcp (source: 12345-192.168.204.3@tcp) 00000100:00100000:3.0:1776178153.852921:0:3347:0:(events.c:361:request_in_callback()) peer: 12345-192.168.204.3@tcp (source: 12345-192.168.204.3@tcp) 00000100:00100000:1.0:1776178153.853023:0:8250:0:(service.c:2152:ptlrpc_server_handle_req_in()) got req x1862456979089664 00000100:00100000:2.0:1776178153.853042:0:6256:0:(service.c:2152:ptlrpc_server_handle_req_in()) got req x1862456979089536 00000100:00100000:1.0:1776178153.853115:0:8250:0:(nrs_fifo.c:179:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.204.3@tcp, seq: 6527 00000100:00100000:1.0:1776178153.853122:0:8250:0:(service.c:2324:ptlrpc_server_handle_request()) Handling RPC req@000000009b5eeec2 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_003:b50963bd-8eba-4432-9a54-8888d00e3fa1+7:17614:x1862456979089664:12345-192.168.204.3@tcp:36:lfs.0 00000100:00100000:2.0:1776178153.853233:0:6256:0:(nrs_fifo.c:179:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.204.3@tcp, seq: 6528 00000100:00100000:2.0:1776178153.853241:0:6256:0:(service.c:2324:ptlrpc_server_handle_request()) Handling RPC req@0000000008b3a3ad pname:cluuid+ref:pid:xid:nid:opc:job mdt00_001:b50963bd-8eba-4432-9a54-8888d00e3fa1+7:2271:x1862456979089536:12345-192.168.204.3@tcp:41:lfs.0 00000100:00100000:1.0:1776178153.853248:0:8250:0:(client.c:1733:ptlrpc_send_new_req()) Sending RPC req@0000000021d5d4e8 pname:cluuid:pid:xid:nid:opc:job mdt00_003:lustre-MDT0001-mdtlov_UUID:8250:1862456998033856:0@lo:1000:mdt00_003.0 00000004:00000024:2.0:1776178153.853274:0:6256:0:(mdt_handler.c:548:mdt_statfs()) blocks cached 0 granted 2097152 pending 0 free 1446039552 avail 1316577280 00000100:00100000:1.0:1776178153.853318:0:8250:0:(events.c:361:request_in_callback()) peer: 12345-0@lo (source: 12345-0@lo) 00000100:00100000:1.0:1776178153.853341:0:8250:0:(client.c:2500:ptlrpc_set_wait()) set 00000000856ca387 going to sleep for 42 seconds 00000100:00100000:2.0:1776178153.853404:0:6256:0:(service.c:2372:ptlrpc_server_handle_request()) Handled RPC req@0000000008b3a3ad pname:cluuid+ref:pid:xid:nid:opc:job mdt00_001:b50963bd-8eba-4432-9a54-8888d00e3fa1+7:2271:x1862456979089536:12345-192.168.204.3@tcp:41:lfs.0 Request processed in 161us (481us total) trans 0 rc 0/0 00000100:00100000:2.0:1776178153.853413:0:6256:0:(nrs_fifo.c:241:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.204.3@tcp, seq: 6528 00000100:00100000:2.0:1776178153.853445:0:14128:0:(service.c:2152:ptlrpc_server_handle_req_in()) got req x1862456998033856 00000100:00100000:2.0:1776178153.853475:0:14128:0:(nrs_fifo.c:179:nrs_fifo_req_get()) NRS start fifo request from 12345-0@lo, seq: 648 00000100:00100000:2.0:1776178153.853480:0:14128:0:(service.c:2324:ptlrpc_server_handle_request()) Handling RPC req@000000004420855e pname:cluuid+ref:pid:xid:nid:opc:job mdt_out00_002:lustre-MDT0001-mdtlov_UUID+6:8250:x1862456998033856:12345-0@lo:1000:mdt00_003.0 00000100:00100000:2.0:1776178153.853546:0:14128:0:(service.c:2372:ptlrpc_server_handle_request()) Handled RPC req@000000004420855e pname:cluuid+ref:pid:xid:nid:opc:job mdt_out00_002:lustre-MDT0001-mdtlov_UUID+6:8250:x1862456998033856:12345-0@lo:1000:mdt00_003.0 Request processed in 66us (229us total) trans 0 rc 0/0 00000100:00100000:2.0:1776178153.853553:0:14128:0:(nrs_fifo.c:241:nrs_fifo_req_stop()) NRS stop fifo request from 12345-0@lo, seq: 648 00000100:00100000:1.0:1776178153.855016:0:8250:0:(client.c:2215:ptlrpc_check_set()) Completed RPC req@0000000021d5d4e8 pname:cluuid:pid:xid:nid:opc:job mdt00_003:lustre-MDT0001-mdtlov_UUID:8250:1862456998033856:0@lo:1000:mdt00_003.0 00000100:00100000:1.0:1776178153.855129:0:8250:0:(client.c:1733:ptlrpc_send_new_req()) Sending RPC req@0000000021d5d4e8 pname:cluuid:pid:xid:nid:opc:job mdt00_003:lustre-MDT0001-mdtlov_UUID:8250:1862456998033920:0@lo:1000:mdt00_003.0 00000100:00100000:1.0:1776178153.855177:0:8250:0:(events.c:361:request_in_callback()) peer: 12345-0@lo (source: 12345-0@lo) 00000100:00100000:1.0:1776178153.855197:0:8250:0:(client.c:2500:ptlrpc_set_wait()) set 00000000856ca387 going to sleep for 42 seconds 00000100:00100000:2.0:1776178153.855941:0:6261:0:(service.c:2152:ptlrpc_server_handle_req_in()) got req x1862456998033920 00000100:00100000:2.0:1776178153.856012:0:6261:0:(nrs_fifo.c:179:nrs_fifo_req_get()) NRS start fifo request from 12345-0@lo, seq: 649 00000100:00100000:2.0:1776178153.856019:0:6261:0:(service.c:2324:ptlrpc_server_handle_request()) Handling RPC req@00000000a6227918 pname:cluuid+ref:pid:xid:nid:opc:job mdt_out00_001:lustre-MDT0001-mdtlov_UUID+6:8250:x1862456998033920:12345-0@lo:1000:mdt00_003.0 00000100:00100000:2.0:1776178153.856160:0:6261:0:(service.c:2372:ptlrpc_server_handle_request()) Handled RPC req@00000000a6227918 pname:cluuid+ref:pid:xid:nid:opc:job mdt_out00_001:lustre-MDT0001-mdtlov_UUID+6:8250:x1862456998033920:12345-0@lo:1000:mdt00_003.0 Request processed in 140us (983us total) trans 0 rc 0/0 00000100:00100000:2.0:1776178153.856168:0:6261:0:(nrs_fifo.c:241:nrs_fifo_req_stop()) NRS stop fifo request from 12345-0@lo, seq: 649 00000100:00100000:3.0:1776178153.856675:0:8250:0:(client.c:2215:ptlrpc_check_set()) Completed RPC req@0000000021d5d4e8 pname:cluuid:pid:xid:nid:opc:job mdt00_003:lustre-MDT0001-mdtlov_UUID:8250:1862456998033920:0@lo:1000:mdt00_003.0 00000100:00100000:3.0:1776178153.856752:0:8250:0:(client.c:1733:ptlrpc_send_new_req()) Sending RPC req@0000000092028508 pname:cluuid:pid:xid:nid:opc:job mdt00_003:lustre-MDT0001-mdtlov_UUID:8250:1862456998033984:0@lo:1000:mdt00_003.0 00000100:00100000:3.0:1776178153.856793:0:8250:0:(events.c:361:request_in_callback()) peer: 12345-0@lo (source: 12345-0@lo) 00000100:00100000:3.0:1776178153.856818:0:8250:0:(client.c:2500:ptlrpc_set_wait()) set 00000000f1769bda going to sleep for 42 seconds 00000100:00100000:2.0:1776178153.857084:0:6261:0:(service.c:2152:ptlrpc_server_handle_req_in()) got req x1862456998033984 00000100:00100000:2.0:1776178153.857116:0:6261:0:(nrs_fifo.c:179:nrs_fifo_req_get()) NRS start fifo request from 12345-0@lo, seq: 650 00000100:00100000:2.0:1776178153.857121:0:6261:0:(service.c:2324:ptlrpc_server_handle_request()) Handling RPC req@00000000f586aea3 pname:cluuid+ref:pid:xid:nid:opc:job mdt_out00_001:lustre-MDT0001-mdtlov_UUID+6:8250:x1862456998033984:12345-0@lo:1000:mdt00_003.0 00000100:00100000:2.0:1776178153.857208:0:6261:0:(service.c:2372:ptlrpc_server_handle_request()) Handled RPC req@00000000f586aea3 pname:cluuid+ref:pid:xid:nid:opc:job mdt_out00_001:lustre-MDT0001-mdtlov_UUID+6:8250:x1862456998033984:12345-0@lo:1000:mdt00_003.0 Request processed in 86us (415us total) trans 0 rc -2/-2 00000100:00100000:2.0:1776178153.857216:0:6261:0:(nrs_fifo.c:241:nrs_fifo_req_stop()) NRS stop fifo request from 12345-0@lo, seq: 650 00000100:00100000:3.0:1776178153.858848:0:8250:0:(client.c:2215:ptlrpc_check_set()) Completed RPC req@0000000092028508 pname:cluuid:pid:xid:nid:opc:job mdt00_003:lustre-MDT0001-mdtlov_UUID:8250:1862456998033984:0@lo:1000:mdt00_003.0 00010000:00010000:3.0:1776178153.858927:0:8250:0:(ldlm_lock.c:766:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(EX) ns: lustre-MDT0000-osp-MDT0001 lock: 0000000055d6a00d/0xa40083ab9bf53342 lrc: 3/0,1 mode: --/EX res: [0x200000007:0x1:0x0].0x0 bits 0x0/0x0 rrc: 3 type: IBT gid 0 flags: 0x10000000000000 nid: local remote: 0x0 expref: -99 pid: 8250 timeout: 0 lvb_type: 0 00010000:00010000:3.0:1776178153.858943:0:8250:0:(ldlm_request.c:1026:ldlm_cli_enqueue()) ### client-side enqueue START, flags 0x1000001000000 ns: lustre-MDT0000-osp-MDT0001 lock: 0000000055d6a00d/0xa40083ab9bf53342 lrc: 3/0,1 mode: --/EX res: [0x200000007:0x1:0x0].0x0 bits 0x2/0x0 rrc: 3 type: IBT gid 0 flags: 0x0 nid: local remote: 0x0 expref: -99 pid: 8250 timeout: 0 lvb_type: 0 00010000:00010000:3.0:1776178153.858954:0:8250:0:(ldlm_request.c:1114:ldlm_cli_enqueue()) ### sending request ns: lustre-MDT0000-osp-MDT0001 lock: 0000000055d6a00d/0xa40083ab9bf53342 lrc: 3/0,1 mode: --/EX res: [0x200000007:0x1:0x0].0x0 bits 0x2/0x0 rrc: 3 type: IBT gid 0 flags: 0x1000000000000 nid: local remote: 0x0 expref: -99 pid: 8250 timeout: 0 lvb_type: 0 00000100:00100000:3.0:1776178153.858971:0:8250:0:(client.c:1733:ptlrpc_send_new_req()) Sending RPC req@000000006c7327ca pname:cluuid:pid:xid:nid:opc:job mdt00_003:lustre-MDT0001-mdtlov_UUID:8250:1862456998034048:0@lo:101:mdt00_003.0 00000100:00100000:3.0:1776178153.859019:0:8250:0:(events.c:361:request_in_callback()) peer: 12345-0@lo (source: 12345-0@lo) 00000100:00100000:3.0:1776178153.859055:0:8250:0:(client.c:2500:ptlrpc_set_wait()) set 00000000b791c5de going to sleep for 42 seconds 00000100:00100000:2.0:1776178153.859653:0:14128:0:(service.c:2152:ptlrpc_server_handle_req_in()) got req x1862456998034048 00000100:00100000:2.0:1776178153.859714:0:14128:0:(nrs_fifo.c:179:nrs_fifo_req_get()) NRS start fifo request from 12345-0@lo, seq: 651 00000100:00100000:2.0:1776178153.859719:0:14128:0:(service.c:2324:ptlrpc_server_handle_request()) Handling RPC req@000000006f8bc860 pname:cluuid+ref:pid:xid:nid:opc:job mdt_out00_002:lustre-MDT0001-mdtlov_UUID+6:8250:x1862456998034048:12345-0@lo:101:mdt00_003.0 00010000:00010000:2.0:1776178153.859734:0:14128:0:(ldlm_lockd.c:1248:ldlm_handle_enqueue0()) ### server-side enqueue handler START 00010000:00010000:2.0:1776178153.859753:0:14128:0:(ldlm_lockd.c:1318:ldlm_handle_enqueue0()) ### server-side enqueue handler, new lock created ns: mdt-lustre-MDT0000_UUID lock: 00000000fcb32840/0xa40083ab9bf53349 lrc: 2/0,0 mode: --/EX res: [0x200000007:0x1:0x0].0x0 bits 0x0/0x0 rrc: 5 type: IBT gid 0 flags: 0x40000000000000 nid: local remote: 0xa40083ab9bf53342 expref: -99 pid: 14128 timeout: 0 lvb_type: 0 00010000:00010000:2.0:1776178153.859788:0:14128:0:(ldlm_lock.c:675:ldlm_add_bl_work_item()) ### lock incompatible; sending blocking AST. ns: mdt-lustre-MDT0000_UUID lock: 00000000dcea299c/0xa40083ab9bf5333b lrc: 2/0,0 mode: PR/PR res: [0x200000007:0x1:0x0].0x0 bits 0x13/0x0 rrc: 5 type: IBT gid 0 flags: 0x40200000000000 nid: 192.168.204.3@tcp remote: 0x771f68aa1eb27a8c expref: 6 pid: 8250 timeout: 0 lvb_type: 0 00010000:00010000:2.0:1776178153.859796:0:14128:0:(ldlm_resource.c:1650:ldlm_resource_add_lock()) ### About to add this lock ns: mdt-lustre-MDT0000_UUID lock: 00000000fcb32840/0xa40083ab9bf53349 lrc: 4/0,0 mode: --/EX res: [0x200000007:0x1:0x0].0x0 bits 0x2/0x0 rrc: 5 type: IBT gid 0 flags: 0x50000001000000 nid: 0@lo remote: 0xa40083ab9bf53342 expref: 7 pid: 14128 timeout: 0 lvb_type: 0 00010000:00010000:2.0:1776178153.859832:0:14128:0:(ldlm_lockd.c:953:ldlm_server_blocking_ast()) ### server preparing blocking AST ns: mdt-lustre-MDT0000_UUID lock: 00000000dcea299c/0xa40083ab9bf5333b lrc: 3/0,0 mode: PR/PR res: [0x200000007:0x1:0x0].0x0 bits 0x13/0x0 rrc: 5 type: IBT gid 0 flags: 0x50200000000020 nid: 192.168.204.3@tcp remote: 0x771f68aa1eb27a8c expref: 6 pid: 8250 timeout: 0 lvb_type: 0 00010000:00010000:2.0:1776178153.859844:0:14128:0:(ldlm_lockd.c:489:ldlm_add_waiting_lock()) ### adding to wait list(timeout: 100, AT: on) ns: mdt-lustre-MDT0000_UUID lock: 00000000dcea299c/0xa40083ab9bf5333b lrc: 4/0,0 mode: PR/PR res: [0x200000007:0x1:0x0].0x0 bits 0x13/0x0 rrc: 5 type: IBT gid 0 flags: 0x70200400000020 nid: 192.168.204.3@tcp remote: 0x771f68aa1eb27a8c expref: 6 pid: 8250 timeout: 961 lvb_type: 0 00000100:00100000:2.0:1776178153.859854:0:14128:0:(client.c:1733:ptlrpc_send_new_req()) Sending RPC req@000000007a321680 pname:cluuid:pid:xid:nid:opc:job mdt_out00_002:lustre-MDT0000_UUID:14128:1862456998034112:192.168.204.3@tcp:104: 00000100:00100000:2.0:1776178153.859905:0:14128:0:(client.c:2500:ptlrpc_set_wait()) set 00000000bd147a94 going to sleep for 6 seconds 00000100:00100000:2.0:1776178153.867133:0:3347:0:(events.c:361:request_in_callback()) peer: 12345-192.168.204.3@tcp (source: 12345-192.168.204.3@tcp) 00000100:00100000:2.0:1776178153.867186:0:6242:0:(service.c:2152:ptlrpc_server_handle_req_in()) got req x1862456979089728 00010000:00010000:2.0:1776178153.867227:0:6242:0:(ldlm_lockd.c:2638:ldlm_cancel_hpreq_check()) ### hpreq cancel/convert lock ns: mdt-lustre-MDT0000_UUID lock: 00000000dcea299c/0xa40083ab9bf5333b lrc: 5/0,0 mode: PR/PR res: [0x200000007:0x1:0x0].0x0 bits 0x13/0x0 rrc: 5 type: IBT gid 0 flags: 0x60200400000020 nid: 192.168.204.3@tcp remote: 0x771f68aa1eb27a8c expref: 7 pid: 8250 timeout: 961 lvb_type: 0 00000100:00100000:2.0:1776178153.867272:0:6242:0:(nrs_fifo.c:179:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.204.3@tcp, seq: 2080 00000100:00100000:2.0:1776178153.867280:0:6242:0:(service.c:2324:ptlrpc_server_handle_request()) Handling RPC req@00000000b7aa289e pname:cluuid+ref:pid:xid:nid:opc:job ldlm_cn00_001:b50963bd-8eba-4432-9a54-8888d00e3fa1+7:2271:x1862456979089728:12345-192.168.204.3@tcp:103:ldlm_bl_02.0 00010000:00010000:2.0:1776178153.867299:0:6242:0:(ldlm_lockd.c:1720:ldlm_request_cancel()) ### server-side cancel handler START: 1 locks, starting at 0 00010000:00010000:2.0:1776178153.867307:0:6242:0:(ldlm_lockd.c:1771:ldlm_request_cancel()) ### server cancels blocked lock after 0s ns: mdt-lustre-MDT0000_UUID lock: 00000000dcea299c/0xa40083ab9bf5333b lrc: 5/0,0 mode: PR/PR res: [0x200000007:0x1:0x0].0x0 bits 0x13/0x0 rrc: 6 type: IBT gid 0 flags: 0x60200400000020 nid: 192.168.204.3@tcp remote: 0x771f68aa1eb27a8c expref: 7 pid: 8250 timeout: 961 lvb_type: 0 00010000:00010000:2.0:1776178153.867320:0:6242:0:(ldlm_lockd.c:562:ldlm_del_waiting_lock()) ### removed ns: mdt-lustre-MDT0000_UUID lock: 00000000dcea299c/0xa40083ab9bf5333b lrc: 4/0,0 mode: PR/PR res: [0x200000007:0x1:0x0].0x0 bits 0x13/0x0 rrc: 6 type: IBT gid 0 flags: 0x50200400000020 nid: 192.168.204.3@tcp remote: 0x771f68aa1eb27a8c expref: 7 pid: 8250 timeout: 961 lvb_type: 0 00010000:00010000:2.0:1776178153.867347:0:6242:0:(ldlm_inodebits.c:94:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x200000007:0x1:0x0].0x0 (00000000ad0d4094) 00010000:00010000:2.0:1776178153.867354:0:6242:0:(ldlm_inodebits.c:113:ldlm_reprocess_inodebits_queue()) ### Reprocessing lock from queue 1 ns: mdt-lustre-MDT0000_UUID lock: 00000000fcb32840/0xa40083ab9bf53349 lrc: 3/0,0 mode: --/EX res: [0x200000007:0x1:0x0].0x0 bits 0x2/0x0 rrc: 6 type: IBT gid 0 flags: 0x50000001000000 nid: 0@lo remote: 0xa40083ab9bf53342 expref: 7 pid: 14128 timeout: 0 lvb_type: 0 00010000:00010000:2.0:1776178153.867367:0:6242:0:(ldlm_lock.c:704:ldlm_add_cp_work_item()) ### lock granted; sending completion AST. ns: mdt-lustre-MDT0000_UUID lock: 00000000fcb32840/0xa40083ab9bf53349 lrc: 3/0,0 mode: EX/EX res: [0x200000007:0x1:0x0].0x0 bits 0x2/0x0 rrc: 6 type: IBT gid 0 flags: 0x50080001000000 nid: 0@lo remote: 0xa40083ab9bf53342 expref: 7 pid: 14128 timeout: 0 lvb_type: 0 00010000:00010000:2.0:1776178153.867377:0:6242:0:(ldlm_lock.c:1064:ldlm_granted_list_add_lock()) ### About to add lock: ns: mdt-lustre-MDT0000_UUID lock: 00000000fcb32840/0xa40083ab9bf53349 lrc: 4/0,0 mode: EX/EX res: [0x200000007:0x1:0x0].0x0 bits 0x2/0x0 rrc: 6 type: IBT gid 0 flags: 0x50080001000000 nid: 0@lo remote: 0xa40083ab9bf53342 expref: 7 pid: 14128 timeout: 0 lvb_type: 0 00010000:00010000:3.0:1776178153.867912:0:14128:0:(ldlm_lock.c:216:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: mdt-lustre-MDT0000_UUID lock: 00000000dcea299c/0xa40083ab9bf5333b lrc: 0/0,0 mode: --/PR res: [0x200000007:0x1:0x0].0x0 bits 0x13/0x0 rrc: 6 type: IBT gid 0 flags: 0x44a01400000020 nid: 192.168.204.3@tcp remote: 0x771f68aa1eb27a8c expref: 7 pid: 8250 timeout: 961 lvb_type: 0 00000100:00100000:3.0:1776178153.867931:0:14128:0:(client.c:2215:ptlrpc_check_set()) Completed RPC req@000000007a321680 pname:cluuid:pid:xid:nid:opc:job mdt_out00_002:lustre-MDT0000_UUID:14128:1862456998034112:192.168.204.3@tcp:104: 00010000:00010000:3.0:1776178153.867959:0:14128:0:(ldlm_lockd.c:1484:ldlm_handle_enqueue0()) ### server-side enqueue handler, sending reply (err=0, rc=0) ns: mdt-lustre-MDT0000_UUID lock: 00000000fcb32840/0xa40083ab9bf53349 lrc: 4/0,0 mode: EX/EX res: [0x200000007:0x1:0x0].0x0 bits 0x2/0x0 rrc: 5 type: IBT gid 0 flags: 0x40000001000000 nid: 0@lo remote: 0xa40083ab9bf53342 expref: 7 pid: 14128 timeout: 0 lvb_type: 0 00010000:00010000:3.0:1776178153.867966:0:14128:0:(ldlm_inodebits.c:94:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x200000007:0x1:0x0].0x0 (00000000ad0d4094) 00010000:00010000:3.0:1776178153.867969:0:14128:0:(ldlm_inodebits.c:96:ldlm_reprocess_inodebits_queue()) Hint 2 00010000:00010000:3.0:1776178153.867971:0:14128:0:(ldlm_lockd.c:1565:ldlm_handle_enqueue0()) ### server-side enqueue handler END (lock 00000000fcb32840, rc 0) 00000100:00100000:3.0:1776178153.868025:0:14128:0:(service.c:2372:ptlrpc_server_handle_request()) Handled RPC req@000000006f8bc860 pname:cluuid+ref:pid:xid:nid:opc:job mdt_out00_002:lustre-MDT0001-mdtlov_UUID+7:8250:x1862456998034048:12345-0@lo:101:mdt00_003.0 Request processed in 8305us (9006us total) trans 0 rc 0/0 00000100:00100000:3.0:1776178153.868033:0:14128:0:(nrs_fifo.c:241:nrs_fifo_req_stop()) NRS stop fifo request from 12345-0@lo, seq: 651 00000100:00100000:3.0:1776178153.868085:0:8250:0:(client.c:2215:ptlrpc_check_set()) Completed RPC req@000000006c7327ca pname:cluuid:pid:xid:nid:opc:job mdt00_003:lustre-MDT0001-mdtlov_UUID:8250:1862456998034048:0@lo:101:mdt00_003.0 00010000:00010000:3.0:1776178153.868101:0:8250:0:(ldlm_resource.c:1650:ldlm_resource_add_lock()) ### About to add this lock ns: lustre-MDT0000-osp-MDT0001 lock: 0000000055d6a00d/0xa40083ab9bf53342 lrc: 4/0,1 mode: --/EX res: [0x200000007:0x1:0x0].0x0 bits 0x2/0x0 rrc: 3 type: IBT gid 0 flags: 0x11000001000000 nid: local remote: 0xa40083ab9bf53349 expref: -99 pid: 8250 timeout: 0 lvb_type: 0 00010000:00010000:3.0:1776178153.868110:0:8250:0:(ldlm_request.c:283:ldlm_completion_ast()) ### client-side enqueue returned a blocked locksleeping ns: lustre-MDT0000-osp-MDT0001 lock: 0000000055d6a00d/0xa40083ab9bf53342 lrc: 4/0,1 mode: --/EX res: [0x200000007:0x1:0x0].0x0 bits 0x2/0x0 rrc: 3 type: IBT gid 0 flags: 0x1000001000000 nid: local remote: 0xa40083ab9bf53349 expref: -99 pid: 8250 timeout: 0 lvb_type: 0 00010000:00010000:2.0:1776178153.869297:0:6242:0:(ldlm_lockd.c:1069:ldlm_server_completion_ast()) ### server preparing completion AST ns: mdt-lustre-MDT0000_UUID lock: 00000000fcb32840/0xa40083ab9bf53349 lrc: 3/0,0 mode: EX/EX res: [0x200000007:0x1:0x0].0x0 bits 0x2/0x0 rrc: 5 type: IBT gid 0 flags: 0x40000001000000 nid: 0@lo remote: 0xa40083ab9bf53342 expref: 6 pid: 14128 timeout: 0 lvb_type: 0 00000100:00100000:2.0:1776178153.869324:0:6242:0:(client.c:1733:ptlrpc_send_new_req()) Sending RPC req@0000000045560455 pname:cluuid:pid:xid:nid:opc:job ldlm_cn00_001:lustre-MDT0000_UUID:6242:1862456998034176:0@lo:105: 00000100:00100000:2.0:1776178153.869381:0:6242:0:(events.c:361:request_in_callback()) peer: 12345-0@lo (source: 12345-0@lo) 00000100:00100000:2.0:1776178153.869763:0:6242:0:(client.c:2500:ptlrpc_set_wait()) set 00000000f8ca7cd7 going to sleep for 6 seconds 00000100:00100000:0.0:1776178153.869969:0:6240:0:(service.c:2152:ptlrpc_server_handle_req_in()) got req x1862456998034176 00000100:00100000:0.0:1776178153.870031:0:6240:0:(nrs_fifo.c:179:nrs_fifo_req_get()) NRS start fifo request from 12345-0@lo, seq: 65 00000100:00100000:0.0:1776178153.870038:0:6240:0:(service.c:2324:ptlrpc_server_handle_request()) Handling RPC req@0000000056080284 pname:cluuid+ref:pid:xid:nid:opc:job ldlm_cb00_001:lustre-MDT0001-mdtlov_UUID+4:6242:x1862456998034176:12345-0@lo:105: 00010000:00010000:0.0:1776178153.870050:0:6240:0:(ldlm_lockd.c:2480:ldlm_callback_handler()) ### completion ast ns: lustre-MDT0000-osp-MDT0001 lock: 0000000055d6a00d/0xa40083ab9bf53342 lrc: 5/0,1 mode: --/EX res: [0x200000007:0x1:0x0].0x0 bits 0x2/0x0 rrc: 3 type: IBT gid 0 flags: 0x1000001000000 nid: local remote: 0xa40083ab9bf53349 expref: -99 pid: 8250 timeout: 0 lvb_type: 0 00010000:00010000:0.0:1776178153.870098:0:6240:0:(ldlm_lockd.c:1938:ldlm_handle_cp_callback()) ### client completion callback handler START ns: lustre-MDT0000-osp-MDT0001 lock: 0000000055d6a00d/0xa40083ab9bf53342 lrc: 5/0,1 mode: --/EX res: [0x200000007:0x1:0x0].0x0 bits 0x2/0x0 rrc: 3 type: IBT gid 0 flags: 0x1000001000000 nid: local remote: 0xa40083ab9bf53349 expref: -99 pid: 8250 timeout: 0 lvb_type: 0 00010000:00010000:0.0:1776178153.870107:0:6240:0:(ldlm_lockd.c:2015:ldlm_handle_cp_callback()) ### completion AST, new policy data ns: lustre-MDT0000-osp-MDT0001 lock: 0000000055d6a00d/0xa40083ab9bf53342 lrc: 5/0,1 mode: --/EX res: [0x200000007:0x1:0x0].0x0 bits 0x2/0x0 rrc: 3 type: IBT gid 0 flags: 0x11000001000000 nid: local remote: 0xa40083ab9bf53349 expref: -99 pid: 8250 timeout: 0 lvb_type: 0 00010000:00010000:0.0:1776178153.870116:0:6240:0:(ldlm_lock.c:704:ldlm_add_cp_work_item()) ### lock granted; sending completion AST. ns: lustre-MDT0000-osp-MDT0001 lock: 0000000055d6a00d/0xa40083ab9bf53342 lrc: 5/0,1 mode: EX/EX res: [0x200000007:0x1:0x0].0x0 bits 0x2/0x0 rrc: 3 type: IBT gid 0 flags: 0x11080001000000 nid: local remote: 0xa40083ab9bf53349 expref: -99 pid: 8250 timeout: 0 lvb_type: 0 00010000:00010000:0.0:1776178153.870123:0:6240:0:(ldlm_lock.c:1064:ldlm_granted_list_add_lock()) ### About to add lock: ns: lustre-MDT0000-osp-MDT0001 lock: 0000000055d6a00d/0xa40083ab9bf53342 lrc: 6/0,1 mode: EX/EX res: [0x200000007:0x1:0x0].0x0 bits 0x2/0x0 rrc: 3 type: IBT gid 0 flags: 0x11080001000000 nid: local remote: 0xa40083ab9bf53349 expref: -99 pid: 8250 timeout: 0 lvb_type: 0 00010000:00010000:0.0:1776178153.870131:0:6240:0:(ldlm_lockd.c:2043:ldlm_handle_cp_callback()) ### callback handler finished, about to run_ast_work ns: lustre-MDT0000-osp-MDT0001 lock: 0000000055d6a00d/0xa40083ab9bf53342 lrc: 6/0,1 mode: EX/EX res: [0x200000007:0x1:0x0].0x0 bits 0x2/0x0 rrc: 3 type: IBT gid 0 flags: 0x1080001000000 nid: local remote: 0xa40083ab9bf53349 expref: -99 pid: 8250 timeout: 0 lvb_type: 0 00010000:00010000:0.0:1776178153.870163:0:6240:0:(ldlm_lockd.c:2054:ldlm_handle_cp_callback()) ### client completion callback handler END (lock 0000000055d6a00d) 00000100:00100000:0.0:1776178153.870200:0:6240:0:(service.c:2372:ptlrpc_server_handle_request()) Handled RPC req@0000000056080284 pname:cluuid+ref:pid:xid:nid:opc:job ldlm_cb00_001:lustre-MDT0001-mdtlov_UUID+4:6242:x1862456998034176:12345-0@lo:105: Request processed in 162us (821us total) trans 0 rc 0/0 00000100:00100000:0.0:1776178153.870207:0:6240:0:(nrs_fifo.c:241:nrs_fifo_req_stop()) NRS stop fifo request from 12345-0@lo, seq: 65 00010000:00010000:3.0:1776178153.870279:0:8250:0:(ldlm_request.c:199:ldlm_completion_tail()) ### client-side enqueue: granted ns: lustre-MDT0000-osp-MDT0001 lock: 0000000055d6a00d/0xa40083ab9bf53342 lrc: 4/0,1 mode: EX/EX res: [0x200000007:0x1:0x0].0x0 bits 0x2/0x0 rrc: 3 type: IBT gid 0 flags: 0x1000001000000 nid: local remote: 0xa40083ab9bf53349 expref: -99 pid: 8250 timeout: 0 lvb_type: 0 00010000:00010000:3.0:1776178153.870287:0:8250:0:(ldlm_request.c:796:ldlm_cli_enqueue_fini()) ### client-side enqueue END ns: lustre-MDT0000-osp-MDT0001 lock: 0000000055d6a00d/0xa40083ab9bf53342 lrc: 4/0,1 mode: EX/EX res: [0x200000007:0x1:0x0].0x0 bits 0x2/0x0 rrc: 3 type: IBT gid 0 flags: 0x1000001000000 nid: local remote: 0xa40083ab9bf53349 expref: -99 pid: 8250 timeout: 0 lvb_type: 0 00000004:00080000:3.0:1776178153.870313:0:8250:0:(osp_object.c:1389:osp_invalidate()) Invalidate osp_object [0x200000007:0x1:0x0] 00000100:00100000:3.0:1776178153.870406:0:8250:0:(client.c:1733:ptlrpc_send_new_req()) Sending RPC req@000000006c7327ca pname:cluuid:pid:xid:nid:opc:job mdt00_003:lustre-MDT0001-mdtlov_UUID:8250:1862456998034240:0@lo:1000:mdt00_003.0 00000100:00100000:3.0:1776178153.870443:0:8250:0:(events.c:361:request_in_callback()) peer: 12345-0@lo (source: 12345-0@lo) 00000100:00100000:3.0:1776178153.870461:0:8250:0:(client.c:2500:ptlrpc_set_wait()) set 000000007f119eed going to sleep for 42 seconds 00000100:00100000:2.0:1776178153.870619:0:6242:0:(client.c:2215:ptlrpc_check_set()) Completed RPC req@0000000045560455 pname:cluuid:pid:xid:nid:opc:job ldlm_cn00_001:lustre-MDT0000_UUID:6242:1862456998034176:0@lo:105: 00010000:00010000:2.0:1776178153.870636:0:6242:0:(ldlm_lockd.c:1782:ldlm_request_cancel()) ### server-side cancel handler END 00000100:00100000:2.0:1776178153.871055:0:6242:0:(service.c:2372:ptlrpc_server_handle_request()) Handled RPC req@00000000b7aa289e pname:cluuid+ref:pid:xid:nid:opc:job ldlm_cn00_001:b50963bd-8eba-4432-9a54-8888d00e3fa1+6:2271:x1862456979089728:12345-192.168.204.3@tcp:103:ldlm_bl_02.0 Request processed in 3773us (3932us total) trans 0 rc 0/0 00000100:00100000:2.0:1776178153.871091:0:6242:0:(nrs_fifo.c:241:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.204.3@tcp, seq: 2080 00000100:00100000:0.0:1776178153.871305:0:14128:0:(service.c:2152:ptlrpc_server_handle_req_in()) got req x1862456998034240 00000100:00100000:0.0:1776178153.871398:0:14128:0:(nrs_fifo.c:179:nrs_fifo_req_get()) NRS start fifo request from 12345-0@lo, seq: 652 00000100:00100000:0.0:1776178153.871403:0:14128:0:(service.c:2324:ptlrpc_server_handle_request()) Handling RPC req@0000000092028508 pname:cluuid+ref:pid:xid:nid:opc:job mdt_out00_002:lustre-MDT0001-mdtlov_UUID+7:8250:x1862456998034240:12345-0@lo:1000:mdt00_003.0 00000100:00100000:0.0:1776178153.871488:0:14128:0:(service.c:2372:ptlrpc_server_handle_request()) Handled RPC req@0000000092028508 pname:cluuid+ref:pid:xid:nid:opc:job mdt_out00_002:lustre-MDT0001-mdtlov_UUID+7:8250:x1862456998034240:12345-0@lo:1000:mdt00_003.0 Request processed in 84us (1045us total) trans 0 rc 0/0 00000100:00100000:0.0:1776178153.871494:0:14128:0:(nrs_fifo.c:241:nrs_fifo_req_stop()) NRS stop fifo request from 12345-0@lo, seq: 652 00000100:00100000:2.0:1776178153.871633:0:8250:0:(client.c:2215:ptlrpc_check_set()) Completed RPC req@000000006c7327ca pname:cluuid:pid:xid:nid:opc:job mdt00_003:lustre-MDT0001-mdtlov_UUID:8250:1862456998034240:0@lo:1000:mdt00_003.0 00000100:00100000:2.0:1776178153.871713:0:8250:0:(client.c:1733:ptlrpc_send_new_req()) Sending RPC req@0000000045560455 pname:cluuid:pid:xid:nid:opc:job mdt00_003:lustre-MDT0001-mdtlov_UUID:8250:1862456998034304:0@lo:1000:mdt00_003.0 00000100:00100000:2.0:1776178153.871746:0:8250:0:(events.c:361:request_in_callback()) peer: 12345-0@lo (source: 12345-0@lo) 00000100:00100000:2.0:1776178153.875732:0:8250:0:(client.c:2500:ptlrpc_set_wait()) set 00000000bd147a94 going to sleep for 42 seconds 00000100:00100000:3.0:1776178153.877267:0:6261:0:(service.c:2152:ptlrpc_server_handle_req_in()) got req x1862456998034304 00000100:00100000:3.0:1776178153.877874:0:6261:0:(nrs_fifo.c:179:nrs_fifo_req_get()) NRS start fifo request from 12345-0@lo, seq: 653 00000100:00100000:3.0:1776178153.877882:0:6261:0:(service.c:2324:ptlrpc_server_handle_request()) Handling RPC req@000000006be5e5b9 pname:cluuid+ref:pid:xid:nid:opc:job mdt_out00_001:lustre-MDT0001-mdtlov_UUID+7:8250:x1862456998034304:12345-0@lo:1000:mdt00_003.0 00000100:00100000:3.0:1776178153.878025:0:6261:0:(service.c:2372:ptlrpc_server_handle_request()) Handled RPC req@000000006be5e5b9 pname:cluuid+ref:pid:xid:nid:opc:job mdt_out00_001:lustre-MDT0001-mdtlov_UUID+7:8250:x1862456998034304:12345-0@lo:1000:mdt00_003.0 Request processed in 142us (6278us total) trans 0 rc -2/-2 00000100:00100000:3.0:1776178153.878032:0:6261:0:(nrs_fifo.c:241:nrs_fifo_req_stop()) NRS stop fifo request from 12345-0@lo, seq: 653 00000100:00100000:2.0:1776178153.882017:0:8250:0:(client.c:2215:ptlrpc_check_set()) Completed RPC req@0000000045560455 pname:cluuid:pid:xid:nid:opc:job mdt00_003:lustre-MDT0001-mdtlov_UUID:8250:1862456998034304:0@lo:1000:mdt00_003.0 00000100:00100000:2.0:1776178153.882277:0:8250:0:(client.c:1733:ptlrpc_send_new_req()) Sending RPC req@0000000045560455 pname:cluuid:pid:xid:nid:opc:job mdt00_003:lustre-MDT0001-mdtlov_UUID:8250:1862456998034368:0@lo:1000:mdt00_003.0 00000100:00100000:2.0:1776178153.882339:0:8250:0:(events.c:361:request_in_callback()) peer: 12345-0@lo (source: 12345-0@lo) 00000100:00100000:2.0:1776178153.882363:0:8250:0:(client.c:2500:ptlrpc_set_wait()) set 00000000bd147a94 going to sleep for 42 seconds 00000100:00100000:2.0:1776178153.882388:0:15213:0:(service.c:2152:ptlrpc_server_handle_req_in()) got req x1862456998034368 00000100:00100000:2.0:1776178153.882431:0:15213:0:(nrs_fifo.c:179:nrs_fifo_req_get()) NRS start fifo request from 12345-0@lo, seq: 654 00000100:00100000:2.0:1776178153.882438:0:15213:0:(service.c:2324:ptlrpc_server_handle_request()) Handling RPC req@00000000a1262b14 pname:cluuid+ref:pid:xid:nid:opc:job mdt_out00_003:lustre-MDT0001-mdtlov_UUID+7:8250:x1862456998034368:12345-0@lo:1000:mdt00_003.0 00000100:00100000:2.0:1776178153.882505:0:15213:0:(service.c:2372:ptlrpc_server_handle_request()) Handled RPC req@00000000a1262b14 pname:cluuid+ref:pid:xid:nid:opc:job mdt_out00_003:lustre-MDT0001-mdtlov_UUID+7:8250:x1862456998034368:12345-0@lo:1000:mdt00_003.0 Request processed in 68us (168us total) trans 0 rc 0/0 00000100:00100000:2.0:1776178153.882515:0:15213:0:(nrs_fifo.c:241:nrs_fifo_req_stop()) NRS stop fifo request from 12345-0@lo, seq: 654 00000100:00100000:2.0:1776178153.882643:0:8250:0:(client.c:2215:ptlrpc_check_set()) Completed RPC req@0000000045560455 pname:cluuid:pid:xid:nid:opc:job mdt00_003:lustre-MDT0001-mdtlov_UUID:8250:1862456998034368:0@lo:1000:mdt00_003.0 00000100:00100000:2.0:1776178153.882701:0:8250:0:(client.c:1733:ptlrpc_send_new_req()) Sending RPC req@0000000045560455 pname:cluuid:pid:xid:nid:opc:job mdt00_003:lustre-MDT0001-mdtlov_UUID:8250:1862456998034432:0@lo:1000:mdt00_003.0 00000100:00100000:2.0:1776178153.882722:0:8250:0:(events.c:361:request_in_callback()) peer: 12345-0@lo (source: 12345-0@lo) 00000100:00100000:2.0:1776178153.882732:0:8250:0:(client.c:2500:ptlrpc_set_wait()) set 00000000bd147a94 going to sleep for 42 seconds 00000100:00100000:2.0:1776178153.882742:0:15213:0:(service.c:2152:ptlrpc_server_handle_req_in()) got req x1862456998034432 00000100:00100000:2.0:1776178153.882756:0:15213:0:(nrs_fifo.c:179:nrs_fifo_req_get()) NRS start fifo request from 12345-0@lo, seq: 655 00000100:00100000:2.0:1776178153.882760:0:15213:0:(service.c:2324:ptlrpc_server_handle_request()) Handling RPC req@00000000d668bb9c pname:cluuid+ref:pid:xid:nid:opc:job mdt_out00_003:lustre-MDT0001-mdtlov_UUID+7:8250:x1862456998034432:12345-0@lo:1000:mdt00_003.0 00000100:00100000:2.0:1776178153.882798:0:15213:0:(service.c:2372:ptlrpc_server_handle_request()) Handled RPC req@00000000d668bb9c pname:cluuid+ref:pid:xid:nid:opc:job mdt_out00_003:lustre-MDT0001-mdtlov_UUID+7:8250:x1862456998034432:12345-0@lo:1000:mdt00_003.0 Request processed in 37us (75us total) trans 0 rc 0/0 00000100:00100000:2.0:1776178153.882804:0:15213:0:(nrs_fifo.c:241:nrs_fifo_req_stop()) NRS stop fifo request from 12345-0@lo, seq: 655 00000100:00100000:2.0:1776178153.883666:0:8250:0:(client.c:2215:ptlrpc_check_set()) Completed RPC req@0000000045560455 pname:cluuid:pid:xid:nid:opc:job mdt00_003:lustre-MDT0001-mdtlov_UUID:8250:1862456998034432:0@lo:1000:mdt00_003.0 00000100:00100000:2.0:1776178153.885082:0:8250:0:(client.c:1733:ptlrpc_send_new_req()) Sending RPC req@0000000045560455 pname:cluuid:pid:xid:nid:opc:job mdt00_003:lustre-MDT0001-mdtlov_UUID:8250:1862456998034496:0@lo:1000:mdt00_003.0 00000100:00100000:2.0:1776178153.885155:0:8250:0:(events.c:361:request_in_callback()) peer: 12345-0@lo (source: 12345-0@lo) 00000100:00100000:2.0:1776178153.885188:0:8250:0:(client.c:2500:ptlrpc_set_wait()) set 00000000bd147a94 going to sleep for 42 seconds 00000100:00100000:0.0:1776178153.885746:0:14128:0:(service.c:2152:ptlrpc_server_handle_req_in()) got req x1862456998034496 00000100:00100000:0.0:1776178153.885804:0:14128:0:(nrs_fifo.c:179:nrs_fifo_req_get()) NRS start fifo request from 12345-0@lo, seq: 656 00000100:00100000:0.0:1776178153.885811:0:14128:0:(service.c:2324:ptlrpc_server_handle_request()) Handling RPC req@000000007a321680 pname:cluuid+ref:pid:xid:nid:opc:job mdt_out00_002:lustre-MDT0001-mdtlov_UUID+7:8250:x1862456998034496:12345-0@lo:1000:mdt00_003.0 00000100:00100000:0.0:1776178153.886046:0:14128:0:(service.c:2372:ptlrpc_server_handle_request()) Handled RPC req@000000007a321680 pname:cluuid+ref:pid:xid:nid:opc:job mdt_out00_002:lustre-MDT0001-mdtlov_UUID+7:8250:x1862456998034496:12345-0@lo:1000:mdt00_003.0 Request processed in 235us (892us total) trans 0 rc 0/0 00000100:00100000:0.0:1776178153.886053:0:14128:0:(nrs_fifo.c:241:nrs_fifo_req_stop()) NRS stop fifo request from 12345-0@lo, seq: 656 00000100:00100000:2.0:1776178153.886162:0:8250:0:(client.c:2215:ptlrpc_check_set()) Completed RPC req@0000000045560455 pname:cluuid:pid:xid:nid:opc:job mdt00_003:lustre-MDT0001-mdtlov_UUID:8250:1862456998034496:0@lo:1000:mdt00_003.0 00010000:00010000:2.0:1776178153.888246:0:9224:0:(ldlm_lockd.c:2792:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000040:00080000:2.0:1776178153.888298:0:8250:0:(llog_osd.c:738:llog_osd_write_rec()) added record [0x240001b70:0x1:0x0].9, 1040 off40336 00000040:00080000:2.0:1776178153.888365:0:8250:0:(llog_osd.c:738:llog_osd_write_rec()) added record [0x200000bd1:0x1:0x0].8, 1192 off38944 00000004:00080000:2.0:1776178153.888370:0:8250:0:(osp_trans.c:1598:osp_trans_stop()) lustre-MDT0000-osp-MDT0001: add oth 00000000f8ca7cd7 with version 20 00000004:00080000:2.0:1776178153.888387:0:18102:0:(osp_trans.c:1343:osp_get_next_request()) ou 000000009121b27e version 20 rpc_version 20 00000100:00100000:2.0:1776178153.888443:0:18102:0:(client.c:1733:ptlrpc_send_new_req()) Sending RPC req@0000000045560455 pname:cluuid:pid:xid:nid:opc:job osp_up0-1:lustre-MDT0001-mdtlov_UUID:18102:1862456998034560:0@lo:1000:osp_up0-1.0 00000100:00100000:2.0:1776178153.888510:0:18102:0:(events.c:361:request_in_callback()) peer: 12345-0@lo (source: 12345-0@lo) 00000100:00100000:2.0:1776178153.889783:0:18102:0:(client.c:2500:ptlrpc_set_wait()) set 000000000b150ab3 going to sleep for 42 seconds 00000100:00100000:0.0:1776178153.895206:0:14128:0:(service.c:2152:ptlrpc_server_handle_req_in()) got req x1862456998034560 00000100:00100000:0.0:1776178153.895288:0:14128:0:(nrs_fifo.c:179:nrs_fifo_req_get()) NRS start fifo request from 12345-0@lo, seq: 657 00000100:00100000:0.0:1776178153.895295:0:14128:0:(service.c:2324:ptlrpc_server_handle_request()) Handling RPC req@000000006c7327ca pname:cluuid+ref:pid:xid:nid:opc:job mdt_out00_002:lustre-MDT0001-mdtlov_UUID+7:18102:x1862456998034560:12345-0@lo:1000:osp_up0-1.0 00000100:00100000:0.0:1776178153.897099:0:14128:0:(service.c:2372:ptlrpc_server_handle_request()) Handled RPC req@000000006c7327ca pname:cluuid+ref:pid:xid:nid:opc:job mdt_out00_002:lustre-MDT0001-mdtlov_UUID+8:18102:x1862456998034560:12345-0@lo:1000:osp_up0-1.0 Request processed in 1794us (8581us total) trans 12884901944 rc 0/0 00000100:00100000:0.0:1776178153.897115:0:14128:0:(nrs_fifo.c:241:nrs_fifo_req_stop()) NRS stop fifo request from 12345-0@lo, seq: 657 00000100:00100000:0.0:1776178153.897202:0:18102:0:(client.c:2215:ptlrpc_check_set()) Completed RPC req@0000000045560455 pname:cluuid:pid:xid:nid:opc:job osp_up0-1:lustre-MDT0001-mdtlov_UUID:18102:1862456998034560:0@lo:1000:osp_up0-1.0 00010000:00010000:2.0:1776178153.899959:0:8250:0:(ldlm_lock.c:766:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: mdt-lustre-MDT0001_UUID lock: 000000007e7ea26f/0xa40083ab9bf53350 lrc: 3/0,1 mode: --/PW res: [0x240000402:0x7f7:0x0].0x0 bits 0x0/0x0 rrc: 2 type: IBT gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 8250 timeout: 0 lvb_type: 0 00010000:00010000:2.0:1776178153.899994:0:8250:0:(ldlm_lock.c:1064:ldlm_granted_list_add_lock()) ### About to add lock: ns: mdt-lustre-MDT0001_UUID lock: 000000007e7ea26f/0xa40083ab9bf53350 lrc: 3/0,1 mode: PW/PW res: [0x240000402:0x7f7:0x0].0x0 bits 0x2/0x0 rrc: 2 type: IBT gid 0 flags: 0x50210000000000 nid: local remote: 0x0 expref: -99 pid: 8250 timeout: 0 lvb_type: 0 00010000:00010000:2.0:1776178153.900015:0:8250:0:(ldlm_request.c:528:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: mdt-lustre-MDT0001_UUID lock: 000000007e7ea26f/0xa40083ab9bf53350 lrc: 3/0,1 mode: PW/PW res: [0x240000402:0x7f7:0x0].0x0 bits 0x2/0x0 rrc: 2 type: IBT gid 0 flags: 0x40210000000000 nid: local remote: 0x0 expref: -99 pid: 8250 timeout: 0 lvb_type: 0 00000004:00010000:2.0:1776178153.900034:0:8250:0:(mdt_handler.c:3992:mdt_save_lock()) ### save lock request 000000009b5eeec2 reply state 00000000a764b8cb transno 17179876233 ns: mdt-lustre-MDT0001_UUID lock: 000000007e7ea26f/0xa40083ab9bf53350 lrc: 3/0,1 mode: PW/PW res: [0x240000402:0x7f7:0x0].0x0 bits 0x2/0x0 rrc: 2 type: IBT gid 0 flags: 0x40210000000000 nid: local remote: 0x0 expref: -99 pid: 8250 timeout: 0 lvb_type: 0 00000004:00200000:2.0:1776178153.900047:0:8250:0:(mdt_handler.c:912:mdt_pack_attr2body()) [0x240000402:0x7f7:0x0]: returning size 4096 00000004:00080000:2.0:1776178153.900084:0:8250:0:(osp_object.c:1389:osp_invalidate()) Invalidate osp_object [0x200000007:0x1:0x0] 00010000:00010000:2.0:1776178153.900097:0:8250:0:(ldlm_lock.c:820:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(EX) ns: lustre-MDT0000-osp-MDT0001 lock: 0000000055d6a00d/0xa40083ab9bf53342 lrc: 4/0,1 mode: EX/EX res: [0x200000007:0x1:0x0].0x0 bits 0x2/0x0 rrc: 3 type: IBT gid 0 flags: 0x11000001000000 nid: local remote: 0xa40083ab9bf53349 expref: -99 pid: 8250 timeout: 0 lvb_type: 0 00010000:00010000:2.0:1776178153.900106:0:8250:0:(ldlm_lock.c:909:ldlm_lock_decref_internal()) ### do not add lock into lru list ns: lustre-MDT0000-osp-MDT0001 lock: 0000000055d6a00d/0xa40083ab9bf53342 lrc: 3/0,0 mode: EX/EX res: [0x200000007:0x1:0x0].0x0 bits 0x2/0x0 rrc: 3 type: IBT gid 0 flags: 0x11000001000000 nid: local remote: 0xa40083ab9bf53349 expref: -99 pid: 8250 timeout: 0 lvb_type: 0 00000100:00100000:2.0:1776178153.900210:0:8250:0:(service.c:2372:ptlrpc_server_handle_request()) Handled RPC req@000000009b5eeec2 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_003:b50963bd-8eba-4432-9a54-8888d00e3fa1+8:17614:x1862456979089664:12345-192.168.204.3@tcp:36:lfs.0 Request processed in 47084us (47343us total) trans 17179876233 rc 0/0 00000100:00100000:2.0:1776178153.900222:0:8250:0:(nrs_fifo.c:241:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.204.3@tcp, seq: 6527 00010000:00010000:1.0:1776178153.904327:0:3340:0:(ldlm_lock.c:1064:ldlm_granted_list_add_lock()) ### About to add lock: ns: mdt-lustre-MDT0001_UUID lock: 000000007e7ea26f/0xa40083ab9bf53350 lrc: 3/0,1 mode: COS/COS res: [0x240000402:0x7f7:0x0].0x0 bits 0x2/0x0 rrc: 2 type: IBT gid 0 flags: 0x50210000000000 nid: local remote: 0x0 expref: -99 pid: 8250 timeout: 0 lvb_type: 0 00010000:00010000:1.0:1776178153.904356:0:3340:0:(ldlm_inodebits.c:94:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x240000402:0x7f7:0x0].0x0 (000000006b762dc5) 00010000:00010000:1.0:1776178153.904360:0:3340:0:(ldlm_inodebits.c:96:ldlm_reprocess_inodebits_queue()) Hint 2 00000100:00100000:3.0:1776178153.948336:0:3348:0:(events.c:361:request_in_callback()) peer: 12345-192.168.204.3@tcp (source: 12345-192.168.204.3@tcp) 00000100:00100000:2.0:1776178153.948441:0:8250:0:(service.c:2152:ptlrpc_server_handle_req_in()) got req x1862456979089856 00000100:00100000:2.0:1776178153.948507:0:8250:0:(nrs_fifo.c:179:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.204.3@tcp, seq: 6529 00000100:00100000:2.0:1776178153.948514:0:8250:0:(service.c:2324:ptlrpc_server_handle_request()) Handling RPC req@0000000068adf51c pname:cluuid+ref:pid:xid:nid:opc:job mdt00_003:b50963bd-8eba-4432-9a54-8888d00e3fa1+6:17618:x1862456979089856:12345-192.168.204.3@tcp:101:lfs.0 00010000:00010000:2.0:1776178153.948543:0:8250:0:(ldlm_lockd.c:1248:ldlm_handle_enqueue0()) ### server-side enqueue handler START 00010000:00010000:2.0:1776178153.948570:0:8250:0:(ldlm_lockd.c:1318:ldlm_handle_enqueue0()) ### server-side enqueue handler, new lock created ns: mdt-lustre-MDT0000_UUID lock: 00000000d3e952db/0xa40083ab9bf53357 lrc: 2/0,0 mode: --/CR res: [0x200000007:0x1:0x0].0x0 bits 0x0/0x0 rrc: 5 type: IBT gid 0 flags: 0x40000000000000 nid: local remote: 0x771f68aa1eb27a93 expref: -99 pid: 8250 timeout: 0 lvb_type: 0 00010000:00010000:2.0:1776178153.948650:0:8250:0:(ldlm_lock.c:766:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(CR) ns: mdt-lustre-MDT0000_UUID lock: 00000000b27cf00e/0xa40083ab9bf5335e lrc: 3/1,0 mode: --/CR res: [0x200000007:0x1:0x0].0x0 bits 0x0/0x0 rrc: 6 type: IBT gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 8250 timeout: 0 lvb_type: 0 00010000:00010000:2.0:1776178153.948680:0:8250:0:(ldlm_lock.c:675:ldlm_add_bl_work_item()) ### lock incompatible; sending blocking AST. ns: mdt-lustre-MDT0000_UUID lock: 00000000fcb32840/0xa40083ab9bf53349 lrc: 2/0,0 mode: EX/EX res: [0x200000007:0x1:0x0].0x0 bits 0x2/0x0 rrc: 6 type: IBT gid 0 flags: 0x40000001000000 nid: 0@lo remote: 0xa40083ab9bf53342 expref: 7 pid: 14128 timeout: 0 lvb_type: 0 00010000:00010000:2.0:1776178153.948689:0:8250:0:(ldlm_resource.c:1650:ldlm_resource_add_lock()) ### About to add this lock ns: mdt-lustre-MDT0000_UUID lock: 00000000b27cf00e/0xa40083ab9bf5335e lrc: 4/1,0 mode: --/CR res: [0x200000007:0x1:0x0].0x0 bits 0x2/0x0 rrc: 6 type: IBT gid 0 flags: 0x50210000000000 nid: local remote: 0x0 expref: -99 pid: 8250 timeout: 0 lvb_type: 0 00010000:00010000:2.0:1776178153.948731:0:8250:0:(ldlm_lockd.c:953:ldlm_server_blocking_ast()) ### server preparing blocking AST ns: mdt-lustre-MDT0000_UUID lock: 00000000fcb32840/0xa40083ab9bf53349 lrc: 3/0,0 mode: EX/EX res: [0x200000007:0x1:0x0].0x0 bits 0x2/0x0 rrc: 6 type: IBT gid 0 flags: 0x50000001000020 nid: 0@lo remote: 0xa40083ab9bf53342 expref: 7 pid: 14128 timeout: 0 lvb_type: 0 00000100:00100000:2.0:1776178153.948750:0:8250:0:(client.c:1733:ptlrpc_send_new_req()) Sending RPC req@0000000098e424d0 pname:cluuid:pid:xid:nid:opc:job mdt00_003:lustre-MDT0000_UUID:8250:1862456998034624:0@lo:104: 00000100:00100000:2.0:1776178153.948804:0:8250:0:(events.c:361:request_in_callback()) peer: 12345-0@lo (source: 12345-0@lo) 00000100:00100000:2.0:1776178153.948841:0:8250:0:(client.c:2500:ptlrpc_set_wait()) set 000000005a11b75b going to sleep for 6 seconds 00000100:00100000:1.0:1776178153.948925:0:10005:0:(service.c:2152:ptlrpc_server_handle_req_in()) got req x1862456998034624 00000100:00100000:1.0:1776178153.949005:0:10005:0:(nrs_fifo.c:179:nrs_fifo_req_get()) NRS start fifo request from 12345-0@lo, seq: 66 00000100:00100000:1.0:1776178153.949009:0:10005:0:(service.c:2324:ptlrpc_server_handle_request()) Handling RPC req@000000001f170bf7 pname:cluuid+ref:pid:xid:nid:opc:job ldlm_cb00_002:lustre-MDT0001-mdtlov_UUID+4:8250:x1862456998034624:12345-0@lo:104: 00010000:00010000:1.0:1776178153.949022:0:10005:0:(ldlm_lockd.c:2468:ldlm_callback_handler()) ### blocking ast ns: lustre-MDT0000-osp-MDT0001 lock: 0000000055d6a00d/0xa40083ab9bf53342 lrc: 3/0,0 mode: EX/EX res: [0x200000007:0x1:0x0].0x0 bits 0x2/0x0 rrc: 3 type: IBT gid 0 flags: 0x1400001000000 nid: local remote: 0xa40083ab9bf53349 expref: -99 pid: 8250 timeout: 0 lvb_type: 0 00010000:00010000:1.0:1776178153.949117:0:10005:0:(ldlm_lockd.c:2159:__ldlm_bl_to_thread()) added 1/1 locks to priority blp list, 1 blwis in pool 00000100:00100000:1.0:1776178153.949123:0:10005:0:(service.c:2372:ptlrpc_server_handle_request()) Handled RPC req@000000001f170bf7 pname:cluuid+ref:pid:xid:nid:opc:job ldlm_cb00_002:lustre-MDT0001-mdtlov_UUID+4:8250:x1862456998034624:12345-0@lo:104: Request processed in 113us (319us total) trans 0 rc 0/0 00000100:00100000:1.0:1776178153.949130:0:10005:0:(nrs_fifo.c:241:nrs_fifo_req_stop()) NRS stop fifo request from 12345-0@lo, seq: 66 00000100:00100000:2.0:1776178153.949201:0:8250:0:(client.c:2215:ptlrpc_check_set()) Completed RPC req@0000000098e424d0 pname:cluuid:pid:xid:nid:opc:job mdt00_003:lustre-MDT0000_UUID:8250:1862456998034624:0@lo:104: 00010000:00010000:2.0:1776178153.949224:0:8250:0:(ldlm_request.c:283:ldlm_completion_ast()) ### client-side enqueue returned a blocked locksleeping ns: mdt-lustre-MDT0000_UUID lock: 00000000b27cf00e/0xa40083ab9bf5335e lrc: 3/1,0 mode: --/CR res: [0x200000007:0x1:0x0].0x0 bits 0x2/0x0 rrc: 6 type: IBT gid 0 flags: 0x40210000000000 nid: local remote: 0x0 expref: -99 pid: 8250 timeout: 0 lvb_type: 0 00010000:00010000:3.0:1776178153.949329:0:6245:0:(ldlm_lockd.c:2789:ldlm_bl_get_work()) Got 1 locks of 0 total in blp. (0 blwis in pool) 00010000:00010000:3.0:1776178153.949335:0:6245:0:(ldlm_lockd.c:1879:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: lustre-MDT0000-osp-MDT0001 lock: 0000000055d6a00d/0xa40083ab9bf53342 lrc: 3/0,0 mode: EX/EX res: [0x200000007:0x1:0x0].0x0 bits 0x2/0x0 rrc: 3 type: IBT gid 0 flags: 0x1400001000000 nid: local remote: 0xa40083ab9bf53349 expref: -99 pid: 8250 timeout: 0 lvb_type: 0 00010000:00010000:3.0:1776178153.949345:0:6245:0:(ldlm_lockd.c:1893:ldlm_handle_bl_callback()) Lock 0000000055d6a00d already unused, calling callback (0000000067e5ddd8) 00010000:00010000:3.0:1776178153.949351:0:6245:0:(ldlm_request.c:1232:ldlm_cli_cancel_local()) ### client-side cancel ns: lustre-MDT0000-osp-MDT0001 lock: 0000000055d6a00d/0xa40083ab9bf53342 lrc: 4/0,0 mode: EX/EX res: [0x200000007:0x1:0x0].0x0 bits 0x2/0x2 rrc: 3 type: IBT gid 0 flags: 0x1408401000000 nid: local remote: 0xa40083ab9bf53349 expref: -99 pid: 8250 timeout: 0 lvb_type: 0 00000004:00010000:3.0:1776178153.949361:0:6245:0:(mdt_handler.c:3608:mdt_remote_blocking_ast()) ### Revoke remote lock ns: lustre-MDT0000-osp-MDT0001 lock: 0000000055d6a00d/0xa40083ab9bf53342 lrc: 4/0,0 mode: EX/EX res: [0x200000007:0x1:0x0].0x0 bits 0x2/0x2 rrc: 3 type: IBT gid 0 flags: 0x1409401000000 nid: local remote: 0xa40083ab9bf53349 expref: -99 pid: 8250 timeout: 0 lvb_type: 0 00010000:00010000:3.0:1776178153.949396:0:6245:0:(ldlm_request.c:1279:__ldlm_pack_lock()) ### packing ns: lustre-MDT0000-osp-MDT0001 lock: 0000000055d6a00d/0xa40083ab9bf53342 lrc: 2/0,0 mode: --/EX res: [0x200000007:0x1:0x0].0x0 bits 0x2/0x2 rrc: 3 type: IBT gid 0 flags: 0x5c09401000020 nid: local remote: 0xa40083ab9bf53349 expref: -99 pid: 8250 timeout: 0 lvb_type: 0 00010000:00010000:3.0:1776178153.949404:0:6245:0:(ldlm_request.c:1325:ldlm_cancel_pack()) 1 locks packed 00010000:00010000:3.0:1776178153.949443:0:6245:0:(ldlm_lockd.c:1903:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: lustre-MDT0000-osp-MDT0001 lock: 0000000055d6a00d/0xa40083ab9bf53342 lrc: 1/0,0 mode: --/EX res: [0x200000007:0x1:0x0].0x0 bits 0x2/0x2 rrc: 3 type: IBT gid 0 flags: 0x5c09401000020 nid: local remote: 0xa40083ab9bf53349 expref: -99 pid: 8250 timeout: 0 lvb_type: 0 00010000:00010000:3.0:1776178153.949452:0:6245:0:(ldlm_lock.c:216:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: lustre-MDT0000-osp-MDT0001 lock: 0000000055d6a00d/0xa40083ab9bf53342 lrc: 0/0,0 mode: --/EX res: [0x200000007:0x1:0x0].0x0 bits 0x2/0x2 rrc: 3 type: IBT gid 0 flags: 0x5c09401000020 nid: local remote: 0xa40083ab9bf53349 expref: -99 pid: 8250 timeout: 0 lvb_type: 0 00010000:00010000:3.0:1776178153.949470:0:6245:0:(ldlm_lockd.c:2792:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00010000:00010000:3.0:1776178153.949471:0:6245:0:(ldlm_lockd.c:2792:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00010000:00010000:3.0:1776178153.949473:0:6245:0:(ldlm_lockd.c:2792:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:3.0:1776178153.949481:0:3354:0:(ptlrpcd.c:415:ptlrpcd_check()) transfer 1 async RPCs [0->1] 00000100:00100000:3.0:1776178153.949494:0:3354:0:(client.c:1733:ptlrpc_send_new_req()) Sending RPC req@00000000ae838296 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_01:lustre-MDT0001-mdtlov_UUID:3354:1862456998034688:0@lo:103:ldlm_bl_02.0 00000100:00100000:3.0:1776178153.949534:0:3354:0:(events.c:361:request_in_callback()) peer: 12345-0@lo (source: 12345-0@lo) 00000100:00100000:2.0:1776178153.949687:0:6242:0:(service.c:2152:ptlrpc_server_handle_req_in()) got req x1862456998034688 00010000:00010000:2.0:1776178153.949705:0:6242:0:(ldlm_lockd.c:2638:ldlm_cancel_hpreq_check()) ### hpreq cancel/convert lock ns: mdt-lustre-MDT0000_UUID lock: 00000000fcb32840/0xa40083ab9bf53349 lrc: 3/0,0 mode: EX/EX res: [0x200000007:0x1:0x0].0x0 bits 0x2/0x0 rrc: 6 type: IBT gid 0 flags: 0x40000401000020 nid: 0@lo remote: 0xa40083ab9bf53342 expref: 8 pid: 14128 timeout: 0 lvb_type: 0 00000100:00100000:2.0:1776178153.949734:0:6242:0:(nrs_fifo.c:179:nrs_fifo_req_get()) NRS start fifo request from 12345-0@lo, seq: 2081 00000100:00100000:2.0:1776178153.949738:0:6242:0:(service.c:2324:ptlrpc_server_handle_request()) Handling RPC req@00000000b9a3856b pname:cluuid+ref:pid:xid:nid:opc:job ldlm_cn00_001:lustre-MDT0001-mdtlov_UUID+8:3354:x1862456998034688:12345-0@lo:103:ldlm_bl_02.0 00010000:00010000:2.0:1776178153.949748:0:6242:0:(ldlm_lockd.c:1720:ldlm_request_cancel()) ### server-side cancel handler START: 1 locks, starting at 0 00000001:00080000:2.0:1776178153.960399:0:20134:0:(tgt_lastrcvd.c:913:tgt_cb_last_committed()) lustre-MDT0000: transno 12884901944 is committed 00010000:00010000:0.0:1776178153.960762:0:6242:0:(ldlm_lock.c:216:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: mdt-lustre-MDT0000_UUID lock: 00000000fcb32840/0xa40083ab9bf53349 lrc: 0/0,0 mode: --/EX res: [0x200000007:0x1:0x0].0x0 bits 0x2/0x0 rrc: 7 type: IBT gid 0 flags: 0x44801401000020 nid: 0@lo remote: 0xa40083ab9bf53342 expref: 7 pid: 14128 timeout: 0 lvb_type: 0 00010000:00010000:0.0:1776178153.960786:0:6242:0:(ldlm_inodebits.c:94:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x200000007:0x1:0x0].0x0 (00000000ad0d4094) 00010000:00010000:0.0:1776178153.960790:0:6242:0:(ldlm_inodebits.c:113:ldlm_reprocess_inodebits_queue()) ### Reprocessing lock from queue 1 ns: mdt-lustre-MDT0000_UUID lock: 00000000b27cf00e/0xa40083ab9bf5335e lrc: 3/1,0 mode: --/CR res: [0x200000007:0x1:0x0].0x0 bits 0x2/0x0 rrc: 6 type: IBT gid 0 flags: 0x40210000000000 nid: local remote: 0x0 expref: -99 pid: 8250 timeout: 0 lvb_type: 0 00010000:00010000:0.0:1776178153.960800:0:6242:0:(ldlm_lock.c:704:ldlm_add_cp_work_item()) ### lock granted; sending completion AST. ns: mdt-lustre-MDT0000_UUID lock: 00000000b27cf00e/0xa40083ab9bf5335e lrc: 3/1,0 mode: CR/CR res: [0x200000007:0x1:0x0].0x0 bits 0x2/0x0 rrc: 6 type: IBT gid 0 flags: 0x40290000000000 nid: local remote: 0x0 expref: -99 pid: 8250 timeout: 0 lvb_type: 0 00010000:00010000:0.0:1776178153.960809:0:6242:0:(ldlm_lock.c:1064:ldlm_granted_list_add_lock()) ### About to add lock: ns: mdt-lustre-MDT0000_UUID lock: 00000000b27cf00e/0xa40083ab9bf5335e lrc: 4/1,0 mode: CR/CR res: [0x200000007:0x1:0x0].0x0 bits 0x2/0x0 rrc: 6 type: IBT gid 0 flags: 0x40290000000000 nid: local remote: 0x0 expref: -99 pid: 8250 timeout: 0 lvb_type: 0 00010000:00010000:0.0:1776178153.960846:0:6242:0:(ldlm_lockd.c:1782:ldlm_request_cancel()) ### server-side cancel handler END 00000100:00100000:0.0:1776178153.960925:0:6242:0:(service.c:2372:ptlrpc_server_handle_request()) Handled RPC req@00000000b9a3856b pname:cluuid+ref:pid:xid:nid:opc:job ldlm_cn00_001:lustre-MDT0001-mdtlov_UUID+6:3354:x1862456998034688:12345-0@lo:103:ldlm_bl_02.0 Request processed in 11185us (11390us total) trans 0 rc 0/0 00000100:00100000:0.0:1776178153.960934:0:6242:0:(nrs_fifo.c:241:nrs_fifo_req_stop()) NRS stop fifo request from 12345-0@lo, seq: 2081 00000100:00100000:3.0:1776178153.961021:0:3354:0:(client.c:2215:ptlrpc_check_set()) Completed RPC req@00000000ae838296 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_01:lustre-MDT0001-mdtlov_UUID:3354:1862456998034688:0@lo:103:ldlm_bl_02.0 00010000:00010000:2.0:1776178153.961249:0:8250:0:(ldlm_request.c:199:ldlm_completion_tail()) ### client-side enqueue: granted ns: mdt-lustre-MDT0000_UUID lock: 00000000b27cf00e/0xa40083ab9bf5335e lrc: 3/1,0 mode: CR/CR res: [0x200000007:0x1:0x0].0x0 bits 0x2/0x0 rrc: 5 type: IBT gid 0 flags: 0x40210000000000 nid: local remote: 0x0 expref: -99 pid: 8250 timeout: 0 lvb_type: 0 00010000:00010000:2.0:1776178153.961259:0:8250:0:(ldlm_request.c:528:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: mdt-lustre-MDT0000_UUID lock: 00000000b27cf00e/0xa40083ab9bf5335e lrc: 3/1,0 mode: CR/CR res: [0x200000007:0x1:0x0].0x0 bits 0x2/0x0 rrc: 5 type: IBT gid 0 flags: 0x40210000000000 nid: local remote: 0x0 expref: -99 pid: 8250 timeout: 0 lvb_type: 0 00010000:00010000:2.0:1776178153.961288:0:8250:0:(ldlm_lock.c:766:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PR) ns: mdt-lustre-MDT0000_UUID lock: 00000000a71646e9/0xa40083ab9bf53365 lrc: 3/1,0 mode: --/PR res: [0x200000007:0x1:0x0].0x58a1b1aa bits 0x0/0x0 rrc: 2 type: IBT gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 8250 timeout: 0 lvb_type: 0 00010000:00010000:2.0:1776178153.961300:0:8250:0:(ldlm_lock.c:1064:ldlm_granted_list_add_lock()) ### About to add lock: ns: mdt-lustre-MDT0000_UUID lock: 00000000a71646e9/0xa40083ab9bf53365 lrc: 3/1,0 mode: PR/PR res: [0x200000007:0x1:0x0].0x58a1b1aa bits 0x2/0x0 rrc: 2 type: IBT gid 0 flags: 0x50210000000000 nid: local remote: 0x0 expref: -99 pid: 8250 timeout: 0 lvb_type: 0 00010000:00010000:2.0:1776178153.961309:0:8250:0:(ldlm_request.c:528:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: mdt-lustre-MDT0000_UUID lock: 00000000a71646e9/0xa40083ab9bf53365 lrc: 3/1,0 mode: PR/PR res: [0x200000007:0x1:0x0].0x58a1b1aa bits 0x2/0x0 rrc: 2 type: IBT gid 0 flags: 0x40210000000000 nid: local remote: 0x0 expref: -99 pid: 8250 timeout: 0 lvb_type: 0 00000100:00100000:2.0:1776178153.961444:0:8250:0:(client.c:1733:ptlrpc_send_new_req()) Sending RPC req@0000000098e424d0 pname:cluuid:pid:xid:nid:opc:job mdt00_003:lustre-MDT0000-mdtlov_UUID:8250:1862456998034752:0@lo:1000:mdt00_003.0 00000100:00100000:2.0:1776178153.961493:0:8250:0:(events.c:361:request_in_callback()) peer: 12345-0@lo (source: 12345-0@lo) 00000100:00100000:2.0:1776178153.961512:0:8250:0:(client.c:2500:ptlrpc_set_wait()) set 000000001023bbe3 going to sleep for 42 seconds 00000100:00100000:0.0:1776178153.962129:0:15213:0:(service.c:2152:ptlrpc_server_handle_req_in()) got req x1862456998034752 00000100:00100000:0.0:1776178153.962182:0:15213:0:(nrs_fifo.c:179:nrs_fifo_req_get()) NRS start fifo request from 12345-0@lo, seq: 658 00000100:00100000:0.0:1776178153.962188:0:15213:0:(service.c:2324:ptlrpc_server_handle_request()) Handling RPC req@000000006b0ae3d8 pname:cluuid+ref:pid:xid:nid:opc:job mdt_out00_003:lustre-MDT0000-mdtlov_UUID+5:8250:x1862456998034752:12345-0@lo:1000:mdt00_003.0 00000100:00100000:0.0:1776178153.962280:0:15213:0:(service.c:2372:ptlrpc_server_handle_request()) Handled RPC req@000000006b0ae3d8 pname:cluuid+ref:pid:xid:nid:opc:job mdt_out00_003:lustre-MDT0000-mdtlov_UUID+5:8250:x1862456998034752:12345-0@lo:1000:mdt00_003.0 Request processed in 91us (789us total) trans 0 rc 0/0 00000100:00100000:0.0:1776178153.962287:0:15213:0:(nrs_fifo.c:241:nrs_fifo_req_stop()) NRS stop fifo request from 12345-0@lo, seq: 658 00000100:00100000:2.0:1776178153.962427:0:8250:0:(client.c:2215:ptlrpc_check_set()) Completed RPC req@0000000098e424d0 pname:cluuid:pid:xid:nid:opc:job mdt00_003:lustre-MDT0000-mdtlov_UUID:8250:1862456998034752:0@lo:1000:mdt00_003.0 00010000:00010000:2.0:1776178153.962486:0:8250:0:(ldlm_lock.c:766:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PR) ns: mdt-lustre-MDT0000_UUID lock: 00000000801fce1a/0xa40083ab9bf5336c lrc: 3/1,0 mode: --/PR res: [0x240000402:0x7f7:0x0].0x0 bits 0x0/0x0 rrc: 2 type: IBT gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 8250 timeout: 0 lvb_type: 0 00010000:00010000:2.0:1776178153.962500:0:8250:0:(ldlm_lock.c:1064:ldlm_granted_list_add_lock()) ### About to add lock: ns: mdt-lustre-MDT0000_UUID lock: 00000000801fce1a/0xa40083ab9bf5336c lrc: 3/1,0 mode: PR/PR res: [0x240000402:0x7f7:0x0].0x0 bits 0x1/0x0 rrc: 2 type: IBT gid 0 flags: 0x50210000000000 nid: local remote: 0x0 expref: -99 pid: 8250 timeout: 0 lvb_type: 0 00010000:00010000:2.0:1776178153.962509:0:8250:0:(ldlm_request.c:528:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: mdt-lustre-MDT0000_UUID lock: 00000000801fce1a/0xa40083ab9bf5336c lrc: 3/1,0 mode: PR/PR res: [0x240000402:0x7f7:0x0].0x0 bits 0x1/0x0 rrc: 2 type: IBT gid 0 flags: 0x40210000000000 nid: local remote: 0x0 expref: -99 pid: 8250 timeout: 0 lvb_type: 0 00000004:00080000:2.0:1776178153.962519:0:8250:0:(osp_object.c:1389:osp_invalidate()) Invalidate osp_object [0x240000402:0x7f7:0x0] 00010000:00010000:2.0:1776178153.962529:0:8250:0:(ldlm_lock.c:820:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(CR) ns: mdt-lustre-MDT0000_UUID lock: 00000000b27cf00e/0xa40083ab9bf5335e lrc: 3/1,0 mode: CR/CR res: [0x200000007:0x1:0x0].0x0 bits 0x2/0x0 rrc: 5 type: IBT gid 0 flags: 0x50210000000000 nid: local remote: 0x0 expref: -99 pid: 8250 timeout: 0 lvb_type: 0 00010000:00010000:2.0:1776178153.962537:0:8250:0:(ldlm_lock.c:879:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: mdt-lustre-MDT0000_UUID lock: 00000000b27cf00e/0xa40083ab9bf5335e lrc: 2/0,0 mode: CR/CR res: [0x200000007:0x1:0x0].0x0 bits 0x2/0x0 rrc: 5 type: IBT gid 0 flags: 0x50210400000000 nid: local remote: 0x0 expref: -99 pid: 8250 timeout: 0 lvb_type: 0 00010000:00010000:2.0:1776178153.962545:0:8250:0:(ldlm_lockd.c:1879:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: mdt-lustre-MDT0000_UUID lock: 00000000b27cf00e/0xa40083ab9bf5335e lrc: 3/0,0 mode: CR/CR res: [0x200000007:0x1:0x0].0x0 bits 0x2/0x0 rrc: 5 type: IBT gid 0 flags: 0x40210400000000 nid: local remote: 0x0 expref: -99 pid: 8250 timeout: 0 lvb_type: 0 00010000:00010000:2.0:1776178153.962552:0:8250:0:(ldlm_lockd.c:1893:ldlm_handle_bl_callback()) Lock 00000000b27cf00e already unused, calling callback (0000000079e78348) 00010000:00010000:2.0:1776178153.962556:0:8250:0:(ldlm_request.c:364:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: mdt-lustre-MDT0000_UUID lock: 00000000b27cf00e/0xa40083ab9bf5335e lrc: 3/0,0 mode: CR/CR res: [0x200000007:0x1:0x0].0x0 bits 0x2/0x0 rrc: 5 type: IBT gid 0 flags: 0x40210400000000 nid: local remote: 0x0 expref: -99 pid: 8250 timeout: 0 lvb_type: 0 00010000:00010000:2.0:1776178153.962564:0:8250:0:(ldlm_request.c:1257:ldlm_cli_cancel_local()) ### server-side local cancel ns: mdt-lustre-MDT0000_UUID lock: 00000000b27cf00e/0xa40083ab9bf5335e lrc: 4/0,0 mode: CR/CR res: [0x200000007:0x1:0x0].0x0 bits 0x2/0x0 rrc: 5 type: IBT gid 0 flags: 0x40218400000000 nid: local remote: 0x0 expref: -99 pid: 8250 timeout: 0 lvb_type: 0 00010000:00010000:2.0:1776178153.962576:0:8250:0:(ldlm_inodebits.c:94:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x200000007:0x1:0x0].0x0 (00000000ad0d4094) 00010000:00010000:2.0:1776178153.962578:0:8250:0:(ldlm_inodebits.c:96:ldlm_reprocess_inodebits_queue()) Hint 2 00010000:00010000:2.0:1776178153.962583:0:8250:0:(ldlm_lockd.c:1903:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: mdt-lustre-MDT0000_UUID lock: 00000000b27cf00e/0xa40083ab9bf5335e lrc: 2/0,0 mode: --/CR res: [0x200000007:0x1:0x0].0x0 bits 0x2/0x0 rrc: 5 type: IBT gid 0 flags: 0x44a19400000000 nid: local remote: 0x0 expref: -99 pid: 8250 timeout: 0 lvb_type: 0 00010000:00010000:2.0:1776178153.962589:0:8250:0:(ldlm_lock.c:216:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: mdt-lustre-MDT0000_UUID lock: 00000000b27cf00e/0xa40083ab9bf5335e lrc: 0/0,0 mode: --/CR res: [0x200000007:0x1:0x0].0x0 bits 0x2/0x0 rrc: 5 type: IBT gid 0 flags: 0x44a19400000000 nid: local remote: 0x0 expref: -99 pid: 8250 timeout: 0 lvb_type: 0 00010000:00010000:2.0:1776178153.962598:0:8250:0:(ldlm_lock.c:820:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PR) ns: mdt-lustre-MDT0000_UUID lock: 00000000a71646e9/0xa40083ab9bf53365 lrc: 3/1,0 mode: PR/PR res: [0x200000007:0x1:0x0].0x58a1b1aa bits 0x2/0x0 rrc: 2 type: IBT gid 0 flags: 0x50210000000000 nid: local remote: 0x0 expref: -99 pid: 8250 timeout: 0 lvb_type: 0 00010000:00010000:2.0:1776178153.962604:0:8250:0:(ldlm_lock.c:879:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: mdt-lustre-MDT0000_UUID lock: 00000000a71646e9/0xa40083ab9bf53365 lrc: 2/0,0 mode: PR/PR res: [0x200000007:0x1:0x0].0x58a1b1aa bits 0x2/0x0 rrc: 2 type: IBT gid 0 flags: 0x50210400000000 nid: local remote: 0x0 expref: -99 pid: 8250 timeout: 0 lvb_type: 0 00010000:00010000:2.0:1776178153.962610:0:8250:0:(ldlm_lockd.c:1879:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: mdt-lustre-MDT0000_UUID lock: 00000000a71646e9/0xa40083ab9bf53365 lrc: 3/0,0 mode: PR/PR res: [0x200000007:0x1:0x0].0x58a1b1aa bits 0x2/0x0 rrc: 2 type: IBT gid 0 flags: 0x40210400000000 nid: local remote: 0x0 expref: -99 pid: 8250 timeout: 0 lvb_type: 0 00010000:00010000:2.0:1776178153.962620:0:8250:0:(ldlm_lockd.c:1893:ldlm_handle_bl_callback()) Lock 00000000a71646e9 already unused, calling callback (0000000079e78348) 00010000:00010000:2.0:1776178153.962622:0:8250:0:(ldlm_request.c:364:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: mdt-lustre-MDT0000_UUID lock: 00000000a71646e9/0xa40083ab9bf53365 lrc: 3/0,0 mode: PR/PR res: [0x200000007:0x1:0x0].0x58a1b1aa bits 0x2/0x0 rrc: 2 type: IBT gid 0 flags: 0x40210400000000 nid: local remote: 0x0 expref: -99 pid: 8250 timeout: 0 lvb_type: 0 00010000:00010000:2.0:1776178153.962628:0:8250:0:(ldlm_request.c:1257:ldlm_cli_cancel_local()) ### server-side local cancel ns: mdt-lustre-MDT0000_UUID lock: 00000000a71646e9/0xa40083ab9bf53365 lrc: 4/0,0 mode: PR/PR res: [0x200000007:0x1:0x0].0x58a1b1aa bits 0x2/0x0 rrc: 2 type: IBT gid 0 flags: 0x40218400000000 nid: local remote: 0x0 expref: -99 pid: 8250 timeout: 0 lvb_type: 0 00010000:00010000:2.0:1776178153.962635:0:8250:0:(ldlm_inodebits.c:94:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x200000007:0x1:0x0].0x58a1b1aa (00000000f1a25a38) 00010000:00010000:2.0:1776178153.962637:0:8250:0:(ldlm_inodebits.c:96:ldlm_reprocess_inodebits_queue()) Hint 2 00010000:00010000:2.0:1776178153.962639:0:8250:0:(ldlm_lockd.c:1903:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: mdt-lustre-MDT0000_UUID lock: 00000000a71646e9/0xa40083ab9bf53365 lrc: 2/0,0 mode: --/PR res: [0x200000007:0x1:0x0].0x58a1b1aa bits 0x2/0x0 rrc: 2 type: IBT gid 0 flags: 0x44a19400000000 nid: local remote: 0x0 expref: -99 pid: 8250 timeout: 0 lvb_type: 0 00010000:00010000:2.0:1776178153.962645:0:8250:0:(ldlm_lock.c:216:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: mdt-lustre-MDT0000_UUID lock: 00000000a71646e9/0xa40083ab9bf53365 lrc: 0/0,0 mode: --/PR res: [0x200000007:0x1:0x0].0x58a1b1aa bits 0x2/0x0 rrc: 2 type: IBT gid 0 flags: 0x44a19400000000 nid: local remote: 0x0 expref: -99 pid: 8250 timeout: 0 lvb_type: 0 00010000:00010000:2.0:1776178153.962699:0:8250:0:(ldlm_lock.c:216:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: mdt-lustre-MDT0000_UUID lock: 00000000d3e952db/0xa40083ab9bf53357 lrc: 0/0,0 mode: --/CR res: [0x200000007:0x1:0x0].0x0 bits 0x1/0x0 rrc: 4 type: IBT gid 0 flags: 0x44000000000000 nid: 192.168.204.3@tcp remote: 0x771f68aa1eb27a93 expref: 8 pid: 8250 timeout: 0 lvb_type: 0 00010000:00010000:2.0:1776178153.962713:0:8250:0:(ldlm_lockd.c:1484:ldlm_handle_enqueue0()) ### server-side enqueue handler, sending reply (err=0, rc=0) ns: mdt-lustre-MDT0000_UUID lock: 00000000801fce1a/0xa40083ab9bf5336c lrc: 3/0,0 mode: PR/PR res: [0x240000402:0x7f7:0x0].0x0 bits 0x1/0x0 rrc: 2 type: IBT gid 0 flags: 0x40200000000000 nid: 192.168.204.3@tcp remote: 0x771f68aa1eb27a93 expref: 7 pid: 8250 timeout: 0 lvb_type: 0 00010000:00010000:2.0:1776178153.962719:0:8250:0:(ldlm_inodebits.c:94:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x240000402:0x7f7:0x0].0x0 (00000000940da82c) 00010000:00010000:2.0:1776178153.962722:0:8250:0:(ldlm_inodebits.c:96:ldlm_reprocess_inodebits_queue()) Hint 1 00010000:00010000:2.0:1776178153.962724:0:8250:0:(ldlm_lockd.c:1565:ldlm_handle_enqueue0()) ### server-side enqueue handler END (lock 00000000801fce1a, rc 0) 00000100:00100000:2.0:1776178153.962783:0:8250:0:(service.c:2372:ptlrpc_server_handle_request()) Handled RPC req@0000000068adf51c pname:cluuid+ref:pid:xid:nid:opc:job mdt00_003:b50963bd-8eba-4432-9a54-8888d00e3fa1+7:17618:x1862456979089856:12345-192.168.204.3@tcp:101:lfs.0 Request processed in 14268us (14456us total) trans 0 rc 0/0 00000100:00100000:2.0:1776178153.962791:0:8250:0:(nrs_fifo.c:241:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.204.3@tcp, seq: 6529 00000100:00100000:0.0:1776178153.968390:0:3347:0:(events.c:361:request_in_callback()) peer: 12345-192.168.204.3@tcp (source: 12345-192.168.204.3@tcp) 00000100:00100000:2.0:1776178153.970204:0:8250:0:(service.c:2152:ptlrpc_server_handle_req_in()) got req x1862456979089984 00000100:00100000:2.0:1776178153.970300:0:8250:0:(nrs_fifo.c:179:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.204.3@tcp, seq: 6530 00000100:00100000:2.0:1776178153.970310:0:8250:0:(service.c:2324:ptlrpc_server_handle_request()) Handling RPC req@00000000e2ca56cc pname:cluuid+ref:pid:xid:nid:opc:job mdt00_003:b50963bd-8eba-4432-9a54-8888d00e3fa1+8:17618:x1862456979089984:12345-192.168.204.3@tcp:101:lfs.0 00010000:00010000:2.0:1776178153.970365:0:8250:0:(ldlm_lockd.c:1248:ldlm_handle_enqueue0()) ### server-side enqueue handler START 00010000:00010000:2.0:1776178153.970401:0:8250:0:(ldlm_lockd.c:1318:ldlm_handle_enqueue0()) ### server-side enqueue handler, new lock created ns: mdt-lustre-MDT0001_UUID lock: 000000007ae7b5f6/0xa40083ab9bf53373 lrc: 2/0,0 mode: --/CR res: [0x240000402:0x7f7:0x0].0x0 bits 0x0/0x0 rrc: 3 type: IBT gid 0 flags: 0x40000000000000 nid: local remote: 0x771f68aa1eb27a9a expref: -99 pid: 8250 timeout: 0 lvb_type: 0 00000004:00200000:2.0:1776178153.970587:0:8250:0:(mdt_handler.c:912:mdt_pack_attr2body()) [0x240000402:0x7f7:0x0]: returning size 4096 00010000:00010000:2.0:1776178153.970760:0:8250:0:(ldlm_lockd.c:1484:ldlm_handle_enqueue0()) ### server-side enqueue handler, sending reply (err=301, rc=0) ns: mdt-lustre-MDT0001_UUID lock: 000000007ae7b5f6/0xa40083ab9bf53373 lrc: 1/0,0 mode: --/CR res: [0x240000402:0x7f7:0x0].0x0 bits 0x1/0x0 rrc: 3 type: IBT gid 0 flags: 0x44000000000000 nid: 192.168.204.3@tcp remote: 0x771f68aa1eb27a9a expref: 10 pid: 8250 timeout: 0 lvb_type: 0 00010000:00010000:2.0:1776178153.970771:0:8250:0:(ldlm_lock.c:216:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: mdt-lustre-MDT0001_UUID lock: 000000007ae7b5f6/0xa40083ab9bf53373 lrc: 0/0,0 mode: --/CR res: [0x240000402:0x7f7:0x0].0x0 bits 0x1/0x0 rrc: 3 type: IBT gid 0 flags: 0x44000000000000 nid: 192.168.204.3@tcp remote: 0x771f68aa1eb27a9a expref: 10 pid: 8250 timeout: 0 lvb_type: 0 00010000:00010000:2.0:1776178153.970788:0:8250:0:(ldlm_lockd.c:1565:ldlm_handle_enqueue0()) ### server-side enqueue handler END (lock 000000007ae7b5f6, rc 0) 00000100:00100000:2.0:1776178153.970908:0:8250:0:(service.c:2372:ptlrpc_server_handle_request()) Handled RPC req@00000000e2ca56cc pname:cluuid+ref:pid:xid:nid:opc:job mdt00_003:b50963bd-8eba-4432-9a54-8888d00e3fa1+9:17618:x1862456979089984:12345-192.168.204.3@tcp:101:lfs.0 Request processed in 599us (2527us total) trans 17179876234 rc 301/301 00000100:00100000:2.0:1776178153.970917:0:8250:0:(nrs_fifo.c:241:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.204.3@tcp, seq: 6530 00000100:00100000:3.0:1776178153.978913:0:3346:0:(events.c:361:request_in_callback()) peer: 12345-192.168.204.3@tcp (source: 12345-192.168.204.3@tcp) 00000100:00100000:2.0:1776178153.981099:0:6259:0:(service.c:2152:ptlrpc_server_handle_req_in()) got req x1862456979090048 00000100:00100000:2.0:1776178153.981199:0:6259:0:(nrs_fifo.c:179:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.204.3@tcp, seq: 2133 00000100:00100000:2.0:1776178153.981207:0:6259:0:(service.c:2324:ptlrpc_server_handle_request()) Handling RPC req@0000000049197666 pname:cluuid+ref:pid:xid:nid:opc:job mdt_rdpg00_001:b50963bd-8eba-4432-9a54-8888d00e3fa1+9:17618:x1862456979090048:12345-192.168.204.3@tcp:33:lfs.0 00000004:00200000:2.0:1776178153.981331:0:6259:0:(mdt_handler.c:912:mdt_pack_attr2body()) [0x240000402:0x7f7:0x0]: returning size 4096 00000004:00200000:2.0:1776178153.981337:0:6259:0:(mdt_handler.c:1523:mdt_getattr_internal()) dirent count 2 stripe count 1 MDT count 2 00000100:00100000:2.0:1776178153.981451:0:6259:0:(service.c:2372:ptlrpc_server_handle_request()) Handled RPC req@0000000049197666 pname:cluuid+ref:pid:xid:nid:opc:job mdt_rdpg00_001:b50963bd-8eba-4432-9a54-8888d00e3fa1+9:17618:x1862456979090048:12345-192.168.204.3@tcp:33:lfs.0 Request processed in 243us (2548us total) trans 0 rc 0/0 00000100:00100000:2.0:1776178153.981463:0:6259:0:(nrs_fifo.c:241:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.204.3@tcp, seq: 2133 00000100:00100000:3.0:1776178154.001226:0:3348:0:(events.c:361:request_in_callback()) peer: 12345-192.168.204.3@tcp (source: 12345-192.168.204.3@tcp) 00000100:00100000:2.0:1776178154.001455:0:6259:0:(service.c:2152:ptlrpc_server_handle_req_in()) got req x1862456979090176 00000100:00100000:2.0:1776178154.001528:0:6259:0:(nrs_fifo.c:179:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.204.3@tcp, seq: 2134 00000100:00100000:2.0:1776178154.001537:0:6259:0:(service.c:2324:ptlrpc_server_handle_request()) Handling RPC req@00000000ae838296 pname:cluuid+ref:pid:xid:nid:opc:job mdt_rdpg00_001:b50963bd-8eba-4432-9a54-8888d00e3fa1+9:17618:x1862456979090176:12345-192.168.204.3@tcp:35:lfs.0 00000100:00100000:2.0:1776178154.001807:0:6259:0:(service.c:2372:ptlrpc_server_handle_request()) Handled RPC req@00000000ae838296 pname:cluuid+ref:pid:xid:nid:opc:job mdt_rdpg00_001:b50963bd-8eba-4432-9a54-8888d00e3fa1+10:17618:x1862456979090176:12345-192.168.204.3@tcp:35:lfs.0 Request processed in 269us (594us total) trans 17179876235 rc 0/0 00000100:00100000:2.0:1776178154.001815:0:6259:0:(nrs_fifo.c:241:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.204.3@tcp, seq: 2134 00000100:00100000:0.0:1776178154.059730:0:3347:0:(events.c:361:request_in_callback()) peer: 12345-192.168.204.3@tcp (source: 12345-192.168.204.3@tcp) 00000100:00100000:2.0:1776178154.059849:0:8250:0:(service.c:2152:ptlrpc_server_handle_req_in()) got req x1862456979090304 00000100:00100000:2.0:1776178154.059912:0:8250:0:(nrs_fifo.c:179:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.204.3@tcp, seq: 6531 00000100:00100000:2.0:1776178154.059919:0:8250:0:(service.c:2324:ptlrpc_server_handle_request()) Handling RPC req@00000000e5c18d8a pname:cluuid+ref:pid:xid:nid:opc:job mdt00_003:b50963bd-8eba-4432-9a54-8888d00e3fa1+10:17621:x1862456979090304:12345-192.168.204.3@tcp:101:touch.0 00010000:00010000:2.0:1776178154.059946:0:8250:0:(ldlm_lockd.c:1248:ldlm_handle_enqueue0()) ### server-side enqueue handler START 00010000:00010000:2.0:1776178154.059982:0:8250:0:(ldlm_lockd.c:1318:ldlm_handle_enqueue0()) ### server-side enqueue handler, new lock created ns: mdt-lustre-MDT0001_UUID lock: 000000007ae7b5f6/0xa40083ab9bf53381 lrc: 2/0,0 mode: --/CW res: [0x240000402:0x7f7:0x0].0x0 bits 0x0/0x0 rrc: 3 type: IBT gid 0 flags: 0x40000000000000 nid: local remote: 0x771f68aa1eb27aa1 expref: -99 pid: 8250 timeout: 0 lvb_type: 0 00010000:00010000:2.0:1776178154.060130:0:8250:0:(ldlm_lock.c:766:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(CW) ns: mdt-lustre-MDT0001_UUID lock: 00000000d3e952db/0xa40083ab9bf53388 lrc: 3/0,1 mode: --/CW res: [0x240000402:0x7f7:0x0].0x0 bits 0x0/0x0 rrc: 4 type: IBT gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 8250 timeout: 0 lvb_type: 0 00010000:00010000:2.0:1776178154.060156:0:8250:0:(ldlm_lock.c:1064:ldlm_granted_list_add_lock()) ### About to add lock: ns: mdt-lustre-MDT0001_UUID lock: 00000000d3e952db/0xa40083ab9bf53388 lrc: 3/0,1 mode: CW/CW res: [0x240000402:0x7f7:0x0].0x0 bits 0x2/0x0 rrc: 4 type: IBT gid 0 flags: 0x50210000000000 nid: local remote: 0x0 expref: -99 pid: 8250 timeout: 0 lvb_type: 0 00010000:00010000:2.0:1776178154.060169:0:8250:0:(ldlm_request.c:528:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: mdt-lustre-MDT0001_UUID lock: 00000000d3e952db/0xa40083ab9bf53388 lrc: 3/0,1 mode: CW/CW res: [0x240000402:0x7f7:0x0].0x0 bits 0x2/0x0 rrc: 4 type: IBT gid 0 flags: 0x40210000000000 nid: local remote: 0x0 expref: -99 pid: 8250 timeout: 0 lvb_type: 0 00010000:00010000:2.0:1776178154.061565:0:8250:0:(ldlm_lock.c:766:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: mdt-lustre-MDT0001_UUID lock: 00000000a71646e9/0xa40083ab9bf5338f lrc: 3/0,1 mode: --/PW res: [0x240000402:0x7f7:0x0].0xcda302ea bits 0x0/0x0 rrc: 2 type: IBT gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 8250 timeout: 0 lvb_type: 0 00010000:00010000:2.0:1776178154.061582:0:8250:0:(ldlm_lock.c:1064:ldlm_granted_list_add_lock()) ### About to add lock: ns: mdt-lustre-MDT0001_UUID lock: 00000000a71646e9/0xa40083ab9bf5338f lrc: 3/0,1 mode: PW/PW res: [0x240000402:0x7f7:0x0].0xcda302ea bits 0x2/0x0 rrc: 2 type: IBT gid 0 flags: 0x50210000000000 nid: local remote: 0x0 expref: -99 pid: 8250 timeout: 0 lvb_type: 0 00010000:00010000:2.0:1776178154.061591:0:8250:0:(ldlm_request.c:528:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: mdt-lustre-MDT0001_UUID lock: 00000000a71646e9/0xa40083ab9bf5338f lrc: 3/0,1 mode: PW/PW res: [0x240000402:0x7f7:0x0].0xcda302ea bits 0x2/0x0 rrc: 2 type: IBT gid 0 flags: 0x40210000000000 nid: local remote: 0x0 expref: -99 pid: 8250 timeout: 0 lvb_type: 0 00000100:00100000:2.0:1776178154.061779:0:8250:0:(client.c:1733:ptlrpc_send_new_req()) Sending RPC req@0000000098e424d0 pname:cluuid:pid:xid:nid:opc:job mdt00_003:lustre-MDT0001-mdtlov_UUID:8250:1862456998034816:0@lo:1000:mdt00_003.0 00000100:00100000:2.0:1776178154.061845:0:8250:0:(events.c:361:request_in_callback()) peer: 12345-0@lo (source: 12345-0@lo) 00000100:00100000:2.0:1776178154.061880:0:8250:0:(client.c:2500:ptlrpc_set_wait()) set 00000000e4b67670 going to sleep for 42 seconds 00000100:00100000:1.0:1776178154.062328:0:14128:0:(service.c:2152:ptlrpc_server_handle_req_in()) got req x1862456998034816 00000100:00100000:1.0:1776178154.062377:0:14128:0:(nrs_fifo.c:179:nrs_fifo_req_get()) NRS start fifo request from 12345-0@lo, seq: 659 00000100:00100000:1.0:1776178154.062383:0:14128:0:(service.c:2324:ptlrpc_server_handle_request()) Handling RPC req@0000000090a7aa50 pname:cluuid+ref:pid:xid:nid:opc:job mdt_out00_002:lustre-MDT0001-mdtlov_UUID+6:8250:x1862456998034816:12345-0@lo:1000:mdt00_003.0 00000100:00100000:1.0:1776178154.062478:0:14128:0:(service.c:2372:ptlrpc_server_handle_request()) Handled RPC req@0000000090a7aa50 pname:cluuid+ref:pid:xid:nid:opc:job mdt_out00_002:lustre-MDT0001-mdtlov_UUID+6:8250:x1862456998034816:12345-0@lo:1000:mdt00_003.0 Request processed in 94us (632us total) trans 0 rc 0/0 00000100:00100000:1.0:1776178154.062485:0:14128:0:(nrs_fifo.c:241:nrs_fifo_req_stop()) NRS stop fifo request from 12345-0@lo, seq: 659 00000100:00100000:2.0:1776178154.065407:0:8250:0:(client.c:2868:ptlrpc_free_committed()) lustre-MDT0000-osp-MDT0001: committing for last_committed 12884901944 gen 3 00000004:00080000:2.0:1776178154.065420:0:8250:0:(osp_trans.c:1044:osp_request_commit_cb()) trans no 12884901944 committed transno 12884901944 00000100:00100000:2.0:1776178154.065451:0:8250:0:(client.c:2215:ptlrpc_check_set()) Completed RPC req@0000000098e424d0 pname:cluuid:pid:xid:nid:opc:job mdt00_003:lustre-MDT0001-mdtlov_UUID:8250:1862456998034816:0@lo:1000:mdt00_003.0 00000004:00080000:3.0:1776178154.066717:0:8250:0:(osp_object.c:1645:osp_create()) lustre-OST0001-osc-MDT0001: Wrote last used FID: [0x2c0000400:0x42e:0x0], index 1: 0 00010000:00010000:3.0:1776178154.066935:0:8250:0:(ldlm_lock.c:766:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(CR) ns: mdt-lustre-MDT0001_UUID lock: 0000000055d6a00d/0xa40083ab9bf53396 lrc: 3/1,0 mode: --/CR res: [0x240000402:0x7f8:0x0].0x0 bits 0x0/0x0 rrc: 2 type: IBT gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 8250 timeout: 0 lvb_type: 0 00010000:00010000:3.0:1776178154.067531:0:8250:0:(ldlm_lock.c:1064:ldlm_granted_list_add_lock()) ### About to add lock: ns: mdt-lustre-MDT0001_UUID lock: 0000000055d6a00d/0xa40083ab9bf53396 lrc: 3/1,0 mode: CR/CR res: [0x240000402:0x7f8:0x0].0x0 bits 0x9/0x0 rrc: 2 type: IBT gid 0 flags: 0x50210000000000 nid: local remote: 0x0 expref: -99 pid: 8250 timeout: 0 lvb_type: 0 00010000:00010000:3.0:1776178154.067547:0:8250:0:(ldlm_request.c:528:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: mdt-lustre-MDT0001_UUID lock: 0000000055d6a00d/0xa40083ab9bf53396 lrc: 3/1,0 mode: CR/CR res: [0x240000402:0x7f8:0x0].0x0 bits 0x9/0x0 rrc: 2 type: IBT gid 0 flags: 0x40210000000000 nid: local remote: 0x0 expref: -99 pid: 8250 timeout: 0 lvb_type: 0 00010000:00010000:3.0:1776178154.067621:0:8250:0:(ldlm_lock.c:820:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(CW) ns: mdt-lustre-MDT0001_UUID lock: 00000000d3e952db/0xa40083ab9bf53388 lrc: 3/0,1 mode: CW/CW res: [0x240000402:0x7f7:0x0].0x0 bits 0x2/0x0 rrc: 4 type: IBT gid 0 flags: 0x50210000000000 nid: local remote: 0x0 expref: -99 pid: 8250 timeout: 0 lvb_type: 0 00010000:00010000:3.0:1776178154.067629:0:8250:0:(ldlm_lock.c:879:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: mdt-lustre-MDT0001_UUID lock: 00000000d3e952db/0xa40083ab9bf53388 lrc: 2/0,0 mode: CW/CW res: [0x240000402:0x7f7:0x0].0x0 bits 0x2/0x0 rrc: 4 type: IBT gid 0 flags: 0x50210400000000 nid: local remote: 0x0 expref: -99 pid: 8250 timeout: 0 lvb_type: 0 00010000:00010000:3.0:1776178154.067635:0:8250:0:(ldlm_lockd.c:1879:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: mdt-lustre-MDT0001_UUID lock: 00000000d3e952db/0xa40083ab9bf53388 lrc: 3/0,0 mode: CW/CW res: [0x240000402:0x7f7:0x0].0x0 bits 0x2/0x0 rrc: 4 type: IBT gid 0 flags: 0x40210400000000 nid: local remote: 0x0 expref: -99 pid: 8250 timeout: 0 lvb_type: 0 00010000:00010000:3.0:1776178154.067641:0:8250:0:(ldlm_lockd.c:1893:ldlm_handle_bl_callback()) Lock 00000000d3e952db already unused, calling callback (0000000079e78348) 00010000:00010000:3.0:1776178154.067645:0:8250:0:(ldlm_request.c:364:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: mdt-lustre-MDT0001_UUID lock: 00000000d3e952db/0xa40083ab9bf53388 lrc: 3/0,0 mode: CW/CW res: [0x240000402:0x7f7:0x0].0x0 bits 0x2/0x0 rrc: 4 type: IBT gid 0 flags: 0x40210400000000 nid: local remote: 0x0 expref: -99 pid: 8250 timeout: 0 lvb_type: 0 00010000:00010000:3.0:1776178154.067659:0:8250:0:(ldlm_request.c:1257:ldlm_cli_cancel_local()) ### server-side local cancel ns: mdt-lustre-MDT0001_UUID lock: 00000000d3e952db/0xa40083ab9bf53388 lrc: 4/0,0 mode: CW/CW res: [0x240000402:0x7f7:0x0].0x0 bits 0x2/0x0 rrc: 4 type: IBT gid 0 flags: 0x40218400000000 nid: local remote: 0x0 expref: -99 pid: 8250 timeout: 0 lvb_type: 0 00010000:00010000:3.0:1776178154.067671:0:8250:0:(ldlm_inodebits.c:94:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x240000402:0x7f7:0x0].0x0 (000000006b762dc5) 00010000:00010000:3.0:1776178154.067673:0:8250:0:(ldlm_inodebits.c:96:ldlm_reprocess_inodebits_queue()) Hint 2 00010000:00010000:3.0:1776178154.067677:0:8250:0:(ldlm_lockd.c:1903:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: mdt-lustre-MDT0001_UUID lock: 00000000d3e952db/0xa40083ab9bf53388 lrc: 2/0,0 mode: --/CW res: [0x240000402:0x7f7:0x0].0x0 bits 0x2/0x0 rrc: 4 type: IBT gid 0 flags: 0x44a19400000000 nid: local remote: 0x0 expref: -99 pid: 8250 timeout: 0 lvb_type: 0 00010000:00010000:3.0:1776178154.067682:0:8250:0:(ldlm_lock.c:216:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: mdt-lustre-MDT0001_UUID lock: 00000000d3e952db/0xa40083ab9bf53388 lrc: 0/0,0 mode: --/CW res: [0x240000402:0x7f7:0x0].0x0 bits 0x2/0x0 rrc: 4 type: IBT gid 0 flags: 0x44a19400000000 nid: local remote: 0x0 expref: -99 pid: 8250 timeout: 0 lvb_type: 0 00000004:00010000:3.0:1776178154.067697:0:8250:0:(mdt_handler.c:3992:mdt_save_lock()) ### save lock request 00000000e5c18d8a reply state 000000001608951f transno 17179876236 ns: mdt-lustre-MDT0001_UUID lock: 00000000a71646e9/0xa40083ab9bf5338f lrc: 3/0,1 mode: PW/PW res: [0x240000402:0x7f7:0x0].0xcda302ea bits 0x2/0x0 rrc: 2 type: IBT gid 0 flags: 0x40210000000000 nid: local remote: 0x0 expref: -99 pid: 8250 timeout: 0 lvb_type: 0 00010000:00010000:3.0:1776178154.067732:0:8250:0:(ldlm_lock.c:216:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: mdt-lustre-MDT0001_UUID lock: 000000007ae7b5f6/0xa40083ab9bf53381 lrc: 0/0,0 mode: --/CW res: [0x240000402:0x7f7:0x0].0x0 bits 0x1/0x0 rrc: 3 type: IBT gid 0 flags: 0x44000000000000 nid: 192.168.204.3@tcp remote: 0x771f68aa1eb27aa1 expref: 13 pid: 8250 timeout: 0 lvb_type: 0 00010000:00010000:3.0:1776178154.067749:0:8250:0:(ldlm_lockd.c:1484:ldlm_handle_enqueue0()) ### server-side enqueue handler, sending reply (err=0, rc=0) ns: mdt-lustre-MDT0001_UUID lock: 0000000055d6a00d/0xa40083ab9bf53396 lrc: 3/0,0 mode: CR/CR res: [0x240000402:0x7f8:0x0].0x0 bits 0x9/0x0 rrc: 2 type: IBT gid 0 flags: 0x40200000000000 nid: 192.168.204.3@tcp remote: 0x771f68aa1eb27aa1 expref: 12 pid: 8250 timeout: 0 lvb_type: 0 00010000:00010000:3.0:1776178154.067755:0:8250:0:(ldlm_inodebits.c:94:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x240000402:0x7f8:0x0].0x0 (00000000099618da) 00010000:00010000:3.0:1776178154.067757:0:8250:0:(ldlm_inodebits.c:96:ldlm_reprocess_inodebits_queue()) Hint 9 00010000:00010000:3.0:1776178154.067759:0:8250:0:(ldlm_lockd.c:1565:ldlm_handle_enqueue0()) ### server-side enqueue handler END (lock 0000000055d6a00d, rc 0) 00000100:00100000:3.0:1776178154.067929:0:8250:0:(service.c:2372:ptlrpc_server_handle_request()) Handled RPC req@00000000e5c18d8a pname:cluuid+ref:pid:xid:nid:opc:job mdt00_003:b50963bd-8eba-4432-9a54-8888d00e3fa1+13:17621:x1862456979090304:12345-192.168.204.3@tcp:101:touch.0 Request processed in 8006us (8204us total) trans 17179876236 rc 0/0 00000100:00100000:3.0:1776178154.067943:0:8250:0:(nrs_fifo.c:241:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.204.3@tcp, seq: 6531 00000100:00100000:2.0:1776178154.082618:0:3346:0:(events.c:361:request_in_callback()) peer: 12345-192.168.204.3@tcp (source: 12345-192.168.204.3@tcp) 00010000:00010000:3.0:1776178154.082766:0:3339:0:(ldlm_lock.c:1064:ldlm_granted_list_add_lock()) ### About to add lock: ns: mdt-lustre-MDT0001_UUID lock: 00000000a71646e9/0xa40083ab9bf5338f lrc: 3/0,1 mode: COS/COS res: [0x240000402:0x7f7:0x0].0xcda302ea bits 0x2/0x0 rrc: 2 type: IBT gid 0 flags: 0x50210000000000 nid: local remote: 0x0 expref: -99 pid: 8250 timeout: 0 lvb_type: 0 00010000:00010000:3.0:1776178154.082790:0:3339:0:(ldlm_inodebits.c:94:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x240000402:0x7f7:0x0].0xcda302ea (00000000f1a25a38) 00010000:00010000:3.0:1776178154.082794:0:3339:0:(ldlm_inodebits.c:96:ldlm_reprocess_inodebits_queue()) Hint 2 00000100:00100000:3.0:1776178154.085420:0:8250:0:(service.c:2152:ptlrpc_server_handle_req_in()) got req x1862456979090432 00000100:00100000:3.0:1776178154.085738:0:8250:0:(nrs_fifo.c:179:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.204.3@tcp, seq: 6532 00000100:00100000:3.0:1776178154.085747:0:8250:0:(service.c:2324:ptlrpc_server_handle_request()) Handling RPC req@0000000098e424d0 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_003:b50963bd-8eba-4432-9a54-8888d00e3fa1+13:17621:x1862456979090432:12345-192.168.204.3@tcp:36:touch.0 00010000:00010000:3.0:1776178154.085929:0:8250:0:(ldlm_lock.c:766:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: mdt-lustre-MDT0001_UUID lock: 00000000dcea299c/0xa40083ab9bf533a4 lrc: 3/0,1 mode: --/PW res: [0x240000402:0x7f8:0x0].0x0 bits 0x0/0x0 rrc: 3 type: IBT gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 8250 timeout: 0 lvb_type: 0 00010000:00010000:3.0:1776178154.085968:0:8250:0:(ldlm_lock.c:1064:ldlm_granted_list_add_lock()) ### About to add lock: ns: mdt-lustre-MDT0001_UUID lock: 00000000dcea299c/0xa40083ab9bf533a4 lrc: 3/0,1 mode: PW/PW res: [0x240000402:0x7f8:0x0].0x0 bits 0x2/0x0 rrc: 3 type: IBT gid 0 flags: 0x50210000000000 nid: local remote: 0x0 expref: -99 pid: 8250 timeout: 0 lvb_type: 0 00010000:00010000:3.0:1776178154.085981:0:8250:0:(ldlm_request.c:528:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: mdt-lustre-MDT0001_UUID lock: 00000000dcea299c/0xa40083ab9bf533a4 lrc: 3/0,1 mode: PW/PW res: [0x240000402:0x7f8:0x0].0x0 bits 0x2/0x0 rrc: 3 type: IBT gid 0 flags: 0x40210000000000 nid: local remote: 0x0 expref: -99 pid: 8250 timeout: 0 lvb_type: 0 00000004:00010000:3.0:1776178154.086194:0:8250:0:(mdt_handler.c:3992:mdt_save_lock()) ### save lock request 0000000098e424d0 reply state 0000000060c236df transno 17179876237 ns: mdt-lustre-MDT0001_UUID lock: 00000000dcea299c/0xa40083ab9bf533a4 lrc: 3/0,1 mode: PW/PW res: [0x240000402:0x7f8:0x0].0x0 bits 0x2/0x0 rrc: 3 type: IBT gid 0 flags: 0x40210000000000 nid: local remote: 0x0 expref: -99 pid: 8250 timeout: 0 lvb_type: 0 00000100:00100000:3.0:1776178154.086311:0:8250:0:(service.c:2372:ptlrpc_server_handle_request()) Handled RPC req@0000000098e424d0 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_003:b50963bd-8eba-4432-9a54-8888d00e3fa1+15:17621:x1862456979090432:12345-192.168.204.3@tcp:36:touch.0 Request processed in 564us (3700us total) trans 17179876237 rc 0/0 00000100:00100000:3.0:1776178154.086319:0:8250:0:(nrs_fifo.c:241:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.204.3@tcp, seq: 6532 00000100:00100000:2.0:1776178154.099373:0:3348:0:(events.c:361:request_in_callback()) peer: 12345-192.168.204.3@tcp (source: 12345-192.168.204.3@tcp) 00010000:00010000:0.0:1776178154.100118:0:3340:0:(ldlm_lock.c:1064:ldlm_granted_list_add_lock()) ### About to add lock: ns: mdt-lustre-MDT0001_UUID lock: 00000000dcea299c/0xa40083ab9bf533a4 lrc: 3/0,1 mode: COS/COS res: [0x240000402:0x7f8:0x0].0x0 bits 0x2/0x0 rrc: 3 type: IBT gid 0 flags: 0x50210000000000 nid: local remote: 0x0 expref: -99 pid: 8250 timeout: 0 lvb_type: 0 00010000:00010000:0.0:1776178154.100134:0:3340:0:(ldlm_inodebits.c:94:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x240000402:0x7f8:0x0].0x0 (00000000099618da) 00010000:00010000:0.0:1776178154.100137:0:3340:0:(ldlm_inodebits.c:96:ldlm_reprocess_inodebits_queue()) Hint 2 00000100:00100000:3.0:1776178154.100393:0:8228:0:(service.c:2152:ptlrpc_server_handle_req_in()) got req x1862456979090496 00000100:00100000:3.0:1776178154.100445:0:8228:0:(nrs_fifo.c:179:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.204.3@tcp, seq: 5257 00000100:00100000:3.0:1776178154.100452:0:8228:0:(service.c:2324:ptlrpc_server_handle_request()) Handling RPC req@000000000e0e1967 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_001:b50963bd-8eba-4432-9a54-8888d00e3fa1+5:2270:x1862456979090496:12345-192.168.204.3@tcp:2:touch.0 00000100:00100000:3.0:1776178154.101352:0:8228:0:(service.c:2372:ptlrpc_server_handle_request()) Handled RPC req@000000000e0e1967 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_001:b50963bd-8eba-4432-9a54-8888d00e3fa1+6:2270:x1862456979090496:12345-192.168.204.3@tcp:2:touch.0 Request processed in 898us (1987us total) trans 4294969324 rc 0/0 00000100:00100000:3.0:1776178154.101363:0:8228:0:(nrs_fifo.c:241:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.204.3@tcp, seq: 5257 00000100:00100000:0.0:1776178154.127840:0:3347:0:(events.c:361:request_in_callback()) peer: 12345-192.168.204.3@tcp (source: 12345-192.168.204.3@tcp) 00000100:00100000:1.0:1776178154.128119:0:6258:0:(service.c:2152:ptlrpc_server_handle_req_in()) got req x1862456979090624 00000100:00100000:1.0:1776178154.128176:0:6258:0:(nrs_fifo.c:179:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.204.3@tcp, seq: 2135 00000100:00100000:1.0:1776178154.128183:0:6258:0:(service.c:2324:ptlrpc_server_handle_request()) Handling RPC req@000000008d82b734 pname:cluuid+ref:pid:xid:nid:opc:job mdt_rdpg00_000:b50963bd-8eba-4432-9a54-8888d00e3fa1+15:17621:x1862456979090624:12345-192.168.204.3@tcp:35:touch.0 00000100:00100000:1.0:1776178154.128483:0:6258:0:(service.c:2372:ptlrpc_server_handle_request()) Handled RPC req@000000008d82b734 pname:cluuid+ref:pid:xid:nid:opc:job mdt_rdpg00_000:b50963bd-8eba-4432-9a54-8888d00e3fa1+16:17621:x1862456979090624:12345-192.168.204.3@tcp:35:touch.0 Request processed in 298us (652us total) trans 17179876238 rc 0/0 00000100:00100000:1.0:1776178154.128491:0:6258:0:(nrs_fifo.c:241:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.204.3@tcp, seq: 2135 00000100:00100000:3.0:1776178154.154183:0:3346:0:(events.c:361:request_in_callback()) peer: 12345-192.168.204.3@tcp (source: 12345-192.168.204.3@tcp) 00000100:00100000:2.0:1776178154.159469:0:8250:0:(service.c:2152:ptlrpc_server_handle_req_in()) got req x1862456979090752 00000100:00100000:1.0:1776178154.159905:0:8250:0:(nrs_fifo.c:179:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.204.3@tcp, seq: 6533 00000100:00100000:1.0:1776178154.159920:0:8250:0:(service.c:2324:ptlrpc_server_handle_request()) Handling RPC req@00000000599b0cf2 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_003:b50963bd-8eba-4432-9a54-8888d00e3fa1+16:17622:x1862456979090752:12345-192.168.204.3@tcp:101:touch.0 00010000:00010000:1.0:1776178154.159995:0:8250:0:(ldlm_lockd.c:1248:ldlm_handle_enqueue0()) ### server-side enqueue handler START 00010000:00010000:1.0:1776178154.160052:0:8250:0:(ldlm_lockd.c:1318:ldlm_handle_enqueue0()) ### server-side enqueue handler, new lock created ns: mdt-lustre-MDT0001_UUID lock: 0000000050323271/0xa40083ab9bf533ab lrc: 2/0,0 mode: --/CW res: [0x240000402:0x7f7:0x0].0x0 bits 0x0/0x0 rrc: 3 type: IBT gid 0 flags: 0x40000000000000 nid: local remote: 0x771f68aa1eb27aa8 expref: -99 pid: 8250 timeout: 0 lvb_type: 0 00010000:00010000:1.0:1776178154.160874:0:8250:0:(ldlm_lock.c:766:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(CW) ns: mdt-lustre-MDT0001_UUID lock: 000000003e1697a2/0xa40083ab9bf533b2 lrc: 3/0,1 mode: --/CW res: [0x240000402:0x7f7:0x0].0x0 bits 0x0/0x0 rrc: 4 type: IBT gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 8250 timeout: 0 lvb_type: 0 00010000:00010000:1.0:1776178154.160892:0:8250:0:(ldlm_lock.c:1064:ldlm_granted_list_add_lock()) ### About to add lock: ns: mdt-lustre-MDT0001_UUID lock: 000000003e1697a2/0xa40083ab9bf533b2 lrc: 3/0,1 mode: CW/CW res: [0x240000402:0x7f7:0x0].0x0 bits 0x2/0x0 rrc: 4 type: IBT gid 0 flags: 0x50210000000000 nid: local remote: 0x0 expref: -99 pid: 8250 timeout: 0 lvb_type: 0 00010000:00010000:1.0:1776178154.160904:0:8250:0:(ldlm_request.c:528:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: mdt-lustre-MDT0001_UUID lock: 000000003e1697a2/0xa40083ab9bf533b2 lrc: 3/0,1 mode: CW/CW res: [0x240000402:0x7f7:0x0].0x0 bits 0x2/0x0 rrc: 4 type: IBT gid 0 flags: 0x40210000000000 nid: local remote: 0x0 expref: -99 pid: 8250 timeout: 0 lvb_type: 0 00010000:00010000:1.0:1776178154.160919:0:8250:0:(ldlm_lock.c:766:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: mdt-lustre-MDT0001_UUID lock: 000000002f317bcb/0xa40083ab9bf533b9 lrc: 3/0,1 mode: --/PW res: [0x240000402:0x7f7:0x0].0x15fcbbea bits 0x0/0x0 rrc: 2 type: IBT gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 8250 timeout: 0 lvb_type: 0 00010000:00010000:1.0:1776178154.160925:0:8250:0:(ldlm_lock.c:1064:ldlm_granted_list_add_lock()) ### About to add lock: ns: mdt-lustre-MDT0001_UUID lock: 000000002f317bcb/0xa40083ab9bf533b9 lrc: 3/0,1 mode: PW/PW res: [0x240000402:0x7f7:0x0].0x15fcbbea bits 0x2/0x0 rrc: 2 type: IBT gid 0 flags: 0x50210000000000 nid: local remote: 0x0 expref: -99 pid: 8250 timeout: 0 lvb_type: 0 00010000:00010000:1.0:1776178154.160931:0:8250:0:(ldlm_request.c:528:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: mdt-lustre-MDT0001_UUID lock: 000000002f317bcb/0xa40083ab9bf533b9 lrc: 3/0,1 mode: PW/PW res: [0x240000402:0x7f7:0x0].0x15fcbbea bits 0x2/0x0 rrc: 2 type: IBT gid 0 flags: 0x40210000000000 nid: local remote: 0x0 expref: -99 pid: 8250 timeout: 0 lvb_type: 0 00000004:00080000:1.0:1776178154.164807:0:8250:0:(osp_object.c:1645:osp_create()) lustre-OST0000-osc-MDT0001: Wrote last used FID: [0x280000400:0x46e:0x0], index 0: 0 00010000:00010000:1.0:1776178154.165023:0:8250:0:(ldlm_lock.c:766:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(CR) ns: mdt-lustre-MDT0001_UUID lock: 000000000b4a063d/0xa40083ab9bf533c0 lrc: 3/1,0 mode: --/CR res: [0x240000402:0x7f9:0x0].0x0 bits 0x0/0x0 rrc: 2 type: IBT gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 8250 timeout: 0 lvb_type: 0 00010000:00010000:1.0:1776178154.165044:0:8250:0:(ldlm_lock.c:1064:ldlm_granted_list_add_lock()) ### About to add lock: ns: mdt-lustre-MDT0001_UUID lock: 000000000b4a063d/0xa40083ab9bf533c0 lrc: 3/1,0 mode: CR/CR res: [0x240000402:0x7f9:0x0].0x0 bits 0x9/0x0 rrc: 2 type: IBT gid 0 flags: 0x50210000000000 nid: local remote: 0x0 expref: -99 pid: 8250 timeout: 0 lvb_type: 0 00010000:00010000:1.0:1776178154.165056:0:8250:0:(ldlm_request.c:528:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: mdt-lustre-MDT0001_UUID lock: 000000000b4a063d/0xa40083ab9bf533c0 lrc: 3/1,0 mode: CR/CR res: [0x240000402:0x7f9:0x0].0x0 bits 0x9/0x0 rrc: 2 type: IBT gid 0 flags: 0x40210000000000 nid: local remote: 0x0 expref: -99 pid: 8250 timeout: 0 lvb_type: 0 00010000:00010000:1.0:1776178154.165154:0:8250:0:(ldlm_lock.c:820:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(CW) ns: mdt-lustre-MDT0001_UUID lock: 000000003e1697a2/0xa40083ab9bf533b2 lrc: 3/0,1 mode: CW/CW res: [0x240000402:0x7f7:0x0].0x0 bits 0x2/0x0 rrc: 4 type: IBT gid 0 flags: 0x50210000000000 nid: local remote: 0x0 expref: -99 pid: 8250 timeout: 0 lvb_type: 0 00010000:00010000:1.0:1776178154.165161:0:8250:0:(ldlm_lock.c:879:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: mdt-lustre-MDT0001_UUID lock: 000000003e1697a2/0xa40083ab9bf533b2 lrc: 2/0,0 mode: CW/CW res: [0x240000402:0x7f7:0x0].0x0 bits 0x2/0x0 rrc: 4 type: IBT gid 0 flags: 0x50210400000000 nid: local remote: 0x0 expref: -99 pid: 8250 timeout: 0 lvb_type: 0 00010000:00010000:1.0:1776178154.165169:0:8250:0:(ldlm_lockd.c:1879:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: mdt-lustre-MDT0001_UUID lock: 000000003e1697a2/0xa40083ab9bf533b2 lrc: 3/0,0 mode: CW/CW res: [0x240000402:0x7f7:0x0].0x0 bits 0x2/0x0 rrc: 4 type: IBT gid 0 flags: 0x40210400000000 nid: local remote: 0x0 expref: -99 pid: 8250 timeout: 0 lvb_type: 0 00010000:00010000:1.0:1776178154.165175:0:8250:0:(ldlm_lockd.c:1893:ldlm_handle_bl_callback()) Lock 000000003e1697a2 already unused, calling callback (0000000079e78348) 00010000:00010000:1.0:1776178154.165179:0:8250:0:(ldlm_request.c:364:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: mdt-lustre-MDT0001_UUID lock: 000000003e1697a2/0xa40083ab9bf533b2 lrc: 3/0,0 mode: CW/CW res: [0x240000402:0x7f7:0x0].0x0 bits 0x2/0x0 rrc: 4 type: IBT gid 0 flags: 0x40210400000000 nid: local remote: 0x0 expref: -99 pid: 8250 timeout: 0 lvb_type: 0 00010000:00010000:1.0:1776178154.165186:0:8250:0:(ldlm_request.c:1257:ldlm_cli_cancel_local()) ### server-side local cancel ns: mdt-lustre-MDT0001_UUID lock: 000000003e1697a2/0xa40083ab9bf533b2 lrc: 4/0,0 mode: CW/CW res: [0x240000402:0x7f7:0x0].0x0 bits 0x2/0x0 rrc: 4 type: IBT gid 0 flags: 0x40218400000000 nid: local remote: 0x0 expref: -99 pid: 8250 timeout: 0 lvb_type: 0 00010000:00010000:1.0:1776178154.165197:0:8250:0:(ldlm_inodebits.c:94:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x240000402:0x7f7:0x0].0x0 (000000006b762dc5) 00010000:00010000:1.0:1776178154.165199:0:8250:0:(ldlm_inodebits.c:96:ldlm_reprocess_inodebits_queue()) Hint 2 00010000:00010000:1.0:1776178154.165202:0:8250:0:(ldlm_lockd.c:1903:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: mdt-lustre-MDT0001_UUID lock: 000000003e1697a2/0xa40083ab9bf533b2 lrc: 2/0,0 mode: --/CW res: [0x240000402:0x7f7:0x0].0x0 bits 0x2/0x0 rrc: 4 type: IBT gid 0 flags: 0x44a19400000000 nid: local remote: 0x0 expref: -99 pid: 8250 timeout: 0 lvb_type: 0 00010000:00010000:1.0:1776178154.165207:0:8250:0:(ldlm_lock.c:216:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: mdt-lustre-MDT0001_UUID lock: 000000003e1697a2/0xa40083ab9bf533b2 lrc: 0/0,0 mode: --/CW res: [0x240000402:0x7f7:0x0].0x0 bits 0x2/0x0 rrc: 4 type: IBT gid 0 flags: 0x44a19400000000 nid: local remote: 0x0 expref: -99 pid: 8250 timeout: 0 lvb_type: 0 00000004:00010000:1.0:1776178154.165221:0:8250:0:(mdt_handler.c:3992:mdt_save_lock()) ### save lock request 00000000599b0cf2 reply state 00000000ae2721d3 transno 17179876239 ns: mdt-lustre-MDT0001_UUID lock: 000000002f317bcb/0xa40083ab9bf533b9 lrc: 3/0,1 mode: PW/PW res: [0x240000402:0x7f7:0x0].0x15fcbbea bits 0x2/0x0 rrc: 2 type: IBT gid 0 flags: 0x40210000000000 nid: local remote: 0x0 expref: -99 pid: 8250 timeout: 0 lvb_type: 0 00010000:00010000:1.0:1776178154.165254:0:8250:0:(ldlm_lock.c:216:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: mdt-lustre-MDT0001_UUID lock: 0000000050323271/0xa40083ab9bf533ab lrc: 0/0,0 mode: --/CW res: [0x240000402:0x7f7:0x0].0x0 bits 0x1/0x0 rrc: 3 type: IBT gid 0 flags: 0x44000000000000 nid: 192.168.204.3@tcp remote: 0x771f68aa1eb27aa8 expref: 19 pid: 8250 timeout: 0 lvb_type: 0 00010000:00010000:1.0:1776178154.165267:0:8250:0:(ldlm_lockd.c:1484:ldlm_handle_enqueue0()) ### server-side enqueue handler, sending reply (err=0, rc=0) ns: mdt-lustre-MDT0001_UUID lock: 000000000b4a063d/0xa40083ab9bf533c0 lrc: 3/0,0 mode: CR/CR res: [0x240000402:0x7f9:0x0].0x0 bits 0x9/0x0 rrc: 2 type: IBT gid 0 flags: 0x40200000000000 nid: 192.168.204.3@tcp remote: 0x771f68aa1eb27aa8 expref: 18 pid: 8250 timeout: 0 lvb_type: 0 00010000:00010000:1.0:1776178154.165273:0:8250:0:(ldlm_inodebits.c:94:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x240000402:0x7f9:0x0].0x0 (00000000f193fa59) 00010000:00010000:1.0:1776178154.165276:0:8250:0:(ldlm_inodebits.c:96:ldlm_reprocess_inodebits_queue()) Hint 9 00010000:00010000:1.0:1776178154.165278:0:8250:0:(ldlm_lockd.c:1565:ldlm_handle_enqueue0()) ### server-side enqueue handler END (lock 000000000b4a063d, rc 0) 00000100:00100000:1.0:1776178154.165359:0:8250:0:(service.c:2372:ptlrpc_server_handle_request()) Handled RPC req@00000000599b0cf2 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_003:b50963bd-8eba-4432-9a54-8888d00e3fa1+19:17622:x1862456979090752:12345-192.168.204.3@tcp:101:touch.0 Request processed in 5440us (11185us total) trans 17179876239 rc 0/0 00000100:00100000:1.0:1776178154.165369:0:8250:0:(nrs_fifo.c:241:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.204.3@tcp, seq: 6533 00010000:00010000:0.0:1776178154.170791:0:3339:0:(ldlm_lock.c:1064:ldlm_granted_list_add_lock()) ### About to add lock: ns: mdt-lustre-MDT0001_UUID lock: 000000002f317bcb/0xa40083ab9bf533b9 lrc: 3/0,1 mode: COS/COS res: [0x240000402:0x7f7:0x0].0x15fcbbea bits 0x2/0x0 rrc: 2 type: IBT gid 0 flags: 0x50210000000000 nid: local remote: 0x0 expref: -99 pid: 8250 timeout: 0 lvb_type: 0 00010000:00010000:0.0:1776178154.170821:0:3339:0:(ldlm_inodebits.c:94:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x240000402:0x7f7:0x0].0x15fcbbea (000000005a2060ba) 00010000:00010000:0.0:1776178154.170824:0:3339:0:(ldlm_inodebits.c:96:ldlm_reprocess_inodebits_queue()) Hint 2 00000100:00100000:3.0:1776178154.173443:0:3346:0:(events.c:361:request_in_callback()) peer: 12345-192.168.204.3@tcp (source: 12345-192.168.204.3@tcp) 00000100:00100000:1.0:1776178154.175878:0:8250:0:(service.c:2152:ptlrpc_server_handle_req_in()) got req x1862456979090880 00000100:00100000:1.0:1776178154.176420:0:8250:0:(nrs_fifo.c:179:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.204.3@tcp, seq: 6534 00000100:00100000:1.0:1776178154.176432:0:8250:0:(service.c:2324:ptlrpc_server_handle_request()) Handling RPC req@00000000f141ac55 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_003:b50963bd-8eba-4432-9a54-8888d00e3fa1+19:17622:x1862456979090880:12345-192.168.204.3@tcp:36:touch.0 00010000:00010000:1.0:1776178154.176624:0:8250:0:(ldlm_lock.c:766:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: mdt-lustre-MDT0001_UUID lock: 0000000050323271/0xa40083ab9bf533ce lrc: 3/0,1 mode: --/PW res: [0x240000402:0x7f9:0x0].0x0 bits 0x0/0x0 rrc: 3 type: IBT gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 8250 timeout: 0 lvb_type: 0 00010000:00010000:1.0:1776178154.176641:0:8250:0:(ldlm_lock.c:1064:ldlm_granted_list_add_lock()) ### About to add lock: ns: mdt-lustre-MDT0001_UUID lock: 0000000050323271/0xa40083ab9bf533ce lrc: 3/0,1 mode: PW/PW res: [0x240000402:0x7f9:0x0].0x0 bits 0x2/0x0 rrc: 3 type: IBT gid 0 flags: 0x50210000000000 nid: local remote: 0x0 expref: -99 pid: 8250 timeout: 0 lvb_type: 0 00010000:00010000:1.0:1776178154.176660:0:8250:0:(ldlm_request.c:528:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: mdt-lustre-MDT0001_UUID lock: 0000000050323271/0xa40083ab9bf533ce lrc: 3/0,1 mode: PW/PW res: [0x240000402:0x7f9:0x0].0x0 bits 0x2/0x0 rrc: 3 type: IBT gid 0 flags: 0x40210000000000 nid: local remote: 0x0 expref: -99 pid: 8250 timeout: 0 lvb_type: 0 00000004:00010000:1.0:1776178154.176769:0:8250:0:(mdt_handler.c:3992:mdt_save_lock()) ### save lock request 00000000f141ac55 reply state 000000000c1de09b transno 17179876240 ns: mdt-lustre-MDT0001_UUID lock: 0000000050323271/0xa40083ab9bf533ce lrc: 3/0,1 mode: PW/PW res: [0x240000402:0x7f9:0x0].0x0 bits 0x2/0x0 rrc: 3 type: IBT gid 0 flags: 0x40210000000000 nid: local remote: 0x0 expref: -99 pid: 8250 timeout: 0 lvb_type: 0 00000100:00100000:1.0:1776178154.176876:0:8250:0:(service.c:2372:ptlrpc_server_handle_request()) Handled RPC req@00000000f141ac55 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_003:b50963bd-8eba-4432-9a54-8888d00e3fa1+21:17622:x1862456979090880:12345-192.168.204.3@tcp:36:touch.0 Request processed in 446us (3441us total) trans 17179876240 rc 0/0 00000100:00100000:1.0:1776178154.176892:0:8250:0:(nrs_fifo.c:241:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.204.3@tcp, seq: 6534 00010000:00010000:0.0:1776178154.180310:0:3340:0:(ldlm_lock.c:1064:ldlm_granted_list_add_lock()) ### About to add lock: ns: mdt-lustre-MDT0001_UUID lock: 0000000050323271/0xa40083ab9bf533ce lrc: 3/0,1 mode: COS/COS res: [0x240000402:0x7f9:0x0].0x0 bits 0x2/0x0 rrc: 3 type: IBT gid 0 flags: 0x50210000000000 nid: local remote: 0x0 expref: -99 pid: 8250 timeout: 0 lvb_type: 0 00010000:00010000:0.0:1776178154.180349:0:3340:0:(ldlm_inodebits.c:94:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x240000402:0x7f9:0x0].0x0 (00000000f193fa59) 00010000:00010000:0.0:1776178154.180354:0:3340:0:(ldlm_inodebits.c:96:ldlm_reprocess_inodebits_queue()) Hint 2 00000100:00100000:3.0:1776178154.184495:0:3346:0:(events.c:361:request_in_callback()) peer: 12345-192.168.204.3@tcp (source: 12345-192.168.204.3@tcp) 00000100:00100000:2.0:1776178154.186115:0:8228:0:(service.c:2152:ptlrpc_server_handle_req_in()) got req x1862456979090944 00000100:00100000:2.0:1776178154.186186:0:8228:0:(nrs_fifo.c:179:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.204.3@tcp, seq: 5258 00000100:00100000:2.0:1776178154.186194:0:8228:0:(service.c:2324:ptlrpc_server_handle_request()) Handling RPC req@000000001f0b1867 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_001:b50963bd-8eba-4432-9a54-8888d00e3fa1+5:2272:x1862456979090944:12345-192.168.204.3@tcp:2:touch.0 00000100:00100000:2.0:1776178154.186462:0:8228:0:(service.c:2372:ptlrpc_server_handle_request()) Handled RPC req@000000001f0b1867 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_001:b50963bd-8eba-4432-9a54-8888d00e3fa1+6:2272:x1862456979090944:12345-192.168.204.3@tcp:2:touch.0 Request processed in 268us (1975us total) trans 4294969353 rc 0/0 00000100:00100000:2.0:1776178154.186470:0:8228:0:(nrs_fifo.c:241:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.204.3@tcp, seq: 5258 00000100:00100000:3.0:1776178154.196459:0:3348:0:(events.c:361:request_in_callback()) peer: 12345-192.168.204.3@tcp (source: 12345-192.168.204.3@tcp) 00000100:00100000:2.0:1776178154.199039:0:6259:0:(service.c:2152:ptlrpc_server_handle_req_in()) got req x1862456979091072 00000100:00100000:2.0:1776178154.199165:0:6259:0:(nrs_fifo.c:179:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.204.3@tcp, seq: 2136 00000100:00100000:2.0:1776178154.199171:0:6259:0:(service.c:2324:ptlrpc_server_handle_request()) Handling RPC req@0000000053e142bc pname:cluuid+ref:pid:xid:nid:opc:job mdt_rdpg00_001:b50963bd-8eba-4432-9a54-8888d00e3fa1+21:17622:x1862456979091072:12345-192.168.204.3@tcp:35:touch.0 00000100:00100000:2.0:1776178154.199478:0:6259:0:(service.c:2372:ptlrpc_server_handle_request()) Handled RPC req@0000000053e142bc pname:cluuid+ref:pid:xid:nid:opc:job mdt_rdpg00_001:b50963bd-8eba-4432-9a54-8888d00e3fa1+22:17622:x1862456979091072:12345-192.168.204.3@tcp:35:touch.0 Request processed in 306us (3026us total) trans 17179876241 rc 0/0 00000100:00100000:2.0:1776178154.199487:0:6259:0:(nrs_fifo.c:241:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.204.3@tcp, seq: 2136 00000100:00100000:1.0:1776178154.232842:0:3347:0:(events.c:361:request_in_callback()) peer: 12345-192.168.204.3@tcp (source: 12345-192.168.204.3@tcp) 00000100:00100000:3.0:1776178154.233129:0:8250:0:(service.c:2152:ptlrpc_server_handle_req_in()) got req x1862456979091200 00000100:00100000:3.0:1776178154.233196:0:8250:0:(nrs_fifo.c:179:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.204.3@tcp, seq: 6535 00000100:00100000:3.0:1776178154.233204:0:8250:0:(service.c:2324:ptlrpc_server_handle_request()) Handling RPC req@0000000078ae567e pname:cluuid+ref:pid:xid:nid:opc:job mdt00_003:b50963bd-8eba-4432-9a54-8888d00e3fa1+22:17623:x1862456979091200:12345-192.168.204.3@tcp:101:touch.0 00010000:00010000:3.0:1776178154.233233:0:8250:0:(ldlm_lockd.c:1248:ldlm_handle_enqueue0()) ### server-side enqueue handler START 00010000:00010000:3.0:1776178154.233256:0:8250:0:(ldlm_lockd.c:1318:ldlm_handle_enqueue0()) ### server-side enqueue handler, new lock created ns: mdt-lustre-MDT0001_UUID lock: 00000000014a8a18/0xa40083ab9bf533d5 lrc: 2/0,0 mode: --/CW res: [0x240000402:0x7f7:0x0].0x0 bits 0x0/0x0 rrc: 3 type: IBT gid 0 flags: 0x40000000000000 nid: local remote: 0x771f68aa1eb27aaf expref: -99 pid: 8250 timeout: 0 lvb_type: 0 00010000:00010000:3.0:1776178154.233370:0:8250:0:(ldlm_lock.c:766:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(CW) ns: mdt-lustre-MDT0001_UUID lock: 000000002a990613/0xa40083ab9bf533dc lrc: 3/0,1 mode: --/CW res: [0x240000402:0x7f7:0x0].0x0 bits 0x0/0x0 rrc: 4 type: IBT gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 8250 timeout: 0 lvb_type: 0 00010000:00010000:3.0:1776178154.233386:0:8250:0:(ldlm_lock.c:1064:ldlm_granted_list_add_lock()) ### About to add lock: ns: mdt-lustre-MDT0001_UUID lock: 000000002a990613/0xa40083ab9bf533dc lrc: 3/0,1 mode: CW/CW res: [0x240000402:0x7f7:0x0].0x0 bits 0x2/0x0 rrc: 4 type: IBT gid 0 flags: 0x50210000000000 nid: local remote: 0x0 expref: -99 pid: 8250 timeout: 0 lvb_type: 0 00010000:00010000:3.0:1776178154.233405:0:8250:0:(ldlm_request.c:528:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: mdt-lustre-MDT0001_UUID lock: 000000002a990613/0xa40083ab9bf533dc lrc: 3/0,1 mode: CW/CW res: [0x240000402:0x7f7:0x0].0x0 bits 0x2/0x0 rrc: 4 type: IBT gid 0 flags: 0x40210000000000 nid: local remote: 0x0 expref: -99 pid: 8250 timeout: 0 lvb_type: 0 00010000:00010000:3.0:1776178154.233419:0:8250:0:(ldlm_lock.c:766:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: mdt-lustre-MDT0001_UUID lock: 00000000c1e7ea3e/0xa40083ab9bf533e3 lrc: 3/0,1 mode: --/PW res: [0x240000402:0x7f7:0x0].0x5e5674ea bits 0x0/0x0 rrc: 2 type: IBT gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 8250 timeout: 0 lvb_type: 0 00010000:00010000:3.0:1776178154.233426:0:8250:0:(ldlm_lock.c:1064:ldlm_granted_list_add_lock()) ### About to add lock: ns: mdt-lustre-MDT0001_UUID lock: 00000000c1e7ea3e/0xa40083ab9bf533e3 lrc: 3/0,1 mode: PW/PW res: [0x240000402:0x7f7:0x0].0x5e5674ea bits 0x2/0x0 rrc: 2 type: IBT gid 0 flags: 0x50210000000000 nid: local remote: 0x0 expref: -99 pid: 8250 timeout: 0 lvb_type: 0 00010000:00010000:3.0:1776178154.233436:0:8250:0:(ldlm_request.c:528:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: mdt-lustre-MDT0001_UUID lock: 00000000c1e7ea3e/0xa40083ab9bf533e3 lrc: 3/0,1 mode: PW/PW res: [0x240000402:0x7f7:0x0].0x5e5674ea bits 0x2/0x0 rrc: 2 type: IBT gid 0 flags: 0x40210000000000 nid: local remote: 0x0 expref: -99 pid: 8250 timeout: 0 lvb_type: 0 00000004:00080000:3.0:1776178154.233792:0:8250:0:(osp_object.c:1645:osp_create()) lustre-OST0001-osc-MDT0001: Wrote last used FID: [0x2c0000400:0x42f:0x0], index 1: 0 00010000:00010000:3.0:1776178154.233921:0:8250:0:(ldlm_lock.c:766:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(CR) ns: mdt-lustre-MDT0001_UUID lock: 000000006a188f9a/0xa40083ab9bf533ea lrc: 3/1,0 mode: --/CR res: [0x240000402:0x7fa:0x0].0x0 bits 0x0/0x0 rrc: 2 type: IBT gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 8250 timeout: 0 lvb_type: 0 00010000:00010000:3.0:1776178154.233930:0:8250:0:(ldlm_lock.c:1064:ldlm_granted_list_add_lock()) ### About to add lock: ns: mdt-lustre-MDT0001_UUID lock: 000000006a188f9a/0xa40083ab9bf533ea lrc: 3/1,0 mode: CR/CR res: [0x240000402:0x7fa:0x0].0x0 bits 0x9/0x0 rrc: 2 type: IBT gid 0 flags: 0x50210000000000 nid: local remote: 0x0 expref: -99 pid: 8250 timeout: 0 lvb_type: 0 00010000:00010000:3.0:1776178154.233936:0:8250:0:(ldlm_request.c:528:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: mdt-lustre-MDT0001_UUID lock: 000000006a188f9a/0xa40083ab9bf533ea lrc: 3/1,0 mode: CR/CR res: [0x240000402:0x7fa:0x0].0x0 bits 0x9/0x0 rrc: 2 type: IBT gid 0 flags: 0x40210000000000 nid: local remote: 0x0 expref: -99 pid: 8250 timeout: 0 lvb_type: 0 00010000:00010000:3.0:1776178154.233982:0:8250:0:(ldlm_lock.c:820:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(CW) ns: mdt-lustre-MDT0001_UUID lock: 000000002a990613/0xa40083ab9bf533dc lrc: 3/0,1 mode: CW/CW res: [0x240000402:0x7f7:0x0].0x0 bits 0x2/0x0 rrc: 4 type: IBT gid 0 flags: 0x50210000000000 nid: local remote: 0x0 expref: -99 pid: 8250 timeout: 0 lvb_type: 0 00010000:00010000:3.0:1776178154.233989:0:8250:0:(ldlm_lock.c:879:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: mdt-lustre-MDT0001_UUID lock: 000000002a990613/0xa40083ab9bf533dc lrc: 2/0,0 mode: CW/CW res: [0x240000402:0x7f7:0x0].0x0 bits 0x2/0x0 rrc: 4 type: IBT gid 0 flags: 0x50210400000000 nid: local remote: 0x0 expref: -99 pid: 8250 timeout: 0 lvb_type: 0 00010000:00010000:3.0:1776178154.233996:0:8250:0:(ldlm_lockd.c:1879:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: mdt-lustre-MDT0001_UUID lock: 000000002a990613/0xa40083ab9bf533dc lrc: 3/0,0 mode: CW/CW res: [0x240000402:0x7f7:0x0].0x0 bits 0x2/0x0 rrc: 4 type: IBT gid 0 flags: 0x40210400000000 nid: local remote: 0x0 expref: -99 pid: 8250 timeout: 0 lvb_type: 0 00010000:00010000:3.0:1776178154.234002:0:8250:0:(ldlm_lockd.c:1893:ldlm_handle_bl_callback()) Lock 000000002a990613 already unused, calling callback (0000000079e78348) 00010000:00010000:3.0:1776178154.234006:0:8250:0:(ldlm_request.c:364:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: mdt-lustre-MDT0001_UUID lock: 000000002a990613/0xa40083ab9bf533dc lrc: 3/0,0 mode: CW/CW res: [0x240000402:0x7f7:0x0].0x0 bits 0x2/0x0 rrc: 4 type: IBT gid 0 flags: 0x40210400000000 nid: local remote: 0x0 expref: -99 pid: 8250 timeout: 0 lvb_type: 0 00010000:00010000:3.0:1776178154.234013:0:8250:0:(ldlm_request.c:1257:ldlm_cli_cancel_local()) ### server-side local cancel ns: mdt-lustre-MDT0001_UUID lock: 000000002a990613/0xa40083ab9bf533dc lrc: 4/0,0 mode: CW/CW res: [0x240000402:0x7f7:0x0].0x0 bits 0x2/0x0 rrc: 4 type: IBT gid 0 flags: 0x40218400000000 nid: local remote: 0x0 expref: -99 pid: 8250 timeout: 0 lvb_type: 0 00010000:00010000:3.0:1776178154.234023:0:8250:0:(ldlm_inodebits.c:94:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x240000402:0x7f7:0x0].0x0 (000000006b762dc5) 00010000:00010000:3.0:1776178154.234025:0:8250:0:(ldlm_inodebits.c:96:ldlm_reprocess_inodebits_queue()) Hint 2 00010000:00010000:3.0:1776178154.234029:0:8250:0:(ldlm_lockd.c:1903:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: mdt-lustre-MDT0001_UUID lock: 000000002a990613/0xa40083ab9bf533dc lrc: 2/0,0 mode: --/CW res: [0x240000402:0x7f7:0x0].0x0 bits 0x2/0x0 rrc: 4 type: IBT gid 0 flags: 0x44a19400000000 nid: local remote: 0x0 expref: -99 pid: 8250 timeout: 0 lvb_type: 0 00010000:00010000:3.0:1776178154.234034:0:8250:0:(ldlm_lock.c:216:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: mdt-lustre-MDT0001_UUID lock: 000000002a990613/0xa40083ab9bf533dc lrc: 0/0,0 mode: --/CW res: [0x240000402:0x7f7:0x0].0x0 bits 0x2/0x0 rrc: 4 type: IBT gid 0 flags: 0x44a19400000000 nid: local remote: 0x0 expref: -99 pid: 8250 timeout: 0 lvb_type: 0 00000004:00010000:3.0:1776178154.234046:0:8250:0:(mdt_handler.c:3992:mdt_save_lock()) ### save lock request 0000000078ae567e reply state 00000000c4d2442f transno 17179876242 ns: mdt-lustre-MDT0001_UUID lock: 00000000c1e7ea3e/0xa40083ab9bf533e3 lrc: 3/0,1 mode: PW/PW res: [0x240000402:0x7f7:0x0].0x5e5674ea bits 0x2/0x0 rrc: 2 type: IBT gid 0 flags: 0x40210000000000 nid: local remote: 0x0 expref: -99 pid: 8250 timeout: 0 lvb_type: 0 00010000:00010000:3.0:1776178154.234126:0:8250:0:(ldlm_lock.c:216:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: mdt-lustre-MDT0001_UUID lock: 00000000014a8a18/0xa40083ab9bf533d5 lrc: 0/0,0 mode: --/CW res: [0x240000402:0x7f7:0x0].0x0 bits 0x1/0x0 rrc: 3 type: IBT gid 0 flags: 0x44000000000000 nid: 192.168.204.3@tcp remote: 0x771f68aa1eb27aaf expref: 25 pid: 8250 timeout: 0 lvb_type: 0 00010000:00010000:3.0:1776178154.234139:0:8250:0:(ldlm_lockd.c:1484:ldlm_handle_enqueue0()) ### server-side enqueue handler, sending reply (err=0, rc=0) ns: mdt-lustre-MDT0001_UUID lock: 000000006a188f9a/0xa40083ab9bf533ea lrc: 3/0,0 mode: CR/CR res: [0x240000402:0x7fa:0x0].0x0 bits 0x9/0x0 rrc: 2 type: IBT gid 0 flags: 0x40200000000000 nid: 192.168.204.3@tcp remote: 0x771f68aa1eb27aaf expref: 24 pid: 8250 timeout: 0 lvb_type: 0 00010000:00010000:3.0:1776178154.234145:0:8250:0:(ldlm_inodebits.c:94:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x240000402:0x7fa:0x0].0x0 (000000006b0fcbcb) 00010000:00010000:3.0:1776178154.234147:0:8250:0:(ldlm_inodebits.c:96:ldlm_reprocess_inodebits_queue()) Hint 9 00010000:00010000:3.0:1776178154.234149:0:8250:0:(ldlm_lockd.c:1565:ldlm_handle_enqueue0()) ### server-side enqueue handler END (lock 000000006a188f9a, rc 0) 00000100:00100000:3.0:1776178154.234209:0:8250:0:(service.c:2372:ptlrpc_server_handle_request()) Handled RPC req@0000000078ae567e pname:cluuid+ref:pid:xid:nid:opc:job mdt00_003:b50963bd-8eba-4432-9a54-8888d00e3fa1+25:17623:x1862456979091200:12345-192.168.204.3@tcp:101:touch.0 Request processed in 1005us (1375us total) trans 17179876242 rc 0/0 00000100:00100000:3.0:1776178154.234217:0:8250:0:(nrs_fifo.c:241:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.204.3@tcp, seq: 6535 00010000:00010000:0.0:1776178154.237789:0:3339:0:(ldlm_lock.c:1064:ldlm_granted_list_add_lock()) ### About to add lock: ns: mdt-lustre-MDT0001_UUID lock: 00000000c1e7ea3e/0xa40083ab9bf533e3 lrc: 3/0,1 mode: COS/COS res: [0x240000402:0x7f7:0x0].0x5e5674ea bits 0x2/0x0 rrc: 2 type: IBT gid 0 flags: 0x50210000000000 nid: local remote: 0x0 expref: -99 pid: 8250 timeout: 0 lvb_type: 0 00010000:00010000:0.0:1776178154.237811:0:3339:0:(ldlm_inodebits.c:94:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x240000402:0x7f7:0x0].0x5e5674ea (000000007b7d95c1) 00010000:00010000:0.0:1776178154.237814:0:3339:0:(ldlm_inodebits.c:96:ldlm_reprocess_inodebits_queue()) Hint 2 00000100:00100000:1.0:1776178154.240114:0:3347:0:(events.c:361:request_in_callback()) peer: 12345-192.168.204.3@tcp (source: 12345-192.168.204.3@tcp) 00000100:00100000:2.0:1776178154.240257:0:6255:0:(service.c:2152:ptlrpc_server_handle_req_in()) got req x1862456979091328 00000100:00100000:2.0:1776178154.240350:0:6255:0:(nrs_fifo.c:179:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.204.3@tcp, seq: 6536 00000100:00100000:2.0:1776178154.240362:0:6255:0:(service.c:2324:ptlrpc_server_handle_request()) Handling RPC req@0000000021d5d4e8 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_000:b50963bd-8eba-4432-9a54-8888d00e3fa1+25:17623:x1862456979091328:12345-192.168.204.3@tcp:36:touch.0 00010000:00010000:2.0:1776178154.240529:0:6255:0:(ldlm_lock.c:766:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: mdt-lustre-MDT0001_UUID lock: 00000000b27cf00e/0xa40083ab9bf533f8 lrc: 3/0,1 mode: --/PW res: [0x240000402:0x7fa:0x0].0x0 bits 0x0/0x0 rrc: 3 type: IBT gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 6255 timeout: 0 lvb_type: 0 00010000:00010000:2.0:1776178154.240550:0:6255:0:(ldlm_lock.c:1064:ldlm_granted_list_add_lock()) ### About to add lock: ns: mdt-lustre-MDT0001_UUID lock: 00000000b27cf00e/0xa40083ab9bf533f8 lrc: 3/0,1 mode: PW/PW res: [0x240000402:0x7fa:0x0].0x0 bits 0x2/0x0 rrc: 3 type: IBT gid 0 flags: 0x50210000000000 nid: local remote: 0x0 expref: -99 pid: 6255 timeout: 0 lvb_type: 0 00010000:00010000:2.0:1776178154.240561:0:6255:0:(ldlm_request.c:528:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: mdt-lustre-MDT0001_UUID lock: 00000000b27cf00e/0xa40083ab9bf533f8 lrc: 3/0,1 mode: PW/PW res: [0x240000402:0x7fa:0x0].0x0 bits 0x2/0x0 rrc: 3 type: IBT gid 0 flags: 0x40210000000000 nid: local remote: 0x0 expref: -99 pid: 6255 timeout: 0 lvb_type: 0 00000004:00010000:2.0:1776178154.241002:0:6255:0:(mdt_handler.c:3992:mdt_save_lock()) ### save lock request 0000000021d5d4e8 reply state 00000000d8382675 transno 17179876243 ns: mdt-lustre-MDT0001_UUID lock: 00000000b27cf00e/0xa40083ab9bf533f8 lrc: 3/0,1 mode: PW/PW res: [0x240000402:0x7fa:0x0].0x0 bits 0x2/0x0 rrc: 3 type: IBT gid 0 flags: 0x40210000000000 nid: local remote: 0x0 expref: -99 pid: 6255 timeout: 0 lvb_type: 0 00000100:00100000:2.0:1776178154.241136:0:6255:0:(service.c:2372:ptlrpc_server_handle_request()) Handled RPC req@0000000021d5d4e8 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_000:b50963bd-8eba-4432-9a54-8888d00e3fa1+27:17623:x1862456979091328:12345-192.168.204.3@tcp:36:touch.0 Request processed in 774us (1029us total) trans 17179876243 rc 0/0 00000100:00100000:2.0:1776178154.241144:0:6255:0:(nrs_fifo.c:241:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.204.3@tcp, seq: 6536 00010000:00010000:0.0:1776178154.250606:0:3340:0:(ldlm_lock.c:1064:ldlm_granted_list_add_lock()) ### About to add lock: ns: mdt-lustre-MDT0001_UUID lock: 00000000b27cf00e/0xa40083ab9bf533f8 lrc: 3/0,1 mode: COS/COS res: [0x240000402:0x7fa:0x0].0x0 bits 0x2/0x0 rrc: 3 type: IBT gid 0 flags: 0x50210000000000 nid: local remote: 0x0 expref: -99 pid: 6255 timeout: 0 lvb_type: 0 00010000:00010000:0.0:1776178154.250644:0:3340:0:(ldlm_inodebits.c:94:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x240000402:0x7fa:0x0].0x0 (000000006b0fcbcb) 00010000:00010000:0.0:1776178154.250657:0:3340:0:(ldlm_inodebits.c:96:ldlm_reprocess_inodebits_queue()) Hint 2 00000100:00100000:1.0:1776178154.251105:0:3347:0:(events.c:361:request_in_callback()) peer: 12345-192.168.204.3@tcp (source: 12345-192.168.204.3@tcp) 00000100:00100000:0.0:1776178154.251262:0:8227:0:(service.c:2152:ptlrpc_server_handle_req_in()) got req x1862456979091392 00000100:00100000:0.0:1776178154.251322:0:8227:0:(nrs_fifo.c:179:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.204.3@tcp, seq: 5259 00000100:00100000:0.0:1776178154.251329:0:8227:0:(service.c:2324:ptlrpc_server_handle_request()) Handling RPC req@00000000fb56871d pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_000:b50963bd-8eba-4432-9a54-8888d00e3fa1+6:2270:x1862456979091392:12345-192.168.204.3@tcp:2:touch.0 00000100:00100000:0.0:1776178154.251563:0:8227:0:(service.c:2372:ptlrpc_server_handle_request()) Handled RPC req@00000000fb56871d pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_000:b50963bd-8eba-4432-9a54-8888d00e3fa1+7:2270:x1862456979091392:12345-192.168.204.3@tcp:2:touch.0 Request processed in 234us (505us total) trans 4294969325 rc 0/0 00000100:00100000:0.0:1776178154.251571:0:8227:0:(nrs_fifo.c:241:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.204.3@tcp, seq: 5259 00000100:00100000:0.0:1776178154.264170:0:3346:0:(events.c:361:request_in_callback()) peer: 12345-192.168.204.3@tcp (source: 12345-192.168.204.3@tcp) 00000100:00100000:1.0:1776178154.264901:0:6258:0:(service.c:2152:ptlrpc_server_handle_req_in()) got req x1862456979091520 00000100:00100000:1.0:1776178154.265160:0:6258:0:(nrs_fifo.c:179:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.204.3@tcp, seq: 2137 00000100:00100000:1.0:1776178154.265168:0:6258:0:(service.c:2324:ptlrpc_server_handle_request()) Handling RPC req@000000006e2391ef pname:cluuid+ref:pid:xid:nid:opc:job mdt_rdpg00_000:b50963bd-8eba-4432-9a54-8888d00e3fa1+27:17623:x1862456979091520:12345-192.168.204.3@tcp:35:touch.0 00000100:00100000:1.0:1776178154.265488:0:6258:0:(service.c:2372:ptlrpc_server_handle_request()) Handled RPC req@000000006e2391ef pname:cluuid+ref:pid:xid:nid:opc:job mdt_rdpg00_000:b50963bd-8eba-4432-9a54-8888d00e3fa1+28:17623:x1862456979091520:12345-192.168.204.3@tcp:35:touch.0 Request processed in 320us (1326us total) trans 17179876244 rc 0/0 00000100:00100000:1.0:1776178154.265496:0:6258:0:(nrs_fifo.c:241:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.204.3@tcp, seq: 2137 00000100:00100000:3.0:1776178154.291991:0:3348:0:(events.c:361:request_in_callback()) peer: 12345-192.168.204.3@tcp (source: 12345-192.168.204.3@tcp) 00000100:00100000:1.0:1776178154.292204:0:8250:0:(service.c:2152:ptlrpc_server_handle_req_in()) got req x1862456979091648 00000100:00100000:1.0:1776178154.292265:0:8250:0:(nrs_fifo.c:179:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.204.3@tcp, seq: 6537 00000100:00100000:1.0:1776178154.292273:0:8250:0:(service.c:2324:ptlrpc_server_handle_request()) Handling RPC req@000000008c4bfd71 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_003:b50963bd-8eba-4432-9a54-8888d00e3fa1+28:17624:x1862456979091648:12345-192.168.204.3@tcp:101:touch.0 00010000:00010000:1.0:1776178154.292309:0:8250:0:(ldlm_lockd.c:1248:ldlm_handle_enqueue0()) ### server-side enqueue handler START 00010000:00010000:1.0:1776178154.292333:0:8250:0:(ldlm_lockd.c:1318:ldlm_handle_enqueue0()) ### server-side enqueue handler, new lock created ns: mdt-lustre-MDT0001_UUID lock: 000000003e1697a2/0xa40083ab9bf533ff lrc: 2/0,0 mode: --/CW res: [0x240000402:0x7f7:0x0].0x0 bits 0x0/0x0 rrc: 3 type: IBT gid 0 flags: 0x40000000000000 nid: local remote: 0x771f68aa1eb27ab6 expref: -99 pid: 8250 timeout: 0 lvb_type: 0 00010000:00010000:1.0:1776178154.292455:0:8250:0:(ldlm_lock.c:766:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(CW) ns: mdt-lustre-MDT0001_UUID lock: 0000000031bb7277/0xa40083ab9bf53406 lrc: 3/0,1 mode: --/CW res: [0x240000402:0x7f7:0x0].0x0 bits 0x0/0x0 rrc: 4 type: IBT gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 8250 timeout: 0 lvb_type: 0 00010000:00010000:1.0:1776178154.292475:0:8250:0:(ldlm_lock.c:1064:ldlm_granted_list_add_lock()) ### About to add lock: ns: mdt-lustre-MDT0001_UUID lock: 0000000031bb7277/0xa40083ab9bf53406 lrc: 3/0,1 mode: CW/CW res: [0x240000402:0x7f7:0x0].0x0 bits 0x2/0x0 rrc: 4 type: IBT gid 0 flags: 0x50210000000000 nid: local remote: 0x0 expref: -99 pid: 8250 timeout: 0 lvb_type: 0 00010000:00010000:1.0:1776178154.292489:0:8250:0:(ldlm_request.c:528:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: mdt-lustre-MDT0001_UUID lock: 0000000031bb7277/0xa40083ab9bf53406 lrc: 3/0,1 mode: CW/CW res: [0x240000402:0x7f7:0x0].0x0 bits 0x2/0x0 rrc: 4 type: IBT gid 0 flags: 0x40210000000000 nid: local remote: 0x0 expref: -99 pid: 8250 timeout: 0 lvb_type: 0 00010000:00010000:1.0:1776178154.292509:0:8250:0:(ldlm_lock.c:766:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: mdt-lustre-MDT0001_UUID lock: 000000005dc671a2/0xa40083ab9bf5340d lrc: 3/0,1 mode: --/PW res: [0x240000402:0x7f7:0x0].0xa6b02dea bits 0x0/0x0 rrc: 2 type: IBT gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 8250 timeout: 0 lvb_type: 0 00010000:00010000:1.0:1776178154.292518:0:8250:0:(ldlm_lock.c:1064:ldlm_granted_list_add_lock()) ### About to add lock: ns: mdt-lustre-MDT0001_UUID lock: 000000005dc671a2/0xa40083ab9bf5340d lrc: 3/0,1 mode: PW/PW res: [0x240000402:0x7f7:0x0].0xa6b02dea bits 0x2/0x0 rrc: 2 type: IBT gid 0 flags: 0x50210000000000 nid: local remote: 0x0 expref: -99 pid: 8250 timeout: 0 lvb_type: 0 00010000:00010000:1.0:1776178154.292527:0:8250:0:(ldlm_request.c:528:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: mdt-lustre-MDT0001_UUID lock: 000000005dc671a2/0xa40083ab9bf5340d lrc: 3/0,1 mode: PW/PW res: [0x240000402:0x7f7:0x0].0xa6b02dea bits 0x2/0x0 rrc: 2 type: IBT gid 0 flags: 0x40210000000000 nid: local remote: 0x0 expref: -99 pid: 8250 timeout: 0 lvb_type: 0 00000004:00080000:1.0:1776178154.292877:0:8250:0:(osp_object.c:1645:osp_create()) lustre-OST0000-osc-MDT0001: Wrote last used FID: [0x280000400:0x46f:0x0], index 0: 0 00010000:00010000:1.0:1776178154.293017:0:8250:0:(ldlm_lock.c:766:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(CR) ns: mdt-lustre-MDT0001_UUID lock: 00000000f314daa9/0xa40083ab9bf53414 lrc: 3/1,0 mode: --/CR res: [0x240000402:0x7fb:0x0].0x0 bits 0x0/0x0 rrc: 2 type: IBT gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 8250 timeout: 0 lvb_type: 0 00010000:00010000:1.0:1776178154.293026:0:8250:0:(ldlm_lock.c:1064:ldlm_granted_list_add_lock()) ### About to add lock: ns: mdt-lustre-MDT0001_UUID lock: 00000000f314daa9/0xa40083ab9bf53414 lrc: 3/1,0 mode: CR/CR res: [0x240000402:0x7fb:0x0].0x0 bits 0x9/0x0 rrc: 2 type: IBT gid 0 flags: 0x50210000000000 nid: local remote: 0x0 expref: -99 pid: 8250 timeout: 0 lvb_type: 0 00010000:00010000:1.0:1776178154.293033:0:8250:0:(ldlm_request.c:528:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: mdt-lustre-MDT0001_UUID lock: 00000000f314daa9/0xa40083ab9bf53414 lrc: 3/1,0 mode: CR/CR res: [0x240000402:0x7fb:0x0].0x0 bits 0x9/0x0 rrc: 2 type: IBT gid 0 flags: 0x40210000000000 nid: local remote: 0x0 expref: -99 pid: 8250 timeout: 0 lvb_type: 0 00010000:00010000:1.0:1776178154.293095:0:8250:0:(ldlm_lock.c:820:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(CW) ns: mdt-lustre-MDT0001_UUID lock: 0000000031bb7277/0xa40083ab9bf53406 lrc: 3/0,1 mode: CW/CW res: [0x240000402:0x7f7:0x0].0x0 bits 0x2/0x0 rrc: 4 type: IBT gid 0 flags: 0x50210000000000 nid: local remote: 0x0 expref: -99 pid: 8250 timeout: 0 lvb_type: 0 00010000:00010000:1.0:1776178154.293102:0:8250:0:(ldlm_lock.c:879:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: mdt-lustre-MDT0001_UUID lock: 0000000031bb7277/0xa40083ab9bf53406 lrc: 2/0,0 mode: CW/CW res: [0x240000402:0x7f7:0x0].0x0 bits 0x2/0x0 rrc: 4 type: IBT gid 0 flags: 0x50210400000000 nid: local remote: 0x0 expref: -99 pid: 8250 timeout: 0 lvb_type: 0 00010000:00010000:1.0:1776178154.293108:0:8250:0:(ldlm_lockd.c:1879:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: mdt-lustre-MDT0001_UUID lock: 0000000031bb7277/0xa40083ab9bf53406 lrc: 3/0,0 mode: CW/CW res: [0x240000402:0x7f7:0x0].0x0 bits 0x2/0x0 rrc: 4 type: IBT gid 0 flags: 0x40210400000000 nid: local remote: 0x0 expref: -99 pid: 8250 timeout: 0 lvb_type: 0 00010000:00010000:1.0:1776178154.293115:0:8250:0:(ldlm_lockd.c:1893:ldlm_handle_bl_callback()) Lock 0000000031bb7277 already unused, calling callback (0000000079e78348) 00010000:00010000:1.0:1776178154.293118:0:8250:0:(ldlm_request.c:364:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: mdt-lustre-MDT0001_UUID lock: 0000000031bb7277/0xa40083ab9bf53406 lrc: 3/0,0 mode: CW/CW res: [0x240000402:0x7f7:0x0].0x0 bits 0x2/0x0 rrc: 4 type: IBT gid 0 flags: 0x40210400000000 nid: local remote: 0x0 expref: -99 pid: 8250 timeout: 0 lvb_type: 0 00010000:00010000:1.0:1776178154.293124:0:8250:0:(ldlm_request.c:1257:ldlm_cli_cancel_local()) ### server-side local cancel ns: mdt-lustre-MDT0001_UUID lock: 0000000031bb7277/0xa40083ab9bf53406 lrc: 4/0,0 mode: CW/CW res: [0x240000402:0x7f7:0x0].0x0 bits 0x2/0x0 rrc: 4 type: IBT gid 0 flags: 0x40218400000000 nid: local remote: 0x0 expref: -99 pid: 8250 timeout: 0 lvb_type: 0 00010000:00010000:1.0:1776178154.293136:0:8250:0:(ldlm_inodebits.c:94:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x240000402:0x7f7:0x0].0x0 (000000006b762dc5) 00010000:00010000:1.0:1776178154.293139:0:8250:0:(ldlm_inodebits.c:96:ldlm_reprocess_inodebits_queue()) Hint 2 00010000:00010000:1.0:1776178154.293142:0:8250:0:(ldlm_lockd.c:1903:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: mdt-lustre-MDT0001_UUID lock: 0000000031bb7277/0xa40083ab9bf53406 lrc: 2/0,0 mode: --/CW res: [0x240000402:0x7f7:0x0].0x0 bits 0x2/0x0 rrc: 4 type: IBT gid 0 flags: 0x44a19400000000 nid: local remote: 0x0 expref: -99 pid: 8250 timeout: 0 lvb_type: 0 00010000:00010000:1.0:1776178154.293148:0:8250:0:(ldlm_lock.c:216:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: mdt-lustre-MDT0001_UUID lock: 0000000031bb7277/0xa40083ab9bf53406 lrc: 0/0,0 mode: --/CW res: [0x240000402:0x7f7:0x0].0x0 bits 0x2/0x0 rrc: 4 type: IBT gid 0 flags: 0x44a19400000000 nid: local remote: 0x0 expref: -99 pid: 8250 timeout: 0 lvb_type: 0 00000004:00010000:1.0:1776178154.293160:0:8250:0:(mdt_handler.c:3992:mdt_save_lock()) ### save lock request 000000008c4bfd71 reply state 000000000003413f transno 17179876245 ns: mdt-lustre-MDT0001_UUID lock: 000000005dc671a2/0xa40083ab9bf5340d lrc: 3/0,1 mode: PW/PW res: [0x240000402:0x7f7:0x0].0xa6b02dea bits 0x2/0x0 rrc: 2 type: IBT gid 0 flags: 0x40210000000000 nid: local remote: 0x0 expref: -99 pid: 8250 timeout: 0 lvb_type: 0 00010000:00010000:1.0:1776178154.293190:0:8250:0:(ldlm_lock.c:216:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: mdt-lustre-MDT0001_UUID lock: 000000003e1697a2/0xa40083ab9bf533ff lrc: 0/0,0 mode: --/CW res: [0x240000402:0x7f7:0x0].0x0 bits 0x1/0x0 rrc: 3 type: IBT gid 0 flags: 0x44000000000000 nid: 192.168.204.3@tcp remote: 0x771f68aa1eb27ab6 expref: 31 pid: 8250 timeout: 0 lvb_type: 0 00010000:00010000:1.0:1776178154.293201:0:8250:0:(ldlm_lockd.c:1484:ldlm_handle_enqueue0()) ### server-side enqueue handler, sending reply (err=0, rc=0) ns: mdt-lustre-MDT0001_UUID lock: 00000000f314daa9/0xa40083ab9bf53414 lrc: 3/0,0 mode: CR/CR res: [0x240000402:0x7fb:0x0].0x0 bits 0x9/0x0 rrc: 2 type: IBT gid 0 flags: 0x40200000000000 nid: 192.168.204.3@tcp remote: 0x771f68aa1eb27ab6 expref: 30 pid: 8250 timeout: 0 lvb_type: 0 00010000:00010000:1.0:1776178154.293207:0:8250:0:(ldlm_inodebits.c:94:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x240000402:0x7fb:0x0].0x0 (000000009f7659df) 00010000:00010000:1.0:1776178154.293209:0:8250:0:(ldlm_inodebits.c:96:ldlm_reprocess_inodebits_queue()) Hint 9 00010000:00010000:1.0:1776178154.293211:0:8250:0:(ldlm_lockd.c:1565:ldlm_handle_enqueue0()) ### server-side enqueue handler END (lock 00000000f314daa9, rc 0) 00000100:00100000:1.0:1776178154.293285:0:8250:0:(service.c:2372:ptlrpc_server_handle_request()) Handled RPC req@000000008c4bfd71 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_003:b50963bd-8eba-4432-9a54-8888d00e3fa1+31:17624:x1862456979091648:12345-192.168.204.3@tcp:101:touch.0 Request processed in 1011us (1304us total) trans 17179876245 rc 0/0 00000100:00100000:1.0:1776178154.293294:0:8250:0:(nrs_fifo.c:241:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.204.3@tcp, seq: 6537 00010000:00010000:3.0:1776178154.301547:0:3339:0:(ldlm_lock.c:1064:ldlm_granted_list_add_lock()) ### About to add lock: ns: mdt-lustre-MDT0001_UUID lock: 000000005dc671a2/0xa40083ab9bf5340d lrc: 3/0,1 mode: COS/COS res: [0x240000402:0x7f7:0x0].0xa6b02dea bits 0x2/0x0 rrc: 2 type: IBT gid 0 flags: 0x50210000000000 nid: local remote: 0x0 expref: -99 pid: 8250 timeout: 0 lvb_type: 0 00010000:00010000:3.0:1776178154.301575:0:3339:0:(ldlm_inodebits.c:94:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x240000402:0x7f7:0x0].0xa6b02dea (0000000048823789) 00010000:00010000:3.0:1776178154.301578:0:3339:0:(ldlm_inodebits.c:96:ldlm_reprocess_inodebits_queue()) Hint 2 00000100:00100000:0.0:1776178154.314661:0:3348:0:(events.c:361:request_in_callback()) peer: 12345-192.168.204.3@tcp (source: 12345-192.168.204.3@tcp) 00000100:00100000:2.0:1776178154.315249:0:6255:0:(service.c:2152:ptlrpc_server_handle_req_in()) got req x1862456979091776 00000100:00100000:2.0:1776178154.315341:0:6255:0:(nrs_fifo.c:179:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.204.3@tcp, seq: 6538 00000100:00100000:2.0:1776178154.315349:0:6255:0:(service.c:2324:ptlrpc_server_handle_request()) Handling RPC req@000000008245b0ee pname:cluuid+ref:pid:xid:nid:opc:job mdt00_000:b50963bd-8eba-4432-9a54-8888d00e3fa1+31:17624:x1862456979091776:12345-192.168.204.3@tcp:36:touch.0 00010000:00010000:2.0:1776178154.315503:0:6255:0:(ldlm_lock.c:766:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: mdt-lustre-MDT0001_UUID lock: 000000007e5ef416/0xa40083ab9bf53422 lrc: 3/0,1 mode: --/PW res: [0x240000402:0x7fb:0x0].0x0 bits 0x0/0x0 rrc: 3 type: IBT gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 6255 timeout: 0 lvb_type: 0 00010000:00010000:2.0:1776178154.315525:0:6255:0:(ldlm_lock.c:1064:ldlm_granted_list_add_lock()) ### About to add lock: ns: mdt-lustre-MDT0001_UUID lock: 000000007e5ef416/0xa40083ab9bf53422 lrc: 3/0,1 mode: PW/PW res: [0x240000402:0x7fb:0x0].0x0 bits 0x2/0x0 rrc: 3 type: IBT gid 0 flags: 0x50210000000000 nid: local remote: 0x0 expref: -99 pid: 6255 timeout: 0 lvb_type: 0 00010000:00010000:2.0:1776178154.315540:0:6255:0:(ldlm_request.c:528:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: mdt-lustre-MDT0001_UUID lock: 000000007e5ef416/0xa40083ab9bf53422 lrc: 3/0,1 mode: PW/PW res: [0x240000402:0x7fb:0x0].0x0 bits 0x2/0x0 rrc: 3 type: IBT gid 0 flags: 0x40210000000000 nid: local remote: 0x0 expref: -99 pid: 6255 timeout: 0 lvb_type: 0 00000004:00010000:2.0:1776178154.315687:0:6255:0:(mdt_handler.c:3992:mdt_save_lock()) ### save lock request 000000008245b0ee reply state 00000000398f4256 transno 17179876246 ns: mdt-lustre-MDT0001_UUID lock: 000000007e5ef416/0xa40083ab9bf53422 lrc: 3/0,1 mode: PW/PW res: [0x240000402:0x7fb:0x0].0x0 bits 0x2/0x0 rrc: 3 type: IBT gid 0 flags: 0x40210000000000 nid: local remote: 0x0 expref: -99 pid: 6255 timeout: 0 lvb_type: 0 00000100:00100000:2.0:1776178154.315837:0:6255:0:(service.c:2372:ptlrpc_server_handle_request()) Handled RPC req@000000008245b0ee pname:cluuid+ref:pid:xid:nid:opc:job mdt00_000:b50963bd-8eba-4432-9a54-8888d00e3fa1+33:17624:x1862456979091776:12345-192.168.204.3@tcp:36:touch.0 Request processed in 487us (1193us total) trans 17179876246 rc 0/0 00000100:00100000:2.0:1776178154.315844:0:6255:0:(nrs_fifo.c:241:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.204.3@tcp, seq: 6538 00010000:00010000:1.0:1776178154.321988:0:3340:0:(ldlm_lock.c:1064:ldlm_granted_list_add_lock()) ### About to add lock: ns: mdt-lustre-MDT0001_UUID lock: 000000007e5ef416/0xa40083ab9bf53422 lrc: 3/0,1 mode: COS/COS res: [0x240000402:0x7fb:0x0].0x0 bits 0x2/0x0 rrc: 3 type: IBT gid 0 flags: 0x50210000000000 nid: local remote: 0x0 expref: -99 pid: 6255 timeout: 0 lvb_type: 0 00010000:00010000:1.0:1776178154.322012:0:3340:0:(ldlm_inodebits.c:94:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x240000402:0x7fb:0x0].0x0 (000000009f7659df) 00010000:00010000:1.0:1776178154.322015:0:3340:0:(ldlm_inodebits.c:96:ldlm_reprocess_inodebits_queue()) Hint 2 00000100:00100000:0.0:1776178154.330597:0:3348:0:(events.c:361:request_in_callback()) peer: 12345-192.168.204.3@tcp (source: 12345-192.168.204.3@tcp) 00000100:00100000:2.0:1776178154.332705:0:8228:0:(service.c:2152:ptlrpc_server_handle_req_in()) got req x1862456979091840 00000100:00100000:2.0:1776178154.332799:0:8228:0:(nrs_fifo.c:179:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.204.3@tcp, seq: 5260 00000100:00100000:2.0:1776178154.332806:0:8228:0:(service.c:2324:ptlrpc_server_handle_request()) Handling RPC req@00000000dd1281fe pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_001:b50963bd-8eba-4432-9a54-8888d00e3fa1+6:2271:x1862456979091840:12345-192.168.204.3@tcp:2:touch.0 00000100:00100000:2.0:1776178154.333154:0:8228:0:(service.c:2372:ptlrpc_server_handle_request()) Handled RPC req@00000000dd1281fe pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_001:b50963bd-8eba-4432-9a54-8888d00e3fa1+7:2271:x1862456979091840:12345-192.168.204.3@tcp:2:touch.0 Request processed in 347us (2565us total) trans 4294969354 rc 0/0 00000100:00100000:2.0:1776178154.333163:0:8228:0:(nrs_fifo.c:241:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.204.3@tcp, seq: 5260 00000100:00100000:3.0:1776178154.336784:0:3354:0:(client.c:1733:ptlrpc_send_new_req()) Sending RPC req@00000000b821049d pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_01:ac61f839-0e8e-4cac-9c59-08bcfbbad330:3354:1862456998034880:0@lo:400:kworker/u8:2.0 00000100:00100000:3.0:1776178154.336871:0:3354:0:(events.c:361:request_in_callback()) peer: 12345-0@lo (source: 12345-0@lo) 00000100:00100000:3.0:1776178154.336898:0:3354:0:(client.c:1733:ptlrpc_send_new_req()) Sending RPC req@000000005b125149 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_01:lustre-MDT0001-lwp-OST0001_UUID:3354:1862456998035136:0@lo:400:kworker/u8:2.0 00000100:00100000:3.0:1776178154.336927:0:3354:0:(events.c:361:request_in_callback()) peer: 12345-0@lo (source: 12345-0@lo) 00000100:00100000:3.0:1776178154.336955:0:3354:0:(client.c:1733:ptlrpc_send_new_req()) Sending RPC req@000000000c9fb390 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_01:lustre-MDT0000-mdtlov_UUID:3354:1862456998035392:0@lo:400:kworker/u8:2.0 00000100:00100000:3.0:1776178154.336980:0:3354:0:(events.c:361:request_in_callback()) peer: 12345-0@lo (source: 12345-0@lo) 00000100:00100000:3.0:1776178154.337005:0:3354:0:(ptlrpcd.c:415:ptlrpcd_check()) transfer 2 async RPCs [0->1] 00000100:00100000:3.0:1776178154.337011:0:3354:0:(client.c:1733:ptlrpc_send_new_req()) Sending RPC req@000000005fdea934 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_01:lustre-MDT0000-lwp-OST0001_UUID:3354:1862456998035072:0@lo:400:kworker/u8:2.0 00000100:00100000:3.0:1776178154.337043:0:3354:0:(events.c:361:request_in_callback()) peer: 12345-0@lo (source: 12345-0@lo) 00000100:00100000:3.0:1776178154.337087:0:3354:0:(client.c:1733:ptlrpc_send_new_req()) Sending RPC req@000000006b9d8ccb pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_01:lustre-MDT0000-lwp-MDT0001_UUID:3354:1862456998035328:0@lo:400:kworker/u8:2.0 00000100:00100000:3.0:1776178154.337112:0:3354:0:(events.c:361:request_in_callback()) peer: 12345-0@lo (source: 12345-0@lo) 00000100:00100000:0.0:1776178154.337311:0:3354:0:(client.c:1733:ptlrpc_send_new_req()) Sending RPC req@000000005927b2c3 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_01:lustre-MDT0000-mdtlov_UUID:3354:1862456998035648:0@lo:13:osp-pre-0-0.0 00000100:00100000:0.0:1776178154.337342:0:3354:0:(events.c:361:request_in_callback()) peer: 12345-0@lo (source: 12345-0@lo) 00000100:00100000:0.0:1776178154.337372:0:3354:0:(ptlrpcd.c:415:ptlrpcd_check()) transfer 1 async RPCs [0->1] 00000100:00100000:0.0:1776178154.337378:0:3354:0:(client.c:1733:ptlrpc_send_new_req()) Sending RPC req@00000000f0ef812c pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_01:lustre-MDT0000-mdtlov_UUID:3354:1862456998035584:0@lo:13:osp-pre-1-0.0 00000100:00100000:0.0:1776178154.337400:0:3354:0:(events.c:361:request_in_callback()) peer: 12345-0@lo (source: 12345-0@lo) 00000100:00100000:0.0:1776178154.337435:0:8232:0:(service.c:2152:ptlrpc_server_handle_req_in()) got req x1862456998035648 00000100:00100000:0.0:1776178154.337489:0:8232:0:(service.c:2152:ptlrpc_server_handle_req_in()) got req x1862456998035584 00000100:00100000:0.0:1776178154.337505:0:8232:0:(nrs_fifo.c:179:nrs_fifo_req_get()) NRS start fifo request from 12345-0@lo, seq: 439 00000100:00100000:0.0:1776178154.337513:0:8232:0:(service.c:2324:ptlrpc_server_handle_request()) Handling RPC req@00000000f8c48966 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_create00:lustre-MDT0000-mdtlov_UUID+9:3354:x1862456998035648:12345-0@lo:13:osp-pre-0-0.0 00002000:00000024:0.0:1776178154.337547:0:8232:0:(ofd_obd.c:740:ofd_statfs()) blocks cached 315392 granted 484883776 pending 0 free 3923689472 avail 3692167168 00000100:00100000:0.0:1776178154.337593:0:8232:0:(service.c:2372:ptlrpc_server_handle_request()) Handled RPC req@00000000f8c48966 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_create00:lustre-MDT0000-mdtlov_UUID+9:3354:x1862456998035648:12345-0@lo:13:osp-pre-0-0.0 Request processed in 80us (250us total) trans 0 rc 0/0 00000100:00100000:0.0:1776178154.337601:0:8232:0:(nrs_fifo.c:241:nrs_fifo_req_stop()) NRS stop fifo request from 12345-0@lo, seq: 439 00000100:00100000:0.0:1776178154.337618:0:8232:0:(nrs_fifo.c:179:nrs_fifo_req_get()) NRS start fifo request from 12345-0@lo, seq: 440 00000100:00100000:0.0:1776178154.337622:0:8232:0:(service.c:2324:ptlrpc_server_handle_request()) Handling RPC req@000000007fcab602 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_create00:lustre-MDT0000-mdtlov_UUID+8:3354:x1862456998035584:12345-0@lo:13:osp-pre-1-0.0 00002000:00000024:0.0:1776178154.337639:0:8232:0:(ofd_obd.c:740:ofd_statfs()) blocks cached 229376 granted 491746368 pending 0 free 3923689472 avail 3692167168 00000100:00100000:0.0:1776178154.337674:0:8232:0:(service.c:2372:ptlrpc_server_handle_request()) Handled RPC req@000000007fcab602 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_create00:lustre-MDT0000-mdtlov_UUID+8:3354:x1862456998035584:12345-0@lo:13:osp-pre-1-0.0 Request processed in 51us (274us total) trans 0 rc 0/0 00000100:00100000:0.0:1776178154.337680:0:8232:0:(nrs_fifo.c:241:nrs_fifo_req_stop()) NRS stop fifo request from 12345-0@lo, seq: 440 00000100:00100000:0.0:1776178154.337713:0:8227:0:(service.c:2152:ptlrpc_server_handle_req_in()) got req x1862456998035392 00000100:00100000:0.0:1776178154.337733:0:8227:0:(nrs_fifo.c:179:nrs_fifo_req_get()) NRS start fifo request from 12345-0@lo, seq: 460 00000100:00100000:0.0:1776178154.337737:0:8227:0:(service.c:2324:ptlrpc_server_handle_request()) Handling RPC req@00000000a49c1305 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_000:lustre-MDT0000-mdtlov_UUID+9:3354:x1862456998035392:12345-0@lo:400:kworker/u8:2.0 00000100:00100000:0.0:1776178154.337763:0:8227:0:(service.c:2372:ptlrpc_server_handle_request()) Handled RPC req@00000000a49c1305 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_000:lustre-MDT0000-mdtlov_UUID+9:3354:x1862456998035392:12345-0@lo:400:kworker/u8:2.0 Request processed in 26us (783us total) trans 0 rc 0/0 00000100:00100000:0.0:1776178154.337769:0:8227:0:(nrs_fifo.c:241:nrs_fifo_req_stop()) NRS stop fifo request from 12345-0@lo, seq: 460 00000100:00100000:0.0:1776178154.337808:0:3354:0:(client.c:2215:ptlrpc_check_set()) Completed RPC req@00000000f0ef812c pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_01:lustre-MDT0000-mdtlov_UUID:3354:1862456998035584:0@lo:13:osp-pre-1-0.0 00000100:00100000:0.0:1776178154.337821:0:3354:0:(client.c:2215:ptlrpc_check_set()) Completed RPC req@000000005927b2c3 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_01:lustre-MDT0000-mdtlov_UUID:3354:1862456998035648:0@lo:13:osp-pre-0-0.0 00000100:00100000:0.0:1776178154.337830:0:3354:0:(client.c:2215:ptlrpc_check_set()) Completed RPC req@000000000c9fb390 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_01:lustre-MDT0000-mdtlov_UUID:3354:1862456998035392:0@lo:400:kworker/u8:2.0 00000100:00100000:1.0:1776178154.338008:0:3355:0:(client.c:1733:ptlrpc_send_new_req()) Sending RPC req@00000000a23c68f9 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_02:lustre-MDT0000-lwp-OST0000_UUID:3355:1862456998034944:0@lo:400:kworker/u8:2.0 00000100:00100000:1.0:1776178154.338043:0:3355:0:(events.c:361:request_in_callback()) peer: 12345-0@lo (source: 12345-0@lo) 00000100:00100000:1.0:1776178154.338088:0:3355:0:(client.c:1733:ptlrpc_send_new_req()) Sending RPC req@000000005accf721 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_02:lustre-MDT0001-mdtlov_UUID:3355:1862456998035200:0@lo:400:kworker/u8:2.0 00000100:00100000:1.0:1776178154.338113:0:3355:0:(events.c:361:request_in_callback()) peer: 12345-0@lo (source: 12345-0@lo) 00000100:00100000:1.0:1776178154.338125:0:3355:0:(client.c:1733:ptlrpc_send_new_req()) Sending RPC req@000000005f3eedd2 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_02:lustre-MDT0000-mdtlov_UUID:3355:1862456998035456:0@lo:400:kworker/u8:2.0 00000100:00100000:1.0:1776178154.338153:0:3355:0:(events.c:361:request_in_callback()) peer: 12345-0@lo (source: 12345-0@lo) 00000100:00100000:1.0:1776178154.338164:0:3355:0:(client.c:1733:ptlrpc_send_new_req()) Sending RPC req@000000004a9818ac pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_02:lustre-MDT0001-mdtlov_UUID:3355:1862456998035712:0@lo:41:osp-pre-0-1.0 00000100:00100000:1.0:1776178154.338186:0:3355:0:(events.c:361:request_in_callback()) peer: 12345-0@lo (source: 12345-0@lo) 00000100:00100000:1.0:1776178154.338205:0:3355:0:(ptlrpcd.c:415:ptlrpcd_check()) transfer 3 async RPCs [3->2] 00000100:00100000:1.0:1776178154.338211:0:3355:0:(client.c:1733:ptlrpc_send_new_req()) Sending RPC req@0000000078c1d864 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_02:lustre-MDT0001-lwp-OST0000_UUID:3355:1862456998035008:0@lo:400:kworker/u8:2.0 00000100:00100000:1.0:1776178154.338232:0:3355:0:(events.c:361:request_in_callback()) peer: 12345-0@lo (source: 12345-0@lo) 00000100:00100000:1.0:1776178154.338249:0:3355:0:(client.c:1733:ptlrpc_send_new_req()) Sending RPC req@0000000099e8e662 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_02:lustre-MDT0001-mdtlov_UUID:3355:1862456998035264:0@lo:400:kworker/u8:2.0 00000100:00100000:1.0:1776178154.338273:0:3355:0:(events.c:361:request_in_callback()) peer: 12345-0@lo (source: 12345-0@lo) 00000100:00100000:1.0:1776178154.338286:0:3355:0:(client.c:1733:ptlrpc_send_new_req()) Sending RPC req@00000000ddf29755 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_02:lustre-MDT0000-lwp-MDT0000_UUID:3355:1862456998035520:0@lo:400:kworker/u8:2.0 00000100:00100000:1.0:1776178154.338304:0:3355:0:(events.c:361:request_in_callback()) peer: 12345-0@lo (source: 12345-0@lo) 00000100:00100000:1.0:1776178154.338325:0:8250:0:(service.c:2152:ptlrpc_server_handle_req_in()) got req x1862456998035136 00000100:00100000:1.0:1776178154.338349:0:8250:0:(service.c:2152:ptlrpc_server_handle_req_in()) got req x1862456998035072 00000100:00100000:1.0:1776178154.338359:0:8250:0:(service.c:2152:ptlrpc_server_handle_req_in()) got req x1862456998035328 00000100:00100000:1.0:1776178154.338368:0:8250:0:(service.c:2152:ptlrpc_server_handle_req_in()) got req x1862456998034944 00000100:00100000:1.0:1776178154.338376:0:8250:0:(service.c:2152:ptlrpc_server_handle_req_in()) got req x1862456998035008 00000100:00100000:1.0:1776178154.338384:0:8250:0:(service.c:2152:ptlrpc_server_handle_req_in()) got req x1862456998035520 00000100:00100000:1.0:1776178154.338393:0:8250:0:(nrs_fifo.c:179:nrs_fifo_req_get()) NRS start fifo request from 12345-0@lo, seq: 829 00000100:00100000:1.0:1776178154.338396:0:8250:0:(service.c:2324:ptlrpc_server_handle_request()) Handling RPC req@000000006344f537 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_003:lustre-MDT0001-lwp-OST0001_UUID+5:3354:x1862456998035136:12345-0@lo:400:kworker/u8:2.0 00000100:00100000:1.0:1776178154.338431:0:8250:0:(service.c:2372:ptlrpc_server_handle_request()) Handled RPC req@000000006344f537 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_003:lustre-MDT0001-lwp-OST0001_UUID+5:3354:x1862456998035136:12345-0@lo:400:kworker/u8:2.0 Request processed in 35us (1505us total) trans 0 rc 0/0 00000100:00100000:1.0:1776178154.338438:0:8250:0:(nrs_fifo.c:241:nrs_fifo_req_stop()) NRS stop fifo request from 12345-0@lo, seq: 829 00000100:00100000:1.0:1776178154.338448:0:8250:0:(nrs_fifo.c:179:nrs_fifo_req_get()) NRS start fifo request from 12345-0@lo, seq: 830 00000100:00100000:1.0:1776178154.338451:0:8250:0:(service.c:2324:ptlrpc_server_handle_request()) Handling RPC req@0000000050c5b9a3 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_003:lustre-MDT0000-lwp-OST0001_UUID+5:3354:x1862456998035072:12345-0@lo:400:kworker/u8:2.0 00000100:00100000:1.0:1776178154.338471:0:8250:0:(service.c:2372:ptlrpc_server_handle_request()) Handled RPC req@0000000050c5b9a3 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_003:lustre-MDT0000-lwp-OST0001_UUID+5:3354:x1862456998035072:12345-0@lo:400:kworker/u8:2.0 Request processed in 20us (1428us total) trans 0 rc 0/0 00000100:00100000:1.0:1776178154.338476:0:8250:0:(nrs_fifo.c:241:nrs_fifo_req_stop()) NRS stop fifo request from 12345-0@lo, seq: 830 00000100:00100000:1.0:1776178154.338483:0:8250:0:(nrs_fifo.c:179:nrs_fifo_req_get()) NRS start fifo request from 12345-0@lo, seq: 831 00000100:00100000:1.0:1776178154.338486:0:8250:0:(service.c:2324:ptlrpc_server_handle_request()) Handling RPC req@0000000030978cff pname:cluuid+ref:pid:xid:nid:opc:job mdt00_003:lustre-MDT0000-lwp-MDT0001_UUID+5:3354:x1862456998035328:12345-0@lo:400:kworker/u8:2.0 00000100:00100000:1.0:1776178154.338505:0:8250:0:(service.c:2372:ptlrpc_server_handle_request()) Handled RPC req@0000000030978cff pname:cluuid+ref:pid:xid:nid:opc:job mdt00_003:lustre-MDT0000-lwp-MDT0001_UUID+5:3354:x1862456998035328:12345-0@lo:400:kworker/u8:2.0 Request processed in 18us (1393us total) trans 0 rc 0/0 00000100:00100000:1.0:1776178154.338510:0:8250:0:(nrs_fifo.c:241:nrs_fifo_req_stop()) NRS stop fifo request from 12345-0@lo, seq: 831 00000100:00100000:1.0:1776178154.338518:0:8250:0:(nrs_fifo.c:179:nrs_fifo_req_get()) NRS start fifo request from 12345-0@lo, seq: 832 00000100:00100000:1.0:1776178154.338520:0:8250:0:(service.c:2324:ptlrpc_server_handle_request()) Handling RPC req@00000000a2512e82 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_003:lustre-MDT0000-lwp-OST0000_UUID+5:3355:x1862456998034944:12345-0@lo:400:kworker/u8:2.0 00000100:00100000:1.0:1776178154.338543:0:8250:0:(service.c:2372:ptlrpc_server_handle_request()) Handled RPC req@00000000a2512e82 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_003:lustre-MDT0000-lwp-OST0000_UUID+5:3355:x1862456998034944:12345-0@lo:400:kworker/u8:2.0 Request processed in 22us (501us total) trans 0 rc 0/0 00000100:00100000:1.0:1776178154.338548:0:8250:0:(nrs_fifo.c:241:nrs_fifo_req_stop()) NRS stop fifo request from 12345-0@lo, seq: 832 00000100:00100000:1.0:1776178154.338570:0:3355:0:(client.c:2215:ptlrpc_check_set()) Completed RPC req@00000000a23c68f9 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_02:lustre-MDT0000-lwp-OST0000_UUID:3355:1862456998034944:0@lo:400:kworker/u8:2.0 00000100:00100000:1.0:1776178154.338588:0:8250:0:(nrs_fifo.c:179:nrs_fifo_req_get()) NRS start fifo request from 12345-0@lo, seq: 833 00000100:00100000:1.0:1776178154.338590:0:8250:0:(service.c:2324:ptlrpc_server_handle_request()) Handling RPC req@000000008b858f58 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_003:lustre-MDT0001-lwp-OST0000_UUID+5:3355:x1862456998035008:12345-0@lo:400:kworker/u8:2.0 00000100:00100000:1.0:1776178154.338617:0:8250:0:(service.c:2372:ptlrpc_server_handle_request()) Handled RPC req@000000008b858f58 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_003:lustre-MDT0001-lwp-OST0000_UUID+5:3355:x1862456998035008:12345-0@lo:400:kworker/u8:2.0 Request processed in 25us (385us total) trans 0 rc 0/0 00000100:00100000:1.0:1776178154.338622:0:8250:0:(nrs_fifo.c:241:nrs_fifo_req_stop()) NRS stop fifo request from 12345-0@lo, seq: 833 00000100:00100000:1.0:1776178154.338640:0:3355:0:(client.c:2215:ptlrpc_check_set()) Completed RPC req@0000000078c1d864 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_02:lustre-MDT0001-lwp-OST0000_UUID:3355:1862456998035008:0@lo:400:kworker/u8:2.0 00000100:00100000:1.0:1776178154.338673:0:8250:0:(nrs_fifo.c:179:nrs_fifo_req_get()) NRS start fifo request from 12345-0@lo, seq: 834 00000100:00100000:1.0:1776178154.338677:0:8250:0:(service.c:2324:ptlrpc_server_handle_request()) Handling RPC req@00000000812a7ebe pname:cluuid+ref:pid:xid:nid:opc:job mdt00_003:lustre-MDT0000-lwp-MDT0000_UUID+5:3355:x1862456998035520:12345-0@lo:400:kworker/u8:2.0 00000100:00100000:1.0:1776178154.338705:0:8250:0:(service.c:2372:ptlrpc_server_handle_request()) Handled RPC req@00000000812a7ebe pname:cluuid+ref:pid:xid:nid:opc:job mdt00_003:lustre-MDT0000-lwp-MDT0000_UUID+5:3355:x1862456998035520:12345-0@lo:400:kworker/u8:2.0 Request processed in 28us (400us total) trans 0 rc 0/0 00000100:00100000:1.0:1776178154.338710:0:8250:0:(nrs_fifo.c:241:nrs_fifo_req_stop()) NRS stop fifo request from 12345-0@lo, seq: 834 00000100:00100000:1.0:1776178154.338729:0:3355:0:(client.c:2215:ptlrpc_check_set()) Completed RPC req@00000000ddf29755 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_02:lustre-MDT0000-lwp-MDT0000_UUID:3355:1862456998035520:0@lo:400:kworker/u8:2.0 00000100:00100000:0.0:1776178154.339194:0:15213:0:(service.c:2152:ptlrpc_server_handle_req_in()) got req x1862456998035712 00000100:00100000:0.0:1776178154.339235:0:15213:0:(nrs_fifo.c:179:nrs_fifo_req_get()) NRS start fifo request from 12345-0@lo, seq: 660 00000100:00100000:0.0:1776178154.339239:0:15213:0:(service.c:2324:ptlrpc_server_handle_request()) Handling RPC req@000000005d95861f pname:cluuid+ref:pid:xid:nid:opc:job mdt_out00_003:lustre-MDT0001-mdtlov_UUID+6:3355:x1862456998035712:12345-0@lo:41:osp-pre-0-1.0 00000004:00000024:0.0:1776178154.339271:0:15213:0:(mdt_handler.c:548:mdt_statfs()) blocks cached 0 granted 2097152 pending 0 free 1446121472 avail 1316659200 00000100:00100000:0.0:1776178154.339314:0:15213:0:(service.c:2372:ptlrpc_server_handle_request()) Handled RPC req@000000005d95861f pname:cluuid+ref:pid:xid:nid:opc:job mdt_out00_003:lustre-MDT0001-mdtlov_UUID+6:3355:x1862456998035712:12345-0@lo:41:osp-pre-0-1.0 Request processed in 72us (1126us total) trans 0 rc 0/0 00000100:00100000:0.0:1776178154.339319:0:15213:0:(nrs_fifo.c:241:nrs_fifo_req_stop()) NRS stop fifo request from 12345-0@lo, seq: 660 00000100:00100000:0.0:1776178154.339348:0:3355:0:(client.c:2215:ptlrpc_check_set()) Completed RPC req@000000004a9818ac pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_02:lustre-MDT0001-mdtlov_UUID:3355:1862456998035712:0@lo:41:osp-pre-0-1.0 00000100:00100000:0.0:1776178154.339368:0:8227:0:(service.c:2152:ptlrpc_server_handle_req_in()) got req x1862456998035200 00000100:00100000:0.0:1776178154.339389:0:8227:0:(service.c:2152:ptlrpc_server_handle_req_in()) got req x1862456998035456 00000100:00100000:0.0:1776178154.339401:0:8227:0:(service.c:2152:ptlrpc_server_handle_req_in()) got req x1862456998035264 00000100:00100000:0.0:1776178154.339414:0:8227:0:(nrs_fifo.c:179:nrs_fifo_req_get()) NRS start fifo request from 12345-0@lo, seq: 461 00000100:00100000:0.0:1776178154.339418:0:8227:0:(service.c:2324:ptlrpc_server_handle_request()) Handling RPC req@000000002f729f84 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_000:lustre-MDT0001-mdtlov_UUID+5:3355:x1862456998035200:12345-0@lo:400:kworker/u8:2.0 00000100:00100000:0.0:1776178154.339449:0:8227:0:(service.c:2372:ptlrpc_server_handle_request()) Handled RPC req@000000002f729f84 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_000:lustre-MDT0001-mdtlov_UUID+5:3355:x1862456998035200:12345-0@lo:400:kworker/u8:2.0 Request processed in 30us (1335us total) trans 0 rc 0/0 00000100:00100000:0.0:1776178154.339455:0:8227:0:(nrs_fifo.c:241:nrs_fifo_req_stop()) NRS stop fifo request from 12345-0@lo, seq: 461 00000100:00100000:0.0:1776178154.339465:0:8227:0:(nrs_fifo.c:179:nrs_fifo_req_get()) NRS start fifo request from 12345-0@lo, seq: 462 00000100:00100000:0.0:1776178154.339469:0:8227:0:(service.c:2324:ptlrpc_server_handle_request()) Handling RPC req@000000005d3a2b34 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_000:lustre-MDT0000-mdtlov_UUID+8:3355:x1862456998035456:12345-0@lo:400:kworker/u8:2.0 00000100:00100000:0.0:1776178154.339494:0:8227:0:(service.c:2372:ptlrpc_server_handle_request()) Handled RPC req@000000005d3a2b34 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_000:lustre-MDT0000-mdtlov_UUID+8:3355:x1862456998035456:12345-0@lo:400:kworker/u8:2.0 Request processed in 24us (1341us total) trans 0 rc 0/0 00000100:00100000:0.0:1776178154.339500:0:8227:0:(nrs_fifo.c:241:nrs_fifo_req_stop()) NRS stop fifo request from 12345-0@lo, seq: 462 00000100:00100000:0.0:1776178154.339510:0:8227:0:(nrs_fifo.c:179:nrs_fifo_req_get()) NRS start fifo request from 12345-0@lo, seq: 463 00000100:00100000:0.0:1776178154.339513:0:8227:0:(service.c:2324:ptlrpc_server_handle_request()) Handling RPC req@00000000a62daf9a pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_000:lustre-MDT0001-mdtlov_UUID+5:3355:x1862456998035264:12345-0@lo:400:kworker/u8:2.0 00000100:00100000:0.0:1776178154.339535:0:8227:0:(service.c:2372:ptlrpc_server_handle_request()) Handled RPC req@00000000a62daf9a pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_000:lustre-MDT0001-mdtlov_UUID+5:3355:x1862456998035264:12345-0@lo:400:kworker/u8:2.0 Request processed in 22us (1262us total) trans 0 rc 0/0 00000100:00100000:0.0:1776178154.339541:0:8227:0:(nrs_fifo.c:241:nrs_fifo_req_stop()) NRS stop fifo request from 12345-0@lo, seq: 463 00000100:00100000:0.0:1776178154.339561:0:3355:0:(client.c:2215:ptlrpc_check_set()) Completed RPC req@0000000099e8e662 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_02:lustre-MDT0001-mdtlov_UUID:3355:1862456998035264:0@lo:400:kworker/u8:2.0 00000100:00100000:1.0:1776178154.340765:0:3354:0:(client.c:2215:ptlrpc_check_set()) Completed RPC req@000000005fdea934 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_01:lustre-MDT0000-lwp-OST0001_UUID:3354:1862456998035072:0@lo:400:kworker/u8:2.0 00000100:00100000:1.0:1776178154.340789:0:3354:0:(client.c:2215:ptlrpc_check_set()) Completed RPC req@000000006b9d8ccb pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_01:lustre-MDT0000-lwp-MDT0001_UUID:3354:1862456998035328:0@lo:400:kworker/u8:2.0 00000100:00100000:1.0:1776178154.340802:0:3354:0:(client.c:2215:ptlrpc_check_set()) Completed RPC req@000000005b125149 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_01:lustre-MDT0001-lwp-OST0001_UUID:3354:1862456998035136:0@lo:400:kworker/u8:2.0 00000100:00100000:0.0:1776178154.340947:0:3355:0:(client.c:2215:ptlrpc_check_set()) Completed RPC req@000000005accf721 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_02:lustre-MDT0001-mdtlov_UUID:3355:1862456998035200:0@lo:400:kworker/u8:2.0 00000100:00100000:0.0:1776178154.340960:0:3355:0:(client.c:2215:ptlrpc_check_set()) Completed RPC req@000000005f3eedd2 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_02:lustre-MDT0000-mdtlov_UUID:3355:1862456998035456:0@lo:400:kworker/u8:2.0 00000100:00100000:3.0:1776178154.341642:0:20137:0:(service.c:2152:ptlrpc_server_handle_req_in()) got req x1862456998034880 00000100:00100000:3.0:1776178154.341714:0:20137:0:(nrs_fifo.c:179:nrs_fifo_req_get()) NRS start fifo request from 12345-0@lo, seq: 136 00000100:00100000:3.0:1776178154.341719:0:20137:0:(service.c:2324:ptlrpc_server_handle_request()) Handling RPC req@00000000f1dccdd0 pname:cluuid+ref:pid:xid:nid:opc:job ll_mgs_0000:ac61f839-0e8e-4cac-9c59-08bcfbbad330+23:3354:x1862456998034880:12345-0@lo:400:kworker/u8:2.0 00000100:00100000:3.0:1776178154.341821:0:20137:0:(service.c:2372:ptlrpc_server_handle_request()) Handled RPC req@00000000f1dccdd0 pname:cluuid+ref:pid:xid:nid:opc:job ll_mgs_0000:ac61f839-0e8e-4cac-9c59-08bcfbbad330+23:3354:x1862456998034880:12345-0@lo:400:kworker/u8:2.0 Request processed in 101us (4952us total) trans 0 rc 0/0 00000100:00100000:3.0:1776178154.341829:0:20137:0:(nrs_fifo.c:241:nrs_fifo_req_stop()) NRS stop fifo request from 12345-0@lo, seq: 136 00000100:00100000:2.0:1776178154.342013:0:3356:0:(client.c:1733:ptlrpc_send_new_req()) Sending RPC req@0000000045560455 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_03:lustre-MDT0000-mdtlov_UUID:3356:1862456998035776:0@lo:41:osp-pre-1-0.0 00000100:00100000:2.0:1776178154.342082:0:3356:0:(events.c:361:request_in_callback()) peer: 12345-0@lo (source: 12345-0@lo) 00000100:00100000:2.0:1776178154.342572:0:14128:0:(service.c:2152:ptlrpc_server_handle_req_in()) got req x1862456998035776 00000100:00100000:2.0:1776178154.342615:0:15213:0:(nrs_fifo.c:179:nrs_fifo_req_get()) NRS start fifo request from 12345-0@lo, seq: 661 00000100:00100000:2.0:1776178154.342620:0:15213:0:(service.c:2324:ptlrpc_server_handle_request()) Handling RPC req@000000004a3d9287 pname:cluuid+ref:pid:xid:nid:opc:job mdt_out00_003:lustre-MDT0000-mdtlov_UUID+5:3356:x1862456998035776:12345-0@lo:41:osp-pre-1-0.0 00000004:00000024:2.0:1776178154.342641:0:15213:0:(mdt_handler.c:548:mdt_statfs()) blocks cached 0 granted 2097152 pending 0 free 1446039552 avail 1316577280 00000100:00100000:2.0:1776178154.342690:0:15213:0:(service.c:2372:ptlrpc_server_handle_request()) Handled RPC req@000000004a3d9287 pname:cluuid+ref:pid:xid:nid:opc:job mdt_out00_003:lustre-MDT0000-mdtlov_UUID+5:3356:x1862456998035776:12345-0@lo:41:osp-pre-1-0.0 Request processed in 69us (609us total) trans 0 rc 0/0 00000100:00100000:2.0:1776178154.342696:0:15213:0:(nrs_fifo.c:241:nrs_fifo_req_stop()) NRS stop fifo request from 12345-0@lo, seq: 661 00000100:00100000:2.0:1776178154.342737:0:3356:0:(client.c:2215:ptlrpc_check_set()) Completed RPC req@0000000045560455 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_03:lustre-MDT0000-mdtlov_UUID:3356:1862456998035776:0@lo:41:osp-pre-1-0.0 00000100:00100000:0.0:1776178154.344116:0:3354:0:(client.c:2215:ptlrpc_check_set()) Completed RPC req@00000000b821049d pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_01:ac61f839-0e8e-4cac-9c59-08bcfbbad330:3354:1862456998034880:0@lo:400:kworker/u8:2.0 00000100:00100000:0.0:1776178154.356206:0:3347:0:(events.c:361:request_in_callback()) peer: 12345-192.168.204.3@tcp (source: 12345-192.168.204.3@tcp) 00000100:00100000:1.0:1776178154.356833:0:6258:0:(service.c:2152:ptlrpc_server_handle_req_in()) got req x1862456979091968 00000100:00100000:1.0:1776178154.356898:0:6258:0:(nrs_fifo.c:179:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.204.3@tcp, seq: 2138 00000100:00100000:1.0:1776178154.356905:0:6258:0:(service.c:2324:ptlrpc_server_handle_request()) Handling RPC req@000000004a9818ac pname:cluuid+ref:pid:xid:nid:opc:job mdt_rdpg00_000:b50963bd-8eba-4432-9a54-8888d00e3fa1+33:17624:x1862456979091968:12345-192.168.204.3@tcp:35:touch.0 00000100:00100000:1.0:1776178154.357379:0:6258:0:(service.c:2372:ptlrpc_server_handle_request()) Handled RPC req@000000004a9818ac pname:cluuid+ref:pid:xid:nid:opc:job mdt_rdpg00_000:b50963bd-8eba-4432-9a54-8888d00e3fa1+34:17624:x1862456979091968:12345-192.168.204.3@tcp:35:touch.0 Request processed in 472us (1184us total) trans 17179876247 rc 0/0 00000100:00100000:1.0:1776178154.357389:0:6258:0:(nrs_fifo.c:241:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.204.3@tcp, seq: 2138 00000100:00100000:3.0:1776178154.386773:0:3346:0:(events.c:361:request_in_callback()) peer: 12345-192.168.204.3@tcp (source: 12345-192.168.204.3@tcp) 00000100:00100000:2.0:1776178154.387415:0:6255:0:(service.c:2152:ptlrpc_server_handle_req_in()) got req x1862456979092096 00000100:00100000:2.0:1776178154.387498:0:6255:0:(nrs_fifo.c:179:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.204.3@tcp, seq: 6539 00000100:00100000:2.0:1776178154.387506:0:6255:0:(service.c:2324:ptlrpc_server_handle_request()) Handling RPC req@000000009497b17e pname:cluuid+ref:pid:xid:nid:opc:job mdt00_000:b50963bd-8eba-4432-9a54-8888d00e3fa1+34:17626:x1862456979092096:12345-192.168.204.3@tcp:101:touch.0 00010000:00010000:2.0:1776178154.387543:0:6255:0:(ldlm_lockd.c:1248:ldlm_handle_enqueue0()) ### server-side enqueue handler START 00010000:00010000:2.0:1776178154.387569:0:6255:0:(ldlm_lockd.c:1318:ldlm_handle_enqueue0()) ### server-side enqueue handler, new lock created ns: mdt-lustre-MDT0001_UUID lock: 000000002f1f7bd5/0xa40083ab9bf53429 lrc: 2/0,0 mode: --/CW res: [0x240000402:0x7f7:0x0].0x0 bits 0x0/0x0 rrc: 3 type: IBT gid 0 flags: 0x40000000000000 nid: local remote: 0x771f68aa1eb27abd expref: -99 pid: 6255 timeout: 0 lvb_type: 0 00010000:00010000:2.0:1776178154.387737:0:6255:0:(ldlm_lock.c:766:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(CW) ns: mdt-lustre-MDT0001_UUID lock: 000000007da3b625/0xa40083ab9bf53430 lrc: 3/0,1 mode: --/CW res: [0x240000402:0x7f7:0x0].0x0 bits 0x0/0x0 rrc: 4 type: IBT gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 6255 timeout: 0 lvb_type: 0 00010000:00010000:2.0:1776178154.387763:0:6255:0:(ldlm_lock.c:1064:ldlm_granted_list_add_lock()) ### About to add lock: ns: mdt-lustre-MDT0001_UUID lock: 000000007da3b625/0xa40083ab9bf53430 lrc: 3/0,1 mode: CW/CW res: [0x240000402:0x7f7:0x0].0x0 bits 0x2/0x0 rrc: 4 type: IBT gid 0 flags: 0x50210000000000 nid: local remote: 0x0 expref: -99 pid: 6255 timeout: 0 lvb_type: 0 00010000:00010000:2.0:1776178154.387781:0:6255:0:(ldlm_request.c:528:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: mdt-lustre-MDT0001_UUID lock: 000000007da3b625/0xa40083ab9bf53430 lrc: 3/0,1 mode: CW/CW res: [0x240000402:0x7f7:0x0].0x0 bits 0x2/0x0 rrc: 4 type: IBT gid 0 flags: 0x40210000000000 nid: local remote: 0x0 expref: -99 pid: 6255 timeout: 0 lvb_type: 0 00010000:00010000:2.0:1776178154.387795:0:6255:0:(ldlm_lock.c:766:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: mdt-lustre-MDT0001_UUID lock: 000000007ee917b5/0xa40083ab9bf53437 lrc: 3/0,1 mode: --/PW res: [0x240000402:0x7f7:0x0].0xef09e6ea bits 0x0/0x0 rrc: 2 type: IBT gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 6255 timeout: 0 lvb_type: 0 00010000:00010000:2.0:1776178154.387802:0:6255:0:(ldlm_lock.c:1064:ldlm_granted_list_add_lock()) ### About to add lock: ns: mdt-lustre-MDT0001_UUID lock: 000000007ee917b5/0xa40083ab9bf53437 lrc: 3/0,1 mode: PW/PW res: [0x240000402:0x7f7:0x0].0xef09e6ea bits 0x2/0x0 rrc: 2 type: IBT gid 0 flags: 0x50210000000000 nid: local remote: 0x0 expref: -99 pid: 6255 timeout: 0 lvb_type: 0 00010000:00010000:2.0:1776178154.387807:0:6255:0:(ldlm_request.c:528:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: mdt-lustre-MDT0001_UUID lock: 000000007ee917b5/0xa40083ab9bf53437 lrc: 3/0,1 mode: PW/PW res: [0x240000402:0x7f7:0x0].0xef09e6ea bits 0x2/0x0 rrc: 2 type: IBT gid 0 flags: 0x40210000000000 nid: local remote: 0x0 expref: -99 pid: 6255 timeout: 0 lvb_type: 0 00000004:00080000:2.0:1776178154.388243:0:6255:0:(osp_object.c:1645:osp_create()) lustre-OST0001-osc-MDT0001: Wrote last used FID: [0x2c0000400:0x430:0x0], index 1: 0 00010000:00010000:2.0:1776178154.393037:0:6255:0:(ldlm_lock.c:766:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(CR) ns: mdt-lustre-MDT0001_UUID lock: 0000000004511956/0xa40083ab9bf5343e lrc: 3/1,0 mode: --/CR res: [0x240000402:0x7fc:0x0].0x0 bits 0x0/0x0 rrc: 2 type: IBT gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 6255 timeout: 0 lvb_type: 0 00010000:00010000:2.0:1776178154.393086:0:6255:0:(ldlm_lock.c:1064:ldlm_granted_list_add_lock()) ### About to add lock: ns: mdt-lustre-MDT0001_UUID lock: 0000000004511956/0xa40083ab9bf5343e lrc: 3/1,0 mode: CR/CR res: [0x240000402:0x7fc:0x0].0x0 bits 0x9/0x0 rrc: 2 type: IBT gid 0 flags: 0x50210000000000 nid: local remote: 0x0 expref: -99 pid: 6255 timeout: 0 lvb_type: 0 00010000:00010000:2.0:1776178154.393103:0:6255:0:(ldlm_request.c:528:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: mdt-lustre-MDT0001_UUID lock: 0000000004511956/0xa40083ab9bf5343e lrc: 3/1,0 mode: CR/CR res: [0x240000402:0x7fc:0x0].0x0 bits 0x9/0x0 rrc: 2 type: IBT gid 0 flags: 0x40210000000000 nid: local remote: 0x0 expref: -99 pid: 6255 timeout: 0 lvb_type: 0 00010000:00010000:2.0:1776178154.393187:0:6255:0:(ldlm_lock.c:820:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(CW) ns: mdt-lustre-MDT0001_UUID lock: 000000007da3b625/0xa40083ab9bf53430 lrc: 3/0,1 mode: CW/CW res: [0x240000402:0x7f7:0x0].0x0 bits 0x2/0x0 rrc: 4 type: IBT gid 0 flags: 0x50210000000000 nid: local remote: 0x0 expref: -99 pid: 6255 timeout: 0 lvb_type: 0 00010000:00010000:2.0:1776178154.393195:0:6255:0:(ldlm_lock.c:879:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: mdt-lustre-MDT0001_UUID lock: 000000007da3b625/0xa40083ab9bf53430 lrc: 2/0,0 mode: CW/CW res: [0x240000402:0x7f7:0x0].0x0 bits 0x2/0x0 rrc: 4 type: IBT gid 0 flags: 0x50210400000000 nid: local remote: 0x0 expref: -99 pid: 6255 timeout: 0 lvb_type: 0 00010000:00010000:2.0:1776178154.393201:0:6255:0:(ldlm_lockd.c:1879:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: mdt-lustre-MDT0001_UUID lock: 000000007da3b625/0xa40083ab9bf53430 lrc: 3/0,0 mode: CW/CW res: [0x240000402:0x7f7:0x0].0x0 bits 0x2/0x0 rrc: 4 type: IBT gid 0 flags: 0x40210400000000 nid: local remote: 0x0 expref: -99 pid: 6255 timeout: 0 lvb_type: 0 00010000:00010000:2.0:1776178154.393206:0:6255:0:(ldlm_lockd.c:1893:ldlm_handle_bl_callback()) Lock 000000007da3b625 already unused, calling callback (0000000079e78348) 00010000:00010000:2.0:1776178154.393210:0:6255:0:(ldlm_request.c:364:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: mdt-lustre-MDT0001_UUID lock: 000000007da3b625/0xa40083ab9bf53430 lrc: 3/0,0 mode: CW/CW res: [0x240000402:0x7f7:0x0].0x0 bits 0x2/0x0 rrc: 4 type: IBT gid 0 flags: 0x40210400000000 nid: local remote: 0x0 expref: -99 pid: 6255 timeout: 0 lvb_type: 0 00010000:00010000:2.0:1776178154.393217:0:6255:0:(ldlm_request.c:1257:ldlm_cli_cancel_local()) ### server-side local cancel ns: mdt-lustre-MDT0001_UUID lock: 000000007da3b625/0xa40083ab9bf53430 lrc: 4/0,0 mode: CW/CW res: [0x240000402:0x7f7:0x0].0x0 bits 0x2/0x0 rrc: 4 type: IBT gid 0 flags: 0x40218400000000 nid: local remote: 0x0 expref: -99 pid: 6255 timeout: 0 lvb_type: 0 00010000:00010000:2.0:1776178154.393230:0:6255:0:(ldlm_inodebits.c:94:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x240000402:0x7f7:0x0].0x0 (000000006b762dc5) 00010000:00010000:2.0:1776178154.393232:0:6255:0:(ldlm_inodebits.c:96:ldlm_reprocess_inodebits_queue()) Hint 2 00010000:00010000:2.0:1776178154.393235:0:6255:0:(ldlm_lockd.c:1903:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: mdt-lustre-MDT0001_UUID lock: 000000007da3b625/0xa40083ab9bf53430 lrc: 2/0,0 mode: --/CW res: [0x240000402:0x7f7:0x0].0x0 bits 0x2/0x0 rrc: 4 type: IBT gid 0 flags: 0x44a19400000000 nid: local remote: 0x0 expref: -99 pid: 6255 timeout: 0 lvb_type: 0 00010000:00010000:2.0:1776178154.393240:0:6255:0:(ldlm_lock.c:216:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: mdt-lustre-MDT0001_UUID lock: 000000007da3b625/0xa40083ab9bf53430 lrc: 0/0,0 mode: --/CW res: [0x240000402:0x7f7:0x0].0x0 bits 0x2/0x0 rrc: 4 type: IBT gid 0 flags: 0x44a19400000000 nid: local remote: 0x0 expref: -99 pid: 6255 timeout: 0 lvb_type: 0 00000004:00010000:2.0:1776178154.393256:0:6255:0:(mdt_handler.c:3992:mdt_save_lock()) ### save lock request 000000009497b17e reply state 000000002f260d03 transno 17179876248 ns: mdt-lustre-MDT0001_UUID lock: 000000007ee917b5/0xa40083ab9bf53437 lrc: 3/0,1 mode: PW/PW res: [0x240000402:0x7f7:0x0].0xef09e6ea bits 0x2/0x0 rrc: 2 type: IBT gid 0 flags: 0x40210000000000 nid: local remote: 0x0 expref: -99 pid: 6255 timeout: 0 lvb_type: 0 00010000:00010000:2.0:1776178154.393311:0:6255:0:(ldlm_lock.c:216:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: mdt-lustre-MDT0001_UUID lock: 000000002f1f7bd5/0xa40083ab9bf53429 lrc: 0/0,0 mode: --/CW res: [0x240000402:0x7f7:0x0].0x0 bits 0x1/0x0 rrc: 3 type: IBT gid 0 flags: 0x44000000000000 nid: 192.168.204.3@tcp remote: 0x771f68aa1eb27abd expref: 37 pid: 6255 timeout: 0 lvb_type: 0 00010000:00010000:2.0:1776178154.393325:0:6255:0:(ldlm_lockd.c:1484:ldlm_handle_enqueue0()) ### server-side enqueue handler, sending reply (err=0, rc=0) ns: mdt-lustre-MDT0001_UUID lock: 0000000004511956/0xa40083ab9bf5343e lrc: 3/0,0 mode: CR/CR res: [0x240000402:0x7fc:0x0].0x0 bits 0x9/0x0 rrc: 2 type: IBT gid 0 flags: 0x40200000000000 nid: 192.168.204.3@tcp remote: 0x771f68aa1eb27abd expref: 36 pid: 6255 timeout: 0 lvb_type: 0 00010000:00010000:2.0:1776178154.393336:0:6255:0:(ldlm_inodebits.c:94:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x240000402:0x7fc:0x0].0x0 (0000000064abeb1c) 00010000:00010000:2.0:1776178154.393339:0:6255:0:(ldlm_inodebits.c:96:ldlm_reprocess_inodebits_queue()) Hint 9 00010000:00010000:2.0:1776178154.393341:0:6255:0:(ldlm_lockd.c:1565:ldlm_handle_enqueue0()) ### server-side enqueue handler END (lock 0000000004511956, rc 0) 00000100:00100000:2.0:1776178154.393445:0:6255:0:(service.c:2372:ptlrpc_server_handle_request()) Handled RPC req@000000009497b17e pname:cluuid+ref:pid:xid:nid:opc:job mdt00_000:b50963bd-8eba-4432-9a54-8888d00e3fa1+37:17626:x1862456979092096:12345-192.168.204.3@tcp:101:touch.0 Request processed in 5938us (6685us total) trans 17179876248 rc 0/0 00000100:00100000:2.0:1776178154.393455:0:6255:0:(nrs_fifo.c:241:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.204.3@tcp, seq: 6539 00010000:00010000:3.0:1776178154.402662:0:3339:0:(ldlm_lock.c:1064:ldlm_granted_list_add_lock()) ### About to add lock: ns: mdt-lustre-MDT0001_UUID lock: 000000007ee917b5/0xa40083ab9bf53437 lrc: 3/0,1 mode: COS/COS res: [0x240000402:0x7f7:0x0].0xef09e6ea bits 0x2/0x0 rrc: 2 type: IBT gid 0 flags: 0x50210000000000 nid: local remote: 0x0 expref: -99 pid: 6255 timeout: 0 lvb_type: 0 00010000:00010000:3.0:1776178154.402686:0:3339:0:(ldlm_inodebits.c:94:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x240000402:0x7f7:0x0].0xef09e6ea (000000003c4b13ad) 00010000:00010000:3.0:1776178154.402689:0:3339:0:(ldlm_inodebits.c:96:ldlm_reprocess_inodebits_queue()) Hint 2 00000100:00100000:3.0:1776178154.406165:0:3346:0:(events.c:361:request_in_callback()) peer: 12345-192.168.204.3@tcp (source: 12345-192.168.204.3@tcp) 00000100:00100000:2.0:1776178154.406333:0:6255:0:(service.c:2152:ptlrpc_server_handle_req_in()) got req x1862456979092224 00000100:00100000:2.0:1776178154.406384:0:6255:0:(nrs_fifo.c:179:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.204.3@tcp, seq: 6540 00000100:00100000:2.0:1776178154.406391:0:6255:0:(service.c:2324:ptlrpc_server_handle_request()) Handling RPC req@000000001d072e30 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_000:b50963bd-8eba-4432-9a54-8888d00e3fa1+37:17626:x1862456979092224:12345-192.168.204.3@tcp:36:touch.0 00010000:00010000:2.0:1776178154.406510:0:6255:0:(ldlm_lock.c:766:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: mdt-lustre-MDT0001_UUID lock: 000000002f1f7bd5/0xa40083ab9bf5344c lrc: 3/0,1 mode: --/PW res: [0x240000402:0x7fc:0x0].0x0 bits 0x0/0x0 rrc: 3 type: IBT gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 6255 timeout: 0 lvb_type: 0 00010000:00010000:2.0:1776178154.406528:0:6255:0:(ldlm_lock.c:1064:ldlm_granted_list_add_lock()) ### About to add lock: ns: mdt-lustre-MDT0001_UUID lock: 000000002f1f7bd5/0xa40083ab9bf5344c lrc: 3/0,1 mode: PW/PW res: [0x240000402:0x7fc:0x0].0x0 bits 0x2/0x0 rrc: 3 type: IBT gid 0 flags: 0x50210000000000 nid: local remote: 0x0 expref: -99 pid: 6255 timeout: 0 lvb_type: 0 00010000:00010000:2.0:1776178154.406540:0:6255:0:(ldlm_request.c:528:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: mdt-lustre-MDT0001_UUID lock: 000000002f1f7bd5/0xa40083ab9bf5344c lrc: 3/0,1 mode: PW/PW res: [0x240000402:0x7fc:0x0].0x0 bits 0x2/0x0 rrc: 3 type: IBT gid 0 flags: 0x40210000000000 nid: local remote: 0x0 expref: -99 pid: 6255 timeout: 0 lvb_type: 0 00000004:00010000:2.0:1776178154.406676:0:6255:0:(mdt_handler.c:3992:mdt_save_lock()) ### save lock request 000000001d072e30 reply state 00000000e7fde0f1 transno 17179876249 ns: mdt-lustre-MDT0001_UUID lock: 000000002f1f7bd5/0xa40083ab9bf5344c lrc: 3/0,1 mode: PW/PW res: [0x240000402:0x7fc:0x0].0x0 bits 0x2/0x0 rrc: 3 type: IBT gid 0 flags: 0x40210000000000 nid: local remote: 0x0 expref: -99 pid: 6255 timeout: 0 lvb_type: 0 00000100:00100000:2.0:1776178154.406806:0:6255:0:(service.c:2372:ptlrpc_server_handle_request()) Handled RPC req@000000001d072e30 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_000:b50963bd-8eba-4432-9a54-8888d00e3fa1+39:17626:x1862456979092224:12345-192.168.204.3@tcp:36:touch.0 Request processed in 415us (650us total) trans 17179876249 rc 0/0 00000100:00100000:2.0:1776178154.406814:0:6255:0:(nrs_fifo.c:241:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.204.3@tcp, seq: 6540 00010000:00010000:1.0:1776178154.414473:0:3340:0:(ldlm_lock.c:1064:ldlm_granted_list_add_lock()) ### About to add lock: ns: mdt-lustre-MDT0001_UUID lock: 000000002f1f7bd5/0xa40083ab9bf5344c lrc: 3/0,1 mode: COS/COS res: [0x240000402:0x7fc:0x0].0x0 bits 0x2/0x0 rrc: 3 type: IBT gid 0 flags: 0x50210000000000 nid: local remote: 0x0 expref: -99 pid: 6255 timeout: 0 lvb_type: 0 00010000:00010000:1.0:1776178154.414501:0:3340:0:(ldlm_inodebits.c:94:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x240000402:0x7fc:0x0].0x0 (0000000064abeb1c) 00010000:00010000:1.0:1776178154.414505:0:3340:0:(ldlm_inodebits.c:96:ldlm_reprocess_inodebits_queue()) Hint 2 00000100:00100000:0.0:1776178154.415899:0:3346:0:(events.c:361:request_in_callback()) peer: 12345-192.168.204.3@tcp (source: 12345-192.168.204.3@tcp) 00000100:00100000:2.0:1776178154.416331:0:8228:0:(service.c:2152:ptlrpc_server_handle_req_in()) got req x1862456979092288 00000100:00100000:2.0:1776178154.416446:0:8228:0:(nrs_fifo.c:179:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.204.3@tcp, seq: 5261 00000100:00100000:2.0:1776178154.416460:0:8228:0:(service.c:2324:ptlrpc_server_handle_request()) Handling RPC req@000000005f3eedd2 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_001:b50963bd-8eba-4432-9a54-8888d00e3fa1+7:2271:x1862456979092288:12345-192.168.204.3@tcp:2:touch.0 00000100:00100000:2.0:1776178154.416780:0:8228:0:(service.c:2372:ptlrpc_server_handle_request()) Handled RPC req@000000005f3eedd2 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_001:b50963bd-8eba-4432-9a54-8888d00e3fa1+8:2271:x1862456979092288:12345-192.168.204.3@tcp:2:touch.0 Request processed in 321us (891us total) trans 4294969326 rc 0/0 00000100:00100000:2.0:1776178154.416789:0:8228:0:(nrs_fifo.c:241:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.204.3@tcp, seq: 5261 00000100:00100000:0.0:1776178154.420856:0:3348:0:(events.c:361:request_in_callback()) peer: 12345-192.168.204.3@tcp (source: 12345-192.168.204.3@tcp) 00000100:00100000:2.0:1776178154.421748:0:6259:0:(service.c:2152:ptlrpc_server_handle_req_in()) got req x1862456979092416 00000100:00100000:2.0:1776178154.421822:0:6259:0:(nrs_fifo.c:179:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.204.3@tcp, seq: 2139 00000100:00100000:2.0:1776178154.421835:0:6259:0:(service.c:2324:ptlrpc_server_handle_request()) Handling RPC req@00000000a1dcd78e pname:cluuid+ref:pid:xid:nid:opc:job mdt_rdpg00_001:b50963bd-8eba-4432-9a54-8888d00e3fa1+39:17626:x1862456979092416:12345-192.168.204.3@tcp:35:touch.0 00000100:00100000:2.0:1776178154.422300:0:6259:0:(service.c:2372:ptlrpc_server_handle_request()) Handled RPC req@00000000a1dcd78e pname:cluuid+ref:pid:xid:nid:opc:job mdt_rdpg00_001:b50963bd-8eba-4432-9a54-8888d00e3fa1+40:17626:x1862456979092416:12345-192.168.204.3@tcp:35:touch.0 Request processed in 464us (1453us total) trans 17179876250 rc 0/0 00000100:00100000:2.0:1776178154.422313:0:6259:0:(nrs_fifo.c:241:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.204.3@tcp, seq: 2139 00000100:00100000:1.0:1776178154.438527:0:3347:0:(events.c:361:request_in_callback()) peer: 12345-192.168.204.3@tcp (source: 12345-192.168.204.3@tcp) 00000100:00100000:0.0:1776178154.438687:0:6256:0:(service.c:2152:ptlrpc_server_handle_req_in()) got req x1862456979092544 00000100:00100000:0.0:1776178154.438749:0:6256:0:(nrs_fifo.c:179:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.204.3@tcp, seq: 6541 00000100:00100000:0.0:1776178154.438756:0:6256:0:(service.c:2324:ptlrpc_server_handle_request()) Handling RPC req@0000000017d54e92 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_001:b50963bd-8eba-4432-9a54-8888d00e3fa1+40:17627:x1862456979092544:12345-192.168.204.3@tcp:101:touch.0 00010000:00010000:0.0:1776178154.438781:0:6256:0:(ldlm_lockd.c:1248:ldlm_handle_enqueue0()) ### server-side enqueue handler START 00010000:00010000:0.0:1776178154.438802:0:6256:0:(ldlm_lockd.c:1318:ldlm_handle_enqueue0()) ### server-side enqueue handler, new lock created ns: mdt-lustre-MDT0001_UUID lock: 0000000068850c85/0xa40083ab9bf53453 lrc: 2/0,0 mode: --/CW res: [0x240000402:0x7f7:0x0].0x0 bits 0x0/0x0 rrc: 3 type: IBT gid 0 flags: 0x40000000000000 nid: local remote: 0x771f68aa1eb27ac4 expref: -99 pid: 6256 timeout: 0 lvb_type: 0 00010000:00010000:0.0:1776178154.439176:0:6256:0:(ldlm_lock.c:766:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(CW) ns: mdt-lustre-MDT0001_UUID lock: 00000000f51fd0d7/0xa40083ab9bf5345a lrc: 3/0,1 mode: --/CW res: [0x240000402:0x7f7:0x0].0x0 bits 0x0/0x0 rrc: 4 type: IBT gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 6256 timeout: 0 lvb_type: 0 00010000:00010000:0.0:1776178154.439193:0:6256:0:(ldlm_lock.c:1064:ldlm_granted_list_add_lock()) ### About to add lock: ns: mdt-lustre-MDT0001_UUID lock: 00000000f51fd0d7/0xa40083ab9bf5345a lrc: 3/0,1 mode: CW/CW res: [0x240000402:0x7f7:0x0].0x0 bits 0x2/0x0 rrc: 4 type: IBT gid 0 flags: 0x50210000000000 nid: local remote: 0x0 expref: -99 pid: 6256 timeout: 0 lvb_type: 0 00010000:00010000:0.0:1776178154.439206:0:6256:0:(ldlm_request.c:528:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: mdt-lustre-MDT0001_UUID lock: 00000000f51fd0d7/0xa40083ab9bf5345a lrc: 3/0,1 mode: CW/CW res: [0x240000402:0x7f7:0x0].0x0 bits 0x2/0x0 rrc: 4 type: IBT gid 0 flags: 0x40210000000000 nid: local remote: 0x0 expref: -99 pid: 6256 timeout: 0 lvb_type: 0 00010000:00010000:0.0:1776178154.439223:0:6256:0:(ldlm_lock.c:766:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: mdt-lustre-MDT0001_UUID lock: 0000000030bf6b1e/0xa40083ab9bf53461 lrc: 3/0,1 mode: --/PW res: [0x240000402:0x7f7:0x0].0x37639fea bits 0x0/0x0 rrc: 2 type: IBT gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 6256 timeout: 0 lvb_type: 0 00010000:00010000:0.0:1776178154.439233:0:6256:0:(ldlm_lock.c:1064:ldlm_granted_list_add_lock()) ### About to add lock: ns: mdt-lustre-MDT0001_UUID lock: 0000000030bf6b1e/0xa40083ab9bf53461 lrc: 3/0,1 mode: PW/PW res: [0x240000402:0x7f7:0x0].0x37639fea bits 0x2/0x0 rrc: 2 type: IBT gid 0 flags: 0x50210000000000 nid: local remote: 0x0 expref: -99 pid: 6256 timeout: 0 lvb_type: 0 00010000:00010000:0.0:1776178154.439241:0:6256:0:(ldlm_request.c:528:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: mdt-lustre-MDT0001_UUID lock: 0000000030bf6b1e/0xa40083ab9bf53461 lrc: 3/0,1 mode: PW/PW res: [0x240000402:0x7f7:0x0].0x37639fea bits 0x2/0x0 rrc: 2 type: IBT gid 0 flags: 0x40210000000000 nid: local remote: 0x0 expref: -99 pid: 6256 timeout: 0 lvb_type: 0 00000004:00080000:0.0:1776178154.439808:0:6256:0:(osp_object.c:1645:osp_create()) lustre-OST0000-osc-MDT0001: Wrote last used FID: [0x280000400:0x470:0x0], index 0: 0 00010000:00010000:0.0:1776178154.440567:0:6256:0:(ldlm_lock.c:766:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(CR) ns: mdt-lustre-MDT0001_UUID lock: 0000000095e53696/0xa40083ab9bf53468 lrc: 3/1,0 mode: --/CR res: [0x240000402:0x7fd:0x0].0x0 bits 0x0/0x0 rrc: 2 type: IBT gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 6256 timeout: 0 lvb_type: 0 00010000:00010000:0.0:1776178154.440583:0:6256:0:(ldlm_lock.c:1064:ldlm_granted_list_add_lock()) ### About to add lock: ns: mdt-lustre-MDT0001_UUID lock: 0000000095e53696/0xa40083ab9bf53468 lrc: 3/1,0 mode: CR/CR res: [0x240000402:0x7fd:0x0].0x0 bits 0x9/0x0 rrc: 2 type: IBT gid 0 flags: 0x50210000000000 nid: local remote: 0x0 expref: -99 pid: 6256 timeout: 0 lvb_type: 0 00010000:00010000:0.0:1776178154.440654:0:6256:0:(ldlm_request.c:528:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: mdt-lustre-MDT0001_UUID lock: 0000000095e53696/0xa40083ab9bf53468 lrc: 3/1,0 mode: CR/CR res: [0x240000402:0x7fd:0x0].0x0 bits 0x9/0x0 rrc: 2 type: IBT gid 0 flags: 0x40210000000000 nid: local remote: 0x0 expref: -99 pid: 6256 timeout: 0 lvb_type: 0 00010000:00010000:0.0:1776178154.440769:0:6256:0:(ldlm_lock.c:820:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(CW) ns: mdt-lustre-MDT0001_UUID lock: 00000000f51fd0d7/0xa40083ab9bf5345a lrc: 3/0,1 mode: CW/CW res: [0x240000402:0x7f7:0x0].0x0 bits 0x2/0x0 rrc: 4 type: IBT gid 0 flags: 0x50210000000000 nid: local remote: 0x0 expref: -99 pid: 6256 timeout: 0 lvb_type: 0 00010000:00010000:0.0:1776178154.440781:0:6256:0:(ldlm_lock.c:879:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: mdt-lustre-MDT0001_UUID lock: 00000000f51fd0d7/0xa40083ab9bf5345a lrc: 2/0,0 mode: CW/CW res: [0x240000402:0x7f7:0x0].0x0 bits 0x2/0x0 rrc: 4 type: IBT gid 0 flags: 0x50210400000000 nid: local remote: 0x0 expref: -99 pid: 6256 timeout: 0 lvb_type: 0 00010000:00010000:0.0:1776178154.440788:0:6256:0:(ldlm_lockd.c:1879:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: mdt-lustre-MDT0001_UUID lock: 00000000f51fd0d7/0xa40083ab9bf5345a lrc: 3/0,0 mode: CW/CW res: [0x240000402:0x7f7:0x0].0x0 bits 0x2/0x0 rrc: 4 type: IBT gid 0 flags: 0x40210400000000 nid: local remote: 0x0 expref: -99 pid: 6256 timeout: 0 lvb_type: 0 00010000:00010000:0.0:1776178154.440793:0:6256:0:(ldlm_lockd.c:1893:ldlm_handle_bl_callback()) Lock 00000000f51fd0d7 already unused, calling callback (0000000079e78348) 00010000:00010000:0.0:1776178154.440797:0:6256:0:(ldlm_request.c:364:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: mdt-lustre-MDT0001_UUID lock: 00000000f51fd0d7/0xa40083ab9bf5345a lrc: 3/0,0 mode: CW/CW res: [0x240000402:0x7f7:0x0].0x0 bits 0x2/0x0 rrc: 4 type: IBT gid 0 flags: 0x40210400000000 nid: local remote: 0x0 expref: -99 pid: 6256 timeout: 0 lvb_type: 0 00010000:00010000:0.0:1776178154.440810:0:6256:0:(ldlm_request.c:1257:ldlm_cli_cancel_local()) ### server-side local cancel ns: mdt-lustre-MDT0001_UUID lock: 00000000f51fd0d7/0xa40083ab9bf5345a lrc: 4/0,0 mode: CW/CW res: [0x240000402:0x7f7:0x0].0x0 bits 0x2/0x0 rrc: 4 type: IBT gid 0 flags: 0x40218400000000 nid: local remote: 0x0 expref: -99 pid: 6256 timeout: 0 lvb_type: 0 00010000:00010000:0.0:1776178154.440824:0:6256:0:(ldlm_inodebits.c:94:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x240000402:0x7f7:0x0].0x0 (000000006b762dc5) 00010000:00010000:0.0:1776178154.440829:0:6256:0:(ldlm_inodebits.c:96:ldlm_reprocess_inodebits_queue()) Hint 2 00010000:00010000:0.0:1776178154.440835:0:6256:0:(ldlm_lockd.c:1903:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: mdt-lustre-MDT0001_UUID lock: 00000000f51fd0d7/0xa40083ab9bf5345a lrc: 2/0,0 mode: --/CW res: [0x240000402:0x7f7:0x0].0x0 bits 0x2/0x0 rrc: 4 type: IBT gid 0 flags: 0x44a19400000000 nid: local remote: 0x0 expref: -99 pid: 6256 timeout: 0 lvb_type: 0 00010000:00010000:0.0:1776178154.440844:0:6256:0:(ldlm_lock.c:216:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: mdt-lustre-MDT0001_UUID lock: 00000000f51fd0d7/0xa40083ab9bf5345a lrc: 0/0,0 mode: --/CW res: [0x240000402:0x7f7:0x0].0x0 bits 0x2/0x0 rrc: 4 type: IBT gid 0 flags: 0x44a19400000000 nid: local remote: 0x0 expref: -99 pid: 6256 timeout: 0 lvb_type: 0 00000004:00010000:0.0:1776178154.440865:0:6256:0:(mdt_handler.c:3992:mdt_save_lock()) ### save lock request 0000000017d54e92 reply state 00000000ecc3fff8 transno 17179876251 ns: mdt-lustre-MDT0001_UUID lock: 0000000030bf6b1e/0xa40083ab9bf53461 lrc: 3/0,1 mode: PW/PW res: [0x240000402:0x7f7:0x0].0x37639fea bits 0x2/0x0 rrc: 2 type: IBT gid 0 flags: 0x40210000000000 nid: local remote: 0x0 expref: -99 pid: 6256 timeout: 0 lvb_type: 0 00010000:00010000:0.0:1776178154.444853:0:6256:0:(ldlm_lock.c:216:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: mdt-lustre-MDT0001_UUID lock: 0000000068850c85/0xa40083ab9bf53453 lrc: 0/0,0 mode: --/CW res: [0x240000402:0x7f7:0x0].0x0 bits 0x1/0x0 rrc: 3 type: IBT gid 0 flags: 0x44000000000000 nid: 192.168.204.3@tcp remote: 0x771f68aa1eb27ac4 expref: 43 pid: 6256 timeout: 0 lvb_type: 0 00010000:00010000:0.0:1776178154.444893:0:6256:0:(ldlm_lockd.c:1484:ldlm_handle_enqueue0()) ### server-side enqueue handler, sending reply (err=0, rc=0) ns: mdt-lustre-MDT0001_UUID lock: 0000000095e53696/0xa40083ab9bf53468 lrc: 3/0,0 mode: CR/CR res: [0x240000402:0x7fd:0x0].0x0 bits 0x9/0x0 rrc: 2 type: IBT gid 0 flags: 0x40200000000000 nid: 192.168.204.3@tcp remote: 0x771f68aa1eb27ac4 expref: 42 pid: 6256 timeout: 0 lvb_type: 0 00010000:00010000:0.0:1776178154.444904:0:6256:0:(ldlm_inodebits.c:94:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x240000402:0x7fd:0x0].0x0 (000000008d47e759) 00010000:00010000:0.0:1776178154.444906:0:6256:0:(ldlm_inodebits.c:96:ldlm_reprocess_inodebits_queue()) Hint 9 00010000:00010000:0.0:1776178154.444909:0:6256:0:(ldlm_lockd.c:1565:ldlm_handle_enqueue0()) ### server-side enqueue handler END (lock 0000000095e53696, rc 0) 00000100:00100000:0.0:1776178154.445009:0:6256:0:(service.c:2372:ptlrpc_server_handle_request()) Handled RPC req@0000000017d54e92 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_001:b50963bd-8eba-4432-9a54-8888d00e3fa1+43:17627:x1862456979092544:12345-192.168.204.3@tcp:101:touch.0 Request processed in 6250us (6489us total) trans 17179876251 rc 0/0 00000100:00100000:0.0:1776178154.445018:0:6256:0:(nrs_fifo.c:241:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.204.3@tcp, seq: 6541 00010000:00010000:3.0:1776178154.447611:0:3339:0:(ldlm_lock.c:1064:ldlm_granted_list_add_lock()) ### About to add lock: ns: mdt-lustre-MDT0001_UUID lock: 0000000030bf6b1e/0xa40083ab9bf53461 lrc: 3/0,1 mode: COS/COS res: [0x240000402:0x7f7:0x0].0x37639fea bits 0x2/0x0 rrc: 2 type: IBT gid 0 flags: 0x50210000000000 nid: local remote: 0x0 expref: -99 pid: 6256 timeout: 0 lvb_type: 0 00010000:00010000:3.0:1776178154.447636:0:3339:0:(ldlm_inodebits.c:94:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x240000402:0x7f7:0x0].0x37639fea (000000000a3c1046) 00010000:00010000:3.0:1776178154.447640:0:3339:0:(ldlm_inodebits.c:96:ldlm_reprocess_inodebits_queue()) Hint 2 00000100:00100000:1.0:1776178154.450660:0:3347:0:(events.c:361:request_in_callback()) peer: 12345-192.168.204.3@tcp (source: 12345-192.168.204.3@tcp) 00000100:00100000:0.0:1776178154.452582:0:6256:0:(service.c:2152:ptlrpc_server_handle_req_in()) got req x1862456979092672 00000100:00100000:0.0:1776178154.452708:0:6256:0:(nrs_fifo.c:179:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.204.3@tcp, seq: 6542 00000100:00100000:0.0:1776178154.452723:0:6256:0:(service.c:2324:ptlrpc_server_handle_request()) Handling RPC req@000000007b15d0e4 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_001:b50963bd-8eba-4432-9a54-8888d00e3fa1+43:17627:x1862456979092672:12345-192.168.204.3@tcp:36:touch.0 00010000:00010000:0.0:1776178154.452894:0:6256:0:(ldlm_lock.c:766:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: mdt-lustre-MDT0001_UUID lock: 0000000068850c85/0xa40083ab9bf53476 lrc: 3/0,1 mode: --/PW res: [0x240000402:0x7fd:0x0].0x0 bits 0x0/0x0 rrc: 3 type: IBT gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 6256 timeout: 0 lvb_type: 0 00010000:00010000:0.0:1776178154.452923:0:6256:0:(ldlm_lock.c:1064:ldlm_granted_list_add_lock()) ### About to add lock: ns: mdt-lustre-MDT0001_UUID lock: 0000000068850c85/0xa40083ab9bf53476 lrc: 3/0,1 mode: PW/PW res: [0x240000402:0x7fd:0x0].0x0 bits 0x2/0x0 rrc: 3 type: IBT gid 0 flags: 0x50210000000000 nid: local remote: 0x0 expref: -99 pid: 6256 timeout: 0 lvb_type: 0 00010000:00010000:0.0:1776178154.452963:0:6256:0:(ldlm_request.c:528:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: mdt-lustre-MDT0001_UUID lock: 0000000068850c85/0xa40083ab9bf53476 lrc: 3/0,1 mode: PW/PW res: [0x240000402:0x7fd:0x0].0x0 bits 0x2/0x0 rrc: 3 type: IBT gid 0 flags: 0x40210000000000 nid: local remote: 0x0 expref: -99 pid: 6256 timeout: 0 lvb_type: 0 00000004:00010000:0.0:1776178154.453609:0:6256:0:(mdt_handler.c:3992:mdt_save_lock()) ### save lock request 000000007b15d0e4 reply state 00000000ca2087c2 transno 17179876252 ns: mdt-lustre-MDT0001_UUID lock: 0000000068850c85/0xa40083ab9bf53476 lrc: 3/0,1 mode: PW/PW res: [0x240000402:0x7fd:0x0].0x0 bits 0x2/0x0 rrc: 3 type: IBT gid 0 flags: 0x40210000000000 nid: local remote: 0x0 expref: -99 pid: 6256 timeout: 0 lvb_type: 0 00000100:00100000:0.0:1776178154.456724:0:6256:0:(service.c:2372:ptlrpc_server_handle_request()) Handled RPC req@000000007b15d0e4 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_001:b50963bd-8eba-4432-9a54-8888d00e3fa1+45:17627:x1862456979092672:12345-192.168.204.3@tcp:36:touch.0 Request processed in 3980us (6055us total) trans 17179876252 rc 0/0 00000100:00100000:0.0:1776178154.456756:0:6256:0:(nrs_fifo.c:241:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.204.3@tcp, seq: 6542 00000100:00100000:0.0:1776178154.465026:0:3346:0:(events.c:361:request_in_callback()) peer: 12345-192.168.204.3@tcp (source: 12345-192.168.204.3@tcp) 00010000:00010000:1.0:1776178154.465110:0:3340:0:(ldlm_lock.c:1064:ldlm_granted_list_add_lock()) ### About to add lock: ns: mdt-lustre-MDT0001_UUID lock: 0000000068850c85/0xa40083ab9bf53476 lrc: 3/0,1 mode: COS/COS res: [0x240000402:0x7fd:0x0].0x0 bits 0x2/0x0 rrc: 3 type: IBT gid 0 flags: 0x50210000000000 nid: local remote: 0x0 expref: -99 pid: 6256 timeout: 0 lvb_type: 0 00010000:00010000:1.0:1776178154.465129:0:3340:0:(ldlm_inodebits.c:94:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x240000402:0x7fd:0x0].0x0 (000000008d47e759) 00010000:00010000:1.0:1776178154.465132:0:3340:0:(ldlm_inodebits.c:96:ldlm_reprocess_inodebits_queue()) Hint 2 00000100:00100000:2.0:1776178154.465297:0:20762:0:(service.c:2152:ptlrpc_server_handle_req_in()) got req x1862456979092736 00000100:00100000:2.0:1776178154.465354:0:20762:0:(nrs_fifo.c:179:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.204.3@tcp, seq: 5262 00000100:00100000:2.0:1776178154.465361:0:20762:0:(service.c:2324:ptlrpc_server_handle_request()) Handling RPC req@0000000099e8e662 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_013:b50963bd-8eba-4432-9a54-8888d00e3fa1+7:2270:x1862456979092736:12345-192.168.204.3@tcp:2:touch.0 00000100:00100000:2.0:1776178154.465583:0:20762:0:(service.c:2372:ptlrpc_server_handle_request()) Handled RPC req@0000000099e8e662 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_013:b50963bd-8eba-4432-9a54-8888d00e3fa1+8:2270:x1862456979092736:12345-192.168.204.3@tcp:2:touch.0 Request processed in 222us (567us total) trans 4294969355 rc 0/0 00000100:00100000:2.0:1776178154.465602:0:20762:0:(nrs_fifo.c:241:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.204.3@tcp, seq: 5262 00000100:00100000:3.0:1776178154.472456:0:3348:0:(events.c:361:request_in_callback()) peer: 12345-192.168.204.3@tcp (source: 12345-192.168.204.3@tcp) 00000100:00100000:1.0:1776178154.472796:0:6258:0:(service.c:2152:ptlrpc_server_handle_req_in()) got req x1862456979092864 00000100:00100000:1.0:1776178154.472994:0:6258:0:(nrs_fifo.c:179:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.204.3@tcp, seq: 2140 00000100:00100000:1.0:1776178154.473004:0:6258:0:(service.c:2324:ptlrpc_server_handle_request()) Handling RPC req@000000006c57be21 pname:cluuid+ref:pid:xid:nid:opc:job mdt_rdpg00_000:b50963bd-8eba-4432-9a54-8888d00e3fa1+45:17627:x1862456979092864:12345-192.168.204.3@tcp:35:touch.0 00000100:00100000:1.0:1776178154.473857:0:6258:0:(service.c:2372:ptlrpc_server_handle_request()) Handled RPC req@000000006c57be21 pname:cluuid+ref:pid:xid:nid:opc:job mdt_rdpg00_000:b50963bd-8eba-4432-9a54-8888d00e3fa1+46:17627:x1862456979092864:12345-192.168.204.3@tcp:35:touch.0 Request processed in 850us (1410us total) trans 17179876253 rc 0/0 00000100:00100000:1.0:1776178154.473869:0:6258:0:(nrs_fifo.c:241:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.204.3@tcp, seq: 2140 00000100:00100000:1.0:1776178154.495906:0:3347:0:(events.c:361:request_in_callback()) peer: 12345-192.168.204.3@tcp (source: 12345-192.168.204.3@tcp) 00000100:00100000:0.0:1776178154.496301:0:6256:0:(service.c:2152:ptlrpc_server_handle_req_in()) got req x1862456979092992 00000100:00100000:0.0:1776178154.496364:0:6256:0:(nrs_fifo.c:179:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.204.3@tcp, seq: 6543 00000100:00100000:0.0:1776178154.496372:0:6256:0:(service.c:2324:ptlrpc_server_handle_request()) Handling RPC req@00000000fed3e037 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_001:b50963bd-8eba-4432-9a54-8888d00e3fa1+46:17628:x1862456979092992:12345-192.168.204.3@tcp:101:touch.0 00010000:00010000:0.0:1776178154.496400:0:6256:0:(ldlm_lockd.c:1248:ldlm_handle_enqueue0()) ### server-side enqueue handler START 00010000:00010000:0.0:1776178154.496421:0:6256:0:(ldlm_lockd.c:1318:ldlm_handle_enqueue0()) ### server-side enqueue handler, new lock created ns: mdt-lustre-MDT0001_UUID lock: 00000000f51fd0d7/0xa40083ab9bf5347d lrc: 2/0,0 mode: --/CW res: [0x240000402:0x7f7:0x0].0x0 bits 0x0/0x0 rrc: 3 type: IBT gid 0 flags: 0x40000000000000 nid: local remote: 0x771f68aa1eb27acb expref: -99 pid: 6256 timeout: 0 lvb_type: 0 00010000:00010000:0.0:1776178154.496532:0:6256:0:(ldlm_lock.c:766:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(CW) ns: mdt-lustre-MDT0001_UUID lock: 000000006b206cc9/0xa40083ab9bf53484 lrc: 3/0,1 mode: --/CW res: [0x240000402:0x7f7:0x0].0x0 bits 0x0/0x0 rrc: 4 type: IBT gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 6256 timeout: 0 lvb_type: 0 00010000:00010000:0.0:1776178154.496561:0:6256:0:(ldlm_lock.c:1064:ldlm_granted_list_add_lock()) ### About to add lock: ns: mdt-lustre-MDT0001_UUID lock: 000000006b206cc9/0xa40083ab9bf53484 lrc: 3/0,1 mode: CW/CW res: [0x240000402:0x7f7:0x0].0x0 bits 0x2/0x0 rrc: 4 type: IBT gid 0 flags: 0x50210000000000 nid: local remote: 0x0 expref: -99 pid: 6256 timeout: 0 lvb_type: 0 00010000:00010000:0.0:1776178154.496573:0:6256:0:(ldlm_request.c:528:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: mdt-lustre-MDT0001_UUID lock: 000000006b206cc9/0xa40083ab9bf53484 lrc: 3/0,1 mode: CW/CW res: [0x240000402:0x7f7:0x0].0x0 bits 0x2/0x0 rrc: 4 type: IBT gid 0 flags: 0x40210000000000 nid: local remote: 0x0 expref: -99 pid: 6256 timeout: 0 lvb_type: 0 00010000:00010000:0.0:1776178154.496587:0:6256:0:(ldlm_lock.c:766:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: mdt-lustre-MDT0001_UUID lock: 000000008813523d/0xa40083ab9bf5348b lrc: 3/0,1 mode: --/PW res: [0x240000402:0x7f7:0x0].0x7fbd58ea bits 0x0/0x0 rrc: 2 type: IBT gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 6256 timeout: 0 lvb_type: 0 00010000:00010000:0.0:1776178154.496593:0:6256:0:(ldlm_lock.c:1064:ldlm_granted_list_add_lock()) ### About to add lock: ns: mdt-lustre-MDT0001_UUID lock: 000000008813523d/0xa40083ab9bf5348b lrc: 3/0,1 mode: PW/PW res: [0x240000402:0x7f7:0x0].0x7fbd58ea bits 0x2/0x0 rrc: 2 type: IBT gid 0 flags: 0x50210000000000 nid: local remote: 0x0 expref: -99 pid: 6256 timeout: 0 lvb_type: 0 00010000:00010000:0.0:1776178154.496599:0:6256:0:(ldlm_request.c:528:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: mdt-lustre-MDT0001_UUID lock: 000000008813523d/0xa40083ab9bf5348b lrc: 3/0,1 mode: PW/PW res: [0x240000402:0x7f7:0x0].0x7fbd58ea bits 0x2/0x0 rrc: 2 type: IBT gid 0 flags: 0x40210000000000 nid: local remote: 0x0 expref: -99 pid: 6256 timeout: 0 lvb_type: 0 00000004:00080000:0.0:1776178154.496911:0:6256:0:(osp_object.c:1645:osp_create()) lustre-OST0001-osc-MDT0001: Wrote last used FID: [0x2c0000400:0x431:0x0], index 1: 0 00010000:00010000:0.0:1776178154.497055:0:6256:0:(ldlm_lock.c:766:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(CR) ns: mdt-lustre-MDT0001_UUID lock: 00000000a211f94c/0xa40083ab9bf53492 lrc: 3/1,0 mode: --/CR res: [0x240000402:0x7fe:0x0].0x0 bits 0x0/0x0 rrc: 2 type: IBT gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 6256 timeout: 0 lvb_type: 0 00010000:00010000:0.0:1776178154.497619:0:6256:0:(ldlm_lock.c:1064:ldlm_granted_list_add_lock()) ### About to add lock: ns: mdt-lustre-MDT0001_UUID lock: 00000000a211f94c/0xa40083ab9bf53492 lrc: 3/1,0 mode: CR/CR res: [0x240000402:0x7fe:0x0].0x0 bits 0x9/0x0 rrc: 2 type: IBT gid 0 flags: 0x50210000000000 nid: local remote: 0x0 expref: -99 pid: 6256 timeout: 0 lvb_type: 0 00010000:00010000:0.0:1776178154.497629:0:6256:0:(ldlm_request.c:528:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: mdt-lustre-MDT0001_UUID lock: 00000000a211f94c/0xa40083ab9bf53492 lrc: 3/1,0 mode: CR/CR res: [0x240000402:0x7fe:0x0].0x0 bits 0x9/0x0 rrc: 2 type: IBT gid 0 flags: 0x40210000000000 nid: local remote: 0x0 expref: -99 pid: 6256 timeout: 0 lvb_type: 0 00010000:00010000:0.0:1776178154.497700:0:6256:0:(ldlm_lock.c:820:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(CW) ns: mdt-lustre-MDT0001_UUID lock: 000000006b206cc9/0xa40083ab9bf53484 lrc: 3/0,1 mode: CW/CW res: [0x240000402:0x7f7:0x0].0x0 bits 0x2/0x0 rrc: 4 type: IBT gid 0 flags: 0x50210000000000 nid: local remote: 0x0 expref: -99 pid: 6256 timeout: 0 lvb_type: 0 00010000:00010000:0.0:1776178154.497707:0:6256:0:(ldlm_lock.c:879:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: mdt-lustre-MDT0001_UUID lock: 000000006b206cc9/0xa40083ab9bf53484 lrc: 2/0,0 mode: CW/CW res: [0x240000402:0x7f7:0x0].0x0 bits 0x2/0x0 rrc: 4 type: IBT gid 0 flags: 0x50210400000000 nid: local remote: 0x0 expref: -99 pid: 6256 timeout: 0 lvb_type: 0 00010000:00010000:0.0:1776178154.497714:0:6256:0:(ldlm_lockd.c:1879:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: mdt-lustre-MDT0001_UUID lock: 000000006b206cc9/0xa40083ab9bf53484 lrc: 3/0,0 mode: CW/CW res: [0x240000402:0x7f7:0x0].0x0 bits 0x2/0x0 rrc: 4 type: IBT gid 0 flags: 0x40210400000000 nid: local remote: 0x0 expref: -99 pid: 6256 timeout: 0 lvb_type: 0 00010000:00010000:0.0:1776178154.497721:0:6256:0:(ldlm_lockd.c:1893:ldlm_handle_bl_callback()) Lock 000000006b206cc9 already unused, calling callback (0000000079e78348) 00010000:00010000:0.0:1776178154.497724:0:6256:0:(ldlm_request.c:364:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: mdt-lustre-MDT0001_UUID lock: 000000006b206cc9/0xa40083ab9bf53484 lrc: 3/0,0 mode: CW/CW res: [0x240000402:0x7f7:0x0].0x0 bits 0x2/0x0 rrc: 4 type: IBT gid 0 flags: 0x40210400000000 nid: local remote: 0x0 expref: -99 pid: 6256 timeout: 0 lvb_type: 0 00010000:00010000:0.0:1776178154.497731:0:6256:0:(ldlm_request.c:1257:ldlm_cli_cancel_local()) ### server-side local cancel ns: mdt-lustre-MDT0001_UUID lock: 000000006b206cc9/0xa40083ab9bf53484 lrc: 4/0,0 mode: CW/CW res: [0x240000402:0x7f7:0x0].0x0 bits 0x2/0x0 rrc: 4 type: IBT gid 0 flags: 0x40218400000000 nid: local remote: 0x0 expref: -99 pid: 6256 timeout: 0 lvb_type: 0 00010000:00010000:0.0:1776178154.497743:0:6256:0:(ldlm_inodebits.c:94:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x240000402:0x7f7:0x0].0x0 (000000006b762dc5) 00010000:00010000:0.0:1776178154.497746:0:6256:0:(ldlm_inodebits.c:96:ldlm_reprocess_inodebits_queue()) Hint 2 00010000:00010000:0.0:1776178154.497750:0:6256:0:(ldlm_lockd.c:1903:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: mdt-lustre-MDT0001_UUID lock: 000000006b206cc9/0xa40083ab9bf53484 lrc: 2/0,0 mode: --/CW res: [0x240000402:0x7f7:0x0].0x0 bits 0x2/0x0 rrc: 4 type: IBT gid 0 flags: 0x44a19400000000 nid: local remote: 0x0 expref: -99 pid: 6256 timeout: 0 lvb_type: 0 00010000:00010000:0.0:1776178154.497755:0:6256:0:(ldlm_lock.c:216:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: mdt-lustre-MDT0001_UUID lock: 000000006b206cc9/0xa40083ab9bf53484 lrc: 0/0,0 mode: --/CW res: [0x240000402:0x7f7:0x0].0x0 bits 0x2/0x0 rrc: 4 type: IBT gid 0 flags: 0x44a19400000000 nid: local remote: 0x0 expref: -99 pid: 6256 timeout: 0 lvb_type: 0 00000004:00010000:0.0:1776178154.497768:0:6256:0:(mdt_handler.c:3992:mdt_save_lock()) ### save lock request 00000000fed3e037 reply state 000000009aa4f42f transno 17179876254 ns: mdt-lustre-MDT0001_UUID lock: 000000008813523d/0xa40083ab9bf5348b lrc: 3/0,1 mode: PW/PW res: [0x240000402:0x7f7:0x0].0x7fbd58ea bits 0x2/0x0 rrc: 2 type: IBT gid 0 flags: 0x40210000000000 nid: local remote: 0x0 expref: -99 pid: 6256 timeout: 0 lvb_type: 0 00010000:00010000:0.0:1776178154.497799:0:6256:0:(ldlm_lock.c:216:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: mdt-lustre-MDT0001_UUID lock: 00000000f51fd0d7/0xa40083ab9bf5347d lrc: 0/0,0 mode: --/CW res: [0x240000402:0x7f7:0x0].0x0 bits 0x1/0x0 rrc: 3 type: IBT gid 0 flags: 0x44000000000000 nid: 192.168.204.3@tcp remote: 0x771f68aa1eb27acb expref: 49 pid: 6256 timeout: 0 lvb_type: 0 00010000:00010000:0.0:1776178154.497812:0:6256:0:(ldlm_lockd.c:1484:ldlm_handle_enqueue0()) ### server-side enqueue handler, sending reply (err=0, rc=0) ns: mdt-lustre-MDT0001_UUID lock: 00000000a211f94c/0xa40083ab9bf53492 lrc: 3/0,0 mode: CR/CR res: [0x240000402:0x7fe:0x0].0x0 bits 0x9/0x0 rrc: 2 type: IBT gid 0 flags: 0x40200000000000 nid: 192.168.204.3@tcp remote: 0x771f68aa1eb27acb expref: 48 pid: 6256 timeout: 0 lvb_type: 0 00010000:00010000:0.0:1776178154.497819:0:6256:0:(ldlm_inodebits.c:94:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x240000402:0x7fe:0x0].0x0 (00000000498a007f) 00010000:00010000:0.0:1776178154.497821:0:6256:0:(ldlm_inodebits.c:96:ldlm_reprocess_inodebits_queue()) Hint 9 00010000:00010000:0.0:1776178154.497823:0:6256:0:(ldlm_lockd.c:1565:ldlm_handle_enqueue0()) ### server-side enqueue handler END (lock 00000000a211f94c, rc 0) 00000100:00100000:0.0:1776178154.497899:0:6256:0:(service.c:2372:ptlrpc_server_handle_request()) Handled RPC req@00000000fed3e037 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_001:b50963bd-8eba-4432-9a54-8888d00e3fa1+49:17628:x1862456979092992:12345-192.168.204.3@tcp:101:touch.0 Request processed in 1526us (2003us total) trans 17179876254 rc 0/0 00000100:00100000:0.0:1776178154.497909:0:6256:0:(nrs_fifo.c:241:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.204.3@tcp, seq: 6543 00010000:00010000:3.0:1776178154.503899:0:3339:0:(ldlm_lock.c:1064:ldlm_granted_list_add_lock()) ### About to add lock: ns: mdt-lustre-MDT0001_UUID lock: 000000008813523d/0xa40083ab9bf5348b lrc: 3/0,1 mode: COS/COS res: [0x240000402:0x7f7:0x0].0x7fbd58ea bits 0x2/0x0 rrc: 2 type: IBT gid 0 flags: 0x50210000000000 nid: local remote: 0x0 expref: -99 pid: 6256 timeout: 0 lvb_type: 0 00010000:00010000:3.0:1776178154.503920:0:3339:0:(ldlm_inodebits.c:94:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x240000402:0x7f7:0x0].0x7fbd58ea (000000008c340be4) 00010000:00010000:3.0:1776178154.503923:0:3339:0:(ldlm_inodebits.c:96:ldlm_reprocess_inodebits_queue()) Hint 2 00000100:00100000:1.0:1776178154.510671:0:3347:0:(events.c:361:request_in_callback()) peer: 12345-192.168.204.3@tcp (source: 12345-192.168.204.3@tcp) 00000100:00100000:2.0:1776178154.510773:0:6255:0:(service.c:2152:ptlrpc_server_handle_req_in()) got req x1862456979093120 00000100:00100000:2.0:1776178154.510838:0:6255:0:(nrs_fifo.c:179:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.204.3@tcp, seq: 6544 00000100:00100000:2.0:1776178154.510846:0:6255:0:(service.c:2324:ptlrpc_server_handle_request()) Handling RPC req@00000000ae838296 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_000:b50963bd-8eba-4432-9a54-8888d00e3fa1+49:17628:x1862456979093120:12345-192.168.204.3@tcp:36:touch.0 00010000:00010000:2.0:1776178154.510989:0:6255:0:(ldlm_lock.c:766:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: mdt-lustre-MDT0001_UUID lock: 000000007da3b625/0xa40083ab9bf534a0 lrc: 3/0,1 mode: --/PW res: [0x240000402:0x7fe:0x0].0x0 bits 0x0/0x0 rrc: 3 type: IBT gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 6255 timeout: 0 lvb_type: 0 00010000:00010000:2.0:1776178154.511025:0:6255:0:(ldlm_lock.c:1064:ldlm_granted_list_add_lock()) ### About to add lock: ns: mdt-lustre-MDT0001_UUID lock: 000000007da3b625/0xa40083ab9bf534a0 lrc: 3/0,1 mode: PW/PW res: [0x240000402:0x7fe:0x0].0x0 bits 0x2/0x0 rrc: 3 type: IBT gid 0 flags: 0x50210000000000 nid: local remote: 0x0 expref: -99 pid: 6255 timeout: 0 lvb_type: 0 00010000:00010000:2.0:1776178154.511043:0:6255:0:(ldlm_request.c:528:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: mdt-lustre-MDT0001_UUID lock: 000000007da3b625/0xa40083ab9bf534a0 lrc: 3/0,1 mode: PW/PW res: [0x240000402:0x7fe:0x0].0x0 bits 0x2/0x0 rrc: 3 type: IBT gid 0 flags: 0x40210000000000 nid: local remote: 0x0 expref: -99 pid: 6255 timeout: 0 lvb_type: 0 00000004:00010000:2.0:1776178154.511188:0:6255:0:(mdt_handler.c:3992:mdt_save_lock()) ### save lock request 00000000ae838296 reply state 00000000621dc145 transno 17179876255 ns: mdt-lustre-MDT0001_UUID lock: 000000007da3b625/0xa40083ab9bf534a0 lrc: 3/0,1 mode: PW/PW res: [0x240000402:0x7fe:0x0].0x0 bits 0x2/0x0 rrc: 3 type: IBT gid 0 flags: 0x40210000000000 nid: local remote: 0x0 expref: -99 pid: 6255 timeout: 0 lvb_type: 0 00000100:00100000:2.0:1776178154.511308:0:6255:0:(service.c:2372:ptlrpc_server_handle_request()) Handled RPC req@00000000ae838296 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_000:b50963bd-8eba-4432-9a54-8888d00e3fa1+51:17628:x1862456979093120:12345-192.168.204.3@tcp:36:touch.0 Request processed in 461us (644us total) trans 17179876255 rc 0/0 00000100:00100000:2.0:1776178154.511315:0:6255:0:(nrs_fifo.c:241:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.204.3@tcp, seq: 6544 00000100:00100000:2.0:1776178154.522607:0:3346:0:(events.c:361:request_in_callback()) peer: 12345-192.168.204.3@tcp (source: 12345-192.168.204.3@tcp) 00000100:00100000:3.0:1776178154.523435:0:8228:0:(service.c:2152:ptlrpc_server_handle_req_in()) got req x1862456979093184 00000100:00100000:3.0:1776178154.523565:0:8228:0:(nrs_fifo.c:179:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.204.3@tcp, seq: 5263 00000100:00100000:3.0:1776178154.523573:0:8228:0:(service.c:2324:ptlrpc_server_handle_request()) Handling RPC req@0000000043589f56 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_001:b50963bd-8eba-4432-9a54-8888d00e3fa1+8:2271:x1862456979093184:12345-192.168.204.3@tcp:2:touch.0 00000100:00100000:3.0:1776178154.523911:0:8228:0:(service.c:2372:ptlrpc_server_handle_request()) Handled RPC req@0000000043589f56 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_001:b50963bd-8eba-4432-9a54-8888d00e3fa1+9:2271:x1862456979093184:12345-192.168.204.3@tcp:2:touch.0 Request processed in 336us (1314us total) trans 4294969327 rc 0/0 00000100:00100000:3.0:1776178154.523921:0:8228:0:(nrs_fifo.c:241:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.204.3@tcp, seq: 5263 00000100:00100000:3.0:1776178154.527348:0:3348:0:(events.c:361:request_in_callback()) peer: 12345-192.168.204.3@tcp (source: 12345-192.168.204.3@tcp) 00010000:00010000:1.0:1776178154.527982:0:3340:0:(ldlm_lock.c:1064:ldlm_granted_list_add_lock()) ### About to add lock: ns: mdt-lustre-MDT0001_UUID lock: 000000007da3b625/0xa40083ab9bf534a0 lrc: 3/0,1 mode: COS/COS res: [0x240000402:0x7fe:0x0].0x0 bits 0x2/0x0 rrc: 3 type: IBT gid 0 flags: 0x50210000000000 nid: local remote: 0x0 expref: -99 pid: 6255 timeout: 0 lvb_type: 0 00010000:00010000:1.0:1776178154.528003:0:3340:0:(ldlm_inodebits.c:94:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x240000402:0x7fe:0x0].0x0 (00000000498a007f) 00010000:00010000:1.0:1776178154.528006:0:3340:0:(ldlm_inodebits.c:96:ldlm_reprocess_inodebits_queue()) Hint 2 00000100:00100000:2.0:1776178154.528340:0:6258:0:(service.c:2152:ptlrpc_server_handle_req_in()) got req x1862456979093312 00000100:00100000:2.0:1776178154.528456:0:6258:0:(nrs_fifo.c:179:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.204.3@tcp, seq: 2141 00000100:00100000:2.0:1776178154.528464:0:6258:0:(service.c:2324:ptlrpc_server_handle_request()) Handling RPC req@000000001d2ef500 pname:cluuid+ref:pid:xid:nid:opc:job mdt_rdpg00_000:b50963bd-8eba-4432-9a54-8888d00e3fa1+51:17628:x1862456979093312:12345-192.168.204.3@tcp:35:touch.0 00000100:00100000:2.0:1776178154.528860:0:6258:0:(service.c:2372:ptlrpc_server_handle_request()) Handled RPC req@000000001d2ef500 pname:cluuid+ref:pid:xid:nid:opc:job mdt_rdpg00_000:b50963bd-8eba-4432-9a54-8888d00e3fa1+52:17628:x1862456979093312:12345-192.168.204.3@tcp:35:touch.0 Request processed in 396us (1526us total) trans 17179876256 rc 0/0 00000100:00100000:2.0:1776178154.528868:0:6258:0:(nrs_fifo.c:241:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.204.3@tcp, seq: 2141 00010000:00010000:2.0:1776178154.920309:0:6244:0:(ldlm_lockd.c:2792:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:0.0:1776178155.741053:0:3347:0:(events.c:361:request_in_callback()) peer: 12345-192.168.204.3@tcp (source: 12345-192.168.204.3@tcp) 00000100:00100000:1.0:1776178155.741399:0:6256:0:(service.c:2152:ptlrpc_server_handle_req_in()) got req x1862456979093376 00000100:00100000:1.0:1776178155.741461:0:6256:0:(nrs_fifo.c:179:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.204.3@tcp, seq: 6545 00000100:00100000:1.0:1776178155.741469:0:6256:0:(service.c:2324:ptlrpc_server_handle_request()) Handling RPC req@000000006b9d8ccb pname:cluuid+ref:pid:xid:nid:opc:job mdt00_001:b50963bd-8eba-4432-9a54-8888d00e3fa1+52:17639:x1862456979093376:12345-192.168.204.3@tcp:101:chmod.0 00010000:00010000:1.0:1776178155.741496:0:6256:0:(ldlm_lockd.c:1248:ldlm_handle_enqueue0()) ### server-side enqueue handler START 00010000:00010000:1.0:1776178155.741516:0:6256:0:(ldlm_lockd.c:1318:ldlm_handle_enqueue0()) ### server-side enqueue handler, new lock created ns: mdt-lustre-MDT0001_UUID lock: 000000003e1697a2/0xa40083ab9bf534a7 lrc: 2/0,0 mode: --/CR res: [0x240000402:0x7f7:0x0].0x0 bits 0x0/0x0 rrc: 3 type: IBT gid 0 flags: 0x40000000000000 nid: local remote: 0x771f68aa1eb27ad2 expref: -99 pid: 6256 timeout: 0 lvb_type: 0 00010000:00010000:1.0:1776178155.741579:0:6256:0:(ldlm_lock.c:766:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PR) ns: mdt-lustre-MDT0001_UUID lock: 0000000031bb7277/0xa40083ab9bf534ae lrc: 3/1,0 mode: --/PR res: [0x240000402:0x7f8:0x0].0x0 bits 0x0/0x0 rrc: 4 type: IBT gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 6256 timeout: 0 lvb_type: 0 00010000:00010000:1.0:1776178155.741595:0:6256:0:(ldlm_lock.c:1064:ldlm_granted_list_add_lock()) ### About to add lock: ns: mdt-lustre-MDT0001_UUID lock: 0000000031bb7277/0xa40083ab9bf534ae lrc: 3/1,0 mode: PR/PR res: [0x240000402:0x7f8:0x0].0x0 bits 0x5b/0x0 rrc: 4 type: IBT gid 0 flags: 0x50210000000000 nid: local remote: 0x0 expref: -99 pid: 6256 timeout: 0 lvb_type: 0 00010000:00010000:1.0:1776178155.741609:0:6256:0:(ldlm_request.c:528:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: mdt-lustre-MDT0001_UUID lock: 0000000031bb7277/0xa40083ab9bf534ae lrc: 3/1,0 mode: PR/PR res: [0x240000402:0x7f8:0x0].0x0 bits 0x5b/0x0 rrc: 4 type: IBT gid 0 flags: 0x40210000000000 nid: local remote: 0x0 expref: -99 pid: 6256 timeout: 0 lvb_type: 0 00000004:00200000:1.0:1776178155.741663:0:6256:0:(mdt_handler.c:912:mdt_pack_attr2body()) [0x240000402:0x7f8:0x0]: returning size 0 00000004:00200000:1.0:1776178155.741667:0:6256:0:(mdt_handler.c:1523:mdt_getattr_internal()) dirent count 0 stripe count 1 MDT count 2 00000004:00010000:1.0:1776178155.741698:0:6256:0:(mdt_handler.c:2353:mdt_getattr_name_lock()) ### Returning lock to client ns: mdt-lustre-MDT0001_UUID lock: 0000000031bb7277/0xa40083ab9bf534ae lrc: 3/1,0 mode: PR/PR res: [0x240000402:0x7f8:0x0].0x0 bits 0x5b/0x0 rrc: 4 type: IBT gid 0 flags: 0x40210000000000 nid: local remote: 0x0 expref: -99 pid: 6256 timeout: 0 lvb_type: 0 00010000:00010000:1.0:1776178155.741711:0:6256:0:(ldlm_lock.c:1064:ldlm_granted_list_add_lock()) ### About to add lock: ns: mdt-lustre-MDT0001_UUID lock: 0000000031bb7277/0xa40083ab9bf534ae lrc: 3/1,0 mode: PR/PR res: [0x240000402:0x7f8:0x0].0x0 bits 0x1b/0x0 rrc: 4 type: IBT gid 0 flags: 0x50210000000000 nid: local remote: 0x0 expref: -99 pid: 6256 timeout: 0 lvb_type: 0 00010000:00010000:1.0:1776178155.741734:0:6256:0:(ldlm_lock.c:216:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: mdt-lustre-MDT0001_UUID lock: 000000003e1697a2/0xa40083ab9bf534a7 lrc: 0/0,0 mode: --/CR res: [0x240000402:0x7f7:0x0].0x0 bits 0x2/0x0 rrc: 3 type: IBT gid 0 flags: 0x44000000000000 nid: 192.168.204.3@tcp remote: 0x771f68aa1eb27ad2 expref: 54 pid: 6256 timeout: 0 lvb_type: 0 00010000:00010000:1.0:1776178155.741751:0:6256:0:(ldlm_lockd.c:1484:ldlm_handle_enqueue0()) ### server-side enqueue handler, sending reply (err=0, rc=0) ns: mdt-lustre-MDT0001_UUID lock: 0000000031bb7277/0xa40083ab9bf534ae lrc: 3/0,0 mode: PR/PR res: [0x240000402:0x7f8:0x0].0x0 bits 0x1b/0x0 rrc: 4 type: IBT gid 0 flags: 0x40200000000000 nid: 192.168.204.3@tcp remote: 0x771f68aa1eb27ad2 expref: 53 pid: 6256 timeout: 0 lvb_type: 0 00010000:00010000:1.0:1776178155.741759:0:6256:0:(ldlm_inodebits.c:94:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x240000402:0x7f8:0x0].0x0 (00000000099618da) 00010000:00010000:1.0:1776178155.741762:0:6256:0:(ldlm_inodebits.c:96:ldlm_reprocess_inodebits_queue()) Hint 1b 00010000:00010000:1.0:1776178155.741765:0:6256:0:(ldlm_lockd.c:1565:ldlm_handle_enqueue0()) ### server-side enqueue handler END (lock 0000000031bb7277, rc 0) 00000100:00100000:1.0:1776178155.741825:0:6256:0:(service.c:2372:ptlrpc_server_handle_request()) Handled RPC req@000000006b9d8ccb pname:cluuid+ref:pid:xid:nid:opc:job mdt00_001:b50963bd-8eba-4432-9a54-8888d00e3fa1+53:17639:x1862456979093376:12345-192.168.204.3@tcp:101:chmod.0 Request processed in 356us (781us total) trans 0 rc 0/0 00000100:00100000:1.0:1776178155.741846:0:6256:0:(nrs_fifo.c:241:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.204.3@tcp, seq: 6545 00000100:00100000:0.0:1776178155.751615:0:3346:0:(events.c:361:request_in_callback()) peer: 12345-192.168.204.3@tcp (source: 12345-192.168.204.3@tcp) 00000100:00100000:1.0:1776178155.751716:0:6255:0:(service.c:2152:ptlrpc_server_handle_req_in()) got req x1862456979093440 00000100:00100000:1.0:1776178155.751784:0:6255:0:(nrs_fifo.c:179:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.204.3@tcp, seq: 6546 00000100:00100000:1.0:1776178155.751792:0:6255:0:(service.c:2324:ptlrpc_server_handle_request()) Handling RPC req@00000000ddf29755 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_000:b50963bd-8eba-4432-9a54-8888d00e3fa1+53:17642:x1862456979093440:12345-192.168.204.3@tcp:101:chmod.0 00010000:00010000:1.0:1776178155.751816:0:6255:0:(ldlm_lockd.c:1248:ldlm_handle_enqueue0()) ### server-side enqueue handler START 00010000:00010000:1.0:1776178155.751844:0:6255:0:(ldlm_lockd.c:1318:ldlm_handle_enqueue0()) ### server-side enqueue handler, new lock created ns: mdt-lustre-MDT0001_UUID lock: 00000000a1e024a5/0xa40083ab9bf534b5 lrc: 2/0,0 mode: --/CR res: [0x240000402:0x7f7:0x0].0x0 bits 0x0/0x0 rrc: 3 type: IBT gid 0 flags: 0x40000000000000 nid: local remote: 0x771f68aa1eb27ad9 expref: -99 pid: 6255 timeout: 0 lvb_type: 0 00010000:00010000:1.0:1776178155.751934:0:6255:0:(ldlm_lock.c:766:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PR) ns: mdt-lustre-MDT0001_UUID lock: 0000000014581b1a/0xa40083ab9bf534bc lrc: 3/1,0 mode: --/PR res: [0x240000402:0x7fb:0x0].0x0 bits 0x0/0x0 rrc: 4 type: IBT gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 6255 timeout: 0 lvb_type: 0 00010000:00010000:1.0:1776178155.751952:0:6255:0:(ldlm_lock.c:1064:ldlm_granted_list_add_lock()) ### About to add lock: ns: mdt-lustre-MDT0001_UUID lock: 0000000014581b1a/0xa40083ab9bf534bc lrc: 3/1,0 mode: PR/PR res: [0x240000402:0x7fb:0x0].0x0 bits 0x5b/0x0 rrc: 4 type: IBT gid 0 flags: 0x50210000000000 nid: local remote: 0x0 expref: -99 pid: 6255 timeout: 0 lvb_type: 0 00010000:00010000:1.0:1776178155.751967:0:6255:0:(ldlm_request.c:528:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: mdt-lustre-MDT0001_UUID lock: 0000000014581b1a/0xa40083ab9bf534bc lrc: 3/1,0 mode: PR/PR res: [0x240000402:0x7fb:0x0].0x0 bits 0x5b/0x0 rrc: 4 type: IBT gid 0 flags: 0x40210000000000 nid: local remote: 0x0 expref: -99 pid: 6255 timeout: 0 lvb_type: 0 00000004:00200000:1.0:1776178155.752019:0:6255:0:(mdt_handler.c:912:mdt_pack_attr2body()) [0x240000402:0x7fb:0x0]: returning size 0 00000004:00200000:1.0:1776178155.752024:0:6255:0:(mdt_handler.c:1523:mdt_getattr_internal()) dirent count 0 stripe count 1 MDT count 2 00000004:00010000:1.0:1776178155.752043:0:6255:0:(mdt_handler.c:2353:mdt_getattr_name_lock()) ### Returning lock to client ns: mdt-lustre-MDT0001_UUID lock: 0000000014581b1a/0xa40083ab9bf534bc lrc: 3/1,0 mode: PR/PR res: [0x240000402:0x7fb:0x0].0x0 bits 0x5b/0x0 rrc: 4 type: IBT gid 0 flags: 0x40210000000000 nid: local remote: 0x0 expref: -99 pid: 6255 timeout: 0 lvb_type: 0 00010000:00010000:1.0:1776178155.752055:0:6255:0:(ldlm_lock.c:1064:ldlm_granted_list_add_lock()) ### About to add lock: ns: mdt-lustre-MDT0001_UUID lock: 0000000014581b1a/0xa40083ab9bf534bc lrc: 3/1,0 mode: PR/PR res: [0x240000402:0x7fb:0x0].0x0 bits 0x1b/0x0 rrc: 4 type: IBT gid 0 flags: 0x50210000000000 nid: local remote: 0x0 expref: -99 pid: 6255 timeout: 0 lvb_type: 0 00010000:00010000:1.0:1776178155.752118:0:6255:0:(ldlm_lock.c:216:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: mdt-lustre-MDT0001_UUID lock: 00000000a1e024a5/0xa40083ab9bf534b5 lrc: 0/0,0 mode: --/CR res: [0x240000402:0x7f7:0x0].0x0 bits 0x2/0x0 rrc: 3 type: IBT gid 0 flags: 0x44000000000000 nid: 192.168.204.3@tcp remote: 0x771f68aa1eb27ad9 expref: 55 pid: 6255 timeout: 0 lvb_type: 0 00010000:00010000:1.0:1776178155.752136:0:6255:0:(ldlm_lockd.c:1484:ldlm_handle_enqueue0()) ### server-side enqueue handler, sending reply (err=0, rc=0) ns: mdt-lustre-MDT0001_UUID lock: 0000000014581b1a/0xa40083ab9bf534bc lrc: 3/0,0 mode: PR/PR res: [0x240000402:0x7fb:0x0].0x0 bits 0x1b/0x0 rrc: 4 type: IBT gid 0 flags: 0x40200000000000 nid: 192.168.204.3@tcp remote: 0x771f68aa1eb27ad9 expref: 54 pid: 6255 timeout: 0 lvb_type: 0 00010000:00010000:1.0:1776178155.752145:0:6255:0:(ldlm_inodebits.c:94:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x240000402:0x7fb:0x0].0x0 (000000009f7659df) 00010000:00010000:1.0:1776178155.752147:0:6255:0:(ldlm_inodebits.c:96:ldlm_reprocess_inodebits_queue()) Hint 1b 00010000:00010000:1.0:1776178155.752150:0:6255:0:(ldlm_lockd.c:1565:ldlm_handle_enqueue0()) ### server-side enqueue handler END (lock 0000000014581b1a, rc 0) 00000100:00100000:1.0:1776178155.752211:0:6255:0:(service.c:2372:ptlrpc_server_handle_request()) Handled RPC req@00000000ddf29755 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_000:b50963bd-8eba-4432-9a54-8888d00e3fa1+54:17642:x1862456979093440:12345-192.168.204.3@tcp:101:chmod.0 Request processed in 419us (609us total) trans 0 rc 0/0 00000100:00100000:1.0:1776178155.752219:0:6255:0:(nrs_fifo.c:241:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.204.3@tcp, seq: 6546 00000100:00100000:3.0:1776178155.756627:0:3348:0:(events.c:361:request_in_callback()) peer: 12345-192.168.204.3@tcp (source: 12345-192.168.204.3@tcp) 00000100:00100000:3.0:1776178155.756667:0:3348:0:(events.c:361:request_in_callback()) peer: 12345-192.168.204.3@tcp (source: 12345-192.168.204.3@tcp) 00000100:00100000:3.0:1776178155.756699:0:3348:0:(events.c:361:request_in_callback()) peer: 12345-192.168.204.3@tcp (source: 12345-192.168.204.3@tcp) 00000100:00100000:1.0:1776178155.756768:0:8228:0:(service.c:2152:ptlrpc_server_handle_req_in()) got req x1862456979093568 00000100:00100000:1.0:1776178155.756825:0:8228:0:(nrs_fifo.c:179:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.204.3@tcp, seq: 5264 00000100:00100000:1.0:1776178155.756832:0:8228:0:(service.c:2324:ptlrpc_server_handle_request()) Handling RPC req@00000000f7d56588 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_001:b50963bd-8eba-4432-9a54-8888d00e3fa1+9:2273:x1862456979093568:12345-192.168.204.3@tcp:101:chmod.0 00010000:00010000:1.0:1776178155.756852:0:8228:0:(ldlm_lockd.c:1248:ldlm_handle_enqueue0()) ### server-side enqueue handler START 00010000:00010000:1.0:1776178155.756891:0:8228:0:(ldlm_lockd.c:1318:ldlm_handle_enqueue0()) ### server-side enqueue handler, new lock created ns: filter-lustre-OST0001_UUID lock: 000000001d465f75/0xa40083ab9bf534c3 lrc: 2/0,0 mode: --/PR res: [0x2c0000400:0x42e:0x0].0x0 rrc: 2 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x771f68aa1eb27ae7 expref: -99 pid: 8228 timeout: 0 lvb_type: 0 00002000:00010000:1.0:1776178155.756918:0:8228:0:(ofd_lvb.c:170:ofd_lvbo_init()) res: [0x2c0000400:0x42e:0x0] initial LVB size: 0, mtime: 0x69de53e9, atime: 0x69de53e9, ctime: 0x69de53e9, blocks: 0x0 00010000:00010000:1.0:1776178155.756942:0:8228:0:(ldlm_extent.c:318:ldlm_check_contention()) contended locks = 0 00010000:00010000:1.0:1776178155.756944:0:8228:0:(ldlm_extent.c:318:ldlm_check_contention()) contended locks = 0 00010000:00010000:1.0:1776178155.756954:0:8228:0:(ldlm_resource.c:1650:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0001_UUID lock: 000000001d465f75/0xa40083ab9bf534c3 lrc: 3/0,0 mode: PR/PR res: [0x2c0000400:0x42e:0x0].0x0 rrc: 2 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x40000000000000 nid: 192.168.204.3@tcp remote: 0x771f68aa1eb27ae7 expref: 10 pid: 8228 timeout: 0 lvb_type: 1 00010000:00010000:1.0:1776178155.756976:0:8228:0:(ldlm_lockd.c:1484:ldlm_handle_enqueue0()) ### server-side enqueue handler, sending reply (err=0, rc=0) ns: filter-lustre-OST0001_UUID lock: 000000001d465f75/0xa40083ab9bf534c3 lrc: 3/0,0 mode: PR/PR res: [0x2c0000400:0x42e:0x0].0x0 rrc: 2 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x40000000000000 nid: 192.168.204.3@tcp remote: 0x771f68aa1eb27ae7 expref: 10 pid: 8228 timeout: 0 lvb_type: 1 00010000:00010000:1.0:1776178155.756992:0:8228:0:(ldlm_lockd.c:1565:ldlm_handle_enqueue0()) ### server-side enqueue handler END (lock 000000001d465f75, rc 0) 00000100:00100000:1.0:1776178155.757081:0:8228:0:(service.c:2372:ptlrpc_server_handle_request()) Handled RPC req@00000000f7d56588 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_001:b50963bd-8eba-4432-9a54-8888d00e3fa1+10:2273:x1862456979093568:12345-192.168.204.3@tcp:101:chmod.0 Request processed in 224us (389us total) trans 0 rc 0/0 00000100:00100000:1.0:1776178155.757090:0:8228:0:(nrs_fifo.c:241:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.204.3@tcp, seq: 5264 00000100:00100000:1.0:1776178155.757174:0:6255:0:(service.c:2152:ptlrpc_server_handle_req_in()) got req x1862456979093504 00000100:00100000:1.0:1776178155.757196:0:6255:0:(service.c:2152:ptlrpc_server_handle_req_in()) got req x1862456979093632 00000100:00100000:1.0:1776178155.757211:0:6255:0:(nrs_fifo.c:179:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.204.3@tcp, seq: 6547 00000100:00100000:1.0:1776178155.757215:0:6255:0:(service.c:2324:ptlrpc_server_handle_request()) Handling RPC req@000000002e8f0791 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_000:b50963bd-8eba-4432-9a54-8888d00e3fa1+55:17641:x1862456979093504:12345-192.168.204.3@tcp:101:chmod.0 00010000:00010000:1.0:1776178155.757226:0:6255:0:(ldlm_lockd.c:1248:ldlm_handle_enqueue0()) ### server-side enqueue handler START 00010000:00010000:1.0:1776178155.757237:0:6255:0:(ldlm_lockd.c:1318:ldlm_handle_enqueue0()) ### server-side enqueue handler, new lock created ns: mdt-lustre-MDT0001_UUID lock: 0000000022bf14ed/0xa40083ab9bf534ca lrc: 2/0,0 mode: --/CR res: [0x240000402:0x7f7:0x0].0x0 bits 0x0/0x0 rrc: 3 type: IBT gid 0 flags: 0x40000000000000 nid: local remote: 0x771f68aa1eb27ae0 expref: -99 pid: 6255 timeout: 0 lvb_type: 0 00010000:00010000:1.0:1776178155.757288:0:6255:0:(ldlm_lock.c:766:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PR) ns: mdt-lustre-MDT0001_UUID lock: 000000002f608ee6/0xa40083ab9bf534d1 lrc: 3/1,0 mode: --/PR res: [0x240000402:0x7fa:0x0].0x0 bits 0x0/0x0 rrc: 4 type: IBT gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 6255 timeout: 0 lvb_type: 0 00010000:00010000:1.0:1776178155.757302:0:6255:0:(ldlm_lock.c:1064:ldlm_granted_list_add_lock()) ### About to add lock: ns: mdt-lustre-MDT0001_UUID lock: 000000002f608ee6/0xa40083ab9bf534d1 lrc: 3/1,0 mode: PR/PR res: [0x240000402:0x7fa:0x0].0x0 bits 0x5b/0x0 rrc: 4 type: IBT gid 0 flags: 0x50210000000000 nid: local remote: 0x0 expref: -99 pid: 6255 timeout: 0 lvb_type: 0 00010000:00010000:1.0:1776178155.757312:0:6255:0:(ldlm_request.c:528:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: mdt-lustre-MDT0001_UUID lock: 000000002f608ee6/0xa40083ab9bf534d1 lrc: 3/1,0 mode: PR/PR res: [0x240000402:0x7fa:0x0].0x0 bits 0x5b/0x0 rrc: 4 type: IBT gid 0 flags: 0x40210000000000 nid: local remote: 0x0 expref: -99 pid: 6255 timeout: 0 lvb_type: 0 00000004:00200000:1.0:1776178155.757359:0:6255:0:(mdt_handler.c:912:mdt_pack_attr2body()) [0x240000402:0x7fa:0x0]: returning size 0 00000004:00200000:1.0:1776178155.757363:0:6255:0:(mdt_handler.c:1523:mdt_getattr_internal()) dirent count 0 stripe count 1 MDT count 2 00000004:00010000:1.0:1776178155.757383:0:6255:0:(mdt_handler.c:2353:mdt_getattr_name_lock()) ### Returning lock to client ns: mdt-lustre-MDT0001_UUID lock: 000000002f608ee6/0xa40083ab9bf534d1 lrc: 3/1,0 mode: PR/PR res: [0x240000402:0x7fa:0x0].0x0 bits 0x5b/0x0 rrc: 4 type: IBT gid 0 flags: 0x40210000000000 nid: local remote: 0x0 expref: -99 pid: 6255 timeout: 0 lvb_type: 0 00010000:00010000:1.0:1776178155.757395:0:6255:0:(ldlm_lock.c:1064:ldlm_granted_list_add_lock()) ### About to add lock: ns: mdt-lustre-MDT0001_UUID lock: 000000002f608ee6/0xa40083ab9bf534d1 lrc: 3/1,0 mode: PR/PR res: [0x240000402:0x7fa:0x0].0x0 bits 0x1b/0x0 rrc: 4 type: IBT gid 0 flags: 0x50210000000000 nid: local remote: 0x0 expref: -99 pid: 6255 timeout: 0 lvb_type: 0 00010000:00010000:1.0:1776178155.757416:0:6255:0:(ldlm_lock.c:216:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: mdt-lustre-MDT0001_UUID lock: 0000000022bf14ed/0xa40083ab9bf534ca lrc: 0/0,0 mode: --/CR res: [0x240000402:0x7f7:0x0].0x0 bits 0x2/0x0 rrc: 3 type: IBT gid 0 flags: 0x44000000000000 nid: 192.168.204.3@tcp remote: 0x771f68aa1eb27ae0 expref: 57 pid: 6255 timeout: 0 lvb_type: 0 00010000:00010000:1.0:1776178155.757434:0:6255:0:(ldlm_lockd.c:1484:ldlm_handle_enqueue0()) ### server-side enqueue handler, sending reply (err=0, rc=0) ns: mdt-lustre-MDT0001_UUID lock: 000000002f608ee6/0xa40083ab9bf534d1 lrc: 3/0,0 mode: PR/PR res: [0x240000402:0x7fa:0x0].0x0 bits 0x1b/0x0 rrc: 4 type: IBT gid 0 flags: 0x40200000000000 nid: 192.168.204.3@tcp remote: 0x771f68aa1eb27ae0 expref: 56 pid: 6255 timeout: 0 lvb_type: 0 00010000:00010000:1.0:1776178155.757442:0:6255:0:(ldlm_inodebits.c:94:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x240000402:0x7fa:0x0].0x0 (000000006b0fcbcb) 00010000:00010000:1.0:1776178155.757445:0:6255:0:(ldlm_inodebits.c:96:ldlm_reprocess_inodebits_queue()) Hint 1b 00010000:00010000:1.0:1776178155.757448:0:6255:0:(ldlm_lockd.c:1565:ldlm_handle_enqueue0()) ### server-side enqueue handler END (lock 000000002f608ee6, rc 0) 00000100:00100000:1.0:1776178155.757474:0:6255:0:(service.c:2372:ptlrpc_server_handle_request()) Handled RPC req@000000002e8f0791 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_000:b50963bd-8eba-4432-9a54-8888d00e3fa1+56:17641:x1862456979093504:12345-192.168.204.3@tcp:101:chmod.0 Request processed in 258us (864us total) trans 0 rc 0/0 00000100:00100000:1.0:1776178155.757480:0:6255:0:(nrs_fifo.c:241:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.204.3@tcp, seq: 6547 00000100:00100000:1.0:1776178155.757492:0:6255:0:(nrs_fifo.c:179:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.204.3@tcp, seq: 6548 00000100:00100000:1.0:1776178155.757497:0:6255:0:(service.c:2324:ptlrpc_server_handle_request()) Handling RPC req@0000000084604fbc pname:cluuid+ref:pid:xid:nid:opc:job mdt00_000:b50963bd-8eba-4432-9a54-8888d00e3fa1+55:17640:x1862456979093632:12345-192.168.204.3@tcp:101:chmod.0 00010000:00010000:1.0:1776178155.757503:0:6255:0:(ldlm_lockd.c:1248:ldlm_handle_enqueue0()) ### server-side enqueue handler START 00010000:00010000:1.0:1776178155.757509:0:6255:0:(ldlm_lockd.c:1318:ldlm_handle_enqueue0()) ### server-side enqueue handler, new lock created ns: mdt-lustre-MDT0001_UUID lock: 000000007e0712b8/0xa40083ab9bf534d8 lrc: 2/0,0 mode: --/CR res: [0x240000402:0x7f7:0x0].0x0 bits 0x0/0x0 rrc: 3 type: IBT gid 0 flags: 0x40000000000000 nid: local remote: 0x771f68aa1eb27aee expref: -99 pid: 6255 timeout: 0 lvb_type: 0 00010000:00010000:1.0:1776178155.757532:0:6255:0:(ldlm_lock.c:766:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PR) ns: mdt-lustre-MDT0001_UUID lock: 000000006844664c/0xa40083ab9bf534df lrc: 3/1,0 mode: --/PR res: [0x240000402:0x7f9:0x0].0x0 bits 0x0/0x0 rrc: 4 type: IBT gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 6255 timeout: 0 lvb_type: 0 00010000:00010000:1.0:1776178155.757541:0:6255:0:(ldlm_lock.c:1064:ldlm_granted_list_add_lock()) ### About to add lock: ns: mdt-lustre-MDT0001_UUID lock: 000000006844664c/0xa40083ab9bf534df lrc: 3/1,0 mode: PR/PR res: [0x240000402:0x7f9:0x0].0x0 bits 0x5b/0x0 rrc: 4 type: IBT gid 0 flags: 0x50210000000000 nid: local remote: 0x0 expref: -99 pid: 6255 timeout: 0 lvb_type: 0 00010000:00010000:1.0:1776178155.757549:0:6255:0:(ldlm_request.c:528:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: mdt-lustre-MDT0001_UUID lock: 000000006844664c/0xa40083ab9bf534df lrc: 3/1,0 mode: PR/PR res: [0x240000402:0x7f9:0x0].0x0 bits 0x5b/0x0 rrc: 4 type: IBT gid 0 flags: 0x40210000000000 nid: local remote: 0x0 expref: -99 pid: 6255 timeout: 0 lvb_type: 0 00000004:00200000:1.0:1776178155.757568:0:6255:0:(mdt_handler.c:912:mdt_pack_attr2body()) [0x240000402:0x7f9:0x0]: returning size 0 00000004:00200000:1.0:1776178155.757570:0:6255:0:(mdt_handler.c:1523:mdt_getattr_internal()) dirent count 0 stripe count 1 MDT count 2 00000004:00010000:1.0:1776178155.757578:0:6255:0:(mdt_handler.c:2353:mdt_getattr_name_lock()) ### Returning lock to client ns: mdt-lustre-MDT0001_UUID lock: 000000006844664c/0xa40083ab9bf534df lrc: 3/1,0 mode: PR/PR res: [0x240000402:0x7f9:0x0].0x0 bits 0x5b/0x0 rrc: 4 type: IBT gid 0 flags: 0x40210000000000 nid: local remote: 0x0 expref: -99 pid: 6255 timeout: 0 lvb_type: 0 00010000:00010000:1.0:1776178155.757585:0:6255:0:(ldlm_lock.c:1064:ldlm_granted_list_add_lock()) ### About to add lock: ns: mdt-lustre-MDT0001_UUID lock: 000000006844664c/0xa40083ab9bf534df lrc: 3/1,0 mode: PR/PR res: [0x240000402:0x7f9:0x0].0x0 bits 0x1b/0x0 rrc: 4 type: IBT gid 0 flags: 0x50210000000000 nid: local remote: 0x0 expref: -99 pid: 6255 timeout: 0 lvb_type: 0 00010000:00010000:1.0:1776178155.757597:0:6255:0:(ldlm_lock.c:216:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: mdt-lustre-MDT0001_UUID lock: 000000007e0712b8/0xa40083ab9bf534d8 lrc: 0/0,0 mode: --/CR res: [0x240000402:0x7f7:0x0].0x0 bits 0x2/0x0 rrc: 3 type: IBT gid 0 flags: 0x44000000000000 nid: 192.168.204.3@tcp remote: 0x771f68aa1eb27aee expref: 57 pid: 6255 timeout: 0 lvb_type: 0 00010000:00010000:1.0:1776178155.757608:0:6255:0:(ldlm_lockd.c:1484:ldlm_handle_enqueue0()) ### server-side enqueue handler, sending reply (err=0, rc=0) ns: mdt-lustre-MDT0001_UUID lock: 000000006844664c/0xa40083ab9bf534df lrc: 3/0,0 mode: PR/PR res: [0x240000402:0x7f9:0x0].0x0 bits 0x1b/0x0 rrc: 4 type: IBT gid 0 flags: 0x40200000000000 nid: 192.168.204.3@tcp remote: 0x771f68aa1eb27aee expref: 56 pid: 6255 timeout: 0 lvb_type: 0 00010000:00010000:1.0:1776178155.757614:0:6255:0:(ldlm_inodebits.c:94:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x240000402:0x7f9:0x0].0x0 (00000000f193fa59) 00010000:00010000:1.0:1776178155.757617:0:6255:0:(ldlm_inodebits.c:96:ldlm_reprocess_inodebits_queue()) Hint 1b 00010000:00010000:1.0:1776178155.757618:0:6255:0:(ldlm_lockd.c:1565:ldlm_handle_enqueue0()) ### server-side enqueue handler END (lock 000000006844664c, rc 0) 00000100:00100000:1.0:1776178155.757650:0:6255:0:(service.c:2372:ptlrpc_server_handle_request()) Handled RPC req@0000000084604fbc pname:cluuid+ref:pid:xid:nid:opc:job mdt00_000:b50963bd-8eba-4432-9a54-8888d00e3fa1+56:17640:x1862456979093632:12345-192.168.204.3@tcp:101:chmod.0 Request processed in 153us (951us total) trans 0 rc 0/0 00000100:00100000:1.0:1776178155.757655:0:6255:0:(nrs_fifo.c:241:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.204.3@tcp, seq: 6548 00000100:00100000:1.0:1776178155.775985:0:3348:0:(events.c:361:request_in_callback()) peer: 12345-192.168.204.3@tcp (source: 12345-192.168.204.3@tcp) 00000100:00100000:1.0:1776178155.776032:0:3348:0:(events.c:361:request_in_callback()) peer: 12345-192.168.204.3@tcp (source: 12345-192.168.204.3@tcp) 00000100:00100000:1.0:1776178155.776146:0:3348:0:(events.c:361:request_in_callback()) peer: 12345-192.168.204.3@tcp (source: 12345-192.168.204.3@tcp) 00000100:00100000:1.0:1776178155.776166:0:3348:0:(events.c:361:request_in_callback()) peer: 12345-192.168.204.3@tcp (source: 12345-192.168.204.3@tcp) 00000100:00100000:1.0:1776178155.787671:0:3348:0:(events.c:361:request_in_callback()) peer: 12345-192.168.204.3@tcp (source: 12345-192.168.204.3@tcp) 00000100:00100000:1.0:1776178155.787702:0:3348:0:(events.c:361:request_in_callback()) peer: 12345-192.168.204.3@tcp (source: 12345-192.168.204.3@tcp) 00000100:00100000:2.0:1776178155.808867:0:11281:0:(service.c:2152:ptlrpc_server_handle_req_in()) got req x1862456979093696 00000100:00100000:2.0:1776178155.808975:0:11281:0:(service.c:2152:ptlrpc_server_handle_req_in()) got req x1862456979093760 00000100:00100000:2.0:1776178155.808994:0:11281:0:(service.c:2152:ptlrpc_server_handle_req_in()) got req x1862456979093824 00000100:00100000:2.0:1776178155.809014:0:11281:0:(nrs_fifo.c:179:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.204.3@tcp, seq: 6549 00000100:00100000:2.0:1776178155.809030:0:11281:0:(service.c:2324:ptlrpc_server_handle_request()) Handling RPC req@0000000053e142bc pname:cluuid+ref:pid:xid:nid:opc:job mdt00_004:b50963bd-8eba-4432-9a54-8888d00e3fa1+58:17644:x1862456979093696:12345-192.168.204.3@tcp:101:chmod.0 00010000:00010000:2.0:1776178155.809054:0:11281:0:(ldlm_lockd.c:1248:ldlm_handle_enqueue0()) ### server-side enqueue handler START 00010000:00010000:2.0:1776178155.809093:0:11281:0:(ldlm_lockd.c:1318:ldlm_handle_enqueue0()) ### server-side enqueue handler, new lock created ns: mdt-lustre-MDT0001_UUID lock: 000000001e6d7318/0xa40083ab9bf534e6 lrc: 2/0,0 mode: --/CR res: [0x240000402:0x7f7:0x0].0x0 bits 0x0/0x0 rrc: 3 type: IBT gid 0 flags: 0x40000000000000 nid: local remote: 0x771f68aa1eb27af5 expref: -99 pid: 11281 timeout: 0 lvb_type: 0 00010000:00010000:2.0:1776178155.809166:0:11281:0:(ldlm_lock.c:766:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PR) ns: mdt-lustre-MDT0001_UUID lock: 000000007ec324d4/0xa40083ab9bf534ed lrc: 3/1,0 mode: --/PR res: [0x240000402:0x7fd:0x0].0x0 bits 0x0/0x0 rrc: 4 type: IBT gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 11281 timeout: 0 lvb_type: 0 00010000:00010000:2.0:1776178155.809183:0:11281:0:(ldlm_lock.c:1064:ldlm_granted_list_add_lock()) ### About to add lock: ns: mdt-lustre-MDT0001_UUID lock: 000000007ec324d4/0xa40083ab9bf534ed lrc: 3/1,0 mode: PR/PR res: [0x240000402:0x7fd:0x0].0x0 bits 0x5b/0x0 rrc: 4 type: IBT gid 0 flags: 0x50210000000000 nid: local remote: 0x0 expref: -99 pid: 11281 timeout: 0 lvb_type: 0 00010000:00010000:2.0:1776178155.809194:0:11281:0:(ldlm_request.c:528:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: mdt-lustre-MDT0001_UUID lock: 000000007ec324d4/0xa40083ab9bf534ed lrc: 3/1,0 mode: PR/PR res: [0x240000402:0x7fd:0x0].0x0 bits 0x5b/0x0 rrc: 4 type: IBT gid 0 flags: 0x40210000000000 nid: local remote: 0x0 expref: -99 pid: 11281 timeout: 0 lvb_type: 0 00000004:00200000:2.0:1776178155.809248:0:11281:0:(mdt_handler.c:912:mdt_pack_attr2body()) [0x240000402:0x7fd:0x0]: returning size 0 00000004:00200000:2.0:1776178155.809252:0:11281:0:(mdt_handler.c:1523:mdt_getattr_internal()) dirent count 0 stripe count 1 MDT count 2 00000004:00010000:2.0:1776178155.809281:0:11281:0:(mdt_handler.c:2353:mdt_getattr_name_lock()) ### Returning lock to client ns: mdt-lustre-MDT0001_UUID lock: 000000007ec324d4/0xa40083ab9bf534ed lrc: 3/1,0 mode: PR/PR res: [0x240000402:0x7fd:0x0].0x0 bits 0x5b/0x0 rrc: 4 type: IBT gid 0 flags: 0x40210000000000 nid: local remote: 0x0 expref: -99 pid: 11281 timeout: 0 lvb_type: 0 00010000:00010000:2.0:1776178155.809293:0:11281:0:(ldlm_lock.c:1064:ldlm_granted_list_add_lock()) ### About to add lock: ns: mdt-lustre-MDT0001_UUID lock: 000000007ec324d4/0xa40083ab9bf534ed lrc: 3/1,0 mode: PR/PR res: [0x240000402:0x7fd:0x0].0x0 bits 0x1b/0x0 rrc: 4 type: IBT gid 0 flags: 0x50210000000000 nid: local remote: 0x0 expref: -99 pid: 11281 timeout: 0 lvb_type: 0 00010000:00010000:2.0:1776178155.809319:0:11281:0:(ldlm_lock.c:216:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: mdt-lustre-MDT0001_UUID lock: 000000001e6d7318/0xa40083ab9bf534e6 lrc: 0/0,0 mode: --/CR res: [0x240000402:0x7f7:0x0].0x0 bits 0x2/0x0 rrc: 3 type: IBT gid 0 flags: 0x44000000000000 nid: 192.168.204.3@tcp remote: 0x771f68aa1eb27af5 expref: 60 pid: 11281 timeout: 0 lvb_type: 0 00010000:00010000:2.0:1776178155.809339:0:11281:0:(ldlm_lockd.c:1484:ldlm_handle_enqueue0()) ### server-side enqueue handler, sending reply (err=0, rc=0) ns: mdt-lustre-MDT0001_UUID lock: 000000007ec324d4/0xa40083ab9bf534ed lrc: 3/0,0 mode: PR/PR res: [0x240000402:0x7fd:0x0].0x0 bits 0x1b/0x0 rrc: 4 type: IBT gid 0 flags: 0x40200000000000 nid: 192.168.204.3@tcp remote: 0x771f68aa1eb27af5 expref: 59 pid: 11281 timeout: 0 lvb_type: 0 00010000:00010000:2.0:1776178155.809347:0:11281:0:(ldlm_inodebits.c:94:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x240000402:0x7fd:0x0].0x0 (000000008d47e759) 00010000:00010000:2.0:1776178155.809350:0:11281:0:(ldlm_inodebits.c:96:ldlm_reprocess_inodebits_queue()) Hint 1b 00010000:00010000:2.0:1776178155.809353:0:11281:0:(ldlm_lockd.c:1565:ldlm_handle_enqueue0()) ### server-side enqueue handler END (lock 000000007ec324d4, rc 0) 00000100:00100000:2.0:1776178155.809419:0:11281:0:(service.c:2372:ptlrpc_server_handle_request()) Handled RPC req@0000000053e142bc pname:cluuid+ref:pid:xid:nid:opc:job mdt00_004:b50963bd-8eba-4432-9a54-8888d00e3fa1+59:17644:x1862456979093696:12345-192.168.204.3@tcp:101:chmod.0 Request processed in 397us (33443us total) trans 0 rc 0/0 00000100:00100000:2.0:1776178155.809427:0:11281:0:(nrs_fifo.c:241:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.204.3@tcp, seq: 6549 00000100:00100000:2.0:1776178155.809445:0:11281:0:(nrs_fifo.c:179:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.204.3@tcp, seq: 6550 00000100:00100000:2.0:1776178155.809450:0:11281:0:(service.c:2324:ptlrpc_server_handle_request()) Handling RPC req@00000000b821049d pname:cluuid+ref:pid:xid:nid:opc:job mdt00_004:b50963bd-8eba-4432-9a54-8888d00e3fa1+58:17643:x1862456979093760:12345-192.168.204.3@tcp:101:chmod.0 00010000:00010000:2.0:1776178155.809469:0:11281:0:(ldlm_lockd.c:1248:ldlm_handle_enqueue0()) ### server-side enqueue handler START 00010000:00010000:2.0:1776178155.809477:0:11281:0:(ldlm_lockd.c:1318:ldlm_handle_enqueue0()) ### server-side enqueue handler, new lock created ns: mdt-lustre-MDT0001_UUID lock: 000000007ae7b5f6/0xa40083ab9bf534f4 lrc: 2/0,0 mode: --/CR res: [0x240000402:0x7f7:0x0].0x0 bits 0x0/0x0 rrc: 3 type: IBT gid 0 flags: 0x40000000000000 nid: local remote: 0x771f68aa1eb27afc expref: -99 pid: 11281 timeout: 0 lvb_type: 0 00010000:00010000:2.0:1776178155.809503:0:11281:0:(ldlm_lock.c:766:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PR) ns: mdt-lustre-MDT0001_UUID lock: 00000000d3e952db/0xa40083ab9bf534fb lrc: 3/1,0 mode: --/PR res: [0x240000402:0x7fc:0x0].0x0 bits 0x0/0x0 rrc: 4 type: IBT gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 11281 timeout: 0 lvb_type: 0 00010000:00010000:2.0:1776178155.809511:0:11281:0:(ldlm_lock.c:1064:ldlm_granted_list_add_lock()) ### About to add lock: ns: mdt-lustre-MDT0001_UUID lock: 00000000d3e952db/0xa40083ab9bf534fb lrc: 3/1,0 mode: PR/PR res: [0x240000402:0x7fc:0x0].0x0 bits 0x5b/0x0 rrc: 4 type: IBT gid 0 flags: 0x50210000000000 nid: local remote: 0x0 expref: -99 pid: 11281 timeout: 0 lvb_type: 0 00010000:00010000:2.0:1776178155.809517:0:11281:0:(ldlm_request.c:528:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: mdt-lustre-MDT0001_UUID lock: 00000000d3e952db/0xa40083ab9bf534fb lrc: 3/1,0 mode: PR/PR res: [0x240000402:0x7fc:0x0].0x0 bits 0x5b/0x0 rrc: 4 type: IBT gid 0 flags: 0x40210000000000 nid: local remote: 0x0 expref: -99 pid: 11281 timeout: 0 lvb_type: 0 00000004:00200000:2.0:1776178155.809536:0:11281:0:(mdt_handler.c:912:mdt_pack_attr2body()) [0x240000402:0x7fc:0x0]: returning size 0 00000004:00200000:2.0:1776178155.809538:0:11281:0:(mdt_handler.c:1523:mdt_getattr_internal()) dirent count 0 stripe count 1 MDT count 2 00000004:00010000:2.0:1776178155.809546:0:11281:0:(mdt_handler.c:2353:mdt_getattr_name_lock()) ### Returning lock to client ns: mdt-lustre-MDT0001_UUID lock: 00000000d3e952db/0xa40083ab9bf534fb lrc: 3/1,0 mode: PR/PR res: [0x240000402:0x7fc:0x0].0x0 bits 0x5b/0x0 rrc: 4 type: IBT gid 0 flags: 0x40210000000000 nid: local remote: 0x0 expref: -99 pid: 11281 timeout: 0 lvb_type: 0 00010000:00010000:2.0:1776178155.809553:0:11281:0:(ldlm_lock.c:1064:ldlm_granted_list_add_lock()) ### About to add lock: ns: mdt-lustre-MDT0001_UUID lock: 00000000d3e952db/0xa40083ab9bf534fb lrc: 3/1,0 mode: PR/PR res: [0x240000402:0x7fc:0x0].0x0 bits 0x1b/0x0 rrc: 4 type: IBT gid 0 flags: 0x50210000000000 nid: local remote: 0x0 expref: -99 pid: 11281 timeout: 0 lvb_type: 0 00010000:00010000:2.0:1776178155.809565:0:11281:0:(ldlm_lock.c:216:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: mdt-lustre-MDT0001_UUID lock: 000000007ae7b5f6/0xa40083ab9bf534f4 lrc: 0/0,0 mode: --/CR res: [0x240000402:0x7f7:0x0].0x0 bits 0x2/0x0 rrc: 3 type: IBT gid 0 flags: 0x44000000000000 nid: 192.168.204.3@tcp remote: 0x771f68aa1eb27afc expref: 60 pid: 11281 timeout: 0 lvb_type: 0 00010000:00010000:2.0:1776178155.809575:0:11281:0:(ldlm_lockd.c:1484:ldlm_handle_enqueue0()) ### server-side enqueue handler, sending reply (err=0, rc=0) ns: mdt-lustre-MDT0001_UUID lock: 00000000d3e952db/0xa40083ab9bf534fb lrc: 3/0,0 mode: PR/PR res: [0x240000402:0x7fc:0x0].0x0 bits 0x1b/0x0 rrc: 4 type: IBT gid 0 flags: 0x40200000000000 nid: 192.168.204.3@tcp remote: 0x771f68aa1eb27afc expref: 59 pid: 11281 timeout: 0 lvb_type: 0 00010000:00010000:2.0:1776178155.809581:0:11281:0:(ldlm_inodebits.c:94:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x240000402:0x7fc:0x0].0x0 (0000000064abeb1c) 00010000:00010000:2.0:1776178155.809584:0:11281:0:(ldlm_inodebits.c:96:ldlm_reprocess_inodebits_queue()) Hint 1b 00010000:00010000:2.0:1776178155.809586:0:11281:0:(ldlm_lockd.c:1565:ldlm_handle_enqueue0()) ### server-side enqueue handler END (lock 00000000d3e952db, rc 0) 00000100:00100000:2.0:1776178155.809606:0:11281:0:(service.c:2372:ptlrpc_server_handle_request()) Handled RPC req@00000000b821049d pname:cluuid+ref:pid:xid:nid:opc:job mdt00_004:b50963bd-8eba-4432-9a54-8888d00e3fa1+59:17643:x1862456979093760:12345-192.168.204.3@tcp:101:chmod.0 Request processed in 155us (33581us total) trans 0 rc 0/0 00000100:00100000:2.0:1776178155.809612:0:11281:0:(nrs_fifo.c:241:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.204.3@tcp, seq: 6550 00000100:00100000:2.0:1776178155.809638:0:11281:0:(nrs_fifo.c:179:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.204.3@tcp, seq: 6551 00000100:00100000:2.0:1776178155.809642:0:11281:0:(service.c:2324:ptlrpc_server_handle_request()) Handling RPC req@000000005b125149 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_004:b50963bd-8eba-4432-9a54-8888d00e3fa1+58:17645:x1862456979093824:12345-192.168.204.3@tcp:101:chmod.0 00010000:00010000:2.0:1776178155.809650:0:11281:0:(ldlm_lockd.c:1248:ldlm_handle_enqueue0()) ### server-side enqueue handler START 00010000:00010000:2.0:1776178155.809656:0:11281:0:(ldlm_lockd.c:1318:ldlm_handle_enqueue0()) ### server-side enqueue handler, new lock created ns: mdt-lustre-MDT0001_UUID lock: 00000000fcb32840/0xa40083ab9bf53502 lrc: 2/0,0 mode: --/CR res: [0x240000402:0x7f7:0x0].0x0 bits 0x0/0x0 rrc: 3 type: IBT gid 0 flags: 0x40000000000000 nid: local remote: 0x771f68aa1eb27b03 expref: -99 pid: 11281 timeout: 0 lvb_type: 0 00010000:00010000:2.0:1776178155.809680:0:11281:0:(ldlm_lock.c:766:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PR) ns: mdt-lustre-MDT0001_UUID lock: 000000007f7ce364/0xa40083ab9bf53509 lrc: 3/1,0 mode: --/PR res: [0x240000402:0x7fe:0x0].0x0 bits 0x0/0x0 rrc: 4 type: IBT gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 11281 timeout: 0 lvb_type: 0 00010000:00010000:2.0:1776178155.809688:0:11281:0:(ldlm_lock.c:1064:ldlm_granted_list_add_lock()) ### About to add lock: ns: mdt-lustre-MDT0001_UUID lock: 000000007f7ce364/0xa40083ab9bf53509 lrc: 3/1,0 mode: PR/PR res: [0x240000402:0x7fe:0x0].0x0 bits 0x5b/0x0 rrc: 4 type: IBT gid 0 flags: 0x50210000000000 nid: local remote: 0x0 expref: -99 pid: 11281 timeout: 0 lvb_type: 0 00010000:00010000:2.0:1776178155.809694:0:11281:0:(ldlm_request.c:528:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: mdt-lustre-MDT0001_UUID lock: 000000007f7ce364/0xa40083ab9bf53509 lrc: 3/1,0 mode: PR/PR res: [0x240000402:0x7fe:0x0].0x0 bits 0x5b/0x0 rrc: 4 type: IBT gid 0 flags: 0x40210000000000 nid: local remote: 0x0 expref: -99 pid: 11281 timeout: 0 lvb_type: 0 00000004:00200000:2.0:1776178155.809712:0:11281:0:(mdt_handler.c:912:mdt_pack_attr2body()) [0x240000402:0x7fe:0x0]: returning size 0 00000004:00200000:2.0:1776178155.809714:0:11281:0:(mdt_handler.c:1523:mdt_getattr_internal()) dirent count 0 stripe count 1 MDT count 2 00000004:00010000:2.0:1776178155.809722:0:11281:0:(mdt_handler.c:2353:mdt_getattr_name_lock()) ### Returning lock to client ns: mdt-lustre-MDT0001_UUID lock: 000000007f7ce364/0xa40083ab9bf53509 lrc: 3/1,0 mode: PR/PR res: [0x240000402:0x7fe:0x0].0x0 bits 0x5b/0x0 rrc: 4 type: IBT gid 0 flags: 0x40210000000000 nid: local remote: 0x0 expref: -99 pid: 11281 timeout: 0 lvb_type: 0 00010000:00010000:2.0:1776178155.809729:0:11281:0:(ldlm_lock.c:1064:ldlm_granted_list_add_lock()) ### About to add lock: ns: mdt-lustre-MDT0001_UUID lock: 000000007f7ce364/0xa40083ab9bf53509 lrc: 3/1,0 mode: PR/PR res: [0x240000402:0x7fe:0x0].0x0 bits 0x1b/0x0 rrc: 4 type: IBT gid 0 flags: 0x50210000000000 nid: local remote: 0x0 expref: -99 pid: 11281 timeout: 0 lvb_type: 0 00010000:00010000:2.0:1776178155.809741:0:11281:0:(ldlm_lock.c:216:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: mdt-lustre-MDT0001_UUID lock: 00000000fcb32840/0xa40083ab9bf53502 lrc: 0/0,0 mode: --/CR res: [0x240000402:0x7f7:0x0].0x0 bits 0x2/0x0 rrc: 3 type: IBT gid 0 flags: 0x44000000000000 nid: 192.168.204.3@tcp remote: 0x771f68aa1eb27b03 expref: 60 pid: 11281 timeout: 0 lvb_type: 0 00010000:00010000:2.0:1776178155.809758:0:11281:0:(ldlm_lockd.c:1484:ldlm_handle_enqueue0()) ### server-side enqueue handler, sending reply (err=0, rc=0) ns: mdt-lustre-MDT0001_UUID lock: 000000007f7ce364/0xa40083ab9bf53509 lrc: 3/0,0 mode: PR/PR res: [0x240000402:0x7fe:0x0].0x0 bits 0x1b/0x0 rrc: 4 type: IBT gid 0 flags: 0x40200000000000 nid: 192.168.204.3@tcp remote: 0x771f68aa1eb27b03 expref: 59 pid: 11281 timeout: 0 lvb_type: 0 00010000:00010000:2.0:1776178155.809763:0:11281:0:(ldlm_inodebits.c:94:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x240000402:0x7fe:0x0].0x0 (00000000498a007f) 00010000:00010000:2.0:1776178155.809765:0:11281:0:(ldlm_inodebits.c:96:ldlm_reprocess_inodebits_queue()) Hint 1b 00010000:00010000:2.0:1776178155.809768:0:11281:0:(ldlm_lockd.c:1565:ldlm_handle_enqueue0()) ### server-side enqueue handler END (lock 000000007f7ce364, rc 0) 00000100:00100000:2.0:1776178155.809787:0:11281:0:(service.c:2372:ptlrpc_server_handle_request()) Handled RPC req@000000005b125149 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_004:b50963bd-8eba-4432-9a54-8888d00e3fa1+59:17645:x1862456979093824:12345-192.168.204.3@tcp:101:chmod.0 Request processed in 143us (33620us total) trans 0 rc 0/0 00000100:00100000:2.0:1776178155.809793:0:11281:0:(nrs_fifo.c:241:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.204.3@tcp, seq: 6551 00000100:00100000:1.0:1776178155.811876:0:6255:0:(service.c:2152:ptlrpc_server_handle_req_in()) got req x1862456979094080 00000100:00100000:1.0:1776178155.811935:0:6255:0:(nrs_fifo.c:179:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.204.3@tcp, seq: 6552 00000100:00100000:1.0:1776178155.811942:0:6255:0:(service.c:2324:ptlrpc_server_handle_request()) Handling RPC req@0000000078c1d864 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_000:b50963bd-8eba-4432-9a54-8888d00e3fa1+59:17639:x1862456979094080:12345-192.168.204.3@tcp:36:chmod.0 00010000:00010000:1.0:1776178155.811992:0:6255:0:(ldlm_lockd.c:1720:ldlm_request_cancel()) ### server-side cancel handler START: 2 locks, starting at 0 00010000:00010000:1.0:1776178155.812017:0:6255:0:(ldlm_lock.c:216:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: mdt-lustre-MDT0001_UUID lock: 0000000031bb7277/0xa40083ab9bf534ae lrc: 0/0,0 mode: --/PR res: [0x240000402:0x7f8:0x0].0x0 bits 0x1b/0x0 rrc: 5 type: IBT gid 0 flags: 0x44a01000000000 nid: 192.168.204.3@tcp remote: 0x771f68aa1eb27ad2 expref: 59 pid: 6256 timeout: 0 lvb_type: 0 00010000:00010000:1.0:1776178155.812497:0:6255:0:(ldlm_lock.c:216:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: mdt-lustre-MDT0001_UUID lock: 0000000055d6a00d/0xa40083ab9bf53396 lrc: 0/0,0 mode: --/CR res: [0x240000402:0x7f8:0x0].0x0 bits 0x9/0x0 rrc: 4 type: IBT gid 0 flags: 0x44a01000000000 nid: 192.168.204.3@tcp remote: 0x771f68aa1eb27aa1 expref: 58 pid: 8250 timeout: 0 lvb_type: 0 00010000:00010000:1.0:1776178155.812510:0:6255:0:(ldlm_inodebits.c:94:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x240000402:0x7f8:0x0].0x0 (00000000099618da) 00010000:00010000:1.0:1776178155.812516:0:6255:0:(ldlm_lockd.c:1782:ldlm_request_cancel()) ### server-side cancel handler END 00010000:00010000:1.0:1776178155.812550:0:6255:0:(ldlm_lock.c:766:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: mdt-lustre-MDT0001_UUID lock: 000000006db4ec1d/0xa40083ab9bf53510 lrc: 3/0,1 mode: --/PW res: [0x240000402:0x7f8:0x0].0x0 bits 0x0/0x0 rrc: 3 type: IBT gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 6255 timeout: 0 lvb_type: 0 00010000:00010000:1.0:1776178155.812565:0:6255:0:(ldlm_lock.c:1064:ldlm_granted_list_add_lock()) ### About to add lock: ns: mdt-lustre-MDT0001_UUID lock: 000000006db4ec1d/0xa40083ab9bf53510 lrc: 3/0,1 mode: PW/PW res: [0x240000402:0x7f8:0x0].0x0 bits 0x13/0x0 rrc: 3 type: IBT gid 0 flags: 0x50210000000000 nid: local remote: 0x0 expref: -99 pid: 6255 timeout: 0 lvb_type: 0 00010000:00010000:1.0:1776178155.812575:0:6255:0:(ldlm_request.c:528:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: mdt-lustre-MDT0001_UUID lock: 000000006db4ec1d/0xa40083ab9bf53510 lrc: 3/0,1 mode: PW/PW res: [0x240000402:0x7f8:0x0].0x0 bits 0x13/0x0 rrc: 3 type: IBT gid 0 flags: 0x40210000000000 nid: local remote: 0x0 expref: -99 pid: 6255 timeout: 0 lvb_type: 0 00000004:00010000:1.0:1776178155.812742:0:6255:0:(mdt_handler.c:3992:mdt_save_lock()) ### save lock request 0000000078c1d864 reply state 000000005dbdbd63 transno 17179876257 ns: mdt-lustre-MDT0001_UUID lock: 000000006db4ec1d/0xa40083ab9bf53510 lrc: 3/0,1 mode: PW/PW res: [0x240000402:0x7f8:0x0].0x0 bits 0x13/0x0 rrc: 3 type: IBT gid 0 flags: 0x40210000000000 nid: local remote: 0x0 expref: -99 pid: 6255 timeout: 0 lvb_type: 0 00010000:02000000:1.0:1776178155.812797:0:6255:0:(libcfs_fail.h:96:cfs_fail_check_set()) *** cfs_fail_loc=15a, val=0*** 00000100:00100000:0.0:1776178155.814273:0:20762:0:(service.c:2152:ptlrpc_server_handle_req_in()) got req x1862456979093888 00000100:00100000:0.0:1776178155.814335:0:20762:0:(service.c:2152:ptlrpc_server_handle_req_in()) got req x1862456979093952 00000100:00100000:0.0:1776178155.814357:0:20762:0:(nrs_fifo.c:179:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.204.3@tcp, seq: 5265 00000100:00100000:0.0:1776178155.814363:0:20762:0:(service.c:2324:ptlrpc_server_handle_request()) Handling RPC req@000000005accf721 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_013:b50963bd-8eba-4432-9a54-8888d00e3fa1+9:2271:x1862456979093888:12345-192.168.204.3@tcp:101:chmod.0 00010000:00010000:0.0:1776178155.814384:0:20762:0:(ldlm_lockd.c:1248:ldlm_handle_enqueue0()) ### server-side enqueue handler START 00010000:00010000:0.0:1776178155.814409:0:20762:0:(ldlm_lockd.c:1318:ldlm_handle_enqueue0()) ### server-side enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 00000000f51fd0d7/0xa40083ab9bf53517 lrc: 2/0,0 mode: --/PR res: [0x280000400:0x46f:0x0].0x0 rrc: 2 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x771f68aa1eb27b0a expref: -99 pid: 20762 timeout: 0 lvb_type: 0 00002000:00010000:0.0:1776178155.814432:0:20762:0:(ofd_lvb.c:170:ofd_lvbo_init()) res: [0x280000400:0x46f:0x0] initial LVB size: 0, mtime: 0x69de53e9, atime: 0x69de53e9, ctime: 0x69de53e9, blocks: 0x0 00010000:00010000:0.0:1776178155.814454:0:20762:0:(ldlm_extent.c:318:ldlm_check_contention()) contended locks = 0 00010000:00010000:0.0:1776178155.814457:0:20762:0:(ldlm_extent.c:318:ldlm_check_contention()) contended locks = 0 00010000:00010000:0.0:1776178155.814467:0:20762:0:(ldlm_resource.c:1650:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 00000000f51fd0d7/0xa40083ab9bf53517 lrc: 3/0,0 mode: PR/PR res: [0x280000400:0x46f:0x0].0x0 rrc: 2 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x40000000000000 nid: 192.168.204.3@tcp remote: 0x771f68aa1eb27b0a expref: 10 pid: 20762 timeout: 0 lvb_type: 1 00010000:00010000:0.0:1776178155.814492:0:20762:0:(ldlm_lockd.c:1484:ldlm_handle_enqueue0()) ### server-side enqueue handler, sending reply (err=0, rc=0) ns: filter-lustre-OST0000_UUID lock: 00000000f51fd0d7/0xa40083ab9bf53517 lrc: 3/0,0 mode: PR/PR res: [0x280000400:0x46f:0x0].0x0 rrc: 2 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x40000000000000 nid: 192.168.204.3@tcp remote: 0x771f68aa1eb27b0a expref: 10 pid: 20762 timeout: 0 lvb_type: 1 00010000:00010000:0.0:1776178155.814507:0:20762:0:(ldlm_lockd.c:1565:ldlm_handle_enqueue0()) ### server-side enqueue handler END (lock 00000000f51fd0d7, rc 0) 00000100:00100000:0.0:1776178155.814575:0:20762:0:(service.c:2372:ptlrpc_server_handle_request()) Handled RPC req@000000005accf721 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_013:b50963bd-8eba-4432-9a54-8888d00e3fa1+10:2271:x1862456979093888:12345-192.168.204.3@tcp:101:chmod.0 Request processed in 210us (38429us total) trans 0 rc 0/0 00000100:00100000:0.0:1776178155.814582:0:20762:0:(nrs_fifo.c:241:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.204.3@tcp, seq: 5265 00000100:00100000:0.0:1776178155.814600:0:20762:0:(nrs_fifo.c:179:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.204.3@tcp, seq: 5266 00000100:00100000:0.0:1776178155.814605:0:20762:0:(service.c:2324:ptlrpc_server_handle_request()) Handling RPC req@000000005fdea934 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_013:b50963bd-8eba-4432-9a54-8888d00e3fa1+9:2272:x1862456979093952:12345-192.168.204.3@tcp:101:chmod.0 00010000:00010000:0.0:1776178155.814612:0:20762:0:(ldlm_lockd.c:1248:ldlm_handle_enqueue0()) ### server-side enqueue handler START 00010000:00010000:0.0:1776178155.814635:0:20762:0:(ldlm_lockd.c:1318:ldlm_handle_enqueue0()) ### server-side enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 000000006b206cc9/0xa40083ab9bf5351e lrc: 2/0,0 mode: --/PR res: [0x280000400:0x46e:0x0].0x0 rrc: 2 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x771f68aa1eb27b11 expref: -99 pid: 20762 timeout: 0 lvb_type: 0 00002000:00010000:0.0:1776178155.814646:0:20762:0:(ofd_lvb.c:170:ofd_lvbo_init()) res: [0x280000400:0x46e:0x0] initial LVB size: 0, mtime: 0x69de53e9, atime: 0x69de53e9, ctime: 0x69de53e9, blocks: 0x0 00010000:00010000:0.0:1776178155.814658:0:20762:0:(ldlm_extent.c:318:ldlm_check_contention()) contended locks = 0 00010000:00010000:0.0:1776178155.814659:0:20762:0:(ldlm_extent.c:318:ldlm_check_contention()) contended locks = 0 00010000:00010000:0.0:1776178155.814662:0:20762:0:(ldlm_resource.c:1650:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 000000006b206cc9/0xa40083ab9bf5351e lrc: 3/0,0 mode: PR/PR res: [0x280000400:0x46e:0x0].0x0 rrc: 2 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x40000000000000 nid: 192.168.204.3@tcp remote: 0x771f68aa1eb27b11 expref: 10 pid: 20762 timeout: 0 lvb_type: 1 00010000:00010000:0.0:1776178155.814671:0:20762:0:(ldlm_lockd.c:1484:ldlm_handle_enqueue0()) ### server-side enqueue handler, sending reply (err=0, rc=0) ns: filter-lustre-OST0000_UUID lock: 000000006b206cc9/0xa40083ab9bf5351e lrc: 3/0,0 mode: PR/PR res: [0x280000400:0x46e:0x0].0x0 rrc: 2 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x40000000000000 nid: 192.168.204.3@tcp remote: 0x771f68aa1eb27b11 expref: 10 pid: 20762 timeout: 0 lvb_type: 1 00010000:00010000:0.0:1776178155.814679:0:20762:0:(ldlm_lockd.c:1565:ldlm_handle_enqueue0()) ### server-side enqueue handler END (lock 000000006b206cc9, rc 0) 00000100:00100000:0.0:1776178155.814697:0:20762:0:(service.c:2372:ptlrpc_server_handle_request()) Handled RPC req@000000005fdea934 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_013:b50963bd-8eba-4432-9a54-8888d00e3fa1+10:2272:x1862456979093952:12345-192.168.204.3@tcp:101:chmod.0 Request processed in 92us (27039us total) trans 0 rc 0/0 00000100:00100000:0.0:1776178155.814703:0:20762:0:(nrs_fifo.c:241:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.204.3@tcp, seq: 5266 00000100:00100000:0.0:1776178155.814760:0:3348:0:(events.c:361:request_in_callback()) peer: 12345-192.168.204.3@tcp (source: 12345-192.168.204.3@tcp) 00000100:00100000:0.0:1776178155.814787:0:20762:0:(service.c:2152:ptlrpc_server_handle_req_in()) got req x1862456979094144 00000100:00100000:0.0:1776178155.814806:0:20762:0:(nrs_fifo.c:179:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.204.3@tcp, seq: 5267 00000100:00100000:0.0:1776178155.814810:0:20762:0:(service.c:2324:ptlrpc_server_handle_request()) Handling RPC req@000000000c9fb390 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_013:b50963bd-8eba-4432-9a54-8888d00e3fa1+10:2270:x1862456979094144:12345-192.168.204.3@tcp:101:chmod.0 00010000:00010000:0.0:1776178155.814818:0:20762:0:(ldlm_lockd.c:1248:ldlm_handle_enqueue0()) ### server-side enqueue handler START 00010000:00010000:0.0:1776178155.814830:0:20762:0:(ldlm_lockd.c:1318:ldlm_handle_enqueue0()) ### server-side enqueue handler, new lock created ns: filter-lustre-OST0001_UUID lock: 00000000d999afdf/0xa40083ab9bf53525 lrc: 2/0,0 mode: --/PR res: [0x2c0000400:0x42f:0x0].0x0 rrc: 2 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x771f68aa1eb27b18 expref: -99 pid: 20762 timeout: 0 lvb_type: 0 00002000:00010000:0.0:1776178155.814843:0:20762:0:(ofd_lvb.c:170:ofd_lvbo_init()) res: [0x2c0000400:0x42f:0x0] initial LVB size: 0, mtime: 0x69de53e9, atime: 0x69de53e9, ctime: 0x69de53e9, blocks: 0x0 00010000:00010000:0.0:1776178155.814856:0:20762:0:(ldlm_extent.c:318:ldlm_check_contention()) contended locks = 0 00010000:00010000:0.0:1776178155.814857:0:20762:0:(ldlm_extent.c:318:ldlm_check_contention()) contended locks = 0 00010000:00010000:0.0:1776178155.814861:0:20762:0:(ldlm_resource.c:1650:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0001_UUID lock: 00000000d999afdf/0xa40083ab9bf53525 lrc: 3/0,0 mode: PR/PR res: [0x2c0000400:0x42f:0x0].0x0 rrc: 2 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x40000000000000 nid: 192.168.204.3@tcp remote: 0x771f68aa1eb27b18 expref: 11 pid: 20762 timeout: 0 lvb_type: 1 00010000:00010000:0.0:1776178155.814871:0:20762:0:(ldlm_lockd.c:1484:ldlm_handle_enqueue0()) ### server-side enqueue handler, sending reply (err=0, rc=0) ns: filter-lustre-OST0001_UUID lock: 00000000d999afdf/0xa40083ab9bf53525 lrc: 3/0,0 mode: PR/PR res: [0x2c0000400:0x42f:0x0].0x0 rrc: 2 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x40000000000000 nid: 192.168.204.3@tcp remote: 0x771f68aa1eb27b18 expref: 11 pid: 20762 timeout: 0 lvb_type: 1 00010000:00010000:0.0:1776178155.814879:0:20762:0:(ldlm_lockd.c:1565:ldlm_handle_enqueue0()) ### server-side enqueue handler END (lock 00000000d999afdf, rc 0) 00000100:00100000:0.0:1776178155.814898:0:20762:0:(service.c:2372:ptlrpc_server_handle_request()) Handled RPC req@000000000c9fb390 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_013:b50963bd-8eba-4432-9a54-8888d00e3fa1+11:2270:x1862456979094144:12345-192.168.204.3@tcp:101:chmod.0 Request processed in 87us (140us total) trans 0 rc 0/0 00000100:00100000:0.0:1776178155.814904:0:20762:0:(nrs_fifo.c:241:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.204.3@tcp, seq: 5267 00000100:00100000:0.0:1776178155.814971:0:3348:0:(events.c:361:request_in_callback()) peer: 12345-192.168.204.3@tcp (source: 12345-192.168.204.3@tcp) 00000100:00100000:0.0:1776178155.815023:0:3348:0:(events.c:361:request_in_callback()) peer: 12345-192.168.204.3@tcp (source: 12345-192.168.204.3@tcp) 00000100:00100000:0.0:1776178155.815046:0:20140:0:(service.c:2152:ptlrpc_server_handle_req_in()) got req x1862456979094208 00000100:00100000:0.0:1776178155.815099:0:20140:0:(nrs_fifo.c:179:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.204.3@tcp, seq: 137 00000100:00100000:0.0:1776178155.815103:0:20140:0:(service.c:2324:ptlrpc_server_handle_request()) Handling RPC req@00000000322375a6 pname:cluuid+ref:pid:xid:nid:opc:job ll_mgs_0002:5b5c66b1-1a2f-4651-8548-0c75b9a38a94+9:2270:x1862456979094208:12345-192.168.204.3@tcp:400:kworker/u8:3.0 00000100:00100000:0.0:1776178155.815131:0:20140:0:(service.c:2372:ptlrpc_server_handle_request()) Handled RPC req@00000000322375a6 pname:cluuid+ref:pid:xid:nid:opc:job ll_mgs_0002:5b5c66b1-1a2f-4651-8548-0c75b9a38a94+9:2270:x1862456979094208:12345-192.168.204.3@tcp:400:kworker/u8:3.0 Request processed in 27us (109us total) trans 0 rc 0/0 00000100:00100000:0.0:1776178155.815137:0:20140:0:(nrs_fifo.c:241:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.204.3@tcp, seq: 137 00000100:00100000:2.0:1776178155.819640:0:11281:0:(service.c:2152:ptlrpc_server_handle_req_in()) got req x1862456979094272 00000100:00100000:2.0:1776178155.819704:0:11281:0:(nrs_fifo.c:179:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.204.3@tcp, seq: 835 00000100:00100000:2.0:1776178155.819712:0:11281:0:(service.c:2324:ptlrpc_server_handle_request()) Handling RPC req@000000005927b2c3 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_004:b50963bd-8eba-4432-9a54-8888d00e3fa1+7:2270:x1862456979094272:12345-192.168.204.3@tcp:400:kworker/u8:3.0 00000100:00100000:2.0:1776178155.820041:0:11281:0:(service.c:2372:ptlrpc_server_handle_request()) Handled RPC req@000000005927b2c3 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_004:b50963bd-8eba-4432-9a54-8888d00e3fa1+7:2270:x1862456979094272:12345-192.168.204.3@tcp:400:kworker/u8:3.0 Request processed in 328us (5069us total) trans 0 rc 0/0 00000100:00100000:2.0:1776178155.820049:0:11281:0:(nrs_fifo.c:241:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.204.3@tcp, seq: 835 00000100:00100000:0.0:1776178155.821562:0:3348:0:(events.c:361:request_in_callback()) peer: 12345-192.168.204.3@tcp (source: 12345-192.168.204.3@tcp) 00000100:00100000:0.0:1776178155.821591:0:3348:0:(events.c:361:request_in_callback()) peer: 12345-192.168.204.3@tcp (source: 12345-192.168.204.3@tcp) 00000100:00100000:0.0:1776178155.821664:0:3348:0:(events.c:361:request_in_callback()) peer: 12345-192.168.204.3@tcp (source: 12345-192.168.204.3@tcp) 00000100:00100000:2.0:1776178155.822169:0:20769:0:(service.c:2152:ptlrpc_server_handle_req_in()) got req x1862456979094336 00000100:00100000:2.0:1776178155.822209:0:20769:0:(service.c:2152:ptlrpc_server_handle_req_in()) got req x1862456979094400 00000100:00100000:2.0:1776178155.822403:0:20769:0:(service.c:2152:ptlrpc_server_handle_req_in()) got req x1862456979094464 00010000:00080000:1.0:1776178155.825583:0:6255:0:(ldlm_lib.c:3333:target_send_reply()) Schedule reply immediately 00000100:00100000:1.0:1776178155.825641:0:6255:0:(service.c:2372:ptlrpc_server_handle_request()) Handled RPC req@0000000078c1d864 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_000:b50963bd-8eba-4432-9a54-8888d00e3fa1+59:17639:x1862456979094080:12345-192.168.204.3@tcp:36:chmod.0 Request processed in 13691us (37931us total) trans 17179876257 rc 0/0 00000100:00100000:1.0:1776178155.825657:0:6255:0:(nrs_fifo.c:241:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.204.3@tcp, seq: 6552 00000100:00100000:1.0:1776178155.826040:0:20759:0:(nrs_fifo.c:179:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.204.3@tcp, seq: 5268 00000100:00100000:1.0:1776178155.826050:0:20759:0:(service.c:2324:ptlrpc_server_handle_request()) Handling RPC req@00000000f0ef812c pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_010:b50963bd-8eba-4432-9a54-8888d00e3fa1+10:2270:x1862456979094336:12345-192.168.204.3@tcp:101:chmod.0 00010000:00010000:1.0:1776178155.826109:0:20759:0:(ldlm_lockd.c:1248:ldlm_handle_enqueue0()) ### server-side enqueue handler START 00010000:00010000:1.0:1776178155.826138:0:20759:0:(ldlm_lockd.c:1318:ldlm_handle_enqueue0()) ### server-side enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 00000000a1e024a5/0xa40083ab9bf5352c lrc: 2/0,0 mode: --/PR res: [0x280000400:0x470:0x0].0x0 rrc: 2 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x771f68aa1eb27b1f expref: -99 pid: 20759 timeout: 0 lvb_type: 0 00002000:00010000:1.0:1776178155.826163:0:20759:0:(ofd_lvb.c:170:ofd_lvbo_init()) res: [0x280000400:0x470:0x0] initial LVB size: 0, mtime: 0x69de53ea, atime: 0x69de53ea, ctime: 0x69de53ea, blocks: 0x0 00010000:00010000:1.0:1776178155.826184:0:20759:0:(ldlm_extent.c:318:ldlm_check_contention()) contended locks = 0 00010000:00010000:1.0:1776178155.826186:0:20759:0:(ldlm_extent.c:318:ldlm_check_contention()) contended locks = 0 00010000:00010000:1.0:1776178155.826196:0:20759:0:(ldlm_resource.c:1650:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 00000000a1e024a5/0xa40083ab9bf5352c lrc: 3/0,0 mode: PR/PR res: [0x280000400:0x470:0x0].0x0 rrc: 2 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x40000000000000 nid: 192.168.204.3@tcp remote: 0x771f68aa1eb27b1f expref: 11 pid: 20759 timeout: 0 lvb_type: 1 00010000:00010000:1.0:1776178155.826213:0:20759:0:(ldlm_lockd.c:1484:ldlm_handle_enqueue0()) ### server-side enqueue handler, sending reply (err=0, rc=0) ns: filter-lustre-OST0000_UUID lock: 00000000a1e024a5/0xa40083ab9bf5352c lrc: 3/0,0 mode: PR/PR res: [0x280000400:0x470:0x0].0x0 rrc: 2 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x40000000000000 nid: 192.168.204.3@tcp remote: 0x771f68aa1eb27b1f expref: 11 pid: 20759 timeout: 0 lvb_type: 1 00010000:00010000:1.0:1776178155.826226:0:20759:0:(ldlm_lockd.c:1565:ldlm_handle_enqueue0()) ### server-side enqueue handler END (lock 00000000a1e024a5, rc 0) 00000100:00100000:1.0:1776178155.826292:0:20759:0:(service.c:2372:ptlrpc_server_handle_request()) Handled RPC req@00000000f0ef812c pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_010:b50963bd-8eba-4432-9a54-8888d00e3fa1+11:2270:x1862456979094336:12345-192.168.204.3@tcp:101:chmod.0 Request processed in 242us (4732us total) trans 0 rc 0/0 00000100:00100000:1.0:1776178155.826297:0:20759:0:(nrs_fifo.c:241:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.204.3@tcp, seq: 5268 00000100:00100000:1.0:1776178155.826326:0:20759:0:(nrs_fifo.c:179:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.204.3@tcp, seq: 5269 00000100:00100000:1.0:1776178155.826333:0:20759:0:(service.c:2324:ptlrpc_server_handle_request()) Handling RPC req@000000007307bd05 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_010:b50963bd-8eba-4432-9a54-8888d00e3fa1+12:2271:x1862456979094400:12345-192.168.204.3@tcp:101:chmod.0 00010000:00010000:1.0:1776178155.826343:0:20759:0:(ldlm_lockd.c:1248:ldlm_handle_enqueue0()) ### server-side enqueue handler START 00010000:00010000:1.0:1776178155.826354:0:20759:0:(ldlm_lockd.c:1318:ldlm_handle_enqueue0()) ### server-side enqueue handler, new lock created ns: filter-lustre-OST0001_UUID lock: 000000003e1697a2/0xa40083ab9bf53533 lrc: 2/0,0 mode: --/PR res: [0x2c0000400:0x431:0x0].0x0 rrc: 2 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x771f68aa1eb27b26 expref: -99 pid: 20759 timeout: 0 lvb_type: 0 00002000:00010000:1.0:1776178155.826369:0:20759:0:(ofd_lvb.c:170:ofd_lvbo_init()) res: [0x2c0000400:0x431:0x0] initial LVB size: 0, mtime: 0x69de53ea, atime: 0x69de53ea, ctime: 0x69de53ea, blocks: 0x0 00010000:00010000:1.0:1776178155.826377:0:20759:0:(ldlm_extent.c:318:ldlm_check_contention()) contended locks = 0 00010000:00010000:1.0:1776178155.826379:0:20759:0:(ldlm_extent.c:318:ldlm_check_contention()) contended locks = 0 00010000:00010000:1.0:1776178155.826382:0:20759:0:(ldlm_resource.c:1650:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0001_UUID lock: 000000003e1697a2/0xa40083ab9bf53533 lrc: 3/0,0 mode: PR/PR res: [0x2c0000400:0x431:0x0].0x0 rrc: 2 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x40000000000000 nid: 192.168.204.3@tcp remote: 0x771f68aa1eb27b26 expref: 13 pid: 20759 timeout: 0 lvb_type: 1 00010000:00010000:1.0:1776178155.826391:0:20759:0:(ldlm_lockd.c:1484:ldlm_handle_enqueue0()) ### server-side enqueue handler, sending reply (err=0, rc=0) ns: filter-lustre-OST0001_UUID lock: 000000003e1697a2/0xa40083ab9bf53533 lrc: 3/0,0 mode: PR/PR res: [0x2c0000400:0x431:0x0].0x0 rrc: 2 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x40000000000000 nid: 192.168.204.3@tcp remote: 0x771f68aa1eb27b26 expref: 13 pid: 20759 timeout: 0 lvb_type: 1 00010000:00010000:1.0:1776178155.826400:0:20759:0:(ldlm_lockd.c:1565:ldlm_handle_enqueue0()) ### server-side enqueue handler END (lock 000000003e1697a2, rc 0) 00000100:00100000:1.0:1776178155.826418:0:20759:0:(service.c:2372:ptlrpc_server_handle_request()) Handled RPC req@000000007307bd05 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_010:b50963bd-8eba-4432-9a54-8888d00e3fa1+13:2271:x1862456979094400:12345-192.168.204.3@tcp:101:chmod.0 Request processed in 86us (4827us total) trans 0 rc 0/0 00000100:00100000:1.0:1776178155.826424:0:20759:0:(nrs_fifo.c:241:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.204.3@tcp, seq: 5269 00000100:00100000:1.0:1776178155.826437:0:20759:0:(nrs_fifo.c:179:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.204.3@tcp, seq: 5270 00000100:00100000:1.0:1776178155.826440:0:20759:0:(service.c:2324:ptlrpc_server_handle_request()) Handling RPC req@00000000cd102128 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_010:b50963bd-8eba-4432-9a54-8888d00e3fa1+12:2273:x1862456979094464:12345-192.168.204.3@tcp:101:chmod.0 00010000:00010000:1.0:1776178155.826448:0:20759:0:(ldlm_lockd.c:1248:ldlm_handle_enqueue0()) ### server-side enqueue handler START 00010000:00010000:1.0:1776178155.826460:0:20759:0:(ldlm_lockd.c:1318:ldlm_handle_enqueue0()) ### server-side enqueue handler, new lock created ns: filter-lustre-OST0001_UUID lock: 000000002abc4399/0xa40083ab9bf5353a lrc: 2/0,0 mode: --/PR res: [0x2c0000400:0x430:0x0].0x0 rrc: 2 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x771f68aa1eb27b2d expref: -99 pid: 20759 timeout: 0 lvb_type: 0 00002000:00010000:1.0:1776178155.826472:0:20759:0:(ofd_lvb.c:170:ofd_lvbo_init()) res: [0x2c0000400:0x430:0x0] initial LVB size: 0, mtime: 0x69de53ea, atime: 0x69de53ea, ctime: 0x69de53ea, blocks: 0x0 00010000:00010000:1.0:1776178155.826481:0:20759:0:(ldlm_extent.c:318:ldlm_check_contention()) contended locks = 0 00010000:00010000:1.0:1776178155.826483:0:20759:0:(ldlm_extent.c:318:ldlm_check_contention()) contended locks = 0 00010000:00010000:1.0:1776178155.826486:0:20759:0:(ldlm_resource.c:1650:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0001_UUID lock: 000000002abc4399/0xa40083ab9bf5353a lrc: 3/0,0 mode: PR/PR res: [0x2c0000400:0x430:0x0].0x0 rrc: 2 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x40000000000000 nid: 192.168.204.3@tcp remote: 0x771f68aa1eb27b2d expref: 13 pid: 20759 timeout: 0 lvb_type: 1 00010000:00010000:1.0:1776178155.826494:0:20759:0:(ldlm_lockd.c:1484:ldlm_handle_enqueue0()) ### server-side enqueue handler, sending reply (err=0, rc=0) ns: filter-lustre-OST0001_UUID lock: 000000002abc4399/0xa40083ab9bf5353a lrc: 3/0,0 mode: PR/PR res: [0x2c0000400:0x430:0x0].0x0 rrc: 2 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x40000000000000 nid: 192.168.204.3@tcp remote: 0x771f68aa1eb27b2d expref: 13 pid: 20759 timeout: 0 lvb_type: 1 00010000:00010000:1.0:1776178155.826501:0:20759:0:(ldlm_lockd.c:1565:ldlm_handle_enqueue0()) ### server-side enqueue handler END (lock 000000002abc4399, rc 0) 00000100:00100000:1.0:1776178155.826523:0:20759:0:(service.c:2372:ptlrpc_server_handle_request()) Handled RPC req@00000000cd102128 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_010:b50963bd-8eba-4432-9a54-8888d00e3fa1+13:2273:x1862456979094464:12345-192.168.204.3@tcp:101:chmod.0 Request processed in 81us (4858us total) trans 0 rc 0/0 00000100:00100000:1.0:1776178155.826528:0:20759:0:(nrs_fifo.c:241:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.204.3@tcp, seq: 5270 00010000:00010000:3.0:1776178155.827163:0:3339:0:(ldlm_lock.c:1064:ldlm_granted_list_add_lock()) ### About to add lock: ns: mdt-lustre-MDT0001_UUID lock: 000000006db4ec1d/0xa40083ab9bf53510 lrc: 3/0,1 mode: COS/COS res: [0x240000402:0x7f8:0x0].0x0 bits 0x13/0x0 rrc: 3 type: IBT gid 0 flags: 0x50210000000000 nid: local remote: 0x0 expref: -99 pid: 6255 timeout: 0 lvb_type: 0 00010000:00010000:3.0:1776178155.827175:0:3339:0:(ldlm_inodebits.c:94:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x240000402:0x7f8:0x0].0x0 (00000000099618da) 00010000:00010000:3.0:1776178155.827178:0:3339:0:(ldlm_inodebits.c:96:ldlm_reprocess_inodebits_queue()) Hint 13 00000100:00100000:3.0:1776178155.827241:0:3348:0:(events.c:361:request_in_callback()) peer: 12345-192.168.204.3@tcp (source: 12345-192.168.204.3@tcp) 00000100:00100000:3.0:1776178155.827299:0:3348:0:(events.c:361:request_in_callback()) peer: 12345-192.168.204.3@tcp (source: 12345-192.168.204.3@tcp) 00000100:00100000:3.0:1776178155.827328:0:3348:0:(events.c:361:request_in_callback()) peer: 12345-192.168.204.3@tcp (source: 12345-192.168.204.3@tcp) 00000100:00100000:1.0:1776178155.827876:0:8250:0:(service.c:2152:ptlrpc_server_handle_req_in()) got req x1862456979094592 00000100:00100000:1.0:1776178155.828054:0:8250:0:(service.c:2152:ptlrpc_server_handle_req_in()) got req x1862456979094720 00000100:00100000:2.0:1776178155.828217:0:6256:0:(service.c:2152:ptlrpc_server_handle_req_in()) got req x1862456979094848 00000100:00100000:3.0:1776178155.828291:0:6256:0:(nrs_fifo.c:179:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.204.3@tcp, seq: 6553 00000100:00100000:3.0:1776178155.828297:0:6256:0:(service.c:2324:ptlrpc_server_handle_request()) Handling RPC req@000000008b5ee68d pname:cluuid+ref:pid:xid:nid:opc:job mdt00_001:b50963bd-8eba-4432-9a54-8888d00e3fa1+61:17641:x1862456979094592:12345-192.168.204.3@tcp:36:chmod.0 00010000:00010000:3.0:1776178155.828367:0:6256:0:(ldlm_lockd.c:1720:ldlm_request_cancel()) ### server-side cancel handler START: 2 locks, starting at 0 00010000:00010000:3.0:1776178155.828394:0:6256:0:(ldlm_lock.c:216:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: mdt-lustre-MDT0001_UUID lock: 000000002f608ee6/0xa40083ab9bf534d1 lrc: 0/0,0 mode: --/PR res: [0x240000402:0x7fa:0x0].0x0 bits 0x1b/0x0 rrc: 5 type: IBT gid 0 flags: 0x44a01000000000 nid: 192.168.204.3@tcp remote: 0x771f68aa1eb27ae0 expref: 61 pid: 6255 timeout: 0 lvb_type: 0 00010000:00010000:3.0:1776178155.828423:0:6256:0:(ldlm_lock.c:216:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: mdt-lustre-MDT0001_UUID lock: 000000006a188f9a/0xa40083ab9bf533ea lrc: 0/0,0 mode: --/CR res: [0x240000402:0x7fa:0x0].0x0 bits 0x9/0x0 rrc: 4 type: IBT gid 0 flags: 0x44a01000000000 nid: 192.168.204.3@tcp remote: 0x771f68aa1eb27aaf expref: 60 pid: 8250 timeout: 0 lvb_type: 0 00010000:00010000:3.0:1776178155.828431:0:6256:0:(ldlm_inodebits.c:94:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x240000402:0x7fa:0x0].0x0 (000000006b0fcbcb) 00010000:00010000:3.0:1776178155.828435:0:6256:0:(ldlm_lockd.c:1782:ldlm_request_cancel()) ### server-side cancel handler END 00010000:00010000:3.0:1776178155.828476:0:6256:0:(ldlm_lock.c:766:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: mdt-lustre-MDT0001_UUID lock: 00000000014a8a18/0xa40083ab9bf53541 lrc: 3/0,1 mode: --/PW res: [0x240000402:0x7fa:0x0].0x0 bits 0x0/0x0 rrc: 3 type: IBT gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 6256 timeout: 0 lvb_type: 0 00010000:00010000:3.0:1776178155.828492:0:6256:0:(ldlm_lock.c:1064:ldlm_granted_list_add_lock()) ### About to add lock: ns: mdt-lustre-MDT0001_UUID lock: 00000000014a8a18/0xa40083ab9bf53541 lrc: 3/0,1 mode: PW/PW res: [0x240000402:0x7fa:0x0].0x0 bits 0x13/0x0 rrc: 3 type: IBT gid 0 flags: 0x50210000000000 nid: local remote: 0x0 expref: -99 pid: 6256 timeout: 0 lvb_type: 0 00000100:00100000:1.0:1776178155.828496:0:8250:0:(nrs_fifo.c:179:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.204.3@tcp, seq: 6554 00000100:00100000:1.0:1776178155.828500:0:8250:0:(service.c:2324:ptlrpc_server_handle_request()) Handling RPC req@00000000a9455d7d pname:cluuid+ref:pid:xid:nid:opc:job mdt00_003:b50963bd-8eba-4432-9a54-8888d00e3fa1+59:17642:x1862456979094720:12345-192.168.204.3@tcp:36:chmod.0 00010000:00010000:3.0:1776178155.828503:0:6256:0:(ldlm_request.c:528:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: mdt-lustre-MDT0001_UUID lock: 00000000014a8a18/0xa40083ab9bf53541 lrc: 3/0,1 mode: PW/PW res: [0x240000402:0x7fa:0x0].0x0 bits 0x13/0x0 rrc: 3 type: IBT gid 0 flags: 0x40210000000000 nid: local remote: 0x0 expref: -99 pid: 6256 timeout: 0 lvb_type: 0 00010000:00010000:1.0:1776178155.828521:0:8250:0:(ldlm_lockd.c:1720:ldlm_request_cancel()) ### server-side cancel handler START: 2 locks, starting at 0 00010000:00010000:1.0:1776178155.828533:0:8250:0:(ldlm_lock.c:216:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: mdt-lustre-MDT0001_UUID lock: 0000000014581b1a/0xa40083ab9bf534bc lrc: 0/0,0 mode: --/PR res: [0x240000402:0x7fb:0x0].0x0 bits 0x1b/0x0 rrc: 5 type: IBT gid 0 flags: 0x44a01000000000 nid: 192.168.204.3@tcp remote: 0x771f68aa1eb27ad9 expref: 59 pid: 6255 timeout: 0 lvb_type: 0 00010000:00010000:1.0:1776178155.828549:0:8250:0:(ldlm_lock.c:216:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: mdt-lustre-MDT0001_UUID lock: 00000000f314daa9/0xa40083ab9bf53414 lrc: 0/0,0 mode: --/CR res: [0x240000402:0x7fb:0x0].0x0 bits 0x9/0x0 rrc: 4 type: IBT gid 0 flags: 0x44a01000000000 nid: 192.168.204.3@tcp remote: 0x771f68aa1eb27ab6 expref: 58 pid: 8250 timeout: 0 lvb_type: 0 00010000:00010000:1.0:1776178155.828555:0:8250:0:(ldlm_inodebits.c:94:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x240000402:0x7fb:0x0].0x0 (000000009f7659df) 00010000:00010000:1.0:1776178155.828558:0:8250:0:(ldlm_lockd.c:1782:ldlm_request_cancel()) ### server-side cancel handler END 00010000:00010000:1.0:1776178155.828572:0:8250:0:(ldlm_lock.c:766:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: mdt-lustre-MDT0001_UUID lock: 000000007e0712b8/0xa40083ab9bf53548 lrc: 3/0,1 mode: --/PW res: [0x240000402:0x7fb:0x0].0x0 bits 0x0/0x0 rrc: 3 type: IBT gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 8250 timeout: 0 lvb_type: 0 00010000:00010000:1.0:1776178155.828581:0:8250:0:(ldlm_lock.c:1064:ldlm_granted_list_add_lock()) ### About to add lock: ns: mdt-lustre-MDT0001_UUID lock: 000000007e0712b8/0xa40083ab9bf53548 lrc: 3/0,1 mode: PW/PW res: [0x240000402:0x7fb:0x0].0x0 bits 0x13/0x0 rrc: 3 type: IBT gid 0 flags: 0x50210000000000 nid: local remote: 0x0 expref: -99 pid: 8250 timeout: 0 lvb_type: 0 00010000:00010000:1.0:1776178155.828589:0:8250:0:(ldlm_request.c:528:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: mdt-lustre-MDT0001_UUID lock: 000000007e0712b8/0xa40083ab9bf53548 lrc: 3/0,1 mode: PW/PW res: [0x240000402:0x7fb:0x0].0x0 bits 0x13/0x0 rrc: 3 type: IBT gid 0 flags: 0x40210000000000 nid: local remote: 0x0 expref: -99 pid: 8250 timeout: 0 lvb_type: 0 00000004:00010000:1.0:1776178155.828997:0:8250:0:(mdt_handler.c:3992:mdt_save_lock()) ### save lock request 00000000a9455d7d reply state 000000007996f1e6 transno 17179876259 ns: mdt-lustre-MDT0001_UUID lock: 000000007e0712b8/0xa40083ab9bf53548 lrc: 3/0,1 mode: PW/PW res: [0x240000402:0x7fb:0x0].0x0 bits 0x13/0x0 rrc: 3 type: IBT gid 0 flags: 0x40210000000000 nid: local remote: 0x0 expref: -99 pid: 8250 timeout: 0 lvb_type: 0 00010000:02000000:1.0:1776178155.829048:0:8250:0:(libcfs_fail.h:96:cfs_fail_check_set()) *** cfs_fail_loc=15a, val=0*** 00010000:00080000:1.0:1776178155.829053:0:8250:0:(ldlm_lib.c:3333:target_send_reply()) Schedule reply immediately 00000100:00100000:1.0:1776178155.829087:0:8250:0:(service.c:2372:ptlrpc_server_handle_request()) Handled RPC req@00000000a9455d7d pname:cluuid+ref:pid:xid:nid:opc:job mdt00_003:b50963bd-8eba-4432-9a54-8888d00e3fa1+60:17642:x1862456979094720:12345-192.168.204.3@tcp:36:chmod.0 Request processed in 585us (1787us total) trans 17179876259 rc 0/0 00000100:00100000:1.0:1776178155.829095:0:8250:0:(nrs_fifo.c:241:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.204.3@tcp, seq: 6554 00000100:00100000:1.0:1776178155.829113:0:8250:0:(nrs_fifo.c:179:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.204.3@tcp, seq: 6555 00000100:00100000:1.0:1776178155.829117:0:8250:0:(service.c:2324:ptlrpc_server_handle_request()) Handling RPC req@00000000c8afe7da pname:cluuid+ref:pid:xid:nid:opc:job mdt00_003:b50963bd-8eba-4432-9a54-8888d00e3fa1+59:17640:x1862456979094848:12345-192.168.204.3@tcp:36:chmod.0 00010000:00010000:1.0:1776178155.829135:0:8250:0:(ldlm_lockd.c:1720:ldlm_request_cancel()) ### server-side cancel handler START: 2 locks, starting at 0 00010000:00010000:1.0:1776178155.829147:0:8250:0:(ldlm_lock.c:216:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: mdt-lustre-MDT0001_UUID lock: 000000006844664c/0xa40083ab9bf534df lrc: 0/0,0 mode: --/PR res: [0x240000402:0x7f9:0x0].0x0 bits 0x1b/0x0 rrc: 5 type: IBT gid 0 flags: 0x44a01000000000 nid: 192.168.204.3@tcp remote: 0x771f68aa1eb27aee expref: 59 pid: 6255 timeout: 0 lvb_type: 0 00010000:00010000:1.0:1776178155.829159:0:8250:0:(ldlm_lock.c:216:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: mdt-lustre-MDT0001_UUID lock: 000000000b4a063d/0xa40083ab9bf533c0 lrc: 0/0,0 mode: --/CR res: [0x240000402:0x7f9:0x0].0x0 bits 0x9/0x0 rrc: 4 type: IBT gid 0 flags: 0x44a01000000000 nid: 192.168.204.3@tcp remote: 0x771f68aa1eb27aa8 expref: 58 pid: 8250 timeout: 0 lvb_type: 0 00010000:00010000:1.0:1776178155.829167:0:8250:0:(ldlm_inodebits.c:94:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x240000402:0x7f9:0x0].0x0 (00000000f193fa59) 00010000:00010000:1.0:1776178155.829170:0:8250:0:(ldlm_lockd.c:1782:ldlm_request_cancel()) ### server-side cancel handler END 00010000:00010000:1.0:1776178155.829180:0:8250:0:(ldlm_lock.c:766:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: mdt-lustre-MDT0001_UUID lock: 0000000022bf14ed/0xa40083ab9bf5354f lrc: 3/0,1 mode: --/PW res: [0x240000402:0x7f9:0x0].0x0 bits 0x0/0x0 rrc: 3 type: IBT gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 8250 timeout: 0 lvb_type: 0 00010000:00010000:1.0:1776178155.829188:0:8250:0:(ldlm_lock.c:1064:ldlm_granted_list_add_lock()) ### About to add lock: ns: mdt-lustre-MDT0001_UUID lock: 0000000022bf14ed/0xa40083ab9bf5354f lrc: 3/0,1 mode: PW/PW res: [0x240000402:0x7f9:0x0].0x0 bits 0x13/0x0 rrc: 3 type: IBT gid 0 flags: 0x50210000000000 nid: local remote: 0x0 expref: -99 pid: 8250 timeout: 0 lvb_type: 0 00010000:00010000:1.0:1776178155.829200:0:8250:0:(ldlm_request.c:528:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: mdt-lustre-MDT0001_UUID lock: 0000000022bf14ed/0xa40083ab9bf5354f lrc: 3/0,1 mode: PW/PW res: [0x240000402:0x7f9:0x0].0x0 bits 0x13/0x0 rrc: 3 type: IBT gid 0 flags: 0x40210000000000 nid: local remote: 0x0 expref: -99 pid: 8250 timeout: 0 lvb_type: 0 00000004:00010000:1.0:1776178155.829256:0:8250:0:(mdt_handler.c:3992:mdt_save_lock()) ### save lock request 00000000c8afe7da reply state 000000004b17e827 transno 17179876260 ns: mdt-lustre-MDT0001_UUID lock: 0000000022bf14ed/0xa40083ab9bf5354f lrc: 3/0,1 mode: PW/PW res: [0x240000402:0x7f9:0x0].0x0 bits 0x13/0x0 rrc: 3 type: IBT gid 0 flags: 0x40210000000000 nid: local remote: 0x0 expref: -99 pid: 8250 timeout: 0 lvb_type: 0 00010000:02000000:1.0:1776178155.829279:0:8250:0:(libcfs_fail.h:96:cfs_fail_check_set()) *** cfs_fail_loc=15a, val=0*** 00010000:00080000:1.0:1776178155.829281:0:8250:0:(ldlm_lib.c:3333:target_send_reply()) Schedule reply immediately 00000100:00100000:1.0:1776178155.829292:0:8250:0:(service.c:2372:ptlrpc_server_handle_request()) Handled RPC req@00000000c8afe7da pname:cluuid+ref:pid:xid:nid:opc:job mdt00_003:b50963bd-8eba-4432-9a54-8888d00e3fa1+59:17640:x1862456979094848:12345-192.168.204.3@tcp:36:chmod.0 Request processed in 174us (1964us total) trans 17179876260 rc 0/0 00000100:00100000:1.0:1776178155.829298:0:8250:0:(nrs_fifo.c:241:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.204.3@tcp, seq: 6555 00010000:00010000:1.0:1776178155.829453:0:3340:0:(ldlm_lock.c:1064:ldlm_granted_list_add_lock()) ### About to add lock: ns: mdt-lustre-MDT0001_UUID lock: 000000007e0712b8/0xa40083ab9bf53548 lrc: 3/0,1 mode: COS/COS res: [0x240000402:0x7fb:0x0].0x0 bits 0x13/0x0 rrc: 3 type: IBT gid 0 flags: 0x50210000000000 nid: local remote: 0x0 expref: -99 pid: 8250 timeout: 0 lvb_type: 0 00010000:00010000:1.0:1776178155.829462:0:3340:0:(ldlm_inodebits.c:94:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x240000402:0x7fb:0x0].0x0 (000000009f7659df) 00010000:00010000:1.0:1776178155.829464:0:3340:0:(ldlm_inodebits.c:96:ldlm_reprocess_inodebits_queue()) Hint 13 00010000:00010000:1.0:1776178155.829474:0:3339:0:(ldlm_lock.c:1064:ldlm_granted_list_add_lock()) ### About to add lock: ns: mdt-lustre-MDT0001_UUID lock: 0000000022bf14ed/0xa40083ab9bf5354f lrc: 3/0,1 mode: COS/COS res: [0x240000402:0x7f9:0x0].0x0 bits 0x13/0x0 rrc: 3 type: IBT gid 0 flags: 0x50210000000000 nid: local remote: 0x0 expref: -99 pid: 8250 timeout: 0 lvb_type: 0 00010000:00010000:1.0:1776178155.829481:0:3339:0:(ldlm_inodebits.c:94:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x240000402:0x7f9:0x0].0x0 (00000000f193fa59) 00010000:00010000:1.0:1776178155.829483:0:3339:0:(ldlm_inodebits.c:96:ldlm_reprocess_inodebits_queue()) Hint 13 00000004:00010000:3.0:1776178155.832708:0:6256:0:(mdt_handler.c:3992:mdt_save_lock()) ### save lock request 000000008b5ee68d reply state 0000000094c8cb62 transno 17179876258 ns: mdt-lustre-MDT0001_UUID lock: 00000000014a8a18/0xa40083ab9bf53541 lrc: 3/0,1 mode: PW/PW res: [0x240000402:0x7fa:0x0].0x0 bits 0x13/0x0 rrc: 3 type: IBT gid 0 flags: 0x40210000000000 nid: local remote: 0x0 expref: -99 pid: 6256 timeout: 0 lvb_type: 0 00010000:02000000:3.0:1776178155.832997:0:6256:0:(libcfs_fail.h:96:cfs_fail_check_set()) *** cfs_fail_loc=15a, val=0*** 00010000:00080000:3.0:1776178155.833002:0:6256:0:(ldlm_lib.c:3333:target_send_reply()) Schedule reply immediately 00000100:00100000:3.0:1776178155.833081:0:6256:0:(service.c:2372:ptlrpc_server_handle_request()) Handled RPC req@000000008b5ee68d pname:cluuid+ref:pid:xid:nid:opc:job mdt00_001:b50963bd-8eba-4432-9a54-8888d00e3fa1+59:17641:x1862456979094592:12345-192.168.204.3@tcp:36:chmod.0 Request processed in 4779us (5837us total) trans 17179876258 rc 0/0 00000100:00100000:3.0:1776178155.833091:0:6256:0:(nrs_fifo.c:241:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.204.3@tcp, seq: 6553 00010000:00010000:2.0:1776178155.833172:0:3340:0:(ldlm_lock.c:1064:ldlm_granted_list_add_lock()) ### About to add lock: ns: mdt-lustre-MDT0001_UUID lock: 00000000014a8a18/0xa40083ab9bf53541 lrc: 3/0,1 mode: COS/COS res: [0x240000402:0x7fa:0x0].0x0 bits 0x13/0x0 rrc: 3 type: IBT gid 0 flags: 0x50210000000000 nid: local remote: 0x0 expref: -99 pid: 6256 timeout: 0 lvb_type: 0 00010000:00010000:2.0:1776178155.833185:0:3340:0:(ldlm_inodebits.c:94:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x240000402:0x7fa:0x0].0x0 (000000006b0fcbcb) 00010000:00010000:2.0:1776178155.833188:0:3340:0:(ldlm_inodebits.c:96:ldlm_reprocess_inodebits_queue()) Hint 13 00000100:00100000:3.0:1776178155.836543:0:3348:0:(events.c:361:request_in_callback()) peer: 12345-192.168.204.3@tcp (source: 12345-192.168.204.3@tcp) 00000100:00100000:0.0:1776178155.837717:0:6256:0:(service.c:2152:ptlrpc_server_handle_req_in()) got req x1862456979094976 00000100:00100000:0.0:1776178155.837783:0:6256:0:(nrs_fifo.c:179:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.204.3@tcp, seq: 6556 00000100:00100000:0.0:1776178155.837791:0:6256:0:(service.c:2324:ptlrpc_server_handle_request()) Handling RPC req@00000000df20940e pname:cluuid+ref:pid:xid:nid:opc:job mdt00_001:b50963bd-8eba-4432-9a54-8888d00e3fa1+59:17645:x1862456979094976:12345-192.168.204.3@tcp:36:chmod.0 00010000:00010000:0.0:1776178155.837846:0:6256:0:(ldlm_lockd.c:1720:ldlm_request_cancel()) ### server-side cancel handler START: 2 locks, starting at 0 00010000:00010000:0.0:1776178155.837884:0:6256:0:(ldlm_lock.c:216:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: mdt-lustre-MDT0001_UUID lock: 000000007f7ce364/0xa40083ab9bf53509 lrc: 0/0,0 mode: --/PR res: [0x240000402:0x7fe:0x0].0x0 bits 0x1b/0x0 rrc: 5 type: IBT gid 0 flags: 0x44a01000000000 nid: 192.168.204.3@tcp remote: 0x771f68aa1eb27b03 expref: 59 pid: 11281 timeout: 0 lvb_type: 0 00010000:00010000:0.0:1776178155.837905:0:6256:0:(ldlm_lock.c:216:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: mdt-lustre-MDT0001_UUID lock: 00000000a211f94c/0xa40083ab9bf53492 lrc: 0/0,0 mode: --/CR res: [0x240000402:0x7fe:0x0].0x0 bits 0x9/0x0 rrc: 4 type: IBT gid 0 flags: 0x44a01000000000 nid: 192.168.204.3@tcp remote: 0x771f68aa1eb27acb expref: 58 pid: 6256 timeout: 0 lvb_type: 0 00010000:00010000:0.0:1776178155.837927:0:6256:0:(ldlm_inodebits.c:94:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x240000402:0x7fe:0x0].0x0 (00000000498a007f) 00010000:00010000:0.0:1776178155.837932:0:6256:0:(ldlm_lockd.c:1782:ldlm_request_cancel()) ### server-side cancel handler END 00010000:00010000:0.0:1776178155.837962:0:6256:0:(ldlm_lock.c:766:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: mdt-lustre-MDT0001_UUID lock: 0000000011d684d8/0xa40083ab9bf53556 lrc: 3/0,1 mode: --/PW res: [0x240000402:0x7fe:0x0].0x0 bits 0x0/0x0 rrc: 3 type: IBT gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 6256 timeout: 0 lvb_type: 0 00010000:00010000:0.0:1776178155.837978:0:6256:0:(ldlm_lock.c:1064:ldlm_granted_list_add_lock()) ### About to add lock: ns: mdt-lustre-MDT0001_UUID lock: 0000000011d684d8/0xa40083ab9bf53556 lrc: 3/0,1 mode: PW/PW res: [0x240000402:0x7fe:0x0].0x0 bits 0x13/0x0 rrc: 3 type: IBT gid 0 flags: 0x50210000000000 nid: local remote: 0x0 expref: -99 pid: 6256 timeout: 0 lvb_type: 0 00010000:00010000:0.0:1776178155.837989:0:6256:0:(ldlm_request.c:528:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: mdt-lustre-MDT0001_UUID lock: 0000000011d684d8/0xa40083ab9bf53556 lrc: 3/0,1 mode: PW/PW res: [0x240000402:0x7fe:0x0].0x0 bits 0x13/0x0 rrc: 3 type: IBT gid 0 flags: 0x40210000000000 nid: local remote: 0x0 expref: -99 pid: 6256 timeout: 0 lvb_type: 0 00000004:00010000:0.0:1776178155.838167:0:6256:0:(mdt_handler.c:3992:mdt_save_lock()) ### save lock request 00000000df20940e reply state 00000000afcbd82c transno 17179876261 ns: mdt-lustre-MDT0001_UUID lock: 0000000011d684d8/0xa40083ab9bf53556 lrc: 3/0,1 mode: PW/PW res: [0x240000402:0x7fe:0x0].0x0 bits 0x13/0x0 rrc: 3 type: IBT gid 0 flags: 0x40210000000000 nid: local remote: 0x0 expref: -99 pid: 6256 timeout: 0 lvb_type: 0 00010000:02000000:0.0:1776178155.838216:0:6256:0:(libcfs_fail.h:96:cfs_fail_check_set()) *** cfs_fail_loc=15a, val=0*** 00010000:00080000:0.0:1776178155.838220:0:6256:0:(ldlm_lib.c:3333:target_send_reply()) Schedule reply immediately 00000100:00100000:0.0:1776178155.838236:0:6256:0:(service.c:2372:ptlrpc_server_handle_request()) Handled RPC req@00000000df20940e pname:cluuid+ref:pid:xid:nid:opc:job mdt00_001:b50963bd-8eba-4432-9a54-8888d00e3fa1+59:17645:x1862456979094976:12345-192.168.204.3@tcp:36:chmod.0 Request processed in 446us (1703us total) trans 17179876261 rc 0/0 00000100:00100000:0.0:1776178155.838243:0:6256:0:(nrs_fifo.c:241:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.204.3@tcp, seq: 6556 00010000:00010000:1.0:1776178155.838284:0:3339:0:(ldlm_lock.c:1064:ldlm_granted_list_add_lock()) ### About to add lock: ns: mdt-lustre-MDT0001_UUID lock: 0000000011d684d8/0xa40083ab9bf53556 lrc: 3/0,1 mode: COS/COS res: [0x240000402:0x7fe:0x0].0x0 bits 0x13/0x0 rrc: 3 type: IBT gid 0 flags: 0x50210000000000 nid: local remote: 0x0 expref: -99 pid: 6256 timeout: 0 lvb_type: 0 00010000:00010000:1.0:1776178155.838293:0:3339:0:(ldlm_inodebits.c:94:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x240000402:0x7fe:0x0].0x0 (00000000498a007f) 00010000:00010000:1.0:1776178155.838296:0:3339:0:(ldlm_inodebits.c:96:ldlm_reprocess_inodebits_queue()) Hint 13 00000100:00100000:1.0:1776178155.840567:0:3347:0:(events.c:361:request_in_callback()) peer: 12345-192.168.204.3@tcp (source: 12345-192.168.204.3@tcp) 00000100:00100000:1.0:1776178155.840674:0:3347:0:(events.c:361:request_in_callback()) peer: 12345-192.168.204.3@tcp (source: 12345-192.168.204.3@tcp) 00000100:00100000:1.0:1776178155.840722:0:6255:0:(service.c:2152:ptlrpc_server_handle_req_in()) got req x1862456979095104 00000100:00100000:1.0:1776178155.840768:0:6255:0:(service.c:2152:ptlrpc_server_handle_req_in()) got req x1862456979095232 00000100:00100000:1.0:1776178155.840799:0:6255:0:(nrs_fifo.c:179:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.204.3@tcp, seq: 6557 00000100:00100000:1.0:1776178155.840808:0:6255:0:(service.c:2324:ptlrpc_server_handle_request()) Handling RPC req@00000000a23c68f9 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_000:b50963bd-8eba-4432-9a54-8888d00e3fa1+60:17643:x1862456979095104:12345-192.168.204.3@tcp:36:chmod.0 00010000:00010000:1.0:1776178155.840900:0:6255:0:(ldlm_lockd.c:1720:ldlm_request_cancel()) ### server-side cancel handler START: 2 locks, starting at 0 00000100:00100000:3.0:1776178155.841431:0:6256:0:(nrs_fifo.c:179:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.204.3@tcp, seq: 6558 00000100:00100000:3.0:1776178155.841437:0:6256:0:(service.c:2324:ptlrpc_server_handle_request()) Handling RPC req@000000002c6408bc pname:cluuid+ref:pid:xid:nid:opc:job mdt00_001:b50963bd-8eba-4432-9a54-8888d00e3fa1+60:17644:x1862456979095232:12345-192.168.204.3@tcp:36:chmod.0 00010000:00010000:1.0:1776178155.842157:0:6255:0:(ldlm_lock.c:216:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: mdt-lustre-MDT0001_UUID lock: 00000000d3e952db/0xa40083ab9bf534fb lrc: 0/0,0 mode: --/PR res: [0x240000402:0x7fc:0x0].0x0 bits 0x1b/0x0 rrc: 5 type: IBT gid 0 flags: 0x44a01000000000 nid: 192.168.204.3@tcp remote: 0x771f68aa1eb27afc expref: 60 pid: 11281 timeout: 0 lvb_type: 0 00010000:00010000:1.0:1776178155.842179:0:6255:0:(ldlm_lock.c:216:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: mdt-lustre-MDT0001_UUID lock: 0000000004511956/0xa40083ab9bf5343e lrc: 0/0,0 mode: --/CR res: [0x240000402:0x7fc:0x0].0x0 bits 0x9/0x0 rrc: 4 type: IBT gid 0 flags: 0x44a01000000000 nid: 192.168.204.3@tcp remote: 0x771f68aa1eb27abd expref: 59 pid: 6255 timeout: 0 lvb_type: 0 00010000:00010000:1.0:1776178155.842188:0:6255:0:(ldlm_inodebits.c:94:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x240000402:0x7fc:0x0].0x0 (0000000064abeb1c) 00010000:00010000:1.0:1776178155.842194:0:6255:0:(ldlm_lockd.c:1782:ldlm_request_cancel()) ### server-side cancel handler END 00010000:00010000:1.0:1776178155.842241:0:6255:0:(ldlm_lock.c:766:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: mdt-lustre-MDT0001_UUID lock: 0000000031bb7277/0xa40083ab9bf5355d lrc: 3/0,1 mode: --/PW res: [0x240000402:0x7fc:0x0].0x0 bits 0x0/0x0 rrc: 3 type: IBT gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 6255 timeout: 0 lvb_type: 0 00010000:00010000:1.0:1776178155.842255:0:6255:0:(ldlm_lock.c:1064:ldlm_granted_list_add_lock()) ### About to add lock: ns: mdt-lustre-MDT0001_UUID lock: 0000000031bb7277/0xa40083ab9bf5355d lrc: 3/0,1 mode: PW/PW res: [0x240000402:0x7fc:0x0].0x0 bits 0x13/0x0 rrc: 3 type: IBT gid 0 flags: 0x50210000000000 nid: local remote: 0x0 expref: -99 pid: 6255 timeout: 0 lvb_type: 0 00010000:00010000:1.0:1776178155.842264:0:6255:0:(ldlm_request.c:528:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: mdt-lustre-MDT0001_UUID lock: 0000000031bb7277/0xa40083ab9bf5355d lrc: 3/0,1 mode: PW/PW res: [0x240000402:0x7fc:0x0].0x0 bits 0x13/0x0 rrc: 3 type: IBT gid 0 flags: 0x40210000000000 nid: local remote: 0x0 expref: -99 pid: 6255 timeout: 0 lvb_type: 0 00010000:00010000:3.0:1776178155.842424:0:6256:0:(ldlm_lockd.c:1720:ldlm_request_cancel()) ### server-side cancel handler START: 2 locks, starting at 0 00010000:00010000:3.0:1776178155.842436:0:6256:0:(ldlm_lock.c:216:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: mdt-lustre-MDT0001_UUID lock: 000000007ec324d4/0xa40083ab9bf534ed lrc: 0/0,0 mode: --/PR res: [0x240000402:0x7fd:0x0].0x0 bits 0x1b/0x0 rrc: 5 type: IBT gid 0 flags: 0x44a01000000000 nid: 192.168.204.3@tcp remote: 0x771f68aa1eb27af5 expref: 58 pid: 11281 timeout: 0 lvb_type: 0 00010000:00010000:3.0:1776178155.842459:0:6256:0:(ldlm_lock.c:216:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: mdt-lustre-MDT0001_UUID lock: 0000000095e53696/0xa40083ab9bf53468 lrc: 0/0,0 mode: --/CR res: [0x240000402:0x7fd:0x0].0x0 bits 0x9/0x0 rrc: 4 type: IBT gid 0 flags: 0x44a01000000000 nid: 192.168.204.3@tcp remote: 0x771f68aa1eb27ac4 expref: 57 pid: 6256 timeout: 0 lvb_type: 0 00010000:00010000:3.0:1776178155.842469:0:6256:0:(ldlm_inodebits.c:94:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x240000402:0x7fd:0x0].0x0 (000000008d47e759) 00010000:00010000:3.0:1776178155.842472:0:6256:0:(ldlm_lockd.c:1782:ldlm_request_cancel()) ### server-side cancel handler END 00010000:00010000:3.0:1776178155.842489:0:6256:0:(ldlm_lock.c:766:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: mdt-lustre-MDT0001_UUID lock: 000000006a188f9a/0xa40083ab9bf53564 lrc: 3/0,1 mode: --/PW res: [0x240000402:0x7fd:0x0].0x0 bits 0x0/0x0 rrc: 3 type: IBT gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 6256 timeout: 0 lvb_type: 0 00010000:00010000:3.0:1776178155.842497:0:6256:0:(ldlm_lock.c:1064:ldlm_granted_list_add_lock()) ### About to add lock: ns: mdt-lustre-MDT0001_UUID lock: 000000006a188f9a/0xa40083ab9bf53564 lrc: 3/0,1 mode: PW/PW res: [0x240000402:0x7fd:0x0].0x0 bits 0x13/0x0 rrc: 3 type: IBT gid 0 flags: 0x50210000000000 nid: local remote: 0x0 expref: -99 pid: 6256 timeout: 0 lvb_type: 0 00010000:00010000:3.0:1776178155.842505:0:6256:0:(ldlm_request.c:528:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: mdt-lustre-MDT0001_UUID lock: 000000006a188f9a/0xa40083ab9bf53564 lrc: 3/0,1 mode: PW/PW res: [0x240000402:0x7fd:0x0].0x0 bits 0x13/0x0 rrc: 3 type: IBT gid 0 flags: 0x40210000000000 nid: local remote: 0x0 expref: -99 pid: 6256 timeout: 0 lvb_type: 0 00000004:00010000:3.0:1776178155.842671:0:6256:0:(mdt_handler.c:3992:mdt_save_lock()) ### save lock request 000000002c6408bc reply state 0000000031230bec transno 17179876262 ns: mdt-lustre-MDT0001_UUID lock: 000000006a188f9a/0xa40083ab9bf53564 lrc: 3/0,1 mode: PW/PW res: [0x240000402:0x7fd:0x0].0x0 bits 0x13/0x0 rrc: 3 type: IBT gid 0 flags: 0x40210000000000 nid: local remote: 0x0 expref: -99 pid: 6256 timeout: 0 lvb_type: 0 00000004:00010000:1.0:1776178155.842691:0:6255:0:(mdt_handler.c:3992:mdt_save_lock()) ### save lock request 00000000a23c68f9 reply state 00000000405feffb transno 17179876263 ns: mdt-lustre-MDT0001_UUID lock: 0000000031bb7277/0xa40083ab9bf5355d lrc: 3/0,1 mode: PW/PW res: [0x240000402:0x7fc:0x0].0x0 bits 0x13/0x0 rrc: 3 type: IBT gid 0 flags: 0x40210000000000 nid: local remote: 0x0 expref: -99 pid: 6255 timeout: 0 lvb_type: 0 00010000:02000000:1.0:1776178155.842719:0:6255:0:(libcfs_fail.h:96:cfs_fail_check_set()) *** cfs_fail_loc=15a, val=0*** 00010000:02000000:3.0:1776178155.842719:0:6256:0:(libcfs_fail.h:96:cfs_fail_check_set()) *** cfs_fail_loc=15a, val=0*** 00010000:00080000:3.0:1776178155.842724:0:6256:0:(ldlm_lib.c:3333:target_send_reply()) Schedule reply immediately 00000100:00100000:3.0:1776178155.843216:0:6256:0:(service.c:2372:ptlrpc_server_handle_request()) Handled RPC req@000000002c6408bc pname:cluuid+ref:pid:xid:nid:opc:job mdt00_001:b50963bd-8eba-4432-9a54-8888d00e3fa1+60:17644:x1862456979095232:12345-192.168.204.3@tcp:36:chmod.0 Request processed in 1776us (2540us total) trans 17179876262 rc 0/0 00000100:00100000:3.0:1776178155.843226:0:6256:0:(nrs_fifo.c:241:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.204.3@tcp, seq: 6558 00010000:00010000:3.0:1776178155.843264:0:3340:0:(ldlm_lock.c:1064:ldlm_granted_list_add_lock()) ### About to add lock: ns: mdt-lustre-MDT0001_UUID lock: 000000006a188f9a/0xa40083ab9bf53564 lrc: 3/0,1 mode: COS/COS res: [0x240000402:0x7fd:0x0].0x0 bits 0x13/0x0 rrc: 3 type: IBT gid 0 flags: 0x50210000000000 nid: local remote: 0x0 expref: -99 pid: 6256 timeout: 0 lvb_type: 0 00010000:00080000:1.0:1776178155.843271:0:6255:0:(ldlm_lib.c:3333:target_send_reply()) Schedule reply immediately 00010000:00010000:3.0:1776178155.843273:0:3340:0:(ldlm_inodebits.c:94:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x240000402:0x7fd:0x0].0x0 (000000008d47e759) 00010000:00010000:3.0:1776178155.843276:0:3340:0:(ldlm_inodebits.c:96:ldlm_reprocess_inodebits_queue()) Hint 13 00000100:00100000:1.0:1776178155.843282:0:6255:0:(service.c:2372:ptlrpc_server_handle_request()) Handled RPC req@00000000a23c68f9 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_000:b50963bd-8eba-4432-9a54-8888d00e3fa1+59:17643:x1862456979095104:12345-192.168.204.3@tcp:36:chmod.0 Request processed in 2476us (2723us total) trans 17179876263 rc 0/0 00000100:00100000:1.0:1776178155.843288:0:6255:0:(nrs_fifo.c:241:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.204.3@tcp, seq: 6557 00010000:00010000:1.0:1776178155.845808:0:3339:0:(ldlm_lock.c:1064:ldlm_granted_list_add_lock()) ### About to add lock: ns: mdt-lustre-MDT0001_UUID lock: 0000000031bb7277/0xa40083ab9bf5355d lrc: 3/0,1 mode: COS/COS res: [0x240000402:0x7fc:0x0].0x0 bits 0x13/0x0 rrc: 3 type: IBT gid 0 flags: 0x50210000000000 nid: local remote: 0x0 expref: -99 pid: 6255 timeout: 0 lvb_type: 0 00010000:00010000:1.0:1776178155.845829:0:3339:0:(ldlm_inodebits.c:94:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x240000402:0x7fc:0x0].0x0 (0000000064abeb1c) 00010000:00010000:1.0:1776178155.845832:0:3339:0:(ldlm_inodebits.c:96:ldlm_reprocess_inodebits_queue()) Hint 13 00000100:00100000:2.0:1776178155.872600:0:3353:0:(client.c:1733:ptlrpc_send_new_req()) Sending RPC req@00000000c3b47349 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_00:lustre-MDT0001-mdtlov_UUID:3353:1862456998035840:0@lo:13:osp-pre-1-1.0 00000100:00100000:2.0:1776178155.872694:0:3353:0:(events.c:361:request_in_callback()) peer: 12345-0@lo (source: 12345-0@lo) 00000100:00100000:3.0:1776178155.873254:0:8231:0:(service.c:2152:ptlrpc_server_handle_req_in()) got req x1862456998035840 00000100:00100000:3.0:1776178155.873298:0:8231:0:(nrs_fifo.c:179:nrs_fifo_req_get()) NRS start fifo request from 12345-0@lo, seq: 441 00000100:00100000:3.0:1776178155.873305:0:8231:0:(service.c:2324:ptlrpc_server_handle_request()) Handling RPC req@000000008d82b734 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_create00:lustre-MDT0001-mdtlov_UUID+5:3353:x1862456998035840:12345-0@lo:13:osp-pre-1-1.0 00002000:00000024:3.0:1776178155.873337:0:8231:0:(ofd_obd.c:740:ofd_statfs()) blocks cached 229376 granted 491746368 pending 0 free 3923689472 avail 3692167168 00000100:00100000:3.0:1776178155.873386:0:8231:0:(service.c:2372:ptlrpc_server_handle_request()) Handled RPC req@000000008d82b734 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_create00:lustre-MDT0001-mdtlov_UUID+5:3353:x1862456998035840:12345-0@lo:13:osp-pre-1-1.0 Request processed in 80us (693us total) trans 0 rc 0/0 00000100:00100000:3.0:1776178155.873393:0:8231:0:(nrs_fifo.c:241:nrs_fifo_req_stop()) NRS stop fifo request from 12345-0@lo, seq: 441 00000100:00100000:2.0:1776178155.873605:0:3353:0:(client.c:2215:ptlrpc_check_set()) Completed RPC req@00000000c3b47349 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_00:lustre-MDT0001-mdtlov_UUID:3353:1862456998035840:0@lo:13:osp-pre-1-1.0 00010000:00010000:3.0:1776178155.939426:0:9225:0:(ldlm_lockd.c:2792:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:2.0:1776178156.384837:0:3353:0:(ptlrpcd.c:415:ptlrpcd_check()) transfer 1 async RPCs [1->0] 00000100:00100000:2.0:1776178156.384865:0:3353:0:(client.c:1733:ptlrpc_send_new_req()) Sending RPC req@0000000081ef7fa0 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_00:lustre-MDT0001-mdtlov_UUID:3353:1862456998035904:0@lo:13:osp-pre-0-1.0 00000100:00100000:2.0:1776178156.384961:0:3353:0:(events.c:361:request_in_callback()) peer: 12345-0@lo (source: 12345-0@lo) 00000100:00100000:0.0:1776178156.385338:0:9554:0:(service.c:2152:ptlrpc_server_handle_req_in()) got req x1862456998035904 00000100:00100000:0.0:1776178156.385395:0:9554:0:(nrs_fifo.c:179:nrs_fifo_req_get()) NRS start fifo request from 12345-0@lo, seq: 442 00000100:00100000:0.0:1776178156.385402:0:9554:0:(service.c:2324:ptlrpc_server_handle_request()) Handling RPC req@000000006e2391ef pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_create00:lustre-MDT0001-mdtlov_UUID+5:3353:x1862456998035904:12345-0@lo:13:osp-pre-0-1.0 00002000:00000024:0.0:1776178156.385437:0:9554:0:(ofd_obd.c:740:ofd_statfs()) blocks cached 315392 granted 484883776 pending 0 free 3923689472 avail 3692167168 00000100:00100000:0.0:1776178156.385491:0:9554:0:(service.c:2372:ptlrpc_server_handle_request()) Handled RPC req@000000006e2391ef pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_create00:lustre-MDT0001-mdtlov_UUID+5:3353:x1862456998035904:12345-0@lo:13:osp-pre-0-1.0 Request processed in 88us (531us total) trans 0 rc 0/0 00000100:00100000:0.0:1776178156.385498:0:9554:0:(nrs_fifo.c:241:nrs_fifo_req_stop()) NRS stop fifo request from 12345-0@lo, seq: 442 00000100:00100000:2.0:1776178156.386029:0:3353:0:(client.c:2215:ptlrpc_check_set()) Completed RPC req@0000000081ef7fa0 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_00:lustre-MDT0001-mdtlov_UUID:3353:1862456998035904:0@lo:13:osp-pre-0-1.0 00010000:00010000:2.0:1776178156.964175:0:9224:0:(ldlm_lockd.c:2792:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000001:00080000:1.0:1776178157.418139:0:8225:0:(tgt_lastrcvd.c:913:tgt_cb_last_committed()) lustre-OST0000: transno 4294969355 is committed 00000001:00080000:1.0:1776178157.418162:0:8225:0:(tgt_lastrcvd.c:913:tgt_cb_last_committed()) lustre-OST0000: transno 4294969352 is committed 00000001:00080000:3.0:1776178157.422077:0:9217:0:(tgt_lastrcvd.c:913:tgt_cb_last_committed()) lustre-OST0001: transno 4294969327 is committed 00000001:00080000:3.0:1776178157.422108:0:9217:0:(tgt_lastrcvd.c:913:tgt_cb_last_committed()) lustre-OST0001: transno 4294969323 is committed 00010000:00010000:1.0:1776178157.986706:0:6245:0:(ldlm_lockd.c:2792:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00010000:00010000:2.0:1776178159.010241:0:6244:0:(ldlm_lockd.c:2792:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:3.0:1776178159.456779:0:3356:0:(client.c:1733:ptlrpc_send_new_req()) Sending RPC req@00000000a0a4914c pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_03:lustre-MDT0000-mdtlov_UUID:3356:1862456998036032:0@lo:41:osp-pre-1-0.0 00000100:00100000:3.0:1776178159.456870:0:3356:0:(events.c:361:request_in_callback()) peer: 12345-0@lo (source: 12345-0@lo) 00000100:00100000:1.0:1776178159.457120:0:14128:0:(service.c:2152:ptlrpc_server_handle_req_in()) got req x1862456998036032 00000100:00100000:1.0:1776178159.457170:0:14128:0:(nrs_fifo.c:179:nrs_fifo_req_get()) NRS start fifo request from 12345-0@lo, seq: 662 00000100:00100000:1.0:1776178159.457176:0:14128:0:(service.c:2324:ptlrpc_server_handle_request()) Handling RPC req@00000000574c4997 pname:cluuid+ref:pid:xid:nid:opc:job mdt_out00_002:lustre-MDT0000-mdtlov_UUID+5:3356:x1862456998036032:12345-0@lo:41:osp-pre-1-0.0 00000004:00000024:1.0:1776178159.457217:0:14128:0:(mdt_handler.c:548:mdt_statfs()) blocks cached 0 granted 2097152 pending 0 free 1446035456 avail 1316573184 00000100:00100000:0.0:1776178159.457342:0:3355:0:(client.c:1733:ptlrpc_send_new_req()) Sending RPC req@000000004a9818ac pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_02:ac61f839-0e8e-4cac-9c59-08bcfbbad330:3355:1862456998035968:0@lo:400:kworker/u8:0.0 00000100:00100000:0.0:1776178159.457376:0:3355:0:(events.c:361:request_in_callback()) peer: 12345-0@lo (source: 12345-0@lo) 00000100:00100000:0.0:1776178159.457463:0:20137:0:(service.c:2152:ptlrpc_server_handle_req_in()) got req x1862456998035968 00000100:00100000:0.0:1776178159.457486:0:20137:0:(nrs_fifo.c:179:nrs_fifo_req_get()) NRS start fifo request from 12345-0@lo, seq: 138 00000100:00100000:0.0:1776178159.457490:0:20137:0:(service.c:2324:ptlrpc_server_handle_request()) Handling RPC req@00000000f0ff59c2 pname:cluuid+ref:pid:xid:nid:opc:job ll_mgs_0000:ac61f839-0e8e-4cac-9c59-08bcfbbad330+23:3355:x1862456998035968:12345-0@lo:400:kworker/u8:0.0 00000100:00100000:0.0:1776178159.457521:0:20137:0:(service.c:2372:ptlrpc_server_handle_request()) Handled RPC req@00000000f0ff59c2 pname:cluuid+ref:pid:xid:nid:opc:job ll_mgs_0000:ac61f839-0e8e-4cac-9c59-08bcfbbad330+23:3355:x1862456998035968:12345-0@lo:400:kworker/u8:0.0 Request processed in 30us (144us total) trans 0 rc 0/0 00000100:00100000:0.0:1776178159.457528:0:20137:0:(nrs_fifo.c:241:nrs_fifo_req_stop()) NRS stop fifo request from 12345-0@lo, seq: 138 00000100:00100000:0.0:1776178159.457571:0:3353:0:(client.c:1733:ptlrpc_send_new_req()) Sending RPC req@000000004eee88be pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_00:lustre-MDT0000-mdtlov_UUID:3353:1862456998036096:0@lo:13:osp-pre-0-0.0 00000100:00100000:0.0:1776178159.457595:0:3353:0:(events.c:361:request_in_callback()) peer: 12345-0@lo (source: 12345-0@lo) 00000100:00100000:0.0:1776178159.457611:0:3353:0:(ptlrpcd.c:415:ptlrpcd_check()) transfer 1 async RPCs [1->0] 00000100:00100000:0.0:1776178159.457615:0:3353:0:(client.c:1733:ptlrpc_send_new_req()) Sending RPC req@000000009a8d7b96 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_00:lustre-MDT0000-mdtlov_UUID:3353:1862456998036160:0@lo:13:osp-pre-1-0.0 00000100:00100000:0.0:1776178159.457635:0:3353:0:(events.c:361:request_in_callback()) peer: 12345-0@lo (source: 12345-0@lo) 00000100:00100000:0.0:1776178159.457664:0:8231:0:(service.c:2152:ptlrpc_server_handle_req_in()) got req x1862456998036096 00000100:00100000:0.0:1776178159.457682:0:8231:0:(service.c:2152:ptlrpc_server_handle_req_in()) got req x1862456998036160 00000100:00100000:0.0:1776178159.457692:0:8231:0:(nrs_fifo.c:179:nrs_fifo_req_get()) NRS start fifo request from 12345-0@lo, seq: 443 00000100:00100000:0.0:1776178159.457695:0:8231:0:(service.c:2324:ptlrpc_server_handle_request()) Handling RPC req@00000000e8d76f60 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_create00:lustre-MDT0000-mdtlov_UUID+5:3353:x1862456998036096:12345-0@lo:13:osp-pre-0-0.0 00002000:00000024:0.0:1776178159.457714:0:8231:0:(ofd_obd.c:740:ofd_statfs()) blocks cached 315392 granted 484883776 pending 0 free 3923689472 avail 3692167168 00000100:00100000:0.0:1776178159.457741:0:8231:0:(service.c:2372:ptlrpc_server_handle_request()) Handled RPC req@00000000e8d76f60 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_create00:lustre-MDT0000-mdtlov_UUID+5:3353:x1862456998036096:12345-0@lo:13:osp-pre-0-0.0 Request processed in 45us (145us total) trans 0 rc 0/0 00000100:00100000:0.0:1776178159.457747:0:8231:0:(nrs_fifo.c:241:nrs_fifo_req_stop()) NRS stop fifo request from 12345-0@lo, seq: 443 00000100:00100000:0.0:1776178159.457757:0:8231:0:(nrs_fifo.c:179:nrs_fifo_req_get()) NRS start fifo request from 12345-0@lo, seq: 444 00000100:00100000:0.0:1776178159.457760:0:8231:0:(service.c:2324:ptlrpc_server_handle_request()) Handling RPC req@0000000043f3da0e pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_create00:lustre-MDT0000-mdtlov_UUID+5:3353:x1862456998036160:12345-0@lo:13:osp-pre-1-0.0 00002000:00000024:0.0:1776178159.457775:0:8231:0:(ofd_obd.c:740:ofd_statfs()) blocks cached 229376 granted 491746368 pending 0 free 3923689472 avail 3692167168 00000100:00100000:0.0:1776178159.457797:0:8231:0:(service.c:2372:ptlrpc_server_handle_request()) Handled RPC req@0000000043f3da0e pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_create00:lustre-MDT0000-mdtlov_UUID+5:3353:x1862456998036160:12345-0@lo:13:osp-pre-1-0.0 Request processed in 36us (163us total) trans 0 rc 0/0 00000100:00100000:0.0:1776178159.457803:0:8231:0:(nrs_fifo.c:241:nrs_fifo_req_stop()) NRS stop fifo request from 12345-0@lo, seq: 444 00000100:00100000:0.0:1776178159.457837:0:3355:0:(client.c:2215:ptlrpc_check_set()) Completed RPC req@000000004a9818ac pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_02:ac61f839-0e8e-4cac-9c59-08bcfbbad330:3355:1862456998035968:0@lo:400:kworker/u8:0.0 00000100:00100000:0.0:1776178159.457863:0:3353:0:(client.c:2868:ptlrpc_free_committed()) lustre-OST0001-osc-MDT0000: committing for last_committed 4294969323 gen 2 00000004:00080000:0.0:1776178159.457869:0:3353:0:(osp_sync.c:508:osp_sync_request_commit_cb()) commit req 000000009d5112f6, transno 4294969321 00000004:00080000:0.0:1776178159.457877:0:3353:0:(osp_sync.c:508:osp_sync_request_commit_cb()) commit req 0000000067671fc8, transno 4294969322 00000004:00080000:0.0:1776178159.457879:0:3353:0:(osp_sync.c:508:osp_sync_request_commit_cb()) commit req 0000000023210d1c, transno 4294969323 00000100:00100000:0.0:1776178159.457886:0:3353:0:(client.c:2215:ptlrpc_check_set()) Completed RPC req@000000009a8d7b96 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_00:lustre-MDT0000-mdtlov_UUID:3353:1862456998036160:0@lo:13:osp-pre-1-0.0 00000100:00100000:0.0:1776178159.457893:0:3353:0:(client.c:2868:ptlrpc_free_committed()) lustre-OST0000-osc-MDT0000: committing for last_committed 4294969352 gen 2 00000004:00080000:0.0:1776178159.457896:0:3353:0:(osp_sync.c:508:osp_sync_request_commit_cb()) commit req 000000005643ab1b, transno 4294969349 00000004:00080000:0.0:1776178159.457902:0:3353:0:(osp_sync.c:508:osp_sync_request_commit_cb()) commit req 00000000a59bfc89, transno 4294969350 00000004:00080000:0.0:1776178159.457905:0:3353:0:(osp_sync.c:508:osp_sync_request_commit_cb()) commit req 00000000575d0558, transno 4294969351 00000004:00080000:0.0:1776178159.457906:0:3353:0:(osp_sync.c:508:osp_sync_request_commit_cb()) commit req 00000000ecb64602, transno 4294969352 00000100:00100000:0.0:1776178159.457911:0:3353:0:(client.c:2215:ptlrpc_check_set()) Completed RPC req@000000004eee88be pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_00:lustre-MDT0000-mdtlov_UUID:3353:1862456998036096:0@lo:13:osp-pre-0-0.0 00000040:00100000:0.0:1776178159.457948:0:20150:0:(llog.c:235:llog_cancel_arr_rec()) Canceling 4 records, first 3 in log [0x1:0xc:0x0] 00000004:00080000:0.0:1776178159.458019:0:20150:0:(osp_sync.c:1124:osp_sync_process_committed()) lustre-OST0000-osc-MDT0000: 0 changes, 0 in progress, 0 in flight, 4 done 00000100:00100000:2.0:1776178159.459516:0:3353:0:(client.c:1733:ptlrpc_send_new_req()) Sending RPC req@0000000045560455 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_00:lustre-MDT0001-lwp-OST0000_UUID:3353:1862456998036352:0@lo:400:kworker/u8:0.0 00000100:00100000:2.0:1776178159.459586:0:3353:0:(events.c:361:request_in_callback()) peer: 12345-0@lo (source: 12345-0@lo) 00000100:00100000:2.0:1776178159.459924:0:3354:0:(client.c:1733:ptlrpc_send_new_req()) Sending RPC req@000000007886b426 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_01:lustre-MDT0000-lwp-OST0001_UUID:3354:1862456998036416:0@lo:400:kworker/u8:0.0 00000100:00100000:2.0:1776178159.459952:0:3354:0:(events.c:361:request_in_callback()) peer: 12345-0@lo (source: 12345-0@lo) 00000100:00100000:2.0:1776178159.460367:0:3353:0:(client.c:1733:ptlrpc_send_new_req()) Sending RPC req@00000000165897bf pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_00:lustre-MDT0001-mdtlov_UUID:3353:1862456998036608:0@lo:400:kworker/u8:0.0 00000100:00100000:2.0:1776178159.460400:0:3353:0:(events.c:361:request_in_callback()) peer: 12345-0@lo (source: 12345-0@lo) 00000100:00100000:3.0:1776178159.460880:0:3356:0:(client.c:1733:ptlrpc_send_new_req()) Sending RPC req@000000004a9818ac pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_03:lustre-MDT0000-lwp-OST0000_UUID:3356:1862456998036288:0@lo:400:kworker/u8:0.0 00000100:00100000:3.0:1776178159.460928:0:3356:0:(events.c:361:request_in_callback()) peer: 12345-0@lo (source: 12345-0@lo) 00000100:00100000:3.0:1776178159.460952:0:3356:0:(client.c:1733:ptlrpc_send_new_req()) Sending RPC req@000000002dd26ce5 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_03:lustre-MDT0001-mdtlov_UUID:3356:1862456998036544:0@lo:400:kworker/u8:0.0 00000100:00100000:3.0:1776178159.460979:0:3356:0:(events.c:361:request_in_callback()) peer: 12345-0@lo (source: 12345-0@lo) 00000100:00100000:2.0:1776178159.461118:0:8228:0:(service.c:2152:ptlrpc_server_handle_req_in()) got req x1862456998036608 00000100:00100000:2.0:1776178159.461169:0:8228:0:(nrs_fifo.c:179:nrs_fifo_req_get()) NRS start fifo request from 12345-0@lo, seq: 464 00000100:00100000:2.0:1776178159.461175:0:8228:0:(service.c:2324:ptlrpc_server_handle_request()) Handling RPC req@00000000c8ff7fb4 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_001:lustre-MDT0001-mdtlov_UUID+5:3353:x1862456998036608:12345-0@lo:400:kworker/u8:0.0 00000100:00100000:2.0:1776178159.461857:0:8228:0:(service.c:2372:ptlrpc_server_handle_request()) Handled RPC req@00000000c8ff7fb4 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_001:lustre-MDT0001-mdtlov_UUID+5:3353:x1862456998036608:12345-0@lo:400:kworker/u8:0.0 Request processed in 678us (1454us total) trans 0 rc 0/0 00000100:00100000:2.0:1776178159.461867:0:8228:0:(nrs_fifo.c:241:nrs_fifo_req_stop()) NRS stop fifo request from 12345-0@lo, seq: 464 00000100:00100000:1.0:1776178159.462203:0:14128:0:(service.c:2372:ptlrpc_server_handle_request()) Handled RPC req@00000000574c4997 pname:cluuid+ref:pid:xid:nid:opc:job mdt_out00_002:lustre-MDT0000-mdtlov_UUID+5:3356:x1862456998036032:12345-0@lo:41:osp-pre-1-0.0 Request processed in 5025us (5333us total) trans 0 rc 0/0 00000100:00100000:1.0:1776178159.462214:0:14128:0:(nrs_fifo.c:241:nrs_fifo_req_stop()) NRS stop fifo request from 12345-0@lo, seq: 662 00000100:00100000:1.0:1776178159.462239:0:14128:0:(service.c:2152:ptlrpc_server_handle_req_in()) got req x1862456998036544 00000100:00100000:1.0:1776178159.462275:0:14128:0:(nrs_fifo.c:179:nrs_fifo_req_get()) NRS start fifo request from 12345-0@lo, seq: 663 00000100:00100000:1.0:1776178159.462280:0:14128:0:(service.c:2324:ptlrpc_server_handle_request()) Handling RPC req@00000000c3b47349 pname:cluuid+ref:pid:xid:nid:opc:job mdt_out00_002:lustre-MDT0001-mdtlov_UUID+6:3356:x1862456998036544:12345-0@lo:400:kworker/u8:0.0 00000100:00100000:1.0:1776178159.462319:0:14128:0:(service.c:2372:ptlrpc_server_handle_request()) Handled RPC req@00000000c3b47349 pname:cluuid+ref:pid:xid:nid:opc:job mdt_out00_002:lustre-MDT0001-mdtlov_UUID+6:3356:x1862456998036544:12345-0@lo:400:kworker/u8:0.0 Request processed in 38us (1340us total) trans 0 rc 0/0 00000100:00100000:1.0:1776178159.462324:0:14128:0:(nrs_fifo.c:241:nrs_fifo_req_stop()) NRS stop fifo request from 12345-0@lo, seq: 663 00000100:00100000:1.0:1776178159.462353:0:6255:0:(service.c:2152:ptlrpc_server_handle_req_in()) got req x1862456998036352 00000100:00100000:1.0:1776178159.462379:0:6255:0:(service.c:2152:ptlrpc_server_handle_req_in()) got req x1862456998036416 00000100:00100000:1.0:1776178159.462396:0:6255:0:(service.c:2152:ptlrpc_server_handle_req_in()) got req x1862456998036288 00000100:00100000:1.0:1776178159.462407:0:6255:0:(nrs_fifo.c:179:nrs_fifo_req_get()) NRS start fifo request from 12345-0@lo, seq: 836 00000100:00100000:1.0:1776178159.462411:0:6255:0:(service.c:2324:ptlrpc_server_handle_request()) Handling RPC req@00000000bc95ec51 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_000:lustre-MDT0001-lwp-OST0000_UUID+5:3353:x1862456998036352:12345-0@lo:400:kworker/u8:0.0 00000100:00100000:1.0:1776178159.462438:0:6255:0:(service.c:2372:ptlrpc_server_handle_request()) Handled RPC req@00000000bc95ec51 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_000:lustre-MDT0001-lwp-OST0000_UUID+5:3353:x1862456998036352:12345-0@lo:400:kworker/u8:0.0 Request processed in 27us (2854us total) trans 0 rc 0/0 00000100:00100000:1.0:1776178159.462448:0:6255:0:(nrs_fifo.c:241:nrs_fifo_req_stop()) NRS stop fifo request from 12345-0@lo, seq: 836 00000100:00100000:1.0:1776178159.462459:0:6255:0:(nrs_fifo.c:179:nrs_fifo_req_get()) NRS start fifo request from 12345-0@lo, seq: 837 00000100:00100000:1.0:1776178159.462463:0:6255:0:(service.c:2324:ptlrpc_server_handle_request()) Handling RPC req@000000002b69594e pname:cluuid+ref:pid:xid:nid:opc:job mdt00_000:lustre-MDT0000-lwp-OST0001_UUID+5:3354:x1862456998036416:12345-0@lo:400:kworker/u8:0.0 00000100:00100000:1.0:1776178159.462484:0:6255:0:(service.c:2372:ptlrpc_server_handle_request()) Handled RPC req@000000002b69594e pname:cluuid+ref:pid:xid:nid:opc:job mdt00_000:lustre-MDT0000-lwp-OST0001_UUID+5:3354:x1862456998036416:12345-0@lo:400:kworker/u8:0.0 Request processed in 21us (2533us total) trans 0 rc 0/0 00000100:00100000:1.0:1776178159.462489:0:6255:0:(nrs_fifo.c:241:nrs_fifo_req_stop()) NRS stop fifo request from 12345-0@lo, seq: 837 00000100:00100000:1.0:1776178159.462497:0:6255:0:(nrs_fifo.c:179:nrs_fifo_req_get()) NRS start fifo request from 12345-0@lo, seq: 838 00000100:00100000:1.0:1776178159.462500:0:6255:0:(service.c:2324:ptlrpc_server_handle_request()) Handling RPC req@000000000489d393 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_000:lustre-MDT0000-lwp-OST0000_UUID+5:3356:x1862456998036288:12345-0@lo:400:kworker/u8:0.0 00000100:00100000:1.0:1776178159.462523:0:6255:0:(service.c:2372:ptlrpc_server_handle_request()) Handled RPC req@000000000489d393 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_000:lustre-MDT0000-lwp-OST0000_UUID+5:3356:x1862456998036288:12345-0@lo:400:kworker/u8:0.0 Request processed in 22us (1595us total) trans 0 rc 0/0 00000100:00100000:1.0:1776178159.462529:0:6255:0:(nrs_fifo.c:241:nrs_fifo_req_stop()) NRS stop fifo request from 12345-0@lo, seq: 838 00000100:00100000:2.0:1776178159.462922:0:3354:0:(client.c:2215:ptlrpc_check_set()) Completed RPC req@000000007886b426 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_01:lustre-MDT0000-lwp-OST0001_UUID:3354:1862456998036416:0@lo:400:kworker/u8:0.0 00000100:00100000:2.0:1776178159.462962:0:3353:0:(client.c:2215:ptlrpc_check_set()) Completed RPC req@00000000165897bf pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_00:lustre-MDT0001-mdtlov_UUID:3353:1862456998036608:0@lo:400:kworker/u8:0.0 00000100:00100000:2.0:1776178159.462972:0:3353:0:(client.c:2215:ptlrpc_check_set()) Completed RPC req@0000000045560455 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_00:lustre-MDT0001-lwp-OST0000_UUID:3353:1862456998036352:0@lo:400:kworker/u8:0.0 00000100:00100000:2.0:1776178159.463359:0:3354:0:(client.c:1733:ptlrpc_send_new_req()) Sending RPC req@00000000165897bf pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_01:lustre-MDT0001-mdtlov_UUID:3354:1862456998036672:0@lo:400:kworker/u8:0.0 00000100:00100000:2.0:1776178159.463403:0:3354:0:(events.c:361:request_in_callback()) peer: 12345-0@lo (source: 12345-0@lo) 00000100:00100000:2.0:1776178159.464113:0:20762:0:(service.c:2152:ptlrpc_server_handle_req_in()) got req x1862456998036672 00000100:00100000:2.0:1776178159.464155:0:20762:0:(nrs_fifo.c:179:nrs_fifo_req_get()) NRS start fifo request from 12345-0@lo, seq: 465 00000100:00100000:2.0:1776178159.464160:0:20762:0:(service.c:2324:ptlrpc_server_handle_request()) Handling RPC req@00000000505e62f8 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_013:lustre-MDT0001-mdtlov_UUID+5:3354:x1862456998036672:12345-0@lo:400:kworker/u8:0.0 00000001:00080000:1.0:1776178159.464328:0:18097:0:(tgt_lastrcvd.c:913:tgt_cb_last_committed()) lustre-MDT0001: transno 17179876263 is committed 00000020:00080000:1.0:1776178159.464387:0:18097:0:(update_trans.c:80:top_multiple_thandle_dump()) lustre-MDT0001-osd tmt 00000000bd147a94 refcount 2 committed 0 result 0 batchid 12884901896 00000020:00080000:1.0:1776178159.464392:0:18097:0:(update_trans.c:89:top_multiple_thandle_dump()) st 0000000059a03372 obd lustre-MDT0000-osp-MDT0001 committed 1 started 1 stopped 1 result 0 sub_th 00000000f8ca7cd7 00000020:00080000:1.0:1776178159.464395:0:18097:0:(update_trans.c:94:top_multiple_thandle_dump()) cookie [0x200000bd1:0x1:0x0].8 00000020:00080000:1.0:1776178159.464398:0:18097:0:(update_trans.c:89:top_multiple_thandle_dump()) st 00000000970e1f1b obd lustre-MDT0001-osd committed 1 started 1 stopped 1 result 0 sub_th 000000001946190b 00000020:00080000:1.0:1776178159.464400:0:18097:0:(update_trans.c:94:top_multiple_thandle_dump()) cookie [0x240001b70:0x1:0x0].9 00000020:00080000:1.0:1776178159.464416:0:18116:0:(update_trans.c:1589:distribute_txn_commit_thread()) lustre-MDT0001: batchid: 12884901896 committed batchid 12884901895 00000020:00080000:1.0:1776178159.464468:0:18116:0:(update_trans.c:1589:distribute_txn_commit_thread()) lustre-MDT0001: batchid: 0 committed batchid 12884901895 00000100:00100000:1.0:1776178159.464506:0:3355:0:(client.c:1733:ptlrpc_send_new_req()) Sending RPC req@000000004eee88be pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_02:lustre-MDT0001-mdtlov_UUID:3355:1862456998036224:0@lo:41:osp-pre-0-1.0 00000100:00100000:1.0:1776178159.464543:0:3355:0:(events.c:361:request_in_callback()) peer: 12345-0@lo (source: 12345-0@lo) 00000100:00100000:2.0:1776178159.465325:0:20762:0:(service.c:2372:ptlrpc_server_handle_request()) Handled RPC req@00000000505e62f8 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_013:lustre-MDT0001-mdtlov_UUID+5:3354:x1862456998036672:12345-0@lo:400:kworker/u8:0.0 Request processed in 1162us (1921us total) trans 0 rc 0/0 00000100:00100000:2.0:1776178159.465336:0:20762:0:(nrs_fifo.c:241:nrs_fifo_req_stop()) NRS stop fifo request from 12345-0@lo, seq: 465 00000100:00100000:2.0:1776178159.465387:0:3354:0:(client.c:2215:ptlrpc_check_set()) Completed RPC req@00000000165897bf pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_01:lustre-MDT0001-mdtlov_UUID:3354:1862456998036672:0@lo:400:kworker/u8:0.0 00000040:00100000:2.0:1776178159.465790:0:20152:0:(llog.c:235:llog_cancel_arr_rec()) Canceling 3 records, first 4 in log [0x1:0xe:0x0] 00000004:00080000:2.0:1776178159.465853:0:20152:0:(osp_sync.c:1124:osp_sync_process_committed()) lustre-OST0001-osc-MDT0000: 0 changes, 0 in progress, 0 in flight, 3 done 00000100:00100000:3.0:1776178159.468054:0:3356:0:(client.c:2215:ptlrpc_check_set()) Completed RPC req@00000000a0a4914c pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_03:lustre-MDT0000-mdtlov_UUID:3356:1862456998036032:0@lo:41:osp-pre-1-0.0 00000100:00100000:3.0:1776178159.468118:0:3356:0:(client.c:1733:ptlrpc_send_new_req()) Sending RPC req@0000000025f8393e pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_03:lustre-MDT0000-lwp-MDT0000_UUID:3356:1862456998036800:0@lo:400:kworker/u8:0.0 00000100:00100000:3.0:1776178159.468187:0:3356:0:(events.c:361:request_in_callback()) peer: 12345-0@lo (source: 12345-0@lo) 00000100:00100000:3.0:1776178159.468242:0:6256:0:(service.c:2152:ptlrpc_server_handle_req_in()) got req x1862456998036800 00000100:00100000:3.0:1776178159.468287:0:6256:0:(nrs_fifo.c:179:nrs_fifo_req_get()) NRS start fifo request from 12345-0@lo, seq: 839 00000100:00100000:3.0:1776178159.468292:0:6256:0:(service.c:2324:ptlrpc_server_handle_request()) Handling RPC req@00000000a0a4914c pname:cluuid+ref:pid:xid:nid:opc:job mdt00_001:lustre-MDT0000-lwp-MDT0000_UUID+5:3356:x1862456998036800:12345-0@lo:400:kworker/u8:0.0 00000100:00100000:3.0:1776178159.468336:0:6256:0:(service.c:2372:ptlrpc_server_handle_request()) Handled RPC req@00000000a0a4914c pname:cluuid+ref:pid:xid:nid:opc:job mdt00_001:lustre-MDT0000-lwp-MDT0000_UUID+5:3356:x1862456998036800:12345-0@lo:400:kworker/u8:0.0 Request processed in 43us (159us total) trans 0 rc 0/0 00000100:00100000:3.0:1776178159.468342:0:6256:0:(nrs_fifo.c:241:nrs_fifo_req_stop()) NRS stop fifo request from 12345-0@lo, seq: 839 00010000:00010000:3.0:1776178159.468370:0:3340:0:(ldlm_lock.c:820:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(COS) ns: mdt-lustre-MDT0001_UUID lock: 000000007e7ea26f/0xa40083ab9bf53350 lrc: 3/0,1 mode: COS/COS res: [0x240000402:0x7f7:0x0].0x0 bits 0x2/0x0 rrc: 2 type: IBT gid 0 flags: 0x50210000000000 nid: local remote: 0x0 expref: -99 pid: 8250 timeout: 0 lvb_type: 0 00010000:00010000:3.0:1776178159.468381:0:3340:0:(ldlm_lock.c:879:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: mdt-lustre-MDT0001_UUID lock: 000000007e7ea26f/0xa40083ab9bf53350 lrc: 2/0,0 mode: COS/COS res: [0x240000402:0x7f7:0x0].0x0 bits 0x2/0x0 rrc: 2 type: IBT gid 0 flags: 0x50210400000000 nid: local remote: 0x0 expref: -99 pid: 8250 timeout: 0 lvb_type: 0 00010000:00010000:3.0:1776178159.468388:0:3340:0:(ldlm_lockd.c:1879:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: mdt-lustre-MDT0001_UUID lock: 000000007e7ea26f/0xa40083ab9bf53350 lrc: 3/0,0 mode: COS/COS res: [0x240000402:0x7f7:0x0].0x0 bits 0x2/0x0 rrc: 2 type: IBT gid 0 flags: 0x40210400000000 nid: local remote: 0x0 expref: -99 pid: 8250 timeout: 0 lvb_type: 0 00010000:00010000:3.0:1776178159.468395:0:3340:0:(ldlm_lockd.c:1893:ldlm_handle_bl_callback()) Lock 000000007e7ea26f already unused, calling callback (0000000079e78348) 00010000:00010000:3.0:1776178159.468399:0:3340:0:(ldlm_request.c:364:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: mdt-lustre-MDT0001_UUID lock: 000000007e7ea26f/0xa40083ab9bf53350 lrc: 3/0,0 mode: COS/COS res: [0x240000402:0x7f7:0x0].0x0 bits 0x2/0x0 rrc: 2 type: IBT gid 0 flags: 0x40210400000000 nid: local remote: 0x0 expref: -99 pid: 8250 timeout: 0 lvb_type: 0 00010000:00010000:3.0:1776178159.468406:0:3340:0:(ldlm_request.c:1257:ldlm_cli_cancel_local()) ### server-side local cancel ns: mdt-lustre-MDT0001_UUID lock: 000000007e7ea26f/0xa40083ab9bf53350 lrc: 4/0,0 mode: COS/COS res: [0x240000402:0x7f7:0x0].0x0 bits 0x2/0x0 rrc: 2 type: IBT gid 0 flags: 0x40218400000000 nid: local remote: 0x0 expref: -99 pid: 8250 timeout: 0 lvb_type: 0 00010000:00010000:3.0:1776178159.468421:0:3340:0:(ldlm_inodebits.c:94:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x240000402:0x7f7:0x0].0x0 (000000006b762dc5) 00010000:00010000:3.0:1776178159.468424:0:3340:0:(ldlm_inodebits.c:96:ldlm_reprocess_inodebits_queue()) Hint 2 00010000:00010000:3.0:1776178159.468429:0:3340:0:(ldlm_lockd.c:1903:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: mdt-lustre-MDT0001_UUID lock: 000000007e7ea26f/0xa40083ab9bf53350 lrc: 2/0,0 mode: --/COS res: [0x240000402:0x7f7:0x0].0x0 bits 0x2/0x0 rrc: 2 type: IBT gid 0 flags: 0x44a19400000000 nid: local remote: 0x0 expref: -99 pid: 8250 timeout: 0 lvb_type: 0 00010000:00010000:3.0:1776178159.468435:0:3340:0:(ldlm_lock.c:216:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: mdt-lustre-MDT0001_UUID lock: 000000007e7ea26f/0xa40083ab9bf53350 lrc: 0/0,0 mode: --/COS res: [0x240000402:0x7f7:0x0].0x0 bits 0x2/0x0 rrc: 2 type: IBT gid 0 flags: 0x44a19400000000 nid: local remote: 0x0 expref: -99 pid: 8250 timeout: 0 lvb_type: 0 00010000:00010000:3.0:1776178159.468456:0:3340:0:(ldlm_lock.c:820:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(COS) ns: mdt-lustre-MDT0001_UUID lock: 00000000a71646e9/0xa40083ab9bf5338f lrc: 3/0,1 mode: COS/COS res: [0x240000402:0x7f7:0x0].0xcda302ea bits 0x2/0x0 rrc: 2 type: IBT gid 0 flags: 0x50210000000000 nid: local remote: 0x0 expref: -99 pid: 8250 timeout: 0 lvb_type: 0 00010000:00010000:3.0:1776178159.468462:0:3340:0:(ldlm_lock.c:879:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: mdt-lustre-MDT0001_UUID lock: 00000000a71646e9/0xa40083ab9bf5338f lrc: 2/0,0 mode: COS/COS res: [0x240000402:0x7f7:0x0].0xcda302ea bits 0x2/0x0 rrc: 2 type: IBT gid 0 flags: 0x50210400000000 nid: local remote: 0x0 expref: -99 pid: 8250 timeout: 0 lvb_type: 0 00010000:00010000:3.0:1776178159.468466:0:3340:0:(ldlm_lockd.c:1879:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: mdt-lustre-MDT0001_UUID lock: 00000000a71646e9/0xa40083ab9bf5338f lrc: 3/0,0 mode: COS/COS res: [0x240000402:0x7f7:0x0].0xcda302ea bits 0x2/0x0 rrc: 2 type: IBT gid 0 flags: 0x40210400000000 nid: local remote: 0x0 expref: -99 pid: 8250 timeout: 0 lvb_type: 0 00010000:00010000:3.0:1776178159.468470:0:3340:0:(ldlm_lockd.c:1893:ldlm_handle_bl_callback()) Lock 00000000a71646e9 already unused, calling callback (0000000079e78348) 00010000:00010000:3.0:1776178159.468472:0:3340:0:(ldlm_request.c:364:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: mdt-lustre-MDT0001_UUID lock: 00000000a71646e9/0xa40083ab9bf5338f lrc: 3/0,0 mode: COS/COS res: [0x240000402:0x7f7:0x0].0xcda302ea bits 0x2/0x0 rrc: 2 type: IBT gid 0 flags: 0x40210400000000 nid: local remote: 0x0 expref: -99 pid: 8250 timeout: 0 lvb_type: 0 00010000:00010000:3.0:1776178159.468477:0:3340:0:(ldlm_request.c:1257:ldlm_cli_cancel_local()) ### server-side local cancel ns: mdt-lustre-MDT0001_UUID lock: 00000000a71646e9/0xa40083ab9bf5338f lrc: 4/0,0 mode: COS/COS res: [0x240000402:0x7f7:0x0].0xcda302ea bits 0x2/0x0 rrc: 2 type: IBT gid 0 flags: 0x40218400000000 nid: local remote: 0x0 expref: -99 pid: 8250 timeout: 0 lvb_type: 0 00010000:00010000:3.0:1776178159.468484:0:3340:0:(ldlm_inodebits.c:94:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x240000402:0x7f7:0x0].0xcda302ea (00000000f1a25a38) 00010000:00010000:3.0:1776178159.468486:0:3340:0:(ldlm_inodebits.c:96:ldlm_reprocess_inodebits_queue()) Hint 2 00010000:00010000:3.0:1776178159.468488:0:3340:0:(ldlm_lockd.c:1903:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: mdt-lustre-MDT0001_UUID lock: 00000000a71646e9/0xa40083ab9bf5338f lrc: 2/0,0 mode: --/COS res: [0x240000402:0x7f7:0x0].0xcda302ea bits 0x2/0x0 rrc: 2 type: IBT gid 0 flags: 0x44a19400000000 nid: local remote: 0x0 expref: -99 pid: 8250 timeout: 0 lvb_type: 0 00010000:00010000:3.0:1776178159.468493:0:3340:0:(ldlm_lock.c:216:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: mdt-lustre-MDT0001_UUID lock: 00000000a71646e9/0xa40083ab9bf5338f lrc: 0/0,0 mode: --/COS res: [0x240000402:0x7f7:0x0].0xcda302ea bits 0x2/0x0 rrc: 2 type: IBT gid 0 flags: 0x44a19400000000 nid: local remote: 0x0 expref: -99 pid: 8250 timeout: 0 lvb_type: 0 00010000:00010000:3.0:1776178159.468507:0:3340:0:(ldlm_lock.c:820:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(COS) ns: mdt-lustre-MDT0001_UUID lock: 00000000dcea299c/0xa40083ab9bf533a4 lrc: 3/0,1 mode: COS/COS res: [0x240000402:0x7f8:0x0].0x0 bits 0x2/0x0 rrc: 3 type: IBT gid 0 flags: 0x50210000000000 nid: local remote: 0x0 expref: -99 pid: 8250 timeout: 0 lvb_type: 0 00010000:00010000:3.0:1776178159.468513:0:3340:0:(ldlm_lock.c:879:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: mdt-lustre-MDT0001_UUID lock: 00000000dcea299c/0xa40083ab9bf533a4 lrc: 2/0,0 mode: COS/COS res: [0x240000402:0x7f8:0x0].0x0 bits 0x2/0x0 rrc: 3 type: IBT gid 0 flags: 0x50210400000000 nid: local remote: 0x0 expref: -99 pid: 8250 timeout: 0 lvb_type: 0 00010000:00010000:3.0:1776178159.468518:0:3340:0:(ldlm_lockd.c:1879:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: mdt-lustre-MDT0001_UUID lock: 00000000dcea299c/0xa40083ab9bf533a4 lrc: 3/0,0 mode: COS/COS res: [0x240000402:0x7f8:0x0].0x0 bits 0x2/0x0 rrc: 3 type: IBT gid 0 flags: 0x40210400000000 nid: local remote: 0x0 expref: -99 pid: 8250 timeout: 0 lvb_type: 0 00010000:00010000:3.0:1776178159.468524:0:3340:0:(ldlm_lockd.c:1893:ldlm_handle_bl_callback()) Lock 00000000dcea299c already unused, calling callback (0000000079e78348) 00010000:00010000:3.0:1776178159.468526:0:3340:0:(ldlm_request.c:364:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: mdt-lustre-MDT0001_UUID lock: 00000000dcea299c/0xa40083ab9bf533a4 lrc: 3/0,0 mode: COS/COS res: [0x240000402:0x7f8:0x0].0x0 bits 0x2/0x0 rrc: 3 type: IBT gid 0 flags: 0x40210400000000 nid: local remote: 0x0 expref: -99 pid: 8250 timeout: 0 lvb_type: 0 00010000:00010000:3.0:1776178159.468532:0:3340:0:(ldlm_request.c:1257:ldlm_cli_cancel_local()) ### server-side local cancel ns: mdt-lustre-MDT0001_UUID lock: 00000000dcea299c/0xa40083ab9bf533a4 lrc: 4/0,0 mode: COS/COS res: [0x240000402:0x7f8:0x0].0x0 bits 0x2/0x0 rrc: 3 type: IBT gid 0 flags: 0x40218400000000 nid: local remote: 0x0 expref: -99 pid: 8250 timeout: 0 lvb_type: 0 00000100:00100000:1.0:1776178159.468621:0:3355:0:(client.c:1733:ptlrpc_send_new_req()) Sending RPC req@000000006c748d80 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_02:lustre-MDT0001-lwp-OST0001_UUID:3355:1862456998036480:0@lo:400:kworker/u8:0.0 00000100:00100000:1.0:1776178159.468648:0:3355:0:(events.c:361:request_in_callback()) peer: 12345-0@lo (source: 12345-0@lo) 00000100:00100000:1.0:1776178159.468663:0:3355:0:(client.c:1733:ptlrpc_send_new_req()) Sending RPC req@00000000165897bf pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_02:lustre-MDT0000-lwp-MDT0001_UUID:3355:1862456998036736:0@lo:400:kworker/u8:0.0 00000100:00100000:1.0:1776178159.468681:0:3355:0:(events.c:361:request_in_callback()) peer: 12345-0@lo (source: 12345-0@lo) 00000100:00100000:0.0:1776178159.469050:0:8250:0:(service.c:2152:ptlrpc_server_handle_req_in()) got req x1862456998036480 00000100:00100000:0.0:1776178159.469086:0:8250:0:(service.c:2152:ptlrpc_server_handle_req_in()) got req x1862456998036736 00000100:00100000:0.0:1776178159.469101:0:8250:0:(nrs_fifo.c:179:nrs_fifo_req_get()) NRS start fifo request from 12345-0@lo, seq: 840 00000100:00100000:0.0:1776178159.469105:0:8250:0:(service.c:2324:ptlrpc_server_handle_request()) Handling RPC req@00000000d384e871 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_003:lustre-MDT0001-lwp-OST0001_UUID+5:3355:x1862456998036480:12345-0@lo:400:kworker/u8:0.0 00000100:00100000:0.0:1776178159.469140:0:8250:0:(service.c:2372:ptlrpc_server_handle_request()) Handled RPC req@00000000d384e871 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_003:lustre-MDT0001-lwp-OST0001_UUID+5:3355:x1862456998036480:12345-0@lo:400:kworker/u8:0.0 Request processed in 34us (492us total) trans 0 rc 0/0 00000100:00100000:0.0:1776178159.469149:0:8250:0:(nrs_fifo.c:241:nrs_fifo_req_stop()) NRS stop fifo request from 12345-0@lo, seq: 840 00000100:00100000:0.0:1776178159.469160:0:8250:0:(nrs_fifo.c:179:nrs_fifo_req_get()) NRS start fifo request from 12345-0@lo, seq: 841 00000100:00100000:0.0:1776178159.469163:0:8250:0:(service.c:2324:ptlrpc_server_handle_request()) Handling RPC req@00000000664eb80b pname:cluuid+ref:pid:xid:nid:opc:job mdt00_003:lustre-MDT0000-lwp-MDT0001_UUID+5:3355:x1862456998036736:12345-0@lo:400:kworker/u8:0.0 00000100:00100000:0.0:1776178159.469182:0:8250:0:(service.c:2372:ptlrpc_server_handle_request()) Handled RPC req@00000000664eb80b pname:cluuid+ref:pid:xid:nid:opc:job mdt00_003:lustre-MDT0000-lwp-MDT0001_UUID+5:3355:x1862456998036736:12345-0@lo:400:kworker/u8:0.0 Request processed in 19us (502us total) trans 0 rc 0/0 00000100:00100000:0.0:1776178159.469187:0:8250:0:(nrs_fifo.c:241:nrs_fifo_req_stop()) NRS stop fifo request from 12345-0@lo, seq: 841 00000100:00100000:0.0:1776178159.469225:0:3355:0:(client.c:2215:ptlrpc_check_set()) Completed RPC req@00000000165897bf pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_02:lustre-MDT0000-lwp-MDT0001_UUID:3355:1862456998036736:0@lo:400:kworker/u8:0.0 00000100:00100000:0.0:1776178159.469235:0:3355:0:(client.c:2215:ptlrpc_check_set()) Completed RPC req@000000006c748d80 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_02:lustre-MDT0001-lwp-OST0001_UUID:3355:1862456998036480:0@lo:400:kworker/u8:0.0 00010000:00010000:3.0:1776178159.469243:0:3340:0:(ldlm_inodebits.c:94:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x240000402:0x7f8:0x0].0x0 (00000000099618da) 00010000:00010000:3.0:1776178159.469248:0:3340:0:(ldlm_inodebits.c:96:ldlm_reprocess_inodebits_queue()) Hint 2 00010000:00010000:3.0:1776178159.469253:0:3340:0:(ldlm_lockd.c:1903:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: mdt-lustre-MDT0001_UUID lock: 00000000dcea299c/0xa40083ab9bf533a4 lrc: 2/0,0 mode: --/COS res: [0x240000402:0x7f8:0x0].0x0 bits 0x2/0x0 rrc: 3 type: IBT gid 0 flags: 0x44a19400000000 nid: local remote: 0x0 expref: -99 pid: 8250 timeout: 0 lvb_type: 0 00010000:00010000:3.0:1776178159.469260:0:3340:0:(ldlm_lock.c:216:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: mdt-lustre-MDT0001_UUID lock: 00000000dcea299c/0xa40083ab9bf533a4 lrc: 0/0,0 mode: --/COS res: [0x240000402:0x7f8:0x0].0x0 bits 0x2/0x0 rrc: 3 type: IBT gid 0 flags: 0x44a19400000000 nid: local remote: 0x0 expref: -99 pid: 8250 timeout: 0 lvb_type: 0 00010000:00010000:3.0:1776178159.469274:0:3340:0:(ldlm_lock.c:820:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(COS) ns: mdt-lustre-MDT0001_UUID lock: 000000002f317bcb/0xa40083ab9bf533b9 lrc: 3/0,1 mode: COS/COS res: [0x240000402:0x7f7:0x0].0x15fcbbea bits 0x2/0x0 rrc: 2 type: IBT gid 0 flags: 0x50210000000000 nid: local remote: 0x0 expref: -99 pid: 8250 timeout: 0 lvb_type: 0 00010000:00010000:3.0:1776178159.469278:0:3340:0:(ldlm_lock.c:879:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: mdt-lustre-MDT0001_UUID lock: 000000002f317bcb/0xa40083ab9bf533b9 lrc: 2/0,0 mode: COS/COS res: [0x240000402:0x7f7:0x0].0x15fcbbea bits 0x2/0x0 rrc: 2 type: IBT gid 0 flags: 0x50210400000000 nid: local remote: 0x0 expref: -99 pid: 8250 timeout: 0 lvb_type: 0 00010000:00010000:3.0:1776178159.469283:0:3340:0:(ldlm_lockd.c:1879:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: mdt-lustre-MDT0001_UUID lock: 000000002f317bcb/0xa40083ab9bf533b9 lrc: 3/0,0 mode: COS/COS res: [0x240000402:0x7f7:0x0].0x15fcbbea bits 0x2/0x0 rrc: 2 type: IBT gid 0 flags: 0x40210400000000 nid: local remote: 0x0 expref: -99 pid: 8250 timeout: 0 lvb_type: 0 00010000:00010000:3.0:1776178159.469293:0:3340:0:(ldlm_lockd.c:1893:ldlm_handle_bl_callback()) Lock 000000002f317bcb already unused, calling callback (0000000079e78348) 00010000:00010000:3.0:1776178159.469295:0:3340:0:(ldlm_request.c:364:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: mdt-lustre-MDT0001_UUID lock: 000000002f317bcb/0xa40083ab9bf533b9 lrc: 3/0,0 mode: COS/COS res: [0x240000402:0x7f7:0x0].0x15fcbbea bits 0x2/0x0 rrc: 2 type: IBT gid 0 flags: 0x40210400000000 nid: local remote: 0x0 expref: -99 pid: 8250 timeout: 0 lvb_type: 0 00010000:00010000:3.0:1776178159.469300:0:3340:0:(ldlm_request.c:1257:ldlm_cli_cancel_local()) ### server-side local cancel ns: mdt-lustre-MDT0001_UUID lock: 000000002f317bcb/0xa40083ab9bf533b9 lrc: 4/0,0 mode: COS/COS res: [0x240000402:0x7f7:0x0].0x15fcbbea bits 0x2/0x0 rrc: 2 type: IBT gid 0 flags: 0x40218400000000 nid: local remote: 0x0 expref: -99 pid: 8250 timeout: 0 lvb_type: 0 00010000:00010000:3.0:1776178159.469308:0:3340:0:(ldlm_inodebits.c:94:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x240000402:0x7f7:0x0].0x15fcbbea (000000005a2060ba) 00010000:00010000:3.0:1776178159.469310:0:3340:0:(ldlm_inodebits.c:96:ldlm_reprocess_inodebits_queue()) Hint 2 00010000:00010000:3.0:1776178159.469312:0:3340:0:(ldlm_lockd.c:1903:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: mdt-lustre-MDT0001_UUID lock: 000000002f317bcb/0xa40083ab9bf533b9 lrc: 2/0,0 mode: --/COS res: [0x240000402:0x7f7:0x0].0x15fcbbea bits 0x2/0x0 rrc: 2 type: IBT gid 0 flags: 0x44a19400000000 nid: local remote: 0x0 expref: -99 pid: 8250 timeout: 0 lvb_type: 0 00010000:00010000:3.0:1776178159.469318:0:3340:0:(ldlm_lock.c:216:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: mdt-lustre-MDT0001_UUID lock: 000000002f317bcb/0xa40083ab9bf533b9 lrc: 0/0,0 mode: --/COS res: [0x240000402:0x7f7:0x0].0x15fcbbea bits 0x2/0x0 rrc: 2 type: IBT gid 0 flags: 0x44a19400000000 nid: local remote: 0x0 expref: -99 pid: 8250 timeout: 0 lvb_type: 0 00010000:00010000:3.0:1776178159.469331:0:3340:0:(ldlm_lock.c:820:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(COS) ns: mdt-lustre-MDT0001_UUID lock: 0000000050323271/0xa40083ab9bf533ce lrc: 3/0,1 mode: COS/COS res: [0x240000402:0x7f9:0x0].0x0 bits 0x2/0x0 rrc: 3 type: IBT gid 0 flags: 0x50210000000000 nid: local remote: 0x0 expref: -99 pid: 8250 timeout: 0 lvb_type: 0 00010000:00010000:3.0:1776178159.469336:0:3340:0:(ldlm_lock.c:879:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: mdt-lustre-MDT0001_UUID lock: 0000000050323271/0xa40083ab9bf533ce lrc: 2/0,0 mode: COS/COS res: [0x240000402:0x7f9:0x0].0x0 bits 0x2/0x0 rrc: 3 type: IBT gid 0 flags: 0x50210400000000 nid: local remote: 0x0 expref: -99 pid: 8250 timeout: 0 lvb_type: 0 00010000:00010000:3.0:1776178159.469341:0:3340:0:(ldlm_lockd.c:1879:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: mdt-lustre-MDT0001_UUID lock: 0000000050323271/0xa40083ab9bf533ce lrc: 3/0,0 mode: COS/COS res: [0x240000402:0x7f9:0x0].0x0 bits 0x2/0x0 rrc: 3 type: IBT gid 0 flags: 0x40210400000000 nid: local remote: 0x0 expref: -99 pid: 8250 timeout: 0 lvb_type: 0 00010000:00010000:3.0:1776178159.469346:0:3340:0:(ldlm_lockd.c:1893:ldlm_handle_bl_callback()) Lock 0000000050323271 already unused, calling callback (0000000079e78348) 00010000:00010000:3.0:1776178159.469348:0:3340:0:(ldlm_request.c:364:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: mdt-lustre-MDT0001_UUID lock: 0000000050323271/0xa40083ab9bf533ce lrc: 3/0,0 mode: COS/COS res: [0x240000402:0x7f9:0x0].0x0 bits 0x2/0x0 rrc: 3 type: IBT gid 0 flags: 0x40210400000000 nid: local remote: 0x0 expref: -99 pid: 8250 timeout: 0 lvb_type: 0 00010000:00010000:3.0:1776178159.469353:0:3340:0:(ldlm_request.c:1257:ldlm_cli_cancel_local()) ### server-side local cancel ns: mdt-lustre-MDT0001_UUID lock: 0000000050323271/0xa40083ab9bf533ce lrc: 4/0,0 mode: COS/COS res: [0x240000402:0x7f9:0x0].0x0 bits 0x2/0x0 rrc: 3 type: IBT gid 0 flags: 0x40218400000000 nid: local remote: 0x0 expref: -99 pid: 8250 timeout: 0 lvb_type: 0 00010000:00010000:3.0:1776178159.469360:0:3340:0:(ldlm_inodebits.c:94:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x240000402:0x7f9:0x0].0x0 (00000000f193fa59) 00010000:00010000:3.0:1776178159.469362:0:3340:0:(ldlm_inodebits.c:96:ldlm_reprocess_inodebits_queue()) Hint 2 00010000:00010000:3.0:1776178159.469363:0:3340:0:(ldlm_lockd.c:1903:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: mdt-lustre-MDT0001_UUID lock: 0000000050323271/0xa40083ab9bf533ce lrc: 2/0,0 mode: --/COS res: [0x240000402:0x7f9:0x0].0x0 bits 0x2/0x0 rrc: 3 type: IBT gid 0 flags: 0x44a19400000000 nid: local remote: 0x0 expref: -99 pid: 8250 timeout: 0 lvb_type: 0 00010000:00010000:3.0:1776178159.469368:0:3340:0:(ldlm_lock.c:216:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: mdt-lustre-MDT0001_UUID lock: 0000000050323271/0xa40083ab9bf533ce lrc: 0/0,0 mode: --/COS res: [0x240000402:0x7f9:0x0].0x0 bits 0x2/0x0 rrc: 3 type: IBT gid 0 flags: 0x44a19400000000 nid: local remote: 0x0 expref: -99 pid: 8250 timeout: 0 lvb_type: 0 00010000:00010000:3.0:1776178159.469378:0:3340:0:(ldlm_lock.c:820:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(COS) ns: mdt-lustre-MDT0001_UUID lock: 00000000c1e7ea3e/0xa40083ab9bf533e3 lrc: 3/0,1 mode: COS/COS res: [0x240000402:0x7f7:0x0].0x5e5674ea bits 0x2/0x0 rrc: 2 type: IBT gid 0 flags: 0x50210000000000 nid: local remote: 0x0 expref: -99 pid: 8250 timeout: 0 lvb_type: 0 00010000:00010000:3.0:1776178159.469382:0:3340:0:(ldlm_lock.c:879:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: mdt-lustre-MDT0001_UUID lock: 00000000c1e7ea3e/0xa40083ab9bf533e3 lrc: 2/0,0 mode: COS/COS res: [0x240000402:0x7f7:0x0].0x5e5674ea bits 0x2/0x0 rrc: 2 type: IBT gid 0 flags: 0x50210400000000 nid: local remote: 0x0 expref: -99 pid: 8250 timeout: 0 lvb_type: 0 00010000:00010000:3.0:1776178159.469387:0:3340:0:(ldlm_lockd.c:1879:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: mdt-lustre-MDT0001_UUID lock: 00000000c1e7ea3e/0xa40083ab9bf533e3 lrc: 3/0,0 mode: COS/COS res: [0x240000402:0x7f7:0x0].0x5e5674ea bits 0x2/0x0 rrc: 2 type: IBT gid 0 flags: 0x40210400000000 nid: local remote: 0x0 expref: -99 pid: 8250 timeout: 0 lvb_type: 0 00010000:00010000:3.0:1776178159.469392:0:3340:0:(ldlm_lockd.c:1893:ldlm_handle_bl_callback()) Lock 00000000c1e7ea3e already unused, calling callback (0000000079e78348) 00010000:00010000:3.0:1776178159.469394:0:3340:0:(ldlm_request.c:364:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: mdt-lustre-MDT0001_UUID lock: 00000000c1e7ea3e/0xa40083ab9bf533e3 lrc: 3/0,0 mode: COS/COS res: [0x240000402:0x7f7:0x0].0x5e5674ea bits 0x2/0x0 rrc: 2 type: IBT gid 0 flags: 0x40210400000000 nid: local remote: 0x0 expref: -99 pid: 8250 timeout: 0 lvb_type: 0 00010000:00010000:3.0:1776178159.469399:0:3340:0:(ldlm_request.c:1257:ldlm_cli_cancel_local()) ### server-side local cancel ns: mdt-lustre-MDT0001_UUID lock: 00000000c1e7ea3e/0xa40083ab9bf533e3 lrc: 4/0,0 mode: COS/COS res: [0x240000402:0x7f7:0x0].0x5e5674ea bits 0x2/0x0 rrc: 2 type: IBT gid 0 flags: 0x40218400000000 nid: local remote: 0x0 expref: -99 pid: 8250 timeout: 0 lvb_type: 0 00010000:00010000:3.0:1776178159.469406:0:3340:0:(ldlm_inodebits.c:94:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x240000402:0x7f7:0x0].0x5e5674ea (000000007b7d95c1) 00010000:00010000:3.0:1776178159.469408:0:3340:0:(ldlm_inodebits.c:96:ldlm_reprocess_inodebits_queue()) Hint 2 00010000:00010000:3.0:1776178159.469410:0:3340:0:(ldlm_lockd.c:1903:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: mdt-lustre-MDT0001_UUID lock: 00000000c1e7ea3e/0xa40083ab9bf533e3 lrc: 2/0,0 mode: --/COS res: [0x240000402:0x7f7:0x0].0x5e5674ea bits 0x2/0x0 rrc: 2 type: IBT gid 0 flags: 0x44a19400000000 nid: local remote: 0x0 expref: -99 pid: 8250 timeout: 0 lvb_type: 0 00010000:00010000:3.0:1776178159.469420:0:3340:0:(ldlm_lock.c:216:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: mdt-lustre-MDT0001_UUID lock: 00000000c1e7ea3e/0xa40083ab9bf533e3 lrc: 0/0,0 mode: --/COS res: [0x240000402:0x7f7:0x0].0x5e5674ea bits 0x2/0x0 rrc: 2 type: IBT gid 0 flags: 0x44a19400000000 nid: local remote: 0x0 expref: -99 pid: 8250 timeout: 0 lvb_type: 0 00010000:00010000:3.0:1776178159.469434:0:3340:0:(ldlm_lock.c:820:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(COS) ns: mdt-lustre-MDT0001_UUID lock: 00000000b27cf00e/0xa40083ab9bf533f8 lrc: 3/0,1 mode: COS/COS res: [0x240000402:0x7fa:0x0].0x0 bits 0x2/0x0 rrc: 3 type: IBT gid 0 flags: 0x50210000000000 nid: local remote: 0x0 expref: -99 pid: 6255 timeout: 0 lvb_type: 0 00010000:00010000:3.0:1776178159.469440:0:3340:0:(ldlm_lock.c:879:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: mdt-lustre-MDT0001_UUID lock: 00000000b27cf00e/0xa40083ab9bf533f8 lrc: 2/0,0 mode: COS/COS res: [0x240000402:0x7fa:0x0].0x0 bits 0x2/0x0 rrc: 3 type: IBT gid 0 flags: 0x50210400000000 nid: local remote: 0x0 expref: -99 pid: 6255 timeout: 0 lvb_type: 0 00010000:00010000:3.0:1776178159.469445:0:3340:0:(ldlm_lockd.c:1879:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: mdt-lustre-MDT0001_UUID lock: 00000000b27cf00e/0xa40083ab9bf533f8 lrc: 3/0,0 mode: COS/COS res: [0x240000402:0x7fa:0x0].0x0 bits 0x2/0x0 rrc: 3 type: IBT gid 0 flags: 0x40210400000000 nid: local remote: 0x0 expref: -99 pid: 6255 timeout: 0 lvb_type: 0 00010000:00010000:3.0:1776178159.469451:0:3340:0:(ldlm_lockd.c:1893:ldlm_handle_bl_callback()) Lock 00000000b27cf00e already unused, calling callback (0000000079e78348) 00010000:00010000:3.0:1776178159.469454:0:3340:0:(ldlm_request.c:364:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: mdt-lustre-MDT0001_UUID lock: 00000000b27cf00e/0xa40083ab9bf533f8 lrc: 3/0,0 mode: COS/COS res: [0x240000402:0x7fa:0x0].0x0 bits 0x2/0x0 rrc: 3 type: IBT gid 0 flags: 0x40210400000000 nid: local remote: 0x0 expref: -99 pid: 6255 timeout: 0 lvb_type: 0 00010000:00010000:3.0:1776178159.469459:0:3340:0:(ldlm_request.c:1257:ldlm_cli_cancel_local()) ### server-side local cancel ns: mdt-lustre-MDT0001_UUID lock: 00000000b27cf00e/0xa40083ab9bf533f8 lrc: 4/0,0 mode: COS/COS res: [0x240000402:0x7fa:0x0].0x0 bits 0x2/0x0 rrc: 3 type: IBT gid 0 flags: 0x40218400000000 nid: local remote: 0x0 expref: -99 pid: 6255 timeout: 0 lvb_type: 0 00010000:00010000:3.0:1776178159.469468:0:3340:0:(ldlm_inodebits.c:94:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x240000402:0x7fa:0x0].0x0 (000000006b0fcbcb) 00010000:00010000:3.0:1776178159.469470:0:3340:0:(ldlm_inodebits.c:96:ldlm_reprocess_inodebits_queue()) Hint 2 00010000:00010000:3.0:1776178159.469472:0:3340:0:(ldlm_lockd.c:1903:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: mdt-lustre-MDT0001_UUID lock: 00000000b27cf00e/0xa40083ab9bf533f8 lrc: 2/0,0 mode: --/COS res: [0x240000402:0x7fa:0x0].0x0 bits 0x2/0x0 rrc: 3 type: IBT gid 0 flags: 0x44a19400000000 nid: local remote: 0x0 expref: -99 pid: 6255 timeout: 0 lvb_type: 0 00010000:00010000:3.0:1776178159.469478:0:3340:0:(ldlm_lock.c:216:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: mdt-lustre-MDT0001_UUID lock: 00000000b27cf00e/0xa40083ab9bf533f8 lrc: 0/0,0 mode: --/COS res: [0x240000402:0x7fa:0x0].0x0 bits 0x2/0x0 rrc: 3 type: IBT gid 0 flags: 0x44a19400000000 nid: local remote: 0x0 expref: -99 pid: 6255 timeout: 0 lvb_type: 0 00010000:00010000:3.0:1776178159.469489:0:3340:0:(ldlm_lock.c:820:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(COS) ns: mdt-lustre-MDT0001_UUID lock: 000000005dc671a2/0xa40083ab9bf5340d lrc: 3/0,1 mode: COS/COS res: [0x240000402:0x7f7:0x0].0xa6b02dea bits 0x2/0x0 rrc: 2 type: IBT gid 0 flags: 0x50210000000000 nid: local remote: 0x0 expref: -99 pid: 8250 timeout: 0 lvb_type: 0 00010000:00010000:3.0:1776178159.469498:0:3340:0:(ldlm_lock.c:879:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: mdt-lustre-MDT0001_UUID lock: 000000005dc671a2/0xa40083ab9bf5340d lrc: 2/0,0 mode: COS/COS res: [0x240000402:0x7f7:0x0].0xa6b02dea bits 0x2/0x0 rrc: 2 type: IBT gid 0 flags: 0x50210400000000 nid: local remote: 0x0 expref: -99 pid: 8250 timeout: 0 lvb_type: 0 00010000:00010000:3.0:1776178159.469504:0:3340:0:(ldlm_lockd.c:1879:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: mdt-lustre-MDT0001_UUID lock: 000000005dc671a2/0xa40083ab9bf5340d lrc: 3/0,0 mode: COS/COS res: [0x240000402:0x7f7:0x0].0xa6b02dea bits 0x2/0x0 rrc: 2 type: IBT gid 0 flags: 0x40210400000000 nid: local remote: 0x0 expref: -99 pid: 8250 timeout: 0 lvb_type: 0 00010000:00010000:3.0:1776178159.469509:0:3340:0:(ldlm_lockd.c:1893:ldlm_handle_bl_callback()) Lock 000000005dc671a2 already unused, calling callback (0000000079e78348) 00010000:00010000:3.0:1776178159.469510:0:3340:0:(ldlm_request.c:364:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: mdt-lustre-MDT0001_UUID lock: 000000005dc671a2/0xa40083ab9bf5340d lrc: 3/0,0 mode: COS/COS res: [0x240000402:0x7f7:0x0].0xa6b02dea bits 0x2/0x0 rrc: 2 type: IBT gid 0 flags: 0x40210400000000 nid: local remote: 0x0 expref: -99 pid: 8250 timeout: 0 lvb_type: 0 00010000:00010000:3.0:1776178159.469516:0:3340:0:(ldlm_request.c:1257:ldlm_cli_cancel_local()) ### server-side local cancel ns: mdt-lustre-MDT0001_UUID lock: 000000005dc671a2/0xa40083ab9bf5340d lrc: 4/0,0 mode: COS/COS res: [0x240000402:0x7f7:0x0].0xa6b02dea bits 0x2/0x0 rrc: 2 type: IBT gid 0 flags: 0x40218400000000 nid: local remote: 0x0 expref: -99 pid: 8250 timeout: 0 lvb_type: 0 00010000:00010000:3.0:1776178159.469524:0:3340:0:(ldlm_inodebits.c:94:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x240000402:0x7f7:0x0].0xa6b02dea (0000000048823789) 00010000:00010000:3.0:1776178159.469526:0:3340:0:(ldlm_inodebits.c:96:ldlm_reprocess_inodebits_queue()) Hint 2 00010000:00010000:3.0:1776178159.469528:0:3340:0:(ldlm_lockd.c:1903:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: mdt-lustre-MDT0001_UUID lock: 000000005dc671a2/0xa40083ab9bf5340d lrc: 2/0,0 mode: --/COS res: [0x240000402:0x7f7:0x0].0xa6b02dea bits 0x2/0x0 rrc: 2 type: IBT gid 0 flags: 0x44a19400000000 nid: local remote: 0x0 expref: -99 pid: 8250 timeout: 0 lvb_type: 0 00010000:00010000:3.0:1776178159.469533:0:3340:0:(ldlm_lock.c:216:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: mdt-lustre-MDT0001_UUID lock: 000000005dc671a2/0xa40083ab9bf5340d lrc: 0/0,0 mode: --/COS res: [0x240000402:0x7f7:0x0].0xa6b02dea bits 0x2/0x0 rrc: 2 type: IBT gid 0 flags: 0x44a19400000000 nid: local remote: 0x0 expref: -99 pid: 8250 timeout: 0 lvb_type: 0 00010000:00010000:3.0:1776178159.469562:0:3340:0:(ldlm_lock.c:820:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(COS) ns: mdt-lustre-MDT0001_UUID lock: 000000007e5ef416/0xa40083ab9bf53422 lrc: 3/0,1 mode: COS/COS res: [0x240000402:0x7fb:0x0].0x0 bits 0x2/0x0 rrc: 3 type: IBT gid 0 flags: 0x50210000000000 nid: local remote: 0x0 expref: -99 pid: 6255 timeout: 0 lvb_type: 0 00010000:00010000:3.0:1776178159.469569:0:3340:0:(ldlm_lock.c:879:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: mdt-lustre-MDT0001_UUID lock: 000000007e5ef416/0xa40083ab9bf53422 lrc: 2/0,0 mode: COS/COS res: [0x240000402:0x7fb:0x0].0x0 bits 0x2/0x0 rrc: 3 type: IBT gid 0 flags: 0x50210400000000 nid: local remote: 0x0 expref: -99 pid: 6255 timeout: 0 lvb_type: 0 00010000:00010000:3.0:1776178159.469574:0:3340:0:(ldlm_lockd.c:1879:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: mdt-lustre-MDT0001_UUID lock: 000000007e5ef416/0xa40083ab9bf53422 lrc: 3/0,0 mode: COS/COS res: [0x240000402:0x7fb:0x0].0x0 bits 0x2/0x0 rrc: 3 type: IBT gid 0 flags: 0x40210400000000 nid: local remote: 0x0 expref: -99 pid: 6255 timeout: 0 lvb_type: 0 00010000:00010000:3.0:1776178159.469584:0:3340:0:(ldlm_lockd.c:1893:ldlm_handle_bl_callback()) Lock 000000007e5ef416 already unused, calling callback (0000000079e78348) 00010000:00010000:3.0:1776178159.469586:0:3340:0:(ldlm_request.c:364:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: mdt-lustre-MDT0001_UUID lock: 000000007e5ef416/0xa40083ab9bf53422 lrc: 3/0,0 mode: COS/COS res: [0x240000402:0x7fb:0x0].0x0 bits 0x2/0x0 rrc: 3 type: IBT gid 0 flags: 0x40210400000000 nid: local remote: 0x0 expref: -99 pid: 6255 timeout: 0 lvb_type: 0 00010000:00010000:3.0:1776178159.469591:0:3340:0:(ldlm_request.c:1257:ldlm_cli_cancel_local()) ### server-side local cancel ns: mdt-lustre-MDT0001_UUID lock: 000000007e5ef416/0xa40083ab9bf53422 lrc: 4/0,0 mode: COS/COS res: [0x240000402:0x7fb:0x0].0x0 bits 0x2/0x0 rrc: 3 type: IBT gid 0 flags: 0x40218400000000 nid: local remote: 0x0 expref: -99 pid: 6255 timeout: 0 lvb_type: 0 00010000:00010000:3.0:1776178159.469598:0:3340:0:(ldlm_inodebits.c:94:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x240000402:0x7fb:0x0].0x0 (000000009f7659df) 00010000:00010000:3.0:1776178159.469600:0:3340:0:(ldlm_inodebits.c:96:ldlm_reprocess_inodebits_queue()) Hint 2 00010000:00010000:3.0:1776178159.469602:0:3340:0:(ldlm_lockd.c:1903:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: mdt-lustre-MDT0001_UUID lock: 000000007e5ef416/0xa40083ab9bf53422 lrc: 2/0,0 mode: --/COS res: [0x240000402:0x7fb:0x0].0x0 bits 0x2/0x0 rrc: 3 type: IBT gid 0 flags: 0x44a19400000000 nid: local remote: 0x0 expref: -99 pid: 6255 timeout: 0 lvb_type: 0 00010000:00010000:3.0:1776178159.469608:0:3340:0:(ldlm_lock.c:216:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: mdt-lustre-MDT0001_UUID lock: 000000007e5ef416/0xa40083ab9bf53422 lrc: 0/0,0 mode: --/COS res: [0x240000402:0x7fb:0x0].0x0 bits 0x2/0x0 rrc: 3 type: IBT gid 0 flags: 0x44a19400000000 nid: local remote: 0x0 expref: -99 pid: 6255 timeout: 0 lvb_type: 0 00010000:00010000:3.0:1776178159.469618:0:3340:0:(ldlm_lock.c:820:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(COS) ns: mdt-lustre-MDT0001_UUID lock: 000000007ee917b5/0xa40083ab9bf53437 lrc: 3/0,1 mode: COS/COS res: [0x240000402:0x7f7:0x0].0xef09e6ea bits 0x2/0x0 rrc: 2 type: IBT gid 0 flags: 0x50210000000000 nid: local remote: 0x0 expref: -99 pid: 6255 timeout: 0 lvb_type: 0 00010000:00010000:3.0:1776178159.469624:0:3340:0:(ldlm_lock.c:879:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: mdt-lustre-MDT0001_UUID lock: 000000007ee917b5/0xa40083ab9bf53437 lrc: 2/0,0 mode: COS/COS res: [0x240000402:0x7f7:0x0].0xef09e6ea bits 0x2/0x0 rrc: 2 type: IBT gid 0 flags: 0x50210400000000 nid: local remote: 0x0 expref: -99 pid: 6255 timeout: 0 lvb_type: 0 00010000:00010000:3.0:1776178159.469629:0:3340:0:(ldlm_lockd.c:1879:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: mdt-lustre-MDT0001_UUID lock: 000000007ee917b5/0xa40083ab9bf53437 lrc: 3/0,0 mode: COS/COS res: [0x240000402:0x7f7:0x0].0xef09e6ea bits 0x2/0x0 rrc: 2 type: IBT gid 0 flags: 0x40210400000000 nid: local remote: 0x0 expref: -99 pid: 6255 timeout: 0 lvb_type: 0 00010000:00010000:3.0:1776178159.469635:0:3340:0:(ldlm_lockd.c:1893:ldlm_handle_bl_callback()) Lock 000000007ee917b5 already unused, calling callback (0000000079e78348) 00010000:00010000:3.0:1776178159.469638:0:3340:0:(ldlm_request.c:364:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: mdt-lustre-MDT0001_UUID lock: 000000007ee917b5/0xa40083ab9bf53437 lrc: 3/0,0 mode: COS/COS res: [0x240000402:0x7f7:0x0].0xef09e6ea bits 0x2/0x0 rrc: 2 type: IBT gid 0 flags: 0x40210400000000 nid: local remote: 0x0 expref: -99 pid: 6255 timeout: 0 lvb_type: 0 00010000:00010000:3.0:1776178159.469643:0:3340:0:(ldlm_request.c:1257:ldlm_cli_cancel_local()) ### server-side local cancel ns: mdt-lustre-MDT0001_UUID lock: 000000007ee917b5/0xa40083ab9bf53437 lrc: 4/0,0 mode: COS/COS res: [0x240000402:0x7f7:0x0].0xef09e6ea bits 0x2/0x0 rrc: 2 type: IBT gid 0 flags: 0x40218400000000 nid: local remote: 0x0 expref: -99 pid: 6255 timeout: 0 lvb_type: 0 00010000:00010000:3.0:1776178159.469651:0:3340:0:(ldlm_inodebits.c:94:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x240000402:0x7f7:0x0].0xef09e6ea (000000003c4b13ad) 00010000:00010000:3.0:1776178159.469653:0:3340:0:(ldlm_inodebits.c:96:ldlm_reprocess_inodebits_queue()) Hint 2 00010000:00010000:3.0:1776178159.469655:0:3340:0:(ldlm_lockd.c:1903:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: mdt-lustre-MDT0001_UUID lock: 000000007ee917b5/0xa40083ab9bf53437 lrc: 2/0,0 mode: --/COS res: [0x240000402:0x7f7:0x0].0xef09e6ea bits 0x2/0x0 rrc: 2 type: IBT gid 0 flags: 0x44a19400000000 nid: local remote: 0x0 expref: -99 pid: 6255 timeout: 0 lvb_type: 0 00010000:00010000:3.0:1776178159.469660:0:3340:0:(ldlm_lock.c:216:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: mdt-lustre-MDT0001_UUID lock: 000000007ee917b5/0xa40083ab9bf53437 lrc: 0/0,0 mode: --/COS res: [0x240000402:0x7f7:0x0].0xef09e6ea bits 0x2/0x0 rrc: 2 type: IBT gid 0 flags: 0x44a19400000000 nid: local remote: 0x0 expref: -99 pid: 6255 timeout: 0 lvb_type: 0 00010000:00010000:3.0:1776178159.469672:0:3340:0:(ldlm_lock.c:820:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(COS) ns: mdt-lustre-MDT0001_UUID lock: 000000002f1f7bd5/0xa40083ab9bf5344c lrc: 3/0,1 mode: COS/COS res: [0x240000402:0x7fc:0x0].0x0 bits 0x2/0x0 rrc: 3 type: IBT gid 0 flags: 0x50210000000000 nid: local remote: 0x0 expref: -99 pid: 6255 timeout: 0 lvb_type: 0 00010000:00010000:3.0:1776178159.469678:0:3340:0:(ldlm_lock.c:879:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: mdt-lustre-MDT0001_UUID lock: 000000002f1f7bd5/0xa40083ab9bf5344c lrc: 2/0,0 mode: COS/COS res: [0x240000402:0x7fc:0x0].0x0 bits 0x2/0x0 rrc: 3 type: IBT gid 0 flags: 0x50210400000000 nid: local remote: 0x0 expref: -99 pid: 6255 timeout: 0 lvb_type: 0 00010000:00010000:3.0:1776178159.469683:0:3340:0:(ldlm_lockd.c:1879:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: mdt-lustre-MDT0001_UUID lock: 000000002f1f7bd5/0xa40083ab9bf5344c lrc: 3/0,0 mode: COS/COS res: [0x240000402:0x7fc:0x0].0x0 bits 0x2/0x0 rrc: 3 type: IBT gid 0 flags: 0x40210400000000 nid: local remote: 0x0 expref: -99 pid: 6255 timeout: 0 lvb_type: 0 00010000:00010000:3.0:1776178159.469689:0:3340:0:(ldlm_lockd.c:1893:ldlm_handle_bl_callback()) Lock 000000002f1f7bd5 already unused, calling callback (0000000079e78348) 00010000:00010000:3.0:1776178159.469691:0:3340:0:(ldlm_request.c:364:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: mdt-lustre-MDT0001_UUID lock: 000000002f1f7bd5/0xa40083ab9bf5344c lrc: 3/0,0 mode: COS/COS res: [0x240000402:0x7fc:0x0].0x0 bits 0x2/0x0 rrc: 3 type: IBT gid 0 flags: 0x40210400000000 nid: local remote: 0x0 expref: -99 pid: 6255 timeout: 0 lvb_type: 0 00010000:00010000:3.0:1776178159.469697:0:3340:0:(ldlm_request.c:1257:ldlm_cli_cancel_local()) ### server-side local cancel ns: mdt-lustre-MDT0001_UUID lock: 000000002f1f7bd5/0xa40083ab9bf5344c lrc: 4/0,0 mode: COS/COS res: [0x240000402:0x7fc:0x0].0x0 bits 0x2/0x0 rrc: 3 type: IBT gid 0 flags: 0x40218400000000 nid: local remote: 0x0 expref: -99 pid: 6255 timeout: 0 lvb_type: 0 00010000:00010000:3.0:1776178159.469704:0:3340:0:(ldlm_inodebits.c:94:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x240000402:0x7fc:0x0].0x0 (0000000064abeb1c) 00010000:00010000:3.0:1776178159.469706:0:3340:0:(ldlm_inodebits.c:96:ldlm_reprocess_inodebits_queue()) Hint 2 00010000:00010000:3.0:1776178159.469708:0:3340:0:(ldlm_lockd.c:1903:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: mdt-lustre-MDT0001_UUID lock: 000000002f1f7bd5/0xa40083ab9bf5344c lrc: 2/0,0 mode: --/COS res: [0x240000402:0x7fc:0x0].0x0 bits 0x2/0x0 rrc: 3 type: IBT gid 0 flags: 0x44a19400000000 nid: local remote: 0x0 expref: -99 pid: 6255 timeout: 0 lvb_type: 0 00010000:00010000:3.0:1776178159.469717:0:3340:0:(ldlm_lock.c:216:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: mdt-lustre-MDT0001_UUID lock: 000000002f1f7bd5/0xa40083ab9bf5344c lrc: 0/0,0 mode: --/COS res: [0x240000402:0x7fc:0x0].0x0 bits 0x2/0x0 rrc: 3 type: IBT gid 0 flags: 0x44a19400000000 nid: local remote: 0x0 expref: -99 pid: 6255 timeout: 0 lvb_type: 0 00010000:00010000:3.0:1776178159.469726:0:3340:0:(ldlm_lock.c:820:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(COS) ns: mdt-lustre-MDT0001_UUID lock: 0000000030bf6b1e/0xa40083ab9bf53461 lrc: 3/0,1 mode: COS/COS res: [0x240000402:0x7f7:0x0].0x37639fea bits 0x2/0x0 rrc: 2 type: IBT gid 0 flags: 0x50210000000000 nid: local remote: 0x0 expref: -99 pid: 6256 timeout: 0 lvb_type: 0 00010000:00010000:3.0:1776178159.469732:0:3340:0:(ldlm_lock.c:879:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: mdt-lustre-MDT0001_UUID lock: 0000000030bf6b1e/0xa40083ab9bf53461 lrc: 2/0,0 mode: COS/COS res: [0x240000402:0x7f7:0x0].0x37639fea bits 0x2/0x0 rrc: 2 type: IBT gid 0 flags: 0x50210400000000 nid: local remote: 0x0 expref: -99 pid: 6256 timeout: 0 lvb_type: 0 00010000:00010000:3.0:1776178159.469747:0:3340:0:(ldlm_lockd.c:1879:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: mdt-lustre-MDT0001_UUID lock: 0000000030bf6b1e/0xa40083ab9bf53461 lrc: 3/0,0 mode: COS/COS res: [0x240000402:0x7f7:0x0].0x37639fea bits 0x2/0x0 rrc: 2 type: IBT gid 0 flags: 0x40210400000000 nid: local remote: 0x0 expref: -99 pid: 6256 timeout: 0 lvb_type: 0 00010000:00010000:3.0:1776178159.469754:0:3340:0:(ldlm_lockd.c:1893:ldlm_handle_bl_callback()) Lock 0000000030bf6b1e already unused, calling callback (0000000079e78348) 00010000:00010000:3.0:1776178159.469756:0:3340:0:(ldlm_request.c:364:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: mdt-lustre-MDT0001_UUID lock: 0000000030bf6b1e/0xa40083ab9bf53461 lrc: 3/0,0 mode: COS/COS res: [0x240000402:0x7f7:0x0].0x37639fea bits 0x2/0x0 rrc: 2 type: IBT gid 0 flags: 0x40210400000000 nid: local remote: 0x0 expref: -99 pid: 6256 timeout: 0 lvb_type: 0 00010000:00010000:3.0:1776178159.469761:0:3340:0:(ldlm_request.c:1257:ldlm_cli_cancel_local()) ### server-side local cancel ns: mdt-lustre-MDT0001_UUID lock: 0000000030bf6b1e/0xa40083ab9bf53461 lrc: 4/0,0 mode: COS/COS res: [0x240000402:0x7f7:0x0].0x37639fea bits 0x2/0x0 rrc: 2 type: IBT gid 0 flags: 0x40218400000000 nid: local remote: 0x0 expref: -99 pid: 6256 timeout: 0 lvb_type: 0 00010000:00010000:3.0:1776178159.469769:0:3340:0:(ldlm_inodebits.c:94:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x240000402:0x7f7:0x0].0x37639fea (000000000a3c1046) 00010000:00010000:3.0:1776178159.469771:0:3340:0:(ldlm_inodebits.c:96:ldlm_reprocess_inodebits_queue()) Hint 2 00010000:00010000:3.0:1776178159.469773:0:3340:0:(ldlm_lockd.c:1903:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: mdt-lustre-MDT0001_UUID lock: 0000000030bf6b1e/0xa40083ab9bf53461 lrc: 2/0,0 mode: --/COS res: [0x240000402:0x7f7:0x0].0x37639fea bits 0x2/0x0 rrc: 2 type: IBT gid 0 flags: 0x44a19400000000 nid: local remote: 0x0 expref: -99 pid: 6256 timeout: 0 lvb_type: 0 00010000:00010000:3.0:1776178159.469779:0:3340:0:(ldlm_lock.c:216:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: mdt-lustre-MDT0001_UUID lock: 0000000030bf6b1e/0xa40083ab9bf53461 lrc: 0/0,0 mode: --/COS res: [0x240000402:0x7f7:0x0].0x37639fea bits 0x2/0x0 rrc: 2 type: IBT gid 0 flags: 0x44a19400000000 nid: local remote: 0x0 expref: -99 pid: 6256 timeout: 0 lvb_type: 0 00010000:00010000:3.0:1776178159.469791:0:3340:0:(ldlm_lock.c:820:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(COS) ns: mdt-lustre-MDT0001_UUID lock: 0000000068850c85/0xa40083ab9bf53476 lrc: 3/0,1 mode: COS/COS res: [0x240000402:0x7fd:0x0].0x0 bits 0x2/0x0 rrc: 3 type: IBT gid 0 flags: 0x50210000000000 nid: local remote: 0x0 expref: -99 pid: 6256 timeout: 0 lvb_type: 0 00010000:00010000:3.0:1776178159.469801:0:3340:0:(ldlm_lock.c:879:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: mdt-lustre-MDT0001_UUID lock: 0000000068850c85/0xa40083ab9bf53476 lrc: 2/0,0 mode: COS/COS res: [0x240000402:0x7fd:0x0].0x0 bits 0x2/0x0 rrc: 3 type: IBT gid 0 flags: 0x50210400000000 nid: local remote: 0x0 expref: -99 pid: 6256 timeout: 0 lvb_type: 0 00010000:00010000:3.0:1776178159.469806:0:3340:0:(ldlm_lockd.c:1879:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: mdt-lustre-MDT0001_UUID lock: 0000000068850c85/0xa40083ab9bf53476 lrc: 3/0,0 mode: COS/COS res: [0x240000402:0x7fd:0x0].0x0 bits 0x2/0x0 rrc: 3 type: IBT gid 0 flags: 0x40210400000000 nid: local remote: 0x0 expref: -99 pid: 6256 timeout: 0 lvb_type: 0 00010000:00010000:3.0:1776178159.469811:0:3340:0:(ldlm_lockd.c:1893:ldlm_handle_bl_callback()) Lock 0000000068850c85 already unused, calling callback (0000000079e78348) 00010000:00010000:3.0:1776178159.469813:0:3340:0:(ldlm_request.c:364:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: mdt-lustre-MDT0001_UUID lock: 0000000068850c85/0xa40083ab9bf53476 lrc: 3/0,0 mode: COS/COS res: [0x240000402:0x7fd:0x0].0x0 bits 0x2/0x0 rrc: 3 type: IBT gid 0 flags: 0x40210400000000 nid: local remote: 0x0 expref: -99 pid: 6256 timeout: 0 lvb_type: 0 00010000:00010000:3.0:1776178159.469818:0:3340:0:(ldlm_request.c:1257:ldlm_cli_cancel_local()) ### server-side local cancel ns: mdt-lustre-MDT0001_UUID lock: 0000000068850c85/0xa40083ab9bf53476 lrc: 4/0,0 mode: COS/COS res: [0x240000402:0x7fd:0x0].0x0 bits 0x2/0x0 rrc: 3 type: IBT gid 0 flags: 0x40218400000000 nid: local remote: 0x0 expref: -99 pid: 6256 timeout: 0 lvb_type: 0 00010000:00010000:3.0:1776178159.469825:0:3340:0:(ldlm_inodebits.c:94:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x240000402:0x7fd:0x0].0x0 (000000008d47e759) 00010000:00010000:3.0:1776178159.469827:0:3340:0:(ldlm_inodebits.c:96:ldlm_reprocess_inodebits_queue()) Hint 2 00010000:00010000:3.0:1776178159.469829:0:3340:0:(ldlm_lockd.c:1903:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: mdt-lustre-MDT0001_UUID lock: 0000000068850c85/0xa40083ab9bf53476 lrc: 2/0,0 mode: --/COS res: [0x240000402:0x7fd:0x0].0x0 bits 0x2/0x0 rrc: 3 type: IBT gid 0 flags: 0x44a19400000000 nid: local remote: 0x0 expref: -99 pid: 6256 timeout: 0 lvb_type: 0 00010000:00010000:3.0:1776178159.469833:0:3340:0:(ldlm_lock.c:216:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: mdt-lustre-MDT0001_UUID lock: 0000000068850c85/0xa40083ab9bf53476 lrc: 0/0,0 mode: --/COS res: [0x240000402:0x7fd:0x0].0x0 bits 0x2/0x0 rrc: 3 type: IBT gid 0 flags: 0x44a19400000000 nid: local remote: 0x0 expref: -99 pid: 6256 timeout: 0 lvb_type: 0 00010000:00010000:3.0:1776178159.469842:0:3340:0:(ldlm_lock.c:820:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(COS) ns: mdt-lustre-MDT0001_UUID lock: 000000008813523d/0xa40083ab9bf5348b lrc: 3/0,1 mode: COS/COS res: [0x240000402:0x7f7:0x0].0x7fbd58ea bits 0x2/0x0 rrc: 2 type: IBT gid 0 flags: 0x50210000000000 nid: local remote: 0x0 expref: -99 pid: 6256 timeout: 0 lvb_type: 0 00010000:00010000:3.0:1776178159.469848:0:3340:0:(ldlm_lock.c:879:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: mdt-lustre-MDT0001_UUID lock: 000000008813523d/0xa40083ab9bf5348b lrc: 2/0,0 mode: COS/COS res: [0x240000402:0x7f7:0x0].0x7fbd58ea bits 0x2/0x0 rrc: 2 type: IBT gid 0 flags: 0x50210400000000 nid: local remote: 0x0 expref: -99 pid: 6256 timeout: 0 lvb_type: 0 00010000:00010000:3.0:1776178159.469853:0:3340:0:(ldlm_lockd.c:1879:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: mdt-lustre-MDT0001_UUID lock: 000000008813523d/0xa40083ab9bf5348b lrc: 3/0,0 mode: COS/COS res: [0x240000402:0x7f7:0x0].0x7fbd58ea bits 0x2/0x0 rrc: 2 type: IBT gid 0 flags: 0x40210400000000 nid: local remote: 0x0 expref: -99 pid: 6256 timeout: 0 lvb_type: 0 00010000:00010000:3.0:1776178159.469862:0:3340:0:(ldlm_lockd.c:1893:ldlm_handle_bl_callback()) Lock 000000008813523d already unused, calling callback (0000000079e78348) 00010000:00010000:3.0:1776178159.469864:0:3340:0:(ldlm_request.c:364:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: mdt-lustre-MDT0001_UUID lock: 000000008813523d/0xa40083ab9bf5348b lrc: 3/0,0 mode: COS/COS res: [0x240000402:0x7f7:0x0].0x7fbd58ea bits 0x2/0x0 rrc: 2 type: IBT gid 0 flags: 0x40210400000000 nid: local remote: 0x0 expref: -99 pid: 6256 timeout: 0 lvb_type: 0 00010000:00010000:3.0:1776178159.469870:0:3340:0:(ldlm_request.c:1257:ldlm_cli_cancel_local()) ### server-side local cancel ns: mdt-lustre-MDT0001_UUID lock: 000000008813523d/0xa40083ab9bf5348b lrc: 4/0,0 mode: COS/COS res: [0x240000402:0x7f7:0x0].0x7fbd58ea bits 0x2/0x0 rrc: 2 type: IBT gid 0 flags: 0x40218400000000 nid: local remote: 0x0 expref: -99 pid: 6256 timeout: 0 lvb_type: 0 00010000:00010000:3.0:1776178159.469877:0:3340:0:(ldlm_inodebits.c:94:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x240000402:0x7f7:0x0].0x7fbd58ea (000000008c340be4) 00010000:00010000:3.0:1776178159.469879:0:3340:0:(ldlm_inodebits.c:96:ldlm_reprocess_inodebits_queue()) Hint 2 00010000:00010000:3.0:1776178159.469881:0:3340:0:(ldlm_lockd.c:1903:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: mdt-lustre-MDT0001_UUID lock: 000000008813523d/0xa40083ab9bf5348b lrc: 2/0,0 mode: --/COS res: [0x240000402:0x7f7:0x0].0x7fbd58ea bits 0x2/0x0 rrc: 2 type: IBT gid 0 flags: 0x44a19400000000 nid: local remote: 0x0 expref: -99 pid: 6256 timeout: 0 lvb_type: 0 00010000:00010000:3.0:1776178159.470487:0:3340:0:(ldlm_lock.c:216:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: mdt-lustre-MDT0001_UUID lock: 000000008813523d/0xa40083ab9bf5348b lrc: 0/0,0 mode: --/COS res: [0x240000402:0x7f7:0x0].0x7fbd58ea bits 0x2/0x0 rrc: 2 type: IBT gid 0 flags: 0x44a19400000000 nid: local remote: 0x0 expref: -99 pid: 6256 timeout: 0 lvb_type: 0 00010000:00010000:3.0:1776178159.470504:0:3340:0:(ldlm_lock.c:820:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(COS) ns: mdt-lustre-MDT0001_UUID lock: 000000007da3b625/0xa40083ab9bf534a0 lrc: 3/0,1 mode: COS/COS res: [0x240000402:0x7fe:0x0].0x0 bits 0x2/0x0 rrc: 3 type: IBT gid 0 flags: 0x50210000000000 nid: local remote: 0x0 expref: -99 pid: 6255 timeout: 0 lvb_type: 0 00010000:00010000:3.0:1776178159.470510:0:3340:0:(ldlm_lock.c:879:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: mdt-lustre-MDT0001_UUID lock: 000000007da3b625/0xa40083ab9bf534a0 lrc: 2/0,0 mode: COS/COS res: [0x240000402:0x7fe:0x0].0x0 bits 0x2/0x0 rrc: 3 type: IBT gid 0 flags: 0x50210400000000 nid: local remote: 0x0 expref: -99 pid: 6255 timeout: 0 lvb_type: 0 00010000:00010000:3.0:1776178159.470515:0:3340:0:(ldlm_lockd.c:1879:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: mdt-lustre-MDT0001_UUID lock: 000000007da3b625/0xa40083ab9bf534a0 lrc: 3/0,0 mode: COS/COS res: [0x240000402:0x7fe:0x0].0x0 bits 0x2/0x0 rrc: 3 type: IBT gid 0 flags: 0x40210400000000 nid: local remote: 0x0 expref: -99 pid: 6255 timeout: 0 lvb_type: 0 00010000:00010000:3.0:1776178159.470520:0:3340:0:(ldlm_lockd.c:1893:ldlm_handle_bl_callback()) Lock 000000007da3b625 already unused, calling callback (0000000079e78348) 00010000:00010000:3.0:1776178159.470522:0:3340:0:(ldlm_request.c:364:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: mdt-lustre-MDT0001_UUID lock: 000000007da3b625/0xa40083ab9bf534a0 lrc: 3/0,0 mode: COS/COS res: [0x240000402:0x7fe:0x0].0x0 bits 0x2/0x0 rrc: 3 type: IBT gid 0 flags: 0x40210400000000 nid: local remote: 0x0 expref: -99 pid: 6255 timeout: 0 lvb_type: 0 00010000:00010000:3.0:1776178159.470527:0:3340:0:(ldlm_request.c:1257:ldlm_cli_cancel_local()) ### server-side local cancel ns: mdt-lustre-MDT0001_UUID lock: 000000007da3b625/0xa40083ab9bf534a0 lrc: 4/0,0 mode: COS/COS res: [0x240000402:0x7fe:0x0].0x0 bits 0x2/0x0 rrc: 3 type: IBT gid 0 flags: 0x40218400000000 nid: local remote: 0x0 expref: -99 pid: 6255 timeout: 0 lvb_type: 0 00010000:00010000:3.0:1776178159.470600:0:3340:0:(ldlm_inodebits.c:94:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x240000402:0x7fe:0x0].0x0 (00000000498a007f) 00010000:00010000:3.0:1776178159.470603:0:3340:0:(ldlm_inodebits.c:96:ldlm_reprocess_inodebits_queue()) Hint 2 00010000:00010000:3.0:1776178159.470605:0:3340:0:(ldlm_lockd.c:1903:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: mdt-lustre-MDT0001_UUID lock: 000000007da3b625/0xa40083ab9bf534a0 lrc: 2/0,0 mode: --/COS res: [0x240000402:0x7fe:0x0].0x0 bits 0x2/0x0 rrc: 3 type: IBT gid 0 flags: 0x44a19400000000 nid: local remote: 0x0 expref: -99 pid: 6255 timeout: 0 lvb_type: 0 00010000:00010000:3.0:1776178159.470611:0:3340:0:(ldlm_lock.c:216:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: mdt-lustre-MDT0001_UUID lock: 000000007da3b625/0xa40083ab9bf534a0 lrc: 0/0,0 mode: --/COS res: [0x240000402:0x7fe:0x0].0x0 bits 0x2/0x0 rrc: 3 type: IBT gid 0 flags: 0x44a19400000000 nid: local remote: 0x0 expref: -99 pid: 6255 timeout: 0 lvb_type: 0 00010000:00010000:3.0:1776178159.470623:0:3340:0:(ldlm_lock.c:820:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(COS) ns: mdt-lustre-MDT0001_UUID lock: 000000006db4ec1d/0xa40083ab9bf53510 lrc: 3/0,1 mode: COS/COS res: [0x240000402:0x7f8:0x0].0x0 bits 0x13/0x0 rrc: 2 type: IBT gid 0 flags: 0x50210000000000 nid: local remote: 0x0 expref: -99 pid: 6255 timeout: 0 lvb_type: 0 00010000:00010000:3.0:1776178159.470628:0:3340:0:(ldlm_lock.c:879:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: mdt-lustre-MDT0001_UUID lock: 000000006db4ec1d/0xa40083ab9bf53510 lrc: 2/0,0 mode: COS/COS res: [0x240000402:0x7f8:0x0].0x0 bits 0x13/0x0 rrc: 2 type: IBT gid 0 flags: 0x50210400000000 nid: local remote: 0x0 expref: -99 pid: 6255 timeout: 0 lvb_type: 0 00010000:00010000:3.0:1776178159.470633:0:3340:0:(ldlm_lockd.c:1879:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: mdt-lustre-MDT0001_UUID lock: 000000006db4ec1d/0xa40083ab9bf53510 lrc: 3/0,0 mode: COS/COS res: [0x240000402:0x7f8:0x0].0x0 bits 0x13/0x0 rrc: 2 type: IBT gid 0 flags: 0x40210400000000 nid: local remote: 0x0 expref: -99 pid: 6255 timeout: 0 lvb_type: 0 00010000:00010000:3.0:1776178159.470637:0:3340:0:(ldlm_lockd.c:1893:ldlm_handle_bl_callback()) Lock 000000006db4ec1d already unused, calling callback (0000000079e78348) 00010000:00010000:3.0:1776178159.470639:0:3340:0:(ldlm_request.c:364:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: mdt-lustre-MDT0001_UUID lock: 000000006db4ec1d/0xa40083ab9bf53510 lrc: 3/0,0 mode: COS/COS res: [0x240000402:0x7f8:0x0].0x0 bits 0x13/0x0 rrc: 2 type: IBT gid 0 flags: 0x40210400000000 nid: local remote: 0x0 expref: -99 pid: 6255 timeout: 0 lvb_type: 0 00010000:00010000:3.0:1776178159.470644:0:3340:0:(ldlm_request.c:1257:ldlm_cli_cancel_local()) ### server-side local cancel ns: mdt-lustre-MDT0001_UUID lock: 000000006db4ec1d/0xa40083ab9bf53510 lrc: 4/0,0 mode: COS/COS res: [0x240000402:0x7f8:0x0].0x0 bits 0x13/0x0 rrc: 2 type: IBT gid 0 flags: 0x40218400000000 nid: local remote: 0x0 expref: -99 pid: 6255 timeout: 0 lvb_type: 0 00010000:00010000:3.0:1776178159.470650:0:3340:0:(ldlm_inodebits.c:94:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x240000402:0x7f8:0x0].0x0 (00000000099618da) 00010000:00010000:3.0:1776178159.470653:0:3340:0:(ldlm_inodebits.c:96:ldlm_reprocess_inodebits_queue()) Hint 13 00010000:00010000:3.0:1776178159.470654:0:3340:0:(ldlm_lockd.c:1903:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: mdt-lustre-MDT0001_UUID lock: 000000006db4ec1d/0xa40083ab9bf53510 lrc: 2/0,0 mode: --/COS res: [0x240000402:0x7f8:0x0].0x0 bits 0x13/0x0 rrc: 2 type: IBT gid 0 flags: 0x44a19400000000 nid: local remote: 0x0 expref: -99 pid: 6255 timeout: 0 lvb_type: 0 00010000:00010000:3.0:1776178159.470664:0:3340:0:(ldlm_lock.c:216:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: mdt-lustre-MDT0001_UUID lock: 000000006db4ec1d/0xa40083ab9bf53510 lrc: 0/0,0 mode: --/COS res: [0x240000402:0x7f8:0x0].0x0 bits 0x13/0x0 rrc: 2 type: IBT gid 0 flags: 0x44a19400000000 nid: local remote: 0x0 expref: -99 pid: 6255 timeout: 0 lvb_type: 0 00010000:00010000:3.0:1776178159.470684:0:3340:0:(ldlm_lock.c:820:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(COS) ns: mdt-lustre-MDT0001_UUID lock: 000000007e0712b8/0xa40083ab9bf53548 lrc: 3/0,1 mode: COS/COS res: [0x240000402:0x7fb:0x0].0x0 bits 0x13/0x0 rrc: 2 type: IBT gid 0 flags: 0x50210000000000 nid: local remote: 0x0 expref: -99 pid: 8250 timeout: 0 lvb_type: 0 00010000:00010000:3.0:1776178159.470691:0:3340:0:(ldlm_lock.c:879:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: mdt-lustre-MDT0001_UUID lock: 000000007e0712b8/0xa40083ab9bf53548 lrc: 2/0,0 mode: COS/COS res: [0x240000402:0x7fb:0x0].0x0 bits 0x13/0x0 rrc: 2 type: IBT gid 0 flags: 0x50210400000000 nid: local remote: 0x0 expref: -99 pid: 8250 timeout: 0 lvb_type: 0 00010000:00010000:3.0:1776178159.470695:0:3340:0:(ldlm_lockd.c:1879:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: mdt-lustre-MDT0001_UUID lock: 000000007e0712b8/0xa40083ab9bf53548 lrc: 3/0,0 mode: COS/COS res: [0x240000402:0x7fb:0x0].0x0 bits 0x13/0x0 rrc: 2 type: IBT gid 0 flags: 0x40210400000000 nid: local remote: 0x0 expref: -99 pid: 8250 timeout: 0 lvb_type: 0 00010000:00010000:3.0:1776178159.470700:0:3340:0:(ldlm_lockd.c:1893:ldlm_handle_bl_callback()) Lock 000000007e0712b8 already unused, calling callback (0000000079e78348) 00010000:00010000:3.0:1776178159.470702:0:3340:0:(ldlm_request.c:364:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: mdt-lustre-MDT0001_UUID lock: 000000007e0712b8/0xa40083ab9bf53548 lrc: 3/0,0 mode: COS/COS res: [0x240000402:0x7fb:0x0].0x0 bits 0x13/0x0 rrc: 2 type: IBT gid 0 flags: 0x40210400000000 nid: local remote: 0x0 expref: -99 pid: 8250 timeout: 0 lvb_type: 0 00010000:00010000:3.0:1776178159.470707:0:3340:0:(ldlm_request.c:1257:ldlm_cli_cancel_local()) ### server-side local cancel ns: mdt-lustre-MDT0001_UUID lock: 000000007e0712b8/0xa40083ab9bf53548 lrc: 4/0,0 mode: COS/COS res: [0x240000402:0x7fb:0x0].0x0 bits 0x13/0x0 rrc: 2 type: IBT gid 0 flags: 0x40218400000000 nid: local remote: 0x0 expref: -99 pid: 8250 timeout: 0 lvb_type: 0 00010000:00010000:3.0:1776178159.470713:0:3340:0:(ldlm_inodebits.c:94:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x240000402:0x7fb:0x0].0x0 (000000009f7659df) 00010000:00010000:3.0:1776178159.470715:0:3340:0:(ldlm_inodebits.c:96:ldlm_reprocess_inodebits_queue()) Hint 13 00010000:00010000:3.0:1776178159.470717:0:3340:0:(ldlm_lockd.c:1903:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: mdt-lustre-MDT0001_UUID lock: 000000007e0712b8/0xa40083ab9bf53548 lrc: 2/0,0 mode: --/COS res: [0x240000402:0x7fb:0x0].0x0 bits 0x13/0x0 rrc: 2 type: IBT gid 0 flags: 0x44a19400000000 nid: local remote: 0x0 expref: -99 pid: 8250 timeout: 0 lvb_type: 0 00010000:00010000:3.0:1776178159.470722:0:3340:0:(ldlm_lock.c:216:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: mdt-lustre-MDT0001_UUID lock: 000000007e0712b8/0xa40083ab9bf53548 lrc: 0/0,0 mode: --/COS res: [0x240000402:0x7fb:0x0].0x0 bits 0x13/0x0 rrc: 2 type: IBT gid 0 flags: 0x44a19400000000 nid: local remote: 0x0 expref: -99 pid: 8250 timeout: 0 lvb_type: 0 00010000:00010000:3.0:1776178159.470735:0:3340:0:(ldlm_lock.c:820:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(COS) ns: mdt-lustre-MDT0001_UUID lock: 0000000022bf14ed/0xa40083ab9bf5354f lrc: 3/0,1 mode: COS/COS res: [0x240000402:0x7f9:0x0].0x0 bits 0x13/0x0 rrc: 2 type: IBT gid 0 flags: 0x50210000000000 nid: local remote: 0x0 expref: -99 pid: 8250 timeout: 0 lvb_type: 0 00010000:00010000:3.0:1776178159.470749:0:3340:0:(ldlm_lock.c:879:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: mdt-lustre-MDT0001_UUID lock: 0000000022bf14ed/0xa40083ab9bf5354f lrc: 2/0,0 mode: COS/COS res: [0x240000402:0x7f9:0x0].0x0 bits 0x13/0x0 rrc: 2 type: IBT gid 0 flags: 0x50210400000000 nid: local remote: 0x0 expref: -99 pid: 8250 timeout: 0 lvb_type: 0 00010000:00010000:3.0:1776178159.470753:0:3340:0:(ldlm_lockd.c:1879:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: mdt-lustre-MDT0001_UUID lock: 0000000022bf14ed/0xa40083ab9bf5354f lrc: 3/0,0 mode: COS/COS res: [0x240000402:0x7f9:0x0].0x0 bits 0x13/0x0 rrc: 2 type: IBT gid 0 flags: 0x40210400000000 nid: local remote: 0x0 expref: -99 pid: 8250 timeout: 0 lvb_type: 0 00010000:00010000:3.0:1776178159.471199:0:3340:0:(ldlm_lockd.c:1893:ldlm_handle_bl_callback()) Lock 0000000022bf14ed already unused, calling callback (0000000079e78348) 00010000:00010000:3.0:1776178159.471202:0:3340:0:(ldlm_request.c:364:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: mdt-lustre-MDT0001_UUID lock: 0000000022bf14ed/0xa40083ab9bf5354f lrc: 3/0,0 mode: COS/COS res: [0x240000402:0x7f9:0x0].0x0 bits 0x13/0x0 rrc: 2 type: IBT gid 0 flags: 0x40210400000000 nid: local remote: 0x0 expref: -99 pid: 8250 timeout: 0 lvb_type: 0 00010000:00010000:3.0:1776178159.471209:0:3340:0:(ldlm_request.c:1257:ldlm_cli_cancel_local()) ### server-side local cancel ns: mdt-lustre-MDT0001_UUID lock: 0000000022bf14ed/0xa40083ab9bf5354f lrc: 4/0,0 mode: COS/COS res: [0x240000402:0x7f9:0x0].0x0 bits 0x13/0x0 rrc: 2 type: IBT gid 0 flags: 0x40218400000000 nid: local remote: 0x0 expref: -99 pid: 8250 timeout: 0 lvb_type: 0 00010000:00010000:3.0:1776178159.471217:0:3340:0:(ldlm_inodebits.c:94:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x240000402:0x7f9:0x0].0x0 (00000000f193fa59) 00010000:00010000:3.0:1776178159.471219:0:3340:0:(ldlm_inodebits.c:96:ldlm_reprocess_inodebits_queue()) Hint 13 00010000:00010000:3.0:1776178159.471221:0:3340:0:(ldlm_lockd.c:1903:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: mdt-lustre-MDT0001_UUID lock: 0000000022bf14ed/0xa40083ab9bf5354f lrc: 2/0,0 mode: --/COS res: [0x240000402:0x7f9:0x0].0x0 bits 0x13/0x0 rrc: 2 type: IBT gid 0 flags: 0x44a19400000000 nid: local remote: 0x0 expref: -99 pid: 8250 timeout: 0 lvb_type: 0 00010000:00010000:3.0:1776178159.471227:0:3340:0:(ldlm_lock.c:216:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: mdt-lustre-MDT0001_UUID lock: 0000000022bf14ed/0xa40083ab9bf5354f lrc: 0/0,0 mode: --/COS res: [0x240000402:0x7f9:0x0].0x0 bits 0x13/0x0 rrc: 2 type: IBT gid 0 flags: 0x44a19400000000 nid: local remote: 0x0 expref: -99 pid: 8250 timeout: 0 lvb_type: 0 00010000:00010000:3.0:1776178159.471243:0:3340:0:(ldlm_lock.c:820:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(COS) ns: mdt-lustre-MDT0001_UUID lock: 00000000014a8a18/0xa40083ab9bf53541 lrc: 3/0,1 mode: COS/COS res: [0x240000402:0x7fa:0x0].0x0 bits 0x13/0x0 rrc: 2 type: IBT gid 0 flags: 0x50210000000000 nid: local remote: 0x0 expref: -99 pid: 6256 timeout: 0 lvb_type: 0 00010000:00010000:3.0:1776178159.471249:0:3340:0:(ldlm_lock.c:879:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: mdt-lustre-MDT0001_UUID lock: 00000000014a8a18/0xa40083ab9bf53541 lrc: 2/0,0 mode: COS/COS res: [0x240000402:0x7fa:0x0].0x0 bits 0x13/0x0 rrc: 2 type: IBT gid 0 flags: 0x50210400000000 nid: local remote: 0x0 expref: -99 pid: 6256 timeout: 0 lvb_type: 0 00010000:00010000:3.0:1776178159.471254:0:3340:0:(ldlm_lockd.c:1879:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: mdt-lustre-MDT0001_UUID lock: 00000000014a8a18/0xa40083ab9bf53541 lrc: 3/0,0 mode: COS/COS res: [0x240000402:0x7fa:0x0].0x0 bits 0x13/0x0 rrc: 2 type: IBT gid 0 flags: 0x40210400000000 nid: local remote: 0x0 expref: -99 pid: 6256 timeout: 0 lvb_type: 0 00010000:00010000:3.0:1776178159.471263:0:3340:0:(ldlm_lockd.c:1893:ldlm_handle_bl_callback()) Lock 00000000014a8a18 already unused, calling callback (0000000079e78348) 00010000:00010000:3.0:1776178159.471265:0:3340:0:(ldlm_request.c:364:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: mdt-lustre-MDT0001_UUID lock: 00000000014a8a18/0xa40083ab9bf53541 lrc: 3/0,0 mode: COS/COS res: [0x240000402:0x7fa:0x0].0x0 bits 0x13/0x0 rrc: 2 type: IBT gid 0 flags: 0x40210400000000 nid: local remote: 0x0 expref: -99 pid: 6256 timeout: 0 lvb_type: 0 00010000:00010000:3.0:1776178159.471270:0:3340:0:(ldlm_request.c:1257:ldlm_cli_cancel_local()) ### server-side local cancel ns: mdt-lustre-MDT0001_UUID lock: 00000000014a8a18/0xa40083ab9bf53541 lrc: 4/0,0 mode: COS/COS res: [0x240000402:0x7fa:0x0].0x0 bits 0x13/0x0 rrc: 2 type: IBT gid 0 flags: 0x40218400000000 nid: local remote: 0x0 expref: -99 pid: 6256 timeout: 0 lvb_type: 0 00010000:00010000:3.0:1776178159.471278:0:3340:0:(ldlm_inodebits.c:94:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x240000402:0x7fa:0x0].0x0 (000000006b0fcbcb) 00010000:00010000:3.0:1776178159.471280:0:3340:0:(ldlm_inodebits.c:96:ldlm_reprocess_inodebits_queue()) Hint 13 00010000:00010000:3.0:1776178159.471282:0:3340:0:(ldlm_lockd.c:1903:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: mdt-lustre-MDT0001_UUID lock: 00000000014a8a18/0xa40083ab9bf53541 lrc: 2/0,0 mode: --/COS res: [0x240000402:0x7fa:0x0].0x0 bits 0x13/0x0 rrc: 2 type: IBT gid 0 flags: 0x44a19400000000 nid: local remote: 0x0 expref: -99 pid: 6256 timeout: 0 lvb_type: 0 00010000:00010000:3.0:1776178159.471287:0:3340:0:(ldlm_lock.c:216:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: mdt-lustre-MDT0001_UUID lock: 00000000014a8a18/0xa40083ab9bf53541 lrc: 0/0,0 mode: --/COS res: [0x240000402:0x7fa:0x0].0x0 bits 0x13/0x0 rrc: 2 type: IBT gid 0 flags: 0x44a19400000000 nid: local remote: 0x0 expref: -99 pid: 6256 timeout: 0 lvb_type: 0 00010000:00010000:3.0:1776178159.471299:0:3340:0:(ldlm_lock.c:820:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(COS) ns: mdt-lustre-MDT0001_UUID lock: 0000000011d684d8/0xa40083ab9bf53556 lrc: 3/0,1 mode: COS/COS res: [0x240000402:0x7fe:0x0].0x0 bits 0x13/0x0 rrc: 2 type: IBT gid 0 flags: 0x50210000000000 nid: local remote: 0x0 expref: -99 pid: 6256 timeout: 0 lvb_type: 0 00010000:00010000:3.0:1776178159.471305:0:3340:0:(ldlm_lock.c:879:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: mdt-lustre-MDT0001_UUID lock: 0000000011d684d8/0xa40083ab9bf53556 lrc: 2/0,0 mode: COS/COS res: [0x240000402:0x7fe:0x0].0x0 bits 0x13/0x0 rrc: 2 type: IBT gid 0 flags: 0x50210400000000 nid: local remote: 0x0 expref: -99 pid: 6256 timeout: 0 lvb_type: 0 00010000:00010000:3.0:1776178159.471310:0:3340:0:(ldlm_lockd.c:1879:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: mdt-lustre-MDT0001_UUID lock: 0000000011d684d8/0xa40083ab9bf53556 lrc: 3/0,0 mode: COS/COS res: [0x240000402:0x7fe:0x0].0x0 bits 0x13/0x0 rrc: 2 type: IBT gid 0 flags: 0x40210400000000 nid: local remote: 0x0 expref: -99 pid: 6256 timeout: 0 lvb_type: 0 00010000:00010000:3.0:1776178159.471314:0:3340:0:(ldlm_lockd.c:1893:ldlm_handle_bl_callback()) Lock 0000000011d684d8 already unused, calling callback (0000000079e78348) 00010000:00010000:3.0:1776178159.471316:0:3340:0:(ldlm_request.c:364:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: mdt-lustre-MDT0001_UUID lock: 0000000011d684d8/0xa40083ab9bf53556 lrc: 3/0,0 mode: COS/COS res: [0x240000402:0x7fe:0x0].0x0 bits 0x13/0x0 rrc: 2 type: IBT gid 0 flags: 0x40210400000000 nid: local remote: 0x0 expref: -99 pid: 6256 timeout: 0 lvb_type: 0 00010000:00010000:3.0:1776178159.471321:0:3340:0:(ldlm_request.c:1257:ldlm_cli_cancel_local()) ### server-side local cancel ns: mdt-lustre-MDT0001_UUID lock: 0000000011d684d8/0xa40083ab9bf53556 lrc: 4/0,0 mode: COS/COS res: [0x240000402:0x7fe:0x0].0x0 bits 0x13/0x0 rrc: 2 type: IBT gid 0 flags: 0x40218400000000 nid: local remote: 0x0 expref: -99 pid: 6256 timeout: 0 lvb_type: 0 00010000:00010000:3.0:1776178159.471328:0:3340:0:(ldlm_inodebits.c:94:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x240000402:0x7fe:0x0].0x0 (00000000498a007f) 00010000:00010000:3.0:1776178159.471330:0:3340:0:(ldlm_inodebits.c:96:ldlm_reprocess_inodebits_queue()) Hint 13 00010000:00010000:3.0:1776178159.471332:0:3340:0:(ldlm_lockd.c:1903:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: mdt-lustre-MDT0001_UUID lock: 0000000011d684d8/0xa40083ab9bf53556 lrc: 2/0,0 mode: --/COS res: [0x240000402:0x7fe:0x0].0x0 bits 0x13/0x0 rrc: 2 type: IBT gid 0 flags: 0x44a19400000000 nid: local remote: 0x0 expref: -99 pid: 6256 timeout: 0 lvb_type: 0 00010000:00010000:3.0:1776178159.471337:0:3340:0:(ldlm_lock.c:216:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: mdt-lustre-MDT0001_UUID lock: 0000000011d684d8/0xa40083ab9bf53556 lrc: 0/0,0 mode: --/COS res: [0x240000402:0x7fe:0x0].0x0 bits 0x13/0x0 rrc: 2 type: IBT gid 0 flags: 0x44a19400000000 nid: local remote: 0x0 expref: -99 pid: 6256 timeout: 0 lvb_type: 0 00010000:00010000:3.0:1776178159.471349:0:3340:0:(ldlm_lock.c:820:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(COS) ns: mdt-lustre-MDT0001_UUID lock: 0000000031bb7277/0xa40083ab9bf5355d lrc: 3/0,1 mode: COS/COS res: [0x240000402:0x7fc:0x0].0x0 bits 0x13/0x0 rrc: 2 type: IBT gid 0 flags: 0x50210000000000 nid: local remote: 0x0 expref: -99 pid: 6255 timeout: 0 lvb_type: 0 00010000:00010000:3.0:1776178159.471355:0:3340:0:(ldlm_lock.c:879:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: mdt-lustre-MDT0001_UUID lock: 0000000031bb7277/0xa40083ab9bf5355d lrc: 2/0,0 mode: COS/COS res: [0x240000402:0x7fc:0x0].0x0 bits 0x13/0x0 rrc: 2 type: IBT gid 0 flags: 0x50210400000000 nid: local remote: 0x0 expref: -99 pid: 6255 timeout: 0 lvb_type: 0 00010000:00010000:3.0:1776178159.471361:0:3340:0:(ldlm_lockd.c:1879:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: mdt-lustre-MDT0001_UUID lock: 0000000031bb7277/0xa40083ab9bf5355d lrc: 3/0,0 mode: COS/COS res: [0x240000402:0x7fc:0x0].0x0 bits 0x13/0x0 rrc: 2 type: IBT gid 0 flags: 0x40210400000000 nid: local remote: 0x0 expref: -99 pid: 6255 timeout: 0 lvb_type: 0 00010000:00010000:3.0:1776178159.471366:0:3340:0:(ldlm_lockd.c:1893:ldlm_handle_bl_callback()) Lock 0000000031bb7277 already unused, calling callback (0000000079e78348) 00010000:00010000:3.0:1776178159.471368:0:3340:0:(ldlm_request.c:364:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: mdt-lustre-MDT0001_UUID lock: 0000000031bb7277/0xa40083ab9bf5355d lrc: 3/0,0 mode: COS/COS res: [0x240000402:0x7fc:0x0].0x0 bits 0x13/0x0 rrc: 2 type: IBT gid 0 flags: 0x40210400000000 nid: local remote: 0x0 expref: -99 pid: 6255 timeout: 0 lvb_type: 0 00010000:00010000:3.0:1776178159.471373:0:3340:0:(ldlm_request.c:1257:ldlm_cli_cancel_local()) ### server-side local cancel ns: mdt-lustre-MDT0001_UUID lock: 0000000031bb7277/0xa40083ab9bf5355d lrc: 4/0,0 mode: COS/COS res: [0x240000402:0x7fc:0x0].0x0 bits 0x13/0x0 rrc: 2 type: IBT gid 0 flags: 0x40218400000000 nid: local remote: 0x0 expref: -99 pid: 6255 timeout: 0 lvb_type: 0 00010000:00010000:3.0:1776178159.471380:0:3340:0:(ldlm_inodebits.c:94:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x240000402:0x7fc:0x0].0x0 (0000000064abeb1c) 00010000:00010000:3.0:1776178159.471382:0:3340:0:(ldlm_inodebits.c:96:ldlm_reprocess_inodebits_queue()) Hint 13 00010000:00010000:3.0:1776178159.471383:0:3340:0:(ldlm_lockd.c:1903:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: mdt-lustre-MDT0001_UUID lock: 0000000031bb7277/0xa40083ab9bf5355d lrc: 2/0,0 mode: --/COS res: [0x240000402:0x7fc:0x0].0x0 bits 0x13/0x0 rrc: 2 type: IBT gid 0 flags: 0x44a19400000000 nid: local remote: 0x0 expref: -99 pid: 6255 timeout: 0 lvb_type: 0 00010000:00010000:3.0:1776178159.471393:0:3340:0:(ldlm_lock.c:216:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: mdt-lustre-MDT0001_UUID lock: 0000000031bb7277/0xa40083ab9bf5355d lrc: 0/0,0 mode: --/COS res: [0x240000402:0x7fc:0x0].0x0 bits 0x13/0x0 rrc: 2 type: IBT gid 0 flags: 0x44a19400000000 nid: local remote: 0x0 expref: -99 pid: 6255 timeout: 0 lvb_type: 0 00010000:00010000:3.0:1776178159.471406:0:3340:0:(ldlm_lock.c:820:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(COS) ns: mdt-lustre-MDT0001_UUID lock: 000000006a188f9a/0xa40083ab9bf53564 lrc: 3/0,1 mode: COS/COS res: [0x240000402:0x7fd:0x0].0x0 bits 0x13/0x0 rrc: 2 type: IBT gid 0 flags: 0x50210000000000 nid: local remote: 0x0 expref: -99 pid: 6256 timeout: 0 lvb_type: 0 00010000:00010000:3.0:1776178159.471411:0:3340:0:(ldlm_lock.c:879:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: mdt-lustre-MDT0001_UUID lock: 000000006a188f9a/0xa40083ab9bf53564 lrc: 2/0,0 mode: COS/COS res: [0x240000402:0x7fd:0x0].0x0 bits 0x13/0x0 rrc: 2 type: IBT gid 0 flags: 0x50210400000000 nid: local remote: 0x0 expref: -99 pid: 6256 timeout: 0 lvb_type: 0 00010000:00010000:3.0:1776178159.471417:0:3340:0:(ldlm_lockd.c:1879:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: mdt-lustre-MDT0001_UUID lock: 000000006a188f9a/0xa40083ab9bf53564 lrc: 3/0,0 mode: COS/COS res: [0x240000402:0x7fd:0x0].0x0 bits 0x13/0x0 rrc: 2 type: IBT gid 0 flags: 0x40210400000000 nid: local remote: 0x0 expref: -99 pid: 6256 timeout: 0 lvb_type: 0 00010000:00010000:3.0:1776178159.471421:0:3340:0:(ldlm_lockd.c:1893:ldlm_handle_bl_callback()) Lock 000000006a188f9a already unused, calling callback (0000000079e78348) 00010000:00010000:3.0:1776178159.471423:0:3340:0:(ldlm_request.c:364:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: mdt-lustre-MDT0001_UUID lock: 000000006a188f9a/0xa40083ab9bf53564 lrc: 3/0,0 mode: COS/COS res: [0x240000402:0x7fd:0x0].0x0 bits 0x13/0x0 rrc: 2 type: IBT gid 0 flags: 0x40210400000000 nid: local remote: 0x0 expref: -99 pid: 6256 timeout: 0 lvb_type: 0 00010000:00010000:3.0:1776178159.471428:0:3340:0:(ldlm_request.c:1257:ldlm_cli_cancel_local()) ### server-side local cancel ns: mdt-lustre-MDT0001_UUID lock: 000000006a188f9a/0xa40083ab9bf53564 lrc: 4/0,0 mode: COS/COS res: [0x240000402:0x7fd:0x0].0x0 bits 0x13/0x0 rrc: 2 type: IBT gid 0 flags: 0x40218400000000 nid: local remote: 0x0 expref: -99 pid: 6256 timeout: 0 lvb_type: 0 00010000:00010000:3.0:1776178159.471435:0:3340:0:(ldlm_inodebits.c:94:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x240000402:0x7fd:0x0].0x0 (000000008d47e759) 00010000:00010000:3.0:1776178159.471437:0:3340:0:(ldlm_inodebits.c:96:ldlm_reprocess_inodebits_queue()) Hint 13 00010000:00010000:3.0:1776178159.471438:0:3340:0:(ldlm_lockd.c:1903:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: mdt-lustre-MDT0001_UUID lock: 000000006a188f9a/0xa40083ab9bf53564 lrc: 2/0,0 mode: --/COS res: [0x240000402:0x7fd:0x0].0x0 bits 0x13/0x0 rrc: 2 type: IBT gid 0 flags: 0x44a19400000000 nid: local remote: 0x0 expref: -99 pid: 6256 timeout: 0 lvb_type: 0 00010000:00010000:3.0:1776178159.471444:0:3340:0:(ldlm_lock.c:216:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: mdt-lustre-MDT0001_UUID lock: 000000006a188f9a/0xa40083ab9bf53564 lrc: 0/0,0 mode: --/COS res: [0x240000402:0x7fd:0x0].0x0 bits 0x13/0x0 rrc: 2 type: IBT gid 0 flags: 0x44a19400000000 nid: local remote: 0x0 expref: -99 pid: 6256 timeout: 0 lvb_type: 0 00000100:00100000:3.0:1776178159.471509:0:3356:0:(client.c:2215:ptlrpc_check_set()) Completed RPC req@000000004a9818ac pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_03:lustre-MDT0000-lwp-OST0000_UUID:3356:1862456998036288:0@lo:400:kworker/u8:0.0 00000100:00100000:3.0:1776178159.471523:0:3356:0:(client.c:2215:ptlrpc_check_set()) Completed RPC req@000000002dd26ce5 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_03:lustre-MDT0001-mdtlov_UUID:3356:1862456998036544:0@lo:400:kworker/u8:0.0 00000100:00100000:3.0:1776178159.471777:0:3356:0:(client.c:2215:ptlrpc_check_set()) Completed RPC req@0000000025f8393e pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_03:lustre-MDT0000-lwp-MDT0000_UUID:3356:1862456998036800:0@lo:400:kworker/u8:0.0 00000100:00100000:2.0:1776178159.473723:0:6261:0:(service.c:2152:ptlrpc_server_handle_req_in()) got req x1862456998036224 00000100:00100000:2.0:1776178159.473795:0:6261:0:(nrs_fifo.c:179:nrs_fifo_req_get()) NRS start fifo request from 12345-0@lo, seq: 664 00000100:00100000:2.0:1776178159.473802:0:6261:0:(service.c:2324:ptlrpc_server_handle_request()) Handling RPC req@0000000015f283a6 pname:cluuid+ref:pid:xid:nid:opc:job mdt_out00_001:lustre-MDT0001-mdtlov_UUID+6:3355:x1862456998036224:12345-0@lo:41:osp-pre-0-1.0 00000004:00000024:2.0:1776178159.473839:0:6261:0:(mdt_handler.c:548:mdt_statfs()) blocks cached 0 granted 2097152 pending 0 free 1446121472 avail 1316659200 00000100:00100000:2.0:1776178159.473924:0:6261:0:(service.c:2372:ptlrpc_server_handle_request()) Handled RPC req@0000000015f283a6 pname:cluuid+ref:pid:xid:nid:opc:job mdt_out00_001:lustre-MDT0001-mdtlov_UUID+6:3355:x1862456998036224:12345-0@lo:41:osp-pre-0-1.0 Request processed in 121us (9380us total) trans 0 rc 0/0 00000100:00100000:2.0:1776178159.473931:0:6261:0:(nrs_fifo.c:241:nrs_fifo_req_stop()) NRS stop fifo request from 12345-0@lo, seq: 664 00000100:00100000:3.0:1776178159.474006:0:3355:0:(client.c:2215:ptlrpc_check_set()) Completed RPC req@000000004eee88be pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_02:lustre-MDT0001-mdtlov_UUID:3355:1862456998036224:0@lo:41:osp-pre-0-1.0 00010000:00010000:3.0:1776178160.034493:0:9225:0:(ldlm_lockd.c:2792:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:1.0:1776178160.933226:0:3346:0:(events.c:361:request_in_callback()) peer: 12345-192.168.204.3@tcp (source: 12345-192.168.204.3@tcp) 00000100:00100000:1.0:1776178160.933296:0:3346:0:(events.c:361:request_in_callback()) peer: 12345-192.168.204.3@tcp (source: 12345-192.168.204.3@tcp) 00000100:00100000:1.0:1776178160.933361:0:3346:0:(events.c:361:request_in_callback()) peer: 12345-192.168.204.3@tcp (source: 12345-192.168.204.3@tcp) 00000100:00100000:1.0:1776178160.933390:0:3346:0:(events.c:361:request_in_callback()) peer: 12345-192.168.204.3@tcp (source: 12345-192.168.204.3@tcp) 00000100:00100000:1.0:1776178160.933431:0:3346:0:(events.c:361:request_in_callback()) peer: 12345-192.168.204.3@tcp (source: 12345-192.168.204.3@tcp) 00000100:00100000:1.0:1776178160.933477:0:6257:0:(service.c:2152:ptlrpc_server_handle_req_in()) got req x1862456979095360 00000100:00100000:1.0:1776178160.933551:0:6257:0:(service.c:2152:ptlrpc_server_handle_req_in()) got req x1862456979095424 00000100:00100000:1.0:1776178160.933577:0:6257:0:(nrs_fifo.c:179:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.204.3@tcp, seq: 842 00000100:00100000:1.0:1776178160.933586:0:6257:0:(service.c:2324:ptlrpc_server_handle_request()) Handling RPC req@0000000083b11d73 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_002:b50963bd-8eba-4432-9a54-8888d00e3fa1+7:2273:x1862456979095360:12345-192.168.204.3@tcp:400:kworker/u8:4.0 00000100:00100000:1.0:1776178160.933673:0:6257:0:(service.c:2372:ptlrpc_server_handle_request()) Handled RPC req@0000000083b11d73 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_002:b50963bd-8eba-4432-9a54-8888d00e3fa1+7:2273:x1862456979095360:12345-192.168.204.3@tcp:400:kworker/u8:4.0 Request processed in 86us (462us total) trans 0 rc 0/0 00000100:00100000:1.0:1776178160.933682:0:6257:0:(nrs_fifo.c:241:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.204.3@tcp, seq: 842 00000100:00100000:1.0:1776178160.933701:0:6257:0:(nrs_fifo.c:179:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.204.3@tcp, seq: 843 00000100:00100000:1.0:1776178160.933706:0:6257:0:(service.c:2324:ptlrpc_server_handle_request()) Handling RPC req@00000000a0c64d21 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_002:b50963bd-8eba-4432-9a54-8888d00e3fa1+6:2273:x1862456979095424:12345-192.168.204.3@tcp:400:kworker/u8:4.0 00000100:00100000:1.0:1776178160.933736:0:6257:0:(service.c:2372:ptlrpc_server_handle_request()) Handled RPC req@00000000a0c64d21 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_002:b50963bd-8eba-4432-9a54-8888d00e3fa1+6:2273:x1862456979095424:12345-192.168.204.3@tcp:400:kworker/u8:4.0 Request processed in 29us (345us total) trans 0 rc 0/0 00000100:00100000:1.0:1776178160.933757:0:6257:0:(nrs_fifo.c:241:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.204.3@tcp, seq: 843 00000100:00100000:1.0:1776178160.933779:0:20140:0:(service.c:2152:ptlrpc_server_handle_req_in()) got req x1862456979095296 00000100:00100000:1.0:1776178160.933800:0:20140:0:(nrs_fifo.c:179:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.204.3@tcp, seq: 139 00000100:00100000:1.0:1776178160.933804:0:20140:0:(service.c:2324:ptlrpc_server_handle_request()) Handling RPC req@00000000ba1c7e2e pname:cluuid+ref:pid:xid:nid:opc:job ll_mgs_0002:5b5c66b1-1a2f-4651-8548-0c75b9a38a94+9:2273:x1862456979095296:12345-192.168.204.3@tcp:400:kworker/u8:4.0 00000100:00100000:1.0:1776178160.933826:0:20140:0:(service.c:2372:ptlrpc_server_handle_request()) Handled RPC req@00000000ba1c7e2e pname:cluuid+ref:pid:xid:nid:opc:job ll_mgs_0002:5b5c66b1-1a2f-4651-8548-0c75b9a38a94+9:2273:x1862456979095296:12345-192.168.204.3@tcp:400:kworker/u8:4.0 Request processed in 21us (466us total) trans 0 rc 0/0 00000100:00100000:1.0:1776178160.933842:0:20140:0:(nrs_fifo.c:241:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.204.3@tcp, seq: 139 00000100:00100000:1.0:1776178160.933872:0:20762:0:(service.c:2152:ptlrpc_server_handle_req_in()) got req x1862456979095488 00000100:00100000:1.0:1776178160.933893:0:20762:0:(service.c:2152:ptlrpc_server_handle_req_in()) got req x1862456979095552 00000100:00100000:1.0:1776178160.933908:0:20762:0:(nrs_fifo.c:179:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.204.3@tcp, seq: 466 00000100:00100000:1.0:1776178160.933912:0:20762:0:(service.c:2324:ptlrpc_server_handle_request()) Handling RPC req@00000000b85458e9 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_013:b50963bd-8eba-4432-9a54-8888d00e3fa1+8:2273:x1862456979095488:12345-192.168.204.3@tcp:400:kworker/u8:4.0 00000100:00100000:1.0:1776178160.933950:0:20762:0:(service.c:2372:ptlrpc_server_handle_request()) Handled RPC req@00000000b85458e9 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_013:b50963bd-8eba-4432-9a54-8888d00e3fa1+8:2273:x1862456979095488:12345-192.168.204.3@tcp:400:kworker/u8:4.0 Request processed in 37us (655us total) trans 0 rc 0/0 00000100:00100000:1.0:1776178160.933956:0:20762:0:(nrs_fifo.c:241:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.204.3@tcp, seq: 466 00000100:00100000:1.0:1776178160.933966:0:20762:0:(nrs_fifo.c:179:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.204.3@tcp, seq: 467 00000100:00100000:1.0:1776178160.933970:0:20762:0:(service.c:2324:ptlrpc_server_handle_request()) Handling RPC req@000000007d766b2a pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_013:b50963bd-8eba-4432-9a54-8888d00e3fa1+9:2273:x1862456979095552:12345-192.168.204.3@tcp:400:kworker/u8:4.0 00000100:00100000:1.0:1776178160.933990:0:20762:0:(service.c:2372:ptlrpc_server_handle_request()) Handled RPC req@000000007d766b2a pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_013:b50963bd-8eba-4432-9a54-8888d00e3fa1+9:2273:x1862456979095552:12345-192.168.204.3@tcp:400:kworker/u8:4.0 Request processed in 19us (560us total) trans 0 rc 0/0 00000100:00100000:1.0:1776178160.933995:0:20762:0:(nrs_fifo.c:241:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.204.3@tcp, seq: 467 00000100:00100000:0.0:1776178160.993585:0:3354:0:(ptlrpcd.c:415:ptlrpcd_check()) transfer 1 async RPCs [0->1] 00000100:00100000:0.0:1776178160.993609:0:3354:0:(client.c:1733:ptlrpc_send_new_req()) Sending RPC req@00000000f7d56588 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_01:lustre-MDT0001-mdtlov_UUID:3354:1862456998036864:0@lo:13:osp-pre-1-1.0 00000100:00100000:0.0:1776178160.993689:0:3354:0:(events.c:361:request_in_callback()) peer: 12345-0@lo (source: 12345-0@lo) 00000100:00100000:1.0:1776178160.994145:0:8232:0:(service.c:2152:ptlrpc_server_handle_req_in()) got req x1862456998036864 00000100:00100000:1.0:1776178160.994200:0:8232:0:(nrs_fifo.c:179:nrs_fifo_req_get()) NRS start fifo request from 12345-0@lo, seq: 445 00000100:00100000:1.0:1776178160.994206:0:8232:0:(service.c:2324:ptlrpc_server_handle_request()) Handling RPC req@000000009a8d7b96 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_create00:lustre-MDT0001-mdtlov_UUID+5:3354:x1862456998036864:12345-0@lo:13:osp-pre-1-1.0 00002000:00000024:1.0:1776178160.994246:0:8232:0:(ofd_obd.c:740:ofd_statfs()) blocks cached 229376 granted 491746368 pending 0 free 3923689472 avail 3692167168 00000100:00100000:1.0:1776178160.994299:0:8232:0:(service.c:2372:ptlrpc_server_handle_request()) Handled RPC req@000000009a8d7b96 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_create00:lustre-MDT0001-mdtlov_UUID+5:3354:x1862456998036864:12345-0@lo:13:osp-pre-1-1.0 Request processed in 93us (611us total) trans 0 rc 0/0 00000100:00100000:1.0:1776178160.994306:0:8232:0:(nrs_fifo.c:241:nrs_fifo_req_stop()) NRS stop fifo request from 12345-0@lo, seq: 445 00000100:00100000:2.0:1776178160.994473:0:3354:0:(client.c:2215:ptlrpc_check_set()) Completed RPC req@00000000f7d56588 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_01:lustre-MDT0001-mdtlov_UUID:3354:1862456998036864:0@lo:13:osp-pre-1-1.0 00010000:00010000:2.0:1776178161.058507:0:9224:0:(ldlm_lockd.c:2792:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:3.0:1776178161.504443:0:3353:0:(ptlrpcd.c:415:ptlrpcd_check()) transfer 1 async RPCs [1->0] 00000100:00100000:3.0:1776178161.504478:0:3353:0:(client.c:1733:ptlrpc_send_new_req()) Sending RPC req@00000000c19c19b9 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_00:lustre-MDT0001-mdtlov_UUID:3353:1862456998036928:0@lo:13:osp-pre-0-1.0 00000100:00100000:3.0:1776178161.504575:0:3353:0:(events.c:361:request_in_callback()) peer: 12345-0@lo (source: 12345-0@lo) 00000100:00100000:0.0:1776178161.504712:0:9554:0:(service.c:2152:ptlrpc_server_handle_req_in()) got req x1862456998036928 00000100:00100000:0.0:1776178161.504771:0:9554:0:(nrs_fifo.c:179:nrs_fifo_req_get()) NRS start fifo request from 12345-0@lo, seq: 446 00000100:00100000:0.0:1776178161.504778:0:9554:0:(service.c:2324:ptlrpc_server_handle_request()) Handling RPC req@00000000f7d56588 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_create00:lustre-MDT0001-mdtlov_UUID+5:3353:x1862456998036928:12345-0@lo:13:osp-pre-0-1.0 00002000:00000024:0.0:1776178161.504812:0:9554:0:(ofd_obd.c:740:ofd_statfs()) blocks cached 315392 granted 484883776 pending 0 free 3923689472 avail 3692167168 00000100:00100000:0.0:1776178161.504865:0:9554:0:(service.c:2372:ptlrpc_server_handle_request()) Handled RPC req@00000000f7d56588 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_create00:lustre-MDT0001-mdtlov_UUID+5:3353:x1862456998036928:12345-0@lo:13:osp-pre-0-1.0 Request processed in 86us (292us total) trans 0 rc 0/0 00000100:00100000:0.0:1776178161.504872:0:9554:0:(nrs_fifo.c:241:nrs_fifo_req_stop()) NRS stop fifo request from 12345-0@lo, seq: 446 00000100:00100000:3.0:1776178161.505025:0:3353:0:(client.c:2215:ptlrpc_check_set()) Completed RPC req@00000000c19c19b9 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_00:lustre-MDT0001-mdtlov_UUID:3353:1862456998036928:0@lo:13:osp-pre-0-1.0 00010000:00010000:1.0:1776178162.084267:0:6245:0:(ldlm_lockd.c:2792:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00010000:00010000:2.0:1776178163.104459:0:6244:0:(ldlm_lockd.c:2792:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00010000:00010000:1.0:1776178164.128689:0:9225:0:(ldlm_lockd.c:2792:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:2.0:1776178164.576466:0:3354:0:(client.c:1733:ptlrpc_send_new_req()) Sending RPC req@000000006c748d80 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_01:lustre-MDT0000-lwp-OST0001_UUID:3354:1862456998037184:0@lo:400:kworker/u8:0.0 00000100:00100000:2.0:1776178164.576596:0:3354:0:(events.c:361:request_in_callback()) peer: 12345-0@lo (source: 12345-0@lo) 00000100:00100000:2.0:1776178164.576647:0:3354:0:(client.c:1733:ptlrpc_send_new_req()) Sending RPC req@00000000e1d2e300 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_01:lustre-MDT0001-mdtlov_UUID:3354:1862456998037440:0@lo:400:kworker/u8:0.0 00000100:00100000:2.0:1776178164.576729:0:3354:0:(events.c:361:request_in_callback()) peer: 12345-0@lo (source: 12345-0@lo) 00000100:00100000:2.0:1776178164.576780:0:3354:0:(client.c:1733:ptlrpc_send_new_req()) Sending RPC req@000000003b436fee pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_01:lustre-MDT0000-mdtlov_UUID:3354:1862456998037696:0@lo:400:kworker/u8:0.0 00000100:00100000:2.0:1776178164.576803:0:3354:0:(events.c:361:request_in_callback()) peer: 12345-0@lo (source: 12345-0@lo) 00000100:00100000:2.0:1776178164.576828:0:3354:0:(ptlrpcd.c:415:ptlrpcd_check()) transfer 3 async RPCs [0->1] 00000100:00100000:2.0:1776178164.576834:0:3354:0:(client.c:1733:ptlrpc_send_new_req()) Sending RPC req@000000002dd26ce5 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_01:lustre-MDT0001-lwp-OST0000_UUID:3354:1862456998037120:0@lo:400:kworker/u8:0.0 00000100:00100000:2.0:1776178164.576853:0:3354:0:(events.c:361:request_in_callback()) peer: 12345-0@lo (source: 12345-0@lo) 00000100:00100000:2.0:1776178164.576867:0:3354:0:(client.c:1733:ptlrpc_send_new_req()) Sending RPC req@00000000e1965319 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_01:lustre-MDT0001-mdtlov_UUID:3354:1862456998037376:0@lo:400:kworker/u8:0.0 00000100:00100000:2.0:1776178164.576886:0:3354:0:(events.c:361:request_in_callback()) peer: 12345-0@lo (source: 12345-0@lo) 00000100:00100000:3.0:1776178164.576893:0:6256:0:(service.c:2152:ptlrpc_server_handle_req_in()) got req x1862456998037184 00000100:00100000:2.0:1776178164.576898:0:3354:0:(client.c:1733:ptlrpc_send_new_req()) Sending RPC req@00000000775bd6f2 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_01:lustre-MDT0000-mdtlov_UUID:3354:1862456998037632:0@lo:400:kworker/u8:0.0 00000100:00100000:2.0:1776178164.576916:0:3354:0:(events.c:361:request_in_callback()) peer: 12345-0@lo (source: 12345-0@lo) 00000100:00100000:3.0:1776178164.576942:0:6256:0:(service.c:2152:ptlrpc_server_handle_req_in()) got req x1862456998037120 00000100:00100000:2.0:1776178164.576944:0:20769:0:(service.c:2152:ptlrpc_server_handle_req_in()) got req x1862456998037440 00000100:00100000:3.0:1776178164.576961:0:6256:0:(nrs_fifo.c:179:nrs_fifo_req_get()) NRS start fifo request from 12345-0@lo, seq: 844 00000100:00100000:2.0:1776178164.576965:0:20769:0:(service.c:2152:ptlrpc_server_handle_req_in()) got req x1862456998037696 00000100:00100000:3.0:1776178164.576967:0:6256:0:(service.c:2324:ptlrpc_server_handle_request()) Handling RPC req@0000000039cf69e0 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_001:lustre-MDT0000-lwp-OST0001_UUID+5:3354:x1862456998037184:12345-0@lo:400:kworker/u8:0.0 00000100:00100000:2.0:1776178164.576979:0:20769:0:(service.c:2152:ptlrpc_server_handle_req_in()) got req x1862456998037376 00000100:00100000:2.0:1776178164.576990:0:20769:0:(service.c:2152:ptlrpc_server_handle_req_in()) got req x1862456998037632 00000100:00100000:2.0:1776178164.577002:0:20769:0:(nrs_fifo.c:179:nrs_fifo_req_get()) NRS start fifo request from 12345-0@lo, seq: 468 00000100:00100000:2.0:1776178164.577006:0:20769:0:(service.c:2324:ptlrpc_server_handle_request()) Handling RPC req@000000009d7a00b9 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_018:lustre-MDT0001-mdtlov_UUID+5:3354:x1862456998037440:12345-0@lo:400:kworker/u8:0.0 00000100:00100000:3.0:1776178164.577025:0:6256:0:(service.c:2372:ptlrpc_server_handle_request()) Handled RPC req@0000000039cf69e0 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_001:lustre-MDT0000-lwp-OST0001_UUID+5:3354:x1862456998037184:12345-0@lo:400:kworker/u8:0.0 Request processed in 58us (433us total) trans 0 rc 0/0 00000100:00100000:2.0:1776178164.577030:0:20769:0:(service.c:2372:ptlrpc_server_handle_request()) Handled RPC req@000000009d7a00b9 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_018:lustre-MDT0001-mdtlov_UUID+5:3354:x1862456998037440:12345-0@lo:400:kworker/u8:0.0 Request processed in 24us (304us total) trans 0 rc 0/0 00000100:00100000:3.0:1776178164.577033:0:6256:0:(nrs_fifo.c:241:nrs_fifo_req_stop()) NRS stop fifo request from 12345-0@lo, seq: 844 00000100:00100000:2.0:1776178164.577034:0:20769:0:(nrs_fifo.c:241:nrs_fifo_req_stop()) NRS stop fifo request from 12345-0@lo, seq: 468 00000100:00100000:3.0:1776178164.577047:0:6256:0:(nrs_fifo.c:179:nrs_fifo_req_get()) NRS start fifo request from 12345-0@lo, seq: 845 00000100:00100000:3.0:1776178164.577050:0:6256:0:(service.c:2324:ptlrpc_server_handle_request()) Handling RPC req@00000000e1d307fb pname:cluuid+ref:pid:xid:nid:opc:job mdt00_001:lustre-MDT0001-lwp-OST0000_UUID+5:3354:x1862456998037120:12345-0@lo:400:kworker/u8:0.0 00000100:00100000:2.0:1776178164.577080:0:20769:0:(nrs_fifo.c:179:nrs_fifo_req_get()) NRS start fifo request from 12345-0@lo, seq: 469 00000100:00100000:2.0:1776178164.577083:0:20769:0:(service.c:2324:ptlrpc_server_handle_request()) Handling RPC req@000000008aab9f17 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_018:lustre-MDT0000-mdtlov_UUID+5:3354:x1862456998037696:12345-0@lo:400:kworker/u8:0.0 00000100:00100000:3.0:1776178164.577096:0:6256:0:(service.c:2372:ptlrpc_server_handle_request()) Handled RPC req@00000000e1d307fb pname:cluuid+ref:pid:xid:nid:opc:job mdt00_001:lustre-MDT0001-lwp-OST0000_UUID+5:3354:x1862456998037120:12345-0@lo:400:kworker/u8:0.0 Request processed in 45us (243us total) trans 0 rc 0/0 00000100:00100000:2.0:1776178164.577102:0:20769:0:(service.c:2372:ptlrpc_server_handle_request()) Handled RPC req@000000008aab9f17 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_018:lustre-MDT0000-mdtlov_UUID+5:3354:x1862456998037696:12345-0@lo:400:kworker/u8:0.0 Request processed in 19us (300us total) trans 0 rc 0/0 00000100:00100000:3.0:1776178164.577102:0:6256:0:(nrs_fifo.c:241:nrs_fifo_req_stop()) NRS stop fifo request from 12345-0@lo, seq: 845 00000100:00100000:2.0:1776178164.577107:0:20769:0:(nrs_fifo.c:241:nrs_fifo_req_stop()) NRS stop fifo request from 12345-0@lo, seq: 469 00000100:00100000:2.0:1776178164.577115:0:20769:0:(nrs_fifo.c:179:nrs_fifo_req_get()) NRS start fifo request from 12345-0@lo, seq: 470 00000100:00100000:2.0:1776178164.577118:0:20769:0:(service.c:2324:ptlrpc_server_handle_request()) Handling RPC req@00000000dab5abff pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_018:lustre-MDT0001-mdtlov_UUID+5:3354:x1862456998037376:12345-0@lo:400:kworker/u8:0.0 00000100:00100000:2.0:1776178164.577138:0:20769:0:(service.c:2372:ptlrpc_server_handle_request()) Handled RPC req@00000000dab5abff pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_018:lustre-MDT0001-mdtlov_UUID+5:3354:x1862456998037376:12345-0@lo:400:kworker/u8:0.0 Request processed in 19us (252us total) trans 0 rc 0/0 00000100:00100000:2.0:1776178164.577143:0:20769:0:(nrs_fifo.c:241:nrs_fifo_req_stop()) NRS stop fifo request from 12345-0@lo, seq: 470 00000100:00100000:3.0:1776178164.577143:0:3356:0:(client.c:1733:ptlrpc_send_new_req()) Sending RPC req@0000000025f8393e pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_03:lustre-MDT0000-lwp-OST0000_UUID:3356:1862456998037056:0@lo:400:kworker/u8:0.0 00000100:00100000:2.0:1776178164.577152:0:20769:0:(nrs_fifo.c:179:nrs_fifo_req_get()) NRS start fifo request from 12345-0@lo, seq: 471 00000100:00100000:2.0:1776178164.577155:0:20769:0:(service.c:2324:ptlrpc_server_handle_request()) Handling RPC req@00000000f44a2faf pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_018:lustre-MDT0000-mdtlov_UUID+5:3354:x1862456998037632:12345-0@lo:400:kworker/u8:0.0 00000100:00100000:3.0:1776178164.577167:0:3356:0:(events.c:361:request_in_callback()) peer: 12345-0@lo (source: 12345-0@lo) 00000100:00100000:2.0:1776178164.577173:0:20769:0:(service.c:2372:ptlrpc_server_handle_request()) Handled RPC req@00000000f44a2faf pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_018:lustre-MDT0000-mdtlov_UUID+5:3354:x1862456998037632:12345-0@lo:400:kworker/u8:0.0 Request processed in 17us (257us total) trans 0 rc 0/0 00000100:00100000:2.0:1776178164.577178:0:20769:0:(nrs_fifo.c:241:nrs_fifo_req_stop()) NRS stop fifo request from 12345-0@lo, seq: 471 00000100:00100000:2.0:1776178164.577222:0:3354:0:(client.c:2215:ptlrpc_check_set()) Completed RPC req@000000002dd26ce5 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_01:lustre-MDT0001-lwp-OST0000_UUID:3354:1862456998037120:0@lo:400:kworker/u8:0.0 00000100:00100000:2.0:1776178164.577231:0:3354:0:(client.c:2215:ptlrpc_check_set()) Completed RPC req@00000000e1965319 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_01:lustre-MDT0001-mdtlov_UUID:3354:1862456998037376:0@lo:400:kworker/u8:0.0 00000100:00100000:2.0:1776178164.577238:0:3354:0:(client.c:2215:ptlrpc_check_set()) Completed RPC req@00000000775bd6f2 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_01:lustre-MDT0000-mdtlov_UUID:3354:1862456998037632:0@lo:400:kworker/u8:0.0 00000100:00100000:2.0:1776178164.577245:0:3354:0:(client.c:2215:ptlrpc_check_set()) Completed RPC req@000000006c748d80 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_01:lustre-MDT0000-lwp-OST0001_UUID:3354:1862456998037184:0@lo:400:kworker/u8:0.0 00000100:00100000:2.0:1776178164.577252:0:3354:0:(client.c:2215:ptlrpc_check_set()) Completed RPC req@00000000e1d2e300 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_01:lustre-MDT0001-mdtlov_UUID:3354:1862456998037440:0@lo:400:kworker/u8:0.0 00000100:00100000:2.0:1776178164.577259:0:3354:0:(client.c:2215:ptlrpc_check_set()) Completed RPC req@000000003b436fee pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_01:lustre-MDT0000-mdtlov_UUID:3354:1862456998037696:0@lo:400:kworker/u8:0.0 00000100:00100000:3.0:1776178164.577298:0:6256:0:(service.c:2152:ptlrpc_server_handle_req_in()) got req x1862456998037056 00000100:00100000:2.0:1776178164.577305:0:3356:0:(client.c:1733:ptlrpc_send_new_req()) Sending RPC req@000000007886b426 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_03:lustre-MDT0001-mdtlov_UUID:3356:1862456998037312:0@lo:400:kworker/u8:0.0 00000100:00100000:3.0:1776178164.577320:0:6256:0:(nrs_fifo.c:179:nrs_fifo_req_get()) NRS start fifo request from 12345-0@lo, seq: 846 00000100:00100000:3.0:1776178164.577323:0:6256:0:(service.c:2324:ptlrpc_server_handle_request()) Handling RPC req@000000006be1db3a pname:cluuid+ref:pid:xid:nid:opc:job mdt00_001:lustre-MDT0000-lwp-OST0000_UUID+5:3356:x1862456998037056:12345-0@lo:400:kworker/u8:0.0 00000100:00100000:2.0:1776178164.577327:0:3356:0:(events.c:361:request_in_callback()) peer: 12345-0@lo (source: 12345-0@lo) 00000100:00100000:2.0:1776178164.577340:0:3356:0:(client.c:1733:ptlrpc_send_new_req()) Sending RPC req@0000000000167c64 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_03:lustre-MDT0000-mdtlov_UUID:3356:1862456998037568:0@lo:400:kworker/u8:0.0 00000100:00100000:3.0:1776178164.577348:0:6256:0:(service.c:2372:ptlrpc_server_handle_request()) Handled RPC req@000000006be1db3a pname:cluuid+ref:pid:xid:nid:opc:job mdt00_001:lustre-MDT0000-lwp-OST0000_UUID+5:3356:x1862456998037056:12345-0@lo:400:kworker/u8:0.0 Request processed in 24us (181us total) trans 0 rc 0/0 00000100:00100000:3.0:1776178164.577353:0:6256:0:(nrs_fifo.c:241:nrs_fifo_req_stop()) NRS stop fifo request from 12345-0@lo, seq: 846 00000100:00100000:2.0:1776178164.577356:0:3356:0:(events.c:361:request_in_callback()) peer: 12345-0@lo (source: 12345-0@lo) 00000100:00100000:2.0:1776178164.577366:0:3356:0:(client.c:1733:ptlrpc_send_new_req()) Sending RPC req@000000007794b3ac pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_03:lustre-MDT0000-mdtlov_UUID:3356:1862456998037824:0@lo:41:osp-pre-1-0.0 00000100:00100000:2.0:1776178164.577384:0:3356:0:(events.c:361:request_in_callback()) peer: 12345-0@lo (source: 12345-0@lo) 00000100:00100000:2.0:1776178164.577406:0:3356:0:(client.c:2215:ptlrpc_check_set()) Completed RPC req@0000000025f8393e pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_03:lustre-MDT0000-lwp-OST0000_UUID:3356:1862456998037056:0@lo:400:kworker/u8:0.0 00000100:00100000:3.0:1776178164.577408:0:6261:0:(service.c:2152:ptlrpc_server_handle_req_in()) got req x1862456998037312 00000100:00100000:2.0:1776178164.577414:0:3356:0:(ptlrpcd.c:415:ptlrpcd_check()) transfer 4 async RPCs [2->3] 00000100:00100000:2.0:1776178164.577418:0:3356:0:(client.c:1733:ptlrpc_send_new_req()) Sending RPC req@0000000092f98018 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_03:ac61f839-0e8e-4cac-9c59-08bcfbbad330:3356:1862456998036992:0@lo:400:kworker/u8:0.0 00000100:00100000:3.0:1776178164.577427:0:6261:0:(service.c:2152:ptlrpc_server_handle_req_in()) got req x1862456998037568 00000100:00100000:3.0:1776178164.577434:0:6261:0:(service.c:2152:ptlrpc_server_handle_req_in()) got req x1862456998037824 00000100:00100000:2.0:1776178164.577443:0:3356:0:(events.c:361:request_in_callback()) peer: 12345-0@lo (source: 12345-0@lo) 00000100:00100000:3.0:1776178164.577443:0:6261:0:(nrs_fifo.c:179:nrs_fifo_req_get()) NRS start fifo request from 12345-0@lo, seq: 665 00000100:00100000:3.0:1776178164.577445:0:6261:0:(service.c:2324:ptlrpc_server_handle_request()) Handling RPC req@000000003b436fee pname:cluuid+ref:pid:xid:nid:opc:job mdt_out00_001:lustre-MDT0001-mdtlov_UUID+6:3356:x1862456998037312:12345-0@lo:400:kworker/u8:0.0 00000100:00100000:2.0:1776178164.577468:0:3356:0:(client.c:1733:ptlrpc_send_new_req()) Sending RPC req@00000000165897bf pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_03:lustre-MDT0001-lwp-OST0001_UUID:3356:1862456998037248:0@lo:400:kworker/u8:0.0 00000100:00100000:3.0:1776178164.577482:0:6261:0:(service.c:2372:ptlrpc_server_handle_request()) Handled RPC req@000000003b436fee pname:cluuid+ref:pid:xid:nid:opc:job mdt_out00_001:lustre-MDT0001-mdtlov_UUID+6:3356:x1862456998037312:12345-0@lo:400:kworker/u8:0.0 Request processed in 36us (155us total) trans 0 rc 0/0 00000100:00100000:2.0:1776178164.577485:0:3356:0:(events.c:361:request_in_callback()) peer: 12345-0@lo (source: 12345-0@lo) 00000100:00100000:3.0:1776178164.577488:0:6261:0:(nrs_fifo.c:241:nrs_fifo_req_stop()) NRS stop fifo request from 12345-0@lo, seq: 665 00000100:00100000:2.0:1776178164.577497:0:3356:0:(client.c:1733:ptlrpc_send_new_req()) Sending RPC req@000000002cf093e4 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_03:lustre-MDT0000-lwp-MDT0001_UUID:3356:1862456998037504:0@lo:400:kworker/u8:0.0 00000100:00100000:3.0:1776178164.577498:0:6261:0:(nrs_fifo.c:179:nrs_fifo_req_get()) NRS start fifo request from 12345-0@lo, seq: 666 00000100:00100000:3.0:1776178164.577501:0:6261:0:(service.c:2324:ptlrpc_server_handle_request()) Handling RPC req@00000000775bd6f2 pname:cluuid+ref:pid:xid:nid:opc:job mdt_out00_001:lustre-MDT0000-mdtlov_UUID+6:3356:x1862456998037568:12345-0@lo:400:kworker/u8:0.0 00000100:00100000:2.0:1776178164.577513:0:3356:0:(events.c:361:request_in_callback()) peer: 12345-0@lo (source: 12345-0@lo) 00000100:00100000:2.0:1776178164.577520:0:3356:0:(client.c:1733:ptlrpc_send_new_req()) Sending RPC req@00000000a92aab25 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_03:lustre-MDT0000-lwp-MDT0000_UUID:3356:1862456998037760:0@lo:400:kworker/u8:0.0 00000100:00100000:3.0:1776178164.577521:0:6261:0:(service.c:2372:ptlrpc_server_handle_request()) Handled RPC req@00000000775bd6f2 pname:cluuid+ref:pid:xid:nid:opc:job mdt_out00_001:lustre-MDT0000-mdtlov_UUID+6:3356:x1862456998037568:12345-0@lo:400:kworker/u8:0.0 Request processed in 19us (164us total) trans 0 rc 0/0 00000100:00100000:3.0:1776178164.577526:0:6261:0:(nrs_fifo.c:241:nrs_fifo_req_stop()) NRS stop fifo request from 12345-0@lo, seq: 666 00000100:00100000:3.0:1776178164.577534:0:6261:0:(nrs_fifo.c:179:nrs_fifo_req_get()) NRS start fifo request from 12345-0@lo, seq: 667 00000100:00100000:2.0:1776178164.577536:0:3356:0:(events.c:361:request_in_callback()) peer: 12345-0@lo (source: 12345-0@lo) 00000100:00100000:3.0:1776178164.577538:0:6261:0:(service.c:2324:ptlrpc_server_handle_request()) Handling RPC req@00000000eafab700 pname:cluuid+ref:pid:xid:nid:opc:job mdt_out00_001:lustre-MDT0000-mdtlov_UUID+5:3356:x1862456998037824:12345-0@lo:41:osp-pre-1-0.0 00000100:00100000:2.0:1776178164.577548:0:3356:0:(client.c:2215:ptlrpc_check_set()) Completed RPC req@000000007886b426 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_03:lustre-MDT0001-mdtlov_UUID:3356:1862456998037312:0@lo:400:kworker/u8:0.0 00000100:00100000:2.0:1776178164.577555:0:3356:0:(client.c:2215:ptlrpc_check_set()) Completed RPC req@0000000000167c64 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_03:lustre-MDT0000-mdtlov_UUID:3356:1862456998037568:0@lo:400:kworker/u8:0.0 00000004:00000024:3.0:1776178164.577561:0:6261:0:(mdt_handler.c:548:mdt_statfs()) blocks cached 0 granted 2097152 pending 0 free 1446035456 avail 1316573184 00000100:00100000:2.0:1776178164.577590:0:6256:0:(service.c:2152:ptlrpc_server_handle_req_in()) got req x1862456998037248 00000100:00100000:3.0:1776178164.577590:0:6261:0:(service.c:2372:ptlrpc_server_handle_request()) Handled RPC req@00000000eafab700 pname:cluuid+ref:pid:xid:nid:opc:job mdt_out00_001:lustre-MDT0000-mdtlov_UUID+5:3356:x1862456998037824:12345-0@lo:41:osp-pre-1-0.0 Request processed in 52us (206us total) trans 0 rc 0/0 00000100:00100000:3.0:1776178164.577596:0:6261:0:(nrs_fifo.c:241:nrs_fifo_req_stop()) NRS stop fifo request from 12345-0@lo, seq: 667 00000100:00100000:2.0:1776178164.577603:0:6256:0:(service.c:2152:ptlrpc_server_handle_req_in()) got req x1862456998037504 00000100:00100000:2.0:1776178164.577610:0:6256:0:(service.c:2152:ptlrpc_server_handle_req_in()) got req x1862456998037760 00000100:00100000:3.0:1776178164.577618:0:3356:0:(client.c:2215:ptlrpc_check_set()) Completed RPC req@000000007794b3ac pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_03:lustre-MDT0000-mdtlov_UUID:3356:1862456998037824:0@lo:41:osp-pre-1-0.0 00000100:00100000:2.0:1776178164.577619:0:6256:0:(nrs_fifo.c:179:nrs_fifo_req_get()) NRS start fifo request from 12345-0@lo, seq: 847 00000100:00100000:2.0:1776178164.577622:0:6256:0:(service.c:2324:ptlrpc_server_handle_request()) Handling RPC req@00000000e1d2e300 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_001:lustre-MDT0001-lwp-OST0001_UUID+5:3356:x1862456998037248:12345-0@lo:400:kworker/u8:0.0 00000100:00100000:2.0:1776178164.577653:0:6256:0:(service.c:2372:ptlrpc_server_handle_request()) Handled RPC req@00000000e1d2e300 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_001:lustre-MDT0001-lwp-OST0001_UUID+5:3356:x1862456998037248:12345-0@lo:400:kworker/u8:0.0 Request processed in 30us (167us total) trans 0 rc 0/0 00000100:00100000:2.0:1776178164.577660:0:6256:0:(nrs_fifo.c:241:nrs_fifo_req_stop()) NRS stop fifo request from 12345-0@lo, seq: 847 00000100:00100000:2.0:1776178164.577670:0:6256:0:(nrs_fifo.c:179:nrs_fifo_req_get()) NRS start fifo request from 12345-0@lo, seq: 848 00000100:00100000:3.0:1776178164.577670:0:3356:0:(client.c:2215:ptlrpc_check_set()) Completed RPC req@00000000165897bf pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_03:lustre-MDT0001-lwp-OST0001_UUID:3356:1862456998037248:0@lo:400:kworker/u8:0.0 00000100:00100000:2.0:1776178164.577674:0:6256:0:(service.c:2324:ptlrpc_server_handle_request()) Handling RPC req@00000000e1965319 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_001:lustre-MDT0000-lwp-MDT0001_UUID+5:3356:x1862456998037504:12345-0@lo:400:kworker/u8:0.0 00000100:00100000:2.0:1776178164.577709:0:6256:0:(service.c:2372:ptlrpc_server_handle_request()) Handled RPC req@00000000e1965319 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_001:lustre-MDT0000-lwp-MDT0001_UUID+5:3356:x1862456998037504:12345-0@lo:400:kworker/u8:0.0 Request processed in 34us (195us total) trans 0 rc 0/0 00000100:00100000:2.0:1776178164.577715:0:6256:0:(nrs_fifo.c:241:nrs_fifo_req_stop()) NRS stop fifo request from 12345-0@lo, seq: 848 00000100:00100000:2.0:1776178164.577726:0:6256:0:(nrs_fifo.c:179:nrs_fifo_req_get()) NRS start fifo request from 12345-0@lo, seq: 849 00000100:00100000:2.0:1776178164.577729:0:6256:0:(service.c:2324:ptlrpc_server_handle_request()) Handling RPC req@0000000025f8393e pname:cluuid+ref:pid:xid:nid:opc:job mdt00_001:lustre-MDT0000-lwp-MDT0000_UUID+5:3356:x1862456998037760:12345-0@lo:400:kworker/u8:0.0 00000100:00100000:2.0:1776178164.577790:0:6256:0:(service.c:2372:ptlrpc_server_handle_request()) Handled RPC req@0000000025f8393e pname:cluuid+ref:pid:xid:nid:opc:job mdt00_001:lustre-MDT0000-lwp-MDT0000_UUID+5:3356:x1862456998037760:12345-0@lo:400:kworker/u8:0.0 Request processed in 59us (253us total) trans 0 rc 0/0 00000100:00100000:2.0:1776178164.577795:0:6256:0:(nrs_fifo.c:241:nrs_fifo_req_stop()) NRS stop fifo request from 12345-0@lo, seq: 849 00000100:00100000:3.0:1776178164.578353:0:3356:0:(client.c:2215:ptlrpc_check_set()) Completed RPC req@000000002cf093e4 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_03:lustre-MDT0000-lwp-MDT0001_UUID:3356:1862456998037504:0@lo:400:kworker/u8:0.0 00000100:00100000:3.0:1776178164.578363:0:3356:0:(client.c:2215:ptlrpc_check_set()) Completed RPC req@00000000a92aab25 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_03:lustre-MDT0000-lwp-MDT0000_UUID:3356:1862456998037760:0@lo:400:kworker/u8:0.0 00000100:00100000:1.0:1776178164.578369:0:20137:0:(service.c:2152:ptlrpc_server_handle_req_in()) got req x1862456998036992 00000100:00100000:1.0:1776178164.578409:0:20137:0:(nrs_fifo.c:179:nrs_fifo_req_get()) NRS start fifo request from 12345-0@lo, seq: 140 00000100:00100000:1.0:1776178164.578413:0:20137:0:(service.c:2324:ptlrpc_server_handle_request()) Handling RPC req@000000005db97027 pname:cluuid+ref:pid:xid:nid:opc:job ll_mgs_0000:ac61f839-0e8e-4cac-9c59-08bcfbbad330+23:3356:x1862456998036992:12345-0@lo:400:kworker/u8:0.0 00000100:00100000:1.0:1776178164.578484:0:20137:0:(service.c:2372:ptlrpc_server_handle_request()) Handled RPC req@000000005db97027 pname:cluuid+ref:pid:xid:nid:opc:job ll_mgs_0000:ac61f839-0e8e-4cac-9c59-08bcfbbad330+23:3356:x1862456998036992:12345-0@lo:400:kworker/u8:0.0 Request processed in 70us (1040us total) trans 0 rc 0/0 00000100:00100000:1.0:1776178164.578497:0:20137:0:(nrs_fifo.c:241:nrs_fifo_req_stop()) NRS stop fifo request from 12345-0@lo, seq: 140 00000100:00100000:1.0:1776178164.578743:0:3353:0:(client.c:1733:ptlrpc_send_new_req()) Sending RPC req@0000000065dbeee0 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_00:lustre-MDT0000-mdtlov_UUID:3353:1862456998037888:0@lo:13:osp-pre-0-0.0 00000100:00100000:0.0:1776178164.578779:0:3354:0:(client.c:1733:ptlrpc_send_new_req()) Sending RPC req@00000000fecf727a pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_01:lustre-MDT0001-mdtlov_UUID:3354:1862456998037952:0@lo:41:osp-pre-0-1.0 00000100:00100000:1.0:1776178164.578783:0:3353:0:(events.c:361:request_in_callback()) peer: 12345-0@lo (source: 12345-0@lo) 00000100:00100000:0.0:1776178164.578807:0:3354:0:(events.c:361:request_in_callback()) peer: 12345-0@lo (source: 12345-0@lo) 00000100:00100000:1.0:1776178164.578814:0:8232:0:(service.c:2152:ptlrpc_server_handle_req_in()) got req x1862456998037888 00000100:00100000:0.0:1776178164.578832:0:15213:0:(service.c:2152:ptlrpc_server_handle_req_in()) got req x1862456998037952 00000100:00100000:1.0:1776178164.578836:0:8232:0:(nrs_fifo.c:179:nrs_fifo_req_get()) NRS start fifo request from 12345-0@lo, seq: 447 00000100:00100000:1.0:1776178164.578839:0:8232:0:(service.c:2324:ptlrpc_server_handle_request()) Handling RPC req@00000000f433e404 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_create00:lustre-MDT0000-mdtlov_UUID+5:3353:x1862456998037888:12345-0@lo:13:osp-pre-0-0.0 00000100:00100000:0.0:1776178164.578851:0:15213:0:(nrs_fifo.c:179:nrs_fifo_req_get()) NRS start fifo request from 12345-0@lo, seq: 668 00000100:00100000:0.0:1776178164.578855:0:15213:0:(service.c:2324:ptlrpc_server_handle_request()) Handling RPC req@000000008259bbb1 pname:cluuid+ref:pid:xid:nid:opc:job mdt_out00_003:lustre-MDT0001-mdtlov_UUID+6:3354:x1862456998037952:12345-0@lo:41:osp-pre-0-1.0 00002000:00000024:1.0:1776178164.578858:0:8232:0:(ofd_obd.c:740:ofd_statfs()) blocks cached 315392 granted 484883776 pending 0 free 3923689472 avail 3692167168 00000004:00000024:0.0:1776178164.578870:0:15213:0:(mdt_handler.c:548:mdt_statfs()) blocks cached 0 granted 2097152 pending 0 free 1446121472 avail 1316659200 00000100:00100000:1.0:1776178164.578887:0:8232:0:(service.c:2372:ptlrpc_server_handle_request()) Handled RPC req@00000000f433e404 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_create00:lustre-MDT0000-mdtlov_UUID+5:3353:x1862456998037888:12345-0@lo:13:osp-pre-0-0.0 Request processed in 47us (104us total) trans 0 rc 0/0 00000100:00100000:1.0:1776178164.578893:0:8232:0:(nrs_fifo.c:241:nrs_fifo_req_stop()) NRS stop fifo request from 12345-0@lo, seq: 447 00000100:00100000:0.0:1776178164.578896:0:15213:0:(service.c:2372:ptlrpc_server_handle_request()) Handled RPC req@000000008259bbb1 pname:cluuid+ref:pid:xid:nid:opc:job mdt_out00_003:lustre-MDT0001-mdtlov_UUID+6:3354:x1862456998037952:12345-0@lo:41:osp-pre-0-1.0 Request processed in 41us (89us total) trans 0 rc 0/0 00000100:00100000:0.0:1776178164.578901:0:15213:0:(nrs_fifo.c:241:nrs_fifo_req_stop()) NRS stop fifo request from 12345-0@lo, seq: 668 00000100:00100000:0.0:1776178164.578923:0:3354:0:(client.c:2215:ptlrpc_check_set()) Completed RPC req@00000000fecf727a pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_01:lustre-MDT0001-mdtlov_UUID:3354:1862456998037952:0@lo:41:osp-pre-0-1.0 00000100:00100000:1.0:1776178164.578923:0:3353:0:(client.c:2215:ptlrpc_check_set()) Completed RPC req@0000000065dbeee0 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_00:lustre-MDT0000-mdtlov_UUID:3353:1862456998037888:0@lo:13:osp-pre-0-0.0 00000100:00100000:3.0:1776178164.578992:0:3356:0:(client.c:2215:ptlrpc_check_set()) Completed RPC req@0000000092f98018 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_03:ac61f839-0e8e-4cac-9c59-08bcfbbad330:3356:1862456998036992:0@lo:400:kworker/u8:0.0 00000100:00100000:0.0:1776178164.579377:0:3356:0:(ptlrpcd.c:415:ptlrpcd_check()) transfer 1 async RPCs [2->3] 00000100:00100000:0.0:1776178164.579395:0:3356:0:(client.c:1733:ptlrpc_send_new_req()) Sending RPC req@000000006c748d80 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_03:lustre-MDT0000-mdtlov_UUID:3356:1862456998038016:0@lo:13:osp-pre-1-0.0 00000100:00100000:0.0:1776178164.579437:0:3356:0:(events.c:361:request_in_callback()) peer: 12345-0@lo (source: 12345-0@lo) 00000100:00100000:1.0:1776178164.580136:0:9554:0:(service.c:2152:ptlrpc_server_handle_req_in()) got req x1862456998038016 00000100:00100000:1.0:1776178164.580205:0:9554:0:(nrs_fifo.c:179:nrs_fifo_req_get()) NRS start fifo request from 12345-0@lo, seq: 448 00000100:00100000:1.0:1776178164.580210:0:9554:0:(service.c:2324:ptlrpc_server_handle_request()) Handling RPC req@00000000fecf727a pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_create00:lustre-MDT0000-mdtlov_UUID+5:3356:x1862456998038016:12345-0@lo:13:osp-pre-1-0.0 00002000:00000024:1.0:1776178164.580235:0:9554:0:(ofd_obd.c:740:ofd_statfs()) blocks cached 229376 granted 491746368 pending 0 free 3923689472 avail 3692167168 00000100:00100000:1.0:1776178164.580289:0:9554:0:(service.c:2372:ptlrpc_server_handle_request()) Handled RPC req@00000000fecf727a pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_create00:lustre-MDT0000-mdtlov_UUID+5:3356:x1862456998038016:12345-0@lo:13:osp-pre-1-0.0 Request processed in 79us (854us total) trans 0 rc 0/0 00000100:00100000:1.0:1776178164.580296:0:9554:0:(nrs_fifo.c:241:nrs_fifo_req_stop()) NRS stop fifo request from 12345-0@lo, seq: 448 00000100:00100000:2.0:1776178164.580557:0:3356:0:(client.c:2215:ptlrpc_check_set()) Completed RPC req@000000006c748d80 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_03:lustre-MDT0000-mdtlov_UUID:3356:1862456998038016:0@lo:13:osp-pre-1-0.0 00010000:00010000:2.0:1776178165.152417:0:9224:0:(ldlm_lockd.c:2792:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000020:00080000:3.0:1776178165.604674:0:18097:0:(update_trans.c:1350:distribute_txn_batchid_cb()) lustre-MDT0001: 12884901896 batchid updated 00000020:00080000:1.0:1776178165.604752:0:18116:0:(update_trans.c:1589:distribute_txn_commit_thread()) lustre-MDT0001: batchid: 0 committed batchid 12884901896 00000040:00100000:1.0:1776178165.604767:0:18116:0:(llog.c:235:llog_cancel_arr_rec()) Canceling 1 records, first 8 in log [0x200000bd1:0x1:0x0] 00000004:00080000:1.0:1776178165.604818:0:18116:0:(osp_trans.c:1598:osp_trans_stop()) lustre-MDT0000-osp-MDT0001: add oth 0000000007a274bd with version 21 00000020:00080000:1.0:1776178165.604833:0:18116:0:(update_trans.c:1312:distribute_txn_cancel_records()) lustre-MDT0000-osp-MDT0001: batchid 12884901896 cancel update log [0x200000bd1:0x1:0x0].8: rc = 0 00000040:00100000:1.0:1776178165.604840:0:18116:0:(llog.c:235:llog_cancel_arr_rec()) Canceling 1 records, first 9 in log [0x240001b70:0x1:0x0] 00000020:00080000:1.0:1776178165.604908:0:18116:0:(update_trans.c:1312:distribute_txn_cancel_records()) lustre-MDT0001-osd: batchid 12884901896 cancel update log [0x240001b70:0x1:0x0].9: rc = 0 00000020:00080000:1.0:1776178165.604920:0:18116:0:(update_trans.c:1589:distribute_txn_commit_thread()) lustre-MDT0001: batchid: 0 committed batchid 12884901896 00000004:00080000:2.0:1776178165.605188:0:18102:0:(osp_trans.c:1343:osp_get_next_request()) ou 000000009121b27e version 21 rpc_version 21 00000100:00100000:2.0:1776178165.605241:0:18102:0:(client.c:1733:ptlrpc_send_new_req()) Sending RPC req@000000006c748d80 pname:cluuid:pid:xid:nid:opc:job osp_up0-1:lustre-MDT0001-mdtlov_UUID:18102:1862456998038080:0@lo:1000:osp_up0-1.0 00000100:00100000:2.0:1776178165.605307:0:18102:0:(events.c:361:request_in_callback()) peer: 12345-0@lo (source: 12345-0@lo) 00000100:00100000:2.0:1776178165.606364:0:18102:0:(client.c:2500:ptlrpc_set_wait()) set 000000002c805ee7 going to sleep for 42 seconds 00000100:00100000:0.0:1776178165.606669:0:15213:0:(service.c:2152:ptlrpc_server_handle_req_in()) got req x1862456998038080 00000100:00100000:0.0:1776178165.606724:0:15213:0:(nrs_fifo.c:179:nrs_fifo_req_get()) NRS start fifo request from 12345-0@lo, seq: 669 00000100:00100000:0.0:1776178165.606731:0:15213:0:(service.c:2324:ptlrpc_server_handle_request()) Handling RPC req@000000002dd26ce5 pname:cluuid+ref:pid:xid:nid:opc:job mdt_out00_003:lustre-MDT0001-mdtlov_UUID+6:18102:x1862456998038080:12345-0@lo:1000:osp_up0-1.0 00000100:00100000:0.0:1776178165.606929:0:15213:0:(service.c:2372:ptlrpc_server_handle_request()) Handled RPC req@000000002dd26ce5 pname:cluuid+ref:pid:xid:nid:opc:job mdt_out00_003:lustre-MDT0001-mdtlov_UUID+7:18102:x1862456998038080:12345-0@lo:1000:osp_up0-1.0 Request processed in 197us (1622us total) trans 12884901945 rc 0/0 00000100:00100000:0.0:1776178165.606937:0:15213:0:(nrs_fifo.c:241:nrs_fifo_req_stop()) NRS stop fifo request from 12345-0@lo, seq: 669 00000100:00100000:2.0:1776178165.608338:0:18102:0:(client.c:2215:ptlrpc_check_set()) Completed RPC req@000000006c748d80 pname:cluuid:pid:xid:nid:opc:job osp_up0-1:lustre-MDT0001-mdtlov_UUID:18102:1862456998038080:0@lo:1000:osp_up0-1.0 00000100:00100000:2.0:1776178166.060252:0:3347:0:(events.c:361:request_in_callback()) peer: 12345-192.168.204.3@tcp (source: 12345-192.168.204.3@tcp) 00000100:00100000:2.0:1776178166.060303:0:3347:0:(events.c:361:request_in_callback()) peer: 12345-192.168.204.3@tcp (source: 12345-192.168.204.3@tcp) 00000100:00100000:1.0:1776178166.060396:0:20140:0:(service.c:2152:ptlrpc_server_handle_req_in()) got req x1862456979095616 00000100:00100000:1.0:1776178166.060462:0:20140:0:(nrs_fifo.c:179:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.204.3@tcp, seq: 141 00000100:00100000:1.0:1776178166.060469:0:20140:0:(service.c:2324:ptlrpc_server_handle_request()) Handling RPC req@0000000017512ce0 pname:cluuid+ref:pid:xid:nid:opc:job ll_mgs_0002:5b5c66b1-1a2f-4651-8548-0c75b9a38a94+9:2271:x1862456979095616:12345-192.168.204.3@tcp:400:kworker/u8:2.0 00000100:00100000:1.0:1776178166.060532:0:20140:0:(service.c:2372:ptlrpc_server_handle_request()) Handled RPC req@0000000017512ce0 pname:cluuid+ref:pid:xid:nid:opc:job ll_mgs_0002:5b5c66b1-1a2f-4651-8548-0c75b9a38a94+9:2271:x1862456979095616:12345-192.168.204.3@tcp:400:kworker/u8:2.0 Request processed in 63us (289us total) trans 0 rc 0/0 00000100:00100000:1.0:1776178166.060539:0:20140:0:(nrs_fifo.c:241:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.204.3@tcp, seq: 141 00000100:00100000:2.0:1776178166.060753:0:3347:0:(events.c:361:request_in_callback()) peer: 12345-192.168.204.3@tcp (source: 12345-192.168.204.3@tcp) 00000100:00100000:2.0:1776178166.060778:0:3347:0:(events.c:361:request_in_callback()) peer: 12345-192.168.204.3@tcp (source: 12345-192.168.204.3@tcp) 00000100:00100000:1.0:1776178166.060810:0:14440:0:(service.c:2152:ptlrpc_server_handle_req_in()) got req x1862456979095872 00000100:00100000:1.0:1776178166.060841:0:14440:0:(nrs_fifo.c:179:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.204.3@tcp, seq: 472 00000100:00100000:1.0:1776178166.060845:0:14440:0:(service.c:2324:ptlrpc_server_handle_request()) Handling RPC req@00000000165897bf pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_004:b50963bd-8eba-4432-9a54-8888d00e3fa1+9:2271:x1862456979095872:12345-192.168.204.3@tcp:400:kworker/u8:2.0 00000100:00100000:1.0:1776178166.060879:0:14440:0:(service.c:2372:ptlrpc_server_handle_request()) Handled RPC req@00000000165897bf pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_004:b50963bd-8eba-4432-9a54-8888d00e3fa1+9:2271:x1862456979095872:12345-192.168.204.3@tcp:400:kworker/u8:2.0 Request processed in 33us (127us total) trans 0 rc 0/0 00000100:00100000:1.0:1776178166.060885:0:14440:0:(nrs_fifo.c:241:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.204.3@tcp, seq: 472 00000100:00100000:2.0:1776178166.065696:0:3347:0:(events.c:361:request_in_callback()) peer: 12345-192.168.204.3@tcp (source: 12345-192.168.204.3@tcp) 00000100:00100000:3.0:1776178166.065915:0:8227:0:(service.c:2152:ptlrpc_server_handle_req_in()) got req x1862456979095808 00000100:00100000:3.0:1776178166.065971:0:8227:0:(nrs_fifo.c:179:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.204.3@tcp, seq: 473 00000100:00100000:3.0:1776178166.065977:0:8227:0:(service.c:2324:ptlrpc_server_handle_request()) Handling RPC req@0000000000167c64 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_000:b50963bd-8eba-4432-9a54-8888d00e3fa1+8:2271:x1862456979095808:12345-192.168.204.3@tcp:400:kworker/u8:2.0 00000100:00100000:3.0:1776178166.066039:0:8227:0:(service.c:2372:ptlrpc_server_handle_request()) Handled RPC req@0000000000167c64 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_000:b50963bd-8eba-4432-9a54-8888d00e3fa1+8:2271:x1862456979095808:12345-192.168.204.3@tcp:400:kworker/u8:2.0 Request processed in 62us (352us total) trans 0 rc 0/0 00000100:00100000:3.0:1776178166.066046:0:8227:0:(nrs_fifo.c:241:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.204.3@tcp, seq: 473 00000100:00100000:0.0:1776178166.071584:0:8250:0:(service.c:2152:ptlrpc_server_handle_req_in()) got req x1862456979095744 00000100:00100000:0.0:1776178166.071679:0:8250:0:(service.c:2152:ptlrpc_server_handle_req_in()) got req x1862456979095680 00000100:00100000:0.0:1776178166.071785:0:8250:0:(nrs_fifo.c:179:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.204.3@tcp, seq: 850 00000100:00100000:0.0:1776178166.071793:0:8250:0:(service.c:2324:ptlrpc_server_handle_request()) Handling RPC req@0000000092f98018 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_003:b50963bd-8eba-4432-9a54-8888d00e3fa1+6:2271:x1862456979095744:12345-192.168.204.3@tcp:400:kworker/u8:2.0 00000100:00100000:0.0:1776178166.071892:0:8250:0:(service.c:2372:ptlrpc_server_handle_request()) Handled RPC req@0000000092f98018 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_003:b50963bd-8eba-4432-9a54-8888d00e3fa1+6:2271:x1862456979095744:12345-192.168.204.3@tcp:400:kworker/u8:2.0 Request processed in 99us (11589us total) trans 0 rc 0/0 00000100:00100000:0.0:1776178166.071900:0:8250:0:(nrs_fifo.c:241:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.204.3@tcp, seq: 850 00000100:00100000:0.0:1776178166.071919:0:8250:0:(nrs_fifo.c:179:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.204.3@tcp, seq: 851 00000100:00100000:0.0:1776178166.071923:0:8250:0:(service.c:2324:ptlrpc_server_handle_request()) Handling RPC req@00000000a92aab25 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_003:b50963bd-8eba-4432-9a54-8888d00e3fa1+7:2271:x1862456979095680:12345-192.168.204.3@tcp:400:kworker/u8:2.0 00000100:00100000:0.0:1776178166.071947:0:8250:0:(service.c:2372:ptlrpc_server_handle_request()) Handled RPC req@00000000a92aab25 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_003:b50963bd-8eba-4432-9a54-8888d00e3fa1+7:2271:x1862456979095680:12345-192.168.204.3@tcp:400:kworker/u8:2.0 Request processed in 23us (11170us total) trans 0 rc 0/0 00000100:00100000:0.0:1776178166.071954:0:8250:0:(nrs_fifo.c:241:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.204.3@tcp, seq: 851 00000100:00100000:2.0:1776178166.116280:0:3356:0:(client.c:1733:ptlrpc_send_new_req()) Sending RPC req@000000007794b3ac pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_03:lustre-MDT0001-mdtlov_UUID:3356:1862456998038144:0@lo:13:osp-pre-1-1.0 00000100:00100000:2.0:1776178166.116381:0:3356:0:(events.c:361:request_in_callback()) peer: 12345-0@lo (source: 12345-0@lo) 00000100:00100000:0.0:1776178166.117253:0:8232:0:(service.c:2152:ptlrpc_server_handle_req_in()) got req x1862456998038144 00000100:00100000:0.0:1776178166.117320:0:8232:0:(nrs_fifo.c:179:nrs_fifo_req_get()) NRS start fifo request from 12345-0@lo, seq: 449 00000100:00100000:0.0:1776178166.117327:0:8232:0:(service.c:2324:ptlrpc_server_handle_request()) Handling RPC req@000000007886b426 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_create00:lustre-MDT0001-mdtlov_UUID+5:3356:x1862456998038144:12345-0@lo:13:osp-pre-1-1.0 00002000:00000024:0.0:1776178166.117362:0:8232:0:(ofd_obd.c:740:ofd_statfs()) blocks cached 229376 granted 491746368 pending 0 free 3923689472 avail 3692167168 00000100:00100000:0.0:1776178166.120471:0:8232:0:(service.c:2372:ptlrpc_server_handle_request()) Handled RPC req@000000007886b426 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_create00:lustre-MDT0001-mdtlov_UUID+5:3356:x1862456998038144:12345-0@lo:13:osp-pre-1-1.0 Request processed in 3135us (4083us total) trans 0 rc 0/0 00000100:00100000:0.0:1776178166.120487:0:8232:0:(nrs_fifo.c:241:nrs_fifo_req_stop()) NRS stop fifo request from 12345-0@lo, seq: 449 00000100:00100000:2.0:1776178166.124359:0:3356:0:(client.c:2215:ptlrpc_check_set()) Completed RPC req@000000007794b3ac pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_03:lustre-MDT0001-mdtlov_UUID:3356:1862456998038144:0@lo:13:osp-pre-1-1.0 00010000:00010000:1.0:1776178166.176585:0:6245:0:(ldlm_lockd.c:2792:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:1.0:1776178166.624442:0:3353:0:(client.c:1733:ptlrpc_send_new_req()) Sending RPC req@000000004ee84694 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_00:lustre-MDT0001-mdtlov_UUID:3353:1862456998038208:0@lo:13:osp-pre-0-1.0 00000100:00100000:1.0:1776178166.624515:0:3353:0:(events.c:361:request_in_callback()) peer: 12345-0@lo (source: 12345-0@lo) 00000100:00100000:3.0:1776178166.624777:0:8232:0:(service.c:2152:ptlrpc_server_handle_req_in()) got req x1862456998038208 00000100:00100000:3.0:1776178166.624830:0:8232:0:(nrs_fifo.c:179:nrs_fifo_req_get()) NRS start fifo request from 12345-0@lo, seq: 450 00000100:00100000:3.0:1776178166.624836:0:8232:0:(service.c:2324:ptlrpc_server_handle_request()) Handling RPC req@0000000065dbeee0 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_create00:lustre-MDT0001-mdtlov_UUID+5:3353:x1862456998038208:12345-0@lo:13:osp-pre-0-1.0 00002000:00000024:3.0:1776178166.624865:0:8232:0:(ofd_obd.c:740:ofd_statfs()) blocks cached 315392 granted 484883776 pending 0 free 3923689472 avail 3692167168 00000100:00100000:3.0:1776178166.624906:0:8232:0:(service.c:2372:ptlrpc_server_handle_request()) Handled RPC req@0000000065dbeee0 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_create00:lustre-MDT0001-mdtlov_UUID+5:3353:x1862456998038208:12345-0@lo:13:osp-pre-0-1.0 Request processed in 69us (392us total) trans 0 rc 0/0 00000100:00100000:3.0:1776178166.624914:0:8232:0:(nrs_fifo.c:241:nrs_fifo_req_stop()) NRS stop fifo request from 12345-0@lo, seq: 450 00000100:00100000:1.0:1776178166.625006:0:3353:0:(client.c:2215:ptlrpc_check_set()) Completed RPC req@000000004ee84694 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_00:lustre-MDT0001-mdtlov_UUID:3353:1862456998038208:0@lo:13:osp-pre-0-1.0 00010000:00010000:2.0:1776178167.200330:0:6244:0:(ldlm_lockd.c:2792:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00010000:00010000:2.0:1776178168.225435:0:9225:0:(ldlm_lockd.c:2792:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00010000:00010000:2.0:1776178169.248469:0:9224:0:(ldlm_lockd.c:2792:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:2.0:1776178169.698016:0:3354:0:(client.c:1733:ptlrpc_send_new_req()) Sending RPC req@00000000889a4355 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_01:lustre-MDT0000-mdtlov_UUID:3354:1862456998038272:0@lo:13:osp-pre-0-0.0 00000100:00100000:2.0:1776178169.698102:0:3354:0:(events.c:361:request_in_callback()) peer: 12345-0@lo (source: 12345-0@lo) 00000100:00100000:2.0:1776178169.698142:0:3356:0:(ptlrpcd.c:415:ptlrpcd_check()) transfer 1 async RPCs [2->3] 00000100:00100000:2.0:1776178169.698148:0:3356:0:(client.c:1733:ptlrpc_send_new_req()) Sending RPC req@00000000ebb70e84 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_03:lustre-MDT0001-mdtlov_UUID:3356:1862456998038336:0@lo:41:osp-pre-0-1.0 00000100:00100000:2.0:1776178169.698171:0:3356:0:(events.c:361:request_in_callback()) peer: 12345-0@lo (source: 12345-0@lo) 00000100:00100000:3.0:1776178169.698366:0:8232:0:(service.c:2152:ptlrpc_server_handle_req_in()) got req x1862456998038272 00000100:00100000:3.0:1776178169.698406:0:8232:0:(nrs_fifo.c:179:nrs_fifo_req_get()) NRS start fifo request from 12345-0@lo, seq: 451 00000100:00100000:3.0:1776178169.698412:0:8232:0:(service.c:2324:ptlrpc_server_handle_request()) Handling RPC req@000000001c9b18be pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_create00:lustre-MDT0000-mdtlov_UUID+5:3354:x1862456998038272:12345-0@lo:13:osp-pre-0-0.0 00002000:00000024:3.0:1776178169.698439:0:8232:0:(ofd_obd.c:740:ofd_statfs()) blocks cached 315392 granted 484883776 pending 0 free 3923689472 avail 3692167168 00000100:00100000:3.0:1776178169.698475:0:8232:0:(service.c:2372:ptlrpc_server_handle_request()) Handled RPC req@000000001c9b18be pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_create00:lustre-MDT0000-mdtlov_UUID+5:3354:x1862456998038272:12345-0@lo:13:osp-pre-0-0.0 Request processed in 63us (376us total) trans 0 rc 0/0 00000100:00100000:3.0:1776178169.698482:0:8232:0:(nrs_fifo.c:241:nrs_fifo_req_stop()) NRS stop fifo request from 12345-0@lo, seq: 451 00000100:00100000:3.0:1776178169.698699:0:3353:0:(client.c:1733:ptlrpc_send_new_req()) Sending RPC req@00000000b26c52d3 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_00:lustre-MDT0000-lwp-OST0000_UUID:3353:1862456998038464:0@lo:400:kworker/u8:2.0 00000100:00100000:3.0:1776178169.698723:0:3353:0:(events.c:361:request_in_callback()) peer: 12345-0@lo (source: 12345-0@lo) 00000100:00100000:3.0:1776178169.698735:0:3353:0:(client.c:1733:ptlrpc_send_new_req()) Sending RPC req@000000006ea63760 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_00:lustre-MDT0001-mdtlov_UUID:3353:1862456998038720:0@lo:400:kworker/u8:2.0 00000100:00100000:3.0:1776178169.698751:0:3353:0:(events.c:361:request_in_callback()) peer: 12345-0@lo (source: 12345-0@lo) 00000100:00100000:3.0:1776178169.698759:0:3353:0:(client.c:1733:ptlrpc_send_new_req()) Sending RPC req@00000000b7afa2a2 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_00:lustre-MDT0000-mdtlov_UUID:3353:1862456998038976:0@lo:400:kworker/u8:2.0 00000100:00100000:3.0:1776178169.698773:0:3353:0:(events.c:361:request_in_callback()) peer: 12345-0@lo (source: 12345-0@lo) 00000100:00100000:3.0:1776178169.698786:0:3353:0:(ptlrpcd.c:415:ptlrpcd_check()) transfer 3 async RPCs [1->0] 00000100:00100000:3.0:1776178169.698790:0:3353:0:(client.c:1733:ptlrpc_send_new_req()) Sending RPC req@00000000660d337f pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_00:lustre-MDT0001-lwp-OST0000_UUID:3353:1862456998038528:0@lo:400:kworker/u8:2.0 00000100:00100000:3.0:1776178169.698804:0:3353:0:(events.c:361:request_in_callback()) peer: 12345-0@lo (source: 12345-0@lo) 00000100:00100000:3.0:1776178169.698812:0:3353:0:(client.c:1733:ptlrpc_send_new_req()) Sending RPC req@00000000f52e98d8 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_00:lustre-MDT0001-mdtlov_UUID:3353:1862456998038784:0@lo:400:kworker/u8:2.0 00000100:00100000:3.0:1776178169.699445:0:3353:0:(events.c:361:request_in_callback()) peer: 12345-0@lo (source: 12345-0@lo) 00000100:00100000:3.0:1776178169.699458:0:3353:0:(client.c:1733:ptlrpc_send_new_req()) Sending RPC req@000000007608d8e9 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_00:lustre-MDT0000-lwp-MDT0000_UUID:3353:1862456998039040:0@lo:400:kworker/u8:2.0 00000100:00100000:3.0:1776178169.699476:0:3353:0:(events.c:361:request_in_callback()) peer: 12345-0@lo (source: 12345-0@lo) 00000100:00100000:3.0:1776178169.699496:0:8227:0:(service.c:2152:ptlrpc_server_handle_req_in()) got req x1862456998038720 00000100:00100000:3.0:1776178169.699516:0:8227:0:(service.c:2152:ptlrpc_server_handle_req_in()) got req x1862456998038976 00000100:00100000:3.0:1776178169.699526:0:8227:0:(service.c:2152:ptlrpc_server_handle_req_in()) got req x1862456998038784 00000100:00100000:3.0:1776178169.699538:0:8227:0:(nrs_fifo.c:179:nrs_fifo_req_get()) NRS start fifo request from 12345-0@lo, seq: 474 00000100:00100000:3.0:1776178169.699542:0:8227:0:(service.c:2324:ptlrpc_server_handle_request()) Handling RPC req@000000008a10a1f2 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_000:lustre-MDT0001-mdtlov_UUID+5:3353:x1862456998038720:12345-0@lo:400:kworker/u8:2.0 00000100:00100000:3.0:1776178169.699570:0:8227:0:(service.c:2372:ptlrpc_server_handle_request()) Handled RPC req@000000008a10a1f2 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_000:lustre-MDT0001-mdtlov_UUID+5:3353:x1862456998038720:12345-0@lo:400:kworker/u8:2.0 Request processed in 28us (819us total) trans 0 rc 0/0 00000100:00100000:3.0:1776178169.699576:0:8227:0:(nrs_fifo.c:241:nrs_fifo_req_stop()) NRS stop fifo request from 12345-0@lo, seq: 474 00000100:00100000:3.0:1776178169.699585:0:8227:0:(nrs_fifo.c:179:nrs_fifo_req_get()) NRS start fifo request from 12345-0@lo, seq: 475 00000100:00100000:3.0:1776178169.699588:0:8227:0:(service.c:2324:ptlrpc_server_handle_request()) Handling RPC req@0000000037e96ea5 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_000:lustre-MDT0000-mdtlov_UUID+5:3353:x1862456998038976:12345-0@lo:400:kworker/u8:2.0 00000100:00100000:0.0:1776178169.699759:0:8228:0:(nrs_fifo.c:179:nrs_fifo_req_get()) NRS start fifo request from 12345-0@lo, seq: 476 00000100:00100000:0.0:1776178169.699764:0:8228:0:(service.c:2324:ptlrpc_server_handle_request()) Handling RPC req@000000006c64aaae pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_001:lustre-MDT0001-mdtlov_UUID+5:3353:x1862456998038784:12345-0@lo:400:kworker/u8:2.0 00000100:00100000:0.0:1776178169.699792:0:8228:0:(service.c:2372:ptlrpc_server_handle_request()) Handled RPC req@000000006c64aaae pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_001:lustre-MDT0001-mdtlov_UUID+5:3353:x1862456998038784:12345-0@lo:400:kworker/u8:2.0 Request processed in 28us (350us total) trans 0 rc 0/0 00000100:00100000:0.0:1776178169.699797:0:8228:0:(nrs_fifo.c:241:nrs_fifo_req_stop()) NRS stop fifo request from 12345-0@lo, seq: 476 00000100:00100000:0.0:1776178169.699813:0:8250:0:(service.c:2152:ptlrpc_server_handle_req_in()) got req x1862456998038464 00000100:00100000:0.0:1776178169.699846:0:8250:0:(service.c:2152:ptlrpc_server_handle_req_in()) got req x1862456998038528 00000100:00100000:0.0:1776178169.699857:0:8250:0:(service.c:2152:ptlrpc_server_handle_req_in()) got req x1862456998039040 00000100:00100000:0.0:1776178169.699867:0:8250:0:(nrs_fifo.c:179:nrs_fifo_req_get()) NRS start fifo request from 12345-0@lo, seq: 852 00000100:00100000:0.0:1776178169.699870:0:8250:0:(service.c:2324:ptlrpc_server_handle_request()) Handling RPC req@000000007d4744cc pname:cluuid+ref:pid:xid:nid:opc:job mdt00_003:lustre-MDT0000-lwp-OST0000_UUID+5:3353:x1862456998038464:12345-0@lo:400:kworker/u8:2.0 00000100:00100000:0.0:1776178169.699892:0:8250:0:(service.c:2372:ptlrpc_server_handle_request()) Handled RPC req@000000007d4744cc pname:cluuid+ref:pid:xid:nid:opc:job mdt00_003:lustre-MDT0000-lwp-OST0000_UUID+5:3353:x1862456998038464:12345-0@lo:400:kworker/u8:2.0 Request processed in 21us (1169us total) trans 0 rc 0/0 00000100:00100000:0.0:1776178169.699897:0:8250:0:(nrs_fifo.c:241:nrs_fifo_req_stop()) NRS stop fifo request from 12345-0@lo, seq: 852 00000100:00100000:0.0:1776178169.699907:0:8250:0:(nrs_fifo.c:179:nrs_fifo_req_get()) NRS start fifo request from 12345-0@lo, seq: 853 00000100:00100000:0.0:1776178169.699909:0:8250:0:(service.c:2324:ptlrpc_server_handle_request()) Handling RPC req@00000000e53deeb6 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_003:lustre-MDT0001-lwp-OST0000_UUID+5:3353:x1862456998038528:12345-0@lo:400:kworker/u8:2.0 00000100:00100000:0.0:1776178169.699925:0:8250:0:(service.c:2372:ptlrpc_server_handle_request()) Handled RPC req@00000000e53deeb6 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_003:lustre-MDT0001-lwp-OST0000_UUID+5:3353:x1862456998038528:12345-0@lo:400:kworker/u8:2.0 Request processed in 15us (1122us total) trans 0 rc 0/0 00000100:00100000:0.0:1776178169.699930:0:8250:0:(nrs_fifo.c:241:nrs_fifo_req_stop()) NRS stop fifo request from 12345-0@lo, seq: 853 00000100:00100000:0.0:1776178169.699937:0:8250:0:(nrs_fifo.c:179:nrs_fifo_req_get()) NRS start fifo request from 12345-0@lo, seq: 854 00000100:00100000:0.0:1776178169.699940:0:8250:0:(service.c:2324:ptlrpc_server_handle_request()) Handling RPC req@00000000f6a6e923 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_003:lustre-MDT0000-lwp-MDT0000_UUID+5:3353:x1862456998039040:12345-0@lo:400:kworker/u8:2.0 00000100:00100000:0.0:1776178169.699954:0:8250:0:(service.c:2372:ptlrpc_server_handle_request()) Handled RPC req@00000000f6a6e923 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_003:lustre-MDT0000-lwp-MDT0000_UUID+5:3353:x1862456998039040:12345-0@lo:400:kworker/u8:2.0 Request processed in 14us (478us total) trans 0 rc 0/0 00000100:00100000:0.0:1776178169.699959:0:8250:0:(nrs_fifo.c:241:nrs_fifo_req_stop()) NRS stop fifo request from 12345-0@lo, seq: 854 00000100:00100000:0.0:1776178169.700131:0:3353:0:(client.c:2215:ptlrpc_check_set()) Completed RPC req@00000000660d337f pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_00:lustre-MDT0001-lwp-OST0000_UUID:3353:1862456998038528:0@lo:400:kworker/u8:2.0 00000100:00100000:0.0:1776178169.700139:0:3353:0:(client.c:2215:ptlrpc_check_set()) Completed RPC req@00000000f52e98d8 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_00:lustre-MDT0001-mdtlov_UUID:3353:1862456998038784:0@lo:400:kworker/u8:2.0 00000100:00100000:0.0:1776178169.700145:0:3353:0:(client.c:2215:ptlrpc_check_set()) Completed RPC req@000000007608d8e9 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_00:lustre-MDT0000-lwp-MDT0000_UUID:3353:1862456998039040:0@lo:400:kworker/u8:2.0 00000100:00100000:0.0:1776178169.700153:0:3353:0:(client.c:2215:ptlrpc_check_set()) Completed RPC req@00000000b26c52d3 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_00:lustre-MDT0000-lwp-OST0000_UUID:3353:1862456998038464:0@lo:400:kworker/u8:2.0 00000100:00100000:0.0:1776178169.700159:0:3353:0:(client.c:2215:ptlrpc_check_set()) Completed RPC req@000000006ea63760 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_00:lustre-MDT0001-mdtlov_UUID:3353:1862456998038720:0@lo:400:kworker/u8:2.0 00000100:00100000:0.0:1776178169.700166:0:3353:0:(client.c:2215:ptlrpc_check_set()) Completed RPC req@00000000b7afa2a2 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_00:lustre-MDT0000-mdtlov_UUID:3353:1862456998038976:0@lo:400:kworker/u8:2.0 00000100:00100000:2.0:1776178169.700403:0:3356:0:(client.c:1733:ptlrpc_send_new_req()) Sending RPC req@0000000008fe8f11 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_03:ac61f839-0e8e-4cac-9c59-08bcfbbad330:3356:1862456998038400:0@lo:400:kworker/u8:2.0 00000100:00100000:2.0:1776178169.700432:0:3356:0:(events.c:361:request_in_callback()) peer: 12345-0@lo (source: 12345-0@lo) 00000100:00100000:2.0:1776178169.700456:0:3356:0:(client.c:1733:ptlrpc_send_new_req()) Sending RPC req@0000000024b881b8 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_03:lustre-MDT0001-lwp-OST0001_UUID:3356:1862456998038656:0@lo:400:kworker/u8:2.0 00000100:00100000:2.0:1776178169.700473:0:3356:0:(events.c:361:request_in_callback()) peer: 12345-0@lo (source: 12345-0@lo) 00000100:00100000:2.0:1776178169.700482:0:3356:0:(client.c:1733:ptlrpc_send_new_req()) Sending RPC req@000000000c014cb0 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_03:lustre-MDT0000-mdtlov_UUID:3356:1862456998038912:0@lo:400:kworker/u8:2.0 00000100:00100000:2.0:1776178169.700498:0:3356:0:(events.c:361:request_in_callback()) peer: 12345-0@lo (source: 12345-0@lo) 00000100:00100000:2.0:1776178169.700508:0:3356:0:(client.c:1733:ptlrpc_send_new_req()) Sending RPC req@00000000c7e83057 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_03:lustre-MDT0000-mdtlov_UUID:3356:1862456998039168:0@lo:41:osp-pre-1-0.0 00000100:00100000:2.0:1776178169.700523:0:3356:0:(events.c:361:request_in_callback()) peer: 12345-0@lo (source: 12345-0@lo) 00000100:00100000:2.0:1776178169.700536:0:3356:0:(ptlrpcd.c:415:ptlrpcd_check()) transfer 3 async RPCs [2->3] 00000100:00100000:2.0:1776178169.700541:0:3356:0:(client.c:1733:ptlrpc_send_new_req()) Sending RPC req@000000004943ee04 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_03:lustre-MDT0000-lwp-OST0001_UUID:3356:1862456998038592:0@lo:400:kworker/u8:2.0 00000100:00100000:2.0:1776178169.700555:0:3356:0:(events.c:361:request_in_callback()) peer: 12345-0@lo (source: 12345-0@lo) 00000100:00100000:2.0:1776178169.700564:0:3356:0:(client.c:1733:ptlrpc_send_new_req()) Sending RPC req@0000000002c47d13 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_03:lustre-MDT0000-lwp-MDT0001_UUID:3356:1862456998038848:0@lo:400:kworker/u8:2.0 00000100:00100000:2.0:1776178169.700579:0:3356:0:(events.c:361:request_in_callback()) peer: 12345-0@lo (source: 12345-0@lo) 00000100:00100000:2.0:1776178169.700588:0:3356:0:(client.c:1733:ptlrpc_send_new_req()) Sending RPC req@00000000737321c9 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_03:lustre-MDT0000-mdtlov_UUID:3356:1862456998039104:0@lo:13:osp-pre-1-0.0 00000100:00100000:2.0:1776178169.700602:0:3356:0:(events.c:361:request_in_callback()) peer: 12345-0@lo (source: 12345-0@lo) 00000100:00100000:2.0:1776178169.700631:0:15213:0:(service.c:2152:ptlrpc_server_handle_req_in()) got req x1862456998038336 00000100:00100000:2.0:1776178169.700660:0:15213:0:(service.c:2152:ptlrpc_server_handle_req_in()) got req x1862456998038912 00000100:00100000:2.0:1776178169.700668:0:15213:0:(service.c:2152:ptlrpc_server_handle_req_in()) got req x1862456998039168 00000100:00100000:2.0:1776178169.700677:0:15213:0:(nrs_fifo.c:179:nrs_fifo_req_get()) NRS start fifo request from 12345-0@lo, seq: 670 00000100:00100000:2.0:1776178169.700681:0:15213:0:(service.c:2324:ptlrpc_server_handle_request()) Handling RPC req@00000000d1a7070f pname:cluuid+ref:pid:xid:nid:opc:job mdt_out00_003:lustre-MDT0001-mdtlov_UUID+7:3356:x1862456998038336:12345-0@lo:41:osp-pre-0-1.0 00000004:00000024:2.0:1776178169.700701:0:15213:0:(mdt_handler.c:548:mdt_statfs()) blocks cached 0 granted 2097152 pending 0 free 1446121472 avail 1316659200 00000100:00100000:2.0:1776178169.700727:0:15213:0:(service.c:2372:ptlrpc_server_handle_request()) Handled RPC req@00000000d1a7070f pname:cluuid+ref:pid:xid:nid:opc:job mdt_out00_003:lustre-MDT0001-mdtlov_UUID+7:3356:x1862456998038336:12345-0@lo:41:osp-pre-0-1.0 Request processed in 46us (2556us total) trans 0 rc 0/0 00000100:00100000:2.0:1776178169.700732:0:15213:0:(nrs_fifo.c:241:nrs_fifo_req_stop()) NRS stop fifo request from 12345-0@lo, seq: 670 00000100:00100000:2.0:1776178169.700744:0:15213:0:(nrs_fifo.c:179:nrs_fifo_req_get()) NRS start fifo request from 12345-0@lo, seq: 671 00000100:00100000:2.0:1776178169.700746:0:15213:0:(service.c:2324:ptlrpc_server_handle_request()) Handling RPC req@00000000e1865444 pname:cluuid+ref:pid:xid:nid:opc:job mdt_out00_003:lustre-MDT0000-mdtlov_UUID+6:3356:x1862456998038912:12345-0@lo:400:kworker/u8:2.0 00000100:00100000:2.0:1776178169.700764:0:15213:0:(service.c:2372:ptlrpc_server_handle_request()) Handled RPC req@00000000e1865444 pname:cluuid+ref:pid:xid:nid:opc:job mdt_out00_003:lustre-MDT0000-mdtlov_UUID+6:3356:x1862456998038912:12345-0@lo:400:kworker/u8:2.0 Request processed in 17us (266us total) trans 0 rc 0/0 00000100:00100000:2.0:1776178169.700769:0:15213:0:(nrs_fifo.c:241:nrs_fifo_req_stop()) NRS stop fifo request from 12345-0@lo, seq: 671 00000100:00100000:2.0:1776178169.700776:0:15213:0:(nrs_fifo.c:179:nrs_fifo_req_get()) NRS start fifo request from 12345-0@lo, seq: 672 00000100:00100000:2.0:1776178169.700779:0:15213:0:(service.c:2324:ptlrpc_server_handle_request()) Handling RPC req@00000000a343c008 pname:cluuid+ref:pid:xid:nid:opc:job mdt_out00_003:lustre-MDT0000-mdtlov_UUID+5:3356:x1862456998039168:12345-0@lo:41:osp-pre-1-0.0 00000004:00000024:2.0:1776178169.700790:0:15213:0:(mdt_handler.c:548:mdt_statfs()) blocks cached 0 granted 2097152 pending 0 free 1446035456 avail 1316573184 00000100:00100000:2.0:1776178169.700805:0:15213:0:(service.c:2372:ptlrpc_server_handle_request()) Handled RPC req@00000000a343c008 pname:cluuid+ref:pid:xid:nid:opc:job mdt_out00_003:lustre-MDT0000-mdtlov_UUID+5:3356:x1862456998039168:12345-0@lo:41:osp-pre-1-0.0 Request processed in 26us (283us total) trans 0 rc 0/0 00000100:00100000:2.0:1776178169.700810:0:15213:0:(nrs_fifo.c:241:nrs_fifo_req_stop()) NRS stop fifo request from 12345-0@lo, seq: 672 00000100:00100000:2.0:1776178169.700836:0:6256:0:(service.c:2152:ptlrpc_server_handle_req_in()) got req x1862456998038656 00000100:00100000:2.0:1776178169.700852:0:6256:0:(service.c:2152:ptlrpc_server_handle_req_in()) got req x1862456998038592 00000100:00100000:2.0:1776178169.700859:0:6256:0:(service.c:2152:ptlrpc_server_handle_req_in()) got req x1862456998038848 00000100:00100000:2.0:1776178169.700867:0:6256:0:(nrs_fifo.c:179:nrs_fifo_req_get()) NRS start fifo request from 12345-0@lo, seq: 855 00000100:00100000:2.0:1776178169.700870:0:6256:0:(service.c:2324:ptlrpc_server_handle_request()) Handling RPC req@00000000c31dc70e pname:cluuid+ref:pid:xid:nid:opc:job mdt00_001:lustre-MDT0001-lwp-OST0001_UUID+5:3356:x1862456998038656:12345-0@lo:400:kworker/u8:2.0 00000100:00100000:2.0:1776178169.700890:0:6256:0:(service.c:2372:ptlrpc_server_handle_request()) Handled RPC req@00000000c31dc70e pname:cluuid+ref:pid:xid:nid:opc:job mdt00_001:lustre-MDT0001-lwp-OST0001_UUID+5:3356:x1862456998038656:12345-0@lo:400:kworker/u8:2.0 Request processed in 20us (418us total) trans 0 rc 0/0 00000100:00100000:2.0:1776178169.700896:0:6256:0:(nrs_fifo.c:241:nrs_fifo_req_stop()) NRS stop fifo request from 12345-0@lo, seq: 855 00000100:00100000:2.0:1776178169.700903:0:6256:0:(nrs_fifo.c:179:nrs_fifo_req_get()) NRS start fifo request from 12345-0@lo, seq: 856 00000100:00100000:2.0:1776178169.700906:0:6256:0:(service.c:2324:ptlrpc_server_handle_request()) Handling RPC req@0000000045ff6e42 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_001:lustre-MDT0000-lwp-OST0001_UUID+5:3356:x1862456998038592:12345-0@lo:400:kworker/u8:2.0 00000100:00100000:2.0:1776178169.700921:0:6256:0:(service.c:2372:ptlrpc_server_handle_request()) Handled RPC req@0000000045ff6e42 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_001:lustre-MDT0000-lwp-OST0001_UUID+5:3356:x1862456998038592:12345-0@lo:400:kworker/u8:2.0 Request processed in 14us (366us total) trans 0 rc 0/0 00000100:00100000:2.0:1776178169.700926:0:6256:0:(nrs_fifo.c:241:nrs_fifo_req_stop()) NRS stop fifo request from 12345-0@lo, seq: 856 00000100:00100000:2.0:1776178169.700933:0:6256:0:(nrs_fifo.c:179:nrs_fifo_req_get()) NRS start fifo request from 12345-0@lo, seq: 857 00000100:00100000:2.0:1776178169.700936:0:6256:0:(service.c:2324:ptlrpc_server_handle_request()) Handling RPC req@0000000002da67e2 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_001:lustre-MDT0000-lwp-MDT0001_UUID+5:3356:x1862456998038848:12345-0@lo:400:kworker/u8:2.0 00000100:00100000:2.0:1776178169.700950:0:6256:0:(service.c:2372:ptlrpc_server_handle_request()) Handled RPC req@0000000002da67e2 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_001:lustre-MDT0000-lwp-MDT0001_UUID+5:3356:x1862456998038848:12345-0@lo:400:kworker/u8:2.0 Request processed in 14us (372us total) trans 0 rc 0/0 00000100:00100000:2.0:1776178169.700955:0:6256:0:(nrs_fifo.c:241:nrs_fifo_req_stop()) NRS stop fifo request from 12345-0@lo, seq: 857 00000100:00100000:2.0:1776178169.700975:0:9554:0:(service.c:2152:ptlrpc_server_handle_req_in()) got req x1862456998039104 00000100:00100000:2.0:1776178169.700990:0:9554:0:(nrs_fifo.c:179:nrs_fifo_req_get()) NRS start fifo request from 12345-0@lo, seq: 452 00000100:00100000:2.0:1776178169.700993:0:9554:0:(service.c:2324:ptlrpc_server_handle_request()) Handling RPC req@00000000702f2be6 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_create00:lustre-MDT0000-mdtlov_UUID+6:3356:x1862456998039104:12345-0@lo:13:osp-pre-1-0.0 00002000:00000024:2.0:1776178169.701007:0:9554:0:(ofd_obd.c:740:ofd_statfs()) blocks cached 229376 granted 491746368 pending 0 free 3923689472 avail 3692167168 00000100:00100000:2.0:1776178169.701027:0:9554:0:(service.c:2372:ptlrpc_server_handle_request()) Handled RPC req@00000000702f2be6 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_create00:lustre-MDT0000-mdtlov_UUID+6:3356:x1862456998039104:12345-0@lo:13:osp-pre-1-0.0 Request processed in 34us (425us total) trans 0 rc 0/0 00000100:00100000:2.0:1776178169.701032:0:9554:0:(nrs_fifo.c:241:nrs_fifo_req_stop()) NRS stop fifo request from 12345-0@lo, seq: 452 00000100:00100000:2.0:1776178169.701081:0:3354:0:(client.c:2215:ptlrpc_check_set()) Completed RPC req@00000000889a4355 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_01:lustre-MDT0000-mdtlov_UUID:3354:1862456998038272:0@lo:13:osp-pre-0-0.0 00000100:00100000:2.0:1776178169.701101:0:3356:0:(client.c:2215:ptlrpc_check_set()) Completed RPC req@000000004943ee04 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_03:lustre-MDT0000-lwp-OST0001_UUID:3356:1862456998038592:0@lo:400:kworker/u8:2.0 00000100:00100000:2.0:1776178169.701108:0:3356:0:(client.c:2215:ptlrpc_check_set()) Completed RPC req@0000000002c47d13 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_03:lustre-MDT0000-lwp-MDT0001_UUID:3356:1862456998038848:0@lo:400:kworker/u8:2.0 00000100:00100000:2.0:1776178169.701116:0:3356:0:(client.c:2215:ptlrpc_check_set()) Completed RPC req@00000000737321c9 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_03:lustre-MDT0000-mdtlov_UUID:3356:1862456998039104:0@lo:13:osp-pre-1-0.0 00000100:00100000:2.0:1776178169.701122:0:3356:0:(client.c:2215:ptlrpc_check_set()) Completed RPC req@0000000024b881b8 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_03:lustre-MDT0001-lwp-OST0001_UUID:3356:1862456998038656:0@lo:400:kworker/u8:2.0 00000100:00100000:2.0:1776178169.701128:0:3356:0:(client.c:2215:ptlrpc_check_set()) Completed RPC req@000000000c014cb0 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_03:lustre-MDT0000-mdtlov_UUID:3356:1862456998038912:0@lo:400:kworker/u8:2.0 00000100:00100000:2.0:1776178169.701134:0:3356:0:(client.c:2215:ptlrpc_check_set()) Completed RPC req@00000000c7e83057 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_03:lustre-MDT0000-mdtlov_UUID:3356:1862456998039168:0@lo:41:osp-pre-1-0.0 00000100:00100000:2.0:1776178169.701143:0:3356:0:(client.c:2215:ptlrpc_check_set()) Completed RPC req@00000000ebb70e84 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_03:lustre-MDT0001-mdtlov_UUID:3356:1862456998038336:0@lo:41:osp-pre-0-1.0 00000100:00100000:3.0:1776178169.701337:0:8227:0:(service.c:2372:ptlrpc_server_handle_request()) Handled RPC req@0000000037e96ea5 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_000:lustre-MDT0000-mdtlov_UUID+5:3353:x1862456998038976:12345-0@lo:400:kworker/u8:2.0 Request processed in 16us (832us total) trans 0 rc 0/0 00000100:00100000:3.0:1776178169.701344:0:8227:0:(nrs_fifo.c:241:nrs_fifo_req_stop()) NRS stop fifo request from 12345-0@lo, seq: 475 00000100:00100000:0.0:1776178169.701635:0:20137:0:(service.c:2152:ptlrpc_server_handle_req_in()) got req x1862456998038400 00000100:00100000:0.0:1776178169.701673:0:20137:0:(nrs_fifo.c:179:nrs_fifo_req_get()) NRS start fifo request from 12345-0@lo, seq: 142 00000100:00100000:0.0:1776178169.701677:0:20137:0:(service.c:2324:ptlrpc_server_handle_request()) Handling RPC req@000000004d415026 pname:cluuid+ref:pid:xid:nid:opc:job ll_mgs_0000:ac61f839-0e8e-4cac-9c59-08bcfbbad330+23:3356:x1862456998038400:12345-0@lo:400:kworker/u8:2.0 00000100:00100000:0.0:1776178169.701709:0:20137:0:(service.c:2372:ptlrpc_server_handle_request()) Handled RPC req@000000004d415026 pname:cluuid+ref:pid:xid:nid:opc:job ll_mgs_0000:ac61f839-0e8e-4cac-9c59-08bcfbbad330+23:3356:x1862456998038400:12345-0@lo:400:kworker/u8:2.0 Request processed in 31us (1277us total) trans 0 rc 0/0 00000100:00100000:0.0:1776178169.701714:0:20137:0:(nrs_fifo.c:241:nrs_fifo_req_stop()) NRS stop fifo request from 12345-0@lo, seq: 142 00000100:00100000:2.0:1776178169.701967:0:3356:0:(client.c:2215:ptlrpc_check_set()) Completed RPC req@0000000008fe8f11 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_03:ac61f839-0e8e-4cac-9c59-08bcfbbad330:3356:1862456998038400:0@lo:400:kworker/u8:2.0 00010000:00010000:1.0:1776178170.274901:0:6245:0:(ldlm_lockd.c:2792:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:2.0:1776178171.174443:0:3347:0:(events.c:361:request_in_callback()) peer: 12345-192.168.204.3@tcp (source: 12345-192.168.204.3@tcp) 00000100:00100000:2.0:1776178171.174507:0:3347:0:(events.c:361:request_in_callback()) peer: 12345-192.168.204.3@tcp (source: 12345-192.168.204.3@tcp) 00000100:00100000:2.0:1776178171.174543:0:3347:0:(events.c:361:request_in_callback()) peer: 12345-192.168.204.3@tcp (source: 12345-192.168.204.3@tcp) 00000100:00100000:2.0:1776178171.174561:0:3347:0:(events.c:361:request_in_callback()) peer: 12345-192.168.204.3@tcp (source: 12345-192.168.204.3@tcp) 00000100:00100000:2.0:1776178171.174605:0:3347:0:(events.c:361:request_in_callback()) peer: 12345-192.168.204.3@tcp (source: 12345-192.168.204.3@tcp) 00000100:00100000:2.0:1776178171.174653:0:6257:0:(service.c:2152:ptlrpc_server_handle_req_in()) got req x1862456979096000 00000100:00100000:2.0:1776178171.174700:0:6257:0:(service.c:2152:ptlrpc_server_handle_req_in()) got req x1862456979096064 00000100:00100000:2.0:1776178171.174717:0:6257:0:(nrs_fifo.c:179:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.204.3@tcp, seq: 858 00000100:00100000:2.0:1776178171.174735:0:6257:0:(service.c:2324:ptlrpc_server_handle_request()) Handling RPC req@00000000c7e83057 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_002:b50963bd-8eba-4432-9a54-8888d00e3fa1+7:2272:x1862456979096000:12345-192.168.204.3@tcp:400:kworker/u8:3.0 00000100:00100000:2.0:1776178171.174804:0:6257:0:(service.c:2372:ptlrpc_server_handle_request()) Handled RPC req@00000000c7e83057 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_002:b50963bd-8eba-4432-9a54-8888d00e3fa1+7:2272:x1862456979096000:12345-192.168.204.3@tcp:400:kworker/u8:3.0 Request processed in 80us (373us total) trans 0 rc 0/0 00000100:00100000:2.0:1776178171.174812:0:6257:0:(nrs_fifo.c:241:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.204.3@tcp, seq: 858 00000100:00100000:2.0:1776178171.174829:0:6257:0:(nrs_fifo.c:179:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.204.3@tcp, seq: 859 00000100:00100000:2.0:1776178171.174833:0:6257:0:(service.c:2324:ptlrpc_server_handle_request()) Handling RPC req@0000000001246e58 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_002:b50963bd-8eba-4432-9a54-8888d00e3fa1+6:2273:x1862456979096064:12345-192.168.204.3@tcp:400:kworker/u8:3.0 00000100:00100000:2.0:1776178171.174855:0:6257:0:(service.c:2372:ptlrpc_server_handle_request()) Handled RPC req@0000000001246e58 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_002:b50963bd-8eba-4432-9a54-8888d00e3fa1+6:2273:x1862456979096064:12345-192.168.204.3@tcp:400:kworker/u8:3.0 Request processed in 22us (295us total) trans 0 rc 0/0 00000100:00100000:2.0:1776178171.174860:0:6257:0:(nrs_fifo.c:241:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.204.3@tcp, seq: 859 00000100:00100000:3.0:1776178171.175560:0:20140:0:(service.c:2152:ptlrpc_server_handle_req_in()) got req x1862456979095936 00000100:00100000:3.0:1776178171.175599:0:20140:0:(nrs_fifo.c:179:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.204.3@tcp, seq: 143 00000100:00100000:3.0:1776178171.175604:0:20140:0:(service.c:2324:ptlrpc_server_handle_request()) Handling RPC req@00000000d1b75234 pname:cluuid+ref:pid:xid:nid:opc:job ll_mgs_0002:5b5c66b1-1a2f-4651-8548-0c75b9a38a94+9:2273:x1862456979095936:12345-192.168.204.3@tcp:400:kworker/u8:3.0 00000100:00100000:3.0:1776178171.175647:0:20140:0:(service.c:2372:ptlrpc_server_handle_request()) Handled RPC req@00000000d1b75234 pname:cluuid+ref:pid:xid:nid:opc:job ll_mgs_0002:5b5c66b1-1a2f-4651-8548-0c75b9a38a94+9:2273:x1862456979095936:12345-192.168.204.3@tcp:400:kworker/u8:3.0 Request processed in 43us (1141us total) trans 0 rc 0/0 00000100:00100000:3.0:1776178171.175654:0:20140:0:(nrs_fifo.c:241:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.204.3@tcp, seq: 143 00000100:00100000:3.0:1776178171.175688:0:8227:0:(service.c:2152:ptlrpc_server_handle_req_in()) got req x1862456979096128 00000100:00100000:3.0:1776178171.175705:0:8227:0:(service.c:2152:ptlrpc_server_handle_req_in()) got req x1862456979096192 00000100:00100000:3.0:1776178171.176176:0:8227:0:(nrs_fifo.c:179:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.204.3@tcp, seq: 477 00000100:00100000:3.0:1776178171.176182:0:8227:0:(service.c:2324:ptlrpc_server_handle_request()) Handling RPC req@00000000f871d6a5 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_000:b50963bd-8eba-4432-9a54-8888d00e3fa1+8:2272:x1862456979096128:12345-192.168.204.3@tcp:400:kworker/u8:3.0 00000100:00100000:3.0:1776178171.176213:0:8227:0:(service.c:2372:ptlrpc_server_handle_request()) Handled RPC req@00000000f871d6a5 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_000:b50963bd-8eba-4432-9a54-8888d00e3fa1+8:2272:x1862456979096128:12345-192.168.204.3@tcp:400:kworker/u8:3.0 Request processed in 31us (1670us total) trans 0 rc 0/0 00000100:00100000:0.0:1776178171.189557:0:20759:0:(nrs_fifo.c:179:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.204.3@tcp, seq: 478 00000100:00100000:0.0:1776178171.189577:0:20759:0:(service.c:2324:ptlrpc_server_handle_request()) Handling RPC req@00000000ecea139c pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_010:b50963bd-8eba-4432-9a54-8888d00e3fa1+9:2272:x1862456979096192:12345-192.168.204.3@tcp:400:kworker/u8:3.0 00000100:00100000:0.0:1776178171.193277:0:20759:0:(service.c:2372:ptlrpc_server_handle_request()) Handled RPC req@00000000ecea139c pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_010:b50963bd-8eba-4432-9a54-8888d00e3fa1+9:2272:x1862456979096192:12345-192.168.204.3@tcp:400:kworker/u8:3.0 Request processed in 3699us (18665us total) trans 0 rc 0/0 00000100:00100000:0.0:1776178171.193293:0:20759:0:(nrs_fifo.c:241:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.204.3@tcp, seq: 478 00000100:00100000:3.0:1776178171.194091:0:8227:0:(nrs_fifo.c:241:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.204.3@tcp, seq: 477 00000100:00100000:2.0:1776178171.232475:0:3354:0:(ptlrpcd.c:415:ptlrpcd_check()) transfer 1 async RPCs [0->1] 00000100:00100000:2.0:1776178171.248997:0:3354:0:(client.c:1733:ptlrpc_send_new_req()) Sending RPC req@0000000076f23bfa pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_01:lustre-MDT0001-mdtlov_UUID:3354:1862456998039232:0@lo:13:osp-pre-1-1.0 00000100:00100000:2.0:1776178171.249097:0:3354:0:(events.c:361:request_in_callback()) peer: 12345-0@lo (source: 12345-0@lo) 00000100:00100000:3.0:1776178171.249845:0:8232:0:(service.c:2152:ptlrpc_server_handle_req_in()) got req x1862456998039232 00000100:00100000:3.0:1776178171.249908:0:8232:0:(nrs_fifo.c:179:nrs_fifo_req_get()) NRS start fifo request from 12345-0@lo, seq: 453 00000100:00100000:3.0:1776178171.249915:0:8232:0:(service.c:2324:ptlrpc_server_handle_request()) Handling RPC req@00000000949e3fea pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_create00:lustre-MDT0001-mdtlov_UUID+5:3354:x1862456998039232:12345-0@lo:13:osp-pre-1-1.0 00002000:00000024:3.0:1776178171.249942:0:8232:0:(ofd_obd.c:740:ofd_statfs()) blocks cached 229376 granted 491746368 pending 0 free 3923689472 avail 3692167168 00000100:00100000:3.0:1776178171.250021:0:8232:0:(service.c:2372:ptlrpc_server_handle_request()) Handled RPC req@00000000949e3fea pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_create00:lustre-MDT0001-mdtlov_UUID+5:3354:x1862456998039232:12345-0@lo:13:osp-pre-1-1.0 Request processed in 105us (925us total) trans 0 rc 0/0 00000100:00100000:3.0:1776178171.250029:0:8232:0:(nrs_fifo.c:241:nrs_fifo_req_stop()) NRS stop fifo request from 12345-0@lo, seq: 453 00000100:00100000:0.0:1776178171.250143:0:3354:0:(client.c:2215:ptlrpc_check_set()) Completed RPC req@0000000076f23bfa pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_01:lustre-MDT0001-mdtlov_UUID:3354:1862456998039232:0@lo:13:osp-pre-1-1.0 00010000:00010000:2.0:1776178171.297512:0:6244:0:(ldlm_lockd.c:2792:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000001:00080000:2.0:1776178171.751522:0:20134:0:(tgt_lastrcvd.c:913:tgt_cb_last_committed()) lustre-MDT0000: transno 12884901945 is committed 00000100:00100000:0.0:1776178171.757375:0:3353:0:(ptlrpcd.c:415:ptlrpcd_check()) transfer 1 async RPCs [1->0] 00000100:00100000:0.0:1776178171.757398:0:3353:0:(client.c:1733:ptlrpc_send_new_req()) Sending RPC req@00000000b3ddbcd2 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_00:lustre-MDT0001-mdtlov_UUID:3353:1862456998039296:0@lo:13:osp-pre-0-1.0 00000100:00100000:0.0:1776178171.757461:0:3353:0:(events.c:361:request_in_callback()) peer: 12345-0@lo (source: 12345-0@lo) 00000100:00100000:0.0:1776178171.757525:0:9554:0:(service.c:2152:ptlrpc_server_handle_req_in()) got req x1862456998039296 00000100:00100000:0.0:1776178171.757569:0:9554:0:(nrs_fifo.c:179:nrs_fifo_req_get()) NRS start fifo request from 12345-0@lo, seq: 454 00000100:00100000:0.0:1776178171.757575:0:9554:0:(service.c:2324:ptlrpc_server_handle_request()) Handling RPC req@00000000bd6cd251 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_create00:lustre-MDT0001-mdtlov_UUID+5:3353:x1862456998039296:12345-0@lo:13:osp-pre-0-1.0 00002000:00000024:0.0:1776178171.757602:0:9554:0:(ofd_obd.c:740:ofd_statfs()) blocks cached 315392 granted 484883776 pending 0 free 3923689472 avail 3692167168 00000100:00100000:0.0:1776178171.757649:0:9554:0:(service.c:2372:ptlrpc_server_handle_request()) Handled RPC req@00000000bd6cd251 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_create00:lustre-MDT0001-mdtlov_UUID+5:3353:x1862456998039296:12345-0@lo:13:osp-pre-0-1.0 Request processed in 73us (188us total) trans 0 rc 0/0 00000100:00100000:0.0:1776178171.757656:0:9554:0:(nrs_fifo.c:241:nrs_fifo_req_stop()) NRS stop fifo request from 12345-0@lo, seq: 454 00000100:00100000:0.0:1776178171.757696:0:3353:0:(client.c:2215:ptlrpc_check_set()) Completed RPC req@00000000b3ddbcd2 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_00:lustre-MDT0001-mdtlov_UUID:3353:1862456998039296:0@lo:13:osp-pre-0-1.0 00010000:00010000:2.0:1776178172.321049:0:9225:0:(ldlm_lockd.c:2792:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00010000:00010000:2.0:1776178173.348125:0:9224:0:(ldlm_lockd.c:2792:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00010000:00010000:1.0:1776178174.375171:0:6245:0:(ldlm_lockd.c:2792:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:3.0:1776178174.817498:0:3355:0:(client.c:1733:ptlrpc_send_new_req()) Sending RPC req@000000003fa4e1db pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_02:lustre-MDT0000-mdtlov_UUID:3355:1862456998039360:0@lo:13:osp-pre-1-0.0 00000100:00100000:3.0:1776178174.817574:0:3355:0:(events.c:361:request_in_callback()) peer: 12345-0@lo (source: 12345-0@lo) 00000100:00100000:3.0:1776178174.817642:0:8232:0:(service.c:2152:ptlrpc_server_handle_req_in()) got req x1862456998039360 00000100:00100000:3.0:1776178174.817691:0:8232:0:(nrs_fifo.c:179:nrs_fifo_req_get()) NRS start fifo request from 12345-0@lo, seq: 455 00000100:00100000:3.0:1776178174.817698:0:8232:0:(service.c:2324:ptlrpc_server_handle_request()) Handling RPC req@0000000076f23bfa pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_create00:lustre-MDT0000-mdtlov_UUID+5:3355:x1862456998039360:12345-0@lo:13:osp-pre-1-0.0 00002000:00000024:3.0:1776178174.817739:0:8232:0:(ofd_obd.c:740:ofd_statfs()) blocks cached 229376 granted 491746368 pending 0 free 3923689472 avail 3692167168 00000100:00100000:3.0:1776178174.817785:0:8232:0:(service.c:2372:ptlrpc_server_handle_request()) Handled RPC req@0000000076f23bfa pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_create00:lustre-MDT0000-mdtlov_UUID+5:3355:x1862456998039360:12345-0@lo:13:osp-pre-1-0.0 Request processed in 86us (212us total) trans 0 rc 0/0 00000100:00100000:3.0:1776178174.817792:0:8232:0:(nrs_fifo.c:241:nrs_fifo_req_stop()) NRS stop fifo request from 12345-0@lo, seq: 455 00000100:00100000:3.0:1776178174.817847:0:3355:0:(client.c:2215:ptlrpc_check_set()) Completed RPC req@000000003fa4e1db pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_02:lustre-MDT0000-mdtlov_UUID:3355:1862456998039360:0@lo:13:osp-pre-1-0.0 00000100:00100000:3.0:1776178174.817917:0:3356:0:(client.c:1733:ptlrpc_send_new_req()) Sending RPC req@000000003fa4e1db pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_03:lustre-MDT0000-mdtlov_UUID:3356:1862456998039424:0@lo:41:osp-pre-1-0.0 00000100:00100000:3.0:1776178174.817945:0:3356:0:(events.c:361:request_in_callback()) peer: 12345-0@lo (source: 12345-0@lo) 00000100:00100000:2.0:1776178174.818176:0:3354:0:(client.c:1733:ptlrpc_send_new_req()) Sending RPC req@0000000056a48d5b pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_01:lustre-MDT0000-lwp-OST0000_UUID:3354:1862456998039552:0@lo:400:kworker/u8:0.0 00000100:00100000:2.0:1776178174.818208:0:3354:0:(events.c:361:request_in_callback()) peer: 12345-0@lo (source: 12345-0@lo) 00000100:00100000:2.0:1776178174.818227:0:3354:0:(client.c:1733:ptlrpc_send_new_req()) Sending RPC req@00000000eec4b6d0 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_01:lustre-MDT0001-mdtlov_UUID:3354:1862456998039808:0@lo:400:kworker/u8:0.0 00000100:00100000:2.0:1776178174.818248:0:3354:0:(events.c:361:request_in_callback()) peer: 12345-0@lo (source: 12345-0@lo) 00000100:00100000:2.0:1776178174.818276:0:3354:0:(client.c:1733:ptlrpc_send_new_req()) Sending RPC req@0000000018bd7474 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_01:lustre-MDT0000-mdtlov_UUID:3354:1862456998040064:0@lo:400:kworker/u8:0.0 00000100:00100000:2.0:1776178174.818300:0:3354:0:(events.c:361:request_in_callback()) peer: 12345-0@lo (source: 12345-0@lo) 00000100:00100000:2.0:1776178174.818319:0:3354:0:(ptlrpcd.c:415:ptlrpcd_check()) transfer 3 async RPCs [0->1] 00000100:00100000:2.0:1776178174.818324:0:3354:0:(client.c:1733:ptlrpc_send_new_req()) Sending RPC req@000000009b04a931 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_01:ac61f839-0e8e-4cac-9c59-08bcfbbad330:3354:1862456998039488:0@lo:400:kworker/u8:0.0 00000100:00100000:2.0:1776178174.818367:0:3354:0:(events.c:361:request_in_callback()) peer: 12345-0@lo (source: 12345-0@lo) 00000100:00100000:2.0:1776178174.818384:0:3354:0:(client.c:1733:ptlrpc_send_new_req()) Sending RPC req@00000000a979415d pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_01:lustre-MDT0001-lwp-OST0001_UUID:3354:1862456998039744:0@lo:400:kworker/u8:0.0 00000100:00100000:2.0:1776178174.818405:0:3354:0:(events.c:361:request_in_callback()) peer: 12345-0@lo (source: 12345-0@lo) 00000100:00100000:2.0:1776178174.818418:0:3354:0:(client.c:1733:ptlrpc_send_new_req()) Sending RPC req@0000000020763766 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_01:lustre-MDT0000-lwp-MDT0001_UUID:3354:1862456998040000:0@lo:400:kworker/u8:0.0 00000100:00100000:2.0:1776178174.818437:0:3354:0:(events.c:361:request_in_callback()) peer: 12345-0@lo (source: 12345-0@lo) 00000100:00100000:2.0:1776178174.818457:0:3355:0:(client.c:1733:ptlrpc_send_new_req()) Sending RPC req@0000000022264af5 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_02:lustre-MDT0001-lwp-OST0000_UUID:3355:1862456998039616:0@lo:400:kworker/u8:0.0 00000100:00100000:2.0:1776178174.818474:0:3355:0:(events.c:361:request_in_callback()) peer: 12345-0@lo (source: 12345-0@lo) 00000100:00100000:2.0:1776178174.818487:0:3355:0:(client.c:1733:ptlrpc_send_new_req()) Sending RPC req@000000007c827612 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_02:lustre-MDT0001-mdtlov_UUID:3355:1862456998039872:0@lo:400:kworker/u8:0.0 00000100:00100000:2.0:1776178174.818503:0:3355:0:(events.c:361:request_in_callback()) peer: 12345-0@lo (source: 12345-0@lo) 00000100:00100000:2.0:1776178174.818514:0:3355:0:(client.c:1733:ptlrpc_send_new_req()) Sending RPC req@000000002549e7f5 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_02:lustre-MDT0000-lwp-MDT0000_UUID:3355:1862456998040128:0@lo:400:kworker/u8:0.0 00000100:00100000:2.0:1776178174.818529:0:3355:0:(events.c:361:request_in_callback()) peer: 12345-0@lo (source: 12345-0@lo) 00000100:00100000:2.0:1776178174.818540:0:3355:0:(ptlrpcd.c:415:ptlrpcd_check()) transfer 2 async RPCs [3->2] 00000100:00100000:2.0:1776178174.818545:0:3355:0:(client.c:1733:ptlrpc_send_new_req()) Sending RPC req@0000000014e690d4 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_02:lustre-MDT0000-lwp-OST0001_UUID:3355:1862456998039680:0@lo:400:kworker/u8:0.0 00000100:00100000:2.0:1776178174.818563:0:3355:0:(events.c:361:request_in_callback()) peer: 12345-0@lo (source: 12345-0@lo) 00000100:00100000:2.0:1776178174.818570:0:3355:0:(client.c:1733:ptlrpc_send_new_req()) Sending RPC req@00000000a101cbf3 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_02:lustre-MDT0001-mdtlov_UUID:3355:1862456998039936:0@lo:400:kworker/u8:0.0 00000100:00100000:2.0:1776178174.818585:0:3355:0:(events.c:361:request_in_callback()) peer: 12345-0@lo (source: 12345-0@lo) 00000100:00100000:2.0:1776178174.818613:0:6255:0:(service.c:2152:ptlrpc_server_handle_req_in()) got req x1862456998039552 00000100:00100000:2.0:1776178174.818636:0:6255:0:(service.c:2152:ptlrpc_server_handle_req_in()) got req x1862456998039744 00000100:00100000:2.0:1776178174.818646:0:6255:0:(service.c:2152:ptlrpc_server_handle_req_in()) got req x1862456998040000 00000100:00100000:2.0:1776178174.818654:0:6255:0:(service.c:2152:ptlrpc_server_handle_req_in()) got req x1862456998039616 00000100:00100000:2.0:1776178174.818661:0:6255:0:(service.c:2152:ptlrpc_server_handle_req_in()) got req x1862456998040128 00000100:00100000:2.0:1776178174.818668:0:6255:0:(service.c:2152:ptlrpc_server_handle_req_in()) got req x1862456998039680 00000100:00100000:2.0:1776178174.818679:0:6255:0:(nrs_fifo.c:179:nrs_fifo_req_get()) NRS start fifo request from 12345-0@lo, seq: 860 00000100:00100000:2.0:1776178174.818682:0:6255:0:(service.c:2324:ptlrpc_server_handle_request()) Handling RPC req@00000000803faf52 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_000:lustre-MDT0000-lwp-OST0000_UUID+5:3354:x1862456998039552:12345-0@lo:400:kworker/u8:0.0 00000100:00100000:2.0:1776178174.818723:0:6255:0:(service.c:2372:ptlrpc_server_handle_request()) Handled RPC req@00000000803faf52 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_000:lustre-MDT0000-lwp-OST0000_UUID+5:3354:x1862456998039552:12345-0@lo:400:kworker/u8:0.0 Request processed in 39us (515us total) trans 0 rc 0/0 00000100:00100000:2.0:1776178174.818729:0:6255:0:(nrs_fifo.c:241:nrs_fifo_req_stop()) NRS stop fifo request from 12345-0@lo, seq: 860 00000100:00100000:2.0:1776178174.818742:0:6255:0:(nrs_fifo.c:179:nrs_fifo_req_get()) NRS start fifo request from 12345-0@lo, seq: 861 00000100:00100000:2.0:1776178174.818745:0:6255:0:(service.c:2324:ptlrpc_server_handle_request()) Handling RPC req@00000000e67f6ac0 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_000:lustre-MDT0001-lwp-OST0001_UUID+5:3354:x1862456998039744:12345-0@lo:400:kworker/u8:0.0 00000100:00100000:2.0:1776178174.818769:0:6255:0:(service.c:2372:ptlrpc_server_handle_request()) Handled RPC req@00000000e67f6ac0 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_000:lustre-MDT0001-lwp-OST0001_UUID+5:3354:x1862456998039744:12345-0@lo:400:kworker/u8:0.0 Request processed in 23us (363us total) trans 0 rc 0/0 00000100:00100000:2.0:1776178174.818774:0:6255:0:(nrs_fifo.c:241:nrs_fifo_req_stop()) NRS stop fifo request from 12345-0@lo, seq: 861 00000100:00100000:2.0:1776178174.818785:0:6255:0:(nrs_fifo.c:179:nrs_fifo_req_get()) NRS start fifo request from 12345-0@lo, seq: 862 00000100:00100000:2.0:1776178174.818788:0:6255:0:(service.c:2324:ptlrpc_server_handle_request()) Handling RPC req@000000008be25fe6 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_000:lustre-MDT0000-lwp-MDT0001_UUID+5:3354:x1862456998040000:12345-0@lo:400:kworker/u8:0.0 00000100:00100000:2.0:1776178174.818807:0:6255:0:(service.c:2372:ptlrpc_server_handle_request()) Handled RPC req@000000008be25fe6 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_000:lustre-MDT0000-lwp-MDT0001_UUID+5:3354:x1862456998040000:12345-0@lo:400:kworker/u8:0.0 Request processed in 19us (371us total) trans 0 rc 0/0 00000100:00100000:2.0:1776178174.818812:0:6255:0:(nrs_fifo.c:241:nrs_fifo_req_stop()) NRS stop fifo request from 12345-0@lo, seq: 862 00000100:00100000:2.0:1776178174.818821:0:6255:0:(nrs_fifo.c:179:nrs_fifo_req_get()) NRS start fifo request from 12345-0@lo, seq: 863 00000100:00100000:2.0:1776178174.818824:0:6255:0:(service.c:2324:ptlrpc_server_handle_request()) Handling RPC req@0000000047a42b33 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_000:lustre-MDT0001-lwp-OST0000_UUID+5:3355:x1862456998039616:12345-0@lo:400:kworker/u8:0.0 00000100:00100000:2.0:1776178174.818847:0:6255:0:(service.c:2372:ptlrpc_server_handle_request()) Handled RPC req@0000000047a42b33 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_000:lustre-MDT0001-lwp-OST0000_UUID+5:3355:x1862456998039616:12345-0@lo:400:kworker/u8:0.0 Request processed in 22us (372us total) trans 0 rc 0/0 00000100:00100000:2.0:1776178174.818853:0:6255:0:(nrs_fifo.c:241:nrs_fifo_req_stop()) NRS stop fifo request from 12345-0@lo, seq: 863 00000100:00100000:2.0:1776178174.818862:0:6255:0:(nrs_fifo.c:179:nrs_fifo_req_get()) NRS start fifo request from 12345-0@lo, seq: 864 00000100:00100000:2.0:1776178174.818866:0:6255:0:(service.c:2324:ptlrpc_server_handle_request()) Handling RPC req@000000004e1a7b1b pname:cluuid+ref:pid:xid:nid:opc:job mdt00_000:lustre-MDT0000-lwp-MDT0000_UUID+5:3355:x1862456998040128:12345-0@lo:400:kworker/u8:0.0 00000100:00100000:2.0:1776178174.818885:0:6255:0:(service.c:2372:ptlrpc_server_handle_request()) Handled RPC req@000000004e1a7b1b pname:cluuid+ref:pid:xid:nid:opc:job mdt00_000:lustre-MDT0000-lwp-MDT0000_UUID+5:3355:x1862456998040128:12345-0@lo:400:kworker/u8:0.0 Request processed in 19us (356us total) trans 0 rc 0/0 00000100:00100000:2.0:1776178174.818890:0:6255:0:(nrs_fifo.c:241:nrs_fifo_req_stop()) NRS stop fifo request from 12345-0@lo, seq: 864 00000100:00100000:2.0:1776178174.818899:0:6255:0:(nrs_fifo.c:179:nrs_fifo_req_get()) NRS start fifo request from 12345-0@lo, seq: 865 00000100:00100000:2.0:1776178174.818902:0:6255:0:(service.c:2324:ptlrpc_server_handle_request()) Handling RPC req@000000007d6ef583 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_000:lustre-MDT0000-lwp-OST0001_UUID+5:3355:x1862456998039680:12345-0@lo:400:kworker/u8:0.0 00000100:00100000:2.0:1776178174.818921:0:6255:0:(service.c:2372:ptlrpc_server_handle_request()) Handled RPC req@000000007d6ef583 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_000:lustre-MDT0000-lwp-OST0001_UUID+5:3355:x1862456998039680:12345-0@lo:400:kworker/u8:0.0 Request processed in 18us (358us total) trans 0 rc 0/0 00000100:00100000:2.0:1776178174.818926:0:6255:0:(nrs_fifo.c:241:nrs_fifo_req_stop()) NRS stop fifo request from 12345-0@lo, seq: 865 00000100:00100000:2.0:1776178174.818942:0:6260:0:(service.c:2152:ptlrpc_server_handle_req_in()) got req x1862456998039424 00000100:00100000:2.0:1776178174.818961:0:6260:0:(service.c:2152:ptlrpc_server_handle_req_in()) got req x1862456998039808 00000100:00100000:2.0:1776178174.818976:0:6260:0:(nrs_fifo.c:179:nrs_fifo_req_get()) NRS start fifo request from 12345-0@lo, seq: 673 00000100:00100000:2.0:1776178174.818979:0:6260:0:(service.c:2324:ptlrpc_server_handle_request()) Handling RPC req@0000000008fe8f11 pname:cluuid+ref:pid:xid:nid:opc:job mdt_out00_000:lustre-MDT0000-mdtlov_UUID+5:3356:x1862456998039424:12345-0@lo:41:osp-pre-1-0.0 00000004:00000024:2.0:1776178174.819001:0:6260:0:(mdt_handler.c:548:mdt_statfs()) blocks cached 0 granted 2097152 pending 0 free 1446035456 avail 1316573184 00000100:00100000:2.0:1776178174.819034:0:6260:0:(service.c:2372:ptlrpc_server_handle_request()) Handled RPC req@0000000008fe8f11 pname:cluuid+ref:pid:xid:nid:opc:job mdt_out00_000:lustre-MDT0000-mdtlov_UUID+5:3356:x1862456998039424:12345-0@lo:41:osp-pre-1-0.0 Request processed in 53us (1089us total) trans 0 rc 0/0 00000100:00100000:2.0:1776178174.819040:0:6260:0:(nrs_fifo.c:241:nrs_fifo_req_stop()) NRS stop fifo request from 12345-0@lo, seq: 673 00000100:00100000:2.0:1776178174.819051:0:6260:0:(nrs_fifo.c:179:nrs_fifo_req_get()) NRS start fifo request from 12345-0@lo, seq: 674 00000100:00100000:2.0:1776178174.819054:0:6260:0:(service.c:2324:ptlrpc_server_handle_request()) Handling RPC req@000000008c50846d pname:cluuid+ref:pid:xid:nid:opc:job mdt_out00_000:lustre-MDT0001-mdtlov_UUID+6:3354:x1862456998039808:12345-0@lo:400:kworker/u8:0.0 00000100:00100000:2.0:1776178174.819102:0:6260:0:(service.c:2372:ptlrpc_server_handle_request()) Handled RPC req@000000008c50846d pname:cluuid+ref:pid:xid:nid:opc:job mdt_out00_000:lustre-MDT0001-mdtlov_UUID+6:3354:x1862456998039808:12345-0@lo:400:kworker/u8:0.0 Request processed in 46us (853us total) trans 0 rc 0/0 00000100:00100000:2.0:1776178174.819107:0:6260:0:(nrs_fifo.c:241:nrs_fifo_req_stop()) NRS stop fifo request from 12345-0@lo, seq: 674 00000100:00100000:2.0:1776178174.819124:0:20755:0:(service.c:2152:ptlrpc_server_handle_req_in()) got req x1862456998040064 00000100:00100000:2.0:1776178174.819147:0:20755:0:(service.c:2152:ptlrpc_server_handle_req_in()) got req x1862456998039872 00000100:00100000:2.0:1776178174.819160:0:20755:0:(service.c:2152:ptlrpc_server_handle_req_in()) got req x1862456998039936 00000100:00100000:2.0:1776178174.819176:0:20755:0:(nrs_fifo.c:179:nrs_fifo_req_get()) NRS start fifo request from 12345-0@lo, seq: 479 00000100:00100000:2.0:1776178174.819179:0:20755:0:(service.c:2324:ptlrpc_server_handle_request()) Handling RPC req@0000000056e345df pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_006:lustre-MDT0000-mdtlov_UUID+5:3354:x1862456998040064:12345-0@lo:400:kworker/u8:0.0 00000100:00100000:2.0:1776178174.819203:0:20755:0:(service.c:2372:ptlrpc_server_handle_request()) Handled RPC req@0000000056e345df pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_006:lustre-MDT0000-mdtlov_UUID+5:3354:x1862456998040064:12345-0@lo:400:kworker/u8:0.0 Request processed in 24us (903us total) trans 0 rc 0/0 00000100:00100000:2.0:1776178174.819209:0:20755:0:(nrs_fifo.c:241:nrs_fifo_req_stop()) NRS stop fifo request from 12345-0@lo, seq: 479 00000100:00100000:2.0:1776178174.819219:0:20755:0:(nrs_fifo.c:179:nrs_fifo_req_get()) NRS start fifo request from 12345-0@lo, seq: 480 00000100:00100000:2.0:1776178174.819222:0:20755:0:(service.c:2324:ptlrpc_server_handle_request()) Handling RPC req@000000001da3f43e pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_006:lustre-MDT0001-mdtlov_UUID+5:3355:x1862456998039872:12345-0@lo:400:kworker/u8:0.0 00000100:00100000:2.0:1776178174.819242:0:20755:0:(service.c:2372:ptlrpc_server_handle_request()) Handled RPC req@000000001da3f43e pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_006:lustre-MDT0001-mdtlov_UUID+5:3355:x1862456998039872:12345-0@lo:400:kworker/u8:0.0 Request processed in 19us (739us total) trans 0 rc 0/0 00000100:00100000:2.0:1776178174.819247:0:20755:0:(nrs_fifo.c:241:nrs_fifo_req_stop()) NRS stop fifo request from 12345-0@lo, seq: 480 00000100:00100000:3.0:1776178174.822443:0:3353:0:(client.c:1733:ptlrpc_send_new_req()) Sending RPC req@00000000d28be450 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_00:lustre-MDT0000-mdtlov_UUID:3353:1862456998040256:0@lo:13:osp-pre-0-0.0 00000100:00100000:3.0:1776178174.822512:0:3353:0:(events.c:361:request_in_callback()) peer: 12345-0@lo (source: 12345-0@lo) 00000100:00100000:3.0:1776178174.822566:0:9554:0:(service.c:2152:ptlrpc_server_handle_req_in()) got req x1862456998040256 00000100:00100000:3.0:1776178174.822605:0:9554:0:(nrs_fifo.c:179:nrs_fifo_req_get()) NRS start fifo request from 12345-0@lo, seq: 456 00000100:00100000:3.0:1776178174.822610:0:9554:0:(service.c:2324:ptlrpc_server_handle_request()) Handling RPC req@000000000c014cb0 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_create00:lustre-MDT0000-mdtlov_UUID+5:3353:x1862456998040256:12345-0@lo:13:osp-pre-0-0.0 00002000:00000024:3.0:1776178174.822640:0:9554:0:(ofd_obd.c:740:ofd_statfs()) blocks cached 315392 granted 484883776 pending 0 free 3923689472 avail 3692167168 00000100:00100000:3.0:1776178174.822678:0:9554:0:(service.c:2372:ptlrpc_server_handle_request()) Handled RPC req@000000000c014cb0 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_create00:lustre-MDT0000-mdtlov_UUID+5:3353:x1862456998040256:12345-0@lo:13:osp-pre-0-0.0 Request processed in 67us (168us total) trans 0 rc 0/0 00000100:00100000:3.0:1776178174.822685:0:9554:0:(nrs_fifo.c:241:nrs_fifo_req_stop()) NRS stop fifo request from 12345-0@lo, seq: 456 00000100:00100000:3.0:1776178174.822730:0:3353:0:(client.c:2215:ptlrpc_check_set()) Completed RPC req@00000000d28be450 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_00:lustre-MDT0000-mdtlov_UUID:3353:1862456998040256:0@lo:13:osp-pre-0-0.0 00000100:00100000:3.0:1776178174.822799:0:3354:0:(client.c:2215:ptlrpc_check_set()) Completed RPC req@00000000a979415d pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_01:lustre-MDT0001-lwp-OST0001_UUID:3354:1862456998039744:0@lo:400:kworker/u8:0.0 00000100:00100000:3.0:1776178174.822809:0:3354:0:(client.c:2215:ptlrpc_check_set()) Completed RPC req@0000000020763766 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_01:lustre-MDT0000-lwp-MDT0001_UUID:3354:1862456998040000:0@lo:400:kworker/u8:0.0 00000100:00100000:3.0:1776178174.822820:0:3354:0:(client.c:2215:ptlrpc_check_set()) Completed RPC req@0000000056a48d5b pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_01:lustre-MDT0000-lwp-OST0000_UUID:3354:1862456998039552:0@lo:400:kworker/u8:0.0 00000100:00100000:3.0:1776178174.822829:0:3354:0:(client.c:2868:ptlrpc_free_committed()) lustre-MDT0000-osp-MDT0001: committing for last_committed 12884901945 gen 3 00000004:00080000:3.0:1776178174.822834:0:3354:0:(osp_trans.c:1044:osp_request_commit_cb()) trans no 12884901945 committed transno 12884901945 00000100:00100000:3.0:1776178174.822852:0:3354:0:(client.c:2215:ptlrpc_check_set()) Completed RPC req@00000000eec4b6d0 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_01:lustre-MDT0001-mdtlov_UUID:3354:1862456998039808:0@lo:400:kworker/u8:0.0 00000100:00100000:3.0:1776178174.822864:0:3354:0:(client.c:2215:ptlrpc_check_set()) Completed RPC req@0000000018bd7474 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_01:lustre-MDT0000-mdtlov_UUID:3354:1862456998040064:0@lo:400:kworker/u8:0.0 00000100:00100000:3.0:1776178174.822932:0:20137:0:(service.c:2152:ptlrpc_server_handle_req_in()) got req x1862456998039488 00000100:00100000:3.0:1776178174.822959:0:20137:0:(nrs_fifo.c:179:nrs_fifo_req_get()) NRS start fifo request from 12345-0@lo, seq: 144 00000100:00100000:3.0:1776178174.822963:0:20137:0:(service.c:2324:ptlrpc_server_handle_request()) Handling RPC req@00000000e22b0439 pname:cluuid+ref:pid:xid:nid:opc:job ll_mgs_0000:ac61f839-0e8e-4cac-9c59-08bcfbbad330+23:3354:x1862456998039488:12345-0@lo:400:kworker/u8:0.0 00000100:00100000:3.0:1776178174.822999:0:20137:0:(service.c:2372:ptlrpc_server_handle_request()) Handled RPC req@00000000e22b0439 pname:cluuid+ref:pid:xid:nid:opc:job ll_mgs_0000:ac61f839-0e8e-4cac-9c59-08bcfbbad330+23:3354:x1862456998039488:12345-0@lo:400:kworker/u8:0.0 Request processed in 35us (4632us total) trans 0 rc 0/0 00000100:00100000:3.0:1776178174.823005:0:20137:0:(nrs_fifo.c:241:nrs_fifo_req_stop()) NRS stop fifo request from 12345-0@lo, seq: 144 00000100:00100000:3.0:1776178174.823034:0:3354:0:(client.c:2215:ptlrpc_check_set()) Completed RPC req@000000009b04a931 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_01:ac61f839-0e8e-4cac-9c59-08bcfbbad330:3354:1862456998039488:0@lo:400:kworker/u8:0.0 00000100:00100000:2.0:1776178174.824346:0:20755:0:(nrs_fifo.c:179:nrs_fifo_req_get()) NRS start fifo request from 12345-0@lo, seq: 481 00000100:00100000:2.0:1776178174.824367:0:20755:0:(service.c:2324:ptlrpc_server_handle_request()) Handling RPC req@00000000f351bcc3 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_006:lustre-MDT0001-mdtlov_UUID+5:3355:x1862456998039936:12345-0@lo:400:kworker/u8:0.0 00000100:00100000:2.0:1776178174.824443:0:20755:0:(service.c:2372:ptlrpc_server_handle_request()) Handled RPC req@00000000f351bcc3 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_006:lustre-MDT0001-mdtlov_UUID+5:3355:x1862456998039936:12345-0@lo:400:kworker/u8:0.0 Request processed in 82us (5858us total) trans 0 rc 0/0 00000100:00100000:0.0:1776178174.827015:0:3355:0:(client.c:2215:ptlrpc_check_set()) Completed RPC req@0000000014e690d4 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_02:lustre-MDT0000-lwp-OST0001_UUID:3355:1862456998039680:0@lo:400:kworker/u8:0.0 00000100:00100000:0.0:1776178174.827036:0:3355:0:(client.c:2215:ptlrpc_check_set()) Completed RPC req@00000000a101cbf3 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_02:lustre-MDT0001-mdtlov_UUID:3355:1862456998039936:0@lo:400:kworker/u8:0.0 00000100:00100000:0.0:1776178174.827049:0:3355:0:(client.c:2215:ptlrpc_check_set()) Completed RPC req@0000000022264af5 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_02:lustre-MDT0001-lwp-OST0000_UUID:3355:1862456998039616:0@lo:400:kworker/u8:0.0 00000100:00100000:0.0:1776178174.827073:0:3355:0:(client.c:2215:ptlrpc_check_set()) Completed RPC req@000000007c827612 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_02:lustre-MDT0001-mdtlov_UUID:3355:1862456998039872:0@lo:400:kworker/u8:0.0 00000100:00100000:0.0:1776178174.827085:0:3355:0:(client.c:2215:ptlrpc_check_set()) Completed RPC req@000000002549e7f5 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_02:lustre-MDT0000-lwp-MDT0000_UUID:3355:1862456998040128:0@lo:400:kworker/u8:0.0 00000100:00100000:3.0:1776178174.827085:0:3356:0:(client.c:1733:ptlrpc_send_new_req()) Sending RPC req@00000000b3ddbcd2 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_03:lustre-MDT0001-mdtlov_UUID:3356:1862456998040192:0@lo:41:osp-pre-0-1.0 00000100:00100000:0.0:1776178174.827182:0:3356:0:(events.c:361:request_in_callback()) peer: 12345-0@lo (source: 12345-0@lo) 00000100:00100000:2.0:1776178174.827225:0:20755:0:(nrs_fifo.c:241:nrs_fifo_req_stop()) NRS stop fifo request from 12345-0@lo, seq: 481 00000100:00100000:0.0:1776178174.827231:0:3356:0:(client.c:2215:ptlrpc_check_set()) Completed RPC req@000000003fa4e1db pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_03:lustre-MDT0000-mdtlov_UUID:3356:1862456998039424:0@lo:41:osp-pre-1-0.0 00000100:00100000:3.0:1776178174.827313:0:6261:0:(service.c:2152:ptlrpc_server_handle_req_in()) got req x1862456998040192 00000100:00100000:3.0:1776178174.827354:0:6261:0:(nrs_fifo.c:179:nrs_fifo_req_get()) NRS start fifo request from 12345-0@lo, seq: 675 00000100:00100000:3.0:1776178174.827359:0:6261:0:(service.c:2324:ptlrpc_server_handle_request()) Handling RPC req@00000000f9d811fb pname:cluuid+ref:pid:xid:nid:opc:job mdt_out00_001:lustre-MDT0001-mdtlov_UUID+6:3356:x1862456998040192:12345-0@lo:41:osp-pre-0-1.0 00000004:00000024:3.0:1776178174.827392:0:6261:0:(mdt_handler.c:548:mdt_statfs()) blocks cached 0 granted 2097152 pending 0 free 1446121472 avail 1316659200 00000100:00100000:3.0:1776178174.827478:0:6261:0:(service.c:2372:ptlrpc_server_handle_request()) Handled RPC req@00000000f9d811fb pname:cluuid+ref:pid:xid:nid:opc:job mdt_out00_001:lustre-MDT0001-mdtlov_UUID+6:3356:x1862456998040192:12345-0@lo:41:osp-pre-0-1.0 Request processed in 117us (297us total) trans 0 rc 0/0 00000100:00100000:3.0:1776178174.827484:0:6261:0:(nrs_fifo.c:241:nrs_fifo_req_stop()) NRS stop fifo request from 12345-0@lo, seq: 675 00000100:00100000:2.0:1776178174.827536:0:3356:0:(client.c:2215:ptlrpc_check_set()) Completed RPC req@00000000b3ddbcd2 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_03:lustre-MDT0001-mdtlov_UUID:3356:1862456998040192:0@lo:41:osp-pre-0-1.0 00010000:00010000:2.0:1776178175.392531:0:6244:0:(ldlm_lockd.c:2792:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:3.0:1776178176.293666:0:3346:0:(events.c:361:request_in_callback()) peer: 12345-192.168.204.3@tcp (source: 12345-192.168.204.3@tcp) 00000100:00100000:3.0:1776178176.293756:0:3346:0:(events.c:361:request_in_callback()) peer: 12345-192.168.204.3@tcp (source: 12345-192.168.204.3@tcp) 00000100:00100000:3.0:1776178176.293805:0:3346:0:(events.c:361:request_in_callback()) peer: 12345-192.168.204.3@tcp (source: 12345-192.168.204.3@tcp) 00000100:00100000:1.0:1776178176.294245:0:20755:0:(service.c:2152:ptlrpc_server_handle_req_in()) got req x1862456979096512 00000100:00100000:2.0:1776178176.294245:0:11281:0:(service.c:2152:ptlrpc_server_handle_req_in()) got req x1862456979096384 00000100:00100000:1.0:1776178176.294348:0:20755:0:(nrs_fifo.c:179:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.204.3@tcp, seq: 482 00000100:00100000:1.0:1776178176.294357:0:20755:0:(service.c:2324:ptlrpc_server_handle_request()) Handling RPC req@0000000002c47d13 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_006:b50963bd-8eba-4432-9a54-8888d00e3fa1+9:2272:x1862456979096512:12345-192.168.204.3@tcp:400:kworker/u8:3.0 00000100:00100000:2.0:1776178176.294406:0:11281:0:(nrs_fifo.c:179:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.204.3@tcp, seq: 866 00000100:00100000:2.0:1776178176.294412:0:11281:0:(service.c:2324:ptlrpc_server_handle_request()) Handling RPC req@0000000024b881b8 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_004:b50963bd-8eba-4432-9a54-8888d00e3fa1+6:2272:x1862456979096384:12345-192.168.204.3@tcp:400:kworker/u8:3.0 00000100:00100000:0.0:1776178176.294507:0:20140:0:(service.c:2152:ptlrpc_server_handle_req_in()) got req x1862456979096256 00000100:00100000:0.0:1776178176.294536:0:20140:0:(nrs_fifo.c:179:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.204.3@tcp, seq: 145 00000100:00100000:0.0:1776178176.294541:0:20140:0:(service.c:2324:ptlrpc_server_handle_request()) Handling RPC req@00000000b8c10765 pname:cluuid+ref:pid:xid:nid:opc:job ll_mgs_0002:5b5c66b1-1a2f-4651-8548-0c75b9a38a94+9:2272:x1862456979096256:12345-192.168.204.3@tcp:400:kworker/u8:3.0 00000100:00100000:1.0:1776178176.294605:0:20755:0:(service.c:2372:ptlrpc_server_handle_request()) Handled RPC req@0000000002c47d13 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_006:b50963bd-8eba-4432-9a54-8888d00e3fa1+9:2272:x1862456979096512:12345-192.168.204.3@tcp:400:kworker/u8:3.0 Request processed in 249us (800us total) trans 0 rc 0/0 00000100:00100000:1.0:1776178176.294618:0:20755:0:(nrs_fifo.c:241:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.204.3@tcp, seq: 482 00000100:00100000:0.0:1776178176.298009:0:20140:0:(service.c:2372:ptlrpc_server_handle_request()) Handled RPC req@00000000b8c10765 pname:cluuid+ref:pid:xid:nid:opc:job ll_mgs_0002:5b5c66b1-1a2f-4651-8548-0c75b9a38a94+9:2272:x1862456979096256:12345-192.168.204.3@tcp:400:kworker/u8:3.0 Request processed in 3455us (4346us total) trans 0 rc 0/0 00000100:00100000:0.0:1776178176.298027:0:20140:0:(nrs_fifo.c:241:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.204.3@tcp, seq: 145 00000100:00100000:2.0:1776178176.298292:0:11281:0:(service.c:2372:ptlrpc_server_handle_request()) Handled RPC req@0000000024b881b8 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_004:b50963bd-8eba-4432-9a54-8888d00e3fa1+6:2272:x1862456979096384:12345-192.168.204.3@tcp:400:kworker/u8:3.0 Request processed in 3584us (4241us total) trans 0 rc 0/0 00000100:00100000:2.0:1776178176.298304:0:11281:0:(nrs_fifo.c:241:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.204.3@tcp, seq: 866 00000100:00100000:1.0:1776178176.302521:0:3347:0:(events.c:361:request_in_callback()) peer: 12345-192.168.204.3@tcp (source: 12345-192.168.204.3@tcp) 00000100:00100000:2.0:1776178176.304164:0:11281:0:(service.c:2152:ptlrpc_server_handle_req_in()) got req x1862456979096320 00000100:00100000:2.0:1776178176.304268:0:11281:0:(nrs_fifo.c:179:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.204.3@tcp, seq: 867 00000100:00100000:2.0:1776178176.304277:0:11281:0:(service.c:2324:ptlrpc_server_handle_request()) Handling RPC req@00000000b6612f78 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_004:b50963bd-8eba-4432-9a54-8888d00e3fa1+7:2273:x1862456979096320:12345-192.168.204.3@tcp:400:kworker/u8:3.0 00000100:00100000:2.0:1776178176.304419:0:11281:0:(service.c:2372:ptlrpc_server_handle_request()) Handled RPC req@00000000b6612f78 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_004:b50963bd-8eba-4432-9a54-8888d00e3fa1+7:2273:x1862456979096320:12345-192.168.204.3@tcp:400:kworker/u8:3.0 Request processed in 143us (1915us total) trans 0 rc 0/0 00000100:00100000:2.0:1776178176.304429:0:11281:0:(nrs_fifo.c:241:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.204.3@tcp, seq: 867 00000100:00100000:1.0:1776178176.307545:0:3347:0:(events.c:361:request_in_callback()) peer: 12345-192.168.204.3@tcp (source: 12345-192.168.204.3@tcp) 00000100:00100000:2.0:1776178176.308686:0:8227:0:(service.c:2152:ptlrpc_server_handle_req_in()) got req x1862456979096448 00000100:00100000:2.0:1776178176.308771:0:8227:0:(nrs_fifo.c:179:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.204.3@tcp, seq: 483 00000100:00100000:2.0:1776178176.308780:0:8227:0:(service.c:2324:ptlrpc_server_handle_request()) Handling RPC req@000000000b5ad4fd pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_000:b50963bd-8eba-4432-9a54-8888d00e3fa1+8:2273:x1862456979096448:12345-192.168.204.3@tcp:400:kworker/u8:3.0 00000100:00100000:2.0:1776178176.310221:0:8227:0:(service.c:2372:ptlrpc_server_handle_request()) Handled RPC req@000000000b5ad4fd pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_000:b50963bd-8eba-4432-9a54-8888d00e3fa1+8:2273:x1862456979096448:12345-192.168.204.3@tcp:400:kworker/u8:3.0 Request processed in 1427us (2685us total) trans 0 rc 0/0 00000100:00100000:2.0:1776178176.310250:0:8227:0:(nrs_fifo.c:241:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.204.3@tcp, seq: 483 00000100:00100000:0.0:1776178176.353864:0:3354:0:(client.c:1733:ptlrpc_send_new_req()) Sending RPC req@000000004943ee04 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_01:lustre-MDT0001-mdtlov_UUID:3354:1862456998040320:0@lo:13:osp-pre-1-1.0 00000100:00100000:0.0:1776178176.353945:0:3354:0:(events.c:361:request_in_callback()) peer: 12345-0@lo (source: 12345-0@lo) 00000100:00100000:3.0:1776178176.354044:0:8232:0:(service.c:2152:ptlrpc_server_handle_req_in()) got req x1862456998040320 00000100:00100000:3.0:1776178176.354123:0:8232:0:(nrs_fifo.c:179:nrs_fifo_req_get()) NRS start fifo request from 12345-0@lo, seq: 457 00000100:00100000:3.0:1776178176.354129:0:8232:0:(service.c:2324:ptlrpc_server_handle_request()) Handling RPC req@0000000040909b59 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_create00:lustre-MDT0001-mdtlov_UUID+5:3354:x1862456998040320:12345-0@lo:13:osp-pre-1-1.0 00002000:00000024:3.0:1776178176.354152:0:8232:0:(ofd_obd.c:740:ofd_statfs()) blocks cached 229376 granted 491746368 pending 0 free 3923689472 avail 3692167168 00000100:00100000:3.0:1776178176.354303:0:8232:0:(service.c:2372:ptlrpc_server_handle_request()) Handled RPC req@0000000040909b59 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_create00:lustre-MDT0001-mdtlov_UUID+5:3354:x1862456998040320:12345-0@lo:13:osp-pre-1-1.0 Request processed in 172us (359us total) trans 0 rc 0/0 00000100:00100000:3.0:1776178176.354312:0:8232:0:(nrs_fifo.c:241:nrs_fifo_req_stop()) NRS stop fifo request from 12345-0@lo, seq: 457 00000100:00100000:2.0:1776178176.354643:0:3354:0:(client.c:2215:ptlrpc_check_set()) Completed RPC req@000000004943ee04 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_01:lustre-MDT0001-mdtlov_UUID:3354:1862456998040320:0@lo:13:osp-pre-1-1.0 00010000:00010000:2.0:1776178176.416366:0:9225:0:(ldlm_lockd.c:2792:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:2.0:1776178176.865521:0:3355:0:(client.c:1733:ptlrpc_send_new_req()) Sending RPC req@0000000023fb6238 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_02:lustre-MDT0001-mdtlov_UUID:3355:1862456998040384:0@lo:13:osp-pre-0-1.0 00000100:00100000:2.0:1776178176.865635:0:3355:0:(events.c:361:request_in_callback()) peer: 12345-0@lo (source: 12345-0@lo) 00000100:00100000:0.0:1776178176.867394:0:9554:0:(service.c:2152:ptlrpc_server_handle_req_in()) got req x1862456998040384 00000100:00100000:0.0:1776178176.867483:0:9554:0:(nrs_fifo.c:179:nrs_fifo_req_get()) NRS start fifo request from 12345-0@lo, seq: 458 00000100:00100000:0.0:1776178176.867492:0:9554:0:(service.c:2324:ptlrpc_server_handle_request()) Handling RPC req@000000001946bc3d pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_create00:lustre-MDT0001-mdtlov_UUID+5:3355:x1862456998040384:12345-0@lo:13:osp-pre-0-1.0 00002000:00000024:0.0:1776178176.867534:0:9554:0:(ofd_obd.c:740:ofd_statfs()) blocks cached 315392 granted 484883776 pending 0 free 3923689472 avail 3692167168 00000100:00100000:0.0:1776178176.867743:0:9554:0:(service.c:2372:ptlrpc_server_handle_request()) Handled RPC req@000000001946bc3d pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_create00:lustre-MDT0001-mdtlov_UUID+5:3355:x1862456998040384:12345-0@lo:13:osp-pre-0-1.0 Request processed in 250us (2109us total) trans 0 rc 0/0 00000100:00100000:0.0:1776178176.867752:0:9554:0:(nrs_fifo.c:241:nrs_fifo_req_stop()) NRS stop fifo request from 12345-0@lo, seq: 458 00000100:00100000:2.0:1776178176.885340:0:3355:0:(client.c:2215:ptlrpc_check_set()) Completed RPC req@0000000023fb6238 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_02:lustre-MDT0001-mdtlov_UUID:3355:1862456998040384:0@lo:13:osp-pre-0-1.0 00010000:00010000:2.0:1776178177.441675:0:9224:0:(ldlm_lockd.c:2792:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00010000:00010000:1.0:1776178178.464463:0:6245:0:(ldlm_lockd.c:2792:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00010000:00010000:2.0:1776178179.489576:0:6244:0:(ldlm_lockd.c:2792:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:3.0:1776178179.938461:0:3353:0:(client.c:1733:ptlrpc_send_new_req()) Sending RPC req@00000000af9d051c pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_00:lustre-MDT0000-mdtlov_UUID:3353:1862456998040512:0@lo:13:osp-pre-1-0.0 00000100:00100000:3.0:1776178179.938571:0:3353:0:(events.c:361:request_in_callback()) peer: 12345-0@lo (source: 12345-0@lo) 00000100:00100000:3.0:1776178179.938697:0:3355:0:(ptlrpcd.c:415:ptlrpcd_check()) transfer 1 async RPCs [3->2] 00000100:00100000:3.0:1776178179.938707:0:3355:0:(client.c:1733:ptlrpc_send_new_req()) Sending RPC req@00000000b7afa2a2 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_02:lustre-MDT0000-mdtlov_UUID:3355:1862456998040448:0@lo:41:osp-pre-1-0.0 00000100:00100000:3.0:1776178179.938736:0:3355:0:(events.c:361:request_in_callback()) peer: 12345-0@lo (source: 12345-0@lo) 00000100:00100000:3.0:1776178179.938899:0:3354:0:(client.c:1733:ptlrpc_send_new_req()) Sending RPC req@000000007608d8e9 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_01:ac61f839-0e8e-4cac-9c59-08bcfbbad330:3354:1862456998040576:0@lo:400:kworker/u8:0.0 00000100:00100000:3.0:1776178179.938944:0:3354:0:(events.c:361:request_in_callback()) peer: 12345-0@lo (source: 12345-0@lo) 00000100:00100000:3.0:1776178179.938958:0:3354:0:(client.c:1733:ptlrpc_send_new_req()) Sending RPC req@000000004943ee04 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_01:lustre-MDT0001-lwp-OST0001_UUID:3354:1862456998040832:0@lo:400:kworker/u8:0.0 00000100:00100000:3.0:1776178179.938981:0:3354:0:(events.c:361:request_in_callback()) peer: 12345-0@lo (source: 12345-0@lo) 00000100:00100000:3.0:1776178179.938992:0:3354:0:(client.c:1733:ptlrpc_send_new_req()) Sending RPC req@00000000a101cbf3 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_01:lustre-MDT0000-lwp-MDT0001_UUID:3354:1862456998041088:0@lo:400:kworker/u8:0.0 00000100:00100000:3.0:1776178179.939009:0:3354:0:(events.c:361:request_in_callback()) peer: 12345-0@lo (source: 12345-0@lo) 00000100:00100000:3.0:1776178179.939026:0:3354:0:(ptlrpcd.c:415:ptlrpcd_check()) transfer 2 async RPCs [0->1] 00000100:00100000:3.0:1776178179.939031:0:3354:0:(client.c:1733:ptlrpc_send_new_req()) Sending RPC req@000000007794b3ac pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_01:lustre-MDT0000-lwp-OST0001_UUID:3354:1862456998040768:0@lo:400:kworker/u8:0.0 00000100:00100000:3.0:1776178179.939051:0:3354:0:(events.c:361:request_in_callback()) peer: 12345-0@lo (source: 12345-0@lo) 00000100:00100000:3.0:1776178179.939086:0:3354:0:(client.c:1733:ptlrpc_send_new_req()) Sending RPC req@0000000022264af5 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_01:lustre-MDT0001-mdtlov_UUID:3354:1862456998041024:0@lo:400:kworker/u8:0.0 00000100:00100000:3.0:1776178179.939108:0:3354:0:(events.c:361:request_in_callback()) peer: 12345-0@lo (source: 12345-0@lo) 00000100:00100000:3.0:1776178179.939148:0:6257:0:(service.c:2152:ptlrpc_server_handle_req_in()) got req x1862456998040832 00000100:00100000:3.0:1776178179.939206:0:6257:0:(service.c:2152:ptlrpc_server_handle_req_in()) got req x1862456998041088 00000100:00100000:3.0:1776178179.939221:0:6257:0:(service.c:2152:ptlrpc_server_handle_req_in()) got req x1862456998040768 00000100:00100000:3.0:1776178179.939233:0:6257:0:(nrs_fifo.c:179:nrs_fifo_req_get()) NRS start fifo request from 12345-0@lo, seq: 868 00000100:00100000:3.0:1776178179.939239:0:6257:0:(service.c:2324:ptlrpc_server_handle_request()) Handling RPC req@00000000eec4b6d0 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_002:lustre-MDT0001-lwp-OST0001_UUID+5:3354:x1862456998040832:12345-0@lo:400:kworker/u8:0.0 00000100:00100000:3.0:1776178179.939293:0:6257:0:(service.c:2372:ptlrpc_server_handle_request()) Handled RPC req@00000000eec4b6d0 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_002:lustre-MDT0001-lwp-OST0001_UUID+5:3354:x1862456998040832:12345-0@lo:400:kworker/u8:0.0 Request processed in 54us (311us total) trans 0 rc 0/0 00000100:00100000:0.0:1776178179.939581:0:9554:0:(service.c:2152:ptlrpc_server_handle_req_in()) got req x1862456998040512 00000100:00100000:2.0:1776178179.939585:0:11281:0:(nrs_fifo.c:179:nrs_fifo_req_get()) NRS start fifo request from 12345-0@lo, seq: 869 00000100:00100000:2.0:1776178179.939598:0:11281:0:(service.c:2324:ptlrpc_server_handle_request()) Handling RPC req@0000000056a48d5b pname:cluuid+ref:pid:xid:nid:opc:job mdt00_004:lustre-MDT0000-lwp-MDT0001_UUID+5:3354:x1862456998041088:12345-0@lo:400:kworker/u8:0.0 00000100:00100000:0.0:1776178179.939622:0:9554:0:(nrs_fifo.c:179:nrs_fifo_req_get()) NRS start fifo request from 12345-0@lo, seq: 459 00000100:00100000:0.0:1776178179.939630:0:9554:0:(service.c:2324:ptlrpc_server_handle_request()) Handling RPC req@000000006ea63760 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_create00:lustre-MDT0000-mdtlov_UUID+5:3353:x1862456998040512:12345-0@lo:13:osp-pre-1-0.0 00002000:00000024:0.0:1776178179.939674:0:9554:0:(ofd_obd.c:740:ofd_statfs()) blocks cached 229376 granted 491746368 pending 0 free 3923689472 avail 3692167168 00000100:00100000:2.0:1776178179.939712:0:11281:0:(service.c:2372:ptlrpc_server_handle_request()) Handled RPC req@0000000056a48d5b pname:cluuid+ref:pid:xid:nid:opc:job mdt00_004:lustre-MDT0000-lwp-MDT0001_UUID+5:3354:x1862456998041088:12345-0@lo:400:kworker/u8:0.0 Request processed in 114us (701us total) trans 0 rc 0/0 00000100:00100000:2.0:1776178179.939724:0:11281:0:(nrs_fifo.c:241:nrs_fifo_req_stop()) NRS stop fifo request from 12345-0@lo, seq: 869 00000100:00100000:0.0:1776178179.939743:0:9554:0:(service.c:2372:ptlrpc_server_handle_request()) Handled RPC req@000000006ea63760 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_create00:lustre-MDT0000-mdtlov_UUID+5:3353:x1862456998040512:12345-0@lo:13:osp-pre-1-0.0 Request processed in 112us (1172us total) trans 0 rc 0/0 00000100:00100000:0.0:1776178179.939761:0:9554:0:(nrs_fifo.c:241:nrs_fifo_req_stop()) NRS stop fifo request from 12345-0@lo, seq: 459 00000100:00100000:2.0:1776178179.939771:0:11281:0:(nrs_fifo.c:179:nrs_fifo_req_get()) NRS start fifo request from 12345-0@lo, seq: 870 00000100:00100000:2.0:1776178179.939778:0:11281:0:(service.c:2324:ptlrpc_server_handle_request()) Handling RPC req@0000000020763766 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_004:lustre-MDT0000-lwp-OST0001_UUID+5:3354:x1862456998040768:12345-0@lo:400:kworker/u8:0.0 00000100:00100000:0.0:1776178179.939801:0:15213:0:(service.c:2152:ptlrpc_server_handle_req_in()) got req x1862456998040448 00000100:00100000:0.0:1776178179.939830:0:15213:0:(nrs_fifo.c:179:nrs_fifo_req_get()) NRS start fifo request from 12345-0@lo, seq: 676 00000100:00100000:0.0:1776178179.939835:0:15213:0:(service.c:2324:ptlrpc_server_handle_request()) Handling RPC req@00000000b26c52d3 pname:cluuid+ref:pid:xid:nid:opc:job mdt_out00_003:lustre-MDT0000-mdtlov_UUID+5:3355:x1862456998040448:12345-0@lo:41:osp-pre-1-0.0 00000004:00000024:0.0:1776178179.939860:0:15213:0:(mdt_handler.c:548:mdt_statfs()) blocks cached 0 granted 2097152 pending 0 free 1446035456 avail 1316573184 00000100:00100000:0.0:1776178179.939891:0:15213:0:(service.c:2372:ptlrpc_server_handle_request()) Handled RPC req@00000000b26c52d3 pname:cluuid+ref:pid:xid:nid:opc:job mdt_out00_003:lustre-MDT0000-mdtlov_UUID+5:3355:x1862456998040448:12345-0@lo:41:osp-pre-1-0.0 Request processed in 55us (1155us total) trans 0 rc 0/0 00000100:00100000:0.0:1776178179.939897:0:15213:0:(nrs_fifo.c:241:nrs_fifo_req_stop()) NRS stop fifo request from 12345-0@lo, seq: 676 00000100:00100000:0.0:1776178179.939915:0:20140:0:(service.c:2152:ptlrpc_server_handle_req_in()) got req x1862456998040576 00000100:00100000:0.0:1776178179.939936:0:20140:0:(nrs_fifo.c:179:nrs_fifo_req_get()) NRS start fifo request from 12345-0@lo, seq: 146 00000100:00100000:0.0:1776178179.939939:0:20140:0:(service.c:2324:ptlrpc_server_handle_request()) Handling RPC req@00000000b59e728a pname:cluuid+ref:pid:xid:nid:opc:job ll_mgs_0002:ac61f839-0e8e-4cac-9c59-08bcfbbad330+23:3354:x1862456998040576:12345-0@lo:400:kworker/u8:0.0 00000100:00100000:0.0:1776178179.939963:0:20140:0:(service.c:2372:ptlrpc_server_handle_request()) Handled RPC req@00000000b59e728a pname:cluuid+ref:pid:xid:nid:opc:job ll_mgs_0002:ac61f839-0e8e-4cac-9c59-08bcfbbad330+23:3354:x1862456998040576:12345-0@lo:400:kworker/u8:0.0 Request processed in 23us (1020us total) trans 0 rc 0/0 00000100:00100000:0.0:1776178179.939968:0:20140:0:(nrs_fifo.c:241:nrs_fifo_req_stop()) NRS stop fifo request from 12345-0@lo, seq: 146 00000100:00100000:0.0:1776178179.940026:0:3356:0:(client.c:1733:ptlrpc_send_new_req()) Sending RPC req@00000000660d337f pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_03:lustre-MDT0001-lwp-OST0000_UUID:3356:1862456998040704:0@lo:400:kworker/u8:0.0 00000100:00100000:0.0:1776178179.940049:0:3356:0:(events.c:361:request_in_callback()) peer: 12345-0@lo (source: 12345-0@lo) 00000100:00100000:0.0:1776178179.940079:0:3356:0:(client.c:1733:ptlrpc_send_new_req()) Sending RPC req@000000007c827612 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_03:lustre-MDT0001-mdtlov_UUID:3356:1862456998040960:0@lo:400:kworker/u8:0.0 00000100:00100000:0.0:1776178179.940097:0:3356:0:(events.c:361:request_in_callback()) peer: 12345-0@lo (source: 12345-0@lo) 00000100:00100000:0.0:1776178179.940109:0:3356:0:(client.c:1733:ptlrpc_send_new_req()) Sending RPC req@000000009b04a931 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_03:lustre-MDT0000-lwp-MDT0000_UUID:3356:1862456998041216:0@lo:400:kworker/u8:0.0 00000100:00100000:0.0:1776178179.940126:0:3356:0:(events.c:361:request_in_callback()) peer: 12345-0@lo (source: 12345-0@lo) 00000100:00100000:0.0:1776178179.940140:0:3356:0:(ptlrpcd.c:415:ptlrpcd_check()) transfer 3 async RPCs [2->3] 00000100:00100000:0.0:1776178179.940144:0:3356:0:(client.c:1733:ptlrpc_send_new_req()) Sending RPC req@00000000f52e98d8 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_03:lustre-MDT0000-lwp-OST0000_UUID:3356:1862456998040640:0@lo:400:kworker/u8:0.0 00000100:00100000:0.0:1776178179.940170:0:3356:0:(events.c:361:request_in_callback()) peer: 12345-0@lo (source: 12345-0@lo) 00000100:00100000:0.0:1776178179.940179:0:3356:0:(client.c:1733:ptlrpc_send_new_req()) Sending RPC req@000000003fa4e1db pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_03:lustre-MDT0001-mdtlov_UUID:3356:1862456998040896:0@lo:400:kworker/u8:0.0 00000100:00100000:0.0:1776178179.940198:0:3356:0:(events.c:361:request_in_callback()) peer: 12345-0@lo (source: 12345-0@lo) 00000100:00100000:0.0:1776178179.940209:0:3356:0:(client.c:1733:ptlrpc_send_new_req()) Sending RPC req@0000000014e690d4 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_03:lustre-MDT0000-mdtlov_UUID:3356:1862456998041152:0@lo:400:kworker/u8:0.0 00000100:00100000:0.0:1776178179.940227:0:3356:0:(events.c:361:request_in_callback()) peer: 12345-0@lo (source: 12345-0@lo) 00000100:00100000:0.0:1776178179.940252:0:20755:0:(service.c:2152:ptlrpc_server_handle_req_in()) got req x1862456998041024 00000100:00100000:0.0:1776178179.940275:0:20755:0:(service.c:2152:ptlrpc_server_handle_req_in()) got req x1862456998040960 00000100:00100000:0.0:1776178179.940290:0:20755:0:(service.c:2152:ptlrpc_server_handle_req_in()) got req x1862456998041152 00000100:00100000:0.0:1776178179.940318:0:20755:0:(nrs_fifo.c:179:nrs_fifo_req_get()) NRS start fifo request from 12345-0@lo, seq: 484 00000100:00100000:0.0:1776178179.940322:0:20755:0:(service.c:2324:ptlrpc_server_handle_request()) Handling RPC req@00000000a979415d pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_006:lustre-MDT0001-mdtlov_UUID+5:3354:x1862456998041024:12345-0@lo:400:kworker/u8:0.0 00000100:00100000:0.0:1776178179.940347:0:20755:0:(service.c:2372:ptlrpc_server_handle_request()) Handled RPC req@00000000a979415d pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_006:lustre-MDT0001-mdtlov_UUID+5:3354:x1862456998041024:12345-0@lo:400:kworker/u8:0.0 Request processed in 25us (1239us total) trans 0 rc 0/0 00000100:00100000:0.0:1776178179.940357:0:20755:0:(nrs_fifo.c:241:nrs_fifo_req_stop()) NRS stop fifo request from 12345-0@lo, seq: 484 00000100:00100000:0.0:1776178179.940369:0:20755:0:(nrs_fifo.c:179:nrs_fifo_req_get()) NRS start fifo request from 12345-0@lo, seq: 485 00000100:00100000:0.0:1776178179.940372:0:20755:0:(service.c:2324:ptlrpc_server_handle_request()) Handling RPC req@00000000792dcbe4 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_006:lustre-MDT0001-mdtlov_UUID+5:3356:x1862456998040960:12345-0@lo:400:kworker/u8:0.0 00000100:00100000:0.0:1776178179.940394:0:20755:0:(service.c:2372:ptlrpc_server_handle_request()) Handled RPC req@00000000792dcbe4 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_006:lustre-MDT0001-mdtlov_UUID+5:3356:x1862456998040960:12345-0@lo:400:kworker/u8:0.0 Request processed in 21us (297us total) trans 0 rc 0/0 00000100:00100000:0.0:1776178179.940400:0:20755:0:(nrs_fifo.c:241:nrs_fifo_req_stop()) NRS stop fifo request from 12345-0@lo, seq: 485 00000100:00100000:0.0:1776178179.940410:0:20755:0:(nrs_fifo.c:179:nrs_fifo_req_get()) NRS start fifo request from 12345-0@lo, seq: 486 00000100:00100000:0.0:1776178179.940413:0:20755:0:(service.c:2324:ptlrpc_server_handle_request()) Handling RPC req@00000000b3ddbcd2 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_006:lustre-MDT0000-mdtlov_UUID+5:3356:x1862456998041152:12345-0@lo:400:kworker/u8:0.0 00000100:00100000:0.0:1776178179.940432:0:20755:0:(service.c:2372:ptlrpc_server_handle_request()) Handled RPC req@00000000b3ddbcd2 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_006:lustre-MDT0000-mdtlov_UUID+5:3356:x1862456998041152:12345-0@lo:400:kworker/u8:0.0 Request processed in 18us (206us total) trans 0 rc 0/0 00000100:00100000:0.0:1776178179.940438:0:20755:0:(nrs_fifo.c:241:nrs_fifo_req_stop()) NRS stop fifo request from 12345-0@lo, seq: 486 00000100:00100000:0.0:1776178179.940455:0:8250:0:(service.c:2152:ptlrpc_server_handle_req_in()) got req x1862456998040704 00000100:00100000:0.0:1776178179.940468:0:8250:0:(service.c:2152:ptlrpc_server_handle_req_in()) got req x1862456998041216 00000100:00100000:0.0:1776178179.940489:0:8250:0:(service.c:2152:ptlrpc_server_handle_req_in()) got req x1862456998040640 00000100:00100000:0.0:1776178179.940498:0:8250:0:(nrs_fifo.c:179:nrs_fifo_req_get()) NRS start fifo request from 12345-0@lo, seq: 871 00000100:00100000:0.0:1776178179.940501:0:8250:0:(service.c:2324:ptlrpc_server_handle_request()) Handling RPC req@00000000a1c5224b pname:cluuid+ref:pid:xid:nid:opc:job mdt00_003:lustre-MDT0001-lwp-OST0000_UUID+5:3356:x1862456998040704:12345-0@lo:400:kworker/u8:0.0 00000100:00100000:0.0:1776178179.940524:0:8250:0:(service.c:2372:ptlrpc_server_handle_request()) Handled RPC req@00000000a1c5224b pname:cluuid+ref:pid:xid:nid:opc:job mdt00_003:lustre-MDT0001-lwp-OST0000_UUID+5:3356:x1862456998040704:12345-0@lo:400:kworker/u8:0.0 Request processed in 22us (475us total) trans 0 rc 0/0 00000100:00100000:0.0:1776178179.940529:0:8250:0:(nrs_fifo.c:241:nrs_fifo_req_stop()) NRS stop fifo request from 12345-0@lo, seq: 871 00000100:00100000:0.0:1776178179.940537:0:8250:0:(nrs_fifo.c:179:nrs_fifo_req_get()) NRS start fifo request from 12345-0@lo, seq: 872 00000100:00100000:0.0:1776178179.940540:0:8250:0:(service.c:2324:ptlrpc_server_handle_request()) Handling RPC req@00000000cdd7022b pname:cluuid+ref:pid:xid:nid:opc:job mdt00_003:lustre-MDT0000-lwp-MDT0000_UUID+5:3356:x1862456998041216:12345-0@lo:400:kworker/u8:0.0 00000100:00100000:0.0:1776178179.940561:0:8250:0:(service.c:2372:ptlrpc_server_handle_request()) Handled RPC req@00000000cdd7022b pname:cluuid+ref:pid:xid:nid:opc:job mdt00_003:lustre-MDT0000-lwp-MDT0000_UUID+5:3356:x1862456998041216:12345-0@lo:400:kworker/u8:0.0 Request processed in 20us (434us total) trans 0 rc 0/0 00000100:00100000:0.0:1776178179.940566:0:8250:0:(nrs_fifo.c:241:nrs_fifo_req_stop()) NRS stop fifo request from 12345-0@lo, seq: 872 00000100:00100000:0.0:1776178179.940575:0:8250:0:(nrs_fifo.c:179:nrs_fifo_req_get()) NRS start fifo request from 12345-0@lo, seq: 873 00000100:00100000:0.0:1776178179.940580:0:8250:0:(service.c:2324:ptlrpc_server_handle_request()) Handling RPC req@000000006e3f7b7f pname:cluuid+ref:pid:xid:nid:opc:job mdt00_003:lustre-MDT0000-lwp-OST0000_UUID+5:3356:x1862456998040640:12345-0@lo:400:kworker/u8:0.0 00000100:00100000:0.0:1776178179.940599:0:8250:0:(service.c:2372:ptlrpc_server_handle_request()) Handled RPC req@000000006e3f7b7f pname:cluuid+ref:pid:xid:nid:opc:job mdt00_003:lustre-MDT0000-lwp-OST0000_UUID+5:3356:x1862456998040640:12345-0@lo:400:kworker/u8:0.0 Request processed in 19us (428us total) trans 0 rc 0/0 00000100:00100000:0.0:1776178179.940604:0:8250:0:(nrs_fifo.c:241:nrs_fifo_req_stop()) NRS stop fifo request from 12345-0@lo, seq: 873 00000100:00100000:0.0:1776178179.940642:0:3355:0:(client.c:2215:ptlrpc_check_set()) Completed RPC req@00000000b7afa2a2 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_02:lustre-MDT0000-mdtlov_UUID:3355:1862456998040448:0@lo:41:osp-pre-1-0.0 00000100:00100000:0.0:1776178179.940665:0:15213:0:(service.c:2152:ptlrpc_server_handle_req_in()) got req x1862456998040896 00000100:00100000:0.0:1776178179.940681:0:15213:0:(nrs_fifo.c:179:nrs_fifo_req_get()) NRS start fifo request from 12345-0@lo, seq: 677 00000100:00100000:0.0:1776178179.940684:0:15213:0:(service.c:2324:ptlrpc_server_handle_request()) Handling RPC req@0000000023fb6238 pname:cluuid+ref:pid:xid:nid:opc:job mdt_out00_003:lustre-MDT0001-mdtlov_UUID+6:3356:x1862456998040896:12345-0@lo:400:kworker/u8:0.0 00000100:00100000:0.0:1776178179.940711:0:15213:0:(service.c:2372:ptlrpc_server_handle_request()) Handled RPC req@0000000023fb6238 pname:cluuid+ref:pid:xid:nid:opc:job mdt_out00_003:lustre-MDT0001-mdtlov_UUID+6:3356:x1862456998040896:12345-0@lo:400:kworker/u8:0.0 Request processed in 26us (511us total) trans 0 rc 0/0 00000100:00100000:0.0:1776178179.940718:0:15213:0:(nrs_fifo.c:241:nrs_fifo_req_stop()) NRS stop fifo request from 12345-0@lo, seq: 677 00000100:00100000:0.0:1776178179.940755:0:3354:0:(client.c:2215:ptlrpc_check_set()) Completed RPC req@0000000022264af5 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_01:lustre-MDT0001-mdtlov_UUID:3354:1862456998041024:0@lo:400:kworker/u8:0.0 00000100:00100000:2.0:1776178179.940764:0:11281:0:(service.c:2372:ptlrpc_server_handle_request()) Handled RPC req@0000000020763766 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_004:lustre-MDT0000-lwp-OST0001_UUID+5:3354:x1862456998040768:12345-0@lo:400:kworker/u8:0.0 Request processed in 981us (1707us total) trans 0 rc 0/0 00000100:00100000:0.0:1776178179.940771:0:3354:0:(client.c:2215:ptlrpc_check_set()) Completed RPC req@000000007608d8e9 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_01:ac61f839-0e8e-4cac-9c59-08bcfbbad330:3354:1862456998040576:0@lo:400:kworker/u8:0.0 00000100:00100000:2.0:1776178179.940782:0:11281:0:(nrs_fifo.c:241:nrs_fifo_req_stop()) NRS stop fifo request from 12345-0@lo, seq: 870 00000100:00100000:0.0:1776178179.940794:0:3354:0:(client.c:2215:ptlrpc_check_set()) Completed RPC req@000000004943ee04 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_01:lustre-MDT0001-lwp-OST0001_UUID:3354:1862456998040832:0@lo:400:kworker/u8:0.0 00000100:00100000:0.0:1776178179.940808:0:3354:0:(client.c:2215:ptlrpc_check_set()) Completed RPC req@00000000a101cbf3 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_01:lustre-MDT0000-lwp-MDT0001_UUID:3354:1862456998041088:0@lo:400:kworker/u8:0.0 00000100:00100000:0.0:1776178179.940849:0:3354:0:(ptlrpcd.c:415:ptlrpcd_check()) transfer 1 async RPCs [0->1] 00000100:00100000:0.0:1776178179.940860:0:3354:0:(client.c:1733:ptlrpc_send_new_req()) Sending RPC req@00000000226852f3 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_01:lustre-MDT0001-mdtlov_UUID:3354:1862456998041280:0@lo:41:osp-pre-0-1.0 00000100:00100000:0.0:1776178179.940891:0:3354:0:(events.c:361:request_in_callback()) peer: 12345-0@lo (source: 12345-0@lo) 00000100:00100000:0.0:1776178179.940925:0:3354:0:(client.c:2215:ptlrpc_check_set()) Completed RPC req@000000007794b3ac pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_01:lustre-MDT0000-lwp-OST0001_UUID:3354:1862456998040768:0@lo:400:kworker/u8:0.0 00000100:00100000:2.0:1776178179.940956:0:6261:0:(service.c:2152:ptlrpc_server_handle_req_in()) got req x1862456998041280 00000100:00100000:0.0:1776178179.940961:0:3356:0:(client.c:2215:ptlrpc_check_set()) Completed RPC req@00000000f52e98d8 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_03:lustre-MDT0000-lwp-OST0000_UUID:3356:1862456998040640:0@lo:400:kworker/u8:0.0 00000100:00100000:0.0:1776178179.940971:0:3356:0:(client.c:2215:ptlrpc_check_set()) Completed RPC req@000000003fa4e1db pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_03:lustre-MDT0001-mdtlov_UUID:3356:1862456998040896:0@lo:400:kworker/u8:0.0 00000100:00100000:0.0:1776178179.940984:0:3356:0:(client.c:2215:ptlrpc_check_set()) Completed RPC req@0000000014e690d4 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_03:lustre-MDT0000-mdtlov_UUID:3356:1862456998041152:0@lo:400:kworker/u8:0.0 00000100:00100000:0.0:1776178179.940995:0:3356:0:(client.c:2215:ptlrpc_check_set()) Completed RPC req@00000000660d337f pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_03:lustre-MDT0001-lwp-OST0000_UUID:3356:1862456998040704:0@lo:400:kworker/u8:0.0 00000100:00100000:2.0:1776178179.941011:0:6261:0:(nrs_fifo.c:179:nrs_fifo_req_get()) NRS start fifo request from 12345-0@lo, seq: 678 00000100:00100000:0.0:1776178179.941015:0:3356:0:(client.c:2215:ptlrpc_check_set()) Completed RPC req@000000007c827612 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_03:lustre-MDT0001-mdtlov_UUID:3356:1862456998040960:0@lo:400:kworker/u8:0.0 00000100:00100000:2.0:1776178179.941022:0:6261:0:(service.c:2324:ptlrpc_server_handle_request()) Handling RPC req@00000000ebb70e84 pname:cluuid+ref:pid:xid:nid:opc:job mdt_out00_001:lustre-MDT0001-mdtlov_UUID+6:3354:x1862456998041280:12345-0@lo:41:osp-pre-0-1.0 00000100:00100000:0.0:1776178179.941031:0:3356:0:(client.c:2215:ptlrpc_check_set()) Completed RPC req@000000009b04a931 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_03:lustre-MDT0000-lwp-MDT0000_UUID:3356:1862456998041216:0@lo:400:kworker/u8:0.0 00000004:00000024:2.0:1776178179.941087:0:6261:0:(mdt_handler.c:548:mdt_statfs()) blocks cached 0 granted 2097152 pending 0 free 1446121472 avail 1316659200 00000100:00100000:0.0:1776178179.941118:0:3353:0:(client.c:2215:ptlrpc_check_set()) Completed RPC req@00000000af9d051c pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_00:lustre-MDT0000-mdtlov_UUID:3353:1862456998040512:0@lo:13:osp-pre-1-0.0 00000100:00100000:2.0:1776178179.941152:0:6261:0:(service.c:2372:ptlrpc_server_handle_request()) Handled RPC req@00000000ebb70e84 pname:cluuid+ref:pid:xid:nid:opc:job mdt_out00_001:lustre-MDT0001-mdtlov_UUID+6:3354:x1862456998041280:12345-0@lo:41:osp-pre-0-1.0 Request processed in 132us (260us total) trans 0 rc 0/0 00000100:00100000:2.0:1776178179.941173:0:6261:0:(nrs_fifo.c:241:nrs_fifo_req_stop()) NRS stop fifo request from 12345-0@lo, seq: 678 00000100:00100000:0.0:1776178179.941211:0:3354:0:(client.c:2215:ptlrpc_check_set()) Completed RPC req@00000000226852f3 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_01:lustre-MDT0001-mdtlov_UUID:3354:1862456998041280:0@lo:41:osp-pre-0-1.0 00000100:00100000:3.0:1776178179.942031:0:6257:0:(nrs_fifo.c:241:nrs_fifo_req_stop()) NRS stop fifo request from 12345-0@lo, seq: 868 00000100:00100000:0.0:1776178179.943701:0:3354:0:(client.c:1733:ptlrpc_send_new_req()) Sending RPC req@0000000044f82452 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_01:lustre-MDT0000-mdtlov_UUID:3354:1862456998041344:0@lo:13:osp-pre-0-0.0 00000100:00100000:0.0:1776178179.943791:0:3354:0:(events.c:361:request_in_callback()) peer: 12345-0@lo (source: 12345-0@lo) 00000100:00100000:1.0:1776178179.966499:0:8232:0:(service.c:2152:ptlrpc_server_handle_req_in()) got req x1862456998041344 00000100:00100000:1.0:1776178179.966579:0:8232:0:(nrs_fifo.c:179:nrs_fifo_req_get()) NRS start fifo request from 12345-0@lo, seq: 460 00000100:00100000:1.0:1776178179.966587:0:8232:0:(service.c:2324:ptlrpc_server_handle_request()) Handling RPC req@00000000af9d051c pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_create00:lustre-MDT0000-mdtlov_UUID+5:3354:x1862456998041344:12345-0@lo:13:osp-pre-0-0.0 00002000:00000024:1.0:1776178179.966628:0:8232:0:(ofd_obd.c:740:ofd_statfs()) blocks cached 315392 granted 484883776 pending 0 free 3923689472 avail 3692167168 00000100:00100000:1.0:1776178179.966759:0:8232:0:(service.c:2372:ptlrpc_server_handle_request()) Handled RPC req@00000000af9d051c pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_create00:lustre-MDT0000-mdtlov_UUID+5:3354:x1862456998041344:12345-0@lo:13:osp-pre-0-0.0 Request processed in 172us (22969us total) trans 0 rc 0/0 00000100:00100000:1.0:1776178179.966767:0:8232:0:(nrs_fifo.c:241:nrs_fifo_req_stop()) NRS stop fifo request from 12345-0@lo, seq: 460 00000100:00100000:2.0:1776178179.969236:0:3354:0:(client.c:2215:ptlrpc_check_set()) Completed RPC req@0000000044f82452 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_01:lustre-MDT0000-mdtlov_UUID:3354:1862456998041344:0@lo:13:osp-pre-0-0.0 00010000:00010000:2.0:1776178180.513221:0:9225:0:(ldlm_lockd.c:2792:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:3.0:1776178181.416180:0:3346:0:(events.c:361:request_in_callback()) peer: 12345-192.168.204.3@tcp (source: 12345-192.168.204.3@tcp) 00000100:00100000:3.0:1776178181.416232:0:3346:0:(events.c:361:request_in_callback()) peer: 12345-192.168.204.3@tcp (source: 12345-192.168.204.3@tcp) 00000100:00100000:3.0:1776178181.416264:0:3346:0:(events.c:361:request_in_callback()) peer: 12345-192.168.204.3@tcp (source: 12345-192.168.204.3@tcp) 00000100:00100000:3.0:1776178181.416311:0:3346:0:(events.c:361:request_in_callback()) peer: 12345-192.168.204.3@tcp (source: 12345-192.168.204.3@tcp) 00000100:00100000:3.0:1776178181.416387:0:3346:0:(events.c:361:request_in_callback()) peer: 12345-192.168.204.3@tcp (source: 12345-192.168.204.3@tcp) 00000100:00100000:3.0:1776178181.416425:0:20762:0:(service.c:2152:ptlrpc_server_handle_req_in()) got req x1862456979096832 00000100:00100000:3.0:1776178181.416472:0:20762:0:(service.c:2152:ptlrpc_server_handle_req_in()) got req x1862456979096768 00000100:00100000:1.0:1776178181.416548:0:20137:0:(service.c:2152:ptlrpc_server_handle_req_in()) got req x1862456979096576 00000100:00100000:1.0:1776178181.416580:0:20137:0:(nrs_fifo.c:179:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.204.3@tcp, seq: 147 00000100:00100000:1.0:1776178181.416589:0:20137:0:(service.c:2324:ptlrpc_server_handle_request()) Handling RPC req@00000000dd6bfaa2 pname:cluuid+ref:pid:xid:nid:opc:job ll_mgs_0000:5b5c66b1-1a2f-4651-8548-0c75b9a38a94+9:2273:x1862456979096576:12345-192.168.204.3@tcp:400:kworker/u8:3.0 00000100:00100000:1.0:1776178181.416688:0:20137:0:(service.c:2372:ptlrpc_server_handle_request()) Handled RPC req@00000000dd6bfaa2 pname:cluuid+ref:pid:xid:nid:opc:job ll_mgs_0000:5b5c66b1-1a2f-4651-8548-0c75b9a38a94+9:2273:x1862456979096576:12345-192.168.204.3@tcp:400:kworker/u8:3.0 Request processed in 99us (457us total) trans 0 rc 0/0 00000100:00100000:1.0:1776178181.416696:0:20137:0:(nrs_fifo.c:241:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.204.3@tcp, seq: 147 00000100:00100000:0.0:1776178181.417154:0:6257:0:(service.c:2152:ptlrpc_server_handle_req_in()) got req x1862456979096640 00000100:00100000:0.0:1776178181.417195:0:6257:0:(service.c:2152:ptlrpc_server_handle_req_in()) got req x1862456979096704 00000100:00100000:0.0:1776178181.417218:0:6257:0:(nrs_fifo.c:179:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.204.3@tcp, seq: 874 00000100:00100000:0.0:1776178181.417223:0:6257:0:(service.c:2324:ptlrpc_server_handle_request()) Handling RPC req@000000009b04a931 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_002:b50963bd-8eba-4432-9a54-8888d00e3fa1+7:2272:x1862456979096640:12345-192.168.204.3@tcp:400:kworker/u8:3.0 00000100:00100000:0.0:1776178181.417266:0:6257:0:(service.c:2372:ptlrpc_server_handle_request()) Handled RPC req@000000009b04a931 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_002:b50963bd-8eba-4432-9a54-8888d00e3fa1+7:2272:x1862456979096640:12345-192.168.204.3@tcp:400:kworker/u8:3.0 Request processed in 43us (1095us total) trans 0 rc 0/0 00000100:00100000:0.0:1776178181.417274:0:6257:0:(nrs_fifo.c:241:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.204.3@tcp, seq: 874 00000100:00100000:0.0:1776178181.417290:0:6257:0:(nrs_fifo.c:179:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.204.3@tcp, seq: 875 00000100:00100000:0.0:1776178181.417294:0:6257:0:(service.c:2324:ptlrpc_server_handle_request()) Handling RPC req@000000007c827612 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_002:b50963bd-8eba-4432-9a54-8888d00e3fa1+6:2273:x1862456979096704:12345-192.168.204.3@tcp:400:kworker/u8:3.0 00000100:00100000:0.0:1776178181.417332:0:6257:0:(service.c:2372:ptlrpc_server_handle_request()) Handled RPC req@000000007c827612 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_002:b50963bd-8eba-4432-9a54-8888d00e3fa1+6:2273:x1862456979096704:12345-192.168.204.3@tcp:400:kworker/u8:3.0 Request processed in 37us (1068us total) trans 0 rc 0/0 00000100:00100000:0.0:1776178181.417339:0:6257:0:(nrs_fifo.c:241:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.204.3@tcp, seq: 875 00000100:00100000:3.0:1776178181.417589:0:20762:0:(nrs_fifo.c:179:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.204.3@tcp, seq: 487 00000100:00100000:3.0:1776178181.417596:0:20762:0:(service.c:2324:ptlrpc_server_handle_request()) Handling RPC req@0000000014e690d4 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_013:b50963bd-8eba-4432-9a54-8888d00e3fa1+9:2273:x1862456979096832:12345-192.168.204.3@tcp:400:kworker/u8:3.0 00000100:00100000:3.0:1776178181.419645:0:20762:0:(service.c:2372:ptlrpc_server_handle_request()) Handled RPC req@0000000014e690d4 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_013:b50963bd-8eba-4432-9a54-8888d00e3fa1+9:2273:x1862456979096832:12345-192.168.204.3@tcp:400:kworker/u8:3.0 Request processed in 2041us (3327us total) trans 0 rc 0/0 00000100:00100000:3.0:1776178181.419658:0:20762:0:(nrs_fifo.c:241:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.204.3@tcp, seq: 487 00000100:00100000:3.0:1776178181.419692:0:20762:0:(nrs_fifo.c:179:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.204.3@tcp, seq: 488 00000100:00100000:3.0:1776178181.419699:0:20762:0:(service.c:2324:ptlrpc_server_handle_request()) Handling RPC req@00000000a101cbf3 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_013:b50963bd-8eba-4432-9a54-8888d00e3fa1+8:2272:x1862456979096768:12345-192.168.204.3@tcp:400:kworker/u8:3.0 00000100:00100000:3.0:1776178181.419749:0:20762:0:(service.c:2372:ptlrpc_server_handle_request()) Handled RPC req@00000000a101cbf3 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_013:b50963bd-8eba-4432-9a54-8888d00e3fa1+8:2272:x1862456979096768:12345-192.168.204.3@tcp:400:kworker/u8:3.0 Request processed in 51us (3363us total) trans 0 rc 0/0 00000100:00100000:3.0:1776178181.419754:0:20762:0:(nrs_fifo.c:241:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.204.3@tcp, seq: 488 00000100:00100000:0.0:1776178181.473428:0:3355:0:(client.c:1733:ptlrpc_send_new_req()) Sending RPC req@0000000022264af5 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_02:lustre-MDT0001-mdtlov_UUID:3355:1862456998041408:0@lo:13:osp-pre-1-1.0 00000100:00100000:0.0:1776178181.473514:0:3355:0:(events.c:361:request_in_callback()) peer: 12345-0@lo (source: 12345-0@lo) 00000100:00100000:3.0:1776178181.474360:0:9554:0:(service.c:2152:ptlrpc_server_handle_req_in()) got req x1862456998041408 00000100:00100000:3.0:1776178181.474429:0:9554:0:(nrs_fifo.c:179:nrs_fifo_req_get()) NRS start fifo request from 12345-0@lo, seq: 461 00000100:00100000:3.0:1776178181.474435:0:9554:0:(service.c:2324:ptlrpc_server_handle_request()) Handling RPC req@00000000737321c9 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_create00:lustre-MDT0001-mdtlov_UUID+5:3355:x1862456998041408:12345-0@lo:13:osp-pre-1-1.0 00002000:00000024:3.0:1776178181.474469:0:9554:0:(ofd_obd.c:740:ofd_statfs()) blocks cached 229376 granted 491746368 pending 0 free 3923689472 avail 3692167168 00000100:00100000:3.0:1776178181.474527:0:9554:0:(service.c:2372:ptlrpc_server_handle_request()) Handled RPC req@00000000737321c9 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_create00:lustre-MDT0001-mdtlov_UUID+5:3355:x1862456998041408:12345-0@lo:13:osp-pre-1-1.0 Request processed in 92us (1015us total) trans 0 rc 0/0 00000100:00100000:3.0:1776178181.474534:0:9554:0:(nrs_fifo.c:241:nrs_fifo_req_stop()) NRS stop fifo request from 12345-0@lo, seq: 461 00000100:00100000:0.0:1776178181.474817:0:3355:0:(client.c:2215:ptlrpc_check_set()) Completed RPC req@0000000022264af5 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_02:lustre-MDT0001-mdtlov_UUID:3355:1862456998041408:0@lo:13:osp-pre-1-1.0 00010000:00010000:2.0:1776178181.536264:0:9224:0:(ldlm_lockd.c:2792:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:1.0:1776178181.984397:0:3356:0:(client.c:1733:ptlrpc_send_new_req()) Sending RPC req@000000004ee84694 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_03:lustre-MDT0001-mdtlov_UUID:3356:1862456998041472:0@lo:13:osp-pre-0-1.0 00000100:00100000:1.0:1776178181.984479:0:3356:0:(events.c:361:request_in_callback()) peer: 12345-0@lo (source: 12345-0@lo) 00000100:00100000:2.0:1776178181.984715:0:8232:0:(service.c:2152:ptlrpc_server_handle_req_in()) got req x1862456998041472 00000100:00100000:2.0:1776178181.984780:0:8232:0:(nrs_fifo.c:179:nrs_fifo_req_get()) NRS start fifo request from 12345-0@lo, seq: 462 00000100:00100000:2.0:1776178181.984786:0:8232:0:(service.c:2324:ptlrpc_server_handle_request()) Handling RPC req@00000000fab910b9 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_create00:lustre-MDT0001-mdtlov_UUID+5:3356:x1862456998041472:12345-0@lo:13:osp-pre-0-1.0 00002000:00000024:2.0:1776178181.984815:0:8232:0:(ofd_obd.c:740:ofd_statfs()) blocks cached 315392 granted 484883776 pending 0 free 3923689472 avail 3692167168 00000100:00100000:2.0:1776178181.991521:0:8232:0:(service.c:2372:ptlrpc_server_handle_request()) Handled RPC req@00000000fab910b9 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_create00:lustre-MDT0001-mdtlov_UUID+5:3356:x1862456998041472:12345-0@lo:13:osp-pre-0-1.0 Request processed in 6725us (7035us total) trans 0 rc 0/0 00000100:00100000:2.0:1776178181.991539:0:8232:0:(nrs_fifo.c:241:nrs_fifo_req_stop()) NRS stop fifo request from 12345-0@lo, seq: 462 00000100:00100000:0.0:1776178182.001856:0:3356:0:(client.c:2215:ptlrpc_check_set()) Completed RPC req@000000004ee84694 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_03:lustre-MDT0001-mdtlov_UUID:3356:1862456998041472:0@lo:13:osp-pre-0-1.0 00010000:00010000:1.0:1776178182.563088:0:6245:0:(ldlm_lockd.c:2792:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00010000:00010000:2.0:1776178183.586170:0:6244:0:(ldlm_lockd.c:2792:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00010000:00010000:2.0:1776178184.683414:0:9225:0:(ldlm_lockd.c:2792:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:3.0:1776178185.065784:0:3353:0:(client.c:1733:ptlrpc_send_new_req()) Sending RPC req@0000000022264af5 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_00:lustre-MDT0000-mdtlov_UUID:3353:1862456998041536:0@lo:13:osp-pre-1-0.0 00000100:00100000:3.0:1776178185.065861:0:3353:0:(events.c:361:request_in_callback()) peer: 12345-0@lo (source: 12345-0@lo) 00000100:00100000:3.0:1776178185.065943:0:8232:0:(service.c:2152:ptlrpc_server_handle_req_in()) got req x1862456998041536 00000100:00100000:3.0:1776178185.065994:0:8232:0:(nrs_fifo.c:179:nrs_fifo_req_get()) NRS start fifo request from 12345-0@lo, seq: 463 00000100:00100000:3.0:1776178185.066000:0:8232:0:(service.c:2324:ptlrpc_server_handle_request()) Handling RPC req@000000006c748d80 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_create00:lustre-MDT0000-mdtlov_UUID+5:3353:x1862456998041536:12345-0@lo:13:osp-pre-1-0.0 00002000:00000024:3.0:1776178185.066034:0:8232:0:(ofd_obd.c:740:ofd_statfs()) blocks cached 229376 granted 491746368 pending 0 free 3923689472 avail 3692167168 00000100:00100000:3.0:1776178185.066113:0:8232:0:(service.c:2372:ptlrpc_server_handle_request()) Handled RPC req@000000006c748d80 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_create00:lustre-MDT0000-mdtlov_UUID+5:3353:x1862456998041536:12345-0@lo:13:osp-pre-1-0.0 Request processed in 112us (254us total) trans 0 rc 0/0 00000100:00100000:3.0:1776178185.066120:0:8232:0:(nrs_fifo.c:241:nrs_fifo_req_stop()) NRS stop fifo request from 12345-0@lo, seq: 463 00000100:00100000:3.0:1776178185.066166:0:3353:0:(client.c:2215:ptlrpc_check_set()) Completed RPC req@0000000022264af5 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_00:lustre-MDT0000-mdtlov_UUID:3353:1862456998041536:0@lo:13:osp-pre-1-0.0 00000100:00100000:3.0:1776178185.066316:0:3353:0:(client.c:1733:ptlrpc_send_new_req()) Sending RPC req@0000000013f6d767 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_00:lustre-MDT0000-lwp-OST0001_UUID:3353:1862456998041792:0@lo:400:kworker/u8:0.0 00000100:00100000:3.0:1776178185.066342:0:3353:0:(events.c:361:request_in_callback()) peer: 12345-0@lo (source: 12345-0@lo) 00000100:00100000:3.0:1776178185.066382:0:3353:0:(client.c:1733:ptlrpc_send_new_req()) Sending RPC req@00000000dd52d0d0 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_00:lustre-MDT0001-mdtlov_UUID:3353:1862456998042048:0@lo:400:kworker/u8:0.0 00000100:00100000:3.0:1776178185.066406:0:3353:0:(events.c:361:request_in_callback()) peer: 12345-0@lo (source: 12345-0@lo) 00000100:00100000:3.0:1776178185.066420:0:3353:0:(client.c:1733:ptlrpc_send_new_req()) Sending RPC req@00000000f62fe06f pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_00:lustre-MDT0000-lwp-MDT0000_UUID:3353:1862456998042304:0@lo:400:kworker/u8:0.0 00000100:00100000:3.0:1776178185.066437:0:3353:0:(events.c:361:request_in_callback()) peer: 12345-0@lo (source: 12345-0@lo) 00000100:00100000:3.0:1776178185.066450:0:3353:0:(ptlrpcd.c:415:ptlrpcd_check()) transfer 3 async RPCs [1->0] 00000100:00100000:3.0:1776178185.066454:0:3353:0:(client.c:1733:ptlrpc_send_new_req()) Sending RPC req@0000000022264af5 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_00:ac61f839-0e8e-4cac-9c59-08bcfbbad330:3353:1862456998041600:0@lo:400:kworker/u8:0.0 00000100:00100000:3.0:1776178185.066475:0:3353:0:(events.c:361:request_in_callback()) peer: 12345-0@lo (source: 12345-0@lo) 00000100:00100000:3.0:1776178185.066487:0:3353:0:(client.c:1733:ptlrpc_send_new_req()) Sending RPC req@0000000096181210 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_00:lustre-MDT0001-lwp-OST0001_UUID:3353:1862456998041856:0@lo:400:kworker/u8:0.0 00000100:00100000:3.0:1776178185.066503:0:3353:0:(events.c:361:request_in_callback()) peer: 12345-0@lo (source: 12345-0@lo) 00000100:00100000:3.0:1776178185.066512:0:3353:0:(client.c:1733:ptlrpc_send_new_req()) Sending RPC req@0000000087a31eb2 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_00:lustre-MDT0000-lwp-MDT0001_UUID:3353:1862456998042112:0@lo:400:kworker/u8:0.0 00000100:00100000:3.0:1776178185.066528:0:3353:0:(events.c:361:request_in_callback()) peer: 12345-0@lo (source: 12345-0@lo) 00000100:00100000:3.0:1776178185.066565:0:3356:0:(client.c:1733:ptlrpc_send_new_req()) Sending RPC req@000000005459e48f pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_03:lustre-MDT0001-lwp-OST0000_UUID:3356:1862456998041728:0@lo:400:kworker/u8:0.0 00000100:00100000:3.0:1776178185.066587:0:3356:0:(events.c:361:request_in_callback()) peer: 12345-0@lo (source: 12345-0@lo) 00000100:00100000:3.0:1776178185.066601:0:3356:0:(client.c:1733:ptlrpc_send_new_req()) Sending RPC req@0000000008605f3d pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_03:lustre-MDT0001-mdtlov_UUID:3356:1862456998041984:0@lo:400:kworker/u8:0.0 00000100:00100000:3.0:1776178185.066619:0:3356:0:(events.c:361:request_in_callback()) peer: 12345-0@lo (source: 12345-0@lo) 00000100:00100000:3.0:1776178185.066632:0:3356:0:(client.c:1733:ptlrpc_send_new_req()) Sending RPC req@000000008a6fdb8d pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_03:lustre-MDT0000-mdtlov_UUID:3356:1862456998042240:0@lo:400:kworker/u8:0.0 00000100:00100000:3.0:1776178185.066653:0:3356:0:(events.c:361:request_in_callback()) peer: 12345-0@lo (source: 12345-0@lo) 00000100:00100000:3.0:1776178185.066668:0:3356:0:(ptlrpcd.c:415:ptlrpcd_check()) transfer 3 async RPCs [2->3] 00000100:00100000:3.0:1776178185.066673:0:3356:0:(client.c:1733:ptlrpc_send_new_req()) Sending RPC req@000000002cf093e4 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_03:lustre-MDT0000-lwp-OST0000_UUID:3356:1862456998041664:0@lo:400:kworker/u8:0.0 00000100:00100000:3.0:1776178185.066692:0:3356:0:(events.c:361:request_in_callback()) peer: 12345-0@lo (source: 12345-0@lo) 00000100:00100000:3.0:1776178185.066699:0:3356:0:(client.c:1733:ptlrpc_send_new_req()) Sending RPC req@0000000023da22ef pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_03:lustre-MDT0001-mdtlov_UUID:3356:1862456998041920:0@lo:400:kworker/u8:0.0 00000100:00100000:3.0:1776178185.066719:0:3356:0:(events.c:361:request_in_callback()) peer: 12345-0@lo (source: 12345-0@lo) 00000100:00100000:3.0:1776178185.066733:0:3356:0:(client.c:1733:ptlrpc_send_new_req()) Sending RPC req@000000004c88c002 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_03:lustre-MDT0000-mdtlov_UUID:3356:1862456998042176:0@lo:400:kworker/u8:0.0 00000100:00100000:3.0:1776178185.066750:0:3356:0:(events.c:361:request_in_callback()) peer: 12345-0@lo (source: 12345-0@lo) 00000100:00100000:3.0:1776178185.066800:0:3354:0:(client.c:1733:ptlrpc_send_new_req()) Sending RPC req@00000000ac4116de pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_01:lustre-MDT0001-mdtlov_UUID:3354:1862456998042368:0@lo:41:osp-pre-0-1.0 00000100:00100000:3.0:1776178185.066822:0:3354:0:(events.c:361:request_in_callback()) peer: 12345-0@lo (source: 12345-0@lo) 00000100:00100000:3.0:1776178185.066864:0:8228:0:(service.c:2152:ptlrpc_server_handle_req_in()) got req x1862456998042048 00000100:00100000:3.0:1776178185.066888:0:8228:0:(service.c:2152:ptlrpc_server_handle_req_in()) got req x1862456998041984 00000100:00100000:3.0:1776178185.066904:0:8228:0:(service.c:2152:ptlrpc_server_handle_req_in()) got req x1862456998042240 00000100:00100000:3.0:1776178185.066920:0:8228:0:(nrs_fifo.c:179:nrs_fifo_req_get()) NRS start fifo request from 12345-0@lo, seq: 489 00000100:00100000:3.0:1776178185.066923:0:8228:0:(service.c:2324:ptlrpc_server_handle_request()) Handling RPC req@00000000a562d27c pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_001:lustre-MDT0001-mdtlov_UUID+5:3353:x1862456998042048:12345-0@lo:400:kworker/u8:0.0 00000100:00100000:3.0:1776178185.066958:0:8228:0:(service.c:2372:ptlrpc_server_handle_request()) Handled RPC req@00000000a562d27c pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_001:lustre-MDT0001-mdtlov_UUID+5:3353:x1862456998042048:12345-0@lo:400:kworker/u8:0.0 Request processed in 34us (551us total) trans 0 rc 0/0 00000100:00100000:3.0:1776178185.066964:0:8228:0:(nrs_fifo.c:241:nrs_fifo_req_stop()) NRS stop fifo request from 12345-0@lo, seq: 489 00000100:00100000:3.0:1776178185.066974:0:8228:0:(nrs_fifo.c:179:nrs_fifo_req_get()) NRS start fifo request from 12345-0@lo, seq: 490 00000100:00100000:3.0:1776178185.066977:0:8228:0:(service.c:2324:ptlrpc_server_handle_request()) Handling RPC req@00000000c38303b3 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_001:lustre-MDT0001-mdtlov_UUID+5:3356:x1862456998041984:12345-0@lo:400:kworker/u8:0.0 00000100:00100000:3.0:1776178185.067002:0:8228:0:(service.c:2372:ptlrpc_server_handle_request()) Handled RPC req@00000000c38303b3 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_001:lustre-MDT0001-mdtlov_UUID+5:3356:x1862456998041984:12345-0@lo:400:kworker/u8:0.0 Request processed in 24us (382us total) trans 0 rc 0/0 00000100:00100000:3.0:1776178185.067007:0:8228:0:(nrs_fifo.c:241:nrs_fifo_req_stop()) NRS stop fifo request from 12345-0@lo, seq: 490 00000100:00100000:3.0:1776178185.067016:0:8228:0:(nrs_fifo.c:179:nrs_fifo_req_get()) NRS start fifo request from 12345-0@lo, seq: 491 00000100:00100000:3.0:1776178185.067019:0:8228:0:(service.c:2324:ptlrpc_server_handle_request()) Handling RPC req@000000007309261c pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_001:lustre-MDT0000-mdtlov_UUID+5:3356:x1862456998042240:12345-0@lo:400:kworker/u8:0.0 00000100:00100000:3.0:1776178185.067039:0:8228:0:(service.c:2372:ptlrpc_server_handle_request()) Handled RPC req@000000007309261c pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_001:lustre-MDT0000-mdtlov_UUID+5:3356:x1862456998042240:12345-0@lo:400:kworker/u8:0.0 Request processed in 19us (386us total) trans 0 rc 0/0 00000100:00100000:3.0:1776178185.067044:0:8228:0:(nrs_fifo.c:241:nrs_fifo_req_stop()) NRS stop fifo request from 12345-0@lo, seq: 491 00000100:00100000:3.0:1776178185.067095:0:6255:0:(service.c:2152:ptlrpc_server_handle_req_in()) got req x1862456998041792 00000100:00100000:3.0:1776178185.067114:0:6255:0:(nrs_fifo.c:179:nrs_fifo_req_get()) NRS start fifo request from 12345-0@lo, seq: 876 00000100:00100000:3.0:1776178185.067117:0:6255:0:(service.c:2324:ptlrpc_server_handle_request()) Handling RPC req@000000009bbbb40c pname:cluuid+ref:pid:xid:nid:opc:job mdt00_000:lustre-MDT0000-lwp-OST0001_UUID+5:3353:x1862456998041792:12345-0@lo:400:kworker/u8:0.0 00000100:00100000:3.0:1776178185.067144:0:6255:0:(service.c:2372:ptlrpc_server_handle_request()) Handled RPC req@000000009bbbb40c pname:cluuid+ref:pid:xid:nid:opc:job mdt00_000:lustre-MDT0000-lwp-OST0001_UUID+5:3353:x1862456998041792:12345-0@lo:400:kworker/u8:0.0 Request processed in 26us (802us total) trans 0 rc 0/0 00000100:00100000:3.0:1776178185.067150:0:6255:0:(nrs_fifo.c:241:nrs_fifo_req_stop()) NRS stop fifo request from 12345-0@lo, seq: 876 00000100:00100000:3.0:1776178185.067160:0:6255:0:(service.c:2152:ptlrpc_server_handle_req_in()) got req x1862456998042304 00000100:00100000:3.0:1776178185.067170:0:6255:0:(service.c:2152:ptlrpc_server_handle_req_in()) got req x1862456998041856 00000100:00100000:3.0:1776178185.067179:0:6255:0:(service.c:2152:ptlrpc_server_handle_req_in()) got req x1862456998042112 00000100:00100000:3.0:1776178185.067187:0:6255:0:(service.c:2152:ptlrpc_server_handle_req_in()) got req x1862456998041728 00000100:00100000:3.0:1776178185.067195:0:6255:0:(service.c:2152:ptlrpc_server_handle_req_in()) got req x1862456998041664 00000100:00100000:3.0:1776178185.067204:0:6255:0:(nrs_fifo.c:179:nrs_fifo_req_get()) NRS start fifo request from 12345-0@lo, seq: 877 00000100:00100000:3.0:1776178185.067207:0:6255:0:(service.c:2324:ptlrpc_server_handle_request()) Handling RPC req@00000000e4231891 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_000:lustre-MDT0000-lwp-MDT0000_UUID+5:3353:x1862456998042304:12345-0@lo:400:kworker/u8:0.0 00000100:00100000:3.0:1776178185.067230:0:6255:0:(service.c:2372:ptlrpc_server_handle_request()) Handled RPC req@00000000e4231891 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_000:lustre-MDT0000-lwp-MDT0000_UUID+5:3353:x1862456998042304:12345-0@lo:400:kworker/u8:0.0 Request processed in 22us (793us total) trans 0 rc 0/0 00000100:00100000:3.0:1776178185.067236:0:6255:0:(nrs_fifo.c:241:nrs_fifo_req_stop()) NRS stop fifo request from 12345-0@lo, seq: 877 00000100:00100000:3.0:1776178185.067246:0:6255:0:(nrs_fifo.c:179:nrs_fifo_req_get()) NRS start fifo request from 12345-0@lo, seq: 878 00000100:00100000:3.0:1776178185.067249:0:6255:0:(service.c:2324:ptlrpc_server_handle_request()) Handling RPC req@00000000239d7b9a pname:cluuid+ref:pid:xid:nid:opc:job mdt00_000:lustre-MDT0001-lwp-OST0001_UUID+5:3353:x1862456998041856:12345-0@lo:400:kworker/u8:0.0 00000100:00100000:3.0:1776178185.067268:0:6255:0:(service.c:2372:ptlrpc_server_handle_request()) Handled RPC req@00000000239d7b9a pname:cluuid+ref:pid:xid:nid:opc:job mdt00_000:lustre-MDT0001-lwp-OST0001_UUID+5:3353:x1862456998041856:12345-0@lo:400:kworker/u8:0.0 Request processed in 19us (766us total) trans 0 rc 0/0 00000100:00100000:3.0:1776178185.067277:0:6255:0:(nrs_fifo.c:241:nrs_fifo_req_stop()) NRS stop fifo request from 12345-0@lo, seq: 878 00000100:00100000:3.0:1776178185.067285:0:6255:0:(nrs_fifo.c:179:nrs_fifo_req_get()) NRS start fifo request from 12345-0@lo, seq: 879 00000100:00100000:3.0:1776178185.067288:0:6255:0:(service.c:2324:ptlrpc_server_handle_request()) Handling RPC req@000000000daa3c9a pname:cluuid+ref:pid:xid:nid:opc:job mdt00_000:lustre-MDT0000-lwp-MDT0001_UUID+5:3353:x1862456998042112:12345-0@lo:400:kworker/u8:0.0 00000100:00100000:3.0:1776178185.067307:0:6255:0:(service.c:2372:ptlrpc_server_handle_request()) Handled RPC req@000000000daa3c9a pname:cluuid+ref:pid:xid:nid:opc:job mdt00_000:lustre-MDT0000-lwp-MDT0001_UUID+5:3353:x1862456998042112:12345-0@lo:400:kworker/u8:0.0 Request processed in 19us (779us total) trans 0 rc 0/0 00000100:00100000:3.0:1776178185.067313:0:6255:0:(nrs_fifo.c:241:nrs_fifo_req_stop()) NRS stop fifo request from 12345-0@lo, seq: 879 00000100:00100000:3.0:1776178185.067322:0:6255:0:(nrs_fifo.c:179:nrs_fifo_req_get()) NRS start fifo request from 12345-0@lo, seq: 880 00000100:00100000:3.0:1776178185.067325:0:6255:0:(service.c:2324:ptlrpc_server_handle_request()) Handling RPC req@0000000042ac1e35 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_000:lustre-MDT0001-lwp-OST0000_UUID+5:3356:x1862456998041728:12345-0@lo:400:kworker/u8:0.0 00000100:00100000:3.0:1776178185.067343:0:6255:0:(service.c:2372:ptlrpc_server_handle_request()) Handled RPC req@0000000042ac1e35 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_000:lustre-MDT0001-lwp-OST0000_UUID+5:3356:x1862456998041728:12345-0@lo:400:kworker/u8:0.0 Request processed in 18us (757us total) trans 0 rc 0/0 00000100:00100000:3.0:1776178185.067349:0:6255:0:(nrs_fifo.c:241:nrs_fifo_req_stop()) NRS stop fifo request from 12345-0@lo, seq: 880 00000100:00100000:3.0:1776178185.067357:0:6255:0:(nrs_fifo.c:179:nrs_fifo_req_get()) NRS start fifo request from 12345-0@lo, seq: 881 00000100:00100000:3.0:1776178185.067360:0:6255:0:(service.c:2324:ptlrpc_server_handle_request()) Handling RPC req@00000000ffd05959 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_000:lustre-MDT0000-lwp-OST0000_UUID+5:3356:x1862456998041664:12345-0@lo:400:kworker/u8:0.0 00000100:00100000:3.0:1776178185.067379:0:6255:0:(service.c:2372:ptlrpc_server_handle_request()) Handled RPC req@00000000ffd05959 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_000:lustre-MDT0000-lwp-OST0000_UUID+5:3356:x1862456998041664:12345-0@lo:400:kworker/u8:0.0 Request processed in 18us (687us total) trans 0 rc 0/0 00000100:00100000:3.0:1776178185.067384:0:6255:0:(nrs_fifo.c:241:nrs_fifo_req_stop()) NRS stop fifo request from 12345-0@lo, seq: 881 00000100:00100000:3.0:1776178185.067412:0:3356:0:(client.c:2215:ptlrpc_check_set()) Completed RPC req@000000002cf093e4 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_03:lustre-MDT0000-lwp-OST0000_UUID:3356:1862456998041664:0@lo:400:kworker/u8:0.0 00000100:00100000:3.0:1776178185.067420:0:3356:0:(client.c:2215:ptlrpc_check_set()) Completed RPC req@000000005459e48f pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_03:lustre-MDT0001-lwp-OST0000_UUID:3356:1862456998041728:0@lo:400:kworker/u8:0.0 00000100:00100000:3.0:1776178185.067428:0:3356:0:(client.c:2215:ptlrpc_check_set()) Completed RPC req@0000000008605f3d pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_03:lustre-MDT0001-mdtlov_UUID:3356:1862456998041984:0@lo:400:kworker/u8:0.0 00000100:00100000:3.0:1776178185.067435:0:3356:0:(client.c:2215:ptlrpc_check_set()) Completed RPC req@000000008a6fdb8d pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_03:lustre-MDT0000-mdtlov_UUID:3356:1862456998042240:0@lo:400:kworker/u8:0.0 00000100:00100000:3.0:1776178185.067459:0:3353:0:(client.c:2215:ptlrpc_check_set()) Completed RPC req@0000000096181210 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_00:lustre-MDT0001-lwp-OST0001_UUID:3353:1862456998041856:0@lo:400:kworker/u8:0.0 00000100:00100000:3.0:1776178185.067466:0:3353:0:(client.c:2215:ptlrpc_check_set()) Completed RPC req@0000000087a31eb2 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_00:lustre-MDT0000-lwp-MDT0001_UUID:3353:1862456998042112:0@lo:400:kworker/u8:0.0 00000100:00100000:3.0:1776178185.067474:0:3353:0:(client.c:2215:ptlrpc_check_set()) Completed RPC req@0000000013f6d767 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_00:lustre-MDT0000-lwp-OST0001_UUID:3353:1862456998041792:0@lo:400:kworker/u8:0.0 00000100:00100000:3.0:1776178185.067481:0:3353:0:(client.c:2215:ptlrpc_check_set()) Completed RPC req@00000000dd52d0d0 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_00:lustre-MDT0001-mdtlov_UUID:3353:1862456998042048:0@lo:400:kworker/u8:0.0 00000100:00100000:3.0:1776178185.067487:0:3353:0:(client.c:2215:ptlrpc_check_set()) Completed RPC req@00000000f62fe06f pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_00:lustre-MDT0000-lwp-MDT0000_UUID:3353:1862456998042304:0@lo:400:kworker/u8:0.0 00000100:00100000:3.0:1776178185.067542:0:20140:0:(service.c:2152:ptlrpc_server_handle_req_in()) got req x1862456998041600 00000100:00100000:3.0:1776178185.067565:0:20140:0:(nrs_fifo.c:179:nrs_fifo_req_get()) NRS start fifo request from 12345-0@lo, seq: 148 00000100:00100000:3.0:1776178185.067568:0:20140:0:(service.c:2324:ptlrpc_server_handle_request()) Handling RPC req@00000000ddaf121b pname:cluuid+ref:pid:xid:nid:opc:job ll_mgs_0002:ac61f839-0e8e-4cac-9c59-08bcfbbad330+23:3353:x1862456998041600:12345-0@lo:400:kworker/u8:0.0 00000100:00100000:3.0:1776178185.067596:0:20140:0:(service.c:2372:ptlrpc_server_handle_request()) Handled RPC req@00000000ddaf121b pname:cluuid+ref:pid:xid:nid:opc:job ll_mgs_0002:ac61f839-0e8e-4cac-9c59-08bcfbbad330+23:3353:x1862456998041600:12345-0@lo:400:kworker/u8:0.0 Request processed in 27us (1121us total) trans 0 rc 0/0 00000100:00100000:3.0:1776178185.067602:0:20140:0:(nrs_fifo.c:241:nrs_fifo_req_stop()) NRS stop fifo request from 12345-0@lo, seq: 148 00000100:00100000:3.0:1776178185.067633:0:3353:0:(client.c:2215:ptlrpc_check_set()) Completed RPC req@0000000022264af5 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_00:ac61f839-0e8e-4cac-9c59-08bcfbbad330:3353:1862456998041600:0@lo:400:kworker/u8:0.0 00000100:00100000:3.0:1776178185.067666:0:14128:0:(service.c:2152:ptlrpc_server_handle_req_in()) got req x1862456998041920 00000100:00100000:3.0:1776178185.067686:0:14128:0:(service.c:2152:ptlrpc_server_handle_req_in()) got req x1862456998042176 00000100:00100000:3.0:1776178185.067694:0:14128:0:(service.c:2152:ptlrpc_server_handle_req_in()) got req x1862456998042368 00000100:00100000:3.0:1776178185.067702:0:14128:0:(nrs_fifo.c:179:nrs_fifo_req_get()) NRS start fifo request from 12345-0@lo, seq: 679 00000100:00100000:3.0:1776178185.067705:0:14128:0:(service.c:2324:ptlrpc_server_handle_request()) Handling RPC req@000000001ee743a7 pname:cluuid+ref:pid:xid:nid:opc:job mdt_out00_002:lustre-MDT0001-mdtlov_UUID+7:3356:x1862456998041920:12345-0@lo:400:kworker/u8:0.0 00000100:00100000:3.0:1776178185.067737:0:14128:0:(service.c:2372:ptlrpc_server_handle_request()) Handled RPC req@000000001ee743a7 pname:cluuid+ref:pid:xid:nid:opc:job mdt_out00_002:lustre-MDT0001-mdtlov_UUID+7:3356:x1862456998041920:12345-0@lo:400:kworker/u8:0.0 Request processed in 30us (1017us total) trans 0 rc 0/0 00000100:00100000:3.0:1776178185.067743:0:14128:0:(nrs_fifo.c:241:nrs_fifo_req_stop()) NRS stop fifo request from 12345-0@lo, seq: 679 00000100:00100000:3.0:1776178185.067753:0:14128:0:(nrs_fifo.c:179:nrs_fifo_req_get()) NRS start fifo request from 12345-0@lo, seq: 680 00000100:00100000:3.0:1776178185.067756:0:14128:0:(service.c:2324:ptlrpc_server_handle_request()) Handling RPC req@0000000038b19922 pname:cluuid+ref:pid:xid:nid:opc:job mdt_out00_002:lustre-MDT0000-mdtlov_UUID+5:3356:x1862456998042176:12345-0@lo:400:kworker/u8:0.0 00000100:00100000:3.0:1776178185.067778:0:14128:0:(service.c:2372:ptlrpc_server_handle_request()) Handled RPC req@0000000038b19922 pname:cluuid+ref:pid:xid:nid:opc:job mdt_out00_002:lustre-MDT0000-mdtlov_UUID+5:3356:x1862456998042176:12345-0@lo:400:kworker/u8:0.0 Request processed in 22us (1029us total) trans 0 rc 0/0 00000100:00100000:3.0:1776178185.067783:0:14128:0:(nrs_fifo.c:241:nrs_fifo_req_stop()) NRS stop fifo request from 12345-0@lo, seq: 680 00000100:00100000:3.0:1776178185.067793:0:14128:0:(nrs_fifo.c:179:nrs_fifo_req_get()) NRS start fifo request from 12345-0@lo, seq: 681 00000100:00100000:3.0:1776178185.067796:0:14128:0:(service.c:2324:ptlrpc_server_handle_request()) Handling RPC req@000000000bdf54a0 pname:cluuid+ref:pid:xid:nid:opc:job mdt_out00_002:lustre-MDT0001-mdtlov_UUID+6:3354:x1862456998042368:12345-0@lo:41:osp-pre-0-1.0 00000004:00000024:3.0:1776178185.067817:0:14128:0:(mdt_handler.c:548:mdt_statfs()) blocks cached 0 granted 2097152 pending 0 free 1446121472 avail 1316659200 00000100:00100000:3.0:1776178185.067847:0:14128:0:(service.c:2372:ptlrpc_server_handle_request()) Handled RPC req@000000000bdf54a0 pname:cluuid+ref:pid:xid:nid:opc:job mdt_out00_002:lustre-MDT0001-mdtlov_UUID+6:3354:x1862456998042368:12345-0@lo:41:osp-pre-0-1.0 Request processed in 50us (1025us total) trans 0 rc 0/0 00000100:00100000:3.0:1776178185.067852:0:14128:0:(nrs_fifo.c:241:nrs_fifo_req_stop()) NRS stop fifo request from 12345-0@lo, seq: 681 00000100:00100000:3.0:1776178185.067880:0:3354:0:(client.c:2215:ptlrpc_check_set()) Completed RPC req@00000000ac4116de pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_01:lustre-MDT0001-mdtlov_UUID:3354:1862456998042368:0@lo:41:osp-pre-0-1.0 00000100:00100000:3.0:1776178185.067899:0:3356:0:(client.c:2215:ptlrpc_check_set()) Completed RPC req@0000000023da22ef pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_03:lustre-MDT0001-mdtlov_UUID:3356:1862456998041920:0@lo:400:kworker/u8:0.0 00000100:00100000:3.0:1776178185.067907:0:3356:0:(client.c:2215:ptlrpc_check_set()) Completed RPC req@000000004c88c002 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_03:lustre-MDT0000-mdtlov_UUID:3356:1862456998042176:0@lo:400:kworker/u8:0.0 00000100:00100000:3.0:1776178185.098041:0:3355:0:(client.c:1733:ptlrpc_send_new_req()) Sending RPC req@00000000ac4116de pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_02:lustre-MDT0000-mdtlov_UUID:3355:1862456998042432:0@lo:13:osp-pre-0-0.0 00000100:00100000:3.0:1776178185.098274:0:3355:0:(events.c:361:request_in_callback()) peer: 12345-0@lo (source: 12345-0@lo) 00000100:00100000:3.0:1776178185.098366:0:9554:0:(service.c:2152:ptlrpc_server_handle_req_in()) got req x1862456998042432 00000100:00100000:3.0:1776178185.098415:0:9554:0:(nrs_fifo.c:179:nrs_fifo_req_get()) NRS start fifo request from 12345-0@lo, seq: 464 00000100:00100000:3.0:1776178185.098422:0:9554:0:(service.c:2324:ptlrpc_server_handle_request()) Handling RPC req@00000000034781c2 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_create00:lustre-MDT0000-mdtlov_UUID+5:3355:x1862456998042432:12345-0@lo:13:osp-pre-0-0.0 00002000:00000024:3.0:1776178185.098462:0:9554:0:(ofd_obd.c:740:ofd_statfs()) blocks cached 315392 granted 484883776 pending 0 free 3923689472 avail 3692167168 00000100:00100000:3.0:1776178185.098505:0:9554:0:(service.c:2372:ptlrpc_server_handle_request()) Handled RPC req@00000000034781c2 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_create00:lustre-MDT0000-mdtlov_UUID+5:3355:x1862456998042432:12345-0@lo:13:osp-pre-0-0.0 Request processed in 82us (232us total) trans 0 rc 0/0 00000100:00100000:3.0:1776178185.098514:0:9554:0:(nrs_fifo.c:241:nrs_fifo_req_stop()) NRS stop fifo request from 12345-0@lo, seq: 464 00000100:00100000:3.0:1776178185.098569:0:3355:0:(client.c:2215:ptlrpc_check_set()) Completed RPC req@00000000ac4116de pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_02:lustre-MDT0000-mdtlov_UUID:3355:1862456998042432:0@lo:13:osp-pre-0-0.0 00000100:00100000:0.0:1776178185.134003:0:3355:0:(ptlrpcd.c:415:ptlrpcd_check()) transfer 1 async RPCs [3->2] 00000100:00100000:0.0:1776178185.134042:0:3355:0:(client.c:1733:ptlrpc_send_new_req()) Sending RPC req@00000000ca8b81ba pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_02:lustre-MDT0000-mdtlov_UUID:3355:1862456998042496:0@lo:41:osp-pre-1-0.0 00000100:00100000:0.0:1776178185.134141:0:3355:0:(events.c:361:request_in_callback()) peer: 12345-0@lo (source: 12345-0@lo) 00000100:00100000:2.0:1776178185.134239:0:6260:0:(service.c:2152:ptlrpc_server_handle_req_in()) got req x1862456998042496 00000100:00100000:2.0:1776178185.134295:0:6260:0:(nrs_fifo.c:179:nrs_fifo_req_get()) NRS start fifo request from 12345-0@lo, seq: 682 00000100:00100000:2.0:1776178185.134301:0:6260:0:(service.c:2324:ptlrpc_server_handle_request()) Handling RPC req@000000004ee84694 pname:cluuid+ref:pid:xid:nid:opc:job mdt_out00_000:lustre-MDT0000-mdtlov_UUID+5:3355:x1862456998042496:12345-0@lo:41:osp-pre-1-0.0 00000004:00000024:2.0:1776178185.134333:0:6260:0:(mdt_handler.c:548:mdt_statfs()) blocks cached 0 granted 2097152 pending 0 free 1446035456 avail 1316573184 00000100:00100000:2.0:1776178185.134397:0:6260:0:(service.c:2372:ptlrpc_server_handle_request()) Handled RPC req@000000004ee84694 pname:cluuid+ref:pid:xid:nid:opc:job mdt_out00_000:lustre-MDT0000-mdtlov_UUID+5:3355:x1862456998042496:12345-0@lo:41:osp-pre-1-0.0 Request processed in 95us (257us total) trans 0 rc 0/0 00000100:00100000:2.0:1776178185.134404:0:6260:0:(nrs_fifo.c:241:nrs_fifo_req_stop()) NRS stop fifo request from 12345-0@lo, seq: 682 00000100:00100000:1.0:1776178185.134497:0:3355:0:(client.c:2215:ptlrpc_check_set()) Completed RPC req@00000000ca8b81ba pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_02:lustre-MDT0000-mdtlov_UUID:3355:1862456998042496:0@lo:41:osp-pre-1-0.0 00010000:00010000:2.0:1776178185.696591:0:9224:0:(ldlm_lockd.c:2792:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:3.0:1776178186.541808:0:3346:0:(events.c:361:request_in_callback()) peer: 12345-192.168.204.3@tcp (source: 12345-192.168.204.3@tcp) 00000100:00100000:3.0:1776178186.541869:0:3346:0:(events.c:361:request_in_callback()) peer: 12345-192.168.204.3@tcp (source: 12345-192.168.204.3@tcp) 00000100:00100000:3.0:1776178186.541904:0:3346:0:(events.c:361:request_in_callback()) peer: 12345-192.168.204.3@tcp (source: 12345-192.168.204.3@tcp) 00000100:00100000:3.0:1776178186.541925:0:3346:0:(events.c:361:request_in_callback()) peer: 12345-192.168.204.3@tcp (source: 12345-192.168.204.3@tcp) 00000100:00100000:3.0:1776178186.541956:0:3346:0:(events.c:361:request_in_callback()) peer: 12345-192.168.204.3@tcp (source: 12345-192.168.204.3@tcp) 00000100:00100000:1.0:1776178186.542141:0:20759:0:(service.c:2152:ptlrpc_server_handle_req_in()) got req x1862456979097088 00000100:00100000:1.0:1776178186.542189:0:20759:0:(service.c:2152:ptlrpc_server_handle_req_in()) got req x1862456979097152 00000100:00100000:1.0:1776178186.542208:0:20759:0:(nrs_fifo.c:179:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.204.3@tcp, seq: 492 00000100:00100000:1.0:1776178186.542216:0:20759:0:(service.c:2324:ptlrpc_server_handle_request()) Handling RPC req@00000000f97fd38e pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_010:b50963bd-8eba-4432-9a54-8888d00e3fa1+8:2273:x1862456979097088:12345-192.168.204.3@tcp:400:kworker/u8:2.0 00000100:00100000:1.0:1776178186.542308:0:20759:0:(service.c:2372:ptlrpc_server_handle_request()) Handled RPC req@00000000f97fd38e pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_010:b50963bd-8eba-4432-9a54-8888d00e3fa1+8:2273:x1862456979097088:12345-192.168.204.3@tcp:400:kworker/u8:2.0 Request processed in 93us (405us total) trans 0 rc 0/0 00000100:00100000:1.0:1776178186.542317:0:20759:0:(nrs_fifo.c:241:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.204.3@tcp, seq: 492 00000100:00100000:1.0:1776178186.542333:0:20759:0:(nrs_fifo.c:179:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.204.3@tcp, seq: 493 00000100:00100000:1.0:1776178186.542337:0:20759:0:(service.c:2324:ptlrpc_server_handle_request()) Handling RPC req@00000000196ec025 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_010:b50963bd-8eba-4432-9a54-8888d00e3fa1+9:2273:x1862456979097152:12345-192.168.204.3@tcp:400:kworker/u8:2.0 00000100:00100000:1.0:1776178186.542365:0:20759:0:(service.c:2372:ptlrpc_server_handle_request()) Handled RPC req@00000000196ec025 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_010:b50963bd-8eba-4432-9a54-8888d00e3fa1+9:2273:x1862456979097152:12345-192.168.204.3@tcp:400:kworker/u8:2.0 Request processed in 28us (410us total) trans 0 rc 0/0 00000100:00100000:1.0:1776178186.542371:0:20759:0:(nrs_fifo.c:241:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.204.3@tcp, seq: 493 00000100:00100000:0.0:1776178186.545175:0:20137:0:(service.c:2152:ptlrpc_server_handle_req_in()) got req x1862456979096896 00000100:00100000:0.0:1776178186.545248:0:20137:0:(nrs_fifo.c:179:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.204.3@tcp, seq: 149 00000100:00100000:0.0:1776178186.545254:0:20137:0:(service.c:2324:ptlrpc_server_handle_request()) Handling RPC req@000000003ef1d595 pname:cluuid+ref:pid:xid:nid:opc:job ll_mgs_0000:5b5c66b1-1a2f-4651-8548-0c75b9a38a94+9:2272:x1862456979096896:12345-192.168.204.3@tcp:400:kworker/u8:2.0 00000100:00100000:0.0:1776178186.545350:0:20137:0:(service.c:2372:ptlrpc_server_handle_request()) Handled RPC req@000000003ef1d595 pname:cluuid+ref:pid:xid:nid:opc:job ll_mgs_0000:5b5c66b1-1a2f-4651-8548-0c75b9a38a94+9:2272:x1862456979096896:12345-192.168.204.3@tcp:400:kworker/u8:2.0 Request processed in 95us (3555us total) trans 0 rc 0/0 00000100:00100000:0.0:1776178186.545359:0:20137:0:(nrs_fifo.c:241:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.204.3@tcp, seq: 149 00000100:00100000:2.0:1776178186.552178:0:11281:0:(service.c:2152:ptlrpc_server_handle_req_in()) got req x1862456979096960 00000100:00100000:3.0:1776178186.552748:0:6256:0:(service.c:2152:ptlrpc_server_handle_req_in()) got req x1862456979097024 00000100:00100000:2.0:1776178186.552808:0:11281:0:(nrs_fifo.c:179:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.204.3@tcp, seq: 882 00000100:00100000:2.0:1776178186.552816:0:11281:0:(service.c:2324:ptlrpc_server_handle_request()) Handling RPC req@00000000ac4116de pname:cluuid+ref:pid:xid:nid:opc:job mdt00_004:b50963bd-8eba-4432-9a54-8888d00e3fa1+7:2273:x1862456979096960:12345-192.168.204.3@tcp:400:kworker/u8:2.0 00000100:00100000:2.0:1776178186.553057:0:11281:0:(service.c:2372:ptlrpc_server_handle_request()) Handled RPC req@00000000ac4116de pname:cluuid+ref:pid:xid:nid:opc:job mdt00_004:b50963bd-8eba-4432-9a54-8888d00e3fa1+7:2273:x1862456979096960:12345-192.168.204.3@tcp:400:kworker/u8:2.0 Request processed in 240us (11187us total) trans 0 rc 0/0 00000100:00100000:2.0:1776178186.553088:0:11281:0:(nrs_fifo.c:241:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.204.3@tcp, seq: 882 00000100:00100000:2.0:1776178186.553107:0:11281:0:(nrs_fifo.c:179:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.204.3@tcp, seq: 883 00000100:00100000:2.0:1776178186.553112:0:11281:0:(service.c:2324:ptlrpc_server_handle_request()) Handling RPC req@000000008b978a8a pname:cluuid+ref:pid:xid:nid:opc:job mdt00_004:b50963bd-8eba-4432-9a54-8888d00e3fa1+6:2273:x1862456979097024:12345-192.168.204.3@tcp:400:kworker/u8:2.0 00000100:00100000:2.0:1776178186.553144:0:11281:0:(service.c:2372:ptlrpc_server_handle_request()) Handled RPC req@000000008b978a8a pname:cluuid+ref:pid:xid:nid:opc:job mdt00_004:b50963bd-8eba-4432-9a54-8888d00e3fa1+6:2273:x1862456979097024:12345-192.168.204.3@tcp:400:kworker/u8:2.0 Request processed in 31us (11219us total) trans 0 rc 0/0 00000100:00100000:2.0:1776178186.553150:0:11281:0:(nrs_fifo.c:241:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.204.3@tcp, seq: 883 00000100:00100000:0.0:1776178186.594655:0:3353:0:(client.c:1733:ptlrpc_send_new_req()) Sending RPC req@00000000f45ed991 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_00:lustre-MDT0001-mdtlov_UUID:3353:1862456998042560:0@lo:13:osp-pre-1-1.0 00000100:00100000:0.0:1776178186.594736:0:3353:0:(events.c:361:request_in_callback()) peer: 12345-0@lo (source: 12345-0@lo) 00000100:00100000:3.0:1776178186.595747:0:8232:0:(service.c:2152:ptlrpc_server_handle_req_in()) got req x1862456998042560 00000100:00100000:3.0:1776178186.595815:0:8232:0:(nrs_fifo.c:179:nrs_fifo_req_get()) NRS start fifo request from 12345-0@lo, seq: 465 00000100:00100000:3.0:1776178186.595823:0:8232:0:(service.c:2324:ptlrpc_server_handle_request()) Handling RPC req@00000000c19c19b9 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_create00:lustre-MDT0001-mdtlov_UUID+5:3353:x1862456998042560:12345-0@lo:13:osp-pre-1-1.0 00002000:00000024:3.0:1776178186.595849:0:8232:0:(ofd_obd.c:740:ofd_statfs()) blocks cached 229376 granted 491746368 pending 0 free 3923689472 avail 3692167168 00000100:00100000:3.0:1776178186.595931:0:8232:0:(service.c:2372:ptlrpc_server_handle_request()) Handled RPC req@00000000c19c19b9 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_create00:lustre-MDT0001-mdtlov_UUID+5:3353:x1862456998042560:12345-0@lo:13:osp-pre-1-1.0 Request processed in 108us (1197us total) trans 0 rc 0/0 00000100:00100000:3.0:1776178186.595939:0:8232:0:(nrs_fifo.c:241:nrs_fifo_req_stop()) NRS stop fifo request from 12345-0@lo, seq: 465 00000100:00100000:1.0:1776178186.596027:0:3353:0:(client.c:2215:ptlrpc_check_set()) Completed RPC req@00000000f45ed991 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_00:lustre-MDT0001-mdtlov_UUID:3353:1862456998042560:0@lo:13:osp-pre-1-1.0 00010000:00010000:1.0:1776178186.720523:0:6245:0:(ldlm_lockd.c:2792:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:1.0:1776178187.104309:0:3354:0:(client.c:1733:ptlrpc_send_new_req()) Sending RPC req@000000004eee88be pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_01:lustre-MDT0001-mdtlov_UUID:3354:1862456998042624:0@lo:13:osp-pre-0-1.0 00000100:00100000:1.0:1776178187.104388:0:3354:0:(events.c:361:request_in_callback()) peer: 12345-0@lo (source: 12345-0@lo) 00000100:00100000:3.0:1776178187.104498:0:9554:0:(service.c:2152:ptlrpc_server_handle_req_in()) got req x1862456998042624 00000100:00100000:3.0:1776178187.104560:0:9554:0:(nrs_fifo.c:179:nrs_fifo_req_get()) NRS start fifo request from 12345-0@lo, seq: 466 00000100:00100000:3.0:1776178187.104574:0:9554:0:(service.c:2324:ptlrpc_server_handle_request()) Handling RPC req@00000000ca8b81ba pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_create00:lustre-MDT0001-mdtlov_UUID+5:3354:x1862456998042624:12345-0@lo:13:osp-pre-0-1.0 00002000:00000024:3.0:1776178187.104603:0:9554:0:(ofd_obd.c:740:ofd_statfs()) blocks cached 315392 granted 484883776 pending 0 free 3923689472 avail 3692167168 00000100:00100000:3.0:1776178187.105367:0:9554:0:(service.c:2372:ptlrpc_server_handle_request()) Handled RPC req@00000000ca8b81ba pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_create00:lustre-MDT0001-mdtlov_UUID+5:3354:x1862456998042624:12345-0@lo:13:osp-pre-0-1.0 Request processed in 789us (976us total) trans 0 rc 0/0 00000100:00100000:3.0:1776178187.105376:0:9554:0:(nrs_fifo.c:241:nrs_fifo_req_stop()) NRS stop fifo request from 12345-0@lo, seq: 466 00000100:00100000:1.0:1776178187.105455:0:3354:0:(client.c:2215:ptlrpc_check_set()) Completed RPC req@000000004eee88be pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_01:lustre-MDT0001-mdtlov_UUID:3354:1862456998042624:0@lo:13:osp-pre-0-1.0 00010000:00010000:2.0:1776178187.747419:0:6244:0:(ldlm_lockd.c:2792:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00010000:00010000:2.0:1776178188.772208:0:9225:0:(ldlm_lockd.c:2792:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00010000:00010000:2.0:1776178189.798096:0:9224:0:(ldlm_lockd.c:2792:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:3.0:1776178190.176384:0:3356:0:(client.c:1733:ptlrpc_send_new_req()) Sending RPC req@0000000007e413b7 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_03:lustre-MDT0000-mdtlov_UUID:3356:1862456998042752:0@lo:13:osp-pre-1-0.0 00000100:00100000:1.0:1776178190.176465:0:3355:0:(client.c:1733:ptlrpc_send_new_req()) Sending RPC req@00000000c37c187e pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_02:lustre-MDT0000-mdtlov_UUID:3355:1862456998042688:0@lo:13:osp-pre-0-0.0 00000100:00100000:3.0:1776178190.176511:0:3356:0:(events.c:361:request_in_callback()) peer: 12345-0@lo (source: 12345-0@lo) 00000100:00100000:3.0:1776178190.176577:0:3355:0:(events.c:361:request_in_callback()) peer: 12345-0@lo (source: 12345-0@lo) 00000100:00100000:3.0:1776178190.176616:0:9554:0:(service.c:2152:ptlrpc_server_handle_req_in()) got req x1862456998042752 00000100:00100000:3.0:1776178190.176682:0:9554:0:(service.c:2152:ptlrpc_server_handle_req_in()) got req x1862456998042688 00000100:00100000:3.0:1776178190.176698:0:9554:0:(nrs_fifo.c:179:nrs_fifo_req_get()) NRS start fifo request from 12345-0@lo, seq: 467 00000100:00100000:3.0:1776178190.176705:0:9554:0:(service.c:2324:ptlrpc_server_handle_request()) Handling RPC req@00000000c237605d pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_create00:lustre-MDT0000-mdtlov_UUID+5:3356:x1862456998042752:12345-0@lo:13:osp-pre-1-0.0 00002000:00000024:3.0:1776178190.176740:0:9554:0:(ofd_obd.c:740:ofd_statfs()) blocks cached 229376 granted 491746368 pending 0 free 3923689472 avail 3692167168 00000100:00100000:2.0:1776178190.176943:0:8232:0:(nrs_fifo.c:179:nrs_fifo_req_get()) NRS start fifo request from 12345-0@lo, seq: 468 00000100:00100000:2.0:1776178190.176949:0:8232:0:(service.c:2324:ptlrpc_server_handle_request()) Handling RPC req@000000009a62f69c pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_create00:lustre-MDT0000-mdtlov_UUID+5:3355:x1862456998042688:12345-0@lo:13:osp-pre-0-0.0 00002000:00000024:2.0:1776178190.176984:0:8232:0:(ofd_obd.c:740:ofd_statfs()) blocks cached 315392 granted 484883776 pending 0 free 3923689472 avail 3692167168 00000100:00100000:1.0:1776178190.177199:0:3356:0:(client.c:2215:ptlrpc_check_set()) Completed RPC req@0000000007e413b7 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_03:lustre-MDT0000-mdtlov_UUID:3356:1862456998042752:0@lo:13:osp-pre-1-0.0 00000100:00100000:1.0:1776178190.177350:0:3356:0:(client.c:1733:ptlrpc_send_new_req()) Sending RPC req@0000000044f82452 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_03:lustre-MDT0001-lwp-OST0000_UUID:3356:1862456998043008:0@lo:400:kworker/u8:0.0 00000100:00100000:0.0:1776178190.177367:0:3354:0:(client.c:1733:ptlrpc_send_new_req()) Sending RPC req@0000000054130213 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_01:ac61f839-0e8e-4cac-9c59-08bcfbbad330:3354:1862456998042880:0@lo:400:kworker/u8:0.0 00000100:00100000:2.0:1776178190.178435:0:8232:0:(service.c:2372:ptlrpc_server_handle_request()) Handled RPC req@000000009a62f69c pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_create00:lustre-MDT0000-mdtlov_UUID+5:3355:x1862456998042688:12345-0@lo:13:osp-pre-0-0.0 Request processed in 1479us (1852us total) trans 0 rc 0/0 00000100:00100000:2.0:1776178190.178445:0:8232:0:(nrs_fifo.c:241:nrs_fifo_req_stop()) NRS stop fifo request from 12345-0@lo, seq: 468 00000100:00100000:2.0:1776178190.178500:0:3355:0:(client.c:1733:ptlrpc_send_new_req()) Sending RPC req@0000000028ade006 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_02:lustre-MDT0000-lwp-OST0000_UUID:3355:1862456998042944:0@lo:400:kworker/u8:0.0 00000100:00100000:2.0:1776178190.178542:0:3355:0:(events.c:361:request_in_callback()) peer: 12345-0@lo (source: 12345-0@lo) 00000100:00100000:2.0:1776178190.178563:0:3355:0:(client.c:1733:ptlrpc_send_new_req()) Sending RPC req@0000000007075047 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_02:lustre-MDT0001-mdtlov_UUID:3355:1862456998043200:0@lo:400:kworker/u8:0.0 00000100:00100000:2.0:1776178190.178591:0:3355:0:(events.c:361:request_in_callback()) peer: 12345-0@lo (source: 12345-0@lo) 00000100:00100000:2.0:1776178190.178608:0:3355:0:(client.c:1733:ptlrpc_send_new_req()) Sending RPC req@000000001c465e98 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_02:lustre-MDT0000-mdtlov_UUID:3355:1862456998043456:0@lo:400:kworker/u8:0.0 00000100:00100000:2.0:1776178190.178629:0:3355:0:(events.c:361:request_in_callback()) peer: 12345-0@lo (source: 12345-0@lo) 00000100:00100000:2.0:1776178190.178666:0:3355:0:(client.c:2215:ptlrpc_check_set()) Completed RPC req@00000000c37c187e pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_02:lustre-MDT0000-mdtlov_UUID:3355:1862456998042688:0@lo:13:osp-pre-0-0.0 00000100:00100000:2.0:1776178190.178705:0:11281:0:(service.c:2152:ptlrpc_server_handle_req_in()) got req x1862456998042944 00000100:00100000:2.0:1776178190.178797:0:11281:0:(nrs_fifo.c:179:nrs_fifo_req_get()) NRS start fifo request from 12345-0@lo, seq: 884 00000100:00100000:2.0:1776178190.178802:0:11281:0:(service.c:2324:ptlrpc_server_handle_request()) Handling RPC req@000000002549e7f5 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_004:lustre-MDT0000-lwp-OST0000_UUID+5:3355:x1862456998042944:12345-0@lo:400:kworker/u8:0.0 00000100:00100000:2.0:1776178190.178851:0:11281:0:(service.c:2372:ptlrpc_server_handle_request()) Handled RPC req@000000002549e7f5 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_004:lustre-MDT0000-lwp-OST0000_UUID+5:3355:x1862456998042944:12345-0@lo:400:kworker/u8:0.0 Request processed in 48us (310us total) trans 0 rc 0/0 00000100:00100000:2.0:1776178190.178858:0:11281:0:(nrs_fifo.c:241:nrs_fifo_req_stop()) NRS stop fifo request from 12345-0@lo, seq: 884 00000100:00100000:2.0:1776178190.178877:0:6260:0:(service.c:2152:ptlrpc_server_handle_req_in()) got req x1862456998043200 00000100:00100000:2.0:1776178190.178901:0:6260:0:(service.c:2152:ptlrpc_server_handle_req_in()) got req x1862456998043456 00000100:00100000:2.0:1776178190.178919:0:6260:0:(nrs_fifo.c:179:nrs_fifo_req_get()) NRS start fifo request from 12345-0@lo, seq: 683 00000100:00100000:2.0:1776178190.178923:0:6260:0:(service.c:2324:ptlrpc_server_handle_request()) Handling RPC req@0000000018bd7474 pname:cluuid+ref:pid:xid:nid:opc:job mdt_out00_000:lustre-MDT0001-mdtlov_UUID+6:3355:x1862456998043200:12345-0@lo:400:kworker/u8:0.0 00000100:00100000:2.0:1776178190.178954:0:6260:0:(service.c:2372:ptlrpc_server_handle_request()) Handled RPC req@0000000018bd7474 pname:cluuid+ref:pid:xid:nid:opc:job mdt_out00_000:lustre-MDT0001-mdtlov_UUID+6:3355:x1862456998043200:12345-0@lo:400:kworker/u8:0.0 Request processed in 31us (362us total) trans 0 rc 0/0 00000100:00100000:2.0:1776178190.179299:0:6260:0:(nrs_fifo.c:241:nrs_fifo_req_stop()) NRS stop fifo request from 12345-0@lo, seq: 683 00000100:00100000:2.0:1776178190.179313:0:6260:0:(nrs_fifo.c:179:nrs_fifo_req_get()) NRS start fifo request from 12345-0@lo, seq: 684 00000100:00100000:2.0:1776178190.179316:0:6260:0:(service.c:2324:ptlrpc_server_handle_request()) Handling RPC req@0000000028dd2f9a pname:cluuid+ref:pid:xid:nid:opc:job mdt_out00_000:lustre-MDT0000-mdtlov_UUID+5:3355:x1862456998043456:12345-0@lo:400:kworker/u8:0.0 00000100:00100000:2.0:1776178190.179347:0:6260:0:(service.c:2372:ptlrpc_server_handle_request()) Handled RPC req@0000000028dd2f9a pname:cluuid+ref:pid:xid:nid:opc:job mdt_out00_000:lustre-MDT0000-mdtlov_UUID+5:3355:x1862456998043456:12345-0@lo:400:kworker/u8:0.0 Request processed in 30us (717us total) trans 0 rc 0/0 00000100:00100000:2.0:1776178190.179352:0:6260:0:(nrs_fifo.c:241:nrs_fifo_req_stop()) NRS stop fifo request from 12345-0@lo, seq: 684 00000100:00100000:2.0:1776178190.179388:0:3355:0:(client.c:2215:ptlrpc_check_set()) Completed RPC req@0000000028ade006 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_02:lustre-MDT0000-lwp-OST0000_UUID:3355:1862456998042944:0@lo:400:kworker/u8:0.0 00000100:00100000:2.0:1776178190.179397:0:3355:0:(client.c:2215:ptlrpc_check_set()) Completed RPC req@0000000007075047 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_02:lustre-MDT0001-mdtlov_UUID:3355:1862456998043200:0@lo:400:kworker/u8:0.0 00000100:00100000:2.0:1776178190.179404:0:3355:0:(client.c:2215:ptlrpc_check_set()) Completed RPC req@000000001c465e98 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_02:lustre-MDT0000-mdtlov_UUID:3355:1862456998043456:0@lo:400:kworker/u8:0.0 00000100:00100000:2.0:1776178190.179450:0:3353:0:(client.c:1733:ptlrpc_send_new_req()) Sending RPC req@00000000226852f3 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_00:lustre-MDT0000-mdtlov_UUID:3353:1862456998042816:0@lo:41:osp-pre-1-0.0 00000100:00100000:2.0:1776178190.179481:0:3353:0:(events.c:361:request_in_callback()) peer: 12345-0@lo (source: 12345-0@lo) 00000100:00100000:2.0:1776178190.179497:0:3353:0:(client.c:1733:ptlrpc_send_new_req()) Sending RPC req@00000000d28be450 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_00:lustre-MDT0000-lwp-OST0001_UUID:3353:1862456998043072:0@lo:400:kworker/u8:0.0 00000100:00100000:2.0:1776178190.179519:0:3353:0:(events.c:361:request_in_callback()) peer: 12345-0@lo (source: 12345-0@lo) 00000100:00100000:2.0:1776178190.179533:0:3353:0:(client.c:1733:ptlrpc_send_new_req()) Sending RPC req@00000000cc7f07e3 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_00:lustre-MDT0001-mdtlov_UUID:3353:1862456998043328:0@lo:400:kworker/u8:0.0 00000100:00100000:2.0:1776178190.179556:0:3353:0:(events.c:361:request_in_callback()) peer: 12345-0@lo (source: 12345-0@lo) 00000100:00100000:2.0:1776178190.179582:0:14128:0:(service.c:2152:ptlrpc_server_handle_req_in()) got req x1862456998042816 00000100:00100000:2.0:1776178190.179603:0:14128:0:(nrs_fifo.c:179:nrs_fifo_req_get()) NRS start fifo request from 12345-0@lo, seq: 685 00000100:00100000:2.0:1776178190.179606:0:14128:0:(service.c:2324:ptlrpc_server_handle_request()) Handling RPC req@000000000354b8ca pname:cluuid+ref:pid:xid:nid:opc:job mdt_out00_002:lustre-MDT0000-mdtlov_UUID+5:3353:x1862456998042816:12345-0@lo:41:osp-pre-1-0.0 00000004:00000024:2.0:1776178190.179630:0:14128:0:(mdt_handler.c:548:mdt_statfs()) blocks cached 0 granted 2097152 pending 0 free 1446035456 avail 1316573184 00000100:00100000:2.0:1776178190.179662:0:14128:0:(service.c:2372:ptlrpc_server_handle_request()) Handled RPC req@000000000354b8ca pname:cluuid+ref:pid:xid:nid:opc:job mdt_out00_002:lustre-MDT0000-mdtlov_UUID+5:3353:x1862456998042816:12345-0@lo:41:osp-pre-1-0.0 Request processed in 55us (182us total) trans 0 rc 0/0 00000100:00100000:2.0:1776178190.179667:0:14128:0:(nrs_fifo.c:241:nrs_fifo_req_stop()) NRS stop fifo request from 12345-0@lo, seq: 685 00000100:00100000:2.0:1776178190.179684:0:6257:0:(service.c:2152:ptlrpc_server_handle_req_in()) got req x1862456998043072 00000100:00100000:2.0:1776178190.179705:0:6257:0:(nrs_fifo.c:179:nrs_fifo_req_get()) NRS start fifo request from 12345-0@lo, seq: 885 00000100:00100000:2.0:1776178190.179708:0:6257:0:(service.c:2324:ptlrpc_server_handle_request()) Handling RPC req@0000000013888310 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_002:lustre-MDT0000-lwp-OST0001_UUID+5:3353:x1862456998043072:12345-0@lo:400:kworker/u8:0.0 00000100:00100000:2.0:1776178190.179912:0:6257:0:(service.c:2372:ptlrpc_server_handle_request()) Handled RPC req@0000000013888310 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_002:lustre-MDT0000-lwp-OST0001_UUID+5:3353:x1862456998043072:12345-0@lo:400:kworker/u8:0.0 Request processed in 202us (392us total) trans 0 rc 0/0 00000100:00100000:2.0:1776178190.179919:0:6257:0:(nrs_fifo.c:241:nrs_fifo_req_stop()) NRS stop fifo request from 12345-0@lo, seq: 885 00000100:00100000:2.0:1776178190.179953:0:3353:0:(client.c:2215:ptlrpc_check_set()) Completed RPC req@00000000226852f3 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_00:lustre-MDT0000-mdtlov_UUID:3353:1862456998042816:0@lo:41:osp-pre-1-0.0 00000100:00100000:2.0:1776178190.179975:0:3353:0:(client.c:2215:ptlrpc_check_set()) Completed RPC req@00000000d28be450 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_00:lustre-MDT0000-lwp-OST0001_UUID:3353:1862456998043072:0@lo:400:kworker/u8:0.0 00000100:00100000:2.0:1776178190.180022:0:20755:0:(service.c:2152:ptlrpc_server_handle_req_in()) got req x1862456998043328 00000100:00100000:2.0:1776178190.180132:0:20755:0:(nrs_fifo.c:179:nrs_fifo_req_get()) NRS start fifo request from 12345-0@lo, seq: 494 00000100:00100000:2.0:1776178190.180136:0:20755:0:(service.c:2324:ptlrpc_server_handle_request()) Handling RPC req@00000000c8bb8784 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_006:lustre-MDT0001-mdtlov_UUID+5:3353:x1862456998043328:12345-0@lo:400:kworker/u8:0.0 00000100:00100000:2.0:1776178190.180166:0:20755:0:(service.c:2372:ptlrpc_server_handle_request()) Handled RPC req@00000000c8bb8784 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_006:lustre-MDT0001-mdtlov_UUID+5:3353:x1862456998043328:12345-0@lo:400:kworker/u8:0.0 Request processed in 30us (611us total) trans 0 rc 0/0 00000100:00100000:2.0:1776178190.180172:0:20755:0:(nrs_fifo.c:241:nrs_fifo_req_stop()) NRS stop fifo request from 12345-0@lo, seq: 494 00000100:00100000:2.0:1776178190.180192:0:3353:0:(client.c:2215:ptlrpc_check_set()) Completed RPC req@00000000cc7f07e3 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_00:lustre-MDT0001-mdtlov_UUID:3353:1862456998043328:0@lo:400:kworker/u8:0.0 00000100:00100000:3.0:1776178190.189621:0:9554:0:(service.c:2372:ptlrpc_server_handle_request()) Handled RPC req@00000000c237605d pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_create00:lustre-MDT0000-mdtlov_UUID+5:3356:x1862456998042752:12345-0@lo:13:osp-pre-1-0.0 Request processed in 12901us (13097us total) trans 0 rc 0/0 00000100:00100000:3.0:1776178190.189642:0:9554:0:(nrs_fifo.c:241:nrs_fifo_req_stop()) NRS stop fifo request from 12345-0@lo, seq: 467 00000100:00100000:1.0:1776178190.201794:0:3356:0:(events.c:361:request_in_callback()) peer: 12345-0@lo (source: 12345-0@lo) 00000100:00100000:1.0:1776178190.201856:0:3356:0:(client.c:1733:ptlrpc_send_new_req()) Sending RPC req@00000000bebb3448 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_03:lustre-MDT0001-mdtlov_UUID:3356:1862456998043264:0@lo:400:kworker/u8:0.0 00000100:00100000:1.0:1776178190.201913:0:3356:0:(events.c:361:request_in_callback()) peer: 12345-0@lo (source: 12345-0@lo) 00000100:00100000:1.0:1776178190.201943:0:3356:0:(client.c:1733:ptlrpc_send_new_req()) Sending RPC req@000000007307bd05 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_03:lustre-MDT0001-mdtlov_UUID:3356:1862456998043520:0@lo:41:osp-pre-0-1.0 00000100:00100000:1.0:1776178190.201975:0:3356:0:(events.c:361:request_in_callback()) peer: 12345-0@lo (source: 12345-0@lo) 00000100:00100000:1.0:1776178190.201998:0:3356:0:(client.c:1733:ptlrpc_send_new_req()) Sending RPC req@00000000375ba3dd pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_03:lustre-MDT0000-lwp-MDT0000_UUID:3356:1862456998043584:0@lo:400:kworker/u8:0.0 00000100:00100000:1.0:1776178190.202017:0:3356:0:(events.c:361:request_in_callback()) peer: 12345-0@lo (source: 12345-0@lo) 00000100:00100000:2.0:1776178190.202414:0:11281:0:(service.c:2152:ptlrpc_server_handle_req_in()) got req x1862456998043008 00000100:00100000:1.0:1776178190.202420:0:15213:0:(service.c:2152:ptlrpc_server_handle_req_in()) got req x1862456998043520 00000100:00100000:1.0:1776178190.202462:0:15213:0:(nrs_fifo.c:179:nrs_fifo_req_get()) NRS start fifo request from 12345-0@lo, seq: 686 00000100:00100000:1.0:1776178190.202468:0:15213:0:(service.c:2324:ptlrpc_server_handle_request()) Handling RPC req@000000001c465e98 pname:cluuid+ref:pid:xid:nid:opc:job mdt_out00_003:lustre-MDT0001-mdtlov_UUID+6:3356:x1862456998043520:12345-0@lo:41:osp-pre-0-1.0 00000004:00000024:1.0:1776178190.202501:0:15213:0:(mdt_handler.c:548:mdt_statfs()) blocks cached 0 granted 2097152 pending 0 free 1446121472 avail 1316659200 00000100:00100000:2.0:1776178190.202566:0:11281:0:(service.c:2152:ptlrpc_server_handle_req_in()) got req x1862456998043584 00000100:00100000:2.0:1776178190.202583:0:11281:0:(nrs_fifo.c:179:nrs_fifo_req_get()) NRS start fifo request from 12345-0@lo, seq: 886 00000100:00100000:2.0:1776178190.202588:0:11281:0:(service.c:2324:ptlrpc_server_handle_request()) Handling RPC req@0000000098aa1ab6 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_004:lustre-MDT0001-lwp-OST0000_UUID+5:3356:x1862456998043008:12345-0@lo:400:kworker/u8:0.0 00000100:00100000:3.0:1776178190.202694:0:8250:0:(nrs_fifo.c:179:nrs_fifo_req_get()) NRS start fifo request from 12345-0@lo, seq: 887 00000100:00100000:3.0:1776178190.202700:0:8250:0:(service.c:2324:ptlrpc_server_handle_request()) Handling RPC req@0000000007075047 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_003:lustre-MDT0000-lwp-MDT0000_UUID+5:3356:x1862456998043584:12345-0@lo:400:kworker/u8:0.0 00000100:00100000:1.0:1776178190.202739:0:15213:0:(service.c:2372:ptlrpc_server_handle_request()) Handled RPC req@000000001c465e98 pname:cluuid+ref:pid:xid:nid:opc:job mdt_out00_003:lustre-MDT0001-mdtlov_UUID+6:3356:x1862456998043520:12345-0@lo:41:osp-pre-0-1.0 Request processed in 270us (764us total) trans 0 rc 0/0 00000100:00100000:1.0:1776178190.202746:0:15213:0:(nrs_fifo.c:241:nrs_fifo_req_stop()) NRS stop fifo request from 12345-0@lo, seq: 686 00000100:00100000:2.0:1776178190.209953:0:11281:0:(service.c:2372:ptlrpc_server_handle_request()) Handled RPC req@0000000098aa1ab6 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_004:lustre-MDT0001-lwp-OST0000_UUID+5:3356:x1862456998043008:12345-0@lo:400:kworker/u8:0.0 Request processed in 7342us (8155us total) trans 0 rc 0/0 00000100:00100000:2.0:1776178190.209986:0:11281:0:(nrs_fifo.c:241:nrs_fifo_req_stop()) NRS stop fifo request from 12345-0@lo, seq: 886 00000100:00100000:2.0:1776178190.210155:0:8227:0:(service.c:2152:ptlrpc_server_handle_req_in()) got req x1862456998043264 00000100:00100000:2.0:1776178190.210224:0:8227:0:(nrs_fifo.c:179:nrs_fifo_req_get()) NRS start fifo request from 12345-0@lo, seq: 495 00000100:00100000:2.0:1776178190.210230:0:8227:0:(service.c:2324:ptlrpc_server_handle_request()) Handling RPC req@00000000cc7f07e3 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_000:lustre-MDT0001-mdtlov_UUID+5:3356:x1862456998043264:12345-0@lo:400:kworker/u8:0.0 00000100:00100000:2.0:1776178190.211088:0:8227:0:(service.c:2372:ptlrpc_server_handle_request()) Handled RPC req@00000000cc7f07e3 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_000:lustre-MDT0001-mdtlov_UUID+5:3356:x1862456998043264:12345-0@lo:400:kworker/u8:0.0 Request processed in 824us (9142us total) trans 0 rc 0/0 00000100:00100000:2.0:1776178190.211097:0:8227:0:(nrs_fifo.c:241:nrs_fifo_req_stop()) NRS stop fifo request from 12345-0@lo, seq: 495 00000100:00100000:1.0:1776178190.257175:0:3356:0:(client.c:2215:ptlrpc_check_set()) Completed RPC req@0000000044f82452 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_03:lustre-MDT0001-lwp-OST0000_UUID:3356:1862456998043008:0@lo:400:kworker/u8:0.0 00000100:00100000:2.0:1776178190.257326:0:3356:0:(client.c:2215:ptlrpc_check_set()) Completed RPC req@00000000bebb3448 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_03:lustre-MDT0001-mdtlov_UUID:3356:1862456998043264:0@lo:400:kworker/u8:0.0 00000100:00100000:2.0:1776178190.257349:0:3356:0:(client.c:2215:ptlrpc_check_set()) Completed RPC req@000000007307bd05 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_03:lustre-MDT0001-mdtlov_UUID:3356:1862456998043520:0@lo:41:osp-pre-0-1.0 00000100:00100000:2.0:1776178190.257716:0:8250:0:(service.c:2372:ptlrpc_server_handle_request()) Handled RPC req@0000000007075047 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_003:lustre-MDT0000-lwp-MDT0000_UUID+5:3356:x1862456998043584:12345-0@lo:400:kworker/u8:0.0 Request processed in 55013us (55696us total) trans 0 rc 0/0 00000100:00100000:2.0:1776178190.257727:0:8250:0:(nrs_fifo.c:241:nrs_fifo_req_stop()) NRS stop fifo request from 12345-0@lo, seq: 887 00000100:00100000:1.0:1776178190.257795:0:3356:0:(client.c:2215:ptlrpc_check_set()) Completed RPC req@00000000375ba3dd pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_03:lustre-MDT0000-lwp-MDT0000_UUID:3356:1862456998043584:0@lo:400:kworker/u8:0.0 00000100:00100000:0.0:1776178190.263555:0:3354:0:(events.c:361:request_in_callback()) peer: 12345-0@lo (source: 12345-0@lo) 00000100:00100000:0.0:1776178190.265108:0:3354:0:(client.c:1733:ptlrpc_send_new_req()) Sending RPC req@00000000889a4355 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_01:lustre-MDT0001-lwp-OST0001_UUID:3354:1862456998043136:0@lo:400:kworker/u8:0.0 00000100:00100000:0.0:1776178190.265158:0:3354:0:(events.c:361:request_in_callback()) peer: 12345-0@lo (source: 12345-0@lo) 00000100:00100000:0.0:1776178190.265171:0:3354:0:(client.c:1733:ptlrpc_send_new_req()) Sending RPC req@00000000be94e5d7 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_01:lustre-MDT0000-lwp-MDT0001_UUID:3354:1862456998043392:0@lo:400:kworker/u8:0.0 00000100:00100000:0.0:1776178190.265189:0:3354:0:(events.c:361:request_in_callback()) peer: 12345-0@lo (source: 12345-0@lo) 00000100:00100000:2.0:1776178190.265295:0:8250:0:(service.c:2152:ptlrpc_server_handle_req_in()) got req x1862456998043136 00000100:00100000:2.0:1776178190.265346:0:8250:0:(service.c:2152:ptlrpc_server_handle_req_in()) got req x1862456998043392 00000100:00100000:2.0:1776178190.265364:0:8250:0:(nrs_fifo.c:179:nrs_fifo_req_get()) NRS start fifo request from 12345-0@lo, seq: 888 00000100:00100000:2.0:1776178190.265370:0:8250:0:(service.c:2324:ptlrpc_server_handle_request()) Handling RPC req@00000000cd102128 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_003:lustre-MDT0001-lwp-OST0001_UUID+5:3354:x1862456998043136:12345-0@lo:400:kworker/u8:0.0 00000100:00100000:2.0:1776178190.265423:0:8250:0:(service.c:2372:ptlrpc_server_handle_request()) Handled RPC req@00000000cd102128 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_003:lustre-MDT0001-lwp-OST0001_UUID+5:3354:x1862456998043136:12345-0@lo:400:kworker/u8:0.0 Request processed in 52us (265us total) trans 0 rc 0/0 00000100:00100000:2.0:1776178190.265430:0:8250:0:(nrs_fifo.c:241:nrs_fifo_req_stop()) NRS stop fifo request from 12345-0@lo, seq: 888 00000100:00100000:2.0:1776178190.265445:0:8250:0:(nrs_fifo.c:179:nrs_fifo_req_get()) NRS start fifo request from 12345-0@lo, seq: 889 00000100:00100000:2.0:1776178190.265449:0:8250:0:(service.c:2324:ptlrpc_server_handle_request()) Handling RPC req@0000000081ef7fa0 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_003:lustre-MDT0000-lwp-MDT0001_UUID+5:3354:x1862456998043392:12345-0@lo:400:kworker/u8:0.0 00000100:00100000:2.0:1776178190.265469:0:8250:0:(service.c:2372:ptlrpc_server_handle_request()) Handled RPC req@0000000081ef7fa0 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_003:lustre-MDT0000-lwp-MDT0001_UUID+5:3354:x1862456998043392:12345-0@lo:400:kworker/u8:0.0 Request processed in 20us (279us total) trans 0 rc 0/0 00000100:00100000:2.0:1776178190.265474:0:8250:0:(nrs_fifo.c:241:nrs_fifo_req_stop()) NRS stop fifo request from 12345-0@lo, seq: 889 00000100:00100000:1.0:1776178190.265662:0:20140:0:(service.c:2152:ptlrpc_server_handle_req_in()) got req x1862456998042880 00000100:00100000:1.0:1776178190.265698:0:20140:0:(nrs_fifo.c:179:nrs_fifo_req_get()) NRS start fifo request from 12345-0@lo, seq: 150 00000100:00100000:1.0:1776178190.265701:0:20140:0:(service.c:2324:ptlrpc_server_handle_request()) Handling RPC req@00000000b4b6b848 pname:cluuid+ref:pid:xid:nid:opc:job ll_mgs_0002:ac61f839-0e8e-4cac-9c59-08bcfbbad330+23:3354:x1862456998042880:12345-0@lo:400:kworker/u8:0.0 00000100:00100000:1.0:1776178190.265732:0:20140:0:(service.c:2372:ptlrpc_server_handle_request()) Handled RPC req@00000000b4b6b848 pname:cluuid+ref:pid:xid:nid:opc:job ll_mgs_0002:ac61f839-0e8e-4cac-9c59-08bcfbbad330+23:3354:x1862456998042880:12345-0@lo:400:kworker/u8:0.0 Request processed in 30us (2186us total) trans 0 rc 0/0 00000100:00100000:1.0:1776178190.265737:0:20140:0:(nrs_fifo.c:241:nrs_fifo_req_stop()) NRS stop fifo request from 12345-0@lo, seq: 150 00000100:00100000:0.0:1776178190.267143:0:3354:0:(client.c:2215:ptlrpc_check_set()) Completed RPC req@0000000054130213 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_01:ac61f839-0e8e-4cac-9c59-08bcfbbad330:3354:1862456998042880:0@lo:400:kworker/u8:0.0 00000100:00100000:0.0:1776178190.267161:0:3354:0:(client.c:2215:ptlrpc_check_set()) Completed RPC req@00000000889a4355 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_01:lustre-MDT0001-lwp-OST0001_UUID:3354:1862456998043136:0@lo:400:kworker/u8:0.0 00000100:00100000:0.0:1776178190.267171:0:3354:0:(client.c:2215:ptlrpc_check_set()) Completed RPC req@00000000be94e5d7 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_01:lustre-MDT0000-lwp-MDT0001_UUID:3354:1862456998043392:0@lo:400:kworker/u8:0.0 00010000:00010000:1.0:1776178190.816470:0:6245:0:(ldlm_lockd.c:2792:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:1.0:1776178191.651392:0:3348:0:(events.c:361:request_in_callback()) peer: 12345-192.168.204.3@tcp (source: 12345-192.168.204.3@tcp) 00000100:00100000:1.0:1776178191.651652:0:3348:0:(events.c:361:request_in_callback()) peer: 12345-192.168.204.3@tcp (source: 12345-192.168.204.3@tcp) 00000100:00100000:1.0:1776178191.651694:0:3348:0:(events.c:361:request_in_callback()) peer: 12345-192.168.204.3@tcp (source: 12345-192.168.204.3@tcp) 00000100:00100000:0.0:1776178191.651816:0:20755:0:(service.c:2152:ptlrpc_server_handle_req_in()) got req x1862456979097472 00000100:00100000:0.0:1776178191.651881:0:20755:0:(nrs_fifo.c:179:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.204.3@tcp, seq: 496 00000100:00100000:0.0:1776178191.651888:0:20755:0:(service.c:2324:ptlrpc_server_handle_request()) Handling RPC req@00000000be94e5d7 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_006:b50963bd-8eba-4432-9a54-8888d00e3fa1+9:2270:x1862456979097472:12345-192.168.204.3@tcp:400:kworker/u8:3.0 00000100:00100000:0.0:1776178191.651978:0:20755:0:(service.c:2372:ptlrpc_server_handle_request()) Handled RPC req@00000000be94e5d7 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_006:b50963bd-8eba-4432-9a54-8888d00e3fa1+9:2270:x1862456979097472:12345-192.168.204.3@tcp:400:kworker/u8:3.0 Request processed in 90us (284us total) trans 0 rc 0/0 00000100:00100000:0.0:1776178191.651987:0:20755:0:(nrs_fifo.c:241:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.204.3@tcp, seq: 496 00000100:00100000:3.0:1776178191.653027:0:6256:0:(service.c:2152:ptlrpc_server_handle_req_in()) got req x1862456979097344 00000100:00100000:3.0:1776178191.653092:0:6256:0:(nrs_fifo.c:179:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.204.3@tcp, seq: 890 00000100:00100000:3.0:1776178191.653097:0:6256:0:(service.c:2324:ptlrpc_server_handle_request()) Handling RPC req@00000000375ba3dd pname:cluuid+ref:pid:xid:nid:opc:job mdt00_001:b50963bd-8eba-4432-9a54-8888d00e3fa1+6:2270:x1862456979097344:12345-192.168.204.3@tcp:400:kworker/u8:3.0 00000100:00100000:3.0:1776178191.653342:0:6256:0:(service.c:2372:ptlrpc_server_handle_request()) Handled RPC req@00000000375ba3dd pname:cluuid+ref:pid:xid:nid:opc:job mdt00_001:b50963bd-8eba-4432-9a54-8888d00e3fa1+6:2270:x1862456979097344:12345-192.168.204.3@tcp:400:kworker/u8:3.0 Request processed in 243us (1689us total) trans 0 rc 0/0 00000100:00100000:3.0:1776178191.653349:0:6256:0:(nrs_fifo.c:241:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.204.3@tcp, seq: 890 00000100:00100000:3.0:1776178191.654102:0:3348:0:(events.c:361:request_in_callback()) peer: 12345-192.168.204.3@tcp (source: 12345-192.168.204.3@tcp) 00000100:00100000:0.0:1776178191.654261:0:11281:0:(service.c:2152:ptlrpc_server_handle_req_in()) got req x1862456979097280 00000100:00100000:0.0:1776178191.654293:0:11281:0:(nrs_fifo.c:179:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.204.3@tcp, seq: 891 00000100:00100000:0.0:1776178191.654297:0:11281:0:(service.c:2324:ptlrpc_server_handle_request()) Handling RPC req@00000000e82039bb pname:cluuid+ref:pid:xid:nid:opc:job mdt00_004:b50963bd-8eba-4432-9a54-8888d00e3fa1+7:2271:x1862456979097280:12345-192.168.204.3@tcp:400:kworker/u8:3.0 00000100:00100000:0.0:1776178191.654335:0:11281:0:(service.c:2372:ptlrpc_server_handle_request()) Handled RPC req@00000000e82039bb pname:cluuid+ref:pid:xid:nid:opc:job mdt00_004:b50963bd-8eba-4432-9a54-8888d00e3fa1+7:2271:x1862456979097280:12345-192.168.204.3@tcp:400:kworker/u8:3.0 Request processed in 38us (236us total) trans 0 rc 0/0 00000100:00100000:0.0:1776178191.654342:0:11281:0:(nrs_fifo.c:241:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.204.3@tcp, seq: 891 00000100:00100000:2.0:1776178191.656043:0:20137:0:(service.c:2152:ptlrpc_server_handle_req_in()) got req x1862456979097216 00000100:00100000:2.0:1776178191.656125:0:20137:0:(nrs_fifo.c:179:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.204.3@tcp, seq: 151 00000100:00100000:2.0:1776178191.656501:0:20137:0:(service.c:2324:ptlrpc_server_handle_request()) Handling RPC req@000000001333a9de pname:cluuid+ref:pid:xid:nid:opc:job ll_mgs_0000:5b5c66b1-1a2f-4651-8548-0c75b9a38a94+9:2270:x1862456979097216:12345-192.168.204.3@tcp:400:kworker/u8:3.0 00000100:00100000:2.0:1776178191.656567:0:20137:0:(service.c:2372:ptlrpc_server_handle_request()) Handled RPC req@000000001333a9de pname:cluuid+ref:pid:xid:nid:opc:job ll_mgs_0000:5b5c66b1-1a2f-4651-8548-0c75b9a38a94+9:2270:x1862456979097216:12345-192.168.204.3@tcp:400:kworker/u8:3.0 Request processed in 69us (5194us total) trans 0 rc 0/0 00000100:00100000:2.0:1776178191.656574:0:20137:0:(nrs_fifo.c:241:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.204.3@tcp, seq: 151 00000100:00100000:3.0:1776178191.658934:0:3348:0:(events.c:361:request_in_callback()) peer: 12345-192.168.204.3@tcp (source: 12345-192.168.204.3@tcp) 00000100:00100000:1.0:1776178191.660277:0:8227:0:(service.c:2152:ptlrpc_server_handle_req_in()) got req x1862456979097408 00000100:00100000:1.0:1776178191.660345:0:8227:0:(nrs_fifo.c:179:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.204.3@tcp, seq: 497 00000100:00100000:1.0:1776178191.660352:0:8227:0:(service.c:2324:ptlrpc_server_handle_request()) Handling RPC req@00000000d81deb92 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_000:b50963bd-8eba-4432-9a54-8888d00e3fa1+8:2271:x1862456979097408:12345-192.168.204.3@tcp:400:kworker/u8:3.0 00000100:00100000:1.0:1776178191.661300:0:8227:0:(service.c:2372:ptlrpc_server_handle_request()) Handled RPC req@00000000d81deb92 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_000:b50963bd-8eba-4432-9a54-8888d00e3fa1+8:2271:x1862456979097408:12345-192.168.204.3@tcp:400:kworker/u8:3.0 Request processed in 944us (2368us total) trans 0 rc 0/0 00000100:00100000:1.0:1776178191.661313:0:8227:0:(nrs_fifo.c:241:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.204.3@tcp, seq: 497 00000100:00100000:0.0:1776178191.714766:0:3354:0:(ptlrpcd.c:415:ptlrpcd_check()) transfer 1 async RPCs [0->1] 00000100:00100000:0.0:1776178191.714792:0:3354:0:(client.c:1733:ptlrpc_send_new_req()) Sending RPC req@0000000057946847 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_01:lustre-MDT0001-mdtlov_UUID:3354:1862456998043648:0@lo:13:osp-pre-1-1.0 00000100:00100000:0.0:1776178191.714865:0:3354:0:(events.c:361:request_in_callback()) peer: 12345-0@lo (source: 12345-0@lo) 00000100:00100000:3.0:1776178191.715017:0:9554:0:(service.c:2152:ptlrpc_server_handle_req_in()) got req x1862456998043648 00000100:00100000:3.0:1776178191.715098:0:9554:0:(nrs_fifo.c:179:nrs_fifo_req_get()) NRS start fifo request from 12345-0@lo, seq: 469 00000100:00100000:3.0:1776178191.715105:0:9554:0:(service.c:2324:ptlrpc_server_handle_request()) Handling RPC req@000000007307bd05 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_create00:lustre-MDT0001-mdtlov_UUID+5:3354:x1862456998043648:12345-0@lo:13:osp-pre-1-1.0 00002000:00000024:3.0:1776178191.715135:0:9554:0:(ofd_obd.c:740:ofd_statfs()) blocks cached 229376 granted 491746368 pending 0 free 3923689472 avail 3692167168 00000100:00100000:3.0:1776178191.715179:0:9554:0:(service.c:2372:ptlrpc_server_handle_request()) Handled RPC req@000000007307bd05 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_create00:lustre-MDT0001-mdtlov_UUID+5:3354:x1862456998043648:12345-0@lo:13:osp-pre-1-1.0 Request processed in 74us (315us total) trans 0 rc 0/0 00000100:00100000:3.0:1776178191.715187:0:9554:0:(nrs_fifo.c:241:nrs_fifo_req_stop()) NRS stop fifo request from 12345-0@lo, seq: 469 00000100:00100000:0.0:1776178191.715300:0:3354:0:(client.c:2215:ptlrpc_check_set()) Completed RPC req@0000000057946847 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_01:lustre-MDT0001-mdtlov_UUID:3354:1862456998043648:0@lo:13:osp-pre-1-1.0 00010000:00010000:2.0:1776178191.842894:0:6244:0:(ldlm_lockd.c:2792:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:2.0:1776178192.224747:0:3354:0:(client.c:1733:ptlrpc_send_new_req()) Sending RPC req@000000004eee88be pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_01:lustre-MDT0001-mdtlov_UUID:3354:1862456998043712:0@lo:13:osp-pre-0-1.0 00000100:00100000:2.0:1776178192.224823:0:3354:0:(events.c:361:request_in_callback()) peer: 12345-0@lo (source: 12345-0@lo) 00000100:00100000:0.0:1776178192.225169:0:8232:0:(service.c:2152:ptlrpc_server_handle_req_in()) got req x1862456998043712 00000100:00100000:0.0:1776178192.225229:0:8232:0:(nrs_fifo.c:179:nrs_fifo_req_get()) NRS start fifo request from 12345-0@lo, seq: 470 00000100:00100000:0.0:1776178192.225236:0:8232:0:(service.c:2324:ptlrpc_server_handle_request()) Handling RPC req@000000001f88d899 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_create00:lustre-MDT0001-mdtlov_UUID+5:3354:x1862456998043712:12345-0@lo:13:osp-pre-0-1.0 00002000:00000024:0.0:1776178192.225267:0:8232:0:(ofd_obd.c:740:ofd_statfs()) blocks cached 315392 granted 484883776 pending 0 free 3923689472 avail 3692167168 00000100:00100000:0.0:1776178192.225322:0:8232:0:(service.c:2372:ptlrpc_server_handle_request()) Handled RPC req@000000001f88d899 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_create00:lustre-MDT0001-mdtlov_UUID+5:3354:x1862456998043712:12345-0@lo:13:osp-pre-0-1.0 Request processed in 86us (499us total) trans 0 rc 0/0 00000100:00100000:0.0:1776178192.225329:0:8232:0:(nrs_fifo.c:241:nrs_fifo_req_stop()) NRS stop fifo request from 12345-0@lo, seq: 470 00000100:00100000:2.0:1776178192.225411:0:3354:0:(client.c:2215:ptlrpc_check_set()) Completed RPC req@000000004eee88be pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_01:lustre-MDT0001-mdtlov_UUID:3354:1862456998043712:0@lo:13:osp-pre-0-1.0 00010000:00010000:2.0:1776178192.864638:0:9225:0:(ldlm_lockd.c:2792:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00010000:00010000:2.0:1776178193.890706:0:9224:0:(ldlm_lockd.c:2792:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00010000:00010000:1.0:1776178194.913956:0:6245:0:(ldlm_lockd.c:2792:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:0.0:1776178195.296508:0:3356:0:(client.c:1733:ptlrpc_send_new_req()) Sending RPC req@0000000067c1ae4f pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_03:lustre-MDT0000-lwp-OST0000_UUID:3356:1862456998043840:0@lo:400:kworker/u8:0.0 00000100:00100000:0.0:1776178195.296594:0:3356:0:(events.c:361:request_in_callback()) peer: 12345-0@lo (source: 12345-0@lo) 00000100:00100000:0.0:1776178195.296637:0:3356:0:(ptlrpcd.c:415:ptlrpcd_check()) transfer 1 async RPCs [2->3] 00000100:00100000:0.0:1776178195.296643:0:3356:0:(client.c:1733:ptlrpc_send_new_req()) Sending RPC req@00000000bebb3448 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_03:ac61f839-0e8e-4cac-9c59-08bcfbbad330:3356:1862456998043776:0@lo:400:kworker/u8:0.0 00000100:00100000:0.0:1776178195.296691:0:3356:0:(events.c:361:request_in_callback()) peer: 12345-0@lo (source: 12345-0@lo) 00000100:00100000:0.0:1776178195.296738:0:20140:0:(service.c:2152:ptlrpc_server_handle_req_in()) got req x1862456998043776 00000100:00100000:0.0:1776178195.296787:0:20140:0:(nrs_fifo.c:179:nrs_fifo_req_get()) NRS start fifo request from 12345-0@lo, seq: 152 00000100:00100000:0.0:1776178195.296793:0:20140:0:(service.c:2324:ptlrpc_server_handle_request()) Handling RPC req@00000000b4857168 pname:cluuid+ref:pid:xid:nid:opc:job ll_mgs_0002:ac61f839-0e8e-4cac-9c59-08bcfbbad330+23:3356:x1862456998043776:12345-0@lo:400:kworker/u8:0.0 00000100:00100000:0.0:1776178195.296839:0:20140:0:(service.c:2372:ptlrpc_server_handle_request()) Handled RPC req@00000000b4857168 pname:cluuid+ref:pid:xid:nid:opc:job ll_mgs_0002:ac61f839-0e8e-4cac-9c59-08bcfbbad330+23:3356:x1862456998043776:12345-0@lo:400:kworker/u8:0.0 Request processed in 46us (149us total) trans 0 rc 0/0 00000100:00100000:0.0:1776178195.296847:0:20140:0:(nrs_fifo.c:241:nrs_fifo_req_stop()) NRS stop fifo request from 12345-0@lo, seq: 152 00000100:00100000:0.0:1776178195.296911:0:3356:0:(client.c:2215:ptlrpc_check_set()) Completed RPC req@00000000bebb3448 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_03:ac61f839-0e8e-4cac-9c59-08bcfbbad330:3356:1862456998043776:0@lo:400:kworker/u8:0.0 00000100:00100000:0.0:1776178195.296956:0:6256:0:(service.c:2152:ptlrpc_server_handle_req_in()) got req x1862456998043840 00000100:00100000:0.0:1776178195.296988:0:6256:0:(nrs_fifo.c:179:nrs_fifo_req_get()) NRS start fifo request from 12345-0@lo, seq: 892 00000100:00100000:0.0:1776178195.296992:0:6256:0:(service.c:2324:ptlrpc_server_handle_request()) Handling RPC req@000000004eee88be pname:cluuid+ref:pid:xid:nid:opc:job mdt00_001:lustre-MDT0000-lwp-OST0000_UUID+5:3356:x1862456998043840:12345-0@lo:400:kworker/u8:0.0 00000100:00100000:0.0:1776178195.297027:0:6256:0:(service.c:2372:ptlrpc_server_handle_request()) Handled RPC req@000000004eee88be pname:cluuid+ref:pid:xid:nid:opc:job mdt00_001:lustre-MDT0000-lwp-OST0000_UUID+5:3356:x1862456998043840:12345-0@lo:400:kworker/u8:0.0 Request processed in 34us (434us total) trans 0 rc 0/0 00000100:00100000:0.0:1776178195.297033:0:6256:0:(nrs_fifo.c:241:nrs_fifo_req_stop()) NRS stop fifo request from 12345-0@lo, seq: 892 00000100:00100000:0.0:1776178195.297088:0:3356:0:(client.c:2215:ptlrpc_check_set()) Completed RPC req@0000000067c1ae4f pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_03:lustre-MDT0000-lwp-OST0000_UUID:3356:1862456998043840:0@lo:400:kworker/u8:0.0 00000100:00100000:0.0:1776178195.297403:0:3356:0:(client.c:1733:ptlrpc_send_new_req()) Sending RPC req@0000000067c1ae4f pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_03:lustre-MDT0001-mdtlov_UUID:3356:1862456998044096:0@lo:400:kworker/u8:0.0 00000100:00100000:0.0:1776178195.297433:0:3356:0:(events.c:361:request_in_callback()) peer: 12345-0@lo (source: 12345-0@lo) 00000100:00100000:0.0:1776178195.297459:0:3356:0:(client.c:1733:ptlrpc_send_new_req()) Sending RPC req@00000000dad865ab pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_03:lustre-MDT0000-mdtlov_UUID:3356:1862456998044352:0@lo:400:kworker/u8:0.0 00000100:00100000:0.0:1776178195.297478:0:3356:0:(events.c:361:request_in_callback()) peer: 12345-0@lo (source: 12345-0@lo) 00000100:00100000:0.0:1776178195.297490:0:3356:0:(ptlrpcd.c:415:ptlrpcd_check()) transfer 3 async RPCs [2->3] 00000100:00100000:0.0:1776178195.297495:0:3356:0:(client.c:1733:ptlrpc_send_new_req()) Sending RPC req@00000000103d2c47 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_03:lustre-MDT0001-lwp-OST0001_UUID:3356:1862456998044032:0@lo:400:kworker/u8:0.0 00000100:00100000:0.0:1776178195.297513:0:3356:0:(events.c:361:request_in_callback()) peer: 12345-0@lo (source: 12345-0@lo) 00000100:00100000:0.0:1776178195.297524:0:3356:0:(client.c:1733:ptlrpc_send_new_req()) Sending RPC req@000000005039d2a8 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_03:lustre-MDT0000-lwp-MDT0001_UUID:3356:1862456998044288:0@lo:400:kworker/u8:0.0 00000100:00100000:0.0:1776178195.297540:0:3356:0:(events.c:361:request_in_callback()) peer: 12345-0@lo (source: 12345-0@lo) 00000100:00100000:0.0:1776178195.297549:0:3356:0:(client.c:1733:ptlrpc_send_new_req()) Sending RPC req@000000008f3792e7 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_03:lustre-MDT0000-lwp-MDT0000_UUID:3356:1862456998044544:0@lo:400:kworker/u8:0.0 00000100:00100000:0.0:1776178195.297564:0:3356:0:(events.c:361:request_in_callback()) peer: 12345-0@lo (source: 12345-0@lo) 00000100:00100000:0.0:1776178195.297585:0:11281:0:(service.c:2152:ptlrpc_server_handle_req_in()) got req x1862456998044032 00000100:00100000:0.0:1776178195.297606:0:11281:0:(service.c:2152:ptlrpc_server_handle_req_in()) got req x1862456998044288 00000100:00100000:0.0:1776178195.297619:0:11281:0:(service.c:2152:ptlrpc_server_handle_req_in()) got req x1862456998044544 00000100:00100000:0.0:1776178195.297629:0:11281:0:(nrs_fifo.c:179:nrs_fifo_req_get()) NRS start fifo request from 12345-0@lo, seq: 893 00000100:00100000:0.0:1776178195.297633:0:11281:0:(service.c:2324:ptlrpc_server_handle_request()) Handling RPC req@000000003fa4e1db pname:cluuid+ref:pid:xid:nid:opc:job mdt00_004:lustre-MDT0001-lwp-OST0001_UUID+5:3356:x1862456998044032:12345-0@lo:400:kworker/u8:0.0 00000100:00100000:0.0:1776178195.297665:0:11281:0:(service.c:2372:ptlrpc_server_handle_request()) Handled RPC req@000000003fa4e1db pname:cluuid+ref:pid:xid:nid:opc:job mdt00_004:lustre-MDT0001-lwp-OST0001_UUID+5:3356:x1862456998044032:12345-0@lo:400:kworker/u8:0.0 Request processed in 31us (151us total) trans 0 rc 0/0 00000100:00100000:0.0:1776178195.297670:0:11281:0:(nrs_fifo.c:241:nrs_fifo_req_stop()) NRS stop fifo request from 12345-0@lo, seq: 893 00000100:00100000:0.0:1776178195.297680:0:11281:0:(nrs_fifo.c:179:nrs_fifo_req_get()) NRS start fifo request from 12345-0@lo, seq: 894 00000100:00100000:0.0:1776178195.297684:0:11281:0:(service.c:2324:ptlrpc_server_handle_request()) Handling RPC req@00000000f52e98d8 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_004:lustre-MDT0000-lwp-MDT0001_UUID+5:3356:x1862456998044288:12345-0@lo:400:kworker/u8:0.0 00000100:00100000:0.0:1776178195.297707:0:11281:0:(service.c:2372:ptlrpc_server_handle_request()) Handled RPC req@00000000f52e98d8 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_004:lustre-MDT0000-lwp-MDT0001_UUID+5:3356:x1862456998044288:12345-0@lo:400:kworker/u8:0.0 Request processed in 23us (167us total) trans 0 rc 0/0 00000100:00100000:0.0:1776178195.297713:0:11281:0:(nrs_fifo.c:241:nrs_fifo_req_stop()) NRS stop fifo request from 12345-0@lo, seq: 894 00000100:00100000:0.0:1776178195.297724:0:11281:0:(nrs_fifo.c:179:nrs_fifo_req_get()) NRS start fifo request from 12345-0@lo, seq: 895 00000100:00100000:0.0:1776178195.297727:0:11281:0:(service.c:2324:ptlrpc_server_handle_request()) Handling RPC req@000000007794b3ac pname:cluuid+ref:pid:xid:nid:opc:job mdt00_004:lustre-MDT0000-lwp-MDT0000_UUID+5:3356:x1862456998044544:12345-0@lo:400:kworker/u8:0.0 00000100:00100000:0.0:1776178195.297750:0:11281:0:(service.c:2372:ptlrpc_server_handle_request()) Handled RPC req@000000007794b3ac pname:cluuid+ref:pid:xid:nid:opc:job mdt00_004:lustre-MDT0000-lwp-MDT0000_UUID+5:3356:x1862456998044544:12345-0@lo:400:kworker/u8:0.0 Request processed in 22us (186us total) trans 0 rc 0/0 00000100:00100000:0.0:1776178195.297756:0:11281:0:(nrs_fifo.c:241:nrs_fifo_req_stop()) NRS stop fifo request from 12345-0@lo, seq: 895 00000100:00100000:0.0:1776178195.297786:0:3356:0:(client.c:2215:ptlrpc_check_set()) Completed RPC req@00000000103d2c47 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_03:lustre-MDT0001-lwp-OST0001_UUID:3356:1862456998044032:0@lo:400:kworker/u8:0.0 00000100:00100000:0.0:1776178195.297795:0:3356:0:(client.c:2215:ptlrpc_check_set()) Completed RPC req@000000005039d2a8 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_03:lustre-MDT0000-lwp-MDT0001_UUID:3356:1862456998044288:0@lo:400:kworker/u8:0.0 00000100:00100000:0.0:1776178195.297801:0:3356:0:(client.c:2215:ptlrpc_check_set()) Completed RPC req@000000008f3792e7 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_03:lustre-MDT0000-lwp-MDT0000_UUID:3356:1862456998044544:0@lo:400:kworker/u8:0.0 00000100:00100000:1.0:1776178195.297972:0:6261:0:(service.c:2152:ptlrpc_server_handle_req_in()) got req x1862456998044096 00000100:00100000:1.0:1776178195.298007:0:6261:0:(service.c:2152:ptlrpc_server_handle_req_in()) got req x1862456998044352 00000100:00100000:1.0:1776178195.298024:0:6261:0:(nrs_fifo.c:179:nrs_fifo_req_get()) NRS start fifo request from 12345-0@lo, seq: 687 00000100:00100000:1.0:1776178195.298027:0:6261:0:(service.c:2324:ptlrpc_server_handle_request()) Handling RPC req@00000000bebb3448 pname:cluuid+ref:pid:xid:nid:opc:job mdt_out00_001:lustre-MDT0001-mdtlov_UUID+6:3356:x1862456998044096:12345-0@lo:400:kworker/u8:0.0 00000100:00100000:1.0:1776178195.298085:0:6261:0:(service.c:2372:ptlrpc_server_handle_request()) Handled RPC req@00000000bebb3448 pname:cluuid+ref:pid:xid:nid:opc:job mdt_out00_001:lustre-MDT0001-mdtlov_UUID+6:3356:x1862456998044096:12345-0@lo:400:kworker/u8:0.0 Request processed in 56us (651us total) trans 0 rc 0/0 00000100:00100000:1.0:1776178195.298091:0:6261:0:(nrs_fifo.c:241:nrs_fifo_req_stop()) NRS stop fifo request from 12345-0@lo, seq: 687 00000100:00100000:1.0:1776178195.298103:0:6261:0:(nrs_fifo.c:179:nrs_fifo_req_get()) NRS start fifo request from 12345-0@lo, seq: 688 00000100:00100000:1.0:1776178195.298107:0:6261:0:(service.c:2324:ptlrpc_server_handle_request()) Handling RPC req@00000000660d337f pname:cluuid+ref:pid:xid:nid:opc:job mdt_out00_001:lustre-MDT0000-mdtlov_UUID+5:3356:x1862456998044352:12345-0@lo:400:kworker/u8:0.0 00000100:00100000:1.0:1776178195.298132:0:6261:0:(service.c:2372:ptlrpc_server_handle_request()) Handled RPC req@00000000660d337f pname:cluuid+ref:pid:xid:nid:opc:job mdt_out00_001:lustre-MDT0000-mdtlov_UUID+5:3356:x1862456998044352:12345-0@lo:400:kworker/u8:0.0 Request processed in 24us (653us total) trans 0 rc 0/0 00000100:00100000:1.0:1776178195.298137:0:6261:0:(nrs_fifo.c:241:nrs_fifo_req_stop()) NRS stop fifo request from 12345-0@lo, seq: 688 00000100:00100000:1.0:1776178195.298180:0:3356:0:(client.c:2215:ptlrpc_check_set()) Completed RPC req@0000000067c1ae4f pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_03:lustre-MDT0001-mdtlov_UUID:3356:1862456998044096:0@lo:400:kworker/u8:0.0 00000100:00100000:1.0:1776178195.298188:0:3356:0:(client.c:2215:ptlrpc_check_set()) Completed RPC req@00000000dad865ab pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_03:lustre-MDT0000-mdtlov_UUID:3356:1862456998044352:0@lo:400:kworker/u8:0.0 00000100:00100000:3.0:1776178195.298404:0:3353:0:(client.c:1733:ptlrpc_send_new_req()) Sending RPC req@00000000250bc347 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_00:lustre-MDT0001-lwp-OST0000_UUID:3353:1862456998043904:0@lo:400:kworker/u8:0.0 00000100:00100000:3.0:1776178195.298437:0:3353:0:(events.c:361:request_in_callback()) peer: 12345-0@lo (source: 12345-0@lo) 00000100:00100000:3.0:1776178195.298454:0:3353:0:(client.c:1733:ptlrpc_send_new_req()) Sending RPC req@00000000c832ce5d pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_00:lustre-MDT0001-mdtlov_UUID:3353:1862456998044160:0@lo:400:kworker/u8:0.0 00000100:00100000:3.0:1776178195.298478:0:3353:0:(events.c:361:request_in_callback()) peer: 12345-0@lo (source: 12345-0@lo) 00000100:00100000:3.0:1776178195.298496:0:3353:0:(client.c:1733:ptlrpc_send_new_req()) Sending RPC req@0000000028ec8437 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_00:lustre-MDT0000-mdtlov_UUID:3353:1862456998044416:0@lo:400:kworker/u8:0.0 00000100:00100000:3.0:1776178195.298515:0:3353:0:(events.c:361:request_in_callback()) peer: 12345-0@lo (source: 12345-0@lo) 00000100:00100000:3.0:1776178195.298533:0:3353:0:(ptlrpcd.c:415:ptlrpcd_check()) transfer 3 async RPCs [1->0] 00000100:00100000:3.0:1776178195.298539:0:3353:0:(client.c:1733:ptlrpc_send_new_req()) Sending RPC req@000000001e11faca pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_00:lustre-MDT0000-lwp-OST0001_UUID:3353:1862456998043968:0@lo:400:kworker/u8:0.0 00000100:00100000:3.0:1776178195.298563:0:3353:0:(events.c:361:request_in_callback()) peer: 12345-0@lo (source: 12345-0@lo) 00000100:00100000:3.0:1776178195.298577:0:3353:0:(client.c:1733:ptlrpc_send_new_req()) Sending RPC req@000000003174f31d pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_00:lustre-MDT0001-mdtlov_UUID:3353:1862456998044224:0@lo:400:kworker/u8:0.0 00000100:00100000:3.0:1776178195.298600:0:3353:0:(events.c:361:request_in_callback()) peer: 12345-0@lo (source: 12345-0@lo) 00000100:00100000:3.0:1776178195.298615:0:3353:0:(client.c:1733:ptlrpc_send_new_req()) Sending RPC req@000000007bc8dbc1 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_00:lustre-MDT0000-mdtlov_UUID:3353:1862456998044480:0@lo:400:kworker/u8:0.0 00000100:00100000:3.0:1776178195.298635:0:3353:0:(events.c:361:request_in_callback()) peer: 12345-0@lo (source: 12345-0@lo) 00000100:00100000:3.0:1776178195.298661:0:8227:0:(service.c:2152:ptlrpc_server_handle_req_in()) got req x1862456998044160 00000100:00100000:3.0:1776178195.298689:0:8227:0:(service.c:2152:ptlrpc_server_handle_req_in()) got req x1862456998044416 00000100:00100000:3.0:1776178195.298703:0:8227:0:(service.c:2152:ptlrpc_server_handle_req_in()) got req x1862456998044224 00000100:00100000:3.0:1776178195.298716:0:8227:0:(service.c:2152:ptlrpc_server_handle_req_in()) got req x1862456998044480 00000100:00100000:3.0:1776178195.298731:0:8227:0:(nrs_fifo.c:179:nrs_fifo_req_get()) NRS start fifo request from 12345-0@lo, seq: 498 00000100:00100000:3.0:1776178195.298735:0:8227:0:(service.c:2324:ptlrpc_server_handle_request()) Handling RPC req@0000000007e413b7 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_000:lustre-MDT0001-mdtlov_UUID+5:3353:x1862456998044160:12345-0@lo:400:kworker/u8:0.0 00000100:00100000:3.0:1776178195.298771:0:8227:0:(service.c:2372:ptlrpc_server_handle_request()) Handled RPC req@0000000007e413b7 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_000:lustre-MDT0001-mdtlov_UUID+5:3353:x1862456998044160:12345-0@lo:400:kworker/u8:0.0 Request processed in 35us (293us total) trans 0 rc 0/0 00000100:00100000:3.0:1776178195.298778:0:8227:0:(nrs_fifo.c:241:nrs_fifo_req_stop()) NRS stop fifo request from 12345-0@lo, seq: 498 00000100:00100000:3.0:1776178195.298789:0:8227:0:(nrs_fifo.c:179:nrs_fifo_req_get()) NRS start fifo request from 12345-0@lo, seq: 499 00000100:00100000:3.0:1776178195.298792:0:8227:0:(service.c:2324:ptlrpc_server_handle_request()) Handling RPC req@00000000f45ed991 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_000:lustre-MDT0000-mdtlov_UUID+5:3353:x1862456998044416:12345-0@lo:400:kworker/u8:0.0 00000100:00100000:3.0:1776178195.298816:0:8227:0:(service.c:2372:ptlrpc_server_handle_request()) Handled RPC req@00000000f45ed991 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_000:lustre-MDT0000-mdtlov_UUID+5:3353:x1862456998044416:12345-0@lo:400:kworker/u8:0.0 Request processed in 23us (301us total) trans 0 rc 0/0 00000100:00100000:3.0:1776178195.298821:0:8227:0:(nrs_fifo.c:241:nrs_fifo_req_stop()) NRS stop fifo request from 12345-0@lo, seq: 499 00000100:00100000:3.0:1776178195.298831:0:8227:0:(nrs_fifo.c:179:nrs_fifo_req_get()) NRS start fifo request from 12345-0@lo, seq: 500 00000100:00100000:3.0:1776178195.298834:0:8227:0:(service.c:2324:ptlrpc_server_handle_request()) Handling RPC req@0000000022264af5 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_000:lustre-MDT0001-mdtlov_UUID+5:3353:x1862456998044224:12345-0@lo:400:kworker/u8:0.0 00000100:00100000:3.0:1776178195.298856:0:8227:0:(service.c:2372:ptlrpc_server_handle_request()) Handled RPC req@0000000022264af5 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_000:lustre-MDT0001-mdtlov_UUID+5:3353:x1862456998044224:12345-0@lo:400:kworker/u8:0.0 Request processed in 21us (256us total) trans 0 rc 0/0 00000100:00100000:3.0:1776178195.298861:0:8227:0:(nrs_fifo.c:241:nrs_fifo_req_stop()) NRS stop fifo request from 12345-0@lo, seq: 500 00000100:00100000:3.0:1776178195.298885:0:8227:0:(nrs_fifo.c:179:nrs_fifo_req_get()) NRS start fifo request from 12345-0@lo, seq: 501 00000100:00100000:3.0:1776178195.298888:0:8227:0:(service.c:2324:ptlrpc_server_handle_request()) Handling RPC req@000000004c88c002 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_000:lustre-MDT0000-mdtlov_UUID+5:3353:x1862456998044480:12345-0@lo:400:kworker/u8:0.0 00000100:00100000:3.0:1776178195.298913:0:8227:0:(service.c:2372:ptlrpc_server_handle_request()) Handled RPC req@000000004c88c002 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_000:lustre-MDT0000-mdtlov_UUID+5:3353:x1862456998044480:12345-0@lo:400:kworker/u8:0.0 Request processed in 24us (278us total) trans 0 rc 0/0 00000100:00100000:3.0:1776178195.298918:0:8227:0:(nrs_fifo.c:241:nrs_fifo_req_stop()) NRS stop fifo request from 12345-0@lo, seq: 501 00000100:00100000:3.0:1776178195.298954:0:6255:0:(service.c:2152:ptlrpc_server_handle_req_in()) got req x1862456998043904 00000100:00100000:3.0:1776178195.298976:0:6255:0:(service.c:2152:ptlrpc_server_handle_req_in()) got req x1862456998043968 00000100:00100000:3.0:1776178195.298991:0:6255:0:(nrs_fifo.c:179:nrs_fifo_req_get()) NRS start fifo request from 12345-0@lo, seq: 896 00000100:00100000:3.0:1776178195.298994:0:6255:0:(service.c:2324:ptlrpc_server_handle_request()) Handling RPC req@00000000c37c187e pname:cluuid+ref:pid:xid:nid:opc:job mdt00_000:lustre-MDT0001-lwp-OST0000_UUID+5:3353:x1862456998043904:12345-0@lo:400:kworker/u8:0.0 00000100:00100000:3.0:1776178195.299021:0:6255:0:(service.c:2372:ptlrpc_server_handle_request()) Handled RPC req@00000000c37c187e pname:cluuid+ref:pid:xid:nid:opc:job mdt00_000:lustre-MDT0001-lwp-OST0000_UUID+5:3353:x1862456998043904:12345-0@lo:400:kworker/u8:0.0 Request processed in 25us (584us total) trans 0 rc 0/0 00000100:00100000:3.0:1776178195.299026:0:6255:0:(nrs_fifo.c:241:nrs_fifo_req_stop()) NRS stop fifo request from 12345-0@lo, seq: 896 00000100:00100000:3.0:1776178195.299035:0:6255:0:(nrs_fifo.c:179:nrs_fifo_req_get()) NRS start fifo request from 12345-0@lo, seq: 897 00000100:00100000:3.0:1776178195.299038:0:6255:0:(service.c:2324:ptlrpc_server_handle_request()) Handling RPC req@0000000023da22ef pname:cluuid+ref:pid:xid:nid:opc:job mdt00_000:lustre-MDT0000-lwp-OST0001_UUID+5:3353:x1862456998043968:12345-0@lo:400:kworker/u8:0.0 00000100:00100000:3.0:1776178195.299094:0:6255:0:(service.c:2372:ptlrpc_server_handle_request()) Handled RPC req@0000000023da22ef pname:cluuid+ref:pid:xid:nid:opc:job mdt00_000:lustre-MDT0000-lwp-OST0001_UUID+5:3353:x1862456998043968:12345-0@lo:400:kworker/u8:0.0 Request processed in 54us (530us total) trans 0 rc 0/0 00000100:00100000:3.0:1776178195.299099:0:6255:0:(nrs_fifo.c:241:nrs_fifo_req_stop()) NRS stop fifo request from 12345-0@lo, seq: 897 00000100:00100000:3.0:1776178195.299133:0:3353:0:(client.c:2215:ptlrpc_check_set()) Completed RPC req@000000001e11faca pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_00:lustre-MDT0000-lwp-OST0001_UUID:3353:1862456998043968:0@lo:400:kworker/u8:0.0 00000100:00100000:3.0:1776178195.299157:0:3353:0:(client.c:2215:ptlrpc_check_set()) Completed RPC req@000000003174f31d pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_00:lustre-MDT0001-mdtlov_UUID:3353:1862456998044224:0@lo:400:kworker/u8:0.0 00000100:00100000:3.0:1776178195.299167:0:3353:0:(client.c:2215:ptlrpc_check_set()) Completed RPC req@000000007bc8dbc1 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_00:lustre-MDT0000-mdtlov_UUID:3353:1862456998044480:0@lo:400:kworker/u8:0.0 00000100:00100000:3.0:1776178195.299175:0:3353:0:(client.c:2215:ptlrpc_check_set()) Completed RPC req@00000000250bc347 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_00:lustre-MDT0001-lwp-OST0000_UUID:3353:1862456998043904:0@lo:400:kworker/u8:0.0 00000100:00100000:3.0:1776178195.299183:0:3353:0:(client.c:2215:ptlrpc_check_set()) Completed RPC req@00000000c832ce5d pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_00:lustre-MDT0001-mdtlov_UUID:3353:1862456998044160:0@lo:400:kworker/u8:0.0 00000100:00100000:3.0:1776178195.299190:0:3353:0:(client.c:2215:ptlrpc_check_set()) Completed RPC req@0000000028ec8437 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_00:lustre-MDT0000-mdtlov_UUID:3353:1862456998044416:0@lo:400:kworker/u8:0.0 00000100:00100000:1.0:1776178195.299512:0:3356:0:(client.c:1733:ptlrpc_send_new_req()) Sending RPC req@0000000057946847 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_03:lustre-MDT0000-mdtlov_UUID:3356:1862456998044608:0@lo:13:osp-pre-1-0.0 00000100:00100000:1.0:1776178195.299547:0:3356:0:(events.c:361:request_in_callback()) peer: 12345-0@lo (source: 12345-0@lo) 00000100:00100000:3.0:1776178195.299684:0:9554:0:(service.c:2152:ptlrpc_server_handle_req_in()) got req x1862456998044608 00000100:00100000:3.0:1776178195.299717:0:9554:0:(nrs_fifo.c:179:nrs_fifo_req_get()) NRS start fifo request from 12345-0@lo, seq: 471 00000100:00100000:3.0:1776178195.299721:0:9554:0:(service.c:2324:ptlrpc_server_handle_request()) Handling RPC req@00000000dad865ab pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_create00:lustre-MDT0000-mdtlov_UUID+5:3356:x1862456998044608:12345-0@lo:13:osp-pre-1-0.0 00002000:00000024:3.0:1776178195.299747:0:9554:0:(ofd_obd.c:740:ofd_statfs()) blocks cached 229376 granted 491746368 pending 0 free 3923689472 avail 3692167168 00000100:00100000:3.0:1776178195.299792:0:9554:0:(service.c:2372:ptlrpc_server_handle_request()) Handled RPC req@00000000dad865ab pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_create00:lustre-MDT0000-mdtlov_UUID+5:3356:x1862456998044608:12345-0@lo:13:osp-pre-1-0.0 Request processed in 70us (245us total) trans 0 rc 0/0 00000100:00100000:3.0:1776178195.299798:0:9554:0:(nrs_fifo.c:241:nrs_fifo_req_stop()) NRS stop fifo request from 12345-0@lo, seq: 471 00000100:00100000:1.0:1776178195.299996:0:3356:0:(client.c:2215:ptlrpc_check_set()) Completed RPC req@0000000057946847 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_03:lustre-MDT0000-mdtlov_UUID:3356:1862456998044608:0@lo:13:osp-pre-1-0.0 00000100:00100000:2.0:1776178195.302383:0:3354:0:(ptlrpcd.c:415:ptlrpcd_check()) transfer 1 async RPCs [0->1] 00000100:00100000:2.0:1776178195.302415:0:3354:0:(client.c:1733:ptlrpc_send_new_req()) Sending RPC req@000000004943ee04 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_01:lustre-MDT0001-mdtlov_UUID:3354:1862456998044672:0@lo:41:osp-pre-0-1.0 00000100:00100000:2.0:1776178195.302491:0:3354:0:(events.c:361:request_in_callback()) peer: 12345-0@lo (source: 12345-0@lo) 00000100:00100000:0.0:1776178195.302713:0:6260:0:(service.c:2152:ptlrpc_server_handle_req_in()) got req x1862456998044672 00000100:00100000:0.0:1776178195.302773:0:6260:0:(nrs_fifo.c:179:nrs_fifo_req_get()) NRS start fifo request from 12345-0@lo, seq: 689 00000100:00100000:0.0:1776178195.302780:0:6260:0:(service.c:2324:ptlrpc_server_handle_request()) Handling RPC req@00000000fdb309c4 pname:cluuid+ref:pid:xid:nid:opc:job mdt_out00_000:lustre-MDT0001-mdtlov_UUID+6:3354:x1862456998044672:12345-0@lo:41:osp-pre-0-1.0 00000004:00000024:0.0:1776178195.302819:0:6260:0:(mdt_handler.c:548:mdt_statfs()) blocks cached 0 granted 2097152 pending 0 free 1446121472 avail 1316659200 00000100:00100000:0.0:1776178195.302886:0:6260:0:(service.c:2372:ptlrpc_server_handle_request()) Handled RPC req@00000000fdb309c4 pname:cluuid+ref:pid:xid:nid:opc:job mdt_out00_000:lustre-MDT0001-mdtlov_UUID+6:3354:x1862456998044672:12345-0@lo:41:osp-pre-0-1.0 Request processed in 104us (394us total) trans 0 rc 0/0 00000100:00100000:0.0:1776178195.302893:0:6260:0:(nrs_fifo.c:241:nrs_fifo_req_stop()) NRS stop fifo request from 12345-0@lo, seq: 689 00000100:00100000:0.0:1776178195.302948:0:3354:0:(client.c:2215:ptlrpc_check_set()) Completed RPC req@000000004943ee04 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_01:lustre-MDT0001-mdtlov_UUID:3354:1862456998044672:0@lo:41:osp-pre-0-1.0 00000100:00100000:0.0:1776178195.303554:0:3354:0:(client.c:1733:ptlrpc_send_new_req()) Sending RPC req@00000000f62fe06f pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_01:lustre-MDT0000-mdtlov_UUID:3354:1862456998044736:0@lo:13:osp-pre-0-0.0 00000100:00100000:0.0:1776178195.303592:0:3354:0:(events.c:361:request_in_callback()) peer: 12345-0@lo (source: 12345-0@lo) 00000100:00100000:3.0:1776178195.305294:0:8232:0:(service.c:2152:ptlrpc_server_handle_req_in()) got req x1862456998044736 00000100:00100000:3.0:1776178195.305381:0:8232:0:(nrs_fifo.c:179:nrs_fifo_req_get()) NRS start fifo request from 12345-0@lo, seq: 472 00000100:00100000:3.0:1776178195.305387:0:8232:0:(service.c:2324:ptlrpc_server_handle_request()) Handling RPC req@000000004943ee04 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_create00:lustre-MDT0000-mdtlov_UUID+5:3354:x1862456998044736:12345-0@lo:13:osp-pre-0-0.0 00002000:00000024:3.0:1776178195.305421:0:8232:0:(ofd_obd.c:740:ofd_statfs()) blocks cached 315392 granted 484883776 pending 0 free 3923689472 avail 3692167168 00000100:00100000:3.0:1776178195.305499:0:8232:0:(service.c:2372:ptlrpc_server_handle_request()) Handled RPC req@000000004943ee04 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_create00:lustre-MDT0000-mdtlov_UUID+5:3354:x1862456998044736:12345-0@lo:13:osp-pre-0-0.0 Request processed in 111us (1907us total) trans 0 rc 0/0 00000100:00100000:3.0:1776178195.305505:0:8232:0:(nrs_fifo.c:241:nrs_fifo_req_stop()) NRS stop fifo request from 12345-0@lo, seq: 472 00000100:00100000:0.0:1776178195.305706:0:3354:0:(client.c:2215:ptlrpc_check_set()) Completed RPC req@00000000f62fe06f pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_01:lustre-MDT0000-mdtlov_UUID:3354:1862456998044736:0@lo:13:osp-pre-0-0.0 00000100:00100000:2.0:1776178195.306506:0:3355:0:(client.c:1733:ptlrpc_send_new_req()) Sending RPC req@00000000c008a6fb pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_02:lustre-MDT0000-mdtlov_UUID:3355:1862456998044800:0@lo:41:osp-pre-1-0.0 00000100:00100000:2.0:1776178195.306558:0:3355:0:(events.c:361:request_in_callback()) peer: 12345-0@lo (source: 12345-0@lo) 00000100:00100000:0.0:1776178195.306635:0:14128:0:(service.c:2152:ptlrpc_server_handle_req_in()) got req x1862456998044800 00000100:00100000:0.0:1776178195.306682:0:14128:0:(nrs_fifo.c:179:nrs_fifo_req_get()) NRS start fifo request from 12345-0@lo, seq: 690 00000100:00100000:0.0:1776178195.306687:0:14128:0:(service.c:2324:ptlrpc_server_handle_request()) Handling RPC req@00000000b5990455 pname:cluuid+ref:pid:xid:nid:opc:job mdt_out00_002:lustre-MDT0000-mdtlov_UUID+5:3355:x1862456998044800:12345-0@lo:41:osp-pre-1-0.0 00000004:00000024:0.0:1776178195.306713:0:14128:0:(mdt_handler.c:548:mdt_statfs()) blocks cached 0 granted 2097152 pending 0 free 1446035456 avail 1316573184 00000100:00100000:0.0:1776178195.314004:0:14128:0:(service.c:2372:ptlrpc_server_handle_request()) Handled RPC req@00000000b5990455 pname:cluuid+ref:pid:xid:nid:opc:job mdt_out00_002:lustre-MDT0000-mdtlov_UUID+5:3355:x1862456998044800:12345-0@lo:41:osp-pre-1-0.0 Request processed in 7304us (7434us total) trans 0 rc 0/0 00000100:00100000:0.0:1776178195.314025:0:14128:0:(nrs_fifo.c:241:nrs_fifo_req_stop()) NRS stop fifo request from 12345-0@lo, seq: 690 00000100:00100000:2.0:1776178195.314339:0:3355:0:(client.c:2215:ptlrpc_check_set()) Completed RPC req@00000000c008a6fb pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_02:lustre-MDT0000-mdtlov_UUID:3355:1862456998044800:0@lo:41:osp-pre-1-0.0 00010000:00010000:2.0:1776178195.936319:0:6244:0:(ldlm_lockd.c:2792:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:3.0:1776178196.771993:0:3346:0:(events.c:361:request_in_callback()) peer: 12345-192.168.204.3@tcp (source: 12345-192.168.204.3@tcp) 00000100:00100000:3.0:1776178196.772046:0:3346:0:(events.c:361:request_in_callback()) peer: 12345-192.168.204.3@tcp (source: 12345-192.168.204.3@tcp) 00000100:00100000:3.0:1776178196.772129:0:3346:0:(events.c:361:request_in_callback()) peer: 12345-192.168.204.3@tcp (source: 12345-192.168.204.3@tcp) 00000100:00100000:3.0:1776178196.772149:0:3346:0:(events.c:361:request_in_callback()) peer: 12345-192.168.204.3@tcp (source: 12345-192.168.204.3@tcp) 00000100:00100000:3.0:1776178196.772176:0:3346:0:(events.c:361:request_in_callback()) peer: 12345-192.168.204.3@tcp (source: 12345-192.168.204.3@tcp) 00000100:00100000:3.0:1776178196.772211:0:11281:0:(service.c:2152:ptlrpc_server_handle_req_in()) got req x1862456979097664 00000100:00100000:3.0:1776178196.772277:0:11281:0:(service.c:2152:ptlrpc_server_handle_req_in()) got req x1862456979097600 00000100:00100000:3.0:1776178196.772298:0:11281:0:(nrs_fifo.c:179:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.204.3@tcp, seq: 898 00000100:00100000:3.0:1776178196.772315:0:11281:0:(service.c:2324:ptlrpc_server_handle_request()) Handling RPC req@00000000dd52d0d0 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_004:b50963bd-8eba-4432-9a54-8888d00e3fa1+6:2271:x1862456979097664:12345-192.168.204.3@tcp:400:kworker/u8:3.0 00000100:00100000:3.0:1776178196.772390:0:11281:0:(service.c:2372:ptlrpc_server_handle_request()) Handled RPC req@00000000dd52d0d0 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_004:b50963bd-8eba-4432-9a54-8888d00e3fa1+6:2271:x1862456979097664:12345-192.168.204.3@tcp:400:kworker/u8:3.0 Request processed in 75us (345us total) trans 0 rc 0/0 00000100:00100000:3.0:1776178196.772398:0:11281:0:(nrs_fifo.c:241:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.204.3@tcp, seq: 898 00000100:00100000:3.0:1776178196.772417:0:11281:0:(nrs_fifo.c:179:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.204.3@tcp, seq: 899 00000100:00100000:3.0:1776178196.772421:0:11281:0:(service.c:2324:ptlrpc_server_handle_request()) Handling RPC req@000000008a6fdb8d pname:cluuid+ref:pid:xid:nid:opc:job mdt00_004:b50963bd-8eba-4432-9a54-8888d00e3fa1+7:2271:x1862456979097600:12345-192.168.204.3@tcp:400:kworker/u8:3.0 00000100:00100000:3.0:1776178196.772446:0:11281:0:(service.c:2372:ptlrpc_server_handle_request()) Handled RPC req@000000008a6fdb8d pname:cluuid+ref:pid:xid:nid:opc:job mdt00_004:b50963bd-8eba-4432-9a54-8888d00e3fa1+7:2271:x1862456979097600:12345-192.168.204.3@tcp:400:kworker/u8:3.0 Request processed in 24us (298us total) trans 0 rc 0/0 00000100:00100000:3.0:1776178196.772453:0:11281:0:(nrs_fifo.c:241:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.204.3@tcp, seq: 899 00000100:00100000:2.0:1776178196.773019:0:20769:0:(service.c:2152:ptlrpc_server_handle_req_in()) got req x1862456979097792 00000100:00100000:2.0:1776178196.773109:0:20769:0:(service.c:2152:ptlrpc_server_handle_req_in()) got req x1862456979097728 00000100:00100000:2.0:1776178196.773135:0:20769:0:(nrs_fifo.c:179:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.204.3@tcp, seq: 502 00000100:00100000:2.0:1776178196.773139:0:20769:0:(service.c:2324:ptlrpc_server_handle_request()) Handling RPC req@0000000087a31eb2 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_018:b50963bd-8eba-4432-9a54-8888d00e3fa1+9:2271:x1862456979097792:12345-192.168.204.3@tcp:400:kworker/u8:3.0 00000100:00100000:2.0:1776178196.773189:0:20769:0:(service.c:2372:ptlrpc_server_handle_request()) Handled RPC req@0000000087a31eb2 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_018:b50963bd-8eba-4432-9a54-8888d00e3fa1+9:2271:x1862456979097792:12345-192.168.204.3@tcp:400:kworker/u8:3.0 Request processed in 48us (1059us total) trans 0 rc 0/0 00000100:00100000:2.0:1776178196.773195:0:20769:0:(nrs_fifo.c:241:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.204.3@tcp, seq: 502 00000100:00100000:2.0:1776178196.773209:0:20769:0:(nrs_fifo.c:179:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.204.3@tcp, seq: 503 00000100:00100000:2.0:1776178196.773213:0:20769:0:(service.c:2324:ptlrpc_server_handle_request()) Handling RPC req@0000000008605f3d pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_018:b50963bd-8eba-4432-9a54-8888d00e3fa1+8:2271:x1862456979097728:12345-192.168.204.3@tcp:400:kworker/u8:3.0 00000100:00100000:2.0:1776178196.773237:0:20769:0:(service.c:2372:ptlrpc_server_handle_request()) Handled RPC req@0000000008605f3d pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_018:b50963bd-8eba-4432-9a54-8888d00e3fa1+8:2271:x1862456979097728:12345-192.168.204.3@tcp:400:kworker/u8:3.0 Request processed in 23us (1061us total) trans 0 rc 0/0 00000100:00100000:2.0:1776178196.773243:0:20769:0:(nrs_fifo.c:241:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.204.3@tcp, seq: 503 00000100:00100000:2.0:1776178196.773278:0:20137:0:(service.c:2152:ptlrpc_server_handle_req_in()) got req x1862456979097536 00000100:00100000:2.0:1776178196.773316:0:20137:0:(nrs_fifo.c:179:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.204.3@tcp, seq: 153 00000100:00100000:2.0:1776178196.773320:0:20137:0:(service.c:2324:ptlrpc_server_handle_request()) Handling RPC req@0000000088793b53 pname:cluuid+ref:pid:xid:nid:opc:job ll_mgs_0000:5b5c66b1-1a2f-4651-8548-0c75b9a38a94+9:2271:x1862456979097536:12345-192.168.204.3@tcp:400:kworker/u8:3.0 00000100:00100000:2.0:1776178196.773346:0:20137:0:(service.c:2372:ptlrpc_server_handle_request()) Handled RPC req@0000000088793b53 pname:cluuid+ref:pid:xid:nid:opc:job ll_mgs_0000:5b5c66b1-1a2f-4651-8548-0c75b9a38a94+9:2271:x1862456979097536:12345-192.168.204.3@tcp:400:kworker/u8:3.0 Request processed in 25us (1363us total) trans 0 rc 0/0 00000100:00100000:2.0:1776178196.773353:0:20137:0:(nrs_fifo.c:241:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.204.3@tcp, seq: 153 00000100:00100000:1.0:1776178196.833124:0:3356:0:(client.c:1733:ptlrpc_send_new_req()) Sending RPC req@00000000f62fe06f pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_03:lustre-MDT0001-mdtlov_UUID:3356:1862456998044864:0@lo:13:osp-pre-1-1.0 00000100:00100000:1.0:1776178196.833212:0:3356:0:(events.c:361:request_in_callback()) peer: 12345-0@lo (source: 12345-0@lo) 00000100:00100000:2.0:1776178196.833516:0:9554:0:(service.c:2152:ptlrpc_server_handle_req_in()) got req x1862456998044864 00000100:00100000:2.0:1776178196.833578:0:9554:0:(nrs_fifo.c:179:nrs_fifo_req_get()) NRS start fifo request from 12345-0@lo, seq: 473 00000100:00100000:2.0:1776178196.833585:0:9554:0:(service.c:2324:ptlrpc_server_handle_request()) Handling RPC req@000000007d20d045 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_create00:lustre-MDT0001-mdtlov_UUID+5:3356:x1862456998044864:12345-0@lo:13:osp-pre-1-1.0 00002000:00000024:2.0:1776178196.833635:0:9554:0:(ofd_obd.c:740:ofd_statfs()) blocks cached 229376 granted 491746368 pending 0 free 3923689472 avail 3692167168 00000100:00100000:2.0:1776178196.839451:0:9554:0:(service.c:2372:ptlrpc_server_handle_request()) Handled RPC req@000000007d20d045 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_create00:lustre-MDT0001-mdtlov_UUID+5:3356:x1862456998044864:12345-0@lo:13:osp-pre-1-1.0 Request processed in 5856us (6230us total) trans 0 rc 0/0 00000100:00100000:2.0:1776178196.839467:0:9554:0:(nrs_fifo.c:241:nrs_fifo_req_stop()) NRS stop fifo request from 12345-0@lo, seq: 473 00000100:00100000:1.0:1776178196.840409:0:3356:0:(client.c:2215:ptlrpc_check_set()) Completed RPC req@00000000f62fe06f pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_03:lustre-MDT0001-mdtlov_UUID:3356:1862456998044864:0@lo:13:osp-pre-1-1.0 00010000:00010000:2.0:1776178196.961460:0:9225:0:(ldlm_lockd.c:2792:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:2.0:1776178197.345162:0:3353:0:(client.c:1733:ptlrpc_send_new_req()) Sending RPC req@000000007608d8e9 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_00:lustre-MDT0001-mdtlov_UUID:3353:1862456998044928:0@lo:13:osp-pre-0-1.0 00000100:00100000:2.0:1776178197.345515:0:3353:0:(events.c:361:request_in_callback()) peer: 12345-0@lo (source: 12345-0@lo) 00000100:00100000:3.0:1776178197.353547:0:9554:0:(service.c:2152:ptlrpc_server_handle_req_in()) got req x1862456998044928 00000100:00100000:3.0:1776178197.353635:0:9554:0:(nrs_fifo.c:179:nrs_fifo_req_get()) NRS start fifo request from 12345-0@lo, seq: 474 00000100:00100000:3.0:1776178197.353642:0:9554:0:(service.c:2324:ptlrpc_server_handle_request()) Handling RPC req@0000000050bb92d0 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_create00:lustre-MDT0001-mdtlov_UUID+5:3353:x1862456998044928:12345-0@lo:13:osp-pre-0-1.0 00002000:00000024:3.0:1776178197.353677:0:9554:0:(ofd_obd.c:740:ofd_statfs()) blocks cached 315392 granted 484883776 pending 0 free 3923689472 avail 3692167168 00000100:00100000:3.0:1776178197.353864:0:9554:0:(service.c:2372:ptlrpc_server_handle_request()) Handled RPC req@0000000050bb92d0 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_create00:lustre-MDT0001-mdtlov_UUID+5:3353:x1862456998044928:12345-0@lo:13:osp-pre-0-1.0 Request processed in 221us (8351us total) trans 0 rc 0/0 00000100:00100000:3.0:1776178197.353873:0:9554:0:(nrs_fifo.c:241:nrs_fifo_req_stop()) NRS stop fifo request from 12345-0@lo, seq: 474 00000100:00100000:2.0:1776178197.367916:0:3353:0:(client.c:2215:ptlrpc_check_set()) Completed RPC req@000000007608d8e9 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_00:lustre-MDT0001-mdtlov_UUID:3353:1862456998044928:0@lo:13:osp-pre-0-1.0 00010000:00010000:2.0:1776178197.985091:0:9224:0:(ldlm_lockd.c:2792:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00010000:00010000:1.0:1776178199.008275:0:6245:0:(ldlm_lockd.c:2792:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00010000:00010000:2.0:1776178200.034096:0:6244:0:(ldlm_lockd.c:2792:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:1.0:1776178200.416423:0:3356:0:(client.c:1733:ptlrpc_send_new_req()) Sending RPC req@00000000da9f88c0 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_03:lustre-MDT0001-lwp-OST0000_UUID:3356:1862456998045120:0@lo:400:kworker/u8:2.0 00000100:00100000:2.0:1776178200.416439:0:3353:0:(client.c:1733:ptlrpc_send_new_req()) Sending RPC req@000000007c8ec83c pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_00:lustre-MDT0000-lwp-OST0001_UUID:3353:1862456998045184:0@lo:400:kworker/u8:2.0 00000100:00100000:3.0:1776178200.416529:0:3355:0:(client.c:1733:ptlrpc_send_new_req()) Sending RPC req@00000000fcc87e3b pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_02:lustre-MDT0000-lwp-OST0000_UUID:3355:1862456998045056:0@lo:400:kworker/u8:2.0 00000100:00100000:0.0:1776178200.416616:0:3354:0:(client.c:1733:ptlrpc_send_new_req()) Sending RPC req@00000000430a7ee3 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_01:ac61f839-0e8e-4cac-9c59-08bcfbbad330:3354:1862456998044992:0@lo:400:kworker/u8:2.0 00000100:00100000:2.0:1776178200.417002:0:3353:0:(events.c:361:request_in_callback()) peer: 12345-0@lo (source: 12345-0@lo) 00000100:00100000:2.0:1776178200.417034:0:3353:0:(client.c:1733:ptlrpc_send_new_req()) Sending RPC req@000000005e1c4771 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_00:lustre-MDT0001-mdtlov_UUID:3353:1862456998045440:0@lo:400:kworker/u8:2.0 00000100:00100000:2.0:1776178200.417057:0:3353:0:(events.c:361:request_in_callback()) peer: 12345-0@lo (source: 12345-0@lo) 00000100:00100000:2.0:1776178200.417981:0:3353:0:(client.c:1733:ptlrpc_send_new_req()) Sending RPC req@00000000f8049845 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_00:lustre-MDT0000-mdtlov_UUID:3353:1862456998045696:0@lo:400:kworker/u8:2.0 00000100:00100000:2.0:1776178200.418017:0:3353:0:(events.c:361:request_in_callback()) peer: 12345-0@lo (source: 12345-0@lo) 00000100:00100000:2.0:1776178200.418145:0:8228:0:(service.c:2152:ptlrpc_server_handle_req_in()) got req x1862456998045440 00000100:00100000:2.0:1776178200.418274:0:8228:0:(service.c:2152:ptlrpc_server_handle_req_in()) got req x1862456998045696 00000100:00100000:2.0:1776178200.418297:0:8228:0:(nrs_fifo.c:179:nrs_fifo_req_get()) NRS start fifo request from 12345-0@lo, seq: 504 00000100:00100000:2.0:1776178200.418303:0:8228:0:(service.c:2324:ptlrpc_server_handle_request()) Handling RPC req@00000000ad37fc5d pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_001:lustre-MDT0001-mdtlov_UUID+5:3353:x1862456998045440:12345-0@lo:400:kworker/u8:2.0 00000100:00100000:2.0:1776178200.418355:0:8228:0:(service.c:2372:ptlrpc_server_handle_request()) Handled RPC req@00000000ad37fc5d pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_001:lustre-MDT0001-mdtlov_UUID+5:3353:x1862456998045440:12345-0@lo:400:kworker/u8:2.0 Request processed in 51us (1297us total) trans 0 rc 0/0 00000100:00100000:2.0:1776178200.418362:0:8228:0:(nrs_fifo.c:241:nrs_fifo_req_stop()) NRS stop fifo request from 12345-0@lo, seq: 504 00000100:00100000:2.0:1776178200.418380:0:8228:0:(nrs_fifo.c:179:nrs_fifo_req_get()) NRS start fifo request from 12345-0@lo, seq: 505 00000100:00100000:2.0:1776178200.418384:0:8228:0:(service.c:2324:ptlrpc_server_handle_request()) Handling RPC req@00000000033734be pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_001:lustre-MDT0000-mdtlov_UUID+5:3353:x1862456998045696:12345-0@lo:400:kworker/u8:2.0 00000100:00100000:2.0:1776178200.418408:0:8228:0:(service.c:2372:ptlrpc_server_handle_request()) Handled RPC req@00000000033734be pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_001:lustre-MDT0000-mdtlov_UUID+5:3353:x1862456998045696:12345-0@lo:400:kworker/u8:2.0 Request processed in 24us (391us total) trans 0 rc 0/0 00000100:00100000:2.0:1776178200.418414:0:8228:0:(nrs_fifo.c:241:nrs_fifo_req_stop()) NRS stop fifo request from 12345-0@lo, seq: 505 00000100:00100000:2.0:1776178200.418456:0:3353:0:(client.c:2215:ptlrpc_check_set()) Completed RPC req@000000005e1c4771 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_00:lustre-MDT0001-mdtlov_UUID:3353:1862456998045440:0@lo:400:kworker/u8:2.0 00000100:00100000:2.0:1776178200.418465:0:3353:0:(client.c:2215:ptlrpc_check_set()) Completed RPC req@00000000f8049845 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_00:lustre-MDT0000-mdtlov_UUID:3353:1862456998045696:0@lo:400:kworker/u8:2.0 00000100:00100000:3.0:1776178200.418641:0:3355:0:(events.c:361:request_in_callback()) peer: 12345-0@lo (source: 12345-0@lo) 00000100:00100000:3.0:1776178200.418660:0:3355:0:(client.c:1733:ptlrpc_send_new_req()) Sending RPC req@00000000465ea19a pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_02:lustre-MDT0001-mdtlov_UUID:3355:1862456998045312:0@lo:400:kworker/u8:2.0 00000100:00100000:3.0:1776178200.418683:0:3355:0:(events.c:361:request_in_callback()) peer: 12345-0@lo (source: 12345-0@lo) 00000100:00100000:3.0:1776178200.418698:0:3355:0:(client.c:1733:ptlrpc_send_new_req()) Sending RPC req@0000000045aa39de pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_02:lustre-MDT0000-mdtlov_UUID:3355:1862456998045568:0@lo:400:kworker/u8:2.0 00000100:00100000:3.0:1776178200.418714:0:3355:0:(events.c:361:request_in_callback()) peer: 12345-0@lo (source: 12345-0@lo) 00000100:00100000:3.0:1776178200.418726:0:3355:0:(client.c:1733:ptlrpc_send_new_req()) Sending RPC req@00000000f62fe06f pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_02:lustre-MDT0000-mdtlov_UUID:3355:1862456998045824:0@lo:13:osp-pre-1-0.0 00000100:00100000:3.0:1776178200.418748:0:3355:0:(events.c:361:request_in_callback()) peer: 12345-0@lo (source: 12345-0@lo) 00000100:00100000:3.0:1776178200.418778:0:3355:0:(ptlrpcd.c:415:ptlrpcd_check()) transfer 1 async RPCs [3->2] 00000100:00100000:3.0:1776178200.418786:0:3355:0:(client.c:1733:ptlrpc_send_new_req()) Sending RPC req@00000000ead84f6b pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_02:lustre-MDT0000-mdtlov_UUID:3355:1862456998045888:0@lo:13:osp-pre-0-0.0 00000100:00100000:3.0:1776178200.418809:0:3355:0:(events.c:361:request_in_callback()) peer: 12345-0@lo (source: 12345-0@lo) 00000100:00100000:3.0:1776178200.418836:0:6256:0:(service.c:2152:ptlrpc_server_handle_req_in()) got req x1862456998045184 00000100:00100000:3.0:1776178200.418864:0:6256:0:(service.c:2152:ptlrpc_server_handle_req_in()) got req x1862456998045056 00000100:00100000:3.0:1776178200.418878:0:6256:0:(nrs_fifo.c:179:nrs_fifo_req_get()) NRS start fifo request from 12345-0@lo, seq: 900 00000100:00100000:3.0:1776178200.418882:0:6256:0:(service.c:2324:ptlrpc_server_handle_request()) Handling RPC req@00000000ad228ff6 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_001:lustre-MDT0000-lwp-OST0001_UUID+5:3353:x1862456998045184:12345-0@lo:400:kworker/u8:2.0 00000100:00100000:3.0:1776178200.418920:0:6256:0:(service.c:2372:ptlrpc_server_handle_request()) Handled RPC req@00000000ad228ff6 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_001:lustre-MDT0000-lwp-OST0001_UUID+5:3353:x1862456998045184:12345-0@lo:400:kworker/u8:2.0 Request processed in 38us (1921us total) trans 0 rc 0/0 00000100:00100000:3.0:1776178200.418927:0:6256:0:(nrs_fifo.c:241:nrs_fifo_req_stop()) NRS stop fifo request from 12345-0@lo, seq: 900 00000100:00100000:3.0:1776178200.418938:0:6256:0:(nrs_fifo.c:179:nrs_fifo_req_get()) NRS start fifo request from 12345-0@lo, seq: 901 00000100:00100000:3.0:1776178200.418954:0:6256:0:(service.c:2324:ptlrpc_server_handle_request()) Handling RPC req@0000000013f6d767 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_001:lustre-MDT0000-lwp-OST0000_UUID+5:3355:x1862456998045056:12345-0@lo:400:kworker/u8:2.0 00000100:00100000:3.0:1776178200.418993:0:6256:0:(service.c:2372:ptlrpc_server_handle_request()) Handled RPC req@0000000013f6d767 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_001:lustre-MDT0000-lwp-OST0000_UUID+5:3355:x1862456998045056:12345-0@lo:400:kworker/u8:2.0 Request processed in 51us (353us total) trans 0 rc 0/0 00000100:00100000:2.0:1776178200.419000:0:3353:0:(client.c:2215:ptlrpc_check_set()) Completed RPC req@000000007c8ec83c pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_00:lustre-MDT0000-lwp-OST0001_UUID:3353:1862456998045184:0@lo:400:kworker/u8:2.0 00000100:00100000:3.0:1776178200.419001:0:6256:0:(nrs_fifo.c:241:nrs_fifo_req_stop()) NRS stop fifo request from 12345-0@lo, seq: 901 00000100:00100000:3.0:1776178200.419024:0:6260:0:(service.c:2152:ptlrpc_server_handle_req_in()) got req x1862456998045312 00000100:00100000:3.0:1776178200.419049:0:6260:0:(service.c:2152:ptlrpc_server_handle_req_in()) got req x1862456998045568 00000100:00100000:3.0:1776178200.419083:0:6260:0:(nrs_fifo.c:179:nrs_fifo_req_get()) NRS start fifo request from 12345-0@lo, seq: 691 00000100:00100000:3.0:1776178200.419088:0:6260:0:(service.c:2324:ptlrpc_server_handle_request()) Handling RPC req@0000000096181210 pname:cluuid+ref:pid:xid:nid:opc:job mdt_out00_000:lustre-MDT0001-mdtlov_UUID+6:3355:x1862456998045312:12345-0@lo:400:kworker/u8:2.0 00000100:00100000:3.0:1776178200.419119:0:6260:0:(service.c:2372:ptlrpc_server_handle_request()) Handled RPC req@0000000096181210 pname:cluuid+ref:pid:xid:nid:opc:job mdt_out00_000:lustre-MDT0001-mdtlov_UUID+6:3355:x1862456998045312:12345-0@lo:400:kworker/u8:2.0 Request processed in 30us (435us total) trans 0 rc 0/0 00000100:00100000:3.0:1776178200.419125:0:6260:0:(nrs_fifo.c:241:nrs_fifo_req_stop()) NRS stop fifo request from 12345-0@lo, seq: 691 00000100:00100000:3.0:1776178200.419137:0:6260:0:(nrs_fifo.c:179:nrs_fifo_req_get()) NRS start fifo request from 12345-0@lo, seq: 692 00000100:00100000:3.0:1776178200.419141:0:6260:0:(service.c:2324:ptlrpc_server_handle_request()) Handling RPC req@000000005459e48f pname:cluuid+ref:pid:xid:nid:opc:job mdt_out00_000:lustre-MDT0000-mdtlov_UUID+5:3355:x1862456998045568:12345-0@lo:400:kworker/u8:2.0 00000100:00100000:2.0:1776178200.419155:0:9554:0:(service.c:2152:ptlrpc_server_handle_req_in()) got req x1862456998045824 00000100:00100000:3.0:1776178200.419167:0:6260:0:(service.c:2372:ptlrpc_server_handle_request()) Handled RPC req@000000005459e48f pname:cluuid+ref:pid:xid:nid:opc:job mdt_out00_000:lustre-MDT0000-mdtlov_UUID+5:3355:x1862456998045568:12345-0@lo:400:kworker/u8:2.0 Request processed in 26us (452us total) trans 0 rc 0/0 00000100:00100000:3.0:1776178200.419172:0:6260:0:(nrs_fifo.c:241:nrs_fifo_req_stop()) NRS stop fifo request from 12345-0@lo, seq: 692 00000100:00100000:3.0:1776178200.419214:0:3355:0:(client.c:2215:ptlrpc_check_set()) Completed RPC req@00000000fcc87e3b pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_02:lustre-MDT0000-lwp-OST0000_UUID:3355:1862456998045056:0@lo:400:kworker/u8:2.0 00000100:00100000:3.0:1776178200.419223:0:3355:0:(client.c:2215:ptlrpc_check_set()) Completed RPC req@00000000465ea19a pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_02:lustre-MDT0001-mdtlov_UUID:3355:1862456998045312:0@lo:400:kworker/u8:2.0 00000100:00100000:3.0:1776178200.419231:0:3355:0:(client.c:2215:ptlrpc_check_set()) Completed RPC req@0000000045aa39de pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_02:lustre-MDT0000-mdtlov_UUID:3355:1862456998045568:0@lo:400:kworker/u8:2.0 00000100:00100000:3.0:1776178200.419271:0:8232:0:(service.c:2152:ptlrpc_server_handle_req_in()) got req x1862456998045888 00000100:00100000:3.0:1776178200.419293:0:8232:0:(nrs_fifo.c:179:nrs_fifo_req_get()) NRS start fifo request from 12345-0@lo, seq: 475 00000100:00100000:3.0:1776178200.419296:0:8232:0:(service.c:2324:ptlrpc_server_handle_request()) Handling RPC req@0000000039f3bccf pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_create00:lustre-MDT0000-mdtlov_UUID+5:3355:x1862456998045888:12345-0@lo:13:osp-pre-0-0.0 00002000:00000024:3.0:1776178200.419323:0:8232:0:(ofd_obd.c:740:ofd_statfs()) blocks cached 315392 granted 484883776 pending 0 free 3923689472 avail 3692167168 00000100:00100000:2.0:1776178200.419361:0:9554:0:(nrs_fifo.c:179:nrs_fifo_req_get()) NRS start fifo request from 12345-0@lo, seq: 476 00000100:00100000:3.0:1776178200.419364:0:8232:0:(service.c:2372:ptlrpc_server_handle_request()) Handled RPC req@0000000039f3bccf pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_create00:lustre-MDT0000-mdtlov_UUID+5:3355:x1862456998045888:12345-0@lo:13:osp-pre-0-0.0 Request processed in 65us (554us total) trans 0 rc 0/0 00000100:00100000:2.0:1776178200.419369:0:9554:0:(service.c:2324:ptlrpc_server_handle_request()) Handling RPC req@000000002cf093e4 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_create00:lustre-MDT0000-mdtlov_UUID+5:3355:x1862456998045824:12345-0@lo:13:osp-pre-1-0.0 00000100:00100000:3.0:1776178200.419376:0:8232:0:(nrs_fifo.c:241:nrs_fifo_req_stop()) NRS stop fifo request from 12345-0@lo, seq: 475 00002000:00000024:2.0:1776178200.419391:0:9554:0:(ofd_obd.c:740:ofd_statfs()) blocks cached 229376 granted 491746368 pending 0 free 3923689472 avail 3692167168 00000100:00100000:2.0:1776178200.419420:0:9554:0:(service.c:2372:ptlrpc_server_handle_request()) Handled RPC req@000000002cf093e4 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_create00:lustre-MDT0000-mdtlov_UUID+5:3355:x1862456998045824:12345-0@lo:13:osp-pre-1-0.0 Request processed in 53us (673us total) trans 0 rc 0/0 00000100:00100000:2.0:1776178200.419426:0:9554:0:(nrs_fifo.c:241:nrs_fifo_req_stop()) NRS stop fifo request from 12345-0@lo, seq: 476 00000100:00100000:2.0:1776178200.419511:0:3355:0:(client.c:2215:ptlrpc_check_set()) Completed RPC req@00000000ead84f6b pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_02:lustre-MDT0000-mdtlov_UUID:3355:1862456998045888:0@lo:13:osp-pre-0-0.0 00000100:00100000:2.0:1776178200.419522:0:3355:0:(client.c:2215:ptlrpc_check_set()) Completed RPC req@00000000f62fe06f pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_02:lustre-MDT0000-mdtlov_UUID:3355:1862456998045824:0@lo:13:osp-pre-1-0.0 00000100:00100000:2.0:1776178200.420318:0:3353:0:(client.c:1733:ptlrpc_send_new_req()) Sending RPC req@00000000ead84f6b pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_00:lustre-MDT0001-mdtlov_UUID:3353:1862456998045952:0@lo:41:osp-pre-0-1.0 00000100:00100000:1.0:1776178200.420341:0:3356:0:(events.c:361:request_in_callback()) peer: 12345-0@lo (source: 12345-0@lo) 00000100:00100000:2.0:1776178200.420367:0:3353:0:(events.c:361:request_in_callback()) peer: 12345-0@lo (source: 12345-0@lo) 00000100:00100000:1.0:1776178200.420373:0:3356:0:(client.c:1733:ptlrpc_send_new_req()) Sending RPC req@000000004e93e8e2 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_03:lustre-MDT0001-mdtlov_UUID:3356:1862456998045376:0@lo:400:kworker/u8:2.0 00000100:00100000:1.0:1776178200.420407:0:3356:0:(events.c:361:request_in_callback()) peer: 12345-0@lo (source: 12345-0@lo) 00000100:00100000:2.0:1776178200.420411:0:14128:0:(service.c:2152:ptlrpc_server_handle_req_in()) got req x1862456998045952 00000100:00100000:1.0:1776178200.420424:0:3356:0:(client.c:1733:ptlrpc_send_new_req()) Sending RPC req@0000000021d29d48 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_03:lustre-MDT0000-mdtlov_UUID:3356:1862456998045632:0@lo:400:kworker/u8:2.0 00000100:00100000:1.0:1776178200.420451:0:3356:0:(events.c:361:request_in_callback()) peer: 12345-0@lo (source: 12345-0@lo) 00000100:00100000:2.0:1776178200.420451:0:14128:0:(nrs_fifo.c:179:nrs_fifo_req_get()) NRS start fifo request from 12345-0@lo, seq: 693 00000100:00100000:2.0:1776178200.420459:0:14128:0:(service.c:2324:ptlrpc_server_handle_request()) Handling RPC req@00000000df769e31 pname:cluuid+ref:pid:xid:nid:opc:job mdt_out00_002:lustre-MDT0001-mdtlov_UUID+6:3353:x1862456998045952:12345-0@lo:41:osp-pre-0-1.0 00000004:00000024:2.0:1776178200.420488:0:14128:0:(mdt_handler.c:548:mdt_statfs()) blocks cached 0 granted 2097152 pending 0 free 1446121472 avail 1316659200 00000100:00100000:1.0:1776178200.420526:0:20769:0:(service.c:2152:ptlrpc_server_handle_req_in()) got req x1862456998045376 00000100:00100000:2.0:1776178200.420532:0:14128:0:(service.c:2372:ptlrpc_server_handle_request()) Handled RPC req@00000000df769e31 pname:cluuid+ref:pid:xid:nid:opc:job mdt_out00_002:lustre-MDT0001-mdtlov_UUID+6:3353:x1862456998045952:12345-0@lo:41:osp-pre-0-1.0 Request processed in 74us (165us total) trans 0 rc 0/0 00000100:00100000:2.0:1776178200.420543:0:14128:0:(nrs_fifo.c:241:nrs_fifo_req_stop()) NRS stop fifo request from 12345-0@lo, seq: 693 00000100:00100000:1.0:1776178200.420555:0:20769:0:(service.c:2152:ptlrpc_server_handle_req_in()) got req x1862456998045632 00000100:00100000:2.0:1776178200.420570:0:20765:0:(nrs_fifo.c:179:nrs_fifo_req_get()) NRS start fifo request from 12345-0@lo, seq: 506 00000100:00100000:2.0:1776178200.420577:0:20765:0:(service.c:2324:ptlrpc_server_handle_request()) Handling RPC req@0000000045aa39de pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_016:lustre-MDT0001-mdtlov_UUID+5:3356:x1862456998045376:12345-0@lo:400:kworker/u8:2.0 00000100:00100000:1.0:1776178200.420582:0:20769:0:(nrs_fifo.c:179:nrs_fifo_req_get()) NRS start fifo request from 12345-0@lo, seq: 507 00000100:00100000:1.0:1776178200.420587:0:20769:0:(service.c:2324:ptlrpc_server_handle_request()) Handling RPC req@00000000465ea19a pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_018:lustre-MDT0000-mdtlov_UUID+5:3356:x1862456998045632:12345-0@lo:400:kworker/u8:2.0 00000100:00100000:1.0:1776178200.420634:0:20769:0:(service.c:2372:ptlrpc_server_handle_request()) Handled RPC req@00000000465ea19a pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_018:lustre-MDT0000-mdtlov_UUID+5:3356:x1862456998045632:12345-0@lo:400:kworker/u8:2.0 Request processed in 46us (187us total) trans 0 rc 0/0 00000100:00100000:2.0:1776178200.420639:0:20765:0:(service.c:2372:ptlrpc_server_handle_request()) Handled RPC req@0000000045aa39de pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_016:lustre-MDT0001-mdtlov_UUID+5:3356:x1862456998045376:12345-0@lo:400:kworker/u8:2.0 Request processed in 60us (231us total) trans 0 rc 0/0 00000100:00100000:1.0:1776178200.420646:0:20769:0:(nrs_fifo.c:241:nrs_fifo_req_stop()) NRS stop fifo request from 12345-0@lo, seq: 507 00000100:00100000:1.0:1776178200.420672:0:8250:0:(service.c:2152:ptlrpc_server_handle_req_in()) got req x1862456998045120 00000100:00100000:1.0:1776178200.420694:0:8250:0:(nrs_fifo.c:179:nrs_fifo_req_get()) NRS start fifo request from 12345-0@lo, seq: 902 00000100:00100000:1.0:1776178200.420698:0:8250:0:(service.c:2324:ptlrpc_server_handle_request()) Handling RPC req@00000000b03365ad pname:cluuid+ref:pid:xid:nid:opc:job mdt00_003:lustre-MDT0001-lwp-OST0000_UUID+5:3356:x1862456998045120:12345-0@lo:400:kworker/u8:2.0 00000100:00100000:1.0:1776178200.420729:0:8250:0:(service.c:2372:ptlrpc_server_handle_request()) Handled RPC req@00000000b03365ad pname:cluuid+ref:pid:xid:nid:opc:job mdt00_003:lustre-MDT0001-lwp-OST0000_UUID+5:3356:x1862456998045120:12345-0@lo:400:kworker/u8:2.0 Request processed in 31us (392us total) trans 0 rc 0/0 00000100:00100000:1.0:1776178200.420734:0:8250:0:(nrs_fifo.c:241:nrs_fifo_req_stop()) NRS stop fifo request from 12345-0@lo, seq: 902 00000100:00100000:1.0:1776178200.420906:0:3353:0:(client.c:2215:ptlrpc_check_set()) Completed RPC req@00000000ead84f6b pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_00:lustre-MDT0001-mdtlov_UUID:3353:1862456998045952:0@lo:41:osp-pre-0-1.0 00000100:00100000:1.0:1776178200.420924:0:3353:0:(ptlrpcd.c:415:ptlrpcd_check()) transfer 1 async RPCs [1->0] 00000100:00100000:1.0:1776178200.420929:0:3353:0:(client.c:1733:ptlrpc_send_new_req()) Sending RPC req@00000000fcc87e3b pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_00:lustre-MDT0000-mdtlov_UUID:3353:1862456998046016:0@lo:41:osp-pre-1-0.0 00000100:00100000:2.0:1776178200.420955:0:20765:0:(nrs_fifo.c:241:nrs_fifo_req_stop()) NRS stop fifo request from 12345-0@lo, seq: 506 00000100:00100000:1.0:1776178200.421222:0:3353:0:(events.c:361:request_in_callback()) peer: 12345-0@lo (source: 12345-0@lo) 00000100:00100000:1.0:1776178200.421268:0:3356:0:(client.c:2215:ptlrpc_check_set()) Completed RPC req@00000000da9f88c0 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_03:lustre-MDT0001-lwp-OST0000_UUID:3356:1862456998045120:0@lo:400:kworker/u8:2.0 00000100:00100000:1.0:1776178200.421279:0:3356:0:(client.c:2215:ptlrpc_check_set()) Completed RPC req@000000004e93e8e2 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_03:lustre-MDT0001-mdtlov_UUID:3356:1862456998045376:0@lo:400:kworker/u8:2.0 00000100:00100000:1.0:1776178200.421288:0:3356:0:(client.c:2215:ptlrpc_check_set()) Completed RPC req@0000000021d29d48 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_03:lustre-MDT0000-mdtlov_UUID:3356:1862456998045632:0@lo:400:kworker/u8:2.0 00000100:00100000:3.0:1776178200.421369:0:6261:0:(service.c:2152:ptlrpc_server_handle_req_in()) got req x1862456998046016 00000100:00100000:3.0:1776178200.421437:0:6261:0:(nrs_fifo.c:179:nrs_fifo_req_get()) NRS start fifo request from 12345-0@lo, seq: 694 00000100:00100000:3.0:1776178200.421441:0:6261:0:(service.c:2324:ptlrpc_server_handle_request()) Handling RPC req@000000007c8ec83c pname:cluuid+ref:pid:xid:nid:opc:job mdt_out00_001:lustre-MDT0000-mdtlov_UUID+5:3353:x1862456998046016:12345-0@lo:41:osp-pre-1-0.0 00000004:00000024:3.0:1776178200.421465:0:6261:0:(mdt_handler.c:548:mdt_statfs()) blocks cached 0 granted 2097152 pending 0 free 1446035456 avail 1316573184 00000100:00100000:3.0:1776178200.421514:0:6261:0:(service.c:2372:ptlrpc_server_handle_request()) Handled RPC req@000000007c8ec83c pname:cluuid+ref:pid:xid:nid:opc:job mdt_out00_001:lustre-MDT0000-mdtlov_UUID+5:3353:x1862456998046016:12345-0@lo:41:osp-pre-1-0.0 Request processed in 72us (293us total) trans 0 rc 0/0 00000100:00100000:3.0:1776178200.421520:0:6261:0:(nrs_fifo.c:241:nrs_fifo_req_stop()) NRS stop fifo request from 12345-0@lo, seq: 694 00000100:00100000:0.0:1776178200.421533:0:3354:0:(events.c:361:request_in_callback()) peer: 12345-0@lo (source: 12345-0@lo) 00000100:00100000:0.0:1776178200.421560:0:3354:0:(client.c:1733:ptlrpc_send_new_req()) Sending RPC req@0000000058300775 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_01:lustre-MDT0001-lwp-OST0001_UUID:3354:1862456998045248:0@lo:400:kworker/u8:2.0 00000100:00100000:1.0:1776178200.421579:0:3353:0:(client.c:2215:ptlrpc_check_set()) Completed RPC req@00000000fcc87e3b pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_00:lustre-MDT0000-mdtlov_UUID:3353:1862456998046016:0@lo:41:osp-pre-1-0.0 00000100:00100000:0.0:1776178200.421591:0:3354:0:(events.c:361:request_in_callback()) peer: 12345-0@lo (source: 12345-0@lo) 00000100:00100000:1.0:1776178200.421617:0:20140:0:(service.c:2152:ptlrpc_server_handle_req_in()) got req x1862456998044992 00000100:00100000:1.0:1776178200.421646:0:20140:0:(nrs_fifo.c:179:nrs_fifo_req_get()) NRS start fifo request from 12345-0@lo, seq: 154 00000100:00100000:1.0:1776178200.421650:0:20140:0:(service.c:2324:ptlrpc_server_handle_request()) Handling RPC req@000000004c7a25f9 pname:cluuid+ref:pid:xid:nid:opc:job ll_mgs_0002:ac61f839-0e8e-4cac-9c59-08bcfbbad330+23:3354:x1862456998044992:12345-0@lo:400:kworker/u8:2.0 00000100:00100000:1.0:1776178200.421688:0:20140:0:(service.c:2372:ptlrpc_server_handle_request()) Handled RPC req@000000004c7a25f9 pname:cluuid+ref:pid:xid:nid:opc:job ll_mgs_0002:ac61f839-0e8e-4cac-9c59-08bcfbbad330+23:3354:x1862456998044992:12345-0@lo:400:kworker/u8:2.0 Request processed in 37us (157us total) trans 0 rc 0/0 00000100:00100000:0.0:1776178200.421691:0:3354:0:(client.c:1733:ptlrpc_send_new_req()) Sending RPC req@00000000d66c78e0 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_01:lustre-MDT0000-lwp-MDT0001_UUID:3354:1862456998045504:0@lo:400:kworker/u8:2.0 00000100:00100000:1.0:1776178200.421696:0:20140:0:(nrs_fifo.c:241:nrs_fifo_req_stop()) NRS stop fifo request from 12345-0@lo, seq: 154 00000100:00100000:1.0:1776178200.421736:0:6257:0:(service.c:2152:ptlrpc_server_handle_req_in()) got req x1862456998045248 00000100:00100000:1.0:1776178200.421758:0:6257:0:(nrs_fifo.c:179:nrs_fifo_req_get()) NRS start fifo request from 12345-0@lo, seq: 903 00000100:00100000:1.0:1776178200.421762:0:6257:0:(service.c:2324:ptlrpc_server_handle_request()) Handling RPC req@00000000b7afa2a2 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_002:lustre-MDT0001-lwp-OST0001_UUID+5:3354:x1862456998045248:12345-0@lo:400:kworker/u8:2.0 00000100:00100000:1.0:1776178200.421802:0:6257:0:(service.c:2372:ptlrpc_server_handle_request()) Handled RPC req@00000000b7afa2a2 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_002:lustre-MDT0001-lwp-OST0001_UUID+5:3354:x1862456998045248:12345-0@lo:400:kworker/u8:2.0 Request processed in 39us (211us total) trans 0 rc 0/0 00000100:00100000:1.0:1776178200.421808:0:6257:0:(nrs_fifo.c:241:nrs_fifo_req_stop()) NRS stop fifo request from 12345-0@lo, seq: 903 00000100:00100000:0.0:1776178200.421880:0:3354:0:(events.c:361:request_in_callback()) peer: 12345-0@lo (source: 12345-0@lo) 00000100:00100000:0.0:1776178200.421999:0:3354:0:(client.c:1733:ptlrpc_send_new_req()) Sending RPC req@000000009d976bc3 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_01:lustre-MDT0000-lwp-MDT0000_UUID:3354:1862456998045760:0@lo:400:kworker/u8:2.0 00000100:00100000:0.0:1776178200.422020:0:3354:0:(events.c:361:request_in_callback()) peer: 12345-0@lo (source: 12345-0@lo) 00000100:00100000:0.0:1776178200.423641:0:3354:0:(client.c:2215:ptlrpc_check_set()) Completed RPC req@00000000430a7ee3 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_01:ac61f839-0e8e-4cac-9c59-08bcfbbad330:3354:1862456998044992:0@lo:400:kworker/u8:2.0 00000100:00100000:0.0:1776178200.423670:0:3354:0:(client.c:2215:ptlrpc_check_set()) Completed RPC req@0000000058300775 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_01:lustre-MDT0001-lwp-OST0001_UUID:3354:1862456998045248:0@lo:400:kworker/u8:2.0 00000100:00100000:1.0:1776178200.423937:0:8250:0:(service.c:2152:ptlrpc_server_handle_req_in()) got req x1862456998045504 00000100:00100000:1.0:1776178200.423995:0:8250:0:(service.c:2152:ptlrpc_server_handle_req_in()) got req x1862456998045760 00000100:00100000:1.0:1776178200.424107:0:8250:0:(nrs_fifo.c:179:nrs_fifo_req_get()) NRS start fifo request from 12345-0@lo, seq: 904 00000100:00100000:1.0:1776178200.424113:0:8250:0:(service.c:2324:ptlrpc_server_handle_request()) Handling RPC req@000000007608d8e9 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_003:lustre-MDT0000-lwp-MDT0001_UUID+5:3354:x1862456998045504:12345-0@lo:400:kworker/u8:2.0 00000100:00100000:1.0:1776178200.424192:0:8250:0:(service.c:2372:ptlrpc_server_handle_request()) Handled RPC req@000000007608d8e9 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_003:lustre-MDT0000-lwp-MDT0001_UUID+5:3354:x1862456998045504:12345-0@lo:400:kworker/u8:2.0 Request processed in 79us (2313us total) trans 0 rc 0/0 00000100:00100000:1.0:1776178200.424208:0:8250:0:(nrs_fifo.c:241:nrs_fifo_req_stop()) NRS stop fifo request from 12345-0@lo, seq: 904 00000100:00100000:1.0:1776178200.424227:0:8250:0:(nrs_fifo.c:179:nrs_fifo_req_get()) NRS start fifo request from 12345-0@lo, seq: 905 00000100:00100000:1.0:1776178200.424356:0:8250:0:(service.c:2324:ptlrpc_server_handle_request()) Handling RPC req@00000000250bc347 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_003:lustre-MDT0000-lwp-MDT0000_UUID+5:3354:x1862456998045760:12345-0@lo:400:kworker/u8:2.0 00000100:00100000:1.0:1776178200.424379:0:8250:0:(service.c:2372:ptlrpc_server_handle_request()) Handled RPC req@00000000250bc347 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_003:lustre-MDT0000-lwp-MDT0000_UUID+5:3354:x1862456998045760:12345-0@lo:400:kworker/u8:2.0 Request processed in 148us (2359us total) trans 0 rc 0/0 00000100:00100000:1.0:1776178200.424384:0:8250:0:(nrs_fifo.c:241:nrs_fifo_req_stop()) NRS stop fifo request from 12345-0@lo, seq: 905 00000100:00100000:0.0:1776178200.424575:0:3354:0:(client.c:2215:ptlrpc_check_set()) Completed RPC req@00000000d66c78e0 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_01:lustre-MDT0000-lwp-MDT0001_UUID:3354:1862456998045504:0@lo:400:kworker/u8:2.0 00000100:00100000:0.0:1776178200.424583:0:3354:0:(client.c:2215:ptlrpc_check_set()) Completed RPC req@000000009d976bc3 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_01:lustre-MDT0000-lwp-MDT0000_UUID:3354:1862456998045760:0@lo:400:kworker/u8:2.0 00010000:00010000:2.0:1776178201.057003:0:9225:0:(ldlm_lockd.c:2792:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:3.0:1776178201.889692:0:3346:0:(events.c:361:request_in_callback()) peer: 12345-192.168.204.3@tcp (source: 12345-192.168.204.3@tcp) 00000100:00100000:3.0:1776178201.889762:0:3346:0:(events.c:361:request_in_callback()) peer: 12345-192.168.204.3@tcp (source: 12345-192.168.204.3@tcp) 00000100:00100000:3.0:1776178201.889803:0:3346:0:(events.c:361:request_in_callback()) peer: 12345-192.168.204.3@tcp (source: 12345-192.168.204.3@tcp) 00000100:00100000:3.0:1776178201.889825:0:3346:0:(events.c:361:request_in_callback()) peer: 12345-192.168.204.3@tcp (source: 12345-192.168.204.3@tcp) 00000100:00100000:3.0:1776178201.889859:0:3346:0:(events.c:361:request_in_callback()) peer: 12345-192.168.204.3@tcp (source: 12345-192.168.204.3@tcp) 00000100:00100000:3.0:1776178201.889902:0:20765:0:(service.c:2152:ptlrpc_server_handle_req_in()) got req x1862456979098112 00000100:00100000:0.0:1776178201.889924:0:20137:0:(service.c:2152:ptlrpc_server_handle_req_in()) got req x1862456979097856 00000100:00100000:3.0:1776178201.889960:0:20765:0:(service.c:2152:ptlrpc_server_handle_req_in()) got req x1862456979098048 00000100:00100000:0.0:1776178201.889967:0:20137:0:(nrs_fifo.c:179:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.204.3@tcp, seq: 155 00000100:00100000:0.0:1776178201.889976:0:20137:0:(service.c:2324:ptlrpc_server_handle_request()) Handling RPC req@00000000ae8faa77 pname:cluuid+ref:pid:xid:nid:opc:job ll_mgs_0000:5b5c66b1-1a2f-4651-8548-0c75b9a38a94+9:2273:x1862456979097856:12345-192.168.204.3@tcp:400:kworker/u8:2.0 00000100:00100000:3.0:1776178201.889988:0:20765:0:(nrs_fifo.c:179:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.204.3@tcp, seq: 508 00000100:00100000:3.0:1776178201.889994:0:20765:0:(service.c:2324:ptlrpc_server_handle_request()) Handling RPC req@0000000089dbd0da pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_016:b50963bd-8eba-4432-9a54-8888d00e3fa1+9:2273:x1862456979098112:12345-192.168.204.3@tcp:400:kworker/u8:2.0 00000100:00100000:1.0:1776178201.890140:0:8228:0:(nrs_fifo.c:179:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.204.3@tcp, seq: 509 00000100:00100000:1.0:1776178201.890145:0:8228:0:(service.c:2324:ptlrpc_server_handle_request()) Handling RPC req@000000001671af6a pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_001:b50963bd-8eba-4432-9a54-8888d00e3fa1+8:2272:x1862456979098048:12345-192.168.204.3@tcp:400:kworker/u8:2.0 00000100:00100000:2.0:1776178201.890222:0:6257:0:(service.c:2152:ptlrpc_server_handle_req_in()) got req x1862456979097920 00000100:00100000:2.0:1776178201.890259:0:6257:0:(service.c:2152:ptlrpc_server_handle_req_in()) got req x1862456979097984 00000100:00100000:2.0:1776178201.890282:0:6257:0:(nrs_fifo.c:179:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.204.3@tcp, seq: 906 00000100:00100000:2.0:1776178201.890287:0:6257:0:(service.c:2324:ptlrpc_server_handle_request()) Handling RPC req@000000001f977915 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_002:b50963bd-8eba-4432-9a54-8888d00e3fa1+7:2272:x1862456979097920:12345-192.168.204.3@tcp:400:kworker/u8:2.0 00000100:00100000:3.0:1776178201.890438:0:20765:0:(service.c:2372:ptlrpc_server_handle_request()) Handled RPC req@0000000089dbd0da pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_016:b50963bd-8eba-4432-9a54-8888d00e3fa1+9:2273:x1862456979098112:12345-192.168.204.3@tcp:400:kworker/u8:2.0 Request processed in 442us (612us total) trans 0 rc 0/0 00000100:00100000:3.0:1776178201.890446:0:20765:0:(nrs_fifo.c:241:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.204.3@tcp, seq: 508 00000100:00100000:3.0:1776178201.890476:0:11281:0:(nrs_fifo.c:179:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.204.3@tcp, seq: 907 00000100:00100000:3.0:1776178201.890481:0:11281:0:(service.c:2324:ptlrpc_server_handle_request()) Handling RPC req@0000000007d49440 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_004:b50963bd-8eba-4432-9a54-8888d00e3fa1+6:2273:x1862456979097984:12345-192.168.204.3@tcp:400:kworker/u8:2.0 00000100:00100000:0.0:1776178201.893960:0:20137:0:(service.c:2372:ptlrpc_server_handle_request()) Handled RPC req@00000000ae8faa77 pname:cluuid+ref:pid:xid:nid:opc:job ll_mgs_0000:5b5c66b1-1a2f-4651-8548-0c75b9a38a94+9:2273:x1862456979097856:12345-192.168.204.3@tcp:400:kworker/u8:2.0 Request processed in 3976us (4189us total) trans 0 rc 0/0 00000100:00100000:1.0:1776178201.893972:0:8228:0:(service.c:2372:ptlrpc_server_handle_request()) Handled RPC req@000000001671af6a pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_001:b50963bd-8eba-4432-9a54-8888d00e3fa1+8:2272:x1862456979098048:12345-192.168.204.3@tcp:400:kworker/u8:2.0 Request processed in 3822us (4109us total) trans 0 rc 0/0 00000100:00100000:0.0:1776178201.893980:0:20137:0:(nrs_fifo.c:241:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.204.3@tcp, seq: 155 00000100:00100000:1.0:1776178201.893984:0:8228:0:(nrs_fifo.c:241:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.204.3@tcp, seq: 509 00000100:00100000:2.0:1776178201.895517:0:6257:0:(service.c:2372:ptlrpc_server_handle_request()) Handled RPC req@000000001f977915 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_002:b50963bd-8eba-4432-9a54-8888d00e3fa1+7:2272:x1862456979097920:12345-192.168.204.3@tcp:400:kworker/u8:2.0 Request processed in 5220us (5832us total) trans 0 rc 0/0 00000100:00100000:2.0:1776178201.895533:0:6257:0:(nrs_fifo.c:241:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.204.3@tcp, seq: 906 00000100:00100000:3.0:1776178201.901622:0:11281:0:(service.c:2372:ptlrpc_server_handle_request()) Handled RPC req@0000000007d49440 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_004:b50963bd-8eba-4432-9a54-8888d00e3fa1+6:2273:x1862456979097984:12345-192.168.204.3@tcp:400:kworker/u8:2.0 Request processed in 11129us (11809us total) trans 0 rc 0/0 00000100:00100000:3.0:1776178201.901638:0:11281:0:(nrs_fifo.c:241:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.204.3@tcp, seq: 907 00000100:00100000:1.0:1776178201.952808:0:3356:0:(ptlrpcd.c:415:ptlrpcd_check()) transfer 1 async RPCs [2->3] 00000100:00100000:1.0:1776178201.952840:0:3356:0:(client.c:1733:ptlrpc_send_new_req()) Sending RPC req@00000000b0409689 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_03:lustre-MDT0001-mdtlov_UUID:3356:1862456998046080:0@lo:13:osp-pre-1-1.0 00000100:00100000:1.0:1776178201.952924:0:3356:0:(events.c:361:request_in_callback()) peer: 12345-0@lo (source: 12345-0@lo) 00000100:00100000:3.0:1776178201.953156:0:8231:0:(service.c:2152:ptlrpc_server_handle_req_in()) got req x1862456998046080 00000100:00100000:3.0:1776178201.953233:0:8231:0:(nrs_fifo.c:179:nrs_fifo_req_get()) NRS start fifo request from 12345-0@lo, seq: 477 00000100:00100000:3.0:1776178201.953240:0:8231:0:(service.c:2324:ptlrpc_server_handle_request()) Handling RPC req@00000000fcc87e3b pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_create00:lustre-MDT0001-mdtlov_UUID+5:3356:x1862456998046080:12345-0@lo:13:osp-pre-1-1.0 00002000:00000024:3.0:1776178201.953275:0:8231:0:(ofd_obd.c:740:ofd_statfs()) blocks cached 229376 granted 491746368 pending 0 free 3923689472 avail 3692167168 00000100:00100000:3.0:1776178201.953328:0:8231:0:(service.c:2372:ptlrpc_server_handle_request()) Handled RPC req@00000000fcc87e3b pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_create00:lustre-MDT0001-mdtlov_UUID+5:3356:x1862456998046080:12345-0@lo:13:osp-pre-1-1.0 Request processed in 88us (405us total) trans 0 rc 0/0 00000100:00100000:3.0:1776178201.953336:0:8231:0:(nrs_fifo.c:241:nrs_fifo_req_stop()) NRS stop fifo request from 12345-0@lo, seq: 477 00000100:00100000:1.0:1776178201.953523:0:3356:0:(client.c:2215:ptlrpc_check_set()) Completed RPC req@00000000b0409689 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_03:lustre-MDT0001-mdtlov_UUID:3356:1862456998046080:0@lo:13:osp-pre-1-1.0 00010000:00010000:2.0:1776178202.080494:0:9224:0:(ldlm_lockd.c:2792:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:1.0:1776178202.467229:0:3356:0:(client.c:1733:ptlrpc_send_new_req()) Sending RPC req@000000001e11faca pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_03:lustre-MDT0001-mdtlov_UUID:3356:1862456998046144:0@lo:13:osp-pre-0-1.0 00000100:00100000:1.0:1776178202.467312:0:3356:0:(events.c:361:request_in_callback()) peer: 12345-0@lo (source: 12345-0@lo) 00000100:00100000:2.0:1776178202.467488:0:9554:0:(service.c:2152:ptlrpc_server_handle_req_in()) got req x1862456998046144 00000100:00100000:2.0:1776178202.467556:0:9554:0:(nrs_fifo.c:179:nrs_fifo_req_get()) NRS start fifo request from 12345-0@lo, seq: 478 00000100:00100000:2.0:1776178202.467562:0:9554:0:(service.c:2324:ptlrpc_server_handle_request()) Handling RPC req@0000000021d29d48 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_create00:lustre-MDT0001-mdtlov_UUID+5:3356:x1862456998046144:12345-0@lo:13:osp-pre-0-1.0 00002000:00000024:2.0:1776178202.467597:0:9554:0:(ofd_obd.c:740:ofd_statfs()) blocks cached 315392 granted 484883776 pending 0 free 3923689472 avail 3692167168 00000100:00100000:2.0:1776178202.473680:0:9554:0:(service.c:2372:ptlrpc_server_handle_request()) Handled RPC req@0000000021d29d48 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_create00:lustre-MDT0001-mdtlov_UUID+5:3356:x1862456998046144:12345-0@lo:13:osp-pre-0-1.0 Request processed in 6103us (6355us total) trans 0 rc 0/0 00000100:00100000:2.0:1776178202.473703:0:9554:0:(nrs_fifo.c:241:nrs_fifo_req_stop()) NRS stop fifo request from 12345-0@lo, seq: 478 00000100:00100000:1.0:1776178202.473769:0:3356:0:(client.c:2215:ptlrpc_check_set()) Completed RPC req@000000001e11faca pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_03:lustre-MDT0001-mdtlov_UUID:3356:1862456998046144:0@lo:13:osp-pre-0-1.0 00010000:00010000:1.0:1776178203.105967:0:6245:0:(ldlm_lockd.c:2792:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:0.0:1776178203.983890:0:3347:0:(events.c:361:request_in_callback()) peer: 12345-192.168.204.3@tcp (source: 12345-192.168.204.3@tcp) 00000100:00100000:0.0:1776178203.983974:0:3347:0:(events.c:361:request_in_callback()) peer: 12345-192.168.204.3@tcp (source: 12345-192.168.204.3@tcp) 00000100:00100000:0.0:1776178203.984028:0:3347:0:(events.c:361:request_in_callback()) peer: 12345-192.168.204.3@tcp (source: 12345-192.168.204.3@tcp) 00000100:00100000:0.0:1776178203.984124:0:3347:0:(events.c:361:request_in_callback()) peer: 12345-192.168.204.3@tcp (source: 12345-192.168.204.3@tcp) 00000100:00100000:0.0:1776178203.984253:0:3347:0:(events.c:361:request_in_callback()) peer: 12345-192.168.204.3@tcp (source: 12345-192.168.204.3@tcp) 00000100:00100000:0.0:1776178203.984301:0:6257:0:(service.c:2152:ptlrpc_server_handle_req_in()) got req x1862456979098240 00000100:00100000:0.0:1776178203.984356:0:6257:0:(service.c:2152:ptlrpc_server_handle_req_in()) got req x1862456979098304 00000100:00100000:0.0:1776178203.984378:0:6257:0:(nrs_fifo.c:179:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.204.3@tcp, seq: 908 00000100:00100000:0.0:1776178203.984384:0:6257:0:(service.c:2324:ptlrpc_server_handle_request()) Handling RPC req@00000000103d2c47 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_002:b50963bd-8eba-4432-9a54-8888d00e3fa1+7:2270:x1862456979098240:12345-192.168.204.3@tcp:400:kworker/u8:2.0 00000100:00100000:0.0:1776178203.984476:0:6257:0:(service.c:2372:ptlrpc_server_handle_request()) Handled RPC req@00000000103d2c47 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_002:b50963bd-8eba-4432-9a54-8888d00e3fa1+7:2270:x1862456979098240:12345-192.168.204.3@tcp:400:kworker/u8:2.0 Request processed in 91us (448us total) trans 0 rc 0/0 00000100:00100000:0.0:1776178203.984485:0:6257:0:(nrs_fifo.c:241:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.204.3@tcp, seq: 908 00000100:00100000:0.0:1776178203.984505:0:6257:0:(nrs_fifo.c:179:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.204.3@tcp, seq: 909 00000100:00100000:0.0:1776178203.984509:0:6257:0:(service.c:2324:ptlrpc_server_handle_request()) Handling RPC req@000000009d5112f6 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_002:b50963bd-8eba-4432-9a54-8888d00e3fa1+6:2269:x1862456979098304:12345-192.168.204.3@tcp:38:kworker/u8:2.0 00000100:00100000:1.0:1776178203.984632:0:20140:0:(service.c:2152:ptlrpc_server_handle_req_in()) got req x1862456979098176 00000100:00100000:1.0:1776178203.984664:0:20140:0:(nrs_fifo.c:179:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.204.3@tcp, seq: 156 00000100:00100000:1.0:1776178203.984668:0:20140:0:(service.c:2324:ptlrpc_server_handle_request()) Handling RPC req@000000002efeec88 pname:cluuid+ref:pid:xid:nid:opc:job ll_mgs_0002:5b5c66b1-1a2f-4651-8548-0c75b9a38a94+9:2270:x1862456979098176:12345-192.168.204.3@tcp:400:kworker/u8:2.0 00000100:00100000:1.0:1776178203.984698:0:20140:0:(service.c:2372:ptlrpc_server_handle_request()) Handled RPC req@000000002efeec88 pname:cluuid+ref:pid:xid:nid:opc:job ll_mgs_0002:5b5c66b1-1a2f-4651-8548-0c75b9a38a94+9:2270:x1862456979098176:12345-192.168.204.3@tcp:400:kworker/u8:2.0 Request processed in 29us (818us total) trans 0 rc 0/0 00000100:00100000:1.0:1776178203.984724:0:20140:0:(nrs_fifo.c:241:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.204.3@tcp, seq: 156 00000100:00100000:3.0:1776178203.984869:0:20762:0:(service.c:2152:ptlrpc_server_handle_req_in()) got req x1862456979098368 00000100:00100000:3.0:1776178203.984909:0:20762:0:(service.c:2152:ptlrpc_server_handle_req_in()) got req x1862456979098432 00000100:00100000:3.0:1776178203.984930:0:20762:0:(nrs_fifo.c:179:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.204.3@tcp, seq: 510 00000100:00100000:3.0:1776178203.984935:0:20762:0:(service.c:2324:ptlrpc_server_handle_request()) Handling RPC req@0000000067c1ae4f pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_013:b50963bd-8eba-4432-9a54-8888d00e3fa1+8:2270:x1862456979098368:12345-192.168.204.3@tcp:400:kworker/u8:2.0 00000100:00100000:3.0:1776178203.984971:0:20762:0:(service.c:2372:ptlrpc_server_handle_request()) Handled RPC req@0000000067c1ae4f pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_013:b50963bd-8eba-4432-9a54-8888d00e3fa1+8:2270:x1862456979098368:12345-192.168.204.3@tcp:400:kworker/u8:2.0 Request processed in 35us (998us total) trans 0 rc 0/0 00000100:00100000:3.0:1776178203.984977:0:20762:0:(nrs_fifo.c:241:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.204.3@tcp, seq: 510 00000100:00100000:3.0:1776178203.984989:0:20762:0:(nrs_fifo.c:179:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.204.3@tcp, seq: 511 00000100:00100000:3.0:1776178203.984993:0:20762:0:(service.c:2324:ptlrpc_server_handle_request()) Handling RPC req@000000001e11faca pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_013:b50963bd-8eba-4432-9a54-8888d00e3fa1+9:2270:x1862456979098432:12345-192.168.204.3@tcp:400:kworker/u8:2.0 00000100:00100000:3.0:1776178203.985014:0:20762:0:(service.c:2372:ptlrpc_server_handle_request()) Handled RPC req@000000001e11faca pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_013:b50963bd-8eba-4432-9a54-8888d00e3fa1+9:2270:x1862456979098432:12345-192.168.204.3@tcp:400:kworker/u8:2.0 Request processed in 20us (957us total) trans 0 rc 0/0 00000100:00100000:3.0:1776178203.985020:0:20762:0:(nrs_fifo.c:241:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.204.3@tcp, seq: 511 00010000:02000400:0.0:1776178203.986830:0:6257:0:(ldlm_lib.c:914:target_handle_reconnect()) lustre-MDT0001: Client b50963bd-8eba-4432-9a54-8888d00e3fa1 (at 192.168.204.3@tcp) reconnecting 00010000:00080000:0.0:1776178203.991208:0:6257:0:(ldlm_lib.c:1370:target_handle_connect()) lustre-MDT0001: connection from b50963bd-8eba-4432-9a54-8888d00e3fa1@192.168.204.3@tcp t18446615405130674304 exp 0000000069b82a03 cur 911 last 1776178203 00000004:00100000:0.0:1776178203.991250:0:6257:0:(mdt_handler.c:6641:mdt_connect_internal()) lustre-MDT0001: cli 192.168.204.3@tcp supports cksum type f7, return f7 00000100:00080000:0.0:1776178203.991278:0:6257:0:(import.c:86:import_set_state_nolock()) 00000000a3b4b5b0 : changing import state from FULL to RECOVER 00000100:00080000:0.0:1776178203.991288:0:6257:0:(import.c:86:import_set_state_nolock()) 00000000a3b4b5b0 : changing import state from RECOVER to FULL 00000100:00100000:0.0:1776178203.991440:0:6257:0:(service.c:2372:ptlrpc_server_handle_request()) Handled RPC req@000000009d5112f6 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_002:b50963bd-8eba-4432-9a54-8888d00e3fa1+6:2269:x1862456979098304:12345-192.168.204.3@tcp:38:kworker/u8:2.0 Request processed in 6929us (7186us total) trans 0 rc 0/0 00000100:00100000:0.0:1776178203.991450:0:6257:0:(nrs_fifo.c:241:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.204.3@tcp, seq: 909 00000100:00100000:3.0:1776178204.004854:0:3348:0:(events.c:361:request_in_callback()) peer: 12345-192.168.204.3@tcp (source: 12345-192.168.204.3@tcp) 00000100:00100000:3.0:1776178204.004889:0:3348:0:(events.c:361:request_in_callback()) peer: 12345-192.168.204.3@tcp (source: 12345-192.168.204.3@tcp) 00000100:00100000:3.0:1776178204.004924:0:3348:0:(events.c:361:request_in_callback()) peer: 12345-192.168.204.3@tcp (source: 12345-192.168.204.3@tcp) 00000100:00100000:0.0:1776178204.004931:0:6257:0:(service.c:2152:ptlrpc_server_handle_req_in()) got req x1862456979094080 00000100:00100000:3.0:1776178204.004960:0:3348:0:(events.c:361:request_in_callback()) peer: 12345-192.168.204.3@tcp (source: 12345-192.168.204.3@tcp) 00000100:00100000:0.0:1776178204.004995:0:6257:0:(service.c:2152:ptlrpc_server_handle_req_in()) got req x1862456979094848 00000100:00100000:0.0:1776178204.005007:0:6257:0:(service.c:2152:ptlrpc_server_handle_req_in()) got req x1862456979095232 00000100:00100000:0.0:1776178204.005016:0:6257:0:(service.c:2152:ptlrpc_server_handle_req_in()) got req x1862456979094976 00000100:00100000:0.0:1776178204.005028:0:6257:0:(nrs_fifo.c:179:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.204.3@tcp, seq: 6559 00000100:00100000:0.0:1776178204.005035:0:6257:0:(service.c:2324:ptlrpc_server_handle_request()) Handling RPC req@0000000007fa878c pname:cluuid+ref:pid:xid:nid:opc:job mdt00_002:b50963bd-8eba-4432-9a54-8888d00e3fa1+9:17639:x1862456979094080:12345-192.168.204.3@tcp:36:chmod.0 00000004:00080000:0.0:1776178204.005846:0:6257:0:(mdt_recovery.c:188:mdt_req_from_lrd()) @@@ restoring transno req@0000000007fa878c x1862456979094080/t0(0) o36->b50963bd-8eba-4432-9a54-8888d00e3fa1@192.168.204.3@tcp:736/0 lens 488/3152 e 0 to 0 dl 1776178251 ref 1 fl Interpret:/2/0 rc 0/0 job:'chmod.0' 00000004:00000400:0.0:1776178204.005857:0:6257:0:(mdt_recovery.c:200:mdt_req_from_lrd()) @@@ restoring transno req@0000000007fa878c x1862456979094080/t17179876257(0) o36->b50963bd-8eba-4432-9a54-8888d00e3fa1@192.168.204.3@tcp:736/0 lens 488/3152 e 0 to 0 dl 1776178251 ref 1 fl Interpret:/2/0 rc 0/0 job:'chmod.0' 00000100:00100000:1.0:1776178204.008430:0:8250:0:(nrs_fifo.c:179:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.204.3@tcp, seq: 6560 00000100:00100000:1.0:1776178204.008448:0:8250:0:(service.c:2324:ptlrpc_server_handle_request()) Handling RPC req@00000000dd4b6b2f pname:cluuid+ref:pid:xid:nid:opc:job mdt00_003:b50963bd-8eba-4432-9a54-8888d00e3fa1+9:17640:x1862456979094848:12345-192.168.204.3@tcp:36:chmod.0 00000004:00080000:1.0:1776178204.008524:0:8250:0:(mdt_recovery.c:188:mdt_req_from_lrd()) @@@ restoring transno req@00000000dd4b6b2f x1862456979094848/t0(0) o36->b50963bd-8eba-4432-9a54-8888d00e3fa1@192.168.204.3@tcp:736/0 lens 488/3152 e 0 to 0 dl 1776178251 ref 1 fl Interpret:/2/0 rc 0/0 job:'chmod.0' 00000004:00000400:1.0:1776178204.008535:0:8250:0:(mdt_recovery.c:200:mdt_req_from_lrd()) @@@ restoring transno req@00000000dd4b6b2f x1862456979094848/t17179876260(0) o36->b50963bd-8eba-4432-9a54-8888d00e3fa1@192.168.204.3@tcp:736/0 lens 488/3152 e 0 to 0 dl 1776178251 ref 1 fl Interpret:/2/0 rc 0/0 job:'chmod.0' 00000100:00100000:1.0:1776178204.008686:0:8250:0:(service.c:2372:ptlrpc_server_handle_request()) Handled RPC req@00000000dd4b6b2f pname:cluuid+ref:pid:xid:nid:opc:job mdt00_003:b50963bd-8eba-4432-9a54-8888d00e3fa1+9:17640:x1862456979094848:12345-192.168.204.3@tcp:36:chmod.0 Request processed in 242us (3796us total) trans 17179876260 rc 0/0 00000100:00100000:1.0:1776178204.008694:0:8250:0:(nrs_fifo.c:241:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.204.3@tcp, seq: 6560 00000100:00100000:1.0:1776178204.008743:0:8250:0:(nrs_fifo.c:179:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.204.3@tcp, seq: 6561 00000100:00100000:1.0:1776178204.008748:0:8250:0:(service.c:2324:ptlrpc_server_handle_request()) Handling RPC req@00000000ec65370c pname:cluuid+ref:pid:xid:nid:opc:job mdt00_003:b50963bd-8eba-4432-9a54-8888d00e3fa1+8:17644:x1862456979095232:12345-192.168.204.3@tcp:36:chmod.0 00000004:00080000:1.0:1776178204.008770:0:8250:0:(mdt_recovery.c:188:mdt_req_from_lrd()) @@@ restoring transno req@00000000ec65370c x1862456979095232/t0(0) o36->b50963bd-8eba-4432-9a54-8888d00e3fa1@192.168.204.3@tcp:736/0 lens 488/3152 e 0 to 0 dl 1776178251 ref 1 fl Interpret:/2/0 rc 0/0 job:'chmod.0' 00000004:00000400:1.0:1776178204.008777:0:8250:0:(mdt_recovery.c:200:mdt_req_from_lrd()) @@@ restoring transno req@00000000ec65370c x1862456979095232/t17179876262(0) o36->b50963bd-8eba-4432-9a54-8888d00e3fa1@192.168.204.3@tcp:736/0 lens 488/3152 e 0 to 0 dl 1776178251 ref 1 fl Interpret:/2/0 rc 0/0 job:'chmod.0' 00000100:00100000:1.0:1776178204.008818:0:8250:0:(service.c:2372:ptlrpc_server_handle_request()) Handled RPC req@00000000ec65370c pname:cluuid+ref:pid:xid:nid:opc:job mdt00_003:b50963bd-8eba-4432-9a54-8888d00e3fa1+8:17644:x1862456979095232:12345-192.168.204.3@tcp:36:chmod.0 Request processed in 69us (3894us total) trans 17179876262 rc 0/0 00000100:00100000:1.0:1776178204.008824:0:8250:0:(nrs_fifo.c:241:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.204.3@tcp, seq: 6561 00000100:00100000:1.0:1776178204.008842:0:8250:0:(nrs_fifo.c:179:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.204.3@tcp, seq: 6562 00000100:00100000:1.0:1776178204.008847:0:8250:0:(service.c:2324:ptlrpc_server_handle_request()) Handling RPC req@00000000e787b61f pname:cluuid+ref:pid:xid:nid:opc:job mdt00_003:b50963bd-8eba-4432-9a54-8888d00e3fa1+7:17645:x1862456979094976:12345-192.168.204.3@tcp:36:chmod.0 00000004:00080000:1.0:1776178204.008865:0:8250:0:(mdt_recovery.c:188:mdt_req_from_lrd()) @@@ restoring transno req@00000000e787b61f x1862456979094976/t0(0) o36->b50963bd-8eba-4432-9a54-8888d00e3fa1@192.168.204.3@tcp:736/0 lens 488/3152 e 0 to 0 dl 1776178251 ref 1 fl Interpret:/2/0 rc 0/0 job:'chmod.0' 00000004:00000400:1.0:1776178204.008872:0:8250:0:(mdt_recovery.c:200:mdt_req_from_lrd()) @@@ restoring transno req@00000000e787b61f x1862456979094976/t17179876261(0) o36->b50963bd-8eba-4432-9a54-8888d00e3fa1@192.168.204.3@tcp:736/0 lens 488/3152 e 0 to 0 dl 1776178251 ref 1 fl Interpret:/2/0 rc 0/0 job:'chmod.0' 00000100:00100000:1.0:1776178204.008909:0:8250:0:(service.c:2372:ptlrpc_server_handle_request()) Handled RPC req@00000000e787b61f pname:cluuid+ref:pid:xid:nid:opc:job mdt00_003:b50963bd-8eba-4432-9a54-8888d00e3fa1+7:17645:x1862456979094976:12345-192.168.204.3@tcp:36:chmod.0 Request processed in 62us (3949us total) trans 17179876261 rc 0/0 00000100:00100000:1.0:1776178204.008916:0:8250:0:(nrs_fifo.c:241:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.204.3@tcp, seq: 6562 00000100:00100000:1.0:1776178204.009891:0:3347:0:(events.c:361:request_in_callback()) peer: 12345-192.168.204.3@tcp (source: 12345-192.168.204.3@tcp) 00000100:00100000:1.0:1776178204.009958:0:3347:0:(events.c:361:request_in_callback()) peer: 12345-192.168.204.3@tcp (source: 12345-192.168.204.3@tcp) 00000100:00100000:1.0:1776178204.010120:0:3347:0:(events.c:361:request_in_callback()) peer: 12345-192.168.204.3@tcp (source: 12345-192.168.204.3@tcp) 00000100:00100000:3.0:1776178204.010188:0:6255:0:(service.c:2152:ptlrpc_server_handle_req_in()) got req x1862456979094720 00000100:00100000:3.0:1776178204.010229:0:6255:0:(service.c:2152:ptlrpc_server_handle_req_in()) got req x1862456979095104 00000100:00100000:3.0:1776178204.010242:0:6255:0:(nrs_fifo.c:179:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.204.3@tcp, seq: 6563 00000100:00100000:3.0:1776178204.010247:0:6255:0:(service.c:2324:ptlrpc_server_handle_request()) Handling RPC req@000000004e93e8e2 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_000:b50963bd-8eba-4432-9a54-8888d00e3fa1+8:17642:x1862456979094720:12345-192.168.204.3@tcp:36:chmod.0 00000004:00080000:3.0:1776178204.010290:0:6255:0:(mdt_recovery.c:188:mdt_req_from_lrd()) @@@ restoring transno req@000000004e93e8e2 x1862456979094720/t0(0) o36->b50963bd-8eba-4432-9a54-8888d00e3fa1@192.168.204.3@tcp:736/0 lens 488/3152 e 0 to 0 dl 1776178251 ref 1 fl Interpret:/2/0 rc 0/0 job:'chmod.0' 00000004:00000400:3.0:1776178204.010298:0:6255:0:(mdt_recovery.c:200:mdt_req_from_lrd()) @@@ restoring transno req@000000004e93e8e2 x1862456979094720/t17179876259(0) o36->b50963bd-8eba-4432-9a54-8888d00e3fa1@192.168.204.3@tcp:736/0 lens 488/3152 e 0 to 0 dl 1776178251 ref 1 fl Interpret:/2/0 rc 0/0 job:'chmod.0' 00000100:00100000:3.0:1776178204.010376:0:6255:0:(service.c:2372:ptlrpc_server_handle_request()) Handled RPC req@000000004e93e8e2 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_000:b50963bd-8eba-4432-9a54-8888d00e3fa1+8:17642:x1862456979094720:12345-192.168.204.3@tcp:36:chmod.0 Request processed in 128us (489us total) trans 17179876259 rc 0/0 00000100:00100000:3.0:1776178204.010382:0:6255:0:(nrs_fifo.c:241:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.204.3@tcp, seq: 6563 00000100:00100000:3.0:1776178204.010398:0:6255:0:(nrs_fifo.c:179:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.204.3@tcp, seq: 6564 00000100:00100000:3.0:1776178204.010401:0:6255:0:(service.c:2324:ptlrpc_server_handle_request()) Handling RPC req@00000000da9f88c0 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_000:b50963bd-8eba-4432-9a54-8888d00e3fa1+7:17643:x1862456979095104:12345-192.168.204.3@tcp:36:chmod.0 00000004:00080000:3.0:1776178204.010430:0:6255:0:(mdt_recovery.c:188:mdt_req_from_lrd()) @@@ restoring transno req@00000000da9f88c0 x1862456979095104/t0(0) o36->b50963bd-8eba-4432-9a54-8888d00e3fa1@192.168.204.3@tcp:736/0 lens 488/3152 e 0 to 0 dl 1776178251 ref 1 fl Interpret:/2/0 rc 0/0 job:'chmod.0' 00000004:00000400:3.0:1776178204.010437:0:6255:0:(mdt_recovery.c:200:mdt_req_from_lrd()) @@@ restoring transno req@00000000da9f88c0 x1862456979095104/t17179876263(0) o36->b50963bd-8eba-4432-9a54-8888d00e3fa1@192.168.204.3@tcp:736/0 lens 488/3152 e 0 to 0 dl 1776178251 ref 1 fl Interpret:/2/0 rc 0/0 job:'chmod.0' 00000100:00100000:3.0:1776178204.010472:0:6255:0:(service.c:2372:ptlrpc_server_handle_request()) Handled RPC req@00000000da9f88c0 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_000:b50963bd-8eba-4432-9a54-8888d00e3fa1+7:17643:x1862456979095104:12345-192.168.204.3@tcp:36:chmod.0 Request processed in 69us (514us total) trans 17179876263 rc 0/0 00000100:00100000:3.0:1776178204.010478:0:6255:0:(nrs_fifo.c:241:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.204.3@tcp, seq: 6564 00000100:00100000:2.0:1776178204.011772:0:6256:0:(service.c:2152:ptlrpc_server_handle_req_in()) got req x1862456979094592 00000100:00100000:2.0:1776178204.011901:0:6256:0:(nrs_fifo.c:179:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.204.3@tcp, seq: 6565 00000100:00100000:2.0:1776178204.011908:0:6256:0:(service.c:2324:ptlrpc_server_handle_request()) Handling RPC req@00000000f8049845 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_001:b50963bd-8eba-4432-9a54-8888d00e3fa1+7:17641:x1862456979094592:12345-192.168.204.3@tcp:36:chmod.0 00000004:00080000:2.0:1776178204.011968:0:6256:0:(mdt_recovery.c:188:mdt_req_from_lrd()) @@@ restoring transno req@00000000f8049845 x1862456979094592/t0(0) o36->b50963bd-8eba-4432-9a54-8888d00e3fa1@192.168.204.3@tcp:736/0 lens 488/3152 e 0 to 0 dl 1776178251 ref 1 fl Interpret:/2/0 rc 0/0 job:'chmod.0' 00000004:00000400:2.0:1776178204.011977:0:6256:0:(mdt_recovery.c:200:mdt_req_from_lrd()) @@@ restoring transno req@00000000f8049845 x1862456979094592/t17179876258(0) o36->b50963bd-8eba-4432-9a54-8888d00e3fa1@192.168.204.3@tcp:736/0 lens 488/3152 e 0 to 0 dl 1776178251 ref 1 fl Interpret:/2/0 rc 0/0 job:'chmod.0' 00000100:00100000:2.0:1776178204.012118:0:6256:0:(service.c:2372:ptlrpc_server_handle_request()) Handled RPC req@00000000f8049845 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_001:b50963bd-8eba-4432-9a54-8888d00e3fa1+7:17641:x1862456979094592:12345-192.168.204.3@tcp:36:chmod.0 Request processed in 209us (1998us total) trans 17179876258 rc 0/0 00000100:00100000:2.0:1776178204.012125:0:6256:0:(nrs_fifo.c:241:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.204.3@tcp, seq: 6565 00000100:00100000:3.0:1776178204.018397:0:3348:0:(events.c:361:request_in_callback()) peer: 12345-192.168.204.3@tcp (source: 12345-192.168.204.3@tcp) 00000100:00100000:3.0:1776178204.018447:0:3348:0:(events.c:361:request_in_callback()) peer: 12345-192.168.204.3@tcp (source: 12345-192.168.204.3@tcp) 00000100:00100000:3.0:1776178204.018613:0:3348:0:(events.c:361:request_in_callback()) peer: 12345-192.168.204.3@tcp (source: 12345-192.168.204.3@tcp) 00000100:00100000:1.0:1776178204.021408:0:11281:0:(service.c:2152:ptlrpc_server_handle_req_in()) got req x1862456979098944 00000100:00100000:1.0:1776178204.021462:0:11281:0:(service.c:2152:ptlrpc_server_handle_req_in()) got req x1862456979099008 00000100:00100000:1.0:1776178204.021471:0:11281:0:(service.c:2152:ptlrpc_server_handle_req_in()) got req x1862456979099072 00000100:00100000:1.0:1776178204.021487:0:11281:0:(nrs_fifo.c:179:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.204.3@tcp, seq: 6566 00000100:00100000:1.0:1776178204.021494:0:11281:0:(service.c:2324:ptlrpc_server_handle_request()) Handling RPC req@00000000a2082164 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_004:b50963bd-8eba-4432-9a54-8888d00e3fa1+9:17644:x1862456979098944:12345-192.168.204.3@tcp:101:chmod.0 00010000:00010000:1.0:1776178204.021518:0:11281:0:(ldlm_lockd.c:1248:ldlm_handle_enqueue0()) ### server-side enqueue handler START 00010000:00010000:1.0:1776178204.021539:0:11281:0:(ldlm_lockd.c:1318:ldlm_handle_enqueue0()) ### server-side enqueue handler, new lock created ns: mdt-lustre-MDT0001_UUID lock: 000000000b4a063d/0xa40083ab9bf5356b lrc: 2/0,0 mode: --/CR res: [0x240000402:0x7fd:0x0].0x0 bits 0x0/0x0 rrc: 2 type: IBT gid 0 flags: 0x40000000000000 nid: local remote: 0x771f68aa1eb27b34 expref: -99 pid: 11281 timeout: 0 lvb_type: 0 00010000:00010000:1.0:1776178204.021596:0:11281:0:(ldlm_lock.c:1064:ldlm_granted_list_add_lock()) ### About to add lock: ns: mdt-lustre-MDT0001_UUID lock: 000000000b4a063d/0xa40083ab9bf5356b lrc: 3/0,0 mode: CR/CR res: [0x240000402:0x7fd:0x0].0x0 bits 0x8/0x0 rrc: 2 type: IBT gid 0 flags: 0x50000000000000 nid: 192.168.204.3@tcp remote: 0x771f68aa1eb27b34 expref: 10 pid: 11281 timeout: 0 lvb_type: 0 00010000:00010000:1.0:1776178204.021613:0:11281:0:(ldlm_lockd.c:1484:ldlm_handle_enqueue0()) ### server-side enqueue handler, sending reply (err=0, rc=0) ns: mdt-lustre-MDT0001_UUID lock: 000000000b4a063d/0xa40083ab9bf5356b lrc: 3/0,0 mode: CR/CR res: [0x240000402:0x7fd:0x0].0x0 bits 0x8/0x0 rrc: 2 type: IBT gid 0 flags: 0x40000000000000 nid: 192.168.204.3@tcp remote: 0x771f68aa1eb27b34 expref: 10 pid: 11281 timeout: 0 lvb_type: 0 00010000:00010000:1.0:1776178204.021645:0:11281:0:(ldlm_inodebits.c:94:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x240000402:0x7fd:0x0].0x0 (0000000021e0af54) 00010000:00010000:1.0:1776178204.021648:0:11281:0:(ldlm_inodebits.c:96:ldlm_reprocess_inodebits_queue()) Hint 8 00010000:00010000:1.0:1776178204.021651:0:11281:0:(ldlm_lockd.c:1565:ldlm_handle_enqueue0()) ### server-side enqueue handler END (lock 000000000b4a063d, rc 0) 00000100:00100000:1.0:1776178204.021721:0:11281:0:(service.c:2372:ptlrpc_server_handle_request()) Handled RPC req@00000000a2082164 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_004:b50963bd-8eba-4432-9a54-8888d00e3fa1+10:17644:x1862456979098944:12345-192.168.204.3@tcp:101:chmod.0 Request processed in 226us (3336us total) trans 0 rc 0/0 00000100:00100000:1.0:1776178204.021729:0:11281:0:(nrs_fifo.c:241:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.204.3@tcp, seq: 6566 00000100:00100000:1.0:1776178204.021749:0:11281:0:(nrs_fifo.c:179:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.204.3@tcp, seq: 6567 00000100:00100000:1.0:1776178204.021753:0:11281:0:(service.c:2324:ptlrpc_server_handle_request()) Handling RPC req@000000004eda7977 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_004:b50963bd-8eba-4432-9a54-8888d00e3fa1+9:17640:x1862456979099008:12345-192.168.204.3@tcp:101:chmod.0 00010000:00010000:1.0:1776178204.021760:0:11281:0:(ldlm_lockd.c:1248:ldlm_handle_enqueue0()) ### server-side enqueue handler START 00010000:00010000:1.0:1776178204.021770:0:11281:0:(ldlm_lockd.c:1318:ldlm_handle_enqueue0()) ### server-side enqueue handler, new lock created ns: mdt-lustre-MDT0001_UUID lock: 000000006844664c/0xa40083ab9bf53572 lrc: 2/0,0 mode: --/CR res: [0x240000402:0x7f9:0x0].0x0 bits 0x0/0x0 rrc: 2 type: IBT gid 0 flags: 0x40000000000000 nid: local remote: 0x771f68aa1eb27b3b expref: -99 pid: 11281 timeout: 0 lvb_type: 0 00010000:00010000:1.0:1776178204.021797:0:11281:0:(ldlm_lock.c:1064:ldlm_granted_list_add_lock()) ### About to add lock: ns: mdt-lustre-MDT0001_UUID lock: 000000006844664c/0xa40083ab9bf53572 lrc: 3/0,0 mode: CR/CR res: [0x240000402:0x7f9:0x0].0x0 bits 0x8/0x0 rrc: 2 type: IBT gid 0 flags: 0x50000000000000 nid: 192.168.204.3@tcp remote: 0x771f68aa1eb27b3b expref: 10 pid: 11281 timeout: 0 lvb_type: 0 00010000:00010000:1.0:1776178204.021806:0:11281:0:(ldlm_lockd.c:1484:ldlm_handle_enqueue0()) ### server-side enqueue handler, sending reply (err=0, rc=0) ns: mdt-lustre-MDT0001_UUID lock: 000000006844664c/0xa40083ab9bf53572 lrc: 3/0,0 mode: CR/CR res: [0x240000402:0x7f9:0x0].0x0 bits 0x8/0x0 rrc: 2 type: IBT gid 0 flags: 0x40000000000000 nid: 192.168.204.3@tcp remote: 0x771f68aa1eb27b3b expref: 10 pid: 11281 timeout: 0 lvb_type: 0 00010000:00010000:1.0:1776178204.021816:0:11281:0:(ldlm_inodebits.c:94:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x240000402:0x7f9:0x0].0x0 (0000000056a46617) 00010000:00010000:1.0:1776178204.021818:0:11281:0:(ldlm_inodebits.c:96:ldlm_reprocess_inodebits_queue()) Hint 8 00010000:00010000:1.0:1776178204.021820:0:11281:0:(ldlm_lockd.c:1565:ldlm_handle_enqueue0()) ### server-side enqueue handler END (lock 000000006844664c, rc 0) 00000100:00100000:1.0:1776178204.021838:0:11281:0:(service.c:2372:ptlrpc_server_handle_request()) Handled RPC req@000000004eda7977 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_004:b50963bd-8eba-4432-9a54-8888d00e3fa1+10:17640:x1862456979099008:12345-192.168.204.3@tcp:101:chmod.0 Request processed in 85us (3392us total) trans 0 rc 0/0 00000100:00100000:1.0:1776178204.021844:0:11281:0:(nrs_fifo.c:241:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.204.3@tcp, seq: 6567 00000100:00100000:1.0:1776178204.021857:0:11281:0:(nrs_fifo.c:179:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.204.3@tcp, seq: 6568 00000100:00100000:1.0:1776178204.021861:0:11281:0:(service.c:2324:ptlrpc_server_handle_request()) Handling RPC req@0000000084c00ed0 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_004:b50963bd-8eba-4432-9a54-8888d00e3fa1+9:17645:x1862456979099072:12345-192.168.204.3@tcp:101:chmod.0 00010000:00010000:1.0:1776178204.021878:0:11281:0:(ldlm_lockd.c:1248:ldlm_handle_enqueue0()) ### server-side enqueue handler START 00010000:00010000:1.0:1776178204.021890:0:11281:0:(ldlm_lockd.c:1318:ldlm_handle_enqueue0()) ### server-side enqueue handler, new lock created ns: mdt-lustre-MDT0001_UUID lock: 00000000f314daa9/0xa40083ab9bf53579 lrc: 2/0,0 mode: --/CR res: [0x240000402:0x7fe:0x0].0x0 bits 0x0/0x0 rrc: 2 type: IBT gid 0 flags: 0x40000000000000 nid: local remote: 0x771f68aa1eb27b42 expref: -99 pid: 11281 timeout: 0 lvb_type: 0 00010000:00010000:1.0:1776178204.021916:0:11281:0:(ldlm_lock.c:1064:ldlm_granted_list_add_lock()) ### About to add lock: ns: mdt-lustre-MDT0001_UUID lock: 00000000f314daa9/0xa40083ab9bf53579 lrc: 3/0,0 mode: CR/CR res: [0x240000402:0x7fe:0x0].0x0 bits 0x8/0x0 rrc: 2 type: IBT gid 0 flags: 0x50000000000000 nid: 192.168.204.3@tcp remote: 0x771f68aa1eb27b42 expref: 10 pid: 11281 timeout: 0 lvb_type: 0 00010000:00010000:1.0:1776178204.021926:0:11281:0:(ldlm_lockd.c:1484:ldlm_handle_enqueue0()) ### server-side enqueue handler, sending reply (err=0, rc=0) ns: mdt-lustre-MDT0001_UUID lock: 00000000f314daa9/0xa40083ab9bf53579 lrc: 3/0,0 mode: CR/CR res: [0x240000402:0x7fe:0x0].0x0 bits 0x8/0x0 rrc: 2 type: IBT gid 0 flags: 0x40000000000000 nid: 192.168.204.3@tcp remote: 0x771f68aa1eb27b42 expref: 10 pid: 11281 timeout: 0 lvb_type: 0 00010000:00010000:1.0:1776178204.021937:0:11281:0:(ldlm_inodebits.c:94:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x240000402:0x7fe:0x0].0x0 (0000000030fbaafd) 00010000:00010000:1.0:1776178204.021940:0:11281:0:(ldlm_inodebits.c:96:ldlm_reprocess_inodebits_queue()) Hint 8 00010000:00010000:1.0:1776178204.021942:0:11281:0:(ldlm_lockd.c:1565:ldlm_handle_enqueue0()) ### server-side enqueue handler END (lock 00000000f314daa9, rc 0) 00000100:00100000:1.0:1776178204.021961:0:11281:0:(service.c:2372:ptlrpc_server_handle_request()) Handled RPC req@0000000084c00ed0 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_004:b50963bd-8eba-4432-9a54-8888d00e3fa1+10:17645:x1862456979099072:12345-192.168.204.3@tcp:101:chmod.0 Request processed in 100us (3349us total) trans 0 rc 0/0 00000100:00100000:1.0:1776178204.021967:0:11281:0:(nrs_fifo.c:241:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.204.3@tcp, seq: 6568 00000100:00100000:1.0:1776178204.026593:0:3346:0:(events.c:361:request_in_callback()) peer: 12345-192.168.204.3@tcp (source: 12345-192.168.204.3@tcp) 00000100:00100000:1.0:1776178204.026651:0:3346:0:(events.c:361:request_in_callback()) peer: 12345-192.168.204.3@tcp (source: 12345-192.168.204.3@tcp) 00000100:00100000:1.0:1776178204.026691:0:6255:0:(service.c:2152:ptlrpc_server_handle_req_in()) got req x1862456979099136 00000100:00100000:2.0:1776178204.026893:0:6256:0:(service.c:2152:ptlrpc_server_handle_req_in()) got req x1862456979099200 00000100:00100000:2.0:1776178204.026945:0:6256:0:(nrs_fifo.c:179:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.204.3@tcp, seq: 6569 00000100:00100000:2.0:1776178204.026953:0:6256:0:(service.c:2324:ptlrpc_server_handle_request()) Handling RPC req@00000000c008a6fb pname:cluuid+ref:pid:xid:nid:opc:job mdt00_001:b50963bd-8eba-4432-9a54-8888d00e3fa1+11:17643:x1862456979099200:12345-192.168.204.3@tcp:101:chmod.0 00010000:00010000:2.0:1776178204.026985:0:6256:0:(ldlm_lockd.c:1248:ldlm_handle_enqueue0()) ### server-side enqueue handler START 00010000:00010000:2.0:1776178204.027177:0:6256:0:(ldlm_lockd.c:1318:ldlm_handle_enqueue0()) ### server-side enqueue handler, new lock created ns: mdt-lustre-MDT0001_UUID lock: 00000000fcb32840/0xa40083ab9bf53580 lrc: 2/0,0 mode: --/CR res: [0x240000402:0x7fc:0x0].0x0 bits 0x0/0x0 rrc: 2 type: IBT gid 0 flags: 0x40000000000000 nid: local remote: 0x771f68aa1eb27b50 expref: -99 pid: 6256 timeout: 0 lvb_type: 0 00010000:00010000:2.0:1776178204.027248:0:6256:0:(ldlm_lock.c:1064:ldlm_granted_list_add_lock()) ### About to add lock: ns: mdt-lustre-MDT0001_UUID lock: 00000000fcb32840/0xa40083ab9bf53580 lrc: 3/0,0 mode: CR/CR res: [0x240000402:0x7fc:0x0].0x0 bits 0x8/0x0 rrc: 2 type: IBT gid 0 flags: 0x50000000000000 nid: 192.168.204.3@tcp remote: 0x771f68aa1eb27b50 expref: 12 pid: 6256 timeout: 0 lvb_type: 0 00010000:00010000:2.0:1776178204.027262:0:6256:0:(ldlm_lockd.c:1484:ldlm_handle_enqueue0()) ### server-side enqueue handler, sending reply (err=0, rc=0) ns: mdt-lustre-MDT0001_UUID lock: 00000000fcb32840/0xa40083ab9bf53580 lrc: 3/0,0 mode: CR/CR res: [0x240000402:0x7fc:0x0].0x0 bits 0x8/0x0 rrc: 2 type: IBT gid 0 flags: 0x40000000000000 nid: 192.168.204.3@tcp remote: 0x771f68aa1eb27b50 expref: 12 pid: 6256 timeout: 0 lvb_type: 0 00010000:00010000:2.0:1776178204.027280:0:6256:0:(ldlm_inodebits.c:94:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x240000402:0x7fc:0x0].0x0 (00000000c31449d9) 00010000:00010000:2.0:1776178204.027283:0:6256:0:(ldlm_inodebits.c:96:ldlm_reprocess_inodebits_queue()) Hint 8 00010000:00010000:2.0:1776178204.027287:0:6256:0:(ldlm_lockd.c:1565:ldlm_handle_enqueue0()) ### server-side enqueue handler END (lock 00000000fcb32840, rc 0) 00000100:00100000:2.0:1776178204.027353:0:6256:0:(service.c:2372:ptlrpc_server_handle_request()) Handled RPC req@00000000c008a6fb pname:cluuid+ref:pid:xid:nid:opc:job mdt00_001:b50963bd-8eba-4432-9a54-8888d00e3fa1+12:17643:x1862456979099200:12345-192.168.204.3@tcp:101:chmod.0 Request processed in 400us (701us total) trans 0 rc 0/0 00000100:00100000:2.0:1776178204.027361:0:6256:0:(nrs_fifo.c:241:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.204.3@tcp, seq: 6569 00000100:00100000:1.0:1776178204.027813:0:6255:0:(nrs_fifo.c:179:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.204.3@tcp, seq: 6570 00000100:00100000:1.0:1776178204.027818:0:6255:0:(service.c:2324:ptlrpc_server_handle_request()) Handling RPC req@000000005e1c4771 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_000:b50963bd-8eba-4432-9a54-8888d00e3fa1+11:17642:x1862456979099136:12345-192.168.204.3@tcp:101:chmod.0 00010000:00010000:1.0:1776178204.027830:0:6255:0:(ldlm_lockd.c:1248:ldlm_handle_enqueue0()) ### server-side enqueue handler START 00010000:00010000:1.0:1776178204.027844:0:6255:0:(ldlm_lockd.c:1318:ldlm_handle_enqueue0()) ### server-side enqueue handler, new lock created ns: mdt-lustre-MDT0001_UUID lock: 0000000014581b1a/0xa40083ab9bf53587 lrc: 2/0,0 mode: --/CR res: [0x240000402:0x7fb:0x0].0x0 bits 0x0/0x0 rrc: 2 type: IBT gid 0 flags: 0x40000000000000 nid: local remote: 0x771f68aa1eb27b49 expref: -99 pid: 6255 timeout: 0 lvb_type: 0 00010000:00010000:1.0:1776178204.027887:0:6255:0:(ldlm_lock.c:1064:ldlm_granted_list_add_lock()) ### About to add lock: ns: mdt-lustre-MDT0001_UUID lock: 0000000014581b1a/0xa40083ab9bf53587 lrc: 3/0,0 mode: CR/CR res: [0x240000402:0x7fb:0x0].0x0 bits 0x8/0x0 rrc: 2 type: IBT gid 0 flags: 0x50000000000000 nid: 192.168.204.3@tcp remote: 0x771f68aa1eb27b49 expref: 12 pid: 6255 timeout: 0 lvb_type: 0 00010000:00010000:1.0:1776178204.027897:0:6255:0:(ldlm_lockd.c:1484:ldlm_handle_enqueue0()) ### server-side enqueue handler, sending reply (err=0, rc=0) ns: mdt-lustre-MDT0001_UUID lock: 0000000014581b1a/0xa40083ab9bf53587 lrc: 3/0,0 mode: CR/CR res: [0x240000402:0x7fb:0x0].0x0 bits 0x8/0x0 rrc: 2 type: IBT gid 0 flags: 0x40000000000000 nid: 192.168.204.3@tcp remote: 0x771f68aa1eb27b49 expref: 12 pid: 6255 timeout: 0 lvb_type: 0 00010000:00010000:1.0:1776178204.027909:0:6255:0:(ldlm_inodebits.c:94:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x240000402:0x7fb:0x0].0x0 (000000000885de38) 00010000:00010000:1.0:1776178204.027912:0:6255:0:(ldlm_inodebits.c:96:ldlm_reprocess_inodebits_queue()) Hint 8 00010000:00010000:1.0:1776178204.027913:0:6255:0:(ldlm_lockd.c:1565:ldlm_handle_enqueue0()) ### server-side enqueue handler END (lock 0000000014581b1a, rc 0) 00000100:00100000:1.0:1776178204.027941:0:6255:0:(service.c:2372:ptlrpc_server_handle_request()) Handled RPC req@000000005e1c4771 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_000:b50963bd-8eba-4432-9a54-8888d00e3fa1+12:17642:x1862456979099136:12345-192.168.204.3@tcp:101:chmod.0 Request processed in 122us (1357us total) trans 0 rc 0/0 00000100:00100000:1.0:1776178204.027947:0:6255:0:(nrs_fifo.c:241:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.204.3@tcp, seq: 6570 00000100:00100000:1.0:1776178204.034597:0:3346:0:(events.c:361:request_in_callback()) peer: 12345-192.168.204.3@tcp (source: 12345-192.168.204.3@tcp) 00000100:00100000:1.0:1776178204.034655:0:3346:0:(events.c:361:request_in_callback()) peer: 12345-192.168.204.3@tcp (source: 12345-192.168.204.3@tcp) 00000100:00100000:1.0:1776178204.034681:0:3346:0:(events.c:361:request_in_callback()) peer: 12345-192.168.204.3@tcp (source: 12345-192.168.204.3@tcp) 00000100:00100000:1.0:1776178204.034711:0:3346:0:(events.c:361:request_in_callback()) peer: 12345-192.168.204.3@tcp (source: 12345-192.168.204.3@tcp) 00000100:00100000:1.0:1776178204.034732:0:3346:0:(events.c:361:request_in_callback()) peer: 12345-192.168.204.3@tcp (source: 12345-192.168.204.3@tcp) 00000100:00100000:2.0:1776178204.035036:0:8228:0:(service.c:2152:ptlrpc_server_handle_req_in()) got req x1862456979099264 00000100:00100000:2.0:1776178204.035115:0:8228:0:(service.c:2152:ptlrpc_server_handle_req_in()) got req x1862456979099328 00000100:00100000:2.0:1776178204.035132:0:8228:0:(service.c:2152:ptlrpc_server_handle_req_in()) got req x1862456979099392 00000100:00100000:2.0:1776178204.035154:0:8228:0:(service.c:2152:ptlrpc_server_handle_req_in()) got req x1862456979099520 00000100:00100000:2.0:1776178204.035172:0:8228:0:(nrs_fifo.c:179:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.204.3@tcp, seq: 5271 00000100:00100000:2.0:1776178204.035179:0:8228:0:(service.c:2324:ptlrpc_server_handle_request()) Handling RPC req@0000000028ec8437 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_001:b50963bd-8eba-4432-9a54-8888d00e3fa1+10:2272:x1862456979099264:12345-192.168.204.3@tcp:2:chmod.0 00002000:00010000:2.0:1776178204.035366:0:8228:0:(ofd_lvb.c:324:ofd_lvbo_update()) res: [0x2c0000400:0x431:0x0] updating lvb size from disk: 0 -> 0 00002000:00010000:2.0:1776178204.035370:0:8228:0:(ofd_lvb.c:331:ofd_lvbo_update()) res: [0x2c0000400:0x431:0x0] updating lvb mtime from disk: 1776178154 -> 1776178154 00002000:00010000:2.0:1776178204.035372:0:8228:0:(ofd_lvb.c:337:ofd_lvbo_update()) res: [0x2c0000400:0x431:0x0] updating lvb atime from disk: 1776178154 -> 1776178154 00002000:00010000:2.0:1776178204.035374:0:8228:0:(ofd_lvb.c:343:ofd_lvbo_update()) res: [0x2c0000400:0x431:0x0] updating lvb ctime from disk: 1776178154 -> 1776178155 00002000:00010000:2.0:1776178204.035376:0:8228:0:(ofd_lvb.c:349:ofd_lvbo_update()) res: [0x2c0000400:0x431:0x0] updating lvb blocks from disk: 0 -> 0 00000100:00100000:2.0:1776178204.035436:0:8228:0:(service.c:2372:ptlrpc_server_handle_request()) Handled RPC req@0000000028ec8437 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_001:b50963bd-8eba-4432-9a54-8888d00e3fa1+11:2272:x1862456979099264:12345-192.168.204.3@tcp:2:chmod.0 Request processed in 256us (848us total) trans 4294969328 rc 0/0 00000100:00100000:2.0:1776178204.035444:0:8228:0:(nrs_fifo.c:241:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.204.3@tcp, seq: 5271 00000100:00100000:2.0:1776178204.035463:0:8228:0:(nrs_fifo.c:179:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.204.3@tcp, seq: 5272 00000100:00100000:2.0:1776178204.035468:0:8228:0:(service.c:2324:ptlrpc_server_handle_request()) Handling RPC req@00000000c832ce5d pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_001:b50963bd-8eba-4432-9a54-8888d00e3fa1+9:2270:x1862456979099328:12345-192.168.204.3@tcp:2:chmod.0 00002000:00010000:2.0:1776178204.035537:0:8228:0:(ofd_lvb.c:324:ofd_lvbo_update()) res: [0x280000400:0x46e:0x0] updating lvb size from disk: 0 -> 0 00002000:00010000:2.0:1776178204.035540:0:8228:0:(ofd_lvb.c:331:ofd_lvbo_update()) res: [0x280000400:0x46e:0x0] updating lvb mtime from disk: 1776178153 -> 1776178153 00002000:00010000:2.0:1776178204.035541:0:8228:0:(ofd_lvb.c:337:ofd_lvbo_update()) res: [0x280000400:0x46e:0x0] updating lvb atime from disk: 1776178153 -> 1776178153 00002000:00010000:2.0:1776178204.035543:0:8228:0:(ofd_lvb.c:343:ofd_lvbo_update()) res: [0x280000400:0x46e:0x0] updating lvb ctime from disk: 1776178153 -> 1776178155 00002000:00010000:2.0:1776178204.035545:0:8228:0:(ofd_lvb.c:349:ofd_lvbo_update()) res: [0x280000400:0x46e:0x0] updating lvb blocks from disk: 0 -> 0 00000100:00100000:2.0:1776178204.035565:0:8228:0:(service.c:2372:ptlrpc_server_handle_request()) Handled RPC req@00000000c832ce5d pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_001:b50963bd-8eba-4432-9a54-8888d00e3fa1+10:2270:x1862456979099328:12345-192.168.204.3@tcp:2:chmod.0 Request processed in 96us (910us total) trans 4294969356 rc 0/0 00000100:00100000:2.0:1776178204.035571:0:8228:0:(nrs_fifo.c:241:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.204.3@tcp, seq: 5272 00000100:00100000:2.0:1776178204.035583:0:8228:0:(nrs_fifo.c:179:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.204.3@tcp, seq: 5273 00000100:00100000:2.0:1776178204.035587:0:8228:0:(service.c:2324:ptlrpc_server_handle_request()) Handling RPC req@000000007bc8dbc1 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_001:b50963bd-8eba-4432-9a54-8888d00e3fa1+9:2273:x1862456979099392:12345-192.168.204.3@tcp:2:chmod.0 00000100:00100000:1.0:1776178204.036127:0:20759:0:(nrs_fifo.c:179:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.204.3@tcp, seq: 5274 00000100:00100000:1.0:1776178204.036133:0:20759:0:(service.c:2324:ptlrpc_server_handle_request()) Handling RPC req@000000008f3792e7 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_010:b50963bd-8eba-4432-9a54-8888d00e3fa1+10:2272:x1862456979099520:12345-192.168.204.3@tcp:2:chmod.0 00000100:00100000:3.0:1776178204.036341:0:8250:0:(service.c:2152:ptlrpc_server_handle_req_in()) got req x1862456979099456 00000100:00100000:3.0:1776178204.036387:0:8250:0:(nrs_fifo.c:179:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.204.3@tcp, seq: 6571 00000100:00100000:3.0:1776178204.036392:0:8250:0:(service.c:2324:ptlrpc_server_handle_request()) Handling RPC req@000000003174f31d pname:cluuid+ref:pid:xid:nid:opc:job mdt00_003:b50963bd-8eba-4432-9a54-8888d00e3fa1+12:17641:x1862456979099456:12345-192.168.204.3@tcp:101:chmod.0 00010000:00010000:3.0:1776178204.036410:0:8250:0:(ldlm_lockd.c:1248:ldlm_handle_enqueue0()) ### server-side enqueue handler START 00010000:00010000:3.0:1776178204.036435:0:8250:0:(ldlm_lockd.c:1318:ldlm_handle_enqueue0()) ### server-side enqueue handler, new lock created ns: mdt-lustre-MDT0001_UUID lock: 000000006a188f9a/0xa40083ab9bf5358e lrc: 2/0,0 mode: --/CR res: [0x240000402:0x7fa:0x0].0x0 bits 0x0/0x0 rrc: 2 type: IBT gid 0 flags: 0x40000000000000 nid: local remote: 0x771f68aa1eb27b57 expref: -99 pid: 8250 timeout: 0 lvb_type: 0 00010000:00010000:3.0:1776178204.036489:0:8250:0:(ldlm_lock.c:1064:ldlm_granted_list_add_lock()) ### About to add lock: ns: mdt-lustre-MDT0001_UUID lock: 000000006a188f9a/0xa40083ab9bf5358e lrc: 3/0,0 mode: CR/CR res: [0x240000402:0x7fa:0x0].0x0 bits 0x8/0x0 rrc: 2 type: IBT gid 0 flags: 0x50000000000000 nid: 192.168.204.3@tcp remote: 0x771f68aa1eb27b57 expref: 13 pid: 8250 timeout: 0 lvb_type: 0 00010000:00010000:3.0:1776178204.036506:0:8250:0:(ldlm_lockd.c:1484:ldlm_handle_enqueue0()) ### server-side enqueue handler, sending reply (err=0, rc=0) ns: mdt-lustre-MDT0001_UUID lock: 000000006a188f9a/0xa40083ab9bf5358e lrc: 3/0,0 mode: CR/CR res: [0x240000402:0x7fa:0x0].0x0 bits 0x8/0x0 rrc: 2 type: IBT gid 0 flags: 0x40000000000000 nid: 192.168.204.3@tcp remote: 0x771f68aa1eb27b57 expref: 13 pid: 8250 timeout: 0 lvb_type: 0 00010000:00010000:3.0:1776178204.036524:0:8250:0:(ldlm_inodebits.c:94:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x240000402:0x7fa:0x0].0x0 (0000000036d08056) 00010000:00010000:3.0:1776178204.036526:0:8250:0:(ldlm_inodebits.c:96:ldlm_reprocess_inodebits_queue()) Hint 8 00010000:00010000:3.0:1776178204.036530:0:8250:0:(ldlm_lockd.c:1565:ldlm_handle_enqueue0()) ### server-side enqueue handler END (lock 000000006a188f9a, rc 0) 00000100:00100000:3.0:1776178204.036560:0:8250:0:(service.c:2372:ptlrpc_server_handle_request()) Handled RPC req@000000003174f31d pname:cluuid+ref:pid:xid:nid:opc:job mdt00_003:b50963bd-8eba-4432-9a54-8888d00e3fa1+13:17641:x1862456979099456:12345-192.168.204.3@tcp:101:chmod.0 Request processed in 168us (1849us total) trans 0 rc 0/0 00000100:00100000:3.0:1776178204.036567:0:8250:0:(nrs_fifo.c:241:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.204.3@tcp, seq: 6571 00002000:00010000:1.0:1776178204.036856:0:20759:0:(ofd_lvb.c:324:ofd_lvbo_update()) res: [0x2c0000400:0x430:0x0] updating lvb size from disk: 0 -> 0 00002000:00010000:1.0:1776178204.036860:0:20759:0:(ofd_lvb.c:331:ofd_lvbo_update()) res: [0x2c0000400:0x430:0x0] updating lvb mtime from disk: 1776178154 -> 1776178154 00002000:00010000:1.0:1776178204.036862:0:20759:0:(ofd_lvb.c:337:ofd_lvbo_update()) res: [0x2c0000400:0x430:0x0] updating lvb atime from disk: 1776178154 -> 1776178154 00002000:00010000:2.0:1776178204.037109:0:8228:0:(ofd_lvb.c:324:ofd_lvbo_update()) res: [0x280000400:0x470:0x0] updating lvb size from disk: 0 -> 0 00002000:00010000:2.0:1776178204.037112:0:8228:0:(ofd_lvb.c:331:ofd_lvbo_update()) res: [0x280000400:0x470:0x0] updating lvb mtime from disk: 1776178154 -> 1776178154 00002000:00010000:2.0:1776178204.037113:0:8228:0:(ofd_lvb.c:337:ofd_lvbo_update()) res: [0x280000400:0x470:0x0] updating lvb atime from disk: 1776178154 -> 1776178154 00002000:00010000:2.0:1776178204.037115:0:8228:0:(ofd_lvb.c:343:ofd_lvbo_update()) res: [0x280000400:0x470:0x0] updating lvb ctime from disk: 1776178154 -> 1776178155 00002000:00010000:2.0:1776178204.037116:0:8228:0:(ofd_lvb.c:349:ofd_lvbo_update()) res: [0x280000400:0x470:0x0] updating lvb blocks from disk: 0 -> 0 00002000:00010000:1.0:1776178204.037444:0:20759:0:(ofd_lvb.c:343:ofd_lvbo_update()) res: [0x2c0000400:0x430:0x0] updating lvb ctime from disk: 1776178154 -> 1776178155 00002000:00010000:1.0:1776178204.037446:0:20759:0:(ofd_lvb.c:349:ofd_lvbo_update()) res: [0x2c0000400:0x430:0x0] updating lvb blocks from disk: 0 -> 0 00000100:00100000:1.0:1776178204.037496:0:20759:0:(service.c:2372:ptlrpc_server_handle_request()) Handled RPC req@000000008f3792e7 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_010:b50963bd-8eba-4432-9a54-8888d00e3fa1+11:2272:x1862456979099520:12345-192.168.204.3@tcp:2:chmod.0 Request processed in 1361us (2762us total) trans 4294969329 rc 0/0 00000100:00100000:1.0:1776178204.037507:0:20759:0:(nrs_fifo.c:241:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.204.3@tcp, seq: 5274 00000100:00100000:2.0:1776178204.037564:0:8228:0:(service.c:2372:ptlrpc_server_handle_request()) Handled RPC req@000000007bc8dbc1 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_001:b50963bd-8eba-4432-9a54-8888d00e3fa1+10:2273:x1862456979099392:12345-192.168.204.3@tcp:2:chmod.0 Request processed in 1975us (2883us total) trans 4294969357 rc 0/0 00000100:00100000:2.0:1776178204.037572:0:8228:0:(nrs_fifo.c:241:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.204.3@tcp, seq: 5273 00000100:00100000:3.0:1776178204.038480:0:3347:0:(events.c:361:request_in_callback()) peer: 12345-192.168.204.3@tcp (source: 12345-192.168.204.3@tcp) 00000100:00100000:2.0:1776178204.039012:0:8228:0:(service.c:2152:ptlrpc_server_handle_req_in()) got req x1862456979099584 00000100:00100000:2.0:1776178204.039093:0:8228:0:(nrs_fifo.c:179:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.204.3@tcp, seq: 5275 00000100:00100000:2.0:1776178204.039100:0:8228:0:(service.c:2324:ptlrpc_server_handle_request()) Handling RPC req@00000000542c8034 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_001:b50963bd-8eba-4432-9a54-8888d00e3fa1+10:2270:x1862456979099584:12345-192.168.204.3@tcp:2:chmod.0 00002000:00010000:2.0:1776178204.039297:0:8228:0:(ofd_lvb.c:324:ofd_lvbo_update()) res: [0x280000400:0x46f:0x0] updating lvb size from disk: 0 -> 0 00002000:00010000:2.0:1776178204.039300:0:8228:0:(ofd_lvb.c:331:ofd_lvbo_update()) res: [0x280000400:0x46f:0x0] updating lvb mtime from disk: 1776178153 -> 1776178153 00002000:00010000:2.0:1776178204.039302:0:8228:0:(ofd_lvb.c:337:ofd_lvbo_update()) res: [0x280000400:0x46f:0x0] updating lvb atime from disk: 1776178153 -> 1776178153 00002000:00010000:2.0:1776178204.039304:0:8228:0:(ofd_lvb.c:343:ofd_lvbo_update()) res: [0x280000400:0x46f:0x0] updating lvb ctime from disk: 1776178153 -> 1776178155 00002000:00010000:2.0:1776178204.039305:0:8228:0:(ofd_lvb.c:349:ofd_lvbo_update()) res: [0x280000400:0x46f:0x0] updating lvb blocks from disk: 0 -> 0 00000100:00100000:2.0:1776178204.039360:0:8228:0:(service.c:2372:ptlrpc_server_handle_request()) Handled RPC req@00000000542c8034 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_001:b50963bd-8eba-4432-9a54-8888d00e3fa1+11:2270:x1862456979099584:12345-192.168.204.3@tcp:2:chmod.0 Request processed in 260us (886us total) trans 4294969358 rc 0/0 00000100:00100000:2.0:1776178204.039367:0:8228:0:(nrs_fifo.c:241:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.204.3@tcp, seq: 5275 00000100:00100000:1.0:1776178204.041463:0:3347:0:(events.c:361:request_in_callback()) peer: 12345-192.168.204.3@tcp (source: 12345-192.168.204.3@tcp) 00000100:00100000:3.0:1776178204.042278:0:20759:0:(service.c:2152:ptlrpc_server_handle_req_in()) got req x1862456979099648 00000100:00100000:3.0:1776178204.042343:0:20759:0:(nrs_fifo.c:179:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.204.3@tcp, seq: 5276 00000100:00100000:3.0:1776178204.042351:0:20759:0:(service.c:2324:ptlrpc_server_handle_request()) Handling RPC req@000000005039d2a8 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_010:b50963bd-8eba-4432-9a54-8888d00e3fa1+11:2271:x1862456979099648:12345-192.168.204.3@tcp:2:chmod.0 00002000:00010000:3.0:1776178204.042508:0:20759:0:(ofd_lvb.c:324:ofd_lvbo_update()) res: [0x2c0000400:0x42f:0x0] updating lvb size from disk: 0 -> 0 00002000:00010000:3.0:1776178204.042511:0:20759:0:(ofd_lvb.c:331:ofd_lvbo_update()) res: [0x2c0000400:0x42f:0x0] updating lvb mtime from disk: 1776178153 -> 1776178153 00002000:00010000:3.0:1776178204.042513:0:20759:0:(ofd_lvb.c:337:ofd_lvbo_update()) res: [0x2c0000400:0x42f:0x0] updating lvb atime from disk: 1776178153 -> 1776178153 00002000:00010000:3.0:1776178204.042515:0:20759:0:(ofd_lvb.c:343:ofd_lvbo_update()) res: [0x2c0000400:0x42f:0x0] updating lvb ctime from disk: 1776178153 -> 1776178155 00002000:00010000:3.0:1776178204.042518:0:20759:0:(ofd_lvb.c:349:ofd_lvbo_update()) res: [0x2c0000400:0x42f:0x0] updating lvb blocks from disk: 0 -> 0 00000100:00100000:3.0:1776178204.042585:0:20759:0:(service.c:2372:ptlrpc_server_handle_request()) Handled RPC req@000000005039d2a8 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_010:b50963bd-8eba-4432-9a54-8888d00e3fa1+12:2271:x1862456979099648:12345-192.168.204.3@tcp:2:chmod.0 Request processed in 234us (1129us total) trans 4294969330 rc 0/0 00000100:00100000:3.0:1776178204.042593:0:20759:0:(nrs_fifo.c:241:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.204.3@tcp, seq: 5276 00000100:00100000:0.0:1776178204.052524:0:6257:0:(service.c:2372:ptlrpc_server_handle_request()) Handled RPC req@0000000007fa878c pname:cluuid+ref:pid:xid:nid:opc:job mdt00_002:b50963bd-8eba-4432-9a54-8888d00e3fa1+12:17639:x1862456979094080:12345-192.168.204.3@tcp:36:chmod.0 Request processed in 47480us (47669us total) trans 17179876257 rc 0/0 00000100:00100000:0.0:1776178204.052540:0:6257:0:(nrs_fifo.c:241:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.204.3@tcp, seq: 6559 00000100:00100000:1.0:1776178204.058433:0:3347:0:(events.c:361:request_in_callback()) peer: 12345-192.168.204.3@tcp (source: 12345-192.168.204.3@tcp) 00000100:00100000:0.0:1776178204.058616:0:6257:0:(service.c:2152:ptlrpc_server_handle_req_in()) got req x1862456979099712 00000100:00100000:0.0:1776178204.058763:0:6257:0:(nrs_fifo.c:179:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.204.3@tcp, seq: 6572 00000100:00100000:0.0:1776178204.058770:0:6257:0:(service.c:2324:ptlrpc_server_handle_request()) Handling RPC req@000000009d976bc3 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_002:b50963bd-8eba-4432-9a54-8888d00e3fa1+12:17639:x1862456979099712:12345-192.168.204.3@tcp:101:chmod.0 00010000:00010000:0.0:1776178204.058807:0:6257:0:(ldlm_lockd.c:1248:ldlm_handle_enqueue0()) ### server-side enqueue handler START 00010000:00010000:0.0:1776178204.058830:0:6257:0:(ldlm_lockd.c:1318:ldlm_handle_enqueue0()) ### server-side enqueue handler, new lock created ns: mdt-lustre-MDT0001_UUID lock: 00000000a211f94c/0xa40083ab9bf53595 lrc: 2/0,0 mode: --/CR res: [0x240000402:0x7f8:0x0].0x0 bits 0x0/0x0 rrc: 2 type: IBT gid 0 flags: 0x40000000000000 nid: local remote: 0x771f68aa1eb27b5e expref: -99 pid: 6257 timeout: 0 lvb_type: 0 00010000:00010000:1.0:1776178204.065328:0:6257:0:(ldlm_lock.c:1064:ldlm_granted_list_add_lock()) ### About to add lock: ns: mdt-lustre-MDT0001_UUID lock: 00000000a211f94c/0xa40083ab9bf53595 lrc: 3/0,0 mode: CR/CR res: [0x240000402:0x7f8:0x0].0x0 bits 0x8/0x0 rrc: 2 type: IBT gid 0 flags: 0x50000000000000 nid: 192.168.204.3@tcp remote: 0x771f68aa1eb27b5e expref: 13 pid: 6257 timeout: 0 lvb_type: 0 00010000:00010000:1.0:1776178204.065360:0:6257:0:(ldlm_lockd.c:1484:ldlm_handle_enqueue0()) ### server-side enqueue handler, sending reply (err=0, rc=0) ns: mdt-lustre-MDT0001_UUID lock: 00000000a211f94c/0xa40083ab9bf53595 lrc: 3/0,0 mode: CR/CR res: [0x240000402:0x7f8:0x0].0x0 bits 0x8/0x0 rrc: 2 type: IBT gid 0 flags: 0x40000000000000 nid: 192.168.204.3@tcp remote: 0x771f68aa1eb27b5e expref: 13 pid: 6257 timeout: 0 lvb_type: 0 00010000:00010000:1.0:1776178204.065390:0:6257:0:(ldlm_inodebits.c:94:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x240000402:0x7f8:0x0].0x0 (000000000b2f81d2) 00010000:00010000:1.0:1776178204.065393:0:6257:0:(ldlm_inodebits.c:96:ldlm_reprocess_inodebits_queue()) Hint 8 00010000:00010000:1.0:1776178204.065398:0:6257:0:(ldlm_lockd.c:1565:ldlm_handle_enqueue0()) ### server-side enqueue handler END (lock 00000000a211f94c, rc 0) 00000100:00100000:1.0:1776178204.065562:0:6257:0:(service.c:2372:ptlrpc_server_handle_request()) Handled RPC req@000000009d976bc3 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_002:b50963bd-8eba-4432-9a54-8888d00e3fa1+13:17639:x1862456979099712:12345-192.168.204.3@tcp:101:chmod.0 Request processed in 6789us (7136us total) trans 0 rc 0/0 00000100:00100000:1.0:1776178204.065572:0:6257:0:(nrs_fifo.c:241:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.204.3@tcp, seq: 6572 00000100:00100000:1.0:1776178204.068333:0:3346:0:(events.c:361:request_in_callback()) peer: 12345-192.168.204.3@tcp (source: 12345-192.168.204.3@tcp) 00000100:00100000:2.0:1776178204.068837:0:8228:0:(service.c:2152:ptlrpc_server_handle_req_in()) got req x1862456979099776 00000100:00100000:2.0:1776178204.068931:0:8228:0:(nrs_fifo.c:179:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.204.3@tcp, seq: 5277 00000100:00100000:2.0:1776178204.068940:0:8228:0:(service.c:2324:ptlrpc_server_handle_request()) Handling RPC req@00000000d66c78e0 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_001:b50963bd-8eba-4432-9a54-8888d00e3fa1+12:2270:x1862456979099776:12345-192.168.204.3@tcp:2:chmod.0 00002000:00010000:2.0:1776178204.069245:0:8228:0:(ofd_lvb.c:324:ofd_lvbo_update()) res: [0x2c0000400:0x42e:0x0] updating lvb size from disk: 0 -> 0 00002000:00010000:2.0:1776178204.069249:0:8228:0:(ofd_lvb.c:331:ofd_lvbo_update()) res: [0x2c0000400:0x42e:0x0] updating lvb mtime from disk: 1776178153 -> 1776178153 00002000:00010000:2.0:1776178204.069251:0:8228:0:(ofd_lvb.c:337:ofd_lvbo_update()) res: [0x2c0000400:0x42e:0x0] updating lvb atime from disk: 1776178153 -> 1776178153 00002000:00010000:2.0:1776178204.069255:0:8228:0:(ofd_lvb.c:343:ofd_lvbo_update()) res: [0x2c0000400:0x42e:0x0] updating lvb ctime from disk: 1776178153 -> 1776178155 00002000:00010000:2.0:1776178204.069257:0:8228:0:(ofd_lvb.c:349:ofd_lvbo_update()) res: [0x2c0000400:0x42e:0x0] updating lvb blocks from disk: 0 -> 0 00000100:00100000:2.0:1776178204.069322:0:8228:0:(service.c:2372:ptlrpc_server_handle_request()) Handled RPC req@00000000d66c78e0 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_001:b50963bd-8eba-4432-9a54-8888d00e3fa1+13:2270:x1862456979099776:12345-192.168.204.3@tcp:2:chmod.0 Request processed in 382us (1003us total) trans 4294969331 rc 0/0 00000100:00100000:2.0:1776178204.069331:0:8228:0:(nrs_fifo.c:241:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.204.3@tcp, seq: 5277 00010000:00010000:2.0:1776178204.128406:0:6244:0:(ldlm_lockd.c:2792:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:2.0:1776178204.131665:0:3348:0:(events.c:361:request_in_callback()) peer: 12345-192.168.204.3@tcp (source: 12345-192.168.204.3@tcp) 00000100:00100000:1.0:1776178204.132964:0:6257:0:(service.c:2152:ptlrpc_server_handle_req_in()) got req x1862456979099840 00000100:00100000:1.0:1776178204.133054:0:6257:0:(nrs_fifo.c:179:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.204.3@tcp, seq: 6573 00000100:00100000:1.0:1776178204.133107:0:6257:0:(service.c:2324:ptlrpc_server_handle_request()) Handling RPC req@00000000314f395b pname:cluuid+ref:pid:xid:nid:opc:job mdt00_002:b50963bd-8eba-4432-9a54-8888d00e3fa1+13:17664:x1862456979099840:12345-192.168.204.3@tcp:101:checkstat.0 00010000:00010000:1.0:1776178204.133129:0:6257:0:(ldlm_lockd.c:1248:ldlm_handle_enqueue0()) ### server-side enqueue handler START 00010000:00010000:1.0:1776178204.133163:0:6257:0:(ldlm_lockd.c:1318:ldlm_handle_enqueue0()) ### server-side enqueue handler, new lock created ns: mdt-lustre-MDT0001_UUID lock: 000000006b0afb52/0xa40083ab9bf5359c lrc: 2/0,0 mode: --/CR res: [0x240000402:0x7f7:0x0].0x0 bits 0x0/0x0 rrc: 2 type: IBT gid 0 flags: 0x40000000000000 nid: local remote: 0x771f68aa1eb27b65 expref: -99 pid: 6257 timeout: 0 lvb_type: 0 00010000:00010000:1.0:1776178204.133226:0:6257:0:(ldlm_lock.c:766:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(CR) ns: mdt-lustre-MDT0001_UUID lock: 0000000056462e0c/0xa40083ab9bf535a3 lrc: 3/1,0 mode: --/CR res: [0x240000402:0x7f7:0x0].0x0 bits 0x0/0x0 rrc: 3 type: IBT gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 6257 timeout: 0 lvb_type: 0 00010000:00010000:1.0:1776178204.133241:0:6257:0:(ldlm_lock.c:1064:ldlm_granted_list_add_lock()) ### About to add lock: ns: mdt-lustre-MDT0001_UUID lock: 0000000056462e0c/0xa40083ab9bf535a3 lrc: 3/1,0 mode: CR/CR res: [0x240000402:0x7f7:0x0].0x0 bits 0x2/0x0 rrc: 3 type: IBT gid 0 flags: 0x50210000000000 nid: local remote: 0x0 expref: -99 pid: 6257 timeout: 0 lvb_type: 0 00010000:00010000:1.0:1776178204.133252:0:6257:0:(ldlm_request.c:528:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: mdt-lustre-MDT0001_UUID lock: 0000000056462e0c/0xa40083ab9bf535a3 lrc: 3/1,0 mode: CR/CR res: [0x240000402:0x7f7:0x0].0x0 bits 0x2/0x0 rrc: 3 type: IBT gid 0 flags: 0x40210000000000 nid: local remote: 0x0 expref: -99 pid: 6257 timeout: 0 lvb_type: 0 00010000:00010000:1.0:1776178204.133264:0:6257:0:(ldlm_lock.c:766:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PR) ns: mdt-lustre-MDT0001_UUID lock: 00000000daa24072/0xa40083ab9bf535aa lrc: 3/1,0 mode: --/PR res: [0x240000402:0x7f7:0x0].0xcda302ea bits 0x0/0x0 rrc: 2 type: IBT gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 6257 timeout: 0 lvb_type: 0 00010000:00010000:1.0:1776178204.133270:0:6257:0:(ldlm_lock.c:1064:ldlm_granted_list_add_lock()) ### About to add lock: ns: mdt-lustre-MDT0001_UUID lock: 00000000daa24072/0xa40083ab9bf535aa lrc: 3/1,0 mode: PR/PR res: [0x240000402:0x7f7:0x0].0xcda302ea bits 0x2/0x0 rrc: 2 type: IBT gid 0 flags: 0x50210000000000 nid: local remote: 0x0 expref: -99 pid: 6257 timeout: 0 lvb_type: 0 00010000:00010000:1.0:1776178204.133276:0:6257:0:(ldlm_request.c:528:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: mdt-lustre-MDT0001_UUID lock: 00000000daa24072/0xa40083ab9bf535aa lrc: 3/1,0 mode: PR/PR res: [0x240000402:0x7f7:0x0].0xcda302ea bits 0x2/0x0 rrc: 2 type: IBT gid 0 flags: 0x40210000000000 nid: local remote: 0x0 expref: -99 pid: 6257 timeout: 0 lvb_type: 0 00010000:00010000:1.0:1776178204.133351:0:6257:0:(ldlm_lock.c:766:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PR) ns: mdt-lustre-MDT0001_UUID lock: 000000002cc4819c/0xa40083ab9bf535b1 lrc: 3/1,0 mode: --/PR res: [0x240000402:0x7f8:0x0].0x0 bits 0x0/0x0 rrc: 3 type: IBT gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 6257 timeout: 0 lvb_type: 0 00010000:00010000:1.0:1776178204.133359:0:6257:0:(ldlm_lock.c:1064:ldlm_granted_list_add_lock()) ### About to add lock: ns: mdt-lustre-MDT0001_UUID lock: 000000002cc4819c/0xa40083ab9bf535b1 lrc: 3/1,0 mode: PR/PR res: [0x240000402:0x7f8:0x0].0x0 bits 0x5b/0x0 rrc: 3 type: IBT gid 0 flags: 0x50210000000000 nid: local remote: 0x0 expref: -99 pid: 6257 timeout: 0 lvb_type: 0 00010000:00010000:1.0:1776178204.133371:0:6257:0:(ldlm_request.c:528:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: mdt-lustre-MDT0001_UUID lock: 000000002cc4819c/0xa40083ab9bf535b1 lrc: 3/1,0 mode: PR/PR res: [0x240000402:0x7f8:0x0].0x0 bits 0x5b/0x0 rrc: 3 type: IBT gid 0 flags: 0x40210000000000 nid: local remote: 0x0 expref: -99 pid: 6257 timeout: 0 lvb_type: 0 00000004:00200000:1.0:1776178204.133415:0:6257:0:(mdt_handler.c:912:mdt_pack_attr2body()) [0x240000402:0x7f8:0x0]: returning size 0 00000004:00200000:1.0:1776178204.133419:0:6257:0:(mdt_handler.c:1523:mdt_getattr_internal()) dirent count 0 stripe count 1 MDT count 2 00000004:00010000:1.0:1776178204.133460:0:6257:0:(mdt_handler.c:2353:mdt_getattr_name_lock()) ### Returning lock to client ns: mdt-lustre-MDT0001_UUID lock: 000000002cc4819c/0xa40083ab9bf535b1 lrc: 3/1,0 mode: PR/PR res: [0x240000402:0x7f8:0x0].0x0 bits 0x5b/0x0 rrc: 3 type: IBT gid 0 flags: 0x40210000000000 nid: local remote: 0x0 expref: -99 pid: 6257 timeout: 0 lvb_type: 0 00010000:00010000:1.0:1776178204.133472:0:6257:0:(ldlm_lock.c:1064:ldlm_granted_list_add_lock()) ### About to add lock: ns: mdt-lustre-MDT0001_UUID lock: 000000002cc4819c/0xa40083ab9bf535b1 lrc: 3/1,0 mode: PR/PR res: [0x240000402:0x7f8:0x0].0x0 bits 0x1b/0x0 rrc: 3 type: IBT gid 0 flags: 0x50210000000000 nid: local remote: 0x0 expref: -99 pid: 6257 timeout: 0 lvb_type: 0 00010000:00010000:1.0:1776178204.133480:0:6257:0:(ldlm_lock.c:820:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(CR) ns: mdt-lustre-MDT0001_UUID lock: 0000000056462e0c/0xa40083ab9bf535a3 lrc: 3/1,0 mode: CR/CR res: [0x240000402:0x7f7:0x0].0x0 bits 0x2/0x0 rrc: 3 type: IBT gid 0 flags: 0x50210000000000 nid: local remote: 0x0 expref: -99 pid: 6257 timeout: 0 lvb_type: 0 00010000:00010000:1.0:1776178204.133486:0:6257:0:(ldlm_lock.c:879:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: mdt-lustre-MDT0001_UUID lock: 0000000056462e0c/0xa40083ab9bf535a3 lrc: 2/0,0 mode: CR/CR res: [0x240000402:0x7f7:0x0].0x0 bits 0x2/0x0 rrc: 3 type: IBT gid 0 flags: 0x50210400000000 nid: local remote: 0x0 expref: -99 pid: 6257 timeout: 0 lvb_type: 0 00010000:00010000:1.0:1776178204.133493:0:6257:0:(ldlm_lockd.c:1879:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: mdt-lustre-MDT0001_UUID lock: 0000000056462e0c/0xa40083ab9bf535a3 lrc: 3/0,0 mode: CR/CR res: [0x240000402:0x7f7:0x0].0x0 bits 0x2/0x0 rrc: 3 type: IBT gid 0 flags: 0x40210400000000 nid: local remote: 0x0 expref: -99 pid: 6257 timeout: 0 lvb_type: 0 00010000:00010000:1.0:1776178204.133499:0:6257:0:(ldlm_lockd.c:1893:ldlm_handle_bl_callback()) Lock 0000000056462e0c already unused, calling callback (0000000079e78348) 00010000:00010000:1.0:1776178204.133503:0:6257:0:(ldlm_request.c:364:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: mdt-lustre-MDT0001_UUID lock: 0000000056462e0c/0xa40083ab9bf535a3 lrc: 3/0,0 mode: CR/CR res: [0x240000402:0x7f7:0x0].0x0 bits 0x2/0x0 rrc: 3 type: IBT gid 0 flags: 0x40210400000000 nid: local remote: 0x0 expref: -99 pid: 6257 timeout: 0 lvb_type: 0 00010000:00010000:1.0:1776178204.133510:0:6257:0:(ldlm_request.c:1257:ldlm_cli_cancel_local()) ### server-side local cancel ns: mdt-lustre-MDT0001_UUID lock: 0000000056462e0c/0xa40083ab9bf535a3 lrc: 4/0,0 mode: CR/CR res: [0x240000402:0x7f7:0x0].0x0 bits 0x2/0x0 rrc: 3 type: IBT gid 0 flags: 0x40218400000000 nid: local remote: 0x0 expref: -99 pid: 6257 timeout: 0 lvb_type: 0 00010000:00010000:1.0:1776178204.133521:0:6257:0:(ldlm_inodebits.c:94:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x240000402:0x7f7:0x0].0x0 (000000001fd37005) 00010000:00010000:1.0:1776178204.133524:0:6257:0:(ldlm_inodebits.c:96:ldlm_reprocess_inodebits_queue()) Hint 2 00010000:00010000:1.0:1776178204.133528:0:6257:0:(ldlm_lockd.c:1903:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: mdt-lustre-MDT0001_UUID lock: 0000000056462e0c/0xa40083ab9bf535a3 lrc: 2/0,0 mode: --/CR res: [0x240000402:0x7f7:0x0].0x0 bits 0x2/0x0 rrc: 3 type: IBT gid 0 flags: 0x44a19400000000 nid: local remote: 0x0 expref: -99 pid: 6257 timeout: 0 lvb_type: 0 00010000:00010000:1.0:1776178204.133534:0:6257:0:(ldlm_lock.c:216:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: mdt-lustre-MDT0001_UUID lock: 0000000056462e0c/0xa40083ab9bf535a3 lrc: 0/0,0 mode: --/CR res: [0x240000402:0x7f7:0x0].0x0 bits 0x2/0x0 rrc: 3 type: IBT gid 0 flags: 0x44a19400000000 nid: local remote: 0x0 expref: -99 pid: 6257 timeout: 0 lvb_type: 0 00010000:00010000:1.0:1776178204.133548:0:6257:0:(ldlm_lock.c:820:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PR) ns: mdt-lustre-MDT0001_UUID lock: 00000000daa24072/0xa40083ab9bf535aa lrc: 3/1,0 mode: PR/PR res: [0x240000402:0x7f7:0x0].0xcda302ea bits 0x2/0x0 rrc: 2 type: IBT gid 0 flags: 0x50210000000000 nid: local remote: 0x0 expref: -99 pid: 6257 timeout: 0 lvb_type: 0 00010000:00010000:1.0:1776178204.133552:0:6257:0:(ldlm_lock.c:879:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: mdt-lustre-MDT0001_UUID lock: 00000000daa24072/0xa40083ab9bf535aa lrc: 2/0,0 mode: PR/PR res: [0x240000402:0x7f7:0x0].0xcda302ea bits 0x2/0x0 rrc: 2 type: IBT gid 0 flags: 0x50210400000000 nid: local remote: 0x0 expref: -99 pid: 6257 timeout: 0 lvb_type: 0 00010000:00010000:1.0:1776178204.133558:0:6257:0:(ldlm_lockd.c:1879:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: mdt-lustre-MDT0001_UUID lock: 00000000daa24072/0xa40083ab9bf535aa lrc: 3/0,0 mode: PR/PR res: [0x240000402:0x7f7:0x0].0xcda302ea bits 0x2/0x0 rrc: 2 type: IBT gid 0 flags: 0x40210400000000 nid: local remote: 0x0 expref: -99 pid: 6257 timeout: 0 lvb_type: 0 00010000:00010000:1.0:1776178204.133563:0:6257:0:(ldlm_lockd.c:1893:ldlm_handle_bl_callback()) Lock 00000000daa24072 already unused, calling callback (0000000079e78348) 00010000:00010000:1.0:1776178204.133565:0:6257:0:(ldlm_request.c:364:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: mdt-lustre-MDT0001_UUID lock: 00000000daa24072/0xa40083ab9bf535aa lrc: 3/0,0 mode: PR/PR res: [0x240000402:0x7f7:0x0].0xcda302ea bits 0x2/0x0 rrc: 2 type: IBT gid 0 flags: 0x40210400000000 nid: local remote: 0x0 expref: -99 pid: 6257 timeout: 0 lvb_type: 0 00010000:00010000:1.0:1776178204.133570:0:6257:0:(ldlm_request.c:1257:ldlm_cli_cancel_local()) ### server-side local cancel ns: mdt-lustre-MDT0001_UUID lock: 00000000daa24072/0xa40083ab9bf535aa lrc: 4/0,0 mode: PR/PR res: [0x240000402:0x7f7:0x0].0xcda302ea bits 0x2/0x0 rrc: 2 type: IBT gid 0 flags: 0x40218400000000 nid: local remote: 0x0 expref: -99 pid: 6257 timeout: 0 lvb_type: 0 00010000:00010000:1.0:1776178204.133577:0:6257:0:(ldlm_inodebits.c:94:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x240000402:0x7f7:0x0].0xcda302ea (000000009742f5ea) 00010000:00010000:1.0:1776178204.133579:0:6257:0:(ldlm_inodebits.c:96:ldlm_reprocess_inodebits_queue()) Hint 2 00010000:00010000:1.0:1776178204.133581:0:6257:0:(ldlm_lockd.c:1903:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: mdt-lustre-MDT0001_UUID lock: 00000000daa24072/0xa40083ab9bf535aa lrc: 2/0,0 mode: --/PR res: [0x240000402:0x7f7:0x0].0xcda302ea bits 0x2/0x0 rrc: 2 type: IBT gid 0 flags: 0x44a19400000000 nid: local remote: 0x0 expref: -99 pid: 6257 timeout: 0 lvb_type: 0 00010000:00010000:1.0:1776178204.133586:0:6257:0:(ldlm_lock.c:216:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: mdt-lustre-MDT0001_UUID lock: 00000000daa24072/0xa40083ab9bf535aa lrc: 0/0,0 mode: --/PR res: [0x240000402:0x7f7:0x0].0xcda302ea bits 0x2/0x0 rrc: 2 type: IBT gid 0 flags: 0x44a19400000000 nid: local remote: 0x0 expref: -99 pid: 6257 timeout: 0 lvb_type: 0 00010000:00010000:1.0:1776178204.133615:0:6257:0:(ldlm_lock.c:216:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: mdt-lustre-MDT0001_UUID lock: 000000006b0afb52/0xa40083ab9bf5359c lrc: 0/0,0 mode: --/CR res: [0x240000402:0x7f7:0x0].0x0 bits 0x2/0x0 rrc: 2 type: IBT gid 0 flags: 0x44000000000000 nid: 192.168.204.3@tcp remote: 0x771f68aa1eb27b65 expref: 15 pid: 6257 timeout: 0 lvb_type: 0 00010000:00010000:1.0:1776178204.133632:0:6257:0:(ldlm_lockd.c:1484:ldlm_handle_enqueue0()) ### server-side enqueue handler, sending reply (err=0, rc=0) ns: mdt-lustre-MDT0001_UUID lock: 000000002cc4819c/0xa40083ab9bf535b1 lrc: 3/0,0 mode: PR/PR res: [0x240000402:0x7f8:0x0].0x0 bits 0x1b/0x0 rrc: 3 type: IBT gid 0 flags: 0x40200000000000 nid: 192.168.204.3@tcp remote: 0x771f68aa1eb27b65 expref: 14 pid: 6257 timeout: 0 lvb_type: 0 00010000:00010000:1.0:1776178204.133639:0:6257:0:(ldlm_inodebits.c:94:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x240000402:0x7f8:0x0].0x0 (000000000b2f81d2) 00010000:00010000:1.0:1776178204.133641:0:6257:0:(ldlm_inodebits.c:96:ldlm_reprocess_inodebits_queue()) Hint 1b 00010000:00010000:1.0:1776178204.133643:0:6257:0:(ldlm_lockd.c:1565:ldlm_handle_enqueue0()) ### server-side enqueue handler END (lock 000000002cc4819c, rc 0) 00000100:00100000:1.0:1776178204.133740:0:6257:0:(service.c:2372:ptlrpc_server_handle_request()) Handled RPC req@00000000314f395b pname:cluuid+ref:pid:xid:nid:opc:job mdt00_002:b50963bd-8eba-4432-9a54-8888d00e3fa1+14:17664:x1862456979099840:12345-192.168.204.3@tcp:101:checkstat.0 Request processed in 678us (2084us total) trans 0 rc 0/0 00000100:00100000:1.0:1776178204.133748:0:6257:0:(nrs_fifo.c:241:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.204.3@tcp, seq: 6573 00000100:00100000:1.0:1776178204.156744:0:3347:0:(events.c:361:request_in_callback()) peer: 12345-192.168.204.3@tcp (source: 12345-192.168.204.3@tcp) 00000100:00100000:3.0:1776178204.156848:0:11281:0:(service.c:2152:ptlrpc_server_handle_req_in()) got req x1862456979099904 00000100:00100000:3.0:1776178204.156911:0:11281:0:(nrs_fifo.c:179:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.204.3@tcp, seq: 6574 00000100:00100000:3.0:1776178204.156918:0:11281:0:(service.c:2324:ptlrpc_server_handle_request()) Handling RPC req@0000000058300775 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_004:b50963bd-8eba-4432-9a54-8888d00e3fa1+14:17665:x1862456979099904:12345-192.168.204.3@tcp:101:checkstat.0 00010000:00010000:3.0:1776178204.156942:0:11281:0:(ldlm_lockd.c:1248:ldlm_handle_enqueue0()) ### server-side enqueue handler START 00010000:00010000:3.0:1776178204.156980:0:11281:0:(ldlm_lockd.c:1318:ldlm_handle_enqueue0()) ### server-side enqueue handler, new lock created ns: mdt-lustre-MDT0001_UUID lock: 00000000014a8a18/0xa40083ab9bf535b8 lrc: 2/0,0 mode: --/CR res: [0x240000402:0x7f7:0x0].0x0 bits 0x0/0x0 rrc: 2 type: IBT gid 0 flags: 0x40000000000000 nid: local remote: 0x771f68aa1eb27b6c expref: -99 pid: 11281 timeout: 0 lvb_type: 0 00010000:00010000:3.0:1776178204.157118:0:11281:0:(ldlm_lock.c:766:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(CR) ns: mdt-lustre-MDT0001_UUID lock: 00000000c1e7ea3e/0xa40083ab9bf535bf lrc: 3/1,0 mode: --/CR res: [0x240000402:0x7f7:0x0].0x0 bits 0x0/0x0 rrc: 3 type: IBT gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 11281 timeout: 0 lvb_type: 0 00010000:00010000:3.0:1776178204.157134:0:11281:0:(ldlm_lock.c:1064:ldlm_granted_list_add_lock()) ### About to add lock: ns: mdt-lustre-MDT0001_UUID lock: 00000000c1e7ea3e/0xa40083ab9bf535bf lrc: 3/1,0 mode: CR/CR res: [0x240000402:0x7f7:0x0].0x0 bits 0x2/0x0 rrc: 3 type: IBT gid 0 flags: 0x50210000000000 nid: local remote: 0x0 expref: -99 pid: 11281 timeout: 0 lvb_type: 0 00010000:00010000:3.0:1776178204.157145:0:11281:0:(ldlm_request.c:528:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: mdt-lustre-MDT0001_UUID lock: 00000000c1e7ea3e/0xa40083ab9bf535bf lrc: 3/1,0 mode: CR/CR res: [0x240000402:0x7f7:0x0].0x0 bits 0x2/0x0 rrc: 3 type: IBT gid 0 flags: 0x40210000000000 nid: local remote: 0x0 expref: -99 pid: 11281 timeout: 0 lvb_type: 0 00010000:00010000:3.0:1776178204.157156:0:11281:0:(ldlm_lock.c:766:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PR) ns: mdt-lustre-MDT0001_UUID lock: 00000000dcea299c/0xa40083ab9bf535c6 lrc: 3/1,0 mode: --/PR res: [0x240000402:0x7f7:0x0].0x15fcbbea bits 0x0/0x0 rrc: 2 type: IBT gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 11281 timeout: 0 lvb_type: 0 00010000:00010000:3.0:1776178204.157163:0:11281:0:(ldlm_lock.c:1064:ldlm_granted_list_add_lock()) ### About to add lock: ns: mdt-lustre-MDT0001_UUID lock: 00000000dcea299c/0xa40083ab9bf535c6 lrc: 3/1,0 mode: PR/PR res: [0x240000402:0x7f7:0x0].0x15fcbbea bits 0x2/0x0 rrc: 2 type: IBT gid 0 flags: 0x50210000000000 nid: local remote: 0x0 expref: -99 pid: 11281 timeout: 0 lvb_type: 0 00010000:00010000:3.0:1776178204.157169:0:11281:0:(ldlm_request.c:528:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: mdt-lustre-MDT0001_UUID lock: 00000000dcea299c/0xa40083ab9bf535c6 lrc: 3/1,0 mode: PR/PR res: [0x240000402:0x7f7:0x0].0x15fcbbea bits 0x2/0x0 rrc: 2 type: IBT gid 0 flags: 0x40210000000000 nid: local remote: 0x0 expref: -99 pid: 11281 timeout: 0 lvb_type: 0 00010000:00010000:3.0:1776178204.157281:0:11281:0:(ldlm_lock.c:766:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PR) ns: mdt-lustre-MDT0001_UUID lock: 000000002a990613/0xa40083ab9bf535cd lrc: 3/1,0 mode: --/PR res: [0x240000402:0x7f9:0x0].0x0 bits 0x0/0x0 rrc: 3 type: IBT gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 11281 timeout: 0 lvb_type: 0 00010000:00010000:3.0:1776178204.157290:0:11281:0:(ldlm_lock.c:1064:ldlm_granted_list_add_lock()) ### About to add lock: ns: mdt-lustre-MDT0001_UUID lock: 000000002a990613/0xa40083ab9bf535cd lrc: 3/1,0 mode: PR/PR res: [0x240000402:0x7f9:0x0].0x0 bits 0x5b/0x0 rrc: 3 type: IBT gid 0 flags: 0x50210000000000 nid: local remote: 0x0 expref: -99 pid: 11281 timeout: 0 lvb_type: 0 00010000:00010000:3.0:1776178204.157297:0:11281:0:(ldlm_request.c:528:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: mdt-lustre-MDT0001_UUID lock: 000000002a990613/0xa40083ab9bf535cd lrc: 3/1,0 mode: PR/PR res: [0x240000402:0x7f9:0x0].0x0 bits 0x5b/0x0 rrc: 3 type: IBT gid 0 flags: 0x40210000000000 nid: local remote: 0x0 expref: -99 pid: 11281 timeout: 0 lvb_type: 0 00000004:00200000:3.0:1776178204.157340:0:11281:0:(mdt_handler.c:912:mdt_pack_attr2body()) [0x240000402:0x7f9:0x0]: returning size 0 00000004:00200000:3.0:1776178204.157346:0:11281:0:(mdt_handler.c:1523:mdt_getattr_internal()) dirent count 0 stripe count 1 MDT count 2 00000004:00010000:3.0:1776178204.157371:0:11281:0:(mdt_handler.c:2353:mdt_getattr_name_lock()) ### Returning lock to client ns: mdt-lustre-MDT0001_UUID lock: 000000002a990613/0xa40083ab9bf535cd lrc: 3/1,0 mode: PR/PR res: [0x240000402:0x7f9:0x0].0x0 bits 0x5b/0x0 rrc: 3 type: IBT gid 0 flags: 0x40210000000000 nid: local remote: 0x0 expref: -99 pid: 11281 timeout: 0 lvb_type: 0 00010000:00010000:3.0:1776178204.157383:0:11281:0:(ldlm_lock.c:1064:ldlm_granted_list_add_lock()) ### About to add lock: ns: mdt-lustre-MDT0001_UUID lock: 000000002a990613/0xa40083ab9bf535cd lrc: 3/1,0 mode: PR/PR res: [0x240000402:0x7f9:0x0].0x0 bits 0x1b/0x0 rrc: 3 type: IBT gid 0 flags: 0x50210000000000 nid: local remote: 0x0 expref: -99 pid: 11281 timeout: 0 lvb_type: 0 00010000:00010000:3.0:1776178204.157391:0:11281:0:(ldlm_lock.c:820:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(CR) ns: mdt-lustre-MDT0001_UUID lock: 00000000c1e7ea3e/0xa40083ab9bf535bf lrc: 3/1,0 mode: CR/CR res: [0x240000402:0x7f7:0x0].0x0 bits 0x2/0x0 rrc: 3 type: IBT gid 0 flags: 0x50210000000000 nid: local remote: 0x0 expref: -99 pid: 11281 timeout: 0 lvb_type: 0 00010000:00010000:3.0:1776178204.157397:0:11281:0:(ldlm_lock.c:879:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: mdt-lustre-MDT0001_UUID lock: 00000000c1e7ea3e/0xa40083ab9bf535bf lrc: 2/0,0 mode: CR/CR res: [0x240000402:0x7f7:0x0].0x0 bits 0x2/0x0 rrc: 3 type: IBT gid 0 flags: 0x50210400000000 nid: local remote: 0x0 expref: -99 pid: 11281 timeout: 0 lvb_type: 0 00010000:00010000:3.0:1776178204.157404:0:11281:0:(ldlm_lockd.c:1879:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: mdt-lustre-MDT0001_UUID lock: 00000000c1e7ea3e/0xa40083ab9bf535bf lrc: 3/0,0 mode: CR/CR res: [0x240000402:0x7f7:0x0].0x0 bits 0x2/0x0 rrc: 3 type: IBT gid 0 flags: 0x40210400000000 nid: local remote: 0x0 expref: -99 pid: 11281 timeout: 0 lvb_type: 0 00010000:00010000:3.0:1776178204.157410:0:11281:0:(ldlm_lockd.c:1893:ldlm_handle_bl_callback()) Lock 00000000c1e7ea3e already unused, calling callback (0000000079e78348) 00010000:00010000:3.0:1776178204.157414:0:11281:0:(ldlm_request.c:364:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: mdt-lustre-MDT0001_UUID lock: 00000000c1e7ea3e/0xa40083ab9bf535bf lrc: 3/0,0 mode: CR/CR res: [0x240000402:0x7f7:0x0].0x0 bits 0x2/0x0 rrc: 3 type: IBT gid 0 flags: 0x40210400000000 nid: local remote: 0x0 expref: -99 pid: 11281 timeout: 0 lvb_type: 0 00010000:00010000:3.0:1776178204.157425:0:11281:0:(ldlm_request.c:1257:ldlm_cli_cancel_local()) ### server-side local cancel ns: mdt-lustre-MDT0001_UUID lock: 00000000c1e7ea3e/0xa40083ab9bf535bf lrc: 4/0,0 mode: CR/CR res: [0x240000402:0x7f7:0x0].0x0 bits 0x2/0x0 rrc: 3 type: IBT gid 0 flags: 0x40218400000000 nid: local remote: 0x0 expref: -99 pid: 11281 timeout: 0 lvb_type: 0 00010000:00010000:3.0:1776178204.157436:0:11281:0:(ldlm_inodebits.c:94:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x240000402:0x7f7:0x0].0x0 (0000000022e38283) 00010000:00010000:3.0:1776178204.157438:0:11281:0:(ldlm_inodebits.c:96:ldlm_reprocess_inodebits_queue()) Hint 2 00010000:00010000:3.0:1776178204.157443:0:11281:0:(ldlm_lockd.c:1903:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: mdt-lustre-MDT0001_UUID lock: 00000000c1e7ea3e/0xa40083ab9bf535bf lrc: 2/0,0 mode: --/CR res: [0x240000402:0x7f7:0x0].0x0 bits 0x2/0x0 rrc: 3 type: IBT gid 0 flags: 0x44a19400000000 nid: local remote: 0x0 expref: -99 pid: 11281 timeout: 0 lvb_type: 0 00010000:00010000:3.0:1776178204.157449:0:11281:0:(ldlm_lock.c:216:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: mdt-lustre-MDT0001_UUID lock: 00000000c1e7ea3e/0xa40083ab9bf535bf lrc: 0/0,0 mode: --/CR res: [0x240000402:0x7f7:0x0].0x0 bits 0x2/0x0 rrc: 3 type: IBT gid 0 flags: 0x44a19400000000 nid: local remote: 0x0 expref: -99 pid: 11281 timeout: 0 lvb_type: 0 00010000:00010000:3.0:1776178204.157462:0:11281:0:(ldlm_lock.c:820:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PR) ns: mdt-lustre-MDT0001_UUID lock: 00000000dcea299c/0xa40083ab9bf535c6 lrc: 3/1,0 mode: PR/PR res: [0x240000402:0x7f7:0x0].0x15fcbbea bits 0x2/0x0 rrc: 2 type: IBT gid 0 flags: 0x50210000000000 nid: local remote: 0x0 expref: -99 pid: 11281 timeout: 0 lvb_type: 0 00010000:00010000:3.0:1776178204.157467:0:11281:0:(ldlm_lock.c:879:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: mdt-lustre-MDT0001_UUID lock: 00000000dcea299c/0xa40083ab9bf535c6 lrc: 2/0,0 mode: PR/PR res: [0x240000402:0x7f7:0x0].0x15fcbbea bits 0x2/0x0 rrc: 2 type: IBT gid 0 flags: 0x50210400000000 nid: local remote: 0x0 expref: -99 pid: 11281 timeout: 0 lvb_type: 0 00010000:00010000:3.0:1776178204.157471:0:11281:0:(ldlm_lockd.c:1879:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: mdt-lustre-MDT0001_UUID lock: 00000000dcea299c/0xa40083ab9bf535c6 lrc: 3/0,0 mode: PR/PR res: [0x240000402:0x7f7:0x0].0x15fcbbea bits 0x2/0x0 rrc: 2 type: IBT gid 0 flags: 0x40210400000000 nid: local remote: 0x0 expref: -99 pid: 11281 timeout: 0 lvb_type: 0 00010000:00010000:3.0:1776178204.157476:0:11281:0:(ldlm_lockd.c:1893:ldlm_handle_bl_callback()) Lock 00000000dcea299c already unused, calling callback (0000000079e78348) 00010000:00010000:3.0:1776178204.157478:0:11281:0:(ldlm_request.c:364:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: mdt-lustre-MDT0001_UUID lock: 00000000dcea299c/0xa40083ab9bf535c6 lrc: 3/0,0 mode: PR/PR res: [0x240000402:0x7f7:0x0].0x15fcbbea bits 0x2/0x0 rrc: 2 type: IBT gid 0 flags: 0x40210400000000 nid: local remote: 0x0 expref: -99 pid: 11281 timeout: 0 lvb_type: 0 00010000:00010000:3.0:1776178204.157483:0:11281:0:(ldlm_request.c:1257:ldlm_cli_cancel_local()) ### server-side local cancel ns: mdt-lustre-MDT0001_UUID lock: 00000000dcea299c/0xa40083ab9bf535c6 lrc: 4/0,0 mode: PR/PR res: [0x240000402:0x7f7:0x0].0x15fcbbea bits 0x2/0x0 rrc: 2 type: IBT gid 0 flags: 0x40218400000000 nid: local remote: 0x0 expref: -99 pid: 11281 timeout: 0 lvb_type: 0 00010000:00010000:3.0:1776178204.157488:0:11281:0:(ldlm_inodebits.c:94:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x240000402:0x7f7:0x0].0x15fcbbea (000000009f2a4d4c) 00010000:00010000:3.0:1776178204.157490:0:11281:0:(ldlm_inodebits.c:96:ldlm_reprocess_inodebits_queue()) Hint 2 00010000:00010000:3.0:1776178204.157492:0:11281:0:(ldlm_lockd.c:1903:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: mdt-lustre-MDT0001_UUID lock: 00000000dcea299c/0xa40083ab9bf535c6 lrc: 2/0,0 mode: --/PR res: [0x240000402:0x7f7:0x0].0x15fcbbea bits 0x2/0x0 rrc: 2 type: IBT gid 0 flags: 0x44a19400000000 nid: local remote: 0x0 expref: -99 pid: 11281 timeout: 0 lvb_type: 0 00010000:00010000:3.0:1776178204.157496:0:11281:0:(ldlm_lock.c:216:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: mdt-lustre-MDT0001_UUID lock: 00000000dcea299c/0xa40083ab9bf535c6 lrc: 0/0,0 mode: --/PR res: [0x240000402:0x7f7:0x0].0x15fcbbea bits 0x2/0x0 rrc: 2 type: IBT gid 0 flags: 0x44a19400000000 nid: local remote: 0x0 expref: -99 pid: 11281 timeout: 0 lvb_type: 0 00010000:00010000:3.0:1776178204.157525:0:11281:0:(ldlm_lock.c:216:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: mdt-lustre-MDT0001_UUID lock: 00000000014a8a18/0xa40083ab9bf535b8 lrc: 0/0,0 mode: --/CR res: [0x240000402:0x7f7:0x0].0x0 bits 0x2/0x0 rrc: 2 type: IBT gid 0 flags: 0x44000000000000 nid: 192.168.204.3@tcp remote: 0x771f68aa1eb27b6c expref: 16 pid: 11281 timeout: 0 lvb_type: 0 00010000:00010000:3.0:1776178204.157539:0:11281:0:(ldlm_lockd.c:1484:ldlm_handle_enqueue0()) ### server-side enqueue handler, sending reply (err=0, rc=0) ns: mdt-lustre-MDT0001_UUID lock: 000000002a990613/0xa40083ab9bf535cd lrc: 3/0,0 mode: PR/PR res: [0x240000402:0x7f9:0x0].0x0 bits 0x1b/0x0 rrc: 3 type: IBT gid 0 flags: 0x40200000000000 nid: 192.168.204.3@tcp remote: 0x771f68aa1eb27b6c expref: 15 pid: 11281 timeout: 0 lvb_type: 0 00010000:00010000:3.0:1776178204.157545:0:11281:0:(ldlm_inodebits.c:94:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x240000402:0x7f9:0x0].0x0 (0000000056a46617) 00010000:00010000:3.0:1776178204.157547:0:11281:0:(ldlm_inodebits.c:96:ldlm_reprocess_inodebits_queue()) Hint 1b 00010000:00010000:3.0:1776178204.157549:0:11281:0:(ldlm_lockd.c:1565:ldlm_handle_enqueue0()) ### server-side enqueue handler END (lock 000000002a990613, rc 0) 00000100:00100000:3.0:1776178204.157619:0:11281:0:(service.c:2372:ptlrpc_server_handle_request()) Handled RPC req@0000000058300775 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_004:b50963bd-8eba-4432-9a54-8888d00e3fa1+15:17665:x1862456979099904:12345-192.168.204.3@tcp:101:checkstat.0 Request processed in 699us (884us total) trans 0 rc 0/0 00000100:00100000:3.0:1776178204.157627:0:11281:0:(nrs_fifo.c:241:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.204.3@tcp, seq: 6574 00000100:00100000:2.0:1776178204.175385:0:3346:0:(events.c:361:request_in_callback()) peer: 12345-192.168.204.3@tcp (source: 12345-192.168.204.3@tcp) 00000100:00100000:3.0:1776178204.175545:0:6257:0:(service.c:2152:ptlrpc_server_handle_req_in()) got req x1862456979099968 00000100:00100000:3.0:1776178204.175612:0:6257:0:(nrs_fifo.c:179:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.204.3@tcp, seq: 6575 00000100:00100000:3.0:1776178204.175620:0:6257:0:(service.c:2324:ptlrpc_server_handle_request()) Handling RPC req@000000004547bc6e pname:cluuid+ref:pid:xid:nid:opc:job mdt00_002:b50963bd-8eba-4432-9a54-8888d00e3fa1+15:17666:x1862456979099968:12345-192.168.204.3@tcp:101:checkstat.0 00010000:00010000:3.0:1776178204.175655:0:6257:0:(ldlm_lockd.c:1248:ldlm_handle_enqueue0()) ### server-side enqueue handler START 00010000:00010000:3.0:1776178204.175682:0:6257:0:(ldlm_lockd.c:1318:ldlm_handle_enqueue0()) ### server-side enqueue handler, new lock created ns: mdt-lustre-MDT0001_UUID lock: 00000000014a8a18/0xa40083ab9bf535d4 lrc: 2/0,0 mode: --/CR res: [0x240000402:0x7f7:0x0].0x0 bits 0x0/0x0 rrc: 2 type: IBT gid 0 flags: 0x40000000000000 nid: local remote: 0x771f68aa1eb27b73 expref: -99 pid: 6257 timeout: 0 lvb_type: 0 00010000:00010000:3.0:1776178204.175756:0:6257:0:(ldlm_lock.c:766:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(CR) ns: mdt-lustre-MDT0001_UUID lock: 00000000dcea299c/0xa40083ab9bf535db lrc: 3/1,0 mode: --/CR res: [0x240000402:0x7f7:0x0].0x0 bits 0x0/0x0 rrc: 3 type: IBT gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 6257 timeout: 0 lvb_type: 0 00010000:00010000:3.0:1776178204.175773:0:6257:0:(ldlm_lock.c:1064:ldlm_granted_list_add_lock()) ### About to add lock: ns: mdt-lustre-MDT0001_UUID lock: 00000000dcea299c/0xa40083ab9bf535db lrc: 3/1,0 mode: CR/CR res: [0x240000402:0x7f7:0x0].0x0 bits 0x2/0x0 rrc: 3 type: IBT gid 0 flags: 0x50210000000000 nid: local remote: 0x0 expref: -99 pid: 6257 timeout: 0 lvb_type: 0 00010000:00010000:3.0:1776178204.175787:0:6257:0:(ldlm_request.c:528:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: mdt-lustre-MDT0001_UUID lock: 00000000dcea299c/0xa40083ab9bf535db lrc: 3/1,0 mode: CR/CR res: [0x240000402:0x7f7:0x0].0x0 bits 0x2/0x0 rrc: 3 type: IBT gid 0 flags: 0x40210000000000 nid: local remote: 0x0 expref: -99 pid: 6257 timeout: 0 lvb_type: 0 00010000:00010000:3.0:1776178204.175801:0:6257:0:(ldlm_lock.c:766:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PR) ns: mdt-lustre-MDT0001_UUID lock: 00000000c1e7ea3e/0xa40083ab9bf535e2 lrc: 3/1,0 mode: --/PR res: [0x240000402:0x7f7:0x0].0x5e5674ea bits 0x0/0x0 rrc: 2 type: IBT gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 6257 timeout: 0 lvb_type: 0 00010000:00010000:3.0:1776178204.175808:0:6257:0:(ldlm_lock.c:1064:ldlm_granted_list_add_lock()) ### About to add lock: ns: mdt-lustre-MDT0001_UUID lock: 00000000c1e7ea3e/0xa40083ab9bf535e2 lrc: 3/1,0 mode: PR/PR res: [0x240000402:0x7f7:0x0].0x5e5674ea bits 0x2/0x0 rrc: 2 type: IBT gid 0 flags: 0x50210000000000 nid: local remote: 0x0 expref: -99 pid: 6257 timeout: 0 lvb_type: 0 00010000:00010000:3.0:1776178204.175815:0:6257:0:(ldlm_request.c:528:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: mdt-lustre-MDT0001_UUID lock: 00000000c1e7ea3e/0xa40083ab9bf535e2 lrc: 3/1,0 mode: PR/PR res: [0x240000402:0x7f7:0x0].0x5e5674ea bits 0x2/0x0 rrc: 2 type: IBT gid 0 flags: 0x40210000000000 nid: local remote: 0x0 expref: -99 pid: 6257 timeout: 0 lvb_type: 0 00010000:00010000:3.0:1776178204.175895:0:6257:0:(ldlm_lock.c:766:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PR) ns: mdt-lustre-MDT0001_UUID lock: 000000008cbcbaf5/0xa40083ab9bf535e9 lrc: 3/1,0 mode: --/PR res: [0x240000402:0x7fa:0x0].0x0 bits 0x0/0x0 rrc: 3 type: IBT gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 6257 timeout: 0 lvb_type: 0 00010000:00010000:3.0:1776178204.175905:0:6257:0:(ldlm_lock.c:1064:ldlm_granted_list_add_lock()) ### About to add lock: ns: mdt-lustre-MDT0001_UUID lock: 000000008cbcbaf5/0xa40083ab9bf535e9 lrc: 3/1,0 mode: PR/PR res: [0x240000402:0x7fa:0x0].0x0 bits 0x5b/0x0 rrc: 3 type: IBT gid 0 flags: 0x50210000000000 nid: local remote: 0x0 expref: -99 pid: 6257 timeout: 0 lvb_type: 0 00010000:00010000:3.0:1776178204.175913:0:6257:0:(ldlm_request.c:528:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: mdt-lustre-MDT0001_UUID lock: 000000008cbcbaf5/0xa40083ab9bf535e9 lrc: 3/1,0 mode: PR/PR res: [0x240000402:0x7fa:0x0].0x0 bits 0x5b/0x0 rrc: 3 type: IBT gid 0 flags: 0x40210000000000 nid: local remote: 0x0 expref: -99 pid: 6257 timeout: 0 lvb_type: 0 00000004:00200000:3.0:1776178204.175956:0:6257:0:(mdt_handler.c:912:mdt_pack_attr2body()) [0x240000402:0x7fa:0x0]: returning size 0 00000004:00200000:3.0:1776178204.175972:0:6257:0:(mdt_handler.c:1523:mdt_getattr_internal()) dirent count 0 stripe count 1 MDT count 2 00000004:00010000:3.0:1776178204.176008:0:6257:0:(mdt_handler.c:2353:mdt_getattr_name_lock()) ### Returning lock to client ns: mdt-lustre-MDT0001_UUID lock: 000000008cbcbaf5/0xa40083ab9bf535e9 lrc: 3/1,0 mode: PR/PR res: [0x240000402:0x7fa:0x0].0x0 bits 0x5b/0x0 rrc: 3 type: IBT gid 0 flags: 0x40210000000000 nid: local remote: 0x0 expref: -99 pid: 6257 timeout: 0 lvb_type: 0 00010000:00010000:3.0:1776178204.176022:0:6257:0:(ldlm_lock.c:1064:ldlm_granted_list_add_lock()) ### About to add lock: ns: mdt-lustre-MDT0001_UUID lock: 000000008cbcbaf5/0xa40083ab9bf535e9 lrc: 3/1,0 mode: PR/PR res: [0x240000402:0x7fa:0x0].0x0 bits 0x1b/0x0 rrc: 3 type: IBT gid 0 flags: 0x50210000000000 nid: local remote: 0x0 expref: -99 pid: 6257 timeout: 0 lvb_type: 0 00010000:00010000:3.0:1776178204.176032:0:6257:0:(ldlm_lock.c:820:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(CR) ns: mdt-lustre-MDT0001_UUID lock: 00000000dcea299c/0xa40083ab9bf535db lrc: 3/1,0 mode: CR/CR res: [0x240000402:0x7f7:0x0].0x0 bits 0x2/0x0 rrc: 3 type: IBT gid 0 flags: 0x50210000000000 nid: local remote: 0x0 expref: -99 pid: 6257 timeout: 0 lvb_type: 0 00010000:00010000:3.0:1776178204.176040:0:6257:0:(ldlm_lock.c:879:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: mdt-lustre-MDT0001_UUID lock: 00000000dcea299c/0xa40083ab9bf535db lrc: 2/0,0 mode: CR/CR res: [0x240000402:0x7f7:0x0].0x0 bits 0x2/0x0 rrc: 3 type: IBT gid 0 flags: 0x50210400000000 nid: local remote: 0x0 expref: -99 pid: 6257 timeout: 0 lvb_type: 0 00010000:00010000:3.0:1776178204.176048:0:6257:0:(ldlm_lockd.c:1879:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: mdt-lustre-MDT0001_UUID lock: 00000000dcea299c/0xa40083ab9bf535db lrc: 3/0,0 mode: CR/CR res: [0x240000402:0x7f7:0x0].0x0 bits 0x2/0x0 rrc: 3 type: IBT gid 0 flags: 0x40210400000000 nid: local remote: 0x0 expref: -99 pid: 6257 timeout: 0 lvb_type: 0 00010000:00010000:3.0:1776178204.176054:0:6257:0:(ldlm_lockd.c:1893:ldlm_handle_bl_callback()) Lock 00000000dcea299c already unused, calling callback (0000000079e78348) 00010000:00010000:3.0:1776178204.176383:0:6257:0:(ldlm_request.c:364:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: mdt-lustre-MDT0001_UUID lock: 00000000dcea299c/0xa40083ab9bf535db lrc: 3/0,0 mode: CR/CR res: [0x240000402:0x7f7:0x0].0x0 bits 0x2/0x0 rrc: 3 type: IBT gid 0 flags: 0x40210400000000 nid: local remote: 0x0 expref: -99 pid: 6257 timeout: 0 lvb_type: 0 00010000:00010000:3.0:1776178204.176397:0:6257:0:(ldlm_request.c:1257:ldlm_cli_cancel_local()) ### server-side local cancel ns: mdt-lustre-MDT0001_UUID lock: 00000000dcea299c/0xa40083ab9bf535db lrc: 4/0,0 mode: CR/CR res: [0x240000402:0x7f7:0x0].0x0 bits 0x2/0x0 rrc: 3 type: IBT gid 0 flags: 0x40218400000000 nid: local remote: 0x0 expref: -99 pid: 6257 timeout: 0 lvb_type: 0 00010000:00010000:3.0:1776178204.176413:0:6257:0:(ldlm_inodebits.c:94:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x240000402:0x7f7:0x0].0x0 (0000000022e38283) 00010000:00010000:3.0:1776178204.176417:0:6257:0:(ldlm_inodebits.c:96:ldlm_reprocess_inodebits_queue()) Hint 2 00010000:00010000:3.0:1776178204.176424:0:6257:0:(ldlm_lockd.c:1903:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: mdt-lustre-MDT0001_UUID lock: 00000000dcea299c/0xa40083ab9bf535db lrc: 2/0,0 mode: --/CR res: [0x240000402:0x7f7:0x0].0x0 bits 0x2/0x0 rrc: 3 type: IBT gid 0 flags: 0x44a19400000000 nid: local remote: 0x0 expref: -99 pid: 6257 timeout: 0 lvb_type: 0 00010000:00010000:3.0:1776178204.176430:0:6257:0:(ldlm_lock.c:216:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: mdt-lustre-MDT0001_UUID lock: 00000000dcea299c/0xa40083ab9bf535db lrc: 0/0,0 mode: --/CR res: [0x240000402:0x7f7:0x0].0x0 bits 0x2/0x0 rrc: 3 type: IBT gid 0 flags: 0x44a19400000000 nid: local remote: 0x0 expref: -99 pid: 6257 timeout: 0 lvb_type: 0 00010000:00010000:3.0:1776178204.176448:0:6257:0:(ldlm_lock.c:820:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PR) ns: mdt-lustre-MDT0001_UUID lock: 00000000c1e7ea3e/0xa40083ab9bf535e2 lrc: 3/1,0 mode: PR/PR res: [0x240000402:0x7f7:0x0].0x5e5674ea bits 0x2/0x0 rrc: 2 type: IBT gid 0 flags: 0x50210000000000 nid: local remote: 0x0 expref: -99 pid: 6257 timeout: 0 lvb_type: 0 00010000:00010000:3.0:1776178204.176453:0:6257:0:(ldlm_lock.c:879:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: mdt-lustre-MDT0001_UUID lock: 00000000c1e7ea3e/0xa40083ab9bf535e2 lrc: 2/0,0 mode: PR/PR res: [0x240000402:0x7f7:0x0].0x5e5674ea bits 0x2/0x0 rrc: 2 type: IBT gid 0 flags: 0x50210400000000 nid: local remote: 0x0 expref: -99 pid: 6257 timeout: 0 lvb_type: 0 00010000:00010000:3.0:1776178204.176460:0:6257:0:(ldlm_lockd.c:1879:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: mdt-lustre-MDT0001_UUID lock: 00000000c1e7ea3e/0xa40083ab9bf535e2 lrc: 3/0,0 mode: PR/PR res: [0x240000402:0x7f7:0x0].0x5e5674ea bits 0x2/0x0 rrc: 2 type: IBT gid 0 flags: 0x40210400000000 nid: local remote: 0x0 expref: -99 pid: 6257 timeout: 0 lvb_type: 0 00010000:00010000:3.0:1776178204.176471:0:6257:0:(ldlm_lockd.c:1893:ldlm_handle_bl_callback()) Lock 00000000c1e7ea3e already unused, calling callback (0000000079e78348) 00010000:00010000:3.0:1776178204.176473:0:6257:0:(ldlm_request.c:364:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: mdt-lustre-MDT0001_UUID lock: 00000000c1e7ea3e/0xa40083ab9bf535e2 lrc: 3/0,0 mode: PR/PR res: [0x240000402:0x7f7:0x0].0x5e5674ea bits 0x2/0x0 rrc: 2 type: IBT gid 0 flags: 0x40210400000000 nid: local remote: 0x0 expref: -99 pid: 6257 timeout: 0 lvb_type: 0 00010000:00010000:3.0:1776178204.176480:0:6257:0:(ldlm_request.c:1257:ldlm_cli_cancel_local()) ### server-side local cancel ns: mdt-lustre-MDT0001_UUID lock: 00000000c1e7ea3e/0xa40083ab9bf535e2 lrc: 4/0,0 mode: PR/PR res: [0x240000402:0x7f7:0x0].0x5e5674ea bits 0x2/0x0 rrc: 2 type: IBT gid 0 flags: 0x40218400000000 nid: local remote: 0x0 expref: -99 pid: 6257 timeout: 0 lvb_type: 0 00010000:00010000:3.0:1776178204.176486:0:6257:0:(ldlm_inodebits.c:94:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x240000402:0x7f7:0x0].0x5e5674ea (000000009f2a4d4c) 00010000:00010000:3.0:1776178204.176488:0:6257:0:(ldlm_inodebits.c:96:ldlm_reprocess_inodebits_queue()) Hint 2 00010000:00010000:3.0:1776178204.176490:0:6257:0:(ldlm_lockd.c:1903:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: mdt-lustre-MDT0001_UUID lock: 00000000c1e7ea3e/0xa40083ab9bf535e2 lrc: 2/0,0 mode: --/PR res: [0x240000402:0x7f7:0x0].0x5e5674ea bits 0x2/0x0 rrc: 2 type: IBT gid 0 flags: 0x44a19400000000 nid: local remote: 0x0 expref: -99 pid: 6257 timeout: 0 lvb_type: 0 00010000:00010000:3.0:1776178204.176494:0:6257:0:(ldlm_lock.c:216:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: mdt-lustre-MDT0001_UUID lock: 00000000c1e7ea3e/0xa40083ab9bf535e2 lrc: 0/0,0 mode: --/PR res: [0x240000402:0x7f7:0x0].0x5e5674ea bits 0x2/0x0 rrc: 2 type: IBT gid 0 flags: 0x44a19400000000 nid: local remote: 0x0 expref: -99 pid: 6257 timeout: 0 lvb_type: 0 00010000:00010000:3.0:1776178204.176532:0:6257:0:(ldlm_lock.c:216:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: mdt-lustre-MDT0001_UUID lock: 00000000014a8a18/0xa40083ab9bf535d4 lrc: 0/0,0 mode: --/CR res: [0x240000402:0x7f7:0x0].0x0 bits 0x2/0x0 rrc: 2 type: IBT gid 0 flags: 0x44000000000000 nid: 192.168.204.3@tcp remote: 0x771f68aa1eb27b73 expref: 17 pid: 6257 timeout: 0 lvb_type: 0 00010000:00010000:3.0:1776178204.176547:0:6257:0:(ldlm_lockd.c:1484:ldlm_handle_enqueue0()) ### server-side enqueue handler, sending reply (err=0, rc=0) ns: mdt-lustre-MDT0001_UUID lock: 000000008cbcbaf5/0xa40083ab9bf535e9 lrc: 3/0,0 mode: PR/PR res: [0x240000402:0x7fa:0x0].0x0 bits 0x1b/0x0 rrc: 3 type: IBT gid 0 flags: 0x40200000000000 nid: 192.168.204.3@tcp remote: 0x771f68aa1eb27b73 expref: 16 pid: 6257 timeout: 0 lvb_type: 0 00010000:00010000:3.0:1776178204.176554:0:6257:0:(ldlm_inodebits.c:94:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x240000402:0x7fa:0x0].0x0 (0000000036d08056) 00010000:00010000:3.0:1776178204.176556:0:6257:0:(ldlm_inodebits.c:96:ldlm_reprocess_inodebits_queue()) Hint 1b 00010000:00010000:3.0:1776178204.176558:0:6257:0:(ldlm_lockd.c:1565:ldlm_handle_enqueue0()) ### server-side enqueue handler END (lock 000000008cbcbaf5, rc 0) 00000100:00100000:3.0:1776178204.176707:0:6257:0:(service.c:2372:ptlrpc_server_handle_request()) Handled RPC req@000000004547bc6e pname:cluuid+ref:pid:xid:nid:opc:job mdt00_002:b50963bd-8eba-4432-9a54-8888d00e3fa1+16:17666:x1862456979099968:12345-192.168.204.3@tcp:101:checkstat.0 Request processed in 1076us (1324us total) trans 0 rc 0/0 00000100:00100000:3.0:1776178204.176717:0:6257:0:(nrs_fifo.c:241:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.204.3@tcp, seq: 6575 00000100:00100000:2.0:1776178204.188931:0:3348:0:(events.c:361:request_in_callback()) peer: 12345-192.168.204.3@tcp (source: 12345-192.168.204.3@tcp) 00000100:00100000:3.0:1776178204.189134:0:6257:0:(service.c:2152:ptlrpc_server_handle_req_in()) got req x1862456979100032 00000100:00100000:3.0:1776178204.189198:0:6257:0:(nrs_fifo.c:179:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.204.3@tcp, seq: 6576 00000100:00100000:3.0:1776178204.189207:0:6257:0:(service.c:2324:ptlrpc_server_handle_request()) Handling RPC req@00000000ead84f6b pname:cluuid+ref:pid:xid:nid:opc:job mdt00_002:b50963bd-8eba-4432-9a54-8888d00e3fa1+16:17667:x1862456979100032:12345-192.168.204.3@tcp:101:checkstat.0 00010000:00010000:3.0:1776178204.189229:0:6257:0:(ldlm_lockd.c:1248:ldlm_handle_enqueue0()) ### server-side enqueue handler START 00010000:00010000:3.0:1776178204.189252:0:6257:0:(ldlm_lockd.c:1318:ldlm_handle_enqueue0()) ### server-side enqueue handler, new lock created ns: mdt-lustre-MDT0001_UUID lock: 00000000014a8a18/0xa40083ab9bf535f0 lrc: 2/0,0 mode: --/CR res: [0x240000402:0x7f7:0x0].0x0 bits 0x0/0x0 rrc: 2 type: IBT gid 0 flags: 0x40000000000000 nid: local remote: 0x771f68aa1eb27b7a expref: -99 pid: 6257 timeout: 0 lvb_type: 0 00010000:00010000:3.0:1776178204.189322:0:6257:0:(ldlm_lock.c:766:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(CR) ns: mdt-lustre-MDT0001_UUID lock: 00000000c1e7ea3e/0xa40083ab9bf535f7 lrc: 3/1,0 mode: --/CR res: [0x240000402:0x7f7:0x0].0x0 bits 0x0/0x0 rrc: 3 type: IBT gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 6257 timeout: 0 lvb_type: 0 00010000:00010000:3.0:1776178204.189337:0:6257:0:(ldlm_lock.c:1064:ldlm_granted_list_add_lock()) ### About to add lock: ns: mdt-lustre-MDT0001_UUID lock: 00000000c1e7ea3e/0xa40083ab9bf535f7 lrc: 3/1,0 mode: CR/CR res: [0x240000402:0x7f7:0x0].0x0 bits 0x2/0x0 rrc: 3 type: IBT gid 0 flags: 0x50210000000000 nid: local remote: 0x0 expref: -99 pid: 6257 timeout: 0 lvb_type: 0 00010000:00010000:3.0:1776178204.189349:0:6257:0:(ldlm_request.c:528:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: mdt-lustre-MDT0001_UUID lock: 00000000c1e7ea3e/0xa40083ab9bf535f7 lrc: 3/1,0 mode: CR/CR res: [0x240000402:0x7f7:0x0].0x0 bits 0x2/0x0 rrc: 3 type: IBT gid 0 flags: 0x40210000000000 nid: local remote: 0x0 expref: -99 pid: 6257 timeout: 0 lvb_type: 0 00010000:00010000:3.0:1776178204.189360:0:6257:0:(ldlm_lock.c:766:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PR) ns: mdt-lustre-MDT0001_UUID lock: 00000000dcea299c/0xa40083ab9bf535fe lrc: 3/1,0 mode: --/PR res: [0x240000402:0x7f7:0x0].0xa6b02dea bits 0x0/0x0 rrc: 2 type: IBT gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 6257 timeout: 0 lvb_type: 0 00010000:00010000:3.0:1776178204.189367:0:6257:0:(ldlm_lock.c:1064:ldlm_granted_list_add_lock()) ### About to add lock: ns: mdt-lustre-MDT0001_UUID lock: 00000000dcea299c/0xa40083ab9bf535fe lrc: 3/1,0 mode: PR/PR res: [0x240000402:0x7f7:0x0].0xa6b02dea bits 0x2/0x0 rrc: 2 type: IBT gid 0 flags: 0x50210000000000 nid: local remote: 0x0 expref: -99 pid: 6257 timeout: 0 lvb_type: 0 00010000:00010000:3.0:1776178204.189372:0:6257:0:(ldlm_request.c:528:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: mdt-lustre-MDT0001_UUID lock: 00000000dcea299c/0xa40083ab9bf535fe lrc: 3/1,0 mode: PR/PR res: [0x240000402:0x7f7:0x0].0xa6b02dea bits 0x2/0x0 rrc: 2 type: IBT gid 0 flags: 0x40210000000000 nid: local remote: 0x0 expref: -99 pid: 6257 timeout: 0 lvb_type: 0 00010000:00010000:3.0:1776178204.189837:0:6257:0:(ldlm_lock.c:766:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PR) ns: mdt-lustre-MDT0001_UUID lock: 00000000712476de/0xa40083ab9bf53605 lrc: 3/1,0 mode: --/PR res: [0x240000402:0x7fb:0x0].0x0 bits 0x0/0x0 rrc: 3 type: IBT gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 6257 timeout: 0 lvb_type: 0 00010000:00010000:3.0:1776178204.189858:0:6257:0:(ldlm_lock.c:1064:ldlm_granted_list_add_lock()) ### About to add lock: ns: mdt-lustre-MDT0001_UUID lock: 00000000712476de/0xa40083ab9bf53605 lrc: 3/1,0 mode: PR/PR res: [0x240000402:0x7fb:0x0].0x0 bits 0x5b/0x0 rrc: 3 type: IBT gid 0 flags: 0x50210000000000 nid: local remote: 0x0 expref: -99 pid: 6257 timeout: 0 lvb_type: 0 00010000:00010000:3.0:1776178204.189871:0:6257:0:(ldlm_request.c:528:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: mdt-lustre-MDT0001_UUID lock: 00000000712476de/0xa40083ab9bf53605 lrc: 3/1,0 mode: PR/PR res: [0x240000402:0x7fb:0x0].0x0 bits 0x5b/0x0 rrc: 3 type: IBT gid 0 flags: 0x40210000000000 nid: local remote: 0x0 expref: -99 pid: 6257 timeout: 0 lvb_type: 0 00000004:00200000:3.0:1776178204.189909:0:6257:0:(mdt_handler.c:912:mdt_pack_attr2body()) [0x240000402:0x7fb:0x0]: returning size 0 00000004:00200000:3.0:1776178204.189913:0:6257:0:(mdt_handler.c:1523:mdt_getattr_internal()) dirent count 0 stripe count 1 MDT count 2 00000004:00010000:3.0:1776178204.189933:0:6257:0:(mdt_handler.c:2353:mdt_getattr_name_lock()) ### Returning lock to client ns: mdt-lustre-MDT0001_UUID lock: 00000000712476de/0xa40083ab9bf53605 lrc: 3/1,0 mode: PR/PR res: [0x240000402:0x7fb:0x0].0x0 bits 0x5b/0x0 rrc: 3 type: IBT gid 0 flags: 0x40210000000000 nid: local remote: 0x0 expref: -99 pid: 6257 timeout: 0 lvb_type: 0 00010000:00010000:3.0:1776178204.189949:0:6257:0:(ldlm_lock.c:1064:ldlm_granted_list_add_lock()) ### About to add lock: ns: mdt-lustre-MDT0001_UUID lock: 00000000712476de/0xa40083ab9bf53605 lrc: 3/1,0 mode: PR/PR res: [0x240000402:0x7fb:0x0].0x0 bits 0x1b/0x0 rrc: 3 type: IBT gid 0 flags: 0x50210000000000 nid: local remote: 0x0 expref: -99 pid: 6257 timeout: 0 lvb_type: 0 00010000:00010000:3.0:1776178204.189957:0:6257:0:(ldlm_lock.c:820:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(CR) ns: mdt-lustre-MDT0001_UUID lock: 00000000c1e7ea3e/0xa40083ab9bf535f7 lrc: 3/1,0 mode: CR/CR res: [0x240000402:0x7f7:0x0].0x0 bits 0x2/0x0 rrc: 3 type: IBT gid 0 flags: 0x50210000000000 nid: local remote: 0x0 expref: -99 pid: 6257 timeout: 0 lvb_type: 0 00010000:00010000:3.0:1776178204.189963:0:6257:0:(ldlm_lock.c:879:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: mdt-lustre-MDT0001_UUID lock: 00000000c1e7ea3e/0xa40083ab9bf535f7 lrc: 2/0,0 mode: CR/CR res: [0x240000402:0x7f7:0x0].0x0 bits 0x2/0x0 rrc: 3 type: IBT gid 0 flags: 0x50210400000000 nid: local remote: 0x0 expref: -99 pid: 6257 timeout: 0 lvb_type: 0 00010000:00010000:3.0:1776178204.189970:0:6257:0:(ldlm_lockd.c:1879:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: mdt-lustre-MDT0001_UUID lock: 00000000c1e7ea3e/0xa40083ab9bf535f7 lrc: 3/0,0 mode: CR/CR res: [0x240000402:0x7f7:0x0].0x0 bits 0x2/0x0 rrc: 3 type: IBT gid 0 flags: 0x40210400000000 nid: local remote: 0x0 expref: -99 pid: 6257 timeout: 0 lvb_type: 0 00010000:00010000:3.0:1776178204.189975:0:6257:0:(ldlm_lockd.c:1893:ldlm_handle_bl_callback()) Lock 00000000c1e7ea3e already unused, calling callback (0000000079e78348) 00010000:00010000:3.0:1776178204.189979:0:6257:0:(ldlm_request.c:364:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: mdt-lustre-MDT0001_UUID lock: 00000000c1e7ea3e/0xa40083ab9bf535f7 lrc: 3/0,0 mode: CR/CR res: [0x240000402:0x7f7:0x0].0x0 bits 0x2/0x0 rrc: 3 type: IBT gid 0 flags: 0x40210400000000 nid: local remote: 0x0 expref: -99 pid: 6257 timeout: 0 lvb_type: 0 00010000:00010000:3.0:1776178204.189985:0:6257:0:(ldlm_request.c:1257:ldlm_cli_cancel_local()) ### server-side local cancel ns: mdt-lustre-MDT0001_UUID lock: 00000000c1e7ea3e/0xa40083ab9bf535f7 lrc: 4/0,0 mode: CR/CR res: [0x240000402:0x7f7:0x0].0x0 bits 0x2/0x0 rrc: 3 type: IBT gid 0 flags: 0x40218400000000 nid: local remote: 0x0 expref: -99 pid: 6257 timeout: 0 lvb_type: 0 00010000:00010000:3.0:1776178204.189996:0:6257:0:(ldlm_inodebits.c:94:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x240000402:0x7f7:0x0].0x0 (0000000022e38283) 00010000:00010000:3.0:1776178204.189999:0:6257:0:(ldlm_inodebits.c:96:ldlm_reprocess_inodebits_queue()) Hint 2 00010000:00010000:3.0:1776178204.190003:0:6257:0:(ldlm_lockd.c:1903:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: mdt-lustre-MDT0001_UUID lock: 00000000c1e7ea3e/0xa40083ab9bf535f7 lrc: 2/0,0 mode: --/CR res: [0x240000402:0x7f7:0x0].0x0 bits 0x2/0x0 rrc: 3 type: IBT gid 0 flags: 0x44a19400000000 nid: local remote: 0x0 expref: -99 pid: 6257 timeout: 0 lvb_type: 0 00010000:00010000:3.0:1776178204.190010:0:6257:0:(ldlm_lock.c:216:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: mdt-lustre-MDT0001_UUID lock: 00000000c1e7ea3e/0xa40083ab9bf535f7 lrc: 0/0,0 mode: --/CR res: [0x240000402:0x7f7:0x0].0x0 bits 0x2/0x0 rrc: 3 type: IBT gid 0 flags: 0x44a19400000000 nid: local remote: 0x0 expref: -99 pid: 6257 timeout: 0 lvb_type: 0 00010000:00010000:3.0:1776178204.190021:0:6257:0:(ldlm_lock.c:820:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PR) ns: mdt-lustre-MDT0001_UUID lock: 00000000dcea299c/0xa40083ab9bf535fe lrc: 3/1,0 mode: PR/PR res: [0x240000402:0x7f7:0x0].0xa6b02dea bits 0x2/0x0 rrc: 2 type: IBT gid 0 flags: 0x50210000000000 nid: local remote: 0x0 expref: -99 pid: 6257 timeout: 0 lvb_type: 0 00010000:00010000:3.0:1776178204.190028:0:6257:0:(ldlm_lock.c:879:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: mdt-lustre-MDT0001_UUID lock: 00000000dcea299c/0xa40083ab9bf535fe lrc: 2/0,0 mode: PR/PR res: [0x240000402:0x7f7:0x0].0xa6b02dea bits 0x2/0x0 rrc: 2 type: IBT gid 0 flags: 0x50210400000000 nid: local remote: 0x0 expref: -99 pid: 6257 timeout: 0 lvb_type: 0 00010000:00010000:3.0:1776178204.190032:0:6257:0:(ldlm_lockd.c:1879:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: mdt-lustre-MDT0001_UUID lock: 00000000dcea299c/0xa40083ab9bf535fe lrc: 3/0,0 mode: PR/PR res: [0x240000402:0x7f7:0x0].0xa6b02dea bits 0x2/0x0 rrc: 2 type: IBT gid 0 flags: 0x40210400000000 nid: local remote: 0x0 expref: -99 pid: 6257 timeout: 0 lvb_type: 0 00010000:00010000:3.0:1776178204.190038:0:6257:0:(ldlm_lockd.c:1893:ldlm_handle_bl_callback()) Lock 00000000dcea299c already unused, calling callback (0000000079e78348) 00010000:00010000:3.0:1776178204.190039:0:6257:0:(ldlm_request.c:364:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: mdt-lustre-MDT0001_UUID lock: 00000000dcea299c/0xa40083ab9bf535fe lrc: 3/0,0 mode: PR/PR res: [0x240000402:0x7f7:0x0].0xa6b02dea bits 0x2/0x0 rrc: 2 type: IBT gid 0 flags: 0x40210400000000 nid: local remote: 0x0 expref: -99 pid: 6257 timeout: 0 lvb_type: 0 00010000:00010000:3.0:1776178204.190045:0:6257:0:(ldlm_request.c:1257:ldlm_cli_cancel_local()) ### server-side local cancel ns: mdt-lustre-MDT0001_UUID lock: 00000000dcea299c/0xa40083ab9bf535fe lrc: 4/0,0 mode: PR/PR res: [0x240000402:0x7f7:0x0].0xa6b02dea bits 0x2/0x0 rrc: 2 type: IBT gid 0 flags: 0x40218400000000 nid: local remote: 0x0 expref: -99 pid: 6257 timeout: 0 lvb_type: 0 00010000:00010000:3.0:1776178204.190051:0:6257:0:(ldlm_inodebits.c:94:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x240000402:0x7f7:0x0].0xa6b02dea (000000009f2a4d4c) 00010000:00010000:3.0:1776178204.190054:0:6257:0:(ldlm_inodebits.c:96:ldlm_reprocess_inodebits_queue()) Hint 2 00010000:00010000:3.0:1776178204.190055:0:6257:0:(ldlm_lockd.c:1903:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: mdt-lustre-MDT0001_UUID lock: 00000000dcea299c/0xa40083ab9bf535fe lrc: 2/0,0 mode: --/PR res: [0x240000402:0x7f7:0x0].0xa6b02dea bits 0x2/0x0 rrc: 2 type: IBT gid 0 flags: 0x44a19400000000 nid: local remote: 0x0 expref: -99 pid: 6257 timeout: 0 lvb_type: 0 00010000:00010000:3.0:1776178204.190339:0:6257:0:(ldlm_lock.c:216:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: mdt-lustre-MDT0001_UUID lock: 00000000dcea299c/0xa40083ab9bf535fe lrc: 0/0,0 mode: --/PR res: [0x240000402:0x7f7:0x0].0xa6b02dea bits 0x2/0x0 rrc: 2 type: IBT gid 0 flags: 0x44a19400000000 nid: local remote: 0x0 expref: -99 pid: 6257 timeout: 0 lvb_type: 0 00010000:00010000:3.0:1776178204.190378:0:6257:0:(ldlm_lock.c:216:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: mdt-lustre-MDT0001_UUID lock: 00000000014a8a18/0xa40083ab9bf535f0 lrc: 0/0,0 mode: --/CR res: [0x240000402:0x7f7:0x0].0x0 bits 0x2/0x0 rrc: 2 type: IBT gid 0 flags: 0x44000000000000 nid: 192.168.204.3@tcp remote: 0x771f68aa1eb27b7a expref: 18 pid: 6257 timeout: 0 lvb_type: 0 00010000:00010000:3.0:1776178204.190393:0:6257:0:(ldlm_lockd.c:1484:ldlm_handle_enqueue0()) ### server-side enqueue handler, sending reply (err=0, rc=0) ns: mdt-lustre-MDT0001_UUID lock: 00000000712476de/0xa40083ab9bf53605 lrc: 3/0,0 mode: PR/PR res: [0x240000402:0x7fb:0x0].0x0 bits 0x1b/0x0 rrc: 3 type: IBT gid 0 flags: 0x40200000000000 nid: 192.168.204.3@tcp remote: 0x771f68aa1eb27b7a expref: 17 pid: 6257 timeout: 0 lvb_type: 0 00010000:00010000:3.0:1776178204.190399:0:6257:0:(ldlm_inodebits.c:94:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x240000402:0x7fb:0x0].0x0 (000000000885de38) 00010000:00010000:3.0:1776178204.190401:0:6257:0:(ldlm_inodebits.c:96:ldlm_reprocess_inodebits_queue()) Hint 1b 00010000:00010000:3.0:1776178204.190403:0:6257:0:(ldlm_lockd.c:1565:ldlm_handle_enqueue0()) ### server-side enqueue handler END (lock 00000000712476de, rc 0) 00000100:00100000:2.0:1776178204.190858:0:6257:0:(service.c:2372:ptlrpc_server_handle_request()) Handled RPC req@00000000ead84f6b pname:cluuid+ref:pid:xid:nid:opc:job mdt00_002:b50963bd-8eba-4432-9a54-8888d00e3fa1+17:17667:x1862456979100032:12345-192.168.204.3@tcp:101:checkstat.0 Request processed in 1649us (1935us total) trans 0 rc 0/0 00000100:00100000:2.0:1776178204.190869:0:6257:0:(nrs_fifo.c:241:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.204.3@tcp, seq: 6576 00000100:00100000:1.0:1776178204.202438:0:3347:0:(events.c:361:request_in_callback()) peer: 12345-192.168.204.3@tcp (source: 12345-192.168.204.3@tcp) 00000100:00100000:2.0:1776178204.202673:0:6257:0:(service.c:2152:ptlrpc_server_handle_req_in()) got req x1862456979100096 00000100:00100000:2.0:1776178204.202754:0:6257:0:(nrs_fifo.c:179:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.204.3@tcp, seq: 6577 00000100:00100000:2.0:1776178204.202761:0:6257:0:(service.c:2324:ptlrpc_server_handle_request()) Handling RPC req@00000000430a7ee3 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_002:b50963bd-8eba-4432-9a54-8888d00e3fa1+17:17668:x1862456979100096:12345-192.168.204.3@tcp:101:checkstat.0 00010000:00010000:2.0:1776178204.202786:0:6257:0:(ldlm_lockd.c:1248:ldlm_handle_enqueue0()) ### server-side enqueue handler START 00010000:00010000:2.0:1776178204.202813:0:6257:0:(ldlm_lockd.c:1318:ldlm_handle_enqueue0()) ### server-side enqueue handler, new lock created ns: mdt-lustre-MDT0001_UUID lock: 000000007ae7b5f6/0xa40083ab9bf5360c lrc: 2/0,0 mode: --/CR res: [0x240000402:0x7f7:0x0].0x0 bits 0x0/0x0 rrc: 2 type: IBT gid 0 flags: 0x40000000000000 nid: local remote: 0x771f68aa1eb27b81 expref: -99 pid: 6257 timeout: 0 lvb_type: 0 00010000:00010000:2.0:1776178204.202881:0:6257:0:(ldlm_lock.c:766:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(CR) ns: mdt-lustre-MDT0001_UUID lock: 000000001e6d7318/0xa40083ab9bf53613 lrc: 3/1,0 mode: --/CR res: [0x240000402:0x7f7:0x0].0x0 bits 0x0/0x0 rrc: 3 type: IBT gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 6257 timeout: 0 lvb_type: 0 00010000:00010000:2.0:1776178204.202895:0:6257:0:(ldlm_lock.c:1064:ldlm_granted_list_add_lock()) ### About to add lock: ns: mdt-lustre-MDT0001_UUID lock: 000000001e6d7318/0xa40083ab9bf53613 lrc: 3/1,0 mode: CR/CR res: [0x240000402:0x7f7:0x0].0x0 bits 0x2/0x0 rrc: 3 type: IBT gid 0 flags: 0x50210000000000 nid: local remote: 0x0 expref: -99 pid: 6257 timeout: 0 lvb_type: 0 00010000:00010000:2.0:1776178204.202905:0:6257:0:(ldlm_request.c:528:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: mdt-lustre-MDT0001_UUID lock: 000000001e6d7318/0xa40083ab9bf53613 lrc: 3/1,0 mode: CR/CR res: [0x240000402:0x7f7:0x0].0x0 bits 0x2/0x0 rrc: 3 type: IBT gid 0 flags: 0x40210000000000 nid: local remote: 0x0 expref: -99 pid: 6257 timeout: 0 lvb_type: 0 00010000:00010000:2.0:1776178204.202923:0:6257:0:(ldlm_lock.c:766:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PR) ns: mdt-lustre-MDT0001_UUID lock: 00000000f779de26/0xa40083ab9bf5361a lrc: 3/1,0 mode: --/PR res: [0x240000402:0x7f7:0x0].0xef09e6ea bits 0x0/0x0 rrc: 2 type: IBT gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 6257 timeout: 0 lvb_type: 0 00010000:00010000:2.0:1776178204.202930:0:6257:0:(ldlm_lock.c:1064:ldlm_granted_list_add_lock()) ### About to add lock: ns: mdt-lustre-MDT0001_UUID lock: 00000000f779de26/0xa40083ab9bf5361a lrc: 3/1,0 mode: PR/PR res: [0x240000402:0x7f7:0x0].0xef09e6ea bits 0x2/0x0 rrc: 2 type: IBT gid 0 flags: 0x50210000000000 nid: local remote: 0x0 expref: -99 pid: 6257 timeout: 0 lvb_type: 0 00010000:00010000:2.0:1776178204.202936:0:6257:0:(ldlm_request.c:528:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: mdt-lustre-MDT0001_UUID lock: 00000000f779de26/0xa40083ab9bf5361a lrc: 3/1,0 mode: PR/PR res: [0x240000402:0x7f7:0x0].0xef09e6ea bits 0x2/0x0 rrc: 2 type: IBT gid 0 flags: 0x40210000000000 nid: local remote: 0x0 expref: -99 pid: 6257 timeout: 0 lvb_type: 0 00010000:00010000:2.0:1776178204.203170:0:6257:0:(ldlm_lock.c:766:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PR) ns: mdt-lustre-MDT0001_UUID lock: 0000000025b13886/0xa40083ab9bf53621 lrc: 3/1,0 mode: --/PR res: [0x240000402:0x7fc:0x0].0x0 bits 0x0/0x0 rrc: 3 type: IBT gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 6257 timeout: 0 lvb_type: 0 00010000:00010000:2.0:1776178204.203179:0:6257:0:(ldlm_lock.c:1064:ldlm_granted_list_add_lock()) ### About to add lock: ns: mdt-lustre-MDT0001_UUID lock: 0000000025b13886/0xa40083ab9bf53621 lrc: 3/1,0 mode: PR/PR res: [0x240000402:0x7fc:0x0].0x0 bits 0x5b/0x0 rrc: 3 type: IBT gid 0 flags: 0x50210000000000 nid: local remote: 0x0 expref: -99 pid: 6257 timeout: 0 lvb_type: 0 00010000:00010000:2.0:1776178204.203186:0:6257:0:(ldlm_request.c:528:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: mdt-lustre-MDT0001_UUID lock: 0000000025b13886/0xa40083ab9bf53621 lrc: 3/1,0 mode: PR/PR res: [0x240000402:0x7fc:0x0].0x0 bits 0x5b/0x0 rrc: 3 type: IBT gid 0 flags: 0x40210000000000 nid: local remote: 0x0 expref: -99 pid: 6257 timeout: 0 lvb_type: 0 00000004:00200000:2.0:1776178204.203228:0:6257:0:(mdt_handler.c:912:mdt_pack_attr2body()) [0x240000402:0x7fc:0x0]: returning size 0 00000004:00200000:2.0:1776178204.203232:0:6257:0:(mdt_handler.c:1523:mdt_getattr_internal()) dirent count 0 stripe count 1 MDT count 2 00000004:00010000:2.0:1776178204.203253:0:6257:0:(mdt_handler.c:2353:mdt_getattr_name_lock()) ### Returning lock to client ns: mdt-lustre-MDT0001_UUID lock: 0000000025b13886/0xa40083ab9bf53621 lrc: 3/1,0 mode: PR/PR res: [0x240000402:0x7fc:0x0].0x0 bits 0x5b/0x0 rrc: 3 type: IBT gid 0 flags: 0x40210000000000 nid: local remote: 0x0 expref: -99 pid: 6257 timeout: 0 lvb_type: 0 00010000:00010000:2.0:1776178204.203266:0:6257:0:(ldlm_lock.c:1064:ldlm_granted_list_add_lock()) ### About to add lock: ns: mdt-lustre-MDT0001_UUID lock: 0000000025b13886/0xa40083ab9bf53621 lrc: 3/1,0 mode: PR/PR res: [0x240000402:0x7fc:0x0].0x0 bits 0x1b/0x0 rrc: 3 type: IBT gid 0 flags: 0x50210000000000 nid: local remote: 0x0 expref: -99 pid: 6257 timeout: 0 lvb_type: 0 00010000:00010000:2.0:1776178204.203274:0:6257:0:(ldlm_lock.c:820:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(CR) ns: mdt-lustre-MDT0001_UUID lock: 000000001e6d7318/0xa40083ab9bf53613 lrc: 3/1,0 mode: CR/CR res: [0x240000402:0x7f7:0x0].0x0 bits 0x2/0x0 rrc: 3 type: IBT gid 0 flags: 0x50210000000000 nid: local remote: 0x0 expref: -99 pid: 6257 timeout: 0 lvb_type: 0 00010000:00010000:2.0:1776178204.203280:0:6257:0:(ldlm_lock.c:879:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: mdt-lustre-MDT0001_UUID lock: 000000001e6d7318/0xa40083ab9bf53613 lrc: 2/0,0 mode: CR/CR res: [0x240000402:0x7f7:0x0].0x0 bits 0x2/0x0 rrc: 3 type: IBT gid 0 flags: 0x50210400000000 nid: local remote: 0x0 expref: -99 pid: 6257 timeout: 0 lvb_type: 0 00010000:00010000:2.0:1776178204.203287:0:6257:0:(ldlm_lockd.c:1879:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: mdt-lustre-MDT0001_UUID lock: 000000001e6d7318/0xa40083ab9bf53613 lrc: 3/0,0 mode: CR/CR res: [0x240000402:0x7f7:0x0].0x0 bits 0x2/0x0 rrc: 3 type: IBT gid 0 flags: 0x40210400000000 nid: local remote: 0x0 expref: -99 pid: 6257 timeout: 0 lvb_type: 0 00010000:00010000:2.0:1776178204.203293:0:6257:0:(ldlm_lockd.c:1893:ldlm_handle_bl_callback()) Lock 000000001e6d7318 already unused, calling callback (0000000079e78348) 00010000:00010000:2.0:1776178204.203296:0:6257:0:(ldlm_request.c:364:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: mdt-lustre-MDT0001_UUID lock: 000000001e6d7318/0xa40083ab9bf53613 lrc: 3/0,0 mode: CR/CR res: [0x240000402:0x7f7:0x0].0x0 bits 0x2/0x0 rrc: 3 type: IBT gid 0 flags: 0x40210400000000 nid: local remote: 0x0 expref: -99 pid: 6257 timeout: 0 lvb_type: 0 00010000:00010000:2.0:1776178204.203303:0:6257:0:(ldlm_request.c:1257:ldlm_cli_cancel_local()) ### server-side local cancel ns: mdt-lustre-MDT0001_UUID lock: 000000001e6d7318/0xa40083ab9bf53613 lrc: 4/0,0 mode: CR/CR res: [0x240000402:0x7f7:0x0].0x0 bits 0x2/0x0 rrc: 3 type: IBT gid 0 flags: 0x40218400000000 nid: local remote: 0x0 expref: -99 pid: 6257 timeout: 0 lvb_type: 0 00010000:00010000:2.0:1776178204.203314:0:6257:0:(ldlm_inodebits.c:94:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x240000402:0x7f7:0x0].0x0 (00000000e4751492) 00010000:00010000:2.0:1776178204.203317:0:6257:0:(ldlm_inodebits.c:96:ldlm_reprocess_inodebits_queue()) Hint 2 00010000:00010000:2.0:1776178204.203322:0:6257:0:(ldlm_lockd.c:1903:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: mdt-lustre-MDT0001_UUID lock: 000000001e6d7318/0xa40083ab9bf53613 lrc: 2/0,0 mode: --/CR res: [0x240000402:0x7f7:0x0].0x0 bits 0x2/0x0 rrc: 3 type: IBT gid 0 flags: 0x44a19400000000 nid: local remote: 0x0 expref: -99 pid: 6257 timeout: 0 lvb_type: 0 00010000:00010000:2.0:1776178204.203328:0:6257:0:(ldlm_lock.c:216:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: mdt-lustre-MDT0001_UUID lock: 000000001e6d7318/0xa40083ab9bf53613 lrc: 0/0,0 mode: --/CR res: [0x240000402:0x7f7:0x0].0x0 bits 0x2/0x0 rrc: 3 type: IBT gid 0 flags: 0x44a19400000000 nid: local remote: 0x0 expref: -99 pid: 6257 timeout: 0 lvb_type: 0 00010000:00010000:2.0:1776178204.203342:0:6257:0:(ldlm_lock.c:820:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PR) ns: mdt-lustre-MDT0001_UUID lock: 00000000f779de26/0xa40083ab9bf5361a lrc: 3/1,0 mode: PR/PR res: [0x240000402:0x7f7:0x0].0xef09e6ea bits 0x2/0x0 rrc: 2 type: IBT gid 0 flags: 0x50210000000000 nid: local remote: 0x0 expref: -99 pid: 6257 timeout: 0 lvb_type: 0 00010000:00010000:2.0:1776178204.203347:0:6257:0:(ldlm_lock.c:879:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: mdt-lustre-MDT0001_UUID lock: 00000000f779de26/0xa40083ab9bf5361a lrc: 2/0,0 mode: PR/PR res: [0x240000402:0x7f7:0x0].0xef09e6ea bits 0x2/0x0 rrc: 2 type: IBT gid 0 flags: 0x50210400000000 nid: local remote: 0x0 expref: -99 pid: 6257 timeout: 0 lvb_type: 0 00010000:00010000:2.0:1776178204.203352:0:6257:0:(ldlm_lockd.c:1879:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: mdt-lustre-MDT0001_UUID lock: 00000000f779de26/0xa40083ab9bf5361a lrc: 3/0,0 mode: PR/PR res: [0x240000402:0x7f7:0x0].0xef09e6ea bits 0x2/0x0 rrc: 2 type: IBT gid 0 flags: 0x40210400000000 nid: local remote: 0x0 expref: -99 pid: 6257 timeout: 0 lvb_type: 0 00010000:00010000:2.0:1776178204.203361:0:6257:0:(ldlm_lockd.c:1893:ldlm_handle_bl_callback()) Lock 00000000f779de26 already unused, calling callback (0000000079e78348) 00010000:00010000:2.0:1776178204.203362:0:6257:0:(ldlm_request.c:364:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: mdt-lustre-MDT0001_UUID lock: 00000000f779de26/0xa40083ab9bf5361a lrc: 3/0,0 mode: PR/PR res: [0x240000402:0x7f7:0x0].0xef09e6ea bits 0x2/0x0 rrc: 2 type: IBT gid 0 flags: 0x40210400000000 nid: local remote: 0x0 expref: -99 pid: 6257 timeout: 0 lvb_type: 0 00010000:00010000:2.0:1776178204.203368:0:6257:0:(ldlm_request.c:1257:ldlm_cli_cancel_local()) ### server-side local cancel ns: mdt-lustre-MDT0001_UUID lock: 00000000f779de26/0xa40083ab9bf5361a lrc: 4/0,0 mode: PR/PR res: [0x240000402:0x7f7:0x0].0xef09e6ea bits 0x2/0x0 rrc: 2 type: IBT gid 0 flags: 0x40218400000000 nid: local remote: 0x0 expref: -99 pid: 6257 timeout: 0 lvb_type: 0 00010000:00010000:2.0:1776178204.203374:0:6257:0:(ldlm_inodebits.c:94:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x240000402:0x7f7:0x0].0xef09e6ea (0000000043c2d1ef) 00010000:00010000:2.0:1776178204.203376:0:6257:0:(ldlm_inodebits.c:96:ldlm_reprocess_inodebits_queue()) Hint 2 00010000:00010000:2.0:1776178204.203377:0:6257:0:(ldlm_lockd.c:1903:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: mdt-lustre-MDT0001_UUID lock: 00000000f779de26/0xa40083ab9bf5361a lrc: 2/0,0 mode: --/PR res: [0x240000402:0x7f7:0x0].0xef09e6ea bits 0x2/0x0 rrc: 2 type: IBT gid 0 flags: 0x44a19400000000 nid: local remote: 0x0 expref: -99 pid: 6257 timeout: 0 lvb_type: 0 00010000:00010000:2.0:1776178204.203382:0:6257:0:(ldlm_lock.c:216:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: mdt-lustre-MDT0001_UUID lock: 00000000f779de26/0xa40083ab9bf5361a lrc: 0/0,0 mode: --/PR res: [0x240000402:0x7f7:0x0].0xef09e6ea bits 0x2/0x0 rrc: 2 type: IBT gid 0 flags: 0x44a19400000000 nid: local remote: 0x0 expref: -99 pid: 6257 timeout: 0 lvb_type: 0 00010000:00010000:2.0:1776178204.203413:0:6257:0:(ldlm_lock.c:216:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: mdt-lustre-MDT0001_UUID lock: 000000007ae7b5f6/0xa40083ab9bf5360c lrc: 0/0,0 mode: --/CR res: [0x240000402:0x7f7:0x0].0x0 bits 0x2/0x0 rrc: 2 type: IBT gid 0 flags: 0x44000000000000 nid: 192.168.204.3@tcp remote: 0x771f68aa1eb27b81 expref: 19 pid: 6257 timeout: 0 lvb_type: 0 00010000:00010000:2.0:1776178204.203428:0:6257:0:(ldlm_lockd.c:1484:ldlm_handle_enqueue0()) ### server-side enqueue handler, sending reply (err=0, rc=0) ns: mdt-lustre-MDT0001_UUID lock: 0000000025b13886/0xa40083ab9bf53621 lrc: 3/0,0 mode: PR/PR res: [0x240000402:0x7fc:0x0].0x0 bits 0x1b/0x0 rrc: 3 type: IBT gid 0 flags: 0x40200000000000 nid: 192.168.204.3@tcp remote: 0x771f68aa1eb27b81 expref: 18 pid: 6257 timeout: 0 lvb_type: 0 00010000:00010000:2.0:1776178204.203435:0:6257:0:(ldlm_inodebits.c:94:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x240000402:0x7fc:0x0].0x0 (00000000c31449d9) 00010000:00010000:2.0:1776178204.203437:0:6257:0:(ldlm_inodebits.c:96:ldlm_reprocess_inodebits_queue()) Hint 1b 00010000:00010000:2.0:1776178204.203439:0:6257:0:(ldlm_lockd.c:1565:ldlm_handle_enqueue0()) ### server-side enqueue handler END (lock 0000000025b13886, rc 0) 00000100:00100000:2.0:1776178204.203558:0:6257:0:(service.c:2372:ptlrpc_server_handle_request()) Handled RPC req@00000000430a7ee3 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_002:b50963bd-8eba-4432-9a54-8888d00e3fa1+18:17668:x1862456979100096:12345-192.168.204.3@tcp:101:checkstat.0 Request processed in 796us (1132us total) trans 0 rc 0/0 00000100:00100000:2.0:1776178204.203568:0:6257:0:(nrs_fifo.c:241:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.204.3@tcp, seq: 6577 00000100:00100000:1.0:1776178204.220430:0:3346:0:(events.c:361:request_in_callback()) peer: 12345-192.168.204.3@tcp (source: 12345-192.168.204.3@tcp) 00000100:00100000:2.0:1776178204.220640:0:6257:0:(service.c:2152:ptlrpc_server_handle_req_in()) got req x1862456979100160 00000100:00100000:2.0:1776178204.220711:0:6257:0:(nrs_fifo.c:179:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.204.3@tcp, seq: 6578 00000100:00100000:2.0:1776178204.220719:0:6257:0:(service.c:2324:ptlrpc_server_handle_request()) Handling RPC req@0000000057946847 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_002:b50963bd-8eba-4432-9a54-8888d00e3fa1+18:17669:x1862456979100160:12345-192.168.204.3@tcp:101:checkstat.0 00010000:00010000:2.0:1776178204.220741:0:6257:0:(ldlm_lockd.c:1248:ldlm_handle_enqueue0()) ### server-side enqueue handler START 00010000:00010000:2.0:1776178204.220769:0:6257:0:(ldlm_lockd.c:1318:ldlm_handle_enqueue0()) ### server-side enqueue handler, new lock created ns: mdt-lustre-MDT0001_UUID lock: 000000007ae7b5f6/0xa40083ab9bf53628 lrc: 2/0,0 mode: --/CR res: [0x240000402:0x7f7:0x0].0x0 bits 0x0/0x0 rrc: 2 type: IBT gid 0 flags: 0x40000000000000 nid: local remote: 0x771f68aa1eb27b88 expref: -99 pid: 6257 timeout: 0 lvb_type: 0 00010000:00010000:2.0:1776178204.220831:0:6257:0:(ldlm_lock.c:766:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(CR) ns: mdt-lustre-MDT0001_UUID lock: 00000000f779de26/0xa40083ab9bf5362f lrc: 3/1,0 mode: --/CR res: [0x240000402:0x7f7:0x0].0x0 bits 0x0/0x0 rrc: 3 type: IBT gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 6257 timeout: 0 lvb_type: 0 00010000:00010000:2.0:1776178204.220846:0:6257:0:(ldlm_lock.c:1064:ldlm_granted_list_add_lock()) ### About to add lock: ns: mdt-lustre-MDT0001_UUID lock: 00000000f779de26/0xa40083ab9bf5362f lrc: 3/1,0 mode: CR/CR res: [0x240000402:0x7f7:0x0].0x0 bits 0x2/0x0 rrc: 3 type: IBT gid 0 flags: 0x50210000000000 nid: local remote: 0x0 expref: -99 pid: 6257 timeout: 0 lvb_type: 0 00010000:00010000:2.0:1776178204.220858:0:6257:0:(ldlm_request.c:528:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: mdt-lustre-MDT0001_UUID lock: 00000000f779de26/0xa40083ab9bf5362f lrc: 3/1,0 mode: CR/CR res: [0x240000402:0x7f7:0x0].0x0 bits 0x2/0x0 rrc: 3 type: IBT gid 0 flags: 0x40210000000000 nid: local remote: 0x0 expref: -99 pid: 6257 timeout: 0 lvb_type: 0 00010000:00010000:2.0:1776178204.220871:0:6257:0:(ldlm_lock.c:766:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PR) ns: mdt-lustre-MDT0001_UUID lock: 000000001e6d7318/0xa40083ab9bf53636 lrc: 3/1,0 mode: --/PR res: [0x240000402:0x7f7:0x0].0x37639fea bits 0x0/0x0 rrc: 2 type: IBT gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 6257 timeout: 0 lvb_type: 0 00010000:00010000:2.0:1776178204.220877:0:6257:0:(ldlm_lock.c:1064:ldlm_granted_list_add_lock()) ### About to add lock: ns: mdt-lustre-MDT0001_UUID lock: 000000001e6d7318/0xa40083ab9bf53636 lrc: 3/1,0 mode: PR/PR res: [0x240000402:0x7f7:0x0].0x37639fea bits 0x2/0x0 rrc: 2 type: IBT gid 0 flags: 0x50210000000000 nid: local remote: 0x0 expref: -99 pid: 6257 timeout: 0 lvb_type: 0 00010000:00010000:2.0:1776178204.220884:0:6257:0:(ldlm_request.c:528:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: mdt-lustre-MDT0001_UUID lock: 000000001e6d7318/0xa40083ab9bf53636 lrc: 3/1,0 mode: PR/PR res: [0x240000402:0x7f7:0x0].0x37639fea bits 0x2/0x0 rrc: 2 type: IBT gid 0 flags: 0x40210000000000 nid: local remote: 0x0 expref: -99 pid: 6257 timeout: 0 lvb_type: 0 00010000:00010000:2.0:1776178204.220977:0:6257:0:(ldlm_lock.c:766:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PR) ns: mdt-lustre-MDT0001_UUID lock: 000000000b5c0143/0xa40083ab9bf5363d lrc: 3/1,0 mode: --/PR res: [0x240000402:0x7fd:0x0].0x0 bits 0x0/0x0 rrc: 3 type: IBT gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 6257 timeout: 0 lvb_type: 0 00010000:00010000:2.0:1776178204.220986:0:6257:0:(ldlm_lock.c:1064:ldlm_granted_list_add_lock()) ### About to add lock: ns: mdt-lustre-MDT0001_UUID lock: 000000000b5c0143/0xa40083ab9bf5363d lrc: 3/1,0 mode: PR/PR res: [0x240000402:0x7fd:0x0].0x0 bits 0x5b/0x0 rrc: 3 type: IBT gid 0 flags: 0x50210000000000 nid: local remote: 0x0 expref: -99 pid: 6257 timeout: 0 lvb_type: 0 00010000:00010000:2.0:1776178204.220993:0:6257:0:(ldlm_request.c:528:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: mdt-lustre-MDT0001_UUID lock: 000000000b5c0143/0xa40083ab9bf5363d lrc: 3/1,0 mode: PR/PR res: [0x240000402:0x7fd:0x0].0x0 bits 0x5b/0x0 rrc: 3 type: IBT gid 0 flags: 0x40210000000000 nid: local remote: 0x0 expref: -99 pid: 6257 timeout: 0 lvb_type: 0 00000004:00200000:2.0:1776178204.221043:0:6257:0:(mdt_handler.c:912:mdt_pack_attr2body()) [0x240000402:0x7fd:0x0]: returning size 0 00000004:00200000:2.0:1776178204.221048:0:6257:0:(mdt_handler.c:1523:mdt_getattr_internal()) dirent count 0 stripe count 1 MDT count 2 00000004:00010000:2.0:1776178204.221108:0:6257:0:(mdt_handler.c:2353:mdt_getattr_name_lock()) ### Returning lock to client ns: mdt-lustre-MDT0001_UUID lock: 000000000b5c0143/0xa40083ab9bf5363d lrc: 3/1,0 mode: PR/PR res: [0x240000402:0x7fd:0x0].0x0 bits 0x5b/0x0 rrc: 3 type: IBT gid 0 flags: 0x40210000000000 nid: local remote: 0x0 expref: -99 pid: 6257 timeout: 0 lvb_type: 0 00010000:00010000:2.0:1776178204.221123:0:6257:0:(ldlm_lock.c:1064:ldlm_granted_list_add_lock()) ### About to add lock: ns: mdt-lustre-MDT0001_UUID lock: 000000000b5c0143/0xa40083ab9bf5363d lrc: 3/1,0 mode: PR/PR res: [0x240000402:0x7fd:0x0].0x0 bits 0x1b/0x0 rrc: 3 type: IBT gid 0 flags: 0x50210000000000 nid: local remote: 0x0 expref: -99 pid: 6257 timeout: 0 lvb_type: 0 00010000:00010000:2.0:1776178204.221132:0:6257:0:(ldlm_lock.c:820:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(CR) ns: mdt-lustre-MDT0001_UUID lock: 00000000f779de26/0xa40083ab9bf5362f lrc: 3/1,0 mode: CR/CR res: [0x240000402:0x7f7:0x0].0x0 bits 0x2/0x0 rrc: 3 type: IBT gid 0 flags: 0x50210000000000 nid: local remote: 0x0 expref: -99 pid: 6257 timeout: 0 lvb_type: 0 00010000:00010000:2.0:1776178204.221137:0:6257:0:(ldlm_lock.c:879:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: mdt-lustre-MDT0001_UUID lock: 00000000f779de26/0xa40083ab9bf5362f lrc: 2/0,0 mode: CR/CR res: [0x240000402:0x7f7:0x0].0x0 bits 0x2/0x0 rrc: 3 type: IBT gid 0 flags: 0x50210400000000 nid: local remote: 0x0 expref: -99 pid: 6257 timeout: 0 lvb_type: 0 00010000:00010000:2.0:1776178204.221145:0:6257:0:(ldlm_lockd.c:1879:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: mdt-lustre-MDT0001_UUID lock: 00000000f779de26/0xa40083ab9bf5362f lrc: 3/0,0 mode: CR/CR res: [0x240000402:0x7f7:0x0].0x0 bits 0x2/0x0 rrc: 3 type: IBT gid 0 flags: 0x40210400000000 nid: local remote: 0x0 expref: -99 pid: 6257 timeout: 0 lvb_type: 0 00010000:00010000:2.0:1776178204.221151:0:6257:0:(ldlm_lockd.c:1893:ldlm_handle_bl_callback()) Lock 00000000f779de26 already unused, calling callback (0000000079e78348) 00010000:00010000:2.0:1776178204.221155:0:6257:0:(ldlm_request.c:364:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: mdt-lustre-MDT0001_UUID lock: 00000000f779de26/0xa40083ab9bf5362f lrc: 3/0,0 mode: CR/CR res: [0x240000402:0x7f7:0x0].0x0 bits 0x2/0x0 rrc: 3 type: IBT gid 0 flags: 0x40210400000000 nid: local remote: 0x0 expref: -99 pid: 6257 timeout: 0 lvb_type: 0 00010000:00010000:2.0:1776178204.221162:0:6257:0:(ldlm_request.c:1257:ldlm_cli_cancel_local()) ### server-side local cancel ns: mdt-lustre-MDT0001_UUID lock: 00000000f779de26/0xa40083ab9bf5362f lrc: 4/0,0 mode: CR/CR res: [0x240000402:0x7f7:0x0].0x0 bits 0x2/0x0 rrc: 3 type: IBT gid 0 flags: 0x40218400000000 nid: local remote: 0x0 expref: -99 pid: 6257 timeout: 0 lvb_type: 0 00010000:00010000:2.0:1776178204.221176:0:6257:0:(ldlm_inodebits.c:94:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x240000402:0x7f7:0x0].0x0 (00000000e4751492) 00010000:00010000:2.0:1776178204.221178:0:6257:0:(ldlm_inodebits.c:96:ldlm_reprocess_inodebits_queue()) Hint 2 00010000:00010000:2.0:1776178204.221184:0:6257:0:(ldlm_lockd.c:1903:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: mdt-lustre-MDT0001_UUID lock: 00000000f779de26/0xa40083ab9bf5362f lrc: 2/0,0 mode: --/CR res: [0x240000402:0x7f7:0x0].0x0 bits 0x2/0x0 rrc: 3 type: IBT gid 0 flags: 0x44a19400000000 nid: local remote: 0x0 expref: -99 pid: 6257 timeout: 0 lvb_type: 0 00010000:00010000:2.0:1776178204.221190:0:6257:0:(ldlm_lock.c:216:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: mdt-lustre-MDT0001_UUID lock: 00000000f779de26/0xa40083ab9bf5362f lrc: 0/0,0 mode: --/CR res: [0x240000402:0x7f7:0x0].0x0 bits 0x2/0x0 rrc: 3 type: IBT gid 0 flags: 0x44a19400000000 nid: local remote: 0x0 expref: -99 pid: 6257 timeout: 0 lvb_type: 0 00010000:00010000:2.0:1776178204.221206:0:6257:0:(ldlm_lock.c:820:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PR) ns: mdt-lustre-MDT0001_UUID lock: 000000001e6d7318/0xa40083ab9bf53636 lrc: 3/1,0 mode: PR/PR res: [0x240000402:0x7f7:0x0].0x37639fea bits 0x2/0x0 rrc: 2 type: IBT gid 0 flags: 0x50210000000000 nid: local remote: 0x0 expref: -99 pid: 6257 timeout: 0 lvb_type: 0 00010000:00010000:2.0:1776178204.221212:0:6257:0:(ldlm_lock.c:879:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: mdt-lustre-MDT0001_UUID lock: 000000001e6d7318/0xa40083ab9bf53636 lrc: 2/0,0 mode: PR/PR res: [0x240000402:0x7f7:0x0].0x37639fea bits 0x2/0x0 rrc: 2 type: IBT gid 0 flags: 0x50210400000000 nid: local remote: 0x0 expref: -99 pid: 6257 timeout: 0 lvb_type: 0 00010000:00010000:2.0:1776178204.221216:0:6257:0:(ldlm_lockd.c:1879:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: mdt-lustre-MDT0001_UUID lock: 000000001e6d7318/0xa40083ab9bf53636 lrc: 3/0,0 mode: PR/PR res: [0x240000402:0x7f7:0x0].0x37639fea bits 0x2/0x0 rrc: 2 type: IBT gid 0 flags: 0x40210400000000 nid: local remote: 0x0 expref: -99 pid: 6257 timeout: 0 lvb_type: 0 00010000:00010000:2.0:1776178204.221221:0:6257:0:(ldlm_lockd.c:1893:ldlm_handle_bl_callback()) Lock 000000001e6d7318 already unused, calling callback (0000000079e78348) 00010000:00010000:2.0:1776178204.221223:0:6257:0:(ldlm_request.c:364:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: mdt-lustre-MDT0001_UUID lock: 000000001e6d7318/0xa40083ab9bf53636 lrc: 3/0,0 mode: PR/PR res: [0x240000402:0x7f7:0x0].0x37639fea bits 0x2/0x0 rrc: 2 type: IBT gid 0 flags: 0x40210400000000 nid: local remote: 0x0 expref: -99 pid: 6257 timeout: 0 lvb_type: 0 00010000:00010000:2.0:1776178204.221228:0:6257:0:(ldlm_request.c:1257:ldlm_cli_cancel_local()) ### server-side local cancel ns: mdt-lustre-MDT0001_UUID lock: 000000001e6d7318/0xa40083ab9bf53636 lrc: 4/0,0 mode: PR/PR res: [0x240000402:0x7f7:0x0].0x37639fea bits 0x2/0x0 rrc: 2 type: IBT gid 0 flags: 0x40218400000000 nid: local remote: 0x0 expref: -99 pid: 6257 timeout: 0 lvb_type: 0 00010000:00010000:2.0:1776178204.221234:0:6257:0:(ldlm_inodebits.c:94:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x240000402:0x7f7:0x0].0x37639fea (0000000043c2d1ef) 00010000:00010000:2.0:1776178204.221236:0:6257:0:(ldlm_inodebits.c:96:ldlm_reprocess_inodebits_queue()) Hint 2 00010000:00010000:2.0:1776178204.221237:0:6257:0:(ldlm_lockd.c:1903:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: mdt-lustre-MDT0001_UUID lock: 000000001e6d7318/0xa40083ab9bf53636 lrc: 2/0,0 mode: --/PR res: [0x240000402:0x7f7:0x0].0x37639fea bits 0x2/0x0 rrc: 2 type: IBT gid 0 flags: 0x44a19400000000 nid: local remote: 0x0 expref: -99 pid: 6257 timeout: 0 lvb_type: 0 00010000:00010000:2.0:1776178204.221242:0:6257:0:(ldlm_lock.c:216:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: mdt-lustre-MDT0001_UUID lock: 000000001e6d7318/0xa40083ab9bf53636 lrc: 0/0,0 mode: --/PR res: [0x240000402:0x7f7:0x0].0x37639fea bits 0x2/0x0 rrc: 2 type: IBT gid 0 flags: 0x44a19400000000 nid: local remote: 0x0 expref: -99 pid: 6257 timeout: 0 lvb_type: 0 00010000:00010000:2.0:1776178204.221274:0:6257:0:(ldlm_lock.c:216:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: mdt-lustre-MDT0001_UUID lock: 000000007ae7b5f6/0xa40083ab9bf53628 lrc: 0/0,0 mode: --/CR res: [0x240000402:0x7f7:0x0].0x0 bits 0x2/0x0 rrc: 2 type: IBT gid 0 flags: 0x44000000000000 nid: 192.168.204.3@tcp remote: 0x771f68aa1eb27b88 expref: 20 pid: 6257 timeout: 0 lvb_type: 0 00010000:00010000:2.0:1776178204.221288:0:6257:0:(ldlm_lockd.c:1484:ldlm_handle_enqueue0()) ### server-side enqueue handler, sending reply (err=0, rc=0) ns: mdt-lustre-MDT0001_UUID lock: 000000000b5c0143/0xa40083ab9bf5363d lrc: 3/0,0 mode: PR/PR res: [0x240000402:0x7fd:0x0].0x0 bits 0x1b/0x0 rrc: 3 type: IBT gid 0 flags: 0x40200000000000 nid: 192.168.204.3@tcp remote: 0x771f68aa1eb27b88 expref: 19 pid: 6257 timeout: 0 lvb_type: 0 00010000:00010000:2.0:1776178204.221295:0:6257:0:(ldlm_inodebits.c:94:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x240000402:0x7fd:0x0].0x0 (0000000021e0af54) 00010000:00010000:2.0:1776178204.221297:0:6257:0:(ldlm_inodebits.c:96:ldlm_reprocess_inodebits_queue()) Hint 1b 00010000:00010000:2.0:1776178204.221299:0:6257:0:(ldlm_lockd.c:1565:ldlm_handle_enqueue0()) ### server-side enqueue handler END (lock 000000000b5c0143, rc 0) 00000100:00100000:2.0:1776178204.221381:0:6257:0:(service.c:2372:ptlrpc_server_handle_request()) Handled RPC req@0000000057946847 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_002:b50963bd-8eba-4432-9a54-8888d00e3fa1+19:17669:x1862456979100160:12345-192.168.204.3@tcp:101:checkstat.0 Request processed in 662us (958us total) trans 0 rc 0/0 00000100:00100000:2.0:1776178204.221390:0:6257:0:(nrs_fifo.c:241:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.204.3@tcp, seq: 6578 00000100:00100000:1.0:1776178204.235599:0:3348:0:(events.c:361:request_in_callback()) peer: 12345-192.168.204.3@tcp (source: 12345-192.168.204.3@tcp) 00000100:00100000:3.0:1776178204.235810:0:11281:0:(service.c:2152:ptlrpc_server_handle_req_in()) got req x1862456979100224 00000100:00100000:3.0:1776178204.235870:0:11281:0:(nrs_fifo.c:179:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.204.3@tcp, seq: 6579 00000100:00100000:3.0:1776178204.235877:0:11281:0:(service.c:2324:ptlrpc_server_handle_request()) Handling RPC req@000000000ff6c1d2 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_004:b50963bd-8eba-4432-9a54-8888d00e3fa1+19:17670:x1862456979100224:12345-192.168.204.3@tcp:101:checkstat.0 00010000:00010000:3.0:1776178204.235903:0:11281:0:(ldlm_lockd.c:1248:ldlm_handle_enqueue0()) ### server-side enqueue handler START 00010000:00010000:3.0:1776178204.235931:0:11281:0:(ldlm_lockd.c:1318:ldlm_handle_enqueue0()) ### server-side enqueue handler, new lock created ns: mdt-lustre-MDT0001_UUID lock: 00000000014a8a18/0xa40083ab9bf53644 lrc: 2/0,0 mode: --/CR res: [0x240000402:0x7f7:0x0].0x0 bits 0x0/0x0 rrc: 2 type: IBT gid 0 flags: 0x40000000000000 nid: local remote: 0x771f68aa1eb27b8f expref: -99 pid: 11281 timeout: 0 lvb_type: 0 00010000:00010000:3.0:1776178204.236009:0:11281:0:(ldlm_lock.c:766:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(CR) ns: mdt-lustre-MDT0001_UUID lock: 00000000dcea299c/0xa40083ab9bf5364b lrc: 3/1,0 mode: --/CR res: [0x240000402:0x7f7:0x0].0x0 bits 0x0/0x0 rrc: 3 type: IBT gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 11281 timeout: 0 lvb_type: 0 00010000:00010000:3.0:1776178204.236025:0:11281:0:(ldlm_lock.c:1064:ldlm_granted_list_add_lock()) ### About to add lock: ns: mdt-lustre-MDT0001_UUID lock: 00000000dcea299c/0xa40083ab9bf5364b lrc: 3/1,0 mode: CR/CR res: [0x240000402:0x7f7:0x0].0x0 bits 0x2/0x0 rrc: 3 type: IBT gid 0 flags: 0x50210000000000 nid: local remote: 0x0 expref: -99 pid: 11281 timeout: 0 lvb_type: 0 00010000:00010000:3.0:1776178204.236037:0:11281:0:(ldlm_request.c:528:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: mdt-lustre-MDT0001_UUID lock: 00000000dcea299c/0xa40083ab9bf5364b lrc: 3/1,0 mode: CR/CR res: [0x240000402:0x7f7:0x0].0x0 bits 0x2/0x0 rrc: 3 type: IBT gid 0 flags: 0x40210000000000 nid: local remote: 0x0 expref: -99 pid: 11281 timeout: 0 lvb_type: 0 00010000:00010000:3.0:1776178204.236049:0:11281:0:(ldlm_lock.c:766:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PR) ns: mdt-lustre-MDT0001_UUID lock: 00000000c1e7ea3e/0xa40083ab9bf53652 lrc: 3/1,0 mode: --/PR res: [0x240000402:0x7f7:0x0].0x7fbd58ea bits 0x0/0x0 rrc: 2 type: IBT gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 11281 timeout: 0 lvb_type: 0 00010000:00010000:3.0:1776178204.236055:0:11281:0:(ldlm_lock.c:1064:ldlm_granted_list_add_lock()) ### About to add lock: ns: mdt-lustre-MDT0001_UUID lock: 00000000c1e7ea3e/0xa40083ab9bf53652 lrc: 3/1,0 mode: PR/PR res: [0x240000402:0x7f7:0x0].0x7fbd58ea bits 0x2/0x0 rrc: 2 type: IBT gid 0 flags: 0x50210000000000 nid: local remote: 0x0 expref: -99 pid: 11281 timeout: 0 lvb_type: 0 00010000:00010000:3.0:1776178204.236115:0:11281:0:(ldlm_request.c:528:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: mdt-lustre-MDT0001_UUID lock: 00000000c1e7ea3e/0xa40083ab9bf53652 lrc: 3/1,0 mode: PR/PR res: [0x240000402:0x7f7:0x0].0x7fbd58ea bits 0x2/0x0 rrc: 2 type: IBT gid 0 flags: 0x40210000000000 nid: local remote: 0x0 expref: -99 pid: 11281 timeout: 0 lvb_type: 0 00010000:00010000:3.0:1776178204.236195:0:11281:0:(ldlm_lock.c:766:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PR) ns: mdt-lustre-MDT0001_UUID lock: 000000006cb7334a/0xa40083ab9bf53659 lrc: 3/1,0 mode: --/PR res: [0x240000402:0x7fe:0x0].0x0 bits 0x0/0x0 rrc: 3 type: IBT gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 11281 timeout: 0 lvb_type: 0 00010000:00010000:3.0:1776178204.236204:0:11281:0:(ldlm_lock.c:1064:ldlm_granted_list_add_lock()) ### About to add lock: ns: mdt-lustre-MDT0001_UUID lock: 000000006cb7334a/0xa40083ab9bf53659 lrc: 3/1,0 mode: PR/PR res: [0x240000402:0x7fe:0x0].0x0 bits 0x5b/0x0 rrc: 3 type: IBT gid 0 flags: 0x50210000000000 nid: local remote: 0x0 expref: -99 pid: 11281 timeout: 0 lvb_type: 0 00010000:00010000:3.0:1776178204.236210:0:11281:0:(ldlm_request.c:528:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: mdt-lustre-MDT0001_UUID lock: 000000006cb7334a/0xa40083ab9bf53659 lrc: 3/1,0 mode: PR/PR res: [0x240000402:0x7fe:0x0].0x0 bits 0x5b/0x0 rrc: 3 type: IBT gid 0 flags: 0x40210000000000 nid: local remote: 0x0 expref: -99 pid: 11281 timeout: 0 lvb_type: 0 00000004:00200000:3.0:1776178204.236245:0:11281:0:(mdt_handler.c:912:mdt_pack_attr2body()) [0x240000402:0x7fe:0x0]: returning size 0 00000004:00200000:3.0:1776178204.236249:0:11281:0:(mdt_handler.c:1523:mdt_getattr_internal()) dirent count 0 stripe count 1 MDT count 2 00000004:00010000:3.0:1776178204.236267:0:11281:0:(mdt_handler.c:2353:mdt_getattr_name_lock()) ### Returning lock to client ns: mdt-lustre-MDT0001_UUID lock: 000000006cb7334a/0xa40083ab9bf53659 lrc: 3/1,0 mode: PR/PR res: [0x240000402:0x7fe:0x0].0x0 bits 0x5b/0x0 rrc: 3 type: IBT gid 0 flags: 0x40210000000000 nid: local remote: 0x0 expref: -99 pid: 11281 timeout: 0 lvb_type: 0 00010000:00010000:3.0:1776178204.236279:0:11281:0:(ldlm_lock.c:1064:ldlm_granted_list_add_lock()) ### About to add lock: ns: mdt-lustre-MDT0001_UUID lock: 000000006cb7334a/0xa40083ab9bf53659 lrc: 3/1,0 mode: PR/PR res: [0x240000402:0x7fe:0x0].0x0 bits 0x1b/0x0 rrc: 3 type: IBT gid 0 flags: 0x50210000000000 nid: local remote: 0x0 expref: -99 pid: 11281 timeout: 0 lvb_type: 0 00010000:00010000:3.0:1776178204.236286:0:11281:0:(ldlm_lock.c:820:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(CR) ns: mdt-lustre-MDT0001_UUID lock: 00000000dcea299c/0xa40083ab9bf5364b lrc: 3/1,0 mode: CR/CR res: [0x240000402:0x7f7:0x0].0x0 bits 0x2/0x0 rrc: 3 type: IBT gid 0 flags: 0x50210000000000 nid: local remote: 0x0 expref: -99 pid: 11281 timeout: 0 lvb_type: 0 00010000:00010000:3.0:1776178204.236292:0:11281:0:(ldlm_lock.c:879:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: mdt-lustre-MDT0001_UUID lock: 00000000dcea299c/0xa40083ab9bf5364b lrc: 2/0,0 mode: CR/CR res: [0x240000402:0x7f7:0x0].0x0 bits 0x2/0x0 rrc: 3 type: IBT gid 0 flags: 0x50210400000000 nid: local remote: 0x0 expref: -99 pid: 11281 timeout: 0 lvb_type: 0 00010000:00010000:3.0:1776178204.236298:0:11281:0:(ldlm_lockd.c:1879:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: mdt-lustre-MDT0001_UUID lock: 00000000dcea299c/0xa40083ab9bf5364b lrc: 3/0,0 mode: CR/CR res: [0x240000402:0x7f7:0x0].0x0 bits 0x2/0x0 rrc: 3 type: IBT gid 0 flags: 0x40210400000000 nid: local remote: 0x0 expref: -99 pid: 11281 timeout: 0 lvb_type: 0 00010000:00010000:3.0:1776178204.236305:0:11281:0:(ldlm_lockd.c:1893:ldlm_handle_bl_callback()) Lock 00000000dcea299c already unused, calling callback (0000000079e78348) 00010000:00010000:3.0:1776178204.236308:0:11281:0:(ldlm_request.c:364:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: mdt-lustre-MDT0001_UUID lock: 00000000dcea299c/0xa40083ab9bf5364b lrc: 3/0,0 mode: CR/CR res: [0x240000402:0x7f7:0x0].0x0 bits 0x2/0x0 rrc: 3 type: IBT gid 0 flags: 0x40210400000000 nid: local remote: 0x0 expref: -99 pid: 11281 timeout: 0 lvb_type: 0 00010000:00010000:3.0:1776178204.236316:0:11281:0:(ldlm_request.c:1257:ldlm_cli_cancel_local()) ### server-side local cancel ns: mdt-lustre-MDT0001_UUID lock: 00000000dcea299c/0xa40083ab9bf5364b lrc: 4/0,0 mode: CR/CR res: [0x240000402:0x7f7:0x0].0x0 bits 0x2/0x0 rrc: 3 type: IBT gid 0 flags: 0x40218400000000 nid: local remote: 0x0 expref: -99 pid: 11281 timeout: 0 lvb_type: 0 00010000:00010000:3.0:1776178204.236327:0:11281:0:(ldlm_inodebits.c:94:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x240000402:0x7f7:0x0].0x0 (0000000022e38283) 00010000:00010000:3.0:1776178204.236330:0:11281:0:(ldlm_inodebits.c:96:ldlm_reprocess_inodebits_queue()) Hint 2 00010000:00010000:3.0:1776178204.236334:0:11281:0:(ldlm_lockd.c:1903:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: mdt-lustre-MDT0001_UUID lock: 00000000dcea299c/0xa40083ab9bf5364b lrc: 2/0,0 mode: --/CR res: [0x240000402:0x7f7:0x0].0x0 bits 0x2/0x0 rrc: 3 type: IBT gid 0 flags: 0x44a19400000000 nid: local remote: 0x0 expref: -99 pid: 11281 timeout: 0 lvb_type: 0 00010000:00010000:3.0:1776178204.236340:0:11281:0:(ldlm_lock.c:216:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: mdt-lustre-MDT0001_UUID lock: 00000000dcea299c/0xa40083ab9bf5364b lrc: 0/0,0 mode: --/CR res: [0x240000402:0x7f7:0x0].0x0 bits 0x2/0x0 rrc: 3 type: IBT gid 0 flags: 0x44a19400000000 nid: local remote: 0x0 expref: -99 pid: 11281 timeout: 0 lvb_type: 0 00010000:00010000:3.0:1776178204.236352:0:11281:0:(ldlm_lock.c:820:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PR) ns: mdt-lustre-MDT0001_UUID lock: 00000000c1e7ea3e/0xa40083ab9bf53652 lrc: 3/1,0 mode: PR/PR res: [0x240000402:0x7f7:0x0].0x7fbd58ea bits 0x2/0x0 rrc: 2 type: IBT gid 0 flags: 0x50210000000000 nid: local remote: 0x0 expref: -99 pid: 11281 timeout: 0 lvb_type: 0 00010000:00010000:3.0:1776178204.236357:0:11281:0:(ldlm_lock.c:879:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: mdt-lustre-MDT0001_UUID lock: 00000000c1e7ea3e/0xa40083ab9bf53652 lrc: 2/0,0 mode: PR/PR res: [0x240000402:0x7f7:0x0].0x7fbd58ea bits 0x2/0x0 rrc: 2 type: IBT gid 0 flags: 0x50210400000000 nid: local remote: 0x0 expref: -99 pid: 11281 timeout: 0 lvb_type: 0 00010000:00010000:3.0:1776178204.236362:0:11281:0:(ldlm_lockd.c:1879:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: mdt-lustre-MDT0001_UUID lock: 00000000c1e7ea3e/0xa40083ab9bf53652 lrc: 3/0,0 mode: PR/PR res: [0x240000402:0x7f7:0x0].0x7fbd58ea bits 0x2/0x0 rrc: 2 type: IBT gid 0 flags: 0x40210400000000 nid: local remote: 0x0 expref: -99 pid: 11281 timeout: 0 lvb_type: 0 00010000:00010000:3.0:1776178204.236366:0:11281:0:(ldlm_lockd.c:1893:ldlm_handle_bl_callback()) Lock 00000000c1e7ea3e already unused, calling callback (0000000079e78348) 00010000:00010000:3.0:1776178204.236368:0:11281:0:(ldlm_request.c:364:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: mdt-lustre-MDT0001_UUID lock: 00000000c1e7ea3e/0xa40083ab9bf53652 lrc: 3/0,0 mode: PR/PR res: [0x240000402:0x7f7:0x0].0x7fbd58ea bits 0x2/0x0 rrc: 2 type: IBT gid 0 flags: 0x40210400000000 nid: local remote: 0x0 expref: -99 pid: 11281 timeout: 0 lvb_type: 0 00010000:00010000:3.0:1776178204.236374:0:11281:0:(ldlm_request.c:1257:ldlm_cli_cancel_local()) ### server-side local cancel ns: mdt-lustre-MDT0001_UUID lock: 00000000c1e7ea3e/0xa40083ab9bf53652 lrc: 4/0,0 mode: PR/PR res: [0x240000402:0x7f7:0x0].0x7fbd58ea bits 0x2/0x0 rrc: 2 type: IBT gid 0 flags: 0x40218400000000 nid: local remote: 0x0 expref: -99 pid: 11281 timeout: 0 lvb_type: 0 00010000:00010000:3.0:1776178204.236380:0:11281:0:(ldlm_inodebits.c:94:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x240000402:0x7f7:0x0].0x7fbd58ea (000000009f2a4d4c) 00010000:00010000:3.0:1776178204.236383:0:11281:0:(ldlm_inodebits.c:96:ldlm_reprocess_inodebits_queue()) Hint 2 00010000:00010000:3.0:1776178204.236384:0:11281:0:(ldlm_lockd.c:1903:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: mdt-lustre-MDT0001_UUID lock: 00000000c1e7ea3e/0xa40083ab9bf53652 lrc: 2/0,0 mode: --/PR res: [0x240000402:0x7f7:0x0].0x7fbd58ea bits 0x2/0x0 rrc: 2 type: IBT gid 0 flags: 0x44a19400000000 nid: local remote: 0x0 expref: -99 pid: 11281 timeout: 0 lvb_type: 0 00010000:00010000:3.0:1776178204.236390:0:11281:0:(ldlm_lock.c:216:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: mdt-lustre-MDT0001_UUID lock: 00000000c1e7ea3e/0xa40083ab9bf53652 lrc: 0/0,0 mode: --/PR res: [0x240000402:0x7f7:0x0].0x7fbd58ea bits 0x2/0x0 rrc: 2 type: IBT gid 0 flags: 0x44a19400000000 nid: local remote: 0x0 expref: -99 pid: 11281 timeout: 0 lvb_type: 0 00010000:00010000:3.0:1776178204.236419:0:11281:0:(ldlm_lock.c:216:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: mdt-lustre-MDT0001_UUID lock: 00000000014a8a18/0xa40083ab9bf53644 lrc: 0/0,0 mode: --/CR res: [0x240000402:0x7f7:0x0].0x0 bits 0x2/0x0 rrc: 2 type: IBT gid 0 flags: 0x44000000000000 nid: 192.168.204.3@tcp remote: 0x771f68aa1eb27b8f expref: 21 pid: 11281 timeout: 0 lvb_type: 0 00010000:00010000:3.0:1776178204.236434:0:11281:0:(ldlm_lockd.c:1484:ldlm_handle_enqueue0()) ### server-side enqueue handler, sending reply (err=0, rc=0) ns: mdt-lustre-MDT0001_UUID lock: 000000006cb7334a/0xa40083ab9bf53659 lrc: 3/0,0 mode: PR/PR res: [0x240000402:0x7fe:0x0].0x0 bits 0x1b/0x0 rrc: 3 type: IBT gid 0 flags: 0x40200000000000 nid: 192.168.204.3@tcp remote: 0x771f68aa1eb27b8f expref: 20 pid: 11281 timeout: 0 lvb_type: 0 00010000:00010000:3.0:1776178204.236441:0:11281:0:(ldlm_inodebits.c:94:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x240000402:0x7fe:0x0].0x0 (0000000030fbaafd) 00010000:00010000:3.0:1776178204.236443:0:11281:0:(ldlm_inodebits.c:96:ldlm_reprocess_inodebits_queue()) Hint 1b 00010000:00010000:3.0:1776178204.236445:0:11281:0:(ldlm_lockd.c:1565:ldlm_handle_enqueue0()) ### server-side enqueue handler END (lock 000000006cb7334a, rc 0) 00000100:00100000:3.0:1776178204.236514:0:11281:0:(service.c:2372:ptlrpc_server_handle_request()) Handled RPC req@000000000ff6c1d2 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_004:b50963bd-8eba-4432-9a54-8888d00e3fa1+20:17670:x1862456979100224:12345-192.168.204.3@tcp:101:checkstat.0 Request processed in 636us (936us total) trans 0 rc 0/0 00000100:00100000:3.0:1776178204.236522:0:11281:0:(nrs_fifo.c:241:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.204.3@tcp, seq: 6579 00000100:00100000:1.0:1776178204.258623:0:3347:0:(events.c:361:request_in_callback()) peer: 12345-192.168.204.3@tcp (source: 12345-192.168.204.3@tcp) 00000100:00100000:2.0:1776178204.258809:0:6257:0:(service.c:2152:ptlrpc_server_handle_req_in()) got req x1862456979100288 00000100:00100000:2.0:1776178204.258868:0:6257:0:(nrs_fifo.c:179:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.204.3@tcp, seq: 6580 00000100:00100000:2.0:1776178204.258875:0:6257:0:(service.c:2324:ptlrpc_server_handle_request()) Handling RPC req@000000005f1d3350 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_002:b50963bd-8eba-4432-9a54-8888d00e3fa1+20:17671:x1862456979100288:12345-192.168.204.3@tcp:101:rm.0 00010000:00010000:2.0:1776178204.258897:0:6257:0:(ldlm_lockd.c:1248:ldlm_handle_enqueue0()) ### server-side enqueue handler START 00010000:00010000:2.0:1776178204.258923:0:6257:0:(ldlm_lockd.c:1318:ldlm_handle_enqueue0()) ### server-side enqueue handler, new lock created ns: mdt-lustre-MDT0001_UUID lock: 000000007ae7b5f6/0xa40083ab9bf53660 lrc: 2/0,0 mode: --/CR res: [0x200000007:0x1:0x0].0x0 bits 0x0/0x0 rrc: 2 type: IBT gid 0 flags: 0x40000000000000 nid: local remote: 0x771f68aa1eb27b96 expref: -99 pid: 6257 timeout: 0 lvb_type: 0 00010000:00010000:2.0:1776178204.259153:0:6257:0:(ldlm_lock.c:766:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PR) ns: mdt-lustre-MDT0001_UUID lock: 000000001e6d7318/0xa40083ab9bf53667 lrc: 3/1,0 mode: --/PR res: [0x240000402:0x7f7:0x0].0x0 bits 0x0/0x0 rrc: 2 type: IBT gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 6257 timeout: 0 lvb_type: 0 00010000:00010000:2.0:1776178204.259170:0:6257:0:(ldlm_lock.c:1064:ldlm_granted_list_add_lock()) ### About to add lock: ns: mdt-lustre-MDT0001_UUID lock: 000000001e6d7318/0xa40083ab9bf53667 lrc: 3/1,0 mode: PR/PR res: [0x240000402:0x7f7:0x0].0x0 bits 0x12/0x0 rrc: 2 type: IBT gid 0 flags: 0x50210000000000 nid: local remote: 0x0 expref: -99 pid: 6257 timeout: 0 lvb_type: 0 00010000:00010000:2.0:1776178204.259183:0:6257:0:(ldlm_request.c:528:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: mdt-lustre-MDT0001_UUID lock: 000000001e6d7318/0xa40083ab9bf53667 lrc: 3/1,0 mode: PR/PR res: [0x240000402:0x7f7:0x0].0x0 bits 0x12/0x0 rrc: 2 type: IBT gid 0 flags: 0x40210000000000 nid: local remote: 0x0 expref: -99 pid: 6257 timeout: 0 lvb_type: 0 00000004:00200000:2.0:1776178204.259233:0:6257:0:(mdt_handler.c:912:mdt_pack_attr2body()) [0x240000402:0x7f7:0x0]: returning size 4096 00000004:00200000:2.0:1776178204.259237:0:6257:0:(mdt_handler.c:1523:mdt_getattr_internal()) dirent count 9 stripe count 1 MDT count 2 00000004:00010000:2.0:1776178204.259255:0:6257:0:(mdt_handler.c:2353:mdt_getattr_name_lock()) ### Returning lock to client ns: mdt-lustre-MDT0001_UUID lock: 000000001e6d7318/0xa40083ab9bf53667 lrc: 3/1,0 mode: PR/PR res: [0x240000402:0x7f7:0x0].0x0 bits 0x12/0x0 rrc: 2 type: IBT gid 0 flags: 0x40210000000000 nid: local remote: 0x0 expref: -99 pid: 6257 timeout: 0 lvb_type: 0 00010000:00010000:2.0:1776178204.259283:0:6257:0:(ldlm_lock.c:216:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: mdt-lustre-MDT0001_UUID lock: 000000007ae7b5f6/0xa40083ab9bf53660 lrc: 0/0,0 mode: --/CR res: [0x200000007:0x1:0x0].0x0 bits 0x2/0x0 rrc: 2 type: IBT gid 0 flags: 0x44000000000000 nid: 192.168.204.3@tcp remote: 0x771f68aa1eb27b96 expref: 22 pid: 6257 timeout: 0 lvb_type: 0 00010000:00010000:2.0:1776178204.259513:0:6257:0:(ldlm_lockd.c:1484:ldlm_handle_enqueue0()) ### server-side enqueue handler, sending reply (err=0, rc=0) ns: mdt-lustre-MDT0001_UUID lock: 000000001e6d7318/0xa40083ab9bf53667 lrc: 3/0,0 mode: PR/PR res: [0x240000402:0x7f7:0x0].0x0 bits 0x12/0x0 rrc: 2 type: IBT gid 0 flags: 0x40200000000000 nid: 192.168.204.3@tcp remote: 0x771f68aa1eb27b96 expref: 21 pid: 6257 timeout: 0 lvb_type: 0 00010000:00010000:2.0:1776178204.259524:0:6257:0:(ldlm_inodebits.c:94:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x240000402:0x7f7:0x0].0x0 (0000000043c2d1ef) 00010000:00010000:2.0:1776178204.259527:0:6257:0:(ldlm_inodebits.c:96:ldlm_reprocess_inodebits_queue()) Hint 12 00010000:00010000:2.0:1776178204.259531:0:6257:0:(ldlm_lockd.c:1565:ldlm_handle_enqueue0()) ### server-side enqueue handler END (lock 000000001e6d7318, rc 0) 00000100:00100000:2.0:1776178204.259610:0:6257:0:(service.c:2372:ptlrpc_server_handle_request()) Handled RPC req@000000005f1d3350 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_002:b50963bd-8eba-4432-9a54-8888d00e3fa1+21:17671:x1862456979100288:12345-192.168.204.3@tcp:101:rm.0 Request processed in 735us (996us total) trans 0 rc 0/0 00000100:00100000:2.0:1776178204.259619:0:6257:0:(nrs_fifo.c:241:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.204.3@tcp, seq: 6580 00000100:00100000:1.0:1776178204.266037:0:3346:0:(events.c:361:request_in_callback()) peer: 12345-192.168.204.3@tcp (source: 12345-192.168.204.3@tcp) 00000100:00100000:2.0:1776178204.266122:0:6257:0:(service.c:2152:ptlrpc_server_handle_req_in()) got req x1862456979100416 00000100:00100000:2.0:1776178204.266194:0:6257:0:(nrs_fifo.c:179:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.204.3@tcp, seq: 6581 00000100:00100000:2.0:1776178204.266202:0:6257:0:(service.c:2324:ptlrpc_server_handle_request()) Handling RPC req@0000000008c9fb19 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_002:b50963bd-8eba-4432-9a54-8888d00e3fa1+21:17671:x1862456979100416:12345-192.168.204.3@tcp:101:rm.0 00010000:00010000:2.0:1776178204.266230:0:6257:0:(ldlm_lockd.c:1248:ldlm_handle_enqueue0()) ### server-side enqueue handler START 00010000:00010000:2.0:1776178204.266256:0:6257:0:(ldlm_lockd.c:1318:ldlm_handle_enqueue0()) ### server-side enqueue handler, new lock created ns: mdt-lustre-MDT0001_UUID lock: 000000007ae7b5f6/0xa40083ab9bf5366e lrc: 2/0,0 mode: --/CR res: [0x200000007:0x1:0x0].0x0 bits 0x0/0x0 rrc: 2 type: IBT gid 0 flags: 0x40000000000000 nid: local remote: 0x771f68aa1eb27b9d expref: -99 pid: 6257 timeout: 0 lvb_type: 0 00000004:00200000:2.0:1776178204.266425:0:6257:0:(mdt_handler.c:912:mdt_pack_attr2body()) [0x240000402:0x7f7:0x0]: returning size 4096 00010000:00010000:2.0:1776178204.266585:0:6257:0:(ldlm_lockd.c:1484:ldlm_handle_enqueue0()) ### server-side enqueue handler, sending reply (err=301, rc=0) ns: mdt-lustre-MDT0001_UUID lock: 000000007ae7b5f6/0xa40083ab9bf5366e lrc: 1/0,0 mode: --/CR res: [0x200000007:0x1:0x0].0x0 bits 0x1/0x0 rrc: 2 type: IBT gid 0 flags: 0x44000000000000 nid: 192.168.204.3@tcp remote: 0x771f68aa1eb27b9d expref: 23 pid: 6257 timeout: 0 lvb_type: 0 00010000:00010000:2.0:1776178204.266606:0:6257:0:(ldlm_lock.c:216:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: mdt-lustre-MDT0001_UUID lock: 000000007ae7b5f6/0xa40083ab9bf5366e lrc: 0/0,0 mode: --/CR res: [0x200000007:0x1:0x0].0x0 bits 0x1/0x0 rrc: 2 type: IBT gid 0 flags: 0x44000000000000 nid: 192.168.204.3@tcp remote: 0x771f68aa1eb27b9d expref: 23 pid: 6257 timeout: 0 lvb_type: 0 00010000:00010000:2.0:1776178204.266628:0:6257:0:(ldlm_lockd.c:1565:ldlm_handle_enqueue0()) ### server-side enqueue handler END (lock 000000007ae7b5f6, rc 0) 00000100:00100000:2.0:1776178204.266708:0:6257:0:(service.c:2372:ptlrpc_server_handle_request()) Handled RPC req@0000000008c9fb19 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_002:b50963bd-8eba-4432-9a54-8888d00e3fa1+22:17671:x1862456979100416:12345-192.168.204.3@tcp:101:rm.0 Request processed in 506us (685us total) trans 17179876264 rc 301/301 00000100:00100000:2.0:1776178204.266716:0:6257:0:(nrs_fifo.c:241:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.204.3@tcp, seq: 6581 00000100:00100000:3.0:1776178204.275890:0:3348:0:(events.c:361:request_in_callback()) peer: 12345-192.168.204.3@tcp (source: 12345-192.168.204.3@tcp) 00000100:00100000:2.0:1776178204.276894:0:6258:0:(service.c:2152:ptlrpc_server_handle_req_in()) got req x1862456979100480 00000100:00100000:2.0:1776178204.276984:0:6258:0:(nrs_fifo.c:179:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.204.3@tcp, seq: 2142 00000100:00100000:2.0:1776178204.277003:0:6258:0:(service.c:2324:ptlrpc_server_handle_request()) Handling RPC req@0000000028422d92 pname:cluuid+ref:pid:xid:nid:opc:job mdt_rdpg00_000:b50963bd-8eba-4432-9a54-8888d00e3fa1+22:17671:x1862456979100480:12345-192.168.204.3@tcp:37:rm.0 00000100:00100000:2.0:1776178204.294294:0:6258:0:(service.c:2372:ptlrpc_server_handle_request()) Handled RPC req@0000000028422d92 pname:cluuid+ref:pid:xid:nid:opc:job mdt_rdpg00_000:b50963bd-8eba-4432-9a54-8888d00e3fa1+22:17671:x1862456979100480:12345-192.168.204.3@tcp:37:rm.0 Request processed in 17290us (18406us total) trans 0 rc 0/0 00000100:00100000:2.0:1776178204.294320:0:6258:0:(nrs_fifo.c:241:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.204.3@tcp, seq: 2142 00000100:00100000:2.0:1776178204.301239:0:3346:0:(events.c:361:request_in_callback()) peer: 12345-192.168.204.3@tcp (source: 12345-192.168.204.3@tcp) 00000100:00100000:1.0:1776178204.301514:0:6258:0:(service.c:2152:ptlrpc_server_handle_req_in()) got req x1862456979100608 00000100:00100000:1.0:1776178204.301574:0:6258:0:(nrs_fifo.c:179:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.204.3@tcp, seq: 2143 00000100:00100000:1.0:1776178204.301582:0:6258:0:(service.c:2324:ptlrpc_server_handle_request()) Handling RPC req@00000000f62fe06f pname:cluuid+ref:pid:xid:nid:opc:job mdt_rdpg00_000:b50963bd-8eba-4432-9a54-8888d00e3fa1+22:17671:x1862456979100608:12345-192.168.204.3@tcp:35:rm.0 00000100:00100000:1.0:1776178204.301849:0:6258:0:(service.c:2372:ptlrpc_server_handle_request()) Handled RPC req@00000000f62fe06f pname:cluuid+ref:pid:xid:nid:opc:job mdt_rdpg00_000:b50963bd-8eba-4432-9a54-8888d00e3fa1+23:17671:x1862456979100608:12345-192.168.204.3@tcp:35:rm.0 Request processed in 266us (621us total) trans 17179876265 rc 0/0 00000100:00100000:1.0:1776178204.301856:0:6258:0:(nrs_fifo.c:241:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.204.3@tcp, seq: 2143 00000100:00100000:3.0:1776178204.307190:0:3348:0:(events.c:361:request_in_callback()) peer: 12345-192.168.204.3@tcp (source: 12345-192.168.204.3@tcp) 00000100:00100000:2.0:1776178204.307487:0:11281:0:(service.c:2152:ptlrpc_server_handle_req_in()) got req x1862456979100736 00000100:00100000:2.0:1776178204.307543:0:11281:0:(nrs_fifo.c:179:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.204.3@tcp, seq: 6582 00000100:00100000:2.0:1776178204.307551:0:11281:0:(service.c:2324:ptlrpc_server_handle_request()) Handling RPC req@00000000c324c57b pname:cluuid+ref:pid:xid:nid:opc:job mdt00_004:b50963bd-8eba-4432-9a54-8888d00e3fa1+23:17671:x1862456979100736:12345-192.168.204.3@tcp:101:rm.0 00010000:00010000:2.0:1776178204.307576:0:11281:0:(ldlm_lockd.c:1248:ldlm_handle_enqueue0()) ### server-side enqueue handler START 00010000:00010000:2.0:1776178204.307610:0:11281:0:(ldlm_lockd.c:1318:ldlm_handle_enqueue0()) ### server-side enqueue handler, new lock created ns: mdt-lustre-MDT0001_UUID lock: 000000007ae7b5f6/0xa40083ab9bf5367c lrc: 2/0,0 mode: --/CR res: [0x200000007:0x1:0x0].0x0 bits 0x0/0x0 rrc: 2 type: IBT gid 0 flags: 0x40000000000000 nid: local remote: 0x771f68aa1eb27ba4 expref: -99 pid: 11281 timeout: 0 lvb_type: 0 00010000:00010000:2.0:1776178204.307721:0:11281:0:(ldlm_lock.c:766:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(CR) ns: mdt-lustre-MDT0001_UUID lock: 00000000f779de26/0xa40083ab9bf53683 lrc: 3/1,0 mode: --/CR res: [0x240000402:0x7f7:0x0].0x0 bits 0x0/0x0 rrc: 3 type: IBT gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 11281 timeout: 0 lvb_type: 0 00010000:00010000:2.0:1776178204.307736:0:11281:0:(ldlm_lock.c:1064:ldlm_granted_list_add_lock()) ### About to add lock: ns: mdt-lustre-MDT0001_UUID lock: 00000000f779de26/0xa40083ab9bf53683 lrc: 3/1,0 mode: CR/CR res: [0x240000402:0x7f7:0x0].0x0 bits 0x5/0x0 rrc: 3 type: IBT gid 0 flags: 0x50210000000000 nid: local remote: 0x0 expref: -99 pid: 11281 timeout: 0 lvb_type: 0 00010000:00010000:2.0:1776178204.307748:0:11281:0:(ldlm_request.c:528:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: mdt-lustre-MDT0001_UUID lock: 00000000f779de26/0xa40083ab9bf53683 lrc: 3/1,0 mode: CR/CR res: [0x240000402:0x7f7:0x0].0x0 bits 0x5/0x0 rrc: 3 type: IBT gid 0 flags: 0x40210000000000 nid: local remote: 0x0 expref: -99 pid: 11281 timeout: 0 lvb_type: 0 00000004:00200000:2.0:1776178204.307761:0:11281:0:(mdt_handler.c:912:mdt_pack_attr2body()) [0x240000402:0x7f7:0x0]: returning size 4096 00010000:00010000:2.0:1776178204.307861:0:11281:0:(ldlm_lock.c:216:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: mdt-lustre-MDT0001_UUID lock: 000000007ae7b5f6/0xa40083ab9bf5367c lrc: 0/0,0 mode: --/CR res: [0x200000007:0x1:0x0].0x0 bits 0x1/0x0 rrc: 2 type: IBT gid 0 flags: 0x44000000000000 nid: 192.168.204.3@tcp remote: 0x771f68aa1eb27ba4 expref: 26 pid: 11281 timeout: 0 lvb_type: 0 00010000:00010000:2.0:1776178204.307886:0:11281:0:(ldlm_lockd.c:1484:ldlm_handle_enqueue0()) ### server-side enqueue handler, sending reply (err=0, rc=0) ns: mdt-lustre-MDT0001_UUID lock: 00000000f779de26/0xa40083ab9bf53683 lrc: 3/0,0 mode: CR/CR res: [0x240000402:0x7f7:0x0].0x0 bits 0x5/0x0 rrc: 3 type: IBT gid 0 flags: 0x40200000000000 nid: 192.168.204.3@tcp remote: 0x771f68aa1eb27ba4 expref: 25 pid: 11281 timeout: 0 lvb_type: 0 00010000:00010000:2.0:1776178204.307901:0:11281:0:(ldlm_inodebits.c:94:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x240000402:0x7f7:0x0].0x0 (0000000043c2d1ef) 00010000:00010000:2.0:1776178204.307904:0:11281:0:(ldlm_inodebits.c:96:ldlm_reprocess_inodebits_queue()) Hint 5 00010000:00010000:2.0:1776178204.307908:0:11281:0:(ldlm_lockd.c:1565:ldlm_handle_enqueue0()) ### server-side enqueue handler END (lock 00000000f779de26, rc 0) 00000100:00100000:2.0:1776178204.307987:0:11281:0:(service.c:2372:ptlrpc_server_handle_request()) Handled RPC req@00000000c324c57b pname:cluuid+ref:pid:xid:nid:opc:job mdt00_004:b50963bd-8eba-4432-9a54-8888d00e3fa1+25:17671:x1862456979100736:12345-192.168.204.3@tcp:101:rm.0 Request processed in 436us (804us total) trans 17179876266 rc 0/0 00000100:00100000:2.0:1776178204.308003:0:11281:0:(nrs_fifo.c:241:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.204.3@tcp, seq: 6582 00000100:00100000:2.0:1776178204.315284:0:3347:0:(events.c:361:request_in_callback()) peer: 12345-192.168.204.3@tcp (source: 12345-192.168.204.3@tcp) 00000100:00100000:1.0:1776178204.315484:0:6257:0:(service.c:2152:ptlrpc_server_handle_req_in()) got req x1862456979100800 00000100:00100000:1.0:1776178204.315547:0:6257:0:(nrs_fifo.c:179:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.204.3@tcp, seq: 6583 00000100:00100000:1.0:1776178204.315554:0:6257:0:(service.c:2324:ptlrpc_server_handle_request()) Handling RPC req@00000000226852f3 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_002:b50963bd-8eba-4432-9a54-8888d00e3fa1+7:17671:x1862456979100800:12345-192.168.204.3@tcp:41:rm.0 00000004:00000024:1.0:1776178204.315626:0:6257:0:(mdt_handler.c:548:mdt_statfs()) blocks cached 0 granted 2097152 pending 0 free 7847378944 avail 7383789568 00000100:00100000:1.0:1776178204.315713:0:6257:0:(service.c:2372:ptlrpc_server_handle_request()) Handled RPC req@00000000226852f3 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_002:b50963bd-8eba-4432-9a54-8888d00e3fa1+7:17671:x1862456979100800:12345-192.168.204.3@tcp:41:rm.0 Request processed in 157us (437us total) trans 0 rc 0/0 00000100:00100000:1.0:1776178204.315720:0:6257:0:(nrs_fifo.c:241:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.204.3@tcp, seq: 6583 00000100:00100000:3.0:1776178204.319929:0:3346:0:(events.c:361:request_in_callback()) peer: 12345-192.168.204.3@tcp (source: 12345-192.168.204.3@tcp) 00000100:00100000:2.0:1776178204.322260:0:11281:0:(service.c:2152:ptlrpc_server_handle_req_in()) got req x1862456979100928 00000100:00100000:2.0:1776178204.322359:0:11281:0:(nrs_fifo.c:179:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.204.3@tcp, seq: 6584 00000100:00100000:2.0:1776178204.322368:0:11281:0:(service.c:2324:ptlrpc_server_handle_request()) Handling RPC req@000000008c3fbdb6 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_004:b50963bd-8eba-4432-9a54-8888d00e3fa1+25:17671:x1862456979100928:12345-192.168.204.3@tcp:36:rm.0 00010000:00010000:2.0:1776178204.322436:0:11281:0:(ldlm_lockd.c:1720:ldlm_request_cancel()) ### server-side cancel handler START: 3 locks, starting at 0 00010000:00010000:2.0:1776178204.322474:0:11281:0:(ldlm_lock.c:216:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: mdt-lustre-MDT0001_UUID lock: 000000000b5c0143/0xa40083ab9bf5363d lrc: 0/0,0 mode: --/PR res: [0x240000402:0x7fd:0x0].0x0 bits 0x1b/0x0 rrc: 4 type: IBT gid 0 flags: 0x44a01000000000 nid: 192.168.204.3@tcp remote: 0x771f68aa1eb27b88 expref: 25 pid: 6257 timeout: 0 lvb_type: 0 00010000:00010000:2.0:1776178204.322503:0:11281:0:(ldlm_lock.c:216:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: mdt-lustre-MDT0001_UUID lock: 000000000b4a063d/0xa40083ab9bf5356b lrc: 0/0,0 mode: --/CR res: [0x240000402:0x7fd:0x0].0x0 bits 0x8/0x0 rrc: 3 type: IBT gid 0 flags: 0x44801000000000 nid: 192.168.204.3@tcp remote: 0x771f68aa1eb27b34 expref: 24 pid: 11281 timeout: 0 lvb_type: 0 00010000:00010000:2.0:1776178204.322517:0:11281:0:(ldlm_inodebits.c:94:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x240000402:0x7fd:0x0].0x0 (0000000021e0af54) 00010000:00010000:2.0:1776178204.322535:0:11281:0:(ldlm_lock.c:216:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: mdt-lustre-MDT0001_UUID lock: 000000001e6d7318/0xa40083ab9bf53667 lrc: 0/0,0 mode: --/PR res: [0x240000402:0x7f7:0x0].0x0 bits 0x12/0x0 rrc: 4 type: IBT gid 0 flags: 0x44a01000000000 nid: 192.168.204.3@tcp remote: 0x771f68aa1eb27b96 expref: 23 pid: 6257 timeout: 0 lvb_type: 0 00010000:00010000:2.0:1776178204.322543:0:11281:0:(ldlm_inodebits.c:94:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x240000402:0x7f7:0x0].0x0 (0000000043c2d1ef) 00010000:00010000:2.0:1776178204.322546:0:11281:0:(ldlm_lockd.c:1782:ldlm_request_cancel()) ### server-side cancel handler END 00010000:00010000:2.0:1776178204.322578:0:11281:0:(ldlm_lock.c:766:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(CW) ns: mdt-lustre-MDT0001_UUID lock: 000000007ae7b5f6/0xa40083ab9bf53691 lrc: 3/0,1 mode: --/CW res: [0x240000402:0x7f7:0x0].0x0 bits 0x0/0x0 rrc: 3 type: IBT gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 11281 timeout: 0 lvb_type: 0 00010000:00010000:2.0:1776178204.322811:0:11281:0:(ldlm_lock.c:1064:ldlm_granted_list_add_lock()) ### About to add lock: ns: mdt-lustre-MDT0001_UUID lock: 000000007ae7b5f6/0xa40083ab9bf53691 lrc: 3/0,1 mode: CW/CW res: [0x240000402:0x7f7:0x0].0x0 bits 0x2/0x0 rrc: 3 type: IBT gid 0 flags: 0x50210000000000 nid: local remote: 0x0 expref: -99 pid: 11281 timeout: 0 lvb_type: 0 00010000:00010000:2.0:1776178204.322823:0:11281:0:(ldlm_request.c:528:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: mdt-lustre-MDT0001_UUID lock: 000000007ae7b5f6/0xa40083ab9bf53691 lrc: 3/0,1 mode: CW/CW res: [0x240000402:0x7f7:0x0].0x0 bits 0x2/0x0 rrc: 3 type: IBT gid 0 flags: 0x40210000000000 nid: local remote: 0x0 expref: -99 pid: 11281 timeout: 0 lvb_type: 0 00010000:00010000:2.0:1776178204.322840:0:11281:0:(ldlm_lock.c:766:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: mdt-lustre-MDT0001_UUID lock: 000000006cc9c357/0xa40083ab9bf53698 lrc: 3/0,1 mode: --/PW res: [0x240000402:0x7f7:0x0].0x37639fea bits 0x0/0x0 rrc: 2 type: IBT gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 11281 timeout: 0 lvb_type: 0 00010000:00010000:2.0:1776178204.322846:0:11281:0:(ldlm_lock.c:1064:ldlm_granted_list_add_lock()) ### About to add lock: ns: mdt-lustre-MDT0001_UUID lock: 000000006cc9c357/0xa40083ab9bf53698 lrc: 3/0,1 mode: PW/PW res: [0x240000402:0x7f7:0x0].0x37639fea bits 0x2/0x0 rrc: 2 type: IBT gid 0 flags: 0x50210000000000 nid: local remote: 0x0 expref: -99 pid: 11281 timeout: 0 lvb_type: 0 00010000:00010000:2.0:1776178204.322852:0:11281:0:(ldlm_request.c:528:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: mdt-lustre-MDT0001_UUID lock: 000000006cc9c357/0xa40083ab9bf53698 lrc: 3/0,1 mode: PW/PW res: [0x240000402:0x7f7:0x0].0x37639fea bits 0x2/0x0 rrc: 2 type: IBT gid 0 flags: 0x40210000000000 nid: local remote: 0x0 expref: -99 pid: 11281 timeout: 0 lvb_type: 0 00010000:00010000:2.0:1776178204.322928:0:11281:0:(ldlm_lock.c:766:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(EX) ns: mdt-lustre-MDT0001_UUID lock: 000000000396420a/0xa40083ab9bf5369f lrc: 3/0,1 mode: --/EX res: [0x240000402:0x7fd:0x0].0x0 bits 0x0/0x0 rrc: 2 type: IBT gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 11281 timeout: 0 lvb_type: 0 00010000:00010000:2.0:1776178204.322934:0:11281:0:(ldlm_lock.c:1064:ldlm_granted_list_add_lock()) ### About to add lock: ns: mdt-lustre-MDT0001_UUID lock: 000000000396420a/0xa40083ab9bf5369f lrc: 3/0,1 mode: EX/EX res: [0x240000402:0x7fd:0x0].0x0 bits 0x3/0x0 rrc: 2 type: IBT gid 0 flags: 0x50210000000000 nid: local remote: 0x0 expref: -99 pid: 11281 timeout: 0 lvb_type: 0 00010000:00010000:2.0:1776178204.322940:0:11281:0:(ldlm_request.c:528:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: mdt-lustre-MDT0001_UUID lock: 000000000396420a/0xa40083ab9bf5369f lrc: 3/0,1 mode: EX/EX res: [0x240000402:0x7fd:0x0].0x0 bits 0x3/0x0 rrc: 2 type: IBT gid 0 flags: 0x40210000000000 nid: local remote: 0x0 expref: -99 pid: 11281 timeout: 0 lvb_type: 0 00000040:00080000:2.0:1776178204.323236:0:11281:0:(llog_osd.c:738:llog_osd_write_rec()) added record [0x1:0x10:0x0].1006, 64 off72576 00000004:00080000:2.0:1776178204.323242:0:11281:0:(osp_sync.c:1658:osp_sync_add_commit_cb()) lustre-OST0000-osc-MDT0001: add commit cb at 911650181708ns, next at 912650179791ns, rc = 0 00010000:00010000:2.0:1776178204.323292:0:11281:0:(ldlm_lock.c:766:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: mdt-lustre-MDT0001_UUID lock: 00000000e4217a5c/0xa40083ab9bf536a6 lrc: 3/0,1 mode: --/PW res: [0x240000402:0x7fd:0x0].0x0 bits 0x0/0x0 rrc: 3 type: IBT gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 11281 timeout: 0 lvb_type: 0 00010000:00010000:2.0:1776178204.323319:0:11281:0:(ldlm_lock.c:1064:ldlm_granted_list_add_lock()) ### About to add lock: ns: mdt-lustre-MDT0001_UUID lock: 00000000e4217a5c/0xa40083ab9bf536a6 lrc: 3/0,1 mode: PW/PW res: [0x240000402:0x7fd:0x0].0x0 bits 0x40/0x0 rrc: 3 type: IBT gid 18446635898622804416 flags: 0x50010080000000 nid: local remote: 0x0 expref: -99 pid: 11281 timeout: 0 lvb_type: 0 00010000:00010000:2.0:1776178204.323328:0:11281:0:(ldlm_request.c:528:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: mdt-lustre-MDT0001_UUID lock: 00000000e4217a5c/0xa40083ab9bf536a6 lrc: 3/0,1 mode: PW/PW res: [0x240000402:0x7fd:0x0].0x0 bits 0x40/0x0 rrc: 3 type: IBT gid 18446635898622804416 flags: 0x40010080000000 nid: local remote: 0x0 expref: -99 pid: 11281 timeout: 0 lvb_type: 0 00010000:00010000:2.0:1776178204.323336:0:11281:0:(ldlm_lock.c:942:ldlm_lock_decref_and_cancel()) ### ldlm_lock_decref(PW) ns: mdt-lustre-MDT0001_UUID lock: 00000000e4217a5c/0xa40083ab9bf536a6 lrc: 4/0,1 mode: PW/PW res: [0x240000402:0x7fd:0x0].0x0 bits 0x40/0x0 rrc: 3 type: IBT gid 18446635898622804416 flags: 0x40010080000000 nid: local remote: 0x0 expref: -99 pid: 11281 timeout: 0 lvb_type: 0 00010000:00010000:2.0:1776178204.323343:0:11281:0:(ldlm_lock.c:820:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: mdt-lustre-MDT0001_UUID lock: 00000000e4217a5c/0xa40083ab9bf536a6 lrc: 4/0,1 mode: PW/PW res: [0x240000402:0x7fd:0x0].0x0 bits 0x40/0x0 rrc: 3 type: IBT gid 18446635898622804416 flags: 0x50010480000000 nid: local remote: 0x0 expref: -99 pid: 11281 timeout: 0 lvb_type: 0 00010000:00010000:2.0:1776178204.323348:0:11281:0:(ldlm_lock.c:879:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: mdt-lustre-MDT0001_UUID lock: 00000000e4217a5c/0xa40083ab9bf536a6 lrc: 3/0,0 mode: PW/PW res: [0x240000402:0x7fd:0x0].0x0 bits 0x40/0x0 rrc: 3 type: IBT gid 18446635898622804416 flags: 0x50010480000000 nid: local remote: 0x0 expref: -99 pid: 11281 timeout: 0 lvb_type: 0 00010000:00010000:2.0:1776178204.323371:0:11281:0:(ldlm_lockd.c:2159:__ldlm_bl_to_thread()) added 1/1 locks to regular blp list, 1 blwis in pool 00000004:00010000:2.0:1776178204.323391:0:11281:0:(mdt_handler.c:3992:mdt_save_lock()) ### save lock request 000000008c3fbdb6 reply state 00000000e7846b5f transno 17179876267 ns: mdt-lustre-MDT0001_UUID lock: 000000000396420a/0xa40083ab9bf5369f lrc: 3/0,1 mode: EX/EX res: [0x240000402:0x7fd:0x0].0x0 bits 0x3/0x0 rrc: 3 type: IBT gid 0 flags: 0x40210000000000 nid: local remote: 0x0 expref: -99 pid: 11281 timeout: 0 lvb_type: 0 00010000:00010000:2.0:1776178204.323513:0:11281:0:(ldlm_lock.c:820:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(CW) ns: mdt-lustre-MDT0001_UUID lock: 000000007ae7b5f6/0xa40083ab9bf53691 lrc: 3/0,1 mode: CW/CW res: [0x240000402:0x7f7:0x0].0x0 bits 0x2/0x0 rrc: 3 type: IBT gid 0 flags: 0x50210000000000 nid: local remote: 0x0 expref: -99 pid: 11281 timeout: 0 lvb_type: 0 00010000:00010000:2.0:1776178204.323519:0:11281:0:(ldlm_lock.c:879:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: mdt-lustre-MDT0001_UUID lock: 000000007ae7b5f6/0xa40083ab9bf53691 lrc: 2/0,0 mode: CW/CW res: [0x240000402:0x7f7:0x0].0x0 bits 0x2/0x0 rrc: 3 type: IBT gid 0 flags: 0x50210400000000 nid: local remote: 0x0 expref: -99 pid: 11281 timeout: 0 lvb_type: 0 00010000:00010000:2.0:1776178204.323525:0:11281:0:(ldlm_lockd.c:1879:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: mdt-lustre-MDT0001_UUID lock: 000000007ae7b5f6/0xa40083ab9bf53691 lrc: 3/0,0 mode: CW/CW res: [0x240000402:0x7f7:0x0].0x0 bits 0x2/0x0 rrc: 3 type: IBT gid 0 flags: 0x40210400000000 nid: local remote: 0x0 expref: -99 pid: 11281 timeout: 0 lvb_type: 0 00010000:00010000:2.0:1776178204.323531:0:11281:0:(ldlm_lockd.c:1893:ldlm_handle_bl_callback()) Lock 000000007ae7b5f6 already unused, calling callback (0000000079e78348) 00010000:00010000:2.0:1776178204.323535:0:11281:0:(ldlm_request.c:364:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: mdt-lustre-MDT0001_UUID lock: 000000007ae7b5f6/0xa40083ab9bf53691 lrc: 3/0,0 mode: CW/CW res: [0x240000402:0x7f7:0x0].0x0 bits 0x2/0x0 rrc: 3 type: IBT gid 0 flags: 0x40210400000000 nid: local remote: 0x0 expref: -99 pid: 11281 timeout: 0 lvb_type: 0 00010000:00010000:2.0:1776178204.323541:0:11281:0:(ldlm_request.c:1257:ldlm_cli_cancel_local()) ### server-side local cancel ns: mdt-lustre-MDT0001_UUID lock: 000000007ae7b5f6/0xa40083ab9bf53691 lrc: 4/0,0 mode: CW/CW res: [0x240000402:0x7f7:0x0].0x0 bits 0x2/0x0 rrc: 3 type: IBT gid 0 flags: 0x40218400000000 nid: local remote: 0x0 expref: -99 pid: 11281 timeout: 0 lvb_type: 0 00010000:00010000:2.0:1776178204.323550:0:11281:0:(ldlm_inodebits.c:94:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x240000402:0x7f7:0x0].0x0 (0000000043c2d1ef) 00010000:00010000:2.0:1776178204.323553:0:11281:0:(ldlm_inodebits.c:96:ldlm_reprocess_inodebits_queue()) Hint 2 00010000:00010000:2.0:1776178204.323556:0:11281:0:(ldlm_lockd.c:1903:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: mdt-lustre-MDT0001_UUID lock: 000000007ae7b5f6/0xa40083ab9bf53691 lrc: 2/0,0 mode: --/CW res: [0x240000402:0x7f7:0x0].0x0 bits 0x2/0x0 rrc: 3 type: IBT gid 0 flags: 0x44a19400000000 nid: local remote: 0x0 expref: -99 pid: 11281 timeout: 0 lvb_type: 0 00010000:00010000:2.0:1776178204.323561:0:11281:0:(ldlm_lock.c:216:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: mdt-lustre-MDT0001_UUID lock: 000000007ae7b5f6/0xa40083ab9bf53691 lrc: 0/0,0 mode: --/CW res: [0x240000402:0x7f7:0x0].0x0 bits 0x2/0x0 rrc: 3 type: IBT gid 0 flags: 0x44a19400000000 nid: local remote: 0x0 expref: -99 pid: 11281 timeout: 0 lvb_type: 0 00000004:00010000:2.0:1776178204.323569:0:11281:0:(mdt_handler.c:3992:mdt_save_lock()) ### save lock request 000000008c3fbdb6 reply state 00000000e7846b5f transno 17179876267 ns: mdt-lustre-MDT0001_UUID lock: 000000006cc9c357/0xa40083ab9bf53698 lrc: 3/0,1 mode: PW/PW res: [0x240000402:0x7f7:0x0].0x37639fea bits 0x2/0x0 rrc: 2 type: IBT gid 0 flags: 0x40210000000000 nid: local remote: 0x0 expref: -99 pid: 11281 timeout: 0 lvb_type: 0 00010000:00010000:1.0:1776178204.323858:0:9225:0:(ldlm_lockd.c:2789:ldlm_bl_get_work()) Got 1 locks of 0 total in blp. (0 blwis in pool) 00010000:00010000:1.0:1776178204.323863:0:9225:0:(ldlm_lockd.c:1879:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: mdt-lustre-MDT0001_UUID lock: 00000000e4217a5c/0xa40083ab9bf536a6 lrc: 2/0,0 mode: PW/PW res: [0x240000402:0x7fd:0x0].0x0 bits 0x40/0x0 rrc: 3 type: IBT gid 18446635898622804416 flags: 0x40010480000000 nid: local remote: 0x0 expref: -99 pid: 11281 timeout: 0 lvb_type: 0 00010000:00010000:1.0:1776178204.323871:0:9225:0:(ldlm_lockd.c:1893:ldlm_handle_bl_callback()) Lock 00000000e4217a5c already unused, calling callback (0000000063c97258) 00010000:00010000:1.0:1776178204.323873:0:9225:0:(ldlm_request.c:364:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: mdt-lustre-MDT0001_UUID lock: 00000000e4217a5c/0xa40083ab9bf536a6 lrc: 2/0,0 mode: PW/PW res: [0x240000402:0x7fd:0x0].0x0 bits 0x40/0x0 rrc: 3 type: IBT gid 18446635898622804416 flags: 0x40010480000000 nid: local remote: 0x0 expref: -99 pid: 11281 timeout: 0 lvb_type: 0 00010000:00010000:1.0:1776178204.323879:0:9225:0:(ldlm_request.c:1257:ldlm_cli_cancel_local()) ### server-side local cancel ns: mdt-lustre-MDT0001_UUID lock: 00000000e4217a5c/0xa40083ab9bf536a6 lrc: 3/0,0 mode: PW/PW res: [0x240000402:0x7fd:0x0].0x0 bits 0x40/0x0 rrc: 3 type: IBT gid 18446635898622804416 flags: 0x40018480000000 nid: local remote: 0x0 expref: -99 pid: 11281 timeout: 0 lvb_type: 0 00010000:00010000:1.0:1776178204.323888:0:9225:0:(ldlm_inodebits.c:94:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x240000402:0x7fd:0x0].0x0 (00000000aeb25ebf) 00010000:00010000:1.0:1776178204.323890:0:9225:0:(ldlm_inodebits.c:96:ldlm_reprocess_inodebits_queue()) Hint 40 00010000:00010000:1.0:1776178204.323892:0:9225:0:(ldlm_lockd.c:1903:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: mdt-lustre-MDT0001_UUID lock: 00000000e4217a5c/0xa40083ab9bf536a6 lrc: 1/0,0 mode: --/PW res: [0x240000402:0x7fd:0x0].0x0 bits 0x40/0x0 rrc: 3 type: IBT gid 18446635898622804416 flags: 0x44819480000000 nid: local remote: 0x0 expref: -99 pid: 11281 timeout: 0 lvb_type: 0 00010000:00010000:1.0:1776178204.323897:0:9225:0:(ldlm_lock.c:216:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: mdt-lustre-MDT0001_UUID lock: 00000000e4217a5c/0xa40083ab9bf536a6 lrc: 0/0,0 mode: --/PW res: [0x240000402:0x7fd:0x0].0x0 bits 0x40/0x0 rrc: 3 type: IBT gid 18446635898622804416 flags: 0x44819480000000 nid: local remote: 0x0 expref: -99 pid: 11281 timeout: 0 lvb_type: 0 00010000:00010000:1.0:1776178204.323909:0:9225:0:(ldlm_lockd.c:2792:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00010000:00010000:1.0:1776178204.323910:0:9225:0:(ldlm_lockd.c:2792:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00010000:00010000:1.0:1776178204.323912:0:9225:0:(ldlm_lockd.c:2792:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:2.0:1776178204.324210:0:11281:0:(service.c:2372:ptlrpc_server_handle_request()) Handled RPC req@000000008c3fbdb6 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_004:b50963bd-8eba-4432-9a54-8888d00e3fa1+24:17671:x1862456979100928:12345-192.168.204.3@tcp:36:rm.0 Request processed in 1839us (4287us total) trans 17179876267 rc 0/0 00000100:00100000:2.0:1776178204.324219:0:11281:0:(nrs_fifo.c:241:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.204.3@tcp, seq: 6584 00000100:00100000:3.0:1776178204.329254:0:3348:0:(events.c:361:request_in_callback()) peer: 12345-192.168.204.3@tcp (source: 12345-192.168.204.3@tcp) 00010000:00010000:1.0:1776178204.329281:0:3339:0:(ldlm_lock.c:1064:ldlm_granted_list_add_lock()) ### About to add lock: ns: mdt-lustre-MDT0001_UUID lock: 000000006cc9c357/0xa40083ab9bf53698 lrc: 3/0,1 mode: COS/COS res: [0x240000402:0x7f7:0x0].0x37639fea bits 0x2/0x0 rrc: 2 type: IBT gid 0 flags: 0x50210000000000 nid: local remote: 0x0 expref: -99 pid: 11281 timeout: 0 lvb_type: 0 00010000:00010000:1.0:1776178204.329301:0:3339:0:(ldlm_inodebits.c:94:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x240000402:0x7f7:0x0].0x37639fea (00000000e4751492) 00010000:00010000:1.0:1776178204.329314:0:3339:0:(ldlm_inodebits.c:96:ldlm_reprocess_inodebits_queue()) Hint 2 00010000:00010000:1.0:1776178204.329320:0:3339:0:(ldlm_lock.c:1064:ldlm_granted_list_add_lock()) ### About to add lock: ns: mdt-lustre-MDT0001_UUID lock: 000000000396420a/0xa40083ab9bf5369f lrc: 3/0,1 mode: COS/COS res: [0x240000402:0x7fd:0x0].0x0 bits 0x3/0x0 rrc: 2 type: IBT gid 0 flags: 0x50210000000000 nid: local remote: 0x0 expref: -99 pid: 11281 timeout: 0 lvb_type: 0 00010000:00010000:1.0:1776178204.329327:0:3339:0:(ldlm_inodebits.c:94:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x240000402:0x7fd:0x0].0x0 (00000000aeb25ebf) 00010000:00010000:1.0:1776178204.329329:0:3339:0:(ldlm_inodebits.c:96:ldlm_reprocess_inodebits_queue()) Hint 3 00000100:00100000:2.0:1776178204.329415:0:11281:0:(service.c:2152:ptlrpc_server_handle_req_in()) got req x1862456979101056 00000100:00100000:2.0:1776178204.329473:0:11281:0:(nrs_fifo.c:179:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.204.3@tcp, seq: 6585 00000100:00100000:2.0:1776178204.329480:0:11281:0:(service.c:2324:ptlrpc_server_handle_request()) Handling RPC req@0000000047c15ac0 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_004:b50963bd-8eba-4432-9a54-8888d00e3fa1+24:17671:x1862456979101056:12345-192.168.204.3@tcp:36:rm.0 00010000:00010000:2.0:1776178204.329543:0:11281:0:(ldlm_lockd.c:1720:ldlm_request_cancel()) ### server-side cancel handler START: 2 locks, starting at 0 00010000:00010000:2.0:1776178204.329574:0:11281:0:(ldlm_lock.c:216:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: mdt-lustre-MDT0001_UUID lock: 00000000712476de/0xa40083ab9bf53605 lrc: 0/0,0 mode: --/PR res: [0x240000402:0x7fb:0x0].0x0 bits 0x1b/0x0 rrc: 4 type: IBT gid 0 flags: 0x44a01000000000 nid: 192.168.204.3@tcp remote: 0x771f68aa1eb27b7a expref: 24 pid: 6257 timeout: 0 lvb_type: 0 00010000:00010000:2.0:1776178204.329768:0:11281:0:(ldlm_lock.c:216:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: mdt-lustre-MDT0001_UUID lock: 0000000014581b1a/0xa40083ab9bf53587 lrc: 0/0,0 mode: --/CR res: [0x240000402:0x7fb:0x0].0x0 bits 0x8/0x0 rrc: 3 type: IBT gid 0 flags: 0x44801000000000 nid: 192.168.204.3@tcp remote: 0x771f68aa1eb27b49 expref: 23 pid: 6255 timeout: 0 lvb_type: 0 00010000:00010000:2.0:1776178204.329785:0:11281:0:(ldlm_inodebits.c:94:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x240000402:0x7fb:0x0].0x0 (000000000885de38) 00010000:00010000:2.0:1776178204.329795:0:11281:0:(ldlm_lockd.c:1782:ldlm_request_cancel()) ### server-side cancel handler END 00010000:00010000:2.0:1776178204.329820:0:11281:0:(ldlm_lock.c:766:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(CW) ns: mdt-lustre-MDT0001_UUID lock: 0000000073f2c7a9/0xa40083ab9bf536ad lrc: 3/0,1 mode: --/CW res: [0x240000402:0x7f7:0x0].0x0 bits 0x0/0x0 rrc: 3 type: IBT gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 11281 timeout: 0 lvb_type: 0 00010000:00010000:2.0:1776178204.329832:0:11281:0:(ldlm_lock.c:1064:ldlm_granted_list_add_lock()) ### About to add lock: ns: mdt-lustre-MDT0001_UUID lock: 0000000073f2c7a9/0xa40083ab9bf536ad lrc: 3/0,1 mode: CW/CW res: [0x240000402:0x7f7:0x0].0x0 bits 0x2/0x0 rrc: 3 type: IBT gid 0 flags: 0x50210000000000 nid: local remote: 0x0 expref: -99 pid: 11281 timeout: 0 lvb_type: 0 00010000:00010000:2.0:1776178204.329841:0:11281:0:(ldlm_request.c:528:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: mdt-lustre-MDT0001_UUID lock: 0000000073f2c7a9/0xa40083ab9bf536ad lrc: 3/0,1 mode: CW/CW res: [0x240000402:0x7f7:0x0].0x0 bits 0x2/0x0 rrc: 3 type: IBT gid 0 flags: 0x40210000000000 nid: local remote: 0x0 expref: -99 pid: 11281 timeout: 0 lvb_type: 0 00010000:00010000:2.0:1776178204.329854:0:11281:0:(ldlm_lock.c:766:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: mdt-lustre-MDT0001_UUID lock: 00000000a27c19b3/0xa40083ab9bf536b4 lrc: 3/0,1 mode: --/PW res: [0x240000402:0x7f7:0x0].0xa6b02dea bits 0x0/0x0 rrc: 2 type: IBT gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 11281 timeout: 0 lvb_type: 0 00010000:00010000:2.0:1776178204.329859:0:11281:0:(ldlm_lock.c:1064:ldlm_granted_list_add_lock()) ### About to add lock: ns: mdt-lustre-MDT0001_UUID lock: 00000000a27c19b3/0xa40083ab9bf536b4 lrc: 3/0,1 mode: PW/PW res: [0x240000402:0x7f7:0x0].0xa6b02dea bits 0x2/0x0 rrc: 2 type: IBT gid 0 flags: 0x50210000000000 nid: local remote: 0x0 expref: -99 pid: 11281 timeout: 0 lvb_type: 0 00010000:00010000:2.0:1776178204.329864:0:11281:0:(ldlm_request.c:528:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: mdt-lustre-MDT0001_UUID lock: 00000000a27c19b3/0xa40083ab9bf536b4 lrc: 3/0,1 mode: PW/PW res: [0x240000402:0x7f7:0x0].0xa6b02dea bits 0x2/0x0 rrc: 2 type: IBT gid 0 flags: 0x40210000000000 nid: local remote: 0x0 expref: -99 pid: 11281 timeout: 0 lvb_type: 0 00010000:00010000:2.0:1776178204.329944:0:11281:0:(ldlm_lock.c:766:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(EX) ns: mdt-lustre-MDT0001_UUID lock: 000000009a8f90ae/0xa40083ab9bf536bb lrc: 3/0,1 mode: --/EX res: [0x240000402:0x7fb:0x0].0x0 bits 0x0/0x0 rrc: 2 type: IBT gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 11281 timeout: 0 lvb_type: 0 00010000:00010000:2.0:1776178204.329952:0:11281:0:(ldlm_lock.c:1064:ldlm_granted_list_add_lock()) ### About to add lock: ns: mdt-lustre-MDT0001_UUID lock: 000000009a8f90ae/0xa40083ab9bf536bb lrc: 3/0,1 mode: EX/EX res: [0x240000402:0x7fb:0x0].0x0 bits 0x3/0x0 rrc: 2 type: IBT gid 0 flags: 0x50210000000000 nid: local remote: 0x0 expref: -99 pid: 11281 timeout: 0 lvb_type: 0 00010000:00010000:2.0:1776178204.329958:0:11281:0:(ldlm_request.c:528:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: mdt-lustre-MDT0001_UUID lock: 000000009a8f90ae/0xa40083ab9bf536bb lrc: 3/0,1 mode: EX/EX res: [0x240000402:0x7fb:0x0].0x0 bits 0x3/0x0 rrc: 2 type: IBT gid 0 flags: 0x40210000000000 nid: local remote: 0x0 expref: -99 pid: 11281 timeout: 0 lvb_type: 0 00000040:00080000:2.0:1776178204.330232:0:11281:0:(llog_osd.c:738:llog_osd_write_rec()) added record [0x1:0x10:0x0].1007, 64 off72640 00010000:00010000:2.0:1776178204.330279:0:11281:0:(ldlm_lock.c:766:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: mdt-lustre-MDT0001_UUID lock: 00000000540a8f8b/0xa40083ab9bf536c2 lrc: 3/0,1 mode: --/PW res: [0x240000402:0x7fb:0x0].0x0 bits 0x0/0x0 rrc: 3 type: IBT gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 11281 timeout: 0 lvb_type: 0 00010000:00010000:2.0:1776178204.330288:0:11281:0:(ldlm_lock.c:1064:ldlm_granted_list_add_lock()) ### About to add lock: ns: mdt-lustre-MDT0001_UUID lock: 00000000540a8f8b/0xa40083ab9bf536c2 lrc: 3/0,1 mode: PW/PW res: [0x240000402:0x7fb:0x0].0x0 bits 0x40/0x0 rrc: 3 type: IBT gid 18446635898622804416 flags: 0x50010080000000 nid: local remote: 0x0 expref: -99 pid: 11281 timeout: 0 lvb_type: 0 00010000:00010000:2.0:1776178204.330295:0:11281:0:(ldlm_request.c:528:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: mdt-lustre-MDT0001_UUID lock: 00000000540a8f8b/0xa40083ab9bf536c2 lrc: 3/0,1 mode: PW/PW res: [0x240000402:0x7fb:0x0].0x0 bits 0x40/0x0 rrc: 3 type: IBT gid 18446635898622804416 flags: 0x40010080000000 nid: local remote: 0x0 expref: -99 pid: 11281 timeout: 0 lvb_type: 0 00010000:00010000:2.0:1776178204.330302:0:11281:0:(ldlm_lock.c:942:ldlm_lock_decref_and_cancel()) ### ldlm_lock_decref(PW) ns: mdt-lustre-MDT0001_UUID lock: 00000000540a8f8b/0xa40083ab9bf536c2 lrc: 4/0,1 mode: PW/PW res: [0x240000402:0x7fb:0x0].0x0 bits 0x40/0x0 rrc: 3 type: IBT gid 18446635898622804416 flags: 0x40010080000000 nid: local remote: 0x0 expref: -99 pid: 11281 timeout: 0 lvb_type: 0 00010000:00010000:2.0:1776178204.330308:0:11281:0:(ldlm_lock.c:820:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: mdt-lustre-MDT0001_UUID lock: 00000000540a8f8b/0xa40083ab9bf536c2 lrc: 4/0,1 mode: PW/PW res: [0x240000402:0x7fb:0x0].0x0 bits 0x40/0x0 rrc: 3 type: IBT gid 18446635898622804416 flags: 0x50010480000000 nid: local remote: 0x0 expref: -99 pid: 11281 timeout: 0 lvb_type: 0 00010000:00010000:2.0:1776178204.330314:0:11281:0:(ldlm_lock.c:879:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: mdt-lustre-MDT0001_UUID lock: 00000000540a8f8b/0xa40083ab9bf536c2 lrc: 3/0,0 mode: PW/PW res: [0x240000402:0x7fb:0x0].0x0 bits 0x40/0x0 rrc: 3 type: IBT gid 18446635898622804416 flags: 0x50010480000000 nid: local remote: 0x0 expref: -99 pid: 11281 timeout: 0 lvb_type: 0 00010000:00010000:2.0:1776178204.330348:0:11281:0:(ldlm_lockd.c:2159:__ldlm_bl_to_thread()) added 1/1 locks to regular blp list, 1 blwis in pool 00000004:00010000:2.0:1776178204.330372:0:11281:0:(mdt_handler.c:3992:mdt_save_lock()) ### save lock request 0000000047c15ac0 reply state 000000002b94b459 transno 17179876268 ns: mdt-lustre-MDT0001_UUID lock: 000000009a8f90ae/0xa40083ab9bf536bb lrc: 3/0,1 mode: EX/EX res: [0x240000402:0x7fb:0x0].0x0 bits 0x3/0x0 rrc: 3 type: IBT gid 0 flags: 0x40210000000000 nid: local remote: 0x0 expref: -99 pid: 11281 timeout: 0 lvb_type: 0 00010000:00010000:2.0:1776178204.330504:0:11281:0:(ldlm_lock.c:820:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(CW) ns: mdt-lustre-MDT0001_UUID lock: 0000000073f2c7a9/0xa40083ab9bf536ad lrc: 3/0,1 mode: CW/CW res: [0x240000402:0x7f7:0x0].0x0 bits 0x2/0x0 rrc: 3 type: IBT gid 0 flags: 0x50210000000000 nid: local remote: 0x0 expref: -99 pid: 11281 timeout: 0 lvb_type: 0 00010000:00010000:2.0:1776178204.330511:0:11281:0:(ldlm_lock.c:879:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: mdt-lustre-MDT0001_UUID lock: 0000000073f2c7a9/0xa40083ab9bf536ad lrc: 2/0,0 mode: CW/CW res: [0x240000402:0x7f7:0x0].0x0 bits 0x2/0x0 rrc: 3 type: IBT gid 0 flags: 0x50210400000000 nid: local remote: 0x0 expref: -99 pid: 11281 timeout: 0 lvb_type: 0 00010000:00010000:2.0:1776178204.330523:0:11281:0:(ldlm_lockd.c:1879:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: mdt-lustre-MDT0001_UUID lock: 0000000073f2c7a9/0xa40083ab9bf536ad lrc: 3/0,0 mode: CW/CW res: [0x240000402:0x7f7:0x0].0x0 bits 0x2/0x0 rrc: 3 type: IBT gid 0 flags: 0x40210400000000 nid: local remote: 0x0 expref: -99 pid: 11281 timeout: 0 lvb_type: 0 00010000:00010000:2.0:1776178204.330530:0:11281:0:(ldlm_lockd.c:1893:ldlm_handle_bl_callback()) Lock 0000000073f2c7a9 already unused, calling callback (0000000079e78348) 00010000:00010000:2.0:1776178204.330534:0:11281:0:(ldlm_request.c:364:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: mdt-lustre-MDT0001_UUID lock: 0000000073f2c7a9/0xa40083ab9bf536ad lrc: 3/0,0 mode: CW/CW res: [0x240000402:0x7f7:0x0].0x0 bits 0x2/0x0 rrc: 3 type: IBT gid 0 flags: 0x40210400000000 nid: local remote: 0x0 expref: -99 pid: 11281 timeout: 0 lvb_type: 0 00010000:00010000:2.0:1776178204.330541:0:11281:0:(ldlm_request.c:1257:ldlm_cli_cancel_local()) ### server-side local cancel ns: mdt-lustre-MDT0001_UUID lock: 0000000073f2c7a9/0xa40083ab9bf536ad lrc: 4/0,0 mode: CW/CW res: [0x240000402:0x7f7:0x0].0x0 bits 0x2/0x0 rrc: 3 type: IBT gid 0 flags: 0x40218400000000 nid: local remote: 0x0 expref: -99 pid: 11281 timeout: 0 lvb_type: 0 00010000:00010000:2.0:1776178204.330550:0:11281:0:(ldlm_inodebits.c:94:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x240000402:0x7f7:0x0].0x0 (0000000043c2d1ef) 00010000:00010000:2.0:1776178204.330552:0:11281:0:(ldlm_inodebits.c:96:ldlm_reprocess_inodebits_queue()) Hint 2 00010000:00010000:2.0:1776178204.330555:0:11281:0:(ldlm_lockd.c:1903:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: mdt-lustre-MDT0001_UUID lock: 0000000073f2c7a9/0xa40083ab9bf536ad lrc: 2/0,0 mode: --/CW res: [0x240000402:0x7f7:0x0].0x0 bits 0x2/0x0 rrc: 3 type: IBT gid 0 flags: 0x44a19400000000 nid: local remote: 0x0 expref: -99 pid: 11281 timeout: 0 lvb_type: 0 00010000:00010000:2.0:1776178204.330560:0:11281:0:(ldlm_lock.c:216:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: mdt-lustre-MDT0001_UUID lock: 0000000073f2c7a9/0xa40083ab9bf536ad lrc: 0/0,0 mode: --/CW res: [0x240000402:0x7f7:0x0].0x0 bits 0x2/0x0 rrc: 3 type: IBT gid 0 flags: 0x44a19400000000 nid: local remote: 0x0 expref: -99 pid: 11281 timeout: 0 lvb_type: 0 00000004:00010000:2.0:1776178204.330569:0:11281:0:(mdt_handler.c:3992:mdt_save_lock()) ### save lock request 0000000047c15ac0 reply state 000000002b94b459 transno 17179876268 ns: mdt-lustre-MDT0001_UUID lock: 00000000a27c19b3/0xa40083ab9bf536b4 lrc: 3/0,1 mode: PW/PW res: [0x240000402:0x7f7:0x0].0xa6b02dea bits 0x2/0x0 rrc: 2 type: IBT gid 0 flags: 0x40210000000000 nid: local remote: 0x0 expref: -99 pid: 11281 timeout: 0 lvb_type: 0 00010000:00010000:0.0:1776178204.330876:0:9224:0:(ldlm_lockd.c:2789:ldlm_bl_get_work()) Got 1 locks of 0 total in blp. (0 blwis in pool) 00010000:00010000:0.0:1776178204.330884:0:9224:0:(ldlm_lockd.c:1879:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: mdt-lustre-MDT0001_UUID lock: 00000000540a8f8b/0xa40083ab9bf536c2 lrc: 2/0,0 mode: PW/PW res: [0x240000402:0x7fb:0x0].0x0 bits 0x40/0x0 rrc: 3 type: IBT gid 18446635898622804416 flags: 0x40010480000000 nid: local remote: 0x0 expref: -99 pid: 11281 timeout: 0 lvb_type: 0 00010000:00010000:0.0:1776178204.330891:0:9224:0:(ldlm_lockd.c:1893:ldlm_handle_bl_callback()) Lock 00000000540a8f8b already unused, calling callback (0000000063c97258) 00010000:00010000:0.0:1776178204.330895:0:9224:0:(ldlm_request.c:364:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: mdt-lustre-MDT0001_UUID lock: 00000000540a8f8b/0xa40083ab9bf536c2 lrc: 2/0,0 mode: PW/PW res: [0x240000402:0x7fb:0x0].0x0 bits 0x40/0x0 rrc: 3 type: IBT gid 18446635898622804416 flags: 0x40010480000000 nid: local remote: 0x0 expref: -99 pid: 11281 timeout: 0 lvb_type: 0 00010000:00010000:0.0:1776178204.330901:0:9224:0:(ldlm_request.c:1257:ldlm_cli_cancel_local()) ### server-side local cancel ns: mdt-lustre-MDT0001_UUID lock: 00000000540a8f8b/0xa40083ab9bf536c2 lrc: 3/0,0 mode: PW/PW res: [0x240000402:0x7fb:0x0].0x0 bits 0x40/0x0 rrc: 3 type: IBT gid 18446635898622804416 flags: 0x40018480000000 nid: local remote: 0x0 expref: -99 pid: 11281 timeout: 0 lvb_type: 0 00010000:00010000:0.0:1776178204.330910:0:9224:0:(ldlm_inodebits.c:94:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x240000402:0x7fb:0x0].0x0 (000000000bedd4cc) 00010000:00010000:0.0:1776178204.330913:0:9224:0:(ldlm_inodebits.c:96:ldlm_reprocess_inodebits_queue()) Hint 40 00010000:00010000:0.0:1776178204.330916:0:9224:0:(ldlm_lockd.c:1903:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: mdt-lustre-MDT0001_UUID lock: 00000000540a8f8b/0xa40083ab9bf536c2 lrc: 1/0,0 mode: --/PW res: [0x240000402:0x7fb:0x0].0x0 bits 0x40/0x0 rrc: 3 type: IBT gid 18446635898622804416 flags: 0x44819480000000 nid: local remote: 0x0 expref: -99 pid: 11281 timeout: 0 lvb_type: 0 00010000:00010000:0.0:1776178204.330921:0:9224:0:(ldlm_lock.c:216:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: mdt-lustre-MDT0001_UUID lock: 00000000540a8f8b/0xa40083ab9bf536c2 lrc: 0/0,0 mode: --/PW res: [0x240000402:0x7fb:0x0].0x0 bits 0x40/0x0 rrc: 3 type: IBT gid 18446635898622804416 flags: 0x44819480000000 nid: local remote: 0x0 expref: -99 pid: 11281 timeout: 0 lvb_type: 0 00010000:00010000:0.0:1776178204.330934:0:9224:0:(ldlm_lockd.c:2792:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00010000:00010000:0.0:1776178204.330936:0:9224:0:(ldlm_lockd.c:2792:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00010000:00010000:0.0:1776178204.330938:0:9224:0:(ldlm_lockd.c:2792:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:2.0:1776178204.332412:0:11281:0:(service.c:2372:ptlrpc_server_handle_request()) Handled RPC req@0000000047c15ac0 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_004:b50963bd-8eba-4432-9a54-8888d00e3fa1+24:17671:x1862456979101056:12345-192.168.204.3@tcp:36:rm.0 Request processed in 2923us (3160us total) trans 17179876268 rc 0/0 00000100:00100000:2.0:1776178204.332428:0:11281:0:(nrs_fifo.c:241:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.204.3@tcp, seq: 6585 00000100:00100000:1.0:1776178204.344274:0:3347:0:(events.c:361:request_in_callback()) peer: 12345-192.168.204.3@tcp (source: 12345-192.168.204.3@tcp) 00010000:00010000:3.0:1776178204.344434:0:3340:0:(ldlm_lock.c:1064:ldlm_granted_list_add_lock()) ### About to add lock: ns: mdt-lustre-MDT0001_UUID lock: 00000000a27c19b3/0xa40083ab9bf536b4 lrc: 3/0,1 mode: COS/COS res: [0x240000402:0x7f7:0x0].0xa6b02dea bits 0x2/0x0 rrc: 2 type: IBT gid 0 flags: 0x50210000000000 nid: local remote: 0x0 expref: -99 pid: 11281 timeout: 0 lvb_type: 0 00010000:00010000:3.0:1776178204.344454:0:3340:0:(ldlm_inodebits.c:94:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x240000402:0x7f7:0x0].0xa6b02dea (000000009a65918f) 00010000:00010000:3.0:1776178204.344457:0:3340:0:(ldlm_inodebits.c:96:ldlm_reprocess_inodebits_queue()) Hint 2 00010000:00010000:3.0:1776178204.344462:0:3340:0:(ldlm_lock.c:1064:ldlm_granted_list_add_lock()) ### About to add lock: ns: mdt-lustre-MDT0001_UUID lock: 000000009a8f90ae/0xa40083ab9bf536bb lrc: 3/0,1 mode: COS/COS res: [0x240000402:0x7fb:0x0].0x0 bits 0x3/0x0 rrc: 2 type: IBT gid 0 flags: 0x50210000000000 nid: local remote: 0x0 expref: -99 pid: 11281 timeout: 0 lvb_type: 0 00010000:00010000:3.0:1776178204.344469:0:3340:0:(ldlm_inodebits.c:94:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x240000402:0x7fb:0x0].0x0 (000000000bedd4cc) 00010000:00010000:3.0:1776178204.344471:0:3340:0:(ldlm_inodebits.c:96:ldlm_reprocess_inodebits_queue()) Hint 3 00000100:00100000:2.0:1776178204.344551:0:11281:0:(service.c:2152:ptlrpc_server_handle_req_in()) got req x1862456979101184 00000100:00100000:2.0:1776178204.344625:0:11281:0:(nrs_fifo.c:179:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.204.3@tcp, seq: 6586 00000100:00100000:2.0:1776178204.344632:0:11281:0:(service.c:2324:ptlrpc_server_handle_request()) Handling RPC req@000000006459e586 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_004:b50963bd-8eba-4432-9a54-8888d00e3fa1+24:17671:x1862456979101184:12345-192.168.204.3@tcp:36:rm.0 00010000:00010000:2.0:1776178204.344702:0:11281:0:(ldlm_lockd.c:1720:ldlm_request_cancel()) ### server-side cancel handler START: 2 locks, starting at 0 00010000:00010000:2.0:1776178204.344734:0:11281:0:(ldlm_lock.c:216:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: mdt-lustre-MDT0001_UUID lock: 000000002cc4819c/0xa40083ab9bf535b1 lrc: 0/0,0 mode: --/PR res: [0x240000402:0x7f8:0x0].0x0 bits 0x1b/0x0 rrc: 4 type: IBT gid 0 flags: 0x44a01000000000 nid: 192.168.204.3@tcp remote: 0x771f68aa1eb27b65 expref: 24 pid: 6257 timeout: 0 lvb_type: 0 00010000:00010000:2.0:1776178204.344769:0:11281:0:(ldlm_lock.c:216:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: mdt-lustre-MDT0001_UUID lock: 00000000a211f94c/0xa40083ab9bf53595 lrc: 0/0,0 mode: --/CR res: [0x240000402:0x7f8:0x0].0x0 bits 0x8/0x0 rrc: 3 type: IBT gid 0 flags: 0x44801000000000 nid: 192.168.204.3@tcp remote: 0x771f68aa1eb27b5e expref: 23 pid: 6257 timeout: 0 lvb_type: 0 00010000:00010000:2.0:1776178204.344778:0:11281:0:(ldlm_inodebits.c:94:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x240000402:0x7f8:0x0].0x0 (000000000b2f81d2) 00010000:00010000:2.0:1776178204.344786:0:11281:0:(ldlm_lockd.c:1782:ldlm_request_cancel()) ### server-side cancel handler END 00010000:00010000:2.0:1776178204.344811:0:11281:0:(ldlm_lock.c:766:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(CW) ns: mdt-lustre-MDT0001_UUID lock: 000000007ae7b5f6/0xa40083ab9bf536c9 lrc: 3/0,1 mode: --/CW res: [0x240000402:0x7f7:0x0].0x0 bits 0x0/0x0 rrc: 3 type: IBT gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 11281 timeout: 0 lvb_type: 0 00010000:00010000:2.0:1776178204.344823:0:11281:0:(ldlm_lock.c:1064:ldlm_granted_list_add_lock()) ### About to add lock: ns: mdt-lustre-MDT0001_UUID lock: 000000007ae7b5f6/0xa40083ab9bf536c9 lrc: 3/0,1 mode: CW/CW res: [0x240000402:0x7f7:0x0].0x0 bits 0x2/0x0 rrc: 3 type: IBT gid 0 flags: 0x50210000000000 nid: local remote: 0x0 expref: -99 pid: 11281 timeout: 0 lvb_type: 0 00010000:00010000:2.0:1776178204.344832:0:11281:0:(ldlm_request.c:528:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: mdt-lustre-MDT0001_UUID lock: 000000007ae7b5f6/0xa40083ab9bf536c9 lrc: 3/0,1 mode: CW/CW res: [0x240000402:0x7f7:0x0].0x0 bits 0x2/0x0 rrc: 3 type: IBT gid 0 flags: 0x40210000000000 nid: local remote: 0x0 expref: -99 pid: 11281 timeout: 0 lvb_type: 0 00010000:00010000:2.0:1776178204.344845:0:11281:0:(ldlm_lock.c:766:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: mdt-lustre-MDT0001_UUID lock: 000000001e6d7318/0xa40083ab9bf536d0 lrc: 3/0,1 mode: --/PW res: [0x240000402:0x7f7:0x0].0xcda302ea bits 0x0/0x0 rrc: 2 type: IBT gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 11281 timeout: 0 lvb_type: 0 00010000:00010000:2.0:1776178204.344852:0:11281:0:(ldlm_lock.c:1064:ldlm_granted_list_add_lock()) ### About to add lock: ns: mdt-lustre-MDT0001_UUID lock: 000000001e6d7318/0xa40083ab9bf536d0 lrc: 3/0,1 mode: PW/PW res: [0x240000402:0x7f7:0x0].0xcda302ea bits 0x2/0x0 rrc: 2 type: IBT gid 0 flags: 0x50210000000000 nid: local remote: 0x0 expref: -99 pid: 11281 timeout: 0 lvb_type: 0 00010000:00010000:2.0:1776178204.344858:0:11281:0:(ldlm_request.c:528:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: mdt-lustre-MDT0001_UUID lock: 000000001e6d7318/0xa40083ab9bf536d0 lrc: 3/0,1 mode: PW/PW res: [0x240000402:0x7f7:0x0].0xcda302ea bits 0x2/0x0 rrc: 2 type: IBT gid 0 flags: 0x40210000000000 nid: local remote: 0x0 expref: -99 pid: 11281 timeout: 0 lvb_type: 0 00010000:00010000:2.0:1776178204.344930:0:11281:0:(ldlm_lock.c:766:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(EX) ns: mdt-lustre-MDT0001_UUID lock: 000000000b5c0143/0xa40083ab9bf536d7 lrc: 3/0,1 mode: --/EX res: [0x240000402:0x7f8:0x0].0x0 bits 0x0/0x0 rrc: 2 type: IBT gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 11281 timeout: 0 lvb_type: 0 00010000:00010000:2.0:1776178204.344943:0:11281:0:(ldlm_lock.c:1064:ldlm_granted_list_add_lock()) ### About to add lock: ns: mdt-lustre-MDT0001_UUID lock: 000000000b5c0143/0xa40083ab9bf536d7 lrc: 3/0,1 mode: EX/EX res: [0x240000402:0x7f8:0x0].0x0 bits 0x3/0x0 rrc: 2 type: IBT gid 0 flags: 0x50210000000000 nid: local remote: 0x0 expref: -99 pid: 11281 timeout: 0 lvb_type: 0 00010000:00010000:2.0:1776178204.344949:0:11281:0:(ldlm_request.c:528:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: mdt-lustre-MDT0001_UUID lock: 000000000b5c0143/0xa40083ab9bf536d7 lrc: 3/0,1 mode: EX/EX res: [0x240000402:0x7f8:0x0].0x0 bits 0x3/0x0 rrc: 2 type: IBT gid 0 flags: 0x40210000000000 nid: local remote: 0x0 expref: -99 pid: 11281 timeout: 0 lvb_type: 0 00000040:00080000:2.0:1776178204.345557:0:11281:0:(llog_osd.c:738:llog_osd_write_rec()) added record [0x1:0x12:0x0].1006, 64 off72576 00000004:00080000:2.0:1776178204.345565:0:11281:0:(osp_sync.c:1658:osp_sync_add_commit_cb()) lustre-OST0001-osc-MDT0001: add commit cb at 911672504319ns, next at 912672501870ns, rc = 0 00010000:00010000:2.0:1776178204.345842:0:11281:0:(ldlm_lock.c:766:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: mdt-lustre-MDT0001_UUID lock: 00000000c9d7d9fc/0xa40083ab9bf536de lrc: 3/0,1 mode: --/PW res: [0x240000402:0x7f8:0x0].0x0 bits 0x0/0x0 rrc: 3 type: IBT gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 11281 timeout: 0 lvb_type: 0 00010000:00010000:2.0:1776178204.345855:0:11281:0:(ldlm_lock.c:1064:ldlm_granted_list_add_lock()) ### About to add lock: ns: mdt-lustre-MDT0001_UUID lock: 00000000c9d7d9fc/0xa40083ab9bf536de lrc: 3/0,1 mode: PW/PW res: [0x240000402:0x7f8:0x0].0x0 bits 0x40/0x0 rrc: 3 type: IBT gid 18446635898622804416 flags: 0x50010080000000 nid: local remote: 0x0 expref: -99 pid: 11281 timeout: 0 lvb_type: 0 00010000:00010000:2.0:1776178204.345864:0:11281:0:(ldlm_request.c:528:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: mdt-lustre-MDT0001_UUID lock: 00000000c9d7d9fc/0xa40083ab9bf536de lrc: 3/0,1 mode: PW/PW res: [0x240000402:0x7f8:0x0].0x0 bits 0x40/0x0 rrc: 3 type: IBT gid 18446635898622804416 flags: 0x40010080000000 nid: local remote: 0x0 expref: -99 pid: 11281 timeout: 0 lvb_type: 0 00010000:00010000:2.0:1776178204.345872:0:11281:0:(ldlm_lock.c:942:ldlm_lock_decref_and_cancel()) ### ldlm_lock_decref(PW) ns: mdt-lustre-MDT0001_UUID lock: 00000000c9d7d9fc/0xa40083ab9bf536de lrc: 4/0,1 mode: PW/PW res: [0x240000402:0x7f8:0x0].0x0 bits 0x40/0x0 rrc: 3 type: IBT gid 18446635898622804416 flags: 0x40010080000000 nid: local remote: 0x0 expref: -99 pid: 11281 timeout: 0 lvb_type: 0 00010000:00010000:2.0:1776178204.345878:0:11281:0:(ldlm_lock.c:820:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: mdt-lustre-MDT0001_UUID lock: 00000000c9d7d9fc/0xa40083ab9bf536de lrc: 4/0,1 mode: PW/PW res: [0x240000402:0x7f8:0x0].0x0 bits 0x40/0x0 rrc: 3 type: IBT gid 18446635898622804416 flags: 0x50010480000000 nid: local remote: 0x0 expref: -99 pid: 11281 timeout: 0 lvb_type: 0 00010000:00010000:2.0:1776178204.345883:0:11281:0:(ldlm_lock.c:879:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: mdt-lustre-MDT0001_UUID lock: 00000000c9d7d9fc/0xa40083ab9bf536de lrc: 3/0,0 mode: PW/PW res: [0x240000402:0x7f8:0x0].0x0 bits 0x40/0x0 rrc: 3 type: IBT gid 18446635898622804416 flags: 0x50010480000000 nid: local remote: 0x0 expref: -99 pid: 11281 timeout: 0 lvb_type: 0 00010000:00010000:2.0:1776178204.345909:0:11281:0:(ldlm_lockd.c:2159:__ldlm_bl_to_thread()) added 1/1 locks to regular blp list, 1 blwis in pool 00000004:00010000:2.0:1776178204.345926:0:11281:0:(mdt_handler.c:3992:mdt_save_lock()) ### save lock request 000000006459e586 reply state 000000007821093e transno 17179876269 ns: mdt-lustre-MDT0001_UUID lock: 000000000b5c0143/0xa40083ab9bf536d7 lrc: 3/0,1 mode: EX/EX res: [0x240000402:0x7f8:0x0].0x0 bits 0x3/0x0 rrc: 3 type: IBT gid 0 flags: 0x40210000000000 nid: local remote: 0x0 expref: -99 pid: 11281 timeout: 0 lvb_type: 0 00010000:00010000:2.0:1776178204.346023:0:11281:0:(ldlm_lock.c:820:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(CW) ns: mdt-lustre-MDT0001_UUID lock: 000000007ae7b5f6/0xa40083ab9bf536c9 lrc: 3/0,1 mode: CW/CW res: [0x240000402:0x7f7:0x0].0x0 bits 0x2/0x0 rrc: 3 type: IBT gid 0 flags: 0x50210000000000 nid: local remote: 0x0 expref: -99 pid: 11281 timeout: 0 lvb_type: 0 00010000:00010000:2.0:1776178204.346031:0:11281:0:(ldlm_lock.c:879:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: mdt-lustre-MDT0001_UUID lock: 000000007ae7b5f6/0xa40083ab9bf536c9 lrc: 2/0,0 mode: CW/CW res: [0x240000402:0x7f7:0x0].0x0 bits 0x2/0x0 rrc: 3 type: IBT gid 0 flags: 0x50210400000000 nid: local remote: 0x0 expref: -99 pid: 11281 timeout: 0 lvb_type: 0 00010000:00010000:2.0:1776178204.346037:0:11281:0:(ldlm_lockd.c:1879:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: mdt-lustre-MDT0001_UUID lock: 000000007ae7b5f6/0xa40083ab9bf536c9 lrc: 3/0,0 mode: CW/CW res: [0x240000402:0x7f7:0x0].0x0 bits 0x2/0x0 rrc: 3 type: IBT gid 0 flags: 0x40210400000000 nid: local remote: 0x0 expref: -99 pid: 11281 timeout: 0 lvb_type: 0 00010000:00010000:2.0:1776178204.346043:0:11281:0:(ldlm_lockd.c:1893:ldlm_handle_bl_callback()) Lock 000000007ae7b5f6 already unused, calling callback (0000000079e78348) 00010000:00010000:2.0:1776178204.346046:0:11281:0:(ldlm_request.c:364:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: mdt-lustre-MDT0001_UUID lock: 000000007ae7b5f6/0xa40083ab9bf536c9 lrc: 3/0,0 mode: CW/CW res: [0x240000402:0x7f7:0x0].0x0 bits 0x2/0x0 rrc: 3 type: IBT gid 0 flags: 0x40210400000000 nid: local remote: 0x0 expref: -99 pid: 11281 timeout: 0 lvb_type: 0 00010000:00010000:2.0:1776178204.346053:0:11281:0:(ldlm_request.c:1257:ldlm_cli_cancel_local()) ### server-side local cancel ns: mdt-lustre-MDT0001_UUID lock: 000000007ae7b5f6/0xa40083ab9bf536c9 lrc: 4/0,0 mode: CW/CW res: [0x240000402:0x7f7:0x0].0x0 bits 0x2/0x0 rrc: 3 type: IBT gid 0 flags: 0x40218400000000 nid: local remote: 0x0 expref: -99 pid: 11281 timeout: 0 lvb_type: 0 00010000:00010000:2.0:1776178204.346081:0:11281:0:(ldlm_inodebits.c:94:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x240000402:0x7f7:0x0].0x0 (0000000043c2d1ef) 00010000:00010000:2.0:1776178204.346085:0:11281:0:(ldlm_inodebits.c:96:ldlm_reprocess_inodebits_queue()) Hint 2 00010000:00010000:2.0:1776178204.346088:0:11281:0:(ldlm_lockd.c:1903:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: mdt-lustre-MDT0001_UUID lock: 000000007ae7b5f6/0xa40083ab9bf536c9 lrc: 2/0,0 mode: --/CW res: [0x240000402:0x7f7:0x0].0x0 bits 0x2/0x0 rrc: 3 type: IBT gid 0 flags: 0x44a19400000000 nid: local remote: 0x0 expref: -99 pid: 11281 timeout: 0 lvb_type: 0 00010000:00010000:2.0:1776178204.346094:0:11281:0:(ldlm_lock.c:216:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: mdt-lustre-MDT0001_UUID lock: 000000007ae7b5f6/0xa40083ab9bf536c9 lrc: 0/0,0 mode: --/CW res: [0x240000402:0x7f7:0x0].0x0 bits 0x2/0x0 rrc: 3 type: IBT gid 0 flags: 0x44a19400000000 nid: local remote: 0x0 expref: -99 pid: 11281 timeout: 0 lvb_type: 0 00000004:00010000:2.0:1776178204.346104:0:11281:0:(mdt_handler.c:3992:mdt_save_lock()) ### save lock request 000000006459e586 reply state 000000007821093e transno 17179876269 ns: mdt-lustre-MDT0001_UUID lock: 000000001e6d7318/0xa40083ab9bf536d0 lrc: 3/0,1 mode: PW/PW res: [0x240000402:0x7f7:0x0].0xcda302ea bits 0x2/0x0 rrc: 2 type: IBT gid 0 flags: 0x40210000000000 nid: local remote: 0x0 expref: -99 pid: 11281 timeout: 0 lvb_type: 0 00010000:00010000:1.0:1776178204.346236:0:6245:0:(ldlm_lockd.c:2789:ldlm_bl_get_work()) Got 1 locks of 0 total in blp. (0 blwis in pool) 00010000:00010000:1.0:1776178204.346240:0:6245:0:(ldlm_lockd.c:1879:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: mdt-lustre-MDT0001_UUID lock: 00000000c9d7d9fc/0xa40083ab9bf536de lrc: 2/0,0 mode: PW/PW res: [0x240000402:0x7f8:0x0].0x0 bits 0x40/0x0 rrc: 3 type: IBT gid 18446635898622804416 flags: 0x40010480000000 nid: local remote: 0x0 expref: -99 pid: 11281 timeout: 0 lvb_type: 0 00010000:00010000:1.0:1776178204.346247:0:6245:0:(ldlm_lockd.c:1893:ldlm_handle_bl_callback()) Lock 00000000c9d7d9fc already unused, calling callback (0000000063c97258) 00010000:00010000:1.0:1776178204.346249:0:6245:0:(ldlm_request.c:364:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: mdt-lustre-MDT0001_UUID lock: 00000000c9d7d9fc/0xa40083ab9bf536de lrc: 2/0,0 mode: PW/PW res: [0x240000402:0x7f8:0x0].0x0 bits 0x40/0x0 rrc: 3 type: IBT gid 18446635898622804416 flags: 0x40010480000000 nid: local remote: 0x0 expref: -99 pid: 11281 timeout: 0 lvb_type: 0 00010000:00010000:1.0:1776178204.346255:0:6245:0:(ldlm_request.c:1257:ldlm_cli_cancel_local()) ### server-side local cancel ns: mdt-lustre-MDT0001_UUID lock: 00000000c9d7d9fc/0xa40083ab9bf536de lrc: 3/0,0 mode: PW/PW res: [0x240000402:0x7f8:0x0].0x0 bits 0x40/0x0 rrc: 3 type: IBT gid 18446635898622804416 flags: 0x40018480000000 nid: local remote: 0x0 expref: -99 pid: 11281 timeout: 0 lvb_type: 0 00010000:00010000:1.0:1776178204.346263:0:6245:0:(ldlm_inodebits.c:94:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x240000402:0x7f8:0x0].0x0 (00000000d7645e58) 00010000:00010000:1.0:1776178204.346265:0:6245:0:(ldlm_inodebits.c:96:ldlm_reprocess_inodebits_queue()) Hint 40 00010000:00010000:1.0:1776178204.346267:0:6245:0:(ldlm_lockd.c:1903:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: mdt-lustre-MDT0001_UUID lock: 00000000c9d7d9fc/0xa40083ab9bf536de lrc: 1/0,0 mode: --/PW res: [0x240000402:0x7f8:0x0].0x0 bits 0x40/0x0 rrc: 3 type: IBT gid 18446635898622804416 flags: 0x44819480000000 nid: local remote: 0x0 expref: -99 pid: 11281 timeout: 0 lvb_type: 0 00010000:00010000:1.0:1776178204.346273:0:6245:0:(ldlm_lock.c:216:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: mdt-lustre-MDT0001_UUID lock: 00000000c9d7d9fc/0xa40083ab9bf536de lrc: 0/0,0 mode: --/PW res: [0x240000402:0x7f8:0x0].0x0 bits 0x40/0x0 rrc: 3 type: IBT gid 18446635898622804416 flags: 0x44819480000000 nid: local remote: 0x0 expref: -99 pid: 11281 timeout: 0 lvb_type: 0 00010000:00010000:1.0:1776178204.346283:0:6245:0:(ldlm_lockd.c:2792:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00010000:00010000:1.0:1776178204.346284:0:6245:0:(ldlm_lockd.c:2792:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00010000:00010000:1.0:1776178204.346285:0:6245:0:(ldlm_lockd.c:2792:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:2.0:1776178204.346321:0:11281:0:(service.c:2372:ptlrpc_server_handle_request()) Handled RPC req@000000006459e586 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_004:b50963bd-8eba-4432-9a54-8888d00e3fa1+24:17671:x1862456979101184:12345-192.168.204.3@tcp:36:rm.0 Request processed in 1687us (2054us total) trans 17179876269 rc 0/0 00000100:00100000:2.0:1776178204.346331:0:11281:0:(nrs_fifo.c:241:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.204.3@tcp, seq: 6586 00010000:00010000:1.0:1776178204.353802:0:3339:0:(ldlm_lock.c:1064:ldlm_granted_list_add_lock()) ### About to add lock: ns: mdt-lustre-MDT0001_UUID lock: 000000001e6d7318/0xa40083ab9bf536d0 lrc: 3/0,1 mode: COS/COS res: [0x240000402:0x7f7:0x0].0xcda302ea bits 0x2/0x0 rrc: 2 type: IBT gid 0 flags: 0x50210000000000 nid: local remote: 0x0 expref: -99 pid: 11281 timeout: 0 lvb_type: 0 00000100:00100000:3.0:1776178204.353821:0:3346:0:(events.c:361:request_in_callback()) peer: 12345-192.168.204.3@tcp (source: 12345-192.168.204.3@tcp) 00010000:00010000:1.0:1776178204.353831:0:3339:0:(ldlm_inodebits.c:94:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x240000402:0x7f7:0x0].0xcda302ea (0000000059286f6d) 00010000:00010000:1.0:1776178204.353835:0:3339:0:(ldlm_inodebits.c:96:ldlm_reprocess_inodebits_queue()) Hint 2 00010000:00010000:1.0:1776178204.353840:0:3339:0:(ldlm_lock.c:1064:ldlm_granted_list_add_lock()) ### About to add lock: ns: mdt-lustre-MDT0001_UUID lock: 000000000b5c0143/0xa40083ab9bf536d7 lrc: 3/0,1 mode: COS/COS res: [0x240000402:0x7f8:0x0].0x0 bits 0x3/0x0 rrc: 2 type: IBT gid 0 flags: 0x50210000000000 nid: local remote: 0x0 expref: -99 pid: 11281 timeout: 0 lvb_type: 0 00010000:00010000:1.0:1776178204.353846:0:3339:0:(ldlm_inodebits.c:94:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x240000402:0x7f8:0x0].0x0 (00000000d7645e58) 00010000:00010000:1.0:1776178204.353848:0:3339:0:(ldlm_inodebits.c:96:ldlm_reprocess_inodebits_queue()) Hint 3 00000100:00100000:2.0:1776178204.356971:0:11281:0:(service.c:2152:ptlrpc_server_handle_req_in()) got req x1862456979101312 00000100:00100000:2.0:1776178204.357055:0:11281:0:(nrs_fifo.c:179:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.204.3@tcp, seq: 6587 00000100:00100000:2.0:1776178204.357089:0:11281:0:(service.c:2324:ptlrpc_server_handle_request()) Handling RPC req@000000006342fd4b pname:cluuid+ref:pid:xid:nid:opc:job mdt00_004:b50963bd-8eba-4432-9a54-8888d00e3fa1+24:17671:x1862456979101312:12345-192.168.204.3@tcp:36:rm.0 00010000:00010000:2.0:1776178204.357153:0:11281:0:(ldlm_lockd.c:1720:ldlm_request_cancel()) ### server-side cancel handler START: 2 locks, starting at 0 00010000:00010000:2.0:1776178204.357180:0:11281:0:(ldlm_lock.c:216:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: mdt-lustre-MDT0001_UUID lock: 000000008cbcbaf5/0xa40083ab9bf535e9 lrc: 0/0,0 mode: --/PR res: [0x240000402:0x7fa:0x0].0x0 bits 0x1b/0x0 rrc: 4 type: IBT gid 0 flags: 0x44a01000000000 nid: 192.168.204.3@tcp remote: 0x771f68aa1eb27b73 expref: 24 pid: 6257 timeout: 0 lvb_type: 0 00010000:00010000:2.0:1776178204.357202:0:11281:0:(ldlm_lock.c:216:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: mdt-lustre-MDT0001_UUID lock: 000000006a188f9a/0xa40083ab9bf5358e lrc: 0/0,0 mode: --/CR res: [0x240000402:0x7fa:0x0].0x0 bits 0x8/0x0 rrc: 3 type: IBT gid 0 flags: 0x44801000000000 nid: 192.168.204.3@tcp remote: 0x771f68aa1eb27b57 expref: 23 pid: 8250 timeout: 0 lvb_type: 0 00010000:00010000:2.0:1776178204.357213:0:11281:0:(ldlm_inodebits.c:94:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x240000402:0x7fa:0x0].0x0 (0000000036d08056) 00010000:00010000:2.0:1776178204.357224:0:11281:0:(ldlm_lockd.c:1782:ldlm_request_cancel()) ### server-side cancel handler END 00010000:00010000:2.0:1776178204.357251:0:11281:0:(ldlm_lock.c:766:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(CW) ns: mdt-lustre-MDT0001_UUID lock: 0000000073f2c7a9/0xa40083ab9bf536e5 lrc: 3/0,1 mode: --/CW res: [0x240000402:0x7f7:0x0].0x0 bits 0x0/0x0 rrc: 3 type: IBT gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 11281 timeout: 0 lvb_type: 0 00010000:00010000:2.0:1776178204.357277:0:11281:0:(ldlm_lock.c:1064:ldlm_granted_list_add_lock()) ### About to add lock: ns: mdt-lustre-MDT0001_UUID lock: 0000000073f2c7a9/0xa40083ab9bf536e5 lrc: 3/0,1 mode: CW/CW res: [0x240000402:0x7f7:0x0].0x0 bits 0x2/0x0 rrc: 3 type: IBT gid 0 flags: 0x50210000000000 nid: local remote: 0x0 expref: -99 pid: 11281 timeout: 0 lvb_type: 0 00010000:00010000:2.0:1776178204.357288:0:11281:0:(ldlm_request.c:528:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: mdt-lustre-MDT0001_UUID lock: 0000000073f2c7a9/0xa40083ab9bf536e5 lrc: 3/0,1 mode: CW/CW res: [0x240000402:0x7f7:0x0].0x0 bits 0x2/0x0 rrc: 3 type: IBT gid 0 flags: 0x40210000000000 nid: local remote: 0x0 expref: -99 pid: 11281 timeout: 0 lvb_type: 0 00010000:00010000:2.0:1776178204.357313:0:11281:0:(ldlm_lock.c:766:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: mdt-lustre-MDT0001_UUID lock: 0000000096c1e297/0xa40083ab9bf536ec lrc: 3/0,1 mode: --/PW res: [0x240000402:0x7f7:0x0].0x5e5674ea bits 0x0/0x0 rrc: 2 type: IBT gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 11281 timeout: 0 lvb_type: 0 00010000:00010000:2.0:1776178204.357320:0:11281:0:(ldlm_lock.c:1064:ldlm_granted_list_add_lock()) ### About to add lock: ns: mdt-lustre-MDT0001_UUID lock: 0000000096c1e297/0xa40083ab9bf536ec lrc: 3/0,1 mode: PW/PW res: [0x240000402:0x7f7:0x0].0x5e5674ea bits 0x2/0x0 rrc: 2 type: IBT gid 0 flags: 0x50210000000000 nid: local remote: 0x0 expref: -99 pid: 11281 timeout: 0 lvb_type: 0 00010000:00010000:2.0:1776178204.357332:0:11281:0:(ldlm_request.c:528:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: mdt-lustre-MDT0001_UUID lock: 0000000096c1e297/0xa40083ab9bf536ec lrc: 3/0,1 mode: PW/PW res: [0x240000402:0x7f7:0x0].0x5e5674ea bits 0x2/0x0 rrc: 2 type: IBT gid 0 flags: 0x40210000000000 nid: local remote: 0x0 expref: -99 pid: 11281 timeout: 0 lvb_type: 0 00010000:00010000:2.0:1776178204.357411:0:11281:0:(ldlm_lock.c:766:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(EX) ns: mdt-lustre-MDT0001_UUID lock: 00000000c9d7d9fc/0xa40083ab9bf536f3 lrc: 3/0,1 mode: --/EX res: [0x240000402:0x7fa:0x0].0x0 bits 0x0/0x0 rrc: 2 type: IBT gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 11281 timeout: 0 lvb_type: 0 00010000:00010000:2.0:1776178204.357419:0:11281:0:(ldlm_lock.c:1064:ldlm_granted_list_add_lock()) ### About to add lock: ns: mdt-lustre-MDT0001_UUID lock: 00000000c9d7d9fc/0xa40083ab9bf536f3 lrc: 3/0,1 mode: EX/EX res: [0x240000402:0x7fa:0x0].0x0 bits 0x3/0x0 rrc: 2 type: IBT gid 0 flags: 0x50210000000000 nid: local remote: 0x0 expref: -99 pid: 11281 timeout: 0 lvb_type: 0 00010000:00010000:2.0:1776178204.357426:0:11281:0:(ldlm_request.c:528:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: mdt-lustre-MDT0001_UUID lock: 00000000c9d7d9fc/0xa40083ab9bf536f3 lrc: 3/0,1 mode: EX/EX res: [0x240000402:0x7fa:0x0].0x0 bits 0x3/0x0 rrc: 2 type: IBT gid 0 flags: 0x40210000000000 nid: local remote: 0x0 expref: -99 pid: 11281 timeout: 0 lvb_type: 0 00000040:00080000:2.0:1776178204.357643:0:11281:0:(llog_osd.c:738:llog_osd_write_rec()) added record [0x1:0x12:0x0].1007, 64 off72640 00010000:00010000:2.0:1776178204.357687:0:11281:0:(ldlm_lock.c:766:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: mdt-lustre-MDT0001_UUID lock: 00000000540a8f8b/0xa40083ab9bf536fa lrc: 3/0,1 mode: --/PW res: [0x240000402:0x7fa:0x0].0x0 bits 0x0/0x0 rrc: 3 type: IBT gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 11281 timeout: 0 lvb_type: 0 00010000:00010000:2.0:1776178204.357709:0:11281:0:(ldlm_lock.c:1064:ldlm_granted_list_add_lock()) ### About to add lock: ns: mdt-lustre-MDT0001_UUID lock: 00000000540a8f8b/0xa40083ab9bf536fa lrc: 3/0,1 mode: PW/PW res: [0x240000402:0x7fa:0x0].0x0 bits 0x40/0x0 rrc: 3 type: IBT gid 18446635898622804416 flags: 0x50010080000000 nid: local remote: 0x0 expref: -99 pid: 11281 timeout: 0 lvb_type: 0 00010000:00010000:2.0:1776178204.357788:0:11281:0:(ldlm_request.c:528:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: mdt-lustre-MDT0001_UUID lock: 00000000540a8f8b/0xa40083ab9bf536fa lrc: 3/0,1 mode: PW/PW res: [0x240000402:0x7fa:0x0].0x0 bits 0x40/0x0 rrc: 3 type: IBT gid 18446635898622804416 flags: 0x40010080000000 nid: local remote: 0x0 expref: -99 pid: 11281 timeout: 0 lvb_type: 0 00010000:00010000:2.0:1776178204.357797:0:11281:0:(ldlm_lock.c:942:ldlm_lock_decref_and_cancel()) ### ldlm_lock_decref(PW) ns: mdt-lustre-MDT0001_UUID lock: 00000000540a8f8b/0xa40083ab9bf536fa lrc: 4/0,1 mode: PW/PW res: [0x240000402:0x7fa:0x0].0x0 bits 0x40/0x0 rrc: 3 type: IBT gid 18446635898622804416 flags: 0x40010080000000 nid: local remote: 0x0 expref: -99 pid: 11281 timeout: 0 lvb_type: 0 00010000:00010000:2.0:1776178204.357805:0:11281:0:(ldlm_lock.c:820:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: mdt-lustre-MDT0001_UUID lock: 00000000540a8f8b/0xa40083ab9bf536fa lrc: 4/0,1 mode: PW/PW res: [0x240000402:0x7fa:0x0].0x0 bits 0x40/0x0 rrc: 3 type: IBT gid 18446635898622804416 flags: 0x50010480000000 nid: local remote: 0x0 expref: -99 pid: 11281 timeout: 0 lvb_type: 0 00010000:00010000:2.0:1776178204.357812:0:11281:0:(ldlm_lock.c:879:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: mdt-lustre-MDT0001_UUID lock: 00000000540a8f8b/0xa40083ab9bf536fa lrc: 3/0,0 mode: PW/PW res: [0x240000402:0x7fa:0x0].0x0 bits 0x40/0x0 rrc: 3 type: IBT gid 18446635898622804416 flags: 0x50010480000000 nid: local remote: 0x0 expref: -99 pid: 11281 timeout: 0 lvb_type: 0 00010000:00010000:2.0:1776178204.357837:0:11281:0:(ldlm_lockd.c:2159:__ldlm_bl_to_thread()) added 1/1 locks to regular blp list, 1 blwis in pool 00000004:00010000:2.0:1776178204.357857:0:11281:0:(mdt_handler.c:3992:mdt_save_lock()) ### save lock request 000000006342fd4b reply state 0000000039eb7ea9 transno 17179876270 ns: mdt-lustre-MDT0001_UUID lock: 00000000c9d7d9fc/0xa40083ab9bf536f3 lrc: 3/0,1 mode: EX/EX res: [0x240000402:0x7fa:0x0].0x0 bits 0x3/0x0 rrc: 3 type: IBT gid 0 flags: 0x40210000000000 nid: local remote: 0x0 expref: -99 pid: 11281 timeout: 0 lvb_type: 0 00010000:00010000:3.0:1776178204.357936:0:6244:0:(ldlm_lockd.c:2789:ldlm_bl_get_work()) Got 1 locks of 0 total in blp. (0 blwis in pool) 00010000:00010000:3.0:1776178204.357942:0:6244:0:(ldlm_lockd.c:1879:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: mdt-lustre-MDT0001_UUID lock: 00000000540a8f8b/0xa40083ab9bf536fa lrc: 2/0,0 mode: PW/PW res: [0x240000402:0x7fa:0x0].0x0 bits 0x40/0x0 rrc: 3 type: IBT gid 18446635898622804416 flags: 0x40010480000000 nid: local remote: 0x0 expref: -99 pid: 11281 timeout: 0 lvb_type: 0 00010000:00010000:2.0:1776178204.357948:0:11281:0:(ldlm_lock.c:820:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(CW) ns: mdt-lustre-MDT0001_UUID lock: 0000000073f2c7a9/0xa40083ab9bf536e5 lrc: 3/0,1 mode: CW/CW res: [0x240000402:0x7f7:0x0].0x0 bits 0x2/0x0 rrc: 3 type: IBT gid 0 flags: 0x50210000000000 nid: local remote: 0x0 expref: -99 pid: 11281 timeout: 0 lvb_type: 0 00010000:00010000:3.0:1776178204.357950:0:6244:0:(ldlm_lockd.c:1893:ldlm_handle_bl_callback()) Lock 00000000540a8f8b already unused, calling callback (0000000063c97258) 00010000:00010000:2.0:1776178204.357954:0:11281:0:(ldlm_lock.c:879:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: mdt-lustre-MDT0001_UUID lock: 0000000073f2c7a9/0xa40083ab9bf536e5 lrc: 2/0,0 mode: CW/CW res: [0x240000402:0x7f7:0x0].0x0 bits 0x2/0x0 rrc: 3 type: IBT gid 0 flags: 0x50210400000000 nid: local remote: 0x0 expref: -99 pid: 11281 timeout: 0 lvb_type: 0 00010000:00010000:3.0:1776178204.357954:0:6244:0:(ldlm_request.c:364:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: mdt-lustre-MDT0001_UUID lock: 00000000540a8f8b/0xa40083ab9bf536fa lrc: 2/0,0 mode: PW/PW res: [0x240000402:0x7fa:0x0].0x0 bits 0x40/0x0 rrc: 3 type: IBT gid 18446635898622804416 flags: 0x40010480000000 nid: local remote: 0x0 expref: -99 pid: 11281 timeout: 0 lvb_type: 0 00010000:00010000:2.0:1776178204.357960:0:11281:0:(ldlm_lockd.c:1879:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: mdt-lustre-MDT0001_UUID lock: 0000000073f2c7a9/0xa40083ab9bf536e5 lrc: 3/0,0 mode: CW/CW res: [0x240000402:0x7f7:0x0].0x0 bits 0x2/0x0 rrc: 3 type: IBT gid 0 flags: 0x40210400000000 nid: local remote: 0x0 expref: -99 pid: 11281 timeout: 0 lvb_type: 0 00010000:00010000:3.0:1776178204.357962:0:6244:0:(ldlm_request.c:1257:ldlm_cli_cancel_local()) ### server-side local cancel ns: mdt-lustre-MDT0001_UUID lock: 00000000540a8f8b/0xa40083ab9bf536fa lrc: 3/0,0 mode: PW/PW res: [0x240000402:0x7fa:0x0].0x0 bits 0x40/0x0 rrc: 3 type: IBT gid 18446635898622804416 flags: 0x40018480000000 nid: local remote: 0x0 expref: -99 pid: 11281 timeout: 0 lvb_type: 0 00010000:00010000:2.0:1776178204.357965:0:11281:0:(ldlm_lockd.c:1893:ldlm_handle_bl_callback()) Lock 0000000073f2c7a9 already unused, calling callback (0000000079e78348) 00010000:00010000:2.0:1776178204.357967:0:11281:0:(ldlm_request.c:364:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: mdt-lustre-MDT0001_UUID lock: 0000000073f2c7a9/0xa40083ab9bf536e5 lrc: 3/0,0 mode: CW/CW res: [0x240000402:0x7f7:0x0].0x0 bits 0x2/0x0 rrc: 3 type: IBT gid 0 flags: 0x40210400000000 nid: local remote: 0x0 expref: -99 pid: 11281 timeout: 0 lvb_type: 0 00010000:00010000:2.0:1776178204.357971:0:11281:0:(ldlm_request.c:1257:ldlm_cli_cancel_local()) ### server-side local cancel ns: mdt-lustre-MDT0001_UUID lock: 0000000073f2c7a9/0xa40083ab9bf536e5 lrc: 4/0,0 mode: CW/CW res: [0x240000402:0x7f7:0x0].0x0 bits 0x2/0x0 rrc: 3 type: IBT gid 0 flags: 0x40218400000000 nid: local remote: 0x0 expref: -99 pid: 11281 timeout: 0 lvb_type: 0 00010000:00010000:3.0:1776178204.357972:0:6244:0:(ldlm_inodebits.c:94:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x240000402:0x7fa:0x0].0x0 (0000000064abeb1c) 00010000:00010000:3.0:1776178204.357974:0:6244:0:(ldlm_inodebits.c:96:ldlm_reprocess_inodebits_queue()) Hint 40 00010000:00010000:3.0:1776178204.357977:0:6244:0:(ldlm_lockd.c:1903:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: mdt-lustre-MDT0001_UUID lock: 00000000540a8f8b/0xa40083ab9bf536fa lrc: 1/0,0 mode: --/PW res: [0x240000402:0x7fa:0x0].0x0 bits 0x40/0x0 rrc: 3 type: IBT gid 18446635898622804416 flags: 0x44819480000000 nid: local remote: 0x0 expref: -99 pid: 11281 timeout: 0 lvb_type: 0 00010000:00010000:2.0:1776178204.357978:0:11281:0:(ldlm_inodebits.c:94:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x240000402:0x7f7:0x0].0x0 (0000000043c2d1ef) 00010000:00010000:2.0:1776178204.357980:0:11281:0:(ldlm_inodebits.c:96:ldlm_reprocess_inodebits_queue()) Hint 2 00010000:00010000:2.0:1776178204.357982:0:11281:0:(ldlm_lockd.c:1903:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: mdt-lustre-MDT0001_UUID lock: 0000000073f2c7a9/0xa40083ab9bf536e5 lrc: 2/0,0 mode: --/CW res: [0x240000402:0x7f7:0x0].0x0 bits 0x2/0x0 rrc: 3 type: IBT gid 0 flags: 0x44a19400000000 nid: local remote: 0x0 expref: -99 pid: 11281 timeout: 0 lvb_type: 0 00010000:00010000:3.0:1776178204.357983:0:6244:0:(ldlm_lock.c:216:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: mdt-lustre-MDT0001_UUID lock: 00000000540a8f8b/0xa40083ab9bf536fa lrc: 0/0,0 mode: --/PW res: [0x240000402:0x7fa:0x0].0x0 bits 0x40/0x0 rrc: 3 type: IBT gid 18446635898622804416 flags: 0x44819480000000 nid: local remote: 0x0 expref: -99 pid: 11281 timeout: 0 lvb_type: 0 00010000:00010000:2.0:1776178204.357988:0:11281:0:(ldlm_lock.c:216:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: mdt-lustre-MDT0001_UUID lock: 0000000073f2c7a9/0xa40083ab9bf536e5 lrc: 0/0,0 mode: --/CW res: [0x240000402:0x7f7:0x0].0x0 bits 0x2/0x0 rrc: 3 type: IBT gid 0 flags: 0x44a19400000000 nid: local remote: 0x0 expref: -99 pid: 11281 timeout: 0 lvb_type: 0 00000004:00010000:2.0:1776178204.357996:0:11281:0:(mdt_handler.c:3992:mdt_save_lock()) ### save lock request 000000006342fd4b reply state 0000000039eb7ea9 transno 17179876270 ns: mdt-lustre-MDT0001_UUID lock: 0000000096c1e297/0xa40083ab9bf536ec lrc: 3/0,1 mode: PW/PW res: [0x240000402:0x7f7:0x0].0x5e5674ea bits 0x2/0x0 rrc: 2 type: IBT gid 0 flags: 0x40210000000000 nid: local remote: 0x0 expref: -99 pid: 11281 timeout: 0 lvb_type: 0 00010000:00010000:3.0:1776178204.358002:0:6244:0:(ldlm_lockd.c:2792:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00010000:00010000:3.0:1776178204.358004:0:6244:0:(ldlm_lockd.c:2792:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00010000:00010000:3.0:1776178204.358005:0:6244:0:(ldlm_lockd.c:2792:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:2.0:1776178204.358145:0:11281:0:(service.c:2372:ptlrpc_server_handle_request()) Handled RPC req@000000006342fd4b pname:cluuid+ref:pid:xid:nid:opc:job mdt00_004:b50963bd-8eba-4432-9a54-8888d00e3fa1+24:17671:x1862456979101312:12345-192.168.204.3@tcp:36:rm.0 Request processed in 1056us (4328us total) trans 17179876270 rc 0/0 00000100:00100000:2.0:1776178204.358153:0:11281:0:(nrs_fifo.c:241:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.204.3@tcp, seq: 6587 00010000:00010000:0.0:1776178204.362765:0:3340:0:(ldlm_lock.c:1064:ldlm_granted_list_add_lock()) ### About to add lock: ns: mdt-lustre-MDT0001_UUID lock: 0000000096c1e297/0xa40083ab9bf536ec lrc: 3/0,1 mode: COS/COS res: [0x240000402:0x7f7:0x0].0x5e5674ea bits 0x2/0x0 rrc: 2 type: IBT gid 0 flags: 0x50210000000000 nid: local remote: 0x0 expref: -99 pid: 11281 timeout: 0 lvb_type: 0 00010000:00010000:0.0:1776178204.362792:0:3340:0:(ldlm_inodebits.c:94:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x240000402:0x7f7:0x0].0x5e5674ea (00000000314a8d10) 00010000:00010000:0.0:1776178204.362795:0:3340:0:(ldlm_inodebits.c:96:ldlm_reprocess_inodebits_queue()) Hint 2 00010000:00010000:0.0:1776178204.362801:0:3340:0:(ldlm_lock.c:1064:ldlm_granted_list_add_lock()) ### About to add lock: ns: mdt-lustre-MDT0001_UUID lock: 00000000c9d7d9fc/0xa40083ab9bf536f3 lrc: 3/0,1 mode: COS/COS res: [0x240000402:0x7fa:0x0].0x0 bits 0x3/0x0 rrc: 2 type: IBT gid 0 flags: 0x50210000000000 nid: local remote: 0x0 expref: -99 pid: 11281 timeout: 0 lvb_type: 0 00010000:00010000:0.0:1776178204.362808:0:3340:0:(ldlm_inodebits.c:94:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x240000402:0x7fa:0x0].0x0 (0000000064abeb1c) 00010000:00010000:0.0:1776178204.362811:0:3340:0:(ldlm_inodebits.c:96:ldlm_reprocess_inodebits_queue()) Hint 3 00000100:00100000:3.0:1776178204.376626:0:3346:0:(events.c:361:request_in_callback()) peer: 12345-192.168.204.3@tcp (source: 12345-192.168.204.3@tcp) 00000100:00100000:2.0:1776178204.378441:0:11281:0:(service.c:2152:ptlrpc_server_handle_req_in()) got req x1862456979101440 00000100:00100000:3.0:1776178204.378613:0:11281:0:(nrs_fifo.c:179:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.204.3@tcp, seq: 6588 00000100:00100000:3.0:1776178204.378621:0:11281:0:(service.c:2324:ptlrpc_server_handle_request()) Handling RPC req@0000000014250674 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_004:b50963bd-8eba-4432-9a54-8888d00e3fa1+24:17671:x1862456979101440:12345-192.168.204.3@tcp:36:rm.0 00010000:00010000:3.0:1776178204.378683:0:11281:0:(ldlm_lockd.c:1720:ldlm_request_cancel()) ### server-side cancel handler START: 2 locks, starting at 0 00010000:00010000:3.0:1776178204.378721:0:11281:0:(ldlm_lock.c:216:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: mdt-lustre-MDT0001_UUID lock: 000000006cb7334a/0xa40083ab9bf53659 lrc: 0/0,0 mode: --/PR res: [0x240000402:0x7fe:0x0].0x0 bits 0x1b/0x0 rrc: 4 type: IBT gid 0 flags: 0x44a01000000000 nid: 192.168.204.3@tcp remote: 0x771f68aa1eb27b8f expref: 24 pid: 11281 timeout: 0 lvb_type: 0 00010000:00010000:3.0:1776178204.378746:0:11281:0:(ldlm_lock.c:216:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: mdt-lustre-MDT0001_UUID lock: 00000000f314daa9/0xa40083ab9bf53579 lrc: 0/0,0 mode: --/CR res: [0x240000402:0x7fe:0x0].0x0 bits 0x8/0x0 rrc: 3 type: IBT gid 0 flags: 0x44801000000000 nid: 192.168.204.3@tcp remote: 0x771f68aa1eb27b42 expref: 23 pid: 11281 timeout: 0 lvb_type: 0 00010000:00010000:3.0:1776178204.378757:0:11281:0:(ldlm_inodebits.c:94:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x240000402:0x7fe:0x0].0x0 (0000000030fbaafd) 00010000:00010000:3.0:1776178204.378768:0:11281:0:(ldlm_lockd.c:1782:ldlm_request_cancel()) ### server-side cancel handler END 00010000:00010000:3.0:1776178204.378798:0:11281:0:(ldlm_lock.c:766:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(CW) ns: mdt-lustre-MDT0001_UUID lock: 00000000014a8a18/0xa40083ab9bf53701 lrc: 3/0,1 mode: --/CW res: [0x240000402:0x7f7:0x0].0x0 bits 0x0/0x0 rrc: 3 type: IBT gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 11281 timeout: 0 lvb_type: 0 00010000:00010000:3.0:1776178204.378932:0:11281:0:(ldlm_lock.c:1064:ldlm_granted_list_add_lock()) ### About to add lock: ns: mdt-lustre-MDT0001_UUID lock: 00000000014a8a18/0xa40083ab9bf53701 lrc: 3/0,1 mode: CW/CW res: [0x240000402:0x7f7:0x0].0x0 bits 0x2/0x0 rrc: 3 type: IBT gid 0 flags: 0x50210000000000 nid: local remote: 0x0 expref: -99 pid: 11281 timeout: 0 lvb_type: 0 00010000:00010000:3.0:1776178204.378944:0:11281:0:(ldlm_request.c:528:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: mdt-lustre-MDT0001_UUID lock: 00000000014a8a18/0xa40083ab9bf53701 lrc: 3/0,1 mode: CW/CW res: [0x240000402:0x7f7:0x0].0x0 bits 0x2/0x0 rrc: 3 type: IBT gid 0 flags: 0x40210000000000 nid: local remote: 0x0 expref: -99 pid: 11281 timeout: 0 lvb_type: 0 00010000:00010000:3.0:1776178204.378961:0:11281:0:(ldlm_lock.c:766:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: mdt-lustre-MDT0001_UUID lock: 00000000c1e7ea3e/0xa40083ab9bf53708 lrc: 3/0,1 mode: --/PW res: [0x240000402:0x7f7:0x0].0x7fbd58ea bits 0x0/0x0 rrc: 2 type: IBT gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 11281 timeout: 0 lvb_type: 0 00010000:00010000:3.0:1776178204.378968:0:11281:0:(ldlm_lock.c:1064:ldlm_granted_list_add_lock()) ### About to add lock: ns: mdt-lustre-MDT0001_UUID lock: 00000000c1e7ea3e/0xa40083ab9bf53708 lrc: 3/0,1 mode: PW/PW res: [0x240000402:0x7f7:0x0].0x7fbd58ea bits 0x2/0x0 rrc: 2 type: IBT gid 0 flags: 0x50210000000000 nid: local remote: 0x0 expref: -99 pid: 11281 timeout: 0 lvb_type: 0 00010000:00010000:3.0:1776178204.378973:0:11281:0:(ldlm_request.c:528:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: mdt-lustre-MDT0001_UUID lock: 00000000c1e7ea3e/0xa40083ab9bf53708 lrc: 3/0,1 mode: PW/PW res: [0x240000402:0x7f7:0x0].0x7fbd58ea bits 0x2/0x0 rrc: 2 type: IBT gid 0 flags: 0x40210000000000 nid: local remote: 0x0 expref: -99 pid: 11281 timeout: 0 lvb_type: 0 00010000:00010000:3.0:1776178204.379413:0:11281:0:(ldlm_lock.c:766:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(EX) ns: mdt-lustre-MDT0001_UUID lock: 00000000dcea299c/0xa40083ab9bf5370f lrc: 3/0,1 mode: --/EX res: [0x240000402:0x7fe:0x0].0x0 bits 0x0/0x0 rrc: 2 type: IBT gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 11281 timeout: 0 lvb_type: 0 00010000:00010000:3.0:1776178204.379428:0:11281:0:(ldlm_lock.c:1064:ldlm_granted_list_add_lock()) ### About to add lock: ns: mdt-lustre-MDT0001_UUID lock: 00000000dcea299c/0xa40083ab9bf5370f lrc: 3/0,1 mode: EX/EX res: [0x240000402:0x7fe:0x0].0x0 bits 0x3/0x0 rrc: 2 type: IBT gid 0 flags: 0x50210000000000 nid: local remote: 0x0 expref: -99 pid: 11281 timeout: 0 lvb_type: 0 00010000:00010000:3.0:1776178204.379436:0:11281:0:(ldlm_request.c:528:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: mdt-lustre-MDT0001_UUID lock: 00000000dcea299c/0xa40083ab9bf5370f lrc: 3/0,1 mode: EX/EX res: [0x240000402:0x7fe:0x0].0x0 bits 0x3/0x0 rrc: 2 type: IBT gid 0 flags: 0x40210000000000 nid: local remote: 0x0 expref: -99 pid: 11281 timeout: 0 lvb_type: 0 00000040:00080000:3.0:1776178204.379707:0:11281:0:(llog_osd.c:738:llog_osd_write_rec()) added record [0x1:0x12:0x0].1008, 64 off72704 00010000:00010000:3.0:1776178204.379762:0:11281:0:(ldlm_lock.c:766:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: mdt-lustre-MDT0001_UUID lock: 00000000bef7c62f/0xa40083ab9bf53716 lrc: 3/0,1 mode: --/PW res: [0x240000402:0x7fe:0x0].0x0 bits 0x0/0x0 rrc: 3 type: IBT gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 11281 timeout: 0 lvb_type: 0 00010000:00010000:3.0:1776178204.379771:0:11281:0:(ldlm_lock.c:1064:ldlm_granted_list_add_lock()) ### About to add lock: ns: mdt-lustre-MDT0001_UUID lock: 00000000bef7c62f/0xa40083ab9bf53716 lrc: 3/0,1 mode: PW/PW res: [0x240000402:0x7fe:0x0].0x0 bits 0x40/0x0 rrc: 3 type: IBT gid 18446635898622804416 flags: 0x50010080000000 nid: local remote: 0x0 expref: -99 pid: 11281 timeout: 0 lvb_type: 0 00010000:00010000:3.0:1776178204.379779:0:11281:0:(ldlm_request.c:528:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: mdt-lustre-MDT0001_UUID lock: 00000000bef7c62f/0xa40083ab9bf53716 lrc: 3/0,1 mode: PW/PW res: [0x240000402:0x7fe:0x0].0x0 bits 0x40/0x0 rrc: 3 type: IBT gid 18446635898622804416 flags: 0x40010080000000 nid: local remote: 0x0 expref: -99 pid: 11281 timeout: 0 lvb_type: 0 00010000:00010000:3.0:1776178204.379787:0:11281:0:(ldlm_lock.c:942:ldlm_lock_decref_and_cancel()) ### ldlm_lock_decref(PW) ns: mdt-lustre-MDT0001_UUID lock: 00000000bef7c62f/0xa40083ab9bf53716 lrc: 4/0,1 mode: PW/PW res: [0x240000402:0x7fe:0x0].0x0 bits 0x40/0x0 rrc: 3 type: IBT gid 18446635898622804416 flags: 0x40010080000000 nid: local remote: 0x0 expref: -99 pid: 11281 timeout: 0 lvb_type: 0 00010000:00010000:3.0:1776178204.379794:0:11281:0:(ldlm_lock.c:820:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: mdt-lustre-MDT0001_UUID lock: 00000000bef7c62f/0xa40083ab9bf53716 lrc: 4/0,1 mode: PW/PW res: [0x240000402:0x7fe:0x0].0x0 bits 0x40/0x0 rrc: 3 type: IBT gid 18446635898622804416 flags: 0x50010480000000 nid: local remote: 0x0 expref: -99 pid: 11281 timeout: 0 lvb_type: 0 00010000:00010000:3.0:1776178204.379800:0:11281:0:(ldlm_lock.c:879:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: mdt-lustre-MDT0001_UUID lock: 00000000bef7c62f/0xa40083ab9bf53716 lrc: 3/0,0 mode: PW/PW res: [0x240000402:0x7fe:0x0].0x0 bits 0x40/0x0 rrc: 3 type: IBT gid 18446635898622804416 flags: 0x50010480000000 nid: local remote: 0x0 expref: -99 pid: 11281 timeout: 0 lvb_type: 0 00010000:00010000:3.0:1776178204.379974:0:11281:0:(ldlm_lockd.c:2159:__ldlm_bl_to_thread()) added 1/1 locks to regular blp list, 1 blwis in pool 00000004:00010000:3.0:1776178204.380267:0:11281:0:(mdt_handler.c:3992:mdt_save_lock()) ### save lock request 0000000014250674 reply state 0000000057b7ebbe transno 17179876271 ns: mdt-lustre-MDT0001_UUID lock: 00000000dcea299c/0xa40083ab9bf5370f lrc: 3/0,1 mode: EX/EX res: [0x240000402:0x7fe:0x0].0x0 bits 0x3/0x0 rrc: 3 type: IBT gid 0 flags: 0x40210000000000 nid: local remote: 0x0 expref: -99 pid: 11281 timeout: 0 lvb_type: 0 00010000:00010000:3.0:1776178204.380379:0:11281:0:(ldlm_lock.c:820:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(CW) ns: mdt-lustre-MDT0001_UUID lock: 00000000014a8a18/0xa40083ab9bf53701 lrc: 3/0,1 mode: CW/CW res: [0x240000402:0x7f7:0x0].0x0 bits 0x2/0x0 rrc: 3 type: IBT gid 0 flags: 0x50210000000000 nid: local remote: 0x0 expref: -99 pid: 11281 timeout: 0 lvb_type: 0 00010000:00010000:3.0:1776178204.380386:0:11281:0:(ldlm_lock.c:879:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: mdt-lustre-MDT0001_UUID lock: 00000000014a8a18/0xa40083ab9bf53701 lrc: 2/0,0 mode: CW/CW res: [0x240000402:0x7f7:0x0].0x0 bits 0x2/0x0 rrc: 3 type: IBT gid 0 flags: 0x50210400000000 nid: local remote: 0x0 expref: -99 pid: 11281 timeout: 0 lvb_type: 0 00010000:00010000:3.0:1776178204.380393:0:11281:0:(ldlm_lockd.c:1879:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: mdt-lustre-MDT0001_UUID lock: 00000000014a8a18/0xa40083ab9bf53701 lrc: 3/0,0 mode: CW/CW res: [0x240000402:0x7f7:0x0].0x0 bits 0x2/0x0 rrc: 3 type: IBT gid 0 flags: 0x40210400000000 nid: local remote: 0x0 expref: -99 pid: 11281 timeout: 0 lvb_type: 0 00010000:00010000:3.0:1776178204.380398:0:11281:0:(ldlm_lockd.c:1893:ldlm_handle_bl_callback()) Lock 00000000014a8a18 already unused, calling callback (0000000079e78348) 00010000:00010000:3.0:1776178204.380402:0:11281:0:(ldlm_request.c:364:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: mdt-lustre-MDT0001_UUID lock: 00000000014a8a18/0xa40083ab9bf53701 lrc: 3/0,0 mode: CW/CW res: [0x240000402:0x7f7:0x0].0x0 bits 0x2/0x0 rrc: 3 type: IBT gid 0 flags: 0x40210400000000 nid: local remote: 0x0 expref: -99 pid: 11281 timeout: 0 lvb_type: 0 00010000:00010000:3.0:1776178204.380408:0:11281:0:(ldlm_request.c:1257:ldlm_cli_cancel_local()) ### server-side local cancel ns: mdt-lustre-MDT0001_UUID lock: 00000000014a8a18/0xa40083ab9bf53701 lrc: 4/0,0 mode: CW/CW res: [0x240000402:0x7f7:0x0].0x0 bits 0x2/0x0 rrc: 3 type: IBT gid 0 flags: 0x40218400000000 nid: local remote: 0x0 expref: -99 pid: 11281 timeout: 0 lvb_type: 0 00010000:00010000:3.0:1776178204.380420:0:11281:0:(ldlm_inodebits.c:94:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x240000402:0x7f7:0x0].0x0 (0000000043c2d1ef) 00010000:00010000:3.0:1776178204.380423:0:11281:0:(ldlm_inodebits.c:96:ldlm_reprocess_inodebits_queue()) Hint 2 00010000:00010000:3.0:1776178204.380426:0:11281:0:(ldlm_lockd.c:1903:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: mdt-lustre-MDT0001_UUID lock: 00000000014a8a18/0xa40083ab9bf53701 lrc: 2/0,0 mode: --/CW res: [0x240000402:0x7f7:0x0].0x0 bits 0x2/0x0 rrc: 3 type: IBT gid 0 flags: 0x44a19400000000 nid: local remote: 0x0 expref: -99 pid: 11281 timeout: 0 lvb_type: 0 00010000:00010000:3.0:1776178204.380432:0:11281:0:(ldlm_lock.c:216:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: mdt-lustre-MDT0001_UUID lock: 00000000014a8a18/0xa40083ab9bf53701 lrc: 0/0,0 mode: --/CW res: [0x240000402:0x7f7:0x0].0x0 bits 0x2/0x0 rrc: 3 type: IBT gid 0 flags: 0x44a19400000000 nid: local remote: 0x0 expref: -99 pid: 11281 timeout: 0 lvb_type: 0 00000004:00010000:3.0:1776178204.380441:0:11281:0:(mdt_handler.c:3992:mdt_save_lock()) ### save lock request 0000000014250674 reply state 0000000057b7ebbe transno 17179876271 ns: mdt-lustre-MDT0001_UUID lock: 00000000c1e7ea3e/0xa40083ab9bf53708 lrc: 3/0,1 mode: PW/PW res: [0x240000402:0x7f7:0x0].0x7fbd58ea bits 0x2/0x0 rrc: 2 type: IBT gid 0 flags: 0x40210000000000 nid: local remote: 0x0 expref: -99 pid: 11281 timeout: 0 lvb_type: 0 00000100:00100000:3.0:1776178204.380540:0:11281:0:(service.c:2372:ptlrpc_server_handle_request()) Handled RPC req@0000000014250674 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_004:b50963bd-8eba-4432-9a54-8888d00e3fa1+24:17671:x1862456979101440:12345-192.168.204.3@tcp:36:rm.0 Request processed in 1918us (3921us total) trans 17179876271 rc 0/0 00000100:00100000:3.0:1776178204.380550:0:11281:0:(nrs_fifo.c:241:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.204.3@tcp, seq: 6588 00000100:00100000:3.0:1776178204.385915:0:3348:0:(events.c:361:request_in_callback()) peer: 12345-192.168.204.3@tcp (source: 12345-192.168.204.3@tcp) 00010000:00010000:2.0:1776178204.386104:0:3339:0:(ldlm_lock.c:1064:ldlm_granted_list_add_lock()) ### About to add lock: ns: mdt-lustre-MDT0001_UUID lock: 00000000c1e7ea3e/0xa40083ab9bf53708 lrc: 3/0,1 mode: COS/COS res: [0x240000402:0x7f7:0x0].0x7fbd58ea bits 0x2/0x0 rrc: 2 type: IBT gid 0 flags: 0x50210000000000 nid: local remote: 0x0 expref: -99 pid: 11281 timeout: 0 lvb_type: 0 00010000:00010000:2.0:1776178204.386122:0:3339:0:(ldlm_inodebits.c:94:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x240000402:0x7f7:0x0].0x7fbd58ea (0000000022e38283) 00010000:00010000:2.0:1776178204.386126:0:3339:0:(ldlm_inodebits.c:96:ldlm_reprocess_inodebits_queue()) Hint 2 00000100:00100000:0.0:1776178204.386134:0:11281:0:(service.c:2152:ptlrpc_server_handle_req_in()) got req x1862456979101568 00010000:00010000:2.0:1776178204.386141:0:3339:0:(ldlm_lock.c:1064:ldlm_granted_list_add_lock()) ### About to add lock: ns: mdt-lustre-MDT0001_UUID lock: 00000000dcea299c/0xa40083ab9bf5370f lrc: 3/0,1 mode: COS/COS res: [0x240000402:0x7fe:0x0].0x0 bits 0x3/0x0 rrc: 3 type: IBT gid 0 flags: 0x50210000000000 nid: local remote: 0x0 expref: -99 pid: 11281 timeout: 0 lvb_type: 0 00010000:00010000:2.0:1776178204.386149:0:3339:0:(ldlm_inodebits.c:94:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x240000402:0x7fe:0x0].0x0 (000000009f2a4d4c) 00010000:00010000:2.0:1776178204.386151:0:3339:0:(ldlm_inodebits.c:96:ldlm_reprocess_inodebits_queue()) Hint 3 00000100:00100000:0.0:1776178204.386203:0:11281:0:(nrs_fifo.c:179:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.204.3@tcp, seq: 6589 00000100:00100000:0.0:1776178204.386211:0:11281:0:(service.c:2324:ptlrpc_server_handle_request()) Handling RPC req@00000000ef862c6d pname:cluuid+ref:pid:xid:nid:opc:job mdt00_004:b50963bd-8eba-4432-9a54-8888d00e3fa1+24:17671:x1862456979101568:12345-192.168.204.3@tcp:36:rm.0 00010000:00010000:0.0:1776178204.386269:0:11281:0:(ldlm_lockd.c:1720:ldlm_request_cancel()) ### server-side cancel handler START: 2 locks, starting at 0 00010000:00010000:0.0:1776178204.386291:0:11281:0:(ldlm_lock.c:216:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: mdt-lustre-MDT0001_UUID lock: 0000000025b13886/0xa40083ab9bf53621 lrc: 0/0,0 mode: --/PR res: [0x240000402:0x7fc:0x0].0x0 bits 0x1b/0x0 rrc: 4 type: IBT gid 0 flags: 0x44a01000000000 nid: 192.168.204.3@tcp remote: 0x771f68aa1eb27b81 expref: 24 pid: 6257 timeout: 0 lvb_type: 0 00010000:00010000:0.0:1776178204.386311:0:11281:0:(ldlm_lock.c:216:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: mdt-lustre-MDT0001_UUID lock: 00000000fcb32840/0xa40083ab9bf53580 lrc: 0/0,0 mode: --/CR res: [0x240000402:0x7fc:0x0].0x0 bits 0x8/0x0 rrc: 3 type: IBT gid 0 flags: 0x44801000000000 nid: 192.168.204.3@tcp remote: 0x771f68aa1eb27b50 expref: 23 pid: 6256 timeout: 0 lvb_type: 0 00010000:00010000:0.0:1776178204.386319:0:11281:0:(ldlm_inodebits.c:94:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x240000402:0x7fc:0x0].0x0 (00000000c31449d9) 00010000:00010000:0.0:1776178204.386326:0:11281:0:(ldlm_lockd.c:1782:ldlm_request_cancel()) ### server-side cancel handler END 00010000:00010000:0.0:1776178204.386354:0:11281:0:(ldlm_lock.c:766:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(CW) ns: mdt-lustre-MDT0001_UUID lock: 00000000a211f94c/0xa40083ab9bf5371d lrc: 3/0,1 mode: --/CW res: [0x240000402:0x7f7:0x0].0x0 bits 0x0/0x0 rrc: 3 type: IBT gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 11281 timeout: 0 lvb_type: 0 00010000:00010000:0.0:1776178204.386367:0:11281:0:(ldlm_lock.c:1064:ldlm_granted_list_add_lock()) ### About to add lock: ns: mdt-lustre-MDT0001_UUID lock: 00000000a211f94c/0xa40083ab9bf5371d lrc: 3/0,1 mode: CW/CW res: [0x240000402:0x7f7:0x0].0x0 bits 0x2/0x0 rrc: 3 type: IBT gid 0 flags: 0x50210000000000 nid: local remote: 0x0 expref: -99 pid: 11281 timeout: 0 lvb_type: 0 00010000:00010000:0.0:1776178204.386376:0:11281:0:(ldlm_request.c:528:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: mdt-lustre-MDT0001_UUID lock: 00000000a211f94c/0xa40083ab9bf5371d lrc: 3/0,1 mode: CW/CW res: [0x240000402:0x7f7:0x0].0x0 bits 0x2/0x0 rrc: 3 type: IBT gid 0 flags: 0x40210000000000 nid: local remote: 0x0 expref: -99 pid: 11281 timeout: 0 lvb_type: 0 00010000:00010000:0.0:1776178204.386394:0:11281:0:(ldlm_lock.c:766:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: mdt-lustre-MDT0001_UUID lock: 0000000011d684d8/0xa40083ab9bf53724 lrc: 3/0,1 mode: --/PW res: [0x240000402:0x7f7:0x0].0xef09e6ea bits 0x0/0x0 rrc: 2 type: IBT gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 11281 timeout: 0 lvb_type: 0 00010000:00010000:0.0:1776178204.386401:0:11281:0:(ldlm_lock.c:1064:ldlm_granted_list_add_lock()) ### About to add lock: ns: mdt-lustre-MDT0001_UUID lock: 0000000011d684d8/0xa40083ab9bf53724 lrc: 3/0,1 mode: PW/PW res: [0x240000402:0x7f7:0x0].0xef09e6ea bits 0x2/0x0 rrc: 2 type: IBT gid 0 flags: 0x50210000000000 nid: local remote: 0x0 expref: -99 pid: 11281 timeout: 0 lvb_type: 0 00010000:00010000:0.0:1776178204.386407:0:11281:0:(ldlm_request.c:528:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: mdt-lustre-MDT0001_UUID lock: 0000000011d684d8/0xa40083ab9bf53724 lrc: 3/0,1 mode: PW/PW res: [0x240000402:0x7f7:0x0].0xef09e6ea bits 0x2/0x0 rrc: 2 type: IBT gid 0 flags: 0x40210000000000 nid: local remote: 0x0 expref: -99 pid: 11281 timeout: 0 lvb_type: 0 00010000:00010000:0.0:1776178204.386493:0:11281:0:(ldlm_lock.c:766:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(EX) ns: mdt-lustre-MDT0001_UUID lock: 000000008813523d/0xa40083ab9bf5372b lrc: 3/0,1 mode: --/EX res: [0x240000402:0x7fc:0x0].0x0 bits 0x0/0x0 rrc: 2 type: IBT gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 11281 timeout: 0 lvb_type: 0 00010000:00010000:0.0:1776178204.386502:0:11281:0:(ldlm_lock.c:1064:ldlm_granted_list_add_lock()) ### About to add lock: ns: mdt-lustre-MDT0001_UUID lock: 000000008813523d/0xa40083ab9bf5372b lrc: 3/0,1 mode: EX/EX res: [0x240000402:0x7fc:0x0].0x0 bits 0x3/0x0 rrc: 2 type: IBT gid 0 flags: 0x50210000000000 nid: local remote: 0x0 expref: -99 pid: 11281 timeout: 0 lvb_type: 0 00010000:00010000:0.0:1776178204.386508:0:11281:0:(ldlm_request.c:528:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: mdt-lustre-MDT0001_UUID lock: 000000008813523d/0xa40083ab9bf5372b lrc: 3/0,1 mode: EX/EX res: [0x240000402:0x7fc:0x0].0x0 bits 0x3/0x0 rrc: 2 type: IBT gid 0 flags: 0x40210000000000 nid: local remote: 0x0 expref: -99 pid: 11281 timeout: 0 lvb_type: 0 00000040:00080000:0.0:1776178204.386768:0:11281:0:(llog_osd.c:738:llog_osd_write_rec()) added record [0x1:0x12:0x0].1009, 64 off72768 00010000:00010000:0.0:1776178204.386817:0:11281:0:(ldlm_lock.c:766:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: mdt-lustre-MDT0001_UUID lock: 0000000068850c85/0xa40083ab9bf53732 lrc: 3/0,1 mode: --/PW res: [0x240000402:0x7fc:0x0].0x0 bits 0x0/0x0 rrc: 3 type: IBT gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 11281 timeout: 0 lvb_type: 0 00010000:00010000:0.0:1776178204.386827:0:11281:0:(ldlm_lock.c:1064:ldlm_granted_list_add_lock()) ### About to add lock: ns: mdt-lustre-MDT0001_UUID lock: 0000000068850c85/0xa40083ab9bf53732 lrc: 3/0,1 mode: PW/PW res: [0x240000402:0x7fc:0x0].0x0 bits 0x40/0x0 rrc: 3 type: IBT gid 18446635898622804416 flags: 0x50010080000000 nid: local remote: 0x0 expref: -99 pid: 11281 timeout: 0 lvb_type: 0 00010000:00010000:0.0:1776178204.386836:0:11281:0:(ldlm_request.c:528:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: mdt-lustre-MDT0001_UUID lock: 0000000068850c85/0xa40083ab9bf53732 lrc: 3/0,1 mode: PW/PW res: [0x240000402:0x7fc:0x0].0x0 bits 0x40/0x0 rrc: 3 type: IBT gid 18446635898622804416 flags: 0x40010080000000 nid: local remote: 0x0 expref: -99 pid: 11281 timeout: 0 lvb_type: 0 00010000:00010000:0.0:1776178204.386844:0:11281:0:(ldlm_lock.c:942:ldlm_lock_decref_and_cancel()) ### ldlm_lock_decref(PW) ns: mdt-lustre-MDT0001_UUID lock: 0000000068850c85/0xa40083ab9bf53732 lrc: 4/0,1 mode: PW/PW res: [0x240000402:0x7fc:0x0].0x0 bits 0x40/0x0 rrc: 3 type: IBT gid 18446635898622804416 flags: 0x40010080000000 nid: local remote: 0x0 expref: -99 pid: 11281 timeout: 0 lvb_type: 0 00010000:00010000:0.0:1776178204.386852:0:11281:0:(ldlm_lock.c:820:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: mdt-lustre-MDT0001_UUID lock: 0000000068850c85/0xa40083ab9bf53732 lrc: 4/0,1 mode: PW/PW res: [0x240000402:0x7fc:0x0].0x0 bits 0x40/0x0 rrc: 3 type: IBT gid 18446635898622804416 flags: 0x50010480000000 nid: local remote: 0x0 expref: -99 pid: 11281 timeout: 0 lvb_type: 0 00010000:00010000:0.0:1776178204.386859:0:11281:0:(ldlm_lock.c:879:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: mdt-lustre-MDT0001_UUID lock: 0000000068850c85/0xa40083ab9bf53732 lrc: 3/0,0 mode: PW/PW res: [0x240000402:0x7fc:0x0].0x0 bits 0x40/0x0 rrc: 3 type: IBT gid 18446635898622804416 flags: 0x50010480000000 nid: local remote: 0x0 expref: -99 pid: 11281 timeout: 0 lvb_type: 0 00010000:00010000:0.0:1776178204.386886:0:11281:0:(ldlm_lockd.c:2159:__ldlm_bl_to_thread()) added 1/2 locks to regular blp list, 2 blwis in pool 00000004:00010000:0.0:1776178204.386904:0:11281:0:(mdt_handler.c:3992:mdt_save_lock()) ### save lock request 00000000ef862c6d reply state 0000000026cddacf transno 17179876272 ns: mdt-lustre-MDT0001_UUID lock: 000000008813523d/0xa40083ab9bf5372b lrc: 3/0,1 mode: EX/EX res: [0x240000402:0x7fc:0x0].0x0 bits 0x3/0x0 rrc: 3 type: IBT gid 0 flags: 0x40210000000000 nid: local remote: 0x0 expref: -99 pid: 11281 timeout: 0 lvb_type: 0 00010000:00010000:2.0:1776178204.386932:0:9224:0:(ldlm_lockd.c:2789:ldlm_bl_get_work()) Got 1 locks of 1 total in blp. (1 blwis in pool) 00010000:00010000:2.0:1776178204.386940:0:9224:0:(ldlm_lockd.c:1879:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: mdt-lustre-MDT0001_UUID lock: 00000000bef7c62f/0xa40083ab9bf53716 lrc: 2/0,0 mode: PW/PW res: [0x240000402:0x7fe:0x0].0x0 bits 0x40/0x0 rrc: 3 type: IBT gid 18446635898622804416 flags: 0x40010480000000 nid: local remote: 0x0 expref: -99 pid: 11281 timeout: 0 lvb_type: 0 00010000:00010000:2.0:1776178204.386949:0:9224:0:(ldlm_lockd.c:1893:ldlm_handle_bl_callback()) Lock 00000000bef7c62f already unused, calling callback (0000000063c97258) 00010000:00010000:2.0:1776178204.386952:0:9224:0:(ldlm_request.c:364:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: mdt-lustre-MDT0001_UUID lock: 00000000bef7c62f/0xa40083ab9bf53716 lrc: 2/0,0 mode: PW/PW res: [0x240000402:0x7fe:0x0].0x0 bits 0x40/0x0 rrc: 3 type: IBT gid 18446635898622804416 flags: 0x40010480000000 nid: local remote: 0x0 expref: -99 pid: 11281 timeout: 0 lvb_type: 0 00010000:00010000:2.0:1776178204.386961:0:9224:0:(ldlm_request.c:1257:ldlm_cli_cancel_local()) ### server-side local cancel ns: mdt-lustre-MDT0001_UUID lock: 00000000bef7c62f/0xa40083ab9bf53716 lrc: 3/0,0 mode: PW/PW res: [0x240000402:0x7fe:0x0].0x0 bits 0x40/0x0 rrc: 3 type: IBT gid 18446635898622804416 flags: 0x40018480000000 nid: local remote: 0x0 expref: -99 pid: 11281 timeout: 0 lvb_type: 0 00010000:00010000:2.0:1776178204.386974:0:9224:0:(ldlm_inodebits.c:94:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x240000402:0x7fe:0x0].0x0 (000000009f2a4d4c) 00010000:00010000:2.0:1776178204.386976:0:9224:0:(ldlm_inodebits.c:96:ldlm_reprocess_inodebits_queue()) Hint 40 00010000:00010000:2.0:1776178204.386981:0:9224:0:(ldlm_lockd.c:1903:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: mdt-lustre-MDT0001_UUID lock: 00000000bef7c62f/0xa40083ab9bf53716 lrc: 1/0,0 mode: --/PW res: [0x240000402:0x7fe:0x0].0x0 bits 0x40/0x0 rrc: 3 type: IBT gid 18446635898622804416 flags: 0x44819480000000 nid: local remote: 0x0 expref: -99 pid: 11281 timeout: 0 lvb_type: 0 00010000:00010000:2.0:1776178204.386988:0:9224:0:(ldlm_lock.c:216:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: mdt-lustre-MDT0001_UUID lock: 00000000bef7c62f/0xa40083ab9bf53716 lrc: 0/0,0 mode: --/PW res: [0x240000402:0x7fe:0x0].0x0 bits 0x40/0x0 rrc: 3 type: IBT gid 18446635898622804416 flags: 0x44819480000000 nid: local remote: 0x0 expref: -99 pid: 11281 timeout: 0 lvb_type: 0 00010000:00010000:0.0:1776178204.387015:0:11281:0:(ldlm_lock.c:820:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(CW) ns: mdt-lustre-MDT0001_UUID lock: 00000000a211f94c/0xa40083ab9bf5371d lrc: 3/0,1 mode: CW/CW res: [0x240000402:0x7f7:0x0].0x0 bits 0x2/0x0 rrc: 3 type: IBT gid 0 flags: 0x50210000000000 nid: local remote: 0x0 expref: -99 pid: 11281 timeout: 0 lvb_type: 0 00010000:00010000:2.0:1776178204.387023:0:9224:0:(ldlm_lockd.c:2789:ldlm_bl_get_work()) Got 1 locks of 0 total in blp. (0 blwis in pool) 00010000:00010000:0.0:1776178204.387025:0:11281:0:(ldlm_lock.c:879:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: mdt-lustre-MDT0001_UUID lock: 00000000a211f94c/0xa40083ab9bf5371d lrc: 2/0,0 mode: CW/CW res: [0x240000402:0x7f7:0x0].0x0 bits 0x2/0x0 rrc: 3 type: IBT gid 0 flags: 0x50210400000000 nid: local remote: 0x0 expref: -99 pid: 11281 timeout: 0 lvb_type: 0 00010000:00010000:2.0:1776178204.387027:0:9224:0:(ldlm_lockd.c:1879:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: mdt-lustre-MDT0001_UUID lock: 0000000068850c85/0xa40083ab9bf53732 lrc: 2/0,0 mode: PW/PW res: [0x240000402:0x7fc:0x0].0x0 bits 0x40/0x0 rrc: 3 type: IBT gid 18446635898622804416 flags: 0x40010480000000 nid: local remote: 0x0 expref: -99 pid: 11281 timeout: 0 lvb_type: 0 00010000:00010000:0.0:1776178204.387037:0:11281:0:(ldlm_lockd.c:1879:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: mdt-lustre-MDT0001_UUID lock: 00000000a211f94c/0xa40083ab9bf5371d lrc: 3/0,0 mode: CW/CW res: [0x240000402:0x7f7:0x0].0x0 bits 0x2/0x0 rrc: 3 type: IBT gid 0 flags: 0x40210400000000 nid: local remote: 0x0 expref: -99 pid: 11281 timeout: 0 lvb_type: 0 00010000:00010000:2.0:1776178204.387040:0:9224:0:(ldlm_lockd.c:1893:ldlm_handle_bl_callback()) Lock 0000000068850c85 already unused, calling callback (0000000063c97258) 00010000:00010000:2.0:1776178204.387044:0:9224:0:(ldlm_request.c:364:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: mdt-lustre-MDT0001_UUID lock: 0000000068850c85/0xa40083ab9bf53732 lrc: 2/0,0 mode: PW/PW res: [0x240000402:0x7fc:0x0].0x0 bits 0x40/0x0 rrc: 3 type: IBT gid 18446635898622804416 flags: 0x40010480000000 nid: local remote: 0x0 expref: -99 pid: 11281 timeout: 0 lvb_type: 0 00010000:00010000:0.0:1776178204.387048:0:11281:0:(ldlm_lockd.c:1893:ldlm_handle_bl_callback()) Lock 00000000a211f94c already unused, calling callback (0000000079e78348) 00010000:00010000:0.0:1776178204.387053:0:11281:0:(ldlm_request.c:364:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: mdt-lustre-MDT0001_UUID lock: 00000000a211f94c/0xa40083ab9bf5371d lrc: 3/0,0 mode: CW/CW res: [0x240000402:0x7f7:0x0].0x0 bits 0x2/0x0 rrc: 3 type: IBT gid 0 flags: 0x40210400000000 nid: local remote: 0x0 expref: -99 pid: 11281 timeout: 0 lvb_type: 0 00010000:00010000:2.0:1776178204.387055:0:9224:0:(ldlm_request.c:1257:ldlm_cli_cancel_local()) ### server-side local cancel ns: mdt-lustre-MDT0001_UUID lock: 0000000068850c85/0xa40083ab9bf53732 lrc: 3/0,0 mode: PW/PW res: [0x240000402:0x7fc:0x0].0x0 bits 0x40/0x0 rrc: 3 type: IBT gid 18446635898622804416 flags: 0x40018480000000 nid: local remote: 0x0 expref: -99 pid: 11281 timeout: 0 lvb_type: 0 00010000:00010000:0.0:1776178204.387082:0:11281:0:(ldlm_request.c:1257:ldlm_cli_cancel_local()) ### server-side local cancel ns: mdt-lustre-MDT0001_UUID lock: 00000000a211f94c/0xa40083ab9bf5371d lrc: 4/0,0 mode: CW/CW res: [0x240000402:0x7f7:0x0].0x0 bits 0x2/0x0 rrc: 3 type: IBT gid 0 flags: 0x40218400000000 nid: local remote: 0x0 expref: -99 pid: 11281 timeout: 0 lvb_type: 0 00010000:00010000:0.0:1776178204.387091:0:11281:0:(ldlm_inodebits.c:94:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x240000402:0x7f7:0x0].0x0 (0000000043c2d1ef) 00010000:00010000:0.0:1776178204.387095:0:11281:0:(ldlm_inodebits.c:96:ldlm_reprocess_inodebits_queue()) Hint 2 00010000:00010000:0.0:1776178204.387098:0:11281:0:(ldlm_lockd.c:1903:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: mdt-lustre-MDT0001_UUID lock: 00000000a211f94c/0xa40083ab9bf5371d lrc: 2/0,0 mode: --/CW res: [0x240000402:0x7f7:0x0].0x0 bits 0x2/0x0 rrc: 3 type: IBT gid 0 flags: 0x44a19400000000 nid: local remote: 0x0 expref: -99 pid: 11281 timeout: 0 lvb_type: 0 00010000:00010000:2.0:1776178204.387099:0:9224:0:(ldlm_inodebits.c:94:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x240000402:0x7fc:0x0].0x0 (00000000abeb6ce0) 00010000:00010000:2.0:1776178204.387104:0:9224:0:(ldlm_inodebits.c:96:ldlm_reprocess_inodebits_queue()) Hint 40 00010000:00010000:2.0:1776178204.387106:0:9224:0:(ldlm_lockd.c:1903:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: mdt-lustre-MDT0001_UUID lock: 0000000068850c85/0xa40083ab9bf53732 lrc: 1/0,0 mode: --/PW res: [0x240000402:0x7fc:0x0].0x0 bits 0x40/0x0 rrc: 3 type: IBT gid 18446635898622804416 flags: 0x44819480000000 nid: local remote: 0x0 expref: -99 pid: 11281 timeout: 0 lvb_type: 0 00010000:00010000:0.0:1776178204.387109:0:11281:0:(ldlm_lock.c:216:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: mdt-lustre-MDT0001_UUID lock: 00000000a211f94c/0xa40083ab9bf5371d lrc: 0/0,0 mode: --/CW res: [0x240000402:0x7f7:0x0].0x0 bits 0x2/0x0 rrc: 3 type: IBT gid 0 flags: 0x44a19400000000 nid: local remote: 0x0 expref: -99 pid: 11281 timeout: 0 lvb_type: 0 00010000:00010000:2.0:1776178204.387119:0:9224:0:(ldlm_lock.c:216:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: mdt-lustre-MDT0001_UUID lock: 0000000068850c85/0xa40083ab9bf53732 lrc: 0/0,0 mode: --/PW res: [0x240000402:0x7fc:0x0].0x0 bits 0x40/0x0 rrc: 3 type: IBT gid 18446635898622804416 flags: 0x44819480000000 nid: local remote: 0x0 expref: -99 pid: 11281 timeout: 0 lvb_type: 0 00000004:00010000:0.0:1776178204.387125:0:11281:0:(mdt_handler.c:3992:mdt_save_lock()) ### save lock request 00000000ef862c6d reply state 0000000026cddacf transno 17179876272 ns: mdt-lustre-MDT0001_UUID lock: 0000000011d684d8/0xa40083ab9bf53724 lrc: 3/0,1 mode: PW/PW res: [0x240000402:0x7f7:0x0].0xef09e6ea bits 0x2/0x0 rrc: 2 type: IBT gid 0 flags: 0x40210000000000 nid: local remote: 0x0 expref: -99 pid: 11281 timeout: 0 lvb_type: 0 00000100:00100000:0.0:1776178204.387203:0:11281:0:(service.c:2372:ptlrpc_server_handle_request()) Handled RPC req@00000000ef862c6d pname:cluuid+ref:pid:xid:nid:opc:job mdt00_004:b50963bd-8eba-4432-9a54-8888d00e3fa1+24:17671:x1862456979101568:12345-192.168.204.3@tcp:36:rm.0 Request processed in 991us (1297us total) trans 17179876272 rc 0/0 00000100:00100000:0.0:1776178204.387212:0:11281:0:(nrs_fifo.c:241:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.204.3@tcp, seq: 6589 00010000:00010000:2.0:1776178204.387351:0:9224:0:(ldlm_lockd.c:2792:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00010000:00010000:2.0:1776178204.387354:0:9224:0:(ldlm_lockd.c:2792:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00010000:00010000:2.0:1776178204.387355:0:9224:0:(ldlm_lockd.c:2792:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00010000:00010000:1.0:1776178204.388369:0:9225:0:(ldlm_lockd.c:2792:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:2.0:1776178204.390916:0:3347:0:(events.c:361:request_in_callback()) peer: 12345-192.168.204.3@tcp (source: 12345-192.168.204.3@tcp) 00000100:00100000:3.0:1776178204.391386:0:11281:0:(service.c:2152:ptlrpc_server_handle_req_in()) got req x1862456979101696 00000100:00100000:3.0:1776178204.391444:0:11281:0:(nrs_fifo.c:179:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.204.3@tcp, seq: 6590 00000100:00100000:3.0:1776178204.391451:0:11281:0:(service.c:2324:ptlrpc_server_handle_request()) Handling RPC req@00000000889a4355 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_004:b50963bd-8eba-4432-9a54-8888d00e3fa1+24:17671:x1862456979101696:12345-192.168.204.3@tcp:36:rm.0 00010000:00010000:3.0:1776178204.391511:0:11281:0:(ldlm_lockd.c:1720:ldlm_request_cancel()) ### server-side cancel handler START: 2 locks, starting at 0 00010000:00010000:3.0:1776178204.391544:0:11281:0:(ldlm_lock.c:216:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: mdt-lustre-MDT0001_UUID lock: 000000002a990613/0xa40083ab9bf535cd lrc: 0/0,0 mode: --/PR res: [0x240000402:0x7f9:0x0].0x0 bits 0x1b/0x0 rrc: 4 type: IBT gid 0 flags: 0x44a01000000000 nid: 192.168.204.3@tcp remote: 0x771f68aa1eb27b6c expref: 24 pid: 11281 timeout: 0 lvb_type: 0 00010000:00010000:3.0:1776178204.391569:0:11281:0:(ldlm_lock.c:216:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: mdt-lustre-MDT0001_UUID lock: 000000006844664c/0xa40083ab9bf53572 lrc: 0/0,0 mode: --/CR res: [0x240000402:0x7f9:0x0].0x0 bits 0x8/0x0 rrc: 3 type: IBT gid 0 flags: 0x44801000000000 nid: 192.168.204.3@tcp remote: 0x771f68aa1eb27b3b expref: 23 pid: 11281 timeout: 0 lvb_type: 0 00010000:00010000:3.0:1776178204.391580:0:11281:0:(ldlm_inodebits.c:94:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x240000402:0x7f9:0x0].0x0 (0000000056a46617) 00010000:00010000:3.0:1776178204.391590:0:11281:0:(ldlm_lockd.c:1782:ldlm_request_cancel()) ### server-side cancel handler END 00010000:00010000:3.0:1776178204.391621:0:11281:0:(ldlm_lock.c:766:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(CW) ns: mdt-lustre-MDT0001_UUID lock: 000000006cb7334a/0xa40083ab9bf53739 lrc: 3/0,1 mode: --/CW res: [0x240000402:0x7f7:0x0].0x0 bits 0x0/0x0 rrc: 3 type: IBT gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 11281 timeout: 0 lvb_type: 0 00010000:00010000:3.0:1776178204.391637:0:11281:0:(ldlm_lock.c:1064:ldlm_granted_list_add_lock()) ### About to add lock: ns: mdt-lustre-MDT0001_UUID lock: 000000006cb7334a/0xa40083ab9bf53739 lrc: 3/0,1 mode: CW/CW res: [0x240000402:0x7f7:0x0].0x0 bits 0x2/0x0 rrc: 3 type: IBT gid 0 flags: 0x50210000000000 nid: local remote: 0x0 expref: -99 pid: 11281 timeout: 0 lvb_type: 0 00010000:00010000:3.0:1776178204.391647:0:11281:0:(ldlm_request.c:528:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: mdt-lustre-MDT0001_UUID lock: 000000006cb7334a/0xa40083ab9bf53739 lrc: 3/0,1 mode: CW/CW res: [0x240000402:0x7f7:0x0].0x0 bits 0x2/0x0 rrc: 3 type: IBT gid 0 flags: 0x40210000000000 nid: local remote: 0x0 expref: -99 pid: 11281 timeout: 0 lvb_type: 0 00010000:00010000:3.0:1776178204.391661:0:11281:0:(ldlm_lock.c:766:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: mdt-lustre-MDT0001_UUID lock: 00000000b335466f/0xa40083ab9bf53740 lrc: 3/0,1 mode: --/PW res: [0x240000402:0x7f7:0x0].0x15fcbbea bits 0x0/0x0 rrc: 2 type: IBT gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 11281 timeout: 0 lvb_type: 0 00010000:00010000:3.0:1776178204.391668:0:11281:0:(ldlm_lock.c:1064:ldlm_granted_list_add_lock()) ### About to add lock: ns: mdt-lustre-MDT0001_UUID lock: 00000000b335466f/0xa40083ab9bf53740 lrc: 3/0,1 mode: PW/PW res: [0x240000402:0x7f7:0x0].0x15fcbbea bits 0x2/0x0 rrc: 2 type: IBT gid 0 flags: 0x50210000000000 nid: local remote: 0x0 expref: -99 pid: 11281 timeout: 0 lvb_type: 0 00010000:00010000:3.0:1776178204.391678:0:11281:0:(ldlm_request.c:528:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: mdt-lustre-MDT0001_UUID lock: 00000000b335466f/0xa40083ab9bf53740 lrc: 3/0,1 mode: PW/PW res: [0x240000402:0x7f7:0x0].0x15fcbbea bits 0x2/0x0 rrc: 2 type: IBT gid 0 flags: 0x40210000000000 nid: local remote: 0x0 expref: -99 pid: 11281 timeout: 0 lvb_type: 0 00010000:00010000:3.0:1776178204.391772:0:11281:0:(ldlm_lock.c:766:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(EX) ns: mdt-lustre-MDT0001_UUID lock: 00000000575572c2/0xa40083ab9bf53747 lrc: 3/0,1 mode: --/EX res: [0x240000402:0x7f9:0x0].0x0 bits 0x0/0x0 rrc: 2 type: IBT gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 11281 timeout: 0 lvb_type: 0 00010000:00010000:3.0:1776178204.391781:0:11281:0:(ldlm_lock.c:1064:ldlm_granted_list_add_lock()) ### About to add lock: ns: mdt-lustre-MDT0001_UUID lock: 00000000575572c2/0xa40083ab9bf53747 lrc: 3/0,1 mode: EX/EX res: [0x240000402:0x7f9:0x0].0x0 bits 0x3/0x0 rrc: 2 type: IBT gid 0 flags: 0x50210000000000 nid: local remote: 0x0 expref: -99 pid: 11281 timeout: 0 lvb_type: 0 00010000:00010000:3.0:1776178204.391787:0:11281:0:(ldlm_request.c:528:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: mdt-lustre-MDT0001_UUID lock: 00000000575572c2/0xa40083ab9bf53747 lrc: 3/0,1 mode: EX/EX res: [0x240000402:0x7f9:0x0].0x0 bits 0x3/0x0 rrc: 2 type: IBT gid 0 flags: 0x40210000000000 nid: local remote: 0x0 expref: -99 pid: 11281 timeout: 0 lvb_type: 0 00000040:00080000:3.0:1776178204.392686:0:11281:0:(llog_osd.c:738:llog_osd_write_rec()) added record [0x1:0x10:0x0].1008, 64 off72704 00010000:00010000:3.0:1776178204.392759:0:11281:0:(ldlm_lock.c:766:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: mdt-lustre-MDT0001_UUID lock: 0000000068a2dd4a/0xa40083ab9bf5374e lrc: 3/0,1 mode: --/PW res: [0x240000402:0x7f9:0x0].0x0 bits 0x0/0x0 rrc: 3 type: IBT gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 11281 timeout: 0 lvb_type: 0 00010000:00010000:3.0:1776178204.392772:0:11281:0:(ldlm_lock.c:1064:ldlm_granted_list_add_lock()) ### About to add lock: ns: mdt-lustre-MDT0001_UUID lock: 0000000068a2dd4a/0xa40083ab9bf5374e lrc: 3/0,1 mode: PW/PW res: [0x240000402:0x7f9:0x0].0x0 bits 0x40/0x0 rrc: 3 type: IBT gid 18446635898622804416 flags: 0x50010080000000 nid: local remote: 0x0 expref: -99 pid: 11281 timeout: 0 lvb_type: 0 00010000:00010000:3.0:1776178204.392781:0:11281:0:(ldlm_request.c:528:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: mdt-lustre-MDT0001_UUID lock: 0000000068a2dd4a/0xa40083ab9bf5374e lrc: 3/0,1 mode: PW/PW res: [0x240000402:0x7f9:0x0].0x0 bits 0x40/0x0 rrc: 3 type: IBT gid 18446635898622804416 flags: 0x40010080000000 nid: local remote: 0x0 expref: -99 pid: 11281 timeout: 0 lvb_type: 0 00010000:00010000:3.0:1776178204.392789:0:11281:0:(ldlm_lock.c:942:ldlm_lock_decref_and_cancel()) ### ldlm_lock_decref(PW) ns: mdt-lustre-MDT0001_UUID lock: 0000000068a2dd4a/0xa40083ab9bf5374e lrc: 4/0,1 mode: PW/PW res: [0x240000402:0x7f9:0x0].0x0 bits 0x40/0x0 rrc: 3 type: IBT gid 18446635898622804416 flags: 0x40010080000000 nid: local remote: 0x0 expref: -99 pid: 11281 timeout: 0 lvb_type: 0 00010000:00010000:3.0:1776178204.392795:0:11281:0:(ldlm_lock.c:820:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: mdt-lustre-MDT0001_UUID lock: 0000000068a2dd4a/0xa40083ab9bf5374e lrc: 4/0,1 mode: PW/PW res: [0x240000402:0x7f9:0x0].0x0 bits 0x40/0x0 rrc: 3 type: IBT gid 18446635898622804416 flags: 0x50010480000000 nid: local remote: 0x0 expref: -99 pid: 11281 timeout: 0 lvb_type: 0 00010000:00010000:3.0:1776178204.392800:0:11281:0:(ldlm_lock.c:879:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: mdt-lustre-MDT0001_UUID lock: 0000000068a2dd4a/0xa40083ab9bf5374e lrc: 3/0,0 mode: PW/PW res: [0x240000402:0x7f9:0x0].0x0 bits 0x40/0x0 rrc: 3 type: IBT gid 18446635898622804416 flags: 0x50010480000000 nid: local remote: 0x0 expref: -99 pid: 11281 timeout: 0 lvb_type: 0 00010000:00010000:3.0:1776178204.392815:0:11281:0:(ldlm_lockd.c:2159:__ldlm_bl_to_thread()) added 1/1 locks to regular blp list, 1 blwis in pool 00000004:00010000:3.0:1776178204.392831:0:11281:0:(mdt_handler.c:3992:mdt_save_lock()) ### save lock request 00000000889a4355 reply state 00000000d1d6cc04 transno 17179876273 ns: mdt-lustre-MDT0001_UUID lock: 00000000575572c2/0xa40083ab9bf53747 lrc: 3/0,1 mode: EX/EX res: [0x240000402:0x7f9:0x0].0x0 bits 0x3/0x0 rrc: 3 type: IBT gid 0 flags: 0x40210000000000 nid: local remote: 0x0 expref: -99 pid: 11281 timeout: 0 lvb_type: 0 00010000:00010000:3.0:1776178204.392926:0:11281:0:(ldlm_lock.c:820:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(CW) ns: mdt-lustre-MDT0001_UUID lock: 000000006cb7334a/0xa40083ab9bf53739 lrc: 3/0,1 mode: CW/CW res: [0x240000402:0x7f7:0x0].0x0 bits 0x2/0x0 rrc: 3 type: IBT gid 0 flags: 0x50210000000000 nid: local remote: 0x0 expref: -99 pid: 11281 timeout: 0 lvb_type: 0 00010000:00010000:3.0:1776178204.392933:0:11281:0:(ldlm_lock.c:879:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: mdt-lustre-MDT0001_UUID lock: 000000006cb7334a/0xa40083ab9bf53739 lrc: 2/0,0 mode: CW/CW res: [0x240000402:0x7f7:0x0].0x0 bits 0x2/0x0 rrc: 3 type: IBT gid 0 flags: 0x50210400000000 nid: local remote: 0x0 expref: -99 pid: 11281 timeout: 0 lvb_type: 0 00010000:00010000:3.0:1776178204.392939:0:11281:0:(ldlm_lockd.c:1879:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: mdt-lustre-MDT0001_UUID lock: 000000006cb7334a/0xa40083ab9bf53739 lrc: 3/0,0 mode: CW/CW res: [0x240000402:0x7f7:0x0].0x0 bits 0x2/0x0 rrc: 3 type: IBT gid 0 flags: 0x40210400000000 nid: local remote: 0x0 expref: -99 pid: 11281 timeout: 0 lvb_type: 0 00010000:00010000:3.0:1776178204.392945:0:11281:0:(ldlm_lockd.c:1893:ldlm_handle_bl_callback()) Lock 000000006cb7334a already unused, calling callback (0000000079e78348) 00010000:00010000:3.0:1776178204.392949:0:11281:0:(ldlm_request.c:364:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: mdt-lustre-MDT0001_UUID lock: 000000006cb7334a/0xa40083ab9bf53739 lrc: 3/0,0 mode: CW/CW res: [0x240000402:0x7f7:0x0].0x0 bits 0x2/0x0 rrc: 3 type: IBT gid 0 flags: 0x40210400000000 nid: local remote: 0x0 expref: -99 pid: 11281 timeout: 0 lvb_type: 0 00010000:00010000:3.0:1776178204.392956:0:11281:0:(ldlm_request.c:1257:ldlm_cli_cancel_local()) ### server-side local cancel ns: mdt-lustre-MDT0001_UUID lock: 000000006cb7334a/0xa40083ab9bf53739 lrc: 4/0,0 mode: CW/CW res: [0x240000402:0x7f7:0x0].0x0 bits 0x2/0x0 rrc: 3 type: IBT gid 0 flags: 0x40218400000000 nid: local remote: 0x0 expref: -99 pid: 11281 timeout: 0 lvb_type: 0 00010000:00010000:3.0:1776178204.392965:0:11281:0:(ldlm_inodebits.c:94:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x240000402:0x7f7:0x0].0x0 (0000000043c2d1ef) 00010000:00010000:3.0:1776178204.392968:0:11281:0:(ldlm_inodebits.c:96:ldlm_reprocess_inodebits_queue()) Hint 2 00010000:00010000:3.0:1776178204.392972:0:11281:0:(ldlm_lockd.c:1903:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: mdt-lustre-MDT0001_UUID lock: 000000006cb7334a/0xa40083ab9bf53739 lrc: 2/0,0 mode: --/CW res: [0x240000402:0x7f7:0x0].0x0 bits 0x2/0x0 rrc: 3 type: IBT gid 0 flags: 0x44a19400000000 nid: local remote: 0x0 expref: -99 pid: 11281 timeout: 0 lvb_type: 0 00010000:00010000:3.0:1776178204.392978:0:11281:0:(ldlm_lock.c:216:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: mdt-lustre-MDT0001_UUID lock: 000000006cb7334a/0xa40083ab9bf53739 lrc: 0/0,0 mode: --/CW res: [0x240000402:0x7f7:0x0].0x0 bits 0x2/0x0 rrc: 3 type: IBT gid 0 flags: 0x44a19400000000 nid: local remote: 0x0 expref: -99 pid: 11281 timeout: 0 lvb_type: 0 00000004:00010000:3.0:1776178204.393000:0:11281:0:(mdt_handler.c:3992:mdt_save_lock()) ### save lock request 00000000889a4355 reply state 00000000d1d6cc04 transno 17179876273 ns: mdt-lustre-MDT0001_UUID lock: 00000000b335466f/0xa40083ab9bf53740 lrc: 3/0,1 mode: PW/PW res: [0x240000402:0x7f7:0x0].0x15fcbbea bits 0x2/0x0 rrc: 2 type: IBT gid 0 flags: 0x40210000000000 nid: local remote: 0x0 expref: -99 pid: 11281 timeout: 0 lvb_type: 0 00010000:00010000:0.0:1776178204.393167:0:3340:0:(ldlm_lock.c:1064:ldlm_granted_list_add_lock()) ### About to add lock: ns: mdt-lustre-MDT0001_UUID lock: 0000000011d684d8/0xa40083ab9bf53724 lrc: 3/0,1 mode: COS/COS res: [0x240000402:0x7f7:0x0].0xef09e6ea bits 0x2/0x0 rrc: 2 type: IBT gid 0 flags: 0x50210000000000 nid: local remote: 0x0 expref: -99 pid: 11281 timeout: 0 lvb_type: 0 00010000:00010000:0.0:1776178204.393177:0:3340:0:(ldlm_inodebits.c:94:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x240000402:0x7f7:0x0].0xef09e6ea (0000000026efbdea) 00010000:00010000:0.0:1776178204.393179:0:3340:0:(ldlm_inodebits.c:96:ldlm_reprocess_inodebits_queue()) Hint 2 00010000:00010000:0.0:1776178204.393182:0:3340:0:(ldlm_lock.c:1064:ldlm_granted_list_add_lock()) ### About to add lock: ns: mdt-lustre-MDT0001_UUID lock: 000000008813523d/0xa40083ab9bf5372b lrc: 3/0,1 mode: COS/COS res: [0x240000402:0x7fc:0x0].0x0 bits 0x3/0x0 rrc: 2 type: IBT gid 0 flags: 0x50210000000000 nid: local remote: 0x0 expref: -99 pid: 11281 timeout: 0 lvb_type: 0 00010000:00010000:0.0:1776178204.393187:0:3340:0:(ldlm_inodebits.c:94:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x240000402:0x7fc:0x0].0x0 (00000000abeb6ce0) 00010000:00010000:0.0:1776178204.393189:0:3340:0:(ldlm_inodebits.c:96:ldlm_reprocess_inodebits_queue()) Hint 3 00000100:00100000:3.0:1776178204.393544:0:11281:0:(service.c:2372:ptlrpc_server_handle_request()) Handled RPC req@00000000889a4355 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_004:b50963bd-8eba-4432-9a54-8888d00e3fa1+24:17671:x1862456979101696:12345-192.168.204.3@tcp:36:rm.0 Request processed in 2090us (2635us total) trans 17179876273 rc 0/0 00000100:00100000:3.0:1776178204.393553:0:11281:0:(nrs_fifo.c:241:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.204.3@tcp, seq: 6590 00010000:00010000:1.0:1776178204.393572:0:6245:0:(ldlm_lockd.c:2789:ldlm_bl_get_work()) Got 1 locks of 0 total in blp. (0 blwis in pool) 00010000:00010000:1.0:1776178204.393578:0:6245:0:(ldlm_lockd.c:1879:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: mdt-lustre-MDT0001_UUID lock: 0000000068a2dd4a/0xa40083ab9bf5374e lrc: 2/0,0 mode: PW/PW res: [0x240000402:0x7f9:0x0].0x0 bits 0x40/0x0 rrc: 3 type: IBT gid 18446635898622804416 flags: 0x40010480000000 nid: local remote: 0x0 expref: -99 pid: 11281 timeout: 0 lvb_type: 0 00010000:00010000:1.0:1776178204.393587:0:6245:0:(ldlm_lockd.c:1893:ldlm_handle_bl_callback()) Lock 0000000068a2dd4a already unused, calling callback (0000000063c97258) 00010000:00010000:1.0:1776178204.393590:0:6245:0:(ldlm_request.c:364:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: mdt-lustre-MDT0001_UUID lock: 0000000068a2dd4a/0xa40083ab9bf5374e lrc: 2/0,0 mode: PW/PW res: [0x240000402:0x7f9:0x0].0x0 bits 0x40/0x0 rrc: 3 type: IBT gid 18446635898622804416 flags: 0x40010480000000 nid: local remote: 0x0 expref: -99 pid: 11281 timeout: 0 lvb_type: 0 00010000:00010000:1.0:1776178204.393597:0:6245:0:(ldlm_request.c:1257:ldlm_cli_cancel_local()) ### server-side local cancel ns: mdt-lustre-MDT0001_UUID lock: 0000000068a2dd4a/0xa40083ab9bf5374e lrc: 3/0,0 mode: PW/PW res: [0x240000402:0x7f9:0x0].0x0 bits 0x40/0x0 rrc: 3 type: IBT gid 18446635898622804416 flags: 0x40018480000000 nid: local remote: 0x0 expref: -99 pid: 11281 timeout: 0 lvb_type: 0 00010000:00010000:1.0:1776178204.393607:0:6245:0:(ldlm_inodebits.c:94:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x240000402:0x7f9:0x0].0x0 (00000000de0abd11) 00010000:00010000:1.0:1776178204.393611:0:6245:0:(ldlm_inodebits.c:96:ldlm_reprocess_inodebits_queue()) Hint 40 00010000:00010000:1.0:1776178204.393614:0:6245:0:(ldlm_lockd.c:1903:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: mdt-lustre-MDT0001_UUID lock: 0000000068a2dd4a/0xa40083ab9bf5374e lrc: 1/0,0 mode: --/PW res: [0x240000402:0x7f9:0x0].0x0 bits 0x40/0x0 rrc: 3 type: IBT gid 18446635898622804416 flags: 0x44819480000000 nid: local remote: 0x0 expref: -99 pid: 11281 timeout: 0 lvb_type: 0 00010000:00010000:1.0:1776178204.393619:0:6245:0:(ldlm_lock.c:216:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: mdt-lustre-MDT0001_UUID lock: 0000000068a2dd4a/0xa40083ab9bf5374e lrc: 0/0,0 mode: --/PW res: [0x240000402:0x7f9:0x0].0x0 bits 0x40/0x0 rrc: 3 type: IBT gid 18446635898622804416 flags: 0x44819480000000 nid: local remote: 0x0 expref: -99 pid: 11281 timeout: 0 lvb_type: 0 00010000:00010000:1.0:1776178204.393633:0:6245:0:(ldlm_lockd.c:2792:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00010000:00010000:1.0:1776178204.393635:0:6245:0:(ldlm_lockd.c:2792:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00010000:00010000:1.0:1776178204.393637:0:6245:0:(ldlm_lockd.c:2792:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:3.0:1776178204.398894:0:3346:0:(events.c:361:request_in_callback()) peer: 12345-192.168.204.3@tcp (source: 12345-192.168.204.3@tcp) 00000100:00100000:0.0:1776178204.399021:0:11281:0:(service.c:2152:ptlrpc_server_handle_req_in()) got req x1862456979101760 00000100:00100000:0.0:1776178204.399148:0:11281:0:(nrs_fifo.c:179:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.204.3@tcp, seq: 6591 00000100:00100000:0.0:1776178204.399157:0:11281:0:(service.c:2324:ptlrpc_server_handle_request()) Handling RPC req@00000000952003a4 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_004:b50963bd-8eba-4432-9a54-8888d00e3fa1+7:17671:x1862456979101760:12345-192.168.204.3@tcp:101:rm.0 00010000:00010000:0.0:1776178204.399187:0:11281:0:(ldlm_lockd.c:1248:ldlm_handle_enqueue0()) ### server-side enqueue handler START 00010000:00010000:0.0:1776178204.399216:0:11281:0:(ldlm_lockd.c:1318:ldlm_handle_enqueue0()) ### server-side enqueue handler, new lock created ns: mdt-lustre-MDT0000_UUID lock: 0000000030bf6b1e/0xa40083ab9bf53755 lrc: 2/0,0 mode: --/CR res: [0x200000007:0x1:0x0].0x0 bits 0x0/0x0 rrc: 4 type: IBT gid 0 flags: 0x40000000000000 nid: local remote: 0x771f68aa1eb27bab expref: -99 pid: 11281 timeout: 0 lvb_type: 0 00010000:00010000:0.0:1776178204.399298:0:11281:0:(ldlm_lock.c:766:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(CR) ns: mdt-lustre-MDT0000_UUID lock: 0000000095e53696/0xa40083ab9bf5375c lrc: 3/1,0 mode: --/CR res: [0x200000007:0x1:0x0].0x0 bits 0x0/0x0 rrc: 5 type: IBT gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 11281 timeout: 0 lvb_type: 0 00010000:00010000:0.0:1776178204.399319:0:11281:0:(ldlm_lock.c:1064:ldlm_granted_list_add_lock()) ### About to add lock: ns: mdt-lustre-MDT0000_UUID lock: 0000000095e53696/0xa40083ab9bf5375c lrc: 3/1,0 mode: CR/CR res: [0x200000007:0x1:0x0].0x0 bits 0x2/0x0 rrc: 5 type: IBT gid 0 flags: 0x50210000000000 nid: local remote: 0x0 expref: -99 pid: 11281 timeout: 0 lvb_type: 0 00010000:00010000:0.0:1776178204.399333:0:11281:0:(ldlm_request.c:528:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: mdt-lustre-MDT0000_UUID lock: 0000000095e53696/0xa40083ab9bf5375c lrc: 3/1,0 mode: CR/CR res: [0x200000007:0x1:0x0].0x0 bits 0x2/0x0 rrc: 5 type: IBT gid 0 flags: 0x40210000000000 nid: local remote: 0x0 expref: -99 pid: 11281 timeout: 0 lvb_type: 0 00010000:00010000:0.0:1776178204.399347:0:11281:0:(ldlm_lock.c:766:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PR) ns: mdt-lustre-MDT0000_UUID lock: 000000004bd1d7cb/0xa40083ab9bf53763 lrc: 3/1,0 mode: --/PR res: [0x200000007:0x1:0x0].0x58a1b1aa bits 0x0/0x0 rrc: 2 type: IBT gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 11281 timeout: 0 lvb_type: 0 00010000:00010000:0.0:1776178204.399354:0:11281:0:(ldlm_lock.c:1064:ldlm_granted_list_add_lock()) ### About to add lock: ns: mdt-lustre-MDT0000_UUID lock: 000000004bd1d7cb/0xa40083ab9bf53763 lrc: 3/1,0 mode: PR/PR res: [0x200000007:0x1:0x0].0x58a1b1aa bits 0x2/0x0 rrc: 2 type: IBT gid 0 flags: 0x50210000000000 nid: local remote: 0x0 expref: -99 pid: 11281 timeout: 0 lvb_type: 0 00010000:00010000:0.0:1776178204.399359:0:11281:0:(ldlm_request.c:528:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: mdt-lustre-MDT0000_UUID lock: 000000004bd1d7cb/0xa40083ab9bf53763 lrc: 3/1,0 mode: PR/PR res: [0x200000007:0x1:0x0].0x58a1b1aa bits 0x2/0x0 rrc: 2 type: IBT gid 0 flags: 0x40210000000000 nid: local remote: 0x0 expref: -99 pid: 11281 timeout: 0 lvb_type: 0 00010000:00010000:0.0:1776178204.399426:0:11281:0:(ldlm_lock.c:766:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PR) ns: mdt-lustre-MDT0000_UUID lock: 000000007797f5e2/0xa40083ab9bf5376a lrc: 3/1,0 mode: --/PR res: [0x240000402:0x7f7:0x0].0x0 bits 0x0/0x0 rrc: 3 type: IBT gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 11281 timeout: 0 lvb_type: 0 00010000:00010000:0.0:1776178204.399436:0:11281:0:(ldlm_lock.c:1064:ldlm_granted_list_add_lock()) ### About to add lock: ns: mdt-lustre-MDT0000_UUID lock: 000000007797f5e2/0xa40083ab9bf5376a lrc: 3/1,0 mode: PR/PR res: [0x240000402:0x7f7:0x0].0x0 bits 0x1/0x0 rrc: 3 type: IBT gid 0 flags: 0x50210000000000 nid: local remote: 0x0 expref: -99 pid: 11281 timeout: 0 lvb_type: 0 00010000:00010000:0.0:1776178204.399442:0:11281:0:(ldlm_request.c:528:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: mdt-lustre-MDT0000_UUID lock: 000000007797f5e2/0xa40083ab9bf5376a lrc: 3/1,0 mode: PR/PR res: [0x240000402:0x7f7:0x0].0x0 bits 0x1/0x0 rrc: 3 type: IBT gid 0 flags: 0x40210000000000 nid: local remote: 0x0 expref: -99 pid: 11281 timeout: 0 lvb_type: 0 00000004:00080000:0.0:1776178204.399451:0:11281:0:(osp_object.c:1389:osp_invalidate()) Invalidate osp_object [0x240000402:0x7f7:0x0] 00000004:00010000:0.0:1776178204.399469:0:11281:0:(mdt_handler.c:2353:mdt_getattr_name_lock()) ### Returning lock to client ns: mdt-lustre-MDT0000_UUID lock: 000000007797f5e2/0xa40083ab9bf5376a lrc: 3/1,0 mode: PR/PR res: [0x240000402:0x7f7:0x0].0x0 bits 0x1/0x0 rrc: 3 type: IBT gid 0 flags: 0x40210000000000 nid: local remote: 0x0 expref: -99 pid: 11281 timeout: 0 lvb_type: 0 00010000:00010000:0.0:1776178204.399482:0:11281:0:(ldlm_lock.c:820:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(CR) ns: mdt-lustre-MDT0000_UUID lock: 0000000095e53696/0xa40083ab9bf5375c lrc: 3/1,0 mode: CR/CR res: [0x200000007:0x1:0x0].0x0 bits 0x2/0x0 rrc: 5 type: IBT gid 0 flags: 0x50210000000000 nid: local remote: 0x0 expref: -99 pid: 11281 timeout: 0 lvb_type: 0 00010000:00010000:0.0:1776178204.399494:0:11281:0:(ldlm_lock.c:879:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: mdt-lustre-MDT0000_UUID lock: 0000000095e53696/0xa40083ab9bf5375c lrc: 2/0,0 mode: CR/CR res: [0x200000007:0x1:0x0].0x0 bits 0x2/0x0 rrc: 5 type: IBT gid 0 flags: 0x50210400000000 nid: local remote: 0x0 expref: -99 pid: 11281 timeout: 0 lvb_type: 0 00010000:00010000:0.0:1776178204.399501:0:11281:0:(ldlm_lockd.c:1879:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: mdt-lustre-MDT0000_UUID lock: 0000000095e53696/0xa40083ab9bf5375c lrc: 3/0,0 mode: CR/CR res: [0x200000007:0x1:0x0].0x0 bits 0x2/0x0 rrc: 5 type: IBT gid 0 flags: 0x40210400000000 nid: local remote: 0x0 expref: -99 pid: 11281 timeout: 0 lvb_type: 0 00010000:00010000:0.0:1776178204.399509:0:11281:0:(ldlm_lockd.c:1893:ldlm_handle_bl_callback()) Lock 0000000095e53696 already unused, calling callback (0000000079e78348) 00010000:00010000:0.0:1776178204.399512:0:11281:0:(ldlm_request.c:364:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: mdt-lustre-MDT0000_UUID lock: 0000000095e53696/0xa40083ab9bf5375c lrc: 3/0,0 mode: CR/CR res: [0x200000007:0x1:0x0].0x0 bits 0x2/0x0 rrc: 5 type: IBT gid 0 flags: 0x40210400000000 nid: local remote: 0x0 expref: -99 pid: 11281 timeout: 0 lvb_type: 0 00010000:00010000:0.0:1776178204.399520:0:11281:0:(ldlm_request.c:1257:ldlm_cli_cancel_local()) ### server-side local cancel ns: mdt-lustre-MDT0000_UUID lock: 0000000095e53696/0xa40083ab9bf5375c lrc: 4/0,0 mode: CR/CR res: [0x200000007:0x1:0x0].0x0 bits 0x2/0x0 rrc: 5 type: IBT gid 0 flags: 0x40218400000000 nid: local remote: 0x0 expref: -99 pid: 11281 timeout: 0 lvb_type: 0 00010000:00010000:0.0:1776178204.399532:0:11281:0:(ldlm_inodebits.c:94:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x200000007:0x1:0x0].0x0 (00000000ad0d4094) 00010000:00010000:0.0:1776178204.399536:0:11281:0:(ldlm_inodebits.c:96:ldlm_reprocess_inodebits_queue()) Hint 2 00010000:00010000:0.0:1776178204.399540:0:11281:0:(ldlm_lockd.c:1903:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: mdt-lustre-MDT0000_UUID lock: 0000000095e53696/0xa40083ab9bf5375c lrc: 2/0,0 mode: --/CR res: [0x200000007:0x1:0x0].0x0 bits 0x2/0x0 rrc: 5 type: IBT gid 0 flags: 0x44a19400000000 nid: local remote: 0x0 expref: -99 pid: 11281 timeout: 0 lvb_type: 0 00010000:00010000:0.0:1776178204.399545:0:11281:0:(ldlm_lock.c:216:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: mdt-lustre-MDT0000_UUID lock: 0000000095e53696/0xa40083ab9bf5375c lrc: 0/0,0 mode: --/CR res: [0x200000007:0x1:0x0].0x0 bits 0x2/0x0 rrc: 5 type: IBT gid 0 flags: 0x44a19400000000 nid: local remote: 0x0 expref: -99 pid: 11281 timeout: 0 lvb_type: 0 00010000:00010000:0.0:1776178204.399559:0:11281:0:(ldlm_lock.c:820:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PR) ns: mdt-lustre-MDT0000_UUID lock: 000000004bd1d7cb/0xa40083ab9bf53763 lrc: 3/1,0 mode: PR/PR res: [0x200000007:0x1:0x0].0x58a1b1aa bits 0x2/0x0 rrc: 2 type: IBT gid 0 flags: 0x50210000000000 nid: local remote: 0x0 expref: -99 pid: 11281 timeout: 0 lvb_type: 0 00010000:00010000:0.0:1776178204.399565:0:11281:0:(ldlm_lock.c:879:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: mdt-lustre-MDT0000_UUID lock: 000000004bd1d7cb/0xa40083ab9bf53763 lrc: 2/0,0 mode: PR/PR res: [0x200000007:0x1:0x0].0x58a1b1aa bits 0x2/0x0 rrc: 2 type: IBT gid 0 flags: 0x50210400000000 nid: local remote: 0x0 expref: -99 pid: 11281 timeout: 0 lvb_type: 0 00010000:00010000:0.0:1776178204.399570:0:11281:0:(ldlm_lockd.c:1879:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: mdt-lustre-MDT0000_UUID lock: 000000004bd1d7cb/0xa40083ab9bf53763 lrc: 3/0,0 mode: PR/PR res: [0x200000007:0x1:0x0].0x58a1b1aa bits 0x2/0x0 rrc: 2 type: IBT gid 0 flags: 0x40210400000000 nid: local remote: 0x0 expref: -99 pid: 11281 timeout: 0 lvb_type: 0 00010000:00010000:0.0:1776178204.399580:0:11281:0:(ldlm_lockd.c:1893:ldlm_handle_bl_callback()) Lock 000000004bd1d7cb already unused, calling callback (0000000079e78348) 00010000:00010000:0.0:1776178204.399581:0:11281:0:(ldlm_request.c:364:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: mdt-lustre-MDT0000_UUID lock: 000000004bd1d7cb/0xa40083ab9bf53763 lrc: 3/0,0 mode: PR/PR res: [0x200000007:0x1:0x0].0x58a1b1aa bits 0x2/0x0 rrc: 2 type: IBT gid 0 flags: 0x40210400000000 nid: local remote: 0x0 expref: -99 pid: 11281 timeout: 0 lvb_type: 0 00010000:00010000:0.0:1776178204.399587:0:11281:0:(ldlm_request.c:1257:ldlm_cli_cancel_local()) ### server-side local cancel ns: mdt-lustre-MDT0000_UUID lock: 000000004bd1d7cb/0xa40083ab9bf53763 lrc: 4/0,0 mode: PR/PR res: [0x200000007:0x1:0x0].0x58a1b1aa bits 0x2/0x0 rrc: 2 type: IBT gid 0 flags: 0x40218400000000 nid: local remote: 0x0 expref: -99 pid: 11281 timeout: 0 lvb_type: 0 00010000:00010000:0.0:1776178204.399592:0:11281:0:(ldlm_inodebits.c:94:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x200000007:0x1:0x0].0x58a1b1aa (00000000c9b683f9) 00010000:00010000:0.0:1776178204.399594:0:11281:0:(ldlm_inodebits.c:96:ldlm_reprocess_inodebits_queue()) Hint 2 00010000:00010000:0.0:1776178204.399596:0:11281:0:(ldlm_lockd.c:1903:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: mdt-lustre-MDT0000_UUID lock: 000000004bd1d7cb/0xa40083ab9bf53763 lrc: 2/0,0 mode: --/PR res: [0x200000007:0x1:0x0].0x58a1b1aa bits 0x2/0x0 rrc: 2 type: IBT gid 0 flags: 0x44a19400000000 nid: local remote: 0x0 expref: -99 pid: 11281 timeout: 0 lvb_type: 0 00010000:00010000:0.0:1776178204.399601:0:11281:0:(ldlm_lock.c:216:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: mdt-lustre-MDT0000_UUID lock: 000000004bd1d7cb/0xa40083ab9bf53763 lrc: 0/0,0 mode: --/PR res: [0x200000007:0x1:0x0].0x58a1b1aa bits 0x2/0x0 rrc: 2 type: IBT gid 0 flags: 0x44a19400000000 nid: local remote: 0x0 expref: -99 pid: 11281 timeout: 0 lvb_type: 0 00010000:00010000:0.0:1776178204.399631:0:11281:0:(ldlm_lock.c:216:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: mdt-lustre-MDT0000_UUID lock: 0000000030bf6b1e/0xa40083ab9bf53755 lrc: 0/0,0 mode: --/CR res: [0x200000007:0x1:0x0].0x0 bits 0x2/0x0 rrc: 4 type: IBT gid 0 flags: 0x44000000000000 nid: 192.168.204.3@tcp remote: 0x771f68aa1eb27bab expref: 9 pid: 11281 timeout: 0 lvb_type: 0 00010000:00010000:0.0:1776178204.399642:0:11281:0:(ldlm_lockd.c:1484:ldlm_handle_enqueue0()) ### server-side enqueue handler, sending reply (err=0, rc=0) ns: mdt-lustre-MDT0000_UUID lock: 000000007797f5e2/0xa40083ab9bf5376a lrc: 3/0,0 mode: PR/PR res: [0x240000402:0x7f7:0x0].0x0 bits 0x1/0x0 rrc: 3 type: IBT gid 0 flags: 0x40200000000000 nid: 192.168.204.3@tcp remote: 0x771f68aa1eb27bab expref: 8 pid: 11281 timeout: 0 lvb_type: 0 00010000:00010000:0.0:1776178204.399648:0:11281:0:(ldlm_inodebits.c:94:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x240000402:0x7f7:0x0].0x0 (00000000940da82c) 00010000:00010000:0.0:1776178204.399651:0:11281:0:(ldlm_inodebits.c:96:ldlm_reprocess_inodebits_queue()) Hint 1 00010000:00010000:0.0:1776178204.399652:0:11281:0:(ldlm_lockd.c:1565:ldlm_handle_enqueue0()) ### server-side enqueue handler END (lock 000000007797f5e2, rc 0) 00000100:00100000:0.0:1776178204.399731:0:11281:0:(service.c:2372:ptlrpc_server_handle_request()) Handled RPC req@00000000952003a4 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_004:b50963bd-8eba-4432-9a54-8888d00e3fa1+8:17671:x1862456979101760:12345-192.168.204.3@tcp:101:rm.0 Request processed in 573us (849us total) trans 0 rc 0/0 00000100:00100000:0.0:1776178204.399739:0:11281:0:(nrs_fifo.c:241:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.204.3@tcp, seq: 6591 00000100:00100000:0.0:1776178204.403379:0:3348:0:(events.c:361:request_in_callback()) peer: 12345-192.168.204.3@tcp (source: 12345-192.168.204.3@tcp) 00000100:00100000:0.0:1776178204.403491:0:3348:0:(events.c:361:request_in_callback()) peer: 12345-192.168.204.3@tcp (source: 12345-192.168.204.3@tcp) 00000100:00100000:3.0:1776178204.403671:0:6242:0:(service.c:2152:ptlrpc_server_handle_req_in()) got req x1862456979101888 00000100:00100000:3.0:1776178204.403759:0:6242:0:(nrs_fifo.c:179:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.204.3@tcp, seq: 45 00000100:00100000:3.0:1776178204.403767:0:6242:0:(service.c:2324:ptlrpc_server_handle_request()) Handling RPC req@0000000087bcb6a9 pname:cluuid+ref:pid:xid:nid:opc:job ldlm_cn00_001:b50963bd-8eba-4432-9a54-8888d00e3fa1+8:2271:x1862456979101888:12345-192.168.204.3@tcp:103:ldlm_bl_04.0 00010000:00010000:3.0:1776178204.403789:0:6242:0:(ldlm_lockd.c:1720:ldlm_request_cancel()) ### server-side cancel handler START: 1 locks, starting at 0 00010000:00010000:3.0:1776178204.403817:0:6242:0:(ldlm_lock.c:216:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: mdt-lustre-MDT0000_UUID lock: 000000007797f5e2/0xa40083ab9bf5376a lrc: 0/0,0 mode: --/PR res: [0x240000402:0x7f7:0x0].0x0 bits 0x1/0x0 rrc: 4 type: IBT gid 0 flags: 0x44a01000000000 nid: 192.168.204.3@tcp remote: 0x771f68aa1eb27bab expref: 8 pid: 11281 timeout: 0 lvb_type: 0 00010000:00010000:3.0:1776178204.403838:0:6242:0:(ldlm_inodebits.c:94:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x240000402:0x7f7:0x0].0x0 (00000000940da82c) 00010000:00010000:3.0:1776178204.403844:0:6242:0:(ldlm_lockd.c:1782:ldlm_request_cancel()) ### server-side cancel handler END 00000100:00100000:3.0:1776178204.403895:0:6242:0:(service.c:2372:ptlrpc_server_handle_request()) Handled RPC req@0000000087bcb6a9 pname:cluuid+ref:pid:xid:nid:opc:job ldlm_cn00_001:b50963bd-8eba-4432-9a54-8888d00e3fa1+7:2271:x1862456979101888:12345-192.168.204.3@tcp:103:ldlm_bl_04.0 Request processed in 128us (405us total) trans 0 rc 0/0 00000100:00100000:3.0:1776178204.403903:0:6242:0:(nrs_fifo.c:241:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.204.3@tcp, seq: 45 00010000:00010000:2.0:1776178204.405370:0:3339:0:(ldlm_lock.c:1064:ldlm_granted_list_add_lock()) ### About to add lock: ns: mdt-lustre-MDT0001_UUID lock: 00000000b335466f/0xa40083ab9bf53740 lrc: 3/0,1 mode: COS/COS res: [0x240000402:0x7f7:0x0].0x15fcbbea bits 0x2/0x0 rrc: 2 type: IBT gid 0 flags: 0x50210000000000 nid: local remote: 0x0 expref: -99 pid: 11281 timeout: 0 lvb_type: 0 00010000:00010000:2.0:1776178204.405394:0:3339:0:(ldlm_inodebits.c:94:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x240000402:0x7f7:0x0].0x15fcbbea (00000000f97f1796) 00010000:00010000:2.0:1776178204.405398:0:3339:0:(ldlm_inodebits.c:96:ldlm_reprocess_inodebits_queue()) Hint 2 00010000:00010000:2.0:1776178204.405403:0:3339:0:(ldlm_lock.c:1064:ldlm_granted_list_add_lock()) ### About to add lock: ns: mdt-lustre-MDT0001_UUID lock: 00000000575572c2/0xa40083ab9bf53747 lrc: 3/0,1 mode: COS/COS res: [0x240000402:0x7f9:0x0].0x0 bits 0x3/0x0 rrc: 2 type: IBT gid 0 flags: 0x50210000000000 nid: local remote: 0x0 expref: -99 pid: 11281 timeout: 0 lvb_type: 0 00010000:00010000:2.0:1776178204.405408:0:3339:0:(ldlm_inodebits.c:94:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x240000402:0x7f9:0x0].0x0 (00000000de0abd11) 00010000:00010000:2.0:1776178204.405410:0:3339:0:(ldlm_inodebits.c:96:ldlm_reprocess_inodebits_queue()) Hint 3 00000100:00100000:1.0:1776178204.405940:0:6257:0:(service.c:2152:ptlrpc_server_handle_req_in()) got req x1862456979101824 00000100:00100000:1.0:1776178204.406398:0:6257:0:(nrs_fifo.c:179:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.204.3@tcp, seq: 6592 00000100:00100000:1.0:1776178204.406406:0:6257:0:(service.c:2324:ptlrpc_server_handle_request()) Handling RPC req@0000000067671fc8 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_002:b50963bd-8eba-4432-9a54-8888d00e3fa1+24:17671:x1862456979101824:12345-192.168.204.3@tcp:101:rm.0 00010000:00010000:1.0:1776178204.406433:0:6257:0:(ldlm_lockd.c:1248:ldlm_handle_enqueue0()) ### server-side enqueue handler START 00010000:00010000:1.0:1776178204.406453:0:6257:0:(ldlm_lockd.c:1318:ldlm_handle_enqueue0()) ### server-side enqueue handler, new lock created ns: mdt-lustre-MDT0001_UUID lock: 000000006b0afb52/0xa40083ab9bf53771 lrc: 2/0,0 mode: --/CR res: [0x240000402:0x7f7:0x0].0x0 bits 0x0/0x0 rrc: 3 type: IBT gid 0 flags: 0x40000000000000 nid: local remote: 0x771f68aa1eb27bb2 expref: -99 pid: 6257 timeout: 0 lvb_type: 0 00010000:00010000:1.0:1776178204.406508:0:6257:0:(ldlm_lock.c:766:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PR) ns: mdt-lustre-MDT0001_UUID lock: 00000000daa24072/0xa40083ab9bf53778 lrc: 3/1,0 mode: --/PR res: [0x240000402:0x7f7:0x0].0x0 bits 0x0/0x0 rrc: 4 type: IBT gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 6257 timeout: 0 lvb_type: 0 00010000:00010000:1.0:1776178204.406521:0:6257:0:(ldlm_lock.c:1064:ldlm_granted_list_add_lock()) ### About to add lock: ns: mdt-lustre-MDT0001_UUID lock: 00000000daa24072/0xa40083ab9bf53778 lrc: 3/1,0 mode: PR/PR res: [0x240000402:0x7f7:0x0].0x0 bits 0x12/0x0 rrc: 4 type: IBT gid 0 flags: 0x50210000000000 nid: local remote: 0x0 expref: -99 pid: 6257 timeout: 0 lvb_type: 0 00010000:00010000:1.0:1776178204.406531:0:6257:0:(ldlm_request.c:528:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: mdt-lustre-MDT0001_UUID lock: 00000000daa24072/0xa40083ab9bf53778 lrc: 3/1,0 mode: PR/PR res: [0x240000402:0x7f7:0x0].0x0 bits 0x12/0x0 rrc: 4 type: IBT gid 0 flags: 0x40210000000000 nid: local remote: 0x0 expref: -99 pid: 6257 timeout: 0 lvb_type: 0 00000004:00200000:1.0:1776178204.406575:0:6257:0:(mdt_handler.c:912:mdt_pack_attr2body()) [0x240000402:0x7f7:0x0]: returning size 4096 00000004:00200000:1.0:1776178204.406579:0:6257:0:(mdt_handler.c:1523:mdt_getattr_internal()) dirent count 2 stripe count 1 MDT count 2 00010000:00010000:1.0:1776178204.406618:0:6257:0:(ldlm_lock.c:216:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: mdt-lustre-MDT0001_UUID lock: 000000006b0afb52/0xa40083ab9bf53771 lrc: 0/0,0 mode: --/CR res: [0x240000402:0x7f7:0x0].0x0 bits 0x2/0x0 rrc: 4 type: IBT gid 0 flags: 0x44000000000000 nid: 192.168.204.3@tcp remote: 0x771f68aa1eb27bb2 expref: 26 pid: 6257 timeout: 0 lvb_type: 0 00010000:00010000:1.0:1776178204.406636:0:6257:0:(ldlm_lockd.c:1484:ldlm_handle_enqueue0()) ### server-side enqueue handler, sending reply (err=0, rc=0) ns: mdt-lustre-MDT0001_UUID lock: 00000000daa24072/0xa40083ab9bf53778 lrc: 3/0,0 mode: PR/PR res: [0x240000402:0x7f7:0x0].0x0 bits 0x12/0x0 rrc: 3 type: IBT gid 0 flags: 0x40200000000000 nid: 192.168.204.3@tcp remote: 0x771f68aa1eb27bb2 expref: 25 pid: 6257 timeout: 0 lvb_type: 0 00010000:00010000:1.0:1776178204.406643:0:6257:0:(ldlm_inodebits.c:94:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x240000402:0x7f7:0x0].0x0 (0000000043c2d1ef) 00010000:00010000:1.0:1776178204.406645:0:6257:0:(ldlm_inodebits.c:96:ldlm_reprocess_inodebits_queue()) Hint 12 00010000:00010000:1.0:1776178204.406648:0:6257:0:(ldlm_lockd.c:1565:ldlm_handle_enqueue0()) ### server-side enqueue handler END (lock 00000000daa24072, rc 0) 00000100:00100000:1.0:1776178204.406731:0:6257:0:(service.c:2372:ptlrpc_server_handle_request()) Handled RPC req@0000000067671fc8 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_002:b50963bd-8eba-4432-9a54-8888d00e3fa1+25:17671:x1862456979101824:12345-192.168.204.3@tcp:101:rm.0 Request processed in 324us (3364us total) trans 0 rc 0/0 00000100:00100000:1.0:1776178204.406740:0:6257:0:(nrs_fifo.c:241:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.204.3@tcp, seq: 6592 00000100:00100000:3.0:1776178204.410866:0:3348:0:(events.c:361:request_in_callback()) peer: 12345-192.168.204.3@tcp (source: 12345-192.168.204.3@tcp) 00000100:00100000:1.0:1776178204.411650:0:20770:0:(service.c:2152:ptlrpc_server_handle_req_in()) got req x1862456979101952 00000100:00100000:1.0:1776178204.411750:0:20770:0:(nrs_fifo.c:179:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.204.3@tcp, seq: 46 00000100:00100000:1.0:1776178204.411758:0:20770:0:(service.c:2324:ptlrpc_server_handle_request()) Handling RPC req@00000000b0409689 pname:cluuid+ref:pid:xid:nid:opc:job ldlm_cn00_005:b50963bd-8eba-4432-9a54-8888d00e3fa1+7:2271:x1862456979101952:12345-192.168.204.3@tcp:103:rm.0 00010000:00010000:1.0:1776178204.411783:0:20770:0:(ldlm_lockd.c:1720:ldlm_request_cancel()) ### server-side cancel handler START: 1 locks, starting at 0 00010000:00010000:1.0:1776178204.411813:0:20770:0:(ldlm_lock.c:216:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: mdt-lustre-MDT0000_UUID lock: 00000000801fce1a/0xa40083ab9bf5336c lrc: 0/0,0 mode: --/PR res: [0x240000402:0x7f7:0x0].0x0 bits 0x1/0x0 rrc: 3 type: IBT gid 0 flags: 0x44a01000000000 nid: 192.168.204.3@tcp remote: 0x771f68aa1eb27a93 expref: 7 pid: 8250 timeout: 0 lvb_type: 0 00010000:00010000:1.0:1776178204.411835:0:20770:0:(ldlm_inodebits.c:94:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x240000402:0x7f7:0x0].0x0 (00000000940da82c) 00010000:00010000:1.0:1776178204.411847:0:20770:0:(ldlm_lockd.c:1782:ldlm_request_cancel()) ### server-side cancel handler END 00000100:00100000:1.0:1776178204.411965:0:20770:0:(service.c:2372:ptlrpc_server_handle_request()) Handled RPC req@00000000b0409689 pname:cluuid+ref:pid:xid:nid:opc:job ldlm_cn00_005:b50963bd-8eba-4432-9a54-8888d00e3fa1+6:2271:x1862456979101952:12345-192.168.204.3@tcp:103:rm.0 Request processed in 207us (1111us total) trans 0 rc 0/0 00000100:00100000:1.0:1776178204.411973:0:20770:0:(nrs_fifo.c:241:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.204.3@tcp, seq: 46 00000100:00100000:2.0:1776178204.413867:0:3346:0:(events.c:361:request_in_callback()) peer: 12345-192.168.204.3@tcp (source: 12345-192.168.204.3@tcp) 00000100:00100000:0.0:1776178204.415487:0:6259:0:(service.c:2152:ptlrpc_server_handle_req_in()) got req x1862456979102080 00000100:00100000:0.0:1776178204.415556:0:6259:0:(nrs_fifo.c:179:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.204.3@tcp, seq: 2144 00000100:00100000:0.0:1776178204.415563:0:6259:0:(service.c:2324:ptlrpc_server_handle_request()) Handling RPC req@0000000054130213 pname:cluuid+ref:pid:xid:nid:opc:job mdt_rdpg00_001:b50963bd-8eba-4432-9a54-8888d00e3fa1+25:17671:x1862456979102080:12345-192.168.204.3@tcp:35:rm.0 00000100:00100000:0.0:1776178204.415862:0:6259:0:(service.c:2372:ptlrpc_server_handle_request()) Handled RPC req@0000000054130213 pname:cluuid+ref:pid:xid:nid:opc:job mdt_rdpg00_001:b50963bd-8eba-4432-9a54-8888d00e3fa1+26:17671:x1862456979102080:12345-192.168.204.3@tcp:35:rm.0 Request processed in 298us (2003us total) trans 17179876274 rc 0/0 00000100:00100000:0.0:1776178204.415873:0:6259:0:(nrs_fifo.c:241:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.204.3@tcp, seq: 2144 00000100:00100000:3.0:1776178204.418362:0:3347:0:(events.c:361:request_in_callback()) peer: 12345-192.168.204.3@tcp (source: 12345-192.168.204.3@tcp) 00000100:00100000:1.0:1776178204.418475:0:6257:0:(service.c:2152:ptlrpc_server_handle_req_in()) got req x1862456979102208 00000100:00100000:1.0:1776178204.418532:0:6257:0:(nrs_fifo.c:179:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.204.3@tcp, seq: 6593 00000100:00100000:1.0:1776178204.418540:0:6257:0:(service.c:2324:ptlrpc_server_handle_request()) Handling RPC req@000000008c469ec0 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_002:b50963bd-8eba-4432-9a54-8888d00e3fa1+26:17671:x1862456979102208:12345-192.168.204.3@tcp:36:rm.0 00010000:00010000:1.0:1776178204.418604:0:6257:0:(ldlm_lockd.c:1720:ldlm_request_cancel()) ### server-side cancel handler START: 2 locks, starting at 0 00010000:00010000:1.0:1776178204.418634:0:6257:0:(ldlm_lock.c:216:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: mdt-lustre-MDT0001_UUID lock: 00000000daa24072/0xa40083ab9bf53778 lrc: 0/0,0 mode: --/PR res: [0x240000402:0x7f7:0x0].0x0 bits 0x12/0x0 rrc: 4 type: IBT gid 0 flags: 0x44a01000000000 nid: 192.168.204.3@tcp remote: 0x771f68aa1eb27bb2 expref: 26 pid: 6257 timeout: 0 lvb_type: 0 00010000:00010000:1.0:1776178204.418658:0:6257:0:(ldlm_lock.c:216:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: mdt-lustre-MDT0001_UUID lock: 00000000f779de26/0xa40083ab9bf53683 lrc: 0/0,0 mode: --/CR res: [0x240000402:0x7f7:0x0].0x0 bits 0x5/0x0 rrc: 3 type: IBT gid 0 flags: 0x44a01000000000 nid: 192.168.204.3@tcp remote: 0x771f68aa1eb27ba4 expref: 25 pid: 11281 timeout: 0 lvb_type: 0 00010000:00010000:1.0:1776178204.418671:0:6257:0:(ldlm_inodebits.c:94:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x240000402:0x7f7:0x0].0x0 (0000000043c2d1ef) 00010000:00010000:1.0:1776178204.418682:0:6257:0:(ldlm_lockd.c:1782:ldlm_request_cancel()) ### server-side cancel handler END 00010000:00010000:1.0:1776178204.418742:0:6257:0:(ldlm_lock.c:766:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(EX) ns: lustre-MDT0000-osp-MDT0001 lock: 0000000056462e0c/0xa40083ab9bf5377f lrc: 3/0,1 mode: --/EX res: [0x200000007:0x1:0x0].0x0 bits 0x0/0x0 rrc: 3 type: IBT gid 0 flags: 0x10000000000000 nid: local remote: 0x0 expref: -99 pid: 6257 timeout: 0 lvb_type: 0 00010000:00010000:1.0:1776178204.418751:0:6257:0:(ldlm_request.c:1026:ldlm_cli_enqueue()) ### client-side enqueue START, flags 0x1000001000000 ns: lustre-MDT0000-osp-MDT0001 lock: 0000000056462e0c/0xa40083ab9bf5377f lrc: 3/0,1 mode: --/EX res: [0x200000007:0x1:0x0].0x0 bits 0x2/0x0 rrc: 3 type: IBT gid 0 flags: 0x0 nid: local remote: 0x0 expref: -99 pid: 6257 timeout: 0 lvb_type: 0 00010000:00010000:1.0:1776178204.418762:0:6257:0:(ldlm_request.c:1114:ldlm_cli_enqueue()) ### sending request ns: lustre-MDT0000-osp-MDT0001 lock: 0000000056462e0c/0xa40083ab9bf5377f lrc: 3/0,1 mode: --/EX res: [0x200000007:0x1:0x0].0x0 bits 0x2/0x0 rrc: 3 type: IBT gid 0 flags: 0x1000000000000 nid: local remote: 0x0 expref: -99 pid: 6257 timeout: 0 lvb_type: 0 00000100:00100000:1.0:1776178204.418787:0:6257:0:(client.c:1733:ptlrpc_send_new_req()) Sending RPC req@00000000eeeaf4ad pname:cluuid:pid:xid:nid:opc:job mdt00_002:lustre-MDT0001-mdtlov_UUID:6257:1862456998046208:0@lo:101:mdt00_002.0 00000100:00100000:1.0:1776178204.418841:0:6257:0:(events.c:361:request_in_callback()) peer: 12345-0@lo (source: 12345-0@lo) 00000100:00100000:1.0:1776178204.418868:0:6257:0:(client.c:2500:ptlrpc_set_wait()) set 00000000c557aaed going to sleep for 42 seconds 00000100:00100000:3.0:1776178204.419195:0:6261:0:(service.c:2152:ptlrpc_server_handle_req_in()) got req x1862456998046208 00000100:00100000:3.0:1776178204.419285:0:6261:0:(nrs_fifo.c:179:nrs_fifo_req_get()) NRS start fifo request from 12345-0@lo, seq: 695 00000100:00100000:3.0:1776178204.419290:0:6261:0:(service.c:2324:ptlrpc_server_handle_request()) Handling RPC req@0000000040899e7d pname:cluuid+ref:pid:xid:nid:opc:job mdt_out00_001:lustre-MDT0001-mdtlov_UUID+6:6257:x1862456998046208:12345-0@lo:101:mdt00_002.0 00010000:00010000:3.0:1776178204.419304:0:6261:0:(ldlm_lockd.c:1248:ldlm_handle_enqueue0()) ### server-side enqueue handler START 00010000:00010000:3.0:1776178204.419323:0:6261:0:(ldlm_lockd.c:1318:ldlm_handle_enqueue0()) ### server-side enqueue handler, new lock created ns: mdt-lustre-MDT0000_UUID lock: 000000006cb7334a/0xa40083ab9bf53786 lrc: 2/0,0 mode: --/EX res: [0x200000007:0x1:0x0].0x0 bits 0x0/0x0 rrc: 4 type: IBT gid 0 flags: 0x40000000000000 nid: local remote: 0xa40083ab9bf5377f expref: -99 pid: 6261 timeout: 0 lvb_type: 0 00010000:00010000:3.0:1776178204.419359:0:6261:0:(ldlm_lock.c:1064:ldlm_granted_list_add_lock()) ### About to add lock: ns: mdt-lustre-MDT0000_UUID lock: 000000006cb7334a/0xa40083ab9bf53786 lrc: 3/0,0 mode: EX/EX res: [0x200000007:0x1:0x0].0x0 bits 0x2/0x0 rrc: 4 type: IBT gid 0 flags: 0x50000001000000 nid: 0@lo remote: 0xa40083ab9bf5377f expref: 7 pid: 6261 timeout: 0 lvb_type: 0 00010000:00010000:3.0:1776178204.419375:0:6261:0:(ldlm_lockd.c:1484:ldlm_handle_enqueue0()) ### server-side enqueue handler, sending reply (err=0, rc=0) ns: mdt-lustre-MDT0000_UUID lock: 000000006cb7334a/0xa40083ab9bf53786 lrc: 3/0,0 mode: EX/EX res: [0x200000007:0x1:0x0].0x0 bits 0x2/0x0 rrc: 4 type: IBT gid 0 flags: 0x40000001000000 nid: 0@lo remote: 0xa40083ab9bf5377f expref: 7 pid: 6261 timeout: 0 lvb_type: 0 00010000:00010000:3.0:1776178204.419382:0:6261:0:(ldlm_inodebits.c:94:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x200000007:0x1:0x0].0x0 (00000000ad0d4094) 00010000:00010000:3.0:1776178204.419385:0:6261:0:(ldlm_inodebits.c:96:ldlm_reprocess_inodebits_queue()) Hint 2 00010000:00010000:3.0:1776178204.419388:0:6261:0:(ldlm_lockd.c:1565:ldlm_handle_enqueue0()) ### server-side enqueue handler END (lock 000000006cb7334a, rc 0) 00000100:00100000:3.0:1776178204.419439:0:6261:0:(service.c:2372:ptlrpc_server_handle_request()) Handled RPC req@0000000040899e7d pname:cluuid+ref:pid:xid:nid:opc:job mdt_out00_001:lustre-MDT0001-mdtlov_UUID+7:6257:x1862456998046208:12345-0@lo:101:mdt00_002.0 Request processed in 148us (597us total) trans 0 rc 0/0 00000100:00100000:3.0:1776178204.419446:0:6261:0:(nrs_fifo.c:241:nrs_fifo_req_stop()) NRS stop fifo request from 12345-0@lo, seq: 695 00000100:00100000:1.0:1776178204.419535:0:6257:0:(client.c:2215:ptlrpc_check_set()) Completed RPC req@00000000eeeaf4ad pname:cluuid:pid:xid:nid:opc:job mdt00_002:lustre-MDT0001-mdtlov_UUID:6257:1862456998046208:0@lo:101:mdt00_002.0 00010000:00010000:1.0:1776178204.419553:0:6257:0:(ldlm_lock.c:1064:ldlm_granted_list_add_lock()) ### About to add lock: ns: lustre-MDT0000-osp-MDT0001 lock: 0000000056462e0c/0xa40083ab9bf5377f lrc: 4/0,1 mode: EX/EX res: [0x200000007:0x1:0x0].0x0 bits 0x2/0x0 rrc: 3 type: IBT gid 0 flags: 0x11000001000000 nid: local remote: 0xa40083ab9bf53786 expref: -99 pid: 6257 timeout: 0 lvb_type: 0 00010000:00010000:1.0:1776178204.419563:0:6257:0:(ldlm_request.c:796:ldlm_cli_enqueue_fini()) ### client-side enqueue END ns: lustre-MDT0000-osp-MDT0001 lock: 0000000056462e0c/0xa40083ab9bf5377f lrc: 4/0,1 mode: EX/EX res: [0x200000007:0x1:0x0].0x0 bits 0x2/0x0 rrc: 3 type: IBT gid 0 flags: 0x1000001000000 nid: local remote: 0xa40083ab9bf53786 expref: -99 pid: 6257 timeout: 0 lvb_type: 0 00000004:00080000:1.0:1776178204.419585:0:6257:0:(osp_object.c:1389:osp_invalidate()) Invalidate osp_object [0x200000007:0x1:0x0] 00000100:00100000:1.0:1776178204.419631:0:6257:0:(client.c:1733:ptlrpc_send_new_req()) Sending RPC req@00000000eeeaf4ad pname:cluuid:pid:xid:nid:opc:job mdt00_002:lustre-MDT0001-mdtlov_UUID:6257:1862456998046272:0@lo:1000:mdt00_002.0 00000100:00100000:1.0:1776178204.419687:0:6257:0:(events.c:361:request_in_callback()) peer: 12345-0@lo (source: 12345-0@lo) 00000100:00100000:1.0:1776178204.420623:0:6257:0:(client.c:2500:ptlrpc_set_wait()) set 00000000780f0fb4 going to sleep for 42 seconds 00000100:00100000:0.0:1776178204.420724:0:14128:0:(service.c:2152:ptlrpc_server_handle_req_in()) got req x1862456998046272 00000100:00100000:0.0:1776178204.420787:0:14128:0:(nrs_fifo.c:179:nrs_fifo_req_get()) NRS start fifo request from 12345-0@lo, seq: 696 00000100:00100000:0.0:1776178204.420793:0:14128:0:(service.c:2324:ptlrpc_server_handle_request()) Handling RPC req@000000006ce0dfc8 pname:cluuid+ref:pid:xid:nid:opc:job mdt_out00_002:lustre-MDT0001-mdtlov_UUID+7:6257:x1862456998046272:12345-0@lo:1000:mdt00_002.0 00000100:00100000:0.0:1776178204.420905:0:14128:0:(service.c:2372:ptlrpc_server_handle_request()) Handled RPC req@000000006ce0dfc8 pname:cluuid+ref:pid:xid:nid:opc:job mdt_out00_002:lustre-MDT0001-mdtlov_UUID+7:6257:x1862456998046272:12345-0@lo:1000:mdt00_002.0 Request processed in 112us (1218us total) trans 0 rc 0/0 00000100:00100000:0.0:1776178204.420913:0:14128:0:(nrs_fifo.c:241:nrs_fifo_req_stop()) NRS stop fifo request from 12345-0@lo, seq: 696 00000100:00100000:1.0:1776178204.421027:0:6257:0:(client.c:2215:ptlrpc_check_set()) Completed RPC req@00000000eeeaf4ad pname:cluuid:pid:xid:nid:opc:job mdt00_002:lustre-MDT0001-mdtlov_UUID:6257:1862456998046272:0@lo:1000:mdt00_002.0 00000100:00100000:1.0:1776178204.421122:0:6257:0:(client.c:1733:ptlrpc_send_new_req()) Sending RPC req@00000000eeeaf4ad pname:cluuid:pid:xid:nid:opc:job mdt00_002:lustre-MDT0001-mdtlov_UUID:6257:1862456998046336:0@lo:1000:mdt00_002.0 00000100:00100000:1.0:1776178204.421165:0:6257:0:(events.c:361:request_in_callback()) peer: 12345-0@lo (source: 12345-0@lo) 00000100:00100000:1.0:1776178204.421186:0:6257:0:(client.c:2500:ptlrpc_set_wait()) set 00000000780f0fb4 going to sleep for 42 seconds 00000100:00100000:0.0:1776178204.421493:0:6260:0:(service.c:2152:ptlrpc_server_handle_req_in()) got req x1862456998046336 00000100:00100000:0.0:1776178204.421527:0:6260:0:(nrs_fifo.c:179:nrs_fifo_req_get()) NRS start fifo request from 12345-0@lo, seq: 697 00000100:00100000:0.0:1776178204.421532:0:6260:0:(service.c:2324:ptlrpc_server_handle_request()) Handling RPC req@00000000758405de pname:cluuid+ref:pid:xid:nid:opc:job mdt_out00_000:lustre-MDT0001-mdtlov_UUID+7:6257:x1862456998046336:12345-0@lo:1000:mdt00_002.0 00000100:00100000:0.0:1776178204.421627:0:6260:0:(service.c:2372:ptlrpc_server_handle_request()) Handled RPC req@00000000758405de pname:cluuid+ref:pid:xid:nid:opc:job mdt_out00_000:lustre-MDT0001-mdtlov_UUID+7:6257:x1862456998046336:12345-0@lo:1000:mdt00_002.0 Request processed in 94us (462us total) trans 0 rc 1/1 00000100:00100000:0.0:1776178204.421634:0:6260:0:(nrs_fifo.c:241:nrs_fifo_req_stop()) NRS stop fifo request from 12345-0@lo, seq: 697 00000100:00100000:1.0:1776178204.421721:0:6257:0:(client.c:2215:ptlrpc_check_set()) Completed RPC req@00000000eeeaf4ad pname:cluuid:pid:xid:nid:opc:job mdt00_002:lustre-MDT0001-mdtlov_UUID:6257:1862456998046336:0@lo:1000:mdt00_002.0 00010000:00010000:1.0:1776178204.421783:0:6257:0:(ldlm_lock.c:766:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(EX) ns: lustre-MDT0000-osp-MDT0001 lock: 00000000b2e83690/0xa40083ab9bf5378d lrc: 3/0,1 mode: --/EX res: [0x240000402:0x7f7:0x0].0x0 bits 0x0/0x0 rrc: 2 type: IBT gid 0 flags: 0x10000000000000 nid: local remote: 0x0 expref: -99 pid: 6257 timeout: 0 lvb_type: 0 00010000:00010000:1.0:1776178204.421794:0:6257:0:(ldlm_request.c:1026:ldlm_cli_enqueue()) ### client-side enqueue START, flags 0x1000001000000 ns: lustre-MDT0000-osp-MDT0001 lock: 00000000b2e83690/0xa40083ab9bf5378d lrc: 3/0,1 mode: --/EX res: [0x240000402:0x7f7:0x0].0x0 bits 0x1/0x0 rrc: 2 type: IBT gid 0 flags: 0x0 nid: local remote: 0x0 expref: -99 pid: 6257 timeout: 0 lvb_type: 0 00010000:00010000:1.0:1776178204.421804:0:6257:0:(ldlm_request.c:1114:ldlm_cli_enqueue()) ### sending request ns: lustre-MDT0000-osp-MDT0001 lock: 00000000b2e83690/0xa40083ab9bf5378d lrc: 3/0,1 mode: --/EX res: [0x240000402:0x7f7:0x0].0x0 bits 0x1/0x0 rrc: 2 type: IBT gid 0 flags: 0x1000000000000 nid: local remote: 0x0 expref: -99 pid: 6257 timeout: 0 lvb_type: 0 00000100:00100000:1.0:1776178204.421816:0:6257:0:(client.c:1733:ptlrpc_send_new_req()) Sending RPC req@00000000eeeaf4ad pname:cluuid:pid:xid:nid:opc:job mdt00_002:lustre-MDT0001-mdtlov_UUID:6257:1862456998046400:0@lo:101:mdt00_002.0 00000100:00100000:1.0:1776178204.421840:0:6257:0:(events.c:361:request_in_callback()) peer: 12345-0@lo (source: 12345-0@lo) 00000100:00100000:1.0:1776178204.421896:0:6257:0:(client.c:2500:ptlrpc_set_wait()) set 00000000c557aaed going to sleep for 42 seconds 00000100:00100000:3.0:1776178204.421969:0:6261:0:(service.c:2152:ptlrpc_server_handle_req_in()) got req x1862456998046400 00000100:00100000:2.0:1776178204.422339:0:15213:0:(nrs_fifo.c:179:nrs_fifo_req_get()) NRS start fifo request from 12345-0@lo, seq: 698 00000100:00100000:2.0:1776178204.422344:0:15213:0:(service.c:2324:ptlrpc_server_handle_request()) Handling RPC req@0000000053c46e8d pname:cluuid+ref:pid:xid:nid:opc:job mdt_out00_003:lustre-MDT0001-mdtlov_UUID+7:6257:x1862456998046400:12345-0@lo:101:mdt00_002.0 00010000:00010000:2.0:1776178204.422355:0:15213:0:(ldlm_lockd.c:1248:ldlm_handle_enqueue0()) ### server-side enqueue handler START 00010000:00010000:2.0:1776178204.422373:0:15213:0:(ldlm_lockd.c:1318:ldlm_handle_enqueue0()) ### server-side enqueue handler, new lock created ns: mdt-lustre-MDT0000_UUID lock: 0000000073f2c7a9/0xa40083ab9bf53794 lrc: 2/0,0 mode: --/EX res: [0x240000402:0x7f7:0x0].0x0 bits 0x0/0x0 rrc: 2 type: IBT gid 0 flags: 0x40000000000000 nid: local remote: 0xa40083ab9bf5378d expref: -99 pid: 15213 timeout: 0 lvb_type: 0 00010000:00010000:2.0:1776178204.422397:0:15213:0:(ldlm_lock.c:1064:ldlm_granted_list_add_lock()) ### About to add lock: ns: mdt-lustre-MDT0000_UUID lock: 0000000073f2c7a9/0xa40083ab9bf53794 lrc: 3/0,0 mode: EX/EX res: [0x240000402:0x7f7:0x0].0x0 bits 0x1/0x0 rrc: 2 type: IBT gid 0 flags: 0x50000001000000 nid: 0@lo remote: 0xa40083ab9bf5378d expref: 8 pid: 15213 timeout: 0 lvb_type: 0 00010000:00010000:2.0:1776178204.422409:0:15213:0:(ldlm_lockd.c:1484:ldlm_handle_enqueue0()) ### server-side enqueue handler, sending reply (err=0, rc=0) ns: mdt-lustre-MDT0000_UUID lock: 0000000073f2c7a9/0xa40083ab9bf53794 lrc: 3/0,0 mode: EX/EX res: [0x240000402:0x7f7:0x0].0x0 bits 0x1/0x0 rrc: 2 type: IBT gid 0 flags: 0x40000001000000 nid: 0@lo remote: 0xa40083ab9bf5378d expref: 8 pid: 15213 timeout: 0 lvb_type: 0 00010000:00010000:2.0:1776178204.422417:0:15213:0:(ldlm_inodebits.c:94:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x240000402:0x7f7:0x0].0x0 (000000003c4b13ad) 00010000:00010000:2.0:1776178204.422420:0:15213:0:(ldlm_inodebits.c:96:ldlm_reprocess_inodebits_queue()) Hint 1 00010000:00010000:2.0:1776178204.422423:0:15213:0:(ldlm_lockd.c:1565:ldlm_handle_enqueue0()) ### server-side enqueue handler END (lock 0000000073f2c7a9, rc 0) 00000100:00100000:2.0:1776178204.422454:0:15213:0:(service.c:2372:ptlrpc_server_handle_request()) Handled RPC req@0000000053c46e8d pname:cluuid+ref:pid:xid:nid:opc:job mdt_out00_003:lustre-MDT0001-mdtlov_UUID+8:6257:x1862456998046400:12345-0@lo:101:mdt00_002.0 Request processed in 109us (613us total) trans 0 rc 0/0 00000100:00100000:2.0:1776178204.422460:0:15213:0:(nrs_fifo.c:241:nrs_fifo_req_stop()) NRS stop fifo request from 12345-0@lo, seq: 698 00000100:00100000:3.0:1776178204.422524:0:6257:0:(client.c:2215:ptlrpc_check_set()) Completed RPC req@00000000eeeaf4ad pname:cluuid:pid:xid:nid:opc:job mdt00_002:lustre-MDT0001-mdtlov_UUID:6257:1862456998046400:0@lo:101:mdt00_002.0 00010000:00010000:3.0:1776178204.422538:0:6257:0:(ldlm_lock.c:1064:ldlm_granted_list_add_lock()) ### About to add lock: ns: lustre-MDT0000-osp-MDT0001 lock: 00000000b2e83690/0xa40083ab9bf5378d lrc: 4/0,1 mode: EX/EX res: [0x240000402:0x7f7:0x0].0x0 bits 0x1/0x0 rrc: 2 type: IBT gid 0 flags: 0x11000001000000 nid: local remote: 0xa40083ab9bf53794 expref: -99 pid: 6257 timeout: 0 lvb_type: 0 00010000:00010000:3.0:1776178204.422547:0:6257:0:(ldlm_request.c:796:ldlm_cli_enqueue_fini()) ### client-side enqueue END ns: lustre-MDT0000-osp-MDT0001 lock: 00000000b2e83690/0xa40083ab9bf5378d lrc: 4/0,1 mode: EX/EX res: [0x240000402:0x7f7:0x0].0x0 bits 0x1/0x0 rrc: 2 type: IBT gid 0 flags: 0x1000001000000 nid: local remote: 0xa40083ab9bf53794 expref: -99 pid: 6257 timeout: 0 lvb_type: 0 00010000:00010000:3.0:1776178204.422572:0:6257:0:(ldlm_lock.c:766:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(EX) ns: mdt-lustre-MDT0001_UUID lock: 000000002a990613/0xa40083ab9bf5379b lrc: 3/0,1 mode: --/EX res: [0x240000402:0x7f7:0x0].0x0 bits 0x0/0x0 rrc: 2 type: IBT gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 6257 timeout: 0 lvb_type: 0 00010000:00010000:3.0:1776178204.422580:0:6257:0:(ldlm_lock.c:1064:ldlm_granted_list_add_lock()) ### About to add lock: ns: mdt-lustre-MDT0001_UUID lock: 000000002a990613/0xa40083ab9bf5379b lrc: 3/0,1 mode: EX/EX res: [0x240000402:0x7f7:0x0].0x0 bits 0x2/0x0 rrc: 2 type: IBT gid 0 flags: 0x50210001000000 nid: local remote: 0x0 expref: -99 pid: 6257 timeout: 0 lvb_type: 0 00010000:00010000:3.0:1776178204.422588:0:6257:0:(ldlm_request.c:528:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: mdt-lustre-MDT0001_UUID lock: 000000002a990613/0xa40083ab9bf5379b lrc: 3/0,1 mode: EX/EX res: [0x240000402:0x7f7:0x0].0x0 bits 0x2/0x0 rrc: 2 type: IBT gid 0 flags: 0x40210001000000 nid: local remote: 0x0 expref: -99 pid: 6257 timeout: 0 lvb_type: 0 00000040:00080000:3.0:1776178204.422968:0:6257:0:(llog_osd.c:738:llog_osd_write_rec()) added record [0x240001b70:0x1:0x0].10, 432 off40768 00000040:00080000:3.0:1776178204.423040:0:6257:0:(llog_osd.c:738:llog_osd_write_rec()) added record [0x200000bd1:0x1:0x0].9, 512 off39456 00000004:00080000:3.0:1776178204.423044:0:6257:0:(osp_trans.c:1598:osp_trans_stop()) lustre-MDT0000-osp-MDT0001: add oth 000000003a5eaf8c with version 22 00000004:00080000:3.0:1776178204.423088:0:18102:0:(osp_trans.c:1343:osp_get_next_request()) ou 000000009121b27e version 22 rpc_version 22 00000100:00100000:3.0:1776178204.423121:0:18102:0:(client.c:1733:ptlrpc_send_new_req()) Sending RPC req@0000000095c019b3 pname:cluuid:pid:xid:nid:opc:job osp_up0-1:lustre-MDT0001-mdtlov_UUID:18102:1862456998046464:0@lo:1000:osp_up0-1.0 00000100:00100000:3.0:1776178204.423322:0:18102:0:(events.c:361:request_in_callback()) peer: 12345-0@lo (source: 12345-0@lo) 00000100:00100000:3.0:1776178204.423787:0:18102:0:(client.c:2500:ptlrpc_set_wait()) set 000000000cb56daf going to sleep for 42 seconds 00000100:00100000:0.0:1776178204.423871:0:14128:0:(service.c:2152:ptlrpc_server_handle_req_in()) got req x1862456998046464 00000100:00100000:1.0:1776178204.423976:0:14128:0:(nrs_fifo.c:179:nrs_fifo_req_get()) NRS start fifo request from 12345-0@lo, seq: 699 00000100:00100000:1.0:1776178204.423982:0:14128:0:(service.c:2324:ptlrpc_server_handle_request()) Handling RPC req@00000000ad01e608 pname:cluuid+ref:pid:xid:nid:opc:job mdt_out00_002:lustre-MDT0001-mdtlov_UUID+8:18102:x1862456998046464:12345-0@lo:1000:osp_up0-1.0 00000100:00100000:1.0:1776178204.424819:0:14128:0:(service.c:2372:ptlrpc_server_handle_request()) Handled RPC req@00000000ad01e608 pname:cluuid+ref:pid:xid:nid:opc:job mdt_out00_002:lustre-MDT0001-mdtlov_UUID+9:18102:x1862456998046464:12345-0@lo:1000:osp_up0-1.0 Request processed in 833us (1494us total) trans 12884901946 rc 0/0 00000100:00100000:1.0:1776178204.424829:0:14128:0:(nrs_fifo.c:241:nrs_fifo_req_stop()) NRS stop fifo request from 12345-0@lo, seq: 699 00000100:00100000:3.0:1776178204.425007:0:18102:0:(client.c:2215:ptlrpc_check_set()) Completed RPC req@0000000095c019b3 pname:cluuid:pid:xid:nid:opc:job osp_up0-1:lustre-MDT0001-mdtlov_UUID:18102:1862456998046464:0@lo:1000:osp_up0-1.0 00000004:00200000:1.0:1776178204.425380:0:6257:0:(mdt_handler.c:912:mdt_pack_attr2body()) [0x240000402:0x7f7:0x0]: returning size 4096 00000004:00010000:1.0:1776178204.425402:0:6257:0:(mdt_handler.c:3992:mdt_save_lock()) ### save lock request 000000008c469ec0 reply state 00000000ada31b8f transno 17179876275 ns: mdt-lustre-MDT0001_UUID lock: 000000002a990613/0xa40083ab9bf5379b lrc: 3/0,1 mode: EX/EX res: [0x240000402:0x7f7:0x0].0x0 bits 0x2/0x0 rrc: 2 type: IBT gid 0 flags: 0x40210001000000 nid: local remote: 0x0 expref: -99 pid: 6257 timeout: 0 lvb_type: 0 00010000:00010000:1.0:1776178204.425418:0:6257:0:(ldlm_lock.c:820:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(EX) ns: lustre-MDT0000-osp-MDT0001 lock: 00000000b2e83690/0xa40083ab9bf5378d lrc: 4/0,1 mode: EX/EX res: [0x240000402:0x7f7:0x0].0x0 bits 0x1/0x0 rrc: 2 type: IBT gid 0 flags: 0x11000001000000 nid: local remote: 0xa40083ab9bf53794 expref: -99 pid: 6257 timeout: 0 lvb_type: 0 00010000:00010000:1.0:1776178204.425426:0:6257:0:(ldlm_lock.c:909:ldlm_lock_decref_internal()) ### do not add lock into lru list ns: lustre-MDT0000-osp-MDT0001 lock: 00000000b2e83690/0xa40083ab9bf5378d lrc: 3/0,0 mode: EX/EX res: [0x240000402:0x7f7:0x0].0x0 bits 0x1/0x0 rrc: 2 type: IBT gid 0 flags: 0x11000001000000 nid: local remote: 0xa40083ab9bf53794 expref: -99 pid: 6257 timeout: 0 lvb_type: 0 00000004:00080000:1.0:1776178204.425560:0:6257:0:(osp_object.c:1389:osp_invalidate()) Invalidate osp_object [0x200000007:0x1:0x0] 00010000:00010000:1.0:1776178204.425566:0:6257:0:(ldlm_lock.c:820:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(EX) ns: lustre-MDT0000-osp-MDT0001 lock: 0000000056462e0c/0xa40083ab9bf5377f lrc: 4/0,1 mode: EX/EX res: [0x200000007:0x1:0x0].0x0 bits 0x2/0x0 rrc: 3 type: IBT gid 0 flags: 0x11000001000000 nid: local remote: 0xa40083ab9bf53786 expref: -99 pid: 6257 timeout: 0 lvb_type: 0 00010000:00010000:1.0:1776178204.425578:0:6257:0:(ldlm_lock.c:909:ldlm_lock_decref_internal()) ### do not add lock into lru list ns: lustre-MDT0000-osp-MDT0001 lock: 0000000056462e0c/0xa40083ab9bf5377f lrc: 3/0,0 mode: EX/EX res: [0x200000007:0x1:0x0].0x0 bits 0x2/0x0 rrc: 3 type: IBT gid 0 flags: 0x11000001000000 nid: local remote: 0xa40083ab9bf53786 expref: -99 pid: 6257 timeout: 0 lvb_type: 0 00000100:00100000:1.0:1776178204.425706:0:6257:0:(service.c:2372:ptlrpc_server_handle_request()) Handled RPC req@000000008c469ec0 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_002:b50963bd-8eba-4432-9a54-8888d00e3fa1+26:17671:x1862456979102208:12345-192.168.204.3@tcp:36:rm.0 Request processed in 7166us (7352us total) trans 17179876275 rc 0/0 00000100:00100000:1.0:1776178204.425715:0:6257:0:(nrs_fifo.c:241:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.204.3@tcp, seq: 6593 00010000:00010000:3.0:1776178204.433516:0:3340:0:(ldlm_lock.c:1064:ldlm_granted_list_add_lock()) ### About to add lock: ns: mdt-lustre-MDT0001_UUID lock: 000000002a990613/0xa40083ab9bf5379b lrc: 3/0,1 mode: COS/COS res: [0x240000402:0x7f7:0x0].0x0 bits 0x2/0x0 rrc: 2 type: IBT gid 0 flags: 0x50210001000000 nid: local remote: 0x0 expref: -99 pid: 6257 timeout: 0 lvb_type: 0 00010000:00010000:3.0:1776178204.433544:0:3340:0:(ldlm_inodebits.c:94:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x240000402:0x7f7:0x0].0x0 (00000000b27056f4) 00010000:00010000:3.0:1776178204.433547:0:3340:0:(ldlm_inodebits.c:96:ldlm_reprocess_inodebits_queue()) Hint 2 00010000:00010000:3.0:1776178205.164916:0:6244:0:(ldlm_lockd.c:2792:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:2.0:1776178205.536241:0:3353:0:(client.c:1733:ptlrpc_send_new_req()) Sending RPC req@0000000044f82452 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_00:lustre-MDT0000-mdtlov_UUID:3353:1862456998046528:0@lo:13:osp-pre-1-0.0 00000100:00100000:2.0:1776178205.536350:0:3353:0:(events.c:361:request_in_callback()) peer: 12345-0@lo (source: 12345-0@lo) 00000100:00100000:2.0:1776178205.536420:0:9554:0:(service.c:2152:ptlrpc_server_handle_req_in()) got req x1862456998046528 00000100:00100000:2.0:1776178205.536480:0:9554:0:(nrs_fifo.c:179:nrs_fifo_req_get()) NRS start fifo request from 12345-0@lo, seq: 479 00000100:00100000:2.0:1776178205.536486:0:9554:0:(service.c:2324:ptlrpc_server_handle_request()) Handling RPC req@00000000d28be450 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_create00:lustre-MDT0000-mdtlov_UUID+5:3353:x1862456998046528:12345-0@lo:13:osp-pre-1-0.0 00002000:00000024:2.0:1776178205.536517:0:9554:0:(ofd_obd.c:740:ofd_statfs()) blocks cached 229376 granted 491746368 pending 0 free 3923689472 avail 3692167168 00000100:00100000:2.0:1776178205.536563:0:9554:0:(service.c:2372:ptlrpc_server_handle_request()) Handled RPC req@00000000d28be450 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_create00:lustre-MDT0000-mdtlov_UUID+5:3353:x1862456998046528:12345-0@lo:13:osp-pre-1-0.0 Request processed in 75us (214us total) trans 0 rc 0/0 00000100:00100000:2.0:1776178205.536570:0:9554:0:(nrs_fifo.c:241:nrs_fifo_req_stop()) NRS stop fifo request from 12345-0@lo, seq: 479 00000100:00100000:2.0:1776178205.536640:0:3353:0:(client.c:2215:ptlrpc_check_set()) Completed RPC req@0000000044f82452 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_00:lustre-MDT0000-mdtlov_UUID:3353:1862456998046528:0@lo:13:osp-pre-1-0.0 00000100:00100000:0.0:1776178205.539107:0:3354:0:(client.c:1733:ptlrpc_send_new_req()) Sending RPC req@0000000023210d1c pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_01:lustre-MDT0000-mdtlov_UUID:3354:1862456998046592:0@lo:13:osp-pre-0-0.0 00000100:00100000:0.0:1776178205.539280:0:3354:0:(events.c:361:request_in_callback()) peer: 12345-0@lo (source: 12345-0@lo) 00000100:00100000:0.0:1776178205.539354:0:8231:0:(service.c:2152:ptlrpc_server_handle_req_in()) got req x1862456998046592 00000100:00100000:0.0:1776178205.539397:0:8231:0:(nrs_fifo.c:179:nrs_fifo_req_get()) NRS start fifo request from 12345-0@lo, seq: 480 00000100:00100000:0.0:1776178205.539403:0:8231:0:(service.c:2324:ptlrpc_server_handle_request()) Handling RPC req@00000000dd1281fe pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_create00:lustre-MDT0000-mdtlov_UUID+5:3354:x1862456998046592:12345-0@lo:13:osp-pre-0-0.0 00002000:00000024:0.0:1776178205.539437:0:8231:0:(ofd_obd.c:740:ofd_statfs()) blocks cached 315392 granted 484883776 pending 0 free 3923689472 avail 3692167168 00000100:00100000:0.0:1776178205.539480:0:8231:0:(service.c:2372:ptlrpc_server_handle_request()) Handled RPC req@00000000dd1281fe pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_create00:lustre-MDT0000-mdtlov_UUID+5:3354:x1862456998046592:12345-0@lo:13:osp-pre-0-0.0 Request processed in 76us (203us total) trans 0 rc 0/0 00000100:00100000:0.0:1776178205.539487:0:8231:0:(nrs_fifo.c:241:nrs_fifo_req_stop()) NRS stop fifo request from 12345-0@lo, seq: 480 00000100:00100000:0.0:1776178205.539619:0:3354:0:(client.c:2215:ptlrpc_check_set()) Completed RPC req@0000000023210d1c pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_01:lustre-MDT0000-mdtlov_UUID:3354:1862456998046592:0@lo:13:osp-pre-0-0.0 00000100:00100000:1.0:1776178205.541507:0:3356:0:(ptlrpcd.c:415:ptlrpcd_check()) transfer 1 async RPCs [2->3] 00000100:00100000:1.0:1776178205.541526:0:3356:0:(client.c:1733:ptlrpc_send_new_req()) Sending RPC req@000000000cde9837 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_03:lustre-MDT0001-mdtlov_UUID:3356:1862456998046656:0@lo:41:osp-pre-0-1.0 00000100:00100000:1.0:1776178205.541585:0:3356:0:(events.c:361:request_in_callback()) peer: 12345-0@lo (source: 12345-0@lo) 00000100:00100000:1.0:1776178205.541932:0:15213:0:(service.c:2152:ptlrpc_server_handle_req_in()) got req x1862456998046656 00000100:00100000:1.0:1776178205.541979:0:15213:0:(nrs_fifo.c:179:nrs_fifo_req_get()) NRS start fifo request from 12345-0@lo, seq: 700 00000100:00100000:1.0:1776178205.542364:0:15213:0:(service.c:2324:ptlrpc_server_handle_request()) Handling RPC req@0000000016c7a5ff pname:cluuid+ref:pid:xid:nid:opc:job mdt_out00_003:lustre-MDT0001-mdtlov_UUID+9:3356:x1862456998046656:12345-0@lo:41:osp-pre-0-1.0 00000004:00000024:1.0:1776178205.542405:0:15213:0:(mdt_handler.c:548:mdt_statfs()) blocks cached 0 granted 2097152 pending 0 free 1446125568 avail 1316663296 00000100:00100000:1.0:1776178205.542463:0:15213:0:(service.c:2372:ptlrpc_server_handle_request()) Handled RPC req@0000000016c7a5ff pname:cluuid+ref:pid:xid:nid:opc:job mdt_out00_003:lustre-MDT0001-mdtlov_UUID+9:3356:x1862456998046656:12345-0@lo:41:osp-pre-0-1.0 Request processed in 100us (878us total) trans 0 rc 0/0 00000100:00100000:1.0:1776178205.542472:0:15213:0:(nrs_fifo.c:241:nrs_fifo_req_stop()) NRS stop fifo request from 12345-0@lo, seq: 700 00000100:00100000:1.0:1776178205.542527:0:3356:0:(client.c:2215:ptlrpc_check_set()) Completed RPC req@000000000cde9837 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_03:lustre-MDT0001-mdtlov_UUID:3356:1862456998046656:0@lo:41:osp-pre-0-1.0 00000100:00100000:3.0:1776178205.543043:0:3355:0:(client.c:1733:ptlrpc_send_new_req()) Sending RPC req@0000000058333a54 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_02:lustre-MDT0001-lwp-OST0000_UUID:3355:1862456998046912:0@lo:400:kworker/u8:1.0 00000100:00100000:3.0:1776178205.543113:0:3355:0:(events.c:361:request_in_callback()) peer: 12345-0@lo (source: 12345-0@lo) 00000100:00100000:3.0:1776178205.543136:0:3355:0:(client.c:1733:ptlrpc_send_new_req()) Sending RPC req@00000000931093aa pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_02:lustre-MDT0001-mdtlov_UUID:3355:1862456998047168:0@lo:400:kworker/u8:1.0 00000100:00100000:3.0:1776178205.543160:0:3355:0:(events.c:361:request_in_callback()) peer: 12345-0@lo (source: 12345-0@lo) 00000100:00100000:3.0:1776178205.543178:0:3355:0:(ptlrpcd.c:415:ptlrpcd_check()) transfer 3 async RPCs [3->2] 00000100:00100000:3.0:1776178205.543183:0:3355:0:(client.c:1733:ptlrpc_send_new_req()) Sending RPC req@00000000a3697847 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_02:lustre-MDT0000-mdtlov_UUID:3355:1862456998046720:0@lo:41:osp-pre-1-0.0 00000100:00100000:3.0:1776178205.543207:0:3355:0:(events.c:361:request_in_callback()) peer: 12345-0@lo (source: 12345-0@lo) 00000100:00100000:3.0:1776178205.543225:0:3355:0:(client.c:1733:ptlrpc_send_new_req()) Sending RPC req@00000000779279f4 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_02:lustre-MDT0000-lwp-OST0001_UUID:3355:1862456998046976:0@lo:400:kworker/u8:1.0 00000100:00100000:3.0:1776178205.543249:0:3355:0:(events.c:361:request_in_callback()) peer: 12345-0@lo (source: 12345-0@lo) 00000100:00100000:3.0:1776178205.543260:0:3355:0:(client.c:1733:ptlrpc_send_new_req()) Sending RPC req@00000000f093b599 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_02:lustre-MDT0000-lwp-MDT0001_UUID:3355:1862456998047232:0@lo:400:kworker/u8:1.0 00000100:00100000:3.0:1776178205.543284:0:3355:0:(events.c:361:request_in_callback()) peer: 12345-0@lo (source: 12345-0@lo) 00000100:00100000:3.0:1776178205.543312:0:3353:0:(client.c:1733:ptlrpc_send_new_req()) Sending RPC req@0000000068680f90 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_00:ac61f839-0e8e-4cac-9c59-08bcfbbad330:3353:1862456998046784:0@lo:400:kworker/u8:1.0 00000100:00100000:3.0:1776178205.543345:0:3353:0:(events.c:361:request_in_callback()) peer: 12345-0@lo (source: 12345-0@lo) 00000100:00100000:3.0:1776178205.543364:0:3353:0:(client.c:1733:ptlrpc_send_new_req()) Sending RPC req@00000000dc1ea8e7 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_00:lustre-MDT0001-lwp-OST0001_UUID:3353:1862456998047040:0@lo:400:kworker/u8:1.0 00000100:00100000:3.0:1776178205.543385:0:3353:0:(events.c:361:request_in_callback()) peer: 12345-0@lo (source: 12345-0@lo) 00000100:00100000:3.0:1776178205.543401:0:3353:0:(client.c:1733:ptlrpc_send_new_req()) Sending RPC req@00000000a5decd95 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_00:lustre-MDT0000-mdtlov_UUID:3353:1862456998047296:0@lo:400:kworker/u8:1.0 00000100:00100000:3.0:1776178205.543421:0:3353:0:(events.c:361:request_in_callback()) peer: 12345-0@lo (source: 12345-0@lo) 00000100:00100000:3.0:1776178205.543439:0:3353:0:(ptlrpcd.c:415:ptlrpcd_check()) transfer 3 async RPCs [1->0] 00000100:00100000:3.0:1776178205.543444:0:3353:0:(client.c:1733:ptlrpc_send_new_req()) Sending RPC req@00000000bc2ea964 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_00:lustre-MDT0000-lwp-OST0000_UUID:3353:1862456998046848:0@lo:400:kworker/u8:1.0 00000100:00100000:3.0:1776178205.543469:0:3353:0:(events.c:361:request_in_callback()) peer: 12345-0@lo (source: 12345-0@lo) 00000100:00100000:3.0:1776178205.543484:0:3353:0:(client.c:1733:ptlrpc_send_new_req()) Sending RPC req@00000000be1ebf42 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_00:lustre-MDT0001-mdtlov_UUID:3353:1862456998047104:0@lo:400:kworker/u8:1.0 00000100:00100000:3.0:1776178205.543507:0:3353:0:(events.c:361:request_in_callback()) peer: 12345-0@lo (source: 12345-0@lo) 00000100:00100000:3.0:1776178205.543522:0:3353:0:(client.c:1733:ptlrpc_send_new_req()) Sending RPC req@000000008c938150 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_00:lustre-MDT0000-lwp-MDT0000_UUID:3353:1862456998047360:0@lo:400:kworker/u8:1.0 00000100:00100000:3.0:1776178205.543544:0:3353:0:(events.c:361:request_in_callback()) peer: 12345-0@lo (source: 12345-0@lo) 00000100:00100000:3.0:1776178205.543571:0:20759:0:(service.c:2152:ptlrpc_server_handle_req_in()) got req x1862456998047168 00000100:00100000:3.0:1776178205.543604:0:20759:0:(service.c:2152:ptlrpc_server_handle_req_in()) got req x1862456998047104 00000100:00100000:0.0:1776178205.543866:0:11281:0:(service.c:2152:ptlrpc_server_handle_req_in()) got req x1862456998046912 00000100:00100000:0.0:1776178205.543890:0:11281:0:(service.c:2152:ptlrpc_server_handle_req_in()) got req x1862456998046976 00000100:00100000:0.0:1776178205.543901:0:11281:0:(service.c:2152:ptlrpc_server_handle_req_in()) got req x1862456998047232 00000100:00100000:0.0:1776178205.543912:0:11281:0:(service.c:2152:ptlrpc_server_handle_req_in()) got req x1862456998047040 00000100:00100000:0.0:1776178205.543921:0:11281:0:(service.c:2152:ptlrpc_server_handle_req_in()) got req x1862456998046848 00000100:00100000:0.0:1776178205.543930:0:11281:0:(service.c:2152:ptlrpc_server_handle_req_in()) got req x1862456998047360 00000100:00100000:0.0:1776178205.543941:0:11281:0:(nrs_fifo.c:179:nrs_fifo_req_get()) NRS start fifo request from 12345-0@lo, seq: 910 00000100:00100000:0.0:1776178205.543946:0:11281:0:(service.c:2324:ptlrpc_server_handle_request()) Handling RPC req@0000000088f30966 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_004:lustre-MDT0001-lwp-OST0000_UUID+5:3355:x1862456998046912:12345-0@lo:400:kworker/u8:1.0 00000100:00100000:0.0:1776178205.544004:0:11281:0:(service.c:2372:ptlrpc_server_handle_request()) Handled RPC req@0000000088f30966 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_004:lustre-MDT0001-lwp-OST0000_UUID+5:3355:x1862456998046912:12345-0@lo:400:kworker/u8:1.0 Request processed in 57us (891us total) trans 0 rc 0/0 00000100:00100000:0.0:1776178205.544011:0:11281:0:(nrs_fifo.c:241:nrs_fifo_req_stop()) NRS stop fifo request from 12345-0@lo, seq: 910 00000100:00100000:0.0:1776178205.544029:0:11281:0:(nrs_fifo.c:179:nrs_fifo_req_get()) NRS start fifo request from 12345-0@lo, seq: 911 00000100:00100000:0.0:1776178205.544032:0:11281:0:(service.c:2324:ptlrpc_server_handle_request()) Handling RPC req@00000000bad9c860 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_004:lustre-MDT0000-lwp-OST0001_UUID+5:3355:x1862456998046976:12345-0@lo:400:kworker/u8:1.0 00000100:00100000:0.0:1776178205.544077:0:11281:0:(service.c:2372:ptlrpc_server_handle_request()) Handled RPC req@00000000bad9c860 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_004:lustre-MDT0000-lwp-OST0001_UUID+5:3355:x1862456998046976:12345-0@lo:400:kworker/u8:1.0 Request processed in 43us (828us total) trans 0 rc 0/0 00000100:00100000:0.0:1776178205.544084:0:11281:0:(nrs_fifo.c:241:nrs_fifo_req_stop()) NRS stop fifo request from 12345-0@lo, seq: 911 00000100:00100000:0.0:1776178205.544098:0:11281:0:(nrs_fifo.c:179:nrs_fifo_req_get()) NRS start fifo request from 12345-0@lo, seq: 912 00000100:00100000:0.0:1776178205.544101:0:11281:0:(service.c:2324:ptlrpc_server_handle_request()) Handling RPC req@00000000f301f00a pname:cluuid+ref:pid:xid:nid:opc:job mdt00_004:lustre-MDT0000-lwp-MDT0001_UUID+5:3355:x1862456998047232:12345-0@lo:400:kworker/u8:1.0 00000100:00100000:0.0:1776178205.544129:0:11281:0:(service.c:2372:ptlrpc_server_handle_request()) Handled RPC req@00000000f301f00a pname:cluuid+ref:pid:xid:nid:opc:job mdt00_004:lustre-MDT0000-lwp-MDT0001_UUID+5:3355:x1862456998047232:12345-0@lo:400:kworker/u8:1.0 Request processed in 27us (845us total) trans 0 rc 0/0 00000100:00100000:0.0:1776178205.544135:0:11281:0:(nrs_fifo.c:241:nrs_fifo_req_stop()) NRS stop fifo request from 12345-0@lo, seq: 912 00000100:00100000:0.0:1776178205.544149:0:11281:0:(nrs_fifo.c:179:nrs_fifo_req_get()) NRS start fifo request from 12345-0@lo, seq: 913 00000100:00100000:0.0:1776178205.544153:0:11281:0:(service.c:2324:ptlrpc_server_handle_request()) Handling RPC req@000000001d6fd88e pname:cluuid+ref:pid:xid:nid:opc:job mdt00_004:lustre-MDT0001-lwp-OST0001_UUID+5:3353:x1862456998047040:12345-0@lo:400:kworker/u8:1.0 00000100:00100000:0.0:1776178205.544181:0:11281:0:(service.c:2372:ptlrpc_server_handle_request()) Handled RPC req@000000001d6fd88e pname:cluuid+ref:pid:xid:nid:opc:job mdt00_004:lustre-MDT0001-lwp-OST0001_UUID+5:3353:x1862456998047040:12345-0@lo:400:kworker/u8:1.0 Request processed in 28us (796us total) trans 0 rc 0/0 00000100:00100000:0.0:1776178205.544187:0:11281:0:(nrs_fifo.c:241:nrs_fifo_req_stop()) NRS stop fifo request from 12345-0@lo, seq: 913 00000100:00100000:0.0:1776178205.544197:0:11281:0:(nrs_fifo.c:179:nrs_fifo_req_get()) NRS start fifo request from 12345-0@lo, seq: 914 00000100:00100000:0.0:1776178205.544200:0:11281:0:(service.c:2324:ptlrpc_server_handle_request()) Handling RPC req@0000000023c02f47 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_004:lustre-MDT0000-lwp-OST0000_UUID+5:3353:x1862456998046848:12345-0@lo:400:kworker/u8:1.0 00000100:00100000:0.0:1776178205.544225:0:11281:0:(service.c:2372:ptlrpc_server_handle_request()) Handled RPC req@0000000023c02f47 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_004:lustre-MDT0000-lwp-OST0000_UUID+5:3353:x1862456998046848:12345-0@lo:400:kworker/u8:1.0 Request processed in 24us (755us total) trans 0 rc 0/0 00000100:00100000:0.0:1776178205.544231:0:11281:0:(nrs_fifo.c:241:nrs_fifo_req_stop()) NRS stop fifo request from 12345-0@lo, seq: 914 00000100:00100000:0.0:1776178205.544241:0:11281:0:(nrs_fifo.c:179:nrs_fifo_req_get()) NRS start fifo request from 12345-0@lo, seq: 915 00000100:00100000:0.0:1776178205.544244:0:11281:0:(service.c:2324:ptlrpc_server_handle_request()) Handling RPC req@00000000337eb961 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_004:lustre-MDT0000-lwp-MDT0000_UUID+5:3353:x1862456998047360:12345-0@lo:400:kworker/u8:1.0 00000100:00100000:0.0:1776178205.544267:0:11281:0:(service.c:2372:ptlrpc_server_handle_request()) Handled RPC req@00000000337eb961 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_004:lustre-MDT0000-lwp-MDT0000_UUID+5:3353:x1862456998047360:12345-0@lo:400:kworker/u8:1.0 Request processed in 22us (722us total) trans 0 rc 0/0 00000100:00100000:0.0:1776178205.544271:0:11281:0:(nrs_fifo.c:241:nrs_fifo_req_stop()) NRS stop fifo request from 12345-0@lo, seq: 915 00000100:00100000:3.0:1776178205.547333:0:20759:0:(nrs_fifo.c:179:nrs_fifo_req_get()) NRS start fifo request from 12345-0@lo, seq: 512 00000100:00100000:3.0:1776178205.547370:0:20759:0:(service.c:2324:ptlrpc_server_handle_request()) Handling RPC req@000000000390171d pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_010:lustre-MDT0001-mdtlov_UUID+5:3355:x1862456998047168:12345-0@lo:400:kworker/u8:1.0 00000100:00100000:3.0:1776178205.547466:0:20759:0:(service.c:2372:ptlrpc_server_handle_request()) Handled RPC req@000000000390171d pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_010:lustre-MDT0001-mdtlov_UUID+5:3355:x1862456998047168:12345-0@lo:400:kworker/u8:1.0 Request processed in 113us (4306us total) trans 0 rc 0/0 00000100:00100000:3.0:1776178205.547474:0:20759:0:(nrs_fifo.c:241:nrs_fifo_req_stop()) NRS stop fifo request from 12345-0@lo, seq: 512 00000100:00100000:3.0:1776178205.547507:0:20759:0:(nrs_fifo.c:179:nrs_fifo_req_get()) NRS start fifo request from 12345-0@lo, seq: 513 00000100:00100000:3.0:1776178205.547512:0:20759:0:(service.c:2324:ptlrpc_server_handle_request()) Handling RPC req@000000008b793df7 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_010:lustre-MDT0001-mdtlov_UUID+5:3353:x1862456998047104:12345-0@lo:400:kworker/u8:1.0 00000100:00100000:3.0:1776178205.547561:0:20759:0:(service.c:2372:ptlrpc_server_handle_request()) Handled RPC req@000000008b793df7 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_010:lustre-MDT0001-mdtlov_UUID+5:3353:x1862456998047104:12345-0@lo:400:kworker/u8:1.0 Request processed in 49us (4054us total) trans 0 rc 0/0 00000100:00100000:3.0:1776178205.547568:0:20759:0:(nrs_fifo.c:241:nrs_fifo_req_stop()) NRS stop fifo request from 12345-0@lo, seq: 513 00000100:00100000:3.0:1776178205.547602:0:14128:0:(service.c:2152:ptlrpc_server_handle_req_in()) got req x1862456998046720 00000100:00100000:3.0:1776178205.547646:0:14128:0:(service.c:2152:ptlrpc_server_handle_req_in()) got req x1862456998047296 00000100:00100000:3.0:1776178205.547680:0:14128:0:(nrs_fifo.c:179:nrs_fifo_req_get()) NRS start fifo request from 12345-0@lo, seq: 701 00000100:00100000:3.0:1776178205.547685:0:14128:0:(service.c:2324:ptlrpc_server_handle_request()) Handling RPC req@0000000009593c4b pname:cluuid+ref:pid:xid:nid:opc:job mdt_out00_002:lustre-MDT0000-mdtlov_UUID+6:3355:x1862456998046720:12345-0@lo:41:osp-pre-1-0.0 00000004:00000024:3.0:1776178205.547723:0:14128:0:(mdt_handler.c:548:mdt_statfs()) blocks cached 0 granted 2097152 pending 0 free 1446039552 avail 1316577280 00000100:00100000:3.0:1776178205.547759:0:14128:0:(service.c:2372:ptlrpc_server_handle_request()) Handled RPC req@0000000009593c4b pname:cluuid+ref:pid:xid:nid:opc:job mdt_out00_002:lustre-MDT0000-mdtlov_UUID+6:3355:x1862456998046720:12345-0@lo:41:osp-pre-1-0.0 Request processed in 73us (4552us total) trans 0 rc 0/0 00000100:00100000:3.0:1776178205.547766:0:14128:0:(nrs_fifo.c:241:nrs_fifo_req_stop()) NRS stop fifo request from 12345-0@lo, seq: 701 00000100:00100000:3.0:1776178205.547778:0:14128:0:(nrs_fifo.c:179:nrs_fifo_req_get()) NRS start fifo request from 12345-0@lo, seq: 702 00000100:00100000:3.0:1776178205.547782:0:14128:0:(service.c:2324:ptlrpc_server_handle_request()) Handling RPC req@000000008f4a0165 pname:cluuid+ref:pid:xid:nid:opc:job mdt_out00_002:lustre-MDT0000-mdtlov_UUID+5:3353:x1862456998047296:12345-0@lo:400:kworker/u8:1.0 00000100:00100000:3.0:1776178205.547812:0:14128:0:(service.c:2372:ptlrpc_server_handle_request()) Handled RPC req@000000008f4a0165 pname:cluuid+ref:pid:xid:nid:opc:job mdt_out00_002:lustre-MDT0000-mdtlov_UUID+5:3353:x1862456998047296:12345-0@lo:400:kworker/u8:1.0 Request processed in 29us (4390us total) trans 0 rc 0/0 00000100:00100000:3.0:1776178205.547819:0:14128:0:(nrs_fifo.c:241:nrs_fifo_req_stop()) NRS stop fifo request from 12345-0@lo, seq: 702 00000100:00100000:3.0:1776178205.547868:0:20137:0:(service.c:2152:ptlrpc_server_handle_req_in()) got req x1862456998046784 00000100:00100000:3.0:1776178205.547898:0:20137:0:(nrs_fifo.c:179:nrs_fifo_req_get()) NRS start fifo request from 12345-0@lo, seq: 157 00000100:00100000:3.0:1776178205.547902:0:20137:0:(service.c:2324:ptlrpc_server_handle_request()) Handling RPC req@0000000021ee30f5 pname:cluuid+ref:pid:xid:nid:opc:job ll_mgs_0000:ac61f839-0e8e-4cac-9c59-08bcfbbad330+23:3353:x1862456998046784:12345-0@lo:400:kworker/u8:1.0 00000100:00100000:3.0:1776178205.547938:0:20137:0:(service.c:2372:ptlrpc_server_handle_request()) Handled RPC req@0000000021ee30f5 pname:cluuid+ref:pid:xid:nid:opc:job ll_mgs_0000:ac61f839-0e8e-4cac-9c59-08bcfbbad330+23:3353:x1862456998046784:12345-0@lo:400:kworker/u8:1.0 Request processed in 36us (4593us total) trans 0 rc 0/0 00000100:00100000:3.0:1776178205.547949:0:20137:0:(nrs_fifo.c:241:nrs_fifo_req_stop()) NRS stop fifo request from 12345-0@lo, seq: 157 00000100:00100000:3.0:1776178205.548083:0:3355:0:(client.c:2215:ptlrpc_check_set()) Completed RPC req@00000000a3697847 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_02:lustre-MDT0000-mdtlov_UUID:3355:1862456998046720:0@lo:41:osp-pre-1-0.0 00000100:00100000:3.0:1776178205.548105:0:3355:0:(client.c:2215:ptlrpc_check_set()) Completed RPC req@00000000779279f4 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_02:lustre-MDT0000-lwp-OST0001_UUID:3355:1862456998046976:0@lo:400:kworker/u8:1.0 00000100:00100000:3.0:1776178205.548129:0:3355:0:(client.c:2215:ptlrpc_check_set()) Completed RPC req@00000000f093b599 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_02:lustre-MDT0000-lwp-MDT0001_UUID:3355:1862456998047232:0@lo:400:kworker/u8:1.0 00000100:00100000:3.0:1776178205.548141:0:3355:0:(client.c:2215:ptlrpc_check_set()) Completed RPC req@0000000058333a54 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_02:lustre-MDT0001-lwp-OST0000_UUID:3355:1862456998046912:0@lo:400:kworker/u8:1.0 00000100:00100000:3.0:1776178205.548153:0:3355:0:(client.c:2215:ptlrpc_check_set()) Completed RPC req@00000000931093aa pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_02:lustre-MDT0001-mdtlov_UUID:3355:1862456998047168:0@lo:400:kworker/u8:1.0 00000100:00100000:3.0:1776178205.548209:0:3353:0:(client.c:2215:ptlrpc_check_set()) Completed RPC req@00000000bc2ea964 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_00:lustre-MDT0000-lwp-OST0000_UUID:3353:1862456998046848:0@lo:400:kworker/u8:1.0 00000100:00100000:3.0:1776178205.548226:0:3353:0:(client.c:2215:ptlrpc_check_set()) Completed RPC req@00000000be1ebf42 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_00:lustre-MDT0001-mdtlov_UUID:3353:1862456998047104:0@lo:400:kworker/u8:1.0 00000100:00100000:3.0:1776178205.548245:0:3353:0:(client.c:2215:ptlrpc_check_set()) Completed RPC req@000000008c938150 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_00:lustre-MDT0000-lwp-MDT0000_UUID:3353:1862456998047360:0@lo:400:kworker/u8:1.0 00000100:00100000:3.0:1776178205.548253:0:3353:0:(client.c:2215:ptlrpc_check_set()) Completed RPC req@0000000068680f90 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_00:ac61f839-0e8e-4cac-9c59-08bcfbbad330:3353:1862456998046784:0@lo:400:kworker/u8:1.0 00000100:00100000:3.0:1776178205.548265:0:3353:0:(client.c:2215:ptlrpc_check_set()) Completed RPC req@00000000dc1ea8e7 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_00:lustre-MDT0001-lwp-OST0001_UUID:3353:1862456998047040:0@lo:400:kworker/u8:1.0 00000100:00100000:3.0:1776178205.548274:0:3353:0:(client.c:2215:ptlrpc_check_set()) Completed RPC req@00000000a5decd95 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_00:lustre-MDT0000-mdtlov_UUID:3353:1862456998047296:0@lo:400:kworker/u8:1.0 00010000:00010000:2.0:1776178206.177327:0:9224:0:(ldlm_lockd.c:2792:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:0.0:1776178207.073321:0:3355:0:(client.c:1733:ptlrpc_send_new_req()) Sending RPC req@00000000a5decd95 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_02:lustre-MDT0001-mdtlov_UUID:3355:1862456998047424:0@lo:13:osp-pre-1-1.0 00000004:00080000:3.0:1776178207.073321:0:18107:0:(osp_sync.c:1658:osp_sync_add_commit_cb()) lustre-OST0001-osc-MDT0001: add commit cb at 914400257714ns, next at 915400254445ns, rc = 0 00000100:00100000:0.0:1776178207.073420:0:3355:0:(events.c:361:request_in_callback()) peer: 12345-0@lo (source: 12345-0@lo) 00000100:00100000:3.0:1776178207.074230:0:8232:0:(service.c:2152:ptlrpc_server_handle_req_in()) got req x1862456998047424 00000100:00100000:3.0:1776178207.074293:0:8232:0:(nrs_fifo.c:179:nrs_fifo_req_get()) NRS start fifo request from 12345-0@lo, seq: 481 00000100:00100000:3.0:1776178207.074300:0:8232:0:(service.c:2324:ptlrpc_server_handle_request()) Handling RPC req@0000000023210d1c pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_create00:lustre-MDT0001-mdtlov_UUID+5:3355:x1862456998047424:12345-0@lo:13:osp-pre-1-1.0 00002000:00000024:3.0:1776178207.074334:0:8232:0:(ofd_obd.c:740:ofd_statfs()) blocks cached 229376 granted 491746368 pending 0 free 3923689472 avail 3692167168 00000100:00100000:3.0:1776178207.074400:0:8232:0:(service.c:2372:ptlrpc_server_handle_request()) Handled RPC req@0000000023210d1c pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_create00:lustre-MDT0001-mdtlov_UUID+5:3355:x1862456998047424:12345-0@lo:13:osp-pre-1-1.0 Request processed in 99us (982us total) trans 0 rc 0/0 00000100:00100000:3.0:1776178207.074408:0:8232:0:(nrs_fifo.c:241:nrs_fifo_req_stop()) NRS stop fifo request from 12345-0@lo, seq: 481 00000100:00100000:1.0:1776178207.074724:0:3355:0:(client.c:2215:ptlrpc_check_set()) Completed RPC req@00000000a5decd95 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_02:lustre-MDT0001-mdtlov_UUID:3355:1862456998047424:0@lo:13:osp-pre-1-1.0 00010000:00010000:1.0:1776178207.200469:0:9225:0:(ldlm_lockd.c:2792:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000004:00080000:0.0:1776178207.590858:0:18105:0:(osp_sync.c:1658:osp_sync_add_commit_cb()) lustre-OST0000-osc-MDT0001: add commit cb at 914917795365ns, next at 915917792898ns, rc = 0 00000100:00100000:3.0:1776178207.590933:0:3355:0:(ptlrpcd.c:415:ptlrpcd_check()) transfer 1 async RPCs [3->2] 00000100:00100000:3.0:1776178207.590955:0:3355:0:(client.c:1733:ptlrpc_send_new_req()) Sending RPC req@00000000f0ef812c pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_02:lustre-MDT0001-mdtlov_UUID:3355:1862456998047488:0@lo:13:osp-pre-0-1.0 00000100:00100000:3.0:1776178207.591037:0:3355:0:(events.c:361:request_in_callback()) peer: 12345-0@lo (source: 12345-0@lo) 00000100:00100000:0.0:1776178207.591285:0:9554:0:(service.c:2152:ptlrpc_server_handle_req_in()) got req x1862456998047488 00000100:00100000:0.0:1776178207.591356:0:9554:0:(nrs_fifo.c:179:nrs_fifo_req_get()) NRS start fifo request from 12345-0@lo, seq: 482 00000100:00100000:0.0:1776178207.591366:0:9554:0:(service.c:2324:ptlrpc_server_handle_request()) Handling RPC req@00000000311b2d82 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_create00:lustre-MDT0001-mdtlov_UUID+5:3355:x1862456998047488:12345-0@lo:13:osp-pre-0-1.0 00002000:00000024:0.0:1776178207.591406:0:9554:0:(ofd_obd.c:740:ofd_statfs()) blocks cached 315392 granted 484883776 pending 0 free 3923689472 avail 3692167168 00000100:00100000:0.0:1776178207.591464:0:9554:0:(service.c:2372:ptlrpc_server_handle_request()) Handled RPC req@00000000311b2d82 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_create00:lustre-MDT0001-mdtlov_UUID+5:3355:x1862456998047488:12345-0@lo:13:osp-pre-0-1.0 Request processed in 97us (429us total) trans 0 rc 0/0 00000100:00100000:0.0:1776178207.591471:0:9554:0:(nrs_fifo.c:241:nrs_fifo_req_stop()) NRS stop fifo request from 12345-0@lo, seq: 482 00000100:00100000:1.0:1776178207.591626:0:3355:0:(client.c:2215:ptlrpc_check_set()) Completed RPC req@00000000f0ef812c pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_02:lustre-MDT0001-mdtlov_UUID:3355:1862456998047488:0@lo:13:osp-pre-0-1.0 00010000:00010000:1.0:1776178208.224282:0:6245:0:(ldlm_lockd.c:2792:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:3.0:1776178209.064233:0:3347:0:(events.c:361:request_in_callback()) peer: 12345-192.168.204.3@tcp (source: 12345-192.168.204.3@tcp) 00000100:00100000:1.0:1776178209.064484:0:20140:0:(service.c:2152:ptlrpc_server_handle_req_in()) got req x1862456979102272 00000100:00100000:1.0:1776178209.064825:0:20140:0:(nrs_fifo.c:179:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.204.3@tcp, seq: 158 00000100:00100000:1.0:1776178209.064836:0:20140:0:(service.c:2324:ptlrpc_server_handle_request()) Handling RPC req@000000006efdb84e pname:cluuid+ref:pid:xid:nid:opc:job ll_mgs_0002:5b5c66b1-1a2f-4651-8548-0c75b9a38a94+9:2270:x1862456979102272:12345-192.168.204.3@tcp:400:kworker/u8:3.0 00000100:00100000:1.0:1776178209.064964:0:20140:0:(service.c:2372:ptlrpc_server_handle_request()) Handled RPC req@000000006efdb84e pname:cluuid+ref:pid:xid:nid:opc:job ll_mgs_0002:5b5c66b1-1a2f-4651-8548-0c75b9a38a94+9:2270:x1862456979102272:12345-192.168.204.3@tcp:400:kworker/u8:3.0 Request processed in 129us (745us total) trans 0 rc 0/0 00000100:00100000:1.0:1776178209.064972:0:20140:0:(nrs_fifo.c:241:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.204.3@tcp, seq: 158 00000100:00100000:1.0:1776178209.066550:0:3348:0:(events.c:361:request_in_callback()) peer: 12345-192.168.204.3@tcp (source: 12345-192.168.204.3@tcp) 00000100:00100000:1.0:1776178209.066616:0:3348:0:(events.c:361:request_in_callback()) peer: 12345-192.168.204.3@tcp (source: 12345-192.168.204.3@tcp) 00000100:00100000:1.0:1776178209.066643:0:3348:0:(events.c:361:request_in_callback()) peer: 12345-192.168.204.3@tcp (source: 12345-192.168.204.3@tcp) 00000100:00100000:1.0:1776178209.066666:0:3348:0:(events.c:361:request_in_callback()) peer: 12345-192.168.204.3@tcp (source: 12345-192.168.204.3@tcp) 00000100:00100000:3.0:1776178209.066975:0:6256:0:(service.c:2152:ptlrpc_server_handle_req_in()) got req x1862456979102336 00000100:00100000:3.0:1776178209.067055:0:6256:0:(service.c:2152:ptlrpc_server_handle_req_in()) got req x1862456979102400 00000100:00100000:3.0:1776178209.067111:0:6256:0:(nrs_fifo.c:179:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.204.3@tcp, seq: 916 00000100:00100000:3.0:1776178209.067118:0:6256:0:(service.c:2324:ptlrpc_server_handle_request()) Handling RPC req@000000000cde9837 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_001:b50963bd-8eba-4432-9a54-8888d00e3fa1+6:2270:x1862456979102336:12345-192.168.204.3@tcp:400:kworker/u8:3.0 00000100:00100000:3.0:1776178209.067210:0:6256:0:(service.c:2372:ptlrpc_server_handle_request()) Handled RPC req@000000000cde9837 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_001:b50963bd-8eba-4432-9a54-8888d00e3fa1+6:2270:x1862456979102336:12345-192.168.204.3@tcp:400:kworker/u8:3.0 Request processed in 91us (670us total) trans 0 rc 0/0 00000100:00100000:3.0:1776178209.067219:0:6256:0:(nrs_fifo.c:241:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.204.3@tcp, seq: 916 00000100:00100000:3.0:1776178209.067238:0:6256:0:(nrs_fifo.c:179:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.204.3@tcp, seq: 917 00000100:00100000:3.0:1776178209.067243:0:6256:0:(service.c:2324:ptlrpc_server_handle_request()) Handling RPC req@000000004c44fad7 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_001:b50963bd-8eba-4432-9a54-8888d00e3fa1+26:2271:x1862456979102400:12345-192.168.204.3@tcp:400:kworker/u8:3.0 00000100:00100000:3.0:1776178209.067272:0:6256:0:(service.c:2372:ptlrpc_server_handle_request()) Handled RPC req@000000004c44fad7 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_001:b50963bd-8eba-4432-9a54-8888d00e3fa1+26:2271:x1862456979102400:12345-192.168.204.3@tcp:400:kworker/u8:3.0 Request processed in 28us (629us total) trans 0 rc 0/0 00000100:00100000:3.0:1776178209.067278:0:6256:0:(nrs_fifo.c:241:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.204.3@tcp, seq: 917 00000100:00100000:3.0:1776178209.067299:0:20762:0:(service.c:2152:ptlrpc_server_handle_req_in()) got req x1862456979102464 00000100:00100000:3.0:1776178209.067325:0:20762:0:(service.c:2152:ptlrpc_server_handle_req_in()) got req x1862456979102528 00000100:00100000:3.0:1776178209.067343:0:20762:0:(nrs_fifo.c:179:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.204.3@tcp, seq: 514 00000100:00100000:3.0:1776178209.067347:0:20762:0:(service.c:2324:ptlrpc_server_handle_request()) Handling RPC req@00000000854f8d52 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_013:b50963bd-8eba-4432-9a54-8888d00e3fa1+11:2271:x1862456979102464:12345-192.168.204.3@tcp:400:kworker/u8:3.0 00000100:00100000:3.0:1776178209.067375:0:20762:0:(service.c:2372:ptlrpc_server_handle_request()) Handled RPC req@00000000854f8d52 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_013:b50963bd-8eba-4432-9a54-8888d00e3fa1+11:2271:x1862456979102464:12345-192.168.204.3@tcp:400:kworker/u8:3.0 Request processed in 27us (770us total) trans 0 rc 0/0 00000100:00100000:3.0:1776178209.067381:0:20762:0:(nrs_fifo.c:241:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.204.3@tcp, seq: 514 00000100:00100000:3.0:1776178209.067391:0:20762:0:(nrs_fifo.c:179:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.204.3@tcp, seq: 515 00000100:00100000:3.0:1776178209.067395:0:20762:0:(service.c:2324:ptlrpc_server_handle_request()) Handling RPC req@00000000d7e84c97 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_013:b50963bd-8eba-4432-9a54-8888d00e3fa1+13:2271:x1862456979102528:12345-192.168.204.3@tcp:400:kworker/u8:3.0 00000100:00100000:3.0:1776178209.067417:0:20762:0:(service.c:2372:ptlrpc_server_handle_request()) Handled RPC req@00000000d7e84c97 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_013:b50963bd-8eba-4432-9a54-8888d00e3fa1+13:2271:x1862456979102528:12345-192.168.204.3@tcp:400:kworker/u8:3.0 Request processed in 22us (751us total) trans 0 rc 0/0 00000100:00100000:3.0:1776178209.067422:0:20762:0:(nrs_fifo.c:241:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.204.3@tcp, seq: 515 00010000:00010000:3.0:1776178209.249533:0:6244:0:(ldlm_lockd.c:2792:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000001:00080000:2.0:1776178209.636534:0:9217:0:(tgt_lastrcvd.c:913:tgt_cb_last_committed()) lustre-OST0001: transno 4294969331 is committed 00000001:00080000:0.0:1776178209.637160:0:8225:0:(tgt_lastrcvd.c:913:tgt_cb_last_committed()) lustre-OST0000: transno 4294969358 is committed 00000004:00080000:1.0:1776178209.637312:0:18097:0:(osp_sync.c:1623:osp_sync_local_commit_cb()) lustre-OST0000-osc-MDT0001: 31 committed 00000004:00080000:1.0:1776178209.637339:0:18097:0:(osp_sync.c:1623:osp_sync_local_commit_cb()) lustre-OST0001-osc-MDT0001: 31 committed 00010000:00010000:1.0:1776178209.637384:0:18097:0:(ldlm_request.c:1232:ldlm_cli_cancel_local()) ### client-side cancel ns: lustre-MDT0000-osp-MDT0001 lock: 0000000056462e0c/0xa40083ab9bf5377f lrc: 3/0,0 mode: EX/EX res: [0x200000007:0x1:0x0].0x0 bits 0x2/0x0 rrc: 3 type: IBT gid 0 flags: 0x1008401000000 nid: local remote: 0xa40083ab9bf53786 expref: -99 pid: 6257 timeout: 0 lvb_type: 0 00000004:00010000:1.0:1776178209.637403:0:18097:0:(mdt_handler.c:3608:mdt_remote_blocking_ast()) ### Revoke remote lock ns: lustre-MDT0000-osp-MDT0001 lock: 0000000056462e0c/0xa40083ab9bf5377f lrc: 3/0,0 mode: EX/EX res: [0x200000007:0x1:0x0].0x0 bits 0x2/0x0 rrc: 3 type: IBT gid 0 flags: 0x1009401000000 nid: local remote: 0xa40083ab9bf53786 expref: -99 pid: 6257 timeout: 0 lvb_type: 0 00010000:00010000:1.0:1776178209.637457:0:18097:0:(ldlm_request.c:1279:__ldlm_pack_lock()) ### packing ns: lustre-MDT0000-osp-MDT0001 lock: 0000000056462e0c/0xa40083ab9bf5377f lrc: 2/0,0 mode: --/EX res: [0x200000007:0x1:0x0].0x0 bits 0x2/0x0 rrc: 3 type: IBT gid 0 flags: 0x5809401000020 nid: local remote: 0xa40083ab9bf53786 expref: -99 pid: 6257 timeout: 0 lvb_type: 0 00010000:00010000:1.0:1776178209.637464:0:18097:0:(ldlm_request.c:1325:ldlm_cancel_pack()) 1 locks packed 00010000:00010000:1.0:1776178209.637480:0:18097:0:(ldlm_lock.c:216:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: lustre-MDT0000-osp-MDT0001 lock: 0000000056462e0c/0xa40083ab9bf5377f lrc: 0/0,0 mode: --/EX res: [0x200000007:0x1:0x0].0x0 bits 0x2/0x0 rrc: 3 type: IBT gid 0 flags: 0x5809401000020 nid: local remote: 0xa40083ab9bf53786 expref: -99 pid: 6257 timeout: 0 lvb_type: 0 00010000:00010000:1.0:1776178209.637493:0:18097:0:(ldlm_request.c:1232:ldlm_cli_cancel_local()) ### client-side cancel ns: lustre-MDT0000-osp-MDT0001 lock: 00000000b2e83690/0xa40083ab9bf5378d lrc: 3/0,0 mode: EX/EX res: [0x240000402:0x7f7:0x0].0x0 bits 0x1/0x0 rrc: 2 type: IBT gid 0 flags: 0x1008401000000 nid: local remote: 0xa40083ab9bf53794 expref: -99 pid: 6257 timeout: 0 lvb_type: 0 00000004:00010000:1.0:1776178209.637499:0:18097:0:(mdt_handler.c:3608:mdt_remote_blocking_ast()) ### Revoke remote lock ns: lustre-MDT0000-osp-MDT0001 lock: 00000000b2e83690/0xa40083ab9bf5378d lrc: 3/0,0 mode: EX/EX res: [0x240000402:0x7f7:0x0].0x0 bits 0x1/0x0 rrc: 2 type: IBT gid 0 flags: 0x1009401000000 nid: local remote: 0xa40083ab9bf53794 expref: -99 pid: 6257 timeout: 0 lvb_type: 0 00000100:00100000:3.0:1776178209.637511:0:3353:0:(client.c:1733:ptlrpc_send_new_req()) Sending RPC req@00000000f95e9da5 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_00:lustre-MDT0001-mdtlov_UUID:3353:1862456998047616:0@lo:6:osp-syn-1-1.0 00010000:00010000:1.0:1776178209.637512:0:18097:0:(ldlm_request.c:1279:__ldlm_pack_lock()) ### packing ns: lustre-MDT0000-osp-MDT0001 lock: 00000000b2e83690/0xa40083ab9bf5378d lrc: 2/0,0 mode: --/EX res: [0x240000402:0x7f7:0x0].0x0 bits 0x1/0x0 rrc: 2 type: IBT gid 0 flags: 0x5809401000020 nid: local remote: 0xa40083ab9bf53794 expref: -99 pid: 6257 timeout: 0 lvb_type: 0 00010000:00010000:1.0:1776178209.637527:0:18097:0:(ldlm_request.c:1325:ldlm_cancel_pack()) 1 locks packed 00010000:00010000:1.0:1776178209.637530:0:18097:0:(ldlm_lock.c:216:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: lustre-MDT0000-osp-MDT0001 lock: 00000000b2e83690/0xa40083ab9bf5378d lrc: 0/0,0 mode: --/EX res: [0x240000402:0x7f7:0x0].0x0 bits 0x1/0x0 rrc: 2 type: IBT gid 0 flags: 0x5809401000020 nid: local remote: 0xa40083ab9bf53794 expref: -99 pid: 6257 timeout: 0 lvb_type: 0 00000001:00080000:1.0:1776178209.637544:0:18097:0:(tgt_lastrcvd.c:913:tgt_cb_last_committed()) lustre-MDT0001: transno 17179876275 is committed 00000004:00080000:1.0:1776178209.637569:0:18097:0:(osp_sync.c:1623:osp_sync_local_commit_cb()) lustre-OST0001-osc-MDT0001: 30 committed 00000004:00080000:1.0:1776178209.637588:0:18097:0:(osp_sync.c:1623:osp_sync_local_commit_cb()) lustre-OST0000-osc-MDT0001: 30 committed 00000100:00100000:1.0:1776178209.637618:0:3354:0:(client.c:1733:ptlrpc_send_new_req()) Sending RPC req@00000000ab295186 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_01:lustre-MDT0001-mdtlov_UUID:3354:1862456998047552:0@lo:103:jbd2/dm-1-8.0 00000100:00100000:3.0:1776178209.637619:0:3353:0:(events.c:361:request_in_callback()) peer: 12345-0@lo (source: 12345-0@lo) 00000100:00100000:0.0:1776178209.637681:0:20759:0:(service.c:2152:ptlrpc_server_handle_req_in()) got req x1862456998047616 00000100:00100000:0.0:1776178209.637831:0:20759:0:(nrs_fifo.c:179:nrs_fifo_req_get()) NRS start fifo request from 12345-0@lo, seq: 5278 00000100:00100000:0.0:1776178209.637839:0:20759:0:(service.c:2324:ptlrpc_server_handle_request()) Handling RPC req@00000000dc1ea8e7 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_010:lustre-MDT0001-mdtlov_UUID+5:3353:x1862456998047616:12345-0@lo:6:osp-syn-1-1.0 00010000:00010000:2.0:1776178209.637868:0:3339:0:(ldlm_lock.c:820:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(COS) ns: mdt-lustre-MDT0001_UUID lock: 000000006cc9c357/0xa40083ab9bf53698 lrc: 3/0,1 mode: COS/COS res: [0x240000402:0x7f7:0x0].0x37639fea bits 0x2/0x0 rrc: 2 type: IBT gid 0 flags: 0x50210000000000 nid: local remote: 0x0 expref: -99 pid: 11281 timeout: 0 lvb_type: 0 00002000:00080000:0.0:1776178209.637873:0:20759:0:(ofd_dev.c:1822:ofd_destroy_hdl()) lustre-OST0001: Destroy object 0x2c0000400:1070 count 1 00010000:00010000:2.0:1776178209.637880:0:3339:0:(ldlm_lock.c:879:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: mdt-lustre-MDT0001_UUID lock: 000000006cc9c357/0xa40083ab9bf53698 lrc: 2/0,0 mode: COS/COS res: [0x240000402:0x7f7:0x0].0x37639fea bits 0x2/0x0 rrc: 2 type: IBT gid 0 flags: 0x50210400000000 nid: local remote: 0x0 expref: -99 pid: 11281 timeout: 0 lvb_type: 0 00010000:00010000:2.0:1776178209.637890:0:3339:0:(ldlm_lockd.c:1879:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: mdt-lustre-MDT0001_UUID lock: 000000006cc9c357/0xa40083ab9bf53698 lrc: 3/0,0 mode: COS/COS res: [0x240000402:0x7f7:0x0].0x37639fea bits 0x2/0x0 rrc: 2 type: IBT gid 0 flags: 0x40210400000000 nid: local remote: 0x0 expref: -99 pid: 11281 timeout: 0 lvb_type: 0 00010000:00010000:2.0:1776178209.637898:0:3339:0:(ldlm_lockd.c:1893:ldlm_handle_bl_callback()) Lock 000000006cc9c357 already unused, calling callback (0000000079e78348) 00010000:00010000:2.0:1776178209.637902:0:3339:0:(ldlm_request.c:364:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: mdt-lustre-MDT0001_UUID lock: 000000006cc9c357/0xa40083ab9bf53698 lrc: 3/0,0 mode: COS/COS res: [0x240000402:0x7f7:0x0].0x37639fea bits 0x2/0x0 rrc: 2 type: IBT gid 0 flags: 0x40210400000000 nid: local remote: 0x0 expref: -99 pid: 11281 timeout: 0 lvb_type: 0 00010000:00010000:2.0:1776178209.637909:0:3339:0:(ldlm_request.c:1257:ldlm_cli_cancel_local()) ### server-side local cancel ns: mdt-lustre-MDT0001_UUID lock: 000000006cc9c357/0xa40083ab9bf53698 lrc: 4/0,0 mode: COS/COS res: [0x240000402:0x7f7:0x0].0x37639fea bits 0x2/0x0 rrc: 2 type: IBT gid 0 flags: 0x40218400000000 nid: local remote: 0x0 expref: -99 pid: 11281 timeout: 0 lvb_type: 0 00010000:00010000:2.0:1776178209.637921:0:3339:0:(ldlm_inodebits.c:94:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x240000402:0x7f7:0x0].0x37639fea (00000000e4751492) 00010000:00010000:2.0:1776178209.637925:0:3339:0:(ldlm_inodebits.c:96:ldlm_reprocess_inodebits_queue()) Hint 2 00010000:00010000:2.0:1776178209.637929:0:3339:0:(ldlm_lockd.c:1903:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: mdt-lustre-MDT0001_UUID lock: 000000006cc9c357/0xa40083ab9bf53698 lrc: 2/0,0 mode: --/COS res: [0x240000402:0x7f7:0x0].0x37639fea bits 0x2/0x0 rrc: 2 type: IBT gid 0 flags: 0x44a19400000000 nid: local remote: 0x0 expref: -99 pid: 11281 timeout: 0 lvb_type: 0 00010000:00010000:2.0:1776178209.637936:0:3339:0:(ldlm_lock.c:216:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: mdt-lustre-MDT0001_UUID lock: 000000006cc9c357/0xa40083ab9bf53698 lrc: 0/0,0 mode: --/COS res: [0x240000402:0x7f7:0x0].0x37639fea bits 0x2/0x0 rrc: 2 type: IBT gid 0 flags: 0x44a19400000000 nid: local remote: 0x0 expref: -99 pid: 11281 timeout: 0 lvb_type: 0 00010000:00010000:2.0:1776178209.637954:0:3339:0:(ldlm_lock.c:820:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(COS) ns: mdt-lustre-MDT0001_UUID lock: 000000000396420a/0xa40083ab9bf5369f lrc: 3/0,1 mode: COS/COS res: [0x240000402:0x7fd:0x0].0x0 bits 0x3/0x0 rrc: 2 type: IBT gid 0 flags: 0x50210000000000 nid: local remote: 0x0 expref: -99 pid: 11281 timeout: 0 lvb_type: 0 00010000:00010000:2.0:1776178209.637960:0:3339:0:(ldlm_lock.c:879:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: mdt-lustre-MDT0001_UUID lock: 000000000396420a/0xa40083ab9bf5369f lrc: 2/0,0 mode: COS/COS res: [0x240000402:0x7fd:0x0].0x0 bits 0x3/0x0 rrc: 2 type: IBT gid 0 flags: 0x50210400000000 nid: local remote: 0x0 expref: -99 pid: 11281 timeout: 0 lvb_type: 0 00010000:00010000:2.0:1776178209.637966:0:3339:0:(ldlm_lockd.c:1879:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: mdt-lustre-MDT0001_UUID lock: 000000000396420a/0xa40083ab9bf5369f lrc: 3/0,0 mode: COS/COS res: [0x240000402:0x7fd:0x0].0x0 bits 0x3/0x0 rrc: 2 type: IBT gid 0 flags: 0x40210400000000 nid: local remote: 0x0 expref: -99 pid: 11281 timeout: 0 lvb_type: 0 00010000:00010000:2.0:1776178209.637971:0:3339:0:(ldlm_lockd.c:1893:ldlm_handle_bl_callback()) Lock 000000000396420a already unused, calling callback (0000000079e78348) 00010000:00010000:2.0:1776178209.637973:0:3339:0:(ldlm_request.c:364:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: mdt-lustre-MDT0001_UUID lock: 000000000396420a/0xa40083ab9bf5369f lrc: 3/0,0 mode: COS/COS res: [0x240000402:0x7fd:0x0].0x0 bits 0x3/0x0 rrc: 2 type: IBT gid 0 flags: 0x40210400000000 nid: local remote: 0x0 expref: -99 pid: 11281 timeout: 0 lvb_type: 0 00010000:00010000:2.0:1776178209.637978:0:3339:0:(ldlm_request.c:1257:ldlm_cli_cancel_local()) ### server-side local cancel ns: mdt-lustre-MDT0001_UUID lock: 000000000396420a/0xa40083ab9bf5369f lrc: 4/0,0 mode: COS/COS res: [0x240000402:0x7fd:0x0].0x0 bits 0x3/0x0 rrc: 2 type: IBT gid 0 flags: 0x40218400000000 nid: local remote: 0x0 expref: -99 pid: 11281 timeout: 0 lvb_type: 0 00010000:00010000:2.0:1776178209.637985:0:3339:0:(ldlm_inodebits.c:94:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x240000402:0x7fd:0x0].0x0 (00000000aeb25ebf) 00010000:00010000:2.0:1776178209.637987:0:3339:0:(ldlm_inodebits.c:96:ldlm_reprocess_inodebits_queue()) Hint 3 00010000:00010000:2.0:1776178209.637989:0:3339:0:(ldlm_lockd.c:1903:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: mdt-lustre-MDT0001_UUID lock: 000000000396420a/0xa40083ab9bf5369f lrc: 2/0,0 mode: --/COS res: [0x240000402:0x7fd:0x0].0x0 bits 0x3/0x0 rrc: 2 type: IBT gid 0 flags: 0x44a19400000000 nid: local remote: 0x0 expref: -99 pid: 11281 timeout: 0 lvb_type: 0 00010000:00010000:2.0:1776178209.637994:0:3339:0:(ldlm_lock.c:216:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: mdt-lustre-MDT0001_UUID lock: 000000000396420a/0xa40083ab9bf5369f lrc: 0/0,0 mode: --/COS res: [0x240000402:0x7fd:0x0].0x0 bits 0x3/0x0 rrc: 2 type: IBT gid 0 flags: 0x44a19400000000 nid: local remote: 0x0 expref: -99 pid: 11281 timeout: 0 lvb_type: 0 00010000:00010000:2.0:1776178209.638014:0:3339:0:(ldlm_lock.c:820:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(COS) ns: mdt-lustre-MDT0001_UUID lock: 00000000a27c19b3/0xa40083ab9bf536b4 lrc: 3/0,1 mode: COS/COS res: [0x240000402:0x7f7:0x0].0xa6b02dea bits 0x2/0x0 rrc: 2 type: IBT gid 0 flags: 0x50210000000000 nid: local remote: 0x0 expref: -99 pid: 11281 timeout: 0 lvb_type: 0 00010000:00010000:2.0:1776178209.638019:0:3339:0:(ldlm_lock.c:879:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: mdt-lustre-MDT0001_UUID lock: 00000000a27c19b3/0xa40083ab9bf536b4 lrc: 2/0,0 mode: COS/COS res: [0x240000402:0x7f7:0x0].0xa6b02dea bits 0x2/0x0 rrc: 2 type: IBT gid 0 flags: 0x50210400000000 nid: local remote: 0x0 expref: -99 pid: 11281 timeout: 0 lvb_type: 0 00010000:00010000:2.0:1776178209.638024:0:3339:0:(ldlm_lockd.c:1879:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: mdt-lustre-MDT0001_UUID lock: 00000000a27c19b3/0xa40083ab9bf536b4 lrc: 3/0,0 mode: COS/COS res: [0x240000402:0x7f7:0x0].0xa6b02dea bits 0x2/0x0 rrc: 2 type: IBT gid 0 flags: 0x40210400000000 nid: local remote: 0x0 expref: -99 pid: 11281 timeout: 0 lvb_type: 0 00010000:00010000:2.0:1776178209.638029:0:3339:0:(ldlm_lockd.c:1893:ldlm_handle_bl_callback()) Lock 00000000a27c19b3 already unused, calling callback (0000000079e78348) 00010000:00010000:2.0:1776178209.638031:0:3339:0:(ldlm_request.c:364:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: mdt-lustre-MDT0001_UUID lock: 00000000a27c19b3/0xa40083ab9bf536b4 lrc: 3/0,0 mode: COS/COS res: [0x240000402:0x7f7:0x0].0xa6b02dea bits 0x2/0x0 rrc: 2 type: IBT gid 0 flags: 0x40210400000000 nid: local remote: 0x0 expref: -99 pid: 11281 timeout: 0 lvb_type: 0 00010000:00010000:2.0:1776178209.638048:0:3339:0:(ldlm_request.c:1257:ldlm_cli_cancel_local()) ### server-side local cancel ns: mdt-lustre-MDT0001_UUID lock: 00000000a27c19b3/0xa40083ab9bf536b4 lrc: 4/0,0 mode: COS/COS res: [0x240000402:0x7f7:0x0].0xa6b02dea bits 0x2/0x0 rrc: 2 type: IBT gid 0 flags: 0x40218400000000 nid: local remote: 0x0 expref: -99 pid: 11281 timeout: 0 lvb_type: 0 00010000:00010000:2.0:1776178209.638056:0:3339:0:(ldlm_inodebits.c:94:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x240000402:0x7f7:0x0].0xa6b02dea (000000009a65918f) 00010000:00010000:2.0:1776178209.638078:0:3339:0:(ldlm_inodebits.c:96:ldlm_reprocess_inodebits_queue()) Hint 2 00010000:00010000:2.0:1776178209.638081:0:3339:0:(ldlm_lockd.c:1903:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: mdt-lustre-MDT0001_UUID lock: 00000000a27c19b3/0xa40083ab9bf536b4 lrc: 2/0,0 mode: --/COS res: [0x240000402:0x7f7:0x0].0xa6b02dea bits 0x2/0x0 rrc: 2 type: IBT gid 0 flags: 0x44a19400000000 nid: local remote: 0x0 expref: -99 pid: 11281 timeout: 0 lvb_type: 0 00010000:00010000:2.0:1776178209.638087:0:3339:0:(ldlm_lock.c:216:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: mdt-lustre-MDT0001_UUID lock: 00000000a27c19b3/0xa40083ab9bf536b4 lrc: 0/0,0 mode: --/COS res: [0x240000402:0x7f7:0x0].0xa6b02dea bits 0x2/0x0 rrc: 2 type: IBT gid 0 flags: 0x44a19400000000 nid: local remote: 0x0 expref: -99 pid: 11281 timeout: 0 lvb_type: 0 00010000:00010000:2.0:1776178209.638098:0:3339:0:(ldlm_lock.c:820:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(COS) ns: mdt-lustre-MDT0001_UUID lock: 000000009a8f90ae/0xa40083ab9bf536bb lrc: 3/0,1 mode: COS/COS res: [0x240000402:0x7fb:0x0].0x0 bits 0x3/0x0 rrc: 2 type: IBT gid 0 flags: 0x50210000000000 nid: local remote: 0x0 expref: -99 pid: 11281 timeout: 0 lvb_type: 0 00010000:00010000:0.0:1776178209.638112:0:20759:0:(ldlm_lock.c:766:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0001_UUID lock: 0000000030bf6b1e/0xa40083ab9bf537a2 lrc: 3/0,1 mode: --/PW res: [0x2c0000400:0x42e:0x0].0x0 rrc: 3 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 20759 timeout: 0 lvb_type: 0 00010000:00010000:0.0:1776178209.638132:0:20759:0:(ldlm_lock.c:675:ldlm_add_bl_work_item()) ### lock incompatible; sending blocking AST. ns: filter-lustre-OST0001_UUID lock: 000000001d465f75/0xa40083ab9bf534c3 lrc: 2/0,0 mode: PR/PR res: [0x2c0000400:0x42e:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x40000000000000 nid: 192.168.204.3@tcp remote: 0x771f68aa1eb27ae7 expref: 8 pid: 8228 timeout: 0 lvb_type: 1 00010000:00010000:0.0:1776178209.638141:0:20759:0:(ldlm_extent.c:318:ldlm_check_contention()) contended locks = 0 00010000:00010000:0.0:1776178209.638143:0:20759:0:(ldlm_extent.c:318:ldlm_check_contention()) contended locks = 0 00010000:00010000:0.0:1776178209.638145:0:20759:0:(ldlm_resource.c:1650:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0001_UUID lock: 0000000030bf6b1e/0xa40083ab9bf537a2 lrc: 4/0,1 mode: --/PW res: [0x2c0000400:0x42e:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x50010080000000 nid: local remote: 0x0 expref: -99 pid: 20759 timeout: 0 lvb_type: 0 00010000:00010000:2.0:1776178209.638150:0:3339:0:(ldlm_lock.c:879:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: mdt-lustre-MDT0001_UUID lock: 000000009a8f90ae/0xa40083ab9bf536bb lrc: 2/0,0 mode: COS/COS res: [0x240000402:0x7fb:0x0].0x0 bits 0x3/0x0 rrc: 2 type: IBT gid 0 flags: 0x50210400000000 nid: local remote: 0x0 expref: -99 pid: 11281 timeout: 0 lvb_type: 0 00010000:00010000:2.0:1776178209.638160:0:3339:0:(ldlm_lockd.c:1879:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: mdt-lustre-MDT0001_UUID lock: 000000009a8f90ae/0xa40083ab9bf536bb lrc: 3/0,0 mode: COS/COS res: [0x240000402:0x7fb:0x0].0x0 bits 0x3/0x0 rrc: 2 type: IBT gid 0 flags: 0x40210400000000 nid: local remote: 0x0 expref: -99 pid: 11281 timeout: 0 lvb_type: 0 00010000:00010000:2.0:1776178209.638164:0:3339:0:(ldlm_lockd.c:1893:ldlm_handle_bl_callback()) Lock 000000009a8f90ae already unused, calling callback (0000000079e78348) 00010000:00010000:2.0:1776178209.638166:0:3339:0:(ldlm_request.c:364:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: mdt-lustre-MDT0001_UUID lock: 000000009a8f90ae/0xa40083ab9bf536bb lrc: 3/0,0 mode: COS/COS res: [0x240000402:0x7fb:0x0].0x0 bits 0x3/0x0 rrc: 2 type: IBT gid 0 flags: 0x40210400000000 nid: local remote: 0x0 expref: -99 pid: 11281 timeout: 0 lvb_type: 0 00010000:00010000:2.0:1776178209.638171:0:3339:0:(ldlm_request.c:1257:ldlm_cli_cancel_local()) ### server-side local cancel ns: mdt-lustre-MDT0001_UUID lock: 000000009a8f90ae/0xa40083ab9bf536bb lrc: 4/0,0 mode: COS/COS res: [0x240000402:0x7fb:0x0].0x0 bits 0x3/0x0 rrc: 2 type: IBT gid 0 flags: 0x40218400000000 nid: local remote: 0x0 expref: -99 pid: 11281 timeout: 0 lvb_type: 0 00010000:00010000:0.0:1776178209.638174:0:20759:0:(ldlm_lockd.c:953:ldlm_server_blocking_ast()) ### server preparing blocking AST ns: filter-lustre-OST0001_UUID lock: 000000001d465f75/0xa40083ab9bf534c3 lrc: 3/0,0 mode: PR/PR res: [0x2c0000400:0x42e:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x50000000010020 nid: 192.168.204.3@tcp remote: 0x771f68aa1eb27ae7 expref: 8 pid: 8228 timeout: 0 lvb_type: 1 00010000:00010000:2.0:1776178209.638178:0:3339:0:(ldlm_inodebits.c:94:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x240000402:0x7fb:0x0].0x0 (000000000bedd4cc) 00010000:00010000:2.0:1776178209.638181:0:3339:0:(ldlm_inodebits.c:96:ldlm_reprocess_inodebits_queue()) Hint 3 00010000:00010000:2.0:1776178209.638183:0:3339:0:(ldlm_lockd.c:1903:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: mdt-lustre-MDT0001_UUID lock: 000000009a8f90ae/0xa40083ab9bf536bb lrc: 2/0,0 mode: --/COS res: [0x240000402:0x7fb:0x0].0x0 bits 0x3/0x0 rrc: 2 type: IBT gid 0 flags: 0x44a19400000000 nid: local remote: 0x0 expref: -99 pid: 11281 timeout: 0 lvb_type: 0 00010000:00010000:0.0:1776178209.638187:0:20759:0:(ldlm_lockd.c:489:ldlm_add_waiting_lock()) ### adding to wait list(timeout: 100, AT: on) ns: filter-lustre-OST0001_UUID lock: 000000001d465f75/0xa40083ab9bf534c3 lrc: 4/0,0 mode: PR/PR res: [0x2c0000400:0x42e:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x70000400010020 nid: 192.168.204.3@tcp remote: 0x771f68aa1eb27ae7 expref: 8 pid: 8228 timeout: 1016 lvb_type: 1 00010000:00010000:2.0:1776178209.638188:0:3339:0:(ldlm_lock.c:216:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: mdt-lustre-MDT0001_UUID lock: 000000009a8f90ae/0xa40083ab9bf536bb lrc: 0/0,0 mode: --/COS res: [0x240000402:0x7fb:0x0].0x0 bits 0x3/0x0 rrc: 2 type: IBT gid 0 flags: 0x44a19400000000 nid: local remote: 0x0 expref: -99 pid: 11281 timeout: 0 lvb_type: 0 00000100:00100000:0.0:1776178209.638200:0:20759:0:(client.c:1733:ptlrpc_send_new_req()) Sending RPC req@0000000023210d1c pname:cluuid:pid:xid:nid:opc:job ll_ost00_010:lustre-OST0001_UUID:20759:1862456998048128:192.168.204.3@tcp:104: 00010000:00010000:2.0:1776178209.638203:0:3339:0:(ldlm_lock.c:820:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(COS) ns: mdt-lustre-MDT0001_UUID lock: 000000001e6d7318/0xa40083ab9bf536d0 lrc: 3/0,1 mode: COS/COS res: [0x240000402:0x7f7:0x0].0xcda302ea bits 0x2/0x0 rrc: 2 type: IBT gid 0 flags: 0x50210000000000 nid: local remote: 0x0 expref: -99 pid: 11281 timeout: 0 lvb_type: 0 00010000:00010000:2.0:1776178209.638209:0:3339:0:(ldlm_lock.c:879:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: mdt-lustre-MDT0001_UUID lock: 000000001e6d7318/0xa40083ab9bf536d0 lrc: 2/0,0 mode: COS/COS res: [0x240000402:0x7f7:0x0].0xcda302ea bits 0x2/0x0 rrc: 2 type: IBT gid 0 flags: 0x50210400000000 nid: local remote: 0x0 expref: -99 pid: 11281 timeout: 0 lvb_type: 0 00010000:00010000:2.0:1776178209.638214:0:3339:0:(ldlm_lockd.c:1879:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: mdt-lustre-MDT0001_UUID lock: 000000001e6d7318/0xa40083ab9bf536d0 lrc: 3/0,0 mode: COS/COS res: [0x240000402:0x7f7:0x0].0xcda302ea bits 0x2/0x0 rrc: 2 type: IBT gid 0 flags: 0x40210400000000 nid: local remote: 0x0 expref: -99 pid: 11281 timeout: 0 lvb_type: 0 00010000:00010000:2.0:1776178209.638219:0:3339:0:(ldlm_lockd.c:1893:ldlm_handle_bl_callback()) Lock 000000001e6d7318 already unused, calling callback (0000000079e78348) 00010000:00010000:2.0:1776178209.638221:0:3339:0:(ldlm_request.c:364:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: mdt-lustre-MDT0001_UUID lock: 000000001e6d7318/0xa40083ab9bf536d0 lrc: 3/0,0 mode: COS/COS res: [0x240000402:0x7f7:0x0].0xcda302ea bits 0x2/0x0 rrc: 2 type: IBT gid 0 flags: 0x40210400000000 nid: local remote: 0x0 expref: -99 pid: 11281 timeout: 0 lvb_type: 0 00010000:00010000:2.0:1776178209.638227:0:3339:0:(ldlm_request.c:1257:ldlm_cli_cancel_local()) ### server-side local cancel ns: mdt-lustre-MDT0001_UUID lock: 000000001e6d7318/0xa40083ab9bf536d0 lrc: 4/0,0 mode: COS/COS res: [0x240000402:0x7f7:0x0].0xcda302ea bits 0x2/0x0 rrc: 2 type: IBT gid 0 flags: 0x40218400000000 nid: local remote: 0x0 expref: -99 pid: 11281 timeout: 0 lvb_type: 0 00010000:00010000:2.0:1776178209.638235:0:3339:0:(ldlm_inodebits.c:94:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x240000402:0x7f7:0x0].0xcda302ea (0000000059286f6d) 00010000:00010000:2.0:1776178209.638238:0:3339:0:(ldlm_inodebits.c:96:ldlm_reprocess_inodebits_queue()) Hint 2 00010000:00010000:2.0:1776178209.638240:0:3339:0:(ldlm_lockd.c:1903:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: mdt-lustre-MDT0001_UUID lock: 000000001e6d7318/0xa40083ab9bf536d0 lrc: 2/0,0 mode: --/COS res: [0x240000402:0x7f7:0x0].0xcda302ea bits 0x2/0x0 rrc: 2 type: IBT gid 0 flags: 0x44a19400000000 nid: local remote: 0x0 expref: -99 pid: 11281 timeout: 0 lvb_type: 0 00010000:00010000:2.0:1776178209.638244:0:3339:0:(ldlm_lock.c:216:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: mdt-lustre-MDT0001_UUID lock: 000000001e6d7318/0xa40083ab9bf536d0 lrc: 0/0,0 mode: --/COS res: [0x240000402:0x7f7:0x0].0xcda302ea bits 0x2/0x0 rrc: 2 type: IBT gid 0 flags: 0x44a19400000000 nid: local remote: 0x0 expref: -99 pid: 11281 timeout: 0 lvb_type: 0 00010000:00010000:2.0:1776178209.638255:0:3339:0:(ldlm_lock.c:820:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(COS) ns: mdt-lustre-MDT0001_UUID lock: 000000000b5c0143/0xa40083ab9bf536d7 lrc: 3/0,1 mode: COS/COS res: [0x240000402:0x7f8:0x0].0x0 bits 0x3/0x0 rrc: 2 type: IBT gid 0 flags: 0x50210000000000 nid: local remote: 0x0 expref: -99 pid: 11281 timeout: 0 lvb_type: 0 00010000:00010000:2.0:1776178209.638261:0:3339:0:(ldlm_lock.c:879:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: mdt-lustre-MDT0001_UUID lock: 000000000b5c0143/0xa40083ab9bf536d7 lrc: 2/0,0 mode: COS/COS res: [0x240000402:0x7f8:0x0].0x0 bits 0x3/0x0 rrc: 2 type: IBT gid 0 flags: 0x50210400000000 nid: local remote: 0x0 expref: -99 pid: 11281 timeout: 0 lvb_type: 0 00010000:00010000:2.0:1776178209.638268:0:3339:0:(ldlm_lockd.c:1879:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: mdt-lustre-MDT0001_UUID lock: 000000000b5c0143/0xa40083ab9bf536d7 lrc: 3/0,0 mode: COS/COS res: [0x240000402:0x7f8:0x0].0x0 bits 0x3/0x0 rrc: 2 type: IBT gid 0 flags: 0x40210400000000 nid: local remote: 0x0 expref: -99 pid: 11281 timeout: 0 lvb_type: 0 00010000:00010000:2.0:1776178209.638272:0:3339:0:(ldlm_lockd.c:1893:ldlm_handle_bl_callback()) Lock 000000000b5c0143 already unused, calling callback (0000000079e78348) 00010000:00010000:2.0:1776178209.638274:0:3339:0:(ldlm_request.c:364:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: mdt-lustre-MDT0001_UUID lock: 000000000b5c0143/0xa40083ab9bf536d7 lrc: 3/0,0 mode: COS/COS res: [0x240000402:0x7f8:0x0].0x0 bits 0x3/0x0 rrc: 2 type: IBT gid 0 flags: 0x40210400000000 nid: local remote: 0x0 expref: -99 pid: 11281 timeout: 0 lvb_type: 0 00010000:00010000:2.0:1776178209.638280:0:3339:0:(ldlm_request.c:1257:ldlm_cli_cancel_local()) ### server-side local cancel ns: mdt-lustre-MDT0001_UUID lock: 000000000b5c0143/0xa40083ab9bf536d7 lrc: 4/0,0 mode: COS/COS res: [0x240000402:0x7f8:0x0].0x0 bits 0x3/0x0 rrc: 2 type: IBT gid 0 flags: 0x40218400000000 nid: local remote: 0x0 expref: -99 pid: 11281 timeout: 0 lvb_type: 0 00010000:00010000:2.0:1776178209.638286:0:3339:0:(ldlm_inodebits.c:94:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x240000402:0x7f8:0x0].0x0 (00000000d7645e58) 00010000:00010000:2.0:1776178209.638289:0:3339:0:(ldlm_inodebits.c:96:ldlm_reprocess_inodebits_queue()) Hint 3 00010000:00010000:2.0:1776178209.638292:0:3339:0:(ldlm_lockd.c:1903:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: mdt-lustre-MDT0001_UUID lock: 000000000b5c0143/0xa40083ab9bf536d7 lrc: 2/0,0 mode: --/COS res: [0x240000402:0x7f8:0x0].0x0 bits 0x3/0x0 rrc: 2 type: IBT gid 0 flags: 0x44a19400000000 nid: local remote: 0x0 expref: -99 pid: 11281 timeout: 0 lvb_type: 0 00010000:00010000:2.0:1776178209.638297:0:3339:0:(ldlm_lock.c:216:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: mdt-lustre-MDT0001_UUID lock: 000000000b5c0143/0xa40083ab9bf536d7 lrc: 0/0,0 mode: --/COS res: [0x240000402:0x7f8:0x0].0x0 bits 0x3/0x0 rrc: 2 type: IBT gid 0 flags: 0x44a19400000000 nid: local remote: 0x0 expref: -99 pid: 11281 timeout: 0 lvb_type: 0 00010000:00010000:2.0:1776178209.638311:0:3339:0:(ldlm_lock.c:820:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(COS) ns: mdt-lustre-MDT0001_UUID lock: 0000000096c1e297/0xa40083ab9bf536ec lrc: 3/0,1 mode: COS/COS res: [0x240000402:0x7f7:0x0].0x5e5674ea bits 0x2/0x0 rrc: 2 type: IBT gid 0 flags: 0x50210000000000 nid: local remote: 0x0 expref: -99 pid: 11281 timeout: 0 lvb_type: 0 00010000:00010000:2.0:1776178209.638316:0:3339:0:(ldlm_lock.c:879:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: mdt-lustre-MDT0001_UUID lock: 0000000096c1e297/0xa40083ab9bf536ec lrc: 2/0,0 mode: COS/COS res: [0x240000402:0x7f7:0x0].0x5e5674ea bits 0x2/0x0 rrc: 2 type: IBT gid 0 flags: 0x50210400000000 nid: local remote: 0x0 expref: -99 pid: 11281 timeout: 0 lvb_type: 0 00010000:00010000:2.0:1776178209.638322:0:3339:0:(ldlm_lockd.c:1879:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: mdt-lustre-MDT0001_UUID lock: 0000000096c1e297/0xa40083ab9bf536ec lrc: 3/0,0 mode: COS/COS res: [0x240000402:0x7f7:0x0].0x5e5674ea bits 0x2/0x0 rrc: 2 type: IBT gid 0 flags: 0x40210400000000 nid: local remote: 0x0 expref: -99 pid: 11281 timeout: 0 lvb_type: 0 00010000:00010000:2.0:1776178209.638327:0:3339:0:(ldlm_lockd.c:1893:ldlm_handle_bl_callback()) Lock 0000000096c1e297 already unused, calling callback (0000000079e78348) 00010000:00010000:2.0:1776178209.638329:0:3339:0:(ldlm_request.c:364:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: mdt-lustre-MDT0001_UUID lock: 0000000096c1e297/0xa40083ab9bf536ec lrc: 3/0,0 mode: COS/COS res: [0x240000402:0x7f7:0x0].0x5e5674ea bits 0x2/0x0 rrc: 2 type: IBT gid 0 flags: 0x40210400000000 nid: local remote: 0x0 expref: -99 pid: 11281 timeout: 0 lvb_type: 0 00010000:00010000:2.0:1776178209.638334:0:3339:0:(ldlm_request.c:1257:ldlm_cli_cancel_local()) ### server-side local cancel ns: mdt-lustre-MDT0001_UUID lock: 0000000096c1e297/0xa40083ab9bf536ec lrc: 4/0,0 mode: COS/COS res: [0x240000402:0x7f7:0x0].0x5e5674ea bits 0x2/0x0 rrc: 2 type: IBT gid 0 flags: 0x40218400000000 nid: local remote: 0x0 expref: -99 pid: 11281 timeout: 0 lvb_type: 0 00010000:00010000:2.0:1776178209.638342:0:3339:0:(ldlm_inodebits.c:94:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x240000402:0x7f7:0x0].0x5e5674ea (00000000314a8d10) 00010000:00010000:2.0:1776178209.638344:0:3339:0:(ldlm_inodebits.c:96:ldlm_reprocess_inodebits_queue()) Hint 2 00010000:00010000:2.0:1776178209.638346:0:3339:0:(ldlm_lockd.c:1903:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: mdt-lustre-MDT0001_UUID lock: 0000000096c1e297/0xa40083ab9bf536ec lrc: 2/0,0 mode: --/COS res: [0x240000402:0x7f7:0x0].0x5e5674ea bits 0x2/0x0 rrc: 2 type: IBT gid 0 flags: 0x44a19400000000 nid: local remote: 0x0 expref: -99 pid: 11281 timeout: 0 lvb_type: 0 00010000:00010000:2.0:1776178209.638350:0:3339:0:(ldlm_lock.c:216:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: mdt-lustre-MDT0001_UUID lock: 0000000096c1e297/0xa40083ab9bf536ec lrc: 0/0,0 mode: --/COS res: [0x240000402:0x7f7:0x0].0x5e5674ea bits 0x2/0x0 rrc: 2 type: IBT gid 0 flags: 0x44a19400000000 nid: local remote: 0x0 expref: -99 pid: 11281 timeout: 0 lvb_type: 0 00010000:00010000:2.0:1776178209.638360:0:3339:0:(ldlm_lock.c:820:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(COS) ns: mdt-lustre-MDT0001_UUID lock: 00000000c9d7d9fc/0xa40083ab9bf536f3 lrc: 3/0,1 mode: COS/COS res: [0x240000402:0x7fa:0x0].0x0 bits 0x3/0x0 rrc: 2 type: IBT gid 0 flags: 0x50210000000000 nid: local remote: 0x0 expref: -99 pid: 11281 timeout: 0 lvb_type: 0 00010000:00010000:2.0:1776178209.638365:0:3339:0:(ldlm_lock.c:879:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: mdt-lustre-MDT0001_UUID lock: 00000000c9d7d9fc/0xa40083ab9bf536f3 lrc: 2/0,0 mode: COS/COS res: [0x240000402:0x7fa:0x0].0x0 bits 0x3/0x0 rrc: 2 type: IBT gid 0 flags: 0x50210400000000 nid: local remote: 0x0 expref: -99 pid: 11281 timeout: 0 lvb_type: 0 00010000:00010000:2.0:1776178209.638371:0:3339:0:(ldlm_lockd.c:1879:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: mdt-lustre-MDT0001_UUID lock: 00000000c9d7d9fc/0xa40083ab9bf536f3 lrc: 3/0,0 mode: COS/COS res: [0x240000402:0x7fa:0x0].0x0 bits 0x3/0x0 rrc: 2 type: IBT gid 0 flags: 0x40210400000000 nid: local remote: 0x0 expref: -99 pid: 11281 timeout: 0 lvb_type: 0 00010000:00010000:2.0:1776178209.638375:0:3339:0:(ldlm_lockd.c:1893:ldlm_handle_bl_callback()) Lock 00000000c9d7d9fc already unused, calling callback (0000000079e78348) 00010000:00010000:2.0:1776178209.638377:0:3339:0:(ldlm_request.c:364:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: mdt-lustre-MDT0001_UUID lock: 00000000c9d7d9fc/0xa40083ab9bf536f3 lrc: 3/0,0 mode: COS/COS res: [0x240000402:0x7fa:0x0].0x0 bits 0x3/0x0 rrc: 2 type: IBT gid 0 flags: 0x40210400000000 nid: local remote: 0x0 expref: -99 pid: 11281 timeout: 0 lvb_type: 0 00010000:00010000:2.0:1776178209.638382:0:3339:0:(ldlm_request.c:1257:ldlm_cli_cancel_local()) ### server-side local cancel ns: mdt-lustre-MDT0001_UUID lock: 00000000c9d7d9fc/0xa40083ab9bf536f3 lrc: 4/0,0 mode: COS/COS res: [0x240000402:0x7fa:0x0].0x0 bits 0x3/0x0 rrc: 2 type: IBT gid 0 flags: 0x40218400000000 nid: local remote: 0x0 expref: -99 pid: 11281 timeout: 0 lvb_type: 0 00010000:00010000:2.0:1776178209.638388:0:3339:0:(ldlm_inodebits.c:94:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x240000402:0x7fa:0x0].0x0 (0000000064abeb1c) 00010000:00010000:2.0:1776178209.638389:0:3339:0:(ldlm_inodebits.c:96:ldlm_reprocess_inodebits_queue()) Hint 3 00010000:00010000:2.0:1776178209.638391:0:3339:0:(ldlm_lockd.c:1903:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: mdt-lustre-MDT0001_UUID lock: 00000000c9d7d9fc/0xa40083ab9bf536f3 lrc: 2/0,0 mode: --/COS res: [0x240000402:0x7fa:0x0].0x0 bits 0x3/0x0 rrc: 2 type: IBT gid 0 flags: 0x44a19400000000 nid: local remote: 0x0 expref: -99 pid: 11281 timeout: 0 lvb_type: 0 00010000:00010000:2.0:1776178209.638716:0:3339:0:(ldlm_lock.c:216:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: mdt-lustre-MDT0001_UUID lock: 00000000c9d7d9fc/0xa40083ab9bf536f3 lrc: 0/0,0 mode: --/COS res: [0x240000402:0x7fa:0x0].0x0 bits 0x3/0x0 rrc: 2 type: IBT gid 0 flags: 0x44a19400000000 nid: local remote: 0x0 expref: -99 pid: 11281 timeout: 0 lvb_type: 0 00010000:00010000:2.0:1776178209.638741:0:3339:0:(ldlm_lock.c:820:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(COS) ns: mdt-lustre-MDT0001_UUID lock: 00000000c1e7ea3e/0xa40083ab9bf53708 lrc: 3/0,1 mode: COS/COS res: [0x240000402:0x7f7:0x0].0x7fbd58ea bits 0x2/0x0 rrc: 2 type: IBT gid 0 flags: 0x50210000000000 nid: local remote: 0x0 expref: -99 pid: 11281 timeout: 0 lvb_type: 0 00010000:00010000:2.0:1776178209.638746:0:3339:0:(ldlm_lock.c:879:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: mdt-lustre-MDT0001_UUID lock: 00000000c1e7ea3e/0xa40083ab9bf53708 lrc: 2/0,0 mode: COS/COS res: [0x240000402:0x7f7:0x0].0x7fbd58ea bits 0x2/0x0 rrc: 2 type: IBT gid 0 flags: 0x50210400000000 nid: local remote: 0x0 expref: -99 pid: 11281 timeout: 0 lvb_type: 0 00010000:00010000:2.0:1776178209.638779:0:3339:0:(ldlm_lockd.c:1879:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: mdt-lustre-MDT0001_UUID lock: 00000000c1e7ea3e/0xa40083ab9bf53708 lrc: 3/0,0 mode: COS/COS res: [0x240000402:0x7f7:0x0].0x7fbd58ea bits 0x2/0x0 rrc: 2 type: IBT gid 0 flags: 0x40210400000000 nid: local remote: 0x0 expref: -99 pid: 11281 timeout: 0 lvb_type: 0 00010000:00010000:2.0:1776178209.638793:0:3339:0:(ldlm_lockd.c:1893:ldlm_handle_bl_callback()) Lock 00000000c1e7ea3e already unused, calling callback (0000000079e78348) 00010000:00010000:2.0:1776178209.638818:0:3339:0:(ldlm_request.c:364:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: mdt-lustre-MDT0001_UUID lock: 00000000c1e7ea3e/0xa40083ab9bf53708 lrc: 3/0,0 mode: COS/COS res: [0x240000402:0x7f7:0x0].0x7fbd58ea bits 0x2/0x0 rrc: 2 type: IBT gid 0 flags: 0x40210400000000 nid: local remote: 0x0 expref: -99 pid: 11281 timeout: 0 lvb_type: 0 00010000:00010000:2.0:1776178209.638847:0:3339:0:(ldlm_request.c:1257:ldlm_cli_cancel_local()) ### server-side local cancel ns: mdt-lustre-MDT0001_UUID lock: 00000000c1e7ea3e/0xa40083ab9bf53708 lrc: 4/0,0 mode: COS/COS res: [0x240000402:0x7f7:0x0].0x7fbd58ea bits 0x2/0x0 rrc: 2 type: IBT gid 0 flags: 0x40218400000000 nid: local remote: 0x0 expref: -99 pid: 11281 timeout: 0 lvb_type: 0 00010000:00010000:2.0:1776178209.638861:0:3339:0:(ldlm_inodebits.c:94:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x240000402:0x7f7:0x0].0x7fbd58ea (0000000022e38283) 00010000:00010000:2.0:1776178209.638863:0:3339:0:(ldlm_inodebits.c:96:ldlm_reprocess_inodebits_queue()) Hint 2 00010000:00010000:2.0:1776178209.638866:0:3339:0:(ldlm_lockd.c:1903:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: mdt-lustre-MDT0001_UUID lock: 00000000c1e7ea3e/0xa40083ab9bf53708 lrc: 2/0,0 mode: --/COS res: [0x240000402:0x7f7:0x0].0x7fbd58ea bits 0x2/0x0 rrc: 2 type: IBT gid 0 flags: 0x44a19400000000 nid: local remote: 0x0 expref: -99 pid: 11281 timeout: 0 lvb_type: 0 00000100:00100000:1.0:1776178209.638909:0:3354:0:(events.c:361:request_in_callback()) peer: 12345-0@lo (source: 12345-0@lo) 00000100:00100000:1.0:1776178209.638945:0:3354:0:(client.c:1733:ptlrpc_send_new_req()) Sending RPC req@00000000f8e38e41 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_01:lustre-MDT0001-mdtlov_UUID:3354:1862456998047808:0@lo:103:jbd2/dm-1-8.0 00000100:00100000:1.0:1776178209.638971:0:3354:0:(events.c:361:request_in_callback()) peer: 12345-0@lo (source: 12345-0@lo) 00000100:00100000:1.0:1776178209.638994:0:3354:0:(ptlrpcd.c:415:ptlrpcd_check()) transfer 2 async RPCs [0->1] 00000100:00100000:1.0:1776178209.639000:0:3354:0:(client.c:1733:ptlrpc_send_new_req()) Sending RPC req@00000000dd1281fe pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_01:lustre-MDT0001-mdtlov_UUID:3354:1862456998047872:0@lo:6:osp-syn-1-1.0 00010000:00010000:2.0:1776178209.639147:0:3339:0:(ldlm_lock.c:216:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: mdt-lustre-MDT0001_UUID lock: 00000000c1e7ea3e/0xa40083ab9bf53708 lrc: 0/0,0 mode: --/COS res: [0x240000402:0x7f7:0x0].0x7fbd58ea bits 0x2/0x0 rrc: 2 type: IBT gid 0 flags: 0x44a19400000000 nid: local remote: 0x0 expref: -99 pid: 11281 timeout: 0 lvb_type: 0 00010000:00010000:2.0:1776178209.639160:0:3339:0:(ldlm_lock.c:820:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(COS) ns: mdt-lustre-MDT0001_UUID lock: 00000000dcea299c/0xa40083ab9bf5370f lrc: 3/0,1 mode: COS/COS res: [0x240000402:0x7fe:0x0].0x0 bits 0x3/0x0 rrc: 2 type: IBT gid 0 flags: 0x50210000000000 nid: local remote: 0x0 expref: -99 pid: 11281 timeout: 0 lvb_type: 0 00010000:00010000:2.0:1776178209.639166:0:3339:0:(ldlm_lock.c:879:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: mdt-lustre-MDT0001_UUID lock: 00000000dcea299c/0xa40083ab9bf5370f lrc: 2/0,0 mode: COS/COS res: [0x240000402:0x7fe:0x0].0x0 bits 0x3/0x0 rrc: 2 type: IBT gid 0 flags: 0x50210400000000 nid: local remote: 0x0 expref: -99 pid: 11281 timeout: 0 lvb_type: 0 00010000:00010000:2.0:1776178209.639171:0:3339:0:(ldlm_lockd.c:1879:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: mdt-lustre-MDT0001_UUID lock: 00000000dcea299c/0xa40083ab9bf5370f lrc: 3/0,0 mode: COS/COS res: [0x240000402:0x7fe:0x0].0x0 bits 0x3/0x0 rrc: 2 type: IBT gid 0 flags: 0x40210400000000 nid: local remote: 0x0 expref: -99 pid: 11281 timeout: 0 lvb_type: 0 00010000:00010000:2.0:1776178209.639178:0:3339:0:(ldlm_lockd.c:1893:ldlm_handle_bl_callback()) Lock 00000000dcea299c already unused, calling callback (0000000079e78348) 00010000:00010000:2.0:1776178209.639180:0:3339:0:(ldlm_request.c:364:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: mdt-lustre-MDT0001_UUID lock: 00000000dcea299c/0xa40083ab9bf5370f lrc: 3/0,0 mode: COS/COS res: [0x240000402:0x7fe:0x0].0x0 bits 0x3/0x0 rrc: 2 type: IBT gid 0 flags: 0x40210400000000 nid: local remote: 0x0 expref: -99 pid: 11281 timeout: 0 lvb_type: 0 00010000:00010000:2.0:1776178209.639186:0:3339:0:(ldlm_request.c:1257:ldlm_cli_cancel_local()) ### server-side local cancel ns: mdt-lustre-MDT0001_UUID lock: 00000000dcea299c/0xa40083ab9bf5370f lrc: 4/0,0 mode: COS/COS res: [0x240000402:0x7fe:0x0].0x0 bits 0x3/0x0 rrc: 2 type: IBT gid 0 flags: 0x40218400000000 nid: local remote: 0x0 expref: -99 pid: 11281 timeout: 0 lvb_type: 0 00010000:00010000:2.0:1776178209.639195:0:3339:0:(ldlm_inodebits.c:94:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x240000402:0x7fe:0x0].0x0 (000000009f2a4d4c) 00010000:00010000:2.0:1776178209.639197:0:3339:0:(ldlm_inodebits.c:96:ldlm_reprocess_inodebits_queue()) Hint 3 00010000:00010000:2.0:1776178209.639199:0:3339:0:(ldlm_lockd.c:1903:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: mdt-lustre-MDT0001_UUID lock: 00000000dcea299c/0xa40083ab9bf5370f lrc: 2/0,0 mode: --/COS res: [0x240000402:0x7fe:0x0].0x0 bits 0x3/0x0 rrc: 2 type: IBT gid 0 flags: 0x44a19400000000 nid: local remote: 0x0 expref: -99 pid: 11281 timeout: 0 lvb_type: 0 00010000:00010000:2.0:1776178209.639205:0:3339:0:(ldlm_lock.c:216:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: mdt-lustre-MDT0001_UUID lock: 00000000dcea299c/0xa40083ab9bf5370f lrc: 0/0,0 mode: --/COS res: [0x240000402:0x7fe:0x0].0x0 bits 0x3/0x0 rrc: 2 type: IBT gid 0 flags: 0x44a19400000000 nid: local remote: 0x0 expref: -99 pid: 11281 timeout: 0 lvb_type: 0 00010000:00010000:2.0:1776178209.639221:0:3339:0:(ldlm_lock.c:820:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(COS) ns: mdt-lustre-MDT0001_UUID lock: 0000000011d684d8/0xa40083ab9bf53724 lrc: 3/0,1 mode: COS/COS res: [0x240000402:0x7f7:0x0].0xef09e6ea bits 0x2/0x0 rrc: 2 type: IBT gid 0 flags: 0x50210000000000 nid: local remote: 0x0 expref: -99 pid: 11281 timeout: 0 lvb_type: 0 00010000:00010000:2.0:1776178209.639227:0:3339:0:(ldlm_lock.c:879:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: mdt-lustre-MDT0001_UUID lock: 0000000011d684d8/0xa40083ab9bf53724 lrc: 2/0,0 mode: COS/COS res: [0x240000402:0x7f7:0x0].0xef09e6ea bits 0x2/0x0 rrc: 2 type: IBT gid 0 flags: 0x50210400000000 nid: local remote: 0x0 expref: -99 pid: 11281 timeout: 0 lvb_type: 0 00010000:00010000:2.0:1776178209.639232:0:3339:0:(ldlm_lockd.c:1879:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: mdt-lustre-MDT0001_UUID lock: 0000000011d684d8/0xa40083ab9bf53724 lrc: 3/0,0 mode: COS/COS res: [0x240000402:0x7f7:0x0].0xef09e6ea bits 0x2/0x0 rrc: 2 type: IBT gid 0 flags: 0x40210400000000 nid: local remote: 0x0 expref: -99 pid: 11281 timeout: 0 lvb_type: 0 00010000:00010000:2.0:1776178209.639237:0:3339:0:(ldlm_lockd.c:1893:ldlm_handle_bl_callback()) Lock 0000000011d684d8 already unused, calling callback (0000000079e78348) 00010000:00010000:2.0:1776178209.639238:0:3339:0:(ldlm_request.c:364:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: mdt-lustre-MDT0001_UUID lock: 0000000011d684d8/0xa40083ab9bf53724 lrc: 3/0,0 mode: COS/COS res: [0x240000402:0x7f7:0x0].0xef09e6ea bits 0x2/0x0 rrc: 2 type: IBT gid 0 flags: 0x40210400000000 nid: local remote: 0x0 expref: -99 pid: 11281 timeout: 0 lvb_type: 0 00010000:00010000:2.0:1776178209.639244:0:3339:0:(ldlm_request.c:1257:ldlm_cli_cancel_local()) ### server-side local cancel ns: mdt-lustre-MDT0001_UUID lock: 0000000011d684d8/0xa40083ab9bf53724 lrc: 4/0,0 mode: COS/COS res: [0x240000402:0x7f7:0x0].0xef09e6ea bits 0x2/0x0 rrc: 2 type: IBT gid 0 flags: 0x40218400000000 nid: local remote: 0x0 expref: -99 pid: 11281 timeout: 0 lvb_type: 0 00010000:00010000:2.0:1776178209.639251:0:3339:0:(ldlm_inodebits.c:94:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x240000402:0x7f7:0x0].0xef09e6ea (0000000026efbdea) 00010000:00010000:2.0:1776178209.639253:0:3339:0:(ldlm_inodebits.c:96:ldlm_reprocess_inodebits_queue()) Hint 2 00010000:00010000:2.0:1776178209.639255:0:3339:0:(ldlm_lockd.c:1903:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: mdt-lustre-MDT0001_UUID lock: 0000000011d684d8/0xa40083ab9bf53724 lrc: 2/0,0 mode: --/COS res: [0x240000402:0x7f7:0x0].0xef09e6ea bits 0x2/0x0 rrc: 2 type: IBT gid 0 flags: 0x44a19400000000 nid: local remote: 0x0 expref: -99 pid: 11281 timeout: 0 lvb_type: 0 00010000:00010000:2.0:1776178209.639266:0:3339:0:(ldlm_lock.c:216:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: mdt-lustre-MDT0001_UUID lock: 0000000011d684d8/0xa40083ab9bf53724 lrc: 0/0,0 mode: --/COS res: [0x240000402:0x7f7:0x0].0xef09e6ea bits 0x2/0x0 rrc: 2 type: IBT gid 0 flags: 0x44a19400000000 nid: local remote: 0x0 expref: -99 pid: 11281 timeout: 0 lvb_type: 0 00010000:00010000:2.0:1776178209.639279:0:3339:0:(ldlm_lock.c:820:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(COS) ns: mdt-lustre-MDT0001_UUID lock: 000000008813523d/0xa40083ab9bf5372b lrc: 3/0,1 mode: COS/COS res: [0x240000402:0x7fc:0x0].0x0 bits 0x3/0x0 rrc: 2 type: IBT gid 0 flags: 0x50210000000000 nid: local remote: 0x0 expref: -99 pid: 11281 timeout: 0 lvb_type: 0 00010000:00010000:2.0:1776178209.639286:0:3339:0:(ldlm_lock.c:879:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: mdt-lustre-MDT0001_UUID lock: 000000008813523d/0xa40083ab9bf5372b lrc: 2/0,0 mode: COS/COS res: [0x240000402:0x7fc:0x0].0x0 bits 0x3/0x0 rrc: 2 type: IBT gid 0 flags: 0x50210400000000 nid: local remote: 0x0 expref: -99 pid: 11281 timeout: 0 lvb_type: 0 00010000:00010000:2.0:1776178209.639290:0:3339:0:(ldlm_lockd.c:1879:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: mdt-lustre-MDT0001_UUID lock: 000000008813523d/0xa40083ab9bf5372b lrc: 3/0,0 mode: COS/COS res: [0x240000402:0x7fc:0x0].0x0 bits 0x3/0x0 rrc: 2 type: IBT gid 0 flags: 0x40210400000000 nid: local remote: 0x0 expref: -99 pid: 11281 timeout: 0 lvb_type: 0 00010000:00010000:2.0:1776178209.639295:0:3339:0:(ldlm_lockd.c:1893:ldlm_handle_bl_callback()) Lock 000000008813523d already unused, calling callback (0000000079e78348) 00010000:00010000:2.0:1776178209.639298:0:3339:0:(ldlm_request.c:364:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: mdt-lustre-MDT0001_UUID lock: 000000008813523d/0xa40083ab9bf5372b lrc: 3/0,0 mode: COS/COS res: [0x240000402:0x7fc:0x0].0x0 bits 0x3/0x0 rrc: 2 type: IBT gid 0 flags: 0x40210400000000 nid: local remote: 0x0 expref: -99 pid: 11281 timeout: 0 lvb_type: 0 00010000:00010000:2.0:1776178209.639304:0:3339:0:(ldlm_request.c:1257:ldlm_cli_cancel_local()) ### server-side local cancel ns: mdt-lustre-MDT0001_UUID lock: 000000008813523d/0xa40083ab9bf5372b lrc: 4/0,0 mode: COS/COS res: [0x240000402:0x7fc:0x0].0x0 bits 0x3/0x0 rrc: 2 type: IBT gid 0 flags: 0x40218400000000 nid: local remote: 0x0 expref: -99 pid: 11281 timeout: 0 lvb_type: 0 00010000:00010000:2.0:1776178209.639312:0:3339:0:(ldlm_inodebits.c:94:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x240000402:0x7fc:0x0].0x0 (00000000abeb6ce0) 00010000:00010000:2.0:1776178209.639313:0:3339:0:(ldlm_inodebits.c:96:ldlm_reprocess_inodebits_queue()) Hint 3 00010000:00010000:2.0:1776178209.639315:0:3339:0:(ldlm_lockd.c:1903:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: mdt-lustre-MDT0001_UUID lock: 000000008813523d/0xa40083ab9bf5372b lrc: 2/0,0 mode: --/COS res: [0x240000402:0x7fc:0x0].0x0 bits 0x3/0x0 rrc: 2 type: IBT gid 0 flags: 0x44a19400000000 nid: local remote: 0x0 expref: -99 pid: 11281 timeout: 0 lvb_type: 0 00010000:00010000:2.0:1776178209.639321:0:3339:0:(ldlm_lock.c:216:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: mdt-lustre-MDT0001_UUID lock: 000000008813523d/0xa40083ab9bf5372b lrc: 0/0,0 mode: --/COS res: [0x240000402:0x7fc:0x0].0x0 bits 0x3/0x0 rrc: 2 type: IBT gid 0 flags: 0x44a19400000000 nid: local remote: 0x0 expref: -99 pid: 11281 timeout: 0 lvb_type: 0 00010000:00010000:2.0:1776178209.639337:0:3339:0:(ldlm_lock.c:820:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(COS) ns: mdt-lustre-MDT0001_UUID lock: 00000000b335466f/0xa40083ab9bf53740 lrc: 3/0,1 mode: COS/COS res: [0x240000402:0x7f7:0x0].0x15fcbbea bits 0x2/0x0 rrc: 2 type: IBT gid 0 flags: 0x50210000000000 nid: local remote: 0x0 expref: -99 pid: 11281 timeout: 0 lvb_type: 0 00010000:00010000:2.0:1776178209.639346:0:3339:0:(ldlm_lock.c:879:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: mdt-lustre-MDT0001_UUID lock: 00000000b335466f/0xa40083ab9bf53740 lrc: 2/0,0 mode: COS/COS res: [0x240000402:0x7f7:0x0].0x15fcbbea bits 0x2/0x0 rrc: 2 type: IBT gid 0 flags: 0x50210400000000 nid: local remote: 0x0 expref: -99 pid: 11281 timeout: 0 lvb_type: 0 00010000:00010000:2.0:1776178209.639351:0:3339:0:(ldlm_lockd.c:1879:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: mdt-lustre-MDT0001_UUID lock: 00000000b335466f/0xa40083ab9bf53740 lrc: 3/0,0 mode: COS/COS res: [0x240000402:0x7f7:0x0].0x15fcbbea bits 0x2/0x0 rrc: 2 type: IBT gid 0 flags: 0x40210400000000 nid: local remote: 0x0 expref: -99 pid: 11281 timeout: 0 lvb_type: 0 00010000:00010000:2.0:1776178209.639356:0:3339:0:(ldlm_lockd.c:1893:ldlm_handle_bl_callback()) Lock 00000000b335466f already unused, calling callback (0000000079e78348) 00010000:00010000:2.0:1776178209.639358:0:3339:0:(ldlm_request.c:364:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: mdt-lustre-MDT0001_UUID lock: 00000000b335466f/0xa40083ab9bf53740 lrc: 3/0,0 mode: COS/COS res: [0x240000402:0x7f7:0x0].0x15fcbbea bits 0x2/0x0 rrc: 2 type: IBT gid 0 flags: 0x40210400000000 nid: local remote: 0x0 expref: -99 pid: 11281 timeout: 0 lvb_type: 0 00010000:00010000:2.0:1776178209.639364:0:3339:0:(ldlm_request.c:1257:ldlm_cli_cancel_local()) ### server-side local cancel ns: mdt-lustre-MDT0001_UUID lock: 00000000b335466f/0xa40083ab9bf53740 lrc: 4/0,0 mode: COS/COS res: [0x240000402:0x7f7:0x0].0x15fcbbea bits 0x2/0x0 rrc: 2 type: IBT gid 0 flags: 0x40218400000000 nid: local remote: 0x0 expref: -99 pid: 11281 timeout: 0 lvb_type: 0 00010000:00010000:2.0:1776178209.639370:0:3339:0:(ldlm_inodebits.c:94:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x240000402:0x7f7:0x0].0x15fcbbea (00000000f97f1796) 00010000:00010000:2.0:1776178209.639372:0:3339:0:(ldlm_inodebits.c:96:ldlm_reprocess_inodebits_queue()) Hint 2 00010000:00010000:2.0:1776178209.639374:0:3339:0:(ldlm_lockd.c:1903:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: mdt-lustre-MDT0001_UUID lock: 00000000b335466f/0xa40083ab9bf53740 lrc: 2/0,0 mode: --/COS res: [0x240000402:0x7f7:0x0].0x15fcbbea bits 0x2/0x0 rrc: 2 type: IBT gid 0 flags: 0x44a19400000000 nid: local remote: 0x0 expref: -99 pid: 11281 timeout: 0 lvb_type: 0 00010000:00010000:2.0:1776178209.639379:0:3339:0:(ldlm_lock.c:216:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: mdt-lustre-MDT0001_UUID lock: 00000000b335466f/0xa40083ab9bf53740 lrc: 0/0,0 mode: --/COS res: [0x240000402:0x7f7:0x0].0x15fcbbea bits 0x2/0x0 rrc: 2 type: IBT gid 0 flags: 0x44a19400000000 nid: local remote: 0x0 expref: -99 pid: 11281 timeout: 0 lvb_type: 0 00010000:00010000:2.0:1776178209.639390:0:3339:0:(ldlm_lock.c:820:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(COS) ns: mdt-lustre-MDT0001_UUID lock: 00000000575572c2/0xa40083ab9bf53747 lrc: 3/0,1 mode: COS/COS res: [0x240000402:0x7f9:0x0].0x0 bits 0x3/0x0 rrc: 2 type: IBT gid 0 flags: 0x50210000000000 nid: local remote: 0x0 expref: -99 pid: 11281 timeout: 0 lvb_type: 0 00010000:00010000:2.0:1776178209.639396:0:3339:0:(ldlm_lock.c:879:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: mdt-lustre-MDT0001_UUID lock: 00000000575572c2/0xa40083ab9bf53747 lrc: 2/0,0 mode: COS/COS res: [0x240000402:0x7f9:0x0].0x0 bits 0x3/0x0 rrc: 2 type: IBT gid 0 flags: 0x50210400000000 nid: local remote: 0x0 expref: -99 pid: 11281 timeout: 0 lvb_type: 0 00010000:00010000:2.0:1776178209.639401:0:3339:0:(ldlm_lockd.c:1879:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: mdt-lustre-MDT0001_UUID lock: 00000000575572c2/0xa40083ab9bf53747 lrc: 3/0,0 mode: COS/COS res: [0x240000402:0x7f9:0x0].0x0 bits 0x3/0x0 rrc: 2 type: IBT gid 0 flags: 0x40210400000000 nid: local remote: 0x0 expref: -99 pid: 11281 timeout: 0 lvb_type: 0 00010000:00010000:2.0:1776178209.639409:0:3339:0:(ldlm_lockd.c:1893:ldlm_handle_bl_callback()) Lock 00000000575572c2 already unused, calling callback (0000000079e78348) 00010000:00010000:2.0:1776178209.639411:0:3339:0:(ldlm_request.c:364:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: mdt-lustre-MDT0001_UUID lock: 00000000575572c2/0xa40083ab9bf53747 lrc: 3/0,0 mode: COS/COS res: [0x240000402:0x7f9:0x0].0x0 bits 0x3/0x0 rrc: 2 type: IBT gid 0 flags: 0x40210400000000 nid: local remote: 0x0 expref: -99 pid: 11281 timeout: 0 lvb_type: 0 00010000:00010000:2.0:1776178209.639416:0:3339:0:(ldlm_request.c:1257:ldlm_cli_cancel_local()) ### server-side local cancel ns: mdt-lustre-MDT0001_UUID lock: 00000000575572c2/0xa40083ab9bf53747 lrc: 4/0,0 mode: COS/COS res: [0x240000402:0x7f9:0x0].0x0 bits 0x3/0x0 rrc: 2 type: IBT gid 0 flags: 0x40218400000000 nid: local remote: 0x0 expref: -99 pid: 11281 timeout: 0 lvb_type: 0 00010000:00010000:2.0:1776178209.639422:0:3339:0:(ldlm_inodebits.c:94:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x240000402:0x7f9:0x0].0x0 (00000000de0abd11) 00010000:00010000:2.0:1776178209.639424:0:3339:0:(ldlm_inodebits.c:96:ldlm_reprocess_inodebits_queue()) Hint 3 00010000:00010000:2.0:1776178209.639426:0:3339:0:(ldlm_lockd.c:1903:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: mdt-lustre-MDT0001_UUID lock: 00000000575572c2/0xa40083ab9bf53747 lrc: 2/0,0 mode: --/COS res: [0x240000402:0x7f9:0x0].0x0 bits 0x3/0x0 rrc: 2 type: IBT gid 0 flags: 0x44a19400000000 nid: local remote: 0x0 expref: -99 pid: 11281 timeout: 0 lvb_type: 0 00010000:00010000:2.0:1776178209.639431:0:3339:0:(ldlm_lock.c:216:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: mdt-lustre-MDT0001_UUID lock: 00000000575572c2/0xa40083ab9bf53747 lrc: 0/0,0 mode: --/COS res: [0x240000402:0x7f9:0x0].0x0 bits 0x3/0x0 rrc: 2 type: IBT gid 0 flags: 0x44a19400000000 nid: local remote: 0x0 expref: -99 pid: 11281 timeout: 0 lvb_type: 0 00010000:00010000:2.0:1776178209.639444:0:3339:0:(ldlm_lock.c:820:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(COS) ns: mdt-lustre-MDT0001_UUID lock: 000000002a990613/0xa40083ab9bf5379b lrc: 3/0,1 mode: COS/COS res: [0x240000402:0x7f7:0x0].0x0 bits 0x2/0x0 rrc: 2 type: IBT gid 0 flags: 0x50210001000000 nid: local remote: 0x0 expref: -99 pid: 6257 timeout: 0 lvb_type: 0 00010000:00010000:2.0:1776178209.639449:0:3339:0:(ldlm_lock.c:879:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: mdt-lustre-MDT0001_UUID lock: 000000002a990613/0xa40083ab9bf5379b lrc: 2/0,0 mode: COS/COS res: [0x240000402:0x7f7:0x0].0x0 bits 0x2/0x0 rrc: 2 type: IBT gid 0 flags: 0x50210401000000 nid: local remote: 0x0 expref: -99 pid: 6257 timeout: 0 lvb_type: 0 00010000:00010000:2.0:1776178209.639453:0:3339:0:(ldlm_lockd.c:1879:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: mdt-lustre-MDT0001_UUID lock: 000000002a990613/0xa40083ab9bf5379b lrc: 3/0,0 mode: COS/COS res: [0x240000402:0x7f7:0x0].0x0 bits 0x2/0x0 rrc: 2 type: IBT gid 0 flags: 0x40210401000000 nid: local remote: 0x0 expref: -99 pid: 6257 timeout: 0 lvb_type: 0 00010000:00010000:2.0:1776178209.639458:0:3339:0:(ldlm_lockd.c:1893:ldlm_handle_bl_callback()) Lock 000000002a990613 already unused, calling callback (0000000079e78348) 00010000:00010000:2.0:1776178209.639459:0:3339:0:(ldlm_request.c:364:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: mdt-lustre-MDT0001_UUID lock: 000000002a990613/0xa40083ab9bf5379b lrc: 3/0,0 mode: COS/COS res: [0x240000402:0x7f7:0x0].0x0 bits 0x2/0x0 rrc: 2 type: IBT gid 0 flags: 0x40210401000000 nid: local remote: 0x0 expref: -99 pid: 6257 timeout: 0 lvb_type: 0 00010000:00010000:2.0:1776178209.639464:0:3339:0:(ldlm_request.c:1257:ldlm_cli_cancel_local()) ### server-side local cancel ns: mdt-lustre-MDT0001_UUID lock: 000000002a990613/0xa40083ab9bf5379b lrc: 4/0,0 mode: COS/COS res: [0x240000402:0x7f7:0x0].0x0 bits 0x2/0x0 rrc: 2 type: IBT gid 0 flags: 0x40218401000000 nid: local remote: 0x0 expref: -99 pid: 6257 timeout: 0 lvb_type: 0 00010000:00010000:2.0:1776178209.639471:0:3339:0:(ldlm_inodebits.c:94:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x240000402:0x7f7:0x0].0x0 (00000000b27056f4) 00010000:00010000:2.0:1776178209.639474:0:3339:0:(ldlm_inodebits.c:96:ldlm_reprocess_inodebits_queue()) Hint 2 00010000:00010000:2.0:1776178209.639476:0:3339:0:(ldlm_lockd.c:1903:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: mdt-lustre-MDT0001_UUID lock: 000000002a990613/0xa40083ab9bf5379b lrc: 2/0,0 mode: --/COS res: [0x240000402:0x7f7:0x0].0x0 bits 0x2/0x0 rrc: 2 type: IBT gid 0 flags: 0x44a19401000000 nid: local remote: 0x0 expref: -99 pid: 6257 timeout: 0 lvb_type: 0 00010000:00010000:2.0:1776178209.639480:0:3339:0:(ldlm_lock.c:216:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: mdt-lustre-MDT0001_UUID lock: 000000002a990613/0xa40083ab9bf5379b lrc: 0/0,0 mode: --/COS res: [0x240000402:0x7f7:0x0].0x0 bits 0x2/0x0 rrc: 2 type: IBT gid 0 flags: 0x44a19401000000 nid: local remote: 0x0 expref: -99 pid: 6257 timeout: 0 lvb_type: 0 00000100:00100000:1.0:1776178209.639582:0:3354:0:(events.c:361:request_in_callback()) peer: 12345-0@lo (source: 12345-0@lo) 00000100:00100000:1.0:1776178209.639622:0:3354:0:(client.c:1733:ptlrpc_send_new_req()) Sending RPC req@00000000f433e404 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_01:lustre-MDT0001-mdtlov_UUID:3354:1862456998048064:0@lo:6:osp-syn-0-1.0 00000100:00100000:1.0:1776178209.639649:0:3354:0:(events.c:361:request_in_callback()) peer: 12345-0@lo (source: 12345-0@lo) 00000100:00100000:1.0:1776178209.639682:0:3355:0:(client.c:1733:ptlrpc_send_new_req()) Sending RPC req@000000009e7a8bb5 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_02:lustre-MDT0001-mdtlov_UUID:3355:1862456998047680:0@lo:6:osp-syn-1-1.0 00000100:00100000:1.0:1776178209.639708:0:3355:0:(events.c:361:request_in_callback()) peer: 12345-0@lo (source: 12345-0@lo) 00000100:00100000:1.0:1776178209.639725:0:3355:0:(client.c:1733:ptlrpc_send_new_req()) Sending RPC req@0000000044f82452 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_02:lustre-MDT0001-mdtlov_UUID:3355:1862456998047936:0@lo:6:osp-syn-0-1.0 00000100:00100000:1.0:1776178209.639745:0:3355:0:(events.c:361:request_in_callback()) peer: 12345-0@lo (source: 12345-0@lo) 00000100:00100000:1.0:1776178209.639763:0:3355:0:(ptlrpcd.c:415:ptlrpcd_check()) transfer 2 async RPCs [3->2] 00000100:00100000:1.0:1776178209.639768:0:3355:0:(client.c:1733:ptlrpc_send_new_req()) Sending RPC req@0000000072865dff pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_02:lustre-MDT0001-mdtlov_UUID:3355:1862456998047744:0@lo:6:osp-syn-1-1.0 00000100:00100000:1.0:1776178209.639791:0:3355:0:(events.c:361:request_in_callback()) peer: 12345-0@lo (source: 12345-0@lo) 00000100:00100000:1.0:1776178209.639807:0:3355:0:(client.c:1733:ptlrpc_send_new_req()) Sending RPC req@0000000028ade006 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_02:lustre-MDT0001-mdtlov_UUID:3355:1862456998048000:0@lo:6:osp-syn-0-1.0 00000100:00100000:1.0:1776178209.639827:0:3355:0:(events.c:361:request_in_callback()) peer: 12345-0@lo (source: 12345-0@lo) 00000100:00100000:1.0:1776178209.639865:0:6242:0:(service.c:2152:ptlrpc_server_handle_req_in()) got req x1862456998047552 00000100:00100000:1.0:1776178209.639926:0:6242:0:(service.c:2152:ptlrpc_server_handle_req_in()) got req x1862456998047808 00000100:00100000:1.0:1776178209.639951:0:6242:0:(nrs_fifo.c:179:nrs_fifo_req_get()) NRS start fifo request from 12345-0@lo, seq: 47 00000100:00100000:1.0:1776178209.639957:0:6242:0:(service.c:2324:ptlrpc_server_handle_request()) Handling RPC req@00000000ba9bc15d pname:cluuid+ref:pid:xid:nid:opc:job ldlm_cn00_001:lustre-MDT0001-mdtlov_UUID+10:3354:x1862456998047552:12345-0@lo:103:jbd2/dm-1-8.0 00010000:00010000:1.0:1776178209.639973:0:6242:0:(ldlm_lockd.c:1720:ldlm_request_cancel()) ### server-side cancel handler START: 1 locks, starting at 0 00010000:00010000:1.0:1776178209.639992:0:6242:0:(ldlm_lock.c:216:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: mdt-lustre-MDT0000_UUID lock: 000000006cb7334a/0xa40083ab9bf53786 lrc: 0/0,0 mode: --/EX res: [0x200000007:0x1:0x0].0x0 bits 0x2/0x0 rrc: 5 type: IBT gid 0 flags: 0x44801001000000 nid: 0@lo remote: 0xa40083ab9bf5377f expref: 10 pid: 6261 timeout: 0 lvb_type: 0 00010000:00010000:1.0:1776178209.640175:0:6242:0:(ldlm_inodebits.c:94:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x200000007:0x1:0x0].0x0 (00000000ad0d4094) 00010000:00010000:1.0:1776178209.640181:0:6242:0:(ldlm_lockd.c:1782:ldlm_request_cancel()) ### server-side cancel handler END 00000100:00100000:1.0:1776178209.640225:0:6242:0:(service.c:2372:ptlrpc_server_handle_request()) Handled RPC req@00000000ba9bc15d pname:cluuid+ref:pid:xid:nid:opc:job ldlm_cn00_001:lustre-MDT0001-mdtlov_UUID+9:3354:x1862456998047552:12345-0@lo:103:jbd2/dm-1-8.0 Request processed in 267us (1316us total) trans 0 rc 0/0 00000100:00100000:1.0:1776178209.640234:0:6242:0:(nrs_fifo.c:241:nrs_fifo_req_stop()) NRS stop fifo request from 12345-0@lo, seq: 47 00000100:00100000:1.0:1776178209.640258:0:6242:0:(nrs_fifo.c:179:nrs_fifo_req_get()) NRS start fifo request from 12345-0@lo, seq: 48 00000100:00100000:1.0:1776178209.640263:0:6242:0:(service.c:2324:ptlrpc_server_handle_request()) Handling RPC req@00000000f4172b4c pname:cluuid+ref:pid:xid:nid:opc:job ldlm_cn00_001:lustre-MDT0001-mdtlov_UUID+8:3354:x1862456998047808:12345-0@lo:103:jbd2/dm-1-8.0 00010000:00010000:1.0:1776178209.640271:0:6242:0:(ldlm_lockd.c:1720:ldlm_request_cancel()) ### server-side cancel handler START: 1 locks, starting at 0 00010000:00010000:1.0:1776178209.640281:0:6242:0:(ldlm_lock.c:216:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: mdt-lustre-MDT0000_UUID lock: 0000000073f2c7a9/0xa40083ab9bf53794 lrc: 0/0,0 mode: --/EX res: [0x240000402:0x7f7:0x0].0x0 bits 0x1/0x0 rrc: 3 type: IBT gid 0 flags: 0x44801001000000 nid: 0@lo remote: 0xa40083ab9bf5378d expref: 8 pid: 15213 timeout: 0 lvb_type: 0 00010000:00010000:1.0:1776178209.640294:0:6242:0:(ldlm_inodebits.c:94:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x240000402:0x7f7:0x0].0x0 (000000003c4b13ad) 00010000:00010000:1.0:1776178209.640302:0:6242:0:(ldlm_lockd.c:1782:ldlm_request_cancel()) ### server-side cancel handler END 00000100:00100000:1.0:1776178209.640322:0:6242:0:(service.c:2372:ptlrpc_server_handle_request()) Handled RPC req@00000000f4172b4c pname:cluuid+ref:pid:xid:nid:opc:job ldlm_cn00_001:lustre-MDT0001-mdtlov_UUID+7:3354:x1862456998047808:12345-0@lo:103:jbd2/dm-1-8.0 Request processed in 60us (1352us total) trans 0 rc 0/0 00000100:00100000:1.0:1776178209.640328:0:6242:0:(nrs_fifo.c:241:nrs_fifo_req_stop()) NRS stop fifo request from 12345-0@lo, seq: 48 00000100:00100000:1.0:1776178209.640354:0:20762:0:(service.c:2152:ptlrpc_server_handle_req_in()) got req x1862456998047872 00000100:00100000:1.0:1776178209.640376:0:20762:0:(service.c:2152:ptlrpc_server_handle_req_in()) got req x1862456998048064 00000100:00100000:1.0:1776178209.640394:0:20762:0:(service.c:2152:ptlrpc_server_handle_req_in()) got req x1862456998047680 00000100:00100000:1.0:1776178209.640408:0:20762:0:(service.c:2152:ptlrpc_server_handle_req_in()) got req x1862456998047936 00000100:00100000:1.0:1776178209.640424:0:20762:0:(service.c:2152:ptlrpc_server_handle_req_in()) got req x1862456998047744 00000100:00100000:1.0:1776178209.640448:0:20762:0:(service.c:2152:ptlrpc_server_handle_req_in()) got req x1862456998048000 00000100:00100000:1.0:1776178209.640465:0:20762:0:(nrs_fifo.c:179:nrs_fifo_req_get()) NRS start fifo request from 12345-0@lo, seq: 5279 00000100:00100000:1.0:1776178209.640474:0:20762:0:(service.c:2324:ptlrpc_server_handle_request()) Handling RPC req@00000000d9bdf497 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_013:lustre-MDT0001-mdtlov_UUID+8:3354:x1862456998047872:12345-0@lo:6:osp-syn-1-1.0 00000100:00100000:3.0:1776178209.640475:0:20758:0:(nrs_fifo.c:179:nrs_fifo_req_get()) NRS start fifo request from 12345-0@lo, seq: 5280 00000100:00100000:3.0:1776178209.640485:0:20758:0:(service.c:2324:ptlrpc_server_handle_request()) Handling RPC req@000000006165f052 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_009:lustre-MDT0001-mdtlov_UUID+7:3354:x1862456998048064:12345-0@lo:6:osp-syn-0-1.0 00002000:00080000:1.0:1776178209.640500:0:20762:0:(ofd_dev.c:1822:ofd_destroy_hdl()) lustre-OST0001: Destroy object 0x2c0000400:1072 count 1 00002000:00080000:3.0:1776178209.640504:0:20758:0:(ofd_dev.c:1822:ofd_destroy_hdl()) lustre-OST0000: Destroy object 0x280000400:1134 count 1 00010000:00010000:1.0:1776178209.640526:0:20762:0:(ldlm_lock.c:766:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0001_UUID lock: 00000000daa24072/0xa40083ab9bf537a9 lrc: 3/0,1 mode: --/PW res: [0x2c0000400:0x430:0x0].0x0 rrc: 3 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 20762 timeout: 0 lvb_type: 0 00010000:00010000:3.0:1776178209.640526:0:20758:0:(ldlm_lock.c:766:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0000_UUID lock: 00000000014a8a18/0xa40083ab9bf537b0 lrc: 3/0,1 mode: --/PW res: [0x280000400:0x46e:0x0].0x0 rrc: 3 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 20758 timeout: 0 lvb_type: 0 00010000:00010000:1.0:1776178209.640555:0:20762:0:(ldlm_lock.c:675:ldlm_add_bl_work_item()) ### lock incompatible; sending blocking AST. ns: filter-lustre-OST0001_UUID lock: 000000002abc4399/0xa40083ab9bf5353a lrc: 2/0,0 mode: PR/PR res: [0x2c0000400:0x430:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x40000000000000 nid: 192.168.204.3@tcp remote: 0x771f68aa1eb27b2d expref: 8 pid: 20759 timeout: 0 lvb_type: 1 00010000:00010000:3.0:1776178209.640556:0:20758:0:(ldlm_lock.c:675:ldlm_add_bl_work_item()) ### lock incompatible; sending blocking AST. ns: filter-lustre-OST0000_UUID lock: 000000006b206cc9/0xa40083ab9bf5351e lrc: 2/0,0 mode: PR/PR res: [0x280000400:0x46e:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x40000000000000 nid: 192.168.204.3@tcp remote: 0x771f68aa1eb27b11 expref: 7 pid: 20762 timeout: 0 lvb_type: 1 00010000:00010000:1.0:1776178209.640575:0:20762:0:(ldlm_extent.c:318:ldlm_check_contention()) contended locks = 0 00010000:00010000:3.0:1776178209.640575:0:20758:0:(ldlm_extent.c:318:ldlm_check_contention()) contended locks = 0 00010000:00010000:1.0:1776178209.640578:0:20762:0:(ldlm_extent.c:318:ldlm_check_contention()) contended locks = 0 00010000:00010000:3.0:1776178209.640579:0:20758:0:(ldlm_extent.c:318:ldlm_check_contention()) contended locks = 0 00010000:00010000:1.0:1776178209.640582:0:20762:0:(ldlm_resource.c:1650:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0001_UUID lock: 00000000daa24072/0xa40083ab9bf537a9 lrc: 4/0,1 mode: --/PW res: [0x2c0000400:0x430:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x50010080000000 nid: local remote: 0x0 expref: -99 pid: 20762 timeout: 0 lvb_type: 0 00010000:00010000:3.0:1776178209.640583:0:20758:0:(ldlm_resource.c:1650:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 00000000014a8a18/0xa40083ab9bf537b0 lrc: 4/0,1 mode: --/PW res: [0x280000400:0x46e:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x50010080000000 nid: local remote: 0x0 expref: -99 pid: 20758 timeout: 0 lvb_type: 0 00010000:00010000:1.0:1776178209.640638:0:20762:0:(ldlm_lockd.c:953:ldlm_server_blocking_ast()) ### server preparing blocking AST ns: filter-lustre-OST0001_UUID lock: 000000002abc4399/0xa40083ab9bf5353a lrc: 3/0,0 mode: PR/PR res: [0x2c0000400:0x430:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x50000000010020 nid: 192.168.204.3@tcp remote: 0x771f68aa1eb27b2d expref: 8 pid: 20759 timeout: 0 lvb_type: 1 00010000:00010000:3.0:1776178209.640639:0:20758:0:(ldlm_lockd.c:953:ldlm_server_blocking_ast()) ### server preparing blocking AST ns: filter-lustre-OST0000_UUID lock: 000000006b206cc9/0xa40083ab9bf5351e lrc: 3/0,0 mode: PR/PR res: [0x280000400:0x46e:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x50000000010020 nid: 192.168.204.3@tcp remote: 0x771f68aa1eb27b11 expref: 7 pid: 20762 timeout: 0 lvb_type: 1 00010000:00010000:1.0:1776178209.640661:0:20762:0:(ldlm_lockd.c:489:ldlm_add_waiting_lock()) ### adding to wait list(timeout: 100, AT: on) ns: filter-lustre-OST0001_UUID lock: 000000002abc4399/0xa40083ab9bf5353a lrc: 4/0,0 mode: PR/PR res: [0x2c0000400:0x430:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x70000400010020 nid: 192.168.204.3@tcp remote: 0x771f68aa1eb27b2d expref: 8 pid: 20759 timeout: 1016 lvb_type: 1 00010000:00010000:3.0:1776178209.640662:0:20758:0:(ldlm_lockd.c:489:ldlm_add_waiting_lock()) ### adding to wait list(timeout: 100, AT: on) ns: filter-lustre-OST0000_UUID lock: 000000006b206cc9/0xa40083ab9bf5351e lrc: 4/0,0 mode: PR/PR res: [0x280000400:0x46e:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x70000400010020 nid: 192.168.204.3@tcp remote: 0x771f68aa1eb27b11 expref: 7 pid: 20762 timeout: 1016 lvb_type: 1 00000100:00100000:1.0:1776178209.640688:0:20762:0:(client.c:1733:ptlrpc_send_new_req()) Sending RPC req@00000000b7440871 pname:cluuid:pid:xid:nid:opc:job ll_ost00_013:lustre-OST0001_UUID:20762:1862456998048256:192.168.204.3@tcp:104: 00000100:00100000:3.0:1776178209.640689:0:20758:0:(client.c:1733:ptlrpc_send_new_req()) Sending RPC req@0000000068680f90 pname:cluuid:pid:xid:nid:opc:job ll_ost00_009:lustre-OST0000_UUID:20758:1862456998048192:192.168.204.3@tcp:104: 00000100:00100000:2.0:1776178209.640783:0:20765:0:(nrs_fifo.c:179:nrs_fifo_req_get()) NRS start fifo request from 12345-0@lo, seq: 5281 00000100:00100000:2.0:1776178209.640788:0:20765:0:(service.c:2324:ptlrpc_server_handle_request()) Handling RPC req@00000000d9d2b484 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_016:lustre-MDT0001-mdtlov_UUID+8:3355:x1862456998047680:12345-0@lo:6:osp-syn-1-1.0 00002000:00080000:2.0:1776178209.640802:0:20765:0:(ofd_dev.c:1822:ofd_destroy_hdl()) lustre-OST0001: Destroy object 0x2c0000400:1071 count 1 00010000:00010000:2.0:1776178209.640813:0:20765:0:(ldlm_lock.c:766:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0001_UUID lock: 000000007ae7b5f6/0xa40083ab9bf537b7 lrc: 3/0,1 mode: --/PW res: [0x2c0000400:0x42f:0x0].0x0 rrc: 3 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 20765 timeout: 0 lvb_type: 0 00010000:00010000:2.0:1776178209.640824:0:20765:0:(ldlm_lock.c:675:ldlm_add_bl_work_item()) ### lock incompatible; sending blocking AST. ns: filter-lustre-OST0001_UUID lock: 00000000d999afdf/0xa40083ab9bf53525 lrc: 2/0,0 mode: PR/PR res: [0x2c0000400:0x42f:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x40000000000000 nid: 192.168.204.3@tcp remote: 0x771f68aa1eb27b18 expref: 8 pid: 20762 timeout: 0 lvb_type: 1 00010000:00010000:2.0:1776178209.640829:0:20765:0:(ldlm_extent.c:318:ldlm_check_contention()) contended locks = 0 00010000:00010000:2.0:1776178209.640831:0:20765:0:(ldlm_extent.c:318:ldlm_check_contention()) contended locks = 0 00010000:00010000:2.0:1776178209.640833:0:20765:0:(ldlm_resource.c:1650:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0001_UUID lock: 000000007ae7b5f6/0xa40083ab9bf537b7 lrc: 4/0,1 mode: --/PW res: [0x2c0000400:0x42f:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x50010080000000 nid: local remote: 0x0 expref: -99 pid: 20765 timeout: 0 lvb_type: 0 00010000:00010000:2.0:1776178209.640850:0:20765:0:(ldlm_lockd.c:953:ldlm_server_blocking_ast()) ### server preparing blocking AST ns: filter-lustre-OST0001_UUID lock: 00000000d999afdf/0xa40083ab9bf53525 lrc: 3/0,0 mode: PR/PR res: [0x2c0000400:0x42f:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x50000000010020 nid: 192.168.204.3@tcp remote: 0x771f68aa1eb27b18 expref: 8 pid: 20762 timeout: 0 lvb_type: 1 00010000:00010000:2.0:1776178209.640858:0:20765:0:(ldlm_lockd.c:489:ldlm_add_waiting_lock()) ### adding to wait list(timeout: 100, AT: on) ns: filter-lustre-OST0001_UUID lock: 00000000d999afdf/0xa40083ab9bf53525 lrc: 4/0,0 mode: PR/PR res: [0x2c0000400:0x42f:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x70000400010020 nid: 192.168.204.3@tcp remote: 0x771f68aa1eb27b18 expref: 8 pid: 20762 timeout: 1016 lvb_type: 1 00000100:00100000:2.0:1776178209.640872:0:20765:0:(client.c:1733:ptlrpc_send_new_req()) Sending RPC req@0000000065dbeee0 pname:cluuid:pid:xid:nid:opc:job ll_ost00_016:lustre-OST0001_UUID:20765:1862456998048320:192.168.204.3@tcp:104: 00000100:00100000:1.0:1776178209.640942:0:20762:0:(client.c:2500:ptlrpc_set_wait()) set 00000000effabc06 going to sleep for 6 seconds 00000100:00100000:1.0:1776178209.640967:0:8227:0:(nrs_fifo.c:179:nrs_fifo_req_get()) NRS start fifo request from 12345-0@lo, seq: 5282 00000100:00100000:1.0:1776178209.640971:0:8227:0:(service.c:2324:ptlrpc_server_handle_request()) Handling RPC req@000000001a990b02 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_000:lustre-MDT0001-mdtlov_UUID+7:3355:x1862456998047936:12345-0@lo:6:osp-syn-0-1.0 00002000:00080000:1.0:1776178209.640985:0:8227:0:(ofd_dev.c:1822:ofd_destroy_hdl()) lustre-OST0000: Destroy object 0x280000400:1136 count 1 00010000:00010000:1.0:1776178209.640997:0:8227:0:(ldlm_lock.c:766:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0000_UUID lock: 000000006b0afb52/0xa40083ab9bf537be lrc: 3/0,1 mode: --/PW res: [0x280000400:0x470:0x0].0x0 rrc: 3 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 8227 timeout: 0 lvb_type: 0 00010000:00010000:1.0:1776178209.641021:0:8227:0:(ldlm_lock.c:675:ldlm_add_bl_work_item()) ### lock incompatible; sending blocking AST. ns: filter-lustre-OST0000_UUID lock: 00000000a1e024a5/0xa40083ab9bf5352c lrc: 2/0,0 mode: PR/PR res: [0x280000400:0x470:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x40000000000000 nid: 192.168.204.3@tcp remote: 0x771f68aa1eb27b1f expref: 7 pid: 20759 timeout: 0 lvb_type: 1 00010000:00010000:1.0:1776178209.641028:0:8227:0:(ldlm_extent.c:318:ldlm_check_contention()) contended locks = 0 00010000:00010000:1.0:1776178209.641029:0:8227:0:(ldlm_extent.c:318:ldlm_check_contention()) contended locks = 0 00010000:00010000:1.0:1776178209.641031:0:8227:0:(ldlm_resource.c:1650:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 000000006b0afb52/0xa40083ab9bf537be lrc: 4/0,1 mode: --/PW res: [0x280000400:0x470:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x50010080000000 nid: local remote: 0x0 expref: -99 pid: 8227 timeout: 0 lvb_type: 0 00000100:00100000:3.0:1776178209.641150:0:20758:0:(client.c:2500:ptlrpc_set_wait()) set 000000003b716127 going to sleep for 6 seconds 00010000:00010000:1.0:1776178209.641157:0:8227:0:(ldlm_lockd.c:953:ldlm_server_blocking_ast()) ### server preparing blocking AST ns: filter-lustre-OST0000_UUID lock: 00000000a1e024a5/0xa40083ab9bf5352c lrc: 3/0,0 mode: PR/PR res: [0x280000400:0x470:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x50000000010020 nid: 192.168.204.3@tcp remote: 0x771f68aa1eb27b1f expref: 7 pid: 20759 timeout: 0 lvb_type: 1 00010000:00010000:1.0:1776178209.641166:0:8227:0:(ldlm_lockd.c:489:ldlm_add_waiting_lock()) ### adding to wait list(timeout: 100, AT: on) ns: filter-lustre-OST0000_UUID lock: 00000000a1e024a5/0xa40083ab9bf5352c lrc: 4/0,0 mode: PR/PR res: [0x280000400:0x470:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x70000400010020 nid: 192.168.204.3@tcp remote: 0x771f68aa1eb27b1f expref: 7 pid: 20759 timeout: 1016 lvb_type: 1 00000100:00100000:1.0:1776178209.641175:0:8227:0:(client.c:1733:ptlrpc_send_new_req()) Sending RPC req@00000000b30b9cdd pname:cluuid:pid:xid:nid:opc:job ll_ost00_000:lustre-OST0000_UUID:8227:1862456998048384:192.168.204.3@tcp:104: 00000100:00100000:1.0:1776178209.641205:0:8227:0:(client.c:2500:ptlrpc_set_wait()) set 00000000d5cddc74 going to sleep for 6 seconds 00000100:00100000:1.0:1776178209.641226:0:20755:0:(nrs_fifo.c:179:nrs_fifo_req_get()) NRS start fifo request from 12345-0@lo, seq: 5283 00000100:00100000:1.0:1776178209.641230:0:20755:0:(service.c:2324:ptlrpc_server_handle_request()) Handling RPC req@00000000d17c6101 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_006:lustre-MDT0001-mdtlov_UUID+8:3355:x1862456998047744:12345-0@lo:6:osp-syn-1-1.0 00002000:00080000:1.0:1776178209.641243:0:20755:0:(ofd_dev.c:1822:ofd_destroy_hdl()) lustre-OST0001: Destroy object 0x2c0000400:1073 count 1 00010000:00010000:1.0:1776178209.641256:0:20755:0:(ldlm_lock.c:766:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0001_UUID lock: 00000000a14e680b/0xa40083ab9bf537c5 lrc: 3/0,1 mode: --/PW res: [0x2c0000400:0x431:0x0].0x0 rrc: 3 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 20755 timeout: 0 lvb_type: 0 00010000:00010000:1.0:1776178209.641267:0:20755:0:(ldlm_lock.c:675:ldlm_add_bl_work_item()) ### lock incompatible; sending blocking AST. ns: filter-lustre-OST0001_UUID lock: 000000003e1697a2/0xa40083ab9bf53533 lrc: 2/0,0 mode: PR/PR res: [0x2c0000400:0x431:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x40000000000000 nid: 192.168.204.3@tcp remote: 0x771f68aa1eb27b26 expref: 8 pid: 20759 timeout: 0 lvb_type: 1 00010000:00010000:1.0:1776178209.641273:0:20755:0:(ldlm_extent.c:318:ldlm_check_contention()) contended locks = 0 00010000:00010000:1.0:1776178209.641274:0:20755:0:(ldlm_extent.c:318:ldlm_check_contention()) contended locks = 0 00010000:00010000:1.0:1776178209.641275:0:20755:0:(ldlm_resource.c:1650:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0001_UUID lock: 00000000a14e680b/0xa40083ab9bf537c5 lrc: 4/0,1 mode: --/PW res: [0x2c0000400:0x431:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x50010080000000 nid: local remote: 0x0 expref: -99 pid: 20755 timeout: 0 lvb_type: 0 00010000:00010000:1.0:1776178209.641297:0:20755:0:(ldlm_lockd.c:953:ldlm_server_blocking_ast()) ### server preparing blocking AST ns: filter-lustre-OST0001_UUID lock: 000000003e1697a2/0xa40083ab9bf53533 lrc: 3/0,0 mode: PR/PR res: [0x2c0000400:0x431:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x50000000010020 nid: 192.168.204.3@tcp remote: 0x771f68aa1eb27b26 expref: 8 pid: 20759 timeout: 0 lvb_type: 1 00010000:00010000:1.0:1776178209.641304:0:20755:0:(ldlm_lockd.c:489:ldlm_add_waiting_lock()) ### adding to wait list(timeout: 100, AT: on) ns: filter-lustre-OST0001_UUID lock: 000000003e1697a2/0xa40083ab9bf53533 lrc: 4/0,0 mode: PR/PR res: [0x2c0000400:0x431:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x70000400010020 nid: 192.168.204.3@tcp remote: 0x771f68aa1eb27b26 expref: 8 pid: 20759 timeout: 1016 lvb_type: 1 00000100:00100000:1.0:1776178209.641313:0:20755:0:(client.c:1733:ptlrpc_send_new_req()) Sending RPC req@00000000eeeaf4ad pname:cluuid:pid:xid:nid:opc:job ll_ost00_006:lustre-OST0001_UUID:20755:1862456998048448:192.168.204.3@tcp:104: 00000100:00100000:1.0:1776178209.641337:0:20755:0:(client.c:2500:ptlrpc_set_wait()) set 000000000b150ab3 going to sleep for 6 seconds 00000100:00100000:1.0:1776178209.641354:0:14440:0:(nrs_fifo.c:179:nrs_fifo_req_get()) NRS start fifo request from 12345-0@lo, seq: 5284 00000100:00100000:1.0:1776178209.641360:0:14440:0:(service.c:2324:ptlrpc_server_handle_request()) Handling RPC req@000000008a1e777f pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_004:lustre-MDT0001-mdtlov_UUID+7:3355:x1862456998048000:12345-0@lo:6:osp-syn-0-1.0 00002000:00080000:1.0:1776178209.641373:0:14440:0:(ofd_dev.c:1822:ofd_destroy_hdl()) lustre-OST0000: Destroy object 0x280000400:1135 count 1 00010000:00010000:1.0:1776178209.641383:0:14440:0:(ldlm_lock.c:766:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: filter-lustre-OST0000_UUID lock: 0000000097975a3b/0xa40083ab9bf537cc lrc: 3/0,1 mode: --/PW res: [0x280000400:0x46f:0x0].0x0 rrc: 3 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 14440 timeout: 0 lvb_type: 0 00010000:00010000:1.0:1776178209.641392:0:14440:0:(ldlm_lock.c:675:ldlm_add_bl_work_item()) ### lock incompatible; sending blocking AST. ns: filter-lustre-OST0000_UUID lock: 00000000f51fd0d7/0xa40083ab9bf53517 lrc: 2/0,0 mode: PR/PR res: [0x280000400:0x46f:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x40000000000000 nid: 192.168.204.3@tcp remote: 0x771f68aa1eb27b0a expref: 7 pid: 20762 timeout: 0 lvb_type: 1 00010000:00010000:1.0:1776178209.641397:0:14440:0:(ldlm_extent.c:318:ldlm_check_contention()) contended locks = 0 00010000:00010000:1.0:1776178209.641398:0:14440:0:(ldlm_extent.c:318:ldlm_check_contention()) contended locks = 0 00010000:00010000:1.0:1776178209.641400:0:14440:0:(ldlm_resource.c:1650:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 0000000097975a3b/0xa40083ab9bf537cc lrc: 4/0,1 mode: --/PW res: [0x280000400:0x46f:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x50010080000000 nid: local remote: 0x0 expref: -99 pid: 14440 timeout: 0 lvb_type: 0 00010000:00010000:1.0:1776178209.641416:0:14440:0:(ldlm_lockd.c:953:ldlm_server_blocking_ast()) ### server preparing blocking AST ns: filter-lustre-OST0000_UUID lock: 00000000f51fd0d7/0xa40083ab9bf53517 lrc: 3/0,0 mode: PR/PR res: [0x280000400:0x46f:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x50000000010020 nid: 192.168.204.3@tcp remote: 0x771f68aa1eb27b0a expref: 7 pid: 20762 timeout: 0 lvb_type: 1 00010000:00010000:1.0:1776178209.641429:0:14440:0:(ldlm_lockd.c:489:ldlm_add_waiting_lock()) ### adding to wait list(timeout: 100, AT: on) ns: filter-lustre-OST0000_UUID lock: 00000000f51fd0d7/0xa40083ab9bf53517 lrc: 4/0,0 mode: PR/PR res: [0x280000400:0x46f:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x70000400010020 nid: 192.168.204.3@tcp remote: 0x771f68aa1eb27b0a expref: 7 pid: 20762 timeout: 1016 lvb_type: 1 00000100:00100000:1.0:1776178209.641437:0:14440:0:(client.c:1733:ptlrpc_send_new_req()) Sending RPC req@00000000a8bae24c pname:cluuid:pid:xid:nid:opc:job ll_ost00_004:lustre-OST0000_UUID:14440:1862456998048512:192.168.204.3@tcp:104: 00000100:00100000:1.0:1776178209.641459:0:14440:0:(client.c:2500:ptlrpc_set_wait()) set 00000000a015a589 going to sleep for 6 seconds 00000100:00100000:1.0:1776178209.641522:0:3354:0:(client.c:2215:ptlrpc_check_set()) Completed RPC req@00000000ab295186 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_01:lustre-MDT0001-mdtlov_UUID:3354:1862456998047552:0@lo:103:jbd2/dm-1-8.0 00000100:00100000:1.0:1776178209.641534:0:3354:0:(client.c:2215:ptlrpc_check_set()) Completed RPC req@00000000f8e38e41 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_01:lustre-MDT0001-mdtlov_UUID:3354:1862456998047808:0@lo:103:jbd2/dm-1-8.0 00000100:00100000:0.0:1776178209.642671:0:20759:0:(client.c:2500:ptlrpc_set_wait()) set 000000000b1f0cc6 going to sleep for 6 seconds 00000100:00100000:2.0:1776178209.643348:0:20765:0:(client.c:2500:ptlrpc_set_wait()) set 00000000b8f976e8 going to sleep for 6 seconds 00000100:00100000:2.0:1776178209.652142:0:20762:0:(client.c:2215:ptlrpc_check_set()) Completed RPC req@00000000b7440871 pname:cluuid:pid:xid:nid:opc:job ll_ost00_013:lustre-OST0001_UUID:20762:1862456998048256:192.168.204.3@tcp:104: 00010000:00010000:2.0:1776178209.652182:0:20762:0:(ldlm_request.c:283:ldlm_completion_ast()) ### client-side enqueue returned a blocked locksleeping ns: filter-lustre-OST0001_UUID lock: 00000000daa24072/0xa40083ab9bf537a9 lrc: 3/0,1 mode: --/PW res: [0x2c0000400:0x430:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x40010080000000 nid: local remote: 0x0 expref: -99 pid: 20762 timeout: 0 lvb_type: 0 00000100:00100000:1.0:1776178209.653446:0:3348:0:(events.c:361:request_in_callback()) peer: 12345-192.168.204.3@tcp (source: 12345-192.168.204.3@tcp) 00000100:00100000:1.0:1776178209.653477:0:3348:0:(events.c:361:request_in_callback()) peer: 12345-192.168.204.3@tcp (source: 12345-192.168.204.3@tcp) 00000100:00100000:1.0:1776178209.653495:0:3348:0:(events.c:361:request_in_callback()) peer: 12345-192.168.204.3@tcp (source: 12345-192.168.204.3@tcp) 00000100:00100000:1.0:1776178209.653538:0:3348:0:(events.c:361:request_in_callback()) peer: 12345-192.168.204.3@tcp (source: 12345-192.168.204.3@tcp) 00000100:00100000:1.0:1776178209.653556:0:3348:0:(events.c:361:request_in_callback()) peer: 12345-192.168.204.3@tcp (source: 12345-192.168.204.3@tcp) 00000100:00100000:1.0:1776178209.653575:0:3348:0:(events.c:361:request_in_callback()) peer: 12345-192.168.204.3@tcp (source: 12345-192.168.204.3@tcp) 00000100:00100000:1.0:1776178209.653655:0:3348:0:(events.c:361:request_in_callback()) peer: 12345-192.168.204.3@tcp (source: 12345-192.168.204.3@tcp) 00000100:00100000:1.0:1776178209.653701:0:6243:0:(service.c:2152:ptlrpc_server_handle_req_in()) got req x1862456979102592 00010000:00010000:1.0:1776178209.653746:0:6243:0:(ldlm_lockd.c:2638:ldlm_cancel_hpreq_check()) ### hpreq cancel/convert lock ns: filter-lustre-OST0001_UUID lock: 000000002abc4399/0xa40083ab9bf5353a lrc: 4/0,0 mode: PR/PR res: [0x2c0000400:0x430:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x60000400010020 nid: 192.168.204.3@tcp remote: 0x771f68aa1eb27b2d expref: 9 pid: 20759 timeout: 1016 lvb_type: 1 00000100:00100000:1.0:1776178209.653779:0:6243:0:(service.c:2152:ptlrpc_server_handle_req_in()) got req x1862456979102720 00010000:00010000:1.0:1776178209.653789:0:6243:0:(ldlm_lockd.c:2638:ldlm_cancel_hpreq_check()) ### hpreq cancel/convert lock ns: filter-lustre-OST0000_UUID lock: 00000000a1e024a5/0xa40083ab9bf5352c lrc: 5/0,0 mode: PR/PR res: [0x280000400:0x470:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x60000400010020 nid: 192.168.204.3@tcp remote: 0x771f68aa1eb27b1f expref: 8 pid: 20759 timeout: 1016 lvb_type: 1 00000100:00100000:1.0:1776178209.653803:0:6243:0:(service.c:2152:ptlrpc_server_handle_req_in()) got req x1862456979102848 00010000:00010000:1.0:1776178209.653811:0:6243:0:(ldlm_lockd.c:2638:ldlm_cancel_hpreq_check()) ### hpreq cancel/convert lock ns: filter-lustre-OST0000_UUID lock: 00000000f51fd0d7/0xa40083ab9bf53517 lrc: 5/0,0 mode: PR/PR res: [0x280000400:0x46f:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x60000400010020 nid: 192.168.204.3@tcp remote: 0x771f68aa1eb27b0a expref: 9 pid: 20762 timeout: 1016 lvb_type: 1 00000100:00100000:1.0:1776178209.653829:0:6243:0:(service.c:2152:ptlrpc_server_handle_req_in()) got req x1862456979102656 00010000:00010000:1.0:1776178209.653837:0:6243:0:(ldlm_lockd.c:2638:ldlm_cancel_hpreq_check()) ### hpreq cancel/convert lock ns: filter-lustre-OST0000_UUID lock: 000000006b206cc9/0xa40083ab9bf5351e lrc: 5/0,0 mode: PR/PR res: [0x280000400:0x46e:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x60000400010020 nid: 192.168.204.3@tcp remote: 0x771f68aa1eb27b11 expref: 10 pid: 20762 timeout: 1016 lvb_type: 1 00000100:00100000:1.0:1776178209.653847:0:6243:0:(service.c:2152:ptlrpc_server_handle_req_in()) got req x1862456979102784 00010000:00010000:1.0:1776178209.653855:0:6243:0:(ldlm_lockd.c:2638:ldlm_cancel_hpreq_check()) ### hpreq cancel/convert lock ns: filter-lustre-OST0001_UUID lock: 000000003e1697a2/0xa40083ab9bf53533 lrc: 5/0,0 mode: PR/PR res: [0x2c0000400:0x431:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x60000400010020 nid: 192.168.204.3@tcp remote: 0x771f68aa1eb27b26 expref: 10 pid: 20759 timeout: 1016 lvb_type: 1 00000100:00100000:1.0:1776178209.653866:0:6243:0:(service.c:2152:ptlrpc_server_handle_req_in()) got req x1862456979102912 00010000:00010000:1.0:1776178209.653877:0:6243:0:(ldlm_lockd.c:2638:ldlm_cancel_hpreq_check()) ### hpreq cancel/convert lock ns: filter-lustre-OST0001_UUID lock: 000000001d465f75/0xa40083ab9bf534c3 lrc: 5/0,0 mode: PR/PR res: [0x2c0000400:0x42e:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x60000400010020 nid: 192.168.204.3@tcp remote: 0x771f68aa1eb27ae7 expref: 11 pid: 8228 timeout: 1016 lvb_type: 1 00000100:00100000:1.0:1776178209.653888:0:6243:0:(service.c:2152:ptlrpc_server_handle_req_in()) got req x1862456979102976 00010000:00010000:1.0:1776178209.653909:0:6243:0:(ldlm_lockd.c:2638:ldlm_cancel_hpreq_check()) ### hpreq cancel/convert lock ns: filter-lustre-OST0001_UUID lock: 00000000d999afdf/0xa40083ab9bf53525 lrc: 5/0,0 mode: PR/PR res: [0x2c0000400:0x42f:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x60000400010020 nid: 192.168.204.3@tcp remote: 0x771f68aa1eb27b18 expref: 12 pid: 20762 timeout: 1016 lvb_type: 1 00000100:00100000:1.0:1776178209.653925:0:6243:0:(nrs_fifo.c:179:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.204.3@tcp, seq: 2082 00000100:00100000:1.0:1776178209.653933:0:6243:0:(service.c:2324:ptlrpc_server_handle_request()) Handling RPC req@000000007058013a pname:cluuid+ref:pid:xid:nid:opc:job ldlm_cn00_002:b50963bd-8eba-4432-9a54-8888d00e3fa1+12:2273:x1862456979102592:12345-192.168.204.3@tcp:103:ldlm_bl_03.0 00010000:00010000:1.0:1776178209.653952:0:6243:0:(ldlm_lockd.c:1720:ldlm_request_cancel()) ### server-side cancel handler START: 1 locks, starting at 0 00010000:00010000:1.0:1776178209.653957:0:6243:0:(ldlm_lockd.c:1771:ldlm_request_cancel()) ### server cancels blocked lock after 0s ns: filter-lustre-OST0001_UUID lock: 000000002abc4399/0xa40083ab9bf5353a lrc: 4/0,0 mode: PR/PR res: [0x2c0000400:0x430:0x0].0x0 rrc: 4 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x60000400010020 nid: 192.168.204.3@tcp remote: 0x771f68aa1eb27b2d expref: 12 pid: 20759 timeout: 1016 lvb_type: 1 00010000:00010000:1.0:1776178209.653969:0:6243:0:(ldlm_lockd.c:562:ldlm_del_waiting_lock()) ### removed ns: filter-lustre-OST0001_UUID lock: 000000002abc4399/0xa40083ab9bf5353a lrc: 3/0,0 mode: PR/PR res: [0x2c0000400:0x430:0x0].0x0 rrc: 4 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x50000400010020 nid: 192.168.204.3@tcp remote: 0x771f68aa1eb27b2d expref: 12 pid: 20759 timeout: 1016 lvb_type: 1 00010000:00010000:1.0:1776178209.654003:0:6243:0:(ldlm_lock.c:216:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0001_UUID lock: 000000002abc4399/0xa40083ab9bf5353a lrc: 0/0,0 mode: --/PR res: [0x2c0000400:0x430:0x0].0x0 rrc: 4 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x44801400010020 nid: 192.168.204.3@tcp remote: 0x771f68aa1eb27b2d expref: 12 pid: 20759 timeout: 1016 lvb_type: 1 00010000:00010000:1.0:1776178209.654024:0:6243:0:(ldlm_extent.c:318:ldlm_check_contention()) contended locks = 0 00010000:00010000:1.0:1776178209.654026:0:6243:0:(ldlm_extent.c:318:ldlm_check_contention()) contended locks = 0 00010000:00010000:1.0:1776178209.654030:0:6243:0:(ldlm_lock.c:704:ldlm_add_cp_work_item()) ### lock granted; sending completion AST. ns: filter-lustre-OST0001_UUID lock: 00000000daa24072/0xa40083ab9bf537a9 lrc: 3/0,1 mode: PW/PW res: [0x2c0000400:0x430:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x40090080000000 nid: local remote: 0x0 expref: -99 pid: 20762 timeout: 0 lvb_type: 0 00010000:00010000:1.0:1776178209.654040:0:6243:0:(ldlm_resource.c:1650:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0001_UUID lock: 00000000daa24072/0xa40083ab9bf537a9 lrc: 4/0,1 mode: PW/PW res: [0x2c0000400:0x430:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x40090080000000 nid: local remote: 0x0 expref: -99 pid: 20762 timeout: 0 lvb_type: 0 00010000:00010000:1.0:1776178209.654109:0:6243:0:(ldlm_lockd.c:1782:ldlm_request_cancel()) ### server-side cancel handler END 00000100:00100000:1.0:1776178209.654170:0:6243:0:(service.c:2372:ptlrpc_server_handle_request()) Handled RPC req@000000007058013a pname:cluuid+ref:pid:xid:nid:opc:job ldlm_cn00_002:b50963bd-8eba-4432-9a54-8888d00e3fa1+11:2273:x1862456979102592:12345-192.168.204.3@tcp:103:ldlm_bl_03.0 Request processed in 237us (733us total) trans 0 rc 0/0 00000100:00100000:1.0:1776178209.654178:0:6243:0:(nrs_fifo.c:241:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.204.3@tcp, seq: 2082 00000100:00100000:1.0:1776178209.654197:0:6243:0:(nrs_fifo.c:179:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.204.3@tcp, seq: 2083 00000100:00100000:1.0:1776178209.654201:0:6243:0:(service.c:2324:ptlrpc_server_handle_request()) Handling RPC req@00000000b8f6795f pname:cluuid+ref:pid:xid:nid:opc:job ldlm_cn00_002:b50963bd-8eba-4432-9a54-8888d00e3fa1+10:2273:x1862456979102720:12345-192.168.204.3@tcp:103:ldlm_bl_03.0 00010000:00010000:1.0:1776178209.654211:0:6243:0:(ldlm_lockd.c:1720:ldlm_request_cancel()) ### server-side cancel handler START: 1 locks, starting at 0 00010000:00010000:1.0:1776178209.654216:0:6243:0:(ldlm_lockd.c:1771:ldlm_request_cancel()) ### server cancels blocked lock after 0s ns: filter-lustre-OST0000_UUID lock: 00000000a1e024a5/0xa40083ab9bf5352c lrc: 5/0,0 mode: PR/PR res: [0x280000400:0x470:0x0].0x0 rrc: 4 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x60000400010020 nid: 192.168.204.3@tcp remote: 0x771f68aa1eb27b1f expref: 10 pid: 20759 timeout: 1016 lvb_type: 1 00010000:00010000:1.0:1776178209.654224:0:6243:0:(ldlm_lockd.c:562:ldlm_del_waiting_lock()) ### removed ns: filter-lustre-OST0000_UUID lock: 00000000a1e024a5/0xa40083ab9bf5352c lrc: 4/0,0 mode: PR/PR res: [0x280000400:0x470:0x0].0x0 rrc: 4 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x50000400010020 nid: 192.168.204.3@tcp remote: 0x771f68aa1eb27b1f expref: 10 pid: 20759 timeout: 1016 lvb_type: 1 00010000:00010000:1.0:1776178209.654245:0:6243:0:(ldlm_extent.c:318:ldlm_check_contention()) contended locks = 0 00010000:00010000:1.0:1776178209.654248:0:6243:0:(ldlm_extent.c:318:ldlm_check_contention()) contended locks = 0 00010000:00010000:1.0:1776178209.654250:0:6243:0:(ldlm_lock.c:704:ldlm_add_cp_work_item()) ### lock granted; sending completion AST. ns: filter-lustre-OST0000_UUID lock: 000000006b0afb52/0xa40083ab9bf537be lrc: 3/0,1 mode: PW/PW res: [0x280000400:0x470:0x0].0x0 rrc: 4 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x50090080000000 nid: local remote: 0x0 expref: -99 pid: 8227 timeout: 0 lvb_type: 0 00010000:00010000:1.0:1776178209.654255:0:6243:0:(ldlm_resource.c:1650:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 000000006b0afb52/0xa40083ab9bf537be lrc: 4/0,1 mode: PW/PW res: [0x280000400:0x470:0x0].0x0 rrc: 4 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x50090080000000 nid: local remote: 0x0 expref: -99 pid: 8227 timeout: 0 lvb_type: 0 00010000:00010000:1.0:1776178209.654266:0:6243:0:(ldlm_lockd.c:1782:ldlm_request_cancel()) ### server-side cancel handler END 00000100:00100000:1.0:1776178209.654281:0:6243:0:(service.c:2372:ptlrpc_server_handle_request()) Handled RPC req@00000000b8f6795f pname:cluuid+ref:pid:xid:nid:opc:job ldlm_cn00_002:b50963bd-8eba-4432-9a54-8888d00e3fa1+10:2273:x1862456979102720:12345-192.168.204.3@tcp:103:ldlm_bl_03.0 Request processed in 79us (805us total) trans 0 rc 0/0 00000100:00100000:1.0:1776178209.654287:0:6243:0:(nrs_fifo.c:241:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.204.3@tcp, seq: 2083 00000100:00100000:1.0:1776178209.654298:0:6243:0:(nrs_fifo.c:179:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.204.3@tcp, seq: 2084 00000100:00100000:1.0:1776178209.654301:0:6243:0:(service.c:2324:ptlrpc_server_handle_request()) Handling RPC req@00000000d8af564d pname:cluuid+ref:pid:xid:nid:opc:job ldlm_cn00_002:b50963bd-8eba-4432-9a54-8888d00e3fa1+9:2273:x1862456979102848:12345-192.168.204.3@tcp:103:ldlm_bl_03.0 00010000:00010000:1.0:1776178209.654308:0:6243:0:(ldlm_lockd.c:1720:ldlm_request_cancel()) ### server-side cancel handler START: 1 locks, starting at 0 00010000:00010000:1.0:1776178209.654312:0:6243:0:(ldlm_lockd.c:1771:ldlm_request_cancel()) ### server cancels blocked lock after 0s ns: filter-lustre-OST0000_UUID lock: 00000000f51fd0d7/0xa40083ab9bf53517 lrc: 5/0,0 mode: PR/PR res: [0x280000400:0x46f:0x0].0x0 rrc: 4 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x60000400010020 nid: 192.168.204.3@tcp remote: 0x771f68aa1eb27b0a expref: 9 pid: 20762 timeout: 1016 lvb_type: 1 00010000:00010000:1.0:1776178209.654319:0:6243:0:(ldlm_lockd.c:562:ldlm_del_waiting_lock()) ### removed ns: filter-lustre-OST0000_UUID lock: 00000000f51fd0d7/0xa40083ab9bf53517 lrc: 4/0,0 mode: PR/PR res: [0x280000400:0x46f:0x0].0x0 rrc: 4 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x50000400010020 nid: 192.168.204.3@tcp remote: 0x771f68aa1eb27b0a expref: 9 pid: 20762 timeout: 1016 lvb_type: 1 00010000:00010000:1.0:1776178209.654333:0:6243:0:(ldlm_extent.c:318:ldlm_check_contention()) contended locks = 0 00010000:00010000:1.0:1776178209.654334:0:6243:0:(ldlm_extent.c:318:ldlm_check_contention()) contended locks = 0 00010000:00010000:1.0:1776178209.654336:0:6243:0:(ldlm_lock.c:704:ldlm_add_cp_work_item()) ### lock granted; sending completion AST. ns: filter-lustre-OST0000_UUID lock: 0000000097975a3b/0xa40083ab9bf537cc lrc: 3/0,1 mode: PW/PW res: [0x280000400:0x46f:0x0].0x0 rrc: 4 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x50090080000000 nid: local remote: 0x0 expref: -99 pid: 14440 timeout: 0 lvb_type: 0 00010000:00010000:1.0:1776178209.654342:0:6243:0:(ldlm_resource.c:1650:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 0000000097975a3b/0xa40083ab9bf537cc lrc: 4/0,1 mode: PW/PW res: [0x280000400:0x46f:0x0].0x0 rrc: 4 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x50090080000000 nid: local remote: 0x0 expref: -99 pid: 14440 timeout: 0 lvb_type: 0 00010000:00010000:1.0:1776178209.654358:0:6243:0:(ldlm_lockd.c:1782:ldlm_request_cancel()) ### server-side cancel handler END 00000100:00100000:1.0:1776178209.654372:0:6243:0:(service.c:2372:ptlrpc_server_handle_request()) Handled RPC req@00000000d8af564d pname:cluuid+ref:pid:xid:nid:opc:job ldlm_cn00_002:b50963bd-8eba-4432-9a54-8888d00e3fa1+9:2273:x1862456979102848:12345-192.168.204.3@tcp:103:ldlm_bl_03.0 Request processed in 70us (877us total) trans 0 rc 0/0 00000100:00100000:1.0:1776178209.654378:0:6243:0:(nrs_fifo.c:241:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.204.3@tcp, seq: 2084 00000100:00100000:1.0:1776178209.654388:0:6243:0:(nrs_fifo.c:179:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.204.3@tcp, seq: 2085 00000100:00100000:1.0:1776178209.654391:0:6243:0:(service.c:2324:ptlrpc_server_handle_request()) Handling RPC req@00000000e8403750 pname:cluuid+ref:pid:xid:nid:opc:job ldlm_cn00_002:b50963bd-8eba-4432-9a54-8888d00e3fa1+8:2273:x1862456979102656:12345-192.168.204.3@tcp:103:ldlm_bl_03.0 00010000:00010000:1.0:1776178209.654397:0:6243:0:(ldlm_lockd.c:1720:ldlm_request_cancel()) ### server-side cancel handler START: 1 locks, starting at 0 00010000:00010000:1.0:1776178209.654402:0:6243:0:(ldlm_lockd.c:1771:ldlm_request_cancel()) ### server cancels blocked lock after 0s ns: filter-lustre-OST0000_UUID lock: 000000006b206cc9/0xa40083ab9bf5351e lrc: 5/0,0 mode: PR/PR res: [0x280000400:0x46e:0x0].0x0 rrc: 4 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x60000400010020 nid: 192.168.204.3@tcp remote: 0x771f68aa1eb27b11 expref: 8 pid: 20762 timeout: 1016 lvb_type: 1 00010000:00010000:1.0:1776178209.654408:0:6243:0:(ldlm_lockd.c:562:ldlm_del_waiting_lock()) ### removed ns: filter-lustre-OST0000_UUID lock: 000000006b206cc9/0xa40083ab9bf5351e lrc: 4/0,0 mode: PR/PR res: [0x280000400:0x46e:0x0].0x0 rrc: 4 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x50000400010020 nid: 192.168.204.3@tcp remote: 0x771f68aa1eb27b11 expref: 8 pid: 20762 timeout: 1016 lvb_type: 1 00010000:00010000:1.0:1776178209.654421:0:6243:0:(ldlm_extent.c:318:ldlm_check_contention()) contended locks = 0 00010000:00010000:1.0:1776178209.654422:0:6243:0:(ldlm_extent.c:318:ldlm_check_contention()) contended locks = 0 00010000:00010000:1.0:1776178209.654425:0:6243:0:(ldlm_lock.c:704:ldlm_add_cp_work_item()) ### lock granted; sending completion AST. ns: filter-lustre-OST0000_UUID lock: 00000000014a8a18/0xa40083ab9bf537b0 lrc: 3/0,1 mode: PW/PW res: [0x280000400:0x46e:0x0].0x0 rrc: 4 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x50090080000000 nid: local remote: 0x0 expref: -99 pid: 20758 timeout: 0 lvb_type: 0 00010000:00010000:1.0:1776178209.654431:0:6243:0:(ldlm_resource.c:1650:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 00000000014a8a18/0xa40083ab9bf537b0 lrc: 4/0,1 mode: PW/PW res: [0x280000400:0x46e:0x0].0x0 rrc: 4 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x50090080000000 nid: local remote: 0x0 expref: -99 pid: 20758 timeout: 0 lvb_type: 0 00010000:00010000:1.0:1776178209.654440:0:6243:0:(ldlm_lockd.c:1782:ldlm_request_cancel()) ### server-side cancel handler END 00000100:00100000:1.0:1776178209.654455:0:6243:0:(service.c:2372:ptlrpc_server_handle_request()) Handled RPC req@00000000e8403750 pname:cluuid+ref:pid:xid:nid:opc:job ldlm_cn00_002:b50963bd-8eba-4432-9a54-8888d00e3fa1+8:2273:x1862456979102656:12345-192.168.204.3@tcp:103:ldlm_bl_03.0 Request processed in 63us (918us total) trans 0 rc 0/0 00000100:00100000:1.0:1776178209.654461:0:6243:0:(nrs_fifo.c:241:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.204.3@tcp, seq: 2085 00000100:00100000:1.0:1776178209.654508:0:6243:0:(nrs_fifo.c:179:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.204.3@tcp, seq: 2086 00000100:00100000:1.0:1776178209.654513:0:6243:0:(service.c:2324:ptlrpc_server_handle_request()) Handling RPC req@000000006522caac pname:cluuid+ref:pid:xid:nid:opc:job ldlm_cn00_002:b50963bd-8eba-4432-9a54-8888d00e3fa1+10:2273:x1862456979102784:12345-192.168.204.3@tcp:103:ldlm_bl_03.0 00010000:00010000:1.0:1776178209.654520:0:6243:0:(ldlm_lockd.c:1720:ldlm_request_cancel()) ### server-side cancel handler START: 1 locks, starting at 0 00010000:00010000:1.0:1776178209.654524:0:6243:0:(ldlm_lockd.c:1771:ldlm_request_cancel()) ### server cancels blocked lock after 0s ns: filter-lustre-OST0001_UUID lock: 000000003e1697a2/0xa40083ab9bf53533 lrc: 5/0,0 mode: PR/PR res: [0x2c0000400:0x431:0x0].0x0 rrc: 4 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x60000400010020 nid: 192.168.204.3@tcp remote: 0x771f68aa1eb27b26 expref: 10 pid: 20759 timeout: 1016 lvb_type: 1 00010000:00010000:1.0:1776178209.654532:0:6243:0:(ldlm_lockd.c:562:ldlm_del_waiting_lock()) ### removed ns: filter-lustre-OST0001_UUID lock: 000000003e1697a2/0xa40083ab9bf53533 lrc: 4/0,0 mode: PR/PR res: [0x2c0000400:0x431:0x0].0x0 rrc: 4 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x50000400010020 nid: 192.168.204.3@tcp remote: 0x771f68aa1eb27b26 expref: 10 pid: 20759 timeout: 1016 lvb_type: 1 00010000:00010000:1.0:1776178209.654544:0:6243:0:(ldlm_extent.c:318:ldlm_check_contention()) contended locks = 0 00010000:00010000:1.0:1776178209.654545:0:6243:0:(ldlm_extent.c:318:ldlm_check_contention()) contended locks = 0 00010000:00010000:1.0:1776178209.654547:0:6243:0:(ldlm_lock.c:704:ldlm_add_cp_work_item()) ### lock granted; sending completion AST. ns: filter-lustre-OST0001_UUID lock: 00000000a14e680b/0xa40083ab9bf537c5 lrc: 3/0,1 mode: PW/PW res: [0x2c0000400:0x431:0x0].0x0 rrc: 4 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x50090080000000 nid: local remote: 0x0 expref: -99 pid: 20755 timeout: 0 lvb_type: 0 00010000:00010000:1.0:1776178209.654553:0:6243:0:(ldlm_resource.c:1650:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0001_UUID lock: 00000000a14e680b/0xa40083ab9bf537c5 lrc: 4/0,1 mode: PW/PW res: [0x2c0000400:0x431:0x0].0x0 rrc: 4 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x50090080000000 nid: local remote: 0x0 expref: -99 pid: 20755 timeout: 0 lvb_type: 0 00010000:00010000:0.0:1776178209.654656:0:20758:0:(ldlm_lock.c:216:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 000000006b206cc9/0xa40083ab9bf5351e lrc: 0/0,0 mode: --/PR res: [0x280000400:0x46e:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x44801400010020 nid: 192.168.204.3@tcp remote: 0x771f68aa1eb27b11 expref: 7 pid: 20762 timeout: 1016 lvb_type: 1 00000100:00100000:0.0:1776178209.654670:0:20758:0:(client.c:2215:ptlrpc_check_set()) Completed RPC req@0000000068680f90 pname:cluuid:pid:xid:nid:opc:job ll_ost00_009:lustre-OST0000_UUID:20758:1862456998048192:192.168.204.3@tcp:104: 00010000:00010000:0.0:1776178209.654694:0:20758:0:(ldlm_request.c:283:ldlm_completion_ast()) ### client-side enqueue returned a blocked locksleeping ns: filter-lustre-OST0000_UUID lock: 00000000014a8a18/0xa40083ab9bf537b0 lrc: 3/0,1 mode: PW/PW res: [0x280000400:0x46e:0x0].0x0 rrc: 2 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x40010080000000 nid: local remote: 0x0 expref: -99 pid: 20758 timeout: 0 lvb_type: 0 00010000:00010000:0.0:1776178209.654705:0:20758:0:(ldlm_request.c:199:ldlm_completion_tail()) ### client-side enqueue: granted ns: filter-lustre-OST0000_UUID lock: 00000000014a8a18/0xa40083ab9bf537b0 lrc: 3/0,1 mode: PW/PW res: [0x280000400:0x46e:0x0].0x0 rrc: 2 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x40010080000000 nid: local remote: 0x0 expref: -99 pid: 20758 timeout: 0 lvb_type: 0 00010000:00010000:0.0:1776178209.654711:0:20758:0:(ldlm_request.c:528:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 00000000014a8a18/0xa40083ab9bf537b0 lrc: 3/0,1 mode: PW/PW res: [0x280000400:0x46e:0x0].0x0 rrc: 2 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x40010080000000 nid: local remote: 0x0 expref: -99 pid: 20758 timeout: 0 lvb_type: 0 00010000:00010000:0.0:1776178209.654720:0:20758:0:(ldlm_lock.c:820:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0000_UUID lock: 00000000014a8a18/0xa40083ab9bf537b0 lrc: 3/0,1 mode: PW/PW res: [0x280000400:0x46e:0x0].0x0 rrc: 2 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x50010080000000 nid: local remote: 0x0 expref: -99 pid: 20758 timeout: 0 lvb_type: 0 00010000:00010000:0.0:1776178209.654726:0:20758:0:(ldlm_lock.c:879:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 00000000014a8a18/0xa40083ab9bf537b0 lrc: 2/0,0 mode: PW/PW res: [0x280000400:0x46e:0x0].0x0 rrc: 2 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x50010480000000 nid: local remote: 0x0 expref: -99 pid: 20758 timeout: 0 lvb_type: 0 00010000:00010000:0.0:1776178209.654746:0:20758:0:(ldlm_lockd.c:2159:__ldlm_bl_to_thread()) added 1/1 locks to regular blp list, 1 blwis in pool 00000100:00100000:0.0:1776178209.655037:0:20758:0:(service.c:2372:ptlrpc_server_handle_request()) Handled RPC req@000000006165f052 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_009:lustre-MDT0001-mdtlov_UUID+8:3354:x1862456998048064:12345-0@lo:6:osp-syn-0-1.0 Request processed in 14552us (15387us total) trans 4294969359 rc 0/0 00000100:00100000:0.0:1776178209.655046:0:20758:0:(nrs_fifo.c:241:nrs_fifo_req_stop()) NRS stop fifo request from 12345-0@lo, seq: 5280 00010000:00010000:0.0:1776178209.655091:0:9224:0:(ldlm_lockd.c:2789:ldlm_bl_get_work()) Got 1 locks of 0 total in blp. (0 blwis in pool) 00010000:00010000:0.0:1776178209.655097:0:9224:0:(ldlm_lockd.c:1879:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 00000000014a8a18/0xa40083ab9bf537b0 lrc: 2/0,0 mode: PW/PW res: [0x280000400:0x46e:0x0].0x0 rrc: 2 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x40010480000000 nid: local remote: 0x0 expref: -99 pid: 20758 timeout: 0 lvb_type: 0 00010000:00010000:0.0:1776178209.655105:0:9224:0:(ldlm_lockd.c:1893:ldlm_handle_bl_callback()) Lock 00000000014a8a18 already unused, calling callback (0000000063c97258) 00010000:00010000:0.0:1776178209.655108:0:9224:0:(ldlm_request.c:364:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 00000000014a8a18/0xa40083ab9bf537b0 lrc: 2/0,0 mode: PW/PW res: [0x280000400:0x46e:0x0].0x0 rrc: 2 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x40010480000000 nid: local remote: 0x0 expref: -99 pid: 20758 timeout: 0 lvb_type: 0 00010000:00010000:0.0:1776178209.655117:0:9224:0:(ldlm_request.c:1257:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 00000000014a8a18/0xa40083ab9bf537b0 lrc: 3/0,0 mode: PW/PW res: [0x280000400:0x46e:0x0].0x0 rrc: 2 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x40018480000000 nid: local remote: 0x0 expref: -99 pid: 20758 timeout: 0 lvb_type: 0 00010000:00010000:0.0:1776178209.655134:0:9224:0:(ldlm_lockd.c:1903:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 00000000014a8a18/0xa40083ab9bf537b0 lrc: 1/0,0 mode: --/PW res: [0x280000400:0x46e:0x0].0x0 rrc: 2 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x44819480000000 nid: local remote: 0x0 expref: -99 pid: 20758 timeout: 0 lvb_type: 0 00010000:00010000:0.0:1776178209.655139:0:9224:0:(ldlm_lock.c:216:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 00000000014a8a18/0xa40083ab9bf537b0 lrc: 0/0,0 mode: --/PW res: [0x280000400:0x46e:0x0].0x0 rrc: 2 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x44819480000000 nid: local remote: 0x0 expref: -99 pid: 20758 timeout: 0 lvb_type: 0 00010000:00010000:0.0:1776178209.655156:0:9224:0:(ldlm_lockd.c:2792:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00010000:00010000:0.0:1776178209.655157:0:9224:0:(ldlm_lockd.c:2792:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00010000:00010000:0.0:1776178209.655158:0:9224:0:(ldlm_lockd.c:2792:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000004:00080000:0.0:1776178209.655206:0:3354:0:(osp_sync.c:562:osp_sync_interpret()) reply req 00000000f433e404/2, rc 0, transno 2063 00000100:00100000:0.0:1776178209.655213:0:3354:0:(client.c:2215:ptlrpc_check_set()) Completed RPC req@00000000f433e404 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_01:lustre-MDT0001-mdtlov_UUID:3354:1862456998048064:0@lo:6:osp-syn-0-1.0 00000100:00100000:0.0:1776178209.655241:0:20759:0:(client.c:2215:ptlrpc_check_set()) Completed RPC req@0000000023210d1c pname:cluuid:pid:xid:nid:opc:job ll_ost00_010:lustre-OST0001_UUID:20759:1862456998048128:192.168.204.3@tcp:104: 00010000:00010000:0.0:1776178209.655254:0:20759:0:(ldlm_request.c:283:ldlm_completion_ast()) ### client-side enqueue returned a blocked locksleeping ns: filter-lustre-OST0001_UUID lock: 0000000030bf6b1e/0xa40083ab9bf537a2 lrc: 3/0,1 mode: --/PW res: [0x2c0000400:0x42e:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x40010080000000 nid: local remote: 0x0 expref: -99 pid: 20759 timeout: 0 lvb_type: 0 00010000:00010000:1.0:1776178209.657459:0:6243:0:(ldlm_lockd.c:1782:ldlm_request_cancel()) ### server-side cancel handler END 00000100:00100000:1.0:1776178209.657552:0:6243:0:(service.c:2372:ptlrpc_server_handle_request()) Handled RPC req@000000006522caac pname:cluuid+ref:pid:xid:nid:opc:job ldlm_cn00_002:b50963bd-8eba-4432-9a54-8888d00e3fa1+10:2273:x1862456979102784:12345-192.168.204.3@tcp:103:ldlm_bl_03.0 Request processed in 3030us (3988us total) trans 0 rc 0/0 00000100:00100000:1.0:1776178209.657566:0:6243:0:(nrs_fifo.c:241:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.204.3@tcp, seq: 2086 00000100:00100000:1.0:1776178209.657622:0:6243:0:(nrs_fifo.c:179:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.204.3@tcp, seq: 2087 00000100:00100000:1.0:1776178209.657632:0:6243:0:(service.c:2324:ptlrpc_server_handle_request()) Handling RPC req@000000008516f059 pname:cluuid+ref:pid:xid:nid:opc:job ldlm_cn00_002:b50963bd-8eba-4432-9a54-8888d00e3fa1+9:2273:x1862456979102912:12345-192.168.204.3@tcp:103:ldlm_bl_03.0 00010000:00010000:1.0:1776178209.657651:0:6243:0:(ldlm_lockd.c:1720:ldlm_request_cancel()) ### server-side cancel handler START: 1 locks, starting at 0 00010000:00010000:1.0:1776178209.657806:0:6243:0:(ldlm_lockd.c:1771:ldlm_request_cancel()) ### server cancels blocked lock after 0s ns: filter-lustre-OST0001_UUID lock: 000000001d465f75/0xa40083ab9bf534c3 lrc: 4/0,0 mode: PR/PR res: [0x2c0000400:0x42e:0x0].0x0 rrc: 4 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x60000400010020 nid: 192.168.204.3@tcp remote: 0x771f68aa1eb27ae7 expref: 9 pid: 8228 timeout: 1016 lvb_type: 1 00010000:00010000:1.0:1776178209.657823:0:6243:0:(ldlm_lockd.c:562:ldlm_del_waiting_lock()) ### removed ns: filter-lustre-OST0001_UUID lock: 000000001d465f75/0xa40083ab9bf534c3 lrc: 3/0,0 mode: PR/PR res: [0x2c0000400:0x42e:0x0].0x0 rrc: 4 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x50000400010020 nid: 192.168.204.3@tcp remote: 0x771f68aa1eb27ae7 expref: 9 pid: 8228 timeout: 1016 lvb_type: 1 00010000:00010000:1.0:1776178209.657850:0:6243:0:(ldlm_lock.c:216:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0001_UUID lock: 000000001d465f75/0xa40083ab9bf534c3 lrc: 0/0,0 mode: --/PR res: [0x2c0000400:0x42e:0x0].0x0 rrc: 4 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x44801400010020 nid: 192.168.204.3@tcp remote: 0x771f68aa1eb27ae7 expref: 9 pid: 8228 timeout: 1016 lvb_type: 1 00010000:00010000:1.0:1776178209.658010:0:6243:0:(ldlm_extent.c:318:ldlm_check_contention()) contended locks = 0 00010000:00010000:1.0:1776178209.658013:0:6243:0:(ldlm_extent.c:318:ldlm_check_contention()) contended locks = 0 00010000:00010000:1.0:1776178209.658017:0:6243:0:(ldlm_lock.c:704:ldlm_add_cp_work_item()) ### lock granted; sending completion AST. ns: filter-lustre-OST0001_UUID lock: 0000000030bf6b1e/0xa40083ab9bf537a2 lrc: 3/0,1 mode: PW/PW res: [0x2c0000400:0x42e:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x40090080000000 nid: local remote: 0x0 expref: -99 pid: 20759 timeout: 0 lvb_type: 0 00010000:00010000:1.0:1776178209.658030:0:6243:0:(ldlm_resource.c:1650:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0001_UUID lock: 0000000030bf6b1e/0xa40083ab9bf537a2 lrc: 4/0,1 mode: PW/PW res: [0x2c0000400:0x42e:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x40090080000000 nid: local remote: 0x0 expref: -99 pid: 20759 timeout: 0 lvb_type: 0 00010000:00010000:1.0:1776178209.658086:0:6243:0:(ldlm_lockd.c:1782:ldlm_request_cancel()) ### server-side cancel handler END 00000100:00100000:1.0:1776178209.658114:0:6243:0:(service.c:2372:ptlrpc_server_handle_request()) Handled RPC req@000000008516f059 pname:cluuid+ref:pid:xid:nid:opc:job ldlm_cn00_002:b50963bd-8eba-4432-9a54-8888d00e3fa1+8:2273:x1862456979102912:12345-192.168.204.3@tcp:103:ldlm_bl_03.0 Request processed in 484us (4539us total) trans 0 rc 0/0 00000100:00100000:1.0:1776178209.658120:0:6243:0:(nrs_fifo.c:241:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.204.3@tcp, seq: 2087 00000100:00100000:1.0:1776178209.658136:0:6243:0:(nrs_fifo.c:179:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.204.3@tcp, seq: 2088 00000100:00100000:1.0:1776178209.658141:0:6243:0:(service.c:2324:ptlrpc_server_handle_request()) Handling RPC req@00000000ba7f03ed pname:cluuid+ref:pid:xid:nid:opc:job ldlm_cn00_002:b50963bd-8eba-4432-9a54-8888d00e3fa1+7:2273:x1862456979102976:12345-192.168.204.3@tcp:103:ldlm_bl_03.0 00010000:00010000:1.0:1776178209.658150:0:6243:0:(ldlm_lockd.c:1720:ldlm_request_cancel()) ### server-side cancel handler START: 1 locks, starting at 0 00010000:00010000:1.0:1776178209.658155:0:6243:0:(ldlm_lockd.c:1771:ldlm_request_cancel()) ### server cancels blocked lock after 0s ns: filter-lustre-OST0001_UUID lock: 00000000d999afdf/0xa40083ab9bf53525 lrc: 5/0,0 mode: PR/PR res: [0x2c0000400:0x42f:0x0].0x0 rrc: 4 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x60000400010020 nid: 192.168.204.3@tcp remote: 0x771f68aa1eb27b18 expref: 7 pid: 20762 timeout: 1016 lvb_type: 1 00010000:00010000:1.0:1776178209.658163:0:6243:0:(ldlm_lockd.c:562:ldlm_del_waiting_lock()) ### removed ns: filter-lustre-OST0001_UUID lock: 00000000d999afdf/0xa40083ab9bf53525 lrc: 4/0,0 mode: PR/PR res: [0x2c0000400:0x42f:0x0].0x0 rrc: 4 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x50000400010020 nid: 192.168.204.3@tcp remote: 0x771f68aa1eb27b18 expref: 7 pid: 20762 timeout: 1016 lvb_type: 1 00010000:00010000:1.0:1776178209.658177:0:6243:0:(ldlm_extent.c:318:ldlm_check_contention()) contended locks = 0 00010000:00010000:1.0:1776178209.658179:0:6243:0:(ldlm_extent.c:318:ldlm_check_contention()) contended locks = 0 00010000:00010000:1.0:1776178209.658181:0:6243:0:(ldlm_lock.c:704:ldlm_add_cp_work_item()) ### lock granted; sending completion AST. ns: filter-lustre-OST0001_UUID lock: 000000007ae7b5f6/0xa40083ab9bf537b7 lrc: 3/0,1 mode: PW/PW res: [0x2c0000400:0x42f:0x0].0x0 rrc: 4 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x50090080000000 nid: local remote: 0x0 expref: -99 pid: 20765 timeout: 0 lvb_type: 0 00010000:00010000:1.0:1776178209.658187:0:6243:0:(ldlm_resource.c:1650:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0001_UUID lock: 000000007ae7b5f6/0xa40083ab9bf537b7 lrc: 4/0,1 mode: PW/PW res: [0x2c0000400:0x42f:0x0].0x0 rrc: 4 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x50090080000000 nid: local remote: 0x0 expref: -99 pid: 20765 timeout: 0 lvb_type: 0 00010000:00010000:1.0:1776178209.658198:0:6243:0:(ldlm_lockd.c:1782:ldlm_request_cancel()) ### server-side cancel handler END 00000100:00100000:1.0:1776178209.658214:0:6243:0:(service.c:2372:ptlrpc_server_handle_request()) Handled RPC req@00000000ba7f03ed pname:cluuid+ref:pid:xid:nid:opc:job ldlm_cn00_002:b50963bd-8eba-4432-9a54-8888d00e3fa1+7:2273:x1862456979102976:12345-192.168.204.3@tcp:103:ldlm_bl_03.0 Request processed in 73us (4560us total) trans 0 rc 0/0 00000100:00100000:1.0:1776178209.658220:0:6243:0:(nrs_fifo.c:241:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.204.3@tcp, seq: 2088 00010000:00010000:1.0:1776178209.658293:0:20765:0:(ldlm_lock.c:216:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0001_UUID lock: 00000000d999afdf/0xa40083ab9bf53525 lrc: 0/0,0 mode: --/PR res: [0x2c0000400:0x42f:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x44801400010020 nid: 192.168.204.3@tcp remote: 0x771f68aa1eb27b18 expref: 6 pid: 20762 timeout: 1016 lvb_type: 1 00000100:00100000:1.0:1776178209.658303:0:20765:0:(client.c:2215:ptlrpc_check_set()) Completed RPC req@0000000065dbeee0 pname:cluuid:pid:xid:nid:opc:job ll_ost00_016:lustre-OST0001_UUID:20765:1862456998048320:192.168.204.3@tcp:104: 00010000:00010000:1.0:1776178209.658325:0:20765:0:(ldlm_request.c:283:ldlm_completion_ast()) ### client-side enqueue returned a blocked locksleeping ns: filter-lustre-OST0001_UUID lock: 000000007ae7b5f6/0xa40083ab9bf537b7 lrc: 3/0,1 mode: PW/PW res: [0x2c0000400:0x42f:0x0].0x0 rrc: 2 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x40010080000000 nid: local remote: 0x0 expref: -99 pid: 20765 timeout: 0 lvb_type: 0 00010000:00010000:1.0:1776178209.658336:0:20765:0:(ldlm_request.c:199:ldlm_completion_tail()) ### client-side enqueue: granted ns: filter-lustre-OST0001_UUID lock: 000000007ae7b5f6/0xa40083ab9bf537b7 lrc: 3/0,1 mode: PW/PW res: [0x2c0000400:0x42f:0x0].0x0 rrc: 2 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x40010080000000 nid: local remote: 0x0 expref: -99 pid: 20765 timeout: 0 lvb_type: 0 00010000:00010000:1.0:1776178209.658343:0:20765:0:(ldlm_request.c:528:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0001_UUID lock: 000000007ae7b5f6/0xa40083ab9bf537b7 lrc: 3/0,1 mode: PW/PW res: [0x2c0000400:0x42f:0x0].0x0 rrc: 2 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x40010080000000 nid: local remote: 0x0 expref: -99 pid: 20765 timeout: 0 lvb_type: 0 00010000:00010000:1.0:1776178209.658352:0:20765:0:(ldlm_lock.c:820:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0001_UUID lock: 000000007ae7b5f6/0xa40083ab9bf537b7 lrc: 3/0,1 mode: PW/PW res: [0x2c0000400:0x42f:0x0].0x0 rrc: 2 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x50010080000000 nid: local remote: 0x0 expref: -99 pid: 20765 timeout: 0 lvb_type: 0 00010000:00010000:1.0:1776178209.658359:0:20765:0:(ldlm_lock.c:879:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0001_UUID lock: 000000007ae7b5f6/0xa40083ab9bf537b7 lrc: 2/0,0 mode: PW/PW res: [0x2c0000400:0x42f:0x0].0x0 rrc: 2 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x50010480000000 nid: local remote: 0x0 expref: -99 pid: 20765 timeout: 0 lvb_type: 0 00010000:00010000:1.0:1776178209.658378:0:20765:0:(ldlm_lockd.c:2159:__ldlm_bl_to_thread()) added 1/1 locks to regular blp list, 1 blwis in pool 00000100:00100000:1.0:1776178209.658704:0:20765:0:(service.c:2372:ptlrpc_server_handle_request()) Handled RPC req@00000000d9d2b484 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_016:lustre-MDT0001-mdtlov_UUID+9:3355:x1862456998047680:12345-0@lo:6:osp-syn-1-1.0 Request processed in 17915us (18995us total) trans 4294969332 rc 0/0 00000100:00100000:1.0:1776178209.658715:0:20765:0:(nrs_fifo.c:241:nrs_fifo_req_stop()) NRS stop fifo request from 12345-0@lo, seq: 5281 00010000:00010000:1.0:1776178209.658740:0:20762:0:(ldlm_request.c:199:ldlm_completion_tail()) ### client-side enqueue: granted ns: filter-lustre-OST0001_UUID lock: 00000000daa24072/0xa40083ab9bf537a9 lrc: 3/0,1 mode: PW/PW res: [0x2c0000400:0x430:0x0].0x0 rrc: 2 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x40010080000000 nid: local remote: 0x0 expref: -99 pid: 20762 timeout: 0 lvb_type: 0 00010000:00010000:1.0:1776178209.658748:0:20762:0:(ldlm_request.c:528:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0001_UUID lock: 00000000daa24072/0xa40083ab9bf537a9 lrc: 3/0,1 mode: PW/PW res: [0x2c0000400:0x430:0x0].0x0 rrc: 2 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x40010080000000 nid: local remote: 0x0 expref: -99 pid: 20762 timeout: 0 lvb_type: 0 00010000:00010000:1.0:1776178209.658755:0:20762:0:(ldlm_lock.c:820:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0001_UUID lock: 00000000daa24072/0xa40083ab9bf537a9 lrc: 3/0,1 mode: PW/PW res: [0x2c0000400:0x430:0x0].0x0 rrc: 2 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x50010080000000 nid: local remote: 0x0 expref: -99 pid: 20762 timeout: 0 lvb_type: 0 00010000:00010000:1.0:1776178209.658760:0:20762:0:(ldlm_lock.c:879:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0001_UUID lock: 00000000daa24072/0xa40083ab9bf537a9 lrc: 2/0,0 mode: PW/PW res: [0x2c0000400:0x430:0x0].0x0 rrc: 2 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x50010480000000 nid: local remote: 0x0 expref: -99 pid: 20762 timeout: 0 lvb_type: 0 00010000:00010000:1.0:1776178209.658774:0:20762:0:(ldlm_lockd.c:2159:__ldlm_bl_to_thread()) added 1/2 locks to regular blp list, 2 blwis in pool 00010000:00010000:0.0:1776178209.659030:0:20759:0:(ldlm_request.c:199:ldlm_completion_tail()) ### client-side enqueue: granted ns: filter-lustre-OST0001_UUID lock: 0000000030bf6b1e/0xa40083ab9bf537a2 lrc: 3/0,1 mode: PW/PW res: [0x2c0000400:0x42e:0x0].0x0 rrc: 2 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x40010080000000 nid: local remote: 0x0 expref: -99 pid: 20759 timeout: 0 lvb_type: 0 00010000:00010000:0.0:1776178209.659039:0:20759:0:(ldlm_request.c:528:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0001_UUID lock: 0000000030bf6b1e/0xa40083ab9bf537a2 lrc: 3/0,1 mode: PW/PW res: [0x2c0000400:0x42e:0x0].0x0 rrc: 2 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x40010080000000 nid: local remote: 0x0 expref: -99 pid: 20759 timeout: 0 lvb_type: 0 00010000:00010000:0.0:1776178209.659046:0:20759:0:(ldlm_lock.c:820:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0001_UUID lock: 0000000030bf6b1e/0xa40083ab9bf537a2 lrc: 3/0,1 mode: PW/PW res: [0x2c0000400:0x42e:0x0].0x0 rrc: 2 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x50010080000000 nid: local remote: 0x0 expref: -99 pid: 20759 timeout: 0 lvb_type: 0 00010000:00010000:0.0:1776178209.659051:0:20759:0:(ldlm_lock.c:879:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0001_UUID lock: 0000000030bf6b1e/0xa40083ab9bf537a2 lrc: 2/0,0 mode: PW/PW res: [0x2c0000400:0x42e:0x0].0x0 rrc: 2 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x50010480000000 nid: local remote: 0x0 expref: -99 pid: 20759 timeout: 0 lvb_type: 0 00010000:00010000:0.0:1776178209.659087:0:20759:0:(ldlm_lockd.c:2159:__ldlm_bl_to_thread()) added 1/3 locks to regular blp list, 3 blwis in pool 00000100:00100000:0.0:1776178209.659201:0:20759:0:(service.c:2372:ptlrpc_server_handle_request()) Handled RPC req@00000000dc1ea8e7 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_010:lustre-MDT0001-mdtlov_UUID+10:3353:x1862456998047616:12345-0@lo:6:osp-syn-1-1.0 Request processed in 21362us (21586us total) trans 4294969334 rc 0/0 00000100:00100000:0.0:1776178209.659209:0:20759:0:(nrs_fifo.c:241:nrs_fifo_req_stop()) NRS stop fifo request from 12345-0@lo, seq: 5278 00010000:00010000:3.0:1776178209.660395:0:14440:0:(ldlm_lock.c:216:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 00000000f51fd0d7/0xa40083ab9bf53517 lrc: 0/0,0 mode: --/PR res: [0x280000400:0x46f:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x44801400010020 nid: 192.168.204.3@tcp remote: 0x771f68aa1eb27b0a expref: 6 pid: 20762 timeout: 1016 lvb_type: 1 00000100:00100000:3.0:1776178209.660419:0:14440:0:(client.c:2215:ptlrpc_check_set()) Completed RPC req@00000000a8bae24c pname:cluuid:pid:xid:nid:opc:job ll_ost00_004:lustre-OST0000_UUID:14440:1862456998048512:192.168.204.3@tcp:104: 00010000:00010000:3.0:1776178209.660445:0:14440:0:(ldlm_request.c:283:ldlm_completion_ast()) ### client-side enqueue returned a blocked locksleeping ns: filter-lustre-OST0000_UUID lock: 0000000097975a3b/0xa40083ab9bf537cc lrc: 3/0,1 mode: PW/PW res: [0x280000400:0x46f:0x0].0x0 rrc: 2 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x40010080000000 nid: local remote: 0x0 expref: -99 pid: 14440 timeout: 0 lvb_type: 0 00010000:00010000:3.0:1776178209.660467:0:6244:0:(ldlm_lockd.c:2789:ldlm_bl_get_work()) Got 1 locks of 2 total in blp. (2 blwis in pool) 00010000:00010000:3.0:1776178209.660473:0:6244:0:(ldlm_lockd.c:1879:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0001_UUID lock: 000000007ae7b5f6/0xa40083ab9bf537b7 lrc: 2/0,0 mode: PW/PW res: [0x2c0000400:0x42f:0x0].0x0 rrc: 2 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x40010480000000 nid: local remote: 0x0 expref: -99 pid: 20765 timeout: 0 lvb_type: 0 00010000:00010000:3.0:1776178209.660480:0:6244:0:(ldlm_lockd.c:1893:ldlm_handle_bl_callback()) Lock 000000007ae7b5f6 already unused, calling callback (0000000063c97258) 00010000:00010000:3.0:1776178209.660484:0:6244:0:(ldlm_request.c:364:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0001_UUID lock: 000000007ae7b5f6/0xa40083ab9bf537b7 lrc: 2/0,0 mode: PW/PW res: [0x2c0000400:0x42f:0x0].0x0 rrc: 2 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x40010480000000 nid: local remote: 0x0 expref: -99 pid: 20765 timeout: 0 lvb_type: 0 00010000:00010000:3.0:1776178209.660494:0:6244:0:(ldlm_request.c:1257:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0001_UUID lock: 000000007ae7b5f6/0xa40083ab9bf537b7 lrc: 3/0,0 mode: PW/PW res: [0x2c0000400:0x42f:0x0].0x0 rrc: 2 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x40018480000000 nid: local remote: 0x0 expref: -99 pid: 20765 timeout: 0 lvb_type: 0 00010000:00010000:3.0:1776178209.660516:0:6244:0:(ldlm_lockd.c:1903:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0001_UUID lock: 000000007ae7b5f6/0xa40083ab9bf537b7 lrc: 1/0,0 mode: --/PW res: [0x2c0000400:0x42f:0x0].0x0 rrc: 2 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x44819480000000 nid: local remote: 0x0 expref: -99 pid: 20765 timeout: 0 lvb_type: 0 00010000:00010000:3.0:1776178209.660522:0:6244:0:(ldlm_lock.c:216:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0001_UUID lock: 000000007ae7b5f6/0xa40083ab9bf537b7 lrc: 0/0,0 mode: --/PW res: [0x2c0000400:0x42f:0x0].0x0 rrc: 2 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x44819480000000 nid: local remote: 0x0 expref: -99 pid: 20765 timeout: 0 lvb_type: 0 00010000:00010000:3.0:1776178209.660539:0:6244:0:(ldlm_lockd.c:2789:ldlm_bl_get_work()) Got 1 locks of 1 total in blp. (1 blwis in pool) 00010000:00010000:3.0:1776178209.660541:0:6244:0:(ldlm_lockd.c:1879:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0001_UUID lock: 00000000daa24072/0xa40083ab9bf537a9 lrc: 2/0,0 mode: PW/PW res: [0x2c0000400:0x430:0x0].0x0 rrc: 2 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x40010480000000 nid: local remote: 0x0 expref: -99 pid: 20762 timeout: 0 lvb_type: 0 00010000:00010000:3.0:1776178209.660547:0:6244:0:(ldlm_lockd.c:1893:ldlm_handle_bl_callback()) Lock 00000000daa24072 already unused, calling callback (0000000063c97258) 00010000:00010000:3.0:1776178209.660548:0:6244:0:(ldlm_request.c:364:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0001_UUID lock: 00000000daa24072/0xa40083ab9bf537a9 lrc: 2/0,0 mode: PW/PW res: [0x2c0000400:0x430:0x0].0x0 rrc: 2 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x40010480000000 nid: local remote: 0x0 expref: -99 pid: 20762 timeout: 0 lvb_type: 0 00010000:00010000:3.0:1776178209.660554:0:6244:0:(ldlm_request.c:1257:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0001_UUID lock: 00000000daa24072/0xa40083ab9bf537a9 lrc: 3/0,0 mode: PW/PW res: [0x2c0000400:0x430:0x0].0x0 rrc: 2 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x40018480000000 nid: local remote: 0x0 expref: -99 pid: 20762 timeout: 0 lvb_type: 0 00010000:00010000:3.0:1776178209.660564:0:6244:0:(ldlm_lockd.c:1903:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0001_UUID lock: 00000000daa24072/0xa40083ab9bf537a9 lrc: 1/0,0 mode: --/PW res: [0x2c0000400:0x430:0x0].0x0 rrc: 2 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x44819480000000 nid: local remote: 0x0 expref: -99 pid: 20762 timeout: 0 lvb_type: 0 00010000:00010000:3.0:1776178209.660570:0:6244:0:(ldlm_lock.c:216:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0001_UUID lock: 00000000daa24072/0xa40083ab9bf537a9 lrc: 0/0,0 mode: --/PW res: [0x2c0000400:0x430:0x0].0x0 rrc: 2 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x44819480000000 nid: local remote: 0x0 expref: -99 pid: 20762 timeout: 0 lvb_type: 0 00010000:00010000:3.0:1776178209.660583:0:6244:0:(ldlm_lockd.c:2789:ldlm_bl_get_work()) Got 1 locks of 0 total in blp. (0 blwis in pool) 00010000:00010000:3.0:1776178209.660586:0:6244:0:(ldlm_lockd.c:1879:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0001_UUID lock: 0000000030bf6b1e/0xa40083ab9bf537a2 lrc: 2/0,0 mode: PW/PW res: [0x2c0000400:0x42e:0x0].0x0 rrc: 2 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x40010480000000 nid: local remote: 0x0 expref: -99 pid: 20759 timeout: 0 lvb_type: 0 00010000:00010000:3.0:1776178209.660607:0:6244:0:(ldlm_lockd.c:1893:ldlm_handle_bl_callback()) Lock 0000000030bf6b1e already unused, calling callback (0000000063c97258) 00010000:00010000:3.0:1776178209.660609:0:6244:0:(ldlm_request.c:364:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0001_UUID lock: 0000000030bf6b1e/0xa40083ab9bf537a2 lrc: 2/0,0 mode: PW/PW res: [0x2c0000400:0x42e:0x0].0x0 rrc: 2 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x40010480000000 nid: local remote: 0x0 expref: -99 pid: 20759 timeout: 0 lvb_type: 0 00010000:00010000:3.0:1776178209.660615:0:6244:0:(ldlm_request.c:1257:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0001_UUID lock: 0000000030bf6b1e/0xa40083ab9bf537a2 lrc: 3/0,0 mode: PW/PW res: [0x2c0000400:0x42e:0x0].0x0 rrc: 2 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x40018480000000 nid: local remote: 0x0 expref: -99 pid: 20759 timeout: 0 lvb_type: 0 00010000:00010000:3.0:1776178209.660625:0:6244:0:(ldlm_lockd.c:1903:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0001_UUID lock: 0000000030bf6b1e/0xa40083ab9bf537a2 lrc: 1/0,0 mode: --/PW res: [0x2c0000400:0x42e:0x0].0x0 rrc: 2 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x44819480000000 nid: local remote: 0x0 expref: -99 pid: 20759 timeout: 0 lvb_type: 0 00010000:00010000:3.0:1776178209.660630:0:6244:0:(ldlm_lock.c:216:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0001_UUID lock: 0000000030bf6b1e/0xa40083ab9bf537a2 lrc: 0/0,0 mode: --/PW res: [0x2c0000400:0x42e:0x0].0x0 rrc: 2 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x44819480000000 nid: local remote: 0x0 expref: -99 pid: 20759 timeout: 0 lvb_type: 0 00010000:00010000:3.0:1776178209.660644:0:6244:0:(ldlm_lockd.c:2792:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00010000:00010000:3.0:1776178209.660646:0:6244:0:(ldlm_lockd.c:2792:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00010000:00010000:3.0:1776178209.660648:0:6244:0:(ldlm_lockd.c:2792:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:1.0:1776178209.661112:0:20762:0:(service.c:2372:ptlrpc_server_handle_request()) Handled RPC req@00000000d9bdf497 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_013:lustre-MDT0001-mdtlov_UUID+9:3354:x1862456998047872:12345-0@lo:6:osp-syn-1-1.0 Request processed in 20637us (21526us total) trans 4294969333 rc 0/0 00000100:00100000:1.0:1776178209.661125:0:20762:0:(nrs_fifo.c:241:nrs_fifo_req_stop()) NRS stop fifo request from 12345-0@lo, seq: 5279 00010000:00010000:1.0:1776178209.662685:0:9225:0:(ldlm_lockd.c:2792:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00010000:00010000:1.0:1776178209.662695:0:6245:0:(ldlm_lockd.c:2792:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000004:00080000:1.0:1776178209.662743:0:3354:0:(osp_sync.c:562:osp_sync_interpret()) reply req 00000000dd1281fe/2, rc 0, transno 2037 00000100:00100000:1.0:1776178209.662755:0:3354:0:(client.c:2215:ptlrpc_check_set()) Completed RPC req@00000000dd1281fe pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_01:lustre-MDT0001-mdtlov_UUID:3354:1862456998047872:0@lo:6:osp-syn-1-1.0 00000004:00080000:1.0:1776178209.662801:0:3355:0:(osp_sync.c:562:osp_sync_interpret()) reply req 000000009e7a8bb5/2, rc 0, transno 2036 00000100:00100000:1.0:1776178209.662805:0:3355:0:(client.c:2215:ptlrpc_check_set()) Completed RPC req@000000009e7a8bb5 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_02:lustre-MDT0001-mdtlov_UUID:3355:1862456998047680:0@lo:6:osp-syn-1-1.0 00010000:00010000:2.0:1776178209.663170:0:20755:0:(ldlm_lock.c:216:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0001_UUID lock: 000000003e1697a2/0xa40083ab9bf53533 lrc: 0/0,0 mode: --/PR res: [0x2c0000400:0x431:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x44801400010020 nid: 192.168.204.3@tcp remote: 0x771f68aa1eb27b26 expref: 5 pid: 20759 timeout: 1016 lvb_type: 1 00000100:00100000:2.0:1776178209.663193:0:20755:0:(client.c:2215:ptlrpc_check_set()) Completed RPC req@00000000eeeaf4ad pname:cluuid:pid:xid:nid:opc:job ll_ost00_006:lustre-OST0001_UUID:20755:1862456998048448:192.168.204.3@tcp:104: 00010000:00010000:2.0:1776178209.663221:0:20755:0:(ldlm_request.c:283:ldlm_completion_ast()) ### client-side enqueue returned a blocked locksleeping ns: filter-lustre-OST0001_UUID lock: 00000000a14e680b/0xa40083ab9bf537c5 lrc: 3/0,1 mode: PW/PW res: [0x2c0000400:0x431:0x0].0x0 rrc: 2 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x40010080000000 nid: local remote: 0x0 expref: -99 pid: 20755 timeout: 0 lvb_type: 0 00010000:00010000:2.0:1776178209.663234:0:20755:0:(ldlm_request.c:199:ldlm_completion_tail()) ### client-side enqueue: granted ns: filter-lustre-OST0001_UUID lock: 00000000a14e680b/0xa40083ab9bf537c5 lrc: 3/0,1 mode: PW/PW res: [0x2c0000400:0x431:0x0].0x0 rrc: 2 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x40010080000000 nid: local remote: 0x0 expref: -99 pid: 20755 timeout: 0 lvb_type: 0 00010000:00010000:2.0:1776178209.663240:0:20755:0:(ldlm_request.c:528:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0001_UUID lock: 00000000a14e680b/0xa40083ab9bf537c5 lrc: 3/0,1 mode: PW/PW res: [0x2c0000400:0x431:0x0].0x0 rrc: 2 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x40010080000000 nid: local remote: 0x0 expref: -99 pid: 20755 timeout: 0 lvb_type: 0 00010000:00010000:2.0:1776178209.663249:0:20755:0:(ldlm_lock.c:820:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0001_UUID lock: 00000000a14e680b/0xa40083ab9bf537c5 lrc: 3/0,1 mode: PW/PW res: [0x2c0000400:0x431:0x0].0x0 rrc: 2 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x50010080000000 nid: local remote: 0x0 expref: -99 pid: 20755 timeout: 0 lvb_type: 0 00010000:00010000:2.0:1776178209.663254:0:20755:0:(ldlm_lock.c:879:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0001_UUID lock: 00000000a14e680b/0xa40083ab9bf537c5 lrc: 2/0,0 mode: PW/PW res: [0x2c0000400:0x431:0x0].0x0 rrc: 2 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x50010480000000 nid: local remote: 0x0 expref: -99 pid: 20755 timeout: 0 lvb_type: 0 00010000:00010000:2.0:1776178209.663283:0:20755:0:(ldlm_lockd.c:2159:__ldlm_bl_to_thread()) added 1/1 locks to regular blp list, 1 blwis in pool 00000100:00100000:2.0:1776178209.663553:0:20755:0:(service.c:2372:ptlrpc_server_handle_request()) Handled RPC req@00000000d17c6101 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_006:lustre-MDT0001-mdtlov_UUID+9:3355:x1862456998047744:12345-0@lo:6:osp-syn-1-1.0 Request processed in 22316us (23755us total) trans 4294969335 rc 0/0 00000100:00100000:2.0:1776178209.663565:0:20755:0:(nrs_fifo.c:241:nrs_fifo_req_stop()) NRS stop fifo request from 12345-0@lo, seq: 5283 00010000:00010000:2.0:1776178209.663621:0:8227:0:(ldlm_lock.c:216:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 00000000a1e024a5/0xa40083ab9bf5352c lrc: 0/0,0 mode: --/PR res: [0x280000400:0x470:0x0].0x0 rrc: 3 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x44801400010020 nid: 192.168.204.3@tcp remote: 0x771f68aa1eb27b1f expref: 5 pid: 20759 timeout: 1016 lvb_type: 1 00000100:00100000:2.0:1776178209.663635:0:8227:0:(client.c:2215:ptlrpc_check_set()) Completed RPC req@00000000b30b9cdd pname:cluuid:pid:xid:nid:opc:job ll_ost00_000:lustre-OST0000_UUID:8227:1862456998048384:192.168.204.3@tcp:104: 00010000:00010000:2.0:1776178209.663650:0:8227:0:(ldlm_request.c:283:ldlm_completion_ast()) ### client-side enqueue returned a blocked locksleeping ns: filter-lustre-OST0000_UUID lock: 000000006b0afb52/0xa40083ab9bf537be lrc: 3/0,1 mode: PW/PW res: [0x280000400:0x470:0x0].0x0 rrc: 2 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x40010080000000 nid: local remote: 0x0 expref: -99 pid: 8227 timeout: 0 lvb_type: 0 00010000:00010000:2.0:1776178209.663657:0:8227:0:(ldlm_request.c:199:ldlm_completion_tail()) ### client-side enqueue: granted ns: filter-lustre-OST0000_UUID lock: 000000006b0afb52/0xa40083ab9bf537be lrc: 3/0,1 mode: PW/PW res: [0x280000400:0x470:0x0].0x0 rrc: 2 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x40010080000000 nid: local remote: 0x0 expref: -99 pid: 8227 timeout: 0 lvb_type: 0 00010000:00010000:2.0:1776178209.663663:0:8227:0:(ldlm_request.c:528:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 000000006b0afb52/0xa40083ab9bf537be lrc: 3/0,1 mode: PW/PW res: [0x280000400:0x470:0x0].0x0 rrc: 2 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x40010080000000 nid: local remote: 0x0 expref: -99 pid: 8227 timeout: 0 lvb_type: 0 00010000:00010000:2.0:1776178209.663671:0:8227:0:(ldlm_lock.c:820:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0000_UUID lock: 000000006b0afb52/0xa40083ab9bf537be lrc: 3/0,1 mode: PW/PW res: [0x280000400:0x470:0x0].0x0 rrc: 2 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x50010080000000 nid: local remote: 0x0 expref: -99 pid: 8227 timeout: 0 lvb_type: 0 00010000:00010000:2.0:1776178209.663678:0:8227:0:(ldlm_lock.c:879:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 000000006b0afb52/0xa40083ab9bf537be lrc: 2/0,0 mode: PW/PW res: [0x280000400:0x470:0x0].0x0 rrc: 2 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x50010480000000 nid: local remote: 0x0 expref: -99 pid: 8227 timeout: 0 lvb_type: 0 00010000:00010000:2.0:1776178209.663697:0:8227:0:(ldlm_lockd.c:2159:__ldlm_bl_to_thread()) added 1/2 locks to regular blp list, 2 blwis in pool 00010000:00010000:1.0:1776178209.664382:0:9224:0:(ldlm_lockd.c:2789:ldlm_bl_get_work()) Got 1 locks of 1 total in blp. (1 blwis in pool) 00010000:00010000:1.0:1776178209.664391:0:9224:0:(ldlm_lockd.c:1879:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0001_UUID lock: 00000000a14e680b/0xa40083ab9bf537c5 lrc: 2/0,0 mode: PW/PW res: [0x2c0000400:0x431:0x0].0x0 rrc: 2 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x40010480000000 nid: local remote: 0x0 expref: -99 pid: 20755 timeout: 0 lvb_type: 0 00010000:00010000:1.0:1776178209.664402:0:9224:0:(ldlm_lockd.c:1893:ldlm_handle_bl_callback()) Lock 00000000a14e680b already unused, calling callback (0000000063c97258) 00010000:00010000:1.0:1776178209.664406:0:9224:0:(ldlm_request.c:364:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0001_UUID lock: 00000000a14e680b/0xa40083ab9bf537c5 lrc: 2/0,0 mode: PW/PW res: [0x2c0000400:0x431:0x0].0x0 rrc: 2 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x40010480000000 nid: local remote: 0x0 expref: -99 pid: 20755 timeout: 0 lvb_type: 0 00010000:00010000:1.0:1776178209.664415:0:9224:0:(ldlm_request.c:1257:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0001_UUID lock: 00000000a14e680b/0xa40083ab9bf537c5 lrc: 3/0,0 mode: PW/PW res: [0x2c0000400:0x431:0x0].0x0 rrc: 2 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x40018480000000 nid: local remote: 0x0 expref: -99 pid: 20755 timeout: 0 lvb_type: 0 00010000:00010000:1.0:1776178209.664436:0:9224:0:(ldlm_lockd.c:1903:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0001_UUID lock: 00000000a14e680b/0xa40083ab9bf537c5 lrc: 1/0,0 mode: --/PW res: [0x2c0000400:0x431:0x0].0x0 rrc: 2 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x44819480000000 nid: local remote: 0x0 expref: -99 pid: 20755 timeout: 0 lvb_type: 0 00010000:00010000:1.0:1776178209.664442:0:9224:0:(ldlm_lock.c:216:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0001_UUID lock: 00000000a14e680b/0xa40083ab9bf537c5 lrc: 0/0,0 mode: --/PW res: [0x2c0000400:0x431:0x0].0x0 rrc: 2 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x44819480000000 nid: local remote: 0x0 expref: -99 pid: 20755 timeout: 0 lvb_type: 0 00010000:00010000:1.0:1776178209.664460:0:9224:0:(ldlm_lockd.c:2789:ldlm_bl_get_work()) Got 1 locks of 0 total in blp. (0 blwis in pool) 00010000:00010000:1.0:1776178209.664462:0:9224:0:(ldlm_lockd.c:1879:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 000000006b0afb52/0xa40083ab9bf537be lrc: 2/0,0 mode: PW/PW res: [0x280000400:0x470:0x0].0x0 rrc: 2 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x40010480000000 nid: local remote: 0x0 expref: -99 pid: 8227 timeout: 0 lvb_type: 0 00010000:00010000:1.0:1776178209.664468:0:9224:0:(ldlm_lockd.c:1893:ldlm_handle_bl_callback()) Lock 000000006b0afb52 already unused, calling callback (0000000063c97258) 00010000:00010000:1.0:1776178209.664469:0:9224:0:(ldlm_request.c:364:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 000000006b0afb52/0xa40083ab9bf537be lrc: 2/0,0 mode: PW/PW res: [0x280000400:0x470:0x0].0x0 rrc: 2 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x40010480000000 nid: local remote: 0x0 expref: -99 pid: 8227 timeout: 0 lvb_type: 0 00010000:00010000:1.0:1776178209.664475:0:9224:0:(ldlm_request.c:1257:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 000000006b0afb52/0xa40083ab9bf537be lrc: 3/0,0 mode: PW/PW res: [0x280000400:0x470:0x0].0x0 rrc: 2 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x40018480000000 nid: local remote: 0x0 expref: -99 pid: 8227 timeout: 0 lvb_type: 0 00010000:00010000:1.0:1776178209.664487:0:9224:0:(ldlm_lockd.c:1903:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 000000006b0afb52/0xa40083ab9bf537be lrc: 1/0,0 mode: --/PW res: [0x280000400:0x470:0x0].0x0 rrc: 2 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x44819480000000 nid: local remote: 0x0 expref: -99 pid: 8227 timeout: 0 lvb_type: 0 00010000:00010000:1.0:1776178209.664492:0:9224:0:(ldlm_lock.c:216:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 000000006b0afb52/0xa40083ab9bf537be lrc: 0/0,0 mode: --/PW res: [0x280000400:0x470:0x0].0x0 rrc: 2 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x44819480000000 nid: local remote: 0x0 expref: -99 pid: 8227 timeout: 0 lvb_type: 0 00010000:00010000:1.0:1776178209.664501:0:9224:0:(ldlm_lockd.c:2792:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00010000:00010000:1.0:1776178209.664502:0:9224:0:(ldlm_lockd.c:2792:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00010000:00010000:1.0:1776178209.664504:0:9224:0:(ldlm_lockd.c:2792:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000004:00080000:1.0:1776178209.664535:0:3355:0:(osp_sync.c:562:osp_sync_interpret()) reply req 0000000072865dff/2, rc 0, transno 2039 00000100:00100000:1.0:1776178209.664541:0:3355:0:(client.c:2215:ptlrpc_check_set()) Completed RPC req@0000000072865dff pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_02:lustre-MDT0001-mdtlov_UUID:3355:1862456998047744:0@lo:6:osp-syn-1-1.0 00000100:00100000:2.0:1776178209.664627:0:8227:0:(service.c:2372:ptlrpc_server_handle_request()) Handled RPC req@000000001a990b02 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_000:lustre-MDT0001-mdtlov_UUID+8:3355:x1862456998047936:12345-0@lo:6:osp-syn-0-1.0 Request processed in 23654us (24880us total) trans 4294969360 rc 0/0 00000100:00100000:2.0:1776178209.664635:0:8227:0:(nrs_fifo.c:241:nrs_fifo_req_stop()) NRS stop fifo request from 12345-0@lo, seq: 5282 00000004:00080000:1.0:1776178209.664881:0:3355:0:(osp_sync.c:562:osp_sync_interpret()) reply req 0000000044f82452/2, rc 0, transno 2064 00000100:00100000:1.0:1776178209.664886:0:3355:0:(client.c:2215:ptlrpc_check_set()) Completed RPC req@0000000044f82452 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_02:lustre-MDT0001-mdtlov_UUID:3355:1862456998047936:0@lo:6:osp-syn-0-1.0 00000004:00080000:3.0:1776178209.665279:0:3353:0:(osp_sync.c:562:osp_sync_interpret()) reply req 00000000f95e9da5/2, rc 0, transno 2038 00000100:00100000:3.0:1776178209.665284:0:3353:0:(client.c:2215:ptlrpc_check_set()) Completed RPC req@00000000f95e9da5 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_00:lustre-MDT0001-mdtlov_UUID:3353:1862456998047616:0@lo:6:osp-syn-1-1.0 00010000:00010000:3.0:1776178209.665302:0:6244:0:(ldlm_lockd.c:2792:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00010000:00010000:3.0:1776178209.665362:0:14440:0:(ldlm_request.c:199:ldlm_completion_tail()) ### client-side enqueue: granted ns: filter-lustre-OST0000_UUID lock: 0000000097975a3b/0xa40083ab9bf537cc lrc: 3/0,1 mode: PW/PW res: [0x280000400:0x46f:0x0].0x0 rrc: 2 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x40010080000000 nid: local remote: 0x0 expref: -99 pid: 14440 timeout: 0 lvb_type: 0 00010000:00010000:3.0:1776178209.665371:0:14440:0:(ldlm_request.c:528:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 0000000097975a3b/0xa40083ab9bf537cc lrc: 3/0,1 mode: PW/PW res: [0x280000400:0x46f:0x0].0x0 rrc: 2 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x40010080000000 nid: local remote: 0x0 expref: -99 pid: 14440 timeout: 0 lvb_type: 0 00010000:00010000:3.0:1776178209.665380:0:14440:0:(ldlm_lock.c:820:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: filter-lustre-OST0000_UUID lock: 0000000097975a3b/0xa40083ab9bf537cc lrc: 3/0,1 mode: PW/PW res: [0x280000400:0x46f:0x0].0x0 rrc: 2 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x50010080000000 nid: local remote: 0x0 expref: -99 pid: 14440 timeout: 0 lvb_type: 0 00010000:00010000:3.0:1776178209.665387:0:14440:0:(ldlm_lock.c:879:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: filter-lustre-OST0000_UUID lock: 0000000097975a3b/0xa40083ab9bf537cc lrc: 2/0,0 mode: PW/PW res: [0x280000400:0x46f:0x0].0x0 rrc: 2 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x50010480000000 nid: local remote: 0x0 expref: -99 pid: 14440 timeout: 0 lvb_type: 0 00010000:00010000:3.0:1776178209.665412:0:14440:0:(ldlm_lockd.c:2159:__ldlm_bl_to_thread()) added 1/1 locks to regular blp list, 1 blwis in pool 00000100:00100000:3.0:1776178209.665644:0:14440:0:(service.c:2372:ptlrpc_server_handle_request()) Handled RPC req@000000008a1e777f pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_004:lustre-MDT0001-mdtlov_UUID+8:3355:x1862456998048000:12345-0@lo:6:osp-syn-0-1.0 Request processed in 24283us (25816us total) trans 4294969361 rc 0/0 00000100:00100000:3.0:1776178209.665654:0:14440:0:(nrs_fifo.c:241:nrs_fifo_req_stop()) NRS stop fifo request from 12345-0@lo, seq: 5284 00000004:00080000:3.0:1776178209.665685:0:3355:0:(osp_sync.c:562:osp_sync_interpret()) reply req 0000000028ade006/2, rc 0, transno 2065 00000100:00100000:3.0:1776178209.665689:0:3355:0:(client.c:2215:ptlrpc_check_set()) Completed RPC req@0000000028ade006 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_02:lustre-MDT0001-mdtlov_UUID:3355:1862456998048000:0@lo:6:osp-syn-0-1.0 00010000:00010000:1.0:1776178209.665742:0:9225:0:(ldlm_lockd.c:2789:ldlm_bl_get_work()) Got 1 locks of 0 total in blp. (0 blwis in pool) 00010000:00010000:1.0:1776178209.665746:0:9225:0:(ldlm_lockd.c:1879:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: filter-lustre-OST0000_UUID lock: 0000000097975a3b/0xa40083ab9bf537cc lrc: 2/0,0 mode: PW/PW res: [0x280000400:0x46f:0x0].0x0 rrc: 2 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x40010480000000 nid: local remote: 0x0 expref: -99 pid: 14440 timeout: 0 lvb_type: 0 00010000:00010000:1.0:1776178209.665754:0:9225:0:(ldlm_lockd.c:1893:ldlm_handle_bl_callback()) Lock 0000000097975a3b already unused, calling callback (0000000063c97258) 00010000:00010000:1.0:1776178209.665756:0:9225:0:(ldlm_request.c:364:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: filter-lustre-OST0000_UUID lock: 0000000097975a3b/0xa40083ab9bf537cc lrc: 2/0,0 mode: PW/PW res: [0x280000400:0x46f:0x0].0x0 rrc: 2 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x40010480000000 nid: local remote: 0x0 expref: -99 pid: 14440 timeout: 0 lvb_type: 0 00010000:00010000:1.0:1776178209.665763:0:9225:0:(ldlm_request.c:1257:ldlm_cli_cancel_local()) ### server-side local cancel ns: filter-lustre-OST0000_UUID lock: 0000000097975a3b/0xa40083ab9bf537cc lrc: 3/0,0 mode: PW/PW res: [0x280000400:0x46f:0x0].0x0 rrc: 2 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x40018480000000 nid: local remote: 0x0 expref: -99 pid: 14440 timeout: 0 lvb_type: 0 00010000:00010000:1.0:1776178209.665780:0:9225:0:(ldlm_lockd.c:1903:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: filter-lustre-OST0000_UUID lock: 0000000097975a3b/0xa40083ab9bf537cc lrc: 1/0,0 mode: --/PW res: [0x280000400:0x46f:0x0].0x0 rrc: 2 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x44819480000000 nid: local remote: 0x0 expref: -99 pid: 14440 timeout: 0 lvb_type: 0 00010000:00010000:1.0:1776178209.665790:0:9225:0:(ldlm_lock.c:216:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: filter-lustre-OST0000_UUID lock: 0000000097975a3b/0xa40083ab9bf537cc lrc: 0/0,0 mode: --/PW res: [0x280000400:0x46f:0x0].0x0 rrc: 2 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x44819480000000 nid: local remote: 0x0 expref: -99 pid: 14440 timeout: 0 lvb_type: 0 00010000:00010000:1.0:1776178209.665804:0:9225:0:(ldlm_lockd.c:2792:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00010000:00010000:1.0:1776178209.665805:0:9225:0:(ldlm_lockd.c:2792:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00010000:00010000:1.0:1776178209.665807:0:9225:0:(ldlm_lockd.c:2792:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00010000:00010000:1.0:1776178210.274751:0:6245:0:(ldlm_lockd.c:2792:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:1.0:1776178210.656688:0:3354:0:(client.c:1733:ptlrpc_send_new_req()) Sending RPC req@00000000fab910b9 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_01:lustre-MDT0000-mdtlov_UUID:3354:1862456998048576:0@lo:13:osp-pre-1-0.0 00000100:00100000:1.0:1776178210.656798:0:3354:0:(events.c:361:request_in_callback()) peer: 12345-0@lo (source: 12345-0@lo) 00000100:00100000:1.0:1776178210.656902:0:8232:0:(service.c:2152:ptlrpc_server_handle_req_in()) got req x1862456998048576 00000100:00100000:1.0:1776178210.656956:0:8232:0:(nrs_fifo.c:179:nrs_fifo_req_get()) NRS start fifo request from 12345-0@lo, seq: 483 00000100:00100000:1.0:1776178210.656963:0:8232:0:(service.c:2324:ptlrpc_server_handle_request()) Handling RPC req@000000007058013a pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_create00:lustre-MDT0000-mdtlov_UUID+5:3354:x1862456998048576:12345-0@lo:13:osp-pre-1-0.0 00002000:00000024:1.0:1776178210.657004:0:8232:0:(ofd_obd.c:740:ofd_statfs()) blocks cached 229376 granted 491746368 pending 0 free 3923689472 avail 3692167168 00000100:00100000:2.0:1776178210.657141:0:3356:0:(ptlrpcd.c:415:ptlrpcd_check()) transfer 1 async RPCs [2->3] 00000100:00100000:2.0:1776178210.657150:0:3356:0:(client.c:1733:ptlrpc_send_new_req()) Sending RPC req@00000000ca8b81ba pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_03:lustre-MDT0001-mdtlov_UUID:3356:1862456998048640:0@lo:41:osp-pre-0-1.0 00000100:00100000:2.0:1776178210.657182:0:3356:0:(events.c:361:request_in_callback()) peer: 12345-0@lo (source: 12345-0@lo) 00000100:00100000:1.0:1776178210.657291:0:8232:0:(service.c:2372:ptlrpc_server_handle_request()) Handled RPC req@000000007058013a pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_create00:lustre-MDT0000-mdtlov_UUID+5:3354:x1862456998048576:12345-0@lo:13:osp-pre-1-0.0 Request processed in 327us (495us total) trans 0 rc 0/0 00000100:00100000:1.0:1776178210.657300:0:8232:0:(nrs_fifo.c:241:nrs_fifo_req_stop()) NRS stop fifo request from 12345-0@lo, seq: 483 00000100:00100000:2.0:1776178210.657462:0:3354:0:(client.c:2215:ptlrpc_check_set()) Completed RPC req@00000000fab910b9 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_01:lustre-MDT0000-mdtlov_UUID:3354:1862456998048576:0@lo:13:osp-pre-1-0.0 00000100:00100000:3.0:1776178210.657857:0:6261:0:(service.c:2152:ptlrpc_server_handle_req_in()) got req x1862456998048640 00000100:00100000:3.0:1776178210.657891:0:6261:0:(nrs_fifo.c:179:nrs_fifo_req_get()) NRS start fifo request from 12345-0@lo, seq: 703 00000100:00100000:3.0:1776178210.657895:0:6261:0:(service.c:2324:ptlrpc_server_handle_request()) Handling RPC req@00000000d28be450 pname:cluuid+ref:pid:xid:nid:opc:job mdt_out00_001:lustre-MDT0001-mdtlov_UUID+7:3356:x1862456998048640:12345-0@lo:41:osp-pre-0-1.0 00000004:00000024:3.0:1776178210.657921:0:6261:0:(mdt_handler.c:548:mdt_statfs()) blocks cached 0 granted 2097152 pending 0 free 1446125568 avail 1316663296 00000100:00100000:3.0:1776178210.657958:0:6261:0:(service.c:2372:ptlrpc_server_handle_request()) Handled RPC req@00000000d28be450 pname:cluuid+ref:pid:xid:nid:opc:job mdt_out00_001:lustre-MDT0001-mdtlov_UUID+7:3356:x1862456998048640:12345-0@lo:41:osp-pre-0-1.0 Request processed in 62us (776us total) trans 0 rc 0/0 00000100:00100000:3.0:1776178210.657963:0:6261:0:(nrs_fifo.c:241:nrs_fifo_req_stop()) NRS stop fifo request from 12345-0@lo, seq: 703 00000100:00100000:2.0:1776178210.657972:0:3356:0:(client.c:1733:ptlrpc_send_new_req()) Sending RPC req@00000000fab910b9 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_03:ac61f839-0e8e-4cac-9c59-08bcfbbad330:3356:1862456998048704:0@lo:400:kworker/u8:2.0 00000100:00100000:3.0:1776178210.657983:0:3355:0:(client.c:1733:ptlrpc_send_new_req()) Sending RPC req@000000000b5ad4fd pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_02:lustre-MDT0000-lwp-OST0001_UUID:3355:1862456998048896:0@lo:400:kworker/u8:2.0 00000100:00100000:3.0:1776178210.658015:0:3355:0:(events.c:361:request_in_callback()) peer: 12345-0@lo (source: 12345-0@lo) 00000100:00100000:2.0:1776178210.658023:0:3356:0:(events.c:361:request_in_callback()) peer: 12345-0@lo (source: 12345-0@lo) 00000100:00100000:3.0:1776178210.658029:0:3355:0:(client.c:1733:ptlrpc_send_new_req()) Sending RPC req@000000004828c19d pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_02:lustre-MDT0000-lwp-MDT0001_UUID:3355:1862456998049152:0@lo:400:kworker/u8:2.0 00000100:00100000:2.0:1776178210.658035:0:3356:0:(client.c:1733:ptlrpc_send_new_req()) Sending RPC req@0000000065dbeee0 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_03:lustre-MDT0001-lwp-OST0001_UUID:3356:1862456998048960:0@lo:400:kworker/u8:2.0 00000100:00100000:3.0:1776178210.658050:0:3355:0:(events.c:361:request_in_callback()) peer: 12345-0@lo (source: 12345-0@lo) 00000100:00100000:2.0:1776178210.658057:0:3354:0:(client.c:1733:ptlrpc_send_new_req()) Sending RPC req@000000007d766b2a pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_01:lustre-MDT0001-lwp-OST0000_UUID:3354:1862456998048832:0@lo:400:kworker/u8:2.0 00000100:00100000:3.0:1776178210.658093:0:3353:0:(client.c:1733:ptlrpc_send_new_req()) Sending RPC req@00000000b85458e9 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_00:lustre-MDT0000-lwp-OST0000_UUID:3353:1862456998048768:0@lo:400:kworker/u8:2.0 00000100:00100000:2.0:1776178210.658099:0:3354:0:(events.c:361:request_in_callback()) peer: 12345-0@lo (source: 12345-0@lo) 00000100:00100000:2.0:1776178210.658114:0:3354:0:(client.c:1733:ptlrpc_send_new_req()) Sending RPC req@0000000042de09e9 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_01:lustre-MDT0001-mdtlov_UUID:3354:1862456998049088:0@lo:400:kworker/u8:2.0 00000100:00100000:3.0:1776178210.658114:0:3353:0:(events.c:361:request_in_callback()) peer: 12345-0@lo (source: 12345-0@lo) 00000100:00100000:3.0:1776178210.658127:0:3353:0:(client.c:1733:ptlrpc_send_new_req()) Sending RPC req@000000009c4a59b3 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_00:lustre-MDT0001-mdtlov_UUID:3353:1862456998049024:0@lo:400:kworker/u8:2.0 00000100:00100000:2.0:1776178210.658134:0:3354:0:(events.c:361:request_in_callback()) peer: 12345-0@lo (source: 12345-0@lo) 00000100:00100000:3.0:1776178210.658151:0:6256:0:(service.c:2152:ptlrpc_server_handle_req_in()) got req x1862456998048896 00000100:00100000:2.0:1776178210.658152:0:3354:0:(client.c:1733:ptlrpc_send_new_req()) Sending RPC req@00000000fd2b8392 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_01:lustre-MDT0000-lwp-MDT0000_UUID:3354:1862456998049344:0@lo:400:kworker/u8:2.0 00000100:00100000:2.0:1776178210.658171:0:3354:0:(events.c:361:request_in_callback()) peer: 12345-0@lo (source: 12345-0@lo) 00000100:00100000:3.0:1776178210.658176:0:6256:0:(service.c:2152:ptlrpc_server_handle_req_in()) got req x1862456998049152 00000100:00100000:3.0:1776178210.658187:0:6256:0:(service.c:2152:ptlrpc_server_handle_req_in()) got req x1862456998048832 00000100:00100000:2.0:1776178210.658192:0:8250:0:(service.c:2152:ptlrpc_server_handle_req_in()) got req x1862456998048768 00000100:00100000:3.0:1776178210.658198:0:6256:0:(service.c:2152:ptlrpc_server_handle_req_in()) got req x1862456998049344 00000100:00100000:3.0:1776178210.658207:0:6256:0:(nrs_fifo.c:179:nrs_fifo_req_get()) NRS start fifo request from 12345-0@lo, seq: 918 00000100:00100000:3.0:1776178210.658211:0:6256:0:(service.c:2324:ptlrpc_server_handle_request()) Handling RPC req@000000008c938150 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_001:lustre-MDT0000-lwp-OST0001_UUID+5:3355:x1862456998048896:12345-0@lo:400:kworker/u8:2.0 00000100:00100000:2.0:1776178210.658272:0:8250:0:(nrs_fifo.c:179:nrs_fifo_req_get()) NRS start fifo request from 12345-0@lo, seq: 919 00000100:00100000:2.0:1776178210.658276:0:8250:0:(service.c:2324:ptlrpc_server_handle_request()) Handling RPC req@00000000be1ebf42 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_003:lustre-MDT0000-lwp-MDT0001_UUID+5:3355:x1862456998049152:12345-0@lo:400:kworker/u8:2.0 00000100:00100000:1.0:1776178210.658285:0:6257:0:(nrs_fifo.c:179:nrs_fifo_req_get()) NRS start fifo request from 12345-0@lo, seq: 920 00000100:00100000:1.0:1776178210.658289:0:6257:0:(service.c:2324:ptlrpc_server_handle_request()) Handling RPC req@000000000a0451ff pname:cluuid+ref:pid:xid:nid:opc:job mdt00_002:lustre-MDT0001-lwp-OST0000_UUID+5:3354:x1862456998048832:12345-0@lo:400:kworker/u8:2.0 00000100:00100000:1.0:1776178210.658321:0:6257:0:(service.c:2372:ptlrpc_server_handle_request()) Handled RPC req@000000000a0451ff pname:cluuid+ref:pid:xid:nid:opc:job mdt00_002:lustre-MDT0001-lwp-OST0000_UUID+5:3354:x1862456998048832:12345-0@lo:400:kworker/u8:2.0 Request processed in 31us (222us total) trans 0 rc 0/0 00000100:00100000:1.0:1776178210.658328:0:6257:0:(nrs_fifo.c:241:nrs_fifo_req_stop()) NRS stop fifo request from 12345-0@lo, seq: 920 00000100:00100000:1.0:1776178210.658341:0:6257:0:(nrs_fifo.c:179:nrs_fifo_req_get()) NRS start fifo request from 12345-0@lo, seq: 921 00000100:00100000:1.0:1776178210.658344:0:6257:0:(service.c:2324:ptlrpc_server_handle_request()) Handling RPC req@00000000bc2ea964 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_002:lustre-MDT0000-lwp-OST0000_UUID+5:3353:x1862456998048768:12345-0@lo:400:kworker/u8:2.0 00000100:00100000:1.0:1776178210.658368:0:6257:0:(service.c:2372:ptlrpc_server_handle_request()) Handled RPC req@00000000bc2ea964 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_002:lustre-MDT0000-lwp-OST0000_UUID+5:3353:x1862456998048768:12345-0@lo:400:kworker/u8:2.0 Request processed in 22us (253us total) trans 0 rc 0/0 00000100:00100000:1.0:1776178210.658374:0:6257:0:(nrs_fifo.c:241:nrs_fifo_req_stop()) NRS stop fifo request from 12345-0@lo, seq: 921 00000100:00100000:1.0:1776178210.658385:0:6257:0:(nrs_fifo.c:179:nrs_fifo_req_get()) NRS start fifo request from 12345-0@lo, seq: 922 00000100:00100000:1.0:1776178210.658387:0:6257:0:(service.c:2324:ptlrpc_server_handle_request()) Handling RPC req@0000000037aace32 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_002:lustre-MDT0000-lwp-MDT0000_UUID+5:3354:x1862456998049344:12345-0@lo:400:kworker/u8:2.0 00000100:00100000:2.0:1776178210.658401:0:8250:0:(service.c:2372:ptlrpc_server_handle_request()) Handled RPC req@00000000be1ebf42 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_003:lustre-MDT0000-lwp-MDT0001_UUID+5:3355:x1862456998049152:12345-0@lo:400:kworker/u8:2.0 Request processed in 124us (350us total) trans 0 rc 0/0 00000100:00100000:2.0:1776178210.658407:0:8250:0:(nrs_fifo.c:241:nrs_fifo_req_stop()) NRS stop fifo request from 12345-0@lo, seq: 919 00000100:00100000:1.0:1776178210.658408:0:6257:0:(service.c:2372:ptlrpc_server_handle_request()) Handled RPC req@0000000037aace32 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_002:lustre-MDT0000-lwp-MDT0000_UUID+5:3354:x1862456998049344:12345-0@lo:400:kworker/u8:2.0 Request processed in 20us (237us total) trans 0 rc 0/0 00000100:00100000:1.0:1776178210.658413:0:6257:0:(nrs_fifo.c:241:nrs_fifo_req_stop()) NRS stop fifo request from 12345-0@lo, seq: 922 00000100:00100000:1.0:1776178210.658427:0:20140:0:(service.c:2152:ptlrpc_server_handle_req_in()) got req x1862456998048704 00000100:00100000:2.0:1776178210.658428:0:3353:0:(events.c:361:request_in_callback()) peer: 12345-0@lo (source: 12345-0@lo) 00000100:00100000:2.0:1776178210.658442:0:3353:0:(client.c:1733:ptlrpc_send_new_req()) Sending RPC req@0000000075189f40 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_00:lustre-MDT0000-mdtlov_UUID:3353:1862456998049280:0@lo:400:kworker/u8:2.0 00000100:00100000:1.0:1776178210.658450:0:20140:0:(nrs_fifo.c:179:nrs_fifo_req_get()) NRS start fifo request from 12345-0@lo, seq: 159 00000100:00100000:1.0:1776178210.658454:0:20140:0:(service.c:2324:ptlrpc_server_handle_request()) Handling RPC req@0000000038b01669 pname:cluuid+ref:pid:xid:nid:opc:job ll_mgs_0002:ac61f839-0e8e-4cac-9c59-08bcfbbad330+23:3356:x1862456998048704:12345-0@lo:400:kworker/u8:2.0 00000100:00100000:2.0:1776178210.658464:0:3353:0:(events.c:361:request_in_callback()) peer: 12345-0@lo (source: 12345-0@lo) 00000100:00100000:1.0:1776178210.658478:0:20140:0:(service.c:2372:ptlrpc_server_handle_request()) Handled RPC req@0000000038b01669 pname:cluuid+ref:pid:xid:nid:opc:job ll_mgs_0002:ac61f839-0e8e-4cac-9c59-08bcfbbad330+23:3356:x1862456998048704:12345-0@lo:400:kworker/u8:2.0 Request processed in 24us (471us total) trans 0 rc 0/0 00000100:00100000:1.0:1776178210.658483:0:20140:0:(nrs_fifo.c:241:nrs_fifo_req_stop()) NRS stop fifo request from 12345-0@lo, seq: 159 00000100:00100000:2.0:1776178210.658493:0:3353:0:(client.c:2215:ptlrpc_check_set()) Completed RPC req@00000000b85458e9 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_00:lustre-MDT0000-lwp-OST0000_UUID:3353:1862456998048768:0@lo:400:kworker/u8:2.0 00000100:00100000:2.0:1776178210.658517:0:8227:0:(service.c:2152:ptlrpc_server_handle_req_in()) got req x1862456998049088 00000100:00100000:1.0:1776178210.658531:0:20755:0:(service.c:2152:ptlrpc_server_handle_req_in()) got req x1862456998049024 00000100:00100000:2.0:1776178210.658543:0:8227:0:(service.c:2152:ptlrpc_server_handle_req_in()) got req x1862456998049280 00000100:00100000:2.0:1776178210.658556:0:8227:0:(nrs_fifo.c:179:nrs_fifo_req_get()) NRS start fifo request from 12345-0@lo, seq: 516 00000100:00100000:2.0:1776178210.658560:0:8227:0:(service.c:2324:ptlrpc_server_handle_request()) Handling RPC req@00000000e933e46b pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_000:lustre-MDT0001-mdtlov_UUID+9:3354:x1862456998049088:12345-0@lo:400:kworker/u8:2.0 00000100:00100000:2.0:1776178210.658599:0:8227:0:(service.c:2372:ptlrpc_server_handle_request()) Handled RPC req@00000000e933e46b pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_000:lustre-MDT0001-mdtlov_UUID+9:3354:x1862456998049088:12345-0@lo:400:kworker/u8:2.0 Request processed in 38us (464us total) trans 0 rc 0/0 00000100:00100000:2.0:1776178210.658605:0:8227:0:(nrs_fifo.c:241:nrs_fifo_req_stop()) NRS stop fifo request from 12345-0@lo, seq: 516 00000100:00100000:2.0:1776178210.658615:0:8227:0:(nrs_fifo.c:179:nrs_fifo_req_get()) NRS start fifo request from 12345-0@lo, seq: 517 00000100:00100000:2.0:1776178210.658619:0:8227:0:(service.c:2324:ptlrpc_server_handle_request()) Handling RPC req@0000000004dd3b9b pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_000:lustre-MDT0000-mdtlov_UUID+5:3353:x1862456998049280:12345-0@lo:400:kworker/u8:2.0 00000100:00100000:2.0:1776178210.658646:0:8227:0:(service.c:2372:ptlrpc_server_handle_request()) Handled RPC req@0000000004dd3b9b pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_000:lustre-MDT0000-mdtlov_UUID+5:3353:x1862456998049280:12345-0@lo:400:kworker/u8:2.0 Request processed in 26us (182us total) trans 0 rc 0/0 00000100:00100000:2.0:1776178210.658651:0:8227:0:(nrs_fifo.c:241:nrs_fifo_req_stop()) NRS stop fifo request from 12345-0@lo, seq: 517 00000100:00100000:2.0:1776178210.658675:0:3353:0:(client.c:2215:ptlrpc_check_set()) Completed RPC req@0000000075189f40 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_00:lustre-MDT0000-mdtlov_UUID:3353:1862456998049280:0@lo:400:kworker/u8:2.0 00000100:00100000:2.0:1776178210.658693:0:3356:0:(events.c:361:request_in_callback()) peer: 12345-0@lo (source: 12345-0@lo) 00000100:00100000:2.0:1776178210.658706:0:3356:0:(client.c:1733:ptlrpc_send_new_req()) Sending RPC req@000000002300ff10 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_03:lustre-MDT0000-mdtlov_UUID:3356:1862456998049216:0@lo:400:kworker/u8:2.0 00000100:00100000:2.0:1776178210.658728:0:3356:0:(events.c:361:request_in_callback()) peer: 12345-0@lo (source: 12345-0@lo) 00000100:00100000:2.0:1776178210.658749:0:3356:0:(client.c:2215:ptlrpc_check_set()) Completed RPC req@00000000ca8b81ba pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_03:lustre-MDT0001-mdtlov_UUID:3356:1862456998048640:0@lo:41:osp-pre-0-1.0 00000100:00100000:2.0:1776178210.658765:0:3356:0:(client.c:2215:ptlrpc_check_set()) Completed RPC req@00000000fab910b9 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_03:ac61f839-0e8e-4cac-9c59-08bcfbbad330:3356:1862456998048704:0@lo:400:kworker/u8:2.0 00000100:00100000:2.0:1776178210.658773:0:3356:0:(ptlrpcd.c:415:ptlrpcd_check()) transfer 1 async RPCs [2->3] 00000100:00100000:2.0:1776178210.658778:0:3356:0:(client.c:1733:ptlrpc_send_new_req()) Sending RPC req@00000000b8f6795f pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_03:lustre-MDT0000-mdtlov_UUID:3356:1862456998049408:0@lo:41:osp-pre-1-0.0 00000100:00100000:2.0:1776178210.658797:0:3356:0:(events.c:361:request_in_callback()) peer: 12345-0@lo (source: 12345-0@lo) 00000100:00100000:3.0:1776178210.658814:0:6256:0:(service.c:2372:ptlrpc_server_handle_request()) Handled RPC req@000000008c938150 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_001:lustre-MDT0000-lwp-OST0001_UUID+5:3355:x1862456998048896:12345-0@lo:400:kworker/u8:2.0 Request processed in 602us (799us total) trans 0 rc 0/0 00000100:00100000:2.0:1776178210.658819:0:6255:0:(service.c:2152:ptlrpc_server_handle_req_in()) got req x1862456998048960 00000100:00100000:3.0:1776178210.658819:0:6256:0:(nrs_fifo.c:241:nrs_fifo_req_stop()) NRS stop fifo request from 12345-0@lo, seq: 918 00000100:00100000:2.0:1776178210.659114:0:6255:0:(nrs_fifo.c:179:nrs_fifo_req_get()) NRS start fifo request from 12345-0@lo, seq: 923 00000100:00100000:2.0:1776178210.659119:0:6255:0:(service.c:2324:ptlrpc_server_handle_request()) Handling RPC req@0000000075189f40 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_000:lustre-MDT0001-lwp-OST0001_UUID+5:3356:x1862456998048960:12345-0@lo:400:kworker/u8:2.0 00000100:00100000:2.0:1776178210.659153:0:6255:0:(service.c:2372:ptlrpc_server_handle_request()) Handled RPC req@0000000075189f40 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_000:lustre-MDT0001-lwp-OST0001_UUID+5:3356:x1862456998048960:12345-0@lo:400:kworker/u8:2.0 Request processed in 33us (459us total) trans 0 rc 0/0 00000100:00100000:2.0:1776178210.659159:0:6255:0:(nrs_fifo.c:241:nrs_fifo_req_stop()) NRS stop fifo request from 12345-0@lo, seq: 923 00000100:00100000:2.0:1776178210.659185:0:3356:0:(client.c:2215:ptlrpc_check_set()) Completed RPC req@0000000065dbeee0 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_03:lustre-MDT0001-lwp-OST0001_UUID:3356:1862456998048960:0@lo:400:kworker/u8:2.0 00000100:00100000:2.0:1776178210.659207:0:3354:0:(client.c:2215:ptlrpc_check_set()) Completed RPC req@000000007d766b2a pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_01:lustre-MDT0001-lwp-OST0000_UUID:3354:1862456998048832:0@lo:400:kworker/u8:2.0 00000100:00100000:2.0:1776178210.659218:0:3354:0:(client.c:2215:ptlrpc_check_set()) Completed RPC req@0000000042de09e9 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_01:lustre-MDT0001-mdtlov_UUID:3354:1862456998049088:0@lo:400:kworker/u8:2.0 00000100:00100000:2.0:1776178210.659227:0:3354:0:(client.c:2215:ptlrpc_check_set()) Completed RPC req@00000000fd2b8392 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_01:lustre-MDT0000-lwp-MDT0000_UUID:3354:1862456998049344:0@lo:400:kworker/u8:2.0 00000100:00100000:2.0:1776178210.659259:0:6260:0:(service.c:2152:ptlrpc_server_handle_req_in()) got req x1862456998049216 00000100:00100000:0.0:1776178210.659264:0:3356:0:(client.c:1733:ptlrpc_send_new_req()) Sending RPC req@0000000023210d1c pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_03:lustre-MDT0000-mdtlov_UUID:3356:1862456998049472:0@lo:13:osp-pre-0-0.0 00000100:00100000:2.0:1776178210.659280:0:6260:0:(service.c:2152:ptlrpc_server_handle_req_in()) got req x1862456998049408 00000100:00100000:0.0:1776178210.659294:0:3356:0:(events.c:361:request_in_callback()) peer: 12345-0@lo (source: 12345-0@lo) 00000100:00100000:2.0:1776178210.659294:0:6260:0:(nrs_fifo.c:179:nrs_fifo_req_get()) NRS start fifo request from 12345-0@lo, seq: 704 00000100:00100000:2.0:1776178210.659297:0:6260:0:(service.c:2324:ptlrpc_server_handle_request()) Handling RPC req@00000000e8195050 pname:cluuid+ref:pid:xid:nid:opc:job mdt_out00_000:lustre-MDT0000-mdtlov_UUID+6:3356:x1862456998049216:12345-0@lo:400:kworker/u8:2.0 00000100:00100000:0.0:1776178210.659319:0:9554:0:(service.c:2152:ptlrpc_server_handle_req_in()) got req x1862456998049472 00000100:00100000:0.0:1776178210.659346:0:9554:0:(nrs_fifo.c:179:nrs_fifo_req_get()) NRS start fifo request from 12345-0@lo, seq: 484 00000100:00100000:0.0:1776178210.659350:0:9554:0:(service.c:2324:ptlrpc_server_handle_request()) Handling RPC req@000000005643ab1b pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_create00:lustre-MDT0000-mdtlov_UUID+5:3356:x1862456998049472:12345-0@lo:13:osp-pre-0-0.0 00002000:00000024:0.0:1776178210.659372:0:9554:0:(ofd_obd.c:740:ofd_statfs()) blocks cached 315392 granted 484883776 pending 0 free 3923689472 avail 3692167168 00000100:00100000:0.0:1776178210.659405:0:9554:0:(service.c:2372:ptlrpc_server_handle_request()) Handled RPC req@000000005643ab1b pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_create00:lustre-MDT0000-mdtlov_UUID+5:3356:x1862456998049472:12345-0@lo:13:osp-pre-0-0.0 Request processed in 54us (111us total) trans 0 rc 0/0 00000100:00100000:0.0:1776178210.659411:0:9554:0:(nrs_fifo.c:241:nrs_fifo_req_stop()) NRS stop fifo request from 12345-0@lo, seq: 484 00000100:00100000:0.0:1776178210.659436:0:3356:0:(client.c:2215:ptlrpc_check_set()) Completed RPC req@0000000023210d1c pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_03:lustre-MDT0000-mdtlov_UUID:3356:1862456998049472:0@lo:13:osp-pre-0-0.0 00000100:00100000:0.0:1776178210.659515:0:14128:0:(nrs_fifo.c:179:nrs_fifo_req_get()) NRS start fifo request from 12345-0@lo, seq: 705 00000100:00100000:0.0:1776178210.659519:0:14128:0:(service.c:2324:ptlrpc_server_handle_request()) Handling RPC req@000000006a3d91b0 pname:cluuid+ref:pid:xid:nid:opc:job mdt_out00_002:lustre-MDT0000-mdtlov_UUID+6:3356:x1862456998049408:12345-0@lo:41:osp-pre-1-0.0 00000004:00000024:0.0:1776178210.659538:0:14128:0:(mdt_handler.c:548:mdt_statfs()) blocks cached 0 granted 2097152 pending 0 free 1446039552 avail 1316577280 00000100:00100000:2.0:1776178210.659646:0:6260:0:(service.c:2372:ptlrpc_server_handle_request()) Handled RPC req@00000000e8195050 pname:cluuid+ref:pid:xid:nid:opc:job mdt_out00_000:lustre-MDT0000-mdtlov_UUID+6:3356:x1862456998049216:12345-0@lo:400:kworker/u8:2.0 Request processed in 346us (916us total) trans 0 rc 0/0 00000100:00100000:2.0:1776178210.659653:0:6260:0:(nrs_fifo.c:241:nrs_fifo_req_stop()) NRS stop fifo request from 12345-0@lo, seq: 704 00000100:00100000:2.0:1776178210.659675:0:3356:0:(client.c:2215:ptlrpc_check_set()) Completed RPC req@000000002300ff10 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_03:lustre-MDT0000-mdtlov_UUID:3356:1862456998049216:0@lo:400:kworker/u8:2.0 00000100:00100000:3.0:1776178210.659675:0:3355:0:(client.c:2215:ptlrpc_check_set()) Completed RPC req@000000000b5ad4fd pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_02:lustre-MDT0000-lwp-OST0001_UUID:3355:1862456998048896:0@lo:400:kworker/u8:2.0 00000100:00100000:3.0:1776178210.659685:0:3355:0:(client.c:2215:ptlrpc_check_set()) Completed RPC req@000000004828c19d pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_02:lustre-MDT0000-lwp-MDT0001_UUID:3355:1862456998049152:0@lo:400:kworker/u8:2.0 00000100:00100000:0.0:1776178210.659912:0:14128:0:(service.c:2372:ptlrpc_server_handle_request()) Handled RPC req@000000006a3d91b0 pname:cluuid+ref:pid:xid:nid:opc:job mdt_out00_002:lustre-MDT0000-mdtlov_UUID+5:3356:x1862456998049408:12345-0@lo:41:osp-pre-1-0.0 Request processed in 392us (1114us total) trans 0 rc 0/0 00000100:00100000:0.0:1776178210.659919:0:14128:0:(nrs_fifo.c:241:nrs_fifo_req_stop()) NRS stop fifo request from 12345-0@lo, seq: 705 00000100:00100000:2.0:1776178210.660127:0:3356:0:(client.c:2215:ptlrpc_check_set()) Completed RPC req@00000000b8f6795f pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_03:lustre-MDT0000-mdtlov_UUID:3356:1862456998049408:0@lo:41:osp-pre-1-0.0 00000100:00100000:1.0:1776178210.660304:0:20755:0:(nrs_fifo.c:179:nrs_fifo_req_get()) NRS start fifo request from 12345-0@lo, seq: 518 00000100:00100000:1.0:1776178210.660308:0:20755:0:(service.c:2324:ptlrpc_server_handle_request()) Handling RPC req@0000000079529182 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_006:lustre-MDT0001-mdtlov_UUID+8:3353:x1862456998049024:12345-0@lo:400:kworker/u8:2.0 00000100:00100000:1.0:1776178210.660351:0:20755:0:(service.c:2372:ptlrpc_server_handle_request()) Handled RPC req@0000000079529182 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_006:lustre-MDT0001-mdtlov_UUID+8:3353:x1862456998049024:12345-0@lo:400:kworker/u8:2.0 Request processed in 42us (1923us total) trans 0 rc 0/0 00000100:00100000:1.0:1776178210.660357:0:20755:0:(nrs_fifo.c:241:nrs_fifo_req_stop()) NRS stop fifo request from 12345-0@lo, seq: 518 00000100:00100000:2.0:1776178210.660645:0:3353:0:(client.c:2215:ptlrpc_check_set()) Completed RPC req@000000009c4a59b3 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_00:lustre-MDT0001-mdtlov_UUID:3353:1862456998049024:0@lo:400:kworker/u8:2.0 00000001:00080000:2.0:1776178210.665223:0:20134:0:(tgt_lastrcvd.c:913:tgt_cb_last_committed()) lustre-MDT0000: transno 12884901946 is committed 00010000:00010000:1.0:1776178211.296297:0:9224:0:(ldlm_lockd.c:2792:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000400:00000080:0.0:1776178211.801380:0:21609:0:(module.c:221:libcfs_ioctl()) libcfs ioctl cmd 3221775648 00000001:02000400:0.0:1776178211.801390:0:21609:0:(debug.c:702:libcfs_debug_mark_buffer()) DEBUG MARKER: == replay-single test 102c: check replay w/o reconstruction with multiple mod RPCs in flight ========================================================== 10:50:09 (1776178209) 00000100:00100000:2.0:1776178211.900242:0:3346:0:(events.c:361:request_in_callback()) peer: 12345-192.168.204.3@tcp (source: 12345-192.168.204.3@tcp) 00000100:00100000:0.0:1776178211.900347:0:6256:0:(service.c:2152:ptlrpc_server_handle_req_in()) got req x1862456979103040 00000100:00100000:0.0:1776178211.900414:0:6256:0:(nrs_fifo.c:179:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.204.3@tcp, seq: 6594 00000100:00100000:0.0:1776178211.900422:0:6256:0:(service.c:2324:ptlrpc_server_handle_request()) Handling RPC req@000000009c4a59b3 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_001:b50963bd-8eba-4432-9a54-8888d00e3fa1+6:18251:x1862456979103040:12345-192.168.204.3@tcp:101:lfs.0 00010000:00010000:0.0:1776178211.900445:0:6256:0:(ldlm_lockd.c:1248:ldlm_handle_enqueue0()) ### server-side enqueue handler START 00010000:00010000:0.0:1776178211.900470:0:6256:0:(ldlm_lockd.c:1318:ldlm_handle_enqueue0()) ### server-side enqueue handler, new lock created ns: mdt-lustre-MDT0000_UUID lock: 000000006b206cc9/0xa40083ab9bf537d3 lrc: 2/0,0 mode: --/CR res: [0x200000007:0x1:0x0].0x0 bits 0x0/0x0 rrc: 4 type: IBT gid 0 flags: 0x40000000000000 nid: local remote: 0x771f68aa1eb27bb9 expref: -99 pid: 6256 timeout: 0 lvb_type: 0 00010000:00010000:0.0:1776178211.900538:0:6256:0:(ldlm_lock.c:766:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(CR) ns: mdt-lustre-MDT0000_UUID lock: 000000004bd1d7cb/0xa40083ab9bf537da lrc: 3/1,0 mode: --/CR res: [0x200000007:0x1:0x0].0x0 bits 0x0/0x0 rrc: 5 type: IBT gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 6256 timeout: 0 lvb_type: 0 00010000:00010000:0.0:1776178211.900568:0:6256:0:(ldlm_lock.c:1064:ldlm_granted_list_add_lock()) ### About to add lock: ns: mdt-lustre-MDT0000_UUID lock: 000000004bd1d7cb/0xa40083ab9bf537da lrc: 3/1,0 mode: CR/CR res: [0x200000007:0x1:0x0].0x0 bits 0x2/0x0 rrc: 5 type: IBT gid 0 flags: 0x50210000000000 nid: local remote: 0x0 expref: -99 pid: 6256 timeout: 0 lvb_type: 0 00010000:00010000:0.0:1776178211.900584:0:6256:0:(ldlm_request.c:528:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: mdt-lustre-MDT0000_UUID lock: 000000004bd1d7cb/0xa40083ab9bf537da lrc: 3/1,0 mode: CR/CR res: [0x200000007:0x1:0x0].0x0 bits 0x2/0x0 rrc: 5 type: IBT gid 0 flags: 0x40210000000000 nid: local remote: 0x0 expref: -99 pid: 6256 timeout: 0 lvb_type: 0 00010000:00010000:0.0:1776178211.900599:0:6256:0:(ldlm_lock.c:766:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PR) ns: mdt-lustre-MDT0000_UUID lock: 0000000095e53696/0xa40083ab9bf537e1 lrc: 3/1,0 mode: --/PR res: [0x200000007:0x1:0x0].0x1317f023 bits 0x0/0x0 rrc: 2 type: IBT gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 6256 timeout: 0 lvb_type: 0 00010000:00010000:0.0:1776178211.900605:0:6256:0:(ldlm_lock.c:1064:ldlm_granted_list_add_lock()) ### About to add lock: ns: mdt-lustre-MDT0000_UUID lock: 0000000095e53696/0xa40083ab9bf537e1 lrc: 3/1,0 mode: PR/PR res: [0x200000007:0x1:0x0].0x1317f023 bits 0x2/0x0 rrc: 2 type: IBT gid 0 flags: 0x50210000000000 nid: local remote: 0x0 expref: -99 pid: 6256 timeout: 0 lvb_type: 0 00010000:00010000:0.0:1776178211.900612:0:6256:0:(ldlm_request.c:528:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: mdt-lustre-MDT0000_UUID lock: 0000000095e53696/0xa40083ab9bf537e1 lrc: 3/1,0 mode: PR/PR res: [0x200000007:0x1:0x0].0x1317f023 bits 0x2/0x0 rrc: 2 type: IBT gid 0 flags: 0x40210000000000 nid: local remote: 0x0 expref: -99 pid: 6256 timeout: 0 lvb_type: 0 00010000:00010000:0.0:1776178211.900657:0:6256:0:(ldlm_lock.c:820:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(CR) ns: mdt-lustre-MDT0000_UUID lock: 000000004bd1d7cb/0xa40083ab9bf537da lrc: 3/1,0 mode: CR/CR res: [0x200000007:0x1:0x0].0x0 bits 0x2/0x0 rrc: 5 type: IBT gid 0 flags: 0x50210000000000 nid: local remote: 0x0 expref: -99 pid: 6256 timeout: 0 lvb_type: 0 00010000:00010000:0.0:1776178211.900665:0:6256:0:(ldlm_lock.c:879:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: mdt-lustre-MDT0000_UUID lock: 000000004bd1d7cb/0xa40083ab9bf537da lrc: 2/0,0 mode: CR/CR res: [0x200000007:0x1:0x0].0x0 bits 0x2/0x0 rrc: 5 type: IBT gid 0 flags: 0x50210400000000 nid: local remote: 0x0 expref: -99 pid: 6256 timeout: 0 lvb_type: 0 00010000:00010000:0.0:1776178211.900671:0:6256:0:(ldlm_lockd.c:1879:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: mdt-lustre-MDT0000_UUID lock: 000000004bd1d7cb/0xa40083ab9bf537da lrc: 3/0,0 mode: CR/CR res: [0x200000007:0x1:0x0].0x0 bits 0x2/0x0 rrc: 5 type: IBT gid 0 flags: 0x40210400000000 nid: local remote: 0x0 expref: -99 pid: 6256 timeout: 0 lvb_type: 0 00010000:00010000:0.0:1776178211.900678:0:6256:0:(ldlm_lockd.c:1893:ldlm_handle_bl_callback()) Lock 000000004bd1d7cb already unused, calling callback (0000000079e78348) 00010000:00010000:0.0:1776178211.900682:0:6256:0:(ldlm_request.c:364:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: mdt-lustre-MDT0000_UUID lock: 000000004bd1d7cb/0xa40083ab9bf537da lrc: 3/0,0 mode: CR/CR res: [0x200000007:0x1:0x0].0x0 bits 0x2/0x0 rrc: 5 type: IBT gid 0 flags: 0x40210400000000 nid: local remote: 0x0 expref: -99 pid: 6256 timeout: 0 lvb_type: 0 00010000:00010000:0.0:1776178211.900689:0:6256:0:(ldlm_request.c:1257:ldlm_cli_cancel_local()) ### server-side local cancel ns: mdt-lustre-MDT0000_UUID lock: 000000004bd1d7cb/0xa40083ab9bf537da lrc: 4/0,0 mode: CR/CR res: [0x200000007:0x1:0x0].0x0 bits 0x2/0x0 rrc: 5 type: IBT gid 0 flags: 0x40218400000000 nid: local remote: 0x0 expref: -99 pid: 6256 timeout: 0 lvb_type: 0 00010000:00010000:0.0:1776178211.900701:0:6256:0:(ldlm_inodebits.c:94:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x200000007:0x1:0x0].0x0 (00000000ad0d4094) 00010000:00010000:0.0:1776178211.900704:0:6256:0:(ldlm_inodebits.c:96:ldlm_reprocess_inodebits_queue()) Hint 2 00010000:00010000:0.0:1776178211.900708:0:6256:0:(ldlm_lockd.c:1903:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: mdt-lustre-MDT0000_UUID lock: 000000004bd1d7cb/0xa40083ab9bf537da lrc: 2/0,0 mode: --/CR res: [0x200000007:0x1:0x0].0x0 bits 0x2/0x0 rrc: 5 type: IBT gid 0 flags: 0x44a19400000000 nid: local remote: 0x0 expref: -99 pid: 6256 timeout: 0 lvb_type: 0 00010000:00010000:0.0:1776178211.900714:0:6256:0:(ldlm_lock.c:216:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: mdt-lustre-MDT0000_UUID lock: 000000004bd1d7cb/0xa40083ab9bf537da lrc: 0/0,0 mode: --/CR res: [0x200000007:0x1:0x0].0x0 bits 0x2/0x0 rrc: 5 type: IBT gid 0 flags: 0x44a19400000000 nid: local remote: 0x0 expref: -99 pid: 6256 timeout: 0 lvb_type: 0 00010000:00010000:0.0:1776178211.900728:0:6256:0:(ldlm_lock.c:820:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PR) ns: mdt-lustre-MDT0000_UUID lock: 0000000095e53696/0xa40083ab9bf537e1 lrc: 3/1,0 mode: PR/PR res: [0x200000007:0x1:0x0].0x1317f023 bits 0x2/0x0 rrc: 2 type: IBT gid 0 flags: 0x50210000000000 nid: local remote: 0x0 expref: -99 pid: 6256 timeout: 0 lvb_type: 0 00010000:00010000:0.0:1776178211.900734:0:6256:0:(ldlm_lock.c:879:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: mdt-lustre-MDT0000_UUID lock: 0000000095e53696/0xa40083ab9bf537e1 lrc: 2/0,0 mode: PR/PR res: [0x200000007:0x1:0x0].0x1317f023 bits 0x2/0x0 rrc: 2 type: IBT gid 0 flags: 0x50210400000000 nid: local remote: 0x0 expref: -99 pid: 6256 timeout: 0 lvb_type: 0 00010000:00010000:0.0:1776178211.900739:0:6256:0:(ldlm_lockd.c:1879:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: mdt-lustre-MDT0000_UUID lock: 0000000095e53696/0xa40083ab9bf537e1 lrc: 3/0,0 mode: PR/PR res: [0x200000007:0x1:0x0].0x1317f023 bits 0x2/0x0 rrc: 2 type: IBT gid 0 flags: 0x40210400000000 nid: local remote: 0x0 expref: -99 pid: 6256 timeout: 0 lvb_type: 0 00010000:00010000:0.0:1776178211.900745:0:6256:0:(ldlm_lockd.c:1893:ldlm_handle_bl_callback()) Lock 0000000095e53696 already unused, calling callback (0000000079e78348) 00010000:00010000:0.0:1776178211.900747:0:6256:0:(ldlm_request.c:364:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: mdt-lustre-MDT0000_UUID lock: 0000000095e53696/0xa40083ab9bf537e1 lrc: 3/0,0 mode: PR/PR res: [0x200000007:0x1:0x0].0x1317f023 bits 0x2/0x0 rrc: 2 type: IBT gid 0 flags: 0x40210400000000 nid: local remote: 0x0 expref: -99 pid: 6256 timeout: 0 lvb_type: 0 00010000:00010000:0.0:1776178211.900752:0:6256:0:(ldlm_request.c:1257:ldlm_cli_cancel_local()) ### server-side local cancel ns: mdt-lustre-MDT0000_UUID lock: 0000000095e53696/0xa40083ab9bf537e1 lrc: 4/0,0 mode: PR/PR res: [0x200000007:0x1:0x0].0x1317f023 bits 0x2/0x0 rrc: 2 type: IBT gid 0 flags: 0x40218400000000 nid: local remote: 0x0 expref: -99 pid: 6256 timeout: 0 lvb_type: 0 00010000:00010000:0.0:1776178211.900759:0:6256:0:(ldlm_inodebits.c:94:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x200000007:0x1:0x0].0x1317f023 (00000000b86c95e3) 00010000:00010000:0.0:1776178211.900761:0:6256:0:(ldlm_inodebits.c:96:ldlm_reprocess_inodebits_queue()) Hint 2 00010000:00010000:0.0:1776178211.900763:0:6256:0:(ldlm_lockd.c:1903:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: mdt-lustre-MDT0000_UUID lock: 0000000095e53696/0xa40083ab9bf537e1 lrc: 2/0,0 mode: --/PR res: [0x200000007:0x1:0x0].0x1317f023 bits 0x2/0x0 rrc: 2 type: IBT gid 0 flags: 0x44a19400000000 nid: local remote: 0x0 expref: -99 pid: 6256 timeout: 0 lvb_type: 0 00010000:00010000:0.0:1776178211.900768:0:6256:0:(ldlm_lock.c:216:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: mdt-lustre-MDT0000_UUID lock: 0000000095e53696/0xa40083ab9bf537e1 lrc: 0/0,0 mode: --/PR res: [0x200000007:0x1:0x0].0x1317f023 bits 0x2/0x0 rrc: 2 type: IBT gid 0 flags: 0x44a19400000000 nid: local remote: 0x0 expref: -99 pid: 6256 timeout: 0 lvb_type: 0 00010000:00010000:0.0:1776178211.900800:0:6256:0:(ldlm_lockd.c:1484:ldlm_handle_enqueue0()) ### server-side enqueue handler, sending reply (err=301, rc=0) ns: mdt-lustre-MDT0000_UUID lock: 000000006b206cc9/0xa40083ab9bf537d3 lrc: 1/0,0 mode: --/CR res: [0x200000007:0x1:0x0].0x0 bits 0x2/0x0 rrc: 4 type: IBT gid 0 flags: 0x44000000000000 nid: 192.168.204.3@tcp remote: 0x771f68aa1eb27bb9 expref: 7 pid: 6256 timeout: 0 lvb_type: 0 00010000:00010000:0.0:1776178211.900808:0:6256:0:(ldlm_lock.c:216:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: mdt-lustre-MDT0000_UUID lock: 000000006b206cc9/0xa40083ab9bf537d3 lrc: 0/0,0 mode: --/CR res: [0x200000007:0x1:0x0].0x0 bits 0x2/0x0 rrc: 4 type: IBT gid 0 flags: 0x44000000000000 nid: 192.168.204.3@tcp remote: 0x771f68aa1eb27bb9 expref: 7 pid: 6256 timeout: 0 lvb_type: 0 00010000:00010000:0.0:1776178211.900817:0:6256:0:(ldlm_lockd.c:1565:ldlm_handle_enqueue0()) ### server-side enqueue handler END (lock 000000006b206cc9, rc 0) 00000100:00100000:0.0:1776178211.900895:0:6256:0:(service.c:2372:ptlrpc_server_handle_request()) Handled RPC req@000000009c4a59b3 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_001:b50963bd-8eba-4432-9a54-8888d00e3fa1+6:18251:x1862456979103040:12345-192.168.204.3@tcp:101:lfs.0 Request processed in 474us (662us total) trans 0 rc 301/301 00000100:00100000:0.0:1776178211.900904:0:6256:0:(nrs_fifo.c:241:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.204.3@tcp, seq: 6594 00000100:00100000:0.0:1776178211.916981:0:3347:0:(events.c:361:request_in_callback()) peer: 12345-192.168.204.3@tcp (source: 12345-192.168.204.3@tcp) 00000100:00100000:2.0:1776178211.917130:0:11281:0:(service.c:2152:ptlrpc_server_handle_req_in()) got req x1862456979103104 00000100:00100000:2.0:1776178211.917201:0:11281:0:(nrs_fifo.c:179:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.204.3@tcp, seq: 6595 00000100:00100000:2.0:1776178211.917210:0:11281:0:(service.c:2324:ptlrpc_server_handle_request()) Handling RPC req@0000000023210d1c pname:cluuid+ref:pid:xid:nid:opc:job mdt00_004:b50963bd-8eba-4432-9a54-8888d00e3fa1+6:18251:x1862456979103104:12345-192.168.204.3@tcp:101:lfs.0 00010000:00010000:2.0:1776178211.917241:0:11281:0:(ldlm_lockd.c:1248:ldlm_handle_enqueue0()) ### server-side enqueue handler START 00010000:00010000:2.0:1776178211.917268:0:11281:0:(ldlm_lockd.c:1318:ldlm_handle_enqueue0()) ### server-side enqueue handler, new lock created ns: mdt-lustre-MDT0000_UUID lock: 00000000c9d7d9fc/0xa40083ab9bf537e8 lrc: 2/0,0 mode: --/CR res: [0x200000007:0x1:0x0].0x0 bits 0x0/0x0 rrc: 4 type: IBT gid 0 flags: 0x40000000000000 nid: local remote: 0x771f68aa1eb27bc0 expref: -99 pid: 11281 timeout: 0 lvb_type: 0 00010000:00010000:2.0:1776178211.917349:0:11281:0:(ldlm_lock.c:766:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(CR) ns: mdt-lustre-MDT0000_UUID lock: 0000000096c1e297/0xa40083ab9bf537ef lrc: 3/1,0 mode: --/CR res: [0x200000007:0x1:0x0].0x0 bits 0x0/0x0 rrc: 5 type: IBT gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 11281 timeout: 0 lvb_type: 0 00010000:00010000:2.0:1776178211.917365:0:11281:0:(ldlm_lock.c:1064:ldlm_granted_list_add_lock()) ### About to add lock: ns: mdt-lustre-MDT0000_UUID lock: 0000000096c1e297/0xa40083ab9bf537ef lrc: 3/1,0 mode: CR/CR res: [0x200000007:0x1:0x0].0x0 bits 0x2/0x0 rrc: 5 type: IBT gid 0 flags: 0x50210000000000 nid: local remote: 0x0 expref: -99 pid: 11281 timeout: 0 lvb_type: 0 00010000:00010000:2.0:1776178211.917378:0:11281:0:(ldlm_request.c:528:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: mdt-lustre-MDT0000_UUID lock: 0000000096c1e297/0xa40083ab9bf537ef lrc: 3/1,0 mode: CR/CR res: [0x200000007:0x1:0x0].0x0 bits 0x2/0x0 rrc: 5 type: IBT gid 0 flags: 0x40210000000000 nid: local remote: 0x0 expref: -99 pid: 11281 timeout: 0 lvb_type: 0 00010000:00010000:2.0:1776178211.917393:0:11281:0:(ldlm_lock.c:766:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PR) ns: mdt-lustre-MDT0000_UUID lock: 000000000b5c0143/0xa40083ab9bf537f6 lrc: 3/1,0 mode: --/PR res: [0x200000007:0x1:0x0].0x1317f023 bits 0x0/0x0 rrc: 2 type: IBT gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 11281 timeout: 0 lvb_type: 0 00010000:00010000:2.0:1776178211.917399:0:11281:0:(ldlm_lock.c:1064:ldlm_granted_list_add_lock()) ### About to add lock: ns: mdt-lustre-MDT0000_UUID lock: 000000000b5c0143/0xa40083ab9bf537f6 lrc: 3/1,0 mode: PR/PR res: [0x200000007:0x1:0x0].0x1317f023 bits 0x2/0x0 rrc: 2 type: IBT gid 0 flags: 0x50210000000000 nid: local remote: 0x0 expref: -99 pid: 11281 timeout: 0 lvb_type: 0 00010000:00010000:2.0:1776178211.917404:0:11281:0:(ldlm_request.c:528:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: mdt-lustre-MDT0000_UUID lock: 000000000b5c0143/0xa40083ab9bf537f6 lrc: 3/1,0 mode: PR/PR res: [0x200000007:0x1:0x0].0x1317f023 bits 0x2/0x0 rrc: 2 type: IBT gid 0 flags: 0x40210000000000 nid: local remote: 0x0 expref: -99 pid: 11281 timeout: 0 lvb_type: 0 00010000:00010000:2.0:1776178211.917449:0:11281:0:(ldlm_lock.c:820:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(CR) ns: mdt-lustre-MDT0000_UUID lock: 0000000096c1e297/0xa40083ab9bf537ef lrc: 3/1,0 mode: CR/CR res: [0x200000007:0x1:0x0].0x0 bits 0x2/0x0 rrc: 5 type: IBT gid 0 flags: 0x50210000000000 nid: local remote: 0x0 expref: -99 pid: 11281 timeout: 0 lvb_type: 0 00010000:00010000:2.0:1776178211.917456:0:11281:0:(ldlm_lock.c:879:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: mdt-lustre-MDT0000_UUID lock: 0000000096c1e297/0xa40083ab9bf537ef lrc: 2/0,0 mode: CR/CR res: [0x200000007:0x1:0x0].0x0 bits 0x2/0x0 rrc: 5 type: IBT gid 0 flags: 0x50210400000000 nid: local remote: 0x0 expref: -99 pid: 11281 timeout: 0 lvb_type: 0 00010000:00010000:2.0:1776178211.917463:0:11281:0:(ldlm_lockd.c:1879:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: mdt-lustre-MDT0000_UUID lock: 0000000096c1e297/0xa40083ab9bf537ef lrc: 3/0,0 mode: CR/CR res: [0x200000007:0x1:0x0].0x0 bits 0x2/0x0 rrc: 5 type: IBT gid 0 flags: 0x40210400000000 nid: local remote: 0x0 expref: -99 pid: 11281 timeout: 0 lvb_type: 0 00010000:00010000:2.0:1776178211.917474:0:11281:0:(ldlm_lockd.c:1893:ldlm_handle_bl_callback()) Lock 0000000096c1e297 already unused, calling callback (0000000079e78348) 00010000:00010000:2.0:1776178211.917478:0:11281:0:(ldlm_request.c:364:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: mdt-lustre-MDT0000_UUID lock: 0000000096c1e297/0xa40083ab9bf537ef lrc: 3/0,0 mode: CR/CR res: [0x200000007:0x1:0x0].0x0 bits 0x2/0x0 rrc: 5 type: IBT gid 0 flags: 0x40210400000000 nid: local remote: 0x0 expref: -99 pid: 11281 timeout: 0 lvb_type: 0 00010000:00010000:2.0:1776178211.917486:0:11281:0:(ldlm_request.c:1257:ldlm_cli_cancel_local()) ### server-side local cancel ns: mdt-lustre-MDT0000_UUID lock: 0000000096c1e297/0xa40083ab9bf537ef lrc: 4/0,0 mode: CR/CR res: [0x200000007:0x1:0x0].0x0 bits 0x2/0x0 rrc: 5 type: IBT gid 0 flags: 0x40218400000000 nid: local remote: 0x0 expref: -99 pid: 11281 timeout: 0 lvb_type: 0 00010000:00010000:2.0:1776178211.917499:0:11281:0:(ldlm_inodebits.c:94:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x200000007:0x1:0x0].0x0 (00000000ad0d4094) 00010000:00010000:2.0:1776178211.917501:0:11281:0:(ldlm_inodebits.c:96:ldlm_reprocess_inodebits_queue()) Hint 2 00010000:00010000:2.0:1776178211.917507:0:11281:0:(ldlm_lockd.c:1903:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: mdt-lustre-MDT0000_UUID lock: 0000000096c1e297/0xa40083ab9bf537ef lrc: 2/0,0 mode: --/CR res: [0x200000007:0x1:0x0].0x0 bits 0x2/0x0 rrc: 5 type: IBT gid 0 flags: 0x44a19400000000 nid: local remote: 0x0 expref: -99 pid: 11281 timeout: 0 lvb_type: 0 00010000:00010000:2.0:1776178211.917513:0:11281:0:(ldlm_lock.c:216:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: mdt-lustre-MDT0000_UUID lock: 0000000096c1e297/0xa40083ab9bf537ef lrc: 0/0,0 mode: --/CR res: [0x200000007:0x1:0x0].0x0 bits 0x2/0x0 rrc: 5 type: IBT gid 0 flags: 0x44a19400000000 nid: local remote: 0x0 expref: -99 pid: 11281 timeout: 0 lvb_type: 0 00010000:00010000:2.0:1776178211.917526:0:11281:0:(ldlm_lock.c:820:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PR) ns: mdt-lustre-MDT0000_UUID lock: 000000000b5c0143/0xa40083ab9bf537f6 lrc: 3/1,0 mode: PR/PR res: [0x200000007:0x1:0x0].0x1317f023 bits 0x2/0x0 rrc: 2 type: IBT gid 0 flags: 0x50210000000000 nid: local remote: 0x0 expref: -99 pid: 11281 timeout: 0 lvb_type: 0 00010000:00010000:2.0:1776178211.917533:0:11281:0:(ldlm_lock.c:879:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: mdt-lustre-MDT0000_UUID lock: 000000000b5c0143/0xa40083ab9bf537f6 lrc: 2/0,0 mode: PR/PR res: [0x200000007:0x1:0x0].0x1317f023 bits 0x2/0x0 rrc: 2 type: IBT gid 0 flags: 0x50210400000000 nid: local remote: 0x0 expref: -99 pid: 11281 timeout: 0 lvb_type: 0 00010000:00010000:2.0:1776178211.917539:0:11281:0:(ldlm_lockd.c:1879:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: mdt-lustre-MDT0000_UUID lock: 000000000b5c0143/0xa40083ab9bf537f6 lrc: 3/0,0 mode: PR/PR res: [0x200000007:0x1:0x0].0x1317f023 bits 0x2/0x0 rrc: 2 type: IBT gid 0 flags: 0x40210400000000 nid: local remote: 0x0 expref: -99 pid: 11281 timeout: 0 lvb_type: 0 00010000:00010000:2.0:1776178211.917543:0:11281:0:(ldlm_lockd.c:1893:ldlm_handle_bl_callback()) Lock 000000000b5c0143 already unused, calling callback (0000000079e78348) 00010000:00010000:2.0:1776178211.917545:0:11281:0:(ldlm_request.c:364:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: mdt-lustre-MDT0000_UUID lock: 000000000b5c0143/0xa40083ab9bf537f6 lrc: 3/0,0 mode: PR/PR res: [0x200000007:0x1:0x0].0x1317f023 bits 0x2/0x0 rrc: 2 type: IBT gid 0 flags: 0x40210400000000 nid: local remote: 0x0 expref: -99 pid: 11281 timeout: 0 lvb_type: 0 00010000:00010000:2.0:1776178211.917562:0:11281:0:(ldlm_request.c:1257:ldlm_cli_cancel_local()) ### server-side local cancel ns: mdt-lustre-MDT0000_UUID lock: 000000000b5c0143/0xa40083ab9bf537f6 lrc: 4/0,0 mode: PR/PR res: [0x200000007:0x1:0x0].0x1317f023 bits 0x2/0x0 rrc: 2 type: IBT gid 0 flags: 0x40218400000000 nid: local remote: 0x0 expref: -99 pid: 11281 timeout: 0 lvb_type: 0 00010000:00010000:2.0:1776178211.917571:0:11281:0:(ldlm_inodebits.c:94:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x200000007:0x1:0x0].0x1317f023 (00000000024b4e8f) 00010000:00010000:2.0:1776178211.917574:0:11281:0:(ldlm_inodebits.c:96:ldlm_reprocess_inodebits_queue()) Hint 2 00010000:00010000:2.0:1776178211.917576:0:11281:0:(ldlm_lockd.c:1903:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: mdt-lustre-MDT0000_UUID lock: 000000000b5c0143/0xa40083ab9bf537f6 lrc: 2/0,0 mode: --/PR res: [0x200000007:0x1:0x0].0x1317f023 bits 0x2/0x0 rrc: 2 type: IBT gid 0 flags: 0x44a19400000000 nid: local remote: 0x0 expref: -99 pid: 11281 timeout: 0 lvb_type: 0 00010000:00010000:2.0:1776178211.917580:0:11281:0:(ldlm_lock.c:216:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: mdt-lustre-MDT0000_UUID lock: 000000000b5c0143/0xa40083ab9bf537f6 lrc: 0/0,0 mode: --/PR res: [0x200000007:0x1:0x0].0x1317f023 bits 0x2/0x0 rrc: 2 type: IBT gid 0 flags: 0x44a19400000000 nid: local remote: 0x0 expref: -99 pid: 11281 timeout: 0 lvb_type: 0 00010000:00010000:2.0:1776178211.917630:0:11281:0:(ldlm_lockd.c:1484:ldlm_handle_enqueue0()) ### server-side enqueue handler, sending reply (err=301, rc=0) ns: mdt-lustre-MDT0000_UUID lock: 00000000c9d7d9fc/0xa40083ab9bf537e8 lrc: 1/0,0 mode: --/CR res: [0x200000007:0x1:0x0].0x0 bits 0x2/0x0 rrc: 4 type: IBT gid 0 flags: 0x44000000000000 nid: 192.168.204.3@tcp remote: 0x771f68aa1eb27bc0 expref: 7 pid: 11281 timeout: 0 lvb_type: 0 00010000:00010000:2.0:1776178211.917638:0:11281:0:(ldlm_lock.c:216:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: mdt-lustre-MDT0000_UUID lock: 00000000c9d7d9fc/0xa40083ab9bf537e8 lrc: 0/0,0 mode: --/CR res: [0x200000007:0x1:0x0].0x0 bits 0x2/0x0 rrc: 4 type: IBT gid 0 flags: 0x44000000000000 nid: 192.168.204.3@tcp remote: 0x771f68aa1eb27bc0 expref: 7 pid: 11281 timeout: 0 lvb_type: 0 00010000:00010000:2.0:1776178211.917648:0:11281:0:(ldlm_lockd.c:1565:ldlm_handle_enqueue0()) ### server-side enqueue handler END (lock 00000000c9d7d9fc, rc 0) 00000100:00100000:2.0:1776178211.917735:0:11281:0:(service.c:2372:ptlrpc_server_handle_request()) Handled RPC req@0000000023210d1c pname:cluuid+ref:pid:xid:nid:opc:job mdt00_004:b50963bd-8eba-4432-9a54-8888d00e3fa1+6:18251:x1862456979103104:12345-192.168.204.3@tcp:101:lfs.0 Request processed in 525us (768us total) trans 0 rc 301/301 00000100:00100000:2.0:1776178211.917744:0:11281:0:(nrs_fifo.c:241:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.204.3@tcp, seq: 6595 00000100:00100000:1.0:1776178211.927711:0:3348:0:(events.c:361:request_in_callback()) peer: 12345-192.168.204.3@tcp (source: 12345-192.168.204.3@tcp) 00000100:00100000:3.0:1776178211.927904:0:6256:0:(service.c:2152:ptlrpc_server_handle_req_in()) got req x1862456979103168 00000100:00100000:3.0:1776178211.927974:0:6256:0:(nrs_fifo.c:179:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.204.3@tcp, seq: 6596 00000100:00100000:3.0:1776178211.927982:0:6256:0:(service.c:2324:ptlrpc_server_handle_request()) Handling RPC req@00000000d8af564d pname:cluuid+ref:pid:xid:nid:opc:job mdt00_001:b50963bd-8eba-4432-9a54-8888d00e3fa1+6:18251:x1862456979103168:12345-192.168.204.3@tcp:101:lfs.0 00010000:00010000:3.0:1776178211.928016:0:6256:0:(ldlm_lockd.c:1248:ldlm_handle_enqueue0()) ### server-side enqueue handler START 00010000:00010000:3.0:1776178211.928039:0:6256:0:(ldlm_lockd.c:1318:ldlm_handle_enqueue0()) ### server-side enqueue handler, new lock created ns: mdt-lustre-MDT0000_UUID lock: 0000000026fd08a4/0xa40083ab9bf537fd lrc: 2/0,0 mode: --/CR res: [0x200000007:0x1:0x0].0x0 bits 0x0/0x0 rrc: 4 type: IBT gid 0 flags: 0x40000000000000 nid: local remote: 0x771f68aa1eb27bc7 expref: -99 pid: 6256 timeout: 0 lvb_type: 0 00010000:00010000:3.0:1776178211.928138:0:6256:0:(ldlm_lock.c:766:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PR) ns: mdt-lustre-MDT0000_UUID lock: 00000000c3ab77cd/0xa40083ab9bf53804 lrc: 3/1,0 mode: --/PR res: [0x200000007:0x1:0x0].0x0 bits 0x0/0x0 rrc: 5 type: IBT gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 6256 timeout: 0 lvb_type: 0 00010000:00010000:3.0:1776178211.928154:0:6256:0:(ldlm_lock.c:1064:ldlm_granted_list_add_lock()) ### About to add lock: ns: mdt-lustre-MDT0000_UUID lock: 00000000c3ab77cd/0xa40083ab9bf53804 lrc: 3/1,0 mode: PR/PR res: [0x200000007:0x1:0x0].0x0 bits 0x13/0x0 rrc: 5 type: IBT gid 0 flags: 0x50210000000000 nid: local remote: 0x0 expref: -99 pid: 6256 timeout: 0 lvb_type: 0 00010000:00010000:3.0:1776178211.928166:0:6256:0:(ldlm_request.c:528:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: mdt-lustre-MDT0000_UUID lock: 00000000c3ab77cd/0xa40083ab9bf53804 lrc: 3/1,0 mode: PR/PR res: [0x200000007:0x1:0x0].0x0 bits 0x13/0x0 rrc: 5 type: IBT gid 0 flags: 0x40210000000000 nid: local remote: 0x0 expref: -99 pid: 6256 timeout: 0 lvb_type: 0 00000004:00200000:3.0:1776178211.928211:0:6256:0:(mdt_handler.c:912:mdt_pack_attr2body()) [0x200000007:0x1:0x0]: returning size 4096 00000004:00200000:3.0:1776178211.928214:0:6256:0:(mdt_handler.c:1523:mdt_getattr_internal()) dirent count 2 stripe count 1 MDT count 2 00000004:00010000:3.0:1776178211.928235:0:6256:0:(mdt_handler.c:2353:mdt_getattr_name_lock()) ### Returning lock to client ns: mdt-lustre-MDT0000_UUID lock: 00000000c3ab77cd/0xa40083ab9bf53804 lrc: 3/1,0 mode: PR/PR res: [0x200000007:0x1:0x0].0x0 bits 0x13/0x0 rrc: 5 type: IBT gid 0 flags: 0x40210000000000 nid: local remote: 0x0 expref: -99 pid: 6256 timeout: 0 lvb_type: 0 00010000:00010000:3.0:1776178211.928259:0:6256:0:(ldlm_lock.c:216:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: mdt-lustre-MDT0000_UUID lock: 0000000026fd08a4/0xa40083ab9bf537fd lrc: 0/0,0 mode: --/CR res: [0x200000007:0x1:0x0].0x0 bits 0x2/0x0 rrc: 5 type: IBT gid 0 flags: 0x44000000000000 nid: 192.168.204.3@tcp remote: 0x771f68aa1eb27bc7 expref: 8 pid: 6256 timeout: 0 lvb_type: 0 00010000:00010000:3.0:1776178211.928275:0:6256:0:(ldlm_lockd.c:1484:ldlm_handle_enqueue0()) ### server-side enqueue handler, sending reply (err=0, rc=0) ns: mdt-lustre-MDT0000_UUID lock: 00000000c3ab77cd/0xa40083ab9bf53804 lrc: 3/0,0 mode: PR/PR res: [0x200000007:0x1:0x0].0x0 bits 0x13/0x0 rrc: 4 type: IBT gid 0 flags: 0x40200000000000 nid: 192.168.204.3@tcp remote: 0x771f68aa1eb27bc7 expref: 7 pid: 6256 timeout: 0 lvb_type: 0 00010000:00010000:3.0:1776178211.928283:0:6256:0:(ldlm_inodebits.c:94:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x200000007:0x1:0x0].0x0 (00000000ad0d4094) 00010000:00010000:3.0:1776178211.928285:0:6256:0:(ldlm_inodebits.c:96:ldlm_reprocess_inodebits_queue()) Hint 13 00010000:00010000:3.0:1776178211.928289:0:6256:0:(ldlm_lockd.c:1565:ldlm_handle_enqueue0()) ### server-side enqueue handler END (lock 00000000c3ab77cd, rc 0) 00000100:00100000:3.0:1776178211.928348:0:6256:0:(service.c:2372:ptlrpc_server_handle_request()) Handled RPC req@00000000d8af564d pname:cluuid+ref:pid:xid:nid:opc:job mdt00_001:b50963bd-8eba-4432-9a54-8888d00e3fa1+7:18251:x1862456979103168:12345-192.168.204.3@tcp:101:lfs.0 Request processed in 366us (650us total) trans 0 rc 0/0 00000100:00100000:3.0:1776178211.928355:0:6256:0:(nrs_fifo.c:241:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.204.3@tcp, seq: 6596 00000100:00100000:1.0:1776178211.932001:0:3346:0:(events.c:361:request_in_callback()) peer: 12345-192.168.204.3@tcp (source: 12345-192.168.204.3@tcp) 00000100:00100000:1.0:1776178211.932045:0:3346:0:(events.c:361:request_in_callback()) peer: 12345-192.168.204.3@tcp (source: 12345-192.168.204.3@tcp) 00000100:00100000:3.0:1776178211.932159:0:6256:0:(service.c:2152:ptlrpc_server_handle_req_in()) got req x1862456979103232 00000100:00100000:3.0:1776178211.932223:0:6256:0:(service.c:2152:ptlrpc_server_handle_req_in()) got req x1862456979103360 00000100:00100000:3.0:1776178211.932245:0:6256:0:(nrs_fifo.c:179:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.204.3@tcp, seq: 6597 00000100:00100000:3.0:1776178211.932251:0:6256:0:(service.c:2324:ptlrpc_server_handle_request()) Handling RPC req@00000000b8f6795f pname:cluuid+ref:pid:xid:nid:opc:job mdt00_001:b50963bd-8eba-4432-9a54-8888d00e3fa1+8:2272:x1862456979103232:12345-192.168.204.3@tcp:41:lfs.0 00000004:00000024:3.0:1776178211.932304:0:6256:0:(mdt_handler.c:548:mdt_statfs()) blocks cached 0 granted 2097152 pending 0 free 1446125568 avail 1316663296 00000100:00100000:3.0:1776178211.932381:0:6256:0:(service.c:2372:ptlrpc_server_handle_request()) Handled RPC req@00000000b8f6795f pname:cluuid+ref:pid:xid:nid:opc:job mdt00_001:b50963bd-8eba-4432-9a54-8888d00e3fa1+8:2272:x1862456979103232:12345-192.168.204.3@tcp:41:lfs.0 Request processed in 130us (390us total) trans 0 rc 0/0 00000100:00100000:3.0:1776178211.932389:0:6256:0:(nrs_fifo.c:241:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.204.3@tcp, seq: 6597 00000100:00100000:3.0:1776178211.932405:0:6256:0:(nrs_fifo.c:179:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.204.3@tcp, seq: 6598 00000100:00100000:3.0:1776178211.932410:0:6256:0:(service.c:2324:ptlrpc_server_handle_request()) Handling RPC req@00000000a8bae24c pname:cluuid+ref:pid:xid:nid:opc:job mdt00_001:b50963bd-8eba-4432-9a54-8888d00e3fa1+7:18251:x1862456979103360:12345-192.168.204.3@tcp:36:lfs.0 00010000:00010000:3.0:1776178211.932453:0:6256:0:(ldlm_lockd.c:1720:ldlm_request_cancel()) ### server-side cancel handler START: 1 locks, starting at 0 00010000:00010000:3.0:1776178211.932490:0:6256:0:(ldlm_lock.c:216:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: mdt-lustre-MDT0000_UUID lock: 00000000c3ab77cd/0xa40083ab9bf53804 lrc: 0/0,0 mode: --/PR res: [0x200000007:0x1:0x0].0x0 bits 0x13/0x0 rrc: 5 type: IBT gid 0 flags: 0x44a01000000000 nid: 192.168.204.3@tcp remote: 0x771f68aa1eb27bc7 expref: 7 pid: 6256 timeout: 0 lvb_type: 0 00010000:00010000:3.0:1776178211.932510:0:6256:0:(ldlm_inodebits.c:94:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x200000007:0x1:0x0].0x0 (00000000ad0d4094) 00010000:00010000:3.0:1776178211.932515:0:6256:0:(ldlm_lockd.c:1782:ldlm_request_cancel()) ### server-side cancel handler END 00010000:00010000:3.0:1776178211.932596:0:6256:0:(ldlm_lock.c:766:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(CW) ns: mdt-lustre-MDT0000_UUID lock: 000000005d93f094/0xa40083ab9bf5380b lrc: 3/0,1 mode: --/CW res: [0x200000007:0x1:0x0].0x0 bits 0x0/0x0 rrc: 4 type: IBT gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 6256 timeout: 0 lvb_type: 0 00010000:00010000:3.0:1776178211.935696:0:6256:0:(ldlm_lock.c:1064:ldlm_granted_list_add_lock()) ### About to add lock: ns: mdt-lustre-MDT0000_UUID lock: 000000005d93f094/0xa40083ab9bf5380b lrc: 3/0,1 mode: CW/CW res: [0x200000007:0x1:0x0].0x0 bits 0x2/0x0 rrc: 4 type: IBT gid 0 flags: 0x50210000000000 nid: local remote: 0x0 expref: -99 pid: 6256 timeout: 0 lvb_type: 0 00010000:00010000:3.0:1776178211.935728:0:6256:0:(ldlm_request.c:528:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: mdt-lustre-MDT0000_UUID lock: 000000005d93f094/0xa40083ab9bf5380b lrc: 3/0,1 mode: CW/CW res: [0x200000007:0x1:0x0].0x0 bits 0x2/0x0 rrc: 4 type: IBT gid 0 flags: 0x40210000000000 nid: local remote: 0x0 expref: -99 pid: 6256 timeout: 0 lvb_type: 0 00010000:00010000:3.0:1776178211.935754:0:6256:0:(ldlm_lock.c:766:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: mdt-lustre-MDT0000_UUID lock: 0000000039ef0a83/0xa40083ab9bf53812 lrc: 3/0,1 mode: --/PW res: [0x200000007:0x1:0x0].0x1317f023 bits 0x0/0x0 rrc: 2 type: IBT gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 6256 timeout: 0 lvb_type: 0 00010000:00010000:3.0:1776178211.935765:0:6256:0:(ldlm_lock.c:1064:ldlm_granted_list_add_lock()) ### About to add lock: ns: mdt-lustre-MDT0000_UUID lock: 0000000039ef0a83/0xa40083ab9bf53812 lrc: 3/0,1 mode: PW/PW res: [0x200000007:0x1:0x0].0x1317f023 bits 0x2/0x0 rrc: 2 type: IBT gid 0 flags: 0x50210000000000 nid: local remote: 0x0 expref: -99 pid: 6256 timeout: 0 lvb_type: 0 00010000:00010000:3.0:1776178211.935776:0:6256:0:(ldlm_request.c:528:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: mdt-lustre-MDT0000_UUID lock: 0000000039ef0a83/0xa40083ab9bf53812 lrc: 3/0,1 mode: PW/PW res: [0x200000007:0x1:0x0].0x1317f023 bits 0x2/0x0 rrc: 2 type: IBT gid 0 flags: 0x40210000000000 nid: local remote: 0x0 expref: -99 pid: 6256 timeout: 0 lvb_type: 0 00010000:00010000:3.0:1776178211.936308:0:6256:0:(ldlm_lock.c:766:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: mdt-lustre-MDT0000_UUID lock: 0000000067990ad0/0xa40083ab9bf53819 lrc: 3/0,1 mode: --/PW res: [0x2000013a1:0x9:0x0].0x0 bits 0x0/0x0 rrc: 2 type: IBT gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 6256 timeout: 0 lvb_type: 0 00010000:00010000:3.0:1776178211.936318:0:6256:0:(ldlm_lock.c:1064:ldlm_granted_list_add_lock()) ### About to add lock: ns: mdt-lustre-MDT0000_UUID lock: 0000000067990ad0/0xa40083ab9bf53819 lrc: 3/0,1 mode: PW/PW res: [0x2000013a1:0x9:0x0].0x0 bits 0x2/0x0 rrc: 2 type: IBT gid 0 flags: 0x50210000000000 nid: local remote: 0x0 expref: -99 pid: 6256 timeout: 0 lvb_type: 0 00010000:00010000:3.0:1776178211.936325:0:6256:0:(ldlm_request.c:528:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: mdt-lustre-MDT0000_UUID lock: 0000000067990ad0/0xa40083ab9bf53819 lrc: 3/0,1 mode: PW/PW res: [0x2000013a1:0x9:0x0].0x0 bits 0x2/0x0 rrc: 2 type: IBT gid 0 flags: 0x40210000000000 nid: local remote: 0x0 expref: -99 pid: 6256 timeout: 0 lvb_type: 0 00000004:00010000:3.0:1776178211.936334:0:6256:0:(mdt_handler.c:3992:mdt_save_lock()) ### save lock request 00000000a8bae24c reply state 000000001a1bb307 transno 12884901947 ns: mdt-lustre-MDT0000_UUID lock: 0000000067990ad0/0xa40083ab9bf53819 lrc: 3/0,1 mode: PW/PW res: [0x2000013a1:0x9:0x0].0x0 bits 0x2/0x0 rrc: 2 type: IBT gid 0 flags: 0x40210000000000 nid: local remote: 0x0 expref: -99 pid: 6256 timeout: 0 lvb_type: 0 00000004:00200000:3.0:1776178211.936344:0:6256:0:(mdt_handler.c:912:mdt_pack_attr2body()) [0x2000013a1:0x9:0x0]: returning size 4096 00010000:00010000:3.0:1776178211.936348:0:6256:0:(ldlm_lock.c:820:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(CW) ns: mdt-lustre-MDT0000_UUID lock: 000000005d93f094/0xa40083ab9bf5380b lrc: 3/0,1 mode: CW/CW res: [0x200000007:0x1:0x0].0x0 bits 0x2/0x0 rrc: 4 type: IBT gid 0 flags: 0x50210000000000 nid: local remote: 0x0 expref: -99 pid: 6256 timeout: 0 lvb_type: 0 00010000:00010000:3.0:1776178211.936354:0:6256:0:(ldlm_lock.c:879:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: mdt-lustre-MDT0000_UUID lock: 000000005d93f094/0xa40083ab9bf5380b lrc: 2/0,0 mode: CW/CW res: [0x200000007:0x1:0x0].0x0 bits 0x2/0x0 rrc: 4 type: IBT gid 0 flags: 0x50210400000000 nid: local remote: 0x0 expref: -99 pid: 6256 timeout: 0 lvb_type: 0 00010000:00010000:3.0:1776178211.936360:0:6256:0:(ldlm_lockd.c:1879:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: mdt-lustre-MDT0000_UUID lock: 000000005d93f094/0xa40083ab9bf5380b lrc: 3/0,0 mode: CW/CW res: [0x200000007:0x1:0x0].0x0 bits 0x2/0x0 rrc: 4 type: IBT gid 0 flags: 0x40210400000000 nid: local remote: 0x0 expref: -99 pid: 6256 timeout: 0 lvb_type: 0 00010000:00010000:3.0:1776178211.936364:0:6256:0:(ldlm_lockd.c:1893:ldlm_handle_bl_callback()) Lock 000000005d93f094 already unused, calling callback (0000000079e78348) 00010000:00010000:3.0:1776178211.936368:0:6256:0:(ldlm_request.c:364:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: mdt-lustre-MDT0000_UUID lock: 000000005d93f094/0xa40083ab9bf5380b lrc: 3/0,0 mode: CW/CW res: [0x200000007:0x1:0x0].0x0 bits 0x2/0x0 rrc: 4 type: IBT gid 0 flags: 0x40210400000000 nid: local remote: 0x0 expref: -99 pid: 6256 timeout: 0 lvb_type: 0 00010000:00010000:3.0:1776178211.936378:0:6256:0:(ldlm_request.c:1257:ldlm_cli_cancel_local()) ### server-side local cancel ns: mdt-lustre-MDT0000_UUID lock: 000000005d93f094/0xa40083ab9bf5380b lrc: 4/0,0 mode: CW/CW res: [0x200000007:0x1:0x0].0x0 bits 0x2/0x0 rrc: 4 type: IBT gid 0 flags: 0x40218400000000 nid: local remote: 0x0 expref: -99 pid: 6256 timeout: 0 lvb_type: 0 00010000:00010000:3.0:1776178211.936388:0:6256:0:(ldlm_inodebits.c:94:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x200000007:0x1:0x0].0x0 (00000000ad0d4094) 00010000:00010000:3.0:1776178211.936390:0:6256:0:(ldlm_inodebits.c:96:ldlm_reprocess_inodebits_queue()) Hint 2 00010000:00010000:3.0:1776178211.936393:0:6256:0:(ldlm_lockd.c:1903:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: mdt-lustre-MDT0000_UUID lock: 000000005d93f094/0xa40083ab9bf5380b lrc: 2/0,0 mode: --/CW res: [0x200000007:0x1:0x0].0x0 bits 0x2/0x0 rrc: 4 type: IBT gid 0 flags: 0x44a19400000000 nid: local remote: 0x0 expref: -99 pid: 6256 timeout: 0 lvb_type: 0 00010000:00010000:3.0:1776178211.936398:0:6256:0:(ldlm_lock.c:216:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: mdt-lustre-MDT0000_UUID lock: 000000005d93f094/0xa40083ab9bf5380b lrc: 0/0,0 mode: --/CW res: [0x200000007:0x1:0x0].0x0 bits 0x2/0x0 rrc: 4 type: IBT gid 0 flags: 0x44a19400000000 nid: local remote: 0x0 expref: -99 pid: 6256 timeout: 0 lvb_type: 0 00000004:00010000:3.0:1776178211.936407:0:6256:0:(mdt_handler.c:3992:mdt_save_lock()) ### save lock request 00000000a8bae24c reply state 000000001a1bb307 transno 12884901947 ns: mdt-lustre-MDT0000_UUID lock: 0000000039ef0a83/0xa40083ab9bf53812 lrc: 3/0,1 mode: PW/PW res: [0x200000007:0x1:0x0].0x1317f023 bits 0x2/0x0 rrc: 2 type: IBT gid 0 flags: 0x40210000000000 nid: local remote: 0x0 expref: -99 pid: 6256 timeout: 0 lvb_type: 0 00000100:00100000:3.0:1776178211.936517:0:6256:0:(service.c:2372:ptlrpc_server_handle_request()) Handled RPC req@00000000a8bae24c pname:cluuid+ref:pid:xid:nid:opc:job mdt00_001:b50963bd-8eba-4432-9a54-8888d00e3fa1+8:18251:x1862456979103360:12345-192.168.204.3@tcp:36:lfs.0 Request processed in 4102us (4468us total) trans 12884901947 rc 0/0 00000100:00100000:3.0:1776178211.936527:0:6256:0:(nrs_fifo.c:241:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.204.3@tcp, seq: 6598 00010000:00010000:3.0:1776178211.940637:0:3340:0:(ldlm_lock.c:1064:ldlm_granted_list_add_lock()) ### About to add lock: ns: mdt-lustre-MDT0000_UUID lock: 0000000039ef0a83/0xa40083ab9bf53812 lrc: 3/0,1 mode: COS/COS res: [0x200000007:0x1:0x0].0x1317f023 bits 0x2/0x0 rrc: 2 type: IBT gid 0 flags: 0x50210000000000 nid: local remote: 0x0 expref: -99 pid: 6256 timeout: 0 lvb_type: 0 00010000:00010000:3.0:1776178211.940666:0:3340:0:(ldlm_inodebits.c:94:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x200000007:0x1:0x0].0x1317f023 (00000000ec624a45) 00010000:00010000:3.0:1776178211.940669:0:3340:0:(ldlm_inodebits.c:96:ldlm_reprocess_inodebits_queue()) Hint 2 00010000:00010000:3.0:1776178211.940677:0:3340:0:(ldlm_lock.c:1064:ldlm_granted_list_add_lock()) ### About to add lock: ns: mdt-lustre-MDT0000_UUID lock: 0000000067990ad0/0xa40083ab9bf53819 lrc: 3/0,1 mode: COS/COS res: [0x2000013a1:0x9:0x0].0x0 bits 0x2/0x0 rrc: 2 type: IBT gid 0 flags: 0x50210000000000 nid: local remote: 0x0 expref: -99 pid: 6256 timeout: 0 lvb_type: 0 00010000:00010000:3.0:1776178211.940683:0:3340:0:(ldlm_inodebits.c:94:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x2000013a1:0x9:0x0].0x0 (00000000a041a513) 00010000:00010000:3.0:1776178211.940684:0:3340:0:(ldlm_inodebits.c:96:ldlm_reprocess_inodebits_queue()) Hint 2 00000100:00100000:1.0:1776178211.967291:0:3348:0:(events.c:361:request_in_callback()) peer: 12345-192.168.204.3@tcp (source: 12345-192.168.204.3@tcp) 00000100:00100000:3.0:1776178211.969430:0:6256:0:(service.c:2152:ptlrpc_server_handle_req_in()) got req x1862456979103488 00000100:00100000:3.0:1776178211.969580:0:6256:0:(nrs_fifo.c:179:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.204.3@tcp, seq: 6599 00000100:00100000:3.0:1776178211.969587:0:6256:0:(service.c:2324:ptlrpc_server_handle_request()) Handling RPC req@000000009d7a00b9 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_001:b50963bd-8eba-4432-9a54-8888d00e3fa1+8:18254:x1862456979103488:12345-192.168.204.3@tcp:101:lfs.0 00010000:00010000:3.0:1776178211.970742:0:6256:0:(ldlm_lockd.c:1248:ldlm_handle_enqueue0()) ### server-side enqueue handler START 00010000:00010000:3.0:1776178211.970776:0:6256:0:(ldlm_lockd.c:1318:ldlm_handle_enqueue0()) ### server-side enqueue handler, new lock created ns: mdt-lustre-MDT0000_UUID lock: 000000005d93f094/0xa40083ab9bf53820 lrc: 2/0,0 mode: --/CR res: [0x200000007:0x1:0x0].0x0 bits 0x0/0x0 rrc: 4 type: IBT gid 0 flags: 0x40000000000000 nid: local remote: 0x771f68aa1eb27bce expref: -99 pid: 6256 timeout: 0 lvb_type: 0 00010000:00010000:3.0:1776178211.971091:0:6256:0:(ldlm_lock.c:766:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(CR) ns: mdt-lustre-MDT0000_UUID lock: 00000000c3ab77cd/0xa40083ab9bf53827 lrc: 3/1,0 mode: --/CR res: [0x200000007:0x1:0x0].0x0 bits 0x0/0x0 rrc: 5 type: IBT gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 6256 timeout: 0 lvb_type: 0 00010000:00010000:3.0:1776178211.971109:0:6256:0:(ldlm_lock.c:1064:ldlm_granted_list_add_lock()) ### About to add lock: ns: mdt-lustre-MDT0000_UUID lock: 00000000c3ab77cd/0xa40083ab9bf53827 lrc: 3/1,0 mode: CR/CR res: [0x200000007:0x1:0x0].0x0 bits 0x2/0x0 rrc: 5 type: IBT gid 0 flags: 0x50210000000000 nid: local remote: 0x0 expref: -99 pid: 6256 timeout: 0 lvb_type: 0 00010000:00010000:3.0:1776178211.971125:0:6256:0:(ldlm_request.c:528:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: mdt-lustre-MDT0000_UUID lock: 00000000c3ab77cd/0xa40083ab9bf53827 lrc: 3/1,0 mode: CR/CR res: [0x200000007:0x1:0x0].0x0 bits 0x2/0x0 rrc: 5 type: IBT gid 0 flags: 0x40210000000000 nid: local remote: 0x0 expref: -99 pid: 6256 timeout: 0 lvb_type: 0 00010000:00010000:3.0:1776178211.971138:0:6256:0:(ldlm_lock.c:766:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PR) ns: mdt-lustre-MDT0000_UUID lock: 0000000026fd08a4/0xa40083ab9bf5382e lrc: 3/1,0 mode: --/PR res: [0x200000007:0x1:0x0].0x1317f023 bits 0x0/0x0 rrc: 3 type: IBT gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 6256 timeout: 0 lvb_type: 0 00010000:00010000:3.0:1776178211.971145:0:6256:0:(ldlm_lock.c:1064:ldlm_granted_list_add_lock()) ### About to add lock: ns: mdt-lustre-MDT0000_UUID lock: 0000000026fd08a4/0xa40083ab9bf5382e lrc: 3/1,0 mode: PR/PR res: [0x200000007:0x1:0x0].0x1317f023 bits 0x2/0x0 rrc: 3 type: IBT gid 0 flags: 0x50210000000000 nid: local remote: 0x0 expref: -99 pid: 6256 timeout: 0 lvb_type: 0 00010000:00010000:3.0:1776178211.971151:0:6256:0:(ldlm_request.c:528:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: mdt-lustre-MDT0000_UUID lock: 0000000026fd08a4/0xa40083ab9bf5382e lrc: 3/1,0 mode: PR/PR res: [0x200000007:0x1:0x0].0x1317f023 bits 0x2/0x0 rrc: 3 type: IBT gid 0 flags: 0x40210000000000 nid: local remote: 0x0 expref: -99 pid: 6256 timeout: 0 lvb_type: 0 00000004:00200000:3.0:1776178211.971262:0:6256:0:(mdt_handler.c:912:mdt_pack_attr2body()) [0x2000013a1:0x9:0x0]: returning size 4096 00010000:00010000:3.0:1776178211.971352:0:6256:0:(ldlm_lock.c:820:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(CR) ns: mdt-lustre-MDT0000_UUID lock: 00000000c3ab77cd/0xa40083ab9bf53827 lrc: 3/1,0 mode: CR/CR res: [0x200000007:0x1:0x0].0x0 bits 0x2/0x0 rrc: 5 type: IBT gid 0 flags: 0x50210000000000 nid: local remote: 0x0 expref: -99 pid: 6256 timeout: 0 lvb_type: 0 00010000:00010000:3.0:1776178211.971360:0:6256:0:(ldlm_lock.c:879:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: mdt-lustre-MDT0000_UUID lock: 00000000c3ab77cd/0xa40083ab9bf53827 lrc: 2/0,0 mode: CR/CR res: [0x200000007:0x1:0x0].0x0 bits 0x2/0x0 rrc: 5 type: IBT gid 0 flags: 0x50210400000000 nid: local remote: 0x0 expref: -99 pid: 6256 timeout: 0 lvb_type: 0 00010000:00010000:3.0:1776178211.971373:0:6256:0:(ldlm_lockd.c:1879:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: mdt-lustre-MDT0000_UUID lock: 00000000c3ab77cd/0xa40083ab9bf53827 lrc: 3/0,0 mode: CR/CR res: [0x200000007:0x1:0x0].0x0 bits 0x2/0x0 rrc: 5 type: IBT gid 0 flags: 0x40210400000000 nid: local remote: 0x0 expref: -99 pid: 6256 timeout: 0 lvb_type: 0 00010000:00010000:3.0:1776178211.971379:0:6256:0:(ldlm_lockd.c:1893:ldlm_handle_bl_callback()) Lock 00000000c3ab77cd already unused, calling callback (0000000079e78348) 00010000:00010000:3.0:1776178211.971383:0:6256:0:(ldlm_request.c:364:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: mdt-lustre-MDT0000_UUID lock: 00000000c3ab77cd/0xa40083ab9bf53827 lrc: 3/0,0 mode: CR/CR res: [0x200000007:0x1:0x0].0x0 bits 0x2/0x0 rrc: 5 type: IBT gid 0 flags: 0x40210400000000 nid: local remote: 0x0 expref: -99 pid: 6256 timeout: 0 lvb_type: 0 00010000:00010000:3.0:1776178211.971390:0:6256:0:(ldlm_request.c:1257:ldlm_cli_cancel_local()) ### server-side local cancel ns: mdt-lustre-MDT0000_UUID lock: 00000000c3ab77cd/0xa40083ab9bf53827 lrc: 4/0,0 mode: CR/CR res: [0x200000007:0x1:0x0].0x0 bits 0x2/0x0 rrc: 5 type: IBT gid 0 flags: 0x40218400000000 nid: local remote: 0x0 expref: -99 pid: 6256 timeout: 0 lvb_type: 0 00010000:00010000:3.0:1776178211.971401:0:6256:0:(ldlm_inodebits.c:94:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x200000007:0x1:0x0].0x0 (00000000ad0d4094) 00010000:00010000:3.0:1776178211.971404:0:6256:0:(ldlm_inodebits.c:96:ldlm_reprocess_inodebits_queue()) Hint 2 00010000:00010000:3.0:1776178211.971407:0:6256:0:(ldlm_lockd.c:1903:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: mdt-lustre-MDT0000_UUID lock: 00000000c3ab77cd/0xa40083ab9bf53827 lrc: 2/0,0 mode: --/CR res: [0x200000007:0x1:0x0].0x0 bits 0x2/0x0 rrc: 5 type: IBT gid 0 flags: 0x44a19400000000 nid: local remote: 0x0 expref: -99 pid: 6256 timeout: 0 lvb_type: 0 00010000:00010000:3.0:1776178211.971413:0:6256:0:(ldlm_lock.c:216:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: mdt-lustre-MDT0000_UUID lock: 00000000c3ab77cd/0xa40083ab9bf53827 lrc: 0/0,0 mode: --/CR res: [0x200000007:0x1:0x0].0x0 bits 0x2/0x0 rrc: 5 type: IBT gid 0 flags: 0x44a19400000000 nid: local remote: 0x0 expref: -99 pid: 6256 timeout: 0 lvb_type: 0 00010000:00010000:3.0:1776178211.971423:0:6256:0:(ldlm_lock.c:820:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PR) ns: mdt-lustre-MDT0000_UUID lock: 0000000026fd08a4/0xa40083ab9bf5382e lrc: 3/1,0 mode: PR/PR res: [0x200000007:0x1:0x0].0x1317f023 bits 0x2/0x0 rrc: 3 type: IBT gid 0 flags: 0x50210000000000 nid: local remote: 0x0 expref: -99 pid: 6256 timeout: 0 lvb_type: 0 00010000:00010000:3.0:1776178211.971428:0:6256:0:(ldlm_lock.c:879:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: mdt-lustre-MDT0000_UUID lock: 0000000026fd08a4/0xa40083ab9bf5382e lrc: 2/0,0 mode: PR/PR res: [0x200000007:0x1:0x0].0x1317f023 bits 0x2/0x0 rrc: 3 type: IBT gid 0 flags: 0x50210400000000 nid: local remote: 0x0 expref: -99 pid: 6256 timeout: 0 lvb_type: 0 00010000:00010000:3.0:1776178211.971433:0:6256:0:(ldlm_lockd.c:1879:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: mdt-lustre-MDT0000_UUID lock: 0000000026fd08a4/0xa40083ab9bf5382e lrc: 3/0,0 mode: PR/PR res: [0x200000007:0x1:0x0].0x1317f023 bits 0x2/0x0 rrc: 3 type: IBT gid 0 flags: 0x40210400000000 nid: local remote: 0x0 expref: -99 pid: 6256 timeout: 0 lvb_type: 0 00010000:00010000:3.0:1776178211.971438:0:6256:0:(ldlm_lockd.c:1893:ldlm_handle_bl_callback()) Lock 0000000026fd08a4 already unused, calling callback (0000000079e78348) 00010000:00010000:3.0:1776178211.971440:0:6256:0:(ldlm_request.c:364:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: mdt-lustre-MDT0000_UUID lock: 0000000026fd08a4/0xa40083ab9bf5382e lrc: 3/0,0 mode: PR/PR res: [0x200000007:0x1:0x0].0x1317f023 bits 0x2/0x0 rrc: 3 type: IBT gid 0 flags: 0x40210400000000 nid: local remote: 0x0 expref: -99 pid: 6256 timeout: 0 lvb_type: 0 00010000:00010000:3.0:1776178211.971446:0:6256:0:(ldlm_request.c:1257:ldlm_cli_cancel_local()) ### server-side local cancel ns: mdt-lustre-MDT0000_UUID lock: 0000000026fd08a4/0xa40083ab9bf5382e lrc: 4/0,0 mode: PR/PR res: [0x200000007:0x1:0x0].0x1317f023 bits 0x2/0x0 rrc: 3 type: IBT gid 0 flags: 0x40218400000000 nid: local remote: 0x0 expref: -99 pid: 6256 timeout: 0 lvb_type: 0 00010000:00010000:3.0:1776178211.971451:0:6256:0:(ldlm_inodebits.c:94:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x200000007:0x1:0x0].0x1317f023 (00000000ec624a45) 00010000:00010000:3.0:1776178211.971453:0:6256:0:(ldlm_inodebits.c:96:ldlm_reprocess_inodebits_queue()) Hint 2 00010000:00010000:3.0:1776178211.971455:0:6256:0:(ldlm_lockd.c:1903:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: mdt-lustre-MDT0000_UUID lock: 0000000026fd08a4/0xa40083ab9bf5382e lrc: 2/0,0 mode: --/PR res: [0x200000007:0x1:0x0].0x1317f023 bits 0x2/0x0 rrc: 3 type: IBT gid 0 flags: 0x44a19400000000 nid: local remote: 0x0 expref: -99 pid: 6256 timeout: 0 lvb_type: 0 00010000:00010000:3.0:1776178211.971459:0:6256:0:(ldlm_lock.c:216:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: mdt-lustre-MDT0000_UUID lock: 0000000026fd08a4/0xa40083ab9bf5382e lrc: 0/0,0 mode: --/PR res: [0x200000007:0x1:0x0].0x1317f023 bits 0x2/0x0 rrc: 3 type: IBT gid 0 flags: 0x44a19400000000 nid: local remote: 0x0 expref: -99 pid: 6256 timeout: 0 lvb_type: 0 00010000:00010000:3.0:1776178211.971486:0:6256:0:(ldlm_lockd.c:1484:ldlm_handle_enqueue0()) ### server-side enqueue handler, sending reply (err=301, rc=0) ns: mdt-lustre-MDT0000_UUID lock: 000000005d93f094/0xa40083ab9bf53820 lrc: 1/0,0 mode: --/CR res: [0x200000007:0x1:0x0].0x0 bits 0x1/0x0 rrc: 4 type: IBT gid 0 flags: 0x44000000000000 nid: 192.168.204.3@tcp remote: 0x771f68aa1eb27bce expref: 10 pid: 6256 timeout: 0 lvb_type: 0 00010000:00010000:3.0:1776178211.971493:0:6256:0:(ldlm_lock.c:216:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: mdt-lustre-MDT0000_UUID lock: 000000005d93f094/0xa40083ab9bf53820 lrc: 0/0,0 mode: --/CR res: [0x200000007:0x1:0x0].0x0 bits 0x1/0x0 rrc: 4 type: IBT gid 0 flags: 0x44000000000000 nid: 192.168.204.3@tcp remote: 0x771f68aa1eb27bce expref: 10 pid: 6256 timeout: 0 lvb_type: 0 00010000:00010000:3.0:1776178211.971502:0:6256:0:(ldlm_lockd.c:1565:ldlm_handle_enqueue0()) ### server-side enqueue handler END (lock 000000005d93f094, rc 0) 00000100:00100000:3.0:1776178211.971606:0:6256:0:(service.c:2372:ptlrpc_server_handle_request()) Handled RPC req@000000009d7a00b9 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_001:b50963bd-8eba-4432-9a54-8888d00e3fa1+9:18254:x1862456979103488:12345-192.168.204.3@tcp:101:lfs.0 Request processed in 2016us (4321us total) trans 12884901948 rc 301/301 00000100:00100000:3.0:1776178211.975292:0:6256:0:(nrs_fifo.c:241:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.204.3@tcp, seq: 6599 00000100:00100000:2.0:1776178211.979411:0:3346:0:(events.c:361:request_in_callback()) peer: 12345-192.168.204.3@tcp (source: 12345-192.168.204.3@tcp) 00000100:00100000:3.0:1776178211.983224:0:6256:0:(service.c:2152:ptlrpc_server_handle_req_in()) got req x1862456979103616 00000100:00100000:3.0:1776178211.983306:0:6256:0:(nrs_fifo.c:179:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.204.3@tcp, seq: 6600 00000100:00100000:3.0:1776178211.983313:0:6256:0:(service.c:2324:ptlrpc_server_handle_request()) Handling RPC req@000000002300ff10 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_001:b50963bd-8eba-4432-9a54-8888d00e3fa1+9:18254:x1862456979103616:12345-192.168.204.3@tcp:101:lfs.0 00010000:00010000:3.0:1776178211.983341:0:6256:0:(ldlm_lockd.c:1248:ldlm_handle_enqueue0()) ### server-side enqueue handler START 00010000:00010000:3.0:1776178211.983363:0:6256:0:(ldlm_lockd.c:1318:ldlm_handle_enqueue0()) ### server-side enqueue handler, new lock created ns: mdt-lustre-MDT0000_UUID lock: 00000000a3795b7d/0xa40083ab9bf5383c lrc: 2/0,0 mode: --/CR res: [0x200000007:0x1:0x0].0x0 bits 0x0/0x0 rrc: 4 type: IBT gid 0 flags: 0x40000000000000 nid: local remote: 0x771f68aa1eb27bd5 expref: -99 pid: 6256 timeout: 0 lvb_type: 0 00010000:00010000:3.0:1776178211.983454:0:6256:0:(ldlm_lock.c:766:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(CR) ns: mdt-lustre-MDT0000_UUID lock: 0000000055220ae2/0xa40083ab9bf53843 lrc: 3/1,0 mode: --/CR res: [0x200000007:0x1:0x0].0x0 bits 0x0/0x0 rrc: 5 type: IBT gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 6256 timeout: 0 lvb_type: 0 00010000:00010000:3.0:1776178211.983470:0:6256:0:(ldlm_lock.c:1064:ldlm_granted_list_add_lock()) ### About to add lock: ns: mdt-lustre-MDT0000_UUID lock: 0000000055220ae2/0xa40083ab9bf53843 lrc: 3/1,0 mode: CR/CR res: [0x200000007:0x1:0x0].0x0 bits 0x2/0x0 rrc: 5 type: IBT gid 0 flags: 0x50210000000000 nid: local remote: 0x0 expref: -99 pid: 6256 timeout: 0 lvb_type: 0 00010000:00010000:3.0:1776178211.983483:0:6256:0:(ldlm_request.c:528:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: mdt-lustre-MDT0000_UUID lock: 0000000055220ae2/0xa40083ab9bf53843 lrc: 3/1,0 mode: CR/CR res: [0x200000007:0x1:0x0].0x0 bits 0x2/0x0 rrc: 5 type: IBT gid 0 flags: 0x40210000000000 nid: local remote: 0x0 expref: -99 pid: 6256 timeout: 0 lvb_type: 0 00010000:00010000:3.0:1776178211.983494:0:6256:0:(ldlm_lock.c:766:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PR) ns: mdt-lustre-MDT0000_UUID lock: 000000008192de90/0xa40083ab9bf5384a lrc: 3/1,0 mode: --/PR res: [0x200000007:0x1:0x0].0x1317f023 bits 0x0/0x0 rrc: 3 type: IBT gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 6256 timeout: 0 lvb_type: 0 00010000:00010000:3.0:1776178211.983501:0:6256:0:(ldlm_lock.c:1064:ldlm_granted_list_add_lock()) ### About to add lock: ns: mdt-lustre-MDT0000_UUID lock: 000000008192de90/0xa40083ab9bf5384a lrc: 3/1,0 mode: PR/PR res: [0x200000007:0x1:0x0].0x1317f023 bits 0x2/0x0 rrc: 3 type: IBT gid 0 flags: 0x50210000000000 nid: local remote: 0x0 expref: -99 pid: 6256 timeout: 0 lvb_type: 0 00010000:00010000:3.0:1776178211.983508:0:6256:0:(ldlm_request.c:528:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: mdt-lustre-MDT0000_UUID lock: 000000008192de90/0xa40083ab9bf5384a lrc: 3/1,0 mode: PR/PR res: [0x200000007:0x1:0x0].0x1317f023 bits 0x2/0x0 rrc: 3 type: IBT gid 0 flags: 0x40210000000000 nid: local remote: 0x0 expref: -99 pid: 6256 timeout: 0 lvb_type: 0 00000004:00200000:3.0:1776178211.983635:0:6256:0:(mdt_handler.c:912:mdt_pack_attr2body()) [0x2000013a1:0x9:0x0]: returning size 4096 00010000:00010000:3.0:1776178211.983756:0:6256:0:(ldlm_lock.c:820:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(CR) ns: mdt-lustre-MDT0000_UUID lock: 0000000055220ae2/0xa40083ab9bf53843 lrc: 3/1,0 mode: CR/CR res: [0x200000007:0x1:0x0].0x0 bits 0x2/0x0 rrc: 5 type: IBT gid 0 flags: 0x50210000000000 nid: local remote: 0x0 expref: -99 pid: 6256 timeout: 0 lvb_type: 0 00010000:00010000:3.0:1776178211.983764:0:6256:0:(ldlm_lock.c:879:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: mdt-lustre-MDT0000_UUID lock: 0000000055220ae2/0xa40083ab9bf53843 lrc: 2/0,0 mode: CR/CR res: [0x200000007:0x1:0x0].0x0 bits 0x2/0x0 rrc: 5 type: IBT gid 0 flags: 0x50210400000000 nid: local remote: 0x0 expref: -99 pid: 6256 timeout: 0 lvb_type: 0 00010000:00010000:3.0:1776178211.983771:0:6256:0:(ldlm_lockd.c:1879:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: mdt-lustre-MDT0000_UUID lock: 0000000055220ae2/0xa40083ab9bf53843 lrc: 3/0,0 mode: CR/CR res: [0x200000007:0x1:0x0].0x0 bits 0x2/0x0 rrc: 5 type: IBT gid 0 flags: 0x40210400000000 nid: local remote: 0x0 expref: -99 pid: 6256 timeout: 0 lvb_type: 0 00010000:00010000:3.0:1776178211.983777:0:6256:0:(ldlm_lockd.c:1893:ldlm_handle_bl_callback()) Lock 0000000055220ae2 already unused, calling callback (0000000079e78348) 00010000:00010000:3.0:1776178211.983781:0:6256:0:(ldlm_request.c:364:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: mdt-lustre-MDT0000_UUID lock: 0000000055220ae2/0xa40083ab9bf53843 lrc: 3/0,0 mode: CR/CR res: [0x200000007:0x1:0x0].0x0 bits 0x2/0x0 rrc: 5 type: IBT gid 0 flags: 0x40210400000000 nid: local remote: 0x0 expref: -99 pid: 6256 timeout: 0 lvb_type: 0 00010000:00010000:3.0:1776178211.983789:0:6256:0:(ldlm_request.c:1257:ldlm_cli_cancel_local()) ### server-side local cancel ns: mdt-lustre-MDT0000_UUID lock: 0000000055220ae2/0xa40083ab9bf53843 lrc: 4/0,0 mode: CR/CR res: [0x200000007:0x1:0x0].0x0 bits 0x2/0x0 rrc: 5 type: IBT gid 0 flags: 0x40218400000000 nid: local remote: 0x0 expref: -99 pid: 6256 timeout: 0 lvb_type: 0 00010000:00010000:3.0:1776178211.983802:0:6256:0:(ldlm_inodebits.c:94:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x200000007:0x1:0x0].0x0 (00000000ad0d4094) 00010000:00010000:3.0:1776178211.983805:0:6256:0:(ldlm_inodebits.c:96:ldlm_reprocess_inodebits_queue()) Hint 2 00010000:00010000:3.0:1776178211.983809:0:6256:0:(ldlm_lockd.c:1903:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: mdt-lustre-MDT0000_UUID lock: 0000000055220ae2/0xa40083ab9bf53843 lrc: 2/0,0 mode: --/CR res: [0x200000007:0x1:0x0].0x0 bits 0x2/0x0 rrc: 5 type: IBT gid 0 flags: 0x44a19400000000 nid: local remote: 0x0 expref: -99 pid: 6256 timeout: 0 lvb_type: 0 00010000:00010000:3.0:1776178211.983814:0:6256:0:(ldlm_lock.c:216:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: mdt-lustre-MDT0000_UUID lock: 0000000055220ae2/0xa40083ab9bf53843 lrc: 0/0,0 mode: --/CR res: [0x200000007:0x1:0x0].0x0 bits 0x2/0x0 rrc: 5 type: IBT gid 0 flags: 0x44a19400000000 nid: local remote: 0x0 expref: -99 pid: 6256 timeout: 0 lvb_type: 0 00010000:00010000:3.0:1776178211.983827:0:6256:0:(ldlm_lock.c:820:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PR) ns: mdt-lustre-MDT0000_UUID lock: 000000008192de90/0xa40083ab9bf5384a lrc: 3/1,0 mode: PR/PR res: [0x200000007:0x1:0x0].0x1317f023 bits 0x2/0x0 rrc: 3 type: IBT gid 0 flags: 0x50210000000000 nid: local remote: 0x0 expref: -99 pid: 6256 timeout: 0 lvb_type: 0 00010000:00010000:3.0:1776178211.983832:0:6256:0:(ldlm_lock.c:879:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: mdt-lustre-MDT0000_UUID lock: 000000008192de90/0xa40083ab9bf5384a lrc: 2/0,0 mode: PR/PR res: [0x200000007:0x1:0x0].0x1317f023 bits 0x2/0x0 rrc: 3 type: IBT gid 0 flags: 0x50210400000000 nid: local remote: 0x0 expref: -99 pid: 6256 timeout: 0 lvb_type: 0 00010000:00010000:3.0:1776178211.983838:0:6256:0:(ldlm_lockd.c:1879:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: mdt-lustre-MDT0000_UUID lock: 000000008192de90/0xa40083ab9bf5384a lrc: 3/0,0 mode: PR/PR res: [0x200000007:0x1:0x0].0x1317f023 bits 0x2/0x0 rrc: 3 type: IBT gid 0 flags: 0x40210400000000 nid: local remote: 0x0 expref: -99 pid: 6256 timeout: 0 lvb_type: 0 00010000:00010000:3.0:1776178211.983842:0:6256:0:(ldlm_lockd.c:1893:ldlm_handle_bl_callback()) Lock 000000008192de90 already unused, calling callback (0000000079e78348) 00010000:00010000:3.0:1776178211.984197:0:6256:0:(ldlm_request.c:364:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: mdt-lustre-MDT0000_UUID lock: 000000008192de90/0xa40083ab9bf5384a lrc: 3/0,0 mode: PR/PR res: [0x200000007:0x1:0x0].0x1317f023 bits 0x2/0x0 rrc: 3 type: IBT gid 0 flags: 0x40210400000000 nid: local remote: 0x0 expref: -99 pid: 6256 timeout: 0 lvb_type: 0 00010000:00010000:3.0:1776178211.984207:0:6256:0:(ldlm_request.c:1257:ldlm_cli_cancel_local()) ### server-side local cancel ns: mdt-lustre-MDT0000_UUID lock: 000000008192de90/0xa40083ab9bf5384a lrc: 4/0,0 mode: PR/PR res: [0x200000007:0x1:0x0].0x1317f023 bits 0x2/0x0 rrc: 3 type: IBT gid 0 flags: 0x40218400000000 nid: local remote: 0x0 expref: -99 pid: 6256 timeout: 0 lvb_type: 0 00010000:00010000:3.0:1776178211.984215:0:6256:0:(ldlm_inodebits.c:94:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x200000007:0x1:0x0].0x1317f023 (00000000ec624a45) 00010000:00010000:3.0:1776178211.984217:0:6256:0:(ldlm_inodebits.c:96:ldlm_reprocess_inodebits_queue()) Hint 2 00010000:00010000:3.0:1776178211.984220:0:6256:0:(ldlm_lockd.c:1903:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: mdt-lustre-MDT0000_UUID lock: 000000008192de90/0xa40083ab9bf5384a lrc: 2/0,0 mode: --/PR res: [0x200000007:0x1:0x0].0x1317f023 bits 0x2/0x0 rrc: 3 type: IBT gid 0 flags: 0x44a19400000000 nid: local remote: 0x0 expref: -99 pid: 6256 timeout: 0 lvb_type: 0 00010000:00010000:3.0:1776178211.984226:0:6256:0:(ldlm_lock.c:216:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: mdt-lustre-MDT0000_UUID lock: 000000008192de90/0xa40083ab9bf5384a lrc: 0/0,0 mode: --/PR res: [0x200000007:0x1:0x0].0x1317f023 bits 0x2/0x0 rrc: 3 type: IBT gid 0 flags: 0x44a19400000000 nid: local remote: 0x0 expref: -99 pid: 6256 timeout: 0 lvb_type: 0 00010000:00010000:3.0:1776178211.984259:0:6256:0:(ldlm_lockd.c:1484:ldlm_handle_enqueue0()) ### server-side enqueue handler, sending reply (err=301, rc=0) ns: mdt-lustre-MDT0000_UUID lock: 00000000a3795b7d/0xa40083ab9bf5383c lrc: 1/0,0 mode: --/CR res: [0x200000007:0x1:0x0].0x0 bits 0x1/0x0 rrc: 4 type: IBT gid 0 flags: 0x44000000000000 nid: 192.168.204.3@tcp remote: 0x771f68aa1eb27bd5 expref: 11 pid: 6256 timeout: 0 lvb_type: 0 00010000:00010000:3.0:1776178211.984267:0:6256:0:(ldlm_lock.c:216:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: mdt-lustre-MDT0000_UUID lock: 00000000a3795b7d/0xa40083ab9bf5383c lrc: 0/0,0 mode: --/CR res: [0x200000007:0x1:0x0].0x0 bits 0x1/0x0 rrc: 4 type: IBT gid 0 flags: 0x44000000000000 nid: 192.168.204.3@tcp remote: 0x771f68aa1eb27bd5 expref: 11 pid: 6256 timeout: 0 lvb_type: 0 00010000:00010000:3.0:1776178211.984276:0:6256:0:(ldlm_lockd.c:1565:ldlm_handle_enqueue0()) ### server-side enqueue handler END (lock 00000000a3795b7d, rc 0) 00000100:00100000:3.0:1776178211.984366:0:6256:0:(service.c:2372:ptlrpc_server_handle_request()) Handled RPC req@000000002300ff10 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_001:b50963bd-8eba-4432-9a54-8888d00e3fa1+10:18254:x1862456979103616:12345-192.168.204.3@tcp:101:lfs.0 Request processed in 1051us (4966us total) trans 12884901949 rc 301/301 00000100:00100000:3.0:1776178211.984375:0:6256:0:(nrs_fifo.c:241:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.204.3@tcp, seq: 6600 00000100:00100000:2.0:1776178211.987568:0:3347:0:(events.c:361:request_in_callback()) peer: 12345-192.168.204.3@tcp (source: 12345-192.168.204.3@tcp) 00000100:00100000:3.0:1776178211.987751:0:6258:0:(service.c:2152:ptlrpc_server_handle_req_in()) got req x1862456979103744 00000100:00100000:3.0:1776178211.987821:0:6258:0:(nrs_fifo.c:179:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.204.3@tcp, seq: 2145 00000100:00100000:3.0:1776178211.987829:0:6258:0:(service.c:2324:ptlrpc_server_handle_request()) Handling RPC req@000000001175d1b2 pname:cluuid+ref:pid:xid:nid:opc:job mdt_rdpg00_000:b50963bd-8eba-4432-9a54-8888d00e3fa1+10:18254:x1862456979103744:12345-192.168.204.3@tcp:35:lfs.0 00000100:00100000:3.0:1776178211.988186:0:6258:0:(service.c:2372:ptlrpc_server_handle_request()) Handled RPC req@000000001175d1b2 pname:cluuid+ref:pid:xid:nid:opc:job mdt_rdpg00_000:b50963bd-8eba-4432-9a54-8888d00e3fa1+11:18254:x1862456979103744:12345-192.168.204.3@tcp:35:lfs.0 Request processed in 357us (639us total) trans 12884901950 rc 0/0 00000100:00100000:3.0:1776178211.988195:0:6258:0:(nrs_fifo.c:241:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.204.3@tcp, seq: 2145 00000100:00100000:2.0:1776178211.993425:0:3348:0:(events.c:361:request_in_callback()) peer: 12345-192.168.204.3@tcp (source: 12345-192.168.204.3@tcp) 00000100:00100000:3.0:1776178211.993757:0:6259:0:(service.c:2152:ptlrpc_server_handle_req_in()) got req x1862456979103808 00000100:00100000:3.0:1776178211.993834:0:6259:0:(nrs_fifo.c:179:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.204.3@tcp, seq: 2146 00000100:00100000:3.0:1776178211.993843:0:6259:0:(service.c:2324:ptlrpc_server_handle_request()) Handling RPC req@00000000fd2b8392 pname:cluuid+ref:pid:xid:nid:opc:job mdt_rdpg00_001:b50963bd-8eba-4432-9a54-8888d00e3fa1+11:18254:x1862456979103808:12345-192.168.204.3@tcp:33:lfs.0 00000004:00200000:3.0:1776178211.993978:0:6259:0:(mdt_handler.c:912:mdt_pack_attr2body()) [0x2000013a1:0x9:0x0]: returning size 4096 00000004:00200000:3.0:1776178211.993982:0:6259:0:(mdt_handler.c:1523:mdt_getattr_internal()) dirent count 2 stripe count 1 MDT count 2 00000100:00100000:3.0:1776178211.994129:0:6259:0:(service.c:2372:ptlrpc_server_handle_request()) Handled RPC req@00000000fd2b8392 pname:cluuid+ref:pid:xid:nid:opc:job mdt_rdpg00_001:b50963bd-8eba-4432-9a54-8888d00e3fa1+11:18254:x1862456979103808:12345-192.168.204.3@tcp:33:lfs.0 Request processed in 286us (719us total) trans 0 rc 0/0 00000100:00100000:3.0:1776178211.994139:0:6259:0:(nrs_fifo.c:241:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.204.3@tcp, seq: 2146 00000100:00100000:2.0:1776178211.996459:0:3346:0:(events.c:361:request_in_callback()) peer: 12345-192.168.204.3@tcp (source: 12345-192.168.204.3@tcp) 00000100:00100000:1.0:1776178211.996673:0:6258:0:(service.c:2152:ptlrpc_server_handle_req_in()) got req x1862456979103936 00000100:00100000:1.0:1776178211.996739:0:6258:0:(nrs_fifo.c:179:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.204.3@tcp, seq: 2147 00000100:00100000:1.0:1776178211.996746:0:6258:0:(service.c:2324:ptlrpc_server_handle_request()) Handling RPC req@0000000042de09e9 pname:cluuid+ref:pid:xid:nid:opc:job mdt_rdpg00_000:b50963bd-8eba-4432-9a54-8888d00e3fa1+11:18254:x1862456979103936:12345-192.168.204.3@tcp:35:lfs.0 00000100:00100000:1.0:1776178211.997025:0:6258:0:(service.c:2372:ptlrpc_server_handle_request()) Handled RPC req@0000000042de09e9 pname:cluuid+ref:pid:xid:nid:opc:job mdt_rdpg00_000:b50963bd-8eba-4432-9a54-8888d00e3fa1+12:18254:x1862456979103936:12345-192.168.204.3@tcp:35:lfs.0 Request processed in 278us (576us total) trans 12884901951 rc 0/0 00000100:00100000:1.0:1776178211.997034:0:6258:0:(nrs_fifo.c:241:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.204.3@tcp, seq: 2147 00000100:00100000:2.0:1776178212.073086:0:3347:0:(events.c:361:request_in_callback()) peer: 12345-192.168.204.3@tcp (source: 12345-192.168.204.3@tcp) 00000100:00100000:0.0:1776178212.073839:0:6256:0:(service.c:2152:ptlrpc_server_handle_req_in()) got req x1862456979104000 00000100:00100000:0.0:1776178212.073931:0:6256:0:(nrs_fifo.c:179:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.204.3@tcp, seq: 6601 00000100:00100000:0.0:1776178212.073938:0:6256:0:(service.c:2324:ptlrpc_server_handle_request()) Handling RPC req@0000000029ef3988 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_001:b50963bd-8eba-4432-9a54-8888d00e3fa1+12:18257:x1862456979104000:12345-192.168.204.3@tcp:101:touch.0 00010000:00010000:0.0:1776178212.073970:0:6256:0:(ldlm_lockd.c:1248:ldlm_handle_enqueue0()) ### server-side enqueue handler START 00010000:00010000:0.0:1776178212.073992:0:6256:0:(ldlm_lockd.c:1318:ldlm_handle_enqueue0()) ### server-side enqueue handler, new lock created ns: mdt-lustre-MDT0000_UUID lock: 000000006b206cc9/0xa40083ab9bf53858 lrc: 2/0,0 mode: --/CR res: [0x200000007:0x1:0x0].0x0 bits 0x0/0x0 rrc: 4 type: IBT gid 0 flags: 0x40000000000000 nid: local remote: 0x771f68aa1eb27bdc expref: -99 pid: 6256 timeout: 0 lvb_type: 0 00010000:00010000:0.0:1776178212.074100:0:6256:0:(ldlm_lock.c:766:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(CR) ns: mdt-lustre-MDT0000_UUID lock: 0000000095e53696/0xa40083ab9bf5385f lrc: 3/1,0 mode: --/CR res: [0x200000007:0x1:0x0].0x0 bits 0x0/0x0 rrc: 5 type: IBT gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 6256 timeout: 0 lvb_type: 0 00010000:00010000:0.0:1776178212.074123:0:6256:0:(ldlm_lock.c:1064:ldlm_granted_list_add_lock()) ### About to add lock: ns: mdt-lustre-MDT0000_UUID lock: 0000000095e53696/0xa40083ab9bf5385f lrc: 3/1,0 mode: CR/CR res: [0x200000007:0x1:0x0].0x0 bits 0x2/0x0 rrc: 5 type: IBT gid 0 flags: 0x50210000000000 nid: local remote: 0x0 expref: -99 pid: 6256 timeout: 0 lvb_type: 0 00010000:00010000:0.0:1776178212.074405:0:6256:0:(ldlm_request.c:528:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: mdt-lustre-MDT0000_UUID lock: 0000000095e53696/0xa40083ab9bf5385f lrc: 3/1,0 mode: CR/CR res: [0x200000007:0x1:0x0].0x0 bits 0x2/0x0 rrc: 5 type: IBT gid 0 flags: 0x40210000000000 nid: local remote: 0x0 expref: -99 pid: 6256 timeout: 0 lvb_type: 0 00010000:00010000:0.0:1776178212.074425:0:6256:0:(ldlm_lock.c:766:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PR) ns: mdt-lustre-MDT0000_UUID lock: 000000004bd1d7cb/0xa40083ab9bf53866 lrc: 3/1,0 mode: --/PR res: [0x200000007:0x1:0x0].0x1317f023 bits 0x0/0x0 rrc: 3 type: IBT gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 6256 timeout: 0 lvb_type: 0 00010000:00010000:0.0:1776178212.074433:0:6256:0:(ldlm_lock.c:1064:ldlm_granted_list_add_lock()) ### About to add lock: ns: mdt-lustre-MDT0000_UUID lock: 000000004bd1d7cb/0xa40083ab9bf53866 lrc: 3/1,0 mode: PR/PR res: [0x200000007:0x1:0x0].0x1317f023 bits 0x2/0x0 rrc: 3 type: IBT gid 0 flags: 0x50210000000000 nid: local remote: 0x0 expref: -99 pid: 6256 timeout: 0 lvb_type: 0 00010000:00010000:0.0:1776178212.074440:0:6256:0:(ldlm_request.c:528:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: mdt-lustre-MDT0000_UUID lock: 000000004bd1d7cb/0xa40083ab9bf53866 lrc: 3/1,0 mode: PR/PR res: [0x200000007:0x1:0x0].0x1317f023 bits 0x2/0x0 rrc: 3 type: IBT gid 0 flags: 0x40210000000000 nid: local remote: 0x0 expref: -99 pid: 6256 timeout: 0 lvb_type: 0 00010000:00010000:0.0:1776178212.074561:0:6256:0:(ldlm_lock.c:766:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PR) ns: mdt-lustre-MDT0000_UUID lock: 00000000a211f94c/0xa40083ab9bf5386d lrc: 3/1,0 mode: --/PR res: [0x2000013a1:0x9:0x0].0x0 bits 0x0/0x0 rrc: 3 type: IBT gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 6256 timeout: 0 lvb_type: 0 00010000:00010000:0.0:1776178212.074570:0:6256:0:(ldlm_lock.c:1064:ldlm_granted_list_add_lock()) ### About to add lock: ns: mdt-lustre-MDT0000_UUID lock: 00000000a211f94c/0xa40083ab9bf5386d lrc: 3/1,0 mode: PR/PR res: [0x2000013a1:0x9:0x0].0x0 bits 0x11/0x0 rrc: 3 type: IBT gid 0 flags: 0x50210000000000 nid: local remote: 0x0 expref: -99 pid: 6256 timeout: 0 lvb_type: 0 00010000:00010000:0.0:1776178212.074577:0:6256:0:(ldlm_request.c:528:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: mdt-lustre-MDT0000_UUID lock: 00000000a211f94c/0xa40083ab9bf5386d lrc: 3/1,0 mode: PR/PR res: [0x2000013a1:0x9:0x0].0x0 bits 0x11/0x0 rrc: 3 type: IBT gid 0 flags: 0x40210000000000 nid: local remote: 0x0 expref: -99 pid: 6256 timeout: 0 lvb_type: 0 00000004:00200000:0.0:1776178212.074619:0:6256:0:(mdt_handler.c:912:mdt_pack_attr2body()) [0x2000013a1:0x9:0x0]: returning size 4096 00000004:00200000:0.0:1776178212.074623:0:6256:0:(mdt_handler.c:1523:mdt_getattr_internal()) dirent count 2 stripe count 1 MDT count 2 00000004:00010000:0.0:1776178212.074646:0:6256:0:(mdt_handler.c:2353:mdt_getattr_name_lock()) ### Returning lock to client ns: mdt-lustre-MDT0000_UUID lock: 00000000a211f94c/0xa40083ab9bf5386d lrc: 3/1,0 mode: PR/PR res: [0x2000013a1:0x9:0x0].0x0 bits 0x11/0x0 rrc: 3 type: IBT gid 0 flags: 0x40210000000000 nid: local remote: 0x0 expref: -99 pid: 6256 timeout: 0 lvb_type: 0 00010000:00010000:0.0:1776178212.074658:0:6256:0:(ldlm_lock.c:820:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(CR) ns: mdt-lustre-MDT0000_UUID lock: 0000000095e53696/0xa40083ab9bf5385f lrc: 3/1,0 mode: CR/CR res: [0x200000007:0x1:0x0].0x0 bits 0x2/0x0 rrc: 5 type: IBT gid 0 flags: 0x50210000000000 nid: local remote: 0x0 expref: -99 pid: 6256 timeout: 0 lvb_type: 0 00010000:00010000:0.0:1776178212.074665:0:6256:0:(ldlm_lock.c:879:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: mdt-lustre-MDT0000_UUID lock: 0000000095e53696/0xa40083ab9bf5385f lrc: 2/0,0 mode: CR/CR res: [0x200000007:0x1:0x0].0x0 bits 0x2/0x0 rrc: 5 type: IBT gid 0 flags: 0x50210400000000 nid: local remote: 0x0 expref: -99 pid: 6256 timeout: 0 lvb_type: 0 00010000:00010000:0.0:1776178212.074682:0:6256:0:(ldlm_lockd.c:1879:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: mdt-lustre-MDT0000_UUID lock: 0000000095e53696/0xa40083ab9bf5385f lrc: 3/0,0 mode: CR/CR res: [0x200000007:0x1:0x0].0x0 bits 0x2/0x0 rrc: 5 type: IBT gid 0 flags: 0x40210400000000 nid: local remote: 0x0 expref: -99 pid: 6256 timeout: 0 lvb_type: 0 00010000:00010000:0.0:1776178212.074690:0:6256:0:(ldlm_lockd.c:1893:ldlm_handle_bl_callback()) Lock 0000000095e53696 already unused, calling callback (0000000079e78348) 00010000:00010000:0.0:1776178212.074695:0:6256:0:(ldlm_request.c:364:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: mdt-lustre-MDT0000_UUID lock: 0000000095e53696/0xa40083ab9bf5385f lrc: 3/0,0 mode: CR/CR res: [0x200000007:0x1:0x0].0x0 bits 0x2/0x0 rrc: 5 type: IBT gid 0 flags: 0x40210400000000 nid: local remote: 0x0 expref: -99 pid: 6256 timeout: 0 lvb_type: 0 00010000:00010000:0.0:1776178212.074703:0:6256:0:(ldlm_request.c:1257:ldlm_cli_cancel_local()) ### server-side local cancel ns: mdt-lustre-MDT0000_UUID lock: 0000000095e53696/0xa40083ab9bf5385f lrc: 4/0,0 mode: CR/CR res: [0x200000007:0x1:0x0].0x0 bits 0x2/0x0 rrc: 5 type: IBT gid 0 flags: 0x40218400000000 nid: local remote: 0x0 expref: -99 pid: 6256 timeout: 0 lvb_type: 0 00010000:00010000:0.0:1776178212.074715:0:6256:0:(ldlm_inodebits.c:94:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x200000007:0x1:0x0].0x0 (00000000ad0d4094) 00010000:00010000:0.0:1776178212.074718:0:6256:0:(ldlm_inodebits.c:96:ldlm_reprocess_inodebits_queue()) Hint 2 00010000:00010000:0.0:1776178212.074723:0:6256:0:(ldlm_lockd.c:1903:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: mdt-lustre-MDT0000_UUID lock: 0000000095e53696/0xa40083ab9bf5385f lrc: 2/0,0 mode: --/CR res: [0x200000007:0x1:0x0].0x0 bits 0x2/0x0 rrc: 5 type: IBT gid 0 flags: 0x44a19400000000 nid: local remote: 0x0 expref: -99 pid: 6256 timeout: 0 lvb_type: 0 00010000:00010000:0.0:1776178212.074729:0:6256:0:(ldlm_lock.c:216:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: mdt-lustre-MDT0000_UUID lock: 0000000095e53696/0xa40083ab9bf5385f lrc: 0/0,0 mode: --/CR res: [0x200000007:0x1:0x0].0x0 bits 0x2/0x0 rrc: 5 type: IBT gid 0 flags: 0x44a19400000000 nid: local remote: 0x0 expref: -99 pid: 6256 timeout: 0 lvb_type: 0 00010000:00010000:0.0:1776178212.074742:0:6256:0:(ldlm_lock.c:820:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PR) ns: mdt-lustre-MDT0000_UUID lock: 000000004bd1d7cb/0xa40083ab9bf53866 lrc: 3/1,0 mode: PR/PR res: [0x200000007:0x1:0x0].0x1317f023 bits 0x2/0x0 rrc: 3 type: IBT gid 0 flags: 0x50210000000000 nid: local remote: 0x0 expref: -99 pid: 6256 timeout: 0 lvb_type: 0 00010000:00010000:0.0:1776178212.074748:0:6256:0:(ldlm_lock.c:879:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: mdt-lustre-MDT0000_UUID lock: 000000004bd1d7cb/0xa40083ab9bf53866 lrc: 2/0,0 mode: PR/PR res: [0x200000007:0x1:0x0].0x1317f023 bits 0x2/0x0 rrc: 3 type: IBT gid 0 flags: 0x50210400000000 nid: local remote: 0x0 expref: -99 pid: 6256 timeout: 0 lvb_type: 0 00010000:00010000:0.0:1776178212.074753:0:6256:0:(ldlm_lockd.c:1879:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: mdt-lustre-MDT0000_UUID lock: 000000004bd1d7cb/0xa40083ab9bf53866 lrc: 3/0,0 mode: PR/PR res: [0x200000007:0x1:0x0].0x1317f023 bits 0x2/0x0 rrc: 3 type: IBT gid 0 flags: 0x40210400000000 nid: local remote: 0x0 expref: -99 pid: 6256 timeout: 0 lvb_type: 0 00010000:00010000:0.0:1776178212.074763:0:6256:0:(ldlm_lockd.c:1893:ldlm_handle_bl_callback()) Lock 000000004bd1d7cb already unused, calling callback (0000000079e78348) 00010000:00010000:0.0:1776178212.074765:0:6256:0:(ldlm_request.c:364:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: mdt-lustre-MDT0000_UUID lock: 000000004bd1d7cb/0xa40083ab9bf53866 lrc: 3/0,0 mode: PR/PR res: [0x200000007:0x1:0x0].0x1317f023 bits 0x2/0x0 rrc: 3 type: IBT gid 0 flags: 0x40210400000000 nid: local remote: 0x0 expref: -99 pid: 6256 timeout: 0 lvb_type: 0 00010000:00010000:0.0:1776178212.074770:0:6256:0:(ldlm_request.c:1257:ldlm_cli_cancel_local()) ### server-side local cancel ns: mdt-lustre-MDT0000_UUID lock: 000000004bd1d7cb/0xa40083ab9bf53866 lrc: 4/0,0 mode: PR/PR res: [0x200000007:0x1:0x0].0x1317f023 bits 0x2/0x0 rrc: 3 type: IBT gid 0 flags: 0x40218400000000 nid: local remote: 0x0 expref: -99 pid: 6256 timeout: 0 lvb_type: 0 00010000:00010000:0.0:1776178212.074777:0:6256:0:(ldlm_inodebits.c:94:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x200000007:0x1:0x0].0x1317f023 (00000000ec624a45) 00010000:00010000:0.0:1776178212.074779:0:6256:0:(ldlm_inodebits.c:96:ldlm_reprocess_inodebits_queue()) Hint 2 00010000:00010000:0.0:1776178212.074781:0:6256:0:(ldlm_lockd.c:1903:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: mdt-lustre-MDT0000_UUID lock: 000000004bd1d7cb/0xa40083ab9bf53866 lrc: 2/0,0 mode: --/PR res: [0x200000007:0x1:0x0].0x1317f023 bits 0x2/0x0 rrc: 3 type: IBT gid 0 flags: 0x44a19400000000 nid: local remote: 0x0 expref: -99 pid: 6256 timeout: 0 lvb_type: 0 00010000:00010000:0.0:1776178212.074786:0:6256:0:(ldlm_lock.c:216:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: mdt-lustre-MDT0000_UUID lock: 000000004bd1d7cb/0xa40083ab9bf53866 lrc: 0/0,0 mode: --/PR res: [0x200000007:0x1:0x0].0x1317f023 bits 0x2/0x0 rrc: 3 type: IBT gid 0 flags: 0x44a19400000000 nid: local remote: 0x0 expref: -99 pid: 6256 timeout: 0 lvb_type: 0 00010000:00010000:0.0:1776178212.074812:0:6256:0:(ldlm_lock.c:216:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: mdt-lustre-MDT0000_UUID lock: 000000006b206cc9/0xa40083ab9bf53858 lrc: 0/0,0 mode: --/CR res: [0x200000007:0x1:0x0].0x0 bits 0x1/0x0 rrc: 4 type: IBT gid 0 flags: 0x44000000000000 nid: 192.168.204.3@tcp remote: 0x771f68aa1eb27bdc expref: 14 pid: 6256 timeout: 0 lvb_type: 0 00010000:00010000:0.0:1776178212.074824:0:6256:0:(ldlm_lockd.c:1484:ldlm_handle_enqueue0()) ### server-side enqueue handler, sending reply (err=0, rc=0) ns: mdt-lustre-MDT0000_UUID lock: 00000000a211f94c/0xa40083ab9bf5386d lrc: 3/0,0 mode: PR/PR res: [0x2000013a1:0x9:0x0].0x0 bits 0x11/0x0 rrc: 3 type: IBT gid 0 flags: 0x40200000000000 nid: 192.168.204.3@tcp remote: 0x771f68aa1eb27bdc expref: 13 pid: 6256 timeout: 0 lvb_type: 0 00010000:00010000:0.0:1776178212.074830:0:6256:0:(ldlm_inodebits.c:94:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x2000013a1:0x9:0x0].0x0 (00000000a041a513) 00010000:00010000:0.0:1776178212.074832:0:6256:0:(ldlm_inodebits.c:96:ldlm_reprocess_inodebits_queue()) Hint 11 00010000:00010000:0.0:1776178212.074834:0:6256:0:(ldlm_lockd.c:1565:ldlm_handle_enqueue0()) ### server-side enqueue handler END (lock 00000000a211f94c, rc 0) 00000100:00100000:0.0:1776178212.074936:0:6256:0:(service.c:2372:ptlrpc_server_handle_request()) Handled RPC req@0000000029ef3988 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_001:b50963bd-8eba-4432-9a54-8888d00e3fa1+13:18257:x1862456979104000:12345-192.168.204.3@tcp:101:touch.0 Request processed in 996us (1879us total) trans 0 rc 0/0 00000100:00100000:0.0:1776178212.074945:0:6256:0:(nrs_fifo.c:241:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.204.3@tcp, seq: 6601 00000100:00100000:2.0:1776178212.086370:0:3348:0:(events.c:361:request_in_callback()) peer: 12345-192.168.204.3@tcp (source: 12345-192.168.204.3@tcp) 00000100:00100000:0.0:1776178212.086454:0:6257:0:(service.c:2152:ptlrpc_server_handle_req_in()) got req x1862456979104128 00000100:00100000:0.0:1776178212.086524:0:6257:0:(nrs_fifo.c:179:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.204.3@tcp, seq: 6602 00000100:00100000:0.0:1776178212.086534:0:6257:0:(service.c:2324:ptlrpc_server_handle_request()) Handling RPC req@0000000035819321 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_002:b50963bd-8eba-4432-9a54-8888d00e3fa1+13:18257:x1862456979104128:12345-192.168.204.3@tcp:101:touch.0 00010000:00010000:0.0:1776178212.086567:0:6257:0:(ldlm_lockd.c:1248:ldlm_handle_enqueue0()) ### server-side enqueue handler START 00010000:00010000:0.0:1776178212.086593:0:6257:0:(ldlm_lockd.c:1318:ldlm_handle_enqueue0()) ### server-side enqueue handler, new lock created ns: mdt-lustre-MDT0000_UUID lock: 000000006b206cc9/0xa40083ab9bf53874 lrc: 2/0,0 mode: --/CW res: [0x2000013a1:0x9:0x0].0x0 bits 0x0/0x0 rrc: 4 type: IBT gid 0 flags: 0x40000000000000 nid: local remote: 0x771f68aa1eb27be3 expref: -99 pid: 6257 timeout: 0 lvb_type: 0 00010000:00010000:0.0:1776178212.086757:0:6257:0:(ldlm_lock.c:766:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(CW) ns: mdt-lustre-MDT0000_UUID lock: 000000004bd1d7cb/0xa40083ab9bf5387b lrc: 3/0,1 mode: --/CW res: [0x2000013a1:0x9:0x0].0x0 bits 0x0/0x0 rrc: 5 type: IBT gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 6257 timeout: 0 lvb_type: 0 00010000:00010000:0.0:1776178212.086775:0:6257:0:(ldlm_lock.c:1064:ldlm_granted_list_add_lock()) ### About to add lock: ns: mdt-lustre-MDT0000_UUID lock: 000000004bd1d7cb/0xa40083ab9bf5387b lrc: 3/0,1 mode: CW/CW res: [0x2000013a1:0x9:0x0].0x0 bits 0x2/0x0 rrc: 5 type: IBT gid 0 flags: 0x50210000000000 nid: local remote: 0x0 expref: -99 pid: 6257 timeout: 0 lvb_type: 0 00010000:00010000:0.0:1776178212.086788:0:6257:0:(ldlm_request.c:528:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: mdt-lustre-MDT0000_UUID lock: 000000004bd1d7cb/0xa40083ab9bf5387b lrc: 3/0,1 mode: CW/CW res: [0x2000013a1:0x9:0x0].0x0 bits 0x2/0x0 rrc: 5 type: IBT gid 0 flags: 0x40210000000000 nid: local remote: 0x0 expref: -99 pid: 6257 timeout: 0 lvb_type: 0 00010000:00010000:0.0:1776178212.086805:0:6257:0:(ldlm_lock.c:766:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: mdt-lustre-MDT0000_UUID lock: 0000000095e53696/0xa40083ab9bf53882 lrc: 3/0,1 mode: --/PW res: [0x2000013a1:0x9:0x0].0xcda302ea bits 0x0/0x0 rrc: 2 type: IBT gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 6257 timeout: 0 lvb_type: 0 00010000:00010000:0.0:1776178212.086813:0:6257:0:(ldlm_lock.c:1064:ldlm_granted_list_add_lock()) ### About to add lock: ns: mdt-lustre-MDT0000_UUID lock: 0000000095e53696/0xa40083ab9bf53882 lrc: 3/0,1 mode: PW/PW res: [0x2000013a1:0x9:0x0].0xcda302ea bits 0x2/0x0 rrc: 2 type: IBT gid 0 flags: 0x50210000000000 nid: local remote: 0x0 expref: -99 pid: 6257 timeout: 0 lvb_type: 0 00010000:00010000:0.0:1776178212.086821:0:6257:0:(ldlm_request.c:528:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: mdt-lustre-MDT0000_UUID lock: 0000000095e53696/0xa40083ab9bf53882 lrc: 3/0,1 mode: PW/PW res: [0x2000013a1:0x9:0x0].0xcda302ea bits 0x2/0x0 rrc: 2 type: IBT gid 0 flags: 0x40210000000000 nid: local remote: 0x0 expref: -99 pid: 6257 timeout: 0 lvb_type: 0 00000004:00080000:0.0:1776178212.087182:0:6257:0:(osp_object.c:1645:osp_create()) lustre-OST0001-osc-MDT0000: Wrote last used FID: [0x100010000:0x45:0x0], index 1: 0 00010000:00010000:0.0:1776178212.087315:0:6257:0:(ldlm_lock.c:766:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(CR) ns: mdt-lustre-MDT0000_UUID lock: 00000000782e253e/0xa40083ab9bf53889 lrc: 3/1,0 mode: --/CR res: [0x2000013a1:0xa:0x0].0x0 bits 0x0/0x0 rrc: 2 type: IBT gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 6257 timeout: 0 lvb_type: 0 00010000:00010000:0.0:1776178212.087323:0:6257:0:(ldlm_lock.c:1064:ldlm_granted_list_add_lock()) ### About to add lock: ns: mdt-lustre-MDT0000_UUID lock: 00000000782e253e/0xa40083ab9bf53889 lrc: 3/1,0 mode: CR/CR res: [0x2000013a1:0xa:0x0].0x0 bits 0x9/0x0 rrc: 2 type: IBT gid 0 flags: 0x50210000000000 nid: local remote: 0x0 expref: -99 pid: 6257 timeout: 0 lvb_type: 0 00010000:00010000:0.0:1776178212.087334:0:6257:0:(ldlm_request.c:528:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: mdt-lustre-MDT0000_UUID lock: 00000000782e253e/0xa40083ab9bf53889 lrc: 3/1,0 mode: CR/CR res: [0x2000013a1:0xa:0x0].0x0 bits 0x9/0x0 rrc: 2 type: IBT gid 0 flags: 0x40210000000000 nid: local remote: 0x0 expref: -99 pid: 6257 timeout: 0 lvb_type: 0 00010000:00010000:0.0:1776178212.087364:0:6257:0:(ldlm_lock.c:820:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(CW) ns: mdt-lustre-MDT0000_UUID lock: 000000004bd1d7cb/0xa40083ab9bf5387b lrc: 3/0,1 mode: CW/CW res: [0x2000013a1:0x9:0x0].0x0 bits 0x2/0x0 rrc: 5 type: IBT gid 0 flags: 0x50210000000000 nid: local remote: 0x0 expref: -99 pid: 6257 timeout: 0 lvb_type: 0 00010000:00010000:0.0:1776178212.087370:0:6257:0:(ldlm_lock.c:879:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: mdt-lustre-MDT0000_UUID lock: 000000004bd1d7cb/0xa40083ab9bf5387b lrc: 2/0,0 mode: CW/CW res: [0x2000013a1:0x9:0x0].0x0 bits 0x2/0x0 rrc: 5 type: IBT gid 0 flags: 0x50210400000000 nid: local remote: 0x0 expref: -99 pid: 6257 timeout: 0 lvb_type: 0 00010000:00010000:0.0:1776178212.087376:0:6257:0:(ldlm_lockd.c:1879:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: mdt-lustre-MDT0000_UUID lock: 000000004bd1d7cb/0xa40083ab9bf5387b lrc: 3/0,0 mode: CW/CW res: [0x2000013a1:0x9:0x0].0x0 bits 0x2/0x0 rrc: 5 type: IBT gid 0 flags: 0x40210400000000 nid: local remote: 0x0 expref: -99 pid: 6257 timeout: 0 lvb_type: 0 00010000:00010000:0.0:1776178212.087381:0:6257:0:(ldlm_lockd.c:1893:ldlm_handle_bl_callback()) Lock 000000004bd1d7cb already unused, calling callback (0000000079e78348) 00010000:00010000:0.0:1776178212.087384:0:6257:0:(ldlm_request.c:364:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: mdt-lustre-MDT0000_UUID lock: 000000004bd1d7cb/0xa40083ab9bf5387b lrc: 3/0,0 mode: CW/CW res: [0x2000013a1:0x9:0x0].0x0 bits 0x2/0x0 rrc: 5 type: IBT gid 0 flags: 0x40210400000000 nid: local remote: 0x0 expref: -99 pid: 6257 timeout: 0 lvb_type: 0 00010000:00010000:0.0:1776178212.087390:0:6257:0:(ldlm_request.c:1257:ldlm_cli_cancel_local()) ### server-side local cancel ns: mdt-lustre-MDT0000_UUID lock: 000000004bd1d7cb/0xa40083ab9bf5387b lrc: 4/0,0 mode: CW/CW res: [0x2000013a1:0x9:0x0].0x0 bits 0x2/0x0 rrc: 5 type: IBT gid 0 flags: 0x40218400000000 nid: local remote: 0x0 expref: -99 pid: 6257 timeout: 0 lvb_type: 0 00010000:00010000:0.0:1776178212.087402:0:6257:0:(ldlm_inodebits.c:94:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x2000013a1:0x9:0x0].0x0 (00000000a041a513) 00010000:00010000:0.0:1776178212.087404:0:6257:0:(ldlm_inodebits.c:96:ldlm_reprocess_inodebits_queue()) Hint 2 00010000:00010000:0.0:1776178212.087408:0:6257:0:(ldlm_lockd.c:1903:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: mdt-lustre-MDT0000_UUID lock: 000000004bd1d7cb/0xa40083ab9bf5387b lrc: 2/0,0 mode: --/CW res: [0x2000013a1:0x9:0x0].0x0 bits 0x2/0x0 rrc: 5 type: IBT gid 0 flags: 0x44a19400000000 nid: local remote: 0x0 expref: -99 pid: 6257 timeout: 0 lvb_type: 0 00010000:00010000:0.0:1776178212.087413:0:6257:0:(ldlm_lock.c:216:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: mdt-lustre-MDT0000_UUID lock: 000000004bd1d7cb/0xa40083ab9bf5387b lrc: 0/0,0 mode: --/CW res: [0x2000013a1:0x9:0x0].0x0 bits 0x2/0x0 rrc: 5 type: IBT gid 0 flags: 0x44a19400000000 nid: local remote: 0x0 expref: -99 pid: 6257 timeout: 0 lvb_type: 0 00000004:00010000:0.0:1776178212.087423:0:6257:0:(mdt_handler.c:3992:mdt_save_lock()) ### save lock request 0000000035819321 reply state 00000000f85bdf7b transno 12884901952 ns: mdt-lustre-MDT0000_UUID lock: 0000000095e53696/0xa40083ab9bf53882 lrc: 3/0,1 mode: PW/PW res: [0x2000013a1:0x9:0x0].0xcda302ea bits 0x2/0x0 rrc: 2 type: IBT gid 0 flags: 0x40210000000000 nid: local remote: 0x0 expref: -99 pid: 6257 timeout: 0 lvb_type: 0 00010000:00010000:0.0:1776178212.087455:0:6257:0:(ldlm_lock.c:216:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: mdt-lustre-MDT0000_UUID lock: 000000006b206cc9/0xa40083ab9bf53874 lrc: 0/0,0 mode: --/CW res: [0x2000013a1:0x9:0x0].0x0 bits 0x1/0x0 rrc: 4 type: IBT gid 0 flags: 0x44000000000000 nid: 192.168.204.3@tcp remote: 0x771f68aa1eb27be3 expref: 16 pid: 6257 timeout: 0 lvb_type: 0 00010000:00010000:0.0:1776178212.087466:0:6257:0:(ldlm_lockd.c:1484:ldlm_handle_enqueue0()) ### server-side enqueue handler, sending reply (err=0, rc=0) ns: mdt-lustre-MDT0000_UUID lock: 00000000782e253e/0xa40083ab9bf53889 lrc: 3/0,0 mode: CR/CR res: [0x2000013a1:0xa:0x0].0x0 bits 0x9/0x0 rrc: 2 type: IBT gid 0 flags: 0x40200000000000 nid: 192.168.204.3@tcp remote: 0x771f68aa1eb27be3 expref: 15 pid: 6257 timeout: 0 lvb_type: 0 00010000:00010000:0.0:1776178212.087472:0:6257:0:(ldlm_inodebits.c:94:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x2000013a1:0xa:0x0].0x0 (000000000a3c1046) 00010000:00010000:0.0:1776178212.087474:0:6257:0:(ldlm_inodebits.c:96:ldlm_reprocess_inodebits_queue()) Hint 9 00010000:00010000:0.0:1776178212.087476:0:6257:0:(ldlm_lockd.c:1565:ldlm_handle_enqueue0()) ### server-side enqueue handler END (lock 00000000782e253e, rc 0) 00000100:00100000:0.0:1776178212.087557:0:6257:0:(service.c:2372:ptlrpc_server_handle_request()) Handled RPC req@0000000035819321 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_002:b50963bd-8eba-4432-9a54-8888d00e3fa1+16:18257:x1862456979104128:12345-192.168.204.3@tcp:101:touch.0 Request processed in 1014us (1185us total) trans 12884901952 rc 0/0 00000100:00100000:0.0:1776178212.087566:0:6257:0:(nrs_fifo.c:241:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.204.3@tcp, seq: 6602 00010000:00010000:2.0:1776178212.094368:0:3339:0:(ldlm_lock.c:1064:ldlm_granted_list_add_lock()) ### About to add lock: ns: mdt-lustre-MDT0000_UUID lock: 0000000095e53696/0xa40083ab9bf53882 lrc: 3/0,1 mode: COS/COS res: [0x2000013a1:0x9:0x0].0xcda302ea bits 0x2/0x0 rrc: 2 type: IBT gid 0 flags: 0x50210000000000 nid: local remote: 0x0 expref: -99 pid: 6257 timeout: 0 lvb_type: 0 00010000:00010000:2.0:1776178212.094395:0:3339:0:(ldlm_inodebits.c:94:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x2000013a1:0x9:0x0].0xcda302ea (00000000b86c95e3) 00010000:00010000:2.0:1776178212.094399:0:3339:0:(ldlm_inodebits.c:96:ldlm_reprocess_inodebits_queue()) Hint 2 00000100:00100000:2.0:1776178212.102926:0:3348:0:(events.c:361:request_in_callback()) peer: 12345-192.168.204.3@tcp (source: 12345-192.168.204.3@tcp) 00000100:00100000:0.0:1776178212.103050:0:6256:0:(service.c:2152:ptlrpc_server_handle_req_in()) got req x1862456979104256 00000100:00100000:0.0:1776178212.103168:0:6256:0:(nrs_fifo.c:179:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.204.3@tcp, seq: 6603 00000100:00100000:0.0:1776178212.103179:0:6256:0:(service.c:2324:ptlrpc_server_handle_request()) Handling RPC req@0000000012b22334 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_001:b50963bd-8eba-4432-9a54-8888d00e3fa1+16:18257:x1862456979104256:12345-192.168.204.3@tcp:36:touch.0 00010000:00010000:0.0:1776178212.103316:0:6256:0:(ldlm_lock.c:766:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: mdt-lustre-MDT0000_UUID lock: 000000006b206cc9/0xa40083ab9bf53897 lrc: 3/0,1 mode: --/PW res: [0x2000013a1:0xa:0x0].0x0 bits 0x0/0x0 rrc: 3 type: IBT gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 6256 timeout: 0 lvb_type: 0 00010000:00010000:0.0:1776178212.103669:0:6256:0:(ldlm_lock.c:1064:ldlm_granted_list_add_lock()) ### About to add lock: ns: mdt-lustre-MDT0000_UUID lock: 000000006b206cc9/0xa40083ab9bf53897 lrc: 3/0,1 mode: PW/PW res: [0x2000013a1:0xa:0x0].0x0 bits 0x2/0x0 rrc: 3 type: IBT gid 0 flags: 0x50210000000000 nid: local remote: 0x0 expref: -99 pid: 6256 timeout: 0 lvb_type: 0 00010000:00010000:0.0:1776178212.103685:0:6256:0:(ldlm_request.c:528:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: mdt-lustre-MDT0000_UUID lock: 000000006b206cc9/0xa40083ab9bf53897 lrc: 3/0,1 mode: PW/PW res: [0x2000013a1:0xa:0x0].0x0 bits 0x2/0x0 rrc: 3 type: IBT gid 0 flags: 0x40210000000000 nid: local remote: 0x0 expref: -99 pid: 6256 timeout: 0 lvb_type: 0 00000004:00010000:1.0:1776178212.103888:0:6256:0:(mdt_handler.c:3992:mdt_save_lock()) ### save lock request 0000000012b22334 reply state 000000004229d9e6 transno 12884901953 ns: mdt-lustre-MDT0000_UUID lock: 000000006b206cc9/0xa40083ab9bf53897 lrc: 3/0,1 mode: PW/PW res: [0x2000013a1:0xa:0x0].0x0 bits 0x2/0x0 rrc: 3 type: IBT gid 0 flags: 0x40210000000000 nid: local remote: 0x0 expref: -99 pid: 6256 timeout: 0 lvb_type: 0 00000100:00100000:1.0:1776178212.104038:0:6256:0:(service.c:2372:ptlrpc_server_handle_request()) Handled RPC req@0000000012b22334 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_001:b50963bd-8eba-4432-9a54-8888d00e3fa1+18:18257:x1862456979104256:12345-192.168.204.3@tcp:36:touch.0 Request processed in 859us (1123us total) trans 12884901953 rc 0/0 00000100:00100000:1.0:1776178212.104046:0:6256:0:(nrs_fifo.c:241:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.204.3@tcp, seq: 6603 00010000:00010000:3.0:1776178212.106160:0:3340:0:(ldlm_lock.c:1064:ldlm_granted_list_add_lock()) ### About to add lock: ns: mdt-lustre-MDT0000_UUID lock: 000000006b206cc9/0xa40083ab9bf53897 lrc: 3/0,1 mode: COS/COS res: [0x2000013a1:0xa:0x0].0x0 bits 0x2/0x0 rrc: 3 type: IBT gid 0 flags: 0x50210000000000 nid: local remote: 0x0 expref: -99 pid: 6256 timeout: 0 lvb_type: 0 00010000:00010000:3.0:1776178212.106184:0:3340:0:(ldlm_inodebits.c:94:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x2000013a1:0xa:0x0].0x0 (000000000a3c1046) 00010000:00010000:3.0:1776178212.106187:0:3340:0:(ldlm_inodebits.c:96:ldlm_reprocess_inodebits_queue()) Hint 2 00000100:00100000:2.0:1776178212.107484:0:3348:0:(events.c:361:request_in_callback()) peer: 12345-192.168.204.3@tcp (source: 12345-192.168.204.3@tcp) 00000100:00100000:0.0:1776178212.107869:0:20762:0:(service.c:2152:ptlrpc_server_handle_req_in()) got req x1862456979104320 00000100:00100000:0.0:1776178212.107935:0:20762:0:(nrs_fifo.c:179:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.204.3@tcp, seq: 5285 00000100:00100000:0.0:1776178212.107943:0:20762:0:(service.c:2324:ptlrpc_server_handle_request()) Handling RPC req@000000004ce46e20 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_013:b50963bd-8eba-4432-9a54-8888d00e3fa1+5:2270:x1862456979104320:12345-192.168.204.3@tcp:2:touch.0 00000100:00100000:0.0:1776178212.108244:0:20762:0:(service.c:2372:ptlrpc_server_handle_request()) Handled RPC req@000000004ce46e20 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_013:b50963bd-8eba-4432-9a54-8888d00e3fa1+6:2270:x1862456979104320:12345-192.168.204.3@tcp:2:touch.0 Request processed in 300us (768us total) trans 4294969336 rc 0/0 00000100:00100000:0.0:1776178212.108253:0:20762:0:(nrs_fifo.c:241:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.204.3@tcp, seq: 5285 00000100:00100000:1.0:1776178212.115284:0:3346:0:(events.c:361:request_in_callback()) peer: 12345-192.168.204.3@tcp (source: 12345-192.168.204.3@tcp) 00000100:00100000:3.0:1776178212.115387:0:6259:0:(service.c:2152:ptlrpc_server_handle_req_in()) got req x1862456979104448 00000100:00100000:3.0:1776178212.115452:0:6259:0:(nrs_fifo.c:179:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.204.3@tcp, seq: 2148 00000100:00100000:3.0:1776178212.115459:0:6259:0:(service.c:2324:ptlrpc_server_handle_request()) Handling RPC req@000000008aab9f17 pname:cluuid+ref:pid:xid:nid:opc:job mdt_rdpg00_001:b50963bd-8eba-4432-9a54-8888d00e3fa1+18:18257:x1862456979104448:12345-192.168.204.3@tcp:35:touch.0 00000100:00100000:3.0:1776178212.115831:0:6259:0:(service.c:2372:ptlrpc_server_handle_request()) Handled RPC req@000000008aab9f17 pname:cluuid+ref:pid:xid:nid:opc:job mdt_rdpg00_001:b50963bd-8eba-4432-9a54-8888d00e3fa1+19:18257:x1862456979104448:12345-192.168.204.3@tcp:35:touch.0 Request processed in 371us (556us total) trans 12884901954 rc 0/0 00000100:00100000:3.0:1776178212.115840:0:6259:0:(nrs_fifo.c:241:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.204.3@tcp, seq: 2148 00000100:00100000:2.0:1776178212.124343:0:3347:0:(events.c:361:request_in_callback()) peer: 12345-192.168.204.3@tcp (source: 12345-192.168.204.3@tcp) 00000100:00100000:1.0:1776178212.124462:0:6256:0:(service.c:2152:ptlrpc_server_handle_req_in()) got req x1862456979104576 00000100:00100000:1.0:1776178212.124527:0:6256:0:(nrs_fifo.c:179:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.204.3@tcp, seq: 6604 00000100:00100000:1.0:1776178212.124535:0:6256:0:(service.c:2324:ptlrpc_server_handle_request()) Handling RPC req@00000000a08c40f7 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_001:b50963bd-8eba-4432-9a54-8888d00e3fa1+19:18258:x1862456979104576:12345-192.168.204.3@tcp:101:touch.0 00010000:00010000:1.0:1776178212.124574:0:6256:0:(ldlm_lockd.c:1248:ldlm_handle_enqueue0()) ### server-side enqueue handler START 00010000:00010000:1.0:1776178212.124596:0:6256:0:(ldlm_lockd.c:1318:ldlm_handle_enqueue0()) ### server-side enqueue handler, new lock created ns: mdt-lustre-MDT0000_UUID lock: 0000000097975a3b/0xa40083ab9bf5389e lrc: 2/0,0 mode: --/CW res: [0x2000013a1:0x9:0x0].0x0 bits 0x0/0x0 rrc: 4 type: IBT gid 0 flags: 0x40000000000000 nid: local remote: 0x771f68aa1eb27bea expref: -99 pid: 6256 timeout: 0 lvb_type: 0 00010000:00010000:1.0:1776178212.124706:0:6256:0:(ldlm_lock.c:766:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(CW) ns: mdt-lustre-MDT0000_UUID lock: 000000006b0afb52/0xa40083ab9bf538a5 lrc: 3/0,1 mode: --/CW res: [0x2000013a1:0x9:0x0].0x0 bits 0x0/0x0 rrc: 5 type: IBT gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 6256 timeout: 0 lvb_type: 0 00010000:00010000:1.0:1776178212.124733:0:6256:0:(ldlm_lock.c:1064:ldlm_granted_list_add_lock()) ### About to add lock: ns: mdt-lustre-MDT0000_UUID lock: 000000006b0afb52/0xa40083ab9bf538a5 lrc: 3/0,1 mode: CW/CW res: [0x2000013a1:0x9:0x0].0x0 bits 0x2/0x0 rrc: 5 type: IBT gid 0 flags: 0x50210000000000 nid: local remote: 0x0 expref: -99 pid: 6256 timeout: 0 lvb_type: 0 00010000:00010000:1.0:1776178212.124746:0:6256:0:(ldlm_request.c:528:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: mdt-lustre-MDT0000_UUID lock: 000000006b0afb52/0xa40083ab9bf538a5 lrc: 3/0,1 mode: CW/CW res: [0x2000013a1:0x9:0x0].0x0 bits 0x2/0x0 rrc: 5 type: IBT gid 0 flags: 0x40210000000000 nid: local remote: 0x0 expref: -99 pid: 6256 timeout: 0 lvb_type: 0 00010000:00010000:1.0:1776178212.124761:0:6256:0:(ldlm_lock.c:766:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: mdt-lustre-MDT0000_UUID lock: 00000000a14e680b/0xa40083ab9bf538ac lrc: 3/0,1 mode: --/PW res: [0x2000013a1:0x9:0x0].0x15fcbbea bits 0x0/0x0 rrc: 2 type: IBT gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 6256 timeout: 0 lvb_type: 0 00010000:00010000:1.0:1776178212.124769:0:6256:0:(ldlm_lock.c:1064:ldlm_granted_list_add_lock()) ### About to add lock: ns: mdt-lustre-MDT0000_UUID lock: 00000000a14e680b/0xa40083ab9bf538ac lrc: 3/0,1 mode: PW/PW res: [0x2000013a1:0x9:0x0].0x15fcbbea bits 0x2/0x0 rrc: 2 type: IBT gid 0 flags: 0x50210000000000 nid: local remote: 0x0 expref: -99 pid: 6256 timeout: 0 lvb_type: 0 00010000:00010000:1.0:1776178212.124776:0:6256:0:(ldlm_request.c:528:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: mdt-lustre-MDT0000_UUID lock: 00000000a14e680b/0xa40083ab9bf538ac lrc: 3/0,1 mode: PW/PW res: [0x2000013a1:0x9:0x0].0x15fcbbea bits 0x2/0x0 rrc: 2 type: IBT gid 0 flags: 0x40210000000000 nid: local remote: 0x0 expref: -99 pid: 6256 timeout: 0 lvb_type: 0 00000004:00080000:1.0:1776178212.125308:0:6256:0:(osp_object.c:1645:osp_create()) lustre-OST0000-osc-MDT0000: Wrote last used FID: [0x100000000:0x46:0x0], index 0: 0 00010000:00010000:1.0:1776178212.125433:0:6256:0:(ldlm_lock.c:766:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(CR) ns: mdt-lustre-MDT0000_UUID lock: 000000001d465f75/0xa40083ab9bf538b3 lrc: 3/1,0 mode: --/CR res: [0x2000013a1:0xb:0x0].0x0 bits 0x0/0x0 rrc: 2 type: IBT gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 6256 timeout: 0 lvb_type: 0 00010000:00010000:1.0:1776178212.125443:0:6256:0:(ldlm_lock.c:1064:ldlm_granted_list_add_lock()) ### About to add lock: ns: mdt-lustre-MDT0000_UUID lock: 000000001d465f75/0xa40083ab9bf538b3 lrc: 3/1,0 mode: CR/CR res: [0x2000013a1:0xb:0x0].0x0 bits 0x9/0x0 rrc: 2 type: IBT gid 0 flags: 0x50210000000000 nid: local remote: 0x0 expref: -99 pid: 6256 timeout: 0 lvb_type: 0 00010000:00010000:1.0:1776178212.125450:0:6256:0:(ldlm_request.c:528:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: mdt-lustre-MDT0000_UUID lock: 000000001d465f75/0xa40083ab9bf538b3 lrc: 3/1,0 mode: CR/CR res: [0x2000013a1:0xb:0x0].0x0 bits 0x9/0x0 rrc: 2 type: IBT gid 0 flags: 0x40210000000000 nid: local remote: 0x0 expref: -99 pid: 6256 timeout: 0 lvb_type: 0 00010000:00010000:1.0:1776178212.125481:0:6256:0:(ldlm_lock.c:820:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(CW) ns: mdt-lustre-MDT0000_UUID lock: 000000006b0afb52/0xa40083ab9bf538a5 lrc: 3/0,1 mode: CW/CW res: [0x2000013a1:0x9:0x0].0x0 bits 0x2/0x0 rrc: 5 type: IBT gid 0 flags: 0x50210000000000 nid: local remote: 0x0 expref: -99 pid: 6256 timeout: 0 lvb_type: 0 00010000:00010000:1.0:1776178212.125487:0:6256:0:(ldlm_lock.c:879:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: mdt-lustre-MDT0000_UUID lock: 000000006b0afb52/0xa40083ab9bf538a5 lrc: 2/0,0 mode: CW/CW res: [0x2000013a1:0x9:0x0].0x0 bits 0x2/0x0 rrc: 5 type: IBT gid 0 flags: 0x50210400000000 nid: local remote: 0x0 expref: -99 pid: 6256 timeout: 0 lvb_type: 0 00010000:00010000:1.0:1776178212.125494:0:6256:0:(ldlm_lockd.c:1879:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: mdt-lustre-MDT0000_UUID lock: 000000006b0afb52/0xa40083ab9bf538a5 lrc: 3/0,0 mode: CW/CW res: [0x2000013a1:0x9:0x0].0x0 bits 0x2/0x0 rrc: 5 type: IBT gid 0 flags: 0x40210400000000 nid: local remote: 0x0 expref: -99 pid: 6256 timeout: 0 lvb_type: 0 00010000:00010000:1.0:1776178212.125500:0:6256:0:(ldlm_lockd.c:1893:ldlm_handle_bl_callback()) Lock 000000006b0afb52 already unused, calling callback (0000000079e78348) 00010000:00010000:1.0:1776178212.125503:0:6256:0:(ldlm_request.c:364:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: mdt-lustre-MDT0000_UUID lock: 000000006b0afb52/0xa40083ab9bf538a5 lrc: 3/0,0 mode: CW/CW res: [0x2000013a1:0x9:0x0].0x0 bits 0x2/0x0 rrc: 5 type: IBT gid 0 flags: 0x40210400000000 nid: local remote: 0x0 expref: -99 pid: 6256 timeout: 0 lvb_type: 0 00010000:00010000:1.0:1776178212.125514:0:6256:0:(ldlm_request.c:1257:ldlm_cli_cancel_local()) ### server-side local cancel ns: mdt-lustre-MDT0000_UUID lock: 000000006b0afb52/0xa40083ab9bf538a5 lrc: 4/0,0 mode: CW/CW res: [0x2000013a1:0x9:0x0].0x0 bits 0x2/0x0 rrc: 5 type: IBT gid 0 flags: 0x40218400000000 nid: local remote: 0x0 expref: -99 pid: 6256 timeout: 0 lvb_type: 0 00010000:00010000:1.0:1776178212.125525:0:6256:0:(ldlm_inodebits.c:94:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x2000013a1:0x9:0x0].0x0 (00000000a041a513) 00010000:00010000:1.0:1776178212.125528:0:6256:0:(ldlm_inodebits.c:96:ldlm_reprocess_inodebits_queue()) Hint 2 00010000:00010000:1.0:1776178212.125532:0:6256:0:(ldlm_lockd.c:1903:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: mdt-lustre-MDT0000_UUID lock: 000000006b0afb52/0xa40083ab9bf538a5 lrc: 2/0,0 mode: --/CW res: [0x2000013a1:0x9:0x0].0x0 bits 0x2/0x0 rrc: 5 type: IBT gid 0 flags: 0x44a19400000000 nid: local remote: 0x0 expref: -99 pid: 6256 timeout: 0 lvb_type: 0 00010000:00010000:1.0:1776178212.125538:0:6256:0:(ldlm_lock.c:216:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: mdt-lustre-MDT0000_UUID lock: 000000006b0afb52/0xa40083ab9bf538a5 lrc: 0/0,0 mode: --/CW res: [0x2000013a1:0x9:0x0].0x0 bits 0x2/0x0 rrc: 5 type: IBT gid 0 flags: 0x44a19400000000 nid: local remote: 0x0 expref: -99 pid: 6256 timeout: 0 lvb_type: 0 00000004:00010000:1.0:1776178212.125565:0:6256:0:(mdt_handler.c:3992:mdt_save_lock()) ### save lock request 00000000a08c40f7 reply state 00000000aab818bc transno 12884901955 ns: mdt-lustre-MDT0000_UUID lock: 00000000a14e680b/0xa40083ab9bf538ac lrc: 3/0,1 mode: PW/PW res: [0x2000013a1:0x9:0x0].0x15fcbbea bits 0x2/0x0 rrc: 2 type: IBT gid 0 flags: 0x40210000000000 nid: local remote: 0x0 expref: -99 pid: 6256 timeout: 0 lvb_type: 0 00010000:00010000:1.0:1776178212.125593:0:6256:0:(ldlm_lock.c:216:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: mdt-lustre-MDT0000_UUID lock: 0000000097975a3b/0xa40083ab9bf5389e lrc: 0/0,0 mode: --/CW res: [0x2000013a1:0x9:0x0].0x0 bits 0x1/0x0 rrc: 4 type: IBT gid 0 flags: 0x44000000000000 nid: 192.168.204.3@tcp remote: 0x771f68aa1eb27bea expref: 22 pid: 6256 timeout: 0 lvb_type: 0 00010000:00010000:1.0:1776178212.125604:0:6256:0:(ldlm_lockd.c:1484:ldlm_handle_enqueue0()) ### server-side enqueue handler, sending reply (err=0, rc=0) ns: mdt-lustre-MDT0000_UUID lock: 000000001d465f75/0xa40083ab9bf538b3 lrc: 3/0,0 mode: CR/CR res: [0x2000013a1:0xb:0x0].0x0 bits 0x9/0x0 rrc: 2 type: IBT gid 0 flags: 0x40200000000000 nid: 192.168.204.3@tcp remote: 0x771f68aa1eb27bea expref: 21 pid: 6256 timeout: 0 lvb_type: 0 00010000:00010000:1.0:1776178212.125611:0:6256:0:(ldlm_inodebits.c:94:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x2000013a1:0xb:0x0].0x0 (000000003275315d) 00010000:00010000:1.0:1776178212.125613:0:6256:0:(ldlm_inodebits.c:96:ldlm_reprocess_inodebits_queue()) Hint 9 00010000:00010000:1.0:1776178212.125614:0:6256:0:(ldlm_lockd.c:1565:ldlm_handle_enqueue0()) ### server-side enqueue handler END (lock 000000001d465f75, rc 0) 00000100:00100000:1.0:1776178212.125688:0:6256:0:(service.c:2372:ptlrpc_server_handle_request()) Handled RPC req@00000000a08c40f7 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_001:b50963bd-8eba-4432-9a54-8888d00e3fa1+22:18258:x1862456979104576:12345-192.168.204.3@tcp:101:touch.0 Request processed in 1152us (1354us total) trans 12884901955 rc 0/0 00000100:00100000:1.0:1776178212.125696:0:6256:0:(nrs_fifo.c:241:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.204.3@tcp, seq: 6604 00000100:00100000:2.0:1776178212.135623:0:3348:0:(events.c:361:request_in_callback()) peer: 12345-192.168.204.3@tcp (source: 12345-192.168.204.3@tcp) 00000100:00100000:1.0:1776178212.135721:0:6256:0:(service.c:2152:ptlrpc_server_handle_req_in()) got req x1862456979104704 00000100:00100000:1.0:1776178212.135783:0:6256:0:(nrs_fifo.c:179:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.204.3@tcp, seq: 6605 00000100:00100000:1.0:1776178212.135791:0:6256:0:(service.c:2324:ptlrpc_server_handle_request()) Handling RPC req@00000000ea88a302 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_001:b50963bd-8eba-4432-9a54-8888d00e3fa1+22:18258:x1862456979104704:12345-192.168.204.3@tcp:36:touch.0 00010000:00010000:1.0:1776178212.135911:0:6256:0:(ldlm_lock.c:766:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: mdt-lustre-MDT0000_UUID lock: 0000000097975a3b/0xa40083ab9bf538c1 lrc: 3/0,1 mode: --/PW res: [0x2000013a1:0xb:0x0].0x0 bits 0x0/0x0 rrc: 3 type: IBT gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 6256 timeout: 0 lvb_type: 0 00010000:00010000:1.0:1776178212.135932:0:6256:0:(ldlm_lock.c:1064:ldlm_granted_list_add_lock()) ### About to add lock: ns: mdt-lustre-MDT0000_UUID lock: 0000000097975a3b/0xa40083ab9bf538c1 lrc: 3/0,1 mode: PW/PW res: [0x2000013a1:0xb:0x0].0x0 bits 0x2/0x0 rrc: 3 type: IBT gid 0 flags: 0x50210000000000 nid: local remote: 0x0 expref: -99 pid: 6256 timeout: 0 lvb_type: 0 00010000:00010000:1.0:1776178212.135945:0:6256:0:(ldlm_request.c:528:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: mdt-lustre-MDT0000_UUID lock: 0000000097975a3b/0xa40083ab9bf538c1 lrc: 3/0,1 mode: PW/PW res: [0x2000013a1:0xb:0x0].0x0 bits 0x2/0x0 rrc: 3 type: IBT gid 0 flags: 0x40210000000000 nid: local remote: 0x0 expref: -99 pid: 6256 timeout: 0 lvb_type: 0 00000004:00010000:1.0:1776178212.136100:0:6256:0:(mdt_handler.c:3992:mdt_save_lock()) ### save lock request 00000000ea88a302 reply state 00000000abf3876e transno 12884901956 ns: mdt-lustre-MDT0000_UUID lock: 0000000097975a3b/0xa40083ab9bf538c1 lrc: 3/0,1 mode: PW/PW res: [0x2000013a1:0xb:0x0].0x0 bits 0x2/0x0 rrc: 3 type: IBT gid 0 flags: 0x40210000000000 nid: local remote: 0x0 expref: -99 pid: 6256 timeout: 0 lvb_type: 0 00000100:00100000:1.0:1776178212.136198:0:6256:0:(service.c:2372:ptlrpc_server_handle_request()) Handled RPC req@00000000ea88a302 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_001:b50963bd-8eba-4432-9a54-8888d00e3fa1+24:18258:x1862456979104704:12345-192.168.204.3@tcp:36:touch.0 Request processed in 406us (585us total) trans 12884901956 rc 0/0 00000100:00100000:1.0:1776178212.136205:0:6256:0:(nrs_fifo.c:241:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.204.3@tcp, seq: 6605 00010000:00010000:0.0:1776178212.137051:0:3339:0:(ldlm_lock.c:1064:ldlm_granted_list_add_lock()) ### About to add lock: ns: mdt-lustre-MDT0000_UUID lock: 00000000a14e680b/0xa40083ab9bf538ac lrc: 3/0,1 mode: COS/COS res: [0x2000013a1:0x9:0x0].0x15fcbbea bits 0x2/0x0 rrc: 2 type: IBT gid 0 flags: 0x50210000000000 nid: local remote: 0x0 expref: -99 pid: 6256 timeout: 0 lvb_type: 0 00010000:00010000:0.0:1776178212.137092:0:3339:0:(ldlm_inodebits.c:94:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x2000013a1:0x9:0x0].0x15fcbbea (000000001fd37005) 00010000:00010000:0.0:1776178212.137095:0:3339:0:(ldlm_inodebits.c:96:ldlm_reprocess_inodebits_queue()) Hint 2 00000100:00100000:3.0:1776178212.139349:0:3346:0:(events.c:361:request_in_callback()) peer: 12345-192.168.204.3@tcp (source: 12345-192.168.204.3@tcp) 00000100:00100000:1.0:1776178212.139461:0:20755:0:(service.c:2152:ptlrpc_server_handle_req_in()) got req x1862456979104768 00000100:00100000:1.0:1776178212.139527:0:20755:0:(nrs_fifo.c:179:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.204.3@tcp, seq: 5286 00000100:00100000:1.0:1776178212.139536:0:20755:0:(service.c:2324:ptlrpc_server_handle_request()) Handling RPC req@00000000931093aa pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_006:b50963bd-8eba-4432-9a54-8888d00e3fa1+5:2273:x1862456979104768:12345-192.168.204.3@tcp:2:touch.0 00000100:00100000:1.0:1776178212.139818:0:20755:0:(service.c:2372:ptlrpc_server_handle_request()) Handled RPC req@00000000931093aa pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_006:b50963bd-8eba-4432-9a54-8888d00e3fa1+6:2273:x1862456979104768:12345-192.168.204.3@tcp:2:touch.0 Request processed in 281us (478us total) trans 4294969362 rc 0/0 00000100:00100000:1.0:1776178212.139826:0:20755:0:(nrs_fifo.c:241:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.204.3@tcp, seq: 5286 00010000:00010000:0.0:1776178212.140335:0:3340:0:(ldlm_lock.c:1064:ldlm_granted_list_add_lock()) ### About to add lock: ns: mdt-lustre-MDT0000_UUID lock: 0000000097975a3b/0xa40083ab9bf538c1 lrc: 3/0,1 mode: COS/COS res: [0x2000013a1:0xb:0x0].0x0 bits 0x2/0x0 rrc: 3 type: IBT gid 0 flags: 0x50210000000000 nid: local remote: 0x0 expref: -99 pid: 6256 timeout: 0 lvb_type: 0 00010000:00010000:0.0:1776178212.140350:0:3340:0:(ldlm_inodebits.c:94:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x2000013a1:0xb:0x0].0x0 (000000003275315d) 00010000:00010000:0.0:1776178212.140353:0:3340:0:(ldlm_inodebits.c:96:ldlm_reprocess_inodebits_queue()) Hint 2 00000100:00100000:3.0:1776178212.142170:0:3347:0:(events.c:361:request_in_callback()) peer: 12345-192.168.204.3@tcp (source: 12345-192.168.204.3@tcp) 00000100:00100000:1.0:1776178212.142718:0:6258:0:(service.c:2152:ptlrpc_server_handle_req_in()) got req x1862456979104896 00000100:00100000:1.0:1776178212.142782:0:6258:0:(nrs_fifo.c:179:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.204.3@tcp, seq: 2149 00000100:00100000:1.0:1776178212.142789:0:6258:0:(service.c:2324:ptlrpc_server_handle_request()) Handling RPC req@0000000058333a54 pname:cluuid+ref:pid:xid:nid:opc:job mdt_rdpg00_000:b50963bd-8eba-4432-9a54-8888d00e3fa1+24:18258:x1862456979104896:12345-192.168.204.3@tcp:35:touch.0 00000100:00100000:1.0:1776178212.143596:0:6258:0:(service.c:2372:ptlrpc_server_handle_request()) Handled RPC req@0000000058333a54 pname:cluuid+ref:pid:xid:nid:opc:job mdt_rdpg00_000:b50963bd-8eba-4432-9a54-8888d00e3fa1+25:18258:x1862456979104896:12345-192.168.204.3@tcp:35:touch.0 Request processed in 803us (1430us total) trans 12884901957 rc 0/0 00000100:00100000:1.0:1776178212.143616:0:6258:0:(nrs_fifo.c:241:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.204.3@tcp, seq: 2149 00000100:00100000:3.0:1776178212.158440:0:3348:0:(events.c:361:request_in_callback()) peer: 12345-192.168.204.3@tcp (source: 12345-192.168.204.3@tcp) 00000100:00100000:2.0:1776178212.158723:0:6257:0:(service.c:2152:ptlrpc_server_handle_req_in()) got req x1862456979105024 00000100:00100000:2.0:1776178212.158799:0:6257:0:(nrs_fifo.c:179:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.204.3@tcp, seq: 6606 00000100:00100000:2.0:1776178212.158808:0:6257:0:(service.c:2324:ptlrpc_server_handle_request()) Handling RPC req@00000000f093b599 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_002:b50963bd-8eba-4432-9a54-8888d00e3fa1+25:18259:x1862456979105024:12345-192.168.204.3@tcp:101:touch.0 00010000:00010000:2.0:1776178212.158841:0:6257:0:(ldlm_lockd.c:1248:ldlm_handle_enqueue0()) ### server-side enqueue handler START 00010000:00010000:2.0:1776178212.158870:0:6257:0:(ldlm_lockd.c:1318:ldlm_handle_enqueue0()) ### server-side enqueue handler, new lock created ns: mdt-lustre-MDT0000_UUID lock: 00000000c9d7d9fc/0xa40083ab9bf538c8 lrc: 2/0,0 mode: --/CW res: [0x2000013a1:0x9:0x0].0x0 bits 0x0/0x0 rrc: 4 type: IBT gid 0 flags: 0x40000000000000 nid: local remote: 0x771f68aa1eb27bf1 expref: -99 pid: 6257 timeout: 0 lvb_type: 0 00010000:00010000:2.0:1776178212.158989:0:6257:0:(ldlm_lock.c:766:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(CW) ns: mdt-lustre-MDT0000_UUID lock: 000000000b5c0143/0xa40083ab9bf538cf lrc: 3/0,1 mode: --/CW res: [0x2000013a1:0x9:0x0].0x0 bits 0x0/0x0 rrc: 5 type: IBT gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 6257 timeout: 0 lvb_type: 0 00010000:00010000:2.0:1776178212.159005:0:6257:0:(ldlm_lock.c:1064:ldlm_granted_list_add_lock()) ### About to add lock: ns: mdt-lustre-MDT0000_UUID lock: 000000000b5c0143/0xa40083ab9bf538cf lrc: 3/0,1 mode: CW/CW res: [0x2000013a1:0x9:0x0].0x0 bits 0x2/0x0 rrc: 5 type: IBT gid 0 flags: 0x50210000000000 nid: local remote: 0x0 expref: -99 pid: 6257 timeout: 0 lvb_type: 0 00010000:00010000:2.0:1776178212.159017:0:6257:0:(ldlm_request.c:528:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: mdt-lustre-MDT0000_UUID lock: 000000000b5c0143/0xa40083ab9bf538cf lrc: 3/0,1 mode: CW/CW res: [0x2000013a1:0x9:0x0].0x0 bits 0x2/0x0 rrc: 5 type: IBT gid 0 flags: 0x40210000000000 nid: local remote: 0x0 expref: -99 pid: 6257 timeout: 0 lvb_type: 0 00010000:00010000:2.0:1776178212.159032:0:6257:0:(ldlm_lock.c:766:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: mdt-lustre-MDT0000_UUID lock: 0000000096c1e297/0xa40083ab9bf538d6 lrc: 3/0,1 mode: --/PW res: [0x2000013a1:0x9:0x0].0x5e5674ea bits 0x0/0x0 rrc: 2 type: IBT gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 6257 timeout: 0 lvb_type: 0 00010000:00010000:2.0:1776178212.159039:0:6257:0:(ldlm_lock.c:1064:ldlm_granted_list_add_lock()) ### About to add lock: ns: mdt-lustre-MDT0000_UUID lock: 0000000096c1e297/0xa40083ab9bf538d6 lrc: 3/0,1 mode: PW/PW res: [0x2000013a1:0x9:0x0].0x5e5674ea bits 0x2/0x0 rrc: 2 type: IBT gid 0 flags: 0x50210000000000 nid: local remote: 0x0 expref: -99 pid: 6257 timeout: 0 lvb_type: 0 00010000:00010000:2.0:1776178212.159045:0:6257:0:(ldlm_request.c:528:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: mdt-lustre-MDT0000_UUID lock: 0000000096c1e297/0xa40083ab9bf538d6 lrc: 3/0,1 mode: PW/PW res: [0x2000013a1:0x9:0x0].0x5e5674ea bits 0x2/0x0 rrc: 2 type: IBT gid 0 flags: 0x40210000000000 nid: local remote: 0x0 expref: -99 pid: 6257 timeout: 0 lvb_type: 0 00000004:00080000:2.0:1776178212.159756:0:6257:0:(osp_object.c:1645:osp_create()) lustre-OST0001-osc-MDT0000: Wrote last used FID: [0x100010000:0x46:0x0], index 1: 0 00010000:00010000:2.0:1776178212.159903:0:6257:0:(ldlm_lock.c:766:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(CR) ns: mdt-lustre-MDT0000_UUID lock: 000000001e6d7318/0xa40083ab9bf538dd lrc: 3/1,0 mode: --/CR res: [0x2000013a1:0xc:0x0].0x0 bits 0x0/0x0 rrc: 2 type: IBT gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 6257 timeout: 0 lvb_type: 0 00010000:00010000:2.0:1776178212.159915:0:6257:0:(ldlm_lock.c:1064:ldlm_granted_list_add_lock()) ### About to add lock: ns: mdt-lustre-MDT0000_UUID lock: 000000001e6d7318/0xa40083ab9bf538dd lrc: 3/1,0 mode: CR/CR res: [0x2000013a1:0xc:0x0].0x0 bits 0x9/0x0 rrc: 2 type: IBT gid 0 flags: 0x50210000000000 nid: local remote: 0x0 expref: -99 pid: 6257 timeout: 0 lvb_type: 0 00010000:00010000:2.0:1776178212.159923:0:6257:0:(ldlm_request.c:528:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: mdt-lustre-MDT0000_UUID lock: 000000001e6d7318/0xa40083ab9bf538dd lrc: 3/1,0 mode: CR/CR res: [0x2000013a1:0xc:0x0].0x0 bits 0x9/0x0 rrc: 2 type: IBT gid 0 flags: 0x40210000000000 nid: local remote: 0x0 expref: -99 pid: 6257 timeout: 0 lvb_type: 0 00010000:00010000:2.0:1776178212.159959:0:6257:0:(ldlm_lock.c:820:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(CW) ns: mdt-lustre-MDT0000_UUID lock: 000000000b5c0143/0xa40083ab9bf538cf lrc: 3/0,1 mode: CW/CW res: [0x2000013a1:0x9:0x0].0x0 bits 0x2/0x0 rrc: 5 type: IBT gid 0 flags: 0x50210000000000 nid: local remote: 0x0 expref: -99 pid: 6257 timeout: 0 lvb_type: 0 00010000:00010000:2.0:1776178212.159966:0:6257:0:(ldlm_lock.c:879:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: mdt-lustre-MDT0000_UUID lock: 000000000b5c0143/0xa40083ab9bf538cf lrc: 2/0,0 mode: CW/CW res: [0x2000013a1:0x9:0x0].0x0 bits 0x2/0x0 rrc: 5 type: IBT gid 0 flags: 0x50210400000000 nid: local remote: 0x0 expref: -99 pid: 6257 timeout: 0 lvb_type: 0 00010000:00010000:2.0:1776178212.159971:0:6257:0:(ldlm_lockd.c:1879:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: mdt-lustre-MDT0000_UUID lock: 000000000b5c0143/0xa40083ab9bf538cf lrc: 3/0,0 mode: CW/CW res: [0x2000013a1:0x9:0x0].0x0 bits 0x2/0x0 rrc: 5 type: IBT gid 0 flags: 0x40210400000000 nid: local remote: 0x0 expref: -99 pid: 6257 timeout: 0 lvb_type: 0 00010000:00010000:2.0:1776178212.159978:0:6257:0:(ldlm_lockd.c:1893:ldlm_handle_bl_callback()) Lock 000000000b5c0143 already unused, calling callback (0000000079e78348) 00010000:00010000:2.0:1776178212.159981:0:6257:0:(ldlm_request.c:364:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: mdt-lustre-MDT0000_UUID lock: 000000000b5c0143/0xa40083ab9bf538cf lrc: 3/0,0 mode: CW/CW res: [0x2000013a1:0x9:0x0].0x0 bits 0x2/0x0 rrc: 5 type: IBT gid 0 flags: 0x40210400000000 nid: local remote: 0x0 expref: -99 pid: 6257 timeout: 0 lvb_type: 0 00010000:00010000:2.0:1776178212.159988:0:6257:0:(ldlm_request.c:1257:ldlm_cli_cancel_local()) ### server-side local cancel ns: mdt-lustre-MDT0000_UUID lock: 000000000b5c0143/0xa40083ab9bf538cf lrc: 4/0,0 mode: CW/CW res: [0x2000013a1:0x9:0x0].0x0 bits 0x2/0x0 rrc: 5 type: IBT gid 0 flags: 0x40218400000000 nid: local remote: 0x0 expref: -99 pid: 6257 timeout: 0 lvb_type: 0 00010000:00010000:2.0:1776178212.159999:0:6257:0:(ldlm_inodebits.c:94:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x2000013a1:0x9:0x0].0x0 (00000000a041a513) 00010000:00010000:2.0:1776178212.160002:0:6257:0:(ldlm_inodebits.c:96:ldlm_reprocess_inodebits_queue()) Hint 2 00010000:00010000:2.0:1776178212.160005:0:6257:0:(ldlm_lockd.c:1903:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: mdt-lustre-MDT0000_UUID lock: 000000000b5c0143/0xa40083ab9bf538cf lrc: 2/0,0 mode: --/CW res: [0x2000013a1:0x9:0x0].0x0 bits 0x2/0x0 rrc: 5 type: IBT gid 0 flags: 0x44a19400000000 nid: local remote: 0x0 expref: -99 pid: 6257 timeout: 0 lvb_type: 0 00010000:00010000:2.0:1776178212.160011:0:6257:0:(ldlm_lock.c:216:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: mdt-lustre-MDT0000_UUID lock: 000000000b5c0143/0xa40083ab9bf538cf lrc: 0/0,0 mode: --/CW res: [0x2000013a1:0x9:0x0].0x0 bits 0x2/0x0 rrc: 5 type: IBT gid 0 flags: 0x44a19400000000 nid: local remote: 0x0 expref: -99 pid: 6257 timeout: 0 lvb_type: 0 00000004:00010000:2.0:1776178212.160024:0:6257:0:(mdt_handler.c:3992:mdt_save_lock()) ### save lock request 00000000f093b599 reply state 000000009c08aa57 transno 12884901958 ns: mdt-lustre-MDT0000_UUID lock: 0000000096c1e297/0xa40083ab9bf538d6 lrc: 3/0,1 mode: PW/PW res: [0x2000013a1:0x9:0x0].0x5e5674ea bits 0x2/0x0 rrc: 2 type: IBT gid 0 flags: 0x40210000000000 nid: local remote: 0x0 expref: -99 pid: 6257 timeout: 0 lvb_type: 0 00010000:00010000:2.0:1776178212.160053:0:6257:0:(ldlm_lock.c:216:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: mdt-lustre-MDT0000_UUID lock: 00000000c9d7d9fc/0xa40083ab9bf538c8 lrc: 0/0,0 mode: --/CW res: [0x2000013a1:0x9:0x0].0x0 bits 0x1/0x0 rrc: 4 type: IBT gid 0 flags: 0x44000000000000 nid: 192.168.204.3@tcp remote: 0x771f68aa1eb27bf1 expref: 28 pid: 6257 timeout: 0 lvb_type: 0 00010000:00010000:2.0:1776178212.160414:0:6257:0:(ldlm_lockd.c:1484:ldlm_handle_enqueue0()) ### server-side enqueue handler, sending reply (err=0, rc=0) ns: mdt-lustre-MDT0000_UUID lock: 000000001e6d7318/0xa40083ab9bf538dd lrc: 3/0,0 mode: CR/CR res: [0x2000013a1:0xc:0x0].0x0 bits 0x9/0x0 rrc: 2 type: IBT gid 0 flags: 0x40200000000000 nid: 192.168.204.3@tcp remote: 0x771f68aa1eb27bf1 expref: 27 pid: 6257 timeout: 0 lvb_type: 0 00010000:00010000:2.0:1776178212.160422:0:6257:0:(ldlm_inodebits.c:94:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x2000013a1:0xc:0x0].0x0 (0000000062690614) 00010000:00010000:2.0:1776178212.160424:0:6257:0:(ldlm_inodebits.c:96:ldlm_reprocess_inodebits_queue()) Hint 9 00010000:00010000:2.0:1776178212.160426:0:6257:0:(ldlm_lockd.c:1565:ldlm_handle_enqueue0()) ### server-side enqueue handler END (lock 000000001e6d7318, rc 0) 00000100:00100000:2.0:1776178212.160572:0:6257:0:(service.c:2372:ptlrpc_server_handle_request()) Handled RPC req@00000000f093b599 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_002:b50963bd-8eba-4432-9a54-8888d00e3fa1+28:18259:x1862456979105024:12345-192.168.204.3@tcp:101:touch.0 Request processed in 1763us (2141us total) trans 12884901958 rc 0/0 00000100:00100000:2.0:1776178212.160581:0:6257:0:(nrs_fifo.c:241:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.204.3@tcp, seq: 6606 00010000:00010000:0.0:1776178212.170818:0:3339:0:(ldlm_lock.c:1064:ldlm_granted_list_add_lock()) ### About to add lock: ns: mdt-lustre-MDT0000_UUID lock: 0000000096c1e297/0xa40083ab9bf538d6 lrc: 3/0,1 mode: COS/COS res: [0x2000013a1:0x9:0x0].0x5e5674ea bits 0x2/0x0 rrc: 2 type: IBT gid 0 flags: 0x50210000000000 nid: local remote: 0x0 expref: -99 pid: 6257 timeout: 0 lvb_type: 0 00010000:00010000:0.0:1776178212.170996:0:3339:0:(ldlm_inodebits.c:94:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x2000013a1:0x9:0x0].0x5e5674ea (00000000024b4e8f) 00010000:00010000:0.0:1776178212.171001:0:3339:0:(ldlm_inodebits.c:96:ldlm_reprocess_inodebits_queue()) Hint 2 00000100:00100000:3.0:1776178212.182881:0:3348:0:(events.c:361:request_in_callback()) peer: 12345-192.168.204.3@tcp (source: 12345-192.168.204.3@tcp) 00000100:00100000:2.0:1776178212.183024:0:6257:0:(service.c:2152:ptlrpc_server_handle_req_in()) got req x1862456979105152 00000100:00100000:2.0:1776178212.183159:0:6257:0:(nrs_fifo.c:179:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.204.3@tcp, seq: 6607 00000100:00100000:2.0:1776178212.183167:0:6257:0:(service.c:2324:ptlrpc_server_handle_request()) Handling RPC req@00000000779279f4 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_002:b50963bd-8eba-4432-9a54-8888d00e3fa1+28:18259:x1862456979105152:12345-192.168.204.3@tcp:36:touch.0 00010000:00010000:2.0:1776178212.183294:0:6257:0:(ldlm_lock.c:766:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: mdt-lustre-MDT0000_UUID lock: 00000000c9d7d9fc/0xa40083ab9bf538eb lrc: 3/0,1 mode: --/PW res: [0x2000013a1:0xc:0x0].0x0 bits 0x0/0x0 rrc: 3 type: IBT gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 6257 timeout: 0 lvb_type: 0 00010000:00010000:2.0:1776178212.183313:0:6257:0:(ldlm_lock.c:1064:ldlm_granted_list_add_lock()) ### About to add lock: ns: mdt-lustre-MDT0000_UUID lock: 00000000c9d7d9fc/0xa40083ab9bf538eb lrc: 3/0,1 mode: PW/PW res: [0x2000013a1:0xc:0x0].0x0 bits 0x2/0x0 rrc: 3 type: IBT gid 0 flags: 0x50210000000000 nid: local remote: 0x0 expref: -99 pid: 6257 timeout: 0 lvb_type: 0 00010000:00010000:2.0:1776178212.183331:0:6257:0:(ldlm_request.c:528:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: mdt-lustre-MDT0000_UUID lock: 00000000c9d7d9fc/0xa40083ab9bf538eb lrc: 3/0,1 mode: PW/PW res: [0x2000013a1:0xc:0x0].0x0 bits 0x2/0x0 rrc: 3 type: IBT gid 0 flags: 0x40210000000000 nid: local remote: 0x0 expref: -99 pid: 6257 timeout: 0 lvb_type: 0 00000004:00010000:2.0:1776178212.183438:0:6257:0:(mdt_handler.c:3992:mdt_save_lock()) ### save lock request 00000000779279f4 reply state 00000000340c9c32 transno 12884901959 ns: mdt-lustre-MDT0000_UUID lock: 00000000c9d7d9fc/0xa40083ab9bf538eb lrc: 3/0,1 mode: PW/PW res: [0x2000013a1:0xc:0x0].0x0 bits 0x2/0x0 rrc: 3 type: IBT gid 0 flags: 0x40210000000000 nid: local remote: 0x0 expref: -99 pid: 6257 timeout: 0 lvb_type: 0 00000100:00100000:2.0:1776178212.183531:0:6257:0:(service.c:2372:ptlrpc_server_handle_request()) Handled RPC req@00000000779279f4 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_002:b50963bd-8eba-4432-9a54-8888d00e3fa1+30:18259:x1862456979105152:12345-192.168.204.3@tcp:36:touch.0 Request processed in 363us (662us total) trans 12884901959 rc 0/0 00000100:00100000:2.0:1776178212.183540:0:6257:0:(nrs_fifo.c:241:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.204.3@tcp, seq: 6607 00010000:00010000:0.0:1776178212.186313:0:3340:0:(ldlm_lock.c:1064:ldlm_granted_list_add_lock()) ### About to add lock: ns: mdt-lustre-MDT0000_UUID lock: 00000000c9d7d9fc/0xa40083ab9bf538eb lrc: 3/0,1 mode: COS/COS res: [0x2000013a1:0xc:0x0].0x0 bits 0x2/0x0 rrc: 3 type: IBT gid 0 flags: 0x50210000000000 nid: local remote: 0x0 expref: -99 pid: 6257 timeout: 0 lvb_type: 0 00010000:00010000:0.0:1776178212.186338:0:3340:0:(ldlm_inodebits.c:94:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x2000013a1:0xc:0x0].0x0 (0000000062690614) 00010000:00010000:0.0:1776178212.186341:0:3340:0:(ldlm_inodebits.c:96:ldlm_reprocess_inodebits_queue()) Hint 2 00000100:00100000:3.0:1776178212.187690:0:3348:0:(events.c:361:request_in_callback()) peer: 12345-192.168.204.3@tcp (source: 12345-192.168.204.3@tcp) 00000100:00100000:2.0:1776178212.187995:0:20762:0:(service.c:2152:ptlrpc_server_handle_req_in()) got req x1862456979105216 00000100:00100000:2.0:1776178212.188051:0:20762:0:(nrs_fifo.c:179:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.204.3@tcp, seq: 5287 00000100:00100000:2.0:1776178212.188092:0:20762:0:(service.c:2324:ptlrpc_server_handle_request()) Handling RPC req@00000000a3697847 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_013:b50963bd-8eba-4432-9a54-8888d00e3fa1+6:2272:x1862456979105216:12345-192.168.204.3@tcp:2:touch.0 00000100:00100000:2.0:1776178212.188313:0:20762:0:(service.c:2372:ptlrpc_server_handle_request()) Handled RPC req@00000000a3697847 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_013:b50963bd-8eba-4432-9a54-8888d00e3fa1+7:2272:x1862456979105216:12345-192.168.204.3@tcp:2:touch.0 Request processed in 253us (629us total) trans 4294969337 rc 0/0 00000100:00100000:2.0:1776178212.188321:0:20762:0:(nrs_fifo.c:241:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.204.3@tcp, seq: 5287 00000100:00100000:2.0:1776178212.190710:0:3346:0:(events.c:361:request_in_callback()) peer: 12345-192.168.204.3@tcp (source: 12345-192.168.204.3@tcp) 00000100:00100000:1.0:1776178212.190816:0:6258:0:(service.c:2152:ptlrpc_server_handle_req_in()) got req x1862456979105344 00000100:00100000:1.0:1776178212.190968:0:6258:0:(nrs_fifo.c:179:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.204.3@tcp, seq: 2150 00000100:00100000:1.0:1776178212.190976:0:6258:0:(service.c:2324:ptlrpc_server_handle_request()) Handling RPC req@000000000cc3fe7b pname:cluuid+ref:pid:xid:nid:opc:job mdt_rdpg00_000:b50963bd-8eba-4432-9a54-8888d00e3fa1+30:18259:x1862456979105344:12345-192.168.204.3@tcp:35:touch.0 00000100:00100000:1.0:1776178212.191306:0:6258:0:(service.c:2372:ptlrpc_server_handle_request()) Handled RPC req@000000000cc3fe7b pname:cluuid+ref:pid:xid:nid:opc:job mdt_rdpg00_000:b50963bd-8eba-4432-9a54-8888d00e3fa1+31:18259:x1862456979105344:12345-192.168.204.3@tcp:35:touch.0 Request processed in 330us (603us total) trans 12884901960 rc 0/0 00000100:00100000:1.0:1776178212.191314:0:6258:0:(nrs_fifo.c:241:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.204.3@tcp, seq: 2150 00000100:00100000:2.0:1776178212.195693:0:3353:0:(client.c:1733:ptlrpc_send_new_req()) Sending RPC req@000000008ee413f4 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_00:lustre-MDT0001-mdtlov_UUID:3353:1862456998049536:0@lo:13:osp-pre-1-1.0 00000100:00100000:2.0:1776178212.195770:0:3353:0:(events.c:361:request_in_callback()) peer: 12345-0@lo (source: 12345-0@lo) 00000100:00100000:1.0:1776178212.195954:0:8232:0:(service.c:2152:ptlrpc_server_handle_req_in()) got req x1862456998049536 00000100:00100000:1.0:1776178212.196016:0:8232:0:(nrs_fifo.c:179:nrs_fifo_req_get()) NRS start fifo request from 12345-0@lo, seq: 485 00000100:00100000:1.0:1776178212.196022:0:8232:0:(service.c:2324:ptlrpc_server_handle_request()) Handling RPC req@0000000029e5c385 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_create00:lustre-MDT0001-mdtlov_UUID+9:3353:x1862456998049536:12345-0@lo:13:osp-pre-1-1.0 00002000:00000024:1.0:1776178212.196052:0:8232:0:(ofd_obd.c:740:ofd_statfs()) blocks cached 229376 granted 491746368 pending 0 free 3923689472 avail 3692167168 00000100:00100000:1.0:1776178212.196123:0:8232:0:(service.c:2372:ptlrpc_server_handle_request()) Handled RPC req@0000000029e5c385 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_create00:lustre-MDT0001-mdtlov_UUID+9:3353:x1862456998049536:12345-0@lo:13:osp-pre-1-1.0 Request processed in 100us (354us total) trans 0 rc 0/0 00000100:00100000:1.0:1776178212.196130:0:8232:0:(nrs_fifo.c:241:nrs_fifo_req_stop()) NRS stop fifo request from 12345-0@lo, seq: 485 00000100:00100000:3.0:1776178212.196314:0:3353:0:(client.c:2215:ptlrpc_check_set()) Completed RPC req@000000008ee413f4 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_00:lustre-MDT0001-mdtlov_UUID:3353:1862456998049536:0@lo:13:osp-pre-1-1.0 00000100:00100000:0.0:1776178212.207327:0:3347:0:(events.c:361:request_in_callback()) peer: 12345-192.168.204.3@tcp (source: 12345-192.168.204.3@tcp) 00000100:00100000:1.0:1776178212.209220:0:6256:0:(service.c:2152:ptlrpc_server_handle_req_in()) got req x1862456979105472 00000100:00100000:1.0:1776178212.209309:0:6256:0:(nrs_fifo.c:179:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.204.3@tcp, seq: 6608 00000100:00100000:1.0:1776178212.209318:0:6256:0:(service.c:2324:ptlrpc_server_handle_request()) Handling RPC req@00000000a59bfc89 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_001:b50963bd-8eba-4432-9a54-8888d00e3fa1+31:18260:x1862456979105472:12345-192.168.204.3@tcp:101:touch.0 00010000:00010000:1.0:1776178212.209352:0:6256:0:(ldlm_lockd.c:1248:ldlm_handle_enqueue0()) ### server-side enqueue handler START 00010000:00010000:1.0:1776178212.209378:0:6256:0:(ldlm_lockd.c:1318:ldlm_handle_enqueue0()) ### server-side enqueue handler, new lock created ns: mdt-lustre-MDT0000_UUID lock: 000000006b0afb52/0xa40083ab9bf538f2 lrc: 2/0,0 mode: --/CW res: [0x2000013a1:0x9:0x0].0x0 bits 0x0/0x0 rrc: 4 type: IBT gid 0 flags: 0x40000000000000 nid: local remote: 0x771f68aa1eb27bf8 expref: -99 pid: 6256 timeout: 0 lvb_type: 0 00010000:00010000:1.0:1776178212.209510:0:6256:0:(ldlm_lock.c:766:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(CW) ns: mdt-lustre-MDT0000_UUID lock: 000000002abc4399/0xa40083ab9bf538f9 lrc: 3/0,1 mode: --/CW res: [0x2000013a1:0x9:0x0].0x0 bits 0x0/0x0 rrc: 5 type: IBT gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 6256 timeout: 0 lvb_type: 0 00010000:00010000:1.0:1776178212.209530:0:6256:0:(ldlm_lock.c:1064:ldlm_granted_list_add_lock()) ### About to add lock: ns: mdt-lustre-MDT0000_UUID lock: 000000002abc4399/0xa40083ab9bf538f9 lrc: 3/0,1 mode: CW/CW res: [0x2000013a1:0x9:0x0].0x0 bits 0x2/0x0 rrc: 5 type: IBT gid 0 flags: 0x50210000000000 nid: local remote: 0x0 expref: -99 pid: 6256 timeout: 0 lvb_type: 0 00010000:00010000:1.0:1776178212.209555:0:6256:0:(ldlm_request.c:528:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: mdt-lustre-MDT0000_UUID lock: 000000002abc4399/0xa40083ab9bf538f9 lrc: 3/0,1 mode: CW/CW res: [0x2000013a1:0x9:0x0].0x0 bits 0x2/0x0 rrc: 5 type: IBT gid 0 flags: 0x40210000000000 nid: local remote: 0x0 expref: -99 pid: 6256 timeout: 0 lvb_type: 0 00010000:00010000:1.0:1776178212.209573:0:6256:0:(ldlm_lock.c:766:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: mdt-lustre-MDT0000_UUID lock: 00000000b2e83690/0xa40083ab9bf53900 lrc: 3/0,1 mode: --/PW res: [0x2000013a1:0x9:0x0].0xa6b02dea bits 0x0/0x0 rrc: 2 type: IBT gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 6256 timeout: 0 lvb_type: 0 00010000:00010000:1.0:1776178212.209580:0:6256:0:(ldlm_lock.c:1064:ldlm_granted_list_add_lock()) ### About to add lock: ns: mdt-lustre-MDT0000_UUID lock: 00000000b2e83690/0xa40083ab9bf53900 lrc: 3/0,1 mode: PW/PW res: [0x2000013a1:0x9:0x0].0xa6b02dea bits 0x2/0x0 rrc: 2 type: IBT gid 0 flags: 0x50210000000000 nid: local remote: 0x0 expref: -99 pid: 6256 timeout: 0 lvb_type: 0 00010000:00010000:1.0:1776178212.209587:0:6256:0:(ldlm_request.c:528:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: mdt-lustre-MDT0000_UUID lock: 00000000b2e83690/0xa40083ab9bf53900 lrc: 3/0,1 mode: PW/PW res: [0x2000013a1:0x9:0x0].0xa6b02dea bits 0x2/0x0 rrc: 2 type: IBT gid 0 flags: 0x40210000000000 nid: local remote: 0x0 expref: -99 pid: 6256 timeout: 0 lvb_type: 0 00000004:00080000:1.0:1776178212.209923:0:6256:0:(osp_object.c:1645:osp_create()) lustre-OST0000-osc-MDT0000: Wrote last used FID: [0x100000000:0x47:0x0], index 0: 0 00010000:00010000:1.0:1776178212.210506:0:6256:0:(ldlm_lock.c:766:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(CR) ns: mdt-lustre-MDT0000_UUID lock: 0000000056462e0c/0xa40083ab9bf53907 lrc: 3/1,0 mode: --/CR res: [0x2000013a1:0xd:0x0].0x0 bits 0x0/0x0 rrc: 2 type: IBT gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 6256 timeout: 0 lvb_type: 0 00010000:00010000:1.0:1776178212.210518:0:6256:0:(ldlm_lock.c:1064:ldlm_granted_list_add_lock()) ### About to add lock: ns: mdt-lustre-MDT0000_UUID lock: 0000000056462e0c/0xa40083ab9bf53907 lrc: 3/1,0 mode: CR/CR res: [0x2000013a1:0xd:0x0].0x0 bits 0x9/0x0 rrc: 2 type: IBT gid 0 flags: 0x50210000000000 nid: local remote: 0x0 expref: -99 pid: 6256 timeout: 0 lvb_type: 0 00010000:00010000:1.0:1776178212.210526:0:6256:0:(ldlm_request.c:528:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: mdt-lustre-MDT0000_UUID lock: 0000000056462e0c/0xa40083ab9bf53907 lrc: 3/1,0 mode: CR/CR res: [0x2000013a1:0xd:0x0].0x0 bits 0x9/0x0 rrc: 2 type: IBT gid 0 flags: 0x40210000000000 nid: local remote: 0x0 expref: -99 pid: 6256 timeout: 0 lvb_type: 0 00010000:00010000:1.0:1776178212.210576:0:6256:0:(ldlm_lock.c:820:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(CW) ns: mdt-lustre-MDT0000_UUID lock: 000000002abc4399/0xa40083ab9bf538f9 lrc: 3/0,1 mode: CW/CW res: [0x2000013a1:0x9:0x0].0x0 bits 0x2/0x0 rrc: 5 type: IBT gid 0 flags: 0x50210000000000 nid: local remote: 0x0 expref: -99 pid: 6256 timeout: 0 lvb_type: 0 00010000:00010000:1.0:1776178212.210583:0:6256:0:(ldlm_lock.c:879:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: mdt-lustre-MDT0000_UUID lock: 000000002abc4399/0xa40083ab9bf538f9 lrc: 2/0,0 mode: CW/CW res: [0x2000013a1:0x9:0x0].0x0 bits 0x2/0x0 rrc: 5 type: IBT gid 0 flags: 0x50210400000000 nid: local remote: 0x0 expref: -99 pid: 6256 timeout: 0 lvb_type: 0 00010000:00010000:1.0:1776178212.210589:0:6256:0:(ldlm_lockd.c:1879:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: mdt-lustre-MDT0000_UUID lock: 000000002abc4399/0xa40083ab9bf538f9 lrc: 3/0,0 mode: CW/CW res: [0x2000013a1:0x9:0x0].0x0 bits 0x2/0x0 rrc: 5 type: IBT gid 0 flags: 0x40210400000000 nid: local remote: 0x0 expref: -99 pid: 6256 timeout: 0 lvb_type: 0 00010000:00010000:1.0:1776178212.210596:0:6256:0:(ldlm_lockd.c:1893:ldlm_handle_bl_callback()) Lock 000000002abc4399 already unused, calling callback (0000000079e78348) 00010000:00010000:1.0:1776178212.210600:0:6256:0:(ldlm_request.c:364:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: mdt-lustre-MDT0000_UUID lock: 000000002abc4399/0xa40083ab9bf538f9 lrc: 3/0,0 mode: CW/CW res: [0x2000013a1:0x9:0x0].0x0 bits 0x2/0x0 rrc: 5 type: IBT gid 0 flags: 0x40210400000000 nid: local remote: 0x0 expref: -99 pid: 6256 timeout: 0 lvb_type: 0 00010000:00010000:1.0:1776178212.210607:0:6256:0:(ldlm_request.c:1257:ldlm_cli_cancel_local()) ### server-side local cancel ns: mdt-lustre-MDT0000_UUID lock: 000000002abc4399/0xa40083ab9bf538f9 lrc: 4/0,0 mode: CW/CW res: [0x2000013a1:0x9:0x0].0x0 bits 0x2/0x0 rrc: 5 type: IBT gid 0 flags: 0x40218400000000 nid: local remote: 0x0 expref: -99 pid: 6256 timeout: 0 lvb_type: 0 00010000:00010000:1.0:1776178212.210618:0:6256:0:(ldlm_inodebits.c:94:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x2000013a1:0x9:0x0].0x0 (00000000a041a513) 00010000:00010000:1.0:1776178212.210620:0:6256:0:(ldlm_inodebits.c:96:ldlm_reprocess_inodebits_queue()) Hint 2 00010000:00010000:1.0:1776178212.210625:0:6256:0:(ldlm_lockd.c:1903:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: mdt-lustre-MDT0000_UUID lock: 000000002abc4399/0xa40083ab9bf538f9 lrc: 2/0,0 mode: --/CW res: [0x2000013a1:0x9:0x0].0x0 bits 0x2/0x0 rrc: 5 type: IBT gid 0 flags: 0x44a19400000000 nid: local remote: 0x0 expref: -99 pid: 6256 timeout: 0 lvb_type: 0 00010000:00010000:1.0:1776178212.210630:0:6256:0:(ldlm_lock.c:216:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: mdt-lustre-MDT0000_UUID lock: 000000002abc4399/0xa40083ab9bf538f9 lrc: 0/0,0 mode: --/CW res: [0x2000013a1:0x9:0x0].0x0 bits 0x2/0x0 rrc: 5 type: IBT gid 0 flags: 0x44a19400000000 nid: local remote: 0x0 expref: -99 pid: 6256 timeout: 0 lvb_type: 0 00000004:00010000:1.0:1776178212.210644:0:6256:0:(mdt_handler.c:3992:mdt_save_lock()) ### save lock request 00000000a59bfc89 reply state 00000000a7faf7d1 transno 12884901961 ns: mdt-lustre-MDT0000_UUID lock: 00000000b2e83690/0xa40083ab9bf53900 lrc: 3/0,1 mode: PW/PW res: [0x2000013a1:0x9:0x0].0xa6b02dea bits 0x2/0x0 rrc: 2 type: IBT gid 0 flags: 0x40210000000000 nid: local remote: 0x0 expref: -99 pid: 6256 timeout: 0 lvb_type: 0 00010000:00010000:1.0:1776178212.210684:0:6256:0:(ldlm_lock.c:216:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: mdt-lustre-MDT0000_UUID lock: 000000006b0afb52/0xa40083ab9bf538f2 lrc: 0/0,0 mode: --/CW res: [0x2000013a1:0x9:0x0].0x0 bits 0x1/0x0 rrc: 4 type: IBT gid 0 flags: 0x44000000000000 nid: 192.168.204.3@tcp remote: 0x771f68aa1eb27bf8 expref: 34 pid: 6256 timeout: 0 lvb_type: 0 00010000:00010000:1.0:1776178212.210697:0:6256:0:(ldlm_lockd.c:1484:ldlm_handle_enqueue0()) ### server-side enqueue handler, sending reply (err=0, rc=0) ns: mdt-lustre-MDT0000_UUID lock: 0000000056462e0c/0xa40083ab9bf53907 lrc: 3/0,0 mode: CR/CR res: [0x2000013a1:0xd:0x0].0x0 bits 0x9/0x0 rrc: 2 type: IBT gid 0 flags: 0x40200000000000 nid: 192.168.204.3@tcp remote: 0x771f68aa1eb27bf8 expref: 33 pid: 6256 timeout: 0 lvb_type: 0 00010000:00010000:1.0:1776178212.210705:0:6256:0:(ldlm_inodebits.c:94:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x2000013a1:0xd:0x0].0x0 (00000000713d9096) 00010000:00010000:1.0:1776178212.210707:0:6256:0:(ldlm_inodebits.c:96:ldlm_reprocess_inodebits_queue()) Hint 9 00010000:00010000:1.0:1776178212.210709:0:6256:0:(ldlm_lockd.c:1565:ldlm_handle_enqueue0()) ### server-side enqueue handler END (lock 0000000056462e0c, rc 0) 00000100:00100000:1.0:1776178212.210808:0:6256:0:(service.c:2372:ptlrpc_server_handle_request()) Handled RPC req@00000000a59bfc89 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_001:b50963bd-8eba-4432-9a54-8888d00e3fa1+34:18260:x1862456979105472:12345-192.168.204.3@tcp:101:touch.0 Request processed in 1490us (3492us total) trans 12884901961 rc 0/0 00000100:00100000:1.0:1776178212.210818:0:6256:0:(nrs_fifo.c:241:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.204.3@tcp, seq: 6608 00000100:00100000:2.0:1776178212.221731:0:3348:0:(events.c:361:request_in_callback()) peer: 12345-192.168.204.3@tcp (source: 12345-192.168.204.3@tcp) 00010000:00010000:0.0:1776178212.221833:0:3339:0:(ldlm_lock.c:1064:ldlm_granted_list_add_lock()) ### About to add lock: ns: mdt-lustre-MDT0000_UUID lock: 00000000b2e83690/0xa40083ab9bf53900 lrc: 3/0,1 mode: COS/COS res: [0x2000013a1:0x9:0x0].0xa6b02dea bits 0x2/0x0 rrc: 2 type: IBT gid 0 flags: 0x50210000000000 nid: local remote: 0x0 expref: -99 pid: 6256 timeout: 0 lvb_type: 0 00010000:00010000:0.0:1776178212.221862:0:3339:0:(ldlm_inodebits.c:94:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x2000013a1:0x9:0x0].0xa6b02dea (0000000057f156ea) 00010000:00010000:0.0:1776178212.221865:0:3339:0:(ldlm_inodebits.c:96:ldlm_reprocess_inodebits_queue()) Hint 2 00000100:00100000:3.0:1776178212.221936:0:6257:0:(service.c:2152:ptlrpc_server_handle_req_in()) got req x1862456979105600 00000100:00100000:3.0:1776178212.222002:0:6257:0:(nrs_fifo.c:179:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.204.3@tcp, seq: 6609 00000100:00100000:3.0:1776178212.222019:0:6257:0:(service.c:2324:ptlrpc_server_handle_request()) Handling RPC req@00000000d0d837c5 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_002:b50963bd-8eba-4432-9a54-8888d00e3fa1+34:18260:x1862456979105600:12345-192.168.204.3@tcp:36:touch.0 00010000:00010000:3.0:1776178212.222178:0:6257:0:(ldlm_lock.c:766:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: mdt-lustre-MDT0000_UUID lock: 00000000a3795b7d/0xa40083ab9bf53915 lrc: 3/0,1 mode: --/PW res: [0x2000013a1:0xd:0x0].0x0 bits 0x0/0x0 rrc: 3 type: IBT gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 6257 timeout: 0 lvb_type: 0 00010000:00010000:3.0:1776178212.222194:0:6257:0:(ldlm_lock.c:1064:ldlm_granted_list_add_lock()) ### About to add lock: ns: mdt-lustre-MDT0000_UUID lock: 00000000a3795b7d/0xa40083ab9bf53915 lrc: 3/0,1 mode: PW/PW res: [0x2000013a1:0xd:0x0].0x0 bits 0x2/0x0 rrc: 3 type: IBT gid 0 flags: 0x50210000000000 nid: local remote: 0x0 expref: -99 pid: 6257 timeout: 0 lvb_type: 0 00010000:00010000:3.0:1776178212.222206:0:6257:0:(ldlm_request.c:528:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: mdt-lustre-MDT0000_UUID lock: 00000000a3795b7d/0xa40083ab9bf53915 lrc: 3/0,1 mode: PW/PW res: [0x2000013a1:0xd:0x0].0x0 bits 0x2/0x0 rrc: 3 type: IBT gid 0 flags: 0x40210000000000 nid: local remote: 0x0 expref: -99 pid: 6257 timeout: 0 lvb_type: 0 00000004:00010000:3.0:1776178212.222342:0:6257:0:(mdt_handler.c:3992:mdt_save_lock()) ### save lock request 00000000d0d837c5 reply state 00000000a39a23b6 transno 12884901962 ns: mdt-lustre-MDT0000_UUID lock: 00000000a3795b7d/0xa40083ab9bf53915 lrc: 3/0,1 mode: PW/PW res: [0x2000013a1:0xd:0x0].0x0 bits 0x2/0x0 rrc: 3 type: IBT gid 0 flags: 0x40210000000000 nid: local remote: 0x0 expref: -99 pid: 6257 timeout: 0 lvb_type: 0 00000100:00100000:3.0:1776178212.222447:0:6257:0:(service.c:2372:ptlrpc_server_handle_request()) Handled RPC req@00000000d0d837c5 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_002:b50963bd-8eba-4432-9a54-8888d00e3fa1+36:18260:x1862456979105600:12345-192.168.204.3@tcp:36:touch.0 Request processed in 428us (726us total) trans 12884901962 rc 0/0 00000100:00100000:3.0:1776178212.222456:0:6257:0:(nrs_fifo.c:241:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.204.3@tcp, seq: 6609 00010000:00010000:0.0:1776178212.229695:0:3340:0:(ldlm_lock.c:1064:ldlm_granted_list_add_lock()) ### About to add lock: ns: mdt-lustre-MDT0000_UUID lock: 00000000a3795b7d/0xa40083ab9bf53915 lrc: 3/0,1 mode: COS/COS res: [0x2000013a1:0xd:0x0].0x0 bits 0x2/0x0 rrc: 3 type: IBT gid 0 flags: 0x50210000000000 nid: local remote: 0x0 expref: -99 pid: 6257 timeout: 0 lvb_type: 0 00010000:00010000:0.0:1776178212.229720:0:3340:0:(ldlm_inodebits.c:94:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x2000013a1:0xd:0x0].0x0 (00000000713d9096) 00010000:00010000:0.0:1776178212.229723:0:3340:0:(ldlm_inodebits.c:96:ldlm_reprocess_inodebits_queue()) Hint 2 00000100:00100000:2.0:1776178212.235223:0:3348:0:(events.c:361:request_in_callback()) peer: 12345-192.168.204.3@tcp (source: 12345-192.168.204.3@tcp) 00000100:00100000:0.0:1776178212.235405:0:20762:0:(service.c:2152:ptlrpc_server_handle_req_in()) got req x1862456979105664 00000100:00100000:0.0:1776178212.235474:0:20762:0:(nrs_fifo.c:179:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.204.3@tcp, seq: 5288 00000100:00100000:0.0:1776178212.235483:0:20762:0:(service.c:2324:ptlrpc_server_handle_request()) Handling RPC req@000000004828c19d pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_013:b50963bd-8eba-4432-9a54-8888d00e3fa1+6:2271:x1862456979105664:12345-192.168.204.3@tcp:2:touch.0 00000100:00100000:0.0:1776178212.235755:0:20762:0:(service.c:2372:ptlrpc_server_handle_request()) Handled RPC req@000000004828c19d pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_013:b50963bd-8eba-4432-9a54-8888d00e3fa1+7:2271:x1862456979105664:12345-192.168.204.3@tcp:2:touch.0 Request processed in 271us (543us total) trans 4294969363 rc 0/0 00000100:00100000:0.0:1776178212.235763:0:20762:0:(nrs_fifo.c:241:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.204.3@tcp, seq: 5288 00000100:00100000:3.0:1776178212.244717:0:3346:0:(events.c:361:request_in_callback()) peer: 12345-192.168.204.3@tcp (source: 12345-192.168.204.3@tcp) 00000100:00100000:1.0:1776178212.244851:0:6258:0:(service.c:2152:ptlrpc_server_handle_req_in()) got req x1862456979105792 00000100:00100000:1.0:1776178212.244930:0:6258:0:(nrs_fifo.c:179:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.204.3@tcp, seq: 2151 00000100:00100000:1.0:1776178212.244938:0:6258:0:(service.c:2324:ptlrpc_server_handle_request()) Handling RPC req@00000000d0aa9e90 pname:cluuid+ref:pid:xid:nid:opc:job mdt_rdpg00_000:b50963bd-8eba-4432-9a54-8888d00e3fa1+36:18260:x1862456979105792:12345-192.168.204.3@tcp:35:touch.0 00000100:00100000:1.0:1776178212.245353:0:6258:0:(service.c:2372:ptlrpc_server_handle_request()) Handled RPC req@00000000d0aa9e90 pname:cluuid+ref:pid:xid:nid:opc:job mdt_rdpg00_000:b50963bd-8eba-4432-9a54-8888d00e3fa1+37:18260:x1862456979105792:12345-192.168.204.3@tcp:35:touch.0 Request processed in 414us (644us total) trans 12884901963 rc 0/0 00000100:00100000:1.0:1776178212.245362:0:6258:0:(nrs_fifo.c:241:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.204.3@tcp, seq: 2151 00000100:00100000:3.0:1776178212.259972:0:3347:0:(events.c:361:request_in_callback()) peer: 12345-192.168.204.3@tcp (source: 12345-192.168.204.3@tcp) 00000100:00100000:1.0:1776178212.260170:0:6256:0:(service.c:2152:ptlrpc_server_handle_req_in()) got req x1862456979105920 00000100:00100000:1.0:1776178212.260251:0:6256:0:(nrs_fifo.c:179:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.204.3@tcp, seq: 6610 00000100:00100000:1.0:1776178212.260259:0:6256:0:(service.c:2324:ptlrpc_server_handle_request()) Handling RPC req@000000008ee413f4 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_001:b50963bd-8eba-4432-9a54-8888d00e3fa1+37:18261:x1862456979105920:12345-192.168.204.3@tcp:101:touch.0 00010000:00010000:1.0:1776178212.260305:0:6256:0:(ldlm_lockd.c:1248:ldlm_handle_enqueue0()) ### server-side enqueue handler START 00010000:00010000:1.0:1776178212.260334:0:6256:0:(ldlm_lockd.c:1318:ldlm_handle_enqueue0()) ### server-side enqueue handler, new lock created ns: mdt-lustre-MDT0000_UUID lock: 000000006b0afb52/0xa40083ab9bf5391c lrc: 2/0,0 mode: --/CW res: [0x2000013a1:0x9:0x0].0x0 bits 0x0/0x0 rrc: 4 type: IBT gid 0 flags: 0x40000000000000 nid: local remote: 0x771f68aa1eb27bff expref: -99 pid: 6256 timeout: 0 lvb_type: 0 00010000:00010000:1.0:1776178212.260444:0:6256:0:(ldlm_lock.c:766:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(CW) ns: mdt-lustre-MDT0000_UUID lock: 000000002abc4399/0xa40083ab9bf53923 lrc: 3/0,1 mode: --/CW res: [0x2000013a1:0x9:0x0].0x0 bits 0x0/0x0 rrc: 5 type: IBT gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 6256 timeout: 0 lvb_type: 0 00010000:00010000:1.0:1776178212.260461:0:6256:0:(ldlm_lock.c:1064:ldlm_granted_list_add_lock()) ### About to add lock: ns: mdt-lustre-MDT0000_UUID lock: 000000002abc4399/0xa40083ab9bf53923 lrc: 3/0,1 mode: CW/CW res: [0x2000013a1:0x9:0x0].0x0 bits 0x2/0x0 rrc: 5 type: IBT gid 0 flags: 0x50210000000000 nid: local remote: 0x0 expref: -99 pid: 6256 timeout: 0 lvb_type: 0 00010000:00010000:1.0:1776178212.260473:0:6256:0:(ldlm_request.c:528:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: mdt-lustre-MDT0000_UUID lock: 000000002abc4399/0xa40083ab9bf53923 lrc: 3/0,1 mode: CW/CW res: [0x2000013a1:0x9:0x0].0x0 bits 0x2/0x0 rrc: 5 type: IBT gid 0 flags: 0x40210000000000 nid: local remote: 0x0 expref: -99 pid: 6256 timeout: 0 lvb_type: 0 00010000:00010000:1.0:1776178212.260488:0:6256:0:(ldlm_lock.c:766:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: mdt-lustre-MDT0000_UUID lock: 00000000a44d8583/0xa40083ab9bf5392a lrc: 3/0,1 mode: --/PW res: [0x2000013a1:0x9:0x0].0xef09e6ea bits 0x0/0x0 rrc: 2 type: IBT gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 6256 timeout: 0 lvb_type: 0 00010000:00010000:1.0:1776178212.260494:0:6256:0:(ldlm_lock.c:1064:ldlm_granted_list_add_lock()) ### About to add lock: ns: mdt-lustre-MDT0000_UUID lock: 00000000a44d8583/0xa40083ab9bf5392a lrc: 3/0,1 mode: PW/PW res: [0x2000013a1:0x9:0x0].0xef09e6ea bits 0x2/0x0 rrc: 2 type: IBT gid 0 flags: 0x50210000000000 nid: local remote: 0x0 expref: -99 pid: 6256 timeout: 0 lvb_type: 0 00010000:00010000:1.0:1776178212.260501:0:6256:0:(ldlm_request.c:528:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: mdt-lustre-MDT0000_UUID lock: 00000000a44d8583/0xa40083ab9bf5392a lrc: 3/0,1 mode: PW/PW res: [0x2000013a1:0x9:0x0].0xef09e6ea bits 0x2/0x0 rrc: 2 type: IBT gid 0 flags: 0x40210000000000 nid: local remote: 0x0 expref: -99 pid: 6256 timeout: 0 lvb_type: 0 00000004:00080000:1.0:1776178212.260886:0:6256:0:(osp_object.c:1645:osp_create()) lustre-OST0001-osc-MDT0000: Wrote last used FID: [0x100010000:0x47:0x0], index 1: 0 00010000:00010000:0.0:1776178212.267941:0:6256:0:(ldlm_lock.c:766:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(CR) ns: mdt-lustre-MDT0000_UUID lock: 000000004bd1d7cb/0xa40083ab9bf53931 lrc: 3/1,0 mode: --/CR res: [0x2000013a1:0xe:0x0].0x0 bits 0x0/0x0 rrc: 2 type: IBT gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 6256 timeout: 0 lvb_type: 0 00010000:00010000:0.0:1776178212.267971:0:6256:0:(ldlm_lock.c:1064:ldlm_granted_list_add_lock()) ### About to add lock: ns: mdt-lustre-MDT0000_UUID lock: 000000004bd1d7cb/0xa40083ab9bf53931 lrc: 3/1,0 mode: CR/CR res: [0x2000013a1:0xe:0x0].0x0 bits 0x9/0x0 rrc: 2 type: IBT gid 0 flags: 0x50210000000000 nid: local remote: 0x0 expref: -99 pid: 6256 timeout: 0 lvb_type: 0 00010000:00010000:0.0:1776178212.268193:0:6256:0:(ldlm_request.c:528:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: mdt-lustre-MDT0000_UUID lock: 000000004bd1d7cb/0xa40083ab9bf53931 lrc: 3/1,0 mode: CR/CR res: [0x2000013a1:0xe:0x0].0x0 bits 0x9/0x0 rrc: 2 type: IBT gid 0 flags: 0x40210000000000 nid: local remote: 0x0 expref: -99 pid: 6256 timeout: 0 lvb_type: 0 00010000:00010000:0.0:1776178212.268288:0:6256:0:(ldlm_lock.c:820:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(CW) ns: mdt-lustre-MDT0000_UUID lock: 000000002abc4399/0xa40083ab9bf53923 lrc: 3/0,1 mode: CW/CW res: [0x2000013a1:0x9:0x0].0x0 bits 0x2/0x0 rrc: 5 type: IBT gid 0 flags: 0x50210000000000 nid: local remote: 0x0 expref: -99 pid: 6256 timeout: 0 lvb_type: 0 00010000:00010000:0.0:1776178212.268403:0:6256:0:(ldlm_lock.c:879:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: mdt-lustre-MDT0000_UUID lock: 000000002abc4399/0xa40083ab9bf53923 lrc: 2/0,0 mode: CW/CW res: [0x2000013a1:0x9:0x0].0x0 bits 0x2/0x0 rrc: 5 type: IBT gid 0 flags: 0x50210400000000 nid: local remote: 0x0 expref: -99 pid: 6256 timeout: 0 lvb_type: 0 00010000:00010000:0.0:1776178212.268413:0:6256:0:(ldlm_lockd.c:1879:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: mdt-lustre-MDT0000_UUID lock: 000000002abc4399/0xa40083ab9bf53923 lrc: 3/0,0 mode: CW/CW res: [0x2000013a1:0x9:0x0].0x0 bits 0x2/0x0 rrc: 5 type: IBT gid 0 flags: 0x40210400000000 nid: local remote: 0x0 expref: -99 pid: 6256 timeout: 0 lvb_type: 0 00010000:00010000:0.0:1776178212.268420:0:6256:0:(ldlm_lockd.c:1893:ldlm_handle_bl_callback()) Lock 000000002abc4399 already unused, calling callback (0000000079e78348) 00010000:00010000:0.0:1776178212.268429:0:6256:0:(ldlm_request.c:364:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: mdt-lustre-MDT0000_UUID lock: 000000002abc4399/0xa40083ab9bf53923 lrc: 3/0,0 mode: CW/CW res: [0x2000013a1:0x9:0x0].0x0 bits 0x2/0x0 rrc: 5 type: IBT gid 0 flags: 0x40210400000000 nid: local remote: 0x0 expref: -99 pid: 6256 timeout: 0 lvb_type: 0 00010000:00010000:0.0:1776178212.268438:0:6256:0:(ldlm_request.c:1257:ldlm_cli_cancel_local()) ### server-side local cancel ns: mdt-lustre-MDT0000_UUID lock: 000000002abc4399/0xa40083ab9bf53923 lrc: 4/0,0 mode: CW/CW res: [0x2000013a1:0x9:0x0].0x0 bits 0x2/0x0 rrc: 5 type: IBT gid 0 flags: 0x40218400000000 nid: local remote: 0x0 expref: -99 pid: 6256 timeout: 0 lvb_type: 0 00010000:00010000:0.0:1776178212.268455:0:6256:0:(ldlm_inodebits.c:94:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x2000013a1:0x9:0x0].0x0 (00000000a041a513) 00010000:00010000:0.0:1776178212.268459:0:6256:0:(ldlm_inodebits.c:96:ldlm_reprocess_inodebits_queue()) Hint 2 00010000:00010000:0.0:1776178212.268465:0:6256:0:(ldlm_lockd.c:1903:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: mdt-lustre-MDT0000_UUID lock: 000000002abc4399/0xa40083ab9bf53923 lrc: 2/0,0 mode: --/CW res: [0x2000013a1:0x9:0x0].0x0 bits 0x2/0x0 rrc: 5 type: IBT gid 0 flags: 0x44a19400000000 nid: local remote: 0x0 expref: -99 pid: 6256 timeout: 0 lvb_type: 0 00010000:00010000:0.0:1776178212.268472:0:6256:0:(ldlm_lock.c:216:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: mdt-lustre-MDT0000_UUID lock: 000000002abc4399/0xa40083ab9bf53923 lrc: 0/0,0 mode: --/CW res: [0x2000013a1:0x9:0x0].0x0 bits 0x2/0x0 rrc: 5 type: IBT gid 0 flags: 0x44a19400000000 nid: local remote: 0x0 expref: -99 pid: 6256 timeout: 0 lvb_type: 0 00000004:00010000:0.0:1776178212.268496:0:6256:0:(mdt_handler.c:3992:mdt_save_lock()) ### save lock request 000000008ee413f4 reply state 000000009e70e4e7 transno 12884901964 ns: mdt-lustre-MDT0000_UUID lock: 00000000a44d8583/0xa40083ab9bf5392a lrc: 3/0,1 mode: PW/PW res: [0x2000013a1:0x9:0x0].0xef09e6ea bits 0x2/0x0 rrc: 2 type: IBT gid 0 flags: 0x40210000000000 nid: local remote: 0x0 expref: -99 pid: 6256 timeout: 0 lvb_type: 0 00010000:00010000:0.0:1776178212.268563:0:6256:0:(ldlm_lock.c:216:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: mdt-lustre-MDT0000_UUID lock: 000000006b0afb52/0xa40083ab9bf5391c lrc: 0/0,0 mode: --/CW res: [0x2000013a1:0x9:0x0].0x0 bits 0x1/0x0 rrc: 4 type: IBT gid 0 flags: 0x44000000000000 nid: 192.168.204.3@tcp remote: 0x771f68aa1eb27bff expref: 40 pid: 6256 timeout: 0 lvb_type: 0 00010000:00010000:0.0:1776178212.268578:0:6256:0:(ldlm_lockd.c:1484:ldlm_handle_enqueue0()) ### server-side enqueue handler, sending reply (err=0, rc=0) ns: mdt-lustre-MDT0000_UUID lock: 000000004bd1d7cb/0xa40083ab9bf53931 lrc: 3/0,0 mode: CR/CR res: [0x2000013a1:0xe:0x0].0x0 bits 0x9/0x0 rrc: 2 type: IBT gid 0 flags: 0x40200000000000 nid: 192.168.204.3@tcp remote: 0x771f68aa1eb27bff expref: 39 pid: 6256 timeout: 0 lvb_type: 0 00010000:00010000:0.0:1776178212.268586:0:6256:0:(ldlm_inodebits.c:94:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x2000013a1:0xe:0x0].0x0 (000000008c340be4) 00010000:00010000:0.0:1776178212.268588:0:6256:0:(ldlm_inodebits.c:96:ldlm_reprocess_inodebits_queue()) Hint 9 00010000:00010000:0.0:1776178212.268590:0:6256:0:(ldlm_lockd.c:1565:ldlm_handle_enqueue0()) ### server-side enqueue handler END (lock 000000004bd1d7cb, rc 0) 00000100:00100000:0.0:1776178212.268759:0:6256:0:(service.c:2372:ptlrpc_server_handle_request()) Handled RPC req@000000008ee413f4 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_001:b50963bd-8eba-4432-9a54-8888d00e3fa1+40:18261:x1862456979105920:12345-192.168.204.3@tcp:101:touch.0 Request processed in 8497us (8796us total) trans 12884901964 rc 0/0 00000100:00100000:0.0:1776178212.268771:0:6256:0:(nrs_fifo.c:241:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.204.3@tcp, seq: 6610 00000100:00100000:2.0:1776178212.279496:0:3348:0:(events.c:361:request_in_callback()) peer: 12345-192.168.204.3@tcp (source: 12345-192.168.204.3@tcp) 00010000:00010000:2.0:1776178212.279540:0:3339:0:(ldlm_lock.c:1064:ldlm_granted_list_add_lock()) ### About to add lock: ns: mdt-lustre-MDT0000_UUID lock: 00000000a44d8583/0xa40083ab9bf5392a lrc: 3/0,1 mode: COS/COS res: [0x2000013a1:0x9:0x0].0xef09e6ea bits 0x2/0x0 rrc: 2 type: IBT gid 0 flags: 0x50210000000000 nid: local remote: 0x0 expref: -99 pid: 6256 timeout: 0 lvb_type: 0 00010000:00010000:2.0:1776178212.279569:0:3339:0:(ldlm_inodebits.c:94:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x2000013a1:0x9:0x0].0xef09e6ea (000000009742f5ea) 00010000:00010000:2.0:1776178212.279572:0:3339:0:(ldlm_inodebits.c:96:ldlm_reprocess_inodebits_queue()) Hint 2 00000100:00100000:2.0:1776178212.279623:0:6256:0:(service.c:2152:ptlrpc_server_handle_req_in()) got req x1862456979106048 00000100:00100000:2.0:1776178212.279681:0:6256:0:(nrs_fifo.c:179:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.204.3@tcp, seq: 6611 00000100:00100000:2.0:1776178212.279689:0:6256:0:(service.c:2324:ptlrpc_server_handle_request()) Handling RPC req@00000000fd2b8392 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_001:b50963bd-8eba-4432-9a54-8888d00e3fa1+40:18261:x1862456979106048:12345-192.168.204.3@tcp:36:touch.0 00010000:00010000:2.0:1776178212.279813:0:6256:0:(ldlm_lock.c:766:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: mdt-lustre-MDT0000_UUID lock: 000000000b5c0143/0xa40083ab9bf5393f lrc: 3/0,1 mode: --/PW res: [0x2000013a1:0xe:0x0].0x0 bits 0x0/0x0 rrc: 3 type: IBT gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 6256 timeout: 0 lvb_type: 0 00010000:00010000:2.0:1776178212.279827:0:6256:0:(ldlm_lock.c:1064:ldlm_granted_list_add_lock()) ### About to add lock: ns: mdt-lustre-MDT0000_UUID lock: 000000000b5c0143/0xa40083ab9bf5393f lrc: 3/0,1 mode: PW/PW res: [0x2000013a1:0xe:0x0].0x0 bits 0x2/0x0 rrc: 3 type: IBT gid 0 flags: 0x50210000000000 nid: local remote: 0x0 expref: -99 pid: 6256 timeout: 0 lvb_type: 0 00010000:00010000:2.0:1776178212.279838:0:6256:0:(ldlm_request.c:528:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: mdt-lustre-MDT0000_UUID lock: 000000000b5c0143/0xa40083ab9bf5393f lrc: 3/0,1 mode: PW/PW res: [0x2000013a1:0xe:0x0].0x0 bits 0x2/0x0 rrc: 3 type: IBT gid 0 flags: 0x40210000000000 nid: local remote: 0x0 expref: -99 pid: 6256 timeout: 0 lvb_type: 0 00000004:00010000:2.0:1776178212.279963:0:6256:0:(mdt_handler.c:3992:mdt_save_lock()) ### save lock request 00000000fd2b8392 reply state 000000003b0ed5a1 transno 12884901965 ns: mdt-lustre-MDT0000_UUID lock: 000000000b5c0143/0xa40083ab9bf5393f lrc: 3/0,1 mode: PW/PW res: [0x2000013a1:0xe:0x0].0x0 bits 0x2/0x0 rrc: 3 type: IBT gid 0 flags: 0x40210000000000 nid: local remote: 0x0 expref: -99 pid: 6256 timeout: 0 lvb_type: 0 00000100:00100000:2.0:1776178212.280466:0:6256:0:(service.c:2372:ptlrpc_server_handle_request()) Handled RPC req@00000000fd2b8392 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_001:b50963bd-8eba-4432-9a54-8888d00e3fa1+42:18261:x1862456979106048:12345-192.168.204.3@tcp:36:touch.0 Request processed in 775us (979us total) trans 12884901965 rc 0/0 00000100:00100000:2.0:1776178212.280476:0:6256:0:(nrs_fifo.c:241:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.204.3@tcp, seq: 6611 00010000:00010000:0.0:1776178212.285572:0:3340:0:(ldlm_lock.c:1064:ldlm_granted_list_add_lock()) ### About to add lock: ns: mdt-lustre-MDT0000_UUID lock: 000000000b5c0143/0xa40083ab9bf5393f lrc: 3/0,1 mode: COS/COS res: [0x2000013a1:0xe:0x0].0x0 bits 0x2/0x0 rrc: 3 type: IBT gid 0 flags: 0x50210000000000 nid: local remote: 0x0 expref: -99 pid: 6256 timeout: 0 lvb_type: 0 00010000:00010000:0.0:1776178212.285604:0:3340:0:(ldlm_inodebits.c:94:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x2000013a1:0xe:0x0].0x0 (000000008c340be4) 00010000:00010000:0.0:1776178212.285607:0:3340:0:(ldlm_inodebits.c:96:ldlm_reprocess_inodebits_queue()) Hint 2 00000100:00100000:2.0:1776178212.285826:0:3348:0:(events.c:361:request_in_callback()) peer: 12345-192.168.204.3@tcp (source: 12345-192.168.204.3@tcp) 00000100:00100000:1.0:1776178212.285943:0:20755:0:(service.c:2152:ptlrpc_server_handle_req_in()) got req x1862456979106112 00000100:00100000:1.0:1776178212.286426:0:20755:0:(nrs_fifo.c:179:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.204.3@tcp, seq: 5289 00000100:00100000:1.0:1776178212.286435:0:20755:0:(service.c:2324:ptlrpc_server_handle_request()) Handling RPC req@0000000042de09e9 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_006:b50963bd-8eba-4432-9a54-8888d00e3fa1+7:2270:x1862456979106112:12345-192.168.204.3@tcp:2:touch.0 00000100:00100000:1.0:1776178212.286722:0:20755:0:(service.c:2372:ptlrpc_server_handle_request()) Handled RPC req@0000000042de09e9 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_006:b50963bd-8eba-4432-9a54-8888d00e3fa1+8:2270:x1862456979106112:12345-192.168.204.3@tcp:2:touch.0 Request processed in 287us (899us total) trans 4294969338 rc 0/0 00000100:00100000:1.0:1776178212.286731:0:20755:0:(nrs_fifo.c:241:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.204.3@tcp, seq: 5289 00000100:00100000:2.0:1776178212.289132:0:3346:0:(events.c:361:request_in_callback()) peer: 12345-192.168.204.3@tcp (source: 12345-192.168.204.3@tcp) 00000100:00100000:3.0:1776178212.289220:0:6259:0:(service.c:2152:ptlrpc_server_handle_req_in()) got req x1862456979106240 00000100:00100000:3.0:1776178212.289265:0:6259:0:(nrs_fifo.c:179:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.204.3@tcp, seq: 2152 00000100:00100000:3.0:1776178212.289273:0:6259:0:(service.c:2324:ptlrpc_server_handle_request()) Handling RPC req@000000000cc3fe7b pname:cluuid+ref:pid:xid:nid:opc:job mdt_rdpg00_001:b50963bd-8eba-4432-9a54-8888d00e3fa1+42:18261:x1862456979106240:12345-192.168.204.3@tcp:35:touch.0 00000100:00100000:3.0:1776178212.289716:0:6259:0:(service.c:2372:ptlrpc_server_handle_request()) Handled RPC req@000000000cc3fe7b pname:cluuid+ref:pid:xid:nid:opc:job mdt_rdpg00_001:b50963bd-8eba-4432-9a54-8888d00e3fa1+43:18261:x1862456979106240:12345-192.168.204.3@tcp:35:touch.0 Request processed in 441us (593us total) trans 12884901966 rc 0/0 00000100:00100000:3.0:1776178212.289725:0:6259:0:(nrs_fifo.c:241:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.204.3@tcp, seq: 2152 00000100:00100000:3.0:1776178212.300407:0:3347:0:(events.c:361:request_in_callback()) peer: 12345-192.168.204.3@tcp (source: 12345-192.168.204.3@tcp) 00000100:00100000:0.0:1776178212.300560:0:6257:0:(service.c:2152:ptlrpc_server_handle_req_in()) got req x1862456979106368 00000100:00100000:0.0:1776178212.300624:0:6257:0:(nrs_fifo.c:179:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.204.3@tcp, seq: 6612 00000100:00100000:0.0:1776178212.300631:0:6257:0:(service.c:2324:ptlrpc_server_handle_request()) Handling RPC req@000000003f8daac1 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_002:b50963bd-8eba-4432-9a54-8888d00e3fa1+43:18262:x1862456979106368:12345-192.168.204.3@tcp:101:touch.0 00010000:00010000:0.0:1776178212.300660:0:6257:0:(ldlm_lockd.c:1248:ldlm_handle_enqueue0()) ### server-side enqueue handler START 00010000:00010000:0.0:1776178212.300682:0:6257:0:(ldlm_lockd.c:1318:ldlm_handle_enqueue0()) ### server-side enqueue handler, new lock created ns: mdt-lustre-MDT0000_UUID lock: 000000000e7a37e0/0xa40083ab9bf53946 lrc: 2/0,0 mode: --/CW res: [0x2000013a1:0x9:0x0].0x0 bits 0x0/0x0 rrc: 4 type: IBT gid 0 flags: 0x40000000000000 nid: local remote: 0x771f68aa1eb27c06 expref: -99 pid: 6257 timeout: 0 lvb_type: 0 00010000:00010000:0.0:1776178212.300789:0:6257:0:(ldlm_lock.c:766:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(CW) ns: mdt-lustre-MDT0000_UUID lock: 000000000b903578/0xa40083ab9bf5394d lrc: 3/0,1 mode: --/CW res: [0x2000013a1:0x9:0x0].0x0 bits 0x0/0x0 rrc: 5 type: IBT gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 6257 timeout: 0 lvb_type: 0 00010000:00010000:0.0:1776178212.300804:0:6257:0:(ldlm_lock.c:1064:ldlm_granted_list_add_lock()) ### About to add lock: ns: mdt-lustre-MDT0000_UUID lock: 000000000b903578/0xa40083ab9bf5394d lrc: 3/0,1 mode: CW/CW res: [0x2000013a1:0x9:0x0].0x0 bits 0x2/0x0 rrc: 5 type: IBT gid 0 flags: 0x50210000000000 nid: local remote: 0x0 expref: -99 pid: 6257 timeout: 0 lvb_type: 0 00010000:00010000:0.0:1776178212.300816:0:6257:0:(ldlm_request.c:528:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: mdt-lustre-MDT0000_UUID lock: 000000000b903578/0xa40083ab9bf5394d lrc: 3/0,1 mode: CW/CW res: [0x2000013a1:0x9:0x0].0x0 bits 0x2/0x0 rrc: 5 type: IBT gid 0 flags: 0x40210000000000 nid: local remote: 0x0 expref: -99 pid: 6257 timeout: 0 lvb_type: 0 00010000:00010000:0.0:1776178212.300829:0:6257:0:(ldlm_lock.c:766:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: mdt-lustre-MDT0000_UUID lock: 00000000f90c1e39/0xa40083ab9bf53954 lrc: 3/0,1 mode: --/PW res: [0x2000013a1:0x9:0x0].0x37639fea bits 0x0/0x0 rrc: 2 type: IBT gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 6257 timeout: 0 lvb_type: 0 00010000:00010000:0.0:1776178212.300835:0:6257:0:(ldlm_lock.c:1064:ldlm_granted_list_add_lock()) ### About to add lock: ns: mdt-lustre-MDT0000_UUID lock: 00000000f90c1e39/0xa40083ab9bf53954 lrc: 3/0,1 mode: PW/PW res: [0x2000013a1:0x9:0x0].0x37639fea bits 0x2/0x0 rrc: 2 type: IBT gid 0 flags: 0x50210000000000 nid: local remote: 0x0 expref: -99 pid: 6257 timeout: 0 lvb_type: 0 00010000:00010000:0.0:1776178212.300853:0:6257:0:(ldlm_request.c:528:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: mdt-lustre-MDT0000_UUID lock: 00000000f90c1e39/0xa40083ab9bf53954 lrc: 3/0,1 mode: PW/PW res: [0x2000013a1:0x9:0x0].0x37639fea bits 0x2/0x0 rrc: 2 type: IBT gid 0 flags: 0x40210000000000 nid: local remote: 0x0 expref: -99 pid: 6257 timeout: 0 lvb_type: 0 00000004:00080000:2.0:1776178212.301751:0:6257:0:(osp_object.c:1645:osp_create()) lustre-OST0000-osc-MDT0000: Wrote last used FID: [0x100000000:0x48:0x0], index 0: 0 00010000:00010000:2.0:1776178212.302536:0:6257:0:(ldlm_lock.c:766:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(CR) ns: mdt-lustre-MDT0000_UUID lock: 000000009a8f90ae/0xa40083ab9bf5395b lrc: 3/1,0 mode: --/CR res: [0x2000013a1:0xf:0x0].0x0 bits 0x0/0x0 rrc: 2 type: IBT gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 6257 timeout: 0 lvb_type: 0 00010000:00010000:2.0:1776178212.302561:0:6257:0:(ldlm_lock.c:1064:ldlm_granted_list_add_lock()) ### About to add lock: ns: mdt-lustre-MDT0000_UUID lock: 000000009a8f90ae/0xa40083ab9bf5395b lrc: 3/1,0 mode: CR/CR res: [0x2000013a1:0xf:0x0].0x0 bits 0x9/0x0 rrc: 2 type: IBT gid 0 flags: 0x50210000000000 nid: local remote: 0x0 expref: -99 pid: 6257 timeout: 0 lvb_type: 0 00010000:00010000:2.0:1776178212.302573:0:6257:0:(ldlm_request.c:528:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: mdt-lustre-MDT0000_UUID lock: 000000009a8f90ae/0xa40083ab9bf5395b lrc: 3/1,0 mode: CR/CR res: [0x2000013a1:0xf:0x0].0x0 bits 0x9/0x0 rrc: 2 type: IBT gid 0 flags: 0x40210000000000 nid: local remote: 0x0 expref: -99 pid: 6257 timeout: 0 lvb_type: 0 00010000:00010000:2.0:1776178212.302622:0:6257:0:(ldlm_lock.c:820:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(CW) ns: mdt-lustre-MDT0000_UUID lock: 000000000b903578/0xa40083ab9bf5394d lrc: 3/0,1 mode: CW/CW res: [0x2000013a1:0x9:0x0].0x0 bits 0x2/0x0 rrc: 5 type: IBT gid 0 flags: 0x50210000000000 nid: local remote: 0x0 expref: -99 pid: 6257 timeout: 0 lvb_type: 0 00010000:00010000:2.0:1776178212.302629:0:6257:0:(ldlm_lock.c:879:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: mdt-lustre-MDT0000_UUID lock: 000000000b903578/0xa40083ab9bf5394d lrc: 2/0,0 mode: CW/CW res: [0x2000013a1:0x9:0x0].0x0 bits 0x2/0x0 rrc: 5 type: IBT gid 0 flags: 0x50210400000000 nid: local remote: 0x0 expref: -99 pid: 6257 timeout: 0 lvb_type: 0 00010000:00010000:2.0:1776178212.302636:0:6257:0:(ldlm_lockd.c:1879:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: mdt-lustre-MDT0000_UUID lock: 000000000b903578/0xa40083ab9bf5394d lrc: 3/0,0 mode: CW/CW res: [0x2000013a1:0x9:0x0].0x0 bits 0x2/0x0 rrc: 5 type: IBT gid 0 flags: 0x40210400000000 nid: local remote: 0x0 expref: -99 pid: 6257 timeout: 0 lvb_type: 0 00010000:00010000:2.0:1776178212.302642:0:6257:0:(ldlm_lockd.c:1893:ldlm_handle_bl_callback()) Lock 000000000b903578 already unused, calling callback (0000000079e78348) 00010000:00010000:2.0:1776178212.302646:0:6257:0:(ldlm_request.c:364:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: mdt-lustre-MDT0000_UUID lock: 000000000b903578/0xa40083ab9bf5394d lrc: 3/0,0 mode: CW/CW res: [0x2000013a1:0x9:0x0].0x0 bits 0x2/0x0 rrc: 5 type: IBT gid 0 flags: 0x40210400000000 nid: local remote: 0x0 expref: -99 pid: 6257 timeout: 0 lvb_type: 0 00010000:00010000:2.0:1776178212.302653:0:6257:0:(ldlm_request.c:1257:ldlm_cli_cancel_local()) ### server-side local cancel ns: mdt-lustre-MDT0000_UUID lock: 000000000b903578/0xa40083ab9bf5394d lrc: 4/0,0 mode: CW/CW res: [0x2000013a1:0x9:0x0].0x0 bits 0x2/0x0 rrc: 5 type: IBT gid 0 flags: 0x40218400000000 nid: local remote: 0x0 expref: -99 pid: 6257 timeout: 0 lvb_type: 0 00010000:00010000:2.0:1776178212.302666:0:6257:0:(ldlm_inodebits.c:94:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x2000013a1:0x9:0x0].0x0 (00000000a041a513) 00010000:00010000:2.0:1776178212.302668:0:6257:0:(ldlm_inodebits.c:96:ldlm_reprocess_inodebits_queue()) Hint 2 00010000:00010000:2.0:1776178212.302673:0:6257:0:(ldlm_lockd.c:1903:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: mdt-lustre-MDT0000_UUID lock: 000000000b903578/0xa40083ab9bf5394d lrc: 2/0,0 mode: --/CW res: [0x2000013a1:0x9:0x0].0x0 bits 0x2/0x0 rrc: 5 type: IBT gid 0 flags: 0x44a19400000000 nid: local remote: 0x0 expref: -99 pid: 6257 timeout: 0 lvb_type: 0 00010000:00010000:2.0:1776178212.302679:0:6257:0:(ldlm_lock.c:216:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: mdt-lustre-MDT0000_UUID lock: 000000000b903578/0xa40083ab9bf5394d lrc: 0/0,0 mode: --/CW res: [0x2000013a1:0x9:0x0].0x0 bits 0x2/0x0 rrc: 5 type: IBT gid 0 flags: 0x44a19400000000 nid: local remote: 0x0 expref: -99 pid: 6257 timeout: 0 lvb_type: 0 00000004:00010000:2.0:1776178212.302694:0:6257:0:(mdt_handler.c:3992:mdt_save_lock()) ### save lock request 000000003f8daac1 reply state 00000000f2b1ed6d transno 12884901967 ns: mdt-lustre-MDT0000_UUID lock: 00000000f90c1e39/0xa40083ab9bf53954 lrc: 3/0,1 mode: PW/PW res: [0x2000013a1:0x9:0x0].0x37639fea bits 0x2/0x0 rrc: 2 type: IBT gid 0 flags: 0x40210000000000 nid: local remote: 0x0 expref: -99 pid: 6257 timeout: 0 lvb_type: 0 00010000:00010000:2.0:1776178212.302736:0:6257:0:(ldlm_lock.c:216:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: mdt-lustre-MDT0000_UUID lock: 000000000e7a37e0/0xa40083ab9bf53946 lrc: 0/0,0 mode: --/CW res: [0x2000013a1:0x9:0x0].0x0 bits 0x1/0x0 rrc: 4 type: IBT gid 0 flags: 0x44000000000000 nid: 192.168.204.3@tcp remote: 0x771f68aa1eb27c06 expref: 46 pid: 6257 timeout: 0 lvb_type: 0 00010000:00010000:2.0:1776178212.302754:0:6257:0:(ldlm_lockd.c:1484:ldlm_handle_enqueue0()) ### server-side enqueue handler, sending reply (err=0, rc=0) ns: mdt-lustre-MDT0000_UUID lock: 000000009a8f90ae/0xa40083ab9bf5395b lrc: 3/0,0 mode: CR/CR res: [0x2000013a1:0xf:0x0].0x0 bits 0x9/0x0 rrc: 2 type: IBT gid 0 flags: 0x40200000000000 nid: 192.168.204.3@tcp remote: 0x771f68aa1eb27c06 expref: 45 pid: 6257 timeout: 0 lvb_type: 0 00010000:00010000:2.0:1776178212.302760:0:6257:0:(ldlm_inodebits.c:94:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x2000013a1:0xf:0x0].0x0 (00000000a5b77ef7) 00010000:00010000:2.0:1776178212.302762:0:6257:0:(ldlm_inodebits.c:96:ldlm_reprocess_inodebits_queue()) Hint 9 00010000:00010000:2.0:1776178212.302764:0:6257:0:(ldlm_lockd.c:1565:ldlm_handle_enqueue0()) ### server-side enqueue handler END (lock 000000009a8f90ae, rc 0) 00000100:00100000:2.0:1776178212.303013:0:6257:0:(service.c:2372:ptlrpc_server_handle_request()) Handled RPC req@000000003f8daac1 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_002:b50963bd-8eba-4432-9a54-8888d00e3fa1+46:18262:x1862456979106368:12345-192.168.204.3@tcp:101:touch.0 Request processed in 2373us (2605us total) trans 12884901967 rc 0/0 00000100:00100000:2.0:1776178212.303023:0:6257:0:(nrs_fifo.c:241:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.204.3@tcp, seq: 6612 00010000:00010000:0.0:1776178212.305726:0:3339:0:(ldlm_lock.c:1064:ldlm_granted_list_add_lock()) ### About to add lock: ns: mdt-lustre-MDT0000_UUID lock: 00000000f90c1e39/0xa40083ab9bf53954 lrc: 3/0,1 mode: COS/COS res: [0x2000013a1:0x9:0x0].0x37639fea bits 0x2/0x0 rrc: 2 type: IBT gid 0 flags: 0x50210000000000 nid: local remote: 0x0 expref: -99 pid: 6257 timeout: 0 lvb_type: 0 00010000:00010000:0.0:1776178212.305751:0:3339:0:(ldlm_inodebits.c:94:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x2000013a1:0x9:0x0].0x37639fea (000000004548771b) 00010000:00010000:0.0:1776178212.305753:0:3339:0:(ldlm_inodebits.c:96:ldlm_reprocess_inodebits_queue()) Hint 2 00000100:00100000:3.0:1776178212.306316:0:3347:0:(events.c:361:request_in_callback()) peer: 12345-192.168.204.3@tcp (source: 12345-192.168.204.3@tcp) 00000100:00100000:2.0:1776178212.307210:0:6257:0:(service.c:2152:ptlrpc_server_handle_req_in()) got req x1862456979106496 00000100:00100000:2.0:1776178212.307265:0:6257:0:(nrs_fifo.c:179:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.204.3@tcp, seq: 6613 00000100:00100000:2.0:1776178212.307271:0:6257:0:(service.c:2324:ptlrpc_server_handle_request()) Handling RPC req@0000000058333a54 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_002:b50963bd-8eba-4432-9a54-8888d00e3fa1+46:18262:x1862456979106496:12345-192.168.204.3@tcp:36:touch.0 00010000:00010000:2.0:1776178212.307600:0:6257:0:(ldlm_lock.c:766:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: mdt-lustre-MDT0000_UUID lock: 00000000a27c19b3/0xa40083ab9bf53969 lrc: 3/0,1 mode: --/PW res: [0x2000013a1:0xf:0x0].0x0 bits 0x0/0x0 rrc: 3 type: IBT gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 6257 timeout: 0 lvb_type: 0 00010000:00010000:2.0:1776178212.307624:0:6257:0:(ldlm_lock.c:1064:ldlm_granted_list_add_lock()) ### About to add lock: ns: mdt-lustre-MDT0000_UUID lock: 00000000a27c19b3/0xa40083ab9bf53969 lrc: 3/0,1 mode: PW/PW res: [0x2000013a1:0xf:0x0].0x0 bits 0x2/0x0 rrc: 3 type: IBT gid 0 flags: 0x50210000000000 nid: local remote: 0x0 expref: -99 pid: 6257 timeout: 0 lvb_type: 0 00010000:00010000:2.0:1776178212.307641:0:6257:0:(ldlm_request.c:528:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: mdt-lustre-MDT0000_UUID lock: 00000000a27c19b3/0xa40083ab9bf53969 lrc: 3/0,1 mode: PW/PW res: [0x2000013a1:0xf:0x0].0x0 bits 0x2/0x0 rrc: 3 type: IBT gid 0 flags: 0x40210000000000 nid: local remote: 0x0 expref: -99 pid: 6257 timeout: 0 lvb_type: 0 00000004:00010000:2.0:1776178212.307773:0:6257:0:(mdt_handler.c:3992:mdt_save_lock()) ### save lock request 0000000058333a54 reply state 0000000073049f7d transno 12884901968 ns: mdt-lustre-MDT0000_UUID lock: 00000000a27c19b3/0xa40083ab9bf53969 lrc: 3/0,1 mode: PW/PW res: [0x2000013a1:0xf:0x0].0x0 bits 0x2/0x0 rrc: 3 type: IBT gid 0 flags: 0x40210000000000 nid: local remote: 0x0 expref: -99 pid: 6257 timeout: 0 lvb_type: 0 00000100:00100000:2.0:1776178212.308012:0:6257:0:(service.c:2372:ptlrpc_server_handle_request()) Handled RPC req@0000000058333a54 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_002:b50963bd-8eba-4432-9a54-8888d00e3fa1+48:18262:x1862456979106496:12345-192.168.204.3@tcp:36:touch.0 Request processed in 739us (1698us total) trans 12884901968 rc 0/0 00000100:00100000:2.0:1776178212.308026:0:6257:0:(nrs_fifo.c:241:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.204.3@tcp, seq: 6613 00010000:00010000:3.0:1776178212.314847:0:3340:0:(ldlm_lock.c:1064:ldlm_granted_list_add_lock()) ### About to add lock: ns: mdt-lustre-MDT0000_UUID lock: 00000000a27c19b3/0xa40083ab9bf53969 lrc: 3/0,1 mode: COS/COS res: [0x2000013a1:0xf:0x0].0x0 bits 0x2/0x0 rrc: 3 type: IBT gid 0 flags: 0x50210000000000 nid: local remote: 0x0 expref: -99 pid: 6257 timeout: 0 lvb_type: 0 00010000:00010000:3.0:1776178212.314876:0:3340:0:(ldlm_inodebits.c:94:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x2000013a1:0xf:0x0].0x0 (00000000a5b77ef7) 00010000:00010000:3.0:1776178212.314879:0:3340:0:(ldlm_inodebits.c:96:ldlm_reprocess_inodebits_queue()) Hint 2 00000100:00100000:3.0:1776178212.315722:0:3347:0:(events.c:361:request_in_callback()) peer: 12345-192.168.204.3@tcp (source: 12345-192.168.204.3@tcp) 00000100:00100000:1.0:1776178212.316974:0:20755:0:(service.c:2152:ptlrpc_server_handle_req_in()) got req x1862456979106560 00000100:00100000:1.0:1776178212.317105:0:20755:0:(nrs_fifo.c:179:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.204.3@tcp, seq: 5290 00000100:00100000:1.0:1776178212.317115:0:20755:0:(service.c:2324:ptlrpc_server_handle_request()) Handling RPC req@000000000390171d pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_006:b50963bd-8eba-4432-9a54-8888d00e3fa1+7:2270:x1862456979106560:12345-192.168.204.3@tcp:2:touch.0 00000100:00100000:1.0:1776178212.317336:0:20755:0:(service.c:2372:ptlrpc_server_handle_request()) Handled RPC req@000000000390171d pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_006:b50963bd-8eba-4432-9a54-8888d00e3fa1+8:2270:x1862456979106560:12345-192.168.204.3@tcp:2:touch.0 Request processed in 222us (1620us total) trans 4294969364 rc 0/0 00000100:00100000:1.0:1776178212.317345:0:20755:0:(nrs_fifo.c:241:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.204.3@tcp, seq: 5290 00010000:00010000:3.0:1776178212.320287:0:6244:0:(ldlm_lockd.c:2792:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:2.0:1776178212.321751:0:3348:0:(events.c:361:request_in_callback()) peer: 12345-192.168.204.3@tcp (source: 12345-192.168.204.3@tcp) 00000100:00100000:3.0:1776178212.321852:0:6259:0:(service.c:2152:ptlrpc_server_handle_req_in()) got req x1862456979106688 00000100:00100000:3.0:1776178212.321910:0:6259:0:(nrs_fifo.c:179:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.204.3@tcp, seq: 2153 00000100:00100000:3.0:1776178212.321917:0:6259:0:(service.c:2324:ptlrpc_server_handle_request()) Handling RPC req@0000000054130213 pname:cluuid+ref:pid:xid:nid:opc:job mdt_rdpg00_001:b50963bd-8eba-4432-9a54-8888d00e3fa1+48:18262:x1862456979106688:12345-192.168.204.3@tcp:35:touch.0 00000100:00100000:3.0:1776178212.322255:0:6259:0:(service.c:2372:ptlrpc_server_handle_request()) Handled RPC req@0000000054130213 pname:cluuid+ref:pid:xid:nid:opc:job mdt_rdpg00_001:b50963bd-8eba-4432-9a54-8888d00e3fa1+49:18262:x1862456979106688:12345-192.168.204.3@tcp:35:touch.0 Request processed in 336us (512us total) trans 12884901969 rc 0/0 00000100:00100000:3.0:1776178212.322263:0:6259:0:(nrs_fifo.c:241:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.204.3@tcp, seq: 2153 00000100:00100000:0.0:1776178212.341596:0:3346:0:(events.c:361:request_in_callback()) peer: 12345-192.168.204.3@tcp (source: 12345-192.168.204.3@tcp) 00000100:00100000:2.0:1776178212.341710:0:6257:0:(service.c:2152:ptlrpc_server_handle_req_in()) got req x1862456979106816 00000100:00100000:2.0:1776178212.341773:0:6257:0:(nrs_fifo.c:179:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.204.3@tcp, seq: 6614 00000100:00100000:2.0:1776178212.341780:0:6257:0:(service.c:2324:ptlrpc_server_handle_request()) Handling RPC req@00000000575d0558 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_002:b50963bd-8eba-4432-9a54-8888d00e3fa1+49:18263:x1862456979106816:12345-192.168.204.3@tcp:101:touch.0 00010000:00010000:2.0:1776178212.341814:0:6257:0:(ldlm_lockd.c:1248:ldlm_handle_enqueue0()) ### server-side enqueue handler START 00010000:00010000:2.0:1776178212.341836:0:6257:0:(ldlm_lockd.c:1318:ldlm_handle_enqueue0()) ### server-side enqueue handler, new lock created ns: mdt-lustre-MDT0000_UUID lock: 000000000396420a/0xa40083ab9bf53970 lrc: 2/0,0 mode: --/CW res: [0x2000013a1:0x9:0x0].0x0 bits 0x0/0x0 rrc: 4 type: IBT gid 0 flags: 0x40000000000000 nid: local remote: 0x771f68aa1eb27c0d expref: -99 pid: 6257 timeout: 0 lvb_type: 0 00010000:00010000:2.0:1776178212.341945:0:6257:0:(ldlm_lock.c:766:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(CW) ns: mdt-lustre-MDT0000_UUID lock: 000000006cc9c357/0xa40083ab9bf53977 lrc: 3/0,1 mode: --/CW res: [0x2000013a1:0x9:0x0].0x0 bits 0x0/0x0 rrc: 5 type: IBT gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 6257 timeout: 0 lvb_type: 0 00010000:00010000:2.0:1776178212.341963:0:6257:0:(ldlm_lock.c:1064:ldlm_granted_list_add_lock()) ### About to add lock: ns: mdt-lustre-MDT0000_UUID lock: 000000006cc9c357/0xa40083ab9bf53977 lrc: 3/0,1 mode: CW/CW res: [0x2000013a1:0x9:0x0].0x0 bits 0x2/0x0 rrc: 5 type: IBT gid 0 flags: 0x50210000000000 nid: local remote: 0x0 expref: -99 pid: 6257 timeout: 0 lvb_type: 0 00010000:00010000:2.0:1776178212.341975:0:6257:0:(ldlm_request.c:528:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: mdt-lustre-MDT0000_UUID lock: 000000006cc9c357/0xa40083ab9bf53977 lrc: 3/0,1 mode: CW/CW res: [0x2000013a1:0x9:0x0].0x0 bits 0x2/0x0 rrc: 5 type: IBT gid 0 flags: 0x40210000000000 nid: local remote: 0x0 expref: -99 pid: 6257 timeout: 0 lvb_type: 0 00010000:00010000:2.0:1776178212.342274:0:6257:0:(ldlm_lock.c:766:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: mdt-lustre-MDT0000_UUID lock: 00000000e4217a5c/0xa40083ab9bf5397e lrc: 3/0,1 mode: --/PW res: [0x2000013a1:0x9:0x0].0x7fbd58ea bits 0x0/0x0 rrc: 2 type: IBT gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 6257 timeout: 0 lvb_type: 0 00010000:00010000:2.0:1776178212.342284:0:6257:0:(ldlm_lock.c:1064:ldlm_granted_list_add_lock()) ### About to add lock: ns: mdt-lustre-MDT0000_UUID lock: 00000000e4217a5c/0xa40083ab9bf5397e lrc: 3/0,1 mode: PW/PW res: [0x2000013a1:0x9:0x0].0x7fbd58ea bits 0x2/0x0 rrc: 2 type: IBT gid 0 flags: 0x50210000000000 nid: local remote: 0x0 expref: -99 pid: 6257 timeout: 0 lvb_type: 0 00010000:00010000:2.0:1776178212.342291:0:6257:0:(ldlm_request.c:528:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: mdt-lustre-MDT0000_UUID lock: 00000000e4217a5c/0xa40083ab9bf5397e lrc: 3/0,1 mode: PW/PW res: [0x2000013a1:0x9:0x0].0x7fbd58ea bits 0x2/0x0 rrc: 2 type: IBT gid 0 flags: 0x40210000000000 nid: local remote: 0x0 expref: -99 pid: 6257 timeout: 0 lvb_type: 0 00000004:00080000:2.0:1776178212.342628:0:6257:0:(osp_object.c:1645:osp_create()) lustre-OST0001-osc-MDT0000: Wrote last used FID: [0x100010000:0x48:0x0], index 1: 0 00010000:00010000:2.0:1776178212.342789:0:6257:0:(ldlm_lock.c:766:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(CR) ns: mdt-lustre-MDT0000_UUID lock: 000000007da3b625/0xa40083ab9bf53985 lrc: 3/1,0 mode: --/CR res: [0x2000013a1:0x10:0x0].0x0 bits 0x0/0x0 rrc: 2 type: IBT gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 6257 timeout: 0 lvb_type: 0 00010000:00010000:2.0:1776178212.342800:0:6257:0:(ldlm_lock.c:1064:ldlm_granted_list_add_lock()) ### About to add lock: ns: mdt-lustre-MDT0000_UUID lock: 000000007da3b625/0xa40083ab9bf53985 lrc: 3/1,0 mode: CR/CR res: [0x2000013a1:0x10:0x0].0x0 bits 0x9/0x0 rrc: 2 type: IBT gid 0 flags: 0x50210000000000 nid: local remote: 0x0 expref: -99 pid: 6257 timeout: 0 lvb_type: 0 00010000:00010000:2.0:1776178212.342808:0:6257:0:(ldlm_request.c:528:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: mdt-lustre-MDT0000_UUID lock: 000000007da3b625/0xa40083ab9bf53985 lrc: 3/1,0 mode: CR/CR res: [0x2000013a1:0x10:0x0].0x0 bits 0x9/0x0 rrc: 2 type: IBT gid 0 flags: 0x40210000000000 nid: local remote: 0x0 expref: -99 pid: 6257 timeout: 0 lvb_type: 0 00010000:00010000:2.0:1776178212.342855:0:6257:0:(ldlm_lock.c:820:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(CW) ns: mdt-lustre-MDT0000_UUID lock: 000000006cc9c357/0xa40083ab9bf53977 lrc: 3/0,1 mode: CW/CW res: [0x2000013a1:0x9:0x0].0x0 bits 0x2/0x0 rrc: 5 type: IBT gid 0 flags: 0x50210000000000 nid: local remote: 0x0 expref: -99 pid: 6257 timeout: 0 lvb_type: 0 00010000:00010000:2.0:1776178212.342862:0:6257:0:(ldlm_lock.c:879:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: mdt-lustre-MDT0000_UUID lock: 000000006cc9c357/0xa40083ab9bf53977 lrc: 2/0,0 mode: CW/CW res: [0x2000013a1:0x9:0x0].0x0 bits 0x2/0x0 rrc: 5 type: IBT gid 0 flags: 0x50210400000000 nid: local remote: 0x0 expref: -99 pid: 6257 timeout: 0 lvb_type: 0 00010000:00010000:2.0:1776178212.342868:0:6257:0:(ldlm_lockd.c:1879:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: mdt-lustre-MDT0000_UUID lock: 000000006cc9c357/0xa40083ab9bf53977 lrc: 3/0,0 mode: CW/CW res: [0x2000013a1:0x9:0x0].0x0 bits 0x2/0x0 rrc: 5 type: IBT gid 0 flags: 0x40210400000000 nid: local remote: 0x0 expref: -99 pid: 6257 timeout: 0 lvb_type: 0 00010000:00010000:2.0:1776178212.342874:0:6257:0:(ldlm_lockd.c:1893:ldlm_handle_bl_callback()) Lock 000000006cc9c357 already unused, calling callback (0000000079e78348) 00010000:00010000:2.0:1776178212.342877:0:6257:0:(ldlm_request.c:364:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: mdt-lustre-MDT0000_UUID lock: 000000006cc9c357/0xa40083ab9bf53977 lrc: 3/0,0 mode: CW/CW res: [0x2000013a1:0x9:0x0].0x0 bits 0x2/0x0 rrc: 5 type: IBT gid 0 flags: 0x40210400000000 nid: local remote: 0x0 expref: -99 pid: 6257 timeout: 0 lvb_type: 0 00010000:00010000:2.0:1776178212.342885:0:6257:0:(ldlm_request.c:1257:ldlm_cli_cancel_local()) ### server-side local cancel ns: mdt-lustre-MDT0000_UUID lock: 000000006cc9c357/0xa40083ab9bf53977 lrc: 4/0,0 mode: CW/CW res: [0x2000013a1:0x9:0x0].0x0 bits 0x2/0x0 rrc: 5 type: IBT gid 0 flags: 0x40218400000000 nid: local remote: 0x0 expref: -99 pid: 6257 timeout: 0 lvb_type: 0 00010000:00010000:2.0:1776178212.342898:0:6257:0:(ldlm_inodebits.c:94:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x2000013a1:0x9:0x0].0x0 (00000000a041a513) 00010000:00010000:2.0:1776178212.342901:0:6257:0:(ldlm_inodebits.c:96:ldlm_reprocess_inodebits_queue()) Hint 2 00010000:00010000:2.0:1776178212.342907:0:6257:0:(ldlm_lockd.c:1903:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: mdt-lustre-MDT0000_UUID lock: 000000006cc9c357/0xa40083ab9bf53977 lrc: 2/0,0 mode: --/CW res: [0x2000013a1:0x9:0x0].0x0 bits 0x2/0x0 rrc: 5 type: IBT gid 0 flags: 0x44a19400000000 nid: local remote: 0x0 expref: -99 pid: 6257 timeout: 0 lvb_type: 0 00010000:00010000:2.0:1776178212.342913:0:6257:0:(ldlm_lock.c:216:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: mdt-lustre-MDT0000_UUID lock: 000000006cc9c357/0xa40083ab9bf53977 lrc: 0/0,0 mode: --/CW res: [0x2000013a1:0x9:0x0].0x0 bits 0x2/0x0 rrc: 5 type: IBT gid 0 flags: 0x44a19400000000 nid: local remote: 0x0 expref: -99 pid: 6257 timeout: 0 lvb_type: 0 00000004:00010000:2.0:1776178212.342934:0:6257:0:(mdt_handler.c:3992:mdt_save_lock()) ### save lock request 00000000575d0558 reply state 00000000ba78875e transno 12884901970 ns: mdt-lustre-MDT0000_UUID lock: 00000000e4217a5c/0xa40083ab9bf5397e lrc: 3/0,1 mode: PW/PW res: [0x2000013a1:0x9:0x0].0x7fbd58ea bits 0x2/0x0 rrc: 2 type: IBT gid 0 flags: 0x40210000000000 nid: local remote: 0x0 expref: -99 pid: 6257 timeout: 0 lvb_type: 0 00010000:00010000:2.0:1776178212.342970:0:6257:0:(ldlm_lock.c:216:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: mdt-lustre-MDT0000_UUID lock: 000000000396420a/0xa40083ab9bf53970 lrc: 0/0,0 mode: --/CW res: [0x2000013a1:0x9:0x0].0x0 bits 0x1/0x0 rrc: 4 type: IBT gid 0 flags: 0x44000000000000 nid: 192.168.204.3@tcp remote: 0x771f68aa1eb27c0d expref: 52 pid: 6257 timeout: 0 lvb_type: 0 00010000:00010000:2.0:1776178212.342995:0:6257:0:(ldlm_lockd.c:1484:ldlm_handle_enqueue0()) ### server-side enqueue handler, sending reply (err=0, rc=0) ns: mdt-lustre-MDT0000_UUID lock: 000000007da3b625/0xa40083ab9bf53985 lrc: 3/0,0 mode: CR/CR res: [0x2000013a1:0x10:0x0].0x0 bits 0x9/0x0 rrc: 2 type: IBT gid 0 flags: 0x40200000000000 nid: 192.168.204.3@tcp remote: 0x771f68aa1eb27c0d expref: 51 pid: 6257 timeout: 0 lvb_type: 0 00010000:00010000:2.0:1776178212.343001:0:6257:0:(ldlm_inodebits.c:94:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x2000013a1:0x10:0x0].0x0 (000000003da5851e) 00010000:00010000:2.0:1776178212.343003:0:6257:0:(ldlm_inodebits.c:96:ldlm_reprocess_inodebits_queue()) Hint 9 00010000:00010000:2.0:1776178212.343005:0:6257:0:(ldlm_lockd.c:1565:ldlm_handle_enqueue0()) ### server-side enqueue handler END (lock 000000007da3b625, rc 0) 00000100:00100000:2.0:1776178212.343132:0:6257:0:(service.c:2372:ptlrpc_server_handle_request()) Handled RPC req@00000000575d0558 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_002:b50963bd-8eba-4432-9a54-8888d00e3fa1+52:18263:x1862456979106816:12345-192.168.204.3@tcp:101:touch.0 Request processed in 1350us (1546us total) trans 12884901970 rc 0/0 00000100:00100000:2.0:1776178212.343141:0:6257:0:(nrs_fifo.c:241:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.204.3@tcp, seq: 6614 00010000:00010000:2.0:1776178212.345610:0:3339:0:(ldlm_lock.c:1064:ldlm_granted_list_add_lock()) ### About to add lock: ns: mdt-lustre-MDT0000_UUID lock: 00000000e4217a5c/0xa40083ab9bf5397e lrc: 3/0,1 mode: COS/COS res: [0x2000013a1:0x9:0x0].0x7fbd58ea bits 0x2/0x0 rrc: 2 type: IBT gid 0 flags: 0x50210000000000 nid: local remote: 0x0 expref: -99 pid: 6257 timeout: 0 lvb_type: 0 00010000:00010000:2.0:1776178212.345633:0:3339:0:(ldlm_inodebits.c:94:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x2000013a1:0x9:0x0].0x7fbd58ea (00000000ccda0888) 00010000:00010000:2.0:1776178212.345636:0:3339:0:(ldlm_inodebits.c:96:ldlm_reprocess_inodebits_queue()) Hint 2 00000100:00100000:0.0:1776178212.347632:0:3346:0:(events.c:361:request_in_callback()) peer: 12345-192.168.204.3@tcp (source: 12345-192.168.204.3@tcp) 00000100:00100000:2.0:1776178212.348001:0:6257:0:(service.c:2152:ptlrpc_server_handle_req_in()) got req x1862456979106944 00000100:00100000:2.0:1776178212.348091:0:6257:0:(nrs_fifo.c:179:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.204.3@tcp, seq: 6615 00000100:00100000:2.0:1776178212.348099:0:6257:0:(service.c:2324:ptlrpc_server_handle_request()) Handling RPC req@00000000ecb64602 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_002:b50963bd-8eba-4432-9a54-8888d00e3fa1+52:18263:x1862456979106944:12345-192.168.204.3@tcp:36:touch.0 00010000:00010000:2.0:1776178212.348226:0:6257:0:(ldlm_lock.c:766:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: mdt-lustre-MDT0000_UUID lock: 000000002f1f7bd5/0xa40083ab9bf53993 lrc: 3/0,1 mode: --/PW res: [0x2000013a1:0x10:0x0].0x0 bits 0x0/0x0 rrc: 3 type: IBT gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 6257 timeout: 0 lvb_type: 0 00010000:00010000:2.0:1776178212.348242:0:6257:0:(ldlm_lock.c:1064:ldlm_granted_list_add_lock()) ### About to add lock: ns: mdt-lustre-MDT0000_UUID lock: 000000002f1f7bd5/0xa40083ab9bf53993 lrc: 3/0,1 mode: PW/PW res: [0x2000013a1:0x10:0x0].0x0 bits 0x2/0x0 rrc: 3 type: IBT gid 0 flags: 0x50210000000000 nid: local remote: 0x0 expref: -99 pid: 6257 timeout: 0 lvb_type: 0 00010000:00010000:2.0:1776178212.348255:0:6257:0:(ldlm_request.c:528:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: mdt-lustre-MDT0000_UUID lock: 000000002f1f7bd5/0xa40083ab9bf53993 lrc: 3/0,1 mode: PW/PW res: [0x2000013a1:0x10:0x0].0x0 bits 0x2/0x0 rrc: 3 type: IBT gid 0 flags: 0x40210000000000 nid: local remote: 0x0 expref: -99 pid: 6257 timeout: 0 lvb_type: 0 00000004:00010000:2.0:1776178212.348381:0:6257:0:(mdt_handler.c:3992:mdt_save_lock()) ### save lock request 00000000ecb64602 reply state 000000003b471bb6 transno 12884901971 ns: mdt-lustre-MDT0000_UUID lock: 000000002f1f7bd5/0xa40083ab9bf53993 lrc: 3/0,1 mode: PW/PW res: [0x2000013a1:0x10:0x0].0x0 bits 0x2/0x0 rrc: 3 type: IBT gid 0 flags: 0x40210000000000 nid: local remote: 0x0 expref: -99 pid: 6257 timeout: 0 lvb_type: 0 00000100:00100000:2.0:1776178212.348491:0:6257:0:(service.c:2372:ptlrpc_server_handle_request()) Handled RPC req@00000000ecb64602 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_002:b50963bd-8eba-4432-9a54-8888d00e3fa1+54:18263:x1862456979106944:12345-192.168.204.3@tcp:36:touch.0 Request processed in 391us (869us total) trans 12884901971 rc 0/0 00000100:00100000:2.0:1776178212.348499:0:6257:0:(nrs_fifo.c:241:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.204.3@tcp, seq: 6615 00010000:00010000:3.0:1776178212.352933:0:3340:0:(ldlm_lock.c:1064:ldlm_granted_list_add_lock()) ### About to add lock: ns: mdt-lustre-MDT0000_UUID lock: 000000002f1f7bd5/0xa40083ab9bf53993 lrc: 3/0,1 mode: COS/COS res: [0x2000013a1:0x10:0x0].0x0 bits 0x2/0x0 rrc: 3 type: IBT gid 0 flags: 0x50210000000000 nid: local remote: 0x0 expref: -99 pid: 6257 timeout: 0 lvb_type: 0 00010000:00010000:3.0:1776178212.352961:0:3340:0:(ldlm_inodebits.c:94:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x2000013a1:0x10:0x0].0x0 (000000003da5851e) 00010000:00010000:3.0:1776178212.352964:0:3340:0:(ldlm_inodebits.c:96:ldlm_reprocess_inodebits_queue()) Hint 2 00000100:00100000:0.0:1776178212.353683:0:3346:0:(events.c:361:request_in_callback()) peer: 12345-192.168.204.3@tcp (source: 12345-192.168.204.3@tcp) 00000100:00100000:1.0:1776178212.353776:0:20755:0:(service.c:2152:ptlrpc_server_handle_req_in()) got req x1862456979107008 00000100:00100000:1.0:1776178212.353833:0:20755:0:(nrs_fifo.c:179:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.204.3@tcp, seq: 5291 00000100:00100000:1.0:1776178212.353846:0:20755:0:(service.c:2324:ptlrpc_server_handle_request()) Handling RPC req@00000000f0ef812c pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_006:b50963bd-8eba-4432-9a54-8888d00e3fa1+8:2271:x1862456979107008:12345-192.168.204.3@tcp:2:touch.0 00000100:00100000:1.0:1776178212.354353:0:20755:0:(service.c:2372:ptlrpc_server_handle_request()) Handled RPC req@00000000f0ef812c pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_006:b50963bd-8eba-4432-9a54-8888d00e3fa1+9:2271:x1862456979107008:12345-192.168.204.3@tcp:2:touch.0 Request processed in 507us (673us total) trans 4294969339 rc 0/0 00000100:00100000:1.0:1776178212.354363:0:20755:0:(nrs_fifo.c:241:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.204.3@tcp, seq: 5291 00000100:00100000:0.0:1776178212.358303:0:3347:0:(events.c:361:request_in_callback()) peer: 12345-192.168.204.3@tcp (source: 12345-192.168.204.3@tcp) 00000100:00100000:3.0:1776178212.358477:0:6258:0:(service.c:2152:ptlrpc_server_handle_req_in()) got req x1862456979107136 00000100:00100000:3.0:1776178212.358532:0:6258:0:(nrs_fifo.c:179:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.204.3@tcp, seq: 2154 00000100:00100000:3.0:1776178212.358548:0:6258:0:(service.c:2324:ptlrpc_server_handle_request()) Handling RPC req@0000000089632c95 pname:cluuid+ref:pid:xid:nid:opc:job mdt_rdpg00_000:b50963bd-8eba-4432-9a54-8888d00e3fa1+54:18263:x1862456979107136:12345-192.168.204.3@tcp:35:touch.0 00000100:00100000:3.0:1776178212.359117:0:6258:0:(service.c:2372:ptlrpc_server_handle_request()) Handled RPC req@0000000089632c95 pname:cluuid+ref:pid:xid:nid:opc:job mdt_rdpg00_000:b50963bd-8eba-4432-9a54-8888d00e3fa1+55:18263:x1862456979107136:12345-192.168.204.3@tcp:35:touch.0 Request processed in 575us (823us total) trans 12884901972 rc 0/0 00000100:00100000:3.0:1776178212.359126:0:6258:0:(nrs_fifo.c:241:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.204.3@tcp, seq: 2154 00000100:00100000:0.0:1776178212.704530:0:3353:0:(ptlrpcd.c:415:ptlrpcd_check()) transfer 1 async RPCs [1->0] 00000100:00100000:0.0:1776178212.704565:0:3353:0:(client.c:1733:ptlrpc_send_new_req()) Sending RPC req@00000000df370bee pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_00:lustre-MDT0001-mdtlov_UUID:3353:1862456998049600:0@lo:13:osp-pre-0-1.0 00000100:00100000:0.0:1776178212.704661:0:3353:0:(events.c:361:request_in_callback()) peer: 12345-0@lo (source: 12345-0@lo) 00000100:00100000:3.0:1776178212.705156:0:9554:0:(service.c:2152:ptlrpc_server_handle_req_in()) got req x1862456998049600 00000100:00100000:3.0:1776178212.705265:0:9554:0:(nrs_fifo.c:179:nrs_fifo_req_get()) NRS start fifo request from 12345-0@lo, seq: 486 00000100:00100000:3.0:1776178212.705272:0:9554:0:(service.c:2324:ptlrpc_server_handle_request()) Handling RPC req@0000000050ea472d pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_create00:lustre-MDT0001-mdtlov_UUID+8:3353:x1862456998049600:12345-0@lo:13:osp-pre-0-1.0 00002000:00000024:3.0:1776178212.705314:0:9554:0:(ofd_obd.c:740:ofd_statfs()) blocks cached 315392 granted 484883776 pending 0 free 3923689472 avail 3692167168 00000100:00100000:3.0:1776178212.705388:0:9554:0:(service.c:2372:ptlrpc_server_handle_request()) Handled RPC req@0000000050ea472d pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_create00:lustre-MDT0001-mdtlov_UUID+8:3353:x1862456998049600:12345-0@lo:13:osp-pre-0-1.0 Request processed in 116us (729us total) trans 0 rc 0/0 00000100:00100000:3.0:1776178212.705398:0:9554:0:(nrs_fifo.c:241:nrs_fifo_req_stop()) NRS stop fifo request from 12345-0@lo, seq: 486 00000100:00100000:0.0:1776178212.705486:0:3353:0:(client.c:2215:ptlrpc_check_set()) Completed RPC req@00000000df370bee pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_00:lustre-MDT0001-mdtlov_UUID:3353:1862456998049600:0@lo:13:osp-pre-0-1.0 00000001:00080000:1.0:1776178213.329402:0:20134:0:(tgt_lastrcvd.c:913:tgt_cb_last_committed()) lustre-MDT0000: transno 12884901972 is committed 00010000:00010000:2.0:1776178213.329533:0:3339:0:(ldlm_lock.c:820:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(COS) ns: mdt-lustre-MDT0000_UUID lock: 0000000039ef0a83/0xa40083ab9bf53812 lrc: 3/0,1 mode: COS/COS res: [0x200000007:0x1:0x0].0x1317f023 bits 0x2/0x0 rrc: 2 type: IBT gid 0 flags: 0x50210000000000 nid: local remote: 0x0 expref: -99 pid: 6256 timeout: 0 lvb_type: 0 00010000:00010000:2.0:1776178213.329548:0:3339:0:(ldlm_lock.c:879:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: mdt-lustre-MDT0000_UUID lock: 0000000039ef0a83/0xa40083ab9bf53812 lrc: 2/0,0 mode: COS/COS res: [0x200000007:0x1:0x0].0x1317f023 bits 0x2/0x0 rrc: 2 type: IBT gid 0 flags: 0x50210400000000 nid: local remote: 0x0 expref: -99 pid: 6256 timeout: 0 lvb_type: 0 00010000:00010000:2.0:1776178213.329565:0:3339:0:(ldlm_lockd.c:1879:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: mdt-lustre-MDT0000_UUID lock: 0000000039ef0a83/0xa40083ab9bf53812 lrc: 3/0,0 mode: COS/COS res: [0x200000007:0x1:0x0].0x1317f023 bits 0x2/0x0 rrc: 2 type: IBT gid 0 flags: 0x40210400000000 nid: local remote: 0x0 expref: -99 pid: 6256 timeout: 0 lvb_type: 0 00010000:00010000:2.0:1776178213.329572:0:3339:0:(ldlm_lockd.c:1893:ldlm_handle_bl_callback()) Lock 0000000039ef0a83 already unused, calling callback (0000000079e78348) 00010000:00010000:2.0:1776178213.329575:0:3339:0:(ldlm_request.c:364:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: mdt-lustre-MDT0000_UUID lock: 0000000039ef0a83/0xa40083ab9bf53812 lrc: 3/0,0 mode: COS/COS res: [0x200000007:0x1:0x0].0x1317f023 bits 0x2/0x0 rrc: 2 type: IBT gid 0 flags: 0x40210400000000 nid: local remote: 0x0 expref: -99 pid: 6256 timeout: 0 lvb_type: 0 00010000:00010000:2.0:1776178213.329590:0:3339:0:(ldlm_request.c:1257:ldlm_cli_cancel_local()) ### server-side local cancel ns: mdt-lustre-MDT0000_UUID lock: 0000000039ef0a83/0xa40083ab9bf53812 lrc: 4/0,0 mode: COS/COS res: [0x200000007:0x1:0x0].0x1317f023 bits 0x2/0x0 rrc: 2 type: IBT gid 0 flags: 0x40218400000000 nid: local remote: 0x0 expref: -99 pid: 6256 timeout: 0 lvb_type: 0 00010000:00010000:2.0:1776178213.329610:0:3339:0:(ldlm_inodebits.c:94:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x200000007:0x1:0x0].0x1317f023 (00000000ec624a45) 00010000:00010000:2.0:1776178213.329613:0:3339:0:(ldlm_inodebits.c:96:ldlm_reprocess_inodebits_queue()) Hint 2 00010000:00010000:2.0:1776178213.329618:0:3339:0:(ldlm_lockd.c:1903:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: mdt-lustre-MDT0000_UUID lock: 0000000039ef0a83/0xa40083ab9bf53812 lrc: 2/0,0 mode: --/COS res: [0x200000007:0x1:0x0].0x1317f023 bits 0x2/0x0 rrc: 2 type: IBT gid 0 flags: 0x44a19400000000 nid: local remote: 0x0 expref: -99 pid: 6256 timeout: 0 lvb_type: 0 00010000:00010000:2.0:1776178213.329630:0:3339:0:(ldlm_lock.c:216:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: mdt-lustre-MDT0000_UUID lock: 0000000039ef0a83/0xa40083ab9bf53812 lrc: 0/0,0 mode: --/COS res: [0x200000007:0x1:0x0].0x1317f023 bits 0x2/0x0 rrc: 2 type: IBT gid 0 flags: 0x44a19400000000 nid: local remote: 0x0 expref: -99 pid: 6256 timeout: 0 lvb_type: 0 00010000:00010000:2.0:1776178213.329652:0:3339:0:(ldlm_lock.c:820:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(COS) ns: mdt-lustre-MDT0000_UUID lock: 0000000067990ad0/0xa40083ab9bf53819 lrc: 3/0,1 mode: COS/COS res: [0x2000013a1:0x9:0x0].0x0 bits 0x2/0x0 rrc: 3 type: IBT gid 0 flags: 0x50210000000000 nid: local remote: 0x0 expref: -99 pid: 6256 timeout: 0 lvb_type: 0 00010000:00010000:2.0:1776178213.329658:0:3339:0:(ldlm_lock.c:879:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: mdt-lustre-MDT0000_UUID lock: 0000000067990ad0/0xa40083ab9bf53819 lrc: 2/0,0 mode: COS/COS res: [0x2000013a1:0x9:0x0].0x0 bits 0x2/0x0 rrc: 3 type: IBT gid 0 flags: 0x50210400000000 nid: local remote: 0x0 expref: -99 pid: 6256 timeout: 0 lvb_type: 0 00010000:00010000:2.0:1776178213.329664:0:3339:0:(ldlm_lockd.c:1879:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: mdt-lustre-MDT0000_UUID lock: 0000000067990ad0/0xa40083ab9bf53819 lrc: 3/0,0 mode: COS/COS res: [0x2000013a1:0x9:0x0].0x0 bits 0x2/0x0 rrc: 3 type: IBT gid 0 flags: 0x40210400000000 nid: local remote: 0x0 expref: -99 pid: 6256 timeout: 0 lvb_type: 0 00010000:00010000:2.0:1776178213.329669:0:3339:0:(ldlm_lockd.c:1893:ldlm_handle_bl_callback()) Lock 0000000067990ad0 already unused, calling callback (0000000079e78348) 00010000:00010000:2.0:1776178213.329671:0:3339:0:(ldlm_request.c:364:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: mdt-lustre-MDT0000_UUID lock: 0000000067990ad0/0xa40083ab9bf53819 lrc: 3/0,0 mode: COS/COS res: [0x2000013a1:0x9:0x0].0x0 bits 0x2/0x0 rrc: 3 type: IBT gid 0 flags: 0x40210400000000 nid: local remote: 0x0 expref: -99 pid: 6256 timeout: 0 lvb_type: 0 00010000:00010000:2.0:1776178213.329677:0:3339:0:(ldlm_request.c:1257:ldlm_cli_cancel_local()) ### server-side local cancel ns: mdt-lustre-MDT0000_UUID lock: 0000000067990ad0/0xa40083ab9bf53819 lrc: 4/0,0 mode: COS/COS res: [0x2000013a1:0x9:0x0].0x0 bits 0x2/0x0 rrc: 3 type: IBT gid 0 flags: 0x40218400000000 nid: local remote: 0x0 expref: -99 pid: 6256 timeout: 0 lvb_type: 0 00010000:00010000:2.0:1776178213.329685:0:3339:0:(ldlm_inodebits.c:94:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x2000013a1:0x9:0x0].0x0 (00000000a041a513) 00010000:00010000:2.0:1776178213.329688:0:3339:0:(ldlm_inodebits.c:96:ldlm_reprocess_inodebits_queue()) Hint 2 00010000:00010000:2.0:1776178213.329690:0:3339:0:(ldlm_lockd.c:1903:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: mdt-lustre-MDT0000_UUID lock: 0000000067990ad0/0xa40083ab9bf53819 lrc: 2/0,0 mode: --/COS res: [0x2000013a1:0x9:0x0].0x0 bits 0x2/0x0 rrc: 3 type: IBT gid 0 flags: 0x44a19400000000 nid: local remote: 0x0 expref: -99 pid: 6256 timeout: 0 lvb_type: 0 00010000:00010000:2.0:1776178213.329695:0:3339:0:(ldlm_lock.c:216:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: mdt-lustre-MDT0000_UUID lock: 0000000067990ad0/0xa40083ab9bf53819 lrc: 0/0,0 mode: --/COS res: [0x2000013a1:0x9:0x0].0x0 bits 0x2/0x0 rrc: 3 type: IBT gid 0 flags: 0x44a19400000000 nid: local remote: 0x0 expref: -99 pid: 6256 timeout: 0 lvb_type: 0 00010000:00010000:2.0:1776178213.329709:0:3339:0:(ldlm_lock.c:820:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(COS) ns: mdt-lustre-MDT0000_UUID lock: 0000000095e53696/0xa40083ab9bf53882 lrc: 3/0,1 mode: COS/COS res: [0x2000013a1:0x9:0x0].0xcda302ea bits 0x2/0x0 rrc: 2 type: IBT gid 0 flags: 0x50210000000000 nid: local remote: 0x0 expref: -99 pid: 6257 timeout: 0 lvb_type: 0 00010000:00010000:2.0:1776178213.329721:0:3339:0:(ldlm_lock.c:879:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: mdt-lustre-MDT0000_UUID lock: 0000000095e53696/0xa40083ab9bf53882 lrc: 2/0,0 mode: COS/COS res: [0x2000013a1:0x9:0x0].0xcda302ea bits 0x2/0x0 rrc: 2 type: IBT gid 0 flags: 0x50210400000000 nid: local remote: 0x0 expref: -99 pid: 6257 timeout: 0 lvb_type: 0 00010000:00010000:2.0:1776178213.329725:0:3339:0:(ldlm_lockd.c:1879:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: mdt-lustre-MDT0000_UUID lock: 0000000095e53696/0xa40083ab9bf53882 lrc: 3/0,0 mode: COS/COS res: [0x2000013a1:0x9:0x0].0xcda302ea bits 0x2/0x0 rrc: 2 type: IBT gid 0 flags: 0x40210400000000 nid: local remote: 0x0 expref: -99 pid: 6257 timeout: 0 lvb_type: 0 00010000:00010000:2.0:1776178213.329731:0:3339:0:(ldlm_lockd.c:1893:ldlm_handle_bl_callback()) Lock 0000000095e53696 already unused, calling callback (0000000079e78348) 00010000:00010000:2.0:1776178213.329732:0:3339:0:(ldlm_request.c:364:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: mdt-lustre-MDT0000_UUID lock: 0000000095e53696/0xa40083ab9bf53882 lrc: 3/0,0 mode: COS/COS res: [0x2000013a1:0x9:0x0].0xcda302ea bits 0x2/0x0 rrc: 2 type: IBT gid 0 flags: 0x40210400000000 nid: local remote: 0x0 expref: -99 pid: 6257 timeout: 0 lvb_type: 0 00010000:00010000:2.0:1776178213.329758:0:3339:0:(ldlm_request.c:1257:ldlm_cli_cancel_local()) ### server-side local cancel ns: mdt-lustre-MDT0000_UUID lock: 0000000095e53696/0xa40083ab9bf53882 lrc: 4/0,0 mode: COS/COS res: [0x2000013a1:0x9:0x0].0xcda302ea bits 0x2/0x0 rrc: 2 type: IBT gid 0 flags: 0x40218400000000 nid: local remote: 0x0 expref: -99 pid: 6257 timeout: 0 lvb_type: 0 00010000:00010000:2.0:1776178213.329766:0:3339:0:(ldlm_inodebits.c:94:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x2000013a1:0x9:0x0].0xcda302ea (00000000b86c95e3) 00010000:00010000:2.0:1776178213.329769:0:3339:0:(ldlm_inodebits.c:96:ldlm_reprocess_inodebits_queue()) Hint 2 00010000:00010000:2.0:1776178213.329771:0:3339:0:(ldlm_lockd.c:1903:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: mdt-lustre-MDT0000_UUID lock: 0000000095e53696/0xa40083ab9bf53882 lrc: 2/0,0 mode: --/COS res: [0x2000013a1:0x9:0x0].0xcda302ea bits 0x2/0x0 rrc: 2 type: IBT gid 0 flags: 0x44a19400000000 nid: local remote: 0x0 expref: -99 pid: 6257 timeout: 0 lvb_type: 0 00010000:00010000:2.0:1776178213.329776:0:3339:0:(ldlm_lock.c:216:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: mdt-lustre-MDT0000_UUID lock: 0000000095e53696/0xa40083ab9bf53882 lrc: 0/0,0 mode: --/COS res: [0x2000013a1:0x9:0x0].0xcda302ea bits 0x2/0x0 rrc: 2 type: IBT gid 0 flags: 0x44a19400000000 nid: local remote: 0x0 expref: -99 pid: 6257 timeout: 0 lvb_type: 0 00010000:00010000:2.0:1776178213.329789:0:3339:0:(ldlm_lock.c:820:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(COS) ns: mdt-lustre-MDT0000_UUID lock: 000000006b206cc9/0xa40083ab9bf53897 lrc: 3/0,1 mode: COS/COS res: [0x2000013a1:0xa:0x0].0x0 bits 0x2/0x0 rrc: 3 type: IBT gid 0 flags: 0x50210000000000 nid: local remote: 0x0 expref: -99 pid: 6256 timeout: 0 lvb_type: 0 00010000:00010000:2.0:1776178213.329794:0:3339:0:(ldlm_lock.c:879:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: mdt-lustre-MDT0000_UUID lock: 000000006b206cc9/0xa40083ab9bf53897 lrc: 2/0,0 mode: COS/COS res: [0x2000013a1:0xa:0x0].0x0 bits 0x2/0x0 rrc: 3 type: IBT gid 0 flags: 0x50210400000000 nid: local remote: 0x0 expref: -99 pid: 6256 timeout: 0 lvb_type: 0 00010000:00010000:2.0:1776178213.329799:0:3339:0:(ldlm_lockd.c:1879:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: mdt-lustre-MDT0000_UUID lock: 000000006b206cc9/0xa40083ab9bf53897 lrc: 3/0,0 mode: COS/COS res: [0x2000013a1:0xa:0x0].0x0 bits 0x2/0x0 rrc: 3 type: IBT gid 0 flags: 0x40210400000000 nid: local remote: 0x0 expref: -99 pid: 6256 timeout: 0 lvb_type: 0 00010000:00010000:2.0:1776178213.329819:0:3339:0:(ldlm_lockd.c:1893:ldlm_handle_bl_callback()) Lock 000000006b206cc9 already unused, calling callback (0000000079e78348) 00010000:00010000:2.0:1776178213.329821:0:3339:0:(ldlm_request.c:364:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: mdt-lustre-MDT0000_UUID lock: 000000006b206cc9/0xa40083ab9bf53897 lrc: 3/0,0 mode: COS/COS res: [0x2000013a1:0xa:0x0].0x0 bits 0x2/0x0 rrc: 3 type: IBT gid 0 flags: 0x40210400000000 nid: local remote: 0x0 expref: -99 pid: 6256 timeout: 0 lvb_type: 0 00010000:00010000:2.0:1776178213.329827:0:3339:0:(ldlm_request.c:1257:ldlm_cli_cancel_local()) ### server-side local cancel ns: mdt-lustre-MDT0000_UUID lock: 000000006b206cc9/0xa40083ab9bf53897 lrc: 4/0,0 mode: COS/COS res: [0x2000013a1:0xa:0x0].0x0 bits 0x2/0x0 rrc: 3 type: IBT gid 0 flags: 0x40218400000000 nid: local remote: 0x0 expref: -99 pid: 6256 timeout: 0 lvb_type: 0 00010000:00010000:2.0:1776178213.329835:0:3339:0:(ldlm_inodebits.c:94:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x2000013a1:0xa:0x0].0x0 (000000000a3c1046) 00010000:00010000:2.0:1776178213.329837:0:3339:0:(ldlm_inodebits.c:96:ldlm_reprocess_inodebits_queue()) Hint 2 00010000:00010000:2.0:1776178213.329839:0:3339:0:(ldlm_lockd.c:1903:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: mdt-lustre-MDT0000_UUID lock: 000000006b206cc9/0xa40083ab9bf53897 lrc: 2/0,0 mode: --/COS res: [0x2000013a1:0xa:0x0].0x0 bits 0x2/0x0 rrc: 3 type: IBT gid 0 flags: 0x44a19400000000 nid: local remote: 0x0 expref: -99 pid: 6256 timeout: 0 lvb_type: 0 00010000:00010000:2.0:1776178213.329845:0:3339:0:(ldlm_lock.c:216:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: mdt-lustre-MDT0000_UUID lock: 000000006b206cc9/0xa40083ab9bf53897 lrc: 0/0,0 mode: --/COS res: [0x2000013a1:0xa:0x0].0x0 bits 0x2/0x0 rrc: 3 type: IBT gid 0 flags: 0x44a19400000000 nid: local remote: 0x0 expref: -99 pid: 6256 timeout: 0 lvb_type: 0 00010000:00010000:2.0:1776178213.329857:0:3339:0:(ldlm_lock.c:820:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(COS) ns: mdt-lustre-MDT0000_UUID lock: 00000000a14e680b/0xa40083ab9bf538ac lrc: 3/0,1 mode: COS/COS res: [0x2000013a1:0x9:0x0].0x15fcbbea bits 0x2/0x0 rrc: 2 type: IBT gid 0 flags: 0x50210000000000 nid: local remote: 0x0 expref: -99 pid: 6256 timeout: 0 lvb_type: 0 00010000:00010000:2.0:1776178213.329863:0:3339:0:(ldlm_lock.c:879:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: mdt-lustre-MDT0000_UUID lock: 00000000a14e680b/0xa40083ab9bf538ac lrc: 2/0,0 mode: COS/COS res: [0x2000013a1:0x9:0x0].0x15fcbbea bits 0x2/0x0 rrc: 2 type: IBT gid 0 flags: 0x50210400000000 nid: local remote: 0x0 expref: -99 pid: 6256 timeout: 0 lvb_type: 0 00010000:00010000:2.0:1776178213.329868:0:3339:0:(ldlm_lockd.c:1879:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: mdt-lustre-MDT0000_UUID lock: 00000000a14e680b/0xa40083ab9bf538ac lrc: 3/0,0 mode: COS/COS res: [0x2000013a1:0x9:0x0].0x15fcbbea bits 0x2/0x0 rrc: 2 type: IBT gid 0 flags: 0x40210400000000 nid: local remote: 0x0 expref: -99 pid: 6256 timeout: 0 lvb_type: 0 00010000:00010000:2.0:1776178213.329873:0:3339:0:(ldlm_lockd.c:1893:ldlm_handle_bl_callback()) Lock 00000000a14e680b already unused, calling callback (0000000079e78348) 00010000:00010000:2.0:1776178213.329876:0:3339:0:(ldlm_request.c:364:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: mdt-lustre-MDT0000_UUID lock: 00000000a14e680b/0xa40083ab9bf538ac lrc: 3/0,0 mode: COS/COS res: [0x2000013a1:0x9:0x0].0x15fcbbea bits 0x2/0x0 rrc: 2 type: IBT gid 0 flags: 0x40210400000000 nid: local remote: 0x0 expref: -99 pid: 6256 timeout: 0 lvb_type: 0 00010000:00010000:2.0:1776178213.329882:0:3339:0:(ldlm_request.c:1257:ldlm_cli_cancel_local()) ### server-side local cancel ns: mdt-lustre-MDT0000_UUID lock: 00000000a14e680b/0xa40083ab9bf538ac lrc: 4/0,0 mode: COS/COS res: [0x2000013a1:0x9:0x0].0x15fcbbea bits 0x2/0x0 rrc: 2 type: IBT gid 0 flags: 0x40218400000000 nid: local remote: 0x0 expref: -99 pid: 6256 timeout: 0 lvb_type: 0 00010000:00010000:2.0:1776178213.329891:0:3339:0:(ldlm_inodebits.c:94:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x2000013a1:0x9:0x0].0x15fcbbea (000000001fd37005) 00010000:00010000:2.0:1776178213.329893:0:3339:0:(ldlm_inodebits.c:96:ldlm_reprocess_inodebits_queue()) Hint 2 00000001:00080000:0.0:1776178213.329895:0:8225:0:(tgt_lastrcvd.c:913:tgt_cb_last_committed()) lustre-OST0000: transno 4294969364 is committed 00010000:00010000:2.0:1776178213.329896:0:3339:0:(ldlm_lockd.c:1903:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: mdt-lustre-MDT0000_UUID lock: 00000000a14e680b/0xa40083ab9bf538ac lrc: 2/0,0 mode: --/COS res: [0x2000013a1:0x9:0x0].0x15fcbbea bits 0x2/0x0 rrc: 2 type: IBT gid 0 flags: 0x44a19400000000 nid: local remote: 0x0 expref: -99 pid: 6256 timeout: 0 lvb_type: 0 00010000:00010000:2.0:1776178213.329905:0:3339:0:(ldlm_lock.c:216:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: mdt-lustre-MDT0000_UUID lock: 00000000a14e680b/0xa40083ab9bf538ac lrc: 0/0,0 mode: --/COS res: [0x2000013a1:0x9:0x0].0x15fcbbea bits 0x2/0x0 rrc: 2 type: IBT gid 0 flags: 0x44a19400000000 nid: local remote: 0x0 expref: -99 pid: 6256 timeout: 0 lvb_type: 0 00000001:00080000:0.0:1776178213.329907:0:8225:0:(tgt_lastrcvd.c:913:tgt_cb_last_committed()) lustre-OST0000: transno 4294969361 is committed 00010000:00010000:2.0:1776178213.329924:0:3339:0:(ldlm_lock.c:820:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(COS) ns: mdt-lustre-MDT0000_UUID lock: 0000000097975a3b/0xa40083ab9bf538c1 lrc: 3/0,1 mode: COS/COS res: [0x2000013a1:0xb:0x0].0x0 bits 0x2/0x0 rrc: 3 type: IBT gid 0 flags: 0x50210000000000 nid: local remote: 0x0 expref: -99 pid: 6256 timeout: 0 lvb_type: 0 00010000:00010000:2.0:1776178213.329931:0:3339:0:(ldlm_lock.c:879:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: mdt-lustre-MDT0000_UUID lock: 0000000097975a3b/0xa40083ab9bf538c1 lrc: 2/0,0 mode: COS/COS res: [0x2000013a1:0xb:0x0].0x0 bits 0x2/0x0 rrc: 3 type: IBT gid 0 flags: 0x50210400000000 nid: local remote: 0x0 expref: -99 pid: 6256 timeout: 0 lvb_type: 0 00010000:00010000:2.0:1776178213.329936:0:3339:0:(ldlm_lockd.c:1879:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: mdt-lustre-MDT0000_UUID lock: 0000000097975a3b/0xa40083ab9bf538c1 lrc: 3/0,0 mode: COS/COS res: [0x2000013a1:0xb:0x0].0x0 bits 0x2/0x0 rrc: 3 type: IBT gid 0 flags: 0x40210400000000 nid: local remote: 0x0 expref: -99 pid: 6256 timeout: 0 lvb_type: 0 00010000:00010000:2.0:1776178213.329942:0:3339:0:(ldlm_lockd.c:1893:ldlm_handle_bl_callback()) Lock 0000000097975a3b already unused, calling callback (0000000079e78348) 00010000:00010000:2.0:1776178213.329944:0:3339:0:(ldlm_request.c:364:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: mdt-lustre-MDT0000_UUID lock: 0000000097975a3b/0xa40083ab9bf538c1 lrc: 3/0,0 mode: COS/COS res: [0x2000013a1:0xb:0x0].0x0 bits 0x2/0x0 rrc: 3 type: IBT gid 0 flags: 0x40210400000000 nid: local remote: 0x0 expref: -99 pid: 6256 timeout: 0 lvb_type: 0 00010000:00010000:2.0:1776178213.329949:0:3339:0:(ldlm_request.c:1257:ldlm_cli_cancel_local()) ### server-side local cancel ns: mdt-lustre-MDT0000_UUID lock: 0000000097975a3b/0xa40083ab9bf538c1 lrc: 4/0,0 mode: COS/COS res: [0x2000013a1:0xb:0x0].0x0 bits 0x2/0x0 rrc: 3 type: IBT gid 0 flags: 0x40218400000000 nid: local remote: 0x0 expref: -99 pid: 6256 timeout: 0 lvb_type: 0 00010000:00010000:2.0:1776178213.329956:0:3339:0:(ldlm_inodebits.c:94:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x2000013a1:0xb:0x0].0x0 (000000003275315d) 00010000:00010000:2.0:1776178213.329958:0:3339:0:(ldlm_inodebits.c:96:ldlm_reprocess_inodebits_queue()) Hint 2 00010000:00010000:2.0:1776178213.329959:0:3339:0:(ldlm_lockd.c:1903:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: mdt-lustre-MDT0000_UUID lock: 0000000097975a3b/0xa40083ab9bf538c1 lrc: 2/0,0 mode: --/COS res: [0x2000013a1:0xb:0x0].0x0 bits 0x2/0x0 rrc: 3 type: IBT gid 0 flags: 0x44a19400000000 nid: local remote: 0x0 expref: -99 pid: 6256 timeout: 0 lvb_type: 0 00010000:00010000:2.0:1776178213.329969:0:3339:0:(ldlm_lock.c:216:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: mdt-lustre-MDT0000_UUID lock: 0000000097975a3b/0xa40083ab9bf538c1 lrc: 0/0,0 mode: --/COS res: [0x2000013a1:0xb:0x0].0x0 bits 0x2/0x0 rrc: 3 type: IBT gid 0 flags: 0x44a19400000000 nid: local remote: 0x0 expref: -99 pid: 6256 timeout: 0 lvb_type: 0 00010000:00010000:2.0:1776178213.329980:0:3339:0:(ldlm_lock.c:820:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(COS) ns: mdt-lustre-MDT0000_UUID lock: 0000000096c1e297/0xa40083ab9bf538d6 lrc: 3/0,1 mode: COS/COS res: [0x2000013a1:0x9:0x0].0x5e5674ea bits 0x2/0x0 rrc: 2 type: IBT gid 0 flags: 0x50210000000000 nid: local remote: 0x0 expref: -99 pid: 6257 timeout: 0 lvb_type: 0 00010000:00010000:2.0:1776178213.329985:0:3339:0:(ldlm_lock.c:879:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: mdt-lustre-MDT0000_UUID lock: 0000000096c1e297/0xa40083ab9bf538d6 lrc: 2/0,0 mode: COS/COS res: [0x2000013a1:0x9:0x0].0x5e5674ea bits 0x2/0x0 rrc: 2 type: IBT gid 0 flags: 0x50210400000000 nid: local remote: 0x0 expref: -99 pid: 6257 timeout: 0 lvb_type: 0 00010000:00010000:2.0:1776178213.329989:0:3339:0:(ldlm_lockd.c:1879:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: mdt-lustre-MDT0000_UUID lock: 0000000096c1e297/0xa40083ab9bf538d6 lrc: 3/0,0 mode: COS/COS res: [0x2000013a1:0x9:0x0].0x5e5674ea bits 0x2/0x0 rrc: 2 type: IBT gid 0 flags: 0x40210400000000 nid: local remote: 0x0 expref: -99 pid: 6257 timeout: 0 lvb_type: 0 00010000:00010000:2.0:1776178213.329994:0:3339:0:(ldlm_lockd.c:1893:ldlm_handle_bl_callback()) Lock 0000000096c1e297 already unused, calling callback (0000000079e78348) 00010000:00010000:2.0:1776178213.329996:0:3339:0:(ldlm_request.c:364:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: mdt-lustre-MDT0000_UUID lock: 0000000096c1e297/0xa40083ab9bf538d6 lrc: 3/0,0 mode: COS/COS res: [0x2000013a1:0x9:0x0].0x5e5674ea bits 0x2/0x0 rrc: 2 type: IBT gid 0 flags: 0x40210400000000 nid: local remote: 0x0 expref: -99 pid: 6257 timeout: 0 lvb_type: 0 00010000:00010000:2.0:1776178213.330000:0:3339:0:(ldlm_request.c:1257:ldlm_cli_cancel_local()) ### server-side local cancel ns: mdt-lustre-MDT0000_UUID lock: 0000000096c1e297/0xa40083ab9bf538d6 lrc: 4/0,0 mode: COS/COS res: [0x2000013a1:0x9:0x0].0x5e5674ea bits 0x2/0x0 rrc: 2 type: IBT gid 0 flags: 0x40218400000000 nid: local remote: 0x0 expref: -99 pid: 6257 timeout: 0 lvb_type: 0 00010000:00010000:2.0:1776178213.330007:0:3339:0:(ldlm_inodebits.c:94:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x2000013a1:0x9:0x0].0x5e5674ea (00000000024b4e8f) 00010000:00010000:2.0:1776178213.330010:0:3339:0:(ldlm_inodebits.c:96:ldlm_reprocess_inodebits_queue()) Hint 2 00010000:00010000:2.0:1776178213.330012:0:3339:0:(ldlm_lockd.c:1903:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: mdt-lustre-MDT0000_UUID lock: 0000000096c1e297/0xa40083ab9bf538d6 lrc: 2/0,0 mode: --/COS res: [0x2000013a1:0x9:0x0].0x5e5674ea bits 0x2/0x0 rrc: 2 type: IBT gid 0 flags: 0x44a19400000000 nid: local remote: 0x0 expref: -99 pid: 6257 timeout: 0 lvb_type: 0 00010000:00010000:2.0:1776178213.330016:0:3339:0:(ldlm_lock.c:216:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: mdt-lustre-MDT0000_UUID lock: 0000000096c1e297/0xa40083ab9bf538d6 lrc: 0/0,0 mode: --/COS res: [0x2000013a1:0x9:0x0].0x5e5674ea bits 0x2/0x0 rrc: 2 type: IBT gid 0 flags: 0x44a19400000000 nid: local remote: 0x0 expref: -99 pid: 6257 timeout: 0 lvb_type: 0 00010000:00010000:2.0:1776178213.330028:0:3339:0:(ldlm_lock.c:820:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(COS) ns: mdt-lustre-MDT0000_UUID lock: 00000000c9d7d9fc/0xa40083ab9bf538eb lrc: 3/0,1 mode: COS/COS res: [0x2000013a1:0xc:0x0].0x0 bits 0x2/0x0 rrc: 3 type: IBT gid 0 flags: 0x50210000000000 nid: local remote: 0x0 expref: -99 pid: 6257 timeout: 0 lvb_type: 0 00010000:00010000:2.0:1776178213.330038:0:3339:0:(ldlm_lock.c:879:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: mdt-lustre-MDT0000_UUID lock: 00000000c9d7d9fc/0xa40083ab9bf538eb lrc: 2/0,0 mode: COS/COS res: [0x2000013a1:0xc:0x0].0x0 bits 0x2/0x0 rrc: 3 type: IBT gid 0 flags: 0x50210400000000 nid: local remote: 0x0 expref: -99 pid: 6257 timeout: 0 lvb_type: 0 00010000:00010000:2.0:1776178213.330044:0:3339:0:(ldlm_lockd.c:1879:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: mdt-lustre-MDT0000_UUID lock: 00000000c9d7d9fc/0xa40083ab9bf538eb lrc: 3/0,0 mode: COS/COS res: [0x2000013a1:0xc:0x0].0x0 bits 0x2/0x0 rrc: 3 type: IBT gid 0 flags: 0x40210400000000 nid: local remote: 0x0 expref: -99 pid: 6257 timeout: 0 lvb_type: 0 00010000:00010000:2.0:1776178213.330048:0:3339:0:(ldlm_lockd.c:1893:ldlm_handle_bl_callback()) Lock 00000000c9d7d9fc already unused, calling callback (0000000079e78348) 00010000:00010000:2.0:1776178213.330050:0:3339:0:(ldlm_request.c:364:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: mdt-lustre-MDT0000_UUID lock: 00000000c9d7d9fc/0xa40083ab9bf538eb lrc: 3/0,0 mode: COS/COS res: [0x2000013a1:0xc:0x0].0x0 bits 0x2/0x0 rrc: 3 type: IBT gid 0 flags: 0x40210400000000 nid: local remote: 0x0 expref: -99 pid: 6257 timeout: 0 lvb_type: 0 00010000:00010000:2.0:1776178213.330055:0:3339:0:(ldlm_request.c:1257:ldlm_cli_cancel_local()) ### server-side local cancel ns: mdt-lustre-MDT0000_UUID lock: 00000000c9d7d9fc/0xa40083ab9bf538eb lrc: 4/0,0 mode: COS/COS res: [0x2000013a1:0xc:0x0].0x0 bits 0x2/0x0 rrc: 3 type: IBT gid 0 flags: 0x40218400000000 nid: local remote: 0x0 expref: -99 pid: 6257 timeout: 0 lvb_type: 0 00010000:00010000:2.0:1776178213.330125:0:3339:0:(ldlm_inodebits.c:94:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x2000013a1:0xc:0x0].0x0 (0000000062690614) 00010000:00010000:2.0:1776178213.330127:0:3339:0:(ldlm_inodebits.c:96:ldlm_reprocess_inodebits_queue()) Hint 2 00010000:00010000:2.0:1776178213.330130:0:3339:0:(ldlm_lockd.c:1903:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: mdt-lustre-MDT0000_UUID lock: 00000000c9d7d9fc/0xa40083ab9bf538eb lrc: 2/0,0 mode: --/COS res: [0x2000013a1:0xc:0x0].0x0 bits 0x2/0x0 rrc: 3 type: IBT gid 0 flags: 0x44a19400000000 nid: local remote: 0x0 expref: -99 pid: 6257 timeout: 0 lvb_type: 0 00010000:00010000:2.0:1776178213.330135:0:3339:0:(ldlm_lock.c:216:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: mdt-lustre-MDT0000_UUID lock: 00000000c9d7d9fc/0xa40083ab9bf538eb lrc: 0/0,0 mode: --/COS res: [0x2000013a1:0xc:0x0].0x0 bits 0x2/0x0 rrc: 3 type: IBT gid 0 flags: 0x44a19400000000 nid: local remote: 0x0 expref: -99 pid: 6257 timeout: 0 lvb_type: 0 00010000:00010000:2.0:1776178213.330147:0:3339:0:(ldlm_lock.c:820:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(COS) ns: mdt-lustre-MDT0000_UUID lock: 00000000b2e83690/0xa40083ab9bf53900 lrc: 3/0,1 mode: COS/COS res: [0x2000013a1:0x9:0x0].0xa6b02dea bits 0x2/0x0 rrc: 2 type: IBT gid 0 flags: 0x50210000000000 nid: local remote: 0x0 expref: -99 pid: 6256 timeout: 0 lvb_type: 0 00010000:00010000:2.0:1776178213.330152:0:3339:0:(ldlm_lock.c:879:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: mdt-lustre-MDT0000_UUID lock: 00000000b2e83690/0xa40083ab9bf53900 lrc: 2/0,0 mode: COS/COS res: [0x2000013a1:0x9:0x0].0xa6b02dea bits 0x2/0x0 rrc: 2 type: IBT gid 0 flags: 0x50210400000000 nid: local remote: 0x0 expref: -99 pid: 6256 timeout: 0 lvb_type: 0 00010000:00010000:2.0:1776178213.330158:0:3339:0:(ldlm_lockd.c:1879:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: mdt-lustre-MDT0000_UUID lock: 00000000b2e83690/0xa40083ab9bf53900 lrc: 3/0,0 mode: COS/COS res: [0x2000013a1:0x9:0x0].0xa6b02dea bits 0x2/0x0 rrc: 2 type: IBT gid 0 flags: 0x40210400000000 nid: local remote: 0x0 expref: -99 pid: 6256 timeout: 0 lvb_type: 0 00010000:00010000:2.0:1776178213.330167:0:3339:0:(ldlm_lockd.c:1893:ldlm_handle_bl_callback()) Lock 00000000b2e83690 already unused, calling callback (0000000079e78348) 00010000:00010000:2.0:1776178213.330169:0:3339:0:(ldlm_request.c:364:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: mdt-lustre-MDT0000_UUID lock: 00000000b2e83690/0xa40083ab9bf53900 lrc: 3/0,0 mode: COS/COS res: [0x2000013a1:0x9:0x0].0xa6b02dea bits 0x2/0x0 rrc: 2 type: IBT gid 0 flags: 0x40210400000000 nid: local remote: 0x0 expref: -99 pid: 6256 timeout: 0 lvb_type: 0 00010000:00010000:2.0:1776178213.330174:0:3339:0:(ldlm_request.c:1257:ldlm_cli_cancel_local()) ### server-side local cancel ns: mdt-lustre-MDT0000_UUID lock: 00000000b2e83690/0xa40083ab9bf53900 lrc: 4/0,0 mode: COS/COS res: [0x2000013a1:0x9:0x0].0xa6b02dea bits 0x2/0x0 rrc: 2 type: IBT gid 0 flags: 0x40218400000000 nid: local remote: 0x0 expref: -99 pid: 6256 timeout: 0 lvb_type: 0 00010000:00010000:2.0:1776178213.330182:0:3339:0:(ldlm_inodebits.c:94:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x2000013a1:0x9:0x0].0xa6b02dea (0000000057f156ea) 00010000:00010000:2.0:1776178213.330184:0:3339:0:(ldlm_inodebits.c:96:ldlm_reprocess_inodebits_queue()) Hint 2 00010000:00010000:2.0:1776178213.330185:0:3339:0:(ldlm_lockd.c:1903:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: mdt-lustre-MDT0000_UUID lock: 00000000b2e83690/0xa40083ab9bf53900 lrc: 2/0,0 mode: --/COS res: [0x2000013a1:0x9:0x0].0xa6b02dea bits 0x2/0x0 rrc: 2 type: IBT gid 0 flags: 0x44a19400000000 nid: local remote: 0x0 expref: -99 pid: 6256 timeout: 0 lvb_type: 0 00010000:00010000:2.0:1776178213.330190:0:3339:0:(ldlm_lock.c:216:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: mdt-lustre-MDT0000_UUID lock: 00000000b2e83690/0xa40083ab9bf53900 lrc: 0/0,0 mode: --/COS res: [0x2000013a1:0x9:0x0].0xa6b02dea bits 0x2/0x0 rrc: 2 type: IBT gid 0 flags: 0x44a19400000000 nid: local remote: 0x0 expref: -99 pid: 6256 timeout: 0 lvb_type: 0 00010000:00010000:2.0:1776178213.330203:0:3339:0:(ldlm_lock.c:820:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(COS) ns: mdt-lustre-MDT0000_UUID lock: 00000000a3795b7d/0xa40083ab9bf53915 lrc: 3/0,1 mode: COS/COS res: [0x2000013a1:0xd:0x0].0x0 bits 0x2/0x0 rrc: 3 type: IBT gid 0 flags: 0x50210000000000 nid: local remote: 0x0 expref: -99 pid: 6257 timeout: 0 lvb_type: 0 00010000:00010000:2.0:1776178213.330209:0:3339:0:(ldlm_lock.c:879:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: mdt-lustre-MDT0000_UUID lock: 00000000a3795b7d/0xa40083ab9bf53915 lrc: 2/0,0 mode: COS/COS res: [0x2000013a1:0xd:0x0].0x0 bits 0x2/0x0 rrc: 3 type: IBT gid 0 flags: 0x50210400000000 nid: local remote: 0x0 expref: -99 pid: 6257 timeout: 0 lvb_type: 0 00010000:00010000:2.0:1776178213.330214:0:3339:0:(ldlm_lockd.c:1879:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: mdt-lustre-MDT0000_UUID lock: 00000000a3795b7d/0xa40083ab9bf53915 lrc: 3/0,0 mode: COS/COS res: [0x2000013a1:0xd:0x0].0x0 bits 0x2/0x0 rrc: 3 type: IBT gid 0 flags: 0x40210400000000 nid: local remote: 0x0 expref: -99 pid: 6257 timeout: 0 lvb_type: 0 00010000:00010000:2.0:1776178213.330219:0:3339:0:(ldlm_lockd.c:1893:ldlm_handle_bl_callback()) Lock 00000000a3795b7d already unused, calling callback (0000000079e78348) 00010000:00010000:2.0:1776178213.330221:0:3339:0:(ldlm_request.c:364:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: mdt-lustre-MDT0000_UUID lock: 00000000a3795b7d/0xa40083ab9bf53915 lrc: 3/0,0 mode: COS/COS res: [0x2000013a1:0xd:0x0].0x0 bits 0x2/0x0 rrc: 3 type: IBT gid 0 flags: 0x40210400000000 nid: local remote: 0x0 expref: -99 pid: 6257 timeout: 0 lvb_type: 0 00010000:00010000:2.0:1776178213.330226:0:3339:0:(ldlm_request.c:1257:ldlm_cli_cancel_local()) ### server-side local cancel ns: mdt-lustre-MDT0000_UUID lock: 00000000a3795b7d/0xa40083ab9bf53915 lrc: 4/0,0 mode: COS/COS res: [0x2000013a1:0xd:0x0].0x0 bits 0x2/0x0 rrc: 3 type: IBT gid 0 flags: 0x40218400000000 nid: local remote: 0x0 expref: -99 pid: 6257 timeout: 0 lvb_type: 0 00010000:00010000:2.0:1776178213.330234:0:3339:0:(ldlm_inodebits.c:94:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x2000013a1:0xd:0x0].0x0 (00000000713d9096) 00010000:00010000:2.0:1776178213.330236:0:3339:0:(ldlm_inodebits.c:96:ldlm_reprocess_inodebits_queue()) Hint 2 00010000:00010000:2.0:1776178213.330237:0:3339:0:(ldlm_lockd.c:1903:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: mdt-lustre-MDT0000_UUID lock: 00000000a3795b7d/0xa40083ab9bf53915 lrc: 2/0,0 mode: --/COS res: [0x2000013a1:0xd:0x0].0x0 bits 0x2/0x0 rrc: 3 type: IBT gid 0 flags: 0x44a19400000000 nid: local remote: 0x0 expref: -99 pid: 6257 timeout: 0 lvb_type: 0 00010000:00010000:2.0:1776178213.330242:0:3339:0:(ldlm_lock.c:216:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: mdt-lustre-MDT0000_UUID lock: 00000000a3795b7d/0xa40083ab9bf53915 lrc: 0/0,0 mode: --/COS res: [0x2000013a1:0xd:0x0].0x0 bits 0x2/0x0 rrc: 3 type: IBT gid 0 flags: 0x44a19400000000 nid: local remote: 0x0 expref: -99 pid: 6257 timeout: 0 lvb_type: 0 00010000:00010000:2.0:1776178213.330253:0:3339:0:(ldlm_lock.c:820:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(COS) ns: mdt-lustre-MDT0000_UUID lock: 00000000a44d8583/0xa40083ab9bf5392a lrc: 3/0,1 mode: COS/COS res: [0x2000013a1:0x9:0x0].0xef09e6ea bits 0x2/0x0 rrc: 2 type: IBT gid 0 flags: 0x50210000000000 nid: local remote: 0x0 expref: -99 pid: 6256 timeout: 0 lvb_type: 0 00010000:00010000:2.0:1776178213.330259:0:3339:0:(ldlm_lock.c:879:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: mdt-lustre-MDT0000_UUID lock: 00000000a44d8583/0xa40083ab9bf5392a lrc: 2/0,0 mode: COS/COS res: [0x2000013a1:0x9:0x0].0xef09e6ea bits 0x2/0x0 rrc: 2 type: IBT gid 0 flags: 0x50210400000000 nid: local remote: 0x0 expref: -99 pid: 6256 timeout: 0 lvb_type: 0 00010000:00010000:2.0:1776178213.330264:0:3339:0:(ldlm_lockd.c:1879:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: mdt-lustre-MDT0000_UUID lock: 00000000a44d8583/0xa40083ab9bf5392a lrc: 3/0,0 mode: COS/COS res: [0x2000013a1:0x9:0x0].0xef09e6ea bits 0x2/0x0 rrc: 2 type: IBT gid 0 flags: 0x40210400000000 nid: local remote: 0x0 expref: -99 pid: 6256 timeout: 0 lvb_type: 0 00010000:00010000:2.0:1776178213.330268:0:3339:0:(ldlm_lockd.c:1893:ldlm_handle_bl_callback()) Lock 00000000a44d8583 already unused, calling callback (0000000079e78348) 00010000:00010000:2.0:1776178213.330270:0:3339:0:(ldlm_request.c:364:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: mdt-lustre-MDT0000_UUID lock: 00000000a44d8583/0xa40083ab9bf5392a lrc: 3/0,0 mode: COS/COS res: [0x2000013a1:0x9:0x0].0xef09e6ea bits 0x2/0x0 rrc: 2 type: IBT gid 0 flags: 0x40210400000000 nid: local remote: 0x0 expref: -99 pid: 6256 timeout: 0 lvb_type: 0 00010000:00010000:2.0:1776178213.330275:0:3339:0:(ldlm_request.c:1257:ldlm_cli_cancel_local()) ### server-side local cancel ns: mdt-lustre-MDT0000_UUID lock: 00000000a44d8583/0xa40083ab9bf5392a lrc: 4/0,0 mode: COS/COS res: [0x2000013a1:0x9:0x0].0xef09e6ea bits 0x2/0x0 rrc: 2 type: IBT gid 0 flags: 0x40218400000000 nid: local remote: 0x0 expref: -99 pid: 6256 timeout: 0 lvb_type: 0 00010000:00010000:2.0:1776178213.330281:0:3339:0:(ldlm_inodebits.c:94:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x2000013a1:0x9:0x0].0xef09e6ea (000000009742f5ea) 00010000:00010000:2.0:1776178213.330284:0:3339:0:(ldlm_inodebits.c:96:ldlm_reprocess_inodebits_queue()) Hint 2 00010000:00010000:2.0:1776178213.330285:0:3339:0:(ldlm_lockd.c:1903:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: mdt-lustre-MDT0000_UUID lock: 00000000a44d8583/0xa40083ab9bf5392a lrc: 2/0,0 mode: --/COS res: [0x2000013a1:0x9:0x0].0xef09e6ea bits 0x2/0x0 rrc: 2 type: IBT gid 0 flags: 0x44a19400000000 nid: local remote: 0x0 expref: -99 pid: 6256 timeout: 0 lvb_type: 0 00010000:00010000:2.0:1776178213.330295:0:3339:0:(ldlm_lock.c:216:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: mdt-lustre-MDT0000_UUID lock: 00000000a44d8583/0xa40083ab9bf5392a lrc: 0/0,0 mode: --/COS res: [0x2000013a1:0x9:0x0].0xef09e6ea bits 0x2/0x0 rrc: 2 type: IBT gid 0 flags: 0x44a19400000000 nid: local remote: 0x0 expref: -99 pid: 6256 timeout: 0 lvb_type: 0 00010000:00010000:2.0:1776178213.330308:0:3339:0:(ldlm_lock.c:820:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(COS) ns: mdt-lustre-MDT0000_UUID lock: 000000000b5c0143/0xa40083ab9bf5393f lrc: 3/0,1 mode: COS/COS res: [0x2000013a1:0xe:0x0].0x0 bits 0x2/0x0 rrc: 3 type: IBT gid 0 flags: 0x50210000000000 nid: local remote: 0x0 expref: -99 pid: 6256 timeout: 0 lvb_type: 0 00010000:00010000:2.0:1776178213.330313:0:3339:0:(ldlm_lock.c:879:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: mdt-lustre-MDT0000_UUID lock: 000000000b5c0143/0xa40083ab9bf5393f lrc: 2/0,0 mode: COS/COS res: [0x2000013a1:0xe:0x0].0x0 bits 0x2/0x0 rrc: 3 type: IBT gid 0 flags: 0x50210400000000 nid: local remote: 0x0 expref: -99 pid: 6256 timeout: 0 lvb_type: 0 00010000:00010000:2.0:1776178213.330319:0:3339:0:(ldlm_lockd.c:1879:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: mdt-lustre-MDT0000_UUID lock: 000000000b5c0143/0xa40083ab9bf5393f lrc: 3/0,0 mode: COS/COS res: [0x2000013a1:0xe:0x0].0x0 bits 0x2/0x0 rrc: 3 type: IBT gid 0 flags: 0x40210400000000 nid: local remote: 0x0 expref: -99 pid: 6256 timeout: 0 lvb_type: 0 00010000:00010000:2.0:1776178213.330324:0:3339:0:(ldlm_lockd.c:1893:ldlm_handle_bl_callback()) Lock 000000000b5c0143 already unused, calling callback (0000000079e78348) 00010000:00010000:2.0:1776178213.330326:0:3339:0:(ldlm_request.c:364:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: mdt-lustre-MDT0000_UUID lock: 000000000b5c0143/0xa40083ab9bf5393f lrc: 3/0,0 mode: COS/COS res: [0x2000013a1:0xe:0x0].0x0 bits 0x2/0x0 rrc: 3 type: IBT gid 0 flags: 0x40210400000000 nid: local remote: 0x0 expref: -99 pid: 6256 timeout: 0 lvb_type: 0 00010000:00010000:2.0:1776178213.330331:0:3339:0:(ldlm_request.c:1257:ldlm_cli_cancel_local()) ### server-side local cancel ns: mdt-lustre-MDT0000_UUID lock: 000000000b5c0143/0xa40083ab9bf5393f lrc: 4/0,0 mode: COS/COS res: [0x2000013a1:0xe:0x0].0x0 bits 0x2/0x0 rrc: 3 type: IBT gid 0 flags: 0x40218400000000 nid: local remote: 0x0 expref: -99 pid: 6256 timeout: 0 lvb_type: 0 00010000:00010000:2.0:1776178213.330337:0:3339:0:(ldlm_inodebits.c:94:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x2000013a1:0xe:0x0].0x0 (000000008c340be4) 00010000:00010000:2.0:1776178213.330339:0:3339:0:(ldlm_inodebits.c:96:ldlm_reprocess_inodebits_queue()) Hint 2 00010000:00010000:2.0:1776178213.330342:0:3339:0:(ldlm_lockd.c:1903:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: mdt-lustre-MDT0000_UUID lock: 000000000b5c0143/0xa40083ab9bf5393f lrc: 2/0,0 mode: --/COS res: [0x2000013a1:0xe:0x0].0x0 bits 0x2/0x0 rrc: 3 type: IBT gid 0 flags: 0x44a19400000000 nid: local remote: 0x0 expref: -99 pid: 6256 timeout: 0 lvb_type: 0 00010000:00010000:2.0:1776178213.330346:0:3339:0:(ldlm_lock.c:216:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: mdt-lustre-MDT0000_UUID lock: 000000000b5c0143/0xa40083ab9bf5393f lrc: 0/0,0 mode: --/COS res: [0x2000013a1:0xe:0x0].0x0 bits 0x2/0x0 rrc: 3 type: IBT gid 0 flags: 0x44a19400000000 nid: local remote: 0x0 expref: -99 pid: 6256 timeout: 0 lvb_type: 0 00010000:00010000:2.0:1776178213.330722:0:3339:0:(ldlm_lock.c:820:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(COS) ns: mdt-lustre-MDT0000_UUID lock: 00000000f90c1e39/0xa40083ab9bf53954 lrc: 3/0,1 mode: COS/COS res: [0x2000013a1:0x9:0x0].0x37639fea bits 0x2/0x0 rrc: 2 type: IBT gid 0 flags: 0x50210000000000 nid: local remote: 0x0 expref: -99 pid: 6257 timeout: 0 lvb_type: 0 00010000:00010000:2.0:1776178213.330734:0:3339:0:(ldlm_lock.c:879:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: mdt-lustre-MDT0000_UUID lock: 00000000f90c1e39/0xa40083ab9bf53954 lrc: 2/0,0 mode: COS/COS res: [0x2000013a1:0x9:0x0].0x37639fea bits 0x2/0x0 rrc: 2 type: IBT gid 0 flags: 0x50210400000000 nid: local remote: 0x0 expref: -99 pid: 6257 timeout: 0 lvb_type: 0 00010000:00010000:2.0:1776178213.330740:0:3339:0:(ldlm_lockd.c:1879:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: mdt-lustre-MDT0000_UUID lock: 00000000f90c1e39/0xa40083ab9bf53954 lrc: 3/0,0 mode: COS/COS res: [0x2000013a1:0x9:0x0].0x37639fea bits 0x2/0x0 rrc: 2 type: IBT gid 0 flags: 0x40210400000000 nid: local remote: 0x0 expref: -99 pid: 6257 timeout: 0 lvb_type: 0 00010000:00010000:2.0:1776178213.330745:0:3339:0:(ldlm_lockd.c:1893:ldlm_handle_bl_callback()) Lock 00000000f90c1e39 already unused, calling callback (0000000079e78348) 00010000:00010000:2.0:1776178213.330747:0:3339:0:(ldlm_request.c:364:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: mdt-lustre-MDT0000_UUID lock: 00000000f90c1e39/0xa40083ab9bf53954 lrc: 3/0,0 mode: COS/COS res: [0x2000013a1:0x9:0x0].0x37639fea bits 0x2/0x0 rrc: 2 type: IBT gid 0 flags: 0x40210400000000 nid: local remote: 0x0 expref: -99 pid: 6257 timeout: 0 lvb_type: 0 00010000:00010000:2.0:1776178213.330753:0:3339:0:(ldlm_request.c:1257:ldlm_cli_cancel_local()) ### server-side local cancel ns: mdt-lustre-MDT0000_UUID lock: 00000000f90c1e39/0xa40083ab9bf53954 lrc: 4/0,0 mode: COS/COS res: [0x2000013a1:0x9:0x0].0x37639fea bits 0x2/0x0 rrc: 2 type: IBT gid 0 flags: 0x40218400000000 nid: local remote: 0x0 expref: -99 pid: 6257 timeout: 0 lvb_type: 0 00010000:00010000:2.0:1776178213.330762:0:3339:0:(ldlm_inodebits.c:94:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x2000013a1:0x9:0x0].0x37639fea (000000004548771b) 00010000:00010000:2.0:1776178213.330764:0:3339:0:(ldlm_inodebits.c:96:ldlm_reprocess_inodebits_queue()) Hint 2 00010000:00010000:2.0:1776178213.330767:0:3339:0:(ldlm_lockd.c:1903:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: mdt-lustre-MDT0000_UUID lock: 00000000f90c1e39/0xa40083ab9bf53954 lrc: 2/0,0 mode: --/COS res: [0x2000013a1:0x9:0x0].0x37639fea bits 0x2/0x0 rrc: 2 type: IBT gid 0 flags: 0x44a19400000000 nid: local remote: 0x0 expref: -99 pid: 6257 timeout: 0 lvb_type: 0 00010000:00010000:2.0:1776178213.330772:0:3339:0:(ldlm_lock.c:216:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: mdt-lustre-MDT0000_UUID lock: 00000000f90c1e39/0xa40083ab9bf53954 lrc: 0/0,0 mode: --/COS res: [0x2000013a1:0x9:0x0].0x37639fea bits 0x2/0x0 rrc: 2 type: IBT gid 0 flags: 0x44a19400000000 nid: local remote: 0x0 expref: -99 pid: 6257 timeout: 0 lvb_type: 0 00010000:00010000:2.0:1776178213.330785:0:3339:0:(ldlm_lock.c:820:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(COS) ns: mdt-lustre-MDT0000_UUID lock: 00000000a27c19b3/0xa40083ab9bf53969 lrc: 3/0,1 mode: COS/COS res: [0x2000013a1:0xf:0x0].0x0 bits 0x2/0x0 rrc: 3 type: IBT gid 0 flags: 0x50210000000000 nid: local remote: 0x0 expref: -99 pid: 6257 timeout: 0 lvb_type: 0 00010000:00010000:2.0:1776178213.330790:0:3339:0:(ldlm_lock.c:879:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: mdt-lustre-MDT0000_UUID lock: 00000000a27c19b3/0xa40083ab9bf53969 lrc: 2/0,0 mode: COS/COS res: [0x2000013a1:0xf:0x0].0x0 bits 0x2/0x0 rrc: 3 type: IBT gid 0 flags: 0x50210400000000 nid: local remote: 0x0 expref: -99 pid: 6257 timeout: 0 lvb_type: 0 00010000:00010000:2.0:1776178213.330796:0:3339:0:(ldlm_lockd.c:1879:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: mdt-lustre-MDT0000_UUID lock: 00000000a27c19b3/0xa40083ab9bf53969 lrc: 3/0,0 mode: COS/COS res: [0x2000013a1:0xf:0x0].0x0 bits 0x2/0x0 rrc: 3 type: IBT gid 0 flags: 0x40210400000000 nid: local remote: 0x0 expref: -99 pid: 6257 timeout: 0 lvb_type: 0 00010000:00010000:2.0:1776178213.330879:0:3339:0:(ldlm_lockd.c:1893:ldlm_handle_bl_callback()) Lock 00000000a27c19b3 already unused, calling callback (0000000079e78348) 00010000:00010000:2.0:1776178213.330882:0:3339:0:(ldlm_request.c:364:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: mdt-lustre-MDT0000_UUID lock: 00000000a27c19b3/0xa40083ab9bf53969 lrc: 3/0,0 mode: COS/COS res: [0x2000013a1:0xf:0x0].0x0 bits 0x2/0x0 rrc: 3 type: IBT gid 0 flags: 0x40210400000000 nid: local remote: 0x0 expref: -99 pid: 6257 timeout: 0 lvb_type: 0 00010000:00010000:2.0:1776178213.330888:0:3339:0:(ldlm_request.c:1257:ldlm_cli_cancel_local()) ### server-side local cancel ns: mdt-lustre-MDT0000_UUID lock: 00000000a27c19b3/0xa40083ab9bf53969 lrc: 4/0,0 mode: COS/COS res: [0x2000013a1:0xf:0x0].0x0 bits 0x2/0x0 rrc: 3 type: IBT gid 0 flags: 0x40218400000000 nid: local remote: 0x0 expref: -99 pid: 6257 timeout: 0 lvb_type: 0 00010000:00010000:2.0:1776178213.330896:0:3339:0:(ldlm_inodebits.c:94:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x2000013a1:0xf:0x0].0x0 (00000000a5b77ef7) 00010000:00010000:2.0:1776178213.330898:0:3339:0:(ldlm_inodebits.c:96:ldlm_reprocess_inodebits_queue()) Hint 2 00010000:00010000:2.0:1776178213.330900:0:3339:0:(ldlm_lockd.c:1903:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: mdt-lustre-MDT0000_UUID lock: 00000000a27c19b3/0xa40083ab9bf53969 lrc: 2/0,0 mode: --/COS res: [0x2000013a1:0xf:0x0].0x0 bits 0x2/0x0 rrc: 3 type: IBT gid 0 flags: 0x44a19400000000 nid: local remote: 0x0 expref: -99 pid: 6257 timeout: 0 lvb_type: 0 00010000:00010000:2.0:1776178213.330905:0:3339:0:(ldlm_lock.c:216:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: mdt-lustre-MDT0000_UUID lock: 00000000a27c19b3/0xa40083ab9bf53969 lrc: 0/0,0 mode: --/COS res: [0x2000013a1:0xf:0x0].0x0 bits 0x2/0x0 rrc: 3 type: IBT gid 0 flags: 0x44a19400000000 nid: local remote: 0x0 expref: -99 pid: 6257 timeout: 0 lvb_type: 0 00010000:00010000:2.0:1776178213.330916:0:3339:0:(ldlm_lock.c:820:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(COS) ns: mdt-lustre-MDT0000_UUID lock: 00000000e4217a5c/0xa40083ab9bf5397e lrc: 3/0,1 mode: COS/COS res: [0x2000013a1:0x9:0x0].0x7fbd58ea bits 0x2/0x0 rrc: 2 type: IBT gid 0 flags: 0x50210000000000 nid: local remote: 0x0 expref: -99 pid: 6257 timeout: 0 lvb_type: 0 00010000:00010000:2.0:1776178213.330922:0:3339:0:(ldlm_lock.c:879:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: mdt-lustre-MDT0000_UUID lock: 00000000e4217a5c/0xa40083ab9bf5397e lrc: 2/0,0 mode: COS/COS res: [0x2000013a1:0x9:0x0].0x7fbd58ea bits 0x2/0x0 rrc: 2 type: IBT gid 0 flags: 0x50210400000000 nid: local remote: 0x0 expref: -99 pid: 6257 timeout: 0 lvb_type: 0 00010000:00010000:2.0:1776178213.330928:0:3339:0:(ldlm_lockd.c:1879:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: mdt-lustre-MDT0000_UUID lock: 00000000e4217a5c/0xa40083ab9bf5397e lrc: 3/0,0 mode: COS/COS res: [0x2000013a1:0x9:0x0].0x7fbd58ea bits 0x2/0x0 rrc: 2 type: IBT gid 0 flags: 0x40210400000000 nid: local remote: 0x0 expref: -99 pid: 6257 timeout: 0 lvb_type: 0 00010000:00010000:2.0:1776178213.330933:0:3339:0:(ldlm_lockd.c:1893:ldlm_handle_bl_callback()) Lock 00000000e4217a5c already unused, calling callback (0000000079e78348) 00010000:00010000:2.0:1776178213.330934:0:3339:0:(ldlm_request.c:364:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: mdt-lustre-MDT0000_UUID lock: 00000000e4217a5c/0xa40083ab9bf5397e lrc: 3/0,0 mode: COS/COS res: [0x2000013a1:0x9:0x0].0x7fbd58ea bits 0x2/0x0 rrc: 2 type: IBT gid 0 flags: 0x40210400000000 nid: local remote: 0x0 expref: -99 pid: 6257 timeout: 0 lvb_type: 0 00010000:00010000:2.0:1776178213.330939:0:3339:0:(ldlm_request.c:1257:ldlm_cli_cancel_local()) ### server-side local cancel ns: mdt-lustre-MDT0000_UUID lock: 00000000e4217a5c/0xa40083ab9bf5397e lrc: 4/0,0 mode: COS/COS res: [0x2000013a1:0x9:0x0].0x7fbd58ea bits 0x2/0x0 rrc: 2 type: IBT gid 0 flags: 0x40218400000000 nid: local remote: 0x0 expref: -99 pid: 6257 timeout: 0 lvb_type: 0 00010000:00010000:2.0:1776178213.330947:0:3339:0:(ldlm_inodebits.c:94:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x2000013a1:0x9:0x0].0x7fbd58ea (00000000ccda0888) 00010000:00010000:2.0:1776178213.330949:0:3339:0:(ldlm_inodebits.c:96:ldlm_reprocess_inodebits_queue()) Hint 2 00010000:00010000:2.0:1776178213.330951:0:3339:0:(ldlm_lockd.c:1903:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: mdt-lustre-MDT0000_UUID lock: 00000000e4217a5c/0xa40083ab9bf5397e lrc: 2/0,0 mode: --/COS res: [0x2000013a1:0x9:0x0].0x7fbd58ea bits 0x2/0x0 rrc: 2 type: IBT gid 0 flags: 0x44a19400000000 nid: local remote: 0x0 expref: -99 pid: 6257 timeout: 0 lvb_type: 0 00010000:00010000:2.0:1776178213.330956:0:3339:0:(ldlm_lock.c:216:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: mdt-lustre-MDT0000_UUID lock: 00000000e4217a5c/0xa40083ab9bf5397e lrc: 0/0,0 mode: --/COS res: [0x2000013a1:0x9:0x0].0x7fbd58ea bits 0x2/0x0 rrc: 2 type: IBT gid 0 flags: 0x44a19400000000 nid: local remote: 0x0 expref: -99 pid: 6257 timeout: 0 lvb_type: 0 00010000:00010000:2.0:1776178213.330969:0:3339:0:(ldlm_lock.c:820:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(COS) ns: mdt-lustre-MDT0000_UUID lock: 000000002f1f7bd5/0xa40083ab9bf53993 lrc: 3/0,1 mode: COS/COS res: [0x2000013a1:0x10:0x0].0x0 bits 0x2/0x0 rrc: 3 type: IBT gid 0 flags: 0x50210000000000 nid: local remote: 0x0 expref: -99 pid: 6257 timeout: 0 lvb_type: 0 00010000:00010000:2.0:1776178213.330974:0:3339:0:(ldlm_lock.c:879:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: mdt-lustre-MDT0000_UUID lock: 000000002f1f7bd5/0xa40083ab9bf53993 lrc: 2/0,0 mode: COS/COS res: [0x2000013a1:0x10:0x0].0x0 bits 0x2/0x0 rrc: 3 type: IBT gid 0 flags: 0x50210400000000 nid: local remote: 0x0 expref: -99 pid: 6257 timeout: 0 lvb_type: 0 00010000:00010000:2.0:1776178213.330978:0:3339:0:(ldlm_lockd.c:1879:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: mdt-lustre-MDT0000_UUID lock: 000000002f1f7bd5/0xa40083ab9bf53993 lrc: 3/0,0 mode: COS/COS res: [0x2000013a1:0x10:0x0].0x0 bits 0x2/0x0 rrc: 3 type: IBT gid 0 flags: 0x40210400000000 nid: local remote: 0x0 expref: -99 pid: 6257 timeout: 0 lvb_type: 0 00010000:00010000:2.0:1776178213.330983:0:3339:0:(ldlm_lockd.c:1893:ldlm_handle_bl_callback()) Lock 000000002f1f7bd5 already unused, calling callback (0000000079e78348) 00010000:00010000:2.0:1776178213.330984:0:3339:0:(ldlm_request.c:364:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: mdt-lustre-MDT0000_UUID lock: 000000002f1f7bd5/0xa40083ab9bf53993 lrc: 3/0,0 mode: COS/COS res: [0x2000013a1:0x10:0x0].0x0 bits 0x2/0x0 rrc: 3 type: IBT gid 0 flags: 0x40210400000000 nid: local remote: 0x0 expref: -99 pid: 6257 timeout: 0 lvb_type: 0 00010000:00010000:2.0:1776178213.330989:0:3339:0:(ldlm_request.c:1257:ldlm_cli_cancel_local()) ### server-side local cancel ns: mdt-lustre-MDT0000_UUID lock: 000000002f1f7bd5/0xa40083ab9bf53993 lrc: 4/0,0 mode: COS/COS res: [0x2000013a1:0x10:0x0].0x0 bits 0x2/0x0 rrc: 3 type: IBT gid 0 flags: 0x40218400000000 nid: local remote: 0x0 expref: -99 pid: 6257 timeout: 0 lvb_type: 0 00010000:00010000:2.0:1776178213.330996:0:3339:0:(ldlm_inodebits.c:94:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x2000013a1:0x10:0x0].0x0 (000000003da5851e) 00010000:00010000:2.0:1776178213.330998:0:3339:0:(ldlm_inodebits.c:96:ldlm_reprocess_inodebits_queue()) Hint 2 00010000:00010000:2.0:1776178213.330999:0:3339:0:(ldlm_lockd.c:1903:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: mdt-lustre-MDT0000_UUID lock: 000000002f1f7bd5/0xa40083ab9bf53993 lrc: 2/0,0 mode: --/COS res: [0x2000013a1:0x10:0x0].0x0 bits 0x2/0x0 rrc: 3 type: IBT gid 0 flags: 0x44a19400000000 nid: local remote: 0x0 expref: -99 pid: 6257 timeout: 0 lvb_type: 0 00010000:00010000:2.0:1776178213.331007:0:3339:0:(ldlm_lock.c:216:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: mdt-lustre-MDT0000_UUID lock: 000000002f1f7bd5/0xa40083ab9bf53993 lrc: 0/0,0 mode: --/COS res: [0x2000013a1:0x10:0x0].0x0 bits 0x2/0x0 rrc: 3 type: IBT gid 0 flags: 0x44a19400000000 nid: local remote: 0x0 expref: -99 pid: 6257 timeout: 0 lvb_type: 0 00000001:00080000:2.0:1776178213.332702:0:9217:0:(tgt_lastrcvd.c:913:tgt_cb_last_committed()) lustre-OST0001: transno 4294969339 is committed 00000001:00080000:2.0:1776178213.332732:0:9217:0:(tgt_lastrcvd.c:913:tgt_cb_last_committed()) lustre-OST0001: transno 4294969335 is committed 00010000:00010000:2.0:1776178213.345642:0:9225:0:(ldlm_lockd.c:2792:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:3.0:1776178213.455768:0:3348:0:(events.c:361:request_in_callback()) peer: 12345-192.168.204.3@tcp (source: 12345-192.168.204.3@tcp) 00000100:00100000:1.0:1776178213.455922:0:6256:0:(service.c:2152:ptlrpc_server_handle_req_in()) got req x1862456979107200 00000100:00100000:1.0:1776178213.455989:0:6256:0:(nrs_fifo.c:179:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.204.3@tcp, seq: 6616 00000100:00100000:1.0:1776178213.455997:0:6256:0:(service.c:2324:ptlrpc_server_handle_request()) Handling RPC req@000000008b793df7 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_001:b50963bd-8eba-4432-9a54-8888d00e3fa1+14:18273:x1862456979107200:12345-192.168.204.3@tcp:101:lfs.0 00010000:00010000:1.0:1776178213.456024:0:6256:0:(ldlm_lockd.c:1248:ldlm_handle_enqueue0()) ### server-side enqueue handler START 00010000:00010000:1.0:1776178213.456049:0:6256:0:(ldlm_lockd.c:1318:ldlm_handle_enqueue0()) ### server-side enqueue handler, new lock created ns: mdt-lustre-MDT0000_UUID lock: 00000000499bab84/0xa40083ab9bf5399a lrc: 2/0,0 mode: --/CR res: [0x200000007:0x1:0x0].0x0 bits 0x0/0x0 rrc: 4 type: IBT gid 0 flags: 0x40000000000000 nid: local remote: 0x771f68aa1eb27c14 expref: -99 pid: 6256 timeout: 0 lvb_type: 0 00010000:00010000:1.0:1776178213.456150:0:6256:0:(ldlm_lock.c:766:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PR) ns: mdt-lustre-MDT0000_UUID lock: 00000000d0f9e56c/0xa40083ab9bf539a1 lrc: 3/1,0 mode: --/PR res: [0x200000007:0x1:0x0].0x0 bits 0x0/0x0 rrc: 5 type: IBT gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 6256 timeout: 0 lvb_type: 0 00010000:00010000:1.0:1776178213.456167:0:6256:0:(ldlm_lock.c:1064:ldlm_granted_list_add_lock()) ### About to add lock: ns: mdt-lustre-MDT0000_UUID lock: 00000000d0f9e56c/0xa40083ab9bf539a1 lrc: 3/1,0 mode: PR/PR res: [0x200000007:0x1:0x0].0x0 bits 0x13/0x0 rrc: 5 type: IBT gid 0 flags: 0x50210000000000 nid: local remote: 0x0 expref: -99 pid: 6256 timeout: 0 lvb_type: 0 00010000:00010000:1.0:1776178213.456180:0:6256:0:(ldlm_request.c:528:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: mdt-lustre-MDT0000_UUID lock: 00000000d0f9e56c/0xa40083ab9bf539a1 lrc: 3/1,0 mode: PR/PR res: [0x200000007:0x1:0x0].0x0 bits 0x13/0x0 rrc: 5 type: IBT gid 0 flags: 0x40210000000000 nid: local remote: 0x0 expref: -99 pid: 6256 timeout: 0 lvb_type: 0 00000004:00200000:1.0:1776178213.456235:0:6256:0:(mdt_handler.c:912:mdt_pack_attr2body()) [0x200000007:0x1:0x0]: returning size 4096 00000004:00200000:1.0:1776178213.456239:0:6256:0:(mdt_handler.c:1523:mdt_getattr_internal()) dirent count 3 stripe count 1 MDT count 2 00000004:00010000:1.0:1776178213.456262:0:6256:0:(mdt_handler.c:2353:mdt_getattr_name_lock()) ### Returning lock to client ns: mdt-lustre-MDT0000_UUID lock: 00000000d0f9e56c/0xa40083ab9bf539a1 lrc: 3/1,0 mode: PR/PR res: [0x200000007:0x1:0x0].0x0 bits 0x13/0x0 rrc: 5 type: IBT gid 0 flags: 0x40210000000000 nid: local remote: 0x0 expref: -99 pid: 6256 timeout: 0 lvb_type: 0 00010000:00010000:1.0:1776178213.456292:0:6256:0:(ldlm_lock.c:216:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: mdt-lustre-MDT0000_UUID lock: 00000000499bab84/0xa40083ab9bf5399a lrc: 0/0,0 mode: --/CR res: [0x200000007:0x1:0x0].0x0 bits 0x2/0x0 rrc: 5 type: IBT gid 0 flags: 0x44000000000000 nid: 192.168.204.3@tcp remote: 0x771f68aa1eb27c14 expref: 16 pid: 6256 timeout: 0 lvb_type: 0 00010000:00010000:1.0:1776178213.456311:0:6256:0:(ldlm_lockd.c:1484:ldlm_handle_enqueue0()) ### server-side enqueue handler, sending reply (err=0, rc=0) ns: mdt-lustre-MDT0000_UUID lock: 00000000d0f9e56c/0xa40083ab9bf539a1 lrc: 3/0,0 mode: PR/PR res: [0x200000007:0x1:0x0].0x0 bits 0x13/0x0 rrc: 4 type: IBT gid 0 flags: 0x40200000000000 nid: 192.168.204.3@tcp remote: 0x771f68aa1eb27c14 expref: 15 pid: 6256 timeout: 0 lvb_type: 0 00010000:00010000:1.0:1776178213.456319:0:6256:0:(ldlm_inodebits.c:94:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x200000007:0x1:0x0].0x0 (00000000ad0d4094) 00010000:00010000:1.0:1776178213.456321:0:6256:0:(ldlm_inodebits.c:96:ldlm_reprocess_inodebits_queue()) Hint 13 00010000:00010000:1.0:1776178213.456325:0:6256:0:(ldlm_lockd.c:1565:ldlm_handle_enqueue0()) ### server-side enqueue handler END (lock 00000000d0f9e56c, rc 0) 00000100:00100000:1.0:1776178213.456387:0:6256:0:(service.c:2372:ptlrpc_server_handle_request()) Handled RPC req@000000008b793df7 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_001:b50963bd-8eba-4432-9a54-8888d00e3fa1+15:18273:x1862456979107200:12345-192.168.204.3@tcp:101:lfs.0 Request processed in 389us (706us total) trans 0 rc 0/0 00000100:00100000:1.0:1776178213.456395:0:6256:0:(nrs_fifo.c:241:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.204.3@tcp, seq: 6616 00000100:00100000:0.0:1776178213.467618:0:3346:0:(events.c:361:request_in_callback()) peer: 12345-192.168.204.3@tcp (source: 12345-192.168.204.3@tcp) 00000100:00100000:2.0:1776178213.467722:0:6257:0:(service.c:2152:ptlrpc_server_handle_req_in()) got req x1862456979107264 00000100:00100000:2.0:1776178213.467791:0:6257:0:(nrs_fifo.c:179:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.204.3@tcp, seq: 6617 00000100:00100000:2.0:1776178213.467799:0:6257:0:(service.c:2324:ptlrpc_server_handle_request()) Handling RPC req@00000000ddc1dc02 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_002:b50963bd-8eba-4432-9a54-8888d00e3fa1+15:18273:x1862456979107264:12345-192.168.204.3@tcp:41:lfs.0 00000004:00000024:2.0:1776178213.467850:0:6257:0:(mdt_handler.c:548:mdt_statfs()) blocks cached 0 granted 2097152 pending 0 free 1446121472 avail 1316659200 00000100:00100000:2.0:1776178213.467943:0:6257:0:(service.c:2372:ptlrpc_server_handle_request()) Handled RPC req@00000000ddc1dc02 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_002:b50963bd-8eba-4432-9a54-8888d00e3fa1+15:18273:x1862456979107264:12345-192.168.204.3@tcp:41:lfs.0 Request processed in 143us (340us total) trans 0 rc 0/0 00000100:00100000:2.0:1776178213.467952:0:6257:0:(nrs_fifo.c:241:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.204.3@tcp, seq: 6617 00000100:00100000:3.0:1776178213.469977:0:3347:0:(events.c:361:request_in_callback()) peer: 12345-192.168.204.3@tcp (source: 12345-192.168.204.3@tcp) 00000100:00100000:0.0:1776178213.470117:0:6256:0:(service.c:2152:ptlrpc_server_handle_req_in()) got req x1862456979107328 00000100:00100000:0.0:1776178213.470164:0:6256:0:(nrs_fifo.c:179:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.204.3@tcp, seq: 6618 00000100:00100000:0.0:1776178213.470171:0:6256:0:(service.c:2324:ptlrpc_server_handle_request()) Handling RPC req@00000000dc1ea8e7 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_001:b50963bd-8eba-4432-9a54-8888d00e3fa1+6:18273:x1862456979107328:12345-192.168.204.3@tcp:41:lfs.0 00000004:00000024:0.0:1776178213.470216:0:6256:0:(mdt_handler.c:548:mdt_statfs()) blocks cached 0 granted 2097152 pending 0 free 1446039552 avail 1316577280 00000100:00100000:0.0:1776178213.470287:0:6256:0:(service.c:2372:ptlrpc_server_handle_request()) Handled RPC req@00000000dc1ea8e7 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_001:b50963bd-8eba-4432-9a54-8888d00e3fa1+6:18273:x1862456979107328:12345-192.168.204.3@tcp:41:lfs.0 Request processed in 115us (316us total) trans 0 rc 0/0 00000100:00100000:0.0:1776178213.470294:0:6256:0:(nrs_fifo.c:241:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.204.3@tcp, seq: 6618 00000100:00100000:2.0:1776178213.476685:0:3348:0:(events.c:361:request_in_callback()) peer: 12345-192.168.204.3@tcp (source: 12345-192.168.204.3@tcp) 00000100:00100000:0.0:1776178213.476930:0:9554:0:(service.c:2152:ptlrpc_server_handle_req_in()) got req x1862456979107392 00000100:00100000:0.0:1776178213.476982:0:9554:0:(nrs_fifo.c:179:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.204.3@tcp, seq: 487 00000100:00100000:0.0:1776178213.476989:0:9554:0:(service.c:2324:ptlrpc_server_handle_request()) Handling RPC req@000000000b5ad4fd pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_create00:b50963bd-8eba-4432-9a54-8888d00e3fa1+5:18273:x1862456979107392:12345-192.168.204.3@tcp:13:lfs.0 00002000:00000024:0.0:1776178213.477016:0:9554:0:(ofd_obd.c:740:ofd_statfs()) blocks cached 315392 granted 484883776 pending 0 free 3923689472 avail 3692167168 00000100:00100000:0.0:1776178213.477099:0:9554:0:(service.c:2372:ptlrpc_server_handle_request()) Handled RPC req@000000000b5ad4fd pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_create00:b50963bd-8eba-4432-9a54-8888d00e3fa1+5:18273:x1862456979107392:12345-192.168.204.3@tcp:13:lfs.0 Request processed in 109us (425us total) trans 0 rc 0/0 00000100:00100000:0.0:1776178213.477108:0:9554:0:(nrs_fifo.c:241:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.204.3@tcp, seq: 487 00000100:00100000:1.0:1776178213.481036:0:3346:0:(events.c:361:request_in_callback()) peer: 12345-192.168.204.3@tcp (source: 12345-192.168.204.3@tcp) 00000100:00100000:2.0:1776178213.481169:0:8232:0:(service.c:2152:ptlrpc_server_handle_req_in()) got req x1862456979107456 00000100:00100000:2.0:1776178213.481238:0:8232:0:(nrs_fifo.c:179:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.204.3@tcp, seq: 488 00000100:00100000:2.0:1776178213.481247:0:8232:0:(service.c:2324:ptlrpc_server_handle_request()) Handling RPC req@000000008aab9f17 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_create00:b50963bd-8eba-4432-9a54-8888d00e3fa1+5:18273:x1862456979107456:12345-192.168.204.3@tcp:13:lfs.0 00002000:00000024:2.0:1776178213.481278:0:8232:0:(ofd_obd.c:740:ofd_statfs()) blocks cached 229376 granted 491746368 pending 0 free 3923689472 avail 3692167168 00000100:00100000:2.0:1776178213.481498:0:8232:0:(service.c:2372:ptlrpc_server_handle_request()) Handled RPC req@000000008aab9f17 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_create00:b50963bd-8eba-4432-9a54-8888d00e3fa1+5:18273:x1862456979107456:12345-192.168.204.3@tcp:13:lfs.0 Request processed in 251us (473us total) trans 0 rc 0/0 00000100:00100000:2.0:1776178213.481505:0:8232:0:(nrs_fifo.c:241:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.204.3@tcp, seq: 488 00000100:00100000:3.0:1776178213.558440:0:3347:0:(events.c:361:request_in_callback()) peer: 12345-192.168.204.3@tcp (source: 12345-192.168.204.3@tcp) 00000100:00100000:2.0:1776178213.558764:0:6257:0:(service.c:2152:ptlrpc_server_handle_req_in()) got req x1862456979107584 00000100:00100000:2.0:1776178213.558830:0:6257:0:(nrs_fifo.c:179:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.204.3@tcp, seq: 6619 00000100:00100000:2.0:1776178213.558838:0:6257:0:(service.c:2324:ptlrpc_server_handle_request()) Handling RPC req@0000000068680f90 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_002:b50963bd-8eba-4432-9a54-8888d00e3fa1+15:18280:x1862456979107584:12345-192.168.204.3@tcp:36:mcreate.0 00010000:00010000:2.0:1776178213.558906:0:6257:0:(ldlm_lockd.c:1720:ldlm_request_cancel()) ### server-side cancel handler START: 1 locks, starting at 0 00010000:00010000:2.0:1776178213.558951:0:6257:0:(ldlm_lock.c:216:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: mdt-lustre-MDT0000_UUID lock: 00000000d0f9e56c/0xa40083ab9bf539a1 lrc: 0/0,0 mode: --/PR res: [0x200000007:0x1:0x0].0x0 bits 0x13/0x0 rrc: 5 type: IBT gid 0 flags: 0x44a01000000000 nid: 192.168.204.3@tcp remote: 0x771f68aa1eb27c14 expref: 15 pid: 6256 timeout: 0 lvb_type: 0 00010000:00010000:2.0:1776178213.558976:0:6257:0:(ldlm_inodebits.c:94:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x200000007:0x1:0x0].0x0 (00000000ad0d4094) 00010000:00010000:2.0:1776178213.558982:0:6257:0:(ldlm_lockd.c:1782:ldlm_request_cancel()) ### server-side cancel handler END 00010000:00010000:2.0:1776178213.559073:0:6257:0:(ldlm_lock.c:766:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(CW) ns: mdt-lustre-MDT0000_UUID lock: 000000002f1f7bd5/0xa40083ab9bf539a8 lrc: 3/0,1 mode: --/CW res: [0x200000007:0x1:0x0].0x0 bits 0x0/0x0 rrc: 4 type: IBT gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 6257 timeout: 0 lvb_type: 0 00010000:00010000:2.0:1776178213.559090:0:6257:0:(ldlm_lock.c:1064:ldlm_granted_list_add_lock()) ### About to add lock: ns: mdt-lustre-MDT0000_UUID lock: 000000002f1f7bd5/0xa40083ab9bf539a8 lrc: 3/0,1 mode: CW/CW res: [0x200000007:0x1:0x0].0x0 bits 0x2/0x0 rrc: 4 type: IBT gid 0 flags: 0x50210000000000 nid: local remote: 0x0 expref: -99 pid: 6257 timeout: 0 lvb_type: 0 00010000:00010000:2.0:1776178213.559101:0:6257:0:(ldlm_request.c:528:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: mdt-lustre-MDT0000_UUID lock: 000000002f1f7bd5/0xa40083ab9bf539a8 lrc: 3/0,1 mode: CW/CW res: [0x200000007:0x1:0x0].0x0 bits 0x2/0x0 rrc: 4 type: IBT gid 0 flags: 0x40210000000000 nid: local remote: 0x0 expref: -99 pid: 6257 timeout: 0 lvb_type: 0 00010000:00010000:2.0:1776178213.559116:0:6257:0:(ldlm_lock.c:766:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: mdt-lustre-MDT0000_UUID lock: 00000000e4217a5c/0xa40083ab9bf539af lrc: 3/0,1 mode: --/PW res: [0x200000007:0x1:0x0].0x2c847881 bits 0x0/0x0 rrc: 2 type: IBT gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 6257 timeout: 0 lvb_type: 0 00010000:00010000:2.0:1776178213.559123:0:6257:0:(ldlm_lock.c:1064:ldlm_granted_list_add_lock()) ### About to add lock: ns: mdt-lustre-MDT0000_UUID lock: 00000000e4217a5c/0xa40083ab9bf539af lrc: 3/0,1 mode: PW/PW res: [0x200000007:0x1:0x0].0x2c847881 bits 0x2/0x0 rrc: 2 type: IBT gid 0 flags: 0x50210000000000 nid: local remote: 0x0 expref: -99 pid: 6257 timeout: 0 lvb_type: 0 00010000:00010000:2.0:1776178213.559129:0:6257:0:(ldlm_request.c:528:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: mdt-lustre-MDT0000_UUID lock: 00000000e4217a5c/0xa40083ab9bf539af lrc: 3/0,1 mode: PW/PW res: [0x200000007:0x1:0x0].0x2c847881 bits 0x2/0x0 rrc: 2 type: IBT gid 0 flags: 0x40210000000000 nid: local remote: 0x0 expref: -99 pid: 6257 timeout: 0 lvb_type: 0 00010000:00010000:2.0:1776178213.560904:0:6257:0:(ldlm_lock.c:820:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(CW) ns: mdt-lustre-MDT0000_UUID lock: 000000002f1f7bd5/0xa40083ab9bf539a8 lrc: 3/0,1 mode: CW/CW res: [0x200000007:0x1:0x0].0x0 bits 0x2/0x0 rrc: 4 type: IBT gid 0 flags: 0x50210000000000 nid: local remote: 0x0 expref: -99 pid: 6257 timeout: 0 lvb_type: 0 00010000:00010000:2.0:1776178213.560930:0:6257:0:(ldlm_lock.c:879:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: mdt-lustre-MDT0000_UUID lock: 000000002f1f7bd5/0xa40083ab9bf539a8 lrc: 2/0,0 mode: CW/CW res: [0x200000007:0x1:0x0].0x0 bits 0x2/0x0 rrc: 4 type: IBT gid 0 flags: 0x50210400000000 nid: local remote: 0x0 expref: -99 pid: 6257 timeout: 0 lvb_type: 0 00010000:00010000:2.0:1776178213.560939:0:6257:0:(ldlm_lockd.c:1879:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: mdt-lustre-MDT0000_UUID lock: 000000002f1f7bd5/0xa40083ab9bf539a8 lrc: 3/0,0 mode: CW/CW res: [0x200000007:0x1:0x0].0x0 bits 0x2/0x0 rrc: 4 type: IBT gid 0 flags: 0x40210400000000 nid: local remote: 0x0 expref: -99 pid: 6257 timeout: 0 lvb_type: 0 00010000:00010000:2.0:1776178213.560951:0:6257:0:(ldlm_lockd.c:1893:ldlm_handle_bl_callback()) Lock 000000002f1f7bd5 already unused, calling callback (0000000079e78348) 00010000:00010000:2.0:1776178213.560955:0:6257:0:(ldlm_request.c:364:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: mdt-lustre-MDT0000_UUID lock: 000000002f1f7bd5/0xa40083ab9bf539a8 lrc: 3/0,0 mode: CW/CW res: [0x200000007:0x1:0x0].0x0 bits 0x2/0x0 rrc: 4 type: IBT gid 0 flags: 0x40210400000000 nid: local remote: 0x0 expref: -99 pid: 6257 timeout: 0 lvb_type: 0 00010000:00010000:2.0:1776178213.560962:0:6257:0:(ldlm_request.c:1257:ldlm_cli_cancel_local()) ### server-side local cancel ns: mdt-lustre-MDT0000_UUID lock: 000000002f1f7bd5/0xa40083ab9bf539a8 lrc: 4/0,0 mode: CW/CW res: [0x200000007:0x1:0x0].0x0 bits 0x2/0x0 rrc: 4 type: IBT gid 0 flags: 0x40218400000000 nid: local remote: 0x0 expref: -99 pid: 6257 timeout: 0 lvb_type: 0 00010000:00010000:2.0:1776178213.560977:0:6257:0:(ldlm_inodebits.c:94:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x200000007:0x1:0x0].0x0 (00000000ad0d4094) 00010000:00010000:2.0:1776178213.560980:0:6257:0:(ldlm_inodebits.c:96:ldlm_reprocess_inodebits_queue()) Hint 2 00010000:00010000:2.0:1776178213.560984:0:6257:0:(ldlm_lockd.c:1903:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: mdt-lustre-MDT0000_UUID lock: 000000002f1f7bd5/0xa40083ab9bf539a8 lrc: 2/0,0 mode: --/CW res: [0x200000007:0x1:0x0].0x0 bits 0x2/0x0 rrc: 4 type: IBT gid 0 flags: 0x44a19400000000 nid: local remote: 0x0 expref: -99 pid: 6257 timeout: 0 lvb_type: 0 00010000:00010000:2.0:1776178213.560989:0:6257:0:(ldlm_lock.c:216:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: mdt-lustre-MDT0000_UUID lock: 000000002f1f7bd5/0xa40083ab9bf539a8 lrc: 0/0,0 mode: --/CW res: [0x200000007:0x1:0x0].0x0 bits 0x2/0x0 rrc: 4 type: IBT gid 0 flags: 0x44a19400000000 nid: local remote: 0x0 expref: -99 pid: 6257 timeout: 0 lvb_type: 0 00000004:00010000:2.0:1776178213.561001:0:6257:0:(mdt_handler.c:3992:mdt_save_lock()) ### save lock request 0000000068680f90 reply state 0000000040fe0756 transno 12884901973 ns: mdt-lustre-MDT0000_UUID lock: 00000000e4217a5c/0xa40083ab9bf539af lrc: 3/0,1 mode: PW/PW res: [0x200000007:0x1:0x0].0x2c847881 bits 0x2/0x0 rrc: 2 type: IBT gid 0 flags: 0x40210000000000 nid: local remote: 0x0 expref: -99 pid: 6257 timeout: 0 lvb_type: 0 00000100:00100000:2.0:1776178213.561155:0:6257:0:(service.c:2372:ptlrpc_server_handle_request()) Handled RPC req@0000000068680f90 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_002:b50963bd-8eba-4432-9a54-8888d00e3fa1+16:18280:x1862456979107584:12345-192.168.204.3@tcp:36:mcreate.0 Request processed in 2313us (2723us total) trans 12884901973 rc 0/0 00000100:00100000:2.0:1776178213.561165:0:6257:0:(nrs_fifo.c:241:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.204.3@tcp, seq: 6619 00010000:00010000:3.0:1776178213.567218:0:3340:0:(ldlm_lock.c:1064:ldlm_granted_list_add_lock()) ### About to add lock: ns: mdt-lustre-MDT0000_UUID lock: 00000000e4217a5c/0xa40083ab9bf539af lrc: 3/0,1 mode: COS/COS res: [0x200000007:0x1:0x0].0x2c847881 bits 0x2/0x0 rrc: 2 type: IBT gid 0 flags: 0x50210000000000 nid: local remote: 0x0 expref: -99 pid: 6257 timeout: 0 lvb_type: 0 00010000:00010000:3.0:1776178213.567242:0:3340:0:(ldlm_inodebits.c:94:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x200000007:0x1:0x0].0x2c847881 (00000000ccda0888) 00010000:00010000:3.0:1776178213.567244:0:3340:0:(ldlm_inodebits.c:96:ldlm_reprocess_inodebits_queue()) Hint 2 00000100:00100000:0.0:1776178213.581136:0:3347:0:(events.c:361:request_in_callback()) peer: 12345-192.168.204.3@tcp (source: 12345-192.168.204.3@tcp) 00000100:00100000:3.0:1776178213.582377:0:6256:0:(service.c:2152:ptlrpc_server_handle_req_in()) got req x1862456979107648 00000100:00100000:3.0:1776178213.582469:0:6256:0:(nrs_fifo.c:179:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.204.3@tcp, seq: 6620 00000100:00100000:3.0:1776178213.582479:0:6256:0:(service.c:2324:ptlrpc_server_handle_request()) Handling RPC req@000000009c55a259 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_001:b50963bd-8eba-4432-9a54-8888d00e3fa1+16:18283:x1862456979107648:12345-192.168.204.3@tcp:101:rm.0 00010000:00010000:3.0:1776178213.582535:0:6256:0:(ldlm_lockd.c:1248:ldlm_handle_enqueue0()) ### server-side enqueue handler START 00010000:00010000:3.0:1776178213.582566:0:6256:0:(ldlm_lockd.c:1318:ldlm_handle_enqueue0()) ### server-side enqueue handler, new lock created ns: mdt-lustre-MDT0000_UUID lock: 000000008192de90/0xa40083ab9bf539b6 lrc: 2/0,0 mode: --/CR res: [0x200000007:0x1:0x0].0x0 bits 0x0/0x0 rrc: 4 type: IBT gid 0 flags: 0x40000000000000 nid: local remote: 0x771f68aa1eb27c1b expref: -99 pid: 6256 timeout: 0 lvb_type: 0 00010000:00010000:3.0:1776178213.582658:0:6256:0:(ldlm_lock.c:766:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(CR) ns: mdt-lustre-MDT0000_UUID lock: 0000000055220ae2/0xa40083ab9bf539bd lrc: 3/1,0 mode: --/CR res: [0x200000007:0x1:0x0].0x0 bits 0x0/0x0 rrc: 5 type: IBT gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 6256 timeout: 0 lvb_type: 0 00010000:00010000:3.0:1776178213.582678:0:6256:0:(ldlm_lock.c:1064:ldlm_granted_list_add_lock()) ### About to add lock: ns: mdt-lustre-MDT0000_UUID lock: 0000000055220ae2/0xa40083ab9bf539bd lrc: 3/1,0 mode: CR/CR res: [0x200000007:0x1:0x0].0x0 bits 0x2/0x0 rrc: 5 type: IBT gid 0 flags: 0x50210000000000 nid: local remote: 0x0 expref: -99 pid: 6256 timeout: 0 lvb_type: 0 00010000:00010000:3.0:1776178213.582693:0:6256:0:(ldlm_request.c:528:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: mdt-lustre-MDT0000_UUID lock: 0000000055220ae2/0xa40083ab9bf539bd lrc: 3/1,0 mode: CR/CR res: [0x200000007:0x1:0x0].0x0 bits 0x2/0x0 rrc: 5 type: IBT gid 0 flags: 0x40210000000000 nid: local remote: 0x0 expref: -99 pid: 6256 timeout: 0 lvb_type: 0 00010000:00010000:3.0:1776178213.582704:0:6256:0:(ldlm_lock.c:766:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PR) ns: mdt-lustre-MDT0000_UUID lock: 000000005d93f094/0xa40083ab9bf539c4 lrc: 3/1,0 mode: --/PR res: [0x200000007:0x1:0x0].0x2c847881 bits 0x0/0x0 rrc: 3 type: IBT gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 6256 timeout: 0 lvb_type: 0 00010000:00010000:3.0:1776178213.582712:0:6256:0:(ldlm_lock.c:1064:ldlm_granted_list_add_lock()) ### About to add lock: ns: mdt-lustre-MDT0000_UUID lock: 000000005d93f094/0xa40083ab9bf539c4 lrc: 3/1,0 mode: PR/PR res: [0x200000007:0x1:0x0].0x2c847881 bits 0x2/0x0 rrc: 3 type: IBT gid 0 flags: 0x50210000000000 nid: local remote: 0x0 expref: -99 pid: 6256 timeout: 0 lvb_type: 0 00010000:00010000:3.0:1776178213.582718:0:6256:0:(ldlm_request.c:528:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: mdt-lustre-MDT0000_UUID lock: 000000005d93f094/0xa40083ab9bf539c4 lrc: 3/1,0 mode: PR/PR res: [0x200000007:0x1:0x0].0x2c847881 bits 0x2/0x0 rrc: 3 type: IBT gid 0 flags: 0x40210000000000 nid: local remote: 0x0 expref: -99 pid: 6256 timeout: 0 lvb_type: 0 00010000:00010000:3.0:1776178213.582866:0:6256:0:(ldlm_lock.c:766:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PR) ns: mdt-lustre-MDT0000_UUID lock: 0000000026fd08a4/0xa40083ab9bf539cb lrc: 3/1,0 mode: --/PR res: [0x2000013a1:0x11:0x0].0x0 bits 0x0/0x0 rrc: 2 type: IBT gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 6256 timeout: 0 lvb_type: 0 00010000:00010000:3.0:1776178213.582875:0:6256:0:(ldlm_lock.c:1064:ldlm_granted_list_add_lock()) ### About to add lock: ns: mdt-lustre-MDT0000_UUID lock: 0000000026fd08a4/0xa40083ab9bf539cb lrc: 3/1,0 mode: PR/PR res: [0x2000013a1:0x11:0x0].0x0 bits 0x5b/0x0 rrc: 2 type: IBT gid 0 flags: 0x50210000000000 nid: local remote: 0x0 expref: -99 pid: 6256 timeout: 0 lvb_type: 0 00010000:00010000:3.0:1776178213.582881:0:6256:0:(ldlm_request.c:528:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: mdt-lustre-MDT0000_UUID lock: 0000000026fd08a4/0xa40083ab9bf539cb lrc: 3/1,0 mode: PR/PR res: [0x2000013a1:0x11:0x0].0x0 bits 0x5b/0x0 rrc: 2 type: IBT gid 0 flags: 0x40210000000000 nid: local remote: 0x0 expref: -99 pid: 6256 timeout: 0 lvb_type: 0 00000004:00200000:3.0:1776178213.582938:0:6256:0:(mdt_handler.c:912:mdt_pack_attr2body()) [0x2000013a1:0x11:0x0]: returning size 0 00000004:00200000:3.0:1776178213.582943:0:6256:0:(mdt_handler.c:1523:mdt_getattr_internal()) dirent count 0 stripe count 1 MDT count 2 00000004:00010000:3.0:1776178213.582969:0:6256:0:(mdt_handler.c:2353:mdt_getattr_name_lock()) ### Returning lock to client ns: mdt-lustre-MDT0000_UUID lock: 0000000026fd08a4/0xa40083ab9bf539cb lrc: 3/1,0 mode: PR/PR res: [0x2000013a1:0x11:0x0].0x0 bits 0x5b/0x0 rrc: 2 type: IBT gid 0 flags: 0x40210000000000 nid: local remote: 0x0 expref: -99 pid: 6256 timeout: 0 lvb_type: 0 00010000:00010000:3.0:1776178213.582983:0:6256:0:(ldlm_lock.c:1064:ldlm_granted_list_add_lock()) ### About to add lock: ns: mdt-lustre-MDT0000_UUID lock: 0000000026fd08a4/0xa40083ab9bf539cb lrc: 3/1,0 mode: PR/PR res: [0x2000013a1:0x11:0x0].0x0 bits 0x1b/0x0 rrc: 2 type: IBT gid 0 flags: 0x50210000000000 nid: local remote: 0x0 expref: -99 pid: 6256 timeout: 0 lvb_type: 0 00010000:00010000:3.0:1776178213.582992:0:6256:0:(ldlm_lock.c:820:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(CR) ns: mdt-lustre-MDT0000_UUID lock: 0000000055220ae2/0xa40083ab9bf539bd lrc: 3/1,0 mode: CR/CR res: [0x200000007:0x1:0x0].0x0 bits 0x2/0x0 rrc: 5 type: IBT gid 0 flags: 0x50210000000000 nid: local remote: 0x0 expref: -99 pid: 6256 timeout: 0 lvb_type: 0 00010000:00010000:3.0:1776178213.582998:0:6256:0:(ldlm_lock.c:879:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: mdt-lustre-MDT0000_UUID lock: 0000000055220ae2/0xa40083ab9bf539bd lrc: 2/0,0 mode: CR/CR res: [0x200000007:0x1:0x0].0x0 bits 0x2/0x0 rrc: 5 type: IBT gid 0 flags: 0x50210400000000 nid: local remote: 0x0 expref: -99 pid: 6256 timeout: 0 lvb_type: 0 00010000:00010000:3.0:1776178213.583006:0:6256:0:(ldlm_lockd.c:1879:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: mdt-lustre-MDT0000_UUID lock: 0000000055220ae2/0xa40083ab9bf539bd lrc: 3/0,0 mode: CR/CR res: [0x200000007:0x1:0x0].0x0 bits 0x2/0x0 rrc: 5 type: IBT gid 0 flags: 0x40210400000000 nid: local remote: 0x0 expref: -99 pid: 6256 timeout: 0 lvb_type: 0 00010000:00010000:3.0:1776178213.583013:0:6256:0:(ldlm_lockd.c:1893:ldlm_handle_bl_callback()) Lock 0000000055220ae2 already unused, calling callback (0000000079e78348) 00010000:00010000:3.0:1776178213.583017:0:6256:0:(ldlm_request.c:364:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: mdt-lustre-MDT0000_UUID lock: 0000000055220ae2/0xa40083ab9bf539bd lrc: 3/0,0 mode: CR/CR res: [0x200000007:0x1:0x0].0x0 bits 0x2/0x0 rrc: 5 type: IBT gid 0 flags: 0x40210400000000 nid: local remote: 0x0 expref: -99 pid: 6256 timeout: 0 lvb_type: 0 00010000:00010000:3.0:1776178213.583031:0:6256:0:(ldlm_request.c:1257:ldlm_cli_cancel_local()) ### server-side local cancel ns: mdt-lustre-MDT0000_UUID lock: 0000000055220ae2/0xa40083ab9bf539bd lrc: 4/0,0 mode: CR/CR res: [0x200000007:0x1:0x0].0x0 bits 0x2/0x0 rrc: 5 type: IBT gid 0 flags: 0x40218400000000 nid: local remote: 0x0 expref: -99 pid: 6256 timeout: 0 lvb_type: 0 00010000:00010000:3.0:1776178213.583046:0:6256:0:(ldlm_inodebits.c:94:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x200000007:0x1:0x0].0x0 (00000000ad0d4094) 00010000:00010000:3.0:1776178213.583049:0:6256:0:(ldlm_inodebits.c:96:ldlm_reprocess_inodebits_queue()) Hint 2 00010000:00010000:3.0:1776178213.583054:0:6256:0:(ldlm_lockd.c:1903:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: mdt-lustre-MDT0000_UUID lock: 0000000055220ae2/0xa40083ab9bf539bd lrc: 2/0,0 mode: --/CR res: [0x200000007:0x1:0x0].0x0 bits 0x2/0x0 rrc: 5 type: IBT gid 0 flags: 0x44a19400000000 nid: local remote: 0x0 expref: -99 pid: 6256 timeout: 0 lvb_type: 0 00010000:00010000:3.0:1776178213.583088:0:6256:0:(ldlm_lock.c:216:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: mdt-lustre-MDT0000_UUID lock: 0000000055220ae2/0xa40083ab9bf539bd lrc: 0/0,0 mode: --/CR res: [0x200000007:0x1:0x0].0x0 bits 0x2/0x0 rrc: 5 type: IBT gid 0 flags: 0x44a19400000000 nid: local remote: 0x0 expref: -99 pid: 6256 timeout: 0 lvb_type: 0 00010000:00010000:3.0:1776178213.583103:0:6256:0:(ldlm_lock.c:820:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PR) ns: mdt-lustre-MDT0000_UUID lock: 000000005d93f094/0xa40083ab9bf539c4 lrc: 3/1,0 mode: PR/PR res: [0x200000007:0x1:0x0].0x2c847881 bits 0x2/0x0 rrc: 3 type: IBT gid 0 flags: 0x50210000000000 nid: local remote: 0x0 expref: -99 pid: 6256 timeout: 0 lvb_type: 0 00010000:00010000:3.0:1776178213.583108:0:6256:0:(ldlm_lock.c:879:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: mdt-lustre-MDT0000_UUID lock: 000000005d93f094/0xa40083ab9bf539c4 lrc: 2/0,0 mode: PR/PR res: [0x200000007:0x1:0x0].0x2c847881 bits 0x2/0x0 rrc: 3 type: IBT gid 0 flags: 0x50210400000000 nid: local remote: 0x0 expref: -99 pid: 6256 timeout: 0 lvb_type: 0 00010000:00010000:3.0:1776178213.583112:0:6256:0:(ldlm_lockd.c:1879:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: mdt-lustre-MDT0000_UUID lock: 000000005d93f094/0xa40083ab9bf539c4 lrc: 3/0,0 mode: PR/PR res: [0x200000007:0x1:0x0].0x2c847881 bits 0x2/0x0 rrc: 3 type: IBT gid 0 flags: 0x40210400000000 nid: local remote: 0x0 expref: -99 pid: 6256 timeout: 0 lvb_type: 0 00010000:00010000:3.0:1776178213.583116:0:6256:0:(ldlm_lockd.c:1893:ldlm_handle_bl_callback()) Lock 000000005d93f094 already unused, calling callback (0000000079e78348) 00010000:00010000:3.0:1776178213.583118:0:6256:0:(ldlm_request.c:364:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: mdt-lustre-MDT0000_UUID lock: 000000005d93f094/0xa40083ab9bf539c4 lrc: 3/0,0 mode: PR/PR res: [0x200000007:0x1:0x0].0x2c847881 bits 0x2/0x0 rrc: 3 type: IBT gid 0 flags: 0x40210400000000 nid: local remote: 0x0 expref: -99 pid: 6256 timeout: 0 lvb_type: 0 00010000:00010000:3.0:1776178213.583123:0:6256:0:(ldlm_request.c:1257:ldlm_cli_cancel_local()) ### server-side local cancel ns: mdt-lustre-MDT0000_UUID lock: 000000005d93f094/0xa40083ab9bf539c4 lrc: 4/0,0 mode: PR/PR res: [0x200000007:0x1:0x0].0x2c847881 bits 0x2/0x0 rrc: 3 type: IBT gid 0 flags: 0x40218400000000 nid: local remote: 0x0 expref: -99 pid: 6256 timeout: 0 lvb_type: 0 00010000:00010000:3.0:1776178213.583129:0:6256:0:(ldlm_inodebits.c:94:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x200000007:0x1:0x0].0x2c847881 (00000000ccda0888) 00010000:00010000:3.0:1776178213.583131:0:6256:0:(ldlm_inodebits.c:96:ldlm_reprocess_inodebits_queue()) Hint 2 00010000:00010000:3.0:1776178213.583133:0:6256:0:(ldlm_lockd.c:1903:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: mdt-lustre-MDT0000_UUID lock: 000000005d93f094/0xa40083ab9bf539c4 lrc: 2/0,0 mode: --/PR res: [0x200000007:0x1:0x0].0x2c847881 bits 0x2/0x0 rrc: 3 type: IBT gid 0 flags: 0x44a19400000000 nid: local remote: 0x0 expref: -99 pid: 6256 timeout: 0 lvb_type: 0 00010000:00010000:3.0:1776178213.583138:0:6256:0:(ldlm_lock.c:216:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: mdt-lustre-MDT0000_UUID lock: 000000005d93f094/0xa40083ab9bf539c4 lrc: 0/0,0 mode: --/PR res: [0x200000007:0x1:0x0].0x2c847881 bits 0x2/0x0 rrc: 3 type: IBT gid 0 flags: 0x44a19400000000 nid: local remote: 0x0 expref: -99 pid: 6256 timeout: 0 lvb_type: 0 00010000:00010000:3.0:1776178213.583167:0:6256:0:(ldlm_lock.c:216:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: mdt-lustre-MDT0000_UUID lock: 000000008192de90/0xa40083ab9bf539b6 lrc: 0/0,0 mode: --/CR res: [0x200000007:0x1:0x0].0x0 bits 0x2/0x0 rrc: 4 type: IBT gid 0 flags: 0x44000000000000 nid: 192.168.204.3@tcp remote: 0x771f68aa1eb27c1b expref: 18 pid: 6256 timeout: 0 lvb_type: 0 00010000:00010000:3.0:1776178213.583180:0:6256:0:(ldlm_lockd.c:1484:ldlm_handle_enqueue0()) ### server-side enqueue handler, sending reply (err=0, rc=0) ns: mdt-lustre-MDT0000_UUID lock: 0000000026fd08a4/0xa40083ab9bf539cb lrc: 3/0,0 mode: PR/PR res: [0x2000013a1:0x11:0x0].0x0 bits 0x1b/0x0 rrc: 2 type: IBT gid 0 flags: 0x40200000000000 nid: 192.168.204.3@tcp remote: 0x771f68aa1eb27c1b expref: 17 pid: 6256 timeout: 0 lvb_type: 0 00010000:00010000:3.0:1776178213.583187:0:6256:0:(ldlm_inodebits.c:94:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x2000013a1:0x11:0x0].0x0 (0000000019a0327e) 00010000:00010000:3.0:1776178213.583189:0:6256:0:(ldlm_inodebits.c:96:ldlm_reprocess_inodebits_queue()) Hint 1b 00010000:00010000:3.0:1776178213.583191:0:6256:0:(ldlm_lockd.c:1565:ldlm_handle_enqueue0()) ### server-side enqueue handler END (lock 0000000026fd08a4, rc 0) 00000100:00100000:3.0:1776178213.583280:0:6256:0:(service.c:2372:ptlrpc_server_handle_request()) Handled RPC req@000000009c55a259 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_001:b50963bd-8eba-4432-9a54-8888d00e3fa1+17:18283:x1862456979107648:12345-192.168.204.3@tcp:101:rm.0 Request processed in 801us (2149us total) trans 0 rc 0/0 00000100:00100000:3.0:1776178213.583289:0:6256:0:(nrs_fifo.c:241:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.204.3@tcp, seq: 6620 00000100:00100000:1.0:1776178213.586938:0:3348:0:(events.c:361:request_in_callback()) peer: 12345-192.168.204.3@tcp (source: 12345-192.168.204.3@tcp) 00000100:00100000:2.0:1776178213.587046:0:6257:0:(service.c:2152:ptlrpc_server_handle_req_in()) got req x1862456979107776 00000100:00100000:2.0:1776178213.587885:0:6257:0:(nrs_fifo.c:179:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.204.3@tcp, seq: 6621 00000100:00100000:2.0:1776178213.587897:0:6257:0:(service.c:2324:ptlrpc_server_handle_request()) Handling RPC req@00000000dab5abff pname:cluuid+ref:pid:xid:nid:opc:job mdt00_002:b50963bd-8eba-4432-9a54-8888d00e3fa1+17:18283:x1862456979107776:12345-192.168.204.3@tcp:36:rm.0 00010000:00010000:2.0:1776178213.587964:0:6257:0:(ldlm_lockd.c:1720:ldlm_request_cancel()) ### server-side cancel handler START: 1 locks, starting at 0 00010000:00010000:2.0:1776178213.587997:0:6257:0:(ldlm_lock.c:216:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: mdt-lustre-MDT0000_UUID lock: 0000000026fd08a4/0xa40083ab9bf539cb lrc: 0/0,0 mode: --/PR res: [0x2000013a1:0x11:0x0].0x0 bits 0x1b/0x0 rrc: 3 type: IBT gid 0 flags: 0x44a01000000000 nid: 192.168.204.3@tcp remote: 0x771f68aa1eb27c1b expref: 17 pid: 6256 timeout: 0 lvb_type: 0 00010000:00010000:2.0:1776178213.588020:0:6257:0:(ldlm_inodebits.c:94:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x2000013a1:0x11:0x0].0x0 (0000000019a0327e) 00010000:00010000:2.0:1776178213.588029:0:6257:0:(ldlm_lockd.c:1782:ldlm_request_cancel()) ### server-side cancel handler END 00010000:00010000:2.0:1776178213.588090:0:6257:0:(ldlm_lock.c:766:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(CW) ns: mdt-lustre-MDT0000_UUID lock: 000000002f1f7bd5/0xa40083ab9bf539d2 lrc: 3/0,1 mode: --/CW res: [0x200000007:0x1:0x0].0x0 bits 0x0/0x0 rrc: 4 type: IBT gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 6257 timeout: 0 lvb_type: 0 00010000:00010000:2.0:1776178213.588109:0:6257:0:(ldlm_lock.c:1064:ldlm_granted_list_add_lock()) ### About to add lock: ns: mdt-lustre-MDT0000_UUID lock: 000000002f1f7bd5/0xa40083ab9bf539d2 lrc: 3/0,1 mode: CW/CW res: [0x200000007:0x1:0x0].0x0 bits 0x2/0x0 rrc: 4 type: IBT gid 0 flags: 0x50210000000000 nid: local remote: 0x0 expref: -99 pid: 6257 timeout: 0 lvb_type: 0 00010000:00010000:2.0:1776178213.588122:0:6257:0:(ldlm_request.c:528:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: mdt-lustre-MDT0000_UUID lock: 000000002f1f7bd5/0xa40083ab9bf539d2 lrc: 3/0,1 mode: CW/CW res: [0x200000007:0x1:0x0].0x0 bits 0x2/0x0 rrc: 4 type: IBT gid 0 flags: 0x40210000000000 nid: local remote: 0x0 expref: -99 pid: 6257 timeout: 0 lvb_type: 0 00010000:00010000:2.0:1776178213.588135:0:6257:0:(ldlm_lock.c:766:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: mdt-lustre-MDT0000_UUID lock: 00000000a27c19b3/0xa40083ab9bf539d9 lrc: 3/0,1 mode: --/PW res: [0x200000007:0x1:0x0].0x2c847881 bits 0x0/0x0 rrc: 3 type: IBT gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 6257 timeout: 0 lvb_type: 0 00010000:00010000:2.0:1776178213.588143:0:6257:0:(ldlm_lock.c:1064:ldlm_granted_list_add_lock()) ### About to add lock: ns: mdt-lustre-MDT0000_UUID lock: 00000000a27c19b3/0xa40083ab9bf539d9 lrc: 3/0,1 mode: PW/PW res: [0x200000007:0x1:0x0].0x2c847881 bits 0x2/0x0 rrc: 3 type: IBT gid 0 flags: 0x50210000000000 nid: local remote: 0x0 expref: -99 pid: 6257 timeout: 0 lvb_type: 0 00010000:00010000:2.0:1776178213.588149:0:6257:0:(ldlm_request.c:528:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: mdt-lustre-MDT0000_UUID lock: 00000000a27c19b3/0xa40083ab9bf539d9 lrc: 3/0,1 mode: PW/PW res: [0x200000007:0x1:0x0].0x2c847881 bits 0x2/0x0 rrc: 3 type: IBT gid 0 flags: 0x40210000000000 nid: local remote: 0x0 expref: -99 pid: 6257 timeout: 0 lvb_type: 0 00010000:00010000:2.0:1776178213.588268:0:6257:0:(ldlm_lock.c:766:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(EX) ns: mdt-lustre-MDT0000_UUID lock: 000000000b5c0143/0xa40083ab9bf539e0 lrc: 3/0,1 mode: --/EX res: [0x2000013a1:0x11:0x0].0x0 bits 0x0/0x0 rrc: 2 type: IBT gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 6257 timeout: 0 lvb_type: 0 00010000:00010000:2.0:1776178213.588276:0:6257:0:(ldlm_lock.c:1064:ldlm_granted_list_add_lock()) ### About to add lock: ns: mdt-lustre-MDT0000_UUID lock: 000000000b5c0143/0xa40083ab9bf539e0 lrc: 3/0,1 mode: EX/EX res: [0x2000013a1:0x11:0x0].0x0 bits 0x3/0x0 rrc: 2 type: IBT gid 0 flags: 0x50210000000000 nid: local remote: 0x0 expref: -99 pid: 6257 timeout: 0 lvb_type: 0 00010000:00010000:2.0:1776178213.588283:0:6257:0:(ldlm_request.c:528:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: mdt-lustre-MDT0000_UUID lock: 000000000b5c0143/0xa40083ab9bf539e0 lrc: 3/0,1 mode: EX/EX res: [0x2000013a1:0x11:0x0].0x0 bits 0x3/0x0 rrc: 2 type: IBT gid 0 flags: 0x40210000000000 nid: local remote: 0x0 expref: -99 pid: 6257 timeout: 0 lvb_type: 0 00010000:00010000:2.0:1776178213.588556:0:6257:0:(ldlm_lock.c:766:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: mdt-lustre-MDT0000_UUID lock: 00000000c9d7d9fc/0xa40083ab9bf539e7 lrc: 3/0,1 mode: --/PW res: [0x2000013a1:0x11:0x0].0x0 bits 0x0/0x0 rrc: 3 type: IBT gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 6257 timeout: 0 lvb_type: 0 00010000:00010000:2.0:1776178213.588565:0:6257:0:(ldlm_lock.c:1064:ldlm_granted_list_add_lock()) ### About to add lock: ns: mdt-lustre-MDT0000_UUID lock: 00000000c9d7d9fc/0xa40083ab9bf539e7 lrc: 3/0,1 mode: PW/PW res: [0x2000013a1:0x11:0x0].0x0 bits 0x40/0x0 rrc: 3 type: IBT gid 18446635902900449728 flags: 0x50010080000000 nid: local remote: 0x0 expref: -99 pid: 6257 timeout: 0 lvb_type: 0 00010000:00010000:2.0:1776178213.588574:0:6257:0:(ldlm_request.c:528:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: mdt-lustre-MDT0000_UUID lock: 00000000c9d7d9fc/0xa40083ab9bf539e7 lrc: 3/0,1 mode: PW/PW res: [0x2000013a1:0x11:0x0].0x0 bits 0x40/0x0 rrc: 3 type: IBT gid 18446635902900449728 flags: 0x40010080000000 nid: local remote: 0x0 expref: -99 pid: 6257 timeout: 0 lvb_type: 0 00010000:00010000:2.0:1776178213.588583:0:6257:0:(ldlm_lock.c:942:ldlm_lock_decref_and_cancel()) ### ldlm_lock_decref(PW) ns: mdt-lustre-MDT0000_UUID lock: 00000000c9d7d9fc/0xa40083ab9bf539e7 lrc: 4/0,1 mode: PW/PW res: [0x2000013a1:0x11:0x0].0x0 bits 0x40/0x0 rrc: 3 type: IBT gid 18446635902900449728 flags: 0x40010080000000 nid: local remote: 0x0 expref: -99 pid: 6257 timeout: 0 lvb_type: 0 00010000:00010000:2.0:1776178213.588590:0:6257:0:(ldlm_lock.c:820:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(PW) ns: mdt-lustre-MDT0000_UUID lock: 00000000c9d7d9fc/0xa40083ab9bf539e7 lrc: 4/0,1 mode: PW/PW res: [0x2000013a1:0x11:0x0].0x0 bits 0x40/0x0 rrc: 3 type: IBT gid 18446635902900449728 flags: 0x50010480000000 nid: local remote: 0x0 expref: -99 pid: 6257 timeout: 0 lvb_type: 0 00010000:00010000:2.0:1776178213.588596:0:6257:0:(ldlm_lock.c:879:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: mdt-lustre-MDT0000_UUID lock: 00000000c9d7d9fc/0xa40083ab9bf539e7 lrc: 3/0,0 mode: PW/PW res: [0x2000013a1:0x11:0x0].0x0 bits 0x40/0x0 rrc: 3 type: IBT gid 18446635902900449728 flags: 0x50010480000000 nid: local remote: 0x0 expref: -99 pid: 6257 timeout: 0 lvb_type: 0 00010000:00010000:2.0:1776178213.588826:0:6257:0:(ldlm_lockd.c:2159:__ldlm_bl_to_thread()) added 1/1 locks to regular blp list, 1 blwis in pool 00000004:00010000:2.0:1776178213.588846:0:6257:0:(mdt_handler.c:3992:mdt_save_lock()) ### save lock request 00000000dab5abff reply state 00000000dd0fad85 transno 12884901974 ns: mdt-lustre-MDT0000_UUID lock: 000000000b5c0143/0xa40083ab9bf539e0 lrc: 3/0,1 mode: EX/EX res: [0x2000013a1:0x11:0x0].0x0 bits 0x3/0x0 rrc: 3 type: IBT gid 0 flags: 0x40210000000000 nid: local remote: 0x0 expref: -99 pid: 6257 timeout: 0 lvb_type: 0 00010000:00010000:2.0:1776178213.588977:0:6257:0:(ldlm_lock.c:820:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(CW) ns: mdt-lustre-MDT0000_UUID lock: 000000002f1f7bd5/0xa40083ab9bf539d2 lrc: 3/0,1 mode: CW/CW res: [0x200000007:0x1:0x0].0x0 bits 0x2/0x0 rrc: 4 type: IBT gid 0 flags: 0x50210000000000 nid: local remote: 0x0 expref: -99 pid: 6257 timeout: 0 lvb_type: 0 00010000:00010000:2.0:1776178213.588983:0:6257:0:(ldlm_lock.c:879:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: mdt-lustre-MDT0000_UUID lock: 000000002f1f7bd5/0xa40083ab9bf539d2 lrc: 2/0,0 mode: CW/CW res: [0x200000007:0x1:0x0].0x0 bits 0x2/0x0 rrc: 4 type: IBT gid 0 flags: 0x50210400000000 nid: local remote: 0x0 expref: -99 pid: 6257 timeout: 0 lvb_type: 0 00010000:00010000:2.0:1776178213.588989:0:6257:0:(ldlm_lockd.c:1879:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: mdt-lustre-MDT0000_UUID lock: 000000002f1f7bd5/0xa40083ab9bf539d2 lrc: 3/0,0 mode: CW/CW res: [0x200000007:0x1:0x0].0x0 bits 0x2/0x0 rrc: 4 type: IBT gid 0 flags: 0x40210400000000 nid: local remote: 0x0 expref: -99 pid: 6257 timeout: 0 lvb_type: 0 00010000:00010000:2.0:1776178213.588994:0:6257:0:(ldlm_lockd.c:1893:ldlm_handle_bl_callback()) Lock 000000002f1f7bd5 already unused, calling callback (0000000079e78348) 00010000:00010000:2.0:1776178213.589000:0:6257:0:(ldlm_request.c:364:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: mdt-lustre-MDT0000_UUID lock: 000000002f1f7bd5/0xa40083ab9bf539d2 lrc: 3/0,0 mode: CW/CW res: [0x200000007:0x1:0x0].0x0 bits 0x2/0x0 rrc: 4 type: IBT gid 0 flags: 0x40210400000000 nid: local remote: 0x0 expref: -99 pid: 6257 timeout: 0 lvb_type: 0 00010000:00010000:2.0:1776178213.589008:0:6257:0:(ldlm_request.c:1257:ldlm_cli_cancel_local()) ### server-side local cancel ns: mdt-lustre-MDT0000_UUID lock: 000000002f1f7bd5/0xa40083ab9bf539d2 lrc: 4/0,0 mode: CW/CW res: [0x200000007:0x1:0x0].0x0 bits 0x2/0x0 rrc: 4 type: IBT gid 0 flags: 0x40218400000000 nid: local remote: 0x0 expref: -99 pid: 6257 timeout: 0 lvb_type: 0 00010000:00010000:2.0:1776178213.589017:0:6257:0:(ldlm_inodebits.c:94:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x200000007:0x1:0x0].0x0 (00000000ad0d4094) 00010000:00010000:2.0:1776178213.589020:0:6257:0:(ldlm_inodebits.c:96:ldlm_reprocess_inodebits_queue()) Hint 2 00010000:00010000:2.0:1776178213.589022:0:6257:0:(ldlm_lockd.c:1903:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: mdt-lustre-MDT0000_UUID lock: 000000002f1f7bd5/0xa40083ab9bf539d2 lrc: 2/0,0 mode: --/CW res: [0x200000007:0x1:0x0].0x0 bits 0x2/0x0 rrc: 4 type: IBT gid 0 flags: 0x44a19400000000 nid: local remote: 0x0 expref: -99 pid: 6257 timeout: 0 lvb_type: 0 00010000:00010000:2.0:1776178213.589028:0:6257:0:(ldlm_lock.c:216:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: mdt-lustre-MDT0000_UUID lock: 000000002f1f7bd5/0xa40083ab9bf539d2 lrc: 0/0,0 mode: --/CW res: [0x200000007:0x1:0x0].0x0 bits 0x2/0x0 rrc: 4 type: IBT gid 0 flags: 0x44a19400000000 nid: local remote: 0x0 expref: -99 pid: 6257 timeout: 0 lvb_type: 0 00000004:00010000:2.0:1776178213.589037:0:6257:0:(mdt_handler.c:3992:mdt_save_lock()) ### save lock request 00000000dab5abff reply state 00000000dd0fad85 transno 12884901974 ns: mdt-lustre-MDT0000_UUID lock: 00000000a27c19b3/0xa40083ab9bf539d9 lrc: 3/0,1 mode: PW/PW res: [0x200000007:0x1:0x0].0x2c847881 bits 0x2/0x0 rrc: 3 type: IBT gid 0 flags: 0x40210000000000 nid: local remote: 0x0 expref: -99 pid: 6257 timeout: 0 lvb_type: 0 00000100:00100000:2.0:1776178213.589168:0:6257:0:(service.c:2372:ptlrpc_server_handle_request()) Handled RPC req@00000000dab5abff pname:cluuid+ref:pid:xid:nid:opc:job mdt00_002:b50963bd-8eba-4432-9a54-8888d00e3fa1+18:18283:x1862456979107776:12345-192.168.204.3@tcp:36:rm.0 Request processed in 1271us (2241us total) trans 12884901974 rc 0/0 00000100:00100000:2.0:1776178213.589178:0:6257:0:(nrs_fifo.c:241:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.204.3@tcp, seq: 6621 00010000:00010000:1.0:1776178213.589764:0:6245:0:(ldlm_lockd.c:2789:ldlm_bl_get_work()) Got 1 locks of 0 total in blp. (0 blwis in pool) 00010000:00010000:1.0:1776178213.589771:0:6245:0:(ldlm_lockd.c:1879:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: mdt-lustre-MDT0000_UUID lock: 00000000c9d7d9fc/0xa40083ab9bf539e7 lrc: 2/0,0 mode: PW/PW res: [0x2000013a1:0x11:0x0].0x0 bits 0x40/0x0 rrc: 3 type: IBT gid 18446635902900449728 flags: 0x40010480000000 nid: local remote: 0x0 expref: -99 pid: 6257 timeout: 0 lvb_type: 0 00010000:00010000:1.0:1776178213.589780:0:6245:0:(ldlm_lockd.c:1893:ldlm_handle_bl_callback()) Lock 00000000c9d7d9fc already unused, calling callback (0000000063c97258) 00010000:00010000:1.0:1776178213.589783:0:6245:0:(ldlm_request.c:364:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: mdt-lustre-MDT0000_UUID lock: 00000000c9d7d9fc/0xa40083ab9bf539e7 lrc: 2/0,0 mode: PW/PW res: [0x2000013a1:0x11:0x0].0x0 bits 0x40/0x0 rrc: 3 type: IBT gid 18446635902900449728 flags: 0x40010480000000 nid: local remote: 0x0 expref: -99 pid: 6257 timeout: 0 lvb_type: 0 00010000:00010000:1.0:1776178213.589790:0:6245:0:(ldlm_request.c:1257:ldlm_cli_cancel_local()) ### server-side local cancel ns: mdt-lustre-MDT0000_UUID lock: 00000000c9d7d9fc/0xa40083ab9bf539e7 lrc: 3/0,0 mode: PW/PW res: [0x2000013a1:0x11:0x0].0x0 bits 0x40/0x0 rrc: 3 type: IBT gid 18446635902900449728 flags: 0x40018480000000 nid: local remote: 0x0 expref: -99 pid: 6257 timeout: 0 lvb_type: 0 00010000:00010000:1.0:1776178213.589804:0:6245:0:(ldlm_inodebits.c:94:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x2000013a1:0x11:0x0].0x0 (00000000024b4e8f) 00010000:00010000:1.0:1776178213.589806:0:6245:0:(ldlm_inodebits.c:96:ldlm_reprocess_inodebits_queue()) Hint 40 00010000:00010000:1.0:1776178213.589809:0:6245:0:(ldlm_lockd.c:1903:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: mdt-lustre-MDT0000_UUID lock: 00000000c9d7d9fc/0xa40083ab9bf539e7 lrc: 1/0,0 mode: --/PW res: [0x2000013a1:0x11:0x0].0x0 bits 0x40/0x0 rrc: 3 type: IBT gid 18446635902900449728 flags: 0x44819480000000 nid: local remote: 0x0 expref: -99 pid: 6257 timeout: 0 lvb_type: 0 00010000:00010000:1.0:1776178213.589815:0:6245:0:(ldlm_lock.c:216:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: mdt-lustre-MDT0000_UUID lock: 00000000c9d7d9fc/0xa40083ab9bf539e7 lrc: 0/0,0 mode: --/PW res: [0x2000013a1:0x11:0x0].0x0 bits 0x40/0x0 rrc: 3 type: IBT gid 18446635902900449728 flags: 0x44819480000000 nid: local remote: 0x0 expref: -99 pid: 6257 timeout: 0 lvb_type: 0 00010000:00010000:1.0:1776178213.589827:0:6245:0:(ldlm_lockd.c:2792:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00010000:00010000:1.0:1776178213.589829:0:6245:0:(ldlm_lockd.c:2792:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00010000:00010000:1.0:1776178213.589830:0:6245:0:(ldlm_lockd.c:2792:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00010000:00010000:2.0:1776178213.593998:0:3339:0:(ldlm_lock.c:1064:ldlm_granted_list_add_lock()) ### About to add lock: ns: mdt-lustre-MDT0000_UUID lock: 00000000a27c19b3/0xa40083ab9bf539d9 lrc: 3/0,1 mode: COS/COS res: [0x200000007:0x1:0x0].0x2c847881 bits 0x2/0x0 rrc: 3 type: IBT gid 0 flags: 0x50210000000000 nid: local remote: 0x0 expref: -99 pid: 6257 timeout: 0 lvb_type: 0 00010000:00010000:2.0:1776178213.594031:0:3339:0:(ldlm_inodebits.c:94:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x200000007:0x1:0x0].0x2c847881 (00000000ccda0888) 00010000:00010000:2.0:1776178213.594035:0:3339:0:(ldlm_inodebits.c:96:ldlm_reprocess_inodebits_queue()) Hint 2 00010000:00010000:2.0:1776178213.594042:0:3339:0:(ldlm_lock.c:1064:ldlm_granted_list_add_lock()) ### About to add lock: ns: mdt-lustre-MDT0000_UUID lock: 000000000b5c0143/0xa40083ab9bf539e0 lrc: 3/0,1 mode: COS/COS res: [0x2000013a1:0x11:0x0].0x0 bits 0x3/0x0 rrc: 2 type: IBT gid 0 flags: 0x50210000000000 nid: local remote: 0x0 expref: -99 pid: 6257 timeout: 0 lvb_type: 0 00010000:00010000:2.0:1776178213.594050:0:3339:0:(ldlm_inodebits.c:94:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x2000013a1:0x11:0x0].0x0 (00000000024b4e8f) 00010000:00010000:2.0:1776178213.594052:0:3339:0:(ldlm_inodebits.c:96:ldlm_reprocess_inodebits_queue()) Hint 3 00000100:00100000:0.0:1776178214.181170:0:3348:0:(events.c:361:request_in_callback()) peer: 12345-192.168.204.3@tcp (source: 12345-192.168.204.3@tcp) 00000100:00100000:0.0:1776178214.181223:0:3348:0:(events.c:361:request_in_callback()) peer: 12345-192.168.204.3@tcp (source: 12345-192.168.204.3@tcp) 00000100:00100000:0.0:1776178214.181253:0:3348:0:(events.c:361:request_in_callback()) peer: 12345-192.168.204.3@tcp (source: 12345-192.168.204.3@tcp) 00000100:00100000:0.0:1776178214.181282:0:3348:0:(events.c:361:request_in_callback()) peer: 12345-192.168.204.3@tcp (source: 12345-192.168.204.3@tcp) 00000100:00100000:0.0:1776178214.181323:0:20755:0:(service.c:2152:ptlrpc_server_handle_req_in()) got req x1862456979107968 00000100:00100000:0.0:1776178214.181379:0:20755:0:(service.c:2152:ptlrpc_server_handle_req_in()) got req x1862456979108032 00000100:00100000:0.0:1776178214.181403:0:20755:0:(nrs_fifo.c:179:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.204.3@tcp, seq: 519 00000100:00100000:0.0:1776178214.181410:0:20755:0:(service.c:2324:ptlrpc_server_handle_request()) Handling RPC req@0000000095214b8e pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_006:b50963bd-8eba-4432-9a54-8888d00e3fa1+5:2273:x1862456979107968:12345-192.168.204.3@tcp:400:kworker/u8:3.0 00000100:00100000:0.0:1776178214.181494:0:20755:0:(service.c:2372:ptlrpc_server_handle_request()) Handled RPC req@0000000095214b8e pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_006:b50963bd-8eba-4432-9a54-8888d00e3fa1+5:2273:x1862456979107968:12345-192.168.204.3@tcp:400:kworker/u8:3.0 Request processed in 83us (271us total) trans 0 rc 0/0 00000100:00100000:0.0:1776178214.181502:0:20755:0:(nrs_fifo.c:241:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.204.3@tcp, seq: 519 00000100:00100000:0.0:1776178214.181531:0:20755:0:(nrs_fifo.c:179:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.204.3@tcp, seq: 520 00000100:00100000:0.0:1776178214.181535:0:20755:0:(service.c:2324:ptlrpc_server_handle_request()) Handling RPC req@00000000df370bee pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_006:b50963bd-8eba-4432-9a54-8888d00e3fa1+5:2273:x1862456979108032:12345-192.168.204.3@tcp:400:kworker/u8:3.0 00000100:00100000:0.0:1776178214.181559:0:20755:0:(service.c:2372:ptlrpc_server_handle_request()) Handled RPC req@00000000df370bee pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_006:b50963bd-8eba-4432-9a54-8888d00e3fa1+5:2273:x1862456979108032:12345-192.168.204.3@tcp:400:kworker/u8:3.0 Request processed in 23us (277us total) trans 0 rc 0/0 00000100:00100000:0.0:1776178214.181565:0:20755:0:(nrs_fifo.c:241:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.204.3@tcp, seq: 520 00000100:00100000:0.0:1776178214.181583:0:6257:0:(service.c:2152:ptlrpc_server_handle_req_in()) got req x1862456979107904 00000100:00100000:0.0:1776178214.181609:0:6257:0:(nrs_fifo.c:179:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.204.3@tcp, seq: 924 00000100:00100000:0.0:1776178214.181613:0:6257:0:(service.c:2324:ptlrpc_server_handle_request()) Handling RPC req@000000004e8fbd9b pname:cluuid+ref:pid:xid:nid:opc:job mdt00_002:b50963bd-8eba-4432-9a54-8888d00e3fa1+6:2273:x1862456979107904:12345-192.168.204.3@tcp:400:kworker/u8:3.0 00000100:00100000:0.0:1776178214.181641:0:6257:0:(service.c:2372:ptlrpc_server_handle_request()) Handled RPC req@000000004e8fbd9b pname:cluuid+ref:pid:xid:nid:opc:job mdt00_002:b50963bd-8eba-4432-9a54-8888d00e3fa1+6:2273:x1862456979107904:12345-192.168.204.3@tcp:400:kworker/u8:3.0 Request processed in 27us (388us total) trans 0 rc 0/0 00000100:00100000:0.0:1776178214.181647:0:6257:0:(nrs_fifo.c:241:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.204.3@tcp, seq: 924 00000100:00100000:2.0:1776178214.182528:0:20137:0:(service.c:2152:ptlrpc_server_handle_req_in()) got req x1862456979107840 00000100:00100000:2.0:1776178214.182581:0:20137:0:(nrs_fifo.c:179:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.204.3@tcp, seq: 160 00000100:00100000:2.0:1776178214.182586:0:20137:0:(service.c:2324:ptlrpc_server_handle_request()) Handling RPC req@000000003524ab73 pname:cluuid+ref:pid:xid:nid:opc:job ll_mgs_0000:5b5c66b1-1a2f-4651-8548-0c75b9a38a94+9:2273:x1862456979107840:12345-192.168.204.3@tcp:400:kworker/u8:3.0 00000100:00100000:2.0:1776178214.182624:0:20137:0:(service.c:2372:ptlrpc_server_handle_request()) Handled RPC req@000000003524ab73 pname:cluuid+ref:pid:xid:nid:opc:job ll_mgs_0000:5b5c66b1-1a2f-4651-8548-0c75b9a38a94+9:2273:x1862456979107840:12345-192.168.204.3@tcp:400:kworker/u8:3.0 Request processed in 38us (1466us total) trans 0 rc 0/0 00000100:00100000:2.0:1776178214.182630:0:20137:0:(nrs_fifo.c:241:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.204.3@tcp, seq: 160 00010000:00010000:2.0:1776178214.370764:0:9224:0:(ldlm_lockd.c:2792:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000020:00000080:3.0:1776178214.521880:0:21709:0:(class_obd.c:302:class_handle_ioctl()) cmd = c008667f 00000020:00000080:3.0:1776178214.521889:0:21709:0:(class_obd.c:126:class_resolve_dev_name()) device name lustre-MDT0000 00000020:00000080:3.0:1776178214.521895:0:21709:0:(class_obd.c:133:class_resolve_dev_name()) device name lustre-MDT0000, dev 5 00010000:00010000:3.0:1776178215.392234:0:6244:0:(ldlm_lockd.c:2792:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:2.0:1776178215.777901:0:3355:0:(client.c:1733:ptlrpc_send_new_req()) Sending RPC req@000000007d766b2a pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_02:lustre-MDT0001-mdtlov_UUID:3355:1862456998049664:0@lo:41:osp-pre-0-1.0 00000100:00100000:2.0:1776178215.777990:0:3355:0:(events.c:361:request_in_callback()) peer: 12345-0@lo (source: 12345-0@lo) 00000100:00100000:2.0:1776178215.778028:0:3355:0:(ptlrpcd.c:415:ptlrpcd_check()) transfer 1 async RPCs [3->2] 00000100:00100000:2.0:1776178215.778034:0:3355:0:(client.c:1733:ptlrpc_send_new_req()) Sending RPC req@0000000065dbeee0 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_02:lustre-MDT0000-mdtlov_UUID:3355:1862456998049728:0@lo:13:osp-pre-1-0.0 00000100:00100000:2.0:1776178215.778089:0:3355:0:(events.c:361:request_in_callback()) peer: 12345-0@lo (source: 12345-0@lo) 00000100:00100000:0.0:1776178215.778422:0:14128:0:(service.c:2152:ptlrpc_server_handle_req_in()) got req x1862456998049664 00000100:00100000:0.0:1776178215.778468:0:14128:0:(nrs_fifo.c:179:nrs_fifo_req_get()) NRS start fifo request from 12345-0@lo, seq: 706 00000100:00100000:0.0:1776178215.778474:0:14128:0:(service.c:2324:ptlrpc_server_handle_request()) Handling RPC req@00000000fab910b9 pname:cluuid+ref:pid:xid:nid:opc:job mdt_out00_002:lustre-MDT0001-mdtlov_UUID+6:3355:x1862456998049664:12345-0@lo:41:osp-pre-0-1.0 00000004:00000024:0.0:1776178215.778520:0:14128:0:(mdt_handler.c:548:mdt_statfs()) blocks cached 0 granted 2097152 pending 0 free 1446121472 avail 1316659200 00010000:00000080:0.0:1776178215.778535:0:14128:0:(ldlm_lib.c:3181:target_committed_to_req()) @@@ not sending last_committed update (1/0) req@00000000fab910b9 x1862456998049664/t0(0) o41->lustre-MDT0001-mdtlov_UUID@0@lo:742/0 lens 224/368 e 0 to 0 dl 1776178257 ref 1 fl Interpret:/0/0 rc 0/0 job:'osp-pre-0-1.0' 00000100:00100000:0.0:1776178215.778580:0:14128:0:(service.c:2372:ptlrpc_server_handle_request()) Handled RPC req@00000000fab910b9 pname:cluuid+ref:pid:xid:nid:opc:job mdt_out00_002:lustre-MDT0001-mdtlov_UUID+6:3355:x1862456998049664:12345-0@lo:41:osp-pre-0-1.0 Request processed in 105us (592us total) trans 0 rc 0/0 00000100:00100000:0.0:1776178215.778587:0:14128:0:(nrs_fifo.c:241:nrs_fifo_req_stop()) NRS stop fifo request from 12345-0@lo, seq: 706 00000100:00100000:0.0:1776178215.778612:0:9554:0:(service.c:2152:ptlrpc_server_handle_req_in()) got req x1862456998049728 00000100:00100000:0.0:1776178215.778635:0:9554:0:(nrs_fifo.c:179:nrs_fifo_req_get()) NRS start fifo request from 12345-0@lo, seq: 489 00000100:00100000:0.0:1776178215.778639:0:9554:0:(service.c:2324:ptlrpc_server_handle_request()) Handling RPC req@00000000ca8b81ba pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_create00:lustre-MDT0000-mdtlov_UUID+5:3355:x1862456998049728:12345-0@lo:13:osp-pre-1-0.0 00002000:00000024:0.0:1776178215.778656:0:9554:0:(ofd_obd.c:740:ofd_statfs()) blocks cached 229376 granted 491746368 pending 0 free 3923689472 avail 3692167168 00000100:00100000:0.0:1776178215.778683:0:9554:0:(service.c:2372:ptlrpc_server_handle_request()) Handled RPC req@00000000ca8b81ba pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_create00:lustre-MDT0000-mdtlov_UUID+5:3355:x1862456998049728:12345-0@lo:13:osp-pre-1-0.0 Request processed in 43us (594us total) trans 0 rc 0/0 00000100:00100000:0.0:1776178215.778689:0:9554:0:(nrs_fifo.c:241:nrs_fifo_req_stop()) NRS stop fifo request from 12345-0@lo, seq: 489 00000100:00100000:0.0:1776178215.778728:0:3353:0:(client.c:1733:ptlrpc_send_new_req()) Sending RPC req@00000000aba13576 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_00:lustre-MDT0000-mdtlov_UUID:3353:1862456998049792:0@lo:41:osp-pre-1-0.0 00000100:00100000:0.0:1776178215.778753:0:3353:0:(events.c:361:request_in_callback()) peer: 12345-0@lo (source: 12345-0@lo) 00000100:00100000:0.0:1776178215.778769:0:3353:0:(ptlrpcd.c:415:ptlrpcd_check()) transfer 1 async RPCs [1->0] 00000100:00100000:0.0:1776178215.778774:0:3353:0:(client.c:1733:ptlrpc_send_new_req()) Sending RPC req@000000006e2391ef pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_00:lustre-MDT0000-mdtlov_UUID:3353:1862456998049856:0@lo:13:osp-pre-0-0.0 00000100:00100000:0.0:1776178215.778794:0:3353:0:(events.c:361:request_in_callback()) peer: 12345-0@lo (source: 12345-0@lo) 00000100:00100000:0.0:1776178215.778819:0:6261:0:(service.c:2152:ptlrpc_server_handle_req_in()) got req x1862456998049792 00000100:00100000:0.0:1776178215.778841:0:6261:0:(nrs_fifo.c:179:nrs_fifo_req_get()) NRS start fifo request from 12345-0@lo, seq: 707 00000100:00100000:0.0:1776178215.778844:0:6261:0:(service.c:2324:ptlrpc_server_handle_request()) Handling RPC req@000000006c57be21 pname:cluuid+ref:pid:xid:nid:opc:job mdt_out00_001:lustre-MDT0000-mdtlov_UUID+5:3353:x1862456998049792:12345-0@lo:41:osp-pre-1-0.0 00000004:00000024:0.0:1776178215.778860:0:6261:0:(mdt_handler.c:548:mdt_statfs()) blocks cached 0 granted 2097152 pending 0 free 1446039552 avail 1316577280 00000100:00100000:0.0:1776178215.778889:0:6261:0:(service.c:2372:ptlrpc_server_handle_request()) Handled RPC req@000000006c57be21 pname:cluuid+ref:pid:xid:nid:opc:job mdt_out00_001:lustre-MDT0000-mdtlov_UUID+5:3353:x1862456998049792:12345-0@lo:41:osp-pre-1-0.0 Request processed in 43us (136us total) trans 0 rc 0/0 00000100:00100000:0.0:1776178215.778895:0:6261:0:(nrs_fifo.c:241:nrs_fifo_req_stop()) NRS stop fifo request from 12345-0@lo, seq: 707 00000100:00100000:0.0:1776178215.778935:0:3355:0:(client.c:2215:ptlrpc_check_set()) Completed RPC req@0000000065dbeee0 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_02:lustre-MDT0000-mdtlov_UUID:3355:1862456998049728:0@lo:13:osp-pre-1-0.0 00000100:00100000:0.0:1776178215.778951:0:3355:0:(client.c:2215:ptlrpc_check_set()) Completed RPC req@000000007d766b2a pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_02:lustre-MDT0001-mdtlov_UUID:3355:1862456998049664:0@lo:41:osp-pre-0-1.0 00000100:00100000:0.0:1776178215.778977:0:8232:0:(service.c:2152:ptlrpc_server_handle_req_in()) got req x1862456998049856 00000100:00100000:0.0:1776178215.778999:0:8232:0:(nrs_fifo.c:179:nrs_fifo_req_get()) NRS start fifo request from 12345-0@lo, seq: 490 00000100:00100000:0.0:1776178215.779004:0:8232:0:(service.c:2324:ptlrpc_server_handle_request()) Handling RPC req@000000001d2ef500 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_create00:lustre-MDT0000-mdtlov_UUID+5:3353:x1862456998049856:12345-0@lo:13:osp-pre-0-0.0 00002000:00000024:0.0:1776178215.779018:0:8232:0:(ofd_obd.c:740:ofd_statfs()) blocks cached 315392 granted 484883776 pending 0 free 3923689472 avail 3692167168 00000100:00100000:0.0:1776178215.779043:0:8232:0:(service.c:2372:ptlrpc_server_handle_request()) Handled RPC req@000000001d2ef500 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_create00:lustre-MDT0000-mdtlov_UUID+5:3353:x1862456998049856:12345-0@lo:13:osp-pre-0-0.0 Request processed in 40us (249us total) trans 0 rc 0/0 00000100:00100000:0.0:1776178215.779049:0:8232:0:(nrs_fifo.c:241:nrs_fifo_req_stop()) NRS stop fifo request from 12345-0@lo, seq: 490 00000100:00100000:0.0:1776178215.779098:0:3353:0:(client.c:2215:ptlrpc_check_set()) Completed RPC req@000000006e2391ef pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_00:lustre-MDT0000-mdtlov_UUID:3353:1862456998049856:0@lo:13:osp-pre-0-0.0 00000100:00100000:0.0:1776178215.779109:0:3353:0:(client.c:2215:ptlrpc_check_set()) Completed RPC req@00000000aba13576 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_00:lustre-MDT0000-mdtlov_UUID:3353:1862456998049792:0@lo:41:osp-pre-1-0.0 00000100:00100000:3.0:1776178215.781830:0:3354:0:(client.c:1733:ptlrpc_send_new_req()) Sending RPC req@00000000c6606435 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_01:lustre-MDT0000-lwp-OST0001_UUID:3354:1862456998050112:0@lo:400:kworker/u8:0.0 00000100:00100000:3.0:1776178215.781901:0:3354:0:(events.c:361:request_in_callback()) peer: 12345-0@lo (source: 12345-0@lo) 00000100:00100000:3.0:1776178215.781929:0:3354:0:(client.c:1733:ptlrpc_send_new_req()) Sending RPC req@00000000761de6a8 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_01:lustre-MDT0000-lwp-MDT0001_UUID:3354:1862456998050368:0@lo:400:kworker/u8:0.0 00000100:00100000:3.0:1776178215.781951:0:3354:0:(events.c:361:request_in_callback()) peer: 12345-0@lo (source: 12345-0@lo) 00000100:00100000:3.0:1776178215.781973:0:3354:0:(ptlrpcd.c:415:ptlrpcd_check()) transfer 2 async RPCs [0->1] 00000100:00100000:3.0:1776178215.781978:0:3354:0:(client.c:1733:ptlrpc_send_new_req()) Sending RPC req@00000000a0c03f2e pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_01:lustre-MDT0001-lwp-OST0000_UUID:3354:1862456998050048:0@lo:400:kworker/u8:0.0 00000100:00100000:3.0:1776178215.781998:0:3354:0:(events.c:361:request_in_callback()) peer: 12345-0@lo (source: 12345-0@lo) 00000100:00100000:3.0:1776178215.782014:0:3354:0:(client.c:1733:ptlrpc_send_new_req()) Sending RPC req@00000000587d8120 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_01:lustre-MDT0001-mdtlov_UUID:3354:1862456998050304:0@lo:400:kworker/u8:0.0 00000100:00100000:3.0:1776178215.782038:0:3354:0:(events.c:361:request_in_callback()) peer: 12345-0@lo (source: 12345-0@lo) 00000100:00100000:3.0:1776178215.782107:0:6255:0:(service.c:2152:ptlrpc_server_handle_req_in()) got req x1862456998050112 00000100:00100000:3.0:1776178215.782152:0:6255:0:(service.c:2152:ptlrpc_server_handle_req_in()) got req x1862456998050368 00000100:00100000:3.0:1776178215.782166:0:6255:0:(service.c:2152:ptlrpc_server_handle_req_in()) got req x1862456998050048 00000100:00100000:3.0:1776178215.782180:0:6255:0:(nrs_fifo.c:179:nrs_fifo_req_get()) NRS start fifo request from 12345-0@lo, seq: 925 00000100:00100000:3.0:1776178215.782186:0:6255:0:(service.c:2324:ptlrpc_server_handle_request()) Handling RPC req@000000003254dbe4 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_000:lustre-MDT0000-lwp-OST0001_UUID+5:3354:x1862456998050112:12345-0@lo:400:kworker/u8:0.0 00010000:00000080:3.0:1776178215.782291:0:6255:0:(ldlm_lib.c:3181:target_committed_to_req()) @@@ not sending last_committed update (1/0) req@000000003254dbe4 x1862456998050112/t0(0) o400->lustre-MDT0000-lwp-OST0001_UUID@0@lo:747/0 lens 224/224 e 0 to 0 dl 1776178262 ref 1 fl Interpret:H/0/0 rc 0/0 job:'kworker/u8:0.0' 00000100:00100000:3.0:1776178215.782335:0:6255:0:(service.c:2372:ptlrpc_server_handle_request()) Handled RPC req@000000003254dbe4 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_000:lustre-MDT0000-lwp-OST0001_UUID+5:3354:x1862456998050112:12345-0@lo:400:kworker/u8:0.0 Request processed in 149us (435us total) trans 0 rc 0/0 00000100:00100000:3.0:1776178215.782343:0:6255:0:(nrs_fifo.c:241:nrs_fifo_req_stop()) NRS stop fifo request from 12345-0@lo, seq: 925 00000100:00100000:3.0:1776178215.782365:0:6255:0:(nrs_fifo.c:179:nrs_fifo_req_get()) NRS start fifo request from 12345-0@lo, seq: 926 00000100:00100000:3.0:1776178215.782368:0:6255:0:(service.c:2324:ptlrpc_server_handle_request()) Handling RPC req@00000000af9df307 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_000:lustre-MDT0000-lwp-MDT0001_UUID+5:3354:x1862456998050368:12345-0@lo:400:kworker/u8:0.0 00010000:00000080:3.0:1776178215.782379:0:6255:0:(ldlm_lib.c:3181:target_committed_to_req()) @@@ not sending last_committed update (1/0) req@00000000af9df307 x1862456998050368/t0(0) o400->lustre-MDT0000-lwp-MDT0001_UUID@0@lo:747/0 lens 224/224 e 0 to 0 dl 1776178262 ref 1 fl Interpret:H/0/0 rc 0/0 job:'kworker/u8:0.0' 00000100:00100000:3.0:1776178215.782402:0:6255:0:(service.c:2372:ptlrpc_server_handle_request()) Handled RPC req@00000000af9df307 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_000:lustre-MDT0000-lwp-MDT0001_UUID+5:3354:x1862456998050368:12345-0@lo:400:kworker/u8:0.0 Request processed in 33us (451us total) trans 0 rc 0/0 00000100:00100000:3.0:1776178215.782407:0:6255:0:(nrs_fifo.c:241:nrs_fifo_req_stop()) NRS stop fifo request from 12345-0@lo, seq: 926 00000100:00100000:3.0:1776178215.782420:0:6255:0:(nrs_fifo.c:179:nrs_fifo_req_get()) NRS start fifo request from 12345-0@lo, seq: 927 00000100:00100000:3.0:1776178215.782423:0:6255:0:(service.c:2324:ptlrpc_server_handle_request()) Handling RPC req@0000000093c329c5 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_000:lustre-MDT0001-lwp-OST0000_UUID+5:3354:x1862456998050048:12345-0@lo:400:kworker/u8:0.0 00000100:00100000:3.0:1776178215.782447:0:6255:0:(service.c:2372:ptlrpc_server_handle_request()) Handled RPC req@0000000093c329c5 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_000:lustre-MDT0001-lwp-OST0000_UUID+5:3354:x1862456998050048:12345-0@lo:400:kworker/u8:0.0 Request processed in 23us (449us total) trans 0 rc 0/0 00000100:00100000:3.0:1776178215.782452:0:6255:0:(nrs_fifo.c:241:nrs_fifo_req_stop()) NRS stop fifo request from 12345-0@lo, seq: 927 00000100:00100000:3.0:1776178215.782474:0:14440:0:(service.c:2152:ptlrpc_server_handle_req_in()) got req x1862456998050304 00000100:00100000:3.0:1776178215.782548:0:14440:0:(nrs_fifo.c:179:nrs_fifo_req_get()) NRS start fifo request from 12345-0@lo, seq: 521 00000100:00100000:3.0:1776178215.782552:0:14440:0:(service.c:2324:ptlrpc_server_handle_request()) Handling RPC req@0000000097804cb0 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_004:lustre-MDT0001-mdtlov_UUID+5:3354:x1862456998050304:12345-0@lo:400:kworker/u8:0.0 00000100:00100000:3.0:1776178215.782583:0:14440:0:(service.c:2372:ptlrpc_server_handle_request()) Handled RPC req@0000000097804cb0 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_004:lustre-MDT0001-mdtlov_UUID+5:3354:x1862456998050304:12345-0@lo:400:kworker/u8:0.0 Request processed in 29us (545us total) trans 0 rc 0/0 00000100:00100000:3.0:1776178215.782588:0:14440:0:(nrs_fifo.c:241:nrs_fifo_req_stop()) NRS stop fifo request from 12345-0@lo, seq: 521 00000100:00100000:3.0:1776178215.782630:0:3355:0:(client.c:1733:ptlrpc_send_new_req()) Sending RPC req@00000000a5decd95 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_02:ac61f839-0e8e-4cac-9c59-08bcfbbad330:3355:1862456998049920:0@lo:400:kworker/u8:0.0 00000100:00100000:3.0:1776178215.782665:0:3355:0:(events.c:361:request_in_callback()) peer: 12345-0@lo (source: 12345-0@lo) 00000100:00100000:3.0:1776178215.782679:0:3355:0:(client.c:1733:ptlrpc_send_new_req()) Sending RPC req@000000006c6f4a36 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_02:lustre-MDT0001-lwp-OST0001_UUID:3355:1862456998050176:0@lo:400:kworker/u8:0.0 00000100:00100000:3.0:1776178215.782699:0:3355:0:(events.c:361:request_in_callback()) peer: 12345-0@lo (source: 12345-0@lo) 00000100:00100000:3.0:1776178215.782713:0:3355:0:(client.c:1733:ptlrpc_send_new_req()) Sending RPC req@0000000033aa56cf pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_02:lustre-MDT0000-lwp-MDT0000_UUID:3355:1862456998050432:0@lo:400:kworker/u8:0.0 00000100:00100000:3.0:1776178215.782733:0:3355:0:(events.c:361:request_in_callback()) peer: 12345-0@lo (source: 12345-0@lo) 00000100:00100000:3.0:1776178215.782744:0:3355:0:(ptlrpcd.c:415:ptlrpcd_check()) transfer 2 async RPCs [3->2] 00000100:00100000:3.0:1776178215.782750:0:3355:0:(client.c:1733:ptlrpc_send_new_req()) Sending RPC req@000000006294d286 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_02:lustre-MDT0000-lwp-OST0000_UUID:3355:1862456998049984:0@lo:400:kworker/u8:0.0 00000100:00100000:3.0:1776178215.782770:0:3355:0:(events.c:361:request_in_callback()) peer: 12345-0@lo (source: 12345-0@lo) 00000100:00100000:3.0:1776178215.782778:0:3355:0:(client.c:1733:ptlrpc_send_new_req()) Sending RPC req@00000000cbfc3bb8 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_02:lustre-MDT0001-mdtlov_UUID:3355:1862456998050240:0@lo:400:kworker/u8:0.0 00000100:00100000:3.0:1776178215.782794:0:3355:0:(events.c:361:request_in_callback()) peer: 12345-0@lo (source: 12345-0@lo) 00000100:00100000:3.0:1776178215.782814:0:20759:0:(service.c:2152:ptlrpc_server_handle_req_in()) got req x1862456998050240 00000100:00100000:3.0:1776178215.782837:0:20759:0:(nrs_fifo.c:179:nrs_fifo_req_get()) NRS start fifo request from 12345-0@lo, seq: 522 00000100:00100000:3.0:1776178215.782840:0:20759:0:(service.c:2324:ptlrpc_server_handle_request()) Handling RPC req@0000000078d5b0b4 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_010:lustre-MDT0001-mdtlov_UUID+5:3355:x1862456998050240:12345-0@lo:400:kworker/u8:0.0 00000100:00100000:3.0:1776178215.782873:0:20759:0:(service.c:2372:ptlrpc_server_handle_request()) Handled RPC req@0000000078d5b0b4 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_010:lustre-MDT0001-mdtlov_UUID+5:3355:x1862456998050240:12345-0@lo:400:kworker/u8:0.0 Request processed in 31us (78us total) trans 0 rc 0/0 00000100:00100000:3.0:1776178215.782879:0:20759:0:(nrs_fifo.c:241:nrs_fifo_req_stop()) NRS stop fifo request from 12345-0@lo, seq: 522 00000100:00100000:3.0:1776178215.782918:0:3355:0:(client.c:2868:ptlrpc_free_committed()) lustre-OST0000-osc-MDT0001: committing for last_committed 4294969361 gen 2 00000004:00080000:3.0:1776178215.782924:0:3355:0:(osp_sync.c:508:osp_sync_request_commit_cb()) commit req 00000000f433e404, transno 4294969359 00000004:00080000:3.0:1776178215.782932:0:3355:0:(osp_sync.c:508:osp_sync_request_commit_cb()) commit req 0000000044f82452, transno 4294969360 00000004:00080000:3.0:1776178215.782935:0:3355:0:(osp_sync.c:508:osp_sync_request_commit_cb()) commit req 0000000028ade006, transno 4294969361 00000100:00100000:3.0:1776178215.782941:0:3355:0:(client.c:2215:ptlrpc_check_set()) Completed RPC req@00000000cbfc3bb8 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_02:lustre-MDT0001-mdtlov_UUID:3355:1862456998050240:0@lo:400:kworker/u8:0.0 00000100:00100000:3.0:1776178215.782979:0:3354:0:(client.c:2215:ptlrpc_check_set()) Completed RPC req@00000000a0c03f2e pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_01:lustre-MDT0001-lwp-OST0000_UUID:3354:1862456998050048:0@lo:400:kworker/u8:0.0 00000100:00100000:3.0:1776178215.782987:0:3354:0:(client.c:2868:ptlrpc_free_committed()) lustre-OST0001-osc-MDT0001: committing for last_committed 4294969335 gen 2 00000004:00080000:3.0:1776178215.782990:0:3354:0:(osp_sync.c:508:osp_sync_request_commit_cb()) commit req 000000009e7a8bb5, transno 4294969332 00000004:00080000:3.0:1776178215.783000:0:3354:0:(osp_sync.c:508:osp_sync_request_commit_cb()) commit req 00000000dd1281fe, transno 4294969333 00000004:00080000:3.0:1776178215.783003:0:3354:0:(osp_sync.c:508:osp_sync_request_commit_cb()) commit req 00000000f95e9da5, transno 4294969334 00000004:00080000:3.0:1776178215.783006:0:3354:0:(osp_sync.c:508:osp_sync_request_commit_cb()) commit req 0000000072865dff, transno 4294969335 00000100:00100000:3.0:1776178215.783010:0:3354:0:(client.c:2215:ptlrpc_check_set()) Completed RPC req@00000000587d8120 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_01:lustre-MDT0001-mdtlov_UUID:3354:1862456998050304:0@lo:400:kworker/u8:0.0 00000100:00100000:3.0:1776178215.783020:0:3354:0:(client.c:2215:ptlrpc_check_set()) Completed RPC req@00000000c6606435 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_01:lustre-MDT0000-lwp-OST0001_UUID:3354:1862456998050112:0@lo:400:kworker/u8:0.0 00000100:00100000:3.0:1776178215.783030:0:3354:0:(client.c:2215:ptlrpc_check_set()) Completed RPC req@00000000761de6a8 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_01:lustre-MDT0000-lwp-MDT0001_UUID:3354:1862456998050368:0@lo:400:kworker/u8:0.0 00000100:00100000:3.0:1776178215.783056:0:6255:0:(service.c:2152:ptlrpc_server_handle_req_in()) got req x1862456998050176 00000100:00100000:3.0:1776178215.783100:0:6255:0:(service.c:2152:ptlrpc_server_handle_req_in()) got req x1862456998050432 00000100:00100000:3.0:1776178215.783110:0:6255:0:(service.c:2152:ptlrpc_server_handle_req_in()) got req x1862456998049984 00000100:00100000:3.0:1776178215.783119:0:6255:0:(nrs_fifo.c:179:nrs_fifo_req_get()) NRS start fifo request from 12345-0@lo, seq: 928 00000100:00100000:3.0:1776178215.783123:0:6255:0:(service.c:2324:ptlrpc_server_handle_request()) Handling RPC req@000000008481f676 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_000:lustre-MDT0001-lwp-OST0001_UUID+5:3355:x1862456998050176:12345-0@lo:400:kworker/u8:0.0 00000100:00100000:3.0:1776178215.783154:0:6255:0:(service.c:2372:ptlrpc_server_handle_request()) Handled RPC req@000000008481f676 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_000:lustre-MDT0001-lwp-OST0001_UUID+5:3355:x1862456998050176:12345-0@lo:400:kworker/u8:0.0 Request processed in 30us (454us total) trans 0 rc 0/0 00000100:00100000:3.0:1776178215.783160:0:6255:0:(nrs_fifo.c:241:nrs_fifo_req_stop()) NRS stop fifo request from 12345-0@lo, seq: 928 00000100:00100000:3.0:1776178215.783170:0:6255:0:(nrs_fifo.c:179:nrs_fifo_req_get()) NRS start fifo request from 12345-0@lo, seq: 929 00000100:00100000:3.0:1776178215.783172:0:6255:0:(service.c:2324:ptlrpc_server_handle_request()) Handling RPC req@00000000061c55b0 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_000:lustre-MDT0000-lwp-MDT0000_UUID+5:3355:x1862456998050432:12345-0@lo:400:kworker/u8:0.0 00010000:00000080:3.0:1776178215.783183:0:6255:0:(ldlm_lib.c:3181:target_committed_to_req()) @@@ not sending last_committed update (1/0) req@00000000061c55b0 x1862456998050432/t0(0) o400->lustre-MDT0000-lwp-MDT0000_UUID@0@lo:747/0 lens 224/224 e 0 to 0 dl 1776178262 ref 1 fl Interpret:H/0/0 rc 0/0 job:'kworker/u8:0.0' 00000100:00100000:3.0:1776178215.783205:0:6255:0:(service.c:2372:ptlrpc_server_handle_request()) Handled RPC req@00000000061c55b0 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_000:lustre-MDT0000-lwp-MDT0000_UUID+5:3355:x1862456998050432:12345-0@lo:400:kworker/u8:0.0 Request processed in 32us (472us total) trans 0 rc 0/0 00000100:00100000:0.0:1776178215.783631:0:6256:0:(nrs_fifo.c:179:nrs_fifo_req_get()) NRS start fifo request from 12345-0@lo, seq: 930 00000100:00100000:0.0:1776178215.783636:0:6256:0:(service.c:2324:ptlrpc_server_handle_request()) Handling RPC req@0000000088a775ba pname:cluuid+ref:pid:xid:nid:opc:job mdt00_001:lustre-MDT0000-lwp-OST0000_UUID+5:3355:x1862456998049984:12345-0@lo:400:kworker/u8:0.0 00010000:00000080:0.0:1776178215.783652:0:6256:0:(ldlm_lib.c:3181:target_committed_to_req()) @@@ not sending last_committed update (1/0) req@0000000088a775ba x1862456998049984/t0(0) o400->lustre-MDT0000-lwp-OST0000_UUID@0@lo:747/0 lens 224/224 e 0 to 0 dl 1776178262 ref 1 fl Interpret:H/0/0 rc 0/0 job:'kworker/u8:0.0' 00000100:00100000:0.0:1776178215.783685:0:6256:0:(service.c:2372:ptlrpc_server_handle_request()) Handled RPC req@0000000088a775ba pname:cluuid+ref:pid:xid:nid:opc:job mdt00_001:lustre-MDT0000-lwp-OST0000_UUID+5:3355:x1862456998049984:12345-0@lo:400:kworker/u8:0.0 Request processed in 48us (914us total) trans 0 rc 0/0 00000100:00100000:0.0:1776178215.783690:0:6256:0:(nrs_fifo.c:241:nrs_fifo_req_stop()) NRS stop fifo request from 12345-0@lo, seq: 930 00000040:00100000:0.0:1776178215.783747:0:18106:0:(llog.c:235:llog_cancel_arr_rec()) Canceling 4 records, first 1008 in log [0x1:0x12:0x0] 00000004:00080000:0.0:1776178215.783846:0:18106:0:(osp_sync.c:1124:osp_sync_process_committed()) lustre-OST0001-osc-MDT0001: 0 changes, 0 in progress, 0 in flight, 4 done 00000100:00100000:2.0:1776178215.784168:0:20140:0:(service.c:2152:ptlrpc_server_handle_req_in()) got req x1862456998049920 00000100:00100000:2.0:1776178215.784204:0:20140:0:(nrs_fifo.c:179:nrs_fifo_req_get()) NRS start fifo request from 12345-0@lo, seq: 161 00000100:00100000:2.0:1776178215.784219:0:20140:0:(service.c:2324:ptlrpc_server_handle_request()) Handling RPC req@0000000048947fd8 pname:cluuid+ref:pid:xid:nid:opc:job ll_mgs_0002:ac61f839-0e8e-4cac-9c59-08bcfbbad330+23:3355:x1862456998049920:12345-0@lo:400:kworker/u8:0.0 00000100:00100000:2.0:1776178215.784258:0:20140:0:(service.c:2372:ptlrpc_server_handle_request()) Handled RPC req@0000000048947fd8 pname:cluuid+ref:pid:xid:nid:opc:job ll_mgs_0002:ac61f839-0e8e-4cac-9c59-08bcfbbad330+23:3355:x1862456998049920:12345-0@lo:400:kworker/u8:0.0 Request processed in 49us (1593us total) trans 0 rc 0/0 00000100:00100000:2.0:1776178215.784264:0:20140:0:(nrs_fifo.c:241:nrs_fifo_req_stop()) NRS stop fifo request from 12345-0@lo, seq: 161 00000040:00100000:2.0:1776178215.784317:0:18104:0:(llog.c:235:llog_cancel_arr_rec()) Canceling 3 records, first 1007 in log [0x1:0x10:0x0] 00000004:00080000:2.0:1776178215.784364:0:18104:0:(osp_sync.c:1124:osp_sync_process_committed()) lustre-OST0000-osc-MDT0001: 0 changes, 0 in progress, 0 in flight, 3 done 00000100:00100000:3.0:1776178215.784869:0:6255:0:(nrs_fifo.c:241:nrs_fifo_req_stop()) NRS stop fifo request from 12345-0@lo, seq: 929 00000100:00100000:3.0:1776178215.784925:0:3355:0:(client.c:2215:ptlrpc_check_set()) Completed RPC req@000000006294d286 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_02:lustre-MDT0000-lwp-OST0000_UUID:3355:1862456998049984:0@lo:400:kworker/u8:0.0 00000100:00100000:3.0:1776178215.784936:0:3355:0:(client.c:2215:ptlrpc_check_set()) Completed RPC req@00000000a5decd95 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_02:ac61f839-0e8e-4cac-9c59-08bcfbbad330:3355:1862456998049920:0@lo:400:kworker/u8:0.0 00000100:00100000:3.0:1776178215.784948:0:3355:0:(client.c:2215:ptlrpc_check_set()) Completed RPC req@000000006c6f4a36 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_02:lustre-MDT0001-lwp-OST0001_UUID:3355:1862456998050176:0@lo:400:kworker/u8:0.0 00000100:00100000:3.0:1776178215.784959:0:3355:0:(client.c:2215:ptlrpc_check_set()) Completed RPC req@0000000033aa56cf pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_02:lustre-MDT0000-lwp-MDT0000_UUID:3355:1862456998050432:0@lo:400:kworker/u8:0.0 00010000:00010000:2.0:1776178216.420496:0:9225:0:(ldlm_lockd.c:2792:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:0.0:1776178217.312306:0:3355:0:(ptlrpcd.c:415:ptlrpcd_check()) transfer 1 async RPCs [3->2] 00000100:00100000:0.0:1776178217.312334:0:3355:0:(client.c:1733:ptlrpc_send_new_req()) Sending RPC req@0000000033aa56cf pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_02:lustre-MDT0001-mdtlov_UUID:3355:1862456998050496:0@lo:13:osp-pre-1-1.0 00000100:00100000:0.0:1776178217.312423:0:3355:0:(events.c:361:request_in_callback()) peer: 12345-0@lo (source: 12345-0@lo) 00000100:00100000:1.0:1776178217.312559:0:9554:0:(service.c:2152:ptlrpc_server_handle_req_in()) got req x1862456998050496 00000100:00100000:1.0:1776178217.312618:0:9554:0:(nrs_fifo.c:179:nrs_fifo_req_get()) NRS start fifo request from 12345-0@lo, seq: 491 00000100:00100000:1.0:1776178217.312625:0:9554:0:(service.c:2324:ptlrpc_server_handle_request()) Handling RPC req@00000000f7d56588 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_create00:lustre-MDT0001-mdtlov_UUID+5:3355:x1862456998050496:12345-0@lo:13:osp-pre-1-1.0 00002000:00000024:1.0:1776178217.312660:0:9554:0:(ofd_obd.c:740:ofd_statfs()) blocks cached 229376 granted 491746368 pending 0 free 3923689472 avail 3692167168 00000100:00100000:1.0:1776178217.312723:0:9554:0:(service.c:2372:ptlrpc_server_handle_request()) Handled RPC req@00000000f7d56588 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_create00:lustre-MDT0001-mdtlov_UUID+5:3355:x1862456998050496:12345-0@lo:13:osp-pre-1-1.0 Request processed in 97us (301us total) trans 0 rc 0/0 00000100:00100000:1.0:1776178217.312732:0:9554:0:(nrs_fifo.c:241:nrs_fifo_req_stop()) NRS stop fifo request from 12345-0@lo, seq: 491 00000100:00100000:3.0:1776178217.312837:0:3355:0:(client.c:2215:ptlrpc_check_set()) Completed RPC req@0000000033aa56cf pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_02:lustre-MDT0001-mdtlov_UUID:3355:1862456998050496:0@lo:13:osp-pre-1-1.0 00010000:00010000:1.0:1776178217.442539:0:6245:0:(ldlm_lockd.c:2792:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:0.0:1776178217.825184:0:3354:0:(ptlrpcd.c:415:ptlrpcd_check()) transfer 1 async RPCs [0->1] 00000100:00100000:0.0:1776178217.825207:0:3354:0:(client.c:1733:ptlrpc_send_new_req()) Sending RPC req@000000009e7a8bb5 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_01:lustre-MDT0001-mdtlov_UUID:3354:1862456998050560:0@lo:13:osp-pre-0-1.0 00000100:00100000:0.0:1776178217.825274:0:3354:0:(events.c:361:request_in_callback()) peer: 12345-0@lo (source: 12345-0@lo) 00000100:00100000:0.0:1776178217.825323:0:8231:0:(service.c:2152:ptlrpc_server_handle_req_in()) got req x1862456998050560 00000100:00100000:0.0:1776178217.825366:0:8231:0:(nrs_fifo.c:179:nrs_fifo_req_get()) NRS start fifo request from 12345-0@lo, seq: 492 00000100:00100000:0.0:1776178217.825371:0:8231:0:(service.c:2324:ptlrpc_server_handle_request()) Handling RPC req@00000000dd1281fe pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_create00:lustre-MDT0001-mdtlov_UUID+5:3354:x1862456998050560:12345-0@lo:13:osp-pre-0-1.0 00002000:00000024:0.0:1776178217.825399:0:8231:0:(ofd_obd.c:740:ofd_statfs()) blocks cached 315392 granted 484883776 pending 0 free 3923689472 avail 3692167168 00000100:00100000:0.0:1776178217.825434:0:8231:0:(service.c:2372:ptlrpc_server_handle_request()) Handled RPC req@00000000dd1281fe pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_create00:lustre-MDT0001-mdtlov_UUID+5:3354:x1862456998050560:12345-0@lo:13:osp-pre-0-1.0 Request processed in 63us (161us total) trans 0 rc 0/0 00000100:00100000:0.0:1776178217.825451:0:8231:0:(nrs_fifo.c:241:nrs_fifo_req_stop()) NRS stop fifo request from 12345-0@lo, seq: 492 00000100:00100000:0.0:1776178217.825493:0:3354:0:(client.c:2215:ptlrpc_check_set()) Completed RPC req@000000009e7a8bb5 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_01:lustre-MDT0001-mdtlov_UUID:3354:1862456998050560:0@lo:13:osp-pre-0-1.0 00010000:00010000:1.0:1776178218.464208:0:9224:0:(ldlm_lockd.c:2792:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:3.0:1776178219.298492:0:3346:0:(events.c:361:request_in_callback()) peer: 12345-192.168.204.3@tcp (source: 12345-192.168.204.3@tcp) 00000100:00100000:3.0:1776178219.298684:0:3346:0:(events.c:361:request_in_callback()) peer: 12345-192.168.204.3@tcp (source: 12345-192.168.204.3@tcp) 00000100:00100000:3.0:1776178219.298801:0:3346:0:(events.c:361:request_in_callback()) peer: 12345-192.168.204.3@tcp (source: 12345-192.168.204.3@tcp) 00000100:00100000:3.0:1776178219.298839:0:3346:0:(events.c:361:request_in_callback()) peer: 12345-192.168.204.3@tcp (source: 12345-192.168.204.3@tcp) 00000100:00100000:3.0:1776178219.298883:0:3346:0:(events.c:361:request_in_callback()) peer: 12345-192.168.204.3@tcp (source: 12345-192.168.204.3@tcp) 00000100:00100000:2.0:1776178219.298930:0:20137:0:(service.c:2152:ptlrpc_server_handle_req_in()) got req x1862456979108096 00000100:00100000:3.0:1776178219.298937:0:20759:0:(service.c:2152:ptlrpc_server_handle_req_in()) got req x1862456979108352 00000100:00100000:3.0:1776178219.299022:0:20759:0:(service.c:2152:ptlrpc_server_handle_req_in()) got req x1862456979108288 00000100:00100000:2.0:1776178219.299027:0:20137:0:(nrs_fifo.c:179:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.204.3@tcp, seq: 162 00000100:00100000:2.0:1776178219.299038:0:20137:0:(service.c:2324:ptlrpc_server_handle_request()) Handling RPC req@00000000ed9d6990 pname:cluuid+ref:pid:xid:nid:opc:job ll_mgs_0000:5b5c66b1-1a2f-4651-8548-0c75b9a38a94+9:2273:x1862456979108096:12345-192.168.204.3@tcp:400:kworker/u8:3.0 00000100:00100000:3.0:1776178219.299049:0:20759:0:(nrs_fifo.c:179:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.204.3@tcp, seq: 523 00000100:00100000:3.0:1776178219.299057:0:20759:0:(service.c:2324:ptlrpc_server_handle_request()) Handling RPC req@000000006c6f4a36 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_010:b50963bd-8eba-4432-9a54-8888d00e3fa1+5:2273:x1862456979108352:12345-192.168.204.3@tcp:400:kworker/u8:3.0 00000100:00100000:2.0:1776178219.299172:0:20137:0:(service.c:2372:ptlrpc_server_handle_request()) Handled RPC req@00000000ed9d6990 pname:cluuid+ref:pid:xid:nid:opc:job ll_mgs_0000:5b5c66b1-1a2f-4651-8548-0c75b9a38a94+9:2273:x1862456979108096:12345-192.168.204.3@tcp:400:kworker/u8:3.0 Request processed in 133us (700us total) trans 0 rc 0/0 00000100:00100000:2.0:1776178219.299186:0:20137:0:(nrs_fifo.c:241:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.204.3@tcp, seq: 162 00000100:00100000:2.0:1776178219.299254:0:8250:0:(service.c:2152:ptlrpc_server_handle_req_in()) got req x1862456979108224 00000100:00100000:3.0:1776178219.299268:0:20759:0:(service.c:2372:ptlrpc_server_handle_request()) Handled RPC req@000000006c6f4a36 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_010:b50963bd-8eba-4432-9a54-8888d00e3fa1+5:2273:x1862456979108352:12345-192.168.204.3@tcp:400:kworker/u8:3.0 Request processed in 207us (464us total) trans 0 rc 0/0 00000100:00100000:3.0:1776178219.299280:0:20759:0:(nrs_fifo.c:241:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.204.3@tcp, seq: 523 00000100:00100000:2.0:1776178219.299297:0:8250:0:(service.c:2152:ptlrpc_server_handle_req_in()) got req x1862456979108160 00000100:00100000:3.0:1776178219.299310:0:20759:0:(nrs_fifo.c:179:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.204.3@tcp, seq: 524 00000100:00100000:3.0:1776178219.299319:0:20759:0:(service.c:2324:ptlrpc_server_handle_request()) Handling RPC req@00000000761de6a8 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_010:b50963bd-8eba-4432-9a54-8888d00e3fa1+5:2273:x1862456979108288:12345-192.168.204.3@tcp:400:kworker/u8:3.0 00000100:00100000:2.0:1776178219.299335:0:8250:0:(nrs_fifo.c:179:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.204.3@tcp, seq: 931 00000100:00100000:2.0:1776178219.299343:0:8250:0:(service.c:2324:ptlrpc_server_handle_request()) Handling RPC req@0000000033aa56cf pname:cluuid+ref:pid:xid:nid:opc:job mdt00_003:b50963bd-8eba-4432-9a54-8888d00e3fa1+6:2273:x1862456979108224:12345-192.168.204.3@tcp:400:kworker/u8:3.0 00000100:00100000:3.0:1776178219.299356:0:20759:0:(service.c:2372:ptlrpc_server_handle_request()) Handled RPC req@00000000761de6a8 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_010:b50963bd-8eba-4432-9a54-8888d00e3fa1+5:2273:x1862456979108288:12345-192.168.204.3@tcp:400:kworker/u8:3.0 Request processed in 37us (473us total) trans 0 rc 0/0 00000100:00100000:3.0:1776178219.299369:0:20759:0:(nrs_fifo.c:241:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.204.3@tcp, seq: 524 00000100:00100000:2.0:1776178219.299391:0:8250:0:(service.c:2372:ptlrpc_server_handle_request()) Handled RPC req@0000000033aa56cf pname:cluuid+ref:pid:xid:nid:opc:job mdt00_003:b50963bd-8eba-4432-9a54-8888d00e3fa1+6:2273:x1862456979108224:12345-192.168.204.3@tcp:400:kworker/u8:3.0 Request processed in 47us (716us total) trans 0 rc 0/0 00000100:00100000:2.0:1776178219.299402:0:8250:0:(nrs_fifo.c:241:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.204.3@tcp, seq: 931 00000100:00100000:2.0:1776178219.299433:0:8250:0:(nrs_fifo.c:179:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.204.3@tcp, seq: 932 00000100:00100000:2.0:1776178219.299438:0:8250:0:(service.c:2324:ptlrpc_server_handle_request()) Handling RPC req@000000006294d286 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_003:b50963bd-8eba-4432-9a54-8888d00e3fa1+18:2273:x1862456979108160:12345-192.168.204.3@tcp:400:kworker/u8:3.0 00010000:00000080:2.0:1776178219.299481:0:8250:0:(ldlm_lib.c:3181:target_committed_to_req()) @@@ not sending last_committed update (1/0) req@000000006294d286 x1862456979108160/t0(0) o400->b50963bd-8eba-4432-9a54-8888d00e3fa1@192.168.204.3@tcp:751/0 lens 224/224 e 0 to 0 dl 1776178266 ref 1 fl Interpret:H/0/0 rc 0/0 job:'kworker/u8:3.0' 00000100:00100000:2.0:1776178219.299515:0:8250:0:(service.c:2372:ptlrpc_server_handle_request()) Handled RPC req@000000006294d286 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_003:b50963bd-8eba-4432-9a54-8888d00e3fa1+18:2273:x1862456979108160:12345-192.168.204.3@tcp:400:kworker/u8:3.0 Request processed in 76us (677us total) trans 0 rc 0/0 00000100:00100000:2.0:1776178219.299523:0:8250:0:(nrs_fifo.c:241:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.204.3@tcp, seq: 932 00010000:00010000:1.0:1776178219.489330:0:6244:0:(ldlm_lockd.c:2792:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000001:00080000:3.0:1776178219.875277:0:20134:0:(tgt_lastrcvd.c:913:tgt_cb_last_committed()) lustre-MDT0000: transno 12884901974 is committed 00010000:00010000:3.0:1776178219.875515:0:3340:0:(ldlm_lock.c:820:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(COS) ns: mdt-lustre-MDT0000_UUID lock: 00000000e4217a5c/0xa40083ab9bf539af lrc: 3/0,1 mode: COS/COS res: [0x200000007:0x1:0x0].0x2c847881 bits 0x2/0x0 rrc: 3 type: IBT gid 0 flags: 0x50210000000000 nid: local remote: 0x0 expref: -99 pid: 6257 timeout: 0 lvb_type: 0 00010000:00010000:3.0:1776178219.875538:0:3340:0:(ldlm_lock.c:879:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: mdt-lustre-MDT0000_UUID lock: 00000000e4217a5c/0xa40083ab9bf539af lrc: 2/0,0 mode: COS/COS res: [0x200000007:0x1:0x0].0x2c847881 bits 0x2/0x0 rrc: 3 type: IBT gid 0 flags: 0x50210400000000 nid: local remote: 0x0 expref: -99 pid: 6257 timeout: 0 lvb_type: 0 00010000:00010000:3.0:1776178219.875547:0:3340:0:(ldlm_lockd.c:1879:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: mdt-lustre-MDT0000_UUID lock: 00000000e4217a5c/0xa40083ab9bf539af lrc: 3/0,0 mode: COS/COS res: [0x200000007:0x1:0x0].0x2c847881 bits 0x2/0x0 rrc: 3 type: IBT gid 0 flags: 0x40210400000000 nid: local remote: 0x0 expref: -99 pid: 6257 timeout: 0 lvb_type: 0 00010000:00010000:3.0:1776178219.875629:0:3340:0:(ldlm_lockd.c:1893:ldlm_handle_bl_callback()) Lock 00000000e4217a5c already unused, calling callback (0000000079e78348) 00010000:00010000:3.0:1776178219.875635:0:3340:0:(ldlm_request.c:364:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: mdt-lustre-MDT0000_UUID lock: 00000000e4217a5c/0xa40083ab9bf539af lrc: 3/0,0 mode: COS/COS res: [0x200000007:0x1:0x0].0x2c847881 bits 0x2/0x0 rrc: 3 type: IBT gid 0 flags: 0x40210400000000 nid: local remote: 0x0 expref: -99 pid: 6257 timeout: 0 lvb_type: 0 00010000:00010000:3.0:1776178219.875645:0:3340:0:(ldlm_request.c:1257:ldlm_cli_cancel_local()) ### server-side local cancel ns: mdt-lustre-MDT0000_UUID lock: 00000000e4217a5c/0xa40083ab9bf539af lrc: 4/0,0 mode: COS/COS res: [0x200000007:0x1:0x0].0x2c847881 bits 0x2/0x0 rrc: 3 type: IBT gid 0 flags: 0x40218400000000 nid: local remote: 0x0 expref: -99 pid: 6257 timeout: 0 lvb_type: 0 00010000:00010000:3.0:1776178219.875681:0:3340:0:(ldlm_inodebits.c:94:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x200000007:0x1:0x0].0x2c847881 (00000000ccda0888) 00010000:00010000:3.0:1776178219.875684:0:3340:0:(ldlm_inodebits.c:96:ldlm_reprocess_inodebits_queue()) Hint 2 00010000:00010000:3.0:1776178219.875690:0:3340:0:(ldlm_lockd.c:1903:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: mdt-lustre-MDT0000_UUID lock: 00000000e4217a5c/0xa40083ab9bf539af lrc: 2/0,0 mode: --/COS res: [0x200000007:0x1:0x0].0x2c847881 bits 0x2/0x0 rrc: 3 type: IBT gid 0 flags: 0x44a19400000000 nid: local remote: 0x0 expref: -99 pid: 6257 timeout: 0 lvb_type: 0 00010000:00010000:3.0:1776178219.875703:0:3340:0:(ldlm_lock.c:216:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: mdt-lustre-MDT0000_UUID lock: 00000000e4217a5c/0xa40083ab9bf539af lrc: 0/0,0 mode: --/COS res: [0x200000007:0x1:0x0].0x2c847881 bits 0x2/0x0 rrc: 3 type: IBT gid 0 flags: 0x44a19400000000 nid: local remote: 0x0 expref: -99 pid: 6257 timeout: 0 lvb_type: 0 00010000:00010000:3.0:1776178219.876057:0:3340:0:(ldlm_lock.c:820:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(COS) ns: mdt-lustre-MDT0000_UUID lock: 00000000a27c19b3/0xa40083ab9bf539d9 lrc: 3/0,1 mode: COS/COS res: [0x200000007:0x1:0x0].0x2c847881 bits 0x2/0x0 rrc: 2 type: IBT gid 0 flags: 0x50210000000000 nid: local remote: 0x0 expref: -99 pid: 6257 timeout: 0 lvb_type: 0 00010000:00010000:3.0:1776178219.878481:0:3340:0:(ldlm_lock.c:879:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: mdt-lustre-MDT0000_UUID lock: 00000000a27c19b3/0xa40083ab9bf539d9 lrc: 2/0,0 mode: COS/COS res: [0x200000007:0x1:0x0].0x2c847881 bits 0x2/0x0 rrc: 2 type: IBT gid 0 flags: 0x50210400000000 nid: local remote: 0x0 expref: -99 pid: 6257 timeout: 0 lvb_type: 0 00010000:00010000:3.0:1776178219.878492:0:3340:0:(ldlm_lockd.c:1879:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: mdt-lustre-MDT0000_UUID lock: 00000000a27c19b3/0xa40083ab9bf539d9 lrc: 3/0,0 mode: COS/COS res: [0x200000007:0x1:0x0].0x2c847881 bits 0x2/0x0 rrc: 2 type: IBT gid 0 flags: 0x40210400000000 nid: local remote: 0x0 expref: -99 pid: 6257 timeout: 0 lvb_type: 0 00010000:00010000:3.0:1776178219.878499:0:3340:0:(ldlm_lockd.c:1893:ldlm_handle_bl_callback()) Lock 00000000a27c19b3 already unused, calling callback (0000000079e78348) 00010000:00010000:3.0:1776178219.878504:0:3340:0:(ldlm_request.c:364:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: mdt-lustre-MDT0000_UUID lock: 00000000a27c19b3/0xa40083ab9bf539d9 lrc: 3/0,0 mode: COS/COS res: [0x200000007:0x1:0x0].0x2c847881 bits 0x2/0x0 rrc: 2 type: IBT gid 0 flags: 0x40210400000000 nid: local remote: 0x0 expref: -99 pid: 6257 timeout: 0 lvb_type: 0 00010000:00010000:3.0:1776178219.878515:0:3340:0:(ldlm_request.c:1257:ldlm_cli_cancel_local()) ### server-side local cancel ns: mdt-lustre-MDT0000_UUID lock: 00000000a27c19b3/0xa40083ab9bf539d9 lrc: 4/0,0 mode: COS/COS res: [0x200000007:0x1:0x0].0x2c847881 bits 0x2/0x0 rrc: 2 type: IBT gid 0 flags: 0x40218400000000 nid: local remote: 0x0 expref: -99 pid: 6257 timeout: 0 lvb_type: 0 00010000:00010000:3.0:1776178219.878540:0:3340:0:(ldlm_inodebits.c:94:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x200000007:0x1:0x0].0x2c847881 (00000000ccda0888) 00010000:00010000:3.0:1776178219.878544:0:3340:0:(ldlm_inodebits.c:96:ldlm_reprocess_inodebits_queue()) Hint 2 00010000:00010000:3.0:1776178219.878550:0:3340:0:(ldlm_lockd.c:1903:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: mdt-lustre-MDT0000_UUID lock: 00000000a27c19b3/0xa40083ab9bf539d9 lrc: 2/0,0 mode: --/COS res: [0x200000007:0x1:0x0].0x2c847881 bits 0x2/0x0 rrc: 2 type: IBT gid 0 flags: 0x44a19400000000 nid: local remote: 0x0 expref: -99 pid: 6257 timeout: 0 lvb_type: 0 00010000:00010000:3.0:1776178219.878569:0:3340:0:(ldlm_lock.c:216:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: mdt-lustre-MDT0000_UUID lock: 00000000a27c19b3/0xa40083ab9bf539d9 lrc: 0/0,0 mode: --/COS res: [0x200000007:0x1:0x0].0x2c847881 bits 0x2/0x0 rrc: 2 type: IBT gid 0 flags: 0x44a19400000000 nid: local remote: 0x0 expref: -99 pid: 6257 timeout: 0 lvb_type: 0 00010000:00010000:3.0:1776178219.878597:0:3340:0:(ldlm_lock.c:820:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(COS) ns: mdt-lustre-MDT0000_UUID lock: 000000000b5c0143/0xa40083ab9bf539e0 lrc: 3/0,1 mode: COS/COS res: [0x2000013a1:0x11:0x0].0x0 bits 0x3/0x0 rrc: 2 type: IBT gid 0 flags: 0x50210000000000 nid: local remote: 0x0 expref: -99 pid: 6257 timeout: 0 lvb_type: 0 00010000:00010000:3.0:1776178219.878604:0:3340:0:(ldlm_lock.c:879:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: mdt-lustre-MDT0000_UUID lock: 000000000b5c0143/0xa40083ab9bf539e0 lrc: 2/0,0 mode: COS/COS res: [0x2000013a1:0x11:0x0].0x0 bits 0x3/0x0 rrc: 2 type: IBT gid 0 flags: 0x50210400000000 nid: local remote: 0x0 expref: -99 pid: 6257 timeout: 0 lvb_type: 0 00010000:00010000:3.0:1776178219.878610:0:3340:0:(ldlm_lockd.c:1879:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: mdt-lustre-MDT0000_UUID lock: 000000000b5c0143/0xa40083ab9bf539e0 lrc: 3/0,0 mode: COS/COS res: [0x2000013a1:0x11:0x0].0x0 bits 0x3/0x0 rrc: 2 type: IBT gid 0 flags: 0x40210400000000 nid: local remote: 0x0 expref: -99 pid: 6257 timeout: 0 lvb_type: 0 00010000:00010000:3.0:1776178219.878621:0:3340:0:(ldlm_lockd.c:1893:ldlm_handle_bl_callback()) Lock 000000000b5c0143 already unused, calling callback (0000000079e78348) 00010000:00010000:3.0:1776178219.878624:0:3340:0:(ldlm_request.c:364:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: mdt-lustre-MDT0000_UUID lock: 000000000b5c0143/0xa40083ab9bf539e0 lrc: 3/0,0 mode: COS/COS res: [0x2000013a1:0x11:0x0].0x0 bits 0x3/0x0 rrc: 2 type: IBT gid 0 flags: 0x40210400000000 nid: local remote: 0x0 expref: -99 pid: 6257 timeout: 0 lvb_type: 0 00010000:00010000:3.0:1776178219.878630:0:3340:0:(ldlm_request.c:1257:ldlm_cli_cancel_local()) ### server-side local cancel ns: mdt-lustre-MDT0000_UUID lock: 000000000b5c0143/0xa40083ab9bf539e0 lrc: 4/0,0 mode: COS/COS res: [0x2000013a1:0x11:0x0].0x0 bits 0x3/0x0 rrc: 2 type: IBT gid 0 flags: 0x40218400000000 nid: local remote: 0x0 expref: -99 pid: 6257 timeout: 0 lvb_type: 0 00010000:00010000:3.0:1776178219.878642:0:3340:0:(ldlm_inodebits.c:94:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x2000013a1:0x11:0x0].0x0 (00000000024b4e8f) 00010000:00010000:3.0:1776178219.878644:0:3340:0:(ldlm_inodebits.c:96:ldlm_reprocess_inodebits_queue()) Hint 3 00010000:00010000:3.0:1776178219.878648:0:3340:0:(ldlm_lockd.c:1903:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: mdt-lustre-MDT0000_UUID lock: 000000000b5c0143/0xa40083ab9bf539e0 lrc: 2/0,0 mode: --/COS res: [0x2000013a1:0x11:0x0].0x0 bits 0x3/0x0 rrc: 2 type: IBT gid 0 flags: 0x44a19400000000 nid: local remote: 0x0 expref: -99 pid: 6257 timeout: 0 lvb_type: 0 00010000:00010000:3.0:1776178219.878654:0:3340:0:(ldlm_lock.c:216:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: mdt-lustre-MDT0000_UUID lock: 000000000b5c0143/0xa40083ab9bf539e0 lrc: 0/0,0 mode: --/COS res: [0x2000013a1:0x11:0x0].0x0 bits 0x3/0x0 rrc: 2 type: IBT gid 0 flags: 0x44a19400000000 nid: local remote: 0x0 expref: -99 pid: 6257 timeout: 0 lvb_type: 0 00010000:00010000:1.0:1776178220.516721:0:9225:0:(ldlm_lockd.c:2792:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000400:00000080:3.0:1776178220.700043:0:22006:0:(module.c:221:libcfs_ioctl()) libcfs ioctl cmd 3221775648 00000001:02000400:3.0:1776178220.700050:0:22006:0:(debug.c:702:libcfs_debug_mark_buffer()) DEBUG MARKER: mds1 REPLAY BARRIER on lustre-MDT0000 00000100:00100000:0.0:1776178220.896472:0:3354:0:(client.c:1733:ptlrpc_send_new_req()) Sending RPC req@000000009e7a8bb5 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_01:lustre-MDT0001-mdtlov_UUID:3354:1862456998050624:0@lo:41:osp-pre-0-1.0 00000100:00100000:0.0:1776178220.896652:0:3354:0:(events.c:361:request_in_callback()) peer: 12345-0@lo (source: 12345-0@lo) 00000100:00100000:0.0:1776178220.896715:0:3355:0:(client.c:1733:ptlrpc_send_new_req()) Sending RPC req@00000000f95e9da5 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_02:lustre-MDT0000-mdtlov_UUID:3355:1862456998050688:0@lo:13:osp-pre-1-0.0 00000100:00100000:0.0:1776178220.896757:0:3355:0:(events.c:361:request_in_callback()) peer: 12345-0@lo (source: 12345-0@lo) 00000100:00100000:0.0:1776178220.896801:0:14128:0:(service.c:2152:ptlrpc_server_handle_req_in()) got req x1862456998050624 00000100:00100000:1.0:1776178220.897089:0:3355:0:(ptlrpcd.c:415:ptlrpcd_check()) transfer 1 async RPCs [3->2] 00000100:00100000:1.0:1776178220.897097:0:3355:0:(client.c:1733:ptlrpc_send_new_req()) Sending RPC req@00000000f44a2faf pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_02:lustre-MDT0000-mdtlov_UUID:3355:1862456998050752:0@lo:41:osp-pre-1-0.0 00000100:00100000:1.0:1776178220.897130:0:3355:0:(events.c:361:request_in_callback()) peer: 12345-0@lo (source: 12345-0@lo) 00000100:00100000:1.0:1776178220.897176:0:15213:0:(service.c:2152:ptlrpc_server_handle_req_in()) got req x1862456998050752 00000100:00100000:1.0:1776178220.897239:0:15213:0:(nrs_fifo.c:179:nrs_fifo_req_get()) NRS start fifo request from 12345-0@lo, seq: 708 00000100:00100000:1.0:1776178220.897251:0:15213:0:(service.c:2324:ptlrpc_server_handle_request()) Handling RPC req@0000000000167c64 pname:cluuid+ref:pid:xid:nid:opc:job mdt_out00_003:lustre-MDT0000-mdtlov_UUID+5:3355:x1862456998050752:12345-0@lo:41:osp-pre-1-0.0 00000004:00000024:1.0:1776178220.897309:0:15213:0:(mdt_handler.c:548:mdt_statfs()) blocks cached 0 granted 2097152 pending 0 free 1446039552 avail 1316577280 00000100:00100000:1.0:1776178220.897393:0:15213:0:(service.c:2372:ptlrpc_server_handle_request()) Handled RPC req@0000000000167c64 pname:cluuid+ref:pid:xid:nid:opc:job mdt_out00_003:lustre-MDT0000-mdtlov_UUID+5:3355:x1862456998050752:12345-0@lo:41:osp-pre-1-0.0 Request processed in 142us (262us total) trans 0 rc 0/0 00000100:00100000:1.0:1776178220.897406:0:15213:0:(nrs_fifo.c:241:nrs_fifo_req_stop()) NRS stop fifo request from 12345-0@lo, seq: 708 00000100:00100000:1.0:1776178220.897489:0:3355:0:(client.c:2215:ptlrpc_check_set()) Completed RPC req@00000000f44a2faf pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_02:lustre-MDT0000-mdtlov_UUID:3355:1862456998050752:0@lo:41:osp-pre-1-0.0 00000100:00100000:0.0:1776178220.897649:0:14128:0:(nrs_fifo.c:179:nrs_fifo_req_get()) NRS start fifo request from 12345-0@lo, seq: 709 00000100:00100000:0.0:1776178220.897657:0:14128:0:(service.c:2324:ptlrpc_server_handle_request()) Handling RPC req@0000000072865dff pname:cluuid+ref:pid:xid:nid:opc:job mdt_out00_002:lustre-MDT0001-mdtlov_UUID+6:3354:x1862456998050624:12345-0@lo:41:osp-pre-0-1.0 00000004:00000024:0.0:1776178220.897682:0:14128:0:(mdt_handler.c:548:mdt_statfs()) blocks cached 0 granted 2097152 pending 0 free 1446121472 avail 1316659200 00010000:00000080:0.0:1776178220.897692:0:14128:0:(ldlm_lib.c:3181:target_committed_to_req()) @@@ not sending last_committed update (1/0) req@0000000072865dff x1862456998050624/t0(0) o41->lustre-MDT0001-mdtlov_UUID@0@lo:747/0 lens 224/368 e 0 to 0 dl 1776178262 ref 1 fl Interpret:/0/0 rc 0/0 job:'osp-pre-0-1.0' 00000100:00100000:0.0:1776178220.897727:0:14128:0:(service.c:2372:ptlrpc_server_handle_request()) Handled RPC req@0000000072865dff pname:cluuid+ref:pid:xid:nid:opc:job mdt_out00_002:lustre-MDT0001-mdtlov_UUID+6:3354:x1862456998050624:12345-0@lo:41:osp-pre-0-1.0 Request processed in 70us (1077us total) trans 0 rc 0/0 00000100:00100000:0.0:1776178220.897732:0:14128:0:(nrs_fifo.c:241:nrs_fifo_req_stop()) NRS stop fifo request from 12345-0@lo, seq: 709 00000100:00100000:0.0:1776178220.897754:0:9554:0:(service.c:2152:ptlrpc_server_handle_req_in()) got req x1862456998050688 00000100:00100000:0.0:1776178220.897778:0:9554:0:(nrs_fifo.c:179:nrs_fifo_req_get()) NRS start fifo request from 12345-0@lo, seq: 493 00000100:00100000:0.0:1776178220.897782:0:9554:0:(service.c:2324:ptlrpc_server_handle_request()) Handling RPC req@00000000aba13576 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_create00:lustre-MDT0000-mdtlov_UUID+5:3355:x1862456998050688:12345-0@lo:13:osp-pre-1-0.0 00002000:00000024:0.0:1776178220.897806:0:9554:0:(ofd_obd.c:740:ofd_statfs()) blocks cached 229376 granted 491746368 pending 0 free 3923689472 avail 3692167168 00000100:00100000:0.0:1776178220.897988:0:9554:0:(service.c:2372:ptlrpc_server_handle_request()) Handled RPC req@00000000aba13576 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_create00:lustre-MDT0000-mdtlov_UUID+5:3355:x1862456998050688:12345-0@lo:13:osp-pre-1-0.0 Request processed in 201us (1228us total) trans 0 rc 0/0 00000100:00100000:0.0:1776178220.897999:0:9554:0:(nrs_fifo.c:241:nrs_fifo_req_stop()) NRS stop fifo request from 12345-0@lo, seq: 493 00000100:00100000:0.0:1776178220.898045:0:3354:0:(client.c:2215:ptlrpc_check_set()) Completed RPC req@000000009e7a8bb5 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_01:lustre-MDT0001-mdtlov_UUID:3354:1862456998050624:0@lo:41:osp-pre-0-1.0 00000100:00100000:1.0:1776178220.898283:0:3355:0:(client.c:2215:ptlrpc_check_set()) Completed RPC req@00000000f95e9da5 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_02:lustre-MDT0000-mdtlov_UUID:3355:1862456998050688:0@lo:13:osp-pre-1-0.0 00000100:00100000:3.0:1776178220.900551:0:3354:0:(ptlrpcd.c:415:ptlrpcd_check()) transfer 1 async RPCs [0->1] 00000100:00100000:3.0:1776178220.900575:0:3354:0:(client.c:1733:ptlrpc_send_new_req()) Sending RPC req@00000000c6606435 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_01:lustre-MDT0000-mdtlov_UUID:3354:1862456998050816:0@lo:13:osp-pre-0-0.0 00000100:00100000:3.0:1776178220.900664:0:3354:0:(events.c:361:request_in_callback()) peer: 12345-0@lo (source: 12345-0@lo) 00000100:00100000:3.0:1776178220.900738:0:8231:0:(service.c:2152:ptlrpc_server_handle_req_in()) got req x1862456998050816 00000100:00100000:3.0:1776178220.900796:0:8231:0:(nrs_fifo.c:179:nrs_fifo_req_get()) NRS start fifo request from 12345-0@lo, seq: 494 00000100:00100000:3.0:1776178220.900802:0:8231:0:(service.c:2324:ptlrpc_server_handle_request()) Handling RPC req@00000000587d8120 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_create00:lustre-MDT0000-mdtlov_UUID+5:3354:x1862456998050816:12345-0@lo:13:osp-pre-0-0.0 00002000:00000024:3.0:1776178220.900838:0:8231:0:(ofd_obd.c:740:ofd_statfs()) blocks cached 315392 granted 484883776 pending 0 free 3923689472 avail 3692167168 00000100:00100000:3.0:1776178220.900896:0:8231:0:(service.c:2372:ptlrpc_server_handle_request()) Handled RPC req@00000000587d8120 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_create00:lustre-MDT0000-mdtlov_UUID+5:3354:x1862456998050816:12345-0@lo:13:osp-pre-0-0.0 Request processed in 92us (234us total) trans 0 rc 0/0 00000100:00100000:3.0:1776178220.900905:0:8231:0:(nrs_fifo.c:241:nrs_fifo_req_stop()) NRS stop fifo request from 12345-0@lo, seq: 494 00000100:00100000:3.0:1776178220.900970:0:3354:0:(client.c:2215:ptlrpc_check_set()) Completed RPC req@00000000c6606435 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_01:lustre-MDT0000-mdtlov_UUID:3354:1862456998050816:0@lo:13:osp-pre-0-0.0 00000100:00100000:0.0:1776178220.901770:0:3356:0:(client.c:1733:ptlrpc_send_new_req()) Sending RPC req@000000009e7a8bb5 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_03:lustre-MDT0001-lwp-OST0000_UUID:3356:1862456998051008:0@lo:400:kworker/u8:3.0 00000100:00100000:0.0:1776178220.901810:0:3356:0:(events.c:361:request_in_callback()) peer: 12345-0@lo (source: 12345-0@lo) 00000100:00100000:0.0:1776178220.901832:0:3356:0:(client.c:1733:ptlrpc_send_new_req()) Sending RPC req@00000000a979415d pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_03:lustre-MDT0001-mdtlov_UUID:3356:1862456998051264:0@lo:400:kworker/u8:3.0 00000100:00100000:0.0:1776178220.901855:0:3356:0:(events.c:361:request_in_callback()) peer: 12345-0@lo (source: 12345-0@lo) 00000100:00100000:0.0:1776178220.901875:0:3356:0:(ptlrpcd.c:415:ptlrpcd_check()) transfer 2 async RPCs [2->3] 00000100:00100000:0.0:1776178220.901879:0:3356:0:(client.c:1733:ptlrpc_send_new_req()) Sending RPC req@00000000f95e9da5 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_03:lustre-MDT0000-lwp-OST0000_UUID:3356:1862456998050944:0@lo:400:kworker/u8:3.0 00000100:00100000:0.0:1776178220.901899:0:3356:0:(events.c:361:request_in_callback()) peer: 12345-0@lo (source: 12345-0@lo) 00000100:00100000:0.0:1776178220.901914:0:3356:0:(client.c:1733:ptlrpc_send_new_req()) Sending RPC req@00000000f871d6a5 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_03:lustre-MDT0001-mdtlov_UUID:3356:1862456998051200:0@lo:400:kworker/u8:3.0 00000100:00100000:0.0:1776178220.901932:0:3356:0:(events.c:361:request_in_callback()) peer: 12345-0@lo (source: 12345-0@lo) 00000100:00100000:0.0:1776178220.901963:0:14440:0:(service.c:2152:ptlrpc_server_handle_req_in()) got req x1862456998051264 00000100:00100000:0.0:1776178220.902031:0:14440:0:(service.c:2152:ptlrpc_server_handle_req_in()) got req x1862456998051200 00000100:00100000:0.0:1776178220.902050:0:14440:0:(nrs_fifo.c:179:nrs_fifo_req_get()) NRS start fifo request from 12345-0@lo, seq: 525 00000100:00100000:0.0:1776178220.902054:0:14440:0:(service.c:2324:ptlrpc_server_handle_request()) Handling RPC req@000000001c9b18be pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_004:lustre-MDT0001-mdtlov_UUID+5:3356:x1862456998051264:12345-0@lo:400:kworker/u8:3.0 00000100:00100000:0.0:1776178220.902144:0:14440:0:(service.c:2372:ptlrpc_server_handle_request()) Handled RPC req@000000001c9b18be pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_004:lustre-MDT0001-mdtlov_UUID+5:3356:x1862456998051264:12345-0@lo:400:kworker/u8:3.0 Request processed in 87us (286us total) trans 0 rc 0/0 00000100:00100000:0.0:1776178220.902155:0:14440:0:(nrs_fifo.c:241:nrs_fifo_req_stop()) NRS stop fifo request from 12345-0@lo, seq: 525 00000100:00100000:0.0:1776178220.902178:0:14440:0:(nrs_fifo.c:179:nrs_fifo_req_get()) NRS start fifo request from 12345-0@lo, seq: 526 00000100:00100000:0.0:1776178220.902184:0:14440:0:(service.c:2324:ptlrpc_server_handle_request()) Handling RPC req@00000000949e3fea pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_004:lustre-MDT0001-mdtlov_UUID+5:3356:x1862456998051200:12345-0@lo:400:kworker/u8:3.0 00000100:00100000:0.0:1776178220.902262:0:14440:0:(service.c:2372:ptlrpc_server_handle_request()) Handled RPC req@00000000949e3fea pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_004:lustre-MDT0001-mdtlov_UUID+5:3356:x1862456998051200:12345-0@lo:400:kworker/u8:3.0 Request processed in 75us (327us total) trans 0 rc 0/0 00000100:00100000:0.0:1776178220.902270:0:14440:0:(nrs_fifo.c:241:nrs_fifo_req_stop()) NRS stop fifo request from 12345-0@lo, seq: 526 00000100:00100000:0.0:1776178220.902309:0:11281:0:(service.c:2152:ptlrpc_server_handle_req_in()) got req x1862456998051008 00000100:00100000:0.0:1776178220.902353:0:11281:0:(service.c:2152:ptlrpc_server_handle_req_in()) got req x1862456998050944 00000100:00100000:0.0:1776178220.902396:0:11281:0:(nrs_fifo.c:179:nrs_fifo_req_get()) NRS start fifo request from 12345-0@lo, seq: 933 00000100:00100000:0.0:1776178220.902403:0:11281:0:(service.c:2324:ptlrpc_server_handle_request()) Handling RPC req@0000000022264af5 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_004:lustre-MDT0001-lwp-OST0000_UUID+5:3356:x1862456998051008:12345-0@lo:400:kworker/u8:3.0 00000100:00100000:0.0:1776178220.902456:0:11281:0:(service.c:2372:ptlrpc_server_handle_request()) Handled RPC req@0000000022264af5 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_004:lustre-MDT0001-lwp-OST0000_UUID+5:3356:x1862456998051008:12345-0@lo:400:kworker/u8:3.0 Request processed in 51us (643us total) trans 0 rc 0/0 00000100:00100000:0.0:1776178220.902464:0:11281:0:(nrs_fifo.c:241:nrs_fifo_req_stop()) NRS stop fifo request from 12345-0@lo, seq: 933 00000100:00100000:0.0:1776178220.902484:0:11281:0:(nrs_fifo.c:179:nrs_fifo_req_get()) NRS start fifo request from 12345-0@lo, seq: 934 00000100:00100000:0.0:1776178220.902490:0:11281:0:(service.c:2324:ptlrpc_server_handle_request()) Handling RPC req@00000000702f2be6 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_004:lustre-MDT0000-lwp-OST0000_UUID+5:3356:x1862456998050944:12345-0@lo:400:kworker/u8:3.0 00010000:00000080:0.0:1776178220.902511:0:11281:0:(ldlm_lib.c:3181:target_committed_to_req()) @@@ not sending last_committed update (1/0) req@00000000702f2be6 x1862456998050944/t0(0) o400->lustre-MDT0000-lwp-OST0000_UUID@0@lo:752/0 lens 224/224 e 0 to 0 dl 1776178267 ref 1 fl Interpret:H/0/0 rc 0/0 job:'kworker/u8:3.0' 00000100:00100000:0.0:1776178220.902555:0:11281:0:(service.c:2372:ptlrpc_server_handle_request()) Handled RPC req@00000000702f2be6 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_004:lustre-MDT0000-lwp-OST0000_UUID+5:3356:x1862456998050944:12345-0@lo:400:kworker/u8:3.0 Request processed in 63us (654us total) trans 0 rc 0/0 00000100:00100000:0.0:1776178220.902563:0:11281:0:(nrs_fifo.c:241:nrs_fifo_req_stop()) NRS stop fifo request from 12345-0@lo, seq: 934 00000100:00100000:0.0:1776178220.902618:0:3356:0:(client.c:2215:ptlrpc_check_set()) Completed RPC req@00000000f95e9da5 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_03:lustre-MDT0000-lwp-OST0000_UUID:3356:1862456998050944:0@lo:400:kworker/u8:3.0 00000100:00100000:0.0:1776178220.902635:0:3356:0:(client.c:2215:ptlrpc_check_set()) Completed RPC req@00000000f871d6a5 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_03:lustre-MDT0001-mdtlov_UUID:3356:1862456998051200:0@lo:400:kworker/u8:3.0 00000100:00100000:0.0:1776178220.902651:0:3356:0:(client.c:2215:ptlrpc_check_set()) Completed RPC req@000000009e7a8bb5 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_03:lustre-MDT0001-lwp-OST0000_UUID:3356:1862456998051008:0@lo:400:kworker/u8:3.0 00000100:00100000:0.0:1776178220.902666:0:3356:0:(client.c:2215:ptlrpc_check_set()) Completed RPC req@00000000a979415d pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_03:lustre-MDT0001-mdtlov_UUID:3356:1862456998051264:0@lo:400:kworker/u8:3.0 00000100:00100000:3.0:1776178220.903246:0:3354:0:(client.c:1733:ptlrpc_send_new_req()) Sending RPC req@000000006e2391ef pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_01:ac61f839-0e8e-4cac-9c59-08bcfbbad330:3354:1862456998050880:0@lo:400:kworker/u8:3.0 00000100:00100000:3.0:1776178220.903301:0:3354:0:(events.c:361:request_in_callback()) peer: 12345-0@lo (source: 12345-0@lo) 00000100:00100000:3.0:1776178220.903518:0:3354:0:(client.c:1733:ptlrpc_send_new_req()) Sending RPC req@00000000ecea139c pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_01:lustre-MDT0001-lwp-OST0001_UUID:3354:1862456998051136:0@lo:400:kworker/u8:3.0 00000100:00100000:3.0:1776178220.903567:0:3354:0:(events.c:361:request_in_callback()) peer: 12345-0@lo (source: 12345-0@lo) 00000100:00100000:3.0:1776178220.903598:0:3354:0:(client.c:1733:ptlrpc_send_new_req()) Sending RPC req@00000000a101cbf3 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_01:lustre-MDT0000-lwp-MDT0000_UUID:3354:1862456998051392:0@lo:400:kworker/u8:3.0 00000100:00100000:3.0:1776178220.903636:0:3354:0:(events.c:361:request_in_callback()) peer: 12345-0@lo (source: 12345-0@lo) 00000100:00100000:3.0:1776178220.903674:0:3354:0:(ptlrpcd.c:415:ptlrpcd_check()) transfer 2 async RPCs [0->1] 00000100:00100000:3.0:1776178220.903680:0:3354:0:(client.c:1733:ptlrpc_send_new_req()) Sending RPC req@000000007886b426 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_01:lustre-MDT0000-lwp-OST0001_UUID:3354:1862456998051072:0@lo:400:kworker/u8:3.0 00000100:00100000:3.0:1776178220.903702:0:3354:0:(events.c:361:request_in_callback()) peer: 12345-0@lo (source: 12345-0@lo) 00000100:00100000:3.0:1776178220.903717:0:3354:0:(client.c:1733:ptlrpc_send_new_req()) Sending RPC req@0000000014e690d4 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_01:lustre-MDT0000-lwp-MDT0001_UUID:3354:1862456998051328:0@lo:400:kworker/u8:3.0 00000100:00100000:3.0:1776178220.903736:0:3354:0:(events.c:361:request_in_callback()) peer: 12345-0@lo (source: 12345-0@lo) 00000100:00100000:3.0:1776178220.905873:0:6256:0:(service.c:2152:ptlrpc_server_handle_req_in()) got req x1862456998051136 00000100:00100000:3.0:1776178220.905935:0:6256:0:(service.c:2152:ptlrpc_server_handle_req_in()) got req x1862456998051392 00000100:00100000:3.0:1776178220.905958:0:6256:0:(service.c:2152:ptlrpc_server_handle_req_in()) got req x1862456998051072 00000100:00100000:3.0:1776178220.905980:0:6256:0:(service.c:2152:ptlrpc_server_handle_req_in()) got req x1862456998051328 00000100:00100000:3.0:1776178220.906010:0:6256:0:(nrs_fifo.c:179:nrs_fifo_req_get()) NRS start fifo request from 12345-0@lo, seq: 935 00000100:00100000:3.0:1776178220.906021:0:6256:0:(service.c:2324:ptlrpc_server_handle_request()) Handling RPC req@00000000c6606435 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_001:lustre-MDT0001-lwp-OST0001_UUID+5:3354:x1862456998051136:12345-0@lo:400:kworker/u8:3.0 00000100:00100000:3.0:1776178220.906164:0:6256:0:(service.c:2372:ptlrpc_server_handle_request()) Handled RPC req@00000000c6606435 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_001:lustre-MDT0001-lwp-OST0001_UUID+5:3354:x1862456998051136:12345-0@lo:400:kworker/u8:3.0 Request processed in 144us (2596us total) trans 0 rc 0/0 00000100:00100000:3.0:1776178220.906174:0:6256:0:(nrs_fifo.c:241:nrs_fifo_req_stop()) NRS stop fifo request from 12345-0@lo, seq: 935 00000100:00100000:3.0:1776178220.906206:0:6256:0:(nrs_fifo.c:179:nrs_fifo_req_get()) NRS start fifo request from 12345-0@lo, seq: 936 00000100:00100000:3.0:1776178220.906213:0:6256:0:(service.c:2324:ptlrpc_server_handle_request()) Handling RPC req@00000000a0c03f2e pname:cluuid+ref:pid:xid:nid:opc:job mdt00_001:lustre-MDT0000-lwp-MDT0000_UUID+5:3354:x1862456998051392:12345-0@lo:400:kworker/u8:3.0 00010000:00000080:3.0:1776178220.906233:0:6256:0:(ldlm_lib.c:3181:target_committed_to_req()) @@@ not sending last_committed update (1/0) req@00000000a0c03f2e x1862456998051392/t0(0) o400->lustre-MDT0000-lwp-MDT0000_UUID@0@lo:752/0 lens 224/224 e 0 to 0 dl 1776178267 ref 1 fl Interpret:H/0/0 rc 0/0 job:'kworker/u8:3.0' 00000100:00100000:3.0:1776178220.906284:0:6256:0:(service.c:2372:ptlrpc_server_handle_request()) Handled RPC req@00000000a0c03f2e pname:cluuid+ref:pid:xid:nid:opc:job mdt00_001:lustre-MDT0000-lwp-MDT0000_UUID+5:3354:x1862456998051392:12345-0@lo:400:kworker/u8:3.0 Request processed in 70us (2648us total) trans 0 rc 0/0 00000100:00100000:3.0:1776178220.906293:0:6256:0:(nrs_fifo.c:241:nrs_fifo_req_stop()) NRS stop fifo request from 12345-0@lo, seq: 936 00000100:00100000:3.0:1776178220.906311:0:6256:0:(nrs_fifo.c:179:nrs_fifo_req_get()) NRS start fifo request from 12345-0@lo, seq: 937 00000100:00100000:3.0:1776178220.906318:0:6256:0:(service.c:2324:ptlrpc_server_handle_request()) Handling RPC req@00000000cbfc3bb8 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_001:lustre-MDT0000-lwp-OST0001_UUID+5:3354:x1862456998051072:12345-0@lo:400:kworker/u8:3.0 00010000:00000080:3.0:1776178220.906334:0:6256:0:(ldlm_lib.c:3181:target_committed_to_req()) @@@ not sending last_committed update (1/0) req@00000000cbfc3bb8 x1862456998051072/t0(0) o400->lustre-MDT0000-lwp-OST0001_UUID@0@lo:752/0 lens 224/224 e 0 to 0 dl 1776178267 ref 1 fl Interpret:H/0/0 rc 0/0 job:'kworker/u8:3.0' 00000100:00100000:3.0:1776178220.906370:0:6256:0:(service.c:2372:ptlrpc_server_handle_request()) Handled RPC req@00000000cbfc3bb8 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_001:lustre-MDT0000-lwp-OST0001_UUID+5:3354:x1862456998051072:12345-0@lo:400:kworker/u8:3.0 Request processed in 51us (2665us total) trans 0 rc 0/0 00000100:00100000:0.0:1776178220.906475:0:20140:0:(service.c:2152:ptlrpc_server_handle_req_in()) got req x1862456998050880 00000100:00100000:0.0:1776178220.906521:0:20140:0:(nrs_fifo.c:179:nrs_fifo_req_get()) NRS start fifo request from 12345-0@lo, seq: 163 00000100:00100000:0.0:1776178220.906530:0:20140:0:(service.c:2324:ptlrpc_server_handle_request()) Handling RPC req@00000000b4107e97 pname:cluuid+ref:pid:xid:nid:opc:job ll_mgs_0002:ac61f839-0e8e-4cac-9c59-08bcfbbad330+23:3354:x1862456998050880:12345-0@lo:400:kworker/u8:3.0 00000100:00100000:0.0:1776178220.906585:0:20140:0:(service.c:2372:ptlrpc_server_handle_request()) Handled RPC req@00000000b4107e97 pname:cluuid+ref:pid:xid:nid:opc:job ll_mgs_0002:ac61f839-0e8e-4cac-9c59-08bcfbbad330+23:3354:x1862456998050880:12345-0@lo:400:kworker/u8:3.0 Request processed in 54us (3284us total) trans 0 rc 0/0 00000100:00100000:0.0:1776178220.906594:0:20140:0:(nrs_fifo.c:241:nrs_fifo_req_stop()) NRS stop fifo request from 12345-0@lo, seq: 163 00000100:00100000:3.0:1776178220.910565:0:6256:0:(nrs_fifo.c:241:nrs_fifo_req_stop()) NRS stop fifo request from 12345-0@lo, seq: 937 00000100:00100000:2.0:1776178220.910774:0:6257:0:(nrs_fifo.c:179:nrs_fifo_req_get()) NRS start fifo request from 12345-0@lo, seq: 938 00000100:00100000:2.0:1776178220.910788:0:6257:0:(service.c:2324:ptlrpc_server_handle_request()) Handling RPC req@00000000bd8e8540 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_002:lustre-MDT0000-lwp-MDT0001_UUID+5:3354:x1862456998051328:12345-0@lo:400:kworker/u8:3.0 00010000:00000080:2.0:1776178220.910826:0:6257:0:(ldlm_lib.c:3181:target_committed_to_req()) @@@ not sending last_committed update (1/0) req@00000000bd8e8540 x1862456998051328/t0(0) o400->lustre-MDT0000-lwp-MDT0001_UUID@0@lo:752/0 lens 224/224 e 0 to 0 dl 1776178267 ref 1 fl Interpret:H/0/0 rc 0/0 job:'kworker/u8:3.0' 00000100:00100000:2.0:1776178220.910920:0:6257:0:(service.c:2372:ptlrpc_server_handle_request()) Handled RPC req@00000000bd8e8540 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_002:lustre-MDT0000-lwp-MDT0001_UUID+5:3354:x1862456998051328:12345-0@lo:400:kworker/u8:3.0 Request processed in 132us (7180us total) trans 0 rc 0/0 00000100:00100000:2.0:1776178220.910933:0:6257:0:(nrs_fifo.c:241:nrs_fifo_req_stop()) NRS stop fifo request from 12345-0@lo, seq: 938 00000100:00100000:3.0:1776178220.918441:0:3354:0:(client.c:2215:ptlrpc_check_set()) Completed RPC req@000000007886b426 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_01:lustre-MDT0000-lwp-OST0001_UUID:3354:1862456998051072:0@lo:400:kworker/u8:3.0 00000100:00100000:3.0:1776178220.918473:0:3354:0:(client.c:2215:ptlrpc_check_set()) Completed RPC req@0000000014e690d4 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_01:lustre-MDT0000-lwp-MDT0001_UUID:3354:1862456998051328:0@lo:400:kworker/u8:3.0 00000100:00100000:3.0:1776178220.918492:0:3354:0:(client.c:2215:ptlrpc_check_set()) Completed RPC req@000000006e2391ef pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_01:ac61f839-0e8e-4cac-9c59-08bcfbbad330:3354:1862456998050880:0@lo:400:kworker/u8:3.0 00000100:00100000:3.0:1776178220.918512:0:3354:0:(client.c:2215:ptlrpc_check_set()) Completed RPC req@00000000ecea139c pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_01:lustre-MDT0001-lwp-OST0001_UUID:3354:1862456998051136:0@lo:400:kworker/u8:3.0 00000100:00100000:3.0:1776178220.918528:0:3354:0:(client.c:2215:ptlrpc_check_set()) Completed RPC req@00000000a101cbf3 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_01:lustre-MDT0000-lwp-MDT0000_UUID:3354:1862456998051392:0@lo:400:kworker/u8:3.0 00010000:00010000:1.0:1776178221.544305:0:6245:0:(ldlm_lockd.c:2792:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:0.0:1776178221.710529:0:3347:0:(events.c:361:request_in_callback()) peer: 12345-192.168.204.3@tcp (source: 12345-192.168.204.3@tcp) 00000100:00100000:3.0:1776178221.710632:0:6256:0:(service.c:2152:ptlrpc_server_handle_req_in()) got req x1862456979108416 00000100:00100000:3.0:1776178221.710714:0:6256:0:(nrs_fifo.c:179:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.204.3@tcp, seq: 6622 00000100:00100000:3.0:1776178221.710724:0:6256:0:(service.c:2324:ptlrpc_server_handle_request()) Handling RPC req@00000000a979415d pname:cluuid+ref:pid:xid:nid:opc:job mdt00_001:b50963bd-8eba-4432-9a54-8888d00e3fa1+14:18361:x1862456979108416:12345-192.168.204.3@tcp:101:chmod.0 00010000:00010000:3.0:1776178221.710756:0:6256:0:(ldlm_lockd.c:1248:ldlm_handle_enqueue0()) ### server-side enqueue handler START 00010000:00010000:3.0:1776178221.710787:0:6256:0:(ldlm_lockd.c:1318:ldlm_handle_enqueue0()) ### server-side enqueue handler, new lock created ns: mdt-lustre-MDT0000_UUID lock: 000000008192de90/0xa40083ab9bf539ee lrc: 2/0,0 mode: --/CR res: [0x2000013a1:0x9:0x0].0x0 bits 0x0/0x0 rrc: 3 type: IBT gid 0 flags: 0x40000000000000 nid: local remote: 0x771f68aa1eb27c22 expref: -99 pid: 6256 timeout: 0 lvb_type: 0 00010000:00010000:3.0:1776178221.710872:0:6256:0:(ldlm_lock.c:766:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PR) ns: mdt-lustre-MDT0000_UUID lock: 000000005d93f094/0xa40083ab9bf539f5 lrc: 3/1,0 mode: --/PR res: [0x2000013a1:0xa:0x0].0x0 bits 0x0/0x0 rrc: 3 type: IBT gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 6256 timeout: 0 lvb_type: 0 00010000:00010000:3.0:1776178221.710890:0:6256:0:(ldlm_lock.c:1064:ldlm_granted_list_add_lock()) ### About to add lock: ns: mdt-lustre-MDT0000_UUID lock: 000000005d93f094/0xa40083ab9bf539f5 lrc: 3/1,0 mode: PR/PR res: [0x2000013a1:0xa:0x0].0x0 bits 0x5b/0x0 rrc: 3 type: IBT gid 0 flags: 0x50210000000000 nid: local remote: 0x0 expref: -99 pid: 6256 timeout: 0 lvb_type: 0 00010000:00010000:3.0:1776178221.710904:0:6256:0:(ldlm_request.c:528:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: mdt-lustre-MDT0000_UUID lock: 000000005d93f094/0xa40083ab9bf539f5 lrc: 3/1,0 mode: PR/PR res: [0x2000013a1:0xa:0x0].0x0 bits 0x5b/0x0 rrc: 3 type: IBT gid 0 flags: 0x40210000000000 nid: local remote: 0x0 expref: -99 pid: 6256 timeout: 0 lvb_type: 0 00000004:00200000:3.0:1776178221.710964:0:6256:0:(mdt_handler.c:912:mdt_pack_attr2body()) [0x2000013a1:0xa:0x0]: returning size 0 00000004:00200000:3.0:1776178221.710970:0:6256:0:(mdt_handler.c:1523:mdt_getattr_internal()) dirent count 0 stripe count 1 MDT count 2 00000004:00010000:3.0:1776178221.710994:0:6256:0:(mdt_handler.c:2353:mdt_getattr_name_lock()) ### Returning lock to client ns: mdt-lustre-MDT0000_UUID lock: 000000005d93f094/0xa40083ab9bf539f5 lrc: 3/1,0 mode: PR/PR res: [0x2000013a1:0xa:0x0].0x0 bits 0x5b/0x0 rrc: 3 type: IBT gid 0 flags: 0x40210000000000 nid: local remote: 0x0 expref: -99 pid: 6256 timeout: 0 lvb_type: 0 00010000:00010000:3.0:1776178221.711007:0:6256:0:(ldlm_lock.c:1064:ldlm_granted_list_add_lock()) ### About to add lock: ns: mdt-lustre-MDT0000_UUID lock: 000000005d93f094/0xa40083ab9bf539f5 lrc: 3/1,0 mode: PR/PR res: [0x2000013a1:0xa:0x0].0x0 bits 0x1b/0x0 rrc: 3 type: IBT gid 0 flags: 0x50210000000000 nid: local remote: 0x0 expref: -99 pid: 6256 timeout: 0 lvb_type: 0 00010000:00010000:3.0:1776178221.711162:0:6256:0:(ldlm_lock.c:216:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: mdt-lustre-MDT0000_UUID lock: 000000008192de90/0xa40083ab9bf539ee lrc: 0/0,0 mode: --/CR res: [0x2000013a1:0x9:0x0].0x0 bits 0x2/0x0 rrc: 3 type: IBT gid 0 flags: 0x44000000000000 nid: 192.168.204.3@tcp remote: 0x771f68aa1eb27c22 expref: 16 pid: 6256 timeout: 0 lvb_type: 0 00010000:00010000:3.0:1776178221.711182:0:6256:0:(ldlm_lockd.c:1484:ldlm_handle_enqueue0()) ### server-side enqueue handler, sending reply (err=0, rc=0) ns: mdt-lustre-MDT0000_UUID lock: 000000005d93f094/0xa40083ab9bf539f5 lrc: 3/0,0 mode: PR/PR res: [0x2000013a1:0xa:0x0].0x0 bits 0x1b/0x0 rrc: 3 type: IBT gid 0 flags: 0x40200000000000 nid: 192.168.204.3@tcp remote: 0x771f68aa1eb27c22 expref: 15 pid: 6256 timeout: 0 lvb_type: 0 00010000:00010000:3.0:1776178221.711190:0:6256:0:(ldlm_inodebits.c:94:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x2000013a1:0xa:0x0].0x0 (000000000a3c1046) 00010000:00010000:3.0:1776178221.711193:0:6256:0:(ldlm_inodebits.c:96:ldlm_reprocess_inodebits_queue()) Hint 1b 00010000:00010000:3.0:1776178221.711197:0:6256:0:(ldlm_lockd.c:1565:ldlm_handle_enqueue0()) ### server-side enqueue handler END (lock 000000005d93f094, rc 0) 00010000:00000080:3.0:1776178221.711205:0:6256:0:(ldlm_lib.c:3181:target_committed_to_req()) @@@ not sending last_committed update (1/0) req@00000000a979415d x1862456979108416/t0(0) o101->b50963bd-8eba-4432-9a54-8888d00e3fa1@192.168.204.3@tcp:753/0 lens 576/696 e 0 to 0 dl 1776178268 ref 1 fl Interpret:/0/0 rc 0/0 job:'chmod.0' 00000100:00100000:3.0:1776178221.712507:0:6256:0:(service.c:2372:ptlrpc_server_handle_request()) Handled RPC req@00000000a979415d pname:cluuid+ref:pid:xid:nid:opc:job mdt00_001:b50963bd-8eba-4432-9a54-8888d00e3fa1+15:18361:x1862456979108416:12345-192.168.204.3@tcp:101:chmod.0 Request processed in 1773us (1986us total) trans 0 rc 0/0 00000100:00100000:3.0:1776178221.712522:0:6256:0:(nrs_fifo.c:241:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.204.3@tcp, seq: 6622 00000100:00100000:2.0:1776178221.723182:0:3348:0:(events.c:361:request_in_callback()) peer: 12345-192.168.204.3@tcp (source: 12345-192.168.204.3@tcp) 00000100:00100000:0.0:1776178221.723249:0:20759:0:(service.c:2152:ptlrpc_server_handle_req_in()) got req x1862456979108480 00000100:00100000:0.0:1776178221.723325:0:20759:0:(nrs_fifo.c:179:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.204.3@tcp, seq: 5292 00000100:00100000:0.0:1776178221.723341:0:20759:0:(service.c:2324:ptlrpc_server_handle_request()) Handling RPC req@00000000f433e404 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_010:b50963bd-8eba-4432-9a54-8888d00e3fa1+5:2270:x1862456979108480:12345-192.168.204.3@tcp:101:chmod.0 00010000:00010000:0.0:1776178221.723374:0:20759:0:(ldlm_lockd.c:1248:ldlm_handle_enqueue0()) ### server-side enqueue handler START 00010000:00010000:0.0:1776178221.723403:0:20759:0:(ldlm_lockd.c:1318:ldlm_handle_enqueue0()) ### server-side enqueue handler, new lock created ns: filter-lustre-OST0001_UUID lock: 00000000f190b39a/0xa40083ab9bf539fc lrc: 2/0,0 mode: --/PR res: [0x45:0x0:0x0].0x0 rrc: 2 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x771f68aa1eb27c29 expref: -99 pid: 20759 timeout: 0 lvb_type: 0 00002000:00010000:0.0:1776178221.723432:0:20759:0:(ofd_lvb.c:170:ofd_lvbo_init()) res: [0x100010000:0x45:0x0] initial LVB size: 0, mtime: 0x69de5423, atime: 0x69de5423, ctime: 0x69de5423, blocks: 0x0 00010000:00010000:0.0:1776178221.723459:0:20759:0:(ldlm_extent.c:318:ldlm_check_contention()) contended locks = 0 00010000:00010000:0.0:1776178221.723462:0:20759:0:(ldlm_extent.c:318:ldlm_check_contention()) contended locks = 0 00010000:00010000:0.0:1776178221.723473:0:20759:0:(ldlm_resource.c:1650:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0001_UUID lock: 00000000f190b39a/0xa40083ab9bf539fc lrc: 3/0,0 mode: PR/PR res: [0x45:0x0:0x0].0x0 rrc: 2 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x40000000000000 nid: 192.168.204.3@tcp remote: 0x771f68aa1eb27c29 expref: 6 pid: 20759 timeout: 0 lvb_type: 1 00010000:00010000:0.0:1776178221.723494:0:20759:0:(ldlm_lockd.c:1484:ldlm_handle_enqueue0()) ### server-side enqueue handler, sending reply (err=0, rc=0) ns: filter-lustre-OST0001_UUID lock: 00000000f190b39a/0xa40083ab9bf539fc lrc: 3/0,0 mode: PR/PR res: [0x45:0x0:0x0].0x0 rrc: 2 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x40000000000000 nid: 192.168.204.3@tcp remote: 0x771f68aa1eb27c29 expref: 6 pid: 20759 timeout: 0 lvb_type: 1 00010000:00010000:0.0:1776178221.723510:0:20759:0:(ldlm_lockd.c:1565:ldlm_handle_enqueue0()) ### server-side enqueue handler END (lock 00000000f190b39a, rc 0) 00000100:00100000:0.0:1776178221.723585:0:20759:0:(service.c:2372:ptlrpc_server_handle_request()) Handled RPC req@00000000f433e404 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_010:b50963bd-8eba-4432-9a54-8888d00e3fa1+6:2270:x1862456979108480:12345-192.168.204.3@tcp:101:chmod.0 Request processed in 251us (414us total) trans 0 rc 0/0 00000100:00100000:0.0:1776178221.723593:0:20759:0:(nrs_fifo.c:241:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.204.3@tcp, seq: 5292 00000100:00100000:1.0:1776178221.731176:0:3346:0:(events.c:361:request_in_callback()) peer: 12345-192.168.204.3@tcp (source: 12345-192.168.204.3@tcp) 00000100:00100000:1.0:1776178221.731218:0:3346:0:(events.c:361:request_in_callback()) peer: 12345-192.168.204.3@tcp (source: 12345-192.168.204.3@tcp) 00000100:00100000:3.0:1776178221.731236:0:6256:0:(service.c:2152:ptlrpc_server_handle_req_in()) got req x1862456979108544 00000100:00100000:1.0:1776178221.731240:0:3346:0:(events.c:361:request_in_callback()) peer: 12345-192.168.204.3@tcp (source: 12345-192.168.204.3@tcp) 00000100:00100000:1.0:1776178221.731270:0:8250:0:(service.c:2152:ptlrpc_server_handle_req_in()) got req x1862456979108672 00000100:00100000:3.0:1776178221.731307:0:6256:0:(service.c:2152:ptlrpc_server_handle_req_in()) got req x1862456979108736 00000100:00100000:1.0:1776178221.734248:0:8250:0:(nrs_fifo.c:179:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.204.3@tcp, seq: 6623 00000100:00100000:1.0:1776178221.734266:0:8250:0:(service.c:2324:ptlrpc_server_handle_request()) Handling RPC req@00000000f44a2faf pname:cluuid+ref:pid:xid:nid:opc:job mdt00_003:b50963bd-8eba-4432-9a54-8888d00e3fa1+17:18364:x1862456979108544:12345-192.168.204.3@tcp:101:chmod.0 00010000:00010000:1.0:1776178221.734299:0:8250:0:(ldlm_lockd.c:1248:ldlm_handle_enqueue0()) ### server-side enqueue handler START 00010000:00010000:1.0:1776178221.734327:0:8250:0:(ldlm_lockd.c:1318:ldlm_handle_enqueue0()) ### server-side enqueue handler, new lock created ns: mdt-lustre-MDT0000_UUID lock: 00000000499bab84/0xa40083ab9bf53a03 lrc: 2/0,0 mode: --/CR res: [0x2000013a1:0x9:0x0].0x0 bits 0x0/0x0 rrc: 3 type: IBT gid 0 flags: 0x40000000000000 nid: local remote: 0x771f68aa1eb27c30 expref: -99 pid: 8250 timeout: 0 lvb_type: 0 00000100:00100000:0.0:1776178221.734461:0:6257:0:(nrs_fifo.c:179:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.204.3@tcp, seq: 6624 00000100:00100000:0.0:1776178221.734468:0:6257:0:(service.c:2324:ptlrpc_server_handle_request()) Handling RPC req@00000000cc7f07e3 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_002:b50963bd-8eba-4432-9a54-8888d00e3fa1+17:18361:x1862456979108672:12345-192.168.204.3@tcp:36:chmod.0 00010000:00010000:0.0:1776178221.734527:0:6257:0:(ldlm_lockd.c:1720:ldlm_request_cancel()) ### server-side cancel handler START: 2 locks, starting at 0 00010000:00010000:0.0:1776178221.734567:0:6257:0:(ldlm_lock.c:216:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: mdt-lustre-MDT0000_UUID lock: 000000005d93f094/0xa40083ab9bf539f5 lrc: 0/0,0 mode: --/PR res: [0x2000013a1:0xa:0x0].0x0 bits 0x1b/0x0 rrc: 4 type: IBT gid 0 flags: 0x44a01000000000 nid: 192.168.204.3@tcp remote: 0x771f68aa1eb27c22 expref: 17 pid: 6256 timeout: 0 lvb_type: 0 00010000:00010000:0.0:1776178221.734602:0:6257:0:(ldlm_lock.c:216:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: mdt-lustre-MDT0000_UUID lock: 00000000782e253e/0xa40083ab9bf53889 lrc: 0/0,0 mode: --/CR res: [0x2000013a1:0xa:0x0].0x0 bits 0x9/0x0 rrc: 3 type: IBT gid 0 flags: 0x44a01000000000 nid: 192.168.204.3@tcp remote: 0x771f68aa1eb27be3 expref: 16 pid: 6257 timeout: 0 lvb_type: 0 00010000:00010000:0.0:1776178221.734614:0:6257:0:(ldlm_inodebits.c:94:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x2000013a1:0xa:0x0].0x0 (000000000a3c1046) 00010000:00010000:0.0:1776178221.734628:0:6257:0:(ldlm_lockd.c:1782:ldlm_request_cancel()) ### server-side cancel handler END 00010000:00010000:0.0:1776178221.734673:0:6257:0:(ldlm_lock.c:766:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: mdt-lustre-MDT0000_UUID lock: 0000000015ae4ee6/0xa40083ab9bf53a0a lrc: 3/0,1 mode: --/PW res: [0x2000013a1:0xa:0x0].0x0 bits 0x0/0x0 rrc: 2 type: IBT gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 6257 timeout: 0 lvb_type: 0 00010000:00010000:0.0:1776178221.734689:0:6257:0:(ldlm_lock.c:1064:ldlm_granted_list_add_lock()) ### About to add lock: ns: mdt-lustre-MDT0000_UUID lock: 0000000015ae4ee6/0xa40083ab9bf53a0a lrc: 3/0,1 mode: PW/PW res: [0x2000013a1:0xa:0x0].0x0 bits 0x13/0x0 rrc: 2 type: IBT gid 0 flags: 0x50210000000000 nid: local remote: 0x0 expref: -99 pid: 6257 timeout: 0 lvb_type: 0 00010000:00010000:0.0:1776178221.734703:0:6257:0:(ldlm_request.c:528:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: mdt-lustre-MDT0000_UUID lock: 0000000015ae4ee6/0xa40083ab9bf53a0a lrc: 3/0,1 mode: PW/PW res: [0x2000013a1:0xa:0x0].0x0 bits 0x13/0x0 rrc: 2 type: IBT gid 0 flags: 0x40210000000000 nid: local remote: 0x0 expref: -99 pid: 6257 timeout: 0 lvb_type: 0 00000004:00010000:0.0:1776178221.735005:0:6257:0:(mdt_handler.c:3992:mdt_save_lock()) ### save lock request 00000000cc7f07e3 reply state 0000000027a4aedc transno 12884901975 ns: mdt-lustre-MDT0000_UUID lock: 0000000015ae4ee6/0xa40083ab9bf53a0a lrc: 3/0,1 mode: PW/PW res: [0x2000013a1:0xa:0x0].0x0 bits 0x13/0x0 rrc: 2 type: IBT gid 0 flags: 0x40210000000000 nid: local remote: 0x0 expref: -99 pid: 6257 timeout: 0 lvb_type: 0 00010000:00000080:0.0:1776178221.735129:0:6257:0:(ldlm_lib.c:3181:target_committed_to_req()) @@@ not sending last_committed update (1/0) req@00000000cc7f07e3 x1862456979108672/t12884901975(0) o36->b50963bd-8eba-4432-9a54-8888d00e3fa1@192.168.204.3@tcp:753/0 lens 488/456 e 0 to 0 dl 1776178268 ref 1 fl Interpret:/0/0 rc 0/0 job:'chmod.0' 00010000:02000000:0.0:1776178221.735141:0:6257:0:(libcfs_fail.h:96:cfs_fail_check_set()) *** cfs_fail_loc=15a, val=0*** 00000100:00100000:3.0:1776178221.736146:0:6256:0:(nrs_fifo.c:179:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.204.3@tcp, seq: 6625 00000100:00100000:3.0:1776178221.736157:0:6256:0:(service.c:2324:ptlrpc_server_handle_request()) Handling RPC req@00000000be94e5d7 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_001:b50963bd-8eba-4432-9a54-8888d00e3fa1+17:18362:x1862456979108736:12345-192.168.204.3@tcp:101:chmod.0 00010000:00010000:3.0:1776178221.736176:0:6256:0:(ldlm_lockd.c:1248:ldlm_handle_enqueue0()) ### server-side enqueue handler START 00010000:00010000:3.0:1776178221.736199:0:6256:0:(ldlm_lockd.c:1318:ldlm_handle_enqueue0()) ### server-side enqueue handler, new lock created ns: mdt-lustre-MDT0000_UUID lock: 000000008192de90/0xa40083ab9bf53a11 lrc: 2/0,0 mode: --/CR res: [0x2000013a1:0x9:0x0].0x0 bits 0x0/0x0 rrc: 5 type: IBT gid 0 flags: 0x40000000000000 nid: local remote: 0x771f68aa1eb27c37 expref: -99 pid: 6256 timeout: 0 lvb_type: 0 00010000:00010000:3.0:1776178221.736270:0:6256:0:(ldlm_lock.c:766:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PR) ns: mdt-lustre-MDT0000_UUID lock: 0000000055220ae2/0xa40083ab9bf53a18 lrc: 3/1,0 mode: --/PR res: [0x2000013a1:0xb:0x0].0x0 bits 0x0/0x0 rrc: 3 type: IBT gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 6256 timeout: 0 lvb_type: 0 00010000:00010000:3.0:1776178221.736286:0:6256:0:(ldlm_lock.c:1064:ldlm_granted_list_add_lock()) ### About to add lock: ns: mdt-lustre-MDT0000_UUID lock: 0000000055220ae2/0xa40083ab9bf53a18 lrc: 3/1,0 mode: PR/PR res: [0x2000013a1:0xb:0x0].0x0 bits 0x5b/0x0 rrc: 3 type: IBT gid 0 flags: 0x50210000000000 nid: local remote: 0x0 expref: -99 pid: 6256 timeout: 0 lvb_type: 0 00010000:00010000:3.0:1776178221.736297:0:6256:0:(ldlm_request.c:528:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: mdt-lustre-MDT0000_UUID lock: 0000000055220ae2/0xa40083ab9bf53a18 lrc: 3/1,0 mode: PR/PR res: [0x2000013a1:0xb:0x0].0x0 bits 0x5b/0x0 rrc: 3 type: IBT gid 0 flags: 0x40210000000000 nid: local remote: 0x0 expref: -99 pid: 6256 timeout: 0 lvb_type: 0 00000004:00200000:3.0:1776178221.736332:0:6256:0:(mdt_handler.c:912:mdt_pack_attr2body()) [0x2000013a1:0xb:0x0]: returning size 0 00000004:00200000:3.0:1776178221.736336:0:6256:0:(mdt_handler.c:1523:mdt_getattr_internal()) dirent count 0 stripe count 1 MDT count 2 00000004:00010000:3.0:1776178221.736352:0:6256:0:(mdt_handler.c:2353:mdt_getattr_name_lock()) ### Returning lock to client ns: mdt-lustre-MDT0000_UUID lock: 0000000055220ae2/0xa40083ab9bf53a18 lrc: 3/1,0 mode: PR/PR res: [0x2000013a1:0xb:0x0].0x0 bits 0x5b/0x0 rrc: 3 type: IBT gid 0 flags: 0x40210000000000 nid: local remote: 0x0 expref: -99 pid: 6256 timeout: 0 lvb_type: 0 00010000:00010000:3.0:1776178221.736379:0:6256:0:(ldlm_lock.c:1064:ldlm_granted_list_add_lock()) ### About to add lock: ns: mdt-lustre-MDT0000_UUID lock: 0000000055220ae2/0xa40083ab9bf53a18 lrc: 3/1,0 mode: PR/PR res: [0x2000013a1:0xb:0x0].0x0 bits 0x1b/0x0 rrc: 3 type: IBT gid 0 flags: 0x50210000000000 nid: local remote: 0x0 expref: -99 pid: 6256 timeout: 0 lvb_type: 0 00010000:00010000:3.0:1776178221.736403:0:6256:0:(ldlm_lock.c:216:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: mdt-lustre-MDT0000_UUID lock: 000000008192de90/0xa40083ab9bf53a11 lrc: 0/0,0 mode: --/CR res: [0x2000013a1:0x9:0x0].0x0 bits 0x2/0x0 rrc: 5 type: IBT gid 0 flags: 0x44000000000000 nid: 192.168.204.3@tcp remote: 0x771f68aa1eb27c37 expref: 19 pid: 6256 timeout: 0 lvb_type: 0 00010000:00010000:3.0:1776178221.736422:0:6256:0:(ldlm_lockd.c:1484:ldlm_handle_enqueue0()) ### server-side enqueue handler, sending reply (err=0, rc=0) ns: mdt-lustre-MDT0000_UUID lock: 0000000055220ae2/0xa40083ab9bf53a18 lrc: 3/0,0 mode: PR/PR res: [0x2000013a1:0xb:0x0].0x0 bits 0x1b/0x0 rrc: 3 type: IBT gid 0 flags: 0x40200000000000 nid: 192.168.204.3@tcp remote: 0x771f68aa1eb27c37 expref: 18 pid: 6256 timeout: 0 lvb_type: 0 00010000:00010000:3.0:1776178221.736434:0:6256:0:(ldlm_inodebits.c:94:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x2000013a1:0xb:0x0].0x0 (000000003275315d) 00010000:00010000:3.0:1776178221.736438:0:6256:0:(ldlm_inodebits.c:96:ldlm_reprocess_inodebits_queue()) Hint 1b 00010000:00010000:3.0:1776178221.736440:0:6256:0:(ldlm_lockd.c:1565:ldlm_handle_enqueue0()) ### server-side enqueue handler END (lock 0000000055220ae2, rc 0) 00010000:00010000:1.0:1776178221.736916:0:8250:0:(ldlm_lock.c:766:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PR) ns: mdt-lustre-MDT0000_UUID lock: 00000000336cd7d4/0xa40083ab9bf53a1f lrc: 3/1,0 mode: --/PR res: [0x2000013a1:0xd:0x0].0x0 bits 0x0/0x0 rrc: 3 type: IBT gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 8250 timeout: 0 lvb_type: 0 00010000:00010000:1.0:1776178221.736925:0:8250:0:(ldlm_lock.c:1064:ldlm_granted_list_add_lock()) ### About to add lock: ns: mdt-lustre-MDT0000_UUID lock: 00000000336cd7d4/0xa40083ab9bf53a1f lrc: 3/1,0 mode: PR/PR res: [0x2000013a1:0xd:0x0].0x0 bits 0x5b/0x0 rrc: 3 type: IBT gid 0 flags: 0x50210000000000 nid: local remote: 0x0 expref: -99 pid: 8250 timeout: 0 lvb_type: 0 00010000:00010000:1.0:1776178221.736932:0:8250:0:(ldlm_request.c:528:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: mdt-lustre-MDT0000_UUID lock: 00000000336cd7d4/0xa40083ab9bf53a1f lrc: 3/1,0 mode: PR/PR res: [0x2000013a1:0xd:0x0].0x0 bits 0x5b/0x0 rrc: 3 type: IBT gid 0 flags: 0x40210000000000 nid: local remote: 0x0 expref: -99 pid: 8250 timeout: 0 lvb_type: 0 00000004:00200000:1.0:1776178221.736957:0:8250:0:(mdt_handler.c:912:mdt_pack_attr2body()) [0x2000013a1:0xd:0x0]: returning size 0 00000004:00200000:1.0:1776178221.736959:0:8250:0:(mdt_handler.c:1523:mdt_getattr_internal()) dirent count 0 stripe count 1 MDT count 2 00000004:00010000:1.0:1776178221.736968:0:8250:0:(mdt_handler.c:2353:mdt_getattr_name_lock()) ### Returning lock to client ns: mdt-lustre-MDT0000_UUID lock: 00000000336cd7d4/0xa40083ab9bf53a1f lrc: 3/1,0 mode: PR/PR res: [0x2000013a1:0xd:0x0].0x0 bits 0x5b/0x0 rrc: 3 type: IBT gid 0 flags: 0x40210000000000 nid: local remote: 0x0 expref: -99 pid: 8250 timeout: 0 lvb_type: 0 00010000:00010000:1.0:1776178221.736977:0:8250:0:(ldlm_lock.c:1064:ldlm_granted_list_add_lock()) ### About to add lock: ns: mdt-lustre-MDT0000_UUID lock: 00000000336cd7d4/0xa40083ab9bf53a1f lrc: 3/1,0 mode: PR/PR res: [0x2000013a1:0xd:0x0].0x0 bits 0x1b/0x0 rrc: 3 type: IBT gid 0 flags: 0x50210000000000 nid: local remote: 0x0 expref: -99 pid: 8250 timeout: 0 lvb_type: 0 00010000:00010000:1.0:1776178221.736993:0:8250:0:(ldlm_lock.c:216:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: mdt-lustre-MDT0000_UUID lock: 00000000499bab84/0xa40083ab9bf53a03 lrc: 0/0,0 mode: --/CR res: [0x2000013a1:0x9:0x0].0x0 bits 0x2/0x0 rrc: 3 type: IBT gid 0 flags: 0x44000000000000 nid: 192.168.204.3@tcp remote: 0x771f68aa1eb27c30 expref: 20 pid: 8250 timeout: 0 lvb_type: 0 00010000:00010000:1.0:1776178221.737006:0:8250:0:(ldlm_lockd.c:1484:ldlm_handle_enqueue0()) ### server-side enqueue handler, sending reply (err=0, rc=0) ns: mdt-lustre-MDT0000_UUID lock: 00000000336cd7d4/0xa40083ab9bf53a1f lrc: 3/0,0 mode: PR/PR res: [0x2000013a1:0xd:0x0].0x0 bits 0x1b/0x0 rrc: 3 type: IBT gid 0 flags: 0x40200000000000 nid: 192.168.204.3@tcp remote: 0x771f68aa1eb27c30 expref: 19 pid: 8250 timeout: 0 lvb_type: 0 00010000:00010000:1.0:1776178221.737026:0:8250:0:(ldlm_inodebits.c:94:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x2000013a1:0xd:0x0].0x0 (00000000713d9096) 00010000:00010000:1.0:1776178221.737028:0:8250:0:(ldlm_inodebits.c:96:ldlm_reprocess_inodebits_queue()) Hint 1b 00010000:00010000:1.0:1776178221.737031:0:8250:0:(ldlm_lockd.c:1565:ldlm_handle_enqueue0()) ### server-side enqueue handler END (lock 00000000336cd7d4, rc 0) 00010000:00000080:1.0:1776178221.737036:0:8250:0:(ldlm_lib.c:3181:target_committed_to_req()) @@@ not sending last_committed update (1/0) req@00000000f44a2faf x1862456979108544/t0(0) o101->b50963bd-8eba-4432-9a54-8888d00e3fa1@192.168.204.3@tcp:753/0 lens 576/696 e 0 to 0 dl 1776178268 ref 1 fl Interpret:/0/0 rc 0/0 job:'chmod.0' 00000100:00100000:1.0:1776178221.737144:0:8250:0:(service.c:2372:ptlrpc_server_handle_request()) Handled RPC req@00000000f44a2faf pname:cluuid+ref:pid:xid:nid:opc:job mdt00_003:b50963bd-8eba-4432-9a54-8888d00e3fa1+19:18364:x1862456979108544:12345-192.168.204.3@tcp:101:chmod.0 Request processed in 2880us (5983us total) trans 0 rc 0/0 00000100:00100000:1.0:1776178221.737152:0:8250:0:(nrs_fifo.c:241:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.204.3@tcp, seq: 6623 00010000:00000080:3.0:1776178221.738437:0:6256:0:(ldlm_lib.c:3181:target_committed_to_req()) @@@ not sending last_committed update (1/0) req@00000000be94e5d7 x1862456979108736/t0(0) o101->b50963bd-8eba-4432-9a54-8888d00e3fa1@192.168.204.3@tcp:753/0 lens 576/696 e 0 to 0 dl 1776178268 ref 1 fl Interpret:/0/0 rc 0/0 job:'chmod.0' 00000100:00100000:3.0:1776178221.738523:0:6256:0:(service.c:2372:ptlrpc_server_handle_request()) Handled RPC req@00000000be94e5d7 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_001:b50963bd-8eba-4432-9a54-8888d00e3fa1+18:18362:x1862456979108736:12345-192.168.204.3@tcp:101:chmod.0 Request processed in 2365us (7281us total) trans 0 rc 0/0 00000100:00100000:3.0:1776178221.738532:0:6256:0:(nrs_fifo.c:241:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.204.3@tcp, seq: 6625 00000100:00100000:2.0:1776178221.739315:0:3347:0:(events.c:361:request_in_callback()) peer: 12345-192.168.204.3@tcp (source: 12345-192.168.204.3@tcp) 00000100:00100000:3.0:1776178221.741284:0:11281:0:(service.c:2152:ptlrpc_server_handle_req_in()) got req x1862456979108800 00000100:00100000:3.0:1776178221.741398:0:11281:0:(nrs_fifo.c:179:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.204.3@tcp, seq: 6626 00000100:00100000:3.0:1776178221.741408:0:11281:0:(service.c:2324:ptlrpc_server_handle_request()) Handling RPC req@0000000044f82452 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_004:b50963bd-8eba-4432-9a54-8888d00e3fa1+18:18363:x1862456979108800:12345-192.168.204.3@tcp:101:chmod.0 00010000:00010000:3.0:1776178221.741442:0:11281:0:(ldlm_lockd.c:1248:ldlm_handle_enqueue0()) ### server-side enqueue handler START 00010000:00010000:3.0:1776178221.741480:0:11281:0:(ldlm_lockd.c:1318:ldlm_handle_enqueue0()) ### server-side enqueue handler, new lock created ns: mdt-lustre-MDT0000_UUID lock: 00000000c3ab77cd/0xa40083ab9bf53a26 lrc: 2/0,0 mode: --/CR res: [0x2000013a1:0x9:0x0].0x0 bits 0x0/0x0 rrc: 3 type: IBT gid 0 flags: 0x40000000000000 nid: local remote: 0x771f68aa1eb27c3e expref: -99 pid: 11281 timeout: 0 lvb_type: 0 00010000:00010000:3.0:1776178221.741559:0:11281:0:(ldlm_lock.c:766:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PR) ns: mdt-lustre-MDT0000_UUID lock: 0000000026fd08a4/0xa40083ab9bf53a2d lrc: 3/1,0 mode: --/PR res: [0x2000013a1:0xc:0x0].0x0 bits 0x0/0x0 rrc: 3 type: IBT gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 11281 timeout: 0 lvb_type: 0 00010000:00010000:3.0:1776178221.741574:0:11281:0:(ldlm_lock.c:1064:ldlm_granted_list_add_lock()) ### About to add lock: ns: mdt-lustre-MDT0000_UUID lock: 0000000026fd08a4/0xa40083ab9bf53a2d lrc: 3/1,0 mode: PR/PR res: [0x2000013a1:0xc:0x0].0x0 bits 0x5b/0x0 rrc: 3 type: IBT gid 0 flags: 0x50210000000000 nid: local remote: 0x0 expref: -99 pid: 11281 timeout: 0 lvb_type: 0 00010000:00010000:3.0:1776178221.741585:0:11281:0:(ldlm_request.c:528:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: mdt-lustre-MDT0000_UUID lock: 0000000026fd08a4/0xa40083ab9bf53a2d lrc: 3/1,0 mode: PR/PR res: [0x2000013a1:0xc:0x0].0x0 bits 0x5b/0x0 rrc: 3 type: IBT gid 0 flags: 0x40210000000000 nid: local remote: 0x0 expref: -99 pid: 11281 timeout: 0 lvb_type: 0 00000004:00200000:3.0:1776178221.742202:0:11281:0:(mdt_handler.c:912:mdt_pack_attr2body()) [0x2000013a1:0xc:0x0]: returning size 0 00000004:00200000:3.0:1776178221.742208:0:11281:0:(mdt_handler.c:1523:mdt_getattr_internal()) dirent count 0 stripe count 1 MDT count 2 00000004:00010000:3.0:1776178221.742239:0:11281:0:(mdt_handler.c:2353:mdt_getattr_name_lock()) ### Returning lock to client ns: mdt-lustre-MDT0000_UUID lock: 0000000026fd08a4/0xa40083ab9bf53a2d lrc: 3/1,0 mode: PR/PR res: [0x2000013a1:0xc:0x0].0x0 bits 0x5b/0x0 rrc: 3 type: IBT gid 0 flags: 0x40210000000000 nid: local remote: 0x0 expref: -99 pid: 11281 timeout: 0 lvb_type: 0 00010000:00010000:3.0:1776178221.742261:0:11281:0:(ldlm_lock.c:1064:ldlm_granted_list_add_lock()) ### About to add lock: ns: mdt-lustre-MDT0000_UUID lock: 0000000026fd08a4/0xa40083ab9bf53a2d lrc: 3/1,0 mode: PR/PR res: [0x2000013a1:0xc:0x0].0x0 bits 0x1b/0x0 rrc: 3 type: IBT gid 0 flags: 0x50210000000000 nid: local remote: 0x0 expref: -99 pid: 11281 timeout: 0 lvb_type: 0 00010000:00010000:3.0:1776178221.742295:0:11281:0:(ldlm_lock.c:216:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: mdt-lustre-MDT0000_UUID lock: 00000000c3ab77cd/0xa40083ab9bf53a26 lrc: 0/0,0 mode: --/CR res: [0x2000013a1:0x9:0x0].0x0 bits 0x2/0x0 rrc: 3 type: IBT gid 0 flags: 0x44000000000000 nid: 192.168.204.3@tcp remote: 0x771f68aa1eb27c3e expref: 20 pid: 11281 timeout: 0 lvb_type: 0 00010000:00010000:3.0:1776178221.742315:0:11281:0:(ldlm_lockd.c:1484:ldlm_handle_enqueue0()) ### server-side enqueue handler, sending reply (err=0, rc=0) ns: mdt-lustre-MDT0000_UUID lock: 0000000026fd08a4/0xa40083ab9bf53a2d lrc: 3/0,0 mode: PR/PR res: [0x2000013a1:0xc:0x0].0x0 bits 0x1b/0x0 rrc: 3 type: IBT gid 0 flags: 0x40200000000000 nid: 192.168.204.3@tcp remote: 0x771f68aa1eb27c3e expref: 19 pid: 11281 timeout: 0 lvb_type: 0 00010000:00010000:3.0:1776178221.742324:0:11281:0:(ldlm_inodebits.c:94:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x2000013a1:0xc:0x0].0x0 (0000000062690614) 00010000:00010000:3.0:1776178221.742327:0:11281:0:(ldlm_inodebits.c:96:ldlm_reprocess_inodebits_queue()) Hint 1b 00010000:00010000:3.0:1776178221.742332:0:11281:0:(ldlm_lockd.c:1565:ldlm_handle_enqueue0()) ### server-side enqueue handler END (lock 0000000026fd08a4, rc 0) 00010000:00000080:3.0:1776178221.742341:0:11281:0:(ldlm_lib.c:3181:target_committed_to_req()) @@@ not sending last_committed update (1/0) req@0000000044f82452 x1862456979108800/t0(0) o101->b50963bd-8eba-4432-9a54-8888d00e3fa1@192.168.204.3@tcp:753/0 lens 576/696 e 0 to 0 dl 1776178268 ref 1 fl Interpret:/0/0 rc 0/0 job:'chmod.0' 00000100:00100000:3.0:1776178221.742439:0:11281:0:(service.c:2372:ptlrpc_server_handle_request()) Handled RPC req@0000000044f82452 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_004:b50963bd-8eba-4432-9a54-8888d00e3fa1+19:18363:x1862456979108800:12345-192.168.204.3@tcp:101:chmod.0 Request processed in 1031us (3127us total) trans 0 rc 0/0 00000100:00100000:3.0:1776178221.742521:0:11281:0:(nrs_fifo.c:241:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.204.3@tcp, seq: 6626 00000100:00100000:1.0:1776178221.749437:0:3348:0:(events.c:361:request_in_callback()) peer: 12345-192.168.204.3@tcp (source: 12345-192.168.204.3@tcp) 00000100:00100000:1.0:1776178221.749626:0:3348:0:(events.c:361:request_in_callback()) peer: 12345-192.168.204.3@tcp (source: 12345-192.168.204.3@tcp) 00000100:00100000:1.0:1776178221.749709:0:3348:0:(events.c:361:request_in_callback()) peer: 12345-192.168.204.3@tcp (source: 12345-192.168.204.3@tcp) 00000100:00100000:3.0:1776178221.749734:0:20762:0:(service.c:2152:ptlrpc_server_handle_req_in()) got req x1862456979108864 00000100:00100000:3.0:1776178221.749806:0:20762:0:(service.c:2152:ptlrpc_server_handle_req_in()) got req x1862456979108928 00000100:00100000:3.0:1776178221.749828:0:20762:0:(nrs_fifo.c:179:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.204.3@tcp, seq: 5293 00000100:00100000:3.0:1776178221.749836:0:20762:0:(service.c:2324:ptlrpc_server_handle_request()) Handling RPC req@00000000f62fe06f pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_013:b50963bd-8eba-4432-9a54-8888d00e3fa1+6:2273:x1862456979108864:12345-192.168.204.3@tcp:101:chmod.0 00010000:00010000:3.0:1776178221.750117:0:20762:0:(ldlm_lockd.c:1248:ldlm_handle_enqueue0()) ### server-side enqueue handler START 00010000:00010000:3.0:1776178221.750140:0:20762:0:(ldlm_lockd.c:1318:ldlm_handle_enqueue0()) ### server-side enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 00000000a3795b7d/0xa40083ab9bf53a34 lrc: 2/0,0 mode: --/PR res: [0x46:0x0:0x0].0x0 rrc: 2 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x771f68aa1eb27c45 expref: -99 pid: 20762 timeout: 0 lvb_type: 0 00002000:00010000:3.0:1776178221.750168:0:20762:0:(ofd_lvb.c:170:ofd_lvbo_init()) res: [0x100000000:0x46:0x0] initial LVB size: 0, mtime: 0x69de5423, atime: 0x69de5423, ctime: 0x69de5423, blocks: 0x0 00010000:00010000:3.0:1776178221.750194:0:20762:0:(ldlm_extent.c:318:ldlm_check_contention()) contended locks = 0 00010000:00010000:3.0:1776178221.750197:0:20762:0:(ldlm_extent.c:318:ldlm_check_contention()) contended locks = 0 00010000:00010000:3.0:1776178221.750208:0:20762:0:(ldlm_resource.c:1650:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 00000000a3795b7d/0xa40083ab9bf53a34 lrc: 3/0,0 mode: PR/PR res: [0x46:0x0:0x0].0x0 rrc: 2 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x40000000000000 nid: 192.168.204.3@tcp remote: 0x771f68aa1eb27c45 expref: 7 pid: 20762 timeout: 0 lvb_type: 1 00010000:00010000:3.0:1776178221.750231:0:20762:0:(ldlm_lockd.c:1484:ldlm_handle_enqueue0()) ### server-side enqueue handler, sending reply (err=0, rc=0) ns: filter-lustre-OST0000_UUID lock: 00000000a3795b7d/0xa40083ab9bf53a34 lrc: 3/0,0 mode: PR/PR res: [0x46:0x0:0x0].0x0 rrc: 2 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x40000000000000 nid: 192.168.204.3@tcp remote: 0x771f68aa1eb27c45 expref: 7 pid: 20762 timeout: 0 lvb_type: 1 00010000:00010000:3.0:1776178221.750247:0:20762:0:(ldlm_lockd.c:1565:ldlm_handle_enqueue0()) ### server-side enqueue handler END (lock 00000000a3795b7d, rc 0) 00000100:00100000:3.0:1776178221.750618:0:20762:0:(service.c:2372:ptlrpc_server_handle_request()) Handled RPC req@00000000f62fe06f pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_013:b50963bd-8eba-4432-9a54-8888d00e3fa1+7:2273:x1862456979108864:12345-192.168.204.3@tcp:101:chmod.0 Request processed in 780us (1268us total) trans 0 rc 0/0 00000100:00100000:3.0:1776178221.750627:0:20762:0:(nrs_fifo.c:241:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.204.3@tcp, seq: 5293 00000100:00100000:3.0:1776178221.750650:0:20762:0:(nrs_fifo.c:179:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.204.3@tcp, seq: 5294 00000100:00100000:3.0:1776178221.750656:0:20762:0:(service.c:2324:ptlrpc_server_handle_request()) Handling RPC req@00000000a562d27c pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_013:b50963bd-8eba-4432-9a54-8888d00e3fa1+6:2271:x1862456979108928:12345-192.168.204.3@tcp:101:chmod.0 00010000:00010000:3.0:1776178221.750665:0:20762:0:(ldlm_lockd.c:1248:ldlm_handle_enqueue0()) ### server-side enqueue handler START 00010000:00010000:3.0:1776178221.750677:0:20762:0:(ldlm_lockd.c:1318:ldlm_handle_enqueue0()) ### server-side enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 0000000067990ad0/0xa40083ab9bf53a3b lrc: 2/0,0 mode: --/PR res: [0x47:0x0:0x0].0x0 rrc: 2 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x771f68aa1eb27c4c expref: -99 pid: 20762 timeout: 0 lvb_type: 0 00002000:00010000:3.0:1776178221.750693:0:20762:0:(ofd_lvb.c:170:ofd_lvbo_init()) res: [0x100000000:0x47:0x0] initial LVB size: 0, mtime: 0x69de5423, atime: 0x69de5423, ctime: 0x69de5423, blocks: 0x0 00010000:00010000:3.0:1776178221.750704:0:20762:0:(ldlm_extent.c:318:ldlm_check_contention()) contended locks = 0 00010000:00010000:3.0:1776178221.750706:0:20762:0:(ldlm_extent.c:318:ldlm_check_contention()) contended locks = 0 00010000:00010000:3.0:1776178221.750709:0:20762:0:(ldlm_resource.c:1650:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 0000000067990ad0/0xa40083ab9bf53a3b lrc: 3/0,0 mode: PR/PR res: [0x47:0x0:0x0].0x0 rrc: 2 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x40000000000000 nid: 192.168.204.3@tcp remote: 0x771f68aa1eb27c4c expref: 7 pid: 20762 timeout: 0 lvb_type: 1 00010000:00010000:3.0:1776178221.750718:0:20762:0:(ldlm_lockd.c:1484:ldlm_handle_enqueue0()) ### server-side enqueue handler, sending reply (err=0, rc=0) ns: filter-lustre-OST0000_UUID lock: 0000000067990ad0/0xa40083ab9bf53a3b lrc: 3/0,0 mode: PR/PR res: [0x47:0x0:0x0].0x0 rrc: 2 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x40000000000000 nid: 192.168.204.3@tcp remote: 0x771f68aa1eb27c4c expref: 7 pid: 20762 timeout: 0 lvb_type: 1 00010000:00010000:3.0:1776178221.750734:0:20762:0:(ldlm_lockd.c:1565:ldlm_handle_enqueue0()) ### server-side enqueue handler END (lock 0000000067990ad0, rc 0) 00000100:00100000:3.0:1776178221.750752:0:20762:0:(service.c:2372:ptlrpc_server_handle_request()) Handled RPC req@00000000a562d27c pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_013:b50963bd-8eba-4432-9a54-8888d00e3fa1+7:2271:x1862456979108928:12345-192.168.204.3@tcp:101:chmod.0 Request processed in 97us (1127us total) trans 0 rc 0/0 00000100:00100000:3.0:1776178221.750758:0:20762:0:(nrs_fifo.c:241:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.204.3@tcp, seq: 5294 00000100:00100000:1.0:1776178221.752649:0:3348:0:(events.c:361:request_in_callback()) peer: 12345-192.168.204.3@tcp (source: 12345-192.168.204.3@tcp) 00000100:00100000:1.0:1776178221.752797:0:20755:0:(service.c:2152:ptlrpc_server_handle_req_in()) got req x1862456979108992 00000100:00100000:1.0:1776178221.753143:0:20755:0:(nrs_fifo.c:179:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.204.3@tcp, seq: 5295 00000100:00100000:1.0:1776178221.753154:0:20755:0:(service.c:2324:ptlrpc_server_handle_request()) Handling RPC req@00000000c38303b3 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_006:b50963bd-8eba-4432-9a54-8888d00e3fa1+6:2270:x1862456979108992:12345-192.168.204.3@tcp:101:chmod.0 00010000:00010000:1.0:1776178221.753180:0:20755:0:(ldlm_lockd.c:1248:ldlm_handle_enqueue0()) ### server-side enqueue handler START 00010000:00010000:1.0:1776178221.753209:0:20755:0:(ldlm_lockd.c:1318:ldlm_handle_enqueue0()) ### server-side enqueue handler, new lock created ns: filter-lustre-OST0001_UUID lock: 0000000015189b1f/0xa40083ab9bf53a42 lrc: 2/0,0 mode: --/PR res: [0x46:0x0:0x0].0x0 rrc: 2 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x771f68aa1eb27c53 expref: -99 pid: 20755 timeout: 0 lvb_type: 0 00002000:00010000:1.0:1776178221.753239:0:20755:0:(ofd_lvb.c:170:ofd_lvbo_init()) res: [0x100010000:0x46:0x0] initial LVB size: 0, mtime: 0x69de5423, atime: 0x69de5423, ctime: 0x69de5423, blocks: 0x0 00010000:00010000:1.0:1776178221.753265:0:20755:0:(ldlm_extent.c:318:ldlm_check_contention()) contended locks = 0 00010000:00010000:1.0:1776178221.753267:0:20755:0:(ldlm_extent.c:318:ldlm_check_contention()) contended locks = 0 00010000:00010000:1.0:1776178221.753282:0:20755:0:(ldlm_resource.c:1650:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0001_UUID lock: 0000000015189b1f/0xa40083ab9bf53a42 lrc: 3/0,0 mode: PR/PR res: [0x46:0x0:0x0].0x0 rrc: 2 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x40000000000000 nid: 192.168.204.3@tcp remote: 0x771f68aa1eb27c53 expref: 7 pid: 20755 timeout: 0 lvb_type: 1 00010000:00010000:1.0:1776178221.753303:0:20755:0:(ldlm_lockd.c:1484:ldlm_handle_enqueue0()) ### server-side enqueue handler, sending reply (err=0, rc=0) ns: filter-lustre-OST0001_UUID lock: 0000000015189b1f/0xa40083ab9bf53a42 lrc: 3/0,0 mode: PR/PR res: [0x46:0x0:0x0].0x0 rrc: 2 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x40000000000000 nid: 192.168.204.3@tcp remote: 0x771f68aa1eb27c53 expref: 7 pid: 20755 timeout: 0 lvb_type: 1 00010000:00010000:1.0:1776178221.753318:0:20755:0:(ldlm_lockd.c:1565:ldlm_handle_enqueue0()) ### server-side enqueue handler END (lock 0000000015189b1f, rc 0) 00000100:00100000:1.0:1776178221.753399:0:20755:0:(service.c:2372:ptlrpc_server_handle_request()) Handled RPC req@00000000c38303b3 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_006:b50963bd-8eba-4432-9a54-8888d00e3fa1+7:2270:x1862456979108992:12345-192.168.204.3@tcp:101:chmod.0 Request processed in 247us (3691us total) trans 0 rc 0/0 00000100:00100000:1.0:1776178221.753407:0:20755:0:(nrs_fifo.c:241:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.204.3@tcp, seq: 5295 00000100:00100000:1.0:1776178221.755267:0:11281:0:(service.c:2152:ptlrpc_server_handle_req_in()) got req x1862456979109056 00000100:00100000:1.0:1776178221.755335:0:6256:0:(nrs_fifo.c:179:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.204.3@tcp, seq: 6627 00000100:00100000:1.0:1776178221.755343:0:6256:0:(service.c:2324:ptlrpc_server_handle_request()) Handling RPC req@000000004c88c002 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_001:b50963bd-8eba-4432-9a54-8888d00e3fa1+19:18367:x1862456979109056:12345-192.168.204.3@tcp:101:chmod.0 00010000:00010000:1.0:1776178221.755380:0:6256:0:(ldlm_lockd.c:1248:ldlm_handle_enqueue0()) ### server-side enqueue handler START 00010000:00010000:1.0:1776178221.755404:0:6256:0:(ldlm_lockd.c:1318:ldlm_handle_enqueue0()) ### server-side enqueue handler, new lock created ns: mdt-lustre-MDT0000_UUID lock: 00000000d0f9e56c/0xa40083ab9bf53a49 lrc: 2/0,0 mode: --/CR res: [0x2000013a1:0x9:0x0].0x0 bits 0x0/0x0 rrc: 3 type: IBT gid 0 flags: 0x40000000000000 nid: local remote: 0x771f68aa1eb27c5a expref: -99 pid: 6256 timeout: 0 lvb_type: 0 00010000:00010000:1.0:1776178221.756513:0:6256:0:(ldlm_lock.c:766:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PR) ns: mdt-lustre-MDT0000_UUID lock: 00000000a44d8583/0xa40083ab9bf53a50 lrc: 3/1,0 mode: --/PR res: [0x2000013a1:0x10:0x0].0x0 bits 0x0/0x0 rrc: 3 type: IBT gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 6256 timeout: 0 lvb_type: 0 00010000:00010000:1.0:1776178221.756546:0:6256:0:(ldlm_lock.c:1064:ldlm_granted_list_add_lock()) ### About to add lock: ns: mdt-lustre-MDT0000_UUID lock: 00000000a44d8583/0xa40083ab9bf53a50 lrc: 3/1,0 mode: PR/PR res: [0x2000013a1:0x10:0x0].0x0 bits 0x5b/0x0 rrc: 3 type: IBT gid 0 flags: 0x50210000000000 nid: local remote: 0x0 expref: -99 pid: 6256 timeout: 0 lvb_type: 0 00010000:00080000:0.0:1776178221.756556:0:6257:0:(ldlm_lib.c:3333:target_send_reply()) Schedule reply immediately 00010000:00010000:1.0:1776178221.756560:0:6256:0:(ldlm_request.c:528:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: mdt-lustre-MDT0000_UUID lock: 00000000a44d8583/0xa40083ab9bf53a50 lrc: 3/1,0 mode: PR/PR res: [0x2000013a1:0x10:0x0].0x0 bits 0x5b/0x0 rrc: 3 type: IBT gid 0 flags: 0x40210000000000 nid: local remote: 0x0 expref: -99 pid: 6256 timeout: 0 lvb_type: 0 00000100:00100000:0.0:1776178221.756578:0:6257:0:(service.c:2372:ptlrpc_server_handle_request()) Handled RPC req@00000000cc7f07e3 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_002:b50963bd-8eba-4432-9a54-8888d00e3fa1+20:18361:x1862456979108672:12345-192.168.204.3@tcp:36:chmod.0 Request processed in 22106us (25356us total) trans 12884901975 rc 0/0 00000100:00100000:0.0:1776178221.756587:0:6257:0:(nrs_fifo.c:241:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.204.3@tcp, seq: 6624 00000004:00200000:1.0:1776178221.756628:0:6256:0:(mdt_handler.c:912:mdt_pack_attr2body()) [0x2000013a1:0x10:0x0]: returning size 0 00000004:00200000:1.0:1776178221.756633:0:6256:0:(mdt_handler.c:1523:mdt_getattr_internal()) dirent count 0 stripe count 1 MDT count 2 00000004:00010000:1.0:1776178221.756660:0:6256:0:(mdt_handler.c:2353:mdt_getattr_name_lock()) ### Returning lock to client ns: mdt-lustre-MDT0000_UUID lock: 00000000a44d8583/0xa40083ab9bf53a50 lrc: 3/1,0 mode: PR/PR res: [0x2000013a1:0x10:0x0].0x0 bits 0x5b/0x0 rrc: 3 type: IBT gid 0 flags: 0x40210000000000 nid: local remote: 0x0 expref: -99 pid: 6256 timeout: 0 lvb_type: 0 00010000:00010000:1.0:1776178221.756674:0:6256:0:(ldlm_lock.c:1064:ldlm_granted_list_add_lock()) ### About to add lock: ns: mdt-lustre-MDT0000_UUID lock: 00000000a44d8583/0xa40083ab9bf53a50 lrc: 3/1,0 mode: PR/PR res: [0x2000013a1:0x10:0x0].0x0 bits 0x1b/0x0 rrc: 3 type: IBT gid 0 flags: 0x50210000000000 nid: local remote: 0x0 expref: -99 pid: 6256 timeout: 0 lvb_type: 0 00010000:00010000:1.0:1776178221.756704:0:6256:0:(ldlm_lock.c:216:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: mdt-lustre-MDT0000_UUID lock: 00000000d0f9e56c/0xa40083ab9bf53a49 lrc: 0/0,0 mode: --/CR res: [0x2000013a1:0x9:0x0].0x0 bits 0x2/0x0 rrc: 3 type: IBT gid 0 flags: 0x44000000000000 nid: 192.168.204.3@tcp remote: 0x771f68aa1eb27c5a expref: 20 pid: 6256 timeout: 0 lvb_type: 0 00010000:00010000:1.0:1776178221.756731:0:6256:0:(ldlm_lockd.c:1484:ldlm_handle_enqueue0()) ### server-side enqueue handler, sending reply (err=0, rc=0) ns: mdt-lustre-MDT0000_UUID lock: 00000000a44d8583/0xa40083ab9bf53a50 lrc: 3/0,0 mode: PR/PR res: [0x2000013a1:0x10:0x0].0x0 bits 0x1b/0x0 rrc: 3 type: IBT gid 0 flags: 0x40200000000000 nid: 192.168.204.3@tcp remote: 0x771f68aa1eb27c5a expref: 19 pid: 6256 timeout: 0 lvb_type: 0 00010000:00010000:1.0:1776178221.756739:0:6256:0:(ldlm_inodebits.c:94:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x2000013a1:0x10:0x0].0x0 (000000003da5851e) 00010000:00010000:1.0:1776178221.756742:0:6256:0:(ldlm_inodebits.c:96:ldlm_reprocess_inodebits_queue()) Hint 1b 00010000:00010000:1.0:1776178221.756746:0:6256:0:(ldlm_lockd.c:1565:ldlm_handle_enqueue0()) ### server-side enqueue handler END (lock 00000000a44d8583, rc 0) 00010000:00000080:1.0:1776178221.756753:0:6256:0:(ldlm_lib.c:3181:target_committed_to_req()) @@@ not sending last_committed update (1/0) req@000000004c88c002 x1862456979109056/t0(0) o101->b50963bd-8eba-4432-9a54-8888d00e3fa1@192.168.204.3@tcp:753/0 lens 576/696 e 0 to 0 dl 1776178268 ref 1 fl Interpret:/0/0 rc 0/0 job:'chmod.0' 00000100:00100000:1.0:1776178221.756845:0:6256:0:(service.c:2372:ptlrpc_server_handle_request()) Handled RPC req@000000004c88c002 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_001:b50963bd-8eba-4432-9a54-8888d00e3fa1+19:18367:x1862456979109056:12345-192.168.204.3@tcp:101:chmod.0 Request processed in 1502us (4209us total) trans 0 rc 0/0 00000100:00100000:1.0:1776178221.756862:0:6256:0:(nrs_fifo.c:241:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.204.3@tcp, seq: 6627 00010000:00010000:0.0:1776178221.756895:0:3339:0:(ldlm_lock.c:1064:ldlm_granted_list_add_lock()) ### About to add lock: ns: mdt-lustre-MDT0000_UUID lock: 0000000015ae4ee6/0xa40083ab9bf53a0a lrc: 3/0,1 mode: COS/COS res: [0x2000013a1:0xa:0x0].0x0 bits 0x13/0x0 rrc: 2 type: IBT gid 0 flags: 0x50210000000000 nid: local remote: 0x0 expref: -99 pid: 6257 timeout: 0 lvb_type: 0 00010000:00010000:0.0:1776178221.756904:0:3339:0:(ldlm_inodebits.c:94:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x2000013a1:0xa:0x0].0x0 (00000000c9b683f9) 00010000:00010000:0.0:1776178221.756907:0:3339:0:(ldlm_inodebits.c:96:ldlm_reprocess_inodebits_queue()) Hint 13 00000100:00100000:3.0:1776178221.757181:0:3347:0:(events.c:361:request_in_callback()) peer: 12345-192.168.204.3@tcp (source: 12345-192.168.204.3@tcp) 00000100:00100000:0.0:1776178221.757906:0:6257:0:(service.c:2152:ptlrpc_server_handle_req_in()) got req x1862456979109120 00000100:00100000:0.0:1776178221.757959:0:6257:0:(nrs_fifo.c:179:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.204.3@tcp, seq: 6628 00000100:00100000:0.0:1776178221.757967:0:6257:0:(service.c:2324:ptlrpc_server_handle_request()) Handling RPC req@0000000044d75ea3 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_002:b50963bd-8eba-4432-9a54-8888d00e3fa1+19:18365:x1862456979109120:12345-192.168.204.3@tcp:101:chmod.0 00010000:00010000:0.0:1776178221.757992:0:6257:0:(ldlm_lockd.c:1248:ldlm_handle_enqueue0()) ### server-side enqueue handler START 00010000:00010000:0.0:1776178221.758028:0:6257:0:(ldlm_lockd.c:1318:ldlm_handle_enqueue0()) ### server-side enqueue handler, new lock created ns: mdt-lustre-MDT0000_UUID lock: 000000001735c8e5/0xa40083ab9bf53a57 lrc: 2/0,0 mode: --/CR res: [0x2000013a1:0x9:0x0].0x0 bits 0x0/0x0 rrc: 3 type: IBT gid 0 flags: 0x40000000000000 nid: local remote: 0x771f68aa1eb27c61 expref: -99 pid: 6257 timeout: 0 lvb_type: 0 00010000:00010000:0.0:1776178221.758115:0:6257:0:(ldlm_lock.c:766:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PR) ns: mdt-lustre-MDT0000_UUID lock: 00000000b4e1b221/0xa40083ab9bf53a5e lrc: 3/1,0 mode: --/PR res: [0x2000013a1:0xe:0x0].0x0 bits 0x0/0x0 rrc: 3 type: IBT gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 6257 timeout: 0 lvb_type: 0 00010000:00010000:0.0:1776178221.758129:0:6257:0:(ldlm_lock.c:1064:ldlm_granted_list_add_lock()) ### About to add lock: ns: mdt-lustre-MDT0000_UUID lock: 00000000b4e1b221/0xa40083ab9bf53a5e lrc: 3/1,0 mode: PR/PR res: [0x2000013a1:0xe:0x0].0x0 bits 0x5b/0x0 rrc: 3 type: IBT gid 0 flags: 0x50210000000000 nid: local remote: 0x0 expref: -99 pid: 6257 timeout: 0 lvb_type: 0 00000100:00100000:1.0:1776178221.758134:0:3346:0:(events.c:361:request_in_callback()) peer: 12345-192.168.204.3@tcp (source: 12345-192.168.204.3@tcp) 00010000:00010000:0.0:1776178221.758139:0:6257:0:(ldlm_request.c:528:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: mdt-lustre-MDT0000_UUID lock: 00000000b4e1b221/0xa40083ab9bf53a5e lrc: 3/1,0 mode: PR/PR res: [0x2000013a1:0xe:0x0].0x0 bits 0x5b/0x0 rrc: 3 type: IBT gid 0 flags: 0x40210000000000 nid: local remote: 0x0 expref: -99 pid: 6257 timeout: 0 lvb_type: 0 00000100:00100000:1.0:1776178221.758162:0:3346:0:(events.c:361:request_in_callback()) peer: 12345-192.168.204.3@tcp (source: 12345-192.168.204.3@tcp) 00000100:00100000:1.0:1776178221.758186:0:6255:0:(service.c:2152:ptlrpc_server_handle_req_in()) got req x1862456979109184 00000004:00200000:0.0:1776178221.758190:0:6257:0:(mdt_handler.c:912:mdt_pack_attr2body()) [0x2000013a1:0xe:0x0]: returning size 0 00000004:00200000:0.0:1776178221.758194:0:6257:0:(mdt_handler.c:1523:mdt_getattr_internal()) dirent count 0 stripe count 1 MDT count 2 00000100:00100000:1.0:1776178221.758207:0:6255:0:(service.c:2152:ptlrpc_server_handle_req_in()) got req x1862456979109312 00000004:00010000:0.0:1776178221.758216:0:6257:0:(mdt_handler.c:2353:mdt_getattr_name_lock()) ### Returning lock to client ns: mdt-lustre-MDT0000_UUID lock: 00000000b4e1b221/0xa40083ab9bf53a5e lrc: 3/1,0 mode: PR/PR res: [0x2000013a1:0xe:0x0].0x0 bits 0x5b/0x0 rrc: 3 type: IBT gid 0 flags: 0x40210000000000 nid: local remote: 0x0 expref: -99 pid: 6257 timeout: 0 lvb_type: 0 00000100:00100000:1.0:1776178221.758217:0:6255:0:(nrs_fifo.c:179:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.204.3@tcp, seq: 6629 00000100:00100000:1.0:1776178221.758222:0:6255:0:(service.c:2324:ptlrpc_server_handle_request()) Handling RPC req@0000000087a31eb2 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_000:b50963bd-8eba-4432-9a54-8888d00e3fa1+22:18366:x1862456979109184:12345-192.168.204.3@tcp:101:chmod.0 00010000:00010000:0.0:1776178221.758229:0:6257:0:(ldlm_lock.c:1064:ldlm_granted_list_add_lock()) ### About to add lock: ns: mdt-lustre-MDT0000_UUID lock: 00000000b4e1b221/0xa40083ab9bf53a5e lrc: 3/1,0 mode: PR/PR res: [0x2000013a1:0xe:0x0].0x0 bits 0x1b/0x0 rrc: 3 type: IBT gid 0 flags: 0x50210000000000 nid: local remote: 0x0 expref: -99 pid: 6257 timeout: 0 lvb_type: 0 00010000:00010000:1.0:1776178221.758229:0:6255:0:(ldlm_lockd.c:1248:ldlm_handle_enqueue0()) ### server-side enqueue handler START 00010000:00010000:1.0:1776178221.758236:0:6255:0:(ldlm_lockd.c:1318:ldlm_handle_enqueue0()) ### server-side enqueue handler, new lock created ns: mdt-lustre-MDT0000_UUID lock: 00000000b2e83690/0xa40083ab9bf53a65 lrc: 2/0,0 mode: --/CR res: [0x2000013a1:0x9:0x0].0x0 bits 0x0/0x0 rrc: 4 type: IBT gid 0 flags: 0x40000000000000 nid: local remote: 0x771f68aa1eb27c68 expref: -99 pid: 6255 timeout: 0 lvb_type: 0 00010000:00010000:1.0:1776178221.758269:0:6255:0:(ldlm_lock.c:766:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PR) ns: mdt-lustre-MDT0000_UUID lock: 0000000097975a3b/0xa40083ab9bf53a6c lrc: 3/1,0 mode: --/PR res: [0x2000013a1:0xf:0x0].0x0 bits 0x0/0x0 rrc: 3 type: IBT gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 6255 timeout: 0 lvb_type: 0 00010000:00010000:1.0:1776178221.758278:0:6255:0:(ldlm_lock.c:1064:ldlm_granted_list_add_lock()) ### About to add lock: ns: mdt-lustre-MDT0000_UUID lock: 0000000097975a3b/0xa40083ab9bf53a6c lrc: 3/1,0 mode: PR/PR res: [0x2000013a1:0xf:0x0].0x0 bits 0x5b/0x0 rrc: 3 type: IBT gid 0 flags: 0x50210000000000 nid: local remote: 0x0 expref: -99 pid: 6255 timeout: 0 lvb_type: 0 00010000:00010000:1.0:1776178221.758292:0:6255:0:(ldlm_request.c:528:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: mdt-lustre-MDT0000_UUID lock: 0000000097975a3b/0xa40083ab9bf53a6c lrc: 3/1,0 mode: PR/PR res: [0x2000013a1:0xf:0x0].0x0 bits 0x5b/0x0 rrc: 3 type: IBT gid 0 flags: 0x40210000000000 nid: local remote: 0x0 expref: -99 pid: 6255 timeout: 0 lvb_type: 0 00000004:00200000:1.0:1776178221.758316:0:6255:0:(mdt_handler.c:912:mdt_pack_attr2body()) [0x2000013a1:0xf:0x0]: returning size 0 00000004:00200000:1.0:1776178221.758318:0:6255:0:(mdt_handler.c:1523:mdt_getattr_internal()) dirent count 0 stripe count 1 MDT count 2 00000004:00010000:1.0:1776178221.758326:0:6255:0:(mdt_handler.c:2353:mdt_getattr_name_lock()) ### Returning lock to client ns: mdt-lustre-MDT0000_UUID lock: 0000000097975a3b/0xa40083ab9bf53a6c lrc: 3/1,0 mode: PR/PR res: [0x2000013a1:0xf:0x0].0x0 bits 0x5b/0x0 rrc: 3 type: IBT gid 0 flags: 0x40210000000000 nid: local remote: 0x0 expref: -99 pid: 6255 timeout: 0 lvb_type: 0 00010000:00010000:1.0:1776178221.758334:0:6255:0:(ldlm_lock.c:1064:ldlm_granted_list_add_lock()) ### About to add lock: ns: mdt-lustre-MDT0000_UUID lock: 0000000097975a3b/0xa40083ab9bf53a6c lrc: 3/1,0 mode: PR/PR res: [0x2000013a1:0xf:0x0].0x0 bits 0x1b/0x0 rrc: 3 type: IBT gid 0 flags: 0x50210000000000 nid: local remote: 0x0 expref: -99 pid: 6255 timeout: 0 lvb_type: 0 00010000:00010000:1.0:1776178221.758355:0:6255:0:(ldlm_lock.c:216:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: mdt-lustre-MDT0000_UUID lock: 00000000b2e83690/0xa40083ab9bf53a65 lrc: 0/0,0 mode: --/CR res: [0x2000013a1:0x9:0x0].0x0 bits 0x2/0x0 rrc: 4 type: IBT gid 0 flags: 0x44000000000000 nid: 192.168.204.3@tcp remote: 0x771f68aa1eb27c68 expref: 24 pid: 6255 timeout: 0 lvb_type: 0 00010000:00010000:1.0:1776178221.758385:0:6255:0:(ldlm_lockd.c:1484:ldlm_handle_enqueue0()) ### server-side enqueue handler, sending reply (err=0, rc=0) ns: mdt-lustre-MDT0000_UUID lock: 0000000097975a3b/0xa40083ab9bf53a6c lrc: 3/0,0 mode: PR/PR res: [0x2000013a1:0xf:0x0].0x0 bits 0x1b/0x0 rrc: 3 type: IBT gid 0 flags: 0x40200000000000 nid: 192.168.204.3@tcp remote: 0x771f68aa1eb27c68 expref: 23 pid: 6255 timeout: 0 lvb_type: 0 00010000:00010000:1.0:1776178221.758393:0:6255:0:(ldlm_inodebits.c:94:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x2000013a1:0xf:0x0].0x0 (00000000a5b77ef7) 00010000:00010000:1.0:1776178221.758396:0:6255:0:(ldlm_inodebits.c:96:ldlm_reprocess_inodebits_queue()) Hint 1b 00010000:00010000:1.0:1776178221.758399:0:6255:0:(ldlm_lockd.c:1565:ldlm_handle_enqueue0()) ### server-side enqueue handler END (lock 0000000097975a3b, rc 0) 00010000:00000080:1.0:1776178221.758406:0:6255:0:(ldlm_lib.c:3181:target_committed_to_req()) @@@ not sending last_committed update (1/0) req@0000000087a31eb2 x1862456979109184/t0(0) o101->b50963bd-8eba-4432-9a54-8888d00e3fa1@192.168.204.3@tcp:753/0 lens 576/696 e 0 to 0 dl 1776178268 ref 1 fl Interpret:/0/0 rc 0/0 job:'chmod.0' 00000100:00100000:3.0:1776178221.758741:0:6256:0:(nrs_fifo.c:179:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.204.3@tcp, seq: 6630 00000100:00100000:3.0:1776178221.758747:0:6256:0:(service.c:2324:ptlrpc_server_handle_request()) Handling RPC req@0000000008605f3d pname:cluuid+ref:pid:xid:nid:opc:job mdt00_001:b50963bd-8eba-4432-9a54-8888d00e3fa1+23:18362:x1862456979109312:12345-192.168.204.3@tcp:36:chmod.0 00010000:00010000:3.0:1776178221.758789:0:6256:0:(ldlm_lockd.c:1720:ldlm_request_cancel()) ### server-side cancel handler START: 2 locks, starting at 0 00010000:00010000:3.0:1776178221.758807:0:6256:0:(ldlm_lock.c:216:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: mdt-lustre-MDT0000_UUID lock: 0000000055220ae2/0xa40083ab9bf53a18 lrc: 0/0,0 mode: --/PR res: [0x2000013a1:0xb:0x0].0x0 bits 0x1b/0x0 rrc: 4 type: IBT gid 0 flags: 0x44a01000000000 nid: 192.168.204.3@tcp remote: 0x771f68aa1eb27c37 expref: 23 pid: 6256 timeout: 0 lvb_type: 0 00010000:00010000:3.0:1776178221.758818:0:6256:0:(ldlm_lock.c:216:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: mdt-lustre-MDT0000_UUID lock: 000000001d465f75/0xa40083ab9bf538b3 lrc: 0/0,0 mode: --/CR res: [0x2000013a1:0xb:0x0].0x0 bits 0x9/0x0 rrc: 3 type: IBT gid 0 flags: 0x44a01000000000 nid: 192.168.204.3@tcp remote: 0x771f68aa1eb27bea expref: 22 pid: 6256 timeout: 0 lvb_type: 0 00010000:00010000:3.0:1776178221.758826:0:6256:0:(ldlm_inodebits.c:94:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x2000013a1:0xb:0x0].0x0 (000000003275315d) 00010000:00010000:3.0:1776178221.758834:0:6256:0:(ldlm_lockd.c:1782:ldlm_request_cancel()) ### server-side cancel handler END 00010000:00010000:3.0:1776178221.758868:0:6256:0:(ldlm_lock.c:766:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: mdt-lustre-MDT0000_UUID lock: 0000000039ef0a83/0xa40083ab9bf53a73 lrc: 3/0,1 mode: --/PW res: [0x2000013a1:0xb:0x0].0x0 bits 0x0/0x0 rrc: 2 type: IBT gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 6256 timeout: 0 lvb_type: 0 00010000:00010000:3.0:1776178221.758876:0:6256:0:(ldlm_lock.c:1064:ldlm_granted_list_add_lock()) ### About to add lock: ns: mdt-lustre-MDT0000_UUID lock: 0000000039ef0a83/0xa40083ab9bf53a73 lrc: 3/0,1 mode: PW/PW res: [0x2000013a1:0xb:0x0].0x0 bits 0x13/0x0 rrc: 2 type: IBT gid 0 flags: 0x50210000000000 nid: local remote: 0x0 expref: -99 pid: 6256 timeout: 0 lvb_type: 0 00010000:00010000:3.0:1776178221.758883:0:6256:0:(ldlm_request.c:528:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: mdt-lustre-MDT0000_UUID lock: 0000000039ef0a83/0xa40083ab9bf53a73 lrc: 3/0,1 mode: PW/PW res: [0x2000013a1:0xb:0x0].0x0 bits 0x13/0x0 rrc: 2 type: IBT gid 0 flags: 0x40210000000000 nid: local remote: 0x0 expref: -99 pid: 6256 timeout: 0 lvb_type: 0 00000004:00010000:3.0:1776178221.759055:0:6256:0:(mdt_handler.c:3992:mdt_save_lock()) ### save lock request 0000000008605f3d reply state 000000007ac84be2 transno 12884901976 ns: mdt-lustre-MDT0000_UUID lock: 0000000039ef0a83/0xa40083ab9bf53a73 lrc: 3/0,1 mode: PW/PW res: [0x2000013a1:0xb:0x0].0x0 bits 0x13/0x0 rrc: 2 type: IBT gid 0 flags: 0x40210000000000 nid: local remote: 0x0 expref: -99 pid: 6256 timeout: 0 lvb_type: 0 00010000:00000080:3.0:1776178221.759099:0:6256:0:(ldlm_lib.c:3181:target_committed_to_req()) @@@ not sending last_committed update (1/0) req@0000000008605f3d x1862456979109312/t12884901976(0) o36->b50963bd-8eba-4432-9a54-8888d00e3fa1@192.168.204.3@tcp:753/0 lens 488/456 e 0 to 0 dl 1776178268 ref 1 fl Interpret:/0/0 rc 0/0 job:'chmod.0' 00010000:02000000:3.0:1776178221.759110:0:6256:0:(libcfs_fail.h:96:cfs_fail_check_set()) *** cfs_fail_loc=15a, val=0*** 00010000:00080000:3.0:1776178221.759114:0:6256:0:(ldlm_lib.c:3333:target_send_reply()) Schedule reply immediately 00000100:00100000:3.0:1776178221.759125:0:6256:0:(service.c:2372:ptlrpc_server_handle_request()) Handled RPC req@0000000008605f3d pname:cluuid+ref:pid:xid:nid:opc:job mdt00_001:b50963bd-8eba-4432-9a54-8888d00e3fa1+23:18362:x1862456979109312:12345-192.168.204.3@tcp:36:chmod.0 Request processed in 378us (963us total) trans 12884901976 rc 0/0 00000100:00100000:3.0:1776178221.759131:0:6256:0:(nrs_fifo.c:241:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.204.3@tcp, seq: 6630 00010000:00010000:3.0:1776178221.759163:0:3340:0:(ldlm_lock.c:1064:ldlm_granted_list_add_lock()) ### About to add lock: ns: mdt-lustre-MDT0000_UUID lock: 0000000039ef0a83/0xa40083ab9bf53a73 lrc: 3/0,1 mode: COS/COS res: [0x2000013a1:0xb:0x0].0x0 bits 0x13/0x0 rrc: 2 type: IBT gid 0 flags: 0x50210000000000 nid: local remote: 0x0 expref: -99 pid: 6256 timeout: 0 lvb_type: 0 00010000:00010000:3.0:1776178221.759171:0:3340:0:(ldlm_inodebits.c:94:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x2000013a1:0xb:0x0].0x0 (000000002f3f796c) 00010000:00010000:3.0:1776178221.759173:0:3340:0:(ldlm_inodebits.c:96:ldlm_reprocess_inodebits_queue()) Hint 13 00010000:00010000:0.0:1776178221.759215:0:6257:0:(ldlm_lock.c:216:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: mdt-lustre-MDT0000_UUID lock: 000000001735c8e5/0xa40083ab9bf53a57 lrc: 0/0,0 mode: --/CR res: [0x2000013a1:0x9:0x0].0x0 bits 0x2/0x0 rrc: 3 type: IBT gid 0 flags: 0x44000000000000 nid: 192.168.204.3@tcp remote: 0x771f68aa1eb27c61 expref: 23 pid: 6257 timeout: 0 lvb_type: 0 00010000:00010000:0.0:1776178221.759233:0:6257:0:(ldlm_lockd.c:1484:ldlm_handle_enqueue0()) ### server-side enqueue handler, sending reply (err=0, rc=0) ns: mdt-lustre-MDT0000_UUID lock: 00000000b4e1b221/0xa40083ab9bf53a5e lrc: 3/0,0 mode: PR/PR res: [0x2000013a1:0xe:0x0].0x0 bits 0x1b/0x0 rrc: 3 type: IBT gid 0 flags: 0x40200000000000 nid: 192.168.204.3@tcp remote: 0x771f68aa1eb27c61 expref: 22 pid: 6257 timeout: 0 lvb_type: 0 00010000:00010000:0.0:1776178221.759253:0:6257:0:(ldlm_inodebits.c:94:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x2000013a1:0xe:0x0].0x0 (000000008c340be4) 00010000:00010000:0.0:1776178221.759255:0:6257:0:(ldlm_inodebits.c:96:ldlm_reprocess_inodebits_queue()) Hint 1b 00010000:00010000:0.0:1776178221.759257:0:6257:0:(ldlm_lockd.c:1565:ldlm_handle_enqueue0()) ### server-side enqueue handler END (lock 00000000b4e1b221, rc 0) 00010000:00000080:0.0:1776178221.759260:0:6257:0:(ldlm_lib.c:3181:target_committed_to_req()) @@@ not sending last_committed update (1/0) req@0000000044d75ea3 x1862456979109120/t0(0) o101->b50963bd-8eba-4432-9a54-8888d00e3fa1@192.168.204.3@tcp:753/0 lens 576/696 e 0 to 0 dl 1776178268 ref 1 fl Interpret:/0/0 rc 0/0 job:'chmod.0' 00000100:00100000:1.0:1776178221.759474:0:6255:0:(service.c:2372:ptlrpc_server_handle_request()) Handled RPC req@0000000087a31eb2 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_000:b50963bd-8eba-4432-9a54-8888d00e3fa1+22:18366:x1862456979109184:12345-192.168.204.3@tcp:101:chmod.0 Request processed in 1249us (1339us total) trans 0 rc 0/0 00000100:00100000:1.0:1776178221.759481:0:6255:0:(nrs_fifo.c:241:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.204.3@tcp, seq: 6629 00000100:00100000:0.0:1776178221.764188:0:6257:0:(service.c:2372:ptlrpc_server_handle_request()) Handled RPC req@0000000044d75ea3 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_002:b50963bd-8eba-4432-9a54-8888d00e3fa1+21:18365:x1862456979109120:12345-192.168.204.3@tcp:101:chmod.0 Request processed in 6207us (6995us total) trans 0 rc 0/0 00000100:00100000:0.0:1776178221.764204:0:6257:0:(nrs_fifo.c:241:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.204.3@tcp, seq: 6628 00000100:00100000:3.0:1776178221.765823:0:3347:0:(events.c:361:request_in_callback()) peer: 12345-192.168.204.3@tcp (source: 12345-192.168.204.3@tcp) 00000100:00100000:3.0:1776178221.767732:0:3346:0:(events.c:361:request_in_callback()) peer: 12345-192.168.204.3@tcp (source: 12345-192.168.204.3@tcp) 00000100:00100000:3.0:1776178221.767776:0:3346:0:(events.c:361:request_in_callback()) peer: 12345-192.168.204.3@tcp (source: 12345-192.168.204.3@tcp) 00000100:00100000:3.0:1776178221.767808:0:3346:0:(events.c:361:request_in_callback()) peer: 12345-192.168.204.3@tcp (source: 12345-192.168.204.3@tcp) 00000100:00100000:3.0:1776178221.767844:0:20759:0:(service.c:2152:ptlrpc_server_handle_req_in()) got req x1862456979109568 00000100:00100000:3.0:1776178221.768085:0:20759:0:(service.c:2152:ptlrpc_server_handle_req_in()) got req x1862456979109696 00000100:00100000:3.0:1776178221.768111:0:20759:0:(nrs_fifo.c:179:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.204.3@tcp, seq: 5296 00000100:00100000:3.0:1776178221.768117:0:20759:0:(service.c:2324:ptlrpc_server_handle_request()) Handling RPC req@00000000b042ff66 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_010:b50963bd-8eba-4432-9a54-8888d00e3fa1+7:2273:x1862456979109568:12345-192.168.204.3@tcp:101:chmod.0 00010000:00010000:3.0:1776178221.768137:0:20759:0:(ldlm_lockd.c:1248:ldlm_handle_enqueue0()) ### server-side enqueue handler START 00010000:00010000:3.0:1776178221.768162:0:20759:0:(ldlm_lockd.c:1318:ldlm_handle_enqueue0()) ### server-side enqueue handler, new lock created ns: filter-lustre-OST0000_UUID lock: 00000000014a8a18/0xa40083ab9bf53a7a lrc: 2/0,0 mode: --/PR res: [0x48:0x0:0x0].0x0 rrc: 2 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x771f68aa1eb27c6f expref: -99 pid: 20759 timeout: 0 lvb_type: 0 00002000:00010000:3.0:1776178221.768192:0:20759:0:(ofd_lvb.c:170:ofd_lvbo_init()) res: [0x100000000:0x48:0x0] initial LVB size: 0, mtime: 0x69de5423, atime: 0x69de5423, ctime: 0x69de5423, blocks: 0x0 00010000:00010000:3.0:1776178221.768217:0:20759:0:(ldlm_extent.c:318:ldlm_check_contention()) contended locks = 0 00010000:00010000:3.0:1776178221.768220:0:20759:0:(ldlm_extent.c:318:ldlm_check_contention()) contended locks = 0 00010000:00010000:3.0:1776178221.768233:0:20759:0:(ldlm_resource.c:1650:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0000_UUID lock: 00000000014a8a18/0xa40083ab9bf53a7a lrc: 3/0,0 mode: PR/PR res: [0x48:0x0:0x0].0x0 rrc: 2 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x40000000000000 nid: 192.168.204.3@tcp remote: 0x771f68aa1eb27c6f expref: 8 pid: 20759 timeout: 0 lvb_type: 1 00010000:00010000:3.0:1776178221.768252:0:20759:0:(ldlm_lockd.c:1484:ldlm_handle_enqueue0()) ### server-side enqueue handler, sending reply (err=0, rc=0) ns: filter-lustre-OST0000_UUID lock: 00000000014a8a18/0xa40083ab9bf53a7a lrc: 3/0,0 mode: PR/PR res: [0x48:0x0:0x0].0x0 rrc: 2 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x40000000000000 nid: 192.168.204.3@tcp remote: 0x771f68aa1eb27c6f expref: 8 pid: 20759 timeout: 0 lvb_type: 1 00010000:00010000:3.0:1776178221.768266:0:20759:0:(ldlm_lockd.c:1565:ldlm_handle_enqueue0()) ### server-side enqueue handler END (lock 00000000014a8a18, rc 0) 00000100:00100000:3.0:1776178221.768324:0:20759:0:(service.c:2372:ptlrpc_server_handle_request()) Handled RPC req@00000000b042ff66 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_010:b50963bd-8eba-4432-9a54-8888d00e3fa1+8:2273:x1862456979109568:12345-192.168.204.3@tcp:101:chmod.0 Request processed in 206us (601us total) trans 0 rc 0/0 00000100:00100000:3.0:1776178221.768331:0:20759:0:(nrs_fifo.c:241:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.204.3@tcp, seq: 5296 00000100:00100000:3.0:1776178221.768348:0:20759:0:(nrs_fifo.c:179:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.204.3@tcp, seq: 5297 00000100:00100000:3.0:1776178221.768353:0:20759:0:(service.c:2324:ptlrpc_server_handle_request()) Handling RPC req@00000000d93806e8 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_010:b50963bd-8eba-4432-9a54-8888d00e3fa1+7:2271:x1862456979109696:12345-192.168.204.3@tcp:101:chmod.0 00010000:00010000:3.0:1776178221.768374:0:20759:0:(ldlm_lockd.c:1248:ldlm_handle_enqueue0()) ### server-side enqueue handler START 00010000:00010000:3.0:1776178221.768386:0:20759:0:(ldlm_lockd.c:1318:ldlm_handle_enqueue0()) ### server-side enqueue handler, new lock created ns: filter-lustre-OST0001_UUID lock: 000000002a990613/0xa40083ab9bf53a81 lrc: 2/0,0 mode: --/PR res: [0x48:0x0:0x0].0x0 rrc: 2 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x771f68aa1eb27c76 expref: -99 pid: 20759 timeout: 0 lvb_type: 0 00002000:00010000:3.0:1776178221.768402:0:20759:0:(ofd_lvb.c:170:ofd_lvbo_init()) res: [0x100010000:0x48:0x0] initial LVB size: 0, mtime: 0x69de5423, atime: 0x69de5423, ctime: 0x69de5423, blocks: 0x0 00010000:00010000:3.0:1776178221.768413:0:20759:0:(ldlm_extent.c:318:ldlm_check_contention()) contended locks = 0 00010000:00010000:3.0:1776178221.768415:0:20759:0:(ldlm_extent.c:318:ldlm_check_contention()) contended locks = 0 00010000:00010000:3.0:1776178221.768419:0:20759:0:(ldlm_resource.c:1650:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0001_UUID lock: 000000002a990613/0xa40083ab9bf53a81 lrc: 3/0,0 mode: PR/PR res: [0x48:0x0:0x0].0x0 rrc: 2 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x40000000000000 nid: 192.168.204.3@tcp remote: 0x771f68aa1eb27c76 expref: 8 pid: 20759 timeout: 0 lvb_type: 1 00010000:00010000:3.0:1776178221.768429:0:20759:0:(ldlm_lockd.c:1484:ldlm_handle_enqueue0()) ### server-side enqueue handler, sending reply (err=0, rc=0) ns: filter-lustre-OST0001_UUID lock: 000000002a990613/0xa40083ab9bf53a81 lrc: 3/0,0 mode: PR/PR res: [0x48:0x0:0x0].0x0 rrc: 2 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x40000000000000 nid: 192.168.204.3@tcp remote: 0x771f68aa1eb27c76 expref: 8 pid: 20759 timeout: 0 lvb_type: 1 00010000:00010000:3.0:1776178221.768437:0:20759:0:(ldlm_lockd.c:1565:ldlm_handle_enqueue0()) ### server-side enqueue handler END (lock 000000002a990613, rc 0) 00000100:00100000:3.0:1776178221.768678:0:20759:0:(service.c:2372:ptlrpc_server_handle_request()) Handled RPC req@00000000d93806e8 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_010:b50963bd-8eba-4432-9a54-8888d00e3fa1+8:2271:x1862456979109696:12345-192.168.204.3@tcp:101:chmod.0 Request processed in 324us (870us total) trans 0 rc 0/0 00000100:00100000:3.0:1776178221.768686:0:20759:0:(nrs_fifo.c:241:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.204.3@tcp, seq: 5297 00000100:00100000:0.0:1776178221.770544:0:6257:0:(service.c:2152:ptlrpc_server_handle_req_in()) got req x1862456979109440 00000100:00100000:0.0:1776178221.770613:0:6257:0:(service.c:2152:ptlrpc_server_handle_req_in()) got req x1862456979109632 00000100:00100000:0.0:1776178221.773135:0:6255:0:(nrs_fifo.c:179:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.204.3@tcp, seq: 6631 00000100:00100000:0.0:1776178221.773159:0:6255:0:(service.c:2324:ptlrpc_server_handle_request()) Handling RPC req@00000000cf8ff617 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_000:b50963bd-8eba-4432-9a54-8888d00e3fa1+22:18364:x1862456979109440:12345-192.168.204.3@tcp:36:chmod.0 00010000:00010000:0.0:1776178221.773231:0:6255:0:(ldlm_lockd.c:1720:ldlm_request_cancel()) ### server-side cancel handler START: 2 locks, starting at 0 00010000:00010000:0.0:1776178221.773274:0:6255:0:(ldlm_lock.c:216:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: mdt-lustre-MDT0000_UUID lock: 00000000336cd7d4/0xa40083ab9bf53a1f lrc: 0/0,0 mode: --/PR res: [0x2000013a1:0xd:0x0].0x0 bits 0x1b/0x0 rrc: 4 type: IBT gid 0 flags: 0x44a01000000000 nid: 192.168.204.3@tcp remote: 0x771f68aa1eb27c30 expref: 22 pid: 8250 timeout: 0 lvb_type: 0 00010000:00010000:0.0:1776178221.773295:0:6255:0:(ldlm_lock.c:216:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: mdt-lustre-MDT0000_UUID lock: 0000000056462e0c/0xa40083ab9bf53907 lrc: 0/0,0 mode: --/CR res: [0x2000013a1:0xd:0x0].0x0 bits 0x9/0x0 rrc: 3 type: IBT gid 0 flags: 0x44a01000000000 nid: 192.168.204.3@tcp remote: 0x771f68aa1eb27bf8 expref: 21 pid: 6256 timeout: 0 lvb_type: 0 00010000:00010000:0.0:1776178221.773304:0:6255:0:(ldlm_inodebits.c:94:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x2000013a1:0xd:0x0].0x0 (00000000713d9096) 00010000:00010000:0.0:1776178221.773319:0:6255:0:(ldlm_lockd.c:1782:ldlm_request_cancel()) ### server-side cancel handler END 00000100:00100000:3.0:1776178221.773338:0:8250:0:(nrs_fifo.c:179:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.204.3@tcp, seq: 6632 00000100:00100000:3.0:1776178221.773346:0:8250:0:(service.c:2324:ptlrpc_server_handle_request()) Handling RPC req@00000000ee83da19 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_003:b50963bd-8eba-4432-9a54-8888d00e3fa1+20:18363:x1862456979109632:12345-192.168.204.3@tcp:36:chmod.0 00010000:00010000:3.0:1776178221.773380:0:8250:0:(ldlm_lockd.c:1720:ldlm_request_cancel()) ### server-side cancel handler START: 2 locks, starting at 0 00010000:00010000:3.0:1776178221.773392:0:8250:0:(ldlm_lock.c:216:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: mdt-lustre-MDT0000_UUID lock: 0000000026fd08a4/0xa40083ab9bf53a2d lrc: 0/0,0 mode: --/PR res: [0x2000013a1:0xc:0x0].0x0 bits 0x1b/0x0 rrc: 4 type: IBT gid 0 flags: 0x44a01000000000 nid: 192.168.204.3@tcp remote: 0x771f68aa1eb27c3e expref: 20 pid: 11281 timeout: 0 lvb_type: 0 00010000:00010000:3.0:1776178221.773406:0:8250:0:(ldlm_lock.c:216:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: mdt-lustre-MDT0000_UUID lock: 000000001e6d7318/0xa40083ab9bf538dd lrc: 0/0,0 mode: --/CR res: [0x2000013a1:0xc:0x0].0x0 bits 0x9/0x0 rrc: 3 type: IBT gid 0 flags: 0x44a01000000000 nid: 192.168.204.3@tcp remote: 0x771f68aa1eb27bf1 expref: 19 pid: 6257 timeout: 0 lvb_type: 0 00010000:00010000:3.0:1776178221.773413:0:8250:0:(ldlm_inodebits.c:94:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x2000013a1:0xc:0x0].0x0 (0000000062690614) 00010000:00010000:3.0:1776178221.773419:0:8250:0:(ldlm_lockd.c:1782:ldlm_request_cancel()) ### server-side cancel handler END 00010000:00010000:3.0:1776178221.773458:0:8250:0:(ldlm_lock.c:766:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: mdt-lustre-MDT0000_UUID lock: 00000000c3ab77cd/0xa40083ab9bf53a88 lrc: 3/0,1 mode: --/PW res: [0x2000013a1:0xc:0x0].0x0 bits 0x0/0x0 rrc: 2 type: IBT gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 8250 timeout: 0 lvb_type: 0 00010000:00010000:3.0:1776178221.773471:0:8250:0:(ldlm_lock.c:1064:ldlm_granted_list_add_lock()) ### About to add lock: ns: mdt-lustre-MDT0000_UUID lock: 00000000c3ab77cd/0xa40083ab9bf53a88 lrc: 3/0,1 mode: PW/PW res: [0x2000013a1:0xc:0x0].0x0 bits 0x13/0x0 rrc: 2 type: IBT gid 0 flags: 0x50210000000000 nid: local remote: 0x0 expref: -99 pid: 8250 timeout: 0 lvb_type: 0 00010000:00010000:3.0:1776178221.773482:0:8250:0:(ldlm_request.c:528:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: mdt-lustre-MDT0000_UUID lock: 00000000c3ab77cd/0xa40083ab9bf53a88 lrc: 3/0,1 mode: PW/PW res: [0x2000013a1:0xc:0x0].0x0 bits 0x13/0x0 rrc: 2 type: IBT gid 0 flags: 0x40210000000000 nid: local remote: 0x0 expref: -99 pid: 8250 timeout: 0 lvb_type: 0 00010000:00010000:0.0:1776178221.773483:0:6255:0:(ldlm_lock.c:766:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: mdt-lustre-MDT0000_UUID lock: 00000000782e253e/0xa40083ab9bf53a8f lrc: 3/0,1 mode: --/PW res: [0x2000013a1:0xd:0x0].0x0 bits 0x0/0x0 rrc: 2 type: IBT gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 6255 timeout: 0 lvb_type: 0 00010000:00010000:0.0:1776178221.773497:0:6255:0:(ldlm_lock.c:1064:ldlm_granted_list_add_lock()) ### About to add lock: ns: mdt-lustre-MDT0000_UUID lock: 00000000782e253e/0xa40083ab9bf53a8f lrc: 3/0,1 mode: PW/PW res: [0x2000013a1:0xd:0x0].0x0 bits 0x13/0x0 rrc: 2 type: IBT gid 0 flags: 0x50210000000000 nid: local remote: 0x0 expref: -99 pid: 6255 timeout: 0 lvb_type: 0 00010000:00010000:0.0:1776178221.773503:0:6255:0:(ldlm_request.c:528:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: mdt-lustre-MDT0000_UUID lock: 00000000782e253e/0xa40083ab9bf53a8f lrc: 3/0,1 mode: PW/PW res: [0x2000013a1:0xd:0x0].0x0 bits 0x13/0x0 rrc: 2 type: IBT gid 0 flags: 0x40210000000000 nid: local remote: 0x0 expref: -99 pid: 6255 timeout: 0 lvb_type: 0 00000004:00010000:0.0:1776178221.773623:0:6255:0:(mdt_handler.c:3992:mdt_save_lock()) ### save lock request 00000000cf8ff617 reply state 000000006e7232c8 transno 12884901977 ns: mdt-lustre-MDT0000_UUID lock: 00000000782e253e/0xa40083ab9bf53a8f lrc: 3/0,1 mode: PW/PW res: [0x2000013a1:0xd:0x0].0x0 bits 0x13/0x0 rrc: 2 type: IBT gid 0 flags: 0x40210000000000 nid: local remote: 0x0 expref: -99 pid: 6255 timeout: 0 lvb_type: 0 00010000:00000080:0.0:1776178221.773673:0:6255:0:(ldlm_lib.c:3181:target_committed_to_req()) @@@ not sending last_committed update (1/0) req@00000000cf8ff617 x1862456979109440/t12884901977(0) o36->b50963bd-8eba-4432-9a54-8888d00e3fa1@192.168.204.3@tcp:753/0 lens 488/456 e 0 to 0 dl 1776178268 ref 1 fl Interpret:/0/0 rc 0/0 job:'chmod.0' 00010000:02000000:0.0:1776178221.773685:0:6255:0:(libcfs_fail.h:96:cfs_fail_check_set()) *** cfs_fail_loc=15a, val=0*** 00010000:00080000:0.0:1776178221.773689:0:6255:0:(ldlm_lib.c:3333:target_send_reply()) Schedule reply immediately 00000100:00100000:0.0:1776178221.773711:0:6255:0:(service.c:2372:ptlrpc_server_handle_request()) Handled RPC req@00000000cf8ff617 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_000:b50963bd-8eba-4432-9a54-8888d00e3fa1+21:18364:x1862456979109440:12345-192.168.204.3@tcp:36:chmod.0 Request processed in 559us (7894us total) trans 12884901977 rc 0/0 00000100:00100000:0.0:1776178221.773718:0:6255:0:(nrs_fifo.c:241:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.204.3@tcp, seq: 6631 00010000:00010000:0.0:1776178221.773763:0:3339:0:(ldlm_lock.c:1064:ldlm_granted_list_add_lock()) ### About to add lock: ns: mdt-lustre-MDT0000_UUID lock: 00000000782e253e/0xa40083ab9bf53a8f lrc: 3/0,1 mode: COS/COS res: [0x2000013a1:0xd:0x0].0x0 bits 0x13/0x0 rrc: 2 type: IBT gid 0 flags: 0x50210000000000 nid: local remote: 0x0 expref: -99 pid: 6255 timeout: 0 lvb_type: 0 00010000:00010000:0.0:1776178221.773770:0:3339:0:(ldlm_inodebits.c:94:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x2000013a1:0xd:0x0].0x0 (0000000078a0d18e) 00010000:00010000:0.0:1776178221.773773:0:3339:0:(ldlm_inodebits.c:96:ldlm_reprocess_inodebits_queue()) Hint 13 00000004:00010000:3.0:1776178221.774082:0:8250:0:(mdt_handler.c:3992:mdt_save_lock()) ### save lock request 00000000ee83da19 reply state 000000006262d7ad transno 12884901978 ns: mdt-lustre-MDT0000_UUID lock: 00000000c3ab77cd/0xa40083ab9bf53a88 lrc: 3/0,1 mode: PW/PW res: [0x2000013a1:0xc:0x0].0x0 bits 0x13/0x0 rrc: 2 type: IBT gid 0 flags: 0x40210000000000 nid: local remote: 0x0 expref: -99 pid: 8250 timeout: 0 lvb_type: 0 00010000:00000080:3.0:1776178221.774112:0:8250:0:(ldlm_lib.c:3181:target_committed_to_req()) @@@ not sending last_committed update (1/0) req@00000000ee83da19 x1862456979109632/t12884901978(0) o36->b50963bd-8eba-4432-9a54-8888d00e3fa1@192.168.204.3@tcp:753/0 lens 488/456 e 0 to 0 dl 1776178268 ref 1 fl Interpret:/0/0 rc 0/0 job:'chmod.0' 00010000:02000000:3.0:1776178221.774120:0:8250:0:(libcfs_fail.h:96:cfs_fail_check_set()) *** cfs_fail_loc=15a, val=0*** 00010000:00080000:3.0:1776178221.774122:0:8250:0:(ldlm_lib.c:3333:target_send_reply()) Schedule reply immediately 00000100:00100000:3.0:1776178221.774138:0:8250:0:(service.c:2372:ptlrpc_server_handle_request()) Handled RPC req@00000000ee83da19 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_003:b50963bd-8eba-4432-9a54-8888d00e3fa1+21:18363:x1862456979109632:12345-192.168.204.3@tcp:36:chmod.0 Request processed in 793us (6372us total) trans 12884901978 rc 0/0 00000100:00100000:3.0:1776178221.774144:0:8250:0:(nrs_fifo.c:241:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.204.3@tcp, seq: 6632 00010000:00010000:0.0:1776178221.774766:0:3340:0:(ldlm_lock.c:1064:ldlm_granted_list_add_lock()) ### About to add lock: ns: mdt-lustre-MDT0000_UUID lock: 00000000c3ab77cd/0xa40083ab9bf53a88 lrc: 3/0,1 mode: COS/COS res: [0x2000013a1:0xc:0x0].0x0 bits 0x13/0x0 rrc: 2 type: IBT gid 0 flags: 0x50210000000000 nid: local remote: 0x0 expref: -99 pid: 8250 timeout: 0 lvb_type: 0 00010000:00010000:0.0:1776178221.774780:0:3340:0:(ldlm_inodebits.c:94:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x2000013a1:0xc:0x0].0x0 (00000000099618da) 00010000:00010000:0.0:1776178221.774783:0:3340:0:(ldlm_inodebits.c:96:ldlm_reprocess_inodebits_queue()) Hint 13 00000100:00100000:3.0:1776178221.780314:0:3346:0:(events.c:361:request_in_callback()) peer: 12345-192.168.204.3@tcp (source: 12345-192.168.204.3@tcp) 00000100:00100000:3.0:1776178221.780359:0:3346:0:(events.c:361:request_in_callback()) peer: 12345-192.168.204.3@tcp (source: 12345-192.168.204.3@tcp) 00000100:00100000:3.0:1776178221.780417:0:3346:0:(events.c:361:request_in_callback()) peer: 12345-192.168.204.3@tcp (source: 12345-192.168.204.3@tcp) 00000100:00100000:3.0:1776178221.780453:0:8250:0:(service.c:2152:ptlrpc_server_handle_req_in()) got req x1862456979109888 00000100:00100000:2.0:1776178221.780467:0:14440:0:(service.c:2152:ptlrpc_server_handle_req_in()) got req x1862456979109760 00000100:00100000:3.0:1776178221.780497:0:8250:0:(service.c:2152:ptlrpc_server_handle_req_in()) got req x1862456979110016 00000100:00100000:2.0:1776178221.780510:0:14440:0:(nrs_fifo.c:179:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.204.3@tcp, seq: 5298 00000100:00100000:2.0:1776178221.780523:0:14440:0:(service.c:2324:ptlrpc_server_handle_request()) Handling RPC req@0000000089769a1e pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_004:b50963bd-8eba-4432-9a54-8888d00e3fa1+8:2271:x1862456979109760:12345-192.168.204.3@tcp:101:chmod.0 00000100:00100000:3.0:1776178221.780523:0:8250:0:(nrs_fifo.c:179:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.204.3@tcp, seq: 6633 00000100:00100000:3.0:1776178221.780531:0:8250:0:(service.c:2324:ptlrpc_server_handle_request()) Handling RPC req@0000000095c7db63 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_003:b50963bd-8eba-4432-9a54-8888d00e3fa1+22:18366:x1862456979109888:12345-192.168.204.3@tcp:36:chmod.0 00010000:00010000:2.0:1776178221.780546:0:14440:0:(ldlm_lockd.c:1248:ldlm_handle_enqueue0()) ### server-side enqueue handler START 00010000:00010000:2.0:1776178221.780578:0:14440:0:(ldlm_lockd.c:1318:ldlm_handle_enqueue0()) ### server-side enqueue handler, new lock created ns: filter-lustre-OST0001_UUID lock: 000000002f1f7bd5/0xa40083ab9bf53a96 lrc: 2/0,0 mode: --/PR res: [0x47:0x0:0x0].0x0 rrc: 2 type: EXT [0->0] (req 0->0) gid 0 flags: 0x40000000000000 nid: local remote: 0x771f68aa1eb27c7d expref: -99 pid: 14440 timeout: 0 lvb_type: 0 00010000:00010000:3.0:1776178221.780583:0:8250:0:(ldlm_lockd.c:1720:ldlm_request_cancel()) ### server-side cancel handler START: 2 locks, starting at 0 00002000:00010000:2.0:1776178221.780609:0:14440:0:(ofd_lvb.c:170:ofd_lvbo_init()) res: [0x100010000:0x47:0x0] initial LVB size: 0, mtime: 0x69de5423, atime: 0x69de5423, ctime: 0x69de5423, blocks: 0x0 00010000:00010000:3.0:1776178221.780617:0:8250:0:(ldlm_lock.c:216:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: mdt-lustre-MDT0000_UUID lock: 0000000097975a3b/0xa40083ab9bf53a6c lrc: 0/0,0 mode: --/PR res: [0x2000013a1:0xf:0x0].0x0 bits 0x1b/0x0 rrc: 4 type: IBT gid 0 flags: 0x44a01000000000 nid: 192.168.204.3@tcp remote: 0x771f68aa1eb27c68 expref: 22 pid: 6255 timeout: 0 lvb_type: 0 00010000:00010000:2.0:1776178221.780629:0:14440:0:(ldlm_extent.c:318:ldlm_check_contention()) contended locks = 0 00010000:00010000:2.0:1776178221.780632:0:14440:0:(ldlm_extent.c:318:ldlm_check_contention()) contended locks = 0 00010000:00010000:2.0:1776178221.780640:0:14440:0:(ldlm_resource.c:1650:ldlm_resource_add_lock()) ### About to add this lock ns: filter-lustre-OST0001_UUID lock: 000000002f1f7bd5/0xa40083ab9bf53a96 lrc: 3/0,0 mode: PR/PR res: [0x47:0x0:0x0].0x0 rrc: 2 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x40000000000000 nid: 192.168.204.3@tcp remote: 0x771f68aa1eb27c7d expref: 9 pid: 14440 timeout: 0 lvb_type: 1 00010000:00010000:3.0:1776178221.780643:0:8250:0:(ldlm_lock.c:216:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: mdt-lustre-MDT0000_UUID lock: 000000009a8f90ae/0xa40083ab9bf5395b lrc: 0/0,0 mode: --/CR res: [0x2000013a1:0xf:0x0].0x0 bits 0x9/0x0 rrc: 3 type: IBT gid 0 flags: 0x44a01000000000 nid: 192.168.204.3@tcp remote: 0x771f68aa1eb27c06 expref: 21 pid: 6257 timeout: 0 lvb_type: 0 00010000:00010000:2.0:1776178221.780666:0:14440:0:(ldlm_lockd.c:1484:ldlm_handle_enqueue0()) ### server-side enqueue handler, sending reply (err=0, rc=0) ns: filter-lustre-OST0001_UUID lock: 000000002f1f7bd5/0xa40083ab9bf53a96 lrc: 3/0,0 mode: PR/PR res: [0x47:0x0:0x0].0x0 rrc: 2 type: EXT [0->18446744073709551615] (req 0->18446744073709551615) gid 0 flags: 0x40000000000000 nid: 192.168.204.3@tcp remote: 0x771f68aa1eb27c7d expref: 9 pid: 14440 timeout: 0 lvb_type: 1 00010000:00010000:3.0:1776178221.780674:0:8250:0:(ldlm_inodebits.c:94:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x2000013a1:0xf:0x0].0x0 (00000000a5b77ef7) 00010000:00010000:2.0:1776178221.780681:0:14440:0:(ldlm_lockd.c:1565:ldlm_handle_enqueue0()) ### server-side enqueue handler END (lock 000000002f1f7bd5, rc 0) 00010000:00010000:3.0:1776178221.780686:0:8250:0:(ldlm_lockd.c:1782:ldlm_request_cancel()) ### server-side cancel handler END 00010000:00010000:3.0:1776178221.780721:0:8250:0:(ldlm_lock.c:766:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: mdt-lustre-MDT0000_UUID lock: 0000000026fd08a4/0xa40083ab9bf53a9d lrc: 3/0,1 mode: --/PW res: [0x2000013a1:0xf:0x0].0x0 bits 0x0/0x0 rrc: 2 type: IBT gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 8250 timeout: 0 lvb_type: 0 00010000:00010000:3.0:1776178221.780735:0:8250:0:(ldlm_lock.c:1064:ldlm_granted_list_add_lock()) ### About to add lock: ns: mdt-lustre-MDT0000_UUID lock: 0000000026fd08a4/0xa40083ab9bf53a9d lrc: 3/0,1 mode: PW/PW res: [0x2000013a1:0xf:0x0].0x0 bits 0x13/0x0 rrc: 2 type: IBT gid 0 flags: 0x50210000000000 nid: local remote: 0x0 expref: -99 pid: 8250 timeout: 0 lvb_type: 0 00010000:00010000:3.0:1776178221.780745:0:8250:0:(ldlm_request.c:528:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: mdt-lustre-MDT0000_UUID lock: 0000000026fd08a4/0xa40083ab9bf53a9d lrc: 3/0,1 mode: PW/PW res: [0x2000013a1:0xf:0x0].0x0 bits 0x13/0x0 rrc: 2 type: IBT gid 0 flags: 0x40210000000000 nid: local remote: 0x0 expref: -99 pid: 8250 timeout: 0 lvb_type: 0 00000100:00100000:2.0:1776178221.780764:0:14440:0:(service.c:2372:ptlrpc_server_handle_request()) Handled RPC req@0000000089769a1e pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_004:b50963bd-8eba-4432-9a54-8888d00e3fa1+9:2271:x1862456979109760:12345-192.168.204.3@tcp:101:chmod.0 Request processed in 245us (405us total) trans 0 rc 0/0 00000100:00100000:2.0:1776178221.780774:0:14440:0:(nrs_fifo.c:241:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.204.3@tcp, seq: 5298 00000004:00010000:3.0:1776178221.780924:0:8250:0:(mdt_handler.c:3992:mdt_save_lock()) ### save lock request 0000000095c7db63 reply state 000000004a16c86e transno 12884901979 ns: mdt-lustre-MDT0000_UUID lock: 0000000026fd08a4/0xa40083ab9bf53a9d lrc: 3/0,1 mode: PW/PW res: [0x2000013a1:0xf:0x0].0x0 bits 0x13/0x0 rrc: 2 type: IBT gid 0 flags: 0x40210000000000 nid: local remote: 0x0 expref: -99 pid: 8250 timeout: 0 lvb_type: 0 00010000:00000080:3.0:1776178221.780964:0:8250:0:(ldlm_lib.c:3181:target_committed_to_req()) @@@ not sending last_committed update (1/0) req@0000000095c7db63 x1862456979109888/t12884901979(0) o36->b50963bd-8eba-4432-9a54-8888d00e3fa1@192.168.204.3@tcp:753/0 lens 488/456 e 0 to 0 dl 1776178268 ref 1 fl Interpret:/0/0 rc 0/0 job:'chmod.0' 00010000:02000000:3.0:1776178221.780973:0:8250:0:(libcfs_fail.h:96:cfs_fail_check_set()) *** cfs_fail_loc=15a, val=0*** 00010000:00080000:3.0:1776178221.780976:0:8250:0:(ldlm_lib.c:3333:target_send_reply()) Schedule reply immediately 00000100:00100000:3.0:1776178221.780996:0:8250:0:(service.c:2372:ptlrpc_server_handle_request()) Handled RPC req@0000000095c7db63 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_003:b50963bd-8eba-4432-9a54-8888d00e3fa1+22:18366:x1862456979109888:12345-192.168.204.3@tcp:36:chmod.0 Request processed in 467us (704us total) trans 12884901979 rc 0/0 00000100:00100000:3.0:1776178221.781002:0:8250:0:(nrs_fifo.c:241:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.204.3@tcp, seq: 6633 00000100:00100000:3.0:1776178221.781015:0:8250:0:(nrs_fifo.c:179:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.204.3@tcp, seq: 6634 00000100:00100000:3.0:1776178221.781020:0:8250:0:(service.c:2324:ptlrpc_server_handle_request()) Handling RPC req@0000000011811c98 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_003:b50963bd-8eba-4432-9a54-8888d00e3fa1+21:18367:x1862456979110016:12345-192.168.204.3@tcp:36:chmod.0 00010000:00010000:2.0:1776178221.781033:0:3339:0:(ldlm_lock.c:1064:ldlm_granted_list_add_lock()) ### About to add lock: ns: mdt-lustre-MDT0000_UUID lock: 0000000026fd08a4/0xa40083ab9bf53a9d lrc: 3/0,1 mode: COS/COS res: [0x2000013a1:0xf:0x0].0x0 bits 0x13/0x0 rrc: 2 type: IBT gid 0 flags: 0x50210000000000 nid: local remote: 0x0 expref: -99 pid: 8250 timeout: 0 lvb_type: 0 00010000:00010000:3.0:1776178221.781040:0:8250:0:(ldlm_lockd.c:1720:ldlm_request_cancel()) ### server-side cancel handler START: 2 locks, starting at 0 00010000:00010000:2.0:1776178221.781044:0:3339:0:(ldlm_inodebits.c:94:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x2000013a1:0xf:0x0].0x0 (000000007b7d95c1) 00010000:00010000:2.0:1776178221.781048:0:3339:0:(ldlm_inodebits.c:96:ldlm_reprocess_inodebits_queue()) Hint 13 00010000:00010000:3.0:1776178221.781055:0:8250:0:(ldlm_lock.c:216:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: mdt-lustre-MDT0000_UUID lock: 00000000a44d8583/0xa40083ab9bf53a50 lrc: 0/0,0 mode: --/PR res: [0x2000013a1:0x10:0x0].0x0 bits 0x1b/0x0 rrc: 4 type: IBT gid 0 flags: 0x44a01000000000 nid: 192.168.204.3@tcp remote: 0x771f68aa1eb27c5a expref: 21 pid: 6256 timeout: 0 lvb_type: 0 00010000:00010000:3.0:1776178221.781595:0:8250:0:(ldlm_lock.c:216:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: mdt-lustre-MDT0000_UUID lock: 000000007da3b625/0xa40083ab9bf53985 lrc: 0/0,0 mode: --/CR res: [0x2000013a1:0x10:0x0].0x0 bits 0x9/0x0 rrc: 3 type: IBT gid 0 flags: 0x44a01000000000 nid: 192.168.204.3@tcp remote: 0x771f68aa1eb27c0d expref: 20 pid: 6257 timeout: 0 lvb_type: 0 00010000:00010000:3.0:1776178221.781604:0:8250:0:(ldlm_inodebits.c:94:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x2000013a1:0x10:0x0].0x0 (000000003da5851e) 00010000:00010000:3.0:1776178221.781611:0:8250:0:(ldlm_lockd.c:1782:ldlm_request_cancel()) ### server-side cancel handler END 00010000:00010000:2.0:1776178221.781707:0:8250:0:(ldlm_lock.c:766:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: mdt-lustre-MDT0000_UUID lock: 0000000096c1e297/0xa40083ab9bf53aa4 lrc: 3/0,1 mode: --/PW res: [0x2000013a1:0x10:0x0].0x0 bits 0x0/0x0 rrc: 2 type: IBT gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 8250 timeout: 0 lvb_type: 0 00010000:00010000:2.0:1776178221.781719:0:8250:0:(ldlm_lock.c:1064:ldlm_granted_list_add_lock()) ### About to add lock: ns: mdt-lustre-MDT0000_UUID lock: 0000000096c1e297/0xa40083ab9bf53aa4 lrc: 3/0,1 mode: PW/PW res: [0x2000013a1:0x10:0x0].0x0 bits 0x13/0x0 rrc: 2 type: IBT gid 0 flags: 0x50210000000000 nid: local remote: 0x0 expref: -99 pid: 8250 timeout: 0 lvb_type: 0 00010000:00010000:2.0:1776178221.781728:0:8250:0:(ldlm_request.c:528:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: mdt-lustre-MDT0000_UUID lock: 0000000096c1e297/0xa40083ab9bf53aa4 lrc: 3/0,1 mode: PW/PW res: [0x2000013a1:0x10:0x0].0x0 bits 0x13/0x0 rrc: 2 type: IBT gid 0 flags: 0x40210000000000 nid: local remote: 0x0 expref: -99 pid: 8250 timeout: 0 lvb_type: 0 00000004:00010000:2.0:1776178221.781898:0:8250:0:(mdt_handler.c:3992:mdt_save_lock()) ### save lock request 0000000011811c98 reply state 00000000c493003b transno 12884901980 ns: mdt-lustre-MDT0000_UUID lock: 0000000096c1e297/0xa40083ab9bf53aa4 lrc: 3/0,1 mode: PW/PW res: [0x2000013a1:0x10:0x0].0x0 bits 0x13/0x0 rrc: 2 type: IBT gid 0 flags: 0x40210000000000 nid: local remote: 0x0 expref: -99 pid: 8250 timeout: 0 lvb_type: 0 00010000:00000080:2.0:1776178221.781931:0:8250:0:(ldlm_lib.c:3181:target_committed_to_req()) @@@ not sending last_committed update (1/0) req@0000000011811c98 x1862456979110016/t12884901980(0) o36->b50963bd-8eba-4432-9a54-8888d00e3fa1@192.168.204.3@tcp:753/0 lens 488/456 e 0 to 0 dl 1776178268 ref 1 fl Interpret:/0/0 rc 0/0 job:'chmod.0' 00010000:02000000:2.0:1776178221.781940:0:8250:0:(libcfs_fail.h:96:cfs_fail_check_set()) *** cfs_fail_loc=15a, val=0*** 00010000:00080000:2.0:1776178221.781942:0:8250:0:(ldlm_lib.c:3333:target_send_reply()) Schedule reply immediately 00000100:00100000:2.0:1776178221.781964:0:8250:0:(service.c:2372:ptlrpc_server_handle_request()) Handled RPC req@0000000011811c98 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_003:b50963bd-8eba-4432-9a54-8888d00e3fa1+21:18367:x1862456979110016:12345-192.168.204.3@tcp:36:chmod.0 Request processed in 943us (1546us total) trans 12884901980 rc 0/0 00000100:00100000:2.0:1776178221.781971:0:8250:0:(nrs_fifo.c:241:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.204.3@tcp, seq: 6634 00010000:00010000:0.0:1776178221.782141:0:3340:0:(ldlm_lock.c:1064:ldlm_granted_list_add_lock()) ### About to add lock: ns: mdt-lustre-MDT0000_UUID lock: 0000000096c1e297/0xa40083ab9bf53aa4 lrc: 3/0,1 mode: COS/COS res: [0x2000013a1:0x10:0x0].0x0 bits 0x13/0x0 rrc: 2 type: IBT gid 0 flags: 0x50210000000000 nid: local remote: 0x0 expref: -99 pid: 8250 timeout: 0 lvb_type: 0 00010000:00010000:0.0:1776178221.782157:0:3340:0:(ldlm_inodebits.c:94:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x2000013a1:0x10:0x0].0x0 (0000000056a46617) 00010000:00010000:0.0:1776178221.782159:0:3340:0:(ldlm_inodebits.c:96:ldlm_reprocess_inodebits_queue()) Hint 13 00000100:00100000:0.0:1776178221.786344:0:3348:0:(events.c:361:request_in_callback()) peer: 12345-192.168.204.3@tcp (source: 12345-192.168.204.3@tcp) 00000100:00100000:2.0:1776178221.786472:0:8250:0:(service.c:2152:ptlrpc_server_handle_req_in()) got req x1862456979110144 00000100:00100000:2.0:1776178221.786533:0:8250:0:(nrs_fifo.c:179:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.204.3@tcp, seq: 6635 00000100:00100000:2.0:1776178221.786539:0:8250:0:(service.c:2324:ptlrpc_server_handle_request()) Handling RPC req@00000000f871d6a5 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_003:b50963bd-8eba-4432-9a54-8888d00e3fa1+21:18365:x1862456979110144:12345-192.168.204.3@tcp:36:chmod.0 00010000:00010000:2.0:1776178221.786596:0:8250:0:(ldlm_lockd.c:1720:ldlm_request_cancel()) ### server-side cancel handler START: 2 locks, starting at 0 00010000:00010000:2.0:1776178221.786624:0:8250:0:(ldlm_lock.c:216:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: mdt-lustre-MDT0000_UUID lock: 00000000b4e1b221/0xa40083ab9bf53a5e lrc: 0/0,0 mode: --/PR res: [0x2000013a1:0xe:0x0].0x0 bits 0x1b/0x0 rrc: 4 type: IBT gid 0 flags: 0x44a01000000000 nid: 192.168.204.3@tcp remote: 0x771f68aa1eb27c61 expref: 21 pid: 6257 timeout: 0 lvb_type: 0 00010000:00010000:2.0:1776178221.786646:0:8250:0:(ldlm_lock.c:216:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: mdt-lustre-MDT0000_UUID lock: 000000004bd1d7cb/0xa40083ab9bf53931 lrc: 0/0,0 mode: --/CR res: [0x2000013a1:0xe:0x0].0x0 bits 0x9/0x0 rrc: 3 type: IBT gid 0 flags: 0x44a01000000000 nid: 192.168.204.3@tcp remote: 0x771f68aa1eb27bff expref: 20 pid: 6256 timeout: 0 lvb_type: 0 00010000:00010000:2.0:1776178221.786657:0:8250:0:(ldlm_inodebits.c:94:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x2000013a1:0xe:0x0].0x0 (000000008c340be4) 00010000:00010000:2.0:1776178221.786665:0:8250:0:(ldlm_lockd.c:1782:ldlm_request_cancel()) ### server-side cancel handler END 00010000:00010000:2.0:1776178221.786695:0:8250:0:(ldlm_lock.c:766:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(PW) ns: mdt-lustre-MDT0000_UUID lock: 000000000396420a/0xa40083ab9bf53aab lrc: 3/0,1 mode: --/PW res: [0x2000013a1:0xe:0x0].0x0 bits 0x0/0x0 rrc: 2 type: IBT gid 0 flags: 0x40000000000000 nid: local remote: 0x0 expref: -99 pid: 8250 timeout: 0 lvb_type: 0 00010000:00010000:2.0:1776178221.786708:0:8250:0:(ldlm_lock.c:1064:ldlm_granted_list_add_lock()) ### About to add lock: ns: mdt-lustre-MDT0000_UUID lock: 000000000396420a/0xa40083ab9bf53aab lrc: 3/0,1 mode: PW/PW res: [0x2000013a1:0xe:0x0].0x0 bits 0x13/0x0 rrc: 2 type: IBT gid 0 flags: 0x50210000000000 nid: local remote: 0x0 expref: -99 pid: 8250 timeout: 0 lvb_type: 0 00010000:00010000:2.0:1776178221.786717:0:8250:0:(ldlm_request.c:528:ldlm_cli_enqueue_local()) ### client-side local enqueue handler, new lock created ns: mdt-lustre-MDT0000_UUID lock: 000000000396420a/0xa40083ab9bf53aab lrc: 3/0,1 mode: PW/PW res: [0x2000013a1:0xe:0x0].0x0 bits 0x13/0x0 rrc: 2 type: IBT gid 0 flags: 0x40210000000000 nid: local remote: 0x0 expref: -99 pid: 8250 timeout: 0 lvb_type: 0 00000004:00010000:2.0:1776178221.786965:0:8250:0:(mdt_handler.c:3992:mdt_save_lock()) ### save lock request 00000000f871d6a5 reply state 000000003d66019f transno 12884901981 ns: mdt-lustre-MDT0000_UUID lock: 000000000396420a/0xa40083ab9bf53aab lrc: 3/0,1 mode: PW/PW res: [0x2000013a1:0xe:0x0].0x0 bits 0x13/0x0 rrc: 2 type: IBT gid 0 flags: 0x40210000000000 nid: local remote: 0x0 expref: -99 pid: 8250 timeout: 0 lvb_type: 0 00010000:00000080:2.0:1776178221.787025:0:8250:0:(ldlm_lib.c:3181:target_committed_to_req()) @@@ not sending last_committed update (1/0) req@00000000f871d6a5 x1862456979110144/t12884901981(0) o36->b50963bd-8eba-4432-9a54-8888d00e3fa1@192.168.204.3@tcp:753/0 lens 488/456 e 0 to 0 dl 1776178268 ref 1 fl Interpret:/0/0 rc 0/0 job:'chmod.0' 00010000:02000000:2.0:1776178221.787037:0:8250:0:(libcfs_fail.h:96:cfs_fail_check_set()) *** cfs_fail_loc=15a, val=0*** 00010000:00080000:2.0:1776178221.787041:0:8250:0:(ldlm_lib.c:3333:target_send_reply()) Schedule reply immediately 00000100:00100000:2.0:1776178221.787084:0:8250:0:(service.c:2372:ptlrpc_server_handle_request()) Handled RPC req@00000000f871d6a5 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_003:b50963bd-8eba-4432-9a54-8888d00e3fa1+21:18365:x1862456979110144:12345-192.168.204.3@tcp:36:chmod.0 Request processed in 544us (751us total) trans 12884901981 rc 0/0 00000100:00100000:2.0:1776178221.787092:0:8250:0:(nrs_fifo.c:241:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.204.3@tcp, seq: 6635 00010000:00010000:3.0:1776178221.787170:0:3339:0:(ldlm_lock.c:1064:ldlm_granted_list_add_lock()) ### About to add lock: ns: mdt-lustre-MDT0000_UUID lock: 000000000396420a/0xa40083ab9bf53aab lrc: 3/0,1 mode: COS/COS res: [0x2000013a1:0xe:0x0].0x0 bits 0x13/0x0 rrc: 2 type: IBT gid 0 flags: 0x50210000000000 nid: local remote: 0x0 expref: -99 pid: 8250 timeout: 0 lvb_type: 0 00010000:00010000:3.0:1776178221.787181:0:3339:0:(ldlm_inodebits.c:94:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x2000013a1:0xe:0x0].0x0 (0000000030fbaafd) 00010000:00010000:3.0:1776178221.787183:0:3339:0:(ldlm_inodebits.c:96:ldlm_reprocess_inodebits_queue()) Hint 13 00000100:00100000:1.0:1776178222.433511:0:3355:0:(client.c:1733:ptlrpc_send_new_req()) Sending RPC req@000000006e2391ef pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_02:lustre-MDT0001-mdtlov_UUID:3355:1862456998051456:0@lo:13:osp-pre-1-1.0 00000100:00100000:1.0:1776178222.433613:0:3355:0:(events.c:361:request_in_callback()) peer: 12345-0@lo (source: 12345-0@lo) 00000100:00100000:3.0:1776178222.433988:0:9554:0:(service.c:2152:ptlrpc_server_handle_req_in()) got req x1862456998051456 00000100:00100000:3.0:1776178222.434052:0:9554:0:(nrs_fifo.c:179:nrs_fifo_req_get()) NRS start fifo request from 12345-0@lo, seq: 495 00000100:00100000:3.0:1776178222.434084:0:9554:0:(service.c:2324:ptlrpc_server_handle_request()) Handling RPC req@0000000067c1ae4f pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_create00:lustre-MDT0001-mdtlov_UUID+5:3355:x1862456998051456:12345-0@lo:13:osp-pre-1-1.0 00002000:00000024:3.0:1776178222.434113:0:9554:0:(ofd_obd.c:740:ofd_statfs()) blocks cached 229376 granted 491746368 pending 0 free 3923689472 avail 3692167168 00000100:00100000:3.0:1776178222.434178:0:9554:0:(service.c:2372:ptlrpc_server_handle_request()) Handled RPC req@0000000067c1ae4f pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_create00:lustre-MDT0001-mdtlov_UUID+5:3355:x1862456998051456:12345-0@lo:13:osp-pre-1-1.0 Request processed in 95us (568us total) trans 0 rc 0/0 00000100:00100000:3.0:1776178222.434186:0:9554:0:(nrs_fifo.c:241:nrs_fifo_req_stop()) NRS stop fifo request from 12345-0@lo, seq: 495 00000100:00100000:2.0:1776178222.437484:0:3355:0:(client.c:2215:ptlrpc_check_set()) Completed RPC req@000000006e2391ef pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_02:lustre-MDT0001-mdtlov_UUID:3355:1862456998051456:0@lo:13:osp-pre-1-1.0 00010000:00010000:2.0:1776178222.561781:0:9224:0:(ldlm_lockd.c:2792:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:3.0:1776178222.946003:0:3356:0:(client.c:1733:ptlrpc_send_new_req()) Sending RPC req@00000000a101cbf3 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_03:lustre-MDT0001-mdtlov_UUID:3356:1862456998051520:0@lo:13:osp-pre-0-1.0 00000100:00100000:3.0:1776178222.946309:0:3356:0:(events.c:361:request_in_callback()) peer: 12345-0@lo (source: 12345-0@lo) 00000100:00100000:0.0:1776178222.955728:0:8231:0:(service.c:2152:ptlrpc_server_handle_req_in()) got req x1862456998051520 00000100:00100000:0.0:1776178222.956283:0:9554:0:(nrs_fifo.c:179:nrs_fifo_req_get()) NRS start fifo request from 12345-0@lo, seq: 496 00000100:00100000:0.0:1776178222.956292:0:9554:0:(service.c:2324:ptlrpc_server_handle_request()) Handling RPC req@000000006e2391ef pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_create00:lustre-MDT0001-mdtlov_UUID+5:3356:x1862456998051520:12345-0@lo:13:osp-pre-0-1.0 00002000:00000024:0.0:1776178222.956348:0:9554:0:(ofd_obd.c:740:ofd_statfs()) blocks cached 315392 granted 484883776 pending 0 free 3923689472 avail 3692167168 00000100:00100000:0.0:1776178222.957087:0:9554:0:(service.c:2372:ptlrpc_server_handle_request()) Handled RPC req@000000006e2391ef pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_create00:lustre-MDT0001-mdtlov_UUID+5:3356:x1862456998051520:12345-0@lo:13:osp-pre-0-1.0 Request processed in 792us (10777us total) trans 0 rc 0/0 00000100:00100000:0.0:1776178222.957098:0:9554:0:(nrs_fifo.c:241:nrs_fifo_req_stop()) NRS stop fifo request from 12345-0@lo, seq: 496 00000100:00100000:1.0:1776178222.957140:0:3356:0:(client.c:2215:ptlrpc_check_set()) Completed RPC req@00000000a101cbf3 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_03:lustre-MDT0001-mdtlov_UUID:3356:1862456998051520:0@lo:13:osp-pre-0-1.0 00010000:00010000:1.0:1776178223.584581:0:6244:0:(ldlm_lockd.c:2792:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:3.0:1776178224.424889:0:3346:0:(events.c:361:request_in_callback()) peer: 12345-192.168.204.3@tcp (source: 12345-192.168.204.3@tcp) 00000100:00100000:3.0:1776178224.424941:0:3346:0:(events.c:361:request_in_callback()) peer: 12345-192.168.204.3@tcp (source: 12345-192.168.204.3@tcp) 00000100:00100000:3.0:1776178224.424996:0:3346:0:(events.c:361:request_in_callback()) peer: 12345-192.168.204.3@tcp (source: 12345-192.168.204.3@tcp) 00000100:00100000:3.0:1776178224.425026:0:3346:0:(events.c:361:request_in_callback()) peer: 12345-192.168.204.3@tcp (source: 12345-192.168.204.3@tcp) 00000100:00100000:3.0:1776178224.425050:0:3346:0:(events.c:361:request_in_callback()) peer: 12345-192.168.204.3@tcp (source: 12345-192.168.204.3@tcp) 00000100:00100000:1.0:1776178224.425149:0:20762:0:(service.c:2152:ptlrpc_server_handle_req_in()) got req x1862456979110400 00000100:00100000:3.0:1776178224.425149:0:20137:0:(service.c:2152:ptlrpc_server_handle_req_in()) got req x1862456979110208 00000100:00100000:1.0:1776178224.425205:0:20762:0:(service.c:2152:ptlrpc_server_handle_req_in()) got req x1862456979110464 00000100:00100000:3.0:1776178224.425208:0:20137:0:(nrs_fifo.c:179:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.204.3@tcp, seq: 164 00000100:00100000:3.0:1776178224.425216:0:20137:0:(service.c:2324:ptlrpc_server_handle_request()) Handling RPC req@00000000b42c55dd pname:cluuid+ref:pid:xid:nid:opc:job ll_mgs_0000:5b5c66b1-1a2f-4651-8548-0c75b9a38a94+9:2270:x1862456979110208:12345-192.168.204.3@tcp:400:kworker/u8:3.0 00000100:00100000:3.0:1776178224.425324:0:20137:0:(service.c:2372:ptlrpc_server_handle_request()) Handled RPC req@00000000b42c55dd pname:cluuid+ref:pid:xid:nid:opc:job ll_mgs_0000:5b5c66b1-1a2f-4651-8548-0c75b9a38a94+9:2270:x1862456979110208:12345-192.168.204.3@tcp:400:kworker/u8:3.0 Request processed in 108us (328us total) trans 0 rc 0/0 00000100:00100000:3.0:1776178224.425332:0:20137:0:(nrs_fifo.c:241:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.204.3@tcp, seq: 164 00000100:00100000:3.0:1776178224.425381:0:8250:0:(service.c:2152:ptlrpc_server_handle_req_in()) got req x1862456979110272 00000100:00100000:3.0:1776178224.425414:0:8250:0:(service.c:2152:ptlrpc_server_handle_req_in()) got req x1862456979110336 00000100:00100000:3.0:1776178224.425432:0:8250:0:(nrs_fifo.c:179:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.204.3@tcp, seq: 939 00000100:00100000:1.0:1776178224.425434:0:20762:0:(nrs_fifo.c:179:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.204.3@tcp, seq: 527 00000100:00100000:3.0:1776178224.425436:0:8250:0:(service.c:2324:ptlrpc_server_handle_request()) Handling RPC req@000000007886b426 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_003:b50963bd-8eba-4432-9a54-8888d00e3fa1+21:2270:x1862456979110272:12345-192.168.204.3@tcp:400:kworker/u8:3.0 00000100:00100000:1.0:1776178224.425438:0:20762:0:(service.c:2324:ptlrpc_server_handle_request()) Handling RPC req@00000000a5decd95 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_013:b50963bd-8eba-4432-9a54-8888d00e3fa1+8:2270:x1862456979110400:12345-192.168.204.3@tcp:400:kworker/u8:3.0 00010000:00000080:3.0:1776178224.425460:0:8250:0:(ldlm_lib.c:3181:target_committed_to_req()) @@@ not sending last_committed update (1/0) req@000000007886b426 x1862456979110272/t0(0) o400->b50963bd-8eba-4432-9a54-8888d00e3fa1@192.168.204.3@tcp:1/0 lens 224/224 e 0 to 0 dl 1776178271 ref 1 fl Interpret:H/0/0 rc 0/0 job:'kworker/u8:3.0' 00000100:00100000:1.0:1776178224.425468:0:20762:0:(service.c:2372:ptlrpc_server_handle_request()) Handled RPC req@00000000a5decd95 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_013:b50963bd-8eba-4432-9a54-8888d00e3fa1+8:2270:x1862456979110400:12345-192.168.204.3@tcp:400:kworker/u8:3.0 Request processed in 30us (528us total) trans 0 rc 0/0 00000100:00100000:1.0:1776178224.425474:0:20762:0:(nrs_fifo.c:241:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.204.3@tcp, seq: 527 00000100:00100000:1.0:1776178224.425486:0:20762:0:(nrs_fifo.c:179:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.204.3@tcp, seq: 528 00000100:00100000:3.0:1776178224.425487:0:8250:0:(service.c:2372:ptlrpc_server_handle_request()) Handled RPC req@000000007886b426 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_003:b50963bd-8eba-4432-9a54-8888d00e3fa1+21:2270:x1862456979110272:12345-192.168.204.3@tcp:400:kworker/u8:3.0 Request processed in 52us (615us total) trans 0 rc 0/0 00000100:00100000:1.0:1776178224.425490:0:20762:0:(service.c:2324:ptlrpc_server_handle_request()) Handling RPC req@0000000022d4dc29 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_013:b50963bd-8eba-4432-9a54-8888d00e3fa1+9:2270:x1862456979110464:12345-192.168.204.3@tcp:400:kworker/u8:3.0 00000100:00100000:3.0:1776178224.425494:0:8250:0:(nrs_fifo.c:241:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.204.3@tcp, seq: 939 00000100:00100000:3.0:1776178224.425504:0:8250:0:(nrs_fifo.c:179:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.204.3@tcp, seq: 940 00000100:00100000:3.0:1776178224.425507:0:8250:0:(service.c:2324:ptlrpc_server_handle_request()) Handling RPC req@00000000699e66b5 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_003:b50963bd-8eba-4432-9a54-8888d00e3fa1+6:2270:x1862456979110336:12345-192.168.204.3@tcp:400:kworker/u8:3.0 00000100:00100000:1.0:1776178224.425509:0:20762:0:(service.c:2372:ptlrpc_server_handle_request()) Handled RPC req@0000000022d4dc29 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_013:b50963bd-8eba-4432-9a54-8888d00e3fa1+9:2270:x1862456979110464:12345-192.168.204.3@tcp:400:kworker/u8:3.0 Request processed in 19us (460us total) trans 0 rc 0/0 00000100:00100000:1.0:1776178224.425515:0:20762:0:(nrs_fifo.c:241:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.204.3@tcp, seq: 528 00000100:00100000:3.0:1776178224.425527:0:8250:0:(service.c:2372:ptlrpc_server_handle_request()) Handled RPC req@00000000699e66b5 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_003:b50963bd-8eba-4432-9a54-8888d00e3fa1+6:2270:x1862456979110336:12345-192.168.204.3@tcp:400:kworker/u8:3.0 Request processed in 19us (501us total) trans 0 rc 0/0 00000100:00100000:3.0:1776178224.425532:0:8250:0:(nrs_fifo.c:241:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.204.3@tcp, seq: 940 00010000:00010000:1.0:1776178224.608257:0:9225:0:(ldlm_lockd.c:2792:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00010000:00010000:2.0:1776178225.632235:0:6245:0:(ldlm_lockd.c:2792:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000020:01000004:0.0:1776178225.856875:0:22200:0:(obd_mount_server.c:1591:server_put_super()) server put_super lustre-MDT0000 00010000:00010000:0.0:1776178225.856905:0:22200:0:(ldlm_lock.c:766:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(CR) ns: MGC192.168.204.103@tcp lock: 0000000058858c41/0xa40083ab9bf32298 lrc: 3/1,0 mode: CR/CR res: [0x65727473756c:0x0:0x0].0x0 rrc: 10 type: PLN flags: 0x11000000000000 nid: local remote: 0xa40083ab9bf3229f expref: -99 pid: 20132 timeout: 0 lvb_type: 0 00010000:00010000:0.0:1776178225.856925:0:22200:0:(ldlm_lock.c:942:ldlm_lock_decref_and_cancel()) ### ldlm_lock_decref(CR) ns: MGC192.168.204.103@tcp lock: 0000000058858c41/0xa40083ab9bf32298 lrc: 3/1,0 mode: CR/CR res: [0x65727473756c:0x0:0x0].0x0 rrc: 10 type: PLN flags: 0x1000000000000 nid: local remote: 0xa40083ab9bf3229f expref: -99 pid: 20132 timeout: 0 lvb_type: 0 00010000:00010000:0.0:1776178225.856932:0:22200:0:(ldlm_lock.c:820:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(CR) ns: MGC192.168.204.103@tcp lock: 0000000058858c41/0xa40083ab9bf32298 lrc: 3/1,0 mode: CR/CR res: [0x65727473756c:0x0:0x0].0x0 rrc: 10 type: PLN flags: 0x11000400000000 nid: local remote: 0xa40083ab9bf3229f expref: -99 pid: 20132 timeout: 0 lvb_type: 0 00010000:00010000:0.0:1776178225.856939:0:22200:0:(ldlm_lock.c:879:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: MGC192.168.204.103@tcp lock: 0000000058858c41/0xa40083ab9bf32298 lrc: 2/0,0 mode: CR/CR res: [0x65727473756c:0x0:0x0].0x0 rrc: 10 type: PLN flags: 0x11000400000000 nid: local remote: 0xa40083ab9bf3229f expref: -99 pid: 20132 timeout: 0 lvb_type: 0 00010000:00010000:0.0:1776178225.856964:0:22200:0:(ldlm_lockd.c:2159:__ldlm_bl_to_thread()) added 1/1 locks to regular blp list, 1 blwis in pool 10000000:01000000:0.0:1776178225.856969:0:22200:0:(mgc_request.c:546:config_log_end()) end config log lustre-client (0) 00000100:00080000:0.0:1776178225.856982:0:22200:0:(import.c:157:ptlrpc_deactivate_import_nolock()) setting import lustre-MDT0000_UUID INVALID 00000100:00100000:0.0:1776178225.856987:0:22200:0:(client.c:2868:ptlrpc_free_committed()) lustre-MDT0000-lwp-MDT0000: committing for last_committed 0 gen 3 00000100:00080000:0.0:1776178225.856993:0:22200:0:(pinger.c:414:ptlrpc_pinger_del_import()) removing pingable import lustre-MDT0000-lwp-MDT0000_UUID->lustre-MDT0000_UUID 00000100:00080000:0.0:1776178225.857024:0:22200:0:(import.c:86:import_set_state_nolock()) 00000000989ee371 lustre-MDT0000_UUID: changing import state from FULL to CONNECTING 00000100:00100000:0.0:1776178225.857047:0:22200:0:(client.c:1733:ptlrpc_send_new_req()) Sending RPC req@00000000ecea139c pname:cluuid:pid:xid:nid:opc:job umount:lustre-MDT0000-lwp-MDT0000_UUID:22200:1862456998051584:0@lo:39:umount.0 00000100:00100000:0.0:1776178225.857157:0:22200:0:(events.c:361:request_in_callback()) peer: 12345-0@lo (source: 12345-0@lo) 00000100:00100000:0.0:1776178225.857186:0:22200:0:(client.c:2500:ptlrpc_set_wait()) set 00000000835d6110 going to sleep for 5 seconds 00000100:00100000:0.0:1776178225.857219:0:11281:0:(service.c:2152:ptlrpc_server_handle_req_in()) got req x1862456998051584 00000100:00100000:0.0:1776178225.857276:0:11281:0:(nrs_fifo.c:179:nrs_fifo_req_get()) NRS start fifo request from 12345-0@lo, seq: 6636 00000100:00100000:0.0:1776178225.857294:0:11281:0:(service.c:2324:ptlrpc_server_handle_request()) Handling RPC req@0000000014e690d4 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_004:lustre-MDT0000-lwp-MDT0000_UUID+5:22200:x1862456998051584:12345-0@lo:39:umount.0 00000020:00000080:0.0:1776178225.857326:0:11281:0:(genops.c:1442:class_disconnect()) disconnect: cookie 0xa40083ab9bf324eb 00000100:00080000:0.0:1776178225.857334:0:11281:0:(import.c:86:import_set_state_nolock()) 0000000097a8d6e8 : changing import state from FULL to CLOSED 00010000:00000080:0.0:1776178225.857446:0:11281:0:(ldlm_lib.c:3181:target_committed_to_req()) @@@ not sending last_committed update (1/0) req@0000000014e690d4 x1862456998051584/t0(0) o39->lustre-MDT0000-lwp-MDT0000_UUID@0@lo:2/0 lens 224/224 e 0 to 0 dl 1776178272 ref 1 fl Interpret:/0/0 rc 0/0 job:'umount.0' 00000100:00100000:0.0:1776178225.857481:0:11281:0:(service.c:2372:ptlrpc_server_handle_request()) Handled RPC req@0000000014e690d4 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_004:lustre-MDT0000-lwp-MDT0000_UUID+1:22200:x1862456998051584:12345-0@lo:39:umount.0 Request processed in 198us (325us total) trans 0 rc 0/0 00000100:00100000:0.0:1776178225.857488:0:11281:0:(nrs_fifo.c:241:nrs_fifo_req_stop()) NRS stop fifo request from 12345-0@lo, seq: 6636 00000020:00000080:0.0:1776178225.857497:0:11281:0:(genops.c:985:class_export_put()) final put 00000000b1fdffbd/lustre-MDT0000-lwp-MDT0000_UUID 00000020:00000080:0.0:1776178225.857527:0:36:0:(genops.c:944:class_export_destroy()) destroying export 00000000b1fdffbd/lustre-MDT0000-lwp-MDT0000_UUID for lustre-MDT0000 00000020:00000080:0.0:1776178225.857597:0:36:0:(genops.c:1164:obd_zombie_import_free()) destroying import 0000000097a8d6e8 for lustre-MDT0000 00000100:00100000:0.0:1776178225.857634:0:22200:0:(client.c:2215:ptlrpc_check_set()) Completed RPC req@00000000ecea139c pname:cluuid:pid:xid:nid:opc:job umount:lustre-MDT0000-lwp-MDT0000_UUID:22200:1862456998051584:0@lo:39:umount.0 00000100:00080000:0.0:1776178225.857650:0:22200:0:(import.c:86:import_set_state_nolock()) 00000000989ee371 lustre-MDT0000_UUID: changing import state from CONNECTING to CLOSED 00000100:00080000:0.0:1776178225.857655:0:22200:0:(import.c:157:ptlrpc_deactivate_import_nolock()) setting import lustre-MDT0000_UUID INVALID 00000100:00100000:0.0:1776178225.857656:0:22200:0:(client.c:2868:ptlrpc_free_committed()) lustre-MDT0000-lwp-MDT0000: committing for last_committed 0 gen 4 00000100:00100000:0.0:1776178225.857659:0:22200:0:(import.c:339:ptlrpc_invalidate_import()) Sleeping 20s for inflight to error out 00010000:00010000:0.0:1776178225.858557:0:22200:0:(ldlm_lock.c:766:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(CR) ns: MGC192.168.204.103@tcp lock: 000000003208ae83/0xa40083ab9bf321c6 lrc: 3/1,0 mode: CR/CR res: [0x65727473756c:0x0:0x0].0x0 rrc: 10 type: PLN flags: 0x11000000000000 nid: local remote: 0xa40083ab9bf321cd expref: -99 pid: 20132 timeout: 0 lvb_type: 0 00010000:00010000:0.0:1776178225.858576:0:22200:0:(ldlm_lock.c:942:ldlm_lock_decref_and_cancel()) ### ldlm_lock_decref(CR) ns: MGC192.168.204.103@tcp lock: 000000003208ae83/0xa40083ab9bf321c6 lrc: 3/1,0 mode: CR/CR res: [0x65727473756c:0x0:0x0].0x0 rrc: 10 type: PLN flags: 0x1000000000000 nid: local remote: 0xa40083ab9bf321cd expref: -99 pid: 20132 timeout: 0 lvb_type: 0 00010000:00010000:0.0:1776178225.858591:0:22200:0:(ldlm_lock.c:820:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(CR) ns: MGC192.168.204.103@tcp lock: 000000003208ae83/0xa40083ab9bf321c6 lrc: 3/1,0 mode: CR/CR res: [0x65727473756c:0x0:0x0].0x0 rrc: 10 type: PLN flags: 0x11000400000000 nid: local remote: 0xa40083ab9bf321cd expref: -99 pid: 20132 timeout: 0 lvb_type: 0 00010000:00010000:0.0:1776178225.858597:0:22200:0:(ldlm_lock.c:879:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: MGC192.168.204.103@tcp lock: 000000003208ae83/0xa40083ab9bf321c6 lrc: 2/0,0 mode: CR/CR res: [0x65727473756c:0x0:0x0].0x0 rrc: 10 type: PLN flags: 0x11000400000000 nid: local remote: 0xa40083ab9bf321cd expref: -99 pid: 20132 timeout: 0 lvb_type: 0 00010000:00010000:0.0:1776178225.858618:0:22200:0:(ldlm_lockd.c:2159:__ldlm_bl_to_thread()) added 1/2 locks to regular blp list, 2 blwis in pool 00010000:00010000:0.0:1776178225.858623:0:22200:0:(ldlm_lock.c:766:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(CR) ns: MGC192.168.204.103@tcp lock: 00000000f2a57534/0xa40083ab9bf323f6 lrc: 3/1,0 mode: CR/CR res: [0x65727473756c:0x2:0x0].0x0 rrc: 3 type: PLN flags: 0x11000000000000 nid: local remote: 0xa40083ab9bf323fd expref: -99 pid: 6253 timeout: 0 lvb_type: 0 00010000:00010000:0.0:1776178225.858628:0:22200:0:(ldlm_lock.c:942:ldlm_lock_decref_and_cancel()) ### ldlm_lock_decref(CR) ns: MGC192.168.204.103@tcp lock: 00000000f2a57534/0xa40083ab9bf323f6 lrc: 3/1,0 mode: CR/CR res: [0x65727473756c:0x2:0x0].0x0 rrc: 3 type: PLN flags: 0x1000000000000 nid: local remote: 0xa40083ab9bf323fd expref: -99 pid: 6253 timeout: 0 lvb_type: 0 00010000:00010000:0.0:1776178225.858632:0:22200:0:(ldlm_lock.c:820:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(CR) ns: MGC192.168.204.103@tcp lock: 00000000f2a57534/0xa40083ab9bf323f6 lrc: 3/1,0 mode: CR/CR res: [0x65727473756c:0x2:0x0].0x0 rrc: 3 type: PLN flags: 0x11000400000000 nid: local remote: 0xa40083ab9bf323fd expref: -99 pid: 6253 timeout: 0 lvb_type: 0 00010000:00010000:0.0:1776178225.858638:0:22200:0:(ldlm_lock.c:879:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: MGC192.168.204.103@tcp lock: 00000000f2a57534/0xa40083ab9bf323f6 lrc: 2/0,0 mode: CR/CR res: [0x65727473756c:0x2:0x0].0x0 rrc: 3 type: PLN flags: 0x11000400000000 nid: local remote: 0xa40083ab9bf323fd expref: -99 pid: 6253 timeout: 0 lvb_type: 0 00010000:00010000:0.0:1776178225.858649:0:22200:0:(ldlm_lockd.c:2159:__ldlm_bl_to_thread()) added 1/3 locks to regular blp list, 3 blwis in pool 00010000:00010000:0.0:1776178225.858653:0:22200:0:(ldlm_lock.c:766:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(CR) ns: MGC192.168.204.103@tcp lock: 0000000069735826/0xa40083ab9bf3228a lrc: 3/1,0 mode: CR/CR res: [0x736d61726170:0x3:0x0].0x0 rrc: 5 type: PLN flags: 0x11000000000000 nid: local remote: 0xa40083ab9bf32291 expref: -99 pid: 20132 timeout: 0 lvb_type: 0 00010000:00010000:0.0:1776178225.858658:0:22200:0:(ldlm_lock.c:942:ldlm_lock_decref_and_cancel()) ### ldlm_lock_decref(CR) ns: MGC192.168.204.103@tcp lock: 0000000069735826/0xa40083ab9bf3228a lrc: 3/1,0 mode: CR/CR res: [0x736d61726170:0x3:0x0].0x0 rrc: 5 type: PLN flags: 0x1000000000000 nid: local remote: 0xa40083ab9bf32291 expref: -99 pid: 20132 timeout: 0 lvb_type: 0 00010000:00010000:0.0:1776178225.858662:0:22200:0:(ldlm_lock.c:820:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(CR) ns: MGC192.168.204.103@tcp lock: 0000000069735826/0xa40083ab9bf3228a lrc: 3/1,0 mode: CR/CR res: [0x736d61726170:0x3:0x0].0x0 rrc: 5 type: PLN flags: 0x11000400000000 nid: local remote: 0xa40083ab9bf32291 expref: -99 pid: 20132 timeout: 0 lvb_type: 0 00010000:00010000:0.0:1776178225.858668:0:22200:0:(ldlm_lock.c:879:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: MGC192.168.204.103@tcp lock: 0000000069735826/0xa40083ab9bf3228a lrc: 2/0,0 mode: CR/CR res: [0x736d61726170:0x3:0x0].0x0 rrc: 5 type: PLN flags: 0x11000400000000 nid: local remote: 0xa40083ab9bf32291 expref: -99 pid: 20132 timeout: 0 lvb_type: 0 00010000:00010000:0.0:1776178225.858676:0:22200:0:(ldlm_lockd.c:2159:__ldlm_bl_to_thread()) added 1/4 locks to regular blp list, 4 blwis in pool 00010000:00010000:0.0:1776178225.858680:0:22200:0:(ldlm_lock.c:766:ldlm_lock_addref_internal_nolock()) ### ldlm_lock_addref(CR) ns: MGC192.168.204.103@tcp lock: 000000006a2bf54b/0xa40083ab9bf321b8 lrc: 3/1,0 mode: CR/CR res: [0x65727473756c:0x5:0x0].0x0 rrc: 3 type: PLN flags: 0x11000000000000 nid: local remote: 0xa40083ab9bf321bf expref: -99 pid: 20132 timeout: 0 lvb_type: 0 00010000:00010000:0.0:1776178225.858685:0:22200:0:(ldlm_lock.c:942:ldlm_lock_decref_and_cancel()) ### ldlm_lock_decref(CR) ns: MGC192.168.204.103@tcp lock: 000000006a2bf54b/0xa40083ab9bf321b8 lrc: 3/1,0 mode: CR/CR res: [0x65727473756c:0x5:0x0].0x0 rrc: 3 type: PLN flags: 0x1000000000000 nid: local remote: 0xa40083ab9bf321bf expref: -99 pid: 20132 timeout: 0 lvb_type: 0 00010000:00010000:0.0:1776178225.858690:0:22200:0:(ldlm_lock.c:820:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(CR) ns: MGC192.168.204.103@tcp lock: 000000006a2bf54b/0xa40083ab9bf321b8 lrc: 3/1,0 mode: CR/CR res: [0x65727473756c:0x5:0x0].0x0 rrc: 3 type: PLN flags: 0x11000400000000 nid: local remote: 0xa40083ab9bf321bf expref: -99 pid: 20132 timeout: 0 lvb_type: 0 00010000:00010000:0.0:1776178225.858694:0:22200:0:(ldlm_lock.c:879:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: MGC192.168.204.103@tcp lock: 000000006a2bf54b/0xa40083ab9bf321b8 lrc: 2/0,0 mode: CR/CR res: [0x65727473756c:0x5:0x0].0x0 rrc: 3 type: PLN flags: 0x11000400000000 nid: local remote: 0xa40083ab9bf321bf expref: -99 pid: 20132 timeout: 0 lvb_type: 0 00010000:00010000:0.0:1776178225.858699:0:22200:0:(ldlm_lockd.c:2159:__ldlm_bl_to_thread()) added 1/5 locks to regular blp list, 5 blwis in pool 10000000:01000000:0.0:1776178225.858701:0:22200:0:(mgc_request.c:546:config_log_end()) end config log lustre-MDT0000 (0) 00000020:01000004:0.0:1776178225.858713:0:22200:0:(obd_mount_server.c:1629:server_put_super()) stopping lustre-MDT0000 00000020:01000000:0.0:1776178225.858718:0:22200:0:(obd_config.c:2324:class_manual_cleanup()) Manual cleanup of lustre-MDT0000 (flags='FA') 00000020:00000080:0.0:1776178225.858722:0:22200:0:(obd_config.c:1384:class_process_config()) processing cmd: cf004 00000020:02000400:0.0:1776178225.858728:0:22200:0:(obd_config.c:854:class_cleanup()) Failing over lustre-MDT0000 00010000:00010000:1.0:1776178225.860841:0:6244:0:(ldlm_lockd.c:2789:ldlm_bl_get_work()) Got 1 locks of 4 total in blp. (4 blwis in pool) 00010000:00010000:1.0:1776178225.860859:0:6244:0:(ldlm_lockd.c:1879:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: MGC192.168.204.103@tcp lock: 0000000058858c41/0xa40083ab9bf32298 lrc: 2/0,0 mode: CR/CR res: [0x65727473756c:0x0:0x0].0x0 rrc: 10 type: PLN flags: 0x1000400000000 nid: local remote: 0xa40083ab9bf3229f expref: -99 pid: 20132 timeout: 0 lvb_type: 0 00010000:00010000:1.0:1776178225.860874:0:6244:0:(ldlm_lockd.c:1893:ldlm_handle_bl_callback()) Lock 0000000058858c41 already unused, calling callback (00000000c05313d7) 10000000:00010000:1.0:1776178225.860878:0:6244:0:(mgc_request.c:1021:mgc_blocking_ast()) ### MGC blocking CB ns: MGC192.168.204.103@tcp lock: 0000000058858c41/0xa40083ab9bf32298 lrc: 2/0,0 mode: CR/CR res: [0x65727473756c:0x0:0x0].0x0 rrc: 10 type: PLN flags: 0x1000400000000 nid: local remote: 0xa40083ab9bf3229f expref: -99 pid: 20132 timeout: 0 lvb_type: 0 00010000:00010000:1.0:1776178225.860889:0:6244:0:(ldlm_request.c:1232:ldlm_cli_cancel_local()) ### client-side cancel ns: MGC192.168.204.103@tcp lock: 0000000058858c41/0xa40083ab9bf32298 lrc: 3/0,0 mode: CR/CR res: [0x65727473756c:0x0:0x0].0x0 rrc: 10 type: PLN flags: 0x1008400000000 nid: local remote: 0xa40083ab9bf3229f expref: -99 pid: 20132 timeout: 0 lvb_type: 0 10000000:00010000:1.0:1776178225.860896:0:6244:0:(mgc_request.c:1027:mgc_blocking_ast()) ### MGC cancel CB ns: MGC192.168.204.103@tcp lock: 0000000058858c41/0xa40083ab9bf32298 lrc: 3/0,0 mode: CR/CR res: [0x65727473756c:0x0:0x0].0x0 rrc: 10 type: PLN flags: 0x1009400000000 nid: local remote: 0xa40083ab9bf3229f expref: -99 pid: 20132 timeout: 0 lvb_type: 0 10000000:01000000:1.0:1776178225.860902:0:6244:0:(mgc_request.c:1031:mgc_blocking_ast()) Lock res [0x65727473756c:0x0:0x0].0x0 (lustre) 10000000:01000000:1.0:1776178225.860905:0:6244:0:(mgc_request.c:1046:mgc_blocking_ast()) log lustre-client: stopping, won't requeue 10000000:01000000:1.0:1776178225.860908:0:6244:0:(mgc_request.c:153:config_log_put()) dropping config log lustre-client 00010000:00010000:1.0:1776178225.860965:0:6244:0:(ldlm_request.c:1279:__ldlm_pack_lock()) ### packing ns: MGC192.168.204.103@tcp lock: 0000000058858c41/0xa40083ab9bf32298 lrc: 2/0,0 mode: --/CR res: [0x65727473756c:0x0:0x0].0x0 rrc: 10 type: PLN flags: 0x5809400000020 nid: local remote: 0xa40083ab9bf3229f expref: -99 pid: 20132 timeout: 0 lvb_type: 0 00010000:00010000:1.0:1776178225.860972:0:6244:0:(ldlm_request.c:1325:ldlm_cancel_pack()) 1 locks packed 00010000:00010000:1.0:1776178225.861009:0:6244:0:(ldlm_lockd.c:1903:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: MGC192.168.204.103@tcp lock: 0000000058858c41/0xa40083ab9bf32298 lrc: 1/0,0 mode: --/CR res: [0x65727473756c:0x0:0x0].0x0 rrc: 10 type: PLN flags: 0x5809400000020 nid: local remote: 0xa40083ab9bf3229f expref: -99 pid: 20132 timeout: 0 lvb_type: 0 00010000:00010000:1.0:1776178225.861016:0:6244:0:(ldlm_lock.c:216:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: MGC192.168.204.103@tcp lock: 0000000058858c41/0xa40083ab9bf32298 lrc: 0/0,0 mode: --/CR res: [0x65727473756c:0x0:0x0].0x0 rrc: 10 type: PLN flags: 0x5809400000020 nid: local remote: 0xa40083ab9bf3229f expref: -99 pid: 20132 timeout: 0 lvb_type: 0 00010000:00010000:1.0:1776178225.861030:0:6244:0:(ldlm_lockd.c:2789:ldlm_bl_get_work()) Got 1 locks of 3 total in blp. (3 blwis in pool) 00010000:00010000:1.0:1776178225.861032:0:6244:0:(ldlm_lockd.c:1879:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: MGC192.168.204.103@tcp lock: 000000003208ae83/0xa40083ab9bf321c6 lrc: 2/0,0 mode: CR/CR res: [0x65727473756c:0x0:0x0].0x0 rrc: 9 type: PLN flags: 0x1000400000000 nid: local remote: 0xa40083ab9bf321cd expref: -99 pid: 20132 timeout: 0 lvb_type: 0 00010000:00010000:1.0:1776178225.861037:0:6244:0:(ldlm_lockd.c:1893:ldlm_handle_bl_callback()) Lock 000000003208ae83 already unused, calling callback (00000000c05313d7) 10000000:00010000:1.0:1776178225.861039:0:6244:0:(mgc_request.c:1021:mgc_blocking_ast()) ### MGC blocking CB ns: MGC192.168.204.103@tcp lock: 000000003208ae83/0xa40083ab9bf321c6 lrc: 2/0,0 mode: CR/CR res: [0x65727473756c:0x0:0x0].0x0 rrc: 9 type: PLN flags: 0x1000400000000 nid: local remote: 0xa40083ab9bf321cd expref: -99 pid: 20132 timeout: 0 lvb_type: 0 00010000:00010000:1.0:1776178225.861044:0:6244:0:(ldlm_request.c:1232:ldlm_cli_cancel_local()) ### client-side cancel ns: MGC192.168.204.103@tcp lock: 000000003208ae83/0xa40083ab9bf321c6 lrc: 3/0,0 mode: CR/CR res: [0x65727473756c:0x0:0x0].0x0 rrc: 9 type: PLN flags: 0x1008400000000 nid: local remote: 0xa40083ab9bf321cd expref: -99 pid: 20132 timeout: 0 lvb_type: 0 10000000:00010000:1.0:1776178225.861049:0:6244:0:(mgc_request.c:1027:mgc_blocking_ast()) ### MGC cancel CB ns: MGC192.168.204.103@tcp lock: 000000003208ae83/0xa40083ab9bf321c6 lrc: 3/0,0 mode: CR/CR res: [0x65727473756c:0x0:0x0].0x0 rrc: 9 type: PLN flags: 0x1009400000000 nid: local remote: 0xa40083ab9bf321cd expref: -99 pid: 20132 timeout: 0 lvb_type: 0 10000000:01000000:1.0:1776178225.861054:0:6244:0:(mgc_request.c:1031:mgc_blocking_ast()) Lock res [0x65727473756c:0x0:0x0].0x0 (lustre) 10000000:01000000:1.0:1776178225.861056:0:6244:0:(mgc_request.c:1046:mgc_blocking_ast()) log lustre-MDT0000: stopping, won't requeue 10000000:01000000:1.0:1776178225.861169:0:6244:0:(mgc_request.c:153:config_log_put()) dropping config log lustre-MDT0000 00010000:00010000:1.0:1776178225.861188:0:6244:0:(ldlm_request.c:1279:__ldlm_pack_lock()) ### packing ns: MGC192.168.204.103@tcp lock: 000000003208ae83/0xa40083ab9bf321c6 lrc: 2/0,0 mode: --/CR res: [0x65727473756c:0x0:0x0].0x0 rrc: 9 type: PLN flags: 0x5809400000020 nid: local remote: 0xa40083ab9bf321cd expref: -99 pid: 20132 timeout: 0 lvb_type: 0 00010000:00010000:1.0:1776178225.861195:0:6244:0:(ldlm_request.c:1325:ldlm_cancel_pack()) 1 locks packed 00010000:00010000:1.0:1776178225.861199:0:6244:0:(ldlm_lockd.c:1903:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: MGC192.168.204.103@tcp lock: 000000003208ae83/0xa40083ab9bf321c6 lrc: 1/0,0 mode: --/CR res: [0x65727473756c:0x0:0x0].0x0 rrc: 9 type: PLN flags: 0x5809400000020 nid: local remote: 0xa40083ab9bf321cd expref: -99 pid: 20132 timeout: 0 lvb_type: 0 00010000:00010000:1.0:1776178225.861205:0:6244:0:(ldlm_lock.c:216:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: MGC192.168.204.103@tcp lock: 000000003208ae83/0xa40083ab9bf321c6 lrc: 0/0,0 mode: --/CR res: [0x65727473756c:0x0:0x0].0x0 rrc: 9 type: PLN flags: 0x5809400000020 nid: local remote: 0xa40083ab9bf321cd expref: -99 pid: 20132 timeout: 0 lvb_type: 0 00010000:00010000:1.0:1776178225.861213:0:6244:0:(ldlm_lockd.c:2789:ldlm_bl_get_work()) Got 1 locks of 2 total in blp. (2 blwis in pool) 00010000:00010000:1.0:1776178225.861216:0:6244:0:(ldlm_lockd.c:1879:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: MGC192.168.204.103@tcp lock: 00000000f2a57534/0xa40083ab9bf323f6 lrc: 2/0,0 mode: CR/CR res: [0x65727473756c:0x2:0x0].0x0 rrc: 3 type: PLN flags: 0x1000400000000 nid: local remote: 0xa40083ab9bf323fd expref: -99 pid: 6253 timeout: 0 lvb_type: 0 00010000:00010000:1.0:1776178225.861221:0:6244:0:(ldlm_lockd.c:1893:ldlm_handle_bl_callback()) Lock 00000000f2a57534 already unused, calling callback (00000000c05313d7) 10000000:00010000:1.0:1776178225.861222:0:6244:0:(mgc_request.c:1021:mgc_blocking_ast()) ### MGC blocking CB ns: MGC192.168.204.103@tcp lock: 00000000f2a57534/0xa40083ab9bf323f6 lrc: 2/0,0 mode: CR/CR res: [0x65727473756c:0x2:0x0].0x0 rrc: 3 type: PLN flags: 0x1000400000000 nid: local remote: 0xa40083ab9bf323fd expref: -99 pid: 6253 timeout: 0 lvb_type: 0 00010000:00010000:1.0:1776178225.861228:0:6244:0:(ldlm_request.c:1232:ldlm_cli_cancel_local()) ### client-side cancel ns: MGC192.168.204.103@tcp lock: 00000000f2a57534/0xa40083ab9bf323f6 lrc: 3/0,0 mode: CR/CR res: [0x65727473756c:0x2:0x0].0x0 rrc: 3 type: PLN flags: 0x1008400000000 nid: local remote: 0xa40083ab9bf323fd expref: -99 pid: 6253 timeout: 0 lvb_type: 0 10000000:00010000:1.0:1776178225.861233:0:6244:0:(mgc_request.c:1027:mgc_blocking_ast()) ### MGC cancel CB ns: MGC192.168.204.103@tcp lock: 00000000f2a57534/0xa40083ab9bf323f6 lrc: 3/0,0 mode: CR/CR res: [0x65727473756c:0x2:0x0].0x0 rrc: 3 type: PLN flags: 0x1009400000000 nid: local remote: 0xa40083ab9bf323fd expref: -99 pid: 6253 timeout: 0 lvb_type: 0 10000000:01000000:1.0:1776178225.861238:0:6244:0:(mgc_request.c:1031:mgc_blocking_ast()) Lock res [0x65727473756c:0x2:0x0].0x0 (lustre) 10000000:01000000:1.0:1776178225.861240:0:6244:0:(mgc_request.c:1046:mgc_blocking_ast()) log lustre-mdtir: stopping, won't requeue 10000000:01000000:1.0:1776178225.861241:0:6244:0:(mgc_request.c:153:config_log_put()) dropping config log lustre-mdtir 00010000:00010000:1.0:1776178225.861253:0:6244:0:(ldlm_request.c:1279:__ldlm_pack_lock()) ### packing ns: MGC192.168.204.103@tcp lock: 00000000f2a57534/0xa40083ab9bf323f6 lrc: 2/0,0 mode: --/CR res: [0x65727473756c:0x2:0x0].0x0 rrc: 3 type: PLN flags: 0x5809400000020 nid: local remote: 0xa40083ab9bf323fd expref: -99 pid: 6253 timeout: 0 lvb_type: 0 00010000:00010000:1.0:1776178225.861259:0:6244:0:(ldlm_request.c:1325:ldlm_cancel_pack()) 1 locks packed 00010000:00010000:1.0:1776178225.861273:0:6244:0:(ldlm_lockd.c:1903:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: MGC192.168.204.103@tcp lock: 00000000f2a57534/0xa40083ab9bf323f6 lrc: 1/0,0 mode: --/CR res: [0x65727473756c:0x2:0x0].0x0 rrc: 3 type: PLN flags: 0x5809400000020 nid: local remote: 0xa40083ab9bf323fd expref: -99 pid: 6253 timeout: 0 lvb_type: 0 00010000:00010000:1.0:1776178225.861278:0:6244:0:(ldlm_lock.c:216:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: MGC192.168.204.103@tcp lock: 00000000f2a57534/0xa40083ab9bf323f6 lrc: 0/0,0 mode: --/CR res: [0x65727473756c:0x2:0x0].0x0 rrc: 3 type: PLN flags: 0x5809400000020 nid: local remote: 0xa40083ab9bf323fd expref: -99 pid: 6253 timeout: 0 lvb_type: 0 00010000:00010000:1.0:1776178225.861297:0:6244:0:(ldlm_lockd.c:2789:ldlm_bl_get_work()) Got 1 locks of 1 total in blp. (1 blwis in pool) 00010000:00010000:1.0:1776178225.861299:0:6244:0:(ldlm_lockd.c:1879:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: MGC192.168.204.103@tcp lock: 0000000069735826/0xa40083ab9bf3228a lrc: 2/0,0 mode: CR/CR res: [0x736d61726170:0x3:0x0].0x0 rrc: 5 type: PLN flags: 0x1000400000000 nid: local remote: 0xa40083ab9bf32291 expref: -99 pid: 20132 timeout: 0 lvb_type: 0 00010000:00010000:1.0:1776178225.861305:0:6244:0:(ldlm_lockd.c:1893:ldlm_handle_bl_callback()) Lock 0000000069735826 already unused, calling callback (00000000c05313d7) 10000000:00010000:1.0:1776178225.861306:0:6244:0:(mgc_request.c:1021:mgc_blocking_ast()) ### MGC blocking CB ns: MGC192.168.204.103@tcp lock: 0000000069735826/0xa40083ab9bf3228a lrc: 2/0,0 mode: CR/CR res: [0x736d61726170:0x3:0x0].0x0 rrc: 5 type: PLN flags: 0x1000400000000 nid: local remote: 0xa40083ab9bf32291 expref: -99 pid: 20132 timeout: 0 lvb_type: 0 00010000:00010000:1.0:1776178225.861312:0:6244:0:(ldlm_request.c:1232:ldlm_cli_cancel_local()) ### client-side cancel ns: MGC192.168.204.103@tcp lock: 0000000069735826/0xa40083ab9bf3228a lrc: 3/0,0 mode: CR/CR res: [0x736d61726170:0x3:0x0].0x0 rrc: 5 type: PLN flags: 0x1008400000000 nid: local remote: 0xa40083ab9bf32291 expref: -99 pid: 20132 timeout: 0 lvb_type: 0 10000000:00010000:1.0:1776178225.861316:0:6244:0:(mgc_request.c:1027:mgc_blocking_ast()) ### MGC cancel CB ns: MGC192.168.204.103@tcp lock: 0000000069735826/0xa40083ab9bf3228a lrc: 3/0,0 mode: CR/CR res: [0x736d61726170:0x3:0x0].0x0 rrc: 5 type: PLN flags: 0x1009400000000 nid: local remote: 0xa40083ab9bf32291 expref: -99 pid: 20132 timeout: 0 lvb_type: 0 10000000:01000000:1.0:1776178225.861322:0:6244:0:(mgc_request.c:1031:mgc_blocking_ast()) Lock res [0x736d61726170:0x3:0x0].0x0 (params) 10000000:01000000:1.0:1776178225.861324:0:6244:0:(mgc_request.c:1046:mgc_blocking_ast()) log params: stopping, won't requeue 10000000:01000000:1.0:1776178225.861326:0:6244:0:(mgc_request.c:153:config_log_put()) dropping config log params 00010000:00010000:1.0:1776178225.861337:0:6244:0:(ldlm_request.c:1279:__ldlm_pack_lock()) ### packing ns: MGC192.168.204.103@tcp lock: 0000000069735826/0xa40083ab9bf3228a lrc: 2/0,0 mode: --/CR res: [0x736d61726170:0x3:0x0].0x0 rrc: 5 type: PLN flags: 0x5809400000020 nid: local remote: 0xa40083ab9bf32291 expref: -99 pid: 20132 timeout: 0 lvb_type: 0 00010000:00010000:1.0:1776178225.861343:0:6244:0:(ldlm_request.c:1325:ldlm_cancel_pack()) 1 locks packed 00010000:00010000:1.0:1776178225.861347:0:6244:0:(ldlm_lockd.c:1903:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: MGC192.168.204.103@tcp lock: 0000000069735826/0xa40083ab9bf3228a lrc: 1/0,0 mode: --/CR res: [0x736d61726170:0x3:0x0].0x0 rrc: 5 type: PLN flags: 0x5809400000020 nid: local remote: 0xa40083ab9bf32291 expref: -99 pid: 20132 timeout: 0 lvb_type: 0 00010000:00010000:1.0:1776178225.861352:0:6244:0:(ldlm_lock.c:216:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: MGC192.168.204.103@tcp lock: 0000000069735826/0xa40083ab9bf3228a lrc: 0/0,0 mode: --/CR res: [0x736d61726170:0x3:0x0].0x0 rrc: 5 type: PLN flags: 0x5809400000020 nid: local remote: 0xa40083ab9bf32291 expref: -99 pid: 20132 timeout: 0 lvb_type: 0 00010000:00010000:1.0:1776178225.861360:0:6244:0:(ldlm_lockd.c:2789:ldlm_bl_get_work()) Got 1 locks of 0 total in blp. (0 blwis in pool) 00010000:00010000:1.0:1776178225.861363:0:6244:0:(ldlm_lockd.c:1879:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: MGC192.168.204.103@tcp lock: 000000006a2bf54b/0xa40083ab9bf321b8 lrc: 2/0,0 mode: CR/CR res: [0x65727473756c:0x5:0x0].0x0 rrc: 3 type: PLN flags: 0x1000400000000 nid: local remote: 0xa40083ab9bf321bf expref: -99 pid: 20132 timeout: 0 lvb_type: 0 00010000:00010000:1.0:1776178225.861372:0:6244:0:(ldlm_lockd.c:1893:ldlm_handle_bl_callback()) Lock 000000006a2bf54b already unused, calling callback (00000000c05313d7) 10000000:00010000:1.0:1776178225.861373:0:6244:0:(mgc_request.c:1021:mgc_blocking_ast()) ### MGC blocking CB ns: MGC192.168.204.103@tcp lock: 000000006a2bf54b/0xa40083ab9bf321b8 lrc: 2/0,0 mode: CR/CR res: [0x65727473756c:0x5:0x0].0x0 rrc: 3 type: PLN flags: 0x1000400000000 nid: local remote: 0xa40083ab9bf321bf expref: -99 pid: 20132 timeout: 0 lvb_type: 0 00010000:00010000:1.0:1776178225.861379:0:6244:0:(ldlm_request.c:1232:ldlm_cli_cancel_local()) ### client-side cancel ns: MGC192.168.204.103@tcp lock: 000000006a2bf54b/0xa40083ab9bf321b8 lrc: 3/0,0 mode: CR/CR res: [0x65727473756c:0x5:0x0].0x0 rrc: 3 type: PLN flags: 0x1008400000000 nid: local remote: 0xa40083ab9bf321bf expref: -99 pid: 20132 timeout: 0 lvb_type: 0 10000000:00010000:1.0:1776178225.861384:0:6244:0:(mgc_request.c:1027:mgc_blocking_ast()) ### MGC cancel CB ns: MGC192.168.204.103@tcp lock: 000000006a2bf54b/0xa40083ab9bf321b8 lrc: 3/0,0 mode: CR/CR res: [0x65727473756c:0x5:0x0].0x0 rrc: 3 type: PLN flags: 0x1009400000000 nid: local remote: 0xa40083ab9bf321bf expref: -99 pid: 20132 timeout: 0 lvb_type: 0 10000000:01000000:1.0:1776178225.861388:0:6244:0:(mgc_request.c:1031:mgc_blocking_ast()) Lock res [0x65727473756c:0x5:0x0].0x0 (lustre) 10000000:01000000:1.0:1776178225.861390:0:6244:0:(mgc_request.c:1046:mgc_blocking_ast()) log lustre-barrier: stopping, won't requeue 10000000:01000000:1.0:1776178225.861391:0:6244:0:(mgc_request.c:153:config_log_put()) dropping config log lustre-barrier 00010000:00010000:1.0:1776178225.861401:0:6244:0:(ldlm_request.c:1279:__ldlm_pack_lock()) ### packing ns: MGC192.168.204.103@tcp lock: 000000006a2bf54b/0xa40083ab9bf321b8 lrc: 2/0,0 mode: --/CR res: [0x65727473756c:0x5:0x0].0x0 rrc: 3 type: PLN flags: 0x5809400000020 nid: local remote: 0xa40083ab9bf321bf expref: -99 pid: 20132 timeout: 0 lvb_type: 0 00010000:00010000:1.0:1776178225.861407:0:6244:0:(ldlm_request.c:1325:ldlm_cancel_pack()) 1 locks packed 00010000:00010000:1.0:1776178225.861410:0:6244:0:(ldlm_lockd.c:1903:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: MGC192.168.204.103@tcp lock: 000000006a2bf54b/0xa40083ab9bf321b8 lrc: 1/0,0 mode: --/CR res: [0x65727473756c:0x5:0x0].0x0 rrc: 3 type: PLN flags: 0x5809400000020 nid: local remote: 0xa40083ab9bf321bf expref: -99 pid: 20132 timeout: 0 lvb_type: 0 00010000:00010000:1.0:1776178225.861415:0:6244:0:(ldlm_lock.c:216:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: MGC192.168.204.103@tcp lock: 000000006a2bf54b/0xa40083ab9bf321b8 lrc: 0/0,0 mode: --/CR res: [0x65727473756c:0x5:0x0].0x0 rrc: 3 type: PLN flags: 0x5809400000020 nid: local remote: 0xa40083ab9bf321bf expref: -99 pid: 20132 timeout: 0 lvb_type: 0 00010000:00010000:1.0:1776178225.861422:0:6244:0:(ldlm_lockd.c:2792:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00010000:00010000:1.0:1776178225.861423:0:6244:0:(ldlm_lockd.c:2792:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00010000:00010000:1.0:1776178225.861424:0:6244:0:(ldlm_lockd.c:2792:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00010000:00010000:1.0:1776178225.861433:0:9225:0:(ldlm_lockd.c:2792:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:1.0:1776178225.861451:0:3353:0:(client.c:1733:ptlrpc_send_new_req()) Sending RPC req@000000001e11faca pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_00:ac61f839-0e8e-4cac-9c59-08bcfbbad330:3353:1862456998051648:0@lo:103:ldlm_bl_01.0 00000100:00100000:1.0:1776178225.861525:0:3353:0:(events.c:361:request_in_callback()) peer: 12345-0@lo (source: 12345-0@lo) 00000100:00100000:1.0:1776178225.861554:0:3353:0:(client.c:1733:ptlrpc_send_new_req()) Sending RPC req@00000000d7e84c97 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_00:ac61f839-0e8e-4cac-9c59-08bcfbbad330:3353:1862456998051904:0@lo:103:ldlm_bl_01.0 00000100:00100000:1.0:1776178225.861575:0:3353:0:(events.c:361:request_in_callback()) peer: 12345-0@lo (source: 12345-0@lo) 00000100:00100000:1.0:1776178225.861598:0:3353:0:(ptlrpcd.c:415:ptlrpcd_check()) transfer 1 async RPCs [1->0] 00000100:00100000:1.0:1776178225.861604:0:3353:0:(client.c:1733:ptlrpc_send_new_req()) Sending RPC req@00000000b30b9cdd pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_00:ac61f839-0e8e-4cac-9c59-08bcfbbad330:3353:1862456998051712:0@lo:103:ldlm_bl_01.0 00000100:00100000:1.0:1776178225.861624:0:3353:0:(events.c:361:request_in_callback()) peer: 12345-0@lo (source: 12345-0@lo) 00000100:00100000:1.0:1776178225.861654:0:3355:0:(client.c:1733:ptlrpc_send_new_req()) Sending RPC req@00000000eeeaf4ad pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_02:ac61f839-0e8e-4cac-9c59-08bcfbbad330:3355:1862456998051776:0@lo:103:ldlm_bl_01.0 00000100:00100000:1.0:1776178225.861678:0:3355:0:(events.c:361:request_in_callback()) peer: 12345-0@lo (source: 12345-0@lo) 00000100:00100000:1.0:1776178225.861695:0:3355:0:(ptlrpcd.c:415:ptlrpcd_check()) transfer 1 async RPCs [3->2] 00000100:00100000:1.0:1776178225.861700:0:3355:0:(client.c:1733:ptlrpc_send_new_req()) Sending RPC req@00000000854f8d52 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_02:ac61f839-0e8e-4cac-9c59-08bcfbbad330:3355:1862456998051840:0@lo:103:ldlm_bl_01.0 00000100:00100000:1.0:1776178225.861721:0:3355:0:(events.c:361:request_in_callback()) peer: 12345-0@lo (source: 12345-0@lo) 00000100:00100000:1.0:1776178225.861758:0:6242:0:(service.c:2152:ptlrpc_server_handle_req_in()) got req x1862456998051648 00000100:00100000:1.0:1776178225.861802:0:6242:0:(service.c:2152:ptlrpc_server_handle_req_in()) got req x1862456998051904 00000100:00100000:1.0:1776178225.861814:0:6242:0:(service.c:2152:ptlrpc_server_handle_req_in()) got req x1862456998051712 00000100:00100000:1.0:1776178225.861823:0:6242:0:(service.c:2152:ptlrpc_server_handle_req_in()) got req x1862456998051776 00000100:00100000:1.0:1776178225.861834:0:6242:0:(service.c:2152:ptlrpc_server_handle_req_in()) got req x1862456998051840 00000100:00100000:1.0:1776178225.861850:0:6242:0:(nrs_fifo.c:179:nrs_fifo_req_get()) NRS start fifo request from 12345-0@lo, seq: 49 00000100:00100000:1.0:1776178225.861857:0:6242:0:(service.c:2324:ptlrpc_server_handle_request()) Handling RPC req@00000000ba9bc15d pname:cluuid+ref:pid:xid:nid:opc:job ldlm_cn00_001:ac61f839-0e8e-4cac-9c59-08bcfbbad330+27:3353:x1862456998051648:12345-0@lo:103:ldlm_bl_01.0 00010000:00010000:1.0:1776178225.861875:0:6242:0:(ldlm_lockd.c:1720:ldlm_request_cancel()) ### server-side cancel handler START: 1 locks, starting at 0 00010000:00010000:1.0:1776178225.861893:0:6242:0:(ldlm_lock.c:216:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: MGS lock: 0000000067188295/0xa40083ab9bf3229f lrc: 0/0,0 mode: --/CR res: [0x65727473756c:0x0:0x0].0x0 rrc: 13 type: PLN flags: 0x44801000000000 nid: 0@lo remote: 0xa40083ab9bf32298 expref: 27 pid: 20139 timeout: 0 lvb_type: 0 00010000:00010000:1.0:1776178225.861908:0:6242:0:(ldlm_lockd.c:1782:ldlm_request_cancel()) ### server-side cancel handler END 00000100:00100000:1.0:1776178225.861941:0:6242:0:(service.c:2372:ptlrpc_server_handle_request()) Handled RPC req@00000000ba9bc15d pname:cluuid+ref:pid:xid:nid:opc:job ldlm_cn00_001:ac61f839-0e8e-4cac-9c59-08bcfbbad330+26:3353:x1862456998051648:12345-0@lo:103:ldlm_bl_01.0 Request processed in 84us (417us total) trans 0 rc 0/0 00000100:00100000:1.0:1776178225.861949:0:6242:0:(nrs_fifo.c:241:nrs_fifo_req_stop()) NRS stop fifo request from 12345-0@lo, seq: 49 00000100:00100000:1.0:1776178225.861966:0:6242:0:(nrs_fifo.c:179:nrs_fifo_req_get()) NRS start fifo request from 12345-0@lo, seq: 50 00000100:00100000:1.0:1776178225.861970:0:6242:0:(service.c:2324:ptlrpc_server_handle_request()) Handling RPC req@00000000f4172b4c pname:cluuid+ref:pid:xid:nid:opc:job ldlm_cn00_001:ac61f839-0e8e-4cac-9c59-08bcfbbad330+25:3353:x1862456998051904:12345-0@lo:103:ldlm_bl_01.0 00010000:00010000:1.0:1776178225.861978:0:6242:0:(ldlm_lockd.c:1720:ldlm_request_cancel()) ### server-side cancel handler START: 1 locks, starting at 0 00010000:00010000:1.0:1776178225.861986:0:6242:0:(ldlm_lock.c:216:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: MGS lock: 000000006423c6b4/0xa40083ab9bf321bf lrc: 0/0,0 mode: --/CR res: [0x65727473756c:0x5:0x0].0x0 rrc: 4 type: PLN flags: 0x44801000000000 nid: 0@lo remote: 0xa40083ab9bf321b8 expref: 25 pid: 20140 timeout: 0 lvb_type: 0 00010000:00010000:1.0:1776178225.861994:0:6242:0:(ldlm_lockd.c:1782:ldlm_request_cancel()) ### server-side cancel handler END 00000100:00100000:1.0:1776178225.862014:0:6242:0:(service.c:2372:ptlrpc_server_handle_request()) Handled RPC req@00000000f4172b4c pname:cluuid+ref:pid:xid:nid:opc:job ldlm_cn00_001:ac61f839-0e8e-4cac-9c59-08bcfbbad330+24:3353:x1862456998051904:12345-0@lo:103:ldlm_bl_01.0 Request processed in 43us (439us total) trans 0 rc 0/0 00000100:00100000:1.0:1776178225.862019:0:6242:0:(nrs_fifo.c:241:nrs_fifo_req_stop()) NRS stop fifo request from 12345-0@lo, seq: 50 00000100:00100000:1.0:1776178225.862030:0:6242:0:(nrs_fifo.c:179:nrs_fifo_req_get()) NRS start fifo request from 12345-0@lo, seq: 51 00000100:00100000:1.0:1776178225.862034:0:6242:0:(service.c:2324:ptlrpc_server_handle_request()) Handling RPC req@00000000b7440871 pname:cluuid+ref:pid:xid:nid:opc:job ldlm_cn00_001:ac61f839-0e8e-4cac-9c59-08bcfbbad330+23:3353:x1862456998051712:12345-0@lo:103:ldlm_bl_01.0 00010000:00010000:1.0:1776178225.862041:0:6242:0:(ldlm_lockd.c:1720:ldlm_request_cancel()) ### server-side cancel handler START: 1 locks, starting at 0 00010000:00010000:1.0:1776178225.862048:0:6242:0:(ldlm_lock.c:216:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: MGS lock: 0000000030c77f58/0xa40083ab9bf321cd lrc: 0/0,0 mode: --/CR res: [0x65727473756c:0x0:0x0].0x0 rrc: 12 type: PLN flags: 0x44801000000000 nid: 0@lo remote: 0xa40083ab9bf321c6 expref: 23 pid: 20139 timeout: 0 lvb_type: 0 00010000:00010000:1.0:1776178225.862056:0:6242:0:(ldlm_lockd.c:1782:ldlm_request_cancel()) ### server-side cancel handler END 00010000:00010000:2.0:1776178225.862242:0:9224:0:(ldlm_lockd.c:2792:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00010000:00010000:2.0:1776178225.862257:0:6245:0:(ldlm_lockd.c:2792:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:2.0:1776178225.863330:0:6241:0:(nrs_fifo.c:179:nrs_fifo_req_get()) NRS start fifo request from 12345-0@lo, seq: 52 00000100:00100000:2.0:1776178225.863338:0:6241:0:(service.c:2324:ptlrpc_server_handle_request()) Handling RPC req@0000000097f5c42c pname:cluuid+ref:pid:xid:nid:opc:job ldlm_cn00_000:ac61f839-0e8e-4cac-9c59-08bcfbbad330+22:3355:x1862456998051776:12345-0@lo:103:ldlm_bl_01.0 00010000:00010000:2.0:1776178225.863355:0:6241:0:(ldlm_lockd.c:1720:ldlm_request_cancel()) ### server-side cancel handler START: 1 locks, starting at 0 00010000:00010000:2.0:1776178225.863373:0:6241:0:(ldlm_lock.c:216:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: MGS lock: 000000000674b12b/0xa40083ab9bf323fd lrc: 0/0,0 mode: --/CR res: [0x65727473756c:0x2:0x0].0x0 rrc: 5 type: PLN flags: 0x44801000000000 nid: 0@lo remote: 0xa40083ab9bf323f6 expref: 22 pid: 20137 timeout: 0 lvb_type: 0 00010000:00010000:2.0:1776178225.863391:0:6241:0:(ldlm_lockd.c:1782:ldlm_request_cancel()) ### server-side cancel handler END 00000100:00100000:2.0:1776178225.863444:0:6241:0:(service.c:2372:ptlrpc_server_handle_request()) Handled RPC req@0000000097f5c42c pname:cluuid+ref:pid:xid:nid:opc:job ldlm_cn00_000:ac61f839-0e8e-4cac-9c59-08bcfbbad330+21:3355:x1862456998051776:12345-0@lo:103:ldlm_bl_01.0 Request processed in 107us (1766us total) trans 0 rc 0/0 00000100:00100000:2.0:1776178225.863450:0:6241:0:(nrs_fifo.c:241:nrs_fifo_req_stop()) NRS stop fifo request from 12345-0@lo, seq: 52 00000100:00100000:2.0:1776178225.863470:0:6241:0:(nrs_fifo.c:179:nrs_fifo_req_get()) NRS start fifo request from 12345-0@lo, seq: 53 00000100:00100000:2.0:1776178225.863474:0:6241:0:(service.c:2324:ptlrpc_server_handle_request()) Handling RPC req@00000000f8e38e41 pname:cluuid+ref:pid:xid:nid:opc:job ldlm_cn00_000:ac61f839-0e8e-4cac-9c59-08bcfbbad330+20:3355:x1862456998051840:12345-0@lo:103:ldlm_bl_01.0 00010000:00010000:2.0:1776178225.863480:0:6241:0:(ldlm_lockd.c:1720:ldlm_request_cancel()) ### server-side cancel handler START: 1 locks, starting at 0 00010000:00010000:2.0:1776178225.863487:0:6241:0:(ldlm_lock.c:216:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: MGS lock: 00000000eca4f621/0xa40083ab9bf32291 lrc: 0/0,0 mode: --/CR res: [0x736d61726170:0x3:0x0].0x0 rrc: 7 type: PLN flags: 0x44801000000000 nid: 0@lo remote: 0xa40083ab9bf3228a expref: 20 pid: 20139 timeout: 0 lvb_type: 0 00010000:00010000:2.0:1776178225.863495:0:6241:0:(ldlm_lockd.c:1782:ldlm_request_cancel()) ### server-side cancel handler END 00000100:00100000:2.0:1776178225.863509:0:6241:0:(service.c:2372:ptlrpc_server_handle_request()) Handled RPC req@00000000f8e38e41 pname:cluuid+ref:pid:xid:nid:opc:job ldlm_cn00_000:ac61f839-0e8e-4cac-9c59-08bcfbbad330+19:3355:x1862456998051840:12345-0@lo:103:ldlm_bl_01.0 Request processed in 35us (1789us total) trans 0 rc 0/0 00000100:00100000:2.0:1776178225.863514:0:6241:0:(nrs_fifo.c:241:nrs_fifo_req_stop()) NRS stop fifo request from 12345-0@lo, seq: 53 00000100:00100000:1.0:1776178225.866193:0:6242:0:(service.c:2372:ptlrpc_server_handle_request()) Handled RPC req@00000000b7440871 pname:cluuid+ref:pid:xid:nid:opc:job ldlm_cn00_001:ac61f839-0e8e-4cac-9c59-08bcfbbad330+18:3353:x1862456998051712:12345-0@lo:103:ldlm_bl_01.0 Request processed in 4146us (4557us total) trans 0 rc 0/0 00000100:00100000:1.0:1776178225.866213:0:6242:0:(nrs_fifo.c:241:nrs_fifo_req_stop()) NRS stop fifo request from 12345-0@lo, seq: 51 00000100:00100000:1.0:1776178225.866390:0:3353:0:(client.c:2215:ptlrpc_check_set()) Completed RPC req@00000000b30b9cdd pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_00:ac61f839-0e8e-4cac-9c59-08bcfbbad330:3353:1862456998051712:0@lo:103:ldlm_bl_01.0 00000100:00100000:1.0:1776178225.866403:0:3353:0:(client.c:2215:ptlrpc_check_set()) Completed RPC req@000000001e11faca pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_00:ac61f839-0e8e-4cac-9c59-08bcfbbad330:3353:1862456998051648:0@lo:103:ldlm_bl_01.0 00000100:00100000:1.0:1776178225.866412:0:3353:0:(client.c:2215:ptlrpc_check_set()) Completed RPC req@00000000d7e84c97 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_00:ac61f839-0e8e-4cac-9c59-08bcfbbad330:3353:1862456998051904:0@lo:103:ldlm_bl_01.0 00000100:00100000:1.0:1776178225.866457:0:3355:0:(client.c:2215:ptlrpc_check_set()) Completed RPC req@00000000854f8d52 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_02:ac61f839-0e8e-4cac-9c59-08bcfbbad330:3355:1862456998051840:0@lo:103:ldlm_bl_01.0 00000100:00100000:1.0:1776178225.866465:0:3355:0:(client.c:2215:ptlrpc_check_set()) Completed RPC req@00000000eeeaf4ad pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_02:ac61f839-0e8e-4cac-9c59-08bcfbbad330:3355:1862456998051776:0@lo:103:ldlm_bl_01.0 00000004:00000080:0.0:1776178225.869716:0:22200:0:(mdt_handler.c:7411:mdt_iocontrol()) handling ioctl cmd 0x40086672 00000001:00080000:0.0:1776178225.872841:0:20134:0:(tgt_lastrcvd.c:913:tgt_cb_last_committed()) lustre-MDT0000: transno 12884901981 is committed 00010000:00010000:0.0:1776178225.872884:0:3340:0:(ldlm_lock.c:820:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(COS) ns: mdt-lustre-MDT0000_UUID lock: 0000000015ae4ee6/0xa40083ab9bf53a0a lrc: 3/0,1 mode: COS/COS res: [0x2000013a1:0xa:0x0].0x0 bits 0x13/0x0 rrc: 2 type: IBT gid 0 flags: 0x50210000000000 nid: local remote: 0x0 expref: -99 pid: 6257 timeout: 0 lvb_type: 0 00010000:00010000:0.0:1776178225.872900:0:3340:0:(ldlm_lock.c:879:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: mdt-lustre-MDT0000_UUID lock: 0000000015ae4ee6/0xa40083ab9bf53a0a lrc: 2/0,0 mode: COS/COS res: [0x2000013a1:0xa:0x0].0x0 bits 0x13/0x0 rrc: 2 type: IBT gid 0 flags: 0x50210400000000 nid: local remote: 0x0 expref: -99 pid: 6257 timeout: 0 lvb_type: 0 00010000:00010000:0.0:1776178225.872909:0:3340:0:(ldlm_lockd.c:1879:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: mdt-lustre-MDT0000_UUID lock: 0000000015ae4ee6/0xa40083ab9bf53a0a lrc: 3/0,0 mode: COS/COS res: [0x2000013a1:0xa:0x0].0x0 bits 0x13/0x0 rrc: 2 type: IBT gid 0 flags: 0x40210400000000 nid: local remote: 0x0 expref: -99 pid: 6257 timeout: 0 lvb_type: 0 00010000:00010000:0.0:1776178225.872916:0:3340:0:(ldlm_lockd.c:1893:ldlm_handle_bl_callback()) Lock 0000000015ae4ee6 already unused, calling callback (0000000079e78348) 00010000:00010000:0.0:1776178225.872921:0:3340:0:(ldlm_request.c:364:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: mdt-lustre-MDT0000_UUID lock: 0000000015ae4ee6/0xa40083ab9bf53a0a lrc: 3/0,0 mode: COS/COS res: [0x2000013a1:0xa:0x0].0x0 bits 0x13/0x0 rrc: 2 type: IBT gid 0 flags: 0x40210400000000 nid: local remote: 0x0 expref: -99 pid: 6257 timeout: 0 lvb_type: 0 00010000:00010000:0.0:1776178225.872929:0:3340:0:(ldlm_request.c:1257:ldlm_cli_cancel_local()) ### server-side local cancel ns: mdt-lustre-MDT0000_UUID lock: 0000000015ae4ee6/0xa40083ab9bf53a0a lrc: 4/0,0 mode: COS/COS res: [0x2000013a1:0xa:0x0].0x0 bits 0x13/0x0 rrc: 2 type: IBT gid 0 flags: 0x40218400000000 nid: local remote: 0x0 expref: -99 pid: 6257 timeout: 0 lvb_type: 0 00010000:00010000:0.0:1776178225.872949:0:3340:0:(ldlm_inodebits.c:94:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x2000013a1:0xa:0x0].0x0 (00000000c9b683f9) 00010000:00010000:0.0:1776178225.872952:0:3340:0:(ldlm_inodebits.c:96:ldlm_reprocess_inodebits_queue()) Hint 13 00010000:00010000:0.0:1776178225.872958:0:3340:0:(ldlm_lockd.c:1903:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: mdt-lustre-MDT0000_UUID lock: 0000000015ae4ee6/0xa40083ab9bf53a0a lrc: 2/0,0 mode: --/COS res: [0x2000013a1:0xa:0x0].0x0 bits 0x13/0x0 rrc: 2 type: IBT gid 0 flags: 0x44a19400000000 nid: local remote: 0x0 expref: -99 pid: 6257 timeout: 0 lvb_type: 0 00010000:00010000:0.0:1776178225.872964:0:3340:0:(ldlm_lock.c:216:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: mdt-lustre-MDT0000_UUID lock: 0000000015ae4ee6/0xa40083ab9bf53a0a lrc: 0/0,0 mode: --/COS res: [0x2000013a1:0xa:0x0].0x0 bits 0x13/0x0 rrc: 2 type: IBT gid 0 flags: 0x44a19400000000 nid: local remote: 0x0 expref: -99 pid: 6257 timeout: 0 lvb_type: 0 00010000:00010000:0.0:1776178225.872989:0:3340:0:(ldlm_lock.c:820:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(COS) ns: mdt-lustre-MDT0000_UUID lock: 0000000039ef0a83/0xa40083ab9bf53a73 lrc: 3/0,1 mode: COS/COS res: [0x2000013a1:0xb:0x0].0x0 bits 0x13/0x0 rrc: 2 type: IBT gid 0 flags: 0x50210000000000 nid: local remote: 0x0 expref: -99 pid: 6256 timeout: 0 lvb_type: 0 00010000:00010000:0.0:1776178225.872996:0:3340:0:(ldlm_lock.c:879:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: mdt-lustre-MDT0000_UUID lock: 0000000039ef0a83/0xa40083ab9bf53a73 lrc: 2/0,0 mode: COS/COS res: [0x2000013a1:0xb:0x0].0x0 bits 0x13/0x0 rrc: 2 type: IBT gid 0 flags: 0x50210400000000 nid: local remote: 0x0 expref: -99 pid: 6256 timeout: 0 lvb_type: 0 00010000:00010000:0.0:1776178225.873001:0:3340:0:(ldlm_lockd.c:1879:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: mdt-lustre-MDT0000_UUID lock: 0000000039ef0a83/0xa40083ab9bf53a73 lrc: 3/0,0 mode: COS/COS res: [0x2000013a1:0xb:0x0].0x0 bits 0x13/0x0 rrc: 2 type: IBT gid 0 flags: 0x40210400000000 nid: local remote: 0x0 expref: -99 pid: 6256 timeout: 0 lvb_type: 0 00010000:00010000:0.0:1776178225.873006:0:3340:0:(ldlm_lockd.c:1893:ldlm_handle_bl_callback()) Lock 0000000039ef0a83 already unused, calling callback (0000000079e78348) 00010000:00010000:0.0:1776178225.873007:0:3340:0:(ldlm_request.c:364:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: mdt-lustre-MDT0000_UUID lock: 0000000039ef0a83/0xa40083ab9bf53a73 lrc: 3/0,0 mode: COS/COS res: [0x2000013a1:0xb:0x0].0x0 bits 0x13/0x0 rrc: 2 type: IBT gid 0 flags: 0x40210400000000 nid: local remote: 0x0 expref: -99 pid: 6256 timeout: 0 lvb_type: 0 00010000:00010000:0.0:1776178225.873013:0:3340:0:(ldlm_request.c:1257:ldlm_cli_cancel_local()) ### server-side local cancel ns: mdt-lustre-MDT0000_UUID lock: 0000000039ef0a83/0xa40083ab9bf53a73 lrc: 4/0,0 mode: COS/COS res: [0x2000013a1:0xb:0x0].0x0 bits 0x13/0x0 rrc: 2 type: IBT gid 0 flags: 0x40218400000000 nid: local remote: 0x0 expref: -99 pid: 6256 timeout: 0 lvb_type: 0 00010000:00010000:0.0:1776178225.873020:0:3340:0:(ldlm_inodebits.c:94:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x2000013a1:0xb:0x0].0x0 (000000002f3f796c) 00010000:00010000:0.0:1776178225.873023:0:3340:0:(ldlm_inodebits.c:96:ldlm_reprocess_inodebits_queue()) Hint 13 00010000:00010000:0.0:1776178225.873025:0:3340:0:(ldlm_lockd.c:1903:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: mdt-lustre-MDT0000_UUID lock: 0000000039ef0a83/0xa40083ab9bf53a73 lrc: 2/0,0 mode: --/COS res: [0x2000013a1:0xb:0x0].0x0 bits 0x13/0x0 rrc: 2 type: IBT gid 0 flags: 0x44a19400000000 nid: local remote: 0x0 expref: -99 pid: 6256 timeout: 0 lvb_type: 0 00010000:00010000:0.0:1776178225.873031:0:3340:0:(ldlm_lock.c:216:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: mdt-lustre-MDT0000_UUID lock: 0000000039ef0a83/0xa40083ab9bf53a73 lrc: 0/0,0 mode: --/COS res: [0x2000013a1:0xb:0x0].0x0 bits 0x13/0x0 rrc: 2 type: IBT gid 0 flags: 0x44a19400000000 nid: local remote: 0x0 expref: -99 pid: 6256 timeout: 0 lvb_type: 0 00010000:00010000:0.0:1776178225.873046:0:3340:0:(ldlm_lock.c:820:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(COS) ns: mdt-lustre-MDT0000_UUID lock: 00000000782e253e/0xa40083ab9bf53a8f lrc: 3/0,1 mode: COS/COS res: [0x2000013a1:0xd:0x0].0x0 bits 0x13/0x0 rrc: 2 type: IBT gid 0 flags: 0x50210000000000 nid: local remote: 0x0 expref: -99 pid: 6255 timeout: 0 lvb_type: 0 00010000:00010000:0.0:1776178225.873052:0:3340:0:(ldlm_lock.c:879:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: mdt-lustre-MDT0000_UUID lock: 00000000782e253e/0xa40083ab9bf53a8f lrc: 2/0,0 mode: COS/COS res: [0x2000013a1:0xd:0x0].0x0 bits 0x13/0x0 rrc: 2 type: IBT gid 0 flags: 0x50210400000000 nid: local remote: 0x0 expref: -99 pid: 6255 timeout: 0 lvb_type: 0 00010000:00010000:0.0:1776178225.873082:0:3340:0:(ldlm_lockd.c:1879:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: mdt-lustre-MDT0000_UUID lock: 00000000782e253e/0xa40083ab9bf53a8f lrc: 3/0,0 mode: COS/COS res: [0x2000013a1:0xd:0x0].0x0 bits 0x13/0x0 rrc: 2 type: IBT gid 0 flags: 0x40210400000000 nid: local remote: 0x0 expref: -99 pid: 6255 timeout: 0 lvb_type: 0 00010000:00010000:0.0:1776178225.873088:0:3340:0:(ldlm_lockd.c:1893:ldlm_handle_bl_callback()) Lock 00000000782e253e already unused, calling callback (0000000079e78348) 00010000:00010000:0.0:1776178225.873090:0:3340:0:(ldlm_request.c:364:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: mdt-lustre-MDT0000_UUID lock: 00000000782e253e/0xa40083ab9bf53a8f lrc: 3/0,0 mode: COS/COS res: [0x2000013a1:0xd:0x0].0x0 bits 0x13/0x0 rrc: 2 type: IBT gid 0 flags: 0x40210400000000 nid: local remote: 0x0 expref: -99 pid: 6255 timeout: 0 lvb_type: 0 00010000:00010000:0.0:1776178225.873095:0:3340:0:(ldlm_request.c:1257:ldlm_cli_cancel_local()) ### server-side local cancel ns: mdt-lustre-MDT0000_UUID lock: 00000000782e253e/0xa40083ab9bf53a8f lrc: 4/0,0 mode: COS/COS res: [0x2000013a1:0xd:0x0].0x0 bits 0x13/0x0 rrc: 2 type: IBT gid 0 flags: 0x40218400000000 nid: local remote: 0x0 expref: -99 pid: 6255 timeout: 0 lvb_type: 0 00010000:00010000:0.0:1776178225.873103:0:3340:0:(ldlm_inodebits.c:94:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x2000013a1:0xd:0x0].0x0 (0000000078a0d18e) 00010000:00010000:0.0:1776178225.873104:0:3340:0:(ldlm_inodebits.c:96:ldlm_reprocess_inodebits_queue()) Hint 13 00010000:00010000:0.0:1776178225.873107:0:3340:0:(ldlm_lockd.c:1903:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: mdt-lustre-MDT0000_UUID lock: 00000000782e253e/0xa40083ab9bf53a8f lrc: 2/0,0 mode: --/COS res: [0x2000013a1:0xd:0x0].0x0 bits 0x13/0x0 rrc: 2 type: IBT gid 0 flags: 0x44a19400000000 nid: local remote: 0x0 expref: -99 pid: 6255 timeout: 0 lvb_type: 0 00010000:00010000:0.0:1776178225.873112:0:3340:0:(ldlm_lock.c:216:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: mdt-lustre-MDT0000_UUID lock: 00000000782e253e/0xa40083ab9bf53a8f lrc: 0/0,0 mode: --/COS res: [0x2000013a1:0xd:0x0].0x0 bits 0x13/0x0 rrc: 2 type: IBT gid 0 flags: 0x44a19400000000 nid: local remote: 0x0 expref: -99 pid: 6255 timeout: 0 lvb_type: 0 00010000:00010000:0.0:1776178225.873132:0:3340:0:(ldlm_lock.c:820:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(COS) ns: mdt-lustre-MDT0000_UUID lock: 00000000c3ab77cd/0xa40083ab9bf53a88 lrc: 3/0,1 mode: COS/COS res: [0x2000013a1:0xc:0x0].0x0 bits 0x13/0x0 rrc: 2 type: IBT gid 0 flags: 0x50210000000000 nid: local remote: 0x0 expref: -99 pid: 8250 timeout: 0 lvb_type: 0 00010000:00010000:0.0:1776178225.873140:0:3340:0:(ldlm_lock.c:879:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: mdt-lustre-MDT0000_UUID lock: 00000000c3ab77cd/0xa40083ab9bf53a88 lrc: 2/0,0 mode: COS/COS res: [0x2000013a1:0xc:0x0].0x0 bits 0x13/0x0 rrc: 2 type: IBT gid 0 flags: 0x50210400000000 nid: local remote: 0x0 expref: -99 pid: 8250 timeout: 0 lvb_type: 0 00010000:00010000:0.0:1776178225.873146:0:3340:0:(ldlm_lockd.c:1879:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: mdt-lustre-MDT0000_UUID lock: 00000000c3ab77cd/0xa40083ab9bf53a88 lrc: 3/0,0 mode: COS/COS res: [0x2000013a1:0xc:0x0].0x0 bits 0x13/0x0 rrc: 2 type: IBT gid 0 flags: 0x40210400000000 nid: local remote: 0x0 expref: -99 pid: 8250 timeout: 0 lvb_type: 0 00010000:00010000:0.0:1776178225.873151:0:3340:0:(ldlm_lockd.c:1893:ldlm_handle_bl_callback()) Lock 00000000c3ab77cd already unused, calling callback (0000000079e78348) 00010000:00010000:0.0:1776178225.873153:0:3340:0:(ldlm_request.c:364:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: mdt-lustre-MDT0000_UUID lock: 00000000c3ab77cd/0xa40083ab9bf53a88 lrc: 3/0,0 mode: COS/COS res: [0x2000013a1:0xc:0x0].0x0 bits 0x13/0x0 rrc: 2 type: IBT gid 0 flags: 0x40210400000000 nid: local remote: 0x0 expref: -99 pid: 8250 timeout: 0 lvb_type: 0 00010000:00010000:0.0:1776178225.873159:0:3340:0:(ldlm_request.c:1257:ldlm_cli_cancel_local()) ### server-side local cancel ns: mdt-lustre-MDT0000_UUID lock: 00000000c3ab77cd/0xa40083ab9bf53a88 lrc: 4/0,0 mode: COS/COS res: [0x2000013a1:0xc:0x0].0x0 bits 0x13/0x0 rrc: 2 type: IBT gid 0 flags: 0x40218400000000 nid: local remote: 0x0 expref: -99 pid: 8250 timeout: 0 lvb_type: 0 00010000:00010000:0.0:1776178225.873167:0:3340:0:(ldlm_inodebits.c:94:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x2000013a1:0xc:0x0].0x0 (00000000099618da) 00010000:00010000:0.0:1776178225.873169:0:3340:0:(ldlm_inodebits.c:96:ldlm_reprocess_inodebits_queue()) Hint 13 00010000:00010000:0.0:1776178225.873171:0:3340:0:(ldlm_lockd.c:1903:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: mdt-lustre-MDT0000_UUID lock: 00000000c3ab77cd/0xa40083ab9bf53a88 lrc: 2/0,0 mode: --/COS res: [0x2000013a1:0xc:0x0].0x0 bits 0x13/0x0 rrc: 2 type: IBT gid 0 flags: 0x44a19400000000 nid: local remote: 0x0 expref: -99 pid: 8250 timeout: 0 lvb_type: 0 00010000:00010000:0.0:1776178225.873177:0:3340:0:(ldlm_lock.c:216:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: mdt-lustre-MDT0000_UUID lock: 00000000c3ab77cd/0xa40083ab9bf53a88 lrc: 0/0,0 mode: --/COS res: [0x2000013a1:0xc:0x0].0x0 bits 0x13/0x0 rrc: 2 type: IBT gid 0 flags: 0x44a19400000000 nid: local remote: 0x0 expref: -99 pid: 8250 timeout: 0 lvb_type: 0 00010000:00010000:0.0:1776178225.873191:0:3340:0:(ldlm_lock.c:820:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(COS) ns: mdt-lustre-MDT0000_UUID lock: 0000000026fd08a4/0xa40083ab9bf53a9d lrc: 3/0,1 mode: COS/COS res: [0x2000013a1:0xf:0x0].0x0 bits 0x13/0x0 rrc: 2 type: IBT gid 0 flags: 0x50210000000000 nid: local remote: 0x0 expref: -99 pid: 8250 timeout: 0 lvb_type: 0 00010000:00010000:0.0:1776178225.873197:0:3340:0:(ldlm_lock.c:879:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: mdt-lustre-MDT0000_UUID lock: 0000000026fd08a4/0xa40083ab9bf53a9d lrc: 2/0,0 mode: COS/COS res: [0x2000013a1:0xf:0x0].0x0 bits 0x13/0x0 rrc: 2 type: IBT gid 0 flags: 0x50210400000000 nid: local remote: 0x0 expref: -99 pid: 8250 timeout: 0 lvb_type: 0 00010000:00010000:0.0:1776178225.873209:0:3340:0:(ldlm_lockd.c:1879:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: mdt-lustre-MDT0000_UUID lock: 0000000026fd08a4/0xa40083ab9bf53a9d lrc: 3/0,0 mode: COS/COS res: [0x2000013a1:0xf:0x0].0x0 bits 0x13/0x0 rrc: 2 type: IBT gid 0 flags: 0x40210400000000 nid: local remote: 0x0 expref: -99 pid: 8250 timeout: 0 lvb_type: 0 00010000:00010000:0.0:1776178225.873215:0:3340:0:(ldlm_lockd.c:1893:ldlm_handle_bl_callback()) Lock 0000000026fd08a4 already unused, calling callback (0000000079e78348) 00010000:00010000:0.0:1776178225.873218:0:3340:0:(ldlm_request.c:364:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: mdt-lustre-MDT0000_UUID lock: 0000000026fd08a4/0xa40083ab9bf53a9d lrc: 3/0,0 mode: COS/COS res: [0x2000013a1:0xf:0x0].0x0 bits 0x13/0x0 rrc: 2 type: IBT gid 0 flags: 0x40210400000000 nid: local remote: 0x0 expref: -99 pid: 8250 timeout: 0 lvb_type: 0 00010000:00010000:0.0:1776178225.873224:0:3340:0:(ldlm_request.c:1257:ldlm_cli_cancel_local()) ### server-side local cancel ns: mdt-lustre-MDT0000_UUID lock: 0000000026fd08a4/0xa40083ab9bf53a9d lrc: 4/0,0 mode: COS/COS res: [0x2000013a1:0xf:0x0].0x0 bits 0x13/0x0 rrc: 2 type: IBT gid 0 flags: 0x40218400000000 nid: local remote: 0x0 expref: -99 pid: 8250 timeout: 0 lvb_type: 0 00010000:00010000:0.0:1776178225.873232:0:3340:0:(ldlm_inodebits.c:94:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x2000013a1:0xf:0x0].0x0 (000000007b7d95c1) 00010000:00010000:0.0:1776178225.873234:0:3340:0:(ldlm_inodebits.c:96:ldlm_reprocess_inodebits_queue()) Hint 13 00010000:00010000:0.0:1776178225.873236:0:3340:0:(ldlm_lockd.c:1903:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: mdt-lustre-MDT0000_UUID lock: 0000000026fd08a4/0xa40083ab9bf53a9d lrc: 2/0,0 mode: --/COS res: [0x2000013a1:0xf:0x0].0x0 bits 0x13/0x0 rrc: 2 type: IBT gid 0 flags: 0x44a19400000000 nid: local remote: 0x0 expref: -99 pid: 8250 timeout: 0 lvb_type: 0 00010000:00010000:0.0:1776178225.873241:0:3340:0:(ldlm_lock.c:216:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: mdt-lustre-MDT0000_UUID lock: 0000000026fd08a4/0xa40083ab9bf53a9d lrc: 0/0,0 mode: --/COS res: [0x2000013a1:0xf:0x0].0x0 bits 0x13/0x0 rrc: 2 type: IBT gid 0 flags: 0x44a19400000000 nid: local remote: 0x0 expref: -99 pid: 8250 timeout: 0 lvb_type: 0 00010000:00010000:0.0:1776178225.873256:0:3340:0:(ldlm_lock.c:820:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(COS) ns: mdt-lustre-MDT0000_UUID lock: 0000000096c1e297/0xa40083ab9bf53aa4 lrc: 3/0,1 mode: COS/COS res: [0x2000013a1:0x10:0x0].0x0 bits 0x13/0x0 rrc: 2 type: IBT gid 0 flags: 0x50210000000000 nid: local remote: 0x0 expref: -99 pid: 8250 timeout: 0 lvb_type: 0 00010000:00010000:0.0:1776178225.873262:0:3340:0:(ldlm_lock.c:879:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: mdt-lustre-MDT0000_UUID lock: 0000000096c1e297/0xa40083ab9bf53aa4 lrc: 2/0,0 mode: COS/COS res: [0x2000013a1:0x10:0x0].0x0 bits 0x13/0x0 rrc: 2 type: IBT gid 0 flags: 0x50210400000000 nid: local remote: 0x0 expref: -99 pid: 8250 timeout: 0 lvb_type: 0 00010000:00010000:0.0:1776178225.873268:0:3340:0:(ldlm_lockd.c:1879:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: mdt-lustre-MDT0000_UUID lock: 0000000096c1e297/0xa40083ab9bf53aa4 lrc: 3/0,0 mode: COS/COS res: [0x2000013a1:0x10:0x0].0x0 bits 0x13/0x0 rrc: 2 type: IBT gid 0 flags: 0x40210400000000 nid: local remote: 0x0 expref: -99 pid: 8250 timeout: 0 lvb_type: 0 00010000:00010000:0.0:1776178225.873273:0:3340:0:(ldlm_lockd.c:1893:ldlm_handle_bl_callback()) Lock 0000000096c1e297 already unused, calling callback (0000000079e78348) 00010000:00010000:0.0:1776178225.873275:0:3340:0:(ldlm_request.c:364:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: mdt-lustre-MDT0000_UUID lock: 0000000096c1e297/0xa40083ab9bf53aa4 lrc: 3/0,0 mode: COS/COS res: [0x2000013a1:0x10:0x0].0x0 bits 0x13/0x0 rrc: 2 type: IBT gid 0 flags: 0x40210400000000 nid: local remote: 0x0 expref: -99 pid: 8250 timeout: 0 lvb_type: 0 00010000:00010000:0.0:1776178225.873282:0:3340:0:(ldlm_request.c:1257:ldlm_cli_cancel_local()) ### server-side local cancel ns: mdt-lustre-MDT0000_UUID lock: 0000000096c1e297/0xa40083ab9bf53aa4 lrc: 4/0,0 mode: COS/COS res: [0x2000013a1:0x10:0x0].0x0 bits 0x13/0x0 rrc: 2 type: IBT gid 0 flags: 0x40218400000000 nid: local remote: 0x0 expref: -99 pid: 8250 timeout: 0 lvb_type: 0 00010000:00010000:0.0:1776178225.873301:0:3340:0:(ldlm_inodebits.c:94:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x2000013a1:0x10:0x0].0x0 (0000000056a46617) 00010000:00010000:0.0:1776178225.873305:0:3340:0:(ldlm_inodebits.c:96:ldlm_reprocess_inodebits_queue()) Hint 13 00010000:00010000:0.0:1776178225.873307:0:3340:0:(ldlm_lockd.c:1903:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: mdt-lustre-MDT0000_UUID lock: 0000000096c1e297/0xa40083ab9bf53aa4 lrc: 2/0,0 mode: --/COS res: [0x2000013a1:0x10:0x0].0x0 bits 0x13/0x0 rrc: 2 type: IBT gid 0 flags: 0x44a19400000000 nid: local remote: 0x0 expref: -99 pid: 8250 timeout: 0 lvb_type: 0 00010000:00010000:0.0:1776178225.873312:0:3340:0:(ldlm_lock.c:216:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: mdt-lustre-MDT0000_UUID lock: 0000000096c1e297/0xa40083ab9bf53aa4 lrc: 0/0,0 mode: --/COS res: [0x2000013a1:0x10:0x0].0x0 bits 0x13/0x0 rrc: 2 type: IBT gid 0 flags: 0x44a19400000000 nid: local remote: 0x0 expref: -99 pid: 8250 timeout: 0 lvb_type: 0 00010000:00010000:0.0:1776178225.873328:0:3340:0:(ldlm_lock.c:820:ldlm_lock_decref_internal_nolock()) ### ldlm_lock_decref(COS) ns: mdt-lustre-MDT0000_UUID lock: 000000000396420a/0xa40083ab9bf53aab lrc: 3/0,1 mode: COS/COS res: [0x2000013a1:0xe:0x0].0x0 bits 0x13/0x0 rrc: 2 type: IBT gid 0 flags: 0x50210000000000 nid: local remote: 0x0 expref: -99 pid: 8250 timeout: 0 lvb_type: 0 00010000:00010000:0.0:1776178225.873334:0:3340:0:(ldlm_lock.c:879:ldlm_lock_decref_internal()) ### final decref done on CBPENDING lock ns: mdt-lustre-MDT0000_UUID lock: 000000000396420a/0xa40083ab9bf53aab lrc: 2/0,0 mode: COS/COS res: [0x2000013a1:0xe:0x0].0x0 bits 0x13/0x0 rrc: 2 type: IBT gid 0 flags: 0x50210400000000 nid: local remote: 0x0 expref: -99 pid: 8250 timeout: 0 lvb_type: 0 00010000:00010000:0.0:1776178225.873340:0:3340:0:(ldlm_lockd.c:1879:ldlm_handle_bl_callback()) ### client blocking AST callback handler ns: mdt-lustre-MDT0000_UUID lock: 000000000396420a/0xa40083ab9bf53aab lrc: 3/0,0 mode: COS/COS res: [0x2000013a1:0xe:0x0].0x0 bits 0x13/0x0 rrc: 2 type: IBT gid 0 flags: 0x40210400000000 nid: local remote: 0x0 expref: -99 pid: 8250 timeout: 0 lvb_type: 0 00010000:00010000:0.0:1776178225.873345:0:3340:0:(ldlm_lockd.c:1893:ldlm_handle_bl_callback()) Lock 000000000396420a already unused, calling callback (0000000079e78348) 00010000:00010000:0.0:1776178225.873358:0:3340:0:(ldlm_request.c:364:ldlm_blocking_ast_nocheck()) ### already unused, calling ldlm_cli_cancel ns: mdt-lustre-MDT0000_UUID lock: 000000000396420a/0xa40083ab9bf53aab lrc: 3/0,0 mode: COS/COS res: [0x2000013a1:0xe:0x0].0x0 bits 0x13/0x0 rrc: 2 type: IBT gid 0 flags: 0x40210400000000 nid: local remote: 0x0 expref: -99 pid: 8250 timeout: 0 lvb_type: 0 00010000:00010000:0.0:1776178225.873364:0:3340:0:(ldlm_request.c:1257:ldlm_cli_cancel_local()) ### server-side local cancel ns: mdt-lustre-MDT0000_UUID lock: 000000000396420a/0xa40083ab9bf53aab lrc: 4/0,0 mode: COS/COS res: [0x2000013a1:0xe:0x0].0x0 bits 0x13/0x0 rrc: 2 type: IBT gid 0 flags: 0x40218400000000 nid: local remote: 0x0 expref: -99 pid: 8250 timeout: 0 lvb_type: 0 00010000:00010000:0.0:1776178225.873372:0:3340:0:(ldlm_inodebits.c:94:ldlm_reprocess_inodebits_queue()) --- Reprocess resource [0x2000013a1:0xe:0x0].0x0 (0000000030fbaafd) 00010000:00010000:0.0:1776178225.873374:0:3340:0:(ldlm_inodebits.c:96:ldlm_reprocess_inodebits_queue()) Hint 13 00010000:00010000:0.0:1776178225.873376:0:3340:0:(ldlm_lockd.c:1903:ldlm_handle_bl_callback()) ### client blocking callback handler END ns: mdt-lustre-MDT0000_UUID lock: 000000000396420a/0xa40083ab9bf53aab lrc: 2/0,0 mode: --/COS res: [0x2000013a1:0xe:0x0].0x0 bits 0x13/0x0 rrc: 2 type: IBT gid 0 flags: 0x44a19400000000 nid: local remote: 0x0 expref: -99 pid: 8250 timeout: 0 lvb_type: 0 00010000:00010000:0.0:1776178225.873382:0:3340:0:(ldlm_lock.c:216:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: mdt-lustre-MDT0000_UUID lock: 000000000396420a/0xa40083ab9bf53aab lrc: 0/0,0 mode: --/COS res: [0x2000013a1:0xe:0x0].0x0 bits 0x13/0x0 rrc: 2 type: IBT gid 0 flags: 0x44a19400000000 nid: local remote: 0x0 expref: -99 pid: 8250 timeout: 0 lvb_type: 0 00000020:00000080:1.0:1776178225.873686:0:22200:0:(obd_config.c:877:class_cleanup()) lustre-MDT0000: forcing exports to disconnect: 5/12 00000020:00080000:1.0:1776178225.873695:0:22200:0:(genops.c:1752:print_export_data()) lustre-MDT0000: ACTIVE 00000000e857adfe lustre-MDT0001-mdtlov_UUID 0@lo 5 (0 0 1) 0 0 0 0: 0000000000000000 12884901946 stale:0 00000020:00080000:1.0:1776178225.873701:0:22200:0:(genops.c:1752:print_export_data()) lustre-MDT0000: ACTIVE 000000007c69b381 lustre-MDT0000-lwp-MDT0001_UUID 0@lo 4 (0 0 0) 0 0 0 0: 0000000000000000 0 stale:0 00000020:00080000:1.0:1776178225.873706:0:22200:0:(genops.c:1752:print_export_data()) lustre-MDT0000: ACTIVE 0000000057f57203 lustre-MDT0000-lwp-OST0001_UUID 0@lo 4 (0 0 0) 0 0 0 0: 0000000000000000 0 stale:0 00000020:00080000:1.0:1776178225.873711:0:22200:0:(genops.c:1752:print_export_data()) lustre-MDT0000: ACTIVE 00000000ecd3a0d3 lustre-MDT0000-lwp-OST0000_UUID 0@lo 4 (0 0 0) 0 0 0 0: 0000000000000000 0 stale:0 00000020:00080000:1.0:1776178225.873717:0:22200:0:(genops.c:1752:print_export_data()) lustre-MDT0000: ACTIVE 00000000117deaf8 b50963bd-8eba-4432-9a54-8888d00e3fa1 192.168.204.3@tcp 6 (0 0 2) 0 0 0 0: 0000000000000000 12884901981 stale:0 00000020:00080000:1.0:1776178225.873721:0:22200:0:(genops.c:1752:print_export_data()) lustre-MDT0000: UNLINKED 00000000ca22e26f b50963bd-8eba-4432-9a54-8888d00e3fa1 1 (0 0 0) 1 0 1 0: 0000000000000000 8589934704 stale:0 00000020:00080000:1.0:1776178225.873727:0:22200:0:(genops.c:1525:class_disconnect_exports()) OBD device 5 (000000006de24446) has exports, disconnecting them 00000020:00080000:1.0:1776178225.873731:0:22200:0:(genops.c:1501:class_disconnect_export_list()) lustre-MDT0000: disconnecting export at 0@lo (00000000e857adfe), last request at 1776178220 00000020:00000080:1.0:1776178225.873740:0:22200:0:(genops.c:1442:class_disconnect()) disconnect: cookie 0xa40083ab9bf324f2 00000020:00010000:1.0:1776178225.873746:0:22200:0:(genops.c:1857:obd_stale_export_put()) Put export 00000000e857adfe: total 2 00000100:00080000:1.0:1776178225.873753:0:22200:0:(import.c:86:import_set_state_nolock()) 00000000599601fd : changing import state from FULL to CLOSED 00000020:00080000:1.0:1776178225.873819:0:22200:0:(genops.c:1506:class_disconnect_export_list()) disconnected export at 0@lo (00000000e857adfe): rc 0 00000020:00080000:1.0:1776178225.873822:0:22200:0:(genops.c:1501:class_disconnect_export_list()) lustre-MDT0000: disconnecting export at 0@lo (000000007c69b381), last request at 1776178220 00000020:00000080:1.0:1776178225.873825:0:22200:0:(genops.c:1442:class_disconnect()) disconnect: cookie 0xa40083ab9bf324e4 00000100:00080000:1.0:1776178225.873829:0:22200:0:(import.c:86:import_set_state_nolock()) 0000000034b61f5b : changing import state from FULL to CLOSED 00000020:00010000:1.0:1776178225.873845:0:9225:0:(genops.c:1843:obd_stale_export_get()) Get export 00000000e857adfe: total 3 00010000:00010000:1.0:1776178225.873868:0:9225:0:(ldlm_lock.c:2634:ldlm_export_cancel_blocked_locks()) Export 00000000e857adfe, canceled 0 locks, left on hash table 1. 00010000:00010000:1.0:1776178225.873896:0:9225:0:(ldlm_lock.c:216:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: mdt-lustre-MDT0000_UUID lock: 00000000076f6c6d/0xa40083ab9bf3255b lrc: 0/0,0 mode: --/PR res: [0x200000007:0x1:0x0].0x0 bits 0x20/0x0 rrc: 3 type: IBT gid 0 flags: 0x44801000000000 nid: 0@lo remote: 0xa40083ab9bf32554 expref: 2 pid: 14128 timeout: 0 lvb_type: 0 00010000:00010000:1.0:1776178225.873912:0:9225:0:(ldlm_lock.c:2662:ldlm_export_cancel_locks()) Export 00000000e857adfe, canceled 1 locks, left on hash table 0. 00010000:00010000:0.0:1776178225.876725:0:6244:0:(ldlm_lockd.c:2792:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000020:00080000:0.0:1776178225.877216:0:22200:0:(genops.c:1506:class_disconnect_export_list()) disconnected export at 0@lo (000000007c69b381): rc 0 00000020:00000080:0.0:1776178225.877221:0:22200:0:(genops.c:985:class_export_put()) final put 000000007c69b381/lustre-MDT0000-lwp-MDT0001_UUID 00000020:00080000:0.0:1776178225.877245:0:22200:0:(genops.c:1501:class_disconnect_export_list()) lustre-MDT0000: disconnecting export at 0@lo (0000000057f57203), last request at 1776178220 00000020:00000080:0.0:1776178225.877254:0:22200:0:(genops.c:1442:class_disconnect()) disconnect: cookie 0xa40083ab9bf324dd 00000100:00080000:0.0:1776178225.877263:0:22200:0:(import.c:86:import_set_state_nolock()) 000000000a4e5daa : changing import state from FULL to CLOSED 00000020:00080000:0.0:1776178225.877321:0:22200:0:(genops.c:1506:class_disconnect_export_list()) disconnected export at 0@lo (0000000057f57203): rc 0 00000020:00000080:0.0:1776178225.877323:0:22200:0:(genops.c:985:class_export_put()) final put 0000000057f57203/lustre-MDT0000-lwp-OST0001_UUID 00000020:00080000:0.0:1776178225.877327:0:22200:0:(genops.c:1501:class_disconnect_export_list()) lustre-MDT0000: disconnecting export at 0@lo (00000000ecd3a0d3), last request at 1776178220 00000020:00000080:0.0:1776178225.877339:0:22200:0:(genops.c:1442:class_disconnect()) disconnect: cookie 0xa40083ab9bf324d6 00000100:00080000:0.0:1776178225.877346:0:22200:0:(import.c:86:import_set_state_nolock()) 00000000ab7617c1 : changing import state from FULL to CLOSED 00000020:00080000:0.0:1776178225.877474:0:22200:0:(genops.c:1506:class_disconnect_export_list()) disconnected export at 0@lo (00000000ecd3a0d3): rc 0 00000020:00000080:0.0:1776178225.877476:0:22200:0:(genops.c:985:class_export_put()) final put 00000000ecd3a0d3/lustre-MDT0000-lwp-OST0000_UUID 00000020:00080000:0.0:1776178225.877482:0:22200:0:(genops.c:1501:class_disconnect_export_list()) lustre-MDT0000: disconnecting export at 192.168.204.3@tcp (00000000117deaf8), last request at 1776178224 00000020:00000080:0.0:1776178225.877484:0:22200:0:(genops.c:1442:class_disconnect()) disconnect: cookie 0xa40083ab9bf323e8 00000020:00010000:0.0:1776178225.877487:0:22200:0:(genops.c:1857:obd_stale_export_put()) Put export 00000000117deaf8: total 3 00000100:00080000:0.0:1776178225.877489:0:22200:0:(import.c:86:import_set_state_nolock()) 00000000095a2387 : changing import state from FULL to CLOSED 00000020:00010000:0.0:1776178225.877501:0:6244:0:(genops.c:1843:obd_stale_export_get()) Get export 00000000117deaf8: total 3 00010000:00010000:0.0:1776178225.877523:0:6244:0:(ldlm_lock.c:2634:ldlm_export_cancel_blocked_locks()) Export 00000000117deaf8, canceled 0 locks, left on hash table 2. 00010000:00010000:0.0:1776178225.877561:0:6244:0:(ldlm_lock.c:216:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: mdt-lustre-MDT0000_UUID lock: 00000000a211f94c/0xa40083ab9bf5386d lrc: 0/0,0 mode: --/PR res: [0x2000013a1:0x9:0x0].0x0 bits 0x11/0x0 rrc: 2 type: IBT gid 0 flags: 0x44a01000000000 nid: 192.168.204.3@tcp remote: 0x771f68aa1eb27bdc expref: 6 pid: 6256 timeout: 0 lvb_type: 0 00010000:00010000:0.0:1776178225.877586:0:6244:0:(ldlm_lock.c:216:ldlm_lock_put()) ### final lock_put on destroyed lock, freeing it. ns: mdt-lustre-MDT0000_UUID lock: 0000000035cf61b1/0xa40083ab9bf52e48 lrc: 0/0,0 mode: --/PR res: [0x200000007:0x1:0x0].0x0 bits 0x11/0x0 rrc: 2 type: IBT gid 0 flags: 0x44a01000000000 nid: 192.168.204.3@tcp remote: 0x771f68aa1eb27943 expref: 5 pid: 6255 timeout: 0 lvb_type: 0 00010000:00010000:0.0:1776178225.877604:0:6244:0:(ldlm_lock.c:2662:ldlm_export_cancel_locks()) Export 00000000117deaf8, canceled 2 locks, left on hash table 0. 00010000:00010000:3.0:1776178225.878505:0:9224:0:(ldlm_lockd.c:2792:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000020:00000080:2.0:1776178225.879199:0:21777:0:(genops.c:944:class_export_destroy()) destroying export 000000007c69b381/lustre-MDT0000-lwp-MDT0001_UUID for lustre-MDT0000 00000020:00000080:2.0:1776178225.879252:0:21777:0:(genops.c:944:class_export_destroy()) destroying export 0000000057f57203/lustre-MDT0000-lwp-OST0001_UUID for lustre-MDT0000 00000020:00000080:2.0:1776178225.879268:0:21777:0:(genops.c:944:class_export_destroy()) destroying export 00000000ecd3a0d3/lustre-MDT0000-lwp-OST0000_UUID for lustre-MDT0000 00000020:00000080:2.0:1776178225.879297:0:21777:0:(genops.c:1164:obd_zombie_import_free()) destroying import 0000000034b61f5b for lustre-MDT0000 00000020:00000080:2.0:1776178225.879303:0:21777:0:(genops.c:1164:obd_zombie_import_free()) destroying import 000000000a4e5daa for lustre-MDT0000 00000020:00000080:2.0:1776178225.879306:0:21777:0:(genops.c:1164:obd_zombie_import_free()) destroying import 00000000ab7617c1 for lustre-MDT0000 00010000:00010000:2.0:1776178225.879318:0:6245:0:(ldlm_lockd.c:2792:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000020:00080000:3.0:1776178225.897777:0:22200:0:(genops.c:1506:class_disconnect_export_list()) disconnected export at 192.168.204.3@tcp (00000000117deaf8): rc 0 00000100:00080000:3.0:1776178225.897830:0:22200:0:(import.c:157:ptlrpc_deactivate_import_nolock()) setting import lustre-MDT0001_UUID INVALID 00000100:00100000:3.0:1776178225.897836:0:22200:0:(client.c:2868:ptlrpc_free_committed()) lustre-MDT0001-osp-MDT0000: committing for last_committed 17179871222 gen 3 00000100:00080000:3.0:1776178225.897844:0:22200:0:(pinger.c:414:ptlrpc_pinger_del_import()) removing pingable import lustre-MDT0000-mdtlov_UUID->lustre-MDT0001_UUID 00000100:00080000:3.0:1776178225.897873:0:22200:0:(import.c:86:import_set_state_nolock()) 000000000af120e1 lustre-MDT0001_UUID: changing import state from FULL to CONNECTING 00000100:00100000:3.0:1776178225.897901:0:22200:0:(client.c:1733:ptlrpc_send_new_req()) Sending RPC req@0000000056ddcb09 pname:cluuid:pid:xid:nid:opc:job umount:lustre-MDT0000-mdtlov_UUID:22200:1862456998051968:0@lo:39:umount.0 00000100:00100000:3.0:1776178225.897980:0:22200:0:(events.c:361:request_in_callback()) peer: 12345-0@lo (source: 12345-0@lo) 00000100:00100000:3.0:1776178225.898019:0:22200:0:(client.c:2500:ptlrpc_set_wait()) set 0000000037173204 going to sleep for 5 seconds 00000100:00100000:2.0:1776178225.901695:0:6261:0:(service.c:2152:ptlrpc_server_handle_req_in()) got req x1862456998051968 00000100:00100000:2.0:1776178225.901788:0:6261:0:(nrs_fifo.c:179:nrs_fifo_req_get()) NRS start fifo request from 12345-0@lo, seq: 710 00000100:00100000:2.0:1776178225.901796:0:6261:0:(service.c:2324:ptlrpc_server_handle_request()) Handling RPC req@0000000039ff8335 pname:cluuid+ref:pid:xid:nid:opc:job mdt_out00_001:lustre-MDT0000-mdtlov_UUID+5:22200:x1862456998051968:12345-0@lo:39:umount.0 00000020:00000080:2.0:1776178225.901840:0:6261:0:(genops.c:1442:class_disconnect()) disconnect: cookie 0xa40083ab9bf3224b 00000100:00080000:2.0:1776178225.901849:0:6261:0:(import.c:86:import_set_state_nolock()) 000000003564f86c : changing import state from FULL to CLOSED 00000001:00000004:2.0:1776178225.901911:0:6261:0:(tgt_lastrcvd.c:705:tgt_server_data_update()) lustre-MDT0001_UUID: mount_count is 4, last_transno is 17179876275 00000100:00100000:2.0:1776178225.902160:0:6261:0:(service.c:2372:ptlrpc_server_handle_request()) Handled RPC req@0000000039ff8335 pname:cluuid+ref:pid:xid:nid:opc:job mdt_out00_001:lustre-MDT0000-mdtlov_UUID+2:22200:x1862456998051968:12345-0@lo:39:umount.0 Request processed in 363us (4179us total) trans 0 rc 0/0 00000100:00100000:2.0:1776178225.902168:0:6261:0:(nrs_fifo.c:241:nrs_fifo_req_stop()) NRS stop fifo request from 12345-0@lo, seq: 710 00010000:00010000:2.0:1776178225.902209:0:9224:0:(ldlm_lockd.c:2792:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:3.0:1776178225.902655:0:22200:0:(client.c:2215:ptlrpc_check_set()) Completed RPC req@0000000056ddcb09 pname:cluuid:pid:xid:nid:opc:job umount:lustre-MDT0000-mdtlov_UUID:22200:1862456998051968:0@lo:39:umount.0 00000100:00080000:3.0:1776178225.902684:0:22200:0:(import.c:86:import_set_state_nolock()) 000000000af120e1 lustre-MDT0001_UUID: changing import state from CONNECTING to CLOSED 00000100:00080000:3.0:1776178225.902692:0:22200:0:(import.c:157:ptlrpc_deactivate_import_nolock()) setting import lustre-MDT0001_UUID INVALID 00000100:00100000:3.0:1776178225.902694:0:22200:0:(client.c:2868:ptlrpc_free_committed()) lustre-MDT0001-osp-MDT0000: committing for last_committed 17179871222 gen 4 00000100:00100000:3.0:1776178225.902699:0:22200:0:(import.c:339:ptlrpc_invalidate_import()) Sleeping 20s for inflight to error out 00000100:00080000:3.0:1776178225.904886:0:22200:0:(import.c:157:ptlrpc_deactivate_import_nolock()) setting import lustre-OST0000_UUID INVALID 00000100:00100000:3.0:1776178225.904898:0:22200:0:(client.c:2868:ptlrpc_free_committed()) lustre-OST0000-osc-MDT0000: committing for last_committed 4294969352 gen 3 00000004:00080000:3.0:1776178225.904921:0:22200:0:(osp_dev.c:1606:osp_import_event()) got inactive 00000100:00080000:3.0:1776178225.904926:0:22200:0:(pinger.c:414:ptlrpc_pinger_del_import()) removing pingable import lustre-MDT0000-mdtlov_UUID->lustre-OST0000_UUID 00000100:00080000:3.0:1776178225.904958:0:22200:0:(import.c:86:import_set_state_nolock()) 00000000aecf43e3 lustre-OST0000_UUID: changing import state from FULL to CONNECTING 00000100:00100000:3.0:1776178225.904984:0:22200:0:(client.c:1733:ptlrpc_send_new_req()) Sending RPC req@0000000056ddcb09 pname:cluuid:pid:xid:nid:opc:job umount:lustre-MDT0000-mdtlov_UUID:22200:1862456998052032:0@lo:9:umount.0 00000100:00100000:3.0:1776178225.905056:0:22200:0:(events.c:361:request_in_callback()) peer: 12345-0@lo (source: 12345-0@lo) 00000100:00100000:3.0:1776178225.905107:0:22200:0:(client.c:2500:ptlrpc_set_wait()) set 0000000037173204 going to sleep for 5 seconds 00000100:00100000:3.0:1776178225.905133:0:20759:0:(service.c:2152:ptlrpc_server_handle_req_in()) got req x1862456998052032 00000100:00100000:3.0:1776178225.905178:0:20759:0:(nrs_fifo.c:179:nrs_fifo_req_get()) NRS start fifo request from 12345-0@lo, seq: 5299 00000100:00100000:3.0:1776178225.905184:0:20759:0:(service.c:2324:ptlrpc_server_handle_request()) Handling RPC req@000000000e0e1967 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_010:lustre-MDT0000-mdtlov_UUID+5:22200:x1862456998052032:12345-0@lo:9:umount.0 00000020:00000080:3.0:1776178225.905217:0:20759:0:(genops.c:1442:class_disconnect()) disconnect: cookie 0xa40083ab9bf32267 00000100:00080000:3.0:1776178225.905224:0:20759:0:(import.c:86:import_set_state_nolock()) 00000000353bb3e3 : changing import state from FULL to CLOSED 00000001:00000004:3.0:1776178225.905260:0:20759:0:(tgt_lastrcvd.c:705:tgt_server_data_update()) lustre-OST0000_UUID: mount_count is 1, last_transno is 4294969364 00000100:00100000:3.0:1776178225.905419:0:20759:0:(service.c:2372:ptlrpc_server_handle_request()) Handled RPC req@000000000e0e1967 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_010:lustre-MDT0000-mdtlov_UUID+2:22200:x1862456998052032:12345-0@lo:9:umount.0 Request processed in 235us (364us total) trans 0 rc 0/0 00000100:00100000:3.0:1776178225.905428:0:20759:0:(nrs_fifo.c:241:nrs_fifo_req_stop()) NRS stop fifo request from 12345-0@lo, seq: 5299 00000100:00100000:3.0:1776178225.905485:0:22200:0:(client.c:2215:ptlrpc_check_set()) Completed RPC req@0000000056ddcb09 pname:cluuid:pid:xid:nid:opc:job umount:lustre-MDT0000-mdtlov_UUID:22200:1862456998052032:0@lo:9:umount.0 00000100:00080000:3.0:1776178225.905502:0:22200:0:(import.c:86:import_set_state_nolock()) 00000000aecf43e3 lustre-OST0000_UUID: changing import state from CONNECTING to CLOSED 00000004:00080000:3.0:1776178225.905506:0:22200:0:(osp_dev.c:1590:osp_import_event()) got disconnected 00000004:00080000:3.0:1776178225.905507:0:22200:0:(osp_dev.c:1606:osp_import_event()) got inactive 00000100:00080000:3.0:1776178225.905509:0:22200:0:(import.c:157:ptlrpc_deactivate_import_nolock()) setting import lustre-OST0000_UUID INVALID 00000100:00100000:3.0:1776178225.905511:0:22200:0:(client.c:2868:ptlrpc_free_committed()) lustre-OST0000-osc-MDT0000: committing for last_committed 4294969352 gen 4 00000004:00080000:3.0:1776178225.905514:0:22200:0:(osp_dev.c:1606:osp_import_event()) got inactive 00000100:00100000:3.0:1776178225.905516:0:22200:0:(import.c:339:ptlrpc_invalidate_import()) Sleeping 20s for inflight to error out 00000100:00080000:3.0:1776178225.905626:0:22200:0:(import.c:157:ptlrpc_deactivate_import_nolock()) setting import lustre-OST0001_UUID INVALID 00000100:00100000:3.0:1776178225.905630:0:22200:0:(client.c:2868:ptlrpc_free_committed()) lustre-OST0001-osc-MDT0000: committing for last_committed 4294969323 gen 3 00000004:00080000:3.0:1776178225.905642:0:22200:0:(osp_dev.c:1606:osp_import_event()) got inactive 00000100:00080000:3.0:1776178225.905644:0:22200:0:(pinger.c:414:ptlrpc_pinger_del_import()) removing pingable import lustre-MDT0000-mdtlov_UUID->lustre-OST0001_UUID 00000100:00080000:3.0:1776178225.905654:0:22200:0:(import.c:86:import_set_state_nolock()) 000000006a88e60b lustre-OST0001_UUID: changing import state from FULL to CONNECTING 00000100:00100000:3.0:1776178225.905663:0:22200:0:(client.c:1733:ptlrpc_send_new_req()) Sending RPC req@0000000056ddcb09 pname:cluuid:pid:xid:nid:opc:job umount:lustre-MDT0000-mdtlov_UUID:22200:1862456998052096:0@lo:9:umount.0 00000100:00100000:3.0:1776178225.905690:0:22200:0:(events.c:361:request_in_callback()) peer: 12345-0@lo (source: 12345-0@lo) 00000100:00100000:3.0:1776178225.905704:0:22200:0:(client.c:2500:ptlrpc_set_wait()) set 0000000037173204 going to sleep for 5 seconds 00000100:00100000:3.0:1776178225.905718:0:20755:0:(service.c:2152:ptlrpc_server_handle_req_in()) got req x1862456998052096 00000100:00100000:3.0:1776178225.905744:0:20755:0:(nrs_fifo.c:179:nrs_fifo_req_get()) NRS start fifo request from 12345-0@lo, seq: 5300 00000100:00100000:3.0:1776178225.905748:0:20755:0:(service.c:2324:ptlrpc_server_handle_request()) Handling RPC req@000000002ab0a797 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_006:lustre-MDT0000-mdtlov_UUID+5:22200:x1862456998052096:12345-0@lo:9:umount.0 00000020:00000080:3.0:1776178225.905765:0:20755:0:(genops.c:1442:class_disconnect()) disconnect: cookie 0xa40083ab9bf32275 00000100:00080000:3.0:1776178225.905769:0:20755:0:(import.c:86:import_set_state_nolock()) 00000000b115c585 : changing import state from FULL to CLOSED 00000001:00000004:3.0:1776178225.905792:0:20755:0:(tgt_lastrcvd.c:705:tgt_server_data_update()) lustre-OST0001_UUID: mount_count is 1, last_transno is 4294969339 00000100:00100000:3.0:1776178225.905879:0:20755:0:(service.c:2372:ptlrpc_server_handle_request()) Handled RPC req@000000002ab0a797 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_006:lustre-MDT0000-mdtlov_UUID+2:22200:x1862456998052096:12345-0@lo:9:umount.0 Request processed in 130us (188us total) trans 0 rc 0/0 00000100:00100000:3.0:1776178225.905885:0:20755:0:(nrs_fifo.c:241:nrs_fifo_req_stop()) NRS stop fifo request from 12345-0@lo, seq: 5300 00000100:00100000:3.0:1776178225.905913:0:22200:0:(client.c:2215:ptlrpc_check_set()) Completed RPC req@0000000056ddcb09 pname:cluuid:pid:xid:nid:opc:job umount:lustre-MDT0000-mdtlov_UUID:22200:1862456998052096:0@lo:9:umount.0 00000100:00080000:3.0:1776178225.905922:0:22200:0:(import.c:86:import_set_state_nolock()) 000000006a88e60b lustre-OST0001_UUID: changing import state from CONNECTING to CLOSED 00000004:00080000:3.0:1776178225.905925:0:22200:0:(osp_dev.c:1590:osp_import_event()) got disconnected 00000004:00080000:3.0:1776178225.905926:0:22200:0:(osp_dev.c:1606:osp_import_event()) got inactive 00000100:00080000:3.0:1776178225.905927:0:22200:0:(import.c:157:ptlrpc_deactivate_import_nolock()) setting import lustre-OST0001_UUID INVALID 00000100:00100000:3.0:1776178225.905928:0:22200:0:(client.c:2868:ptlrpc_free_committed()) lustre-OST0001-osc-MDT0000: committing for last_committed 4294969323 gen 4 00000004:00080000:3.0:1776178225.905930:0:22200:0:(osp_dev.c:1606:osp_import_event()) got inactive 00000100:00100000:3.0:1776178225.905931:0:22200:0:(import.c:339:ptlrpc_invalidate_import()) Sleeping 20s for inflight to error out 00000020:00080000:3.0:1776178225.906096:0:22200:0:(genops.c:1530:class_disconnect_exports()) OBD device 5 (000000006de24446) has no exports 00010000:00010000:2.0:1776178225.906233:0:6245:0:(ldlm_lockd.c:2792:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00010000:00010000:2.0:1776178225.906241:0:9224:0:(ldlm_lockd.c:2792:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000020:00000080:1.0:1776178225.925771:0:9225:0:(genops.c:985:class_export_put()) final put 00000000e857adfe/lustre-MDT0001-mdtlov_UUID 00010000:00010000:1.0:1776178225.926442:0:9225:0:(ldlm_lockd.c:2792:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00010000:00010000:1.0:1776178225.926445:0:9225:0:(ldlm_lockd.c:2792:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00010000:00010000:1.0:1776178225.926448:0:9225:0:(ldlm_lockd.c:2792:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000020:00000080:2.0:1776178225.927138:0:36:0:(genops.c:944:class_export_destroy()) destroying export 00000000e857adfe/lustre-MDT0001-mdtlov_UUID for lustre-MDT0000 00000020:00000080:2.0:1776178225.927215:0:36:0:(genops.c:1164:obd_zombie_import_free()) destroying import 00000000599601fd for lustre-MDT0000 00000020:00000080:0.0:1776178225.931054:0:6244:0:(genops.c:985:class_export_put()) final put 00000000117deaf8/b50963bd-8eba-4432-9a54-8888d00e3fa1 00010000:00010000:0.0:1776178225.931154:0:6244:0:(ldlm_lockd.c:2792:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00010000:00010000:0.0:1776178225.931157:0:6244:0:(ldlm_lockd.c:2792:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00010000:00010000:0.0:1776178225.931160:0:6244:0:(ldlm_lockd.c:2792:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000020:00000080:1.0:1776178225.932494:0:21777:0:(genops.c:944:class_export_destroy()) destroying export 00000000117deaf8/b50963bd-8eba-4432-9a54-8888d00e3fa1 for lustre-MDT0000 00000020:00000080:1.0:1776178225.932585:0:21777:0:(genops.c:1164:obd_zombie_import_free()) destroying import 00000000095a2387 for lustre-MDT0000 00000100:00100000:0.0:1776178226.016585:0:3355:0:(client.c:1733:ptlrpc_send_new_req()) Sending RPC req@00000000a101cbf3 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_02:lustre-MDT0001-lwp-OST0000_UUID:3355:1862456998052224:0@lo:400:kworker/u8:1.0 00000100:00100000:0.0:1776178226.016658:0:3355:0:(events.c:361:request_in_callback()) peer: 12345-0@lo (source: 12345-0@lo) 00000100:00100000:0.0:1776178226.016689:0:3355:0:(client.c:1733:ptlrpc_send_new_req()) Sending RPC req@000000001da3f43e pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_02:lustre-MDT0001-mdtlov_UUID:3355:1862456998052480:0@lo:400:kworker/u8:1.0 00000100:00100000:0.0:1776178226.016711:0:3355:0:(events.c:361:request_in_callback()) peer: 12345-0@lo (source: 12345-0@lo) 00000100:00100000:0.0:1776178226.016735:0:3355:0:(ptlrpcd.c:415:ptlrpcd_check()) transfer 2 async RPCs [3->2] 00000100:00100000:0.0:1776178226.016740:0:3355:0:(client.c:1733:ptlrpc_send_new_req()) Sending RPC req@000000009e7a8bb5 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_02:lustre-MDT0000-lwp-OST0001_UUID:3355:1862456998052288:0@lo:400:kworker/u8:1.0 00000100:00100000:0.0:1776178226.016760:0:3355:0:(events.c:361:request_in_callback()) peer: 12345-0@lo (source: 12345-0@lo) 00000100:00100000:0.0:1776178226.016774:0:3355:0:(client.c:1733:ptlrpc_send_new_req()) Sending RPC req@00000000f351bcc3 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_02:lustre-MDT0001-mdtlov_UUID:3355:1862456998052544:0@lo:400:kworker/u8:1.0 00000100:00100000:0.0:1776178226.016797:0:3355:0:(events.c:361:request_in_callback()) peer: 12345-0@lo (source: 12345-0@lo) 00000100:00100000:0.0:1776178226.016828:0:6257:0:(service.c:2152:ptlrpc_server_handle_req_in()) got req x1862456998052224 00000100:00100000:0.0:1776178226.016877:0:6257:0:(service.c:2152:ptlrpc_server_handle_req_in()) got req x1862456998052288 00000100:00100000:0.0:1776178226.016899:0:6257:0:(nrs_fifo.c:179:nrs_fifo_req_get()) NRS start fifo request from 12345-0@lo, seq: 941 00000100:00100000:0.0:1776178226.016905:0:6257:0:(service.c:2324:ptlrpc_server_handle_request()) Handling RPC req@00000000505e62f8 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_002:lustre-MDT0001-lwp-OST0000_UUID+5:3355:x1862456998052224:12345-0@lo:400:kworker/u8:1.0 00000100:00100000:0.0:1776178226.016955:0:6257:0:(service.c:2372:ptlrpc_server_handle_request()) Handled RPC req@00000000505e62f8 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_002:lustre-MDT0001-lwp-OST0000_UUID+5:3355:x1862456998052224:12345-0@lo:400:kworker/u8:1.0 Request processed in 49us (298us total) trans 0 rc 0/0 00000100:00100000:2.0:1776178226.016956:0:6255:0:(nrs_fifo.c:179:nrs_fifo_req_get()) NRS start fifo request from 12345-0@lo, seq: 6637 00000100:00100000:2.0:1776178226.016961:0:6255:0:(service.c:2324:ptlrpc_server_handle_request()) Handling RPC req@000000006c748d80 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_000:0+-99:3355:x1862456998052288:12345-0@lo:400:kworker/u8:1.0 00000100:00100000:0.0:1776178226.016962:0:6257:0:(nrs_fifo.c:241:nrs_fifo_req_stop()) NRS stop fifo request from 12345-0@lo, seq: 941 00000020:00080000:2.0:1776178226.016966:0:6255:0:(tgt_handler.c:780:tgt_request_handle()) operation 400 on unconnected OST from 12345-0@lo 00000100:00100000:0.0:1776178226.016988:0:20759:0:(service.c:2152:ptlrpc_server_handle_req_in()) got req x1862456998052480 00000100:00100000:2.0:1776178226.016997:0:6255:0:(service.c:2372:ptlrpc_server_handle_request()) Handled RPC req@000000006c748d80 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_000:0+-99:3355:x1862456998052288:12345-0@lo:400:kworker/u8:1.0 Request processed in 36us (237us total) trans 0 rc -107/-107 00000100:00100000:2.0:1776178226.017003:0:6255:0:(nrs_fifo.c:241:nrs_fifo_req_stop()) NRS stop fifo request from 12345-0@lo, seq: 6637 00000100:00100000:0.0:1776178226.017011:0:20759:0:(service.c:2152:ptlrpc_server_handle_req_in()) got req x1862456998052544 00000100:00100000:0.0:1776178226.017026:0:20759:0:(nrs_fifo.c:179:nrs_fifo_req_get()) NRS start fifo request from 12345-0@lo, seq: 529 00000100:00100000:0.0:1776178226.017030:0:20759:0:(service.c:2324:ptlrpc_server_handle_request()) Handling RPC req@00000000165897bf pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_010:lustre-MDT0001-mdtlov_UUID+5:3355:x1862456998052480:12345-0@lo:400:kworker/u8:1.0 00000100:00100000:2.0:1776178226.017050:0:20762:0:(nrs_fifo.c:179:nrs_fifo_req_get()) NRS start fifo request from 12345-0@lo, seq: 530 00000100:00100000:2.0:1776178226.017054:0:20762:0:(service.c:2324:ptlrpc_server_handle_request()) Handling RPC req@000000009016e75e pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_013:lustre-MDT0001-mdtlov_UUID+5:3355:x1862456998052544:12345-0@lo:400:kworker/u8:1.0 00000100:00100000:0.0:1776178226.017078:0:20759:0:(service.c:2372:ptlrpc_server_handle_request()) Handled RPC req@00000000165897bf pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_010:lustre-MDT0001-mdtlov_UUID+5:3355:x1862456998052480:12345-0@lo:400:kworker/u8:1.0 Request processed in 48us (367us total) trans 0 rc 0/0 00000100:00100000:0.0:1776178226.017084:0:20759:0:(nrs_fifo.c:241:nrs_fifo_req_stop()) NRS stop fifo request from 12345-0@lo, seq: 529 00000100:00100000:2.0:1776178226.017105:0:20762:0:(service.c:2372:ptlrpc_server_handle_request()) Handled RPC req@000000009016e75e pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_013:lustre-MDT0001-mdtlov_UUID+5:3355:x1862456998052544:12345-0@lo:400:kworker/u8:1.0 Request processed in 50us (307us total) trans 0 rc 0/0 00000100:00100000:2.0:1776178226.017110:0:20762:0:(nrs_fifo.c:241:nrs_fifo_req_stop()) NRS stop fifo request from 12345-0@lo, seq: 530 00000100:00080000:0.0:1776178226.017129:0:3355:0:(recover.c:232:ptlrpc_request_handle_notconn()) import lustre-MDT0000-lwp-OST0001 of lustre-MDT0000_UUID@192.168.204.103@tcp abruptly disconnected: reconnecting 00000100:00080000:0.0:1776178226.017134:0:3355:0:(import.c:86:import_set_state_nolock()) 00000000dfc8db13 lustre-MDT0000_UUID: changing import state from FULL to DISCONN 00000100:02000400:0.0:1776178226.017138:0:3355:0:(import.c:200:ptlrpc_set_import_discon()) lustre-MDT0000-lwp-OST0001: Connection to lustre-MDT0000 (at 0@lo) was lost; in progress operations using this service will wait for recovery to complete 00000100:00100000:3.0:1776178226.017661:0:3354:0:(client.c:1733:ptlrpc_send_new_req()) Sending RPC req@00000000ecea139c pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_01:lustre-MDT0000-lwp-OST0000_UUID:3354:1862456998052160:0@lo:400:kworker/u8:1.0 00000100:00100000:3.0:1776178226.017699:0:3354:0:(events.c:361:request_in_callback()) peer: 12345-0@lo (source: 12345-0@lo) 00000100:00100000:3.0:1776178226.017730:0:3354:0:(client.c:1733:ptlrpc_send_new_req()) Sending RPC req@0000000056e345df pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_01:lustre-MDT0001-mdtlov_UUID:3354:1862456998052416:0@lo:400:kworker/u8:1.0 00000100:00100000:3.0:1776178226.017756:0:3354:0:(events.c:361:request_in_callback()) peer: 12345-0@lo (source: 12345-0@lo) 00000100:00100000:3.0:1776178226.017776:0:3354:0:(ptlrpcd.c:415:ptlrpcd_check()) transfer 2 async RPCs [0->1] 00000100:00100000:3.0:1776178226.017782:0:3354:0:(client.c:1733:ptlrpc_send_new_req()) Sending RPC req@00000000f95e9da5 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_01:lustre-MDT0001-lwp-OST0001_UUID:3354:1862456998052352:0@lo:400:kworker/u8:1.0 00000100:00100000:3.0:1776178226.017802:0:3354:0:(events.c:361:request_in_callback()) peer: 12345-0@lo (source: 12345-0@lo) 00000100:00100000:3.0:1776178226.017816:0:3354:0:(client.c:1733:ptlrpc_send_new_req()) Sending RPC req@000000001946bc3d pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_01:lustre-MDT0000-lwp-MDT0001_UUID:3354:1862456998052608:0@lo:400:kworker/u8:1.0 00000100:00100000:3.0:1776178226.017835:0:3354:0:(events.c:361:request_in_callback()) peer: 12345-0@lo (source: 12345-0@lo) 00000100:00100000:3.0:1776178226.017864:0:14128:0:(service.c:2152:ptlrpc_server_handle_req_in()) got req x1862456998052416 00000100:00100000:3.0:1776178226.017890:0:14128:0:(nrs_fifo.c:179:nrs_fifo_req_get()) NRS start fifo request from 12345-0@lo, seq: 711 00000100:00100000:3.0:1776178226.017894:0:14128:0:(service.c:2324:ptlrpc_server_handle_request()) Handling RPC req@000000004a9818ac pname:cluuid+ref:pid:xid:nid:opc:job mdt_out00_002:0+-99:3354:x1862456998052416:12345-0@lo:400:kworker/u8:1.0 00000020:00080000:3.0:1776178226.017901:0:14128:0:(tgt_handler.c:780:tgt_request_handle()) operation 400 on unconnected OST from 12345-0@lo 00000100:00100000:3.0:1776178226.017928:0:14128:0:(service.c:2372:ptlrpc_server_handle_request()) Handled RPC req@000000004a9818ac pname:cluuid+ref:pid:xid:nid:opc:job mdt_out00_002:0+-99:3354:x1862456998052416:12345-0@lo:400:kworker/u8:1.0 Request processed in 34us (172us total) trans 0 rc -107/-107 00000100:00100000:3.0:1776178226.017934:0:14128:0:(nrs_fifo.c:241:nrs_fifo_req_stop()) NRS stop fifo request from 12345-0@lo, seq: 711 00000100:00080000:3.0:1776178226.017960:0:3354:0:(recover.c:232:ptlrpc_request_handle_notconn()) import lustre-MDT0000-osp-MDT0001 of lustre-MDT0000_UUID@192.168.204.103@tcp abruptly disconnected: reconnecting 00000100:00080000:3.0:1776178226.017963:0:3354:0:(import.c:86:import_set_state_nolock()) 00000000ba4f1821 lustre-MDT0000_UUID: changing import state from FULL to DISCONN 00000100:02000400:3.0:1776178226.017967:0:3354:0:(import.c:200:ptlrpc_set_import_discon()) lustre-MDT0000-osp-MDT0001: Connection to lustre-MDT0000 (at 0@lo) was lost; in progress operations using this service will wait for recovery to complete 00000100:00080000:3.0:1776178226.017972:0:3354:0:(import.c:86:import_set_state_nolock()) 00000000ba4f1821 lustre-MDT0000_UUID: changing import state from DISCONN to CONNECTING 00000100:00080000:3.0:1776178226.017975:0:3354:0:(import.c:534:import_select_connection()) lustre-MDT0000-osp-MDT0001: connect to NID 0@lo last attempt 687 00000100:00080000:3.0:1776178226.017980:0:3354:0:(import.c:615:import_select_connection()) lustre-MDT0000-osp-MDT0001: import 00000000ba4f1821 using connection 192.168.204.103@tcp/0@lo 00000100:00100000:3.0:1776178226.018007:0:3354:0:(import.c:807:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5) req@000000005d3a2b34 x1862456998052672/t0(0) o38->lustre-MDT0000-osp-MDT0001@0@lo:24/4 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' 00000100:00100000:2.0:1776178226.018023:0:8250:0:(service.c:2152:ptlrpc_server_handle_req_in()) got req x1862456998052160 00000100:00100000:3.0:1776178226.018026:0:3354:0:(client.c:2215:ptlrpc_check_set()) Completed RPC req@0000000056e345df pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_01:lustre-MDT0001-mdtlov_UUID:3354:1862456998052416:0@lo:400:kworker/u8:1.0 00000100:00100000:2.0:1776178226.018045:0:8250:0:(service.c:2152:ptlrpc_server_handle_req_in()) got req x1862456998052352 00000100:00100000:2.0:1776178226.018079:0:8250:0:(service.c:2152:ptlrpc_server_handle_req_in()) got req x1862456998052608 00000100:00100000:3.0:1776178226.018083:0:6256:0:(nrs_fifo.c:179:nrs_fifo_req_get()) NRS start fifo request from 12345-0@lo, seq: 942 00000100:00100000:3.0:1776178226.018087:0:6256:0:(service.c:2324:ptlrpc_server_handle_request()) Handling RPC req@00000000fb56871d pname:cluuid+ref:pid:xid:nid:opc:job mdt00_001:lustre-MDT0001-lwp-OST0001_UUID+5:3354:x1862456998052352:12345-0@lo:400:kworker/u8:1.0 00000100:00100000:2.0:1776178226.018091:0:8250:0:(nrs_fifo.c:179:nrs_fifo_req_get()) NRS start fifo request from 12345-0@lo, seq: 6638 00000100:00100000:2.0:1776178226.018093:0:8250:0:(service.c:2324:ptlrpc_server_handle_request()) Handling RPC req@000000004cbcc47c pname:cluuid+ref:pid:xid:nid:opc:job mdt00_003:0+-99:3354:x1862456998052160:12345-0@lo:400:kworker/u8:1.0 00000020:00080000:2.0:1776178226.018097:0:8250:0:(tgt_handler.c:780:tgt_request_handle()) operation 400 on unconnected OST from 12345-0@lo 00000100:00100000:2.0:1776178226.018122:0:8250:0:(service.c:2372:ptlrpc_server_handle_request()) Handled RPC req@000000004cbcc47c pname:cluuid+ref:pid:xid:nid:opc:job mdt00_003:0+-99:3354:x1862456998052160:12345-0@lo:400:kworker/u8:1.0 Request processed in 27us (423us total) trans 0 rc -107/-107 00000100:00100000:2.0:1776178226.018132:0:8250:0:(nrs_fifo.c:241:nrs_fifo_req_stop()) NRS stop fifo request from 12345-0@lo, seq: 6638 00000100:00100000:2.0:1776178226.018146:0:8250:0:(nrs_fifo.c:179:nrs_fifo_req_get()) NRS start fifo request from 12345-0@lo, seq: 6639 00000100:00100000:2.0:1776178226.018149:0:8250:0:(service.c:2324:ptlrpc_server_handle_request()) Handling RPC req@000000002f729f84 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_003:0+-99:3354:x1862456998052608:12345-0@lo:400:kworker/u8:1.0 00000020:00080000:2.0:1776178226.018153:0:8250:0:(tgt_handler.c:780:tgt_request_handle()) operation 400 on unconnected OST from 12345-0@lo 00000100:00100000:1.0:1776178226.018218:0:3352:0:(client.c:1733:ptlrpc_send_new_req()) Sending RPC req@000000005d3a2b34 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:lustre-MDT0001-mdtlov_UUID:3352:1862456998052672:0@lo:38:ptlrpcd_00_01.0 00000100:00100000:1.0:1776178226.018252:0:3352:0:(events.c:361:request_in_callback()) peer: 12345-0@lo (source: 12345-0@lo) 00000100:00100000:1.0:1776178226.018294:0:14128:0:(service.c:2152:ptlrpc_server_handle_req_in()) got req x1862456998052672 00000100:00100000:1.0:1776178226.018314:0:14128:0:(nrs_fifo.c:179:nrs_fifo_req_get()) NRS start fifo request from 12345-0@lo, seq: 712 00000100:00100000:1.0:1776178226.018318:0:14128:0:(service.c:2324:ptlrpc_server_handle_request()) Handling RPC req@00000000eeeaf4ad pname:cluuid+ref:pid:xid:nid:opc:job mdt_out00_002:0+-99:3352:x1862456998052672:12345-0@lo:38:ptlrpcd_00_01.0 00010000:02000000:1.0:1776178226.018329:0:14128:0:(ldlm_lib.c:1139:target_handle_connect()) lustre-MDT0000: Not available for connect from 0@lo (stopping) 00000100:00100000:3.0:1776178226.018874:0:6256:0:(service.c:2372:ptlrpc_server_handle_request()) Handled RPC req@00000000fb56871d pname:cluuid+ref:pid:xid:nid:opc:job mdt00_001:lustre-MDT0001-lwp-OST0001_UUID+5:3354:x1862456998052352:12345-0@lo:400:kworker/u8:1.0 Request processed in 782us (1069us total) trans 0 rc 0/0 00000100:00100000:3.0:1776178226.018883:0:6256:0:(nrs_fifo.c:241:nrs_fifo_req_stop()) NRS stop fifo request from 12345-0@lo, seq: 942 00000100:00100000:2.0:1776178226.019419:0:8250:0:(service.c:2372:ptlrpc_server_handle_request()) Handled RPC req@000000002f729f84 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_003:0+-99:3354:x1862456998052608:12345-0@lo:400:kworker/u8:1.0 Request processed in 1267us (1582us total) trans 0 rc -107/-107 00000100:00100000:2.0:1776178226.019428:0:8250:0:(nrs_fifo.c:241:nrs_fifo_req_stop()) NRS stop fifo request from 12345-0@lo, seq: 6639 00000100:00100000:2.0:1776178226.019488:0:3354:0:(client.c:2215:ptlrpc_check_set()) Completed RPC req@00000000f95e9da5 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_01:lustre-MDT0001-lwp-OST0001_UUID:3354:1862456998052352:0@lo:400:kworker/u8:1.0 00000100:00080000:2.0:1776178226.019500:0:3354:0:(recover.c:232:ptlrpc_request_handle_notconn()) import lustre-MDT0000-lwp-MDT0001 of lustre-MDT0000_UUID@192.168.204.103@tcp abruptly disconnected: reconnecting 00000100:00080000:2.0:1776178226.019505:0:3354:0:(import.c:86:import_set_state_nolock()) 00000000a7c55b1f lustre-MDT0000_UUID: changing import state from FULL to DISCONN 00000100:02000400:2.0:1776178226.019508:0:3354:0:(import.c:200:ptlrpc_set_import_discon()) lustre-MDT0000-lwp-MDT0001: Connection to lustre-MDT0000 (at 0@lo) was lost; in progress operations using this service will wait for recovery to complete 00000100:00080000:2.0:1776178226.019513:0:3354:0:(import.c:86:import_set_state_nolock()) 00000000a7c55b1f lustre-MDT0000_UUID: changing import state from DISCONN to CONNECTING 00000100:00080000:2.0:1776178226.019517:0:3354:0:(import.c:534:import_select_connection()) lustre-MDT0000-lwp-MDT0001: connect to NID 0@lo last attempt 687 00000100:00080000:2.0:1776178226.019522:0:3354:0:(import.c:615:import_select_connection()) lustre-MDT0000-lwp-MDT0001: import 00000000a7c55b1f using connection 192.168.204.103@tcp/0@lo 00000100:00100000:2.0:1776178226.019549:0:3354:0:(import.c:807:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5) req@0000000028ade006 x1862456998052736/t0(0) o38->lustre-MDT0000-lwp-MDT0001@0@lo:12/10 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' 00000100:00100000:2.0:1776178226.019571:0:3354:0:(client.c:2215:ptlrpc_check_set()) Completed RPC req@000000001946bc3d pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_01:lustre-MDT0000-lwp-MDT0001_UUID:3354:1862456998052608:0@lo:400:kworker/u8:1.0 00000100:00080000:2.0:1776178226.019824:0:3354:0:(recover.c:232:ptlrpc_request_handle_notconn()) import lustre-MDT0000-lwp-OST0000 of lustre-MDT0000_UUID@192.168.204.103@tcp abruptly disconnected: reconnecting 00000100:00080000:2.0:1776178226.019828:0:3354:0:(import.c:86:import_set_state_nolock()) 0000000047601a98 lustre-MDT0000_UUID: changing import state from FULL to DISCONN 00000100:02000400:2.0:1776178226.019832:0:3354:0:(import.c:200:ptlrpc_set_import_discon()) lustre-MDT0000-lwp-OST0000: Connection to lustre-MDT0000 (at 0@lo) was lost; in progress operations using this service will wait for recovery to complete 00000100:00080000:2.0:1776178226.019835:0:3354:0:(import.c:86:import_set_state_nolock()) 0000000047601a98 lustre-MDT0000_UUID: changing import state from DISCONN to CONNECTING 00000100:00080000:2.0:1776178226.019838:0:3354:0:(import.c:534:import_select_connection()) lustre-MDT0000-lwp-OST0000: connect to NID 0@lo last attempt 687 00000100:00080000:2.0:1776178226.019842:0:3354:0:(import.c:615:import_select_connection()) lustre-MDT0000-lwp-OST0000: import 0000000047601a98 using connection 192.168.204.103@tcp/0@lo 00000100:00100000:2.0:1776178226.019856:0:3354:0:(import.c:807:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5) req@00000000b85458e9 x1862456998052800/t0(0) o38->lustre-MDT0000-lwp-OST0000@0@lo:12/10 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' 00000100:00100000:2.0:1776178226.019865:0:3354:0:(client.c:2215:ptlrpc_check_set()) Completed RPC req@00000000ecea139c pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_01:lustre-MDT0000-lwp-OST0000_UUID:3354:1862456998052160:0@lo:400:kworker/u8:1.0 00000100:00100000:3.0:1776178226.020001:0:3352:0:(client.c:1733:ptlrpc_send_new_req()) Sending RPC req@0000000028ade006 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:lustre-MDT0000-lwp-MDT0001_UUID:3352:1862456998052736:0@lo:38:ptlrpcd_00_01.0 00000100:00100000:3.0:1776178226.020086:0:3352:0:(events.c:361:request_in_callback()) peer: 12345-0@lo (source: 12345-0@lo) 00000100:00100000:3.0:1776178226.020103:0:3352:0:(client.c:1733:ptlrpc_send_new_req()) Sending RPC req@00000000b85458e9 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:lustre-MDT0000-lwp-OST0000_UUID:3352:1862456998052800:0@lo:38:ptlrpcd_00_01.0 00000100:00100000:3.0:1776178226.020122:0:3352:0:(events.c:361:request_in_callback()) peer: 12345-0@lo (source: 12345-0@lo) 00000100:00100000:2.0:1776178226.020199:0:6255:0:(service.c:2152:ptlrpc_server_handle_req_in()) got req x1862456998052736 00000100:00100000:2.0:1776178226.020232:0:6255:0:(service.c:2152:ptlrpc_server_handle_req_in()) got req x1862456998052800 00000100:00100000:2.0:1776178226.020248:0:6255:0:(nrs_fifo.c:179:nrs_fifo_req_get()) NRS start fifo request from 12345-0@lo, seq: 6640 00000100:00100000:2.0:1776178226.020253:0:6255:0:(service.c:2324:ptlrpc_server_handle_request()) Handling RPC req@000000001946bc3d pname:cluuid+ref:pid:xid:nid:opc:job mdt00_000:0+-99:3352:x1862456998052736:12345-0@lo:38:ptlrpcd_00_01.0 00000100:00100000:3.0:1776178226.020558:0:8250:0:(nrs_fifo.c:179:nrs_fifo_req_get()) NRS start fifo request from 12345-0@lo, seq: 6641 00000100:00100000:3.0:1776178226.020563:0:8250:0:(service.c:2324:ptlrpc_server_handle_request()) Handling RPC req@0000000056e345df pname:cluuid+ref:pid:xid:nid:opc:job mdt00_003:0+-99:3352:x1862456998052800:12345-0@lo:38:ptlrpcd_00_01.0 00010000:02000000:2.0:1776178226.021475:0:6255:0:(ldlm_lib.c:1139:target_handle_connect()) lustre-MDT0000: Not available for connect from 0@lo (stopping) 00000100:00100000:2.0:1776178226.021571:0:6255:0:(service.c:2372:ptlrpc_server_handle_request()) Handled RPC req@000000001946bc3d pname:cluuid+ref:pid:xid:nid:opc:job mdt00_000:0+-99:3352:x1862456998052736:12345-0@lo:38:ptlrpcd_00_01.0 Request processed in 1316us (1510us total) trans 0 rc -19/-19 00010000:02000000:3.0:1776178226.021710:0:8250:0:(ldlm_lib.c:1139:target_handle_connect()) lustre-MDT0000: Not available for connect from 0@lo (stopping) 00000100:00100000:3.0:1776178226.021745:0:8250:0:(service.c:2372:ptlrpc_server_handle_request()) Handled RPC req@0000000056e345df pname:cluuid+ref:pid:xid:nid:opc:job mdt00_003:0+-99:3352:x1862456998052800:12345-0@lo:38:ptlrpcd_00_01.0 Request processed in 1181us (1623us total) trans 0 rc -19/-19 00000100:00100000:3.0:1776178226.021754:0:8250:0:(nrs_fifo.c:241:nrs_fifo_req_stop()) NRS stop fifo request from 12345-0@lo, seq: 6641 00000100:00100000:2.0:1776178226.021988:0:6255:0:(nrs_fifo.c:241:nrs_fifo_req_stop()) NRS stop fifo request from 12345-0@lo, seq: 6640 00000100:00080000:2.0:1776178226.022053:0:3352:0:(import.c:86:import_set_state_nolock()) 00000000a7c55b1f lustre-MDT0000_UUID: changing import state from CONNECTING to DISCONN 00000100:00080000:2.0:1776178226.022073:0:3352:0:(import.c:1429:ptlrpc_connect_interpret()) recovery of lustre-MDT0000_UUID on 192.168.204.103@tcp failed (-19) 00000100:00100000:2.0:1776178226.022080:0:3352:0:(client.c:2215:ptlrpc_check_set()) Completed RPC req@0000000028ade006 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:lustre-MDT0000-lwp-MDT0001_UUID:3352:1862456998052736:0@lo:38:ptlrpcd_00_01.0 00000100:00080000:2.0:1776178226.022094:0:3352:0:(import.c:86:import_set_state_nolock()) 0000000047601a98 lustre-MDT0000_UUID: changing import state from CONNECTING to DISCONN 00000100:00080000:2.0:1776178226.022097:0:3352:0:(import.c:1429:ptlrpc_connect_interpret()) recovery of lustre-MDT0000_UUID on 192.168.204.103@tcp failed (-19) 00000100:00100000:2.0:1776178226.022100:0:3352:0:(client.c:2215:ptlrpc_check_set()) Completed RPC req@00000000b85458e9 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:lustre-MDT0000-lwp-OST0000_UUID:3352:1862456998052800:0@lo:38:ptlrpcd_00_01.0 00000100:00080000:0.0:1776178226.036912:0:3355:0:(import.c:86:import_set_state_nolock()) 00000000dfc8db13 lustre-MDT0000_UUID: changing import state from DISCONN to CONNECTING 00000100:00080000:0.0:1776178226.036923:0:3355:0:(import.c:534:import_select_connection()) lustre-MDT0000-lwp-OST0001: connect to NID 0@lo last attempt 687 00000100:00080000:0.0:1776178226.036932:0:3355:0:(import.c:615:import_select_connection()) lustre-MDT0000-lwp-OST0001: import 00000000dfc8db13 using connection 192.168.204.103@tcp/0@lo 00000100:00100000:0.0:1776178226.036979:0:3355:0:(import.c:807:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5) req@000000006466ee18 x1862456998052864/t0(0) o38->lustre-MDT0000-lwp-OST0001@0@lo:12/10 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' 00000100:00100000:0.0:1776178226.037005:0:3355:0:(client.c:2215:ptlrpc_check_set()) Completed RPC req@000000009e7a8bb5 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_02:lustre-MDT0000-lwp-OST0001_UUID:3355:1862456998052288:0@lo:400:kworker/u8:1.0 00000100:00100000:0.0:1776178226.037044:0:3355:0:(client.c:2215:ptlrpc_check_set()) Completed RPC req@00000000f351bcc3 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_02:lustre-MDT0001-mdtlov_UUID:3355:1862456998052544:0@lo:400:kworker/u8:1.0 00000100:00100000:0.0:1776178226.037103:0:3355:0:(client.c:2215:ptlrpc_check_set()) Completed RPC req@00000000a101cbf3 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_02:lustre-MDT0001-lwp-OST0000_UUID:3355:1862456998052224:0@lo:400:kworker/u8:1.0 00000100:00100000:0.0:1776178226.037118:0:3355:0:(client.c:2215:ptlrpc_check_set()) Completed RPC req@000000001da3f43e pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_02:lustre-MDT0001-mdtlov_UUID:3355:1862456998052480:0@lo:400:kworker/u8:1.0 00000100:00100000:2.0:1776178226.039542:0:3352:0:(client.c:1733:ptlrpc_send_new_req()) Sending RPC req@000000006466ee18 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:lustre-MDT0000-lwp-OST0001_UUID:3352:1862456998052864:0@lo:38:ptlrpcd_00_02.0 00000100:00100000:2.0:1776178226.039629:0:3352:0:(events.c:361:request_in_callback()) peer: 12345-0@lo (source: 12345-0@lo) 00000100:00100000:0.0:1776178226.039836:0:11281:0:(service.c:2152:ptlrpc_server_handle_req_in()) got req x1862456998052864 00000100:00100000:0.0:1776178226.039886:0:11281:0:(nrs_fifo.c:179:nrs_fifo_req_get()) NRS start fifo request from 12345-0@lo, seq: 6642 00000100:00100000:0.0:1776178226.039892:0:11281:0:(service.c:2324:ptlrpc_server_handle_request()) Handling RPC req@00000000b85458e9 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_004:0+-99:3352:x1862456998052864:12345-0@lo:38:ptlrpcd_00_02.0 00010000:02000000:0.0:1776178226.039910:0:11281:0:(ldlm_lib.c:1139:target_handle_connect()) lustre-MDT0000: Not available for connect from 0@lo (stopping) 00000100:00100000:0.0:1776178226.039963:0:11281:0:(service.c:2372:ptlrpc_server_handle_request()) Handled RPC req@00000000b85458e9 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_004:0+-99:3352:x1862456998052864:12345-0@lo:38:ptlrpcd_00_02.0 Request processed in 69us (335us total) trans 0 rc -19/-19 00000100:00100000:0.0:1776178226.039970:0:11281:0:(nrs_fifo.c:241:nrs_fifo_req_stop()) NRS stop fifo request from 12345-0@lo, seq: 6642 00000100:00080000:2.0:1776178226.040100:0:3352:0:(import.c:86:import_set_state_nolock()) 00000000dfc8db13 lustre-MDT0000_UUID: changing import state from CONNECTING to DISCONN 00000100:00080000:2.0:1776178226.040104:0:3352:0:(import.c:1429:ptlrpc_connect_interpret()) recovery of lustre-MDT0000_UUID on 192.168.204.103@tcp failed (-19) 00000100:00100000:2.0:1776178226.040108:0:3352:0:(client.c:2215:ptlrpc_check_set()) Completed RPC req@000000006466ee18 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:lustre-MDT0000-lwp-OST0001_UUID:3352:1862456998052864:0@lo:38:ptlrpcd_00_02.0 00000100:00100000:2.0:1776178226.059296:0:14128:0:(service.c:2372:ptlrpc_server_handle_request()) Handled RPC req@00000000eeeaf4ad pname:cluuid+ref:pid:xid:nid:opc:job mdt_out00_002:0+-99:3352:x1862456998052672:12345-0@lo:38:ptlrpcd_00_01.0 Request processed in 40954us (41022us total) trans 0 rc -19/-19 00000100:00100000:2.0:1776178226.059322:0:14128:0:(nrs_fifo.c:241:nrs_fifo_req_stop()) NRS stop fifo request from 12345-0@lo, seq: 712 00000100:00080000:0.0:1776178226.059461:0:3352:0:(import.c:86:import_set_state_nolock()) 00000000ba4f1821 lustre-MDT0000_UUID: changing import state from CONNECTING to DISCONN 00000100:00080000:0.0:1776178226.059466:0:3352:0:(import.c:1429:ptlrpc_connect_interpret()) recovery of lustre-MDT0000_UUID on 192.168.204.103@tcp failed (-19) 00000100:00100000:0.0:1776178226.059471:0:3352:0:(client.c:2215:ptlrpc_check_set()) Completed RPC req@000000005d3a2b34 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:lustre-MDT0001-mdtlov_UUID:3352:1862456998052672:0@lo:38:ptlrpcd_00_01.0 00010000:00010000:2.0:1776178226.658200:0:6245:0:(ldlm_lockd.c:2792:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:2.0:1776178227.552960:0:3354:0:(client.c:1733:ptlrpc_send_new_req()) Sending RPC req@000000001da3f43e pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_01:lustre-MDT0001-mdtlov_UUID:3354:1862456998052928:0@lo:13:osp-pre-1-1.0 00000100:00100000:2.0:1776178227.553050:0:3354:0:(events.c:361:request_in_callback()) peer: 12345-0@lo (source: 12345-0@lo) 00000100:00100000:3.0:1776178227.553186:0:8231:0:(service.c:2152:ptlrpc_server_handle_req_in()) got req x1862456998052928 00000100:00100000:3.0:1776178227.553240:0:8231:0:(nrs_fifo.c:179:nrs_fifo_req_get()) NRS start fifo request from 12345-0@lo, seq: 497 00000100:00100000:3.0:1776178227.553247:0:8231:0:(service.c:2324:ptlrpc_server_handle_request()) Handling RPC req@0000000028ade006 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_create00:lustre-MDT0001-mdtlov_UUID+5:3354:x1862456998052928:12345-0@lo:13:osp-pre-1-1.0 00002000:00000024:3.0:1776178227.553293:0:8231:0:(ofd_obd.c:740:ofd_statfs()) blocks cached 229376 granted 491746368 pending 0 free 3923689472 avail 3692167168 00000100:00100000:3.0:1776178227.553770:0:8231:0:(service.c:2372:ptlrpc_server_handle_request()) Handled RPC req@0000000028ade006 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_create00:lustre-MDT0001-mdtlov_UUID+5:3354:x1862456998052928:12345-0@lo:13:osp-pre-1-1.0 Request processed in 521us (721us total) trans 0 rc 0/0 00000100:00100000:3.0:1776178227.553779:0:8231:0:(nrs_fifo.c:241:nrs_fifo_req_stop()) NRS stop fifo request from 12345-0@lo, seq: 497 00000100:00100000:2.0:1776178227.555872:0:3354:0:(client.c:2215:ptlrpc_check_set()) Completed RPC req@000000001da3f43e pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_01:lustre-MDT0001-mdtlov_UUID:3354:1862456998052928:0@lo:13:osp-pre-1-1.0 00010000:00010000:2.0:1776178227.683676:0:9224:0:(ldlm_lockd.c:2792:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:1.0:1776178228.064982:0:3356:0:(ptlrpcd.c:415:ptlrpcd_check()) transfer 1 async RPCs [2->3] 00000100:00100000:1.0:1776178228.065011:0:3356:0:(client.c:1733:ptlrpc_send_new_req()) Sending RPC req@00000000a62daf9a pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_03:lustre-MDT0001-mdtlov_UUID:3356:1862456998052992:0@lo:13:osp-pre-0-1.0 00000100:00100000:1.0:1776178228.065141:0:3356:0:(events.c:361:request_in_callback()) peer: 12345-0@lo (source: 12345-0@lo) 00000100:00100000:3.0:1776178228.065254:0:8231:0:(service.c:2152:ptlrpc_server_handle_req_in()) got req x1862456998052992 00000100:00100000:3.0:1776178228.065310:0:8231:0:(nrs_fifo.c:179:nrs_fifo_req_get()) NRS start fifo request from 12345-0@lo, seq: 498 00000100:00100000:3.0:1776178228.065318:0:8231:0:(service.c:2324:ptlrpc_server_handle_request()) Handling RPC req@00000000854f8d52 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_create00:lustre-MDT0001-mdtlov_UUID+5:3356:x1862456998052992:12345-0@lo:13:osp-pre-0-1.0 00002000:00000024:3.0:1776178228.065352:0:8231:0:(ofd_obd.c:740:ofd_statfs()) blocks cached 315392 granted 484883776 pending 0 free 3923689472 avail 3692167168 00000100:00100000:3.0:1776178228.065409:0:8231:0:(service.c:2372:ptlrpc_server_handle_request()) Handled RPC req@00000000854f8d52 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_create00:lustre-MDT0001-mdtlov_UUID+5:3356:x1862456998052992:12345-0@lo:13:osp-pre-0-1.0 Request processed in 91us (270us total) trans 0 rc 0/0 00000100:00100000:3.0:1776178228.065416:0:8231:0:(nrs_fifo.c:241:nrs_fifo_req_stop()) NRS stop fifo request from 12345-0@lo, seq: 498 00000100:00100000:1.0:1776178228.065557:0:3356:0:(client.c:2215:ptlrpc_check_set()) Completed RPC req@00000000a62daf9a pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_03:lustre-MDT0001-mdtlov_UUID:3356:1862456998052992:0@lo:13:osp-pre-0-1.0 00010000:00010000:2.0:1776178228.705771:0:9225:0:(ldlm_lockd.c:2792:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:0.0:1776178229.537623:0:3348:0:(events.c:361:request_in_callback()) peer: 12345-192.168.204.3@tcp (source: 12345-192.168.204.3@tcp) 00000100:00100000:0.0:1776178229.537679:0:3348:0:(events.c:361:request_in_callback()) peer: 12345-192.168.204.3@tcp (source: 12345-192.168.204.3@tcp) 00000100:00100000:0.0:1776178229.537732:0:3348:0:(events.c:361:request_in_callback()) peer: 12345-192.168.204.3@tcp (source: 12345-192.168.204.3@tcp) 00000100:00100000:0.0:1776178229.537765:0:3348:0:(events.c:361:request_in_callback()) peer: 12345-192.168.204.3@tcp (source: 12345-192.168.204.3@tcp) 00000100:00100000:0.0:1776178229.537796:0:3348:0:(events.c:361:request_in_callback()) peer: 12345-192.168.204.3@tcp (source: 12345-192.168.204.3@tcp) 00000100:00100000:0.0:1776178229.537836:0:14440:0:(service.c:2152:ptlrpc_server_handle_req_in()) got req x1862456979110784 00000100:00100000:0.0:1776178229.537888:0:14440:0:(service.c:2152:ptlrpc_server_handle_req_in()) got req x1862456979110720 00000100:00100000:0.0:1776178229.537910:0:14440:0:(nrs_fifo.c:179:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.204.3@tcp, seq: 531 00000100:00100000:0.0:1776178229.537917:0:14440:0:(service.c:2324:ptlrpc_server_handle_request()) Handling RPC req@000000006466ee18 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_004:b50963bd-8eba-4432-9a54-8888d00e3fa1+9:2270:x1862456979110784:12345-192.168.204.3@tcp:400:kworker/u8:4.0 00000100:00100000:0.0:1776178229.537997:0:14440:0:(service.c:2372:ptlrpc_server_handle_request()) Handled RPC req@000000006466ee18 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_004:b50963bd-8eba-4432-9a54-8888d00e3fa1+9:2270:x1862456979110784:12345-192.168.204.3@tcp:400:kworker/u8:4.0 Request processed in 79us (265us total) trans 0 rc 0/0 00000100:00100000:0.0:1776178229.538005:0:14440:0:(nrs_fifo.c:241:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.204.3@tcp, seq: 531 00000100:00100000:0.0:1776178229.538024:0:14440:0:(nrs_fifo.c:179:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.204.3@tcp, seq: 532 00000100:00100000:0.0:1776178229.538029:0:14440:0:(service.c:2324:ptlrpc_server_handle_request()) Handling RPC req@0000000090cc55f6 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_004:b50963bd-8eba-4432-9a54-8888d00e3fa1+8:2270:x1862456979110720:12345-192.168.204.3@tcp:400:kworker/u8:4.0 00000100:00100000:3.0:1776178229.538172:0:20140:0:(service.c:2152:ptlrpc_server_handle_req_in()) got req x1862456979110528 00000100:00100000:3.0:1776178229.538202:0:20140:0:(nrs_fifo.c:179:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.204.3@tcp, seq: 165 00000100:00100000:3.0:1776178229.538207:0:20140:0:(service.c:2324:ptlrpc_server_handle_request()) Handling RPC req@00000000e31b009d pname:cluuid+ref:pid:xid:nid:opc:job ll_mgs_0002:5b5c66b1-1a2f-4651-8548-0c75b9a38a94+9:2270:x1862456979110528:12345-192.168.204.3@tcp:400:kworker/u8:4.0 00000100:00100000:3.0:1776178229.538247:0:20140:0:(service.c:2372:ptlrpc_server_handle_request()) Handled RPC req@00000000e31b009d pname:cluuid+ref:pid:xid:nid:opc:job ll_mgs_0002:5b5c66b1-1a2f-4651-8548-0c75b9a38a94+9:2270:x1862456979110528:12345-192.168.204.3@tcp:400:kworker/u8:4.0 Request processed in 39us (638us total) trans 0 rc 0/0 00000100:00100000:3.0:1776178229.538254:0:20140:0:(nrs_fifo.c:241:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.204.3@tcp, seq: 165 00000100:00100000:1.0:1776178229.538362:0:11281:0:(service.c:2152:ptlrpc_server_handle_req_in()) got req x1862456979110656 00000100:00100000:1.0:1776178229.538410:0:11281:0:(service.c:2152:ptlrpc_server_handle_req_in()) got req x1862456979110592 00000100:00100000:1.0:1776178229.538431:0:11281:0:(nrs_fifo.c:179:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.204.3@tcp, seq: 943 00000100:00100000:1.0:1776178229.538437:0:11281:0:(service.c:2324:ptlrpc_server_handle_request()) Handling RPC req@00000000a62daf9a pname:cluuid+ref:pid:xid:nid:opc:job mdt00_004:b50963bd-8eba-4432-9a54-8888d00e3fa1+6:2270:x1862456979110656:12345-192.168.204.3@tcp:400:kworker/u8:4.0 00000100:00100000:3.0:1776178229.538449:0:6255:0:(nrs_fifo.c:179:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.204.3@tcp, seq: 6643 00000100:00100000:3.0:1776178229.538455:0:6255:0:(service.c:2324:ptlrpc_server_handle_request()) Handling RPC req@000000005d3a2b34 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_000:0+-99:2270:x1862456979110592:12345-192.168.204.3@tcp:400:kworker/u8:4.0 00000020:00080000:3.0:1776178229.538461:0:6255:0:(tgt_handler.c:780:tgt_request_handle()) operation 400 on unconnected OST from 12345-192.168.204.3@tcp 00000100:00100000:1.0:1776178229.538477:0:11281:0:(service.c:2372:ptlrpc_server_handle_request()) Handled RPC req@00000000a62daf9a pname:cluuid+ref:pid:xid:nid:opc:job mdt00_004:b50963bd-8eba-4432-9a54-8888d00e3fa1+6:2270:x1862456979110656:12345-192.168.204.3@tcp:400:kworker/u8:4.0 Request processed in 39us (798us total) trans 0 rc 0/0 00000100:00100000:1.0:1776178229.538485:0:11281:0:(nrs_fifo.c:241:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.204.3@tcp, seq: 943 00000100:00100000:3.0:1776178229.538492:0:6255:0:(service.c:2372:ptlrpc_server_handle_request()) Handled RPC req@000000005d3a2b34 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_000:0+-99:2270:x1862456979110592:12345-192.168.204.3@tcp:400:kworker/u8:4.0 Request processed in 34us (726us total) trans 0 rc -107/-107 00000100:00100000:3.0:1776178229.538501:0:6255:0:(nrs_fifo.c:241:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.204.3@tcp, seq: 6643 00000100:00100000:0.0:1776178229.539235:0:14440:0:(service.c:2372:ptlrpc_server_handle_request()) Handled RPC req@0000000090cc55f6 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_004:b50963bd-8eba-4432-9a54-8888d00e3fa1+8:2270:x1862456979110720:12345-192.168.204.3@tcp:400:kworker/u8:4.0 Request processed in 1202us (1435us total) trans 0 rc 0/0 00000100:00100000:0.0:1776178229.539245:0:14440:0:(nrs_fifo.c:241:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.204.3@tcp, seq: 532 00000100:00100000:2.0:1776178229.552427:0:3348:0:(events.c:361:request_in_callback()) peer: 12345-192.168.204.3@tcp (source: 12345-192.168.204.3@tcp) 00000100:00100000:0.0:1776178229.552634:0:6256:0:(service.c:2152:ptlrpc_server_handle_req_in()) got req x1862456979110848 00000100:00100000:0.0:1776178229.552680:0:6256:0:(nrs_fifo.c:179:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.204.3@tcp, seq: 6644 00000100:00100000:0.0:1776178229.552686:0:6256:0:(service.c:2324:ptlrpc_server_handle_request()) Handling RPC req@000000007d766b2a pname:cluuid+ref:pid:xid:nid:opc:job mdt00_001:0+-99:2269:x1862456979110848:12345-192.168.204.3@tcp:38:ptlrpcd_00_00.0 00010000:02000000:0.0:1776178229.552703:0:6256:0:(ldlm_lib.c:1139:target_handle_connect()) lustre-MDT0000: Not available for connect from 192.168.204.3@tcp (stopping) 00000100:00100000:0.0:1776178229.562406:0:6256:0:(service.c:2372:ptlrpc_server_handle_request()) Handled RPC req@000000007d766b2a pname:cluuid+ref:pid:xid:nid:opc:job mdt00_001:0+-99:2269:x1862456979110848:12345-192.168.204.3@tcp:38:ptlrpcd_00_00.0 Request processed in 9711us (9983us total) trans 0 rc -19/-19 00000100:00100000:0.0:1776178229.562425:0:6256:0:(nrs_fifo.c:241:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.204.3@tcp, seq: 6644 00010000:00010000:0.0:1776178229.728898:0:6244:0:(ldlm_lockd.c:2792:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00010000:00010000:2.0:1776178230.761087:0:6245:0:(ldlm_lockd.c:2792:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00080000:1.0:1776178231.136338:0:21777:0:(pinger.c:251:ptlrpc_pinger_process_import()) lustre-MDT0000-lwp-OST0000_UUID->lustre-MDT0000_UUID: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00080000:1.0:1776178231.136346:0:21777:0:(recover.c:57:ptlrpc_initiate_recovery()) lustre-MDT0000_UUID: starting recovery 00000100:00080000:1.0:1776178231.136350:0:21777:0:(import.c:86:import_set_state_nolock()) 0000000047601a98 lustre-MDT0000_UUID: changing import state from DISCONN to CONNECTING 00000100:00080000:1.0:1776178231.136359:0:21777:0:(import.c:534:import_select_connection()) lustre-MDT0000-lwp-OST0000: connect to NID 0@lo last attempt 933 00000100:00080000:1.0:1776178231.136363:0:21777:0:(import.c:581:import_select_connection()) lustre-MDT0000-lwp-OST0000: tried all connections, increasing latency to 6s 00000100:00080000:1.0:1776178231.136369:0:21777:0:(import.c:615:import_select_connection()) lustre-MDT0000-lwp-OST0000: import 0000000047601a98 using connection 192.168.204.103@tcp/0@lo 00000100:00100000:1.0:1776178231.136393:0:21777:0:(import.c:807:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5) req@00000000b30b9cdd x1862456998053120/t0(0) o38->lustre-MDT0000-lwp-OST0000@0@lo:12/10 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' 00000100:00080000:1.0:1776178231.136431:0:21777:0:(pinger.c:251:ptlrpc_pinger_process_import()) lustre-MDT0000-lwp-OST0001_UUID->lustre-MDT0000_UUID: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00080000:1.0:1776178231.136435:0:21777:0:(recover.c:57:ptlrpc_initiate_recovery()) lustre-MDT0000_UUID: starting recovery 00000100:00080000:1.0:1776178231.136437:0:21777:0:(import.c:86:import_set_state_nolock()) 00000000dfc8db13 lustre-MDT0000_UUID: changing import state from DISCONN to CONNECTING 00000100:00080000:1.0:1776178231.136440:0:21777:0:(import.c:534:import_select_connection()) lustre-MDT0000-lwp-OST0001: connect to NID 0@lo last attempt 933 00000100:00080000:1.0:1776178231.136442:0:21777:0:(import.c:581:import_select_connection()) lustre-MDT0000-lwp-OST0001: tried all connections, increasing latency to 6s 00000100:00080000:1.0:1776178231.136445:0:21777:0:(import.c:615:import_select_connection()) lustre-MDT0000-lwp-OST0001: import 00000000dfc8db13 using connection 192.168.204.103@tcp/0@lo 00000100:00100000:1.0:1776178231.136456:0:21777:0:(import.c:807:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5) req@00000000f4172b4c x1862456998053248/t0(0) o38->lustre-MDT0000-lwp-OST0001@0@lo:12/10 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' 00000100:00080000:1.0:1776178231.136470:0:21777:0:(pinger.c:251:ptlrpc_pinger_process_import()) lustre-MDT0001-mdtlov_UUID->lustre-MDT0000_UUID: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00080000:1.0:1776178231.136473:0:21777:0:(recover.c:57:ptlrpc_initiate_recovery()) lustre-MDT0000_UUID: starting recovery 00000100:00080000:1.0:1776178231.136474:0:21777:0:(import.c:86:import_set_state_nolock()) 00000000ba4f1821 lustre-MDT0000_UUID: changing import state from DISCONN to CONNECTING 00000100:00080000:1.0:1776178231.136477:0:21777:0:(import.c:534:import_select_connection()) lustre-MDT0000-osp-MDT0001: connect to NID 0@lo last attempt 933 00000100:00080000:1.0:1776178231.136479:0:21777:0:(import.c:581:import_select_connection()) lustre-MDT0000-osp-MDT0001: tried all connections, increasing latency to 6s 00000100:00080000:1.0:1776178231.136482:0:21777:0:(import.c:615:import_select_connection()) lustre-MDT0000-osp-MDT0001: import 00000000ba4f1821 using connection 192.168.204.103@tcp/0@lo 00000100:00100000:1.0:1776178231.136492:0:21777:0:(import.c:807:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5) req@00000000ab295186 x1862456998053376/t0(0) o38->lustre-MDT0000-osp-MDT0001@0@lo:24/4 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' 00000100:00080000:1.0:1776178231.136512:0:21777:0:(pinger.c:251:ptlrpc_pinger_process_import()) lustre-MDT0000-lwp-MDT0001_UUID->lustre-MDT0000_UUID: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00080000:1.0:1776178231.136515:0:21777:0:(recover.c:57:ptlrpc_initiate_recovery()) lustre-MDT0000_UUID: starting recovery 00000100:00080000:1.0:1776178231.136516:0:21777:0:(import.c:86:import_set_state_nolock()) 00000000a7c55b1f lustre-MDT0000_UUID: changing import state from DISCONN to CONNECTING 00000100:00080000:1.0:1776178231.136519:0:21777:0:(import.c:534:import_select_connection()) lustre-MDT0000-lwp-MDT0001: connect to NID 0@lo last attempt 933 00000100:00080000:1.0:1776178231.136521:0:21777:0:(import.c:581:import_select_connection()) lustre-MDT0000-lwp-MDT0001: tried all connections, increasing latency to 6s 00000100:00080000:1.0:1776178231.136524:0:21777:0:(import.c:615:import_select_connection()) lustre-MDT0000-lwp-MDT0001: import 00000000a7c55b1f using connection 192.168.204.103@tcp/0@lo 00000100:00100000:1.0:1776178231.136534:0:21777:0:(import.c:807:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5) req@000000008516f059 x1862456998053568/t0(0) o38->lustre-MDT0000-lwp-MDT0001@0@lo:12/10 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' 00000100:00100000:1.0:1776178231.136668:0:3353:0:(client.c:1733:ptlrpc_send_new_req()) Sending RPC req@00000000ba9bc15d pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_00:lustre-MDT0001-lwp-OST0000_UUID:3353:1862456998053184:0@lo:400:kworker/u8:3.0 00000100:00100000:2.0:1776178231.136732:0:3356:0:(client.c:1733:ptlrpc_send_new_req()) Sending RPC req@00000000d7e84c97 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_03:ac61f839-0e8e-4cac-9c59-08bcfbbad330:3356:1862456998053056:0@lo:400:kworker/u8:3.0 00000100:00100000:1.0:1776178231.136743:0:3353:0:(events.c:361:request_in_callback()) peer: 12345-0@lo (source: 12345-0@lo) 00000100:00100000:3.0:1776178231.136801:0:3355:0:(client.c:1733:ptlrpc_send_new_req()) Sending RPC req@000000001e11faca pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_02:lustre-MDT0001-mdtlov_UUID:3355:1862456998053440:0@lo:400:kworker/u8:3.0 00000100:00100000:0.0:1776178231.136877:0:3352:0:(client.c:1733:ptlrpc_send_new_req()) Sending RPC req@00000000b30b9cdd pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:lustre-MDT0000-lwp-OST0000_UUID:3352:1862456998053120:0@lo:38:kworker/u8:3.0 00000100:00100000:2.0:1776178231.145241:0:3356:0:(events.c:361:request_in_callback()) peer: 12345-0@lo (source: 12345-0@lo) 00000100:00100000:2.0:1776178231.145297:0:3356:0:(client.c:1733:ptlrpc_send_new_req()) Sending RPC req@000000006522caac pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_03:lustre-MDT0001-mdtlov_UUID:3356:1862456998053504:0@lo:400:kworker/u8:3.0 00000100:00100000:2.0:1776178231.145363:0:3356:0:(events.c:361:request_in_callback()) peer: 12345-0@lo (source: 12345-0@lo) 00000100:00100000:2.0:1776178231.145425:0:6256:0:(service.c:2152:ptlrpc_server_handle_req_in()) got req x1862456998053184 00000100:00100000:2.0:1776178231.145471:0:6256:0:(nrs_fifo.c:179:nrs_fifo_req_get()) NRS start fifo request from 12345-0@lo, seq: 944 00000100:00100000:2.0:1776178231.145477:0:6256:0:(service.c:2324:ptlrpc_server_handle_request()) Handling RPC req@00000000ba7f03ed pname:cluuid+ref:pid:xid:nid:opc:job mdt00_001:lustre-MDT0001-lwp-OST0000_UUID+5:3353:x1862456998053184:12345-0@lo:400:kworker/u8:3.0 00000100:00100000:2.0:1776178231.145528:0:6256:0:(service.c:2372:ptlrpc_server_handle_request()) Handled RPC req@00000000ba7f03ed pname:cluuid+ref:pid:xid:nid:opc:job mdt00_001:lustre-MDT0001-lwp-OST0000_UUID+5:3353:x1862456998053184:12345-0@lo:400:kworker/u8:3.0 Request processed in 50us (8785us total) trans 0 rc 0/0 00000100:00100000:2.0:1776178231.145535:0:6256:0:(nrs_fifo.c:241:nrs_fifo_req_stop()) NRS stop fifo request from 12345-0@lo, seq: 944 00000100:00100000:2.0:1776178231.145571:0:20137:0:(service.c:2152:ptlrpc_server_handle_req_in()) got req x1862456998053056 00000100:00100000:2.0:1776178231.145595:0:20137:0:(nrs_fifo.c:179:nrs_fifo_req_get()) NRS start fifo request from 12345-0@lo, seq: 166 00000100:00100000:2.0:1776178231.145600:0:20137:0:(service.c:2324:ptlrpc_server_handle_request()) Handling RPC req@0000000035060500 pname:cluuid+ref:pid:xid:nid:opc:job ll_mgs_0000:ac61f839-0e8e-4cac-9c59-08bcfbbad330+18:3356:x1862456998053056:12345-0@lo:400:kworker/u8:3.0 00000100:00100000:2.0:1776178231.145633:0:20137:0:(service.c:2372:ptlrpc_server_handle_request()) Handled RPC req@0000000035060500 pname:cluuid+ref:pid:xid:nid:opc:job ll_mgs_0000:ac61f839-0e8e-4cac-9c59-08bcfbbad330+18:3356:x1862456998053056:12345-0@lo:400:kworker/u8:3.0 Request processed in 32us (400us total) trans 0 rc 0/0 00000100:00100000:2.0:1776178231.145639:0:20137:0:(nrs_fifo.c:241:nrs_fifo_req_stop()) NRS stop fifo request from 12345-0@lo, seq: 166 00000100:00100000:2.0:1776178231.145679:0:3356:0:(client.c:2215:ptlrpc_check_set()) Completed RPC req@00000000d7e84c97 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_03:ac61f839-0e8e-4cac-9c59-08bcfbbad330:3356:1862456998053056:0@lo:400:kworker/u8:3.0 00000100:00100000:2.0:1776178231.145720:0:14440:0:(service.c:2152:ptlrpc_server_handle_req_in()) got req x1862456998053504 00000100:00100000:2.0:1776178231.145744:0:14440:0:(nrs_fifo.c:179:nrs_fifo_req_get()) NRS start fifo request from 12345-0@lo, seq: 533 00000100:00100000:2.0:1776178231.145748:0:14440:0:(service.c:2324:ptlrpc_server_handle_request()) Handling RPC req@0000000065dbeee0 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_004:lustre-MDT0001-mdtlov_UUID+5:3356:x1862456998053504:12345-0@lo:400:kworker/u8:3.0 00000100:00100000:2.0:1776178231.145781:0:14440:0:(service.c:2372:ptlrpc_server_handle_request()) Handled RPC req@0000000065dbeee0 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_004:lustre-MDT0001-mdtlov_UUID+5:3356:x1862456998053504:12345-0@lo:400:kworker/u8:3.0 Request processed in 32us (418us total) trans 0 rc 0/0 00000100:00100000:2.0:1776178231.145787:0:14440:0:(nrs_fifo.c:241:nrs_fifo_req_stop()) NRS stop fifo request from 12345-0@lo, seq: 533 00000100:00100000:2.0:1776178231.145812:0:3356:0:(client.c:2215:ptlrpc_check_set()) Completed RPC req@000000006522caac pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_03:lustre-MDT0001-mdtlov_UUID:3356:1862456998053504:0@lo:400:kworker/u8:3.0 00000100:00100000:2.0:1776178231.145847:0:3354:0:(client.c:1733:ptlrpc_send_new_req()) Sending RPC req@00000000b7440871 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_01:lustre-MDT0001-lwp-OST0001_UUID:3354:1862456998053312:0@lo:400:kworker/u8:3.0 00000100:00100000:2.0:1776178231.145873:0:3354:0:(events.c:361:request_in_callback()) peer: 12345-0@lo (source: 12345-0@lo) 00000100:00100000:2.0:1776178231.145899:0:6257:0:(service.c:2152:ptlrpc_server_handle_req_in()) got req x1862456998053312 00000100:00100000:2.0:1776178231.145919:0:6257:0:(nrs_fifo.c:179:nrs_fifo_req_get()) NRS start fifo request from 12345-0@lo, seq: 945 00000100:00100000:2.0:1776178231.145922:0:6257:0:(service.c:2324:ptlrpc_server_handle_request()) Handling RPC req@00000000d7e84c97 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_002:lustre-MDT0001-lwp-OST0001_UUID+5:3354:x1862456998053312:12345-0@lo:400:kworker/u8:3.0 00000100:00100000:2.0:1776178231.145952:0:6257:0:(service.c:2372:ptlrpc_server_handle_request()) Handled RPC req@00000000d7e84c97 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_002:lustre-MDT0001-lwp-OST0001_UUID+5:3354:x1862456998053312:12345-0@lo:400:kworker/u8:3.0 Request processed in 28us (78us total) trans 0 rc 0/0 00000100:00100000:2.0:1776178231.145958:0:6257:0:(nrs_fifo.c:241:nrs_fifo_req_stop()) NRS stop fifo request from 12345-0@lo, seq: 945 00000100:00100000:2.0:1776178231.145980:0:3354:0:(client.c:2215:ptlrpc_check_set()) Completed RPC req@00000000b7440871 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_01:lustre-MDT0001-lwp-OST0001_UUID:3354:1862456998053312:0@lo:400:kworker/u8:3.0 00000100:00100000:3.0:1776178231.146858:0:3355:0:(events.c:361:request_in_callback()) peer: 12345-0@lo (source: 12345-0@lo) 00000100:00100000:3.0:1776178231.146898:0:20759:0:(service.c:2152:ptlrpc_server_handle_req_in()) got req x1862456998053440 00000100:00100000:3.0:1776178231.146925:0:20759:0:(nrs_fifo.c:179:nrs_fifo_req_get()) NRS start fifo request from 12345-0@lo, seq: 534 00000100:00100000:3.0:1776178231.146929:0:20759:0:(service.c:2324:ptlrpc_server_handle_request()) Handling RPC req@00000000f351bcc3 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_010:lustre-MDT0001-mdtlov_UUID+5:3355:x1862456998053440:12345-0@lo:400:kworker/u8:3.0 00000100:00100000:3.0:1776178231.146967:0:20759:0:(service.c:2372:ptlrpc_server_handle_request()) Handled RPC req@00000000f351bcc3 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_010:lustre-MDT0001-mdtlov_UUID+5:3355:x1862456998053440:12345-0@lo:400:kworker/u8:3.0 Request processed in 37us (112us total) trans 0 rc 0/0 00000100:00100000:3.0:1776178231.146973:0:20759:0:(nrs_fifo.c:241:nrs_fifo_req_stop()) NRS stop fifo request from 12345-0@lo, seq: 534 00000100:00100000:3.0:1776178231.147005:0:3355:0:(client.c:2215:ptlrpc_check_set()) Completed RPC req@000000001e11faca pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_02:lustre-MDT0001-mdtlov_UUID:3355:1862456998053440:0@lo:400:kworker/u8:3.0 00000100:00100000:1.0:1776178231.147257:0:3353:0:(client.c:2215:ptlrpc_check_set()) Completed RPC req@00000000ba9bc15d pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_00:lustre-MDT0001-lwp-OST0000_UUID:3353:1862456998053184:0@lo:400:kworker/u8:3.0 00000100:00100000:0.0:1776178231.147494:0:3352:0:(events.c:361:request_in_callback()) peer: 12345-0@lo (source: 12345-0@lo) 00000100:00100000:0.0:1776178231.147522:0:3352:0:(client.c:1733:ptlrpc_send_new_req()) Sending RPC req@00000000f4172b4c pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:lustre-MDT0000-lwp-OST0001_UUID:3352:1862456998053248:0@lo:38:kworker/u8:3.0 00000100:00100000:0.0:1776178231.147548:0:3352:0:(events.c:361:request_in_callback()) peer: 12345-0@lo (source: 12345-0@lo) 00000100:00100000:0.0:1776178231.147571:0:3352:0:(client.c:1733:ptlrpc_send_new_req()) Sending RPC req@00000000ab295186 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:lustre-MDT0001-mdtlov_UUID:3352:1862456998053376:0@lo:38:kworker/u8:3.0 00000100:00100000:0.0:1776178231.147598:0:3352:0:(events.c:361:request_in_callback()) peer: 12345-0@lo (source: 12345-0@lo) 00000100:00100000:0.0:1776178231.147611:0:3352:0:(client.c:1733:ptlrpc_send_new_req()) Sending RPC req@000000008516f059 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:lustre-MDT0000-lwp-MDT0001_UUID:3352:1862456998053568:0@lo:38:kworker/u8:3.0 00000100:00100000:0.0:1776178231.147630:0:3352:0:(events.c:361:request_in_callback()) peer: 12345-0@lo (source: 12345-0@lo) 00000100:00100000:0.0:1776178231.147658:0:6255:0:(service.c:2152:ptlrpc_server_handle_req_in()) got req x1862456998053120 00000100:00100000:0.0:1776178231.147680:0:6255:0:(service.c:2152:ptlrpc_server_handle_req_in()) got req x1862456998053248 00000100:00100000:0.0:1776178231.147705:0:6255:0:(service.c:2152:ptlrpc_server_handle_req_in()) got req x1862456998053568 00000100:00100000:0.0:1776178231.147716:0:6255:0:(nrs_fifo.c:179:nrs_fifo_req_get()) NRS start fifo request from 12345-0@lo, seq: 6645 00000100:00100000:0.0:1776178231.147719:0:6255:0:(service.c:2324:ptlrpc_server_handle_request()) Handling RPC req@000000001ea4218d pname:cluuid+ref:pid:xid:nid:opc:job mdt00_000:0+-99:3352:x1862456998053120:12345-0@lo:38:kworker/u8:3.0 00010000:02000000:0.0:1776178231.147732:0:6255:0:(ldlm_lib.c:1139:target_handle_connect()) lustre-MDT0000: Not available for connect from 0@lo (stopping) 00000100:00100000:1.0:1776178231.148366:0:6257:0:(nrs_fifo.c:179:nrs_fifo_req_get()) NRS start fifo request from 12345-0@lo, seq: 6646 00000100:00100000:1.0:1776178231.148373:0:6257:0:(service.c:2324:ptlrpc_server_handle_request()) Handling RPC req@0000000022e48f50 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_002:0+-99:3352:x1862456998053248:12345-0@lo:38:kworker/u8:3.0 00010000:02000000:1.0:1776178231.148382:0:6257:0:(ldlm_lib.c:1139:target_handle_connect()) lustre-MDT0000: Not available for connect from 0@lo (stopping) 00000100:00100000:1.0:1776178231.148425:0:6257:0:(service.c:2372:ptlrpc_server_handle_request()) Handled RPC req@0000000022e48f50 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_002:0+-99:3352:x1862456998053248:12345-0@lo:38:kworker/u8:3.0 Request processed in 52us (876us total) trans 0 rc -19/-19 00000100:00100000:1.0:1776178231.148431:0:6257:0:(nrs_fifo.c:241:nrs_fifo_req_stop()) NRS stop fifo request from 12345-0@lo, seq: 6646 00000100:00100000:1.0:1776178231.148447:0:6257:0:(nrs_fifo.c:179:nrs_fifo_req_get()) NRS start fifo request from 12345-0@lo, seq: 6647 00000100:00100000:1.0:1776178231.148450:0:6257:0:(service.c:2324:ptlrpc_server_handle_request()) Handling RPC req@00000000bd5b5f6f pname:cluuid+ref:pid:xid:nid:opc:job mdt00_002:0+-99:3352:x1862456998053568:12345-0@lo:38:kworker/u8:3.0 00010000:02000000:1.0:1776178231.148456:0:6257:0:(ldlm_lib.c:1139:target_handle_connect()) lustre-MDT0000: Not available for connect from 0@lo (stopping) 00000100:00100000:1.0:1776178231.148479:0:6257:0:(service.c:2372:ptlrpc_server_handle_request()) Handled RPC req@00000000bd5b5f6f pname:cluuid+ref:pid:xid:nid:opc:job mdt00_002:0+-99:3352:x1862456998053568:12345-0@lo:38:kworker/u8:3.0 Request processed in 27us (849us total) trans 0 rc -19/-19 00000100:00100000:1.0:1776178231.148484:0:6257:0:(nrs_fifo.c:241:nrs_fifo_req_stop()) NRS stop fifo request from 12345-0@lo, seq: 6647 00000100:00080000:1.0:1776178231.148532:0:3352:0:(import.c:86:import_set_state_nolock()) 00000000dfc8db13 lustre-MDT0000_UUID: changing import state from CONNECTING to DISCONN 00000100:00080000:1.0:1776178231.148536:0:3352:0:(import.c:1429:ptlrpc_connect_interpret()) recovery of lustre-MDT0000_UUID on 192.168.204.103@tcp failed (-19) 00000100:00100000:1.0:1776178231.148540:0:3352:0:(client.c:2215:ptlrpc_check_set()) Completed RPC req@00000000f4172b4c pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:lustre-MDT0000-lwp-OST0001_UUID:3352:1862456998053248:0@lo:38:kworker/u8:3.0 00000100:00080000:1.0:1776178231.148548:0:3352:0:(import.c:86:import_set_state_nolock()) 00000000a7c55b1f lustre-MDT0000_UUID: changing import state from CONNECTING to DISCONN 00000100:00080000:1.0:1776178231.148551:0:3352:0:(import.c:1429:ptlrpc_connect_interpret()) recovery of lustre-MDT0000_UUID on 192.168.204.103@tcp failed (-19) 00000100:00100000:1.0:1776178231.148553:0:3352:0:(client.c:2215:ptlrpc_check_set()) Completed RPC req@000000008516f059 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:lustre-MDT0000-lwp-MDT0001_UUID:3352:1862456998053568:0@lo:38:kworker/u8:3.0 00000100:00100000:2.0:1776178231.149789:0:14128:0:(service.c:2152:ptlrpc_server_handle_req_in()) got req x1862456998053376 00000100:00100000:2.0:1776178231.149848:0:14128:0:(nrs_fifo.c:179:nrs_fifo_req_get()) NRS start fifo request from 12345-0@lo, seq: 713 00000100:00100000:2.0:1776178231.149852:0:14128:0:(service.c:2324:ptlrpc_server_handle_request()) Handling RPC req@000000001f0581f9 pname:cluuid+ref:pid:xid:nid:opc:job mdt_out00_002:0+-99:3352:x1862456998053376:12345-0@lo:38:kworker/u8:3.0 00010000:02000000:2.0:1776178231.149867:0:14128:0:(ldlm_lib.c:1139:target_handle_connect()) lustre-MDT0000: Not available for connect from 0@lo (stopping) 00000100:00100000:2.0:1776178231.149938:0:14128:0:(service.c:2372:ptlrpc_server_handle_request()) Handled RPC req@000000001f0581f9 pname:cluuid+ref:pid:xid:nid:opc:job mdt_out00_002:0+-99:3352:x1862456998053376:12345-0@lo:38:kworker/u8:3.0 Request processed in 85us (2339us total) trans 0 rc -19/-19 00000100:00100000:2.0:1776178231.149945:0:14128:0:(nrs_fifo.c:241:nrs_fifo_req_stop()) NRS stop fifo request from 12345-0@lo, seq: 713 00000100:00080000:3.0:1776178231.150103:0:3352:0:(import.c:86:import_set_state_nolock()) 00000000ba4f1821 lustre-MDT0000_UUID: changing import state from CONNECTING to DISCONN 00000100:00080000:3.0:1776178231.150107:0:3352:0:(import.c:1429:ptlrpc_connect_interpret()) recovery of lustre-MDT0000_UUID on 192.168.204.103@tcp failed (-19) 00000100:00100000:3.0:1776178231.150112:0:3352:0:(client.c:2215:ptlrpc_check_set()) Completed RPC req@00000000ab295186 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:lustre-MDT0001-mdtlov_UUID:3352:1862456998053376:0@lo:38:kworker/u8:3.0 00000100:00100000:0.0:1776178231.169903:0:6255:0:(service.c:2372:ptlrpc_server_handle_request()) Handled RPC req@000000001ea4218d pname:cluuid+ref:pid:xid:nid:opc:job mdt00_000:0+-99:3352:x1862456998053120:12345-0@lo:38:kworker/u8:3.0 Request processed in 22174us (22401us total) trans 0 rc -19/-19 00000100:00100000:0.0:1776178231.169920:0:6255:0:(nrs_fifo.c:241:nrs_fifo_req_stop()) NRS stop fifo request from 12345-0@lo, seq: 6645 00000100:00080000:1.0:1776178231.172341:0:3352:0:(import.c:86:import_set_state_nolock()) 0000000047601a98 lustre-MDT0000_UUID: changing import state from CONNECTING to DISCONN 00000100:00080000:1.0:1776178231.172353:0:3352:0:(import.c:1429:ptlrpc_connect_interpret()) recovery of lustre-MDT0000_UUID on 192.168.204.103@tcp failed (-19) 00000100:00100000:1.0:1776178231.172361:0:3352:0:(client.c:2215:ptlrpc_check_set()) Completed RPC req@00000000b30b9cdd pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:lustre-MDT0000-lwp-OST0000_UUID:3352:1862456998053120:0@lo:38:kworker/u8:3.0 00000001:00100000:2.0:1776178231.653709:0:9217:0:(tgt_lastrcvd.c:541:tgt_cb_new_client()) lustre-OST0001: committing for initial connect of lustre-MDT0000-mdtlov_UUID 00000020:00000080:2.0:1776178231.653717:0:9217:0:(genops.c:985:class_export_put()) final put 00000000cdb4912e/lustre-MDT0000-mdtlov_UUID 00000001:00100000:0.0:1776178231.654170:0:8225:0:(tgt_lastrcvd.c:541:tgt_cb_new_client()) lustre-OST0000: committing for initial connect of lustre-MDT0000-mdtlov_UUID 00000020:00000080:0.0:1776178231.654183:0:8225:0:(genops.c:985:class_export_put()) final put 000000004268389b/lustre-MDT0000-mdtlov_UUID 00000001:00100000:1.0:1776178231.656121:0:18097:0:(tgt_lastrcvd.c:541:tgt_cb_new_client()) lustre-MDT0001: committing for initial connect of lustre-MDT0000-mdtlov_UUID 00000020:00000080:1.0:1776178231.656130:0:18097:0:(genops.c:985:class_export_put()) final put 000000003e84fc5b/lustre-MDT0000-mdtlov_UUID 00000020:00000080:3.0:1776178231.657521:0:36:0:(genops.c:944:class_export_destroy()) destroying export 00000000cdb4912e/lustre-MDT0000-mdtlov_UUID for lustre-OST0001 00000020:00000080:3.0:1776178231.657710:0:36:0:(genops.c:1164:obd_zombie_import_free()) destroying import 00000000b115c585 for lustre-OST0001 00000020:00000080:2.0:1776178231.661147:0:21777:0:(genops.c:944:class_export_destroy()) destroying export 000000004268389b/lustre-MDT0000-mdtlov_UUID for lustre-OST0000 00000020:00000080:2.0:1776178231.661258:0:21777:0:(genops.c:1164:obd_zombie_import_free()) destroying import 00000000353bb3e3 for lustre-OST0000 00000020:00000080:2.0:1776178231.661387:0:21777:0:(genops.c:944:class_export_destroy()) destroying export 000000003e84fc5b/lustre-MDT0000-mdtlov_UUID for lustre-MDT0001 00000020:00000080:2.0:1776178231.661423:0:21777:0:(genops.c:1164:obd_zombie_import_free()) destroying import 000000003564f86c for lustre-MDT0001 00010000:00010000:2.0:1776178231.776252:0:9224:0:(ldlm_lockd.c:2792:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:2.0:1776178232.672677:0:3354:0:(ptlrpcd.c:415:ptlrpcd_check()) transfer 1 async RPCs [0->1] 00000100:00100000:2.0:1776178232.677225:0:3354:0:(client.c:1733:ptlrpc_send_new_req()) Sending RPC req@000000008c50846d pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_01:lustre-MDT0001-mdtlov_UUID:3354:1862456998053632:0@lo:13:osp-pre-1-1.0 00000100:00100000:2.0:1776178232.677388:0:3354:0:(events.c:361:request_in_callback()) peer: 12345-0@lo (source: 12345-0@lo) 00000100:00100000:0.0:1776178232.679713:0:9554:0:(service.c:2152:ptlrpc_server_handle_req_in()) got req x1862456998053632 00000100:00100000:0.0:1776178232.679780:0:9554:0:(nrs_fifo.c:179:nrs_fifo_req_get()) NRS start fifo request from 12345-0@lo, seq: 499 00000100:00100000:0.0:1776178232.679788:0:9554:0:(service.c:2324:ptlrpc_server_handle_request()) Handling RPC req@0000000040899e7d pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_create00:lustre-MDT0001-mdtlov_UUID+5:3354:x1862456998053632:12345-0@lo:13:osp-pre-1-1.0 00002000:00000024:0.0:1776178232.679824:0:9554:0:(ofd_obd.c:740:ofd_statfs()) blocks cached 229376 granted 491746368 pending 0 free 3923689472 avail 3692167168 00000100:00100000:0.0:1776178232.681550:0:9554:0:(service.c:2372:ptlrpc_server_handle_request()) Handled RPC req@0000000040899e7d pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_create00:lustre-MDT0001-mdtlov_UUID+5:3354:x1862456998053632:12345-0@lo:13:osp-pre-1-1.0 Request processed in 1756us (4157us total) trans 0 rc 0/0 00000100:00100000:0.0:1776178232.681563:0:9554:0:(nrs_fifo.c:241:nrs_fifo_req_stop()) NRS stop fifo request from 12345-0@lo, seq: 499 00000100:00100000:2.0:1776178232.693395:0:3354:0:(client.c:2215:ptlrpc_check_set()) Completed RPC req@000000008c50846d pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_01:lustre-MDT0001-mdtlov_UUID:3354:1862456998053632:0@lo:13:osp-pre-1-1.0 00010000:00010000:2.0:1776178232.803643:0:9225:0:(ldlm_lockd.c:2792:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:2.0:1776178233.186510:0:3354:0:(client.c:1733:ptlrpc_send_new_req()) Sending RPC req@000000001111b219 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_01:lustre-MDT0001-mdtlov_UUID:3354:1862456998053696:0@lo:13:osp-pre-0-1.0 00000100:00100000:2.0:1776178233.186598:0:3354:0:(events.c:361:request_in_callback()) peer: 12345-0@lo (source: 12345-0@lo) 00000100:00100000:0.0:1776178233.186920:0:9554:0:(service.c:2152:ptlrpc_server_handle_req_in()) got req x1862456998053696 00000100:00100000:0.0:1776178233.186980:0:9554:0:(nrs_fifo.c:179:nrs_fifo_req_get()) NRS start fifo request from 12345-0@lo, seq: 500 00000100:00100000:0.0:1776178233.186988:0:9554:0:(service.c:2324:ptlrpc_server_handle_request()) Handling RPC req@000000006ce0dfc8 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_create00:lustre-MDT0001-mdtlov_UUID+5:3354:x1862456998053696:12345-0@lo:13:osp-pre-0-1.0 00002000:00000024:0.0:1776178233.187022:0:9554:0:(ofd_obd.c:740:ofd_statfs()) blocks cached 315392 granted 484883776 pending 0 free 3923689472 avail 3692167168 00000100:00100000:0.0:1776178233.187089:0:9554:0:(service.c:2372:ptlrpc_server_handle_request()) Handled RPC req@000000006ce0dfc8 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_create00:lustre-MDT0001-mdtlov_UUID+5:3354:x1862456998053696:12345-0@lo:13:osp-pre-0-1.0 Request processed in 101us (492us total) trans 0 rc 0/0 00000100:00100000:0.0:1776178233.187095:0:9554:0:(nrs_fifo.c:241:nrs_fifo_req_stop()) NRS stop fifo request from 12345-0@lo, seq: 500 00000100:00100000:2.0:1776178233.187245:0:3354:0:(client.c:2215:ptlrpc_check_set()) Completed RPC req@000000001111b219 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_01:lustre-MDT0001-mdtlov_UUID:3354:1862456998053696:0@lo:13:osp-pre-0-1.0 00010000:00010000:0.0:1776178233.824925:0:6244:0:(ldlm_lockd.c:2792:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:3.0:1776178234.658523:0:3346:0:(events.c:361:request_in_callback()) peer: 12345-192.168.204.3@tcp (source: 12345-192.168.204.3@tcp) 00000100:00100000:2.0:1776178234.660436:0:3348:0:(events.c:361:request_in_callback()) peer: 12345-192.168.204.3@tcp (source: 12345-192.168.204.3@tcp) 00000100:00100000:2.0:1776178234.660477:0:3348:0:(events.c:361:request_in_callback()) peer: 12345-192.168.204.3@tcp (source: 12345-192.168.204.3@tcp) 00000100:00100000:2.0:1776178234.660526:0:3348:0:(events.c:361:request_in_callback()) peer: 12345-192.168.204.3@tcp (source: 12345-192.168.204.3@tcp) 00000100:00100000:1.0:1776178234.660595:0:14440:0:(service.c:2152:ptlrpc_server_handle_req_in()) got req x1862456979111104 00000100:00100000:1.0:1776178234.660644:0:14440:0:(nrs_fifo.c:179:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.204.3@tcp, seq: 535 00000100:00100000:1.0:1776178234.660651:0:14440:0:(service.c:2324:ptlrpc_server_handle_request()) Handling RPC req@00000000758405de pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_004:b50963bd-8eba-4432-9a54-8888d00e3fa1+8:2272:x1862456979111104:12345-192.168.204.3@tcp:400:kworker/u8:3.0 00000100:00100000:1.0:1776178234.660729:0:14440:0:(service.c:2372:ptlrpc_server_handle_request()) Handled RPC req@00000000758405de pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_004:b50963bd-8eba-4432-9a54-8888d00e3fa1+8:2272:x1862456979111104:12345-192.168.204.3@tcp:400:kworker/u8:3.0 Request processed in 77us (251us total) trans 0 rc 0/0 00000100:00100000:1.0:1776178234.660736:0:14440:0:(nrs_fifo.c:241:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.204.3@tcp, seq: 535 00000100:00100000:3.0:1776178234.660994:0:20140:0:(service.c:2152:ptlrpc_server_handle_req_in()) got req x1862456979110912 00000100:00100000:3.0:1776178234.661034:0:20140:0:(nrs_fifo.c:179:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.204.3@tcp, seq: 167 00000100:00100000:3.0:1776178234.661039:0:20140:0:(service.c:2324:ptlrpc_server_handle_request()) Handling RPC req@00000000f1e6b6f9 pname:cluuid+ref:pid:xid:nid:opc:job ll_mgs_0002:5b5c66b1-1a2f-4651-8548-0c75b9a38a94+9:2272:x1862456979110912:12345-192.168.204.3@tcp:400:kworker/u8:3.0 00000100:00100000:3.0:1776178234.661127:0:20140:0:(service.c:2372:ptlrpc_server_handle_request()) Handled RPC req@00000000f1e6b6f9 pname:cluuid+ref:pid:xid:nid:opc:job ll_mgs_0002:5b5c66b1-1a2f-4651-8548-0c75b9a38a94+9:2272:x1862456979110912:12345-192.168.204.3@tcp:400:kworker/u8:3.0 Request processed in 76us (700us total) trans 0 rc 0/0 00000100:00100000:3.0:1776178234.661134:0:20140:0:(nrs_fifo.c:241:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.204.3@tcp, seq: 167 00000100:00100000:2.0:1776178234.670048:0:3348:0:(events.c:361:request_in_callback()) peer: 12345-192.168.204.3@tcp (source: 12345-192.168.204.3@tcp) 00000100:00100000:1.0:1776178234.670272:0:20759:0:(service.c:2152:ptlrpc_server_handle_req_in()) got req x1862456979111168 00000100:00100000:1.0:1776178234.670344:0:20759:0:(nrs_fifo.c:179:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.204.3@tcp, seq: 536 00000100:00100000:1.0:1776178234.670352:0:20759:0:(service.c:2324:ptlrpc_server_handle_request()) Handling RPC req@0000000016c7a5ff pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_010:b50963bd-8eba-4432-9a54-8888d00e3fa1+9:2272:x1862456979111168:12345-192.168.204.3@tcp:400:kworker/u8:3.0 00000100:00100000:1.0:1776178234.670443:0:20759:0:(service.c:2372:ptlrpc_server_handle_request()) Handled RPC req@0000000016c7a5ff pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_010:b50963bd-8eba-4432-9a54-8888d00e3fa1+9:2272:x1862456979111168:12345-192.168.204.3@tcp:400:kworker/u8:3.0 Request processed in 92us (405us total) trans 0 rc 0/0 00000100:00100000:1.0:1776178234.670452:0:20759:0:(nrs_fifo.c:241:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.204.3@tcp, seq: 536 00000100:00100000:0.0:1776178234.670727:0:6255:0:(service.c:2152:ptlrpc_server_handle_req_in()) got req x1862456979110976 00000100:00100000:3.0:1776178234.671022:0:6256:0:(service.c:2152:ptlrpc_server_handle_req_in()) got req x1862456979111040 00000100:00100000:1.0:1776178234.671084:0:11281:0:(nrs_fifo.c:179:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.204.3@tcp, seq: 946 00000100:00100000:1.0:1776178234.671089:0:11281:0:(service.c:2324:ptlrpc_server_handle_request()) Handling RPC req@0000000053c46e8d pname:cluuid+ref:pid:xid:nid:opc:job mdt00_004:b50963bd-8eba-4432-9a54-8888d00e3fa1+6:2272:x1862456979111040:12345-192.168.204.3@tcp:400:kworker/u8:3.0 00000100:00100000:1.0:1776178234.671140:0:11281:0:(service.c:2372:ptlrpc_server_handle_request()) Handled RPC req@0000000053c46e8d pname:cluuid+ref:pid:xid:nid:opc:job mdt00_004:b50963bd-8eba-4432-9a54-8888d00e3fa1+6:2272:x1862456979111040:12345-192.168.204.3@tcp:400:kworker/u8:3.0 Request processed in 49us (10614us total) trans 0 rc 0/0 00000100:00100000:1.0:1776178234.671147:0:11281:0:(nrs_fifo.c:241:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.204.3@tcp, seq: 946 00000100:00100000:1.0:1776178234.671162:0:11281:0:(nrs_fifo.c:179:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.204.3@tcp, seq: 6648 00000100:00100000:1.0:1776178234.671165:0:11281:0:(service.c:2324:ptlrpc_server_handle_request()) Handling RPC req@0000000018bd7474 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_004:0+-99:2269:x1862456979110976:12345-192.168.204.3@tcp:38:kworker/u8:3.0 00010000:02000000:1.0:1776178234.671180:0:11281:0:(ldlm_lib.c:1139:target_handle_connect()) lustre-MDT0000: Not available for connect from 192.168.204.3@tcp (stopping) 00000100:00100000:1.0:1776178234.681232:0:11281:0:(service.c:2372:ptlrpc_server_handle_request()) Handled RPC req@0000000018bd7474 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_004:0+-99:2269:x1862456979110976:12345-192.168.204.3@tcp:38:kworker/u8:3.0 Request processed in 10044us (22698us total) trans 0 rc -19/-19 00000100:00100000:1.0:1776178234.681249:0:11281:0:(nrs_fifo.c:241:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.204.3@tcp, seq: 6648 00010000:00010000:2.0:1776178234.848378:0:6245:0:(ldlm_lockd.c:2792:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00010000:00010000:2.0:1776178235.872639:0:9224:0:(ldlm_lockd.c:2792:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00080000:2.0:1776178236.256834:0:21777:0:(pinger.c:251:ptlrpc_pinger_process_import()) lustre-MDT0000-lwp-OST0000_UUID->lustre-MDT0000_UUID: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00080000:2.0:1776178236.256842:0:21777:0:(recover.c:57:ptlrpc_initiate_recovery()) lustre-MDT0000_UUID: starting recovery 00000100:00080000:2.0:1776178236.256846:0:21777:0:(import.c:86:import_set_state_nolock()) 0000000047601a98 lustre-MDT0000_UUID: changing import state from DISCONN to CONNECTING 00000100:00080000:2.0:1776178236.256857:0:21777:0:(import.c:534:import_select_connection()) lustre-MDT0000-lwp-OST0000: connect to NID 0@lo last attempt 938 00000100:00080000:2.0:1776178236.256862:0:21777:0:(import.c:581:import_select_connection()) lustre-MDT0000-lwp-OST0000: tried all connections, increasing latency to 11s 00000100:00080000:2.0:1776178236.256870:0:21777:0:(import.c:615:import_select_connection()) lustre-MDT0000-lwp-OST0000: import 0000000047601a98 using connection 192.168.204.103@tcp/0@lo 00000100:00100000:2.0:1776178236.256905:0:21777:0:(import.c:807:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5) req@00000000b7440871 x1862456998053824/t0(0) o38->lustre-MDT0000-lwp-OST0000@0@lo:12/10 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' 00000100:00080000:2.0:1776178236.256934:0:21777:0:(pinger.c:251:ptlrpc_pinger_process_import()) lustre-MDT0000-lwp-OST0001_UUID->lustre-MDT0000_UUID: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00080000:2.0:1776178236.256936:0:21777:0:(recover.c:57:ptlrpc_initiate_recovery()) lustre-MDT0000_UUID: starting recovery 00000100:00080000:2.0:1776178236.256938:0:21777:0:(import.c:86:import_set_state_nolock()) 00000000dfc8db13 lustre-MDT0000_UUID: changing import state from DISCONN to CONNECTING 00000100:00080000:2.0:1776178236.256941:0:21777:0:(import.c:534:import_select_connection()) lustre-MDT0000-lwp-OST0001: connect to NID 0@lo last attempt 938 00000100:00080000:2.0:1776178236.256943:0:21777:0:(import.c:581:import_select_connection()) lustre-MDT0000-lwp-OST0001: tried all connections, increasing latency to 11s 00000100:00080000:2.0:1776178236.256946:0:21777:0:(import.c:615:import_select_connection()) lustre-MDT0000-lwp-OST0001: import 00000000dfc8db13 using connection 192.168.204.103@tcp/0@lo 00000100:00100000:2.0:1776178236.256955:0:21777:0:(import.c:807:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5) req@00000000ab295186 x1862456998053952/t0(0) o38->lustre-MDT0000-lwp-OST0001@0@lo:12/10 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' 00000100:00080000:2.0:1776178236.256983:0:21777:0:(pinger.c:251:ptlrpc_pinger_process_import()) lustre-MDT0001-mdtlov_UUID->lustre-MDT0000_UUID: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00080000:2.0:1776178236.256987:0:21777:0:(recover.c:57:ptlrpc_initiate_recovery()) lustre-MDT0000_UUID: starting recovery 00000100:00080000:2.0:1776178236.256988:0:21777:0:(import.c:86:import_set_state_nolock()) 00000000ba4f1821 lustre-MDT0000_UUID: changing import state from DISCONN to CONNECTING 00000100:00080000:2.0:1776178236.256991:0:21777:0:(import.c:534:import_select_connection()) lustre-MDT0000-osp-MDT0001: connect to NID 0@lo last attempt 938 00000100:00080000:2.0:1776178236.256993:0:21777:0:(import.c:581:import_select_connection()) lustre-MDT0000-osp-MDT0001: tried all connections, increasing latency to 11s 00000100:00080000:2.0:1776178236.256996:0:21777:0:(import.c:615:import_select_connection()) lustre-MDT0000-osp-MDT0001: import 00000000ba4f1821 using connection 192.168.204.103@tcp/0@lo 00000100:00100000:2.0:1776178236.257007:0:21777:0:(import.c:807:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5) req@00000000ba9bc15d x1862456998054080/t0(0) o38->lustre-MDT0000-osp-MDT0001@0@lo:24/4 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' 00000100:00080000:2.0:1776178236.257026:0:21777:0:(pinger.c:251:ptlrpc_pinger_process_import()) lustre-MDT0000-lwp-MDT0001_UUID->lustre-MDT0000_UUID: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00080000:2.0:1776178236.257029:0:21777:0:(recover.c:57:ptlrpc_initiate_recovery()) lustre-MDT0000_UUID: starting recovery 00000100:00080000:2.0:1776178236.257030:0:21777:0:(import.c:86:import_set_state_nolock()) 00000000a7c55b1f lustre-MDT0000_UUID: changing import state from DISCONN to CONNECTING 00000100:00080000:2.0:1776178236.257033:0:21777:0:(import.c:534:import_select_connection()) lustre-MDT0000-lwp-MDT0001: connect to NID 0@lo last attempt 938 00000100:00080000:2.0:1776178236.257035:0:21777:0:(import.c:581:import_select_connection()) lustre-MDT0000-lwp-MDT0001: tried all connections, increasing latency to 11s 00000100:00080000:2.0:1776178236.257037:0:21777:0:(import.c:615:import_select_connection()) lustre-MDT0000-lwp-MDT0001: import 00000000a7c55b1f using connection 192.168.204.103@tcp/0@lo 00000100:00100000:2.0:1776178236.257045:0:21777:0:(import.c:807:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5) req@0000000092028508 x1862456998054272/t0(0) o38->lustre-MDT0000-lwp-MDT0001@0@lo:12/10 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' 00000100:00100000:2.0:1776178236.257107:0:3353:0:(client.c:1733:ptlrpc_send_new_req()) Sending RPC req@00000000f4172b4c pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_00:lustre-MDT0001-lwp-OST0001_UUID:3353:1862456998054016:0@lo:400:kworker/u8:3.0 00000100:00100000:2.0:1776178236.257186:0:3353:0:(events.c:361:request_in_callback()) peer: 12345-0@lo (source: 12345-0@lo) 00000100:00100000:2.0:1776178236.257239:0:3353:0:(ptlrpcd.c:415:ptlrpcd_check()) transfer 1 async RPCs [1->0] 00000100:00100000:2.0:1776178236.257244:0:3353:0:(client.c:1733:ptlrpc_send_new_req()) Sending RPC req@000000003095df64 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_00:lustre-MDT0001-mdtlov_UUID:3353:1862456998054144:0@lo:400:kworker/u8:3.0 00000100:00100000:2.0:1776178236.257270:0:3353:0:(events.c:361:request_in_callback()) peer: 12345-0@lo (source: 12345-0@lo) 00000100:00100000:2.0:1776178236.257312:0:14440:0:(service.c:2152:ptlrpc_server_handle_req_in()) got req x1862456998054144 00000100:00100000:2.0:1776178236.257365:0:14440:0:(nrs_fifo.c:179:nrs_fifo_req_get()) NRS start fifo request from 12345-0@lo, seq: 537 00000100:00100000:2.0:1776178236.257372:0:14440:0:(service.c:2324:ptlrpc_server_handle_request()) Handling RPC req@0000000090a7aa50 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_004:lustre-MDT0001-mdtlov_UUID+5:3353:x1862456998054144:12345-0@lo:400:kworker/u8:3.0 00000100:00100000:2.0:1776178236.257430:0:14440:0:(service.c:2372:ptlrpc_server_handle_request()) Handled RPC req@0000000090a7aa50 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_004:lustre-MDT0001-mdtlov_UUID+5:3353:x1862456998054144:12345-0@lo:400:kworker/u8:3.0 Request processed in 57us (159us total) trans 0 rc 0/0 00000100:00100000:2.0:1776178236.257439:0:14440:0:(nrs_fifo.c:241:nrs_fifo_req_stop()) NRS stop fifo request from 12345-0@lo, seq: 537 00000100:00100000:2.0:1776178236.257493:0:3353:0:(client.c:2215:ptlrpc_check_set()) Completed RPC req@000000003095df64 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_00:lustre-MDT0001-mdtlov_UUID:3353:1862456998054144:0@lo:400:kworker/u8:3.0 00000100:00100000:1.0:1776178236.257609:0:3352:0:(client.c:1733:ptlrpc_send_new_req()) Sending RPC req@00000000b7440871 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:lustre-MDT0000-lwp-OST0000_UUID:3352:1862456998053824:0@lo:38:kworker/u8:3.0 00000100:00100000:1.0:1776178236.257643:0:3352:0:(events.c:361:request_in_callback()) peer: 12345-0@lo (source: 12345-0@lo) 00000100:00100000:1.0:1776178236.257666:0:3352:0:(client.c:1733:ptlrpc_send_new_req()) Sending RPC req@00000000ab295186 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:lustre-MDT0000-lwp-OST0001_UUID:3352:1862456998053952:0@lo:38:kworker/u8:3.0 00000100:00100000:1.0:1776178236.257687:0:3352:0:(events.c:361:request_in_callback()) peer: 12345-0@lo (source: 12345-0@lo) 00000100:00100000:1.0:1776178236.257700:0:3352:0:(client.c:1733:ptlrpc_send_new_req()) Sending RPC req@00000000ba9bc15d pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:lustre-MDT0001-mdtlov_UUID:3352:1862456998054080:0@lo:38:kworker/u8:3.0 00000100:00100000:1.0:1776178236.257736:0:3352:0:(events.c:361:request_in_callback()) peer: 12345-0@lo (source: 12345-0@lo) 00000100:00100000:1.0:1776178236.257753:0:3352:0:(client.c:1733:ptlrpc_send_new_req()) Sending RPC req@0000000092028508 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:lustre-MDT0000-lwp-MDT0001_UUID:3352:1862456998054272:0@lo:38:kworker/u8:3.0 00000100:00100000:1.0:1776178236.257772:0:3352:0:(events.c:361:request_in_callback()) peer: 12345-0@lo (source: 12345-0@lo) 00000100:00100000:1.0:1776178236.257801:0:11281:0:(service.c:2152:ptlrpc_server_handle_req_in()) got req x1862456998054016 00000100:00100000:1.0:1776178236.257831:0:11281:0:(service.c:2152:ptlrpc_server_handle_req_in()) got req x1862456998053824 00000100:00100000:1.0:1776178236.257841:0:11281:0:(service.c:2152:ptlrpc_server_handle_req_in()) got req x1862456998053952 00000100:00100000:1.0:1776178236.257848:0:11281:0:(service.c:2152:ptlrpc_server_handle_req_in()) got req x1862456998054272 00000100:00100000:1.0:1776178236.257857:0:11281:0:(nrs_fifo.c:179:nrs_fifo_req_get()) NRS start fifo request from 12345-0@lo, seq: 947 00000100:00100000:1.0:1776178236.257860:0:11281:0:(service.c:2324:ptlrpc_server_handle_request()) Handling RPC req@000000006b0ae3d8 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_004:lustre-MDT0001-lwp-OST0001_UUID+5:3353:x1862456998054016:12345-0@lo:400:kworker/u8:3.0 00000100:00100000:1.0:1776178236.257899:0:11281:0:(service.c:2372:ptlrpc_server_handle_request()) Handled RPC req@000000006b0ae3d8 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_004:lustre-MDT0001-lwp-OST0001_UUID+5:3353:x1862456998054016:12345-0@lo:400:kworker/u8:3.0 Request processed in 38us (714us total) trans 0 rc 0/0 00000100:00100000:1.0:1776178236.257906:0:11281:0:(nrs_fifo.c:241:nrs_fifo_req_stop()) NRS stop fifo request from 12345-0@lo, seq: 947 00000100:00100000:1.0:1776178236.257918:0:11281:0:(nrs_fifo.c:179:nrs_fifo_req_get()) NRS start fifo request from 12345-0@lo, seq: 6649 00000100:00100000:1.0:1776178236.257922:0:11281:0:(service.c:2324:ptlrpc_server_handle_request()) Handling RPC req@000000008516f059 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_004:0+-99:3352:x1862456998053824:12345-0@lo:38:kworker/u8:3.0 00010000:02000000:1.0:1776178236.257936:0:11281:0:(ldlm_lib.c:1139:target_handle_connect()) lustre-MDT0000: Not available for connect from 0@lo (stopping) 00000100:00100000:1.0:1776178236.257985:0:11281:0:(service.c:2372:ptlrpc_server_handle_request()) Handled RPC req@000000008516f059 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_004:0+-99:3352:x1862456998053824:12345-0@lo:38:kworker/u8:3.0 Request processed in 62us (341us total) trans 0 rc -19/-19 00000100:00100000:1.0:1776178236.257991:0:11281:0:(nrs_fifo.c:241:nrs_fifo_req_stop()) NRS stop fifo request from 12345-0@lo, seq: 6649 00000100:00100000:1.0:1776178236.258002:0:11281:0:(nrs_fifo.c:179:nrs_fifo_req_get()) NRS start fifo request from 12345-0@lo, seq: 6650 00000100:00100000:1.0:1776178236.258004:0:11281:0:(service.c:2324:ptlrpc_server_handle_request()) Handling RPC req@000000006522caac pname:cluuid+ref:pid:xid:nid:opc:job mdt00_004:0+-99:3352:x1862456998053952:12345-0@lo:38:kworker/u8:3.0 00010000:02000000:1.0:1776178236.258010:0:11281:0:(ldlm_lib.c:1139:target_handle_connect()) lustre-MDT0000: Not available for connect from 0@lo (stopping) 00000100:00100000:1.0:1776178236.258030:0:11281:0:(service.c:2372:ptlrpc_server_handle_request()) Handled RPC req@000000006522caac pname:cluuid+ref:pid:xid:nid:opc:job mdt00_004:0+-99:3352:x1862456998053952:12345-0@lo:38:kworker/u8:3.0 Request processed in 25us (343us total) trans 0 rc -19/-19 00000100:00100000:1.0:1776178236.258036:0:11281:0:(nrs_fifo.c:241:nrs_fifo_req_stop()) NRS stop fifo request from 12345-0@lo, seq: 6650 00000100:00100000:1.0:1776178236.258046:0:11281:0:(nrs_fifo.c:179:nrs_fifo_req_get()) NRS start fifo request from 12345-0@lo, seq: 6651 00000100:00100000:1.0:1776178236.258049:0:11281:0:(service.c:2324:ptlrpc_server_handle_request()) Handling RPC req@00000000b9a3856b pname:cluuid+ref:pid:xid:nid:opc:job mdt00_004:0+-99:3352:x1862456998054272:12345-0@lo:38:kworker/u8:3.0 00010000:02000000:1.0:1776178236.258054:0:11281:0:(ldlm_lib.c:1139:target_handle_connect()) lustre-MDT0000: Not available for connect from 0@lo (stopping) 00000100:00100000:1.0:1776178236.258113:0:11281:0:(service.c:2372:ptlrpc_server_handle_request()) Handled RPC req@00000000b9a3856b pname:cluuid+ref:pid:xid:nid:opc:job mdt00_004:0+-99:3352:x1862456998054272:12345-0@lo:38:kworker/u8:3.0 Request processed in 62us (339us total) trans 0 rc -19/-19 00000100:00100000:1.0:1776178236.258118:0:11281:0:(nrs_fifo.c:241:nrs_fifo_req_stop()) NRS stop fifo request from 12345-0@lo, seq: 6651 00000100:00100000:1.0:1776178236.258142:0:14128:0:(service.c:2152:ptlrpc_server_handle_req_in()) got req x1862456998054080 00000100:00100000:1.0:1776178236.258159:0:14128:0:(nrs_fifo.c:179:nrs_fifo_req_get()) NRS start fifo request from 12345-0@lo, seq: 714 00000100:00100000:1.0:1776178236.258161:0:14128:0:(service.c:2324:ptlrpc_server_handle_request()) Handling RPC req@00000000ee45921e pname:cluuid+ref:pid:xid:nid:opc:job mdt_out00_002:0+-99:3352:x1862456998054080:12345-0@lo:38:kworker/u8:3.0 00010000:02000000:1.0:1776178236.258167:0:14128:0:(ldlm_lib.c:1139:target_handle_connect()) lustre-MDT0000: Not available for connect from 0@lo (stopping) 00000100:00100000:1.0:1776178236.258186:0:14128:0:(service.c:2372:ptlrpc_server_handle_request()) Handled RPC req@00000000ee45921e pname:cluuid+ref:pid:xid:nid:opc:job mdt_out00_002:0+-99:3352:x1862456998054080:12345-0@lo:38:kworker/u8:3.0 Request processed in 24us (451us total) trans 0 rc -19/-19 00000100:00100000:1.0:1776178236.258191:0:14128:0:(nrs_fifo.c:241:nrs_fifo_req_stop()) NRS stop fifo request from 12345-0@lo, seq: 714 00000100:00080000:1.0:1776178236.258222:0:3352:0:(import.c:86:import_set_state_nolock()) 0000000047601a98 lustre-MDT0000_UUID: changing import state from CONNECTING to DISCONN 00000100:00080000:1.0:1776178236.258226:0:3352:0:(import.c:1429:ptlrpc_connect_interpret()) recovery of lustre-MDT0000_UUID on 192.168.204.103@tcp failed (-19) 00000100:00100000:1.0:1776178236.258229:0:3352:0:(client.c:2215:ptlrpc_check_set()) Completed RPC req@00000000b7440871 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:lustre-MDT0000-lwp-OST0000_UUID:3352:1862456998053824:0@lo:38:kworker/u8:3.0 00000100:00080000:1.0:1776178236.258237:0:3352:0:(import.c:86:import_set_state_nolock()) 00000000dfc8db13 lustre-MDT0000_UUID: changing import state from CONNECTING to DISCONN 00000100:00080000:1.0:1776178236.258239:0:3352:0:(import.c:1429:ptlrpc_connect_interpret()) recovery of lustre-MDT0000_UUID on 192.168.204.103@tcp failed (-19) 00000100:00100000:1.0:1776178236.258241:0:3352:0:(client.c:2215:ptlrpc_check_set()) Completed RPC req@00000000ab295186 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:lustre-MDT0000-lwp-OST0001_UUID:3352:1862456998053952:0@lo:38:kworker/u8:3.0 00000100:00080000:1.0:1776178236.258246:0:3352:0:(import.c:86:import_set_state_nolock()) 00000000ba4f1821 lustre-MDT0000_UUID: changing import state from CONNECTING to DISCONN 00000100:00080000:1.0:1776178236.258248:0:3352:0:(import.c:1429:ptlrpc_connect_interpret()) recovery of lustre-MDT0000_UUID on 192.168.204.103@tcp failed (-19) 00000100:00100000:1.0:1776178236.258251:0:3352:0:(client.c:2215:ptlrpc_check_set()) Completed RPC req@00000000ba9bc15d pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:lustre-MDT0001-mdtlov_UUID:3352:1862456998054080:0@lo:38:kworker/u8:3.0 00000100:00080000:1.0:1776178236.258256:0:3352:0:(import.c:86:import_set_state_nolock()) 00000000a7c55b1f lustre-MDT0000_UUID: changing import state from CONNECTING to DISCONN 00000100:00080000:1.0:1776178236.258259:0:3352:0:(import.c:1429:ptlrpc_connect_interpret()) recovery of lustre-MDT0000_UUID on 192.168.204.103@tcp failed (-19) 00000100:00100000:1.0:1776178236.258260:0:3352:0:(client.c:2215:ptlrpc_check_set()) Completed RPC req@0000000092028508 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:lustre-MDT0000-lwp-MDT0001_UUID:3352:1862456998054272:0@lo:38:kworker/u8:3.0 00000100:00100000:1.0:1776178236.258291:0:3353:0:(client.c:2215:ptlrpc_check_set()) Completed RPC req@00000000f4172b4c pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_00:lustre-MDT0001-lwp-OST0001_UUID:3353:1862456998054016:0@lo:400:kworker/u8:3.0 00000100:00100000:0.0:1776178236.259356:0:3356:0:(client.c:1733:ptlrpc_send_new_req()) Sending RPC req@00000000b30b9cdd pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_03:lustre-MDT0001-lwp-OST0000_UUID:3356:1862456998053888:0@lo:400:kworker/u8:3.0 00000100:00100000:0.0:1776178236.259434:0:3356:0:(events.c:361:request_in_callback()) peer: 12345-0@lo (source: 12345-0@lo) 00000100:00100000:0.0:1776178236.259472:0:3356:0:(ptlrpcd.c:415:ptlrpcd_check()) transfer 2 async RPCs [2->3] 00000100:00100000:0.0:1776178236.259478:0:3356:0:(client.c:1733:ptlrpc_send_new_req()) Sending RPC req@00000000eeeaf4ad pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_03:ac61f839-0e8e-4cac-9c59-08bcfbbad330:3356:1862456998053760:0@lo:400:kworker/u8:3.0 00000100:00100000:0.0:1776178236.259510:0:3356:0:(events.c:361:request_in_callback()) peer: 12345-0@lo (source: 12345-0@lo) 00000100:00100000:0.0:1776178236.259532:0:3356:0:(client.c:1733:ptlrpc_send_new_req()) Sending RPC req@00000000a8c8cb2d pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_03:lustre-MDT0001-mdtlov_UUID:3356:1862456998054208:0@lo:400:kworker/u8:3.0 00000100:00100000:0.0:1776178236.259555:0:3356:0:(events.c:361:request_in_callback()) peer: 12345-0@lo (source: 12345-0@lo) 00000100:00100000:0.0:1776178236.259588:0:6255:0:(service.c:2152:ptlrpc_server_handle_req_in()) got req x1862456998053888 00000100:00100000:0.0:1776178236.259625:0:6255:0:(nrs_fifo.c:179:nrs_fifo_req_get()) NRS start fifo request from 12345-0@lo, seq: 948 00000100:00100000:0.0:1776178236.259629:0:6255:0:(service.c:2324:ptlrpc_server_handle_request()) Handling RPC req@0000000043589f56 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_000:lustre-MDT0001-lwp-OST0000_UUID+5:3356:x1862456998053888:12345-0@lo:400:kworker/u8:3.0 00000100:00100000:0.0:1776178236.259671:0:6255:0:(service.c:2372:ptlrpc_server_handle_request()) Handled RPC req@0000000043589f56 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_000:lustre-MDT0001-lwp-OST0000_UUID+5:3356:x1862456998053888:12345-0@lo:400:kworker/u8:3.0 Request processed in 41us (238us total) trans 0 rc 0/0 00000100:00100000:0.0:1776178236.259677:0:6255:0:(nrs_fifo.c:241:nrs_fifo_req_stop()) NRS stop fifo request from 12345-0@lo, seq: 948 00000100:00100000:0.0:1776178236.259700:0:20137:0:(service.c:2152:ptlrpc_server_handle_req_in()) got req x1862456998053760 00000100:00100000:0.0:1776178236.259724:0:20137:0:(nrs_fifo.c:179:nrs_fifo_req_get()) NRS start fifo request from 12345-0@lo, seq: 168 00000100:00100000:0.0:1776178236.259727:0:20137:0:(service.c:2324:ptlrpc_server_handle_request()) Handling RPC req@000000001091921c pname:cluuid+ref:pid:xid:nid:opc:job ll_mgs_0000:ac61f839-0e8e-4cac-9c59-08bcfbbad330+18:3356:x1862456998053760:12345-0@lo:400:kworker/u8:3.0 00000100:00100000:0.0:1776178236.259757:0:20137:0:(service.c:2372:ptlrpc_server_handle_request()) Handled RPC req@000000001091921c pname:cluuid+ref:pid:xid:nid:opc:job ll_mgs_0000:ac61f839-0e8e-4cac-9c59-08bcfbbad330+18:3356:x1862456998053760:12345-0@lo:400:kworker/u8:3.0 Request processed in 29us (247us total) trans 0 rc 0/0 00000100:00100000:0.0:1776178236.259763:0:20137:0:(nrs_fifo.c:241:nrs_fifo_req_stop()) NRS stop fifo request from 12345-0@lo, seq: 168 00000100:00100000:0.0:1776178236.259793:0:20759:0:(service.c:2152:ptlrpc_server_handle_req_in()) got req x1862456998054208 00000100:00100000:0.0:1776178236.259821:0:20759:0:(nrs_fifo.c:179:nrs_fifo_req_get()) NRS start fifo request from 12345-0@lo, seq: 538 00000100:00100000:0.0:1776178236.259825:0:20759:0:(service.c:2324:ptlrpc_server_handle_request()) Handling RPC req@000000001111b219 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_010:lustre-MDT0001-mdtlov_UUID+5:3356:x1862456998054208:12345-0@lo:400:kworker/u8:3.0 00000100:00100000:0.0:1776178236.259852:0:20759:0:(service.c:2372:ptlrpc_server_handle_request()) Handled RPC req@000000001111b219 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_010:lustre-MDT0001-mdtlov_UUID+5:3356:x1862456998054208:12345-0@lo:400:kworker/u8:3.0 Request processed in 27us (297us total) trans 0 rc 0/0 00000100:00100000:0.0:1776178236.259857:0:20759:0:(nrs_fifo.c:241:nrs_fifo_req_stop()) NRS stop fifo request from 12345-0@lo, seq: 538 00000100:00100000:0.0:1776178236.259898:0:3356:0:(client.c:2215:ptlrpc_check_set()) Completed RPC req@00000000eeeaf4ad pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_03:ac61f839-0e8e-4cac-9c59-08bcfbbad330:3356:1862456998053760:0@lo:400:kworker/u8:3.0 00000100:00100000:0.0:1776178236.259909:0:3356:0:(client.c:2215:ptlrpc_check_set()) Completed RPC req@00000000a8c8cb2d pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_03:lustre-MDT0001-mdtlov_UUID:3356:1862456998054208:0@lo:400:kworker/u8:3.0 00000100:00100000:0.0:1776178236.259917:0:3356:0:(client.c:2215:ptlrpc_check_set()) Completed RPC req@00000000b30b9cdd pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_03:lustre-MDT0001-lwp-OST0000_UUID:3356:1862456998053888:0@lo:400:kworker/u8:3.0 00010000:00010000:2.0:1776178236.900364:0:9225:0:(ldlm_lockd.c:2792:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:0.0:1776178237.792927:0:3356:0:(client.c:1733:ptlrpc_send_new_req()) Sending RPC req@000000001da3f43e pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_03:lustre-MDT0001-mdtlov_UUID:3356:1862456998054336:0@lo:13:osp-pre-1-1.0 00000100:00100000:0.0:1776178237.793033:0:3356:0:(events.c:361:request_in_callback()) peer: 12345-0@lo (source: 12345-0@lo) 00000100:00100000:3.0:1776178237.793341:0:8231:0:(service.c:2152:ptlrpc_server_handle_req_in()) got req x1862456998054336 00000100:00100000:3.0:1776178237.793391:0:8231:0:(nrs_fifo.c:179:nrs_fifo_req_get()) NRS start fifo request from 12345-0@lo, seq: 501 00000100:00100000:3.0:1776178237.793398:0:8231:0:(service.c:2324:ptlrpc_server_handle_request()) Handling RPC req@000000004a3d9287 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_create00:lustre-MDT0001-mdtlov_UUID+5:3356:x1862456998054336:12345-0@lo:13:osp-pre-1-1.0 00002000:00000024:3.0:1776178237.793432:0:8231:0:(ofd_obd.c:740:ofd_statfs()) blocks cached 229376 granted 491746368 pending 0 free 3923689472 avail 3692167168 00000100:00100000:3.0:1776178237.793491:0:8231:0:(service.c:2372:ptlrpc_server_handle_request()) Handled RPC req@000000004a3d9287 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_create00:lustre-MDT0001-mdtlov_UUID+5:3356:x1862456998054336:12345-0@lo:13:osp-pre-1-1.0 Request processed in 92us (460us total) trans 0 rc 0/0 00000100:00100000:3.0:1776178237.793498:0:8231:0:(nrs_fifo.c:241:nrs_fifo_req_stop()) NRS stop fifo request from 12345-0@lo, seq: 501 00000100:00100000:1.0:1776178237.793711:0:3356:0:(client.c:2215:ptlrpc_check_set()) Completed RPC req@000000001da3f43e pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_03:lustre-MDT0001-mdtlov_UUID:3356:1862456998054336:0@lo:13:osp-pre-1-1.0 00010000:00010000:0.0:1776178237.920219:0:6244:0:(ldlm_lockd.c:2792:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:1.0:1776178238.307688:0:3353:0:(client.c:1733:ptlrpc_send_new_req()) Sending RPC req@00000000a49c1305 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_00:lustre-MDT0001-mdtlov_UUID:3353:1862456998054400:0@lo:13:osp-pre-0-1.0 00000100:00100000:1.0:1776178238.307770:0:3353:0:(events.c:361:request_in_callback()) peer: 12345-0@lo (source: 12345-0@lo) 00000100:00100000:0.0:1776178238.308551:0:9554:0:(service.c:2152:ptlrpc_server_handle_req_in()) got req x1862456998054400 00000100:00100000:0.0:1776178238.308621:0:9554:0:(nrs_fifo.c:179:nrs_fifo_req_get()) NRS start fifo request from 12345-0@lo, seq: 502 00000100:00100000:0.0:1776178238.308628:0:9554:0:(service.c:2324:ptlrpc_server_handle_request()) Handling RPC req@000000001f0b1867 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_create00:lustre-MDT0001-mdtlov_UUID+5:3353:x1862456998054400:12345-0@lo:13:osp-pre-0-1.0 00002000:00000024:0.0:1776178238.308660:0:9554:0:(ofd_obd.c:740:ofd_statfs()) blocks cached 315392 granted 484883776 pending 0 free 3923689472 avail 3692167168 00000100:00100000:0.0:1776178238.308727:0:9554:0:(service.c:2372:ptlrpc_server_handle_request()) Handled RPC req@000000001f0b1867 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_create00:lustre-MDT0001-mdtlov_UUID+5:3353:x1862456998054400:12345-0@lo:13:osp-pre-0-1.0 Request processed in 99us (958us total) trans 0 rc 0/0 00000100:00100000:0.0:1776178238.308735:0:9554:0:(nrs_fifo.c:241:nrs_fifo_req_stop()) NRS stop fifo request from 12345-0@lo, seq: 502 00000100:00100000:1.0:1776178238.309285:0:3353:0:(client.c:2215:ptlrpc_check_set()) Completed RPC req@00000000a49c1305 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_00:lustre-MDT0001-mdtlov_UUID:3353:1862456998054400:0@lo:13:osp-pre-0-1.0 00010000:00010000:2.0:1776178238.944352:0:6245:0:(ldlm_lockd.c:2792:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:1.0:1776178239.787253:0:3346:0:(events.c:361:request_in_callback()) peer: 12345-192.168.204.3@tcp (source: 12345-192.168.204.3@tcp) 00000100:00100000:1.0:1776178239.787319:0:3346:0:(events.c:361:request_in_callback()) peer: 12345-192.168.204.3@tcp (source: 12345-192.168.204.3@tcp) 00000100:00100000:1.0:1776178239.787370:0:3346:0:(events.c:361:request_in_callback()) peer: 12345-192.168.204.3@tcp (source: 12345-192.168.204.3@tcp) 00000100:00100000:1.0:1776178239.787412:0:3346:0:(events.c:361:request_in_callback()) peer: 12345-192.168.204.3@tcp (source: 12345-192.168.204.3@tcp) 00000100:00100000:1.0:1776178239.787460:0:3346:0:(events.c:361:request_in_callback()) peer: 12345-192.168.204.3@tcp (source: 12345-192.168.204.3@tcp) 00000100:00100000:1.0:1776178239.787511:0:6257:0:(service.c:2152:ptlrpc_server_handle_req_in()) got req x1862456979111360 00000100:00100000:1.0:1776178239.787584:0:6257:0:(service.c:2152:ptlrpc_server_handle_req_in()) got req x1862456979111296 00000100:00100000:1.0:1776178239.787604:0:6257:0:(nrs_fifo.c:179:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.204.3@tcp, seq: 949 00000100:00100000:1.0:1776178239.787613:0:6257:0:(service.c:2324:ptlrpc_server_handle_request()) Handling RPC req@000000005accf721 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_002:b50963bd-8eba-4432-9a54-8888d00e3fa1+6:2273:x1862456979111360:12345-192.168.204.3@tcp:400:kworker/u8:4.0 00000100:00100000:1.0:1776178239.787710:0:6257:0:(service.c:2372:ptlrpc_server_handle_request()) Handled RPC req@000000005accf721 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_002:b50963bd-8eba-4432-9a54-8888d00e3fa1+6:2273:x1862456979111360:12345-192.168.204.3@tcp:400:kworker/u8:4.0 Request processed in 98us (472us total) trans 0 rc 0/0 00000100:00100000:1.0:1776178239.787720:0:6257:0:(nrs_fifo.c:241:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.204.3@tcp, seq: 949 00000100:00100000:1.0:1776178239.787740:0:6257:0:(nrs_fifo.c:179:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.204.3@tcp, seq: 6652 00000100:00100000:1.0:1776178239.787744:0:6257:0:(service.c:2324:ptlrpc_server_handle_request()) Handling RPC req@00000000eeeaf4ad pname:cluuid+ref:pid:xid:nid:opc:job mdt00_002:0+-99:2269:x1862456979111296:12345-192.168.204.3@tcp:38:kworker/u8:4.0 00010000:02000000:1.0:1776178239.787760:0:6257:0:(ldlm_lib.c:1139:target_handle_connect()) lustre-MDT0000: Not available for connect from 192.168.204.3@tcp (stopping) 00000100:00100000:2.0:1776178239.792302:0:20140:0:(service.c:2152:ptlrpc_server_handle_req_in()) got req x1862456979111232 00000100:00100000:2.0:1776178239.792421:0:20140:0:(nrs_fifo.c:179:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.204.3@tcp, seq: 169 00000100:00100000:2.0:1776178239.792429:0:20140:0:(service.c:2324:ptlrpc_server_handle_request()) Handling RPC req@000000005b4b43d0 pname:cluuid+ref:pid:xid:nid:opc:job ll_mgs_0002:5b5c66b1-1a2f-4651-8548-0c75b9a38a94+9:2273:x1862456979111232:12345-192.168.204.3@tcp:400:kworker/u8:4.0 00000100:00100000:2.0:1776178239.792546:0:20140:0:(service.c:2372:ptlrpc_server_handle_request()) Handled RPC req@000000005b4b43d0 pname:cluuid+ref:pid:xid:nid:opc:job ll_mgs_0002:5b5c66b1-1a2f-4651-8548-0c75b9a38a94+9:2273:x1862456979111232:12345-192.168.204.3@tcp:400:kworker/u8:4.0 Request processed in 116us (5175us total) trans 0 rc 0/0 00000100:00100000:2.0:1776178239.792557:0:20140:0:(nrs_fifo.c:241:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.204.3@tcp, seq: 169 00000100:00100000:0.0:1776178239.793489:0:14440:0:(service.c:2152:ptlrpc_server_handle_req_in()) got req x1862456979111488 00000100:00100000:0.0:1776178239.793551:0:14440:0:(service.c:2152:ptlrpc_server_handle_req_in()) got req x1862456979111424 00000100:00100000:0.0:1776178239.793588:0:14440:0:(nrs_fifo.c:179:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.204.3@tcp, seq: 539 00000100:00100000:0.0:1776178239.793593:0:14440:0:(service.c:2324:ptlrpc_server_handle_request()) Handling RPC req@000000005fdea934 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_004:b50963bd-8eba-4432-9a54-8888d00e3fa1+9:2273:x1862456979111488:12345-192.168.204.3@tcp:400:kworker/u8:4.0 00000100:00100000:0.0:1776178239.793649:0:14440:0:(service.c:2372:ptlrpc_server_handle_request()) Handled RPC req@000000005fdea934 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_004:b50963bd-8eba-4432-9a54-8888d00e3fa1+9:2273:x1862456979111488:12345-192.168.204.3@tcp:400:kworker/u8:4.0 Request processed in 55us (6331us total) trans 0 rc 0/0 00000100:00100000:0.0:1776178239.793656:0:14440:0:(nrs_fifo.c:241:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.204.3@tcp, seq: 539 00000100:00100000:0.0:1776178239.793675:0:14440:0:(nrs_fifo.c:179:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.204.3@tcp, seq: 540 00000100:00100000:0.0:1776178239.793679:0:14440:0:(service.c:2324:ptlrpc_server_handle_request()) Handling RPC req@00000000b30b9cdd pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_004:b50963bd-8eba-4432-9a54-8888d00e3fa1+8:2273:x1862456979111424:12345-192.168.204.3@tcp:400:kworker/u8:4.0 00000100:00100000:0.0:1776178239.793703:0:14440:0:(service.c:2372:ptlrpc_server_handle_request()) Handled RPC req@00000000b30b9cdd pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_004:b50963bd-8eba-4432-9a54-8888d00e3fa1+8:2273:x1862456979111424:12345-192.168.204.3@tcp:400:kworker/u8:4.0 Request processed in 23us (6291us total) trans 0 rc 0/0 00000100:00100000:0.0:1776178239.793709:0:14440:0:(nrs_fifo.c:241:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.204.3@tcp, seq: 540 00000100:00100000:2.0:1776178239.822588:0:6257:0:(service.c:2372:ptlrpc_server_handle_request()) Handled RPC req@00000000eeeaf4ad pname:cluuid+ref:pid:xid:nid:opc:job mdt00_002:0+-99:2269:x1862456979111296:12345-192.168.204.3@tcp:38:kworker/u8:4.0 Request processed in 34833us (35118us total) trans 0 rc -19/-19 00000100:00100000:2.0:1776178239.822609:0:6257:0:(nrs_fifo.c:241:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.204.3@tcp, seq: 6652 00010000:00010000:2.0:1776178239.968324:0:9224:0:(ldlm_lockd.c:2792:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000020:02000400:3.0:1776178240.352627:0:22200:0:(genops.c:1786:obd_exports_barrier()) lustre-MDT0000 is waiting for obd_unlinked_exports more than 8 seconds. The obd refcount = 4. Is it stuck? 00000020:02000400:3.0:1776178240.358240:0:22200:0:(genops.c:1752:print_export_data()) lustre-MDT0000: UNLINKED 00000000ca22e26f b50963bd-8eba-4432-9a54-8888d00e3fa1 1 (0 0 0) 1 0 1 0: 0000000000000000 8589934704 stale:0 00010000:00010000:2.0:1776178240.994086:0:9225:0:(ldlm_lockd.c:2792:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00080000:0.0:1776178241.378353:0:36:0:(pinger.c:251:ptlrpc_pinger_process_import()) lustre-MDT0000-lwp-OST0000_UUID->lustre-MDT0000_UUID: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00080000:0.0:1776178241.378361:0:36:0:(recover.c:57:ptlrpc_initiate_recovery()) lustre-MDT0000_UUID: starting recovery 00000100:00080000:0.0:1776178241.378365:0:36:0:(import.c:86:import_set_state_nolock()) 0000000047601a98 lustre-MDT0000_UUID: changing import state from DISCONN to CONNECTING 00000100:00080000:0.0:1776178241.378373:0:36:0:(import.c:534:import_select_connection()) lustre-MDT0000-lwp-OST0000: connect to NID 0@lo last attempt 943 00000100:00080000:0.0:1776178241.378378:0:36:0:(import.c:581:import_select_connection()) lustre-MDT0000-lwp-OST0000: tried all connections, increasing latency to 16s 00000100:00080000:0.0:1776178241.378384:0:36:0:(import.c:615:import_select_connection()) lustre-MDT0000-lwp-OST0000: import 0000000047601a98 using connection 192.168.204.103@tcp/0@lo 00000100:00100000:3.0:1776178241.378680:0:3354:0:(client.c:1733:ptlrpc_send_new_req()) Sending RPC req@000000005f3eedd2 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_01:ac61f839-0e8e-4cac-9c59-08bcfbbad330:3354:1862456998054464:0@lo:400:kworker/u8:1.0 00000100:00100000:3.0:1776178241.378805:0:3354:0:(events.c:361:request_in_callback()) peer: 12345-0@lo (source: 12345-0@lo) 00000100:00100000:3.0:1776178241.379056:0:20140:0:(service.c:2152:ptlrpc_server_handle_req_in()) got req x1862456998054464 00000100:00100000:3.0:1776178241.379135:0:20140:0:(nrs_fifo.c:179:nrs_fifo_req_get()) NRS start fifo request from 12345-0@lo, seq: 170 00000100:00100000:3.0:1776178241.379143:0:20140:0:(service.c:2324:ptlrpc_server_handle_request()) Handling RPC req@0000000020ee145c pname:cluuid+ref:pid:xid:nid:opc:job ll_mgs_0002:ac61f839-0e8e-4cac-9c59-08bcfbbad330+18:3354:x1862456998054464:12345-0@lo:400:kworker/u8:1.0 00000100:00100000:3.0:1776178241.379199:0:20140:0:(service.c:2372:ptlrpc_server_handle_request()) Handled RPC req@0000000020ee145c pname:cluuid+ref:pid:xid:nid:opc:job ll_mgs_0002:ac61f839-0e8e-4cac-9c59-08bcfbbad330+18:3354:x1862456998054464:12345-0@lo:400:kworker/u8:1.0 Request processed in 56us (395us total) trans 0 rc 0/0 00000100:00100000:3.0:1776178241.379206:0:20140:0:(nrs_fifo.c:241:nrs_fifo_req_stop()) NRS stop fifo request from 12345-0@lo, seq: 170 00000100:00100000:3.0:1776178241.379252:0:3354:0:(client.c:2215:ptlrpc_check_set()) Completed RPC req@000000005f3eedd2 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_01:ac61f839-0e8e-4cac-9c59-08bcfbbad330:3354:1862456998054464:0@lo:400:kworker/u8:1.0 00000100:00100000:0.0:1776178241.379337:0:36:0:(import.c:807:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5) req@0000000099e8e662 x1862456998054528/t0(0) o38->lustre-MDT0000-lwp-OST0000@0@lo:12/10 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' 00000100:00080000:0.0:1776178241.379390:0:36:0:(pinger.c:251:ptlrpc_pinger_process_import()) lustre-MDT0000-lwp-OST0001_UUID->lustre-MDT0000_UUID: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00080000:0.0:1776178241.379405:0:36:0:(recover.c:57:ptlrpc_initiate_recovery()) lustre-MDT0000_UUID: starting recovery 00000100:00080000:0.0:1776178241.379407:0:36:0:(import.c:86:import_set_state_nolock()) 00000000dfc8db13 lustre-MDT0000_UUID: changing import state from DISCONN to CONNECTING 00000100:00080000:0.0:1776178241.379411:0:36:0:(import.c:534:import_select_connection()) lustre-MDT0000-lwp-OST0001: connect to NID 0@lo last attempt 943 00000100:00080000:0.0:1776178241.379414:0:36:0:(import.c:581:import_select_connection()) lustre-MDT0000-lwp-OST0001: tried all connections, increasing latency to 16s 00000100:00080000:0.0:1776178241.379418:0:36:0:(import.c:615:import_select_connection()) lustre-MDT0000-lwp-OST0001: import 00000000dfc8db13 using connection 192.168.204.103@tcp/0@lo 00000100:00100000:1.0:1776178241.379643:0:3356:0:(ptlrpcd.c:415:ptlrpcd_check()) transfer 1 async RPCs [2->3] 00000100:00100000:1.0:1776178241.379657:0:3356:0:(client.c:1733:ptlrpc_send_new_req()) Sending RPC req@000000000c9fb390 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_03:lustre-MDT0001-lwp-OST0000_UUID:3356:1862456998054592:0@lo:400:kworker/u8:1.0 00000100:00100000:1.0:1776178241.379694:0:3356:0:(events.c:361:request_in_callback()) peer: 12345-0@lo (source: 12345-0@lo) 00000100:00100000:1.0:1776178241.379730:0:6257:0:(service.c:2152:ptlrpc_server_handle_req_in()) got req x1862456998054592 00000100:00100000:1.0:1776178241.379775:0:6257:0:(nrs_fifo.c:179:nrs_fifo_req_get()) NRS start fifo request from 12345-0@lo, seq: 950 00000100:00100000:1.0:1776178241.379779:0:6257:0:(service.c:2324:ptlrpc_server_handle_request()) Handling RPC req@00000000f4172b4c pname:cluuid+ref:pid:xid:nid:opc:job mdt00_002:lustre-MDT0001-lwp-OST0000_UUID+5:3356:x1862456998054592:12345-0@lo:400:kworker/u8:1.0 00000100:00100000:1.0:1776178241.379814:0:6257:0:(service.c:2372:ptlrpc_server_handle_request()) Handled RPC req@00000000f4172b4c pname:cluuid+ref:pid:xid:nid:opc:job mdt00_002:lustre-MDT0001-lwp-OST0000_UUID+5:3356:x1862456998054592:12345-0@lo:400:kworker/u8:1.0 Request processed in 35us (121us total) trans 0 rc 0/0 00000100:00100000:1.0:1776178241.379834:0:6257:0:(nrs_fifo.c:241:nrs_fifo_req_stop()) NRS stop fifo request from 12345-0@lo, seq: 950 00000100:00100000:1.0:1776178241.379874:0:3356:0:(client.c:2215:ptlrpc_check_set()) Completed RPC req@000000000c9fb390 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_03:lustre-MDT0001-lwp-OST0000_UUID:3356:1862456998054592:0@lo:400:kworker/u8:1.0 00000100:00100000:2.0:1776178241.380399:0:3352:0:(client.c:1733:ptlrpc_send_new_req()) Sending RPC req@0000000099e8e662 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:lustre-MDT0000-lwp-OST0000_UUID:3352:1862456998054528:0@lo:38:kworker/u8:1.0 00000100:00100000:2.0:1776178241.380446:0:3352:0:(events.c:361:request_in_callback()) peer: 12345-0@lo (source: 12345-0@lo) 00000100:00100000:0.0:1776178241.380667:0:36:0:(import.c:807:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5) req@000000005f3eedd2 x1862456998054656/t0(0) o38->lustre-MDT0000-lwp-OST0001@0@lo:12/10 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' 00000100:00080000:0.0:1776178241.380710:0:36:0:(pinger.c:251:ptlrpc_pinger_process_import()) lustre-MDT0001-mdtlov_UUID->lustre-MDT0000_UUID: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00080000:0.0:1776178241.380714:0:36:0:(recover.c:57:ptlrpc_initiate_recovery()) lustre-MDT0000_UUID: starting recovery 00000100:00080000:0.0:1776178241.380716:0:36:0:(import.c:86:import_set_state_nolock()) 00000000ba4f1821 lustre-MDT0000_UUID: changing import state from DISCONN to CONNECTING 00000100:00080000:0.0:1776178241.380720:0:36:0:(import.c:534:import_select_connection()) lustre-MDT0000-osp-MDT0001: connect to NID 0@lo last attempt 943 00000100:00080000:0.0:1776178241.380722:0:36:0:(import.c:581:import_select_connection()) lustre-MDT0000-osp-MDT0001: tried all connections, increasing latency to 16s 00000100:00080000:0.0:1776178241.380726:0:36:0:(import.c:615:import_select_connection()) lustre-MDT0000-osp-MDT0001: import 00000000ba4f1821 using connection 192.168.204.103@tcp/0@lo 00000100:00100000:0.0:1776178241.380743:0:36:0:(import.c:807:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5) req@000000000c9fb390 x1862456998054784/t0(0) o38->lustre-MDT0000-osp-MDT0001@0@lo:24/4 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' 00000100:00080000:0.0:1776178241.380771:0:36:0:(pinger.c:251:ptlrpc_pinger_process_import()) lustre-MDT0000-lwp-MDT0001_UUID->lustre-MDT0000_UUID: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00080000:0.0:1776178241.380773:0:36:0:(recover.c:57:ptlrpc_initiate_recovery()) lustre-MDT0000_UUID: starting recovery 00000100:00080000:0.0:1776178241.380775:0:36:0:(import.c:86:import_set_state_nolock()) 00000000a7c55b1f lustre-MDT0000_UUID: changing import state from DISCONN to CONNECTING 00000100:00080000:0.0:1776178241.380778:0:36:0:(import.c:534:import_select_connection()) lustre-MDT0000-lwp-MDT0001: connect to NID 0@lo last attempt 943 00000100:00080000:0.0:1776178241.380780:0:36:0:(import.c:581:import_select_connection()) lustre-MDT0000-lwp-MDT0001: tried all connections, increasing latency to 16s 00000100:00080000:0.0:1776178241.380782:0:36:0:(import.c:615:import_select_connection()) lustre-MDT0000-lwp-MDT0001: import 00000000a7c55b1f using connection 192.168.204.103@tcp/0@lo 00000100:00100000:0.0:1776178241.380791:0:36:0:(import.c:807:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5) req@000000007fcab602 x1862456998054976/t0(0) o38->lustre-MDT0000-lwp-MDT0001@0@lo:12/10 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' 00000100:00100000:0.0:1776178241.380812:0:3353:0:(client.c:1733:ptlrpc_send_new_req()) Sending RPC req@00000000f4d20636 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_00:lustre-MDT0001-mdtlov_UUID:3353:1862456998054848:0@lo:400:kworker/u8:1.0 00000100:00100000:1.0:1776178241.381091:0:11281:0:(service.c:2152:ptlrpc_server_handle_req_in()) got req x1862456998054528 00000100:00100000:1.0:1776178241.381121:0:11281:0:(nrs_fifo.c:179:nrs_fifo_req_get()) NRS start fifo request from 12345-0@lo, seq: 6653 00000100:00100000:1.0:1776178241.381126:0:11281:0:(service.c:2324:ptlrpc_server_handle_request()) Handling RPC req@00000000a8c8cb2d pname:cluuid+ref:pid:xid:nid:opc:job mdt00_004:0+-99:3352:x1862456998054528:12345-0@lo:38:kworker/u8:1.0 00010000:02000000:1.0:1776178241.381141:0:11281:0:(ldlm_lib.c:1139:target_handle_connect()) lustre-MDT0000: Not available for connect from 0@lo (stopping) 00000100:00100000:1.0:1776178241.381178:0:11281:0:(service.c:2372:ptlrpc_server_handle_request()) Handled RPC req@00000000a8c8cb2d pname:cluuid+ref:pid:xid:nid:opc:job mdt00_004:0+-99:3352:x1862456998054528:12345-0@lo:38:kworker/u8:1.0 Request processed in 52us (733us total) trans 0 rc -19/-19 00000100:00100000:1.0:1776178241.381185:0:11281:0:(nrs_fifo.c:241:nrs_fifo_req_stop()) NRS stop fifo request from 12345-0@lo, seq: 6653 00000100:00100000:1.0:1776178241.381215:0:3356:0:(client.c:1733:ptlrpc_send_new_req()) Sending RPC req@00000000a49c1305 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_03:lustre-MDT0001-lwp-OST0001_UUID:3356:1862456998054720:0@lo:400:kworker/u8:1.0 00000100:00100000:1.0:1776178241.381239:0:3356:0:(events.c:361:request_in_callback()) peer: 12345-0@lo (source: 12345-0@lo) 00000100:00100000:1.0:1776178241.381265:0:6257:0:(service.c:2152:ptlrpc_server_handle_req_in()) got req x1862456998054720 00000100:00100000:1.0:1776178241.381288:0:6257:0:(nrs_fifo.c:179:nrs_fifo_req_get()) NRS start fifo request from 12345-0@lo, seq: 951 00000100:00100000:1.0:1776178241.381291:0:6257:0:(service.c:2324:ptlrpc_server_handle_request()) Handling RPC req@00000000ba9bc15d pname:cluuid+ref:pid:xid:nid:opc:job mdt00_002:lustre-MDT0001-lwp-OST0001_UUID+5:3356:x1862456998054720:12345-0@lo:400:kworker/u8:1.0 00000100:00100000:1.0:1776178241.381325:0:6257:0:(service.c:2372:ptlrpc_server_handle_request()) Handled RPC req@00000000ba9bc15d pname:cluuid+ref:pid:xid:nid:opc:job mdt00_002:lustre-MDT0001-lwp-OST0001_UUID+5:3356:x1862456998054720:12345-0@lo:400:kworker/u8:1.0 Request processed in 32us (84us total) trans 0 rc 0/0 00000100:00100000:1.0:1776178241.381331:0:6257:0:(nrs_fifo.c:241:nrs_fifo_req_stop()) NRS stop fifo request from 12345-0@lo, seq: 951 00000100:00100000:1.0:1776178241.381360:0:3356:0:(client.c:2215:ptlrpc_check_set()) Completed RPC req@00000000a49c1305 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_03:lustre-MDT0001-lwp-OST0001_UUID:3356:1862456998054720:0@lo:400:kworker/u8:1.0 00000100:00100000:0.0:1776178241.382093:0:3353:0:(events.c:361:request_in_callback()) peer: 12345-0@lo (source: 12345-0@lo) 00000100:00100000:0.0:1776178241.382124:0:3353:0:(ptlrpcd.c:415:ptlrpcd_check()) transfer 1 async RPCs [1->0] 00000100:00100000:0.0:1776178241.382131:0:3353:0:(client.c:1733:ptlrpc_send_new_req()) Sending RPC req@00000000f8c48966 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_00:lustre-MDT0001-mdtlov_UUID:3353:1862456998054912:0@lo:400:kworker/u8:1.0 00000100:00100000:0.0:1776178241.382155:0:3353:0:(events.c:361:request_in_callback()) peer: 12345-0@lo (source: 12345-0@lo) 00000100:00100000:1.0:1776178241.382236:0:20755:0:(service.c:2152:ptlrpc_server_handle_req_in()) got req x1862456998054848 00000100:00100000:1.0:1776178241.382270:0:20755:0:(service.c:2152:ptlrpc_server_handle_req_in()) got req x1862456998054912 00000100:00100000:1.0:1776178241.382287:0:20755:0:(nrs_fifo.c:179:nrs_fifo_req_get()) NRS start fifo request from 12345-0@lo, seq: 541 00000100:00100000:1.0:1776178241.382291:0:20755:0:(service.c:2324:ptlrpc_server_handle_request()) Handling RPC req@0000000056dbc32b pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_006:lustre-MDT0001-mdtlov_UUID+5:3353:x1862456998054848:12345-0@lo:400:kworker/u8:1.0 00000100:00100000:1.0:1776178241.382326:0:20755:0:(service.c:2372:ptlrpc_server_handle_request()) Handled RPC req@0000000056dbc32b pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_006:lustre-MDT0001-mdtlov_UUID+5:3353:x1862456998054848:12345-0@lo:400:kworker/u8:1.0 Request processed in 34us (235us total) trans 0 rc 0/0 00000100:00100000:1.0:1776178241.382331:0:20755:0:(nrs_fifo.c:241:nrs_fifo_req_stop()) NRS stop fifo request from 12345-0@lo, seq: 541 00000100:00100000:1.0:1776178241.382344:0:20755:0:(nrs_fifo.c:179:nrs_fifo_req_get()) NRS start fifo request from 12345-0@lo, seq: 542 00000100:00100000:1.0:1776178241.382347:0:20755:0:(service.c:2324:ptlrpc_server_handle_request()) Handling RPC req@000000005c76b1b3 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_006:lustre-MDT0001-mdtlov_UUID+5:3353:x1862456998054912:12345-0@lo:400:kworker/u8:1.0 00000100:00100000:1.0:1776178241.382368:0:20755:0:(service.c:2372:ptlrpc_server_handle_request()) Handled RPC req@000000005c76b1b3 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_006:lustre-MDT0001-mdtlov_UUID+5:3353:x1862456998054912:12345-0@lo:400:kworker/u8:1.0 Request processed in 21us (214us total) trans 0 rc 0/0 00000100:00100000:1.0:1776178241.382373:0:20755:0:(nrs_fifo.c:241:nrs_fifo_req_stop()) NRS stop fifo request from 12345-0@lo, seq: 542 00000100:00100000:2.0:1776178241.382559:0:3352:0:(client.c:1733:ptlrpc_send_new_req()) Sending RPC req@000000005f3eedd2 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:lustre-MDT0000-lwp-OST0001_UUID:3352:1862456998054656:0@lo:38:kworker/u8:1.0 00000100:00100000:2.0:1776178241.382591:0:3352:0:(events.c:361:request_in_callback()) peer: 12345-0@lo (source: 12345-0@lo) 00000100:00100000:2.0:1776178241.382608:0:3352:0:(client.c:1733:ptlrpc_send_new_req()) Sending RPC req@000000000c9fb390 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:lustre-MDT0001-mdtlov_UUID:3352:1862456998054784:0@lo:38:kworker/u8:1.0 00000100:00100000:2.0:1776178241.382630:0:3352:0:(events.c:361:request_in_callback()) peer: 12345-0@lo (source: 12345-0@lo) 00000100:00100000:2.0:1776178241.382644:0:3352:0:(client.c:1733:ptlrpc_send_new_req()) Sending RPC req@000000007fcab602 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:lustre-MDT0000-lwp-MDT0001_UUID:3352:1862456998054976:0@lo:38:kworker/u8:1.0 00000100:00100000:2.0:1776178241.382665:0:3352:0:(events.c:361:request_in_callback()) peer: 12345-0@lo (source: 12345-0@lo) 00000100:00080000:2.0:1776178241.382688:0:3352:0:(import.c:86:import_set_state_nolock()) 0000000047601a98 lustre-MDT0000_UUID: changing import state from CONNECTING to DISCONN 00000100:00080000:2.0:1776178241.382692:0:3352:0:(import.c:1429:ptlrpc_connect_interpret()) recovery of lustre-MDT0000_UUID on 192.168.204.103@tcp failed (-19) 00000100:00100000:2.0:1776178241.382695:0:3352:0:(client.c:2215:ptlrpc_check_set()) Completed RPC req@0000000099e8e662 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:lustre-MDT0000-lwp-OST0000_UUID:3352:1862456998054528:0@lo:38:kworker/u8:1.0 00000100:00100000:2.0:1776178241.382722:0:15213:0:(service.c:2152:ptlrpc_server_handle_req_in()) got req x1862456998054784 00000100:00100000:2.0:1776178241.382744:0:15213:0:(nrs_fifo.c:179:nrs_fifo_req_get()) NRS start fifo request from 12345-0@lo, seq: 715 00000100:00100000:2.0:1776178241.382748:0:15213:0:(service.c:2324:ptlrpc_server_handle_request()) Handling RPC req@000000003095df64 pname:cluuid+ref:pid:xid:nid:opc:job mdt_out00_003:0+-99:3352:x1862456998054784:12345-0@lo:38:kworker/u8:1.0 00010000:02000000:2.0:1776178241.382760:0:15213:0:(ldlm_lib.c:1139:target_handle_connect()) lustre-MDT0000: Not available for connect from 0@lo (stopping) 00000100:00100000:2.0:1776178241.382792:0:15213:0:(service.c:2372:ptlrpc_server_handle_request()) Handled RPC req@000000003095df64 pname:cluuid+ref:pid:xid:nid:opc:job mdt_out00_003:0+-99:3352:x1862456998054784:12345-0@lo:38:kworker/u8:1.0 Request processed in 42us (161us total) trans 0 rc -19/-19 00000100:00100000:2.0:1776178241.382797:0:15213:0:(nrs_fifo.c:241:nrs_fifo_req_stop()) NRS stop fifo request from 12345-0@lo, seq: 715 00000100:00080000:2.0:1776178241.382902:0:3352:0:(import.c:86:import_set_state_nolock()) 00000000ba4f1821 lustre-MDT0000_UUID: changing import state from CONNECTING to DISCONN 00000100:00080000:2.0:1776178241.382906:0:3352:0:(import.c:1429:ptlrpc_connect_interpret()) recovery of lustre-MDT0000_UUID on 192.168.204.103@tcp failed (-19) 00000100:00100000:2.0:1776178241.382909:0:3352:0:(client.c:2215:ptlrpc_check_set()) Completed RPC req@000000000c9fb390 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:lustre-MDT0001-mdtlov_UUID:3352:1862456998054784:0@lo:38:kworker/u8:1.0 00000100:00100000:2.0:1776178241.382930:0:6257:0:(service.c:2152:ptlrpc_server_handle_req_in()) got req x1862456998054656 00000100:00100000:2.0:1776178241.382948:0:6257:0:(service.c:2152:ptlrpc_server_handle_req_in()) got req x1862456998054976 00000100:00100000:2.0:1776178241.382960:0:6257:0:(nrs_fifo.c:179:nrs_fifo_req_get()) NRS start fifo request from 12345-0@lo, seq: 6654 00000100:00100000:2.0:1776178241.382964:0:6257:0:(service.c:2324:ptlrpc_server_handle_request()) Handling RPC req@0000000092028508 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_002:0+-99:3352:x1862456998054656:12345-0@lo:38:kworker/u8:1.0 00010000:02000000:2.0:1776178241.382971:0:6257:0:(ldlm_lib.c:1139:target_handle_connect()) lustre-MDT0000: Not available for connect from 0@lo (stopping) 00000100:00100000:2.0:1776178241.383010:0:6257:0:(service.c:2372:ptlrpc_server_handle_request()) Handled RPC req@0000000092028508 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_002:0+-99:3352:x1862456998054656:12345-0@lo:38:kworker/u8:1.0 Request processed in 44us (419us total) trans 0 rc -19/-19 00000100:00100000:2.0:1776178241.383015:0:6257:0:(nrs_fifo.c:241:nrs_fifo_req_stop()) NRS stop fifo request from 12345-0@lo, seq: 6654 00000100:00100000:2.0:1776178241.383025:0:6257:0:(nrs_fifo.c:179:nrs_fifo_req_get()) NRS start fifo request from 12345-0@lo, seq: 6655 00000100:00100000:2.0:1776178241.383028:0:6257:0:(service.c:2324:ptlrpc_server_handle_request()) Handling RPC req@000000002dd26ce5 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_002:0+-99:3352:x1862456998054976:12345-0@lo:38:kworker/u8:1.0 00010000:02000000:2.0:1776178241.383034:0:6257:0:(ldlm_lib.c:1139:target_handle_connect()) lustre-MDT0000: Not available for connect from 0@lo (stopping) 00000100:00100000:2.0:1776178241.383053:0:6257:0:(service.c:2372:ptlrpc_server_handle_request()) Handled RPC req@000000002dd26ce5 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_002:0+-99:3352:x1862456998054976:12345-0@lo:38:kworker/u8:1.0 Request processed in 24us (387us total) trans 0 rc -19/-19 00000100:00100000:2.0:1776178241.383057:0:6257:0:(nrs_fifo.c:241:nrs_fifo_req_stop()) NRS stop fifo request from 12345-0@lo, seq: 6655 00000100:00080000:2.0:1776178241.383101:0:3352:0:(import.c:86:import_set_state_nolock()) 00000000dfc8db13 lustre-MDT0000_UUID: changing import state from CONNECTING to DISCONN 00000100:00080000:2.0:1776178241.383105:0:3352:0:(import.c:1429:ptlrpc_connect_interpret()) recovery of lustre-MDT0000_UUID on 192.168.204.103@tcp failed (-19) 00000100:00100000:2.0:1776178241.383108:0:3352:0:(client.c:2215:ptlrpc_check_set()) Completed RPC req@000000005f3eedd2 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:lustre-MDT0000-lwp-OST0001_UUID:3352:1862456998054656:0@lo:38:kworker/u8:1.0 00000100:00080000:2.0:1776178241.383115:0:3352:0:(import.c:86:import_set_state_nolock()) 00000000a7c55b1f lustre-MDT0000_UUID: changing import state from CONNECTING to DISCONN 00000100:00080000:2.0:1776178241.383117:0:3352:0:(import.c:1429:ptlrpc_connect_interpret()) recovery of lustre-MDT0000_UUID on 192.168.204.103@tcp failed (-19) 00000100:00100000:2.0:1776178241.383120:0:3352:0:(client.c:2215:ptlrpc_check_set()) Completed RPC req@000000007fcab602 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:lustre-MDT0000-lwp-MDT0001_UUID:3352:1862456998054976:0@lo:38:kworker/u8:1.0 00000100:00100000:0.0:1776178241.383757:0:3353:0:(client.c:2215:ptlrpc_check_set()) Completed RPC req@00000000f8c48966 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_00:lustre-MDT0001-mdtlov_UUID:3353:1862456998054912:0@lo:400:kworker/u8:1.0 00000100:00100000:0.0:1776178241.383771:0:3353:0:(client.c:2215:ptlrpc_check_set()) Completed RPC req@00000000f4d20636 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_00:lustre-MDT0001-mdtlov_UUID:3353:1862456998054848:0@lo:400:kworker/u8:1.0 00010000:00010000:0.0:1776178242.016217:0:6244:0:(ldlm_lockd.c:2792:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:2.0:1776178242.913188:0:3355:0:(client.c:1733:ptlrpc_send_new_req()) Sending RPC req@000000001da3f43e pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_02:lustre-MDT0001-mdtlov_UUID:3355:1862456998055040:0@lo:13:osp-pre-1-1.0 00000100:00100000:2.0:1776178242.913311:0:3355:0:(events.c:361:request_in_callback()) peer: 12345-0@lo (source: 12345-0@lo) 00000100:00100000:1.0:1776178242.914117:0:8231:0:(service.c:2152:ptlrpc_server_handle_req_in()) got req x1862456998055040 00000100:00100000:1.0:1776178242.914191:0:8231:0:(nrs_fifo.c:179:nrs_fifo_req_get()) NRS start fifo request from 12345-0@lo, seq: 503 00000100:00100000:1.0:1776178242.914200:0:8231:0:(service.c:2324:ptlrpc_server_handle_request()) Handling RPC req@000000001c465e98 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_create00:lustre-MDT0001-mdtlov_UUID+5:3355:x1862456998055040:12345-0@lo:13:osp-pre-1-1.0 00002000:00000024:1.0:1776178242.914241:0:8231:0:(ofd_obd.c:740:ofd_statfs()) blocks cached 229376 granted 491746368 pending 0 free 3923689472 avail 3692167168 00000100:00100000:1.0:1776178242.920943:0:8231:0:(service.c:2372:ptlrpc_server_handle_request()) Handled RPC req@000000001c465e98 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_create00:lustre-MDT0001-mdtlov_UUID+5:3355:x1862456998055040:12345-0@lo:13:osp-pre-1-1.0 Request processed in 6727us (7627us total) trans 0 rc 0/0 00000100:00100000:1.0:1776178242.920973:0:8231:0:(nrs_fifo.c:241:nrs_fifo_req_stop()) NRS stop fifo request from 12345-0@lo, seq: 503 00000100:00100000:2.0:1776178242.925971:0:3355:0:(client.c:2215:ptlrpc_check_set()) Completed RPC req@000000001da3f43e pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_02:lustre-MDT0001-mdtlov_UUID:3355:1862456998055040:0@lo:13:osp-pre-1-1.0 00010000:00010000:2.0:1776178243.041563:0:6245:0:(ldlm_lockd.c:2792:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:1.0:1776178243.424719:0:3356:0:(client.c:1733:ptlrpc_send_new_req()) Sending RPC req@00000000f4d20636 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_03:lustre-MDT0001-mdtlov_UUID:3356:1862456998055104:0@lo:13:osp-pre-0-1.0 00000100:00100000:1.0:1776178243.425111:0:3356:0:(events.c:361:request_in_callback()) peer: 12345-0@lo (source: 12345-0@lo) 00000100:00100000:3.0:1776178243.425279:0:8231:0:(service.c:2152:ptlrpc_server_handle_req_in()) got req x1862456998055104 00000100:00100000:3.0:1776178243.425356:0:8231:0:(nrs_fifo.c:179:nrs_fifo_req_get()) NRS start fifo request from 12345-0@lo, seq: 504 00000100:00100000:3.0:1776178243.425363:0:8231:0:(service.c:2324:ptlrpc_server_handle_request()) Handling RPC req@00000000ab295186 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_create00:lustre-MDT0001-mdtlov_UUID+5:3356:x1862456998055104:12345-0@lo:13:osp-pre-0-1.0 00002000:00000024:3.0:1776178243.425402:0:8231:0:(ofd_obd.c:740:ofd_statfs()) blocks cached 315392 granted 484883776 pending 0 free 3923689472 avail 3692167168 00000100:00100000:3.0:1776178243.425697:0:8231:0:(service.c:2372:ptlrpc_server_handle_request()) Handled RPC req@00000000ab295186 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_create00:lustre-MDT0001-mdtlov_UUID+5:3356:x1862456998055104:12345-0@lo:13:osp-pre-0-1.0 Request processed in 332us (588us total) trans 0 rc 0/0 00000100:00100000:3.0:1776178243.425706:0:8231:0:(nrs_fifo.c:241:nrs_fifo_req_stop()) NRS stop fifo request from 12345-0@lo, seq: 504 00000100:00100000:1.0:1776178243.427211:0:3356:0:(client.c:2215:ptlrpc_check_set()) Completed RPC req@00000000f4d20636 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_03:lustre-MDT0001-mdtlov_UUID:3356:1862456998055104:0@lo:13:osp-pre-0-1.0 00010000:00010000:2.0:1776178244.067183:0:9224:0:(ldlm_lockd.c:2792:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:1.0:1776178244.898372:0:3346:0:(events.c:361:request_in_callback()) peer: 12345-192.168.204.3@tcp (source: 12345-192.168.204.3@tcp) 00000100:00100000:2.0:1776178244.899145:0:20137:0:(service.c:2152:ptlrpc_server_handle_req_in()) got req x1862456979111552 00000100:00100000:2.0:1776178244.899219:0:20137:0:(nrs_fifo.c:179:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.204.3@tcp, seq: 171 00000100:00100000:2.0:1776178244.899230:0:20137:0:(service.c:2324:ptlrpc_server_handle_request()) Handling RPC req@00000000f8a3917d pname:cluuid+ref:pid:xid:nid:opc:job ll_mgs_0000:5b5c66b1-1a2f-4651-8548-0c75b9a38a94+9:2271:x1862456979111552:12345-192.168.204.3@tcp:400:kworker/u8:3.0 00000100:00100000:2.0:1776178244.899344:0:20137:0:(service.c:2372:ptlrpc_server_handle_request()) Handled RPC req@00000000f8a3917d pname:cluuid+ref:pid:xid:nid:opc:job ll_mgs_0000:5b5c66b1-1a2f-4651-8548-0c75b9a38a94+9:2271:x1862456979111552:12345-192.168.204.3@tcp:400:kworker/u8:3.0 Request processed in 115us (983us total) trans 0 rc 0/0 00000100:00100000:2.0:1776178244.899353:0:20137:0:(nrs_fifo.c:241:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.204.3@tcp, seq: 171 00000100:00100000:0.0:1776178244.902404:0:3348:0:(events.c:361:request_in_callback()) peer: 12345-192.168.204.3@tcp (source: 12345-192.168.204.3@tcp) 00000100:00100000:0.0:1776178244.902466:0:3348:0:(events.c:361:request_in_callback()) peer: 12345-192.168.204.3@tcp (source: 12345-192.168.204.3@tcp) 00000100:00100000:1.0:1776178244.902714:0:11281:0:(service.c:2152:ptlrpc_server_handle_req_in()) got req x1862456979111616 00000100:00100000:1.0:1776178244.902786:0:11281:0:(service.c:2152:ptlrpc_server_handle_req_in()) got req x1862456979111680 00000100:00100000:1.0:1776178244.902843:0:11281:0:(nrs_fifo.c:179:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.204.3@tcp, seq: 952 00000100:00100000:1.0:1776178244.902851:0:11281:0:(service.c:2324:ptlrpc_server_handle_request()) Handling RPC req@00000000f4d20636 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_004:b50963bd-8eba-4432-9a54-8888d00e3fa1+6:2271:x1862456979111680:12345-192.168.204.3@tcp:400:kworker/u8:3.0 00000100:00100000:1.0:1776178244.902983:0:11281:0:(service.c:2372:ptlrpc_server_handle_request()) Handled RPC req@00000000f4d20636 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_004:b50963bd-8eba-4432-9a54-8888d00e3fa1+6:2271:x1862456979111680:12345-192.168.204.3@tcp:400:kworker/u8:3.0 Request processed in 132us (516us total) trans 0 rc 0/0 00000100:00100000:1.0:1776178244.902993:0:11281:0:(nrs_fifo.c:241:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.204.3@tcp, seq: 952 00000100:00100000:1.0:1776178244.903015:0:11281:0:(nrs_fifo.c:179:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.204.3@tcp, seq: 6656 00000100:00100000:1.0:1776178244.903033:0:11281:0:(service.c:2324:ptlrpc_server_handle_request()) Handling RPC req@00000000f8c48966 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_004:0+-99:2269:x1862456979111616:12345-192.168.204.3@tcp:38:kworker/u8:3.0 00010000:02000000:1.0:1776178244.903052:0:11281:0:(ldlm_lib.c:1139:target_handle_connect()) lustre-MDT0000: Not available for connect from 192.168.204.3@tcp (stopping) 00000100:00100000:1.0:1776178244.903140:0:11281:0:(service.c:2372:ptlrpc_server_handle_request()) Handled RPC req@00000000f8c48966 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_004:0+-99:2269:x1862456979111616:12345-192.168.204.3@tcp:38:kworker/u8:3.0 Request processed in 106us (762us total) trans 0 rc -19/-19 00000100:00100000:1.0:1776178244.903148:0:11281:0:(nrs_fifo.c:241:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.204.3@tcp, seq: 6656 00000100:00100000:0.0:1776178244.905880:0:3347:0:(events.c:361:request_in_callback()) peer: 12345-192.168.204.3@tcp (source: 12345-192.168.204.3@tcp) 00000100:00100000:0.0:1776178244.905950:0:3347:0:(events.c:361:request_in_callback()) peer: 12345-192.168.204.3@tcp (source: 12345-192.168.204.3@tcp) 00000100:00100000:1.0:1776178244.906099:0:20759:0:(service.c:2152:ptlrpc_server_handle_req_in()) got req x1862456979111808 00000100:00100000:1.0:1776178244.906313:0:20759:0:(service.c:2152:ptlrpc_server_handle_req_in()) got req x1862456979111744 00000100:00100000:2.0:1776178244.912142:0:14440:0:(nrs_fifo.c:179:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.204.3@tcp, seq: 543 00000100:00100000:2.0:1776178244.912156:0:14440:0:(service.c:2324:ptlrpc_server_handle_request()) Handling RPC req@000000007fcab602 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_004:b50963bd-8eba-4432-9a54-8888d00e3fa1+9:2270:x1862456979111808:12345-192.168.204.3@tcp:400:kworker/u8:3.0 00000100:00100000:2.0:1776178244.912307:0:14440:0:(service.c:2372:ptlrpc_server_handle_request()) Handled RPC req@000000007fcab602 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_004:b50963bd-8eba-4432-9a54-8888d00e3fa1+9:2270:x1862456979111808:12345-192.168.204.3@tcp:400:kworker/u8:3.0 Request processed in 152us (6436us total) trans 0 rc 0/0 00000100:00100000:2.0:1776178244.912315:0:14440:0:(nrs_fifo.c:241:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.204.3@tcp, seq: 543 00000100:00100000:2.0:1776178244.912352:0:14440:0:(nrs_fifo.c:179:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.204.3@tcp, seq: 544 00000100:00100000:2.0:1776178244.912357:0:14440:0:(service.c:2324:ptlrpc_server_handle_request()) Handling RPC req@00000000490667a2 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_004:b50963bd-8eba-4432-9a54-8888d00e3fa1+8:2270:x1862456979111744:12345-192.168.204.3@tcp:400:kworker/u8:3.0 00000100:00100000:2.0:1776178244.912899:0:14440:0:(service.c:2372:ptlrpc_server_handle_request()) Handled RPC req@00000000490667a2 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_004:b50963bd-8eba-4432-9a54-8888d00e3fa1+8:2270:x1862456979111744:12345-192.168.204.3@tcp:400:kworker/u8:3.0 Request processed in 540us (6948us total) trans 0 rc 0/0 00000100:00100000:2.0:1776178244.912907:0:14440:0:(nrs_fifo.c:241:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.204.3@tcp, seq: 544 00010000:00010000:2.0:1776178245.089956:0:9225:0:(ldlm_lockd.c:2792:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00010000:00010000:0.0:1776178246.113190:0:6244:0:(ldlm_lockd.c:2792:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00080000:3.0:1776178246.497969:0:36:0:(pinger.c:251:ptlrpc_pinger_process_import()) lustre-MDT0000-lwp-OST0000_UUID->lustre-MDT0000_UUID: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00080000:3.0:1776178246.497980:0:36:0:(recover.c:57:ptlrpc_initiate_recovery()) lustre-MDT0000_UUID: starting recovery 00000100:00080000:3.0:1776178246.497984:0:36:0:(import.c:86:import_set_state_nolock()) 0000000047601a98 lustre-MDT0000_UUID: changing import state from DISCONN to CONNECTING 00000100:00080000:3.0:1776178246.497995:0:36:0:(import.c:534:import_select_connection()) lustre-MDT0000-lwp-OST0000: connect to NID 0@lo last attempt 948 00000100:00080000:3.0:1776178246.498003:0:36:0:(import.c:581:import_select_connection()) lustre-MDT0000-lwp-OST0000: tried all connections, increasing latency to 20s 00000100:00080000:3.0:1776178246.498012:0:36:0:(import.c:615:import_select_connection()) lustre-MDT0000-lwp-OST0000: import 0000000047601a98 using connection 192.168.204.103@tcp/0@lo 00000100:00100000:3.0:1776178246.498037:0:36:0:(import.c:807:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5) req@000000001da3f43e x1862456998055232/t0(0) o38->lustre-MDT0000-lwp-OST0000@0@lo:12/10 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' 00000100:00100000:1.0:1776178246.498263:0:3354:0:(ptlrpcd.c:415:ptlrpcd_check()) transfer 1 async RPCs [0->1] 00000100:00100000:1.0:1776178246.498284:0:3354:0:(client.c:1733:ptlrpc_send_new_req()) Sending RPC req@000000008c50846d pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_01:ac61f839-0e8e-4cac-9c59-08bcfbbad330:3354:1862456998055168:0@lo:400:kworker/u8:1.0 00000100:00100000:1.0:1776178246.498362:0:3354:0:(events.c:361:request_in_callback()) peer: 12345-0@lo (source: 12345-0@lo) 00000100:00100000:1.0:1776178246.498439:0:20140:0:(service.c:2152:ptlrpc_server_handle_req_in()) got req x1862456998055168 00000100:00100000:1.0:1776178246.498495:0:20140:0:(nrs_fifo.c:179:nrs_fifo_req_get()) NRS start fifo request from 12345-0@lo, seq: 172 00000100:00100000:1.0:1776178246.498503:0:20140:0:(service.c:2324:ptlrpc_server_handle_request()) Handling RPC req@000000006fd17377 pname:cluuid+ref:pid:xid:nid:opc:job ll_mgs_0002:ac61f839-0e8e-4cac-9c59-08bcfbbad330+18:3354:x1862456998055168:12345-0@lo:400:kworker/u8:1.0 00000100:00100000:1.0:1776178246.498569:0:20140:0:(service.c:2372:ptlrpc_server_handle_request()) Handled RPC req@000000006fd17377 pname:cluuid+ref:pid:xid:nid:opc:job ll_mgs_0002:ac61f839-0e8e-4cac-9c59-08bcfbbad330+18:3354:x1862456998055168:12345-0@lo:400:kworker/u8:1.0 Request processed in 66us (208us total) trans 0 rc 0/0 00000100:00100000:1.0:1776178246.498577:0:20140:0:(nrs_fifo.c:241:nrs_fifo_req_stop()) NRS stop fifo request from 12345-0@lo, seq: 172 00000100:00100000:1.0:1776178246.498635:0:3354:0:(client.c:2215:ptlrpc_check_set()) Completed RPC req@000000008c50846d pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_01:ac61f839-0e8e-4cac-9c59-08bcfbbad330:3354:1862456998055168:0@lo:400:kworker/u8:1.0 00000100:00100000:2.0:1776178246.498823:0:3352:0:(client.c:1733:ptlrpc_send_new_req()) Sending RPC req@000000001da3f43e pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:lustre-MDT0000-lwp-OST0000_UUID:3352:1862456998055232:0@lo:38:kworker/u8:1.0 00000100:00100000:2.0:1776178246.498871:0:3352:0:(events.c:361:request_in_callback()) peer: 12345-0@lo (source: 12345-0@lo) 00000100:00080000:3.0:1776178246.499052:0:36:0:(pinger.c:251:ptlrpc_pinger_process_import()) lustre-MDT0000-lwp-OST0001_UUID->lustre-MDT0000_UUID: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00100000:0.0:1776178246.499276:0:3353:0:(ptlrpcd.c:415:ptlrpcd_check()) transfer 1 async RPCs [1->0] 00000100:00100000:0.0:1776178246.499288:0:3353:0:(client.c:1733:ptlrpc_send_new_req()) Sending RPC req@000000008c50846d pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_00:lustre-MDT0001-lwp-OST0000_UUID:3353:1862456998055296:0@lo:400:kworker/u8:1.0 00000100:00100000:0.0:1776178246.499319:0:3353:0:(events.c:361:request_in_callback()) peer: 12345-0@lo (source: 12345-0@lo) 00000100:00100000:0.0:1776178246.499352:0:6257:0:(service.c:2152:ptlrpc_server_handle_req_in()) got req x1862456998055232 00000100:00100000:0.0:1776178246.499377:0:6257:0:(service.c:2152:ptlrpc_server_handle_req_in()) got req x1862456998055296 00000100:00100000:0.0:1776178246.499398:0:6257:0:(nrs_fifo.c:179:nrs_fifo_req_get()) NRS start fifo request from 12345-0@lo, seq: 953 00000100:00100000:0.0:1776178246.499402:0:6257:0:(service.c:2324:ptlrpc_server_handle_request()) Handling RPC req@000000006be5e5b9 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_002:lustre-MDT0001-lwp-OST0000_UUID+5:3353:x1862456998055296:12345-0@lo:400:kworker/u8:1.0 00000100:00100000:0.0:1776178246.499441:0:6257:0:(service.c:2372:ptlrpc_server_handle_request()) Handled RPC req@000000006be5e5b9 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_002:lustre-MDT0001-lwp-OST0000_UUID+5:3353:x1862456998055296:12345-0@lo:400:kworker/u8:1.0 Request processed in 38us (121us total) trans 0 rc 0/0 00000100:00100000:0.0:1776178246.499447:0:6257:0:(nrs_fifo.c:241:nrs_fifo_req_stop()) NRS stop fifo request from 12345-0@lo, seq: 953 00000100:00100000:0.0:1776178246.499461:0:6257:0:(nrs_fifo.c:179:nrs_fifo_req_get()) NRS start fifo request from 12345-0@lo, seq: 6657 00000100:00100000:0.0:1776178246.499477:0:6257:0:(service.c:2324:ptlrpc_server_handle_request()) Handling RPC req@00000000bebb3448 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_002:0+-99:3352:x1862456998055232:12345-0@lo:38:kworker/u8:1.0 00010000:02000000:0.0:1776178246.499493:0:6257:0:(ldlm_lib.c:1139:target_handle_connect()) lustre-MDT0000: Not available for connect from 0@lo (stopping) 00000100:00080000:3.0:1776178246.499561:0:36:0:(recover.c:57:ptlrpc_initiate_recovery()) lustre-MDT0000_UUID: starting recovery 00000100:00080000:3.0:1776178246.499565:0:36:0:(import.c:86:import_set_state_nolock()) 00000000dfc8db13 lustre-MDT0000_UUID: changing import state from DISCONN to CONNECTING 00000100:00080000:3.0:1776178246.499570:0:36:0:(import.c:534:import_select_connection()) lustre-MDT0000-lwp-OST0001: connect to NID 0@lo last attempt 948 00000100:00080000:3.0:1776178246.499573:0:36:0:(import.c:581:import_select_connection()) lustre-MDT0000-lwp-OST0001: tried all connections, increasing latency to 20s 00000100:00080000:3.0:1776178246.499577:0:36:0:(import.c:615:import_select_connection()) lustre-MDT0000-lwp-OST0001: import 00000000dfc8db13 using connection 192.168.204.103@tcp/0@lo 00000100:00100000:3.0:1776178246.499602:0:36:0:(import.c:807:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5) req@000000001e11faca x1862456998055360/t0(0) o38->lustre-MDT0000-lwp-OST0001@0@lo:12/10 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' 00000100:00100000:1.0:1776178246.499704:0:3353:0:(client.c:2215:ptlrpc_check_set()) Completed RPC req@000000008c50846d pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_00:lustre-MDT0001-lwp-OST0000_UUID:3353:1862456998055296:0@lo:400:kworker/u8:1.0 00000100:00100000:2.0:1776178246.505753:0:3352:0:(client.c:1733:ptlrpc_send_new_req()) Sending RPC req@000000001e11faca pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:lustre-MDT0000-lwp-OST0001_UUID:3352:1862456998055360:0@lo:38:kworker/u8:1.0 00000100:00100000:2.0:1776178246.505852:0:3352:0:(events.c:361:request_in_callback()) peer: 12345-0@lo (source: 12345-0@lo) 00000100:00100000:1.0:1776178246.506028:0:6256:0:(service.c:2152:ptlrpc_server_handle_req_in()) got req x1862456998055360 00000100:00100000:1.0:1776178246.506119:0:6256:0:(nrs_fifo.c:179:nrs_fifo_req_get()) NRS start fifo request from 12345-0@lo, seq: 6658 00000100:00100000:1.0:1776178246.506127:0:6256:0:(service.c:2324:ptlrpc_server_handle_request()) Handling RPC req@0000000096181210 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_001:0+-99:3352:x1862456998055360:12345-0@lo:38:kworker/u8:1.0 00010000:02000000:1.0:1776178246.506148:0:6256:0:(ldlm_lib.c:1139:target_handle_connect()) lustre-MDT0000: Not available for connect from 0@lo (stopping) 00000100:00100000:1.0:1776178246.506202:0:6256:0:(service.c:2372:ptlrpc_server_handle_request()) Handled RPC req@0000000096181210 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_001:0+-99:3352:x1862456998055360:12345-0@lo:38:kworker/u8:1.0 Request processed in 74us (351us total) trans 0 rc -19/-19 00000100:00100000:1.0:1776178246.506210:0:6256:0:(nrs_fifo.c:241:nrs_fifo_req_stop()) NRS stop fifo request from 12345-0@lo, seq: 6658 00000100:00100000:0.0:1776178246.506396:0:6257:0:(service.c:2372:ptlrpc_server_handle_request()) Handled RPC req@00000000bebb3448 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_002:0+-99:3352:x1862456998055232:12345-0@lo:38:kworker/u8:1.0 Request processed in 6915us (7523us total) trans 0 rc -19/-19 00000100:00080000:3.0:1776178246.506400:0:36:0:(pinger.c:251:ptlrpc_pinger_process_import()) lustre-MDT0001-mdtlov_UUID->lustre-MDT0000_UUID: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00100000:0.0:1776178246.506406:0:6257:0:(nrs_fifo.c:241:nrs_fifo_req_stop()) NRS stop fifo request from 12345-0@lo, seq: 6657 00000100:00080000:3.0:1776178246.506406:0:36:0:(recover.c:57:ptlrpc_initiate_recovery()) lustre-MDT0000_UUID: starting recovery 00000100:00080000:3.0:1776178246.506411:0:36:0:(import.c:86:import_set_state_nolock()) 00000000ba4f1821 lustre-MDT0000_UUID: changing import state from DISCONN to CONNECTING 00000100:00080000:3.0:1776178246.506418:0:36:0:(import.c:534:import_select_connection()) lustre-MDT0000-osp-MDT0001: connect to NID 0@lo last attempt 948 00000100:00080000:3.0:1776178246.506422:0:36:0:(import.c:581:import_select_connection()) lustre-MDT0000-osp-MDT0001: tried all connections, increasing latency to 20s 00000100:00080000:3.0:1776178246.506427:0:36:0:(import.c:615:import_select_connection()) lustre-MDT0000-osp-MDT0001: import 00000000ba4f1821 using connection 192.168.204.103@tcp/0@lo 00000100:00100000:3.0:1776178246.506445:0:36:0:(import.c:807:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5) req@00000000456acdaf x1862456998055488/t0(0) o38->lustre-MDT0000-osp-MDT0001@0@lo:24/4 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' 00000100:00080000:3.0:1776178246.506477:0:36:0:(pinger.c:251:ptlrpc_pinger_process_import()) lustre-MDT0000-lwp-MDT0001_UUID->lustre-MDT0000_UUID: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00080000:3.0:1776178246.506480:0:36:0:(recover.c:57:ptlrpc_initiate_recovery()) lustre-MDT0000_UUID: starting recovery 00000100:00080000:3.0:1776178246.506481:0:36:0:(import.c:86:import_set_state_nolock()) 00000000a7c55b1f lustre-MDT0000_UUID: changing import state from DISCONN to CONNECTING 00000100:00080000:3.0:1776178246.506485:0:36:0:(import.c:534:import_select_connection()) lustre-MDT0000-lwp-MDT0001: connect to NID 0@lo last attempt 948 00000100:00080000:3.0:1776178246.506486:0:36:0:(import.c:581:import_select_connection()) lustre-MDT0000-lwp-MDT0001: tried all connections, increasing latency to 20s 00000100:00080000:3.0:1776178246.506489:0:36:0:(import.c:615:import_select_connection()) lustre-MDT0000-lwp-MDT0001: import 00000000a7c55b1f using connection 192.168.204.103@tcp/0@lo 00000100:00100000:3.0:1776178246.506499:0:36:0:(import.c:807:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5) req@00000000a6227918 x1862456998055680/t0(0) o38->lustre-MDT0000-lwp-MDT0001@0@lo:12/10 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' 00000100:00100000:1.0:1776178246.506505:0:3353:0:(client.c:1733:ptlrpc_send_new_req()) Sending RPC req@000000004420855e pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_00:lustre-MDT0001-mdtlov_UUID:3353:1862456998055616:0@lo:400:kworker/u8:1.0 00000100:00100000:3.0:1776178246.506541:0:3355:0:(client.c:1733:ptlrpc_send_new_req()) Sending RPC req@0000000049b1b339 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_02:lustre-MDT0001-lwp-OST0001_UUID:3355:1862456998055424:0@lo:400:kworker/u8:1.0 00000100:00100000:3.0:1776178246.506571:0:3355:0:(events.c:361:request_in_callback()) peer: 12345-0@lo (source: 12345-0@lo) 00000100:00100000:3.0:1776178246.506586:0:3355:0:(ptlrpcd.c:415:ptlrpcd_check()) transfer 1 async RPCs [3->2] 00000100:00100000:3.0:1776178246.506591:0:3355:0:(client.c:1733:ptlrpc_send_new_req()) Sending RPC req@00000000ed90ff75 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_02:lustre-MDT0001-mdtlov_UUID:3355:1862456998055552:0@lo:400:kworker/u8:1.0 00000100:00100000:0.0:1776178246.506611:0:6257:0:(service.c:2152:ptlrpc_server_handle_req_in()) got req x1862456998055424 00000100:00100000:3.0:1776178246.506618:0:3355:0:(events.c:361:request_in_callback()) peer: 12345-0@lo (source: 12345-0@lo) 00000100:00100000:0.0:1776178246.506655:0:6257:0:(nrs_fifo.c:179:nrs_fifo_req_get()) NRS start fifo request from 12345-0@lo, seq: 954 00000100:00100000:1.0:1776178246.506655:0:3353:0:(events.c:361:request_in_callback()) peer: 12345-0@lo (source: 12345-0@lo) 00000100:00100000:3.0:1776178246.506656:0:20762:0:(service.c:2152:ptlrpc_server_handle_req_in()) got req x1862456998055552 00000100:00100000:0.0:1776178246.506664:0:6257:0:(service.c:2324:ptlrpc_server_handle_request()) Handling RPC req@0000000039ff8335 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_002:lustre-MDT0001-lwp-OST0001_UUID+5:3355:x1862456998055424:12345-0@lo:400:kworker/u8:1.0 00000100:00100000:3.0:1776178246.506694:0:20762:0:(service.c:2152:ptlrpc_server_handle_req_in()) got req x1862456998055616 00000100:00100000:1.0:1776178246.506708:0:20759:0:(nrs_fifo.c:179:nrs_fifo_req_get()) NRS start fifo request from 12345-0@lo, seq: 545 00000100:00100000:1.0:1776178246.506715:0:20759:0:(service.c:2324:ptlrpc_server_handle_request()) Handling RPC req@0000000056ddcb09 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_010:lustre-MDT0001-mdtlov_UUID+5:3355:x1862456998055552:12345-0@lo:400:kworker/u8:1.0 00000100:00100000:0.0:1776178246.506722:0:6257:0:(service.c:2372:ptlrpc_server_handle_request()) Handled RPC req@0000000039ff8335 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_002:lustre-MDT0001-lwp-OST0001_UUID+5:3355:x1862456998055424:12345-0@lo:400:kworker/u8:1.0 Request processed in 57us (149us total) trans 0 rc 0/0 00000100:00100000:3.0:1776178246.506730:0:20762:0:(nrs_fifo.c:179:nrs_fifo_req_get()) NRS start fifo request from 12345-0@lo, seq: 546 00000100:00100000:0.0:1776178246.506731:0:6257:0:(nrs_fifo.c:241:nrs_fifo_req_stop()) NRS stop fifo request from 12345-0@lo, seq: 954 00000100:00100000:3.0:1776178246.506739:0:20762:0:(service.c:2324:ptlrpc_server_handle_request()) Handling RPC req@0000000054fef155 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_013:lustre-MDT0001-mdtlov_UUID+5:3353:x1862456998055616:12345-0@lo:400:kworker/u8:1.0 00000100:00100000:1.0:1776178246.506759:0:20759:0:(service.c:2372:ptlrpc_server_handle_request()) Handled RPC req@0000000056ddcb09 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_010:lustre-MDT0001-mdtlov_UUID+5:3355:x1862456998055552:12345-0@lo:400:kworker/u8:1.0 Request processed in 43us (140us total) trans 0 rc 0/0 00000100:00100000:1.0:1776178246.506767:0:20759:0:(nrs_fifo.c:241:nrs_fifo_req_stop()) NRS stop fifo request from 12345-0@lo, seq: 545 00000100:00100000:0.0:1776178246.506782:0:3355:0:(client.c:2215:ptlrpc_check_set()) Completed RPC req@00000000ed90ff75 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_02:lustre-MDT0001-mdtlov_UUID:3355:1862456998055552:0@lo:400:kworker/u8:1.0 00000100:00100000:3.0:1776178246.506790:0:20762:0:(service.c:2372:ptlrpc_server_handle_request()) Handled RPC req@0000000054fef155 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_013:lustre-MDT0001-mdtlov_UUID+5:3353:x1862456998055616:12345-0@lo:400:kworker/u8:1.0 Request processed in 51us (140us total) trans 0 rc 0/0 00000100:00100000:3.0:1776178246.506798:0:20762:0:(nrs_fifo.c:241:nrs_fifo_req_stop()) NRS stop fifo request from 12345-0@lo, seq: 546 00000100:00100000:0.0:1776178246.506801:0:3355:0:(client.c:2215:ptlrpc_check_set()) Completed RPC req@0000000049b1b339 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_02:lustre-MDT0001-lwp-OST0001_UUID:3355:1862456998055424:0@lo:400:kworker/u8:1.0 00000100:00100000:1.0:1776178246.506813:0:3353:0:(client.c:2215:ptlrpc_check_set()) Completed RPC req@000000004420855e pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_00:lustre-MDT0001-mdtlov_UUID:3353:1862456998055616:0@lo:400:kworker/u8:1.0 00000100:00080000:2.0:1776178246.508201:0:3352:0:(import.c:86:import_set_state_nolock()) 0000000047601a98 lustre-MDT0000_UUID: changing import state from CONNECTING to DISCONN 00000100:00080000:2.0:1776178246.508212:0:3352:0:(import.c:1429:ptlrpc_connect_interpret()) recovery of lustre-MDT0000_UUID on 192.168.204.103@tcp failed (-19) 00000100:00100000:2.0:1776178246.508220:0:3352:0:(client.c:2215:ptlrpc_check_set()) Completed RPC req@000000001da3f43e pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:lustre-MDT0000-lwp-OST0000_UUID:3352:1862456998055232:0@lo:38:kworker/u8:1.0 00000100:00100000:2.0:1776178246.508258:0:3352:0:(client.c:1733:ptlrpc_send_new_req()) Sending RPC req@00000000456acdaf pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:lustre-MDT0001-mdtlov_UUID:3352:1862456998055488:0@lo:38:kworker/u8:1.0 00000100:00100000:2.0:1776178246.508312:0:3352:0:(events.c:361:request_in_callback()) peer: 12345-0@lo (source: 12345-0@lo) 00000100:00100000:2.0:1776178246.508344:0:3352:0:(client.c:1733:ptlrpc_send_new_req()) Sending RPC req@00000000a6227918 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:lustre-MDT0000-lwp-MDT0001_UUID:3352:1862456998055680:0@lo:38:kworker/u8:1.0 00000100:00100000:2.0:1776178246.508370:0:3352:0:(events.c:361:request_in_callback()) peer: 12345-0@lo (source: 12345-0@lo) 00000100:00080000:2.0:1776178246.508401:0:3352:0:(import.c:86:import_set_state_nolock()) 00000000dfc8db13 lustre-MDT0000_UUID: changing import state from CONNECTING to DISCONN 00000100:00080000:2.0:1776178246.508405:0:3352:0:(import.c:1429:ptlrpc_connect_interpret()) recovery of lustre-MDT0000_UUID on 192.168.204.103@tcp failed (-19) 00000100:00100000:2.0:1776178246.508408:0:3352:0:(client.c:2215:ptlrpc_check_set()) Completed RPC req@000000001e11faca pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:lustre-MDT0000-lwp-OST0001_UUID:3352:1862456998055360:0@lo:38:kworker/u8:1.0 00000100:00100000:0.0:1776178246.508558:0:11281:0:(service.c:2152:ptlrpc_server_handle_req_in()) got req x1862456998055680 00000100:00100000:0.0:1776178246.508603:0:11281:0:(nrs_fifo.c:179:nrs_fifo_req_get()) NRS start fifo request from 12345-0@lo, seq: 6659 00000100:00100000:0.0:1776178246.508608:0:11281:0:(service.c:2324:ptlrpc_server_handle_request()) Handling RPC req@00000000931093aa pname:cluuid+ref:pid:xid:nid:opc:job mdt00_004:0+-99:3352:x1862456998055680:12345-0@lo:38:kworker/u8:1.0 00010000:02000000:0.0:1776178246.508627:0:11281:0:(ldlm_lib.c:1139:target_handle_connect()) lustre-MDT0000: Not available for connect from 0@lo (stopping) 00000100:00100000:1.0:1776178246.508680:0:14128:0:(service.c:2152:ptlrpc_server_handle_req_in()) got req x1862456998055488 00000100:00100000:0.0:1776178246.508692:0:11281:0:(service.c:2372:ptlrpc_server_handle_request()) Handled RPC req@00000000931093aa pname:cluuid+ref:pid:xid:nid:opc:job mdt00_004:0+-99:3352:x1862456998055680:12345-0@lo:38:kworker/u8:1.0 Request processed in 81us (320us total) trans 0 rc -19/-19 00000100:00100000:0.0:1776178246.508699:0:11281:0:(nrs_fifo.c:241:nrs_fifo_req_stop()) NRS stop fifo request from 12345-0@lo, seq: 6659 00000100:00100000:1.0:1776178246.508716:0:14128:0:(nrs_fifo.c:179:nrs_fifo_req_get()) NRS start fifo request from 12345-0@lo, seq: 716 00000100:00100000:1.0:1776178246.508722:0:14128:0:(service.c:2324:ptlrpc_server_handle_request()) Handling RPC req@000000005459e48f pname:cluuid+ref:pid:xid:nid:opc:job mdt_out00_002:0+-99:3352:x1862456998055488:12345-0@lo:38:kworker/u8:1.0 00010000:02000000:1.0:1776178246.508732:0:14128:0:(ldlm_lib.c:1139:target_handle_connect()) lustre-MDT0000: Not available for connect from 0@lo (stopping) 00000100:00100000:1.0:1776178246.508763:0:14128:0:(service.c:2372:ptlrpc_server_handle_request()) Handled RPC req@000000005459e48f pname:cluuid+ref:pid:xid:nid:opc:job mdt_out00_002:0+-99:3352:x1862456998055488:12345-0@lo:38:kworker/u8:1.0 Request processed in 41us (453us total) trans 0 rc -19/-19 00000100:00100000:1.0:1776178246.508769:0:14128:0:(nrs_fifo.c:241:nrs_fifo_req_stop()) NRS stop fifo request from 12345-0@lo, seq: 716 00000100:00080000:3.0:1776178246.508854:0:3352:0:(import.c:86:import_set_state_nolock()) 00000000ba4f1821 lustre-MDT0000_UUID: changing import state from CONNECTING to DISCONN 00000100:00080000:3.0:1776178246.508859:0:3352:0:(import.c:1429:ptlrpc_connect_interpret()) recovery of lustre-MDT0000_UUID on 192.168.204.103@tcp failed (-19) 00000100:00100000:3.0:1776178246.508862:0:3352:0:(client.c:2215:ptlrpc_check_set()) Completed RPC req@00000000456acdaf pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:lustre-MDT0001-mdtlov_UUID:3352:1862456998055488:0@lo:38:kworker/u8:1.0 00000100:00080000:3.0:1776178246.508870:0:3352:0:(import.c:86:import_set_state_nolock()) 00000000a7c55b1f lustre-MDT0000_UUID: changing import state from CONNECTING to DISCONN 00000100:00080000:3.0:1776178246.508873:0:3352:0:(import.c:1429:ptlrpc_connect_interpret()) recovery of lustre-MDT0000_UUID on 192.168.204.103@tcp failed (-19) 00000100:00100000:3.0:1776178246.508875:0:3352:0:(client.c:2215:ptlrpc_check_set()) Completed RPC req@00000000a6227918 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:lustre-MDT0000-lwp-MDT0001_UUID:3352:1862456998055680:0@lo:38:kworker/u8:1.0 00010000:00010000:2.0:1776178247.136974:0:6245:0:(ldlm_lockd.c:2792:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:0.0:1776178248.033524:0:3353:0:(ptlrpcd.c:415:ptlrpcd_check()) transfer 1 async RPCs [1->0] 00000100:00100000:0.0:1776178248.033560:0:3353:0:(client.c:1733:ptlrpc_send_new_req()) Sending RPC req@00000000a6227918 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_00:lustre-MDT0001-mdtlov_UUID:3353:1862456998055744:0@lo:13:osp-pre-1-1.0 00000100:00100000:0.0:1776178248.033641:0:3353:0:(events.c:361:request_in_callback()) peer: 12345-0@lo (source: 12345-0@lo) 00000100:00100000:3.0:1776178248.033716:0:9554:0:(service.c:2152:ptlrpc_server_handle_req_in()) got req x1862456998055744 00000100:00100000:3.0:1776178248.033789:0:9554:0:(nrs_fifo.c:179:nrs_fifo_req_get()) NRS start fifo request from 12345-0@lo, seq: 505 00000100:00100000:3.0:1776178248.033795:0:9554:0:(service.c:2324:ptlrpc_server_handle_request()) Handling RPC req@00000000a1262b14 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_create00:lustre-MDT0001-mdtlov_UUID+5:3353:x1862456998055744:12345-0@lo:13:osp-pre-1-1.0 00002000:00000024:3.0:1776178248.033831:0:9554:0:(ofd_obd.c:740:ofd_statfs()) blocks cached 229376 granted 491746368 pending 0 free 3923689472 avail 3692167168 00000100:00100000:2.0:1776178248.035852:0:3353:0:(client.c:2215:ptlrpc_check_set()) Completed RPC req@00000000a6227918 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_00:lustre-MDT0001-mdtlov_UUID:3353:1862456998055744:0@lo:13:osp-pre-1-1.0 00000100:00100000:3.0:1776178248.036434:0:9554:0:(service.c:2372:ptlrpc_server_handle_request()) Handled RPC req@00000000a1262b14 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_create00:lustre-MDT0001-mdtlov_UUID+5:3353:x1862456998055744:12345-0@lo:13:osp-pre-1-1.0 Request processed in 1063us (1219us total) trans 0 rc 0/0 00000100:00100000:3.0:1776178248.036451:0:9554:0:(nrs_fifo.c:241:nrs_fifo_req_stop()) NRS stop fifo request from 12345-0@lo, seq: 505 00010000:00010000:2.0:1776178248.165115:0:9224:0:(ldlm_lockd.c:2792:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:0.0:1776178248.546433:0:3356:0:(ptlrpcd.c:415:ptlrpcd_check()) transfer 1 async RPCs [2->3] 00000100:00100000:0.0:1776178248.546456:0:3356:0:(client.c:1733:ptlrpc_send_new_req()) Sending RPC req@00000000d668bb9c pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_03:lustre-MDT0001-mdtlov_UUID:3356:1862456998055808:0@lo:13:osp-pre-0-1.0 00000100:00100000:0.0:1776178248.546543:0:3356:0:(events.c:361:request_in_callback()) peer: 12345-0@lo (source: 12345-0@lo) 00000100:00100000:3.0:1776178248.547591:0:9554:0:(service.c:2152:ptlrpc_server_handle_req_in()) got req x1862456998055808 00000100:00100000:3.0:1776178248.547664:0:9554:0:(nrs_fifo.c:179:nrs_fifo_req_get()) NRS start fifo request from 12345-0@lo, seq: 506 00000100:00100000:3.0:1776178248.547671:0:9554:0:(service.c:2324:ptlrpc_server_handle_request()) Handling RPC req@000000007a321680 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_create00:lustre-MDT0001-mdtlov_UUID+5:3356:x1862456998055808:12345-0@lo:13:osp-pre-0-1.0 00002000:00000024:3.0:1776178248.547710:0:9554:0:(ofd_obd.c:740:ofd_statfs()) blocks cached 315392 granted 484883776 pending 0 free 3923689472 avail 3692167168 00000100:00100000:3.0:1776178248.547782:0:9554:0:(service.c:2372:ptlrpc_server_handle_request()) Handled RPC req@000000007a321680 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_create00:lustre-MDT0001-mdtlov_UUID+5:3356:x1862456998055808:12345-0@lo:13:osp-pre-0-1.0 Request processed in 111us (1241us total) trans 0 rc 0/0 00000100:00100000:3.0:1776178248.547790:0:9554:0:(nrs_fifo.c:241:nrs_fifo_req_stop()) NRS stop fifo request from 12345-0@lo, seq: 506 00000100:00100000:1.0:1776178248.548155:0:3356:0:(client.c:2215:ptlrpc_check_set()) Completed RPC req@00000000d668bb9c pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_03:lustre-MDT0001-mdtlov_UUID:3356:1862456998055808:0@lo:13:osp-pre-0-1.0 00010000:00010000:2.0:1776178249.184346:0:9225:0:(ldlm_lockd.c:2792:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:1.0:1776178250.019861:0:3346:0:(events.c:361:request_in_callback()) peer: 12345-192.168.204.3@tcp (source: 12345-192.168.204.3@tcp) 00000100:00100000:2.0:1776178250.028603:0:6257:0:(service.c:2152:ptlrpc_server_handle_req_in()) got req x1862456979111936 00000100:00100000:2.0:1776178250.028692:0:6257:0:(nrs_fifo.c:179:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.204.3@tcp, seq: 6660 00000100:00100000:2.0:1776178250.028702:0:6257:0:(service.c:2324:ptlrpc_server_handle_request()) Handling RPC req@00000000b7440871 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_002:0+-99:2269:x1862456979111936:12345-192.168.204.3@tcp:38:kworker/u8:3.0 00010000:02000000:2.0:1776178250.028726:0:6257:0:(ldlm_lib.c:1139:target_handle_connect()) lustre-MDT0000: Not available for connect from 192.168.204.3@tcp (stopping) 00000100:00100000:0.0:1776178250.030005:0:3347:0:(events.c:361:request_in_callback()) peer: 12345-192.168.204.3@tcp (source: 12345-192.168.204.3@tcp) 00000100:00100000:0.0:1776178250.030094:0:3347:0:(events.c:361:request_in_callback()) peer: 12345-192.168.204.3@tcp (source: 12345-192.168.204.3@tcp) 00000100:00100000:0.0:1776178250.030124:0:3347:0:(events.c:361:request_in_callback()) peer: 12345-192.168.204.3@tcp (source: 12345-192.168.204.3@tcp) 00000100:00100000:0.0:1776178250.030202:0:3347:0:(events.c:361:request_in_callback()) peer: 12345-192.168.204.3@tcp (source: 12345-192.168.204.3@tcp) 00000100:00100000:3.0:1776178250.030386:0:20137:0:(service.c:2152:ptlrpc_server_handle_req_in()) got req x1862456979111872 00000100:00100000:3.0:1776178250.030431:0:20137:0:(nrs_fifo.c:179:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.204.3@tcp, seq: 173 00000100:00100000:3.0:1776178250.030437:0:20137:0:(service.c:2324:ptlrpc_server_handle_request()) Handling RPC req@00000000eb350ccb pname:cluuid+ref:pid:xid:nid:opc:job ll_mgs_0000:5b5c66b1-1a2f-4651-8548-0c75b9a38a94+9:2271:x1862456979111872:12345-192.168.204.3@tcp:400:kworker/u8:3.0 00000100:00100000:3.0:1776178250.030511:0:20137:0:(service.c:2372:ptlrpc_server_handle_request()) Handled RPC req@00000000eb350ccb pname:cluuid+ref:pid:xid:nid:opc:job ll_mgs_0000:5b5c66b1-1a2f-4651-8548-0c75b9a38a94+9:2271:x1862456979111872:12345-192.168.204.3@tcp:400:kworker/u8:3.0 Request processed in 73us (511us total) trans 0 rc 0/0 00000100:00100000:3.0:1776178250.030523:0:20137:0:(nrs_fifo.c:241:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.204.3@tcp, seq: 173 00000100:00100000:3.0:1776178250.030560:0:14440:0:(service.c:2152:ptlrpc_server_handle_req_in()) got req x1862456979112064 00000100:00100000:3.0:1776178250.030588:0:14440:0:(service.c:2152:ptlrpc_server_handle_req_in()) got req x1862456979112128 00000100:00100000:3.0:1776178250.030608:0:14440:0:(nrs_fifo.c:179:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.204.3@tcp, seq: 547 00000100:00100000:3.0:1776178250.030612:0:14440:0:(service.c:2324:ptlrpc_server_handle_request()) Handling RPC req@0000000069ebab60 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_004:b50963bd-8eba-4432-9a54-8888d00e3fa1+8:2271:x1862456979112064:12345-192.168.204.3@tcp:400:kworker/u8:3.0 00000100:00100000:3.0:1776178250.030642:0:14440:0:(service.c:2372:ptlrpc_server_handle_request()) Handled RPC req@0000000069ebab60 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_004:b50963bd-8eba-4432-9a54-8888d00e3fa1+8:2271:x1862456979112064:12345-192.168.204.3@tcp:400:kworker/u8:3.0 Request processed in 29us (584us total) trans 0 rc 0/0 00000100:00100000:3.0:1776178250.030648:0:14440:0:(nrs_fifo.c:241:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.204.3@tcp, seq: 547 00000100:00100000:3.0:1776178250.030659:0:14440:0:(nrs_fifo.c:179:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.204.3@tcp, seq: 548 00000100:00100000:3.0:1776178250.030663:0:14440:0:(service.c:2324:ptlrpc_server_handle_request()) Handling RPC req@00000000b7aa289e pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_004:b50963bd-8eba-4432-9a54-8888d00e3fa1+9:2271:x1862456979112128:12345-192.168.204.3@tcp:400:kworker/u8:3.0 00000100:00100000:3.0:1776178250.030685:0:14440:0:(service.c:2372:ptlrpc_server_handle_request()) Handled RPC req@00000000b7aa289e pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_004:b50963bd-8eba-4432-9a54-8888d00e3fa1+9:2271:x1862456979112128:12345-192.168.204.3@tcp:400:kworker/u8:3.0 Request processed in 21us (484us total) trans 0 rc 0/0 00000100:00100000:3.0:1776178250.030691:0:14440:0:(nrs_fifo.c:241:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.204.3@tcp, seq: 548 00000100:00100000:1.0:1776178250.031561:0:11281:0:(service.c:2152:ptlrpc_server_handle_req_in()) got req x1862456979112000 00000100:00100000:1.0:1776178250.031621:0:11281:0:(nrs_fifo.c:179:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.204.3@tcp, seq: 955 00000100:00100000:1.0:1776178250.031626:0:11281:0:(service.c:2324:ptlrpc_server_handle_request()) Handling RPC req@000000007cbab86b pname:cluuid+ref:pid:xid:nid:opc:job mdt00_004:b50963bd-8eba-4432-9a54-8888d00e3fa1+6:2271:x1862456979112000:12345-192.168.204.3@tcp:400:kworker/u8:3.0 00000100:00100000:1.0:1776178250.031681:0:11281:0:(service.c:2372:ptlrpc_server_handle_request()) Handled RPC req@000000007cbab86b pname:cluuid+ref:pid:xid:nid:opc:job mdt00_004:b50963bd-8eba-4432-9a54-8888d00e3fa1+6:2271:x1862456979112000:12345-192.168.204.3@tcp:400:kworker/u8:3.0 Request processed in 55us (1557us total) trans 0 rc 0/0 00000100:00100000:1.0:1776178250.031689:0:11281:0:(nrs_fifo.c:241:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.204.3@tcp, seq: 955 00000100:00100000:3.0:1776178250.053111:0:6257:0:(service.c:2372:ptlrpc_server_handle_request()) Handled RPC req@00000000b7440871 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_002:0+-99:2269:x1862456979111936:12345-192.168.204.3@tcp:38:kworker/u8:3.0 Request processed in 24400us (33257us total) trans 0 rc -19/-19 00000100:00100000:3.0:1776178250.053143:0:6257:0:(nrs_fifo.c:241:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.204.3@tcp, seq: 6660 00010000:00010000:0.0:1776178250.208625:0:6244:0:(ldlm_lockd.c:2792:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00010000:00010000:2.0:1776178251.233443:0:6245:0:(ldlm_lockd.c:2792:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:0.0:1776178251.616460:0:3355:0:(ptlrpcd.c:415:ptlrpcd_check()) transfer 1 async RPCs [3->2] 00000100:00080000:2.0:1776178251.618356:0:37:0:(pinger.c:251:ptlrpc_pinger_process_import()) lustre-MDT0000-lwp-OST0000_UUID->lustre-MDT0000_UUID: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00080000:2.0:1776178251.618371:0:37:0:(recover.c:57:ptlrpc_initiate_recovery()) lustre-MDT0000_UUID: starting recovery 00000100:00080000:2.0:1776178251.618375:0:37:0:(import.c:86:import_set_state_nolock()) 0000000047601a98 lustre-MDT0000_UUID: changing import state from DISCONN to CONNECTING 00000100:00080000:2.0:1776178251.618383:0:37:0:(import.c:534:import_select_connection()) lustre-MDT0000-lwp-OST0000: connect to NID 0@lo last attempt 953 00000100:00080000:2.0:1776178251.618387:0:37:0:(import.c:581:import_select_connection()) lustre-MDT0000-lwp-OST0000: tried all connections, increasing latency to 20s 00000100:00080000:2.0:1776178251.618394:0:37:0:(import.c:615:import_select_connection()) lustre-MDT0000-lwp-OST0000: import 0000000047601a98 using connection 192.168.204.103@tcp/0@lo 00000100:00100000:2.0:1776178251.618452:0:37:0:(import.c:807:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5) req@000000000390171d x1862456998055936/t0(0) o38->lustre-MDT0000-lwp-OST0000@0@lo:12/10 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' 00000100:00080000:2.0:1776178251.618523:0:37:0:(pinger.c:251:ptlrpc_pinger_process_import()) lustre-MDT0000-lwp-OST0001_UUID->lustre-MDT0000_UUID: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00080000:2.0:1776178251.618527:0:37:0:(recover.c:57:ptlrpc_initiate_recovery()) lustre-MDT0000_UUID: starting recovery 00000100:00080000:2.0:1776178251.618529:0:37:0:(import.c:86:import_set_state_nolock()) 00000000dfc8db13 lustre-MDT0000_UUID: changing import state from DISCONN to CONNECTING 00000100:00080000:2.0:1776178251.618532:0:37:0:(import.c:534:import_select_connection()) lustre-MDT0000-lwp-OST0001: connect to NID 0@lo last attempt 953 00000100:00080000:2.0:1776178251.618533:0:37:0:(import.c:581:import_select_connection()) lustre-MDT0000-lwp-OST0001: tried all connections, increasing latency to 20s 00000100:00080000:2.0:1776178251.618537:0:37:0:(import.c:615:import_select_connection()) lustre-MDT0000-lwp-OST0001: import 00000000dfc8db13 using connection 192.168.204.103@tcp/0@lo 00000100:00100000:2.0:1776178251.618546:0:37:0:(import.c:807:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5) req@00000000ad01e608 x1862456998056064/t0(0) o38->lustre-MDT0000-lwp-OST0001@0@lo:12/10 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' 00000100:00080000:2.0:1776178251.618559:0:37:0:(pinger.c:251:ptlrpc_pinger_process_import()) lustre-MDT0001-mdtlov_UUID->lustre-MDT0000_UUID: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00080000:2.0:1776178251.618563:0:37:0:(recover.c:57:ptlrpc_initiate_recovery()) lustre-MDT0000_UUID: starting recovery 00000100:00080000:2.0:1776178251.618564:0:37:0:(import.c:86:import_set_state_nolock()) 00000000ba4f1821 lustre-MDT0000_UUID: changing import state from DISCONN to CONNECTING 00000100:00080000:2.0:1776178251.618567:0:37:0:(import.c:534:import_select_connection()) lustre-MDT0000-osp-MDT0001: connect to NID 0@lo last attempt 953 00000100:00080000:2.0:1776178251.618569:0:37:0:(import.c:581:import_select_connection()) lustre-MDT0000-osp-MDT0001: tried all connections, increasing latency to 20s 00000100:00080000:2.0:1776178251.618572:0:37:0:(import.c:615:import_select_connection()) lustre-MDT0000-osp-MDT0001: import 00000000ba4f1821 using connection 192.168.204.103@tcp/0@lo 00000100:00100000:2.0:1776178251.618582:0:37:0:(import.c:807:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5) req@000000008f4a0165 x1862456998056192/t0(0) o38->lustre-MDT0000-osp-MDT0001@0@lo:24/4 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' 00000100:00100000:1.0:1776178251.618680:0:3353:0:(client.c:1733:ptlrpc_send_new_req()) Sending RPC req@00000000a5decd95 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_00:lustre-MDT0001-lwp-OST0000_UUID:3353:1862456998056000:0@lo:400:kworker/u8:2.0 00000100:00100000:1.0:1776178251.618769:0:3353:0:(events.c:361:request_in_callback()) peer: 12345-0@lo (source: 12345-0@lo) 00000100:00100000:1.0:1776178251.618814:0:3353:0:(ptlrpcd.c:415:ptlrpcd_check()) transfer 1 async RPCs [1->0] 00000100:00100000:1.0:1776178251.618819:0:3353:0:(client.c:1733:ptlrpc_send_new_req()) Sending RPC req@0000000009593c4b pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_00:lustre-MDT0001-lwp-OST0001_UUID:3353:1862456998056128:0@lo:400:kworker/u8:2.0 00000100:00100000:1.0:1776178251.618842:0:3353:0:(events.c:361:request_in_callback()) peer: 12345-0@lo (source: 12345-0@lo) 00000100:00100000:1.0:1776178251.618894:0:8250:0:(service.c:2152:ptlrpc_server_handle_req_in()) got req x1862456998056000 00000100:00100000:1.0:1776178251.618941:0:8250:0:(service.c:2152:ptlrpc_server_handle_req_in()) got req x1862456998056128 00000100:00100000:1.0:1776178251.618964:0:8250:0:(nrs_fifo.c:179:nrs_fifo_req_get()) NRS start fifo request from 12345-0@lo, seq: 956 00000100:00100000:1.0:1776178251.618972:0:8250:0:(service.c:2324:ptlrpc_server_handle_request()) Handling RPC req@000000006165f052 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_003:lustre-MDT0001-lwp-OST0000_UUID+5:3353:x1862456998056000:12345-0@lo:400:kworker/u8:2.0 00000100:00100000:1.0:1776178251.619038:0:8250:0:(service.c:2372:ptlrpc_server_handle_request()) Handled RPC req@000000006165f052 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_003:lustre-MDT0001-lwp-OST0000_UUID+5:3353:x1862456998056000:12345-0@lo:400:kworker/u8:2.0 Request processed in 66us (271us total) trans 0 rc 0/0 00000100:00100000:1.0:1776178251.619045:0:8250:0:(nrs_fifo.c:241:nrs_fifo_req_stop()) NRS stop fifo request from 12345-0@lo, seq: 956 00000100:00100000:1.0:1776178251.619118:0:8250:0:(nrs_fifo.c:179:nrs_fifo_req_get()) NRS start fifo request from 12345-0@lo, seq: 957 00000100:00100000:1.0:1776178251.619122:0:8250:0:(service.c:2324:ptlrpc_server_handle_request()) Handling RPC req@00000000d9d2b484 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_003:lustre-MDT0001-lwp-OST0001_UUID+5:3353:x1862456998056128:12345-0@lo:400:kworker/u8:2.0 00000100:00100000:1.0:1776178251.619146:0:8250:0:(service.c:2372:ptlrpc_server_handle_request()) Handled RPC req@00000000d9d2b484 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_003:lustre-MDT0001-lwp-OST0001_UUID+5:3353:x1862456998056128:12345-0@lo:400:kworker/u8:2.0 Request processed in 23us (304us total) trans 0 rc 0/0 00000100:00100000:1.0:1776178251.619151:0:8250:0:(nrs_fifo.c:241:nrs_fifo_req_stop()) NRS stop fifo request from 12345-0@lo, seq: 957 00000100:00100000:1.0:1776178251.619203:0:3353:0:(client.c:2215:ptlrpc_check_set()) Completed RPC req@0000000009593c4b pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_00:lustre-MDT0001-lwp-OST0001_UUID:3353:1862456998056128:0@lo:400:kworker/u8:2.0 00000100:00100000:1.0:1776178251.619216:0:3353:0:(client.c:2215:ptlrpc_check_set()) Completed RPC req@00000000a5decd95 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_00:lustre-MDT0001-lwp-OST0000_UUID:3353:1862456998056000:0@lo:400:kworker/u8:2.0 00000100:00100000:3.0:1776178251.621710:0:3352:0:(client.c:1733:ptlrpc_send_new_req()) Sending RPC req@000000000390171d pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:lustre-MDT0000-lwp-OST0000_UUID:3352:1862456998055936:0@lo:38:kworker/u8:2.0 00000100:00100000:3.0:1776178251.621828:0:3352:0:(events.c:361:request_in_callback()) peer: 12345-0@lo (source: 12345-0@lo) 00000100:00100000:3.0:1776178251.621872:0:3352:0:(client.c:1733:ptlrpc_send_new_req()) Sending RPC req@00000000ad01e608 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:lustre-MDT0000-lwp-OST0001_UUID:3352:1862456998056064:0@lo:38:kworker/u8:2.0 00000100:00100000:3.0:1776178251.621894:0:3352:0:(events.c:361:request_in_callback()) peer: 12345-0@lo (source: 12345-0@lo) 00000100:00100000:3.0:1776178251.621918:0:3352:0:(client.c:1733:ptlrpc_send_new_req()) Sending RPC req@000000008f4a0165 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:lustre-MDT0001-mdtlov_UUID:3352:1862456998056192:0@lo:38:kworker/u8:2.0 00000100:00100000:3.0:1776178251.621944:0:3352:0:(events.c:361:request_in_callback()) peer: 12345-0@lo (source: 12345-0@lo) 00000100:00100000:3.0:1776178251.622001:0:14128:0:(service.c:2152:ptlrpc_server_handle_req_in()) got req x1862456998056192 00000100:00100000:3.0:1776178251.622050:0:14128:0:(nrs_fifo.c:179:nrs_fifo_req_get()) NRS start fifo request from 12345-0@lo, seq: 717 00000100:00100000:3.0:1776178251.622056:0:14128:0:(service.c:2324:ptlrpc_server_handle_request()) Handling RPC req@0000000056ddcb09 pname:cluuid+ref:pid:xid:nid:opc:job mdt_out00_002:0+-99:3352:x1862456998056192:12345-0@lo:38:kworker/u8:2.0 00010000:02000000:3.0:1776178251.622128:0:14128:0:(ldlm_lib.c:1139:target_handle_connect()) lustre-MDT0000: Not available for connect from 0@lo (stopping) 00000100:00100000:3.0:1776178251.622177:0:14128:0:(service.c:2372:ptlrpc_server_handle_request()) Handled RPC req@0000000056ddcb09 pname:cluuid+ref:pid:xid:nid:opc:job mdt_out00_002:0+-99:3352:x1862456998056192:12345-0@lo:38:kworker/u8:2.0 Request processed in 119us (232us total) trans 0 rc -19/-19 00000100:00100000:3.0:1776178251.622185:0:14128:0:(nrs_fifo.c:241:nrs_fifo_req_stop()) NRS stop fifo request from 12345-0@lo, seq: 717 00000100:00100000:3.0:1776178251.622217:0:6256:0:(service.c:2152:ptlrpc_server_handle_req_in()) got req x1862456998055936 00000100:00100000:3.0:1776178251.622234:0:6256:0:(service.c:2152:ptlrpc_server_handle_req_in()) got req x1862456998056064 00000100:00100000:3.0:1776178251.622256:0:6256:0:(nrs_fifo.c:179:nrs_fifo_req_get()) NRS start fifo request from 12345-0@lo, seq: 6661 00000100:00100000:3.0:1776178251.622260:0:6256:0:(service.c:2324:ptlrpc_server_handle_request()) Handling RPC req@00000000456acdaf pname:cluuid+ref:pid:xid:nid:opc:job mdt00_001:0+-99:3352:x1862456998055936:12345-0@lo:38:kworker/u8:2.0 00010000:02000000:3.0:1776178251.622266:0:6256:0:(ldlm_lib.c:1139:target_handle_connect()) lustre-MDT0000: Not available for connect from 0@lo (stopping) 00000100:00100000:3.0:1776178251.622290:0:6256:0:(service.c:2372:ptlrpc_server_handle_request()) Handled RPC req@00000000456acdaf pname:cluuid+ref:pid:xid:nid:opc:job mdt00_001:0+-99:3352:x1862456998055936:12345-0@lo:38:kworker/u8:2.0 Request processed in 29us (464us total) trans 0 rc -19/-19 00000100:00100000:3.0:1776178251.622296:0:6256:0:(nrs_fifo.c:241:nrs_fifo_req_stop()) NRS stop fifo request from 12345-0@lo, seq: 6661 00000100:00100000:3.0:1776178251.622306:0:6256:0:(nrs_fifo.c:179:nrs_fifo_req_get()) NRS start fifo request from 12345-0@lo, seq: 6662 00000100:00100000:3.0:1776178251.622309:0:6256:0:(service.c:2324:ptlrpc_server_handle_request()) Handling RPC req@0000000022d4dc29 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_001:0+-99:3352:x1862456998056064:12345-0@lo:38:kworker/u8:2.0 00010000:02000000:3.0:1776178251.622315:0:6256:0:(ldlm_lib.c:1139:target_handle_connect()) lustre-MDT0000: Not available for connect from 0@lo (stopping) 00000100:00100000:3.0:1776178251.622334:0:6256:0:(service.c:2372:ptlrpc_server_handle_request()) Handled RPC req@0000000022d4dc29 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_001:0+-99:3352:x1862456998056064:12345-0@lo:38:kworker/u8:2.0 Request processed in 24us (440us total) trans 0 rc -19/-19 00000100:00100000:3.0:1776178251.622340:0:6256:0:(nrs_fifo.c:241:nrs_fifo_req_stop()) NRS stop fifo request from 12345-0@lo, seq: 6662 00000100:00080000:3.0:1776178251.622375:0:3352:0:(import.c:86:import_set_state_nolock()) 0000000047601a98 lustre-MDT0000_UUID: changing import state from CONNECTING to DISCONN 00000100:00080000:3.0:1776178251.622380:0:3352:0:(import.c:1429:ptlrpc_connect_interpret()) recovery of lustre-MDT0000_UUID on 192.168.204.103@tcp failed (-19) 00000100:00100000:3.0:1776178251.622384:0:3352:0:(client.c:2215:ptlrpc_check_set()) Completed RPC req@000000000390171d pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:lustre-MDT0000-lwp-OST0000_UUID:3352:1862456998055936:0@lo:38:kworker/u8:2.0 00000100:00080000:3.0:1776178251.622393:0:3352:0:(import.c:86:import_set_state_nolock()) 00000000dfc8db13 lustre-MDT0000_UUID: changing import state from CONNECTING to DISCONN 00000100:00080000:3.0:1776178251.622395:0:3352:0:(import.c:1429:ptlrpc_connect_interpret()) recovery of lustre-MDT0000_UUID on 192.168.204.103@tcp failed (-19) 00000100:00100000:3.0:1776178251.622397:0:3352:0:(client.c:2215:ptlrpc_check_set()) Completed RPC req@00000000ad01e608 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:lustre-MDT0000-lwp-OST0001_UUID:3352:1862456998056064:0@lo:38:kworker/u8:2.0 00000100:00080000:3.0:1776178251.622406:0:3352:0:(import.c:86:import_set_state_nolock()) 00000000ba4f1821 lustre-MDT0000_UUID: changing import state from CONNECTING to DISCONN 00000100:00080000:3.0:1776178251.622409:0:3352:0:(import.c:1429:ptlrpc_connect_interpret()) recovery of lustre-MDT0000_UUID on 192.168.204.103@tcp failed (-19) 00000100:00100000:3.0:1776178251.622411:0:3352:0:(client.c:2215:ptlrpc_check_set()) Completed RPC req@000000008f4a0165 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:lustre-MDT0001-mdtlov_UUID:3352:1862456998056192:0@lo:38:kworker/u8:2.0 00000100:00100000:0.0:1776178251.623163:0:3355:0:(client.c:1733:ptlrpc_send_new_req()) Sending RPC req@000000001e11faca pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_02:lustre-MDT0001-mdtlov_UUID:3355:1862456998056256:0@lo:400:kworker/u8:2.0 00000100:00100000:0.0:1776178251.623208:0:3355:0:(events.c:361:request_in_callback()) peer: 12345-0@lo (source: 12345-0@lo) 00000100:00100000:0.0:1776178251.623237:0:3355:0:(client.c:1733:ptlrpc_send_new_req()) Sending RPC req@00000000a3697847 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_02:ac61f839-0e8e-4cac-9c59-08bcfbbad330:3355:1862456998055872:0@lo:400:kworker/u8:2.0 00000100:00100000:0.0:1776178251.623261:0:3355:0:(events.c:361:request_in_callback()) peer: 12345-0@lo (source: 12345-0@lo) 00000100:00100000:1.0:1776178251.623341:0:20140:0:(service.c:2152:ptlrpc_server_handle_req_in()) got req x1862456998055872 00000100:00100000:1.0:1776178251.623386:0:20140:0:(nrs_fifo.c:179:nrs_fifo_req_get()) NRS start fifo request from 12345-0@lo, seq: 174 00000100:00100000:1.0:1776178251.623390:0:20140:0:(service.c:2324:ptlrpc_server_handle_request()) Handling RPC req@0000000048e8901d pname:cluuid+ref:pid:xid:nid:opc:job ll_mgs_0002:ac61f839-0e8e-4cac-9c59-08bcfbbad330+18:3355:x1862456998055872:12345-0@lo:400:kworker/u8:2.0 00000100:00100000:1.0:1776178251.623430:0:20140:0:(service.c:2372:ptlrpc_server_handle_request()) Handled RPC req@0000000048e8901d pname:cluuid+ref:pid:xid:nid:opc:job ll_mgs_0002:ac61f839-0e8e-4cac-9c59-08bcfbbad330+18:3355:x1862456998055872:12345-0@lo:400:kworker/u8:2.0 Request processed in 39us (169us total) trans 0 rc 0/0 00000100:00100000:1.0:1776178251.623436:0:20140:0:(nrs_fifo.c:241:nrs_fifo_req_stop()) NRS stop fifo request from 12345-0@lo, seq: 174 00000100:00100000:3.0:1776178251.624101:0:20759:0:(service.c:2152:ptlrpc_server_handle_req_in()) got req x1862456998056256 00000100:00100000:3.0:1776178251.624144:0:20759:0:(nrs_fifo.c:179:nrs_fifo_req_get()) NRS start fifo request from 12345-0@lo, seq: 549 00000100:00100000:3.0:1776178251.624148:0:20759:0:(service.c:2324:ptlrpc_server_handle_request()) Handling RPC req@0000000045560455 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_010:lustre-MDT0001-mdtlov_UUID+5:3355:x1862456998056256:12345-0@lo:400:kworker/u8:2.0 00000100:00100000:3.0:1776178251.624195:0:20759:0:(service.c:2372:ptlrpc_server_handle_request()) Handled RPC req@0000000045560455 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_010:lustre-MDT0001-mdtlov_UUID+5:3355:x1862456998056256:12345-0@lo:400:kworker/u8:2.0 Request processed in 46us (987us total) trans 0 rc 0/0 00000100:00100000:3.0:1776178251.624201:0:20759:0:(nrs_fifo.c:241:nrs_fifo_req_stop()) NRS stop fifo request from 12345-0@lo, seq: 549 00000100:00100000:0.0:1776178251.624418:0:3355:0:(client.c:2215:ptlrpc_check_set()) Completed RPC req@000000001e11faca pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_02:lustre-MDT0001-mdtlov_UUID:3355:1862456998056256:0@lo:400:kworker/u8:2.0 00000100:00100000:0.0:1776178251.624430:0:3355:0:(client.c:2215:ptlrpc_check_set()) Completed RPC req@00000000a3697847 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_02:ac61f839-0e8e-4cac-9c59-08bcfbbad330:3355:1862456998055872:0@lo:400:kworker/u8:2.0 00000100:00080000:2.0:1776178251.626154:0:37:0:(pinger.c:251:ptlrpc_pinger_process_import()) lustre-MDT0000-lwp-MDT0001_UUID->lustre-MDT0000_UUID: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00080000:2.0:1776178251.626161:0:37:0:(recover.c:57:ptlrpc_initiate_recovery()) lustre-MDT0000_UUID: starting recovery 00000100:00080000:2.0:1776178251.626165:0:37:0:(import.c:86:import_set_state_nolock()) 00000000a7c55b1f lustre-MDT0000_UUID: changing import state from DISCONN to CONNECTING 00000100:00080000:2.0:1776178251.626172:0:37:0:(import.c:534:import_select_connection()) lustre-MDT0000-lwp-MDT0001: connect to NID 0@lo last attempt 953 00000100:00080000:2.0:1776178251.626175:0:37:0:(import.c:581:import_select_connection()) lustre-MDT0000-lwp-MDT0001: tried all connections, increasing latency to 20s 00000100:00080000:2.0:1776178251.626181:0:37:0:(import.c:615:import_select_connection()) lustre-MDT0000-lwp-MDT0001: import 00000000a7c55b1f using connection 192.168.204.103@tcp/0@lo 00000100:00100000:2.0:1776178251.626205:0:37:0:(import.c:807:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5) req@0000000037059d8f x1862456998056384/t0(0) o38->lustre-MDT0000-lwp-MDT0001@0@lo:12/10 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' 00000100:00100000:1.0:1776178251.626320:0:3356:0:(client.c:1733:ptlrpc_send_new_req()) Sending RPC req@000000001e11faca pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_03:lustre-MDT0001-mdtlov_UUID:3356:1862456998056320:0@lo:400:kworker/u8:2.0 00000100:00100000:1.0:1776178251.626378:0:3356:0:(events.c:361:request_in_callback()) peer: 12345-0@lo (source: 12345-0@lo) 00000100:00100000:3.0:1776178251.626570:0:3352:0:(client.c:1733:ptlrpc_send_new_req()) Sending RPC req@0000000037059d8f pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:lustre-MDT0000-lwp-MDT0001_UUID:3352:1862456998056384:0@lo:38:kworker/u8:2.0 00000100:00100000:3.0:1776178251.626607:0:3352:0:(events.c:361:request_in_callback()) peer: 12345-0@lo (source: 12345-0@lo) 00000100:00100000:1.0:1776178251.626852:0:6255:0:(service.c:2152:ptlrpc_server_handle_req_in()) got req x1862456998056384 00000100:00100000:1.0:1776178251.626894:0:6255:0:(nrs_fifo.c:179:nrs_fifo_req_get()) NRS start fifo request from 12345-0@lo, seq: 6663 00000100:00100000:1.0:1776178251.626899:0:6255:0:(service.c:2324:ptlrpc_server_handle_request()) Handling RPC req@000000008cfd048c pname:cluuid+ref:pid:xid:nid:opc:job mdt00_000:0+-99:3352:x1862456998056384:12345-0@lo:38:kworker/u8:2.0 00010000:02000000:1.0:1776178251.626914:0:6255:0:(ldlm_lib.c:1139:target_handle_connect()) lustre-MDT0000: Not available for connect from 0@lo (stopping) 00000100:00100000:1.0:1776178251.626952:0:6255:0:(service.c:2372:ptlrpc_server_handle_request()) Handled RPC req@000000008cfd048c pname:cluuid+ref:pid:xid:nid:opc:job mdt00_000:0+-99:3352:x1862456998056384:12345-0@lo:38:kworker/u8:2.0 Request processed in 52us (345us total) trans 0 rc -19/-19 00000100:00100000:1.0:1776178251.626960:0:6255:0:(nrs_fifo.c:241:nrs_fifo_req_stop()) NRS stop fifo request from 12345-0@lo, seq: 6663 00000100:00100000:2.0:1776178251.628038:0:20755:0:(service.c:2152:ptlrpc_server_handle_req_in()) got req x1862456998056320 00000100:00100000:2.0:1776178251.628117:0:20755:0:(nrs_fifo.c:179:nrs_fifo_req_get()) NRS start fifo request from 12345-0@lo, seq: 550 00000100:00100000:2.0:1776178251.628122:0:20755:0:(service.c:2324:ptlrpc_server_handle_request()) Handling RPC req@00000000d17c6101 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_006:lustre-MDT0001-mdtlov_UUID+5:3356:x1862456998056320:12345-0@lo:400:kworker/u8:2.0 00000100:00100000:2.0:1776178251.628195:0:20755:0:(service.c:2372:ptlrpc_server_handle_request()) Handled RPC req@00000000d17c6101 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_006:lustre-MDT0001-mdtlov_UUID+5:3356:x1862456998056320:12345-0@lo:400:kworker/u8:2.0 Request processed in 72us (1817us total) trans 0 rc 0/0 00000100:00100000:2.0:1776178251.628202:0:20755:0:(nrs_fifo.c:241:nrs_fifo_req_stop()) NRS stop fifo request from 12345-0@lo, seq: 550 00000100:00100000:2.0:1776178251.628250:0:3356:0:(client.c:2215:ptlrpc_check_set()) Completed RPC req@000000001e11faca pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_03:lustre-MDT0001-mdtlov_UUID:3356:1862456998056320:0@lo:400:kworker/u8:2.0 00000100:00080000:3.0:1776178251.628423:0:3352:0:(import.c:86:import_set_state_nolock()) 00000000a7c55b1f lustre-MDT0000_UUID: changing import state from CONNECTING to DISCONN 00000100:00080000:3.0:1776178251.628428:0:3352:0:(import.c:1429:ptlrpc_connect_interpret()) recovery of lustre-MDT0000_UUID on 192.168.204.103@tcp failed (-19) 00000100:00100000:3.0:1776178251.628432:0:3352:0:(client.c:2215:ptlrpc_check_set()) Completed RPC req@0000000037059d8f pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:lustre-MDT0000-lwp-MDT0001_UUID:3352:1862456998056384:0@lo:38:kworker/u8:2.0 00010000:00010000:2.0:1776178252.257939:0:9224:0:(ldlm_lockd.c:2792:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:0.0:1776178253.152326:0:3354:0:(ptlrpcd.c:415:ptlrpcd_check()) transfer 1 async RPCs [0->1] 00000100:00100000:0.0:1776178253.152351:0:3354:0:(client.c:1733:ptlrpc_send_new_req()) Sending RPC req@0000000026195e8c pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_01:lustre-MDT0001-mdtlov_UUID:3354:1862456998056448:0@lo:13:osp-pre-1-1.0 00000100:00100000:0.0:1776178253.152424:0:3354:0:(events.c:361:request_in_callback()) peer: 12345-0@lo (source: 12345-0@lo) 00000100:00100000:3.0:1776178253.152521:0:8231:0:(service.c:2152:ptlrpc_server_handle_req_in()) got req x1862456998056448 00000100:00100000:3.0:1776178253.152573:0:8231:0:(nrs_fifo.c:179:nrs_fifo_req_get()) NRS start fifo request from 12345-0@lo, seq: 507 00000100:00100000:3.0:1776178253.152579:0:8231:0:(service.c:2324:ptlrpc_server_handle_request()) Handling RPC req@0000000003506921 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_create00:lustre-MDT0001-mdtlov_UUID+5:3354:x1862456998056448:12345-0@lo:13:osp-pre-1-1.0 00002000:00000024:3.0:1776178253.152610:0:8231:0:(ofd_obd.c:740:ofd_statfs()) blocks cached 229376 granted 491746368 pending 0 free 3923689472 avail 3692167168 00000100:00100000:3.0:1776178253.152667:0:8231:0:(service.c:2372:ptlrpc_server_handle_request()) Handled RPC req@0000000003506921 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_create00:lustre-MDT0001-mdtlov_UUID+5:3354:x1862456998056448:12345-0@lo:13:osp-pre-1-1.0 Request processed in 86us (243us total) trans 0 rc 0/0 00000100:00100000:3.0:1776178253.152674:0:8231:0:(nrs_fifo.c:241:nrs_fifo_req_stop()) NRS stop fifo request from 12345-0@lo, seq: 507 00000100:00100000:2.0:1776178253.152936:0:3354:0:(client.c:2215:ptlrpc_check_set()) Completed RPC req@0000000026195e8c pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_01:lustre-MDT0001-mdtlov_UUID:3354:1862456998056448:0@lo:13:osp-pre-1-1.0 00010000:00010000:2.0:1776178253.280289:0:9225:0:(ldlm_lockd.c:2792:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:1.0:1776178253.664748:0:3353:0:(ptlrpcd.c:415:ptlrpcd_check()) transfer 1 async RPCs [1->0] 00000100:00100000:1.0:1776178253.665489:0:3353:0:(client.c:1733:ptlrpc_send_new_req()) Sending RPC req@0000000022c7a99e pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_00:lustre-MDT0001-mdtlov_UUID:3353:1862456998056512:0@lo:13:osp-pre-0-1.0 00000100:00100000:1.0:1776178253.665581:0:3353:0:(events.c:361:request_in_callback()) peer: 12345-0@lo (source: 12345-0@lo) 00000100:00100000:0.0:1776178253.665734:0:9554:0:(service.c:2152:ptlrpc_server_handle_req_in()) got req x1862456998056512 00000100:00100000:3.0:1776178253.665953:0:8231:0:(nrs_fifo.c:179:nrs_fifo_req_get()) NRS start fifo request from 12345-0@lo, seq: 508 00000100:00100000:3.0:1776178253.665961:0:8231:0:(service.c:2324:ptlrpc_server_handle_request()) Handling RPC req@00000000c59ba3a2 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_create00:lustre-MDT0001-mdtlov_UUID+5:3353:x1862456998056512:12345-0@lo:13:osp-pre-0-1.0 00002000:00000024:3.0:1776178253.665996:0:8231:0:(ofd_obd.c:740:ofd_statfs()) blocks cached 315392 granted 484883776 pending 0 free 3923689472 avail 3692167168 00000100:00100000:3.0:1776178253.666109:0:8231:0:(service.c:2372:ptlrpc_server_handle_request()) Handled RPC req@00000000c59ba3a2 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_create00:lustre-MDT0001-mdtlov_UUID+5:3353:x1862456998056512:12345-0@lo:13:osp-pre-0-1.0 Request processed in 147us (528us total) trans 0 rc 0/0 00000100:00100000:3.0:1776178253.666116:0:8231:0:(nrs_fifo.c:241:nrs_fifo_req_stop()) NRS stop fifo request from 12345-0@lo, seq: 508 00000100:00100000:1.0:1776178253.668551:0:3353:0:(client.c:2215:ptlrpc_check_set()) Completed RPC req@0000000022c7a99e pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_00:lustre-MDT0001-mdtlov_UUID:3353:1862456998056512:0@lo:13:osp-pre-0-1.0 00010000:00010000:0.0:1776178254.304242:0:6244:0:(ldlm_lockd.c:2792:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:0.0:1776178255.146695:0:3348:0:(events.c:361:request_in_callback()) peer: 12345-192.168.204.3@tcp (source: 12345-192.168.204.3@tcp) 00000100:00100000:0.0:1776178255.146759:0:3348:0:(events.c:361:request_in_callback()) peer: 12345-192.168.204.3@tcp (source: 12345-192.168.204.3@tcp) 00000100:00100000:0.0:1776178255.146805:0:3348:0:(events.c:361:request_in_callback()) peer: 12345-192.168.204.3@tcp (source: 12345-192.168.204.3@tcp) 00000100:00100000:0.0:1776178255.146847:0:3348:0:(events.c:361:request_in_callback()) peer: 12345-192.168.204.3@tcp (source: 12345-192.168.204.3@tcp) 00000100:00100000:3.0:1776178255.146880:0:8250:0:(service.c:2152:ptlrpc_server_handle_req_in()) got req x1862456979112320 00000100:00100000:3.0:1776178255.146938:0:8250:0:(service.c:2152:ptlrpc_server_handle_req_in()) got req x1862456979112256 00000100:00100000:0.0:1776178255.146959:0:3348:0:(events.c:361:request_in_callback()) peer: 12345-192.168.204.3@tcp (source: 12345-192.168.204.3@tcp) 00000100:00100000:3.0:1776178255.146963:0:8250:0:(nrs_fifo.c:179:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.204.3@tcp, seq: 958 00000100:00100000:3.0:1776178255.146973:0:8250:0:(service.c:2324:ptlrpc_server_handle_request()) Handling RPC req@0000000007572592 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_003:b50963bd-8eba-4432-9a54-8888d00e3fa1+6:2270:x1862456979112320:12345-192.168.204.3@tcp:400:kworker/u8:3.0 00000100:00100000:0.0:1776178255.147013:0:6257:0:(nrs_fifo.c:179:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.204.3@tcp, seq: 6664 00000100:00100000:0.0:1776178255.147019:0:6257:0:(service.c:2324:ptlrpc_server_handle_request()) Handling RPC req@0000000022c7a99e pname:cluuid+ref:pid:xid:nid:opc:job mdt00_002:0+-99:2269:x1862456979112256:12345-192.168.204.3@tcp:38:kworker/u8:3.0 00010000:02000000:0.0:1776178255.147034:0:6257:0:(ldlm_lib.c:1139:target_handle_connect()) lustre-MDT0000: Not available for connect from 192.168.204.3@tcp (stopping) 00000100:00100000:3.0:1776178255.147097:0:8250:0:(service.c:2372:ptlrpc_server_handle_request()) Handled RPC req@0000000007572592 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_003:b50963bd-8eba-4432-9a54-8888d00e3fa1+6:2270:x1862456979112320:12345-192.168.204.3@tcp:400:kworker/u8:3.0 Request processed in 124us (419us total) trans 0 rc 0/0 00000100:00100000:3.0:1776178255.147106:0:8250:0:(nrs_fifo.c:241:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.204.3@tcp, seq: 958 00000100:00100000:2.0:1776178255.147261:0:20137:0:(service.c:2152:ptlrpc_server_handle_req_in()) got req x1862456979112192 00000100:00100000:2.0:1776178255.147297:0:20137:0:(nrs_fifo.c:179:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.204.3@tcp, seq: 175 00000100:00100000:2.0:1776178255.147302:0:20137:0:(service.c:2324:ptlrpc_server_handle_request()) Handling RPC req@00000000c201d06e pname:cluuid+ref:pid:xid:nid:opc:job ll_mgs_0000:5b5c66b1-1a2f-4651-8548-0c75b9a38a94+9:2271:x1862456979112192:12345-192.168.204.3@tcp:400:kworker/u8:3.0 00000100:00100000:0.0:1776178255.147877:0:20759:0:(service.c:2152:ptlrpc_server_handle_req_in()) got req x1862456979112384 00000100:00100000:0.0:1776178255.147905:0:20759:0:(service.c:2152:ptlrpc_server_handle_req_in()) got req x1862456979112448 00000100:00100000:0.0:1776178255.147923:0:20759:0:(nrs_fifo.c:179:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.204.3@tcp, seq: 551 00000100:00100000:0.0:1776178255.147927:0:20759:0:(service.c:2324:ptlrpc_server_handle_request()) Handling RPC req@0000000007880806 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_010:b50963bd-8eba-4432-9a54-8888d00e3fa1+8:2271:x1862456979112384:12345-192.168.204.3@tcp:400:kworker/u8:3.0 00000100:00100000:0.0:1776178255.147960:0:20759:0:(service.c:2372:ptlrpc_server_handle_request()) Handled RPC req@0000000007880806 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_010:b50963bd-8eba-4432-9a54-8888d00e3fa1+8:2271:x1862456979112384:12345-192.168.204.3@tcp:400:kworker/u8:3.0 Request processed in 31us (1113us total) trans 0 rc 0/0 00000100:00100000:0.0:1776178255.147966:0:20759:0:(nrs_fifo.c:241:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.204.3@tcp, seq: 551 00000100:00100000:0.0:1776178255.147978:0:20759:0:(nrs_fifo.c:179:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.204.3@tcp, seq: 552 00000100:00100000:0.0:1776178255.147982:0:20759:0:(service.c:2324:ptlrpc_server_handle_request()) Handling RPC req@00000000a604eea4 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_010:b50963bd-8eba-4432-9a54-8888d00e3fa1+9:2270:x1862456979112448:12345-192.168.204.3@tcp:400:kworker/u8:3.0 00000100:00100000:0.0:1776178255.148003:0:20759:0:(service.c:2372:ptlrpc_server_handle_request()) Handled RPC req@00000000a604eea4 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_010:b50963bd-8eba-4432-9a54-8888d00e3fa1+9:2270:x1862456979112448:12345-192.168.204.3@tcp:400:kworker/u8:3.0 Request processed in 21us (1047us total) trans 0 rc 0/0 00000100:00100000:0.0:1776178255.148008:0:20759:0:(nrs_fifo.c:241:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.204.3@tcp, seq: 552 00000100:00100000:0.0:1776178255.148038:0:6257:0:(service.c:2372:ptlrpc_server_handle_request()) Handled RPC req@0000000022c7a99e pname:cluuid+ref:pid:xid:nid:opc:job mdt00_002:0+-99:2269:x1862456979112256:12345-192.168.204.3@tcp:38:kworker/u8:3.0 Request processed in 1018us (1279us total) trans 0 rc -19/-19 00000100:00100000:0.0:1776178255.148045:0:6257:0:(nrs_fifo.c:241:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.204.3@tcp, seq: 6664 00000100:00100000:2.0:1776178255.154564:0:20137:0:(service.c:2372:ptlrpc_server_handle_request()) Handled RPC req@00000000c201d06e pname:cluuid+ref:pid:xid:nid:opc:job ll_mgs_0000:5b5c66b1-1a2f-4651-8548-0c75b9a38a94+9:2271:x1862456979112192:12345-192.168.204.3@tcp:400:kworker/u8:3.0 Request processed in 7252us (7750us total) trans 0 rc 0/0 00000100:00100000:2.0:1776178255.155475:0:20137:0:(nrs_fifo.c:241:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.204.3@tcp, seq: 175 00010000:00010000:2.0:1776178255.331134:0:6245:0:(ldlm_lockd.c:2792:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00010000:00010000:2.0:1776178256.353509:0:9224:0:(ldlm_lockd.c:2792:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000020:02000400:3.0:1776178256.736243:0:22200:0:(genops.c:1786:obd_exports_barrier()) lustre-MDT0000 is waiting for obd_unlinked_exports more than 16 seconds. The obd refcount = 4. Is it stuck? 00000100:00080000:2.0:1776178256.738216:0:36:0:(pinger.c:251:ptlrpc_pinger_process_import()) lustre-MDT0000-lwp-OST0000_UUID->lustre-MDT0000_UUID: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00080000:2.0:1776178256.738225:0:36:0:(recover.c:57:ptlrpc_initiate_recovery()) lustre-MDT0000_UUID: starting recovery 00000100:00080000:2.0:1776178256.738230:0:36:0:(import.c:86:import_set_state_nolock()) 0000000047601a98 lustre-MDT0000_UUID: changing import state from DISCONN to CONNECTING 00000100:00080000:2.0:1776178256.738240:0:36:0:(import.c:534:import_select_connection()) lustre-MDT0000-lwp-OST0000: connect to NID 0@lo last attempt 958 00000100:00080000:2.0:1776178256.738244:0:36:0:(import.c:581:import_select_connection()) lustre-MDT0000-lwp-OST0000: tried all connections, increasing latency to 20s 00000100:00080000:2.0:1776178256.738251:0:36:0:(import.c:615:import_select_connection()) lustre-MDT0000-lwp-OST0000: import 0000000047601a98 using connection 192.168.204.103@tcp/0@lo 00000100:00100000:2.0:1776178256.738276:0:36:0:(import.c:807:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5) req@00000000bd795500 x1862456998056640/t0(0) o38->lustre-MDT0000-lwp-OST0000@0@lo:12/10 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' 00000100:00080000:2.0:1776178256.738313:0:36:0:(pinger.c:251:ptlrpc_pinger_process_import()) lustre-MDT0000-lwp-OST0001_UUID->lustre-MDT0000_UUID: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00080000:2.0:1776178256.738316:0:36:0:(recover.c:57:ptlrpc_initiate_recovery()) lustre-MDT0000_UUID: starting recovery 00000100:00080000:2.0:1776178256.738317:0:36:0:(import.c:86:import_set_state_nolock()) 00000000dfc8db13 lustre-MDT0000_UUID: changing import state from DISCONN to CONNECTING 00000100:00080000:2.0:1776178256.738322:0:36:0:(import.c:534:import_select_connection()) lustre-MDT0000-lwp-OST0001: connect to NID 0@lo last attempt 958 00000100:00080000:2.0:1776178256.738324:0:36:0:(import.c:581:import_select_connection()) lustre-MDT0000-lwp-OST0001: tried all connections, increasing latency to 20s 00000100:00080000:2.0:1776178256.738327:0:36:0:(import.c:615:import_select_connection()) lustre-MDT0000-lwp-OST0001: import 00000000dfc8db13 using connection 192.168.204.103@tcp/0@lo 00000100:00100000:2.0:1776178256.738337:0:36:0:(import.c:807:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5) req@0000000037059d8f x1862456998056768/t0(0) o38->lustre-MDT0000-lwp-OST0001@0@lo:12/10 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' 00000100:00080000:2.0:1776178256.738362:0:36:0:(pinger.c:251:ptlrpc_pinger_process_import()) lustre-MDT0001-mdtlov_UUID->lustre-MDT0000_UUID: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00080000:2.0:1776178256.738366:0:36:0:(recover.c:57:ptlrpc_initiate_recovery()) lustre-MDT0000_UUID: starting recovery 00000100:00080000:2.0:1776178256.738367:0:36:0:(import.c:86:import_set_state_nolock()) 00000000ba4f1821 lustre-MDT0000_UUID: changing import state from DISCONN to CONNECTING 00000100:00080000:2.0:1776178256.738371:0:36:0:(import.c:534:import_select_connection()) lustre-MDT0000-osp-MDT0001: connect to NID 0@lo last attempt 958 00000100:00080000:2.0:1776178256.738373:0:36:0:(import.c:581:import_select_connection()) lustre-MDT0000-osp-MDT0001: tried all connections, increasing latency to 20s 00000100:00080000:2.0:1776178256.738376:0:36:0:(import.c:615:import_select_connection()) lustre-MDT0000-osp-MDT0001: import 00000000ba4f1821 using connection 192.168.204.103@tcp/0@lo 00000100:00100000:2.0:1776178256.738387:0:36:0:(import.c:807:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5) req@00000000167dc455 x1862456998056896/t0(0) o38->lustre-MDT0000-osp-MDT0001@0@lo:24/4 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' 00000100:00080000:2.0:1776178256.738406:0:36:0:(pinger.c:251:ptlrpc_pinger_process_import()) lustre-MDT0000-lwp-MDT0001_UUID->lustre-MDT0000_UUID: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00080000:2.0:1776178256.738409:0:36:0:(recover.c:57:ptlrpc_initiate_recovery()) lustre-MDT0000_UUID: starting recovery 00000100:00080000:2.0:1776178256.738410:0:36:0:(import.c:86:import_set_state_nolock()) 00000000a7c55b1f lustre-MDT0000_UUID: changing import state from DISCONN to CONNECTING 00000100:00080000:2.0:1776178256.738413:0:36:0:(import.c:534:import_select_connection()) lustre-MDT0000-lwp-MDT0001: connect to NID 0@lo last attempt 958 00000100:00080000:2.0:1776178256.738415:0:36:0:(import.c:581:import_select_connection()) lustre-MDT0000-lwp-MDT0001: tried all connections, increasing latency to 20s 00000100:00080000:2.0:1776178256.738418:0:36:0:(import.c:615:import_select_connection()) lustre-MDT0000-lwp-MDT0001: import 00000000a7c55b1f using connection 192.168.204.103@tcp/0@lo 00000100:00100000:2.0:1776178256.738427:0:36:0:(import.c:807:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5) req@0000000018082495 x1862456998057088/t0(0) o38->lustre-MDT0000-lwp-MDT0001@0@lo:12/10 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' 00000100:00100000:2.0:1776178256.738461:0:3354:0:(client.c:1733:ptlrpc_send_new_req()) Sending RPC req@00000000bcfff430 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_01:lustre-MDT0001-mdtlov_UUID:3354:1862456998056960:0@lo:400:kworker/u8:1.0 00000100:00100000:2.0:1776178256.738542:0:3354:0:(events.c:361:request_in_callback()) peer: 12345-0@lo (source: 12345-0@lo) 00000100:00100000:2.0:1776178256.738580:0:3354:0:(ptlrpcd.c:415:ptlrpcd_check()) transfer 1 async RPCs [0->1] 00000100:00100000:2.0:1776178256.738586:0:3354:0:(client.c:1733:ptlrpc_send_new_req()) Sending RPC req@000000008e924027 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_01:lustre-MDT0001-lwp-OST0001_UUID:3354:1862456998056832:0@lo:400:kworker/u8:1.0 00000100:00100000:2.0:1776178256.738615:0:3354:0:(events.c:361:request_in_callback()) peer: 12345-0@lo (source: 12345-0@lo) 00000100:00100000:2.0:1776178256.738657:0:6257:0:(service.c:2152:ptlrpc_server_handle_req_in()) got req x1862456998056832 00000100:00100000:2.0:1776178256.738726:0:6257:0:(nrs_fifo.c:179:nrs_fifo_req_get()) NRS start fifo request from 12345-0@lo, seq: 959 00000100:00100000:2.0:1776178256.738734:0:6257:0:(service.c:2324:ptlrpc_server_handle_request()) Handling RPC req@000000008569a159 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_002:lustre-MDT0001-lwp-OST0001_UUID+5:3354:x1862456998056832:12345-0@lo:400:kworker/u8:1.0 00000100:00100000:2.0:1776178256.738790:0:6257:0:(service.c:2372:ptlrpc_server_handle_request()) Handled RPC req@000000008569a159 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_002:lustre-MDT0001-lwp-OST0001_UUID+5:3354:x1862456998056832:12345-0@lo:400:kworker/u8:1.0 Request processed in 55us (175us total) trans 0 rc 0/0 00000100:00100000:2.0:1776178256.738797:0:6257:0:(nrs_fifo.c:241:nrs_fifo_req_stop()) NRS stop fifo request from 12345-0@lo, seq: 959 00000100:00100000:2.0:1776178256.738856:0:3354:0:(client.c:2215:ptlrpc_check_set()) Completed RPC req@000000008e924027 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_01:lustre-MDT0001-lwp-OST0001_UUID:3354:1862456998056832:0@lo:400:kworker/u8:1.0 00000100:00100000:2.0:1776178256.738896:0:3352:0:(client.c:1733:ptlrpc_send_new_req()) Sending RPC req@00000000bd795500 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:lustre-MDT0000-lwp-OST0000_UUID:3352:1862456998056640:0@lo:38:kworker/u8:1.0 00000100:00100000:2.0:1776178256.738919:0:3352:0:(events.c:361:request_in_callback()) peer: 12345-0@lo (source: 12345-0@lo) 00000100:00100000:2.0:1776178256.738932:0:3352:0:(client.c:1733:ptlrpc_send_new_req()) Sending RPC req@0000000037059d8f pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:lustre-MDT0000-lwp-OST0001_UUID:3352:1862456998056768:0@lo:38:kworker/u8:1.0 00000100:00100000:2.0:1776178256.738948:0:3352:0:(events.c:361:request_in_callback()) peer: 12345-0@lo (source: 12345-0@lo) 00000100:00100000:2.0:1776178256.738957:0:3352:0:(client.c:1733:ptlrpc_send_new_req()) Sending RPC req@00000000167dc455 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:lustre-MDT0001-mdtlov_UUID:3352:1862456998056896:0@lo:38:kworker/u8:1.0 00000100:00100000:2.0:1776178256.738979:0:3352:0:(events.c:361:request_in_callback()) peer: 12345-0@lo (source: 12345-0@lo) 00000100:00100000:2.0:1776178256.738993:0:3352:0:(client.c:1733:ptlrpc_send_new_req()) Sending RPC req@0000000018082495 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:lustre-MDT0000-lwp-MDT0001_UUID:3352:1862456998057088:0@lo:38:kworker/u8:1.0 00000100:00100000:2.0:1776178256.739012:0:3352:0:(events.c:361:request_in_callback()) peer: 12345-0@lo (source: 12345-0@lo) 00000100:00100000:2.0:1776178256.739041:0:15213:0:(service.c:2152:ptlrpc_server_handle_req_in()) got req x1862456998056896 00000100:00100000:2.0:1776178256.739092:0:15213:0:(nrs_fifo.c:179:nrs_fifo_req_get()) NRS start fifo request from 12345-0@lo, seq: 718 00000100:00100000:2.0:1776178256.739097:0:15213:0:(service.c:2324:ptlrpc_server_handle_request()) Handling RPC req@00000000b8464782 pname:cluuid+ref:pid:xid:nid:opc:job mdt_out00_003:0+-99:3352:x1862456998056896:12345-0@lo:38:kworker/u8:1.0 00010000:02000000:2.0:1776178256.739115:0:15213:0:(ldlm_lib.c:1139:target_handle_connect()) lustre-MDT0000: Not available for connect from 0@lo (stopping) 00000100:00100000:2.0:1776178256.739148:0:15213:0:(service.c:2372:ptlrpc_server_handle_request()) Handled RPC req@00000000b8464782 pname:cluuid+ref:pid:xid:nid:opc:job mdt_out00_003:0+-99:3352:x1862456998056896:12345-0@lo:38:kworker/u8:1.0 Request processed in 50us (168us total) trans 0 rc -19/-19 00000100:00100000:2.0:1776178256.739153:0:15213:0:(nrs_fifo.c:241:nrs_fifo_req_stop()) NRS stop fifo request from 12345-0@lo, seq: 718 00000100:00100000:2.0:1776178256.739175:0:8250:0:(service.c:2152:ptlrpc_server_handle_req_in()) got req x1862456998056640 00000100:00100000:2.0:1776178256.739192:0:8250:0:(service.c:2152:ptlrpc_server_handle_req_in()) got req x1862456998056768 00000100:00100000:2.0:1776178256.739205:0:8250:0:(service.c:2152:ptlrpc_server_handle_req_in()) got req x1862456998057088 00000100:00100000:2.0:1776178256.739212:0:8250:0:(nrs_fifo.c:179:nrs_fifo_req_get()) NRS start fifo request from 12345-0@lo, seq: 6665 00000100:00100000:2.0:1776178256.739215:0:8250:0:(service.c:2324:ptlrpc_server_handle_request()) Handling RPC req@000000008e924027 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_003:0+-99:3352:x1862456998056640:12345-0@lo:38:kworker/u8:1.0 00010000:02000000:2.0:1776178256.739221:0:8250:0:(ldlm_lib.c:1139:target_handle_connect()) lustre-MDT0000: Not available for connect from 0@lo (stopping) 00000100:00100000:2.0:1776178256.739243:0:8250:0:(service.c:2372:ptlrpc_server_handle_request()) Handled RPC req@000000008e924027 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_003:0+-99:3352:x1862456998056640:12345-0@lo:38:kworker/u8:1.0 Request processed in 27us (323us total) trans 0 rc -19/-19 00000100:00100000:2.0:1776178256.739249:0:8250:0:(nrs_fifo.c:241:nrs_fifo_req_stop()) NRS stop fifo request from 12345-0@lo, seq: 6665 00000100:00100000:2.0:1776178256.739259:0:8250:0:(nrs_fifo.c:179:nrs_fifo_req_get()) NRS start fifo request from 12345-0@lo, seq: 6666 00000100:00100000:2.0:1776178256.739262:0:8250:0:(service.c:2324:ptlrpc_server_handle_request()) Handling RPC req@00000000032739c9 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_003:0+-99:3352:x1862456998056768:12345-0@lo:38:kworker/u8:1.0 00010000:02000000:2.0:1776178256.739268:0:8250:0:(ldlm_lib.c:1139:target_handle_connect()) lustre-MDT0000: Not available for connect from 0@lo (stopping) 00000100:00100000:2.0:1776178256.739622:0:8250:0:(service.c:2372:ptlrpc_server_handle_request()) Handled RPC req@00000000032739c9 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_003:0+-99:3352:x1862456998056768:12345-0@lo:38:kworker/u8:1.0 Request processed in 355us (669us total) trans 0 rc -19/-19 00000100:00100000:2.0:1776178256.739629:0:8250:0:(nrs_fifo.c:241:nrs_fifo_req_stop()) NRS stop fifo request from 12345-0@lo, seq: 6666 00000100:00100000:2.0:1776178256.739641:0:8250:0:(nrs_fifo.c:179:nrs_fifo_req_get()) NRS start fifo request from 12345-0@lo, seq: 6667 00000100:00100000:2.0:1776178256.739645:0:8250:0:(service.c:2324:ptlrpc_server_handle_request()) Handling RPC req@0000000035fd3386 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_003:0+-99:3352:x1862456998057088:12345-0@lo:38:kworker/u8:1.0 00010000:02000000:2.0:1776178256.739652:0:8250:0:(ldlm_lib.c:1139:target_handle_connect()) lustre-MDT0000: Not available for connect from 0@lo (stopping) 00000100:00100000:2.0:1776178256.739672:0:8250:0:(service.c:2372:ptlrpc_server_handle_request()) Handled RPC req@0000000035fd3386 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_003:0+-99:3352:x1862456998057088:12345-0@lo:38:kworker/u8:1.0 Request processed in 27us (660us total) trans 0 rc -19/-19 00000100:00100000:2.0:1776178256.739677:0:8250:0:(nrs_fifo.c:241:nrs_fifo_req_stop()) NRS stop fifo request from 12345-0@lo, seq: 6667 00000100:00080000:2.0:1776178256.739731:0:3352:0:(import.c:86:import_set_state_nolock()) 0000000047601a98 lustre-MDT0000_UUID: changing import state from CONNECTING to DISCONN 00000100:00080000:2.0:1776178256.739736:0:3352:0:(import.c:1429:ptlrpc_connect_interpret()) recovery of lustre-MDT0000_UUID on 192.168.204.103@tcp failed (-19) 00000100:00100000:2.0:1776178256.739739:0:3352:0:(client.c:2215:ptlrpc_check_set()) Completed RPC req@00000000bd795500 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:lustre-MDT0000-lwp-OST0000_UUID:3352:1862456998056640:0@lo:38:kworker/u8:1.0 00000100:00080000:2.0:1776178256.739749:0:3352:0:(import.c:86:import_set_state_nolock()) 00000000dfc8db13 lustre-MDT0000_UUID: changing import state from CONNECTING to DISCONN 00000100:00080000:2.0:1776178256.739751:0:3352:0:(import.c:1429:ptlrpc_connect_interpret()) recovery of lustre-MDT0000_UUID on 192.168.204.103@tcp failed (-19) 00000100:00100000:2.0:1776178256.739754:0:3352:0:(client.c:2215:ptlrpc_check_set()) Completed RPC req@0000000037059d8f pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:lustre-MDT0000-lwp-OST0001_UUID:3352:1862456998056768:0@lo:38:kworker/u8:1.0 00000100:00080000:2.0:1776178256.739760:0:3352:0:(import.c:86:import_set_state_nolock()) 00000000ba4f1821 lustre-MDT0000_UUID: changing import state from CONNECTING to DISCONN 00000100:00080000:2.0:1776178256.739763:0:3352:0:(import.c:1429:ptlrpc_connect_interpret()) recovery of lustre-MDT0000_UUID on 192.168.204.103@tcp failed (-19) 00000100:00100000:2.0:1776178256.739766:0:3352:0:(client.c:2215:ptlrpc_check_set()) Completed RPC req@00000000167dc455 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:lustre-MDT0001-mdtlov_UUID:3352:1862456998056896:0@lo:38:kworker/u8:1.0 00000100:00080000:2.0:1776178256.739772:0:3352:0:(import.c:86:import_set_state_nolock()) 00000000a7c55b1f lustre-MDT0000_UUID: changing import state from CONNECTING to DISCONN 00000100:00080000:2.0:1776178256.739774:0:3352:0:(import.c:1429:ptlrpc_connect_interpret()) recovery of lustre-MDT0000_UUID on 192.168.204.103@tcp failed (-19) 00000100:00100000:2.0:1776178256.739776:0:3352:0:(client.c:2215:ptlrpc_check_set()) Completed RPC req@0000000018082495 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:lustre-MDT0000-lwp-MDT0001_UUID:3352:1862456998057088:0@lo:38:kworker/u8:1.0 00000100:00100000:0.0:1776178256.739957:0:3355:0:(client.c:1733:ptlrpc_send_new_req()) Sending RPC req@00000000b63b7b84 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_02:ac61f839-0e8e-4cac-9c59-08bcfbbad330:3355:1862456998056576:0@lo:400:kworker/u8:1.0 00000100:00100000:0.0:1776178256.739997:0:3355:0:(events.c:361:request_in_callback()) peer: 12345-0@lo (source: 12345-0@lo) 00000100:00100000:0.0:1776178256.740036:0:3355:0:(client.c:1733:ptlrpc_send_new_req()) Sending RPC req@00000000e83ac5a7 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_02:lustre-MDT0001-mdtlov_UUID:3355:1862456998057024:0@lo:400:kworker/u8:1.0 00000100:00100000:0.0:1776178256.740088:0:3355:0:(events.c:361:request_in_callback()) peer: 12345-0@lo (source: 12345-0@lo) 00000100:00100000:0.0:1776178256.740110:0:3355:0:(ptlrpcd.c:415:ptlrpcd_check()) transfer 1 async RPCs [3->2] 00000100:00100000:0.0:1776178256.740118:0:3355:0:(client.c:1733:ptlrpc_send_new_req()) Sending RPC req@0000000052a7c870 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_02:lustre-MDT0001-lwp-OST0000_UUID:3355:1862456998056704:0@lo:400:kworker/u8:1.0 00000100:00100000:0.0:1776178256.740140:0:3355:0:(events.c:361:request_in_callback()) peer: 12345-0@lo (source: 12345-0@lo) 00000100:00100000:0.0:1776178256.740171:0:14440:0:(service.c:2152:ptlrpc_server_handle_req_in()) got req x1862456998056960 00000100:00100000:0.0:1776178256.740211:0:14440:0:(service.c:2152:ptlrpc_server_handle_req_in()) got req x1862456998057024 00000100:00100000:0.0:1776178256.740231:0:14440:0:(nrs_fifo.c:179:nrs_fifo_req_get()) NRS start fifo request from 12345-0@lo, seq: 553 00000100:00100000:0.0:1776178256.740236:0:14440:0:(service.c:2324:ptlrpc_server_handle_request()) Handling RPC req@00000000044bd507 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_004:lustre-MDT0001-mdtlov_UUID+5:3354:x1862456998056960:12345-0@lo:400:kworker/u8:1.0 00000100:00100000:0.0:1776178256.740282:0:14440:0:(service.c:2372:ptlrpc_server_handle_request()) Handled RPC req@00000000044bd507 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_004:lustre-MDT0001-mdtlov_UUID+5:3354:x1862456998056960:12345-0@lo:400:kworker/u8:1.0 Request processed in 46us (1741us total) trans 0 rc 0/0 00000100:00100000:0.0:1776178256.740289:0:14440:0:(nrs_fifo.c:241:nrs_fifo_req_stop()) NRS stop fifo request from 12345-0@lo, seq: 553 00000100:00100000:0.0:1776178256.740303:0:14440:0:(nrs_fifo.c:179:nrs_fifo_req_get()) NRS start fifo request from 12345-0@lo, seq: 554 00000100:00100000:0.0:1776178256.740306:0:14440:0:(service.c:2324:ptlrpc_server_handle_request()) Handling RPC req@0000000097413742 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_004:lustre-MDT0001-mdtlov_UUID+5:3355:x1862456998057024:12345-0@lo:400:kworker/u8:1.0 00000100:00100000:0.0:1776178256.740335:0:14440:0:(service.c:2372:ptlrpc_server_handle_request()) Handled RPC req@0000000097413742 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_004:lustre-MDT0001-mdtlov_UUID+5:3355:x1862456998057024:12345-0@lo:400:kworker/u8:1.0 Request processed in 28us (248us total) trans 0 rc 0/0 00000100:00100000:0.0:1776178256.740344:0:14440:0:(nrs_fifo.c:241:nrs_fifo_req_stop()) NRS stop fifo request from 12345-0@lo, seq: 554 00000100:00100000:0.0:1776178256.740385:0:3354:0:(client.c:2215:ptlrpc_check_set()) Completed RPC req@00000000bcfff430 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_01:lustre-MDT0001-mdtlov_UUID:3354:1862456998056960:0@lo:400:kworker/u8:1.0 00000100:00100000:0.0:1776178256.740405:0:6257:0:(service.c:2152:ptlrpc_server_handle_req_in()) got req x1862456998056704 00000100:00100000:0.0:1776178256.740430:0:6257:0:(nrs_fifo.c:179:nrs_fifo_req_get()) NRS start fifo request from 12345-0@lo, seq: 960 00000100:00100000:0.0:1776178256.740433:0:6257:0:(service.c:2324:ptlrpc_server_handle_request()) Handling RPC req@00000000bd43c651 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_002:lustre-MDT0001-lwp-OST0000_UUID+5:3355:x1862456998056704:12345-0@lo:400:kworker/u8:1.0 00000100:00100000:0.0:1776178256.740463:0:6257:0:(service.c:2372:ptlrpc_server_handle_request()) Handled RPC req@00000000bd43c651 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_002:lustre-MDT0001-lwp-OST0000_UUID+5:3355:x1862456998056704:12345-0@lo:400:kworker/u8:1.0 Request processed in 28us (322us total) trans 0 rc 0/0 00000100:00100000:0.0:1776178256.740468:0:6257:0:(nrs_fifo.c:241:nrs_fifo_req_stop()) NRS stop fifo request from 12345-0@lo, seq: 960 00000100:00100000:0.0:1776178256.740492:0:3355:0:(client.c:2215:ptlrpc_check_set()) Completed RPC req@0000000052a7c870 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_02:lustre-MDT0001-lwp-OST0000_UUID:3355:1862456998056704:0@lo:400:kworker/u8:1.0 00000100:00100000:0.0:1776178256.740501:0:3355:0:(client.c:2215:ptlrpc_check_set()) Completed RPC req@00000000e83ac5a7 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_02:lustre-MDT0001-mdtlov_UUID:3355:1862456998057024:0@lo:400:kworker/u8:1.0 00000100:00100000:2.0:1776178256.741332:0:20137:0:(service.c:2152:ptlrpc_server_handle_req_in()) got req x1862456998056576 00000100:00100000:2.0:1776178256.741389:0:20137:0:(nrs_fifo.c:179:nrs_fifo_req_get()) NRS start fifo request from 12345-0@lo, seq: 176 00000100:00100000:2.0:1776178256.741394:0:20137:0:(service.c:2324:ptlrpc_server_handle_request()) Handling RPC req@000000009c2c404e pname:cluuid+ref:pid:xid:nid:opc:job ll_mgs_0000:ac61f839-0e8e-4cac-9c59-08bcfbbad330+18:3355:x1862456998056576:12345-0@lo:400:kworker/u8:1.0 00000100:00100000:2.0:1776178256.741461:0:20137:0:(service.c:2372:ptlrpc_server_handle_request()) Handled RPC req@000000009c2c404e pname:cluuid+ref:pid:xid:nid:opc:job ll_mgs_0000:ac61f839-0e8e-4cac-9c59-08bcfbbad330+18:3355:x1862456998056576:12345-0@lo:400:kworker/u8:1.0 Request processed in 67us (1465us total) trans 0 rc 0/0 00000100:00100000:2.0:1776178256.741468:0:20137:0:(nrs_fifo.c:241:nrs_fifo_req_stop()) NRS stop fifo request from 12345-0@lo, seq: 176 00000100:00100000:0.0:1776178256.741615:0:3355:0:(client.c:2215:ptlrpc_check_set()) Completed RPC req@00000000b63b7b84 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_02:ac61f839-0e8e-4cac-9c59-08bcfbbad330:3355:1862456998056576:0@lo:400:kworker/u8:1.0 00000020:02000400:3.0:1776178256.755916:0:22200:0:(genops.c:1752:print_export_data()) lustre-MDT0000: UNLINKED 00000000ca22e26f b50963bd-8eba-4432-9a54-8888d00e3fa1 1 (0 0 0) 1 0 1 0: 0000000000000000 8589934704 stale:0 00010000:00010000:2.0:1776178257.376589:0:9225:0:(ldlm_lockd.c:2792:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:2.0:1776178258.277441:0:3355:0:(ptlrpcd.c:415:ptlrpcd_check()) transfer 1 async RPCs [3->2] 00000100:00100000:2.0:1776178258.277468:0:3355:0:(client.c:1733:ptlrpc_send_new_req()) Sending RPC req@0000000058cd86cb pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_02:lustre-MDT0001-mdtlov_UUID:3355:1862456998057152:0@lo:13:osp-pre-1-1.0 00000100:00100000:2.0:1776178258.277556:0:3355:0:(events.c:361:request_in_callback()) peer: 12345-0@lo (source: 12345-0@lo) 00000100:00100000:3.0:1776178258.277794:0:9554:0:(service.c:2152:ptlrpc_server_handle_req_in()) got req x1862456998057152 00000100:00100000:3.0:1776178258.277852:0:9554:0:(nrs_fifo.c:179:nrs_fifo_req_get()) NRS start fifo request from 12345-0@lo, seq: 509 00000100:00100000:3.0:1776178258.277866:0:9554:0:(service.c:2324:ptlrpc_server_handle_request()) Handling RPC req@0000000018082495 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_create00:lustre-MDT0001-mdtlov_UUID+5:3355:x1862456998057152:12345-0@lo:13:osp-pre-1-1.0 00002000:00000024:3.0:1776178258.277898:0:9554:0:(ofd_obd.c:740:ofd_statfs()) blocks cached 229376 granted 491746368 pending 0 free 3923689472 avail 3692167168 00000100:00100000:3.0:1776178258.277944:0:9554:0:(service.c:2372:ptlrpc_server_handle_request()) Handled RPC req@0000000018082495 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_create00:lustre-MDT0001-mdtlov_UUID+5:3355:x1862456998057152:12345-0@lo:13:osp-pre-1-1.0 Request processed in 86us (389us total) trans 0 rc 0/0 00000100:00100000:3.0:1776178258.277951:0:9554:0:(nrs_fifo.c:241:nrs_fifo_req_stop()) NRS stop fifo request from 12345-0@lo, seq: 509 00000100:00100000:2.0:1776178258.278272:0:3355:0:(client.c:2215:ptlrpc_check_set()) Completed RPC req@0000000058cd86cb pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_02:lustre-MDT0001-mdtlov_UUID:3355:1862456998057152:0@lo:13:osp-pre-1-1.0 00010000:00010000:0.0:1776178258.402571:0:6244:0:(ldlm_lockd.c:2792:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:1.0:1776178258.790281:0:3353:0:(client.c:1733:ptlrpc_send_new_req()) Sending RPC req@000000009383c871 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_00:lustre-MDT0001-mdtlov_UUID:3353:1862456998057216:0@lo:13:osp-pre-0-1.0 00000100:00100000:1.0:1776178258.790372:0:3353:0:(events.c:361:request_in_callback()) peer: 12345-0@lo (source: 12345-0@lo) 00000100:00100000:3.0:1776178258.790532:0:8231:0:(service.c:2152:ptlrpc_server_handle_req_in()) got req x1862456998057216 00000100:00100000:3.0:1776178258.790597:0:8231:0:(nrs_fifo.c:179:nrs_fifo_req_get()) NRS start fifo request from 12345-0@lo, seq: 510 00000100:00100000:3.0:1776178258.790603:0:8231:0:(service.c:2324:ptlrpc_server_handle_request()) Handling RPC req@000000000ff6c1d2 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_create00:lustre-MDT0001-mdtlov_UUID+5:3353:x1862456998057216:12345-0@lo:13:osp-pre-0-1.0 00002000:00000024:3.0:1776178258.790633:0:8231:0:(ofd_obd.c:740:ofd_statfs()) blocks cached 315392 granted 484883776 pending 0 free 3923689472 avail 3692167168 00000100:00100000:3.0:1776178258.790702:0:8231:0:(service.c:2372:ptlrpc_server_handle_request()) Handled RPC req@000000000ff6c1d2 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_create00:lustre-MDT0001-mdtlov_UUID+5:3353:x1862456998057216:12345-0@lo:13:osp-pre-0-1.0 Request processed in 97us (331us total) trans 0 rc 0/0 00000100:00100000:3.0:1776178258.790713:0:8231:0:(nrs_fifo.c:241:nrs_fifo_req_stop()) NRS stop fifo request from 12345-0@lo, seq: 510 00000100:00100000:1.0:1776178258.790895:0:3353:0:(client.c:2215:ptlrpc_check_set()) Completed RPC req@000000009383c871 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_00:lustre-MDT0001-mdtlov_UUID:3353:1862456998057216:0@lo:13:osp-pre-0-1.0 00010000:00010000:2.0:1776178259.427424:0:6245:0:(ldlm_lockd.c:2792:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:0.0:1776178260.259372:0:3347:0:(events.c:361:request_in_callback()) peer: 12345-192.168.204.3@tcp (source: 12345-192.168.204.3@tcp) 00000100:00100000:0.0:1776178260.259447:0:3347:0:(events.c:361:request_in_callback()) peer: 12345-192.168.204.3@tcp (source: 12345-192.168.204.3@tcp) 00000100:00100000:1.0:1776178260.259484:0:6255:0:(service.c:2152:ptlrpc_server_handle_req_in()) got req x1862456979112576 00000100:00100000:1.0:1776178260.259542:0:6255:0:(nrs_fifo.c:179:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.204.3@tcp, seq: 6668 00000100:00100000:1.0:1776178260.259549:0:6255:0:(service.c:2324:ptlrpc_server_handle_request()) Handling RPC req@000000002b0faed1 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_000:0+-99:2269:x1862456979112576:12345-192.168.204.3@tcp:38:kworker/u8:4.0 00010000:02000000:1.0:1776178260.259569:0:6255:0:(ldlm_lib.c:1139:target_handle_connect()) lustre-MDT0000: Not available for connect from 192.168.204.3@tcp (stopping) 00000100:00100000:1.0:1776178260.259655:0:6255:0:(service.c:2372:ptlrpc_server_handle_request()) Handled RPC req@000000002b0faed1 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_000:0+-99:2269:x1862456979112576:12345-192.168.204.3@tcp:38:kworker/u8:4.0 Request processed in 105us (300us total) trans 0 rc -19/-19 00000100:00100000:1.0:1776178260.259662:0:6255:0:(nrs_fifo.c:241:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.204.3@tcp, seq: 6668 00000100:00100000:0.0:1776178260.259882:0:3347:0:(events.c:361:request_in_callback()) peer: 12345-192.168.204.3@tcp (source: 12345-192.168.204.3@tcp) 00000100:00100000:0.0:1776178260.259912:0:3347:0:(events.c:361:request_in_callback()) peer: 12345-192.168.204.3@tcp (source: 12345-192.168.204.3@tcp) 00000100:00100000:0.0:1776178260.259938:0:3347:0:(events.c:361:request_in_callback()) peer: 12345-192.168.204.3@tcp (source: 12345-192.168.204.3@tcp) 00000100:00100000:0.0:1776178260.259967:0:20759:0:(service.c:2152:ptlrpc_server_handle_req_in()) got req x1862456979112704 00000100:00100000:0.0:1776178260.260009:0:20759:0:(service.c:2152:ptlrpc_server_handle_req_in()) got req x1862456979112768 00000100:00100000:2.0:1776178260.260028:0:11281:0:(service.c:2152:ptlrpc_server_handle_req_in()) got req x1862456979112640 00000100:00100000:0.0:1776178260.260033:0:20759:0:(nrs_fifo.c:179:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.204.3@tcp, seq: 555 00000100:00100000:0.0:1776178260.260040:0:20759:0:(service.c:2324:ptlrpc_server_handle_request()) Handling RPC req@000000001879c2bc pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_010:b50963bd-8eba-4432-9a54-8888d00e3fa1+8:2272:x1862456979112704:12345-192.168.204.3@tcp:400:kworker/u8:4.0 00000100:00100000:2.0:1776178260.260109:0:11281:0:(nrs_fifo.c:179:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.204.3@tcp, seq: 961 00000100:00100000:2.0:1776178260.260114:0:11281:0:(service.c:2324:ptlrpc_server_handle_request()) Handling RPC req@00000000e36da5a6 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_004:b50963bd-8eba-4432-9a54-8888d00e3fa1+6:2272:x1862456979112640:12345-192.168.204.3@tcp:400:kworker/u8:4.0 00000100:00100000:2.0:1776178260.260153:0:11281:0:(service.c:2372:ptlrpc_server_handle_request()) Handled RPC req@00000000e36da5a6 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_004:b50963bd-8eba-4432-9a54-8888d00e3fa1+6:2272:x1862456979112640:12345-192.168.204.3@tcp:400:kworker/u8:4.0 Request processed in 38us (241us total) trans 0 rc 0/0 00000100:00100000:2.0:1776178260.260158:0:11281:0:(nrs_fifo.c:241:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.204.3@tcp, seq: 961 00000100:00100000:2.0:1776178260.260181:0:20140:0:(service.c:2152:ptlrpc_server_handle_req_in()) got req x1862456979112512 00000100:00100000:2.0:1776178260.260205:0:20140:0:(nrs_fifo.c:179:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.204.3@tcp, seq: 177 00000100:00100000:2.0:1776178260.260210:0:20140:0:(service.c:2324:ptlrpc_server_handle_request()) Handling RPC req@000000005f72129d pname:cluuid+ref:pid:xid:nid:opc:job ll_mgs_0002:5b5c66b1-1a2f-4651-8548-0c75b9a38a94+9:2272:x1862456979112512:12345-192.168.204.3@tcp:400:kworker/u8:4.0 00000100:00100000:2.0:1776178260.260231:0:20140:0:(service.c:2372:ptlrpc_server_handle_request()) Handled RPC req@000000005f72129d pname:cluuid+ref:pid:xid:nid:opc:job ll_mgs_0002:5b5c66b1-1a2f-4651-8548-0c75b9a38a94+9:2272:x1862456979112512:12345-192.168.204.3@tcp:400:kworker/u8:4.0 Request processed in 22us (786us total) trans 0 rc 0/0 00000100:00100000:2.0:1776178260.260238:0:20140:0:(nrs_fifo.c:241:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.204.3@tcp, seq: 177 00000100:00100000:1.0:1776178260.260356:0:20762:0:(nrs_fifo.c:179:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.204.3@tcp, seq: 556 00000100:00100000:1.0:1776178260.260361:0:20762:0:(service.c:2324:ptlrpc_server_handle_request()) Handling RPC req@0000000046115d91 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_013:b50963bd-8eba-4432-9a54-8888d00e3fa1+9:2272:x1862456979112768:12345-192.168.204.3@tcp:400:kworker/u8:4.0 00000100:00100000:1.0:1776178260.260393:0:20762:0:(service.c:2372:ptlrpc_server_handle_request()) Handled RPC req@0000000046115d91 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_013:b50963bd-8eba-4432-9a54-8888d00e3fa1+9:2272:x1862456979112768:12345-192.168.204.3@tcp:400:kworker/u8:4.0 Request processed in 32us (456us total) trans 0 rc 0/0 00000100:00100000:1.0:1776178260.260401:0:20762:0:(nrs_fifo.c:241:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.204.3@tcp, seq: 556 00000100:00100000:0.0:1776178260.280756:0:20759:0:(service.c:2372:ptlrpc_server_handle_request()) Handled RPC req@000000001879c2bc pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_010:b50963bd-8eba-4432-9a54-8888d00e3fa1+8:2272:x1862456979112704:12345-192.168.204.3@tcp:400:kworker/u8:4.0 Request processed in 20700us (20860us total) trans 0 rc 0/0 00000100:00100000:0.0:1776178260.280779:0:20759:0:(nrs_fifo.c:241:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.204.3@tcp, seq: 555 00010000:00010000:2.0:1776178260.449176:0:9224:0:(ldlm_lockd.c:2792:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00010000:00010000:2.0:1776178261.475576:0:9225:0:(ldlm_lockd.c:2792:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00080000:2.0:1776178261.856834:0:36:0:(pinger.c:251:ptlrpc_pinger_process_import()) lustre-MDT0000-lwp-OST0000_UUID->lustre-MDT0000_UUID: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00080000:2.0:1776178261.856845:0:36:0:(recover.c:57:ptlrpc_initiate_recovery()) lustre-MDT0000_UUID: starting recovery 00000100:00080000:2.0:1776178261.856849:0:36:0:(import.c:86:import_set_state_nolock()) 0000000047601a98 lustre-MDT0000_UUID: changing import state from DISCONN to CONNECTING 00000100:00080000:2.0:1776178261.856860:0:36:0:(import.c:534:import_select_connection()) lustre-MDT0000-lwp-OST0000: connect to NID 0@lo last attempt 964 00000100:00080000:2.0:1776178261.856865:0:36:0:(import.c:581:import_select_connection()) lustre-MDT0000-lwp-OST0000: tried all connections, increasing latency to 20s 00000100:00080000:2.0:1776178261.856872:0:36:0:(import.c:615:import_select_connection()) lustre-MDT0000-lwp-OST0000: import 0000000047601a98 using connection 192.168.204.103@tcp/0@lo 00000100:00100000:2.0:1776178261.856908:0:36:0:(import.c:807:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5) req@0000000078361ddf x1862456998057344/t0(0) o38->lustre-MDT0000-lwp-OST0000@0@lo:12/10 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' 00000100:00080000:2.0:1776178261.856963:0:36:0:(pinger.c:251:ptlrpc_pinger_process_import()) lustre-MDT0000-lwp-OST0001_UUID->lustre-MDT0000_UUID: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00080000:2.0:1776178261.856966:0:36:0:(recover.c:57:ptlrpc_initiate_recovery()) lustre-MDT0000_UUID: starting recovery 00000100:00080000:2.0:1776178261.856968:0:36:0:(import.c:86:import_set_state_nolock()) 00000000dfc8db13 lustre-MDT0000_UUID: changing import state from DISCONN to CONNECTING 00000100:00080000:2.0:1776178261.856971:0:36:0:(import.c:534:import_select_connection()) lustre-MDT0000-lwp-OST0001: connect to NID 0@lo last attempt 964 00000100:00080000:2.0:1776178261.856974:0:36:0:(import.c:581:import_select_connection()) lustre-MDT0000-lwp-OST0001: tried all connections, increasing latency to 20s 00000100:00080000:2.0:1776178261.856976:0:36:0:(import.c:615:import_select_connection()) lustre-MDT0000-lwp-OST0001: import 00000000dfc8db13 using connection 192.168.204.103@tcp/0@lo 00000100:00100000:2.0:1776178261.856987:0:36:0:(import.c:807:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5) req@0000000020afd0b3 x1862456998057472/t0(0) o38->lustre-MDT0000-lwp-OST0001@0@lo:12/10 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' 00000100:00080000:2.0:1776178261.857003:0:36:0:(pinger.c:251:ptlrpc_pinger_process_import()) lustre-MDT0001-mdtlov_UUID->lustre-MDT0000_UUID: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00080000:2.0:1776178261.857006:0:36:0:(recover.c:57:ptlrpc_initiate_recovery()) lustre-MDT0000_UUID: starting recovery 00000100:00080000:2.0:1776178261.857008:0:36:0:(import.c:86:import_set_state_nolock()) 00000000ba4f1821 lustre-MDT0000_UUID: changing import state from DISCONN to CONNECTING 00000100:00080000:2.0:1776178261.857011:0:36:0:(import.c:534:import_select_connection()) lustre-MDT0000-osp-MDT0001: connect to NID 0@lo last attempt 964 00000100:00080000:2.0:1776178261.857013:0:36:0:(import.c:581:import_select_connection()) lustre-MDT0000-osp-MDT0001: tried all connections, increasing latency to 20s 00000100:00080000:2.0:1776178261.857017:0:36:0:(import.c:615:import_select_connection()) lustre-MDT0000-osp-MDT0001: import 00000000ba4f1821 using connection 192.168.204.103@tcp/0@lo 00000100:00100000:2.0:1776178261.857028:0:36:0:(import.c:807:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5) req@00000000638772fd x1862456998057600/t0(0) o38->lustre-MDT0000-osp-MDT0001@0@lo:24/4 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' 00000100:00080000:2.0:1776178261.857047:0:36:0:(pinger.c:251:ptlrpc_pinger_process_import()) lustre-MDT0000-lwp-MDT0001_UUID->lustre-MDT0000_UUID: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00080000:2.0:1776178261.857050:0:36:0:(recover.c:57:ptlrpc_initiate_recovery()) lustre-MDT0000_UUID: starting recovery 00000100:00080000:2.0:1776178261.857051:0:36:0:(import.c:86:import_set_state_nolock()) 00000000a7c55b1f lustre-MDT0000_UUID: changing import state from DISCONN to CONNECTING 00000100:00080000:2.0:1776178261.857055:0:36:0:(import.c:534:import_select_connection()) lustre-MDT0000-lwp-MDT0001: connect to NID 0@lo last attempt 964 00000100:00080000:2.0:1776178261.857056:0:36:0:(import.c:581:import_select_connection()) lustre-MDT0000-lwp-MDT0001: tried all connections, increasing latency to 20s 00000100:00080000:2.0:1776178261.857083:0:36:0:(import.c:615:import_select_connection()) lustre-MDT0000-lwp-MDT0001: import 00000000a7c55b1f using connection 192.168.204.103@tcp/0@lo 00000100:00100000:2.0:1776178261.857094:0:36:0:(import.c:807:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5) req@00000000bcefda18 x1862456998057792/t0(0) o38->lustre-MDT0000-lwp-MDT0001@0@lo:12/10 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' 00000100:00100000:2.0:1776178261.857125:0:3355:0:(client.c:1733:ptlrpc_send_new_req()) Sending RPC req@00000000339a9a57 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_02:lustre-MDT0001-lwp-OST0000_UUID:3355:1862456998057408:0@lo:400:kworker/u8:1.0 00000100:00100000:2.0:1776178261.857200:0:3355:0:(events.c:361:request_in_callback()) peer: 12345-0@lo (source: 12345-0@lo) 00000100:00100000:2.0:1776178261.857245:0:3355:0:(ptlrpcd.c:415:ptlrpcd_check()) transfer 1 async RPCs [3->2] 00000100:00100000:2.0:1776178261.857251:0:3355:0:(client.c:1733:ptlrpc_send_new_req()) Sending RPC req@00000000bdad24ac pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_02:lustre-MDT0001-lwp-OST0001_UUID:3355:1862456998057536:0@lo:400:kworker/u8:1.0 00000100:00100000:2.0:1776178261.857276:0:3355:0:(events.c:361:request_in_callback()) peer: 12345-0@lo (source: 12345-0@lo) 00000100:00100000:2.0:1776178261.857315:0:6256:0:(service.c:2152:ptlrpc_server_handle_req_in()) got req x1862456998057408 00000100:00100000:2.0:1776178261.857376:0:6256:0:(service.c:2152:ptlrpc_server_handle_req_in()) got req x1862456998057536 00000100:00100000:2.0:1776178261.857397:0:6256:0:(nrs_fifo.c:179:nrs_fifo_req_get()) NRS start fifo request from 12345-0@lo, seq: 962 00000100:00100000:2.0:1776178261.857404:0:6256:0:(service.c:2324:ptlrpc_server_handle_request()) Handling RPC req@00000000f586aea3 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_001:lustre-MDT0001-lwp-OST0000_UUID+5:3355:x1862456998057408:12345-0@lo:400:kworker/u8:1.0 00000100:00100000:2.0:1776178261.857468:0:6256:0:(service.c:2372:ptlrpc_server_handle_request()) Handled RPC req@00000000f586aea3 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_001:lustre-MDT0001-lwp-OST0000_UUID+5:3355:x1862456998057408:12345-0@lo:400:kworker/u8:1.0 Request processed in 64us (270us total) trans 0 rc 0/0 00000100:00100000:2.0:1776178261.857476:0:6256:0:(nrs_fifo.c:241:nrs_fifo_req_stop()) NRS stop fifo request from 12345-0@lo, seq: 962 00000100:00100000:2.0:1776178261.857493:0:6256:0:(nrs_fifo.c:179:nrs_fifo_req_get()) NRS start fifo request from 12345-0@lo, seq: 963 00000100:00100000:2.0:1776178261.857497:0:6256:0:(service.c:2324:ptlrpc_server_handle_request()) Handling RPC req@000000006f8bc860 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_001:lustre-MDT0001-lwp-OST0001_UUID+5:3355:x1862456998057536:12345-0@lo:400:kworker/u8:1.0 00000100:00100000:2.0:1776178261.857523:0:6256:0:(service.c:2372:ptlrpc_server_handle_request()) Handled RPC req@000000006f8bc860 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_001:lustre-MDT0001-lwp-OST0001_UUID+5:3355:x1862456998057536:12345-0@lo:400:kworker/u8:1.0 Request processed in 25us (248us total) trans 0 rc 0/0 00000100:00100000:2.0:1776178261.857529:0:6256:0:(nrs_fifo.c:241:nrs_fifo_req_stop()) NRS stop fifo request from 12345-0@lo, seq: 963 00000100:00100000:2.0:1776178261.857567:0:3355:0:(client.c:2215:ptlrpc_check_set()) Completed RPC req@00000000bdad24ac pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_02:lustre-MDT0001-lwp-OST0001_UUID:3355:1862456998057536:0@lo:400:kworker/u8:1.0 00000100:00100000:2.0:1776178261.857577:0:3355:0:(client.c:2215:ptlrpc_check_set()) Completed RPC req@00000000339a9a57 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_02:lustre-MDT0001-lwp-OST0000_UUID:3355:1862456998057408:0@lo:400:kworker/u8:1.0 00000100:00100000:1.0:1776178261.857698:0:3353:0:(client.c:1733:ptlrpc_send_new_req()) Sending RPC req@0000000021b0d516 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_00:lustre-MDT0001-mdtlov_UUID:3353:1862456998057664:0@lo:400:kworker/u8:1.0 00000100:00100000:1.0:1776178261.857736:0:3353:0:(events.c:361:request_in_callback()) peer: 12345-0@lo (source: 12345-0@lo) 00000100:00100000:1.0:1776178261.857763:0:3353:0:(ptlrpcd.c:415:ptlrpcd_check()) transfer 2 async RPCs [1->0] 00000100:00100000:1.0:1776178261.857769:0:3353:0:(client.c:1733:ptlrpc_send_new_req()) Sending RPC req@00000000167dc455 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_00:ac61f839-0e8e-4cac-9c59-08bcfbbad330:3353:1862456998057280:0@lo:400:kworker/u8:1.0 00000100:00100000:1.0:1776178261.857800:0:3353:0:(events.c:361:request_in_callback()) peer: 12345-0@lo (source: 12345-0@lo) 00000100:00100000:1.0:1776178261.857813:0:3353:0:(client.c:1733:ptlrpc_send_new_req()) Sending RPC req@00000000d0706d7c pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_00:lustre-MDT0001-mdtlov_UUID:3353:1862456998057728:0@lo:400:kworker/u8:1.0 00000100:00100000:1.0:1776178261.857834:0:3353:0:(events.c:361:request_in_callback()) peer: 12345-0@lo (source: 12345-0@lo) 00000100:00100000:3.0:1776178261.857984:0:3352:0:(client.c:1733:ptlrpc_send_new_req()) Sending RPC req@0000000078361ddf pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:lustre-MDT0000-lwp-OST0000_UUID:3352:1862456998057344:0@lo:38:kworker/u8:1.0 00000100:00100000:3.0:1776178261.858016:0:3352:0:(events.c:361:request_in_callback()) peer: 12345-0@lo (source: 12345-0@lo) 00000100:00100000:3.0:1776178261.858040:0:3352:0:(client.c:1733:ptlrpc_send_new_req()) Sending RPC req@0000000020afd0b3 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:lustre-MDT0000-lwp-OST0001_UUID:3352:1862456998057472:0@lo:38:kworker/u8:1.0 00000100:00100000:3.0:1776178261.858086:0:3352:0:(events.c:361:request_in_callback()) peer: 12345-0@lo (source: 12345-0@lo) 00000100:00100000:3.0:1776178261.858098:0:3352:0:(client.c:1733:ptlrpc_send_new_req()) Sending RPC req@00000000638772fd pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:lustre-MDT0001-mdtlov_UUID:3352:1862456998057600:0@lo:38:kworker/u8:1.0 00000100:00100000:3.0:1776178261.858120:0:3352:0:(events.c:361:request_in_callback()) peer: 12345-0@lo (source: 12345-0@lo) 00000100:00100000:3.0:1776178261.858131:0:3352:0:(client.c:1733:ptlrpc_send_new_req()) Sending RPC req@00000000bcefda18 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:lustre-MDT0000-lwp-MDT0001_UUID:3352:1862456998057792:0@lo:38:kworker/u8:1.0 00000100:00100000:3.0:1776178261.858148:0:3352:0:(events.c:361:request_in_callback()) peer: 12345-0@lo (source: 12345-0@lo) 00000100:00100000:1.0:1776178261.858263:0:6255:0:(service.c:2152:ptlrpc_server_handle_req_in()) got req x1862456998057344 00000100:00100000:1.0:1776178261.858295:0:6255:0:(service.c:2152:ptlrpc_server_handle_req_in()) got req x1862456998057472 00000100:00100000:1.0:1776178261.858304:0:6255:0:(service.c:2152:ptlrpc_server_handle_req_in()) got req x1862456998057792 00000100:00100000:1.0:1776178261.858312:0:6255:0:(nrs_fifo.c:179:nrs_fifo_req_get()) NRS start fifo request from 12345-0@lo, seq: 6669 00000100:00100000:1.0:1776178261.858316:0:6255:0:(service.c:2324:ptlrpc_server_handle_request()) Handling RPC req@0000000024989815 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_000:0+-99:3352:x1862456998057344:12345-0@lo:38:kworker/u8:1.0 00010000:02000000:1.0:1776178261.858328:0:6255:0:(ldlm_lib.c:1139:target_handle_connect()) lustre-MDT0000: Not available for connect from 0@lo (stopping) 00000100:00100000:1.0:1776178261.858363:0:6255:0:(service.c:2372:ptlrpc_server_handle_request()) Handled RPC req@0000000024989815 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_000:0+-99:3352:x1862456998057344:12345-0@lo:38:kworker/u8:1.0 Request processed in 46us (347us total) trans 0 rc -19/-19 00000100:00100000:1.0:1776178261.858370:0:6255:0:(nrs_fifo.c:241:nrs_fifo_req_stop()) NRS stop fifo request from 12345-0@lo, seq: 6669 00000100:00100000:1.0:1776178261.858380:0:6255:0:(nrs_fifo.c:179:nrs_fifo_req_get()) NRS start fifo request from 12345-0@lo, seq: 6670 00000100:00100000:1.0:1776178261.858383:0:6255:0:(service.c:2324:ptlrpc_server_handle_request()) Handling RPC req@00000000eb400454 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_000:0+-99:3352:x1862456998057472:12345-0@lo:38:kworker/u8:1.0 00010000:02000000:1.0:1776178261.858388:0:6255:0:(ldlm_lib.c:1139:target_handle_connect()) lustre-MDT0000: Not available for connect from 0@lo (stopping) 00000100:00100000:1.0:1776178261.858409:0:6255:0:(service.c:2372:ptlrpc_server_handle_request()) Handled RPC req@00000000eb400454 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_000:0+-99:3352:x1862456998057472:12345-0@lo:38:kworker/u8:1.0 Request processed in 25us (323us total) trans 0 rc -19/-19 00000100:00100000:1.0:1776178261.858417:0:6255:0:(nrs_fifo.c:241:nrs_fifo_req_stop()) NRS stop fifo request from 12345-0@lo, seq: 6670 00000100:00100000:1.0:1776178261.858427:0:6255:0:(nrs_fifo.c:179:nrs_fifo_req_get()) NRS start fifo request from 12345-0@lo, seq: 6671 00000100:00100000:1.0:1776178261.858430:0:6255:0:(service.c:2324:ptlrpc_server_handle_request()) Handling RPC req@00000000f0f1167e pname:cluuid+ref:pid:xid:nid:opc:job mdt00_000:0+-99:3352:x1862456998057792:12345-0@lo:38:kworker/u8:1.0 00010000:02000000:1.0:1776178261.858436:0:6255:0:(ldlm_lib.c:1139:target_handle_connect()) lustre-MDT0000: Not available for connect from 0@lo (stopping) 00000100:00100000:1.0:1776178261.858464:0:6255:0:(service.c:2372:ptlrpc_server_handle_request()) Handled RPC req@00000000f0f1167e pname:cluuid+ref:pid:xid:nid:opc:job mdt00_000:0+-99:3352:x1862456998057792:12345-0@lo:38:kworker/u8:1.0 Request processed in 21us (304us total) trans 0 rc -19/-19 00000100:00100000:1.0:1776178261.858469:0:6255:0:(nrs_fifo.c:241:nrs_fifo_req_stop()) NRS stop fifo request from 12345-0@lo, seq: 6671 00000100:00080000:1.0:1776178261.858491:0:3352:0:(import.c:86:import_set_state_nolock()) 0000000047601a98 lustre-MDT0000_UUID: changing import state from CONNECTING to DISCONN 00000100:00080000:1.0:1776178261.858495:0:3352:0:(import.c:1429:ptlrpc_connect_interpret()) recovery of lustre-MDT0000_UUID on 192.168.204.103@tcp failed (-19) 00000100:00100000:1.0:1776178261.858499:0:3352:0:(client.c:2215:ptlrpc_check_set()) Completed RPC req@0000000078361ddf pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:lustre-MDT0000-lwp-OST0000_UUID:3352:1862456998057344:0@lo:38:kworker/u8:1.0 00000100:00080000:1.0:1776178261.858506:0:3352:0:(import.c:86:import_set_state_nolock()) 00000000dfc8db13 lustre-MDT0000_UUID: changing import state from CONNECTING to DISCONN 00000100:00080000:1.0:1776178261.858508:0:3352:0:(import.c:1429:ptlrpc_connect_interpret()) recovery of lustre-MDT0000_UUID on 192.168.204.103@tcp failed (-19) 00000100:00100000:1.0:1776178261.858511:0:3352:0:(client.c:2215:ptlrpc_check_set()) Completed RPC req@0000000020afd0b3 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:lustre-MDT0000-lwp-OST0001_UUID:3352:1862456998057472:0@lo:38:kworker/u8:1.0 00000100:00080000:1.0:1776178261.858517:0:3352:0:(import.c:86:import_set_state_nolock()) 00000000a7c55b1f lustre-MDT0000_UUID: changing import state from CONNECTING to DISCONN 00000100:00080000:1.0:1776178261.858520:0:3352:0:(import.c:1429:ptlrpc_connect_interpret()) recovery of lustre-MDT0000_UUID on 192.168.204.103@tcp failed (-19) 00000100:00100000:1.0:1776178261.858522:0:3352:0:(client.c:2215:ptlrpc_check_set()) Completed RPC req@00000000bcefda18 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:lustre-MDT0000-lwp-MDT0001_UUID:3352:1862456998057792:0@lo:38:kworker/u8:1.0 00000100:00100000:2.0:1776178261.859575:0:20759:0:(service.c:2152:ptlrpc_server_handle_req_in()) got req x1862456998057664 00000100:00100000:2.0:1776178261.859642:0:20759:0:(service.c:2152:ptlrpc_server_handle_req_in()) got req x1862456998057728 00000100:00100000:2.0:1776178261.859674:0:20759:0:(nrs_fifo.c:179:nrs_fifo_req_get()) NRS start fifo request from 12345-0@lo, seq: 557 00000100:00100000:2.0:1776178261.859679:0:20759:0:(service.c:2324:ptlrpc_server_handle_request()) Handling RPC req@000000005f1d3350 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_010:lustre-MDT0001-mdtlov_UUID+5:3353:x1862456998057664:12345-0@lo:400:kworker/u8:1.0 00000100:00100000:0.0:1776178261.859842:0:20755:0:(nrs_fifo.c:179:nrs_fifo_req_get()) NRS start fifo request from 12345-0@lo, seq: 558 00000100:00100000:0.0:1776178261.859848:0:20755:0:(service.c:2324:ptlrpc_server_handle_request()) Handling RPC req@0000000008c9fb19 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_006:lustre-MDT0001-mdtlov_UUID+5:3353:x1862456998057728:12345-0@lo:400:kworker/u8:1.0 00000100:00100000:1.0:1776178261.859978:0:20137:0:(service.c:2152:ptlrpc_server_handle_req_in()) got req x1862456998057280 00000100:00100000:1.0:1776178261.860011:0:20137:0:(nrs_fifo.c:179:nrs_fifo_req_get()) NRS start fifo request from 12345-0@lo, seq: 178 00000100:00100000:1.0:1776178261.860015:0:20137:0:(service.c:2324:ptlrpc_server_handle_request()) Handling RPC req@00000000a6b47274 pname:cluuid+ref:pid:xid:nid:opc:job ll_mgs_0000:ac61f839-0e8e-4cac-9c59-08bcfbbad330+18:3353:x1862456998057280:12345-0@lo:400:kworker/u8:1.0 00000100:00100000:2.0:1776178261.860310:0:20759:0:(service.c:2372:ptlrpc_server_handle_request()) Handled RPC req@000000005f1d3350 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_010:lustre-MDT0001-mdtlov_UUID+5:3353:x1862456998057664:12345-0@lo:400:kworker/u8:1.0 Request processed in 629us (2573us total) trans 0 rc 0/0 00000100:00100000:2.0:1776178261.860317:0:20759:0:(nrs_fifo.c:241:nrs_fifo_req_stop()) NRS stop fifo request from 12345-0@lo, seq: 557 00000100:00100000:2.0:1776178261.860348:0:14128:0:(service.c:2152:ptlrpc_server_handle_req_in()) got req x1862456998057600 00000100:00100000:2.0:1776178261.860368:0:14128:0:(nrs_fifo.c:179:nrs_fifo_req_get()) NRS start fifo request from 12345-0@lo, seq: 719 00000100:00100000:2.0:1776178261.860372:0:14128:0:(service.c:2324:ptlrpc_server_handle_request()) Handling RPC req@000000007d60c0e9 pname:cluuid+ref:pid:xid:nid:opc:job mdt_out00_002:0+-99:3352:x1862456998057600:12345-0@lo:38:kworker/u8:1.0 00010000:02000000:2.0:1776178261.860385:0:14128:0:(ldlm_lib.c:1139:target_handle_connect()) lustre-MDT0000: Not available for connect from 0@lo (stopping) 00000100:00100000:2.0:1776178261.860422:0:14128:0:(service.c:2372:ptlrpc_server_handle_request()) Handled RPC req@000000007d60c0e9 pname:cluuid+ref:pid:xid:nid:opc:job mdt_out00_002:0+-99:3352:x1862456998057600:12345-0@lo:38:kworker/u8:1.0 Request processed in 50us (2303us total) trans 0 rc -19/-19 00000100:00100000:2.0:1776178261.860427:0:14128:0:(nrs_fifo.c:241:nrs_fifo_req_stop()) NRS stop fifo request from 12345-0@lo, seq: 719 00000100:00080000:2.0:1776178261.860479:0:3352:0:(import.c:86:import_set_state_nolock()) 00000000ba4f1821 lustre-MDT0000_UUID: changing import state from CONNECTING to DISCONN 00000100:00080000:2.0:1776178261.860483:0:3352:0:(import.c:1429:ptlrpc_connect_interpret()) recovery of lustre-MDT0000_UUID on 192.168.204.103@tcp failed (-19) 00000100:00100000:2.0:1776178261.860487:0:3352:0:(client.c:2215:ptlrpc_check_set()) Completed RPC req@00000000638772fd pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:lustre-MDT0001-mdtlov_UUID:3352:1862456998057600:0@lo:38:kworker/u8:1.0 00000100:00100000:0.0:1776178261.872057:0:20755:0:(service.c:2372:ptlrpc_server_handle_request()) Handled RPC req@0000000008c9fb19 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_006:lustre-MDT0001-mdtlov_UUID+5:3353:x1862456998057728:12345-0@lo:400:kworker/u8:1.0 Request processed in 12194us (14207us total) trans 0 rc 0/0 00000100:00100000:0.0:1776178261.872094:0:20755:0:(nrs_fifo.c:241:nrs_fifo_req_stop()) NRS stop fifo request from 12345-0@lo, seq: 558 00000100:00100000:1.0:1776178261.883443:0:20137:0:(service.c:2372:ptlrpc_server_handle_request()) Handled RPC req@00000000a6b47274 pname:cluuid+ref:pid:xid:nid:opc:job ll_mgs_0000:ac61f839-0e8e-4cac-9c59-08bcfbbad330+18:3353:x1862456998057280:12345-0@lo:400:kworker/u8:1.0 Request processed in 23414us (25629us total) trans 0 rc 0/0 00000100:00100000:1.0:1776178261.883477:0:20137:0:(nrs_fifo.c:241:nrs_fifo_req_stop()) NRS stop fifo request from 12345-0@lo, seq: 178 00000100:00100000:3.0:1776178261.890829:0:3353:0:(client.c:2215:ptlrpc_check_set()) Completed RPC req@0000000021b0d516 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_00:lustre-MDT0001-mdtlov_UUID:3353:1862456998057664:0@lo:400:kworker/u8:1.0 00000100:00100000:3.0:1776178261.890880:0:3353:0:(client.c:2215:ptlrpc_check_set()) Completed RPC req@00000000167dc455 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_00:ac61f839-0e8e-4cac-9c59-08bcfbbad330:3353:1862456998057280:0@lo:400:kworker/u8:1.0 00000100:00100000:3.0:1776178261.890905:0:3353:0:(client.c:2215:ptlrpc_check_set()) Completed RPC req@00000000d0706d7c pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_00:lustre-MDT0001-mdtlov_UUID:3353:1862456998057728:0@lo:400:kworker/u8:1.0 00010000:00010000:0.0:1776178262.496347:0:6244:0:(ldlm_lockd.c:2792:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:2.0:1776178263.396882:0:3355:0:(client.c:1733:ptlrpc_send_new_req()) Sending RPC req@000000001982f085 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_02:lustre-MDT0001-mdtlov_UUID:3355:1862456998057856:0@lo:13:osp-pre-1-1.0 00000100:00100000:2.0:1776178263.397005:0:3355:0:(events.c:361:request_in_callback()) peer: 12345-0@lo (source: 12345-0@lo) 00000100:00100000:0.0:1776178263.398523:0:9554:0:(service.c:2152:ptlrpc_server_handle_req_in()) got req x1862456998057856 00000100:00100000:0.0:1776178263.398636:0:9554:0:(nrs_fifo.c:179:nrs_fifo_req_get()) NRS start fifo request from 12345-0@lo, seq: 511 00000100:00100000:0.0:1776178263.398645:0:9554:0:(service.c:2324:ptlrpc_server_handle_request()) Handling RPC req@00000000638772fd pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_create00:lustre-MDT0001-mdtlov_UUID+5:3355:x1862456998057856:12345-0@lo:13:osp-pre-1-1.0 00002000:00000024:0.0:1776178263.398704:0:9554:0:(ofd_obd.c:740:ofd_statfs()) blocks cached 229376 granted 491746368 pending 0 free 3923689472 avail 3692167168 00000100:00100000:0.0:1776178263.409820:0:9554:0:(service.c:2372:ptlrpc_server_handle_request()) Handled RPC req@00000000638772fd pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_create00:lustre-MDT0001-mdtlov_UUID+5:3355:x1862456998057856:12345-0@lo:13:osp-pre-1-1.0 Request processed in 11160us (12802us total) trans 0 rc 0/0 00000100:00100000:0.0:1776178263.409847:0:9554:0:(nrs_fifo.c:241:nrs_fifo_req_stop()) NRS stop fifo request from 12345-0@lo, seq: 511 00000100:00100000:2.0:1776178263.410320:0:3355:0:(client.c:2215:ptlrpc_check_set()) Completed RPC req@000000001982f085 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_02:lustre-MDT0001-mdtlov_UUID:3355:1862456998057856:0@lo:13:osp-pre-1-1.0 00010000:00010000:2.0:1776178263.520506:0:6245:0:(ldlm_lockd.c:2792:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:1.0:1776178263.906668:0:3356:0:(client.c:1733:ptlrpc_send_new_req()) Sending RPC req@00000000305b2aed pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_03:lustre-MDT0001-mdtlov_UUID:3356:1862456998057920:0@lo:13:osp-pre-0-1.0 00000100:00100000:1.0:1776178263.906761:0:3356:0:(events.c:361:request_in_callback()) peer: 12345-0@lo (source: 12345-0@lo) 00000100:00100000:0.0:1776178263.906875:0:9554:0:(service.c:2152:ptlrpc_server_handle_req_in()) got req x1862456998057920 00000100:00100000:0.0:1776178263.906933:0:9554:0:(nrs_fifo.c:179:nrs_fifo_req_get()) NRS start fifo request from 12345-0@lo, seq: 512 00000100:00100000:0.0:1776178263.906939:0:9554:0:(service.c:2324:ptlrpc_server_handle_request()) Handling RPC req@000000006459e586 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_create00:lustre-MDT0001-mdtlov_UUID+5:3356:x1862456998057920:12345-0@lo:13:osp-pre-0-1.0 00002000:00000024:0.0:1776178263.906976:0:9554:0:(ofd_obd.c:740:ofd_statfs()) blocks cached 315392 granted 484883776 pending 0 free 3923689472 avail 3692167168 00000100:00100000:0.0:1776178263.907032:0:9554:0:(service.c:2372:ptlrpc_server_handle_request()) Handled RPC req@000000006459e586 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_create00:lustre-MDT0001-mdtlov_UUID+5:3356:x1862456998057920:12345-0@lo:13:osp-pre-0-1.0 Request processed in 93us (273us total) trans 0 rc 0/0 00000100:00100000:0.0:1776178263.907039:0:9554:0:(nrs_fifo.c:241:nrs_fifo_req_stop()) NRS stop fifo request from 12345-0@lo, seq: 512 00000100:00100000:1.0:1776178263.907184:0:3356:0:(client.c:2215:ptlrpc_check_set()) Completed RPC req@00000000305b2aed pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_03:lustre-MDT0001-mdtlov_UUID:3356:1862456998057920:0@lo:13:osp-pre-0-1.0 00010000:00010000:2.0:1776178264.544465:0:9224:0:(ldlm_lockd.c:2792:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:3.0:1776178265.379693:0:3348:0:(events.c:361:request_in_callback()) peer: 12345-192.168.204.3@tcp (source: 12345-192.168.204.3@tcp) 00000100:00100000:2.0:1776178265.380638:0:20137:0:(service.c:2152:ptlrpc_server_handle_req_in()) got req x1862456979112832 00000100:00100000:2.0:1776178265.380738:0:20137:0:(nrs_fifo.c:179:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.204.3@tcp, seq: 179 00000100:00100000:2.0:1776178265.380746:0:20137:0:(service.c:2324:ptlrpc_server_handle_request()) Handling RPC req@0000000080051315 pname:cluuid+ref:pid:xid:nid:opc:job ll_mgs_0000:5b5c66b1-1a2f-4651-8548-0c75b9a38a94+9:2271:x1862456979112832:12345-192.168.204.3@tcp:400:kworker/u8:4.0 00000100:00100000:2.0:1776178265.380843:0:20137:0:(service.c:2372:ptlrpc_server_handle_request()) Handled RPC req@0000000080051315 pname:cluuid+ref:pid:xid:nid:opc:job ll_mgs_0000:5b5c66b1-1a2f-4651-8548-0c75b9a38a94+9:2271:x1862456979112832:12345-192.168.204.3@tcp:400:kworker/u8:4.0 Request processed in 96us (1161us total) trans 0 rc 0/0 00000100:00100000:2.0:1776178265.380852:0:20137:0:(nrs_fifo.c:241:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.204.3@tcp, seq: 179 00000100:00100000:3.0:1776178265.381056:0:3346:0:(events.c:361:request_in_callback()) peer: 12345-192.168.204.3@tcp (source: 12345-192.168.204.3@tcp) 00000100:00100000:1.0:1776178265.383137:0:6256:0:(service.c:2152:ptlrpc_server_handle_req_in()) got req x1862456979112896 00000100:00100000:1.0:1776178265.383302:0:6256:0:(nrs_fifo.c:179:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.204.3@tcp, seq: 6672 00000100:00100000:1.0:1776178265.383310:0:6256:0:(service.c:2324:ptlrpc_server_handle_request()) Handling RPC req@00000000e2b18a6d pname:cluuid+ref:pid:xid:nid:opc:job mdt00_001:0+-99:2269:x1862456979112896:12345-192.168.204.3@tcp:38:kworker/u8:4.0 00010000:02000000:1.0:1776178265.383899:0:6256:0:(ldlm_lib.c:1139:target_handle_connect()) lustre-MDT0000: Not available for connect from 192.168.204.3@tcp (stopping) 00000100:00100000:1.0:1776178265.384183:0:6256:0:(service.c:2372:ptlrpc_server_handle_request()) Handled RPC req@00000000e2b18a6d pname:cluuid+ref:pid:xid:nid:opc:job mdt00_001:0+-99:2269:x1862456979112896:12345-192.168.204.3@tcp:38:kworker/u8:4.0 Request processed in 871us (3127us total) trans 0 rc -19/-19 00000100:00100000:1.0:1776178265.384192:0:6256:0:(nrs_fifo.c:241:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.204.3@tcp, seq: 6672 00000100:00100000:3.0:1776178265.392965:0:3346:0:(events.c:361:request_in_callback()) peer: 12345-192.168.204.3@tcp (source: 12345-192.168.204.3@tcp) 00000100:00100000:3.0:1776178265.393129:0:3346:0:(events.c:361:request_in_callback()) peer: 12345-192.168.204.3@tcp (source: 12345-192.168.204.3@tcp) 00000100:00100000:0.0:1776178265.393262:0:20755:0:(service.c:2152:ptlrpc_server_handle_req_in()) got req x1862456979113088 00000100:00100000:0.0:1776178265.393360:0:20755:0:(nrs_fifo.c:179:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.204.3@tcp, seq: 559 00000100:00100000:0.0:1776178265.393369:0:20755:0:(service.c:2324:ptlrpc_server_handle_request()) Handling RPC req@0000000021d5d4e8 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_006:b50963bd-8eba-4432-9a54-8888d00e3fa1+9:2270:x1862456979113088:12345-192.168.204.3@tcp:400:kworker/u8:4.0 00000100:00100000:0.0:1776178265.393472:0:20755:0:(service.c:2372:ptlrpc_server_handle_request()) Handled RPC req@0000000021d5d4e8 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_006:b50963bd-8eba-4432-9a54-8888d00e3fa1+9:2270:x1862456979113088:12345-192.168.204.3@tcp:400:kworker/u8:4.0 Request processed in 104us (344us total) trans 0 rc 0/0 00000100:00100000:0.0:1776178265.393479:0:20755:0:(nrs_fifo.c:241:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.204.3@tcp, seq: 559 00000100:00100000:3.0:1776178265.395166:0:3348:0:(events.c:361:request_in_callback()) peer: 12345-192.168.204.3@tcp (source: 12345-192.168.204.3@tcp) 00000100:00100000:2.0:1776178265.395365:0:20759:0:(service.c:2152:ptlrpc_server_handle_req_in()) got req x1862456979113024 00000100:00100000:2.0:1776178265.395495:0:20759:0:(nrs_fifo.c:179:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.204.3@tcp, seq: 560 00000100:00100000:2.0:1776178265.395502:0:20759:0:(service.c:2324:ptlrpc_server_handle_request()) Handling RPC req@00000000699e66b5 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_010:b50963bd-8eba-4432-9a54-8888d00e3fa1+8:2271:x1862456979113024:12345-192.168.204.3@tcp:400:kworker/u8:4.0 00000100:00100000:2.0:1776178265.395590:0:20759:0:(service.c:2372:ptlrpc_server_handle_request()) Handled RPC req@00000000699e66b5 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_010:b50963bd-8eba-4432-9a54-8888d00e3fa1+8:2271:x1862456979113024:12345-192.168.204.3@tcp:400:kworker/u8:4.0 Request processed in 88us (433us total) trans 0 rc 0/0 00000100:00100000:2.0:1776178265.395598:0:20759:0:(nrs_fifo.c:241:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.204.3@tcp, seq: 560 00000100:00100000:1.0:1776178265.396567:0:6256:0:(service.c:2152:ptlrpc_server_handle_req_in()) got req x1862456979112960 00000100:00100000:1.0:1776178265.396655:0:6256:0:(nrs_fifo.c:179:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.204.3@tcp, seq: 964 00000100:00100000:1.0:1776178265.396663:0:6256:0:(service.c:2324:ptlrpc_server_handle_request()) Handling RPC req@0000000078ae567e pname:cluuid+ref:pid:xid:nid:opc:job mdt00_001:b50963bd-8eba-4432-9a54-8888d00e3fa1+6:2271:x1862456979112960:12345-192.168.204.3@tcp:400:kworker/u8:4.0 00000100:00100000:1.0:1776178265.396734:0:6256:0:(service.c:2372:ptlrpc_server_handle_request()) Handled RPC req@0000000078ae567e pname:cluuid+ref:pid:xid:nid:opc:job mdt00_001:b50963bd-8eba-4432-9a54-8888d00e3fa1+6:2271:x1862456979112960:12345-192.168.204.3@tcp:400:kworker/u8:4.0 Request processed in 70us (3777us total) trans 0 rc 0/0 00000100:00100000:1.0:1776178265.396741:0:6256:0:(nrs_fifo.c:241:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.204.3@tcp, seq: 964 00010000:00010000:2.0:1776178265.568347:0:9225:0:(ldlm_lockd.c:2792:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00010000:00010000:0.0:1776178266.592323:0:6244:0:(ldlm_lockd.c:2792:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00080000:0.0:1776178266.976365:0:36:0:(pinger.c:251:ptlrpc_pinger_process_import()) lustre-MDT0000-lwp-OST0000_UUID->lustre-MDT0000_UUID: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00080000:0.0:1776178266.976373:0:36:0:(recover.c:57:ptlrpc_initiate_recovery()) lustre-MDT0000_UUID: starting recovery 00000100:00080000:0.0:1776178266.976377:0:36:0:(import.c:86:import_set_state_nolock()) 0000000047601a98 lustre-MDT0000_UUID: changing import state from DISCONN to CONNECTING 00000100:00080000:0.0:1776178266.976385:0:36:0:(import.c:534:import_select_connection()) lustre-MDT0000-lwp-OST0000: connect to NID 0@lo last attempt 969 00000100:00080000:0.0:1776178266.976388:0:36:0:(import.c:581:import_select_connection()) lustre-MDT0000-lwp-OST0000: tried all connections, increasing latency to 20s 00000100:00080000:0.0:1776178266.976394:0:36:0:(import.c:615:import_select_connection()) lustre-MDT0000-lwp-OST0000: import 0000000047601a98 using connection 192.168.204.103@tcp/0@lo 00000100:00100000:0.0:1776178266.976416:0:36:0:(import.c:807:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5) req@00000000e0d60cbc x1862456998058048/t0(0) o38->lustre-MDT0000-lwp-OST0000@0@lo:12/10 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' 00000100:00080000:0.0:1776178266.976452:0:36:0:(pinger.c:251:ptlrpc_pinger_process_import()) lustre-MDT0000-lwp-OST0001_UUID->lustre-MDT0000_UUID: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00080000:0.0:1776178266.976455:0:36:0:(recover.c:57:ptlrpc_initiate_recovery()) lustre-MDT0000_UUID: starting recovery 00000100:00080000:0.0:1776178266.976457:0:36:0:(import.c:86:import_set_state_nolock()) 00000000dfc8db13 lustre-MDT0000_UUID: changing import state from DISCONN to CONNECTING 00000100:00080000:0.0:1776178266.976460:0:36:0:(import.c:534:import_select_connection()) lustre-MDT0000-lwp-OST0001: connect to NID 0@lo last attempt 969 00000100:00080000:0.0:1776178266.976462:0:36:0:(import.c:581:import_select_connection()) lustre-MDT0000-lwp-OST0001: tried all connections, increasing latency to 20s 00000100:00080000:0.0:1776178266.976465:0:36:0:(import.c:615:import_select_connection()) lustre-MDT0000-lwp-OST0001: import 00000000dfc8db13 using connection 192.168.204.103@tcp/0@lo 00000100:00100000:0.0:1776178266.976475:0:36:0:(import.c:807:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5) req@000000006be5e5b9 x1862456998058176/t0(0) o38->lustre-MDT0000-lwp-OST0001@0@lo:12/10 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' 00000100:00080000:0.0:1776178266.976516:0:36:0:(pinger.c:251:ptlrpc_pinger_process_import()) lustre-MDT0001-mdtlov_UUID->lustre-MDT0000_UUID: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00080000:0.0:1776178266.976520:0:36:0:(recover.c:57:ptlrpc_initiate_recovery()) lustre-MDT0000_UUID: starting recovery 00000100:00080000:0.0:1776178266.976521:0:36:0:(import.c:86:import_set_state_nolock()) 00000000ba4f1821 lustre-MDT0000_UUID: changing import state from DISCONN to CONNECTING 00000100:00080000:0.0:1776178266.976525:0:36:0:(import.c:534:import_select_connection()) lustre-MDT0000-osp-MDT0001: connect to NID 0@lo last attempt 969 00000100:00080000:0.0:1776178266.976526:0:36:0:(import.c:581:import_select_connection()) lustre-MDT0000-osp-MDT0001: tried all connections, increasing latency to 20s 00000100:00080000:0.0:1776178266.976529:0:36:0:(import.c:615:import_select_connection()) lustre-MDT0000-osp-MDT0001: import 00000000ba4f1821 using connection 192.168.204.103@tcp/0@lo 00000100:00100000:0.0:1776178266.976540:0:36:0:(import.c:807:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5) req@00000000d668bb9c x1862456998058304/t0(0) o38->lustre-MDT0000-osp-MDT0001@0@lo:24/4 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' 00000100:00080000:0.0:1776178266.976559:0:36:0:(pinger.c:251:ptlrpc_pinger_process_import()) lustre-MDT0000-lwp-MDT0001_UUID->lustre-MDT0000_UUID: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00080000:0.0:1776178266.976562:0:36:0:(recover.c:57:ptlrpc_initiate_recovery()) lustre-MDT0000_UUID: starting recovery 00000100:00080000:0.0:1776178266.976564:0:36:0:(import.c:86:import_set_state_nolock()) 00000000a7c55b1f lustre-MDT0000_UUID: changing import state from DISCONN to CONNECTING 00000100:00080000:0.0:1776178266.976566:0:36:0:(import.c:534:import_select_connection()) lustre-MDT0000-lwp-MDT0001: connect to NID 0@lo last attempt 969 00000100:00080000:0.0:1776178266.976567:0:36:0:(import.c:581:import_select_connection()) lustre-MDT0000-lwp-MDT0001: tried all connections, increasing latency to 20s 00000100:00080000:0.0:1776178266.976570:0:36:0:(import.c:615:import_select_connection()) lustre-MDT0000-lwp-MDT0001: import 00000000a7c55b1f using connection 192.168.204.103@tcp/0@lo 00000100:00100000:0.0:1776178266.976577:0:36:0:(import.c:807:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5) req@000000009383c871 x1862456998058496/t0(0) o38->lustre-MDT0000-lwp-MDT0001@0@lo:12/10 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' 00000100:00100000:0.0:1776178266.976609:0:3355:0:(client.c:1733:ptlrpc_send_new_req()) Sending RPC req@000000007cbab86b pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_02:lustre-MDT0001-lwp-OST0001_UUID:3355:1862456998058240:0@lo:400:kworker/u8:1.0 00000100:00100000:1.0:1776178266.976783:0:3352:0:(client.c:1733:ptlrpc_send_new_req()) Sending RPC req@00000000e0d60cbc pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:lustre-MDT0000-lwp-OST0000_UUID:3352:1862456998058048:0@lo:38:kworker/u8:1.0 00000100:00100000:1.0:1776178266.976837:0:3352:0:(events.c:361:request_in_callback()) peer: 12345-0@lo (source: 12345-0@lo) 00000100:00100000:1.0:1776178266.976870:0:3352:0:(client.c:1733:ptlrpc_send_new_req()) Sending RPC req@000000006be5e5b9 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:lustre-MDT0000-lwp-OST0001_UUID:3352:1862456998058176:0@lo:38:kworker/u8:1.0 00000100:00100000:1.0:1776178266.976892:0:3352:0:(events.c:361:request_in_callback()) peer: 12345-0@lo (source: 12345-0@lo) 00000100:00100000:1.0:1776178266.976907:0:3352:0:(client.c:1733:ptlrpc_send_new_req()) Sending RPC req@00000000d668bb9c pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:lustre-MDT0001-mdtlov_UUID:3352:1862456998058304:0@lo:38:kworker/u8:1.0 00000100:00100000:1.0:1776178266.976932:0:3352:0:(events.c:361:request_in_callback()) peer: 12345-0@lo (source: 12345-0@lo) 00000100:00100000:1.0:1776178266.976945:0:3352:0:(client.c:1733:ptlrpc_send_new_req()) Sending RPC req@000000009383c871 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:lustre-MDT0000-lwp-MDT0001_UUID:3352:1862456998058496:0@lo:38:kworker/u8:1.0 00000100:00100000:1.0:1776178266.976965:0:3352:0:(events.c:361:request_in_callback()) peer: 12345-0@lo (source: 12345-0@lo) 00000100:00100000:1.0:1776178266.977027:0:6256:0:(service.c:2152:ptlrpc_server_handle_req_in()) got req x1862456998058048 00000100:00100000:1.0:1776178266.977088:0:6256:0:(service.c:2152:ptlrpc_server_handle_req_in()) got req x1862456998058176 00000100:00100000:1.0:1776178266.977106:0:6256:0:(service.c:2152:ptlrpc_server_handle_req_in()) got req x1862456998058496 00000100:00100000:1.0:1776178266.977119:0:6256:0:(nrs_fifo.c:179:nrs_fifo_req_get()) NRS start fifo request from 12345-0@lo, seq: 6673 00000100:00100000:1.0:1776178266.977125:0:6256:0:(service.c:2324:ptlrpc_server_handle_request()) Handling RPC req@000000000cde9837 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_001:0+-99:3352:x1862456998058048:12345-0@lo:38:kworker/u8:1.0 00010000:02000000:1.0:1776178266.977139:0:6256:0:(ldlm_lib.c:1139:target_handle_connect()) lustre-MDT0000: Not available for connect from 0@lo (stopping) 00000100:00100000:2.0:1776178266.978580:0:3354:0:(client.c:1733:ptlrpc_send_new_req()) Sending RPC req@000000002a321d27 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_01:lustre-MDT0001-lwp-OST0000_UUID:3354:1862456998058112:0@lo:400:kworker/u8:1.0 00000100:00100000:2.0:1776178266.978641:0:3354:0:(events.c:361:request_in_callback()) peer: 12345-0@lo (source: 12345-0@lo) 00000100:00100000:2.0:1776178266.978680:0:3354:0:(ptlrpcd.c:415:ptlrpcd_check()) transfer 2 async RPCs [0->1] 00000100:00100000:2.0:1776178266.978688:0:3354:0:(client.c:1733:ptlrpc_send_new_req()) Sending RPC req@000000008c593d89 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_01:ac61f839-0e8e-4cac-9c59-08bcfbbad330:3354:1862456998057984:0@lo:400:kworker/u8:1.0 00000100:00100000:2.0:1776178266.978727:0:3354:0:(events.c:361:request_in_callback()) peer: 12345-0@lo (source: 12345-0@lo) 00000100:00100000:2.0:1776178266.978756:0:3354:0:(client.c:1733:ptlrpc_send_new_req()) Sending RPC req@00000000305b2aed pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_01:lustre-MDT0001-mdtlov_UUID:3354:1862456998058432:0@lo:400:kworker/u8:1.0 00000100:00100000:2.0:1776178266.978783:0:3354:0:(events.c:361:request_in_callback()) peer: 12345-0@lo (source: 12345-0@lo) 00000100:00100000:2.0:1776178266.978839:0:15213:0:(service.c:2152:ptlrpc_server_handle_req_in()) got req x1862456998058304 00000100:00100000:2.0:1776178266.978878:0:15213:0:(nrs_fifo.c:179:nrs_fifo_req_get()) NRS start fifo request from 12345-0@lo, seq: 720 00000100:00100000:2.0:1776178266.978884:0:15213:0:(service.c:2324:ptlrpc_server_handle_request()) Handling RPC req@00000000d7e84c97 pname:cluuid+ref:pid:xid:nid:opc:job mdt_out00_003:0+-99:3352:x1862456998058304:12345-0@lo:38:kworker/u8:1.0 00010000:02000000:2.0:1776178266.978899:0:15213:0:(ldlm_lib.c:1139:target_handle_connect()) lustre-MDT0000: Not available for connect from 0@lo (stopping) 00000100:00100000:2.0:1776178266.978952:0:15213:0:(service.c:2372:ptlrpc_server_handle_request()) Handled RPC req@00000000d7e84c97 pname:cluuid+ref:pid:xid:nid:opc:job mdt_out00_003:0+-99:3352:x1862456998058304:12345-0@lo:38:kworker/u8:1.0 Request processed in 67us (2019us total) trans 0 rc -19/-19 00000100:00100000:2.0:1776178266.978960:0:15213:0:(nrs_fifo.c:241:nrs_fifo_req_stop()) NRS stop fifo request from 12345-0@lo, seq: 720 00000100:00100000:2.0:1776178266.978986:0:20759:0:(service.c:2152:ptlrpc_server_handle_req_in()) got req x1862456998058432 00000100:00100000:2.0:1776178266.979029:0:20759:0:(nrs_fifo.c:179:nrs_fifo_req_get()) NRS start fifo request from 12345-0@lo, seq: 561 00000100:00100000:2.0:1776178266.979033:0:20759:0:(service.c:2324:ptlrpc_server_handle_request()) Handling RPC req@00000000bdad24ac pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_010:lustre-MDT0001-mdtlov_UUID+5:3354:x1862456998058432:12345-0@lo:400:kworker/u8:1.0 00000100:00100000:2.0:1776178266.979103:0:20759:0:(service.c:2372:ptlrpc_server_handle_request()) Handled RPC req@00000000bdad24ac pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_010:lustre-MDT0001-mdtlov_UUID+5:3354:x1862456998058432:12345-0@lo:400:kworker/u8:1.0 Request processed in 69us (320us total) trans 0 rc 0/0 00000100:00100000:2.0:1776178266.979109:0:20759:0:(nrs_fifo.c:241:nrs_fifo_req_stop()) NRS stop fifo request from 12345-0@lo, seq: 561 00000100:00100000:2.0:1776178266.979130:0:20140:0:(service.c:2152:ptlrpc_server_handle_req_in()) got req x1862456998057984 00000100:00100000:2.0:1776178266.979150:0:20140:0:(nrs_fifo.c:179:nrs_fifo_req_get()) NRS start fifo request from 12345-0@lo, seq: 180 00000100:00100000:2.0:1776178266.979153:0:20140:0:(service.c:2324:ptlrpc_server_handle_request()) Handling RPC req@000000008f0263f6 pname:cluuid+ref:pid:xid:nid:opc:job ll_mgs_0002:ac61f839-0e8e-4cac-9c59-08bcfbbad330+18:3354:x1862456998057984:12345-0@lo:400:kworker/u8:1.0 00000100:00100000:2.0:1776178266.979175:0:20140:0:(service.c:2372:ptlrpc_server_handle_request()) Handled RPC req@000000008f0263f6 pname:cluuid+ref:pid:xid:nid:opc:job ll_mgs_0002:ac61f839-0e8e-4cac-9c59-08bcfbbad330+18:3354:x1862456998057984:12345-0@lo:400:kworker/u8:1.0 Request processed in 22us (449us total) trans 0 rc 0/0 00000100:00100000:2.0:1776178266.979180:0:20140:0:(nrs_fifo.c:241:nrs_fifo_req_stop()) NRS stop fifo request from 12345-0@lo, seq: 180 00000100:00080000:2.0:1776178266.979227:0:3352:0:(import.c:86:import_set_state_nolock()) 00000000ba4f1821 lustre-MDT0000_UUID: changing import state from CONNECTING to DISCONN 00000100:00080000:2.0:1776178266.979232:0:3352:0:(import.c:1429:ptlrpc_connect_interpret()) recovery of lustre-MDT0000_UUID on 192.168.204.103@tcp failed (-19) 00000100:00100000:2.0:1776178266.979235:0:3352:0:(client.c:2215:ptlrpc_check_set()) Completed RPC req@00000000d668bb9c pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:lustre-MDT0001-mdtlov_UUID:3352:1862456998058304:0@lo:38:kworker/u8:1.0 00000100:00100000:2.0:1776178266.979264:0:3354:0:(client.c:2215:ptlrpc_check_set()) Completed RPC req@000000008c593d89 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_01:ac61f839-0e8e-4cac-9c59-08bcfbbad330:3354:1862456998057984:0@lo:400:kworker/u8:1.0 00000100:00100000:2.0:1776178266.979272:0:3354:0:(client.c:2215:ptlrpc_check_set()) Completed RPC req@00000000305b2aed pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_01:lustre-MDT0001-mdtlov_UUID:3354:1862456998058432:0@lo:400:kworker/u8:1.0 00000100:00100000:2.0:1776178266.979328:0:6257:0:(service.c:2152:ptlrpc_server_handle_req_in()) got req x1862456998058112 00000100:00100000:2.0:1776178266.979351:0:6257:0:(nrs_fifo.c:179:nrs_fifo_req_get()) NRS start fifo request from 12345-0@lo, seq: 965 00000100:00100000:2.0:1776178266.979354:0:6257:0:(service.c:2324:ptlrpc_server_handle_request()) Handling RPC req@00000000339a9a57 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_002:lustre-MDT0001-lwp-OST0000_UUID+5:3354:x1862456998058112:12345-0@lo:400:kworker/u8:1.0 00000100:00100000:2.0:1776178266.979388:0:6257:0:(service.c:2372:ptlrpc_server_handle_request()) Handled RPC req@00000000339a9a57 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_002:lustre-MDT0001-lwp-OST0000_UUID+5:3354:x1862456998058112:12345-0@lo:400:kworker/u8:1.0 Request processed in 33us (747us total) trans 0 rc 0/0 00000100:00100000:2.0:1776178266.979393:0:6257:0:(nrs_fifo.c:241:nrs_fifo_req_stop()) NRS stop fifo request from 12345-0@lo, seq: 965 00000100:00100000:2.0:1776178266.979405:0:6257:0:(nrs_fifo.c:179:nrs_fifo_req_get()) NRS start fifo request from 12345-0@lo, seq: 6674 00000100:00100000:2.0:1776178266.979408:0:6257:0:(service.c:2324:ptlrpc_server_handle_request()) Handling RPC req@000000004c44fad7 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_002:0+-99:3352:x1862456998058176:12345-0@lo:38:kworker/u8:1.0 00010000:02000000:2.0:1776178266.979416:0:6257:0:(ldlm_lib.c:1139:target_handle_connect()) lustre-MDT0000: Not available for connect from 0@lo (stopping) 00000100:00100000:2.0:1776178266.979444:0:6257:0:(service.c:2372:ptlrpc_server_handle_request()) Handled RPC req@000000004c44fad7 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_002:0+-99:3352:x1862456998058176:12345-0@lo:38:kworker/u8:1.0 Request processed in 34us (2551us total) trans 0 rc -19/-19 00000100:00100000:2.0:1776178266.979450:0:6257:0:(nrs_fifo.c:241:nrs_fifo_req_stop()) NRS stop fifo request from 12345-0@lo, seq: 6674 00000100:00100000:2.0:1776178266.979463:0:6257:0:(nrs_fifo.c:179:nrs_fifo_req_get()) NRS start fifo request from 12345-0@lo, seq: 6675 00000100:00100000:2.0:1776178266.979467:0:6257:0:(service.c:2324:ptlrpc_server_handle_request()) Handling RPC req@0000000053c46e8d pname:cluuid+ref:pid:xid:nid:opc:job mdt00_002:0+-99:3352:x1862456998058496:12345-0@lo:38:kworker/u8:1.0 00010000:02000000:2.0:1776178266.979474:0:6257:0:(ldlm_lib.c:1139:target_handle_connect()) lustre-MDT0000: Not available for connect from 0@lo (stopping) 00000100:00100000:2.0:1776178266.979493:0:6257:0:(service.c:2372:ptlrpc_server_handle_request()) Handled RPC req@0000000053c46e8d pname:cluuid+ref:pid:xid:nid:opc:job mdt00_002:0+-99:3352:x1862456998058496:12345-0@lo:38:kworker/u8:1.0 Request processed in 26us (2529us total) trans 0 rc -19/-19 00000100:00100000:2.0:1776178266.979512:0:6257:0:(nrs_fifo.c:241:nrs_fifo_req_stop()) NRS stop fifo request from 12345-0@lo, seq: 6675 00000100:00100000:2.0:1776178266.979532:0:3356:0:(client.c:1733:ptlrpc_send_new_req()) Sending RPC req@00000000490667a2 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_03:lustre-MDT0001-mdtlov_UUID:3356:1862456998058368:0@lo:400:kworker/u8:1.0 00000100:00100000:2.0:1776178266.979556:0:3356:0:(events.c:361:request_in_callback()) peer: 12345-0@lo (source: 12345-0@lo) 00000100:00100000:0.0:1776178266.980139:0:3355:0:(events.c:361:request_in_callback()) peer: 12345-0@lo (source: 12345-0@lo) 00000100:00100000:0.0:1776178266.980175:0:6257:0:(service.c:2152:ptlrpc_server_handle_req_in()) got req x1862456998058240 00000100:00100000:0.0:1776178266.980195:0:6257:0:(nrs_fifo.c:179:nrs_fifo_req_get()) NRS start fifo request from 12345-0@lo, seq: 966 00000100:00100000:0.0:1776178266.980200:0:6257:0:(service.c:2324:ptlrpc_server_handle_request()) Handling RPC req@00000000305b2aed pname:cluuid+ref:pid:xid:nid:opc:job mdt00_002:lustre-MDT0001-lwp-OST0001_UUID+5:3355:x1862456998058240:12345-0@lo:400:kworker/u8:1.0 00000100:00100000:0.0:1776178266.980237:0:6257:0:(service.c:2372:ptlrpc_server_handle_request()) Handled RPC req@00000000305b2aed pname:cluuid+ref:pid:xid:nid:opc:job mdt00_002:lustre-MDT0001-lwp-OST0001_UUID+5:3355:x1862456998058240:12345-0@lo:400:kworker/u8:1.0 Request processed in 36us (99us total) trans 0 rc 0/0 00000100:00100000:0.0:1776178266.980243:0:6257:0:(nrs_fifo.c:241:nrs_fifo_req_stop()) NRS stop fifo request from 12345-0@lo, seq: 966 00000100:00100000:0.0:1776178266.980264:0:20755:0:(service.c:2152:ptlrpc_server_handle_req_in()) got req x1862456998058368 00000100:00100000:0.0:1776178266.980292:0:20755:0:(nrs_fifo.c:179:nrs_fifo_req_get()) NRS start fifo request from 12345-0@lo, seq: 562 00000100:00080000:3.0:1776178266.980588:0:3352:0:(import.c:86:import_set_state_nolock()) 00000000dfc8db13 lustre-MDT0000_UUID: changing import state from CONNECTING to DISCONN 00000100:00080000:3.0:1776178266.980593:0:3352:0:(import.c:1429:ptlrpc_connect_interpret()) recovery of lustre-MDT0000_UUID on 192.168.204.103@tcp failed (-19) 00000100:00100000:3.0:1776178266.980597:0:3352:0:(client.c:2215:ptlrpc_check_set()) Completed RPC req@000000006be5e5b9 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:lustre-MDT0000-lwp-OST0001_UUID:3352:1862456998058176:0@lo:38:kworker/u8:1.0 00000100:00080000:3.0:1776178266.980607:0:3352:0:(import.c:86:import_set_state_nolock()) 00000000a7c55b1f lustre-MDT0000_UUID: changing import state from CONNECTING to DISCONN 00000100:00080000:3.0:1776178266.980609:0:3352:0:(import.c:1429:ptlrpc_connect_interpret()) recovery of lustre-MDT0000_UUID on 192.168.204.103@tcp failed (-19) 00000100:00100000:3.0:1776178266.980612:0:3352:0:(client.c:2215:ptlrpc_check_set()) Completed RPC req@000000009383c871 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:lustre-MDT0000-lwp-MDT0001_UUID:3352:1862456998058496:0@lo:38:kworker/u8:1.0 00000100:00100000:3.0:1776178266.980644:0:3354:0:(client.c:2215:ptlrpc_check_set()) Completed RPC req@000000002a321d27 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_01:lustre-MDT0001-lwp-OST0000_UUID:3354:1862456998058112:0@lo:400:kworker/u8:1.0 00000100:00100000:0.0:1776178266.980914:0:20755:0:(service.c:2324:ptlrpc_server_handle_request()) Handling RPC req@000000006c7327ca pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_006:lustre-MDT0001-mdtlov_UUID+5:3356:x1862456998058368:12345-0@lo:400:kworker/u8:1.0 00000100:00100000:0.0:1776178266.980953:0:20755:0:(service.c:2372:ptlrpc_server_handle_request()) Handled RPC req@000000006c7327ca pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_006:lustre-MDT0001-mdtlov_UUID+5:3356:x1862456998058368:12345-0@lo:400:kworker/u8:1.0 Request processed in 42us (1397us total) trans 0 rc 0/0 00000100:00100000:0.0:1776178266.980959:0:20755:0:(nrs_fifo.c:241:nrs_fifo_req_stop()) NRS stop fifo request from 12345-0@lo, seq: 562 00000100:00100000:0.0:1776178266.980987:0:3355:0:(client.c:2215:ptlrpc_check_set()) Completed RPC req@000000007cbab86b pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_02:lustre-MDT0001-lwp-OST0001_UUID:3355:1862456998058240:0@lo:400:kworker/u8:1.0 00000100:00100000:2.0:1776178266.981435:0:3356:0:(client.c:2215:ptlrpc_check_set()) Completed RPC req@00000000490667a2 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_03:lustre-MDT0001-mdtlov_UUID:3356:1862456998058368:0@lo:400:kworker/u8:1.0 00000100:00100000:1.0:1776178266.997579:0:6256:0:(service.c:2372:ptlrpc_server_handle_request()) Handled RPC req@000000000cde9837 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_001:0+-99:3352:x1862456998058048:12345-0@lo:38:kworker/u8:1.0 Request processed in 20443us (20734us total) trans 0 rc -19/-19 00000100:00100000:1.0:1776178266.997597:0:6256:0:(nrs_fifo.c:241:nrs_fifo_req_stop()) NRS stop fifo request from 12345-0@lo, seq: 6673 00000100:00080000:2.0:1776178266.997873:0:3352:0:(import.c:86:import_set_state_nolock()) 0000000047601a98 lustre-MDT0000_UUID: changing import state from CONNECTING to DISCONN 00000100:00080000:2.0:1776178266.997880:0:3352:0:(import.c:1429:ptlrpc_connect_interpret()) recovery of lustre-MDT0000_UUID on 192.168.204.103@tcp failed (-19) 00000100:00100000:2.0:1776178266.997886:0:3352:0:(client.c:2215:ptlrpc_check_set()) Completed RPC req@00000000e0d60cbc pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:lustre-MDT0000-lwp-OST0000_UUID:3352:1862456998058048:0@lo:38:kworker/u8:1.0 00010000:00010000:2.0:1776178267.616700:0:6245:0:(ldlm_lockd.c:2792:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:1.0:1776178268.513389:0:3353:0:(ptlrpcd.c:415:ptlrpcd_check()) transfer 1 async RPCs [1->0] 00000100:00100000:1.0:1776178268.514174:0:3353:0:(client.c:1733:ptlrpc_send_new_req()) Sending RPC req@0000000039ff8335 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_00:lustre-MDT0001-mdtlov_UUID:3353:1862456998058560:0@lo:13:osp-pre-1-1.0 00000100:00100000:1.0:1776178268.514455:0:3353:0:(events.c:361:request_in_callback()) peer: 12345-0@lo (source: 12345-0@lo) 00000100:00100000:3.0:1776178268.518083:0:8231:0:(service.c:2152:ptlrpc_server_handle_req_in()) got req x1862456998058560 00000100:00100000:3.0:1776178268.518173:0:8231:0:(nrs_fifo.c:179:nrs_fifo_req_get()) NRS start fifo request from 12345-0@lo, seq: 513 00000100:00100000:3.0:1776178268.518182:0:8231:0:(service.c:2324:ptlrpc_server_handle_request()) Handling RPC req@00000000eeeaf4ad pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_create00:lustre-MDT0001-mdtlov_UUID+5:3353:x1862456998058560:12345-0@lo:13:osp-pre-1-1.0 00002000:00000024:3.0:1776178268.518228:0:8231:0:(ofd_obd.c:740:ofd_statfs()) blocks cached 229376 granted 491746368 pending 0 free 3923689472 avail 3692167168 00000100:00100000:3.0:1776178268.518439:0:8231:0:(service.c:2372:ptlrpc_server_handle_request()) Handled RPC req@00000000eeeaf4ad pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_create00:lustre-MDT0001-mdtlov_UUID+5:3353:x1862456998058560:12345-0@lo:13:osp-pre-1-1.0 Request processed in 258us (4169us total) trans 0 rc 0/0 00000100:00100000:3.0:1776178268.518448:0:8231:0:(nrs_fifo.c:241:nrs_fifo_req_stop()) NRS stop fifo request from 12345-0@lo, seq: 513 00000100:00100000:1.0:1776178268.518692:0:3353:0:(client.c:2215:ptlrpc_check_set()) Completed RPC req@0000000039ff8335 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_00:lustre-MDT0001-mdtlov_UUID:3353:1862456998058560:0@lo:13:osp-pre-1-1.0 00010000:00010000:2.0:1776178268.640378:0:9224:0:(ldlm_lockd.c:2792:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:2.0:1776178269.025339:0:3355:0:(client.c:1733:ptlrpc_send_new_req()) Sending RPC req@000000007cbab86b pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_02:lustre-MDT0001-mdtlov_UUID:3355:1862456998058624:0@lo:13:osp-pre-0-1.0 00000100:00100000:2.0:1776178269.025469:0:3355:0:(events.c:361:request_in_callback()) peer: 12345-0@lo (source: 12345-0@lo) 00000100:00100000:0.0:1776178269.025733:0:9554:0:(service.c:2152:ptlrpc_server_handle_req_in()) got req x1862456998058624 00000100:00100000:0.0:1776178269.025817:0:9554:0:(nrs_fifo.c:179:nrs_fifo_req_get()) NRS start fifo request from 12345-0@lo, seq: 514 00000100:00100000:0.0:1776178269.025825:0:9554:0:(service.c:2324:ptlrpc_server_handle_request()) Handling RPC req@00000000544c54c4 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_create00:lustre-MDT0001-mdtlov_UUID+5:3355:x1862456998058624:12345-0@lo:13:osp-pre-0-1.0 00002000:00000024:0.0:1776178269.025866:0:9554:0:(ofd_obd.c:740:ofd_statfs()) blocks cached 315392 granted 484883776 pending 0 free 3923689472 avail 3692167168 00000100:00100000:0.0:1776178269.025977:0:9554:0:(service.c:2372:ptlrpc_server_handle_request()) Handled RPC req@00000000544c54c4 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_create00:lustre-MDT0001-mdtlov_UUID+5:3355:x1862456998058624:12345-0@lo:13:osp-pre-0-1.0 Request processed in 151us (518us total) trans 0 rc 0/0 00000100:00100000:0.0:1776178269.025985:0:9554:0:(nrs_fifo.c:241:nrs_fifo_req_stop()) NRS stop fifo request from 12345-0@lo, seq: 514 00000100:00100000:2.0:1776178269.026349:0:3355:0:(client.c:2215:ptlrpc_check_set()) Completed RPC req@000000007cbab86b pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_02:lustre-MDT0001-mdtlov_UUID:3355:1862456998058624:0@lo:13:osp-pre-0-1.0 00010000:00010000:2.0:1776178269.665040:0:9225:0:(ldlm_lockd.c:2792:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:3.0:1776178270.507398:0:3348:0:(events.c:361:request_in_callback()) peer: 12345-192.168.204.3@tcp (source: 12345-192.168.204.3@tcp) 00000100:00100000:3.0:1776178270.507490:0:3348:0:(events.c:361:request_in_callback()) peer: 12345-192.168.204.3@tcp (source: 12345-192.168.204.3@tcp) 00000100:00100000:3.0:1776178270.507533:0:3348:0:(events.c:361:request_in_callback()) peer: 12345-192.168.204.3@tcp (source: 12345-192.168.204.3@tcp) 00000100:00100000:3.0:1776178270.507564:0:3348:0:(events.c:361:request_in_callback()) peer: 12345-192.168.204.3@tcp (source: 12345-192.168.204.3@tcp) 00000100:00100000:3.0:1776178270.507603:0:3348:0:(events.c:361:request_in_callback()) peer: 12345-192.168.204.3@tcp (source: 12345-192.168.204.3@tcp) 00000100:00100000:2.0:1776178270.507643:0:20137:0:(service.c:2152:ptlrpc_server_handle_req_in()) got req x1862456979113152 00000100:00100000:3.0:1776178270.507643:0:14440:0:(service.c:2152:ptlrpc_server_handle_req_in()) got req x1862456979113408 00000100:00100000:3.0:1776178270.507699:0:14440:0:(service.c:2152:ptlrpc_server_handle_req_in()) got req x1862456979113344 00000100:00100000:2.0:1776178270.507703:0:20137:0:(nrs_fifo.c:179:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.204.3@tcp, seq: 181 00000100:00100000:2.0:1776178270.507711:0:20137:0:(service.c:2324:ptlrpc_server_handle_request()) Handling RPC req@000000000084a3e4 pname:cluuid+ref:pid:xid:nid:opc:job ll_mgs_0000:5b5c66b1-1a2f-4651-8548-0c75b9a38a94+9:2270:x1862456979113152:12345-192.168.204.3@tcp:400:kworker/u8:3.0 00000100:00100000:3.0:1776178270.507716:0:14440:0:(nrs_fifo.c:179:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.204.3@tcp, seq: 563 00000100:00100000:3.0:1776178270.507720:0:14440:0:(service.c:2324:ptlrpc_server_handle_request()) Handling RPC req@0000000022d4dc29 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_004:b50963bd-8eba-4432-9a54-8888d00e3fa1+9:2270:x1862456979113408:12345-192.168.204.3@tcp:400:kworker/u8:3.0 00000100:00100000:2.0:1776178270.507793:0:20137:0:(service.c:2372:ptlrpc_server_handle_request()) Handled RPC req@000000000084a3e4 pname:cluuid+ref:pid:xid:nid:opc:job ll_mgs_0000:5b5c66b1-1a2f-4651-8548-0c75b9a38a94+9:2270:x1862456979113152:12345-192.168.204.3@tcp:400:kworker/u8:3.0 Request processed in 82us (260us total) trans 0 rc 0/0 00000100:00100000:2.0:1776178270.507802:0:20137:0:(nrs_fifo.c:241:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.204.3@tcp, seq: 181 00000100:00100000:2.0:1776178270.507874:0:20759:0:(nrs_fifo.c:179:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.204.3@tcp, seq: 564 00000100:00100000:2.0:1776178270.507879:0:20759:0:(service.c:2324:ptlrpc_server_handle_request()) Handling RPC req@00000000a6227918 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_010:b50963bd-8eba-4432-9a54-8888d00e3fa1+8:2270:x1862456979113344:12345-192.168.204.3@tcp:400:kworker/u8:3.0 00000100:00100000:2.0:1776178270.507907:0:20759:0:(service.c:2372:ptlrpc_server_handle_request()) Handled RPC req@00000000a6227918 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_010:b50963bd-8eba-4432-9a54-8888d00e3fa1+8:2270:x1862456979113344:12345-192.168.204.3@tcp:400:kworker/u8:3.0 Request processed in 28us (343us total) trans 0 rc 0/0 00000100:00100000:2.0:1776178270.507914:0:20759:0:(nrs_fifo.c:241:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.204.3@tcp, seq: 564 00000100:00100000:2.0:1776178270.507939:0:6255:0:(service.c:2152:ptlrpc_server_handle_req_in()) got req x1862456979113280 00000100:00100000:2.0:1776178270.507962:0:6255:0:(service.c:2152:ptlrpc_server_handle_req_in()) got req x1862456979113216 00000100:00100000:2.0:1776178270.507993:0:6255:0:(nrs_fifo.c:179:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.204.3@tcp, seq: 967 00000100:00100000:2.0:1776178270.507997:0:6255:0:(service.c:2324:ptlrpc_server_handle_request()) Handling RPC req@00000000456acdaf pname:cluuid+ref:pid:xid:nid:opc:job mdt00_000:b50963bd-8eba-4432-9a54-8888d00e3fa1+6:2270:x1862456979113280:12345-192.168.204.3@tcp:400:kworker/u8:3.0 00000100:00100000:2.0:1776178270.508026:0:6255:0:(service.c:2372:ptlrpc_server_handle_request()) Handled RPC req@00000000456acdaf pname:cluuid+ref:pid:xid:nid:opc:job mdt00_000:b50963bd-8eba-4432-9a54-8888d00e3fa1+6:2270:x1862456979113280:12345-192.168.204.3@tcp:400:kworker/u8:3.0 Request processed in 28us (649us total) trans 0 rc 0/0 00000100:00100000:2.0:1776178270.508031:0:6255:0:(nrs_fifo.c:241:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.204.3@tcp, seq: 967 00000100:00100000:2.0:1776178270.508044:0:6255:0:(nrs_fifo.c:179:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.204.3@tcp, seq: 6676 00000100:00100000:2.0:1776178270.508047:0:6255:0:(service.c:2324:ptlrpc_server_handle_request()) Handling RPC req@000000004420855e pname:cluuid+ref:pid:xid:nid:opc:job mdt00_000:0+-99:2269:x1862456979113216:12345-192.168.204.3@tcp:38:kworker/u8:3.0 00010000:02000000:2.0:1776178270.508092:0:6255:0:(ldlm_lib.c:1139:target_handle_connect()) lustre-MDT0000: Not available for connect from 192.168.204.3@tcp (stopping) 00000100:00100000:2.0:1776178270.508118:0:6255:0:(service.c:2372:ptlrpc_server_handle_request()) Handled RPC req@000000004420855e pname:cluuid+ref:pid:xid:nid:opc:job mdt00_000:0+-99:2269:x1862456979113216:12345-192.168.204.3@tcp:38:kworker/u8:3.0 Request processed in 70us (515us total) trans 0 rc -19/-19 00000100:00100000:2.0:1776178270.508124:0:6255:0:(nrs_fifo.c:241:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.204.3@tcp, seq: 6676 00000100:00100000:3.0:1776178270.510545:0:14440:0:(service.c:2372:ptlrpc_server_handle_request()) Handled RPC req@0000000022d4dc29 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_004:b50963bd-8eba-4432-9a54-8888d00e3fa1+9:2270:x1862456979113408:12345-192.168.204.3@tcp:400:kworker/u8:3.0 Request processed in 2813us (3045us total) trans 0 rc 0/0 00000100:00100000:3.0:1776178270.510560:0:14440:0:(nrs_fifo.c:241:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.204.3@tcp, seq: 563 00010000:00010000:0.0:1776178270.688221:0:6244:0:(ldlm_lockd.c:2792:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00010000:00010000:1.0:1776178271.712489:0:6245:0:(ldlm_lockd.c:2792:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00080000:2.0:1776178272.099666:0:37:0:(pinger.c:251:ptlrpc_pinger_process_import()) lustre-MDT0000-lwp-OST0000_UUID->lustre-MDT0000_UUID: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00080000:2.0:1776178272.099678:0:37:0:(recover.c:57:ptlrpc_initiate_recovery()) lustre-MDT0000_UUID: starting recovery 00000100:00080000:2.0:1776178272.099684:0:37:0:(import.c:86:import_set_state_nolock()) 0000000047601a98 lustre-MDT0000_UUID: changing import state from DISCONN to CONNECTING 00000100:00080000:2.0:1776178272.099699:0:37:0:(import.c:534:import_select_connection()) lustre-MDT0000-lwp-OST0000: connect to NID 0@lo last attempt 974 00000100:00080000:2.0:1776178272.099705:0:37:0:(import.c:581:import_select_connection()) lustre-MDT0000-lwp-OST0000: tried all connections, increasing latency to 20s 00000100:00080000:2.0:1776178272.099713:0:37:0:(import.c:615:import_select_connection()) lustre-MDT0000-lwp-OST0000: import 0000000047601a98 using connection 192.168.204.103@tcp/0@lo 00000100:00100000:2.0:1776178272.099744:0:37:0:(import.c:807:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5) req@00000000d6e5d9ba x1862456998058752/t0(0) o38->lustre-MDT0000-lwp-OST0000@0@lo:12/10 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' 00000100:00080000:2.0:1776178272.099804:0:37:0:(pinger.c:251:ptlrpc_pinger_process_import()) lustre-MDT0000-lwp-OST0001_UUID->lustre-MDT0000_UUID: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00080000:2.0:1776178272.099808:0:37:0:(recover.c:57:ptlrpc_initiate_recovery()) lustre-MDT0000_UUID: starting recovery 00000100:00100000:3.0:1776178272.099809:0:3356:0:(client.c:1733:ptlrpc_send_new_req()) Sending RPC req@0000000023562990 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_03:ac61f839-0e8e-4cac-9c59-08bcfbbad330:3356:1862456998058688:0@lo:400:kworker/u8:2.0 00000100:00080000:2.0:1776178272.099811:0:37:0:(import.c:86:import_set_state_nolock()) 00000000dfc8db13 lustre-MDT0000_UUID: changing import state from DISCONN to CONNECTING 00000100:00080000:2.0:1776178272.099816:0:37:0:(import.c:534:import_select_connection()) lustre-MDT0000-lwp-OST0001: connect to NID 0@lo last attempt 974 00000100:00080000:2.0:1776178272.099819:0:37:0:(import.c:581:import_select_connection()) lustre-MDT0000-lwp-OST0001: tried all connections, increasing latency to 20s 00000100:00080000:2.0:1776178272.099823:0:37:0:(import.c:615:import_select_connection()) lustre-MDT0000-lwp-OST0001: import 00000000dfc8db13 using connection 192.168.204.103@tcp/0@lo 00000100:00100000:2.0:1776178272.099838:0:37:0:(import.c:807:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5) req@0000000038c09127 x1862456998058880/t0(0) o38->lustre-MDT0000-lwp-OST0001@0@lo:12/10 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' 00000100:00080000:2.0:1776178272.099856:0:37:0:(pinger.c:251:ptlrpc_pinger_process_import()) lustre-MDT0001-mdtlov_UUID->lustre-MDT0000_UUID: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00080000:2.0:1776178272.099859:0:37:0:(recover.c:57:ptlrpc_initiate_recovery()) lustre-MDT0000_UUID: starting recovery 00000100:00080000:2.0:1776178272.099861:0:37:0:(import.c:86:import_set_state_nolock()) 00000000ba4f1821 lustre-MDT0000_UUID: changing import state from DISCONN to CONNECTING 00000100:00080000:2.0:1776178272.099864:0:37:0:(import.c:534:import_select_connection()) lustre-MDT0000-osp-MDT0001: connect to NID 0@lo last attempt 974 00000100:00080000:2.0:1776178272.099866:0:37:0:(import.c:581:import_select_connection()) lustre-MDT0000-osp-MDT0001: tried all connections, increasing latency to 20s 00000100:00080000:2.0:1776178272.099871:0:37:0:(import.c:615:import_select_connection()) lustre-MDT0000-osp-MDT0001: import 00000000ba4f1821 using connection 192.168.204.103@tcp/0@lo 00000100:00100000:2.0:1776178272.099884:0:37:0:(import.c:807:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5) req@00000000167dc455 x1862456998059008/t0(0) o38->lustre-MDT0000-osp-MDT0001@0@lo:24/4 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' 00000100:00080000:2.0:1776178272.099908:0:37:0:(pinger.c:251:ptlrpc_pinger_process_import()) lustre-MDT0000-lwp-MDT0001_UUID->lustre-MDT0000_UUID: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00080000:2.0:1776178272.099911:0:37:0:(recover.c:57:ptlrpc_initiate_recovery()) lustre-MDT0000_UUID: starting recovery 00000100:00080000:2.0:1776178272.099913:0:37:0:(import.c:86:import_set_state_nolock()) 00000000a7c55b1f lustre-MDT0000_UUID: changing import state from DISCONN to CONNECTING 00000100:00080000:2.0:1776178272.099916:0:37:0:(import.c:534:import_select_connection()) lustre-MDT0000-lwp-MDT0001: connect to NID 0@lo last attempt 974 00000100:00080000:2.0:1776178272.099919:0:37:0:(import.c:581:import_select_connection()) lustre-MDT0000-lwp-MDT0001: tried all connections, increasing latency to 20s 00000100:00100000:3.0:1776178272.099922:0:3356:0:(events.c:361:request_in_callback()) peer: 12345-0@lo (source: 12345-0@lo) 00000100:00080000:2.0:1776178272.099925:0:37:0:(import.c:615:import_select_connection()) lustre-MDT0000-lwp-MDT0001: import 00000000a7c55b1f using connection 192.168.204.103@tcp/0@lo 00000100:00100000:2.0:1776178272.099937:0:37:0:(import.c:807:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5) req@0000000020afd0b3 x1862456998059200/t0(0) o38->lustre-MDT0000-lwp-MDT0001@0@lo:12/10 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' 00000100:00100000:2.0:1776178272.099972:0:3354:0:(client.c:1733:ptlrpc_send_new_req()) Sending RPC req@00000000d0706d7c pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_01:lustre-MDT0001-lwp-OST0001_UUID:3354:1862456998058944:0@lo:400:kworker/u8:2.0 00000100:00100000:3.0:1776178272.099975:0:3356:0:(client.c:1733:ptlrpc_send_new_req()) Sending RPC req@00000000bcefda18 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_03:lustre-MDT0001-mdtlov_UUID:3356:1862456998059136:0@lo:400:kworker/u8:2.0 00000100:00100000:3.0:1776178272.100016:0:3356:0:(events.c:361:request_in_callback()) peer: 12345-0@lo (source: 12345-0@lo) 00000100:00100000:2.0:1776178272.100026:0:3354:0:(events.c:361:request_in_callback()) peer: 12345-0@lo (source: 12345-0@lo) 00000100:00100000:3.0:1776178272.100039:0:3356:0:(ptlrpcd.c:415:ptlrpcd_check()) transfer 1 async RPCs [2->3] 00000100:00100000:3.0:1776178272.100048:0:3356:0:(client.c:1733:ptlrpc_send_new_req()) Sending RPC req@0000000021b0d516 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_03:lustre-MDT0001-mdtlov_UUID:3356:1862456998059072:0@lo:400:kworker/u8:2.0 00000100:00100000:2.0:1776178272.100054:0:3354:0:(ptlrpcd.c:415:ptlrpcd_check()) transfer 1 async RPCs [0->1] 00000100:00100000:2.0:1776178272.100114:0:3354:0:(client.c:1733:ptlrpc_send_new_req()) Sending RPC req@00000000b13c16eb pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_01:lustre-MDT0001-lwp-OST0000_UUID:3354:1862456998058816:0@lo:400:kworker/u8:2.0 00000100:00100000:0.0:1776178272.100228:0:20140:0:(service.c:2152:ptlrpc_server_handle_req_in()) got req x1862456998058688 00000100:00100000:0.0:1776178272.100291:0:20140:0:(nrs_fifo.c:179:nrs_fifo_req_get()) NRS start fifo request from 12345-0@lo, seq: 182 00000100:00100000:0.0:1776178272.100298:0:20140:0:(service.c:2324:ptlrpc_server_handle_request()) Handling RPC req@00000000ff2b550f pname:cluuid+ref:pid:xid:nid:opc:job ll_mgs_0002:ac61f839-0e8e-4cac-9c59-08bcfbbad330+18:3356:x1862456998058688:12345-0@lo:400:kworker/u8:2.0 00000100:00100000:1.0:1776178272.100369:0:3352:0:(client.c:1733:ptlrpc_send_new_req()) Sending RPC req@00000000d6e5d9ba pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:lustre-MDT0000-lwp-OST0000_UUID:3352:1862456998058752:0@lo:38:kworker/u8:2.0 00000100:00100000:2.0:1776178272.107366:0:3354:0:(events.c:361:request_in_callback()) peer: 12345-0@lo (source: 12345-0@lo) 00000100:00100000:2.0:1776178272.108623:0:6256:0:(service.c:2152:ptlrpc_server_handle_req_in()) got req x1862456998058944 00000100:00100000:2.0:1776178272.108702:0:6256:0:(service.c:2152:ptlrpc_server_handle_req_in()) got req x1862456998058816 00000100:00100000:2.0:1776178272.108724:0:6256:0:(nrs_fifo.c:179:nrs_fifo_req_get()) NRS start fifo request from 12345-0@lo, seq: 968 00000100:00100000:2.0:1776178272.108731:0:6256:0:(service.c:2324:ptlrpc_server_handle_request()) Handling RPC req@0000000078361ddf pname:cluuid+ref:pid:xid:nid:opc:job mdt00_001:lustre-MDT0001-lwp-OST0001_UUID+5:3354:x1862456998058944:12345-0@lo:400:kworker/u8:2.0 00000100:00100000:2.0:1776178272.108813:0:6256:0:(service.c:2372:ptlrpc_server_handle_request()) Handled RPC req@0000000078361ddf pname:cluuid+ref:pid:xid:nid:opc:job mdt00_001:lustre-MDT0001-lwp-OST0001_UUID+5:3354:x1862456998058944:12345-0@lo:400:kworker/u8:2.0 Request processed in 82us (8789us total) trans 0 rc 0/0 00000100:00100000:2.0:1776178272.108821:0:6256:0:(nrs_fifo.c:241:nrs_fifo_req_stop()) NRS stop fifo request from 12345-0@lo, seq: 968 00000100:00100000:2.0:1776178272.108839:0:6256:0:(nrs_fifo.c:179:nrs_fifo_req_get()) NRS start fifo request from 12345-0@lo, seq: 969 00000100:00100000:2.0:1776178272.108842:0:6256:0:(service.c:2324:ptlrpc_server_handle_request()) Handling RPC req@00000000e83ac5a7 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_001:lustre-MDT0001-lwp-OST0000_UUID+5:3354:x1862456998058816:12345-0@lo:400:kworker/u8:2.0 00000100:00100000:2.0:1776178272.108871:0:6256:0:(service.c:2372:ptlrpc_server_handle_request()) Handled RPC req@00000000e83ac5a7 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_001:lustre-MDT0001-lwp-OST0000_UUID+5:3354:x1862456998058816:12345-0@lo:400:kworker/u8:2.0 Request processed in 28us (1518us total) trans 0 rc 0/0 00000100:00100000:2.0:1776178272.108877:0:6256:0:(nrs_fifo.c:241:nrs_fifo_req_stop()) NRS stop fifo request from 12345-0@lo, seq: 969 00000100:00100000:2.0:1776178272.108922:0:3354:0:(client.c:2215:ptlrpc_check_set()) Completed RPC req@00000000b13c16eb pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_01:lustre-MDT0001-lwp-OST0000_UUID:3354:1862456998058816:0@lo:400:kworker/u8:2.0 00000100:00100000:2.0:1776178272.108933:0:3354:0:(client.c:2215:ptlrpc_check_set()) Completed RPC req@00000000d0706d7c pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_01:lustre-MDT0001-lwp-OST0001_UUID:3354:1862456998058944:0@lo:400:kworker/u8:2.0 00000100:00100000:2.0:1776178272.109088:0:14440:0:(service.c:2152:ptlrpc_server_handle_req_in()) got req x1862456998059136 00000100:00100000:2.0:1776178272.109134:0:14440:0:(nrs_fifo.c:179:nrs_fifo_req_get()) NRS start fifo request from 12345-0@lo, seq: 565 00000100:00100000:2.0:1776178272.109138:0:14440:0:(service.c:2324:ptlrpc_server_handle_request()) Handling RPC req@0000000049b1b339 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_004:lustre-MDT0001-mdtlov_UUID+5:3356:x1862456998059136:12345-0@lo:400:kworker/u8:2.0 00000100:00100000:2.0:1776178272.109172:0:14440:0:(service.c:2372:ptlrpc_server_handle_request()) Handled RPC req@0000000049b1b339 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_004:lustre-MDT0001-mdtlov_UUID+5:3356:x1862456998059136:12345-0@lo:400:kworker/u8:2.0 Request processed in 33us (9156us total) trans 0 rc 0/0 00000100:00100000:2.0:1776178272.109178:0:14440:0:(nrs_fifo.c:241:nrs_fifo_req_stop()) NRS stop fifo request from 12345-0@lo, seq: 565 00000100:00100000:3.0:1776178272.112199:0:3356:0:(events.c:361:request_in_callback()) peer: 12345-0@lo (source: 12345-0@lo) 00000100:00100000:3.0:1776178272.112283:0:3356:0:(client.c:2215:ptlrpc_check_set()) Completed RPC req@00000000bcefda18 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_03:lustre-MDT0001-mdtlov_UUID:3356:1862456998059136:0@lo:400:kworker/u8:2.0 00000100:00100000:2.0:1776178272.112441:0:20755:0:(service.c:2152:ptlrpc_server_handle_req_in()) got req x1862456998059072 00000100:00100000:2.0:1776178272.112514:0:20755:0:(nrs_fifo.c:179:nrs_fifo_req_get()) NRS start fifo request from 12345-0@lo, seq: 566 00000100:00100000:2.0:1776178272.112521:0:20755:0:(service.c:2324:ptlrpc_server_handle_request()) Handling RPC req@00000000ed90ff75 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_006:lustre-MDT0001-mdtlov_UUID+5:3356:x1862456998059072:12345-0@lo:400:kworker/u8:2.0 00000100:00100000:2.0:1776178272.112588:0:20755:0:(service.c:2372:ptlrpc_server_handle_request()) Handled RPC req@00000000ed90ff75 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_006:lustre-MDT0001-mdtlov_UUID+5:3356:x1862456998059072:12345-0@lo:400:kworker/u8:2.0 Request processed in 66us (402us total) trans 0 rc 0/0 00000100:00100000:2.0:1776178272.112596:0:20755:0:(nrs_fifo.c:241:nrs_fifo_req_stop()) NRS stop fifo request from 12345-0@lo, seq: 566 00000100:00100000:3.0:1776178272.112678:0:3356:0:(client.c:2215:ptlrpc_check_set()) Completed RPC req@0000000021b0d516 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_03:lustre-MDT0001-mdtlov_UUID:3356:1862456998059072:0@lo:400:kworker/u8:2.0 00000100:00100000:1.0:1776178272.112909:0:3352:0:(events.c:361:request_in_callback()) peer: 12345-0@lo (source: 12345-0@lo) 00000100:00100000:1.0:1776178272.112943:0:3352:0:(client.c:1733:ptlrpc_send_new_req()) Sending RPC req@0000000038c09127 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:lustre-MDT0000-lwp-OST0001_UUID:3352:1862456998058880:0@lo:38:kworker/u8:2.0 00000100:00100000:2.0:1776178272.112956:0:8250:0:(service.c:2152:ptlrpc_server_handle_req_in()) got req x1862456998058752 00000100:00100000:2.0:1776178272.112989:0:8250:0:(nrs_fifo.c:179:nrs_fifo_req_get()) NRS start fifo request from 12345-0@lo, seq: 6677 00000100:00100000:2.0:1776178272.112993:0:8250:0:(service.c:2324:ptlrpc_server_handle_request()) Handling RPC req@00000000f4172b4c pname:cluuid+ref:pid:xid:nid:opc:job mdt00_003:0+-99:3352:x1862456998058752:12345-0@lo:38:kworker/u8:2.0 00010000:02000000:2.0:1776178272.113009:0:8250:0:(ldlm_lib.c:1139:target_handle_connect()) lustre-MDT0000: Not available for connect from 0@lo (stopping) 00000100:00100000:2.0:1776178272.113043:0:8250:0:(service.c:2372:ptlrpc_server_handle_request()) Handled RPC req@00000000f4172b4c pname:cluuid+ref:pid:xid:nid:opc:job mdt00_003:0+-99:3352:x1862456998058752:12345-0@lo:38:kworker/u8:2.0 Request processed in 50us (135us total) trans 0 rc -19/-19 00000100:00100000:2.0:1776178272.113048:0:8250:0:(nrs_fifo.c:241:nrs_fifo_req_stop()) NRS stop fifo request from 12345-0@lo, seq: 6677 00000100:00100000:1.0:1776178272.113101:0:3352:0:(events.c:361:request_in_callback()) peer: 12345-0@lo (source: 12345-0@lo) 00000100:00100000:1.0:1776178272.113122:0:3352:0:(client.c:1733:ptlrpc_send_new_req()) Sending RPC req@00000000167dc455 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:lustre-MDT0001-mdtlov_UUID:3352:1862456998059008:0@lo:38:kworker/u8:2.0 00000100:00100000:3.0:1776178272.113135:0:6255:0:(service.c:2152:ptlrpc_server_handle_req_in()) got req x1862456998058880 00000100:00100000:1.0:1776178272.113155:0:3352:0:(events.c:361:request_in_callback()) peer: 12345-0@lo (source: 12345-0@lo) 00000100:00100000:3.0:1776178272.113166:0:6255:0:(nrs_fifo.c:179:nrs_fifo_req_get()) NRS start fifo request from 12345-0@lo, seq: 6678 00000100:00100000:3.0:1776178272.113172:0:6255:0:(service.c:2324:ptlrpc_server_handle_request()) Handling RPC req@00000000ba9bc15d pname:cluuid+ref:pid:xid:nid:opc:job mdt00_000:0+-99:3352:x1862456998058880:12345-0@lo:38:kworker/u8:2.0 00000100:00100000:1.0:1776178272.113179:0:3352:0:(client.c:1733:ptlrpc_send_new_req()) Sending RPC req@0000000020afd0b3 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:lustre-MDT0000-lwp-MDT0001_UUID:3352:1862456998059200:0@lo:38:kworker/u8:2.0 00010000:02000000:3.0:1776178272.113181:0:6255:0:(ldlm_lib.c:1139:target_handle_connect()) lustre-MDT0000: Not available for connect from 0@lo (stopping) 00000100:00100000:1.0:1776178272.113214:0:3352:0:(events.c:361:request_in_callback()) peer: 12345-0@lo (source: 12345-0@lo) 00000100:00100000:3.0:1776178272.113222:0:6255:0:(service.c:2372:ptlrpc_server_handle_request()) Handled RPC req@00000000ba9bc15d pname:cluuid+ref:pid:xid:nid:opc:job mdt00_000:0+-99:3352:x1862456998058880:12345-0@lo:38:kworker/u8:2.0 Request processed in 49us (121us total) trans 0 rc -19/-19 00000100:00100000:3.0:1776178272.113230:0:6255:0:(nrs_fifo.c:241:nrs_fifo_req_stop()) NRS stop fifo request from 12345-0@lo, seq: 6678 00000100:00100000:3.0:1776178272.113246:0:6255:0:(service.c:2152:ptlrpc_server_handle_req_in()) got req x1862456998059200 00000100:00080000:1.0:1776178272.113250:0:3352:0:(import.c:86:import_set_state_nolock()) 0000000047601a98 lustre-MDT0000_UUID: changing import state from CONNECTING to DISCONN 00000100:00080000:1.0:1776178272.113255:0:3352:0:(import.c:1429:ptlrpc_connect_interpret()) recovery of lustre-MDT0000_UUID on 192.168.204.103@tcp failed (-19) 00000100:00100000:1.0:1776178272.113260:0:3352:0:(client.c:2215:ptlrpc_check_set()) Completed RPC req@00000000d6e5d9ba pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:lustre-MDT0000-lwp-OST0000_UUID:3352:1862456998058752:0@lo:38:kworker/u8:2.0 00000100:00080000:1.0:1776178272.113271:0:3352:0:(import.c:86:import_set_state_nolock()) 00000000dfc8db13 lustre-MDT0000_UUID: changing import state from CONNECTING to DISCONN 00000100:00100000:3.0:1776178272.113275:0:6255:0:(nrs_fifo.c:179:nrs_fifo_req_get()) NRS start fifo request from 12345-0@lo, seq: 6679 00000100:00080000:1.0:1776178272.113276:0:3352:0:(import.c:1429:ptlrpc_connect_interpret()) recovery of lustre-MDT0000_UUID on 192.168.204.103@tcp failed (-19) 00000100:00100000:3.0:1776178272.113281:0:6255:0:(service.c:2324:ptlrpc_server_handle_request()) Handling RPC req@000000006165f052 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_000:0+-99:3352:x1862456998059200:12345-0@lo:38:kworker/u8:2.0 00000100:00100000:1.0:1776178272.113282:0:3352:0:(client.c:2215:ptlrpc_check_set()) Completed RPC req@0000000038c09127 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:lustre-MDT0000-lwp-OST0001_UUID:3352:1862456998058880:0@lo:38:kworker/u8:2.0 00010000:02000000:3.0:1776178272.113294:0:6255:0:(ldlm_lib.c:1139:target_handle_connect()) lustre-MDT0000: Not available for connect from 0@lo (stopping) 00000100:00100000:1.0:1776178272.113314:0:14128:0:(service.c:2152:ptlrpc_server_handle_req_in()) got req x1862456998059008 00000100:00100000:3.0:1776178272.113328:0:6255:0:(service.c:2372:ptlrpc_server_handle_request()) Handled RPC req@000000006165f052 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_000:0+-99:3352:x1862456998059200:12345-0@lo:38:kworker/u8:2.0 Request processed in 46us (113us total) trans 0 rc -19/-19 00000100:00100000:3.0:1776178272.113336:0:6255:0:(nrs_fifo.c:241:nrs_fifo_req_stop()) NRS stop fifo request from 12345-0@lo, seq: 6679 00000100:00100000:1.0:1776178272.113345:0:14128:0:(nrs_fifo.c:179:nrs_fifo_req_get()) NRS start fifo request from 12345-0@lo, seq: 721 00000100:00100000:1.0:1776178272.113351:0:14128:0:(service.c:2324:ptlrpc_server_handle_request()) Handling RPC req@00000000b7440871 pname:cluuid+ref:pid:xid:nid:opc:job mdt_out00_002:0+-99:3352:x1862456998059008:12345-0@lo:38:kworker/u8:2.0 00010000:02000000:1.0:1776178272.113359:0:14128:0:(ldlm_lib.c:1139:target_handle_connect()) lustre-MDT0000: Not available for connect from 0@lo (stopping) 00000100:00100000:1.0:1776178272.114702:0:14128:0:(service.c:2372:ptlrpc_server_handle_request()) Handled RPC req@00000000b7440871 pname:cluuid+ref:pid:xid:nid:opc:job mdt_out00_002:0+-99:3352:x1862456998059008:12345-0@lo:38:kworker/u8:2.0 Request processed in 1340us (1537us total) trans 0 rc -19/-19 00000100:00100000:1.0:1776178272.114717:0:14128:0:(nrs_fifo.c:241:nrs_fifo_req_stop()) NRS stop fifo request from 12345-0@lo, seq: 721 00000100:00080000:1.0:1776178272.114786:0:3352:0:(import.c:86:import_set_state_nolock()) 00000000ba4f1821 lustre-MDT0000_UUID: changing import state from CONNECTING to DISCONN 00000100:00080000:1.0:1776178272.114791:0:3352:0:(import.c:1429:ptlrpc_connect_interpret()) recovery of lustre-MDT0000_UUID on 192.168.204.103@tcp failed (-19) 00000100:00100000:1.0:1776178272.114795:0:3352:0:(client.c:2215:ptlrpc_check_set()) Completed RPC req@00000000167dc455 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:lustre-MDT0001-mdtlov_UUID:3352:1862456998059008:0@lo:38:kworker/u8:2.0 00000100:00080000:1.0:1776178272.114805:0:3352:0:(import.c:86:import_set_state_nolock()) 00000000a7c55b1f lustre-MDT0000_UUID: changing import state from CONNECTING to DISCONN 00000100:00080000:1.0:1776178272.114808:0:3352:0:(import.c:1429:ptlrpc_connect_interpret()) recovery of lustre-MDT0000_UUID on 192.168.204.103@tcp failed (-19) 00000100:00100000:1.0:1776178272.114811:0:3352:0:(client.c:2215:ptlrpc_check_set()) Completed RPC req@0000000020afd0b3 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:lustre-MDT0000-lwp-MDT0001_UUID:3352:1862456998059200:0@lo:38:kworker/u8:2.0 00000100:00100000:0.0:1776178272.117054:0:20140:0:(service.c:2372:ptlrpc_server_handle_request()) Handled RPC req@00000000ff2b550f pname:cluuid+ref:pid:xid:nid:opc:job ll_mgs_0002:ac61f839-0e8e-4cac-9c59-08bcfbbad330+18:3356:x1862456998058688:12345-0@lo:400:kworker/u8:2.0 Request processed in 16743us (17124us total) trans 0 rc 0/0 00000100:00100000:0.0:1776178272.117090:0:20140:0:(nrs_fifo.c:241:nrs_fifo_req_stop()) NRS stop fifo request from 12345-0@lo, seq: 182 00000100:00100000:3.0:1776178272.117328:0:3356:0:(client.c:2215:ptlrpc_check_set()) Completed RPC req@0000000023562990 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_03:ac61f839-0e8e-4cac-9c59-08bcfbbad330:3356:1862456998058688:0@lo:400:kworker/u8:2.0 00010000:00010000:2.0:1776178272.738122:0:9224:0:(ldlm_lockd.c:2792:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:0.0:1776178273.633430:0:3354:0:(ptlrpcd.c:415:ptlrpcd_check()) transfer 1 async RPCs [0->1] 00000100:00100000:0.0:1776178273.633457:0:3354:0:(client.c:1733:ptlrpc_send_new_req()) Sending RPC req@0000000039ff8335 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_01:lustre-MDT0001-mdtlov_UUID:3354:1862456998059264:0@lo:13:osp-pre-1-1.0 00000100:00100000:0.0:1776178273.633538:0:3354:0:(events.c:361:request_in_callback()) peer: 12345-0@lo (source: 12345-0@lo) 00000100:00100000:1.0:1776178273.633725:0:8231:0:(service.c:2152:ptlrpc_server_handle_req_in()) got req x1862456998059264 00000100:00100000:1.0:1776178273.633774:0:8231:0:(nrs_fifo.c:179:nrs_fifo_req_get()) NRS start fifo request from 12345-0@lo, seq: 515 00000100:00100000:1.0:1776178273.633780:0:8231:0:(service.c:2324:ptlrpc_server_handle_request()) Handling RPC req@000000008c593d89 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_create00:lustre-MDT0001-mdtlov_UUID+5:3354:x1862456998059264:12345-0@lo:13:osp-pre-1-1.0 00002000:00000024:1.0:1776178273.633810:0:8231:0:(ofd_obd.c:740:ofd_statfs()) blocks cached 229376 granted 491746368 pending 0 free 3923689472 avail 3692167168 00000100:00100000:1.0:1776178273.633860:0:8231:0:(service.c:2372:ptlrpc_server_handle_request()) Handled RPC req@000000008c593d89 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_create00:lustre-MDT0001-mdtlov_UUID+5:3354:x1862456998059264:12345-0@lo:13:osp-pre-1-1.0 Request processed in 79us (323us total) trans 0 rc 0/0 00000100:00100000:1.0:1776178273.633878:0:8231:0:(nrs_fifo.c:241:nrs_fifo_req_stop()) NRS stop fifo request from 12345-0@lo, seq: 515 00000100:00100000:3.0:1776178273.633984:0:3354:0:(client.c:2215:ptlrpc_check_set()) Completed RPC req@0000000039ff8335 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_01:lustre-MDT0001-mdtlov_UUID:3354:1862456998059264:0@lo:13:osp-pre-1-1.0 00010000:00010000:2.0:1776178273.760516:0:9225:0:(ldlm_lockd.c:2792:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:2.0:1776178274.145336:0:3353:0:(ptlrpcd.c:415:ptlrpcd_check()) transfer 1 async RPCs [1->0] 00000100:00100000:2.0:1776178274.145378:0:3353:0:(client.c:1733:ptlrpc_send_new_req()) Sending RPC req@00000000d668bb9c pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_00:lustre-MDT0001-mdtlov_UUID:3353:1862456998059328:0@lo:13:osp-pre-0-1.0 00000100:00100000:2.0:1776178274.145478:0:3353:0:(events.c:361:request_in_callback()) peer: 12345-0@lo (source: 12345-0@lo) 00000100:00100000:0.0:1776178274.155719:0:9554:0:(service.c:2152:ptlrpc_server_handle_req_in()) got req x1862456998059328 00000100:00100000:0.0:1776178274.155801:0:9554:0:(nrs_fifo.c:179:nrs_fifo_req_get()) NRS start fifo request from 12345-0@lo, seq: 516 00000100:00100000:0.0:1776178274.155808:0:9554:0:(service.c:2324:ptlrpc_server_handle_request()) Handling RPC req@00000000d0706d7c pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_create00:lustre-MDT0001-mdtlov_UUID+5:3353:x1862456998059328:12345-0@lo:13:osp-pre-0-1.0 00002000:00000024:0.0:1776178274.155854:0:9554:0:(ofd_obd.c:740:ofd_statfs()) blocks cached 315392 granted 484883776 pending 0 free 3923689472 avail 3692167168 00000100:00100000:0.0:1776178274.155970:0:9554:0:(service.c:2372:ptlrpc_server_handle_request()) Handled RPC req@00000000d0706d7c pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_create00:lustre-MDT0001-mdtlov_UUID+5:3353:x1862456998059328:12345-0@lo:13:osp-pre-0-1.0 Request processed in 161us (10493us total) trans 0 rc 0/0 00000100:00100000:0.0:1776178274.155979:0:9554:0:(nrs_fifo.c:241:nrs_fifo_req_stop()) NRS stop fifo request from 12345-0@lo, seq: 516 00000100:00100000:2.0:1776178274.159394:0:3353:0:(client.c:2215:ptlrpc_check_set()) Completed RPC req@00000000d668bb9c pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_00:lustre-MDT0001-mdtlov_UUID:3353:1862456998059328:0@lo:13:osp-pre-0-1.0 00010000:00010000:0.0:1776178274.784246:0:6244:0:(ldlm_lockd.c:2792:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:0.0:1776178275.623891:0:3348:0:(events.c:361:request_in_callback()) peer: 12345-192.168.204.3@tcp (source: 12345-192.168.204.3@tcp) 00000100:00100000:0.0:1776178275.623949:0:3348:0:(events.c:361:request_in_callback()) peer: 12345-192.168.204.3@tcp (source: 12345-192.168.204.3@tcp) 00000100:00100000:0.0:1776178275.623979:0:3348:0:(events.c:361:request_in_callback()) peer: 12345-192.168.204.3@tcp (source: 12345-192.168.204.3@tcp) 00000100:00100000:0.0:1776178275.624014:0:3348:0:(events.c:361:request_in_callback()) peer: 12345-192.168.204.3@tcp (source: 12345-192.168.204.3@tcp) 00000100:00100000:0.0:1776178275.624098:0:20755:0:(service.c:2152:ptlrpc_server_handle_req_in()) got req x1862456979113728 00000100:00100000:0.0:1776178275.624150:0:20755:0:(service.c:2152:ptlrpc_server_handle_req_in()) got req x1862456979113664 00000100:00100000:0.0:1776178275.624169:0:20755:0:(nrs_fifo.c:179:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.204.3@tcp, seq: 567 00000100:00100000:0.0:1776178275.624175:0:20755:0:(service.c:2324:ptlrpc_server_handle_request()) Handling RPC req@000000007cbab86b pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_006:b50963bd-8eba-4432-9a54-8888d00e3fa1+9:2273:x1862456979113728:12345-192.168.204.3@tcp:400:kworker/u8:4.0 00000100:00100000:0.0:1776178275.624245:0:20755:0:(service.c:2372:ptlrpc_server_handle_request()) Handled RPC req@000000007cbab86b pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_006:b50963bd-8eba-4432-9a54-8888d00e3fa1+9:2273:x1862456979113728:12345-192.168.204.3@tcp:400:kworker/u8:4.0 Request processed in 70us (297us total) trans 0 rc 0/0 00000100:00100000:0.0:1776178275.624253:0:20755:0:(nrs_fifo.c:241:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.204.3@tcp, seq: 567 00000100:00100000:0.0:1776178275.624268:0:20755:0:(nrs_fifo.c:179:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.204.3@tcp, seq: 568 00000100:00100000:0.0:1776178275.624272:0:20755:0:(service.c:2324:ptlrpc_server_handle_request()) Handling RPC req@00000000e0d60cbc pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_006:b50963bd-8eba-4432-9a54-8888d00e3fa1+8:2273:x1862456979113664:12345-192.168.204.3@tcp:400:kworker/u8:4.0 00000100:00100000:0.0:1776178275.624517:0:20755:0:(service.c:2372:ptlrpc_server_handle_request()) Handled RPC req@00000000e0d60cbc pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_006:b50963bd-8eba-4432-9a54-8888d00e3fa1+8:2273:x1862456979113664:12345-192.168.204.3@tcp:400:kworker/u8:4.0 Request processed in 243us (502us total) trans 0 rc 0/0 00000100:00100000:0.0:1776178275.624523:0:20755:0:(nrs_fifo.c:241:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.204.3@tcp, seq: 568 00000100:00100000:1.0:1776178275.624911:0:20140:0:(service.c:2152:ptlrpc_server_handle_req_in()) got req x1862456979113472 00000100:00100000:1.0:1776178275.624962:0:20140:0:(nrs_fifo.c:179:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.204.3@tcp, seq: 183 00000100:00100000:1.0:1776178275.624966:0:20140:0:(service.c:2324:ptlrpc_server_handle_request()) Handling RPC req@00000000fb48a5c8 pname:cluuid+ref:pid:xid:nid:opc:job ll_mgs_0002:5b5c66b1-1a2f-4651-8548-0c75b9a38a94+9:2273:x1862456979113472:12345-192.168.204.3@tcp:400:kworker/u8:4.0 00000100:00100000:1.0:1776178275.624998:0:20140:0:(service.c:2372:ptlrpc_server_handle_request()) Handled RPC req@00000000fb48a5c8 pname:cluuid+ref:pid:xid:nid:opc:job ll_mgs_0002:5b5c66b1-1a2f-4651-8548-0c75b9a38a94+9:2273:x1862456979113472:12345-192.168.204.3@tcp:400:kworker/u8:4.0 Request processed in 30us (1018us total) trans 0 rc 0/0 00000100:00100000:1.0:1776178275.625004:0:20140:0:(nrs_fifo.c:241:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.204.3@tcp, seq: 183 00000100:00100000:0.0:1776178275.625051:0:3347:0:(events.c:361:request_in_callback()) peer: 12345-192.168.204.3@tcp (source: 12345-192.168.204.3@tcp) 00000100:00100000:1.0:1776178275.625297:0:6257:0:(service.c:2152:ptlrpc_server_handle_req_in()) got req x1862456979113600 00000100:00100000:1.0:1776178275.625344:0:6257:0:(service.c:2152:ptlrpc_server_handle_req_in()) got req x1862456979113536 00000100:00100000:1.0:1776178275.625373:0:6257:0:(nrs_fifo.c:179:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.204.3@tcp, seq: 970 00000100:00100000:3.0:1776178275.625554:0:6256:0:(nrs_fifo.c:179:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.204.3@tcp, seq: 6680 00000100:00100000:3.0:1776178275.625559:0:6256:0:(service.c:2324:ptlrpc_server_handle_request()) Handling RPC req@00000000490667a2 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_001:0+-99:2269:x1862456979113536:12345-192.168.204.3@tcp:38:kworker/u8:4.0 00010000:02000000:3.0:1776178275.625576:0:6256:0:(ldlm_lib.c:1139:target_handle_connect()) lustre-MDT0000: Not available for connect from 192.168.204.3@tcp (stopping) 00000100:00100000:3.0:1776178275.625618:0:6256:0:(service.c:2372:ptlrpc_server_handle_request()) Handled RPC req@00000000490667a2 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_001:0+-99:2269:x1862456979113536:12345-192.168.204.3@tcp:38:kworker/u8:4.0 Request processed in 58us (570us total) trans 0 rc -19/-19 00000100:00100000:3.0:1776178275.625626:0:6256:0:(nrs_fifo.c:241:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.204.3@tcp, seq: 6680 00000100:00100000:1.0:1776178275.625814:0:6257:0:(service.c:2324:ptlrpc_server_handle_request()) Handling RPC req@00000000d668bb9c pname:cluuid+ref:pid:xid:nid:opc:job mdt00_002:b50963bd-8eba-4432-9a54-8888d00e3fa1+6:2273:x1862456979113600:12345-192.168.204.3@tcp:400:kworker/u8:4.0 00000100:00100000:1.0:1776178275.625862:0:6257:0:(service.c:2372:ptlrpc_server_handle_request()) Handled RPC req@00000000d668bb9c pname:cluuid+ref:pid:xid:nid:opc:job mdt00_002:b50963bd-8eba-4432-9a54-8888d00e3fa1+6:2273:x1862456979113600:12345-192.168.204.3@tcp:400:kworker/u8:4.0 Request processed in 482us (1979us total) trans 0 rc 0/0 00000100:00100000:1.0:1776178275.625869:0:6257:0:(nrs_fifo.c:241:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.204.3@tcp, seq: 970 00010000:00010000:1.0:1776178275.808790:0:6245:0:(ldlm_lockd.c:2792:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00010000:00010000:2.0:1776178276.832354:0:9224:0:(ldlm_lockd.c:2792:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00080000:1.0:1776178277.219113:0:37:0:(pinger.c:251:ptlrpc_pinger_process_import()) lustre-MDT0000-lwp-OST0000_UUID->lustre-MDT0000_UUID: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00080000:1.0:1776178277.219122:0:37:0:(recover.c:57:ptlrpc_initiate_recovery()) lustre-MDT0000_UUID: starting recovery 00000100:00080000:1.0:1776178277.219125:0:37:0:(import.c:86:import_set_state_nolock()) 0000000047601a98 lustre-MDT0000_UUID: changing import state from DISCONN to CONNECTING 00000100:00080000:1.0:1776178277.219133:0:37:0:(import.c:534:import_select_connection()) lustre-MDT0000-lwp-OST0000: connect to NID 0@lo last attempt 979 00000100:00080000:1.0:1776178277.219137:0:37:0:(import.c:581:import_select_connection()) lustre-MDT0000-lwp-OST0000: tried all connections, increasing latency to 20s 00000100:00080000:1.0:1776178277.219142:0:37:0:(import.c:615:import_select_connection()) lustre-MDT0000-lwp-OST0000: import 0000000047601a98 using connection 192.168.204.103@tcp/0@lo 00000100:00100000:1.0:1776178277.219163:0:37:0:(import.c:807:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5) req@000000001a990b02 x1862456998059456/t0(0) o38->lustre-MDT0000-lwp-OST0000@0@lo:12/10 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' 00000100:00080000:1.0:1776178277.219195:0:37:0:(pinger.c:251:ptlrpc_pinger_process_import()) lustre-MDT0000-lwp-OST0001_UUID->lustre-MDT0000_UUID: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00080000:1.0:1776178277.219197:0:37:0:(recover.c:57:ptlrpc_initiate_recovery()) lustre-MDT0000_UUID: starting recovery 00000100:00080000:1.0:1776178277.219199:0:37:0:(import.c:86:import_set_state_nolock()) 00000000dfc8db13 lustre-MDT0000_UUID: changing import state from DISCONN to CONNECTING 00000100:00080000:1.0:1776178277.219202:0:37:0:(import.c:534:import_select_connection()) lustre-MDT0000-lwp-OST0001: connect to NID 0@lo last attempt 979 00000100:00080000:1.0:1776178277.219204:0:37:0:(import.c:581:import_select_connection()) lustre-MDT0000-lwp-OST0001: tried all connections, increasing latency to 20s 00000100:00080000:1.0:1776178277.219206:0:37:0:(import.c:615:import_select_connection()) lustre-MDT0000-lwp-OST0001: import 00000000dfc8db13 using connection 192.168.204.103@tcp/0@lo 00000100:00100000:1.0:1776178277.219214:0:37:0:(import.c:807:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5) req@00000000758405de x1862456998059584/t0(0) o38->lustre-MDT0000-lwp-OST0001@0@lo:12/10 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' 00000100:00080000:1.0:1776178277.219239:0:37:0:(pinger.c:251:ptlrpc_pinger_process_import()) lustre-MDT0001-mdtlov_UUID->lustre-MDT0000_UUID: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00080000:1.0:1776178277.219241:0:37:0:(recover.c:57:ptlrpc_initiate_recovery()) lustre-MDT0000_UUID: starting recovery 00000100:00080000:1.0:1776178277.219243:0:37:0:(import.c:86:import_set_state_nolock()) 00000000ba4f1821 lustre-MDT0000_UUID: changing import state from DISCONN to CONNECTING 00000100:00080000:1.0:1776178277.219246:0:37:0:(import.c:534:import_select_connection()) lustre-MDT0000-osp-MDT0001: connect to NID 0@lo last attempt 979 00000100:00080000:1.0:1776178277.219247:0:37:0:(import.c:581:import_select_connection()) lustre-MDT0000-osp-MDT0001: tried all connections, increasing latency to 20s 00000100:00080000:1.0:1776178277.219249:0:37:0:(import.c:615:import_select_connection()) lustre-MDT0000-osp-MDT0001: import 00000000ba4f1821 using connection 192.168.204.103@tcp/0@lo 00000100:00100000:1.0:1776178277.219257:0:37:0:(import.c:807:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5) req@00000000b30b9cdd x1862456998059712/t0(0) o38->lustre-MDT0000-osp-MDT0001@0@lo:24/4 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' 00000100:00080000:1.0:1776178277.219272:0:37:0:(pinger.c:251:ptlrpc_pinger_process_import()) lustre-MDT0000-lwp-MDT0001_UUID->lustre-MDT0000_UUID: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00080000:1.0:1776178277.219275:0:37:0:(recover.c:57:ptlrpc_initiate_recovery()) lustre-MDT0000_UUID: starting recovery 00000100:00080000:1.0:1776178277.219276:0:37:0:(import.c:86:import_set_state_nolock()) 00000000a7c55b1f lustre-MDT0000_UUID: changing import state from DISCONN to CONNECTING 00000100:00080000:1.0:1776178277.219278:0:37:0:(import.c:534:import_select_connection()) lustre-MDT0000-lwp-MDT0001: connect to NID 0@lo last attempt 979 00000100:00080000:1.0:1776178277.219280:0:37:0:(import.c:581:import_select_connection()) lustre-MDT0000-lwp-MDT0001: tried all connections, increasing latency to 20s 00000100:00080000:1.0:1776178277.219282:0:37:0:(import.c:615:import_select_connection()) lustre-MDT0000-lwp-MDT0001: import 00000000a7c55b1f using connection 192.168.204.103@tcp/0@lo 00000100:00100000:1.0:1776178277.219289:0:37:0:(import.c:807:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5) req@0000000027e376cf x1862456998059904/t0(0) o38->lustre-MDT0000-lwp-MDT0001@0@lo:12/10 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' 00000100:00100000:1.0:1776178277.219369:0:3354:0:(client.c:1733:ptlrpc_send_new_req()) Sending RPC req@0000000081d550e6 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_01:lustre-MDT0001-mdtlov_UUID:3354:1862456998059776:0@lo:400:kworker/u8:2.0 00000100:00100000:2.0:1776178277.219421:0:3352:0:(client.c:1733:ptlrpc_send_new_req()) Sending RPC req@000000001a990b02 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:lustre-MDT0000-lwp-OST0000_UUID:3352:1862456998059456:0@lo:38:kworker/u8:2.0 00000100:00100000:1.0:1776178277.219441:0:3354:0:(events.c:361:request_in_callback()) peer: 12345-0@lo (source: 12345-0@lo) 00000100:00100000:2.0:1776178277.219451:0:3352:0:(events.c:361:request_in_callback()) peer: 12345-0@lo (source: 12345-0@lo) 00000100:00100000:2.0:1776178277.219469:0:3352:0:(client.c:1733:ptlrpc_send_new_req()) Sending RPC req@00000000758405de pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:lustre-MDT0000-lwp-OST0001_UUID:3352:1862456998059584:0@lo:38:kworker/u8:2.0 00000100:00100000:1.0:1776178277.219476:0:3354:0:(ptlrpcd.c:415:ptlrpcd_check()) transfer 1 async RPCs [0->1] 00000100:00100000:1.0:1776178277.219481:0:3354:0:(client.c:1733:ptlrpc_send_new_req()) Sending RPC req@0000000016c7a5ff pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_01:lustre-MDT0001-lwp-OST0001_UUID:3354:1862456998059648:0@lo:400:kworker/u8:2.0 00000100:00100000:2.0:1776178277.219486:0:3352:0:(events.c:361:request_in_callback()) peer: 12345-0@lo (source: 12345-0@lo) 00000100:00100000:2.0:1776178277.219498:0:3352:0:(client.c:1733:ptlrpc_send_new_req()) Sending RPC req@00000000b30b9cdd pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:lustre-MDT0001-mdtlov_UUID:3352:1862456998059712:0@lo:38:kworker/u8:2.0 00000100:00100000:1.0:1776178277.219499:0:3354:0:(events.c:361:request_in_callback()) peer: 12345-0@lo (source: 12345-0@lo) 00000100:00100000:2.0:1776178277.219517:0:3352:0:(events.c:361:request_in_callback()) peer: 12345-0@lo (source: 12345-0@lo) 00000100:00100000:2.0:1776178277.219532:0:3352:0:(client.c:1733:ptlrpc_send_new_req()) Sending RPC req@0000000027e376cf pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:lustre-MDT0000-lwp-MDT0001_UUID:3352:1862456998059904:0@lo:38:kworker/u8:2.0 00000100:00100000:2.0:1776178277.219550:0:3352:0:(events.c:361:request_in_callback()) peer: 12345-0@lo (source: 12345-0@lo) 00000100:00100000:2.0:1776178277.219586:0:8250:0:(service.c:2152:ptlrpc_server_handle_req_in()) got req x1862456998059456 00000100:00100000:1.0:1776178277.219602:0:20759:0:(service.c:2152:ptlrpc_server_handle_req_in()) got req x1862456998059776 00000100:00100000:2.0:1776178277.219615:0:8250:0:(service.c:2152:ptlrpc_server_handle_req_in()) got req x1862456998059584 00000100:00100000:2.0:1776178277.219623:0:8250:0:(service.c:2152:ptlrpc_server_handle_req_in()) got req x1862456998059648 00000100:00100000:1.0:1776178277.219631:0:20759:0:(nrs_fifo.c:179:nrs_fifo_req_get()) NRS start fifo request from 12345-0@lo, seq: 569 00000100:00100000:2.0:1776178277.219636:0:8250:0:(service.c:2152:ptlrpc_server_handle_req_in()) got req x1862456998059904 00000100:00100000:1.0:1776178277.219638:0:20759:0:(service.c:2324:ptlrpc_server_handle_request()) Handling RPC req@00000000efb13deb pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_010:lustre-MDT0001-mdtlov_UUID+5:3354:x1862456998059776:12345-0@lo:400:kworker/u8:2.0 00000100:00100000:2.0:1776178277.219644:0:8250:0:(nrs_fifo.c:179:nrs_fifo_req_get()) NRS start fifo request from 12345-0@lo, seq: 971 00000100:00100000:2.0:1776178277.219647:0:8250:0:(service.c:2324:ptlrpc_server_handle_request()) Handling RPC req@00000000e3063d85 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_003:lustre-MDT0001-lwp-OST0001_UUID+5:3354:x1862456998059648:12345-0@lo:400:kworker/u8:2.0 00000100:00100000:0.0:1776178277.219731:0:3355:0:(client.c:1733:ptlrpc_send_new_req()) Sending RPC req@00000000d9d2b484 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_02:ac61f839-0e8e-4cac-9c59-08bcfbbad330:3355:1862456998059392:0@lo:400:kworker/u8:2.0 00000100:00100000:0.0:1776178277.219764:0:3355:0:(events.c:361:request_in_callback()) peer: 12345-0@lo (source: 12345-0@lo) 00000100:00100000:0.0:1776178277.219781:0:3355:0:(client.c:1733:ptlrpc_send_new_req()) Sending RPC req@00000000d4b41d38 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_02:lustre-MDT0001-mdtlov_UUID:3355:1862456998059840:0@lo:400:kworker/u8:2.0 00000100:00100000:0.0:1776178277.219799:0:3355:0:(events.c:361:request_in_callback()) peer: 12345-0@lo (source: 12345-0@lo) 00000100:00100000:0.0:1776178277.219814:0:3355:0:(ptlrpcd.c:415:ptlrpcd_check()) transfer 1 async RPCs [3->2] 00000100:00100000:0.0:1776178277.219830:0:3355:0:(client.c:1733:ptlrpc_send_new_req()) Sending RPC req@000000008a1e777f pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_02:lustre-MDT0001-lwp-OST0000_UUID:3355:1862456998059520:0@lo:400:kworker/u8:2.0 00000100:00100000:0.0:1776178277.219850:0:3355:0:(events.c:361:request_in_callback()) peer: 12345-0@lo (source: 12345-0@lo) 00000100:00100000:0.0:1776178277.219869:0:11281:0:(service.c:2152:ptlrpc_server_handle_req_in()) got req x1862456998059520 00000100:00100000:0.0:1776178277.219887:0:11281:0:(nrs_fifo.c:179:nrs_fifo_req_get()) NRS start fifo request from 12345-0@lo, seq: 972 00000100:00100000:0.0:1776178277.219891:0:11281:0:(service.c:2324:ptlrpc_server_handle_request()) Handling RPC req@000000009383c871 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_004:lustre-MDT0001-lwp-OST0000_UUID+5:3355:x1862456998059520:12345-0@lo:400:kworker/u8:2.0 00000100:00100000:0.0:1776178277.219920:0:11281:0:(service.c:2372:ptlrpc_server_handle_request()) Handled RPC req@000000009383c871 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_004:lustre-MDT0001-lwp-OST0000_UUID+5:3355:x1862456998059520:12345-0@lo:400:kworker/u8:2.0 Request processed in 29us (70us total) trans 0 rc 0/0 00000100:00100000:0.0:1776178277.219930:0:11281:0:(nrs_fifo.c:241:nrs_fifo_req_stop()) NRS stop fifo request from 12345-0@lo, seq: 972 00000100:00100000:0.0:1776178277.219946:0:11281:0:(nrs_fifo.c:179:nrs_fifo_req_get()) NRS start fifo request from 12345-0@lo, seq: 6681 00000100:00100000:0.0:1776178277.219949:0:11281:0:(service.c:2324:ptlrpc_server_handle_request()) Handling RPC req@00000000b13c16eb pname:cluuid+ref:pid:xid:nid:opc:job mdt00_004:0+-99:3352:x1862456998059456:12345-0@lo:38:kworker/u8:2.0 00010000:02000000:0.0:1776178277.219961:0:11281:0:(ldlm_lib.c:1139:target_handle_connect()) lustre-MDT0000: Not available for connect from 0@lo (stopping) 00000100:00100000:0.0:1776178277.219989:0:11281:0:(service.c:2372:ptlrpc_server_handle_request()) Handled RPC req@00000000b13c16eb pname:cluuid+ref:pid:xid:nid:opc:job mdt00_004:0+-99:3352:x1862456998059456:12345-0@lo:38:kworker/u8:2.0 Request processed in 39us (537us total) trans 0 rc -19/-19 00000100:00100000:0.0:1776178277.219995:0:11281:0:(nrs_fifo.c:241:nrs_fifo_req_stop()) NRS stop fifo request from 12345-0@lo, seq: 6681 00000100:00100000:0.0:1776178277.220006:0:11281:0:(nrs_fifo.c:179:nrs_fifo_req_get()) NRS start fifo request from 12345-0@lo, seq: 6682 00000100:00100000:0.0:1776178277.220009:0:11281:0:(service.c:2324:ptlrpc_server_handle_request()) Handling RPC req@00000000bcfff430 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_004:0+-99:3352:x1862456998059584:12345-0@lo:38:kworker/u8:2.0 00010000:02000000:0.0:1776178277.220014:0:11281:0:(ldlm_lib.c:1139:target_handle_connect()) lustre-MDT0000: Not available for connect from 0@lo (stopping) 00000100:00100000:0.0:1776178277.220032:0:11281:0:(service.c:2372:ptlrpc_server_handle_request()) Handled RPC req@00000000bcfff430 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_004:0+-99:3352:x1862456998059584:12345-0@lo:38:kworker/u8:2.0 Request processed in 22us (546us total) trans 0 rc -19/-19 00000100:00100000:0.0:1776178277.220037:0:11281:0:(nrs_fifo.c:241:nrs_fifo_req_stop()) NRS stop fifo request from 12345-0@lo, seq: 6682 00000100:00100000:0.0:1776178277.220046:0:11281:0:(nrs_fifo.c:179:nrs_fifo_req_get()) NRS start fifo request from 12345-0@lo, seq: 6683 00000100:00100000:0.0:1776178277.220048:0:11281:0:(service.c:2324:ptlrpc_server_handle_request()) Handling RPC req@0000000020afd0b3 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_004:0+-99:3352:x1862456998059904:12345-0@lo:38:kworker/u8:2.0 00010000:02000000:0.0:1776178277.220053:0:11281:0:(ldlm_lib.c:1139:target_handle_connect()) lustre-MDT0000: Not available for connect from 0@lo (stopping) 00000100:00100000:0.0:1776178277.220094:0:11281:0:(service.c:2372:ptlrpc_server_handle_request()) Handled RPC req@0000000020afd0b3 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_004:0+-99:3352:x1862456998059904:12345-0@lo:38:kworker/u8:2.0 Request processed in 44us (543us total) trans 0 rc -19/-19 00000100:00100000:0.0:1776178277.220099:0:11281:0:(nrs_fifo.c:241:nrs_fifo_req_stop()) NRS stop fifo request from 12345-0@lo, seq: 6683 00000100:00100000:0.0:1776178277.220118:0:20137:0:(service.c:2152:ptlrpc_server_handle_req_in()) got req x1862456998059392 00000100:00100000:0.0:1776178277.220142:0:20137:0:(nrs_fifo.c:179:nrs_fifo_req_get()) NRS start fifo request from 12345-0@lo, seq: 184 00000100:00100000:0.0:1776178277.220145:0:20137:0:(service.c:2324:ptlrpc_server_handle_request()) Handling RPC req@0000000060db901b pname:cluuid+ref:pid:xid:nid:opc:job ll_mgs_0000:ac61f839-0e8e-4cac-9c59-08bcfbbad330+18:3355:x1862456998059392:12345-0@lo:400:kworker/u8:2.0 00000100:00100000:0.0:1776178277.220174:0:20137:0:(service.c:2372:ptlrpc_server_handle_request()) Handled RPC req@0000000060db901b pname:cluuid+ref:pid:xid:nid:opc:job ll_mgs_0000:ac61f839-0e8e-4cac-9c59-08bcfbbad330+18:3355:x1862456998059392:12345-0@lo:400:kworker/u8:2.0 Request processed in 27us (409us total) trans 0 rc 0/0 00000100:00100000:0.0:1776178277.220179:0:20137:0:(nrs_fifo.c:241:nrs_fifo_req_stop()) NRS stop fifo request from 12345-0@lo, seq: 184 00000100:00100000:0.0:1776178277.220209:0:14440:0:(service.c:2152:ptlrpc_server_handle_req_in()) got req x1862456998059840 00000100:00100000:0.0:1776178277.220230:0:14440:0:(nrs_fifo.c:179:nrs_fifo_req_get()) NRS start fifo request from 12345-0@lo, seq: 570 00000100:00100000:0.0:1776178277.220234:0:14440:0:(service.c:2324:ptlrpc_server_handle_request()) Handling RPC req@000000002a321d27 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_004:lustre-MDT0001-mdtlov_UUID+5:3355:x1862456998059840:12345-0@lo:400:kworker/u8:2.0 00000100:00100000:0.0:1776178277.220259:0:14440:0:(service.c:2372:ptlrpc_server_handle_request()) Handled RPC req@000000002a321d27 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_004:lustre-MDT0001-mdtlov_UUID+5:3355:x1862456998059840:12345-0@lo:400:kworker/u8:2.0 Request processed in 25us (460us total) trans 0 rc 0/0 00000100:00100000:0.0:1776178277.220264:0:14440:0:(nrs_fifo.c:241:nrs_fifo_req_stop()) NRS stop fifo request from 12345-0@lo, seq: 570 00000100:00100000:0.0:1776178277.220301:0:3355:0:(client.c:2215:ptlrpc_check_set()) Completed RPC req@000000008a1e777f pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_02:lustre-MDT0001-lwp-OST0000_UUID:3355:1862456998059520:0@lo:400:kworker/u8:2.0 00000100:00100000:0.0:1776178277.220323:0:3355:0:(client.c:2215:ptlrpc_check_set()) Completed RPC req@00000000d9d2b484 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_02:ac61f839-0e8e-4cac-9c59-08bcfbbad330:3355:1862456998059392:0@lo:400:kworker/u8:2.0 00000100:00100000:0.0:1776178277.220330:0:3355:0:(client.c:2215:ptlrpc_check_set()) Completed RPC req@00000000d4b41d38 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_02:lustre-MDT0001-mdtlov_UUID:3355:1862456998059840:0@lo:400:kworker/u8:2.0 00000100:00080000:0.0:1776178277.220364:0:3352:0:(import.c:86:import_set_state_nolock()) 0000000047601a98 lustre-MDT0000_UUID: changing import state from CONNECTING to DISCONN 00000100:00080000:0.0:1776178277.220368:0:3352:0:(import.c:1429:ptlrpc_connect_interpret()) recovery of lustre-MDT0000_UUID on 192.168.204.103@tcp failed (-19) 00000100:00100000:0.0:1776178277.220371:0:3352:0:(client.c:2215:ptlrpc_check_set()) Completed RPC req@000000001a990b02 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:lustre-MDT0000-lwp-OST0000_UUID:3352:1862456998059456:0@lo:38:kworker/u8:2.0 00000100:00080000:0.0:1776178277.220378:0:3352:0:(import.c:86:import_set_state_nolock()) 00000000dfc8db13 lustre-MDT0000_UUID: changing import state from CONNECTING to DISCONN 00000100:00080000:0.0:1776178277.220380:0:3352:0:(import.c:1429:ptlrpc_connect_interpret()) recovery of lustre-MDT0000_UUID on 192.168.204.103@tcp failed (-19) 00000100:00100000:0.0:1776178277.220382:0:3352:0:(client.c:2215:ptlrpc_check_set()) Completed RPC req@00000000758405de pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:lustre-MDT0000-lwp-OST0001_UUID:3352:1862456998059584:0@lo:38:kworker/u8:2.0 00000100:00080000:0.0:1776178277.220389:0:3352:0:(import.c:86:import_set_state_nolock()) 00000000a7c55b1f lustre-MDT0000_UUID: changing import state from CONNECTING to DISCONN 00000100:00080000:0.0:1776178277.220391:0:3352:0:(import.c:1429:ptlrpc_connect_interpret()) recovery of lustre-MDT0000_UUID on 192.168.204.103@tcp failed (-19) 00000100:00100000:0.0:1776178277.220393:0:3352:0:(client.c:2215:ptlrpc_check_set()) Completed RPC req@0000000027e376cf pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:lustre-MDT0000-lwp-MDT0001_UUID:3352:1862456998059904:0@lo:38:kworker/u8:2.0 00000100:00100000:0.0:1776178277.220442:0:14128:0:(service.c:2152:ptlrpc_server_handle_req_in()) got req x1862456998059712 00000100:00100000:0.0:1776178277.220461:0:14128:0:(nrs_fifo.c:179:nrs_fifo_req_get()) NRS start fifo request from 12345-0@lo, seq: 722 00000100:00100000:0.0:1776178277.220464:0:14128:0:(service.c:2324:ptlrpc_server_handle_request()) Handling RPC req@0000000023562990 pname:cluuid+ref:pid:xid:nid:opc:job mdt_out00_002:0+-99:3352:x1862456998059712:12345-0@lo:38:kworker/u8:2.0 00010000:02000000:0.0:1776178277.220472:0:14128:0:(ldlm_lib.c:1139:target_handle_connect()) lustre-MDT0000: Not available for connect from 0@lo (stopping) 00000100:00100000:0.0:1776178277.220498:0:14128:0:(service.c:2372:ptlrpc_server_handle_request()) Handled RPC req@0000000023562990 pname:cluuid+ref:pid:xid:nid:opc:job mdt_out00_002:0+-99:3352:x1862456998059712:12345-0@lo:38:kworker/u8:2.0 Request processed in 33us (981us total) trans 0 rc -19/-19 00000100:00100000:0.0:1776178277.220504:0:14128:0:(nrs_fifo.c:241:nrs_fifo_req_stop()) NRS stop fifo request from 12345-0@lo, seq: 722 00000100:00080000:0.0:1776178277.220525:0:3352:0:(import.c:86:import_set_state_nolock()) 00000000ba4f1821 lustre-MDT0000_UUID: changing import state from CONNECTING to DISCONN 00000100:00080000:0.0:1776178277.220528:0:3352:0:(import.c:1429:ptlrpc_connect_interpret()) recovery of lustre-MDT0000_UUID on 192.168.204.103@tcp failed (-19) 00000100:00100000:0.0:1776178277.220531:0:3352:0:(client.c:2215:ptlrpc_check_set()) Completed RPC req@00000000b30b9cdd pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:lustre-MDT0001-mdtlov_UUID:3352:1862456998059712:0@lo:38:kworker/u8:2.0 00000100:00100000:1.0:1776178277.220752:0:20759:0:(service.c:2372:ptlrpc_server_handle_request()) Handled RPC req@00000000efb13deb pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_010:lustre-MDT0001-mdtlov_UUID+5:3354:x1862456998059776:12345-0@lo:400:kworker/u8:2.0 Request processed in 46us (244us total) trans 0 rc 0/0 00000100:00100000:1.0:1776178277.220760:0:20759:0:(nrs_fifo.c:241:nrs_fifo_req_stop()) NRS stop fifo request from 12345-0@lo, seq: 569 00000100:00100000:2.0:1776178277.225762:0:8250:0:(service.c:2372:ptlrpc_server_handle_request()) Handled RPC req@00000000e3063d85 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_003:lustre-MDT0001-lwp-OST0001_UUID+5:3354:x1862456998059648:12345-0@lo:400:kworker/u8:2.0 Request processed in 6104us (6253us total) trans 0 rc 0/0 00000100:00100000:2.0:1776178277.225778:0:8250:0:(nrs_fifo.c:241:nrs_fifo_req_stop()) NRS stop fifo request from 12345-0@lo, seq: 971 00000100:00100000:0.0:1776178277.237433:0:3354:0:(client.c:2215:ptlrpc_check_set()) Completed RPC req@0000000016c7a5ff pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_01:lustre-MDT0001-lwp-OST0001_UUID:3354:1862456998059648:0@lo:400:kworker/u8:2.0 00000100:00100000:0.0:1776178277.237456:0:3354:0:(client.c:2215:ptlrpc_check_set()) Completed RPC req@0000000081d550e6 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_01:lustre-MDT0001-mdtlov_UUID:3354:1862456998059776:0@lo:400:kworker/u8:2.0 00010000:00010000:2.0:1776178277.856451:0:9225:0:(ldlm_lockd.c:2792:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:1.0:1776178278.752450:0:3355:0:(ptlrpcd.c:415:ptlrpcd_check()) transfer 1 async RPCs [3->2] 00000100:00100000:1.0:1776178278.752475:0:3355:0:(client.c:1733:ptlrpc_send_new_req()) Sending RPC req@0000000039ff8335 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_02:lustre-MDT0001-mdtlov_UUID:3355:1862456998059968:0@lo:13:osp-pre-1-1.0 00000100:00100000:1.0:1776178278.752551:0:3355:0:(events.c:361:request_in_callback()) peer: 12345-0@lo (source: 12345-0@lo) 00000100:00100000:0.0:1776178278.752821:0:8231:0:(service.c:2152:ptlrpc_server_handle_req_in()) got req x1862456998059968 00000100:00100000:0.0:1776178278.752885:0:8231:0:(nrs_fifo.c:179:nrs_fifo_req_get()) NRS start fifo request from 12345-0@lo, seq: 517 00000100:00100000:0.0:1776178278.752893:0:8231:0:(service.c:2324:ptlrpc_server_handle_request()) Handling RPC req@0000000013603266 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_create00:lustre-MDT0001-mdtlov_UUID+5:3355:x1862456998059968:12345-0@lo:13:osp-pre-1-1.0 00002000:00000024:0.0:1776178278.752934:0:8231:0:(ofd_obd.c:740:ofd_statfs()) blocks cached 229376 granted 491746368 pending 0 free 3923689472 avail 3692167168 00000100:00100000:0.0:1776178278.753316:0:8231:0:(service.c:2372:ptlrpc_server_handle_request()) Handled RPC req@0000000013603266 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_create00:lustre-MDT0001-mdtlov_UUID+5:3355:x1862456998059968:12345-0@lo:13:osp-pre-1-1.0 Request processed in 422us (764us total) trans 0 rc 0/0 00000100:00100000:0.0:1776178278.753324:0:8231:0:(nrs_fifo.c:241:nrs_fifo_req_stop()) NRS stop fifo request from 12345-0@lo, seq: 517 00000100:00100000:1.0:1776178278.754495:0:3355:0:(client.c:2215:ptlrpc_check_set()) Completed RPC req@0000000039ff8335 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_02:lustre-MDT0001-mdtlov_UUID:3355:1862456998059968:0@lo:13:osp-pre-1-1.0 00010000:00010000:0.0:1776178278.880304:0:6244:0:(ldlm_lockd.c:2792:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:2.0:1776178279.265015:0:3353:0:(client.c:1733:ptlrpc_send_new_req()) Sending RPC req@000000006be5e5b9 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_00:lustre-MDT0001-mdtlov_UUID:3353:1862456998060032:0@lo:13:osp-pre-0-1.0 00000100:00100000:2.0:1776178279.265193:0:3353:0:(events.c:361:request_in_callback()) peer: 12345-0@lo (source: 12345-0@lo) 00000100:00100000:3.0:1776178279.266923:0:8231:0:(service.c:2152:ptlrpc_server_handle_req_in()) got req x1862456998060032 00000100:00100000:3.0:1776178279.266994:0:8231:0:(nrs_fifo.c:179:nrs_fifo_req_get()) NRS start fifo request from 12345-0@lo, seq: 518 00000100:00100000:3.0:1776178279.267001:0:8231:0:(service.c:2324:ptlrpc_server_handle_request()) Handling RPC req@00000000167dc455 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_create00:lustre-MDT0001-mdtlov_UUID+5:3353:x1862456998060032:12345-0@lo:13:osp-pre-0-1.0 00002000:00000024:3.0:1776178279.267038:0:8231:0:(ofd_obd.c:740:ofd_statfs()) blocks cached 315392 granted 484883776 pending 0 free 3923689472 avail 3692167168 00000100:00100000:3.0:1776178279.267155:0:8231:0:(service.c:2372:ptlrpc_server_handle_request()) Handled RPC req@00000000167dc455 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_create00:lustre-MDT0001-mdtlov_UUID+5:3353:x1862456998060032:12345-0@lo:13:osp-pre-0-1.0 Request processed in 152us (1965us total) trans 0 rc 0/0 00000100:00100000:3.0:1776178279.267164:0:8231:0:(nrs_fifo.c:241:nrs_fifo_req_stop()) NRS stop fifo request from 12345-0@lo, seq: 518 00000100:00100000:2.0:1776178279.267336:0:3353:0:(client.c:2215:ptlrpc_check_set()) Completed RPC req@000000006be5e5b9 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_00:lustre-MDT0001-mdtlov_UUID:3353:1862456998060032:0@lo:13:osp-pre-0-1.0 00010000:00010000:1.0:1776178279.905543:0:6245:0:(ldlm_lockd.c:2792:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:0.0:1776178280.740802:0:3348:0:(events.c:361:request_in_callback()) peer: 12345-192.168.204.3@tcp (source: 12345-192.168.204.3@tcp) 00000100:00100000:0.0:1776178280.740852:0:3348:0:(events.c:361:request_in_callback()) peer: 12345-192.168.204.3@tcp (source: 12345-192.168.204.3@tcp) 00000100:00100000:0.0:1776178280.740901:0:3348:0:(events.c:361:request_in_callback()) peer: 12345-192.168.204.3@tcp (source: 12345-192.168.204.3@tcp) 00000100:00100000:3.0:1776178280.740924:0:20759:0:(service.c:2152:ptlrpc_server_handle_req_in()) got req x1862456979114048 00000100:00100000:0.0:1776178280.740929:0:3348:0:(events.c:361:request_in_callback()) peer: 12345-192.168.204.3@tcp (source: 12345-192.168.204.3@tcp) 00000100:00100000:3.0:1776178280.740977:0:20759:0:(nrs_fifo.c:179:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.204.3@tcp, seq: 571 00000100:00100000:3.0:1776178280.740983:0:20759:0:(service.c:2324:ptlrpc_server_handle_request()) Handling RPC req@0000000027e376cf pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_010:b50963bd-8eba-4432-9a54-8888d00e3fa1+9:2270:x1862456979114048:12345-192.168.204.3@tcp:400:kworker/u8:3.0 00000100:00100000:0.0:1776178280.740996:0:3348:0:(events.c:361:request_in_callback()) peer: 12345-192.168.204.3@tcp (source: 12345-192.168.204.3@tcp) 00000100:00100000:0.0:1776178280.741018:0:6255:0:(service.c:2152:ptlrpc_server_handle_req_in()) got req x1862456979113920 00000100:00100000:3.0:1776178280.741048:0:20759:0:(service.c:2372:ptlrpc_server_handle_request()) Handled RPC req@0000000027e376cf pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_010:b50963bd-8eba-4432-9a54-8888d00e3fa1+9:2270:x1862456979114048:12345-192.168.204.3@tcp:400:kworker/u8:3.0 Request processed in 65us (197us total) trans 0 rc 0/0 00000100:00100000:0.0:1776178280.741055:0:6255:0:(service.c:2152:ptlrpc_server_handle_req_in()) got req x1862456979113856 00000100:00100000:3.0:1776178280.741056:0:20759:0:(nrs_fifo.c:241:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.204.3@tcp, seq: 571 00000100:00100000:3.0:1776178280.741093:0:20759:0:(service.c:2152:ptlrpc_server_handle_req_in()) got req x1862456979113984 00000100:00100000:0.0:1776178280.741101:0:6255:0:(nrs_fifo.c:179:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.204.3@tcp, seq: 973 00000100:00100000:0.0:1776178280.741105:0:6255:0:(service.c:2324:ptlrpc_server_handle_request()) Handling RPC req@000000006be5e5b9 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_000:b50963bd-8eba-4432-9a54-8888d00e3fa1+6:2270:x1862456979113920:12345-192.168.204.3@tcp:400:kworker/u8:3.0 00000100:00100000:3.0:1776178280.741112:0:20759:0:(nrs_fifo.c:179:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.204.3@tcp, seq: 572 00000100:00100000:3.0:1776178280.741115:0:20759:0:(service.c:2324:ptlrpc_server_handle_request()) Handling RPC req@00000000d4b41d38 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_010:b50963bd-8eba-4432-9a54-8888d00e3fa1+8:2270:x1862456979113984:12345-192.168.204.3@tcp:400:kworker/u8:3.0 00000100:00100000:0.0:1776178280.741134:0:6255:0:(service.c:2372:ptlrpc_server_handle_request()) Handled RPC req@000000006be5e5b9 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_000:b50963bd-8eba-4432-9a54-8888d00e3fa1+6:2270:x1862456979113920:12345-192.168.204.3@tcp:400:kworker/u8:3.0 Request processed in 27us (341us total) trans 0 rc 0/0 00000100:00100000:3.0:1776178280.741134:0:20759:0:(service.c:2372:ptlrpc_server_handle_request()) Handled RPC req@00000000d4b41d38 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_010:b50963bd-8eba-4432-9a54-8888d00e3fa1+8:2270:x1862456979113984:12345-192.168.204.3@tcp:400:kworker/u8:3.0 Request processed in 18us (138us total) trans 0 rc 0/0 00000100:00100000:3.0:1776178280.741140:0:20759:0:(nrs_fifo.c:241:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.204.3@tcp, seq: 572 00000100:00100000:0.0:1776178280.741141:0:6255:0:(nrs_fifo.c:241:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.204.3@tcp, seq: 973 00000100:00100000:0.0:1776178280.741151:0:6255:0:(nrs_fifo.c:179:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.204.3@tcp, seq: 6684 00000100:00100000:0.0:1776178280.741155:0:6255:0:(service.c:2324:ptlrpc_server_handle_request()) Handling RPC req@0000000081d550e6 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_000:0+-99:2269:x1862456979113856:12345-192.168.204.3@tcp:38:kworker/u8:3.0 00010000:02000000:0.0:1776178280.741169:0:6255:0:(ldlm_lib.c:1139:target_handle_connect()) lustre-MDT0000: Not available for connect from 192.168.204.3@tcp (stopping) 00000100:00100000:0.0:1776178280.741193:0:6255:0:(service.c:2372:ptlrpc_server_handle_request()) Handled RPC req@0000000081d550e6 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_000:0+-99:2269:x1862456979113856:12345-192.168.204.3@tcp:38:kworker/u8:3.0 Request processed in 38us (263us total) trans 0 rc -19/-19 00000100:00100000:0.0:1776178280.741200:0:6255:0:(nrs_fifo.c:241:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.204.3@tcp, seq: 6684 00000100:00100000:1.0:1776178280.741250:0:20140:0:(service.c:2152:ptlrpc_server_handle_req_in()) got req x1862456979113792 00000100:00100000:1.0:1776178280.741290:0:20140:0:(nrs_fifo.c:179:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.204.3@tcp, seq: 185 00000100:00100000:1.0:1776178280.741294:0:20140:0:(service.c:2324:ptlrpc_server_handle_request()) Handling RPC req@000000003e5a23bc pname:cluuid+ref:pid:xid:nid:opc:job ll_mgs_0002:5b5c66b1-1a2f-4651-8548-0c75b9a38a94+9:2270:x1862456979113792:12345-192.168.204.3@tcp:400:kworker/u8:3.0 00000100:00100000:1.0:1776178280.741321:0:20140:0:(service.c:2372:ptlrpc_server_handle_request()) Handled RPC req@000000003e5a23bc pname:cluuid+ref:pid:xid:nid:opc:job ll_mgs_0002:5b5c66b1-1a2f-4651-8548-0c75b9a38a94+9:2270:x1862456979113792:12345-192.168.204.3@tcp:400:kworker/u8:3.0 Request processed in 25us (420us total) trans 0 rc 0/0 00000100:00100000:1.0:1776178280.741326:0:20140:0:(nrs_fifo.c:241:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.204.3@tcp, seq: 185 00010000:00010000:2.0:1776178280.928845:0:9224:0:(ldlm_lockd.c:2792:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00010000:00010000:2.0:1776178281.952338:0:9225:0:(ldlm_lockd.c:2792:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00080000:2.0:1776178282.336531:0:36:0:(pinger.c:251:ptlrpc_pinger_process_import()) lustre-MDT0000-lwp-OST0000_UUID->lustre-MDT0000_UUID: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00080000:2.0:1776178282.336541:0:36:0:(recover.c:57:ptlrpc_initiate_recovery()) lustre-MDT0000_UUID: starting recovery 00000100:00080000:2.0:1776178282.336546:0:36:0:(import.c:86:import_set_state_nolock()) 0000000047601a98 lustre-MDT0000_UUID: changing import state from DISCONN to CONNECTING 00000100:00080000:2.0:1776178282.336555:0:36:0:(import.c:534:import_select_connection()) lustre-MDT0000-lwp-OST0000: connect to NID 0@lo last attempt 984 00000100:00080000:2.0:1776178282.336558:0:36:0:(import.c:581:import_select_connection()) lustre-MDT0000-lwp-OST0000: tried all connections, increasing latency to 20s 00000100:00080000:2.0:1776178282.336565:0:36:0:(import.c:615:import_select_connection()) lustre-MDT0000-lwp-OST0000: import 0000000047601a98 using connection 192.168.204.103@tcp/0@lo 00000100:00100000:1.0:1776178282.336719:0:3354:0:(client.c:1733:ptlrpc_send_new_req()) Sending RPC req@0000000038c09127 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_01:ac61f839-0e8e-4cac-9c59-08bcfbbad330:3354:1862456998060096:0@lo:400:kworker/u8:1.0 00000100:00100000:1.0:1776178282.336799:0:3354:0:(events.c:361:request_in_callback()) peer: 12345-0@lo (source: 12345-0@lo) 00000100:00100000:0.0:1776178282.337103:0:20137:0:(service.c:2152:ptlrpc_server_handle_req_in()) got req x1862456998060096 00000100:00100000:0.0:1776178282.337151:0:20137:0:(nrs_fifo.c:179:nrs_fifo_req_get()) NRS start fifo request from 12345-0@lo, seq: 186 00000100:00100000:0.0:1776178282.337157:0:20137:0:(service.c:2324:ptlrpc_server_handle_request()) Handling RPC req@00000000fc8eef33 pname:cluuid+ref:pid:xid:nid:opc:job ll_mgs_0000:ac61f839-0e8e-4cac-9c59-08bcfbbad330+18:3354:x1862456998060096:12345-0@lo:400:kworker/u8:1.0 00000100:00100000:0.0:1776178282.337229:0:20137:0:(service.c:2372:ptlrpc_server_handle_request()) Handled RPC req@00000000fc8eef33 pname:cluuid+ref:pid:xid:nid:opc:job ll_mgs_0000:ac61f839-0e8e-4cac-9c59-08bcfbbad330+18:3354:x1862456998060096:12345-0@lo:400:kworker/u8:1.0 Request processed in 71us (431us total) trans 0 rc 0/0 00000100:00100000:0.0:1776178282.337236:0:20137:0:(nrs_fifo.c:241:nrs_fifo_req_stop()) NRS stop fifo request from 12345-0@lo, seq: 186 00000100:00100000:1.0:1776178282.337428:0:3354:0:(client.c:2215:ptlrpc_check_set()) Completed RPC req@0000000038c09127 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_01:ac61f839-0e8e-4cac-9c59-08bcfbbad330:3354:1862456998060096:0@lo:400:kworker/u8:1.0 00000100:00100000:2.0:1776178282.337612:0:36:0:(import.c:807:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5) req@00000000d6e5d9ba x1862456998060160/t0(0) o38->lustre-MDT0000-lwp-OST0000@0@lo:12/10 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' 00000100:00080000:2.0:1776178282.337662:0:36:0:(pinger.c:251:ptlrpc_pinger_process_import()) lustre-MDT0000-lwp-OST0001_UUID->lustre-MDT0000_UUID: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00080000:2.0:1776178282.337666:0:36:0:(recover.c:57:ptlrpc_initiate_recovery()) lustre-MDT0000_UUID: starting recovery 00000100:00080000:2.0:1776178282.337668:0:36:0:(import.c:86:import_set_state_nolock()) 00000000dfc8db13 lustre-MDT0000_UUID: changing import state from DISCONN to CONNECTING 00000100:00080000:2.0:1776178282.337671:0:36:0:(import.c:534:import_select_connection()) lustre-MDT0000-lwp-OST0001: connect to NID 0@lo last attempt 984 00000100:00080000:2.0:1776178282.337673:0:36:0:(import.c:581:import_select_connection()) lustre-MDT0000-lwp-OST0001: tried all connections, increasing latency to 20s 00000100:00080000:2.0:1776178282.337677:0:36:0:(import.c:615:import_select_connection()) lustre-MDT0000-lwp-OST0001: import 00000000dfc8db13 using connection 192.168.204.103@tcp/0@lo 00000100:00100000:2.0:1776178282.337690:0:36:0:(import.c:807:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5) req@00000000bcefda18 x1862456998060288/t0(0) o38->lustre-MDT0000-lwp-OST0001@0@lo:12/10 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' 00000100:00100000:3.0:1776178282.337879:0:3356:0:(ptlrpcd.c:415:ptlrpcd_check()) transfer 1 async RPCs [2->3] 00000100:00100000:3.0:1776178282.337891:0:3356:0:(client.c:1733:ptlrpc_send_new_req()) Sending RPC req@0000000021b0d516 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_03:lustre-MDT0001-lwp-OST0000_UUID:3356:1862456998060224:0@lo:400:kworker/u8:1.0 00000100:00100000:3.0:1776178282.337928:0:3356:0:(events.c:361:request_in_callback()) peer: 12345-0@lo (source: 12345-0@lo) 00000100:00100000:3.0:1776178282.337977:0:8250:0:(service.c:2152:ptlrpc_server_handle_req_in()) got req x1862456998060224 00000100:00100000:3.0:1776178282.338014:0:8250:0:(nrs_fifo.c:179:nrs_fifo_req_get()) NRS start fifo request from 12345-0@lo, seq: 974 00000100:00100000:3.0:1776178282.338018:0:8250:0:(service.c:2324:ptlrpc_server_handle_request()) Handling RPC req@000000001982f085 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_003:lustre-MDT0001-lwp-OST0000_UUID+5:3356:x1862456998060224:12345-0@lo:400:kworker/u8:1.0 00000100:00100000:3.0:1776178282.338052:0:8250:0:(service.c:2372:ptlrpc_server_handle_request()) Handled RPC req@000000001982f085 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_003:lustre-MDT0001-lwp-OST0000_UUID+5:3356:x1862456998060224:12345-0@lo:400:kworker/u8:1.0 Request processed in 34us (125us total) trans 0 rc 0/0 00000100:00100000:3.0:1776178282.338058:0:8250:0:(nrs_fifo.c:241:nrs_fifo_req_stop()) NRS stop fifo request from 12345-0@lo, seq: 974 00000100:00100000:3.0:1776178282.338115:0:3356:0:(client.c:2215:ptlrpc_check_set()) Completed RPC req@0000000021b0d516 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_03:lustre-MDT0001-lwp-OST0000_UUID:3356:1862456998060224:0@lo:400:kworker/u8:1.0 00000100:00100000:0.0:1776178282.338345:0:3352:0:(client.c:1733:ptlrpc_send_new_req()) Sending RPC req@00000000d6e5d9ba pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:lustre-MDT0000-lwp-OST0000_UUID:3352:1862456998060160:0@lo:38:kworker/u8:1.0 00000100:00100000:0.0:1776178282.338377:0:3352:0:(events.c:361:request_in_callback()) peer: 12345-0@lo (source: 12345-0@lo) 00000100:00080000:2.0:1776178282.338491:0:36:0:(pinger.c:251:ptlrpc_pinger_process_import()) lustre-MDT0001-mdtlov_UUID->lustre-MDT0000_UUID: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00100000:1.0:1776178282.338612:0:3356:0:(client.c:1733:ptlrpc_send_new_req()) Sending RPC req@0000000021b0d516 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_03:lustre-MDT0001-lwp-OST0001_UUID:3356:1862456998060352:0@lo:400:kworker/u8:1.0 00000100:00100000:1.0:1776178282.338642:0:3356:0:(events.c:361:request_in_callback()) peer: 12345-0@lo (source: 12345-0@lo) 00000100:00100000:0.0:1776178282.338880:0:3352:0:(client.c:1733:ptlrpc_send_new_req()) Sending RPC req@00000000bcefda18 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:lustre-MDT0000-lwp-OST0001_UUID:3352:1862456998060288:0@lo:38:kworker/u8:1.0 00000100:00100000:0.0:1776178282.338908:0:3352:0:(events.c:361:request_in_callback()) peer: 12345-0@lo (source: 12345-0@lo) 00000100:00080000:2.0:1776178282.339240:0:36:0:(recover.c:57:ptlrpc_initiate_recovery()) lustre-MDT0000_UUID: starting recovery 00000100:00080000:2.0:1776178282.339243:0:36:0:(import.c:86:import_set_state_nolock()) 00000000ba4f1821 lustre-MDT0000_UUID: changing import state from DISCONN to CONNECTING 00000100:00080000:2.0:1776178282.339248:0:36:0:(import.c:534:import_select_connection()) lustre-MDT0000-osp-MDT0001: connect to NID 0@lo last attempt 984 00000100:00080000:2.0:1776178282.339250:0:36:0:(import.c:581:import_select_connection()) lustre-MDT0000-osp-MDT0001: tried all connections, increasing latency to 20s 00000100:00080000:2.0:1776178282.339255:0:36:0:(import.c:615:import_select_connection()) lustre-MDT0000-osp-MDT0001: import 00000000ba4f1821 using connection 192.168.204.103@tcp/0@lo 00000100:00100000:2.0:1776178282.339275:0:36:0:(import.c:807:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5) req@0000000038c09127 x1862456998060416/t0(0) o38->lustre-MDT0000-osp-MDT0001@0@lo:24/4 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' 00000100:00080000:2.0:1776178282.339316:0:36:0:(pinger.c:251:ptlrpc_pinger_process_import()) lustre-MDT0000-lwp-MDT0001_UUID->lustre-MDT0000_UUID: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00080000:2.0:1776178282.339320:0:36:0:(recover.c:57:ptlrpc_initiate_recovery()) lustre-MDT0000_UUID: starting recovery 00000100:00080000:2.0:1776178282.339322:0:36:0:(import.c:86:import_set_state_nolock()) 00000000a7c55b1f lustre-MDT0000_UUID: changing import state from DISCONN to CONNECTING 00000100:00080000:2.0:1776178282.339325:0:36:0:(import.c:534:import_select_connection()) lustre-MDT0000-lwp-MDT0001: connect to NID 0@lo last attempt 984 00000100:00080000:2.0:1776178282.339327:0:36:0:(import.c:581:import_select_connection()) lustre-MDT0000-lwp-MDT0001: tried all connections, increasing latency to 20s 00000100:00080000:2.0:1776178282.339330:0:36:0:(import.c:615:import_select_connection()) lustre-MDT0000-lwp-MDT0001: import 00000000a7c55b1f using connection 192.168.204.103@tcp/0@lo 00000100:00100000:2.0:1776178282.339339:0:36:0:(import.c:807:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5) req@0000000037059d8f x1862456998060608/t0(0) o38->lustre-MDT0000-lwp-MDT0001@0@lo:12/10 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' 00000100:00100000:1.0:1776178282.339413:0:6257:0:(service.c:2152:ptlrpc_server_handle_req_in()) got req x1862456998060160 00000100:00100000:1.0:1776178282.339451:0:6257:0:(service.c:2152:ptlrpc_server_handle_req_in()) got req x1862456998060352 00000100:00100000:1.0:1776178282.339472:0:6257:0:(service.c:2152:ptlrpc_server_handle_req_in()) got req x1862456998060288 00000100:00100000:1.0:1776178282.339483:0:6257:0:(nrs_fifo.c:179:nrs_fifo_req_get()) NRS start fifo request from 12345-0@lo, seq: 975 00000100:00100000:1.0:1776178282.339487:0:6257:0:(service.c:2324:ptlrpc_server_handle_request()) Handling RPC req@0000000070c16828 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_002:lustre-MDT0001-lwp-OST0001_UUID+5:3356:x1862456998060352:12345-0@lo:400:kworker/u8:1.0 00000100:00100000:1.0:1776178282.339547:0:6257:0:(service.c:2372:ptlrpc_server_handle_request()) Handled RPC req@0000000070c16828 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_002:lustre-MDT0001-lwp-OST0001_UUID+5:3356:x1862456998060352:12345-0@lo:400:kworker/u8:1.0 Request processed in 59us (905us total) trans 0 rc 0/0 00000100:00100000:1.0:1776178282.339554:0:6257:0:(nrs_fifo.c:241:nrs_fifo_req_stop()) NRS stop fifo request from 12345-0@lo, seq: 975 00000100:00100000:1.0:1776178282.339571:0:6257:0:(nrs_fifo.c:179:nrs_fifo_req_get()) NRS start fifo request from 12345-0@lo, seq: 6685 00000100:00100000:1.0:1776178282.339574:0:6257:0:(service.c:2324:ptlrpc_server_handle_request()) Handling RPC req@0000000016c7a5ff pname:cluuid+ref:pid:xid:nid:opc:job mdt00_002:0+-99:3352:x1862456998060160:12345-0@lo:38:kworker/u8:1.0 00010000:02000000:1.0:1776178282.339597:0:6257:0:(ldlm_lib.c:1139:target_handle_connect()) lustre-MDT0000: Not available for connect from 0@lo (stopping) 00000100:00100000:1.0:1776178282.339627:0:6257:0:(service.c:2372:ptlrpc_server_handle_request()) Handled RPC req@0000000016c7a5ff pname:cluuid+ref:pid:xid:nid:opc:job mdt00_002:0+-99:3352:x1862456998060160:12345-0@lo:38:kworker/u8:1.0 Request processed in 52us (1250us total) trans 0 rc -19/-19 00000100:00100000:1.0:1776178282.339633:0:6257:0:(nrs_fifo.c:241:nrs_fifo_req_stop()) NRS stop fifo request from 12345-0@lo, seq: 6685 00000100:00100000:1.0:1776178282.339644:0:6257:0:(nrs_fifo.c:179:nrs_fifo_req_get()) NRS start fifo request from 12345-0@lo, seq: 6686 00000100:00100000:1.0:1776178282.339647:0:6257:0:(service.c:2324:ptlrpc_server_handle_request()) Handling RPC req@00000000b30b9cdd pname:cluuid+ref:pid:xid:nid:opc:job mdt00_002:0+-99:3352:x1862456998060288:12345-0@lo:38:kworker/u8:1.0 00010000:02000000:1.0:1776178282.339652:0:6257:0:(ldlm_lib.c:1139:target_handle_connect()) lustre-MDT0000: Not available for connect from 0@lo (stopping) 00000100:00100000:1.0:1776178282.339669:0:6257:0:(service.c:2372:ptlrpc_server_handle_request()) Handled RPC req@00000000b30b9cdd pname:cluuid+ref:pid:xid:nid:opc:job mdt00_002:0+-99:3352:x1862456998060288:12345-0@lo:38:kworker/u8:1.0 Request processed in 21us (762us total) trans 0 rc -19/-19 00000100:00100000:1.0:1776178282.339673:0:6257:0:(nrs_fifo.c:241:nrs_fifo_req_stop()) NRS stop fifo request from 12345-0@lo, seq: 6686 00000100:00100000:1.0:1776178282.339690:0:3354:0:(client.c:1733:ptlrpc_send_new_req()) Sending RPC req@0000000052a7c870 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_01:lustre-MDT0001-mdtlov_UUID:3354:1862456998060544:0@lo:400:kworker/u8:1.0 00000100:00100000:1.0:1776178282.339986:0:3354:0:(events.c:361:request_in_callback()) peer: 12345-0@lo (source: 12345-0@lo) 00000100:00100000:1.0:1776178282.340005:0:3354:0:(ptlrpcd.c:415:ptlrpcd_check()) transfer 1 async RPCs [0->1] 00000100:00100000:1.0:1776178282.340010:0:3354:0:(client.c:1733:ptlrpc_send_new_req()) Sending RPC req@00000000b63b7b84 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_01:lustre-MDT0001-mdtlov_UUID:3354:1862456998060480:0@lo:400:kworker/u8:1.0 00000100:00100000:1.0:1776178282.340033:0:3354:0:(events.c:361:request_in_callback()) peer: 12345-0@lo (source: 12345-0@lo) 00000100:00100000:1.0:1776178282.340098:0:3356:0:(client.c:2215:ptlrpc_check_set()) Completed RPC req@0000000021b0d516 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_03:lustre-MDT0001-lwp-OST0001_UUID:3356:1862456998060352:0@lo:400:kworker/u8:1.0 00000100:00100000:0.0:1776178282.348283:0:3352:0:(client.c:1733:ptlrpc_send_new_req()) Sending RPC req@0000000038c09127 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:lustre-MDT0001-mdtlov_UUID:3352:1862456998060416:0@lo:38:kworker/u8:1.0 00000100:00100000:0.0:1776178282.348383:0:3352:0:(events.c:361:request_in_callback()) peer: 12345-0@lo (source: 12345-0@lo) 00000100:00100000:0.0:1776178282.348437:0:3352:0:(client.c:1733:ptlrpc_send_new_req()) Sending RPC req@0000000037059d8f pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:lustre-MDT0000-lwp-MDT0001_UUID:3352:1862456998060608:0@lo:38:kworker/u8:1.0 00000100:00100000:0.0:1776178282.348464:0:3352:0:(events.c:361:request_in_callback()) peer: 12345-0@lo (source: 12345-0@lo) 00000100:00080000:0.0:1776178282.348514:0:3352:0:(import.c:86:import_set_state_nolock()) 00000000dfc8db13 lustre-MDT0000_UUID: changing import state from CONNECTING to DISCONN 00000100:00080000:0.0:1776178282.348519:0:3352:0:(import.c:1429:ptlrpc_connect_interpret()) recovery of lustre-MDT0000_UUID on 192.168.204.103@tcp failed (-19) 00000100:00100000:0.0:1776178282.348524:0:3352:0:(client.c:2215:ptlrpc_check_set()) Completed RPC req@00000000bcefda18 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:lustre-MDT0000-lwp-OST0001_UUID:3352:1862456998060288:0@lo:38:kworker/u8:1.0 00000100:00080000:0.0:1776178282.348536:0:3352:0:(import.c:86:import_set_state_nolock()) 0000000047601a98 lustre-MDT0000_UUID: changing import state from CONNECTING to DISCONN 00000100:00080000:0.0:1776178282.348539:0:3352:0:(import.c:1429:ptlrpc_connect_interpret()) recovery of lustre-MDT0000_UUID on 192.168.204.103@tcp failed (-19) 00000100:00100000:0.0:1776178282.348542:0:3352:0:(client.c:2215:ptlrpc_check_set()) Completed RPC req@00000000d6e5d9ba pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:lustre-MDT0000-lwp-OST0000_UUID:3352:1862456998060160:0@lo:38:kworker/u8:1.0 00000100:00100000:0.0:1776178282.348591:0:11281:0:(service.c:2152:ptlrpc_server_handle_req_in()) got req x1862456998060608 00000100:00100000:0.0:1776178282.348625:0:11281:0:(nrs_fifo.c:179:nrs_fifo_req_get()) NRS start fifo request from 12345-0@lo, seq: 6687 00000100:00100000:0.0:1776178282.348632:0:11281:0:(service.c:2324:ptlrpc_server_handle_request()) Handling RPC req@000000001a990b02 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_004:0+-99:3352:x1862456998060608:12345-0@lo:38:kworker/u8:1.0 00010000:02000000:0.0:1776178282.348646:0:11281:0:(ldlm_lib.c:1139:target_handle_connect()) lustre-MDT0000: Not available for connect from 0@lo (stopping) 00000100:00100000:0.0:1776178282.348685:0:11281:0:(service.c:2372:ptlrpc_server_handle_request()) Handled RPC req@000000001a990b02 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_004:0+-99:3352:x1862456998060608:12345-0@lo:38:kworker/u8:1.0 Request processed in 52us (220us total) trans 0 rc -19/-19 00000100:00100000:0.0:1776178282.348691:0:11281:0:(nrs_fifo.c:241:nrs_fifo_req_stop()) NRS stop fifo request from 12345-0@lo, seq: 6687 00000100:00080000:0.0:1776178282.348725:0:3352:0:(import.c:86:import_set_state_nolock()) 00000000a7c55b1f lustre-MDT0000_UUID: changing import state from CONNECTING to DISCONN 00000100:00080000:0.0:1776178282.348728:0:3352:0:(import.c:1429:ptlrpc_connect_interpret()) recovery of lustre-MDT0000_UUID on 192.168.204.103@tcp failed (-19) 00000100:00100000:0.0:1776178282.348731:0:3352:0:(client.c:2215:ptlrpc_check_set()) Completed RPC req@0000000037059d8f pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:lustre-MDT0000-lwp-MDT0001_UUID:3352:1862456998060608:0@lo:38:kworker/u8:1.0 00000100:00100000:3.0:1776178282.349344:0:14440:0:(service.c:2152:ptlrpc_server_handle_req_in()) got req x1862456998060544 00000100:00100000:3.0:1776178282.349389:0:14440:0:(service.c:2152:ptlrpc_server_handle_req_in()) got req x1862456998060480 00000100:00100000:3.0:1776178282.349411:0:14440:0:(nrs_fifo.c:179:nrs_fifo_req_get()) NRS start fifo request from 12345-0@lo, seq: 573 00000100:00100000:3.0:1776178282.349415:0:14440:0:(service.c:2324:ptlrpc_server_handle_request()) Handling RPC req@000000007cfde41a pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_004:lustre-MDT0001-mdtlov_UUID+5:3354:x1862456998060544:12345-0@lo:400:kworker/u8:1.0 00000100:00100000:3.0:1776178282.349470:0:14440:0:(service.c:2372:ptlrpc_server_handle_request()) Handled RPC req@000000007cfde41a pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_004:lustre-MDT0001-mdtlov_UUID+5:3354:x1862456998060544:12345-0@lo:400:kworker/u8:1.0 Request processed in 54us (9485us total) trans 0 rc 0/0 00000100:00100000:3.0:1776178282.349477:0:14440:0:(nrs_fifo.c:241:nrs_fifo_req_stop()) NRS stop fifo request from 12345-0@lo, seq: 573 00000100:00100000:3.0:1776178282.349490:0:14440:0:(nrs_fifo.c:179:nrs_fifo_req_get()) NRS start fifo request from 12345-0@lo, seq: 574 00000100:00100000:3.0:1776178282.349505:0:14440:0:(service.c:2324:ptlrpc_server_handle_request()) Handling RPC req@000000007e1b5a24 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_004:lustre-MDT0001-mdtlov_UUID+5:3354:x1862456998060480:12345-0@lo:400:kworker/u8:1.0 00000100:00100000:3.0:1776178282.349532:0:14440:0:(service.c:2372:ptlrpc_server_handle_request()) Handled RPC req@000000007e1b5a24 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_004:lustre-MDT0001-mdtlov_UUID+5:3354:x1862456998060480:12345-0@lo:400:kworker/u8:1.0 Request processed in 27us (9500us total) trans 0 rc 0/0 00000100:00100000:3.0:1776178282.349538:0:14440:0:(nrs_fifo.c:241:nrs_fifo_req_stop()) NRS stop fifo request from 12345-0@lo, seq: 574 00000100:00100000:1.0:1776178282.352518:0:15213:0:(service.c:2152:ptlrpc_server_handle_req_in()) got req x1862456998060416 00000100:00100000:1.0:1776178282.352584:0:15213:0:(nrs_fifo.c:179:nrs_fifo_req_get()) NRS start fifo request from 12345-0@lo, seq: 723 00000100:00100000:1.0:1776178282.352589:0:15213:0:(service.c:2324:ptlrpc_server_handle_request()) Handling RPC req@00000000758405de pname:cluuid+ref:pid:xid:nid:opc:job mdt_out00_003:0+-99:3352:x1862456998060416:12345-0@lo:38:kworker/u8:1.0 00010000:02000000:1.0:1776178282.352606:0:15213:0:(ldlm_lib.c:1139:target_handle_connect()) lustre-MDT0000: Not available for connect from 0@lo (stopping) 00000100:00100000:1.0:1776178282.352693:0:15213:0:(service.c:2372:ptlrpc_server_handle_request()) Handled RPC req@00000000758405de pname:cluuid+ref:pid:xid:nid:opc:job mdt_out00_003:0+-99:3352:x1862456998060416:12345-0@lo:38:kworker/u8:1.0 Request processed in 102us (4312us total) trans 0 rc -19/-19 00000100:00100000:1.0:1776178282.352699:0:15213:0:(nrs_fifo.c:241:nrs_fifo_req_stop()) NRS stop fifo request from 12345-0@lo, seq: 723 00000100:00100000:1.0:1776178282.352758:0:3354:0:(client.c:2215:ptlrpc_check_set()) Completed RPC req@00000000b63b7b84 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_01:lustre-MDT0001-mdtlov_UUID:3354:1862456998060480:0@lo:400:kworker/u8:1.0 00000100:00100000:1.0:1776178282.352770:0:3354:0:(client.c:2215:ptlrpc_check_set()) Completed RPC req@0000000052a7c870 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_01:lustre-MDT0001-mdtlov_UUID:3354:1862456998060544:0@lo:400:kworker/u8:1.0 00000100:00080000:3.0:1776178282.355669:0:3352:0:(import.c:86:import_set_state_nolock()) 00000000ba4f1821 lustre-MDT0000_UUID: changing import state from CONNECTING to DISCONN 00000100:00080000:3.0:1776178282.355681:0:3352:0:(import.c:1429:ptlrpc_connect_interpret()) recovery of lustre-MDT0000_UUID on 192.168.204.103@tcp failed (-19) 00000100:00100000:3.0:1776178282.355690:0:3352:0:(client.c:2215:ptlrpc_check_set()) Completed RPC req@0000000038c09127 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:lustre-MDT0001-mdtlov_UUID:3352:1862456998060416:0@lo:38:kworker/u8:1.0 00010000:00010000:0.0:1776178282.976331:0:6244:0:(ldlm_lockd.c:2792:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:2.0:1776178283.872739:0:3356:0:(ptlrpcd.c:415:ptlrpcd_check()) transfer 1 async RPCs [2->3] 00000100:00100000:2.0:1776178283.872766:0:3356:0:(client.c:1733:ptlrpc_send_new_req()) Sending RPC req@0000000058cd86cb pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_03:lustre-MDT0001-mdtlov_UUID:3356:1862456998060672:0@lo:13:osp-pre-1-1.0 00000100:00100000:2.0:1776178283.872842:0:3356:0:(events.c:361:request_in_callback()) peer: 12345-0@lo (source: 12345-0@lo) 00000100:00100000:3.0:1776178283.872978:0:8231:0:(service.c:2152:ptlrpc_server_handle_req_in()) got req x1862456998060672 00000100:00100000:3.0:1776178283.873032:0:8231:0:(nrs_fifo.c:179:nrs_fifo_req_get()) NRS start fifo request from 12345-0@lo, seq: 519 00000100:00100000:3.0:1776178283.873038:0:8231:0:(service.c:2324:ptlrpc_server_handle_request()) Handling RPC req@00000000bd795500 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_create00:lustre-MDT0001-mdtlov_UUID+5:3356:x1862456998060672:12345-0@lo:13:osp-pre-1-1.0 00002000:00000024:3.0:1776178283.873094:0:8231:0:(ofd_obd.c:740:ofd_statfs()) blocks cached 229376 granted 491746368 pending 0 free 3923689472 avail 3692167168 00000100:00100000:3.0:1776178283.873149:0:8231:0:(service.c:2372:ptlrpc_server_handle_request()) Handled RPC req@00000000bd795500 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_create00:lustre-MDT0001-mdtlov_UUID+5:3356:x1862456998060672:12345-0@lo:13:osp-pre-1-1.0 Request processed in 109us (307us total) trans 0 rc 0/0 00000100:00100000:3.0:1776178283.873156:0:8231:0:(nrs_fifo.c:241:nrs_fifo_req_stop()) NRS stop fifo request from 12345-0@lo, seq: 519 00000100:00100000:2.0:1776178283.873404:0:3356:0:(client.c:2215:ptlrpc_check_set()) Completed RPC req@0000000058cd86cb pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_03:lustre-MDT0001-mdtlov_UUID:3356:1862456998060672:0@lo:13:osp-pre-1-1.0 00010000:00010000:1.0:1776178284.010193:0:6245:0:(ldlm_lockd.c:2792:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:1.0:1776178284.387633:0:3355:0:(ptlrpcd.c:415:ptlrpcd_check()) transfer 1 async RPCs [3->2] 00000100:00100000:1.0:1776178284.387660:0:3355:0:(client.c:1733:ptlrpc_send_new_req()) Sending RPC req@00000000d9d2b484 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_02:lustre-MDT0001-mdtlov_UUID:3355:1862456998060736:0@lo:13:osp-pre-0-1.0 00000100:00100000:1.0:1776178284.387739:0:3355:0:(events.c:361:request_in_callback()) peer: 12345-0@lo (source: 12345-0@lo) 00000100:00100000:0.0:1776178284.387935:0:9554:0:(service.c:2152:ptlrpc_server_handle_req_in()) got req x1862456998060736 00000100:00100000:0.0:1776178284.388007:0:9554:0:(nrs_fifo.c:179:nrs_fifo_req_get()) NRS start fifo request from 12345-0@lo, seq: 520 00000100:00100000:0.0:1776178284.388014:0:9554:0:(service.c:2324:ptlrpc_server_handle_request()) Handling RPC req@00000000d94d52a5 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_create00:lustre-MDT0001-mdtlov_UUID+5:3355:x1862456998060736:12345-0@lo:13:osp-pre-0-1.0 00002000:00000024:0.0:1776178284.388043:0:9554:0:(ofd_obd.c:740:ofd_statfs()) blocks cached 315392 granted 484883776 pending 0 free 3923689472 avail 3692167168 00000100:00100000:0.0:1776178284.388115:0:9554:0:(service.c:2372:ptlrpc_server_handle_request()) Handled RPC req@00000000d94d52a5 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_create00:lustre-MDT0001-mdtlov_UUID+5:3355:x1862456998060736:12345-0@lo:13:osp-pre-0-1.0 Request processed in 100us (375us total) trans 0 rc 0/0 00000100:00100000:0.0:1776178284.388127:0:9554:0:(nrs_fifo.c:241:nrs_fifo_req_stop()) NRS stop fifo request from 12345-0@lo, seq: 520 00000100:00100000:2.0:1776178284.388288:0:3355:0:(client.c:2215:ptlrpc_check_set()) Completed RPC req@00000000d9d2b484 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_02:lustre-MDT0001-mdtlov_UUID:3355:1862456998060736:0@lo:13:osp-pre-0-1.0 00010000:00010000:2.0:1776178285.025329:0:9224:0:(ldlm_lockd.c:2792:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:3.0:1776178285.858519:0:3347:0:(events.c:361:request_in_callback()) peer: 12345-192.168.204.3@tcp (source: 12345-192.168.204.3@tcp) 00000100:00100000:3.0:1776178285.858578:0:3347:0:(events.c:361:request_in_callback()) peer: 12345-192.168.204.3@tcp (source: 12345-192.168.204.3@tcp) 00000100:00100000:2.0:1776178285.858625:0:6257:0:(service.c:2152:ptlrpc_server_handle_req_in()) got req x1862456979114176 00000100:00100000:2.0:1776178285.858688:0:6257:0:(nrs_fifo.c:179:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.204.3@tcp, seq: 6688 00000100:00100000:2.0:1776178285.858700:0:6257:0:(service.c:2324:ptlrpc_server_handle_request()) Handling RPC req@0000000026195e8c pname:cluuid+ref:pid:xid:nid:opc:job mdt00_002:0+-99:2269:x1862456979114176:12345-192.168.204.3@tcp:38:kworker/u8:4.0 00010000:02000000:2.0:1776178285.858726:0:6257:0:(ldlm_lib.c:1139:target_handle_connect()) lustre-MDT0000: Not available for connect from 192.168.204.3@tcp (stopping) 00000100:00100000:2.0:1776178285.858813:0:6257:0:(service.c:2372:ptlrpc_server_handle_request()) Handled RPC req@0000000026195e8c pname:cluuid+ref:pid:xid:nid:opc:job mdt00_002:0+-99:2269:x1862456979114176:12345-192.168.204.3@tcp:38:kworker/u8:4.0 Request processed in 114us (307us total) trans 0 rc -19/-19 00000100:00100000:2.0:1776178285.858820:0:6257:0:(nrs_fifo.c:241:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.204.3@tcp, seq: 6688 00000100:00100000:3.0:1776178285.859554:0:3347:0:(events.c:361:request_in_callback()) peer: 12345-192.168.204.3@tcp (source: 12345-192.168.204.3@tcp) 00000100:00100000:3.0:1776178285.859598:0:3347:0:(events.c:361:request_in_callback()) peer: 12345-192.168.204.3@tcp (source: 12345-192.168.204.3@tcp) 00000100:00100000:3.0:1776178285.859638:0:3347:0:(events.c:361:request_in_callback()) peer: 12345-192.168.204.3@tcp (source: 12345-192.168.204.3@tcp) 00000100:00100000:3.0:1776178285.859665:0:20759:0:(service.c:2152:ptlrpc_server_handle_req_in()) got req x1862456979114368 00000100:00100000:3.0:1776178285.859701:0:20759:0:(service.c:2152:ptlrpc_server_handle_req_in()) got req x1862456979114304 00000100:00100000:0.0:1776178285.859711:0:6255:0:(service.c:2152:ptlrpc_server_handle_req_in()) got req x1862456979114240 00000100:00100000:3.0:1776178285.859724:0:20759:0:(nrs_fifo.c:179:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.204.3@tcp, seq: 575 00000100:00100000:3.0:1776178285.859729:0:20759:0:(service.c:2324:ptlrpc_server_handle_request()) Handling RPC req@0000000039ff8335 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_010:b50963bd-8eba-4432-9a54-8888d00e3fa1+9:2270:x1862456979114368:12345-192.168.204.3@tcp:400:kworker/u8:4.0 00000100:00100000:0.0:1776178285.859751:0:6255:0:(nrs_fifo.c:179:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.204.3@tcp, seq: 976 00000100:00100000:0.0:1776178285.859758:0:6255:0:(service.c:2324:ptlrpc_server_handle_request()) Handling RPC req@0000000058cd86cb pname:cluuid+ref:pid:xid:nid:opc:job mdt00_000:b50963bd-8eba-4432-9a54-8888d00e3fa1+6:2270:x1862456979114240:12345-192.168.204.3@tcp:400:kworker/u8:4.0 00000100:00100000:3.0:1776178285.859782:0:20759:0:(service.c:2372:ptlrpc_server_handle_request()) Handled RPC req@0000000039ff8335 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_010:b50963bd-8eba-4432-9a54-8888d00e3fa1+9:2270:x1862456979114368:12345-192.168.204.3@tcp:400:kworker/u8:4.0 Request processed in 52us (230us total) trans 0 rc 0/0 00000100:00100000:3.0:1776178285.859789:0:20759:0:(nrs_fifo.c:241:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.204.3@tcp, seq: 575 00000100:00100000:0.0:1776178285.859801:0:6255:0:(service.c:2372:ptlrpc_server_handle_request()) Handled RPC req@0000000058cd86cb pname:cluuid+ref:pid:xid:nid:opc:job mdt00_000:b50963bd-8eba-4432-9a54-8888d00e3fa1+6:2270:x1862456979114240:12345-192.168.204.3@tcp:400:kworker/u8:4.0 Request processed in 41us (1222us total) trans 0 rc 0/0 00000100:00100000:3.0:1776178285.859808:0:20759:0:(nrs_fifo.c:179:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.204.3@tcp, seq: 576 00000100:00100000:0.0:1776178285.859811:0:6255:0:(nrs_fifo.c:241:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.204.3@tcp, seq: 976 00000100:00100000:3.0:1776178285.859815:0:20759:0:(service.c:2324:ptlrpc_server_handle_request()) Handling RPC req@000000007af1bdf7 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_010:b50963bd-8eba-4432-9a54-8888d00e3fa1+8:2270:x1862456979114304:12345-192.168.204.3@tcp:400:kworker/u8:4.0 00000100:00100000:3.0:1776178285.859845:0:20759:0:(service.c:2372:ptlrpc_server_handle_request()) Handled RPC req@000000007af1bdf7 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_010:b50963bd-8eba-4432-9a54-8888d00e3fa1+8:2270:x1862456979114304:12345-192.168.204.3@tcp:400:kworker/u8:4.0 Request processed in 30us (207us total) trans 0 rc 0/0 00000100:00100000:3.0:1776178285.859852:0:20759:0:(nrs_fifo.c:241:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.204.3@tcp, seq: 576 00000100:00100000:2.0:1776178285.859973:0:20140:0:(service.c:2152:ptlrpc_server_handle_req_in()) got req x1862456979114112 00000100:00100000:2.0:1776178285.860007:0:20140:0:(nrs_fifo.c:179:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.204.3@tcp, seq: 187 00000100:00100000:2.0:1776178285.860011:0:20140:0:(service.c:2324:ptlrpc_server_handle_request()) Handling RPC req@0000000044c7032a pname:cluuid+ref:pid:xid:nid:opc:job ll_mgs_0002:5b5c66b1-1a2f-4651-8548-0c75b9a38a94+9:2270:x1862456979114112:12345-192.168.204.3@tcp:400:kworker/u8:4.0 00000100:00100000:2.0:1776178285.860041:0:20140:0:(service.c:2372:ptlrpc_server_handle_request()) Handled RPC req@0000000044c7032a pname:cluuid+ref:pid:xid:nid:opc:job ll_mgs_0002:5b5c66b1-1a2f-4651-8548-0c75b9a38a94+9:2270:x1862456979114112:12345-192.168.204.3@tcp:400:kworker/u8:4.0 Request processed in 29us (442us total) trans 0 rc 0/0 00000100:00100000:2.0:1776178285.860048:0:20140:0:(nrs_fifo.c:241:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.204.3@tcp, seq: 187 00010000:00010000:2.0:1776178286.048323:0:9225:0:(ldlm_lockd.c:2792:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00010000:00010000:0.0:1776178287.072307:0:6244:0:(ldlm_lockd.c:2792:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00080000:0.0:1776178287.457930:0:37:0:(pinger.c:251:ptlrpc_pinger_process_import()) lustre-MDT0000-lwp-OST0000_UUID->lustre-MDT0000_UUID: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00080000:0.0:1776178287.457939:0:37:0:(recover.c:57:ptlrpc_initiate_recovery()) lustre-MDT0000_UUID: starting recovery 00000100:00080000:0.0:1776178287.457942:0:37:0:(import.c:86:import_set_state_nolock()) 0000000047601a98 lustre-MDT0000_UUID: changing import state from DISCONN to CONNECTING 00000100:00080000:0.0:1776178287.457950:0:37:0:(import.c:534:import_select_connection()) lustre-MDT0000-lwp-OST0000: connect to NID 0@lo last attempt 989 00000100:00080000:0.0:1776178287.457953:0:37:0:(import.c:581:import_select_connection()) lustre-MDT0000-lwp-OST0000: tried all connections, increasing latency to 21s 00000100:00080000:0.0:1776178287.457959:0:37:0:(import.c:615:import_select_connection()) lustre-MDT0000-lwp-OST0000: import 0000000047601a98 using connection 192.168.204.103@tcp/0@lo 00000100:00100000:0.0:1776178287.457984:0:37:0:(import.c:807:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5) req@00000000d9d2b484 x1862456998060864/t0(0) o38->lustre-MDT0000-lwp-OST0000@0@lo:12/10 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' 00000100:00080000:0.0:1776178287.458022:0:37:0:(pinger.c:251:ptlrpc_pinger_process_import()) lustre-MDT0000-lwp-OST0001_UUID->lustre-MDT0000_UUID: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00080000:0.0:1776178287.458025:0:37:0:(recover.c:57:ptlrpc_initiate_recovery()) lustre-MDT0000_UUID: starting recovery 00000100:00080000:0.0:1776178287.458027:0:37:0:(import.c:86:import_set_state_nolock()) 00000000dfc8db13 lustre-MDT0000_UUID: changing import state from DISCONN to CONNECTING 00000100:00080000:0.0:1776178287.458030:0:37:0:(import.c:534:import_select_connection()) lustre-MDT0000-lwp-OST0001: connect to NID 0@lo last attempt 989 00000100:00080000:0.0:1776178287.458031:0:37:0:(import.c:581:import_select_connection()) lustre-MDT0000-lwp-OST0001: tried all connections, increasing latency to 21s 00000100:00080000:0.0:1776178287.458034:0:37:0:(import.c:615:import_select_connection()) lustre-MDT0000-lwp-OST0001: import 00000000dfc8db13 using connection 192.168.204.103@tcp/0@lo 00000100:00100000:0.0:1776178287.458044:0:37:0:(import.c:807:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5) req@000000009e03ef2c x1862456998060992/t0(0) o38->lustre-MDT0000-lwp-OST0001@0@lo:12/10 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' 00000100:00080000:0.0:1776178287.458093:0:37:0:(pinger.c:251:ptlrpc_pinger_process_import()) lustre-MDT0001-mdtlov_UUID->lustre-MDT0000_UUID: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00080000:0.0:1776178287.458096:0:37:0:(recover.c:57:ptlrpc_initiate_recovery()) lustre-MDT0000_UUID: starting recovery 00000100:00080000:0.0:1776178287.458098:0:37:0:(import.c:86:import_set_state_nolock()) 00000000ba4f1821 lustre-MDT0000_UUID: changing import state from DISCONN to CONNECTING 00000100:00080000:0.0:1776178287.458101:0:37:0:(import.c:534:import_select_connection()) lustre-MDT0000-osp-MDT0001: connect to NID 0@lo last attempt 989 00000100:00080000:0.0:1776178287.458103:0:37:0:(import.c:581:import_select_connection()) lustre-MDT0000-osp-MDT0001: tried all connections, increasing latency to 21s 00000100:00080000:0.0:1776178287.458106:0:37:0:(import.c:615:import_select_connection()) lustre-MDT0000-osp-MDT0001: import 00000000ba4f1821 using connection 192.168.204.103@tcp/0@lo 00000100:00100000:0.0:1776178287.458128:0:37:0:(import.c:807:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5) req@00000000ed0a8ae7 x1862456998061120/t0(0) o38->lustre-MDT0000-osp-MDT0001@0@lo:24/4 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' 00000100:00080000:0.0:1776178287.458145:0:37:0:(pinger.c:251:ptlrpc_pinger_process_import()) lustre-MDT0000-lwp-MDT0001_UUID->lustre-MDT0000_UUID: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00080000:0.0:1776178287.458148:0:37:0:(recover.c:57:ptlrpc_initiate_recovery()) lustre-MDT0000_UUID: starting recovery 00000100:00080000:0.0:1776178287.458150:0:37:0:(import.c:86:import_set_state_nolock()) 00000000a7c55b1f lustre-MDT0000_UUID: changing import state from DISCONN to CONNECTING 00000100:00080000:0.0:1776178287.458153:0:37:0:(import.c:534:import_select_connection()) lustre-MDT0000-lwp-MDT0001: connect to NID 0@lo last attempt 989 00000100:00080000:0.0:1776178287.458154:0:37:0:(import.c:581:import_select_connection()) lustre-MDT0000-lwp-MDT0001: tried all connections, increasing latency to 21s 00000100:00080000:0.0:1776178287.458157:0:37:0:(import.c:615:import_select_connection()) lustre-MDT0000-lwp-MDT0001: import 00000000a7c55b1f using connection 192.168.204.103@tcp/0@lo 00000100:00100000:0.0:1776178287.458165:0:37:0:(import.c:807:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5) req@0000000004c8f27c x1862456998061312/t0(0) o38->lustre-MDT0000-lwp-MDT0001@0@lo:12/10 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' 00000100:00100000:0.0:1776178287.458197:0:3355:0:(client.c:1733:ptlrpc_send_new_req()) Sending RPC req@0000000056aa3aa8 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_02:lustre-MDT0001-lwp-OST0001_UUID:3355:1862456998061056:0@lo:400:kworker/u8:2.0 00000100:00100000:0.0:1776178287.458268:0:3355:0:(events.c:361:request_in_callback()) peer: 12345-0@lo (source: 12345-0@lo) 00000100:00100000:0.0:1776178287.458310:0:3355:0:(ptlrpcd.c:415:ptlrpcd_check()) transfer 1 async RPCs [3->2] 00000100:00100000:0.0:1776178287.458315:0:3355:0:(client.c:1733:ptlrpc_send_new_req()) Sending RPC req@00000000e2ca56cc pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_02:lustre-MDT0001-mdtlov_UUID:3355:1862456998061184:0@lo:400:kworker/u8:2.0 00000100:00100000:0.0:1776178287.458341:0:3355:0:(events.c:361:request_in_callback()) peer: 12345-0@lo (source: 12345-0@lo) 00000100:00100000:0.0:1776178287.458392:0:6257:0:(service.c:2152:ptlrpc_server_handle_req_in()) got req x1862456998061056 00000100:00100000:0.0:1776178287.458436:0:6257:0:(nrs_fifo.c:179:nrs_fifo_req_get()) NRS start fifo request from 12345-0@lo, seq: 977 00000100:00100000:0.0:1776178287.458442:0:6257:0:(service.c:2324:ptlrpc_server_handle_request()) Handling RPC req@000000008d82b734 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_002:lustre-MDT0001-lwp-OST0001_UUID+5:3355:x1862456998061056:12345-0@lo:400:kworker/u8:2.0 00000100:00100000:0.0:1776178287.458493:0:6257:0:(service.c:2372:ptlrpc_server_handle_request()) Handled RPC req@000000008d82b734 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_002:lustre-MDT0001-lwp-OST0001_UUID+5:3355:x1862456998061056:12345-0@lo:400:kworker/u8:2.0 Request processed in 50us (226us total) trans 0 rc 0/0 00000100:00100000:0.0:1776178287.458500:0:6257:0:(nrs_fifo.c:241:nrs_fifo_req_stop()) NRS stop fifo request from 12345-0@lo, seq: 977 00000100:00100000:0.0:1776178287.458545:0:3355:0:(client.c:2215:ptlrpc_check_set()) Completed RPC req@0000000056aa3aa8 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_02:lustre-MDT0001-lwp-OST0001_UUID:3355:1862456998061056:0@lo:400:kworker/u8:2.0 00000100:00100000:3.0:1776178287.458636:0:3353:0:(client.c:1733:ptlrpc_send_new_req()) Sending RPC req@000000008a1e777f pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_00:ac61f839-0e8e-4cac-9c59-08bcfbbad330:3353:1862456998060800:0@lo:400:kworker/u8:2.0 00000100:00100000:3.0:1776178287.458674:0:3353:0:(events.c:361:request_in_callback()) peer: 12345-0@lo (source: 12345-0@lo) 00000100:00100000:3.0:1776178287.458698:0:3353:0:(client.c:1733:ptlrpc_send_new_req()) Sending RPC req@00000000e5c18d8a pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_00:lustre-MDT0001-mdtlov_UUID:3353:1862456998061248:0@lo:400:kworker/u8:2.0 00000100:00100000:3.0:1776178287.458719:0:3353:0:(events.c:361:request_in_callback()) peer: 12345-0@lo (source: 12345-0@lo) 00000100:00100000:3.0:1776178287.458736:0:3353:0:(ptlrpcd.c:415:ptlrpcd_check()) transfer 1 async RPCs [1->0] 00000100:00100000:3.0:1776178287.458742:0:3353:0:(client.c:1733:ptlrpc_send_new_req()) Sending RPC req@0000000081fb2a00 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_00:lustre-MDT0001-lwp-OST0000_UUID:3353:1862456998060928:0@lo:400:kworker/u8:2.0 00000100:00100000:3.0:1776178287.458761:0:3353:0:(events.c:361:request_in_callback()) peer: 12345-0@lo (source: 12345-0@lo) 00000100:00100000:3.0:1776178287.458787:0:20759:0:(service.c:2152:ptlrpc_server_handle_req_in()) got req x1862456998061184 00000100:00100000:3.0:1776178287.458811:0:20759:0:(service.c:2152:ptlrpc_server_handle_req_in()) got req x1862456998061248 00000100:00100000:3.0:1776178287.458828:0:20759:0:(nrs_fifo.c:179:nrs_fifo_req_get()) NRS start fifo request from 12345-0@lo, seq: 577 00000100:00100000:3.0:1776178287.458831:0:20759:0:(service.c:2324:ptlrpc_server_handle_request()) Handling RPC req@000000001ff33da0 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_010:lustre-MDT0001-mdtlov_UUID+5:3355:x1862456998061184:12345-0@lo:400:kworker/u8:2.0 00000100:00100000:3.0:1776178287.458864:0:20759:0:(service.c:2372:ptlrpc_server_handle_request()) Handled RPC req@000000001ff33da0 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_010:lustre-MDT0001-mdtlov_UUID+5:3355:x1862456998061184:12345-0@lo:400:kworker/u8:2.0 Request processed in 32us (522us total) trans 0 rc 0/0 00000100:00100000:3.0:1776178287.458870:0:20759:0:(nrs_fifo.c:241:nrs_fifo_req_stop()) NRS stop fifo request from 12345-0@lo, seq: 577 00000100:00100000:3.0:1776178287.458881:0:20759:0:(nrs_fifo.c:179:nrs_fifo_req_get()) NRS start fifo request from 12345-0@lo, seq: 578 00000100:00100000:3.0:1776178287.458884:0:20759:0:(service.c:2324:ptlrpc_server_handle_request()) Handling RPC req@00000000829797f1 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_010:lustre-MDT0001-mdtlov_UUID+5:3353:x1862456998061248:12345-0@lo:400:kworker/u8:2.0 00000100:00100000:3.0:1776178287.458913:0:20759:0:(service.c:2372:ptlrpc_server_handle_request()) Handled RPC req@00000000829797f1 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_010:lustre-MDT0001-mdtlov_UUID+5:3353:x1862456998061248:12345-0@lo:400:kworker/u8:2.0 Request processed in 28us (193us total) trans 0 rc 0/0 00000100:00100000:3.0:1776178287.458919:0:20759:0:(nrs_fifo.c:241:nrs_fifo_req_stop()) NRS stop fifo request from 12345-0@lo, seq: 578 00000100:00100000:3.0:1776178287.458941:0:6255:0:(service.c:2152:ptlrpc_server_handle_req_in()) got req x1862456998060928 00000100:00100000:3.0:1776178287.458960:0:6255:0:(nrs_fifo.c:179:nrs_fifo_req_get()) NRS start fifo request from 12345-0@lo, seq: 978 00000100:00100000:3.0:1776178287.458963:0:6255:0:(service.c:2324:ptlrpc_server_handle_request()) Handling RPC req@000000008dec7778 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_000:lustre-MDT0001-lwp-OST0000_UUID+5:3353:x1862456998060928:12345-0@lo:400:kworker/u8:2.0 00000100:00100000:3.0:1776178287.458989:0:6255:0:(service.c:2372:ptlrpc_server_handle_request()) Handled RPC req@000000008dec7778 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_000:lustre-MDT0001-lwp-OST0000_UUID+5:3353:x1862456998060928:12345-0@lo:400:kworker/u8:2.0 Request processed in 25us (227us total) trans 0 rc 0/0 00000100:00100000:3.0:1776178287.458993:0:6255:0:(nrs_fifo.c:241:nrs_fifo_req_stop()) NRS stop fifo request from 12345-0@lo, seq: 978 00000100:00100000:3.0:1776178287.459016:0:3353:0:(client.c:2215:ptlrpc_check_set()) Completed RPC req@0000000081fb2a00 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_00:lustre-MDT0001-lwp-OST0000_UUID:3353:1862456998060928:0@lo:400:kworker/u8:2.0 00000100:00100000:3.0:1776178287.459025:0:3353:0:(client.c:2215:ptlrpc_check_set()) Completed RPC req@00000000e5c18d8a pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_00:lustre-MDT0001-mdtlov_UUID:3353:1862456998061248:0@lo:400:kworker/u8:2.0 00000100:00100000:3.0:1776178287.459045:0:3355:0:(client.c:2215:ptlrpc_check_set()) Completed RPC req@00000000e2ca56cc pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_02:lustre-MDT0001-mdtlov_UUID:3355:1862456998061184:0@lo:400:kworker/u8:2.0 00000100:00100000:2.0:1776178287.459172:0:3352:0:(client.c:1733:ptlrpc_send_new_req()) Sending RPC req@00000000d9d2b484 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:lustre-MDT0000-lwp-OST0000_UUID:3352:1862456998060864:0@lo:38:kworker/u8:2.0 00000100:00100000:2.0:1776178287.459202:0:3352:0:(events.c:361:request_in_callback()) peer: 12345-0@lo (source: 12345-0@lo) 00000100:00100000:2.0:1776178287.459224:0:3352:0:(client.c:1733:ptlrpc_send_new_req()) Sending RPC req@000000009e03ef2c pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:lustre-MDT0000-lwp-OST0001_UUID:3352:1862456998060992:0@lo:38:kworker/u8:2.0 00000100:00100000:2.0:1776178287.459242:0:3352:0:(events.c:361:request_in_callback()) peer: 12345-0@lo (source: 12345-0@lo) 00000100:00100000:2.0:1776178287.459254:0:3352:0:(client.c:1733:ptlrpc_send_new_req()) Sending RPC req@00000000ed0a8ae7 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:lustre-MDT0001-mdtlov_UUID:3352:1862456998061120:0@lo:38:kworker/u8:2.0 00000100:00100000:2.0:1776178287.459273:0:3352:0:(events.c:361:request_in_callback()) peer: 12345-0@lo (source: 12345-0@lo) 00000100:00100000:2.0:1776178287.459286:0:3352:0:(client.c:1733:ptlrpc_send_new_req()) Sending RPC req@0000000004c8f27c pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:lustre-MDT0000-lwp-MDT0001_UUID:3352:1862456998061312:0@lo:38:kworker/u8:2.0 00000100:00100000:2.0:1776178287.459306:0:3352:0:(events.c:361:request_in_callback()) peer: 12345-0@lo (source: 12345-0@lo) 00000100:00100000:2.0:1776178287.459333:0:14128:0:(service.c:2152:ptlrpc_server_handle_req_in()) got req x1862456998061120 00000100:00100000:2.0:1776178287.459354:0:14128:0:(nrs_fifo.c:179:nrs_fifo_req_get()) NRS start fifo request from 12345-0@lo, seq: 724 00000100:00100000:2.0:1776178287.459357:0:14128:0:(service.c:2324:ptlrpc_server_handle_request()) Handling RPC req@0000000037059d8f pname:cluuid+ref:pid:xid:nid:opc:job mdt_out00_002:0+-99:3352:x1862456998061120:12345-0@lo:38:kworker/u8:2.0 00010000:02000000:2.0:1776178287.459370:0:14128:0:(ldlm_lib.c:1139:target_handle_connect()) lustre-MDT0000: Not available for connect from 0@lo (stopping) 00000100:00100000:2.0:1776178287.459414:0:14128:0:(service.c:2372:ptlrpc_server_handle_request()) Handled RPC req@0000000037059d8f pname:cluuid+ref:pid:xid:nid:opc:job mdt_out00_002:0+-99:3352:x1862456998061120:12345-0@lo:38:kworker/u8:2.0 Request processed in 55us (140us total) trans 0 rc -19/-19 00000100:00100000:2.0:1776178287.459420:0:14128:0:(nrs_fifo.c:241:nrs_fifo_req_stop()) NRS stop fifo request from 12345-0@lo, seq: 724 00000100:00100000:2.0:1776178287.459438:0:8250:0:(service.c:2152:ptlrpc_server_handle_req_in()) got req x1862456998060864 00000100:00100000:2.0:1776178287.459455:0:8250:0:(service.c:2152:ptlrpc_server_handle_req_in()) got req x1862456998060992 00000100:00100000:2.0:1776178287.459468:0:8250:0:(service.c:2152:ptlrpc_server_handle_req_in()) got req x1862456998061312 00000100:00100000:2.0:1776178287.459476:0:8250:0:(nrs_fifo.c:179:nrs_fifo_req_get()) NRS start fifo request from 12345-0@lo, seq: 6689 00000100:00100000:2.0:1776178287.459479:0:8250:0:(service.c:2324:ptlrpc_server_handle_request()) Handling RPC req@0000000052a7c870 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_003:0+-99:3352:x1862456998060864:12345-0@lo:38:kworker/u8:2.0 00010000:02000000:2.0:1776178287.459486:0:8250:0:(ldlm_lib.c:1139:target_handle_connect()) lustre-MDT0000: Not available for connect from 0@lo (stopping) 00000100:00100000:2.0:1776178287.459510:0:8250:0:(service.c:2372:ptlrpc_server_handle_request()) Handled RPC req@0000000052a7c870 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_003:0+-99:3352:x1862456998060864:12345-0@lo:38:kworker/u8:2.0 Request processed in 30us (308us total) trans 0 rc -19/-19 00000100:00100000:2.0:1776178287.459516:0:8250:0:(nrs_fifo.c:241:nrs_fifo_req_stop()) NRS stop fifo request from 12345-0@lo, seq: 6689 00000100:00100000:2.0:1776178287.459526:0:8250:0:(nrs_fifo.c:179:nrs_fifo_req_get()) NRS start fifo request from 12345-0@lo, seq: 6690 00000100:00100000:2.0:1776178287.459529:0:8250:0:(service.c:2324:ptlrpc_server_handle_request()) Handling RPC req@00000000b63b7b84 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_003:0+-99:3352:x1862456998060992:12345-0@lo:38:kworker/u8:2.0 00010000:02000000:2.0:1776178287.459534:0:8250:0:(ldlm_lib.c:1139:target_handle_connect()) lustre-MDT0000: Not available for connect from 0@lo (stopping) 00000100:00100000:2.0:1776178287.459553:0:8250:0:(service.c:2372:ptlrpc_server_handle_request()) Handled RPC req@00000000b63b7b84 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_003:0+-99:3352:x1862456998060992:12345-0@lo:38:kworker/u8:2.0 Request processed in 23us (311us total) trans 0 rc -19/-19 00000100:00100000:2.0:1776178287.459558:0:8250:0:(nrs_fifo.c:241:nrs_fifo_req_stop()) NRS stop fifo request from 12345-0@lo, seq: 6690 00000100:00100000:2.0:1776178287.459568:0:8250:0:(nrs_fifo.c:179:nrs_fifo_req_get()) NRS start fifo request from 12345-0@lo, seq: 6691 00000100:00100000:2.0:1776178287.459571:0:8250:0:(service.c:2324:ptlrpc_server_handle_request()) Handling RPC req@00000000546809f4 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_003:0+-99:3352:x1862456998061312:12345-0@lo:38:kworker/u8:2.0 00010000:02000000:2.0:1776178287.459576:0:8250:0:(ldlm_lib.c:1139:target_handle_connect()) lustre-MDT0000: Not available for connect from 0@lo (stopping) 00000100:00100000:2.0:1776178287.459595:0:8250:0:(service.c:2372:ptlrpc_server_handle_request()) Handled RPC req@00000000546809f4 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_003:0+-99:3352:x1862456998061312:12345-0@lo:38:kworker/u8:2.0 Request processed in 23us (289us total) trans 0 rc -19/-19 00000100:00100000:2.0:1776178287.459601:0:8250:0:(nrs_fifo.c:241:nrs_fifo_req_stop()) NRS stop fifo request from 12345-0@lo, seq: 6691 00000100:00080000:2.0:1776178287.459634:0:3352:0:(import.c:86:import_set_state_nolock()) 0000000047601a98 lustre-MDT0000_UUID: changing import state from CONNECTING to DISCONN 00000100:00080000:2.0:1776178287.459638:0:3352:0:(import.c:1429:ptlrpc_connect_interpret()) recovery of lustre-MDT0000_UUID on 192.168.204.103@tcp failed (-19) 00000100:00100000:2.0:1776178287.459641:0:3352:0:(client.c:2215:ptlrpc_check_set()) Completed RPC req@00000000d9d2b484 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:lustre-MDT0000-lwp-OST0000_UUID:3352:1862456998060864:0@lo:38:kworker/u8:2.0 00000100:00080000:2.0:1776178287.459649:0:3352:0:(import.c:86:import_set_state_nolock()) 00000000dfc8db13 lustre-MDT0000_UUID: changing import state from CONNECTING to DISCONN 00000100:00080000:2.0:1776178287.459652:0:3352:0:(import.c:1429:ptlrpc_connect_interpret()) recovery of lustre-MDT0000_UUID on 192.168.204.103@tcp failed (-19) 00000100:00100000:2.0:1776178287.459654:0:3352:0:(client.c:2215:ptlrpc_check_set()) Completed RPC req@000000009e03ef2c pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:lustre-MDT0000-lwp-OST0001_UUID:3352:1862456998060992:0@lo:38:kworker/u8:2.0 00000100:00080000:2.0:1776178287.459660:0:3352:0:(import.c:86:import_set_state_nolock()) 00000000ba4f1821 lustre-MDT0000_UUID: changing import state from CONNECTING to DISCONN 00000100:00080000:2.0:1776178287.459662:0:3352:0:(import.c:1429:ptlrpc_connect_interpret()) recovery of lustre-MDT0000_UUID on 192.168.204.103@tcp failed (-19) 00000100:00100000:2.0:1776178287.459665:0:3352:0:(client.c:2215:ptlrpc_check_set()) Completed RPC req@00000000ed0a8ae7 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:lustre-MDT0001-mdtlov_UUID:3352:1862456998061120:0@lo:38:kworker/u8:2.0 00000100:00080000:2.0:1776178287.459671:0:3352:0:(import.c:86:import_set_state_nolock()) 00000000a7c55b1f lustre-MDT0000_UUID: changing import state from CONNECTING to DISCONN 00000100:00080000:2.0:1776178287.459674:0:3352:0:(import.c:1429:ptlrpc_connect_interpret()) recovery of lustre-MDT0000_UUID on 192.168.204.103@tcp failed (-19) 00000100:00100000:2.0:1776178287.459675:0:3352:0:(client.c:2215:ptlrpc_check_set()) Completed RPC req@0000000004c8f27c pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:lustre-MDT0000-lwp-MDT0001_UUID:3352:1862456998061312:0@lo:38:kworker/u8:2.0 00000100:00100000:0.0:1776178287.460011:0:20140:0:(service.c:2152:ptlrpc_server_handle_req_in()) got req x1862456998060800 00000100:00100000:0.0:1776178287.460142:0:20140:0:(nrs_fifo.c:179:nrs_fifo_req_get()) NRS start fifo request from 12345-0@lo, seq: 188 00000100:00100000:0.0:1776178287.460147:0:20140:0:(service.c:2324:ptlrpc_server_handle_request()) Handling RPC req@00000000c6a76119 pname:cluuid+ref:pid:xid:nid:opc:job ll_mgs_0002:ac61f839-0e8e-4cac-9c59-08bcfbbad330+18:3353:x1862456998060800:12345-0@lo:400:kworker/u8:2.0 00000100:00100000:0.0:1776178287.460190:0:20140:0:(service.c:2372:ptlrpc_server_handle_request()) Handled RPC req@00000000c6a76119 pname:cluuid+ref:pid:xid:nid:opc:job ll_mgs_0002:ac61f839-0e8e-4cac-9c59-08bcfbbad330+18:3353:x1862456998060800:12345-0@lo:400:kworker/u8:2.0 Request processed in 42us (1516us total) trans 0 rc 0/0 00000100:00100000:0.0:1776178287.460196:0:20140:0:(nrs_fifo.c:241:nrs_fifo_req_stop()) NRS stop fifo request from 12345-0@lo, seq: 188 00000100:00100000:1.0:1776178287.460263:0:3353:0:(client.c:2215:ptlrpc_check_set()) Completed RPC req@000000008a1e777f pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_00:ac61f839-0e8e-4cac-9c59-08bcfbbad330:3353:1862456998060800:0@lo:400:kworker/u8:2.0 00010000:00010000:1.0:1776178288.097802:0:6245:0:(ldlm_lockd.c:2792:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000020:02000400:3.0:1776178288.992210:0:22200:0:(genops.c:1786:obd_exports_barrier()) lustre-MDT0000 is waiting for obd_unlinked_exports more than 32 seconds. The obd refcount = 4. Is it stuck? 00000100:00100000:1.0:1776178288.993100:0:3354:0:(client.c:1733:ptlrpc_send_new_req()) Sending RPC req@00000000fdfa408a pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_01:lustre-MDT0001-mdtlov_UUID:3354:1862456998061376:0@lo:13:osp-pre-1-1.0 00000100:00100000:1.0:1776178288.993183:0:3354:0:(events.c:361:request_in_callback()) peer: 12345-0@lo (source: 12345-0@lo) 00000100:00100000:0.0:1776178288.993304:0:8231:0:(service.c:2152:ptlrpc_server_handle_req_in()) got req x1862456998061376 00000100:00100000:0.0:1776178288.993376:0:8231:0:(nrs_fifo.c:179:nrs_fifo_req_get()) NRS start fifo request from 12345-0@lo, seq: 521 00000100:00100000:0.0:1776178288.993382:0:8231:0:(service.c:2324:ptlrpc_server_handle_request()) Handling RPC req@00000000a13f5de6 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_create00:lustre-MDT0001-mdtlov_UUID+5:3354:x1862456998061376:12345-0@lo:13:osp-pre-1-1.0 00002000:00000024:0.0:1776178288.993420:0:8231:0:(ofd_obd.c:740:ofd_statfs()) blocks cached 229376 granted 491746368 pending 0 free 3923689472 avail 3692167168 00000100:00100000:0.0:1776178288.995201:0:8231:0:(service.c:2372:ptlrpc_server_handle_request()) Handled RPC req@00000000a13f5de6 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_create00:lustre-MDT0001-mdtlov_UUID+5:3354:x1862456998061376:12345-0@lo:13:osp-pre-1-1.0 Request processed in 1807us (2009us total) trans 0 rc 0/0 00000100:00100000:0.0:1776178288.995218:0:8231:0:(nrs_fifo.c:241:nrs_fifo_req_stop()) NRS stop fifo request from 12345-0@lo, seq: 521 00000100:00100000:1.0:1776178288.995353:0:3354:0:(client.c:2215:ptlrpc_check_set()) Completed RPC req@00000000fdfa408a pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_01:lustre-MDT0001-mdtlov_UUID:3354:1862456998061376:0@lo:13:osp-pre-1-1.0 00000020:02000400:3.0:1776178289.010614:0:22200:0:(genops.c:1752:print_export_data()) lustre-MDT0000: UNLINKED 00000000ca22e26f b50963bd-8eba-4432-9a54-8888d00e3fa1 1 (0 0 0) 1 0 1 0: 0000000000000000 8589934704 stale:0 00010000:00010000:2.0:1776178289.120983:0:9224:0:(ldlm_lockd.c:2792:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:2.0:1776178289.508356:0:3356:0:(ptlrpcd.c:415:ptlrpcd_check()) transfer 1 async RPCs [2->3] 00000100:00100000:2.0:1776178289.508389:0:3356:0:(client.c:1733:ptlrpc_send_new_req()) Sending RPC req@0000000056aa3aa8 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_03:lustre-MDT0001-mdtlov_UUID:3356:1862456998061440:0@lo:13:osp-pre-0-1.0 00000100:00100000:2.0:1776178289.508488:0:3356:0:(events.c:361:request_in_callback()) peer: 12345-0@lo (source: 12345-0@lo) 00000100:00100000:0.0:1776178289.509543:0:8231:0:(service.c:2152:ptlrpc_server_handle_req_in()) got req x1862456998061440 00000100:00100000:0.0:1776178289.509606:0:8231:0:(nrs_fifo.c:179:nrs_fifo_req_get()) NRS start fifo request from 12345-0@lo, seq: 522 00000100:00100000:0.0:1776178289.509613:0:8231:0:(service.c:2324:ptlrpc_server_handle_request()) Handling RPC req@00000000a7e4e4e8 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_create00:lustre-MDT0001-mdtlov_UUID+5:3356:x1862456998061440:12345-0@lo:13:osp-pre-0-1.0 00002000:00000024:0.0:1776178289.509648:0:8231:0:(ofd_obd.c:740:ofd_statfs()) blocks cached 315392 granted 484883776 pending 0 free 3923689472 avail 3692167168 00000100:00100000:0.0:1776178289.509718:0:8231:0:(service.c:2372:ptlrpc_server_handle_request()) Handled RPC req@00000000a7e4e4e8 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_create00:lustre-MDT0001-mdtlov_UUID+5:3356:x1862456998061440:12345-0@lo:13:osp-pre-0-1.0 Request processed in 104us (1233us total) trans 0 rc 0/0 00000100:00100000:0.0:1776178289.509725:0:8231:0:(nrs_fifo.c:241:nrs_fifo_req_stop()) NRS stop fifo request from 12345-0@lo, seq: 522 00000100:00100000:0.0:1776178289.509775:0:3356:0:(client.c:2215:ptlrpc_check_set()) Completed RPC req@0000000056aa3aa8 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_03:lustre-MDT0001-mdtlov_UUID:3356:1862456998061440:0@lo:13:osp-pre-0-1.0 00010000:00010000:2.0:1776178290.144579:0:9225:0:(ldlm_lockd.c:2792:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:0.0:1776178290.997280:0:3348:0:(events.c:361:request_in_callback()) peer: 12345-192.168.204.3@tcp (source: 12345-192.168.204.3@tcp) 00000100:00100000:0.0:1776178290.997338:0:3348:0:(events.c:361:request_in_callback()) peer: 12345-192.168.204.3@tcp (source: 12345-192.168.204.3@tcp) 00000100:00100000:0.0:1776178290.997379:0:3348:0:(events.c:361:request_in_callback()) peer: 12345-192.168.204.3@tcp (source: 12345-192.168.204.3@tcp) 00000100:00100000:0.0:1776178290.997411:0:3348:0:(events.c:361:request_in_callback()) peer: 12345-192.168.204.3@tcp (source: 12345-192.168.204.3@tcp) 00000100:00100000:0.0:1776178290.997506:0:3348:0:(events.c:361:request_in_callback()) peer: 12345-192.168.204.3@tcp (source: 12345-192.168.204.3@tcp) 00000100:00100000:0.0:1776178290.997554:0:20762:0:(service.c:2152:ptlrpc_server_handle_req_in()) got req x1862456979114624 00000100:00100000:0.0:1776178290.997606:0:20762:0:(service.c:2152:ptlrpc_server_handle_req_in()) got req x1862456979114688 00000100:00100000:0.0:1776178290.997629:0:20762:0:(nrs_fifo.c:179:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.204.3@tcp, seq: 579 00000100:00100000:0.0:1776178290.997637:0:20762:0:(service.c:2324:ptlrpc_server_handle_request()) Handling RPC req@00000000fb58fc15 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_013:b50963bd-8eba-4432-9a54-8888d00e3fa1+8:2272:x1862456979114624:12345-192.168.204.3@tcp:400:kworker/u8:4.0 00000100:00100000:1.0:1776178290.997664:0:20755:0:(nrs_fifo.c:179:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.204.3@tcp, seq: 580 00000100:00100000:1.0:1776178290.997671:0:20755:0:(service.c:2324:ptlrpc_server_handle_request()) Handling RPC req@0000000004c8f27c pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_006:b50963bd-8eba-4432-9a54-8888d00e3fa1+9:2273:x1862456979114688:12345-192.168.204.3@tcp:400:kworker/u8:4.0 00000100:00100000:0.0:1776178290.997718:0:20762:0:(service.c:2372:ptlrpc_server_handle_request()) Handled RPC req@00000000fb58fc15 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_013:b50963bd-8eba-4432-9a54-8888d00e3fa1+8:2272:x1862456979114624:12345-192.168.204.3@tcp:400:kworker/u8:4.0 Request processed in 81us (338us total) trans 0 rc 0/0 00000100:00100000:0.0:1776178290.997726:0:20762:0:(nrs_fifo.c:241:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.204.3@tcp, seq: 579 00000100:00100000:0.0:1776178290.998500:0:6255:0:(service.c:2152:ptlrpc_server_handle_req_in()) got req x1862456979114560 00000100:00100000:3.0:1776178290.998501:0:6257:0:(service.c:2152:ptlrpc_server_handle_req_in()) got req x1862456979114496 00000100:00100000:3.0:1776178290.998549:0:6257:0:(nrs_fifo.c:179:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.204.3@tcp, seq: 979 00000100:00100000:3.0:1776178290.998555:0:6257:0:(service.c:2324:ptlrpc_server_handle_request()) Handling RPC req@0000000056aa3aa8 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_002:b50963bd-8eba-4432-9a54-8888d00e3fa1+6:2273:x1862456979114560:12345-192.168.204.3@tcp:400:kworker/u8:4.0 00000100:00100000:2.0:1776178290.998606:0:20140:0:(service.c:2152:ptlrpc_server_handle_req_in()) got req x1862456979114432 00000100:00100000:2.0:1776178290.998645:0:20140:0:(nrs_fifo.c:179:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.204.3@tcp, seq: 189 00000100:00100000:2.0:1776178290.998650:0:20140:0:(service.c:2324:ptlrpc_server_handle_request()) Handling RPC req@00000000a2768ffd pname:cluuid+ref:pid:xid:nid:opc:job ll_mgs_0002:5b5c66b1-1a2f-4651-8548-0c75b9a38a94+9:2273:x1862456979114432:12345-192.168.204.3@tcp:400:kworker/u8:4.0 00000100:00100000:0.0:1776178290.998718:0:6255:0:(nrs_fifo.c:179:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.204.3@tcp, seq: 6692 00000100:00100000:0.0:1776178290.998807:0:6255:0:(service.c:2324:ptlrpc_server_handle_request()) Handling RPC req@00000000ed0a8ae7 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_000:0+-99:2269:x1862456979114496:12345-192.168.204.3@tcp:38:kworker/u8:4.0 00010000:02000000:0.0:1776178290.998826:0:6255:0:(ldlm_lib.c:1139:target_handle_connect()) lustre-MDT0000: Not available for connect from 192.168.204.3@tcp (stopping) 00000100:00100000:1.0:1776178291.005117:0:20755:0:(service.c:2372:ptlrpc_server_handle_request()) Handled RPC req@0000000004c8f27c pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_006:b50963bd-8eba-4432-9a54-8888d00e3fa1+9:2273:x1862456979114688:12345-192.168.204.3@tcp:400:kworker/u8:4.0 Request processed in 7432us (7691us total) trans 0 rc 0/0 00000100:00100000:1.0:1776178291.006981:0:20755:0:(nrs_fifo.c:241:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.204.3@tcp, seq: 580 00000100:00100000:3.0:1776178291.007309:0:6257:0:(service.c:2372:ptlrpc_server_handle_request()) Handled RPC req@0000000056aa3aa8 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_002:b50963bd-8eba-4432-9a54-8888d00e3fa1+6:2273:x1862456979114560:12345-192.168.204.3@tcp:400:kworker/u8:4.0 Request processed in 8749us (9968us total) trans 0 rc 0/0 00000100:00100000:3.0:1776178291.007321:0:6257:0:(nrs_fifo.c:241:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.204.3@tcp, seq: 979 00000100:00100000:2.0:1776178291.013115:0:20140:0:(service.c:2372:ptlrpc_server_handle_request()) Handled RPC req@00000000a2768ffd pname:cluuid+ref:pid:xid:nid:opc:job ll_mgs_0002:5b5c66b1-1a2f-4651-8548-0c75b9a38a94+9:2273:x1862456979114432:12345-192.168.204.3@tcp:400:kworker/u8:4.0 Request processed in 14451us (15832us total) trans 0 rc 0/0 00000100:00100000:2.0:1776178291.013135:0:20140:0:(nrs_fifo.c:241:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.204.3@tcp, seq: 189 00000100:00100000:0.0:1776178291.017180:0:6255:0:(service.c:2372:ptlrpc_server_handle_request()) Handled RPC req@00000000ed0a8ae7 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_000:0+-99:2269:x1862456979114496:12345-192.168.204.3@tcp:38:kworker/u8:4.0 Request processed in 18363us (19663us total) trans 0 rc -19/-19 00000100:00100000:0.0:1776178291.017198:0:6255:0:(nrs_fifo.c:241:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.204.3@tcp, seq: 6692 00010000:00010000:0.0:1776178291.168361:0:6244:0:(ldlm_lockd.c:2792:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00010000:00010000:1.0:1776178292.192358:0:6245:0:(ldlm_lockd.c:2792:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00080000:2.0:1776178292.576356:0:36:0:(pinger.c:251:ptlrpc_pinger_process_import()) lustre-MDT0000-lwp-OST0000_UUID->lustre-MDT0000_UUID: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00080000:2.0:1776178292.576369:0:36:0:(recover.c:57:ptlrpc_initiate_recovery()) lustre-MDT0000_UUID: starting recovery 00000100:00080000:2.0:1776178292.576374:0:36:0:(import.c:86:import_set_state_nolock()) 0000000047601a98 lustre-MDT0000_UUID: changing import state from DISCONN to CONNECTING 00000100:00080000:2.0:1776178292.576385:0:36:0:(import.c:534:import_select_connection()) lustre-MDT0000-lwp-OST0000: connect to NID 0@lo last attempt 994 00000100:00080000:2.0:1776178292.576389:0:36:0:(import.c:581:import_select_connection()) lustre-MDT0000-lwp-OST0000: tried all connections, increasing latency to 21s 00000100:00080000:2.0:1776178292.576396:0:36:0:(import.c:615:import_select_connection()) lustre-MDT0000-lwp-OST0000: import 0000000047601a98 using connection 192.168.204.103@tcp/0@lo 00000100:00100000:2.0:1776178292.576423:0:36:0:(import.c:807:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5) req@000000008a1e777f x1862456998061568/t0(0) o38->lustre-MDT0000-lwp-OST0000@0@lo:12/10 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' 00000100:00080000:2.0:1776178292.576475:0:36:0:(pinger.c:251:ptlrpc_pinger_process_import()) lustre-MDT0000-lwp-OST0001_UUID->lustre-MDT0000_UUID: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00080000:2.0:1776178292.576479:0:36:0:(recover.c:57:ptlrpc_initiate_recovery()) lustre-MDT0000_UUID: starting recovery 00000100:00080000:2.0:1776178292.576480:0:36:0:(import.c:86:import_set_state_nolock()) 00000000dfc8db13 lustre-MDT0000_UUID: changing import state from DISCONN to CONNECTING 00000100:00080000:2.0:1776178292.576483:0:36:0:(import.c:534:import_select_connection()) lustre-MDT0000-lwp-OST0001: connect to NID 0@lo last attempt 994 00000100:00080000:2.0:1776178292.576486:0:36:0:(import.c:581:import_select_connection()) lustre-MDT0000-lwp-OST0001: tried all connections, increasing latency to 21s 00000100:00080000:2.0:1776178292.576489:0:36:0:(import.c:615:import_select_connection()) lustre-MDT0000-lwp-OST0001: import 00000000dfc8db13 using connection 192.168.204.103@tcp/0@lo 00000100:00100000:2.0:1776178292.576497:0:36:0:(import.c:807:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5) req@00000000af9e3283 x1862456998061696/t0(0) o38->lustre-MDT0000-lwp-OST0001@0@lo:12/10 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' 00000100:00080000:2.0:1776178292.576510:0:36:0:(pinger.c:251:ptlrpc_pinger_process_import()) lustre-MDT0001-mdtlov_UUID->lustre-MDT0000_UUID: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00080000:2.0:1776178292.576513:0:36:0:(recover.c:57:ptlrpc_initiate_recovery()) lustre-MDT0000_UUID: starting recovery 00000100:00080000:2.0:1776178292.576514:0:36:0:(import.c:86:import_set_state_nolock()) 00000000ba4f1821 lustre-MDT0000_UUID: changing import state from DISCONN to CONNECTING 00000100:00080000:2.0:1776178292.576517:0:36:0:(import.c:534:import_select_connection()) lustre-MDT0000-osp-MDT0001: connect to NID 0@lo last attempt 994 00000100:00080000:2.0:1776178292.576518:0:36:0:(import.c:581:import_select_connection()) lustre-MDT0000-osp-MDT0001: tried all connections, increasing latency to 21s 00000100:00080000:2.0:1776178292.576521:0:36:0:(import.c:615:import_select_connection()) lustre-MDT0000-osp-MDT0001: import 00000000ba4f1821 using connection 192.168.204.103@tcp/0@lo 00000100:00100000:2.0:1776178292.576528:0:36:0:(import.c:807:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5) req@000000009de51106 x1862456998061824/t0(0) o38->lustre-MDT0000-osp-MDT0001@0@lo:24/4 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' 00000100:00080000:2.0:1776178292.576544:0:36:0:(pinger.c:251:ptlrpc_pinger_process_import()) lustre-MDT0000-lwp-MDT0001_UUID->lustre-MDT0000_UUID: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00080000:2.0:1776178292.576546:0:36:0:(recover.c:57:ptlrpc_initiate_recovery()) lustre-MDT0000_UUID: starting recovery 00000100:00080000:2.0:1776178292.576547:0:36:0:(import.c:86:import_set_state_nolock()) 00000000a7c55b1f lustre-MDT0000_UUID: changing import state from DISCONN to CONNECTING 00000100:00080000:2.0:1776178292.576562:0:36:0:(import.c:534:import_select_connection()) lustre-MDT0000-lwp-MDT0001: connect to NID 0@lo last attempt 994 00000100:00080000:2.0:1776178292.576563:0:36:0:(import.c:581:import_select_connection()) lustre-MDT0000-lwp-MDT0001: tried all connections, increasing latency to 21s 00000100:00080000:2.0:1776178292.576566:0:36:0:(import.c:615:import_select_connection()) lustre-MDT0000-lwp-MDT0001: import 00000000a7c55b1f using connection 192.168.204.103@tcp/0@lo 00000100:00100000:2.0:1776178292.576577:0:36:0:(import.c:807:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5) req@000000000d203d1d x1862456998062016/t0(0) o38->lustre-MDT0000-lwp-MDT0001@0@lo:12/10 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' 00000100:00100000:2.0:1776178292.576610:0:3353:0:(client.c:1733:ptlrpc_send_new_req()) Sending RPC req@00000000d9d2b484 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_00:lustre-MDT0001-lwp-OST0000_UUID:3353:1862456998061632:0@lo:400:kworker/u8:1.0 00000100:00100000:3.0:1776178292.576649:0:3355:0:(client.c:1733:ptlrpc_send_new_req()) Sending RPC req@000000003334dea7 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_02:lustre-MDT0001-mdtlov_UUID:3355:1862456998061888:0@lo:400:kworker/u8:1.0 00000100:00100000:2.0:1776178292.576683:0:3353:0:(events.c:361:request_in_callback()) peer: 12345-0@lo (source: 12345-0@lo) 00000100:00100000:3.0:1776178292.576702:0:3355:0:(events.c:361:request_in_callback()) peer: 12345-0@lo (source: 12345-0@lo) 00000100:00100000:3.0:1776178292.576740:0:3355:0:(ptlrpcd.c:415:ptlrpcd_check()) transfer 2 async RPCs [3->2] 00000100:00100000:3.0:1776178292.576889:0:3355:0:(client.c:1733:ptlrpc_send_new_req()) Sending RPC req@00000000fdfa408a pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_02:ac61f839-0e8e-4cac-9c59-08bcfbbad330:3355:1862456998061504:0@lo:400:kworker/u8:1.0 00000100:00100000:1.0:1776178292.576905:0:3352:0:(client.c:1733:ptlrpc_send_new_req()) Sending RPC req@000000008a1e777f pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:lustre-MDT0000-lwp-OST0000_UUID:3352:1862456998061568:0@lo:38:kworker/u8:1.0 00000100:00100000:3.0:1776178292.576926:0:3355:0:(events.c:361:request_in_callback()) peer: 12345-0@lo (source: 12345-0@lo) 00000100:00100000:1.0:1776178292.576942:0:3352:0:(events.c:361:request_in_callback()) peer: 12345-0@lo (source: 12345-0@lo) 00000100:00100000:3.0:1776178292.576947:0:3355:0:(client.c:1733:ptlrpc_send_new_req()) Sending RPC req@0000000015095c30 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_02:lustre-MDT0001-mdtlov_UUID:3355:1862456998061952:0@lo:400:kworker/u8:1.0 00000100:00100000:1.0:1776178292.576964:0:3352:0:(client.c:1733:ptlrpc_send_new_req()) Sending RPC req@00000000af9e3283 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:lustre-MDT0000-lwp-OST0001_UUID:3352:1862456998061696:0@lo:38:kworker/u8:1.0 00000100:00100000:3.0:1776178292.576978:0:3355:0:(events.c:361:request_in_callback()) peer: 12345-0@lo (source: 12345-0@lo) 00000100:00100000:1.0:1776178292.576993:0:3352:0:(events.c:361:request_in_callback()) peer: 12345-0@lo (source: 12345-0@lo) 00000100:00100000:1.0:1776178292.577022:0:3352:0:(client.c:1733:ptlrpc_send_new_req()) Sending RPC req@000000009de51106 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:lustre-MDT0001-mdtlov_UUID:3352:1862456998061824:0@lo:38:kworker/u8:1.0 00000100:00100000:3.0:1776178292.577034:0:6256:0:(service.c:2152:ptlrpc_server_handle_req_in()) got req x1862456998061632 00000100:00100000:1.0:1776178292.577045:0:3352:0:(events.c:361:request_in_callback()) peer: 12345-0@lo (source: 12345-0@lo) 00000100:00100000:1.0:1776178292.577095:0:3352:0:(client.c:1733:ptlrpc_send_new_req()) Sending RPC req@000000000d203d1d pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:lustre-MDT0000-lwp-MDT0001_UUID:3352:1862456998062016:0@lo:38:kworker/u8:1.0 00000100:00100000:1.0:1776178292.577113:0:3352:0:(events.c:361:request_in_callback()) peer: 12345-0@lo (source: 12345-0@lo) 00000100:00100000:1.0:1776178292.577136:0:3354:0:(client.c:1733:ptlrpc_send_new_req()) Sending RPC req@00000000c63a98c1 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_01:lustre-MDT0001-lwp-OST0001_UUID:3354:1862456998061760:0@lo:400:kworker/u8:1.0 00000100:00100000:1.0:1776178292.577154:0:3354:0:(events.c:361:request_in_callback()) peer: 12345-0@lo (source: 12345-0@lo) 00000100:00100000:1.0:1776178292.577180:0:20755:0:(service.c:2152:ptlrpc_server_handle_req_in()) got req x1862456998061888 00000100:00100000:1.0:1776178292.577208:0:20755:0:(service.c:2152:ptlrpc_server_handle_req_in()) got req x1862456998061952 00000100:00100000:1.0:1776178292.577225:0:20755:0:(nrs_fifo.c:179:nrs_fifo_req_get()) NRS start fifo request from 12345-0@lo, seq: 581 00000100:00100000:1.0:1776178292.577230:0:20755:0:(service.c:2324:ptlrpc_server_handle_request()) Handling RPC req@0000000092ddc73d pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_006:lustre-MDT0001-mdtlov_UUID+5:3355:x1862456998061888:12345-0@lo:400:kworker/u8:1.0 00000100:00100000:1.0:1776178292.577273:0:20755:0:(service.c:2372:ptlrpc_server_handle_request()) Handled RPC req@0000000092ddc73d pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_006:lustre-MDT0001-mdtlov_UUID+5:3355:x1862456998061888:12345-0@lo:400:kworker/u8:1.0 Request processed in 43us (574us total) trans 0 rc 0/0 00000100:00100000:1.0:1776178292.577279:0:20755:0:(nrs_fifo.c:241:nrs_fifo_req_stop()) NRS stop fifo request from 12345-0@lo, seq: 581 00000100:00100000:1.0:1776178292.577295:0:20755:0:(nrs_fifo.c:179:nrs_fifo_req_get()) NRS start fifo request from 12345-0@lo, seq: 582 00000100:00100000:1.0:1776178292.577298:0:20755:0:(service.c:2324:ptlrpc_server_handle_request()) Handling RPC req@00000000f064e279 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_006:lustre-MDT0001-mdtlov_UUID+5:3355:x1862456998061952:12345-0@lo:400:kworker/u8:1.0 00000100:00100000:1.0:1776178292.577315:0:20755:0:(service.c:2372:ptlrpc_server_handle_request()) Handled RPC req@00000000f064e279 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_006:lustre-MDT0001-mdtlov_UUID+5:3355:x1862456998061952:12345-0@lo:400:kworker/u8:1.0 Request processed in 17us (340us total) trans 0 rc 0/0 00000100:00100000:1.0:1776178292.577320:0:20755:0:(nrs_fifo.c:241:nrs_fifo_req_stop()) NRS stop fifo request from 12345-0@lo, seq: 582 00000100:00100000:1.0:1776178292.577335:0:6257:0:(service.c:2152:ptlrpc_server_handle_req_in()) got req x1862456998061568 00000100:00100000:1.0:1776178292.577348:0:6257:0:(service.c:2152:ptlrpc_server_handle_req_in()) got req x1862456998061696 00000100:00100000:1.0:1776178292.577355:0:6257:0:(service.c:2152:ptlrpc_server_handle_req_in()) got req x1862456998062016 00000100:00100000:1.0:1776178292.577362:0:6257:0:(service.c:2152:ptlrpc_server_handle_req_in()) got req x1862456998061760 00000100:00100000:1.0:1776178292.577373:0:6257:0:(nrs_fifo.c:179:nrs_fifo_req_get()) NRS start fifo request from 12345-0@lo, seq: 980 00000100:00100000:1.0:1776178292.577376:0:6257:0:(service.c:2324:ptlrpc_server_handle_request()) Handling RPC req@0000000010d80ea5 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_002:lustre-MDT0001-lwp-OST0001_UUID+5:3354:x1862456998061760:12345-0@lo:400:kworker/u8:1.0 00000100:00100000:1.0:1776178292.577404:0:6257:0:(service.c:2372:ptlrpc_server_handle_request()) Handled RPC req@0000000010d80ea5 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_002:lustre-MDT0001-lwp-OST0001_UUID+5:3354:x1862456998061760:12345-0@lo:400:kworker/u8:1.0 Request processed in 27us (250us total) trans 0 rc 0/0 00000100:00100000:1.0:1776178292.577409:0:6257:0:(nrs_fifo.c:241:nrs_fifo_req_stop()) NRS stop fifo request from 12345-0@lo, seq: 980 00000100:00100000:1.0:1776178292.577418:0:6257:0:(nrs_fifo.c:179:nrs_fifo_req_get()) NRS start fifo request from 12345-0@lo, seq: 6693 00000100:00100000:1.0:1776178292.577421:0:6257:0:(service.c:2324:ptlrpc_server_handle_request()) Handling RPC req@000000001b4f0008 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_002:0+-99:3352:x1862456998061568:12345-0@lo:38:kworker/u8:1.0 00010000:02000000:1.0:1776178292.577438:0:6257:0:(ldlm_lib.c:1139:target_handle_connect()) lustre-MDT0000: Not available for connect from 0@lo (stopping) 00000100:00100000:1.0:1776178292.577463:0:6257:0:(service.c:2372:ptlrpc_server_handle_request()) Handled RPC req@000000001b4f0008 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_002:0+-99:3352:x1862456998061568:12345-0@lo:38:kworker/u8:1.0 Request processed in 42us (524us total) trans 0 rc -19/-19 00000100:00100000:1.0:1776178292.577469:0:6257:0:(nrs_fifo.c:241:nrs_fifo_req_stop()) NRS stop fifo request from 12345-0@lo, seq: 6693 00000100:00100000:1.0:1776178292.577477:0:6257:0:(nrs_fifo.c:179:nrs_fifo_req_get()) NRS start fifo request from 12345-0@lo, seq: 6694 00000100:00100000:1.0:1776178292.577479:0:6257:0:(service.c:2324:ptlrpc_server_handle_request()) Handling RPC req@000000001e2473b2 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_002:0+-99:3352:x1862456998061696:12345-0@lo:38:kworker/u8:1.0 00010000:02000000:1.0:1776178292.577484:0:6257:0:(ldlm_lib.c:1139:target_handle_connect()) lustre-MDT0000: Not available for connect from 0@lo (stopping) 00000100:00100000:1.0:1776178292.577499:0:6257:0:(service.c:2372:ptlrpc_server_handle_request()) Handled RPC req@000000001e2473b2 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_002:0+-99:3352:x1862456998061696:12345-0@lo:38:kworker/u8:1.0 Request processed in 19us (509us total) trans 0 rc -19/-19 00000100:00100000:1.0:1776178292.577504:0:6257:0:(nrs_fifo.c:241:nrs_fifo_req_stop()) NRS stop fifo request from 12345-0@lo, seq: 6694 00000100:00100000:1.0:1776178292.577510:0:6257:0:(nrs_fifo.c:179:nrs_fifo_req_get()) NRS start fifo request from 12345-0@lo, seq: 6695 00000100:00100000:1.0:1776178292.577513:0:6257:0:(service.c:2324:ptlrpc_server_handle_request()) Handling RPC req@00000000ee27102e pname:cluuid+ref:pid:xid:nid:opc:job mdt00_002:0+-99:3352:x1862456998062016:12345-0@lo:38:kworker/u8:1.0 00010000:02000000:1.0:1776178292.577517:0:6257:0:(ldlm_lib.c:1139:target_handle_connect()) lustre-MDT0000: Not available for connect from 0@lo (stopping) 00000100:00100000:1.0:1776178292.577529:0:6257:0:(service.c:2372:ptlrpc_server_handle_request()) Handled RPC req@00000000ee27102e pname:cluuid+ref:pid:xid:nid:opc:job mdt00_002:0+-99:3352:x1862456998062016:12345-0@lo:38:kworker/u8:1.0 Request processed in 16us (416us total) trans 0 rc -19/-19 00000100:00100000:1.0:1776178292.577534:0:6257:0:(nrs_fifo.c:241:nrs_fifo_req_stop()) NRS stop fifo request from 12345-0@lo, seq: 6695 00000100:00100000:1.0:1776178292.577549:0:15213:0:(service.c:2152:ptlrpc_server_handle_req_in()) got req x1862456998061824 00000100:00100000:1.0:1776178292.577565:0:15213:0:(nrs_fifo.c:179:nrs_fifo_req_get()) NRS start fifo request from 12345-0@lo, seq: 725 00000100:00100000:1.0:1776178292.577568:0:15213:0:(service.c:2324:ptlrpc_server_handle_request()) Handling RPC req@000000001c8035b1 pname:cluuid+ref:pid:xid:nid:opc:job mdt_out00_003:0+-99:3352:x1862456998061824:12345-0@lo:38:kworker/u8:1.0 00010000:02000000:1.0:1776178292.577574:0:15213:0:(ldlm_lib.c:1139:target_handle_connect()) lustre-MDT0000: Not available for connect from 0@lo (stopping) 00000100:00100000:1.0:1776178292.577591:0:15213:0:(service.c:2372:ptlrpc_server_handle_request()) Handled RPC req@000000001c8035b1 pname:cluuid+ref:pid:xid:nid:opc:job mdt_out00_003:0+-99:3352:x1862456998061824:12345-0@lo:38:kworker/u8:1.0 Request processed in 22us (548us total) trans 0 rc -19/-19 00000100:00100000:1.0:1776178292.577595:0:15213:0:(nrs_fifo.c:241:nrs_fifo_req_stop()) NRS stop fifo request from 12345-0@lo, seq: 725 00000100:00100000:1.0:1776178292.577659:0:3354:0:(client.c:2215:ptlrpc_check_set()) Completed RPC req@00000000c63a98c1 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_01:lustre-MDT0001-lwp-OST0001_UUID:3354:1862456998061760:0@lo:400:kworker/u8:1.0 00000100:00080000:1.0:1776178292.577685:0:3352:0:(import.c:86:import_set_state_nolock()) 0000000047601a98 lustre-MDT0000_UUID: changing import state from CONNECTING to DISCONN 00000100:00080000:1.0:1776178292.577689:0:3352:0:(import.c:1429:ptlrpc_connect_interpret()) recovery of lustre-MDT0000_UUID on 192.168.204.103@tcp failed (-19) 00000100:00100000:1.0:1776178292.577692:0:3352:0:(client.c:2215:ptlrpc_check_set()) Completed RPC req@000000008a1e777f pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:lustre-MDT0000-lwp-OST0000_UUID:3352:1862456998061568:0@lo:38:kworker/u8:1.0 00000100:00080000:1.0:1776178292.577698:0:3352:0:(import.c:86:import_set_state_nolock()) 00000000dfc8db13 lustre-MDT0000_UUID: changing import state from CONNECTING to DISCONN 00000100:00080000:1.0:1776178292.577700:0:3352:0:(import.c:1429:ptlrpc_connect_interpret()) recovery of lustre-MDT0000_UUID on 192.168.204.103@tcp failed (-19) 00000100:00100000:1.0:1776178292.577702:0:3352:0:(client.c:2215:ptlrpc_check_set()) Completed RPC req@00000000af9e3283 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:lustre-MDT0000-lwp-OST0001_UUID:3352:1862456998061696:0@lo:38:kworker/u8:1.0 00000100:00080000:1.0:1776178292.577708:0:3352:0:(import.c:86:import_set_state_nolock()) 00000000ba4f1821 lustre-MDT0000_UUID: changing import state from CONNECTING to DISCONN 00000100:00080000:1.0:1776178292.577710:0:3352:0:(import.c:1429:ptlrpc_connect_interpret()) recovery of lustre-MDT0000_UUID on 192.168.204.103@tcp failed (-19) 00000100:00100000:1.0:1776178292.577862:0:3352:0:(client.c:2215:ptlrpc_check_set()) Completed RPC req@000000009de51106 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:lustre-MDT0001-mdtlov_UUID:3352:1862456998061824:0@lo:38:kworker/u8:1.0 00000100:00080000:1.0:1776178292.577877:0:3352:0:(import.c:86:import_set_state_nolock()) 00000000a7c55b1f lustre-MDT0000_UUID: changing import state from CONNECTING to DISCONN 00000100:00100000:3.0:1776178292.577877:0:6256:0:(nrs_fifo.c:179:nrs_fifo_req_get()) NRS start fifo request from 12345-0@lo, seq: 981 00000100:00080000:1.0:1776178292.577882:0:3352:0:(import.c:1429:ptlrpc_connect_interpret()) recovery of lustre-MDT0000_UUID on 192.168.204.103@tcp failed (-19) 00000100:00100000:3.0:1776178292.577886:0:6256:0:(service.c:2324:ptlrpc_server_handle_request()) Handling RPC req@00000000fb022b32 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_001:lustre-MDT0001-lwp-OST0000_UUID+5:3353:x1862456998061632:12345-0@lo:400:kworker/u8:1.0 00000100:00100000:1.0:1776178292.577887:0:3352:0:(client.c:2215:ptlrpc_check_set()) Completed RPC req@000000000d203d1d pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:lustre-MDT0000-lwp-MDT0001_UUID:3352:1862456998062016:0@lo:38:kworker/u8:1.0 00000100:00100000:1.0:1776178292.577927:0:3355:0:(client.c:2215:ptlrpc_check_set()) Completed RPC req@0000000015095c30 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_02:lustre-MDT0001-mdtlov_UUID:3355:1862456998061952:0@lo:400:kworker/u8:1.0 00000100:00100000:3.0:1776178292.577937:0:6256:0:(service.c:2372:ptlrpc_server_handle_request()) Handled RPC req@00000000fb022b32 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_001:lustre-MDT0001-lwp-OST0000_UUID+5:3353:x1862456998061632:12345-0@lo:400:kworker/u8:1.0 Request processed in 51us (1255us total) trans 0 rc 0/0 00000100:00100000:1.0:1776178292.577941:0:3355:0:(client.c:2215:ptlrpc_check_set()) Completed RPC req@000000003334dea7 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_02:lustre-MDT0001-mdtlov_UUID:3355:1862456998061888:0@lo:400:kworker/u8:1.0 00000100:00100000:3.0:1776178292.577950:0:6256:0:(nrs_fifo.c:241:nrs_fifo_req_stop()) NRS stop fifo request from 12345-0@lo, seq: 981 00000100:00100000:1.0:1776178292.578004:0:20140:0:(service.c:2152:ptlrpc_server_handle_req_in()) got req x1862456998061504 00000100:00100000:1.0:1776178292.578052:0:20140:0:(nrs_fifo.c:179:nrs_fifo_req_get()) NRS start fifo request from 12345-0@lo, seq: 190 00000100:00100000:1.0:1776178292.578056:0:20140:0:(service.c:2324:ptlrpc_server_handle_request()) Handling RPC req@00000000887443f4 pname:cluuid+ref:pid:xid:nid:opc:job ll_mgs_0002:ac61f839-0e8e-4cac-9c59-08bcfbbad330+18:3355:x1862456998061504:12345-0@lo:400:kworker/u8:1.0 00000100:00100000:1.0:1776178292.578110:0:20140:0:(service.c:2372:ptlrpc_server_handle_request()) Handled RPC req@00000000887443f4 pname:cluuid+ref:pid:xid:nid:opc:job ll_mgs_0002:ac61f839-0e8e-4cac-9c59-08bcfbbad330+18:3355:x1862456998061504:12345-0@lo:400:kworker/u8:1.0 Request processed in 54us (1186us total) trans 0 rc 0/0 00000100:00100000:1.0:1776178292.578115:0:20140:0:(nrs_fifo.c:241:nrs_fifo_req_stop()) NRS stop fifo request from 12345-0@lo, seq: 190 00000100:00100000:1.0:1776178292.578147:0:3355:0:(client.c:2215:ptlrpc_check_set()) Completed RPC req@00000000fdfa408a pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_02:ac61f839-0e8e-4cac-9c59-08bcfbbad330:3355:1862456998061504:0@lo:400:kworker/u8:1.0 00000100:00100000:2.0:1776178292.579133:0:3353:0:(client.c:2215:ptlrpc_check_set()) Completed RPC req@00000000d9d2b484 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_00:lustre-MDT0001-lwp-OST0000_UUID:3353:1862456998061632:0@lo:400:kworker/u8:1.0 00010000:00010000:2.0:1776178293.218228:0:9224:0:(ldlm_lockd.c:2792:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:0.0:1776178294.113786:0:3354:0:(ptlrpcd.c:415:ptlrpcd_check()) transfer 1 async RPCs [0->1] 00000100:00100000:0.0:1776178294.113810:0:3354:0:(client.c:1733:ptlrpc_send_new_req()) Sending RPC req@00000000ffd05959 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_01:lustre-MDT0001-mdtlov_UUID:3354:1862456998062080:0@lo:13:osp-pre-1-1.0 00000100:00100000:0.0:1776178294.113888:0:3354:0:(events.c:361:request_in_callback()) peer: 12345-0@lo (source: 12345-0@lo) 00000100:00100000:2.0:1776178294.114046:0:9554:0:(service.c:2152:ptlrpc_server_handle_req_in()) got req x1862456998062080 00000100:00100000:2.0:1776178294.114124:0:9554:0:(nrs_fifo.c:179:nrs_fifo_req_get()) NRS start fifo request from 12345-0@lo, seq: 523 00000100:00100000:2.0:1776178294.114130:0:9554:0:(service.c:2324:ptlrpc_server_handle_request()) Handling RPC req@000000009e03ef2c pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_create00:lustre-MDT0001-mdtlov_UUID+5:3354:x1862456998062080:12345-0@lo:13:osp-pre-1-1.0 00002000:00000024:2.0:1776178294.114161:0:9554:0:(ofd_obd.c:740:ofd_statfs()) blocks cached 229376 granted 491746368 pending 0 free 3923689472 avail 3692167168 00000100:00100000:2.0:1776178294.114217:0:9554:0:(service.c:2372:ptlrpc_server_handle_request()) Handled RPC req@000000009e03ef2c pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_create00:lustre-MDT0001-mdtlov_UUID+5:3354:x1862456998062080:12345-0@lo:13:osp-pre-1-1.0 Request processed in 87us (331us total) trans 0 rc 0/0 00000100:00100000:2.0:1776178294.114225:0:9554:0:(nrs_fifo.c:241:nrs_fifo_req_stop()) NRS stop fifo request from 12345-0@lo, seq: 523 00000100:00100000:1.0:1776178294.114356:0:3354:0:(client.c:2215:ptlrpc_check_set()) Completed RPC req@00000000ffd05959 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_01:lustre-MDT0001-mdtlov_UUID:3354:1862456998062080:0@lo:13:osp-pre-1-1.0 00010000:00010000:2.0:1776178294.243686:0:9225:0:(ldlm_lockd.c:2792:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:3.0:1776178294.625782:0:3353:0:(ptlrpcd.c:415:ptlrpcd_check()) transfer 1 async RPCs [1->0] 00000100:00100000:3.0:1776178294.625809:0:3353:0:(client.c:1733:ptlrpc_send_new_req()) Sending RPC req@00000000e2ca56cc pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_00:lustre-MDT0001-mdtlov_UUID:3353:1862456998062144:0@lo:13:osp-pre-0-1.0 00000100:00100000:3.0:1776178294.625894:0:3353:0:(events.c:361:request_in_callback()) peer: 12345-0@lo (source: 12345-0@lo) 00000100:00100000:0.0:1776178294.626591:0:8231:0:(service.c:2152:ptlrpc_server_handle_req_in()) got req x1862456998062144 00000100:00100000:0.0:1776178294.626674:0:8231:0:(nrs_fifo.c:179:nrs_fifo_req_get()) NRS start fifo request from 12345-0@lo, seq: 524 00000100:00100000:0.0:1776178294.626681:0:8231:0:(service.c:2324:ptlrpc_server_handle_request()) Handling RPC req@00000000609aa1cd pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_create00:lustre-MDT0001-mdtlov_UUID+5:3353:x1862456998062144:12345-0@lo:13:osp-pre-0-1.0 00002000:00000024:0.0:1776178294.626719:0:8231:0:(ofd_obd.c:740:ofd_statfs()) blocks cached 315392 granted 484883776 pending 0 free 3923689472 avail 3692167168 00000100:00100000:0.0:1776178294.626941:0:8231:0:(service.c:2372:ptlrpc_server_handle_request()) Handled RPC req@00000000609aa1cd pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_create00:lustre-MDT0001-mdtlov_UUID+5:3353:x1862456998062144:12345-0@lo:13:osp-pre-0-1.0 Request processed in 259us (1047us total) trans 0 rc 0/0 00000100:00100000:0.0:1776178294.626949:0:8231:0:(nrs_fifo.c:241:nrs_fifo_req_stop()) NRS stop fifo request from 12345-0@lo, seq: 524 00000100:00100000:3.0:1776178294.627103:0:3353:0:(client.c:2215:ptlrpc_check_set()) Completed RPC req@00000000e2ca56cc pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_00:lustre-MDT0001-mdtlov_UUID:3353:1862456998062144:0@lo:13:osp-pre-0-1.0 00010000:00010000:0.0:1776178295.264377:0:6244:0:(ldlm_lockd.c:2792:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:3.0:1776178296.104181:0:3347:0:(events.c:361:request_in_callback()) peer: 12345-192.168.204.3@tcp (source: 12345-192.168.204.3@tcp) 00000100:00100000:3.0:1776178296.104240:0:3347:0:(events.c:361:request_in_callback()) peer: 12345-192.168.204.3@tcp (source: 12345-192.168.204.3@tcp) 00000100:00100000:1.0:1776178296.104262:0:6255:0:(service.c:2152:ptlrpc_server_handle_req_in()) got req x1862456979114816 00000100:00100000:3.0:1776178296.104284:0:3347:0:(events.c:361:request_in_callback()) peer: 12345-192.168.204.3@tcp (source: 12345-192.168.204.3@tcp) 00000100:00100000:1.0:1776178296.104354:0:6255:0:(nrs_fifo.c:179:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.204.3@tcp, seq: 6696 00000100:00100000:1.0:1776178296.104364:0:6255:0:(service.c:2324:ptlrpc_server_handle_request()) Handling RPC req@000000004451d94d pname:cluuid+ref:pid:xid:nid:opc:job mdt00_000:0+-99:2269:x1862456979114816:12345-192.168.204.3@tcp:38:kworker/u8:3.0 00010000:02000000:1.0:1776178296.104398:0:6255:0:(ldlm_lib.c:1139:target_handle_connect()) lustre-MDT0000: Not available for connect from 192.168.204.3@tcp (stopping) 00000100:00100000:0.0:1776178296.104563:0:20137:0:(service.c:2152:ptlrpc_server_handle_req_in()) got req x1862456979114752 00000100:00100000:0.0:1776178296.104612:0:20137:0:(nrs_fifo.c:179:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.204.3@tcp, seq: 191 00000100:00100000:0.0:1776178296.104619:0:20137:0:(service.c:2324:ptlrpc_server_handle_request()) Handling RPC req@0000000076ad720a pname:cluuid+ref:pid:xid:nid:opc:job ll_mgs_0000:5b5c66b1-1a2f-4651-8548-0c75b9a38a94+9:2271:x1862456979114752:12345-192.168.204.3@tcp:400:kworker/u8:3.0 00000100:00100000:1.0:1776178296.104673:0:6255:0:(service.c:2372:ptlrpc_server_handle_request()) Handled RPC req@000000004451d94d pname:cluuid+ref:pid:xid:nid:opc:job mdt00_000:0+-99:2269:x1862456979114816:12345-192.168.204.3@tcp:38:kworker/u8:3.0 Request processed in 307us (505us total) trans 0 rc -19/-19 00000100:00100000:0.0:1776178296.104676:0:20137:0:(service.c:2372:ptlrpc_server_handle_request()) Handled RPC req@0000000076ad720a pname:cluuid+ref:pid:xid:nid:opc:job ll_mgs_0000:5b5c66b1-1a2f-4651-8548-0c75b9a38a94+9:2271:x1862456979114752:12345-192.168.204.3@tcp:400:kworker/u8:3.0 Request processed in 55us (436us total) trans 0 rc 0/0 00000100:00100000:1.0:1776178296.104687:0:6255:0:(nrs_fifo.c:241:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.204.3@tcp, seq: 6696 00000100:00100000:0.0:1776178296.104688:0:20137:0:(nrs_fifo.c:241:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.204.3@tcp, seq: 191 00000100:00100000:3.0:1776178296.107309:0:3347:0:(events.c:361:request_in_callback()) peer: 12345-192.168.204.3@tcp (source: 12345-192.168.204.3@tcp) 00000100:00100000:3.0:1776178296.107363:0:3347:0:(events.c:361:request_in_callback()) peer: 12345-192.168.204.3@tcp (source: 12345-192.168.204.3@tcp) 00000100:00100000:0.0:1776178296.107446:0:8250:0:(service.c:2152:ptlrpc_server_handle_req_in()) got req x1862456979114880 00000100:00100000:0.0:1776178296.107522:0:8250:0:(nrs_fifo.c:179:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.204.3@tcp, seq: 982 00000100:00100000:0.0:1776178296.107532:0:8250:0:(service.c:2324:ptlrpc_server_handle_request()) Handling RPC req@00000000a2dca46e pname:cluuid+ref:pid:xid:nid:opc:job mdt00_003:b50963bd-8eba-4432-9a54-8888d00e3fa1+6:2271:x1862456979114880:12345-192.168.204.3@tcp:400:kworker/u8:3.0 00000100:00100000:0.0:1776178296.107661:0:8250:0:(service.c:2372:ptlrpc_server_handle_request()) Handled RPC req@00000000a2dca46e pname:cluuid+ref:pid:xid:nid:opc:job mdt00_003:b50963bd-8eba-4432-9a54-8888d00e3fa1+6:2271:x1862456979114880:12345-192.168.204.3@tcp:400:kworker/u8:3.0 Request processed in 128us (379us total) trans 0 rc 0/0 00000100:00100000:0.0:1776178296.107669:0:8250:0:(nrs_fifo.c:241:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.204.3@tcp, seq: 982 00000100:00100000:2.0:1776178296.119621:0:20759:0:(service.c:2152:ptlrpc_server_handle_req_in()) got req x1862456979114944 00000100:00100000:2.0:1776178296.119751:0:20759:0:(service.c:2152:ptlrpc_server_handle_req_in()) got req x1862456979115008 00000100:00100000:0.0:1776178296.121350:0:14440:0:(nrs_fifo.c:179:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.204.3@tcp, seq: 583 00000100:00100000:0.0:1776178296.121364:0:14440:0:(service.c:2324:ptlrpc_server_handle_request()) Handling RPC req@00000000e837a7cd pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_004:b50963bd-8eba-4432-9a54-8888d00e3fa1+8:2271:x1862456979114944:12345-192.168.204.3@tcp:400:kworker/u8:3.0 00000100:00100000:0.0:1776178296.121478:0:14440:0:(service.c:2372:ptlrpc_server_handle_request()) Handled RPC req@00000000e837a7cd pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_004:b50963bd-8eba-4432-9a54-8888d00e3fa1+8:2271:x1862456979114944:12345-192.168.204.3@tcp:400:kworker/u8:3.0 Request processed in 116us (17193us total) trans 0 rc 0/0 00000100:00100000:0.0:1776178296.121486:0:14440:0:(nrs_fifo.c:241:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.204.3@tcp, seq: 583 00000100:00100000:0.0:1776178296.121516:0:14440:0:(nrs_fifo.c:179:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.204.3@tcp, seq: 584 00000100:00100000:0.0:1776178296.121520:0:14440:0:(service.c:2324:ptlrpc_server_handle_request()) Handling RPC req@000000005967fd64 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_004:b50963bd-8eba-4432-9a54-8888d00e3fa1+9:2271:x1862456979115008:12345-192.168.204.3@tcp:400:kworker/u8:3.0 00000100:00100000:0.0:1776178296.121557:0:14440:0:(service.c:2372:ptlrpc_server_handle_request()) Handled RPC req@000000005967fd64 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_004:b50963bd-8eba-4432-9a54-8888d00e3fa1+9:2271:x1862456979115008:12345-192.168.204.3@tcp:400:kworker/u8:3.0 Request processed in 37us (14195us total) trans 0 rc 0/0 00000100:00100000:0.0:1776178296.121563:0:14440:0:(nrs_fifo.c:241:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.204.3@tcp, seq: 584 00010000:00010000:1.0:1776178296.288761:0:6245:0:(ldlm_lockd.c:2792:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00010000:00010000:2.0:1776178297.314729:0:9224:0:(ldlm_lockd.c:2792:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00080000:2.0:1776178297.696804:0:36:0:(pinger.c:251:ptlrpc_pinger_process_import()) lustre-MDT0000-lwp-OST0000_UUID->lustre-MDT0000_UUID: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00080000:2.0:1776178297.696813:0:36:0:(recover.c:57:ptlrpc_initiate_recovery()) lustre-MDT0000_UUID: starting recovery 00000100:00080000:2.0:1776178297.696817:0:36:0:(import.c:86:import_set_state_nolock()) 0000000047601a98 lustre-MDT0000_UUID: changing import state from DISCONN to CONNECTING 00000100:00080000:2.0:1776178297.696825:0:36:0:(import.c:534:import_select_connection()) lustre-MDT0000-lwp-OST0000: connect to NID 0@lo last attempt 999 00000100:00080000:2.0:1776178297.696829:0:36:0:(import.c:581:import_select_connection()) lustre-MDT0000-lwp-OST0000: tried all connections, increasing latency to 21s 00000100:00080000:2.0:1776178297.696835:0:36:0:(import.c:615:import_select_connection()) lustre-MDT0000-lwp-OST0000: import 0000000047601a98 using connection 192.168.204.103@tcp/0@lo 00000100:00100000:2.0:1776178297.696856:0:36:0:(import.c:807:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5) req@000000008b978a8a x1862456998062272/t0(0) o38->lustre-MDT0000-lwp-OST0000@0@lo:12/10 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' 00000100:00080000:2.0:1776178297.696902:0:36:0:(pinger.c:251:ptlrpc_pinger_process_import()) lustre-MDT0000-lwp-OST0001_UUID->lustre-MDT0000_UUID: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00080000:2.0:1776178297.696905:0:36:0:(recover.c:57:ptlrpc_initiate_recovery()) lustre-MDT0000_UUID: starting recovery 00000100:00080000:2.0:1776178297.696907:0:36:0:(import.c:86:import_set_state_nolock()) 00000000dfc8db13 lustre-MDT0000_UUID: changing import state from DISCONN to CONNECTING 00000100:00080000:2.0:1776178297.696910:0:36:0:(import.c:534:import_select_connection()) lustre-MDT0000-lwp-OST0001: connect to NID 0@lo last attempt 999 00000100:00080000:2.0:1776178297.696912:0:36:0:(import.c:581:import_select_connection()) lustre-MDT0000-lwp-OST0001: tried all connections, increasing latency to 21s 00000100:00080000:2.0:1776178297.696924:0:36:0:(import.c:615:import_select_connection()) lustre-MDT0000-lwp-OST0001: import 00000000dfc8db13 using connection 192.168.204.103@tcp/0@lo 00000100:00100000:2.0:1776178297.696935:0:36:0:(import.c:807:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5) req@00000000c37c187e x1862456998062400/t0(0) o38->lustre-MDT0000-lwp-OST0001@0@lo:12/10 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' 00000100:00080000:2.0:1776178297.696959:0:36:0:(pinger.c:251:ptlrpc_pinger_process_import()) lustre-MDT0001-mdtlov_UUID->lustre-MDT0000_UUID: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00080000:2.0:1776178297.696963:0:36:0:(recover.c:57:ptlrpc_initiate_recovery()) lustre-MDT0000_UUID: starting recovery 00000100:00080000:2.0:1776178297.696965:0:36:0:(import.c:86:import_set_state_nolock()) 00000000ba4f1821 lustre-MDT0000_UUID: changing import state from DISCONN to CONNECTING 00000100:00080000:2.0:1776178297.696968:0:36:0:(import.c:534:import_select_connection()) lustre-MDT0000-osp-MDT0001: connect to NID 0@lo last attempt 999 00000100:00080000:2.0:1776178297.696969:0:36:0:(import.c:581:import_select_connection()) lustre-MDT0000-osp-MDT0001: tried all connections, increasing latency to 21s 00000100:00080000:2.0:1776178297.696973:0:36:0:(import.c:615:import_select_connection()) lustre-MDT0000-osp-MDT0001: import 00000000ba4f1821 using connection 192.168.204.103@tcp/0@lo 00000100:00100000:2.0:1776178297.696983:0:36:0:(import.c:807:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5) req@000000001ee743a7 x1862456998062528/t0(0) o38->lustre-MDT0000-osp-MDT0001@0@lo:24/4 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' 00000100:00080000:2.0:1776178297.697002:0:36:0:(pinger.c:251:ptlrpc_pinger_process_import()) lustre-MDT0000-lwp-MDT0001_UUID->lustre-MDT0000_UUID: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00080000:2.0:1776178297.697005:0:36:0:(recover.c:57:ptlrpc_initiate_recovery()) lustre-MDT0000_UUID: starting recovery 00000100:00080000:2.0:1776178297.697007:0:36:0:(import.c:86:import_set_state_nolock()) 00000000a7c55b1f lustre-MDT0000_UUID: changing import state from DISCONN to CONNECTING 00000100:00080000:2.0:1776178297.697011:0:36:0:(import.c:534:import_select_connection()) lustre-MDT0000-lwp-MDT0001: connect to NID 0@lo last attempt 999 00000100:00080000:2.0:1776178297.697013:0:36:0:(import.c:581:import_select_connection()) lustre-MDT0000-lwp-MDT0001: tried all connections, increasing latency to 21s 00000100:00080000:2.0:1776178297.697016:0:36:0:(import.c:615:import_select_connection()) lustre-MDT0000-lwp-MDT0001: import 00000000a7c55b1f using connection 192.168.204.103@tcp/0@lo 00000100:00100000:2.0:1776178297.697026:0:36:0:(import.c:807:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5) req@000000007309261c x1862456998062720/t0(0) o38->lustre-MDT0000-lwp-MDT0001@0@lo:12/10 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' 00000100:00100000:0.0:1776178297.697127:0:3356:0:(client.c:1733:ptlrpc_send_new_req()) Sending RPC req@00000000e82039bb pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_03:lustre-MDT0001-lwp-OST0000_UUID:3356:1862456998062336:0@lo:400:kworker/u8:1.0 00000100:00100000:3.0:1776178297.697154:0:3352:0:(client.c:1733:ptlrpc_send_new_req()) Sending RPC req@000000008b978a8a pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:lustre-MDT0000-lwp-OST0000_UUID:3352:1862456998062272:0@lo:38:kworker/u8:1.0 00000100:00100000:0.0:1776178297.697223:0:3356:0:(events.c:361:request_in_callback()) peer: 12345-0@lo (source: 12345-0@lo) 00000100:00100000:3.0:1776178297.697253:0:3353:0:(client.c:1733:ptlrpc_send_new_req()) Sending RPC req@0000000057946847 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_00:lustre-MDT0001-lwp-OST0001_UUID:3353:1862456998062464:0@lo:400:kworker/u8:1.0 00000100:00100000:1.0:1776178297.697295:0:3355:0:(client.c:1733:ptlrpc_send_new_req()) Sending RPC req@00000000ac4116de pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_02:ac61f839-0e8e-4cac-9c59-08bcfbbad330:3355:1862456998062208:0@lo:400:kworker/u8:1.0 00000100:00100000:2.0:1776178297.702114:0:6255:0:(service.c:2152:ptlrpc_server_handle_req_in()) got req x1862456998062336 00000100:00100000:2.0:1776178297.702193:0:6255:0:(nrs_fifo.c:179:nrs_fifo_req_get()) NRS start fifo request from 12345-0@lo, seq: 983 00000100:00100000:2.0:1776178297.702210:0:6255:0:(service.c:2324:ptlrpc_server_handle_request()) Handling RPC req@00000000e5c18d8a pname:cluuid+ref:pid:xid:nid:opc:job mdt00_000:lustre-MDT0001-lwp-OST0000_UUID+5:3356:x1862456998062336:12345-0@lo:400:kworker/u8:1.0 00000100:00100000:3.0:1776178297.702943:0:3353:0:(events.c:361:request_in_callback()) peer: 12345-0@lo (source: 12345-0@lo) 00000100:00100000:3.0:1776178297.703384:0:3353:0:(ptlrpcd.c:415:ptlrpcd_check()) transfer 1 async RPCs [1->0] 00000100:00100000:3.0:1776178297.703484:0:3353:0:(client.c:1733:ptlrpc_send_new_req()) Sending RPC req@0000000038b19922 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_00:lustre-MDT0001-mdtlov_UUID:3353:1862456998062592:0@lo:400:kworker/u8:1.0 00000100:00100000:3.0:1776178297.703529:0:3353:0:(events.c:361:request_in_callback()) peer: 12345-0@lo (source: 12345-0@lo) 00000100:00100000:3.0:1776178297.703582:0:11281:0:(service.c:2152:ptlrpc_server_handle_req_in()) got req x1862456998062464 00000100:00100000:3.0:1776178297.703629:0:11281:0:(nrs_fifo.c:179:nrs_fifo_req_get()) NRS start fifo request from 12345-0@lo, seq: 984 00000100:00100000:3.0:1776178297.703635:0:11281:0:(service.c:2324:ptlrpc_server_handle_request()) Handling RPC req@000000000541929f pname:cluuid+ref:pid:xid:nid:opc:job mdt00_004:lustre-MDT0001-lwp-OST0001_UUID+5:3353:x1862456998062464:12345-0@lo:400:kworker/u8:1.0 00000100:00100000:3.0:1776178297.703694:0:11281:0:(service.c:2372:ptlrpc_server_handle_request()) Handled RPC req@000000000541929f pname:cluuid+ref:pid:xid:nid:opc:job mdt00_004:lustre-MDT0001-lwp-OST0001_UUID+5:3353:x1862456998062464:12345-0@lo:400:kworker/u8:1.0 Request processed in 58us (758us total) trans 0 rc 0/0 00000100:00100000:3.0:1776178297.703702:0:11281:0:(nrs_fifo.c:241:nrs_fifo_req_stop()) NRS stop fifo request from 12345-0@lo, seq: 984 00000100:00100000:3.0:1776178297.703756:0:3353:0:(client.c:2215:ptlrpc_check_set()) Completed RPC req@0000000057946847 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_00:lustre-MDT0001-lwp-OST0001_UUID:3353:1862456998062464:0@lo:400:kworker/u8:1.0 00000100:00100000:3.0:1776178297.703832:0:20762:0:(service.c:2152:ptlrpc_server_handle_req_in()) got req x1862456998062592 00000100:00100000:3.0:1776178297.703860:0:20762:0:(nrs_fifo.c:179:nrs_fifo_req_get()) NRS start fifo request from 12345-0@lo, seq: 585 00000100:00100000:3.0:1776178297.703864:0:20762:0:(service.c:2324:ptlrpc_server_handle_request()) Handling RPC req@00000000002be227 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_013:lustre-MDT0001-mdtlov_UUID+5:3353:x1862456998062592:12345-0@lo:400:kworker/u8:1.0 00000100:00100000:3.0:1776178297.703901:0:20762:0:(service.c:2372:ptlrpc_server_handle_request()) Handled RPC req@00000000002be227 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_013:lustre-MDT0001-mdtlov_UUID+5:3353:x1862456998062592:12345-0@lo:400:kworker/u8:1.0 Request processed in 35us (372us total) trans 0 rc 0/0 00000100:00100000:3.0:1776178297.703907:0:20762:0:(nrs_fifo.c:241:nrs_fifo_req_stop()) NRS stop fifo request from 12345-0@lo, seq: 585 00000100:00100000:3.0:1776178297.703943:0:3353:0:(client.c:2215:ptlrpc_check_set()) Completed RPC req@0000000038b19922 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_00:lustre-MDT0001-mdtlov_UUID:3353:1862456998062592:0@lo:400:kworker/u8:1.0 00000100:00100000:0.0:1776178297.708531:0:3352:0:(events.c:361:request_in_callback()) peer: 12345-0@lo (source: 12345-0@lo) 00000100:00100000:0.0:1776178297.708589:0:3352:0:(client.c:1733:ptlrpc_send_new_req()) Sending RPC req@00000000c37c187e pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:lustre-MDT0000-lwp-OST0001_UUID:3352:1862456998062400:0@lo:38:kworker/u8:1.0 00000100:00100000:0.0:1776178297.708655:0:3352:0:(events.c:361:request_in_callback()) peer: 12345-0@lo (source: 12345-0@lo) 00000100:00100000:0.0:1776178297.708673:0:3352:0:(client.c:1733:ptlrpc_send_new_req()) Sending RPC req@000000001ee743a7 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:lustre-MDT0001-mdtlov_UUID:3352:1862456998062528:0@lo:38:kworker/u8:1.0 00000100:00100000:0.0:1776178297.708700:0:3352:0:(events.c:361:request_in_callback()) peer: 12345-0@lo (source: 12345-0@lo) 00000100:00100000:0.0:1776178297.708724:0:3352:0:(client.c:1733:ptlrpc_send_new_req()) Sending RPC req@000000007309261c pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:lustre-MDT0000-lwp-MDT0001_UUID:3352:1862456998062720:0@lo:38:kworker/u8:1.0 00000100:00100000:1.0:1776178297.709100:0:14128:0:(service.c:2152:ptlrpc_server_handle_req_in()) got req x1862456998062528 00000100:00100000:1.0:1776178297.709147:0:14128:0:(nrs_fifo.c:179:nrs_fifo_req_get()) NRS start fifo request from 12345-0@lo, seq: 726 00000100:00100000:1.0:1776178297.709154:0:14128:0:(service.c:2324:ptlrpc_server_handle_request()) Handling RPC req@000000005f1c2da4 pname:cluuid+ref:pid:xid:nid:opc:job mdt_out00_002:0+-99:3352:x1862456998062528:12345-0@lo:38:kworker/u8:1.0 00010000:02000000:1.0:1776178297.709173:0:14128:0:(ldlm_lib.c:1139:target_handle_connect()) lustre-MDT0000: Not available for connect from 0@lo (stopping) 00000100:00100000:1.0:1776178297.709228:0:14128:0:(service.c:2372:ptlrpc_server_handle_request()) Handled RPC req@000000005f1c2da4 pname:cluuid+ref:pid:xid:nid:opc:job mdt_out00_002:0+-99:3352:x1862456998062528:12345-0@lo:38:kworker/u8:1.0 Request processed in 73us (527us total) trans 0 rc -19/-19 00000100:00100000:1.0:1776178297.709235:0:14128:0:(nrs_fifo.c:241:nrs_fifo_req_stop()) NRS stop fifo request from 12345-0@lo, seq: 726 00000100:00100000:1.0:1776178297.709277:0:3355:0:(events.c:361:request_in_callback()) peer: 12345-0@lo (source: 12345-0@lo) 00000100:00100000:1.0:1776178297.709296:0:3355:0:(client.c:1733:ptlrpc_send_new_req()) Sending RPC req@000000000bdf54a0 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_02:lustre-MDT0001-mdtlov_UUID:3355:1862456998062656:0@lo:400:kworker/u8:1.0 00000100:00100000:1.0:1776178297.709321:0:3355:0:(events.c:361:request_in_callback()) peer: 12345-0@lo (source: 12345-0@lo) 00000100:00100000:1.0:1776178297.709375:0:6256:0:(service.c:2152:ptlrpc_server_handle_req_in()) got req x1862456998062272 00000100:00100000:1.0:1776178297.709395:0:6256:0:(service.c:2152:ptlrpc_server_handle_req_in()) got req x1862456998062400 00000100:00100000:1.0:1776178297.709409:0:6256:0:(nrs_fifo.c:179:nrs_fifo_req_get()) NRS start fifo request from 12345-0@lo, seq: 6697 00000100:00100000:1.0:1776178297.709412:0:6256:0:(service.c:2324:ptlrpc_server_handle_request()) Handling RPC req@0000000081fb2a00 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_001:0+-99:3352:x1862456998062272:12345-0@lo:38:kworker/u8:1.0 00010000:02000000:1.0:1776178297.709419:0:6256:0:(ldlm_lib.c:1139:target_handle_connect()) lustre-MDT0000: Not available for connect from 0@lo (stopping) 00000100:00100000:1.0:1776178297.709445:0:6256:0:(service.c:2372:ptlrpc_server_handle_request()) Handled RPC req@0000000081fb2a00 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_001:0+-99:3352:x1862456998062272:12345-0@lo:38:kworker/u8:1.0 Request processed in 32us (922us total) trans 0 rc -19/-19 00000100:00100000:1.0:1776178297.709451:0:6256:0:(nrs_fifo.c:241:nrs_fifo_req_stop()) NRS stop fifo request from 12345-0@lo, seq: 6697 00000100:00100000:1.0:1776178297.709465:0:6256:0:(nrs_fifo.c:179:nrs_fifo_req_get()) NRS start fifo request from 12345-0@lo, seq: 6698 00000100:00100000:1.0:1776178297.709468:0:6256:0:(service.c:2324:ptlrpc_server_handle_request()) Handling RPC req@00000000e2ca56cc pname:cluuid+ref:pid:xid:nid:opc:job mdt00_001:0+-99:3352:x1862456998062400:12345-0@lo:38:kworker/u8:1.0 00010000:02000000:1.0:1776178297.709473:0:6256:0:(ldlm_lib.c:1139:target_handle_connect()) lustre-MDT0000: Not available for connect from 0@lo (stopping) 00000100:00100000:1.0:1776178297.709495:0:6256:0:(service.c:2372:ptlrpc_server_handle_request()) Handled RPC req@00000000e2ca56cc pname:cluuid+ref:pid:xid:nid:opc:job mdt00_001:0+-99:3352:x1862456998062400:12345-0@lo:38:kworker/u8:1.0 Request processed in 26us (840us total) trans 0 rc -19/-19 00000100:00100000:1.0:1776178297.709500:0:6256:0:(nrs_fifo.c:241:nrs_fifo_req_stop()) NRS stop fifo request from 12345-0@lo, seq: 6698 00000100:00100000:1.0:1776178297.709541:0:20137:0:(service.c:2152:ptlrpc_server_handle_req_in()) got req x1862456998062208 00000100:00100000:1.0:1776178297.709569:0:20137:0:(nrs_fifo.c:179:nrs_fifo_req_get()) NRS start fifo request from 12345-0@lo, seq: 192 00000100:00100000:1.0:1776178297.709573:0:20137:0:(service.c:2324:ptlrpc_server_handle_request()) Handling RPC req@00000000d4927be7 pname:cluuid+ref:pid:xid:nid:opc:job ll_mgs_0000:ac61f839-0e8e-4cac-9c59-08bcfbbad330+18:3355:x1862456998062208:12345-0@lo:400:kworker/u8:1.0 00000100:00100000:1.0:1776178297.709611:0:20137:0:(service.c:2372:ptlrpc_server_handle_request()) Handled RPC req@00000000d4927be7 pname:cluuid+ref:pid:xid:nid:opc:job ll_mgs_0000:ac61f839-0e8e-4cac-9c59-08bcfbbad330+18:3355:x1862456998062208:12345-0@lo:400:kworker/u8:1.0 Request processed in 37us (335us total) trans 0 rc 0/0 00000100:00100000:1.0:1776178297.709617:0:20137:0:(nrs_fifo.c:241:nrs_fifo_req_stop()) NRS stop fifo request from 12345-0@lo, seq: 192 00000100:00100000:1.0:1776178297.709661:0:3355:0:(client.c:2215:ptlrpc_check_set()) Completed RPC req@00000000ac4116de pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_02:ac61f839-0e8e-4cac-9c59-08bcfbbad330:3355:1862456998062208:0@lo:400:kworker/u8:1.0 00000100:00100000:0.0:1776178297.710134:0:3352:0:(events.c:361:request_in_callback()) peer: 12345-0@lo (source: 12345-0@lo) 00000100:00080000:0.0:1776178297.710167:0:3352:0:(import.c:86:import_set_state_nolock()) 0000000047601a98 lustre-MDT0000_UUID: changing import state from CONNECTING to DISCONN 00000100:00080000:0.0:1776178297.710171:0:3352:0:(import.c:1429:ptlrpc_connect_interpret()) recovery of lustre-MDT0000_UUID on 192.168.204.103@tcp failed (-19) 00000100:00100000:0.0:1776178297.710175:0:3352:0:(client.c:2215:ptlrpc_check_set()) Completed RPC req@000000008b978a8a pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:lustre-MDT0000-lwp-OST0000_UUID:3352:1862456998062272:0@lo:38:kworker/u8:1.0 00000100:00080000:0.0:1776178297.710183:0:3352:0:(import.c:86:import_set_state_nolock()) 00000000dfc8db13 lustre-MDT0000_UUID: changing import state from CONNECTING to DISCONN 00000100:00080000:0.0:1776178297.710186:0:3352:0:(import.c:1429:ptlrpc_connect_interpret()) recovery of lustre-MDT0000_UUID on 192.168.204.103@tcp failed (-19) 00000100:00100000:0.0:1776178297.710188:0:3352:0:(client.c:2215:ptlrpc_check_set()) Completed RPC req@00000000c37c187e pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:lustre-MDT0000-lwp-OST0001_UUID:3352:1862456998062400:0@lo:38:kworker/u8:1.0 00000100:00080000:0.0:1776178297.710196:0:3352:0:(import.c:86:import_set_state_nolock()) 00000000ba4f1821 lustre-MDT0000_UUID: changing import state from CONNECTING to DISCONN 00000100:00080000:0.0:1776178297.710209:0:3352:0:(import.c:1429:ptlrpc_connect_interpret()) recovery of lustre-MDT0000_UUID on 192.168.204.103@tcp failed (-19) 00000100:00100000:0.0:1776178297.710212:0:3352:0:(client.c:2215:ptlrpc_check_set()) Completed RPC req@000000001ee743a7 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:lustre-MDT0001-mdtlov_UUID:3352:1862456998062528:0@lo:38:kworker/u8:1.0 00000100:00100000:3.0:1776178297.712183:0:11281:0:(service.c:2152:ptlrpc_server_handle_req_in()) got req x1862456998062720 00000100:00100000:3.0:1776178297.712262:0:11281:0:(nrs_fifo.c:179:nrs_fifo_req_get()) NRS start fifo request from 12345-0@lo, seq: 6699 00000100:00100000:3.0:1776178297.712268:0:11281:0:(service.c:2324:ptlrpc_server_handle_request()) Handling RPC req@00000000028917ab pname:cluuid+ref:pid:xid:nid:opc:job mdt00_004:0+-99:3352:x1862456998062720:12345-0@lo:38:kworker/u8:1.0 00010000:02000000:3.0:1776178297.712286:0:11281:0:(ldlm_lib.c:1139:target_handle_connect()) lustre-MDT0000: Not available for connect from 0@lo (stopping) 00000100:00100000:3.0:1776178297.712356:0:11281:0:(service.c:2372:ptlrpc_server_handle_request()) Handled RPC req@00000000028917ab pname:cluuid+ref:pid:xid:nid:opc:job mdt00_004:0+-99:3352:x1862456998062720:12345-0@lo:38:kworker/u8:1.0 Request processed in 88us (3609us total) trans 0 rc -19/-19 00000100:00100000:3.0:1776178297.712362:0:11281:0:(nrs_fifo.c:241:nrs_fifo_req_stop()) NRS stop fifo request from 12345-0@lo, seq: 6699 00000100:00100000:3.0:1776178297.712389:0:20759:0:(service.c:2152:ptlrpc_server_handle_req_in()) got req x1862456998062656 00000100:00100000:3.0:1776178297.712415:0:20759:0:(nrs_fifo.c:179:nrs_fifo_req_get()) NRS start fifo request from 12345-0@lo, seq: 586 00000100:00100000:3.0:1776178297.712418:0:20759:0:(service.c:2324:ptlrpc_server_handle_request()) Handling RPC req@0000000071333ecf pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_010:lustre-MDT0001-mdtlov_UUID+5:3355:x1862456998062656:12345-0@lo:400:kworker/u8:1.0 00000100:00100000:3.0:1776178297.712454:0:20759:0:(service.c:2372:ptlrpc_server_handle_request()) Handled RPC req@0000000071333ecf pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_010:lustre-MDT0001-mdtlov_UUID+5:3355:x1862456998062656:12345-0@lo:400:kworker/u8:1.0 Request processed in 35us (3133us total) trans 0 rc 0/0 00000100:00100000:3.0:1776178297.712459:0:20759:0:(nrs_fifo.c:241:nrs_fifo_req_stop()) NRS stop fifo request from 12345-0@lo, seq: 586 00000100:00080000:0.0:1776178297.712777:0:3352:0:(import.c:86:import_set_state_nolock()) 00000000a7c55b1f lustre-MDT0000_UUID: changing import state from CONNECTING to DISCONN 00000100:00080000:0.0:1776178297.712782:0:3352:0:(import.c:1429:ptlrpc_connect_interpret()) recovery of lustre-MDT0000_UUID on 192.168.204.103@tcp failed (-19) 00000100:00100000:0.0:1776178297.712786:0:3352:0:(client.c:2215:ptlrpc_check_set()) Completed RPC req@000000007309261c pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:lustre-MDT0000-lwp-MDT0001_UUID:3352:1862456998062720:0@lo:38:kworker/u8:1.0 00000100:00100000:2.0:1776178297.713438:0:6255:0:(service.c:2372:ptlrpc_server_handle_request()) Handled RPC req@00000000e5c18d8a pname:cluuid+ref:pid:xid:nid:opc:job mdt00_000:lustre-MDT0001-lwp-OST0000_UUID+5:3356:x1862456998062336:12345-0@lo:400:kworker/u8:1.0 Request processed in 11236us (16217us total) trans 0 rc 0/0 00000100:00100000:2.0:1776178297.713447:0:6255:0:(nrs_fifo.c:241:nrs_fifo_req_stop()) NRS stop fifo request from 12345-0@lo, seq: 983 00000100:00100000:0.0:1776178297.713628:0:3356:0:(client.c:2215:ptlrpc_check_set()) Completed RPC req@00000000e82039bb pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_03:lustre-MDT0001-lwp-OST0000_UUID:3356:1862456998062336:0@lo:400:kworker/u8:1.0 00000100:00100000:1.0:1776178297.715036:0:3355:0:(client.c:2215:ptlrpc_check_set()) Completed RPC req@000000000bdf54a0 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_02:lustre-MDT0001-mdtlov_UUID:3355:1862456998062656:0@lo:400:kworker/u8:1.0 00010000:00010000:2.0:1776178298.338146:0:9225:0:(ldlm_lockd.c:2792:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:1.0:1776178299.232594:0:3355:0:(ptlrpcd.c:415:ptlrpcd_check()) transfer 1 async RPCs [3->2] 00000100:00100000:1.0:1776178299.233187:0:3355:0:(client.c:1733:ptlrpc_send_new_req()) Sending RPC req@00000000f97fd38e pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_02:lustre-MDT0001-mdtlov_UUID:3355:1862456998062784:0@lo:13:osp-pre-1-1.0 00000100:00100000:1.0:1776178299.233280:0:3355:0:(events.c:361:request_in_callback()) peer: 12345-0@lo (source: 12345-0@lo) 00000100:00100000:2.0:1776178299.233345:0:9554:0:(service.c:2152:ptlrpc_server_handle_req_in()) got req x1862456998062784 00000100:00100000:2.0:1776178299.233423:0:9554:0:(nrs_fifo.c:179:nrs_fifo_req_get()) NRS start fifo request from 12345-0@lo, seq: 525 00000100:00100000:2.0:1776178299.233431:0:9554:0:(service.c:2324:ptlrpc_server_handle_request()) Handling RPC req@000000009220b2b0 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_create00:lustre-MDT0001-mdtlov_UUID+5:3355:x1862456998062784:12345-0@lo:13:osp-pre-1-1.0 00002000:00000024:2.0:1776178299.233471:0:9554:0:(ofd_obd.c:740:ofd_statfs()) blocks cached 229376 granted 491746368 pending 0 free 3923689472 avail 3692167168 00000100:00100000:2.0:1776178299.233531:0:9554:0:(service.c:2372:ptlrpc_server_handle_request()) Handled RPC req@000000009220b2b0 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_create00:lustre-MDT0001-mdtlov_UUID+5:3355:x1862456998062784:12345-0@lo:13:osp-pre-1-1.0 Request processed in 99us (250us total) trans 0 rc 0/0 00000100:00100000:2.0:1776178299.233539:0:9554:0:(nrs_fifo.c:241:nrs_fifo_req_stop()) NRS stop fifo request from 12345-0@lo, seq: 525 00000100:00100000:1.0:1776178299.233676:0:3355:0:(client.c:2215:ptlrpc_check_set()) Completed RPC req@00000000f97fd38e pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_02:lustre-MDT0001-mdtlov_UUID:3355:1862456998062784:0@lo:13:osp-pre-1-1.0 00010000:00010000:0.0:1776178299.360288:0:6244:0:(ldlm_lockd.c:2792:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:3.0:1776178299.744340:0:3353:0:(client.c:1733:ptlrpc_send_new_req()) Sending RPC req@00000000a0db51c1 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_00:lustre-MDT0001-mdtlov_UUID:3353:1862456998062848:0@lo:13:osp-pre-0-1.0 00000100:00100000:3.0:1776178299.744454:0:3353:0:(events.c:361:request_in_callback()) peer: 12345-0@lo (source: 12345-0@lo) 00000100:00100000:0.0:1776178299.745546:0:8231:0:(service.c:2152:ptlrpc_server_handle_req_in()) got req x1862456998062848 00000100:00100000:0.0:1776178299.745620:0:8231:0:(nrs_fifo.c:179:nrs_fifo_req_get()) NRS start fifo request from 12345-0@lo, seq: 526 00000100:00100000:0.0:1776178299.745627:0:8231:0:(service.c:2324:ptlrpc_server_handle_request()) Handling RPC req@000000006d540857 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_create00:lustre-MDT0001-mdtlov_UUID+5:3353:x1862456998062848:12345-0@lo:13:osp-pre-0-1.0 00002000:00000024:0.0:1776178299.745672:0:8231:0:(ofd_obd.c:740:ofd_statfs()) blocks cached 315392 granted 484883776 pending 0 free 3923689472 avail 3692167168 00000100:00100000:0.0:1776178299.745743:0:8231:0:(service.c:2372:ptlrpc_server_handle_request()) Handled RPC req@000000006d540857 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_create00:lustre-MDT0001-mdtlov_UUID+5:3353:x1862456998062848:12345-0@lo:13:osp-pre-0-1.0 Request processed in 116us (1292us total) trans 0 rc 0/0 00000100:00100000:0.0:1776178299.745751:0:8231:0:(nrs_fifo.c:241:nrs_fifo_req_stop()) NRS stop fifo request from 12345-0@lo, seq: 526 00000100:00100000:3.0:1776178299.746123:0:3353:0:(client.c:2215:ptlrpc_check_set()) Completed RPC req@00000000a0db51c1 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_00:lustre-MDT0001-mdtlov_UUID:3353:1862456998062848:0@lo:13:osp-pre-0-1.0 00010000:00010000:1.0:1776178300.384699:0:6245:0:(ldlm_lockd.c:2792:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:0.0:1776178301.224249:0:3348:0:(events.c:361:request_in_callback()) peer: 12345-192.168.204.3@tcp (source: 12345-192.168.204.3@tcp) 00000100:00100000:0.0:1776178301.224305:0:3348:0:(events.c:361:request_in_callback()) peer: 12345-192.168.204.3@tcp (source: 12345-192.168.204.3@tcp) 00000100:00100000:3.0:1776178301.224462:0:20762:0:(service.c:2152:ptlrpc_server_handle_req_in()) got req x1862456979115328 00000100:00100000:3.0:1776178301.224529:0:20762:0:(nrs_fifo.c:179:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.204.3@tcp, seq: 587 00000100:00100000:3.0:1776178301.224536:0:20762:0:(service.c:2324:ptlrpc_server_handle_request()) Handling RPC req@0000000052775143 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_013:b50963bd-8eba-4432-9a54-8888d00e3fa1+9:2270:x1862456979115328:12345-192.168.204.3@tcp:400:kworker/u8:3.0 00000100:00100000:1.0:1776178301.224737:0:8250:0:(service.c:2152:ptlrpc_server_handle_req_in()) got req x1862456979115200 00000100:00100000:1.0:1776178301.224778:0:8250:0:(nrs_fifo.c:179:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.204.3@tcp, seq: 985 00000100:00100000:1.0:1776178301.224782:0:8250:0:(service.c:2324:ptlrpc_server_handle_request()) Handling RPC req@00000000050847c1 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_003:b50963bd-8eba-4432-9a54-8888d00e3fa1+6:2270:x1862456979115200:12345-192.168.204.3@tcp:400:kworker/u8:3.0 00000100:00100000:0.0:1776178301.225018:0:3348:0:(events.c:361:request_in_callback()) peer: 12345-192.168.204.3@tcp (source: 12345-192.168.204.3@tcp) 00000100:00100000:2.0:1776178301.225079:0:20140:0:(service.c:2152:ptlrpc_server_handle_req_in()) got req x1862456979115072 00000100:00100000:2.0:1776178301.225115:0:20140:0:(nrs_fifo.c:179:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.204.3@tcp, seq: 193 00000100:00100000:2.0:1776178301.225120:0:20140:0:(service.c:2324:ptlrpc_server_handle_request()) Handling RPC req@00000000286b1e8d pname:cluuid+ref:pid:xid:nid:opc:job ll_mgs_0002:5b5c66b1-1a2f-4651-8548-0c75b9a38a94+9:2270:x1862456979115072:12345-192.168.204.3@tcp:400:kworker/u8:3.0 00000100:00100000:3.0:1776178301.228479:0:20762:0:(service.c:2372:ptlrpc_server_handle_request()) Handled RPC req@0000000052775143 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_013:b50963bd-8eba-4432-9a54-8888d00e3fa1+9:2270:x1862456979115328:12345-192.168.204.3@tcp:400:kworker/u8:3.0 Request processed in 1017us (1248us total) trans 0 rc 0/0 00000100:00100000:3.0:1776178301.228500:0:20762:0:(nrs_fifo.c:241:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.204.3@tcp, seq: 587 00000100:00100000:1.0:1776178301.230708:0:8250:0:(service.c:2372:ptlrpc_server_handle_request()) Handled RPC req@00000000050847c1 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_003:b50963bd-8eba-4432-9a54-8888d00e3fa1+6:2270:x1862456979115200:12345-192.168.204.3@tcp:400:kworker/u8:3.0 Request processed in 5916us (6460us total) trans 0 rc 0/0 00000100:00100000:1.0:1776178301.230723:0:8250:0:(nrs_fifo.c:241:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.204.3@tcp, seq: 985 00000100:00100000:0.0:1776178301.231511:0:3348:0:(events.c:361:request_in_callback()) peer: 12345-192.168.204.3@tcp (source: 12345-192.168.204.3@tcp) 00000100:00100000:0.0:1776178301.231551:0:3348:0:(events.c:361:request_in_callback()) peer: 12345-192.168.204.3@tcp (source: 12345-192.168.204.3@tcp) 00000100:00100000:2.0:1776178301.231654:0:20140:0:(service.c:2372:ptlrpc_server_handle_request()) Handled RPC req@00000000286b1e8d pname:cluuid+ref:pid:xid:nid:opc:job ll_mgs_0002:5b5c66b1-1a2f-4651-8548-0c75b9a38a94+9:2270:x1862456979115072:12345-192.168.204.3@tcp:400:kworker/u8:3.0 Request processed in 6531us (6635us total) trans 0 rc 0/0 00000100:00100000:2.0:1776178301.231666:0:20140:0:(nrs_fifo.c:241:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.204.3@tcp, seq: 193 00000100:00100000:1.0:1776178301.232097:0:20762:0:(service.c:2152:ptlrpc_server_handle_req_in()) got req x1862456979115264 00000100:00100000:1.0:1776178301.232159:0:20762:0:(nrs_fifo.c:179:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.204.3@tcp, seq: 588 00000100:00100000:1.0:1776178301.232165:0:20762:0:(service.c:2324:ptlrpc_server_handle_request()) Handling RPC req@000000001ef63da8 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_013:b50963bd-8eba-4432-9a54-8888d00e3fa1+8:2270:x1862456979115264:12345-192.168.204.3@tcp:400:kworker/u8:3.0 00000100:00100000:1.0:1776178301.232918:0:20762:0:(service.c:2372:ptlrpc_server_handle_request()) Handled RPC req@000000001ef63da8 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_013:b50963bd-8eba-4432-9a54-8888d00e3fa1+8:2270:x1862456979115264:12345-192.168.204.3@tcp:400:kworker/u8:3.0 Request processed in 749us (1408us total) trans 0 rc 0/0 00000100:00100000:1.0:1776178301.232925:0:20762:0:(nrs_fifo.c:241:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.204.3@tcp, seq: 588 00000100:00100000:1.0:1776178301.232955:0:8250:0:(service.c:2152:ptlrpc_server_handle_req_in()) got req x1862456979115136 00000100:00100000:1.0:1776178301.232984:0:8250:0:(nrs_fifo.c:179:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.204.3@tcp, seq: 6700 00000100:00100000:1.0:1776178301.232988:0:8250:0:(service.c:2324:ptlrpc_server_handle_request()) Handling RPC req@000000009751a398 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_003:0+-99:2269:x1862456979115136:12345-192.168.204.3@tcp:38:kworker/u8:3.0 00010000:02000000:1.0:1776178301.233002:0:8250:0:(ldlm_lib.c:1139:target_handle_connect()) lustre-MDT0000: Not available for connect from 192.168.204.3@tcp (stopping) 00000100:00100000:1.0:1776178301.265028:0:8250:0:(service.c:2372:ptlrpc_server_handle_request()) Handled RPC req@000000009751a398 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_003:0+-99:2269:x1862456979115136:12345-192.168.204.3@tcp:38:kworker/u8:3.0 Request processed in 32024us (33463us total) trans 0 rc -19/-19 00000100:00100000:1.0:1776178301.265048:0:8250:0:(nrs_fifo.c:241:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.204.3@tcp, seq: 6700 00010000:00010000:2.0:1776178301.408305:0:9224:0:(ldlm_lockd.c:2792:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00010000:00010000:2.0:1776178302.432672:0:9225:0:(ldlm_lockd.c:2792:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00080000:1.0:1776178302.816641:0:37:0:(pinger.c:251:ptlrpc_pinger_process_import()) lustre-MDT0000-lwp-OST0000_UUID->lustre-MDT0000_UUID: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00080000:1.0:1776178302.816651:0:37:0:(recover.c:57:ptlrpc_initiate_recovery()) lustre-MDT0000_UUID: starting recovery 00000100:00080000:1.0:1776178302.816655:0:37:0:(import.c:86:import_set_state_nolock()) 0000000047601a98 lustre-MDT0000_UUID: changing import state from DISCONN to CONNECTING 00000100:00080000:1.0:1776178302.816664:0:37:0:(import.c:534:import_select_connection()) lustre-MDT0000-lwp-OST0000: connect to NID 0@lo last attempt 1005 00000100:00080000:1.0:1776178302.816668:0:37:0:(import.c:581:import_select_connection()) lustre-MDT0000-lwp-OST0000: tried all connections, increasing latency to 21s 00000100:00080000:1.0:1776178302.816674:0:37:0:(import.c:615:import_select_connection()) lustre-MDT0000-lwp-OST0000: import 0000000047601a98 using connection 192.168.204.103@tcp/0@lo 00000100:00100000:1.0:1776178302.816698:0:37:0:(import.c:807:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5) req@00000000e178f924 x1862456998062976/t0(0) o38->lustre-MDT0000-lwp-OST0000@0@lo:12/10 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' 00000100:00080000:1.0:1776178302.816741:0:37:0:(pinger.c:251:ptlrpc_pinger_process_import()) lustre-MDT0000-lwp-OST0001_UUID->lustre-MDT0000_UUID: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00080000:1.0:1776178302.816745:0:37:0:(recover.c:57:ptlrpc_initiate_recovery()) lustre-MDT0000_UUID: starting recovery 00000100:00080000:1.0:1776178302.816746:0:37:0:(import.c:86:import_set_state_nolock()) 00000000dfc8db13 lustre-MDT0000_UUID: changing import state from DISCONN to CONNECTING 00000100:00080000:1.0:1776178302.816750:0:37:0:(import.c:534:import_select_connection()) lustre-MDT0000-lwp-OST0001: connect to NID 0@lo last attempt 1005 00000100:00080000:1.0:1776178302.816751:0:37:0:(import.c:581:import_select_connection()) lustre-MDT0000-lwp-OST0001: tried all connections, increasing latency to 21s 00000100:00080000:1.0:1776178302.816755:0:37:0:(import.c:615:import_select_connection()) lustre-MDT0000-lwp-OST0001: import 00000000dfc8db13 using connection 192.168.204.103@tcp/0@lo 00000100:00100000:1.0:1776178302.816765:0:37:0:(import.c:807:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5) req@00000000a66f2317 x1862456998063104/t0(0) o38->lustre-MDT0000-lwp-OST0001@0@lo:12/10 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' 00000100:00080000:1.0:1776178302.816792:0:37:0:(pinger.c:251:ptlrpc_pinger_process_import()) lustre-MDT0001-mdtlov_UUID->lustre-MDT0000_UUID: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00080000:1.0:1776178302.816794:0:37:0:(recover.c:57:ptlrpc_initiate_recovery()) lustre-MDT0000_UUID: starting recovery 00000100:00080000:1.0:1776178302.816795:0:37:0:(import.c:86:import_set_state_nolock()) 00000000ba4f1821 lustre-MDT0000_UUID: changing import state from DISCONN to CONNECTING 00000100:00080000:1.0:1776178302.816798:0:37:0:(import.c:534:import_select_connection()) lustre-MDT0000-osp-MDT0001: connect to NID 0@lo last attempt 1005 00000100:00080000:1.0:1776178302.816800:0:37:0:(import.c:581:import_select_connection()) lustre-MDT0000-osp-MDT0001: tried all connections, increasing latency to 21s 00000100:00080000:1.0:1776178302.816803:0:37:0:(import.c:615:import_select_connection()) lustre-MDT0000-osp-MDT0001: import 00000000ba4f1821 using connection 192.168.204.103@tcp/0@lo 00000100:00100000:1.0:1776178302.816813:0:37:0:(import.c:807:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5) req@00000000865e4f65 x1862456998063232/t0(0) o38->lustre-MDT0000-osp-MDT0001@0@lo:24/4 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' 00000100:00080000:1.0:1776178302.816839:0:37:0:(pinger.c:251:ptlrpc_pinger_process_import()) lustre-MDT0000-lwp-MDT0001_UUID->lustre-MDT0000_UUID: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00080000:1.0:1776178302.816842:0:37:0:(recover.c:57:ptlrpc_initiate_recovery()) lustre-MDT0000_UUID: starting recovery 00000100:00080000:1.0:1776178302.816844:0:37:0:(import.c:86:import_set_state_nolock()) 00000000a7c55b1f lustre-MDT0000_UUID: changing import state from DISCONN to CONNECTING 00000100:00080000:1.0:1776178302.816847:0:37:0:(import.c:534:import_select_connection()) lustre-MDT0000-lwp-MDT0001: connect to NID 0@lo last attempt 1005 00000100:00080000:1.0:1776178302.816849:0:37:0:(import.c:581:import_select_connection()) lustre-MDT0000-lwp-MDT0001: tried all connections, increasing latency to 21s 00000100:00080000:1.0:1776178302.816852:0:37:0:(import.c:615:import_select_connection()) lustre-MDT0000-lwp-MDT0001: import 00000000a7c55b1f using connection 192.168.204.103@tcp/0@lo 00000100:00100000:1.0:1776178302.816862:0:37:0:(import.c:807:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5) req@00000000d1e0f01c x1862456998063424/t0(0) o38->lustre-MDT0000-lwp-MDT0001@0@lo:12/10 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' 00000100:00100000:1.0:1776178302.816893:0:3355:0:(client.c:1733:ptlrpc_send_new_req()) Sending RPC req@00000000a0f2a112 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_02:lustre-MDT0001-lwp-OST0000_UUID:3355:1862456998063040:0@lo:400:kworker/u8:2.0 00000100:00100000:1.0:1776178302.816980:0:3355:0:(events.c:361:request_in_callback()) peer: 12345-0@lo (source: 12345-0@lo) 00000100:00100000:1.0:1776178302.817103:0:3355:0:(ptlrpcd.c:415:ptlrpcd_check()) transfer 1 async RPCs [3->2] 00000100:00100000:1.0:1776178302.817229:0:3355:0:(client.c:1733:ptlrpc_send_new_req()) Sending RPC req@00000000500c0ae6 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_02:lustre-MDT0001-lwp-OST0001_UUID:3355:1862456998063168:0@lo:400:kworker/u8:2.0 00000100:00100000:1.0:1776178302.817306:0:3355:0:(events.c:361:request_in_callback()) peer: 12345-0@lo (source: 12345-0@lo) 00000100:00100000:1.0:1776178302.817346:0:8250:0:(service.c:2152:ptlrpc_server_handle_req_in()) got req x1862456998063040 00000100:00100000:1.0:1776178302.817388:0:8250:0:(service.c:2152:ptlrpc_server_handle_req_in()) got req x1862456998063168 00000100:00100000:1.0:1776178302.817409:0:8250:0:(nrs_fifo.c:179:nrs_fifo_req_get()) NRS start fifo request from 12345-0@lo, seq: 986 00000100:00100000:1.0:1776178302.817415:0:8250:0:(service.c:2324:ptlrpc_server_handle_request()) Handling RPC req@00000000d2922afa pname:cluuid+ref:pid:xid:nid:opc:job mdt00_003:lustre-MDT0001-lwp-OST0000_UUID+5:3355:x1862456998063040:12345-0@lo:400:kworker/u8:2.0 00000100:00100000:1.0:1776178302.817466:0:8250:0:(service.c:2372:ptlrpc_server_handle_request()) Handled RPC req@00000000d2922afa pname:cluuid+ref:pid:xid:nid:opc:job mdt00_003:lustre-MDT0001-lwp-OST0000_UUID+5:3355:x1862456998063040:12345-0@lo:400:kworker/u8:2.0 Request processed in 51us (488us total) trans 0 rc 0/0 00000100:00100000:1.0:1776178302.817474:0:8250:0:(nrs_fifo.c:241:nrs_fifo_req_stop()) NRS stop fifo request from 12345-0@lo, seq: 986 00000100:00100000:1.0:1776178302.817517:0:3355:0:(client.c:2215:ptlrpc_check_set()) Completed RPC req@00000000a0f2a112 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_02:lustre-MDT0001-lwp-OST0000_UUID:3355:1862456998063040:0@lo:400:kworker/u8:2.0 00000100:00100000:1.0:1776178302.817544:0:8250:0:(nrs_fifo.c:179:nrs_fifo_req_get()) NRS start fifo request from 12345-0@lo, seq: 987 00000100:00100000:1.0:1776178302.817547:0:8250:0:(service.c:2324:ptlrpc_server_handle_request()) Handling RPC req@000000000154171f pname:cluuid+ref:pid:xid:nid:opc:job mdt00_003:lustre-MDT0001-lwp-OST0001_UUID+5:3355:x1862456998063168:12345-0@lo:400:kworker/u8:2.0 00000100:00100000:1.0:1776178302.817578:0:8250:0:(service.c:2372:ptlrpc_server_handle_request()) Handled RPC req@000000000154171f pname:cluuid+ref:pid:xid:nid:opc:job mdt00_003:lustre-MDT0001-lwp-OST0001_UUID+5:3355:x1862456998063168:12345-0@lo:400:kworker/u8:2.0 Request processed in 30us (272us total) trans 0 rc 0/0 00000100:00100000:1.0:1776178302.817590:0:8250:0:(nrs_fifo.c:241:nrs_fifo_req_stop()) NRS stop fifo request from 12345-0@lo, seq: 987 00000100:00100000:1.0:1776178302.817613:0:3355:0:(client.c:2215:ptlrpc_check_set()) Completed RPC req@00000000500c0ae6 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_02:lustre-MDT0001-lwp-OST0001_UUID:3355:1862456998063168:0@lo:400:kworker/u8:2.0 00000100:00100000:2.0:1776178302.817700:0:3352:0:(client.c:1733:ptlrpc_send_new_req()) Sending RPC req@00000000e178f924 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:lustre-MDT0000-lwp-OST0000_UUID:3352:1862456998062976:0@lo:38:kworker/u8:2.0 00000100:00100000:2.0:1776178302.817729:0:3352:0:(events.c:361:request_in_callback()) peer: 12345-0@lo (source: 12345-0@lo) 00000100:00100000:2.0:1776178302.817750:0:3352:0:(client.c:1733:ptlrpc_send_new_req()) Sending RPC req@00000000a66f2317 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:lustre-MDT0000-lwp-OST0001_UUID:3352:1862456998063104:0@lo:38:kworker/u8:2.0 00000100:00100000:2.0:1776178302.817770:0:3352:0:(events.c:361:request_in_callback()) peer: 12345-0@lo (source: 12345-0@lo) 00000100:00100000:2.0:1776178302.817786:0:3352:0:(client.c:1733:ptlrpc_send_new_req()) Sending RPC req@00000000865e4f65 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:lustre-MDT0001-mdtlov_UUID:3352:1862456998063232:0@lo:38:kworker/u8:2.0 00000100:00100000:2.0:1776178302.817808:0:3352:0:(events.c:361:request_in_callback()) peer: 12345-0@lo (source: 12345-0@lo) 00000100:00100000:2.0:1776178302.817834:0:3352:0:(client.c:1733:ptlrpc_send_new_req()) Sending RPC req@00000000d1e0f01c pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:lustre-MDT0000-lwp-MDT0001_UUID:3352:1862456998063424:0@lo:38:kworker/u8:2.0 00000100:00100000:2.0:1776178302.817855:0:3352:0:(events.c:361:request_in_callback()) peer: 12345-0@lo (source: 12345-0@lo) 00000100:00100000:2.0:1776178302.817879:0:6255:0:(service.c:2152:ptlrpc_server_handle_req_in()) got req x1862456998062976 00000100:00100000:2.0:1776178302.817895:0:6255:0:(service.c:2152:ptlrpc_server_handle_req_in()) got req x1862456998063104 00000100:00100000:2.0:1776178302.817904:0:6255:0:(service.c:2152:ptlrpc_server_handle_req_in()) got req x1862456998063424 00000100:00100000:2.0:1776178302.817913:0:6255:0:(nrs_fifo.c:179:nrs_fifo_req_get()) NRS start fifo request from 12345-0@lo, seq: 6701 00000100:00100000:2.0:1776178302.817917:0:6255:0:(service.c:2324:ptlrpc_server_handle_request()) Handling RPC req@00000000196ec025 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_000:0+-99:3352:x1862456998062976:12345-0@lo:38:kworker/u8:2.0 00010000:02000000:2.0:1776178302.817929:0:6255:0:(ldlm_lib.c:1139:target_handle_connect()) lustre-MDT0000: Not available for connect from 0@lo (stopping) 00000100:00100000:2.0:1776178302.817960:0:6255:0:(service.c:2372:ptlrpc_server_handle_request()) Handled RPC req@00000000196ec025 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_000:0+-99:3352:x1862456998062976:12345-0@lo:38:kworker/u8:2.0 Request processed in 42us (231us total) trans 0 rc -19/-19 00000100:00100000:2.0:1776178302.817966:0:6255:0:(nrs_fifo.c:241:nrs_fifo_req_stop()) NRS stop fifo request from 12345-0@lo, seq: 6701 00000100:00100000:2.0:1776178302.817977:0:6255:0:(nrs_fifo.c:179:nrs_fifo_req_get()) NRS start fifo request from 12345-0@lo, seq: 6702 00000100:00100000:2.0:1776178302.817980:0:6255:0:(service.c:2324:ptlrpc_server_handle_request()) Handling RPC req@00000000d81deb92 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_000:0+-99:3352:x1862456998063104:12345-0@lo:38:kworker/u8:2.0 00010000:02000000:2.0:1776178302.817986:0:6255:0:(ldlm_lib.c:1139:target_handle_connect()) lustre-MDT0000: Not available for connect from 0@lo (stopping) 00000100:00100000:2.0:1776178302.818006:0:6255:0:(service.c:2372:ptlrpc_server_handle_request()) Handled RPC req@00000000d81deb92 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_000:0+-99:3352:x1862456998063104:12345-0@lo:38:kworker/u8:2.0 Request processed in 24us (236us total) trans 0 rc -19/-19 00000100:00100000:2.0:1776178302.818011:0:6255:0:(nrs_fifo.c:241:nrs_fifo_req_stop()) NRS stop fifo request from 12345-0@lo, seq: 6702 00000100:00100000:2.0:1776178302.818038:0:6255:0:(nrs_fifo.c:179:nrs_fifo_req_get()) NRS start fifo request from 12345-0@lo, seq: 6703 00000100:00100000:2.0:1776178302.818041:0:6255:0:(service.c:2324:ptlrpc_server_handle_request()) Handling RPC req@00000000f45ed991 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_000:0+-99:3352:x1862456998063424:12345-0@lo:38:kworker/u8:2.0 00010000:02000000:2.0:1776178302.818047:0:6255:0:(ldlm_lib.c:1139:target_handle_connect()) lustre-MDT0000: Not available for connect from 0@lo (stopping) 00000100:00100000:2.0:1776178302.818093:0:6255:0:(service.c:2372:ptlrpc_server_handle_request()) Handled RPC req@00000000f45ed991 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_000:0+-99:3352:x1862456998063424:12345-0@lo:38:kworker/u8:2.0 Request processed in 50us (237us total) trans 0 rc -19/-19 00000100:00100000:2.0:1776178302.818098:0:6255:0:(nrs_fifo.c:241:nrs_fifo_req_stop()) NRS stop fifo request from 12345-0@lo, seq: 6703 00000100:00100000:2.0:1776178302.818133:0:15213:0:(service.c:2152:ptlrpc_server_handle_req_in()) got req x1862456998063232 00000100:00100000:2.0:1776178302.818151:0:15213:0:(nrs_fifo.c:179:nrs_fifo_req_get()) NRS start fifo request from 12345-0@lo, seq: 727 00000100:00100000:2.0:1776178302.818154:0:15213:0:(service.c:2324:ptlrpc_server_handle_request()) Handling RPC req@0000000007e413b7 pname:cluuid+ref:pid:xid:nid:opc:job mdt_out00_003:0+-99:3352:x1862456998063232:12345-0@lo:38:kworker/u8:2.0 00010000:02000000:2.0:1776178302.818160:0:15213:0:(ldlm_lib.c:1139:target_handle_connect()) lustre-MDT0000: Not available for connect from 0@lo (stopping) 00000100:00100000:2.0:1776178302.818183:0:15213:0:(service.c:2372:ptlrpc_server_handle_request()) Handled RPC req@0000000007e413b7 pname:cluuid+ref:pid:xid:nid:opc:job mdt_out00_003:0+-99:3352:x1862456998063232:12345-0@lo:38:kworker/u8:2.0 Request processed in 27us (375us total) trans 0 rc -19/-19 00000100:00100000:2.0:1776178302.818188:0:15213:0:(nrs_fifo.c:241:nrs_fifo_req_stop()) NRS stop fifo request from 12345-0@lo, seq: 727 00000100:00080000:2.0:1776178302.818213:0:3352:0:(import.c:86:import_set_state_nolock()) 0000000047601a98 lustre-MDT0000_UUID: changing import state from CONNECTING to DISCONN 00000100:00080000:2.0:1776178302.818216:0:3352:0:(import.c:1429:ptlrpc_connect_interpret()) recovery of lustre-MDT0000_UUID on 192.168.204.103@tcp failed (-19) 00000100:00100000:2.0:1776178302.818220:0:3352:0:(client.c:2215:ptlrpc_check_set()) Completed RPC req@00000000e178f924 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:lustre-MDT0000-lwp-OST0000_UUID:3352:1862456998062976:0@lo:38:kworker/u8:2.0 00000100:00080000:2.0:1776178302.818227:0:3352:0:(import.c:86:import_set_state_nolock()) 00000000dfc8db13 lustre-MDT0000_UUID: changing import state from CONNECTING to DISCONN 00000100:00080000:2.0:1776178302.818229:0:3352:0:(import.c:1429:ptlrpc_connect_interpret()) recovery of lustre-MDT0000_UUID on 192.168.204.103@tcp failed (-19) 00000100:00100000:2.0:1776178302.818231:0:3352:0:(client.c:2215:ptlrpc_check_set()) Completed RPC req@00000000a66f2317 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:lustre-MDT0000-lwp-OST0001_UUID:3352:1862456998063104:0@lo:38:kworker/u8:2.0 00000100:00080000:2.0:1776178302.818411:0:3352:0:(import.c:86:import_set_state_nolock()) 00000000ba4f1821 lustre-MDT0000_UUID: changing import state from CONNECTING to DISCONN 00000100:00080000:2.0:1776178302.818415:0:3352:0:(import.c:1429:ptlrpc_connect_interpret()) recovery of lustre-MDT0000_UUID on 192.168.204.103@tcp failed (-19) 00000100:00100000:2.0:1776178302.818417:0:3352:0:(client.c:2215:ptlrpc_check_set()) Completed RPC req@00000000865e4f65 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:lustre-MDT0001-mdtlov_UUID:3352:1862456998063232:0@lo:38:kworker/u8:2.0 00000100:00080000:2.0:1776178302.818424:0:3352:0:(import.c:86:import_set_state_nolock()) 00000000a7c55b1f lustre-MDT0000_UUID: changing import state from CONNECTING to DISCONN 00000100:00080000:2.0:1776178302.818426:0:3352:0:(import.c:1429:ptlrpc_connect_interpret()) recovery of lustre-MDT0000_UUID on 192.168.204.103@tcp failed (-19) 00000100:00100000:2.0:1776178302.818428:0:3352:0:(client.c:2215:ptlrpc_check_set()) Completed RPC req@00000000d1e0f01c pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:lustre-MDT0000-lwp-MDT0001_UUID:3352:1862456998063424:0@lo:38:kworker/u8:2.0 00000100:00100000:0.0:1776178302.818838:0:3353:0:(client.c:1733:ptlrpc_send_new_req()) Sending RPC req@00000000597c3414 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_00:lustre-MDT0001-mdtlov_UUID:3353:1862456998063296:0@lo:400:kworker/u8:2.0 00000100:00100000:0.0:1776178302.818883:0:3353:0:(events.c:361:request_in_callback()) peer: 12345-0@lo (source: 12345-0@lo) 00000100:00100000:0.0:1776178302.818920:0:3353:0:(ptlrpcd.c:415:ptlrpcd_check()) transfer 2 async RPCs [1->0] 00000100:00100000:0.0:1776178302.818927:0:3353:0:(client.c:1733:ptlrpc_send_new_req()) Sending RPC req@00000000bacf691a pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_00:ac61f839-0e8e-4cac-9c59-08bcfbbad330:3353:1862456998062912:0@lo:400:kworker/u8:2.0 00000100:00100000:0.0:1776178302.818953:0:3353:0:(events.c:361:request_in_callback()) peer: 12345-0@lo (source: 12345-0@lo) 00000100:00100000:0.0:1776178302.818969:0:3353:0:(client.c:1733:ptlrpc_send_new_req()) Sending RPC req@00000000a33eecf4 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_00:lustre-MDT0001-mdtlov_UUID:3353:1862456998063360:0@lo:400:kworker/u8:2.0 00000100:00100000:0.0:1776178302.818989:0:3353:0:(events.c:361:request_in_callback()) peer: 12345-0@lo (source: 12345-0@lo) 00000100:00100000:0.0:1776178302.819629:0:20759:0:(service.c:2152:ptlrpc_server_handle_req_in()) got req x1862456998063296 00000100:00100000:0.0:1776178302.819675:0:20759:0:(service.c:2152:ptlrpc_server_handle_req_in()) got req x1862456998063360 00000100:00100000:0.0:1776178302.819693:0:20759:0:(nrs_fifo.c:179:nrs_fifo_req_get()) NRS start fifo request from 12345-0@lo, seq: 589 00000100:00100000:0.0:1776178302.819698:0:20759:0:(service.c:2324:ptlrpc_server_handle_request()) Handling RPC req@00000000b2b2b654 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_010:lustre-MDT0001-mdtlov_UUID+5:3353:x1862456998063296:12345-0@lo:400:kworker/u8:2.0 00000100:00100000:0.0:1776178302.819750:0:20759:0:(service.c:2372:ptlrpc_server_handle_request()) Handled RPC req@00000000b2b2b654 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_010:lustre-MDT0001-mdtlov_UUID+5:3353:x1862456998063296:12345-0@lo:400:kworker/u8:2.0 Request processed in 52us (867us total) trans 0 rc 0/0 00000100:00100000:0.0:1776178302.819756:0:20759:0:(nrs_fifo.c:241:nrs_fifo_req_stop()) NRS stop fifo request from 12345-0@lo, seq: 589 00000100:00100000:1.0:1776178302.819950:0:20755:0:(nrs_fifo.c:179:nrs_fifo_req_get()) NRS start fifo request from 12345-0@lo, seq: 590 00000100:00100000:1.0:1776178302.819955:0:20755:0:(service.c:2324:ptlrpc_server_handle_request()) Handling RPC req@0000000013888310 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_006:lustre-MDT0001-mdtlov_UUID+5:3353:x1862456998063360:12345-0@lo:400:kworker/u8:2.0 00000100:00100000:1.0:1776178302.819987:0:20755:0:(service.c:2372:ptlrpc_server_handle_request()) Handled RPC req@0000000013888310 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_006:lustre-MDT0001-mdtlov_UUID+5:3353:x1862456998063360:12345-0@lo:400:kworker/u8:2.0 Request processed in 32us (998us total) trans 0 rc 0/0 00000100:00100000:1.0:1776178302.819992:0:20755:0:(nrs_fifo.c:241:nrs_fifo_req_stop()) NRS stop fifo request from 12345-0@lo, seq: 590 00000100:00100000:3.0:1776178302.820320:0:20140:0:(service.c:2152:ptlrpc_server_handle_req_in()) got req x1862456998062912 00000100:00100000:3.0:1776178302.820353:0:20140:0:(nrs_fifo.c:179:nrs_fifo_req_get()) NRS start fifo request from 12345-0@lo, seq: 194 00000100:00100000:3.0:1776178302.820357:0:20140:0:(service.c:2324:ptlrpc_server_handle_request()) Handling RPC req@00000000fd0a07ee pname:cluuid+ref:pid:xid:nid:opc:job ll_mgs_0002:ac61f839-0e8e-4cac-9c59-08bcfbbad330+18:3353:x1862456998062912:12345-0@lo:400:kworker/u8:2.0 00000100:00100000:3.0:1776178302.820395:0:20140:0:(service.c:2372:ptlrpc_server_handle_request()) Handled RPC req@00000000fd0a07ee pname:cluuid+ref:pid:xid:nid:opc:job ll_mgs_0002:ac61f839-0e8e-4cac-9c59-08bcfbbad330+18:3353:x1862456998062912:12345-0@lo:400:kworker/u8:2.0 Request processed in 37us (1441us total) trans 0 rc 0/0 00000100:00100000:3.0:1776178302.820401:0:20140:0:(nrs_fifo.c:241:nrs_fifo_req_stop()) NRS stop fifo request from 12345-0@lo, seq: 194 00000100:00100000:3.0:1776178302.820467:0:3353:0:(client.c:2215:ptlrpc_check_set()) Completed RPC req@00000000bacf691a pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_00:ac61f839-0e8e-4cac-9c59-08bcfbbad330:3353:1862456998062912:0@lo:400:kworker/u8:2.0 00000100:00100000:3.0:1776178302.820484:0:3353:0:(client.c:2215:ptlrpc_check_set()) Completed RPC req@00000000a33eecf4 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_00:lustre-MDT0001-mdtlov_UUID:3353:1862456998063360:0@lo:400:kworker/u8:2.0 00000100:00100000:3.0:1776178302.820494:0:3353:0:(client.c:2215:ptlrpc_check_set()) Completed RPC req@00000000597c3414 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_00:lustre-MDT0001-mdtlov_UUID:3353:1862456998063296:0@lo:400:kworker/u8:2.0 00010000:00010000:0.0:1776178303.456917:0:6244:0:(ldlm_lockd.c:2792:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:1.0:1776178304.355826:0:3355:0:(client.c:1733:ptlrpc_send_new_req()) Sending RPC req@00000000034781c2 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_02:lustre-MDT0001-mdtlov_UUID:3355:1862456998063488:0@lo:13:osp-pre-1-1.0 00000100:00100000:1.0:1776178304.355909:0:3355:0:(events.c:361:request_in_callback()) peer: 12345-0@lo (source: 12345-0@lo) 00000100:00100000:2.0:1776178304.356821:0:9554:0:(service.c:2152:ptlrpc_server_handle_req_in()) got req x1862456998063488 00000100:00100000:2.0:1776178304.356883:0:9554:0:(nrs_fifo.c:179:nrs_fifo_req_get()) NRS start fifo request from 12345-0@lo, seq: 527 00000100:00100000:2.0:1776178304.356889:0:9554:0:(service.c:2324:ptlrpc_server_handle_request()) Handling RPC req@00000000500c0ae6 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_create00:lustre-MDT0001-mdtlov_UUID+5:3355:x1862456998063488:12345-0@lo:13:osp-pre-1-1.0 00002000:00000024:2.0:1776178304.356923:0:9554:0:(ofd_obd.c:740:ofd_statfs()) blocks cached 229376 granted 491746368 pending 0 free 3923689472 avail 3692167168 00000100:00100000:2.0:1776178304.357000:0:9554:0:(service.c:2372:ptlrpc_server_handle_request()) Handled RPC req@00000000500c0ae6 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_create00:lustre-MDT0001-mdtlov_UUID+5:3355:x1862456998063488:12345-0@lo:13:osp-pre-1-1.0 Request processed in 110us (1092us total) trans 0 rc 0/0 00000100:00100000:2.0:1776178304.357008:0:9554:0:(nrs_fifo.c:241:nrs_fifo_req_stop()) NRS stop fifo request from 12345-0@lo, seq: 527 00000100:00100000:1.0:1776178304.357308:0:3355:0:(client.c:2215:ptlrpc_check_set()) Completed RPC req@00000000034781c2 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_02:lustre-MDT0001-mdtlov_UUID:3355:1862456998063488:0@lo:13:osp-pre-1-1.0 00010000:00010000:1.0:1776178304.480339:0:6245:0:(ldlm_lockd.c:2792:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:3.0:1776178304.865463:0:3356:0:(client.c:1733:ptlrpc_send_new_req()) Sending RPC req@00000000ad228ff6 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_03:lustre-MDT0001-mdtlov_UUID:3356:1862456998063552:0@lo:13:osp-pre-0-1.0 00000100:00100000:3.0:1776178304.865546:0:3356:0:(events.c:361:request_in_callback()) peer: 12345-0@lo (source: 12345-0@lo) 00000100:00100000:0.0:1776178304.865922:0:8231:0:(service.c:2152:ptlrpc_server_handle_req_in()) got req x1862456998063552 00000100:00100000:0.0:1776178304.865975:0:8231:0:(nrs_fifo.c:179:nrs_fifo_req_get()) NRS start fifo request from 12345-0@lo, seq: 528 00000100:00100000:0.0:1776178304.865980:0:8231:0:(service.c:2324:ptlrpc_server_handle_request()) Handling RPC req@0000000008b3a3ad pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_create00:lustre-MDT0001-mdtlov_UUID+5:3356:x1862456998063552:12345-0@lo:13:osp-pre-0-1.0 00002000:00000024:0.0:1776178304.866008:0:8231:0:(ofd_obd.c:740:ofd_statfs()) blocks cached 315392 granted 484883776 pending 0 free 3923689472 avail 3692167168 00000100:00100000:0.0:1776178304.866101:0:8231:0:(service.c:2372:ptlrpc_server_handle_request()) Handled RPC req@0000000008b3a3ad pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_create00:lustre-MDT0001-mdtlov_UUID+5:3356:x1862456998063552:12345-0@lo:13:osp-pre-0-1.0 Request processed in 116us (553us total) trans 0 rc 0/0 00000100:00100000:0.0:1776178304.866108:0:8231:0:(nrs_fifo.c:241:nrs_fifo_req_stop()) NRS stop fifo request from 12345-0@lo, seq: 528 00000100:00100000:3.0:1776178304.866185:0:3356:0:(client.c:2215:ptlrpc_check_set()) Completed RPC req@00000000ad228ff6 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_03:lustre-MDT0001-mdtlov_UUID:3356:1862456998063552:0@lo:13:osp-pre-0-1.0 00010000:00010000:2.0:1776178305.504908:0:9224:0:(ldlm_lockd.c:2792:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:0.0:1776178306.339019:0:3346:0:(events.c:361:request_in_callback()) peer: 12345-192.168.204.3@tcp (source: 12345-192.168.204.3@tcp) 00000100:00100000:0.0:1776178306.339212:0:3346:0:(events.c:361:request_in_callback()) peer: 12345-192.168.204.3@tcp (source: 12345-192.168.204.3@tcp) 00000100:00100000:0.0:1776178306.339242:0:3346:0:(events.c:361:request_in_callback()) peer: 12345-192.168.204.3@tcp (source: 12345-192.168.204.3@tcp) 00000100:00100000:0.0:1776178306.339280:0:3346:0:(events.c:361:request_in_callback()) peer: 12345-192.168.204.3@tcp (source: 12345-192.168.204.3@tcp) 00000100:00100000:2.0:1776178306.339347:0:20759:0:(service.c:2152:ptlrpc_server_handle_req_in()) got req x1862456979115584 00000100:00100000:2.0:1776178306.339401:0:20759:0:(nrs_fifo.c:179:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.204.3@tcp, seq: 591 00000100:00100000:2.0:1776178306.339408:0:20759:0:(service.c:2324:ptlrpc_server_handle_request()) Handling RPC req@000000004547bc6e pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_010:b50963bd-8eba-4432-9a54-8888d00e3fa1+8:2272:x1862456979115584:12345-192.168.204.3@tcp:400:kworker/u8:3.0 00000100:00100000:2.0:1776178306.339494:0:20759:0:(service.c:2372:ptlrpc_server_handle_request()) Handled RPC req@000000004547bc6e pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_010:b50963bd-8eba-4432-9a54-8888d00e3fa1+8:2272:x1862456979115584:12345-192.168.204.3@tcp:400:kworker/u8:3.0 Request processed in 85us (252us total) trans 0 rc 0/0 00000100:00100000:2.0:1776178306.339503:0:20759:0:(nrs_fifo.c:241:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.204.3@tcp, seq: 591 00000100:00100000:1.0:1776178306.339765:0:20137:0:(service.c:2152:ptlrpc_server_handle_req_in()) got req x1862456979115392 00000100:00100000:1.0:1776178306.339797:0:20137:0:(nrs_fifo.c:179:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.204.3@tcp, seq: 195 00000100:00100000:1.0:1776178306.339801:0:20137:0:(service.c:2324:ptlrpc_server_handle_request()) Handling RPC req@00000000f240c829 pname:cluuid+ref:pid:xid:nid:opc:job ll_mgs_0000:5b5c66b1-1a2f-4651-8548-0c75b9a38a94+9:2272:x1862456979115392:12345-192.168.204.3@tcp:400:kworker/u8:3.0 00000100:00100000:1.0:1776178306.339827:0:20137:0:(service.c:2372:ptlrpc_server_handle_request()) Handled RPC req@00000000f240c829 pname:cluuid+ref:pid:xid:nid:opc:job ll_mgs_0000:5b5c66b1-1a2f-4651-8548-0c75b9a38a94+9:2272:x1862456979115392:12345-192.168.204.3@tcp:400:kworker/u8:3.0 Request processed in 26us (617us total) trans 0 rc 0/0 00000100:00100000:1.0:1776178306.339833:0:20137:0:(nrs_fifo.c:241:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.204.3@tcp, seq: 195 00000100:00100000:0.0:1776178306.345282:0:3346:0:(events.c:361:request_in_callback()) peer: 12345-192.168.204.3@tcp (source: 12345-192.168.204.3@tcp) 00000100:00100000:2.0:1776178306.347601:0:20762:0:(service.c:2152:ptlrpc_server_handle_req_in()) got req x1862456979115648 00000100:00100000:2.0:1776178306.347682:0:20762:0:(nrs_fifo.c:179:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.204.3@tcp, seq: 592 00000100:00100000:2.0:1776178306.347689:0:20762:0:(service.c:2324:ptlrpc_server_handle_request()) Handling RPC req@0000000028dd2f9a pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_013:b50963bd-8eba-4432-9a54-8888d00e3fa1+9:2272:x1862456979115648:12345-192.168.204.3@tcp:400:kworker/u8:3.0 00000100:00100000:2.0:1776178306.347787:0:20762:0:(service.c:2372:ptlrpc_server_handle_request()) Handled RPC req@0000000028dd2f9a pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_013:b50963bd-8eba-4432-9a54-8888d00e3fa1+9:2272:x1862456979115648:12345-192.168.204.3@tcp:400:kworker/u8:3.0 Request processed in 97us (2512us total) trans 0 rc 0/0 00000100:00100000:2.0:1776178306.347795:0:20762:0:(nrs_fifo.c:241:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.204.3@tcp, seq: 592 00000100:00100000:3.0:1776178306.350897:0:11281:0:(service.c:2152:ptlrpc_server_handle_req_in()) got req x1862456979115456 00000100:00100000:1.0:1776178306.351078:0:8250:0:(service.c:2152:ptlrpc_server_handle_req_in()) got req x1862456979115520 00000100:00100000:1.0:1776178306.351130:0:8250:0:(nrs_fifo.c:179:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.204.3@tcp, seq: 988 00000100:00100000:1.0:1776178306.351137:0:8250:0:(service.c:2324:ptlrpc_server_handle_request()) Handling RPC req@00000000ead84f6b pname:cluuid+ref:pid:xid:nid:opc:job mdt00_003:b50963bd-8eba-4432-9a54-8888d00e3fa1+6:2272:x1862456979115520:12345-192.168.204.3@tcp:400:kworker/u8:3.0 00000100:00100000:1.0:1776178306.351233:0:8250:0:(service.c:2372:ptlrpc_server_handle_request()) Handled RPC req@00000000ead84f6b pname:cluuid+ref:pid:xid:nid:opc:job mdt00_003:b50963bd-8eba-4432-9a54-8888d00e3fa1+6:2272:x1862456979115520:12345-192.168.204.3@tcp:400:kworker/u8:3.0 Request processed in 96us (11955us total) trans 0 rc 0/0 00000100:00100000:1.0:1776178306.351241:0:8250:0:(nrs_fifo.c:241:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.204.3@tcp, seq: 988 00000100:00100000:1.0:1776178306.351257:0:8250:0:(nrs_fifo.c:179:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.204.3@tcp, seq: 6704 00000100:00100000:1.0:1776178306.351261:0:8250:0:(service.c:2324:ptlrpc_server_handle_request()) Handling RPC req@00000000314f395b pname:cluuid+ref:pid:xid:nid:opc:job mdt00_003:0+-99:2269:x1862456979115456:12345-192.168.204.3@tcp:38:kworker/u8:3.0 00010000:02000000:1.0:1776178306.351274:0:8250:0:(ldlm_lib.c:1139:target_handle_connect()) lustre-MDT0000: Not available for connect from 192.168.204.3@tcp (stopping) 00000100:00100000:1.0:1776178306.351300:0:8250:0:(service.c:2372:ptlrpc_server_handle_request()) Handled RPC req@00000000314f395b pname:cluuid+ref:pid:xid:nid:opc:job mdt00_003:0+-99:2269:x1862456979115456:12345-192.168.204.3@tcp:38:kworker/u8:3.0 Request processed in 37us (12293us total) trans 0 rc -19/-19 00000100:00100000:1.0:1776178306.351317:0:8250:0:(nrs_fifo.c:241:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.204.3@tcp, seq: 6704 00010000:00010000:2.0:1776178306.531437:0:9225:0:(ldlm_lockd.c:2792:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00010000:00010000:0.0:1776178307.552883:0:6244:0:(ldlm_lockd.c:2792:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:3.0:1776178307.937180:0:3353:0:(client.c:1733:ptlrpc_send_new_req()) Sending RPC req@000000000354b8ca pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_00:ac61f839-0e8e-4cac-9c59-08bcfbbad330:3353:1862456998063616:0@lo:400:kworker/u8:2.0 00000100:00100000:3.0:1776178307.937258:0:3353:0:(events.c:361:request_in_callback()) peer: 12345-0@lo (source: 12345-0@lo) 00000100:00100000:1.0:1776178307.937321:0:20140:0:(service.c:2152:ptlrpc_server_handle_req_in()) got req x1862456998063616 00000100:00080000:0.0:1776178307.937322:0:37:0:(pinger.c:251:ptlrpc_pinger_process_import()) lustre-MDT0000-lwp-OST0000_UUID->lustre-MDT0000_UUID: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00080000:0.0:1776178307.937328:0:37:0:(recover.c:57:ptlrpc_initiate_recovery()) lustre-MDT0000_UUID: starting recovery 00000100:00080000:0.0:1776178307.937332:0:37:0:(import.c:86:import_set_state_nolock()) 0000000047601a98 lustre-MDT0000_UUID: changing import state from DISCONN to CONNECTING 00000100:00080000:0.0:1776178307.937337:0:37:0:(import.c:534:import_select_connection()) lustre-MDT0000-lwp-OST0000: connect to NID 0@lo last attempt 1010 00000100:00080000:0.0:1776178307.937339:0:37:0:(import.c:581:import_select_connection()) lustre-MDT0000-lwp-OST0000: tried all connections, increasing latency to 21s 00000100:00080000:0.0:1776178307.937345:0:37:0:(import.c:615:import_select_connection()) lustre-MDT0000-lwp-OST0000: import 0000000047601a98 using connection 192.168.204.103@tcp/0@lo 00000100:00100000:0.0:1776178307.937367:0:37:0:(import.c:807:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5) req@00000000fdb309c4 x1862456998063680/t0(0) o38->lustre-MDT0000-lwp-OST0000@0@lo:12/10 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' 00000100:00100000:1.0:1776178307.937374:0:20140:0:(nrs_fifo.c:179:nrs_fifo_req_get()) NRS start fifo request from 12345-0@lo, seq: 196 00000100:00100000:1.0:1776178307.937384:0:20140:0:(service.c:2324:ptlrpc_server_handle_request()) Handling RPC req@0000000091f9a08d pname:cluuid+ref:pid:xid:nid:opc:job ll_mgs_0002:ac61f839-0e8e-4cac-9c59-08bcfbbad330+18:3353:x1862456998063616:12345-0@lo:400:kworker/u8:2.0 00000100:00100000:2.0:1776178307.937413:0:3354:0:(client.c:1733:ptlrpc_send_new_req()) Sending RPC req@00000000b5990455 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_01:lustre-MDT0001-lwp-OST0000_UUID:3354:1862456998063744:0@lo:400:kworker/u8:2.0 00000100:00100000:2.0:1776178307.937456:0:3354:0:(events.c:361:request_in_callback()) peer: 12345-0@lo (source: 12345-0@lo) 00000100:00100000:2.0:1776178307.937493:0:3352:0:(client.c:1733:ptlrpc_send_new_req()) Sending RPC req@00000000fdb309c4 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:lustre-MDT0000-lwp-OST0000_UUID:3352:1862456998063680:0@lo:38:kworker/u8:2.0 00000100:00100000:2.0:1776178307.937517:0:3352:0:(events.c:361:request_in_callback()) peer: 12345-0@lo (source: 12345-0@lo) 00000100:00100000:2.0:1776178307.937569:0:11281:0:(service.c:2152:ptlrpc_server_handle_req_in()) got req x1862456998063744 00000100:00100000:2.0:1776178307.937598:0:11281:0:(service.c:2152:ptlrpc_server_handle_req_in()) got req x1862456998063680 00000100:00100000:2.0:1776178307.937615:0:11281:0:(nrs_fifo.c:179:nrs_fifo_req_get()) NRS start fifo request from 12345-0@lo, seq: 989 00000100:00100000:2.0:1776178307.937619:0:11281:0:(service.c:2324:ptlrpc_server_handle_request()) Handling RPC req@000000009a62f69c pname:cluuid+ref:pid:xid:nid:opc:job mdt00_004:lustre-MDT0001-lwp-OST0000_UUID+5:3354:x1862456998063744:12345-0@lo:400:kworker/u8:2.0 00000100:00100000:2.0:1776178307.937655:0:11281:0:(service.c:2372:ptlrpc_server_handle_request()) Handled RPC req@000000009a62f69c pname:cluuid+ref:pid:xid:nid:opc:job mdt00_004:lustre-MDT0001-lwp-OST0000_UUID+5:3354:x1862456998063744:12345-0@lo:400:kworker/u8:2.0 Request processed in 35us (200us total) trans 0 rc 0/0 00000100:00100000:2.0:1776178307.937662:0:11281:0:(nrs_fifo.c:241:nrs_fifo_req_stop()) NRS stop fifo request from 12345-0@lo, seq: 989 00000100:00100000:2.0:1776178307.937679:0:11281:0:(nrs_fifo.c:179:nrs_fifo_req_get()) NRS start fifo request from 12345-0@lo, seq: 6705 00000100:00100000:2.0:1776178307.937682:0:11281:0:(service.c:2324:ptlrpc_server_handle_request()) Handling RPC req@00000000c237605d pname:cluuid+ref:pid:xid:nid:opc:job mdt00_004:0+-99:3352:x1862456998063680:12345-0@lo:38:kworker/u8:2.0 00010000:02000000:2.0:1776178307.937695:0:11281:0:(ldlm_lib.c:1139:target_handle_connect()) lustre-MDT0000: Not available for connect from 0@lo (stopping) 00000100:00100000:2.0:1776178307.937746:0:11281:0:(service.c:2372:ptlrpc_server_handle_request()) Handled RPC req@00000000c237605d pname:cluuid+ref:pid:xid:nid:opc:job mdt00_004:0+-99:3352:x1862456998063680:12345-0@lo:38:kworker/u8:2.0 Request processed in 62us (228us total) trans 0 rc -19/-19 00000100:00100000:2.0:1776178307.937752:0:11281:0:(nrs_fifo.c:241:nrs_fifo_req_stop()) NRS stop fifo request from 12345-0@lo, seq: 6705 00000100:00080000:2.0:1776178307.937786:0:3352:0:(import.c:86:import_set_state_nolock()) 0000000047601a98 lustre-MDT0000_UUID: changing import state from CONNECTING to DISCONN 00000100:00080000:2.0:1776178307.937790:0:3352:0:(import.c:1429:ptlrpc_connect_interpret()) recovery of lustre-MDT0000_UUID on 192.168.204.103@tcp failed (-19) 00000100:00100000:2.0:1776178307.937794:0:3352:0:(client.c:2215:ptlrpc_check_set()) Completed RPC req@00000000fdb309c4 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:lustre-MDT0000-lwp-OST0000_UUID:3352:1862456998063680:0@lo:38:kworker/u8:2.0 00000100:00100000:2.0:1776178307.937821:0:3354:0:(client.c:2215:ptlrpc_check_set()) Completed RPC req@00000000b5990455 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_01:lustre-MDT0001-lwp-OST0000_UUID:3354:1862456998063744:0@lo:400:kworker/u8:2.0 00000100:00080000:0.0:1776178307.938594:0:37:0:(pinger.c:251:ptlrpc_pinger_process_import()) lustre-MDT0000-lwp-OST0001_UUID->lustre-MDT0000_UUID: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00080000:0.0:1776178307.938599:0:37:0:(recover.c:57:ptlrpc_initiate_recovery()) lustre-MDT0000_UUID: starting recovery 00000100:00080000:0.0:1776178307.938602:0:37:0:(import.c:86:import_set_state_nolock()) 00000000dfc8db13 lustre-MDT0000_UUID: changing import state from DISCONN to CONNECTING 00000100:00080000:0.0:1776178307.938608:0:37:0:(import.c:534:import_select_connection()) lustre-MDT0000-lwp-OST0001: connect to NID 0@lo last attempt 1010 00000100:00080000:0.0:1776178307.938610:0:37:0:(import.c:581:import_select_connection()) lustre-MDT0000-lwp-OST0001: tried all connections, increasing latency to 21s 00000100:00080000:0.0:1776178307.938615:0:37:0:(import.c:615:import_select_connection()) lustre-MDT0000-lwp-OST0001: import 00000000dfc8db13 using connection 192.168.204.103@tcp/0@lo 00000100:00100000:0.0:1776178307.938637:0:37:0:(import.c:807:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5) req@00000000df769e31 x1862456998063808/t0(0) o38->lustre-MDT0000-lwp-OST0001@0@lo:12/10 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' 00000100:00080000:0.0:1776178307.938685:0:37:0:(pinger.c:251:ptlrpc_pinger_process_import()) lustre-MDT0001-mdtlov_UUID->lustre-MDT0000_UUID: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00080000:0.0:1776178307.938688:0:37:0:(recover.c:57:ptlrpc_initiate_recovery()) lustre-MDT0000_UUID: starting recovery 00000100:00080000:0.0:1776178307.938690:0:37:0:(import.c:86:import_set_state_nolock()) 00000000ba4f1821 lustre-MDT0000_UUID: changing import state from DISCONN to CONNECTING 00000100:00080000:0.0:1776178307.938693:0:37:0:(import.c:534:import_select_connection()) lustre-MDT0000-osp-MDT0001: connect to NID 0@lo last attempt 1010 00000100:00080000:0.0:1776178307.938695:0:37:0:(import.c:581:import_select_connection()) lustre-MDT0000-osp-MDT0001: tried all connections, increasing latency to 21s 00000100:00080000:0.0:1776178307.938698:0:37:0:(import.c:615:import_select_connection()) lustre-MDT0000-osp-MDT0001: import 00000000ba4f1821 using connection 192.168.204.103@tcp/0@lo 00000100:00100000:0.0:1776178307.938709:0:37:0:(import.c:807:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5) req@00000000ad37fc5d x1862456998063936/t0(0) o38->lustre-MDT0000-osp-MDT0001@0@lo:24/4 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' 00000100:00080000:0.0:1776178307.938763:0:37:0:(pinger.c:251:ptlrpc_pinger_process_import()) lustre-MDT0000-lwp-MDT0001_UUID->lustre-MDT0000_UUID: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00080000:0.0:1776178307.938766:0:37:0:(recover.c:57:ptlrpc_initiate_recovery()) lustre-MDT0000_UUID: starting recovery 00000100:00080000:0.0:1776178307.938768:0:37:0:(import.c:86:import_set_state_nolock()) 00000000a7c55b1f lustre-MDT0000_UUID: changing import state from DISCONN to CONNECTING 00000100:00080000:0.0:1776178307.938771:0:37:0:(import.c:534:import_select_connection()) lustre-MDT0000-lwp-MDT0001: connect to NID 0@lo last attempt 1010 00000100:00080000:0.0:1776178307.938773:0:37:0:(import.c:581:import_select_connection()) lustre-MDT0000-lwp-MDT0001: tried all connections, increasing latency to 21s 00000100:00080000:0.0:1776178307.938776:0:37:0:(import.c:615:import_select_connection()) lustre-MDT0000-lwp-MDT0001: import 00000000a7c55b1f using connection 192.168.204.103@tcp/0@lo 00000100:00100000:0.0:1776178307.938791:0:37:0:(import.c:807:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5) req@0000000050bb92d0 x1862456998064128/t0(0) o38->lustre-MDT0000-lwp-MDT0001@0@lo:12/10 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' 00000100:00100000:0.0:1776178307.938821:0:3354:0:(ptlrpcd.c:415:ptlrpcd_check()) transfer 1 async RPCs [0->1] 00000100:00100000:0.0:1776178307.938832:0:3354:0:(client.c:1733:ptlrpc_send_new_req()) Sending RPC req@000000001f88d899 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_01:lustre-MDT0001-mdtlov_UUID:3354:1862456998064064:0@lo:400:kworker/u8:2.0 00000100:00100000:0.0:1776178307.938877:0:3354:0:(events.c:361:request_in_callback()) peer: 12345-0@lo (source: 12345-0@lo) 00000100:00100000:0.0:1776178307.939267:0:20759:0:(service.c:2152:ptlrpc_server_handle_req_in()) got req x1862456998064064 00000100:00100000:0.0:1776178307.939307:0:20759:0:(nrs_fifo.c:179:nrs_fifo_req_get()) NRS start fifo request from 12345-0@lo, seq: 593 00000100:00100000:0.0:1776178307.939312:0:20759:0:(service.c:2324:ptlrpc_server_handle_request()) Handling RPC req@00000000b5990455 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_010:lustre-MDT0001-mdtlov_UUID+5:3354:x1862456998064064:12345-0@lo:400:kworker/u8:2.0 00000100:00100000:0.0:1776178307.939353:0:20759:0:(service.c:2372:ptlrpc_server_handle_request()) Handled RPC req@00000000b5990455 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_010:lustre-MDT0001-mdtlov_UUID+5:3354:x1862456998064064:12345-0@lo:400:kworker/u8:2.0 Request processed in 41us (477us total) trans 0 rc 0/0 00000100:00100000:0.0:1776178307.939359:0:20759:0:(nrs_fifo.c:241:nrs_fifo_req_stop()) NRS stop fifo request from 12345-0@lo, seq: 593 00000100:00100000:0.0:1776178307.939398:0:3354:0:(client.c:2215:ptlrpc_check_set()) Completed RPC req@000000001f88d899 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_01:lustre-MDT0001-mdtlov_UUID:3354:1862456998064064:0@lo:400:kworker/u8:2.0 00000100:00100000:0.0:1776178307.939430:0:3355:0:(client.c:1733:ptlrpc_send_new_req()) Sending RPC req@00000000c8bb8784 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_02:lustre-MDT0001-lwp-OST0001_UUID:3355:1862456998063872:0@lo:400:kworker/u8:2.0 00000100:00100000:0.0:1776178307.939455:0:3355:0:(events.c:361:request_in_callback()) peer: 12345-0@lo (source: 12345-0@lo) 00000100:00100000:0.0:1776178307.939471:0:3355:0:(ptlrpcd.c:415:ptlrpcd_check()) transfer 1 async RPCs [3->2] 00000100:00100000:0.0:1776178307.939476:0:3355:0:(client.c:1733:ptlrpc_send_new_req()) Sending RPC req@00000000033734be pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_02:lustre-MDT0001-mdtlov_UUID:3355:1862456998064000:0@lo:400:kworker/u8:2.0 00000100:00100000:0.0:1776178307.939497:0:3355:0:(events.c:361:request_in_callback()) peer: 12345-0@lo (source: 12345-0@lo) 00000100:00100000:0.0:1776178307.939516:0:20762:0:(service.c:2152:ptlrpc_server_handle_req_in()) got req x1862456998064000 00000100:00100000:0.0:1776178307.939538:0:20762:0:(nrs_fifo.c:179:nrs_fifo_req_get()) NRS start fifo request from 12345-0@lo, seq: 594 00000100:00100000:0.0:1776178307.939541:0:20762:0:(service.c:2324:ptlrpc_server_handle_request()) Handling RPC req@00000000fdb309c4 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_013:lustre-MDT0001-mdtlov_UUID+5:3355:x1862456998064000:12345-0@lo:400:kworker/u8:2.0 00000100:00100000:0.0:1776178307.939574:0:20762:0:(service.c:2372:ptlrpc_server_handle_request()) Handled RPC req@00000000fdb309c4 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_013:lustre-MDT0001-mdtlov_UUID+5:3355:x1862456998064000:12345-0@lo:400:kworker/u8:2.0 Request processed in 32us (78us total) trans 0 rc 0/0 00000100:00100000:0.0:1776178307.939579:0:20762:0:(nrs_fifo.c:241:nrs_fifo_req_stop()) NRS stop fifo request from 12345-0@lo, seq: 594 00000100:00100000:0.0:1776178307.939600:0:3355:0:(client.c:2215:ptlrpc_check_set()) Completed RPC req@00000000033734be pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_02:lustre-MDT0001-mdtlov_UUID:3355:1862456998064000:0@lo:400:kworker/u8:2.0 00000100:00100000:2.0:1776178307.939741:0:6257:0:(service.c:2152:ptlrpc_server_handle_req_in()) got req x1862456998063872 00000100:00100000:2.0:1776178307.939778:0:6257:0:(nrs_fifo.c:179:nrs_fifo_req_get()) NRS start fifo request from 12345-0@lo, seq: 990 00000100:00100000:2.0:1776178307.939782:0:6257:0:(service.c:2324:ptlrpc_server_handle_request()) Handling RPC req@000000001f88d899 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_002:lustre-MDT0001-lwp-OST0001_UUID+5:3355:x1862456998063872:12345-0@lo:400:kworker/u8:2.0 00000100:00100000:2.0:1776178307.939822:0:6257:0:(service.c:2372:ptlrpc_server_handle_request()) Handled RPC req@000000001f88d899 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_002:lustre-MDT0001-lwp-OST0001_UUID+5:3355:x1862456998063872:12345-0@lo:400:kworker/u8:2.0 Request processed in 39us (366us total) trans 0 rc 0/0 00000100:00100000:2.0:1776178307.939828:0:6257:0:(nrs_fifo.c:241:nrs_fifo_req_stop()) NRS stop fifo request from 12345-0@lo, seq: 990 00000100:00100000:0.0:1776178307.939856:0:3355:0:(client.c:2215:ptlrpc_check_set()) Completed RPC req@00000000c8bb8784 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_02:lustre-MDT0001-lwp-OST0001_UUID:3355:1862456998063872:0@lo:400:kworker/u8:2.0 00000100:00100000:2.0:1776178307.939872:0:3352:0:(client.c:1733:ptlrpc_send_new_req()) Sending RPC req@00000000df769e31 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:lustre-MDT0000-lwp-OST0001_UUID:3352:1862456998063808:0@lo:38:kworker/u8:2.0 00000100:00100000:2.0:1776178307.939899:0:3352:0:(events.c:361:request_in_callback()) peer: 12345-0@lo (source: 12345-0@lo) 00000100:00100000:2.0:1776178307.939923:0:3352:0:(client.c:1733:ptlrpc_send_new_req()) Sending RPC req@00000000ad37fc5d pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:lustre-MDT0001-mdtlov_UUID:3352:1862456998063936:0@lo:38:kworker/u8:2.0 00000100:00100000:2.0:1776178307.939945:0:3352:0:(events.c:361:request_in_callback()) peer: 12345-0@lo (source: 12345-0@lo) 00000100:00100000:2.0:1776178307.939962:0:3352:0:(client.c:1733:ptlrpc_send_new_req()) Sending RPC req@0000000050bb92d0 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:lustre-MDT0000-lwp-MDT0001_UUID:3352:1862456998064128:0@lo:38:kworker/u8:2.0 00000100:00100000:2.0:1776178307.939981:0:3352:0:(events.c:361:request_in_callback()) peer: 12345-0@lo (source: 12345-0@lo) 00000100:00100000:0.0:1776178307.940012:0:6257:0:(service.c:2152:ptlrpc_server_handle_req_in()) got req x1862456998063808 00000100:00100000:2.0:1776178307.940016:0:14128:0:(service.c:2152:ptlrpc_server_handle_req_in()) got req x1862456998063936 00000100:00100000:0.0:1776178307.940035:0:6257:0:(service.c:2152:ptlrpc_server_handle_req_in()) got req x1862456998064128 00000100:00100000:2.0:1776178307.940045:0:14128:0:(nrs_fifo.c:179:nrs_fifo_req_get()) NRS start fifo request from 12345-0@lo, seq: 728 00000100:00100000:2.0:1776178307.940050:0:14128:0:(service.c:2324:ptlrpc_server_handle_request()) Handling RPC req@00000000f97fd38e pname:cluuid+ref:pid:xid:nid:opc:job mdt_out00_002:0+-99:3352:x1862456998063936:12345-0@lo:38:kworker/u8:2.0 00010000:02000000:2.0:1776178307.940064:0:14128:0:(ldlm_lib.c:1139:target_handle_connect()) lustre-MDT0000: Not available for connect from 0@lo (stopping) 00000100:00100000:0.0:1776178307.940077:0:6257:0:(nrs_fifo.c:179:nrs_fifo_req_get()) NRS start fifo request from 12345-0@lo, seq: 6706 00000100:00100000:0.0:1776178307.940083:0:6257:0:(service.c:2324:ptlrpc_server_handle_request()) Handling RPC req@00000000034781c2 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_002:0+-99:3352:x1862456998063808:12345-0@lo:38:kworker/u8:2.0 00010000:02000000:0.0:1776178307.940092:0:6257:0:(ldlm_lib.c:1139:target_handle_connect()) lustre-MDT0000: Not available for connect from 0@lo (stopping) 00000100:00100000:2.0:1776178307.940133:0:14128:0:(service.c:2372:ptlrpc_server_handle_request()) Handled RPC req@00000000f97fd38e pname:cluuid+ref:pid:xid:nid:opc:job mdt_out00_002:0+-99:3352:x1862456998063936:12345-0@lo:38:kworker/u8:2.0 Request processed in 81us (186us total) trans 0 rc -19/-19 00000100:00100000:0.0:1776178307.940137:0:6257:0:(service.c:2372:ptlrpc_server_handle_request()) Handled RPC req@00000000034781c2 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_002:0+-99:3352:x1862456998063808:12345-0@lo:38:kworker/u8:2.0 Request processed in 51us (236us total) trans 0 rc -19/-19 00000100:00100000:2.0:1776178307.940145:0:14128:0:(nrs_fifo.c:241:nrs_fifo_req_stop()) NRS stop fifo request from 12345-0@lo, seq: 728 00000100:00100000:0.0:1776178307.940148:0:6257:0:(nrs_fifo.c:241:nrs_fifo_req_stop()) NRS stop fifo request from 12345-0@lo, seq: 6706 00000100:00100000:0.0:1776178307.940165:0:6257:0:(nrs_fifo.c:179:nrs_fifo_req_get()) NRS start fifo request from 12345-0@lo, seq: 6707 00000100:00100000:0.0:1776178307.940169:0:6257:0:(service.c:2324:ptlrpc_server_handle_request()) Handling RPC req@000000000bdf54a0 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_002:0+-99:3352:x1862456998064128:12345-0@lo:38:kworker/u8:2.0 00010000:02000000:0.0:1776178307.940176:0:6257:0:(ldlm_lib.c:1139:target_handle_connect()) lustre-MDT0000: Not available for connect from 0@lo (stopping) 00000100:00080000:2.0:1776178307.940191:0:3352:0:(import.c:86:import_set_state_nolock()) 00000000dfc8db13 lustre-MDT0000_UUID: changing import state from CONNECTING to DISCONN 00000100:00080000:2.0:1776178307.940196:0:3352:0:(import.c:1429:ptlrpc_connect_interpret()) recovery of lustre-MDT0000_UUID on 192.168.204.103@tcp failed (-19) 00000100:00100000:2.0:1776178307.940201:0:3352:0:(client.c:2215:ptlrpc_check_set()) Completed RPC req@00000000df769e31 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:lustre-MDT0000-lwp-OST0001_UUID:3352:1862456998063808:0@lo:38:kworker/u8:2.0 00000100:00100000:0.0:1776178307.940204:0:6257:0:(service.c:2372:ptlrpc_server_handle_request()) Handled RPC req@000000000bdf54a0 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_002:0+-99:3352:x1862456998064128:12345-0@lo:38:kworker/u8:2.0 Request processed in 32us (220us total) trans 0 rc -19/-19 00000100:00100000:0.0:1776178307.940212:0:6257:0:(nrs_fifo.c:241:nrs_fifo_req_stop()) NRS stop fifo request from 12345-0@lo, seq: 6707 00000100:00080000:2.0:1776178307.940217:0:3352:0:(import.c:86:import_set_state_nolock()) 00000000ba4f1821 lustre-MDT0000_UUID: changing import state from CONNECTING to DISCONN 00000100:00080000:2.0:1776178307.940221:0:3352:0:(import.c:1429:ptlrpc_connect_interpret()) recovery of lustre-MDT0000_UUID on 192.168.204.103@tcp failed (-19) 00000100:00100000:2.0:1776178307.940226:0:3352:0:(client.c:2215:ptlrpc_check_set()) Completed RPC req@00000000ad37fc5d pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:lustre-MDT0001-mdtlov_UUID:3352:1862456998063936:0@lo:38:kworker/u8:2.0 00000100:00080000:2.0:1776178307.940236:0:3352:0:(import.c:86:import_set_state_nolock()) 00000000a7c55b1f lustre-MDT0000_UUID: changing import state from CONNECTING to DISCONN 00000100:00080000:2.0:1776178307.940239:0:3352:0:(import.c:1429:ptlrpc_connect_interpret()) recovery of lustre-MDT0000_UUID on 192.168.204.103@tcp failed (-19) 00000100:00100000:2.0:1776178307.940242:0:3352:0:(client.c:2215:ptlrpc_check_set()) Completed RPC req@0000000050bb92d0 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:lustre-MDT0000-lwp-MDT0001_UUID:3352:1862456998064128:0@lo:38:kworker/u8:2.0 00000100:00100000:1.0:1776178307.969712:0:20140:0:(service.c:2372:ptlrpc_server_handle_request()) Handled RPC req@0000000091f9a08d pname:cluuid+ref:pid:xid:nid:opc:job ll_mgs_0002:ac61f839-0e8e-4cac-9c59-08bcfbbad330+18:3353:x1862456998063616:12345-0@lo:400:kworker/u8:2.0 Request processed in 32320us (32447us total) trans 0 rc 0/0 00000100:00100000:1.0:1776178307.969741:0:20140:0:(nrs_fifo.c:241:nrs_fifo_req_stop()) NRS stop fifo request from 12345-0@lo, seq: 196 00000100:00100000:3.0:1776178307.974585:0:3353:0:(client.c:2215:ptlrpc_check_set()) Completed RPC req@000000000354b8ca pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_00:ac61f839-0e8e-4cac-9c59-08bcfbbad330:3353:1862456998063616:0@lo:400:kworker/u8:2.0 00010000:00010000:1.0:1776178308.576438:0:6245:0:(ldlm_lockd.c:2792:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:3.0:1776178309.472408:0:3353:0:(ptlrpcd.c:415:ptlrpcd_check()) transfer 1 async RPCs [1->0] 00000100:00100000:3.0:1776178309.472439:0:3353:0:(client.c:1733:ptlrpc_send_new_req()) Sending RPC req@00000000e82039bb pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_00:lustre-MDT0001-mdtlov_UUID:3353:1862456998064192:0@lo:13:osp-pre-1-1.0 00000100:00100000:3.0:1776178309.472530:0:3353:0:(events.c:361:request_in_callback()) peer: 12345-0@lo (source: 12345-0@lo) 00000100:00100000:2.0:1776178309.472624:0:9554:0:(service.c:2152:ptlrpc_server_handle_req_in()) got req x1862456998064192 00000100:00100000:2.0:1776178309.472685:0:9554:0:(nrs_fifo.c:179:nrs_fifo_req_get()) NRS start fifo request from 12345-0@lo, seq: 529 00000100:00100000:2.0:1776178309.472702:0:9554:0:(service.c:2324:ptlrpc_server_handle_request()) Handling RPC req@000000009b5eeec2 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_create00:lustre-MDT0001-mdtlov_UUID+5:3353:x1862456998064192:12345-0@lo:13:osp-pre-1-1.0 00002000:00000024:2.0:1776178309.472740:0:9554:0:(ofd_obd.c:740:ofd_statfs()) blocks cached 229376 granted 491746368 pending 0 free 3923689472 avail 3692167168 00000100:00100000:2.0:1776178309.472832:0:9554:0:(service.c:2372:ptlrpc_server_handle_request()) Handled RPC req@000000009b5eeec2 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_create00:lustre-MDT0001-mdtlov_UUID+5:3353:x1862456998064192:12345-0@lo:13:osp-pre-1-1.0 Request processed in 129us (303us total) trans 0 rc 0/0 00000100:00100000:2.0:1776178309.472840:0:9554:0:(nrs_fifo.c:241:nrs_fifo_req_stop()) NRS stop fifo request from 12345-0@lo, seq: 529 00000100:00100000:3.0:1776178309.472950:0:3353:0:(client.c:2215:ptlrpc_check_set()) Completed RPC req@00000000e82039bb pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_00:lustre-MDT0001-mdtlov_UUID:3353:1862456998064192:0@lo:13:osp-pre-1-1.0 00010000:00010000:2.0:1776178309.608243:0:9224:0:(ldlm_lockd.c:2792:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:3.0:1776178309.984504:0:3356:0:(ptlrpcd.c:415:ptlrpcd_check()) transfer 1 async RPCs [2->3] 00000100:00100000:3.0:1776178309.984547:0:3356:0:(client.c:1733:ptlrpc_send_new_req()) Sending RPC req@00000000c8bb8784 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_03:lustre-MDT0001-mdtlov_UUID:3356:1862456998064256:0@lo:13:osp-pre-0-1.0 00000100:00100000:3.0:1776178309.984641:0:3356:0:(events.c:361:request_in_callback()) peer: 12345-0@lo (source: 12345-0@lo) 00000100:00100000:1.0:1776178309.985009:0:8231:0:(service.c:2152:ptlrpc_server_handle_req_in()) got req x1862456998064256 00000100:00100000:1.0:1776178309.985161:0:8231:0:(nrs_fifo.c:179:nrs_fifo_req_get()) NRS start fifo request from 12345-0@lo, seq: 530 00000100:00100000:1.0:1776178309.985169:0:8231:0:(service.c:2324:ptlrpc_server_handle_request()) Handling RPC req@000000008c469ec0 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_create00:lustre-MDT0001-mdtlov_UUID+5:3356:x1862456998064256:12345-0@lo:13:osp-pre-0-1.0 00002000:00000024:1.0:1776178309.985219:0:8231:0:(ofd_obd.c:740:ofd_statfs()) blocks cached 315392 granted 484883776 pending 0 free 3923689472 avail 3692167168 00000100:00100000:1.0:1776178309.985278:0:8231:0:(service.c:2372:ptlrpc_server_handle_request()) Handled RPC req@000000008c469ec0 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_create00:lustre-MDT0001-mdtlov_UUID+5:3356:x1862456998064256:12345-0@lo:13:osp-pre-0-1.0 Request processed in 110us (639us total) trans 0 rc 0/0 00000100:00100000:1.0:1776178309.985286:0:8231:0:(nrs_fifo.c:241:nrs_fifo_req_stop()) NRS stop fifo request from 12345-0@lo, seq: 530 00000100:00100000:1.0:1776178309.985359:0:3356:0:(client.c:2215:ptlrpc_check_set()) Completed RPC req@00000000c8bb8784 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_03:lustre-MDT0001-mdtlov_UUID:3356:1862456998064256:0@lo:13:osp-pre-0-1.0 00010000:00010000:2.0:1776178310.624934:0:9225:0:(ldlm_lockd.c:2792:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:0.0:1776178311.462956:0:3346:0:(events.c:361:request_in_callback()) peer: 12345-192.168.204.3@tcp (source: 12345-192.168.204.3@tcp) 00000100:00100000:0.0:1776178311.463006:0:3346:0:(events.c:361:request_in_callback()) peer: 12345-192.168.204.3@tcp (source: 12345-192.168.204.3@tcp) 00000100:00100000:0.0:1776178311.463045:0:3346:0:(events.c:361:request_in_callback()) peer: 12345-192.168.204.3@tcp (source: 12345-192.168.204.3@tcp) 00000100:00100000:0.0:1776178311.463126:0:3346:0:(events.c:361:request_in_callback()) peer: 12345-192.168.204.3@tcp (source: 12345-192.168.204.3@tcp) 00000100:00100000:0.0:1776178311.463153:0:3346:0:(events.c:361:request_in_callback()) peer: 12345-192.168.204.3@tcp (source: 12345-192.168.204.3@tcp) 00000100:00100000:0.0:1776178311.463188:0:6257:0:(service.c:2152:ptlrpc_server_handle_req_in()) got req x1862456979115840 00000100:00100000:0.0:1776178311.463239:0:6257:0:(service.c:2152:ptlrpc_server_handle_req_in()) got req x1862456979115776 00000100:00100000:0.0:1776178311.463258:0:6257:0:(nrs_fifo.c:179:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.204.3@tcp, seq: 991 00000100:00100000:0.0:1776178311.463264:0:6257:0:(service.c:2324:ptlrpc_server_handle_request()) Handling RPC req@00000000033734be pname:cluuid+ref:pid:xid:nid:opc:job mdt00_002:b50963bd-8eba-4432-9a54-8888d00e3fa1+6:2270:x1862456979115840:12345-192.168.204.3@tcp:400:kworker/u8:4.0 00000100:00100000:0.0:1776178311.463336:0:6257:0:(service.c:2372:ptlrpc_server_handle_request()) Handled RPC req@00000000033734be pname:cluuid+ref:pid:xid:nid:opc:job mdt00_002:b50963bd-8eba-4432-9a54-8888d00e3fa1+6:2270:x1862456979115840:12345-192.168.204.3@tcp:400:kworker/u8:4.0 Request processed in 70us (329us total) trans 0 rc 0/0 00000100:00100000:0.0:1776178311.463344:0:6257:0:(nrs_fifo.c:241:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.204.3@tcp, seq: 991 00000100:00100000:0.0:1776178311.463362:0:6257:0:(nrs_fifo.c:179:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.204.3@tcp, seq: 6708 00000100:00100000:0.0:1776178311.463366:0:6257:0:(service.c:2324:ptlrpc_server_handle_request()) Handling RPC req@00000000a0db51c1 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_002:0+-99:2269:x1862456979115776:12345-192.168.204.3@tcp:38:kworker/u8:4.0 00010000:02000000:0.0:1776178311.463393:0:6257:0:(ldlm_lib.c:1139:target_handle_connect()) lustre-MDT0000: Not available for connect from 192.168.204.3@tcp (stopping) 00000100:00100000:1.0:1776178311.463532:0:20140:0:(service.c:2152:ptlrpc_server_handle_req_in()) got req x1862456979115712 00000100:00100000:1.0:1776178311.463559:0:20140:0:(nrs_fifo.c:179:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.204.3@tcp, seq: 197 00000100:00100000:1.0:1776178311.463563:0:20140:0:(service.c:2324:ptlrpc_server_handle_request()) Handling RPC req@00000000c0367ec3 pname:cluuid+ref:pid:xid:nid:opc:job ll_mgs_0002:5b5c66b1-1a2f-4651-8548-0c75b9a38a94+9:2271:x1862456979115712:12345-192.168.204.3@tcp:400:kworker/u8:4.0 00000100:00100000:1.0:1776178311.463591:0:20140:0:(service.c:2372:ptlrpc_server_handle_request()) Handled RPC req@00000000c0367ec3 pname:cluuid+ref:pid:xid:nid:opc:job ll_mgs_0002:5b5c66b1-1a2f-4651-8548-0c75b9a38a94+9:2271:x1862456979115712:12345-192.168.204.3@tcp:400:kworker/u8:4.0 Request processed in 27us (646us total) trans 0 rc 0/0 00000100:00100000:1.0:1776178311.463597:0:20140:0:(nrs_fifo.c:241:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.204.3@tcp, seq: 197 00000100:00100000:1.0:1776178311.463664:0:20762:0:(service.c:2152:ptlrpc_server_handle_req_in()) got req x1862456979115968 00000100:00100000:1.0:1776178311.463688:0:20762:0:(service.c:2152:ptlrpc_server_handle_req_in()) got req x1862456979115904 00000100:00100000:1.0:1776178311.463707:0:20762:0:(nrs_fifo.c:179:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.204.3@tcp, seq: 595 00000100:00100000:1.0:1776178311.463710:0:20762:0:(service.c:2324:ptlrpc_server_handle_request()) Handling RPC req@00000000ad228ff6 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_013:b50963bd-8eba-4432-9a54-8888d00e3fa1+9:2270:x1862456979115968:12345-192.168.204.3@tcp:400:kworker/u8:4.0 00000100:00100000:1.0:1776178311.463736:0:20762:0:(service.c:2372:ptlrpc_server_handle_request()) Handled RPC req@00000000ad228ff6 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_013:b50963bd-8eba-4432-9a54-8888d00e3fa1+9:2270:x1862456979115968:12345-192.168.204.3@tcp:400:kworker/u8:4.0 Request processed in 24us (691us total) trans 0 rc 0/0 00000100:00100000:1.0:1776178311.463742:0:20762:0:(nrs_fifo.c:241:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.204.3@tcp, seq: 595 00000100:00100000:1.0:1776178311.463753:0:20762:0:(nrs_fifo.c:179:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.204.3@tcp, seq: 596 00000100:00100000:1.0:1776178311.463758:0:20762:0:(service.c:2324:ptlrpc_server_handle_request()) Handling RPC req@00000000c8bb8784 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_013:b50963bd-8eba-4432-9a54-8888d00e3fa1+8:2271:x1862456979115904:12345-192.168.204.3@tcp:400:kworker/u8:4.0 00000100:00100000:1.0:1776178311.463779:0:20762:0:(service.c:2372:ptlrpc_server_handle_request()) Handled RPC req@00000000c8bb8784 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_013:b50963bd-8eba-4432-9a54-8888d00e3fa1+8:2271:x1862456979115904:12345-192.168.204.3@tcp:400:kworker/u8:4.0 Request processed in 20us (626us total) trans 0 rc 0/0 00000100:00100000:1.0:1776178311.463785:0:20762:0:(nrs_fifo.c:241:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.204.3@tcp, seq: 596 00000100:00100000:0.0:1776178311.464197:0:6257:0:(service.c:2372:ptlrpc_server_handle_request()) Handled RPC req@00000000a0db51c1 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_002:0+-99:2269:x1862456979115776:12345-192.168.204.3@tcp:38:kworker/u8:4.0 Request processed in 828us (1069us total) trans 0 rc -19/-19 00000100:00100000:0.0:1776178311.464206:0:6257:0:(nrs_fifo.c:241:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.204.3@tcp, seq: 6708 00010000:00010000:0.0:1776178311.649855:0:6244:0:(ldlm_lockd.c:2792:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00010000:00010000:1.0:1776178312.672671:0:6245:0:(ldlm_lockd.c:2792:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00080000:1.0:1776178313.056548:0:36:0:(pinger.c:251:ptlrpc_pinger_process_import()) lustre-MDT0000-lwp-OST0000_UUID->lustre-MDT0000_UUID: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00080000:1.0:1776178313.056558:0:36:0:(recover.c:57:ptlrpc_initiate_recovery()) lustre-MDT0000_UUID: starting recovery 00000100:00080000:1.0:1776178313.056563:0:36:0:(import.c:86:import_set_state_nolock()) 0000000047601a98 lustre-MDT0000_UUID: changing import state from DISCONN to CONNECTING 00000100:00080000:1.0:1776178313.056582:0:36:0:(import.c:534:import_select_connection()) lustre-MDT0000-lwp-OST0000: connect to NID 0@lo last attempt 1015 00000100:00080000:1.0:1776178313.056587:0:36:0:(import.c:581:import_select_connection()) lustre-MDT0000-lwp-OST0000: tried all connections, increasing latency to 21s 00000100:00100000:2.0:1776178313.056591:0:3355:0:(ptlrpcd.c:415:ptlrpcd_check()) transfer 1 async RPCs [3->2] 00000100:00080000:1.0:1776178313.056596:0:36:0:(import.c:615:import_select_connection()) lustre-MDT0000-lwp-OST0000: import 0000000047601a98 using connection 192.168.204.103@tcp/0@lo 00000100:00100000:2.0:1776178313.056619:0:3355:0:(client.c:1733:ptlrpc_send_new_req()) Sending RPC req@00000000a0f2a112 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_02:ac61f839-0e8e-4cac-9c59-08bcfbbad330:3355:1862456998064320:0@lo:400:kworker/u8:1.0 00000100:00100000:1.0:1776178313.056635:0:36:0:(import.c:807:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5) req@000000008245b0ee x1862456998064384/t0(0) o38->lustre-MDT0000-lwp-OST0000@0@lo:12/10 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' 00000100:00080000:1.0:1776178313.056685:0:36:0:(pinger.c:251:ptlrpc_pinger_process_import()) lustre-MDT0000-lwp-OST0001_UUID->lustre-MDT0000_UUID: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00080000:1.0:1776178313.056688:0:36:0:(recover.c:57:ptlrpc_initiate_recovery()) lustre-MDT0000_UUID: starting recovery 00000100:00080000:1.0:1776178313.056690:0:36:0:(import.c:86:import_set_state_nolock()) 00000000dfc8db13 lustre-MDT0000_UUID: changing import state from DISCONN to CONNECTING 00000100:00080000:1.0:1776178313.056693:0:36:0:(import.c:534:import_select_connection()) lustre-MDT0000-lwp-OST0001: connect to NID 0@lo last attempt 1015 00000100:00080000:1.0:1776178313.056695:0:36:0:(import.c:581:import_select_connection()) lustre-MDT0000-lwp-OST0001: tried all connections, increasing latency to 21s 00000100:00080000:1.0:1776178313.056699:0:36:0:(import.c:615:import_select_connection()) lustre-MDT0000-lwp-OST0001: import 00000000dfc8db13 using connection 192.168.204.103@tcp/0@lo 00000100:00100000:2.0:1776178313.056725:0:3355:0:(events.c:361:request_in_callback()) peer: 12345-0@lo (source: 12345-0@lo) 00000100:00100000:1.0:1776178313.056726:0:36:0:(import.c:807:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5) req@00000000a33eecf4 x1862456998064512/t0(0) o38->lustre-MDT0000-lwp-OST0001@0@lo:12/10 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' 00000100:00080000:1.0:1776178313.056744:0:36:0:(pinger.c:251:ptlrpc_pinger_process_import()) lustre-MDT0001-mdtlov_UUID->lustre-MDT0000_UUID: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00080000:1.0:1776178313.056747:0:36:0:(recover.c:57:ptlrpc_initiate_recovery()) lustre-MDT0000_UUID: starting recovery 00000100:00080000:1.0:1776178313.056749:0:36:0:(import.c:86:import_set_state_nolock()) 00000000ba4f1821 lustre-MDT0000_UUID: changing import state from DISCONN to CONNECTING 00000100:00080000:1.0:1776178313.056753:0:36:0:(import.c:534:import_select_connection()) lustre-MDT0000-osp-MDT0001: connect to NID 0@lo last attempt 1015 00000100:00080000:1.0:1776178313.056755:0:36:0:(import.c:581:import_select_connection()) lustre-MDT0000-osp-MDT0001: tried all connections, increasing latency to 21s 00000100:00080000:1.0:1776178313.056759:0:36:0:(import.c:615:import_select_connection()) lustre-MDT0000-osp-MDT0001: import 00000000ba4f1821 using connection 192.168.204.103@tcp/0@lo 00000100:00100000:1.0:1776178313.056769:0:36:0:(import.c:807:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5) req@00000000d1e0f01c x1862456998064640/t0(0) o38->lustre-MDT0000-osp-MDT0001@0@lo:24/4 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' 00000100:00080000:1.0:1776178313.056795:0:36:0:(pinger.c:251:ptlrpc_pinger_process_import()) lustre-MDT0000-lwp-MDT0001_UUID->lustre-MDT0000_UUID: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00080000:1.0:1776178313.056799:0:36:0:(recover.c:57:ptlrpc_initiate_recovery()) lustre-MDT0000_UUID: starting recovery 00000100:00080000:1.0:1776178313.056801:0:36:0:(import.c:86:import_set_state_nolock()) 00000000a7c55b1f lustre-MDT0000_UUID: changing import state from DISCONN to CONNECTING 00000100:00080000:1.0:1776178313.056804:0:36:0:(import.c:534:import_select_connection()) lustre-MDT0000-lwp-MDT0001: connect to NID 0@lo last attempt 1015 00000100:00080000:1.0:1776178313.056806:0:36:0:(import.c:581:import_select_connection()) lustre-MDT0000-lwp-MDT0001: tried all connections, increasing latency to 21s 00000100:00080000:1.0:1776178313.056810:0:36:0:(import.c:615:import_select_connection()) lustre-MDT0000-lwp-MDT0001: import 00000000a7c55b1f using connection 192.168.204.103@tcp/0@lo 00000100:00100000:2.0:1776178313.056813:0:20137:0:(service.c:2152:ptlrpc_server_handle_req_in()) got req x1862456998064320 00000100:00100000:1.0:1776178313.056820:0:36:0:(import.c:807:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5) req@00000000e178f924 x1862456998064832/t0(0) o38->lustre-MDT0000-lwp-MDT0001@0@lo:12/10 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' 00000100:00100000:2.0:1776178313.056884:0:20137:0:(nrs_fifo.c:179:nrs_fifo_req_get()) NRS start fifo request from 12345-0@lo, seq: 198 00000100:00100000:2.0:1776178313.056891:0:20137:0:(service.c:2324:ptlrpc_server_handle_request()) Handling RPC req@00000000ecf3e0a2 pname:cluuid+ref:pid:xid:nid:opc:job ll_mgs_0000:ac61f839-0e8e-4cac-9c59-08bcfbbad330+18:3355:x1862456998064320:12345-0@lo:400:kworker/u8:1.0 00000100:00100000:2.0:1776178313.056938:0:20137:0:(service.c:2372:ptlrpc_server_handle_request()) Handled RPC req@00000000ecf3e0a2 pname:cluuid+ref:pid:xid:nid:opc:job ll_mgs_0000:ac61f839-0e8e-4cac-9c59-08bcfbbad330+18:3355:x1862456998064320:12345-0@lo:400:kworker/u8:1.0 Request processed in 47us (216us total) trans 0 rc 0/0 00000100:00100000:2.0:1776178313.056946:0:20137:0:(nrs_fifo.c:241:nrs_fifo_req_stop()) NRS stop fifo request from 12345-0@lo, seq: 198 00000100:00100000:2.0:1776178313.056978:0:3355:0:(client.c:1733:ptlrpc_send_new_req()) Sending RPC req@00000000865e4f65 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_02:lustre-MDT0001-mdtlov_UUID:3355:1862456998064704:0@lo:400:kworker/u8:1.0 00000100:00100000:2.0:1776178313.057004:0:3355:0:(events.c:361:request_in_callback()) peer: 12345-0@lo (source: 12345-0@lo) 00000100:00100000:2.0:1776178313.057030:0:3355:0:(client.c:2215:ptlrpc_check_set()) Completed RPC req@00000000a0f2a112 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_02:ac61f839-0e8e-4cac-9c59-08bcfbbad330:3355:1862456998064320:0@lo:400:kworker/u8:1.0 00000100:00100000:2.0:1776178313.057054:0:3355:0:(ptlrpcd.c:415:ptlrpcd_check()) transfer 1 async RPCs [3->2] 00000100:00100000:0.0:1776178313.057091:0:3352:0:(client.c:1733:ptlrpc_send_new_req()) Sending RPC req@000000008245b0ee pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:lustre-MDT0000-lwp-OST0000_UUID:3352:1862456998064384:0@lo:38:kworker/u8:1.0 00000100:00100000:2.0:1776178313.057092:0:3355:0:(client.c:1733:ptlrpc_send_new_req()) Sending RPC req@00000000a66f2317 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_02:lustre-MDT0001-mdtlov_UUID:3355:1862456998064768:0@lo:400:kworker/u8:1.0 00000100:00100000:2.0:1776178313.057123:0:3355:0:(events.c:361:request_in_callback()) peer: 12345-0@lo (source: 12345-0@lo) 00000100:00100000:0.0:1776178313.057142:0:3352:0:(events.c:361:request_in_callback()) peer: 12345-0@lo (source: 12345-0@lo) 00000100:00100000:2.0:1776178313.057166:0:3354:0:(client.c:1733:ptlrpc_send_new_req()) Sending RPC req@00000000bacf691a pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_01:lustre-MDT0001-lwp-OST0001_UUID:3354:1862456998064576:0@lo:400:kworker/u8:1.0 00000100:00100000:0.0:1776178313.057167:0:3352:0:(client.c:1733:ptlrpc_send_new_req()) Sending RPC req@00000000a33eecf4 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:lustre-MDT0000-lwp-OST0001_UUID:3352:1862456998064512:0@lo:38:kworker/u8:1.0 00000100:00100000:2.0:1776178313.057205:0:3354:0:(events.c:361:request_in_callback()) peer: 12345-0@lo (source: 12345-0@lo) 00000100:00100000:2.0:1776178313.057228:0:3354:0:(ptlrpcd.c:415:ptlrpcd_check()) transfer 1 async RPCs [0->1] 00000100:00100000:2.0:1776178313.057234:0:3354:0:(client.c:1733:ptlrpc_send_new_req()) Sending RPC req@00000000597c3414 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_01:lustre-MDT0001-lwp-OST0000_UUID:3354:1862456998064448:0@lo:400:kworker/u8:1.0 00000100:00100000:0.0:1776178313.057242:0:3352:0:(events.c:361:request_in_callback()) peer: 12345-0@lo (source: 12345-0@lo) 00000100:00100000:2.0:1776178313.057269:0:6256:0:(service.c:2152:ptlrpc_server_handle_req_in()) got req x1862456998064384 00000100:00100000:2.0:1776178313.057293:0:6256:0:(service.c:2152:ptlrpc_server_handle_req_in()) got req x1862456998064576 00000100:00100000:3.0:1776178313.057367:0:14440:0:(service.c:2152:ptlrpc_server_handle_req_in()) got req x1862456998064704 00000100:00100000:3.0:1776178313.057395:0:14440:0:(service.c:2152:ptlrpc_server_handle_req_in()) got req x1862456998064768 00000100:00100000:3.0:1776178313.057411:0:14440:0:(nrs_fifo.c:179:nrs_fifo_req_get()) NRS start fifo request from 12345-0@lo, seq: 597 00000100:00100000:3.0:1776178313.057416:0:14440:0:(service.c:2324:ptlrpc_server_handle_request()) Handling RPC req@000000007309261c pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_004:lustre-MDT0001-mdtlov_UUID+5:3355:x1862456998064704:12345-0@lo:400:kworker/u8:1.0 00000100:00100000:3.0:1776178313.058528:0:14440:0:(service.c:2372:ptlrpc_server_handle_request()) Handled RPC req@000000007309261c pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_004:lustre-MDT0001-mdtlov_UUID+5:3355:x1862456998064704:12345-0@lo:400:kworker/u8:1.0 Request processed in 1108us (1521us total) trans 0 rc 0/0 00000100:00100000:3.0:1776178313.058537:0:14440:0:(nrs_fifo.c:241:nrs_fifo_req_stop()) NRS stop fifo request from 12345-0@lo, seq: 597 00000100:00100000:3.0:1776178313.058555:0:14440:0:(nrs_fifo.c:179:nrs_fifo_req_get()) NRS start fifo request from 12345-0@lo, seq: 598 00000100:00100000:3.0:1776178313.058559:0:14440:0:(service.c:2324:ptlrpc_server_handle_request()) Handling RPC req@000000001ee743a7 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_004:lustre-MDT0001-mdtlov_UUID+5:3355:x1862456998064768:12345-0@lo:400:kworker/u8:1.0 00000100:00100000:2.0:1776178313.058771:0:6256:0:(service.c:2152:ptlrpc_server_handle_req_in()) got req x1862456998064512 00000100:00100000:3.0:1776178313.059546:0:14440:0:(service.c:2372:ptlrpc_server_handle_request()) Handled RPC req@000000001ee743a7 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_004:lustre-MDT0001-mdtlov_UUID+5:3355:x1862456998064768:12345-0@lo:400:kworker/u8:1.0 Request processed in 983us (2420us total) trans 0 rc 0/0 00000100:00100000:3.0:1776178313.059556:0:14440:0:(nrs_fifo.c:241:nrs_fifo_req_stop()) NRS stop fifo request from 12345-0@lo, seq: 598 00000100:00100000:1.0:1776178313.063390:0:8250:0:(nrs_fifo.c:179:nrs_fifo_req_get()) NRS start fifo request from 12345-0@lo, seq: 992 00000100:00100000:3.0:1776178313.063478:0:3355:0:(client.c:2215:ptlrpc_check_set()) Completed RPC req@00000000a66f2317 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_02:lustre-MDT0001-mdtlov_UUID:3355:1862456998064768:0@lo:400:kworker/u8:1.0 00000100:00100000:2.0:1776178313.063505:0:6256:0:(nrs_fifo.c:179:nrs_fifo_req_get()) NRS start fifo request from 12345-0@lo, seq: 6709 00000100:00100000:1.0:1776178313.063506:0:8250:0:(service.c:2324:ptlrpc_server_handle_request()) Handling RPC req@00000000c37c187e pname:cluuid+ref:pid:xid:nid:opc:job mdt00_003:lustre-MDT0001-lwp-OST0001_UUID+5:3354:x1862456998064576:12345-0@lo:400:kworker/u8:1.0 00000100:00100000:2.0:1776178313.063513:0:6256:0:(service.c:2324:ptlrpc_server_handle_request()) Handling RPC req@000000000354b8ca pname:cluuid+ref:pid:xid:nid:opc:job mdt00_001:0+-99:3352:x1862456998064384:12345-0@lo:38:kworker/u8:1.0 00000100:00100000:1.0:1776178313.063604:0:8250:0:(service.c:2372:ptlrpc_server_handle_request()) Handled RPC req@00000000c37c187e pname:cluuid+ref:pid:xid:nid:opc:job mdt00_003:lustre-MDT0001-lwp-OST0001_UUID+5:3354:x1862456998064576:12345-0@lo:400:kworker/u8:1.0 Request processed in 99us (6400us total) trans 0 rc 0/0 00000100:00100000:1.0:1776178313.063612:0:8250:0:(nrs_fifo.c:241:nrs_fifo_req_stop()) NRS stop fifo request from 12345-0@lo, seq: 992 00010000:02000000:2.0:1776178313.063644:0:6256:0:(ldlm_lib.c:1139:target_handle_connect()) lustre-MDT0000: Not available for connect from 0@lo (stopping) 00000100:00100000:1.0:1776178313.063647:0:8250:0:(nrs_fifo.c:179:nrs_fifo_req_get()) NRS start fifo request from 12345-0@lo, seq: 6710 00000100:00100000:1.0:1776178313.063652:0:8250:0:(service.c:2324:ptlrpc_server_handle_request()) Handling RPC req@0000000050bb92d0 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_003:0+-99:3352:x1862456998064512:12345-0@lo:38:kworker/u8:1.0 00010000:02000000:1.0:1776178313.063660:0:8250:0:(ldlm_lib.c:1139:target_handle_connect()) lustre-MDT0000: Not available for connect from 0@lo (stopping) 00000100:00100000:1.0:1776178313.063690:0:8250:0:(service.c:2372:ptlrpc_server_handle_request()) Handled RPC req@0000000050bb92d0 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_003:0+-99:3352:x1862456998064512:12345-0@lo:38:kworker/u8:1.0 Request processed in 37us (6449us total) trans 0 rc -19/-19 00000100:00100000:1.0:1776178313.063696:0:8250:0:(nrs_fifo.c:241:nrs_fifo_req_stop()) NRS stop fifo request from 12345-0@lo, seq: 6710 00000100:00100000:1.0:1776178313.063761:0:3354:0:(events.c:361:request_in_callback()) peer: 12345-0@lo (source: 12345-0@lo) 00000100:00100000:1.0:1776178313.063793:0:3354:0:(client.c:2215:ptlrpc_check_set()) Completed RPC req@00000000bacf691a pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_01:lustre-MDT0001-lwp-OST0001_UUID:3354:1862456998064576:0@lo:400:kworker/u8:1.0 00000100:00100000:1.0:1776178313.063830:0:11281:0:(service.c:2152:ptlrpc_server_handle_req_in()) got req x1862456998064448 00000100:00100000:1.0:1776178313.063859:0:11281:0:(nrs_fifo.c:179:nrs_fifo_req_get()) NRS start fifo request from 12345-0@lo, seq: 993 00000100:00100000:1.0:1776178313.063863:0:11281:0:(service.c:2324:ptlrpc_server_handle_request()) Handling RPC req@00000000a0f2a112 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_004:lustre-MDT0001-lwp-OST0000_UUID+5:3354:x1862456998064448:12345-0@lo:400:kworker/u8:1.0 00000100:00100000:1.0:1776178313.063896:0:11281:0:(service.c:2372:ptlrpc_server_handle_request()) Handled RPC req@00000000a0f2a112 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_004:lustre-MDT0001-lwp-OST0000_UUID+5:3354:x1862456998064448:12345-0@lo:400:kworker/u8:1.0 Request processed in 33us (135us total) trans 0 rc 0/0 00000100:00100000:1.0:1776178313.063902:0:11281:0:(nrs_fifo.c:241:nrs_fifo_req_stop()) NRS stop fifo request from 12345-0@lo, seq: 993 00000100:00100000:2.0:1776178313.063924:0:6256:0:(service.c:2372:ptlrpc_server_handle_request()) Handled RPC req@000000000354b8ca pname:cluuid+ref:pid:xid:nid:opc:job mdt00_001:0+-99:3352:x1862456998064384:12345-0@lo:38:kworker/u8:1.0 Request processed in 410us (6783us total) trans 0 rc -19/-19 00000100:00100000:2.0:1776178313.063931:0:6256:0:(nrs_fifo.c:241:nrs_fifo_req_stop()) NRS stop fifo request from 12345-0@lo, seq: 6709 00000100:00100000:1.0:1776178313.063940:0:3354:0:(client.c:2215:ptlrpc_check_set()) Completed RPC req@00000000597c3414 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_01:lustre-MDT0001-lwp-OST0000_UUID:3354:1862456998064448:0@lo:400:kworker/u8:1.0 00000100:00100000:3.0:1776178313.064253:0:3355:0:(client.c:2215:ptlrpc_check_set()) Completed RPC req@00000000865e4f65 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_02:lustre-MDT0001-mdtlov_UUID:3355:1862456998064704:0@lo:400:kworker/u8:1.0 00000100:00080000:0.0:1776178313.068847:0:3352:0:(import.c:86:import_set_state_nolock()) 00000000dfc8db13 lustre-MDT0000_UUID: changing import state from CONNECTING to DISCONN 00000100:00080000:0.0:1776178313.068859:0:3352:0:(import.c:1429:ptlrpc_connect_interpret()) recovery of lustre-MDT0000_UUID on 192.168.204.103@tcp failed (-19) 00000100:00100000:0.0:1776178313.068866:0:3352:0:(client.c:2215:ptlrpc_check_set()) Completed RPC req@00000000a33eecf4 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:lustre-MDT0000-lwp-OST0001_UUID:3352:1862456998064512:0@lo:38:kworker/u8:1.0 00000100:00100000:0.0:1776178313.068882:0:3352:0:(client.c:1733:ptlrpc_send_new_req()) Sending RPC req@00000000d1e0f01c pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:lustre-MDT0001-mdtlov_UUID:3352:1862456998064640:0@lo:38:kworker/u8:1.0 00000100:00100000:0.0:1776178313.070845:0:3352:0:(events.c:361:request_in_callback()) peer: 12345-0@lo (source: 12345-0@lo) 00000100:00100000:0.0:1776178313.070888:0:3352:0:(client.c:1733:ptlrpc_send_new_req()) Sending RPC req@00000000e178f924 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:lustre-MDT0000-lwp-MDT0001_UUID:3352:1862456998064832:0@lo:38:kworker/u8:1.0 00000100:00100000:0.0:1776178313.070934:0:3352:0:(events.c:361:request_in_callback()) peer: 12345-0@lo (source: 12345-0@lo) 00000100:00100000:2.0:1776178313.071144:0:15213:0:(service.c:2152:ptlrpc_server_handle_req_in()) got req x1862456998064640 00000100:00100000:2.0:1776178313.071183:0:15213:0:(nrs_fifo.c:179:nrs_fifo_req_get()) NRS start fifo request from 12345-0@lo, seq: 729 00000100:00100000:2.0:1776178313.071189:0:15213:0:(service.c:2324:ptlrpc_server_handle_request()) Handling RPC req@00000000ad37fc5d pname:cluuid+ref:pid:xid:nid:opc:job mdt_out00_003:0+-99:3352:x1862456998064640:12345-0@lo:38:kworker/u8:1.0 00010000:02000000:2.0:1776178313.071205:0:15213:0:(ldlm_lib.c:1139:target_handle_connect()) lustre-MDT0000: Not available for connect from 0@lo (stopping) 00000100:00080000:0.0:1776178313.071267:0:3352:0:(import.c:86:import_set_state_nolock()) 0000000047601a98 lustre-MDT0000_UUID: changing import state from CONNECTING to DISCONN 00000100:00080000:0.0:1776178313.071271:0:3352:0:(import.c:1429:ptlrpc_connect_interpret()) recovery of lustre-MDT0000_UUID on 192.168.204.103@tcp failed (-19) 00000100:00100000:0.0:1776178313.071275:0:3352:0:(client.c:2215:ptlrpc_check_set()) Completed RPC req@000000008245b0ee pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:lustre-MDT0000-lwp-OST0000_UUID:3352:1862456998064384:0@lo:38:kworker/u8:1.0 00000100:00100000:3.0:1776178313.073535:0:6257:0:(service.c:2152:ptlrpc_server_handle_req_in()) got req x1862456998064832 00000100:00100000:3.0:1776178313.073855:0:6257:0:(nrs_fifo.c:179:nrs_fifo_req_get()) NRS start fifo request from 12345-0@lo, seq: 6711 00000100:00100000:3.0:1776178313.073862:0:6257:0:(service.c:2324:ptlrpc_server_handle_request()) Handling RPC req@00000000df769e31 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_002:0+-99:3352:x1862456998064832:12345-0@lo:38:kworker/u8:1.0 00010000:02000000:3.0:1776178313.073881:0:6257:0:(ldlm_lib.c:1139:target_handle_connect()) lustre-MDT0000: Not available for connect from 0@lo (stopping) 00000100:00100000:2.0:1776178313.074440:0:15213:0:(service.c:2372:ptlrpc_server_handle_request()) Handled RPC req@00000000ad37fc5d pname:cluuid+ref:pid:xid:nid:opc:job mdt_out00_003:0+-99:3352:x1862456998064640:12345-0@lo:38:kworker/u8:1.0 Request processed in 3248us (3599us total) trans 0 rc -19/-19 00000100:00100000:2.0:1776178313.074449:0:15213:0:(nrs_fifo.c:241:nrs_fifo_req_stop()) NRS stop fifo request from 12345-0@lo, seq: 729 00000100:00080000:0.0:1776178313.074547:0:3352:0:(import.c:86:import_set_state_nolock()) 00000000ba4f1821 lustre-MDT0000_UUID: changing import state from CONNECTING to DISCONN 00000100:00080000:0.0:1776178313.074552:0:3352:0:(import.c:1429:ptlrpc_connect_interpret()) recovery of lustre-MDT0000_UUID on 192.168.204.103@tcp failed (-19) 00000100:00100000:0.0:1776178313.074557:0:3352:0:(client.c:2215:ptlrpc_check_set()) Completed RPC req@00000000d1e0f01c pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:lustre-MDT0001-mdtlov_UUID:3352:1862456998064640:0@lo:38:kworker/u8:1.0 00000100:00100000:3.0:1776178313.079417:0:6257:0:(service.c:2372:ptlrpc_server_handle_request()) Handled RPC req@00000000df769e31 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_002:0+-99:3352:x1862456998064832:12345-0@lo:38:kworker/u8:1.0 Request processed in 5539us (8467us total) trans 0 rc -19/-19 00000100:00100000:3.0:1776178313.079435:0:6257:0:(nrs_fifo.c:241:nrs_fifo_req_stop()) NRS stop fifo request from 12345-0@lo, seq: 6711 00000100:00080000:0.0:1776178313.080938:0:3352:0:(import.c:86:import_set_state_nolock()) 00000000a7c55b1f lustre-MDT0000_UUID: changing import state from CONNECTING to DISCONN 00000100:00080000:0.0:1776178313.080949:0:3352:0:(import.c:1429:ptlrpc_connect_interpret()) recovery of lustre-MDT0000_UUID on 192.168.204.103@tcp failed (-19) 00000100:00100000:0.0:1776178313.080957:0:3352:0:(client.c:2215:ptlrpc_check_set()) Completed RPC req@00000000e178f924 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:lustre-MDT0000-lwp-MDT0001_UUID:3352:1862456998064832:0@lo:38:kworker/u8:1.0 00010000:00010000:2.0:1776178313.696855:0:9224:0:(ldlm_lockd.c:2792:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:3.0:1776178314.593918:0:3353:0:(client.c:1733:ptlrpc_send_new_req()) Sending RPC req@000000008b978a8a pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_00:lustre-MDT0001-mdtlov_UUID:3353:1862456998064896:0@lo:13:osp-pre-1-1.0 00000100:00100000:3.0:1776178314.593985:0:3353:0:(events.c:361:request_in_callback()) peer: 12345-0@lo (source: 12345-0@lo) 00000100:00100000:0.0:1776178314.594699:0:9554:0:(service.c:2152:ptlrpc_server_handle_req_in()) got req x1862456998064896 00000100:00100000:0.0:1776178314.594747:0:9554:0:(nrs_fifo.c:179:nrs_fifo_req_get()) NRS start fifo request from 12345-0@lo, seq: 531 00000100:00100000:0.0:1776178314.594753:0:9554:0:(service.c:2324:ptlrpc_server_handle_request()) Handling RPC req@00000000ddc1dc02 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_create00:lustre-MDT0001-mdtlov_UUID+5:3353:x1862456998064896:12345-0@lo:13:osp-pre-1-1.0 00002000:00000024:0.0:1776178314.594783:0:9554:0:(ofd_obd.c:740:ofd_statfs()) blocks cached 229376 granted 491746368 pending 0 free 3923689472 avail 3692167168 00000100:00100000:0.0:1776178314.594841:0:9554:0:(service.c:2372:ptlrpc_server_handle_request()) Handled RPC req@00000000ddc1dc02 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_create00:lustre-MDT0001-mdtlov_UUID+5:3353:x1862456998064896:12345-0@lo:13:osp-pre-1-1.0 Request processed in 88us (857us total) trans 0 rc 0/0 00000100:00100000:0.0:1776178314.594848:0:9554:0:(nrs_fifo.c:241:nrs_fifo_req_stop()) NRS stop fifo request from 12345-0@lo, seq: 531 00000100:00100000:3.0:1776178314.595107:0:3353:0:(client.c:2215:ptlrpc_check_set()) Completed RPC req@000000008b978a8a pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_00:lustre-MDT0001-mdtlov_UUID:3353:1862456998064896:0@lo:13:osp-pre-1-1.0 00010000:00010000:2.0:1776178314.720546:0:9225:0:(ldlm_lockd.c:2792:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:3.0:1776178315.104494:0:3353:0:(ptlrpcd.c:415:ptlrpcd_check()) transfer 1 async RPCs [1->0] 00000100:00100000:3.0:1776178315.104517:0:3353:0:(client.c:1733:ptlrpc_send_new_req()) Sending RPC req@000000001f977915 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_00:lustre-MDT0001-mdtlov_UUID:3353:1862456998064960:0@lo:13:osp-pre-0-1.0 00000100:00100000:3.0:1776178315.104613:0:3353:0:(events.c:361:request_in_callback()) peer: 12345-0@lo (source: 12345-0@lo) 00000100:00100000:0.0:1776178315.104797:0:8231:0:(service.c:2152:ptlrpc_server_handle_req_in()) got req x1862456998064960 00000100:00100000:0.0:1776178315.104851:0:8231:0:(nrs_fifo.c:179:nrs_fifo_req_get()) NRS start fifo request from 12345-0@lo, seq: 532 00000100:00100000:0.0:1776178315.104856:0:8231:0:(service.c:2324:ptlrpc_server_handle_request()) Handling RPC req@000000009c55a259 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_create00:lustre-MDT0001-mdtlov_UUID+5:3353:x1862456998064960:12345-0@lo:13:osp-pre-0-1.0 00002000:00000024:0.0:1776178315.104896:0:8231:0:(ofd_obd.c:740:ofd_statfs()) blocks cached 315392 granted 484883776 pending 0 free 3923689472 avail 3692167168 00000100:00100000:0.0:1776178315.104948:0:8231:0:(service.c:2372:ptlrpc_server_handle_request()) Handled RPC req@000000009c55a259 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_create00:lustre-MDT0001-mdtlov_UUID+5:3353:x1862456998064960:12345-0@lo:13:osp-pre-0-1.0 Request processed in 91us (337us total) trans 0 rc 0/0 00000100:00100000:0.0:1776178315.104956:0:8231:0:(nrs_fifo.c:241:nrs_fifo_req_stop()) NRS stop fifo request from 12345-0@lo, seq: 532 00000100:00100000:3.0:1776178315.105334:0:3353:0:(client.c:2215:ptlrpc_check_set()) Completed RPC req@000000001f977915 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_00:lustre-MDT0001-mdtlov_UUID:3353:1862456998064960:0@lo:13:osp-pre-0-1.0 00010000:00010000:0.0:1776178315.744830:0:6244:0:(ldlm_lockd.c:2792:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:0.0:1776178316.586374:0:3346:0:(events.c:361:request_in_callback()) peer: 12345-192.168.204.3@tcp (source: 12345-192.168.204.3@tcp) 00000100:00100000:0.0:1776178316.586425:0:3346:0:(events.c:361:request_in_callback()) peer: 12345-192.168.204.3@tcp (source: 12345-192.168.204.3@tcp) 00000100:00100000:0.0:1776178316.586460:0:3346:0:(events.c:361:request_in_callback()) peer: 12345-192.168.204.3@tcp (source: 12345-192.168.204.3@tcp) 00000100:00100000:0.0:1776178316.586494:0:3346:0:(events.c:361:request_in_callback()) peer: 12345-192.168.204.3@tcp (source: 12345-192.168.204.3@tcp) 00000100:00100000:0.0:1776178316.586576:0:3346:0:(events.c:361:request_in_callback()) peer: 12345-192.168.204.3@tcp (source: 12345-192.168.204.3@tcp) 00000100:00100000:1.0:1776178316.586872:0:20762:0:(service.c:2152:ptlrpc_server_handle_req_in()) got req x1862456979116288 00000100:00100000:1.0:1776178316.586915:0:20762:0:(service.c:2152:ptlrpc_server_handle_req_in()) got req x1862456979116224 00000100:00100000:1.0:1776178316.586933:0:20762:0:(nrs_fifo.c:179:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.204.3@tcp, seq: 599 00000100:00100000:1.0:1776178316.586939:0:20762:0:(service.c:2324:ptlrpc_server_handle_request()) Handling RPC req@00000000dd4b6b2f pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_013:b50963bd-8eba-4432-9a54-8888d00e3fa1+9:2273:x1862456979116288:12345-192.168.204.3@tcp:400:kworker/u8:3.0 00000100:00100000:1.0:1776178316.587002:0:20762:0:(service.c:2372:ptlrpc_server_handle_request()) Handled RPC req@00000000dd4b6b2f pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_013:b50963bd-8eba-4432-9a54-8888d00e3fa1+9:2273:x1862456979116288:12345-192.168.204.3@tcp:400:kworker/u8:3.0 Request processed in 63us (577us total) trans 0 rc 0/0 00000100:00100000:1.0:1776178316.587009:0:20762:0:(nrs_fifo.c:241:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.204.3@tcp, seq: 599 00000100:00100000:1.0:1776178316.587025:0:20762:0:(nrs_fifo.c:179:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.204.3@tcp, seq: 600 00000100:00100000:1.0:1776178316.587029:0:20762:0:(service.c:2324:ptlrpc_server_handle_request()) Handling RPC req@00000000ec65370c pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_013:b50963bd-8eba-4432-9a54-8888d00e3fa1+8:2273:x1862456979116224:12345-192.168.204.3@tcp:400:kworker/u8:3.0 00000100:00100000:1.0:1776178316.587048:0:20762:0:(service.c:2372:ptlrpc_server_handle_request()) Handled RPC req@00000000ec65370c pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_013:b50963bd-8eba-4432-9a54-8888d00e3fa1+8:2273:x1862456979116224:12345-192.168.204.3@tcp:400:kworker/u8:3.0 Request processed in 18us (554us total) trans 0 rc 0/0 00000100:00100000:1.0:1776178316.587053:0:20762:0:(nrs_fifo.c:241:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.204.3@tcp, seq: 600 00000100:00100000:2.0:1776178316.587457:0:20140:0:(service.c:2152:ptlrpc_server_handle_req_in()) got req x1862456979116032 00000100:00100000:2.0:1776178316.587494:0:20140:0:(nrs_fifo.c:179:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.204.3@tcp, seq: 199 00000100:00100000:2.0:1776178316.587500:0:20140:0:(service.c:2324:ptlrpc_server_handle_request()) Handling RPC req@000000003960fbcb pname:cluuid+ref:pid:xid:nid:opc:job ll_mgs_0002:5b5c66b1-1a2f-4651-8548-0c75b9a38a94+9:2273:x1862456979116032:12345-192.168.204.3@tcp:400:kworker/u8:3.0 00000100:00100000:2.0:1776178316.587533:0:20140:0:(service.c:2372:ptlrpc_server_handle_request()) Handled RPC req@000000003960fbcb pname:cluuid+ref:pid:xid:nid:opc:job ll_mgs_0002:5b5c66b1-1a2f-4651-8548-0c75b9a38a94+9:2273:x1862456979116032:12345-192.168.204.3@tcp:400:kworker/u8:3.0 Request processed in 33us (1073us total) trans 0 rc 0/0 00000100:00100000:2.0:1776178316.587540:0:20140:0:(nrs_fifo.c:241:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.204.3@tcp, seq: 199 00000100:00100000:0.0:1776178316.587799:0:8250:0:(service.c:2152:ptlrpc_server_handle_req_in()) got req x1862456979116160 00000100:00100000:0.0:1776178316.587830:0:8250:0:(service.c:2152:ptlrpc_server_handle_req_in()) got req x1862456979116096 00000100:00100000:0.0:1776178316.587861:0:8250:0:(nrs_fifo.c:179:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.204.3@tcp, seq: 994 00000100:00100000:0.0:1776178316.587865:0:8250:0:(service.c:2324:ptlrpc_server_handle_request()) Handling RPC req@0000000007d49440 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_003:b50963bd-8eba-4432-9a54-8888d00e3fa1+6:2273:x1862456979116160:12345-192.168.204.3@tcp:400:kworker/u8:3.0 00000100:00100000:0.0:1776178316.587903:0:8250:0:(service.c:2372:ptlrpc_server_handle_request()) Handled RPC req@0000000007d49440 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_003:b50963bd-8eba-4432-9a54-8888d00e3fa1+6:2273:x1862456979116160:12345-192.168.204.3@tcp:400:kworker/u8:3.0 Request processed in 36us (1538us total) trans 0 rc 0/0 00000100:00100000:0.0:1776178316.587908:0:8250:0:(nrs_fifo.c:241:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.204.3@tcp, seq: 994 00000100:00100000:0.0:1776178316.587921:0:8250:0:(nrs_fifo.c:179:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.204.3@tcp, seq: 6712 00000100:00100000:0.0:1776178316.587924:0:8250:0:(service.c:2324:ptlrpc_server_handle_request()) Handling RPC req@00000000e787b61f pname:cluuid+ref:pid:xid:nid:opc:job mdt00_003:0+-99:2269:x1862456979116096:12345-192.168.204.3@tcp:38:kworker/u8:3.0 00010000:02000000:0.0:1776178316.587939:0:8250:0:(ldlm_lib.c:1139:target_handle_connect()) lustre-MDT0000: Not available for connect from 192.168.204.3@tcp (stopping) 00000100:00100000:0.0:1776178316.587965:0:8250:0:(service.c:2372:ptlrpc_server_handle_request()) Handled RPC req@00000000e787b61f pname:cluuid+ref:pid:xid:nid:opc:job mdt00_003:0+-99:2269:x1862456979116096:12345-192.168.204.3@tcp:38:kworker/u8:3.0 Request processed in 39us (1390us total) trans 0 rc -19/-19 00000100:00100000:0.0:1776178316.587971:0:8250:0:(nrs_fifo.c:241:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.204.3@tcp, seq: 6712 00010000:00010000:1.0:1776178316.768426:0:6245:0:(ldlm_lockd.c:2792:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00010000:00010000:2.0:1776178317.792321:0:9224:0:(ldlm_lockd.c:2792:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00080000:0.0:1776178318.176341:0:37:0:(pinger.c:251:ptlrpc_pinger_process_import()) lustre-MDT0000-lwp-OST0000_UUID->lustre-MDT0000_UUID: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00080000:0.0:1776178318.176350:0:37:0:(recover.c:57:ptlrpc_initiate_recovery()) lustre-MDT0000_UUID: starting recovery 00000100:00080000:0.0:1776178318.176353:0:37:0:(import.c:86:import_set_state_nolock()) 0000000047601a98 lustre-MDT0000_UUID: changing import state from DISCONN to CONNECTING 00000100:00080000:0.0:1776178318.176362:0:37:0:(import.c:534:import_select_connection()) lustre-MDT0000-lwp-OST0000: connect to NID 0@lo last attempt 1020 00000100:00080000:0.0:1776178318.176366:0:37:0:(import.c:581:import_select_connection()) lustre-MDT0000-lwp-OST0000: tried all connections, increasing latency to 21s 00000100:00080000:0.0:1776178318.176372:0:37:0:(import.c:615:import_select_connection()) lustre-MDT0000-lwp-OST0000: import 0000000047601a98 using connection 192.168.204.103@tcp/0@lo 00000100:00100000:0.0:1776178318.176404:0:37:0:(import.c:807:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5) req@000000004eda7977 x1862456998065088/t0(0) o38->lustre-MDT0000-lwp-OST0000@0@lo:12/10 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' 00000100:00080000:0.0:1776178318.176433:0:37:0:(pinger.c:251:ptlrpc_pinger_process_import()) lustre-MDT0000-lwp-OST0001_UUID->lustre-MDT0000_UUID: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00080000:0.0:1776178318.176436:0:37:0:(recover.c:57:ptlrpc_initiate_recovery()) lustre-MDT0000_UUID: starting recovery 00000100:00080000:0.0:1776178318.176437:0:37:0:(import.c:86:import_set_state_nolock()) 00000000dfc8db13 lustre-MDT0000_UUID: changing import state from DISCONN to CONNECTING 00000100:00080000:0.0:1776178318.176441:0:37:0:(import.c:534:import_select_connection()) lustre-MDT0000-lwp-OST0001: connect to NID 0@lo last attempt 1020 00000100:00080000:0.0:1776178318.176443:0:37:0:(import.c:581:import_select_connection()) lustre-MDT0000-lwp-OST0001: tried all connections, increasing latency to 21s 00000100:00080000:0.0:1776178318.176446:0:37:0:(import.c:615:import_select_connection()) lustre-MDT0000-lwp-OST0001: import 00000000dfc8db13 using connection 192.168.204.103@tcp/0@lo 00000100:00100000:0.0:1776178318.176466:0:37:0:(import.c:807:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5) req@0000000007fa878c x1862456998065216/t0(0) o38->lustre-MDT0000-lwp-OST0001@0@lo:12/10 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' 00000100:00080000:0.0:1776178318.176490:0:37:0:(pinger.c:251:ptlrpc_pinger_process_import()) lustre-MDT0001-mdtlov_UUID->lustre-MDT0000_UUID: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00080000:0.0:1776178318.176493:0:37:0:(recover.c:57:ptlrpc_initiate_recovery()) lustre-MDT0000_UUID: starting recovery 00000100:00080000:0.0:1776178318.176494:0:37:0:(import.c:86:import_set_state_nolock()) 00000000ba4f1821 lustre-MDT0000_UUID: changing import state from DISCONN to CONNECTING 00000100:00080000:0.0:1776178318.176497:0:37:0:(import.c:534:import_select_connection()) lustre-MDT0000-osp-MDT0001: connect to NID 0@lo last attempt 1020 00000100:00080000:0.0:1776178318.176499:0:37:0:(import.c:581:import_select_connection()) lustre-MDT0000-osp-MDT0001: tried all connections, increasing latency to 21s 00000100:00080000:0.0:1776178318.176502:0:37:0:(import.c:615:import_select_connection()) lustre-MDT0000-osp-MDT0001: import 00000000ba4f1821 using connection 192.168.204.103@tcp/0@lo 00000100:00100000:0.0:1776178318.176511:0:37:0:(import.c:807:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5) req@000000008c3fbdb6 x1862456998065344/t0(0) o38->lustre-MDT0000-osp-MDT0001@0@lo:24/4 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' 00000100:00080000:0.0:1776178318.176542:0:37:0:(pinger.c:251:ptlrpc_pinger_process_import()) lustre-MDT0000-lwp-MDT0001_UUID->lustre-MDT0000_UUID: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00080000:0.0:1776178318.176547:0:37:0:(recover.c:57:ptlrpc_initiate_recovery()) lustre-MDT0000_UUID: starting recovery 00000100:00080000:0.0:1776178318.176548:0:37:0:(import.c:86:import_set_state_nolock()) 00000000a7c55b1f lustre-MDT0000_UUID: changing import state from DISCONN to CONNECTING 00000100:00080000:0.0:1776178318.176551:0:37:0:(import.c:534:import_select_connection()) lustre-MDT0000-lwp-MDT0001: connect to NID 0@lo last attempt 1020 00000100:00080000:0.0:1776178318.176552:0:37:0:(import.c:581:import_select_connection()) lustre-MDT0000-lwp-MDT0001: tried all connections, increasing latency to 21s 00000100:00080000:0.0:1776178318.176555:0:37:0:(import.c:615:import_select_connection()) lustre-MDT0000-lwp-MDT0001: import 00000000a7c55b1f using connection 192.168.204.103@tcp/0@lo 00000100:00100000:0.0:1776178318.176564:0:37:0:(import.c:807:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5) req@0000000014250674 x1862456998065536/t0(0) o38->lustre-MDT0000-lwp-MDT0001@0@lo:12/10 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' 00000100:00100000:1.0:1776178318.176646:0:3354:0:(client.c:1733:ptlrpc_send_new_req()) Sending RPC req@0000000047c15ac0 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_01:lustre-MDT0001-mdtlov_UUID:3354:1862456998065408:0@lo:400:kworker/u8:2.0 00000100:00100000:1.0:1776178318.176733:0:3354:0:(events.c:361:request_in_callback()) peer: 12345-0@lo (source: 12345-0@lo) 00000100:00100000:1.0:1776178318.176780:0:3354:0:(ptlrpcd.c:415:ptlrpcd_check()) transfer 1 async RPCs [0->1] 00000100:00100000:1.0:1776178318.176786:0:3354:0:(client.c:1733:ptlrpc_send_new_req()) Sending RPC req@00000000c324c57b pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_01:lustre-MDT0001-lwp-OST0001_UUID:3354:1862456998065280:0@lo:400:kworker/u8:2.0 00000100:00100000:1.0:1776178318.176813:0:3354:0:(events.c:361:request_in_callback()) peer: 12345-0@lo (source: 12345-0@lo) 00000100:00100000:1.0:1776178318.176838:0:3356:0:(client.c:1733:ptlrpc_send_new_req()) Sending RPC req@0000000084c00ed0 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_03:lustre-MDT0001-lwp-OST0000_UUID:3356:1862456998065152:0@lo:400:kworker/u8:2.0 00000100:00100000:1.0:1776178318.176861:0:3356:0:(events.c:361:request_in_callback()) peer: 12345-0@lo (source: 12345-0@lo) 00000100:00100000:1.0:1776178318.176879:0:3356:0:(ptlrpcd.c:415:ptlrpcd_check()) transfer 2 async RPCs [2->3] 00000100:00100000:1.0:1776178318.176884:0:3356:0:(client.c:1733:ptlrpc_send_new_req()) Sending RPC req@00000000a2082164 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_03:ac61f839-0e8e-4cac-9c59-08bcfbbad330:3356:1862456998065024:0@lo:400:kworker/u8:2.0 00000100:00100000:1.0:1776178318.176912:0:3356:0:(events.c:361:request_in_callback()) peer: 12345-0@lo (source: 12345-0@lo) 00000100:00100000:1.0:1776178318.176927:0:3356:0:(client.c:1733:ptlrpc_send_new_req()) Sending RPC req@000000006342fd4b pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_03:lustre-MDT0001-mdtlov_UUID:3356:1862456998065472:0@lo:400:kworker/u8:2.0 00000100:00100000:1.0:1776178318.176948:0:3356:0:(events.c:361:request_in_callback()) peer: 12345-0@lo (source: 12345-0@lo) 00000100:00100000:1.0:1776178318.176986:0:6256:0:(service.c:2152:ptlrpc_server_handle_req_in()) got req x1862456998065280 00000100:00100000:1.0:1776178318.177037:0:6256:0:(service.c:2152:ptlrpc_server_handle_req_in()) got req x1862456998065152 00000100:00100000:1.0:1776178318.177056:0:6256:0:(nrs_fifo.c:179:nrs_fifo_req_get()) NRS start fifo request from 12345-0@lo, seq: 995 00000100:00100000:1.0:1776178318.177083:0:6256:0:(service.c:2324:ptlrpc_server_handle_request()) Handling RPC req@00000000bacf691a pname:cluuid+ref:pid:xid:nid:opc:job mdt00_001:lustre-MDT0001-lwp-OST0001_UUID+5:3354:x1862456998065280:12345-0@lo:400:kworker/u8:2.0 00000100:00100000:1.0:1776178318.177132:0:6256:0:(service.c:2372:ptlrpc_server_handle_request()) Handled RPC req@00000000bacf691a pname:cluuid+ref:pid:xid:nid:opc:job mdt00_001:lustre-MDT0001-lwp-OST0001_UUID+5:3354:x1862456998065280:12345-0@lo:400:kworker/u8:2.0 Request processed in 49us (318us total) trans 0 rc 0/0 00000100:00100000:1.0:1776178318.177140:0:6256:0:(nrs_fifo.c:241:nrs_fifo_req_stop()) NRS stop fifo request from 12345-0@lo, seq: 995 00000100:00100000:1.0:1776178318.177155:0:6256:0:(nrs_fifo.c:179:nrs_fifo_req_get()) NRS start fifo request from 12345-0@lo, seq: 996 00000100:00100000:1.0:1776178318.177159:0:6256:0:(service.c:2324:ptlrpc_server_handle_request()) Handling RPC req@00000000e178f924 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_001:lustre-MDT0001-lwp-OST0000_UUID+5:3356:x1862456998065152:12345-0@lo:400:kworker/u8:2.0 00000100:00100000:1.0:1776178318.177188:0:6256:0:(service.c:2372:ptlrpc_server_handle_request()) Handled RPC req@00000000e178f924 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_001:lustre-MDT0001-lwp-OST0000_UUID+5:3356:x1862456998065152:12345-0@lo:400:kworker/u8:2.0 Request processed in 28us (326us total) trans 0 rc 0/0 00000100:00100000:1.0:1776178318.177193:0:6256:0:(nrs_fifo.c:241:nrs_fifo_req_stop()) NRS stop fifo request from 12345-0@lo, seq: 996 00000100:00100000:1.0:1776178318.177221:0:20137:0:(service.c:2152:ptlrpc_server_handle_req_in()) got req x1862456998065024 00000100:00100000:1.0:1776178318.177242:0:20137:0:(nrs_fifo.c:179:nrs_fifo_req_get()) NRS start fifo request from 12345-0@lo, seq: 200 00000100:00100000:1.0:1776178318.177245:0:20137:0:(service.c:2324:ptlrpc_server_handle_request()) Handling RPC req@00000000f1dccdd0 pname:cluuid+ref:pid:xid:nid:opc:job ll_mgs_0000:ac61f839-0e8e-4cac-9c59-08bcfbbad330+18:3356:x1862456998065024:12345-0@lo:400:kworker/u8:2.0 00000100:00100000:1.0:1776178318.177271:0:20137:0:(service.c:2372:ptlrpc_server_handle_request()) Handled RPC req@00000000f1dccdd0 pname:cluuid+ref:pid:xid:nid:opc:job ll_mgs_0000:ac61f839-0e8e-4cac-9c59-08bcfbbad330+18:3356:x1862456998065024:12345-0@lo:400:kworker/u8:2.0 Request processed in 25us (359us total) trans 0 rc 0/0 00000100:00100000:1.0:1776178318.177279:0:20137:0:(nrs_fifo.c:241:nrs_fifo_req_stop()) NRS stop fifo request from 12345-0@lo, seq: 200 00000100:00100000:1.0:1776178318.177318:0:3356:0:(client.c:2215:ptlrpc_check_set()) Completed RPC req@00000000a2082164 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_03:ac61f839-0e8e-4cac-9c59-08bcfbbad330:3356:1862456998065024:0@lo:400:kworker/u8:2.0 00000100:00100000:1.0:1776178318.177331:0:3356:0:(client.c:2215:ptlrpc_check_set()) Completed RPC req@0000000084c00ed0 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_03:lustre-MDT0001-lwp-OST0000_UUID:3356:1862456998065152:0@lo:400:kworker/u8:2.0 00000100:00100000:1.0:1776178318.177357:0:3354:0:(client.c:2215:ptlrpc_check_set()) Completed RPC req@00000000c324c57b pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_01:lustre-MDT0001-lwp-OST0001_UUID:3354:1862456998065280:0@lo:400:kworker/u8:2.0 00000100:00100000:3.0:1776178318.177447:0:20759:0:(service.c:2152:ptlrpc_server_handle_req_in()) got req x1862456998065408 00000100:00100000:3.0:1776178318.177477:0:20759:0:(service.c:2152:ptlrpc_server_handle_req_in()) got req x1862456998065472 00000100:00100000:3.0:1776178318.177495:0:20759:0:(nrs_fifo.c:179:nrs_fifo_req_get()) NRS start fifo request from 12345-0@lo, seq: 601 00000100:00100000:3.0:1776178318.177498:0:20759:0:(service.c:2324:ptlrpc_server_handle_request()) Handling RPC req@00000000597c3414 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_010:lustre-MDT0001-mdtlov_UUID+5:3354:x1862456998065408:12345-0@lo:400:kworker/u8:2.0 00000100:00100000:3.0:1776178318.177545:0:20759:0:(service.c:2372:ptlrpc_server_handle_request()) Handled RPC req@00000000597c3414 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_010:lustre-MDT0001-mdtlov_UUID+5:3354:x1862456998065408:12345-0@lo:400:kworker/u8:2.0 Request processed in 45us (814us total) trans 0 rc 0/0 00000100:00100000:3.0:1776178318.177550:0:20759:0:(nrs_fifo.c:241:nrs_fifo_req_stop()) NRS stop fifo request from 12345-0@lo, seq: 601 00000100:00100000:3.0:1776178318.177561:0:20759:0:(nrs_fifo.c:179:nrs_fifo_req_get()) NRS start fifo request from 12345-0@lo, seq: 602 00000100:00100000:3.0:1776178318.177564:0:20759:0:(service.c:2324:ptlrpc_server_handle_request()) Handling RPC req@00000000d1e0f01c pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_010:lustre-MDT0001-mdtlov_UUID+5:3356:x1862456998065472:12345-0@lo:400:kworker/u8:2.0 00000100:00100000:3.0:1776178318.177587:0:20759:0:(service.c:2372:ptlrpc_server_handle_request()) Handled RPC req@00000000d1e0f01c pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_010:lustre-MDT0001-mdtlov_UUID+5:3356:x1862456998065472:12345-0@lo:400:kworker/u8:2.0 Request processed in 23us (639us total) trans 0 rc 0/0 00000100:00100000:3.0:1776178318.177592:0:20759:0:(nrs_fifo.c:241:nrs_fifo_req_stop()) NRS stop fifo request from 12345-0@lo, seq: 602 00000100:00100000:3.0:1776178318.177617:0:3356:0:(client.c:2215:ptlrpc_check_set()) Completed RPC req@000000006342fd4b pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_03:lustre-MDT0001-mdtlov_UUID:3356:1862456998065472:0@lo:400:kworker/u8:2.0 00000100:00100000:3.0:1776178318.177633:0:3354:0:(client.c:2215:ptlrpc_check_set()) Completed RPC req@0000000047c15ac0 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_01:lustre-MDT0001-mdtlov_UUID:3354:1862456998065408:0@lo:400:kworker/u8:2.0 00000100:00100000:2.0:1776178318.177934:0:3352:0:(client.c:1733:ptlrpc_send_new_req()) Sending RPC req@000000004eda7977 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:lustre-MDT0000-lwp-OST0000_UUID:3352:1862456998065088:0@lo:38:kworker/u8:2.0 00000100:00100000:2.0:1776178318.177965:0:3352:0:(events.c:361:request_in_callback()) peer: 12345-0@lo (source: 12345-0@lo) 00000100:00100000:2.0:1776178318.177987:0:3352:0:(client.c:1733:ptlrpc_send_new_req()) Sending RPC req@0000000007fa878c pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:lustre-MDT0000-lwp-OST0001_UUID:3352:1862456998065216:0@lo:38:kworker/u8:2.0 00000100:00100000:2.0:1776178318.178006:0:3352:0:(events.c:361:request_in_callback()) peer: 12345-0@lo (source: 12345-0@lo) 00000100:00100000:2.0:1776178318.178017:0:3352:0:(client.c:1733:ptlrpc_send_new_req()) Sending RPC req@000000008c3fbdb6 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:lustre-MDT0001-mdtlov_UUID:3352:1862456998065344:0@lo:38:kworker/u8:2.0 00000100:00100000:2.0:1776178318.178035:0:3352:0:(events.c:361:request_in_callback()) peer: 12345-0@lo (source: 12345-0@lo) 00000100:00100000:2.0:1776178318.178046:0:3352:0:(client.c:1733:ptlrpc_send_new_req()) Sending RPC req@0000000014250674 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:lustre-MDT0000-lwp-MDT0001_UUID:3352:1862456998065536:0@lo:38:kworker/u8:2.0 00000100:00100000:2.0:1776178318.178077:0:3352:0:(events.c:361:request_in_callback()) peer: 12345-0@lo (source: 12345-0@lo) 00000100:00100000:2.0:1776178318.178106:0:14128:0:(service.c:2152:ptlrpc_server_handle_req_in()) got req x1862456998065344 00000100:00100000:2.0:1776178318.178125:0:14128:0:(nrs_fifo.c:179:nrs_fifo_req_get()) NRS start fifo request from 12345-0@lo, seq: 730 00000100:00100000:2.0:1776178318.178129:0:14128:0:(service.c:2324:ptlrpc_server_handle_request()) Handling RPC req@0000000057946847 pname:cluuid+ref:pid:xid:nid:opc:job mdt_out00_002:0+-99:3352:x1862456998065344:12345-0@lo:38:kworker/u8:2.0 00010000:02000000:2.0:1776178318.178142:0:14128:0:(ldlm_lib.c:1139:target_handle_connect()) lustre-MDT0000: Not available for connect from 0@lo (stopping) 00000100:00100000:2.0:1776178318.178168:0:14128:0:(service.c:2372:ptlrpc_server_handle_request()) Handled RPC req@0000000057946847 pname:cluuid+ref:pid:xid:nid:opc:job mdt_out00_002:0+-99:3352:x1862456998065344:12345-0@lo:38:kworker/u8:2.0 Request processed in 39us (133us total) trans 0 rc -19/-19 00000100:00100000:2.0:1776178318.178174:0:14128:0:(nrs_fifo.c:241:nrs_fifo_req_stop()) NRS stop fifo request from 12345-0@lo, seq: 730 00000100:00100000:2.0:1776178318.178192:0:6255:0:(service.c:2152:ptlrpc_server_handle_req_in()) got req x1862456998065088 00000100:00100000:2.0:1776178318.178204:0:6255:0:(service.c:2152:ptlrpc_server_handle_req_in()) got req x1862456998065216 00000100:00100000:2.0:1776178318.178210:0:6255:0:(service.c:2152:ptlrpc_server_handle_req_in()) got req x1862456998065536 00000100:00100000:2.0:1776178318.178217:0:6255:0:(nrs_fifo.c:179:nrs_fifo_req_get()) NRS start fifo request from 12345-0@lo, seq: 6713 00000100:00100000:2.0:1776178318.178220:0:6255:0:(service.c:2324:ptlrpc_server_handle_request()) Handling RPC req@00000000ac4116de pname:cluuid+ref:pid:xid:nid:opc:job mdt00_000:0+-99:3352:x1862456998065088:12345-0@lo:38:kworker/u8:2.0 00010000:02000000:2.0:1776178318.178227:0:6255:0:(ldlm_lib.c:1139:target_handle_connect()) lustre-MDT0000: Not available for connect from 0@lo (stopping) 00000100:00100000:2.0:1776178318.178244:0:6255:0:(service.c:2372:ptlrpc_server_handle_request()) Handled RPC req@00000000ac4116de pname:cluuid+ref:pid:xid:nid:opc:job mdt00_000:0+-99:3352:x1862456998065088:12345-0@lo:38:kworker/u8:2.0 Request processed in 24us (280us total) trans 0 rc -19/-19 00000100:00100000:2.0:1776178318.178249:0:6255:0:(nrs_fifo.c:241:nrs_fifo_req_stop()) NRS stop fifo request from 12345-0@lo, seq: 6713 00000100:00100000:2.0:1776178318.178257:0:6255:0:(nrs_fifo.c:179:nrs_fifo_req_get()) NRS start fifo request from 12345-0@lo, seq: 6714 00000100:00100000:2.0:1776178318.178259:0:6255:0:(service.c:2324:ptlrpc_server_handle_request()) Handling RPC req@0000000038b19922 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_000:0+-99:3352:x1862456998065216:12345-0@lo:38:kworker/u8:2.0 00010000:02000000:2.0:1776178318.178264:0:6255:0:(ldlm_lib.c:1139:target_handle_connect()) lustre-MDT0000: Not available for connect from 0@lo (stopping) 00000100:00100000:2.0:1776178318.178276:0:6255:0:(service.c:2372:ptlrpc_server_handle_request()) Handled RPC req@0000000038b19922 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_000:0+-99:3352:x1862456998065216:12345-0@lo:38:kworker/u8:2.0 Request processed in 16us (270us total) trans 0 rc -19/-19 00000100:00100000:2.0:1776178318.178281:0:6255:0:(nrs_fifo.c:241:nrs_fifo_req_stop()) NRS stop fifo request from 12345-0@lo, seq: 6714 00000100:00100000:2.0:1776178318.178288:0:6255:0:(nrs_fifo.c:179:nrs_fifo_req_get()) NRS start fifo request from 12345-0@lo, seq: 6715 00000100:00100000:2.0:1776178318.178290:0:6255:0:(service.c:2324:ptlrpc_server_handle_request()) Handling RPC req@00000000ffd05959 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_000:0+-99:3352:x1862456998065536:12345-0@lo:38:kworker/u8:2.0 00010000:02000000:2.0:1776178318.178295:0:6255:0:(ldlm_lib.c:1139:target_handle_connect()) lustre-MDT0000: Not available for connect from 0@lo (stopping) 00000100:00100000:2.0:1776178318.178306:0:6255:0:(service.c:2372:ptlrpc_server_handle_request()) Handled RPC req@00000000ffd05959 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_000:0+-99:3352:x1862456998065536:12345-0@lo:38:kworker/u8:2.0 Request processed in 15us (230us total) trans 0 rc -19/-19 00000100:00100000:2.0:1776178318.178311:0:6255:0:(nrs_fifo.c:241:nrs_fifo_req_stop()) NRS stop fifo request from 12345-0@lo, seq: 6715 00000100:00080000:2.0:1776178318.178328:0:3352:0:(import.c:86:import_set_state_nolock()) 0000000047601a98 lustre-MDT0000_UUID: changing import state from CONNECTING to DISCONN 00000100:00080000:2.0:1776178318.178331:0:3352:0:(import.c:1429:ptlrpc_connect_interpret()) recovery of lustre-MDT0000_UUID on 192.168.204.103@tcp failed (-19) 00000100:00100000:2.0:1776178318.178334:0:3352:0:(client.c:2215:ptlrpc_check_set()) Completed RPC req@000000004eda7977 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:lustre-MDT0000-lwp-OST0000_UUID:3352:1862456998065088:0@lo:38:kworker/u8:2.0 00000100:00080000:2.0:1776178318.178340:0:3352:0:(import.c:86:import_set_state_nolock()) 00000000dfc8db13 lustre-MDT0000_UUID: changing import state from CONNECTING to DISCONN 00000100:00080000:2.0:1776178318.178342:0:3352:0:(import.c:1429:ptlrpc_connect_interpret()) recovery of lustre-MDT0000_UUID on 192.168.204.103@tcp failed (-19) 00000100:00100000:2.0:1776178318.178345:0:3352:0:(client.c:2215:ptlrpc_check_set()) Completed RPC req@0000000007fa878c pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:lustre-MDT0000-lwp-OST0001_UUID:3352:1862456998065216:0@lo:38:kworker/u8:2.0 00000100:00080000:2.0:1776178318.178350:0:3352:0:(import.c:86:import_set_state_nolock()) 00000000ba4f1821 lustre-MDT0000_UUID: changing import state from CONNECTING to DISCONN 00000100:00080000:2.0:1776178318.178352:0:3352:0:(import.c:1429:ptlrpc_connect_interpret()) recovery of lustre-MDT0000_UUID on 192.168.204.103@tcp failed (-19) 00000100:00100000:2.0:1776178318.178354:0:3352:0:(client.c:2215:ptlrpc_check_set()) Completed RPC req@000000008c3fbdb6 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:lustre-MDT0001-mdtlov_UUID:3352:1862456998065344:0@lo:38:kworker/u8:2.0 00000100:00080000:2.0:1776178318.178360:0:3352:0:(import.c:86:import_set_state_nolock()) 00000000a7c55b1f lustre-MDT0000_UUID: changing import state from CONNECTING to DISCONN 00000100:00080000:2.0:1776178318.178362:0:3352:0:(import.c:1429:ptlrpc_connect_interpret()) recovery of lustre-MDT0000_UUID on 192.168.204.103@tcp failed (-19) 00000100:00100000:2.0:1776178318.178364:0:3352:0:(client.c:2215:ptlrpc_check_set()) Completed RPC req@0000000014250674 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:lustre-MDT0000-lwp-MDT0001_UUID:3352:1862456998065536:0@lo:38:kworker/u8:2.0 00010000:00010000:2.0:1776178318.822046:0:9225:0:(ldlm_lockd.c:2792:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:3.0:1776178319.713652:0:3356:0:(client.c:1733:ptlrpc_send_new_req()) Sending RPC req@000000000d203d1d pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_03:lustre-MDT0001-mdtlov_UUID:3356:1862456998065600:0@lo:13:osp-pre-1-1.0 00000100:00100000:3.0:1776178319.713726:0:3356:0:(events.c:361:request_in_callback()) peer: 12345-0@lo (source: 12345-0@lo) 00000100:00100000:0.0:1776178319.713829:0:9554:0:(service.c:2152:ptlrpc_server_handle_req_in()) got req x1862456998065600 00000100:00100000:0.0:1776178319.713893:0:9554:0:(nrs_fifo.c:179:nrs_fifo_req_get()) NRS start fifo request from 12345-0@lo, seq: 533 00000100:00100000:0.0:1776178319.713898:0:9554:0:(service.c:2324:ptlrpc_server_handle_request()) Handling RPC req@000000004e8fbd9b pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_create00:lustre-MDT0001-mdtlov_UUID+5:3356:x1862456998065600:12345-0@lo:13:osp-pre-1-1.0 00002000:00000024:0.0:1776178319.713925:0:9554:0:(ofd_obd.c:740:ofd_statfs()) blocks cached 229376 granted 491746368 pending 0 free 3923689472 avail 3692167168 00000100:00100000:0.0:1776178319.715762:0:9554:0:(service.c:2372:ptlrpc_server_handle_request()) Handled RPC req@000000004e8fbd9b pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_create00:lustre-MDT0001-mdtlov_UUID+5:3356:x1862456998065600:12345-0@lo:13:osp-pre-1-1.0 Request processed in 1858us (2033us total) trans 0 rc 0/0 00000100:00100000:0.0:1776178319.715773:0:9554:0:(nrs_fifo.c:241:nrs_fifo_req_stop()) NRS stop fifo request from 12345-0@lo, seq: 533 00000100:00100000:3.0:1776178319.715843:0:3356:0:(client.c:2215:ptlrpc_check_set()) Completed RPC req@000000000d203d1d pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_03:lustre-MDT0001-mdtlov_UUID:3356:1862456998065600:0@lo:13:osp-pre-1-1.0 00010000:00010000:2.0:1776178319.842341:0:6244:0:(ldlm_lockd.c:2792:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:0.0:1776178320.225654:0:3354:0:(ptlrpcd.c:415:ptlrpcd_check()) transfer 1 async RPCs [0->1] 00000100:00100000:0.0:1776178320.225678:0:3354:0:(client.c:1733:ptlrpc_send_new_req()) Sending RPC req@00000000ef862c6d pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_01:lustre-MDT0001-mdtlov_UUID:3354:1862456998065664:0@lo:13:osp-pre-0-1.0 00000100:00100000:0.0:1776178320.225764:0:3354:0:(events.c:361:request_in_callback()) peer: 12345-0@lo (source: 12345-0@lo) 00000100:00100000:0.0:1776178320.225815:0:9554:0:(service.c:2152:ptlrpc_server_handle_req_in()) got req x1862456998065664 00000100:00100000:0.0:1776178320.225858:0:9554:0:(nrs_fifo.c:179:nrs_fifo_req_get()) NRS start fifo request from 12345-0@lo, seq: 534 00000100:00100000:0.0:1776178320.225864:0:9554:0:(service.c:2324:ptlrpc_server_handle_request()) Handling RPC req@00000000952003a4 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_create00:lustre-MDT0001-mdtlov_UUID+5:3354:x1862456998065664:12345-0@lo:13:osp-pre-0-1.0 00002000:00000024:0.0:1776178320.225890:0:9554:0:(ofd_obd.c:740:ofd_statfs()) blocks cached 315392 granted 484883776 pending 0 free 3923689472 avail 3692167168 00000100:00100000:0.0:1776178320.225929:0:9554:0:(service.c:2372:ptlrpc_server_handle_request()) Handled RPC req@00000000952003a4 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_create00:lustre-MDT0001-mdtlov_UUID+5:3354:x1862456998065664:12345-0@lo:13:osp-pre-0-1.0 Request processed in 64us (166us total) trans 0 rc 0/0 00000100:00100000:0.0:1776178320.225936:0:9554:0:(nrs_fifo.c:241:nrs_fifo_req_stop()) NRS stop fifo request from 12345-0@lo, seq: 534 00000100:00100000:0.0:1776178320.225977:0:3354:0:(client.c:2215:ptlrpc_check_set()) Completed RPC req@00000000ef862c6d pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_01:lustre-MDT0001-mdtlov_UUID:3354:1862456998065664:0@lo:13:osp-pre-0-1.0 00010000:00010000:1.0:1776178320.864322:0:6245:0:(ldlm_lockd.c:2792:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:3.0:1776178321.700400:0:3348:0:(events.c:361:request_in_callback()) peer: 12345-192.168.204.3@tcp (source: 12345-192.168.204.3@tcp) 00000100:00100000:3.0:1776178321.700448:0:3348:0:(events.c:361:request_in_callback()) peer: 12345-192.168.204.3@tcp (source: 12345-192.168.204.3@tcp) 00000100:00100000:3.0:1776178321.700492:0:3348:0:(events.c:361:request_in_callback()) peer: 12345-192.168.204.3@tcp (source: 12345-192.168.204.3@tcp) 00000100:00100000:3.0:1776178321.700525:0:3348:0:(events.c:361:request_in_callback()) peer: 12345-192.168.204.3@tcp (source: 12345-192.168.204.3@tcp) 00000100:00100000:3.0:1776178321.700592:0:3348:0:(events.c:361:request_in_callback()) peer: 12345-192.168.204.3@tcp (source: 12345-192.168.204.3@tcp) 00000100:00100000:3.0:1776178321.700627:0:14440:0:(service.c:2152:ptlrpc_server_handle_req_in()) got req x1862456979116544 00000100:00100000:0.0:1776178321.700651:0:6257:0:(service.c:2152:ptlrpc_server_handle_req_in()) got req x1862456979116480 00000100:00100000:3.0:1776178321.700675:0:14440:0:(service.c:2152:ptlrpc_server_handle_req_in()) got req x1862456979116608 00000100:00100000:0.0:1776178321.700692:0:6257:0:(service.c:2152:ptlrpc_server_handle_req_in()) got req x1862456979116416 00000100:00100000:3.0:1776178321.700707:0:14440:0:(nrs_fifo.c:179:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.204.3@tcp, seq: 603 00000100:00100000:3.0:1776178321.700716:0:14440:0:(service.c:2324:ptlrpc_server_handle_request()) Handling RPC req@00000000a8c8cb2d pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_004:b50963bd-8eba-4432-9a54-8888d00e3fa1+8:2271:x1862456979116544:12345-192.168.204.3@tcp:400:kworker/u8:3.0 00000100:00100000:0.0:1776178321.700720:0:6257:0:(nrs_fifo.c:179:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.204.3@tcp, seq: 997 00000100:00100000:0.0:1776178321.700725:0:6257:0:(service.c:2324:ptlrpc_server_handle_request()) Handling RPC req@0000000092028508 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_002:b50963bd-8eba-4432-9a54-8888d00e3fa1+6:2271:x1862456979116480:12345-192.168.204.3@tcp:400:kworker/u8:3.0 00000100:00100000:1.0:1776178321.700775:0:20755:0:(nrs_fifo.c:179:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.204.3@tcp, seq: 604 00000100:00100000:1.0:1776178321.700781:0:20755:0:(service.c:2324:ptlrpc_server_handle_request()) Handling RPC req@0000000095214b8e pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_006:b50963bd-8eba-4432-9a54-8888d00e3fa1+9:2271:x1862456979116608:12345-192.168.204.3@tcp:400:kworker/u8:3.0 00000100:00100000:0.0:1776178321.700817:0:6257:0:(service.c:2372:ptlrpc_server_handle_request()) Handled RPC req@0000000092028508 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_002:b50963bd-8eba-4432-9a54-8888d00e3fa1+6:2271:x1862456979116480:12345-192.168.204.3@tcp:400:kworker/u8:3.0 Request processed in 89us (323us total) trans 0 rc 0/0 00000100:00100000:1.0:1776178321.700819:0:20755:0:(service.c:2372:ptlrpc_server_handle_request()) Handled RPC req@0000000095214b8e pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_006:b50963bd-8eba-4432-9a54-8888d00e3fa1+9:2271:x1862456979116608:12345-192.168.204.3@tcp:400:kworker/u8:3.0 Request processed in 34us (291us total) trans 0 rc 0/0 00000100:00100000:0.0:1776178321.700833:0:6257:0:(nrs_fifo.c:241:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.204.3@tcp, seq: 997 00000100:00100000:1.0:1776178321.700833:0:20755:0:(nrs_fifo.c:241:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.204.3@tcp, seq: 604 00000100:00100000:0.0:1776178321.700858:0:6257:0:(nrs_fifo.c:179:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.204.3@tcp, seq: 6716 00000100:00100000:0.0:1776178321.700864:0:6257:0:(service.c:2324:ptlrpc_server_handle_request()) Handling RPC req@00000000ec2011d6 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_002:0+-99:2269:x1862456979116416:12345-192.168.204.3@tcp:38:kworker/u8:3.0 00010000:02000000:0.0:1776178321.700878:0:6257:0:(ldlm_lib.c:1139:target_handle_connect()) lustre-MDT0000: Not available for connect from 192.168.204.3@tcp (stopping) 00000100:00100000:0.0:1776178321.700909:0:6257:0:(service.c:2372:ptlrpc_server_handle_request()) Handled RPC req@00000000ec2011d6 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_002:0+-99:2269:x1862456979116416:12345-192.168.204.3@tcp:38:kworker/u8:3.0 Request processed in 44us (316us total) trans 0 rc -19/-19 00000100:00100000:0.0:1776178321.700916:0:6257:0:(nrs_fifo.c:241:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.204.3@tcp, seq: 6716 00000100:00100000:2.0:1776178321.701440:0:20140:0:(service.c:2152:ptlrpc_server_handle_req_in()) got req x1862456979116352 00000100:00100000:2.0:1776178321.701490:0:20140:0:(nrs_fifo.c:179:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.204.3@tcp, seq: 201 00000100:00100000:2.0:1776178321.701494:0:20140:0:(service.c:2324:ptlrpc_server_handle_request()) Handling RPC req@00000000322375a6 pname:cluuid+ref:pid:xid:nid:opc:job ll_mgs_0002:5b5c66b1-1a2f-4651-8548-0c75b9a38a94+9:2271:x1862456979116352:12345-192.168.204.3@tcp:400:kworker/u8:3.0 00000100:00100000:2.0:1776178321.701537:0:20140:0:(service.c:2372:ptlrpc_server_handle_request()) Handled RPC req@00000000322375a6 pname:cluuid+ref:pid:xid:nid:opc:job ll_mgs_0002:5b5c66b1-1a2f-4651-8548-0c75b9a38a94+9:2271:x1862456979116352:12345-192.168.204.3@tcp:400:kworker/u8:3.0 Request processed in 42us (1145us total) trans 0 rc 0/0 00000100:00100000:2.0:1776178321.701544:0:20140:0:(nrs_fifo.c:241:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.204.3@tcp, seq: 201 00000100:00100000:3.0:1776178321.701864:0:14440:0:(service.c:2372:ptlrpc_server_handle_request()) Handled RPC req@00000000a8c8cb2d pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_004:b50963bd-8eba-4432-9a54-8888d00e3fa1+8:2271:x1862456979116544:12345-192.168.204.3@tcp:400:kworker/u8:3.0 Request processed in 1150us (1416us total) trans 0 rc 0/0 00000100:00100000:3.0:1776178321.701872:0:14440:0:(nrs_fifo.c:241:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.204.3@tcp, seq: 603 00010000:00010000:2.0:1776178321.888527:0:9224:0:(ldlm_lockd.c:2792:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00010000:00010000:2.0:1776178322.912411:0:9225:0:(ldlm_lockd.c:2792:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00080000:0.0:1776178323.296298:0:37:0:(pinger.c:251:ptlrpc_pinger_process_import()) lustre-MDT0000-lwp-OST0000_UUID->lustre-MDT0000_UUID: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00080000:0.0:1776178323.296307:0:37:0:(recover.c:57:ptlrpc_initiate_recovery()) lustre-MDT0000_UUID: starting recovery 00000100:00080000:0.0:1776178323.296310:0:37:0:(import.c:86:import_set_state_nolock()) 0000000047601a98 lustre-MDT0000_UUID: changing import state from DISCONN to CONNECTING 00000100:00080000:0.0:1776178323.296318:0:37:0:(import.c:534:import_select_connection()) lustre-MDT0000-lwp-OST0000: connect to NID 0@lo last attempt 1025 00000100:00080000:0.0:1776178323.296321:0:37:0:(import.c:581:import_select_connection()) lustre-MDT0000-lwp-OST0000: tried all connections, increasing latency to 21s 00000100:00080000:0.0:1776178323.296327:0:37:0:(import.c:615:import_select_connection()) lustre-MDT0000-lwp-OST0000: import 0000000047601a98 using connection 192.168.204.103@tcp/0@lo 00000100:00100000:0.0:1776178323.296351:0:37:0:(import.c:807:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5) req@0000000028422d92 x1862456998065792/t0(0) o38->lustre-MDT0000-lwp-OST0000@0@lo:12/10 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' 00000100:00080000:0.0:1776178323.296399:0:37:0:(pinger.c:251:ptlrpc_pinger_process_import()) lustre-MDT0000-lwp-OST0001_UUID->lustre-MDT0000_UUID: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00080000:0.0:1776178323.296403:0:37:0:(recover.c:57:ptlrpc_initiate_recovery()) lustre-MDT0000_UUID: starting recovery 00000100:00080000:0.0:1776178323.296404:0:37:0:(import.c:86:import_set_state_nolock()) 00000000dfc8db13 lustre-MDT0000_UUID: changing import state from DISCONN to CONNECTING 00000100:00080000:0.0:1776178323.296408:0:37:0:(import.c:534:import_select_connection()) lustre-MDT0000-lwp-OST0001: connect to NID 0@lo last attempt 1025 00000100:00080000:0.0:1776178323.296410:0:37:0:(import.c:581:import_select_connection()) lustre-MDT0000-lwp-OST0001: tried all connections, increasing latency to 21s 00000100:00080000:0.0:1776178323.296414:0:37:0:(import.c:615:import_select_connection()) lustre-MDT0000-lwp-OST0001: import 00000000dfc8db13 using connection 192.168.204.103@tcp/0@lo 00000100:00100000:0.0:1776178323.296424:0:37:0:(import.c:807:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5) req@000000001671af6a x1862456998065920/t0(0) o38->lustre-MDT0000-lwp-OST0001@0@lo:12/10 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' 00000100:00100000:3.0:1776178323.296445:0:3353:0:(ptlrpcd.c:415:ptlrpcd_check()) transfer 1 async RPCs [1->0] 00000100:00080000:0.0:1776178323.296448:0:37:0:(pinger.c:251:ptlrpc_pinger_process_import()) lustre-MDT0001-mdtlov_UUID->lustre-MDT0000_UUID: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00080000:0.0:1776178323.296450:0:37:0:(recover.c:57:ptlrpc_initiate_recovery()) lustre-MDT0000_UUID: starting recovery 00000100:00080000:0.0:1776178323.296452:0:37:0:(import.c:86:import_set_state_nolock()) 00000000ba4f1821 lustre-MDT0000_UUID: changing import state from DISCONN to CONNECTING 00000100:00080000:0.0:1776178323.296454:0:37:0:(import.c:534:import_select_connection()) lustre-MDT0000-osp-MDT0001: connect to NID 0@lo last attempt 1025 00000100:00080000:0.0:1776178323.296456:0:37:0:(import.c:581:import_select_connection()) lustre-MDT0000-osp-MDT0001: tried all connections, increasing latency to 21s 00000100:00080000:0.0:1776178323.296458:0:37:0:(import.c:615:import_select_connection()) lustre-MDT0000-osp-MDT0001: import 00000000ba4f1821 using connection 192.168.204.103@tcp/0@lo 00000100:00100000:3.0:1776178323.296461:0:3353:0:(client.c:1733:ptlrpc_send_new_req()) Sending RPC req@00000000ef862c6d pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_00:ac61f839-0e8e-4cac-9c59-08bcfbbad330:3353:1862456998065728:0@lo:400:kworker/u8:2.0 00000100:00100000:0.0:1776178323.296466:0:37:0:(import.c:807:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5) req@00000000b0409689 x1862456998066048/t0(0) o38->lustre-MDT0000-osp-MDT0001@0@lo:24/4 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' 00000100:00080000:0.0:1776178323.296481:0:37:0:(pinger.c:251:ptlrpc_pinger_process_import()) lustre-MDT0000-lwp-MDT0001_UUID->lustre-MDT0000_UUID: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00080000:0.0:1776178323.296484:0:37:0:(recover.c:57:ptlrpc_initiate_recovery()) lustre-MDT0000_UUID: starting recovery 00000100:00080000:0.0:1776178323.296485:0:37:0:(import.c:86:import_set_state_nolock()) 00000000a7c55b1f lustre-MDT0000_UUID: changing import state from DISCONN to CONNECTING 00000100:00080000:0.0:1776178323.296488:0:37:0:(import.c:534:import_select_connection()) lustre-MDT0000-lwp-MDT0001: connect to NID 0@lo last attempt 1025 00000100:00080000:0.0:1776178323.296490:0:37:0:(import.c:581:import_select_connection()) lustre-MDT0000-lwp-MDT0001: tried all connections, increasing latency to 21s 00000100:00080000:0.0:1776178323.296492:0:37:0:(import.c:615:import_select_connection()) lustre-MDT0000-lwp-MDT0001: import 00000000a7c55b1f using connection 192.168.204.103@tcp/0@lo 00000100:00100000:0.0:1776178323.296498:0:37:0:(import.c:807:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5) req@000000008c3fbdb6 x1862456998066240/t0(0) o38->lustre-MDT0000-lwp-MDT0001@0@lo:12/10 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' 00000100:00100000:0.0:1776178323.296524:0:3355:0:(client.c:1733:ptlrpc_send_new_req()) Sending RPC req@0000000089dbd0da pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_02:lustre-MDT0001-lwp-OST0000_UUID:3355:1862456998065856:0@lo:400:kworker/u8:2.0 00000100:00100000:3.0:1776178323.296537:0:3353:0:(events.c:361:request_in_callback()) peer: 12345-0@lo (source: 12345-0@lo) 00000100:00100000:0.0:1776178323.296558:0:3355:0:(events.c:361:request_in_callback()) peer: 12345-0@lo (source: 12345-0@lo) 00000100:00100000:3.0:1776178323.296582:0:3353:0:(client.c:1733:ptlrpc_send_new_req()) Sending RPC req@0000000039f3bccf pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_00:lustre-MDT0001-mdtlov_UUID:3353:1862456998066112:0@lo:400:kworker/u8:2.0 00000100:00100000:0.0:1776178323.296585:0:3355:0:(ptlrpcd.c:415:ptlrpcd_check()) transfer 1 async RPCs [3->2] 00000100:00100000:0.0:1776178323.296589:0:3355:0:(client.c:1733:ptlrpc_send_new_req()) Sending RPC req@00000000542c8034 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_02:lustre-MDT0001-lwp-OST0001_UUID:3355:1862456998065984:0@lo:400:kworker/u8:2.0 00000100:00100000:0.0:1776178323.296608:0:3355:0:(events.c:361:request_in_callback()) peer: 12345-0@lo (source: 12345-0@lo) 00000100:00100000:3.0:1776178323.296612:0:3353:0:(events.c:361:request_in_callback()) peer: 12345-0@lo (source: 12345-0@lo) 00000100:00100000:3.0:1776178323.296627:0:3353:0:(ptlrpcd.c:415:ptlrpcd_check()) transfer 1 async RPCs [1->0] 00000100:00100000:3.0:1776178323.296631:0:3353:0:(client.c:1733:ptlrpc_send_new_req()) Sending RPC req@0000000014250674 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_00:lustre-MDT0001-mdtlov_UUID:3353:1862456998066176:0@lo:400:kworker/u8:2.0 00000100:00100000:0.0:1776178323.296649:0:6256:0:(service.c:2152:ptlrpc_server_handle_req_in()) got req x1862456998065856 00000100:00100000:3.0:1776178323.296651:0:3353:0:(events.c:361:request_in_callback()) peer: 12345-0@lo (source: 12345-0@lo) 00000100:00100000:3.0:1776178323.296689:0:20137:0:(service.c:2152:ptlrpc_server_handle_req_in()) got req x1862456998065728 00000100:00100000:0.0:1776178323.296698:0:6256:0:(service.c:2152:ptlrpc_server_handle_req_in()) got req x1862456998065984 00000100:00100000:0.0:1776178323.296720:0:6256:0:(nrs_fifo.c:179:nrs_fifo_req_get()) NRS start fifo request from 12345-0@lo, seq: 998 00000100:00100000:3.0:1776178323.296720:0:20137:0:(nrs_fifo.c:179:nrs_fifo_req_get()) NRS start fifo request from 12345-0@lo, seq: 202 00000100:00100000:0.0:1776178323.296727:0:6256:0:(service.c:2324:ptlrpc_server_handle_request()) Handling RPC req@0000000007fa878c pname:cluuid+ref:pid:xid:nid:opc:job mdt00_001:lustre-MDT0001-lwp-OST0000_UUID+5:3355:x1862456998065856:12345-0@lo:400:kworker/u8:2.0 00000100:00100000:3.0:1776178323.296735:0:20137:0:(service.c:2324:ptlrpc_server_handle_request()) Handling RPC req@00000000f0ff59c2 pname:cluuid+ref:pid:xid:nid:opc:job ll_mgs_0000:ac61f839-0e8e-4cac-9c59-08bcfbbad330+18:3353:x1862456998065728:12345-0@lo:400:kworker/u8:2.0 00000100:00100000:0.0:1776178323.296775:0:6256:0:(service.c:2372:ptlrpc_server_handle_request()) Handled RPC req@0000000007fa878c pname:cluuid+ref:pid:xid:nid:opc:job mdt00_001:lustre-MDT0001-lwp-OST0000_UUID+5:3355:x1862456998065856:12345-0@lo:400:kworker/u8:2.0 Request processed in 48us (217us total) trans 0 rc 0/0 00000100:00100000:3.0:1776178323.296775:0:20137:0:(service.c:2372:ptlrpc_server_handle_request()) Handled RPC req@00000000f0ff59c2 pname:cluuid+ref:pid:xid:nid:opc:job ll_mgs_0000:ac61f839-0e8e-4cac-9c59-08bcfbbad330+18:3353:x1862456998065728:12345-0@lo:400:kworker/u8:2.0 Request processed in 39us (238us total) trans 0 rc 0/0 00000100:00100000:0.0:1776178323.296783:0:6256:0:(nrs_fifo.c:241:nrs_fifo_req_stop()) NRS stop fifo request from 12345-0@lo, seq: 998 00000100:00100000:3.0:1776178323.296783:0:20137:0:(nrs_fifo.c:241:nrs_fifo_req_stop()) NRS stop fifo request from 12345-0@lo, seq: 202 00000100:00100000:0.0:1776178323.296798:0:6256:0:(nrs_fifo.c:179:nrs_fifo_req_get()) NRS start fifo request from 12345-0@lo, seq: 999 00000100:00100000:0.0:1776178323.296801:0:6256:0:(service.c:2324:ptlrpc_server_handle_request()) Handling RPC req@000000004eda7977 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_001:lustre-MDT0001-lwp-OST0001_UUID+5:3355:x1862456998065984:12345-0@lo:400:kworker/u8:2.0 00000100:00100000:3.0:1776178323.296812:0:20759:0:(service.c:2152:ptlrpc_server_handle_req_in()) got req x1862456998066112 00000100:00100000:0.0:1776178323.296824:0:6256:0:(service.c:2372:ptlrpc_server_handle_request()) Handled RPC req@000000004eda7977 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_001:lustre-MDT0001-lwp-OST0001_UUID+5:3355:x1862456998065984:12345-0@lo:400:kworker/u8:2.0 Request processed in 22us (215us total) trans 0 rc 0/0 00000100:00100000:0.0:1776178323.296829:0:6256:0:(nrs_fifo.c:241:nrs_fifo_req_stop()) NRS stop fifo request from 12345-0@lo, seq: 999 00000100:00100000:3.0:1776178323.296833:0:20759:0:(service.c:2152:ptlrpc_server_handle_req_in()) got req x1862456998066176 00000100:00100000:3.0:1776178323.296849:0:20759:0:(nrs_fifo.c:179:nrs_fifo_req_get()) NRS start fifo request from 12345-0@lo, seq: 605 00000100:00100000:0.0:1776178323.296852:0:20762:0:(nrs_fifo.c:179:nrs_fifo_req_get()) NRS start fifo request from 12345-0@lo, seq: 606 00000100:00100000:3.0:1776178323.296852:0:20759:0:(service.c:2324:ptlrpc_server_handle_request()) Handling RPC req@000000009bbbb40c pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_010:lustre-MDT0001-mdtlov_UUID+5:3353:x1862456998066112:12345-0@lo:400:kworker/u8:2.0 00000100:00100000:0.0:1776178323.296855:0:20762:0:(service.c:2324:ptlrpc_server_handle_request()) Handling RPC req@00000000e4231891 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_013:lustre-MDT0001-mdtlov_UUID+5:3353:x1862456998066176:12345-0@lo:400:kworker/u8:2.0 00000100:00100000:0.0:1776178323.296877:0:20762:0:(service.c:2372:ptlrpc_server_handle_request()) Handled RPC req@00000000e4231891 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_013:lustre-MDT0001-mdtlov_UUID+5:3353:x1862456998066176:12345-0@lo:400:kworker/u8:2.0 Request processed in 21us (225us total) trans 0 rc 0/0 00000100:00100000:3.0:1776178323.296877:0:20759:0:(service.c:2372:ptlrpc_server_handle_request()) Handled RPC req@000000009bbbb40c pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_010:lustre-MDT0001-mdtlov_UUID+5:3353:x1862456998066112:12345-0@lo:400:kworker/u8:2.0 Request processed in 24us (265us total) trans 0 rc 0/0 00000100:00100000:0.0:1776178323.296882:0:20762:0:(nrs_fifo.c:241:nrs_fifo_req_stop()) NRS stop fifo request from 12345-0@lo, seq: 606 00000100:00100000:1.0:1776178323.296944:0:3352:0:(client.c:1733:ptlrpc_send_new_req()) Sending RPC req@0000000028422d92 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:lustre-MDT0000-lwp-OST0000_UUID:3352:1862456998065792:0@lo:38:kworker/u8:2.0 00000100:00100000:1.0:1776178323.296971:0:3352:0:(events.c:361:request_in_callback()) peer: 12345-0@lo (source: 12345-0@lo) 00000100:00100000:3.0:1776178323.296982:0:20759:0:(nrs_fifo.c:241:nrs_fifo_req_stop()) NRS stop fifo request from 12345-0@lo, seq: 605 00000100:00100000:1.0:1776178323.296985:0:3352:0:(client.c:1733:ptlrpc_send_new_req()) Sending RPC req@000000001671af6a pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:lustre-MDT0000-lwp-OST0001_UUID:3352:1862456998065920:0@lo:38:kworker/u8:2.0 00000100:00100000:1.0:1776178323.297003:0:3352:0:(events.c:361:request_in_callback()) peer: 12345-0@lo (source: 12345-0@lo) 00000100:00100000:1.0:1776178323.297013:0:3352:0:(client.c:1733:ptlrpc_send_new_req()) Sending RPC req@00000000b0409689 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:lustre-MDT0001-mdtlov_UUID:3352:1862456998066048:0@lo:38:kworker/u8:2.0 00000100:00100000:3.0:1776178323.297021:0:3353:0:(client.c:2215:ptlrpc_check_set()) Completed RPC req@0000000014250674 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_00:lustre-MDT0001-mdtlov_UUID:3353:1862456998066176:0@lo:400:kworker/u8:2.0 00000100:00100000:3.0:1776178323.297031:0:3353:0:(client.c:2215:ptlrpc_check_set()) Completed RPC req@0000000039f3bccf pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_00:lustre-MDT0001-mdtlov_UUID:3353:1862456998066112:0@lo:400:kworker/u8:2.0 00000100:00100000:1.0:1776178323.297032:0:3352:0:(events.c:361:request_in_callback()) peer: 12345-0@lo (source: 12345-0@lo) 00000100:00100000:3.0:1776178323.297038:0:3353:0:(client.c:2215:ptlrpc_check_set()) Completed RPC req@00000000ef862c6d pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_00:ac61f839-0e8e-4cac-9c59-08bcfbbad330:3353:1862456998065728:0@lo:400:kworker/u8:2.0 00000100:00100000:1.0:1776178323.297044:0:3352:0:(client.c:1733:ptlrpc_send_new_req()) Sending RPC req@000000008c3fbdb6 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:lustre-MDT0000-lwp-MDT0001_UUID:3352:1862456998066240:0@lo:38:kworker/u8:2.0 00000100:00100000:1.0:1776178323.297086:0:3352:0:(events.c:361:request_in_callback()) peer: 12345-0@lo (source: 12345-0@lo) 00000100:00100000:3.0:1776178323.297104:0:6257:0:(service.c:2152:ptlrpc_server_handle_req_in()) got req x1862456998065792 00000100:00100000:1.0:1776178323.297108:0:15213:0:(service.c:2152:ptlrpc_server_handle_req_in()) got req x1862456998066048 00000100:00100000:3.0:1776178323.297120:0:6257:0:(service.c:2152:ptlrpc_server_handle_req_in()) got req x1862456998065920 00000100:00100000:1.0:1776178323.297123:0:15213:0:(nrs_fifo.c:179:nrs_fifo_req_get()) NRS start fifo request from 12345-0@lo, seq: 731 00000100:00100000:1.0:1776178323.297126:0:15213:0:(service.c:2324:ptlrpc_server_handle_request()) Handling RPC req@00000000865e4f65 pname:cluuid+ref:pid:xid:nid:opc:job mdt_out00_003:0+-99:3352:x1862456998066048:12345-0@lo:38:kworker/u8:2.0 00000100:00100000:3.0:1776178323.297127:0:6257:0:(service.c:2152:ptlrpc_server_handle_req_in()) got req x1862456998066240 00000100:00100000:3.0:1776178323.297134:0:6257:0:(nrs_fifo.c:179:nrs_fifo_req_get()) NRS start fifo request from 12345-0@lo, seq: 6717 00000100:00100000:3.0:1776178323.297137:0:6257:0:(service.c:2324:ptlrpc_server_handle_request()) Handling RPC req@000000008245b0ee pname:cluuid+ref:pid:xid:nid:opc:job mdt00_002:0+-99:3352:x1862456998065792:12345-0@lo:38:kworker/u8:2.0 00010000:02000000:1.0:1776178323.297138:0:15213:0:(ldlm_lib.c:1139:target_handle_connect()) lustre-MDT0000: Not available for connect from 0@lo (stopping) 00010000:02000000:3.0:1776178323.297142:0:6257:0:(ldlm_lib.c:1139:target_handle_connect()) lustre-MDT0000: Not available for connect from 0@lo (stopping) 00000100:00100000:1.0:1776178323.297166:0:15213:0:(service.c:2372:ptlrpc_server_handle_request()) Handled RPC req@00000000865e4f65 pname:cluuid+ref:pid:xid:nid:opc:job mdt_out00_003:0+-99:3352:x1862456998066048:12345-0@lo:38:kworker/u8:2.0 Request processed in 39us (134us total) trans 0 rc -19/-19 00000100:00100000:3.0:1776178323.297166:0:6257:0:(service.c:2372:ptlrpc_server_handle_request()) Handled RPC req@000000008245b0ee pname:cluuid+ref:pid:xid:nid:opc:job mdt00_002:0+-99:3352:x1862456998065792:12345-0@lo:38:kworker/u8:2.0 Request processed in 28us (195us total) trans 0 rc -19/-19 00000100:00100000:1.0:1776178323.297172:0:15213:0:(nrs_fifo.c:241:nrs_fifo_req_stop()) NRS stop fifo request from 12345-0@lo, seq: 731 00000100:00100000:3.0:1776178323.297172:0:6257:0:(nrs_fifo.c:241:nrs_fifo_req_stop()) NRS stop fifo request from 12345-0@lo, seq: 6717 00000100:00100000:3.0:1776178323.297181:0:6257:0:(nrs_fifo.c:179:nrs_fifo_req_get()) NRS start fifo request from 12345-0@lo, seq: 6718 00000100:00100000:3.0:1776178323.297184:0:6257:0:(service.c:2324:ptlrpc_server_handle_request()) Handling RPC req@00000000a33eecf4 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_002:0+-99:3352:x1862456998065920:12345-0@lo:38:kworker/u8:2.0 00010000:02000000:3.0:1776178323.297189:0:6257:0:(ldlm_lib.c:1139:target_handle_connect()) lustre-MDT0000: Not available for connect from 0@lo (stopping) 00000100:00080000:1.0:1776178323.297201:0:3352:0:(import.c:86:import_set_state_nolock()) 0000000047601a98 lustre-MDT0000_UUID: changing import state from CONNECTING to DISCONN 00000100:00080000:1.0:1776178323.297204:0:3352:0:(import.c:1429:ptlrpc_connect_interpret()) recovery of lustre-MDT0000_UUID on 192.168.204.103@tcp failed (-19) 00000100:00100000:3.0:1776178323.297206:0:6257:0:(service.c:2372:ptlrpc_server_handle_request()) Handled RPC req@00000000a33eecf4 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_002:0+-99:3352:x1862456998065920:12345-0@lo:38:kworker/u8:2.0 Request processed in 22us (203us total) trans 0 rc -19/-19 00000100:00100000:1.0:1776178323.297207:0:3352:0:(client.c:2215:ptlrpc_check_set()) Completed RPC req@0000000028422d92 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:lustre-MDT0000-lwp-OST0000_UUID:3352:1862456998065792:0@lo:38:kworker/u8:2.0 00000100:00100000:3.0:1776178323.297211:0:6257:0:(nrs_fifo.c:241:nrs_fifo_req_stop()) NRS stop fifo request from 12345-0@lo, seq: 6718 00000100:00080000:1.0:1776178323.297213:0:3352:0:(import.c:86:import_set_state_nolock()) 00000000dfc8db13 lustre-MDT0000_UUID: changing import state from CONNECTING to DISCONN 00000100:00080000:1.0:1776178323.297215:0:3352:0:(import.c:1429:ptlrpc_connect_interpret()) recovery of lustre-MDT0000_UUID on 192.168.204.103@tcp failed (-19) 00000100:00100000:1.0:1776178323.297217:0:3352:0:(client.c:2215:ptlrpc_check_set()) Completed RPC req@000000001671af6a pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:lustre-MDT0000-lwp-OST0001_UUID:3352:1862456998065920:0@lo:38:kworker/u8:2.0 00000100:00100000:3.0:1776178323.297220:0:6257:0:(nrs_fifo.c:179:nrs_fifo_req_get()) NRS start fifo request from 12345-0@lo, seq: 6719 00000100:00100000:3.0:1776178323.297223:0:6257:0:(service.c:2324:ptlrpc_server_handle_request()) Handling RPC req@00000000a66f2317 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_002:0+-99:3352:x1862456998066240:12345-0@lo:38:kworker/u8:2.0 00000100:00080000:1.0:1776178323.297224:0:3352:0:(import.c:86:import_set_state_nolock()) 00000000ba4f1821 lustre-MDT0000_UUID: changing import state from CONNECTING to DISCONN 00000100:00080000:1.0:1776178323.297226:0:3352:0:(import.c:1429:ptlrpc_connect_interpret()) recovery of lustre-MDT0000_UUID on 192.168.204.103@tcp failed (-19) 00000100:00100000:1.0:1776178323.297228:0:3352:0:(client.c:2215:ptlrpc_check_set()) Completed RPC req@00000000b0409689 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:lustre-MDT0001-mdtlov_UUID:3352:1862456998066048:0@lo:38:kworker/u8:2.0 00010000:02000000:3.0:1776178323.297228:0:6257:0:(ldlm_lib.c:1139:target_handle_connect()) lustre-MDT0000: Not available for connect from 0@lo (stopping) 00000100:00100000:2.0:1776178323.297349:0:3355:0:(client.c:2215:ptlrpc_check_set()) Completed RPC req@00000000542c8034 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_02:lustre-MDT0001-lwp-OST0001_UUID:3355:1862456998065984:0@lo:400:kworker/u8:2.0 00000100:00100000:2.0:1776178323.297360:0:3355:0:(client.c:2215:ptlrpc_check_set()) Completed RPC req@0000000089dbd0da pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_02:lustre-MDT0001-lwp-OST0000_UUID:3355:1862456998065856:0@lo:400:kworker/u8:2.0 00000100:00100000:3.0:1776178323.297925:0:6257:0:(service.c:2372:ptlrpc_server_handle_request()) Handled RPC req@00000000a66f2317 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_002:0+-99:3352:x1862456998066240:12345-0@lo:38:kworker/u8:2.0 Request processed in 699us (837us total) trans 0 rc -19/-19 00000100:00100000:3.0:1776178323.297933:0:6257:0:(nrs_fifo.c:241:nrs_fifo_req_stop()) NRS stop fifo request from 12345-0@lo, seq: 6719 00000100:00080000:1.0:1776178323.298132:0:3352:0:(import.c:86:import_set_state_nolock()) 00000000a7c55b1f lustre-MDT0000_UUID: changing import state from CONNECTING to DISCONN 00000100:00080000:1.0:1776178323.298137:0:3352:0:(import.c:1429:ptlrpc_connect_interpret()) recovery of lustre-MDT0000_UUID on 192.168.204.103@tcp failed (-19) 00000100:00100000:1.0:1776178323.298141:0:3352:0:(client.c:2215:ptlrpc_check_set()) Completed RPC req@000000008c3fbdb6 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:lustre-MDT0000-lwp-MDT0001_UUID:3352:1862456998066240:0@lo:38:kworker/u8:2.0 00010000:00010000:2.0:1776178323.936581:0:6244:0:(ldlm_lockd.c:2792:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:3.0:1776178324.833715:0:3356:0:(ptlrpcd.c:415:ptlrpcd_check()) transfer 1 async RPCs [2->3] 00000100:00100000:3.0:1776178324.833748:0:3356:0:(client.c:1733:ptlrpc_send_new_req()) Sending RPC req@000000000d203d1d pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_03:lustre-MDT0001-mdtlov_UUID:3356:1862456998066304:0@lo:13:osp-pre-1-1.0 00000100:00100000:3.0:1776178324.833832:0:3356:0:(events.c:361:request_in_callback()) peer: 12345-0@lo (source: 12345-0@lo) 00000100:00100000:1.0:1776178324.833910:0:8231:0:(service.c:2152:ptlrpc_server_handle_req_in()) got req x1862456998066304 00000100:00100000:1.0:1776178324.833976:0:8231:0:(nrs_fifo.c:179:nrs_fifo_req_get()) NRS start fifo request from 12345-0@lo, seq: 535 00000100:00100000:1.0:1776178324.833983:0:8231:0:(service.c:2324:ptlrpc_server_handle_request()) Handling RPC req@00000000239d7b9a pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_create00:lustre-MDT0001-mdtlov_UUID+5:3356:x1862456998066304:12345-0@lo:13:osp-pre-1-1.0 00002000:00000024:1.0:1776178324.834018:0:8231:0:(ofd_obd.c:740:ofd_statfs()) blocks cached 229376 granted 491746368 pending 0 free 3923689472 avail 3692167168 00000100:00100000:1.0:1776178324.834095:0:8231:0:(service.c:2372:ptlrpc_server_handle_request()) Handled RPC req@00000000239d7b9a pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_create00:lustre-MDT0001-mdtlov_UUID+5:3356:x1862456998066304:12345-0@lo:13:osp-pre-1-1.0 Request processed in 112us (265us total) trans 0 rc 0/0 00000100:00100000:1.0:1776178324.834104:0:8231:0:(nrs_fifo.c:241:nrs_fifo_req_stop()) NRS stop fifo request from 12345-0@lo, seq: 535 00000100:00100000:3.0:1776178324.834209:0:3356:0:(client.c:2215:ptlrpc_check_set()) Completed RPC req@000000000d203d1d pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_03:lustre-MDT0001-mdtlov_UUID:3356:1862456998066304:0@lo:13:osp-pre-1-1.0 00010000:00010000:1.0:1776178324.960449:0:6245:0:(ldlm_lockd.c:2792:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:2.0:1776178325.344941:0:3355:0:(ptlrpcd.c:415:ptlrpcd_check()) transfer 1 async RPCs [3->2] 00000100:00100000:2.0:1776178325.344972:0:3355:0:(client.c:1733:ptlrpc_send_new_req()) Sending RPC req@0000000047c15ac0 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_02:lustre-MDT0001-mdtlov_UUID:3355:1862456998066368:0@lo:13:osp-pre-0-1.0 00000100:00100000:2.0:1776178325.345084:0:3355:0:(events.c:361:request_in_callback()) peer: 12345-0@lo (source: 12345-0@lo) 00000100:00100000:0.0:1776178325.345505:0:9554:0:(service.c:2152:ptlrpc_server_handle_req_in()) got req x1862456998066368 00000100:00100000:0.0:1776178325.346751:0:9554:0:(nrs_fifo.c:179:nrs_fifo_req_get()) NRS start fifo request from 12345-0@lo, seq: 536 00000100:00100000:0.0:1776178325.346766:0:9554:0:(service.c:2324:ptlrpc_server_handle_request()) Handling RPC req@000000008b978a8a pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_create00:lustre-MDT0001-mdtlov_UUID+5:3355:x1862456998066368:12345-0@lo:13:osp-pre-0-1.0 00002000:00000024:0.0:1776178325.346824:0:9554:0:(ofd_obd.c:740:ofd_statfs()) blocks cached 315392 granted 484883776 pending 0 free 3923689472 avail 3692167168 00000100:00100000:0.0:1776178325.346938:0:9554:0:(service.c:2372:ptlrpc_server_handle_request()) Handled RPC req@000000008b978a8a pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_create00:lustre-MDT0001-mdtlov_UUID+5:3355:x1862456998066368:12345-0@lo:13:osp-pre-0-1.0 Request processed in 173us (1855us total) trans 0 rc 0/0 00000100:00100000:0.0:1776178325.346945:0:9554:0:(nrs_fifo.c:241:nrs_fifo_req_stop()) NRS stop fifo request from 12345-0@lo, seq: 536 00000100:00100000:2.0:1776178325.347106:0:3355:0:(client.c:2215:ptlrpc_check_set()) Completed RPC req@0000000047c15ac0 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_02:lustre-MDT0001-mdtlov_UUID:3355:1862456998066368:0@lo:13:osp-pre-0-1.0 00010000:00010000:2.0:1776178325.985217:0:9224:0:(ldlm_lockd.c:2792:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:1.0:1776178326.821566:0:3347:0:(events.c:361:request_in_callback()) peer: 12345-192.168.204.3@tcp (source: 12345-192.168.204.3@tcp) 00000100:00100000:1.0:1776178326.821692:0:3347:0:(events.c:361:request_in_callback()) peer: 12345-192.168.204.3@tcp (source: 12345-192.168.204.3@tcp) 00000100:00100000:1.0:1776178326.821731:0:3347:0:(events.c:361:request_in_callback()) peer: 12345-192.168.204.3@tcp (source: 12345-192.168.204.3@tcp) 00000100:00100000:1.0:1776178326.821763:0:3347:0:(events.c:361:request_in_callback()) peer: 12345-192.168.204.3@tcp (source: 12345-192.168.204.3@tcp) 00000100:00100000:0.0:1776178326.821849:0:20762:0:(service.c:2152:ptlrpc_server_handle_req_in()) got req x1862456979116864 00000100:00100000:0.0:1776178326.821926:0:20762:0:(nrs_fifo.c:179:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.204.3@tcp, seq: 607 00000100:00100000:0.0:1776178326.821934:0:20762:0:(service.c:2324:ptlrpc_server_handle_request()) Handling RPC req@00000000d9d2b484 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_013:b50963bd-8eba-4432-9a54-8888d00e3fa1+8:2272:x1862456979116864:12345-192.168.204.3@tcp:400:kworker/u8:4.0 00000100:00100000:0.0:1776178326.822053:0:20762:0:(service.c:2372:ptlrpc_server_handle_request()) Handled RPC req@00000000d9d2b484 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_013:b50963bd-8eba-4432-9a54-8888d00e3fa1+8:2272:x1862456979116864:12345-192.168.204.3@tcp:400:kworker/u8:4.0 Request processed in 118us (322us total) trans 0 rc 0/0 00000100:00100000:0.0:1776178326.822104:0:20762:0:(nrs_fifo.c:241:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.204.3@tcp, seq: 607 00000100:00100000:2.0:1776178326.822380:0:20140:0:(service.c:2152:ptlrpc_server_handle_req_in()) got req x1862456979116672 00000100:00100000:2.0:1776178326.822424:0:20140:0:(nrs_fifo.c:179:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.204.3@tcp, seq: 203 00000100:00100000:2.0:1776178326.822429:0:20140:0:(service.c:2324:ptlrpc_server_handle_request()) Handling RPC req@00000000ba1c7e2e pname:cluuid+ref:pid:xid:nid:opc:job ll_mgs_0002:5b5c66b1-1a2f-4651-8548-0c75b9a38a94+9:2272:x1862456979116672:12345-192.168.204.3@tcp:400:kworker/u8:4.0 00000100:00100000:2.0:1776178326.822471:0:20140:0:(service.c:2372:ptlrpc_server_handle_request()) Handled RPC req@00000000ba1c7e2e pname:cluuid+ref:pid:xid:nid:opc:job ll_mgs_0002:5b5c66b1-1a2f-4651-8548-0c75b9a38a94+9:2272:x1862456979116672:12345-192.168.204.3@tcp:400:kworker/u8:4.0 Request processed in 42us (780us total) trans 0 rc 0/0 00000100:00100000:2.0:1776178326.822478:0:20140:0:(nrs_fifo.c:241:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.204.3@tcp, seq: 203 00000100:00100000:1.0:1776178326.827307:0:3347:0:(events.c:361:request_in_callback()) peer: 12345-192.168.204.3@tcp (source: 12345-192.168.204.3@tcp) 00000100:00100000:3.0:1776178326.827649:0:6257:0:(service.c:2152:ptlrpc_server_handle_req_in()) got req x1862456979116736 00000100:00100000:0.0:1776178326.827789:0:14440:0:(service.c:2152:ptlrpc_server_handle_req_in()) got req x1862456979116928 00000100:00100000:0.0:1776178326.827863:0:14440:0:(nrs_fifo.c:179:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.204.3@tcp, seq: 608 00000100:00100000:0.0:1776178326.827872:0:14440:0:(service.c:2324:ptlrpc_server_handle_request()) Handling RPC req@000000003334dea7 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_004:b50963bd-8eba-4432-9a54-8888d00e3fa1+9:2273:x1862456979116928:12345-192.168.204.3@tcp:400:kworker/u8:4.0 00000100:00100000:0.0:1776178326.827974:0:14440:0:(service.c:2372:ptlrpc_server_handle_request()) Handled RPC req@000000003334dea7 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_004:b50963bd-8eba-4432-9a54-8888d00e3fa1+9:2273:x1862456979116928:12345-192.168.204.3@tcp:400:kworker/u8:4.0 Request processed in 102us (681us total) trans 0 rc 0/0 00000100:00100000:0.0:1776178326.827983:0:14440:0:(nrs_fifo.c:241:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.204.3@tcp, seq: 608 00000100:00100000:2.0:1776178326.828146:0:11281:0:(service.c:2152:ptlrpc_server_handle_req_in()) got req x1862456979116800 00000100:00100000:2.0:1776178326.828186:0:11281:0:(nrs_fifo.c:179:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.204.3@tcp, seq: 1000 00000100:00100000:2.0:1776178326.828192:0:11281:0:(service.c:2324:ptlrpc_server_handle_request()) Handling RPC req@000000008a1e777f pname:cluuid+ref:pid:xid:nid:opc:job mdt00_004:b50963bd-8eba-4432-9a54-8888d00e3fa1+6:2273:x1862456979116800:12345-192.168.204.3@tcp:400:kworker/u8:4.0 00000100:00100000:2.0:1776178326.828228:0:11281:0:(service.c:2372:ptlrpc_server_handle_request()) Handled RPC req@000000008a1e777f pname:cluuid+ref:pid:xid:nid:opc:job mdt00_004:b50963bd-8eba-4432-9a54-8888d00e3fa1+6:2273:x1862456979116800:12345-192.168.204.3@tcp:400:kworker/u8:4.0 Request processed in 36us (6464us total) trans 0 rc 0/0 00000100:00100000:2.0:1776178326.828635:0:11281:0:(nrs_fifo.c:241:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.204.3@tcp, seq: 1000 00000100:00100000:0.0:1776178326.828643:0:6256:0:(nrs_fifo.c:179:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.204.3@tcp, seq: 6720 00000100:00100000:0.0:1776178326.828650:0:6256:0:(service.c:2324:ptlrpc_server_handle_request()) Handling RPC req@00000000fdfa408a pname:cluuid+ref:pid:xid:nid:opc:job mdt00_001:0+-99:2269:x1862456979116736:12345-192.168.204.3@tcp:38:kworker/u8:4.0 00010000:02000000:0.0:1776178326.828681:0:6256:0:(ldlm_lib.c:1139:target_handle_connect()) lustre-MDT0000: Not available for connect from 192.168.204.3@tcp (stopping) 00000100:00100000:0.0:1776178326.828732:0:6256:0:(service.c:2372:ptlrpc_server_handle_request()) Handled RPC req@00000000fdfa408a pname:cluuid+ref:pid:xid:nid:opc:job mdt00_001:0+-99:2269:x1862456979116736:12345-192.168.204.3@tcp:38:kworker/u8:4.0 Request processed in 82us (7186us total) trans 0 rc -19/-19 00000100:00100000:0.0:1776178326.828739:0:6256:0:(nrs_fifo.c:241:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.204.3@tcp, seq: 6720 00010000:00010000:2.0:1776178327.008478:0:9225:0:(ldlm_lockd.c:2792:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00010000:00010000:2.0:1776178328.032897:0:6244:0:(ldlm_lockd.c:2792:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00080000:3.0:1776178328.417697:0:36:0:(pinger.c:251:ptlrpc_pinger_process_import()) lustre-MDT0000-lwp-OST0000_UUID->lustre-MDT0000_UUID: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00080000:3.0:1776178328.417708:0:36:0:(recover.c:57:ptlrpc_initiate_recovery()) lustre-MDT0000_UUID: starting recovery 00000100:00080000:3.0:1776178328.417713:0:36:0:(import.c:86:import_set_state_nolock()) 0000000047601a98 lustre-MDT0000_UUID: changing import state from DISCONN to CONNECTING 00000100:00080000:3.0:1776178328.417725:0:36:0:(import.c:534:import_select_connection()) lustre-MDT0000-lwp-OST0000: connect to NID 0@lo last attempt 1030 00000100:00080000:3.0:1776178328.417728:0:36:0:(import.c:581:import_select_connection()) lustre-MDT0000-lwp-OST0000: tried all connections, increasing latency to 21s 00000100:00080000:3.0:1776178328.417737:0:36:0:(import.c:615:import_select_connection()) lustre-MDT0000-lwp-OST0000: import 0000000047601a98 using connection 192.168.204.103@tcp/0@lo 00000100:00100000:3.0:1776178328.417764:0:36:0:(import.c:807:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5) req@0000000042ac1e35 x1862456998066496/t0(0) o38->lustre-MDT0000-lwp-OST0000@0@lo:12/10 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' 00000100:00080000:3.0:1776178328.417801:0:36:0:(pinger.c:251:ptlrpc_pinger_process_import()) lustre-MDT0000-lwp-OST0001_UUID->lustre-MDT0000_UUID: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00080000:3.0:1776178328.417804:0:36:0:(recover.c:57:ptlrpc_initiate_recovery()) lustre-MDT0000_UUID: starting recovery 00000100:00080000:3.0:1776178328.417806:0:36:0:(import.c:86:import_set_state_nolock()) 00000000dfc8db13 lustre-MDT0000_UUID: changing import state from DISCONN to CONNECTING 00000100:00080000:3.0:1776178328.417809:0:36:0:(import.c:534:import_select_connection()) lustre-MDT0000-lwp-OST0001: connect to NID 0@lo last attempt 1030 00000100:00080000:3.0:1776178328.417811:0:36:0:(import.c:581:import_select_connection()) lustre-MDT0000-lwp-OST0001: tried all connections, increasing latency to 21s 00000100:00080000:3.0:1776178328.417815:0:36:0:(import.c:615:import_select_connection()) lustre-MDT0000-lwp-OST0001: import 00000000dfc8db13 using connection 192.168.204.103@tcp/0@lo 00000100:00100000:3.0:1776178328.417827:0:36:0:(import.c:807:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5) req@000000008a6fdb8d x1862456998066624/t0(0) o38->lustre-MDT0000-lwp-OST0001@0@lo:12/10 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' 00000100:00080000:3.0:1776178328.417856:0:36:0:(pinger.c:251:ptlrpc_pinger_process_import()) lustre-MDT0001-mdtlov_UUID->lustre-MDT0000_UUID: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00080000:3.0:1776178328.417860:0:36:0:(recover.c:57:ptlrpc_initiate_recovery()) lustre-MDT0000_UUID: starting recovery 00000100:00080000:3.0:1776178328.417862:0:36:0:(import.c:86:import_set_state_nolock()) 00000000ba4f1821 lustre-MDT0000_UUID: changing import state from DISCONN to CONNECTING 00000100:00080000:3.0:1776178328.417865:0:36:0:(import.c:534:import_select_connection()) lustre-MDT0000-osp-MDT0001: connect to NID 0@lo last attempt 1030 00000100:00080000:3.0:1776178328.417867:0:36:0:(import.c:581:import_select_connection()) lustre-MDT0000-osp-MDT0001: tried all connections, increasing latency to 21s 00000100:00080000:3.0:1776178328.417871:0:36:0:(import.c:615:import_select_connection()) lustre-MDT0000-osp-MDT0001: import 00000000ba4f1821 using connection 192.168.204.103@tcp/0@lo 00000100:00100000:3.0:1776178328.417885:0:36:0:(import.c:807:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5) req@0000000008605f3d x1862456998066752/t0(0) o38->lustre-MDT0000-osp-MDT0001@0@lo:24/4 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' 00000100:00080000:3.0:1776178328.417909:0:36:0:(pinger.c:251:ptlrpc_pinger_process_import()) lustre-MDT0000-lwp-MDT0001_UUID->lustre-MDT0000_UUID: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00080000:3.0:1776178328.417912:0:36:0:(recover.c:57:ptlrpc_initiate_recovery()) lustre-MDT0000_UUID: starting recovery 00000100:00080000:3.0:1776178328.417913:0:36:0:(import.c:86:import_set_state_nolock()) 00000000a7c55b1f lustre-MDT0000_UUID: changing import state from DISCONN to CONNECTING 00000100:00080000:3.0:1776178328.417916:0:36:0:(import.c:534:import_select_connection()) lustre-MDT0000-lwp-MDT0001: connect to NID 0@lo last attempt 1030 00000100:00100000:2.0:1776178328.417918:0:3352:0:(client.c:1733:ptlrpc_send_new_req()) Sending RPC req@0000000042ac1e35 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:lustre-MDT0000-lwp-OST0000_UUID:3352:1862456998066496:0@lo:38:kworker/u8:1.0 00000100:00080000:3.0:1776178328.417918:0:36:0:(import.c:581:import_select_connection()) lustre-MDT0000-lwp-MDT0001: tried all connections, increasing latency to 21s 00000100:00080000:3.0:1776178328.417921:0:36:0:(import.c:615:import_select_connection()) lustre-MDT0000-lwp-MDT0001: import 00000000a7c55b1f using connection 192.168.204.103@tcp/0@lo 00000100:00100000:3.0:1776178328.417931:0:36:0:(import.c:807:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5) req@00000000a562d27c x1862456998066944/t0(0) o38->lustre-MDT0000-lwp-MDT0001@0@lo:12/10 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' 00000100:00100000:3.0:1776178328.417952:0:3356:0:(client.c:1733:ptlrpc_send_new_req()) Sending RPC req@0000000087a31eb2 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_03:lustre-MDT0001-mdtlov_UUID:3356:1862456998066816:0@lo:400:kworker/u8:1.0 00000100:00100000:0.0:1776178328.418030:0:3353:0:(client.c:1733:ptlrpc_send_new_req()) Sending RPC req@000000000daa3c9a pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_00:ac61f839-0e8e-4cac-9c59-08bcfbbad330:3353:1862456998066432:0@lo:400:kworker/u8:1.0 00000100:00100000:1.0:1776178328.418113:0:3354:0:(client.c:1733:ptlrpc_send_new_req()) Sending RPC req@00000000dd52d0d0 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_01:lustre-MDT0001-lwp-OST0000_UUID:3354:1862456998066560:0@lo:400:kworker/u8:1.0 00000100:00100000:3.0:1776178328.418202:0:3356:0:(events.c:361:request_in_callback()) peer: 12345-0@lo (source: 12345-0@lo) 00000100:00100000:3.0:1776178328.418255:0:3356:0:(ptlrpcd.c:415:ptlrpcd_check()) transfer 1 async RPCs [2->3] 00000100:00100000:3.0:1776178328.418261:0:3356:0:(client.c:1733:ptlrpc_send_new_req()) Sending RPC req@000000004c88c002 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_03:lustre-MDT0001-lwp-OST0001_UUID:3356:1862456998066688:0@lo:400:kworker/u8:1.0 00000100:00100000:3.0:1776178328.418295:0:3356:0:(events.c:361:request_in_callback()) peer: 12345-0@lo (source: 12345-0@lo) 00000100:00100000:3.0:1776178328.418370:0:6257:0:(service.c:2152:ptlrpc_server_handle_req_in()) got req x1862456998066688 00000100:00100000:3.0:1776178328.418445:0:6257:0:(nrs_fifo.c:179:nrs_fifo_req_get()) NRS start fifo request from 12345-0@lo, seq: 1001 00000100:00100000:3.0:1776178328.418453:0:6257:0:(service.c:2324:ptlrpc_server_handle_request()) Handling RPC req@0000000076452107 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_002:lustre-MDT0001-lwp-OST0001_UUID+5:3356:x1862456998066688:12345-0@lo:400:kworker/u8:1.0 00000100:00100000:3.0:1776178328.418538:0:6257:0:(service.c:2372:ptlrpc_server_handle_request()) Handled RPC req@0000000076452107 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_002:lustre-MDT0001-lwp-OST0001_UUID+5:3356:x1862456998066688:12345-0@lo:400:kworker/u8:1.0 Request processed in 84us (242us total) trans 0 rc 0/0 00000100:00100000:3.0:1776178328.418547:0:6257:0:(nrs_fifo.c:241:nrs_fifo_req_stop()) NRS stop fifo request from 12345-0@lo, seq: 1001 00000100:00100000:3.0:1776178328.418611:0:3356:0:(client.c:2215:ptlrpc_check_set()) Completed RPC req@000000004c88c002 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_03:lustre-MDT0001-lwp-OST0001_UUID:3356:1862456998066688:0@lo:400:kworker/u8:1.0 00000100:00100000:3.0:1776178328.418694:0:20762:0:(service.c:2152:ptlrpc_server_handle_req_in()) got req x1862456998066816 00000100:00100000:3.0:1776178328.418725:0:20762:0:(nrs_fifo.c:179:nrs_fifo_req_get()) NRS start fifo request from 12345-0@lo, seq: 609 00000100:00100000:3.0:1776178328.418730:0:20762:0:(service.c:2324:ptlrpc_server_handle_request()) Handling RPC req@00000000c38303b3 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_013:lustre-MDT0001-mdtlov_UUID+5:3356:x1862456998066816:12345-0@lo:400:kworker/u8:1.0 00000100:00100000:3.0:1776178328.418769:0:20762:0:(service.c:2372:ptlrpc_server_handle_request()) Handled RPC req@00000000c38303b3 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_013:lustre-MDT0001-mdtlov_UUID+5:3356:x1862456998066816:12345-0@lo:400:kworker/u8:1.0 Request processed in 38us (570us total) trans 0 rc 0/0 00000100:00100000:3.0:1776178328.418775:0:20762:0:(nrs_fifo.c:241:nrs_fifo_req_stop()) NRS stop fifo request from 12345-0@lo, seq: 609 00000100:00100000:3.0:1776178328.418798:0:3356:0:(client.c:2215:ptlrpc_check_set()) Completed RPC req@0000000087a31eb2 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_03:lustre-MDT0001-mdtlov_UUID:3356:1862456998066816:0@lo:400:kworker/u8:1.0 00000100:00100000:2.0:1776178328.426123:0:3352:0:(events.c:361:request_in_callback()) peer: 12345-0@lo (source: 12345-0@lo) 00000100:00100000:2.0:1776178328.426250:0:3352:0:(client.c:1733:ptlrpc_send_new_req()) Sending RPC req@000000008a6fdb8d pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:lustre-MDT0000-lwp-OST0001_UUID:3352:1862456998066624:0@lo:38:kworker/u8:1.0 00000100:00100000:2.0:1776178328.426325:0:3352:0:(events.c:361:request_in_callback()) peer: 12345-0@lo (source: 12345-0@lo) 00000100:00100000:2.0:1776178328.426363:0:3352:0:(client.c:1733:ptlrpc_send_new_req()) Sending RPC req@0000000008605f3d pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:lustre-MDT0001-mdtlov_UUID:3352:1862456998066752:0@lo:38:kworker/u8:1.0 00000100:00100000:2.0:1776178328.426406:0:3352:0:(events.c:361:request_in_callback()) peer: 12345-0@lo (source: 12345-0@lo) 00000100:00100000:2.0:1776178328.426452:0:3352:0:(client.c:1733:ptlrpc_send_new_req()) Sending RPC req@00000000a562d27c pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:lustre-MDT0000-lwp-MDT0001_UUID:3352:1862456998066944:0@lo:38:kworker/u8:1.0 00000100:00100000:2.0:1776178328.426483:0:3352:0:(events.c:361:request_in_callback()) peer: 12345-0@lo (source: 12345-0@lo) 00000100:00100000:2.0:1776178328.427350:0:14128:0:(service.c:2152:ptlrpc_server_handle_req_in()) got req x1862456998066752 00000100:00100000:2.0:1776178328.427439:0:14128:0:(nrs_fifo.c:179:nrs_fifo_req_get()) NRS start fifo request from 12345-0@lo, seq: 732 00000100:00100000:2.0:1776178328.427446:0:14128:0:(service.c:2324:ptlrpc_server_handle_request()) Handling RPC req@000000003a3139fc pname:cluuid+ref:pid:xid:nid:opc:job mdt_out00_002:0+-99:3352:x1862456998066752:12345-0@lo:38:kworker/u8:1.0 00010000:02000000:2.0:1776178328.427468:0:14128:0:(ldlm_lib.c:1139:target_handle_connect()) lustre-MDT0000: Not available for connect from 0@lo (stopping) 00000100:00100000:2.0:1776178328.427545:0:14128:0:(service.c:2372:ptlrpc_server_handle_request()) Handled RPC req@000000003a3139fc pname:cluuid+ref:pid:xid:nid:opc:job mdt_out00_002:0+-99:3352:x1862456998066752:12345-0@lo:38:kworker/u8:1.0 Request processed in 98us (1138us total) trans 0 rc -19/-19 00000100:00100000:2.0:1776178328.427555:0:14128:0:(nrs_fifo.c:241:nrs_fifo_req_stop()) NRS stop fifo request from 12345-0@lo, seq: 732 00000100:00100000:2.0:1776178328.427706:0:11281:0:(service.c:2152:ptlrpc_server_handle_req_in()) got req x1862456998066496 00000100:00100000:2.0:1776178328.427731:0:11281:0:(service.c:2152:ptlrpc_server_handle_req_in()) got req x1862456998066624 00000100:00100000:2.0:1776178328.427746:0:11281:0:(service.c:2152:ptlrpc_server_handle_req_in()) got req x1862456998066944 00000100:00100000:2.0:1776178328.427755:0:11281:0:(nrs_fifo.c:179:nrs_fifo_req_get()) NRS start fifo request from 12345-0@lo, seq: 6721 00000100:00100000:2.0:1776178328.427759:0:11281:0:(service.c:2324:ptlrpc_server_handle_request()) Handling RPC req@00000000e82039bb pname:cluuid+ref:pid:xid:nid:opc:job mdt00_004:0+-99:3352:x1862456998066496:12345-0@lo:38:kworker/u8:1.0 00010000:02000000:2.0:1776178328.427766:0:11281:0:(ldlm_lib.c:1139:target_handle_connect()) lustre-MDT0000: Not available for connect from 0@lo (stopping) 00000100:00100000:2.0:1776178328.427793:0:11281:0:(service.c:2372:ptlrpc_server_handle_request()) Handled RPC req@00000000e82039bb pname:cluuid+ref:pid:xid:nid:opc:job mdt00_004:0+-99:3352:x1862456998066496:12345-0@lo:38:kworker/u8:1.0 Request processed in 33us (1683us total) trans 0 rc -19/-19 00000100:00100000:2.0:1776178328.427798:0:11281:0:(nrs_fifo.c:241:nrs_fifo_req_stop()) NRS stop fifo request from 12345-0@lo, seq: 6721 00000100:00100000:2.0:1776178328.427809:0:11281:0:(nrs_fifo.c:179:nrs_fifo_req_get()) NRS start fifo request from 12345-0@lo, seq: 6722 00000100:00100000:2.0:1776178328.427812:0:11281:0:(service.c:2324:ptlrpc_server_handle_request()) Handling RPC req@000000000d203d1d pname:cluuid+ref:pid:xid:nid:opc:job mdt00_004:0+-99:3352:x1862456998066624:12345-0@lo:38:kworker/u8:1.0 00010000:02000000:2.0:1776178328.427817:0:11281:0:(ldlm_lib.c:1139:target_handle_connect()) lustre-MDT0000: Not available for connect from 0@lo (stopping) 00000100:00100000:2.0:1776178328.427837:0:11281:0:(service.c:2372:ptlrpc_server_handle_request()) Handled RPC req@000000000d203d1d pname:cluuid+ref:pid:xid:nid:opc:job mdt00_004:0+-99:3352:x1862456998066624:12345-0@lo:38:kworker/u8:1.0 Request processed in 24us (1513us total) trans 0 rc -19/-19 00000100:00100000:2.0:1776178328.427842:0:11281:0:(nrs_fifo.c:241:nrs_fifo_req_stop()) NRS stop fifo request from 12345-0@lo, seq: 6722 00000100:00100000:0.0:1776178328.428385:0:3353:0:(events.c:361:request_in_callback()) peer: 12345-0@lo (source: 12345-0@lo) 00000100:00100000:0.0:1776178328.428424:0:3353:0:(client.c:1733:ptlrpc_send_new_req()) Sending RPC req@00000000f62fe06f pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_00:lustre-MDT0001-mdtlov_UUID:3353:1862456998066880:0@lo:400:kworker/u8:1.0 00000100:00100000:0.0:1776178328.428458:0:3353:0:(events.c:361:request_in_callback()) peer: 12345-0@lo (source: 12345-0@lo) 00000100:00100000:0.0:1776178328.428504:0:14440:0:(service.c:2152:ptlrpc_server_handle_req_in()) got req x1862456998066880 00000100:00100000:0.0:1776178328.428541:0:14440:0:(nrs_fifo.c:179:nrs_fifo_req_get()) NRS start fifo request from 12345-0@lo, seq: 610 00000100:00100000:0.0:1776178328.428545:0:14440:0:(service.c:2324:ptlrpc_server_handle_request()) Handling RPC req@000000006342fd4b pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_004:lustre-MDT0001-mdtlov_UUID+5:3353:x1862456998066880:12345-0@lo:400:kworker/u8:1.0 00000100:00100000:0.0:1776178328.428596:0:14440:0:(service.c:2372:ptlrpc_server_handle_request()) Handled RPC req@000000006342fd4b pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_004:lustre-MDT0001-mdtlov_UUID+5:3353:x1862456998066880:12345-0@lo:400:kworker/u8:1.0 Request processed in 50us (137us total) trans 0 rc 0/0 00000100:00100000:0.0:1776178328.428606:0:14440:0:(nrs_fifo.c:241:nrs_fifo_req_stop()) NRS stop fifo request from 12345-0@lo, seq: 610 00000100:00100000:0.0:1776178328.428630:0:20137:0:(service.c:2152:ptlrpc_server_handle_req_in()) got req x1862456998066432 00000100:00100000:0.0:1776178328.428658:0:20137:0:(nrs_fifo.c:179:nrs_fifo_req_get()) NRS start fifo request from 12345-0@lo, seq: 204 00000100:00100000:0.0:1776178328.428661:0:20137:0:(service.c:2324:ptlrpc_server_handle_request()) Handling RPC req@000000005db97027 pname:cluuid+ref:pid:xid:nid:opc:job ll_mgs_0000:ac61f839-0e8e-4cac-9c59-08bcfbbad330+18:3353:x1862456998066432:12345-0@lo:400:kworker/u8:1.0 00000100:00100000:0.0:1776178328.428694:0:20137:0:(service.c:2372:ptlrpc_server_handle_request()) Handled RPC req@000000005db97027 pname:cluuid+ref:pid:xid:nid:opc:job ll_mgs_0000:ac61f839-0e8e-4cac-9c59-08bcfbbad330+18:3353:x1862456998066432:12345-0@lo:400:kworker/u8:1.0 Request processed in 31us (312us total) trans 0 rc 0/0 00000100:00100000:0.0:1776178328.428700:0:20137:0:(nrs_fifo.c:241:nrs_fifo_req_stop()) NRS stop fifo request from 12345-0@lo, seq: 204 00000100:00100000:0.0:1776178328.428756:0:3353:0:(client.c:2215:ptlrpc_check_set()) Completed RPC req@000000000daa3c9a pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_00:ac61f839-0e8e-4cac-9c59-08bcfbbad330:3353:1862456998066432:0@lo:400:kworker/u8:1.0 00000100:00100000:0.0:1776178328.428771:0:3353:0:(client.c:2215:ptlrpc_check_set()) Completed RPC req@00000000f62fe06f pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_00:lustre-MDT0001-mdtlov_UUID:3353:1862456998066880:0@lo:400:kworker/u8:1.0 00000100:00080000:0.0:1776178328.428842:0:3352:0:(import.c:86:import_set_state_nolock()) 0000000047601a98 lustre-MDT0000_UUID: changing import state from CONNECTING to DISCONN 00000100:00080000:0.0:1776178328.428847:0:3352:0:(import.c:1429:ptlrpc_connect_interpret()) recovery of lustre-MDT0000_UUID on 192.168.204.103@tcp failed (-19) 00000100:00100000:0.0:1776178328.428851:0:3352:0:(client.c:2215:ptlrpc_check_set()) Completed RPC req@0000000042ac1e35 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:lustre-MDT0000-lwp-OST0000_UUID:3352:1862456998066496:0@lo:38:kworker/u8:1.0 00000100:00080000:0.0:1776178328.428862:0:3352:0:(import.c:86:import_set_state_nolock()) 00000000dfc8db13 lustre-MDT0000_UUID: changing import state from CONNECTING to DISCONN 00000100:00080000:0.0:1776178328.428865:0:3352:0:(import.c:1429:ptlrpc_connect_interpret()) recovery of lustre-MDT0000_UUID on 192.168.204.103@tcp failed (-19) 00000100:00100000:0.0:1776178328.428867:0:3352:0:(client.c:2215:ptlrpc_check_set()) Completed RPC req@000000008a6fdb8d pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:lustre-MDT0000-lwp-OST0001_UUID:3352:1862456998066624:0@lo:38:kworker/u8:1.0 00000100:00080000:0.0:1776178328.428878:0:3352:0:(import.c:86:import_set_state_nolock()) 00000000ba4f1821 lustre-MDT0000_UUID: changing import state from CONNECTING to DISCONN 00000100:00080000:0.0:1776178328.428881:0:3352:0:(import.c:1429:ptlrpc_connect_interpret()) recovery of lustre-MDT0000_UUID on 192.168.204.103@tcp failed (-19) 00000100:00100000:0.0:1776178328.428883:0:3352:0:(client.c:2215:ptlrpc_check_set()) Completed RPC req@0000000008605f3d pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:lustre-MDT0001-mdtlov_UUID:3352:1862456998066752:0@lo:38:kworker/u8:1.0 00000100:00100000:3.0:1776178328.431164:0:6256:0:(nrs_fifo.c:179:nrs_fifo_req_get()) NRS start fifo request from 12345-0@lo, seq: 6723 00000100:00100000:3.0:1776178328.431181:0:6256:0:(service.c:2324:ptlrpc_server_handle_request()) Handling RPC req@000000002c6408bc pname:cluuid+ref:pid:xid:nid:opc:job mdt00_001:0+-99:3352:x1862456998066944:12345-0@lo:38:kworker/u8:1.0 00010000:02000000:3.0:1776178328.431206:0:6256:0:(ldlm_lib.c:1139:target_handle_connect()) lustre-MDT0000: Not available for connect from 0@lo (stopping) 00000100:00100000:3.0:1776178328.431323:0:6256:0:(service.c:2372:ptlrpc_server_handle_request()) Handled RPC req@000000002c6408bc pname:cluuid+ref:pid:xid:nid:opc:job mdt00_001:0+-99:3352:x1862456998066944:12345-0@lo:38:kworker/u8:1.0 Request processed in 145us (4839us total) trans 0 rc -19/-19 00000100:00100000:3.0:1776178328.431341:0:6256:0:(nrs_fifo.c:241:nrs_fifo_req_stop()) NRS stop fifo request from 12345-0@lo, seq: 6723 00000100:00080000:3.0:1776178328.431437:0:3352:0:(import.c:86:import_set_state_nolock()) 00000000a7c55b1f lustre-MDT0000_UUID: changing import state from CONNECTING to DISCONN 00000100:00080000:3.0:1776178328.431442:0:3352:0:(import.c:1429:ptlrpc_connect_interpret()) recovery of lustre-MDT0000_UUID on 192.168.204.103@tcp failed (-19) 00000100:00100000:3.0:1776178328.431448:0:3352:0:(client.c:2215:ptlrpc_check_set()) Completed RPC req@00000000a562d27c pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:lustre-MDT0000-lwp-MDT0001_UUID:3352:1862456998066944:0@lo:38:kworker/u8:1.0 00000100:00100000:1.0:1776178328.435723:0:3354:0:(events.c:361:request_in_callback()) peer: 12345-0@lo (source: 12345-0@lo) 00000100:00100000:2.0:1776178328.436272:0:11281:0:(service.c:2152:ptlrpc_server_handle_req_in()) got req x1862456998066560 00000100:00100000:2.0:1776178328.436382:0:11281:0:(nrs_fifo.c:179:nrs_fifo_req_get()) NRS start fifo request from 12345-0@lo, seq: 1002 00000100:00100000:2.0:1776178328.436392:0:11281:0:(service.c:2324:ptlrpc_server_handle_request()) Handling RPC req@0000000015095c30 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_004:lustre-MDT0001-lwp-OST0000_UUID+5:3354:x1862456998066560:12345-0@lo:400:kworker/u8:1.0 00000100:00100000:2.0:1776178328.436937:0:11281:0:(service.c:2372:ptlrpc_server_handle_request()) Handled RPC req@0000000015095c30 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_004:lustre-MDT0001-lwp-OST0000_UUID+5:3354:x1862456998066560:12345-0@lo:400:kworker/u8:1.0 Request processed in 546us (1233us total) trans 0 rc 0/0 00000100:00100000:2.0:1776178328.436946:0:11281:0:(nrs_fifo.c:241:nrs_fifo_req_stop()) NRS stop fifo request from 12345-0@lo, seq: 1002 00000100:00100000:1.0:1776178328.441128:0:3354:0:(client.c:2215:ptlrpc_check_set()) Completed RPC req@00000000dd52d0d0 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_01:lustre-MDT0001-lwp-OST0000_UUID:3354:1862456998066560:0@lo:400:kworker/u8:1.0 00010000:00010000:2.0:1776178329.056772:0:6245:0:(ldlm_lockd.c:2792:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:1.0:1776178329.953344:0:3354:0:(client.c:1733:ptlrpc_send_new_req()) Sending RPC req@00000000d384e871 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_01:lustre-MDT0001-mdtlov_UUID:3354:1862456998067008:0@lo:13:osp-pre-1-1.0 00000100:00100000:1.0:1776178329.953484:0:3354:0:(events.c:361:request_in_callback()) peer: 12345-0@lo (source: 12345-0@lo) 00000100:00100000:2.0:1776178329.953587:0:9554:0:(service.c:2152:ptlrpc_server_handle_req_in()) got req x1862456998067008 00000100:00100000:2.0:1776178329.954467:0:9554:0:(nrs_fifo.c:179:nrs_fifo_req_get()) NRS start fifo request from 12345-0@lo, seq: 537 00000100:00100000:2.0:1776178329.954478:0:9554:0:(service.c:2324:ptlrpc_server_handle_request()) Handling RPC req@000000009de51106 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_create00:lustre-MDT0001-mdtlov_UUID+5:3354:x1862456998067008:12345-0@lo:13:osp-pre-1-1.0 00002000:00000024:2.0:1776178329.954523:0:9554:0:(ofd_obd.c:740:ofd_statfs()) blocks cached 229376 granted 491746368 pending 0 free 3923689472 avail 3692167168 00000100:00100000:3.0:1776178329.955284:0:3354:0:(client.c:2215:ptlrpc_check_set()) Completed RPC req@00000000d384e871 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_01:lustre-MDT0001-mdtlov_UUID:3354:1862456998067008:0@lo:13:osp-pre-1-1.0 00000100:00100000:2.0:1776178329.956745:0:9554:0:(service.c:2372:ptlrpc_server_handle_request()) Handled RPC req@000000009de51106 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_create00:lustre-MDT0001-mdtlov_UUID+5:3354:x1862456998067008:12345-0@lo:13:osp-pre-1-1.0 Request processed in 2262us (3258us total) trans 0 rc 0/0 00000100:00100000:2.0:1776178329.956758:0:9554:0:(nrs_fifo.c:241:nrs_fifo_req_stop()) NRS stop fifo request from 12345-0@lo, seq: 537 00010000:00010000:2.0:1776178330.082591:0:9224:0:(ldlm_lockd.c:2792:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:1.0:1776178330.470817:0:3356:0:(ptlrpcd.c:415:ptlrpcd_check()) transfer 1 async RPCs [2->3] 00000100:00100000:1.0:1776178330.470853:0:3356:0:(client.c:1733:ptlrpc_send_new_req()) Sending RPC req@00000000c324c57b pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_03:lustre-MDT0001-mdtlov_UUID:3356:1862456998067072:0@lo:13:osp-pre-0-1.0 00000100:00100000:1.0:1776178330.470938:0:3356:0:(events.c:361:request_in_callback()) peer: 12345-0@lo (source: 12345-0@lo) 00000100:00100000:2.0:1776178330.471024:0:9554:0:(service.c:2152:ptlrpc_server_handle_req_in()) got req x1862456998067072 00000100:00100000:2.0:1776178330.471101:0:9554:0:(nrs_fifo.c:179:nrs_fifo_req_get()) NRS start fifo request from 12345-0@lo, seq: 538 00000100:00100000:2.0:1776178330.471109:0:9554:0:(service.c:2324:ptlrpc_server_handle_request()) Handling RPC req@00000000af9e3283 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_create00:lustre-MDT0001-mdtlov_UUID+5:3356:x1862456998067072:12345-0@lo:13:osp-pre-0-1.0 00002000:00000024:2.0:1776178330.471144:0:9554:0:(ofd_obd.c:740:ofd_statfs()) blocks cached 315392 granted 484883776 pending 0 free 3923689472 avail 3692167168 00000100:00100000:2.0:1776178330.471204:0:9554:0:(service.c:2372:ptlrpc_server_handle_request()) Handled RPC req@00000000af9e3283 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_create00:lustre-MDT0001-mdtlov_UUID+5:3356:x1862456998067072:12345-0@lo:13:osp-pre-0-1.0 Request processed in 95us (268us total) trans 0 rc 0/0 00000100:00100000:2.0:1776178330.471211:0:9554:0:(nrs_fifo.c:241:nrs_fifo_req_stop()) NRS stop fifo request from 12345-0@lo, seq: 538 00000100:00100000:1.0:1776178330.471269:0:3356:0:(client.c:2215:ptlrpc_check_set()) Completed RPC req@00000000c324c57b pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_03:lustre-MDT0001-mdtlov_UUID:3356:1862456998067072:0@lo:13:osp-pre-0-1.0 00010000:00010000:2.0:1776178331.104374:0:9225:0:(ldlm_lockd.c:2792:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:1.0:1776178331.946130:0:3347:0:(events.c:361:request_in_callback()) peer: 12345-192.168.204.3@tcp (source: 12345-192.168.204.3@tcp) 00000100:00100000:1.0:1776178331.946289:0:3347:0:(events.c:361:request_in_callback()) peer: 12345-192.168.204.3@tcp (source: 12345-192.168.204.3@tcp) 00000100:00100000:1.0:1776178331.946333:0:3347:0:(events.c:361:request_in_callback()) peer: 12345-192.168.204.3@tcp (source: 12345-192.168.204.3@tcp) 00000100:00100000:1.0:1776178331.946365:0:3347:0:(events.c:361:request_in_callback()) peer: 12345-192.168.204.3@tcp (source: 12345-192.168.204.3@tcp) 00000100:00100000:2.0:1776178331.946454:0:20140:0:(service.c:2152:ptlrpc_server_handle_req_in()) got req x1862456979116992 00000100:00100000:2.0:1776178331.946524:0:20140:0:(nrs_fifo.c:179:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.204.3@tcp, seq: 205 00000100:00100000:2.0:1776178331.946534:0:20140:0:(service.c:2324:ptlrpc_server_handle_request()) Handling RPC req@0000000017512ce0 pname:cluuid+ref:pid:xid:nid:opc:job ll_mgs_0002:5b5c66b1-1a2f-4651-8548-0c75b9a38a94+9:2272:x1862456979116992:12345-192.168.204.3@tcp:400:kworker/u8:3.0 00000100:00100000:2.0:1776178331.946635:0:20140:0:(service.c:2372:ptlrpc_server_handle_request()) Handled RPC req@0000000017512ce0 pname:cluuid+ref:pid:xid:nid:opc:job ll_mgs_0002:5b5c66b1-1a2f-4651-8548-0c75b9a38a94+9:2272:x1862456979116992:12345-192.168.204.3@tcp:400:kworker/u8:3.0 Request processed in 100us (346us total) trans 0 rc 0/0 00000100:00100000:2.0:1776178331.946645:0:20140:0:(nrs_fifo.c:241:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.204.3@tcp, seq: 205 00000100:00100000:3.0:1776178331.950803:0:20762:0:(service.c:2152:ptlrpc_server_handle_req_in()) got req x1862456979117184 00000100:00100000:3.0:1776178331.950903:0:20762:0:(nrs_fifo.c:179:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.204.3@tcp, seq: 611 00000100:00100000:3.0:1776178331.950911:0:20762:0:(service.c:2324:ptlrpc_server_handle_request()) Handling RPC req@0000000038c09127 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_013:b50963bd-8eba-4432-9a54-8888d00e3fa1+8:2272:x1862456979117184:12345-192.168.204.3@tcp:400:kworker/u8:3.0 00000100:00100000:3.0:1776178331.951014:0:20762:0:(service.c:2372:ptlrpc_server_handle_request()) Handled RPC req@0000000038c09127 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_013:b50963bd-8eba-4432-9a54-8888d00e3fa1+8:2272:x1862456979117184:12345-192.168.204.3@tcp:400:kworker/u8:3.0 Request processed in 103us (4680us total) trans 0 rc 0/0 00000100:00100000:3.0:1776178331.951023:0:20762:0:(nrs_fifo.c:241:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.204.3@tcp, seq: 611 00000100:00100000:0.0:1776178331.964550:0:6257:0:(service.c:2152:ptlrpc_server_handle_req_in()) got req x1862456979117056 00000100:00100000:1.0:1776178331.964650:0:3347:0:(events.c:361:request_in_callback()) peer: 12345-192.168.204.3@tcp (source: 12345-192.168.204.3@tcp) 00000100:00100000:0.0:1776178331.964692:0:6257:0:(service.c:2152:ptlrpc_server_handle_req_in()) got req x1862456979117120 00000100:00100000:1.0:1776178331.964726:0:14440:0:(service.c:2152:ptlrpc_server_handle_req_in()) got req x1862456979117248 00000100:00100000:1.0:1776178331.964760:0:14440:0:(nrs_fifo.c:179:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.204.3@tcp, seq: 612 00000100:00100000:1.0:1776178331.964767:0:14440:0:(service.c:2324:ptlrpc_server_handle_request()) Handling RPC req@00000000bcefda18 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_004:b50963bd-8eba-4432-9a54-8888d00e3fa1+9:2272:x1862456979117248:12345-192.168.204.3@tcp:400:kworker/u8:3.0 00000100:00100000:1.0:1776178331.964852:0:14440:0:(service.c:2372:ptlrpc_server_handle_request()) Handled RPC req@00000000bcefda18 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_004:b50963bd-8eba-4432-9a54-8888d00e3fa1+9:2272:x1862456979117248:12345-192.168.204.3@tcp:400:kworker/u8:3.0 Request processed in 85us (209us total) trans 0 rc 0/0 00000100:00100000:1.0:1776178331.964860:0:14440:0:(nrs_fifo.c:241:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.204.3@tcp, seq: 612 00000100:00100000:2.0:1776178331.969607:0:6255:0:(nrs_fifo.c:179:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.204.3@tcp, seq: 1003 00000100:00100000:2.0:1776178331.969635:0:6255:0:(service.c:2324:ptlrpc_server_handle_request()) Handling RPC req@00000000d6e5d9ba pname:cluuid+ref:pid:xid:nid:opc:job mdt00_000:b50963bd-8eba-4432-9a54-8888d00e3fa1+6:2272:x1862456979117120:12345-192.168.204.3@tcp:400:kworker/u8:3.0 00000100:00100000:2.0:1776178331.969748:0:6255:0:(service.c:2372:ptlrpc_server_handle_request()) Handled RPC req@00000000d6e5d9ba pname:cluuid+ref:pid:xid:nid:opc:job mdt00_000:b50963bd-8eba-4432-9a54-8888d00e3fa1+6:2272:x1862456979117120:12345-192.168.204.3@tcp:400:kworker/u8:3.0 Request processed in 117us (23383us total) trans 0 rc 0/0 00000100:00100000:2.0:1776178331.969764:0:6255:0:(nrs_fifo.c:241:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.204.3@tcp, seq: 1003 00000100:00100000:2.0:1776178331.969800:0:6255:0:(nrs_fifo.c:179:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.204.3@tcp, seq: 6724 00000100:00100000:2.0:1776178331.969807:0:6255:0:(service.c:2324:ptlrpc_server_handle_request()) Handling RPC req@00000000c63a98c1 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_000:0+-99:2269:x1862456979117056:12345-192.168.204.3@tcp:38:kworker/u8:3.0 00010000:02000000:2.0:1776178331.969824:0:6255:0:(ldlm_lib.c:1139:target_handle_connect()) lustre-MDT0000: Not available for connect from 192.168.204.3@tcp (stopping) 00000100:00100000:2.0:1776178331.969855:0:6255:0:(service.c:2372:ptlrpc_server_handle_request()) Handled RPC req@00000000c63a98c1 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_000:0+-99:2269:x1862456979117056:12345-192.168.204.3@tcp:38:kworker/u8:3.0 Request processed in 48us (23740us total) trans 0 rc -19/-19 00000100:00100000:2.0:1776178331.969862:0:6255:0:(nrs_fifo.c:241:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.204.3@tcp, seq: 6724 00010000:00010000:2.0:1776178332.128363:0:6244:0:(ldlm_lockd.c:2792:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00010000:00010000:2.0:1776178333.152573:0:6245:0:(ldlm_lockd.c:2792:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00080000:3.0:1776178333.536825:0:36:0:(pinger.c:251:ptlrpc_pinger_process_import()) lustre-MDT0000-lwp-OST0000_UUID->lustre-MDT0000_UUID: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00080000:3.0:1776178333.536834:0:36:0:(recover.c:57:ptlrpc_initiate_recovery()) lustre-MDT0000_UUID: starting recovery 00000100:00080000:3.0:1776178333.536837:0:36:0:(import.c:86:import_set_state_nolock()) 0000000047601a98 lustre-MDT0000_UUID: changing import state from DISCONN to CONNECTING 00000100:00080000:3.0:1776178333.536846:0:36:0:(import.c:534:import_select_connection()) lustre-MDT0000-lwp-OST0000: connect to NID 0@lo last attempt 1035 00000100:00080000:3.0:1776178333.536852:0:36:0:(import.c:581:import_select_connection()) lustre-MDT0000-lwp-OST0000: tried all connections, increasing latency to 21s 00000100:00080000:3.0:1776178333.536858:0:36:0:(import.c:615:import_select_connection()) lustre-MDT0000-lwp-OST0000: import 0000000047601a98 using connection 192.168.204.103@tcp/0@lo 00000100:00100000:3.0:1776178333.536880:0:36:0:(import.c:807:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5) req@000000007216887e x1862456998067200/t0(0) o38->lustre-MDT0000-lwp-OST0000@0@lo:12/10 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' 00000100:00080000:3.0:1776178333.536927:0:36:0:(pinger.c:251:ptlrpc_pinger_process_import()) lustre-MDT0000-lwp-OST0001_UUID->lustre-MDT0000_UUID: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00080000:3.0:1776178333.536930:0:36:0:(recover.c:57:ptlrpc_initiate_recovery()) lustre-MDT0000_UUID: starting recovery 00000100:00080000:3.0:1776178333.536932:0:36:0:(import.c:86:import_set_state_nolock()) 00000000dfc8db13 lustre-MDT0000_UUID: changing import state from DISCONN to CONNECTING 00000100:00080000:3.0:1776178333.536935:0:36:0:(import.c:534:import_select_connection()) lustre-MDT0000-lwp-OST0001: connect to NID 0@lo last attempt 1035 00000100:00080000:3.0:1776178333.536938:0:36:0:(import.c:581:import_select_connection()) lustre-MDT0000-lwp-OST0001: tried all connections, increasing latency to 21s 00000100:00080000:3.0:1776178333.536940:0:36:0:(import.c:615:import_select_connection()) lustre-MDT0000-lwp-OST0001: import 00000000dfc8db13 using connection 192.168.204.103@tcp/0@lo 00000100:00100000:3.0:1776178333.536953:0:36:0:(import.c:807:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5) req@000000000a0451ff x1862456998067328/t0(0) o38->lustre-MDT0000-lwp-OST0001@0@lo:12/10 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' 00000100:00080000:3.0:1776178333.536967:0:36:0:(pinger.c:251:ptlrpc_pinger_process_import()) lustre-MDT0001-mdtlov_UUID->lustre-MDT0000_UUID: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00080000:3.0:1776178333.536970:0:36:0:(recover.c:57:ptlrpc_initiate_recovery()) lustre-MDT0000_UUID: starting recovery 00000100:00080000:3.0:1776178333.536971:0:36:0:(import.c:86:import_set_state_nolock()) 00000000ba4f1821 lustre-MDT0000_UUID: changing import state from DISCONN to CONNECTING 00000100:00080000:3.0:1776178333.536975:0:36:0:(import.c:534:import_select_connection()) lustre-MDT0000-osp-MDT0001: connect to NID 0@lo last attempt 1035 00000100:00080000:3.0:1776178333.536976:0:36:0:(import.c:581:import_select_connection()) lustre-MDT0000-osp-MDT0001: tried all connections, increasing latency to 21s 00000100:00080000:3.0:1776178333.536980:0:36:0:(import.c:615:import_select_connection()) lustre-MDT0000-osp-MDT0001: import 00000000ba4f1821 using connection 192.168.204.103@tcp/0@lo 00000100:00100000:3.0:1776178333.536993:0:36:0:(import.c:807:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5) req@0000000075189f40 x1862456998067456/t0(0) o38->lustre-MDT0000-osp-MDT0001@0@lo:24/4 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' 00000100:00080000:3.0:1776178333.537013:0:36:0:(pinger.c:251:ptlrpc_pinger_process_import()) lustre-MDT0000-lwp-MDT0001_UUID->lustre-MDT0000_UUID: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00080000:3.0:1776178333.537017:0:36:0:(recover.c:57:ptlrpc_initiate_recovery()) lustre-MDT0000_UUID: starting recovery 00000100:00080000:3.0:1776178333.537019:0:36:0:(import.c:86:import_set_state_nolock()) 00000000a7c55b1f lustre-MDT0000_UUID: changing import state from DISCONN to CONNECTING 00000100:00080000:3.0:1776178333.537022:0:36:0:(import.c:534:import_select_connection()) lustre-MDT0000-lwp-MDT0001: connect to NID 0@lo last attempt 1035 00000100:00080000:3.0:1776178333.537024:0:36:0:(import.c:581:import_select_connection()) lustre-MDT0000-lwp-MDT0001: tried all connections, increasing latency to 21s 00000100:00080000:3.0:1776178333.537029:0:36:0:(import.c:615:import_select_connection()) lustre-MDT0000-lwp-MDT0001: import 00000000a7c55b1f using connection 192.168.204.103@tcp/0@lo 00000100:00100000:3.0:1776178333.537039:0:36:0:(import.c:807:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5) req@0000000029ef3988 x1862456998067648/t0(0) o38->lustre-MDT0000-lwp-MDT0001@0@lo:12/10 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' 00000100:00100000:3.0:1776178333.537104:0:3354:0:(client.c:1733:ptlrpc_send_new_req()) Sending RPC req@0000000037aace32 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_01:lustre-MDT0001-lwp-OST0001_UUID:3354:1862456998067392:0@lo:400:kworker/u8:1.0 00000100:00100000:2.0:1776178333.537338:0:3355:0:(client.c:1733:ptlrpc_send_new_req()) Sending RPC req@000000009c4a59b3 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_02:lustre-MDT0001-mdtlov_UUID:3355:1862456998067520:0@lo:400:kworker/u8:1.0 00000100:00100000:2.0:1776178333.537419:0:3355:0:(events.c:361:request_in_callback()) peer: 12345-0@lo (source: 12345-0@lo) 00000100:00100000:2.0:1776178333.537469:0:3355:0:(ptlrpcd.c:415:ptlrpcd_check()) transfer 2 async RPCs [3->2] 00000100:00100000:2.0:1776178333.537476:0:3355:0:(client.c:1733:ptlrpc_send_new_req()) Sending RPC req@00000000be2fd98d pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_02:ac61f839-0e8e-4cac-9c59-08bcfbbad330:3355:1862456998067136:0@lo:400:kworker/u8:1.0 00000100:00100000:2.0:1776178333.537506:0:3355:0:(events.c:361:request_in_callback()) peer: 12345-0@lo (source: 12345-0@lo) 00000100:00100000:2.0:1776178333.537534:0:3355:0:(client.c:1733:ptlrpc_send_new_req()) Sending RPC req@000000002300ff10 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_02:lustre-MDT0001-mdtlov_UUID:3355:1862456998067584:0@lo:400:kworker/u8:1.0 00000100:00100000:2.0:1776178333.537557:0:3355:0:(events.c:361:request_in_callback()) peer: 12345-0@lo (source: 12345-0@lo) 00000100:00100000:2.0:1776178333.537604:0:20137:0:(service.c:2152:ptlrpc_server_handle_req_in()) got req x1862456998067136 00000100:00100000:2.0:1776178333.537663:0:20137:0:(nrs_fifo.c:179:nrs_fifo_req_get()) NRS start fifo request from 12345-0@lo, seq: 206 00000100:00100000:2.0:1776178333.537672:0:20137:0:(service.c:2324:ptlrpc_server_handle_request()) Handling RPC req@000000004d415026 pname:cluuid+ref:pid:xid:nid:opc:job ll_mgs_0000:ac61f839-0e8e-4cac-9c59-08bcfbbad330+18:3355:x1862456998067136:12345-0@lo:400:kworker/u8:1.0 00000100:00100000:2.0:1776178333.537739:0:20137:0:(service.c:2372:ptlrpc_server_handle_request()) Handled RPC req@000000004d415026 pname:cluuid+ref:pid:xid:nid:opc:job ll_mgs_0000:ac61f839-0e8e-4cac-9c59-08bcfbbad330+18:3355:x1862456998067136:12345-0@lo:400:kworker/u8:1.0 Request processed in 67us (232us total) trans 0 rc 0/0 00000100:00100000:2.0:1776178333.537746:0:20137:0:(nrs_fifo.c:241:nrs_fifo_req_stop()) NRS stop fifo request from 12345-0@lo, seq: 206 00000100:00100000:2.0:1776178333.537784:0:14440:0:(service.c:2152:ptlrpc_server_handle_req_in()) got req x1862456998067520 00000100:00100000:2.0:1776178333.537812:0:14440:0:(service.c:2152:ptlrpc_server_handle_req_in()) got req x1862456998067584 00000100:00100000:2.0:1776178333.537831:0:14440:0:(nrs_fifo.c:179:nrs_fifo_req_get()) NRS start fifo request from 12345-0@lo, seq: 613 00000100:00100000:2.0:1776178333.537834:0:14440:0:(service.c:2324:ptlrpc_server_handle_request()) Handling RPC req@00000000664eb80b pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_004:lustre-MDT0001-mdtlov_UUID+5:3355:x1862456998067520:12345-0@lo:400:kworker/u8:1.0 00000100:00100000:2.0:1776178333.537869:0:14440:0:(service.c:2372:ptlrpc_server_handle_request()) Handled RPC req@00000000664eb80b pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_004:lustre-MDT0001-mdtlov_UUID+5:3355:x1862456998067520:12345-0@lo:400:kworker/u8:1.0 Request processed in 33us (451us total) trans 0 rc 0/0 00000100:00100000:2.0:1776178333.537878:0:14440:0:(nrs_fifo.c:241:nrs_fifo_req_stop()) NRS stop fifo request from 12345-0@lo, seq: 613 00000100:00100000:2.0:1776178333.537888:0:14440:0:(nrs_fifo.c:179:nrs_fifo_req_get()) NRS start fifo request from 12345-0@lo, seq: 614 00000100:00100000:2.0:1776178333.537891:0:14440:0:(service.c:2324:ptlrpc_server_handle_request()) Handling RPC req@0000000083b11d73 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_004:lustre-MDT0001-mdtlov_UUID+5:3355:x1862456998067584:12345-0@lo:400:kworker/u8:1.0 00000100:00100000:2.0:1776178333.537916:0:14440:0:(service.c:2372:ptlrpc_server_handle_request()) Handled RPC req@0000000083b11d73 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_004:lustre-MDT0001-mdtlov_UUID+5:3355:x1862456998067584:12345-0@lo:400:kworker/u8:1.0 Request processed in 24us (358us total) trans 0 rc 0/0 00000100:00100000:2.0:1776178333.537922:0:14440:0:(nrs_fifo.c:241:nrs_fifo_req_stop()) NRS stop fifo request from 12345-0@lo, seq: 614 00000100:00100000:2.0:1776178333.537977:0:3355:0:(client.c:2215:ptlrpc_check_set()) Completed RPC req@00000000be2fd98d pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_02:ac61f839-0e8e-4cac-9c59-08bcfbbad330:3355:1862456998067136:0@lo:400:kworker/u8:1.0 00000100:00100000:2.0:1776178333.537991:0:3355:0:(client.c:2215:ptlrpc_check_set()) Completed RPC req@000000002300ff10 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_02:lustre-MDT0001-mdtlov_UUID:3355:1862456998067584:0@lo:400:kworker/u8:1.0 00000100:00100000:2.0:1776178333.538000:0:3355:0:(client.c:2215:ptlrpc_check_set()) Completed RPC req@000000009c4a59b3 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_02:lustre-MDT0001-mdtlov_UUID:3355:1862456998067520:0@lo:400:kworker/u8:1.0 00000100:00100000:3.0:1776178333.538907:0:3354:0:(events.c:361:request_in_callback()) peer: 12345-0@lo (source: 12345-0@lo) 00000100:00100000:3.0:1776178333.538955:0:3354:0:(ptlrpcd.c:415:ptlrpcd_check()) transfer 1 async RPCs [0->1] 00000100:00100000:3.0:1776178333.538967:0:3354:0:(client.c:1733:ptlrpc_send_new_req()) Sending RPC req@00000000542ce52f pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_01:lustre-MDT0001-lwp-OST0000_UUID:3354:1862456998067264:0@lo:400:kworker/u8:1.0 00000100:00100000:3.0:1776178333.539014:0:3354:0:(events.c:361:request_in_callback()) peer: 12345-0@lo (source: 12345-0@lo) 00000100:00100000:0.0:1776178333.539137:0:3352:0:(client.c:1733:ptlrpc_send_new_req()) Sending RPC req@000000007216887e pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:lustre-MDT0000-lwp-OST0000_UUID:3352:1862456998067200:0@lo:38:kworker/u8:1.0 00000100:00100000:0.0:1776178333.539186:0:3352:0:(events.c:361:request_in_callback()) peer: 12345-0@lo (source: 12345-0@lo) 00000100:00100000:0.0:1776178333.539221:0:3352:0:(client.c:1733:ptlrpc_send_new_req()) Sending RPC req@000000000a0451ff pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:lustre-MDT0000-lwp-OST0001_UUID:3352:1862456998067328:0@lo:38:kworker/u8:1.0 00000100:00100000:0.0:1776178333.539253:0:3352:0:(events.c:361:request_in_callback()) peer: 12345-0@lo (source: 12345-0@lo) 00000100:00100000:0.0:1776178333.539268:0:3352:0:(client.c:1733:ptlrpc_send_new_req()) Sending RPC req@0000000075189f40 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:lustre-MDT0001-mdtlov_UUID:3352:1862456998067456:0@lo:38:kworker/u8:1.0 00000100:00100000:0.0:1776178333.539301:0:3352:0:(events.c:361:request_in_callback()) peer: 12345-0@lo (source: 12345-0@lo) 00000100:00100000:0.0:1776178333.539320:0:3352:0:(client.c:1733:ptlrpc_send_new_req()) Sending RPC req@0000000029ef3988 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:lustre-MDT0000-lwp-MDT0001_UUID:3352:1862456998067648:0@lo:38:kworker/u8:1.0 00000100:00100000:0.0:1776178333.539345:0:3352:0:(events.c:361:request_in_callback()) peer: 12345-0@lo (source: 12345-0@lo) 00000100:00100000:0.0:1776178333.539389:0:15213:0:(service.c:2152:ptlrpc_server_handle_req_in()) got req x1862456998067456 00000100:00100000:0.0:1776178333.539426:0:15213:0:(nrs_fifo.c:179:nrs_fifo_req_get()) NRS start fifo request from 12345-0@lo, seq: 733 00000100:00100000:0.0:1776178333.539433:0:15213:0:(service.c:2324:ptlrpc_server_handle_request()) Handling RPC req@000000001f977915 pname:cluuid+ref:pid:xid:nid:opc:job mdt_out00_003:0+-99:3352:x1862456998067456:12345-0@lo:38:kworker/u8:1.0 00010000:02000000:0.0:1776178333.539457:0:15213:0:(ldlm_lib.c:1139:target_handle_connect()) lustre-MDT0000: Not available for connect from 0@lo (stopping) 00000100:00100000:0.0:1776178333.539511:0:15213:0:(service.c:2372:ptlrpc_server_handle_request()) Handled RPC req@000000001f977915 pname:cluuid+ref:pid:xid:nid:opc:job mdt_out00_003:0+-99:3352:x1862456998067456:12345-0@lo:38:kworker/u8:1.0 Request processed in 78us (211us total) trans 0 rc -19/-19 00000100:00100000:0.0:1776178333.539532:0:15213:0:(nrs_fifo.c:241:nrs_fifo_req_stop()) NRS stop fifo request from 12345-0@lo, seq: 733 00000100:00100000:0.0:1776178333.539560:0:6256:0:(service.c:2152:ptlrpc_server_handle_req_in()) got req x1862456998067392 00000100:00100000:0.0:1776178333.539594:0:6256:0:(service.c:2152:ptlrpc_server_handle_req_in()) got req x1862456998067264 00000100:00100000:0.0:1776178333.539604:0:6256:0:(service.c:2152:ptlrpc_server_handle_req_in()) got req x1862456998067200 00000100:00100000:0.0:1776178333.539614:0:6256:0:(service.c:2152:ptlrpc_server_handle_req_in()) got req x1862456998067328 00000100:00100000:0.0:1776178333.539622:0:6256:0:(service.c:2152:ptlrpc_server_handle_req_in()) got req x1862456998067648 00000100:00100000:0.0:1776178333.539631:0:6256:0:(nrs_fifo.c:179:nrs_fifo_req_get()) NRS start fifo request from 12345-0@lo, seq: 1004 00000100:00100000:0.0:1776178333.539635:0:6256:0:(service.c:2324:ptlrpc_server_handle_request()) Handling RPC req@0000000035819321 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_001:lustre-MDT0001-lwp-OST0001_UUID+5:3354:x1862456998067392:12345-0@lo:400:kworker/u8:1.0 00000100:00100000:0.0:1776178333.539685:0:6256:0:(service.c:2372:ptlrpc_server_handle_request()) Handled RPC req@0000000035819321 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_001:lustre-MDT0001-lwp-OST0001_UUID+5:3354:x1862456998067392:12345-0@lo:400:kworker/u8:1.0 Request processed in 49us (785us total) trans 0 rc 0/0 00000100:00100000:0.0:1776178333.539691:0:6256:0:(nrs_fifo.c:241:nrs_fifo_req_stop()) NRS stop fifo request from 12345-0@lo, seq: 1004 00000100:00100000:0.0:1776178333.539703:0:6256:0:(nrs_fifo.c:179:nrs_fifo_req_get()) NRS start fifo request from 12345-0@lo, seq: 1005 00000100:00100000:0.0:1776178333.539706:0:6256:0:(service.c:2324:ptlrpc_server_handle_request()) Handling RPC req@0000000012b22334 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_001:lustre-MDT0001-lwp-OST0000_UUID+5:3354:x1862456998067264:12345-0@lo:400:kworker/u8:1.0 00000100:00100000:0.0:1776178333.539933:0:6256:0:(service.c:2372:ptlrpc_server_handle_request()) Handled RPC req@0000000012b22334 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_001:lustre-MDT0001-lwp-OST0000_UUID+5:3354:x1862456998067264:12345-0@lo:400:kworker/u8:1.0 Request processed in 225us (919us total) trans 0 rc 0/0 00000100:00100000:0.0:1776178333.539940:0:6256:0:(nrs_fifo.c:241:nrs_fifo_req_stop()) NRS stop fifo request from 12345-0@lo, seq: 1005 00000100:00100000:0.0:1776178333.539952:0:6256:0:(nrs_fifo.c:179:nrs_fifo_req_get()) NRS start fifo request from 12345-0@lo, seq: 6725 00000100:00100000:0.0:1776178333.539956:0:6256:0:(service.c:2324:ptlrpc_server_handle_request()) Handling RPC req@0000000084c00ed0 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_001:0+-99:3352:x1862456998067200:12345-0@lo:38:kworker/u8:1.0 00010000:02000000:0.0:1776178333.539963:0:6256:0:(ldlm_lib.c:1139:target_handle_connect()) lustre-MDT0000: Not available for connect from 0@lo (stopping) 00000100:00100000:0.0:1776178333.539982:0:6256:0:(service.c:2372:ptlrpc_server_handle_request()) Handled RPC req@0000000084c00ed0 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_001:0+-99:3352:x1862456998067200:12345-0@lo:38:kworker/u8:1.0 Request processed in 25us (798us total) trans 0 rc -19/-19 00000100:00100000:0.0:1776178333.539988:0:6256:0:(nrs_fifo.c:241:nrs_fifo_req_stop()) NRS stop fifo request from 12345-0@lo, seq: 6725 00000100:00100000:0.0:1776178333.539996:0:6256:0:(nrs_fifo.c:179:nrs_fifo_req_get()) NRS start fifo request from 12345-0@lo, seq: 6726 00000100:00100000:0.0:1776178333.539999:0:6256:0:(service.c:2324:ptlrpc_server_handle_request()) Handling RPC req@00000000a2082164 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_001:0+-99:3352:x1862456998067328:12345-0@lo:38:kworker/u8:1.0 00010000:02000000:0.0:1776178333.540004:0:6256:0:(ldlm_lib.c:1139:target_handle_connect()) lustre-MDT0000: Not available for connect from 0@lo (stopping) 00000100:00100000:0.0:1776178333.540021:0:6256:0:(service.c:2372:ptlrpc_server_handle_request()) Handled RPC req@00000000a2082164 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_001:0+-99:3352:x1862456998067328:12345-0@lo:38:kworker/u8:1.0 Request processed in 21us (768us total) trans 0 rc -19/-19 00000100:00100000:0.0:1776178333.540026:0:6256:0:(nrs_fifo.c:241:nrs_fifo_req_stop()) NRS stop fifo request from 12345-0@lo, seq: 6726 00000100:00100000:0.0:1776178333.540034:0:6256:0:(nrs_fifo.c:179:nrs_fifo_req_get()) NRS start fifo request from 12345-0@lo, seq: 6727 00000100:00100000:0.0:1776178333.540037:0:6256:0:(service.c:2324:ptlrpc_server_handle_request()) Handling RPC req@00000000c324c57b pname:cluuid+ref:pid:xid:nid:opc:job mdt00_001:0+-99:3352:x1862456998067648:12345-0@lo:38:kworker/u8:1.0 00010000:02000000:0.0:1776178333.540042:0:6256:0:(ldlm_lib.c:1139:target_handle_connect()) lustre-MDT0000: Not available for connect from 0@lo (stopping) 00000100:00100000:0.0:1776178333.540102:0:6256:0:(service.c:2372:ptlrpc_server_handle_request()) Handled RPC req@00000000c324c57b pname:cluuid+ref:pid:xid:nid:opc:job mdt00_001:0+-99:3352:x1862456998067648:12345-0@lo:38:kworker/u8:1.0 Request processed in 21us (713us total) trans 0 rc -19/-19 00000100:00100000:0.0:1776178333.540108:0:6256:0:(nrs_fifo.c:241:nrs_fifo_req_stop()) NRS stop fifo request from 12345-0@lo, seq: 6727 00000100:00080000:0.0:1776178333.540144:0:3352:0:(import.c:86:import_set_state_nolock()) 0000000047601a98 lustre-MDT0000_UUID: changing import state from CONNECTING to DISCONN 00000100:00080000:0.0:1776178333.540148:0:3352:0:(import.c:1429:ptlrpc_connect_interpret()) recovery of lustre-MDT0000_UUID on 192.168.204.103@tcp failed (-19) 00000100:00100000:0.0:1776178333.540153:0:3352:0:(client.c:2215:ptlrpc_check_set()) Completed RPC req@000000007216887e pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:lustre-MDT0000-lwp-OST0000_UUID:3352:1862456998067200:0@lo:38:kworker/u8:1.0 00000100:00100000:2.0:1776178333.540321:0:3354:0:(client.c:2215:ptlrpc_check_set()) Completed RPC req@00000000542ce52f pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_01:lustre-MDT0001-lwp-OST0000_UUID:3354:1862456998067264:0@lo:400:kworker/u8:1.0 00000100:00100000:2.0:1776178333.540333:0:3354:0:(client.c:2215:ptlrpc_check_set()) Completed RPC req@0000000037aace32 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_01:lustre-MDT0001-lwp-OST0001_UUID:3354:1862456998067392:0@lo:400:kworker/u8:1.0 00000100:00080000:0.0:1776178333.541284:0:3352:0:(import.c:86:import_set_state_nolock()) 00000000dfc8db13 lustre-MDT0000_UUID: changing import state from CONNECTING to DISCONN 00000100:00080000:0.0:1776178333.541290:0:3352:0:(import.c:1429:ptlrpc_connect_interpret()) recovery of lustre-MDT0000_UUID on 192.168.204.103@tcp failed (-19) 00000100:00100000:0.0:1776178333.541295:0:3352:0:(client.c:2215:ptlrpc_check_set()) Completed RPC req@000000000a0451ff pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:lustre-MDT0000-lwp-OST0001_UUID:3352:1862456998067328:0@lo:38:kworker/u8:1.0 00000100:00080000:0.0:1776178333.541312:0:3352:0:(import.c:86:import_set_state_nolock()) 00000000ba4f1821 lustre-MDT0000_UUID: changing import state from CONNECTING to DISCONN 00000100:00080000:0.0:1776178333.541315:0:3352:0:(import.c:1429:ptlrpc_connect_interpret()) recovery of lustre-MDT0000_UUID on 192.168.204.103@tcp failed (-19) 00000100:00100000:0.0:1776178333.541317:0:3352:0:(client.c:2215:ptlrpc_check_set()) Completed RPC req@0000000075189f40 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:lustre-MDT0001-mdtlov_UUID:3352:1862456998067456:0@lo:38:kworker/u8:1.0 00000100:00080000:0.0:1776178333.541327:0:3352:0:(import.c:86:import_set_state_nolock()) 00000000a7c55b1f lustre-MDT0000_UUID: changing import state from CONNECTING to DISCONN 00000100:00080000:0.0:1776178333.541329:0:3352:0:(import.c:1429:ptlrpc_connect_interpret()) recovery of lustre-MDT0000_UUID on 192.168.204.103@tcp failed (-19) 00000100:00100000:0.0:1776178333.541332:0:3352:0:(client.c:2215:ptlrpc_check_set()) Completed RPC req@0000000029ef3988 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:lustre-MDT0000-lwp-MDT0001_UUID:3352:1862456998067648:0@lo:38:kworker/u8:1.0 00010000:00010000:2.0:1776178334.176375:0:9224:0:(ldlm_lockd.c:2792:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:0.0:1776178335.073508:0:3353:0:(client.c:1733:ptlrpc_send_new_req()) Sending RPC req@00000000a0c64d21 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_00:lustre-MDT0001-mdtlov_UUID:3353:1862456998067712:0@lo:13:osp-pre-1-1.0 00000100:00100000:0.0:1776178335.073602:0:3353:0:(events.c:361:request_in_callback()) peer: 12345-0@lo (source: 12345-0@lo) 00000100:00100000:1.0:1776178335.073724:0:9554:0:(service.c:2152:ptlrpc_server_handle_req_in()) got req x1862456998067712 00000100:00100000:1.0:1776178335.073787:0:9554:0:(nrs_fifo.c:179:nrs_fifo_req_get()) NRS start fifo request from 12345-0@lo, seq: 539 00000100:00100000:1.0:1776178335.073793:0:9554:0:(service.c:2324:ptlrpc_server_handle_request()) Handling RPC req@0000000047c15ac0 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_create00:lustre-MDT0001-mdtlov_UUID+5:3353:x1862456998067712:12345-0@lo:13:osp-pre-1-1.0 00002000:00000024:1.0:1776178335.073833:0:9554:0:(ofd_obd.c:740:ofd_statfs()) blocks cached 229376 granted 491746368 pending 0 free 3923689472 avail 3692167168 00000100:00100000:1.0:1776178335.077738:0:9554:0:(service.c:2372:ptlrpc_server_handle_request()) Handled RPC req@0000000047c15ac0 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_create00:lustre-MDT0001-mdtlov_UUID+5:3353:x1862456998067712:12345-0@lo:13:osp-pre-1-1.0 Request processed in 3933us (4126us total) trans 0 rc 0/0 00000100:00100000:1.0:1776178335.077757:0:9554:0:(nrs_fifo.c:241:nrs_fifo_req_stop()) NRS stop fifo request from 12345-0@lo, seq: 539 00000100:00100000:0.0:1776178335.080832:0:3353:0:(client.c:2215:ptlrpc_check_set()) Completed RPC req@00000000a0c64d21 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_00:lustre-MDT0001-mdtlov_UUID:3353:1862456998067712:0@lo:13:osp-pre-1-1.0 00010000:00010000:2.0:1776178335.204877:0:9225:0:(ldlm_lockd.c:2792:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:1.0:1776178335.584815:0:3353:0:(ptlrpcd.c:415:ptlrpcd_check()) transfer 1 async RPCs [1->0] 00000100:00100000:1.0:1776178335.584841:0:3353:0:(client.c:1733:ptlrpc_send_new_req()) Sending RPC req@000000008c3fbdb6 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_00:lustre-MDT0001-mdtlov_UUID:3353:1862456998067776:0@lo:13:osp-pre-0-1.0 00000100:00100000:1.0:1776178335.584928:0:3353:0:(events.c:361:request_in_callback()) peer: 12345-0@lo (source: 12345-0@lo) 00000100:00100000:0.0:1776178335.585116:0:9554:0:(service.c:2152:ptlrpc_server_handle_req_in()) got req x1862456998067776 00000100:00100000:0.0:1776178335.585207:0:9554:0:(nrs_fifo.c:179:nrs_fifo_req_get()) NRS start fifo request from 12345-0@lo, seq: 540 00000100:00100000:0.0:1776178335.585213:0:9554:0:(service.c:2324:ptlrpc_server_handle_request()) Handling RPC req@0000000021b0d516 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_create00:lustre-MDT0001-mdtlov_UUID+5:3353:x1862456998067776:12345-0@lo:13:osp-pre-0-1.0 00002000:00000024:0.0:1776178335.585250:0:9554:0:(ofd_obd.c:740:ofd_statfs()) blocks cached 315392 granted 484883776 pending 0 free 3923689472 avail 3692167168 00000100:00100000:0.0:1776178335.585315:0:9554:0:(service.c:2372:ptlrpc_server_handle_request()) Handled RPC req@0000000021b0d516 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_create00:lustre-MDT0001-mdtlov_UUID+5:3353:x1862456998067776:12345-0@lo:13:osp-pre-0-1.0 Request processed in 100us (387us total) trans 0 rc 0/0 00000100:00100000:0.0:1776178335.585322:0:9554:0:(nrs_fifo.c:241:nrs_fifo_req_stop()) NRS stop fifo request from 12345-0@lo, seq: 540 00000100:00100000:1.0:1776178335.585615:0:3353:0:(client.c:2215:ptlrpc_check_set()) Completed RPC req@000000008c3fbdb6 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_00:lustre-MDT0001-mdtlov_UUID:3353:1862456998067776:0@lo:13:osp-pre-0-1.0 00010000:00010000:2.0:1776178336.228119:0:6244:0:(ldlm_lockd.c:2792:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:3.0:1776178337.057457:0:3347:0:(events.c:361:request_in_callback()) peer: 12345-192.168.204.3@tcp (source: 12345-192.168.204.3@tcp) 00000100:00100000:2.0:1776178337.057594:0:6255:0:(service.c:2152:ptlrpc_server_handle_req_in()) got req x1862456979117376 00000100:00100000:2.0:1776178337.057678:0:6255:0:(nrs_fifo.c:179:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.204.3@tcp, seq: 6728 00000100:00100000:2.0:1776178337.057694:0:6255:0:(service.c:2324:ptlrpc_server_handle_request()) Handling RPC req@00000000a08c40f7 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_000:0+-99:2269:x1862456979117376:12345-192.168.204.3@tcp:38:kworker/u8:3.0 00010000:02000000:2.0:1776178337.057725:0:6255:0:(ldlm_lib.c:1139:target_handle_connect()) lustre-MDT0000: Not available for connect from 192.168.204.3@tcp (stopping) 00000100:00100000:2.0:1776178337.057814:0:6255:0:(service.c:2372:ptlrpc_server_handle_request()) Handled RPC req@00000000a08c40f7 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_000:0+-99:2269:x1862456979117376:12345-192.168.204.3@tcp:38:kworker/u8:3.0 Request processed in 124us (367us total) trans 0 rc -19/-19 00000100:00100000:2.0:1776178337.057822:0:6255:0:(nrs_fifo.c:241:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.204.3@tcp, seq: 6728 00000100:00100000:1.0:1776178337.058520:0:3348:0:(events.c:361:request_in_callback()) peer: 12345-192.168.204.3@tcp (source: 12345-192.168.204.3@tcp) 00000100:00100000:1.0:1776178337.058563:0:3348:0:(events.c:361:request_in_callback()) peer: 12345-192.168.204.3@tcp (source: 12345-192.168.204.3@tcp) 00000100:00100000:1.0:1776178337.058598:0:3348:0:(events.c:361:request_in_callback()) peer: 12345-192.168.204.3@tcp (source: 12345-192.168.204.3@tcp) 00000100:00100000:2.0:1776178337.058639:0:20762:0:(service.c:2152:ptlrpc_server_handle_req_in()) got req x1862456979117504 00000100:00100000:1.0:1776178337.058644:0:3348:0:(events.c:361:request_in_callback()) peer: 12345-192.168.204.3@tcp (source: 12345-192.168.204.3@tcp) 00000100:00100000:2.0:1776178337.058687:0:20762:0:(service.c:2152:ptlrpc_server_handle_req_in()) got req x1862456979117568 00000100:00100000:3.0:1776178337.058714:0:6255:0:(service.c:2152:ptlrpc_server_handle_req_in()) got req x1862456979117440 00000100:00100000:0.0:1776178337.058772:0:20140:0:(service.c:2152:ptlrpc_server_handle_req_in()) got req x1862456979117312 00000100:00100000:0.0:1776178337.058804:0:20140:0:(nrs_fifo.c:179:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.204.3@tcp, seq: 207 00000100:00100000:0.0:1776178337.058809:0:20140:0:(service.c:2324:ptlrpc_server_handle_request()) Handling RPC req@00000000d1b75234 pname:cluuid+ref:pid:xid:nid:opc:job ll_mgs_0002:5b5c66b1-1a2f-4651-8548-0c75b9a38a94+9:2271:x1862456979117312:12345-192.168.204.3@tcp:400:kworker/u8:3.0 00000100:00100000:0.0:1776178337.058858:0:20140:0:(service.c:2372:ptlrpc_server_handle_request()) Handled RPC req@00000000d1b75234 pname:cluuid+ref:pid:xid:nid:opc:job ll_mgs_0002:5b5c66b1-1a2f-4651-8548-0c75b9a38a94+9:2271:x1862456979117312:12345-192.168.204.3@tcp:400:kworker/u8:3.0 Request processed in 49us (342us total) trans 0 rc 0/0 00000100:00100000:0.0:1776178337.058865:0:20140:0:(nrs_fifo.c:241:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.204.3@tcp, seq: 207 00000100:00100000:3.0:1776178337.059028:0:6255:0:(nrs_fifo.c:179:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.204.3@tcp, seq: 1006 00000100:00100000:3.0:1776178337.059032:0:6255:0:(service.c:2324:ptlrpc_server_handle_request()) Handling RPC req@0000000023da22ef pname:cluuid+ref:pid:xid:nid:opc:job mdt00_000:b50963bd-8eba-4432-9a54-8888d00e3fa1+6:2271:x1862456979117440:12345-192.168.204.3@tcp:400:kworker/u8:3.0 00000100:00100000:3.0:1776178337.059099:0:6255:0:(service.c:2372:ptlrpc_server_handle_request()) Handled RPC req@0000000023da22ef pname:cluuid+ref:pid:xid:nid:opc:job mdt00_000:b50963bd-8eba-4432-9a54-8888d00e3fa1+6:2271:x1862456979117440:12345-192.168.204.3@tcp:400:kworker/u8:3.0 Request processed in 66us (501us total) trans 0 rc 0/0 00000100:00100000:3.0:1776178337.059106:0:6255:0:(nrs_fifo.c:241:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.204.3@tcp, seq: 1006 00000100:00100000:1.0:1776178337.059455:0:20759:0:(nrs_fifo.c:179:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.204.3@tcp, seq: 615 00000100:00100000:1.0:1776178337.059461:0:20759:0:(service.c:2324:ptlrpc_server_handle_request()) Handling RPC req@000000007fe70ab5 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_010:b50963bd-8eba-4432-9a54-8888d00e3fa1+8:2271:x1862456979117504:12345-192.168.204.3@tcp:400:kworker/u8:3.0 00000100:00100000:1.0:1776178337.059501:0:20759:0:(service.c:2372:ptlrpc_server_handle_request()) Handled RPC req@000000007fe70ab5 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_010:b50963bd-8eba-4432-9a54-8888d00e3fa1+8:2271:x1862456979117504:12345-192.168.204.3@tcp:400:kworker/u8:3.0 Request processed in 40us (939us total) trans 0 rc 0/0 00000100:00100000:1.0:1776178337.059507:0:20759:0:(nrs_fifo.c:241:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.204.3@tcp, seq: 615 00000100:00100000:1.0:1776178337.059728:0:20759:0:(nrs_fifo.c:179:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.204.3@tcp, seq: 616 00000100:00100000:1.0:1776178337.059732:0:20759:0:(service.c:2324:ptlrpc_server_handle_request()) Handling RPC req@0000000078fb1a26 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_010:b50963bd-8eba-4432-9a54-8888d00e3fa1+9:2271:x1862456979117568:12345-192.168.204.3@tcp:400:kworker/u8:3.0 00000100:00100000:1.0:1776178337.059753:0:20759:0:(service.c:2372:ptlrpc_server_handle_request()) Handled RPC req@0000000078fb1a26 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_010:b50963bd-8eba-4432-9a54-8888d00e3fa1+9:2271:x1862456979117568:12345-192.168.204.3@tcp:400:kworker/u8:3.0 Request processed in 21us (1112us total) trans 0 rc 0/0 00000100:00100000:1.0:1776178337.059758:0:20759:0:(nrs_fifo.c:241:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.204.3@tcp, seq: 616 00010000:00010000:2.0:1776178337.248430:0:6245:0:(ldlm_lockd.c:2792:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00010000:00010000:2.0:1776178338.272320:0:9224:0:(ldlm_lockd.c:2792:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00080000:1.0:1776178338.656380:0:37:0:(pinger.c:251:ptlrpc_pinger_process_import()) lustre-MDT0000-lwp-OST0000_UUID->lustre-MDT0000_UUID: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00080000:1.0:1776178338.656390:0:37:0:(recover.c:57:ptlrpc_initiate_recovery()) lustre-MDT0000_UUID: starting recovery 00000100:00080000:1.0:1776178338.656394:0:37:0:(import.c:86:import_set_state_nolock()) 0000000047601a98 lustre-MDT0000_UUID: changing import state from DISCONN to CONNECTING 00000100:00080000:1.0:1776178338.656403:0:37:0:(import.c:534:import_select_connection()) lustre-MDT0000-lwp-OST0000: connect to NID 0@lo last attempt 1040 00000100:00080000:1.0:1776178338.656407:0:37:0:(import.c:581:import_select_connection()) lustre-MDT0000-lwp-OST0000: tried all connections, increasing latency to 21s 00000100:00080000:1.0:1776178338.656412:0:37:0:(import.c:615:import_select_connection()) lustre-MDT0000-lwp-OST0000: import 0000000047601a98 using connection 192.168.204.103@tcp/0@lo 00000100:00100000:1.0:1776178338.656448:0:37:0:(import.c:807:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5) req@00000000747e17cb x1862456998067904/t0(0) o38->lustre-MDT0000-lwp-OST0000@0@lo:12/10 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' 00000100:00080000:1.0:1776178338.656478:0:37:0:(pinger.c:251:ptlrpc_pinger_process_import()) lustre-MDT0000-lwp-OST0001_UUID->lustre-MDT0000_UUID: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00080000:1.0:1776178338.656481:0:37:0:(recover.c:57:ptlrpc_initiate_recovery()) lustre-MDT0000_UUID: starting recovery 00000100:00080000:1.0:1776178338.656483:0:37:0:(import.c:86:import_set_state_nolock()) 00000000dfc8db13 lustre-MDT0000_UUID: changing import state from DISCONN to CONNECTING 00000100:00080000:1.0:1776178338.656486:0:37:0:(import.c:534:import_select_connection()) lustre-MDT0000-lwp-OST0001: connect to NID 0@lo last attempt 1040 00000100:00080000:1.0:1776178338.656488:0:37:0:(import.c:581:import_select_connection()) lustre-MDT0000-lwp-OST0001: tried all connections, increasing latency to 21s 00000100:00080000:1.0:1776178338.656491:0:37:0:(import.c:615:import_select_connection()) lustre-MDT0000-lwp-OST0001: import 00000000dfc8db13 using connection 192.168.204.103@tcp/0@lo 00000100:00100000:1.0:1776178338.656500:0:37:0:(import.c:807:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5) req@000000008f48ae7c x1862456998068032/t0(0) o38->lustre-MDT0000-lwp-OST0001@0@lo:12/10 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' 00000100:00080000:1.0:1776178338.656552:0:37:0:(pinger.c:251:ptlrpc_pinger_process_import()) lustre-MDT0001-mdtlov_UUID->lustre-MDT0000_UUID: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00080000:1.0:1776178338.656555:0:37:0:(recover.c:57:ptlrpc_initiate_recovery()) lustre-MDT0000_UUID: starting recovery 00000100:00080000:1.0:1776178338.656556:0:37:0:(import.c:86:import_set_state_nolock()) 00000000ba4f1821 lustre-MDT0000_UUID: changing import state from DISCONN to CONNECTING 00000100:00080000:1.0:1776178338.656559:0:37:0:(import.c:534:import_select_connection()) lustre-MDT0000-osp-MDT0001: connect to NID 0@lo last attempt 1040 00000100:00080000:1.0:1776178338.656561:0:37:0:(import.c:581:import_select_connection()) lustre-MDT0000-osp-MDT0001: tried all connections, increasing latency to 21s 00000100:00080000:1.0:1776178338.656564:0:37:0:(import.c:615:import_select_connection()) lustre-MDT0000-osp-MDT0001: import 00000000ba4f1821 using connection 192.168.204.103@tcp/0@lo 00000100:00100000:1.0:1776178338.656575:0:37:0:(import.c:807:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5) req@000000001e11faca x1862456998068160/t0(0) o38->lustre-MDT0000-osp-MDT0001@0@lo:24/4 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' 00000100:00080000:1.0:1776178338.656594:0:37:0:(pinger.c:251:ptlrpc_pinger_process_import()) lustre-MDT0000-lwp-MDT0001_UUID->lustre-MDT0000_UUID: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00080000:1.0:1776178338.656599:0:37:0:(recover.c:57:ptlrpc_initiate_recovery()) lustre-MDT0000_UUID: starting recovery 00000100:00080000:1.0:1776178338.656601:0:37:0:(import.c:86:import_set_state_nolock()) 00000000a7c55b1f lustre-MDT0000_UUID: changing import state from DISCONN to CONNECTING 00000100:00080000:1.0:1776178338.656604:0:37:0:(import.c:534:import_select_connection()) lustre-MDT0000-lwp-MDT0001: connect to NID 0@lo last attempt 1040 00000100:00080000:1.0:1776178338.656606:0:37:0:(import.c:581:import_select_connection()) lustre-MDT0000-lwp-MDT0001: tried all connections, increasing latency to 21s 00000100:00080000:1.0:1776178338.656608:0:37:0:(import.c:615:import_select_connection()) lustre-MDT0000-lwp-MDT0001: import 00000000a7c55b1f using connection 192.168.204.103@tcp/0@lo 00000100:00100000:1.0:1776178338.656619:0:37:0:(import.c:807:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5) req@0000000069ebab60 x1862456998068352/t0(0) o38->lustre-MDT0000-lwp-MDT0001@0@lo:12/10 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' 00000100:00100000:1.0:1776178338.656659:0:3353:0:(client.c:1733:ptlrpc_send_new_req()) Sending RPC req@00000000b3188344 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_00:lustre-MDT0001-lwp-OST0001_UUID:3353:1862456998068096:0@lo:400:kworker/u8:2.0 00000100:00100000:1.0:1776178338.656735:0:3353:0:(events.c:361:request_in_callback()) peer: 12345-0@lo (source: 12345-0@lo) 00000100:00100000:1.0:1776178338.656772:0:3353:0:(ptlrpcd.c:415:ptlrpcd_check()) transfer 1 async RPCs [1->0] 00000100:00100000:1.0:1776178338.656778:0:3353:0:(client.c:1733:ptlrpc_send_new_req()) Sending RPC req@00000000250bc347 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_00:lustre-MDT0001-mdtlov_UUID:3353:1862456998068224:0@lo:400:kworker/u8:2.0 00000100:00100000:1.0:1776178338.656806:0:3353:0:(events.c:361:request_in_callback()) peer: 12345-0@lo (source: 12345-0@lo) 00000100:00100000:3.0:1776178338.656897:0:3356:0:(client.c:1733:ptlrpc_send_new_req()) Sending RPC req@00000000de25df9f pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_03:lustre-MDT0001-lwp-OST0000_UUID:3356:1862456998067968:0@lo:400:kworker/u8:2.0 00000100:00100000:3.0:1776178338.656926:0:3356:0:(events.c:361:request_in_callback()) peer: 12345-0@lo (source: 12345-0@lo) 00000100:00100000:3.0:1776178338.656951:0:3356:0:(ptlrpcd.c:415:ptlrpcd_check()) transfer 2 async RPCs [2->3] 00000100:00100000:3.0:1776178338.656956:0:3356:0:(client.c:1733:ptlrpc_send_new_req()) Sending RPC req@00000000a430b18f pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_03:ac61f839-0e8e-4cac-9c59-08bcfbbad330:3356:1862456998067840:0@lo:400:kworker/u8:2.0 00000100:00100000:3.0:1776178338.656984:0:3356:0:(events.c:361:request_in_callback()) peer: 12345-0@lo (source: 12345-0@lo) 00000100:00100000:3.0:1776178338.656997:0:3356:0:(client.c:1733:ptlrpc_send_new_req()) Sending RPC req@00000000103d2c47 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_03:lustre-MDT0001-mdtlov_UUID:3356:1862456998068288:0@lo:400:kworker/u8:2.0 00000100:00100000:3.0:1776178338.657015:0:3356:0:(events.c:361:request_in_callback()) peer: 12345-0@lo (source: 12345-0@lo) 00000100:00100000:3.0:1776178338.657049:0:8250:0:(service.c:2152:ptlrpc_server_handle_req_in()) got req x1862456998068096 00000100:00100000:3.0:1776178338.657121:0:8250:0:(service.c:2152:ptlrpc_server_handle_req_in()) got req x1862456998067968 00000100:00100000:3.0:1776178338.657153:0:8250:0:(nrs_fifo.c:179:nrs_fifo_req_get()) NRS start fifo request from 12345-0@lo, seq: 1007 00000100:00100000:3.0:1776178338.657160:0:8250:0:(service.c:2324:ptlrpc_server_handle_request()) Handling RPC req@00000000b7aa289e pname:cluuid+ref:pid:xid:nid:opc:job mdt00_003:lustre-MDT0001-lwp-OST0001_UUID+5:3353:x1862456998068096:12345-0@lo:400:kworker/u8:2.0 00000100:00100000:3.0:1776178338.657218:0:8250:0:(service.c:2372:ptlrpc_server_handle_request()) Handled RPC req@00000000b7aa289e pname:cluuid+ref:pid:xid:nid:opc:job mdt00_003:lustre-MDT0001-lwp-OST0001_UUID+5:3353:x1862456998068096:12345-0@lo:400:kworker/u8:2.0 Request processed in 58us (485us total) trans 0 rc 0/0 00000100:00100000:3.0:1776178338.657226:0:8250:0:(nrs_fifo.c:241:nrs_fifo_req_stop()) NRS stop fifo request from 12345-0@lo, seq: 1007 00000100:00100000:3.0:1776178338.657243:0:8250:0:(nrs_fifo.c:179:nrs_fifo_req_get()) NRS start fifo request from 12345-0@lo, seq: 1008 00000100:00100000:3.0:1776178338.657247:0:8250:0:(service.c:2324:ptlrpc_server_handle_request()) Handling RPC req@00000000ea88a302 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_003:lustre-MDT0001-lwp-OST0000_UUID+5:3356:x1862456998067968:12345-0@lo:400:kworker/u8:2.0 00000100:00100000:3.0:1776178338.657273:0:8250:0:(service.c:2372:ptlrpc_server_handle_request()) Handled RPC req@00000000ea88a302 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_003:lustre-MDT0001-lwp-OST0000_UUID+5:3356:x1862456998067968:12345-0@lo:400:kworker/u8:2.0 Request processed in 26us (348us total) trans 0 rc 0/0 00000100:00100000:3.0:1776178338.657279:0:8250:0:(nrs_fifo.c:241:nrs_fifo_req_stop()) NRS stop fifo request from 12345-0@lo, seq: 1008 00000100:00100000:3.0:1776178338.657330:0:3356:0:(client.c:2215:ptlrpc_check_set()) Completed RPC req@00000000de25df9f pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_03:lustre-MDT0001-lwp-OST0000_UUID:3356:1862456998067968:0@lo:400:kworker/u8:2.0 00000100:00100000:3.0:1776178338.657363:0:3353:0:(client.c:2215:ptlrpc_check_set()) Completed RPC req@00000000b3188344 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_00:lustre-MDT0001-lwp-OST0001_UUID:3353:1862456998068096:0@lo:400:kworker/u8:2.0 00000100:00100000:1.0:1776178338.657714:0:20759:0:(service.c:2152:ptlrpc_server_handle_req_in()) got req x1862456998068224 00000100:00100000:0.0:1776178338.657826:0:3352:0:(client.c:1733:ptlrpc_send_new_req()) Sending RPC req@00000000747e17cb pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:lustre-MDT0000-lwp-OST0000_UUID:3352:1862456998067904:0@lo:38:kworker/u8:2.0 00000100:00100000:0.0:1776178338.657862:0:3352:0:(events.c:361:request_in_callback()) peer: 12345-0@lo (source: 12345-0@lo) 00000100:00100000:0.0:1776178338.657891:0:3352:0:(client.c:1733:ptlrpc_send_new_req()) Sending RPC req@000000008f48ae7c pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:lustre-MDT0000-lwp-OST0001_UUID:3352:1862456998068032:0@lo:38:kworker/u8:2.0 00000100:00100000:0.0:1776178338.657911:0:3352:0:(events.c:361:request_in_callback()) peer: 12345-0@lo (source: 12345-0@lo) 00000100:00100000:0.0:1776178338.657923:0:3352:0:(client.c:1733:ptlrpc_send_new_req()) Sending RPC req@000000001e11faca pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:lustre-MDT0001-mdtlov_UUID:3352:1862456998068160:0@lo:38:kworker/u8:2.0 00000100:00100000:0.0:1776178338.657946:0:3352:0:(events.c:361:request_in_callback()) peer: 12345-0@lo (source: 12345-0@lo) 00000100:00100000:0.0:1776178338.657960:0:3352:0:(client.c:1733:ptlrpc_send_new_req()) Sending RPC req@0000000069ebab60 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:lustre-MDT0000-lwp-MDT0001_UUID:3352:1862456998068352:0@lo:38:kworker/u8:2.0 00000100:00100000:0.0:1776178338.657979:0:3352:0:(events.c:361:request_in_callback()) peer: 12345-0@lo (source: 12345-0@lo) 00000100:00100000:3.0:1776178338.658042:0:11281:0:(service.c:2152:ptlrpc_server_handle_req_in()) got req x1862456998067904 00000100:00100000:3.0:1776178338.658086:0:11281:0:(service.c:2152:ptlrpc_server_handle_req_in()) got req x1862456998068032 00000100:00100000:3.0:1776178338.658096:0:11281:0:(service.c:2152:ptlrpc_server_handle_req_in()) got req x1862456998068352 00000100:00100000:3.0:1776178338.658105:0:11281:0:(nrs_fifo.c:179:nrs_fifo_req_get()) NRS start fifo request from 12345-0@lo, seq: 6729 00000100:00100000:3.0:1776178338.658108:0:11281:0:(service.c:2324:ptlrpc_server_handle_request()) Handling RPC req@0000000089dbd0da pname:cluuid+ref:pid:xid:nid:opc:job mdt00_004:0+-99:3352:x1862456998067904:12345-0@lo:38:kworker/u8:2.0 00010000:02000000:3.0:1776178338.658121:0:11281:0:(ldlm_lib.c:1139:target_handle_connect()) lustre-MDT0000: Not available for connect from 0@lo (stopping) 00000100:00100000:3.0:1776178338.658167:0:11281:0:(service.c:2372:ptlrpc_server_handle_request()) Handled RPC req@0000000089dbd0da pname:cluuid+ref:pid:xid:nid:opc:job mdt00_004:0+-99:3352:x1862456998067904:12345-0@lo:38:kworker/u8:2.0 Request processed in 57us (305us total) trans 0 rc -19/-19 00000100:00100000:3.0:1776178338.658173:0:11281:0:(nrs_fifo.c:241:nrs_fifo_req_stop()) NRS stop fifo request from 12345-0@lo, seq: 6729 00000100:00100000:3.0:1776178338.658184:0:11281:0:(nrs_fifo.c:179:nrs_fifo_req_get()) NRS start fifo request from 12345-0@lo, seq: 6730 00000100:00100000:3.0:1776178338.658187:0:11281:0:(service.c:2324:ptlrpc_server_handle_request()) Handling RPC req@00000000542c8034 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_004:0+-99:3352:x1862456998068032:12345-0@lo:38:kworker/u8:2.0 00010000:02000000:3.0:1776178338.658193:0:11281:0:(ldlm_lib.c:1139:target_handle_connect()) lustre-MDT0000: Not available for connect from 0@lo (stopping) 00000100:00100000:3.0:1776178338.658211:0:11281:0:(service.c:2372:ptlrpc_server_handle_request()) Handled RPC req@00000000542c8034 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_004:0+-99:3352:x1862456998068032:12345-0@lo:38:kworker/u8:2.0 Request processed in 23us (300us total) trans 0 rc -19/-19 00000100:00100000:3.0:1776178338.658216:0:11281:0:(nrs_fifo.c:241:nrs_fifo_req_stop()) NRS stop fifo request from 12345-0@lo, seq: 6730 00000100:00100000:3.0:1776178338.658225:0:11281:0:(nrs_fifo.c:179:nrs_fifo_req_get()) NRS start fifo request from 12345-0@lo, seq: 6731 00000100:00100000:3.0:1776178338.658228:0:11281:0:(service.c:2324:ptlrpc_server_handle_request()) Handling RPC req@000000001671af6a pname:cluuid+ref:pid:xid:nid:opc:job mdt00_004:0+-99:3352:x1862456998068352:12345-0@lo:38:kworker/u8:2.0 00010000:02000000:3.0:1776178338.658234:0:11281:0:(ldlm_lib.c:1139:target_handle_connect()) lustre-MDT0000: Not available for connect from 0@lo (stopping) 00000100:00100000:3.0:1776178338.658252:0:11281:0:(service.c:2372:ptlrpc_server_handle_request()) Handled RPC req@000000001671af6a pname:cluuid+ref:pid:xid:nid:opc:job mdt00_004:0+-99:3352:x1862456998068352:12345-0@lo:38:kworker/u8:2.0 Request processed in 23us (273us total) trans 0 rc -19/-19 00000100:00100000:3.0:1776178338.658257:0:11281:0:(nrs_fifo.c:241:nrs_fifo_req_stop()) NRS stop fifo request from 12345-0@lo, seq: 6731 00000100:00100000:3.0:1776178338.658305:0:14128:0:(service.c:2152:ptlrpc_server_handle_req_in()) got req x1862456998068160 00000100:00100000:3.0:1776178338.658324:0:14128:0:(nrs_fifo.c:179:nrs_fifo_req_get()) NRS start fifo request from 12345-0@lo, seq: 734 00000100:00100000:3.0:1776178338.658329:0:14128:0:(service.c:2324:ptlrpc_server_handle_request()) Handling RPC req@00000000b0409689 pname:cluuid+ref:pid:xid:nid:opc:job mdt_out00_002:0+-99:3352:x1862456998068160:12345-0@lo:38:kworker/u8:2.0 00010000:02000000:3.0:1776178338.658335:0:14128:0:(ldlm_lib.c:1139:target_handle_connect()) lustre-MDT0000: Not available for connect from 0@lo (stopping) 00000100:00100000:3.0:1776178338.658359:0:14128:0:(service.c:2372:ptlrpc_server_handle_request()) Handled RPC req@00000000b0409689 pname:cluuid+ref:pid:xid:nid:opc:job mdt_out00_002:0+-99:3352:x1862456998068160:12345-0@lo:38:kworker/u8:2.0 Request processed in 30us (413us total) trans 0 rc -19/-19 00000100:00100000:3.0:1776178338.658364:0:14128:0:(nrs_fifo.c:241:nrs_fifo_req_stop()) NRS stop fifo request from 12345-0@lo, seq: 734 00000100:00100000:3.0:1776178338.658399:0:20137:0:(service.c:2152:ptlrpc_server_handle_req_in()) got req x1862456998067840 00000100:00100000:3.0:1776178338.658424:0:20137:0:(nrs_fifo.c:179:nrs_fifo_req_get()) NRS start fifo request from 12345-0@lo, seq: 208 00000100:00100000:3.0:1776178338.658428:0:20137:0:(service.c:2324:ptlrpc_server_handle_request()) Handling RPC req@00000000e22b0439 pname:cluuid+ref:pid:xid:nid:opc:job ll_mgs_0000:ac61f839-0e8e-4cac-9c59-08bcfbbad330+18:3356:x1862456998067840:12345-0@lo:400:kworker/u8:2.0 00000100:00100000:3.0:1776178338.658466:0:20137:0:(service.c:2372:ptlrpc_server_handle_request()) Handled RPC req@00000000e22b0439 pname:cluuid+ref:pid:xid:nid:opc:job ll_mgs_0000:ac61f839-0e8e-4cac-9c59-08bcfbbad330+18:3356:x1862456998067840:12345-0@lo:400:kworker/u8:2.0 Request processed in 37us (1481us total) trans 0 rc 0/0 00000100:00100000:3.0:1776178338.658472:0:20137:0:(nrs_fifo.c:241:nrs_fifo_req_stop()) NRS stop fifo request from 12345-0@lo, seq: 208 00000100:00100000:3.0:1776178338.658513:0:3356:0:(client.c:2215:ptlrpc_check_set()) Completed RPC req@00000000a430b18f pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_03:ac61f839-0e8e-4cac-9c59-08bcfbbad330:3356:1862456998067840:0@lo:400:kworker/u8:2.0 00000100:00080000:3.0:1776178338.658568:0:3352:0:(import.c:86:import_set_state_nolock()) 0000000047601a98 lustre-MDT0000_UUID: changing import state from CONNECTING to DISCONN 00000100:00080000:3.0:1776178338.658572:0:3352:0:(import.c:1429:ptlrpc_connect_interpret()) recovery of lustre-MDT0000_UUID on 192.168.204.103@tcp failed (-19) 00000100:00100000:3.0:1776178338.658575:0:3352:0:(client.c:2215:ptlrpc_check_set()) Completed RPC req@00000000747e17cb pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:lustre-MDT0000-lwp-OST0000_UUID:3352:1862456998067904:0@lo:38:kworker/u8:2.0 00000100:00080000:3.0:1776178338.658582:0:3352:0:(import.c:86:import_set_state_nolock()) 00000000dfc8db13 lustre-MDT0000_UUID: changing import state from CONNECTING to DISCONN 00000100:00080000:3.0:1776178338.658585:0:3352:0:(import.c:1429:ptlrpc_connect_interpret()) recovery of lustre-MDT0000_UUID on 192.168.204.103@tcp failed (-19) 00000100:00100000:3.0:1776178338.658587:0:3352:0:(client.c:2215:ptlrpc_check_set()) Completed RPC req@000000008f48ae7c pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:lustre-MDT0000-lwp-OST0001_UUID:3352:1862456998068032:0@lo:38:kworker/u8:2.0 00000100:00080000:3.0:1776178338.658593:0:3352:0:(import.c:86:import_set_state_nolock()) 00000000ba4f1821 lustre-MDT0000_UUID: changing import state from CONNECTING to DISCONN 00000100:00080000:3.0:1776178338.658596:0:3352:0:(import.c:1429:ptlrpc_connect_interpret()) recovery of lustre-MDT0000_UUID on 192.168.204.103@tcp failed (-19) 00000100:00100000:3.0:1776178338.658598:0:3352:0:(client.c:2215:ptlrpc_check_set()) Completed RPC req@000000001e11faca pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:lustre-MDT0001-mdtlov_UUID:3352:1862456998068160:0@lo:38:kworker/u8:2.0 00000100:00080000:3.0:1776178338.658607:0:3352:0:(import.c:86:import_set_state_nolock()) 00000000a7c55b1f lustre-MDT0000_UUID: changing import state from CONNECTING to DISCONN 00000100:00080000:3.0:1776178338.658609:0:3352:0:(import.c:1429:ptlrpc_connect_interpret()) recovery of lustre-MDT0000_UUID on 192.168.204.103@tcp failed (-19) 00000100:00100000:3.0:1776178338.658611:0:3352:0:(client.c:2215:ptlrpc_check_set()) Completed RPC req@0000000069ebab60 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:lustre-MDT0000-lwp-MDT0001_UUID:3352:1862456998068352:0@lo:38:kworker/u8:2.0 00000100:00100000:1.0:1776178338.658728:0:20759:0:(service.c:2152:ptlrpc_server_handle_req_in()) got req x1862456998068288 00000100:00100000:2.0:1776178338.662957:0:20762:0:(nrs_fifo.c:179:nrs_fifo_req_get()) NRS start fifo request from 12345-0@lo, seq: 617 00000100:00100000:2.0:1776178338.662972:0:20762:0:(service.c:2324:ptlrpc_server_handle_request()) Handling RPC req@0000000045560455 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_013:lustre-MDT0001-mdtlov_UUID+5:3353:x1862456998068224:12345-0@lo:400:kworker/u8:2.0 00000100:00100000:2.0:1776178338.663147:0:20762:0:(service.c:2372:ptlrpc_server_handle_request()) Handled RPC req@0000000045560455 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_013:lustre-MDT0001-mdtlov_UUID+5:3353:x1862456998068224:12345-0@lo:400:kworker/u8:2.0 Request processed in 177us (6340us total) trans 0 rc 0/0 00000100:00100000:2.0:1776178338.663156:0:20762:0:(nrs_fifo.c:241:nrs_fifo_req_stop()) NRS stop fifo request from 12345-0@lo, seq: 617 00000100:00100000:2.0:1776178338.663189:0:20762:0:(nrs_fifo.c:179:nrs_fifo_req_get()) NRS start fifo request from 12345-0@lo, seq: 618 00000100:00100000:2.0:1776178338.663194:0:20762:0:(service.c:2324:ptlrpc_server_handle_request()) Handling RPC req@00000000d0d837c5 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_013:lustre-MDT0001-mdtlov_UUID+5:3356:x1862456998068288:12345-0@lo:400:kworker/u8:2.0 00000100:00100000:2.0:1776178338.663245:0:20762:0:(service.c:2372:ptlrpc_server_handle_request()) Handled RPC req@00000000d0d837c5 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_013:lustre-MDT0001-mdtlov_UUID+5:3356:x1862456998068288:12345-0@lo:400:kworker/u8:2.0 Request processed in 51us (6229us total) trans 0 rc 0/0 00000100:00100000:2.0:1776178338.663251:0:20762:0:(nrs_fifo.c:241:nrs_fifo_req_stop()) NRS stop fifo request from 12345-0@lo, seq: 618 00000100:00100000:2.0:1776178338.663310:0:3356:0:(client.c:2215:ptlrpc_check_set()) Completed RPC req@00000000103d2c47 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_03:lustre-MDT0001-mdtlov_UUID:3356:1862456998068288:0@lo:400:kworker/u8:2.0 00000100:00100000:3.0:1776178338.665874:0:3353:0:(client.c:2215:ptlrpc_check_set()) Completed RPC req@00000000250bc347 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_00:lustre-MDT0001-mdtlov_UUID:3353:1862456998068224:0@lo:400:kworker/u8:2.0 00010000:00010000:2.0:1776178339.299387:0:9225:0:(ldlm_lockd.c:2792:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:0.0:1776178340.193258:0:3355:0:(ptlrpcd.c:415:ptlrpcd_check()) transfer 1 async RPCs [3->2] 00000100:00100000:0.0:1776178340.193291:0:3355:0:(client.c:1733:ptlrpc_send_new_req()) Sending RPC req@00000000b6612f78 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_02:lustre-MDT0001-mdtlov_UUID:3355:1862456998068416:0@lo:13:osp-pre-1-1.0 00000100:00100000:0.0:1776178340.193381:0:3355:0:(events.c:361:request_in_callback()) peer: 12345-0@lo (source: 12345-0@lo) 00000100:00100000:1.0:1776178340.193611:0:8231:0:(service.c:2152:ptlrpc_server_handle_req_in()) got req x1862456998068416 00000100:00100000:1.0:1776178340.193687:0:8231:0:(nrs_fifo.c:179:nrs_fifo_req_get()) NRS start fifo request from 12345-0@lo, seq: 541 00000100:00100000:1.0:1776178340.193696:0:8231:0:(service.c:2324:ptlrpc_server_handle_request()) Handling RPC req@0000000028422d92 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_create00:lustre-MDT0001-mdtlov_UUID+5:3355:x1862456998068416:12345-0@lo:13:osp-pre-1-1.0 00002000:00000024:1.0:1776178340.193748:0:8231:0:(ofd_obd.c:740:ofd_statfs()) blocks cached 229376 granted 491746368 pending 0 free 3923689472 avail 3692167168 00000100:00100000:1.0:1776178340.193815:0:8231:0:(service.c:2372:ptlrpc_server_handle_request()) Handled RPC req@0000000028422d92 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_create00:lustre-MDT0001-mdtlov_UUID+5:3355:x1862456998068416:12345-0@lo:13:osp-pre-1-1.0 Request processed in 119us (436us total) trans 0 rc 0/0 00000100:00100000:1.0:1776178340.193823:0:8231:0:(nrs_fifo.c:241:nrs_fifo_req_stop()) NRS stop fifo request from 12345-0@lo, seq: 541 00000100:00100000:2.0:1776178340.194309:0:3355:0:(client.c:2215:ptlrpc_check_set()) Completed RPC req@00000000b6612f78 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_02:lustre-MDT0001-mdtlov_UUID:3355:1862456998068416:0@lo:13:osp-pre-1-1.0 00010000:00010000:2.0:1776178340.320431:0:6244:0:(ldlm_lockd.c:2792:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:2.0:1776178340.704433:0:3354:0:(ptlrpcd.c:415:ptlrpcd_check()) transfer 1 async RPCs [0->1] 00000100:00100000:2.0:1776178340.704470:0:3354:0:(client.c:1733:ptlrpc_send_new_req()) Sending RPC req@00000000ef862c6d pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_01:lustre-MDT0001-mdtlov_UUID:3354:1862456998068480:0@lo:13:osp-pre-0-1.0 00000100:00100000:2.0:1776178340.704558:0:3354:0:(events.c:361:request_in_callback()) peer: 12345-0@lo (source: 12345-0@lo) 00000100:00100000:0.0:1776178340.704623:0:9554:0:(service.c:2152:ptlrpc_server_handle_req_in()) got req x1862456998068480 00000100:00100000:0.0:1776178340.704688:0:9554:0:(nrs_fifo.c:179:nrs_fifo_req_get()) NRS start fifo request from 12345-0@lo, seq: 542 00000100:00100000:0.0:1776178340.704696:0:9554:0:(service.c:2324:ptlrpc_server_handle_request()) Handling RPC req@00000000b6612f78 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_create00:lustre-MDT0001-mdtlov_UUID+5:3354:x1862456998068480:12345-0@lo:13:osp-pre-0-1.0 00002000:00000024:0.0:1776178340.704739:0:9554:0:(ofd_obd.c:740:ofd_statfs()) blocks cached 315392 granted 484883776 pending 0 free 3923689472 avail 3692167168 00000100:00100000:0.0:1776178340.704869:0:9554:0:(service.c:2372:ptlrpc_server_handle_request()) Handled RPC req@00000000b6612f78 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_create00:lustre-MDT0001-mdtlov_UUID+5:3354:x1862456998068480:12345-0@lo:13:osp-pre-0-1.0 Request processed in 163us (302us total) trans 0 rc 0/0 00000100:00100000:0.0:1776178340.704877:0:9554:0:(nrs_fifo.c:241:nrs_fifo_req_stop()) NRS stop fifo request from 12345-0@lo, seq: 542 00000100:00100000:2.0:1776178340.708819:0:3354:0:(client.c:2215:ptlrpc_check_set()) Completed RPC req@00000000ef862c6d pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_01:lustre-MDT0001-mdtlov_UUID:3354:1862456998068480:0@lo:13:osp-pre-0-1.0 00010000:00010000:2.0:1776178341.344400:0:6245:0:(ldlm_lockd.c:2792:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:0.0:1776178342.185909:0:3347:0:(events.c:361:request_in_callback()) peer: 12345-192.168.204.3@tcp (source: 12345-192.168.204.3@tcp) 00000100:00100000:0.0:1776178342.186050:0:3347:0:(events.c:361:request_in_callback()) peer: 12345-192.168.204.3@tcp (source: 12345-192.168.204.3@tcp) 00000100:00100000:0.0:1776178342.186128:0:3347:0:(events.c:361:request_in_callback()) peer: 12345-192.168.204.3@tcp (source: 12345-192.168.204.3@tcp) 00000100:00100000:0.0:1776178342.186224:0:3347:0:(events.c:361:request_in_callback()) peer: 12345-192.168.204.3@tcp (source: 12345-192.168.204.3@tcp) 00000100:00100000:0.0:1776178342.186339:0:3347:0:(events.c:361:request_in_callback()) peer: 12345-192.168.204.3@tcp (source: 12345-192.168.204.3@tcp) 00000100:00100000:0.0:1776178342.186387:0:20755:0:(service.c:2152:ptlrpc_server_handle_req_in()) got req x1862456979117888 00000100:00100000:0.0:1776178342.186435:0:20755:0:(service.c:2152:ptlrpc_server_handle_req_in()) got req x1862456979117824 00000100:00100000:0.0:1776178342.186461:0:20755:0:(nrs_fifo.c:179:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.204.3@tcp, seq: 619 00000100:00100000:0.0:1776178342.186468:0:20755:0:(service.c:2324:ptlrpc_server_handle_request()) Handling RPC req@00000000ef862c6d pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_006:b50963bd-8eba-4432-9a54-8888d00e3fa1+9:2270:x1862456979117888:12345-192.168.204.3@tcp:400:kworker/u8:4.0 00000100:00100000:0.0:1776178342.186548:0:20755:0:(service.c:2372:ptlrpc_server_handle_request()) Handled RPC req@00000000ef862c6d pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_006:b50963bd-8eba-4432-9a54-8888d00e3fa1+9:2270:x1862456979117888:12345-192.168.204.3@tcp:400:kworker/u8:4.0 Request processed in 79us (325us total) trans 0 rc 0/0 00000100:00100000:0.0:1776178342.186555:0:20755:0:(nrs_fifo.c:241:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.204.3@tcp, seq: 619 00000100:00100000:0.0:1776178342.186587:0:20755:0:(nrs_fifo.c:179:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.204.3@tcp, seq: 620 00000100:00100000:0.0:1776178342.186591:0:20755:0:(service.c:2324:ptlrpc_server_handle_request()) Handling RPC req@00000000dd4b6b2f pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_006:b50963bd-8eba-4432-9a54-8888d00e3fa1+8:2270:x1862456979117824:12345-192.168.204.3@tcp:400:kworker/u8:4.0 00000100:00100000:0.0:1776178342.186616:0:20755:0:(service.c:2372:ptlrpc_server_handle_request()) Handled RPC req@00000000dd4b6b2f pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_006:b50963bd-8eba-4432-9a54-8888d00e3fa1+8:2270:x1862456979117824:12345-192.168.204.3@tcp:400:kworker/u8:4.0 Request processed in 24us (279us total) trans 0 rc 0/0 00000100:00100000:0.0:1776178342.186623:0:20755:0:(nrs_fifo.c:241:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.204.3@tcp, seq: 620 00000100:00100000:2.0:1776178342.192955:0:6256:0:(service.c:2152:ptlrpc_server_handle_req_in()) got req x1862456979117696 00000100:00100000:2.0:1776178342.193042:0:6256:0:(service.c:2152:ptlrpc_server_handle_req_in()) got req x1862456979117760 00000100:00100000:2.0:1776178342.193111:0:6256:0:(nrs_fifo.c:179:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.204.3@tcp, seq: 1009 00000100:00100000:2.0:1776178342.193118:0:6256:0:(service.c:2324:ptlrpc_server_handle_request()) Handling RPC req@0000000014250674 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_001:b50963bd-8eba-4432-9a54-8888d00e3fa1+6:2270:x1862456979117760:12345-192.168.204.3@tcp:400:kworker/u8:4.0 00000100:00100000:2.0:1776178342.193216:0:6256:0:(service.c:2372:ptlrpc_server_handle_request()) Handled RPC req@0000000014250674 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_001:b50963bd-8eba-4432-9a54-8888d00e3fa1+6:2270:x1862456979117760:12345-192.168.204.3@tcp:400:kworker/u8:4.0 Request processed in 98us (7088us total) trans 0 rc 0/0 00000100:00100000:2.0:1776178342.193224:0:6256:0:(nrs_fifo.c:241:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.204.3@tcp, seq: 1009 00000100:00100000:2.0:1776178342.193251:0:6256:0:(nrs_fifo.c:179:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.204.3@tcp, seq: 6732 00000100:00100000:2.0:1776178342.193256:0:6256:0:(service.c:2324:ptlrpc_server_handle_request()) Handling RPC req@0000000039f3bccf pname:cluuid+ref:pid:xid:nid:opc:job mdt00_001:0+-99:2269:x1862456979117696:12345-192.168.204.3@tcp:38:kworker/u8:4.0 00010000:02000000:2.0:1776178342.193274:0:6256:0:(ldlm_lib.c:1139:target_handle_connect()) lustre-MDT0000: Not available for connect from 192.168.204.3@tcp (stopping) 00000100:00100000:2.0:1776178342.193303:0:6256:0:(service.c:2372:ptlrpc_server_handle_request()) Handled RPC req@0000000039f3bccf pname:cluuid+ref:pid:xid:nid:opc:job mdt00_001:0+-99:2269:x1862456979117696:12345-192.168.204.3@tcp:38:kworker/u8:4.0 Request processed in 47us (7255us total) trans 0 rc -19/-19 00000100:00100000:2.0:1776178342.193311:0:6256:0:(nrs_fifo.c:241:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.204.3@tcp, seq: 6732 00000100:00100000:3.0:1776178342.193964:0:20140:0:(service.c:2152:ptlrpc_server_handle_req_in()) got req x1862456979117632 00000100:00100000:3.0:1776178342.194011:0:20140:0:(nrs_fifo.c:179:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.204.3@tcp, seq: 209 00000100:00100000:3.0:1776178342.194127:0:20140:0:(service.c:2324:ptlrpc_server_handle_request()) Handling RPC req@00000000b8c10765 pname:cluuid+ref:pid:xid:nid:opc:job ll_mgs_0002:5b5c66b1-1a2f-4651-8548-0c75b9a38a94+9:2271:x1862456979117632:12345-192.168.204.3@tcp:400:kworker/u8:4.0 00000100:00100000:3.0:1776178342.194181:0:20140:0:(service.c:2372:ptlrpc_server_handle_request()) Handled RPC req@00000000b8c10765 pname:cluuid+ref:pid:xid:nid:opc:job ll_mgs_0002:5b5c66b1-1a2f-4651-8548-0c75b9a38a94+9:2271:x1862456979117632:12345-192.168.204.3@tcp:400:kworker/u8:4.0 Request processed in 56us (8286us total) trans 0 rc 0/0 00000100:00100000:3.0:1776178342.194187:0:20140:0:(nrs_fifo.c:241:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.204.3@tcp, seq: 209 00010000:00010000:2.0:1776178342.368234:0:9224:0:(ldlm_lockd.c:2792:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00010000:00010000:2.0:1776178343.395676:0:9225:0:(ldlm_lockd.c:2792:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00080000:0.0:1776178343.776422:0:37:0:(pinger.c:251:ptlrpc_pinger_process_import()) lustre-MDT0000-lwp-OST0000_UUID->lustre-MDT0000_UUID: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00080000:0.0:1776178343.776435:0:37:0:(recover.c:57:ptlrpc_initiate_recovery()) lustre-MDT0000_UUID: starting recovery 00000100:00080000:0.0:1776178343.776440:0:37:0:(import.c:86:import_set_state_nolock()) 0000000047601a98 lustre-MDT0000_UUID: changing import state from DISCONN to CONNECTING 00000100:00080000:0.0:1776178343.776453:0:37:0:(import.c:534:import_select_connection()) lustre-MDT0000-lwp-OST0000: connect to NID 0@lo last attempt 1045 00000100:00080000:0.0:1776178343.776459:0:37:0:(import.c:581:import_select_connection()) lustre-MDT0000-lwp-OST0000: tried all connections, increasing latency to 21s 00000100:00080000:0.0:1776178343.776469:0:37:0:(import.c:615:import_select_connection()) lustre-MDT0000-lwp-OST0000: import 0000000047601a98 using connection 192.168.204.103@tcp/0@lo 00000100:00100000:0.0:1776178343.776501:0:37:0:(import.c:807:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5) req@000000006342fd4b x1862456998068608/t0(0) o38->lustre-MDT0000-lwp-OST0000@0@lo:12/10 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' 00000100:00080000:0.0:1776178343.776545:0:37:0:(pinger.c:251:ptlrpc_pinger_process_import()) lustre-MDT0000-lwp-OST0001_UUID->lustre-MDT0000_UUID: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00080000:0.0:1776178343.776550:0:37:0:(recover.c:57:ptlrpc_initiate_recovery()) lustre-MDT0000_UUID: starting recovery 00000100:00080000:0.0:1776178343.776552:0:37:0:(import.c:86:import_set_state_nolock()) 00000000dfc8db13 lustre-MDT0000_UUID: changing import state from DISCONN to CONNECTING 00000100:00080000:0.0:1776178343.776556:0:37:0:(import.c:534:import_select_connection()) lustre-MDT0000-lwp-OST0001: connect to NID 0@lo last attempt 1045 00000100:00080000:0.0:1776178343.776559:0:37:0:(import.c:581:import_select_connection()) lustre-MDT0000-lwp-OST0001: tried all connections, increasing latency to 21s 00000100:00080000:0.0:1776178343.776564:0:37:0:(import.c:615:import_select_connection()) lustre-MDT0000-lwp-OST0001: import 00000000dfc8db13 using connection 192.168.204.103@tcp/0@lo 00000100:00100000:0.0:1776178343.776584:0:37:0:(import.c:807:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5) req@0000000030978cff x1862456998068736/t0(0) o38->lustre-MDT0000-lwp-OST0001@0@lo:12/10 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' 00000100:00080000:0.0:1776178343.776612:0:37:0:(pinger.c:251:ptlrpc_pinger_process_import()) lustre-MDT0001-mdtlov_UUID->lustre-MDT0000_UUID: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00080000:0.0:1776178343.776616:0:37:0:(recover.c:57:ptlrpc_initiate_recovery()) lustre-MDT0000_UUID: starting recovery 00000100:00080000:0.0:1776178343.776618:0:37:0:(import.c:86:import_set_state_nolock()) 00000000ba4f1821 lustre-MDT0000_UUID: changing import state from DISCONN to CONNECTING 00000100:00080000:0.0:1776178343.776623:0:37:0:(import.c:534:import_select_connection()) lustre-MDT0000-osp-MDT0001: connect to NID 0@lo last attempt 1045 00000100:00080000:0.0:1776178343.776626:0:37:0:(import.c:581:import_select_connection()) lustre-MDT0000-osp-MDT0001: tried all connections, increasing latency to 21s 00000100:00080000:0.0:1776178343.776630:0:37:0:(import.c:615:import_select_connection()) lustre-MDT0000-osp-MDT0001: import 00000000ba4f1821 using connection 192.168.204.103@tcp/0@lo 00000100:00100000:0.0:1776178343.776648:0:37:0:(import.c:807:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5) req@000000001d072e30 x1862456998068864/t0(0) o38->lustre-MDT0000-osp-MDT0001@0@lo:24/4 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' 00000100:00080000:0.0:1776178343.776697:0:37:0:(pinger.c:251:ptlrpc_pinger_process_import()) lustre-MDT0000-lwp-MDT0001_UUID->lustre-MDT0000_UUID: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00080000:0.0:1776178343.776702:0:37:0:(recover.c:57:ptlrpc_initiate_recovery()) lustre-MDT0000_UUID: starting recovery 00000100:00080000:0.0:1776178343.776704:0:37:0:(import.c:86:import_set_state_nolock()) 00000000a7c55b1f lustre-MDT0000_UUID: changing import state from DISCONN to CONNECTING 00000100:00080000:0.0:1776178343.776709:0:37:0:(import.c:534:import_select_connection()) lustre-MDT0000-lwp-MDT0001: connect to NID 0@lo last attempt 1045 00000100:00080000:0.0:1776178343.776711:0:37:0:(import.c:581:import_select_connection()) lustre-MDT0000-lwp-MDT0001: tried all connections, increasing latency to 21s 00000100:00080000:0.0:1776178343.776715:0:37:0:(import.c:615:import_select_connection()) lustre-MDT0000-lwp-MDT0001: import 00000000a7c55b1f using connection 192.168.204.103@tcp/0@lo 00000100:00100000:0.0:1776178343.776732:0:37:0:(import.c:807:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5) req@00000000a9455d7d x1862456998069056/t0(0) o38->lustre-MDT0000-lwp-MDT0001@0@lo:12/10 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' 00000100:00100000:2.0:1776178343.776889:0:3354:0:(client.c:1733:ptlrpc_send_new_req()) Sending RPC req@00000000ec65370c pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_01:ac61f839-0e8e-4cac-9c59-08bcfbbad330:3354:1862456998068544:0@lo:400:kworker/u8:2.0 00000100:00100000:2.0:1776178343.776982:0:3354:0:(events.c:361:request_in_callback()) peer: 12345-0@lo (source: 12345-0@lo) 00000100:00100000:2.0:1776178343.777022:0:3354:0:(client.c:1733:ptlrpc_send_new_req()) Sending RPC req@0000000084604fbc pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_01:lustre-MDT0001-mdtlov_UUID:3354:1862456998068992:0@lo:400:kworker/u8:2.0 00000100:00100000:2.0:1776178343.777089:0:3354:0:(events.c:361:request_in_callback()) peer: 12345-0@lo (source: 12345-0@lo) 00000100:00100000:2.0:1776178343.777114:0:3354:0:(ptlrpcd.c:415:ptlrpcd_check()) transfer 1 async RPCs [0->1] 00000100:00100000:2.0:1776178343.777120:0:3354:0:(client.c:1733:ptlrpc_send_new_req()) Sending RPC req@000000002e8f0791 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_01:lustre-MDT0001-mdtlov_UUID:3354:1862456998068928:0@lo:400:kworker/u8:2.0 00000100:00100000:2.0:1776178343.777143:0:3354:0:(events.c:361:request_in_callback()) peer: 12345-0@lo (source: 12345-0@lo) 00000100:00100000:2.0:1776178343.777170:0:3355:0:(client.c:1733:ptlrpc_send_new_req()) Sending RPC req@000000008c3fbdb6 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_02:lustre-MDT0001-lwp-OST0000_UUID:3355:1862456998068672:0@lo:400:kworker/u8:2.0 00000100:00100000:2.0:1776178343.777193:0:3355:0:(events.c:361:request_in_callback()) peer: 12345-0@lo (source: 12345-0@lo) 00000100:00100000:2.0:1776178343.777208:0:3355:0:(ptlrpcd.c:415:ptlrpcd_check()) transfer 1 async RPCs [3->2] 00000100:00100000:2.0:1776178343.777211:0:3355:0:(client.c:1733:ptlrpc_send_new_req()) Sending RPC req@000000009497b17e pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_02:lustre-MDT0001-lwp-OST0001_UUID:3355:1862456998068800:0@lo:400:kworker/u8:2.0 00000100:00100000:2.0:1776178343.777231:0:3355:0:(events.c:361:request_in_callback()) peer: 12345-0@lo (source: 12345-0@lo) 00000100:00100000:2.0:1776178343.777263:0:20762:0:(service.c:2152:ptlrpc_server_handle_req_in()) got req x1862456998068992 00000100:00100000:2.0:1776178343.777310:0:20762:0:(service.c:2152:ptlrpc_server_handle_req_in()) got req x1862456998068928 00000100:00100000:2.0:1776178343.777333:0:20762:0:(nrs_fifo.c:179:nrs_fifo_req_get()) NRS start fifo request from 12345-0@lo, seq: 621 00000100:00100000:2.0:1776178343.777338:0:20762:0:(service.c:2324:ptlrpc_server_handle_request()) Handling RPC req@00000000889a4355 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_013:lustre-MDT0001-mdtlov_UUID+5:3354:x1862456998068992:12345-0@lo:400:kworker/u8:2.0 00000100:00100000:2.0:1776178343.777387:0:20762:0:(service.c:2372:ptlrpc_server_handle_request()) Handled RPC req@00000000889a4355 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_013:lustre-MDT0001-mdtlov_UUID+5:3354:x1862456998068992:12345-0@lo:400:kworker/u8:2.0 Request processed in 48us (298us total) trans 0 rc 0/0 00000100:00100000:2.0:1776178343.777393:0:20762:0:(nrs_fifo.c:241:nrs_fifo_req_stop()) NRS stop fifo request from 12345-0@lo, seq: 621 00000100:00100000:2.0:1776178343.777409:0:20762:0:(nrs_fifo.c:179:nrs_fifo_req_get()) NRS start fifo request from 12345-0@lo, seq: 622 00000100:00100000:2.0:1776178343.777413:0:20762:0:(service.c:2324:ptlrpc_server_handle_request()) Handling RPC req@0000000044f82452 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_013:lustre-MDT0001-mdtlov_UUID+5:3354:x1862456998068928:12345-0@lo:400:kworker/u8:2.0 00000100:00100000:2.0:1776178343.777438:0:20762:0:(service.c:2372:ptlrpc_server_handle_request()) Handled RPC req@0000000044f82452 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_013:lustre-MDT0001-mdtlov_UUID+5:3354:x1862456998068928:12345-0@lo:400:kworker/u8:2.0 Request processed in 24us (294us total) trans 0 rc 0/0 00000100:00100000:2.0:1776178343.777442:0:20762:0:(nrs_fifo.c:241:nrs_fifo_req_stop()) NRS stop fifo request from 12345-0@lo, seq: 622 00000100:00100000:2.0:1776178343.777462:0:8250:0:(service.c:2152:ptlrpc_server_handle_req_in()) got req x1862456998068672 00000100:00100000:2.0:1776178343.777487:0:8250:0:(service.c:2152:ptlrpc_server_handle_req_in()) got req x1862456998068800 00000100:00100000:2.0:1776178343.777500:0:8250:0:(nrs_fifo.c:179:nrs_fifo_req_get()) NRS start fifo request from 12345-0@lo, seq: 1010 00000100:00100000:2.0:1776178343.777503:0:8250:0:(service.c:2324:ptlrpc_server_handle_request()) Handling RPC req@00000000b85458e9 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_003:lustre-MDT0001-lwp-OST0000_UUID+5:3355:x1862456998068672:12345-0@lo:400:kworker/u8:2.0 00000100:00100000:2.0:1776178343.777533:0:8250:0:(service.c:2372:ptlrpc_server_handle_request()) Handled RPC req@00000000b85458e9 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_003:lustre-MDT0001-lwp-OST0000_UUID+5:3355:x1862456998068672:12345-0@lo:400:kworker/u8:2.0 Request processed in 28us (338us total) trans 0 rc 0/0 00000100:00100000:2.0:1776178343.777537:0:8250:0:(nrs_fifo.c:241:nrs_fifo_req_stop()) NRS stop fifo request from 12345-0@lo, seq: 1010 00000100:00100000:2.0:1776178343.777548:0:8250:0:(nrs_fifo.c:179:nrs_fifo_req_get()) NRS start fifo request from 12345-0@lo, seq: 1011 00000100:00100000:2.0:1776178343.777551:0:8250:0:(service.c:2324:ptlrpc_server_handle_request()) Handling RPC req@000000007d766b2a pname:cluuid+ref:pid:xid:nid:opc:job mdt00_003:lustre-MDT0001-lwp-OST0001_UUID+5:3355:x1862456998068800:12345-0@lo:400:kworker/u8:2.0 00000100:00100000:2.0:1776178343.777573:0:8250:0:(service.c:2372:ptlrpc_server_handle_request()) Handled RPC req@000000007d766b2a pname:cluuid+ref:pid:xid:nid:opc:job mdt00_003:lustre-MDT0001-lwp-OST0001_UUID+5:3355:x1862456998068800:12345-0@lo:400:kworker/u8:2.0 Request processed in 21us (342us total) trans 0 rc 0/0 00000100:00100000:2.0:1776178343.777578:0:8250:0:(nrs_fifo.c:241:nrs_fifo_req_stop()) NRS stop fifo request from 12345-0@lo, seq: 1011 00000100:00100000:2.0:1776178343.777626:0:3355:0:(client.c:2215:ptlrpc_check_set()) Completed RPC req@000000009497b17e pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_02:lustre-MDT0001-lwp-OST0001_UUID:3355:1862456998068800:0@lo:400:kworker/u8:2.0 00000100:00100000:2.0:1776178343.777635:0:3355:0:(client.c:2215:ptlrpc_check_set()) Completed RPC req@000000008c3fbdb6 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_02:lustre-MDT0001-lwp-OST0000_UUID:3355:1862456998068672:0@lo:400:kworker/u8:2.0 00000100:00100000:2.0:1776178343.777660:0:3354:0:(client.c:2215:ptlrpc_check_set()) Completed RPC req@000000002e8f0791 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_01:lustre-MDT0001-mdtlov_UUID:3354:1862456998068928:0@lo:400:kworker/u8:2.0 00000100:00100000:2.0:1776178343.777669:0:3354:0:(client.c:2215:ptlrpc_check_set()) Completed RPC req@0000000084604fbc pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_01:lustre-MDT0001-mdtlov_UUID:3354:1862456998068992:0@lo:400:kworker/u8:2.0 00000100:00100000:0.0:1776178343.778007:0:20137:0:(service.c:2152:ptlrpc_server_handle_req_in()) got req x1862456998068544 00000100:00100000:0.0:1776178343.778055:0:20137:0:(nrs_fifo.c:179:nrs_fifo_req_get()) NRS start fifo request from 12345-0@lo, seq: 210 00000100:00100000:0.0:1776178343.778079:0:20137:0:(service.c:2324:ptlrpc_server_handle_request()) Handling RPC req@00000000b59e728a pname:cluuid+ref:pid:xid:nid:opc:job ll_mgs_0000:ac61f839-0e8e-4cac-9c59-08bcfbbad330+18:3354:x1862456998068544:12345-0@lo:400:kworker/u8:2.0 00000100:00100000:0.0:1776178343.778119:0:20137:0:(service.c:2372:ptlrpc_server_handle_request()) Handled RPC req@00000000b59e728a pname:cluuid+ref:pid:xid:nid:opc:job ll_mgs_0000:ac61f839-0e8e-4cac-9c59-08bcfbbad330+18:3354:x1862456998068544:12345-0@lo:400:kworker/u8:2.0 Request processed in 39us (1138us total) trans 0 rc 0/0 00000100:00100000:0.0:1776178343.778125:0:20137:0:(nrs_fifo.c:241:nrs_fifo_req_stop()) NRS stop fifo request from 12345-0@lo, seq: 210 00000100:00100000:3.0:1776178343.778221:0:3352:0:(client.c:1733:ptlrpc_send_new_req()) Sending RPC req@000000006342fd4b pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:lustre-MDT0000-lwp-OST0000_UUID:3352:1862456998068608:0@lo:38:kworker/u8:2.0 00000100:00100000:3.0:1776178343.778253:0:3352:0:(events.c:361:request_in_callback()) peer: 12345-0@lo (source: 12345-0@lo) 00000100:00100000:3.0:1776178343.778277:0:3352:0:(client.c:1733:ptlrpc_send_new_req()) Sending RPC req@0000000030978cff pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:lustre-MDT0000-lwp-OST0001_UUID:3352:1862456998068736:0@lo:38:kworker/u8:2.0 00000100:00100000:3.0:1776178343.778296:0:3352:0:(events.c:361:request_in_callback()) peer: 12345-0@lo (source: 12345-0@lo) 00000100:00100000:3.0:1776178343.778310:0:3352:0:(client.c:1733:ptlrpc_send_new_req()) Sending RPC req@000000001d072e30 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:lustre-MDT0001-mdtlov_UUID:3352:1862456998068864:0@lo:38:kworker/u8:2.0 00000100:00100000:3.0:1776178343.778333:0:3352:0:(events.c:361:request_in_callback()) peer: 12345-0@lo (source: 12345-0@lo) 00000100:00100000:3.0:1776178343.778347:0:3352:0:(client.c:1733:ptlrpc_send_new_req()) Sending RPC req@00000000a9455d7d pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:lustre-MDT0000-lwp-MDT0001_UUID:3352:1862456998069056:0@lo:38:kworker/u8:2.0 00000100:00100000:3.0:1776178343.778367:0:3352:0:(events.c:361:request_in_callback()) peer: 12345-0@lo (source: 12345-0@lo) 00000100:00100000:3.0:1776178343.778397:0:6257:0:(service.c:2152:ptlrpc_server_handle_req_in()) got req x1862456998068608 00000100:00100000:3.0:1776178343.778417:0:6257:0:(service.c:2152:ptlrpc_server_handle_req_in()) got req x1862456998068736 00000100:00100000:3.0:1776178343.778425:0:6257:0:(service.c:2152:ptlrpc_server_handle_req_in()) got req x1862456998069056 00000100:00100000:3.0:1776178343.778436:0:6257:0:(nrs_fifo.c:179:nrs_fifo_req_get()) NRS start fifo request from 12345-0@lo, seq: 6733 00000100:00100000:3.0:1776178343.778439:0:6257:0:(service.c:2324:ptlrpc_server_handle_request()) Handling RPC req@00000000fd2b8392 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_002:0+-99:3352:x1862456998068608:12345-0@lo:38:kworker/u8:2.0 00010000:02000000:3.0:1776178343.778451:0:6257:0:(ldlm_lib.c:1139:target_handle_connect()) lustre-MDT0000: Not available for connect from 0@lo (stopping) 00000100:00100000:3.0:1776178343.778483:0:6257:0:(service.c:2372:ptlrpc_server_handle_request()) Handled RPC req@00000000fd2b8392 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_002:0+-99:3352:x1862456998068608:12345-0@lo:38:kworker/u8:2.0 Request processed in 42us (229us total) trans 0 rc -19/-19 00000100:00100000:3.0:1776178343.778489:0:6257:0:(nrs_fifo.c:241:nrs_fifo_req_stop()) NRS stop fifo request from 12345-0@lo, seq: 6733 00000100:00100000:3.0:1776178343.778502:0:6257:0:(nrs_fifo.c:179:nrs_fifo_req_get()) NRS start fifo request from 12345-0@lo, seq: 6734 00000100:00100000:3.0:1776178343.778505:0:6257:0:(service.c:2324:ptlrpc_server_handle_request()) Handling RPC req@00000000e933e46b pname:cluuid+ref:pid:xid:nid:opc:job mdt00_002:0+-99:3352:x1862456998068736:12345-0@lo:38:kworker/u8:2.0 00010000:02000000:3.0:1776178343.778511:0:6257:0:(ldlm_lib.c:1139:target_handle_connect()) lustre-MDT0000: Not available for connect from 0@lo (stopping) 00000100:00100000:3.0:1776178343.778531:0:6257:0:(service.c:2372:ptlrpc_server_handle_request()) Handled RPC req@00000000e933e46b pname:cluuid+ref:pid:xid:nid:opc:job mdt00_002:0+-99:3352:x1862456998068736:12345-0@lo:38:kworker/u8:2.0 Request processed in 24us (235us total) trans 0 rc -19/-19 00000100:00100000:3.0:1776178343.778536:0:6257:0:(nrs_fifo.c:241:nrs_fifo_req_stop()) NRS stop fifo request from 12345-0@lo, seq: 6734 00000100:00100000:3.0:1776178343.778546:0:6257:0:(nrs_fifo.c:179:nrs_fifo_req_get()) NRS start fifo request from 12345-0@lo, seq: 6735 00000100:00100000:3.0:1776178343.778549:0:6257:0:(service.c:2324:ptlrpc_server_handle_request()) Handling RPC req@0000000079529182 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_002:0+-99:3352:x1862456998069056:12345-0@lo:38:kworker/u8:2.0 00010000:02000000:3.0:1776178343.778555:0:6257:0:(ldlm_lib.c:1139:target_handle_connect()) lustre-MDT0000: Not available for connect from 0@lo (stopping) 00000100:00100000:3.0:1776178343.778575:0:6257:0:(service.c:2372:ptlrpc_server_handle_request()) Handled RPC req@0000000079529182 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_002:0+-99:3352:x1862456998069056:12345-0@lo:38:kworker/u8:2.0 Request processed in 25us (208us total) trans 0 rc -19/-19 00000100:00100000:3.0:1776178343.778580:0:6257:0:(nrs_fifo.c:241:nrs_fifo_req_stop()) NRS stop fifo request from 12345-0@lo, seq: 6735 00000100:00100000:3.0:1776178343.778599:0:15213:0:(service.c:2152:ptlrpc_server_handle_req_in()) got req x1862456998068864 00000100:00100000:3.0:1776178343.778618:0:15213:0:(nrs_fifo.c:179:nrs_fifo_req_get()) NRS start fifo request from 12345-0@lo, seq: 735 00000100:00100000:3.0:1776178343.778622:0:15213:0:(service.c:2324:ptlrpc_server_handle_request()) Handling RPC req@0000000004dd3b9b pname:cluuid+ref:pid:xid:nid:opc:job mdt_out00_003:0+-99:3352:x1862456998068864:12345-0@lo:38:kworker/u8:2.0 00010000:02000000:3.0:1776178343.778628:0:15213:0:(ldlm_lib.c:1139:target_handle_connect()) lustre-MDT0000: Not available for connect from 0@lo (stopping) 00000100:00100000:3.0:1776178343.778647:0:15213:0:(service.c:2372:ptlrpc_server_handle_request()) Handled RPC req@0000000004dd3b9b pname:cluuid+ref:pid:xid:nid:opc:job mdt_out00_003:0+-99:3352:x1862456998068864:12345-0@lo:38:kworker/u8:2.0 Request processed in 24us (314us total) trans 0 rc -19/-19 00000100:00100000:3.0:1776178343.778652:0:15213:0:(nrs_fifo.c:241:nrs_fifo_req_stop()) NRS stop fifo request from 12345-0@lo, seq: 735 00000100:00080000:3.0:1776178343.778702:0:3352:0:(import.c:86:import_set_state_nolock()) 0000000047601a98 lustre-MDT0000_UUID: changing import state from CONNECTING to DISCONN 00000100:00080000:3.0:1776178343.778705:0:3352:0:(import.c:1429:ptlrpc_connect_interpret()) recovery of lustre-MDT0000_UUID on 192.168.204.103@tcp failed (-19) 00000100:00100000:3.0:1776178343.778709:0:3352:0:(client.c:2215:ptlrpc_check_set()) Completed RPC req@000000006342fd4b pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:lustre-MDT0000-lwp-OST0000_UUID:3352:1862456998068608:0@lo:38:kworker/u8:2.0 00000100:00080000:3.0:1776178343.778716:0:3352:0:(import.c:86:import_set_state_nolock()) 00000000dfc8db13 lustre-MDT0000_UUID: changing import state from CONNECTING to DISCONN 00000100:00080000:3.0:1776178343.778718:0:3352:0:(import.c:1429:ptlrpc_connect_interpret()) recovery of lustre-MDT0000_UUID on 192.168.204.103@tcp failed (-19) 00000100:00100000:3.0:1776178343.778720:0:3352:0:(client.c:2215:ptlrpc_check_set()) Completed RPC req@0000000030978cff pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:lustre-MDT0000-lwp-OST0001_UUID:3352:1862456998068736:0@lo:38:kworker/u8:2.0 00000100:00080000:3.0:1776178343.778726:0:3352:0:(import.c:86:import_set_state_nolock()) 00000000ba4f1821 lustre-MDT0000_UUID: changing import state from CONNECTING to DISCONN 00000100:00080000:3.0:1776178343.778728:0:3352:0:(import.c:1429:ptlrpc_connect_interpret()) recovery of lustre-MDT0000_UUID on 192.168.204.103@tcp failed (-19) 00000100:00100000:3.0:1776178343.778730:0:3352:0:(client.c:2215:ptlrpc_check_set()) Completed RPC req@000000001d072e30 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:lustre-MDT0001-mdtlov_UUID:3352:1862456998068864:0@lo:38:kworker/u8:2.0 00000100:00080000:3.0:1776178343.778736:0:3352:0:(import.c:86:import_set_state_nolock()) 00000000a7c55b1f lustre-MDT0000_UUID: changing import state from CONNECTING to DISCONN 00000100:00080000:3.0:1776178343.778738:0:3352:0:(import.c:1429:ptlrpc_connect_interpret()) recovery of lustre-MDT0000_UUID on 192.168.204.103@tcp failed (-19) 00000100:00100000:3.0:1776178343.778740:0:3352:0:(client.c:2215:ptlrpc_check_set()) Completed RPC req@00000000a9455d7d pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:lustre-MDT0000-lwp-MDT0001_UUID:3352:1862456998069056:0@lo:38:kworker/u8:2.0 00000100:00100000:2.0:1776178343.779106:0:3354:0:(client.c:2215:ptlrpc_check_set()) Completed RPC req@00000000ec65370c pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_01:ac61f839-0e8e-4cac-9c59-08bcfbbad330:3354:1862456998068544:0@lo:400:kworker/u8:2.0 00010000:00010000:2.0:1776178344.421477:0:6244:0:(ldlm_lockd.c:2792:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:0.0:1776178345.312619:0:3355:0:(client.c:1733:ptlrpc_send_new_req()) Sending RPC req@00000000f433e404 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_02:lustre-MDT0001-mdtlov_UUID:3355:1862456998069120:0@lo:13:osp-pre-1-1.0 00000100:00100000:0.0:1776178345.312721:0:3355:0:(events.c:361:request_in_callback()) peer: 12345-0@lo (source: 12345-0@lo) 00000100:00100000:1.0:1776178345.312930:0:8231:0:(service.c:2152:ptlrpc_server_handle_req_in()) got req x1862456998069120 00000100:00100000:1.0:1776178345.313020:0:8231:0:(nrs_fifo.c:179:nrs_fifo_req_get()) NRS start fifo request from 12345-0@lo, seq: 543 00000100:00100000:1.0:1776178345.313027:0:8231:0:(service.c:2324:ptlrpc_server_handle_request()) Handling RPC req@00000000c8afe7da pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_create00:lustre-MDT0001-mdtlov_UUID+5:3355:x1862456998069120:12345-0@lo:13:osp-pre-1-1.0 00002000:00000024:1.0:1776178345.313101:0:8231:0:(ofd_obd.c:740:ofd_statfs()) blocks cached 229376 granted 491746368 pending 0 free 3923689472 avail 3692167168 00000100:00100000:1.0:1776178345.313170:0:8231:0:(service.c:2372:ptlrpc_server_handle_request()) Handled RPC req@00000000c8afe7da pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_create00:lustre-MDT0001-mdtlov_UUID+5:3355:x1862456998069120:12345-0@lo:13:osp-pre-1-1.0 Request processed in 142us (451us total) trans 0 rc 0/0 00000100:00100000:1.0:1776178345.313179:0:8231:0:(nrs_fifo.c:241:nrs_fifo_req_stop()) NRS stop fifo request from 12345-0@lo, seq: 543 00000100:00100000:2.0:1776178345.313424:0:3355:0:(client.c:2215:ptlrpc_check_set()) Completed RPC req@00000000f433e404 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_02:lustre-MDT0001-mdtlov_UUID:3355:1862456998069120:0@lo:13:osp-pre-1-1.0 00010000:00010000:2.0:1776178345.441840:0:6245:0:(ldlm_lockd.c:2792:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:3.0:1776178345.824396:0:3356:0:(client.c:1733:ptlrpc_send_new_req()) Sending RPC req@000000008b5ee68d pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_03:lustre-MDT0001-mdtlov_UUID:3356:1862456998069184:0@lo:13:osp-pre-0-1.0 00000100:00100000:3.0:1776178345.824472:0:3356:0:(events.c:361:request_in_callback()) peer: 12345-0@lo (source: 12345-0@lo) 00000100:00100000:0.0:1776178345.824630:0:9554:0:(service.c:2152:ptlrpc_server_handle_req_in()) got req x1862456998069184 00000100:00100000:0.0:1776178345.824694:0:9554:0:(nrs_fifo.c:179:nrs_fifo_req_get()) NRS start fifo request from 12345-0@lo, seq: 544 00000100:00100000:0.0:1776178345.824701:0:9554:0:(service.c:2324:ptlrpc_server_handle_request()) Handling RPC req@000000004ce46e20 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_create00:lustre-MDT0001-mdtlov_UUID+5:3356:x1862456998069184:12345-0@lo:13:osp-pre-0-1.0 00002000:00000024:0.0:1776178345.824733:0:9554:0:(ofd_obd.c:740:ofd_statfs()) blocks cached 315392 granted 484883776 pending 0 free 3923689472 avail 3692167168 00000100:00100000:0.0:1776178345.824793:0:9554:0:(service.c:2372:ptlrpc_server_handle_request()) Handled RPC req@000000004ce46e20 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_create00:lustre-MDT0001-mdtlov_UUID+5:3356:x1862456998069184:12345-0@lo:13:osp-pre-0-1.0 Request processed in 91us (323us total) trans 0 rc 0/0 00000100:00100000:0.0:1776178345.824802:0:9554:0:(nrs_fifo.c:241:nrs_fifo_req_stop()) NRS stop fifo request from 12345-0@lo, seq: 544 00000100:00100000:3.0:1776178345.824894:0:3356:0:(client.c:2215:ptlrpc_check_set()) Completed RPC req@000000008b5ee68d pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_03:lustre-MDT0001-mdtlov_UUID:3356:1862456998069184:0@lo:13:osp-pre-0-1.0 00010000:00010000:2.0:1776178346.464461:0:9224:0:(ldlm_lockd.c:2792:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:0.0:1776178347.302772:0:3346:0:(events.c:361:request_in_callback()) peer: 12345-192.168.204.3@tcp (source: 12345-192.168.204.3@tcp) 00000100:00100000:0.0:1776178347.302870:0:3346:0:(events.c:361:request_in_callback()) peer: 12345-192.168.204.3@tcp (source: 12345-192.168.204.3@tcp) 00000100:00100000:0.0:1776178347.302913:0:3346:0:(events.c:361:request_in_callback()) peer: 12345-192.168.204.3@tcp (source: 12345-192.168.204.3@tcp) 00000100:00100000:0.0:1776178347.302938:0:3346:0:(events.c:361:request_in_callback()) peer: 12345-192.168.204.3@tcp (source: 12345-192.168.204.3@tcp) 00000100:00100000:0.0:1776178347.303035:0:3346:0:(events.c:361:request_in_callback()) peer: 12345-192.168.204.3@tcp (source: 12345-192.168.204.3@tcp) 00000100:00100000:1.0:1776178347.303106:0:20759:0:(service.c:2152:ptlrpc_server_handle_req_in()) got req x1862456979118208 00000100:00100000:1.0:1776178347.303158:0:20759:0:(service.c:2152:ptlrpc_server_handle_req_in()) got req x1862456979118144 00000100:00100000:1.0:1776178347.303176:0:20759:0:(nrs_fifo.c:179:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.204.3@tcp, seq: 623 00000100:00100000:1.0:1776178347.303183:0:20759:0:(service.c:2324:ptlrpc_server_handle_request()) Handling RPC req@000000000489d393 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_010:b50963bd-8eba-4432-9a54-8888d00e3fa1+9:2270:x1862456979118208:12345-192.168.204.3@tcp:400:kworker/u8:3.0 00000100:00100000:1.0:1776178347.303254:0:20759:0:(service.c:2372:ptlrpc_server_handle_request()) Handled RPC req@000000000489d393 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_010:b50963bd-8eba-4432-9a54-8888d00e3fa1+9:2270:x1862456979118208:12345-192.168.204.3@tcp:400:kworker/u8:3.0 Request processed in 71us (385us total) trans 0 rc 0/0 00000100:00100000:1.0:1776178347.303261:0:20759:0:(nrs_fifo.c:241:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.204.3@tcp, seq: 623 00000100:00100000:1.0:1776178347.303276:0:20759:0:(nrs_fifo.c:179:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.204.3@tcp, seq: 624 00000100:00100000:1.0:1776178347.303280:0:20759:0:(service.c:2324:ptlrpc_server_handle_request()) Handling RPC req@00000000a0a4914c pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_010:b50963bd-8eba-4432-9a54-8888d00e3fa1+8:2270:x1862456979118144:12345-192.168.204.3@tcp:400:kworker/u8:3.0 00000100:00100000:1.0:1776178347.303302:0:20759:0:(service.c:2372:ptlrpc_server_handle_request()) Handled RPC req@00000000a0a4914c pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_010:b50963bd-8eba-4432-9a54-8888d00e3fa1+8:2270:x1862456979118144:12345-192.168.204.3@tcp:400:kworker/u8:3.0 Request processed in 21us (364us total) trans 0 rc 0/0 00000100:00100000:1.0:1776178347.303308:0:20759:0:(nrs_fifo.c:241:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.204.3@tcp, seq: 624 00000100:00100000:3.0:1776178347.303568:0:20140:0:(service.c:2152:ptlrpc_server_handle_req_in()) got req x1862456979117952 00000100:00100000:3.0:1776178347.303609:0:20140:0:(nrs_fifo.c:179:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.204.3@tcp, seq: 211 00000100:00100000:3.0:1776178347.303613:0:20140:0:(service.c:2324:ptlrpc_server_handle_request()) Handling RPC req@00000000dd6bfaa2 pname:cluuid+ref:pid:xid:nid:opc:job ll_mgs_0002:5b5c66b1-1a2f-4651-8548-0c75b9a38a94+9:2270:x1862456979117952:12345-192.168.204.3@tcp:400:kworker/u8:3.0 00000100:00100000:3.0:1776178347.304891:0:20140:0:(service.c:2372:ptlrpc_server_handle_request()) Handled RPC req@00000000dd6bfaa2 pname:cluuid+ref:pid:xid:nid:opc:job ll_mgs_0002:5b5c66b1-1a2f-4651-8548-0c75b9a38a94+9:2270:x1862456979117952:12345-192.168.204.3@tcp:400:kworker/u8:3.0 Request processed in 1273us (1975us total) trans 0 rc 0/0 00000100:00100000:3.0:1776178347.304903:0:20140:0:(nrs_fifo.c:241:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.204.3@tcp, seq: 211 00000100:00100000:2.0:1776178347.306513:0:6256:0:(service.c:2152:ptlrpc_server_handle_req_in()) got req x1862456979118080 00000100:00100000:1.0:1776178347.306743:0:6255:0:(service.c:2152:ptlrpc_server_handle_req_in()) got req x1862456979118016 00000100:00100000:1.0:1776178347.306776:0:6255:0:(nrs_fifo.c:179:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.204.3@tcp, seq: 1012 00000100:00100000:1.0:1776178347.306783:0:6255:0:(service.c:2324:ptlrpc_server_handle_request()) Handling RPC req@00000000df20940e pname:cluuid+ref:pid:xid:nid:opc:job mdt00_000:b50963bd-8eba-4432-9a54-8888d00e3fa1+6:2270:x1862456979118080:12345-192.168.204.3@tcp:400:kworker/u8:3.0 00000100:00100000:1.0:1776178347.306853:0:6255:0:(service.c:2372:ptlrpc_server_handle_request()) Handled RPC req@00000000df20940e pname:cluuid+ref:pid:xid:nid:opc:job mdt00_000:b50963bd-8eba-4432-9a54-8888d00e3fa1+6:2270:x1862456979118080:12345-192.168.204.3@tcp:400:kworker/u8:3.0 Request processed in 70us (4090us total) trans 0 rc 0/0 00000100:00100000:1.0:1776178347.306860:0:6255:0:(nrs_fifo.c:241:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.204.3@tcp, seq: 1012 00000100:00100000:1.0:1776178347.306882:0:6255:0:(nrs_fifo.c:179:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.204.3@tcp, seq: 6736 00000100:00100000:1.0:1776178347.306886:0:6255:0:(service.c:2324:ptlrpc_server_handle_request()) Handling RPC req@00000000574c4997 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_000:0+-99:2269:x1862456979118016:12345-192.168.204.3@tcp:38:kworker/u8:3.0 00010000:02000000:1.0:1776178347.306900:0:6255:0:(ldlm_lib.c:1139:target_handle_connect()) lustre-MDT0000: Not available for connect from 192.168.204.3@tcp (stopping) 00000100:00100000:1.0:1776178347.306927:0:6255:0:(service.c:2372:ptlrpc_server_handle_request()) Handled RPC req@00000000574c4997 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_000:0+-99:2269:x1862456979118016:12345-192.168.204.3@tcp:38:kworker/u8:3.0 Request processed in 40us (3892us total) trans 0 rc -19/-19 00000100:00100000:1.0:1776178347.306934:0:6255:0:(nrs_fifo.c:241:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.204.3@tcp, seq: 6736 00010000:00010000:2.0:1776178347.488855:0:9225:0:(ldlm_lockd.c:2792:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00010000:00010000:2.0:1776178348.512516:0:6244:0:(ldlm_lockd.c:2792:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00080000:3.0:1776178348.897803:0:36:0:(pinger.c:251:ptlrpc_pinger_process_import()) lustre-MDT0000-lwp-OST0000_UUID->lustre-MDT0000_UUID: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00080000:3.0:1776178348.897812:0:36:0:(recover.c:57:ptlrpc_initiate_recovery()) lustre-MDT0000_UUID: starting recovery 00000100:00080000:3.0:1776178348.897817:0:36:0:(import.c:86:import_set_state_nolock()) 0000000047601a98 lustre-MDT0000_UUID: changing import state from DISCONN to CONNECTING 00000100:00080000:3.0:1776178348.897827:0:36:0:(import.c:534:import_select_connection()) lustre-MDT0000-lwp-OST0000: connect to NID 0@lo last attempt 1051 00000100:00080000:3.0:1776178348.897830:0:36:0:(import.c:581:import_select_connection()) lustre-MDT0000-lwp-OST0000: tried all connections, increasing latency to 21s 00000100:00080000:3.0:1776178348.897836:0:36:0:(import.c:615:import_select_connection()) lustre-MDT0000-lwp-OST0000: import 0000000047601a98 using connection 192.168.204.103@tcp/0@lo 00000100:00100000:3.0:1776178348.897860:0:36:0:(import.c:807:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5) req@0000000042de09e9 x1862456998069312/t0(0) o38->lustre-MDT0000-lwp-OST0000@0@lo:12/10 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' 00000100:00080000:3.0:1776178348.897888:0:36:0:(pinger.c:251:ptlrpc_pinger_process_import()) lustre-MDT0000-lwp-OST0001_UUID->lustre-MDT0000_UUID: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00080000:3.0:1776178348.897892:0:36:0:(recover.c:57:ptlrpc_initiate_recovery()) lustre-MDT0000_UUID: starting recovery 00000100:00080000:3.0:1776178348.897894:0:36:0:(import.c:86:import_set_state_nolock()) 00000000dfc8db13 lustre-MDT0000_UUID: changing import state from DISCONN to CONNECTING 00000100:00080000:3.0:1776178348.897897:0:36:0:(import.c:534:import_select_connection()) lustre-MDT0000-lwp-OST0001: connect to NID 0@lo last attempt 1051 00000100:00080000:3.0:1776178348.897900:0:36:0:(import.c:581:import_select_connection()) lustre-MDT0000-lwp-OST0001: tried all connections, increasing latency to 21s 00000100:00080000:3.0:1776178348.897904:0:36:0:(import.c:615:import_select_connection()) lustre-MDT0000-lwp-OST0001: import 00000000dfc8db13 using connection 192.168.204.103@tcp/0@lo 00000100:00100000:3.0:1776178348.897913:0:36:0:(import.c:807:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5) req@000000006a3d91b0 x1862456998069440/t0(0) o38->lustre-MDT0000-lwp-OST0001@0@lo:12/10 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' 00000100:00080000:3.0:1776178348.897956:0:36:0:(pinger.c:251:ptlrpc_pinger_process_import()) lustre-MDT0001-mdtlov_UUID->lustre-MDT0000_UUID: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00080000:3.0:1776178348.897960:0:36:0:(recover.c:57:ptlrpc_initiate_recovery()) lustre-MDT0000_UUID: starting recovery 00000100:00080000:3.0:1776178348.897961:0:36:0:(import.c:86:import_set_state_nolock()) 00000000ba4f1821 lustre-MDT0000_UUID: changing import state from DISCONN to CONNECTING 00000100:00080000:3.0:1776178348.897965:0:36:0:(import.c:534:import_select_connection()) lustre-MDT0000-osp-MDT0001: connect to NID 0@lo last attempt 1051 00000100:00080000:3.0:1776178348.897967:0:36:0:(import.c:581:import_select_connection()) lustre-MDT0000-osp-MDT0001: tried all connections, increasing latency to 21s 00000100:00080000:3.0:1776178348.897971:0:36:0:(import.c:615:import_select_connection()) lustre-MDT0000-osp-MDT0001: import 00000000ba4f1821 using connection 192.168.204.103@tcp/0@lo 00000100:00100000:3.0:1776178348.897985:0:36:0:(import.c:807:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5) req@0000000075189f40 x1862456998069568/t0(0) o38->lustre-MDT0000-osp-MDT0001@0@lo:24/4 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' 00000100:00080000:3.0:1776178348.898002:0:36:0:(pinger.c:251:ptlrpc_pinger_process_import()) lustre-MDT0000-lwp-MDT0001_UUID->lustre-MDT0000_UUID: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00080000:3.0:1776178348.898006:0:36:0:(recover.c:57:ptlrpc_initiate_recovery()) lustre-MDT0000_UUID: starting recovery 00000100:00080000:3.0:1776178348.898007:0:36:0:(import.c:86:import_set_state_nolock()) 00000000a7c55b1f lustre-MDT0000_UUID: changing import state from DISCONN to CONNECTING 00000100:00080000:3.0:1776178348.898011:0:36:0:(import.c:534:import_select_connection()) lustre-MDT0000-lwp-MDT0001: connect to NID 0@lo last attempt 1051 00000100:00080000:3.0:1776178348.898013:0:36:0:(import.c:581:import_select_connection()) lustre-MDT0000-lwp-MDT0001: tried all connections, increasing latency to 21s 00000100:00080000:3.0:1776178348.898016:0:36:0:(import.c:615:import_select_connection()) lustre-MDT0000-lwp-MDT0001: import 00000000a7c55b1f using connection 192.168.204.103@tcp/0@lo 00000100:00100000:3.0:1776178348.898025:0:36:0:(import.c:807:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5) req@0000000037aace32 x1862456998069760/t0(0) o38->lustre-MDT0000-lwp-MDT0001@0@lo:12/10 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' 00000100:00100000:3.0:1776178348.898248:0:3355:0:(client.c:1733:ptlrpc_send_new_req()) Sending RPC req@0000000029ef3988 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_02:lustre-MDT0001-lwp-OST0001_UUID:3355:1862456998069504:0@lo:400:kworker/u8:1.0 00000100:00100000:3.0:1776178348.898312:0:3355:0:(events.c:361:request_in_callback()) peer: 12345-0@lo (source: 12345-0@lo) 00000100:00100000:3.0:1776178348.898344:0:3355:0:(ptlrpcd.c:415:ptlrpcd_check()) transfer 1 async RPCs [3->2] 00000100:00100000:3.0:1776178348.898349:0:3355:0:(client.c:1733:ptlrpc_send_new_req()) Sending RPC req@000000000a0451ff pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_02:lustre-MDT0001-mdtlov_UUID:3355:1862456998069632:0@lo:400:kworker/u8:1.0 00000100:00100000:3.0:1776178348.898454:0:3355:0:(events.c:361:request_in_callback()) peer: 12345-0@lo (source: 12345-0@lo) 00000100:00100000:3.0:1776178348.898494:0:20755:0:(service.c:2152:ptlrpc_server_handle_req_in()) got req x1862456998069632 00000100:00100000:3.0:1776178348.898540:0:20755:0:(nrs_fifo.c:179:nrs_fifo_req_get()) NRS start fifo request from 12345-0@lo, seq: 625 00000100:00100000:3.0:1776178348.898547:0:20755:0:(service.c:2324:ptlrpc_server_handle_request()) Handling RPC req@000000009c4a59b3 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_006:lustre-MDT0001-mdtlov_UUID+5:3355:x1862456998069632:12345-0@lo:400:kworker/u8:1.0 00000100:00100000:3.0:1776178348.898600:0:20755:0:(service.c:2372:ptlrpc_server_handle_request()) Handled RPC req@000000009c4a59b3 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_006:lustre-MDT0001-mdtlov_UUID+5:3355:x1862456998069632:12345-0@lo:400:kworker/u8:1.0 Request processed in 53us (147us total) trans 0 rc 0/0 00000100:00100000:3.0:1776178348.898608:0:20755:0:(nrs_fifo.c:241:nrs_fifo_req_stop()) NRS stop fifo request from 12345-0@lo, seq: 625 00000100:00100000:3.0:1776178348.898670:0:3355:0:(client.c:2215:ptlrpc_check_set()) Completed RPC req@000000000a0451ff pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_02:lustre-MDT0001-mdtlov_UUID:3355:1862456998069632:0@lo:400:kworker/u8:1.0 00000100:00100000:3.0:1776178348.898699:0:11281:0:(service.c:2152:ptlrpc_server_handle_req_in()) got req x1862456998069504 00000100:00100000:3.0:1776178348.898726:0:11281:0:(nrs_fifo.c:179:nrs_fifo_req_get()) NRS start fifo request from 12345-0@lo, seq: 1013 00000100:00100000:3.0:1776178348.898730:0:11281:0:(service.c:2324:ptlrpc_server_handle_request()) Handling RPC req@00000000542ce52f pname:cluuid+ref:pid:xid:nid:opc:job mdt00_004:lustre-MDT0001-lwp-OST0001_UUID+5:3355:x1862456998069504:12345-0@lo:400:kworker/u8:1.0 00000100:00100000:3.0:1776178348.898762:0:11281:0:(service.c:2372:ptlrpc_server_handle_request()) Handled RPC req@00000000542ce52f pname:cluuid+ref:pid:xid:nid:opc:job mdt00_004:lustre-MDT0001-lwp-OST0001_UUID+5:3355:x1862456998069504:12345-0@lo:400:kworker/u8:1.0 Request processed in 31us (451us total) trans 0 rc 0/0 00000100:00100000:3.0:1776178348.898767:0:11281:0:(nrs_fifo.c:241:nrs_fifo_req_stop()) NRS stop fifo request from 12345-0@lo, seq: 1013 00000100:00100000:3.0:1776178348.898793:0:3355:0:(client.c:2215:ptlrpc_check_set()) Completed RPC req@0000000029ef3988 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_02:lustre-MDT0001-lwp-OST0001_UUID:3355:1862456998069504:0@lo:400:kworker/u8:1.0 00000100:00100000:3.0:1776178348.898810:0:3352:0:(client.c:1733:ptlrpc_send_new_req()) Sending RPC req@0000000042de09e9 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:lustre-MDT0000-lwp-OST0000_UUID:3352:1862456998069312:0@lo:38:kworker/u8:1.0 00000100:00100000:3.0:1776178348.898838:0:3352:0:(events.c:361:request_in_callback()) peer: 12345-0@lo (source: 12345-0@lo) 00000100:00100000:3.0:1776178348.898852:0:3352:0:(client.c:1733:ptlrpc_send_new_req()) Sending RPC req@000000006a3d91b0 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:lustre-MDT0000-lwp-OST0001_UUID:3352:1862456998069440:0@lo:38:kworker/u8:1.0 00000100:00100000:0.0:1776178348.902162:0:6256:0:(service.c:2152:ptlrpc_server_handle_req_in()) got req x1862456998069312 00000100:00100000:0.0:1776178348.902260:0:6256:0:(nrs_fifo.c:179:nrs_fifo_req_get()) NRS start fifo request from 12345-0@lo, seq: 6737 00000100:00100000:0.0:1776178348.902269:0:6256:0:(service.c:2324:ptlrpc_server_handle_request()) Handling RPC req@0000000029ef3988 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_001:0+-99:3352:x1862456998069312:12345-0@lo:38:kworker/u8:1.0 00010000:02000000:0.0:1776178348.902300:0:6256:0:(ldlm_lib.c:1139:target_handle_connect()) lustre-MDT0000: Not available for connect from 0@lo (stopping) 00000100:00100000:2.0:1776178348.902422:0:3354:0:(client.c:1733:ptlrpc_send_new_req()) Sending RPC req@00000000e8195050 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_01:lustre-MDT0001-lwp-OST0000_UUID:3354:1862456998069376:0@lo:400:kworker/u8:1.0 00000100:00100000:0.0:1776178348.906252:0:3353:0:(client.c:1733:ptlrpc_send_new_req()) Sending RPC req@000000004828c19d pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_00:ac61f839-0e8e-4cac-9c59-08bcfbbad330:3353:1862456998069248:0@lo:400:kworker/u8:1.0 00000100:00100000:0.0:1776178348.906401:0:3353:0:(events.c:361:request_in_callback()) peer: 12345-0@lo (source: 12345-0@lo) 00000100:00100000:2.0:1776178348.906579:0:3354:0:(events.c:361:request_in_callback()) peer: 12345-0@lo (source: 12345-0@lo) 00000100:00100000:2.0:1776178348.906646:0:11281:0:(service.c:2152:ptlrpc_server_handle_req_in()) got req x1862456998069376 00000100:00100000:2.0:1776178348.906730:0:11281:0:(nrs_fifo.c:179:nrs_fifo_req_get()) NRS start fifo request from 12345-0@lo, seq: 1014 00000100:00100000:2.0:1776178348.906736:0:11281:0:(service.c:2324:ptlrpc_server_handle_request()) Handling RPC req@00000000f433e404 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_004:lustre-MDT0001-lwp-OST0000_UUID+5:3354:x1862456998069376:12345-0@lo:400:kworker/u8:1.0 00000100:00100000:2.0:1776178348.906793:0:11281:0:(service.c:2372:ptlrpc_server_handle_request()) Handled RPC req@00000000f433e404 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_004:lustre-MDT0001-lwp-OST0000_UUID+5:3354:x1862456998069376:12345-0@lo:400:kworker/u8:1.0 Request processed in 57us (215us total) trans 0 rc 0/0 00000100:00100000:2.0:1776178348.906801:0:11281:0:(nrs_fifo.c:241:nrs_fifo_req_stop()) NRS stop fifo request from 12345-0@lo, seq: 1014 00000100:00100000:2.0:1776178348.906851:0:3354:0:(client.c:2215:ptlrpc_check_set()) Completed RPC req@00000000e8195050 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_01:lustre-MDT0001-lwp-OST0000_UUID:3354:1862456998069376:0@lo:400:kworker/u8:1.0 00000100:00100000:1.0:1776178348.909282:0:6256:0:(service.c:2372:ptlrpc_server_handle_request()) Handled RPC req@0000000029ef3988 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_001:0+-99:3352:x1862456998069312:12345-0@lo:38:kworker/u8:1.0 Request processed in 6998us (10428us total) trans 0 rc -19/-19 00000100:00100000:1.0:1776178348.909307:0:6256:0:(nrs_fifo.c:241:nrs_fifo_req_stop()) NRS stop fifo request from 12345-0@lo, seq: 6737 00000100:00100000:1.0:1776178348.911317:0:20137:0:(service.c:2152:ptlrpc_server_handle_req_in()) got req x1862456998069248 00000100:00100000:1.0:1776178348.911404:0:20137:0:(nrs_fifo.c:179:nrs_fifo_req_get()) NRS start fifo request from 12345-0@lo, seq: 212 00000100:00100000:1.0:1776178348.911413:0:20137:0:(service.c:2324:ptlrpc_server_handle_request()) Handling RPC req@00000000ddaf121b pname:cluuid+ref:pid:xid:nid:opc:job ll_mgs_0000:ac61f839-0e8e-4cac-9c59-08bcfbbad330+18:3353:x1862456998069248:12345-0@lo:400:kworker/u8:1.0 00000100:00100000:1.0:1776178348.911533:0:20137:0:(service.c:2372:ptlrpc_server_handle_request()) Handled RPC req@00000000ddaf121b pname:cluuid+ref:pid:xid:nid:opc:job ll_mgs_0000:ac61f839-0e8e-4cac-9c59-08bcfbbad330+18:3353:x1862456998069248:12345-0@lo:400:kworker/u8:1.0 Request processed in 121us (5134us total) trans 0 rc 0/0 00000100:00100000:1.0:1776178348.911541:0:20137:0:(nrs_fifo.c:241:nrs_fifo_req_stop()) NRS stop fifo request from 12345-0@lo, seq: 212 00000100:00100000:3.0:1776178348.912023:0:3352:0:(events.c:361:request_in_callback()) peer: 12345-0@lo (source: 12345-0@lo) 00000100:00100000:3.0:1776178348.912087:0:3352:0:(client.c:1733:ptlrpc_send_new_req()) Sending RPC req@0000000075189f40 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:lustre-MDT0001-mdtlov_UUID:3352:1862456998069568:0@lo:38:kworker/u8:1.0 00000100:00100000:3.0:1776178348.912131:0:3352:0:(events.c:361:request_in_callback()) peer: 12345-0@lo (source: 12345-0@lo) 00000100:00100000:3.0:1776178348.912167:0:14128:0:(service.c:2152:ptlrpc_server_handle_req_in()) got req x1862456998069568 00000100:00100000:3.0:1776178348.912194:0:14128:0:(nrs_fifo.c:179:nrs_fifo_req_get()) NRS start fifo request from 12345-0@lo, seq: 736 00000100:00100000:3.0:1776178348.912199:0:14128:0:(service.c:2324:ptlrpc_server_handle_request()) Handling RPC req@000000002300ff10 pname:cluuid+ref:pid:xid:nid:opc:job mdt_out00_002:0+-99:3352:x1862456998069568:12345-0@lo:38:kworker/u8:1.0 00010000:02000000:3.0:1776178348.912217:0:14128:0:(ldlm_lib.c:1139:target_handle_connect()) lustre-MDT0000: Not available for connect from 0@lo (stopping) 00000100:00100000:3.0:1776178348.912249:0:14128:0:(service.c:2372:ptlrpc_server_handle_request()) Handled RPC req@000000002300ff10 pname:cluuid+ref:pid:xid:nid:opc:job mdt_out00_002:0+-99:3352:x1862456998069568:12345-0@lo:38:kworker/u8:1.0 Request processed in 49us (117us total) trans 0 rc -19/-19 00000100:00100000:3.0:1776178348.912254:0:14128:0:(nrs_fifo.c:241:nrs_fifo_req_stop()) NRS stop fifo request from 12345-0@lo, seq: 736 00000100:00100000:3.0:1776178348.912276:0:3352:0:(client.c:1733:ptlrpc_send_new_req()) Sending RPC req@0000000037aace32 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:lustre-MDT0000-lwp-MDT0001_UUID:3352:1862456998069760:0@lo:38:kworker/u8:1.0 00000100:00100000:3.0:1776178348.912298:0:3352:0:(events.c:361:request_in_callback()) peer: 12345-0@lo (source: 12345-0@lo) 00000100:00100000:3.0:1776178348.912332:0:6255:0:(service.c:2152:ptlrpc_server_handle_req_in()) got req x1862456998069440 00000100:00100000:3.0:1776178348.912353:0:6255:0:(service.c:2152:ptlrpc_server_handle_req_in()) got req x1862456998069760 00000100:00100000:3.0:1776178348.912370:0:6255:0:(nrs_fifo.c:179:nrs_fifo_req_get()) NRS start fifo request from 12345-0@lo, seq: 6738 00000100:00100000:3.0:1776178348.912373:0:6255:0:(service.c:2324:ptlrpc_server_handle_request()) Handling RPC req@000000000a0451ff pname:cluuid+ref:pid:xid:nid:opc:job mdt00_000:0+-99:3352:x1862456998069440:12345-0@lo:38:kworker/u8:1.0 00010000:02000000:3.0:1776178348.912392:0:6255:0:(ldlm_lib.c:1139:target_handle_connect()) lustre-MDT0000: Not available for connect from 0@lo (stopping) 00000100:00100000:3.0:1776178348.912418:0:6255:0:(service.c:2372:ptlrpc_server_handle_request()) Handled RPC req@000000000a0451ff pname:cluuid+ref:pid:xid:nid:opc:job mdt00_000:0+-99:3352:x1862456998069440:12345-0@lo:38:kworker/u8:1.0 Request processed in 44us (398us total) trans 0 rc -19/-19 00000100:00100000:3.0:1776178348.912424:0:6255:0:(nrs_fifo.c:241:nrs_fifo_req_stop()) NRS stop fifo request from 12345-0@lo, seq: 6738 00000100:00100000:3.0:1776178348.912447:0:6255:0:(nrs_fifo.c:179:nrs_fifo_req_get()) NRS start fifo request from 12345-0@lo, seq: 6739 00000100:00100000:3.0:1776178348.912451:0:6255:0:(service.c:2324:ptlrpc_server_handle_request()) Handling RPC req@00000000be2fd98d pname:cluuid+ref:pid:xid:nid:opc:job mdt00_000:0+-99:3352:x1862456998069760:12345-0@lo:38:kworker/u8:1.0 00010000:02000000:3.0:1776178348.912457:0:6255:0:(ldlm_lib.c:1139:target_handle_connect()) lustre-MDT0000: Not available for connect from 0@lo (stopping) 00000100:00100000:3.0:1776178348.912478:0:6255:0:(service.c:2372:ptlrpc_server_handle_request()) Handled RPC req@00000000be2fd98d pname:cluuid+ref:pid:xid:nid:opc:job mdt00_000:0+-99:3352:x1862456998069760:12345-0@lo:38:kworker/u8:1.0 Request processed in 26us (180us total) trans 0 rc -19/-19 00000100:00100000:3.0:1776178348.912484:0:6255:0:(nrs_fifo.c:241:nrs_fifo_req_stop()) NRS stop fifo request from 12345-0@lo, seq: 6739 00000100:00080000:3.0:1776178348.912521:0:3352:0:(import.c:86:import_set_state_nolock()) 0000000047601a98 lustre-MDT0000_UUID: changing import state from CONNECTING to DISCONN 00000100:00080000:3.0:1776178348.912526:0:3352:0:(import.c:1429:ptlrpc_connect_interpret()) recovery of lustre-MDT0000_UUID on 192.168.204.103@tcp failed (-19) 00000100:00100000:3.0:1776178348.912530:0:3352:0:(client.c:2215:ptlrpc_check_set()) Completed RPC req@0000000042de09e9 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:lustre-MDT0000-lwp-OST0000_UUID:3352:1862456998069312:0@lo:38:kworker/u8:1.0 00000100:00080000:3.0:1776178348.912540:0:3352:0:(import.c:86:import_set_state_nolock()) 00000000dfc8db13 lustre-MDT0000_UUID: changing import state from CONNECTING to DISCONN 00000100:00080000:3.0:1776178348.912543:0:3352:0:(import.c:1429:ptlrpc_connect_interpret()) recovery of lustre-MDT0000_UUID on 192.168.204.103@tcp failed (-19) 00000100:00100000:3.0:1776178348.912545:0:3352:0:(client.c:2215:ptlrpc_check_set()) Completed RPC req@000000006a3d91b0 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:lustre-MDT0000-lwp-OST0001_UUID:3352:1862456998069440:0@lo:38:kworker/u8:1.0 00000100:00080000:3.0:1776178348.912551:0:3352:0:(import.c:86:import_set_state_nolock()) 00000000ba4f1821 lustre-MDT0000_UUID: changing import state from CONNECTING to DISCONN 00000100:00080000:3.0:1776178348.912554:0:3352:0:(import.c:1429:ptlrpc_connect_interpret()) recovery of lustre-MDT0000_UUID on 192.168.204.103@tcp failed (-19) 00000100:00100000:3.0:1776178348.912557:0:3352:0:(client.c:2215:ptlrpc_check_set()) Completed RPC req@0000000075189f40 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:lustre-MDT0001-mdtlov_UUID:3352:1862456998069568:0@lo:38:kworker/u8:1.0 00000100:00080000:3.0:1776178348.912563:0:3352:0:(import.c:86:import_set_state_nolock()) 00000000a7c55b1f lustre-MDT0000_UUID: changing import state from CONNECTING to DISCONN 00000100:00080000:3.0:1776178348.912883:0:3352:0:(import.c:1429:ptlrpc_connect_interpret()) recovery of lustre-MDT0000_UUID on 192.168.204.103@tcp failed (-19) 00000100:00100000:3.0:1776178348.912887:0:3352:0:(client.c:2215:ptlrpc_check_set()) Completed RPC req@0000000037aace32 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:lustre-MDT0000-lwp-MDT0001_UUID:3352:1862456998069760:0@lo:38:kworker/u8:1.0 00000100:00100000:2.0:1776178348.913228:0:3353:0:(client.c:1733:ptlrpc_send_new_req()) Sending RPC req@000000007216887e pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_00:lustre-MDT0001-mdtlov_UUID:3353:1862456998069696:0@lo:400:kworker/u8:1.0 00000100:00100000:2.0:1776178348.913267:0:3353:0:(events.c:361:request_in_callback()) peer: 12345-0@lo (source: 12345-0@lo) 00000100:00100000:2.0:1776178348.913309:0:3353:0:(client.c:2215:ptlrpc_check_set()) Completed RPC req@000000004828c19d pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_00:ac61f839-0e8e-4cac-9c59-08bcfbbad330:3353:1862456998069248:0@lo:400:kworker/u8:1.0 00000100:00100000:3.0:1776178348.913325:0:20762:0:(service.c:2152:ptlrpc_server_handle_req_in()) got req x1862456998069696 00000100:00100000:3.0:1776178348.913368:0:20762:0:(nrs_fifo.c:179:nrs_fifo_req_get()) NRS start fifo request from 12345-0@lo, seq: 626 00000100:00100000:3.0:1776178348.913372:0:20762:0:(service.c:2324:ptlrpc_server_handle_request()) Handling RPC req@0000000065dbeee0 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_013:lustre-MDT0001-mdtlov_UUID+5:3353:x1862456998069696:12345-0@lo:400:kworker/u8:1.0 00000100:00100000:3.0:1776178348.913423:0:20762:0:(service.c:2372:ptlrpc_server_handle_request()) Handled RPC req@0000000065dbeee0 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_013:lustre-MDT0001-mdtlov_UUID+5:3353:x1862456998069696:12345-0@lo:400:kworker/u8:1.0 Request processed in 50us (155us total) trans 0 rc 0/0 00000100:00100000:3.0:1776178348.915748:0:20762:0:(nrs_fifo.c:241:nrs_fifo_req_stop()) NRS stop fifo request from 12345-0@lo, seq: 626 00000100:00100000:0.0:1776178348.916160:0:3353:0:(client.c:2215:ptlrpc_check_set()) Completed RPC req@000000007216887e pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_00:lustre-MDT0001-mdtlov_UUID:3353:1862456998069696:0@lo:400:kworker/u8:1.0 00010000:00010000:2.0:1776178349.539012:0:6245:0:(ldlm_lockd.c:2792:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:0.0:1776178350.433254:0:3354:0:(client.c:1733:ptlrpc_send_new_req()) Sending RPC req@0000000015f283a6 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_01:lustre-MDT0001-mdtlov_UUID:3354:1862456998069824:0@lo:13:osp-pre-1-1.0 00000100:00100000:0.0:1776178350.433394:0:3354:0:(events.c:361:request_in_callback()) peer: 12345-0@lo (source: 12345-0@lo) 00000100:00100000:3.0:1776178350.433777:0:8231:0:(service.c:2152:ptlrpc_server_handle_req_in()) got req x1862456998069824 00000100:00100000:3.0:1776178350.433856:0:8231:0:(nrs_fifo.c:179:nrs_fifo_req_get()) NRS start fifo request from 12345-0@lo, seq: 545 00000100:00100000:3.0:1776178350.433864:0:8231:0:(service.c:2324:ptlrpc_server_handle_request()) Handling RPC req@00000000c3b47349 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_create00:lustre-MDT0001-mdtlov_UUID+5:3354:x1862456998069824:12345-0@lo:13:osp-pre-1-1.0 00002000:00000024:3.0:1776178350.433905:0:8231:0:(ofd_obd.c:740:ofd_statfs()) blocks cached 229376 granted 491746368 pending 0 free 3923689472 avail 3692167168 00000100:00100000:3.0:1776178350.433980:0:8231:0:(service.c:2372:ptlrpc_server_handle_request()) Handled RPC req@00000000c3b47349 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_create00:lustre-MDT0001-mdtlov_UUID+5:3354:x1862456998069824:12345-0@lo:13:osp-pre-1-1.0 Request processed in 115us (588us total) trans 0 rc 0/0 00000100:00100000:3.0:1776178350.433988:0:8231:0:(nrs_fifo.c:241:nrs_fifo_req_stop()) NRS stop fifo request from 12345-0@lo, seq: 545 00000100:00100000:2.0:1776178350.434103:0:3354:0:(client.c:2215:ptlrpc_check_set()) Completed RPC req@0000000015f283a6 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_01:lustre-MDT0001-mdtlov_UUID:3354:1862456998069824:0@lo:13:osp-pre-1-1.0 00010000:00010000:2.0:1776178350.562714:0:9224:0:(ldlm_lockd.c:2792:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:3.0:1776178350.944275:0:3355:0:(client.c:1733:ptlrpc_send_new_req()) Sending RPC req@000000006c57be21 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_02:lustre-MDT0001-mdtlov_UUID:3355:1862456998069888:0@lo:13:osp-pre-0-1.0 00000100:00100000:3.0:1776178350.944378:0:3355:0:(events.c:361:request_in_callback()) peer: 12345-0@lo (source: 12345-0@lo) 00000100:00100000:1.0:1776178350.944828:0:9554:0:(service.c:2152:ptlrpc_server_handle_req_in()) got req x1862456998069888 00000100:00100000:1.0:1776178350.945009:0:9554:0:(nrs_fifo.c:179:nrs_fifo_req_get()) NRS start fifo request from 12345-0@lo, seq: 546 00000100:00100000:1.0:1776178350.945016:0:9554:0:(service.c:2324:ptlrpc_server_handle_request()) Handling RPC req@0000000037aace32 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_create00:lustre-MDT0001-mdtlov_UUID+5:3355:x1862456998069888:12345-0@lo:13:osp-pre-0-1.0 00002000:00000024:1.0:1776178350.945048:0:9554:0:(ofd_obd.c:740:ofd_statfs()) blocks cached 315392 granted 484883776 pending 0 free 3923689472 avail 3692167168 00000100:00100000:1.0:1776178350.945128:0:9554:0:(service.c:2372:ptlrpc_server_handle_request()) Handled RPC req@0000000037aace32 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_create00:lustre-MDT0001-mdtlov_UUID+5:3355:x1862456998069888:12345-0@lo:13:osp-pre-0-1.0 Request processed in 112us (751us total) trans 0 rc 0/0 00000100:00100000:1.0:1776178350.945135:0:9554:0:(nrs_fifo.c:241:nrs_fifo_req_stop()) NRS stop fifo request from 12345-0@lo, seq: 546 00000100:00100000:0.0:1776178350.945210:0:3355:0:(client.c:2215:ptlrpc_check_set()) Completed RPC req@000000006c57be21 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_02:lustre-MDT0001-mdtlov_UUID:3355:1862456998069888:0@lo:13:osp-pre-0-1.0 00010000:00010000:2.0:1776178351.584898:0:9225:0:(ldlm_lockd.c:2792:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:0.0:1776178352.419943:0:3347:0:(events.c:361:request_in_callback()) peer: 12345-192.168.204.3@tcp (source: 12345-192.168.204.3@tcp) 00000100:00100000:0.0:1776178352.419996:0:3347:0:(events.c:361:request_in_callback()) peer: 12345-192.168.204.3@tcp (source: 12345-192.168.204.3@tcp) 00000100:00100000:0.0:1776178352.420039:0:3347:0:(events.c:361:request_in_callback()) peer: 12345-192.168.204.3@tcp (source: 12345-192.168.204.3@tcp) 00000100:00100000:0.0:1776178352.420129:0:3347:0:(events.c:361:request_in_callback()) peer: 12345-192.168.204.3@tcp (source: 12345-192.168.204.3@tcp) 00000100:00100000:3.0:1776178352.420251:0:6256:0:(service.c:2152:ptlrpc_server_handle_req_in()) got req x1862456979118400 00000100:00100000:3.0:1776178352.420313:0:6256:0:(nrs_fifo.c:179:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.204.3@tcp, seq: 1015 00000100:00100000:3.0:1776178352.420320:0:6256:0:(service.c:2324:ptlrpc_server_handle_request()) Handling RPC req@000000008b5ee68d pname:cluuid+ref:pid:xid:nid:opc:job mdt00_001:b50963bd-8eba-4432-9a54-8888d00e3fa1+6:2271:x1862456979118400:12345-192.168.204.3@tcp:400:kworker/u8:3.0 00000100:00100000:3.0:1776178352.420401:0:6256:0:(service.c:2372:ptlrpc_server_handle_request()) Handled RPC req@000000008b5ee68d pname:cluuid+ref:pid:xid:nid:opc:job mdt00_001:b50963bd-8eba-4432-9a54-8888d00e3fa1+6:2271:x1862456979118400:12345-192.168.204.3@tcp:400:kworker/u8:3.0 Request processed in 80us (362us total) trans 0 rc 0/0 00000100:00100000:3.0:1776178352.420409:0:6256:0:(nrs_fifo.c:241:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.204.3@tcp, seq: 1015 00000100:00100000:3.0:1776178352.420435:0:20762:0:(service.c:2152:ptlrpc_server_handle_req_in()) got req x1862456979118464 00000100:00100000:3.0:1776178352.420461:0:20762:0:(service.c:2152:ptlrpc_server_handle_req_in()) got req x1862456979118528 00000100:00100000:3.0:1776178352.420478:0:20762:0:(nrs_fifo.c:179:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.204.3@tcp, seq: 627 00000100:00100000:3.0:1776178352.420482:0:20762:0:(service.c:2324:ptlrpc_server_handle_request()) Handling RPC req@000000006c57be21 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_013:b50963bd-8eba-4432-9a54-8888d00e3fa1+8:2271:x1862456979118464:12345-192.168.204.3@tcp:400:kworker/u8:3.0 00000100:00100000:3.0:1776178352.420507:0:20762:0:(service.c:2372:ptlrpc_server_handle_request()) Handled RPC req@000000006c57be21 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_013:b50963bd-8eba-4432-9a54-8888d00e3fa1+8:2271:x1862456979118464:12345-192.168.204.3@tcp:400:kworker/u8:3.0 Request processed in 24us (512us total) trans 0 rc 0/0 00000100:00100000:3.0:1776178352.420513:0:20762:0:(nrs_fifo.c:241:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.204.3@tcp, seq: 627 00000100:00100000:3.0:1776178352.420523:0:20762:0:(nrs_fifo.c:179:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.204.3@tcp, seq: 628 00000100:00100000:3.0:1776178352.420527:0:20762:0:(service.c:2324:ptlrpc_server_handle_request()) Handling RPC req@00000000a9455d7d pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_013:b50963bd-8eba-4432-9a54-8888d00e3fa1+9:2271:x1862456979118528:12345-192.168.204.3@tcp:400:kworker/u8:3.0 00000100:00100000:3.0:1776178352.420548:0:20762:0:(service.c:2372:ptlrpc_server_handle_request()) Handled RPC req@00000000a9455d7d pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_013:b50963bd-8eba-4432-9a54-8888d00e3fa1+9:2271:x1862456979118528:12345-192.168.204.3@tcp:400:kworker/u8:3.0 Request processed in 20us (419us total) trans 0 rc 0/0 00000100:00100000:3.0:1776178352.420553:0:20762:0:(nrs_fifo.c:241:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.204.3@tcp, seq: 628 00000100:00100000:0.0:1776178352.421051:0:3346:0:(events.c:361:request_in_callback()) peer: 12345-192.168.204.3@tcp (source: 12345-192.168.204.3@tcp) 00000100:00100000:3.0:1776178352.421135:0:6256:0:(service.c:2152:ptlrpc_server_handle_req_in()) got req x1862456979118336 00000100:00100000:3.0:1776178352.421158:0:6256:0:(nrs_fifo.c:179:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.204.3@tcp, seq: 6740 00000100:00100000:3.0:1776178352.421164:0:6256:0:(service.c:2324:ptlrpc_server_handle_request()) Handling RPC req@000000001d072e30 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_001:0+-99:2269:x1862456979118336:12345-192.168.204.3@tcp:38:kworker/u8:3.0 00010000:02000000:3.0:1776178352.421186:0:6256:0:(ldlm_lib.c:1139:target_handle_connect()) lustre-MDT0000: Not available for connect from 192.168.204.3@tcp (stopping) 00000100:00100000:3.0:1776178352.421314:0:6256:0:(service.c:2372:ptlrpc_server_handle_request()) Handled RPC req@000000001d072e30 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_001:0+-99:2269:x1862456979118336:12345-192.168.204.3@tcp:38:kworker/u8:3.0 Request processed in 148us (264us total) trans 0 rc -19/-19 00000100:00100000:3.0:1776178352.421324:0:6256:0:(nrs_fifo.c:241:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.204.3@tcp, seq: 6740 00000100:00100000:1.0:1776178352.421401:0:20140:0:(service.c:2152:ptlrpc_server_handle_req_in()) got req x1862456979118272 00000100:00100000:1.0:1776178352.421441:0:20140:0:(nrs_fifo.c:179:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.204.3@tcp, seq: 213 00000100:00100000:1.0:1776178352.421446:0:20140:0:(service.c:2324:ptlrpc_server_handle_request()) Handling RPC req@000000003ef1d595 pname:cluuid+ref:pid:xid:nid:opc:job ll_mgs_0002:5b5c66b1-1a2f-4651-8548-0c75b9a38a94+9:2271:x1862456979118272:12345-192.168.204.3@tcp:400:kworker/u8:3.0 00000100:00100000:1.0:1776178352.421476:0:20140:0:(service.c:2372:ptlrpc_server_handle_request()) Handled RPC req@000000003ef1d595 pname:cluuid+ref:pid:xid:nid:opc:job ll_mgs_0002:5b5c66b1-1a2f-4651-8548-0c75b9a38a94+9:2271:x1862456979118272:12345-192.168.204.3@tcp:400:kworker/u8:3.0 Request processed in 30us (1542us total) trans 0 rc 0/0 00000100:00100000:1.0:1776178352.421482:0:20140:0:(nrs_fifo.c:241:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.204.3@tcp, seq: 213 00010000:00010000:2.0:1776178352.610009:0:6244:0:(ldlm_lockd.c:2792:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00010000:00010000:2.0:1776178353.632550:0:6245:0:(ldlm_lockd.c:2792:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00080000:1.0:1776178354.025853:0:36:0:(pinger.c:251:ptlrpc_pinger_process_import()) lustre-MDT0000-lwp-OST0000_UUID->lustre-MDT0000_UUID: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00080000:1.0:1776178354.025861:0:36:0:(recover.c:57:ptlrpc_initiate_recovery()) lustre-MDT0000_UUID: starting recovery 00000100:00080000:1.0:1776178354.025865:0:36:0:(import.c:86:import_set_state_nolock()) 0000000047601a98 lustre-MDT0000_UUID: changing import state from DISCONN to CONNECTING 00000100:00080000:1.0:1776178354.025875:0:36:0:(import.c:534:import_select_connection()) lustre-MDT0000-lwp-OST0000: connect to NID 0@lo last attempt 1056 00000100:00080000:1.0:1776178354.025879:0:36:0:(import.c:581:import_select_connection()) lustre-MDT0000-lwp-OST0000: tried all connections, increasing latency to 21s 00000100:00080000:1.0:1776178354.025885:0:36:0:(import.c:615:import_select_connection()) lustre-MDT0000-lwp-OST0000: import 0000000047601a98 using connection 192.168.204.103@tcp/0@lo 00000100:00100000:3.0:1776178354.025889:0:3355:0:(ptlrpcd.c:415:ptlrpcd_check()) transfer 1 async RPCs [3->2] 00000100:00100000:1.0:1776178354.025906:0:36:0:(import.c:807:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5) req@00000000a604eea4 x1862456998070016/t0(0) o38->lustre-MDT0000-lwp-OST0000@0@lo:12/10 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' 00000100:00100000:3.0:1776178354.025908:0:3355:0:(client.c:1733:ptlrpc_send_new_req()) Sending RPC req@0000000007880806 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_02:ac61f839-0e8e-4cac-9c59-08bcfbbad330:3355:1862456998069952:0@lo:400:kworker/u8:1.0 00000100:00080000:1.0:1776178354.025937:0:36:0:(pinger.c:251:ptlrpc_pinger_process_import()) lustre-MDT0000-lwp-OST0001_UUID->lustre-MDT0000_UUID: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00080000:1.0:1776178354.025940:0:36:0:(recover.c:57:ptlrpc_initiate_recovery()) lustre-MDT0000_UUID: starting recovery 00000100:00080000:1.0:1776178354.025941:0:36:0:(import.c:86:import_set_state_nolock()) 00000000dfc8db13 lustre-MDT0000_UUID: changing import state from DISCONN to CONNECTING 00000100:00080000:1.0:1776178354.025944:0:36:0:(import.c:534:import_select_connection()) lustre-MDT0000-lwp-OST0001: connect to NID 0@lo last attempt 1056 00000100:00080000:1.0:1776178354.025946:0:36:0:(import.c:581:import_select_connection()) lustre-MDT0000-lwp-OST0001: tried all connections, increasing latency to 21s 00000100:00080000:1.0:1776178354.025948:0:36:0:(import.c:615:import_select_connection()) lustre-MDT0000-lwp-OST0001: import 00000000dfc8db13 using connection 192.168.204.103@tcp/0@lo 00000100:00100000:1.0:1776178354.025957:0:36:0:(import.c:807:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5) req@0000000046115d91 x1862456998070144/t0(0) o38->lustre-MDT0000-lwp-OST0001@0@lo:12/10 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' 00000100:00080000:1.0:1776178354.025969:0:36:0:(pinger.c:251:ptlrpc_pinger_process_import()) lustre-MDT0001-mdtlov_UUID->lustre-MDT0000_UUID: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00080000:1.0:1776178354.025972:0:36:0:(recover.c:57:ptlrpc_initiate_recovery()) lustre-MDT0000_UUID: starting recovery 00000100:00080000:1.0:1776178354.025974:0:36:0:(import.c:86:import_set_state_nolock()) 00000000ba4f1821 lustre-MDT0000_UUID: changing import state from DISCONN to CONNECTING 00000100:00080000:1.0:1776178354.025976:0:36:0:(import.c:534:import_select_connection()) lustre-MDT0000-osp-MDT0001: connect to NID 0@lo last attempt 1056 00000100:00080000:1.0:1776178354.025978:0:36:0:(import.c:581:import_select_connection()) lustre-MDT0000-osp-MDT0001: tried all connections, increasing latency to 21s 00000100:00080000:1.0:1776178354.025980:0:36:0:(import.c:615:import_select_connection()) lustre-MDT0000-osp-MDT0001: import 00000000ba4f1821 using connection 192.168.204.103@tcp/0@lo 00000100:00100000:3.0:1776178354.025983:0:3355:0:(events.c:361:request_in_callback()) peer: 12345-0@lo (source: 12345-0@lo) 00000100:00100000:1.0:1776178354.025990:0:36:0:(import.c:807:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5) req@000000007cbab86b x1862456998070272/t0(0) o38->lustre-MDT0000-osp-MDT0001@0@lo:24/4 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' 00000100:00080000:1.0:1776178354.026006:0:36:0:(pinger.c:251:ptlrpc_pinger_process_import()) lustre-MDT0000-lwp-MDT0001_UUID->lustre-MDT0000_UUID: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00100000:3.0:1776178354.026008:0:3355:0:(client.c:1733:ptlrpc_send_new_req()) Sending RPC req@00000000e0d60cbc pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_02:lustre-MDT0001-mdtlov_UUID:3355:1862456998070336:0@lo:400:kworker/u8:1.0 00000100:00080000:1.0:1776178354.026010:0:36:0:(recover.c:57:ptlrpc_initiate_recovery()) lustre-MDT0000_UUID: starting recovery 00000100:00080000:1.0:1776178354.026011:0:36:0:(import.c:86:import_set_state_nolock()) 00000000a7c55b1f lustre-MDT0000_UUID: changing import state from DISCONN to CONNECTING 00000100:00080000:1.0:1776178354.026014:0:36:0:(import.c:534:import_select_connection()) lustre-MDT0000-lwp-MDT0001: connect to NID 0@lo last attempt 1056 00000100:00080000:1.0:1776178354.026016:0:36:0:(import.c:581:import_select_connection()) lustre-MDT0000-lwp-MDT0001: tried all connections, increasing latency to 21s 00000100:00080000:1.0:1776178354.026018:0:36:0:(import.c:615:import_select_connection()) lustre-MDT0000-lwp-MDT0001: import 00000000a7c55b1f using connection 192.168.204.103@tcp/0@lo 00000100:00100000:1.0:1776178354.026029:0:36:0:(import.c:807:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5) req@0000000069ebab60 x1862456998070464/t0(0) o38->lustre-MDT0000-lwp-MDT0001@0@lo:12/10 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' 00000100:00100000:3.0:1776178354.026043:0:3355:0:(events.c:361:request_in_callback()) peer: 12345-0@lo (source: 12345-0@lo) 00000100:00100000:1.0:1776178354.026055:0:3353:0:(client.c:1733:ptlrpc_send_new_req()) Sending RPC req@0000000097413742 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_00:lustre-MDT0001-lwp-OST0000_UUID:3353:1862456998070080:0@lo:400:kworker/u8:1.0 00000100:00100000:3.0:1776178354.026084:0:3355:0:(ptlrpcd.c:415:ptlrpcd_check()) transfer 1 async RPCs [3->2] 00000100:00100000:3.0:1776178354.026089:0:3355:0:(client.c:1733:ptlrpc_send_new_req()) Sending RPC req@000000002a321d27 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_02:lustre-MDT0001-mdtlov_UUID:3355:1862456998070400:0@lo:400:kworker/u8:1.0 00000100:00100000:3.0:1776178354.026110:0:3355:0:(events.c:361:request_in_callback()) peer: 12345-0@lo (source: 12345-0@lo) 00000100:00100000:1.0:1776178354.026116:0:3353:0:(events.c:361:request_in_callback()) peer: 12345-0@lo (source: 12345-0@lo) 00000100:00100000:1.0:1776178354.026133:0:3353:0:(ptlrpcd.c:415:ptlrpcd_check()) transfer 1 async RPCs [1->0] 00000100:00100000:1.0:1776178354.026137:0:3353:0:(client.c:1733:ptlrpc_send_new_req()) Sending RPC req@000000001879c2bc pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_00:lustre-MDT0001-lwp-OST0001_UUID:3353:1862456998070208:0@lo:400:kworker/u8:1.0 00000100:00100000:1.0:1776178354.026159:0:3353:0:(events.c:361:request_in_callback()) peer: 12345-0@lo (source: 12345-0@lo) 00000100:00100000:3.0:1776178354.026162:0:20759:0:(service.c:2152:ptlrpc_server_handle_req_in()) got req x1862456998070336 00000100:00100000:1.0:1776178354.026193:0:11281:0:(service.c:2152:ptlrpc_server_handle_req_in()) got req x1862456998070080 00000100:00100000:3.0:1776178354.026206:0:20759:0:(service.c:2152:ptlrpc_server_handle_req_in()) got req x1862456998070400 00000100:00100000:1.0:1776178354.026216:0:11281:0:(service.c:2152:ptlrpc_server_handle_req_in()) got req x1862456998070208 00000100:00100000:3.0:1776178354.026223:0:20759:0:(nrs_fifo.c:179:nrs_fifo_req_get()) NRS start fifo request from 12345-0@lo, seq: 629 00000100:00100000:3.0:1776178354.026228:0:20759:0:(service.c:2324:ptlrpc_server_handle_request()) Handling RPC req@0000000075189f40 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_010:lustre-MDT0001-mdtlov_UUID+5:3355:x1862456998070336:12345-0@lo:400:kworker/u8:1.0 00000100:00100000:1.0:1776178354.026231:0:11281:0:(nrs_fifo.c:179:nrs_fifo_req_get()) NRS start fifo request from 12345-0@lo, seq: 1016 00000100:00100000:1.0:1776178354.026234:0:11281:0:(service.c:2324:ptlrpc_server_handle_request()) Handling RPC req@00000000efb13deb pname:cluuid+ref:pid:xid:nid:opc:job mdt00_004:lustre-MDT0001-lwp-OST0000_UUID+5:3353:x1862456998070080:12345-0@lo:400:kworker/u8:1.0 00000100:00100000:2.0:1776178354.026358:0:20137:0:(service.c:2152:ptlrpc_server_handle_req_in()) got req x1862456998069952 00000100:00100000:2.0:1776178354.026387:0:20137:0:(nrs_fifo.c:179:nrs_fifo_req_get()) NRS start fifo request from 12345-0@lo, seq: 214 00000100:00100000:2.0:1776178354.026391:0:20137:0:(service.c:2324:ptlrpc_server_handle_request()) Handling RPC req@00000000b4b6b848 pname:cluuid+ref:pid:xid:nid:opc:job ll_mgs_0000:ac61f839-0e8e-4cac-9c59-08bcfbbad330+18:3355:x1862456998069952:12345-0@lo:400:kworker/u8:1.0 00000100:00100000:0.0:1776178354.026561:0:20762:0:(nrs_fifo.c:179:nrs_fifo_req_get()) NRS start fifo request from 12345-0@lo, seq: 630 00000100:00100000:0.0:1776178354.026566:0:20762:0:(service.c:2324:ptlrpc_server_handle_request()) Handling RPC req@000000006a3d91b0 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_013:lustre-MDT0001-mdtlov_UUID+5:3355:x1862456998070400:12345-0@lo:400:kworker/u8:1.0 00000100:00100000:1.0:1776178354.027007:0:11281:0:(service.c:2372:ptlrpc_server_handle_request()) Handled RPC req@00000000efb13deb pname:cluuid+ref:pid:xid:nid:opc:job mdt00_004:lustre-MDT0001-lwp-OST0000_UUID+5:3353:x1862456998070080:12345-0@lo:400:kworker/u8:1.0 Request processed in 770us (889us total) trans 0 rc 0/0 00000100:00100000:1.0:1776178354.027017:0:11281:0:(nrs_fifo.c:241:nrs_fifo_req_stop()) NRS stop fifo request from 12345-0@lo, seq: 1016 00000100:00100000:1.0:1776178354.027036:0:11281:0:(nrs_fifo.c:179:nrs_fifo_req_get()) NRS start fifo request from 12345-0@lo, seq: 1017 00000100:00100000:1.0:1776178354.027039:0:11281:0:(service.c:2324:ptlrpc_server_handle_request()) Handling RPC req@0000000027e376cf pname:cluuid+ref:pid:xid:nid:opc:job mdt00_004:lustre-MDT0001-lwp-OST0001_UUID+5:3353:x1862456998070208:12345-0@lo:400:kworker/u8:1.0 00000100:00100000:1.0:1776178354.027088:0:11281:0:(service.c:2372:ptlrpc_server_handle_request()) Handled RPC req@0000000027e376cf pname:cluuid+ref:pid:xid:nid:opc:job mdt00_004:lustre-MDT0001-lwp-OST0001_UUID+5:3353:x1862456998070208:12345-0@lo:400:kworker/u8:1.0 Request processed in 48us (929us total) trans 0 rc 0/0 00000100:00100000:1.0:1776178354.027093:0:11281:0:(nrs_fifo.c:241:nrs_fifo_req_stop()) NRS stop fifo request from 12345-0@lo, seq: 1017 00000100:00100000:1.0:1776178354.027130:0:3353:0:(client.c:2215:ptlrpc_check_set()) Completed RPC req@000000001879c2bc pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_00:lustre-MDT0001-lwp-OST0001_UUID:3353:1862456998070208:0@lo:400:kworker/u8:1.0 00000100:00100000:1.0:1776178354.027141:0:3353:0:(client.c:2215:ptlrpc_check_set()) Completed RPC req@0000000097413742 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_00:lustre-MDT0001-lwp-OST0000_UUID:3353:1862456998070080:0@lo:400:kworker/u8:1.0 00000100:00100000:1.0:1776178354.027193:0:3352:0:(client.c:1733:ptlrpc_send_new_req()) Sending RPC req@00000000a604eea4 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:lustre-MDT0000-lwp-OST0000_UUID:3352:1862456998070016:0@lo:38:kworker/u8:1.0 00000100:00100000:1.0:1776178354.027220:0:3352:0:(events.c:361:request_in_callback()) peer: 12345-0@lo (source: 12345-0@lo) 00000100:00100000:1.0:1776178354.027238:0:6256:0:(service.c:2152:ptlrpc_server_handle_req_in()) got req x1862456998070016 00000100:00100000:1.0:1776178354.027256:0:6256:0:(nrs_fifo.c:179:nrs_fifo_req_get()) NRS start fifo request from 12345-0@lo, seq: 6741 00000100:00100000:1.0:1776178354.027259:0:6256:0:(service.c:2324:ptlrpc_server_handle_request()) Handling RPC req@0000000097413742 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_001:0+-99:3352:x1862456998070016:12345-0@lo:38:kworker/u8:1.0 00010000:02000000:1.0:1776178354.027272:0:6256:0:(ldlm_lib.c:1139:target_handle_connect()) lustre-MDT0000: Not available for connect from 0@lo (stopping) 00000100:00100000:1.0:1776178354.027352:0:6256:0:(service.c:2372:ptlrpc_server_handle_request()) Handled RPC req@0000000097413742 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_001:0+-99:3352:x1862456998070016:12345-0@lo:38:kworker/u8:1.0 Request processed in 91us (131us total) trans 0 rc -19/-19 00000100:00100000:1.0:1776178354.027358:0:6256:0:(nrs_fifo.c:241:nrs_fifo_req_stop()) NRS stop fifo request from 12345-0@lo, seq: 6741 00000100:00100000:1.0:1776178354.027377:0:3352:0:(client.c:1733:ptlrpc_send_new_req()) Sending RPC req@0000000046115d91 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:lustre-MDT0000-lwp-OST0001_UUID:3352:1862456998070144:0@lo:38:kworker/u8:1.0 00000100:00100000:1.0:1776178354.027537:0:3352:0:(events.c:361:request_in_callback()) peer: 12345-0@lo (source: 12345-0@lo) 00000100:00100000:1.0:1776178354.027559:0:8250:0:(service.c:2152:ptlrpc_server_handle_req_in()) got req x1862456998070144 00000100:00100000:1.0:1776178354.027577:0:8250:0:(nrs_fifo.c:179:nrs_fifo_req_get()) NRS start fifo request from 12345-0@lo, seq: 6742 00000100:00100000:1.0:1776178354.027581:0:8250:0:(service.c:2324:ptlrpc_server_handle_request()) Handling RPC req@000000001879c2bc pname:cluuid+ref:pid:xid:nid:opc:job mdt00_003:0+-99:3352:x1862456998070144:12345-0@lo:38:kworker/u8:1.0 00010000:02000000:1.0:1776178354.027587:0:8250:0:(ldlm_lib.c:1139:target_handle_connect()) lustre-MDT0000: Not available for connect from 0@lo (stopping) 00000100:00100000:1.0:1776178354.027609:0:8250:0:(service.c:2372:ptlrpc_server_handle_request()) Handled RPC req@000000001879c2bc pname:cluuid+ref:pid:xid:nid:opc:job mdt00_003:0+-99:3352:x1862456998070144:12345-0@lo:38:kworker/u8:1.0 Request processed in 27us (73us total) trans 0 rc -19/-19 00000100:00100000:1.0:1776178354.027613:0:8250:0:(nrs_fifo.c:241:nrs_fifo_req_stop()) NRS stop fifo request from 12345-0@lo, seq: 6742 00000100:00100000:1.0:1776178354.027628:0:3352:0:(client.c:1733:ptlrpc_send_new_req()) Sending RPC req@000000007cbab86b pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:lustre-MDT0001-mdtlov_UUID:3352:1862456998070272:0@lo:38:kworker/u8:1.0 00000100:00100000:1.0:1776178354.027653:0:3352:0:(events.c:361:request_in_callback()) peer: 12345-0@lo (source: 12345-0@lo) 00000100:00100000:1.0:1776178354.027825:0:3352:0:(client.c:1733:ptlrpc_send_new_req()) Sending RPC req@0000000069ebab60 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:lustre-MDT0000-lwp-MDT0001_UUID:3352:1862456998070464:0@lo:38:kworker/u8:1.0 00000100:00100000:1.0:1776178354.027860:0:3352:0:(events.c:361:request_in_callback()) peer: 12345-0@lo (source: 12345-0@lo) 00000100:00100000:1.0:1776178354.027882:0:6256:0:(service.c:2152:ptlrpc_server_handle_req_in()) got req x1862456998070464 00000100:00100000:1.0:1776178354.027898:0:6256:0:(nrs_fifo.c:179:nrs_fifo_req_get()) NRS start fifo request from 12345-0@lo, seq: 6743 00000100:00100000:1.0:1776178354.027901:0:6256:0:(service.c:2324:ptlrpc_server_handle_request()) Handling RPC req@0000000021d5d4e8 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_001:0+-99:3352:x1862456998070464:12345-0@lo:38:kworker/u8:1.0 00010000:02000000:1.0:1776178354.027908:0:6256:0:(ldlm_lib.c:1139:target_handle_connect()) lustre-MDT0000: Not available for connect from 0@lo (stopping) 00000100:00100000:1.0:1776178354.027928:0:6256:0:(service.c:2372:ptlrpc_server_handle_request()) Handled RPC req@0000000021d5d4e8 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_001:0+-99:3352:x1862456998070464:12345-0@lo:38:kworker/u8:1.0 Request processed in 26us (69us total) trans 0 rc -19/-19 00000100:00100000:1.0:1776178354.027934:0:6256:0:(nrs_fifo.c:241:nrs_fifo_req_stop()) NRS stop fifo request from 12345-0@lo, seq: 6743 00000100:00080000:1.0:1776178354.027958:0:3352:0:(import.c:86:import_set_state_nolock()) 0000000047601a98 lustre-MDT0000_UUID: changing import state from CONNECTING to DISCONN 00000100:00080000:1.0:1776178354.027961:0:3352:0:(import.c:1429:ptlrpc_connect_interpret()) recovery of lustre-MDT0000_UUID on 192.168.204.103@tcp failed (-19) 00000100:00100000:1.0:1776178354.027965:0:3352:0:(client.c:2215:ptlrpc_check_set()) Completed RPC req@00000000a604eea4 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:lustre-MDT0000-lwp-OST0000_UUID:3352:1862456998070016:0@lo:38:kworker/u8:1.0 00000100:00080000:1.0:1776178354.027973:0:3352:0:(import.c:86:import_set_state_nolock()) 00000000dfc8db13 lustre-MDT0000_UUID: changing import state from CONNECTING to DISCONN 00000100:00080000:1.0:1776178354.027975:0:3352:0:(import.c:1429:ptlrpc_connect_interpret()) recovery of lustre-MDT0000_UUID on 192.168.204.103@tcp failed (-19) 00000100:00100000:1.0:1776178354.027977:0:3352:0:(client.c:2215:ptlrpc_check_set()) Completed RPC req@0000000046115d91 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:lustre-MDT0000-lwp-OST0001_UUID:3352:1862456998070144:0@lo:38:kworker/u8:1.0 00000100:00080000:1.0:1776178354.027984:0:3352:0:(import.c:86:import_set_state_nolock()) 00000000a7c55b1f lustre-MDT0000_UUID: changing import state from CONNECTING to DISCONN 00000100:00080000:1.0:1776178354.027986:0:3352:0:(import.c:1429:ptlrpc_connect_interpret()) recovery of lustre-MDT0000_UUID on 192.168.204.103@tcp failed (-19) 00000100:00100000:1.0:1776178354.027988:0:3352:0:(client.c:2215:ptlrpc_check_set()) Completed RPC req@0000000069ebab60 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:lustre-MDT0000-lwp-MDT0001_UUID:3352:1862456998070464:0@lo:38:kworker/u8:1.0 00000100:00100000:1.0:1776178354.028057:0:3355:0:(client.c:2215:ptlrpc_check_set()) Completed RPC req@00000000e0d60cbc pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_02:lustre-MDT0001-mdtlov_UUID:3355:1862456998070336:0@lo:400:kworker/u8:1.0 00000100:00100000:1.0:1776178354.028106:0:15213:0:(service.c:2152:ptlrpc_server_handle_req_in()) got req x1862456998070272 00000100:00100000:1.0:1776178354.028126:0:15213:0:(nrs_fifo.c:179:nrs_fifo_req_get()) NRS start fifo request from 12345-0@lo, seq: 737 00000100:00100000:1.0:1776178354.028129:0:15213:0:(service.c:2324:ptlrpc_server_handle_request()) Handling RPC req@00000000d4b41d38 pname:cluuid+ref:pid:xid:nid:opc:job mdt_out00_003:0+-99:3352:x1862456998070272:12345-0@lo:38:kworker/u8:1.0 00010000:02000000:1.0:1776178354.028136:0:15213:0:(ldlm_lib.c:1139:target_handle_connect()) lustre-MDT0000: Not available for connect from 0@lo (stopping) 00000100:00100000:1.0:1776178354.028162:0:15213:0:(service.c:2372:ptlrpc_server_handle_request()) Handled RPC req@00000000d4b41d38 pname:cluuid+ref:pid:xid:nid:opc:job mdt_out00_003:0+-99:3352:x1862456998070272:12345-0@lo:38:kworker/u8:1.0 Request processed in 32us (509us total) trans 0 rc -19/-19 00000100:00100000:1.0:1776178354.028168:0:15213:0:(nrs_fifo.c:241:nrs_fifo_req_stop()) NRS stop fifo request from 12345-0@lo, seq: 737 00000100:00080000:1.0:1776178354.028187:0:3352:0:(import.c:86:import_set_state_nolock()) 00000000ba4f1821 lustre-MDT0000_UUID: changing import state from CONNECTING to DISCONN 00000100:00080000:1.0:1776178354.028190:0:3352:0:(import.c:1429:ptlrpc_connect_interpret()) recovery of lustre-MDT0000_UUID on 192.168.204.103@tcp failed (-19) 00000100:00100000:1.0:1776178354.028192:0:3352:0:(client.c:2215:ptlrpc_check_set()) Completed RPC req@000000007cbab86b pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:lustre-MDT0001-mdtlov_UUID:3352:1862456998070272:0@lo:38:kworker/u8:1.0 00000100:00100000:3.0:1776178354.030829:0:20759:0:(service.c:2372:ptlrpc_server_handle_request()) Handled RPC req@0000000075189f40 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_010:lustre-MDT0001-mdtlov_UUID+5:3355:x1862456998070336:12345-0@lo:400:kworker/u8:1.0 Request processed in 4592us (4777us total) trans 0 rc 0/0 00000100:00100000:3.0:1776178354.030856:0:20759:0:(nrs_fifo.c:241:nrs_fifo_req_stop()) NRS stop fifo request from 12345-0@lo, seq: 629 00000100:00100000:2.0:1776178354.045420:0:20137:0:(service.c:2372:ptlrpc_server_handle_request()) Handled RPC req@00000000b4b6b848 pname:cluuid+ref:pid:xid:nid:opc:job ll_mgs_0000:ac61f839-0e8e-4cac-9c59-08bcfbbad330+18:3355:x1862456998069952:12345-0@lo:400:kworker/u8:1.0 Request processed in 19018us (19428us total) trans 0 rc 0/0 00000100:00100000:2.0:1776178354.045436:0:20137:0:(nrs_fifo.c:241:nrs_fifo_req_stop()) NRS stop fifo request from 12345-0@lo, seq: 214 00000100:00100000:1.0:1776178354.045734:0:3355:0:(client.c:2215:ptlrpc_check_set()) Completed RPC req@0000000007880806 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_02:ac61f839-0e8e-4cac-9c59-08bcfbbad330:3355:1862456998069952:0@lo:400:kworker/u8:1.0 00000100:00100000:0.0:1776178354.046361:0:20762:0:(service.c:2372:ptlrpc_server_handle_request()) Handled RPC req@000000006a3d91b0 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_013:lustre-MDT0001-mdtlov_UUID+5:3355:x1862456998070400:12345-0@lo:400:kworker/u8:1.0 Request processed in 19790us (20246us total) trans 0 rc 0/0 00000100:00100000:0.0:1776178354.046373:0:20762:0:(nrs_fifo.c:241:nrs_fifo_req_stop()) NRS stop fifo request from 12345-0@lo, seq: 630 00000100:00100000:3.0:1776178354.046915:0:3355:0:(client.c:2215:ptlrpc_check_set()) Completed RPC req@000000002a321d27 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_02:lustre-MDT0001-mdtlov_UUID:3355:1862456998070400:0@lo:400:kworker/u8:1.0 00000020:02000400:3.0:1776178354.531588:0:22200:0:(genops.c:1786:obd_exports_barrier()) lustre-MDT0000 is waiting for obd_unlinked_exports more than 64 seconds. The obd refcount = 4. Is it stuck? 00000020:02000400:3.0:1776178354.539431:0:22200:0:(genops.c:1752:print_export_data()) lustre-MDT0000: UNLINKED 00000000ca22e26f b50963bd-8eba-4432-9a54-8888d00e3fa1 1 (0 0 0) 1 0 1 0: 0000000000000000 8589934704 stale:0 00010000:00010000:2.0:1776178354.657816:0:9224:0:(ldlm_lockd.c:2792:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:3.0:1776178355.552349:0:3354:0:(ptlrpcd.c:415:ptlrpcd_check()) transfer 1 async RPCs [0->1] 00000100:00100000:3.0:1776178355.552375:0:3354:0:(client.c:1733:ptlrpc_send_new_req()) Sending RPC req@0000000015f283a6 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_01:lustre-MDT0001-mdtlov_UUID:3354:1862456998070528:0@lo:13:osp-pre-1-1.0 00000100:00100000:3.0:1776178355.552449:0:3354:0:(events.c:361:request_in_callback()) peer: 12345-0@lo (source: 12345-0@lo) 00000100:00100000:1.0:1776178355.552595:0:9554:0:(service.c:2152:ptlrpc_server_handle_req_in()) got req x1862456998070528 00000100:00100000:1.0:1776178355.552654:0:9554:0:(nrs_fifo.c:179:nrs_fifo_req_get()) NRS start fifo request from 12345-0@lo, seq: 547 00000100:00100000:1.0:1776178355.552661:0:9554:0:(service.c:2324:ptlrpc_server_handle_request()) Handling RPC req@0000000042de09e9 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_create00:lustre-MDT0001-mdtlov_UUID+5:3354:x1862456998070528:12345-0@lo:13:osp-pre-1-1.0 00002000:00000024:1.0:1776178355.552690:0:9554:0:(ofd_obd.c:740:ofd_statfs()) blocks cached 229376 granted 491746368 pending 0 free 3923689472 avail 3692167168 00000100:00100000:1.0:1776178355.556080:0:9554:0:(service.c:2372:ptlrpc_server_handle_request()) Handled RPC req@0000000042de09e9 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_create00:lustre-MDT0001-mdtlov_UUID+5:3354:x1862456998070528:12345-0@lo:13:osp-pre-1-1.0 Request processed in 3393us (3607us total) trans 0 rc 0/0 00000100:00100000:1.0:1776178355.556099:0:9554:0:(nrs_fifo.c:241:nrs_fifo_req_stop()) NRS stop fifo request from 12345-0@lo, seq: 547 00000100:00100000:3.0:1776178355.558937:0:3354:0:(client.c:2215:ptlrpc_check_set()) Completed RPC req@0000000015f283a6 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_01:lustre-MDT0001-mdtlov_UUID:3354:1862456998070528:0@lo:13:osp-pre-1-1.0 00010000:00010000:2.0:1776178355.680336:0:9225:0:(ldlm_lockd.c:2792:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:1.0:1776178356.068974:0:3353:0:(ptlrpcd.c:415:ptlrpcd_check()) transfer 1 async RPCs [1->0] 00000100:00100000:1.0:1776178356.069012:0:3353:0:(client.c:1733:ptlrpc_send_new_req()) Sending RPC req@0000000030978cff pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_00:lustre-MDT0001-mdtlov_UUID:3353:1862456998070592:0@lo:13:osp-pre-0-1.0 00000100:00100000:1.0:1776178356.069123:0:3353:0:(events.c:361:request_in_callback()) peer: 12345-0@lo (source: 12345-0@lo) 00000100:00100000:0.0:1776178356.071022:0:9554:0:(service.c:2152:ptlrpc_server_handle_req_in()) got req x1862456998070592 00000100:00100000:0.0:1776178356.071118:0:9554:0:(nrs_fifo.c:179:nrs_fifo_req_get()) NRS start fifo request from 12345-0@lo, seq: 548 00000100:00100000:0.0:1776178356.071128:0:9554:0:(service.c:2324:ptlrpc_server_handle_request()) Handling RPC req@00000000699e66b5 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_create00:lustre-MDT0001-mdtlov_UUID+5:3353:x1862456998070592:12345-0@lo:13:osp-pre-0-1.0 00002000:00000024:0.0:1776178356.071163:0:9554:0:(ofd_obd.c:740:ofd_statfs()) blocks cached 315392 granted 484883776 pending 0 free 3923689472 avail 3692167168 00000100:00100000:0.0:1776178356.071257:0:9554:0:(service.c:2372:ptlrpc_server_handle_request()) Handled RPC req@00000000699e66b5 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_create00:lustre-MDT0001-mdtlov_UUID+5:3353:x1862456998070592:12345-0@lo:13:osp-pre-0-1.0 Request processed in 129us (2136us total) trans 0 rc 0/0 00000100:00100000:0.0:1776178356.071266:0:9554:0:(nrs_fifo.c:241:nrs_fifo_req_stop()) NRS stop fifo request from 12345-0@lo, seq: 548 00000100:00100000:1.0:1776178356.071494:0:3353:0:(client.c:2215:ptlrpc_check_set()) Completed RPC req@0000000030978cff pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_00:lustre-MDT0001-mdtlov_UUID:3353:1862456998070592:0@lo:13:osp-pre-0-1.0 00010000:00010000:2.0:1776178356.704501:0:6244:0:(ldlm_lockd.c:2792:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:3.0:1776178357.539984:0:3348:0:(events.c:361:request_in_callback()) peer: 12345-192.168.204.3@tcp (source: 12345-192.168.204.3@tcp) 00000100:00100000:3.0:1776178357.540037:0:3348:0:(events.c:361:request_in_callback()) peer: 12345-192.168.204.3@tcp (source: 12345-192.168.204.3@tcp) 00000100:00100000:2.0:1776178357.540120:0:20137:0:(service.c:2152:ptlrpc_server_handle_req_in()) got req x1862456979118592 00000100:00100000:2.0:1776178357.540180:0:20137:0:(nrs_fifo.c:179:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.204.3@tcp, seq: 215 00000100:00100000:2.0:1776178357.540186:0:20137:0:(service.c:2324:ptlrpc_server_handle_request()) Handling RPC req@000000001333a9de pname:cluuid+ref:pid:xid:nid:opc:job ll_mgs_0000:5b5c66b1-1a2f-4651-8548-0c75b9a38a94+9:2272:x1862456979118592:12345-192.168.204.3@tcp:400:kworker/u8:3.0 00000100:00100000:2.0:1776178357.540269:0:20137:0:(service.c:2372:ptlrpc_server_handle_request()) Handled RPC req@000000001333a9de pname:cluuid+ref:pid:xid:nid:opc:job ll_mgs_0000:5b5c66b1-1a2f-4651-8548-0c75b9a38a94+9:2272:x1862456979118592:12345-192.168.204.3@tcp:400:kworker/u8:3.0 Request processed in 82us (310us total) trans 0 rc 0/0 00000100:00100000:2.0:1776178357.540276:0:20137:0:(nrs_fifo.c:241:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.204.3@tcp, seq: 215 00000100:00100000:0.0:1776178357.540284:0:20762:0:(service.c:2152:ptlrpc_server_handle_req_in()) got req x1862456979118784 00000100:00100000:0.0:1776178357.540330:0:20762:0:(nrs_fifo.c:179:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.204.3@tcp, seq: 631 00000100:00100000:0.0:1776178357.540337:0:20762:0:(service.c:2324:ptlrpc_server_handle_request()) Handling RPC req@000000007216887e pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_013:b50963bd-8eba-4432-9a54-8888d00e3fa1+8:2272:x1862456979118784:12345-192.168.204.3@tcp:400:kworker/u8:3.0 00000100:00100000:0.0:1776178357.540377:0:20762:0:(service.c:2372:ptlrpc_server_handle_request()) Handled RPC req@000000007216887e pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_013:b50963bd-8eba-4432-9a54-8888d00e3fa1+8:2272:x1862456979118784:12345-192.168.204.3@tcp:400:kworker/u8:3.0 Request processed in 41us (340us total) trans 0 rc 0/0 00000100:00100000:0.0:1776178357.540386:0:20762:0:(nrs_fifo.c:241:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.204.3@tcp, seq: 631 00000100:00100000:3.0:1776178357.543612:0:3348:0:(events.c:361:request_in_callback()) peer: 12345-192.168.204.3@tcp (source: 12345-192.168.204.3@tcp) 00000100:00100000:3.0:1776178357.543782:0:3348:0:(events.c:361:request_in_callback()) peer: 12345-192.168.204.3@tcp (source: 12345-192.168.204.3@tcp) 00000100:00100000:3.0:1776178357.543822:0:3348:0:(events.c:361:request_in_callback()) peer: 12345-192.168.204.3@tcp (source: 12345-192.168.204.3@tcp) 00000100:00100000:2.0:1776178357.543895:0:20759:0:(service.c:2152:ptlrpc_server_handle_req_in()) got req x1862456979118848 00000100:00100000:2.0:1776178357.544398:0:20759:0:(nrs_fifo.c:179:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.204.3@tcp, seq: 632 00000100:00100000:2.0:1776178357.544409:0:20759:0:(service.c:2324:ptlrpc_server_handle_request()) Handling RPC req@00000000e8195050 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_010:b50963bd-8eba-4432-9a54-8888d00e3fa1+9:2272:x1862456979118848:12345-192.168.204.3@tcp:400:kworker/u8:3.0 00000100:00100000:2.0:1776178357.544519:0:20759:0:(service.c:2372:ptlrpc_server_handle_request()) Handled RPC req@00000000e8195050 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_010:b50963bd-8eba-4432-9a54-8888d00e3fa1+9:2272:x1862456979118848:12345-192.168.204.3@tcp:400:kworker/u8:3.0 Request processed in 112us (747us total) trans 0 rc 0/0 00000100:00100000:2.0:1776178357.544528:0:20759:0:(nrs_fifo.c:241:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.204.3@tcp, seq: 632 00000100:00100000:1.0:1776178357.558116:0:6255:0:(service.c:2152:ptlrpc_server_handle_req_in()) got req x1862456979118720 00000100:00100000:0.0:1776178357.558392:0:6255:0:(service.c:2152:ptlrpc_server_handle_req_in()) got req x1862456979118656 00000100:00100000:0.0:1776178357.558437:0:6255:0:(nrs_fifo.c:179:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.204.3@tcp, seq: 1018 00000100:00100000:0.0:1776178357.558445:0:6255:0:(service.c:2324:ptlrpc_server_handle_request()) Handling RPC req@000000004828c19d pname:cluuid+ref:pid:xid:nid:opc:job mdt00_000:b50963bd-8eba-4432-9a54-8888d00e3fa1+6:2272:x1862456979118720:12345-192.168.204.3@tcp:400:kworker/u8:3.0 00000100:00100000:0.0:1776178357.558562:0:6255:0:(service.c:2372:ptlrpc_server_handle_request()) Handled RPC req@000000004828c19d pname:cluuid+ref:pid:xid:nid:opc:job mdt00_000:b50963bd-8eba-4432-9a54-8888d00e3fa1+6:2272:x1862456979118720:12345-192.168.204.3@tcp:400:kworker/u8:3.0 Request processed in 117us (14960us total) trans 0 rc 0/0 00000100:00100000:1.0:1776178357.558565:0:6256:0:(nrs_fifo.c:179:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.204.3@tcp, seq: 6744 00000100:00100000:1.0:1776178357.558579:0:6256:0:(service.c:2324:ptlrpc_server_handle_request()) Handling RPC req@00000000250bc347 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_001:0+-99:2269:x1862456979118656:12345-192.168.204.3@tcp:38:kworker/u8:3.0 00000100:00100000:0.0:1776178357.558584:0:6255:0:(nrs_fifo.c:241:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.204.3@tcp, seq: 1018 00010000:02000000:1.0:1776178357.558604:0:6256:0:(ldlm_lib.c:1139:target_handle_connect()) lustre-MDT0000: Not available for connect from 192.168.204.3@tcp (stopping) 00000100:00100000:1.0:1776178357.558859:0:6256:0:(service.c:2372:ptlrpc_server_handle_request()) Handled RPC req@00000000250bc347 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_001:0+-99:2269:x1862456979118656:12345-192.168.204.3@tcp:38:kworker/u8:3.0 Request processed in 279us (15036us total) trans 0 rc -19/-19 00000100:00100000:1.0:1776178357.558867:0:6256:0:(nrs_fifo.c:241:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.204.3@tcp, seq: 6744 00010000:00010000:2.0:1776178357.728593:0:6245:0:(ldlm_lockd.c:2792:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00010000:00010000:2.0:1776178358.754761:0:9224:0:(ldlm_lockd.c:2792:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00080000:1.0:1776178359.136476:0:37:0:(pinger.c:251:ptlrpc_pinger_process_import()) lustre-MDT0000-lwp-OST0000_UUID->lustre-MDT0000_UUID: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00080000:1.0:1776178359.136491:0:37:0:(recover.c:57:ptlrpc_initiate_recovery()) lustre-MDT0000_UUID: starting recovery 00000100:00080000:1.0:1776178359.136498:0:37:0:(import.c:86:import_set_state_nolock()) 0000000047601a98 lustre-MDT0000_UUID: changing import state from DISCONN to CONNECTING 00000100:00080000:1.0:1776178359.136511:0:37:0:(import.c:534:import_select_connection()) lustre-MDT0000-lwp-OST0000: connect to NID 0@lo last attempt 1061 00000100:00080000:1.0:1776178359.136516:0:37:0:(import.c:581:import_select_connection()) lustre-MDT0000-lwp-OST0000: tried all connections, increasing latency to 21s 00000100:00080000:1.0:1776178359.136526:0:37:0:(import.c:615:import_select_connection()) lustre-MDT0000-lwp-OST0000: import 0000000047601a98 using connection 192.168.204.103@tcp/0@lo 00000100:00100000:1.0:1776178359.136562:0:37:0:(import.c:807:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5) req@0000000022d4dc29 x1862456998070720/t0(0) o38->lustre-MDT0000-lwp-OST0000@0@lo:12/10 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' 00000100:00080000:1.0:1776178359.136610:0:37:0:(pinger.c:251:ptlrpc_pinger_process_import()) lustre-MDT0000-lwp-OST0001_UUID->lustre-MDT0000_UUID: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00080000:1.0:1776178359.136613:0:37:0:(recover.c:57:ptlrpc_initiate_recovery()) lustre-MDT0000_UUID: starting recovery 00000100:00080000:1.0:1776178359.136615:0:37:0:(import.c:86:import_set_state_nolock()) 00000000dfc8db13 lustre-MDT0000_UUID: changing import state from DISCONN to CONNECTING 00000100:00080000:1.0:1776178359.136619:0:37:0:(import.c:534:import_select_connection()) lustre-MDT0000-lwp-OST0001: connect to NID 0@lo last attempt 1061 00000100:00080000:1.0:1776178359.136621:0:37:0:(import.c:581:import_select_connection()) lustre-MDT0000-lwp-OST0001: tried all connections, increasing latency to 21s 00000100:00080000:1.0:1776178359.136625:0:37:0:(import.c:615:import_select_connection()) lustre-MDT0000-lwp-OST0001: import 00000000dfc8db13 using connection 192.168.204.103@tcp/0@lo 00000100:00100000:1.0:1776178359.136637:0:37:0:(import.c:807:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5) req@00000000ed90ff75 x1862456998070848/t0(0) o38->lustre-MDT0000-lwp-OST0001@0@lo:12/10 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' 00000100:00080000:1.0:1776178359.136669:0:37:0:(pinger.c:251:ptlrpc_pinger_process_import()) lustre-MDT0001-mdtlov_UUID->lustre-MDT0000_UUID: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00080000:1.0:1776178359.136672:0:37:0:(recover.c:57:ptlrpc_initiate_recovery()) lustre-MDT0000_UUID: starting recovery 00000100:00080000:1.0:1776178359.136674:0:37:0:(import.c:86:import_set_state_nolock()) 00000000ba4f1821 lustre-MDT0000_UUID: changing import state from DISCONN to CONNECTING 00000100:00080000:1.0:1776178359.136678:0:37:0:(import.c:534:import_select_connection()) lustre-MDT0000-osp-MDT0001: connect to NID 0@lo last attempt 1061 00000100:00080000:1.0:1776178359.136680:0:37:0:(import.c:581:import_select_connection()) lustre-MDT0000-osp-MDT0001: tried all connections, increasing latency to 21s 00000100:00080000:1.0:1776178359.136684:0:37:0:(import.c:615:import_select_connection()) lustre-MDT0000-osp-MDT0001: import 00000000ba4f1821 using connection 192.168.204.103@tcp/0@lo 00000100:00100000:1.0:1776178359.137129:0:37:0:(import.c:807:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5) req@000000002a321d27 x1862456998070976/t0(0) o38->lustre-MDT0000-osp-MDT0001@0@lo:24/4 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' 00000100:00080000:1.0:1776178359.137155:0:37:0:(pinger.c:251:ptlrpc_pinger_process_import()) lustre-MDT0000-lwp-MDT0001_UUID->lustre-MDT0000_UUID: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00080000:1.0:1776178359.137159:0:37:0:(recover.c:57:ptlrpc_initiate_recovery()) lustre-MDT0000_UUID: starting recovery 00000100:00080000:1.0:1776178359.137161:0:37:0:(import.c:86:import_set_state_nolock()) 00000000a7c55b1f lustre-MDT0000_UUID: changing import state from DISCONN to CONNECTING 00000100:00080000:1.0:1776178359.137164:0:37:0:(import.c:534:import_select_connection()) lustre-MDT0000-lwp-MDT0001: connect to NID 0@lo last attempt 1061 00000100:00080000:1.0:1776178359.137166:0:37:0:(import.c:581:import_select_connection()) lustre-MDT0000-lwp-MDT0001: tried all connections, increasing latency to 21s 00000100:00080000:1.0:1776178359.137170:0:37:0:(import.c:615:import_select_connection()) lustre-MDT0000-lwp-MDT0001: import 00000000a7c55b1f using connection 192.168.204.103@tcp/0@lo 00000100:00100000:1.0:1776178359.137181:0:37:0:(import.c:807:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5) req@00000000e0d60cbc x1862456998071168/t0(0) o38->lustre-MDT0000-lwp-MDT0001@0@lo:12/10 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' 00000100:00100000:1.0:1776178359.137230:0:3353:0:(client.c:1733:ptlrpc_send_new_req()) Sending RPC req@0000000039ff8335 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_00:lustre-MDT0001-lwp-OST0001_UUID:3353:1862456998070912:0@lo:400:kworker/u8:2.0 00000100:00100000:1.0:1776178359.137329:0:3353:0:(events.c:361:request_in_callback()) peer: 12345-0@lo (source: 12345-0@lo) 00000100:00100000:1.0:1776178359.137468:0:3353:0:(ptlrpcd.c:415:ptlrpcd_check()) transfer 1 async RPCs [1->0] 00000100:00100000:1.0:1776178359.137476:0:3353:0:(client.c:1733:ptlrpc_send_new_req()) Sending RPC req@0000000007880806 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_00:lustre-MDT0001-mdtlov_UUID:3353:1862456998071040:0@lo:400:kworker/u8:2.0 00000100:00100000:1.0:1776178359.137511:0:3353:0:(events.c:361:request_in_callback()) peer: 12345-0@lo (source: 12345-0@lo) 00000100:00100000:3.0:1776178359.137634:0:3355:0:(client.c:1733:ptlrpc_send_new_req()) Sending RPC req@00000000a6227918 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_02:ac61f839-0e8e-4cac-9c59-08bcfbbad330:3355:1862456998070656:0@lo:400:kworker/u8:2.0 00000100:00100000:3.0:1776178359.137679:0:3355:0:(events.c:361:request_in_callback()) peer: 12345-0@lo (source: 12345-0@lo) 00000100:00100000:3.0:1776178359.137713:0:3355:0:(client.c:1733:ptlrpc_send_new_req()) Sending RPC req@000000007cbab86b pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_02:lustre-MDT0001-mdtlov_UUID:3355:1862456998071104:0@lo:400:kworker/u8:2.0 00000100:00100000:3.0:1776178359.137738:0:3355:0:(events.c:361:request_in_callback()) peer: 12345-0@lo (source: 12345-0@lo) 00000100:00100000:3.0:1776178359.137772:0:3355:0:(ptlrpcd.c:415:ptlrpcd_check()) transfer 1 async RPCs [3->2] 00000100:00100000:3.0:1776178359.137778:0:3355:0:(client.c:1733:ptlrpc_send_new_req()) Sending RPC req@0000000049b1b339 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_02:lustre-MDT0001-lwp-OST0000_UUID:3355:1862456998070784:0@lo:400:kworker/u8:2.0 00000100:00100000:3.0:1776178359.137802:0:3355:0:(events.c:361:request_in_callback()) peer: 12345-0@lo (source: 12345-0@lo) 00000100:00100000:3.0:1776178359.137857:0:11281:0:(service.c:2152:ptlrpc_server_handle_req_in()) got req x1862456998070912 00000100:00100000:3.0:1776178359.137916:0:11281:0:(service.c:2152:ptlrpc_server_handle_req_in()) got req x1862456998070784 00000100:00100000:3.0:1776178359.137938:0:11281:0:(nrs_fifo.c:179:nrs_fifo_req_get()) NRS start fifo request from 12345-0@lo, seq: 1019 00000100:00100000:3.0:1776178359.137947:0:11281:0:(service.c:2324:ptlrpc_server_handle_request()) Handling RPC req@0000000069ebab60 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_004:lustre-MDT0001-lwp-OST0001_UUID+5:3353:x1862456998070912:12345-0@lo:400:kworker/u8:2.0 00000100:00100000:3.0:1776178359.138015:0:11281:0:(service.c:2372:ptlrpc_server_handle_request()) Handled RPC req@0000000069ebab60 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_004:lustre-MDT0001-lwp-OST0001_UUID+5:3353:x1862456998070912:12345-0@lo:400:kworker/u8:2.0 Request processed in 68us (689us total) trans 0 rc 0/0 00000100:00100000:3.0:1776178359.138024:0:11281:0:(nrs_fifo.c:241:nrs_fifo_req_stop()) NRS stop fifo request from 12345-0@lo, seq: 1019 00000100:00100000:3.0:1776178359.138042:0:11281:0:(nrs_fifo.c:179:nrs_fifo_req_get()) NRS start fifo request from 12345-0@lo, seq: 1020 00000100:00100000:3.0:1776178359.138047:0:11281:0:(service.c:2324:ptlrpc_server_handle_request()) Handling RPC req@000000001e11faca pname:cluuid+ref:pid:xid:nid:opc:job mdt00_004:lustre-MDT0001-lwp-OST0000_UUID+5:3355:x1862456998070784:12345-0@lo:400:kworker/u8:2.0 00000100:00100000:3.0:1776178359.138107:0:11281:0:(service.c:2372:ptlrpc_server_handle_request()) Handled RPC req@000000001e11faca pname:cluuid+ref:pid:xid:nid:opc:job mdt00_004:lustre-MDT0001-lwp-OST0000_UUID+5:3355:x1862456998070784:12345-0@lo:400:kworker/u8:2.0 Request processed in 60us (305us total) trans 0 rc 0/0 00000100:00100000:3.0:1776178359.138117:0:11281:0:(nrs_fifo.c:241:nrs_fifo_req_stop()) NRS stop fifo request from 12345-0@lo, seq: 1020 00000100:00100000:3.0:1776178359.138139:0:20762:0:(service.c:2152:ptlrpc_server_handle_req_in()) got req x1862456998071040 00000100:00100000:3.0:1776178359.138161:0:20762:0:(service.c:2152:ptlrpc_server_handle_req_in()) got req x1862456998071104 00000100:00100000:3.0:1776178359.138178:0:20762:0:(nrs_fifo.c:179:nrs_fifo_req_get()) NRS start fifo request from 12345-0@lo, seq: 633 00000100:00100000:3.0:1776178359.138181:0:20762:0:(service.c:2324:ptlrpc_server_handle_request()) Handling RPC req@0000000046115d91 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_013:lustre-MDT0001-mdtlov_UUID+5:3353:x1862456998071040:12345-0@lo:400:kworker/u8:2.0 00000100:00100000:3.0:1776178359.138212:0:20762:0:(service.c:2372:ptlrpc_server_handle_request()) Handled RPC req@0000000046115d91 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_013:lustre-MDT0001-mdtlov_UUID+5:3353:x1862456998071040:12345-0@lo:400:kworker/u8:2.0 Request processed in 29us (701us total) trans 0 rc 0/0 00000100:00100000:3.0:1776178359.138217:0:20762:0:(nrs_fifo.c:241:nrs_fifo_req_stop()) NRS stop fifo request from 12345-0@lo, seq: 633 00000100:00100000:3.0:1776178359.138228:0:20762:0:(nrs_fifo.c:179:nrs_fifo_req_get()) NRS start fifo request from 12345-0@lo, seq: 634 00000100:00100000:3.0:1776178359.138231:0:20762:0:(service.c:2324:ptlrpc_server_handle_request()) Handling RPC req@00000000103d2c47 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_013:lustre-MDT0001-mdtlov_UUID+5:3355:x1862456998071104:12345-0@lo:400:kworker/u8:2.0 00000100:00100000:3.0:1776178359.138255:0:20762:0:(service.c:2372:ptlrpc_server_handle_request()) Handled RPC req@00000000103d2c47 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_013:lustre-MDT0001-mdtlov_UUID+5:3355:x1862456998071104:12345-0@lo:400:kworker/u8:2.0 Request processed in 23us (517us total) trans 0 rc 0/0 00000100:00100000:3.0:1776178359.138260:0:20762:0:(nrs_fifo.c:241:nrs_fifo_req_stop()) NRS stop fifo request from 12345-0@lo, seq: 634 00000100:00100000:3.0:1776178359.138308:0:3355:0:(client.c:2215:ptlrpc_check_set()) Completed RPC req@0000000049b1b339 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_02:lustre-MDT0001-lwp-OST0000_UUID:3355:1862456998070784:0@lo:400:kworker/u8:2.0 00000100:00100000:3.0:1776178359.138318:0:3355:0:(client.c:2215:ptlrpc_check_set()) Completed RPC req@000000007cbab86b pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_02:lustre-MDT0001-mdtlov_UUID:3355:1862456998071104:0@lo:400:kworker/u8:2.0 00000100:00100000:3.0:1776178359.138353:0:3353:0:(client.c:2215:ptlrpc_check_set()) Completed RPC req@0000000007880806 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_00:lustre-MDT0001-mdtlov_UUID:3353:1862456998071040:0@lo:400:kworker/u8:2.0 00000100:00100000:3.0:1776178359.138364:0:3353:0:(client.c:2215:ptlrpc_check_set()) Completed RPC req@0000000039ff8335 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_00:lustre-MDT0001-lwp-OST0001_UUID:3353:1862456998070912:0@lo:400:kworker/u8:2.0 00000100:00100000:0.0:1776178359.139652:0:3352:0:(client.c:1733:ptlrpc_send_new_req()) Sending RPC req@0000000022d4dc29 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:lustre-MDT0000-lwp-OST0000_UUID:3352:1862456998070720:0@lo:38:kworker/u8:2.0 00000100:00100000:0.0:1776178359.139734:0:3352:0:(events.c:361:request_in_callback()) peer: 12345-0@lo (source: 12345-0@lo) 00000100:00100000:0.0:1776178359.139774:0:3352:0:(client.c:1733:ptlrpc_send_new_req()) Sending RPC req@00000000ed90ff75 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:lustre-MDT0000-lwp-OST0001_UUID:3352:1862456998070848:0@lo:38:kworker/u8:2.0 00000100:00100000:0.0:1776178359.139801:0:3352:0:(events.c:361:request_in_callback()) peer: 12345-0@lo (source: 12345-0@lo) 00000100:00100000:0.0:1776178359.139820:0:3352:0:(client.c:1733:ptlrpc_send_new_req()) Sending RPC req@000000002a321d27 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:lustre-MDT0001-mdtlov_UUID:3352:1862456998070976:0@lo:38:kworker/u8:2.0 00000100:00100000:0.0:1776178359.139845:0:3352:0:(events.c:361:request_in_callback()) peer: 12345-0@lo (source: 12345-0@lo) 00000100:00100000:0.0:1776178359.139861:0:3352:0:(client.c:1733:ptlrpc_send_new_req()) Sending RPC req@00000000e0d60cbc pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:lustre-MDT0000-lwp-MDT0001_UUID:3352:1862456998071168:0@lo:38:kworker/u8:2.0 00000100:00100000:0.0:1776178359.139883:0:3352:0:(events.c:361:request_in_callback()) peer: 12345-0@lo (source: 12345-0@lo) 00000100:00100000:0.0:1776178359.139929:0:14128:0:(service.c:2152:ptlrpc_server_handle_req_in()) got req x1862456998070976 00000100:00100000:0.0:1776178359.139964:0:14128:0:(nrs_fifo.c:179:nrs_fifo_req_get()) NRS start fifo request from 12345-0@lo, seq: 738 00000100:00100000:0.0:1776178359.139969:0:14128:0:(service.c:2324:ptlrpc_server_handle_request()) Handling RPC req@000000009497b17e pname:cluuid+ref:pid:xid:nid:opc:job mdt_out00_002:0+-99:3352:x1862456998070976:12345-0@lo:38:kworker/u8:2.0 00010000:02000000:0.0:1776178359.139989:0:14128:0:(ldlm_lib.c:1139:target_handle_connect()) lustre-MDT0000: Not available for connect from 0@lo (stopping) 00000100:00100000:0.0:1776178359.140033:0:14128:0:(service.c:2372:ptlrpc_server_handle_request()) Handled RPC req@000000009497b17e pname:cluuid+ref:pid:xid:nid:opc:job mdt_out00_002:0+-99:3352:x1862456998070976:12345-0@lo:38:kworker/u8:2.0 Request processed in 63us (188us total) trans 0 rc -19/-19 00000100:00100000:0.0:1776178359.140039:0:14128:0:(nrs_fifo.c:241:nrs_fifo_req_stop()) NRS stop fifo request from 12345-0@lo, seq: 738 00000100:00100000:0.0:1776178359.140097:0:8250:0:(service.c:2152:ptlrpc_server_handle_req_in()) got req x1862456998070720 00000100:00100000:0.0:1776178359.140116:0:8250:0:(service.c:2152:ptlrpc_server_handle_req_in()) got req x1862456998070848 00000100:00100000:0.0:1776178359.140123:0:8250:0:(service.c:2152:ptlrpc_server_handle_req_in()) got req x1862456998071168 00000100:00100000:0.0:1776178359.140132:0:8250:0:(nrs_fifo.c:179:nrs_fifo_req_get()) NRS start fifo request from 12345-0@lo, seq: 6745 00000100:00100000:0.0:1776178359.140136:0:8250:0:(service.c:2324:ptlrpc_server_handle_request()) Handling RPC req@0000000084604fbc pname:cluuid+ref:pid:xid:nid:opc:job mdt00_003:0+-99:3352:x1862456998070720:12345-0@lo:38:kworker/u8:2.0 00010000:02000000:0.0:1776178359.140144:0:8250:0:(ldlm_lib.c:1139:target_handle_connect()) lustre-MDT0000: Not available for connect from 0@lo (stopping) 00000100:00100000:0.0:1776178359.140168:0:8250:0:(service.c:2372:ptlrpc_server_handle_request()) Handled RPC req@0000000084604fbc pname:cluuid+ref:pid:xid:nid:opc:job mdt00_003:0+-99:3352:x1862456998070720:12345-0@lo:38:kworker/u8:2.0 Request processed in 31us (437us total) trans 0 rc -19/-19 00000100:00100000:0.0:1776178359.140174:0:8250:0:(nrs_fifo.c:241:nrs_fifo_req_stop()) NRS stop fifo request from 12345-0@lo, seq: 6745 00000100:00100000:0.0:1776178359.140185:0:8250:0:(nrs_fifo.c:179:nrs_fifo_req_get()) NRS start fifo request from 12345-0@lo, seq: 6746 00000100:00100000:0.0:1776178359.140188:0:8250:0:(service.c:2324:ptlrpc_server_handle_request()) Handling RPC req@000000002e8f0791 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_003:0+-99:3352:x1862456998070848:12345-0@lo:38:kworker/u8:2.0 00010000:02000000:0.0:1776178359.140194:0:8250:0:(ldlm_lib.c:1139:target_handle_connect()) lustre-MDT0000: Not available for connect from 0@lo (stopping) 00000100:00100000:0.0:1776178359.140214:0:8250:0:(service.c:2372:ptlrpc_server_handle_request()) Handled RPC req@000000002e8f0791 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_003:0+-99:3352:x1862456998070848:12345-0@lo:38:kworker/u8:2.0 Request processed in 24us (413us total) trans 0 rc -19/-19 00000100:00100000:0.0:1776178359.140219:0:8250:0:(nrs_fifo.c:241:nrs_fifo_req_stop()) NRS stop fifo request from 12345-0@lo, seq: 6746 00000100:00100000:0.0:1776178359.140228:0:8250:0:(nrs_fifo.c:179:nrs_fifo_req_get()) NRS start fifo request from 12345-0@lo, seq: 6747 00000100:00100000:0.0:1776178359.140231:0:8250:0:(service.c:2324:ptlrpc_server_handle_request()) Handling RPC req@0000000030978cff pname:cluuid+ref:pid:xid:nid:opc:job mdt00_003:0+-99:3352:x1862456998071168:12345-0@lo:38:kworker/u8:2.0 00010000:02000000:0.0:1776178359.140237:0:8250:0:(ldlm_lib.c:1139:target_handle_connect()) lustre-MDT0000: Not available for connect from 0@lo (stopping) 00000100:00100000:0.0:1776178359.140255:0:8250:0:(service.c:2372:ptlrpc_server_handle_request()) Handled RPC req@0000000030978cff pname:cluuid+ref:pid:xid:nid:opc:job mdt00_003:0+-99:3352:x1862456998071168:12345-0@lo:38:kworker/u8:2.0 Request processed in 23us (372us total) trans 0 rc -19/-19 00000100:00100000:0.0:1776178359.140261:0:8250:0:(nrs_fifo.c:241:nrs_fifo_req_stop()) NRS stop fifo request from 12345-0@lo, seq: 6747 00000100:00080000:0.0:1776178359.140299:0:3352:0:(import.c:86:import_set_state_nolock()) 0000000047601a98 lustre-MDT0000_UUID: changing import state from CONNECTING to DISCONN 00000100:00080000:0.0:1776178359.140303:0:3352:0:(import.c:1429:ptlrpc_connect_interpret()) recovery of lustre-MDT0000_UUID on 192.168.204.103@tcp failed (-19) 00000100:00100000:0.0:1776178359.140308:0:3352:0:(client.c:2215:ptlrpc_check_set()) Completed RPC req@0000000022d4dc29 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:lustre-MDT0000-lwp-OST0000_UUID:3352:1862456998070720:0@lo:38:kworker/u8:2.0 00000100:00080000:0.0:1776178359.140316:0:3352:0:(import.c:86:import_set_state_nolock()) 00000000dfc8db13 lustre-MDT0000_UUID: changing import state from CONNECTING to DISCONN 00000100:00080000:0.0:1776178359.140319:0:3352:0:(import.c:1429:ptlrpc_connect_interpret()) recovery of lustre-MDT0000_UUID on 192.168.204.103@tcp failed (-19) 00000100:00100000:0.0:1776178359.140322:0:3352:0:(client.c:2215:ptlrpc_check_set()) Completed RPC req@00000000ed90ff75 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:lustre-MDT0000-lwp-OST0001_UUID:3352:1862456998070848:0@lo:38:kworker/u8:2.0 00000100:00080000:0.0:1776178359.140328:0:3352:0:(import.c:86:import_set_state_nolock()) 00000000ba4f1821 lustre-MDT0000_UUID: changing import state from CONNECTING to DISCONN 00000100:00080000:0.0:1776178359.140330:0:3352:0:(import.c:1429:ptlrpc_connect_interpret()) recovery of lustre-MDT0000_UUID on 192.168.204.103@tcp failed (-19) 00000100:00100000:0.0:1776178359.140332:0:3352:0:(client.c:2215:ptlrpc_check_set()) Completed RPC req@000000002a321d27 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:lustre-MDT0001-mdtlov_UUID:3352:1862456998070976:0@lo:38:kworker/u8:2.0 00000100:00080000:0.0:1776178359.140338:0:3352:0:(import.c:86:import_set_state_nolock()) 00000000a7c55b1f lustre-MDT0000_UUID: changing import state from CONNECTING to DISCONN 00000100:00080000:0.0:1776178359.140340:0:3352:0:(import.c:1429:ptlrpc_connect_interpret()) recovery of lustre-MDT0000_UUID on 192.168.204.103@tcp failed (-19) 00000100:00100000:0.0:1776178359.140342:0:3352:0:(client.c:2215:ptlrpc_check_set()) Completed RPC req@00000000e0d60cbc pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:lustre-MDT0000-lwp-MDT0001_UUID:3352:1862456998071168:0@lo:38:kworker/u8:2.0 00000100:00100000:1.0:1776178359.140741:0:20140:0:(service.c:2152:ptlrpc_server_handle_req_in()) got req x1862456998070656 00000100:00100000:1.0:1776178359.140795:0:20140:0:(nrs_fifo.c:179:nrs_fifo_req_get()) NRS start fifo request from 12345-0@lo, seq: 216 00000100:00100000:1.0:1776178359.140800:0:20140:0:(service.c:2324:ptlrpc_server_handle_request()) Handling RPC req@00000000b4857168 pname:cluuid+ref:pid:xid:nid:opc:job ll_mgs_0002:ac61f839-0e8e-4cac-9c59-08bcfbbad330+18:3355:x1862456998070656:12345-0@lo:400:kworker/u8:2.0 00000100:00100000:1.0:1776178359.140862:0:20140:0:(service.c:2372:ptlrpc_server_handle_request()) Handled RPC req@00000000b4857168 pname:cluuid+ref:pid:xid:nid:opc:job ll_mgs_0002:ac61f839-0e8e-4cac-9c59-08bcfbbad330+18:3355:x1862456998070656:12345-0@lo:400:kworker/u8:2.0 Request processed in 62us (3184us total) trans 0 rc 0/0 00000100:00100000:1.0:1776178359.140870:0:20140:0:(nrs_fifo.c:241:nrs_fifo_req_stop()) NRS stop fifo request from 12345-0@lo, seq: 216 00000100:00100000:3.0:1776178359.154246:0:3355:0:(client.c:2215:ptlrpc_check_set()) Completed RPC req@00000000a6227918 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_02:ac61f839-0e8e-4cac-9c59-08bcfbbad330:3355:1862456998070656:0@lo:400:kworker/u8:2.0 00010000:00010000:2.0:1776178359.776906:0:9225:0:(ldlm_lockd.c:2792:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:1.0:1776178360.673459:0:3355:0:(ptlrpcd.c:415:ptlrpcd_check()) transfer 1 async RPCs [3->2] 00000100:00100000:1.0:1776178360.674162:0:3355:0:(client.c:1733:ptlrpc_send_new_req()) Sending RPC req@00000000d28be450 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_02:lustre-MDT0001-mdtlov_UUID:3355:1862456998071232:0@lo:13:osp-pre-1-1.0 00000100:00100000:1.0:1776178360.674293:0:3355:0:(events.c:361:request_in_callback()) peer: 12345-0@lo (source: 12345-0@lo) 00000100:00100000:2.0:1776178360.674441:0:9554:0:(service.c:2152:ptlrpc_server_handle_req_in()) got req x1862456998071232 00000100:00100000:2.0:1776178360.674535:0:9554:0:(nrs_fifo.c:179:nrs_fifo_req_get()) NRS start fifo request from 12345-0@lo, seq: 549 00000100:00100000:2.0:1776178360.674543:0:9554:0:(service.c:2324:ptlrpc_server_handle_request()) Handling RPC req@00000000a604eea4 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_create00:lustre-MDT0001-mdtlov_UUID+5:3355:x1862456998071232:12345-0@lo:13:osp-pre-1-1.0 00002000:00000024:2.0:1776178360.675330:0:9554:0:(ofd_obd.c:740:ofd_statfs()) blocks cached 229376 granted 491746368 pending 0 free 3923689472 avail 3692167168 00000100:00100000:2.0:1776178360.675430:0:9554:0:(service.c:2372:ptlrpc_server_handle_request()) Handled RPC req@00000000a604eea4 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_create00:lustre-MDT0001-mdtlov_UUID+5:3355:x1862456998071232:12345-0@lo:13:osp-pre-1-1.0 Request processed in 885us (1138us total) trans 0 rc 0/0 00000100:00100000:2.0:1776178360.675441:0:9554:0:(nrs_fifo.c:241:nrs_fifo_req_stop()) NRS stop fifo request from 12345-0@lo, seq: 549 00000100:00100000:1.0:1776178360.675652:0:3355:0:(client.c:2215:ptlrpc_check_set()) Completed RPC req@00000000d28be450 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_02:lustre-MDT0001-mdtlov_UUID:3355:1862456998071232:0@lo:13:osp-pre-1-1.0 00010000:00010000:2.0:1776178360.801103:0:6244:0:(ldlm_lockd.c:2792:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:3.0:1776178361.184580:0:3353:0:(client.c:1733:ptlrpc_send_new_req()) Sending RPC req@0000000083bc4c0f pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_00:lustre-MDT0001-mdtlov_UUID:3353:1862456998071296:0@lo:13:osp-pre-0-1.0 00000100:00100000:3.0:1776178361.184721:0:3353:0:(events.c:361:request_in_callback()) peer: 12345-0@lo (source: 12345-0@lo) 00000100:00100000:2.0:1776178361.184858:0:8231:0:(service.c:2152:ptlrpc_server_handle_req_in()) got req x1862456998071296 00000100:00100000:2.0:1776178361.184930:0:8231:0:(nrs_fifo.c:179:nrs_fifo_req_get()) NRS start fifo request from 12345-0@lo, seq: 550 00000100:00100000:2.0:1776178361.184938:0:8231:0:(service.c:2324:ptlrpc_server_handle_request()) Handling RPC req@000000008f48ae7c pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_create00:lustre-MDT0001-mdtlov_UUID+5:3353:x1862456998071296:12345-0@lo:13:osp-pre-0-1.0 00002000:00000024:2.0:1776178361.184992:0:8231:0:(ofd_obd.c:740:ofd_statfs()) blocks cached 315392 granted 484883776 pending 0 free 3923689472 avail 3692167168 00000100:00100000:2.0:1776178361.185089:0:8231:0:(service.c:2372:ptlrpc_server_handle_request()) Handled RPC req@000000008f48ae7c pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_create00:lustre-MDT0001-mdtlov_UUID+5:3353:x1862456998071296:12345-0@lo:13:osp-pre-0-1.0 Request processed in 151us (370us total) trans 0 rc 0/0 00000100:00100000:2.0:1776178361.185097:0:8231:0:(nrs_fifo.c:241:nrs_fifo_req_stop()) NRS stop fifo request from 12345-0@lo, seq: 550 00000100:00100000:3.0:1776178361.185498:0:3353:0:(client.c:2215:ptlrpc_check_set()) Completed RPC req@0000000083bc4c0f pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_00:lustre-MDT0001-mdtlov_UUID:3353:1862456998071296:0@lo:13:osp-pre-0-1.0 00010000:00010000:2.0:1776178361.827170:0:6245:0:(ldlm_lockd.c:2792:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:0.0:1776178362.661589:0:3347:0:(events.c:361:request_in_callback()) peer: 12345-192.168.204.3@tcp (source: 12345-192.168.204.3@tcp) 00000100:00100000:1.0:1776178362.661710:0:20137:0:(service.c:2152:ptlrpc_server_handle_req_in()) got req x1862456979118912 00000100:00100000:1.0:1776178362.661767:0:20137:0:(nrs_fifo.c:179:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.204.3@tcp, seq: 217 00000100:00100000:1.0:1776178362.661773:0:20137:0:(service.c:2324:ptlrpc_server_handle_request()) Handling RPC req@0000000088793b53 pname:cluuid+ref:pid:xid:nid:opc:job ll_mgs_0000:5b5c66b1-1a2f-4651-8548-0c75b9a38a94+9:2271:x1862456979118912:12345-192.168.204.3@tcp:400:kworker/u8:4.0 00000100:00100000:1.0:1776178362.661848:0:20137:0:(service.c:2372:ptlrpc_server_handle_request()) Handled RPC req@0000000088793b53 pname:cluuid+ref:pid:xid:nid:opc:job ll_mgs_0000:5b5c66b1-1a2f-4651-8548-0c75b9a38a94+9:2271:x1862456979118912:12345-192.168.204.3@tcp:400:kworker/u8:4.0 Request processed in 74us (269us total) trans 0 rc 0/0 00000100:00100000:1.0:1776178362.661856:0:20137:0:(nrs_fifo.c:241:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.204.3@tcp, seq: 217 00000100:00100000:2.0:1776178362.664920:0:3346:0:(events.c:361:request_in_callback()) peer: 12345-192.168.204.3@tcp (source: 12345-192.168.204.3@tcp) 00000100:00100000:2.0:1776178362.665044:0:3346:0:(events.c:361:request_in_callback()) peer: 12345-192.168.204.3@tcp (source: 12345-192.168.204.3@tcp) 00000100:00100000:2.0:1776178362.665129:0:3346:0:(events.c:361:request_in_callback()) peer: 12345-192.168.204.3@tcp (source: 12345-192.168.204.3@tcp) 00000100:00100000:1.0:1776178362.665334:0:14440:0:(service.c:2152:ptlrpc_server_handle_req_in()) got req x1862456979119104 00000100:00100000:1.0:1776178362.665404:0:14440:0:(nrs_fifo.c:179:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.204.3@tcp, seq: 635 00000100:00100000:1.0:1776178362.665412:0:14440:0:(service.c:2324:ptlrpc_server_handle_request()) Handling RPC req@00000000d28be450 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_004:b50963bd-8eba-4432-9a54-8888d00e3fa1+8:2271:x1862456979119104:12345-192.168.204.3@tcp:400:kworker/u8:4.0 00000100:00100000:1.0:1776178362.665493:0:14440:0:(service.c:2372:ptlrpc_server_handle_request()) Handled RPC req@00000000d28be450 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_004:b50963bd-8eba-4432-9a54-8888d00e3fa1+8:2271:x1862456979119104:12345-192.168.204.3@tcp:400:kworker/u8:4.0 Request processed in 79us (449us total) trans 0 rc 0/0 00000100:00100000:1.0:1776178362.665501:0:14440:0:(nrs_fifo.c:241:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.204.3@tcp, seq: 635 00000100:00100000:2.0:1776178362.671129:0:3346:0:(events.c:361:request_in_callback()) peer: 12345-192.168.204.3@tcp (source: 12345-192.168.204.3@tcp) 00000100:00100000:1.0:1776178362.671233:0:20755:0:(service.c:2152:ptlrpc_server_handle_req_in()) got req x1862456979119168 00000100:00100000:1.0:1776178362.671288:0:20755:0:(nrs_fifo.c:179:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.204.3@tcp, seq: 636 00000100:00100000:1.0:1776178362.671295:0:20755:0:(service.c:2324:ptlrpc_server_handle_request()) Handling RPC req@00000000664eb80b pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_006:b50963bd-8eba-4432-9a54-8888d00e3fa1+9:2271:x1862456979119168:12345-192.168.204.3@tcp:400:kworker/u8:4.0 00000100:00100000:1.0:1776178362.671377:0:20755:0:(service.c:2372:ptlrpc_server_handle_request()) Handled RPC req@00000000664eb80b pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_006:b50963bd-8eba-4432-9a54-8888d00e3fa1+9:2271:x1862456979119168:12345-192.168.204.3@tcp:400:kworker/u8:4.0 Request processed in 81us (258us total) trans 0 rc 0/0 00000100:00100000:1.0:1776178362.671395:0:20755:0:(nrs_fifo.c:241:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.204.3@tcp, seq: 636 00000100:00100000:0.0:1776178362.672323:0:6255:0:(service.c:2152:ptlrpc_server_handle_req_in()) got req x1862456979118976 00000100:00100000:1.0:1776178362.672977:0:6255:0:(service.c:2152:ptlrpc_server_handle_req_in()) got req x1862456979119040 00000100:00100000:1.0:1776178362.673017:0:6255:0:(nrs_fifo.c:179:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.204.3@tcp, seq: 1021 00000100:00100000:1.0:1776178362.673023:0:6255:0:(service.c:2324:ptlrpc_server_handle_request()) Handling RPC req@0000000015f283a6 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_000:b50963bd-8eba-4432-9a54-8888d00e3fa1+6:2271:x1862456979119040:12345-192.168.204.3@tcp:400:kworker/u8:4.0 00000100:00100000:1.0:1776178362.673135:0:6255:0:(service.c:2372:ptlrpc_server_handle_request()) Handled RPC req@0000000015f283a6 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_000:b50963bd-8eba-4432-9a54-8888d00e3fa1+6:2271:x1862456979119040:12345-192.168.204.3@tcp:400:kworker/u8:4.0 Request processed in 111us (8006us total) trans 0 rc 0/0 00000100:00100000:1.0:1776178362.673142:0:6255:0:(nrs_fifo.c:241:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.204.3@tcp, seq: 1021 00000100:00100000:1.0:1776178362.673160:0:6255:0:(nrs_fifo.c:179:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.204.3@tcp, seq: 6748 00000100:00100000:1.0:1776178362.673164:0:6255:0:(service.c:2324:ptlrpc_server_handle_request()) Handling RPC req@00000000fab910b9 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_000:0+-99:2269:x1862456979118976:12345-192.168.204.3@tcp:38:kworker/u8:4.0 00010000:02000000:1.0:1776178362.673179:0:6255:0:(ldlm_lib.c:1139:target_handle_connect()) lustre-MDT0000: Not available for connect from 192.168.204.3@tcp (stopping) 00000100:00100000:1.0:1776178362.673201:0:6255:0:(service.c:2372:ptlrpc_server_handle_request()) Handled RPC req@00000000fab910b9 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_000:0+-99:2269:x1862456979118976:12345-192.168.204.3@tcp:38:kworker/u8:4.0 Request processed in 37us (8290us total) trans 0 rc -19/-19 00000100:00100000:1.0:1776178362.673208:0:6255:0:(nrs_fifo.c:241:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.204.3@tcp, seq: 6748 00010000:00010000:2.0:1776178362.859463:0:9224:0:(ldlm_lockd.c:2792:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00010000:00010000:2.0:1776178363.873829:0:9225:0:(ldlm_lockd.c:2792:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00080000:2.0:1776178364.256515:0:36:0:(pinger.c:251:ptlrpc_pinger_process_import()) lustre-MDT0000-lwp-OST0000_UUID->lustre-MDT0000_UUID: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00080000:2.0:1776178364.256522:0:36:0:(recover.c:57:ptlrpc_initiate_recovery()) lustre-MDT0000_UUID: starting recovery 00000100:00080000:2.0:1776178364.256526:0:36:0:(import.c:86:import_set_state_nolock()) 0000000047601a98 lustre-MDT0000_UUID: changing import state from DISCONN to CONNECTING 00000100:00080000:2.0:1776178364.256535:0:36:0:(import.c:534:import_select_connection()) lustre-MDT0000-lwp-OST0000: connect to NID 0@lo last attempt 1066 00000100:00080000:2.0:1776178364.256539:0:36:0:(import.c:581:import_select_connection()) lustre-MDT0000-lwp-OST0000: tried all connections, increasing latency to 21s 00000100:00080000:2.0:1776178364.256546:0:36:0:(import.c:615:import_select_connection()) lustre-MDT0000-lwp-OST0000: import 0000000047601a98 using connection 192.168.204.103@tcp/0@lo 00000100:00100000:2.0:1776178364.256570:0:36:0:(import.c:807:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5) req@00000000a0c64d21 x1862456998071424/t0(0) o38->lustre-MDT0000-lwp-OST0000@0@lo:12/10 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' 00000100:00080000:2.0:1776178364.256612:0:36:0:(pinger.c:251:ptlrpc_pinger_process_import()) lustre-MDT0000-lwp-OST0001_UUID->lustre-MDT0000_UUID: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00080000:2.0:1776178364.256616:0:36:0:(recover.c:57:ptlrpc_initiate_recovery()) lustre-MDT0000_UUID: starting recovery 00000100:00080000:2.0:1776178364.256617:0:36:0:(import.c:86:import_set_state_nolock()) 00000000dfc8db13 lustre-MDT0000_UUID: changing import state from DISCONN to CONNECTING 00000100:00080000:2.0:1776178364.256620:0:36:0:(import.c:534:import_select_connection()) lustre-MDT0000-lwp-OST0001: connect to NID 0@lo last attempt 1066 00000100:00080000:2.0:1776178364.256622:0:36:0:(import.c:581:import_select_connection()) lustre-MDT0000-lwp-OST0001: tried all connections, increasing latency to 21s 00000100:00080000:2.0:1776178364.256625:0:36:0:(import.c:615:import_select_connection()) lustre-MDT0000-lwp-OST0001: import 00000000dfc8db13 using connection 192.168.204.103@tcp/0@lo 00000100:00100000:2.0:1776178364.256637:0:36:0:(import.c:807:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5) req@00000000ec65370c x1862456998071552/t0(0) o38->lustre-MDT0000-lwp-OST0001@0@lo:12/10 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' 00000100:00080000:2.0:1776178364.256662:0:36:0:(pinger.c:251:ptlrpc_pinger_process_import()) lustre-MDT0001-mdtlov_UUID->lustre-MDT0000_UUID: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00080000:2.0:1776178364.256665:0:36:0:(recover.c:57:ptlrpc_initiate_recovery()) lustre-MDT0000_UUID: starting recovery 00000100:00080000:2.0:1776178364.256666:0:36:0:(import.c:86:import_set_state_nolock()) 00000000ba4f1821 lustre-MDT0000_UUID: changing import state from DISCONN to CONNECTING 00000100:00080000:2.0:1776178364.256669:0:36:0:(import.c:534:import_select_connection()) lustre-MDT0000-osp-MDT0001: connect to NID 0@lo last attempt 1066 00000100:00080000:2.0:1776178364.256671:0:36:0:(import.c:581:import_select_connection()) lustre-MDT0000-osp-MDT0001: tried all connections, increasing latency to 21s 00000100:00080000:2.0:1776178364.256674:0:36:0:(import.c:615:import_select_connection()) lustre-MDT0000-osp-MDT0001: import 00000000ba4f1821 using connection 192.168.204.103@tcp/0@lo 00000100:00100000:2.0:1776178364.256685:0:36:0:(import.c:807:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5) req@000000008c3fbdb6 x1862456998071680/t0(0) o38->lustre-MDT0000-osp-MDT0001@0@lo:24/4 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' 00000100:00080000:2.0:1776178364.256706:0:36:0:(pinger.c:251:ptlrpc_pinger_process_import()) lustre-MDT0000-lwp-MDT0001_UUID->lustre-MDT0000_UUID: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00080000:2.0:1776178364.256709:0:36:0:(recover.c:57:ptlrpc_initiate_recovery()) lustre-MDT0000_UUID: starting recovery 00000100:00080000:2.0:1776178364.256710:0:36:0:(import.c:86:import_set_state_nolock()) 00000000a7c55b1f lustre-MDT0000_UUID: changing import state from DISCONN to CONNECTING 00000100:00080000:2.0:1776178364.256714:0:36:0:(import.c:534:import_select_connection()) lustre-MDT0000-lwp-MDT0001: connect to NID 0@lo last attempt 1066 00000100:00080000:2.0:1776178364.256715:0:36:0:(import.c:581:import_select_connection()) lustre-MDT0000-lwp-MDT0001: tried all connections, increasing latency to 21s 00000100:00080000:2.0:1776178364.256721:0:36:0:(import.c:615:import_select_connection()) lustre-MDT0000-lwp-MDT0001: import 00000000a7c55b1f using connection 192.168.204.103@tcp/0@lo 00000100:00100000:2.0:1776178364.256743:0:36:0:(import.c:807:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5) req@000000001d51eec7 x1862456998071872/t0(0) o38->lustre-MDT0000-lwp-MDT0001@0@lo:12/10 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' 00000100:00100000:3.0:1776178364.256839:0:3353:0:(client.c:1733:ptlrpc_send_new_req()) Sending RPC req@00000000a30eb964 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_00:lustre-MDT0001-mdtlov_UUID:3353:1862456998071744:0@lo:400:kworker/u8:1.0 00000100:00100000:3.0:1776178364.256924:0:3353:0:(events.c:361:request_in_callback()) peer: 12345-0@lo (source: 12345-0@lo) 00000100:00100000:3.0:1776178364.256970:0:3353:0:(ptlrpcd.c:415:ptlrpcd_check()) transfer 2 async RPCs [1->0] 00000100:00100000:3.0:1776178364.256975:0:3353:0:(client.c:1733:ptlrpc_send_new_req()) Sending RPC req@0000000083b11d73 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_00:ac61f839-0e8e-4cac-9c59-08bcfbbad330:3353:1862456998071360:0@lo:400:kworker/u8:1.0 00000100:00100000:3.0:1776178364.257006:0:3353:0:(events.c:361:request_in_callback()) peer: 12345-0@lo (source: 12345-0@lo) 00000100:00100000:3.0:1776178364.257018:0:3353:0:(client.c:1733:ptlrpc_send_new_req()) Sending RPC req@00000000b766edf6 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_00:lustre-MDT0001-mdtlov_UUID:3353:1862456998071808:0@lo:400:kworker/u8:1.0 00000100:00100000:3.0:1776178364.257036:0:3353:0:(events.c:361:request_in_callback()) peer: 12345-0@lo (source: 12345-0@lo) 00000100:00100000:3.0:1776178364.257084:0:3356:0:(client.c:1733:ptlrpc_send_new_req()) Sending RPC req@000000006342fd4b pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_03:lustre-MDT0001-lwp-OST0001_UUID:3356:1862456998071616:0@lo:400:kworker/u8:1.0 00000100:00100000:3.0:1776178364.257108:0:3356:0:(events.c:361:request_in_callback()) peer: 12345-0@lo (source: 12345-0@lo) 00000100:00100000:3.0:1776178364.257123:0:3356:0:(ptlrpcd.c:415:ptlrpcd_check()) transfer 1 async RPCs [2->3] 00000100:00100000:3.0:1776178364.257127:0:3356:0:(client.c:1733:ptlrpc_send_new_req()) Sending RPC req@00000000d384e871 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_03:lustre-MDT0001-lwp-OST0000_UUID:3356:1862456998071488:0@lo:400:kworker/u8:1.0 00000100:00100000:3.0:1776178364.257160:0:3356:0:(events.c:361:request_in_callback()) peer: 12345-0@lo (source: 12345-0@lo) 00000100:00100000:3.0:1776178364.257188:0:6256:0:(service.c:2152:ptlrpc_server_handle_req_in()) got req x1862456998071616 00000100:00100000:3.0:1776178364.257223:0:6256:0:(service.c:2152:ptlrpc_server_handle_req_in()) got req x1862456998071488 00000100:00100000:3.0:1776178364.257237:0:6256:0:(nrs_fifo.c:179:nrs_fifo_req_get()) NRS start fifo request from 12345-0@lo, seq: 1022 00000100:00100000:3.0:1776178364.257242:0:6256:0:(service.c:2324:ptlrpc_server_handle_request()) Handling RPC req@00000000b3188344 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_001:lustre-MDT0001-lwp-OST0001_UUID+5:3356:x1862456998071616:12345-0@lo:400:kworker/u8:1.0 00000100:00100000:3.0:1776178364.257287:0:6256:0:(service.c:2372:ptlrpc_server_handle_request()) Handled RPC req@00000000b3188344 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_001:lustre-MDT0001-lwp-OST0001_UUID+5:3356:x1862456998071616:12345-0@lo:400:kworker/u8:1.0 Request processed in 44us (179us total) trans 0 rc 0/0 00000100:00100000:3.0:1776178364.257294:0:6256:0:(nrs_fifo.c:241:nrs_fifo_req_stop()) NRS stop fifo request from 12345-0@lo, seq: 1022 00000100:00100000:3.0:1776178364.257308:0:6256:0:(nrs_fifo.c:179:nrs_fifo_req_get()) NRS start fifo request from 12345-0@lo, seq: 1023 00000100:00100000:3.0:1776178364.257311:0:6256:0:(service.c:2324:ptlrpc_server_handle_request()) Handling RPC req@00000000de25df9f pname:cluuid+ref:pid:xid:nid:opc:job mdt00_001:lustre-MDT0001-lwp-OST0000_UUID+5:3356:x1862456998071488:12345-0@lo:400:kworker/u8:1.0 00000100:00100000:3.0:1776178364.257333:0:6256:0:(service.c:2372:ptlrpc_server_handle_request()) Handled RPC req@00000000de25df9f pname:cluuid+ref:pid:xid:nid:opc:job mdt00_001:lustre-MDT0001-lwp-OST0000_UUID+5:3356:x1862456998071488:12345-0@lo:400:kworker/u8:1.0 Request processed in 21us (173us total) trans 0 rc 0/0 00000100:00100000:3.0:1776178364.257338:0:6256:0:(nrs_fifo.c:241:nrs_fifo_req_stop()) NRS stop fifo request from 12345-0@lo, seq: 1023 00000100:00100000:3.0:1776178364.257381:0:3356:0:(client.c:2215:ptlrpc_check_set()) Completed RPC req@00000000d384e871 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_03:lustre-MDT0001-lwp-OST0000_UUID:3356:1862456998071488:0@lo:400:kworker/u8:1.0 00000100:00100000:3.0:1776178364.257390:0:3356:0:(client.c:2215:ptlrpc_check_set()) Completed RPC req@000000006342fd4b pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_03:lustre-MDT0001-lwp-OST0001_UUID:3356:1862456998071616:0@lo:400:kworker/u8:1.0 00000100:00100000:1.0:1776178364.257474:0:20755:0:(service.c:2152:ptlrpc_server_handle_req_in()) got req x1862456998071744 00000100:00100000:1.0:1776178364.257511:0:20755:0:(service.c:2152:ptlrpc_server_handle_req_in()) got req x1862456998071808 00000100:00100000:1.0:1776178364.257526:0:20755:0:(nrs_fifo.c:179:nrs_fifo_req_get()) NRS start fifo request from 12345-0@lo, seq: 637 00000100:00100000:1.0:1776178364.257529:0:20755:0:(service.c:2324:ptlrpc_server_handle_request()) Handling RPC req@00000000747e17cb pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_006:lustre-MDT0001-mdtlov_UUID+5:3353:x1862456998071744:12345-0@lo:400:kworker/u8:1.0 00000100:00100000:1.0:1776178364.257567:0:20755:0:(service.c:2372:ptlrpc_server_handle_request()) Handled RPC req@00000000747e17cb pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_006:lustre-MDT0001-mdtlov_UUID+5:3353:x1862456998071744:12345-0@lo:400:kworker/u8:1.0 Request processed in 36us (644us total) trans 0 rc 0/0 00000100:00100000:1.0:1776178364.257573:0:20755:0:(nrs_fifo.c:241:nrs_fifo_req_stop()) NRS stop fifo request from 12345-0@lo, seq: 637 00000100:00100000:1.0:1776178364.257585:0:20755:0:(nrs_fifo.c:179:nrs_fifo_req_get()) NRS start fifo request from 12345-0@lo, seq: 638 00000100:00100000:1.0:1776178364.257588:0:20755:0:(service.c:2324:ptlrpc_server_handle_request()) Handling RPC req@00000000a430b18f pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_006:lustre-MDT0001-mdtlov_UUID+5:3353:x1862456998071808:12345-0@lo:400:kworker/u8:1.0 00000100:00100000:1.0:1776178364.257609:0:20755:0:(service.c:2372:ptlrpc_server_handle_request()) Handled RPC req@00000000a430b18f pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_006:lustre-MDT0001-mdtlov_UUID+5:3353:x1862456998071808:12345-0@lo:400:kworker/u8:1.0 Request processed in 21us (573us total) trans 0 rc 0/0 00000100:00100000:1.0:1776178364.257614:0:20755:0:(nrs_fifo.c:241:nrs_fifo_req_stop()) NRS stop fifo request from 12345-0@lo, seq: 638 00000100:00100000:1.0:1776178364.257637:0:3353:0:(client.c:2215:ptlrpc_check_set()) Completed RPC req@00000000b766edf6 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_00:lustre-MDT0001-mdtlov_UUID:3353:1862456998071808:0@lo:400:kworker/u8:1.0 00000100:00100000:1.0:1776178364.257657:0:3353:0:(client.c:2215:ptlrpc_check_set()) Completed RPC req@00000000a30eb964 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_00:lustre-MDT0001-mdtlov_UUID:3353:1862456998071744:0@lo:400:kworker/u8:1.0 00000100:00100000:2.0:1776178364.258097:0:20140:0:(service.c:2152:ptlrpc_server_handle_req_in()) got req x1862456998071360 00000100:00100000:2.0:1776178364.258185:0:20140:0:(nrs_fifo.c:179:nrs_fifo_req_get()) NRS start fifo request from 12345-0@lo, seq: 218 00000100:00100000:2.0:1776178364.258189:0:20140:0:(service.c:2324:ptlrpc_server_handle_request()) Handling RPC req@000000004c7a25f9 pname:cluuid+ref:pid:xid:nid:opc:job ll_mgs_0002:ac61f839-0e8e-4cac-9c59-08bcfbbad330+18:3353:x1862456998071360:12345-0@lo:400:kworker/u8:1.0 00000100:00100000:2.0:1776178364.258231:0:20140:0:(service.c:2372:ptlrpc_server_handle_request()) Handled RPC req@000000004c7a25f9 pname:cluuid+ref:pid:xid:nid:opc:job ll_mgs_0002:ac61f839-0e8e-4cac-9c59-08bcfbbad330+18:3353:x1862456998071360:12345-0@lo:400:kworker/u8:1.0 Request processed in 40us (1224us total) trans 0 rc 0/0 00000100:00100000:2.0:1776178364.258236:0:20140:0:(nrs_fifo.c:241:nrs_fifo_req_stop()) NRS stop fifo request from 12345-0@lo, seq: 218 00000100:00100000:3.0:1776178364.258326:0:3353:0:(client.c:2215:ptlrpc_check_set()) Completed RPC req@0000000083b11d73 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_00:ac61f839-0e8e-4cac-9c59-08bcfbbad330:3353:1862456998071360:0@lo:400:kworker/u8:1.0 00000100:00100000:0.0:1776178364.258607:0:3352:0:(client.c:1733:ptlrpc_send_new_req()) Sending RPC req@00000000a0c64d21 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:lustre-MDT0000-lwp-OST0000_UUID:3352:1862456998071424:0@lo:38:kworker/u8:1.0 00000100:00100000:0.0:1776178364.258642:0:3352:0:(events.c:361:request_in_callback()) peer: 12345-0@lo (source: 12345-0@lo) 00000100:00100000:0.0:1776178364.258675:0:3352:0:(client.c:1733:ptlrpc_send_new_req()) Sending RPC req@00000000ec65370c pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:lustre-MDT0000-lwp-OST0001_UUID:3352:1862456998071552:0@lo:38:kworker/u8:1.0 00000100:00100000:0.0:1776178364.258695:0:3352:0:(events.c:361:request_in_callback()) peer: 12345-0@lo (source: 12345-0@lo) 00000100:00100000:0.0:1776178364.258710:0:3352:0:(client.c:1733:ptlrpc_send_new_req()) Sending RPC req@000000008c3fbdb6 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:lustre-MDT0001-mdtlov_UUID:3352:1862456998071680:0@lo:38:kworker/u8:1.0 00000100:00100000:0.0:1776178364.258746:0:3352:0:(events.c:361:request_in_callback()) peer: 12345-0@lo (source: 12345-0@lo) 00000100:00100000:0.0:1776178364.258766:0:3352:0:(client.c:1733:ptlrpc_send_new_req()) Sending RPC req@000000001d51eec7 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:lustre-MDT0000-lwp-MDT0001_UUID:3352:1862456998071872:0@lo:38:kworker/u8:1.0 00000100:00100000:0.0:1776178364.258789:0:3352:0:(events.c:361:request_in_callback()) peer: 12345-0@lo (source: 12345-0@lo) 00000100:00100000:0.0:1776178364.258819:0:11281:0:(service.c:2152:ptlrpc_server_handle_req_in()) got req x1862456998071424 00000100:00100000:0.0:1776178364.258840:0:11281:0:(service.c:2152:ptlrpc_server_handle_req_in()) got req x1862456998071552 00000100:00100000:0.0:1776178364.258860:0:11281:0:(service.c:2152:ptlrpc_server_handle_req_in()) got req x1862456998071872 00000100:00100000:0.0:1776178364.258870:0:11281:0:(nrs_fifo.c:179:nrs_fifo_req_get()) NRS start fifo request from 12345-0@lo, seq: 6749 00000100:00100000:0.0:1776178364.258874:0:11281:0:(service.c:2324:ptlrpc_server_handle_request()) Handling RPC req@0000000039cf69e0 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_004:0+-99:3352:x1862456998071424:12345-0@lo:38:kworker/u8:1.0 00010000:02000000:0.0:1776178364.258892:0:11281:0:(ldlm_lib.c:1139:target_handle_connect()) lustre-MDT0000: Not available for connect from 0@lo (stopping) 00000100:00100000:0.0:1776178364.258929:0:11281:0:(service.c:2372:ptlrpc_server_handle_request()) Handled RPC req@0000000039cf69e0 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_004:0+-99:3352:x1862456998071424:12345-0@lo:38:kworker/u8:1.0 Request processed in 54us (287us total) trans 0 rc -19/-19 00000100:00100000:0.0:1776178364.258935:0:11281:0:(nrs_fifo.c:241:nrs_fifo_req_stop()) NRS stop fifo request from 12345-0@lo, seq: 6749 00000100:00100000:0.0:1776178364.258945:0:11281:0:(nrs_fifo.c:179:nrs_fifo_req_get()) NRS start fifo request from 12345-0@lo, seq: 6750 00000100:00100000:0.0:1776178364.258948:0:11281:0:(service.c:2324:ptlrpc_server_handle_request()) Handling RPC req@00000000e1d307fb pname:cluuid+ref:pid:xid:nid:opc:job mdt00_004:0+-99:3352:x1862456998071552:12345-0@lo:38:kworker/u8:1.0 00010000:02000000:0.0:1776178364.258954:0:11281:0:(ldlm_lib.c:1139:target_handle_connect()) lustre-MDT0000: Not available for connect from 0@lo (stopping) 00000100:00100000:0.0:1776178364.258975:0:11281:0:(service.c:2372:ptlrpc_server_handle_request()) Handled RPC req@00000000e1d307fb pname:cluuid+ref:pid:xid:nid:opc:job mdt00_004:0+-99:3352:x1862456998071552:12345-0@lo:38:kworker/u8:1.0 Request processed in 26us (280us total) trans 0 rc -19/-19 00000100:00100000:0.0:1776178364.258980:0:11281:0:(nrs_fifo.c:241:nrs_fifo_req_stop()) NRS stop fifo request from 12345-0@lo, seq: 6750 00000100:00100000:0.0:1776178364.258990:0:11281:0:(nrs_fifo.c:179:nrs_fifo_req_get()) NRS start fifo request from 12345-0@lo, seq: 6751 00000100:00100000:0.0:1776178364.258993:0:11281:0:(service.c:2324:ptlrpc_server_handle_request()) Handling RPC req@000000009d7a00b9 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_004:0+-99:3352:x1862456998071872:12345-0@lo:38:kworker/u8:1.0 00010000:02000000:0.0:1776178364.258998:0:11281:0:(ldlm_lib.c:1139:target_handle_connect()) lustre-MDT0000: Not available for connect from 0@lo (stopping) 00000100:00100000:0.0:1776178364.259015:0:11281:0:(service.c:2372:ptlrpc_server_handle_request()) Handled RPC req@000000009d7a00b9 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_004:0+-99:3352:x1862456998071872:12345-0@lo:38:kworker/u8:1.0 Request processed in 20us (226us total) trans 0 rc -19/-19 00000100:00100000:0.0:1776178364.259020:0:11281:0:(nrs_fifo.c:241:nrs_fifo_req_stop()) NRS stop fifo request from 12345-0@lo, seq: 6751 00000100:00100000:0.0:1776178364.259041:0:15213:0:(service.c:2152:ptlrpc_server_handle_req_in()) got req x1862456998071680 00000100:00100000:0.0:1776178364.259090:0:15213:0:(nrs_fifo.c:179:nrs_fifo_req_get()) NRS start fifo request from 12345-0@lo, seq: 739 00000100:00100000:0.0:1776178364.259094:0:15213:0:(service.c:2324:ptlrpc_server_handle_request()) Handling RPC req@00000000a92aab25 pname:cluuid+ref:pid:xid:nid:opc:job mdt_out00_003:0+-99:3352:x1862456998071680:12345-0@lo:38:kworker/u8:1.0 00010000:02000000:0.0:1776178364.259101:0:15213:0:(ldlm_lib.c:1139:target_handle_connect()) lustre-MDT0000: Not available for connect from 0@lo (stopping) 00000100:00100000:0.0:1776178364.259125:0:15213:0:(service.c:2372:ptlrpc_server_handle_request()) Handled RPC req@00000000a92aab25 pname:cluuid+ref:pid:xid:nid:opc:job mdt_out00_003:0+-99:3352:x1862456998071680:12345-0@lo:38:kworker/u8:1.0 Request processed in 31us (380us total) trans 0 rc -19/-19 00000100:00100000:0.0:1776178364.259131:0:15213:0:(nrs_fifo.c:241:nrs_fifo_req_stop()) NRS stop fifo request from 12345-0@lo, seq: 739 00000100:00080000:0.0:1776178364.259180:0:3352:0:(import.c:86:import_set_state_nolock()) 0000000047601a98 lustre-MDT0000_UUID: changing import state from CONNECTING to DISCONN 00000100:00080000:0.0:1776178364.259184:0:3352:0:(import.c:1429:ptlrpc_connect_interpret()) recovery of lustre-MDT0000_UUID on 192.168.204.103@tcp failed (-19) 00000100:00100000:0.0:1776178364.259188:0:3352:0:(client.c:2215:ptlrpc_check_set()) Completed RPC req@00000000a0c64d21 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:lustre-MDT0000-lwp-OST0000_UUID:3352:1862456998071424:0@lo:38:kworker/u8:1.0 00000100:00080000:0.0:1776178364.259196:0:3352:0:(import.c:86:import_set_state_nolock()) 00000000dfc8db13 lustre-MDT0000_UUID: changing import state from CONNECTING to DISCONN 00000100:00080000:0.0:1776178364.259199:0:3352:0:(import.c:1429:ptlrpc_connect_interpret()) recovery of lustre-MDT0000_UUID on 192.168.204.103@tcp failed (-19) 00000100:00100000:0.0:1776178364.259201:0:3352:0:(client.c:2215:ptlrpc_check_set()) Completed RPC req@00000000ec65370c pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:lustre-MDT0000-lwp-OST0001_UUID:3352:1862456998071552:0@lo:38:kworker/u8:1.0 00000100:00080000:0.0:1776178364.259207:0:3352:0:(import.c:86:import_set_state_nolock()) 00000000ba4f1821 lustre-MDT0000_UUID: changing import state from CONNECTING to DISCONN 00000100:00080000:0.0:1776178364.259209:0:3352:0:(import.c:1429:ptlrpc_connect_interpret()) recovery of lustre-MDT0000_UUID on 192.168.204.103@tcp failed (-19) 00000100:00100000:0.0:1776178364.259211:0:3352:0:(client.c:2215:ptlrpc_check_set()) Completed RPC req@000000008c3fbdb6 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:lustre-MDT0001-mdtlov_UUID:3352:1862456998071680:0@lo:38:kworker/u8:1.0 00000100:00080000:0.0:1776178364.259217:0:3352:0:(import.c:86:import_set_state_nolock()) 00000000a7c55b1f lustre-MDT0000_UUID: changing import state from CONNECTING to DISCONN 00000100:00080000:0.0:1776178364.259220:0:3352:0:(import.c:1429:ptlrpc_connect_interpret()) recovery of lustre-MDT0000_UUID on 192.168.204.103@tcp failed (-19) 00000100:00100000:0.0:1776178364.259222:0:3352:0:(client.c:2215:ptlrpc_check_set()) Completed RPC req@000000001d51eec7 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:lustre-MDT0000-lwp-MDT0001_UUID:3352:1862456998071872:0@lo:38:kworker/u8:1.0 00010000:00010000:2.0:1776178364.896453:0:6244:0:(ldlm_lockd.c:2792:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:1.0:1776178365.797385:0:3355:0:(client.c:1733:ptlrpc_send_new_req()) Sending RPC req@00000000478fd8d5 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_02:lustre-MDT0001-mdtlov_UUID:3355:1862456998071936:0@lo:13:osp-pre-1-1.0 00000100:00100000:1.0:1776178365.797476:0:3355:0:(events.c:361:request_in_callback()) peer: 12345-0@lo (source: 12345-0@lo) 00000100:00100000:2.0:1776178365.797780:0:9554:0:(service.c:2152:ptlrpc_server_handle_req_in()) got req x1862456998071936 00000100:00100000:2.0:1776178365.797865:0:9554:0:(nrs_fifo.c:179:nrs_fifo_req_get()) NRS start fifo request from 12345-0@lo, seq: 551 00000100:00100000:2.0:1776178365.797871:0:9554:0:(service.c:2324:ptlrpc_server_handle_request()) Handling RPC req@0000000037645a66 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_create00:lustre-MDT0001-mdtlov_UUID+5:3355:x1862456998071936:12345-0@lo:13:osp-pre-1-1.0 00002000:00000024:2.0:1776178365.797903:0:9554:0:(ofd_obd.c:740:ofd_statfs()) blocks cached 229376 granted 491746368 pending 0 free 3923689472 avail 3692167168 00000100:00100000:2.0:1776178365.797945:0:9554:0:(service.c:2372:ptlrpc_server_handle_request()) Handled RPC req@0000000037645a66 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_create00:lustre-MDT0001-mdtlov_UUID+5:3355:x1862456998071936:12345-0@lo:13:osp-pre-1-1.0 Request processed in 74us (481us total) trans 0 rc 0/0 00000100:00100000:2.0:1776178365.797951:0:9554:0:(nrs_fifo.c:241:nrs_fifo_req_stop()) NRS stop fifo request from 12345-0@lo, seq: 551 00000100:00100000:3.0:1776178365.798277:0:3355:0:(client.c:2215:ptlrpc_check_set()) Completed RPC req@00000000478fd8d5 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_02:lustre-MDT0001-mdtlov_UUID:3355:1862456998071936:0@lo:13:osp-pre-1-1.0 00010000:00010000:2.0:1776178365.921458:0:6245:0:(ldlm_lockd.c:2792:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:1.0:1776178366.305358:0:3355:0:(ptlrpcd.c:415:ptlrpcd_check()) transfer 1 async RPCs [3->2] 00000100:00100000:1.0:1776178366.305380:0:3355:0:(client.c:1733:ptlrpc_send_new_req()) Sending RPC req@00000000dab5abff pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_02:lustre-MDT0001-mdtlov_UUID:3355:1862456998072000:0@lo:13:osp-pre-0-1.0 00000100:00100000:1.0:1776178366.305442:0:3355:0:(events.c:361:request_in_callback()) peer: 12345-0@lo (source: 12345-0@lo) 00000100:00100000:2.0:1776178366.308828:0:9554:0:(service.c:2152:ptlrpc_server_handle_req_in()) got req x1862456998072000 00000100:00100000:2.0:1776178366.308907:0:9554:0:(nrs_fifo.c:179:nrs_fifo_req_get()) NRS start fifo request from 12345-0@lo, seq: 552 00000100:00100000:2.0:1776178366.308914:0:9554:0:(service.c:2324:ptlrpc_server_handle_request()) Handling RPC req@00000000a1262b14 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_create00:lustre-MDT0001-mdtlov_UUID+5:3355:x1862456998072000:12345-0@lo:13:osp-pre-0-1.0 00002000:00000024:2.0:1776178366.308954:0:9554:0:(ofd_obd.c:740:ofd_statfs()) blocks cached 315392 granted 484883776 pending 0 free 3923689472 avail 3692167168 00000100:00100000:2.0:1776178366.309044:0:9554:0:(service.c:2372:ptlrpc_server_handle_request()) Handled RPC req@00000000a1262b14 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_create00:lustre-MDT0001-mdtlov_UUID+5:3355:x1862456998072000:12345-0@lo:13:osp-pre-0-1.0 Request processed in 130us (3602us total) trans 0 rc 0/0 00000100:00100000:2.0:1776178366.309051:0:9554:0:(nrs_fifo.c:241:nrs_fifo_req_stop()) NRS stop fifo request from 12345-0@lo, seq: 552 00000100:00100000:3.0:1776178366.309256:0:3355:0:(client.c:2215:ptlrpc_check_set()) Completed RPC req@00000000dab5abff pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_02:lustre-MDT0001-mdtlov_UUID:3355:1862456998072000:0@lo:13:osp-pre-0-1.0 00010000:00010000:2.0:1776178366.944336:0:9224:0:(ldlm_lockd.c:2792:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:1.0:1776178367.786287:0:3347:0:(events.c:361:request_in_callback()) peer: 12345-192.168.204.3@tcp (source: 12345-192.168.204.3@tcp) 00000100:00100000:1.0:1776178367.786341:0:3347:0:(events.c:361:request_in_callback()) peer: 12345-192.168.204.3@tcp (source: 12345-192.168.204.3@tcp) 00000100:00100000:1.0:1776178367.786371:0:3347:0:(events.c:361:request_in_callback()) peer: 12345-192.168.204.3@tcp (source: 12345-192.168.204.3@tcp) 00000100:00100000:1.0:1776178367.786480:0:3347:0:(events.c:361:request_in_callback()) peer: 12345-192.168.204.3@tcp (source: 12345-192.168.204.3@tcp) 00000100:00100000:2.0:1776178367.787107:0:20140:0:(service.c:2152:ptlrpc_server_handle_req_in()) got req x1862456979119232 00000100:00100000:2.0:1776178367.787167:0:20140:0:(nrs_fifo.c:179:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.204.3@tcp, seq: 219 00000100:00100000:2.0:1776178367.787173:0:20140:0:(service.c:2324:ptlrpc_server_handle_request()) Handling RPC req@00000000ae8faa77 pname:cluuid+ref:pid:xid:nid:opc:job ll_mgs_0002:5b5c66b1-1a2f-4651-8548-0c75b9a38a94+9:2272:x1862456979119232:12345-192.168.204.3@tcp:400:kworker/u8:4.0 00000100:00100000:2.0:1776178367.787254:0:20140:0:(service.c:2372:ptlrpc_server_handle_request()) Handled RPC req@00000000ae8faa77 pname:cluuid+ref:pid:xid:nid:opc:job ll_mgs_0002:5b5c66b1-1a2f-4651-8548-0c75b9a38a94+9:2272:x1862456979119232:12345-192.168.204.3@tcp:400:kworker/u8:4.0 Request processed in 80us (976us total) trans 0 rc 0/0 00000100:00100000:2.0:1776178367.787262:0:20140:0:(nrs_fifo.c:241:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.204.3@tcp, seq: 219 00000100:00100000:1.0:1776178367.787471:0:3347:0:(events.c:361:request_in_callback()) peer: 12345-192.168.204.3@tcp (source: 12345-192.168.204.3@tcp) 00000100:00100000:0.0:1776178367.788446:0:6255:0:(service.c:2152:ptlrpc_server_handle_req_in()) got req x1862456979119296 00000100:00100000:0.0:1776178367.788482:0:6255:0:(service.c:2152:ptlrpc_server_handle_req_in()) got req x1862456979119360 00000100:00100000:0.0:1776178367.788508:0:6255:0:(nrs_fifo.c:179:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.204.3@tcp, seq: 1024 00000100:00100000:0.0:1776178367.788513:0:6255:0:(service.c:2324:ptlrpc_server_handle_request()) Handling RPC req@0000000003506921 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_000:b50963bd-8eba-4432-9a54-8888d00e3fa1+6:2273:x1862456979119360:12345-192.168.204.3@tcp:400:kworker/u8:4.0 00000100:00100000:0.0:1776178367.788564:0:6255:0:(service.c:2372:ptlrpc_server_handle_request()) Handled RPC req@0000000003506921 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_000:b50963bd-8eba-4432-9a54-8888d00e3fa1+6:2273:x1862456979119360:12345-192.168.204.3@tcp:400:kworker/u8:4.0 Request processed in 50us (2193us total) trans 0 rc 0/0 00000100:00100000:0.0:1776178367.788571:0:6255:0:(nrs_fifo.c:241:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.204.3@tcp, seq: 1024 00000100:00100000:0.0:1776178367.788601:0:6255:0:(nrs_fifo.c:179:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.204.3@tcp, seq: 6752 00000100:00100000:0.0:1776178367.788605:0:6255:0:(service.c:2324:ptlrpc_server_handle_request()) Handling RPC req@000000007a321680 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_000:0+-99:2269:x1862456979119296:12345-192.168.204.3@tcp:38:kworker/u8:4.0 00010000:02000000:0.0:1776178367.788619:0:6255:0:(ldlm_lib.c:1139:target_handle_connect()) lustre-MDT0000: Not available for connect from 192.168.204.3@tcp (stopping) 00000100:00100000:3.0:1776178367.790650:0:20762:0:(service.c:2152:ptlrpc_server_handle_req_in()) got req x1862456979119488 00000100:00100000:3.0:1776178367.790734:0:20762:0:(service.c:2152:ptlrpc_server_handle_req_in()) got req x1862456979119424 00000100:00100000:3.0:1776178367.790778:0:20762:0:(nrs_fifo.c:179:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.204.3@tcp, seq: 639 00000100:00100000:3.0:1776178367.790785:0:20762:0:(service.c:2324:ptlrpc_server_handle_request()) Handling RPC req@000000008c593d89 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_013:b50963bd-8eba-4432-9a54-8888d00e3fa1+9:2273:x1862456979119488:12345-192.168.204.3@tcp:400:kworker/u8:4.0 00000100:00100000:3.0:1776178367.790857:0:20762:0:(service.c:2372:ptlrpc_server_handle_request()) Handled RPC req@000000008c593d89 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_013:b50963bd-8eba-4432-9a54-8888d00e3fa1+9:2273:x1862456979119488:12345-192.168.204.3@tcp:400:kworker/u8:4.0 Request processed in 72us (4378us total) trans 0 rc 0/0 00000100:00100000:3.0:1776178367.790864:0:20762:0:(nrs_fifo.c:241:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.204.3@tcp, seq: 639 00000100:00100000:3.0:1776178367.790878:0:20762:0:(nrs_fifo.c:179:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.204.3@tcp, seq: 640 00000100:00100000:3.0:1776178367.790881:0:20762:0:(service.c:2324:ptlrpc_server_handle_request()) Handling RPC req@00000000a604eea4 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_013:b50963bd-8eba-4432-9a54-8888d00e3fa1+8:2273:x1862456979119424:12345-192.168.204.3@tcp:400:kworker/u8:4.0 00000100:00100000:3.0:1776178367.790901:0:20762:0:(service.c:2372:ptlrpc_server_handle_request()) Handled RPC req@00000000a604eea4 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_013:b50963bd-8eba-4432-9a54-8888d00e3fa1+8:2273:x1862456979119424:12345-192.168.204.3@tcp:400:kworker/u8:4.0 Request processed in 19us (3431us total) trans 0 rc 0/0 00000100:00100000:3.0:1776178367.790906:0:20762:0:(nrs_fifo.c:241:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.204.3@tcp, seq: 640 00000100:00100000:0.0:1776178367.817309:0:6255:0:(service.c:2372:ptlrpc_server_handle_request()) Handled RPC req@000000007a321680 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_000:0+-99:2269:x1862456979119296:12345-192.168.204.3@tcp:38:kworker/u8:4.0 Request processed in 28692us (30957us total) trans 0 rc -19/-19 00000100:00100000:0.0:1776178367.817327:0:6255:0:(nrs_fifo.c:241:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.204.3@tcp, seq: 6752 00010000:00010000:2.0:1776178367.970147:0:9225:0:(ldlm_lockd.c:2792:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00010000:00010000:3.0:1776178368.992440:0:6244:0:(ldlm_lockd.c:2792:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:3.0:1776178369.377773:0:3353:0:(client.c:1733:ptlrpc_send_new_req()) Sending RPC req@000000005643ab1b pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_00:ac61f839-0e8e-4cac-9c59-08bcfbbad330:3353:1862456998072064:0@lo:400:kworker/u8:1.0 00000100:00100000:3.0:1776178369.377881:0:3353:0:(events.c:361:request_in_callback()) peer: 12345-0@lo (source: 12345-0@lo) 00000100:00100000:1.0:1776178369.378009:0:20137:0:(service.c:2152:ptlrpc_server_handle_req_in()) got req x1862456998072064 00000100:00100000:1.0:1776178369.378119:0:20137:0:(nrs_fifo.c:179:nrs_fifo_req_get()) NRS start fifo request from 12345-0@lo, seq: 220 00000100:00100000:1.0:1776178369.378128:0:20137:0:(service.c:2324:ptlrpc_server_handle_request()) Handling RPC req@000000002efeec88 pname:cluuid+ref:pid:xid:nid:opc:job ll_mgs_0000:ac61f839-0e8e-4cac-9c59-08bcfbbad330+18:3353:x1862456998072064:12345-0@lo:400:kworker/u8:1.0 00000100:00080000:2.0:1776178369.378140:0:36:0:(pinger.c:251:ptlrpc_pinger_process_import()) lustre-MDT0000-lwp-OST0000_UUID->lustre-MDT0000_UUID: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00080000:2.0:1776178369.378146:0:36:0:(recover.c:57:ptlrpc_initiate_recovery()) lustre-MDT0000_UUID: starting recovery 00000100:00080000:2.0:1776178369.378151:0:36:0:(import.c:86:import_set_state_nolock()) 0000000047601a98 lustre-MDT0000_UUID: changing import state from DISCONN to CONNECTING 00000100:00080000:2.0:1776178369.378158:0:36:0:(import.c:534:import_select_connection()) lustre-MDT0000-lwp-OST0000: connect to NID 0@lo last attempt 1071 00000100:00080000:2.0:1776178369.378161:0:36:0:(import.c:581:import_select_connection()) lustre-MDT0000-lwp-OST0000: tried all connections, increasing latency to 21s 00000100:00080000:2.0:1776178369.378166:0:36:0:(import.c:615:import_select_connection()) lustre-MDT0000-lwp-OST0000: import 0000000047601a98 using connection 192.168.204.103@tcp/0@lo 00000100:00100000:2.0:1776178369.378196:0:36:0:(import.c:807:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5) req@00000000dd1281fe x1862456998072128/t0(0) o38->lustre-MDT0000-lwp-OST0000@0@lo:12/10 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' 00000100:00100000:0.0:1776178369.378232:0:3352:0:(client.c:1733:ptlrpc_send_new_req()) Sending RPC req@00000000dd1281fe pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:lustre-MDT0000-lwp-OST0000_UUID:3352:1862456998072128:0@lo:38:kworker/u8:1.0 00000100:00100000:0.0:1776178369.378274:0:3352:0:(events.c:361:request_in_callback()) peer: 12345-0@lo (source: 12345-0@lo) 00000100:00100000:0.0:1776178369.378323:0:6255:0:(service.c:2152:ptlrpc_server_handle_req_in()) got req x1862456998072128 00000100:00100000:0.0:1776178369.378365:0:6255:0:(nrs_fifo.c:179:nrs_fifo_req_get()) NRS start fifo request from 12345-0@lo, seq: 6753 00000100:00100000:0.0:1776178369.378369:0:6255:0:(service.c:2324:ptlrpc_server_handle_request()) Handling RPC req@00000000f44a2faf pname:cluuid+ref:pid:xid:nid:opc:job mdt00_000:0+-99:3352:x1862456998072128:12345-0@lo:38:kworker/u8:1.0 00010000:02000000:0.0:1776178369.378387:0:6255:0:(ldlm_lib.c:1139:target_handle_connect()) lustre-MDT0000: Not available for connect from 0@lo (stopping) 00000100:00100000:0.0:1776178369.378424:0:6255:0:(service.c:2372:ptlrpc_server_handle_request()) Handled RPC req@00000000f44a2faf pname:cluuid+ref:pid:xid:nid:opc:job mdt00_000:0+-99:3352:x1862456998072128:12345-0@lo:38:kworker/u8:1.0 Request processed in 54us (150us total) trans 0 rc -19/-19 00000100:00100000:0.0:1776178369.378431:0:6255:0:(nrs_fifo.c:241:nrs_fifo_req_stop()) NRS stop fifo request from 12345-0@lo, seq: 6753 00000100:00080000:0.0:1776178369.378465:0:3352:0:(import.c:86:import_set_state_nolock()) 0000000047601a98 lustre-MDT0000_UUID: changing import state from CONNECTING to DISCONN 00000100:00080000:0.0:1776178369.378469:0:3352:0:(import.c:1429:ptlrpc_connect_interpret()) recovery of lustre-MDT0000_UUID on 192.168.204.103@tcp failed (-19) 00000100:00100000:0.0:1776178369.378472:0:3352:0:(client.c:2215:ptlrpc_check_set()) Completed RPC req@00000000dd1281fe pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:lustre-MDT0000-lwp-OST0000_UUID:3352:1862456998072128:0@lo:38:kworker/u8:1.0 00000100:00100000:0.0:1776178369.378528:0:3354:0:(client.c:1733:ptlrpc_send_new_req()) Sending RPC req@00000000e64148ab pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_01:lustre-MDT0001-lwp-OST0000_UUID:3354:1862456998072192:0@lo:400:kworker/u8:1.0 00000100:00100000:0.0:1776178369.378749:0:3354:0:(events.c:361:request_in_callback()) peer: 12345-0@lo (source: 12345-0@lo) 00000100:00100000:0.0:1776178369.378784:0:6257:0:(service.c:2152:ptlrpc_server_handle_req_in()) got req x1862456998072192 00000100:00100000:0.0:1776178369.378812:0:6257:0:(nrs_fifo.c:179:nrs_fifo_req_get()) NRS start fifo request from 12345-0@lo, seq: 1025 00000100:00100000:0.0:1776178369.378815:0:6257:0:(service.c:2324:ptlrpc_server_handle_request()) Handling RPC req@000000003b436fee pname:cluuid+ref:pid:xid:nid:opc:job mdt00_002:lustre-MDT0001-lwp-OST0000_UUID+5:3354:x1862456998072192:12345-0@lo:400:kworker/u8:1.0 00000100:00100000:0.0:1776178369.378852:0:6257:0:(service.c:2372:ptlrpc_server_handle_request()) Handled RPC req@000000003b436fee pname:cluuid+ref:pid:xid:nid:opc:job mdt00_002:lustre-MDT0001-lwp-OST0000_UUID+5:3354:x1862456998072192:12345-0@lo:400:kworker/u8:1.0 Request processed in 35us (104us total) trans 0 rc 0/0 00000100:00100000:0.0:1776178369.378858:0:6257:0:(nrs_fifo.c:241:nrs_fifo_req_stop()) NRS stop fifo request from 12345-0@lo, seq: 1025 00000100:00100000:0.0:1776178369.378883:0:3354:0:(client.c:2215:ptlrpc_check_set()) Completed RPC req@00000000e64148ab pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_01:lustre-MDT0001-lwp-OST0000_UUID:3354:1862456998072192:0@lo:400:kworker/u8:1.0 00000100:00080000:2.0:1776178369.382199:0:36:0:(pinger.c:251:ptlrpc_pinger_process_import()) lustre-MDT0000-lwp-OST0001_UUID->lustre-MDT0000_UUID: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00080000:2.0:1776178369.382216:0:36:0:(recover.c:57:ptlrpc_initiate_recovery()) lustre-MDT0000_UUID: starting recovery 00000100:00080000:2.0:1776178369.382220:0:36:0:(import.c:86:import_set_state_nolock()) 00000000dfc8db13 lustre-MDT0000_UUID: changing import state from DISCONN to CONNECTING 00000100:00080000:2.0:1776178369.382231:0:36:0:(import.c:534:import_select_connection()) lustre-MDT0000-lwp-OST0001: connect to NID 0@lo last attempt 1071 00000100:00080000:2.0:1776178369.382235:0:36:0:(import.c:581:import_select_connection()) lustre-MDT0000-lwp-OST0001: tried all connections, increasing latency to 21s 00000100:00080000:2.0:1776178369.382244:0:36:0:(import.c:615:import_select_connection()) lustre-MDT0000-lwp-OST0001: import 00000000dfc8db13 using connection 192.168.204.103@tcp/0@lo 00000100:00100000:2.0:1776178369.382308:0:36:0:(import.c:807:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5) req@0000000047458d9f x1862456998072256/t0(0) o38->lustre-MDT0000-lwp-OST0001@0@lo:12/10 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' 00000100:00080000:2.0:1776178369.382366:0:36:0:(pinger.c:251:ptlrpc_pinger_process_import()) lustre-MDT0001-mdtlov_UUID->lustre-MDT0000_UUID: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00080000:2.0:1776178369.382381:0:36:0:(recover.c:57:ptlrpc_initiate_recovery()) lustre-MDT0000_UUID: starting recovery 00000100:00080000:2.0:1776178369.382384:0:36:0:(import.c:86:import_set_state_nolock()) 00000000ba4f1821 lustre-MDT0000_UUID: changing import state from DISCONN to CONNECTING 00000100:00080000:2.0:1776178369.382387:0:36:0:(import.c:534:import_select_connection()) lustre-MDT0000-osp-MDT0001: connect to NID 0@lo last attempt 1071 00000100:00080000:2.0:1776178369.382389:0:36:0:(import.c:581:import_select_connection()) lustre-MDT0000-osp-MDT0001: tried all connections, increasing latency to 21s 00000100:00080000:2.0:1776178369.382392:0:36:0:(import.c:615:import_select_connection()) lustre-MDT0000-osp-MDT0001: import 00000000ba4f1821 using connection 192.168.204.103@tcp/0@lo 00000100:00100000:2.0:1776178369.382406:0:36:0:(import.c:807:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5) req@000000005927b2c3 x1862456998072384/t0(0) o38->lustre-MDT0000-osp-MDT0001@0@lo:24/4 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' 00000100:00080000:2.0:1776178369.382430:0:36:0:(pinger.c:251:ptlrpc_pinger_process_import()) lustre-MDT0000-lwp-MDT0001_UUID->lustre-MDT0000_UUID: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00080000:2.0:1776178369.382446:0:36:0:(recover.c:57:ptlrpc_initiate_recovery()) lustre-MDT0000_UUID: starting recovery 00000100:00080000:2.0:1776178369.382447:0:36:0:(import.c:86:import_set_state_nolock()) 00000000a7c55b1f lustre-MDT0000_UUID: changing import state from DISCONN to CONNECTING 00000100:00080000:2.0:1776178369.382451:0:36:0:(import.c:534:import_select_connection()) lustre-MDT0000-lwp-MDT0001: connect to NID 0@lo last attempt 1071 00000100:00080000:2.0:1776178369.382453:0:36:0:(import.c:581:import_select_connection()) lustre-MDT0000-lwp-MDT0001: tried all connections, increasing latency to 21s 00000100:00080000:2.0:1776178369.382456:0:36:0:(import.c:615:import_select_connection()) lustre-MDT0000-lwp-MDT0001: import 00000000a7c55b1f using connection 192.168.204.103@tcp/0@lo 00000100:00100000:2.0:1776178369.382468:0:36:0:(import.c:807:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5) req@0000000023210d1c x1862456998072576/t0(0) o38->lustre-MDT0000-lwp-MDT0001@0@lo:12/10 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' 00000100:00100000:2.0:1776178369.382507:0:3356:0:(client.c:1733:ptlrpc_send_new_req()) Sending RPC req@000000009d5112f6 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_03:lustre-MDT0001-mdtlov_UUID:3356:1862456998072448:0@lo:400:kworker/u8:1.0 00000100:00100000:2.0:1776178369.382605:0:3356:0:(events.c:361:request_in_callback()) peer: 12345-0@lo (source: 12345-0@lo) 00000100:00100000:2.0:1776178369.382648:0:3356:0:(ptlrpcd.c:415:ptlrpcd_check()) transfer 1 async RPCs [2->3] 00000100:00100000:2.0:1776178369.382654:0:3356:0:(client.c:1733:ptlrpc_send_new_req()) Sending RPC req@00000000b5702a7a pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_03:lustre-MDT0001-lwp-OST0001_UUID:3356:1862456998072320:0@lo:400:kworker/u8:1.0 00000100:00100000:2.0:1776178369.382683:0:3356:0:(events.c:361:request_in_callback()) peer: 12345-0@lo (source: 12345-0@lo) 00000100:00100000:2.0:1776178369.382723:0:20755:0:(service.c:2152:ptlrpc_server_handle_req_in()) got req x1862456998072448 00000100:00100000:2.0:1776178369.382771:0:20755:0:(nrs_fifo.c:179:nrs_fifo_req_get()) NRS start fifo request from 12345-0@lo, seq: 641 00000100:00100000:2.0:1776178369.382777:0:20755:0:(service.c:2324:ptlrpc_server_handle_request()) Handling RPC req@00000000a59bfc89 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_006:lustre-MDT0001-mdtlov_UUID+5:3356:x1862456998072448:12345-0@lo:400:kworker/u8:1.0 00000100:00100000:2.0:1776178369.382829:0:20755:0:(service.c:2372:ptlrpc_server_handle_request()) Handled RPC req@00000000a59bfc89 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_006:lustre-MDT0001-mdtlov_UUID+5:3356:x1862456998072448:12345-0@lo:400:kworker/u8:1.0 Request processed in 50us (225us total) trans 0 rc 0/0 00000100:00100000:2.0:1776178369.382837:0:20755:0:(nrs_fifo.c:241:nrs_fifo_req_stop()) NRS stop fifo request from 12345-0@lo, seq: 641 00000100:00100000:2.0:1776178369.382879:0:3354:0:(ptlrpcd.c:415:ptlrpcd_check()) transfer 1 async RPCs [0->1] 00000100:00100000:2.0:1776178369.382887:0:3354:0:(client.c:1733:ptlrpc_send_new_req()) Sending RPC req@0000000067671fc8 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_01:lustre-MDT0001-mdtlov_UUID:3354:1862456998072512:0@lo:400:kworker/u8:1.0 00000100:00100000:2.0:1776178369.382911:0:3354:0:(events.c:361:request_in_callback()) peer: 12345-0@lo (source: 12345-0@lo) 00000100:00100000:2.0:1776178369.382936:0:14440:0:(service.c:2152:ptlrpc_server_handle_req_in()) got req x1862456998072512 00000100:00100000:2.0:1776178369.382957:0:14440:0:(nrs_fifo.c:179:nrs_fifo_req_get()) NRS start fifo request from 12345-0@lo, seq: 642 00000100:00100000:2.0:1776178369.382960:0:14440:0:(service.c:2324:ptlrpc_server_handle_request()) Handling RPC req@00000000ecb64602 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_004:lustre-MDT0001-mdtlov_UUID+5:3354:x1862456998072512:12345-0@lo:400:kworker/u8:1.0 00000100:00100000:2.0:1776178369.382995:0:14440:0:(service.c:2372:ptlrpc_server_handle_request()) Handled RPC req@00000000ecb64602 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_004:lustre-MDT0001-mdtlov_UUID+5:3354:x1862456998072512:12345-0@lo:400:kworker/u8:1.0 Request processed in 34us (84us total) trans 0 rc 0/0 00000100:00100000:2.0:1776178369.383001:0:14440:0:(nrs_fifo.c:241:nrs_fifo_req_stop()) NRS stop fifo request from 12345-0@lo, seq: 642 00000100:00100000:2.0:1776178369.383043:0:3354:0:(client.c:2215:ptlrpc_check_set()) Completed RPC req@0000000067671fc8 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_01:lustre-MDT0001-mdtlov_UUID:3354:1862456998072512:0@lo:400:kworker/u8:1.0 00000100:00100000:2.0:1776178369.383109:0:3356:0:(client.c:2215:ptlrpc_check_set()) Completed RPC req@000000009d5112f6 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_03:lustre-MDT0001-mdtlov_UUID:3356:1862456998072448:0@lo:400:kworker/u8:1.0 00000100:00100000:2.0:1776178369.383130:0:6255:0:(service.c:2152:ptlrpc_server_handle_req_in()) got req x1862456998072320 00000100:00100000:2.0:1776178369.383162:0:6255:0:(nrs_fifo.c:179:nrs_fifo_req_get()) NRS start fifo request from 12345-0@lo, seq: 1026 00000100:00100000:2.0:1776178369.383166:0:6255:0:(service.c:2324:ptlrpc_server_handle_request()) Handling RPC req@00000000575d0558 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_000:lustre-MDT0001-lwp-OST0001_UUID+5:3356:x1862456998072320:12345-0@lo:400:kworker/u8:1.0 00000100:00100000:2.0:1776178369.383199:0:6255:0:(service.c:2372:ptlrpc_server_handle_request()) Handled RPC req@00000000575d0558 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_000:lustre-MDT0001-lwp-OST0001_UUID+5:3356:x1862456998072320:12345-0@lo:400:kworker/u8:1.0 Request processed in 32us (516us total) trans 0 rc 0/0 00000100:00100000:2.0:1776178369.383205:0:6255:0:(nrs_fifo.c:241:nrs_fifo_req_stop()) NRS stop fifo request from 12345-0@lo, seq: 1026 00000100:00100000:2.0:1776178369.383226:0:3356:0:(client.c:2215:ptlrpc_check_set()) Completed RPC req@00000000b5702a7a pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_03:lustre-MDT0001-lwp-OST0001_UUID:3356:1862456998072320:0@lo:400:kworker/u8:1.0 00000100:00100000:1.0:1776178369.383698:0:20137:0:(service.c:2372:ptlrpc_server_handle_request()) Handled RPC req@000000002efeec88 pname:cluuid+ref:pid:xid:nid:opc:job ll_mgs_0000:ac61f839-0e8e-4cac-9c59-08bcfbbad330+18:3353:x1862456998072064:12345-0@lo:400:kworker/u8:1.0 Request processed in 5568us (5816us total) trans 0 rc 0/0 00000100:00100000:1.0:1776178369.383718:0:20137:0:(nrs_fifo.c:241:nrs_fifo_req_stop()) NRS stop fifo request from 12345-0@lo, seq: 220 00000100:00100000:0.0:1776178369.388118:0:3352:0:(client.c:1733:ptlrpc_send_new_req()) Sending RPC req@0000000047458d9f pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:lustre-MDT0000-lwp-OST0001_UUID:3352:1862456998072256:0@lo:38:kworker/u8:1.0 00000100:00100000:0.0:1776178369.388214:0:3352:0:(events.c:361:request_in_callback()) peer: 12345-0@lo (source: 12345-0@lo) 00000100:00100000:0.0:1776178369.388249:0:3352:0:(client.c:1733:ptlrpc_send_new_req()) Sending RPC req@000000005927b2c3 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:lustre-MDT0001-mdtlov_UUID:3352:1862456998072384:0@lo:38:kworker/u8:1.0 00000100:00100000:0.0:1776178369.388287:0:3352:0:(events.c:361:request_in_callback()) peer: 12345-0@lo (source: 12345-0@lo) 00000100:00100000:0.0:1776178369.388318:0:3352:0:(client.c:1733:ptlrpc_send_new_req()) Sending RPC req@0000000023210d1c pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:lustre-MDT0000-lwp-MDT0001_UUID:3352:1862456998072576:0@lo:38:kworker/u8:1.0 00000100:00100000:0.0:1776178369.388339:0:3352:0:(events.c:361:request_in_callback()) peer: 12345-0@lo (source: 12345-0@lo) 00000100:00100000:0.0:1776178369.388402:0:6255:0:(service.c:2152:ptlrpc_server_handle_req_in()) got req x1862456998072256 00000100:00100000:0.0:1776178369.388443:0:6255:0:(service.c:2152:ptlrpc_server_handle_req_in()) got req x1862456998072576 00000100:00100000:0.0:1776178369.388465:0:8250:0:(nrs_fifo.c:179:nrs_fifo_req_get()) NRS start fifo request from 12345-0@lo, seq: 6754 00000100:00100000:0.0:1776178369.388471:0:8250:0:(service.c:2324:ptlrpc_server_handle_request()) Handling RPC req@00000000775bd6f2 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_003:0+-99:3352:x1862456998072256:12345-0@lo:38:kworker/u8:1.0 00010000:02000000:0.0:1776178369.388486:0:8250:0:(ldlm_lib.c:1139:target_handle_connect()) lustre-MDT0000: Not available for connect from 0@lo (stopping) 00000100:00100000:0.0:1776178369.388525:0:8250:0:(service.c:2372:ptlrpc_server_handle_request()) Handled RPC req@00000000775bd6f2 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_003:0+-99:3352:x1862456998072256:12345-0@lo:38:kworker/u8:1.0 Request processed in 54us (313us total) trans 0 rc -19/-19 00000100:00100000:0.0:1776178369.388531:0:8250:0:(nrs_fifo.c:241:nrs_fifo_req_stop()) NRS stop fifo request from 12345-0@lo, seq: 6754 00000100:00100000:1.0:1776178369.388732:0:6257:0:(nrs_fifo.c:179:nrs_fifo_req_get()) NRS start fifo request from 12345-0@lo, seq: 6755 00000100:00100000:1.0:1776178369.388738:0:6257:0:(service.c:2324:ptlrpc_server_handle_request()) Handling RPC req@0000000000167c64 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_002:0+-99:3352:x1862456998072576:12345-0@lo:38:kworker/u8:1.0 00010000:02000000:1.0:1776178369.388745:0:6257:0:(ldlm_lib.c:1139:target_handle_connect()) lustre-MDT0000: Not available for connect from 0@lo (stopping) 00000100:00100000:1.0:1776178369.388777:0:6257:0:(service.c:2372:ptlrpc_server_handle_request()) Handled RPC req@0000000000167c64 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_002:0+-99:3352:x1862456998072576:12345-0@lo:38:kworker/u8:1.0 Request processed in 38us (437us total) trans 0 rc -19/-19 00000100:00100000:1.0:1776178369.388782:0:6257:0:(nrs_fifo.c:241:nrs_fifo_req_stop()) NRS stop fifo request from 12345-0@lo, seq: 6755 00000100:00080000:1.0:1776178369.388840:0:3352:0:(import.c:86:import_set_state_nolock()) 00000000dfc8db13 lustre-MDT0000_UUID: changing import state from CONNECTING to DISCONN 00000100:00080000:1.0:1776178369.388855:0:3352:0:(import.c:1429:ptlrpc_connect_interpret()) recovery of lustre-MDT0000_UUID on 192.168.204.103@tcp failed (-19) 00000100:00100000:1.0:1776178369.388860:0:3352:0:(client.c:2215:ptlrpc_check_set()) Completed RPC req@0000000047458d9f pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:lustre-MDT0000-lwp-OST0001_UUID:3352:1862456998072256:0@lo:38:kworker/u8:1.0 00000100:00080000:1.0:1776178369.388871:0:3352:0:(import.c:86:import_set_state_nolock()) 00000000a7c55b1f lustre-MDT0000_UUID: changing import state from CONNECTING to DISCONN 00000100:00080000:1.0:1776178369.388874:0:3352:0:(import.c:1429:ptlrpc_connect_interpret()) recovery of lustre-MDT0000_UUID on 192.168.204.103@tcp failed (-19) 00000100:00100000:1.0:1776178369.388876:0:3352:0:(client.c:2215:ptlrpc_check_set()) Completed RPC req@0000000023210d1c pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:lustre-MDT0000-lwp-MDT0001_UUID:3352:1862456998072576:0@lo:38:kworker/u8:1.0 00000100:00100000:2.0:1776178369.389889:0:14128:0:(service.c:2152:ptlrpc_server_handle_req_in()) got req x1862456998072384 00000100:00100000:2.0:1776178369.389934:0:14128:0:(nrs_fifo.c:179:nrs_fifo_req_get()) NRS start fifo request from 12345-0@lo, seq: 740 00000100:00100000:2.0:1776178369.389939:0:14128:0:(service.c:2324:ptlrpc_server_handle_request()) Handling RPC req@00000000eafab700 pname:cluuid+ref:pid:xid:nid:opc:job mdt_out00_002:0+-99:3352:x1862456998072384:12345-0@lo:38:kworker/u8:1.0 00010000:02000000:2.0:1776178369.389952:0:14128:0:(ldlm_lib.c:1139:target_handle_connect()) lustre-MDT0000: Not available for connect from 0@lo (stopping) 00000100:00100000:2.0:1776178369.390007:0:14128:0:(service.c:2372:ptlrpc_server_handle_request()) Handled RPC req@00000000eafab700 pname:cluuid+ref:pid:xid:nid:opc:job mdt_out00_002:0+-99:3352:x1862456998072384:12345-0@lo:38:kworker/u8:1.0 Request processed in 67us (1719us total) trans 0 rc -19/-19 00000100:00100000:2.0:1776178369.390013:0:14128:0:(nrs_fifo.c:241:nrs_fifo_req_stop()) NRS stop fifo request from 12345-0@lo, seq: 740 00000100:00080000:1.0:1776178369.390273:0:3352:0:(import.c:86:import_set_state_nolock()) 00000000ba4f1821 lustre-MDT0000_UUID: changing import state from CONNECTING to DISCONN 00000100:00080000:1.0:1776178369.390280:0:3352:0:(import.c:1429:ptlrpc_connect_interpret()) recovery of lustre-MDT0000_UUID on 192.168.204.103@tcp failed (-19) 00000100:00100000:1.0:1776178369.390286:0:3352:0:(client.c:2215:ptlrpc_check_set()) Completed RPC req@000000005927b2c3 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:lustre-MDT0001-mdtlov_UUID:3352:1862456998072384:0@lo:38:kworker/u8:1.0 00000100:00100000:3.0:1776178369.393531:0:3353:0:(client.c:2215:ptlrpc_check_set()) Completed RPC req@000000005643ab1b pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_00:ac61f839-0e8e-4cac-9c59-08bcfbbad330:3353:1862456998072064:0@lo:400:kworker/u8:1.0 00010000:00010000:2.0:1776178370.016330:0:6245:0:(ldlm_lockd.c:2792:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:0.0:1776178370.912412:0:3354:0:(client.c:1733:ptlrpc_send_new_req()) Sending RPC req@00000000b5702a7a pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_01:lustre-MDT0001-mdtlov_UUID:3354:1862456998072640:0@lo:13:osp-pre-1-1.0 00000100:00100000:0.0:1776178370.912503:0:3354:0:(events.c:361:request_in_callback()) peer: 12345-0@lo (source: 12345-0@lo) 00000100:00100000:1.0:1776178370.912679:0:8231:0:(service.c:2152:ptlrpc_server_handle_req_in()) got req x1862456998072640 00000100:00100000:1.0:1776178370.912786:0:8231:0:(nrs_fifo.c:179:nrs_fifo_req_get()) NRS start fifo request from 12345-0@lo, seq: 553 00000100:00100000:1.0:1776178370.912791:0:8231:0:(service.c:2324:ptlrpc_server_handle_request()) Handling RPC req@00000000dab5abff pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_create00:lustre-MDT0001-mdtlov_UUID+5:3354:x1862456998072640:12345-0@lo:13:osp-pre-1-1.0 00002000:00000024:1.0:1776178370.912823:0:8231:0:(ofd_obd.c:740:ofd_statfs()) blocks cached 229376 granted 491746368 pending 0 free 3923689472 avail 3692167168 00000100:00100000:1.0:1776178370.912874:0:8231:0:(service.c:2372:ptlrpc_server_handle_request()) Handled RPC req@00000000dab5abff pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_create00:lustre-MDT0001-mdtlov_UUID+5:3354:x1862456998072640:12345-0@lo:13:osp-pre-1-1.0 Request processed in 81us (372us total) trans 0 rc 0/0 00000100:00100000:1.0:1776178370.912881:0:8231:0:(nrs_fifo.c:241:nrs_fifo_req_stop()) NRS stop fifo request from 12345-0@lo, seq: 553 00000100:00100000:0.0:1776178370.912946:0:3354:0:(client.c:2215:ptlrpc_check_set()) Completed RPC req@00000000b5702a7a pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_01:lustre-MDT0001-mdtlov_UUID:3354:1862456998072640:0@lo:13:osp-pre-1-1.0 00010000:00010000:2.0:1776178371.040572:0:9224:0:(ldlm_lockd.c:2792:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:3.0:1776178371.425848:0:3356:0:(ptlrpcd.c:415:ptlrpcd_check()) transfer 1 async RPCs [2->3] 00000100:00100000:3.0:1776178371.425871:0:3356:0:(client.c:1733:ptlrpc_send_new_req()) Sending RPC req@000000008aab9f17 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_03:lustre-MDT0001-mdtlov_UUID:3356:1862456998072704:0@lo:13:osp-pre-0-1.0 00000100:00100000:3.0:1776178371.425954:0:3356:0:(events.c:361:request_in_callback()) peer: 12345-0@lo (source: 12345-0@lo) 00000100:00100000:1.0:1776178371.426250:0:8231:0:(service.c:2152:ptlrpc_server_handle_req_in()) got req x1862456998072704 00000100:00100000:1.0:1776178371.426306:0:8231:0:(nrs_fifo.c:179:nrs_fifo_req_get()) NRS start fifo request from 12345-0@lo, seq: 554 00000100:00100000:1.0:1776178371.426312:0:8231:0:(service.c:2324:ptlrpc_server_handle_request()) Handling RPC req@0000000067c1ae4f pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_create00:lustre-MDT0001-mdtlov_UUID+5:3356:x1862456998072704:12345-0@lo:13:osp-pre-0-1.0 00002000:00000024:1.0:1776178371.426342:0:8231:0:(ofd_obd.c:740:ofd_statfs()) blocks cached 315392 granted 484883776 pending 0 free 3923689472 avail 3692167168 00000100:00100000:1.0:1776178371.426393:0:8231:0:(service.c:2372:ptlrpc_server_handle_request()) Handled RPC req@0000000067c1ae4f pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_create00:lustre-MDT0001-mdtlov_UUID+5:3356:x1862456998072704:12345-0@lo:13:osp-pre-0-1.0 Request processed in 81us (440us total) trans 0 rc 0/0 00000100:00100000:1.0:1776178371.426399:0:8231:0:(nrs_fifo.c:241:nrs_fifo_req_stop()) NRS stop fifo request from 12345-0@lo, seq: 554 00000100:00100000:3.0:1776178371.426488:0:3356:0:(client.c:2215:ptlrpc_check_set()) Completed RPC req@000000008aab9f17 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_03:lustre-MDT0001-mdtlov_UUID:3356:1862456998072704:0@lo:13:osp-pre-0-1.0 00010000:00010000:2.0:1776178372.064302:0:9225:0:(ldlm_lockd.c:2792:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:1.0:1776178372.901200:0:3347:0:(events.c:361:request_in_callback()) peer: 12345-192.168.204.3@tcp (source: 12345-192.168.204.3@tcp) 00000100:00100000:0.0:1776178372.901326:0:6256:0:(service.c:2152:ptlrpc_server_handle_req_in()) got req x1862456979119616 00000100:00100000:0.0:1776178372.901376:0:6256:0:(nrs_fifo.c:179:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.204.3@tcp, seq: 6756 00000100:00100000:0.0:1776178372.901384:0:6256:0:(service.c:2324:ptlrpc_server_handle_request()) Handling RPC req@00000000e0d60cbc pname:cluuid+ref:pid:xid:nid:opc:job mdt00_001:0+-99:2269:x1862456979119616:12345-192.168.204.3@tcp:38:kworker/u8:4.0 00010000:02000000:0.0:1776178372.901406:0:6256:0:(ldlm_lib.c:1139:target_handle_connect()) lustre-MDT0000: Not available for connect from 192.168.204.3@tcp (stopping) 00000100:00100000:0.0:1776178372.901547:0:6256:0:(service.c:2372:ptlrpc_server_handle_request()) Handled RPC req@00000000e0d60cbc pname:cluuid+ref:pid:xid:nid:opc:job mdt00_001:0+-99:2269:x1862456979119616:12345-192.168.204.3@tcp:38:kworker/u8:4.0 Request processed in 161us (356us total) trans 0 rc -19/-19 00000100:00100000:0.0:1776178372.901555:0:6256:0:(nrs_fifo.c:241:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.204.3@tcp, seq: 6756 00000100:00100000:2.0:1776178372.901780:0:3346:0:(events.c:361:request_in_callback()) peer: 12345-192.168.204.3@tcp (source: 12345-192.168.204.3@tcp) 00000100:00100000:2.0:1776178372.901816:0:3346:0:(events.c:361:request_in_callback()) peer: 12345-192.168.204.3@tcp (source: 12345-192.168.204.3@tcp) 00000100:00100000:3.0:1776178372.901883:0:20137:0:(service.c:2152:ptlrpc_server_handle_req_in()) got req x1862456979119552 00000100:00100000:2.0:1776178372.901904:0:3346:0:(events.c:361:request_in_callback()) peer: 12345-192.168.204.3@tcp (source: 12345-192.168.204.3@tcp) 00000100:00100000:3.0:1776178372.901926:0:20137:0:(nrs_fifo.c:179:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.204.3@tcp, seq: 221 00000100:00100000:3.0:1776178372.901935:0:20137:0:(service.c:2324:ptlrpc_server_handle_request()) Handling RPC req@0000000021ee30f5 pname:cluuid+ref:pid:xid:nid:opc:job ll_mgs_0000:5b5c66b1-1a2f-4651-8548-0c75b9a38a94+9:2272:x1862456979119552:12345-192.168.204.3@tcp:400:kworker/u8:4.0 00000100:00100000:2.0:1776178372.901939:0:3346:0:(events.c:361:request_in_callback()) peer: 12345-192.168.204.3@tcp (source: 12345-192.168.204.3@tcp) 00000100:00100000:3.0:1776178372.901985:0:20137:0:(service.c:2372:ptlrpc_server_handle_request()) Handled RPC req@0000000021ee30f5 pname:cluuid+ref:pid:xid:nid:opc:job ll_mgs_0000:5b5c66b1-1a2f-4651-8548-0c75b9a38a94+9:2272:x1862456979119552:12345-192.168.204.3@tcp:400:kworker/u8:4.0 Request processed in 52us (206us total) trans 0 rc 0/0 00000100:00100000:3.0:1776178372.901992:0:20137:0:(nrs_fifo.c:241:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.204.3@tcp, seq: 221 00000100:00100000:3.0:1776178372.902028:0:8250:0:(service.c:2152:ptlrpc_server_handle_req_in()) got req x1862456979119680 00000100:00100000:3.0:1776178372.902091:0:8250:0:(nrs_fifo.c:179:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.204.3@tcp, seq: 1027 00000100:00100000:3.0:1776178372.902096:0:8250:0:(service.c:2324:ptlrpc_server_handle_request()) Handling RPC req@0000000067671fc8 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_003:b50963bd-8eba-4432-9a54-8888d00e3fa1+6:2272:x1862456979119680:12345-192.168.204.3@tcp:400:kworker/u8:4.0 00000100:00100000:3.0:1776178372.902128:0:8250:0:(service.c:2372:ptlrpc_server_handle_request()) Handled RPC req@0000000067671fc8 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_003:b50963bd-8eba-4432-9a54-8888d00e3fa1+6:2272:x1862456979119680:12345-192.168.204.3@tcp:400:kworker/u8:4.0 Request processed in 32us (226us total) trans 0 rc 0/0 00000100:00100000:3.0:1776178372.902134:0:8250:0:(nrs_fifo.c:241:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.204.3@tcp, seq: 1027 00000100:00100000:0.0:1776178372.903817:0:20755:0:(service.c:2152:ptlrpc_server_handle_req_in()) got req x1862456979119808 00000100:00100000:3.0:1776178372.903817:0:14440:0:(service.c:2152:ptlrpc_server_handle_req_in()) got req x1862456979119744 00000100:00100000:3.0:1776178372.903882:0:14440:0:(nrs_fifo.c:179:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.204.3@tcp, seq: 643 00000100:00100000:3.0:1776178372.903888:0:14440:0:(service.c:2324:ptlrpc_server_handle_request()) Handling RPC req@000000009d5112f6 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_004:b50963bd-8eba-4432-9a54-8888d00e3fa1+8:2272:x1862456979119744:12345-192.168.204.3@tcp:400:kworker/u8:4.0 00000100:00100000:3.0:1776178372.903953:0:14440:0:(service.c:2372:ptlrpc_server_handle_request()) Handled RPC req@000000009d5112f6 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_004:b50963bd-8eba-4432-9a54-8888d00e3fa1+8:2272:x1862456979119744:12345-192.168.204.3@tcp:400:kworker/u8:4.0 Request processed in 65us (2137us total) trans 0 rc 0/0 00000100:00100000:3.0:1776178372.903960:0:14440:0:(nrs_fifo.c:241:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.204.3@tcp, seq: 643 00000100:00100000:0.0:1776178372.904720:0:20755:0:(nrs_fifo.c:179:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.204.3@tcp, seq: 644 00000100:00100000:0.0:1776178372.904726:0:20755:0:(service.c:2324:ptlrpc_server_handle_request()) Handling RPC req@00000000f0ef812c pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_006:b50963bd-8eba-4432-9a54-8888d00e3fa1+9:2272:x1862456979119808:12345-192.168.204.3@tcp:400:kworker/u8:4.0 00000100:00100000:0.0:1776178372.904988:0:20755:0:(service.c:2372:ptlrpc_server_handle_request()) Handled RPC req@00000000f0ef812c pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_006:b50963bd-8eba-4432-9a54-8888d00e3fa1+9:2272:x1862456979119808:12345-192.168.204.3@tcp:400:kworker/u8:4.0 Request processed in 260us (3053us total) trans 0 rc 0/0 00000100:00100000:0.0:1776178372.904996:0:20755:0:(nrs_fifo.c:241:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.204.3@tcp, seq: 644 00010000:00010000:3.0:1776178373.089902:0:6244:0:(ldlm_lockd.c:2792:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00010000:00010000:2.0:1776178374.114660:0:6245:0:(ldlm_lockd.c:2792:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00080000:2.0:1776178374.496584:0:36:0:(pinger.c:251:ptlrpc_pinger_process_import()) lustre-MDT0000-lwp-OST0000_UUID->lustre-MDT0000_UUID: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00080000:2.0:1776178374.496594:0:36:0:(recover.c:57:ptlrpc_initiate_recovery()) lustre-MDT0000_UUID: starting recovery 00000100:00080000:2.0:1776178374.496598:0:36:0:(import.c:86:import_set_state_nolock()) 0000000047601a98 lustre-MDT0000_UUID: changing import state from DISCONN to CONNECTING 00000100:00080000:2.0:1776178374.496610:0:36:0:(import.c:534:import_select_connection()) lustre-MDT0000-lwp-OST0000: connect to NID 0@lo last attempt 1076 00000100:00080000:2.0:1776178374.496614:0:36:0:(import.c:581:import_select_connection()) lustre-MDT0000-lwp-OST0000: tried all connections, increasing latency to 21s 00000100:00080000:2.0:1776178374.496624:0:36:0:(import.c:615:import_select_connection()) lustre-MDT0000-lwp-OST0000: import 0000000047601a98 using connection 192.168.204.103@tcp/0@lo 00000100:00100000:2.0:1776178374.496655:0:36:0:(import.c:807:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5) req@0000000072865dff x1862456998072832/t0(0) o38->lustre-MDT0000-lwp-OST0000@0@lo:12/10 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' 00000100:00080000:2.0:1776178374.496700:0:36:0:(pinger.c:251:ptlrpc_pinger_process_import()) lustre-MDT0000-lwp-OST0001_UUID->lustre-MDT0000_UUID: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00080000:2.0:1776178374.496704:0:36:0:(recover.c:57:ptlrpc_initiate_recovery()) lustre-MDT0000_UUID: starting recovery 00000100:00080000:2.0:1776178374.496705:0:36:0:(import.c:86:import_set_state_nolock()) 00000000dfc8db13 lustre-MDT0000_UUID: changing import state from DISCONN to CONNECTING 00000100:00080000:2.0:1776178374.496709:0:36:0:(import.c:534:import_select_connection()) lustre-MDT0000-lwp-OST0001: connect to NID 0@lo last attempt 1076 00000100:00080000:2.0:1776178374.496711:0:36:0:(import.c:581:import_select_connection()) lustre-MDT0000-lwp-OST0001: tried all connections, increasing latency to 21s 00000100:00080000:2.0:1776178374.496715:0:36:0:(import.c:615:import_select_connection()) lustre-MDT0000-lwp-OST0001: import 00000000dfc8db13 using connection 192.168.204.103@tcp/0@lo 00000100:00100000:2.0:1776178374.496724:0:36:0:(import.c:807:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5) req@00000000f95e9da5 x1862456998072960/t0(0) o38->lustre-MDT0000-lwp-OST0001@0@lo:12/10 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' 00000100:00080000:2.0:1776178374.496739:0:36:0:(pinger.c:251:ptlrpc_pinger_process_import()) lustre-MDT0001-mdtlov_UUID->lustre-MDT0000_UUID: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00080000:2.0:1776178374.496742:0:36:0:(recover.c:57:ptlrpc_initiate_recovery()) lustre-MDT0000_UUID: starting recovery 00000100:00080000:2.0:1776178374.496743:0:36:0:(import.c:86:import_set_state_nolock()) 00000000ba4f1821 lustre-MDT0000_UUID: changing import state from DISCONN to CONNECTING 00000100:00080000:2.0:1776178374.496745:0:36:0:(import.c:534:import_select_connection()) lustre-MDT0000-osp-MDT0001: connect to NID 0@lo last attempt 1076 00000100:00080000:2.0:1776178374.496748:0:36:0:(import.c:581:import_select_connection()) lustre-MDT0000-osp-MDT0001: tried all connections, increasing latency to 21s 00000100:00080000:2.0:1776178374.496751:0:36:0:(import.c:615:import_select_connection()) lustre-MDT0000-osp-MDT0001: import 00000000ba4f1821 using connection 192.168.204.103@tcp/0@lo 00000100:00100000:2.0:1776178374.496764:0:36:0:(import.c:807:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5) req@000000005643ab1b x1862456998073088/t0(0) o38->lustre-MDT0000-osp-MDT0001@0@lo:24/4 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' 00000100:00080000:2.0:1776178374.496784:0:36:0:(pinger.c:251:ptlrpc_pinger_process_import()) lustre-MDT0000-lwp-MDT0001_UUID->lustre-MDT0000_UUID: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00080000:2.0:1776178374.496787:0:36:0:(recover.c:57:ptlrpc_initiate_recovery()) lustre-MDT0000_UUID: starting recovery 00000100:00080000:2.0:1776178374.496789:0:36:0:(import.c:86:import_set_state_nolock()) 00000000a7c55b1f lustre-MDT0000_UUID: changing import state from DISCONN to CONNECTING 00000100:00080000:2.0:1776178374.496791:0:36:0:(import.c:534:import_select_connection()) lustre-MDT0000-lwp-MDT0001: connect to NID 0@lo last attempt 1076 00000100:00080000:2.0:1776178374.496793:0:36:0:(import.c:581:import_select_connection()) lustre-MDT0000-lwp-MDT0001: tried all connections, increasing latency to 21s 00000100:00080000:2.0:1776178374.496796:0:36:0:(import.c:615:import_select_connection()) lustre-MDT0000-lwp-MDT0001: import 00000000a7c55b1f using connection 192.168.204.103@tcp/0@lo 00000100:00100000:2.0:1776178374.496804:0:36:0:(import.c:807:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5) req@0000000047458d9f x1862456998073280/t0(0) o38->lustre-MDT0000-lwp-MDT0001@0@lo:12/10 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' 00000100:00100000:0.0:1776178374.496953:0:3355:0:(client.c:1733:ptlrpc_send_new_req()) Sending RPC req@000000005927b2c3 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_02:lustre-MDT0001-mdtlov_UUID:3355:1862456998073152:0@lo:400:kworker/u8:1.0 00000100:00100000:1.0:1776178374.497042:0:3352:0:(client.c:1733:ptlrpc_send_new_req()) Sending RPC req@0000000072865dff pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:lustre-MDT0000-lwp-OST0000_UUID:3352:1862456998072832:0@lo:38:kworker/u8:1.0 00000100:00100000:1.0:1776178374.497114:0:3352:0:(events.c:361:request_in_callback()) peer: 12345-0@lo (source: 12345-0@lo) 00000100:00100000:1.0:1776178374.497155:0:3352:0:(client.c:1733:ptlrpc_send_new_req()) Sending RPC req@00000000f95e9da5 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:lustre-MDT0000-lwp-OST0001_UUID:3352:1862456998072960:0@lo:38:kworker/u8:1.0 00000100:00100000:0.0:1776178374.497168:0:3355:0:(events.c:361:request_in_callback()) peer: 12345-0@lo (source: 12345-0@lo) 00000100:00100000:1.0:1776178374.497182:0:3352:0:(events.c:361:request_in_callback()) peer: 12345-0@lo (source: 12345-0@lo) 00000100:00100000:1.0:1776178374.497198:0:3352:0:(client.c:1733:ptlrpc_send_new_req()) Sending RPC req@000000005643ab1b pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:lustre-MDT0001-mdtlov_UUID:3352:1862456998073088:0@lo:38:kworker/u8:1.0 00000100:00100000:0.0:1776178374.497201:0:3355:0:(ptlrpcd.c:415:ptlrpcd_check()) transfer 2 async RPCs [3->2] 00000100:00100000:0.0:1776178374.497209:0:3355:0:(client.c:1733:ptlrpc_send_new_req()) Sending RPC req@00000000df370bee pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_02:ac61f839-0e8e-4cac-9c59-08bcfbbad330:3355:1862456998072768:0@lo:400:kworker/u8:1.0 00000100:00100000:1.0:1776178374.497225:0:3352:0:(events.c:361:request_in_callback()) peer: 12345-0@lo (source: 12345-0@lo) 00000100:00100000:1.0:1776178374.497248:0:3352:0:(client.c:1733:ptlrpc_send_new_req()) Sending RPC req@0000000047458d9f pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:lustre-MDT0000-lwp-MDT0001_UUID:3352:1862456998073280:0@lo:38:kworker/u8:1.0 00000100:00100000:0.0:1776178374.497251:0:3355:0:(events.c:361:request_in_callback()) peer: 12345-0@lo (source: 12345-0@lo) 00000100:00100000:0.0:1776178374.497269:0:3355:0:(client.c:1733:ptlrpc_send_new_req()) Sending RPC req@0000000023210d1c pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_02:lustre-MDT0001-mdtlov_UUID:3355:1862456998073216:0@lo:400:kworker/u8:1.0 00000100:00100000:1.0:1776178374.497280:0:3352:0:(events.c:361:request_in_callback()) peer: 12345-0@lo (source: 12345-0@lo) 00000100:00100000:1.0:1776178374.497317:0:6255:0:(service.c:2152:ptlrpc_server_handle_req_in()) got req x1862456998072832 00000100:00100000:1.0:1776178374.497343:0:6255:0:(service.c:2152:ptlrpc_server_handle_req_in()) got req x1862456998072960 00000100:00100000:1.0:1776178374.497353:0:6255:0:(service.c:2152:ptlrpc_server_handle_req_in()) got req x1862456998073280 00000100:00100000:1.0:1776178374.497363:0:6255:0:(nrs_fifo.c:179:nrs_fifo_req_get()) NRS start fifo request from 12345-0@lo, seq: 6757 00000100:00100000:1.0:1776178374.497368:0:6255:0:(service.c:2324:ptlrpc_server_handle_request()) Handling RPC req@000000002a321d27 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_000:0+-99:3352:x1862456998072832:12345-0@lo:38:kworker/u8:1.0 00010000:02000000:1.0:1776178374.497382:0:6255:0:(ldlm_lib.c:1139:target_handle_connect()) lustre-MDT0000: Not available for connect from 0@lo (stopping) 00000100:00100000:1.0:1776178374.497428:0:6255:0:(service.c:2372:ptlrpc_server_handle_request()) Handled RPC req@000000002a321d27 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_000:0+-99:3352:x1862456998072832:12345-0@lo:38:kworker/u8:1.0 Request processed in 59us (316us total) trans 0 rc -19/-19 00000100:00100000:1.0:1776178374.497434:0:6255:0:(nrs_fifo.c:241:nrs_fifo_req_stop()) NRS stop fifo request from 12345-0@lo, seq: 6757 00000100:00100000:1.0:1776178374.497462:0:6255:0:(nrs_fifo.c:179:nrs_fifo_req_get()) NRS start fifo request from 12345-0@lo, seq: 6758 00000100:00100000:1.0:1776178374.497465:0:6255:0:(service.c:2324:ptlrpc_server_handle_request()) Handling RPC req@0000000007880806 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_000:0+-99:3352:x1862456998072960:12345-0@lo:38:kworker/u8:1.0 00010000:02000000:1.0:1776178374.497471:0:6255:0:(ldlm_lib.c:1139:target_handle_connect()) lustre-MDT0000: Not available for connect from 0@lo (stopping) 00000100:00100000:1.0:1776178374.497491:0:6255:0:(service.c:2372:ptlrpc_server_handle_request()) Handled RPC req@0000000007880806 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_000:0+-99:3352:x1862456998072960:12345-0@lo:38:kworker/u8:1.0 Request processed in 25us (311us total) trans 0 rc -19/-19 00000100:00100000:1.0:1776178374.497497:0:6255:0:(nrs_fifo.c:241:nrs_fifo_req_stop()) NRS stop fifo request from 12345-0@lo, seq: 6758 00000100:00100000:1.0:1776178374.497505:0:6255:0:(nrs_fifo.c:179:nrs_fifo_req_get()) NRS start fifo request from 12345-0@lo, seq: 6759 00000100:00100000:1.0:1776178374.497508:0:6255:0:(service.c:2324:ptlrpc_server_handle_request()) Handling RPC req@000000009e03ef2c pname:cluuid+ref:pid:xid:nid:opc:job mdt00_000:0+-99:3352:x1862456998073280:12345-0@lo:38:kworker/u8:1.0 00010000:02000000:1.0:1776178374.497513:0:6255:0:(ldlm_lib.c:1139:target_handle_connect()) lustre-MDT0000: Not available for connect from 0@lo (stopping) 00000100:00100000:1.0:1776178374.497529:0:6255:0:(service.c:2372:ptlrpc_server_handle_request()) Handled RPC req@000000009e03ef2c pname:cluuid+ref:pid:xid:nid:opc:job mdt00_000:0+-99:3352:x1862456998073280:12345-0@lo:38:kworker/u8:1.0 Request processed in 20us (250us total) trans 0 rc -19/-19 00000100:00100000:1.0:1776178374.497533:0:6255:0:(nrs_fifo.c:241:nrs_fifo_req_stop()) NRS stop fifo request from 12345-0@lo, seq: 6759 00000100:00080000:1.0:1776178374.497571:0:3352:0:(import.c:86:import_set_state_nolock()) 0000000047601a98 lustre-MDT0000_UUID: changing import state from CONNECTING to DISCONN 00000100:00080000:1.0:1776178374.497576:0:3352:0:(import.c:1429:ptlrpc_connect_interpret()) recovery of lustre-MDT0000_UUID on 192.168.204.103@tcp failed (-19) 00000100:00100000:1.0:1776178374.497579:0:3352:0:(client.c:2215:ptlrpc_check_set()) Completed RPC req@0000000072865dff pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:lustre-MDT0000-lwp-OST0000_UUID:3352:1862456998072832:0@lo:38:kworker/u8:1.0 00000100:00080000:1.0:1776178374.497588:0:3352:0:(import.c:86:import_set_state_nolock()) 00000000dfc8db13 lustre-MDT0000_UUID: changing import state from CONNECTING to DISCONN 00000100:00080000:1.0:1776178374.497590:0:3352:0:(import.c:1429:ptlrpc_connect_interpret()) recovery of lustre-MDT0000_UUID on 192.168.204.103@tcp failed (-19) 00000100:00100000:1.0:1776178374.497592:0:3352:0:(client.c:2215:ptlrpc_check_set()) Completed RPC req@00000000f95e9da5 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:lustre-MDT0000-lwp-OST0001_UUID:3352:1862456998072960:0@lo:38:kworker/u8:1.0 00000100:00080000:1.0:1776178374.497603:0:3352:0:(import.c:86:import_set_state_nolock()) 00000000a7c55b1f lustre-MDT0000_UUID: changing import state from CONNECTING to DISCONN 00000100:00080000:1.0:1776178374.497606:0:3352:0:(import.c:1429:ptlrpc_connect_interpret()) recovery of lustre-MDT0000_UUID on 192.168.204.103@tcp failed (-19) 00000100:00100000:1.0:1776178374.497608:0:3352:0:(client.c:2215:ptlrpc_check_set()) Completed RPC req@0000000047458d9f pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:lustre-MDT0000-lwp-MDT0001_UUID:3352:1862456998073280:0@lo:38:kworker/u8:1.0 00000100:00100000:1.0:1776178374.497661:0:15213:0:(service.c:2152:ptlrpc_server_handle_req_in()) got req x1862456998073088 00000100:00100000:1.0:1776178374.497679:0:15213:0:(nrs_fifo.c:179:nrs_fifo_req_get()) NRS start fifo request from 12345-0@lo, seq: 741 00000100:00100000:1.0:1776178374.497683:0:15213:0:(service.c:2324:ptlrpc_server_handle_request()) Handling RPC req@000000007cbab86b pname:cluuid+ref:pid:xid:nid:opc:job mdt_out00_003:0+-99:3352:x1862456998073088:12345-0@lo:38:kworker/u8:1.0 00010000:02000000:1.0:1776178374.497689:0:15213:0:(ldlm_lib.c:1139:target_handle_connect()) lustre-MDT0000: Not available for connect from 0@lo (stopping) 00000100:00100000:1.0:1776178374.497715:0:15213:0:(service.c:2372:ptlrpc_server_handle_request()) Handled RPC req@000000007cbab86b pname:cluuid+ref:pid:xid:nid:opc:job mdt_out00_003:0+-99:3352:x1862456998073088:12345-0@lo:38:kworker/u8:1.0 Request processed in 32us (493us total) trans 0 rc -19/-19 00000100:00100000:1.0:1776178374.497721:0:15213:0:(nrs_fifo.c:241:nrs_fifo_req_stop()) NRS stop fifo request from 12345-0@lo, seq: 741 00000100:00100000:1.0:1776178374.497737:0:20759:0:(service.c:2152:ptlrpc_server_handle_req_in()) got req x1862456998073152 00000100:00100000:1.0:1776178374.497762:0:20759:0:(nrs_fifo.c:179:nrs_fifo_req_get()) NRS start fifo request from 12345-0@lo, seq: 645 00000100:00100000:1.0:1776178374.497765:0:20759:0:(service.c:2324:ptlrpc_server_handle_request()) Handling RPC req@0000000083bc4c0f pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_010:lustre-MDT0001-mdtlov_UUID+5:3355:x1862456998073152:12345-0@lo:400:kworker/u8:1.0 00000100:00100000:1.0:1776178374.497798:0:20759:0:(service.c:2372:ptlrpc_server_handle_request()) Handled RPC req@0000000083bc4c0f pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_010:lustre-MDT0001-mdtlov_UUID+5:3355:x1862456998073152:12345-0@lo:400:kworker/u8:1.0 Request processed in 32us (632us total) trans 0 rc 0/0 00000100:00100000:1.0:1776178374.497803:0:20759:0:(nrs_fifo.c:241:nrs_fifo_req_stop()) NRS stop fifo request from 12345-0@lo, seq: 645 00000100:00100000:1.0:1776178374.497818:0:3354:0:(client.c:1733:ptlrpc_send_new_req()) Sending RPC req@00000000b5702a7a pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_01:lustre-MDT0001-lwp-OST0001_UUID:3354:1862456998073024:0@lo:400:kworker/u8:1.0 00000100:00100000:1.0:1776178374.497839:0:3354:0:(events.c:361:request_in_callback()) peer: 12345-0@lo (source: 12345-0@lo) 00000100:00100000:1.0:1776178374.497852:0:3354:0:(ptlrpcd.c:415:ptlrpcd_check()) transfer 1 async RPCs [0->1] 00000100:00100000:1.0:1776178374.497873:0:3354:0:(client.c:1733:ptlrpc_send_new_req()) Sending RPC req@000000009e7a8bb5 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_01:lustre-MDT0001-lwp-OST0000_UUID:3354:1862456998072896:0@lo:400:kworker/u8:1.0 00000100:00100000:1.0:1776178374.497896:0:3354:0:(events.c:361:request_in_callback()) peer: 12345-0@lo (source: 12345-0@lo) 00000100:00100000:1.0:1776178374.497919:0:6257:0:(service.c:2152:ptlrpc_server_handle_req_in()) got req x1862456998073024 00000100:00100000:1.0:1776178374.497939:0:6257:0:(service.c:2152:ptlrpc_server_handle_req_in()) got req x1862456998072896 00000100:00100000:1.0:1776178374.497949:0:6257:0:(nrs_fifo.c:179:nrs_fifo_req_get()) NRS start fifo request from 12345-0@lo, seq: 1028 00000100:00100000:1.0:1776178374.497952:0:6257:0:(service.c:2324:ptlrpc_server_handle_request()) Handling RPC req@000000001ff33da0 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_002:lustre-MDT0001-lwp-OST0001_UUID+5:3354:x1862456998073024:12345-0@lo:400:kworker/u8:1.0 00000100:00100000:1.0:1776178374.497985:0:6257:0:(service.c:2372:ptlrpc_server_handle_request()) Handled RPC req@000000001ff33da0 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_002:lustre-MDT0001-lwp-OST0001_UUID+5:3354:x1862456998073024:12345-0@lo:400:kworker/u8:1.0 Request processed in 32us (146us total) trans 0 rc 0/0 00000100:00100000:1.0:1776178374.497991:0:6257:0:(nrs_fifo.c:241:nrs_fifo_req_stop()) NRS stop fifo request from 12345-0@lo, seq: 1028 00000100:00100000:1.0:1776178374.498004:0:6257:0:(nrs_fifo.c:179:nrs_fifo_req_get()) NRS start fifo request from 12345-0@lo, seq: 1029 00000100:00100000:1.0:1776178374.498008:0:6257:0:(service.c:2324:ptlrpc_server_handle_request()) Handling RPC req@00000000fb58fc15 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_002:lustre-MDT0001-lwp-OST0000_UUID+5:3354:x1862456998072896:12345-0@lo:400:kworker/u8:1.0 00000100:00100000:1.0:1776178374.498033:0:6257:0:(service.c:2372:ptlrpc_server_handle_request()) Handled RPC req@00000000fb58fc15 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_002:lustre-MDT0001-lwp-OST0000_UUID+5:3354:x1862456998072896:12345-0@lo:400:kworker/u8:1.0 Request processed in 24us (137us total) trans 0 rc 0/0 00000100:00100000:1.0:1776178374.498038:0:6257:0:(nrs_fifo.c:241:nrs_fifo_req_stop()) NRS stop fifo request from 12345-0@lo, seq: 1029 00000100:00100000:1.0:1776178374.498091:0:3354:0:(client.c:2215:ptlrpc_check_set()) Completed RPC req@000000009e7a8bb5 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_01:lustre-MDT0001-lwp-OST0000_UUID:3354:1862456998072896:0@lo:400:kworker/u8:1.0 00000100:00100000:1.0:1776178374.498103:0:3354:0:(client.c:2215:ptlrpc_check_set()) Completed RPC req@00000000b5702a7a pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_01:lustre-MDT0001-lwp-OST0001_UUID:3354:1862456998073024:0@lo:400:kworker/u8:1.0 00000100:00080000:1.0:1776178374.498126:0:3352:0:(import.c:86:import_set_state_nolock()) 00000000ba4f1821 lustre-MDT0000_UUID: changing import state from CONNECTING to DISCONN 00000100:00080000:1.0:1776178374.498129:0:3352:0:(import.c:1429:ptlrpc_connect_interpret()) recovery of lustre-MDT0000_UUID on 192.168.204.103@tcp failed (-19) 00000100:00100000:1.0:1776178374.498132:0:3352:0:(client.c:2215:ptlrpc_check_set()) Completed RPC req@000000005643ab1b pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:lustre-MDT0001-mdtlov_UUID:3352:1862456998073088:0@lo:38:kworker/u8:1.0 00000100:00100000:0.0:1776178374.498197:0:3355:0:(events.c:361:request_in_callback()) peer: 12345-0@lo (source: 12345-0@lo) 00000100:00100000:0.0:1776178374.498227:0:3355:0:(client.c:2215:ptlrpc_check_set()) Completed RPC req@000000005927b2c3 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_02:lustre-MDT0001-mdtlov_UUID:3355:1862456998073152:0@lo:400:kworker/u8:1.0 00000100:00100000:3.0:1776178374.498547:0:20137:0:(service.c:2152:ptlrpc_server_handle_req_in()) got req x1862456998072768 00000100:00100000:3.0:1776178374.498598:0:20137:0:(nrs_fifo.c:179:nrs_fifo_req_get()) NRS start fifo request from 12345-0@lo, seq: 222 00000100:00100000:3.0:1776178374.498602:0:20137:0:(service.c:2324:ptlrpc_server_handle_request()) Handling RPC req@000000006efdb84e pname:cluuid+ref:pid:xid:nid:opc:job ll_mgs_0000:ac61f839-0e8e-4cac-9c59-08bcfbbad330+18:3355:x1862456998072768:12345-0@lo:400:kworker/u8:1.0 00000100:00100000:3.0:1776178374.498645:0:20137:0:(service.c:2372:ptlrpc_server_handle_request()) Handled RPC req@000000006efdb84e pname:cluuid+ref:pid:xid:nid:opc:job ll_mgs_0000:ac61f839-0e8e-4cac-9c59-08bcfbbad330+18:3355:x1862456998072768:12345-0@lo:400:kworker/u8:1.0 Request processed in 42us (1397us total) trans 0 rc 0/0 00000100:00100000:3.0:1776178374.498651:0:20137:0:(nrs_fifo.c:241:nrs_fifo_req_stop()) NRS stop fifo request from 12345-0@lo, seq: 222 00000100:00100000:3.0:1776178374.498689:0:3355:0:(client.c:2215:ptlrpc_check_set()) Completed RPC req@00000000df370bee pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_02:ac61f839-0e8e-4cac-9c59-08bcfbbad330:3355:1862456998072768:0@lo:400:kworker/u8:1.0 00000100:00100000:1.0:1776178374.501212:0:20755:0:(service.c:2152:ptlrpc_server_handle_req_in()) got req x1862456998073216 00000100:00100000:1.0:1776178374.501313:0:20755:0:(nrs_fifo.c:179:nrs_fifo_req_get()) NRS start fifo request from 12345-0@lo, seq: 646 00000100:00100000:1.0:1776178374.501319:0:20755:0:(service.c:2324:ptlrpc_server_handle_request()) Handling RPC req@000000008aab9f17 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_006:lustre-MDT0001-mdtlov_UUID+5:3355:x1862456998073216:12345-0@lo:400:kworker/u8:1.0 00000100:00100000:1.0:1776178374.502127:0:20755:0:(service.c:2372:ptlrpc_server_handle_request()) Handled RPC req@000000008aab9f17 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_006:lustre-MDT0001-mdtlov_UUID+5:3355:x1862456998073216:12345-0@lo:400:kworker/u8:1.0 Request processed in 805us (3928us total) trans 0 rc 0/0 00000100:00100000:1.0:1776178374.502137:0:20755:0:(nrs_fifo.c:241:nrs_fifo_req_stop()) NRS stop fifo request from 12345-0@lo, seq: 646 00000100:00100000:3.0:1776178374.507260:0:3355:0:(client.c:2215:ptlrpc_check_set()) Completed RPC req@0000000023210d1c pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_02:lustre-MDT0001-mdtlov_UUID:3355:1862456998073216:0@lo:400:kworker/u8:1.0 00010000:00010000:2.0:1776178375.140011:0:9224:0:(ldlm_lockd.c:2792:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:1.0:1776178376.032665:0:3354:0:(ptlrpcd.c:415:ptlrpcd_check()) transfer 1 async RPCs [0->1] 00000100:00100000:1.0:1776178376.032695:0:3354:0:(client.c:1733:ptlrpc_send_new_req()) Sending RPC req@00000000e64148ab pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_01:lustre-MDT0001-mdtlov_UUID:3354:1862456998073344:0@lo:13:osp-pre-1-1.0 00000100:00100000:1.0:1776178376.032783:0:3354:0:(events.c:361:request_in_callback()) peer: 12345-0@lo (source: 12345-0@lo) 00000100:00100000:2.0:1776178376.033011:0:9554:0:(service.c:2152:ptlrpc_server_handle_req_in()) got req x1862456998073344 00000100:00100000:2.0:1776178376.033090:0:9554:0:(nrs_fifo.c:179:nrs_fifo_req_get()) NRS start fifo request from 12345-0@lo, seq: 555 00000100:00100000:2.0:1776178376.033097:0:9554:0:(service.c:2324:ptlrpc_server_handle_request()) Handling RPC req@0000000004c8f27c pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_create00:lustre-MDT0001-mdtlov_UUID+5:3354:x1862456998073344:12345-0@lo:13:osp-pre-1-1.0 00002000:00000024:2.0:1776178376.033129:0:9554:0:(ofd_obd.c:740:ofd_statfs()) blocks cached 229376 granted 491746368 pending 0 free 3923689472 avail 3692167168 00000100:00100000:2.0:1776178376.033192:0:9554:0:(service.c:2372:ptlrpc_server_handle_request()) Handled RPC req@0000000004c8f27c pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_create00:lustre-MDT0001-mdtlov_UUID+5:3354:x1862456998073344:12345-0@lo:13:osp-pre-1-1.0 Request processed in 94us (409us total) trans 0 rc 0/0 00000100:00100000:2.0:1776178376.033198:0:9554:0:(nrs_fifo.c:241:nrs_fifo_req_stop()) NRS stop fifo request from 12345-0@lo, seq: 555 00000100:00100000:0.0:1776178376.033489:0:3354:0:(client.c:2215:ptlrpc_check_set()) Completed RPC req@00000000e64148ab pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_01:lustre-MDT0001-mdtlov_UUID:3354:1862456998073344:0@lo:13:osp-pre-1-1.0 00010000:00010000:2.0:1776178376.163831:0:9225:0:(ldlm_lockd.c:2792:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:3.0:1776178376.544868:0:3353:0:(ptlrpcd.c:415:ptlrpcd_check()) transfer 1 async RPCs [1->0] 00000100:00100000:3.0:1776178376.544895:0:3353:0:(client.c:1733:ptlrpc_send_new_req()) Sending RPC req@0000000063c20b43 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_00:lustre-MDT0001-mdtlov_UUID:3353:1862456998073408:0@lo:13:osp-pre-0-1.0 00000100:00100000:3.0:1776178376.544986:0:3353:0:(events.c:361:request_in_callback()) peer: 12345-0@lo (source: 12345-0@lo) 00000100:00100000:2.0:1776178376.545103:0:8231:0:(service.c:2152:ptlrpc_server_handle_req_in()) got req x1862456998073408 00000100:00100000:2.0:1776178376.545159:0:8231:0:(nrs_fifo.c:179:nrs_fifo_req_get()) NRS start fifo request from 12345-0@lo, seq: 556 00000100:00100000:2.0:1776178376.545164:0:8231:0:(service.c:2324:ptlrpc_server_handle_request()) Handling RPC req@000000008f48ae7c pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_create00:lustre-MDT0001-mdtlov_UUID+5:3353:x1862456998073408:12345-0@lo:13:osp-pre-0-1.0 00002000:00000024:2.0:1776178376.545193:0:8231:0:(ofd_obd.c:740:ofd_statfs()) blocks cached 315392 granted 484883776 pending 0 free 3923689472 avail 3692167168 00000100:00100000:2.0:1776178376.545235:0:8231:0:(service.c:2372:ptlrpc_server_handle_request()) Handled RPC req@000000008f48ae7c pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_create00:lustre-MDT0001-mdtlov_UUID+5:3353:x1862456998073408:12345-0@lo:13:osp-pre-0-1.0 Request processed in 71us (251us total) trans 0 rc 0/0 00000100:00100000:2.0:1776178376.545243:0:8231:0:(nrs_fifo.c:241:nrs_fifo_req_stop()) NRS stop fifo request from 12345-0@lo, seq: 556 00000100:00100000:3.0:1776178376.545404:0:3353:0:(client.c:2215:ptlrpc_check_set()) Completed RPC req@0000000063c20b43 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_00:lustre-MDT0001-mdtlov_UUID:3353:1862456998073408:0@lo:13:osp-pre-0-1.0 00010000:00010000:3.0:1776178377.186323:0:6244:0:(ldlm_lockd.c:2792:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:1.0:1776178378.024337:0:3347:0:(events.c:361:request_in_callback()) peer: 12345-192.168.204.3@tcp (source: 12345-192.168.204.3@tcp) 00000100:00100000:1.0:1776178378.024396:0:3347:0:(events.c:361:request_in_callback()) peer: 12345-192.168.204.3@tcp (source: 12345-192.168.204.3@tcp) 00000100:00100000:1.0:1776178378.024429:0:3347:0:(events.c:361:request_in_callback()) peer: 12345-192.168.204.3@tcp (source: 12345-192.168.204.3@tcp) 00000100:00100000:2.0:1776178378.024455:0:6256:0:(service.c:2152:ptlrpc_server_handle_req_in()) got req x1862456979120000 00000100:00100000:1.0:1776178378.024473:0:3347:0:(events.c:361:request_in_callback()) peer: 12345-192.168.204.3@tcp (source: 12345-192.168.204.3@tcp) 00000100:00100000:1.0:1776178378.024506:0:3347:0:(events.c:361:request_in_callback()) peer: 12345-192.168.204.3@tcp (source: 12345-192.168.204.3@tcp) 00000100:00100000:0.0:1776178378.024575:0:20755:0:(service.c:2152:ptlrpc_server_handle_req_in()) got req x1862456979120128 00000100:00100000:0.0:1776178378.024600:0:20755:0:(service.c:2152:ptlrpc_server_handle_req_in()) got req x1862456979120064 00000100:00100000:0.0:1776178378.024617:0:20755:0:(nrs_fifo.c:179:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.204.3@tcp, seq: 647 00000100:00100000:0.0:1776178378.024623:0:20755:0:(service.c:2324:ptlrpc_server_handle_request()) Handling RPC req@0000000018b5e3c6 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_006:b50963bd-8eba-4432-9a54-8888d00e3fa1+9:2273:x1862456979120128:12345-192.168.204.3@tcp:400:kworker/u8:3.0 00000100:00100000:0.0:1776178378.024715:0:20755:0:(service.c:2372:ptlrpc_server_handle_request()) Handled RPC req@0000000018b5e3c6 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_006:b50963bd-8eba-4432-9a54-8888d00e3fa1+9:2273:x1862456979120128:12345-192.168.204.3@tcp:400:kworker/u8:3.0 Request processed in 91us (319us total) trans 0 rc 0/0 00000100:00100000:0.0:1776178378.024723:0:20755:0:(nrs_fifo.c:241:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.204.3@tcp, seq: 647 00000100:00100000:0.0:1776178378.024744:0:20755:0:(nrs_fifo.c:179:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.204.3@tcp, seq: 648 00000100:00100000:0.0:1776178378.024747:0:20755:0:(service.c:2324:ptlrpc_server_handle_request()) Handling RPC req@0000000043a010c5 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_006:b50963bd-8eba-4432-9a54-8888d00e3fa1+8:2273:x1862456979120064:12345-192.168.204.3@tcp:400:kworker/u8:3.0 00000100:00100000:0.0:1776178378.024772:0:20755:0:(service.c:2372:ptlrpc_server_handle_request()) Handled RPC req@0000000043a010c5 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_006:b50963bd-8eba-4432-9a54-8888d00e3fa1+8:2273:x1862456979120064:12345-192.168.204.3@tcp:400:kworker/u8:3.0 Request processed in 24us (300us total) trans 0 rc 0/0 00000100:00100000:0.0:1776178378.024778:0:20755:0:(nrs_fifo.c:241:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.204.3@tcp, seq: 648 00000100:00100000:3.0:1776178378.025798:0:20140:0:(service.c:2152:ptlrpc_server_handle_req_in()) got req x1862456979119872 00000100:00100000:3.0:1776178378.025981:0:20140:0:(nrs_fifo.c:179:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.204.3@tcp, seq: 223 00000100:00100000:3.0:1776178378.025990:0:20140:0:(service.c:2324:ptlrpc_server_handle_request()) Handling RPC req@0000000038b01669 pname:cluuid+ref:pid:xid:nid:opc:job ll_mgs_0002:5b5c66b1-1a2f-4651-8548-0c75b9a38a94+9:2273:x1862456979119872:12345-192.168.204.3@tcp:400:kworker/u8:3.0 00000100:00100000:3.0:1776178378.026039:0:20140:0:(service.c:2372:ptlrpc_server_handle_request()) Handled RPC req@0000000038b01669 pname:cluuid+ref:pid:xid:nid:opc:job ll_mgs_0002:5b5c66b1-1a2f-4651-8548-0c75b9a38a94+9:2273:x1862456979119872:12345-192.168.204.3@tcp:400:kworker/u8:3.0 Request processed in 49us (1610us total) trans 0 rc 0/0 00000100:00100000:3.0:1776178378.026046:0:20140:0:(nrs_fifo.c:241:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.204.3@tcp, seq: 223 00000100:00100000:2.0:1776178378.036842:0:6256:0:(service.c:2152:ptlrpc_server_handle_req_in()) got req x1862456979119936 00000100:00100000:2.0:1776178378.037000:0:6256:0:(nrs_fifo.c:179:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.204.3@tcp, seq: 1030 00000100:00100000:2.0:1776178378.037012:0:6256:0:(service.c:2324:ptlrpc_server_handle_request()) Handling RPC req@00000000a770fd7b pname:cluuid+ref:pid:xid:nid:opc:job mdt00_001:b50963bd-8eba-4432-9a54-8888d00e3fa1+6:2273:x1862456979120000:12345-192.168.204.3@tcp:400:kworker/u8:3.0 00000100:00100000:2.0:1776178378.037115:0:6256:0:(service.c:2372:ptlrpc_server_handle_request()) Handled RPC req@00000000a770fd7b pname:cluuid+ref:pid:xid:nid:opc:job mdt00_001:b50963bd-8eba-4432-9a54-8888d00e3fa1+6:2273:x1862456979120000:12345-192.168.204.3@tcp:400:kworker/u8:3.0 Request processed in 105us (12790us total) trans 0 rc 0/0 00000100:00100000:2.0:1776178378.037123:0:6256:0:(nrs_fifo.c:241:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.204.3@tcp, seq: 1030 00000100:00100000:2.0:1776178378.037158:0:6256:0:(nrs_fifo.c:179:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.204.3@tcp, seq: 6760 00000100:00100000:2.0:1776178378.037162:0:6256:0:(service.c:2324:ptlrpc_server_handle_request()) Handling RPC req@00000000f720f286 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_001:0+-99:2269:x1862456979119936:12345-192.168.204.3@tcp:38:kworker/u8:3.0 00010000:02000000:2.0:1776178378.037176:0:6256:0:(ldlm_lib.c:1139:target_handle_connect()) lustre-MDT0000: Not available for connect from 192.168.204.3@tcp (stopping) 00000100:00100000:2.0:1776178378.037204:0:6256:0:(service.c:2372:ptlrpc_server_handle_request()) Handled RPC req@00000000f720f286 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_001:0+-99:2269:x1862456979119936:12345-192.168.204.3@tcp:38:kworker/u8:3.0 Request processed in 41us (12699us total) trans 0 rc -19/-19 00000100:00100000:2.0:1776178378.037211:0:6256:0:(nrs_fifo.c:241:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.204.3@tcp, seq: 6760 00010000:00010000:2.0:1776178378.208343:0:6245:0:(ldlm_lockd.c:2792:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00010000:00010000:2.0:1776178379.232398:0:9224:0:(ldlm_lockd.c:2792:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:2.0:1776178379.618609:0:3356:0:(ptlrpcd.c:415:ptlrpcd_check()) transfer 1 async RPCs [2->3] 00000100:00080000:3.0:1776178379.618634:0:37:0:(pinger.c:251:ptlrpc_pinger_process_import()) lustre-MDT0000-lwp-OST0000_UUID->lustre-MDT0000_UUID: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00080000:3.0:1776178379.618642:0:37:0:(recover.c:57:ptlrpc_initiate_recovery()) lustre-MDT0000_UUID: starting recovery 00000100:00080000:3.0:1776178379.618645:0:37:0:(import.c:86:import_set_state_nolock()) 0000000047601a98 lustre-MDT0000_UUID: changing import state from DISCONN to CONNECTING 00000100:00080000:3.0:1776178379.618655:0:37:0:(import.c:534:import_select_connection()) lustre-MDT0000-lwp-OST0000: connect to NID 0@lo last attempt 1081 00000100:00080000:3.0:1776178379.618659:0:37:0:(import.c:581:import_select_connection()) lustre-MDT0000-lwp-OST0000: tried all connections, increasing latency to 21s 00000100:00080000:3.0:1776178379.618666:0:37:0:(import.c:615:import_select_connection()) lustre-MDT0000-lwp-OST0000: import 0000000047601a98 using connection 192.168.204.103@tcp/0@lo 00000100:00100000:3.0:1776178379.618690:0:37:0:(import.c:807:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5) req@0000000047c15ac0 x1862456998073536/t0(0) o38->lustre-MDT0000-lwp-OST0000@0@lo:12/10 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' 00000100:00100000:1.0:1776178379.618781:0:3352:0:(client.c:1733:ptlrpc_send_new_req()) Sending RPC req@0000000047c15ac0 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:lustre-MDT0000-lwp-OST0000_UUID:3352:1862456998073536:0@lo:38:kworker/u8:2.0 00000100:00100000:1.0:1776178379.618852:0:3352:0:(events.c:361:request_in_callback()) peer: 12345-0@lo (source: 12345-0@lo) 00000100:00100000:1.0:1776178379.618914:0:11281:0:(service.c:2152:ptlrpc_server_handle_req_in()) got req x1862456998073536 00000100:00100000:1.0:1776178379.618959:0:11281:0:(nrs_fifo.c:179:nrs_fifo_req_get()) NRS start fifo request from 12345-0@lo, seq: 6761 00000100:00100000:1.0:1776178379.618965:0:11281:0:(service.c:2324:ptlrpc_server_handle_request()) Handling RPC req@00000000af523f53 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_004:0+-99:3352:x1862456998073536:12345-0@lo:38:kworker/u8:2.0 00010000:02000000:1.0:1776178379.618980:0:11281:0:(ldlm_lib.c:1139:target_handle_connect()) lustre-MDT0000: Not available for connect from 0@lo (stopping) 00000100:00100000:1.0:1776178379.619023:0:11281:0:(service.c:2372:ptlrpc_server_handle_request()) Handled RPC req@00000000af523f53 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_004:0+-99:3352:x1862456998073536:12345-0@lo:38:kworker/u8:2.0 Request processed in 56us (171us total) trans 0 rc -19/-19 00000100:00100000:1.0:1776178379.619029:0:11281:0:(nrs_fifo.c:241:nrs_fifo_req_stop()) NRS stop fifo request from 12345-0@lo, seq: 6761 00000100:00080000:1.0:1776178379.619098:0:3352:0:(import.c:86:import_set_state_nolock()) 0000000047601a98 lustre-MDT0000_UUID: changing import state from CONNECTING to DISCONN 00000100:00080000:1.0:1776178379.619102:0:3352:0:(import.c:1429:ptlrpc_connect_interpret()) recovery of lustre-MDT0000_UUID on 192.168.204.103@tcp failed (-19) 00000100:00100000:1.0:1776178379.619106:0:3352:0:(client.c:2215:ptlrpc_check_set()) Completed RPC req@0000000047c15ac0 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:lustre-MDT0000-lwp-OST0000_UUID:3352:1862456998073536:0@lo:38:kworker/u8:2.0 00000100:00100000:0.0:1776178379.619390:0:3355:0:(ptlrpcd.c:415:ptlrpcd_check()) transfer 1 async RPCs [3->2] 00000100:00100000:0.0:1776178379.619404:0:3355:0:(client.c:1733:ptlrpc_send_new_req()) Sending RPC req@0000000028422d92 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_02:lustre-MDT0001-lwp-OST0000_UUID:3355:1862456998073600:0@lo:400:kworker/u8:2.0 00000100:00100000:0.0:1776178379.619438:0:3355:0:(events.c:361:request_in_callback()) peer: 12345-0@lo (source: 12345-0@lo) 00000100:00100000:1.0:1776178379.619817:0:8250:0:(service.c:2152:ptlrpc_server_handle_req_in()) got req x1862456998073600 00000100:00100000:1.0:1776178379.619859:0:8250:0:(nrs_fifo.c:179:nrs_fifo_req_get()) NRS start fifo request from 12345-0@lo, seq: 1031 00000100:00100000:1.0:1776178379.619863:0:8250:0:(service.c:2324:ptlrpc_server_handle_request()) Handling RPC req@0000000025f8393e pname:cluuid+ref:pid:xid:nid:opc:job mdt00_003:lustre-MDT0001-lwp-OST0000_UUID+5:3355:x1862456998073600:12345-0@lo:400:kworker/u8:2.0 00000100:00100000:1.0:1776178379.619916:0:8250:0:(service.c:2372:ptlrpc_server_handle_request()) Handled RPC req@0000000025f8393e pname:cluuid+ref:pid:xid:nid:opc:job mdt00_003:lustre-MDT0001-lwp-OST0000_UUID+5:3355:x1862456998073600:12345-0@lo:400:kworker/u8:2.0 Request processed in 52us (478us total) trans 0 rc 0/0 00000100:00100000:1.0:1776178379.619922:0:8250:0:(nrs_fifo.c:241:nrs_fifo_req_stop()) NRS stop fifo request from 12345-0@lo, seq: 1031 00000100:00100000:1.0:1776178379.619954:0:3355:0:(client.c:2215:ptlrpc_check_set()) Completed RPC req@0000000028422d92 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_02:lustre-MDT0001-lwp-OST0000_UUID:3355:1862456998073600:0@lo:400:kworker/u8:2.0 00000100:00100000:2.0:1776178379.623655:0:3356:0:(client.c:1733:ptlrpc_send_new_req()) Sending RPC req@00000000952003a4 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_03:ac61f839-0e8e-4cac-9c59-08bcfbbad330:3356:1862456998073472:0@lo:400:kworker/u8:2.0 00000100:00100000:2.0:1776178379.623741:0:3356:0:(events.c:361:request_in_callback()) peer: 12345-0@lo (source: 12345-0@lo) 00000100:00080000:3.0:1776178379.625289:0:37:0:(pinger.c:251:ptlrpc_pinger_process_import()) lustre-MDT0000-lwp-OST0001_UUID->lustre-MDT0000_UUID: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00080000:3.0:1776178379.625299:0:37:0:(recover.c:57:ptlrpc_initiate_recovery()) lustre-MDT0000_UUID: starting recovery 00000100:00080000:3.0:1776178379.625302:0:37:0:(import.c:86:import_set_state_nolock()) 00000000dfc8db13 lustre-MDT0000_UUID: changing import state from DISCONN to CONNECTING 00000100:00080000:3.0:1776178379.625310:0:37:0:(import.c:534:import_select_connection()) lustre-MDT0000-lwp-OST0001: connect to NID 0@lo last attempt 1081 00000100:00080000:3.0:1776178379.625314:0:37:0:(import.c:581:import_select_connection()) lustre-MDT0000-lwp-OST0001: tried all connections, increasing latency to 21s 00000100:00080000:3.0:1776178379.625320:0:37:0:(import.c:615:import_select_connection()) lustre-MDT0000-lwp-OST0001: import 00000000dfc8db13 using connection 192.168.204.103@tcp/0@lo 00000100:00100000:3.0:1776178379.625370:0:37:0:(import.c:807:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5) req@000000008c3fbdb6 x1862456998073664/t0(0) o38->lustre-MDT0000-lwp-OST0001@0@lo:12/10 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' 00000100:00080000:3.0:1776178379.625406:0:37:0:(pinger.c:251:ptlrpc_pinger_process_import()) lustre-MDT0001-mdtlov_UUID->lustre-MDT0000_UUID: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00080000:3.0:1776178379.625409:0:37:0:(recover.c:57:ptlrpc_initiate_recovery()) lustre-MDT0000_UUID: starting recovery 00000100:00080000:3.0:1776178379.625411:0:37:0:(import.c:86:import_set_state_nolock()) 00000000ba4f1821 lustre-MDT0000_UUID: changing import state from DISCONN to CONNECTING 00000100:00080000:3.0:1776178379.625414:0:37:0:(import.c:534:import_select_connection()) lustre-MDT0000-osp-MDT0001: connect to NID 0@lo last attempt 1081 00000100:00080000:3.0:1776178379.625415:0:37:0:(import.c:581:import_select_connection()) lustre-MDT0000-osp-MDT0001: tried all connections, increasing latency to 21s 00000100:00080000:3.0:1776178379.625418:0:37:0:(import.c:615:import_select_connection()) lustre-MDT0000-osp-MDT0001: import 00000000ba4f1821 using connection 192.168.204.103@tcp/0@lo 00000100:00100000:3.0:1776178379.625426:0:37:0:(import.c:807:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5) req@000000006342fd4b x1862456998073792/t0(0) o38->lustre-MDT0000-osp-MDT0001@0@lo:24/4 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' 00000100:00080000:3.0:1776178379.625443:0:37:0:(pinger.c:251:ptlrpc_pinger_process_import()) lustre-MDT0000-lwp-MDT0001_UUID->lustre-MDT0000_UUID: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00080000:3.0:1776178379.625446:0:37:0:(recover.c:57:ptlrpc_initiate_recovery()) lustre-MDT0000_UUID: starting recovery 00000100:00080000:3.0:1776178379.625447:0:37:0:(import.c:86:import_set_state_nolock()) 00000000a7c55b1f lustre-MDT0000_UUID: changing import state from DISCONN to CONNECTING 00000100:00080000:3.0:1776178379.625449:0:37:0:(import.c:534:import_select_connection()) lustre-MDT0000-lwp-MDT0001: connect to NID 0@lo last attempt 1081 00000100:00080000:3.0:1776178379.625451:0:37:0:(import.c:581:import_select_connection()) lustre-MDT0000-lwp-MDT0001: tried all connections, increasing latency to 21s 00000100:00080000:3.0:1776178379.625453:0:37:0:(import.c:615:import_select_connection()) lustre-MDT0000-lwp-MDT0001: import 00000000a7c55b1f using connection 192.168.204.103@tcp/0@lo 00000100:00100000:3.0:1776178379.625460:0:37:0:(import.c:807:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5) req@000000000b5ad4fd x1862456998073984/t0(0) o38->lustre-MDT0000-lwp-MDT0001@0@lo:12/10 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' 00000100:00100000:3.0:1776178379.625492:0:3353:0:(client.c:1733:ptlrpc_send_new_req()) Sending RPC req@00000000ec65370c pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_00:lustre-MDT0001-lwp-OST0001_UUID:3353:1862456998073728:0@lo:400:kworker/u8:2.0 00000100:00100000:3.0:1776178379.625558:0:3353:0:(events.c:361:request_in_callback()) peer: 12345-0@lo (source: 12345-0@lo) 00000100:00100000:3.0:1776178379.625590:0:3353:0:(ptlrpcd.c:415:ptlrpcd_check()) transfer 1 async RPCs [1->0] 00000100:00100000:3.0:1776178379.625595:0:3353:0:(client.c:1733:ptlrpc_send_new_req()) Sending RPC req@0000000028422d92 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_00:lustre-MDT0001-mdtlov_UUID:3353:1862456998073856:0@lo:400:kworker/u8:2.0 00000100:00100000:3.0:1776178379.625615:0:3353:0:(events.c:361:request_in_callback()) peer: 12345-0@lo (source: 12345-0@lo) 00000100:00100000:3.0:1776178379.625660:0:11281:0:(service.c:2152:ptlrpc_server_handle_req_in()) got req x1862456998073728 00000100:00100000:3.0:1776178379.625707:0:11281:0:(nrs_fifo.c:179:nrs_fifo_req_get()) NRS start fifo request from 12345-0@lo, seq: 1032 00000100:00100000:3.0:1776178379.625713:0:11281:0:(service.c:2324:ptlrpc_server_handle_request()) Handling RPC req@00000000ca8b81ba pname:cluuid+ref:pid:xid:nid:opc:job mdt00_004:lustre-MDT0001-lwp-OST0001_UUID+5:3353:x1862456998073728:12345-0@lo:400:kworker/u8:2.0 00000100:00100000:3.0:1776178379.625763:0:11281:0:(service.c:2372:ptlrpc_server_handle_request()) Handled RPC req@00000000ca8b81ba pname:cluuid+ref:pid:xid:nid:opc:job mdt00_004:lustre-MDT0001-lwp-OST0001_UUID+5:3353:x1862456998073728:12345-0@lo:400:kworker/u8:2.0 Request processed in 49us (206us total) trans 0 rc 0/0 00000100:00100000:3.0:1776178379.625770:0:11281:0:(nrs_fifo.c:241:nrs_fifo_req_stop()) NRS stop fifo request from 12345-0@lo, seq: 1032 00000100:00100000:3.0:1776178379.625809:0:3353:0:(client.c:2215:ptlrpc_check_set()) Completed RPC req@00000000ec65370c pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_00:lustre-MDT0001-lwp-OST0001_UUID:3353:1862456998073728:0@lo:400:kworker/u8:2.0 00000100:00100000:3.0:1776178379.625842:0:14440:0:(service.c:2152:ptlrpc_server_handle_req_in()) got req x1862456998073856 00000100:00100000:3.0:1776178379.625863:0:14440:0:(nrs_fifo.c:179:nrs_fifo_req_get()) NRS start fifo request from 12345-0@lo, seq: 649 00000100:00100000:3.0:1776178379.625867:0:14440:0:(service.c:2324:ptlrpc_server_handle_request()) Handling RPC req@0000000028ade006 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_004:lustre-MDT0001-mdtlov_UUID+5:3353:x1862456998073856:12345-0@lo:400:kworker/u8:2.0 00000100:00100000:3.0:1776178379.625893:0:14440:0:(service.c:2372:ptlrpc_server_handle_request()) Handled RPC req@0000000028ade006 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_004:lustre-MDT0001-mdtlov_UUID+5:3353:x1862456998073856:12345-0@lo:400:kworker/u8:2.0 Request processed in 26us (278us total) trans 0 rc 0/0 00000100:00100000:3.0:1776178379.625899:0:14440:0:(nrs_fifo.c:241:nrs_fifo_req_stop()) NRS stop fifo request from 12345-0@lo, seq: 649 00000100:00100000:3.0:1776178379.625919:0:3353:0:(client.c:2215:ptlrpc_check_set()) Completed RPC req@0000000028422d92 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_00:lustre-MDT0001-mdtlov_UUID:3353:1862456998073856:0@lo:400:kworker/u8:2.0 00000100:00100000:1.0:1776178379.626552:0:3352:0:(client.c:1733:ptlrpc_send_new_req()) Sending RPC req@000000008c3fbdb6 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:lustre-MDT0000-lwp-OST0001_UUID:3352:1862456998073664:0@lo:38:kworker/u8:2.0 00000100:00100000:1.0:1776178379.626603:0:3352:0:(events.c:361:request_in_callback()) peer: 12345-0@lo (source: 12345-0@lo) 00000100:00100000:1.0:1776178379.626633:0:3352:0:(client.c:1733:ptlrpc_send_new_req()) Sending RPC req@000000006342fd4b pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:lustre-MDT0001-mdtlov_UUID:3352:1862456998073792:0@lo:38:kworker/u8:2.0 00000100:00100000:1.0:1776178379.626656:0:3352:0:(events.c:361:request_in_callback()) peer: 12345-0@lo (source: 12345-0@lo) 00000100:00100000:1.0:1776178379.626669:0:3352:0:(client.c:1733:ptlrpc_send_new_req()) Sending RPC req@000000000b5ad4fd pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:lustre-MDT0000-lwp-MDT0001_UUID:3352:1862456998073984:0@lo:38:kworker/u8:2.0 00000100:00100000:1.0:1776178379.626688:0:3352:0:(events.c:361:request_in_callback()) peer: 12345-0@lo (source: 12345-0@lo) 00000100:00100000:1.0:1776178379.626718:0:3355:0:(client.c:1733:ptlrpc_send_new_req()) Sending RPC req@0000000047c15ac0 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_02:lustre-MDT0001-mdtlov_UUID:3355:1862456998073920:0@lo:400:kworker/u8:2.0 00000100:00100000:1.0:1776178379.626744:0:3355:0:(events.c:361:request_in_callback()) peer: 12345-0@lo (source: 12345-0@lo) 00000100:00100000:1.0:1776178379.626777:0:8250:0:(service.c:2152:ptlrpc_server_handle_req_in()) got req x1862456998073664 00000100:00100000:1.0:1776178379.626814:0:6255:0:(service.c:2152:ptlrpc_server_handle_req_in()) got req x1862456998073984 00000100:00100000:1.0:1776178379.626834:0:6255:0:(nrs_fifo.c:179:nrs_fifo_req_get()) NRS start fifo request from 12345-0@lo, seq: 6762 00000100:00100000:1.0:1776178379.626837:0:6255:0:(service.c:2324:ptlrpc_server_handle_request()) Handling RPC req@000000008bb01090 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_000:0+-99:3352:x1862456998073664:12345-0@lo:38:kworker/u8:2.0 00010000:02000000:1.0:1776178379.626854:0:6255:0:(ldlm_lib.c:1139:target_handle_connect()) lustre-MDT0000: Not available for connect from 0@lo (stopping) 00000100:00100000:1.0:1776178379.626888:0:6255:0:(service.c:2372:ptlrpc_server_handle_request()) Handled RPC req@000000008bb01090 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_000:0+-99:3352:x1862456998073664:12345-0@lo:38:kworker/u8:2.0 Request processed in 49us (285us total) trans 0 rc -19/-19 00000100:00100000:1.0:1776178379.626894:0:6255:0:(nrs_fifo.c:241:nrs_fifo_req_stop()) NRS stop fifo request from 12345-0@lo, seq: 6762 00000100:00100000:1.0:1776178379.626907:0:6255:0:(nrs_fifo.c:179:nrs_fifo_req_get()) NRS start fifo request from 12345-0@lo, seq: 6763 00000100:00100000:1.0:1776178379.626910:0:6255:0:(service.c:2324:ptlrpc_server_handle_request()) Handling RPC req@00000000dff3309c pname:cluuid+ref:pid:xid:nid:opc:job mdt00_000:0+-99:3352:x1862456998073984:12345-0@lo:38:kworker/u8:2.0 00010000:02000000:1.0:1776178379.626916:0:6255:0:(ldlm_lib.c:1139:target_handle_connect()) lustre-MDT0000: Not available for connect from 0@lo (stopping) 00000100:00100000:1.0:1776178379.626935:0:6255:0:(service.c:2372:ptlrpc_server_handle_request()) Handled RPC req@00000000dff3309c pname:cluuid+ref:pid:xid:nid:opc:job mdt00_000:0+-99:3352:x1862456998073984:12345-0@lo:38:kworker/u8:2.0 Request processed in 24us (247us total) trans 0 rc -19/-19 00000100:00100000:1.0:1776178379.626939:0:6255:0:(nrs_fifo.c:241:nrs_fifo_req_stop()) NRS stop fifo request from 12345-0@lo, seq: 6763 00000100:00080000:1.0:1776178379.626976:0:3352:0:(import.c:86:import_set_state_nolock()) 00000000dfc8db13 lustre-MDT0000_UUID: changing import state from CONNECTING to DISCONN 00000100:00080000:1.0:1776178379.626979:0:3352:0:(import.c:1429:ptlrpc_connect_interpret()) recovery of lustre-MDT0000_UUID on 192.168.204.103@tcp failed (-19) 00000100:00100000:1.0:1776178379.626983:0:3352:0:(client.c:2215:ptlrpc_check_set()) Completed RPC req@000000008c3fbdb6 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:lustre-MDT0000-lwp-OST0001_UUID:3352:1862456998073664:0@lo:38:kworker/u8:2.0 00000100:00080000:1.0:1776178379.626992:0:3352:0:(import.c:86:import_set_state_nolock()) 00000000a7c55b1f lustre-MDT0000_UUID: changing import state from CONNECTING to DISCONN 00000100:00080000:1.0:1776178379.626994:0:3352:0:(import.c:1429:ptlrpc_connect_interpret()) recovery of lustre-MDT0000_UUID on 192.168.204.103@tcp failed (-19) 00000100:00100000:1.0:1776178379.626996:0:3352:0:(client.c:2215:ptlrpc_check_set()) Completed RPC req@000000000b5ad4fd pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:lustre-MDT0000-lwp-MDT0001_UUID:3352:1862456998073984:0@lo:38:kworker/u8:2.0 00000100:00100000:3.0:1776178379.627683:0:20762:0:(service.c:2152:ptlrpc_server_handle_req_in()) got req x1862456998073920 00000100:00100000:3.0:1776178379.627747:0:20762:0:(nrs_fifo.c:179:nrs_fifo_req_get()) NRS start fifo request from 12345-0@lo, seq: 650 00000100:00100000:3.0:1776178379.627751:0:20762:0:(service.c:2324:ptlrpc_server_handle_request()) Handling RPC req@0000000053234520 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_013:lustre-MDT0001-mdtlov_UUID+5:3355:x1862456998073920:12345-0@lo:400:kworker/u8:2.0 00000100:00100000:3.0:1776178379.627817:0:20762:0:(service.c:2372:ptlrpc_server_handle_request()) Handled RPC req@0000000053234520 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_013:lustre-MDT0001-mdtlov_UUID+5:3355:x1862456998073920:12345-0@lo:400:kworker/u8:2.0 Request processed in 64us (1072us total) trans 0 rc 0/0 00000100:00100000:3.0:1776178379.627822:0:20762:0:(nrs_fifo.c:241:nrs_fifo_req_stop()) NRS stop fifo request from 12345-0@lo, seq: 650 00000100:00100000:2.0:1776178379.627943:0:14128:0:(service.c:2152:ptlrpc_server_handle_req_in()) got req x1862456998073792 00000100:00100000:2.0:1776178379.627974:0:14128:0:(nrs_fifo.c:179:nrs_fifo_req_get()) NRS start fifo request from 12345-0@lo, seq: 742 00000100:00100000:2.0:1776178379.627978:0:14128:0:(service.c:2324:ptlrpc_server_handle_request()) Handling RPC req@00000000c1934814 pname:cluuid+ref:pid:xid:nid:opc:job mdt_out00_002:0+-99:3352:x1862456998073792:12345-0@lo:38:kworker/u8:2.0 00010000:02000000:2.0:1776178379.627989:0:14128:0:(ldlm_lib.c:1139:target_handle_connect()) lustre-MDT0000: Not available for connect from 0@lo (stopping) 00000100:00100000:2.0:1776178379.628024:0:14128:0:(service.c:2372:ptlrpc_server_handle_request()) Handled RPC req@00000000c1934814 pname:cluuid+ref:pid:xid:nid:opc:job mdt_out00_002:0+-99:3352:x1862456998073792:12345-0@lo:38:kworker/u8:2.0 Request processed in 45us (1367us total) trans 0 rc -19/-19 00000100:00100000:2.0:1776178379.628033:0:14128:0:(nrs_fifo.c:241:nrs_fifo_req_stop()) NRS stop fifo request from 12345-0@lo, seq: 742 00000100:00080000:2.0:1776178379.628107:0:3352:0:(import.c:86:import_set_state_nolock()) 00000000ba4f1821 lustre-MDT0000_UUID: changing import state from CONNECTING to DISCONN 00000100:00080000:2.0:1776178379.628112:0:3352:0:(import.c:1429:ptlrpc_connect_interpret()) recovery of lustre-MDT0000_UUID on 192.168.204.103@tcp failed (-19) 00000100:00100000:2.0:1776178379.628116:0:3352:0:(client.c:2215:ptlrpc_check_set()) Completed RPC req@000000006342fd4b pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:lustre-MDT0001-mdtlov_UUID:3352:1862456998073792:0@lo:38:kworker/u8:2.0 00000100:00100000:0.0:1776178379.628793:0:20137:0:(service.c:2152:ptlrpc_server_handle_req_in()) got req x1862456998073472 00000100:00100000:0.0:1776178379.628840:0:20137:0:(nrs_fifo.c:179:nrs_fifo_req_get()) NRS start fifo request from 12345-0@lo, seq: 224 00000100:00100000:0.0:1776178379.628844:0:20137:0:(service.c:2324:ptlrpc_server_handle_request()) Handling RPC req@000000003524ab73 pname:cluuid+ref:pid:xid:nid:opc:job ll_mgs_0000:ac61f839-0e8e-4cac-9c59-08bcfbbad330+18:3356:x1862456998073472:12345-0@lo:400:kworker/u8:2.0 00000100:00100000:0.0:1776178379.628911:0:20137:0:(service.c:2372:ptlrpc_server_handle_request()) Handled RPC req@000000003524ab73 pname:cluuid+ref:pid:xid:nid:opc:job ll_mgs_0000:ac61f839-0e8e-4cac-9c59-08bcfbbad330+18:3356:x1862456998073472:12345-0@lo:400:kworker/u8:2.0 Request processed in 66us (5171us total) trans 0 rc 0/0 00000100:00100000:0.0:1776178379.628918:0:20137:0:(nrs_fifo.c:241:nrs_fifo_req_stop()) NRS stop fifo request from 12345-0@lo, seq: 224 00000100:00100000:2.0:1776178379.629195:0:3356:0:(client.c:2215:ptlrpc_check_set()) Completed RPC req@00000000952003a4 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_03:ac61f839-0e8e-4cac-9c59-08bcfbbad330:3356:1862456998073472:0@lo:400:kworker/u8:2.0 00000100:00100000:1.0:1776178379.629408:0:3355:0:(client.c:2215:ptlrpc_check_set()) Completed RPC req@0000000047c15ac0 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_02:lustre-MDT0001-mdtlov_UUID:3355:1862456998073920:0@lo:400:kworker/u8:2.0 00010000:00010000:2.0:1776178380.257302:0:9225:0:(ldlm_lockd.c:2792:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:1.0:1776178381.152500:0:3355:0:(ptlrpcd.c:415:ptlrpcd_check()) transfer 1 async RPCs [3->2] 00000100:00100000:1.0:1776178381.152534:0:3355:0:(client.c:1733:ptlrpc_send_new_req()) Sending RPC req@00000000dd1281fe pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_02:lustre-MDT0001-mdtlov_UUID:3355:1862456998074048:0@lo:13:osp-pre-1-1.0 00000100:00100000:1.0:1776178381.152621:0:3355:0:(events.c:361:request_in_callback()) peer: 12345-0@lo (source: 12345-0@lo) 00000100:00100000:0.0:1776178381.152722:0:9554:0:(service.c:2152:ptlrpc_server_handle_req_in()) got req x1862456998074048 00000100:00100000:0.0:1776178381.152772:0:9554:0:(nrs_fifo.c:179:nrs_fifo_req_get()) NRS start fifo request from 12345-0@lo, seq: 557 00000100:00100000:0.0:1776178381.152778:0:9554:0:(service.c:2324:ptlrpc_server_handle_request()) Handling RPC req@000000003316d46e pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_create00:lustre-MDT0001-mdtlov_UUID+5:3355:x1862456998074048:12345-0@lo:13:osp-pre-1-1.0 00002000:00000024:0.0:1776178381.152808:0:9554:0:(ofd_obd.c:740:ofd_statfs()) blocks cached 229376 granted 491746368 pending 0 free 3923689472 avail 3692167168 00000100:00100000:0.0:1776178381.152856:0:9554:0:(service.c:2372:ptlrpc_server_handle_request()) Handled RPC req@000000003316d46e pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_create00:lustre-MDT0001-mdtlov_UUID+5:3355:x1862456998074048:12345-0@lo:13:osp-pre-1-1.0 Request processed in 77us (235us total) trans 0 rc 0/0 00000100:00100000:0.0:1776178381.152862:0:9554:0:(nrs_fifo.c:241:nrs_fifo_req_stop()) NRS stop fifo request from 12345-0@lo, seq: 557 00000100:00100000:1.0:1776178381.153161:0:3355:0:(client.c:2215:ptlrpc_check_set()) Completed RPC req@00000000dd1281fe pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_02:lustre-MDT0001-mdtlov_UUID:3355:1862456998074048:0@lo:13:osp-pre-1-1.0 00010000:00010000:3.0:1776178381.280411:0:6244:0:(ldlm_lockd.c:2792:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:1.0:1776178381.665378:0:3354:0:(ptlrpcd.c:415:ptlrpcd_check()) transfer 1 async RPCs [0->1] 00000100:00100000:1.0:1776178381.665404:0:3354:0:(client.c:1733:ptlrpc_send_new_req()) Sending RPC req@0000000092f98018 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_01:lustre-MDT0001-mdtlov_UUID:3354:1862456998074112:0@lo:13:osp-pre-0-1.0 00000100:00100000:1.0:1776178381.665475:0:3354:0:(events.c:361:request_in_callback()) peer: 12345-0@lo (source: 12345-0@lo) 00000100:00100000:2.0:1776178381.665579:0:8231:0:(service.c:2152:ptlrpc_server_handle_req_in()) got req x1862456998074112 00000100:00100000:2.0:1776178381.665629:0:8231:0:(nrs_fifo.c:179:nrs_fifo_req_get()) NRS start fifo request from 12345-0@lo, seq: 558 00000100:00100000:2.0:1776178381.665635:0:8231:0:(service.c:2324:ptlrpc_server_handle_request()) Handling RPC req@000000002973935f pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_create00:lustre-MDT0001-mdtlov_UUID+5:3354:x1862456998074112:12345-0@lo:13:osp-pre-0-1.0 00002000:00000024:2.0:1776178381.665665:0:8231:0:(ofd_obd.c:740:ofd_statfs()) blocks cached 315392 granted 484883776 pending 0 free 3923689472 avail 3692167168 00000100:00100000:2.0:1776178381.665713:0:8231:0:(service.c:2372:ptlrpc_server_handle_request()) Handled RPC req@000000002973935f pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_create00:lustre-MDT0001-mdtlov_UUID+5:3354:x1862456998074112:12345-0@lo:13:osp-pre-0-1.0 Request processed in 78us (239us total) trans 0 rc 0/0 00000100:00100000:2.0:1776178381.665721:0:8231:0:(nrs_fifo.c:241:nrs_fifo_req_stop()) NRS stop fifo request from 12345-0@lo, seq: 558 00000100:00100000:1.0:1776178381.665977:0:3354:0:(client.c:2215:ptlrpc_check_set()) Completed RPC req@0000000092f98018 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_01:lustre-MDT0001-mdtlov_UUID:3354:1862456998074112:0@lo:13:osp-pre-0-1.0 00010000:00010000:2.0:1776178382.304318:0:6245:0:(ldlm_lockd.c:2792:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:0.0:1776178383.140578:0:3348:0:(events.c:361:request_in_callback()) peer: 12345-192.168.204.3@tcp (source: 12345-192.168.204.3@tcp) 00000100:00100000:2.0:1776178383.140780:0:6256:0:(service.c:2152:ptlrpc_server_handle_req_in()) got req x1862456979120256 00000100:00100000:2.0:1776178383.140838:0:6256:0:(nrs_fifo.c:179:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.204.3@tcp, seq: 6764 00000100:00100000:2.0:1776178383.140846:0:6256:0:(service.c:2324:ptlrpc_server_handle_request()) Handling RPC req@00000000505e62f8 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_001:0+-99:2269:x1862456979120256:12345-192.168.204.3@tcp:38:kworker/u8:3.0 00010000:02000000:2.0:1776178383.140868:0:6256:0:(ldlm_lib.c:1139:target_handle_connect()) lustre-MDT0000: Not available for connect from 192.168.204.3@tcp (stopping) 00000100:00100000:2.0:1776178383.140945:0:6256:0:(service.c:2372:ptlrpc_server_handle_request()) Handled RPC req@00000000505e62f8 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_001:0+-99:2269:x1862456979120256:12345-192.168.204.3@tcp:38:kworker/u8:3.0 Request processed in 97us (375us total) trans 0 rc -19/-19 00000100:00100000:2.0:1776178383.140952:0:6256:0:(nrs_fifo.c:241:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.204.3@tcp, seq: 6764 00000100:00100000:0.0:1776178383.142153:0:3346:0:(events.c:361:request_in_callback()) peer: 12345-192.168.204.3@tcp (source: 12345-192.168.204.3@tcp) 00000100:00100000:0.0:1776178383.142187:0:3346:0:(events.c:361:request_in_callback()) peer: 12345-192.168.204.3@tcp (source: 12345-192.168.204.3@tcp) 00000100:00100000:0.0:1776178383.142213:0:3346:0:(events.c:361:request_in_callback()) peer: 12345-192.168.204.3@tcp (source: 12345-192.168.204.3@tcp) 00000100:00100000:0.0:1776178383.142235:0:3346:0:(events.c:361:request_in_callback()) peer: 12345-192.168.204.3@tcp (source: 12345-192.168.204.3@tcp) 00000100:00100000:1.0:1776178383.142309:0:14440:0:(service.c:2152:ptlrpc_server_handle_req_in()) got req x1862456979120384 00000100:00100000:1.0:1776178383.142353:0:14440:0:(service.c:2152:ptlrpc_server_handle_req_in()) got req x1862456979120448 00000100:00100000:1.0:1776178383.142384:0:14440:0:(nrs_fifo.c:179:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.204.3@tcp, seq: 651 00000100:00100000:1.0:1776178383.142389:0:14440:0:(service.c:2324:ptlrpc_server_handle_request()) Handling RPC req@000000006c748d80 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_004:b50963bd-8eba-4432-9a54-8888d00e3fa1+8:2272:x1862456979120384:12345-192.168.204.3@tcp:400:kworker/u8:3.0 00000100:00100000:1.0:1776178383.142442:0:14440:0:(service.c:2372:ptlrpc_server_handle_request()) Handled RPC req@000000006c748d80 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_004:b50963bd-8eba-4432-9a54-8888d00e3fa1+8:2272:x1862456979120384:12345-192.168.204.3@tcp:400:kworker/u8:3.0 Request processed in 52us (255us total) trans 0 rc 0/0 00000100:00100000:1.0:1776178383.142448:0:14440:0:(nrs_fifo.c:241:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.204.3@tcp, seq: 651 00000100:00100000:1.0:1776178383.142463:0:14440:0:(nrs_fifo.c:179:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.204.3@tcp, seq: 652 00000100:00100000:1.0:1776178383.142467:0:14440:0:(service.c:2324:ptlrpc_server_handle_request()) Handling RPC req@00000000165897bf pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_004:b50963bd-8eba-4432-9a54-8888d00e3fa1+9:2272:x1862456979120448:12345-192.168.204.3@tcp:400:kworker/u8:3.0 00000100:00100000:1.0:1776178383.142487:0:14440:0:(service.c:2372:ptlrpc_server_handle_request()) Handled RPC req@00000000165897bf pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_004:b50963bd-8eba-4432-9a54-8888d00e3fa1+9:2272:x1862456979120448:12345-192.168.204.3@tcp:400:kworker/u8:3.0 Request processed in 19us (253us total) trans 0 rc 0/0 00000100:00100000:1.0:1776178383.142493:0:14440:0:(nrs_fifo.c:241:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.204.3@tcp, seq: 652 00000100:00100000:2.0:1776178383.142739:0:6256:0:(service.c:2152:ptlrpc_server_handle_req_in()) got req x1862456979120320 00000100:00100000:2.0:1776178383.142783:0:6256:0:(nrs_fifo.c:179:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.204.3@tcp, seq: 1033 00000100:00100000:2.0:1776178383.142787:0:6256:0:(service.c:2324:ptlrpc_server_handle_request()) Handling RPC req@000000002dd26ce5 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_001:b50963bd-8eba-4432-9a54-8888d00e3fa1+6:2272:x1862456979120320:12345-192.168.204.3@tcp:400:kworker/u8:3.0 00000100:00100000:2.0:1776178383.142827:0:6256:0:(service.c:2372:ptlrpc_server_handle_request()) Handled RPC req@000000002dd26ce5 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_001:b50963bd-8eba-4432-9a54-8888d00e3fa1+6:2272:x1862456979120320:12345-192.168.204.3@tcp:400:kworker/u8:3.0 Request processed in 39us (614us total) trans 0 rc 0/0 00000100:00100000:2.0:1776178383.142833:0:6256:0:(nrs_fifo.c:241:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.204.3@tcp, seq: 1033 00000100:00100000:3.0:1776178383.143246:0:20140:0:(service.c:2152:ptlrpc_server_handle_req_in()) got req x1862456979120192 00000100:00100000:3.0:1776178383.143295:0:20140:0:(nrs_fifo.c:179:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.204.3@tcp, seq: 225 00000100:00100000:3.0:1776178383.143301:0:20140:0:(service.c:2324:ptlrpc_server_handle_request()) Handling RPC req@0000000048947fd8 pname:cluuid+ref:pid:xid:nid:opc:job ll_mgs_0002:5b5c66b1-1a2f-4651-8548-0c75b9a38a94+9:2272:x1862456979120192:12345-192.168.204.3@tcp:400:kworker/u8:3.0 00000100:00100000:3.0:1776178383.143340:0:20140:0:(service.c:2372:ptlrpc_server_handle_request()) Handled RPC req@0000000048947fd8 pname:cluuid+ref:pid:xid:nid:opc:job ll_mgs_0002:5b5c66b1-1a2f-4651-8548-0c75b9a38a94+9:2272:x1862456979120192:12345-192.168.204.3@tcp:400:kworker/u8:3.0 Request processed in 38us (1192us total) trans 0 rc 0/0 00000100:00100000:3.0:1776178383.143346:0:20140:0:(nrs_fifo.c:241:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.204.3@tcp, seq: 225 00010000:00010000:2.0:1776178383.328542:0:9224:0:(ldlm_lockd.c:2792:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00010000:00010000:2.0:1776178384.355176:0:9225:0:(ldlm_lockd.c:2792:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00080000:1.0:1776178384.738158:0:37:0:(pinger.c:251:ptlrpc_pinger_process_import()) lustre-MDT0000-lwp-OST0000_UUID->lustre-MDT0000_UUID: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00080000:1.0:1776178384.738166:0:37:0:(recover.c:57:ptlrpc_initiate_recovery()) lustre-MDT0000_UUID: starting recovery 00000100:00080000:1.0:1776178384.738170:0:37:0:(import.c:86:import_set_state_nolock()) 0000000047601a98 lustre-MDT0000_UUID: changing import state from DISCONN to CONNECTING 00000100:00080000:1.0:1776178384.738178:0:37:0:(import.c:534:import_select_connection()) lustre-MDT0000-lwp-OST0000: connect to NID 0@lo last attempt 1086 00000100:00080000:1.0:1776178384.738182:0:37:0:(import.c:581:import_select_connection()) lustre-MDT0000-lwp-OST0000: tried all connections, increasing latency to 21s 00000100:00080000:1.0:1776178384.738187:0:37:0:(import.c:615:import_select_connection()) lustre-MDT0000-lwp-OST0000: import 0000000047601a98 using connection 192.168.204.103@tcp/0@lo 00000100:00100000:1.0:1776178384.738209:0:37:0:(import.c:807:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5) req@000000006a209b80 x1862456998074240/t0(0) o38->lustre-MDT0000-lwp-OST0000@0@lo:12/10 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' 00000100:00080000:1.0:1776178384.738243:0:37:0:(pinger.c:251:ptlrpc_pinger_process_import()) lustre-MDT0000-lwp-OST0001_UUID->lustre-MDT0000_UUID: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00080000:1.0:1776178384.738265:0:37:0:(recover.c:57:ptlrpc_initiate_recovery()) lustre-MDT0000_UUID: starting recovery 00000100:00080000:1.0:1776178384.738267:0:37:0:(import.c:86:import_set_state_nolock()) 00000000dfc8db13 lustre-MDT0000_UUID: changing import state from DISCONN to CONNECTING 00000100:00080000:1.0:1776178384.738271:0:37:0:(import.c:534:import_select_connection()) lustre-MDT0000-lwp-OST0001: connect to NID 0@lo last attempt 1086 00000100:00080000:1.0:1776178384.738273:0:37:0:(import.c:581:import_select_connection()) lustre-MDT0000-lwp-OST0001: tried all connections, increasing latency to 21s 00000100:00080000:1.0:1776178384.738278:0:37:0:(import.c:615:import_select_connection()) lustre-MDT0000-lwp-OST0001: import 00000000dfc8db13 using connection 192.168.204.103@tcp/0@lo 00000100:00100000:1.0:1776178384.738292:0:37:0:(import.c:807:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5) req@00000000cb1b73cb x1862456998074368/t0(0) o38->lustre-MDT0000-lwp-OST0001@0@lo:12/10 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' 00000100:00100000:3.0:1776178384.738305:0:3353:0:(ptlrpcd.c:415:ptlrpcd_check()) transfer 1 async RPCs [1->0] 00000100:00080000:1.0:1776178384.738309:0:37:0:(pinger.c:251:ptlrpc_pinger_process_import()) lustre-MDT0001-mdtlov_UUID->lustre-MDT0000_UUID: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00080000:1.0:1776178384.738313:0:37:0:(recover.c:57:ptlrpc_initiate_recovery()) lustre-MDT0000_UUID: starting recovery 00000100:00080000:1.0:1776178384.738315:0:37:0:(import.c:86:import_set_state_nolock()) 00000000ba4f1821 lustre-MDT0000_UUID: changing import state from DISCONN to CONNECTING 00000100:00080000:1.0:1776178384.738318:0:37:0:(import.c:534:import_select_connection()) lustre-MDT0000-osp-MDT0001: connect to NID 0@lo last attempt 1086 00000100:00080000:1.0:1776178384.738320:0:37:0:(import.c:581:import_select_connection()) lustre-MDT0000-osp-MDT0001: tried all connections, increasing latency to 21s 00000100:00080000:1.0:1776178384.738324:0:37:0:(import.c:615:import_select_connection()) lustre-MDT0000-osp-MDT0001: import 00000000ba4f1821 using connection 192.168.204.103@tcp/0@lo 00000100:00100000:3.0:1776178384.738336:0:3353:0:(client.c:1733:ptlrpc_send_new_req()) Sending RPC req@00000000ba273b2a pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_00:ac61f839-0e8e-4cac-9c59-08bcfbbad330:3353:1862456998074176:0@lo:400:kworker/u8:2.0 00000100:00100000:3.0:1776178384.738411:0:3353:0:(events.c:361:request_in_callback()) peer: 12345-0@lo (source: 12345-0@lo) 00000100:00100000:3.0:1776178384.738445:0:3356:0:(client.c:1733:ptlrpc_send_new_req()) Sending RPC req@00000000cc80b69e pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_03:lustre-MDT0001-lwp-OST0001_UUID:3356:1862456998074432:0@lo:400:kworker/u8:2.0 00000100:00100000:3.0:1776178384.738473:0:3356:0:(events.c:361:request_in_callback()) peer: 12345-0@lo (source: 12345-0@lo) 00000100:00100000:3.0:1776178384.738488:0:3356:0:(ptlrpcd.c:415:ptlrpcd_check()) transfer 1 async RPCs [2->3] 00000100:00100000:3.0:1776178384.738492:0:3356:0:(client.c:1733:ptlrpc_send_new_req()) Sending RPC req@00000000f9929c4b pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_03:lustre-MDT0001-lwp-OST0000_UUID:3356:1862456998074304:0@lo:400:kworker/u8:2.0 00000100:00100000:3.0:1776178384.738514:0:3356:0:(events.c:361:request_in_callback()) peer: 12345-0@lo (source: 12345-0@lo) 00000100:00100000:3.0:1776178384.738553:0:20137:0:(service.c:2152:ptlrpc_server_handle_req_in()) got req x1862456998074176 00000100:00100000:3.0:1776178384.738591:0:20137:0:(nrs_fifo.c:179:nrs_fifo_req_get()) NRS start fifo request from 12345-0@lo, seq: 226 00000100:00100000:3.0:1776178384.738596:0:20137:0:(service.c:2324:ptlrpc_server_handle_request()) Handling RPC req@00000000ed9d6990 pname:cluuid+ref:pid:xid:nid:opc:job ll_mgs_0000:ac61f839-0e8e-4cac-9c59-08bcfbbad330+18:3353:x1862456998074176:12345-0@lo:400:kworker/u8:2.0 00000100:00100000:3.0:1776178384.738634:0:20137:0:(service.c:2372:ptlrpc_server_handle_request()) Handled RPC req@00000000ed9d6990 pname:cluuid+ref:pid:xid:nid:opc:job ll_mgs_0000:ac61f839-0e8e-4cac-9c59-08bcfbbad330+18:3353:x1862456998074176:12345-0@lo:400:kworker/u8:2.0 Request processed in 38us (224us total) trans 0 rc 0/0 00000100:00100000:3.0:1776178384.738640:0:20137:0:(nrs_fifo.c:241:nrs_fifo_req_stop()) NRS stop fifo request from 12345-0@lo, seq: 226 00000100:00100000:3.0:1776178384.738693:0:3353:0:(client.c:2215:ptlrpc_check_set()) Completed RPC req@00000000ba273b2a pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_00:ac61f839-0e8e-4cac-9c59-08bcfbbad330:3353:1862456998074176:0@lo:400:kworker/u8:2.0 00000100:00100000:2.0:1776178384.738712:0:3352:0:(client.c:1733:ptlrpc_send_new_req()) Sending RPC req@000000006a209b80 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:lustre-MDT0000-lwp-OST0000_UUID:3352:1862456998074240:0@lo:38:kworker/u8:2.0 00000100:00100000:3.0:1776178384.738742:0:6256:0:(service.c:2152:ptlrpc_server_handle_req_in()) got req x1862456998074432 00000100:00100000:2.0:1776178384.738747:0:3352:0:(events.c:361:request_in_callback()) peer: 12345-0@lo (source: 12345-0@lo) 00000100:00100000:1.0:1776178384.738762:0:37:0:(import.c:807:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5) req@00000000d2194868 x1862456998074496/t0(0) o38->lustre-MDT0000-osp-MDT0001@0@lo:24/4 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' 00000100:00100000:2.0:1776178384.738769:0:3352:0:(client.c:1733:ptlrpc_send_new_req()) Sending RPC req@00000000cb1b73cb pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:lustre-MDT0000-lwp-OST0001_UUID:3352:1862456998074368:0@lo:38:kworker/u8:2.0 00000100:00100000:3.0:1776178384.738774:0:6256:0:(service.c:2152:ptlrpc_server_handle_req_in()) got req x1862456998074304 00000100:00100000:3.0:1776178384.738797:0:6256:0:(service.c:2152:ptlrpc_server_handle_req_in()) got req x1862456998074240 00000100:00100000:2.0:1776178384.738801:0:3352:0:(events.c:361:request_in_callback()) peer: 12345-0@lo (source: 12345-0@lo) 00000100:00080000:1.0:1776178384.738810:0:37:0:(pinger.c:251:ptlrpc_pinger_process_import()) lustre-MDT0000-lwp-MDT0001_UUID->lustre-MDT0000_UUID: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00080000:1.0:1776178384.738814:0:37:0:(recover.c:57:ptlrpc_initiate_recovery()) lustre-MDT0000_UUID: starting recovery 00000100:00080000:1.0:1776178384.738816:0:37:0:(import.c:86:import_set_state_nolock()) 00000000a7c55b1f lustre-MDT0000_UUID: changing import state from DISCONN to CONNECTING 00000100:00100000:2.0:1776178384.738819:0:3352:0:(client.c:1733:ptlrpc_send_new_req()) Sending RPC req@00000000d2194868 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:lustre-MDT0001-mdtlov_UUID:3352:1862456998074496:0@lo:38:kworker/u8:2.0 00000100:00080000:1.0:1776178384.738821:0:37:0:(import.c:534:import_select_connection()) lustre-MDT0000-lwp-MDT0001: connect to NID 0@lo last attempt 1086 00000100:00080000:1.0:1776178384.738824:0:37:0:(import.c:581:import_select_connection()) lustre-MDT0000-lwp-MDT0001: tried all connections, increasing latency to 21s 00000100:00080000:1.0:1776178384.738829:0:37:0:(import.c:615:import_select_connection()) lustre-MDT0000-lwp-MDT0001: import 00000000a7c55b1f using connection 192.168.204.103@tcp/0@lo 00000100:00100000:1.0:1776178384.738842:0:37:0:(import.c:807:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5) req@0000000017d54e92 x1862456998074688/t0(0) o38->lustre-MDT0000-lwp-MDT0001@0@lo:12/10 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' 00000100:00100000:2.0:1776178384.738853:0:3352:0:(events.c:361:request_in_callback()) peer: 12345-0@lo (source: 12345-0@lo) 00000100:00100000:1.0:1776178384.738874:0:11281:0:(service.c:2152:ptlrpc_server_handle_req_in()) got req x1862456998074368 00000100:00100000:2.0:1776178384.738876:0:3352:0:(client.c:1733:ptlrpc_send_new_req()) Sending RPC req@0000000017d54e92 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:lustre-MDT0000-lwp-MDT0001_UUID:3352:1862456998074688:0@lo:38:kworker/u8:2.0 00000100:00100000:1.0:1776178384.738895:0:11281:0:(nrs_fifo.c:179:nrs_fifo_req_get()) NRS start fifo request from 12345-0@lo, seq: 1034 00000100:00100000:1.0:1776178384.738903:0:11281:0:(service.c:2324:ptlrpc_server_handle_request()) Handling RPC req@00000000b6612f78 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_004:lustre-MDT0001-lwp-OST0001_UUID+5:3356:x1862456998074432:12345-0@lo:400:kworker/u8:2.0 00000100:00100000:2.0:1776178384.738904:0:3352:0:(events.c:361:request_in_callback()) peer: 12345-0@lo (source: 12345-0@lo) 00000100:00100000:2.0:1776178384.738931:0:15213:0:(service.c:2152:ptlrpc_server_handle_req_in()) got req x1862456998074496 00000100:00100000:1.0:1776178384.738952:0:11281:0:(service.c:2372:ptlrpc_server_handle_request()) Handled RPC req@00000000b6612f78 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_004:lustre-MDT0001-lwp-OST0001_UUID+5:3356:x1862456998074432:12345-0@lo:400:kworker/u8:2.0 Request processed in 50us (478us total) trans 0 rc 0/0 00000100:00100000:2.0:1776178384.738957:0:15213:0:(nrs_fifo.c:179:nrs_fifo_req_get()) NRS start fifo request from 12345-0@lo, seq: 743 00000100:00100000:1.0:1776178384.738966:0:11281:0:(nrs_fifo.c:241:nrs_fifo_req_stop()) NRS stop fifo request from 12345-0@lo, seq: 1034 00000100:00100000:2.0:1776178384.738966:0:15213:0:(service.c:2324:ptlrpc_server_handle_request()) Handling RPC req@00000000a30eb964 pname:cluuid+ref:pid:xid:nid:opc:job mdt_out00_003:0+-99:3352:x1862456998074496:12345-0@lo:38:kworker/u8:2.0 00010000:02000000:2.0:1776178384.738981:0:15213:0:(ldlm_lib.c:1139:target_handle_connect()) lustre-MDT0000: Not available for connect from 0@lo (stopping) 00000100:00100000:1.0:1776178384.738984:0:11281:0:(service.c:2152:ptlrpc_server_handle_req_in()) got req x1862456998074688 00000100:00100000:1.0:1776178384.739001:0:11281:0:(nrs_fifo.c:179:nrs_fifo_req_get()) NRS start fifo request from 12345-0@lo, seq: 1035 00000100:00100000:1.0:1776178384.739006:0:11281:0:(service.c:2324:ptlrpc_server_handle_request()) Handling RPC req@00000000a0c64d21 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_004:lustre-MDT0001-lwp-OST0000_UUID+5:3356:x1862456998074304:12345-0@lo:400:kworker/u8:2.0 00000100:00100000:2.0:1776178384.739016:0:15213:0:(service.c:2372:ptlrpc_server_handle_request()) Handled RPC req@00000000a30eb964 pname:cluuid+ref:pid:xid:nid:opc:job mdt_out00_003:0+-99:3352:x1862456998074496:12345-0@lo:38:kworker/u8:2.0 Request processed in 51us (164us total) trans 0 rc -19/-19 00000100:00100000:2.0:1776178384.739023:0:15213:0:(nrs_fifo.c:241:nrs_fifo_req_stop()) NRS stop fifo request from 12345-0@lo, seq: 743 00000100:00100000:1.0:1776178384.739041:0:11281:0:(service.c:2372:ptlrpc_server_handle_request()) Handled RPC req@00000000a0c64d21 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_004:lustre-MDT0001-lwp-OST0000_UUID+5:3356:x1862456998074304:12345-0@lo:400:kworker/u8:2.0 Request processed in 34us (526us total) trans 0 rc 0/0 00000100:00100000:1.0:1776178384.739051:0:11281:0:(nrs_fifo.c:241:nrs_fifo_req_stop()) NRS stop fifo request from 12345-0@lo, seq: 1035 00000100:00080000:2.0:1776178384.739057:0:3352:0:(import.c:86:import_set_state_nolock()) 00000000ba4f1821 lustre-MDT0000_UUID: changing import state from CONNECTING to DISCONN 00000100:00080000:2.0:1776178384.739079:0:3352:0:(import.c:1429:ptlrpc_connect_interpret()) recovery of lustre-MDT0000_UUID on 192.168.204.103@tcp failed (-19) 00000100:00100000:2.0:1776178384.739084:0:3352:0:(client.c:2215:ptlrpc_check_set()) Completed RPC req@00000000d2194868 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:lustre-MDT0001-mdtlov_UUID:3352:1862456998074496:0@lo:38:kworker/u8:2.0 00000100:00100000:1.0:1776178384.739105:0:11281:0:(nrs_fifo.c:179:nrs_fifo_req_get()) NRS start fifo request from 12345-0@lo, seq: 6765 00000100:00100000:1.0:1776178384.739109:0:11281:0:(service.c:2324:ptlrpc_server_handle_request()) Handling RPC req@000000001d51eec7 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_004:0+-99:3352:x1862456998074368:12345-0@lo:38:kworker/u8:2.0 00010000:02000000:1.0:1776178384.739116:0:11281:0:(ldlm_lib.c:1139:target_handle_connect()) lustre-MDT0000: Not available for connect from 0@lo (stopping) 00000100:00100000:2.0:1776178384.739124:0:6255:0:(nrs_fifo.c:179:nrs_fifo_req_get()) NRS start fifo request from 12345-0@lo, seq: 6766 00000100:00100000:2.0:1776178384.739130:0:6255:0:(service.c:2324:ptlrpc_server_handle_request()) Handling RPC req@00000000b766edf6 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_000:0+-99:3352:x1862456998074688:12345-0@lo:38:kworker/u8:2.0 00010000:02000000:2.0:1776178384.739137:0:6255:0:(ldlm_lib.c:1139:target_handle_connect()) lustre-MDT0000: Not available for connect from 0@lo (stopping) 00000100:00100000:1.0:1776178384.739146:0:11281:0:(service.c:2372:ptlrpc_server_handle_request()) Handled RPC req@000000001d51eec7 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_004:0+-99:3352:x1862456998074368:12345-0@lo:38:kworker/u8:2.0 Request processed in 35us (345us total) trans 0 rc -19/-19 00000100:00100000:1.0:1776178384.739153:0:11281:0:(nrs_fifo.c:241:nrs_fifo_req_stop()) NRS stop fifo request from 12345-0@lo, seq: 6765 00000100:00100000:2.0:1776178384.739170:0:6255:0:(service.c:2372:ptlrpc_server_handle_request()) Handled RPC req@00000000b766edf6 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_000:0+-99:3352:x1862456998074688:12345-0@lo:38:kworker/u8:2.0 Request processed in 39us (268us total) trans 0 rc -19/-19 00000100:00100000:2.0:1776178384.739177:0:6255:0:(nrs_fifo.c:241:nrs_fifo_req_stop()) NRS stop fifo request from 12345-0@lo, seq: 6766 00000100:00100000:1.0:1776178384.739184:0:3354:0:(client.c:1733:ptlrpc_send_new_req()) Sending RPC req@00000000812a7ebe pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_01:lustre-MDT0001-mdtlov_UUID:3354:1862456998074624:0@lo:400:kworker/u8:2.0 00000100:00080000:2.0:1776178384.739212:0:3352:0:(import.c:86:import_set_state_nolock()) 00000000a7c55b1f lustre-MDT0000_UUID: changing import state from CONNECTING to DISCONN 00000100:00080000:2.0:1776178384.739216:0:3352:0:(import.c:1429:ptlrpc_connect_interpret()) recovery of lustre-MDT0000_UUID on 192.168.204.103@tcp failed (-19) 00000100:00100000:1.0:1776178384.739221:0:3354:0:(events.c:361:request_in_callback()) peer: 12345-0@lo (source: 12345-0@lo) 00000100:00100000:2.0:1776178384.739223:0:3352:0:(client.c:2215:ptlrpc_check_set()) Completed RPC req@0000000017d54e92 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:lustre-MDT0000-lwp-MDT0001_UUID:3352:1862456998074688:0@lo:38:kworker/u8:2.0 00000100:00080000:2.0:1776178384.739237:0:3352:0:(import.c:86:import_set_state_nolock()) 00000000dfc8db13 lustre-MDT0000_UUID: changing import state from CONNECTING to DISCONN 00000100:00080000:2.0:1776178384.739240:0:3352:0:(import.c:1429:ptlrpc_connect_interpret()) recovery of lustre-MDT0000_UUID on 192.168.204.103@tcp failed (-19) 00000100:00100000:2.0:1776178384.739243:0:3352:0:(client.c:2215:ptlrpc_check_set()) Completed RPC req@00000000cb1b73cb pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:lustre-MDT0000-lwp-OST0001_UUID:3352:1862456998074368:0@lo:38:kworker/u8:2.0 00000100:00100000:1.0:1776178384.739245:0:3354:0:(ptlrpcd.c:415:ptlrpcd_check()) transfer 1 async RPCs [0->1] 00000100:00100000:1.0:1776178384.739253:0:3354:0:(client.c:1733:ptlrpc_send_new_req()) Sending RPC req@000000007d5bffd0 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_01:lustre-MDT0001-mdtlov_UUID:3354:1862456998074560:0@lo:400:kworker/u8:2.0 00000100:00100000:1.0:1776178384.739289:0:3354:0:(events.c:361:request_in_callback()) peer: 12345-0@lo (source: 12345-0@lo) 00000100:00100000:2.0:1776178384.739325:0:20762:0:(service.c:2152:ptlrpc_server_handle_req_in()) got req x1862456998074624 00000100:00100000:1.0:1776178384.739342:0:3356:0:(client.c:2215:ptlrpc_check_set()) Completed RPC req@00000000f9929c4b pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_03:lustre-MDT0001-lwp-OST0000_UUID:3356:1862456998074304:0@lo:400:kworker/u8:2.0 00000100:00100000:2.0:1776178384.739351:0:20762:0:(service.c:2152:ptlrpc_server_handle_req_in()) got req x1862456998074560 00000100:00100000:1.0:1776178384.739358:0:3356:0:(client.c:2215:ptlrpc_check_set()) Completed RPC req@00000000cc80b69e pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_03:lustre-MDT0001-lwp-OST0001_UUID:3356:1862456998074432:0@lo:400:kworker/u8:2.0 00000100:00100000:2.0:1776178384.739379:0:20762:0:(nrs_fifo.c:179:nrs_fifo_req_get()) NRS start fifo request from 12345-0@lo, seq: 653 00000100:00100000:2.0:1776178384.739384:0:20762:0:(service.c:2324:ptlrpc_server_handle_request()) Handling RPC req@000000007b15d0e4 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_013:lustre-MDT0001-mdtlov_UUID+5:3354:x1862456998074624:12345-0@lo:400:kworker/u8:2.0 00000100:00100000:1.0:1776178384.739395:0:20755:0:(nrs_fifo.c:179:nrs_fifo_req_get()) NRS start fifo request from 12345-0@lo, seq: 654 00000100:00100000:1.0:1776178384.739401:0:20755:0:(service.c:2324:ptlrpc_server_handle_request()) Handling RPC req@00000000fed3e037 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_006:lustre-MDT0001-mdtlov_UUID+5:3354:x1862456998074560:12345-0@lo:400:kworker/u8:2.0 00000100:00100000:2.0:1776178384.739434:0:20762:0:(service.c:2372:ptlrpc_server_handle_request()) Handled RPC req@000000007b15d0e4 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_013:lustre-MDT0001-mdtlov_UUID+5:3354:x1862456998074624:12345-0@lo:400:kworker/u8:2.0 Request processed in 49us (216us total) trans 0 rc 0/0 00000100:00100000:2.0:1776178384.739440:0:20762:0:(nrs_fifo.c:241:nrs_fifo_req_stop()) NRS stop fifo request from 12345-0@lo, seq: 653 00000100:00100000:1.0:1776178384.740058:0:20755:0:(service.c:2372:ptlrpc_server_handle_request()) Handled RPC req@00000000fed3e037 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_006:lustre-MDT0001-mdtlov_UUID+5:3354:x1862456998074560:12345-0@lo:400:kworker/u8:2.0 Request processed in 655us (767us total) trans 0 rc 0/0 00000100:00100000:1.0:1776178384.740138:0:20755:0:(nrs_fifo.c:241:nrs_fifo_req_stop()) NRS stop fifo request from 12345-0@lo, seq: 654 00000100:00100000:1.0:1776178384.740173:0:3354:0:(client.c:2215:ptlrpc_check_set()) Completed RPC req@000000007d5bffd0 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_01:lustre-MDT0001-mdtlov_UUID:3354:1862456998074560:0@lo:400:kworker/u8:2.0 00000100:00100000:1.0:1776178384.740183:0:3354:0:(client.c:2215:ptlrpc_check_set()) Completed RPC req@00000000812a7ebe pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_01:lustre-MDT0001-mdtlov_UUID:3354:1862456998074624:0@lo:400:kworker/u8:2.0 00000100:00100000:3.0:1776178384.740697:0:6256:0:(nrs_fifo.c:179:nrs_fifo_req_get()) NRS start fifo request from 12345-0@lo, seq: 6767 00000100:00100000:3.0:1776178384.740702:0:6256:0:(service.c:2324:ptlrpc_server_handle_request()) Handling RPC req@00000000478fd8d5 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_001:0+-99:3352:x1862456998074240:12345-0@lo:38:kworker/u8:2.0 00010000:02000000:3.0:1776178384.740715:0:6256:0:(ldlm_lib.c:1139:target_handle_connect()) lustre-MDT0000: Not available for connect from 0@lo (stopping) 00000100:00100000:3.0:1776178384.742212:0:6256:0:(service.c:2372:ptlrpc_server_handle_request()) Handled RPC req@00000000478fd8d5 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_001:0+-99:3352:x1862456998074240:12345-0@lo:38:kworker/u8:2.0 Request processed in 1501us (3459us total) trans 0 rc -19/-19 00000100:00100000:3.0:1776178384.742226:0:6256:0:(nrs_fifo.c:241:nrs_fifo_req_stop()) NRS stop fifo request from 12345-0@lo, seq: 6767 00000100:00080000:2.0:1776178384.742565:0:3352:0:(import.c:86:import_set_state_nolock()) 0000000047601a98 lustre-MDT0000_UUID: changing import state from CONNECTING to DISCONN 00000100:00080000:2.0:1776178384.742571:0:3352:0:(import.c:1429:ptlrpc_connect_interpret()) recovery of lustre-MDT0000_UUID on 192.168.204.103@tcp failed (-19) 00000100:00100000:2.0:1776178384.742578:0:3352:0:(client.c:2215:ptlrpc_check_set()) Completed RPC req@000000006a209b80 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:lustre-MDT0000-lwp-OST0000_UUID:3352:1862456998074240:0@lo:38:kworker/u8:2.0 00010000:00010000:3.0:1776178385.376350:0:6244:0:(ldlm_lockd.c:2792:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:1.0:1776178386.272429:0:3356:0:(ptlrpcd.c:415:ptlrpcd_check()) transfer 1 async RPCs [2->3] 00000100:00100000:1.0:1776178386.272455:0:3356:0:(client.c:1733:ptlrpc_send_new_req()) Sending RPC req@00000000dd1281fe pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_03:lustre-MDT0001-mdtlov_UUID:3356:1862456998074752:0@lo:13:osp-pre-1-1.0 00000100:00100000:1.0:1776178386.272527:0:3356:0:(events.c:361:request_in_callback()) peer: 12345-0@lo (source: 12345-0@lo) 00000100:00100000:0.0:1776178386.272634:0:9554:0:(service.c:2152:ptlrpc_server_handle_req_in()) got req x1862456998074752 00000100:00100000:0.0:1776178386.272682:0:9554:0:(nrs_fifo.c:179:nrs_fifo_req_get()) NRS start fifo request from 12345-0@lo, seq: 559 00000100:00100000:0.0:1776178386.272688:0:9554:0:(service.c:2324:ptlrpc_server_handle_request()) Handling RPC req@00000000812a7ebe pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_create00:lustre-MDT0001-mdtlov_UUID+5:3356:x1862456998074752:12345-0@lo:13:osp-pre-1-1.0 00002000:00000024:0.0:1776178386.272716:0:9554:0:(ofd_obd.c:740:ofd_statfs()) blocks cached 229376 granted 491746368 pending 0 free 3923689472 avail 3692167168 00000100:00100000:0.0:1776178386.272765:0:9554:0:(service.c:2372:ptlrpc_server_handle_request()) Handled RPC req@00000000812a7ebe pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_create00:lustre-MDT0001-mdtlov_UUID+5:3356:x1862456998074752:12345-0@lo:13:osp-pre-1-1.0 Request processed in 76us (240us total) trans 0 rc 0/0 00000100:00100000:0.0:1776178386.272772:0:9554:0:(nrs_fifo.c:241:nrs_fifo_req_stop()) NRS stop fifo request from 12345-0@lo, seq: 559 00000100:00100000:1.0:1776178386.272850:0:3356:0:(client.c:2215:ptlrpc_check_set()) Completed RPC req@00000000dd1281fe pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_03:lustre-MDT0001-mdtlov_UUID:3356:1862456998074752:0@lo:13:osp-pre-1-1.0 00010000:00010000:2.0:1776178386.400420:0:6245:0:(ldlm_lockd.c:2792:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:1.0:1776178386.786405:0:3356:0:(client.c:1733:ptlrpc_send_new_req()) Sending RPC req@0000000092f98018 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_03:lustre-MDT0001-mdtlov_UUID:3356:1862456998074816:0@lo:13:osp-pre-0-1.0 00000100:00100000:1.0:1776178386.786486:0:3356:0:(events.c:361:request_in_callback()) peer: 12345-0@lo (source: 12345-0@lo) 00000100:00100000:2.0:1776178386.786706:0:8231:0:(service.c:2152:ptlrpc_server_handle_req_in()) got req x1862456998074816 00000100:00100000:2.0:1776178386.786768:0:8231:0:(nrs_fifo.c:179:nrs_fifo_req_get()) NRS start fifo request from 12345-0@lo, seq: 560 00000100:00100000:2.0:1776178386.786774:0:8231:0:(service.c:2324:ptlrpc_server_handle_request()) Handling RPC req@000000007d5bffd0 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_create00:lustre-MDT0001-mdtlov_UUID+5:3356:x1862456998074816:12345-0@lo:13:osp-pre-0-1.0 00002000:00000024:2.0:1776178386.786812:0:8231:0:(ofd_obd.c:740:ofd_statfs()) blocks cached 315392 granted 484883776 pending 0 free 3923689472 avail 3692167168 00000100:00100000:2.0:1776178386.786869:0:8231:0:(service.c:2372:ptlrpc_server_handle_request()) Handled RPC req@000000007d5bffd0 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_create00:lustre-MDT0001-mdtlov_UUID+5:3356:x1862456998074816:12345-0@lo:13:osp-pre-0-1.0 Request processed in 94us (383us total) trans 0 rc 0/0 00000100:00100000:2.0:1776178386.786877:0:8231:0:(nrs_fifo.c:241:nrs_fifo_req_stop()) NRS stop fifo request from 12345-0@lo, seq: 560 00000100:00100000:1.0:1776178386.787186:0:3356:0:(client.c:2215:ptlrpc_check_set()) Completed RPC req@0000000092f98018 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_03:lustre-MDT0001-mdtlov_UUID:3356:1862456998074816:0@lo:13:osp-pre-0-1.0 00010000:00010000:2.0:1776178387.424422:0:9224:0:(ldlm_lockd.c:2792:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:3.0:1776178388.258230:0:3348:0:(events.c:361:request_in_callback()) peer: 12345-192.168.204.3@tcp (source: 12345-192.168.204.3@tcp) 00000100:00100000:3.0:1776178388.258286:0:3348:0:(events.c:361:request_in_callback()) peer: 12345-192.168.204.3@tcp (source: 12345-192.168.204.3@tcp) 00000100:00100000:3.0:1776178388.258322:0:3348:0:(events.c:361:request_in_callback()) peer: 12345-192.168.204.3@tcp (source: 12345-192.168.204.3@tcp) 00000100:00100000:3.0:1776178388.258352:0:3348:0:(events.c:361:request_in_callback()) peer: 12345-192.168.204.3@tcp (source: 12345-192.168.204.3@tcp) 00000100:00100000:3.0:1776178388.258373:0:3348:0:(events.c:361:request_in_callback()) peer: 12345-192.168.204.3@tcp (source: 12345-192.168.204.3@tcp) 00000100:00100000:3.0:1776178388.258413:0:20759:0:(service.c:2152:ptlrpc_server_handle_req_in()) got req x1862456979120704 00000100:00100000:3.0:1776178388.258467:0:20759:0:(service.c:2152:ptlrpc_server_handle_req_in()) got req x1862456979120768 00000100:00100000:3.0:1776178388.258491:0:20759:0:(nrs_fifo.c:179:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.204.3@tcp, seq: 655 00000100:00100000:3.0:1776178388.258498:0:20759:0:(service.c:2324:ptlrpc_server_handle_request()) Handling RPC req@00000000d384e871 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_010:b50963bd-8eba-4432-9a54-8888d00e3fa1+8:2271:x1862456979120704:12345-192.168.204.3@tcp:400:kworker/u8:3.0 00000100:00100000:3.0:1776178388.258572:0:20759:0:(service.c:2372:ptlrpc_server_handle_request()) Handled RPC req@00000000d384e871 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_010:b50963bd-8eba-4432-9a54-8888d00e3fa1+8:2271:x1862456979120704:12345-192.168.204.3@tcp:400:kworker/u8:3.0 Request processed in 73us (249us total) trans 0 rc 0/0 00000100:00100000:3.0:1776178388.258580:0:20759:0:(nrs_fifo.c:241:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.204.3@tcp, seq: 655 00000100:00100000:3.0:1776178388.258598:0:20759:0:(nrs_fifo.c:179:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.204.3@tcp, seq: 656 00000100:00100000:3.0:1776178388.258602:0:20759:0:(service.c:2324:ptlrpc_server_handle_request()) Handling RPC req@0000000047c15ac0 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_010:b50963bd-8eba-4432-9a54-8888d00e3fa1+9:2271:x1862456979120768:12345-192.168.204.3@tcp:400:kworker/u8:3.0 00000100:00100000:3.0:1776178388.258626:0:20759:0:(service.c:2372:ptlrpc_server_handle_request()) Handled RPC req@0000000047c15ac0 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_010:b50963bd-8eba-4432-9a54-8888d00e3fa1+9:2271:x1862456979120768:12345-192.168.204.3@tcp:400:kworker/u8:3.0 Request processed in 23us (253us total) trans 0 rc 0/0 00000100:00100000:3.0:1776178388.258632:0:20759:0:(nrs_fifo.c:241:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.204.3@tcp, seq: 656 00000100:00100000:0.0:1776178388.258869:0:6256:0:(service.c:2152:ptlrpc_server_handle_req_in()) got req x1862456979120576 00000100:00100000:0.0:1776178388.258904:0:6256:0:(service.c:2152:ptlrpc_server_handle_req_in()) got req x1862456979120640 00000100:00100000:2.0:1776178388.258929:0:8250:0:(nrs_fifo.c:179:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.204.3@tcp, seq: 1036 00000100:00100000:2.0:1776178388.258941:0:8250:0:(service.c:2324:ptlrpc_server_handle_request()) Handling RPC req@000000000b5ad4fd pname:cluuid+ref:pid:xid:nid:opc:job mdt00_003:b50963bd-8eba-4432-9a54-8888d00e3fa1+6:2271:x1862456979120640:12345-192.168.204.3@tcp:400:kworker/u8:3.0 00000100:00100000:0.0:1776178388.258942:0:6256:0:(nrs_fifo.c:179:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.204.3@tcp, seq: 6768 00000100:00100000:0.0:1776178388.258951:0:6256:0:(service.c:2324:ptlrpc_server_handle_request()) Handling RPC req@0000000083b11d73 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_001:0+-99:2269:x1862456979120576:12345-192.168.204.3@tcp:38:kworker/u8:3.0 00010000:02000000:0.0:1776178388.258972:0:6256:0:(ldlm_lib.c:1139:target_handle_connect()) lustre-MDT0000: Not available for connect from 192.168.204.3@tcp (stopping) 00000100:00100000:2.0:1776178388.258986:0:8250:0:(service.c:2372:ptlrpc_server_handle_request()) Handled RPC req@000000000b5ad4fd pname:cluuid+ref:pid:xid:nid:opc:job mdt00_003:b50963bd-8eba-4432-9a54-8888d00e3fa1+6:2271:x1862456979120640:12345-192.168.204.3@tcp:400:kworker/u8:3.0 Request processed in 48us (634us total) trans 0 rc 0/0 00000100:00100000:2.0:1776178388.258994:0:8250:0:(nrs_fifo.c:241:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.204.3@tcp, seq: 1036 00000100:00100000:0.0:1776178388.259013:0:6256:0:(service.c:2372:ptlrpc_server_handle_request()) Handled RPC req@0000000083b11d73 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_001:0+-99:2269:x1862456979120576:12345-192.168.204.3@tcp:38:kworker/u8:3.0 Request processed in 61us (791us total) trans 0 rc -19/-19 00000100:00100000:0.0:1776178388.259022:0:6256:0:(nrs_fifo.c:241:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.204.3@tcp, seq: 6768 00000100:00100000:1.0:1776178388.259480:0:20140:0:(service.c:2152:ptlrpc_server_handle_req_in()) got req x1862456979120512 00000100:00100000:1.0:1776178388.259520:0:20140:0:(nrs_fifo.c:179:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.204.3@tcp, seq: 227 00000100:00100000:1.0:1776178388.259524:0:20140:0:(service.c:2324:ptlrpc_server_handle_request()) Handling RPC req@00000000b4107e97 pname:cluuid+ref:pid:xid:nid:opc:job ll_mgs_0002:5b5c66b1-1a2f-4651-8548-0c75b9a38a94+9:2271:x1862456979120512:12345-192.168.204.3@tcp:400:kworker/u8:3.0 00000100:00100000:1.0:1776178388.259563:0:20140:0:(service.c:2372:ptlrpc_server_handle_request()) Handled RPC req@00000000b4107e97 pname:cluuid+ref:pid:xid:nid:opc:job ll_mgs_0002:5b5c66b1-1a2f-4651-8548-0c75b9a38a94+9:2271:x1862456979120512:12345-192.168.204.3@tcp:400:kworker/u8:3.0 Request processed in 37us (1278us total) trans 0 rc 0/0 00000100:00100000:1.0:1776178388.259570:0:20140:0:(nrs_fifo.c:241:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.204.3@tcp, seq: 227 00010000:00010000:2.0:1776178388.448382:0:9225:0:(ldlm_lockd.c:2792:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00010000:00010000:3.0:1776178389.472371:0:6244:0:(ldlm_lockd.c:2792:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00080000:0.0:1776178389.856648:0:36:0:(pinger.c:251:ptlrpc_pinger_process_import()) lustre-MDT0000-lwp-OST0000_UUID->lustre-MDT0000_UUID: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00080000:0.0:1776178389.856659:0:36:0:(recover.c:57:ptlrpc_initiate_recovery()) lustre-MDT0000_UUID: starting recovery 00000100:00080000:0.0:1776178389.856665:0:36:0:(import.c:86:import_set_state_nolock()) 0000000047601a98 lustre-MDT0000_UUID: changing import state from DISCONN to CONNECTING 00000100:00080000:0.0:1776178389.856675:0:36:0:(import.c:534:import_select_connection()) lustre-MDT0000-lwp-OST0000: connect to NID 0@lo last attempt 1092 00000100:00080000:0.0:1776178389.856680:0:36:0:(import.c:581:import_select_connection()) lustre-MDT0000-lwp-OST0000: tried all connections, increasing latency to 21s 00000100:00080000:0.0:1776178389.856688:0:36:0:(import.c:615:import_select_connection()) lustre-MDT0000-lwp-OST0000: import 0000000047601a98 using connection 192.168.204.103@tcp/0@lo 00000100:00100000:0.0:1776178389.856718:0:36:0:(import.c:807:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5) req@0000000092f98018 x1862456998074944/t0(0) o38->lustre-MDT0000-lwp-OST0000@0@lo:12/10 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' 00000100:00100000:1.0:1776178389.856727:0:3354:0:(ptlrpcd.c:415:ptlrpcd_check()) transfer 1 async RPCs [0->1] 00000100:00100000:1.0:1776178389.856755:0:3354:0:(client.c:1733:ptlrpc_send_new_req()) Sending RPC req@0000000063c20b43 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_01:ac61f839-0e8e-4cac-9c59-08bcfbbad330:3354:1862456998074880:0@lo:400:kworker/u8:1.0 00000100:00080000:0.0:1776178389.856768:0:36:0:(pinger.c:251:ptlrpc_pinger_process_import()) lustre-MDT0000-lwp-OST0001_UUID->lustre-MDT0000_UUID: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00080000:0.0:1776178389.856771:0:36:0:(recover.c:57:ptlrpc_initiate_recovery()) lustre-MDT0000_UUID: starting recovery 00000100:00080000:0.0:1776178389.856773:0:36:0:(import.c:86:import_set_state_nolock()) 00000000dfc8db13 lustre-MDT0000_UUID: changing import state from DISCONN to CONNECTING 00000100:00080000:0.0:1776178389.856777:0:36:0:(import.c:534:import_select_connection()) lustre-MDT0000-lwp-OST0001: connect to NID 0@lo last attempt 1092 00000100:00080000:0.0:1776178389.856779:0:36:0:(import.c:581:import_select_connection()) lustre-MDT0000-lwp-OST0001: tried all connections, increasing latency to 21s 00000100:00080000:0.0:1776178389.856783:0:36:0:(import.c:615:import_select_connection()) lustre-MDT0000-lwp-OST0001: import 00000000dfc8db13 using connection 192.168.204.103@tcp/0@lo 00000100:00100000:0.0:1776178389.856796:0:36:0:(import.c:807:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5) req@00000000380b40c4 x1862456998075072/t0(0) o38->lustre-MDT0000-lwp-OST0001@0@lo:12/10 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' 00000100:00100000:1.0:1776178389.856834:0:3354:0:(events.c:361:request_in_callback()) peer: 12345-0@lo (source: 12345-0@lo) 00000100:00080000:0.0:1776178389.856838:0:36:0:(pinger.c:251:ptlrpc_pinger_process_import()) lustre-MDT0001-mdtlov_UUID->lustre-MDT0000_UUID: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00080000:0.0:1776178389.856843:0:36:0:(recover.c:57:ptlrpc_initiate_recovery()) lustre-MDT0000_UUID: starting recovery 00000100:00080000:0.0:1776178389.856845:0:36:0:(import.c:86:import_set_state_nolock()) 00000000ba4f1821 lustre-MDT0000_UUID: changing import state from DISCONN to CONNECTING 00000100:00080000:0.0:1776178389.856850:0:36:0:(import.c:534:import_select_connection()) lustre-MDT0000-osp-MDT0001: connect to NID 0@lo last attempt 1092 00000100:00080000:0.0:1776178389.856852:0:36:0:(import.c:581:import_select_connection()) lustre-MDT0000-osp-MDT0001: tried all connections, increasing latency to 21s 00000100:00080000:0.0:1776178389.856856:0:36:0:(import.c:615:import_select_connection()) lustre-MDT0000-osp-MDT0001: import 00000000ba4f1821 using connection 192.168.204.103@tcp/0@lo 00000100:00100000:1.0:1776178389.856862:0:3354:0:(client.c:1733:ptlrpc_send_new_req()) Sending RPC req@000000001f101832 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_01:lustre-MDT0001-lwp-OST0000_UUID:3354:1862456998075008:0@lo:400:kworker/u8:1.0 00000100:00100000:0.0:1776178389.856870:0:36:0:(import.c:807:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5) req@0000000098e424d0 x1862456998075200/t0(0) o38->lustre-MDT0000-osp-MDT0001@0@lo:24/4 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' 00000100:00100000:1.0:1776178389.856890:0:3354:0:(events.c:361:request_in_callback()) peer: 12345-0@lo (source: 12345-0@lo) 00000100:00080000:0.0:1776178389.856899:0:36:0:(pinger.c:251:ptlrpc_pinger_process_import()) lustre-MDT0000-lwp-MDT0001_UUID->lustre-MDT0000_UUID: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00080000:0.0:1776178389.856903:0:36:0:(recover.c:57:ptlrpc_initiate_recovery()) lustre-MDT0000_UUID: starting recovery 00000100:00080000:0.0:1776178389.856904:0:36:0:(import.c:86:import_set_state_nolock()) 00000000a7c55b1f lustre-MDT0000_UUID: changing import state from DISCONN to CONNECTING 00000100:00100000:1.0:1776178389.856905:0:3354:0:(ptlrpcd.c:415:ptlrpcd_check()) transfer 1 async RPCs [0->1] 00000100:00080000:0.0:1776178389.856910:0:36:0:(import.c:534:import_select_connection()) lustre-MDT0000-lwp-MDT0001: connect to NID 0@lo last attempt 1092 00000100:00100000:1.0:1776178389.856913:0:3354:0:(client.c:1733:ptlrpc_send_new_req()) Sending RPC req@000000000e0e1967 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_01:lustre-MDT0001-mdtlov_UUID:3354:1862456998075328:0@lo:400:kworker/u8:1.0 00000100:00080000:0.0:1776178389.856914:0:36:0:(import.c:581:import_select_connection()) lustre-MDT0000-lwp-MDT0001: tried all connections, increasing latency to 21s 00000100:00080000:0.0:1776178389.856919:0:36:0:(import.c:615:import_select_connection()) lustre-MDT0000-lwp-MDT0001: import 00000000a7c55b1f using connection 192.168.204.103@tcp/0@lo 00000100:00100000:0.0:1776178389.856933:0:36:0:(import.c:807:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5) req@0000000090a7aa50 x1862456998075392/t0(0) o38->lustre-MDT0000-lwp-MDT0001@0@lo:12/10 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' 00000100:00100000:1.0:1776178389.856941:0:3354:0:(events.c:361:request_in_callback()) peer: 12345-0@lo (source: 12345-0@lo) 00000100:00100000:1.0:1776178389.856972:0:3356:0:(client.c:1733:ptlrpc_send_new_req()) Sending RPC req@000000006b0ae3d8 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_03:lustre-MDT0001-mdtlov_UUID:3356:1862456998075264:0@lo:400:kworker/u8:1.0 00000100:00100000:1.0:1776178389.856997:0:3356:0:(events.c:361:request_in_callback()) peer: 12345-0@lo (source: 12345-0@lo) 00000100:00100000:1.0:1776178389.857017:0:3356:0:(ptlrpcd.c:415:ptlrpcd_check()) transfer 1 async RPCs [2->3] 00000100:00100000:1.0:1776178389.857021:0:3356:0:(client.c:1733:ptlrpc_send_new_req()) Sending RPC req@000000007d5cfd93 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_03:lustre-MDT0001-lwp-OST0001_UUID:3356:1862456998075136:0@lo:400:kworker/u8:1.0 00000100:00100000:1.0:1776178389.857043:0:3356:0:(events.c:361:request_in_callback()) peer: 12345-0@lo (source: 12345-0@lo) 00000100:00100000:3.0:1776178389.857145:0:3352:0:(client.c:1733:ptlrpc_send_new_req()) Sending RPC req@0000000092f98018 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:lustre-MDT0000-lwp-OST0000_UUID:3352:1862456998074944:0@lo:38:kworker/u8:1.0 00000100:00100000:3.0:1776178389.857184:0:3352:0:(events.c:361:request_in_callback()) peer: 12345-0@lo (source: 12345-0@lo) 00000100:00100000:2.0:1776178389.857198:0:20137:0:(service.c:2152:ptlrpc_server_handle_req_in()) got req x1862456998074880 00000100:00100000:3.0:1776178389.857200:0:3352:0:(client.c:1733:ptlrpc_send_new_req()) Sending RPC req@00000000380b40c4 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:lustre-MDT0000-lwp-OST0001_UUID:3352:1862456998075072:0@lo:38:kworker/u8:1.0 00000100:00100000:3.0:1776178389.857216:0:3352:0:(events.c:361:request_in_callback()) peer: 12345-0@lo (source: 12345-0@lo) 00000100:00100000:3.0:1776178389.857230:0:3352:0:(client.c:1733:ptlrpc_send_new_req()) Sending RPC req@0000000098e424d0 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:lustre-MDT0001-mdtlov_UUID:3352:1862456998075200:0@lo:38:kworker/u8:1.0 00000100:00100000:2.0:1776178389.857244:0:20137:0:(nrs_fifo.c:179:nrs_fifo_req_get()) NRS start fifo request from 12345-0@lo, seq: 228 00000100:00100000:3.0:1776178389.857251:0:3352:0:(events.c:361:request_in_callback()) peer: 12345-0@lo (source: 12345-0@lo) 00000100:00100000:2.0:1776178389.857252:0:20137:0:(service.c:2324:ptlrpc_server_handle_request()) Handling RPC req@00000000b42c55dd pname:cluuid+ref:pid:xid:nid:opc:job ll_mgs_0000:ac61f839-0e8e-4cac-9c59-08bcfbbad330+18:3354:x1862456998074880:12345-0@lo:400:kworker/u8:1.0 00000100:00100000:3.0:1776178389.857266:0:3352:0:(client.c:1733:ptlrpc_send_new_req()) Sending RPC req@0000000090a7aa50 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:lustre-MDT0000-lwp-MDT0001_UUID:3352:1862456998075392:0@lo:38:kworker/u8:1.0 00000100:00100000:3.0:1776178389.857285:0:3352:0:(events.c:361:request_in_callback()) peer: 12345-0@lo (source: 12345-0@lo) 00000100:00100000:3.0:1776178389.857323:0:6256:0:(service.c:2152:ptlrpc_server_handle_req_in()) got req x1862456998075008 00000100:00100000:3.0:1776178389.857348:0:6256:0:(service.c:2152:ptlrpc_server_handle_req_in()) got req x1862456998075136 00000100:00100000:3.0:1776178389.857358:0:6256:0:(service.c:2152:ptlrpc_server_handle_req_in()) got req x1862456998074944 00000100:00100000:3.0:1776178389.857367:0:6256:0:(service.c:2152:ptlrpc_server_handle_req_in()) got req x1862456998075072 00000100:00100000:3.0:1776178389.857374:0:6256:0:(service.c:2152:ptlrpc_server_handle_req_in()) got req x1862456998075392 00000100:00100000:3.0:1776178389.857384:0:6256:0:(nrs_fifo.c:179:nrs_fifo_req_get()) NRS start fifo request from 12345-0@lo, seq: 1037 00000100:00100000:3.0:1776178389.857387:0:6256:0:(service.c:2324:ptlrpc_server_handle_request()) Handling RPC req@00000000cc80b69e pname:cluuid+ref:pid:xid:nid:opc:job mdt00_001:lustre-MDT0001-lwp-OST0000_UUID+5:3354:x1862456998075008:12345-0@lo:400:kworker/u8:1.0 00000100:00100000:0.0:1776178389.857426:0:20755:0:(service.c:2152:ptlrpc_server_handle_req_in()) got req x1862456998075328 00000100:00100000:3.0:1776178389.857451:0:6256:0:(service.c:2372:ptlrpc_server_handle_request()) Handled RPC req@00000000cc80b69e pname:cluuid+ref:pid:xid:nid:opc:job mdt00_001:lustre-MDT0001-lwp-OST0000_UUID+5:3354:x1862456998075008:12345-0@lo:400:kworker/u8:1.0 Request processed in 62us (562us total) trans 0 rc 0/0 00000100:00100000:3.0:1776178389.857459:0:6256:0:(nrs_fifo.c:241:nrs_fifo_req_stop()) NRS stop fifo request from 12345-0@lo, seq: 1037 00000100:00100000:3.0:1776178389.857482:0:6256:0:(nrs_fifo.c:179:nrs_fifo_req_get()) NRS start fifo request from 12345-0@lo, seq: 1038 00000100:00100000:3.0:1776178389.857486:0:6256:0:(service.c:2324:ptlrpc_server_handle_request()) Handling RPC req@000000009016e75e pname:cluuid+ref:pid:xid:nid:opc:job mdt00_001:lustre-MDT0001-lwp-OST0001_UUID+5:3356:x1862456998075136:12345-0@lo:400:kworker/u8:1.0 00000100:00100000:3.0:1776178389.857516:0:6256:0:(service.c:2372:ptlrpc_server_handle_request()) Handled RPC req@000000009016e75e pname:cluuid+ref:pid:xid:nid:opc:job mdt00_001:lustre-MDT0001-lwp-OST0001_UUID+5:3356:x1862456998075136:12345-0@lo:400:kworker/u8:1.0 Request processed in 29us (472us total) trans 0 rc 0/0 00000100:00100000:3.0:1776178389.857522:0:6256:0:(nrs_fifo.c:241:nrs_fifo_req_stop()) NRS stop fifo request from 12345-0@lo, seq: 1038 00000100:00100000:3.0:1776178389.857532:0:6256:0:(nrs_fifo.c:179:nrs_fifo_req_get()) NRS start fifo request from 12345-0@lo, seq: 6769 00000100:00100000:3.0:1776178389.857535:0:6256:0:(service.c:2324:ptlrpc_server_handle_request()) Handling RPC req@00000000952003a4 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_001:0+-99:3352:x1862456998074944:12345-0@lo:38:kworker/u8:1.0 00010000:02000000:3.0:1776178389.857551:0:6256:0:(ldlm_lib.c:1139:target_handle_connect()) lustre-MDT0000: Not available for connect from 0@lo (stopping) 00000100:00100000:3.0:1776178389.857582:0:6256:0:(service.c:2372:ptlrpc_server_handle_request()) Handled RPC req@00000000952003a4 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_001:0+-99:3352:x1862456998074944:12345-0@lo:38:kworker/u8:1.0 Request processed in 46us (399us total) trans 0 rc -19/-19 00000100:00100000:3.0:1776178389.857589:0:6256:0:(nrs_fifo.c:241:nrs_fifo_req_stop()) NRS stop fifo request from 12345-0@lo, seq: 6769 00000100:00100000:3.0:1776178389.857599:0:6256:0:(nrs_fifo.c:179:nrs_fifo_req_get()) NRS start fifo request from 12345-0@lo, seq: 6770 00000100:00100000:3.0:1776178389.857602:0:6256:0:(service.c:2324:ptlrpc_server_handle_request()) Handling RPC req@000000006342fd4b pname:cluuid+ref:pid:xid:nid:opc:job mdt00_001:0+-99:3352:x1862456998075072:12345-0@lo:38:kworker/u8:1.0 00010000:02000000:3.0:1776178389.857609:0:6256:0:(ldlm_lib.c:1139:target_handle_connect()) lustre-MDT0000: Not available for connect from 0@lo (stopping) 00000100:00100000:2.0:1776178389.857627:0:20137:0:(service.c:2372:ptlrpc_server_handle_request()) Handled RPC req@00000000b42c55dd pname:cluuid+ref:pid:xid:nid:opc:job ll_mgs_0000:ac61f839-0e8e-4cac-9c59-08bcfbbad330+18:3354:x1862456998074880:12345-0@lo:400:kworker/u8:1.0 Request processed in 376us (796us total) trans 0 rc 0/0 00000100:00100000:3.0:1776178389.857627:0:6256:0:(service.c:2372:ptlrpc_server_handle_request()) Handled RPC req@000000006342fd4b pname:cluuid+ref:pid:xid:nid:opc:job mdt00_001:0+-99:3352:x1862456998075072:12345-0@lo:38:kworker/u8:1.0 Request processed in 24us (410us total) trans 0 rc -19/-19 00000100:00100000:3.0:1776178389.857632:0:6256:0:(nrs_fifo.c:241:nrs_fifo_req_stop()) NRS stop fifo request from 12345-0@lo, seq: 6770 00000100:00100000:2.0:1776178389.857633:0:20137:0:(nrs_fifo.c:241:nrs_fifo_req_stop()) NRS stop fifo request from 12345-0@lo, seq: 228 00000100:00100000:3.0:1776178389.857644:0:6256:0:(nrs_fifo.c:179:nrs_fifo_req_get()) NRS start fifo request from 12345-0@lo, seq: 6771 00000100:00100000:3.0:1776178389.857647:0:6256:0:(service.c:2324:ptlrpc_server_handle_request()) Handling RPC req@0000000028422d92 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_001:0+-99:3352:x1862456998075392:12345-0@lo:38:kworker/u8:1.0 00010000:02000000:3.0:1776178389.857653:0:6256:0:(ldlm_lib.c:1139:target_handle_connect()) lustre-MDT0000: Not available for connect from 0@lo (stopping) 00000100:00100000:0.0:1776178389.857670:0:20755:0:(service.c:2152:ptlrpc_server_handle_req_in()) got req x1862456998075264 00000100:00100000:3.0:1776178389.857675:0:6256:0:(service.c:2372:ptlrpc_server_handle_request()) Handled RPC req@0000000028422d92 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_001:0+-99:3352:x1862456998075392:12345-0@lo:38:kworker/u8:1.0 Request processed in 26us (389us total) trans 0 rc -19/-19 00000100:00100000:3.0:1776178389.857681:0:6256:0:(nrs_fifo.c:241:nrs_fifo_req_stop()) NRS stop fifo request from 12345-0@lo, seq: 6771 00000100:00100000:2.0:1776178389.857686:0:14128:0:(service.c:2152:ptlrpc_server_handle_req_in()) got req x1862456998075200 00000100:00100000:0.0:1776178389.857704:0:20755:0:(nrs_fifo.c:179:nrs_fifo_req_get()) NRS start fifo request from 12345-0@lo, seq: 657 00000100:00100000:0.0:1776178389.857711:0:20755:0:(service.c:2324:ptlrpc_server_handle_request()) Handling RPC req@00000000f9929c4b pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_006:lustre-MDT0001-mdtlov_UUID+5:3354:x1862456998075328:12345-0@lo:400:kworker/u8:1.0 00000100:00100000:2.0:1776178389.857716:0:14128:0:(nrs_fifo.c:179:nrs_fifo_req_get()) NRS start fifo request from 12345-0@lo, seq: 744 00000100:00100000:2.0:1776178389.857721:0:14128:0:(service.c:2324:ptlrpc_server_handle_request()) Handling RPC req@000000008c3fbdb6 pname:cluuid+ref:pid:xid:nid:opc:job mdt_out00_002:0+-99:3352:x1862456998075200:12345-0@lo:38:kworker/u8:1.0 00000100:00100000:3.0:1776178389.857726:0:3356:0:(client.c:2215:ptlrpc_check_set()) Completed RPC req@000000007d5cfd93 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_03:lustre-MDT0001-lwp-OST0001_UUID:3356:1862456998075136:0@lo:400:kworker/u8:1.0 00010000:02000000:2.0:1776178389.857727:0:14128:0:(ldlm_lib.c:1139:target_handle_connect()) lustre-MDT0000: Not available for connect from 0@lo (stopping) 00000100:00080000:3.0:1776178389.857761:0:3352:0:(import.c:86:import_set_state_nolock()) 0000000047601a98 lustre-MDT0000_UUID: changing import state from CONNECTING to DISCONN 00000100:00080000:3.0:1776178389.857765:0:3352:0:(import.c:1429:ptlrpc_connect_interpret()) recovery of lustre-MDT0000_UUID on 192.168.204.103@tcp failed (-19) 00000100:00100000:3.0:1776178389.857773:0:3352:0:(client.c:2215:ptlrpc_check_set()) Completed RPC req@0000000092f98018 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:lustre-MDT0000-lwp-OST0000_UUID:3352:1862456998074944:0@lo:38:kworker/u8:1.0 00000100:00100000:0.0:1776178389.857781:0:20755:0:(service.c:2372:ptlrpc_server_handle_request()) Handled RPC req@00000000f9929c4b pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_006:lustre-MDT0001-mdtlov_UUID+5:3354:x1862456998075328:12345-0@lo:400:kworker/u8:1.0 Request processed in 67us (839us total) trans 0 rc 0/0 00000100:00100000:2.0:1776178389.857782:0:14128:0:(service.c:2372:ptlrpc_server_handle_request()) Handled RPC req@000000008c3fbdb6 pname:cluuid+ref:pid:xid:nid:opc:job mdt_out00_002:0+-99:3352:x1862456998075200:12345-0@lo:38:kworker/u8:1.0 Request processed in 58us (528us total) trans 0 rc -19/-19 00000100:00080000:3.0:1776178389.857782:0:3352:0:(import.c:86:import_set_state_nolock()) 00000000dfc8db13 lustre-MDT0000_UUID: changing import state from CONNECTING to DISCONN 00000100:00080000:3.0:1776178389.857787:0:3352:0:(import.c:1429:ptlrpc_connect_interpret()) recovery of lustre-MDT0000_UUID on 192.168.204.103@tcp failed (-19) 00000100:00100000:3.0:1776178389.857790:0:3352:0:(client.c:2215:ptlrpc_check_set()) Completed RPC req@00000000380b40c4 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:lustre-MDT0000-lwp-OST0001_UUID:3352:1862456998075072:0@lo:38:kworker/u8:1.0 00000100:00100000:2.0:1776178389.857795:0:14128:0:(nrs_fifo.c:241:nrs_fifo_req_stop()) NRS stop fifo request from 12345-0@lo, seq: 744 00000100:00100000:0.0:1776178389.857799:0:20755:0:(nrs_fifo.c:241:nrs_fifo_req_stop()) NRS stop fifo request from 12345-0@lo, seq: 657 00000100:00080000:3.0:1776178389.857801:0:3352:0:(import.c:86:import_set_state_nolock()) 00000000ba4f1821 lustre-MDT0000_UUID: changing import state from CONNECTING to DISCONN 00000100:00080000:3.0:1776178389.857804:0:3352:0:(import.c:1429:ptlrpc_connect_interpret()) recovery of lustre-MDT0000_UUID on 192.168.204.103@tcp failed (-19) 00000100:00100000:3.0:1776178389.857808:0:3352:0:(client.c:2215:ptlrpc_check_set()) Completed RPC req@0000000098e424d0 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:lustre-MDT0001-mdtlov_UUID:3352:1862456998075200:0@lo:38:kworker/u8:1.0 00000100:00080000:3.0:1776178389.857818:0:3352:0:(import.c:86:import_set_state_nolock()) 00000000a7c55b1f lustre-MDT0000_UUID: changing import state from CONNECTING to DISCONN 00000100:00080000:3.0:1776178389.857820:0:3352:0:(import.c:1429:ptlrpc_connect_interpret()) recovery of lustre-MDT0000_UUID on 192.168.204.103@tcp failed (-19) 00000100:00100000:3.0:1776178389.857822:0:3352:0:(client.c:2215:ptlrpc_check_set()) Completed RPC req@0000000090a7aa50 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:lustre-MDT0000-lwp-MDT0001_UUID:3352:1862456998075392:0@lo:38:kworker/u8:1.0 00000100:00100000:0.0:1776178389.857832:0:20755:0:(nrs_fifo.c:179:nrs_fifo_req_get()) NRS start fifo request from 12345-0@lo, seq: 658 00000100:00100000:2.0:1776178389.857839:0:3354:0:(client.c:2215:ptlrpc_check_set()) Completed RPC req@000000000e0e1967 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_01:lustre-MDT0001-mdtlov_UUID:3354:1862456998075328:0@lo:400:kworker/u8:1.0 00000100:00100000:0.0:1776178389.857840:0:20755:0:(service.c:2324:ptlrpc_server_handle_request()) Handling RPC req@00000000a62daf9a pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_006:lustre-MDT0001-mdtlov_UUID+5:3356:x1862456998075264:12345-0@lo:400:kworker/u8:1.0 00000100:00100000:2.0:1776178389.857859:0:3354:0:(client.c:2215:ptlrpc_check_set()) Completed RPC req@000000001f101832 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_01:lustre-MDT0001-lwp-OST0000_UUID:3354:1862456998075008:0@lo:400:kworker/u8:1.0 00000100:00100000:2.0:1776178389.857868:0:3354:0:(client.c:2215:ptlrpc_check_set()) Completed RPC req@0000000063c20b43 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_01:ac61f839-0e8e-4cac-9c59-08bcfbbad330:3354:1862456998074880:0@lo:400:kworker/u8:1.0 00000100:00100000:0.0:1776178389.857891:0:20755:0:(service.c:2372:ptlrpc_server_handle_request()) Handled RPC req@00000000a62daf9a pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_006:lustre-MDT0001-mdtlov_UUID+5:3356:x1862456998075264:12345-0@lo:400:kworker/u8:1.0 Request processed in 53us (893us total) trans 0 rc 0/0 00000100:00100000:3.0:1776178389.857897:0:3356:0:(client.c:2215:ptlrpc_check_set()) Completed RPC req@000000006b0ae3d8 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_03:lustre-MDT0001-mdtlov_UUID:3356:1862456998075264:0@lo:400:kworker/u8:1.0 00000100:00100000:0.0:1776178389.857901:0:20755:0:(nrs_fifo.c:241:nrs_fifo_req_stop()) NRS stop fifo request from 12345-0@lo, seq: 658 00010000:00010000:2.0:1776178390.499720:0:6245:0:(ldlm_lockd.c:2792:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:3.0:1776178391.392709:0:3353:0:(ptlrpcd.c:415:ptlrpcd_check()) transfer 1 async RPCs [1->0] 00000100:00100000:3.0:1776178391.396919:0:3353:0:(client.c:1733:ptlrpc_send_new_req()) Sending RPC req@00000000e64148ab pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_00:lustre-MDT0001-mdtlov_UUID:3353:1862456998075456:0@lo:13:osp-pre-1-1.0 00000100:00100000:3.0:1776178391.397045:0:3353:0:(events.c:361:request_in_callback()) peer: 12345-0@lo (source: 12345-0@lo) 00000100:00100000:0.0:1776178391.397483:0:9554:0:(service.c:2152:ptlrpc_server_handle_req_in()) got req x1862456998075456 00000100:00100000:0.0:1776178391.397664:0:9554:0:(nrs_fifo.c:179:nrs_fifo_req_get()) NRS start fifo request from 12345-0@lo, seq: 561 00000100:00100000:0.0:1776178391.397674:0:9554:0:(service.c:2324:ptlrpc_server_handle_request()) Handling RPC req@00000000ec65370c pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_create00:lustre-MDT0001-mdtlov_UUID+5:3353:x1862456998075456:12345-0@lo:13:osp-pre-1-1.0 00002000:00000024:0.0:1776178391.397727:0:9554:0:(ofd_obd.c:740:ofd_statfs()) blocks cached 229376 granted 491746368 pending 0 free 3923689472 avail 3692167168 00000100:00100000:0.0:1776178391.397809:0:9554:0:(service.c:2372:ptlrpc_server_handle_request()) Handled RPC req@00000000ec65370c pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_create00:lustre-MDT0001-mdtlov_UUID+5:3353:x1862456998075456:12345-0@lo:13:osp-pre-1-1.0 Request processed in 135us (766us total) trans 0 rc 0/0 00000100:00100000:0.0:1776178391.397816:0:9554:0:(nrs_fifo.c:241:nrs_fifo_req_stop()) NRS stop fifo request from 12345-0@lo, seq: 561 00000100:00100000:3.0:1776178391.397941:0:3353:0:(client.c:2215:ptlrpc_check_set()) Completed RPC req@00000000e64148ab pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_00:lustre-MDT0001-mdtlov_UUID:3353:1862456998075456:0@lo:13:osp-pre-1-1.0 00010000:00010000:2.0:1776178391.521357:0:9224:0:(ldlm_lockd.c:2792:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:3.0:1776178391.904615:0:3355:0:(client.c:1733:ptlrpc_send_new_req()) Sending RPC req@000000006b0ae3d8 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_02:lustre-MDT0001-mdtlov_UUID:3355:1862456998075520:0@lo:13:osp-pre-0-1.0 00000100:00100000:3.0:1776178391.904679:0:3355:0:(events.c:361:request_in_callback()) peer: 12345-0@lo (source: 12345-0@lo) 00000100:00100000:0.0:1776178391.904772:0:9554:0:(service.c:2152:ptlrpc_server_handle_req_in()) got req x1862456998075520 00000100:00100000:0.0:1776178391.904817:0:9554:0:(nrs_fifo.c:179:nrs_fifo_req_get()) NRS start fifo request from 12345-0@lo, seq: 562 00000100:00100000:0.0:1776178391.904823:0:9554:0:(service.c:2324:ptlrpc_server_handle_request()) Handling RPC req@0000000063c20b43 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_create00:lustre-MDT0001-mdtlov_UUID+5:3355:x1862456998075520:12345-0@lo:13:osp-pre-0-1.0 00002000:00000024:0.0:1776178391.904856:0:9554:0:(ofd_obd.c:740:ofd_statfs()) blocks cached 315392 granted 484883776 pending 0 free 3923689472 avail 3692167168 00000100:00100000:0.0:1776178391.904893:0:9554:0:(service.c:2372:ptlrpc_server_handle_request()) Handled RPC req@0000000063c20b43 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_create00:lustre-MDT0001-mdtlov_UUID+5:3355:x1862456998075520:12345-0@lo:13:osp-pre-0-1.0 Request processed in 70us (215us total) trans 0 rc 0/0 00000100:00100000:0.0:1776178391.904899:0:9554:0:(nrs_fifo.c:241:nrs_fifo_req_stop()) NRS stop fifo request from 12345-0@lo, seq: 562 00000100:00100000:3.0:1776178391.906987:0:3355:0:(client.c:2215:ptlrpc_check_set()) Completed RPC req@000000006b0ae3d8 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_02:lustre-MDT0001-mdtlov_UUID:3355:1862456998075520:0@lo:13:osp-pre-0-1.0 00010000:00010000:2.0:1776178392.551887:0:9225:0:(ldlm_lockd.c:2792:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:3.0:1776178393.380589:0:3348:0:(events.c:361:request_in_callback()) peer: 12345-192.168.204.3@tcp (source: 12345-192.168.204.3@tcp) 00000100:00100000:3.0:1776178393.380696:0:3348:0:(events.c:361:request_in_callback()) peer: 12345-192.168.204.3@tcp (source: 12345-192.168.204.3@tcp) 00000100:00100000:3.0:1776178393.380733:0:3348:0:(events.c:361:request_in_callback()) peer: 12345-192.168.204.3@tcp (source: 12345-192.168.204.3@tcp) 00000100:00100000:3.0:1776178393.380768:0:3348:0:(events.c:361:request_in_callback()) peer: 12345-192.168.204.3@tcp (source: 12345-192.168.204.3@tcp) 00000100:00100000:3.0:1776178393.380826:0:3348:0:(events.c:361:request_in_callback()) peer: 12345-192.168.204.3@tcp (source: 12345-192.168.204.3@tcp) 00000100:00100000:3.0:1776178393.380873:0:6257:0:(service.c:2152:ptlrpc_server_handle_req_in()) got req x1862456979120960 00000100:00100000:3.0:1776178393.380935:0:6257:0:(service.c:2152:ptlrpc_server_handle_req_in()) got req x1862456979120896 00000100:00100000:3.0:1776178393.380952:0:6257:0:(nrs_fifo.c:179:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.204.3@tcp, seq: 1039 00000100:00100000:3.0:1776178393.380960:0:6257:0:(service.c:2324:ptlrpc_server_handle_request()) Handling RPC req@00000000699e66b5 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_002:b50963bd-8eba-4432-9a54-8888d00e3fa1+6:2271:x1862456979120960:12345-192.168.204.3@tcp:400:kworker/u8:3.0 00000100:00100000:3.0:1776178393.381057:0:6257:0:(service.c:2372:ptlrpc_server_handle_request()) Handled RPC req@00000000699e66b5 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_002:b50963bd-8eba-4432-9a54-8888d00e3fa1+6:2271:x1862456979120960:12345-192.168.204.3@tcp:400:kworker/u8:3.0 Request processed in 97us (323us total) trans 0 rc 0/0 00000100:00100000:3.0:1776178393.381107:0:6257:0:(nrs_fifo.c:241:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.204.3@tcp, seq: 1039 00000100:00100000:3.0:1776178393.381128:0:6257:0:(nrs_fifo.c:179:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.204.3@tcp, seq: 6772 00000100:00100000:3.0:1776178393.381132:0:6257:0:(service.c:2324:ptlrpc_server_handle_request()) Handling RPC req@00000000ed90ff75 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_002:0+-99:2269:x1862456979120896:12345-192.168.204.3@tcp:38:kworker/u8:3.0 00010000:02000000:3.0:1776178393.381150:0:6257:0:(ldlm_lib.c:1139:target_handle_connect()) lustre-MDT0000: Not available for connect from 192.168.204.3@tcp (stopping) 00000100:00100000:3.0:1776178393.381178:0:6257:0:(service.c:2372:ptlrpc_server_handle_request()) Handled RPC req@00000000ed90ff75 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_002:0+-99:2269:x1862456979120896:12345-192.168.204.3@tcp:38:kworker/u8:3.0 Request processed in 46us (354us total) trans 0 rc -19/-19 00000100:00100000:3.0:1776178393.381186:0:6257:0:(nrs_fifo.c:241:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.204.3@tcp, seq: 6772 00000100:00100000:0.0:1776178393.383485:0:20755:0:(service.c:2152:ptlrpc_server_handle_req_in()) got req x1862456979121024 00000100:00100000:0.0:1776178393.383577:0:20755:0:(service.c:2152:ptlrpc_server_handle_req_in()) got req x1862456979121088 00000100:00100000:0.0:1776178393.383609:0:20755:0:(nrs_fifo.c:179:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.204.3@tcp, seq: 659 00000100:00100000:0.0:1776178393.383615:0:20755:0:(service.c:2324:ptlrpc_server_handle_request()) Handling RPC req@0000000092f98018 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_006:b50963bd-8eba-4432-9a54-8888d00e3fa1+8:2271:x1862456979121024:12345-192.168.204.3@tcp:400:kworker/u8:3.0 00000100:00100000:0.0:1776178393.383704:0:20755:0:(service.c:2372:ptlrpc_server_handle_request()) Handled RPC req@0000000092f98018 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_006:b50963bd-8eba-4432-9a54-8888d00e3fa1+8:2271:x1862456979121024:12345-192.168.204.3@tcp:400:kworker/u8:3.0 Request processed in 89us (3009us total) trans 0 rc 0/0 00000100:00100000:0.0:1776178393.383714:0:20755:0:(nrs_fifo.c:241:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.204.3@tcp, seq: 659 00000100:00100000:0.0:1776178393.383736:0:20755:0:(nrs_fifo.c:179:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.204.3@tcp, seq: 660 00000100:00100000:0.0:1776178393.383740:0:20755:0:(service.c:2324:ptlrpc_server_handle_request()) Handling RPC req@00000000a6227918 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_006:b50963bd-8eba-4432-9a54-8888d00e3fa1+9:2271:x1862456979121088:12345-192.168.204.3@tcp:400:kworker/u8:3.0 00000100:00100000:0.0:1776178393.383762:0:20755:0:(service.c:2372:ptlrpc_server_handle_request()) Handled RPC req@00000000a6227918 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_006:b50963bd-8eba-4432-9a54-8888d00e3fa1+9:2271:x1862456979121088:12345-192.168.204.3@tcp:400:kworker/u8:3.0 Request processed in 21us (2994us total) trans 0 rc 0/0 00000100:00100000:0.0:1776178393.383767:0:20755:0:(nrs_fifo.c:241:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.204.3@tcp, seq: 660 00000100:00100000:2.0:1776178393.392135:0:20140:0:(service.c:2152:ptlrpc_server_handle_req_in()) got req x1862456979120832 00000100:00100000:2.0:1776178393.394104:0:20140:0:(nrs_fifo.c:179:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.204.3@tcp, seq: 229 00000100:00100000:2.0:1776178393.394118:0:20140:0:(service.c:2324:ptlrpc_server_handle_request()) Handling RPC req@00000000e31b009d pname:cluuid+ref:pid:xid:nid:opc:job ll_mgs_0002:5b5c66b1-1a2f-4651-8548-0c75b9a38a94+9:2271:x1862456979120832:12345-192.168.204.3@tcp:400:kworker/u8:3.0 00000100:00100000:2.0:1776178393.394213:0:20140:0:(service.c:2372:ptlrpc_server_handle_request()) Handled RPC req@00000000e31b009d pname:cluuid+ref:pid:xid:nid:opc:job ll_mgs_0002:5b5c66b1-1a2f-4651-8548-0c75b9a38a94+9:2271:x1862456979120832:12345-192.168.204.3@tcp:400:kworker/u8:3.0 Request processed in 98us (13635us total) trans 0 rc 0/0 00000100:00100000:2.0:1776178393.394221:0:20140:0:(nrs_fifo.c:241:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.204.3@tcp, seq: 229 00010000:00010000:3.0:1776178393.569868:0:6244:0:(ldlm_lockd.c:2792:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00010000:00010000:2.0:1776178394.592413:0:6245:0:(ldlm_lockd.c:2792:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00080000:1.0:1776178394.976684:0:37:0:(pinger.c:251:ptlrpc_pinger_process_import()) lustre-MDT0000-lwp-OST0000_UUID->lustre-MDT0000_UUID: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00080000:1.0:1776178394.976695:0:37:0:(recover.c:57:ptlrpc_initiate_recovery()) lustre-MDT0000_UUID: starting recovery 00000100:00080000:1.0:1776178394.976699:0:37:0:(import.c:86:import_set_state_nolock()) 0000000047601a98 lustre-MDT0000_UUID: changing import state from DISCONN to CONNECTING 00000100:00080000:1.0:1776178394.976710:0:37:0:(import.c:534:import_select_connection()) lustre-MDT0000-lwp-OST0000: connect to NID 0@lo last attempt 1097 00000100:00080000:1.0:1776178394.976715:0:37:0:(import.c:581:import_select_connection()) lustre-MDT0000-lwp-OST0000: tried all connections, increasing latency to 21s 00000100:00080000:1.0:1776178394.976723:0:37:0:(import.c:615:import_select_connection()) lustre-MDT0000-lwp-OST0000: import 0000000047601a98 using connection 192.168.204.103@tcp/0@lo 00000100:00100000:1.0:1776178394.976754:0:37:0:(import.c:807:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5) req@000000006a209b80 x1862456998075648/t0(0) o38->lustre-MDT0000-lwp-OST0000@0@lo:12/10 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' 00000100:00080000:1.0:1776178394.976804:0:37:0:(pinger.c:251:ptlrpc_pinger_process_import()) lustre-MDT0000-lwp-OST0001_UUID->lustre-MDT0000_UUID: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00080000:1.0:1776178394.976807:0:37:0:(recover.c:57:ptlrpc_initiate_recovery()) lustre-MDT0000_UUID: starting recovery 00000100:00080000:1.0:1776178394.976809:0:37:0:(import.c:86:import_set_state_nolock()) 00000000dfc8db13 lustre-MDT0000_UUID: changing import state from DISCONN to CONNECTING 00000100:00080000:1.0:1776178394.976812:0:37:0:(import.c:534:import_select_connection()) lustre-MDT0000-lwp-OST0001: connect to NID 0@lo last attempt 1097 00000100:00080000:1.0:1776178394.976814:0:37:0:(import.c:581:import_select_connection()) lustre-MDT0000-lwp-OST0001: tried all connections, increasing latency to 21s 00000100:00080000:1.0:1776178394.976818:0:37:0:(import.c:615:import_select_connection()) lustre-MDT0000-lwp-OST0001: import 00000000dfc8db13 using connection 192.168.204.103@tcp/0@lo 00000100:00100000:1.0:1776178394.976828:0:37:0:(import.c:807:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5) req@0000000017d54e92 x1862456998075776/t0(0) o38->lustre-MDT0000-lwp-OST0001@0@lo:12/10 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' 00000100:00080000:1.0:1776178394.976844:0:37:0:(pinger.c:251:ptlrpc_pinger_process_import()) lustre-MDT0001-mdtlov_UUID->lustre-MDT0000_UUID: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00080000:1.0:1776178394.976847:0:37:0:(recover.c:57:ptlrpc_initiate_recovery()) lustre-MDT0000_UUID: starting recovery 00000100:00080000:1.0:1776178394.976849:0:37:0:(import.c:86:import_set_state_nolock()) 00000000ba4f1821 lustre-MDT0000_UUID: changing import state from DISCONN to CONNECTING 00000100:00080000:1.0:1776178394.976852:0:37:0:(import.c:534:import_select_connection()) lustre-MDT0000-osp-MDT0001: connect to NID 0@lo last attempt 1097 00000100:00080000:1.0:1776178394.976854:0:37:0:(import.c:581:import_select_connection()) lustre-MDT0000-osp-MDT0001: tried all connections, increasing latency to 21s 00000100:00080000:1.0:1776178394.976858:0:37:0:(import.c:615:import_select_connection()) lustre-MDT0000-osp-MDT0001: import 00000000ba4f1821 using connection 192.168.204.103@tcp/0@lo 00000100:00100000:1.0:1776178394.976868:0:37:0:(import.c:807:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5) req@00000000ba273b2a x1862456998075904/t0(0) o38->lustre-MDT0000-osp-MDT0001@0@lo:24/4 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' 00000100:00080000:1.0:1776178394.976895:0:37:0:(pinger.c:251:ptlrpc_pinger_process_import()) lustre-MDT0000-lwp-MDT0001_UUID->lustre-MDT0000_UUID: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00080000:1.0:1776178394.976899:0:37:0:(recover.c:57:ptlrpc_initiate_recovery()) lustre-MDT0000_UUID: starting recovery 00000100:00080000:1.0:1776178394.976900:0:37:0:(import.c:86:import_set_state_nolock()) 00000000a7c55b1f lustre-MDT0000_UUID: changing import state from DISCONN to CONNECTING 00000100:00080000:1.0:1776178394.976903:0:37:0:(import.c:534:import_select_connection()) lustre-MDT0000-lwp-MDT0001: connect to NID 0@lo last attempt 1097 00000100:00080000:1.0:1776178394.976905:0:37:0:(import.c:581:import_select_connection()) lustre-MDT0000-lwp-MDT0001: tried all connections, increasing latency to 21s 00000100:00080000:1.0:1776178394.976908:0:37:0:(import.c:615:import_select_connection()) lustre-MDT0000-lwp-MDT0001: import 00000000a7c55b1f using connection 192.168.204.103@tcp/0@lo 00000100:00100000:1.0:1776178394.976918:0:37:0:(import.c:807:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5) req@000000006247494d x1862456998076096/t0(0) o38->lustre-MDT0000-lwp-MDT0001@0@lo:12/10 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' 00000100:00100000:1.0:1776178394.976960:0:3354:0:(client.c:1733:ptlrpc_send_new_req()) Sending RPC req@00000000d2194868 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_01:lustre-MDT0001-lwp-OST0001_UUID:3354:1862456998075840:0@lo:400:kworker/u8:2.0 00000100:00100000:1.0:1776178394.977092:0:3354:0:(events.c:361:request_in_callback()) peer: 12345-0@lo (source: 12345-0@lo) 00000100:00100000:1.0:1776178394.977140:0:3354:0:(ptlrpcd.c:415:ptlrpcd_check()) transfer 1 async RPCs [0->1] 00000100:00100000:1.0:1776178394.977147:0:3354:0:(client.c:1733:ptlrpc_send_new_req()) Sending RPC req@00000000cb1b73cb pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_01:lustre-MDT0001-lwp-OST0000_UUID:3354:1862456998075712:0@lo:400:kworker/u8:2.0 00000100:00100000:1.0:1776178394.977171:0:3354:0:(events.c:361:request_in_callback()) peer: 12345-0@lo (source: 12345-0@lo) 00000100:00100000:1.0:1776178394.977226:0:6255:0:(service.c:2152:ptlrpc_server_handle_req_in()) got req x1862456998075840 00000100:00100000:1.0:1776178394.977277:0:6255:0:(service.c:2152:ptlrpc_server_handle_req_in()) got req x1862456998075712 00000100:00100000:1.0:1776178394.977299:0:6255:0:(nrs_fifo.c:179:nrs_fifo_req_get()) NRS start fifo request from 12345-0@lo, seq: 1040 00000100:00100000:1.0:1776178394.977305:0:6255:0:(service.c:2324:ptlrpc_server_handle_request()) Handling RPC req@0000000015909b2d pname:cluuid+ref:pid:xid:nid:opc:job mdt00_000:lustre-MDT0001-lwp-OST0001_UUID+5:3354:x1862456998075840:12345-0@lo:400:kworker/u8:2.0 00000100:00100000:1.0:1776178394.977364:0:6255:0:(service.c:2372:ptlrpc_server_handle_request()) Handled RPC req@0000000015909b2d pname:cluuid+ref:pid:xid:nid:opc:job mdt00_000:lustre-MDT0001-lwp-OST0001_UUID+5:3354:x1862456998075840:12345-0@lo:400:kworker/u8:2.0 Request processed in 58us (307us total) trans 0 rc 0/0 00000100:00100000:1.0:1776178394.977372:0:6255:0:(nrs_fifo.c:241:nrs_fifo_req_stop()) NRS stop fifo request from 12345-0@lo, seq: 1040 00000100:00100000:1.0:1776178394.977389:0:6255:0:(nrs_fifo.c:179:nrs_fifo_req_get()) NRS start fifo request from 12345-0@lo, seq: 1041 00000100:00100000:1.0:1776178394.977392:0:6255:0:(service.c:2324:ptlrpc_server_handle_request()) Handling RPC req@00000000a2f460e7 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_000:lustre-MDT0001-lwp-OST0000_UUID+5:3354:x1862456998075712:12345-0@lo:400:kworker/u8:2.0 00000100:00100000:1.0:1776178394.977418:0:6255:0:(service.c:2372:ptlrpc_server_handle_request()) Handled RPC req@00000000a2f460e7 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_000:lustre-MDT0001-lwp-OST0000_UUID+5:3354:x1862456998075712:12345-0@lo:400:kworker/u8:2.0 Request processed in 24us (246us total) trans 0 rc 0/0 00000100:00100000:1.0:1776178394.977423:0:6255:0:(nrs_fifo.c:241:nrs_fifo_req_stop()) NRS stop fifo request from 12345-0@lo, seq: 1041 00000100:00100000:1.0:1776178394.977479:0:3354:0:(client.c:2215:ptlrpc_check_set()) Completed RPC req@00000000cb1b73cb pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_01:lustre-MDT0001-lwp-OST0000_UUID:3354:1862456998075712:0@lo:400:kworker/u8:2.0 00000100:00100000:1.0:1776178394.977490:0:3354:0:(client.c:2215:ptlrpc_check_set()) Completed RPC req@00000000d2194868 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_01:lustre-MDT0001-lwp-OST0001_UUID:3354:1862456998075840:0@lo:400:kworker/u8:2.0 00000100:00100000:3.0:1776178394.977588:0:3356:0:(client.c:1733:ptlrpc_send_new_req()) Sending RPC req@000000006466ee18 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_03:ac61f839-0e8e-4cac-9c59-08bcfbbad330:3356:1862456998075584:0@lo:400:kworker/u8:2.0 00000100:00100000:3.0:1776178394.977636:0:3356:0:(events.c:361:request_in_callback()) peer: 12345-0@lo (source: 12345-0@lo) 00000100:00100000:3.0:1776178394.977664:0:3356:0:(client.c:1733:ptlrpc_send_new_req()) Sending RPC req@00000000bfb13eb6 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_03:lustre-MDT0001-mdtlov_UUID:3356:1862456998076032:0@lo:400:kworker/u8:2.0 00000100:00100000:3.0:1776178394.977695:0:3356:0:(events.c:361:request_in_callback()) peer: 12345-0@lo (source: 12345-0@lo) 00000100:00100000:3.0:1776178394.977718:0:3356:0:(ptlrpcd.c:415:ptlrpcd_check()) transfer 1 async RPCs [2->3] 00000100:00100000:3.0:1776178394.977724:0:3356:0:(client.c:1733:ptlrpc_send_new_req()) Sending RPC req@000000001f0b1867 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_03:lustre-MDT0001-mdtlov_UUID:3356:1862456998075968:0@lo:400:kworker/u8:2.0 00000100:00100000:3.0:1776178394.977748:0:3356:0:(events.c:361:request_in_callback()) peer: 12345-0@lo (source: 12345-0@lo) 00000100:00100000:3.0:1776178394.977781:0:14440:0:(service.c:2152:ptlrpc_server_handle_req_in()) got req x1862456998076032 00000100:00100000:3.0:1776178394.977809:0:14440:0:(service.c:2152:ptlrpc_server_handle_req_in()) got req x1862456998075968 00000100:00100000:3.0:1776178394.977827:0:14440:0:(nrs_fifo.c:179:nrs_fifo_req_get()) NRS start fifo request from 12345-0@lo, seq: 661 00000100:00100000:3.0:1776178394.977831:0:14440:0:(service.c:2324:ptlrpc_server_handle_request()) Handling RPC req@0000000022d4dc29 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_004:lustre-MDT0001-mdtlov_UUID+5:3356:x1862456998076032:12345-0@lo:400:kworker/u8:2.0 00000100:00100000:3.0:1776178394.977867:0:14440:0:(service.c:2372:ptlrpc_server_handle_request()) Handled RPC req@0000000022d4dc29 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_004:lustre-MDT0001-mdtlov_UUID+5:3356:x1862456998076032:12345-0@lo:400:kworker/u8:2.0 Request processed in 35us (171us total) trans 0 rc 0/0 00000100:00100000:3.0:1776178394.977873:0:14440:0:(nrs_fifo.c:241:nrs_fifo_req_stop()) NRS stop fifo request from 12345-0@lo, seq: 661 00000100:00100000:3.0:1776178394.977884:0:14440:0:(nrs_fifo.c:179:nrs_fifo_req_get()) NRS start fifo request from 12345-0@lo, seq: 662 00000100:00100000:3.0:1776178394.977888:0:14440:0:(service.c:2324:ptlrpc_server_handle_request()) Handling RPC req@0000000039ff8335 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_004:lustre-MDT0001-mdtlov_UUID+5:3356:x1862456998075968:12345-0@lo:400:kworker/u8:2.0 00000100:00100000:3.0:1776178394.977912:0:14440:0:(service.c:2372:ptlrpc_server_handle_request()) Handled RPC req@0000000039ff8335 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_004:lustre-MDT0001-mdtlov_UUID+5:3356:x1862456998075968:12345-0@lo:400:kworker/u8:2.0 Request processed in 23us (164us total) trans 0 rc 0/0 00000100:00100000:3.0:1776178394.977918:0:14440:0:(nrs_fifo.c:241:nrs_fifo_req_stop()) NRS stop fifo request from 12345-0@lo, seq: 662 00000100:00100000:3.0:1776178394.977946:0:3356:0:(client.c:2215:ptlrpc_check_set()) Completed RPC req@000000001f0b1867 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_03:lustre-MDT0001-mdtlov_UUID:3356:1862456998075968:0@lo:400:kworker/u8:2.0 00000100:00100000:3.0:1776178394.977957:0:3356:0:(client.c:2215:ptlrpc_check_set()) Completed RPC req@00000000bfb13eb6 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_03:lustre-MDT0001-mdtlov_UUID:3356:1862456998076032:0@lo:400:kworker/u8:2.0 00000100:00100000:2.0:1776178394.977991:0:3352:0:(client.c:1733:ptlrpc_send_new_req()) Sending RPC req@000000006a209b80 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:lustre-MDT0000-lwp-OST0000_UUID:3352:1862456998075648:0@lo:38:kworker/u8:2.0 00000100:00100000:2.0:1776178394.978026:0:3352:0:(events.c:361:request_in_callback()) peer: 12345-0@lo (source: 12345-0@lo) 00000100:00100000:2.0:1776178394.978051:0:3352:0:(client.c:1733:ptlrpc_send_new_req()) Sending RPC req@0000000017d54e92 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:lustre-MDT0000-lwp-OST0001_UUID:3352:1862456998075776:0@lo:38:kworker/u8:2.0 00000100:00100000:2.0:1776178394.978111:0:3352:0:(events.c:361:request_in_callback()) peer: 12345-0@lo (source: 12345-0@lo) 00000100:00100000:2.0:1776178394.978131:0:3352:0:(client.c:1733:ptlrpc_send_new_req()) Sending RPC req@00000000ba273b2a pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:lustre-MDT0001-mdtlov_UUID:3352:1862456998075904:0@lo:38:kworker/u8:2.0 00000100:00100000:2.0:1776178394.978160:0:3352:0:(events.c:361:request_in_callback()) peer: 12345-0@lo (source: 12345-0@lo) 00000100:00100000:0.0:1776178394.978180:0:6256:0:(service.c:2152:ptlrpc_server_handle_req_in()) got req x1862456998075648 00000100:00100000:2.0:1776178394.978182:0:3352:0:(client.c:1733:ptlrpc_send_new_req()) Sending RPC req@000000006247494d pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:lustre-MDT0000-lwp-MDT0001_UUID:3352:1862456998076096:0@lo:38:kworker/u8:2.0 00000100:00100000:2.0:1776178394.978206:0:3352:0:(events.c:361:request_in_callback()) peer: 12345-0@lo (source: 12345-0@lo) 00000100:00100000:2.0:1776178394.978293:0:15213:0:(service.c:2152:ptlrpc_server_handle_req_in()) got req x1862456998075904 00000100:00100000:1.0:1776178394.978313:0:20140:0:(service.c:2152:ptlrpc_server_handle_req_in()) got req x1862456998075584 00000100:00100000:2.0:1776178394.978324:0:15213:0:(nrs_fifo.c:179:nrs_fifo_req_get()) NRS start fifo request from 12345-0@lo, seq: 745 00000100:00100000:2.0:1776178394.978330:0:15213:0:(service.c:2324:ptlrpc_server_handle_request()) Handling RPC req@000000005927b2c3 pname:cluuid+ref:pid:xid:nid:opc:job mdt_out00_003:0+-99:3352:x1862456998075904:12345-0@lo:38:kworker/u8:2.0 00010000:02000000:2.0:1776178394.978347:0:15213:0:(ldlm_lib.c:1139:target_handle_connect()) lustre-MDT0000: Not available for connect from 0@lo (stopping) 00000100:00100000:1.0:1776178394.978352:0:20140:0:(nrs_fifo.c:179:nrs_fifo_req_get()) NRS start fifo request from 12345-0@lo, seq: 230 00000100:00100000:1.0:1776178394.978359:0:20140:0:(service.c:2324:ptlrpc_server_handle_request()) Handling RPC req@0000000035060500 pname:cluuid+ref:pid:xid:nid:opc:job ll_mgs_0002:ac61f839-0e8e-4cac-9c59-08bcfbbad330+18:3356:x1862456998075584:12345-0@lo:400:kworker/u8:2.0 00000100:00100000:2.0:1776178394.978394:0:15213:0:(service.c:2372:ptlrpc_server_handle_request()) Handled RPC req@000000005927b2c3 pname:cluuid+ref:pid:xid:nid:opc:job mdt_out00_003:0+-99:3352:x1862456998075904:12345-0@lo:38:kworker/u8:2.0 Request processed in 63us (232us total) trans 0 rc -19/-19 00000100:00100000:2.0:1776178394.978401:0:15213:0:(nrs_fifo.c:241:nrs_fifo_req_stop()) NRS stop fifo request from 12345-0@lo, seq: 745 00000100:00100000:1.0:1776178394.978412:0:20140:0:(service.c:2372:ptlrpc_server_handle_request()) Handled RPC req@0000000035060500 pname:cluuid+ref:pid:xid:nid:opc:job ll_mgs_0002:ac61f839-0e8e-4cac-9c59-08bcfbbad330+18:3356:x1862456998075584:12345-0@lo:400:kworker/u8:2.0 Request processed in 51us (774us total) trans 0 rc 0/0 00000100:00100000:1.0:1776178394.978419:0:20140:0:(nrs_fifo.c:241:nrs_fifo_req_stop()) NRS stop fifo request from 12345-0@lo, seq: 230 00000100:00100000:1.0:1776178394.978450:0:3356:0:(client.c:2215:ptlrpc_check_set()) Completed RPC req@000000006466ee18 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_03:ac61f839-0e8e-4cac-9c59-08bcfbbad330:3356:1862456998075584:0@lo:400:kworker/u8:2.0 00000100:00100000:0.0:1776178394.987704:0:6256:0:(service.c:2152:ptlrpc_server_handle_req_in()) got req x1862456998075776 00000100:00100000:0.0:1776178394.987768:0:6256:0:(service.c:2152:ptlrpc_server_handle_req_in()) got req x1862456998076096 00000100:00100000:0.0:1776178394.987790:0:6256:0:(nrs_fifo.c:179:nrs_fifo_req_get()) NRS start fifo request from 12345-0@lo, seq: 6773 00000100:00100000:0.0:1776178394.987797:0:6256:0:(service.c:2324:ptlrpc_server_handle_request()) Handling RPC req@0000000023210d1c pname:cluuid+ref:pid:xid:nid:opc:job mdt00_001:0+-99:3352:x1862456998075648:12345-0@lo:38:kworker/u8:2.0 00010000:02000000:0.0:1776178394.987822:0:6256:0:(ldlm_lib.c:1139:target_handle_connect()) lustre-MDT0000: Not available for connect from 0@lo (stopping) 00000100:00100000:3.0:1776178394.988106:0:6257:0:(nrs_fifo.c:179:nrs_fifo_req_get()) NRS start fifo request from 12345-0@lo, seq: 6774 00000100:00100000:3.0:1776178394.988112:0:6257:0:(service.c:2324:ptlrpc_server_handle_request()) Handling RPC req@00000000df370bee pname:cluuid+ref:pid:xid:nid:opc:job mdt00_002:0+-99:3352:x1862456998075776:12345-0@lo:38:kworker/u8:2.0 00010000:02000000:3.0:1776178394.988121:0:6257:0:(ldlm_lib.c:1139:target_handle_connect()) lustre-MDT0000: Not available for connect from 0@lo (stopping) 00000100:00100000:3.0:1776178394.988217:0:6257:0:(service.c:2372:ptlrpc_server_handle_request()) Handled RPC req@00000000df370bee pname:cluuid+ref:pid:xid:nid:opc:job mdt00_002:0+-99:3352:x1862456998075776:12345-0@lo:38:kworker/u8:2.0 Request processed in 103us (10105us total) trans 0 rc -19/-19 00000100:00100000:3.0:1776178394.988225:0:6257:0:(nrs_fifo.c:241:nrs_fifo_req_stop()) NRS stop fifo request from 12345-0@lo, seq: 6774 00000100:00100000:3.0:1776178394.988252:0:6257:0:(nrs_fifo.c:179:nrs_fifo_req_get()) NRS start fifo request from 12345-0@lo, seq: 6775 00000100:00100000:3.0:1776178394.988255:0:6257:0:(service.c:2324:ptlrpc_server_handle_request()) Handling RPC req@000000005643ab1b pname:cluuid+ref:pid:xid:nid:opc:job mdt00_002:0+-99:3352:x1862456998076096:12345-0@lo:38:kworker/u8:2.0 00010000:02000000:3.0:1776178394.988262:0:6257:0:(ldlm_lib.c:1139:target_handle_connect()) lustre-MDT0000: Not available for connect from 0@lo (stopping) 00000100:00100000:3.0:1776178394.988286:0:6257:0:(service.c:2372:ptlrpc_server_handle_request()) Handled RPC req@000000005643ab1b pname:cluuid+ref:pid:xid:nid:opc:job mdt00_002:0+-99:3352:x1862456998076096:12345-0@lo:38:kworker/u8:2.0 Request processed in 30us (10079us total) trans 0 rc -19/-19 00000100:00100000:3.0:1776178394.988291:0:6257:0:(nrs_fifo.c:241:nrs_fifo_req_stop()) NRS stop fifo request from 12345-0@lo, seq: 6775 00000100:00080000:3.0:1776178394.988533:0:3352:0:(import.c:86:import_set_state_nolock()) 00000000dfc8db13 lustre-MDT0000_UUID: changing import state from CONNECTING to DISCONN 00000100:00080000:3.0:1776178394.988539:0:3352:0:(import.c:1429:ptlrpc_connect_interpret()) recovery of lustre-MDT0000_UUID on 192.168.204.103@tcp failed (-19) 00000100:00100000:3.0:1776178394.988544:0:3352:0:(client.c:2215:ptlrpc_check_set()) Completed RPC req@0000000017d54e92 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:lustre-MDT0000-lwp-OST0001_UUID:3352:1862456998075776:0@lo:38:kworker/u8:2.0 00000100:00080000:3.0:1776178394.988953:0:3352:0:(import.c:86:import_set_state_nolock()) 00000000ba4f1821 lustre-MDT0000_UUID: changing import state from CONNECTING to DISCONN 00000100:00080000:3.0:1776178394.988958:0:3352:0:(import.c:1429:ptlrpc_connect_interpret()) recovery of lustre-MDT0000_UUID on 192.168.204.103@tcp failed (-19) 00000100:00100000:3.0:1776178394.988962:0:3352:0:(client.c:2215:ptlrpc_check_set()) Completed RPC req@00000000ba273b2a pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:lustre-MDT0001-mdtlov_UUID:3352:1862456998075904:0@lo:38:kworker/u8:2.0 00000100:00080000:3.0:1776178394.988973:0:3352:0:(import.c:86:import_set_state_nolock()) 00000000a7c55b1f lustre-MDT0000_UUID: changing import state from CONNECTING to DISCONN 00000100:00080000:3.0:1776178394.988976:0:3352:0:(import.c:1429:ptlrpc_connect_interpret()) recovery of lustre-MDT0000_UUID on 192.168.204.103@tcp failed (-19) 00000100:00100000:3.0:1776178394.988978:0:3352:0:(client.c:2215:ptlrpc_check_set()) Completed RPC req@000000006247494d pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:lustre-MDT0000-lwp-MDT0001_UUID:3352:1862456998076096:0@lo:38:kworker/u8:2.0 00000100:00100000:0.0:1776178394.991622:0:6256:0:(service.c:2372:ptlrpc_server_handle_request()) Handled RPC req@0000000023210d1c pname:cluuid+ref:pid:xid:nid:opc:job mdt00_001:0+-99:3352:x1862456998075648:12345-0@lo:38:kworker/u8:2.0 Request processed in 3810us (13581us total) trans 0 rc -19/-19 00000100:00100000:0.0:1776178394.991643:0:6256:0:(nrs_fifo.c:241:nrs_fifo_req_stop()) NRS stop fifo request from 12345-0@lo, seq: 6773 00000100:00080000:3.0:1776178394.991820:0:3352:0:(import.c:86:import_set_state_nolock()) 0000000047601a98 lustre-MDT0000_UUID: changing import state from CONNECTING to DISCONN 00000100:00080000:3.0:1776178394.991826:0:3352:0:(import.c:1429:ptlrpc_connect_interpret()) recovery of lustre-MDT0000_UUID on 192.168.204.103@tcp failed (-19) 00000100:00100000:3.0:1776178394.991833:0:3352:0:(client.c:2215:ptlrpc_check_set()) Completed RPC req@000000006a209b80 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:lustre-MDT0000-lwp-OST0000_UUID:3352:1862456998075648:0@lo:38:kworker/u8:2.0 00010000:00010000:2.0:1776178395.619057:0:9224:0:(ldlm_lockd.c:2792:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:3.0:1776178396.514668:0:3353:0:(client.c:1733:ptlrpc_send_new_req()) Sending RPC req@00000000b5702a7a pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_00:lustre-MDT0001-mdtlov_UUID:3353:1862456998076160:0@lo:13:osp-pre-1-1.0 00000100:00100000:3.0:1776178396.514812:0:3353:0:(events.c:361:request_in_callback()) peer: 12345-0@lo (source: 12345-0@lo) 00000100:00100000:2.0:1776178396.515387:0:8231:0:(service.c:2152:ptlrpc_server_handle_req_in()) got req x1862456998076160 00000100:00100000:2.0:1776178396.515476:0:8231:0:(nrs_fifo.c:179:nrs_fifo_req_get()) NRS start fifo request from 12345-0@lo, seq: 563 00000100:00100000:2.0:1776178396.515485:0:8231:0:(service.c:2324:ptlrpc_server_handle_request()) Handling RPC req@0000000049b1b339 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_create00:lustre-MDT0001-mdtlov_UUID+5:3353:x1862456998076160:12345-0@lo:13:osp-pre-1-1.0 00002000:00000024:2.0:1776178396.515539:0:8231:0:(ofd_obd.c:740:ofd_statfs()) blocks cached 229376 granted 491746368 pending 0 free 3923689472 avail 3692167168 00000100:00100000:2.0:1776178396.515622:0:8231:0:(service.c:2372:ptlrpc_server_handle_request()) Handled RPC req@0000000049b1b339 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_create00:lustre-MDT0001-mdtlov_UUID+5:3353:x1862456998076160:12345-0@lo:13:osp-pre-1-1.0 Request processed in 136us (813us total) trans 0 rc 0/0 00000100:00100000:2.0:1776178396.515631:0:8231:0:(nrs_fifo.c:241:nrs_fifo_req_stop()) NRS stop fifo request from 12345-0@lo, seq: 563 00000100:00100000:3.0:1776178396.515818:0:3353:0:(client.c:2215:ptlrpc_check_set()) Completed RPC req@00000000b5702a7a pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_00:lustre-MDT0001-mdtlov_UUID:3353:1862456998076160:0@lo:13:osp-pre-1-1.0 00010000:00010000:2.0:1776178396.643009:0:9225:0:(ldlm_lockd.c:2792:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:3.0:1776178397.026701:0:3353:0:(ptlrpcd.c:415:ptlrpcd_check()) transfer 1 async RPCs [1->0] 00000100:00100000:3.0:1776178397.026736:0:3353:0:(client.c:1733:ptlrpc_send_new_req()) Sending RPC req@000000001f101832 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_00:lustre-MDT0001-mdtlov_UUID:3353:1862456998076224:0@lo:13:osp-pre-0-1.0 00000100:00100000:3.0:1776178397.026832:0:3353:0:(events.c:361:request_in_callback()) peer: 12345-0@lo (source: 12345-0@lo) 00000100:00100000:0.0:1776178397.026957:0:9554:0:(service.c:2152:ptlrpc_server_handle_req_in()) got req x1862456998076224 00000100:00100000:0.0:1776178397.027031:0:9554:0:(nrs_fifo.c:179:nrs_fifo_req_get()) NRS start fifo request from 12345-0@lo, seq: 564 00000100:00100000:0.0:1776178397.027036:0:9554:0:(service.c:2324:ptlrpc_server_handle_request()) Handling RPC req@00000000239d7b9a pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_create00:lustre-MDT0001-mdtlov_UUID+5:3353:x1862456998076224:12345-0@lo:13:osp-pre-0-1.0 00002000:00000024:0.0:1776178397.027099:0:9554:0:(ofd_obd.c:740:ofd_statfs()) blocks cached 315392 granted 484883776 pending 0 free 3923689472 avail 3692167168 00000100:00100000:0.0:1776178397.027150:0:9554:0:(service.c:2372:ptlrpc_server_handle_request()) Handled RPC req@00000000239d7b9a pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_create00:lustre-MDT0001-mdtlov_UUID+5:3353:x1862456998076224:12345-0@lo:13:osp-pre-0-1.0 Request processed in 112us (319us total) trans 0 rc 0/0 00000100:00100000:0.0:1776178397.027158:0:9554:0:(nrs_fifo.c:241:nrs_fifo_req_stop()) NRS stop fifo request from 12345-0@lo, seq: 564 00000100:00100000:3.0:1776178397.027474:0:3353:0:(client.c:2215:ptlrpc_check_set()) Completed RPC req@000000001f101832 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_00:lustre-MDT0001-mdtlov_UUID:3353:1862456998076224:0@lo:13:osp-pre-0-1.0 00010000:00010000:3.0:1776178397.667651:0:6244:0:(ldlm_lockd.c:2792:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:3.0:1776178398.499150:0:3347:0:(events.c:361:request_in_callback()) peer: 12345-192.168.204.3@tcp (source: 12345-192.168.204.3@tcp) 00000100:00100000:3.0:1776178398.499202:0:3347:0:(events.c:361:request_in_callback()) peer: 12345-192.168.204.3@tcp (source: 12345-192.168.204.3@tcp) 00000100:00100000:1.0:1776178398.499325:0:20137:0:(service.c:2152:ptlrpc_server_handle_req_in()) got req x1862456979121152 00000100:00100000:1.0:1776178398.499389:0:20137:0:(nrs_fifo.c:179:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.204.3@tcp, seq: 231 00000100:00100000:1.0:1776178398.499396:0:20137:0:(service.c:2324:ptlrpc_server_handle_request()) Handling RPC req@00000000f1e6b6f9 pname:cluuid+ref:pid:xid:nid:opc:job ll_mgs_0000:5b5c66b1-1a2f-4651-8548-0c75b9a38a94+9:2272:x1862456979121152:12345-192.168.204.3@tcp:400:kworker/u8:4.0 00000100:00100000:1.0:1776178398.499468:0:20137:0:(service.c:2372:ptlrpc_server_handle_request()) Handled RPC req@00000000f1e6b6f9 pname:cluuid+ref:pid:xid:nid:opc:job ll_mgs_0000:5b5c66b1-1a2f-4651-8548-0c75b9a38a94+9:2272:x1862456979121152:12345-192.168.204.3@tcp:400:kworker/u8:4.0 Request processed in 71us (328us total) trans 0 rc 0/0 00000100:00100000:1.0:1776178398.499476:0:20137:0:(nrs_fifo.c:241:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.204.3@tcp, seq: 231 00000100:00100000:3.0:1776178398.499672:0:3347:0:(events.c:361:request_in_callback()) peer: 12345-192.168.204.3@tcp (source: 12345-192.168.204.3@tcp) 00000100:00100000:2.0:1776178398.502151:0:8250:0:(service.c:2152:ptlrpc_server_handle_req_in()) got req x1862456979121216 00000100:00100000:2.0:1776178398.502218:0:8250:0:(service.c:2152:ptlrpc_server_handle_req_in()) got req x1862456979121280 00000100:00100000:2.0:1776178398.502283:0:8250:0:(nrs_fifo.c:179:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.204.3@tcp, seq: 1042 00000100:00100000:1.0:1776178398.502423:0:6255:0:(nrs_fifo.c:179:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.204.3@tcp, seq: 6776 00000100:00100000:1.0:1776178398.502429:0:6255:0:(service.c:2324:ptlrpc_server_handle_request()) Handling RPC req@00000000e4231891 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_000:0+-99:2269:x1862456979121216:12345-192.168.204.3@tcp:38:kworker/u8:4.0 00010000:02000000:1.0:1776178398.502452:0:6255:0:(ldlm_lib.c:1139:target_handle_connect()) lustre-MDT0000: Not available for connect from 192.168.204.3@tcp (stopping) 00000100:00100000:2.0:1776178398.502498:0:8250:0:(service.c:2324:ptlrpc_server_handle_request()) Handling RPC req@000000009bbbb40c pname:cluuid+ref:pid:xid:nid:opc:job mdt00_003:b50963bd-8eba-4432-9a54-8888d00e3fa1+6:2273:x1862456979121280:12345-192.168.204.3@tcp:400:kworker/u8:4.0 00000100:00100000:3.0:1776178398.508806:0:3347:0:(events.c:361:request_in_callback()) peer: 12345-192.168.204.3@tcp (source: 12345-192.168.204.3@tcp) 00000100:00100000:1.0:1776178398.509741:0:6255:0:(service.c:2372:ptlrpc_server_handle_request()) Handled RPC req@00000000e4231891 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_000:0+-99:2269:x1862456979121216:12345-192.168.204.3@tcp:38:kworker/u8:4.0 Request processed in 7301us (10529us total) trans 0 rc -19/-19 00000100:00100000:1.0:1776178398.509763:0:6255:0:(nrs_fifo.c:241:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.204.3@tcp, seq: 6776 00000100:00100000:1.0:1776178398.509870:0:20755:0:(service.c:2152:ptlrpc_server_handle_req_in()) got req x1862456979121344 00000100:00100000:1.0:1776178398.509924:0:20755:0:(nrs_fifo.c:179:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.204.3@tcp, seq: 663 00000100:00100000:1.0:1776178398.509932:0:20755:0:(service.c:2324:ptlrpc_server_handle_request()) Handling RPC req@00000000c38303b3 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_006:b50963bd-8eba-4432-9a54-8888d00e3fa1+8:2272:x1862456979121344:12345-192.168.204.3@tcp:400:kworker/u8:4.0 00000100:00100000:3.0:1776178398.511495:0:3347:0:(events.c:361:request_in_callback()) peer: 12345-192.168.204.3@tcp (source: 12345-192.168.204.3@tcp) 00000100:00100000:3.0:1776178398.511541:0:20762:0:(service.c:2152:ptlrpc_server_handle_req_in()) got req x1862456979121408 00000100:00100000:1.0:1776178398.511622:0:20755:0:(service.c:2372:ptlrpc_server_handle_request()) Handled RPC req@00000000c38303b3 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_006:b50963bd-8eba-4432-9a54-8888d00e3fa1+8:2272:x1862456979121344:12345-192.168.204.3@tcp:400:kworker/u8:4.0 Request processed in 1687us (2824us total) trans 0 rc 0/0 00000100:00100000:1.0:1776178398.511636:0:20755:0:(nrs_fifo.c:241:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.204.3@tcp, seq: 663 00000100:00100000:3.0:1776178398.514037:0:20762:0:(nrs_fifo.c:179:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.204.3@tcp, seq: 664 00000100:00100000:2.0:1776178398.515095:0:8250:0:(service.c:2372:ptlrpc_server_handle_request()) Handled RPC req@000000009bbbb40c pname:cluuid+ref:pid:xid:nid:opc:job mdt00_003:b50963bd-8eba-4432-9a54-8888d00e3fa1+6:2273:x1862456979121280:12345-192.168.204.3@tcp:400:kworker/u8:4.0 Request processed in 12590us (15415us total) trans 0 rc 0/0 00000100:00100000:2.0:1776178398.515114:0:8250:0:(nrs_fifo.c:241:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.204.3@tcp, seq: 1042 00000100:00100000:3.0:1776178398.515213:0:20762:0:(service.c:2324:ptlrpc_server_handle_request()) Handling RPC req@00000000dd52d0d0 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_013:b50963bd-8eba-4432-9a54-8888d00e3fa1+9:2273:x1862456979121408:12345-192.168.204.3@tcp:400:kworker/u8:4.0 00000100:00100000:3.0:1776178398.515285:0:20762:0:(service.c:2372:ptlrpc_server_handle_request()) Handled RPC req@00000000dd52d0d0 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_013:b50963bd-8eba-4432-9a54-8888d00e3fa1+9:2273:x1862456979121408:12345-192.168.204.3@tcp:400:kworker/u8:4.0 Request processed in 77us (3799us total) trans 0 rc 0/0 00000100:00100000:3.0:1776178398.515365:0:20762:0:(nrs_fifo.c:241:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.204.3@tcp, seq: 664 00010000:00010000:2.0:1776178398.688386:0:6245:0:(ldlm_lockd.c:2792:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00010000:00010000:2.0:1776178399.712331:0:9224:0:(ldlm_lockd.c:2792:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00080000:2.0:1776178400.099347:0:36:0:(pinger.c:251:ptlrpc_pinger_process_import()) lustre-MDT0000-lwp-OST0000_UUID->lustre-MDT0000_UUID: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00080000:2.0:1776178400.099356:0:36:0:(recover.c:57:ptlrpc_initiate_recovery()) lustre-MDT0000_UUID: starting recovery 00000100:00080000:2.0:1776178400.099359:0:36:0:(import.c:86:import_set_state_nolock()) 0000000047601a98 lustre-MDT0000_UUID: changing import state from DISCONN to CONNECTING 00000100:00080000:2.0:1776178400.099368:0:36:0:(import.c:534:import_select_connection()) lustre-MDT0000-lwp-OST0000: connect to NID 0@lo last attempt 1102 00000100:00080000:2.0:1776178400.099371:0:36:0:(import.c:581:import_select_connection()) lustre-MDT0000-lwp-OST0000: tried all connections, increasing latency to 21s 00000100:00080000:2.0:1776178400.099378:0:36:0:(import.c:615:import_select_connection()) lustre-MDT0000-lwp-OST0000: import 0000000047601a98 using connection 192.168.204.103@tcp/0@lo 00000100:00100000:2.0:1776178400.099403:0:36:0:(import.c:807:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5) req@0000000047458d9f x1862456998076352/t0(0) o38->lustre-MDT0000-lwp-OST0000@0@lo:12/10 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' 00000100:00080000:2.0:1776178400.099434:0:36:0:(pinger.c:251:ptlrpc_pinger_process_import()) lustre-MDT0000-lwp-OST0001_UUID->lustre-MDT0000_UUID: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00080000:2.0:1776178400.099437:0:36:0:(recover.c:57:ptlrpc_initiate_recovery()) lustre-MDT0000_UUID: starting recovery 00000100:00080000:2.0:1776178400.099438:0:36:0:(import.c:86:import_set_state_nolock()) 00000000dfc8db13 lustre-MDT0000_UUID: changing import state from DISCONN to CONNECTING 00000100:00080000:2.0:1776178400.099442:0:36:0:(import.c:534:import_select_connection()) lustre-MDT0000-lwp-OST0001: connect to NID 0@lo last attempt 1102 00000100:00080000:2.0:1776178400.099443:0:36:0:(import.c:581:import_select_connection()) lustre-MDT0000-lwp-OST0001: tried all connections, increasing latency to 21s 00000100:00080000:2.0:1776178400.099447:0:36:0:(import.c:615:import_select_connection()) lustre-MDT0000-lwp-OST0001: import 00000000dfc8db13 using connection 192.168.204.103@tcp/0@lo 00000100:00100000:2.0:1776178400.099456:0:36:0:(import.c:807:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5) req@0000000072865dff x1862456998076480/t0(0) o38->lustre-MDT0000-lwp-OST0001@0@lo:12/10 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' 00000100:00080000:2.0:1776178400.099480:0:36:0:(pinger.c:251:ptlrpc_pinger_process_import()) lustre-MDT0001-mdtlov_UUID->lustre-MDT0000_UUID: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00080000:2.0:1776178400.099483:0:36:0:(recover.c:57:ptlrpc_initiate_recovery()) lustre-MDT0000_UUID: starting recovery 00000100:00080000:2.0:1776178400.099484:0:36:0:(import.c:86:import_set_state_nolock()) 00000000ba4f1821 lustre-MDT0000_UUID: changing import state from DISCONN to CONNECTING 00000100:00080000:2.0:1776178400.099488:0:36:0:(import.c:534:import_select_connection()) lustre-MDT0000-osp-MDT0001: connect to NID 0@lo last attempt 1102 00000100:00080000:2.0:1776178400.099489:0:36:0:(import.c:581:import_select_connection()) lustre-MDT0000-osp-MDT0001: tried all connections, increasing latency to 21s 00000100:00080000:2.0:1776178400.099492:0:36:0:(import.c:615:import_select_connection()) lustre-MDT0000-osp-MDT0001: import 00000000ba4f1821 using connection 192.168.204.103@tcp/0@lo 00000100:00100000:2.0:1776178400.099501:0:36:0:(import.c:807:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5) req@00000000e64148ab x1862456998076608/t0(0) o38->lustre-MDT0000-osp-MDT0001@0@lo:24/4 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' 00000100:00080000:2.0:1776178400.099518:0:36:0:(pinger.c:251:ptlrpc_pinger_process_import()) lustre-MDT0000-lwp-MDT0001_UUID->lustre-MDT0000_UUID: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00080000:2.0:1776178400.099521:0:36:0:(recover.c:57:ptlrpc_initiate_recovery()) lustre-MDT0000_UUID: starting recovery 00000100:00080000:2.0:1776178400.099522:0:36:0:(import.c:86:import_set_state_nolock()) 00000000a7c55b1f lustre-MDT0000_UUID: changing import state from DISCONN to CONNECTING 00000100:00080000:2.0:1776178400.099524:0:36:0:(import.c:534:import_select_connection()) lustre-MDT0000-lwp-MDT0001: connect to NID 0@lo last attempt 1102 00000100:00080000:2.0:1776178400.099526:0:36:0:(import.c:581:import_select_connection()) lustre-MDT0000-lwp-MDT0001: tried all connections, increasing latency to 21s 00000100:00080000:2.0:1776178400.099529:0:36:0:(import.c:615:import_select_connection()) lustre-MDT0000-lwp-MDT0001: import 00000000a7c55b1f using connection 192.168.204.103@tcp/0@lo 00000100:00100000:2.0:1776178400.099536:0:36:0:(import.c:807:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5) req@00000000bd795500 x1862456998076800/t0(0) o38->lustre-MDT0000-lwp-MDT0001@0@lo:12/10 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' 00000100:00100000:1.0:1776178400.099714:0:3356:0:(client.c:1733:ptlrpc_send_new_req()) Sending RPC req@00000000f95e9da5 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_03:lustre-MDT0001-lwp-OST0000_UUID:3356:1862456998076416:0@lo:400:kworker/u8:1.0 00000100:00100000:1.0:1776178400.099778:0:3356:0:(events.c:361:request_in_callback()) peer: 12345-0@lo (source: 12345-0@lo) 00000100:00100000:1.0:1776178400.099821:0:3356:0:(ptlrpcd.c:415:ptlrpcd_check()) transfer 2 async RPCs [2->3] 00000100:00100000:1.0:1776178400.099827:0:3356:0:(client.c:1733:ptlrpc_send_new_req()) Sending RPC req@000000009e7a8bb5 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_03:ac61f839-0e8e-4cac-9c59-08bcfbbad330:3356:1862456998076288:0@lo:400:kworker/u8:1.0 00000100:00100000:1.0:1776178400.099856:0:3356:0:(events.c:361:request_in_callback()) peer: 12345-0@lo (source: 12345-0@lo) 00000100:00100000:1.0:1776178400.099869:0:3356:0:(client.c:1733:ptlrpc_send_new_req()) Sending RPC req@0000000013603266 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_03:lustre-MDT0001-mdtlov_UUID:3356:1862456998076736:0@lo:400:kworker/u8:1.0 00000100:00100000:1.0:1776178400.099891:0:3356:0:(events.c:361:request_in_callback()) peer: 12345-0@lo (source: 12345-0@lo) 00000100:00100000:1.0:1776178400.099912:0:3354:0:(client.c:1733:ptlrpc_send_new_req()) Sending RPC req@00000000dd1281fe pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_01:lustre-MDT0001-mdtlov_UUID:3354:1862456998076672:0@lo:400:kworker/u8:1.0 00000100:00100000:1.0:1776178400.099933:0:3354:0:(events.c:361:request_in_callback()) peer: 12345-0@lo (source: 12345-0@lo) 00000100:00100000:1.0:1776178400.099948:0:3354:0:(ptlrpcd.c:415:ptlrpcd_check()) transfer 1 async RPCs [0->1] 00000100:00100000:1.0:1776178400.099951:0:3354:0:(client.c:1733:ptlrpc_send_new_req()) Sending RPC req@00000000b5702a7a pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_01:lustre-MDT0001-lwp-OST0001_UUID:3354:1862456998076544:0@lo:400:kworker/u8:1.0 00000100:00100000:1.0:1776178400.099983:0:3354:0:(events.c:361:request_in_callback()) peer: 12345-0@lo (source: 12345-0@lo) 00000100:00100000:1.0:1776178400.100015:0:20755:0:(service.c:2152:ptlrpc_server_handle_req_in()) got req x1862456998076736 00000100:00100000:1.0:1776178400.100053:0:20755:0:(service.c:2152:ptlrpc_server_handle_req_in()) got req x1862456998076672 00000100:00100000:1.0:1776178400.100094:0:20755:0:(nrs_fifo.c:179:nrs_fifo_req_get()) NRS start fifo request from 12345-0@lo, seq: 665 00000100:00100000:1.0:1776178400.100099:0:20755:0:(service.c:2324:ptlrpc_server_handle_request()) Handling RPC req@000000009126d8c6 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_006:lustre-MDT0001-mdtlov_UUID+5:3356:x1862456998076736:12345-0@lo:400:kworker/u8:1.0 00000100:00100000:1.0:1776178400.100143:0:20755:0:(service.c:2372:ptlrpc_server_handle_request()) Handled RPC req@000000009126d8c6 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_006:lustre-MDT0001-mdtlov_UUID+5:3356:x1862456998076736:12345-0@lo:400:kworker/u8:1.0 Request processed in 43us (251us total) trans 0 rc 0/0 00000100:00100000:1.0:1776178400.100150:0:20755:0:(nrs_fifo.c:241:nrs_fifo_req_stop()) NRS stop fifo request from 12345-0@lo, seq: 665 00000100:00100000:1.0:1776178400.100166:0:20755:0:(nrs_fifo.c:179:nrs_fifo_req_get()) NRS start fifo request from 12345-0@lo, seq: 666 00000100:00100000:1.0:1776178400.100170:0:20755:0:(service.c:2324:ptlrpc_server_handle_request()) Handling RPC req@000000001554c6b4 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_006:lustre-MDT0001-mdtlov_UUID+5:3354:x1862456998076672:12345-0@lo:400:kworker/u8:1.0 00000100:00100000:1.0:1776178400.100201:0:20755:0:(service.c:2372:ptlrpc_server_handle_request()) Handled RPC req@000000001554c6b4 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_006:lustre-MDT0001-mdtlov_UUID+5:3354:x1862456998076672:12345-0@lo:400:kworker/u8:1.0 Request processed in 30us (268us total) trans 0 rc 0/0 00000100:00100000:1.0:1776178400.100206:0:20755:0:(nrs_fifo.c:241:nrs_fifo_req_stop()) NRS stop fifo request from 12345-0@lo, seq: 666 00000100:00100000:1.0:1776178400.100226:0:6255:0:(service.c:2152:ptlrpc_server_handle_req_in()) got req x1862456998076416 00000100:00100000:1.0:1776178400.100248:0:6255:0:(service.c:2152:ptlrpc_server_handle_req_in()) got req x1862456998076544 00000100:00100000:1.0:1776178400.100262:0:6255:0:(nrs_fifo.c:179:nrs_fifo_req_get()) NRS start fifo request from 12345-0@lo, seq: 1043 00000100:00100000:1.0:1776178400.100266:0:6255:0:(service.c:2324:ptlrpc_server_handle_request()) Handling RPC req@00000000f6af7e5b pname:cluuid+ref:pid:xid:nid:opc:job mdt00_000:lustre-MDT0001-lwp-OST0000_UUID+5:3356:x1862456998076416:12345-0@lo:400:kworker/u8:1.0 00000100:00100000:1.0:1776178400.100294:0:6255:0:(service.c:2372:ptlrpc_server_handle_request()) Handled RPC req@00000000f6af7e5b pname:cluuid+ref:pid:xid:nid:opc:job mdt00_000:lustre-MDT0001-lwp-OST0000_UUID+5:3356:x1862456998076416:12345-0@lo:400:kworker/u8:1.0 Request processed in 28us (518us total) trans 0 rc 0/0 00000100:00100000:1.0:1776178400.100300:0:6255:0:(nrs_fifo.c:241:nrs_fifo_req_stop()) NRS stop fifo request from 12345-0@lo, seq: 1043 00000100:00100000:1.0:1776178400.100311:0:6255:0:(nrs_fifo.c:179:nrs_fifo_req_get()) NRS start fifo request from 12345-0@lo, seq: 1044 00000100:00100000:1.0:1776178400.100314:0:6255:0:(service.c:2324:ptlrpc_server_handle_request()) Handling RPC req@00000000124cea36 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_000:lustre-MDT0001-lwp-OST0001_UUID+5:3354:x1862456998076544:12345-0@lo:400:kworker/u8:1.0 00000100:00100000:1.0:1776178400.100336:0:6255:0:(service.c:2372:ptlrpc_server_handle_request()) Handled RPC req@00000000124cea36 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_000:lustre-MDT0001-lwp-OST0001_UUID+5:3354:x1862456998076544:12345-0@lo:400:kworker/u8:1.0 Request processed in 21us (354us total) trans 0 rc 0/0 00000100:00100000:1.0:1776178400.100341:0:6255:0:(nrs_fifo.c:241:nrs_fifo_req_stop()) NRS stop fifo request from 12345-0@lo, seq: 1044 00000100:00100000:1.0:1776178400.100375:0:3354:0:(client.c:2215:ptlrpc_check_set()) Completed RPC req@00000000b5702a7a pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_01:lustre-MDT0001-lwp-OST0001_UUID:3354:1862456998076544:0@lo:400:kworker/u8:1.0 00000100:00100000:1.0:1776178400.100385:0:3354:0:(client.c:2215:ptlrpc_check_set()) Completed RPC req@00000000dd1281fe pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_01:lustre-MDT0001-mdtlov_UUID:3354:1862456998076672:0@lo:400:kworker/u8:1.0 00000100:00100000:1.0:1776178400.100412:0:3356:0:(client.c:2215:ptlrpc_check_set()) Completed RPC req@0000000013603266 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_03:lustre-MDT0001-mdtlov_UUID:3356:1862456998076736:0@lo:400:kworker/u8:1.0 00000100:00100000:1.0:1776178400.100420:0:3356:0:(client.c:2215:ptlrpc_check_set()) Completed RPC req@00000000f95e9da5 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_03:lustre-MDT0001-lwp-OST0000_UUID:3356:1862456998076416:0@lo:400:kworker/u8:1.0 00000100:00100000:3.0:1776178400.100515:0:3352:0:(client.c:1733:ptlrpc_send_new_req()) Sending RPC req@0000000047458d9f pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:lustre-MDT0000-lwp-OST0000_UUID:3352:1862456998076352:0@lo:38:kworker/u8:1.0 00000100:00100000:3.0:1776178400.100542:0:3352:0:(events.c:361:request_in_callback()) peer: 12345-0@lo (source: 12345-0@lo) 00000100:00100000:3.0:1776178400.100560:0:3352:0:(client.c:1733:ptlrpc_send_new_req()) Sending RPC req@0000000072865dff pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:lustre-MDT0000-lwp-OST0001_UUID:3352:1862456998076480:0@lo:38:kworker/u8:1.0 00000100:00100000:3.0:1776178400.100579:0:3352:0:(events.c:361:request_in_callback()) peer: 12345-0@lo (source: 12345-0@lo) 00000100:00100000:3.0:1776178400.100592:0:3352:0:(client.c:1733:ptlrpc_send_new_req()) Sending RPC req@00000000e64148ab pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:lustre-MDT0001-mdtlov_UUID:3352:1862456998076608:0@lo:38:kworker/u8:1.0 00000100:00100000:3.0:1776178400.100613:0:3352:0:(events.c:361:request_in_callback()) peer: 12345-0@lo (source: 12345-0@lo) 00000100:00100000:3.0:1776178400.100630:0:3352:0:(client.c:1733:ptlrpc_send_new_req()) Sending RPC req@00000000bd795500 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:lustre-MDT0000-lwp-MDT0001_UUID:3352:1862456998076800:0@lo:38:kworker/u8:1.0 00000100:00100000:3.0:1776178400.100651:0:3352:0:(events.c:361:request_in_callback()) peer: 12345-0@lo (source: 12345-0@lo) 00000100:00100000:3.0:1776178400.100685:0:6257:0:(service.c:2152:ptlrpc_server_handle_req_in()) got req x1862456998076352 00000100:00100000:3.0:1776178400.100699:0:6257:0:(service.c:2152:ptlrpc_server_handle_req_in()) got req x1862456998076480 00000100:00100000:3.0:1776178400.100706:0:6257:0:(service.c:2152:ptlrpc_server_handle_req_in()) got req x1862456998076800 00000100:00100000:3.0:1776178400.100714:0:6257:0:(nrs_fifo.c:179:nrs_fifo_req_get()) NRS start fifo request from 12345-0@lo, seq: 6777 00000100:00100000:3.0:1776178400.100717:0:6257:0:(service.c:2324:ptlrpc_server_handle_request()) Handling RPC req@00000000a562d27c pname:cluuid+ref:pid:xid:nid:opc:job mdt00_002:0+-99:3352:x1862456998076352:12345-0@lo:38:kworker/u8:1.0 00010000:02000000:3.0:1776178400.100730:0:6257:0:(ldlm_lib.c:1139:target_handle_connect()) lustre-MDT0000: Not available for connect from 0@lo (stopping) 00000100:00100000:3.0:1776178400.100766:0:6257:0:(service.c:2372:ptlrpc_server_handle_request()) Handled RPC req@00000000a562d27c pname:cluuid+ref:pid:xid:nid:opc:job mdt00_002:0+-99:3352:x1862456998076352:12345-0@lo:38:kworker/u8:1.0 Request processed in 47us (224us total) trans 0 rc -19/-19 00000100:00100000:3.0:1776178400.100771:0:6257:0:(nrs_fifo.c:241:nrs_fifo_req_stop()) NRS stop fifo request from 12345-0@lo, seq: 6777 00000100:00100000:3.0:1776178400.100781:0:6257:0:(nrs_fifo.c:179:nrs_fifo_req_get()) NRS start fifo request from 12345-0@lo, seq: 6778 00000100:00100000:3.0:1776178400.100785:0:6257:0:(service.c:2324:ptlrpc_server_handle_request()) Handling RPC req@0000000008605f3d pname:cluuid+ref:pid:xid:nid:opc:job mdt00_002:0+-99:3352:x1862456998076480:12345-0@lo:38:kworker/u8:1.0 00010000:02000000:3.0:1776178400.100791:0:6257:0:(ldlm_lib.c:1139:target_handle_connect()) lustre-MDT0000: Not available for connect from 0@lo (stopping) 00000100:00100000:3.0:1776178400.100811:0:6257:0:(service.c:2372:ptlrpc_server_handle_request()) Handled RPC req@0000000008605f3d pname:cluuid+ref:pid:xid:nid:opc:job mdt00_002:0+-99:3352:x1862456998076480:12345-0@lo:38:kworker/u8:1.0 Request processed in 25us (232us total) trans 0 rc -19/-19 00000100:00100000:3.0:1776178400.100816:0:6257:0:(nrs_fifo.c:241:nrs_fifo_req_stop()) NRS stop fifo request from 12345-0@lo, seq: 6778 00000100:00100000:3.0:1776178400.100825:0:6257:0:(nrs_fifo.c:179:nrs_fifo_req_get()) NRS start fifo request from 12345-0@lo, seq: 6779 00000100:00100000:3.0:1776178400.100828:0:6257:0:(service.c:2324:ptlrpc_server_handle_request()) Handling RPC req@0000000042ac1e35 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_002:0+-99:3352:x1862456998076800:12345-0@lo:38:kworker/u8:1.0 00010000:02000000:3.0:1776178400.100833:0:6257:0:(ldlm_lib.c:1139:target_handle_connect()) lustre-MDT0000: Not available for connect from 0@lo (stopping) 00000100:00100000:3.0:1776178400.100852:0:6257:0:(service.c:2372:ptlrpc_server_handle_request()) Handled RPC req@0000000042ac1e35 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_002:0+-99:3352:x1862456998076800:12345-0@lo:38:kworker/u8:1.0 Request processed in 23us (202us total) trans 0 rc -19/-19 00000100:00100000:3.0:1776178400.100858:0:6257:0:(nrs_fifo.c:241:nrs_fifo_req_stop()) NRS stop fifo request from 12345-0@lo, seq: 6779 00000100:00100000:3.0:1776178400.100877:0:14128:0:(service.c:2152:ptlrpc_server_handle_req_in()) got req x1862456998076608 00000100:00100000:3.0:1776178400.100893:0:14128:0:(nrs_fifo.c:179:nrs_fifo_req_get()) NRS start fifo request from 12345-0@lo, seq: 746 00000100:00100000:3.0:1776178400.100897:0:14128:0:(service.c:2324:ptlrpc_server_handle_request()) Handling RPC req@000000008a6fdb8d pname:cluuid+ref:pid:xid:nid:opc:job mdt_out00_002:0+-99:3352:x1862456998076608:12345-0@lo:38:kworker/u8:1.0 00010000:02000000:3.0:1776178400.100902:0:14128:0:(ldlm_lib.c:1139:target_handle_connect()) lustre-MDT0000: Not available for connect from 0@lo (stopping) 00000100:00100000:3.0:1776178400.100924:0:14128:0:(service.c:2372:ptlrpc_server_handle_request()) Handled RPC req@000000008a6fdb8d pname:cluuid+ref:pid:xid:nid:opc:job mdt_out00_002:0+-99:3352:x1862456998076608:12345-0@lo:38:kworker/u8:1.0 Request processed in 26us (311us total) trans 0 rc -19/-19 00000100:00100000:3.0:1776178400.100929:0:14128:0:(nrs_fifo.c:241:nrs_fifo_req_stop()) NRS stop fifo request from 12345-0@lo, seq: 746 00000100:00080000:3.0:1776178400.100952:0:3352:0:(import.c:86:import_set_state_nolock()) 0000000047601a98 lustre-MDT0000_UUID: changing import state from CONNECTING to DISCONN 00000100:00080000:3.0:1776178400.100956:0:3352:0:(import.c:1429:ptlrpc_connect_interpret()) recovery of lustre-MDT0000_UUID on 192.168.204.103@tcp failed (-19) 00000100:00100000:3.0:1776178400.100959:0:3352:0:(client.c:2215:ptlrpc_check_set()) Completed RPC req@0000000047458d9f pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:lustre-MDT0000-lwp-OST0000_UUID:3352:1862456998076352:0@lo:38:kworker/u8:1.0 00000100:00080000:3.0:1776178400.100977:0:3352:0:(import.c:86:import_set_state_nolock()) 00000000dfc8db13 lustre-MDT0000_UUID: changing import state from CONNECTING to DISCONN 00000100:00080000:3.0:1776178400.100979:0:3352:0:(import.c:1429:ptlrpc_connect_interpret()) recovery of lustre-MDT0000_UUID on 192.168.204.103@tcp failed (-19) 00000100:00100000:3.0:1776178400.100982:0:3352:0:(client.c:2215:ptlrpc_check_set()) Completed RPC req@0000000072865dff pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:lustre-MDT0000-lwp-OST0001_UUID:3352:1862456998076480:0@lo:38:kworker/u8:1.0 00000100:00080000:3.0:1776178400.100988:0:3352:0:(import.c:86:import_set_state_nolock()) 00000000ba4f1821 lustre-MDT0000_UUID: changing import state from CONNECTING to DISCONN 00000100:00080000:3.0:1776178400.100990:0:3352:0:(import.c:1429:ptlrpc_connect_interpret()) recovery of lustre-MDT0000_UUID on 192.168.204.103@tcp failed (-19) 00000100:00100000:3.0:1776178400.100992:0:3352:0:(client.c:2215:ptlrpc_check_set()) Completed RPC req@00000000e64148ab pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:lustre-MDT0001-mdtlov_UUID:3352:1862456998076608:0@lo:38:kworker/u8:1.0 00000100:00080000:3.0:1776178400.100999:0:3352:0:(import.c:86:import_set_state_nolock()) 00000000a7c55b1f lustre-MDT0000_UUID: changing import state from CONNECTING to DISCONN 00000100:00080000:3.0:1776178400.101002:0:3352:0:(import.c:1429:ptlrpc_connect_interpret()) recovery of lustre-MDT0000_UUID on 192.168.204.103@tcp failed (-19) 00000100:00100000:3.0:1776178400.101004:0:3352:0:(client.c:2215:ptlrpc_check_set()) Completed RPC req@00000000bd795500 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:lustre-MDT0000-lwp-MDT0001_UUID:3352:1862456998076800:0@lo:38:kworker/u8:1.0 00000100:00100000:0.0:1776178400.101470:0:20140:0:(service.c:2152:ptlrpc_server_handle_req_in()) got req x1862456998076288 00000100:00100000:0.0:1776178400.101519:0:20140:0:(nrs_fifo.c:179:nrs_fifo_req_get()) NRS start fifo request from 12345-0@lo, seq: 232 00000100:00100000:0.0:1776178400.101523:0:20140:0:(service.c:2324:ptlrpc_server_handle_request()) Handling RPC req@000000001091921c pname:cluuid+ref:pid:xid:nid:opc:job ll_mgs_0002:ac61f839-0e8e-4cac-9c59-08bcfbbad330+18:3356:x1862456998076288:12345-0@lo:400:kworker/u8:1.0 00000100:00100000:0.0:1776178400.101579:0:20140:0:(service.c:2372:ptlrpc_server_handle_request()) Handled RPC req@000000001091921c pname:cluuid+ref:pid:xid:nid:opc:job ll_mgs_0002:ac61f839-0e8e-4cac-9c59-08bcfbbad330+18:3356:x1862456998076288:12345-0@lo:400:kworker/u8:1.0 Request processed in 55us (1722us total) trans 0 rc 0/0 00000100:00100000:0.0:1776178400.101585:0:20140:0:(nrs_fifo.c:241:nrs_fifo_req_stop()) NRS stop fifo request from 12345-0@lo, seq: 232 00000100:00100000:3.0:1776178400.102097:0:3356:0:(client.c:2215:ptlrpc_check_set()) Completed RPC req@000000009e7a8bb5 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_03:ac61f839-0e8e-4cac-9c59-08bcfbbad330:3356:1862456998076288:0@lo:400:kworker/u8:1.0 00010000:00010000:2.0:1776178400.736411:0:9225:0:(ldlm_lockd.c:2792:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:0.0:1776178401.633148:0:3355:0:(ptlrpcd.c:415:ptlrpcd_check()) transfer 1 async RPCs [3->2] 00000100:00100000:0.0:1776178401.638392:0:3355:0:(client.c:1733:ptlrpc_send_new_req()) Sending RPC req@00000000bd795500 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_02:lustre-MDT0001-mdtlov_UUID:3355:1862456998076864:0@lo:13:osp-pre-1-1.0 00000100:00100000:0.0:1776178401.638488:0:3355:0:(events.c:361:request_in_callback()) peer: 12345-0@lo (source: 12345-0@lo) 00000100:00100000:2.0:1776178401.638894:0:8231:0:(service.c:2152:ptlrpc_server_handle_req_in()) got req x1862456998076864 00000100:00100000:2.0:1776178401.638970:0:8231:0:(nrs_fifo.c:179:nrs_fifo_req_get()) NRS start fifo request from 12345-0@lo, seq: 565 00000100:00100000:2.0:1776178401.638977:0:8231:0:(service.c:2324:ptlrpc_server_handle_request()) Handling RPC req@000000000e0e1967 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_create00:lustre-MDT0001-mdtlov_UUID+5:3355:x1862456998076864:12345-0@lo:13:osp-pre-1-1.0 00002000:00000024:2.0:1776178401.639019:0:8231:0:(ofd_obd.c:740:ofd_statfs()) blocks cached 229376 granted 491746368 pending 0 free 3923689472 avail 3692167168 00000100:00100000:2.0:1776178401.639121:0:8231:0:(service.c:2372:ptlrpc_server_handle_request()) Handled RPC req@000000000e0e1967 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_create00:lustre-MDT0001-mdtlov_UUID+5:3355:x1862456998076864:12345-0@lo:13:osp-pre-1-1.0 Request processed in 144us (634us total) trans 0 rc 0/0 00000100:00100000:2.0:1776178401.639129:0:8231:0:(nrs_fifo.c:241:nrs_fifo_req_stop()) NRS stop fifo request from 12345-0@lo, seq: 565 00000100:00100000:3.0:1776178401.639280:0:3355:0:(client.c:2215:ptlrpc_check_set()) Completed RPC req@00000000bd795500 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_02:lustre-MDT0001-mdtlov_UUID:3355:1862456998076864:0@lo:13:osp-pre-1-1.0 00010000:00010000:3.0:1776178401.765569:0:6244:0:(ldlm_lockd.c:2792:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:3.0:1776178402.144414:0:3353:0:(client.c:1733:ptlrpc_send_new_req()) Sending RPC req@0000000090a7aa50 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_00:lustre-MDT0001-mdtlov_UUID:3353:1862456998076928:0@lo:13:osp-pre-0-1.0 00000100:00100000:3.0:1776178402.144531:0:3353:0:(events.c:361:request_in_callback()) peer: 12345-0@lo (source: 12345-0@lo) 00000100:00100000:0.0:1776178402.144748:0:9554:0:(service.c:2152:ptlrpc_server_handle_req_in()) got req x1862456998076928 00000100:00100000:0.0:1776178402.144815:0:9554:0:(nrs_fifo.c:179:nrs_fifo_req_get()) NRS start fifo request from 12345-0@lo, seq: 566 00000100:00100000:0.0:1776178402.144821:0:9554:0:(service.c:2324:ptlrpc_server_handle_request()) Handling RPC req@00000000f62fe06f pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_create00:lustre-MDT0001-mdtlov_UUID+5:3353:x1862456998076928:12345-0@lo:13:osp-pre-0-1.0 00002000:00000024:0.0:1776178402.144856:0:9554:0:(ofd_obd.c:740:ofd_statfs()) blocks cached 315392 granted 484883776 pending 0 free 3923689472 avail 3692167168 00000100:00100000:0.0:1776178402.144909:0:9554:0:(service.c:2372:ptlrpc_server_handle_request()) Handled RPC req@00000000f62fe06f pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_create00:lustre-MDT0001-mdtlov_UUID+5:3353:x1862456998076928:12345-0@lo:13:osp-pre-0-1.0 Request processed in 88us (380us total) trans 0 rc 0/0 00000100:00100000:0.0:1776178402.144917:0:9554:0:(nrs_fifo.c:241:nrs_fifo_req_stop()) NRS stop fifo request from 12345-0@lo, seq: 566 00000100:00100000:3.0:1776178402.145204:0:3353:0:(client.c:2215:ptlrpc_check_set()) Completed RPC req@0000000090a7aa50 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_00:lustre-MDT0001-mdtlov_UUID:3353:1862456998076928:0@lo:13:osp-pre-0-1.0 00010000:00010000:2.0:1776178402.784460:0:6245:0:(ldlm_lockd.c:2792:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:0.0:1776178403.618229:0:3348:0:(events.c:361:request_in_callback()) peer: 12345-192.168.204.3@tcp (source: 12345-192.168.204.3@tcp) 00000100:00100000:0.0:1776178403.618278:0:3348:0:(events.c:361:request_in_callback()) peer: 12345-192.168.204.3@tcp (source: 12345-192.168.204.3@tcp) 00000100:00100000:0.0:1776178403.618332:0:3348:0:(events.c:361:request_in_callback()) peer: 12345-192.168.204.3@tcp (source: 12345-192.168.204.3@tcp) 00000100:00100000:0.0:1776178403.618365:0:3348:0:(events.c:361:request_in_callback()) peer: 12345-192.168.204.3@tcp (source: 12345-192.168.204.3@tcp) 00000100:00100000:0.0:1776178403.618463:0:3348:0:(events.c:361:request_in_callback()) peer: 12345-192.168.204.3@tcp (source: 12345-192.168.204.3@tcp) 00000100:00100000:0.0:1776178403.618521:0:6256:0:(service.c:2152:ptlrpc_server_handle_req_in()) got req x1862456979121600 00000100:00100000:0.0:1776178403.618600:0:6256:0:(service.c:2152:ptlrpc_server_handle_req_in()) got req x1862456979121536 00000100:00100000:0.0:1776178403.618620:0:6256:0:(nrs_fifo.c:179:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.204.3@tcp, seq: 1045 00000100:00100000:0.0:1776178403.618630:0:6256:0:(service.c:2324:ptlrpc_server_handle_request()) Handling RPC req@0000000098e424d0 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_001:b50963bd-8eba-4432-9a54-8888d00e3fa1+6:2270:x1862456979121600:12345-192.168.204.3@tcp:400:kworker/u8:3.0 00000100:00100000:0.0:1776178403.618731:0:6256:0:(service.c:2372:ptlrpc_server_handle_request()) Handled RPC req@0000000098e424d0 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_001:b50963bd-8eba-4432-9a54-8888d00e3fa1+6:2270:x1862456979121600:12345-192.168.204.3@tcp:400:kworker/u8:3.0 Request processed in 103us (511us total) trans 0 rc 0/0 00000100:00100000:0.0:1776178403.618739:0:6256:0:(nrs_fifo.c:241:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.204.3@tcp, seq: 1045 00000100:00100000:0.0:1776178403.618762:0:6256:0:(nrs_fifo.c:179:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.204.3@tcp, seq: 6780 00000100:00100000:0.0:1776178403.618766:0:6256:0:(service.c:2324:ptlrpc_server_handle_request()) Handling RPC req@0000000090a7aa50 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_001:0+-99:2269:x1862456979121536:12345-192.168.204.3@tcp:38:kworker/u8:3.0 00000100:00100000:3.0:1776178403.618776:0:20137:0:(service.c:2152:ptlrpc_server_handle_req_in()) got req x1862456979121472 00010000:02000000:0.0:1776178403.618785:0:6256:0:(ldlm_lib.c:1139:target_handle_connect()) lustre-MDT0000: Not available for connect from 192.168.204.3@tcp (stopping) 00000100:00100000:3.0:1776178403.618817:0:20137:0:(nrs_fifo.c:179:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.204.3@tcp, seq: 233 00000100:00100000:3.0:1776178403.618824:0:20137:0:(service.c:2324:ptlrpc_server_handle_request()) Handling RPC req@000000005b4b43d0 pname:cluuid+ref:pid:xid:nid:opc:job ll_mgs_0000:5b5c66b1-1a2f-4651-8548-0c75b9a38a94+9:2270:x1862456979121472:12345-192.168.204.3@tcp:400:kworker/u8:3.0 00000100:00100000:0.0:1776178403.618830:0:6256:0:(service.c:2372:ptlrpc_server_handle_request()) Handled RPC req@0000000090a7aa50 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_001:0+-99:2269:x1862456979121536:12345-192.168.204.3@tcp:38:kworker/u8:3.0 Request processed in 49us (353us total) trans 0 rc -19/-19 00000100:00100000:0.0:1776178403.618838:0:6256:0:(nrs_fifo.c:241:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.204.3@tcp, seq: 6780 00000100:00100000:3.0:1776178403.618861:0:20137:0:(service.c:2372:ptlrpc_server_handle_request()) Handled RPC req@000000005b4b43d0 pname:cluuid+ref:pid:xid:nid:opc:job ll_mgs_0000:5b5c66b1-1a2f-4651-8548-0c75b9a38a94+9:2270:x1862456979121472:12345-192.168.204.3@tcp:400:kworker/u8:3.0 Request processed in 35us (527us total) trans 0 rc 0/0 00000100:00100000:3.0:1776178403.618868:0:20137:0:(nrs_fifo.c:241:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.204.3@tcp, seq: 233 00000100:00100000:3.0:1776178403.618921:0:14440:0:(service.c:2152:ptlrpc_server_handle_req_in()) got req x1862456979121728 00000100:00100000:3.0:1776178403.618951:0:14440:0:(service.c:2152:ptlrpc_server_handle_req_in()) got req x1862456979121664 00000100:00100000:3.0:1776178403.618975:0:14440:0:(nrs_fifo.c:179:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.204.3@tcp, seq: 667 00000100:00100000:3.0:1776178403.618980:0:14440:0:(service.c:2324:ptlrpc_server_handle_request()) Handling RPC req@00000000380b40c4 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_004:b50963bd-8eba-4432-9a54-8888d00e3fa1+9:2270:x1862456979121728:12345-192.168.204.3@tcp:400:kworker/u8:3.0 00000100:00100000:3.0:1776178403.619014:0:14440:0:(service.c:2372:ptlrpc_server_handle_request()) Handled RPC req@00000000380b40c4 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_004:b50963bd-8eba-4432-9a54-8888d00e3fa1+9:2270:x1862456979121728:12345-192.168.204.3@tcp:400:kworker/u8:3.0 Request processed in 33us (736us total) trans 0 rc 0/0 00000100:00100000:3.0:1776178403.619020:0:14440:0:(nrs_fifo.c:241:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.204.3@tcp, seq: 667 00000100:00100000:3.0:1776178403.619033:0:14440:0:(nrs_fifo.c:179:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.204.3@tcp, seq: 668 00000100:00100000:3.0:1776178403.619036:0:14440:0:(service.c:2324:ptlrpc_server_handle_request()) Handling RPC req@000000007d5cfd93 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_004:b50963bd-8eba-4432-9a54-8888d00e3fa1+8:2270:x1862456979121664:12345-192.168.204.3@tcp:400:kworker/u8:3.0 00000100:00100000:3.0:1776178403.619092:0:14440:0:(service.c:2372:ptlrpc_server_handle_request()) Handled RPC req@000000007d5cfd93 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_004:b50963bd-8eba-4432-9a54-8888d00e3fa1+8:2270:x1862456979121664:12345-192.168.204.3@tcp:400:kworker/u8:3.0 Request processed in 22us (695us total) trans 0 rc 0/0 00000100:00100000:3.0:1776178403.619099:0:14440:0:(nrs_fifo.c:241:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.204.3@tcp, seq: 668 00010000:00010000:2.0:1776178403.808628:0:9224:0:(ldlm_lockd.c:2792:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00010000:00010000:2.0:1776178404.839084:0:9225:0:(ldlm_lockd.c:2792:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00080000:0.0:1776178405.220334:0:36:0:(pinger.c:251:ptlrpc_pinger_process_import()) lustre-MDT0000-lwp-OST0000_UUID->lustre-MDT0000_UUID: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00080000:0.0:1776178405.220343:0:36:0:(recover.c:57:ptlrpc_initiate_recovery()) lustre-MDT0000_UUID: starting recovery 00000100:00080000:0.0:1776178405.220346:0:36:0:(import.c:86:import_set_state_nolock()) 0000000047601a98 lustre-MDT0000_UUID: changing import state from DISCONN to CONNECTING 00000100:00080000:0.0:1776178405.220354:0:36:0:(import.c:534:import_select_connection()) lustre-MDT0000-lwp-OST0000: connect to NID 0@lo last attempt 1107 00000100:00080000:0.0:1776178405.220366:0:36:0:(import.c:581:import_select_connection()) lustre-MDT0000-lwp-OST0000: tried all connections, increasing latency to 21s 00000100:00080000:0.0:1776178405.220372:0:36:0:(import.c:615:import_select_connection()) lustre-MDT0000-lwp-OST0000: import 0000000047601a98 using connection 192.168.204.103@tcp/0@lo 00000100:00100000:0.0:1776178405.220394:0:36:0:(import.c:807:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5) req@000000006b0ae3d8 x1862456998077056/t0(0) o38->lustre-MDT0000-lwp-OST0000@0@lo:12/10 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' 00000100:00080000:0.0:1776178405.220443:0:36:0:(pinger.c:251:ptlrpc_pinger_process_import()) lustre-MDT0000-lwp-OST0001_UUID->lustre-MDT0000_UUID: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00080000:0.0:1776178405.220447:0:36:0:(recover.c:57:ptlrpc_initiate_recovery()) lustre-MDT0000_UUID: starting recovery 00000100:00080000:0.0:1776178405.220449:0:36:0:(import.c:86:import_set_state_nolock()) 00000000dfc8db13 lustre-MDT0000_UUID: changing import state from DISCONN to CONNECTING 00000100:00080000:0.0:1776178405.220453:0:36:0:(import.c:534:import_select_connection()) lustre-MDT0000-lwp-OST0001: connect to NID 0@lo last attempt 1107 00000100:00080000:0.0:1776178405.220455:0:36:0:(import.c:581:import_select_connection()) lustre-MDT0000-lwp-OST0001: tried all connections, increasing latency to 21s 00000100:00080000:0.0:1776178405.220459:0:36:0:(import.c:615:import_select_connection()) lustre-MDT0000-lwp-OST0001: import 00000000dfc8db13 using connection 192.168.204.103@tcp/0@lo 00000100:00100000:0.0:1776178405.220469:0:36:0:(import.c:807:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5) req@000000002b69594e x1862456998077184/t0(0) o38->lustre-MDT0000-lwp-OST0001@0@lo:12/10 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' 00000100:00100000:2.0:1776178405.220480:0:3352:0:(client.c:1733:ptlrpc_send_new_req()) Sending RPC req@000000006b0ae3d8 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:lustre-MDT0000-lwp-OST0000_UUID:3352:1862456998077056:0@lo:38:kworker/u8:1.0 00000100:00080000:0.0:1776178405.220482:0:36:0:(pinger.c:251:ptlrpc_pinger_process_import()) lustre-MDT0001-mdtlov_UUID->lustre-MDT0000_UUID: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00080000:0.0:1776178405.220485:0:36:0:(recover.c:57:ptlrpc_initiate_recovery()) lustre-MDT0000_UUID: starting recovery 00000100:00080000:0.0:1776178405.220486:0:36:0:(import.c:86:import_set_state_nolock()) 00000000ba4f1821 lustre-MDT0000_UUID: changing import state from DISCONN to CONNECTING 00000100:00080000:0.0:1776178405.220489:0:36:0:(import.c:534:import_select_connection()) lustre-MDT0000-osp-MDT0001: connect to NID 0@lo last attempt 1107 00000100:00080000:0.0:1776178405.220491:0:36:0:(import.c:581:import_select_connection()) lustre-MDT0000-osp-MDT0001: tried all connections, increasing latency to 21s 00000100:00080000:0.0:1776178405.220495:0:36:0:(import.c:615:import_select_connection()) lustre-MDT0000-osp-MDT0001: import 00000000ba4f1821 using connection 192.168.204.103@tcp/0@lo 00000100:00100000:0.0:1776178405.220504:0:36:0:(import.c:807:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5) req@000000007058013a x1862456998077312/t0(0) o38->lustre-MDT0000-osp-MDT0001@0@lo:24/4 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' 00000100:00080000:0.0:1776178405.220521:0:36:0:(pinger.c:251:ptlrpc_pinger_process_import()) lustre-MDT0000-lwp-MDT0001_UUID->lustre-MDT0000_UUID: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00080000:0.0:1776178405.220523:0:36:0:(recover.c:57:ptlrpc_initiate_recovery()) lustre-MDT0000_UUID: starting recovery 00000100:00080000:0.0:1776178405.220524:0:36:0:(import.c:86:import_set_state_nolock()) 00000000a7c55b1f lustre-MDT0000_UUID: changing import state from DISCONN to CONNECTING 00000100:00080000:0.0:1776178405.220527:0:36:0:(import.c:534:import_select_connection()) lustre-MDT0000-lwp-MDT0001: connect to NID 0@lo last attempt 1107 00000100:00080000:0.0:1776178405.220529:0:36:0:(import.c:581:import_select_connection()) lustre-MDT0000-lwp-MDT0001: tried all connections, increasing latency to 21s 00000100:00080000:0.0:1776178405.220531:0:36:0:(import.c:615:import_select_connection()) lustre-MDT0000-lwp-MDT0001: import 00000000a7c55b1f using connection 192.168.204.103@tcp/0@lo 00000100:00100000:0.0:1776178405.220538:0:36:0:(import.c:807:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5) req@00000000a8bae24c x1862456998077504/t0(0) o38->lustre-MDT0000-lwp-MDT0001@0@lo:12/10 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' 00000100:00100000:2.0:1776178405.220554:0:3352:0:(events.c:361:request_in_callback()) peer: 12345-0@lo (source: 12345-0@lo) 00000100:00100000:0.0:1776178405.220595:0:6255:0:(service.c:2152:ptlrpc_server_handle_req_in()) got req x1862456998077056 00000100:00100000:2.0:1776178405.220603:0:3352:0:(client.c:1733:ptlrpc_send_new_req()) Sending RPC req@000000002b69594e pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:lustre-MDT0000-lwp-OST0001_UUID:3352:1862456998077184:0@lo:38:kworker/u8:1.0 00000100:00100000:2.0:1776178405.220624:0:3352:0:(events.c:361:request_in_callback()) peer: 12345-0@lo (source: 12345-0@lo) 00000100:00100000:3.0:1776178405.220701:0:3353:0:(client.c:1733:ptlrpc_send_new_req()) Sending RPC req@00000000d8af564d pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_00:lustre-MDT0001-mdtlov_UUID:3353:1862456998077376:0@lo:400:kworker/u8:1.0 00000100:00100000:3.0:1776178405.220736:0:3353:0:(events.c:361:request_in_callback()) peer: 12345-0@lo (source: 12345-0@lo) 00000100:00100000:3.0:1776178405.220759:0:3353:0:(ptlrpcd.c:415:ptlrpcd_check()) transfer 2 async RPCs [1->0] 00000100:00100000:3.0:1776178405.220770:0:3353:0:(client.c:1733:ptlrpc_send_new_req()) Sending RPC req@000000001f101832 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_00:ac61f839-0e8e-4cac-9c59-08bcfbbad330:3353:1862456998076992:0@lo:400:kworker/u8:1.0 00000100:00100000:3.0:1776178405.220799:0:3353:0:(events.c:361:request_in_callback()) peer: 12345-0@lo (source: 12345-0@lo) 00000100:00100000:3.0:1776178405.220814:0:3353:0:(client.c:1733:ptlrpc_send_new_req()) Sending RPC req@00000000b8f6795f pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_00:lustre-MDT0001-mdtlov_UUID:3353:1862456998077440:0@lo:400:kworker/u8:1.0 00000100:00100000:3.0:1776178405.220834:0:3353:0:(events.c:361:request_in_callback()) peer: 12345-0@lo (source: 12345-0@lo) 00000100:00100000:3.0:1776178405.220862:0:20759:0:(service.c:2152:ptlrpc_server_handle_req_in()) got req x1862456998077376 00000100:00100000:3.0:1776178405.220906:0:20759:0:(service.c:2152:ptlrpc_server_handle_req_in()) got req x1862456998077440 00000100:00100000:3.0:1776178405.220930:0:20759:0:(nrs_fifo.c:179:nrs_fifo_req_get()) NRS start fifo request from 12345-0@lo, seq: 669 00000100:00100000:3.0:1776178405.220937:0:20759:0:(service.c:2324:ptlrpc_server_handle_request()) Handling RPC req@000000000daa3c9a pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_010:lustre-MDT0001-mdtlov_UUID+5:3353:x1862456998077376:12345-0@lo:400:kworker/u8:1.0 00000100:00100000:3.0:1776178405.220990:0:20759:0:(service.c:2372:ptlrpc_server_handle_request()) Handled RPC req@000000000daa3c9a pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_010:lustre-MDT0001-mdtlov_UUID+5:3353:x1862456998077376:12345-0@lo:400:kworker/u8:1.0 Request processed in 53us (254us total) trans 0 rc 0/0 00000100:00100000:3.0:1776178405.220997:0:20759:0:(nrs_fifo.c:241:nrs_fifo_req_stop()) NRS stop fifo request from 12345-0@lo, seq: 669 00000100:00100000:3.0:1776178405.221013:0:20759:0:(nrs_fifo.c:179:nrs_fifo_req_get()) NRS start fifo request from 12345-0@lo, seq: 670 00000100:00100000:3.0:1776178405.221017:0:20759:0:(service.c:2324:ptlrpc_server_handle_request()) Handling RPC req@0000000087a31eb2 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_010:lustre-MDT0001-mdtlov_UUID+5:3353:x1862456998077440:12345-0@lo:400:kworker/u8:1.0 00000100:00100000:3.0:1776178405.221041:0:20759:0:(service.c:2372:ptlrpc_server_handle_request()) Handled RPC req@0000000087a31eb2 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_010:lustre-MDT0001-mdtlov_UUID+5:3353:x1862456998077440:12345-0@lo:400:kworker/u8:1.0 Request processed in 24us (207us total) trans 0 rc 0/0 00000100:00100000:3.0:1776178405.221046:0:20759:0:(nrs_fifo.c:241:nrs_fifo_req_stop()) NRS stop fifo request from 12345-0@lo, seq: 670 00000100:00100000:3.0:1776178405.221107:0:20140:0:(service.c:2152:ptlrpc_server_handle_req_in()) got req x1862456998076992 00000100:00100000:3.0:1776178405.221134:0:20140:0:(nrs_fifo.c:179:nrs_fifo_req_get()) NRS start fifo request from 12345-0@lo, seq: 234 00000100:00100000:3.0:1776178405.221137:0:20140:0:(service.c:2324:ptlrpc_server_handle_request()) Handling RPC req@0000000020ee145c pname:cluuid+ref:pid:xid:nid:opc:job ll_mgs_0002:ac61f839-0e8e-4cac-9c59-08bcfbbad330+18:3353:x1862456998076992:12345-0@lo:400:kworker/u8:1.0 00000100:00100000:3.0:1776178405.221165:0:20140:0:(service.c:2372:ptlrpc_server_handle_request()) Handled RPC req@0000000020ee145c pname:cluuid+ref:pid:xid:nid:opc:job ll_mgs_0002:ac61f839-0e8e-4cac-9c59-08bcfbbad330+18:3353:x1862456998076992:12345-0@lo:400:kworker/u8:1.0 Request processed in 26us (365us total) trans 0 rc 0/0 00000100:00100000:3.0:1776178405.221171:0:20140:0:(nrs_fifo.c:241:nrs_fifo_req_stop()) NRS stop fifo request from 12345-0@lo, seq: 234 00000100:00100000:3.0:1776178405.221205:0:3356:0:(client.c:1733:ptlrpc_send_new_req()) Sending RPC req@000000007886b426 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_03:lustre-MDT0001-lwp-OST0001_UUID:3356:1862456998077248:0@lo:400:kworker/u8:1.0 00000100:00100000:3.0:1776178405.221230:0:3355:0:(client.c:1733:ptlrpc_send_new_req()) Sending RPC req@00000000bc95ec51 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_02:lustre-MDT0001-lwp-OST0000_UUID:3355:1862456998077120:0@lo:400:kworker/u8:1.0 00000100:00100000:3.0:1776178405.221568:0:3353:0:(client.c:2215:ptlrpc_check_set()) Completed RPC req@000000001f101832 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_00:ac61f839-0e8e-4cac-9c59-08bcfbbad330:3353:1862456998076992:0@lo:400:kworker/u8:1.0 00000100:00100000:3.0:1776178405.221584:0:3353:0:(client.c:2215:ptlrpc_check_set()) Completed RPC req@00000000b8f6795f pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_00:lustre-MDT0001-mdtlov_UUID:3353:1862456998077440:0@lo:400:kworker/u8:1.0 00000100:00100000:3.0:1776178405.221591:0:3353:0:(client.c:2215:ptlrpc_check_set()) Completed RPC req@00000000d8af564d pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_00:lustre-MDT0001-mdtlov_UUID:3353:1862456998077376:0@lo:400:kworker/u8:1.0 00000100:00100000:2.0:1776178405.225813:0:3352:0:(client.c:1733:ptlrpc_send_new_req()) Sending RPC req@000000007058013a pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:lustre-MDT0001-mdtlov_UUID:3352:1862456998077312:0@lo:38:kworker/u8:1.0 00000100:00100000:2.0:1776178405.225905:0:3352:0:(events.c:361:request_in_callback()) peer: 12345-0@lo (source: 12345-0@lo) 00000100:00100000:2.0:1776178405.225924:0:3352:0:(client.c:1733:ptlrpc_send_new_req()) Sending RPC req@00000000a8bae24c pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:lustre-MDT0000-lwp-MDT0001_UUID:3352:1862456998077504:0@lo:38:kworker/u8:1.0 00000100:00100000:2.0:1776178405.225948:0:3352:0:(events.c:361:request_in_callback()) peer: 12345-0@lo (source: 12345-0@lo) 00000100:00100000:0.0:1776178405.229512:0:6255:0:(service.c:2152:ptlrpc_server_handle_req_in()) got req x1862456998077184 00000100:00100000:0.0:1776178405.229602:0:6255:0:(nrs_fifo.c:179:nrs_fifo_req_get()) NRS start fifo request from 12345-0@lo, seq: 6781 00000100:00100000:0.0:1776178405.229610:0:6255:0:(service.c:2324:ptlrpc_server_handle_request()) Handling RPC req@00000000bd795500 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_000:0+-99:3352:x1862456998077056:12345-0@lo:38:kworker/u8:1.0 00010000:02000000:0.0:1776178405.229637:0:6255:0:(ldlm_lib.c:1139:target_handle_connect()) lustre-MDT0000: Not available for connect from 0@lo (stopping) 00000100:00100000:0.0:1776178405.229750:0:6255:0:(service.c:2372:ptlrpc_server_handle_request()) Handled RPC req@00000000bd795500 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_000:0+-99:3352:x1862456998077056:12345-0@lo:38:kworker/u8:1.0 Request processed in 139us (9196us total) trans 0 rc -19/-19 00000100:00100000:0.0:1776178405.229758:0:6255:0:(nrs_fifo.c:241:nrs_fifo_req_stop()) NRS stop fifo request from 12345-0@lo, seq: 6781 00000100:00100000:0.0:1776178405.229777:0:6255:0:(nrs_fifo.c:179:nrs_fifo_req_get()) NRS start fifo request from 12345-0@lo, seq: 6782 00000100:00100000:0.0:1776178405.229781:0:6255:0:(service.c:2324:ptlrpc_server_handle_request()) Handling RPC req@00000000a7e4e4e8 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_000:0+-99:3352:x1862456998077184:12345-0@lo:38:kworker/u8:1.0 00010000:02000000:0.0:1776178405.229789:0:6255:0:(ldlm_lib.c:1139:target_handle_connect()) lustre-MDT0000: Not available for connect from 0@lo (stopping) 00000100:00100000:0.0:1776178405.229812:0:6255:0:(service.c:2372:ptlrpc_server_handle_request()) Handled RPC req@00000000a7e4e4e8 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_000:0+-99:3352:x1862456998077184:12345-0@lo:38:kworker/u8:1.0 Request processed in 30us (9188us total) trans 0 rc -19/-19 00000100:00100000:0.0:1776178405.229817:0:6255:0:(nrs_fifo.c:241:nrs_fifo_req_stop()) NRS stop fifo request from 12345-0@lo, seq: 6782 00000100:00100000:0.0:1776178405.229913:0:15213:0:(service.c:2152:ptlrpc_server_handle_req_in()) got req x1862456998077312 00000100:00100000:0.0:1776178405.229945:0:15213:0:(nrs_fifo.c:179:nrs_fifo_req_get()) NRS start fifo request from 12345-0@lo, seq: 747 00000100:00100000:0.0:1776178405.229949:0:15213:0:(service.c:2324:ptlrpc_server_handle_request()) Handling RPC req@000000001d2ef500 pname:cluuid+ref:pid:xid:nid:opc:job mdt_out00_003:0+-99:3352:x1862456998077312:12345-0@lo:38:kworker/u8:1.0 00010000:02000000:0.0:1776178405.229956:0:15213:0:(ldlm_lib.c:1139:target_handle_connect()) lustre-MDT0000: Not available for connect from 0@lo (stopping) 00000100:00100000:0.0:1776178405.229982:0:15213:0:(service.c:2372:ptlrpc_server_handle_request()) Handled RPC req@000000001d2ef500 pname:cluuid+ref:pid:xid:nid:opc:job mdt_out00_003:0+-99:3352:x1862456998077312:12345-0@lo:38:kworker/u8:1.0 Request processed in 32us (4078us total) trans 0 rc -19/-19 00000100:00100000:0.0:1776178405.229987:0:15213:0:(nrs_fifo.c:241:nrs_fifo_req_stop()) NRS stop fifo request from 12345-0@lo, seq: 747 00000100:00100000:1.0:1776178405.231332:0:8250:0:(service.c:2152:ptlrpc_server_handle_req_in()) got req x1862456998077504 00000100:00100000:1.0:1776178405.231398:0:8250:0:(nrs_fifo.c:179:nrs_fifo_req_get()) NRS start fifo request from 12345-0@lo, seq: 6783 00000100:00100000:1.0:1776178405.231404:0:8250:0:(service.c:2324:ptlrpc_server_handle_request()) Handling RPC req@00000000f7d56588 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_003:0+-99:3352:x1862456998077504:12345-0@lo:38:kworker/u8:1.0 00010000:02000000:1.0:1776178405.231422:0:8250:0:(ldlm_lib.c:1139:target_handle_connect()) lustre-MDT0000: Not available for connect from 0@lo (stopping) 00000100:00100000:1.0:1776178405.233696:0:8250:0:(service.c:2372:ptlrpc_server_handle_request()) Handled RPC req@00000000f7d56588 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_003:0+-99:3352:x1862456998077504:12345-0@lo:38:kworker/u8:1.0 Request processed in 2281us (7736us total) trans 0 rc -19/-19 00000100:00100000:3.0:1776178405.233829:0:3355:0:(events.c:361:request_in_callback()) peer: 12345-0@lo (source: 12345-0@lo) 00000100:00100000:1.0:1776178405.233842:0:8250:0:(nrs_fifo.c:241:nrs_fifo_req_stop()) NRS stop fifo request from 12345-0@lo, seq: 6783 00000100:00100000:0.0:1776178405.233894:0:3356:0:(events.c:361:request_in_callback()) peer: 12345-0@lo (source: 12345-0@lo) 00000100:00100000:1.0:1776178405.233906:0:8250:0:(service.c:2152:ptlrpc_server_handle_req_in()) got req x1862456998077120 00000100:00100000:3.0:1776178405.233932:0:11281:0:(service.c:2152:ptlrpc_server_handle_req_in()) got req x1862456998077248 00000100:00100000:3.0:1776178405.233976:0:6256:0:(nrs_fifo.c:179:nrs_fifo_req_get()) NRS start fifo request from 12345-0@lo, seq: 1046 00000100:00100000:3.0:1776178405.233983:0:6256:0:(service.c:2324:ptlrpc_server_handle_request()) Handling RPC req@00000000ba7f03ed pname:cluuid+ref:pid:xid:nid:opc:job mdt00_001:lustre-MDT0001-lwp-OST0001_UUID+5:3356:x1862456998077248:12345-0@lo:400:kworker/u8:1.0 00000100:00100000:3.0:1776178405.234096:0:6256:0:(service.c:2372:ptlrpc_server_handle_request()) Handled RPC req@00000000ba7f03ed pname:cluuid+ref:pid:xid:nid:opc:job mdt00_001:lustre-MDT0001-lwp-OST0001_UUID+5:3356:x1862456998077248:12345-0@lo:400:kworker/u8:1.0 Request processed in 112us (204us total) trans 0 rc 0/0 00000100:00100000:3.0:1776178405.234103:0:6256:0:(nrs_fifo.c:241:nrs_fifo_req_stop()) NRS stop fifo request from 12345-0@lo, seq: 1046 00000100:00100000:3.0:1776178405.234120:0:6256:0:(nrs_fifo.c:179:nrs_fifo_req_get()) NRS start fifo request from 12345-0@lo, seq: 1047 00000100:00100000:3.0:1776178405.234124:0:6256:0:(service.c:2324:ptlrpc_server_handle_request()) Handling RPC req@000000004c88c002 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_001:lustre-MDT0001-lwp-OST0000_UUID+5:3355:x1862456998077120:12345-0@lo:400:kworker/u8:1.0 00000100:00100000:3.0:1776178405.234155:0:6256:0:(service.c:2372:ptlrpc_server_handle_request()) Handled RPC req@000000004c88c002 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_001:lustre-MDT0001-lwp-OST0000_UUID+5:3355:x1862456998077120:12345-0@lo:400:kworker/u8:1.0 Request processed in 31us (330us total) trans 0 rc 0/0 00000100:00100000:3.0:1776178405.234161:0:6256:0:(nrs_fifo.c:241:nrs_fifo_req_stop()) NRS stop fifo request from 12345-0@lo, seq: 1047 00000100:00080000:2.0:1776178405.234611:0:3352:0:(import.c:86:import_set_state_nolock()) 00000000a7c55b1f lustre-MDT0000_UUID: changing import state from CONNECTING to DISCONN 00000100:00080000:2.0:1776178405.234618:0:3352:0:(import.c:1429:ptlrpc_connect_interpret()) recovery of lustre-MDT0000_UUID on 192.168.204.103@tcp failed (-19) 00000100:00100000:2.0:1776178405.234623:0:3352:0:(client.c:2215:ptlrpc_check_set()) Completed RPC req@00000000a8bae24c pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:lustre-MDT0000-lwp-MDT0001_UUID:3352:1862456998077504:0@lo:38:kworker/u8:1.0 00000100:00080000:2.0:1776178405.234637:0:3352:0:(import.c:86:import_set_state_nolock()) 0000000047601a98 lustre-MDT0000_UUID: changing import state from CONNECTING to DISCONN 00000100:00080000:2.0:1776178405.234640:0:3352:0:(import.c:1429:ptlrpc_connect_interpret()) recovery of lustre-MDT0000_UUID on 192.168.204.103@tcp failed (-19) 00000100:00100000:2.0:1776178405.234643:0:3352:0:(client.c:2215:ptlrpc_check_set()) Completed RPC req@000000006b0ae3d8 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:lustre-MDT0000-lwp-OST0000_UUID:3352:1862456998077056:0@lo:38:kworker/u8:1.0 00000100:00080000:2.0:1776178405.234678:0:3352:0:(import.c:86:import_set_state_nolock()) 00000000dfc8db13 lustre-MDT0000_UUID: changing import state from CONNECTING to DISCONN 00000100:00080000:2.0:1776178405.234681:0:3352:0:(import.c:1429:ptlrpc_connect_interpret()) recovery of lustre-MDT0000_UUID on 192.168.204.103@tcp failed (-19) 00000100:00100000:2.0:1776178405.234683:0:3352:0:(client.c:2215:ptlrpc_check_set()) Completed RPC req@000000002b69594e pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:lustre-MDT0000-lwp-OST0001_UUID:3352:1862456998077184:0@lo:38:kworker/u8:1.0 00000100:00080000:2.0:1776178405.234695:0:3352:0:(import.c:86:import_set_state_nolock()) 00000000ba4f1821 lustre-MDT0000_UUID: changing import state from CONNECTING to DISCONN 00000100:00080000:2.0:1776178405.234698:0:3352:0:(import.c:1429:ptlrpc_connect_interpret()) recovery of lustre-MDT0000_UUID on 192.168.204.103@tcp failed (-19) 00000100:00100000:2.0:1776178405.234701:0:3352:0:(client.c:2215:ptlrpc_check_set()) Completed RPC req@000000007058013a pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:lustre-MDT0001-mdtlov_UUID:3352:1862456998077312:0@lo:38:kworker/u8:1.0 00000100:00100000:0.0:1776178405.234810:0:3356:0:(client.c:2215:ptlrpc_check_set()) Completed RPC req@000000007886b426 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_03:lustre-MDT0001-lwp-OST0001_UUID:3356:1862456998077248:0@lo:400:kworker/u8:1.0 00000100:00100000:3.0:1776178405.234926:0:3355:0:(client.c:2215:ptlrpc_check_set()) Completed RPC req@00000000bc95ec51 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_02:lustre-MDT0001-lwp-OST0000_UUID:3355:1862456998077120:0@lo:400:kworker/u8:1.0 00010000:00010000:3.0:1776178405.860895:0:6244:0:(ldlm_lockd.c:2792:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:0.0:1776178406.753266:0:3356:0:(ptlrpcd.c:415:ptlrpcd_check()) transfer 1 async RPCs [2->3] 00000100:00100000:0.0:1776178406.753296:0:3356:0:(client.c:1733:ptlrpc_send_new_req()) Sending RPC req@00000000c8afe7da pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_03:lustre-MDT0001-mdtlov_UUID:3356:1862456998077568:0@lo:13:osp-pre-1-1.0 00000100:00100000:0.0:1776178406.753386:0:3356:0:(events.c:361:request_in_callback()) peer: 12345-0@lo (source: 12345-0@lo) 00000100:00100000:2.0:1776178406.753503:0:8231:0:(service.c:2152:ptlrpc_server_handle_req_in()) got req x1862456998077568 00000100:00100000:2.0:1776178406.753578:0:8231:0:(nrs_fifo.c:179:nrs_fifo_req_get()) NRS start fifo request from 12345-0@lo, seq: 567 00000100:00100000:2.0:1776178406.753585:0:8231:0:(service.c:2324:ptlrpc_server_handle_request()) Handling RPC req@000000008516f059 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_create00:lustre-MDT0001-mdtlov_UUID+5:3356:x1862456998077568:12345-0@lo:13:osp-pre-1-1.0 00002000:00000024:2.0:1776178406.753632:0:8231:0:(ofd_obd.c:740:ofd_statfs()) blocks cached 229376 granted 491746368 pending 0 free 3923689472 avail 3692167168 00000100:00100000:2.0:1776178406.753713:0:8231:0:(service.c:2372:ptlrpc_server_handle_request()) Handled RPC req@000000008516f059 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_create00:lustre-MDT0001-mdtlov_UUID+5:3356:x1862456998077568:12345-0@lo:13:osp-pre-1-1.0 Request processed in 128us (329us total) trans 0 rc 0/0 00000100:00100000:2.0:1776178406.753721:0:8231:0:(nrs_fifo.c:241:nrs_fifo_req_stop()) NRS stop fifo request from 12345-0@lo, seq: 567 00000100:00100000:1.0:1776178406.756415:0:3356:0:(client.c:2215:ptlrpc_check_set()) Completed RPC req@00000000c8afe7da pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_03:lustre-MDT0001-mdtlov_UUID:3356:1862456998077568:0@lo:13:osp-pre-1-1.0 00010000:00010000:2.0:1776178406.880811:0:6245:0:(ldlm_lockd.c:2792:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:3.0:1776178407.267916:0:3355:0:(ptlrpcd.c:415:ptlrpcd_check()) transfer 1 async RPCs [3->2] 00000100:00100000:3.0:1776178407.268645:0:3355:0:(client.c:1733:ptlrpc_send_new_req()) Sending RPC req@000000006522caac pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_02:lustre-MDT0001-mdtlov_UUID:3355:1862456998077632:0@lo:13:osp-pre-0-1.0 00000100:00100000:3.0:1776178407.268733:0:3355:0:(events.c:361:request_in_callback()) peer: 12345-0@lo (source: 12345-0@lo) 00000100:00100000:0.0:1776178407.268906:0:9554:0:(service.c:2152:ptlrpc_server_handle_req_in()) got req x1862456998077632 00000100:00100000:0.0:1776178407.269051:0:9554:0:(nrs_fifo.c:179:nrs_fifo_req_get()) NRS start fifo request from 12345-0@lo, seq: 568 00000100:00100000:0.0:1776178407.269090:0:9554:0:(service.c:2324:ptlrpc_server_handle_request()) Handling RPC req@000000006b0ae3d8 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_create00:lustre-MDT0001-mdtlov_UUID+5:3355:x1862456998077632:12345-0@lo:13:osp-pre-0-1.0 00002000:00000024:0.0:1776178407.269592:0:9554:0:(ofd_obd.c:740:ofd_statfs()) blocks cached 315392 granted 484883776 pending 0 free 3923689472 avail 3692167168 00000100:00100000:0.0:1776178407.269655:0:9554:0:(service.c:2372:ptlrpc_server_handle_request()) Handled RPC req@000000006b0ae3d8 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_create00:lustre-MDT0001-mdtlov_UUID+5:3355:x1862456998077632:12345-0@lo:13:osp-pre-0-1.0 Request processed in 594us (921us total) trans 0 rc 0/0 00000100:00100000:0.0:1776178407.269662:0:9554:0:(nrs_fifo.c:241:nrs_fifo_req_stop()) NRS stop fifo request from 12345-0@lo, seq: 568 00000100:00100000:3.0:1776178407.269747:0:3355:0:(client.c:2215:ptlrpc_check_set()) Completed RPC req@000000006522caac pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_02:lustre-MDT0001-mdtlov_UUID:3355:1862456998077632:0@lo:13:osp-pre-0-1.0 00010000:00010000:2.0:1776178407.906479:0:9224:0:(ldlm_lockd.c:2792:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:0.0:1776178408.737730:0:3348:0:(events.c:361:request_in_callback()) peer: 12345-192.168.204.3@tcp (source: 12345-192.168.204.3@tcp) 00000100:00100000:0.0:1776178408.737789:0:3348:0:(events.c:361:request_in_callback()) peer: 12345-192.168.204.3@tcp (source: 12345-192.168.204.3@tcp) 00000100:00100000:0.0:1776178408.737834:0:3348:0:(events.c:361:request_in_callback()) peer: 12345-192.168.204.3@tcp (source: 12345-192.168.204.3@tcp) 00000100:00100000:1.0:1776178408.737940:0:6255:0:(service.c:2152:ptlrpc_server_handle_req_in()) got req x1862456979121856 00000100:00100000:1.0:1776178408.738001:0:6255:0:(service.c:2152:ptlrpc_server_handle_req_in()) got req x1862456979121920 00000100:00100000:1.0:1776178408.738036:0:6255:0:(nrs_fifo.c:179:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.204.3@tcp, seq: 1048 00000100:00100000:1.0:1776178408.738044:0:6255:0:(service.c:2324:ptlrpc_server_handle_request()) Handling RPC req@000000007058013a pname:cluuid+ref:pid:xid:nid:opc:job mdt00_000:b50963bd-8eba-4432-9a54-8888d00e3fa1+6:2270:x1862456979121920:12345-192.168.204.3@tcp:400:kworker/u8:3.0 00000100:00100000:1.0:1776178408.738194:0:6255:0:(service.c:2372:ptlrpc_server_handle_request()) Handled RPC req@000000007058013a pname:cluuid+ref:pid:xid:nid:opc:job mdt00_000:b50963bd-8eba-4432-9a54-8888d00e3fa1+6:2270:x1862456979121920:12345-192.168.204.3@tcp:400:kworker/u8:3.0 Request processed in 149us (359us total) trans 0 rc 0/0 00000100:00100000:1.0:1776178408.738202:0:6255:0:(nrs_fifo.c:241:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.204.3@tcp, seq: 1048 00000100:00100000:1.0:1776178408.738221:0:6255:0:(nrs_fifo.c:179:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.204.3@tcp, seq: 6784 00000100:00100000:1.0:1776178408.738225:0:6255:0:(service.c:2324:ptlrpc_server_handle_request()) Handling RPC req@000000006522caac pname:cluuid+ref:pid:xid:nid:opc:job mdt00_000:0+-99:2269:x1862456979121856:12345-192.168.204.3@tcp:38:kworker/u8:3.0 00010000:02000000:1.0:1776178408.738240:0:6255:0:(ldlm_lib.c:1139:target_handle_connect()) lustre-MDT0000: Not available for connect from 192.168.204.3@tcp (stopping) 00000100:00100000:1.0:1776178408.738267:0:6255:0:(service.c:2372:ptlrpc_server_handle_request()) Handled RPC req@000000006522caac pname:cluuid+ref:pid:xid:nid:opc:job mdt00_000:0+-99:2269:x1862456979121856:12345-192.168.204.3@tcp:38:kworker/u8:3.0 Request processed in 41us (550us total) trans 0 rc -19/-19 00000100:00100000:1.0:1776178408.738274:0:6255:0:(nrs_fifo.c:241:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.204.3@tcp, seq: 6784 00000100:00100000:0.0:1776178408.738531:0:20137:0:(service.c:2152:ptlrpc_server_handle_req_in()) got req x1862456979121792 00000100:00100000:0.0:1776178408.738571:0:20137:0:(nrs_fifo.c:179:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.204.3@tcp, seq: 235 00000100:00100000:0.0:1776178408.738575:0:20137:0:(service.c:2324:ptlrpc_server_handle_request()) Handling RPC req@00000000f8a3917d pname:cluuid+ref:pid:xid:nid:opc:job ll_mgs_0000:5b5c66b1-1a2f-4651-8548-0c75b9a38a94+9:2271:x1862456979121792:12345-192.168.204.3@tcp:400:kworker/u8:3.0 00000100:00100000:0.0:1776178408.738609:0:20137:0:(service.c:2372:ptlrpc_server_handle_request()) Handled RPC req@00000000f8a3917d pname:cluuid+ref:pid:xid:nid:opc:job ll_mgs_0000:5b5c66b1-1a2f-4651-8548-0c75b9a38a94+9:2271:x1862456979121792:12345-192.168.204.3@tcp:400:kworker/u8:3.0 Request processed in 33us (821us total) trans 0 rc 0/0 00000100:00100000:0.0:1776178408.738616:0:20137:0:(nrs_fifo.c:241:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.204.3@tcp, seq: 235 00000100:00100000:0.0:1776178408.744036:0:3348:0:(events.c:361:request_in_callback()) peer: 12345-192.168.204.3@tcp (source: 12345-192.168.204.3@tcp) 00000100:00100000:3.0:1776178408.744233:0:14440:0:(service.c:2152:ptlrpc_server_handle_req_in()) got req x1862456979121984 00000100:00100000:3.0:1776178408.744314:0:14440:0:(nrs_fifo.c:179:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.204.3@tcp, seq: 671 00000100:00100000:3.0:1776178408.744323:0:14440:0:(service.c:2324:ptlrpc_server_handle_request()) Handling RPC req@00000000a8bae24c pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_004:b50963bd-8eba-4432-9a54-8888d00e3fa1+8:2271:x1862456979121984:12345-192.168.204.3@tcp:400:kworker/u8:3.0 00000100:00100000:3.0:1776178408.746663:0:14440:0:(service.c:2372:ptlrpc_server_handle_request()) Handled RPC req@00000000a8bae24c pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_004:b50963bd-8eba-4432-9a54-8888d00e3fa1+8:2271:x1862456979121984:12345-192.168.204.3@tcp:400:kworker/u8:3.0 Request processed in 2331us (2645us total) trans 0 rc 0/0 00000100:00100000:3.0:1776178408.746681:0:14440:0:(nrs_fifo.c:241:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.204.3@tcp, seq: 671 00000100:00100000:0.0:1776178408.749699:0:3348:0:(events.c:361:request_in_callback()) peer: 12345-192.168.204.3@tcp (source: 12345-192.168.204.3@tcp) 00000100:00100000:3.0:1776178408.754344:0:14440:0:(service.c:2152:ptlrpc_server_handle_req_in()) got req x1862456979122048 00000100:00100000:3.0:1776178408.754503:0:14440:0:(nrs_fifo.c:179:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.204.3@tcp, seq: 672 00000100:00100000:3.0:1776178408.754513:0:14440:0:(service.c:2324:ptlrpc_server_handle_request()) Handling RPC req@00000000d8af564d pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_004:b50963bd-8eba-4432-9a54-8888d00e3fa1+9:2270:x1862456979122048:12345-192.168.204.3@tcp:400:kworker/u8:3.0 00000100:00100000:3.0:1776178408.754634:0:14440:0:(service.c:2372:ptlrpc_server_handle_request()) Handled RPC req@00000000d8af564d pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_004:b50963bd-8eba-4432-9a54-8888d00e3fa1+9:2270:x1862456979122048:12345-192.168.204.3@tcp:400:kworker/u8:3.0 Request processed in 121us (4943us total) trans 0 rc 0/0 00000100:00100000:3.0:1776178408.754642:0:14440:0:(nrs_fifo.c:241:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.204.3@tcp, seq: 672 00010000:00010000:2.0:1776178408.928514:0:9225:0:(ldlm_lockd.c:2792:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00010000:00010000:3.0:1776178409.953354:0:6244:0:(ldlm_lockd.c:2792:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00080000:3.0:1776178410.338708:0:37:0:(pinger.c:251:ptlrpc_pinger_process_import()) lustre-MDT0000-lwp-OST0000_UUID->lustre-MDT0000_UUID: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00080000:3.0:1776178410.338718:0:37:0:(recover.c:57:ptlrpc_initiate_recovery()) lustre-MDT0000_UUID: starting recovery 00000100:00080000:3.0:1776178410.338722:0:37:0:(import.c:86:import_set_state_nolock()) 0000000047601a98 lustre-MDT0000_UUID: changing import state from DISCONN to CONNECTING 00000100:00080000:3.0:1776178410.338731:0:37:0:(import.c:534:import_select_connection()) lustre-MDT0000-lwp-OST0000: connect to NID 0@lo last attempt 1112 00000100:00080000:3.0:1776178410.338735:0:37:0:(import.c:581:import_select_connection()) lustre-MDT0000-lwp-OST0000: tried all connections, increasing latency to 21s 00000100:00080000:3.0:1776178410.338742:0:37:0:(import.c:615:import_select_connection()) lustre-MDT0000-lwp-OST0000: import 0000000047601a98 using connection 192.168.204.103@tcp/0@lo 00000100:00100000:3.0:1776178410.338779:0:37:0:(import.c:807:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5) req@0000000013f6d767 x1862456998077760/t0(0) o38->lustre-MDT0000-lwp-OST0000@0@lo:12/10 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' 00000100:00080000:3.0:1776178410.338809:0:37:0:(pinger.c:251:ptlrpc_pinger_process_import()) lustre-MDT0000-lwp-OST0001_UUID->lustre-MDT0000_UUID: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00080000:3.0:1776178410.338812:0:37:0:(recover.c:57:ptlrpc_initiate_recovery()) lustre-MDT0000_UUID: starting recovery 00000100:00080000:3.0:1776178410.338814:0:37:0:(import.c:86:import_set_state_nolock()) 00000000dfc8db13 lustre-MDT0000_UUID: changing import state from DISCONN to CONNECTING 00000100:00080000:3.0:1776178410.338817:0:37:0:(import.c:534:import_select_connection()) lustre-MDT0000-lwp-OST0001: connect to NID 0@lo last attempt 1112 00000100:00080000:3.0:1776178410.338818:0:37:0:(import.c:581:import_select_connection()) lustre-MDT0000-lwp-OST0001: tried all connections, increasing latency to 21s 00000100:00080000:3.0:1776178410.338822:0:37:0:(import.c:615:import_select_connection()) lustre-MDT0000-lwp-OST0001: import 00000000dfc8db13 using connection 192.168.204.103@tcp/0@lo 00000100:00100000:3.0:1776178410.338832:0:37:0:(import.c:807:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5) req@000000001c465e98 x1862456998077888/t0(0) o38->lustre-MDT0000-lwp-OST0001@0@lo:12/10 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' 00000100:00080000:3.0:1776178410.338865:0:37:0:(pinger.c:251:ptlrpc_pinger_process_import()) lustre-MDT0001-mdtlov_UUID->lustre-MDT0000_UUID: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00080000:3.0:1776178410.338868:0:37:0:(recover.c:57:ptlrpc_initiate_recovery()) lustre-MDT0000_UUID: starting recovery 00000100:00080000:3.0:1776178410.338869:0:37:0:(import.c:86:import_set_state_nolock()) 00000000ba4f1821 lustre-MDT0000_UUID: changing import state from DISCONN to CONNECTING 00000100:00080000:3.0:1776178410.338872:0:37:0:(import.c:534:import_select_connection()) lustre-MDT0000-osp-MDT0001: connect to NID 0@lo last attempt 1112 00000100:00080000:3.0:1776178410.338875:0:37:0:(import.c:581:import_select_connection()) lustre-MDT0000-osp-MDT0001: tried all connections, increasing latency to 21s 00000100:00080000:3.0:1776178410.338878:0:37:0:(import.c:615:import_select_connection()) lustre-MDT0000-osp-MDT0001: import 00000000ba4f1821 using connection 192.168.204.103@tcp/0@lo 00000100:00100000:3.0:1776178410.338889:0:37:0:(import.c:807:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5) req@0000000098aa1ab6 x1862456998078016/t0(0) o38->lustre-MDT0000-osp-MDT0001@0@lo:24/4 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' 00000100:00080000:3.0:1776178410.338907:0:37:0:(pinger.c:251:ptlrpc_pinger_process_import()) lustre-MDT0000-lwp-MDT0001_UUID->lustre-MDT0000_UUID: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00080000:3.0:1776178410.338910:0:37:0:(recover.c:57:ptlrpc_initiate_recovery()) lustre-MDT0000_UUID: starting recovery 00000100:00080000:3.0:1776178410.338912:0:37:0:(import.c:86:import_set_state_nolock()) 00000000a7c55b1f lustre-MDT0000_UUID: changing import state from DISCONN to CONNECTING 00000100:00080000:3.0:1776178410.338914:0:37:0:(import.c:534:import_select_connection()) lustre-MDT0000-lwp-MDT0001: connect to NID 0@lo last attempt 1112 00000100:00080000:3.0:1776178410.338916:0:37:0:(import.c:581:import_select_connection()) lustre-MDT0000-lwp-MDT0001: tried all connections, increasing latency to 21s 00000100:00080000:3.0:1776178410.338919:0:37:0:(import.c:615:import_select_connection()) lustre-MDT0000-lwp-MDT0001: import 00000000a7c55b1f using connection 192.168.204.103@tcp/0@lo 00000100:00100000:3.0:1776178410.338927:0:37:0:(import.c:807:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5) req@00000000be94e5d7 x1862456998078208/t0(0) o38->lustre-MDT0000-lwp-MDT0001@0@lo:12/10 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' 00000100:00100000:3.0:1776178410.338959:0:3356:0:(client.c:1733:ptlrpc_send_new_req()) Sending RPC req@0000000007075047 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_03:lustre-MDT0001-mdtlov_UUID:3356:1862456998078080:0@lo:400:kworker/u8:2.0 00000100:00100000:3.0:1776178410.339035:0:3356:0:(events.c:361:request_in_callback()) peer: 12345-0@lo (source: 12345-0@lo) 00000100:00100000:3.0:1776178410.339114:0:3356:0:(ptlrpcd.c:415:ptlrpcd_check()) transfer 1 async RPCs [2->3] 00000100:00100000:3.0:1776178410.339119:0:3356:0:(client.c:1733:ptlrpc_send_new_req()) Sending RPC req@00000000f88b2465 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_03:lustre-MDT0001-lwp-OST0001_UUID:3356:1862456998077952:0@lo:400:kworker/u8:2.0 00000100:00100000:3.0:1776178410.339145:0:3356:0:(events.c:361:request_in_callback()) peer: 12345-0@lo (source: 12345-0@lo) 00000100:00100000:3.0:1776178410.339182:0:6256:0:(service.c:2152:ptlrpc_server_handle_req_in()) got req x1862456998077952 00000100:00100000:3.0:1776178410.339226:0:6256:0:(nrs_fifo.c:179:nrs_fifo_req_get()) NRS start fifo request from 12345-0@lo, seq: 1049 00000100:00100000:3.0:1776178410.339231:0:6256:0:(service.c:2324:ptlrpc_server_handle_request()) Handling RPC req@00000000bebb3448 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_001:lustre-MDT0001-lwp-OST0001_UUID+5:3356:x1862456998077952:12345-0@lo:400:kworker/u8:2.0 00000100:00100000:3.0:1776178410.339281:0:6256:0:(service.c:2372:ptlrpc_server_handle_request()) Handled RPC req@00000000bebb3448 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_001:lustre-MDT0001-lwp-OST0001_UUID+5:3356:x1862456998077952:12345-0@lo:400:kworker/u8:2.0 Request processed in 48us (135us total) trans 0 rc 0/0 00000100:00100000:3.0:1776178410.339289:0:6256:0:(nrs_fifo.c:241:nrs_fifo_req_stop()) NRS stop fifo request from 12345-0@lo, seq: 1049 00000100:00100000:3.0:1776178410.339344:0:3356:0:(client.c:2215:ptlrpc_check_set()) Completed RPC req@00000000f88b2465 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_03:lustre-MDT0001-lwp-OST0001_UUID:3356:1862456998077952:0@lo:400:kworker/u8:2.0 00000100:00100000:3.0:1776178410.339381:0:14440:0:(service.c:2152:ptlrpc_server_handle_req_in()) got req x1862456998078080 00000100:00100000:3.0:1776178410.339419:0:14440:0:(nrs_fifo.c:179:nrs_fifo_req_get()) NRS start fifo request from 12345-0@lo, seq: 673 00000100:00100000:3.0:1776178410.339422:0:14440:0:(service.c:2324:ptlrpc_server_handle_request()) Handling RPC req@00000000cc7f07e3 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_004:lustre-MDT0001-mdtlov_UUID+5:3356:x1862456998078080:12345-0@lo:400:kworker/u8:2.0 00000100:00100000:3.0:1776178410.339455:0:14440:0:(service.c:2372:ptlrpc_server_handle_request()) Handled RPC req@00000000cc7f07e3 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_004:lustre-MDT0001-mdtlov_UUID+5:3356:x1862456998078080:12345-0@lo:400:kworker/u8:2.0 Request processed in 31us (421us total) trans 0 rc 0/0 00000100:00100000:3.0:1776178410.339461:0:14440:0:(nrs_fifo.c:241:nrs_fifo_req_stop()) NRS stop fifo request from 12345-0@lo, seq: 673 00000100:00100000:3.0:1776178410.339492:0:3356:0:(client.c:2215:ptlrpc_check_set()) Completed RPC req@0000000007075047 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_03:lustre-MDT0001-mdtlov_UUID:3356:1862456998078080:0@lo:400:kworker/u8:2.0 00000100:00100000:0.0:1776178410.339584:0:3353:0:(client.c:1733:ptlrpc_send_new_req()) Sending RPC req@000000001f101832 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_00:ac61f839-0e8e-4cac-9c59-08bcfbbad330:3353:1862456998077696:0@lo:400:kworker/u8:2.0 00000100:00100000:0.0:1776178410.339624:0:3353:0:(events.c:361:request_in_callback()) peer: 12345-0@lo (source: 12345-0@lo) 00000100:00100000:0.0:1776178410.339648:0:3353:0:(client.c:1733:ptlrpc_send_new_req()) Sending RPC req@00000000375ba3dd pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_00:lustre-MDT0001-mdtlov_UUID:3353:1862456998078144:0@lo:400:kworker/u8:2.0 00000100:00100000:0.0:1776178410.339667:0:3353:0:(events.c:361:request_in_callback()) peer: 12345-0@lo (source: 12345-0@lo) 00000100:00100000:0.0:1776178410.339681:0:3353:0:(ptlrpcd.c:415:ptlrpcd_check()) transfer 1 async RPCs [1->0] 00000100:00100000:0.0:1776178410.339685:0:3353:0:(client.c:1733:ptlrpc_send_new_req()) Sending RPC req@0000000096181210 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_00:lustre-MDT0001-lwp-OST0000_UUID:3353:1862456998077824:0@lo:400:kworker/u8:2.0 00000100:00100000:0.0:1776178410.339706:0:3353:0:(events.c:361:request_in_callback()) peer: 12345-0@lo (source: 12345-0@lo) 00000100:00100000:2.0:1776178410.339862:0:8250:0:(service.c:2152:ptlrpc_server_handle_req_in()) got req x1862456998077824 00000100:00100000:2.0:1776178410.339897:0:8250:0:(nrs_fifo.c:179:nrs_fifo_req_get()) NRS start fifo request from 12345-0@lo, seq: 1050 00000100:00100000:2.0:1776178410.339901:0:8250:0:(service.c:2324:ptlrpc_server_handle_request()) Handling RPC req@00000000e7c1b281 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_003:lustre-MDT0001-lwp-OST0000_UUID+5:3353:x1862456998077824:12345-0@lo:400:kworker/u8:2.0 00000100:00100000:2.0:1776178410.339939:0:8250:0:(service.c:2372:ptlrpc_server_handle_request()) Handled RPC req@00000000e7c1b281 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_003:lustre-MDT0001-lwp-OST0000_UUID+5:3353:x1862456998077824:12345-0@lo:400:kworker/u8:2.0 Request processed in 37us (233us total) trans 0 rc 0/0 00000100:00100000:2.0:1776178410.339945:0:8250:0:(nrs_fifo.c:241:nrs_fifo_req_stop()) NRS stop fifo request from 12345-0@lo, seq: 1050 00000100:00100000:2.0:1776178410.339973:0:3353:0:(client.c:2215:ptlrpc_check_set()) Completed RPC req@0000000096181210 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_00:lustre-MDT0001-lwp-OST0000_UUID:3353:1862456998077824:0@lo:400:kworker/u8:2.0 00000100:00100000:2.0:1776178410.339989:0:3352:0:(client.c:1733:ptlrpc_send_new_req()) Sending RPC req@0000000013f6d767 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:lustre-MDT0000-lwp-OST0000_UUID:3352:1862456998077760:0@lo:38:kworker/u8:2.0 00000100:00100000:2.0:1776178410.340015:0:3352:0:(events.c:361:request_in_callback()) peer: 12345-0@lo (source: 12345-0@lo) 00000100:00100000:2.0:1776178410.340033:0:8250:0:(service.c:2152:ptlrpc_server_handle_req_in()) got req x1862456998077760 00000100:00100000:2.0:1776178410.340054:0:8250:0:(nrs_fifo.c:179:nrs_fifo_req_get()) NRS start fifo request from 12345-0@lo, seq: 6785 00000100:00100000:2.0:1776178410.340057:0:8250:0:(service.c:2324:ptlrpc_server_handle_request()) Handling RPC req@00000000c3b47349 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_003:0+-99:3352:x1862456998077760:12345-0@lo:38:kworker/u8:2.0 00010000:02000000:2.0:1776178410.340095:0:8250:0:(ldlm_lib.c:1139:target_handle_connect()) lustre-MDT0000: Not available for connect from 0@lo (stopping) 00000100:00100000:2.0:1776178410.340123:0:8250:0:(service.c:2372:ptlrpc_server_handle_request()) Handled RPC req@00000000c3b47349 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_003:0+-99:3352:x1862456998077760:12345-0@lo:38:kworker/u8:2.0 Request processed in 65us (108us total) trans 0 rc -19/-19 00000100:00100000:2.0:1776178410.340129:0:8250:0:(nrs_fifo.c:241:nrs_fifo_req_stop()) NRS stop fifo request from 12345-0@lo, seq: 6785 00000100:00100000:2.0:1776178410.340147:0:3352:0:(client.c:1733:ptlrpc_send_new_req()) Sending RPC req@000000001c465e98 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:lustre-MDT0000-lwp-OST0001_UUID:3352:1862456998077888:0@lo:38:kworker/u8:2.0 00000100:00100000:2.0:1776178410.340169:0:3352:0:(events.c:361:request_in_callback()) peer: 12345-0@lo (source: 12345-0@lo) 00000100:00100000:2.0:1776178410.340186:0:6255:0:(service.c:2152:ptlrpc_server_handle_req_in()) got req x1862456998077888 00000100:00100000:2.0:1776178410.340202:0:6255:0:(nrs_fifo.c:179:nrs_fifo_req_get()) NRS start fifo request from 12345-0@lo, seq: 6786 00000100:00100000:2.0:1776178410.340205:0:6255:0:(service.c:2324:ptlrpc_server_handle_request()) Handling RPC req@00000000c8afe7da pname:cluuid+ref:pid:xid:nid:opc:job mdt00_000:0+-99:3352:x1862456998077888:12345-0@lo:38:kworker/u8:2.0 00010000:02000000:2.0:1776178410.340212:0:6255:0:(ldlm_lib.c:1139:target_handle_connect()) lustre-MDT0000: Not available for connect from 0@lo (stopping) 00000100:00100000:2.0:1776178410.340235:0:6255:0:(service.c:2372:ptlrpc_server_handle_request()) Handled RPC req@00000000c8afe7da pname:cluuid+ref:pid:xid:nid:opc:job mdt00_000:0+-99:3352:x1862456998077888:12345-0@lo:38:kworker/u8:2.0 Request processed in 29us (66us total) trans 0 rc -19/-19 00000100:00100000:2.0:1776178410.340240:0:6255:0:(nrs_fifo.c:241:nrs_fifo_req_stop()) NRS stop fifo request from 12345-0@lo, seq: 6786 00000100:00100000:2.0:1776178410.340256:0:3352:0:(client.c:1733:ptlrpc_send_new_req()) Sending RPC req@0000000098aa1ab6 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:lustre-MDT0001-mdtlov_UUID:3352:1862456998078016:0@lo:38:kworker/u8:2.0 00000100:00100000:2.0:1776178410.340279:0:3352:0:(events.c:361:request_in_callback()) peer: 12345-0@lo (source: 12345-0@lo) 00000100:00100000:2.0:1776178410.340301:0:14128:0:(service.c:2152:ptlrpc_server_handle_req_in()) got req x1862456998078016 00000100:00100000:2.0:1776178410.340318:0:14128:0:(nrs_fifo.c:179:nrs_fifo_req_get()) NRS start fifo request from 12345-0@lo, seq: 748 00000100:00100000:2.0:1776178410.340322:0:14128:0:(service.c:2324:ptlrpc_server_handle_request()) Handling RPC req@0000000096181210 pname:cluuid+ref:pid:xid:nid:opc:job mdt_out00_002:0+-99:3352:x1862456998078016:12345-0@lo:38:kworker/u8:2.0 00010000:02000000:2.0:1776178410.340328:0:14128:0:(ldlm_lib.c:1139:target_handle_connect()) lustre-MDT0000: Not available for connect from 0@lo (stopping) 00000100:00100000:2.0:1776178410.340349:0:14128:0:(service.c:2372:ptlrpc_server_handle_request()) Handled RPC req@0000000096181210 pname:cluuid+ref:pid:xid:nid:opc:job mdt_out00_002:0+-99:3352:x1862456998078016:12345-0@lo:38:kworker/u8:2.0 Request processed in 27us (71us total) trans 0 rc -19/-19 00000100:00100000:2.0:1776178410.340355:0:14128:0:(nrs_fifo.c:241:nrs_fifo_req_stop()) NRS stop fifo request from 12345-0@lo, seq: 748 00000100:00100000:2.0:1776178410.340370:0:3352:0:(client.c:1733:ptlrpc_send_new_req()) Sending RPC req@00000000be94e5d7 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:lustre-MDT0000-lwp-MDT0001_UUID:3352:1862456998078208:0@lo:38:kworker/u8:2.0 00000100:00100000:2.0:1776178410.340392:0:3352:0:(events.c:361:request_in_callback()) peer: 12345-0@lo (source: 12345-0@lo) 00000100:00100000:3.0:1776178410.340593:0:20140:0:(service.c:2152:ptlrpc_server_handle_req_in()) got req x1862456998077696 00000100:00100000:3.0:1776178410.340629:0:20140:0:(nrs_fifo.c:179:nrs_fifo_req_get()) NRS start fifo request from 12345-0@lo, seq: 236 00000100:00100000:3.0:1776178410.340633:0:20140:0:(service.c:2324:ptlrpc_server_handle_request()) Handling RPC req@000000006fd17377 pname:cluuid+ref:pid:xid:nid:opc:job ll_mgs_0002:ac61f839-0e8e-4cac-9c59-08bcfbbad330+18:3353:x1862456998077696:12345-0@lo:400:kworker/u8:2.0 00000100:00100000:3.0:1776178410.346701:0:20140:0:(service.c:2372:ptlrpc_server_handle_request()) Handled RPC req@000000006fd17377 pname:cluuid+ref:pid:xid:nid:opc:job ll_mgs_0002:ac61f839-0e8e-4cac-9c59-08bcfbbad330+18:3353:x1862456998077696:12345-0@lo:400:kworker/u8:2.0 Request processed in 6050us (7060us total) trans 0 rc 0/0 00000100:00100000:3.0:1776178410.346724:0:20140:0:(nrs_fifo.c:241:nrs_fifo_req_stop()) NRS stop fifo request from 12345-0@lo, seq: 236 00000100:00100000:3.0:1776178410.346830:0:20755:0:(service.c:2152:ptlrpc_server_handle_req_in()) got req x1862456998078144 00000100:00100000:3.0:1776178410.346968:0:20755:0:(nrs_fifo.c:179:nrs_fifo_req_get()) NRS start fifo request from 12345-0@lo, seq: 674 00000100:00100000:3.0:1776178410.346975:0:20755:0:(service.c:2324:ptlrpc_server_handle_request()) Handling RPC req@00000000b8f6795f pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_006:lustre-MDT0001-mdtlov_UUID+5:3353:x1862456998078144:12345-0@lo:400:kworker/u8:2.0 00000100:00100000:3.0:1776178410.347053:0:20755:0:(service.c:2372:ptlrpc_server_handle_request()) Handled RPC req@00000000b8f6795f pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_006:lustre-MDT0001-mdtlov_UUID+5:3353:x1862456998078144:12345-0@lo:400:kworker/u8:2.0 Request processed in 78us (7384us total) trans 0 rc 0/0 00000100:00100000:3.0:1776178410.347085:0:20755:0:(nrs_fifo.c:241:nrs_fifo_req_stop()) NRS stop fifo request from 12345-0@lo, seq: 674 00000100:00100000:0.0:1776178410.347204:0:11281:0:(service.c:2152:ptlrpc_server_handle_req_in()) got req x1862456998078208 00000100:00100000:0.0:1776178410.347243:0:11281:0:(nrs_fifo.c:179:nrs_fifo_req_get()) NRS start fifo request from 12345-0@lo, seq: 6787 00000100:00100000:0.0:1776178410.347247:0:11281:0:(service.c:2324:ptlrpc_server_handle_request()) Handling RPC req@00000000bc95ec51 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_004:0+-99:3352:x1862456998078208:12345-0@lo:38:kworker/u8:2.0 00010000:02000000:0.0:1776178410.347268:0:11281:0:(ldlm_lib.c:1139:target_handle_connect()) lustre-MDT0000: Not available for connect from 0@lo (stopping) 00000100:00100000:0.0:1776178410.347310:0:11281:0:(service.c:2372:ptlrpc_server_handle_request()) Handled RPC req@00000000bc95ec51 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_004:0+-99:3352:x1862456998078208:12345-0@lo:38:kworker/u8:2.0 Request processed in 62us (6918us total) trans 0 rc -19/-19 00000100:00100000:0.0:1776178410.347319:0:11281:0:(nrs_fifo.c:241:nrs_fifo_req_stop()) NRS stop fifo request from 12345-0@lo, seq: 6787 00000100:00100000:0.0:1776178410.347415:0:3353:0:(client.c:2215:ptlrpc_check_set()) Completed RPC req@000000001f101832 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_00:ac61f839-0e8e-4cac-9c59-08bcfbbad330:3353:1862456998077696:0@lo:400:kworker/u8:2.0 00000100:00100000:0.0:1776178410.347437:0:3353:0:(client.c:2215:ptlrpc_check_set()) Completed RPC req@00000000375ba3dd pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_00:lustre-MDT0001-mdtlov_UUID:3353:1862456998078144:0@lo:400:kworker/u8:2.0 00000100:00080000:3.0:1776178410.349215:0:3352:0:(import.c:86:import_set_state_nolock()) 0000000047601a98 lustre-MDT0000_UUID: changing import state from CONNECTING to DISCONN 00000100:00080000:3.0:1776178410.349226:0:3352:0:(import.c:1429:ptlrpc_connect_interpret()) recovery of lustre-MDT0000_UUID on 192.168.204.103@tcp failed (-19) 00000100:00100000:3.0:1776178410.349234:0:3352:0:(client.c:2215:ptlrpc_check_set()) Completed RPC req@0000000013f6d767 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:lustre-MDT0000-lwp-OST0000_UUID:3352:1862456998077760:0@lo:38:kworker/u8:2.0 00000100:00080000:3.0:1776178410.349251:0:3352:0:(import.c:86:import_set_state_nolock()) 00000000dfc8db13 lustre-MDT0000_UUID: changing import state from CONNECTING to DISCONN 00000100:00080000:3.0:1776178410.349254:0:3352:0:(import.c:1429:ptlrpc_connect_interpret()) recovery of lustre-MDT0000_UUID on 192.168.204.103@tcp failed (-19) 00000100:00100000:3.0:1776178410.349257:0:3352:0:(client.c:2215:ptlrpc_check_set()) Completed RPC req@000000001c465e98 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:lustre-MDT0000-lwp-OST0001_UUID:3352:1862456998077888:0@lo:38:kworker/u8:2.0 00000100:00080000:3.0:1776178410.349271:0:3352:0:(import.c:86:import_set_state_nolock()) 00000000ba4f1821 lustre-MDT0000_UUID: changing import state from CONNECTING to DISCONN 00000100:00080000:3.0:1776178410.349274:0:3352:0:(import.c:1429:ptlrpc_connect_interpret()) recovery of lustre-MDT0000_UUID on 192.168.204.103@tcp failed (-19) 00000100:00100000:3.0:1776178410.349277:0:3352:0:(client.c:2215:ptlrpc_check_set()) Completed RPC req@0000000098aa1ab6 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:lustre-MDT0001-mdtlov_UUID:3352:1862456998078016:0@lo:38:kworker/u8:2.0 00000100:00080000:3.0:1776178410.349285:0:3352:0:(import.c:86:import_set_state_nolock()) 00000000a7c55b1f lustre-MDT0000_UUID: changing import state from CONNECTING to DISCONN 00000100:00080000:3.0:1776178410.349288:0:3352:0:(import.c:1429:ptlrpc_connect_interpret()) recovery of lustre-MDT0000_UUID on 192.168.204.103@tcp failed (-19) 00000100:00100000:3.0:1776178410.349290:0:3352:0:(client.c:2215:ptlrpc_check_set()) Completed RPC req@00000000be94e5d7 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:lustre-MDT0000-lwp-MDT0001_UUID:3352:1862456998078208:0@lo:38:kworker/u8:2.0 00010000:00010000:2.0:1776178410.976341:0:6245:0:(ldlm_lockd.c:2792:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:1.0:1776178411.872472:0:3354:0:(client.c:1733:ptlrpc_send_new_req()) Sending RPC req@000000007886b426 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_01:lustre-MDT0001-mdtlov_UUID:3354:1862456998078272:0@lo:13:osp-pre-1-1.0 00000100:00100000:1.0:1776178411.872589:0:3354:0:(events.c:361:request_in_callback()) peer: 12345-0@lo (source: 12345-0@lo) 00000100:00100000:2.0:1776178411.872853:0:9554:0:(service.c:2152:ptlrpc_server_handle_req_in()) got req x1862456998078272 00000100:00100000:2.0:1776178411.872910:0:9554:0:(nrs_fifo.c:179:nrs_fifo_req_get()) NRS start fifo request from 12345-0@lo, seq: 569 00000100:00100000:2.0:1776178411.872917:0:9554:0:(service.c:2324:ptlrpc_server_handle_request()) Handling RPC req@000000008d6accd5 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_create00:lustre-MDT0001-mdtlov_UUID+5:3354:x1862456998078272:12345-0@lo:13:osp-pre-1-1.0 00002000:00000024:2.0:1776178411.872960:0:9554:0:(ofd_obd.c:740:ofd_statfs()) blocks cached 229376 granted 491746368 pending 0 free 3923689472 avail 3692167168 00000100:00100000:2.0:1776178411.873025:0:9554:0:(service.c:2372:ptlrpc_server_handle_request()) Handled RPC req@000000008d6accd5 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_create00:lustre-MDT0001-mdtlov_UUID+5:3354:x1862456998078272:12345-0@lo:13:osp-pre-1-1.0 Request processed in 108us (438us total) trans 0 rc 0/0 00000100:00100000:2.0:1776178411.873034:0:9554:0:(nrs_fifo.c:241:nrs_fifo_req_stop()) NRS stop fifo request from 12345-0@lo, seq: 569 00000100:00100000:3.0:1776178411.873172:0:3354:0:(client.c:2215:ptlrpc_check_set()) Completed RPC req@000000007886b426 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_01:lustre-MDT0001-mdtlov_UUID:3354:1862456998078272:0@lo:13:osp-pre-1-1.0 00010000:00010000:2.0:1776178412.000381:0:9224:0:(ldlm_lockd.c:2792:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:3.0:1776178412.384742:0:3355:0:(client.c:1733:ptlrpc_send_new_req()) Sending RPC req@00000000803faf52 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_02:lustre-MDT0001-mdtlov_UUID:3355:1862456998078336:0@lo:13:osp-pre-0-1.0 00000100:00100000:3.0:1776178412.384841:0:3355:0:(events.c:361:request_in_callback()) peer: 12345-0@lo (source: 12345-0@lo) 00000100:00100000:1.0:1776178412.384904:0:8231:0:(service.c:2152:ptlrpc_server_handle_req_in()) got req x1862456998078336 00000100:00100000:1.0:1776178412.384984:0:8231:0:(nrs_fifo.c:179:nrs_fifo_req_get()) NRS start fifo request from 12345-0@lo, seq: 570 00000100:00100000:1.0:1776178412.384992:0:8231:0:(service.c:2324:ptlrpc_server_handle_request()) Handling RPC req@00000000be94e5d7 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_create00:lustre-MDT0001-mdtlov_UUID+5:3355:x1862456998078336:12345-0@lo:13:osp-pre-0-1.0 00002000:00000024:1.0:1776178412.385030:0:8231:0:(ofd_obd.c:740:ofd_statfs()) blocks cached 315392 granted 484883776 pending 0 free 3923689472 avail 3692167168 00000100:00100000:1.0:1776178412.385118:0:8231:0:(service.c:2372:ptlrpc_server_handle_request()) Handled RPC req@00000000be94e5d7 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_create00:lustre-MDT0001-mdtlov_UUID+5:3355:x1862456998078336:12345-0@lo:13:osp-pre-0-1.0 Request processed in 126us (279us total) trans 0 rc 0/0 00000100:00100000:1.0:1776178412.385127:0:8231:0:(nrs_fifo.c:241:nrs_fifo_req_stop()) NRS stop fifo request from 12345-0@lo, seq: 570 00000100:00100000:3.0:1776178412.385197:0:3355:0:(client.c:2215:ptlrpc_check_set()) Completed RPC req@00000000803faf52 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_02:lustre-MDT0001-mdtlov_UUID:3355:1862456998078336:0@lo:13:osp-pre-0-1.0 00010000:00010000:2.0:1776178413.024712:0:9225:0:(ldlm_lockd.c:2792:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:1.0:1776178413.858917:0:3346:0:(events.c:361:request_in_callback()) peer: 12345-192.168.204.3@tcp (source: 12345-192.168.204.3@tcp) 00000100:00100000:1.0:1776178413.858966:0:3346:0:(events.c:361:request_in_callback()) peer: 12345-192.168.204.3@tcp (source: 12345-192.168.204.3@tcp) 00000100:00100000:1.0:1776178413.859001:0:3346:0:(events.c:361:request_in_callback()) peer: 12345-192.168.204.3@tcp (source: 12345-192.168.204.3@tcp) 00000100:00100000:1.0:1776178413.859042:0:3346:0:(events.c:361:request_in_callback()) peer: 12345-192.168.204.3@tcp (source: 12345-192.168.204.3@tcp) 00000100:00100000:2.0:1776178413.859102:0:8250:0:(service.c:2152:ptlrpc_server_handle_req_in()) got req x1862456979122240 00000100:00100000:1.0:1776178413.859115:0:3346:0:(events.c:361:request_in_callback()) peer: 12345-192.168.204.3@tcp (source: 12345-192.168.204.3@tcp) 00000100:00100000:1.0:1776178413.859145:0:14440:0:(service.c:2152:ptlrpc_server_handle_req_in()) got req x1862456979122368 00000100:00100000:2.0:1776178413.859166:0:8250:0:(service.c:2152:ptlrpc_server_handle_req_in()) got req x1862456979122176 00000100:00100000:1.0:1776178413.859180:0:14440:0:(service.c:2152:ptlrpc_server_handle_req_in()) got req x1862456979122304 00000100:00100000:1.0:1776178413.859214:0:14440:0:(nrs_fifo.c:179:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.204.3@tcp, seq: 675 00000100:00100000:2.0:1776178413.859214:0:8250:0:(nrs_fifo.c:179:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.204.3@tcp, seq: 1051 00000100:00100000:1.0:1776178413.859229:0:14440:0:(service.c:2324:ptlrpc_server_handle_request()) Handling RPC req@0000000068adf51c pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_004:b50963bd-8eba-4432-9a54-8888d00e3fa1+9:2270:x1862456979122368:12345-192.168.204.3@tcp:400:kworker/u8:3.0 00000100:00100000:2.0:1776178413.859229:0:8250:0:(service.c:2324:ptlrpc_server_handle_request()) Handling RPC req@00000000a9cfe89c pname:cluuid+ref:pid:xid:nid:opc:job mdt00_003:b50963bd-8eba-4432-9a54-8888d00e3fa1+6:2270:x1862456979122240:12345-192.168.204.3@tcp:400:kworker/u8:3.0 00000100:00100000:3.0:1776178413.859426:0:6256:0:(nrs_fifo.c:179:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.204.3@tcp, seq: 6788 00000100:00100000:3.0:1776178413.859431:0:6256:0:(service.c:2324:ptlrpc_server_handle_request()) Handling RPC req@00000000f141ac55 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_001:0+-99:2269:x1862456979122176:12345-192.168.204.3@tcp:38:kworker/u8:3.0 00010000:02000000:3.0:1776178413.859449:0:6256:0:(ldlm_lib.c:1139:target_handle_connect()) lustre-MDT0000: Not available for connect from 192.168.204.3@tcp (stopping) 00000100:00100000:0.0:1776178413.859529:0:20755:0:(nrs_fifo.c:179:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.204.3@tcp, seq: 676 00000100:00100000:0.0:1776178413.859535:0:20755:0:(service.c:2324:ptlrpc_server_handle_request()) Handling RPC req@00000000599b0cf2 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_006:b50963bd-8eba-4432-9a54-8888d00e3fa1+8:2270:x1862456979122304:12345-192.168.204.3@tcp:400:kworker/u8:3.0 00000100:00100000:1.0:1776178413.859600:0:14440:0:(service.c:2372:ptlrpc_server_handle_request()) Handled RPC req@0000000068adf51c pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_004:b50963bd-8eba-4432-9a54-8888d00e3fa1+9:2270:x1862456979122368:12345-192.168.204.3@tcp:400:kworker/u8:3.0 Request processed in 376us (598us total) trans 0 rc 0/0 00000100:00100000:1.0:1776178413.859610:0:14440:0:(nrs_fifo.c:241:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.204.3@tcp, seq: 675 00000100:00100000:2.0:1776178413.863108:0:8250:0:(service.c:2372:ptlrpc_server_handle_request()) Handled RPC req@00000000a9cfe89c pname:cluuid+ref:pid:xid:nid:opc:job mdt00_003:b50963bd-8eba-4432-9a54-8888d00e3fa1+6:2270:x1862456979122240:12345-192.168.204.3@tcp:400:kworker/u8:3.0 Request processed in 3871us (4128us total) trans 0 rc 0/0 00000100:00100000:2.0:1776178413.863131:0:8250:0:(nrs_fifo.c:241:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.204.3@tcp, seq: 1051 00000100:00100000:2.0:1776178413.863256:0:20140:0:(service.c:2152:ptlrpc_server_handle_req_in()) got req x1862456979122112 00000100:00100000:2.0:1776178413.863734:0:20140:0:(nrs_fifo.c:179:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.204.3@tcp, seq: 237 00000100:00100000:2.0:1776178413.863740:0:20140:0:(service.c:2324:ptlrpc_server_handle_request()) Handling RPC req@00000000eb350ccb pname:cluuid+ref:pid:xid:nid:opc:job ll_mgs_0002:5b5c66b1-1a2f-4651-8548-0c75b9a38a94+9:2270:x1862456979122112:12345-192.168.204.3@tcp:400:kworker/u8:3.0 00000100:00100000:2.0:1776178413.863802:0:20140:0:(service.c:2372:ptlrpc_server_handle_request()) Handled RPC req@00000000eb350ccb pname:cluuid+ref:pid:xid:nid:opc:job ll_mgs_0002:5b5c66b1-1a2f-4651-8548-0c75b9a38a94+9:2270:x1862456979122112:12345-192.168.204.3@tcp:400:kworker/u8:3.0 Request processed in 61us (4898us total) trans 0 rc 0/0 00000100:00100000:2.0:1776178413.863808:0:20140:0:(nrs_fifo.c:241:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.204.3@tcp, seq: 237 00000100:00100000:3.0:1776178413.865249:0:6256:0:(service.c:2372:ptlrpc_server_handle_request()) Handled RPC req@00000000f141ac55 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_001:0+-99:2269:x1862456979122176:12345-192.168.204.3@tcp:38:kworker/u8:3.0 Request processed in 5808us (6126us total) trans 0 rc -19/-19 00000100:00100000:3.0:1776178413.865270:0:6256:0:(nrs_fifo.c:241:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.204.3@tcp, seq: 6788 00000100:00100000:0.0:1776178413.868368:0:20755:0:(service.c:2372:ptlrpc_server_handle_request()) Handled RPC req@00000000599b0cf2 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_006:b50963bd-8eba-4432-9a54-8888d00e3fa1+8:2270:x1862456979122304:12345-192.168.204.3@tcp:400:kworker/u8:3.0 Request processed in 8816us (9310us total) trans 0 rc 0/0 00000100:00100000:0.0:1776178413.868399:0:20755:0:(nrs_fifo.c:241:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.204.3@tcp, seq: 676 00010000:00010000:3.0:1776178414.048907:0:6244:0:(ldlm_lockd.c:2792:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00010000:00010000:2.0:1776178415.075251:0:6245:0:(ldlm_lockd.c:2792:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00080000:2.0:1776178415.456527:0:36:0:(pinger.c:251:ptlrpc_pinger_process_import()) lustre-MDT0000-lwp-OST0000_UUID->lustre-MDT0000_UUID: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00080000:2.0:1776178415.456540:0:36:0:(recover.c:57:ptlrpc_initiate_recovery()) lustre-MDT0000_UUID: starting recovery 00000100:00080000:2.0:1776178415.456544:0:36:0:(import.c:86:import_set_state_nolock()) 0000000047601a98 lustre-MDT0000_UUID: changing import state from DISCONN to CONNECTING 00000100:00080000:2.0:1776178415.456558:0:36:0:(import.c:534:import_select_connection()) lustre-MDT0000-lwp-OST0000: connect to NID 0@lo last attempt 1117 00000100:00080000:2.0:1776178415.456562:0:36:0:(import.c:581:import_select_connection()) lustre-MDT0000-lwp-OST0000: tried all connections, increasing latency to 21s 00000100:00080000:2.0:1776178415.456571:0:36:0:(import.c:615:import_select_connection()) lustre-MDT0000-lwp-OST0000: import 0000000047601a98 using connection 192.168.204.103@tcp/0@lo 00000100:00100000:2.0:1776178415.456621:0:36:0:(import.c:807:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5) req@000000007886b426 x1862456998078464/t0(0) o38->lustre-MDT0000-lwp-OST0000@0@lo:12/10 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' 00000100:00080000:2.0:1776178415.456689:0:36:0:(pinger.c:251:ptlrpc_pinger_process_import()) lustre-MDT0000-lwp-OST0001_UUID->lustre-MDT0000_UUID: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00080000:2.0:1776178415.456693:0:36:0:(recover.c:57:ptlrpc_initiate_recovery()) lustre-MDT0000_UUID: starting recovery 00000100:00080000:2.0:1776178415.456695:0:36:0:(import.c:86:import_set_state_nolock()) 00000000dfc8db13 lustre-MDT0000_UUID: changing import state from DISCONN to CONNECTING 00000100:00080000:2.0:1776178415.456700:0:36:0:(import.c:534:import_select_connection()) lustre-MDT0000-lwp-OST0001: connect to NID 0@lo last attempt 1117 00000100:00080000:2.0:1776178415.456702:0:36:0:(import.c:581:import_select_connection()) lustre-MDT0000-lwp-OST0001: tried all connections, increasing latency to 21s 00000100:00080000:2.0:1776178415.456707:0:36:0:(import.c:615:import_select_connection()) lustre-MDT0000-lwp-OST0001: import 00000000dfc8db13 using connection 192.168.204.103@tcp/0@lo 00000100:00100000:2.0:1776178415.456722:0:36:0:(import.c:807:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5) req@000000004ce46e20 x1862456998078592/t0(0) o38->lustre-MDT0000-lwp-OST0001@0@lo:12/10 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' 00000100:00080000:2.0:1776178415.456750:0:36:0:(pinger.c:251:ptlrpc_pinger_process_import()) lustre-MDT0001-mdtlov_UUID->lustre-MDT0000_UUID: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00080000:2.0:1776178415.456754:0:36:0:(recover.c:57:ptlrpc_initiate_recovery()) lustre-MDT0000_UUID: starting recovery 00000100:00080000:2.0:1776178415.456755:0:36:0:(import.c:86:import_set_state_nolock()) 00000000ba4f1821 lustre-MDT0000_UUID: changing import state from DISCONN to CONNECTING 00000100:00080000:2.0:1776178415.456758:0:36:0:(import.c:534:import_select_connection()) lustre-MDT0000-osp-MDT0001: connect to NID 0@lo last attempt 1117 00000100:00080000:2.0:1776178415.456760:0:36:0:(import.c:581:import_select_connection()) lustre-MDT0000-osp-MDT0001: tried all connections, increasing latency to 21s 00000100:00080000:2.0:1776178415.456763:0:36:0:(import.c:615:import_select_connection()) lustre-MDT0000-osp-MDT0001: import 00000000ba4f1821 using connection 192.168.204.103@tcp/0@lo 00000100:00100000:2.0:1776178415.456776:0:36:0:(import.c:807:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5) req@0000000042de09e9 x1862456998078720/t0(0) o38->lustre-MDT0000-osp-MDT0001@0@lo:24/4 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' 00000100:00080000:2.0:1776178415.456806:0:36:0:(pinger.c:251:ptlrpc_pinger_process_import()) lustre-MDT0000-lwp-MDT0001_UUID->lustre-MDT0000_UUID: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00080000:2.0:1776178415.456812:0:36:0:(recover.c:57:ptlrpc_initiate_recovery()) lustre-MDT0000_UUID: starting recovery 00000100:00080000:2.0:1776178415.456813:0:36:0:(import.c:86:import_set_state_nolock()) 00000000a7c55b1f lustre-MDT0000_UUID: changing import state from DISCONN to CONNECTING 00000100:00080000:2.0:1776178415.456816:0:36:0:(import.c:534:import_select_connection()) lustre-MDT0000-lwp-MDT0001: connect to NID 0@lo last attempt 1117 00000100:00080000:2.0:1776178415.456818:0:36:0:(import.c:581:import_select_connection()) lustre-MDT0000-lwp-MDT0001: tried all connections, increasing latency to 21s 00000100:00080000:2.0:1776178415.456821:0:36:0:(import.c:615:import_select_connection()) lustre-MDT0000-lwp-MDT0001: import 00000000a7c55b1f using connection 192.168.204.103@tcp/0@lo 00000100:00100000:2.0:1776178415.456831:0:36:0:(import.c:807:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5) req@000000006b9d8ccb x1862456998078912/t0(0) o38->lustre-MDT0000-lwp-MDT0001@0@lo:12/10 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' 00000100:00100000:0.0:1776178415.456958:0:3353:0:(client.c:1733:ptlrpc_send_new_req()) Sending RPC req@0000000029e5c385 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_00:lustre-MDT0001-lwp-OST0000_UUID:3353:1862456998078528:0@lo:400:kworker/u8:1.0 00000100:00100000:0.0:1776178415.457121:0:3353:0:(events.c:361:request_in_callback()) peer: 12345-0@lo (source: 12345-0@lo) 00000100:00100000:0.0:1776178415.457172:0:3353:0:(ptlrpcd.c:415:ptlrpcd_check()) transfer 1 async RPCs [1->0] 00000100:00100000:0.0:1776178415.457178:0:3353:0:(client.c:1733:ptlrpc_send_new_req()) Sending RPC req@0000000037aace32 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_00:lustre-MDT0001-lwp-OST0001_UUID:3353:1862456998078656:0@lo:400:kworker/u8:1.0 00000100:00100000:0.0:1776178415.457201:0:3353:0:(events.c:361:request_in_callback()) peer: 12345-0@lo (source: 12345-0@lo) 00000100:00100000:0.0:1776178415.457260:0:8250:0:(service.c:2152:ptlrpc_server_handle_req_in()) got req x1862456998078528 00000100:00100000:0.0:1776178415.457330:0:8250:0:(service.c:2152:ptlrpc_server_handle_req_in()) got req x1862456998078656 00000100:00100000:0.0:1776178415.457354:0:8250:0:(nrs_fifo.c:179:nrs_fifo_req_get()) NRS start fifo request from 12345-0@lo, seq: 1052 00000100:00100000:0.0:1776178415.457365:0:8250:0:(service.c:2324:ptlrpc_server_handle_request()) Handling RPC req@00000000e67f6ac0 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_003:lustre-MDT0001-lwp-OST0000_UUID+5:3353:x1862456998078528:12345-0@lo:400:kworker/u8:1.0 00000100:00100000:0.0:1776178415.457448:0:8250:0:(service.c:2372:ptlrpc_server_handle_request()) Handled RPC req@00000000e67f6ac0 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_003:lustre-MDT0001-lwp-OST0000_UUID+5:3353:x1862456998078528:12345-0@lo:400:kworker/u8:1.0 Request processed in 84us (328us total) trans 0 rc 0/0 00000100:00100000:0.0:1776178415.457457:0:8250:0:(nrs_fifo.c:241:nrs_fifo_req_stop()) NRS stop fifo request from 12345-0@lo, seq: 1052 00000100:00100000:0.0:1776178415.457481:0:8250:0:(nrs_fifo.c:179:nrs_fifo_req_get()) NRS start fifo request from 12345-0@lo, seq: 1053 00000100:00100000:0.0:1776178415.457484:0:8250:0:(service.c:2324:ptlrpc_server_handle_request()) Handling RPC req@000000008be25fe6 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_003:lustre-MDT0001-lwp-OST0001_UUID+5:3353:x1862456998078656:12345-0@lo:400:kworker/u8:1.0 00000100:00100000:0.0:1776178415.457518:0:8250:0:(service.c:2372:ptlrpc_server_handle_request()) Handled RPC req@000000008be25fe6 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_003:lustre-MDT0001-lwp-OST0001_UUID+5:3353:x1862456998078656:12345-0@lo:400:kworker/u8:1.0 Request processed in 33us (317us total) trans 0 rc 0/0 00000100:00100000:0.0:1776178415.457526:0:8250:0:(nrs_fifo.c:241:nrs_fifo_req_stop()) NRS stop fifo request from 12345-0@lo, seq: 1053 00000100:00100000:3.0:1776178415.457540:0:3355:0:(client.c:1733:ptlrpc_send_new_req()) Sending RPC req@000000006344f537 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_02:lustre-MDT0001-mdtlov_UUID:3355:1862456998078784:0@lo:400:kworker/u8:1.0 00000100:00100000:0.0:1776178415.457574:0:3353:0:(client.c:2215:ptlrpc_check_set()) Completed RPC req@0000000037aace32 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_00:lustre-MDT0001-lwp-OST0001_UUID:3353:1862456998078656:0@lo:400:kworker/u8:1.0 00000100:00100000:0.0:1776178415.457589:0:3353:0:(client.c:2215:ptlrpc_check_set()) Completed RPC req@0000000029e5c385 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_00:lustre-MDT0001-lwp-OST0000_UUID:3353:1862456998078528:0@lo:400:kworker/u8:1.0 00000100:00100000:3.0:1776178415.457601:0:3355:0:(events.c:361:request_in_callback()) peer: 12345-0@lo (source: 12345-0@lo) 00000100:00100000:1.0:1776178415.457782:0:3356:0:(client.c:1733:ptlrpc_send_new_req()) Sending RPC req@000000002b69594e pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_03:ac61f839-0e8e-4cac-9c59-08bcfbbad330:3356:1862456998078400:0@lo:400:kworker/u8:1.0 00000100:00100000:1.0:1776178415.457904:0:3356:0:(events.c:361:request_in_callback()) peer: 12345-0@lo (source: 12345-0@lo) 00000100:00100000:1.0:1776178415.457965:0:3356:0:(client.c:1733:ptlrpc_send_new_req()) Sending RPC req@0000000050c5b9a3 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_03:lustre-MDT0001-mdtlov_UUID:3356:1862456998078848:0@lo:400:kworker/u8:1.0 00000100:00100000:1.0:1776178415.457993:0:3356:0:(events.c:361:request_in_callback()) peer: 12345-0@lo (source: 12345-0@lo) 00000100:00100000:1.0:1776178415.458129:0:20137:0:(service.c:2152:ptlrpc_server_handle_req_in()) got req x1862456998078400 00000100:00100000:1.0:1776178415.458186:0:20137:0:(nrs_fifo.c:179:nrs_fifo_req_get()) NRS start fifo request from 12345-0@lo, seq: 238 00000100:00100000:1.0:1776178415.458193:0:20137:0:(service.c:2324:ptlrpc_server_handle_request()) Handling RPC req@0000000048e8901d pname:cluuid+ref:pid:xid:nid:opc:job ll_mgs_0000:ac61f839-0e8e-4cac-9c59-08bcfbbad330+18:3356:x1862456998078400:12345-0@lo:400:kworker/u8:1.0 00000100:00100000:1.0:1776178415.458252:0:20137:0:(service.c:2372:ptlrpc_server_handle_request()) Handled RPC req@0000000048e8901d pname:cluuid+ref:pid:xid:nid:opc:job ll_mgs_0000:ac61f839-0e8e-4cac-9c59-08bcfbbad330+18:3356:x1862456998078400:12345-0@lo:400:kworker/u8:1.0 Request processed in 58us (353us total) trans 0 rc 0/0 00000100:00100000:1.0:1776178415.458259:0:20137:0:(nrs_fifo.c:241:nrs_fifo_req_stop()) NRS stop fifo request from 12345-0@lo, seq: 238 00000100:00100000:1.0:1776178415.458362:0:3356:0:(client.c:2215:ptlrpc_check_set()) Completed RPC req@000000002b69594e pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_03:ac61f839-0e8e-4cac-9c59-08bcfbbad330:3356:1862456998078400:0@lo:400:kworker/u8:1.0 00000100:00100000:0.0:1776178415.460488:0:20755:0:(service.c:2152:ptlrpc_server_handle_req_in()) got req x1862456998078784 00000100:00100000:3.0:1776178415.460903:0:3352:0:(client.c:1733:ptlrpc_send_new_req()) Sending RPC req@000000007886b426 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:lustre-MDT0000-lwp-OST0000_UUID:3352:1862456998078464:0@lo:38:kworker/u8:1.0 00000100:00100000:3.0:1776178415.461033:0:3352:0:(events.c:361:request_in_callback()) peer: 12345-0@lo (source: 12345-0@lo) 00000100:00100000:3.0:1776178415.461099:0:3352:0:(client.c:1733:ptlrpc_send_new_req()) Sending RPC req@000000004ce46e20 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:lustre-MDT0000-lwp-OST0001_UUID:3352:1862456998078592:0@lo:38:kworker/u8:1.0 00000100:00100000:1.0:1776178415.461385:0:6255:0:(service.c:2152:ptlrpc_server_handle_req_in()) got req x1862456998078464 00000100:00100000:2.0:1776178415.461459:0:20759:0:(service.c:2152:ptlrpc_server_handle_req_in()) got req x1862456998078848 00000100:00100000:3.0:1776178415.461537:0:3352:0:(events.c:361:request_in_callback()) peer: 12345-0@lo (source: 12345-0@lo) 00000100:00100000:1.0:1776178415.461556:0:6255:0:(service.c:2152:ptlrpc_server_handle_req_in()) got req x1862456998078592 00000100:00100000:3.0:1776178415.461564:0:3352:0:(client.c:1733:ptlrpc_send_new_req()) Sending RPC req@0000000042de09e9 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:lustre-MDT0001-mdtlov_UUID:3352:1862456998078720:0@lo:38:kworker/u8:1.0 00000100:00100000:1.0:1776178415.461599:0:6255:0:(nrs_fifo.c:179:nrs_fifo_req_get()) NRS start fifo request from 12345-0@lo, seq: 6789 00000100:00100000:1.0:1776178415.461608:0:6255:0:(service.c:2324:ptlrpc_server_handle_request()) Handling RPC req@000000001c465e98 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_000:0+-99:3352:x1862456998078464:12345-0@lo:38:kworker/u8:1.0 00000100:00100000:3.0:1776178415.461620:0:3352:0:(events.c:361:request_in_callback()) peer: 12345-0@lo (source: 12345-0@lo) 00010000:02000000:1.0:1776178415.461633:0:6255:0:(ldlm_lib.c:1139:target_handle_connect()) lustre-MDT0000: Not available for connect from 0@lo (stopping) 00000100:00100000:3.0:1776178415.461646:0:3352:0:(client.c:1733:ptlrpc_send_new_req()) Sending RPC req@000000006b9d8ccb pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:lustre-MDT0000-lwp-MDT0001_UUID:3352:1862456998078912:0@lo:38:kworker/u8:1.0 00000100:00100000:3.0:1776178415.461696:0:3352:0:(events.c:361:request_in_callback()) peer: 12345-0@lo (source: 12345-0@lo) 00000100:00100000:0.0:1776178415.461774:0:20755:0:(nrs_fifo.c:179:nrs_fifo_req_get()) NRS start fifo request from 12345-0@lo, seq: 677 00000100:00100000:0.0:1776178415.461781:0:20755:0:(service.c:2324:ptlrpc_server_handle_request()) Handling RPC req@0000000098aa1ab6 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_006:lustre-MDT0001-mdtlov_UUID+5:3355:x1862456998078784:12345-0@lo:400:kworker/u8:1.0 00000100:00100000:3.0:1776178415.462528:0:6257:0:(service.c:2152:ptlrpc_server_handle_req_in()) got req x1862456998078912 00000100:00100000:3.0:1776178415.462558:0:6257:0:(nrs_fifo.c:179:nrs_fifo_req_get()) NRS start fifo request from 12345-0@lo, seq: 6790 00000100:00100000:3.0:1776178415.462562:0:6257:0:(service.c:2324:ptlrpc_server_handle_request()) Handling RPC req@0000000007075047 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_002:0+-99:3352:x1862456998078592:12345-0@lo:38:kworker/u8:1.0 00010000:02000000:3.0:1776178415.462578:0:6257:0:(ldlm_lib.c:1139:target_handle_connect()) lustre-MDT0000: Not available for connect from 0@lo (stopping) 00000100:00100000:3.0:1776178415.462636:0:6257:0:(service.c:2372:ptlrpc_server_handle_request()) Handled RPC req@0000000007075047 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_002:0+-99:3352:x1862456998078592:12345-0@lo:38:kworker/u8:1.0 Request processed in 73us (1100us total) trans 0 rc -19/-19 00000100:00100000:3.0:1776178415.462644:0:6257:0:(nrs_fifo.c:241:nrs_fifo_req_stop()) NRS stop fifo request from 12345-0@lo, seq: 6790 00000100:00100000:3.0:1776178415.462663:0:6257:0:(nrs_fifo.c:179:nrs_fifo_req_get()) NRS start fifo request from 12345-0@lo, seq: 6791 00000100:00100000:3.0:1776178415.462666:0:6257:0:(service.c:2324:ptlrpc_server_handle_request()) Handling RPC req@00000000375ba3dd pname:cluuid+ref:pid:xid:nid:opc:job mdt00_002:0+-99:3352:x1862456998078912:12345-0@lo:38:kworker/u8:1.0 00010000:02000000:3.0:1776178415.462683:0:6257:0:(ldlm_lib.c:1139:target_handle_connect()) lustre-MDT0000: Not available for connect from 0@lo (stopping) 00000100:00100000:3.0:1776178415.462708:0:6257:0:(service.c:2372:ptlrpc_server_handle_request()) Handled RPC req@00000000375ba3dd pname:cluuid+ref:pid:xid:nid:opc:job mdt00_002:0+-99:3352:x1862456998078912:12345-0@lo:38:kworker/u8:1.0 Request processed in 39us (1012us total) trans 0 rc -19/-19 00000100:00100000:3.0:1776178415.462714:0:6257:0:(nrs_fifo.c:241:nrs_fifo_req_stop()) NRS stop fifo request from 12345-0@lo, seq: 6791 00000100:00100000:1.0:1776178415.463629:0:6255:0:(service.c:2372:ptlrpc_server_handle_request()) Handled RPC req@000000001c465e98 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_000:0+-99:3352:x1862456998078464:12345-0@lo:38:kworker/u8:1.0 Request processed in 2008us (2586us total) trans 0 rc -19/-19 00000100:00100000:1.0:1776178415.463651:0:6255:0:(nrs_fifo.c:241:nrs_fifo_req_stop()) NRS stop fifo request from 12345-0@lo, seq: 6789 00000100:00100000:1.0:1776178415.463801:0:15213:0:(service.c:2152:ptlrpc_server_handle_req_in()) got req x1862456998078720 00000100:00100000:1.0:1776178415.463854:0:15213:0:(nrs_fifo.c:179:nrs_fifo_req_get()) NRS start fifo request from 12345-0@lo, seq: 749 00000100:00100000:1.0:1776178415.463860:0:15213:0:(service.c:2324:ptlrpc_server_handle_request()) Handling RPC req@00000000f88b2465 pname:cluuid+ref:pid:xid:nid:opc:job mdt_out00_003:0+-99:3352:x1862456998078720:12345-0@lo:38:kworker/u8:1.0 00010000:02000000:1.0:1776178415.463881:0:15213:0:(ldlm_lib.c:1139:target_handle_connect()) lustre-MDT0000: Not available for connect from 0@lo (stopping) 00000100:00100000:1.0:1776178415.463973:0:15213:0:(service.c:2372:ptlrpc_server_handle_request()) Handled RPC req@00000000f88b2465 pname:cluuid+ref:pid:xid:nid:opc:job mdt_out00_003:0+-99:3352:x1862456998078720:12345-0@lo:38:kworker/u8:1.0 Request processed in 112us (2355us total) trans 0 rc -19/-19 00000100:00100000:1.0:1776178415.463979:0:15213:0:(nrs_fifo.c:241:nrs_fifo_req_stop()) NRS stop fifo request from 12345-0@lo, seq: 749 00000100:00080000:1.0:1776178415.464116:0:3352:0:(import.c:86:import_set_state_nolock()) 0000000047601a98 lustre-MDT0000_UUID: changing import state from CONNECTING to DISCONN 00000100:00080000:1.0:1776178415.464121:0:3352:0:(import.c:1429:ptlrpc_connect_interpret()) recovery of lustre-MDT0000_UUID on 192.168.204.103@tcp failed (-19) 00000100:00100000:1.0:1776178415.464127:0:3352:0:(client.c:2215:ptlrpc_check_set()) Completed RPC req@000000007886b426 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:lustre-MDT0000-lwp-OST0000_UUID:3352:1862456998078464:0@lo:38:kworker/u8:1.0 00000100:00080000:1.0:1776178415.464141:0:3352:0:(import.c:86:import_set_state_nolock()) 00000000dfc8db13 lustre-MDT0000_UUID: changing import state from CONNECTING to DISCONN 00000100:00080000:1.0:1776178415.464144:0:3352:0:(import.c:1429:ptlrpc_connect_interpret()) recovery of lustre-MDT0000_UUID on 192.168.204.103@tcp failed (-19) 00000100:00100000:1.0:1776178415.464147:0:3352:0:(client.c:2215:ptlrpc_check_set()) Completed RPC req@000000004ce46e20 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:lustre-MDT0000-lwp-OST0001_UUID:3352:1862456998078592:0@lo:38:kworker/u8:1.0 00000100:00080000:1.0:1776178415.464156:0:3352:0:(import.c:86:import_set_state_nolock()) 00000000ba4f1821 lustre-MDT0000_UUID: changing import state from CONNECTING to DISCONN 00000100:00080000:1.0:1776178415.464159:0:3352:0:(import.c:1429:ptlrpc_connect_interpret()) recovery of lustre-MDT0000_UUID on 192.168.204.103@tcp failed (-19) 00000100:00100000:1.0:1776178415.464161:0:3352:0:(client.c:2215:ptlrpc_check_set()) Completed RPC req@0000000042de09e9 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:lustre-MDT0001-mdtlov_UUID:3352:1862456998078720:0@lo:38:kworker/u8:1.0 00000100:00080000:1.0:1776178415.464172:0:3352:0:(import.c:86:import_set_state_nolock()) 00000000a7c55b1f lustre-MDT0000_UUID: changing import state from CONNECTING to DISCONN 00000100:00080000:1.0:1776178415.464176:0:3352:0:(import.c:1429:ptlrpc_connect_interpret()) recovery of lustre-MDT0000_UUID on 192.168.204.103@tcp failed (-19) 00000100:00100000:1.0:1776178415.464178:0:3352:0:(client.c:2215:ptlrpc_check_set()) Completed RPC req@000000006b9d8ccb pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:lustre-MDT0000-lwp-MDT0001_UUID:3352:1862456998078912:0@lo:38:kworker/u8:1.0 00000100:00100000:2.0:1776178415.466717:0:20759:0:(nrs_fifo.c:179:nrs_fifo_req_get()) NRS start fifo request from 12345-0@lo, seq: 678 00000100:00100000:2.0:1776178415.466741:0:20759:0:(service.c:2324:ptlrpc_server_handle_request()) Handling RPC req@000000008c4bfd71 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_010:lustre-MDT0001-mdtlov_UUID+5:3356:x1862456998078848:12345-0@lo:400:kworker/u8:1.0 00000100:00100000:0.0:1776178415.468907:0:20755:0:(service.c:2372:ptlrpc_server_handle_request()) Handled RPC req@0000000098aa1ab6 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_006:lustre-MDT0001-mdtlov_UUID+5:3355:x1862456998078784:12345-0@lo:400:kworker/u8:1.0 Request processed in 7108us (11289us total) trans 0 rc 0/0 00000100:00100000:3.0:1776178415.468924:0:3355:0:(client.c:2215:ptlrpc_check_set()) Completed RPC req@000000006344f537 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_02:lustre-MDT0001-mdtlov_UUID:3355:1862456998078784:0@lo:400:kworker/u8:1.0 00000100:00100000:0.0:1776178415.468934:0:20755:0:(nrs_fifo.c:241:nrs_fifo_req_stop()) NRS stop fifo request from 12345-0@lo, seq: 677 00000100:00100000:2.0:1776178415.472087:0:20759:0:(service.c:2372:ptlrpc_server_handle_request()) Handled RPC req@000000008c4bfd71 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_010:lustre-MDT0001-mdtlov_UUID+5:3356:x1862456998078848:12345-0@lo:400:kworker/u8:1.0 Request processed in 5313us (14055us total) trans 0 rc 0/0 00000100:00100000:2.0:1776178415.472117:0:20759:0:(nrs_fifo.c:241:nrs_fifo_req_stop()) NRS stop fifo request from 12345-0@lo, seq: 678 00000100:00100000:1.0:1776178415.477136:0:3356:0:(client.c:2215:ptlrpc_check_set()) Completed RPC req@0000000050c5b9a3 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_03:lustre-MDT0001-mdtlov_UUID:3356:1862456998078848:0@lo:400:kworker/u8:1.0 00010000:00010000:2.0:1776178416.097187:0:9224:0:(ldlm_lockd.c:2792:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:3.0:1776178416.992661:0:3354:0:(ptlrpcd.c:415:ptlrpcd_check()) transfer 1 async RPCs [0->1] 00000100:00100000:3.0:1776178416.992710:0:3354:0:(client.c:1733:ptlrpc_send_new_req()) Sending RPC req@00000000ddf29755 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_01:lustre-MDT0001-mdtlov_UUID:3354:1862456998078976:0@lo:13:osp-pre-1-1.0 00000100:00100000:3.0:1776178416.992822:0:3354:0:(events.c:361:request_in_callback()) peer: 12345-0@lo (source: 12345-0@lo) 00000100:00100000:2.0:1776178416.993304:0:9554:0:(service.c:2152:ptlrpc_server_handle_req_in()) got req x1862456998078976 00000100:00100000:2.0:1776178416.993409:0:9554:0:(nrs_fifo.c:179:nrs_fifo_req_get()) NRS start fifo request from 12345-0@lo, seq: 571 00000100:00100000:2.0:1776178416.993417:0:9554:0:(service.c:2324:ptlrpc_server_handle_request()) Handling RPC req@0000000013f6d767 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_create00:lustre-MDT0001-mdtlov_UUID+5:3354:x1862456998078976:12345-0@lo:13:osp-pre-1-1.0 00002000:00000024:2.0:1776178416.993491:0:9554:0:(ofd_obd.c:740:ofd_statfs()) blocks cached 229376 granted 491746368 pending 0 free 3923689472 avail 3692167168 00000100:00100000:2.0:1776178416.993572:0:9554:0:(service.c:2372:ptlrpc_server_handle_request()) Handled RPC req@0000000013f6d767 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_create00:lustre-MDT0001-mdtlov_UUID+5:3354:x1862456998078976:12345-0@lo:13:osp-pre-1-1.0 Request processed in 154us (751us total) trans 0 rc 0/0 00000100:00100000:2.0:1776178416.993583:0:9554:0:(nrs_fifo.c:241:nrs_fifo_req_stop()) NRS stop fifo request from 12345-0@lo, seq: 571 00000100:00100000:3.0:1776178416.995057:0:3354:0:(client.c:2215:ptlrpc_check_set()) Completed RPC req@00000000ddf29755 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_01:lustre-MDT0001-mdtlov_UUID:3354:1862456998078976:0@lo:13:osp-pre-1-1.0 00010000:00010000:1.0:1776178417.120528:0:9225:0:(ldlm_lockd.c:2792:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:3.0:1776178417.505406:0:3354:0:(client.c:1733:ptlrpc_send_new_req()) Sending RPC req@0000000047a42b33 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_01:lustre-MDT0001-mdtlov_UUID:3354:1862456998079040:0@lo:13:osp-pre-0-1.0 00000100:00100000:3.0:1776178417.505525:0:3354:0:(events.c:361:request_in_callback()) peer: 12345-0@lo (source: 12345-0@lo) 00000100:00100000:2.0:1776178417.506057:0:8231:0:(service.c:2152:ptlrpc_server_handle_req_in()) got req x1862456998079040 00000100:00100000:2.0:1776178417.506171:0:8231:0:(nrs_fifo.c:179:nrs_fifo_req_get()) NRS start fifo request from 12345-0@lo, seq: 572 00000100:00100000:2.0:1776178417.506179:0:8231:0:(service.c:2324:ptlrpc_server_handle_request()) Handling RPC req@0000000052775143 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_create00:lustre-MDT0001-mdtlov_UUID+5:3354:x1862456998079040:12345-0@lo:13:osp-pre-0-1.0 00002000:00000024:2.0:1776178417.506225:0:8231:0:(ofd_obd.c:740:ofd_statfs()) blocks cached 315392 granted 484883776 pending 0 free 3923689472 avail 3692167168 00000100:00100000:2.0:1776178417.506354:0:8231:0:(service.c:2372:ptlrpc_server_handle_request()) Handled RPC req@0000000052775143 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_create00:lustre-MDT0001-mdtlov_UUID+5:3354:x1862456998079040:12345-0@lo:13:osp-pre-0-1.0 Request processed in 175us (830us total) trans 0 rc 0/0 00000100:00100000:2.0:1776178417.506365:0:8231:0:(nrs_fifo.c:241:nrs_fifo_req_stop()) NRS stop fifo request from 12345-0@lo, seq: 572 00000100:00100000:3.0:1776178417.508762:0:3354:0:(client.c:2215:ptlrpc_check_set()) Completed RPC req@0000000047a42b33 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_01:lustre-MDT0001-mdtlov_UUID:3354:1862456998079040:0@lo:13:osp-pre-0-1.0 00010000:00010000:3.0:1776178418.144730:0:6244:0:(ldlm_lockd.c:2792:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:2.0:1776178418.997944:0:3347:0:(events.c:361:request_in_callback()) peer: 12345-192.168.204.3@tcp (source: 12345-192.168.204.3@tcp) 00000100:00100000:1.0:1776178418.998248:0:20140:0:(service.c:2152:ptlrpc_server_handle_req_in()) got req x1862456979122432 00000100:00100000:1.0:1776178418.998312:0:20140:0:(nrs_fifo.c:179:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.204.3@tcp, seq: 239 00000100:00100000:1.0:1776178418.998320:0:20140:0:(service.c:2324:ptlrpc_server_handle_request()) Handling RPC req@00000000c201d06e pname:cluuid+ref:pid:xid:nid:opc:job ll_mgs_0002:5b5c66b1-1a2f-4651-8548-0c75b9a38a94+9:2272:x1862456979122432:12345-192.168.204.3@tcp:400:kworker/u8:4.0 00000100:00100000:1.0:1776178418.998409:0:20140:0:(service.c:2372:ptlrpc_server_handle_request()) Handled RPC req@00000000c201d06e pname:cluuid+ref:pid:xid:nid:opc:job ll_mgs_0002:5b5c66b1-1a2f-4651-8548-0c75b9a38a94+9:2272:x1862456979122432:12345-192.168.204.3@tcp:400:kworker/u8:4.0 Request processed in 89us (475us total) trans 0 rc 0/0 00000100:00100000:1.0:1776178418.998417:0:20140:0:(nrs_fifo.c:241:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.204.3@tcp, seq: 239 00000100:00100000:2.0:1776178418.998668:0:3347:0:(events.c:361:request_in_callback()) peer: 12345-192.168.204.3@tcp (source: 12345-192.168.204.3@tcp) 00000100:00100000:2.0:1776178418.998866:0:3347:0:(events.c:361:request_in_callback()) peer: 12345-192.168.204.3@tcp (source: 12345-192.168.204.3@tcp) 00000100:00100000:2.0:1776178418.998904:0:3347:0:(events.c:361:request_in_callback()) peer: 12345-192.168.204.3@tcp (source: 12345-192.168.204.3@tcp) 00000100:00100000:2.0:1776178418.998933:0:3347:0:(events.c:361:request_in_callback()) peer: 12345-192.168.204.3@tcp (source: 12345-192.168.204.3@tcp) 00000100:00100000:2.0:1776178418.998963:0:20759:0:(service.c:2152:ptlrpc_server_handle_req_in()) got req x1862456979122688 00000100:00100000:1.0:1776178418.999197:0:6256:0:(service.c:2152:ptlrpc_server_handle_req_in()) got req x1862456979122560 00000100:00100000:1.0:1776178418.999233:0:6256:0:(service.c:2152:ptlrpc_server_handle_req_in()) got req x1862456979122496 00000100:00100000:1.0:1776178418.999250:0:6256:0:(nrs_fifo.c:179:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.204.3@tcp, seq: 1054 00000100:00100000:1.0:1776178418.999255:0:6256:0:(service.c:2324:ptlrpc_server_handle_request()) Handling RPC req@000000005f3eedd2 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_001:b50963bd-8eba-4432-9a54-8888d00e3fa1+6:2273:x1862456979122560:12345-192.168.204.3@tcp:400:kworker/u8:4.0 00000100:00100000:1.0:1776178418.999301:0:6256:0:(service.c:2372:ptlrpc_server_handle_request()) Handled RPC req@000000005f3eedd2 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_001:b50963bd-8eba-4432-9a54-8888d00e3fa1+6:2273:x1862456979122560:12345-192.168.204.3@tcp:400:kworker/u8:4.0 Request processed in 46us (634us total) trans 0 rc 0/0 00000100:00100000:1.0:1776178418.999308:0:6256:0:(nrs_fifo.c:241:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.204.3@tcp, seq: 1054 00000100:00100000:1.0:1776178418.999321:0:6256:0:(nrs_fifo.c:179:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.204.3@tcp, seq: 6792 00000100:00100000:1.0:1776178418.999325:0:6256:0:(service.c:2324:ptlrpc_server_handle_request()) Handling RPC req@000000000c9fb390 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_001:0+-99:2269:x1862456979122496:12345-192.168.204.3@tcp:38:kworker/u8:4.0 00010000:02000000:1.0:1776178418.999342:0:6256:0:(ldlm_lib.c:1139:target_handle_connect()) lustre-MDT0000: Not available for connect from 192.168.204.3@tcp (stopping) 00000100:00100000:1.0:1776178418.999369:0:6256:0:(service.c:2372:ptlrpc_server_handle_request()) Handled RPC req@000000000c9fb390 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_001:0+-99:2269:x1862456979122496:12345-192.168.204.3@tcp:38:kworker/u8:4.0 Request processed in 43us (504us total) trans 0 rc -19/-19 00000100:00100000:1.0:1776178418.999375:0:6256:0:(nrs_fifo.c:241:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.204.3@tcp, seq: 6792 00000100:00100000:2.0:1776178418.999846:0:20759:0:(service.c:2152:ptlrpc_server_handle_req_in()) got req x1862456979122624 00000100:00100000:2.0:1776178418.999866:0:20759:0:(nrs_fifo.c:179:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.204.3@tcp, seq: 679 00000100:00100000:2.0:1776178418.999871:0:20759:0:(service.c:2324:ptlrpc_server_handle_request()) Handling RPC req@0000000099e8e662 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_010:b50963bd-8eba-4432-9a54-8888d00e3fa1+9:2273:x1862456979122688:12345-192.168.204.3@tcp:400:kworker/u8:4.0 00000100:00100000:0.0:1776178419.000290:0:20755:0:(nrs_fifo.c:179:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.204.3@tcp, seq: 680 00000100:00100000:0.0:1776178419.000296:0:20755:0:(service.c:2324:ptlrpc_server_handle_request()) Handling RPC req@00000000a49c1305 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_006:b50963bd-8eba-4432-9a54-8888d00e3fa1+8:2272:x1862456979122624:12345-192.168.204.3@tcp:400:kworker/u8:4.0 00000100:00100000:0.0:1776178419.000352:0:20755:0:(service.c:2372:ptlrpc_server_handle_request()) Handled RPC req@00000000a49c1305 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_006:b50963bd-8eba-4432-9a54-8888d00e3fa1+8:2272:x1862456979122624:12345-192.168.204.3@tcp:400:kworker/u8:4.0 Request processed in 56us (1419us total) trans 0 rc 0/0 00000100:00100000:0.0:1776178419.000359:0:20755:0:(nrs_fifo.c:241:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.204.3@tcp, seq: 680 00000100:00100000:2.0:1776178419.000427:0:20759:0:(service.c:2372:ptlrpc_server_handle_request()) Handled RPC req@0000000099e8e662 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_010:b50963bd-8eba-4432-9a54-8888d00e3fa1+9:2273:x1862456979122688:12345-192.168.204.3@tcp:400:kworker/u8:4.0 Request processed in 554us (1527us total) trans 0 rc 0/0 00000100:00100000:2.0:1776178419.000434:0:20759:0:(nrs_fifo.c:241:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.204.3@tcp, seq: 679 00010000:00010000:2.0:1776178419.169545:0:6245:0:(ldlm_lockd.c:2792:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00010000:00010000:2.0:1776178420.194791:0:9224:0:(ldlm_lockd.c:2792:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00080000:2.0:1776178420.579238:0:36:0:(pinger.c:251:ptlrpc_pinger_process_import()) lustre-MDT0000-lwp-OST0000_UUID->lustre-MDT0000_UUID: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00080000:2.0:1776178420.579247:0:36:0:(recover.c:57:ptlrpc_initiate_recovery()) lustre-MDT0000_UUID: starting recovery 00000100:00080000:2.0:1776178420.579251:0:36:0:(import.c:86:import_set_state_nolock()) 0000000047601a98 lustre-MDT0000_UUID: changing import state from DISCONN to CONNECTING 00000100:00080000:2.0:1776178420.579261:0:36:0:(import.c:534:import_select_connection()) lustre-MDT0000-lwp-OST0000: connect to NID 0@lo last attempt 1122 00000100:00080000:2.0:1776178420.579265:0:36:0:(import.c:581:import_select_connection()) lustre-MDT0000-lwp-OST0000: tried all connections, increasing latency to 21s 00000100:00080000:2.0:1776178420.579273:0:36:0:(import.c:615:import_select_connection()) lustre-MDT0000-lwp-OST0000: import 0000000047601a98 using connection 192.168.204.103@tcp/0@lo 00000100:00100000:2.0:1776178420.579297:0:36:0:(import.c:807:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5) req@0000000050c5b9a3 x1862456998079168/t0(0) o38->lustre-MDT0000-lwp-OST0000@0@lo:12/10 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' 00000100:00080000:2.0:1776178420.579319:0:36:0:(pinger.c:251:ptlrpc_pinger_process_import()) lustre-MDT0000-lwp-OST0001_UUID->lustre-MDT0000_UUID: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00080000:2.0:1776178420.579324:0:36:0:(recover.c:57:ptlrpc_initiate_recovery()) lustre-MDT0000_UUID: starting recovery 00000100:00080000:2.0:1776178420.579327:0:36:0:(import.c:86:import_set_state_nolock()) 00000000dfc8db13 lustre-MDT0000_UUID: changing import state from DISCONN to CONNECTING 00000100:00080000:2.0:1776178420.579331:0:36:0:(import.c:534:import_select_connection()) lustre-MDT0000-lwp-OST0001: connect to NID 0@lo last attempt 1122 00000100:00080000:2.0:1776178420.579333:0:36:0:(import.c:581:import_select_connection()) lustre-MDT0000-lwp-OST0001: tried all connections, increasing latency to 21s 00000100:00080000:2.0:1776178420.579336:0:36:0:(import.c:615:import_select_connection()) lustre-MDT0000-lwp-OST0001: import 00000000dfc8db13 using connection 192.168.204.103@tcp/0@lo 00000100:00100000:2.0:1776178420.579346:0:36:0:(import.c:807:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5) req@000000006b9d8ccb x1862456998079296/t0(0) o38->lustre-MDT0000-lwp-OST0001@0@lo:12/10 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' 00000100:00080000:2.0:1776178420.579376:0:36:0:(pinger.c:251:ptlrpc_pinger_process_import()) lustre-MDT0001-mdtlov_UUID->lustre-MDT0000_UUID: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00080000:2.0:1776178420.579379:0:36:0:(recover.c:57:ptlrpc_initiate_recovery()) lustre-MDT0000_UUID: starting recovery 00000100:00080000:2.0:1776178420.579381:0:36:0:(import.c:86:import_set_state_nolock()) 00000000ba4f1821 lustre-MDT0000_UUID: changing import state from DISCONN to CONNECTING 00000100:00080000:2.0:1776178420.579383:0:36:0:(import.c:534:import_select_connection()) lustre-MDT0000-osp-MDT0001: connect to NID 0@lo last attempt 1122 00000100:00080000:2.0:1776178420.579385:0:36:0:(import.c:581:import_select_connection()) lustre-MDT0000-osp-MDT0001: tried all connections, increasing latency to 21s 00000100:00080000:2.0:1776178420.579389:0:36:0:(import.c:615:import_select_connection()) lustre-MDT0000-osp-MDT0001: import 00000000ba4f1821 using connection 192.168.204.103@tcp/0@lo 00000100:00100000:2.0:1776178420.579401:0:36:0:(import.c:807:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5) req@000000007309261c x1862456998079424/t0(0) o38->lustre-MDT0000-osp-MDT0001@0@lo:24/4 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' 00000100:00080000:2.0:1776178420.579422:0:36:0:(pinger.c:251:ptlrpc_pinger_process_import()) lustre-MDT0000-lwp-MDT0001_UUID->lustre-MDT0000_UUID: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00080000:2.0:1776178420.579425:0:36:0:(recover.c:57:ptlrpc_initiate_recovery()) lustre-MDT0000_UUID: starting recovery 00000100:00080000:2.0:1776178420.579427:0:36:0:(import.c:86:import_set_state_nolock()) 00000000a7c55b1f lustre-MDT0000_UUID: changing import state from DISCONN to CONNECTING 00000100:00080000:2.0:1776178420.579430:0:36:0:(import.c:534:import_select_connection()) lustre-MDT0000-lwp-MDT0001: connect to NID 0@lo last attempt 1122 00000100:00080000:2.0:1776178420.579432:0:36:0:(import.c:581:import_select_connection()) lustre-MDT0000-lwp-MDT0001: tried all connections, increasing latency to 21s 00000100:00080000:2.0:1776178420.579435:0:36:0:(import.c:615:import_select_connection()) lustre-MDT0000-lwp-MDT0001: import 00000000a7c55b1f using connection 192.168.204.103@tcp/0@lo 00000100:00100000:2.0:1776178420.579445:0:36:0:(import.c:807:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5) req@000000006e2391ef x1862456998079616/t0(0) o38->lustre-MDT0000-lwp-MDT0001@0@lo:12/10 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' 00000100:00100000:2.0:1776178420.579477:0:3353:0:(client.c:1733:ptlrpc_send_new_req()) Sending RPC req@000000008b978a8a pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_00:lustre-MDT0001-lwp-OST0001_UUID:3353:1862456998079360:0@lo:400:kworker/u8:1.0 00000100:00100000:2.0:1776178420.579549:0:3353:0:(events.c:361:request_in_callback()) peer: 12345-0@lo (source: 12345-0@lo) 00000100:00100000:2.0:1776178420.579592:0:3353:0:(ptlrpcd.c:415:ptlrpcd_check()) transfer 1 async RPCs [1->0] 00000100:00100000:2.0:1776178420.579598:0:3353:0:(client.c:1733:ptlrpc_send_new_req()) Sending RPC req@000000001ee743a7 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_00:lustre-MDT0001-mdtlov_UUID:3353:1862456998079488:0@lo:400:kworker/u8:1.0 00000100:00100000:2.0:1776178420.579628:0:3353:0:(events.c:361:request_in_callback()) peer: 12345-0@lo (source: 12345-0@lo) 00000100:00100000:1.0:1776178420.579632:0:3356:0:(client.c:1733:ptlrpc_send_new_req()) Sending RPC req@000000006344f537 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_03:lustre-MDT0001-lwp-OST0000_UUID:3356:1862456998079232:0@lo:400:kworker/u8:1.0 00000100:00100000:2.0:1776178420.579668:0:14440:0:(service.c:2152:ptlrpc_server_handle_req_in()) got req x1862456998079488 00000100:00100000:1.0:1776178420.579673:0:3356:0:(events.c:361:request_in_callback()) peer: 12345-0@lo (source: 12345-0@lo) 00000100:00100000:1.0:1776178420.579697:0:3356:0:(ptlrpcd.c:415:ptlrpcd_check()) transfer 2 async RPCs [2->3] 00000100:00100000:1.0:1776178420.579714:0:3356:0:(client.c:1733:ptlrpc_send_new_req()) Sending RPC req@00000000ddf29755 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_03:ac61f839-0e8e-4cac-9c59-08bcfbbad330:3356:1862456998079104:0@lo:400:kworker/u8:1.0 00000100:00100000:2.0:1776178420.579728:0:14440:0:(nrs_fifo.c:179:nrs_fifo_req_get()) NRS start fifo request from 12345-0@lo, seq: 681 00000100:00100000:2.0:1776178420.579736:0:14440:0:(service.c:2324:ptlrpc_server_handle_request()) Handling RPC req@00000000a13f5de6 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_004:lustre-MDT0001-mdtlov_UUID+5:3353:x1862456998079488:12345-0@lo:400:kworker/u8:1.0 00000100:00100000:1.0:1776178420.579744:0:3356:0:(events.c:361:request_in_callback()) peer: 12345-0@lo (source: 12345-0@lo) 00000100:00100000:1.0:1776178420.579768:0:3356:0:(client.c:1733:ptlrpc_send_new_req()) Sending RPC req@00000000aba13576 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_03:lustre-MDT0001-mdtlov_UUID:3356:1862456998079552:0@lo:400:kworker/u8:1.0 00000100:00100000:1.0:1776178420.579786:0:3356:0:(events.c:361:request_in_callback()) peer: 12345-0@lo (source: 12345-0@lo) 00000100:00100000:2.0:1776178420.579791:0:14440:0:(service.c:2372:ptlrpc_server_handle_request()) Handled RPC req@00000000a13f5de6 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_004:lustre-MDT0001-mdtlov_UUID+5:3353:x1862456998079488:12345-0@lo:400:kworker/u8:1.0 Request processed in 55us (163us total) trans 0 rc 0/0 00000100:00100000:2.0:1776178420.579798:0:14440:0:(nrs_fifo.c:241:nrs_fifo_req_stop()) NRS stop fifo request from 12345-0@lo, seq: 681 00000100:00100000:1.0:1776178420.579808:0:3352:0:(client.c:1733:ptlrpc_send_new_req()) Sending RPC req@0000000050c5b9a3 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:lustre-MDT0000-lwp-OST0000_UUID:3352:1862456998079168:0@lo:38:kworker/u8:1.0 00000100:00100000:2.0:1776178420.579814:0:14440:0:(service.c:2152:ptlrpc_server_handle_req_in()) got req x1862456998079552 00000100:00100000:1.0:1776178420.579829:0:3352:0:(events.c:361:request_in_callback()) peer: 12345-0@lo (source: 12345-0@lo) 00000100:00100000:2.0:1776178420.579833:0:14440:0:(nrs_fifo.c:179:nrs_fifo_req_get()) NRS start fifo request from 12345-0@lo, seq: 682 00000100:00100000:2.0:1776178420.579836:0:14440:0:(service.c:2324:ptlrpc_server_handle_request()) Handling RPC req@0000000053e142bc pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_004:lustre-MDT0001-mdtlov_UUID+5:3356:x1862456998079552:12345-0@lo:400:kworker/u8:1.0 00000100:00100000:1.0:1776178420.579841:0:3352:0:(client.c:1733:ptlrpc_send_new_req()) Sending RPC req@000000006b9d8ccb pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:lustre-MDT0000-lwp-OST0001_UUID:3352:1862456998079296:0@lo:38:kworker/u8:1.0 00000100:00100000:1.0:1776178420.579857:0:3352:0:(events.c:361:request_in_callback()) peer: 12345-0@lo (source: 12345-0@lo) 00000100:00100000:2.0:1776178420.579863:0:14440:0:(service.c:2372:ptlrpc_server_handle_request()) Handled RPC req@0000000053e142bc pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_004:lustre-MDT0001-mdtlov_UUID+5:3356:x1862456998079552:12345-0@lo:400:kworker/u8:1.0 Request processed in 26us (77us total) trans 0 rc 0/0 00000100:00100000:1.0:1776178420.579867:0:3352:0:(client.c:1733:ptlrpc_send_new_req()) Sending RPC req@000000007309261c pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:lustre-MDT0001-mdtlov_UUID:3352:1862456998079424:0@lo:38:kworker/u8:1.0 00000100:00100000:2.0:1776178420.579869:0:14440:0:(nrs_fifo.c:241:nrs_fifo_req_stop()) NRS stop fifo request from 12345-0@lo, seq: 682 00000100:00100000:1.0:1776178420.579887:0:3352:0:(events.c:361:request_in_callback()) peer: 12345-0@lo (source: 12345-0@lo) 00000100:00100000:1.0:1776178420.579899:0:3352:0:(client.c:1733:ptlrpc_send_new_req()) Sending RPC req@000000006e2391ef pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:lustre-MDT0000-lwp-MDT0001_UUID:3352:1862456998079616:0@lo:38:kworker/u8:1.0 00000100:00100000:2.0:1776178420.579906:0:3353:0:(client.c:2215:ptlrpc_check_set()) Completed RPC req@000000001ee743a7 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_00:lustre-MDT0001-mdtlov_UUID:3353:1862456998079488:0@lo:400:kworker/u8:1.0 00000100:00100000:1.0:1776178420.579915:0:3352:0:(events.c:361:request_in_callback()) peer: 12345-0@lo (source: 12345-0@lo) 00000100:00100000:1.0:1776178420.579940:0:11281:0:(service.c:2152:ptlrpc_server_handle_req_in()) got req x1862456998079360 00000100:00100000:2.0:1776178420.579956:0:6256:0:(service.c:2152:ptlrpc_server_handle_req_in()) got req x1862456998079232 00000100:00100000:1.0:1776178420.579960:0:11281:0:(nrs_fifo.c:179:nrs_fifo_req_get()) NRS start fifo request from 12345-0@lo, seq: 1055 00000100:00100000:1.0:1776178420.579964:0:11281:0:(service.c:2324:ptlrpc_server_handle_request()) Handling RPC req@00000000d94d52a5 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_004:lustre-MDT0001-lwp-OST0001_UUID+5:3353:x1862456998079360:12345-0@lo:400:kworker/u8:1.0 00000100:00100000:2.0:1776178420.579968:0:6256:0:(service.c:2152:ptlrpc_server_handle_req_in()) got req x1862456998079168 00000100:00100000:2.0:1776178420.579975:0:6256:0:(service.c:2152:ptlrpc_server_handle_req_in()) got req x1862456998079296 00000100:00100000:2.0:1776178420.579982:0:6256:0:(service.c:2152:ptlrpc_server_handle_req_in()) got req x1862456998079616 00000100:00100000:2.0:1776178420.579991:0:6256:0:(nrs_fifo.c:179:nrs_fifo_req_get()) NRS start fifo request from 12345-0@lo, seq: 1056 00000100:00100000:2.0:1776178420.579994:0:6256:0:(service.c:2324:ptlrpc_server_handle_request()) Handling RPC req@00000000ae838296 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_001:lustre-MDT0001-lwp-OST0000_UUID+5:3356:x1862456998079232:12345-0@lo:400:kworker/u8:1.0 00000100:00100000:1.0:1776178420.579998:0:11281:0:(service.c:2372:ptlrpc_server_handle_request()) Handled RPC req@00000000d94d52a5 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_004:lustre-MDT0001-lwp-OST0001_UUID+5:3353:x1862456998079360:12345-0@lo:400:kworker/u8:1.0 Request processed in 33us (452us total) trans 0 rc 0/0 00000100:00100000:1.0:1776178420.580003:0:11281:0:(nrs_fifo.c:241:nrs_fifo_req_stop()) NRS stop fifo request from 12345-0@lo, seq: 1055 00000100:00100000:1.0:1776178420.580016:0:11281:0:(nrs_fifo.c:179:nrs_fifo_req_get()) NRS start fifo request from 12345-0@lo, seq: 6793 00000100:00100000:2.0:1776178420.580019:0:6256:0:(service.c:2372:ptlrpc_server_handle_request()) Handled RPC req@00000000ae838296 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_001:lustre-MDT0001-lwp-OST0000_UUID+5:3356:x1862456998079232:12345-0@lo:400:kworker/u8:1.0 Request processed in 24us (346us total) trans 0 rc 0/0 00000100:00100000:1.0:1776178420.580020:0:11281:0:(service.c:2324:ptlrpc_server_handle_request()) Handling RPC req@00000000b821049d pname:cluuid+ref:pid:xid:nid:opc:job mdt00_004:0+-99:3352:x1862456998079168:12345-0@lo:38:kworker/u8:1.0 00000100:00100000:2.0:1776178420.580024:0:6256:0:(nrs_fifo.c:241:nrs_fifo_req_stop()) NRS stop fifo request from 12345-0@lo, seq: 1056 00010000:02000000:1.0:1776178420.580034:0:11281:0:(ldlm_lib.c:1139:target_handle_connect()) lustre-MDT0000: Not available for connect from 0@lo (stopping) 00000100:00100000:2.0:1776178420.580034:0:6256:0:(nrs_fifo.c:179:nrs_fifo_req_get()) NRS start fifo request from 12345-0@lo, seq: 6794 00000100:00100000:2.0:1776178420.580036:0:6256:0:(service.c:2324:ptlrpc_server_handle_request()) Handling RPC req@000000005b125149 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_001:0+-99:3352:x1862456998079296:12345-0@lo:38:kworker/u8:1.0 00010000:02000000:2.0:1776178420.580041:0:6256:0:(ldlm_lib.c:1139:target_handle_connect()) lustre-MDT0000: Not available for connect from 0@lo (stopping) 00000100:00100000:3.0:1776178420.580244:0:20137:0:(service.c:2152:ptlrpc_server_handle_req_in()) got req x1862456998079104 00000100:00100000:3.0:1776178420.580275:0:20137:0:(nrs_fifo.c:179:nrs_fifo_req_get()) NRS start fifo request from 12345-0@lo, seq: 240 00000100:00100000:3.0:1776178420.580279:0:20137:0:(service.c:2324:ptlrpc_server_handle_request()) Handling RPC req@000000009c2c404e pname:cluuid+ref:pid:xid:nid:opc:job ll_mgs_0000:ac61f839-0e8e-4cac-9c59-08bcfbbad330+18:3356:x1862456998079104:12345-0@lo:400:kworker/u8:1.0 00000100:00100000:0.0:1776178420.580927:0:3356:0:(client.c:2215:ptlrpc_check_set()) Completed RPC req@00000000aba13576 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_03:lustre-MDT0001-mdtlov_UUID:3356:1862456998079552:0@lo:400:kworker/u8:1.0 00000100:00100000:0.0:1776178420.580944:0:3356:0:(client.c:2215:ptlrpc_check_set()) Completed RPC req@000000006344f537 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_03:lustre-MDT0001-lwp-OST0000_UUID:3356:1862456998079232:0@lo:400:kworker/u8:1.0 00000100:00100000:0.0:1776178420.580978:0:6255:0:(nrs_fifo.c:179:nrs_fifo_req_get()) NRS start fifo request from 12345-0@lo, seq: 6795 00000100:00100000:0.0:1776178420.580982:0:6255:0:(service.c:2324:ptlrpc_server_handle_request()) Handling RPC req@00000000a23c68f9 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_000:0+-99:3352:x1862456998079616:12345-0@lo:38:kworker/u8:1.0 00010000:02000000:0.0:1776178420.580992:0:6255:0:(ldlm_lib.c:1139:target_handle_connect()) lustre-MDT0000: Not available for connect from 0@lo (stopping) 00000100:00100000:3.0:1776178420.589585:0:20137:0:(service.c:2372:ptlrpc_server_handle_request()) Handled RPC req@000000009c2c404e pname:cluuid+ref:pid:xid:nid:opc:job ll_mgs_0000:ac61f839-0e8e-4cac-9c59-08bcfbbad330+18:3356:x1862456998079104:12345-0@lo:400:kworker/u8:1.0 Request processed in 9281us (9816us total) trans 0 rc 0/0 00000100:00100000:3.0:1776178420.589605:0:20137:0:(nrs_fifo.c:241:nrs_fifo_req_stop()) NRS stop fifo request from 12345-0@lo, seq: 240 00000100:00100000:3.0:1776178420.590304:0:3356:0:(client.c:2215:ptlrpc_check_set()) Completed RPC req@00000000ddf29755 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_03:ac61f839-0e8e-4cac-9c59-08bcfbbad330:3356:1862456998079104:0@lo:400:kworker/u8:1.0 00000100:00080000:3.0:1776178420.590547:0:3352:0:(import.c:86:import_set_state_nolock()) 0000000047601a98 lustre-MDT0000_UUID: changing import state from CONNECTING to DISCONN 00000100:00080000:3.0:1776178420.590553:0:3352:0:(import.c:1429:ptlrpc_connect_interpret()) recovery of lustre-MDT0000_UUID on 192.168.204.103@tcp failed (-19) 00000100:00100000:3.0:1776178420.590557:0:3352:0:(client.c:2215:ptlrpc_check_set()) Completed RPC req@0000000050c5b9a3 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:lustre-MDT0000-lwp-OST0000_UUID:3352:1862456998079168:0@lo:38:kworker/u8:1.0 00000100:00080000:3.0:1776178420.590568:0:3352:0:(import.c:86:import_set_state_nolock()) 00000000dfc8db13 lustre-MDT0000_UUID: changing import state from CONNECTING to DISCONN 00000100:00080000:3.0:1776178420.591650:0:3352:0:(import.c:1429:ptlrpc_connect_interpret()) recovery of lustre-MDT0000_UUID on 192.168.204.103@tcp failed (-19) 00000100:00100000:3.0:1776178420.591661:0:3352:0:(client.c:2215:ptlrpc_check_set()) Completed RPC req@000000006b9d8ccb pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:lustre-MDT0000-lwp-OST0001_UUID:3352:1862456998079296:0@lo:38:kworker/u8:1.0 00000100:00100000:3.0:1776178420.591797:0:3353:0:(client.c:2215:ptlrpc_check_set()) Completed RPC req@000000008b978a8a pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_00:lustre-MDT0001-lwp-OST0001_UUID:3353:1862456998079360:0@lo:400:kworker/u8:1.0 00000100:00100000:3.0:1776178420.592188:0:14128:0:(service.c:2152:ptlrpc_server_handle_req_in()) got req x1862456998079424 00000100:00100000:3.0:1776178420.592248:0:14128:0:(nrs_fifo.c:179:nrs_fifo_req_get()) NRS start fifo request from 12345-0@lo, seq: 750 00000100:00100000:3.0:1776178420.592256:0:14128:0:(service.c:2324:ptlrpc_server_handle_request()) Handling RPC req@0000000078c1d864 pname:cluuid+ref:pid:xid:nid:opc:job mdt_out00_002:0+-99:3352:x1862456998079424:12345-0@lo:38:kworker/u8:1.0 00010000:02000000:3.0:1776178420.592275:0:14128:0:(ldlm_lib.c:1139:target_handle_connect()) lustre-MDT0000: Not available for connect from 0@lo (stopping) 00000100:00100000:3.0:1776178420.592367:0:14128:0:(service.c:2372:ptlrpc_server_handle_request()) Handled RPC req@0000000078c1d864 pname:cluuid+ref:pid:xid:nid:opc:job mdt_out00_002:0+-99:3352:x1862456998079424:12345-0@lo:38:kworker/u8:1.0 Request processed in 110us (12478us total) trans 0 rc -19/-19 00000100:00100000:3.0:1776178420.592376:0:14128:0:(nrs_fifo.c:241:nrs_fifo_req_stop()) NRS stop fifo request from 12345-0@lo, seq: 750 00000100:00080000:3.0:1776178420.592414:0:3352:0:(import.c:86:import_set_state_nolock()) 00000000ba4f1821 lustre-MDT0000_UUID: changing import state from CONNECTING to DISCONN 00000100:00080000:3.0:1776178420.592419:0:3352:0:(import.c:1429:ptlrpc_connect_interpret()) recovery of lustre-MDT0000_UUID on 192.168.204.103@tcp failed (-19) 00000100:00100000:3.0:1776178420.592422:0:3352:0:(client.c:2215:ptlrpc_check_set()) Completed RPC req@000000007309261c pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:lustre-MDT0001-mdtlov_UUID:3352:1862456998079424:0@lo:38:kworker/u8:1.0 00000100:00100000:2.0:1776178420.593046:0:6256:0:(service.c:2372:ptlrpc_server_handle_request()) Handled RPC req@000000005b125149 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_001:0+-99:3352:x1862456998079296:12345-0@lo:38:kworker/u8:1.0 Request processed in 13006us (13185us total) trans 0 rc -19/-19 00000100:00100000:2.0:1776178420.593056:0:6256:0:(nrs_fifo.c:241:nrs_fifo_req_stop()) NRS stop fifo request from 12345-0@lo, seq: 6794 00000100:00100000:1.0:1776178420.595898:0:11281:0:(service.c:2372:ptlrpc_server_handle_request()) Handled RPC req@00000000b821049d pname:cluuid+ref:pid:xid:nid:opc:job mdt00_004:0+-99:3352:x1862456998079168:12345-0@lo:38:kworker/u8:1.0 Request processed in 15859us (16051us total) trans 0 rc -19/-19 00000100:00100000:1.0:1776178420.595929:0:11281:0:(nrs_fifo.c:241:nrs_fifo_req_stop()) NRS stop fifo request from 12345-0@lo, seq: 6793 00000100:00100000:0.0:1776178420.596590:0:6255:0:(service.c:2372:ptlrpc_server_handle_request()) Handled RPC req@00000000a23c68f9 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_000:0+-99:3352:x1862456998079616:12345-0@lo:38:kworker/u8:1.0 Request processed in 15604us (16671us total) trans 0 rc -19/-19 00000100:00100000:0.0:1776178420.596599:0:6255:0:(nrs_fifo.c:241:nrs_fifo_req_stop()) NRS stop fifo request from 12345-0@lo, seq: 6795 00000100:00080000:3.0:1776178420.596745:0:3352:0:(import.c:86:import_set_state_nolock()) 00000000a7c55b1f lustre-MDT0000_UUID: changing import state from CONNECTING to DISCONN 00000100:00080000:3.0:1776178420.596751:0:3352:0:(import.c:1429:ptlrpc_connect_interpret()) recovery of lustre-MDT0000_UUID on 192.168.204.103@tcp failed (-19) 00000100:00100000:3.0:1776178420.596756:0:3352:0:(client.c:2215:ptlrpc_check_set()) Completed RPC req@000000006e2391ef pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:lustre-MDT0000-lwp-MDT0001_UUID:3352:1862456998079616:0@lo:38:kworker/u8:1.0 00010000:00010000:1.0:1776178421.217108:0:9225:0:(ldlm_lockd.c:2792:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:0.0:1776178422.114799:0:3355:0:(ptlrpcd.c:415:ptlrpcd_check()) transfer 1 async RPCs [3->2] 00000100:00100000:0.0:1776178422.114831:0:3355:0:(client.c:1733:ptlrpc_send_new_req()) Sending RPC req@000000009220b2b0 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_02:lustre-MDT0001-mdtlov_UUID:3355:1862456998079680:0@lo:13:osp-pre-1-1.0 00000100:00100000:0.0:1776178422.114913:0:3355:0:(events.c:361:request_in_callback()) peer: 12345-0@lo (source: 12345-0@lo) 00000100:00100000:2.0:1776178422.115390:0:9554:0:(service.c:2152:ptlrpc_server_handle_req_in()) got req x1862456998079680 00000100:00100000:2.0:1776178422.115454:0:9554:0:(nrs_fifo.c:179:nrs_fifo_req_get()) NRS start fifo request from 12345-0@lo, seq: 573 00000100:00100000:2.0:1776178422.115460:0:9554:0:(service.c:2324:ptlrpc_server_handle_request()) Handling RPC req@000000004e1a7b1b pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_create00:lustre-MDT0001-mdtlov_UUID+5:3355:x1862456998079680:12345-0@lo:13:osp-pre-1-1.0 00002000:00000024:2.0:1776178422.115492:0:9554:0:(ofd_obd.c:740:ofd_statfs()) blocks cached 229376 granted 491746368 pending 0 free 3923689472 avail 3692167168 00000100:00100000:2.0:1776178422.115555:0:9554:0:(service.c:2372:ptlrpc_server_handle_request()) Handled RPC req@000000004e1a7b1b pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_create00:lustre-MDT0001-mdtlov_UUID+5:3355:x1862456998079680:12345-0@lo:13:osp-pre-1-1.0 Request processed in 94us (643us total) trans 0 rc 0/0 00000100:00100000:2.0:1776178422.115562:0:9554:0:(nrs_fifo.c:241:nrs_fifo_req_stop()) NRS stop fifo request from 12345-0@lo, seq: 573 00000100:00100000:0.0:1776178422.115657:0:3355:0:(client.c:2215:ptlrpc_check_set()) Completed RPC req@000000009220b2b0 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_02:lustre-MDT0001-mdtlov_UUID:3355:1862456998079680:0@lo:13:osp-pre-1-1.0 00010000:00010000:3.0:1776178422.240376:0:6244:0:(ldlm_lockd.c:2792:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:1.0:1776178422.624549:0:3354:0:(ptlrpcd.c:415:ptlrpcd_check()) transfer 1 async RPCs [0->1] 00000100:00100000:1.0:1776178422.624573:0:3354:0:(client.c:1733:ptlrpc_send_new_req()) Sending RPC req@000000007d6ef583 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_01:lustre-MDT0001-mdtlov_UUID:3354:1862456998079744:0@lo:13:osp-pre-0-1.0 00000100:00100000:1.0:1776178422.624649:0:3354:0:(events.c:361:request_in_callback()) peer: 12345-0@lo (source: 12345-0@lo) 00000100:00100000:2.0:1776178422.624888:0:8231:0:(service.c:2152:ptlrpc_server_handle_req_in()) got req x1862456998079744 00000100:00100000:2.0:1776178422.625186:0:8231:0:(nrs_fifo.c:179:nrs_fifo_req_get()) NRS start fifo request from 12345-0@lo, seq: 574 00000100:00100000:2.0:1776178422.625194:0:8231:0:(service.c:2324:ptlrpc_server_handle_request()) Handling RPC req@00000000b9a3856b pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_create00:lustre-MDT0001-mdtlov_UUID+5:3354:x1862456998079744:12345-0@lo:13:osp-pre-0-1.0 00002000:00000024:2.0:1776178422.625232:0:8231:0:(ofd_obd.c:740:ofd_statfs()) blocks cached 315392 granted 484883776 pending 0 free 3923689472 avail 3692167168 00000100:00100000:2.0:1776178422.625297:0:8231:0:(service.c:2372:ptlrpc_server_handle_request()) Handled RPC req@00000000b9a3856b pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_create00:lustre-MDT0001-mdtlov_UUID+5:3354:x1862456998079744:12345-0@lo:13:osp-pre-0-1.0 Request processed in 102us (648us total) trans 0 rc 0/0 00000100:00100000:2.0:1776178422.625305:0:8231:0:(nrs_fifo.c:241:nrs_fifo_req_stop()) NRS stop fifo request from 12345-0@lo, seq: 574 00000100:00100000:1.0:1776178422.625346:0:3354:0:(client.c:2215:ptlrpc_check_set()) Completed RPC req@000000007d6ef583 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_01:lustre-MDT0001-mdtlov_UUID:3354:1862456998079744:0@lo:13:osp-pre-0-1.0 00010000:00010000:2.0:1776178423.264747:0:6245:0:(ldlm_lockd.c:2792:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:1.0:1776178424.098644:0:3348:0:(events.c:361:request_in_callback()) peer: 12345-192.168.204.3@tcp (source: 12345-192.168.204.3@tcp) 00000100:00100000:1.0:1776178424.098700:0:3348:0:(events.c:361:request_in_callback()) peer: 12345-192.168.204.3@tcp (source: 12345-192.168.204.3@tcp) 00000100:00100000:1.0:1776178424.098742:0:3348:0:(events.c:361:request_in_callback()) peer: 12345-192.168.204.3@tcp (source: 12345-192.168.204.3@tcp) 00000100:00100000:0.0:1776178424.098744:0:6255:0:(service.c:2152:ptlrpc_server_handle_req_in()) got req x1862456979122880 00000100:00100000:1.0:1776178424.098794:0:3348:0:(events.c:361:request_in_callback()) peer: 12345-192.168.204.3@tcp (source: 12345-192.168.204.3@tcp) 00000100:00100000:1.0:1776178424.098832:0:11281:0:(nrs_fifo.c:179:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.204.3@tcp, seq: 1057 00000100:00100000:1.0:1776178424.098842:0:11281:0:(service.c:2324:ptlrpc_server_handle_request()) Handling RPC req@000000005accf721 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_004:b50963bd-8eba-4432-9a54-8888d00e3fa1+6:2273:x1862456979122880:12345-192.168.204.3@tcp:400:kworker/u8:4.0 00000100:00100000:1.0:1776178424.098954:0:11281:0:(service.c:2372:ptlrpc_server_handle_request()) Handled RPC req@000000005accf721 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_004:b50963bd-8eba-4432-9a54-8888d00e3fa1+6:2273:x1862456979122880:12345-192.168.204.3@tcp:400:kworker/u8:4.0 Request processed in 112us (319us total) trans 0 rc 0/0 00000100:00100000:1.0:1776178424.098964:0:11281:0:(nrs_fifo.c:241:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.204.3@tcp, seq: 1057 00000100:00100000:2.0:1776178424.099155:0:20759:0:(service.c:2152:ptlrpc_server_handle_req_in()) got req x1862456979123008 00000100:00100000:2.0:1776178424.099199:0:20759:0:(service.c:2152:ptlrpc_server_handle_req_in()) got req x1862456979122944 00000100:00100000:2.0:1776178424.099224:0:20759:0:(nrs_fifo.c:179:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.204.3@tcp, seq: 683 00000100:00100000:2.0:1776178424.099229:0:20759:0:(service.c:2324:ptlrpc_server_handle_request()) Handling RPC req@000000005fdea934 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_010:b50963bd-8eba-4432-9a54-8888d00e3fa1+9:2273:x1862456979123008:12345-192.168.204.3@tcp:400:kworker/u8:4.0 00000100:00100000:2.0:1776178424.099283:0:20759:0:(service.c:2372:ptlrpc_server_handle_request()) Handled RPC req@000000005fdea934 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_010:b50963bd-8eba-4432-9a54-8888d00e3fa1+9:2273:x1862456979123008:12345-192.168.204.3@tcp:400:kworker/u8:4.0 Request processed in 45us (532us total) trans 0 rc 0/0 00000100:00100000:2.0:1776178424.099289:0:20759:0:(nrs_fifo.c:241:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.204.3@tcp, seq: 683 00000100:00100000:2.0:1776178424.099304:0:20759:0:(nrs_fifo.c:179:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.204.3@tcp, seq: 684 00000100:00100000:2.0:1776178424.099309:0:20759:0:(service.c:2324:ptlrpc_server_handle_request()) Handling RPC req@00000000d5d07a1d pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_010:b50963bd-8eba-4432-9a54-8888d00e3fa1+8:2272:x1862456979122944:12345-192.168.204.3@tcp:400:kworker/u8:4.0 00000100:00100000:2.0:1776178424.099334:0:20759:0:(service.c:2372:ptlrpc_server_handle_request()) Handled RPC req@00000000d5d07a1d pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_010:b50963bd-8eba-4432-9a54-8888d00e3fa1+8:2272:x1862456979122944:12345-192.168.204.3@tcp:400:kworker/u8:4.0 Request processed in 24us (542us total) trans 0 rc 0/0 00000100:00100000:2.0:1776178424.099339:0:20759:0:(nrs_fifo.c:241:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.204.3@tcp, seq: 684 00000100:00100000:3.0:1776178424.100258:0:20140:0:(service.c:2152:ptlrpc_server_handle_req_in()) got req x1862456979122752 00000100:00100000:3.0:1776178424.100313:0:20140:0:(nrs_fifo.c:179:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.204.3@tcp, seq: 241 00000100:00100000:3.0:1776178424.100319:0:20140:0:(service.c:2324:ptlrpc_server_handle_request()) Handling RPC req@000000005f72129d pname:cluuid+ref:pid:xid:nid:opc:job ll_mgs_0002:5b5c66b1-1a2f-4651-8548-0c75b9a38a94+9:2272:x1862456979122752:12345-192.168.204.3@tcp:400:kworker/u8:4.0 00000100:00100000:3.0:1776178424.100372:0:20140:0:(service.c:2372:ptlrpc_server_handle_request()) Handled RPC req@000000005f72129d pname:cluuid+ref:pid:xid:nid:opc:job ll_mgs_0002:5b5c66b1-1a2f-4651-8548-0c75b9a38a94+9:2272:x1862456979122752:12345-192.168.204.3@tcp:400:kworker/u8:4.0 Request processed in 52us (1672us total) trans 0 rc 0/0 00000100:00100000:3.0:1776178424.100378:0:20140:0:(nrs_fifo.c:241:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.204.3@tcp, seq: 241 00000100:00100000:1.0:1776178424.100440:0:3348:0:(events.c:361:request_in_callback()) peer: 12345-192.168.204.3@tcp (source: 12345-192.168.204.3@tcp) 00000100:00100000:3.0:1776178424.100586:0:11281:0:(service.c:2152:ptlrpc_server_handle_req_in()) got req x1862456979122816 00000100:00100000:3.0:1776178424.100730:0:11281:0:(nrs_fifo.c:179:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.204.3@tcp, seq: 6796 00000100:00100000:3.0:1776178424.100736:0:11281:0:(service.c:2324:ptlrpc_server_handle_request()) Handling RPC req@000000009126d8c6 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_004:0+-99:2269:x1862456979122816:12345-192.168.204.3@tcp:38:kworker/u8:4.0 00010000:02000000:3.0:1776178424.100754:0:11281:0:(ldlm_lib.c:1139:target_handle_connect()) lustre-MDT0000: Not available for connect from 192.168.204.3@tcp (stopping) 00000100:00100000:3.0:1776178424.100788:0:11281:0:(service.c:2372:ptlrpc_server_handle_request()) Handled RPC req@000000009126d8c6 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_004:0+-99:2269:x1862456979122816:12345-192.168.204.3@tcp:38:kworker/u8:4.0 Request processed in 51us (350us total) trans 0 rc -19/-19 00000100:00100000:3.0:1776178424.100794:0:11281:0:(nrs_fifo.c:241:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.204.3@tcp, seq: 6796 00010000:00010000:2.0:1776178424.291551:0:9224:0:(ldlm_lockd.c:2792:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00010000:00010000:1.0:1776178425.312528:0:9225:0:(ldlm_lockd.c:2792:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00080000:3.0:1776178425.697166:0:36:0:(pinger.c:251:ptlrpc_pinger_process_import()) lustre-MDT0000-lwp-OST0000_UUID->lustre-MDT0000_UUID: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00080000:3.0:1776178425.697174:0:36:0:(recover.c:57:ptlrpc_initiate_recovery()) lustre-MDT0000_UUID: starting recovery 00000100:00080000:3.0:1776178425.697178:0:36:0:(import.c:86:import_set_state_nolock()) 0000000047601a98 lustre-MDT0000_UUID: changing import state from DISCONN to CONNECTING 00000100:00080000:3.0:1776178425.697189:0:36:0:(import.c:534:import_select_connection()) lustre-MDT0000-lwp-OST0000: connect to NID 0@lo last attempt 1127 00000100:00080000:3.0:1776178425.697193:0:36:0:(import.c:581:import_select_connection()) lustre-MDT0000-lwp-OST0000: tried all connections, increasing latency to 21s 00000100:00080000:3.0:1776178425.697203:0:36:0:(import.c:615:import_select_connection()) lustre-MDT0000-lwp-OST0000: import 0000000047601a98 using connection 192.168.204.103@tcp/0@lo 00000100:00100000:3.0:1776178425.697233:0:36:0:(import.c:807:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5) req@00000000b2b2b654 x1862456998079872/t0(0) o38->lustre-MDT0000-lwp-OST0000@0@lo:12/10 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' 00000100:00080000:3.0:1776178425.697289:0:36:0:(pinger.c:251:ptlrpc_pinger_process_import()) lustre-MDT0000-lwp-OST0001_UUID->lustre-MDT0000_UUID: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00080000:3.0:1776178425.697293:0:36:0:(recover.c:57:ptlrpc_initiate_recovery()) lustre-MDT0000_UUID: starting recovery 00000100:00080000:3.0:1776178425.697294:0:36:0:(import.c:86:import_set_state_nolock()) 00000000dfc8db13 lustre-MDT0000_UUID: changing import state from DISCONN to CONNECTING 00000100:00080000:3.0:1776178425.697297:0:36:0:(import.c:534:import_select_connection()) lustre-MDT0000-lwp-OST0001: connect to NID 0@lo last attempt 1127 00000100:00080000:3.0:1776178425.697299:0:36:0:(import.c:581:import_select_connection()) lustre-MDT0000-lwp-OST0001: tried all connections, increasing latency to 21s 00000100:00080000:3.0:1776178425.697303:0:36:0:(import.c:615:import_select_connection()) lustre-MDT0000-lwp-OST0001: import 00000000dfc8db13 using connection 192.168.204.103@tcp/0@lo 00000100:00100000:3.0:1776178425.697314:0:36:0:(import.c:807:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5) req@000000004547bc6e x1862456998080000/t0(0) o38->lustre-MDT0000-lwp-OST0001@0@lo:12/10 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' 00000100:00080000:3.0:1776178425.697330:0:36:0:(pinger.c:251:ptlrpc_pinger_process_import()) lustre-MDT0001-mdtlov_UUID->lustre-MDT0000_UUID: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00080000:3.0:1776178425.697333:0:36:0:(recover.c:57:ptlrpc_initiate_recovery()) lustre-MDT0000_UUID: starting recovery 00000100:00080000:3.0:1776178425.697334:0:36:0:(import.c:86:import_set_state_nolock()) 00000000ba4f1821 lustre-MDT0000_UUID: changing import state from DISCONN to CONNECTING 00000100:00080000:3.0:1776178425.697338:0:36:0:(import.c:534:import_select_connection()) lustre-MDT0000-osp-MDT0001: connect to NID 0@lo last attempt 1127 00000100:00080000:3.0:1776178425.697339:0:36:0:(import.c:581:import_select_connection()) lustre-MDT0000-osp-MDT0001: tried all connections, increasing latency to 21s 00000100:00080000:3.0:1776178425.697342:0:36:0:(import.c:615:import_select_connection()) lustre-MDT0000-osp-MDT0001: import 00000000ba4f1821 using connection 192.168.204.103@tcp/0@lo 00000100:00100000:3.0:1776178425.697352:0:36:0:(import.c:807:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5) req@00000000b5990455 x1862456998080128/t0(0) o38->lustre-MDT0000-osp-MDT0001@0@lo:24/4 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' 00000100:00080000:3.0:1776178425.697370:0:36:0:(pinger.c:251:ptlrpc_pinger_process_import()) lustre-MDT0000-lwp-MDT0001_UUID->lustre-MDT0000_UUID: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00080000:3.0:1776178425.697374:0:36:0:(recover.c:57:ptlrpc_initiate_recovery()) lustre-MDT0000_UUID: starting recovery 00000100:00080000:3.0:1776178425.697375:0:36:0:(import.c:86:import_set_state_nolock()) 00000000a7c55b1f lustre-MDT0000_UUID: changing import state from DISCONN to CONNECTING 00000100:00080000:3.0:1776178425.697378:0:36:0:(import.c:534:import_select_connection()) lustre-MDT0000-lwp-MDT0001: connect to NID 0@lo last attempt 1127 00000100:00080000:3.0:1776178425.697380:0:36:0:(import.c:581:import_select_connection()) lustre-MDT0000-lwp-MDT0001: tried all connections, increasing latency to 21s 00000100:00080000:3.0:1776178425.697382:0:36:0:(import.c:615:import_select_connection()) lustre-MDT0000-lwp-MDT0001: import 00000000a7c55b1f using connection 192.168.204.103@tcp/0@lo 00000100:00100000:3.0:1776178425.697391:0:36:0:(import.c:807:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5) req@00000000c8bb8784 x1862456998080320/t0(0) o38->lustre-MDT0000-lwp-MDT0001@0@lo:12/10 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' 00000100:00100000:3.0:1776178425.697423:0:3355:0:(client.c:1733:ptlrpc_send_new_req()) Sending RPC req@0000000013888310 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_02:lustre-MDT0001-lwp-OST0000_UUID:3355:1862456998079936:0@lo:400:kworker/u8:1.0 00000100:00100000:3.0:1776178425.697533:0:3355:0:(events.c:361:request_in_callback()) peer: 12345-0@lo (source: 12345-0@lo) 00000100:00100000:3.0:1776178425.697581:0:3355:0:(ptlrpcd.c:415:ptlrpcd_check()) transfer 1 async RPCs [3->2] 00000100:00100000:3.0:1776178425.697587:0:3355:0:(client.c:1733:ptlrpc_send_new_req()) Sending RPC req@0000000028dd2f9a pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_02:lustre-MDT0001-lwp-OST0001_UUID:3355:1862456998080064:0@lo:400:kworker/u8:1.0 00000100:00100000:3.0:1776178425.697609:0:3355:0:(events.c:361:request_in_callback()) peer: 12345-0@lo (source: 12345-0@lo) 00000100:00100000:3.0:1776178425.697660:0:11281:0:(service.c:2152:ptlrpc_server_handle_req_in()) got req x1862456998079936 00000100:00100000:3.0:1776178425.697709:0:11281:0:(service.c:2152:ptlrpc_server_handle_req_in()) got req x1862456998080064 00000100:00100000:3.0:1776178425.697728:0:11281:0:(nrs_fifo.c:179:nrs_fifo_req_get()) NRS start fifo request from 12345-0@lo, seq: 1058 00000100:00100000:3.0:1776178425.697735:0:11281:0:(service.c:2324:ptlrpc_server_handle_request()) Handling RPC req@0000000050ea472d pname:cluuid+ref:pid:xid:nid:opc:job mdt00_004:lustre-MDT0001-lwp-OST0000_UUID+5:3355:x1862456998079936:12345-0@lo:400:kworker/u8:1.0 00000100:00100000:3.0:1776178425.697815:0:11281:0:(service.c:2372:ptlrpc_server_handle_request()) Handled RPC req@0000000050ea472d pname:cluuid+ref:pid:xid:nid:opc:job mdt00_004:lustre-MDT0001-lwp-OST0000_UUID+5:3355:x1862456998079936:12345-0@lo:400:kworker/u8:1.0 Request processed in 80us (284us total) trans 0 rc 0/0 00000100:00100000:3.0:1776178425.697823:0:11281:0:(nrs_fifo.c:241:nrs_fifo_req_stop()) NRS stop fifo request from 12345-0@lo, seq: 1058 00000100:00100000:3.0:1776178425.697843:0:11281:0:(nrs_fifo.c:179:nrs_fifo_req_get()) NRS start fifo request from 12345-0@lo, seq: 1059 00000100:00100000:3.0:1776178425.697847:0:11281:0:(service.c:2324:ptlrpc_server_handle_request()) Handling RPC req@00000000609aa1cd pname:cluuid+ref:pid:xid:nid:opc:job mdt00_004:lustre-MDT0001-lwp-OST0001_UUID+5:3355:x1862456998080064:12345-0@lo:400:kworker/u8:1.0 00000100:00100000:3.0:1776178425.697870:0:11281:0:(service.c:2372:ptlrpc_server_handle_request()) Handled RPC req@00000000609aa1cd pname:cluuid+ref:pid:xid:nid:opc:job mdt00_004:lustre-MDT0001-lwp-OST0001_UUID+5:3355:x1862456998080064:12345-0@lo:400:kworker/u8:1.0 Request processed in 23us (261us total) trans 0 rc 0/0 00000100:00100000:3.0:1776178425.697875:0:11281:0:(nrs_fifo.c:241:nrs_fifo_req_stop()) NRS stop fifo request from 12345-0@lo, seq: 1059 00000100:00100000:3.0:1776178425.697920:0:3355:0:(client.c:2215:ptlrpc_check_set()) Completed RPC req@0000000028dd2f9a pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_02:lustre-MDT0001-lwp-OST0001_UUID:3355:1862456998080064:0@lo:400:kworker/u8:1.0 00000100:00100000:3.0:1776178425.697931:0:3355:0:(client.c:2215:ptlrpc_check_set()) Completed RPC req@0000000013888310 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_02:lustre-MDT0001-lwp-OST0000_UUID:3355:1862456998079936:0@lo:400:kworker/u8:1.0 00000100:00100000:0.0:1776178425.698109:0:3352:0:(client.c:1733:ptlrpc_send_new_req()) Sending RPC req@00000000b2b2b654 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:lustre-MDT0000-lwp-OST0000_UUID:3352:1862456998079872:0@lo:38:kworker/u8:1.0 00000100:00100000:0.0:1776178425.698133:0:3352:0:(events.c:361:request_in_callback()) peer: 12345-0@lo (source: 12345-0@lo) 00000100:00100000:0.0:1776178425.698151:0:3352:0:(client.c:1733:ptlrpc_send_new_req()) Sending RPC req@000000004547bc6e pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:lustre-MDT0000-lwp-OST0001_UUID:3352:1862456998080000:0@lo:38:kworker/u8:1.0 00000100:00100000:0.0:1776178425.698166:0:3352:0:(events.c:361:request_in_callback()) peer: 12345-0@lo (source: 12345-0@lo) 00000100:00100000:0.0:1776178425.698176:0:3352:0:(client.c:1733:ptlrpc_send_new_req()) Sending RPC req@00000000b5990455 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:lustre-MDT0001-mdtlov_UUID:3352:1862456998080128:0@lo:38:kworker/u8:1.0 00000100:00100000:0.0:1776178425.698193:0:3352:0:(events.c:361:request_in_callback()) peer: 12345-0@lo (source: 12345-0@lo) 00000100:00100000:0.0:1776178425.698204:0:3352:0:(client.c:1733:ptlrpc_send_new_req()) Sending RPC req@00000000c8bb8784 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:lustre-MDT0000-lwp-MDT0001_UUID:3352:1862456998080320:0@lo:38:kworker/u8:1.0 00000100:00100000:0.0:1776178425.698217:0:3352:0:(events.c:361:request_in_callback()) peer: 12345-0@lo (source: 12345-0@lo) 00000100:00100000:0.0:1776178425.698239:0:15213:0:(service.c:2152:ptlrpc_server_handle_req_in()) got req x1862456998080128 00000100:00100000:0.0:1776178425.698254:0:15213:0:(nrs_fifo.c:179:nrs_fifo_req_get()) NRS start fifo request from 12345-0@lo, seq: 751 00000100:00100000:0.0:1776178425.698257:0:15213:0:(service.c:2324:ptlrpc_server_handle_request()) Handling RPC req@0000000056e345df pname:cluuid+ref:pid:xid:nid:opc:job mdt_out00_003:0+-99:3352:x1862456998080128:12345-0@lo:38:kworker/u8:1.0 00010000:02000000:0.0:1776178425.698269:0:15213:0:(ldlm_lib.c:1139:target_handle_connect()) lustre-MDT0000: Not available for connect from 0@lo (stopping) 00000100:00100000:0.0:1776178425.698291:0:15213:0:(service.c:2372:ptlrpc_server_handle_request()) Handled RPC req@0000000056e345df pname:cluuid+ref:pid:xid:nid:opc:job mdt_out00_003:0+-99:3352:x1862456998080128:12345-0@lo:38:kworker/u8:1.0 Request processed in 33us (97us total) trans 0 rc -19/-19 00000100:00100000:0.0:1776178425.698294:0:15213:0:(nrs_fifo.c:241:nrs_fifo_req_stop()) NRS stop fifo request from 12345-0@lo, seq: 751 00000100:00100000:0.0:1776178425.698308:0:6257:0:(service.c:2152:ptlrpc_server_handle_req_in()) got req x1862456998079872 00000100:00100000:0.0:1776178425.698315:0:6257:0:(service.c:2152:ptlrpc_server_handle_req_in()) got req x1862456998080000 00000100:00100000:0.0:1776178425.698319:0:6257:0:(service.c:2152:ptlrpc_server_handle_req_in()) got req x1862456998080320 00000100:00100000:0.0:1776178425.698324:0:6257:0:(nrs_fifo.c:179:nrs_fifo_req_get()) NRS start fifo request from 12345-0@lo, seq: 6797 00000100:00100000:0.0:1776178425.698326:0:6257:0:(service.c:2324:ptlrpc_server_handle_request()) Handling RPC req@000000002549e7f5 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_002:0+-99:3352:x1862456998079872:12345-0@lo:38:kworker/u8:1.0 00010000:02000000:0.0:1776178425.698330:0:6257:0:(ldlm_lib.c:1139:target_handle_connect()) lustre-MDT0000: Not available for connect from 0@lo (stopping) 00000100:00100000:0.0:1776178425.698343:0:6257:0:(service.c:2372:ptlrpc_server_handle_request()) Handled RPC req@000000002549e7f5 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_002:0+-99:3352:x1862456998079872:12345-0@lo:38:kworker/u8:1.0 Request processed in 16us (210us total) trans 0 rc -19/-19 00000100:00100000:0.0:1776178425.698346:0:6257:0:(nrs_fifo.c:241:nrs_fifo_req_stop()) NRS stop fifo request from 12345-0@lo, seq: 6797 00000100:00100000:0.0:1776178425.698352:0:6257:0:(nrs_fifo.c:179:nrs_fifo_req_get()) NRS start fifo request from 12345-0@lo, seq: 6798 00000100:00100000:0.0:1776178425.698353:0:6257:0:(service.c:2324:ptlrpc_server_handle_request()) Handling RPC req@00000000226852f3 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_002:0+-99:3352:x1862456998080000:12345-0@lo:38:kworker/u8:1.0 00010000:02000000:0.0:1776178425.698357:0:6257:0:(ldlm_lib.c:1139:target_handle_connect()) lustre-MDT0000: Not available for connect from 0@lo (stopping) 00000100:00100000:0.0:1776178425.698366:0:6257:0:(service.c:2372:ptlrpc_server_handle_request()) Handled RPC req@00000000226852f3 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_002:0+-99:3352:x1862456998080000:12345-0@lo:38:kworker/u8:1.0 Request processed in 12us (200us total) trans 0 rc -19/-19 00000100:00100000:0.0:1776178425.698369:0:6257:0:(nrs_fifo.c:241:nrs_fifo_req_stop()) NRS stop fifo request from 12345-0@lo, seq: 6798 00000100:00100000:0.0:1776178425.698373:0:6257:0:(nrs_fifo.c:179:nrs_fifo_req_get()) NRS start fifo request from 12345-0@lo, seq: 6799 00000100:00100000:0.0:1776178425.698375:0:6257:0:(service.c:2324:ptlrpc_server_handle_request()) Handling RPC req@000000001946bc3d pname:cluuid+ref:pid:xid:nid:opc:job mdt00_002:0+-99:3352:x1862456998080320:12345-0@lo:38:kworker/u8:1.0 00010000:02000000:0.0:1776178425.698378:0:6257:0:(ldlm_lib.c:1139:target_handle_connect()) lustre-MDT0000: Not available for connect from 0@lo (stopping) 00000100:00100000:0.0:1776178425.698386:0:6257:0:(service.c:2372:ptlrpc_server_handle_request()) Handled RPC req@000000001946bc3d pname:cluuid+ref:pid:xid:nid:opc:job mdt00_002:0+-99:3352:x1862456998080320:12345-0@lo:38:kworker/u8:1.0 Request processed in 11us (169us total) trans 0 rc -19/-19 00000100:00100000:0.0:1776178425.698389:0:6257:0:(nrs_fifo.c:241:nrs_fifo_req_stop()) NRS stop fifo request from 12345-0@lo, seq: 6799 00000100:00080000:0.0:1776178425.698404:0:3352:0:(import.c:86:import_set_state_nolock()) 0000000047601a98 lustre-MDT0000_UUID: changing import state from CONNECTING to DISCONN 00000100:00080000:0.0:1776178425.698407:0:3352:0:(import.c:1429:ptlrpc_connect_interpret()) recovery of lustre-MDT0000_UUID on 192.168.204.103@tcp failed (-19) 00000100:00100000:0.0:1776178425.698409:0:3352:0:(client.c:2215:ptlrpc_check_set()) Completed RPC req@00000000b2b2b654 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:lustre-MDT0000-lwp-OST0000_UUID:3352:1862456998079872:0@lo:38:kworker/u8:1.0 00000100:00080000:0.0:1776178425.698413:0:3352:0:(import.c:86:import_set_state_nolock()) 00000000dfc8db13 lustre-MDT0000_UUID: changing import state from CONNECTING to DISCONN 00000100:00080000:0.0:1776178425.698415:0:3352:0:(import.c:1429:ptlrpc_connect_interpret()) recovery of lustre-MDT0000_UUID on 192.168.204.103@tcp failed (-19) 00000100:00100000:0.0:1776178425.698416:0:3352:0:(client.c:2215:ptlrpc_check_set()) Completed RPC req@000000004547bc6e pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:lustre-MDT0000-lwp-OST0001_UUID:3352:1862456998080000:0@lo:38:kworker/u8:1.0 00000100:00080000:0.0:1776178425.698420:0:3352:0:(import.c:86:import_set_state_nolock()) 00000000ba4f1821 lustre-MDT0000_UUID: changing import state from CONNECTING to DISCONN 00000100:00080000:0.0:1776178425.698422:0:3352:0:(import.c:1429:ptlrpc_connect_interpret()) recovery of lustre-MDT0000_UUID on 192.168.204.103@tcp failed (-19) 00000100:00100000:0.0:1776178425.698424:0:3352:0:(client.c:2215:ptlrpc_check_set()) Completed RPC req@00000000b5990455 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:lustre-MDT0001-mdtlov_UUID:3352:1862456998080128:0@lo:38:kworker/u8:1.0 00000100:00080000:0.0:1776178425.698428:0:3352:0:(import.c:86:import_set_state_nolock()) 00000000a7c55b1f lustre-MDT0000_UUID: changing import state from CONNECTING to DISCONN 00000100:00080000:0.0:1776178425.698430:0:3352:0:(import.c:1429:ptlrpc_connect_interpret()) recovery of lustre-MDT0000_UUID on 192.168.204.103@tcp failed (-19) 00000100:00100000:0.0:1776178425.698431:0:3352:0:(client.c:2215:ptlrpc_check_set()) Completed RPC req@00000000c8bb8784 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:lustre-MDT0000-lwp-MDT0001_UUID:3352:1862456998080320:0@lo:38:kworker/u8:1.0 00000100:00100000:1.0:1776178425.698517:0:3354:0:(client.c:1733:ptlrpc_send_new_req()) Sending RPC req@000000001ef63da8 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_01:ac61f839-0e8e-4cac-9c59-08bcfbbad330:3354:1862456998079808:0@lo:400:kworker/u8:1.0 00000100:00100000:1.0:1776178425.698557:0:3354:0:(events.c:361:request_in_callback()) peer: 12345-0@lo (source: 12345-0@lo) 00000100:00100000:1.0:1776178425.698580:0:3354:0:(client.c:1733:ptlrpc_send_new_req()) Sending RPC req@00000000ad228ff6 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_01:lustre-MDT0001-mdtlov_UUID:3354:1862456998080256:0@lo:400:kworker/u8:1.0 00000100:00100000:1.0:1776178425.698603:0:3354:0:(events.c:361:request_in_callback()) peer: 12345-0@lo (source: 12345-0@lo) 00000100:00100000:1.0:1776178425.698623:0:3354:0:(ptlrpcd.c:415:ptlrpcd_check()) transfer 1 async RPCs [0->1] 00000100:00100000:1.0:1776178425.698628:0:3354:0:(client.c:1733:ptlrpc_send_new_req()) Sending RPC req@00000000fdb309c4 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_01:lustre-MDT0001-mdtlov_UUID:3354:1862456998080192:0@lo:400:kworker/u8:1.0 00000100:00100000:1.0:1776178425.698648:0:3354:0:(events.c:361:request_in_callback()) peer: 12345-0@lo (source: 12345-0@lo) 00000100:00100000:1.0:1776178425.698681:0:20762:0:(service.c:2152:ptlrpc_server_handle_req_in()) got req x1862456998080256 00000100:00100000:1.0:1776178425.698711:0:20762:0:(service.c:2152:ptlrpc_server_handle_req_in()) got req x1862456998080192 00000100:00100000:1.0:1776178425.698727:0:20762:0:(nrs_fifo.c:179:nrs_fifo_req_get()) NRS start fifo request from 12345-0@lo, seq: 685 00000100:00100000:1.0:1776178425.698729:0:20762:0:(service.c:2324:ptlrpc_server_handle_request()) Handling RPC req@000000001554c6b4 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_013:lustre-MDT0001-mdtlov_UUID+5:3354:x1862456998080256:12345-0@lo:400:kworker/u8:1.0 00000100:00100000:1.0:1776178425.698754:0:20762:0:(service.c:2372:ptlrpc_server_handle_request()) Handled RPC req@000000001554c6b4 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_013:lustre-MDT0001-mdtlov_UUID+5:3354:x1862456998080256:12345-0@lo:400:kworker/u8:1.0 Request processed in 24us (151us total) trans 0 rc 0/0 00000100:00100000:1.0:1776178425.698757:0:20762:0:(nrs_fifo.c:241:nrs_fifo_req_stop()) NRS stop fifo request from 12345-0@lo, seq: 685 00000100:00100000:1.0:1776178425.698765:0:20762:0:(nrs_fifo.c:179:nrs_fifo_req_get()) NRS start fifo request from 12345-0@lo, seq: 686 00000100:00100000:1.0:1776178425.698767:0:20762:0:(service.c:2324:ptlrpc_server_handle_request()) Handling RPC req@0000000068adf51c pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_013:lustre-MDT0001-mdtlov_UUID+5:3354:x1862456998080192:12345-0@lo:400:kworker/u8:1.0 00000100:00100000:1.0:1776178425.698780:0:20762:0:(service.c:2372:ptlrpc_server_handle_request()) Handled RPC req@0000000068adf51c pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_013:lustre-MDT0001-mdtlov_UUID+5:3354:x1862456998080192:12345-0@lo:400:kworker/u8:1.0 Request processed in 12us (133us total) trans 0 rc 0/0 00000100:00100000:1.0:1776178425.698783:0:20762:0:(nrs_fifo.c:241:nrs_fifo_req_stop()) NRS stop fifo request from 12345-0@lo, seq: 686 00000100:00100000:1.0:1776178425.698803:0:3354:0:(client.c:2215:ptlrpc_check_set()) Completed RPC req@00000000fdb309c4 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_01:lustre-MDT0001-mdtlov_UUID:3354:1862456998080192:0@lo:400:kworker/u8:1.0 00000100:00100000:1.0:1776178425.698808:0:3354:0:(client.c:2215:ptlrpc_check_set()) Completed RPC req@00000000ad228ff6 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_01:lustre-MDT0001-mdtlov_UUID:3354:1862456998080256:0@lo:400:kworker/u8:1.0 00000100:00100000:0.0:1776178425.698865:0:20137:0:(service.c:2152:ptlrpc_server_handle_req_in()) got req x1862456998079808 00000100:00100000:0.0:1776178425.698900:0:20137:0:(nrs_fifo.c:179:nrs_fifo_req_get()) NRS start fifo request from 12345-0@lo, seq: 242 00000100:00100000:0.0:1776178425.698904:0:20137:0:(service.c:2324:ptlrpc_server_handle_request()) Handling RPC req@00000000a6b47274 pname:cluuid+ref:pid:xid:nid:opc:job ll_mgs_0000:ac61f839-0e8e-4cac-9c59-08bcfbbad330+18:3354:x1862456998079808:12345-0@lo:400:kworker/u8:1.0 00000100:00100000:0.0:1776178425.698934:0:20137:0:(service.c:2372:ptlrpc_server_handle_request()) Handled RPC req@00000000a6b47274 pname:cluuid+ref:pid:xid:nid:opc:job ll_mgs_0000:ac61f839-0e8e-4cac-9c59-08bcfbbad330+18:3354:x1862456998079808:12345-0@lo:400:kworker/u8:1.0 Request processed in 29us (377us total) trans 0 rc 0/0 00000100:00100000:0.0:1776178425.698939:0:20137:0:(nrs_fifo.c:241:nrs_fifo_req_stop()) NRS stop fifo request from 12345-0@lo, seq: 242 00000100:00100000:1.0:1776178425.699137:0:3354:0:(client.c:2215:ptlrpc_check_set()) Completed RPC req@000000001ef63da8 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_01:ac61f839-0e8e-4cac-9c59-08bcfbbad330:3354:1862456998079808:0@lo:400:kworker/u8:1.0 00010000:00010000:3.0:1776178426.336375:0:6244:0:(ldlm_lockd.c:2792:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:0.0:1776178427.232815:0:3356:0:(ptlrpcd.c:415:ptlrpcd_check()) transfer 1 async RPCs [2->3] 00000100:00100000:0.0:1776178427.232849:0:3356:0:(client.c:1733:ptlrpc_send_new_req()) Sending RPC req@00000000a13f5de6 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_03:lustre-MDT0001-mdtlov_UUID:3356:1862456998080384:0@lo:13:osp-pre-1-1.0 00000100:00100000:0.0:1776178427.232945:0:3356:0:(events.c:361:request_in_callback()) peer: 12345-0@lo (source: 12345-0@lo) 00000100:00100000:2.0:1776178427.233056:0:8231:0:(service.c:2152:ptlrpc_server_handle_req_in()) got req x1862456998080384 00000100:00100000:2.0:1776178427.233163:0:8231:0:(nrs_fifo.c:179:nrs_fifo_req_get()) NRS start fifo request from 12345-0@lo, seq: 575 00000100:00100000:2.0:1776178427.233170:0:8231:0:(service.c:2324:ptlrpc_server_handle_request()) Handling RPC req@0000000018bd7474 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_create00:lustre-MDT0001-mdtlov_UUID+5:3356:x1862456998080384:12345-0@lo:13:osp-pre-1-1.0 00002000:00000024:2.0:1776178427.233212:0:8231:0:(ofd_obd.c:740:ofd_statfs()) blocks cached 229376 granted 491746368 pending 0 free 3923689472 avail 3692167168 00000100:00100000:2.0:1776178427.233262:0:8231:0:(service.c:2372:ptlrpc_server_handle_request()) Handled RPC req@0000000018bd7474 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_create00:lustre-MDT0001-mdtlov_UUID+5:3356:x1862456998080384:12345-0@lo:13:osp-pre-1-1.0 Request processed in 92us (320us total) trans 0 rc 0/0 00000100:00100000:2.0:1776178427.233269:0:8231:0:(nrs_fifo.c:241:nrs_fifo_req_stop()) NRS stop fifo request from 12345-0@lo, seq: 575 00000100:00100000:1.0:1776178427.233823:0:3356:0:(client.c:2215:ptlrpc_check_set()) Completed RPC req@00000000a13f5de6 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_03:lustre-MDT0001-mdtlov_UUID:3356:1862456998080384:0@lo:13:osp-pre-1-1.0 00010000:00010000:2.0:1776178427.360425:0:6245:0:(ldlm_lockd.c:2792:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:1.0:1776178427.744279:0:3356:0:(client.c:1733:ptlrpc_send_new_req()) Sending RPC req@000000001da3f43e pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_03:lustre-MDT0001-mdtlov_UUID:3356:1862456998080448:0@lo:13:osp-pre-0-1.0 00000100:00100000:1.0:1776178427.744360:0:3356:0:(events.c:361:request_in_callback()) peer: 12345-0@lo (source: 12345-0@lo) 00000100:00100000:1.0:1776178427.744491:0:9554:0:(service.c:2152:ptlrpc_server_handle_req_in()) got req x1862456998080448 00000100:00100000:1.0:1776178427.744542:0:9554:0:(nrs_fifo.c:179:nrs_fifo_req_get()) NRS start fifo request from 12345-0@lo, seq: 576 00000100:00100000:1.0:1776178427.744547:0:9554:0:(service.c:2324:ptlrpc_server_handle_request()) Handling RPC req@00000000599b0cf2 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_create00:lustre-MDT0001-mdtlov_UUID+5:3356:x1862456998080448:12345-0@lo:13:osp-pre-0-1.0 00002000:00000024:1.0:1776178427.744576:0:9554:0:(ofd_obd.c:740:ofd_statfs()) blocks cached 315392 granted 484883776 pending 0 free 3923689472 avail 3692167168 00000100:00100000:1.0:1776178427.744630:0:9554:0:(service.c:2372:ptlrpc_server_handle_request()) Handled RPC req@00000000599b0cf2 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_create00:lustre-MDT0001-mdtlov_UUID+5:3356:x1862456998080448:12345-0@lo:13:osp-pre-0-1.0 Request processed in 81us (269us total) trans 0 rc 0/0 00000100:00100000:1.0:1776178427.744641:0:9554:0:(nrs_fifo.c:241:nrs_fifo_req_stop()) NRS stop fifo request from 12345-0@lo, seq: 576 00000100:00100000:3.0:1776178427.744929:0:3356:0:(client.c:2215:ptlrpc_check_set()) Completed RPC req@000000001da3f43e pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_03:lustre-MDT0001-mdtlov_UUID:3356:1862456998080448:0@lo:13:osp-pre-0-1.0 00010000:00010000:2.0:1776178428.386317:0:9224:0:(ldlm_lockd.c:2792:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:1.0:1776178429.228454:0:3348:0:(events.c:361:request_in_callback()) peer: 12345-192.168.204.3@tcp (source: 12345-192.168.204.3@tcp) 00000100:00100000:1.0:1776178429.228505:0:3348:0:(events.c:361:request_in_callback()) peer: 12345-192.168.204.3@tcp (source: 12345-192.168.204.3@tcp) 00000100:00100000:1.0:1776178429.228546:0:3348:0:(events.c:361:request_in_callback()) peer: 12345-192.168.204.3@tcp (source: 12345-192.168.204.3@tcp) 00000100:00100000:1.0:1776178429.228583:0:3348:0:(events.c:361:request_in_callback()) peer: 12345-192.168.204.3@tcp (source: 12345-192.168.204.3@tcp) 00000100:00100000:1.0:1776178429.228610:0:3348:0:(events.c:361:request_in_callback()) peer: 12345-192.168.204.3@tcp (source: 12345-192.168.204.3@tcp) 00000100:00100000:1.0:1776178429.228649:0:14440:0:(service.c:2152:ptlrpc_server_handle_req_in()) got req x1862456979123328 00000100:00100000:1.0:1776178429.228696:0:14440:0:(service.c:2152:ptlrpc_server_handle_req_in()) got req x1862456979123264 00000100:00100000:1.0:1776178429.228717:0:14440:0:(nrs_fifo.c:179:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.204.3@tcp, seq: 687 00000100:00100000:1.0:1776178429.228724:0:14440:0:(service.c:2324:ptlrpc_server_handle_request()) Handling RPC req@0000000053e142bc pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_004:b50963bd-8eba-4432-9a54-8888d00e3fa1+9:2273:x1862456979123328:12345-192.168.204.3@tcp:400:kworker/u8:4.0 00000100:00100000:1.0:1776178429.228797:0:14440:0:(service.c:2372:ptlrpc_server_handle_request()) Handled RPC req@0000000053e142bc pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_004:b50963bd-8eba-4432-9a54-8888d00e3fa1+9:2273:x1862456979123328:12345-192.168.204.3@tcp:400:kworker/u8:4.0 Request processed in 72us (292us total) trans 0 rc 0/0 00000100:00100000:1.0:1776178429.228804:0:14440:0:(nrs_fifo.c:241:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.204.3@tcp, seq: 687 00000100:00100000:1.0:1776178429.228823:0:14440:0:(nrs_fifo.c:179:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.204.3@tcp, seq: 688 00000100:00100000:1.0:1776178429.228828:0:14440:0:(service.c:2324:ptlrpc_server_handle_request()) Handling RPC req@000000005fdea934 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_004:b50963bd-8eba-4432-9a54-8888d00e3fa1+8:2273:x1862456979123264:12345-192.168.204.3@tcp:400:kworker/u8:4.0 00000100:00100000:1.0:1776178429.228851:0:14440:0:(service.c:2372:ptlrpc_server_handle_request()) Handled RPC req@000000005fdea934 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_004:b50963bd-8eba-4432-9a54-8888d00e3fa1+8:2273:x1862456979123264:12345-192.168.204.3@tcp:400:kworker/u8:4.0 Request processed in 23us (269us total) trans 0 rc 0/0 00000100:00100000:1.0:1776178429.228857:0:14440:0:(nrs_fifo.c:241:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.204.3@tcp, seq: 688 00000100:00100000:0.0:1776178429.228962:0:20140:0:(service.c:2152:ptlrpc_server_handle_req_in()) got req x1862456979123072 00000100:00100000:0.0:1776178429.229000:0:20140:0:(nrs_fifo.c:179:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.204.3@tcp, seq: 243 00000100:00100000:0.0:1776178429.229004:0:20140:0:(service.c:2324:ptlrpc_server_handle_request()) Handling RPC req@0000000080051315 pname:cluuid+ref:pid:xid:nid:opc:job ll_mgs_0002:5b5c66b1-1a2f-4651-8548-0c75b9a38a94+9:2273:x1862456979123072:12345-192.168.204.3@tcp:400:kworker/u8:4.0 00000100:00100000:0.0:1776178429.229032:0:20140:0:(service.c:2372:ptlrpc_server_handle_request()) Handled RPC req@0000000080051315 pname:cluuid+ref:pid:xid:nid:opc:job ll_mgs_0002:5b5c66b1-1a2f-4651-8548-0c75b9a38a94+9:2273:x1862456979123072:12345-192.168.204.3@tcp:400:kworker/u8:4.0 Request processed in 26us (486us total) trans 0 rc 0/0 00000100:00100000:0.0:1776178429.229038:0:20140:0:(nrs_fifo.c:241:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.204.3@tcp, seq: 243 00000100:00100000:2.0:1776178429.229677:0:6256:0:(service.c:2152:ptlrpc_server_handle_req_in()) got req x1862456979123200 00000100:00100000:2.0:1776178429.229715:0:6256:0:(service.c:2152:ptlrpc_server_handle_req_in()) got req x1862456979123136 00000100:00100000:2.0:1776178429.229736:0:6256:0:(nrs_fifo.c:179:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.204.3@tcp, seq: 1060 00000100:00100000:2.0:1776178429.229741:0:6256:0:(service.c:2324:ptlrpc_server_handle_request()) Handling RPC req@000000008c4bfd71 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_001:b50963bd-8eba-4432-9a54-8888d00e3fa1+6:2273:x1862456979123200:12345-192.168.204.3@tcp:400:kworker/u8:4.0 00000100:00100000:2.0:1776178429.229778:0:6256:0:(service.c:2372:ptlrpc_server_handle_request()) Handled RPC req@000000008c4bfd71 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_001:b50963bd-8eba-4432-9a54-8888d00e3fa1+6:2273:x1862456979123200:12345-192.168.204.3@tcp:400:kworker/u8:4.0 Request processed in 36us (1335us total) trans 0 rc 0/0 00000100:00100000:2.0:1776178429.229786:0:6256:0:(nrs_fifo.c:241:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.204.3@tcp, seq: 1060 00000100:00100000:2.0:1776178429.229800:0:6256:0:(nrs_fifo.c:179:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.204.3@tcp, seq: 6800 00000100:00100000:2.0:1776178429.229804:0:6256:0:(service.c:2324:ptlrpc_server_handle_request()) Handling RPC req@000000006247494d pname:cluuid+ref:pid:xid:nid:opc:job mdt00_001:0+-99:2269:x1862456979123136:12345-192.168.204.3@tcp:38:kworker/u8:4.0 00010000:02000000:2.0:1776178429.229817:0:6256:0:(ldlm_lib.c:1139:target_handle_connect()) lustre-MDT0000: Not available for connect from 192.168.204.3@tcp (stopping) 00000100:00100000:2.0:1776178429.229842:0:6256:0:(service.c:2372:ptlrpc_server_handle_request()) Handled RPC req@000000006247494d pname:cluuid+ref:pid:xid:nid:opc:job mdt00_001:0+-99:2269:x1862456979123136:12345-192.168.204.3@tcp:38:kworker/u8:4.0 Request processed in 37us (1232us total) trans 0 rc -19/-19 00000100:00100000:2.0:1776178429.229849:0:6256:0:(nrs_fifo.c:241:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.204.3@tcp, seq: 6800 00010000:00010000:1.0:1776178429.412027:0:9225:0:(ldlm_lockd.c:2792:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00010000:00010000:3.0:1776178430.432571:0:6244:0:(ldlm_lockd.c:2792:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:2.0:1776178430.826188:0:3353:0:(client.c:1733:ptlrpc_send_new_req()) Sending RPC req@00000000f351bcc3 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_00:ac61f839-0e8e-4cac-9c59-08bcfbbad330:3353:1862456998080512:0@lo:400:kworker/u8:2.0 00000100:00100000:2.0:1776178430.826276:0:3353:0:(events.c:361:request_in_callback()) peer: 12345-0@lo (source: 12345-0@lo) 00000100:00100000:1.0:1776178430.826394:0:20137:0:(service.c:2152:ptlrpc_server_handle_req_in()) got req x1862456998080512 00000100:00100000:1.0:1776178430.826467:0:20137:0:(nrs_fifo.c:179:nrs_fifo_req_get()) NRS start fifo request from 12345-0@lo, seq: 244 00000100:00100000:1.0:1776178430.826473:0:20137:0:(service.c:2324:ptlrpc_server_handle_request()) Handling RPC req@000000008f0263f6 pname:cluuid+ref:pid:xid:nid:opc:job ll_mgs_0000:ac61f839-0e8e-4cac-9c59-08bcfbbad330+18:3353:x1862456998080512:12345-0@lo:400:kworker/u8:2.0 00000100:00080000:0.0:1776178430.827671:0:37:0:(pinger.c:251:ptlrpc_pinger_process_import()) lustre-MDT0000-lwp-OST0000_UUID->lustre-MDT0000_UUID: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00080000:0.0:1776178430.827680:0:37:0:(recover.c:57:ptlrpc_initiate_recovery()) lustre-MDT0000_UUID: starting recovery 00000100:00080000:0.0:1776178430.827684:0:37:0:(import.c:86:import_set_state_nolock()) 0000000047601a98 lustre-MDT0000_UUID: changing import state from DISCONN to CONNECTING 00000100:00080000:0.0:1776178430.827690:0:37:0:(import.c:534:import_select_connection()) lustre-MDT0000-lwp-OST0000: connect to NID 0@lo last attempt 1133 00000100:00080000:0.0:1776178430.827693:0:37:0:(import.c:581:import_select_connection()) lustre-MDT0000-lwp-OST0000: tried all connections, increasing latency to 21s 00000100:00080000:0.0:1776178430.827698:0:37:0:(import.c:615:import_select_connection()) lustre-MDT0000-lwp-OST0000: import 0000000047601a98 using connection 192.168.204.103@tcp/0@lo 00000100:00100000:0.0:1776178430.827729:0:37:0:(import.c:807:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5) req@000000008c50846d x1862456998080576/t0(0) o38->lustre-MDT0000-lwp-OST0000@0@lo:12/10 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' 00000100:00080000:0.0:1776178430.827777:0:37:0:(pinger.c:251:ptlrpc_pinger_process_import()) lustre-MDT0000-lwp-OST0001_UUID->lustre-MDT0000_UUID: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00080000:0.0:1776178430.827780:0:37:0:(recover.c:57:ptlrpc_initiate_recovery()) lustre-MDT0000_UUID: starting recovery 00000100:00080000:0.0:1776178430.827782:0:37:0:(import.c:86:import_set_state_nolock()) 00000000dfc8db13 lustre-MDT0000_UUID: changing import state from DISCONN to CONNECTING 00000100:00080000:0.0:1776178430.827785:0:37:0:(import.c:534:import_select_connection()) lustre-MDT0000-lwp-OST0001: connect to NID 0@lo last attempt 1133 00000100:00080000:0.0:1776178430.827787:0:37:0:(import.c:581:import_select_connection()) lustre-MDT0000-lwp-OST0001: tried all connections, increasing latency to 21s 00000100:00080000:0.0:1776178430.827790:0:37:0:(import.c:615:import_select_connection()) lustre-MDT0000-lwp-OST0001: import 00000000dfc8db13 using connection 192.168.204.103@tcp/0@lo 00000100:00100000:0.0:1776178430.827799:0:37:0:(import.c:807:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5) req@000000007d6ef583 x1862456998080704/t0(0) o38->lustre-MDT0000-lwp-OST0001@0@lo:12/10 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' 00000100:00080000:0.0:1776178430.827820:0:37:0:(pinger.c:251:ptlrpc_pinger_process_import()) lustre-MDT0001-mdtlov_UUID->lustre-MDT0000_UUID: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00080000:0.0:1776178430.827823:0:37:0:(recover.c:57:ptlrpc_initiate_recovery()) lustre-MDT0000_UUID: starting recovery 00000100:00080000:0.0:1776178430.827824:0:37:0:(import.c:86:import_set_state_nolock()) 00000000ba4f1821 lustre-MDT0000_UUID: changing import state from DISCONN to CONNECTING 00000100:00080000:0.0:1776178430.827827:0:37:0:(import.c:534:import_select_connection()) lustre-MDT0000-osp-MDT0001: connect to NID 0@lo last attempt 1133 00000100:00080000:0.0:1776178430.827828:0:37:0:(import.c:581:import_select_connection()) lustre-MDT0000-osp-MDT0001: tried all connections, increasing latency to 21s 00000100:00080000:0.0:1776178430.827831:0:37:0:(import.c:615:import_select_connection()) lustre-MDT0000-osp-MDT0001: import 00000000ba4f1821 using connection 192.168.204.103@tcp/0@lo 00000100:00100000:0.0:1776178430.827840:0:37:0:(import.c:807:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5) req@00000000803faf52 x1862456998080832/t0(0) o38->lustre-MDT0000-osp-MDT0001@0@lo:24/4 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' 00000100:00080000:0.0:1776178430.827860:0:37:0:(pinger.c:251:ptlrpc_pinger_process_import()) lustre-MDT0000-lwp-MDT0001_UUID->lustre-MDT0000_UUID: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00080000:0.0:1776178430.827863:0:37:0:(recover.c:57:ptlrpc_initiate_recovery()) lustre-MDT0000_UUID: starting recovery 00000100:00080000:0.0:1776178430.827864:0:37:0:(import.c:86:import_set_state_nolock()) 00000000a7c55b1f lustre-MDT0000_UUID: changing import state from DISCONN to CONNECTING 00000100:00080000:0.0:1776178430.827867:0:37:0:(import.c:534:import_select_connection()) lustre-MDT0000-lwp-MDT0001: connect to NID 0@lo last attempt 1133 00000100:00080000:0.0:1776178430.827868:0:37:0:(import.c:581:import_select_connection()) lustre-MDT0000-lwp-MDT0001: tried all connections, increasing latency to 21s 00000100:00080000:0.0:1776178430.827871:0:37:0:(import.c:615:import_select_connection()) lustre-MDT0000-lwp-MDT0001: import 00000000a7c55b1f using connection 192.168.204.103@tcp/0@lo 00000100:00100000:0.0:1776178430.827881:0:37:0:(import.c:807:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5) req@000000006b9d8ccb x1862456998081024/t0(0) o38->lustre-MDT0000-lwp-MDT0001@0@lo:12/10 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' 00000100:00100000:0.0:1776178430.827911:0:3355:0:(client.c:1733:ptlrpc_send_new_req()) Sending RPC req@0000000047a42b33 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_02:lustre-MDT0001-lwp-OST0001_UUID:3355:1862456998080768:0@lo:400:kworker/u8:2.0 00000100:00100000:0.0:1776178430.827960:0:3355:0:(events.c:361:request_in_callback()) peer: 12345-0@lo (source: 12345-0@lo) 00000100:00100000:0.0:1776178430.827988:0:3355:0:(ptlrpcd.c:415:ptlrpcd_check()) transfer 1 async RPCs [3->2] 00000100:00100000:0.0:1776178430.827993:0:3355:0:(client.c:1733:ptlrpc_send_new_req()) Sending RPC req@00000000a49c1305 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_02:lustre-MDT0001-mdtlov_UUID:3355:1862456998080896:0@lo:400:kworker/u8:2.0 00000100:00100000:0.0:1776178430.828015:0:3355:0:(events.c:361:request_in_callback()) peer: 12345-0@lo (source: 12345-0@lo) 00000100:00100000:0.0:1776178430.828046:0:8250:0:(service.c:2152:ptlrpc_server_handle_req_in()) got req x1862456998080768 00000100:00100000:3.0:1776178430.828220:0:3352:0:(client.c:1733:ptlrpc_send_new_req()) Sending RPC req@000000008c50846d pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:lustre-MDT0000-lwp-OST0000_UUID:3352:1862456998080576:0@lo:38:kworker/u8:2.0 00000100:00100000:3.0:1776178430.828255:0:3352:0:(events.c:361:request_in_callback()) peer: 12345-0@lo (source: 12345-0@lo) 00000100:00100000:3.0:1776178430.828272:0:3352:0:(client.c:1733:ptlrpc_send_new_req()) Sending RPC req@000000007d6ef583 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:lustre-MDT0000-lwp-OST0001_UUID:3352:1862456998080704:0@lo:38:kworker/u8:2.0 00000100:00100000:3.0:1776178430.828295:0:3352:0:(events.c:361:request_in_callback()) peer: 12345-0@lo (source: 12345-0@lo) 00000100:00100000:3.0:1776178430.828310:0:3352:0:(client.c:1733:ptlrpc_send_new_req()) Sending RPC req@00000000803faf52 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:lustre-MDT0001-mdtlov_UUID:3352:1862456998080832:0@lo:38:kworker/u8:2.0 00000100:00100000:3.0:1776178430.828332:0:3352:0:(events.c:361:request_in_callback()) peer: 12345-0@lo (source: 12345-0@lo) 00000100:00100000:3.0:1776178430.828347:0:3352:0:(client.c:1733:ptlrpc_send_new_req()) Sending RPC req@000000006b9d8ccb pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:lustre-MDT0000-lwp-MDT0001_UUID:3352:1862456998081024:0@lo:38:kworker/u8:2.0 00000100:00100000:3.0:1776178430.828367:0:3352:0:(events.c:361:request_in_callback()) peer: 12345-0@lo (source: 12345-0@lo) 00000100:00100000:3.0:1776178430.828415:0:14128:0:(service.c:2152:ptlrpc_server_handle_req_in()) got req x1862456998080832 00000100:00100000:3.0:1776178430.828445:0:14128:0:(nrs_fifo.c:179:nrs_fifo_req_get()) NRS start fifo request from 12345-0@lo, seq: 752 00000100:00100000:3.0:1776178430.828449:0:14128:0:(service.c:2324:ptlrpc_server_handle_request()) Handling RPC req@000000009b5eeec2 pname:cluuid+ref:pid:xid:nid:opc:job mdt_out00_002:0+-99:3352:x1862456998080832:12345-0@lo:38:kworker/u8:2.0 00010000:02000000:3.0:1776178430.828463:0:14128:0:(ldlm_lib.c:1139:target_handle_connect()) lustre-MDT0000: Not available for connect from 0@lo (stopping) 00000100:00100000:3.0:1776178430.828495:0:14128:0:(service.c:2372:ptlrpc_server_handle_request()) Handled RPC req@000000009b5eeec2 pname:cluuid+ref:pid:xid:nid:opc:job mdt_out00_002:0+-99:3352:x1862456998080832:12345-0@lo:38:kworker/u8:2.0 Request processed in 45us (163us total) trans 0 rc -19/-19 00000100:00100000:3.0:1776178430.828510:0:14128:0:(nrs_fifo.c:241:nrs_fifo_req_stop()) NRS stop fifo request from 12345-0@lo, seq: 752 00000100:00100000:3.0:1776178430.828530:0:6257:0:(service.c:2152:ptlrpc_server_handle_req_in()) got req x1862456998080576 00000100:00100000:1.0:1776178430.828902:0:20137:0:(service.c:2372:ptlrpc_server_handle_request()) Handled RPC req@000000008f0263f6 pname:cluuid+ref:pid:xid:nid:opc:job ll_mgs_0000:ac61f839-0e8e-4cac-9c59-08bcfbbad330+18:3353:x1862456998080512:12345-0@lo:400:kworker/u8:2.0 Request processed in 2426us (2626us total) trans 0 rc 0/0 00000100:00100000:1.0:1776178430.828911:0:20137:0:(nrs_fifo.c:241:nrs_fifo_req_stop()) NRS stop fifo request from 12345-0@lo, seq: 244 00000100:00100000:1.0:1776178430.828985:0:20759:0:(service.c:2152:ptlrpc_server_handle_req_in()) got req x1862456998080896 00000100:00100000:1.0:1776178430.829012:0:20759:0:(nrs_fifo.c:179:nrs_fifo_req_get()) NRS start fifo request from 12345-0@lo, seq: 689 00000100:00100000:1.0:1776178430.829017:0:20759:0:(service.c:2324:ptlrpc_server_handle_request()) Handling RPC req@00000000ddf29755 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_010:lustre-MDT0001-mdtlov_UUID+5:3355:x1862456998080896:12345-0@lo:400:kworker/u8:2.0 00000100:00100000:1.0:1776178430.829080:0:20759:0:(service.c:2372:ptlrpc_server_handle_request()) Handled RPC req@00000000ddf29755 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_010:lustre-MDT0001-mdtlov_UUID+5:3355:x1862456998080896:12345-0@lo:400:kworker/u8:2.0 Request processed in 63us (1064us total) trans 0 rc 0/0 00000100:00100000:1.0:1776178430.829087:0:20759:0:(nrs_fifo.c:241:nrs_fifo_req_stop()) NRS stop fifo request from 12345-0@lo, seq: 689 00000100:00100000:1.0:1776178430.829110:0:3354:0:(client.c:1733:ptlrpc_send_new_req()) Sending RPC req@000000001da3f43e pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_01:lustre-MDT0001-lwp-OST0000_UUID:3354:1862456998080640:0@lo:400:kworker/u8:2.0 00000100:00100000:1.0:1776178430.829145:0:3354:0:(events.c:361:request_in_callback()) peer: 12345-0@lo (source: 12345-0@lo) 00000100:00100000:1.0:1776178430.829158:0:3354:0:(ptlrpcd.c:415:ptlrpcd_check()) transfer 1 async RPCs [0->1] 00000100:00100000:1.0:1776178430.829163:0:3354:0:(client.c:1733:ptlrpc_send_new_req()) Sending RPC req@0000000099e8e662 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_01:lustre-MDT0001-mdtlov_UUID:3354:1862456998080960:0@lo:400:kworker/u8:2.0 00000100:00100000:1.0:1776178430.829184:0:3354:0:(events.c:361:request_in_callback()) peer: 12345-0@lo (source: 12345-0@lo) 00000100:00100000:0.0:1776178430.829870:0:8250:0:(nrs_fifo.c:179:nrs_fifo_req_get()) NRS start fifo request from 12345-0@lo, seq: 1061 00000100:00100000:0.0:1776178430.829880:0:8250:0:(service.c:2324:ptlrpc_server_handle_request()) Handling RPC req@0000000050c5b9a3 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_003:lustre-MDT0001-lwp-OST0001_UUID+5:3355:x1862456998080768:12345-0@lo:400:kworker/u8:2.0 00000100:00100000:0.0:1776178430.829930:0:8250:0:(service.c:2372:ptlrpc_server_handle_request()) Handled RPC req@0000000050c5b9a3 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_003:lustre-MDT0001-lwp-OST0001_UUID+5:3355:x1862456998080768:12345-0@lo:400:kworker/u8:2.0 Request processed in 52us (1970us total) trans 0 rc 0/0 00000100:00100000:1.0:1776178430.829962:0:20762:0:(service.c:2152:ptlrpc_server_handle_req_in()) got req x1862456998080960 00000100:00100000:1.0:1776178430.829997:0:20762:0:(nrs_fifo.c:179:nrs_fifo_req_get()) NRS start fifo request from 12345-0@lo, seq: 690 00000100:00100000:1.0:1776178430.830000:0:20762:0:(service.c:2324:ptlrpc_server_handle_request()) Handling RPC req@000000001f0b1867 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_013:lustre-MDT0001-mdtlov_UUID+5:3354:x1862456998080960:12345-0@lo:400:kworker/u8:2.0 00000100:00100000:1.0:1776178430.830034:0:20762:0:(service.c:2372:ptlrpc_server_handle_request()) Handled RPC req@000000001f0b1867 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_013:lustre-MDT0001-mdtlov_UUID+5:3354:x1862456998080960:12345-0@lo:400:kworker/u8:2.0 Request processed in 33us (850us total) trans 0 rc 0/0 00000100:00100000:1.0:1776178430.830039:0:20762:0:(nrs_fifo.c:241:nrs_fifo_req_stop()) NRS stop fifo request from 12345-0@lo, seq: 690 00000100:00100000:1.0:1776178430.830104:0:3354:0:(client.c:2215:ptlrpc_check_set()) Completed RPC req@0000000099e8e662 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_01:lustre-MDT0001-mdtlov_UUID:3354:1862456998080960:0@lo:400:kworker/u8:2.0 00000100:00100000:1.0:1776178430.830164:0:6255:0:(service.c:2152:ptlrpc_server_handle_req_in()) got req x1862456998080704 00000100:00100000:3.0:1776178430.837819:0:6257:0:(service.c:2152:ptlrpc_server_handle_req_in()) got req x1862456998081024 00000100:00100000:3.0:1776178430.837857:0:6257:0:(service.c:2152:ptlrpc_server_handle_req_in()) got req x1862456998080640 00000100:00100000:3.0:1776178430.837884:0:6257:0:(nrs_fifo.c:179:nrs_fifo_req_get()) NRS start fifo request from 12345-0@lo, seq: 1062 00000100:00100000:3.0:1776178430.837891:0:6257:0:(service.c:2324:ptlrpc_server_handle_request()) Handling RPC req@00000000bfb13eb6 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_002:lustre-MDT0001-lwp-OST0000_UUID+5:3354:x1862456998080640:12345-0@lo:400:kworker/u8:2.0 00000100:00100000:3.0:1776178430.838035:0:6257:0:(service.c:2372:ptlrpc_server_handle_request()) Handled RPC req@00000000bfb13eb6 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_002:lustre-MDT0001-lwp-OST0000_UUID+5:3354:x1862456998080640:12345-0@lo:400:kworker/u8:2.0 Request processed in 143us (8889us total) trans 0 rc 0/0 00000100:00100000:3.0:1776178430.838042:0:6257:0:(nrs_fifo.c:241:nrs_fifo_req_stop()) NRS stop fifo request from 12345-0@lo, seq: 1062 00000100:00100000:3.0:1776178430.838078:0:6257:0:(nrs_fifo.c:179:nrs_fifo_req_get()) NRS start fifo request from 12345-0@lo, seq: 6801 00000100:00100000:3.0:1776178430.838083:0:6257:0:(service.c:2324:ptlrpc_server_handle_request()) Handling RPC req@000000006d540857 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_002:0+-99:3352:x1862456998080576:12345-0@lo:38:kworker/u8:2.0 00010000:02000000:3.0:1776178430.838100:0:6257:0:(ldlm_lib.c:1139:target_handle_connect()) lustre-MDT0000: Not available for connect from 0@lo (stopping) 00000100:00100000:3.0:1776178430.838143:0:6257:0:(service.c:2372:ptlrpc_server_handle_request()) Handled RPC req@000000006d540857 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_002:0+-99:3352:x1862456998080576:12345-0@lo:38:kworker/u8:2.0 Request processed in 59us (9888us total) trans 0 rc -19/-19 00000100:00100000:3.0:1776178430.838149:0:6257:0:(nrs_fifo.c:241:nrs_fifo_req_stop()) NRS stop fifo request from 12345-0@lo, seq: 6801 00000100:00100000:3.0:1776178430.838160:0:6257:0:(nrs_fifo.c:179:nrs_fifo_req_get()) NRS start fifo request from 12345-0@lo, seq: 6802 00000100:00100000:3.0:1776178430.838163:0:6257:0:(service.c:2324:ptlrpc_server_handle_request()) Handling RPC req@000000008c469ec0 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_002:0+-99:3352:x1862456998081024:12345-0@lo:38:kworker/u8:2.0 00010000:02000000:3.0:1776178430.838169:0:6257:0:(ldlm_lib.c:1139:target_handle_connect()) lustre-MDT0000: Not available for connect from 0@lo (stopping) 00000100:00100000:3.0:1776178430.838191:0:6257:0:(service.c:2372:ptlrpc_server_handle_request()) Handled RPC req@000000008c469ec0 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_002:0+-99:3352:x1862456998081024:12345-0@lo:38:kworker/u8:2.0 Request processed in 27us (9823us total) trans 0 rc -19/-19 00000100:00100000:3.0:1776178430.838195:0:6257:0:(nrs_fifo.c:241:nrs_fifo_req_stop()) NRS stop fifo request from 12345-0@lo, seq: 6802 00000100:00100000:2.0:1776178430.842857:0:3353:0:(client.c:2215:ptlrpc_check_set()) Completed RPC req@00000000f351bcc3 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_00:ac61f839-0e8e-4cac-9c59-08bcfbbad330:3353:1862456998080512:0@lo:400:kworker/u8:2.0 00000100:00100000:0.0:1776178430.842962:0:8250:0:(nrs_fifo.c:241:nrs_fifo_req_stop()) NRS stop fifo request from 12345-0@lo, seq: 1061 00000100:00100000:0.0:1776178430.843024:0:3355:0:(client.c:2215:ptlrpc_check_set()) Completed RPC req@00000000a49c1305 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_02:lustre-MDT0001-mdtlov_UUID:3355:1862456998080896:0@lo:400:kworker/u8:2.0 00000100:00100000:0.0:1776178430.843036:0:3355:0:(client.c:2215:ptlrpc_check_set()) Completed RPC req@0000000047a42b33 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_02:lustre-MDT0001-lwp-OST0001_UUID:3355:1862456998080768:0@lo:400:kworker/u8:2.0 00000100:00100000:1.0:1776178430.843874:0:6255:0:(nrs_fifo.c:179:nrs_fifo_req_get()) NRS start fifo request from 12345-0@lo, seq: 6803 00000100:00100000:1.0:1776178430.843885:0:6255:0:(service.c:2324:ptlrpc_server_handle_request()) Handling RPC req@0000000008b3a3ad pname:cluuid+ref:pid:xid:nid:opc:job mdt00_000:0+-99:3352:x1862456998080704:12345-0@lo:38:kworker/u8:2.0 00010000:02000000:1.0:1776178430.843904:0:6255:0:(ldlm_lib.c:1139:target_handle_connect()) lustre-MDT0000: Not available for connect from 0@lo (stopping) 00000100:00100000:1.0:1776178430.844000:0:6255:0:(service.c:2372:ptlrpc_server_handle_request()) Handled RPC req@0000000008b3a3ad pname:cluuid+ref:pid:xid:nid:opc:job mdt00_000:0+-99:3352:x1862456998080704:12345-0@lo:38:kworker/u8:2.0 Request processed in 114us (15703us total) trans 0 rc -19/-19 00000100:00100000:1.0:1776178430.844008:0:6255:0:(nrs_fifo.c:241:nrs_fifo_req_stop()) NRS stop fifo request from 12345-0@lo, seq: 6803 00000100:00080000:2.0:1776178430.851978:0:3352:0:(import.c:86:import_set_state_nolock()) 0000000047601a98 lustre-MDT0000_UUID: changing import state from CONNECTING to DISCONN 00000100:00080000:2.0:1776178430.851989:0:3352:0:(import.c:1429:ptlrpc_connect_interpret()) recovery of lustre-MDT0000_UUID on 192.168.204.103@tcp failed (-19) 00000100:00100000:2.0:1776178430.851997:0:3352:0:(client.c:2215:ptlrpc_check_set()) Completed RPC req@000000008c50846d pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:lustre-MDT0000-lwp-OST0000_UUID:3352:1862456998080576:0@lo:38:kworker/u8:2.0 00000100:00080000:2.0:1776178430.852011:0:3352:0:(import.c:86:import_set_state_nolock()) 00000000dfc8db13 lustre-MDT0000_UUID: changing import state from CONNECTING to DISCONN 00000100:00080000:2.0:1776178430.852014:0:3352:0:(import.c:1429:ptlrpc_connect_interpret()) recovery of lustre-MDT0000_UUID on 192.168.204.103@tcp failed (-19) 00000100:00100000:2.0:1776178430.852016:0:3352:0:(client.c:2215:ptlrpc_check_set()) Completed RPC req@000000007d6ef583 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:lustre-MDT0000-lwp-OST0001_UUID:3352:1862456998080704:0@lo:38:kworker/u8:2.0 00000100:00080000:2.0:1776178430.852026:0:3352:0:(import.c:86:import_set_state_nolock()) 00000000ba4f1821 lustre-MDT0000_UUID: changing import state from CONNECTING to DISCONN 00000100:00080000:2.0:1776178430.852029:0:3352:0:(import.c:1429:ptlrpc_connect_interpret()) recovery of lustre-MDT0000_UUID on 192.168.204.103@tcp failed (-19) 00000100:00100000:2.0:1776178430.852031:0:3352:0:(client.c:2215:ptlrpc_check_set()) Completed RPC req@00000000803faf52 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:lustre-MDT0001-mdtlov_UUID:3352:1862456998080832:0@lo:38:kworker/u8:2.0 00000100:00080000:2.0:1776178430.852042:0:3352:0:(import.c:86:import_set_state_nolock()) 00000000a7c55b1f lustre-MDT0000_UUID: changing import state from CONNECTING to DISCONN 00000100:00080000:2.0:1776178430.852044:0:3352:0:(import.c:1429:ptlrpc_connect_interpret()) recovery of lustre-MDT0000_UUID on 192.168.204.103@tcp failed (-19) 00000100:00100000:2.0:1776178430.852077:0:3352:0:(client.c:2215:ptlrpc_check_set()) Completed RPC req@000000006b9d8ccb pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:lustre-MDT0000-lwp-MDT0001_UUID:3352:1862456998081024:0@lo:38:kworker/u8:2.0 00000100:00100000:2.0:1776178430.852156:0:3354:0:(client.c:2215:ptlrpc_check_set()) Completed RPC req@000000001da3f43e pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_01:lustre-MDT0001-lwp-OST0000_UUID:3354:1862456998080640:0@lo:400:kworker/u8:2.0 00010000:00010000:2.0:1776178431.456458:0:6245:0:(ldlm_lockd.c:2792:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:3.0:1776178432.352561:0:3353:0:(ptlrpcd.c:415:ptlrpcd_check()) transfer 1 async RPCs [1->0] 00000100:00100000:3.0:1776178432.352595:0:3353:0:(client.c:1733:ptlrpc_send_new_req()) Sending RPC req@00000000500c0ae6 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_00:lustre-MDT0001-mdtlov_UUID:3353:1862456998081088:0@lo:13:osp-pre-1-1.0 00000100:00100000:3.0:1776178432.352707:0:3353:0:(events.c:361:request_in_callback()) peer: 12345-0@lo (source: 12345-0@lo) 00000100:00100000:0.0:1776178432.352818:0:9554:0:(service.c:2152:ptlrpc_server_handle_req_in()) got req x1862456998081088 00000100:00100000:0.0:1776178432.352874:0:9554:0:(nrs_fifo.c:179:nrs_fifo_req_get()) NRS start fifo request from 12345-0@lo, seq: 577 00000100:00100000:0.0:1776178432.352879:0:9554:0:(service.c:2324:ptlrpc_server_handle_request()) Handling RPC req@00000000ddc1dc02 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_create00:lustre-MDT0001-mdtlov_UUID+5:3353:x1862456998081088:12345-0@lo:13:osp-pre-1-1.0 00002000:00000024:0.0:1776178432.352912:0:9554:0:(ofd_obd.c:740:ofd_statfs()) blocks cached 229376 granted 491746368 pending 0 free 3923689472 avail 3692167168 00000100:00100000:0.0:1776178432.352967:0:9554:0:(service.c:2372:ptlrpc_server_handle_request()) Handled RPC req@00000000ddc1dc02 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_create00:lustre-MDT0001-mdtlov_UUID+5:3353:x1862456998081088:12345-0@lo:13:osp-pre-1-1.0 Request processed in 86us (261us total) trans 0 rc 0/0 00000100:00100000:0.0:1776178432.352974:0:9554:0:(nrs_fifo.c:241:nrs_fifo_req_stop()) NRS stop fifo request from 12345-0@lo, seq: 577 00000100:00100000:2.0:1776178432.353150:0:3353:0:(client.c:2215:ptlrpc_check_set()) Completed RPC req@00000000500c0ae6 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_00:lustre-MDT0001-mdtlov_UUID:3353:1862456998081088:0@lo:13:osp-pre-1-1.0 00010000:00010000:2.0:1776178432.480354:0:9224:0:(ldlm_lockd.c:2792:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:3.0:1776178432.865013:0:3356:0:(ptlrpcd.c:415:ptlrpcd_check()) transfer 1 async RPCs [2->3] 00000100:00100000:3.0:1776178432.865035:0:3356:0:(client.c:1733:ptlrpc_send_new_req()) Sending RPC req@00000000a49c1305 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_03:lustre-MDT0001-mdtlov_UUID:3356:1862456998081152:0@lo:13:osp-pre-0-1.0 00000100:00100000:3.0:1776178432.865122:0:3356:0:(events.c:361:request_in_callback()) peer: 12345-0@lo (source: 12345-0@lo) 00000100:00100000:0.0:1776178432.879649:0:8231:0:(service.c:2152:ptlrpc_server_handle_req_in()) got req x1862456998081152 00000100:00100000:0.0:1776178432.879734:0:8231:0:(nrs_fifo.c:179:nrs_fifo_req_get()) NRS start fifo request from 12345-0@lo, seq: 578 00000100:00100000:0.0:1776178432.879740:0:8231:0:(service.c:2324:ptlrpc_server_handle_request()) Handling RPC req@000000009c55a259 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_create00:lustre-MDT0001-mdtlov_UUID+5:3356:x1862456998081152:12345-0@lo:13:osp-pre-0-1.0 00002000:00000024:0.0:1776178432.879777:0:8231:0:(ofd_obd.c:740:ofd_statfs()) blocks cached 315392 granted 484883776 pending 0 free 3923689472 avail 3692167168 00000100:00100000:0.0:1776178432.879862:0:8231:0:(service.c:2372:ptlrpc_server_handle_request()) Handled RPC req@000000009c55a259 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_create00:lustre-MDT0001-mdtlov_UUID+5:3356:x1862456998081152:12345-0@lo:13:osp-pre-0-1.0 Request processed in 121us (14741us total) trans 0 rc 0/0 00000100:00100000:0.0:1776178432.879874:0:8231:0:(nrs_fifo.c:241:nrs_fifo_req_stop()) NRS stop fifo request from 12345-0@lo, seq: 578 00000100:00100000:3.0:1776178432.881455:0:3356:0:(client.c:2215:ptlrpc_check_set()) Completed RPC req@00000000a49c1305 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_03:lustre-MDT0001-mdtlov_UUID:3356:1862456998081152:0@lo:13:osp-pre-0-1.0 00010000:00010000:1.0:1776178433.506179:0:9225:0:(ldlm_lockd.c:2792:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:1.0:1776178434.340761:0:3348:0:(events.c:361:request_in_callback()) peer: 12345-192.168.204.3@tcp (source: 12345-192.168.204.3@tcp) 00000100:00100000:1.0:1776178434.340884:0:3348:0:(events.c:361:request_in_callback()) peer: 12345-192.168.204.3@tcp (source: 12345-192.168.204.3@tcp) 00000100:00100000:1.0:1776178434.340923:0:3348:0:(events.c:361:request_in_callback()) peer: 12345-192.168.204.3@tcp (source: 12345-192.168.204.3@tcp) 00000100:00100000:1.0:1776178434.340958:0:3348:0:(events.c:361:request_in_callback()) peer: 12345-192.168.204.3@tcp (source: 12345-192.168.204.3@tcp) 00000100:00100000:2.0:1776178434.341230:0:20759:0:(service.c:2152:ptlrpc_server_handle_req_in()) got req x1862456979123584 00000100:00100000:2.0:1776178434.341293:0:20759:0:(service.c:2152:ptlrpc_server_handle_req_in()) got req x1862456979123648 00000100:00100000:2.0:1776178434.341339:0:20759:0:(nrs_fifo.c:179:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.204.3@tcp, seq: 691 00000100:00100000:2.0:1776178434.341347:0:20759:0:(service.c:2324:ptlrpc_server_handle_request()) Handling RPC req@000000007886b426 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_010:b50963bd-8eba-4432-9a54-8888d00e3fa1+8:2271:x1862456979123584:12345-192.168.204.3@tcp:400:kworker/u8:3.0 00000100:00100000:3.0:1776178434.341580:0:11281:0:(service.c:2152:ptlrpc_server_handle_req_in()) got req x1862456979123520 00000100:00100000:3.0:1776178434.341632:0:11281:0:(nrs_fifo.c:179:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.204.3@tcp, seq: 1063 00000100:00100000:3.0:1776178434.341637:0:11281:0:(service.c:2324:ptlrpc_server_handle_request()) Handling RPC req@000000002b69594e pname:cluuid+ref:pid:xid:nid:opc:job mdt00_004:b50963bd-8eba-4432-9a54-8888d00e3fa1+6:2271:x1862456979123520:12345-192.168.204.3@tcp:400:kworker/u8:3.0 00000100:00100000:0.0:1776178434.341774:0:20137:0:(service.c:2152:ptlrpc_server_handle_req_in()) got req x1862456979123392 00000100:00100000:0.0:1776178434.342103:0:20137:0:(nrs_fifo.c:179:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.204.3@tcp, seq: 245 00000100:00100000:0.0:1776178434.342109:0:20137:0:(service.c:2324:ptlrpc_server_handle_request()) Handling RPC req@000000000084a3e4 pname:cluuid+ref:pid:xid:nid:opc:job ll_mgs_0000:5b5c66b1-1a2f-4651-8548-0c75b9a38a94+9:2271:x1862456979123392:12345-192.168.204.3@tcp:400:kworker/u8:3.0 00000100:00100000:2.0:1776178434.342182:0:20759:0:(service.c:2372:ptlrpc_server_handle_request()) Handled RPC req@000000007886b426 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_010:b50963bd-8eba-4432-9a54-8888d00e3fa1+8:2271:x1862456979123584:12345-192.168.204.3@tcp:400:kworker/u8:3.0 Request processed in 835us (1298us total) trans 0 rc 0/0 00000100:00100000:2.0:1776178434.342190:0:20759:0:(nrs_fifo.c:241:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.204.3@tcp, seq: 691 00000100:00100000:2.0:1776178434.342208:0:20759:0:(nrs_fifo.c:179:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.204.3@tcp, seq: 692 00000100:00100000:2.0:1776178434.342212:0:20759:0:(service.c:2324:ptlrpc_server_handle_request()) Handling RPC req@0000000013603266 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_010:b50963bd-8eba-4432-9a54-8888d00e3fa1+9:2271:x1862456979123648:12345-192.168.204.3@tcp:400:kworker/u8:3.0 00000100:00100000:3.0:1776178434.342698:0:11281:0:(service.c:2372:ptlrpc_server_handle_request()) Handled RPC req@000000002b69594e pname:cluuid+ref:pid:xid:nid:opc:job mdt00_004:b50963bd-8eba-4432-9a54-8888d00e3fa1+6:2271:x1862456979123520:12345-192.168.204.3@tcp:400:kworker/u8:3.0 Request processed in 1058us (1774us total) trans 0 rc 0/0 00000100:00100000:3.0:1776178434.342705:0:11281:0:(nrs_fifo.c:241:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.204.3@tcp, seq: 1063 00000100:00100000:3.0:1776178434.342753:0:3348:0:(events.c:361:request_in_callback()) peer: 12345-192.168.204.3@tcp (source: 12345-192.168.204.3@tcp) 00000100:00100000:3.0:1776178434.342836:0:11281:0:(service.c:2152:ptlrpc_server_handle_req_in()) got req x1862456979123456 00000100:00100000:3.0:1776178434.342869:0:11281:0:(nrs_fifo.c:179:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.204.3@tcp, seq: 6804 00000100:00100000:3.0:1776178434.342874:0:11281:0:(service.c:2324:ptlrpc_server_handle_request()) Handling RPC req@000000004e8fbd9b pname:cluuid+ref:pid:xid:nid:opc:job mdt00_004:0+-99:2269:x1862456979123456:12345-192.168.204.3@tcp:38:kworker/u8:3.0 00010000:02000000:3.0:1776178434.342892:0:11281:0:(ldlm_lib.c:1139:target_handle_connect()) lustre-MDT0000: Not available for connect from 192.168.204.3@tcp (stopping) 00000100:00100000:0.0:1776178434.342902:0:20137:0:(service.c:2372:ptlrpc_server_handle_request()) Handled RPC req@000000000084a3e4 pname:cluuid+ref:pid:xid:nid:opc:job ll_mgs_0000:5b5c66b1-1a2f-4651-8548-0c75b9a38a94+9:2271:x1862456979123392:12345-192.168.204.3@tcp:400:kworker/u8:3.0 Request processed in 790us (2148us total) trans 0 rc 0/0 00000100:00100000:0.0:1776178434.342912:0:20137:0:(nrs_fifo.c:241:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.204.3@tcp, seq: 245 00000100:00100000:3.0:1776178434.342923:0:11281:0:(service.c:2372:ptlrpc_server_handle_request()) Handled RPC req@000000004e8fbd9b pname:cluuid+ref:pid:xid:nid:opc:job mdt00_004:0+-99:2269:x1862456979123456:12345-192.168.204.3@tcp:38:kworker/u8:3.0 Request processed in 46us (169us total) trans 0 rc -19/-19 00000100:00100000:3.0:1776178434.342931:0:11281:0:(nrs_fifo.c:241:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.204.3@tcp, seq: 6804 00000100:00100000:2.0:1776178434.343055:0:20759:0:(service.c:2372:ptlrpc_server_handle_request()) Handled RPC req@0000000013603266 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_010:b50963bd-8eba-4432-9a54-8888d00e3fa1+9:2271:x1862456979123648:12345-192.168.204.3@tcp:400:kworker/u8:3.0 Request processed in 842us (2097us total) trans 0 rc 0/0 00000100:00100000:2.0:1776178434.343079:0:20759:0:(nrs_fifo.c:241:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.204.3@tcp, seq: 692 00010000:00010000:3.0:1776178434.530810:0:6244:0:(ldlm_lockd.c:2792:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00010000:00010000:2.0:1776178435.553730:0:6245:0:(ldlm_lockd.c:2792:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00080000:2.0:1776178435.939816:0:36:0:(pinger.c:251:ptlrpc_pinger_process_import()) lustre-MDT0000-lwp-OST0000_UUID->lustre-MDT0000_UUID: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00080000:2.0:1776178435.939824:0:36:0:(recover.c:57:ptlrpc_initiate_recovery()) lustre-MDT0000_UUID: starting recovery 00000100:00080000:2.0:1776178435.939828:0:36:0:(import.c:86:import_set_state_nolock()) 0000000047601a98 lustre-MDT0000_UUID: changing import state from DISCONN to CONNECTING 00000100:00080000:2.0:1776178435.939838:0:36:0:(import.c:534:import_select_connection()) lustre-MDT0000-lwp-OST0000: connect to NID 0@lo last attempt 1138 00000100:00080000:2.0:1776178435.939842:0:36:0:(import.c:581:import_select_connection()) lustre-MDT0000-lwp-OST0000: tried all connections, increasing latency to 21s 00000100:00080000:2.0:1776178435.939850:0:36:0:(import.c:615:import_select_connection()) lustre-MDT0000-lwp-OST0000: import 0000000047601a98 using connection 192.168.204.103@tcp/0@lo 00000100:00100000:2.0:1776178435.939875:0:36:0:(import.c:807:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5) req@000000007cfde41a x1862456998081280/t0(0) o38->lustre-MDT0000-lwp-OST0000@0@lo:12/10 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' 00000100:00080000:2.0:1776178435.939924:0:36:0:(pinger.c:251:ptlrpc_pinger_process_import()) lustre-MDT0000-lwp-OST0001_UUID->lustre-MDT0000_UUID: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00080000:2.0:1776178435.939928:0:36:0:(recover.c:57:ptlrpc_initiate_recovery()) lustre-MDT0000_UUID: starting recovery 00000100:00080000:2.0:1776178435.939930:0:36:0:(import.c:86:import_set_state_nolock()) 00000000dfc8db13 lustre-MDT0000_UUID: changing import state from DISCONN to CONNECTING 00000100:00080000:2.0:1776178435.939933:0:36:0:(import.c:534:import_select_connection()) lustre-MDT0000-lwp-OST0001: connect to NID 0@lo last attempt 1138 00000100:00080000:2.0:1776178435.939935:0:36:0:(import.c:581:import_select_connection()) lustre-MDT0000-lwp-OST0001: tried all connections, increasing latency to 21s 00000100:00080000:2.0:1776178435.939939:0:36:0:(import.c:615:import_select_connection()) lustre-MDT0000-lwp-OST0001: import 00000000dfc8db13 using connection 192.168.204.103@tcp/0@lo 00000100:00100000:2.0:1776178435.939952:0:36:0:(import.c:807:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5) req@0000000071333ecf x1862456998081408/t0(0) o38->lustre-MDT0000-lwp-OST0001@0@lo:12/10 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' 00000100:00080000:2.0:1776178435.939969:0:36:0:(pinger.c:251:ptlrpc_pinger_process_import()) lustre-MDT0001-mdtlov_UUID->lustre-MDT0000_UUID: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00080000:2.0:1776178435.939972:0:36:0:(recover.c:57:ptlrpc_initiate_recovery()) lustre-MDT0000_UUID: starting recovery 00000100:00080000:2.0:1776178435.939973:0:36:0:(import.c:86:import_set_state_nolock()) 00000000ba4f1821 lustre-MDT0000_UUID: changing import state from DISCONN to CONNECTING 00000100:00080000:2.0:1776178435.939976:0:36:0:(import.c:534:import_select_connection()) lustre-MDT0000-osp-MDT0001: connect to NID 0@lo last attempt 1138 00000100:00080000:2.0:1776178435.939978:0:36:0:(import.c:581:import_select_connection()) lustre-MDT0000-osp-MDT0001: tried all connections, increasing latency to 21s 00000100:00080000:2.0:1776178435.939981:0:36:0:(import.c:615:import_select_connection()) lustre-MDT0000-osp-MDT0001: import 00000000ba4f1821 using connection 192.168.204.103@tcp/0@lo 00000100:00100000:2.0:1776178435.939991:0:36:0:(import.c:807:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5) req@00000000d1e0f01c x1862456998081536/t0(0) o38->lustre-MDT0000-osp-MDT0001@0@lo:24/4 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' 00000100:00080000:2.0:1776178435.940010:0:36:0:(pinger.c:251:ptlrpc_pinger_process_import()) lustre-MDT0000-lwp-MDT0001_UUID->lustre-MDT0000_UUID: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00080000:2.0:1776178435.940013:0:36:0:(recover.c:57:ptlrpc_initiate_recovery()) lustre-MDT0000_UUID: starting recovery 00000100:00080000:2.0:1776178435.940014:0:36:0:(import.c:86:import_set_state_nolock()) 00000000a7c55b1f lustre-MDT0000_UUID: changing import state from DISCONN to CONNECTING 00000100:00080000:2.0:1776178435.940017:0:36:0:(import.c:534:import_select_connection()) lustre-MDT0000-lwp-MDT0001: connect to NID 0@lo last attempt 1138 00000100:00080000:2.0:1776178435.940019:0:36:0:(import.c:581:import_select_connection()) lustre-MDT0000-lwp-MDT0001: tried all connections, increasing latency to 21s 00000100:00080000:2.0:1776178435.940021:0:36:0:(import.c:615:import_select_connection()) lustre-MDT0000-lwp-MDT0001: import 00000000a7c55b1f using connection 192.168.204.103@tcp/0@lo 00000100:00100000:2.0:1776178435.940033:0:36:0:(import.c:807:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5) req@000000006e2391ef x1862456998081728/t0(0) o38->lustre-MDT0000-lwp-MDT0001@0@lo:12/10 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' 00000100:00100000:2.0:1776178435.940092:0:3353:0:(client.c:1733:ptlrpc_send_new_req()) Sending RPC req@000000007e1b5a24 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_00:lustre-MDT0001-lwp-OST0000_UUID:3353:1862456998081344:0@lo:400:kworker/u8:1.0 00000100:00100000:2.0:1776178435.940170:0:3353:0:(events.c:361:request_in_callback()) peer: 12345-0@lo (source: 12345-0@lo) 00000100:00100000:2.0:1776178435.940206:0:3353:0:(ptlrpcd.c:415:ptlrpcd_check()) transfer 1 async RPCs [1->0] 00000100:00100000:2.0:1776178435.940211:0:3353:0:(client.c:1733:ptlrpc_send_new_req()) Sending RPC req@00000000597c3414 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_00:lustre-MDT0001-lwp-OST0001_UUID:3353:1862456998081472:0@lo:400:kworker/u8:1.0 00000100:00100000:2.0:1776178435.940231:0:3353:0:(events.c:361:request_in_callback()) peer: 12345-0@lo (source: 12345-0@lo) 00000100:00100000:0.0:1776178435.940319:0:3352:0:(client.c:1733:ptlrpc_send_new_req()) Sending RPC req@000000007cfde41a pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:lustre-MDT0000-lwp-OST0000_UUID:3352:1862456998081280:0@lo:38:kworker/u8:1.0 00000100:00100000:0.0:1776178435.940350:0:3352:0:(events.c:361:request_in_callback()) peer: 12345-0@lo (source: 12345-0@lo) 00000100:00100000:0.0:1776178435.940375:0:3352:0:(client.c:1733:ptlrpc_send_new_req()) Sending RPC req@0000000071333ecf pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:lustre-MDT0000-lwp-OST0001_UUID:3352:1862456998081408:0@lo:38:kworker/u8:1.0 00000100:00100000:0.0:1776178435.940396:0:3352:0:(events.c:361:request_in_callback()) peer: 12345-0@lo (source: 12345-0@lo) 00000100:00100000:0.0:1776178435.940410:0:3352:0:(client.c:1733:ptlrpc_send_new_req()) Sending RPC req@00000000d1e0f01c pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:lustre-MDT0001-mdtlov_UUID:3352:1862456998081536:0@lo:38:kworker/u8:1.0 00000100:00100000:0.0:1776178435.940433:0:3352:0:(events.c:361:request_in_callback()) peer: 12345-0@lo (source: 12345-0@lo) 00000100:00100000:0.0:1776178435.940445:0:3352:0:(client.c:1733:ptlrpc_send_new_req()) Sending RPC req@000000006e2391ef pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:lustre-MDT0000-lwp-MDT0001_UUID:3352:1862456998081728:0@lo:38:kworker/u8:1.0 00000100:00100000:0.0:1776178435.940464:0:3352:0:(events.c:361:request_in_callback()) peer: 12345-0@lo (source: 12345-0@lo) 00000100:00100000:0.0:1776178435.940498:0:8250:0:(service.c:2152:ptlrpc_server_handle_req_in()) got req x1862456998081344 00000100:00100000:0.0:1776178435.940534:0:8250:0:(service.c:2152:ptlrpc_server_handle_req_in()) got req x1862456998081472 00000100:00100000:0.0:1776178435.940543:0:8250:0:(service.c:2152:ptlrpc_server_handle_req_in()) got req x1862456998081280 00000100:00100000:0.0:1776178435.940552:0:8250:0:(service.c:2152:ptlrpc_server_handle_req_in()) got req x1862456998081408 00000100:00100000:0.0:1776178435.940560:0:8250:0:(service.c:2152:ptlrpc_server_handle_req_in()) got req x1862456998081728 00000100:00100000:0.0:1776178435.940571:0:8250:0:(nrs_fifo.c:179:nrs_fifo_req_get()) NRS start fifo request from 12345-0@lo, seq: 1064 00000100:00100000:0.0:1776178435.940576:0:8250:0:(service.c:2324:ptlrpc_server_handle_request()) Handling RPC req@00000000aba13576 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_003:lustre-MDT0001-lwp-OST0000_UUID+5:3353:x1862456998081344:12345-0@lo:400:kworker/u8:1.0 00000100:00100000:0.0:1776178435.940625:0:8250:0:(service.c:2372:ptlrpc_server_handle_request()) Handled RPC req@00000000aba13576 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_003:lustre-MDT0001-lwp-OST0000_UUID+5:3353:x1862456998081344:12345-0@lo:400:kworker/u8:1.0 Request processed in 48us (457us total) trans 0 rc 0/0 00000100:00100000:0.0:1776178435.940631:0:8250:0:(nrs_fifo.c:241:nrs_fifo_req_stop()) NRS stop fifo request from 12345-0@lo, seq: 1064 00000100:00100000:0.0:1776178435.940648:0:8250:0:(nrs_fifo.c:179:nrs_fifo_req_get()) NRS start fifo request from 12345-0@lo, seq: 1065 00000100:00100000:0.0:1776178435.940652:0:8250:0:(service.c:2324:ptlrpc_server_handle_request()) Handling RPC req@000000001da3f43e pname:cluuid+ref:pid:xid:nid:opc:job mdt00_003:lustre-MDT0001-lwp-OST0001_UUID+5:3353:x1862456998081472:12345-0@lo:400:kworker/u8:1.0 00000100:00100000:0.0:1776178435.940675:0:8250:0:(service.c:2372:ptlrpc_server_handle_request()) Handled RPC req@000000001da3f43e pname:cluuid+ref:pid:xid:nid:opc:job mdt00_003:lustre-MDT0001-lwp-OST0001_UUID+5:3353:x1862456998081472:12345-0@lo:400:kworker/u8:1.0 Request processed in 22us (444us total) trans 0 rc 0/0 00000100:00100000:0.0:1776178435.940680:0:8250:0:(nrs_fifo.c:241:nrs_fifo_req_stop()) NRS stop fifo request from 12345-0@lo, seq: 1065 00000100:00100000:0.0:1776178435.940690:0:8250:0:(nrs_fifo.c:179:nrs_fifo_req_get()) NRS start fifo request from 12345-0@lo, seq: 6805 00000100:00100000:0.0:1776178435.940694:0:8250:0:(service.c:2324:ptlrpc_server_handle_request()) Handling RPC req@000000006344f537 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_003:0+-99:3352:x1862456998081280:12345-0@lo:38:kworker/u8:1.0 00010000:02000000:0.0:1776178435.940705:0:8250:0:(ldlm_lib.c:1139:target_handle_connect()) lustre-MDT0000: Not available for connect from 0@lo (stopping) 00000100:00100000:0.0:1776178435.940732:0:8250:0:(service.c:2372:ptlrpc_server_handle_request()) Handled RPC req@000000006344f537 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_003:0+-99:3352:x1862456998081280:12345-0@lo:38:kworker/u8:1.0 Request processed in 37us (382us total) trans 0 rc -19/-19 00000100:00100000:0.0:1776178435.940738:0:8250:0:(nrs_fifo.c:241:nrs_fifo_req_stop()) NRS stop fifo request from 12345-0@lo, seq: 6805 00000100:00100000:0.0:1776178435.940747:0:8250:0:(nrs_fifo.c:179:nrs_fifo_req_get()) NRS start fifo request from 12345-0@lo, seq: 6806 00000100:00100000:0.0:1776178435.940750:0:8250:0:(service.c:2324:ptlrpc_server_handle_request()) Handling RPC req@00000000a49c1305 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_003:0+-99:3352:x1862456998081408:12345-0@lo:38:kworker/u8:1.0 00010000:02000000:0.0:1776178435.940755:0:8250:0:(ldlm_lib.c:1139:target_handle_connect()) lustre-MDT0000: Not available for connect from 0@lo (stopping) 00000100:00100000:0.0:1776178435.940774:0:8250:0:(service.c:2372:ptlrpc_server_handle_request()) Handled RPC req@00000000a49c1305 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_003:0+-99:3352:x1862456998081408:12345-0@lo:38:kworker/u8:1.0 Request processed in 23us (378us total) trans 0 rc -19/-19 00000100:00100000:0.0:1776178435.940779:0:8250:0:(nrs_fifo.c:241:nrs_fifo_req_stop()) NRS stop fifo request from 12345-0@lo, seq: 6806 00000100:00100000:0.0:1776178435.940787:0:8250:0:(nrs_fifo.c:179:nrs_fifo_req_get()) NRS start fifo request from 12345-0@lo, seq: 6807 00000100:00100000:0.0:1776178435.940790:0:8250:0:(service.c:2324:ptlrpc_server_handle_request()) Handling RPC req@0000000099e8e662 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_003:0+-99:3352:x1862456998081728:12345-0@lo:38:kworker/u8:1.0 00010000:02000000:0.0:1776178435.940795:0:8250:0:(ldlm_lib.c:1139:target_handle_connect()) lustre-MDT0000: Not available for connect from 0@lo (stopping) 00000100:00100000:0.0:1776178435.940814:0:8250:0:(service.c:2372:ptlrpc_server_handle_request()) Handled RPC req@0000000099e8e662 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_003:0+-99:3352:x1862456998081728:12345-0@lo:38:kworker/u8:1.0 Request processed in 22us (349us total) trans 0 rc -19/-19 00000100:00100000:0.0:1776178435.940819:0:8250:0:(nrs_fifo.c:241:nrs_fifo_req_stop()) NRS stop fifo request from 12345-0@lo, seq: 6807 00000100:00100000:0.0:1776178435.940853:0:3353:0:(client.c:2215:ptlrpc_check_set()) Completed RPC req@00000000597c3414 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_00:lustre-MDT0001-lwp-OST0001_UUID:3353:1862456998081472:0@lo:400:kworker/u8:1.0 00000100:00100000:0.0:1776178435.940865:0:3353:0:(client.c:2215:ptlrpc_check_set()) Completed RPC req@000000007e1b5a24 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_00:lustre-MDT0001-lwp-OST0000_UUID:3353:1862456998081344:0@lo:400:kworker/u8:1.0 00000100:00080000:0.0:1776178435.940895:0:3352:0:(import.c:86:import_set_state_nolock()) 0000000047601a98 lustre-MDT0000_UUID: changing import state from CONNECTING to DISCONN 00000100:00080000:0.0:1776178435.940899:0:3352:0:(import.c:1429:ptlrpc_connect_interpret()) recovery of lustre-MDT0000_UUID on 192.168.204.103@tcp failed (-19) 00000100:00100000:0.0:1776178435.940902:0:3352:0:(client.c:2215:ptlrpc_check_set()) Completed RPC req@000000007cfde41a pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:lustre-MDT0000-lwp-OST0000_UUID:3352:1862456998081280:0@lo:38:kworker/u8:1.0 00000100:00080000:0.0:1776178435.940909:0:3352:0:(import.c:86:import_set_state_nolock()) 00000000dfc8db13 lustre-MDT0000_UUID: changing import state from CONNECTING to DISCONN 00000100:00080000:0.0:1776178435.940911:0:3352:0:(import.c:1429:ptlrpc_connect_interpret()) recovery of lustre-MDT0000_UUID on 192.168.204.103@tcp failed (-19) 00000100:00100000:0.0:1776178435.940913:0:3352:0:(client.c:2215:ptlrpc_check_set()) Completed RPC req@0000000071333ecf pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:lustre-MDT0000-lwp-OST0001_UUID:3352:1862456998081408:0@lo:38:kworker/u8:1.0 00000100:00080000:0.0:1776178435.940920:0:3352:0:(import.c:86:import_set_state_nolock()) 00000000a7c55b1f lustre-MDT0000_UUID: changing import state from CONNECTING to DISCONN 00000100:00080000:0.0:1776178435.940922:0:3352:0:(import.c:1429:ptlrpc_connect_interpret()) recovery of lustre-MDT0000_UUID on 192.168.204.103@tcp failed (-19) 00000100:00100000:0.0:1776178435.940924:0:3352:0:(client.c:2215:ptlrpc_check_set()) Completed RPC req@000000006e2391ef pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:lustre-MDT0000-lwp-MDT0001_UUID:3352:1862456998081728:0@lo:38:kworker/u8:1.0 00000100:00100000:1.0:1776178435.941243:0:3355:0:(client.c:1733:ptlrpc_send_new_req()) Sending RPC req@00000000a13f5de6 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_02:lustre-MDT0001-mdtlov_UUID:3355:1862456998081600:0@lo:400:kworker/u8:1.0 00000100:00100000:1.0:1776178435.941292:0:3355:0:(events.c:361:request_in_callback()) peer: 12345-0@lo (source: 12345-0@lo) 00000100:00100000:1.0:1776178435.941322:0:3355:0:(ptlrpcd.c:415:ptlrpcd_check()) transfer 2 async RPCs [3->2] 00000100:00100000:1.0:1776178435.941328:0:3355:0:(client.c:1733:ptlrpc_send_new_req()) Sending RPC req@00000000500c0ae6 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_02:ac61f839-0e8e-4cac-9c59-08bcfbbad330:3355:1862456998081216:0@lo:400:kworker/u8:1.0 00000100:00100000:1.0:1776178435.941354:0:3355:0:(events.c:361:request_in_callback()) peer: 12345-0@lo (source: 12345-0@lo) 00000100:00100000:1.0:1776178435.941366:0:3355:0:(client.c:1733:ptlrpc_send_new_req()) Sending RPC req@000000009220b2b0 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_02:lustre-MDT0001-mdtlov_UUID:3355:1862456998081664:0@lo:400:kworker/u8:1.0 00000100:00100000:1.0:1776178435.941383:0:3355:0:(events.c:361:request_in_callback()) peer: 12345-0@lo (source: 12345-0@lo) 00000100:00100000:1.0:1776178435.941408:0:20755:0:(service.c:2152:ptlrpc_server_handle_req_in()) got req x1862456998081600 00000100:00100000:1.0:1776178435.941436:0:20755:0:(service.c:2152:ptlrpc_server_handle_req_in()) got req x1862456998081664 00000100:00100000:1.0:1776178435.941452:0:20755:0:(nrs_fifo.c:179:nrs_fifo_req_get()) NRS start fifo request from 12345-0@lo, seq: 693 00000100:00100000:1.0:1776178435.941456:0:20755:0:(service.c:2324:ptlrpc_server_handle_request()) Handling RPC req@00000000a59bfc89 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_006:lustre-MDT0001-mdtlov_UUID+5:3355:x1862456998081600:12345-0@lo:400:kworker/u8:1.0 00000100:00100000:1.0:1776178435.941503:0:20755:0:(service.c:2372:ptlrpc_server_handle_request()) Handled RPC req@00000000a59bfc89 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_006:lustre-MDT0001-mdtlov_UUID+5:3355:x1862456998081600:12345-0@lo:400:kworker/u8:1.0 Request processed in 46us (224us total) trans 0 rc 0/0 00000100:00100000:1.0:1776178435.941509:0:20755:0:(nrs_fifo.c:241:nrs_fifo_req_stop()) NRS stop fifo request from 12345-0@lo, seq: 693 00000100:00100000:1.0:1776178435.941520:0:20755:0:(nrs_fifo.c:179:nrs_fifo_req_get()) NRS start fifo request from 12345-0@lo, seq: 694 00000100:00100000:1.0:1776178435.941524:0:20755:0:(service.c:2324:ptlrpc_server_handle_request()) Handling RPC req@00000000ecb64602 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_006:lustre-MDT0001-mdtlov_UUID+5:3355:x1862456998081664:12345-0@lo:400:kworker/u8:1.0 00000100:00100000:1.0:1776178435.941547:0:20755:0:(service.c:2372:ptlrpc_server_handle_request()) Handled RPC req@00000000ecb64602 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_006:lustre-MDT0001-mdtlov_UUID+5:3355:x1862456998081664:12345-0@lo:400:kworker/u8:1.0 Request processed in 22us (164us total) trans 0 rc 0/0 00000100:00100000:1.0:1776178435.941552:0:20755:0:(nrs_fifo.c:241:nrs_fifo_req_stop()) NRS stop fifo request from 12345-0@lo, seq: 694 00000100:00100000:1.0:1776178435.941583:0:3355:0:(client.c:2215:ptlrpc_check_set()) Completed RPC req@000000009220b2b0 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_02:lustre-MDT0001-mdtlov_UUID:3355:1862456998081664:0@lo:400:kworker/u8:1.0 00000100:00100000:1.0:1776178435.941591:0:3355:0:(client.c:2215:ptlrpc_check_set()) Completed RPC req@00000000a13f5de6 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_02:lustre-MDT0001-mdtlov_UUID:3355:1862456998081600:0@lo:400:kworker/u8:1.0 00000100:00100000:3.0:1776178435.941692:0:15213:0:(service.c:2152:ptlrpc_server_handle_req_in()) got req x1862456998081536 00000100:00100000:3.0:1776178435.941725:0:15213:0:(nrs_fifo.c:179:nrs_fifo_req_get()) NRS start fifo request from 12345-0@lo, seq: 753 00000100:00100000:3.0:1776178435.941729:0:15213:0:(service.c:2324:ptlrpc_server_handle_request()) Handling RPC req@000000006b9d8ccb pname:cluuid+ref:pid:xid:nid:opc:job mdt_out00_003:0+-99:3352:x1862456998081536:12345-0@lo:38:kworker/u8:1.0 00010000:02000000:3.0:1776178435.941737:0:15213:0:(ldlm_lib.c:1139:target_handle_connect()) lustre-MDT0000: Not available for connect from 0@lo (stopping) 00000100:00100000:3.0:1776178435.941769:0:15213:0:(service.c:2372:ptlrpc_server_handle_request()) Handled RPC req@000000006b9d8ccb pname:cluuid+ref:pid:xid:nid:opc:job mdt_out00_003:0+-99:3352:x1862456998081536:12345-0@lo:38:kworker/u8:1.0 Request processed in 39us (1336us total) trans 0 rc -19/-19 00000100:00100000:3.0:1776178435.941774:0:15213:0:(nrs_fifo.c:241:nrs_fifo_req_stop()) NRS stop fifo request from 12345-0@lo, seq: 753 00000100:00100000:0.0:1776178435.947256:0:20137:0:(service.c:2152:ptlrpc_server_handle_req_in()) got req x1862456998081216 00000100:00100000:0.0:1776178435.947337:0:20137:0:(nrs_fifo.c:179:nrs_fifo_req_get()) NRS start fifo request from 12345-0@lo, seq: 246 00000100:00100000:0.0:1776178435.947344:0:20137:0:(service.c:2324:ptlrpc_server_handle_request()) Handling RPC req@00000000ff2b550f pname:cluuid+ref:pid:xid:nid:opc:job ll_mgs_0000:ac61f839-0e8e-4cac-9c59-08bcfbbad330+18:3355:x1862456998081216:12345-0@lo:400:kworker/u8:1.0 00000100:00100000:0.0:1776178435.947451:0:20137:0:(service.c:2372:ptlrpc_server_handle_request()) Handled RPC req@00000000ff2b550f pname:cluuid+ref:pid:xid:nid:opc:job ll_mgs_0000:ac61f839-0e8e-4cac-9c59-08bcfbbad330+18:3355:x1862456998081216:12345-0@lo:400:kworker/u8:1.0 Request processed in 107us (6096us total) trans 0 rc 0/0 00000100:00100000:0.0:1776178435.947459:0:20137:0:(nrs_fifo.c:241:nrs_fifo_req_stop()) NRS stop fifo request from 12345-0@lo, seq: 246 00000100:00080000:0.0:1776178435.947512:0:3352:0:(import.c:86:import_set_state_nolock()) 00000000ba4f1821 lustre-MDT0000_UUID: changing import state from CONNECTING to DISCONN 00000100:00080000:0.0:1776178435.947516:0:3352:0:(import.c:1429:ptlrpc_connect_interpret()) recovery of lustre-MDT0000_UUID on 192.168.204.103@tcp failed (-19) 00000100:00100000:0.0:1776178435.947521:0:3352:0:(client.c:2215:ptlrpc_check_set()) Completed RPC req@00000000d1e0f01c pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:lustre-MDT0001-mdtlov_UUID:3352:1862456998081536:0@lo:38:kworker/u8:1.0 00000100:00100000:2.0:1776178435.947639:0:3355:0:(client.c:2215:ptlrpc_check_set()) Completed RPC req@00000000500c0ae6 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_02:ac61f839-0e8e-4cac-9c59-08bcfbbad330:3355:1862456998081216:0@lo:400:kworker/u8:1.0 00010000:00010000:2.0:1776178436.576422:0:9224:0:(ldlm_lockd.c:2792:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:2.0:1776178437.479612:0:3354:0:(ptlrpcd.c:415:ptlrpcd_check()) transfer 1 async RPCs [0->1] 00000100:00100000:2.0:1776178437.479643:0:3354:0:(client.c:1733:ptlrpc_send_new_req()) Sending RPC req@000000007af1bdf7 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_01:lustre-MDT0001-mdtlov_UUID:3354:1862456998081792:0@lo:13:osp-pre-1-1.0 00000100:00100000:2.0:1776178437.479734:0:3354:0:(events.c:361:request_in_callback()) peer: 12345-0@lo (source: 12345-0@lo) 00000100:00100000:1.0:1776178437.479856:0:9554:0:(service.c:2152:ptlrpc_server_handle_req_in()) got req x1862456998081792 00000100:00100000:1.0:1776178437.479923:0:9554:0:(nrs_fifo.c:179:nrs_fifo_req_get()) NRS start fifo request from 12345-0@lo, seq: 579 00000100:00100000:1.0:1776178437.479930:0:9554:0:(service.c:2324:ptlrpc_server_handle_request()) Handling RPC req@00000000803faf52 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_create00:lustre-MDT0001-mdtlov_UUID+5:3354:x1862456998081792:12345-0@lo:13:osp-pre-1-1.0 00002000:00000024:1.0:1776178437.479962:0:9554:0:(ofd_obd.c:740:ofd_statfs()) blocks cached 229376 granted 491746368 pending 0 free 3923689472 avail 3692167168 00000100:00100000:1.0:1776178437.480009:0:9554:0:(service.c:2372:ptlrpc_server_handle_request()) Handled RPC req@00000000803faf52 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_create00:lustre-MDT0001-mdtlov_UUID+5:3354:x1862456998081792:12345-0@lo:13:osp-pre-1-1.0 Request processed in 79us (276us total) trans 0 rc 0/0 00000100:00100000:1.0:1776178437.480016:0:9554:0:(nrs_fifo.c:241:nrs_fifo_req_stop()) NRS stop fifo request from 12345-0@lo, seq: 579 00000100:00100000:2.0:1776178437.480098:0:3354:0:(client.c:2215:ptlrpc_check_set()) Completed RPC req@000000007af1bdf7 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_01:lustre-MDT0001-mdtlov_UUID:3354:1862456998081792:0@lo:13:osp-pre-1-1.0 00010000:00010000:1.0:1776178437.602162:0:9225:0:(ldlm_lockd.c:2792:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:1.0:1776178437.984523:0:3353:0:(ptlrpcd.c:415:ptlrpcd_check()) transfer 1 async RPCs [1->0] 00000100:00100000:1.0:1776178437.984547:0:3353:0:(client.c:1733:ptlrpc_send_new_req()) Sending RPC req@00000000500c0ae6 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_00:lustre-MDT0001-mdtlov_UUID:3353:1862456998081856:0@lo:13:osp-pre-0-1.0 00000100:00100000:1.0:1776178437.984622:0:3353:0:(events.c:361:request_in_callback()) peer: 12345-0@lo (source: 12345-0@lo) 00000100:00100000:3.0:1776178437.985224:0:8231:0:(service.c:2152:ptlrpc_server_handle_req_in()) got req x1862456998081856 00000100:00100000:3.0:1776178437.985299:0:8231:0:(nrs_fifo.c:179:nrs_fifo_req_get()) NRS start fifo request from 12345-0@lo, seq: 580 00000100:00100000:3.0:1776178437.985305:0:8231:0:(service.c:2324:ptlrpc_server_handle_request()) Handling RPC req@000000009d5112f6 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_create00:lustre-MDT0001-mdtlov_UUID+5:3353:x1862456998081856:12345-0@lo:13:osp-pre-0-1.0 00002000:00000024:3.0:1776178437.985336:0:8231:0:(ofd_obd.c:740:ofd_statfs()) blocks cached 315392 granted 484883776 pending 0 free 3923689472 avail 3692167168 00000100:00100000:3.0:1776178437.985394:0:8231:0:(service.c:2372:ptlrpc_server_handle_request()) Handled RPC req@000000009d5112f6 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_create00:lustre-MDT0001-mdtlov_UUID+5:3353:x1862456998081856:12345-0@lo:13:osp-pre-0-1.0 Request processed in 88us (773us total) trans 0 rc 0/0 00000100:00100000:3.0:1776178437.985401:0:8231:0:(nrs_fifo.c:241:nrs_fifo_req_stop()) NRS stop fifo request from 12345-0@lo, seq: 580 00000100:00100000:0.0:1776178437.985535:0:3353:0:(client.c:2215:ptlrpc_check_set()) Completed RPC req@00000000500c0ae6 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_00:lustre-MDT0001-mdtlov_UUID:3353:1862456998081856:0@lo:13:osp-pre-0-1.0 00010000:00010000:3.0:1776178438.629913:0:6244:0:(ldlm_lockd.c:2792:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:0.0:1776178439.463670:0:3347:0:(events.c:361:request_in_callback()) peer: 12345-192.168.204.3@tcp (source: 12345-192.168.204.3@tcp) 00000100:00100000:0.0:1776178439.463713:0:3347:0:(events.c:361:request_in_callback()) peer: 12345-192.168.204.3@tcp (source: 12345-192.168.204.3@tcp) 00000100:00100000:0.0:1776178439.463746:0:3347:0:(events.c:361:request_in_callback()) peer: 12345-192.168.204.3@tcp (source: 12345-192.168.204.3@tcp) 00000100:00100000:0.0:1776178439.463772:0:3347:0:(events.c:361:request_in_callback()) peer: 12345-192.168.204.3@tcp (source: 12345-192.168.204.3@tcp) 00000100:00100000:0.0:1776178439.463808:0:6255:0:(service.c:2152:ptlrpc_server_handle_req_in()) got req x1862456979123840 00000100:00100000:0.0:1776178439.463857:0:6255:0:(nrs_fifo.c:179:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.204.3@tcp, seq: 1066 00000100:00100000:0.0:1776178439.463864:0:6255:0:(service.c:2324:ptlrpc_server_handle_request()) Handling RPC req@00000000d1e0f01c pname:cluuid+ref:pid:xid:nid:opc:job mdt00_000:b50963bd-8eba-4432-9a54-8888d00e3fa1+6:2271:x1862456979123840:12345-192.168.204.3@tcp:400:kworker/u8:4.0 00000100:00100000:0.0:1776178439.463945:0:6255:0:(service.c:2372:ptlrpc_server_handle_request()) Handled RPC req@00000000d1e0f01c pname:cluuid+ref:pid:xid:nid:opc:job mdt00_000:b50963bd-8eba-4432-9a54-8888d00e3fa1+6:2271:x1862456979123840:12345-192.168.204.3@tcp:400:kworker/u8:4.0 Request processed in 81us (200us total) trans 0 rc 0/0 00000100:00100000:0.0:1776178439.463953:0:6255:0:(nrs_fifo.c:241:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.204.3@tcp, seq: 1066 00000100:00100000:0.0:1776178439.463978:0:20762:0:(service.c:2152:ptlrpc_server_handle_req_in()) got req x1862456979123904 00000100:00100000:0.0:1776178439.464000:0:20762:0:(service.c:2152:ptlrpc_server_handle_req_in()) got req x1862456979123968 00000100:00100000:0.0:1776178439.464016:0:20762:0:(nrs_fifo.c:179:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.204.3@tcp, seq: 695 00000100:00100000:0.0:1776178439.464020:0:20762:0:(service.c:2324:ptlrpc_server_handle_request()) Handling RPC req@00000000500c0ae6 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_013:b50963bd-8eba-4432-9a54-8888d00e3fa1+8:2271:x1862456979123904:12345-192.168.204.3@tcp:400:kworker/u8:4.0 00000100:00100000:0.0:1776178439.464046:0:20762:0:(service.c:2372:ptlrpc_server_handle_request()) Handled RPC req@00000000500c0ae6 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_013:b50963bd-8eba-4432-9a54-8888d00e3fa1+8:2271:x1862456979123904:12345-192.168.204.3@tcp:400:kworker/u8:4.0 Request processed in 24us (333us total) trans 0 rc 0/0 00000100:00100000:0.0:1776178439.464052:0:20762:0:(nrs_fifo.c:241:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.204.3@tcp, seq: 695 00000100:00100000:0.0:1776178439.464095:0:20762:0:(nrs_fifo.c:179:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.204.3@tcp, seq: 696 00000100:00100000:0.0:1776178439.464099:0:20762:0:(service.c:2324:ptlrpc_server_handle_request()) Handling RPC req@00000000a13f5de6 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_013:b50963bd-8eba-4432-9a54-8888d00e3fa1+9:2271:x1862456979123968:12345-192.168.204.3@tcp:400:kworker/u8:4.0 00000100:00100000:0.0:1776178439.464120:0:20762:0:(service.c:2372:ptlrpc_server_handle_request()) Handled RPC req@00000000a13f5de6 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_013:b50963bd-8eba-4432-9a54-8888d00e3fa1+9:2271:x1862456979123968:12345-192.168.204.3@tcp:400:kworker/u8:4.0 Request processed in 21us (349us total) trans 0 rc 0/0 00000100:00100000:0.0:1776178439.464125:0:20762:0:(nrs_fifo.c:241:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.204.3@tcp, seq: 696 00000100:00100000:3.0:1776178439.464945:0:20140:0:(service.c:2152:ptlrpc_server_handle_req_in()) got req x1862456979123712 00000100:00100000:3.0:1776178439.464985:0:20140:0:(nrs_fifo.c:179:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.204.3@tcp, seq: 247 00000100:00100000:3.0:1776178439.464990:0:20140:0:(service.c:2324:ptlrpc_server_handle_request()) Handling RPC req@00000000fb48a5c8 pname:cluuid+ref:pid:xid:nid:opc:job ll_mgs_0002:5b5c66b1-1a2f-4651-8548-0c75b9a38a94+9:2271:x1862456979123712:12345-192.168.204.3@tcp:400:kworker/u8:4.0 00000100:00100000:3.0:1776178439.465028:0:20140:0:(service.c:2372:ptlrpc_server_handle_request()) Handled RPC req@00000000fb48a5c8 pname:cluuid+ref:pid:xid:nid:opc:job ll_mgs_0002:5b5c66b1-1a2f-4651-8548-0c75b9a38a94+9:2271:x1862456979123712:12345-192.168.204.3@tcp:400:kworker/u8:4.0 Request processed in 37us (1368us total) trans 0 rc 0/0 00000100:00100000:3.0:1776178439.465035:0:20140:0:(nrs_fifo.c:241:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.204.3@tcp, seq: 247 00000100:00100000:1.0:1776178439.466145:0:3348:0:(events.c:361:request_in_callback()) peer: 12345-192.168.204.3@tcp (source: 12345-192.168.204.3@tcp) 00000100:00100000:2.0:1776178439.466327:0:6256:0:(service.c:2152:ptlrpc_server_handle_req_in()) got req x1862456979123776 00000100:00100000:2.0:1776178439.466369:0:6256:0:(nrs_fifo.c:179:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.204.3@tcp, seq: 6808 00000100:00100000:2.0:1776178439.466375:0:6256:0:(service.c:2324:ptlrpc_server_handle_request()) Handling RPC req@00000000f0ef812c pname:cluuid+ref:pid:xid:nid:opc:job mdt00_001:0+-99:2269:x1862456979123776:12345-192.168.204.3@tcp:38:kworker/u8:4.0 00010000:02000000:2.0:1776178439.466390:0:6256:0:(ldlm_lib.c:1139:target_handle_connect()) lustre-MDT0000: Not available for connect from 192.168.204.3@tcp (stopping) 00000100:00100000:2.0:1776178439.466434:0:6256:0:(service.c:2372:ptlrpc_server_handle_request()) Handled RPC req@00000000f0ef812c pname:cluuid+ref:pid:xid:nid:opc:job mdt00_001:0+-99:2269:x1862456979123776:12345-192.168.204.3@tcp:38:kworker/u8:4.0 Request processed in 58us (294us total) trans 0 rc -19/-19 00000100:00100000:2.0:1776178439.466440:0:6256:0:(nrs_fifo.c:241:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.204.3@tcp, seq: 6808 00010000:00010000:2.0:1776178439.648827:0:6245:0:(ldlm_lockd.c:2792:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00010000:00010000:2.0:1776178440.672342:0:9224:0:(ldlm_lockd.c:2792:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00080000:2.0:1776178441.056748:0:36:0:(pinger.c:251:ptlrpc_pinger_process_import()) lustre-MDT0000-lwp-OST0000_UUID->lustre-MDT0000_UUID: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00080000:2.0:1776178441.056759:0:36:0:(recover.c:57:ptlrpc_initiate_recovery()) lustre-MDT0000_UUID: starting recovery 00000100:00080000:2.0:1776178441.056764:0:36:0:(import.c:86:import_set_state_nolock()) 0000000047601a98 lustre-MDT0000_UUID: changing import state from DISCONN to CONNECTING 00000100:00080000:2.0:1776178441.056775:0:36:0:(import.c:534:import_select_connection()) lustre-MDT0000-lwp-OST0000: connect to NID 0@lo last attempt 1143 00000100:00080000:2.0:1776178441.056780:0:36:0:(import.c:581:import_select_connection()) lustre-MDT0000-lwp-OST0000: tried all connections, increasing latency to 21s 00000100:00080000:2.0:1776178441.056786:0:36:0:(import.c:615:import_select_connection()) lustre-MDT0000-lwp-OST0000: import 0000000047601a98 using connection 192.168.204.103@tcp/0@lo 00000100:00100000:2.0:1776178441.056810:0:36:0:(import.c:807:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5) req@000000008c50846d x1862456998081984/t0(0) o38->lustre-MDT0000-lwp-OST0000@0@lo:12/10 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' 00000100:00080000:2.0:1776178441.056842:0:36:0:(pinger.c:251:ptlrpc_pinger_process_import()) lustre-MDT0000-lwp-OST0001_UUID->lustre-MDT0000_UUID: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00080000:2.0:1776178441.056845:0:36:0:(recover.c:57:ptlrpc_initiate_recovery()) lustre-MDT0000_UUID: starting recovery 00000100:00080000:2.0:1776178441.056846:0:36:0:(import.c:86:import_set_state_nolock()) 00000000dfc8db13 lustre-MDT0000_UUID: changing import state from DISCONN to CONNECTING 00000100:00080000:2.0:1776178441.056850:0:36:0:(import.c:534:import_select_connection()) lustre-MDT0000-lwp-OST0001: connect to NID 0@lo last attempt 1143 00000100:00080000:2.0:1776178441.056852:0:36:0:(import.c:581:import_select_connection()) lustre-MDT0000-lwp-OST0001: tried all connections, increasing latency to 21s 00000100:00080000:2.0:1776178441.056855:0:36:0:(import.c:615:import_select_connection()) lustre-MDT0000-lwp-OST0001: import 00000000dfc8db13 using connection 192.168.204.103@tcp/0@lo 00000100:00100000:2.0:1776178441.056866:0:36:0:(import.c:807:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5) req@00000000f351bcc3 x1862456998082112/t0(0) o38->lustre-MDT0000-lwp-OST0001@0@lo:12/10 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' 00000100:00080000:2.0:1776178441.056906:0:36:0:(pinger.c:251:ptlrpc_pinger_process_import()) lustre-MDT0001-mdtlov_UUID->lustre-MDT0000_UUID: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00080000:2.0:1776178441.056910:0:36:0:(recover.c:57:ptlrpc_initiate_recovery()) lustre-MDT0000_UUID: starting recovery 00000100:00080000:2.0:1776178441.056911:0:36:0:(import.c:86:import_set_state_nolock()) 00000000ba4f1821 lustre-MDT0000_UUID: changing import state from DISCONN to CONNECTING 00000100:00080000:2.0:1776178441.056914:0:36:0:(import.c:534:import_select_connection()) lustre-MDT0000-osp-MDT0001: connect to NID 0@lo last attempt 1143 00000100:00080000:2.0:1776178441.056916:0:36:0:(import.c:581:import_select_connection()) lustre-MDT0000-osp-MDT0001: tried all connections, increasing latency to 21s 00000100:00080000:2.0:1776178441.056919:0:36:0:(import.c:615:import_select_connection()) lustre-MDT0000-osp-MDT0001: import 00000000ba4f1821 using connection 192.168.204.103@tcp/0@lo 00000100:00100000:2.0:1776178441.056931:0:36:0:(import.c:807:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5) req@00000000a0a4914c x1862456998082240/t0(0) o38->lustre-MDT0000-osp-MDT0001@0@lo:24/4 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' 00000100:00080000:2.0:1776178441.056948:0:36:0:(pinger.c:251:ptlrpc_pinger_process_import()) lustre-MDT0000-lwp-MDT0001_UUID->lustre-MDT0000_UUID: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00080000:2.0:1776178441.056950:0:36:0:(recover.c:57:ptlrpc_initiate_recovery()) lustre-MDT0000_UUID: starting recovery 00000100:00080000:2.0:1776178441.056952:0:36:0:(import.c:86:import_set_state_nolock()) 00000000a7c55b1f lustre-MDT0000_UUID: changing import state from DISCONN to CONNECTING 00000100:00080000:2.0:1776178441.056955:0:36:0:(import.c:534:import_select_connection()) lustre-MDT0000-lwp-MDT0001: connect to NID 0@lo last attempt 1143 00000100:00080000:2.0:1776178441.056956:0:36:0:(import.c:581:import_select_connection()) lustre-MDT0000-lwp-MDT0001: tried all connections, increasing latency to 21s 00000100:00080000:2.0:1776178441.056959:0:36:0:(import.c:615:import_select_connection()) lustre-MDT0000-lwp-MDT0001: import 00000000a7c55b1f using connection 192.168.204.103@tcp/0@lo 00000100:00100000:2.0:1776178441.056969:0:36:0:(import.c:807:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5) req@0000000078fb1a26 x1862456998082432/t0(0) o38->lustre-MDT0000-lwp-MDT0001@0@lo:12/10 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' 00000100:00100000:2.0:1776178441.057002:0:3354:0:(client.c:1733:ptlrpc_send_new_req()) Sending RPC req@000000006c57be21 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_01:lustre-MDT0001-mdtlov_UUID:3354:1862456998082304:0@lo:400:kworker/u8:1.0 00000100:00100000:2.0:1776178441.057101:0:3354:0:(events.c:361:request_in_callback()) peer: 12345-0@lo (source: 12345-0@lo) 00000100:00100000:2.0:1776178441.057144:0:3354:0:(ptlrpcd.c:415:ptlrpcd_check()) transfer 1 async RPCs [0->1] 00000100:00100000:2.0:1776178441.057149:0:3354:0:(client.c:1733:ptlrpc_send_new_req()) Sending RPC req@000000000489d393 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_01:lustre-MDT0001-lwp-OST0001_UUID:3354:1862456998082176:0@lo:400:kworker/u8:1.0 00000100:00100000:2.0:1776178441.057180:0:3354:0:(events.c:361:request_in_callback()) peer: 12345-0@lo (source: 12345-0@lo) 00000100:00100000:2.0:1776178441.057231:0:20759:0:(service.c:2152:ptlrpc_server_handle_req_in()) got req x1862456998082304 00000100:00100000:2.0:1776178441.057271:0:20759:0:(nrs_fifo.c:179:nrs_fifo_req_get()) NRS start fifo request from 12345-0@lo, seq: 697 00000100:00100000:2.0:1776178441.057277:0:20759:0:(service.c:2324:ptlrpc_server_handle_request()) Handling RPC req@00000000103d2c47 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_010:lustre-MDT0001-mdtlov_UUID+5:3354:x1862456998082304:12345-0@lo:400:kworker/u8:1.0 00000100:00100000:2.0:1776178441.057333:0:20759:0:(service.c:2372:ptlrpc_server_handle_request()) Handled RPC req@00000000103d2c47 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_010:lustre-MDT0001-mdtlov_UUID+5:3354:x1862456998082304:12345-0@lo:400:kworker/u8:1.0 Request processed in 55us (234us total) trans 0 rc 0/0 00000100:00100000:2.0:1776178441.057340:0:20759:0:(nrs_fifo.c:241:nrs_fifo_req_stop()) NRS stop fifo request from 12345-0@lo, seq: 697 00000100:00100000:2.0:1776178441.057392:0:3354:0:(client.c:2215:ptlrpc_check_set()) Completed RPC req@000000006c57be21 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_01:lustre-MDT0001-mdtlov_UUID:3354:1862456998082304:0@lo:400:kworker/u8:1.0 00000100:00100000:0.0:1776178441.058035:0:3352:0:(client.c:1733:ptlrpc_send_new_req()) Sending RPC req@000000008c50846d pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:lustre-MDT0000-lwp-OST0000_UUID:3352:1862456998081984:0@lo:38:kworker/u8:1.0 00000100:00100000:0.0:1776178441.058116:0:3352:0:(events.c:361:request_in_callback()) peer: 12345-0@lo (source: 12345-0@lo) 00000100:00100000:0.0:1776178441.058135:0:3352:0:(client.c:1733:ptlrpc_send_new_req()) Sending RPC req@00000000f351bcc3 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:lustre-MDT0000-lwp-OST0001_UUID:3352:1862456998082112:0@lo:38:kworker/u8:1.0 00000100:00100000:0.0:1776178441.058154:0:3352:0:(events.c:361:request_in_callback()) peer: 12345-0@lo (source: 12345-0@lo) 00000100:00100000:0.0:1776178441.058167:0:3352:0:(client.c:1733:ptlrpc_send_new_req()) Sending RPC req@00000000a0a4914c pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:lustre-MDT0001-mdtlov_UUID:3352:1862456998082240:0@lo:38:kworker/u8:1.0 00000100:00100000:0.0:1776178441.058195:0:3352:0:(events.c:361:request_in_callback()) peer: 12345-0@lo (source: 12345-0@lo) 00000100:00100000:0.0:1776178441.058214:0:3352:0:(client.c:1733:ptlrpc_send_new_req()) Sending RPC req@0000000078fb1a26 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:lustre-MDT0000-lwp-MDT0001_UUID:3352:1862456998082432:0@lo:38:kworker/u8:1.0 00000100:00100000:0.0:1776178441.058233:0:3352:0:(events.c:361:request_in_callback()) peer: 12345-0@lo (source: 12345-0@lo) 00000100:00100000:0.0:1776178441.058283:0:14128:0:(service.c:2152:ptlrpc_server_handle_req_in()) got req x1862456998082240 00000100:00100000:0.0:1776178441.058311:0:14128:0:(nrs_fifo.c:179:nrs_fifo_req_get()) NRS start fifo request from 12345-0@lo, seq: 754 00000100:00100000:0.0:1776178441.058316:0:14128:0:(service.c:2324:ptlrpc_server_handle_request()) Handling RPC req@0000000071333ecf pname:cluuid+ref:pid:xid:nid:opc:job mdt_out00_002:0+-99:3352:x1862456998082240:12345-0@lo:38:kworker/u8:1.0 00010000:02000000:0.0:1776178441.058333:0:14128:0:(ldlm_lib.c:1139:target_handle_connect()) lustre-MDT0000: Not available for connect from 0@lo (stopping) 00000100:00100000:0.0:1776178441.058378:0:14128:0:(service.c:2372:ptlrpc_server_handle_request()) Handled RPC req@0000000071333ecf pname:cluuid+ref:pid:xid:nid:opc:job mdt_out00_002:0+-99:3352:x1862456998082240:12345-0@lo:38:kworker/u8:1.0 Request processed in 61us (181us total) trans 0 rc -19/-19 00000100:00100000:0.0:1776178441.058384:0:14128:0:(nrs_fifo.c:241:nrs_fifo_req_stop()) NRS stop fifo request from 12345-0@lo, seq: 754 00000100:00080000:0.0:1776178441.058413:0:3352:0:(import.c:86:import_set_state_nolock()) 00000000ba4f1821 lustre-MDT0000_UUID: changing import state from CONNECTING to DISCONN 00000100:00080000:0.0:1776178441.058417:0:3352:0:(import.c:1429:ptlrpc_connect_interpret()) recovery of lustre-MDT0000_UUID on 192.168.204.103@tcp failed (-19) 00000100:00100000:0.0:1776178441.058421:0:3352:0:(client.c:2215:ptlrpc_check_set()) Completed RPC req@00000000a0a4914c pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:lustre-MDT0001-mdtlov_UUID:3352:1862456998082240:0@lo:38:kworker/u8:1.0 00000100:00100000:3.0:1776178441.060129:0:6257:0:(service.c:2152:ptlrpc_server_handle_req_in()) got req x1862456998082176 00000100:00100000:3.0:1776178441.060203:0:6257:0:(service.c:2152:ptlrpc_server_handle_req_in()) got req x1862456998081984 00000100:00100000:3.0:1776178441.060212:0:6257:0:(service.c:2152:ptlrpc_server_handle_req_in()) got req x1862456998082112 00000100:00100000:3.0:1776178441.060219:0:6257:0:(service.c:2152:ptlrpc_server_handle_req_in()) got req x1862456998082432 00000100:00100000:3.0:1776178441.060235:0:6257:0:(nrs_fifo.c:179:nrs_fifo_req_get()) NRS start fifo request from 12345-0@lo, seq: 1067 00000100:00100000:3.0:1776178441.060241:0:6257:0:(service.c:2324:ptlrpc_server_handle_request()) Handling RPC req@00000000747e17cb pname:cluuid+ref:pid:xid:nid:opc:job mdt00_002:lustre-MDT0001-lwp-OST0001_UUID+5:3354:x1862456998082176:12345-0@lo:400:kworker/u8:1.0 00000100:00100000:1.0:1776178441.060357:0:3355:0:(client.c:1733:ptlrpc_send_new_req()) Sending RPC req@000000007d6ef583 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_02:ac61f839-0e8e-4cac-9c59-08bcfbbad330:3355:1862456998081920:0@lo:400:kworker/u8:1.0 00000100:00100000:3.0:1776178441.060370:0:6257:0:(service.c:2372:ptlrpc_server_handle_request()) Handled RPC req@00000000747e17cb pname:cluuid+ref:pid:xid:nid:opc:job mdt00_002:lustre-MDT0001-lwp-OST0001_UUID+5:3354:x1862456998082176:12345-0@lo:400:kworker/u8:1.0 Request processed in 129us (3191us total) trans 0 rc 0/0 00000100:00100000:3.0:1776178441.060377:0:6257:0:(nrs_fifo.c:241:nrs_fifo_req_stop()) NRS stop fifo request from 12345-0@lo, seq: 1067 00000100:00100000:3.0:1776178441.060396:0:6257:0:(nrs_fifo.c:179:nrs_fifo_req_get()) NRS start fifo request from 12345-0@lo, seq: 6809 00000100:00100000:3.0:1776178441.060399:0:6257:0:(service.c:2324:ptlrpc_server_handle_request()) Handling RPC req@000000009220b2b0 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_002:0+-99:3352:x1862456998081984:12345-0@lo:38:kworker/u8:1.0 00000100:00100000:1.0:1776178441.060404:0:3355:0:(events.c:361:request_in_callback()) peer: 12345-0@lo (source: 12345-0@lo) 00010000:02000000:3.0:1776178441.060413:0:6257:0:(ldlm_lib.c:1139:target_handle_connect()) lustre-MDT0000: Not available for connect from 0@lo (stopping) 00000100:00100000:1.0:1776178441.060430:0:3355:0:(client.c:1733:ptlrpc_send_new_req()) Sending RPC req@00000000a9455d7d pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_02:lustre-MDT0001-mdtlov_UUID:3355:1862456998082368:0@lo:400:kworker/u8:1.0 00000100:00100000:3.0:1776178441.060445:0:6257:0:(service.c:2372:ptlrpc_server_handle_request()) Handled RPC req@000000009220b2b0 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_002:0+-99:3352:x1862456998081984:12345-0@lo:38:kworker/u8:1.0 Request processed in 45us (2330us total) trans 0 rc -19/-19 00000100:00100000:3.0:1776178441.060451:0:6257:0:(nrs_fifo.c:241:nrs_fifo_req_stop()) NRS stop fifo request from 12345-0@lo, seq: 6809 00000100:00100000:1.0:1776178441.060452:0:3355:0:(events.c:361:request_in_callback()) peer: 12345-0@lo (source: 12345-0@lo) 00000100:00100000:3.0:1776178441.060462:0:6257:0:(nrs_fifo.c:179:nrs_fifo_req_get()) NRS start fifo request from 12345-0@lo, seq: 6810 00000100:00100000:3.0:1776178441.060464:0:6257:0:(service.c:2324:ptlrpc_server_handle_request()) Handling RPC req@000000006e2391ef pname:cluuid+ref:pid:xid:nid:opc:job mdt00_002:0+-99:3352:x1862456998082112:12345-0@lo:38:kworker/u8:1.0 00010000:02000000:3.0:1776178441.060470:0:6257:0:(ldlm_lib.c:1139:target_handle_connect()) lustre-MDT0000: Not available for connect from 0@lo (stopping) 00000100:00100000:1.0:1776178441.060484:0:3355:0:(ptlrpcd.c:415:ptlrpcd_check()) transfer 1 async RPCs [3->2] 00000100:00100000:3.0:1776178441.060490:0:6257:0:(service.c:2372:ptlrpc_server_handle_request()) Handled RPC req@000000006e2391ef pname:cluuid+ref:pid:xid:nid:opc:job mdt00_002:0+-99:3352:x1862456998082112:12345-0@lo:38:kworker/u8:1.0 Request processed in 25us (2337us total) trans 0 rc -19/-19 00000100:00100000:1.0:1776178441.060491:0:3355:0:(client.c:1733:ptlrpc_send_new_req()) Sending RPC req@0000000047a42b33 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_02:lustre-MDT0001-lwp-OST0000_UUID:3355:1862456998082048:0@lo:400:kworker/u8:1.0 00000100:00100000:3.0:1776178441.060496:0:6257:0:(nrs_fifo.c:241:nrs_fifo_req_stop()) NRS stop fifo request from 12345-0@lo, seq: 6810 00000100:00100000:3.0:1776178441.060505:0:6257:0:(nrs_fifo.c:179:nrs_fifo_req_get()) NRS start fifo request from 12345-0@lo, seq: 6811 00000100:00100000:3.0:1776178441.060508:0:6257:0:(service.c:2324:ptlrpc_server_handle_request()) Handling RPC req@000000007cfde41a pname:cluuid+ref:pid:xid:nid:opc:job mdt00_002:0+-99:3352:x1862456998082432:12345-0@lo:38:kworker/u8:1.0 00000100:00100000:1.0:1776178441.060513:0:3355:0:(events.c:361:request_in_callback()) peer: 12345-0@lo (source: 12345-0@lo) 00010000:02000000:3.0:1776178441.060514:0:6257:0:(ldlm_lib.c:1139:target_handle_connect()) lustre-MDT0000: Not available for connect from 0@lo (stopping) 00000100:00100000:1.0:1776178441.060536:0:6255:0:(service.c:2152:ptlrpc_server_handle_req_in()) got req x1862456998082048 00000100:00100000:3.0:1776178441.060543:0:6257:0:(service.c:2372:ptlrpc_server_handle_request()) Handled RPC req@000000007cfde41a pname:cluuid+ref:pid:xid:nid:opc:job mdt00_002:0+-99:3352:x1862456998082432:12345-0@lo:38:kworker/u8:1.0 Request processed in 22us (2297us total) trans 0 rc -19/-19 00000100:00100000:3.0:1776178441.060548:0:6257:0:(nrs_fifo.c:241:nrs_fifo_req_stop()) NRS stop fifo request from 12345-0@lo, seq: 6811 00000100:00100000:3.0:1776178441.060595:0:3354:0:(client.c:2215:ptlrpc_check_set()) Completed RPC req@000000000489d393 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_01:lustre-MDT0001-lwp-OST0001_UUID:3354:1862456998082176:0@lo:400:kworker/u8:1.0 00000100:00100000:0.0:1776178441.060622:0:20137:0:(service.c:2152:ptlrpc_server_handle_req_in()) got req x1862456998081920 00000100:00080000:3.0:1776178441.060630:0:3352:0:(import.c:86:import_set_state_nolock()) 0000000047601a98 lustre-MDT0000_UUID: changing import state from CONNECTING to DISCONN 00000100:00080000:3.0:1776178441.060634:0:3352:0:(import.c:1429:ptlrpc_connect_interpret()) recovery of lustre-MDT0000_UUID on 192.168.204.103@tcp failed (-19) 00000100:00100000:3.0:1776178441.060637:0:3352:0:(client.c:2215:ptlrpc_check_set()) Completed RPC req@000000008c50846d pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:lustre-MDT0000-lwp-OST0000_UUID:3352:1862456998081984:0@lo:38:kworker/u8:1.0 00000100:00080000:3.0:1776178441.060646:0:3352:0:(import.c:86:import_set_state_nolock()) 00000000dfc8db13 lustre-MDT0000_UUID: changing import state from CONNECTING to DISCONN 00000100:00080000:3.0:1776178441.060649:0:3352:0:(import.c:1429:ptlrpc_connect_interpret()) recovery of lustre-MDT0000_UUID on 192.168.204.103@tcp failed (-19) 00000100:00100000:3.0:1776178441.060652:0:3352:0:(client.c:2215:ptlrpc_check_set()) Completed RPC req@00000000f351bcc3 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:lustre-MDT0000-lwp-OST0001_UUID:3352:1862456998082112:0@lo:38:kworker/u8:1.0 00000100:00100000:0.0:1776178441.060653:0:20137:0:(nrs_fifo.c:179:nrs_fifo_req_get()) NRS start fifo request from 12345-0@lo, seq: 248 00000100:00100000:0.0:1776178441.060656:0:20137:0:(service.c:2324:ptlrpc_server_handle_request()) Handling RPC req@0000000060db901b pname:cluuid+ref:pid:xid:nid:opc:job ll_mgs_0000:ac61f839-0e8e-4cac-9c59-08bcfbbad330+18:3355:x1862456998081920:12345-0@lo:400:kworker/u8:1.0 00000100:00080000:3.0:1776178441.060660:0:3352:0:(import.c:86:import_set_state_nolock()) 00000000a7c55b1f lustre-MDT0000_UUID: changing import state from CONNECTING to DISCONN 00000100:00080000:3.0:1776178441.060663:0:3352:0:(import.c:1429:ptlrpc_connect_interpret()) recovery of lustre-MDT0000_UUID on 192.168.204.103@tcp failed (-19) 00000100:00100000:3.0:1776178441.060665:0:3352:0:(client.c:2215:ptlrpc_check_set()) Completed RPC req@0000000078fb1a26 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:lustre-MDT0000-lwp-MDT0001_UUID:3352:1862456998082432:0@lo:38:kworker/u8:1.0 00000100:00100000:0.0:1776178441.060690:0:20137:0:(service.c:2372:ptlrpc_server_handle_request()) Handled RPC req@0000000060db901b pname:cluuid+ref:pid:xid:nid:opc:job ll_mgs_0000:ac61f839-0e8e-4cac-9c59-08bcfbbad330+18:3355:x1862456998081920:12345-0@lo:400:kworker/u8:1.0 Request processed in 32us (285us total) trans 0 rc 0/0 00000100:00100000:0.0:1776178441.060695:0:20137:0:(nrs_fifo.c:241:nrs_fifo_req_stop()) NRS stop fifo request from 12345-0@lo, seq: 248 00000100:00100000:3.0:1776178441.060718:0:20755:0:(service.c:2152:ptlrpc_server_handle_req_in()) got req x1862456998082368 00000100:00100000:0.0:1776178441.060732:0:3355:0:(client.c:2215:ptlrpc_check_set()) Completed RPC req@000000007d6ef583 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_02:ac61f839-0e8e-4cac-9c59-08bcfbbad330:3355:1862456998081920:0@lo:400:kworker/u8:1.0 00000100:00100000:3.0:1776178441.060741:0:20755:0:(nrs_fifo.c:179:nrs_fifo_req_get()) NRS start fifo request from 12345-0@lo, seq: 698 00000100:00100000:3.0:1776178441.060745:0:20755:0:(service.c:2324:ptlrpc_server_handle_request()) Handling RPC req@000000009e7a8bb5 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_006:lustre-MDT0001-mdtlov_UUID+5:3355:x1862456998082368:12345-0@lo:400:kworker/u8:1.0 00000100:00100000:3.0:1776178441.060796:0:20755:0:(service.c:2372:ptlrpc_server_handle_request()) Handled RPC req@000000009e7a8bb5 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_006:lustre-MDT0001-mdtlov_UUID+5:3355:x1862456998082368:12345-0@lo:400:kworker/u8:1.0 Request processed in 50us (343us total) trans 0 rc 0/0 00000100:00100000:3.0:1776178441.060801:0:20755:0:(nrs_fifo.c:241:nrs_fifo_req_stop()) NRS stop fifo request from 12345-0@lo, seq: 698 00000100:00100000:0.0:1776178441.062098:0:3355:0:(client.c:2215:ptlrpc_check_set()) Completed RPC req@00000000a9455d7d pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_02:lustre-MDT0001-mdtlov_UUID:3355:1862456998082368:0@lo:400:kworker/u8:1.0 00000100:00100000:1.0:1776178441.067693:0:6255:0:(nrs_fifo.c:179:nrs_fifo_req_get()) NRS start fifo request from 12345-0@lo, seq: 1068 00000100:00100000:1.0:1776178441.068506:0:6255:0:(service.c:2324:ptlrpc_server_handle_request()) Handling RPC req@00000000e64148ab pname:cluuid+ref:pid:xid:nid:opc:job mdt00_000:lustre-MDT0001-lwp-OST0000_UUID+5:3355:x1862456998082048:12345-0@lo:400:kworker/u8:1.0 00000100:00100000:0.0:1776178441.068904:0:6255:0:(service.c:2372:ptlrpc_server_handle_request()) Handled RPC req@00000000e64148ab pname:cluuid+ref:pid:xid:nid:opc:job mdt00_000:lustre-MDT0001-lwp-OST0000_UUID+5:3355:x1862456998082048:12345-0@lo:400:kworker/u8:1.0 Request processed in 1195us (8386us total) trans 0 rc 0/0 00000100:00100000:0.0:1776178441.068917:0:6255:0:(nrs_fifo.c:241:nrs_fifo_req_stop()) NRS stop fifo request from 12345-0@lo, seq: 1068 00000100:00100000:1.0:1776178441.069234:0:3355:0:(client.c:2215:ptlrpc_check_set()) Completed RPC req@0000000047a42b33 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_02:lustre-MDT0001-lwp-OST0000_UUID:3355:1862456998082048:0@lo:400:kworker/u8:1.0 00010000:00010000:1.0:1776178441.696360:0:9225:0:(ldlm_lockd.c:2792:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:1.0:1776178442.592439:0:3355:0:(ptlrpcd.c:415:ptlrpcd_check()) transfer 1 async RPCs [3->2] 00000100:00100000:1.0:1776178442.592494:0:3355:0:(client.c:1733:ptlrpc_send_new_req()) Sending RPC req@00000000829797f1 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_02:lustre-MDT0001-mdtlov_UUID:3355:1862456998082496:0@lo:13:osp-pre-1-1.0 00000100:00100000:1.0:1776178442.592629:0:3355:0:(events.c:361:request_in_callback()) peer: 12345-0@lo (source: 12345-0@lo) 00000100:00100000:2.0:1776178442.592894:0:9554:0:(service.c:2152:ptlrpc_server_handle_req_in()) got req x1862456998082496 00000100:00100000:2.0:1776178442.592951:0:9554:0:(nrs_fifo.c:179:nrs_fifo_req_get()) NRS start fifo request from 12345-0@lo, seq: 581 00000100:00100000:2.0:1776178442.592958:0:9554:0:(service.c:2324:ptlrpc_server_handle_request()) Handling RPC req@0000000072865dff pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_create00:lustre-MDT0001-mdtlov_UUID+5:3355:x1862456998082496:12345-0@lo:13:osp-pre-1-1.0 00002000:00000024:2.0:1776178442.592990:0:9554:0:(ofd_obd.c:740:ofd_statfs()) blocks cached 229376 granted 491746368 pending 0 free 3923689472 avail 3692167168 00000100:00100000:2.0:1776178442.593053:0:9554:0:(service.c:2372:ptlrpc_server_handle_request()) Handled RPC req@0000000072865dff pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_create00:lustre-MDT0001-mdtlov_UUID+5:3355:x1862456998082496:12345-0@lo:13:osp-pre-1-1.0 Request processed in 95us (426us total) trans 0 rc 0/0 00000100:00100000:2.0:1776178442.593082:0:9554:0:(nrs_fifo.c:241:nrs_fifo_req_stop()) NRS stop fifo request from 12345-0@lo, seq: 581 00000100:00100000:1.0:1776178442.593181:0:3355:0:(client.c:2215:ptlrpc_check_set()) Completed RPC req@00000000829797f1 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_02:lustre-MDT0001-mdtlov_UUID:3355:1862456998082496:0@lo:13:osp-pre-1-1.0 00010000:00010000:3.0:1776178442.720344:0:6244:0:(ldlm_lockd.c:2792:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:2.0:1776178443.109486:0:3353:0:(client.c:1733:ptlrpc_send_new_req()) Sending RPC req@000000007e1b5a24 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_00:lustre-MDT0001-mdtlov_UUID:3353:1862456998082560:0@lo:13:osp-pre-0-1.0 00000100:00100000:2.0:1776178443.109582:0:3353:0:(events.c:361:request_in_callback()) peer: 12345-0@lo (source: 12345-0@lo) 00000100:00100000:0.0:1776178443.110129:0:8231:0:(service.c:2152:ptlrpc_server_handle_req_in()) got req x1862456998082560 00000100:00100000:0.0:1776178443.110195:0:8231:0:(nrs_fifo.c:179:nrs_fifo_req_get()) NRS start fifo request from 12345-0@lo, seq: 582 00000100:00100000:0.0:1776178443.110202:0:8231:0:(service.c:2324:ptlrpc_server_handle_request()) Handling RPC req@00000000a430b18f pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_create00:lustre-MDT0001-mdtlov_UUID+5:3353:x1862456998082560:12345-0@lo:13:osp-pre-0-1.0 00002000:00000024:0.0:1776178443.110235:0:8231:0:(ofd_obd.c:740:ofd_statfs()) blocks cached 315392 granted 484883776 pending 0 free 3923689472 avail 3692167168 00000100:00100000:0.0:1776178443.110307:0:8231:0:(service.c:2372:ptlrpc_server_handle_request()) Handled RPC req@00000000a430b18f pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_create00:lustre-MDT0001-mdtlov_UUID+5:3353:x1862456998082560:12345-0@lo:13:osp-pre-0-1.0 Request processed in 105us (727us total) trans 0 rc 0/0 00000100:00100000:0.0:1776178443.110315:0:8231:0:(nrs_fifo.c:241:nrs_fifo_req_stop()) NRS stop fifo request from 12345-0@lo, seq: 582 00000100:00100000:1.0:1776178443.114351:0:3353:0:(client.c:2215:ptlrpc_check_set()) Completed RPC req@000000007e1b5a24 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_00:lustre-MDT0001-mdtlov_UUID:3353:1862456998082560:0@lo:13:osp-pre-0-1.0 00010000:00010000:2.0:1776178443.745213:0:6245:0:(ldlm_lockd.c:2792:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:3.0:1776178444.583925:0:3348:0:(events.c:361:request_in_callback()) peer: 12345-192.168.204.3@tcp (source: 12345-192.168.204.3@tcp) 00000100:00100000:3.0:1776178444.583978:0:3348:0:(events.c:361:request_in_callback()) peer: 12345-192.168.204.3@tcp (source: 12345-192.168.204.3@tcp) 00000100:00100000:3.0:1776178444.584017:0:3348:0:(events.c:361:request_in_callback()) peer: 12345-192.168.204.3@tcp (source: 12345-192.168.204.3@tcp) 00000100:00100000:3.0:1776178444.584046:0:3348:0:(events.c:361:request_in_callback()) peer: 12345-192.168.204.3@tcp (source: 12345-192.168.204.3@tcp) 00000100:00100000:3.0:1776178444.584108:0:3348:0:(events.c:361:request_in_callback()) peer: 12345-192.168.204.3@tcp (source: 12345-192.168.204.3@tcp) 00000100:00100000:3.0:1776178444.584156:0:20137:0:(service.c:2152:ptlrpc_server_handle_req_in()) got req x1862456979124032 00000100:00100000:3.0:1776178444.584207:0:20137:0:(nrs_fifo.c:179:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.204.3@tcp, seq: 249 00000100:00100000:3.0:1776178444.584214:0:20137:0:(service.c:2324:ptlrpc_server_handle_request()) Handling RPC req@000000003e5a23bc pname:cluuid+ref:pid:xid:nid:opc:job ll_mgs_0000:5b5c66b1-1a2f-4651-8548-0c75b9a38a94+9:2270:x1862456979124032:12345-192.168.204.3@tcp:400:kworker/u8:3.0 00000100:00100000:3.0:1776178444.584285:0:20137:0:(service.c:2372:ptlrpc_server_handle_request()) Handled RPC req@000000003e5a23bc pname:cluuid+ref:pid:xid:nid:opc:job ll_mgs_0000:5b5c66b1-1a2f-4651-8548-0c75b9a38a94+9:2270:x1862456979124032:12345-192.168.204.3@tcp:400:kworker/u8:3.0 Request processed in 70us (267us total) trans 0 rc 0/0 00000100:00100000:3.0:1776178444.584294:0:20137:0:(nrs_fifo.c:241:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.204.3@tcp, seq: 249 00000100:00100000:3.0:1776178444.584331:0:6256:0:(service.c:2152:ptlrpc_server_handle_req_in()) got req x1862456979124160 00000100:00100000:3.0:1776178444.584356:0:6256:0:(service.c:2152:ptlrpc_server_handle_req_in()) got req x1862456979124096 00000100:00100000:3.0:1776178444.584370:0:6256:0:(nrs_fifo.c:179:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.204.3@tcp, seq: 1069 00000100:00100000:3.0:1776178444.584374:0:6256:0:(service.c:2324:ptlrpc_server_handle_request()) Handling RPC req@0000000092ddc73d pname:cluuid+ref:pid:xid:nid:opc:job mdt00_001:b50963bd-8eba-4432-9a54-8888d00e3fa1+6:2270:x1862456979124160:12345-192.168.204.3@tcp:400:kworker/u8:3.0 00000100:00100000:3.0:1776178444.584404:0:6256:0:(service.c:2372:ptlrpc_server_handle_request()) Handled RPC req@0000000092ddc73d pname:cluuid+ref:pid:xid:nid:opc:job mdt00_001:b50963bd-8eba-4432-9a54-8888d00e3fa1+6:2270:x1862456979124160:12345-192.168.204.3@tcp:400:kworker/u8:3.0 Request processed in 29us (492us total) trans 0 rc 0/0 00000100:00100000:3.0:1776178444.584410:0:6256:0:(nrs_fifo.c:241:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.204.3@tcp, seq: 1069 00000100:00100000:3.0:1776178444.584422:0:6256:0:(nrs_fifo.c:179:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.204.3@tcp, seq: 6812 00000100:00100000:3.0:1776178444.584426:0:6256:0:(service.c:2324:ptlrpc_server_handle_request()) Handling RPC req@000000005967fd64 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_001:0+-99:2269:x1862456979124096:12345-192.168.204.3@tcp:38:kworker/u8:3.0 00010000:02000000:3.0:1776178444.584439:0:6256:0:(ldlm_lib.c:1139:target_handle_connect()) lustre-MDT0000: Not available for connect from 192.168.204.3@tcp (stopping) 00000100:00100000:3.0:1776178444.584462:0:6256:0:(service.c:2372:ptlrpc_server_handle_request()) Handled RPC req@000000005967fd64 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_001:0+-99:2269:x1862456979124096:12345-192.168.204.3@tcp:38:kworker/u8:3.0 Request processed in 35us (354us total) trans 0 rc -19/-19 00000100:00100000:3.0:1776178444.584467:0:6256:0:(nrs_fifo.c:241:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.204.3@tcp, seq: 6812 00000100:00100000:3.0:1776178444.584486:0:20755:0:(service.c:2152:ptlrpc_server_handle_req_in()) got req x1862456979124288 00000100:00100000:3.0:1776178444.584506:0:20755:0:(service.c:2152:ptlrpc_server_handle_req_in()) got req x1862456979124224 00000100:00100000:3.0:1776178444.584521:0:20755:0:(nrs_fifo.c:179:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.204.3@tcp, seq: 699 00000100:00100000:3.0:1776178444.584524:0:20755:0:(service.c:2324:ptlrpc_server_handle_request()) Handling RPC req@00000000f064e279 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_006:b50963bd-8eba-4432-9a54-8888d00e3fa1+9:2270:x1862456979124288:12345-192.168.204.3@tcp:400:kworker/u8:3.0 00000100:00100000:2.0:1776178444.584653:0:20759:0:(nrs_fifo.c:179:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.204.3@tcp, seq: 700 00000100:00100000:2.0:1776178444.584659:0:20759:0:(service.c:2324:ptlrpc_server_handle_request()) Handling RPC req@00000000e837a7cd pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_010:b50963bd-8eba-4432-9a54-8888d00e3fa1+8:2270:x1862456979124224:12345-192.168.204.3@tcp:400:kworker/u8:3.0 00000100:00100000:2.0:1776178444.584689:0:20759:0:(service.c:2372:ptlrpc_server_handle_request()) Handled RPC req@00000000e837a7cd pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_010:b50963bd-8eba-4432-9a54-8888d00e3fa1+8:2270:x1862456979124224:12345-192.168.204.3@tcp:400:kworker/u8:3.0 Request processed in 30us (644us total) trans 0 rc 0/0 00000100:00100000:2.0:1776178444.584695:0:20759:0:(nrs_fifo.c:241:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.204.3@tcp, seq: 700 00000100:00100000:3.0:1776178444.585007:0:20755:0:(service.c:2372:ptlrpc_server_handle_request()) Handled RPC req@00000000f064e279 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_006:b50963bd-8eba-4432-9a54-8888d00e3fa1+9:2270:x1862456979124288:12345-192.168.204.3@tcp:400:kworker/u8:3.0 Request processed in 480us (1029us total) trans 0 rc 0/0 00000100:00100000:3.0:1776178444.585015:0:20755:0:(nrs_fifo.c:241:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.204.3@tcp, seq: 699 00010000:00010000:2.0:1776178444.769502:0:9224:0:(ldlm_lockd.c:2792:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00010000:00010000:1.0:1776178445.793139:0:9225:0:(ldlm_lockd.c:2792:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00080000:0.0:1776178446.176376:0:37:0:(pinger.c:251:ptlrpc_pinger_process_import()) lustre-MDT0000-lwp-OST0000_UUID->lustre-MDT0000_UUID: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00080000:0.0:1776178446.176386:0:37:0:(recover.c:57:ptlrpc_initiate_recovery()) lustre-MDT0000_UUID: starting recovery 00000100:00080000:0.0:1776178446.176392:0:37:0:(import.c:86:import_set_state_nolock()) 0000000047601a98 lustre-MDT0000_UUID: changing import state from DISCONN to CONNECTING 00000100:00080000:0.0:1776178446.176403:0:37:0:(import.c:534:import_select_connection()) lustre-MDT0000-lwp-OST0000: connect to NID 0@lo last attempt 1148 00000100:00080000:0.0:1776178446.176407:0:37:0:(import.c:581:import_select_connection()) lustre-MDT0000-lwp-OST0000: tried all connections, increasing latency to 21s 00000100:00080000:0.0:1776178446.176413:0:37:0:(import.c:615:import_select_connection()) lustre-MDT0000-lwp-OST0000: import 0000000047601a98 using connection 192.168.204.103@tcp/0@lo 00000100:00100000:0.0:1776178446.176438:0:37:0:(import.c:807:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5) req@000000007e1b5a24 x1862456998082688/t0(0) o38->lustre-MDT0000-lwp-OST0000@0@lo:12/10 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' 00000100:00080000:0.0:1776178446.176489:0:37:0:(pinger.c:251:ptlrpc_pinger_process_import()) lustre-MDT0000-lwp-OST0001_UUID->lustre-MDT0000_UUID: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00080000:0.0:1776178446.176493:0:37:0:(recover.c:57:ptlrpc_initiate_recovery()) lustre-MDT0000_UUID: starting recovery 00000100:00080000:0.0:1776178446.176494:0:37:0:(import.c:86:import_set_state_nolock()) 00000000dfc8db13 lustre-MDT0000_UUID: changing import state from DISCONN to CONNECTING 00000100:00080000:0.0:1776178446.176497:0:37:0:(import.c:534:import_select_connection()) lustre-MDT0000-lwp-OST0001: connect to NID 0@lo last attempt 1148 00000100:00080000:0.0:1776178446.176499:0:37:0:(import.c:581:import_select_connection()) lustre-MDT0000-lwp-OST0001: tried all connections, increasing latency to 21s 00000100:00080000:0.0:1776178446.176502:0:37:0:(import.c:615:import_select_connection()) lustre-MDT0000-lwp-OST0001: import 00000000dfc8db13 using connection 192.168.204.103@tcp/0@lo 00000100:00100000:0.0:1776178446.176511:0:37:0:(import.c:807:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5) req@000000009c4a59b3 x1862456998082816/t0(0) o38->lustre-MDT0000-lwp-OST0001@0@lo:12/10 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' 00000100:00080000:0.0:1776178446.176525:0:37:0:(pinger.c:251:ptlrpc_pinger_process_import()) lustre-MDT0001-mdtlov_UUID->lustre-MDT0000_UUID: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00080000:0.0:1776178446.176528:0:37:0:(recover.c:57:ptlrpc_initiate_recovery()) lustre-MDT0000_UUID: starting recovery 00000100:00080000:0.0:1776178446.176529:0:37:0:(import.c:86:import_set_state_nolock()) 00000000ba4f1821 lustre-MDT0000_UUID: changing import state from DISCONN to CONNECTING 00000100:00080000:0.0:1776178446.176532:0:37:0:(import.c:534:import_select_connection()) lustre-MDT0000-osp-MDT0001: connect to NID 0@lo last attempt 1148 00000100:00080000:0.0:1776178446.176534:0:37:0:(import.c:581:import_select_connection()) lustre-MDT0000-osp-MDT0001: tried all connections, increasing latency to 21s 00000100:00080000:0.0:1776178446.176537:0:37:0:(import.c:615:import_select_connection()) lustre-MDT0000-osp-MDT0001: import 00000000ba4f1821 using connection 192.168.204.103@tcp/0@lo 00000100:00100000:0.0:1776178446.176547:0:37:0:(import.c:807:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5) req@000000006a3d91b0 x1862456998082944/t0(0) o38->lustre-MDT0000-osp-MDT0001@0@lo:24/4 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' 00000100:00080000:0.0:1776178446.176565:0:37:0:(pinger.c:251:ptlrpc_pinger_process_import()) lustre-MDT0000-lwp-MDT0001_UUID->lustre-MDT0000_UUID: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00080000:0.0:1776178446.176568:0:37:0:(recover.c:57:ptlrpc_initiate_recovery()) lustre-MDT0000_UUID: starting recovery 00000100:00080000:0.0:1776178446.176569:0:37:0:(import.c:86:import_set_state_nolock()) 00000000a7c55b1f lustre-MDT0000_UUID: changing import state from DISCONN to CONNECTING 00000100:00080000:0.0:1776178446.176573:0:37:0:(import.c:534:import_select_connection()) lustre-MDT0000-lwp-MDT0001: connect to NID 0@lo last attempt 1148 00000100:00080000:0.0:1776178446.176575:0:37:0:(import.c:581:import_select_connection()) lustre-MDT0000-lwp-MDT0001: tried all connections, increasing latency to 21s 00000100:00080000:0.0:1776178446.176578:0:37:0:(import.c:615:import_select_connection()) lustre-MDT0000-lwp-MDT0001: import 00000000a7c55b1f using connection 192.168.204.103@tcp/0@lo 00000100:00100000:0.0:1776178446.176587:0:37:0:(import.c:807:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5) req@0000000042de09e9 x1862456998083136/t0(0) o38->lustre-MDT0000-lwp-MDT0001@0@lo:12/10 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' 00000100:00100000:0.0:1776178446.176638:0:3356:0:(client.c:1733:ptlrpc_send_new_req()) Sending RPC req@0000000075189f40 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_03:lustre-MDT0001-lwp-OST0001_UUID:3356:1862456998082880:0@lo:400:kworker/u8:2.0 00000100:00100000:0.0:1776178446.176728:0:3356:0:(events.c:361:request_in_callback()) peer: 12345-0@lo (source: 12345-0@lo) 00000100:00100000:1.0:1776178446.176857:0:3353:0:(client.c:1733:ptlrpc_send_new_req()) Sending RPC req@000000007216887e pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_00:lustre-MDT0001-mdtlov_UUID:3353:1862456998083008:0@lo:400:kworker/u8:2.0 00000100:00100000:2.0:1776178446.176916:0:3352:0:(client.c:1733:ptlrpc_send_new_req()) Sending RPC req@000000007e1b5a24 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:lustre-MDT0000-lwp-OST0000_UUID:3352:1862456998082688:0@lo:38:kworker/u8:2.0 00000100:00100000:3.0:1776178446.176980:0:3354:0:(client.c:1733:ptlrpc_send_new_req()) Sending RPC req@00000000597c3414 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_01:ac61f839-0e8e-4cac-9c59-08bcfbbad330:3354:1862456998082624:0@lo:400:kworker/u8:2.0 00000100:00100000:1.0:1776178446.180958:0:3353:0:(events.c:361:request_in_callback()) peer: 12345-0@lo (source: 12345-0@lo) 00000100:00100000:1.0:1776178446.181039:0:3355:0:(client.c:1733:ptlrpc_send_new_req()) Sending RPC req@00000000d0d837c5 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_02:lustre-MDT0001-lwp-OST0000_UUID:3355:1862456998082752:0@lo:400:kworker/u8:2.0 00000100:00100000:1.0:1776178446.181111:0:3355:0:(events.c:361:request_in_callback()) peer: 12345-0@lo (source: 12345-0@lo) 00000100:00100000:1.0:1776178446.181164:0:6255:0:(service.c:2152:ptlrpc_server_handle_req_in()) got req x1862456998082880 00000100:00100000:1.0:1776178446.181206:0:6255:0:(service.c:2152:ptlrpc_server_handle_req_in()) got req x1862456998082752 00000100:00100000:1.0:1776178446.181224:0:6255:0:(nrs_fifo.c:179:nrs_fifo_req_get()) NRS start fifo request from 12345-0@lo, seq: 1070 00000100:00100000:1.0:1776178446.181229:0:6255:0:(service.c:2324:ptlrpc_server_handle_request()) Handling RPC req@000000004ce46e20 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_000:lustre-MDT0001-lwp-OST0001_UUID+5:3356:x1862456998082880:12345-0@lo:400:kworker/u8:2.0 00000100:00100000:1.0:1776178446.181277:0:6255:0:(service.c:2372:ptlrpc_server_handle_request()) Handled RPC req@000000004ce46e20 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_000:lustre-MDT0001-lwp-OST0001_UUID+5:3356:x1862456998082880:12345-0@lo:400:kworker/u8:2.0 Request processed in 47us (4550us total) trans 0 rc 0/0 00000100:00100000:1.0:1776178446.181283:0:6255:0:(nrs_fifo.c:241:nrs_fifo_req_stop()) NRS stop fifo request from 12345-0@lo, seq: 1070 00000100:00100000:1.0:1776178446.181300:0:6255:0:(nrs_fifo.c:179:nrs_fifo_req_get()) NRS start fifo request from 12345-0@lo, seq: 1071 00000100:00100000:1.0:1776178446.181303:0:6255:0:(service.c:2324:ptlrpc_server_handle_request()) Handling RPC req@00000000f95e9da5 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_000:lustre-MDT0001-lwp-OST0000_UUID+5:3355:x1862456998082752:12345-0@lo:400:kworker/u8:2.0 00000100:00100000:1.0:1776178446.181330:0:6255:0:(service.c:2372:ptlrpc_server_handle_request()) Handled RPC req@00000000f95e9da5 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_000:lustre-MDT0001-lwp-OST0000_UUID+5:3355:x1862456998082752:12345-0@lo:400:kworker/u8:2.0 Request processed in 26us (220us total) trans 0 rc 0/0 00000100:00100000:1.0:1776178446.181335:0:6255:0:(nrs_fifo.c:241:nrs_fifo_req_stop()) NRS stop fifo request from 12345-0@lo, seq: 1071 00000100:00100000:1.0:1776178446.181373:0:3355:0:(client.c:2215:ptlrpc_check_set()) Completed RPC req@00000000d0d837c5 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_02:lustre-MDT0001-lwp-OST0000_UUID:3355:1862456998082752:0@lo:400:kworker/u8:2.0 00000100:00100000:1.0:1776178446.181427:0:20755:0:(service.c:2152:ptlrpc_server_handle_req_in()) got req x1862456998083008 00000100:00100000:1.0:1776178446.181453:0:20755:0:(nrs_fifo.c:179:nrs_fifo_req_get()) NRS start fifo request from 12345-0@lo, seq: 701 00000100:00100000:1.0:1776178446.181457:0:20755:0:(service.c:2324:ptlrpc_server_handle_request()) Handling RPC req@0000000047458d9f pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_006:lustre-MDT0001-mdtlov_UUID+5:3353:x1862456998083008:12345-0@lo:400:kworker/u8:2.0 00000100:00100000:1.0:1776178446.181489:0:20755:0:(service.c:2372:ptlrpc_server_handle_request()) Handled RPC req@0000000047458d9f pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_006:lustre-MDT0001-mdtlov_UUID+5:3353:x1862456998083008:12345-0@lo:400:kworker/u8:2.0 Request processed in 32us (541us total) trans 0 rc 0/0 00000100:00100000:1.0:1776178446.181494:0:20755:0:(nrs_fifo.c:241:nrs_fifo_req_stop()) NRS stop fifo request from 12345-0@lo, seq: 701 00000100:00100000:1.0:1776178446.181522:0:3353:0:(client.c:2215:ptlrpc_check_set()) Completed RPC req@000000007216887e pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_00:lustre-MDT0001-mdtlov_UUID:3353:1862456998083008:0@lo:400:kworker/u8:2.0 00000100:00100000:2.0:1776178446.184219:0:3352:0:(events.c:361:request_in_callback()) peer: 12345-0@lo (source: 12345-0@lo) 00000100:00100000:2.0:1776178446.184264:0:3352:0:(client.c:1733:ptlrpc_send_new_req()) Sending RPC req@000000009c4a59b3 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:lustre-MDT0000-lwp-OST0001_UUID:3352:1862456998082816:0@lo:38:kworker/u8:2.0 00000100:00100000:2.0:1776178446.184311:0:3352:0:(events.c:361:request_in_callback()) peer: 12345-0@lo (source: 12345-0@lo) 00000100:00100000:2.0:1776178446.184330:0:3352:0:(client.c:1733:ptlrpc_send_new_req()) Sending RPC req@000000006a3d91b0 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:lustre-MDT0001-mdtlov_UUID:3352:1862456998082944:0@lo:38:kworker/u8:2.0 00000100:00100000:2.0:1776178446.184354:0:3352:0:(events.c:361:request_in_callback()) peer: 12345-0@lo (source: 12345-0@lo) 00000100:00100000:2.0:1776178446.184372:0:3352:0:(client.c:1733:ptlrpc_send_new_req()) Sending RPC req@0000000042de09e9 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:lustre-MDT0000-lwp-MDT0001_UUID:3352:1862456998083136:0@lo:38:kworker/u8:2.0 00000100:00100000:2.0:1776178446.184392:0:3352:0:(events.c:361:request_in_callback()) peer: 12345-0@lo (source: 12345-0@lo) 00000100:00100000:2.0:1776178446.184443:0:11281:0:(service.c:2152:ptlrpc_server_handle_req_in()) got req x1862456998082688 00000100:00100000:2.0:1776178446.184477:0:11281:0:(service.c:2152:ptlrpc_server_handle_req_in()) got req x1862456998082816 00000100:00100000:2.0:1776178446.184489:0:11281:0:(service.c:2152:ptlrpc_server_handle_req_in()) got req x1862456998083136 00000100:00100000:2.0:1776178446.184499:0:11281:0:(nrs_fifo.c:179:nrs_fifo_req_get()) NRS start fifo request from 12345-0@lo, seq: 6813 00000100:00100000:2.0:1776178446.184504:0:11281:0:(service.c:2324:ptlrpc_server_handle_request()) Handling RPC req@0000000078fb1a26 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_004:0+-99:3352:x1862456998082688:12345-0@lo:38:kworker/u8:2.0 00010000:02000000:2.0:1776178446.184518:0:11281:0:(ldlm_lib.c:1139:target_handle_connect()) lustre-MDT0000: Not available for connect from 0@lo (stopping) 00000100:00100000:2.0:1776178446.184555:0:11281:0:(service.c:2372:ptlrpc_server_handle_request()) Handled RPC req@0000000078fb1a26 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_004:0+-99:3352:x1862456998082688:12345-0@lo:38:kworker/u8:2.0 Request processed in 49us (346us total) trans 0 rc -19/-19 00000100:00100000:2.0:1776178446.184562:0:11281:0:(nrs_fifo.c:241:nrs_fifo_req_stop()) NRS stop fifo request from 12345-0@lo, seq: 6813 00000100:00100000:2.0:1776178446.184578:0:11281:0:(nrs_fifo.c:179:nrs_fifo_req_get()) NRS start fifo request from 12345-0@lo, seq: 6814 00000100:00100000:2.0:1776178446.184581:0:11281:0:(service.c:2324:ptlrpc_server_handle_request()) Handling RPC req@00000000a9455d7d pname:cluuid+ref:pid:xid:nid:opc:job mdt00_004:0+-99:3352:x1862456998082816:12345-0@lo:38:kworker/u8:2.0 00010000:02000000:2.0:1776178446.184587:0:11281:0:(ldlm_lib.c:1139:target_handle_connect()) lustre-MDT0000: Not available for connect from 0@lo (stopping) 00000100:00100000:2.0:1776178446.184608:0:11281:0:(service.c:2372:ptlrpc_server_handle_request()) Handled RPC req@00000000a9455d7d pname:cluuid+ref:pid:xid:nid:opc:job mdt00_004:0+-99:3352:x1862456998082816:12345-0@lo:38:kworker/u8:2.0 Request processed in 25us (296us total) trans 0 rc -19/-19 00000100:00100000:2.0:1776178446.184613:0:11281:0:(nrs_fifo.c:241:nrs_fifo_req_stop()) NRS stop fifo request from 12345-0@lo, seq: 6814 00000100:00100000:2.0:1776178446.184623:0:11281:0:(nrs_fifo.c:179:nrs_fifo_req_get()) NRS start fifo request from 12345-0@lo, seq: 6815 00000100:00100000:2.0:1776178446.184987:0:11281:0:(service.c:2324:ptlrpc_server_handle_request()) Handling RPC req@00000000a0a4914c pname:cluuid+ref:pid:xid:nid:opc:job mdt00_004:0+-99:3352:x1862456998083136:12345-0@lo:38:kworker/u8:2.0 00010000:02000000:2.0:1776178446.184995:0:11281:0:(ldlm_lib.c:1139:target_handle_connect()) lustre-MDT0000: Not available for connect from 0@lo (stopping) 00000100:00100000:2.0:1776178446.185020:0:11281:0:(service.c:2372:ptlrpc_server_handle_request()) Handled RPC req@00000000a0a4914c pname:cluuid+ref:pid:xid:nid:opc:job mdt00_004:0+-99:3352:x1862456998083136:12345-0@lo:38:kworker/u8:2.0 Request processed in 35us (627us total) trans 0 rc -19/-19 00000100:00100000:2.0:1776178446.185026:0:11281:0:(nrs_fifo.c:241:nrs_fifo_req_stop()) NRS stop fifo request from 12345-0@lo, seq: 6815 00000100:00080000:2.0:1776178446.185102:0:3352:0:(import.c:86:import_set_state_nolock()) 0000000047601a98 lustre-MDT0000_UUID: changing import state from CONNECTING to DISCONN 00000100:00080000:2.0:1776178446.185107:0:3352:0:(import.c:1429:ptlrpc_connect_interpret()) recovery of lustre-MDT0000_UUID on 192.168.204.103@tcp failed (-19) 00000100:00100000:2.0:1776178446.185111:0:3352:0:(client.c:2215:ptlrpc_check_set()) Completed RPC req@000000007e1b5a24 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:lustre-MDT0000-lwp-OST0000_UUID:3352:1862456998082688:0@lo:38:kworker/u8:2.0 00000100:00080000:2.0:1776178446.185121:0:3352:0:(import.c:86:import_set_state_nolock()) 00000000dfc8db13 lustre-MDT0000_UUID: changing import state from CONNECTING to DISCONN 00000100:00080000:2.0:1776178446.185123:0:3352:0:(import.c:1429:ptlrpc_connect_interpret()) recovery of lustre-MDT0000_UUID on 192.168.204.103@tcp failed (-19) 00000100:00100000:2.0:1776178446.185125:0:3352:0:(client.c:2215:ptlrpc_check_set()) Completed RPC req@000000009c4a59b3 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:lustre-MDT0000-lwp-OST0001_UUID:3352:1862456998082816:0@lo:38:kworker/u8:2.0 00000100:00080000:2.0:1776178446.185132:0:3352:0:(import.c:86:import_set_state_nolock()) 00000000a7c55b1f lustre-MDT0000_UUID: changing import state from CONNECTING to DISCONN 00000100:00080000:2.0:1776178446.185134:0:3352:0:(import.c:1429:ptlrpc_connect_interpret()) recovery of lustre-MDT0000_UUID on 192.168.204.103@tcp failed (-19) 00000100:00100000:2.0:1776178446.185137:0:3352:0:(client.c:2215:ptlrpc_check_set()) Completed RPC req@0000000042de09e9 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:lustre-MDT0000-lwp-MDT0001_UUID:3352:1862456998083136:0@lo:38:kworker/u8:2.0 00000100:00100000:2.0:1776178446.185171:0:15213:0:(service.c:2152:ptlrpc_server_handle_req_in()) got req x1862456998082944 00000100:00100000:2.0:1776178446.185194:0:15213:0:(nrs_fifo.c:179:nrs_fifo_req_get()) NRS start fifo request from 12345-0@lo, seq: 755 00000100:00100000:2.0:1776178446.185198:0:15213:0:(service.c:2324:ptlrpc_server_handle_request()) Handling RPC req@000000000489d393 pname:cluuid+ref:pid:xid:nid:opc:job mdt_out00_003:0+-99:3352:x1862456998082944:12345-0@lo:38:kworker/u8:2.0 00010000:02000000:2.0:1776178446.185205:0:15213:0:(ldlm_lib.c:1139:target_handle_connect()) lustre-MDT0000: Not available for connect from 0@lo (stopping) 00000100:00100000:2.0:1776178446.185232:0:15213:0:(service.c:2372:ptlrpc_server_handle_request()) Handled RPC req@000000000489d393 pname:cluuid+ref:pid:xid:nid:opc:job mdt_out00_003:0+-99:3352:x1862456998082944:12345-0@lo:38:kworker/u8:2.0 Request processed in 33us (878us total) trans 0 rc -19/-19 00000100:00100000:2.0:1776178446.185238:0:15213:0:(nrs_fifo.c:241:nrs_fifo_req_stop()) NRS stop fifo request from 12345-0@lo, seq: 755 00000100:00080000:2.0:1776178446.185257:0:3352:0:(import.c:86:import_set_state_nolock()) 00000000ba4f1821 lustre-MDT0000_UUID: changing import state from CONNECTING to DISCONN 00000100:00080000:2.0:1776178446.185260:0:3352:0:(import.c:1429:ptlrpc_connect_interpret()) recovery of lustre-MDT0000_UUID on 192.168.204.103@tcp failed (-19) 00000100:00100000:2.0:1776178446.185262:0:3352:0:(client.c:2215:ptlrpc_check_set()) Completed RPC req@000000006a3d91b0 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:lustre-MDT0001-mdtlov_UUID:3352:1862456998082944:0@lo:38:kworker/u8:2.0 00000100:00100000:3.0:1776178446.189203:0:3354:0:(events.c:361:request_in_callback()) peer: 12345-0@lo (source: 12345-0@lo) 00000100:00100000:3.0:1776178446.189254:0:3354:0:(client.c:1733:ptlrpc_send_new_req()) Sending RPC req@00000000e8195050 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_01:lustre-MDT0001-mdtlov_UUID:3354:1862456998083072:0@lo:400:kworker/u8:2.0 00000100:00100000:3.0:1776178446.189302:0:3354:0:(events.c:361:request_in_callback()) peer: 12345-0@lo (source: 12345-0@lo) 00000100:00100000:1.0:1776178446.190807:0:20140:0:(service.c:2152:ptlrpc_server_handle_req_in()) got req x1862456998082624 00000100:00100000:1.0:1776178446.190876:0:20140:0:(nrs_fifo.c:179:nrs_fifo_req_get()) NRS start fifo request from 12345-0@lo, seq: 250 00000100:00100000:1.0:1776178446.190883:0:20140:0:(service.c:2324:ptlrpc_server_handle_request()) Handling RPC req@00000000fc8eef33 pname:cluuid+ref:pid:xid:nid:opc:job ll_mgs_0002:ac61f839-0e8e-4cac-9c59-08bcfbbad330+18:3354:x1862456998082624:12345-0@lo:400:kworker/u8:2.0 00000100:00100000:1.0:1776178446.190969:0:20140:0:(service.c:2372:ptlrpc_server_handle_request()) Handled RPC req@00000000fc8eef33 pname:cluuid+ref:pid:xid:nid:opc:job ll_mgs_0002:ac61f839-0e8e-4cac-9c59-08bcfbbad330+18:3354:x1862456998082624:12345-0@lo:400:kworker/u8:2.0 Request processed in 86us (1776us total) trans 0 rc 0/0 00000100:00100000:1.0:1776178446.190976:0:20140:0:(nrs_fifo.c:241:nrs_fifo_req_stop()) NRS stop fifo request from 12345-0@lo, seq: 250 00000100:00100000:3.0:1776178446.191237:0:3354:0:(client.c:2215:ptlrpc_check_set()) Completed RPC req@00000000597c3414 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_01:ac61f839-0e8e-4cac-9c59-08bcfbbad330:3354:1862456998082624:0@lo:400:kworker/u8:2.0 00000100:00100000:2.0:1776178446.191533:0:20762:0:(service.c:2152:ptlrpc_server_handle_req_in()) got req x1862456998083072 00000100:00100000:2.0:1776178446.191571:0:20762:0:(nrs_fifo.c:179:nrs_fifo_req_get()) NRS start fifo request from 12345-0@lo, seq: 702 00000100:00100000:2.0:1776178446.191576:0:20762:0:(service.c:2324:ptlrpc_server_handle_request()) Handling RPC req@00000000002be227 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_013:lustre-MDT0001-mdtlov_UUID+5:3354:x1862456998083072:12345-0@lo:400:kworker/u8:2.0 00000100:00100000:2.0:1776178446.191635:0:20762:0:(service.c:2372:ptlrpc_server_handle_request()) Handled RPC req@00000000002be227 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_013:lustre-MDT0001-mdtlov_UUID+5:3354:x1862456998083072:12345-0@lo:400:kworker/u8:2.0 Request processed in 58us (2333us total) trans 0 rc 0/0 00000100:00100000:2.0:1776178446.191642:0:20762:0:(nrs_fifo.c:241:nrs_fifo_req_stop()) NRS stop fifo request from 12345-0@lo, seq: 702 00000100:00100000:3.0:1776178446.191879:0:3354:0:(client.c:2215:ptlrpc_check_set()) Completed RPC req@00000000e8195050 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_01:lustre-MDT0001-mdtlov_UUID:3354:1862456998083072:0@lo:400:kworker/u8:2.0 00000100:00100000:0.0:1776178446.198468:0:3356:0:(client.c:2215:ptlrpc_check_set()) Completed RPC req@0000000075189f40 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_03:lustre-MDT0001-lwp-OST0001_UUID:3356:1862456998082880:0@lo:400:kworker/u8:2.0 00010000:00010000:3.0:1776178446.816660:0:6244:0:(ldlm_lockd.c:2792:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:1.0:1776178447.717102:0:3355:0:(client.c:1733:ptlrpc_send_new_req()) Sending RPC req@0000000095214b8e pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_02:lustre-MDT0001-mdtlov_UUID:3355:1862456998083200:0@lo:13:osp-pre-1-1.0 00000100:00100000:1.0:1776178447.717193:0:3355:0:(events.c:361:request_in_callback()) peer: 12345-0@lo (source: 12345-0@lo) 00000100:00100000:2.0:1776178447.717340:0:9554:0:(service.c:2152:ptlrpc_server_handle_req_in()) got req x1862456998083200 00000100:00100000:2.0:1776178447.717402:0:9554:0:(nrs_fifo.c:179:nrs_fifo_req_get()) NRS start fifo request from 12345-0@lo, seq: 583 00000100:00100000:2.0:1776178447.717409:0:9554:0:(service.c:2324:ptlrpc_server_handle_request()) Handling RPC req@00000000dd1281fe pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_create00:lustre-MDT0001-mdtlov_UUID+5:3355:x1862456998083200:12345-0@lo:13:osp-pre-1-1.0 00002000:00000024:2.0:1776178447.717441:0:9554:0:(ofd_obd.c:740:ofd_statfs()) blocks cached 229376 granted 491746368 pending 0 free 3923689472 avail 3692167168 00000100:00100000:2.0:1776178447.717496:0:9554:0:(service.c:2372:ptlrpc_server_handle_request()) Handled RPC req@00000000dd1281fe pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_create00:lustre-MDT0001-mdtlov_UUID+5:3355:x1862456998083200:12345-0@lo:13:osp-pre-1-1.0 Request processed in 85us (303us total) trans 0 rc 0/0 00000100:00100000:2.0:1776178447.717503:0:9554:0:(nrs_fifo.c:241:nrs_fifo_req_stop()) NRS stop fifo request from 12345-0@lo, seq: 583 00000100:00100000:1.0:1776178447.717675:0:3355:0:(client.c:2215:ptlrpc_check_set()) Completed RPC req@0000000095214b8e pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_02:lustre-MDT0001-mdtlov_UUID:3355:1862456998083200:0@lo:13:osp-pre-1-1.0 00010000:00010000:2.0:1776178447.840991:0:6245:0:(ldlm_lockd.c:2792:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:1.0:1776178448.224459:0:3355:0:(ptlrpcd.c:415:ptlrpcd_check()) transfer 1 async RPCs [3->2] 00000100:00100000:1.0:1776178448.224489:0:3355:0:(client.c:1733:ptlrpc_send_new_req()) Sending RPC req@0000000075189f40 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_02:lustre-MDT0001-mdtlov_UUID:3355:1862456998083264:0@lo:13:osp-pre-0-1.0 00000100:00100000:1.0:1776178448.224563:0:3355:0:(events.c:361:request_in_callback()) peer: 12345-0@lo (source: 12345-0@lo) 00000100:00100000:3.0:1776178448.224683:0:8231:0:(service.c:2152:ptlrpc_server_handle_req_in()) got req x1862456998083264 00000100:00100000:3.0:1776178448.224753:0:8231:0:(nrs_fifo.c:179:nrs_fifo_req_get()) NRS start fifo request from 12345-0@lo, seq: 584 00000100:00100000:3.0:1776178448.224759:0:8231:0:(service.c:2324:ptlrpc_server_handle_request()) Handling RPC req@00000000b5702a7a pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_create00:lustre-MDT0001-mdtlov_UUID+5:3355:x1862456998083264:12345-0@lo:13:osp-pre-0-1.0 00002000:00000024:3.0:1776178448.224790:0:8231:0:(ofd_obd.c:740:ofd_statfs()) blocks cached 315392 granted 484883776 pending 0 free 3923689472 avail 3692167168 00000100:00100000:3.0:1776178448.224843:0:8231:0:(service.c:2372:ptlrpc_server_handle_request()) Handled RPC req@00000000b5702a7a pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_create00:lustre-MDT0001-mdtlov_UUID+5:3355:x1862456998083264:12345-0@lo:13:osp-pre-0-1.0 Request processed in 82us (281us total) trans 0 rc 0/0 00000100:00100000:3.0:1776178448.224849:0:8231:0:(nrs_fifo.c:241:nrs_fifo_req_stop()) NRS stop fifo request from 12345-0@lo, seq: 584 00000100:00100000:0.0:1776178448.225128:0:3355:0:(client.c:2215:ptlrpc_check_set()) Completed RPC req@0000000075189f40 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_02:lustre-MDT0001-mdtlov_UUID:3355:1862456998083264:0@lo:13:osp-pre-0-1.0 00010000:00010000:1.0:1776178448.864458:0:9224:0:(ldlm_lockd.c:2792:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:3.0:1776178449.698588:0:3348:0:(events.c:361:request_in_callback()) peer: 12345-192.168.204.3@tcp (source: 12345-192.168.204.3@tcp) 00000100:00100000:3.0:1776178449.698658:0:3348:0:(events.c:361:request_in_callback()) peer: 12345-192.168.204.3@tcp (source: 12345-192.168.204.3@tcp) 00000100:00100000:3.0:1776178449.698695:0:3348:0:(events.c:361:request_in_callback()) peer: 12345-192.168.204.3@tcp (source: 12345-192.168.204.3@tcp) 00000100:00100000:3.0:1776178449.698731:0:3348:0:(events.c:361:request_in_callback()) peer: 12345-192.168.204.3@tcp (source: 12345-192.168.204.3@tcp) 00000100:00100000:3.0:1776178449.698760:0:3348:0:(events.c:361:request_in_callback()) peer: 12345-192.168.204.3@tcp (source: 12345-192.168.204.3@tcp) 00000100:00100000:3.0:1776178449.698812:0:20762:0:(service.c:2152:ptlrpc_server_handle_req_in()) got req x1862456979124544 00000100:00100000:3.0:1776178449.698863:0:20762:0:(service.c:2152:ptlrpc_server_handle_req_in()) got req x1862456979124608 00000100:00100000:3.0:1776178449.698887:0:20762:0:(nrs_fifo.c:179:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.204.3@tcp, seq: 703 00000100:00100000:3.0:1776178449.698894:0:20762:0:(service.c:2324:ptlrpc_server_handle_request()) Handling RPC req@0000000095214b8e pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_013:b50963bd-8eba-4432-9a54-8888d00e3fa1+8:2271:x1862456979124544:12345-192.168.204.3@tcp:400:kworker/u8:4.0 00000100:00100000:3.0:1776178449.698986:0:20762:0:(service.c:2372:ptlrpc_server_handle_request()) Handled RPC req@0000000095214b8e pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_013:b50963bd-8eba-4432-9a54-8888d00e3fa1+8:2271:x1862456979124544:12345-192.168.204.3@tcp:400:kworker/u8:4.0 Request processed in 92us (291us total) trans 0 rc 0/0 00000100:00100000:3.0:1776178449.698994:0:20762:0:(nrs_fifo.c:241:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.204.3@tcp, seq: 703 00000100:00100000:3.0:1776178449.699011:0:20762:0:(nrs_fifo.c:179:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.204.3@tcp, seq: 704 00000100:00100000:3.0:1776178449.699015:0:20762:0:(service.c:2324:ptlrpc_server_handle_request()) Handling RPC req@00000000829797f1 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_013:b50963bd-8eba-4432-9a54-8888d00e3fa1+9:2270:x1862456979124608:12345-192.168.204.3@tcp:400:kworker/u8:4.0 00000100:00100000:3.0:1776178449.699051:0:20762:0:(service.c:2372:ptlrpc_server_handle_request()) Handled RPC req@00000000829797f1 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_013:b50963bd-8eba-4432-9a54-8888d00e3fa1+9:2270:x1862456979124608:12345-192.168.204.3@tcp:400:kworker/u8:4.0 Request processed in 35us (321us total) trans 0 rc 0/0 00000100:00100000:3.0:1776178449.699057:0:20762:0:(nrs_fifo.c:241:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.204.3@tcp, seq: 704 00000100:00100000:3.0:1776178449.699148:0:6257:0:(service.c:2152:ptlrpc_server_handle_req_in()) got req x1862456979124480 00000100:00100000:3.0:1776178449.699178:0:6257:0:(service.c:2152:ptlrpc_server_handle_req_in()) got req x1862456979124416 00000100:00100000:3.0:1776178449.699195:0:6257:0:(nrs_fifo.c:179:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.204.3@tcp, seq: 1072 00000100:00100000:3.0:1776178449.699200:0:6257:0:(service.c:2324:ptlrpc_server_handle_request()) Handling RPC req@00000000a8c8cb2d pname:cluuid+ref:pid:xid:nid:opc:job mdt00_002:b50963bd-8eba-4432-9a54-8888d00e3fa1+6:2270:x1862456979124480:12345-192.168.204.3@tcp:400:kworker/u8:4.0 00000100:00100000:3.0:1776178449.699235:0:6257:0:(service.c:2372:ptlrpc_server_handle_request()) Handled RPC req@00000000a8c8cb2d pname:cluuid+ref:pid:xid:nid:opc:job mdt00_002:b50963bd-8eba-4432-9a54-8888d00e3fa1+6:2270:x1862456979124480:12345-192.168.204.3@tcp:400:kworker/u8:4.0 Request processed in 35us (659us total) trans 0 rc 0/0 00000100:00100000:3.0:1776178449.699243:0:6257:0:(nrs_fifo.c:241:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.204.3@tcp, seq: 1072 00000100:00100000:3.0:1776178449.699255:0:6257:0:(nrs_fifo.c:179:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.204.3@tcp, seq: 6816 00000100:00100000:3.0:1776178449.699259:0:6257:0:(service.c:2324:ptlrpc_server_handle_request()) Handling RPC req@000000007af1bdf7 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_002:0+-99:2269:x1862456979124416:12345-192.168.204.3@tcp:38:kworker/u8:4.0 00010000:02000000:3.0:1776178449.699275:0:6257:0:(ldlm_lib.c:1139:target_handle_connect()) lustre-MDT0000: Not available for connect from 192.168.204.3@tcp (stopping) 00000100:00100000:3.0:1776178449.699305:0:6257:0:(service.c:2372:ptlrpc_server_handle_request()) Handled RPC req@000000007af1bdf7 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_002:0+-99:2269:x1862456979124416:12345-192.168.204.3@tcp:38:kworker/u8:4.0 Request processed in 44us (544us total) trans 0 rc -19/-19 00000100:00100000:3.0:1776178449.699311:0:6257:0:(nrs_fifo.c:241:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.204.3@tcp, seq: 6816 00000100:00100000:0.0:1776178449.699819:0:20137:0:(service.c:2152:ptlrpc_server_handle_req_in()) got req x1862456979124352 00000100:00100000:0.0:1776178449.699861:0:20137:0:(nrs_fifo.c:179:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.204.3@tcp, seq: 251 00000100:00100000:0.0:1776178449.699865:0:20137:0:(service.c:2324:ptlrpc_server_handle_request()) Handling RPC req@0000000044c7032a pname:cluuid+ref:pid:xid:nid:opc:job ll_mgs_0000:5b5c66b1-1a2f-4651-8548-0c75b9a38a94+9:2271:x1862456979124352:12345-192.168.204.3@tcp:400:kworker/u8:4.0 00000100:00100000:0.0:1776178449.699909:0:20137:0:(service.c:2372:ptlrpc_server_handle_request()) Handled RPC req@0000000044c7032a pname:cluuid+ref:pid:xid:nid:opc:job ll_mgs_0000:5b5c66b1-1a2f-4651-8548-0c75b9a38a94+9:2271:x1862456979124352:12345-192.168.204.3@tcp:400:kworker/u8:4.0 Request processed in 43us (1251us total) trans 0 rc 0/0 00000100:00100000:0.0:1776178449.699915:0:20137:0:(nrs_fifo.c:241:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.204.3@tcp, seq: 251 00010000:00010000:1.0:1776178449.888513:0:9225:0:(ldlm_lockd.c:2792:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00010000:00010000:3.0:1776178450.912344:0:6244:0:(ldlm_lockd.c:2792:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:0.0:1776178451.297436:0:3354:0:(ptlrpcd.c:415:ptlrpcd_check()) transfer 1 async RPCs [0->1] 00000100:00080000:3.0:1776178451.297460:0:36:0:(pinger.c:251:ptlrpc_pinger_process_import()) lustre-MDT0000-lwp-OST0000_UUID->lustre-MDT0000_UUID: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00080000:3.0:1776178451.297466:0:36:0:(recover.c:57:ptlrpc_initiate_recovery()) lustre-MDT0000_UUID: starting recovery 00000100:00080000:3.0:1776178451.297470:0:36:0:(import.c:86:import_set_state_nolock()) 0000000047601a98 lustre-MDT0000_UUID: changing import state from DISCONN to CONNECTING 00000100:00080000:3.0:1776178451.297479:0:36:0:(import.c:534:import_select_connection()) lustre-MDT0000-lwp-OST0000: connect to NID 0@lo last attempt 1153 00000100:00080000:3.0:1776178451.297481:0:36:0:(import.c:581:import_select_connection()) lustre-MDT0000-lwp-OST0000: tried all connections, increasing latency to 21s 00000100:00080000:3.0:1776178451.297488:0:36:0:(import.c:615:import_select_connection()) lustre-MDT0000-lwp-OST0000: import 0000000047601a98 using connection 192.168.204.103@tcp/0@lo 00000100:00100000:3.0:1776178451.297517:0:36:0:(import.c:807:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5) req@000000007d6ef583 x1862456998083392/t0(0) o38->lustre-MDT0000-lwp-OST0000@0@lo:12/10 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' 00000100:00100000:1.0:1776178451.297604:0:3353:0:(ptlrpcd.c:415:ptlrpcd_check()) transfer 1 async RPCs [1->0] 00000100:00100000:2.0:1776178451.297620:0:3352:0:(client.c:1733:ptlrpc_send_new_req()) Sending RPC req@000000007d6ef583 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:lustre-MDT0000-lwp-OST0000_UUID:3352:1862456998083392:0@lo:38:kworker/u8:1.0 00000100:00100000:1.0:1776178451.297625:0:3353:0:(client.c:1733:ptlrpc_send_new_req()) Sending RPC req@00000000f351bcc3 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_00:lustre-MDT0001-lwp-OST0000_UUID:3353:1862456998083456:0@lo:400:kworker/u8:1.0 00000100:00100000:1.0:1776178451.297716:0:3353:0:(events.c:361:request_in_callback()) peer: 12345-0@lo (source: 12345-0@lo) 00000100:00100000:2.0:1776178451.297764:0:6255:0:(service.c:2152:ptlrpc_server_handle_req_in()) got req x1862456998083456 00000100:00100000:1.0:1776178451.297771:0:3352:0:(events.c:361:request_in_callback()) peer: 12345-0@lo (source: 12345-0@lo) 00000100:00100000:2.0:1776178451.297813:0:6255:0:(service.c:2152:ptlrpc_server_handle_req_in()) got req x1862456998083392 00000100:00100000:0.0:1776178451.299463:0:3354:0:(client.c:1733:ptlrpc_send_new_req()) Sending RPC req@0000000047a42b33 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_01:ac61f839-0e8e-4cac-9c59-08bcfbbad330:3354:1862456998083328:0@lo:400:kworker/u8:1.0 00000100:00100000:0.0:1776178451.299554:0:3354:0:(events.c:361:request_in_callback()) peer: 12345-0@lo (source: 12345-0@lo) 00000100:00100000:1.0:1776178451.299903:0:8250:0:(nrs_fifo.c:179:nrs_fifo_req_get()) NRS start fifo request from 12345-0@lo, seq: 1073 00000100:00100000:1.0:1776178451.299915:0:8250:0:(service.c:2324:ptlrpc_server_handle_request()) Handling RPC req@0000000018b5e3c6 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_003:lustre-MDT0001-lwp-OST0000_UUID+5:3353:x1862456998083456:12345-0@lo:400:kworker/u8:1.0 00000100:00100000:1.0:1776178451.300004:0:8250:0:(service.c:2372:ptlrpc_server_handle_request()) Handled RPC req@0000000018b5e3c6 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_003:lustre-MDT0001-lwp-OST0000_UUID+5:3353:x1862456998083456:12345-0@lo:400:kworker/u8:1.0 Request processed in 91us (2290us total) trans 0 rc 0/0 00000100:00100000:1.0:1776178451.300012:0:8250:0:(nrs_fifo.c:241:nrs_fifo_req_stop()) NRS stop fifo request from 12345-0@lo, seq: 1073 00000100:00100000:1.0:1776178451.300037:0:8250:0:(nrs_fifo.c:179:nrs_fifo_req_get()) NRS start fifo request from 12345-0@lo, seq: 6817 00000100:00100000:1.0:1776178451.300042:0:8250:0:(service.c:2324:ptlrpc_server_handle_request()) Handling RPC req@0000000043a010c5 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_003:0+-99:3352:x1862456998083392:12345-0@lo:38:kworker/u8:1.0 00010000:02000000:1.0:1776178451.300058:0:8250:0:(ldlm_lib.c:1139:target_handle_connect()) lustre-MDT0000: Not available for connect from 0@lo (stopping) 00000100:00100000:1.0:1776178451.300116:0:8250:0:(service.c:2372:ptlrpc_server_handle_request()) Handled RPC req@0000000043a010c5 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_003:0+-99:3352:x1862456998083392:12345-0@lo:38:kworker/u8:1.0 Request processed in 73us (2346us total) trans 0 rc -19/-19 00000100:00100000:1.0:1776178451.300123:0:8250:0:(nrs_fifo.c:241:nrs_fifo_req_stop()) NRS stop fifo request from 12345-0@lo, seq: 6817 00000100:00080000:1.0:1776178451.300205:0:3352:0:(import.c:86:import_set_state_nolock()) 0000000047601a98 lustre-MDT0000_UUID: changing import state from CONNECTING to DISCONN 00000100:00080000:1.0:1776178451.300210:0:3352:0:(import.c:1429:ptlrpc_connect_interpret()) recovery of lustre-MDT0000_UUID on 192.168.204.103@tcp failed (-19) 00000100:00100000:1.0:1776178451.300214:0:3352:0:(client.c:2215:ptlrpc_check_set()) Completed RPC req@000000007d6ef583 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:lustre-MDT0000-lwp-OST0000_UUID:3352:1862456998083392:0@lo:38:kworker/u8:1.0 00000100:00100000:1.0:1776178451.300255:0:3353:0:(client.c:2215:ptlrpc_check_set()) Completed RPC req@00000000f351bcc3 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_00:lustre-MDT0001-lwp-OST0000_UUID:3353:1862456998083456:0@lo:400:kworker/u8:1.0 00000100:00100000:1.0:1776178451.300307:0:20140:0:(service.c:2152:ptlrpc_server_handle_req_in()) got req x1862456998083328 00000100:00100000:1.0:1776178451.300352:0:20140:0:(nrs_fifo.c:179:nrs_fifo_req_get()) NRS start fifo request from 12345-0@lo, seq: 252 00000100:00100000:1.0:1776178451.300356:0:20140:0:(service.c:2324:ptlrpc_server_handle_request()) Handling RPC req@00000000c6a76119 pname:cluuid+ref:pid:xid:nid:opc:job ll_mgs_0002:ac61f839-0e8e-4cac-9c59-08bcfbbad330+18:3354:x1862456998083328:12345-0@lo:400:kworker/u8:1.0 00000100:00100000:1.0:1776178451.300393:0:20140:0:(service.c:2372:ptlrpc_server_handle_request()) Handled RPC req@00000000c6a76119 pname:cluuid+ref:pid:xid:nid:opc:job ll_mgs_0002:ac61f839-0e8e-4cac-9c59-08bcfbbad330+18:3354:x1862456998083328:12345-0@lo:400:kworker/u8:1.0 Request processed in 36us (842us total) trans 0 rc 0/0 00000100:00100000:1.0:1776178451.300399:0:20140:0:(nrs_fifo.c:241:nrs_fifo_req_stop()) NRS stop fifo request from 12345-0@lo, seq: 252 00000100:00100000:1.0:1776178451.300438:0:3354:0:(client.c:2215:ptlrpc_check_set()) Completed RPC req@0000000047a42b33 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_01:ac61f839-0e8e-4cac-9c59-08bcfbbad330:3354:1862456998083328:0@lo:400:kworker/u8:1.0 00000100:00080000:3.0:1776178451.300643:0:36:0:(pinger.c:251:ptlrpc_pinger_process_import()) lustre-MDT0000-lwp-OST0001_UUID->lustre-MDT0000_UUID: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00080000:3.0:1776178451.300650:0:36:0:(recover.c:57:ptlrpc_initiate_recovery()) lustre-MDT0000_UUID: starting recovery 00000100:00080000:3.0:1776178451.300653:0:36:0:(import.c:86:import_set_state_nolock()) 00000000dfc8db13 lustre-MDT0000_UUID: changing import state from DISCONN to CONNECTING 00000100:00080000:3.0:1776178451.300658:0:36:0:(import.c:534:import_select_connection()) lustre-MDT0000-lwp-OST0001: connect to NID 0@lo last attempt 1153 00000100:00080000:3.0:1776178451.300661:0:36:0:(import.c:581:import_select_connection()) lustre-MDT0000-lwp-OST0001: tried all connections, increasing latency to 21s 00000100:00080000:3.0:1776178451.300665:0:36:0:(import.c:615:import_select_connection()) lustre-MDT0000-lwp-OST0001: import 00000000dfc8db13 using connection 192.168.204.103@tcp/0@lo 00000100:00100000:3.0:1776178451.300707:0:36:0:(import.c:807:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5) req@000000008c50846d x1862456998083520/t0(0) o38->lustre-MDT0000-lwp-OST0001@0@lo:12/10 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' 00000100:00080000:3.0:1776178451.300746:0:36:0:(pinger.c:251:ptlrpc_pinger_process_import()) lustre-MDT0001-mdtlov_UUID->lustre-MDT0000_UUID: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00080000:3.0:1776178451.300749:0:36:0:(recover.c:57:ptlrpc_initiate_recovery()) lustre-MDT0000_UUID: starting recovery 00000100:00080000:3.0:1776178451.300750:0:36:0:(import.c:86:import_set_state_nolock()) 00000000ba4f1821 lustre-MDT0000_UUID: changing import state from DISCONN to CONNECTING 00000100:00080000:3.0:1776178451.300753:0:36:0:(import.c:534:import_select_connection()) lustre-MDT0000-osp-MDT0001: connect to NID 0@lo last attempt 1153 00000100:00080000:3.0:1776178451.300755:0:36:0:(import.c:581:import_select_connection()) lustre-MDT0000-osp-MDT0001: tried all connections, increasing latency to 21s 00000100:00080000:3.0:1776178451.300758:0:36:0:(import.c:615:import_select_connection()) lustre-MDT0000-osp-MDT0001: import 00000000ba4f1821 using connection 192.168.204.103@tcp/0@lo 00000100:00100000:3.0:1776178451.300767:0:36:0:(import.c:807:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5) req@00000000f351bcc3 x1862456998083648/t0(0) o38->lustre-MDT0000-osp-MDT0001@0@lo:24/4 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' 00000100:00080000:3.0:1776178451.300785:0:36:0:(pinger.c:251:ptlrpc_pinger_process_import()) lustre-MDT0000-lwp-MDT0001_UUID->lustre-MDT0000_UUID: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00080000:3.0:1776178451.300789:0:36:0:(recover.c:57:ptlrpc_initiate_recovery()) lustre-MDT0000_UUID: starting recovery 00000100:00080000:3.0:1776178451.300790:0:36:0:(import.c:86:import_set_state_nolock()) 00000000a7c55b1f lustre-MDT0000_UUID: changing import state from DISCONN to CONNECTING 00000100:00080000:3.0:1776178451.300792:0:36:0:(import.c:534:import_select_connection()) lustre-MDT0000-lwp-MDT0001: connect to NID 0@lo last attempt 1153 00000100:00080000:3.0:1776178451.300794:0:36:0:(import.c:581:import_select_connection()) lustre-MDT0000-lwp-MDT0001: tried all connections, increasing latency to 21s 00000100:00080000:3.0:1776178451.300797:0:36:0:(import.c:615:import_select_connection()) lustre-MDT0000-lwp-MDT0001: import 00000000a7c55b1f using connection 192.168.204.103@tcp/0@lo 00000100:00100000:3.0:1776178451.300805:0:36:0:(import.c:807:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5) req@00000000ad228ff6 x1862456998083840/t0(0) o38->lustre-MDT0000-lwp-MDT0001@0@lo:12/10 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' 00000100:00100000:3.0:1776178451.300829:0:3355:0:(client.c:1733:ptlrpc_send_new_req()) Sending RPC req@0000000047a42b33 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_02:lustre-MDT0001-lwp-OST0001_UUID:3355:1862456998083584:0@lo:400:kworker/u8:1.0 00000100:00100000:3.0:1776178451.300862:0:3355:0:(events.c:361:request_in_callback()) peer: 12345-0@lo (source: 12345-0@lo) 00000100:00100000:3.0:1776178451.300882:0:3355:0:(ptlrpcd.c:415:ptlrpcd_check()) transfer 1 async RPCs [3->2] 00000100:00100000:3.0:1776178451.300887:0:3355:0:(client.c:1733:ptlrpc_send_new_req()) Sending RPC req@000000007d6ef583 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_02:lustre-MDT0001-mdtlov_UUID:3355:1862456998083712:0@lo:400:kworker/u8:1.0 00000100:00100000:3.0:1776178451.300912:0:3355:0:(events.c:361:request_in_callback()) peer: 12345-0@lo (source: 12345-0@lo) 00000100:00100000:3.0:1776178451.300947:0:20755:0:(service.c:2152:ptlrpc_server_handle_req_in()) got req x1862456998083712 00000100:00100000:3.0:1776178451.300980:0:20755:0:(nrs_fifo.c:179:nrs_fifo_req_get()) NRS start fifo request from 12345-0@lo, seq: 705 00000100:00100000:3.0:1776178451.300984:0:20755:0:(service.c:2324:ptlrpc_server_handle_request()) Handling RPC req@00000000c8bb8784 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_006:lustre-MDT0001-mdtlov_UUID+5:3355:x1862456998083712:12345-0@lo:400:kworker/u8:1.0 00000100:00100000:3.0:1776178451.301027:0:20755:0:(service.c:2372:ptlrpc_server_handle_request()) Handled RPC req@00000000c8bb8784 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_006:lustre-MDT0001-mdtlov_UUID+5:3355:x1862456998083712:12345-0@lo:400:kworker/u8:1.0 Request processed in 43us (115us total) trans 0 rc 0/0 00000100:00100000:3.0:1776178451.301033:0:20755:0:(nrs_fifo.c:241:nrs_fifo_req_stop()) NRS stop fifo request from 12345-0@lo, seq: 705 00000100:00100000:3.0:1776178451.301075:0:3355:0:(client.c:2215:ptlrpc_check_set()) Completed RPC req@000000007d6ef583 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_02:lustre-MDT0001-mdtlov_UUID:3355:1862456998083712:0@lo:400:kworker/u8:1.0 00000100:00100000:3.0:1776178451.301095:0:3352:0:(client.c:1733:ptlrpc_send_new_req()) Sending RPC req@000000008c50846d pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:lustre-MDT0000-lwp-OST0001_UUID:3352:1862456998083520:0@lo:38:kworker/u8:1.0 00000100:00100000:3.0:1776178451.301119:0:3352:0:(events.c:361:request_in_callback()) peer: 12345-0@lo (source: 12345-0@lo) 00000100:00100000:3.0:1776178451.301135:0:3352:0:(client.c:1733:ptlrpc_send_new_req()) Sending RPC req@00000000f351bcc3 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:lustre-MDT0001-mdtlov_UUID:3352:1862456998083648:0@lo:38:kworker/u8:1.0 00000100:00100000:3.0:1776178451.301157:0:3352:0:(events.c:361:request_in_callback()) peer: 12345-0@lo (source: 12345-0@lo) 00000100:00100000:3.0:1776178451.301169:0:3352:0:(client.c:1733:ptlrpc_send_new_req()) Sending RPC req@00000000ad228ff6 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:lustre-MDT0000-lwp-MDT0001_UUID:3352:1862456998083840:0@lo:38:kworker/u8:1.0 00000100:00100000:3.0:1776178451.301187:0:3352:0:(events.c:361:request_in_callback()) peer: 12345-0@lo (source: 12345-0@lo) 00000100:00100000:3.0:1776178451.301221:0:6257:0:(service.c:2152:ptlrpc_server_handle_req_in()) got req x1862456998083584 00000100:00100000:3.0:1776178451.301241:0:6257:0:(service.c:2152:ptlrpc_server_handle_req_in()) got req x1862456998083520 00000100:00100000:3.0:1776178451.301250:0:6257:0:(service.c:2152:ptlrpc_server_handle_req_in()) got req x1862456998083840 00000100:00100000:3.0:1776178451.301263:0:6257:0:(nrs_fifo.c:179:nrs_fifo_req_get()) NRS start fifo request from 12345-0@lo, seq: 1074 00000100:00100000:3.0:1776178451.301267:0:6257:0:(service.c:2324:ptlrpc_server_handle_request()) Handling RPC req@00000000fdb309c4 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_002:lustre-MDT0001-lwp-OST0001_UUID+5:3355:x1862456998083584:12345-0@lo:400:kworker/u8:1.0 00000100:00100000:3.0:1776178451.301396:0:6257:0:(service.c:2372:ptlrpc_server_handle_request()) Handled RPC req@00000000fdb309c4 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_002:lustre-MDT0001-lwp-OST0001_UUID+5:3355:x1862456998083584:12345-0@lo:400:kworker/u8:1.0 Request processed in 127us (534us total) trans 0 rc 0/0 00000100:00100000:3.0:1776178451.301403:0:6257:0:(nrs_fifo.c:241:nrs_fifo_req_stop()) NRS stop fifo request from 12345-0@lo, seq: 1074 00000100:00100000:3.0:1776178451.301416:0:6257:0:(nrs_fifo.c:179:nrs_fifo_req_get()) NRS start fifo request from 12345-0@lo, seq: 6818 00000100:00100000:3.0:1776178451.301419:0:6257:0:(service.c:2324:ptlrpc_server_handle_request()) Handling RPC req@00000000b5990455 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_002:0+-99:3352:x1862456998083520:12345-0@lo:38:kworker/u8:1.0 00010000:02000000:3.0:1776178451.301430:0:6257:0:(ldlm_lib.c:1139:target_handle_connect()) lustre-MDT0000: Not available for connect from 0@lo (stopping) 00000100:00100000:3.0:1776178451.301454:0:6257:0:(service.c:2372:ptlrpc_server_handle_request()) Handled RPC req@00000000b5990455 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_002:0+-99:3352:x1862456998083520:12345-0@lo:38:kworker/u8:1.0 Request processed in 35us (336us total) trans 0 rc -19/-19 00000100:00100000:3.0:1776178451.301460:0:6257:0:(nrs_fifo.c:241:nrs_fifo_req_stop()) NRS stop fifo request from 12345-0@lo, seq: 6818 00000100:00100000:3.0:1776178451.301470:0:6257:0:(nrs_fifo.c:179:nrs_fifo_req_get()) NRS start fifo request from 12345-0@lo, seq: 6819 00000100:00100000:3.0:1776178451.301473:0:6257:0:(service.c:2324:ptlrpc_server_handle_request()) Handling RPC req@00000000b2b2b654 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_002:0+-99:3352:x1862456998083840:12345-0@lo:38:kworker/u8:1.0 00010000:02000000:3.0:1776178451.301478:0:6257:0:(ldlm_lib.c:1139:target_handle_connect()) lustre-MDT0000: Not available for connect from 0@lo (stopping) 00000100:00100000:3.0:1776178451.301497:0:6257:0:(service.c:2372:ptlrpc_server_handle_request()) Handled RPC req@00000000b2b2b654 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_002:0+-99:3352:x1862456998083840:12345-0@lo:38:kworker/u8:1.0 Request processed in 23us (310us total) trans 0 rc -19/-19 00000100:00100000:3.0:1776178451.301502:0:6257:0:(nrs_fifo.c:241:nrs_fifo_req_stop()) NRS stop fifo request from 12345-0@lo, seq: 6819 00000100:00100000:3.0:1776178451.301525:0:3355:0:(client.c:2215:ptlrpc_check_set()) Completed RPC req@0000000047a42b33 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_02:lustre-MDT0001-lwp-OST0001_UUID:3355:1862456998083584:0@lo:400:kworker/u8:1.0 00000100:00100000:3.0:1776178451.301542:0:3353:0:(client.c:1733:ptlrpc_send_new_req()) Sending RPC req@000000001ef63da8 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_00:lustre-MDT0001-mdtlov_UUID:3353:1862456998083776:0@lo:400:kworker/u8:1.0 00000100:00100000:3.0:1776178451.301565:0:3353:0:(events.c:361:request_in_callback()) peer: 12345-0@lo (source: 12345-0@lo) 00000100:00100000:3.0:1776178451.301592:0:20759:0:(service.c:2152:ptlrpc_server_handle_req_in()) got req x1862456998083776 00000100:00100000:3.0:1776178451.301614:0:20759:0:(nrs_fifo.c:179:nrs_fifo_req_get()) NRS start fifo request from 12345-0@lo, seq: 706 00000100:00100000:3.0:1776178451.301618:0:20759:0:(service.c:2324:ptlrpc_server_handle_request()) Handling RPC req@0000000013888310 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_010:lustre-MDT0001-mdtlov_UUID+5:3353:x1862456998083776:12345-0@lo:400:kworker/u8:1.0 00000100:00100000:3.0:1776178451.301648:0:20759:0:(service.c:2372:ptlrpc_server_handle_request()) Handled RPC req@0000000013888310 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_010:lustre-MDT0001-mdtlov_UUID+5:3353:x1862456998083776:12345-0@lo:400:kworker/u8:1.0 Request processed in 30us (83us total) trans 0 rc 0/0 00000100:00100000:3.0:1776178451.301653:0:20759:0:(nrs_fifo.c:241:nrs_fifo_req_stop()) NRS stop fifo request from 12345-0@lo, seq: 706 00000100:00100000:3.0:1776178451.301674:0:3353:0:(client.c:2215:ptlrpc_check_set()) Completed RPC req@000000001ef63da8 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_00:lustre-MDT0001-mdtlov_UUID:3353:1862456998083776:0@lo:400:kworker/u8:1.0 00000100:00080000:3.0:1776178451.301696:0:3352:0:(import.c:86:import_set_state_nolock()) 00000000dfc8db13 lustre-MDT0000_UUID: changing import state from CONNECTING to DISCONN 00000100:00080000:3.0:1776178451.301700:0:3352:0:(import.c:1429:ptlrpc_connect_interpret()) recovery of lustre-MDT0000_UUID on 192.168.204.103@tcp failed (-19) 00000100:00100000:3.0:1776178451.301702:0:3352:0:(client.c:2215:ptlrpc_check_set()) Completed RPC req@000000008c50846d pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:lustre-MDT0000-lwp-OST0001_UUID:3352:1862456998083520:0@lo:38:kworker/u8:1.0 00000100:00080000:3.0:1776178451.301709:0:3352:0:(import.c:86:import_set_state_nolock()) 00000000a7c55b1f lustre-MDT0000_UUID: changing import state from CONNECTING to DISCONN 00000100:00080000:3.0:1776178451.301711:0:3352:0:(import.c:1429:ptlrpc_connect_interpret()) recovery of lustre-MDT0000_UUID on 192.168.204.103@tcp failed (-19) 00000100:00100000:3.0:1776178451.301713:0:3352:0:(client.c:2215:ptlrpc_check_set()) Completed RPC req@00000000ad228ff6 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:lustre-MDT0000-lwp-MDT0001_UUID:3352:1862456998083840:0@lo:38:kworker/u8:1.0 00000100:00100000:1.0:1776178451.302471:0:14128:0:(service.c:2152:ptlrpc_server_handle_req_in()) got req x1862456998083648 00000100:00100000:1.0:1776178451.302509:0:14128:0:(nrs_fifo.c:179:nrs_fifo_req_get()) NRS start fifo request from 12345-0@lo, seq: 756 00000100:00100000:1.0:1776178451.302512:0:14128:0:(service.c:2324:ptlrpc_server_handle_request()) Handling RPC req@000000004547bc6e pname:cluuid+ref:pid:xid:nid:opc:job mdt_out00_002:0+-99:3352:x1862456998083648:12345-0@lo:38:kworker/u8:1.0 00010000:02000000:1.0:1776178451.302523:0:14128:0:(ldlm_lib.c:1139:target_handle_connect()) lustre-MDT0000: Not available for connect from 0@lo (stopping) 00000100:00100000:1.0:1776178451.302569:0:14128:0:(service.c:2372:ptlrpc_server_handle_request()) Handled RPC req@000000004547bc6e pname:cluuid+ref:pid:xid:nid:opc:job mdt_out00_002:0+-99:3352:x1862456998083648:12345-0@lo:38:kworker/u8:1.0 Request processed in 55us (1411us total) trans 0 rc -19/-19 00000100:00100000:1.0:1776178451.302579:0:14128:0:(nrs_fifo.c:241:nrs_fifo_req_stop()) NRS stop fifo request from 12345-0@lo, seq: 756 00000100:00080000:3.0:1776178451.302717:0:3352:0:(import.c:86:import_set_state_nolock()) 00000000ba4f1821 lustre-MDT0000_UUID: changing import state from CONNECTING to DISCONN 00000100:00080000:3.0:1776178451.302721:0:3352:0:(import.c:1429:ptlrpc_connect_interpret()) recovery of lustre-MDT0000_UUID on 192.168.204.103@tcp failed (-19) 00000100:00100000:3.0:1776178451.302725:0:3352:0:(client.c:2215:ptlrpc_check_set()) Completed RPC req@00000000f351bcc3 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:lustre-MDT0001-mdtlov_UUID:3352:1862456998083648:0@lo:38:kworker/u8:1.0 00010000:00010000:2.0:1776178451.936264:0:6245:0:(ldlm_lockd.c:2792:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:2.0:1776178452.833817:0:3353:0:(ptlrpcd.c:415:ptlrpcd_check()) transfer 1 async RPCs [1->0] 00000100:00100000:2.0:1776178452.833848:0:3353:0:(client.c:1733:ptlrpc_send_new_req()) Sending RPC req@00000000ad228ff6 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_00:lustre-MDT0001-mdtlov_UUID:3353:1862456998083904:0@lo:13:osp-pre-1-1.0 00000100:00100000:2.0:1776178452.833978:0:3353:0:(events.c:361:request_in_callback()) peer: 12345-0@lo (source: 12345-0@lo) 00000100:00100000:0.0:1776178452.834129:0:9554:0:(service.c:2152:ptlrpc_server_handle_req_in()) got req x1862456998083904 00000100:00100000:0.0:1776178452.834198:0:9554:0:(nrs_fifo.c:179:nrs_fifo_req_get()) NRS start fifo request from 12345-0@lo, seq: 585 00000100:00100000:0.0:1776178452.834205:0:9554:0:(service.c:2324:ptlrpc_server_handle_request()) Handling RPC req@000000006c57be21 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_create00:lustre-MDT0001-mdtlov_UUID+5:3353:x1862456998083904:12345-0@lo:13:osp-pre-1-1.0 00002000:00000024:0.0:1776178452.834247:0:9554:0:(ofd_obd.c:740:ofd_statfs()) blocks cached 229376 granted 491746368 pending 0 free 3923689472 avail 3692167168 00000100:00100000:0.0:1776178452.834309:0:9554:0:(service.c:2372:ptlrpc_server_handle_request()) Handled RPC req@000000006c57be21 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_create00:lustre-MDT0001-mdtlov_UUID+5:3353:x1862456998083904:12345-0@lo:13:osp-pre-1-1.0 Request processed in 103us (331us total) trans 0 rc 0/0 00000100:00100000:0.0:1776178452.834317:0:9554:0:(nrs_fifo.c:241:nrs_fifo_req_stop()) NRS stop fifo request from 12345-0@lo, seq: 585 00000100:00100000:2.0:1776178452.834732:0:3353:0:(client.c:2215:ptlrpc_check_set()) Completed RPC req@00000000ad228ff6 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_00:lustre-MDT0001-mdtlov_UUID:3353:1862456998083904:0@lo:13:osp-pre-1-1.0 00010000:00010000:1.0:1776178452.960232:0:9224:0:(ldlm_lockd.c:2792:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:3.0:1776178453.344411:0:3355:0:(client.c:1733:ptlrpc_send_new_req()) Sending RPC req@0000000075189f40 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_02:lustre-MDT0001-mdtlov_UUID:3355:1862456998083968:0@lo:13:osp-pre-0-1.0 00000100:00100000:3.0:1776178453.344752:0:3355:0:(events.c:361:request_in_callback()) peer: 12345-0@lo (source: 12345-0@lo) 00000100:00100000:0.0:1776178453.344870:0:8231:0:(service.c:2152:ptlrpc_server_handle_req_in()) got req x1862456998083968 00000100:00100000:0.0:1776178453.345009:0:8231:0:(nrs_fifo.c:179:nrs_fifo_req_get()) NRS start fifo request from 12345-0@lo, seq: 586 00000100:00100000:0.0:1776178453.345016:0:8231:0:(service.c:2324:ptlrpc_server_handle_request()) Handling RPC req@000000001ef63da8 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_create00:lustre-MDT0001-mdtlov_UUID+5:3355:x1862456998083968:12345-0@lo:13:osp-pre-0-1.0 00002000:00000024:0.0:1776178453.345053:0:8231:0:(ofd_obd.c:740:ofd_statfs()) blocks cached 315392 granted 484883776 pending 0 free 3923689472 avail 3692167168 00000100:00100000:0.0:1776178453.345198:0:8231:0:(service.c:2372:ptlrpc_server_handle_request()) Handled RPC req@000000001ef63da8 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_create00:lustre-MDT0001-mdtlov_UUID+5:3355:x1862456998083968:12345-0@lo:13:osp-pre-0-1.0 Request processed in 181us (447us total) trans 0 rc 0/0 00000100:00100000:0.0:1776178453.345207:0:8231:0:(nrs_fifo.c:241:nrs_fifo_req_stop()) NRS stop fifo request from 12345-0@lo, seq: 586 00000100:00100000:3.0:1776178453.345281:0:3355:0:(client.c:2215:ptlrpc_check_set()) Completed RPC req@0000000075189f40 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_02:lustre-MDT0001-mdtlov_UUID:3355:1862456998083968:0@lo:13:osp-pre-0-1.0 00010000:00010000:1.0:1776178453.996512:0:9225:0:(ldlm_lockd.c:2792:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:3.0:1776178454.818566:0:3348:0:(events.c:361:request_in_callback()) peer: 12345-192.168.204.3@tcp (source: 12345-192.168.204.3@tcp) 00000100:00100000:3.0:1776178454.818619:0:3348:0:(events.c:361:request_in_callback()) peer: 12345-192.168.204.3@tcp (source: 12345-192.168.204.3@tcp) 00000100:00100000:3.0:1776178454.818664:0:3348:0:(events.c:361:request_in_callback()) peer: 12345-192.168.204.3@tcp (source: 12345-192.168.204.3@tcp) 00000100:00100000:3.0:1776178454.818704:0:3348:0:(events.c:361:request_in_callback()) peer: 12345-192.168.204.3@tcp (source: 12345-192.168.204.3@tcp) 00000100:00100000:0.0:1776178454.818708:0:20137:0:(service.c:2152:ptlrpc_server_handle_req_in()) got req x1862456979124672 00000100:00100000:3.0:1776178454.818740:0:3348:0:(events.c:361:request_in_callback()) peer: 12345-192.168.204.3@tcp (source: 12345-192.168.204.3@tcp) 00000100:00100000:0.0:1776178454.818768:0:20137:0:(nrs_fifo.c:179:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.204.3@tcp, seq: 253 00000100:00100000:3.0:1776178454.818771:0:20759:0:(service.c:2152:ptlrpc_server_handle_req_in()) got req x1862456979124864 00000100:00100000:0.0:1776178454.818777:0:20137:0:(service.c:2324:ptlrpc_server_handle_request()) Handling RPC req@00000000a2768ffd pname:cluuid+ref:pid:xid:nid:opc:job ll_mgs_0000:5b5c66b1-1a2f-4651-8548-0c75b9a38a94+9:2271:x1862456979124672:12345-192.168.204.3@tcp:400:kworker/u8:4.0 00000100:00100000:3.0:1776178454.818804:0:20759:0:(service.c:2152:ptlrpc_server_handle_req_in()) got req x1862456979124928 00000100:00100000:3.0:1776178454.818840:0:20759:0:(nrs_fifo.c:179:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.204.3@tcp, seq: 707 00000100:00100000:0.0:1776178454.818854:0:20137:0:(service.c:2372:ptlrpc_server_handle_request()) Handled RPC req@00000000a2768ffd pname:cluuid+ref:pid:xid:nid:opc:job ll_mgs_0000:5b5c66b1-1a2f-4651-8548-0c75b9a38a94+9:2271:x1862456979124672:12345-192.168.204.3@tcp:400:kworker/u8:4.0 Request processed in 76us (294us total) trans 0 rc 0/0 00000100:00100000:3.0:1776178454.818856:0:20759:0:(service.c:2324:ptlrpc_server_handle_request()) Handling RPC req@0000000028dd2f9a pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_010:b50963bd-8eba-4432-9a54-8888d00e3fa1+8:2271:x1862456979124864:12345-192.168.204.3@tcp:400:kworker/u8:4.0 00000100:00100000:0.0:1776178454.818868:0:20137:0:(nrs_fifo.c:241:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.204.3@tcp, seq: 253 00000100:00100000:0.0:1776178454.818907:0:6256:0:(service.c:2152:ptlrpc_server_handle_req_in()) got req x1862456979124800 00000100:00100000:0.0:1776178454.818952:0:6256:0:(service.c:2152:ptlrpc_server_handle_req_in()) got req x1862456979124736 00000100:00100000:0.0:1776178454.818971:0:6256:0:(nrs_fifo.c:179:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.204.3@tcp, seq: 1075 00000100:00100000:0.0:1776178454.818976:0:6256:0:(service.c:2324:ptlrpc_server_handle_request()) Handling RPC req@00000000ad228ff6 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_001:b50963bd-8eba-4432-9a54-8888d00e3fa1+6:2271:x1862456979124800:12345-192.168.204.3@tcp:400:kworker/u8:4.0 00000100:00100000:2.0:1776178454.819177:0:20755:0:(nrs_fifo.c:179:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.204.3@tcp, seq: 708 00000100:00100000:2.0:1776178454.819184:0:20755:0:(service.c:2324:ptlrpc_server_handle_request()) Handling RPC req@00000000f351bcc3 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_006:b50963bd-8eba-4432-9a54-8888d00e3fa1+9:2271:x1862456979124928:12345-192.168.204.3@tcp:400:kworker/u8:4.0 00000100:00100000:2.0:1776178454.819218:0:20755:0:(service.c:2372:ptlrpc_server_handle_request()) Handled RPC req@00000000f351bcc3 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_006:b50963bd-8eba-4432-9a54-8888d00e3fa1+9:2271:x1862456979124928:12345-192.168.204.3@tcp:400:kworker/u8:4.0 Request processed in 35us (516us total) trans 0 rc 0/0 00000100:00100000:2.0:1776178454.819225:0:20755:0:(nrs_fifo.c:241:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.204.3@tcp, seq: 708 00000100:00100000:2.0:1776178454.819249:0:6255:0:(nrs_fifo.c:179:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.204.3@tcp, seq: 6820 00000100:00100000:0.0:1776178454.819310:0:6256:0:(service.c:2372:ptlrpc_server_handle_request()) Handled RPC req@00000000ad228ff6 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_001:b50963bd-8eba-4432-9a54-8888d00e3fa1+6:2271:x1862456979124800:12345-192.168.204.3@tcp:400:kworker/u8:4.0 Request processed in 325us (638us total) trans 0 rc 0/0 00000100:00100000:2.0:1776178454.819318:0:6255:0:(service.c:2324:ptlrpc_server_handle_request()) Handling RPC req@000000008c50846d pname:cluuid+ref:pid:xid:nid:opc:job mdt00_000:0+-99:2269:x1862456979124736:12345-192.168.204.3@tcp:38:kworker/u8:4.0 00000100:00100000:0.0:1776178454.819323:0:6256:0:(nrs_fifo.c:241:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.204.3@tcp, seq: 1075 00010000:02000000:2.0:1776178454.819340:0:6255:0:(ldlm_lib.c:1139:target_handle_connect()) lustre-MDT0000: Not available for connect from 192.168.204.3@tcp (stopping) 00000100:00100000:2.0:1776178454.819370:0:6255:0:(service.c:2372:ptlrpc_server_handle_request()) Handled RPC req@000000008c50846d pname:cluuid+ref:pid:xid:nid:opc:job mdt00_000:0+-99:2269:x1862456979124736:12345-192.168.204.3@tcp:38:kworker/u8:4.0 Request processed in 55us (630us total) trans 0 rc -19/-19 00000100:00100000:2.0:1776178454.819377:0:6255:0:(nrs_fifo.c:241:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.204.3@tcp, seq: 6820 00000100:00100000:3.0:1776178454.824133:0:20759:0:(service.c:2372:ptlrpc_server_handle_request()) Handled RPC req@0000000028dd2f9a pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_010:b50963bd-8eba-4432-9a54-8888d00e3fa1+8:2271:x1862456979124864:12345-192.168.204.3@tcp:400:kworker/u8:4.0 Request processed in 5278us (5505us total) trans 0 rc 0/0 00000100:00100000:3.0:1776178454.824150:0:20759:0:(nrs_fifo.c:241:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.204.3@tcp, seq: 707 00010000:00010000:3.0:1776178455.008406:0:6244:0:(ldlm_lockd.c:2792:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00010000:00010000:2.0:1776178456.032313:0:6245:0:(ldlm_lockd.c:2792:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00080000:0.0:1776178456.416868:0:36:0:(pinger.c:251:ptlrpc_pinger_process_import()) lustre-MDT0000-lwp-OST0000_UUID->lustre-MDT0000_UUID: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00080000:0.0:1776178456.416876:0:36:0:(recover.c:57:ptlrpc_initiate_recovery()) lustre-MDT0000_UUID: starting recovery 00000100:00080000:0.0:1776178456.416879:0:36:0:(import.c:86:import_set_state_nolock()) 0000000047601a98 lustre-MDT0000_UUID: changing import state from DISCONN to CONNECTING 00000100:00080000:0.0:1776178456.416897:0:36:0:(import.c:534:import_select_connection()) lustre-MDT0000-lwp-OST0000: connect to NID 0@lo last attempt 1158 00000100:00080000:0.0:1776178456.416900:0:36:0:(import.c:581:import_select_connection()) lustre-MDT0000-lwp-OST0000: tried all connections, increasing latency to 21s 00000100:00080000:0.0:1776178456.416906:0:36:0:(import.c:615:import_select_connection()) lustre-MDT0000-lwp-OST0000: import 0000000047601a98 using connection 192.168.204.103@tcp/0@lo 00000100:00100000:0.0:1776178456.416928:0:36:0:(import.c:807:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5) req@0000000037aace32 x1862456998084096/t0(0) o38->lustre-MDT0000-lwp-OST0000@0@lo:12/10 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' 00000100:00080000:0.0:1776178456.416959:0:36:0:(pinger.c:251:ptlrpc_pinger_process_import()) lustre-MDT0000-lwp-OST0001_UUID->lustre-MDT0000_UUID: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00080000:0.0:1776178456.416962:0:36:0:(recover.c:57:ptlrpc_initiate_recovery()) lustre-MDT0000_UUID: starting recovery 00000100:00080000:0.0:1776178456.416964:0:36:0:(import.c:86:import_set_state_nolock()) 00000000dfc8db13 lustre-MDT0000_UUID: changing import state from DISCONN to CONNECTING 00000100:00080000:0.0:1776178456.416967:0:36:0:(import.c:534:import_select_connection()) lustre-MDT0000-lwp-OST0001: connect to NID 0@lo last attempt 1158 00000100:00080000:0.0:1776178456.416968:0:36:0:(import.c:581:import_select_connection()) lustre-MDT0000-lwp-OST0001: tried all connections, increasing latency to 21s 00000100:00080000:0.0:1776178456.416971:0:36:0:(import.c:615:import_select_connection()) lustre-MDT0000-lwp-OST0001: import 00000000dfc8db13 using connection 192.168.204.103@tcp/0@lo 00000100:00100000:0.0:1776178456.416989:0:36:0:(import.c:807:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5) req@00000000e8195050 x1862456998084224/t0(0) o38->lustre-MDT0000-lwp-OST0001@0@lo:12/10 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' 00000100:00080000:0.0:1776178456.417003:0:36:0:(pinger.c:251:ptlrpc_pinger_process_import()) lustre-MDT0001-mdtlov_UUID->lustre-MDT0000_UUID: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00080000:0.0:1776178456.417005:0:36:0:(recover.c:57:ptlrpc_initiate_recovery()) lustre-MDT0000_UUID: starting recovery 00000100:00080000:0.0:1776178456.417006:0:36:0:(import.c:86:import_set_state_nolock()) 00000000ba4f1821 lustre-MDT0000_UUID: changing import state from DISCONN to CONNECTING 00000100:00080000:0.0:1776178456.417009:0:36:0:(import.c:534:import_select_connection()) lustre-MDT0000-osp-MDT0001: connect to NID 0@lo last attempt 1158 00000100:00080000:0.0:1776178456.417018:0:36:0:(import.c:581:import_select_connection()) lustre-MDT0000-osp-MDT0001: tried all connections, increasing latency to 21s 00000100:00080000:0.0:1776178456.417021:0:36:0:(import.c:615:import_select_connection()) lustre-MDT0000-osp-MDT0001: import 00000000ba4f1821 using connection 192.168.204.103@tcp/0@lo 00000100:00100000:0.0:1776178456.417031:0:36:0:(import.c:807:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5) req@0000000042de09e9 x1862456998084352/t0(0) o38->lustre-MDT0000-osp-MDT0001@0@lo:24/4 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' 00000100:00080000:0.0:1776178456.417047:0:36:0:(pinger.c:251:ptlrpc_pinger_process_import()) lustre-MDT0000-lwp-MDT0001_UUID->lustre-MDT0000_UUID: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00080000:0.0:1776178456.417050:0:36:0:(recover.c:57:ptlrpc_initiate_recovery()) lustre-MDT0000_UUID: starting recovery 00000100:00080000:0.0:1776178456.417051:0:36:0:(import.c:86:import_set_state_nolock()) 00000000a7c55b1f lustre-MDT0000_UUID: changing import state from DISCONN to CONNECTING 00000100:00080000:0.0:1776178456.417089:0:36:0:(import.c:534:import_select_connection()) lustre-MDT0000-lwp-MDT0001: connect to NID 0@lo last attempt 1158 00000100:00080000:0.0:1776178456.417091:0:36:0:(import.c:581:import_select_connection()) lustre-MDT0000-lwp-MDT0001: tried all connections, increasing latency to 21s 00000100:00080000:0.0:1776178456.417093:0:36:0:(import.c:615:import_select_connection()) lustre-MDT0000-lwp-MDT0001: import 00000000a7c55b1f using connection 192.168.204.103@tcp/0@lo 00000100:00100000:0.0:1776178456.417103:0:36:0:(import.c:807:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5) req@00000000d0d837c5 x1862456998084544/t0(0) o38->lustre-MDT0000-lwp-MDT0001@0@lo:12/10 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' 00000100:00100000:0.0:1776178456.417281:0:3355:0:(client.c:1733:ptlrpc_send_new_req()) Sending RPC req@000000009c4a59b3 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_02:lustre-MDT0001-mdtlov_UUID:3355:1862456998084416:0@lo:400:kworker/u8:1.0 00000100:00100000:0.0:1776178456.417379:0:3355:0:(events.c:361:request_in_callback()) peer: 12345-0@lo (source: 12345-0@lo) 00000100:00100000:0.0:1776178456.417418:0:3355:0:(ptlrpcd.c:415:ptlrpcd_check()) transfer 2 async RPCs [3->2] 00000100:00100000:0.0:1776178456.417423:0:3355:0:(client.c:1733:ptlrpc_send_new_req()) Sending RPC req@0000000029e5c385 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_02:ac61f839-0e8e-4cac-9c59-08bcfbbad330:3355:1862456998084032:0@lo:400:kworker/u8:1.0 00000100:00100000:0.0:1776178456.417452:0:3355:0:(events.c:361:request_in_callback()) peer: 12345-0@lo (source: 12345-0@lo) 00000100:00100000:0.0:1776178456.417463:0:3355:0:(client.c:1733:ptlrpc_send_new_req()) Sending RPC req@000000007216887e pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_02:lustre-MDT0001-mdtlov_UUID:3355:1862456998084480:0@lo:400:kworker/u8:1.0 00000100:00100000:0.0:1776178456.417479:0:3355:0:(events.c:361:request_in_callback()) peer: 12345-0@lo (source: 12345-0@lo) 00000100:00100000:0.0:1776178456.417509:0:14440:0:(service.c:2152:ptlrpc_server_handle_req_in()) got req x1862456998084416 00000100:00100000:0.0:1776178456.417543:0:14440:0:(service.c:2152:ptlrpc_server_handle_req_in()) got req x1862456998084480 00000100:00100000:0.0:1776178456.417558:0:14440:0:(nrs_fifo.c:179:nrs_fifo_req_get()) NRS start fifo request from 12345-0@lo, seq: 709 00000100:00100000:0.0:1776178456.417563:0:14440:0:(service.c:2324:ptlrpc_server_handle_request()) Handling RPC req@00000000380b40c4 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_004:lustre-MDT0001-mdtlov_UUID+5:3355:x1862456998084416:12345-0@lo:400:kworker/u8:1.0 00000100:00100000:0.0:1776178456.417615:0:14440:0:(service.c:2372:ptlrpc_server_handle_request()) Handled RPC req@00000000380b40c4 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_004:lustre-MDT0001-mdtlov_UUID+5:3355:x1862456998084416:12345-0@lo:400:kworker/u8:1.0 Request processed in 51us (237us total) trans 0 rc 0/0 00000100:00100000:0.0:1776178456.417623:0:14440:0:(nrs_fifo.c:241:nrs_fifo_req_stop()) NRS stop fifo request from 12345-0@lo, seq: 709 00000100:00100000:0.0:1776178456.417638:0:14440:0:(nrs_fifo.c:179:nrs_fifo_req_get()) NRS start fifo request from 12345-0@lo, seq: 710 00000100:00100000:0.0:1776178456.417641:0:14440:0:(service.c:2324:ptlrpc_server_handle_request()) Handling RPC req@000000007d5cfd93 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_004:lustre-MDT0001-mdtlov_UUID+5:3355:x1862456998084480:12345-0@lo:400:kworker/u8:1.0 00000100:00100000:0.0:1776178456.417662:0:14440:0:(service.c:2372:ptlrpc_server_handle_request()) Handled RPC req@000000007d5cfd93 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_004:lustre-MDT0001-mdtlov_UUID+5:3355:x1862456998084480:12345-0@lo:400:kworker/u8:1.0 Request processed in 21us (183us total) trans 0 rc 0/0 00000100:00100000:0.0:1776178456.417668:0:14440:0:(nrs_fifo.c:241:nrs_fifo_req_stop()) NRS stop fifo request from 12345-0@lo, seq: 710 00000100:00100000:0.0:1776178456.417704:0:3355:0:(client.c:2215:ptlrpc_check_set()) Completed RPC req@000000007216887e pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_02:lustre-MDT0001-mdtlov_UUID:3355:1862456998084480:0@lo:400:kworker/u8:1.0 00000100:00100000:0.0:1776178456.417713:0:3355:0:(client.c:2215:ptlrpc_check_set()) Completed RPC req@000000009c4a59b3 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_02:lustre-MDT0001-mdtlov_UUID:3355:1862456998084416:0@lo:400:kworker/u8:1.0 00000100:00100000:0.0:1776178456.417740:0:3352:0:(client.c:1733:ptlrpc_send_new_req()) Sending RPC req@0000000037aace32 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:lustre-MDT0000-lwp-OST0000_UUID:3352:1862456998084096:0@lo:38:kworker/u8:1.0 00000100:00100000:0.0:1776178456.417763:0:3352:0:(events.c:361:request_in_callback()) peer: 12345-0@lo (source: 12345-0@lo) 00000100:00100000:0.0:1776178456.417774:0:3352:0:(client.c:1733:ptlrpc_send_new_req()) Sending RPC req@00000000e8195050 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:lustre-MDT0000-lwp-OST0001_UUID:3352:1862456998084224:0@lo:38:kworker/u8:1.0 00000100:00100000:0.0:1776178456.417790:0:3352:0:(events.c:361:request_in_callback()) peer: 12345-0@lo (source: 12345-0@lo) 00000100:00100000:0.0:1776178456.417801:0:3352:0:(client.c:1733:ptlrpc_send_new_req()) Sending RPC req@0000000042de09e9 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:lustre-MDT0001-mdtlov_UUID:3352:1862456998084352:0@lo:38:kworker/u8:1.0 00000100:00100000:0.0:1776178456.417818:0:3352:0:(events.c:361:request_in_callback()) peer: 12345-0@lo (source: 12345-0@lo) 00000100:00100000:0.0:1776178456.417829:0:3352:0:(client.c:1733:ptlrpc_send_new_req()) Sending RPC req@00000000d0d837c5 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:lustre-MDT0000-lwp-MDT0001_UUID:3352:1862456998084544:0@lo:38:kworker/u8:1.0 00000100:00100000:0.0:1776178456.417847:0:3352:0:(events.c:361:request_in_callback()) peer: 12345-0@lo (source: 12345-0@lo) 00000100:00100000:0.0:1776178456.417871:0:8250:0:(service.c:2152:ptlrpc_server_handle_req_in()) got req x1862456998084096 00000100:00100000:0.0:1776178456.417899:0:8250:0:(service.c:2152:ptlrpc_server_handle_req_in()) got req x1862456998084224 00000100:00100000:0.0:1776178456.417912:0:8250:0:(service.c:2152:ptlrpc_server_handle_req_in()) got req x1862456998084544 00000100:00100000:0.0:1776178456.417921:0:8250:0:(nrs_fifo.c:179:nrs_fifo_req_get()) NRS start fifo request from 12345-0@lo, seq: 6821 00000100:00100000:0.0:1776178456.417924:0:8250:0:(service.c:2324:ptlrpc_server_handle_request()) Handling RPC req@000000001f101832 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_003:0+-99:3352:x1862456998084096:12345-0@lo:38:kworker/u8:1.0 00010000:02000000:0.0:1776178456.417937:0:8250:0:(ldlm_lib.c:1139:target_handle_connect()) lustre-MDT0000: Not available for connect from 0@lo (stopping) 00000100:00100000:0.0:1776178456.417969:0:8250:0:(service.c:2372:ptlrpc_server_handle_request()) Handled RPC req@000000001f101832 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_003:0+-99:3352:x1862456998084096:12345-0@lo:38:kworker/u8:1.0 Request processed in 44us (205us total) trans 0 rc -19/-19 00000100:00100000:0.0:1776178456.417974:0:8250:0:(nrs_fifo.c:241:nrs_fifo_req_stop()) NRS stop fifo request from 12345-0@lo, seq: 6821 00000100:00100000:0.0:1776178456.417984:0:8250:0:(nrs_fifo.c:179:nrs_fifo_req_get()) NRS start fifo request from 12345-0@lo, seq: 6822 00000100:00100000:0.0:1776178456.417987:0:8250:0:(service.c:2324:ptlrpc_server_handle_request()) Handling RPC req@000000009c4a59b3 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_003:0+-99:3352:x1862456998084224:12345-0@lo:38:kworker/u8:1.0 00010000:02000000:0.0:1776178456.417995:0:8250:0:(ldlm_lib.c:1139:target_handle_connect()) lustre-MDT0000: Not available for connect from 0@lo (stopping) 00000100:00100000:0.0:1776178456.418013:0:8250:0:(service.c:2372:ptlrpc_server_handle_request()) Handled RPC req@000000009c4a59b3 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_003:0+-99:3352:x1862456998084224:12345-0@lo:38:kworker/u8:1.0 Request processed in 25us (223us total) trans 0 rc -19/-19 00000100:00100000:0.0:1776178456.418017:0:8250:0:(nrs_fifo.c:241:nrs_fifo_req_stop()) NRS stop fifo request from 12345-0@lo, seq: 6822 00000100:00100000:0.0:1776178456.418026:0:8250:0:(nrs_fifo.c:179:nrs_fifo_req_get()) NRS start fifo request from 12345-0@lo, seq: 6823 00000100:00100000:0.0:1776178456.418029:0:8250:0:(service.c:2324:ptlrpc_server_handle_request()) Handling RPC req@00000000854f8d52 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_003:0+-99:3352:x1862456998084544:12345-0@lo:38:kworker/u8:1.0 00010000:02000000:0.0:1776178456.418033:0:8250:0:(ldlm_lib.c:1139:target_handle_connect()) lustre-MDT0000: Not available for connect from 0@lo (stopping) 00000100:00100000:0.0:1776178456.418051:0:8250:0:(service.c:2372:ptlrpc_server_handle_request()) Handled RPC req@00000000854f8d52 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_003:0+-99:3352:x1862456998084544:12345-0@lo:38:kworker/u8:1.0 Request processed in 21us (203us total) trans 0 rc -19/-19 00000100:00100000:0.0:1776178456.418055:0:8250:0:(nrs_fifo.c:241:nrs_fifo_req_stop()) NRS stop fifo request from 12345-0@lo, seq: 6823 00000100:00080000:0.0:1776178456.418112:0:3352:0:(import.c:86:import_set_state_nolock()) 0000000047601a98 lustre-MDT0000_UUID: changing import state from CONNECTING to DISCONN 00000100:00080000:0.0:1776178456.418117:0:3352:0:(import.c:1429:ptlrpc_connect_interpret()) recovery of lustre-MDT0000_UUID on 192.168.204.103@tcp failed (-19) 00000100:00100000:0.0:1776178456.418120:0:3352:0:(client.c:2215:ptlrpc_check_set()) Completed RPC req@0000000037aace32 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:lustre-MDT0000-lwp-OST0000_UUID:3352:1862456998084096:0@lo:38:kworker/u8:1.0 00000100:00080000:0.0:1776178456.418127:0:3352:0:(import.c:86:import_set_state_nolock()) 00000000dfc8db13 lustre-MDT0000_UUID: changing import state from CONNECTING to DISCONN 00000100:00080000:0.0:1776178456.418129:0:3352:0:(import.c:1429:ptlrpc_connect_interpret()) recovery of lustre-MDT0000_UUID on 192.168.204.103@tcp failed (-19) 00000100:00100000:0.0:1776178456.418130:0:3352:0:(client.c:2215:ptlrpc_check_set()) Completed RPC req@00000000e8195050 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:lustre-MDT0000-lwp-OST0001_UUID:3352:1862456998084224:0@lo:38:kworker/u8:1.0 00000100:00080000:0.0:1776178456.418136:0:3352:0:(import.c:86:import_set_state_nolock()) 00000000a7c55b1f lustre-MDT0000_UUID: changing import state from CONNECTING to DISCONN 00000100:00080000:0.0:1776178456.418138:0:3352:0:(import.c:1429:ptlrpc_connect_interpret()) recovery of lustre-MDT0000_UUID on 192.168.204.103@tcp failed (-19) 00000100:00100000:0.0:1776178456.418139:0:3352:0:(client.c:2215:ptlrpc_check_set()) Completed RPC req@00000000d0d837c5 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:lustre-MDT0000-lwp-MDT0001_UUID:3352:1862456998084544:0@lo:38:kworker/u8:1.0 00000100:00100000:1.0:1776178456.418496:0:3354:0:(client.c:1733:ptlrpc_send_new_req()) Sending RPC req@000000006a3d91b0 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_01:lustre-MDT0001-lwp-OST0001_UUID:3354:1862456998084288:0@lo:400:kworker/u8:1.0 00000100:00100000:1.0:1776178456.418534:0:3354:0:(events.c:361:request_in_callback()) peer: 12345-0@lo (source: 12345-0@lo) 00000100:00100000:1.0:1776178456.418560:0:3354:0:(ptlrpcd.c:415:ptlrpcd_check()) transfer 1 async RPCs [0->1] 00000100:00100000:1.0:1776178456.418566:0:3354:0:(client.c:1733:ptlrpc_send_new_req()) Sending RPC req@0000000075189f40 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_01:lustre-MDT0001-lwp-OST0000_UUID:3354:1862456998084160:0@lo:400:kworker/u8:1.0 00000100:00100000:1.0:1776178456.418587:0:3354:0:(events.c:361:request_in_callback()) peer: 12345-0@lo (source: 12345-0@lo) 00000100:00100000:1.0:1776178456.418631:0:6256:0:(service.c:2152:ptlrpc_server_handle_req_in()) got req x1862456998084288 00000100:00100000:1.0:1776178456.418662:0:6256:0:(service.c:2152:ptlrpc_server_handle_req_in()) got req x1862456998084160 00000100:00100000:1.0:1776178456.418678:0:6256:0:(nrs_fifo.c:179:nrs_fifo_req_get()) NRS start fifo request from 12345-0@lo, seq: 1076 00000100:00100000:1.0:1776178456.418681:0:6256:0:(service.c:2324:ptlrpc_server_handle_request()) Handling RPC req@0000000053234520 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_001:lustre-MDT0001-lwp-OST0001_UUID+5:3354:x1862456998084288:12345-0@lo:400:kworker/u8:1.0 00000100:00100000:1.0:1776178456.418721:0:6256:0:(service.c:2372:ptlrpc_server_handle_request()) Handled RPC req@0000000053234520 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_001:lustre-MDT0001-lwp-OST0001_UUID+5:3354:x1862456998084288:12345-0@lo:400:kworker/u8:1.0 Request processed in 39us (188us total) trans 0 rc 0/0 00000100:00100000:1.0:1776178456.418727:0:6256:0:(nrs_fifo.c:241:nrs_fifo_req_stop()) NRS stop fifo request from 12345-0@lo, seq: 1076 00000100:00100000:1.0:1776178456.418738:0:6256:0:(nrs_fifo.c:179:nrs_fifo_req_get()) NRS start fifo request from 12345-0@lo, seq: 1077 00000100:00100000:1.0:1776178456.418742:0:6256:0:(service.c:2324:ptlrpc_server_handle_request()) Handling RPC req@000000007b15d0e4 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_001:lustre-MDT0001-lwp-OST0000_UUID+5:3354:x1862456998084160:12345-0@lo:400:kworker/u8:1.0 00000100:00100000:1.0:1776178456.418767:0:6256:0:(service.c:2372:ptlrpc_server_handle_request()) Handled RPC req@000000007b15d0e4 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_001:lustre-MDT0001-lwp-OST0000_UUID+5:3354:x1862456998084160:12345-0@lo:400:kworker/u8:1.0 Request processed in 24us (180us total) trans 0 rc 0/0 00000100:00100000:1.0:1776178456.418772:0:6256:0:(nrs_fifo.c:241:nrs_fifo_req_stop()) NRS stop fifo request from 12345-0@lo, seq: 1077 00000100:00100000:1.0:1776178456.418795:0:20140:0:(service.c:2152:ptlrpc_server_handle_req_in()) got req x1862456998084032 00000100:00100000:1.0:1776178456.418820:0:20140:0:(nrs_fifo.c:179:nrs_fifo_req_get()) NRS start fifo request from 12345-0@lo, seq: 254 00000100:00100000:1.0:1776178456.418824:0:20140:0:(service.c:2324:ptlrpc_server_handle_request()) Handling RPC req@00000000887443f4 pname:cluuid+ref:pid:xid:nid:opc:job ll_mgs_0002:ac61f839-0e8e-4cac-9c59-08bcfbbad330+18:3355:x1862456998084032:12345-0@lo:400:kworker/u8:1.0 00000100:00100000:1.0:1776178456.418853:0:20140:0:(service.c:2372:ptlrpc_server_handle_request()) Handled RPC req@00000000887443f4 pname:cluuid+ref:pid:xid:nid:opc:job ll_mgs_0002:ac61f839-0e8e-4cac-9c59-08bcfbbad330+18:3355:x1862456998084032:12345-0@lo:400:kworker/u8:1.0 Request processed in 29us (1400us total) trans 0 rc 0/0 00000100:00100000:1.0:1776178456.418862:0:20140:0:(nrs_fifo.c:241:nrs_fifo_req_stop()) NRS stop fifo request from 12345-0@lo, seq: 254 00000100:00100000:1.0:1776178456.418913:0:3354:0:(client.c:2215:ptlrpc_check_set()) Completed RPC req@0000000075189f40 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_01:lustre-MDT0001-lwp-OST0000_UUID:3354:1862456998084160:0@lo:400:kworker/u8:1.0 00000100:00100000:1.0:1776178456.418924:0:3354:0:(client.c:2215:ptlrpc_check_set()) Completed RPC req@000000006a3d91b0 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_01:lustre-MDT0001-lwp-OST0001_UUID:3354:1862456998084288:0@lo:400:kworker/u8:1.0 00000100:00100000:1.0:1776178456.418956:0:3355:0:(client.c:2215:ptlrpc_check_set()) Completed RPC req@0000000029e5c385 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_02:ac61f839-0e8e-4cac-9c59-08bcfbbad330:3355:1862456998084032:0@lo:400:kworker/u8:1.0 00000100:00100000:2.0:1776178456.419746:0:15213:0:(service.c:2152:ptlrpc_server_handle_req_in()) got req x1862456998084352 00000100:00100000:2.0:1776178456.419797:0:15213:0:(nrs_fifo.c:179:nrs_fifo_req_get()) NRS start fifo request from 12345-0@lo, seq: 757 00000100:00100000:2.0:1776178456.419802:0:15213:0:(service.c:2324:ptlrpc_server_handle_request()) Handling RPC req@000000007216887e pname:cluuid+ref:pid:xid:nid:opc:job mdt_out00_003:0+-99:3352:x1862456998084352:12345-0@lo:38:kworker/u8:1.0 00010000:02000000:2.0:1776178456.419825:0:15213:0:(ldlm_lib.c:1139:target_handle_connect()) lustre-MDT0000: Not available for connect from 0@lo (stopping) 00000100:00100000:2.0:1776178456.419912:0:15213:0:(service.c:2372:ptlrpc_server_handle_request()) Handled RPC req@000000007216887e pname:cluuid+ref:pid:xid:nid:opc:job mdt_out00_003:0+-99:3352:x1862456998084352:12345-0@lo:38:kworker/u8:1.0 Request processed in 108us (2092us total) trans 0 rc -19/-19 00000100:00100000:2.0:1776178456.419920:0:15213:0:(nrs_fifo.c:241:nrs_fifo_req_stop()) NRS stop fifo request from 12345-0@lo, seq: 757 00000100:00080000:2.0:1776178456.419959:0:3352:0:(import.c:86:import_set_state_nolock()) 00000000ba4f1821 lustre-MDT0000_UUID: changing import state from CONNECTING to DISCONN 00000100:00080000:2.0:1776178456.419964:0:3352:0:(import.c:1429:ptlrpc_connect_interpret()) recovery of lustre-MDT0000_UUID on 192.168.204.103@tcp failed (-19) 00000100:00100000:2.0:1776178456.419969:0:3352:0:(client.c:2215:ptlrpc_check_set()) Completed RPC req@0000000042de09e9 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:lustre-MDT0001-mdtlov_UUID:3352:1862456998084352:0@lo:38:kworker/u8:1.0 00010000:00010000:1.0:1776178457.057353:0:9224:0:(ldlm_lockd.c:2792:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:1.0:1776178457.952720:0:3354:0:(ptlrpcd.c:415:ptlrpcd_check()) transfer 1 async RPCs [0->1] 00000100:00100000:1.0:1776178457.952744:0:3354:0:(client.c:1733:ptlrpc_send_new_req()) Sending RPC req@0000000047a42b33 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_01:lustre-MDT0001-mdtlov_UUID:3354:1862456998084608:0@lo:13:osp-pre-1-1.0 00000100:00100000:1.0:1776178457.952819:0:3354:0:(events.c:361:request_in_callback()) peer: 12345-0@lo (source: 12345-0@lo) 00000100:00100000:0.0:1776178457.953005:0:8231:0:(service.c:2152:ptlrpc_server_handle_req_in()) got req x1862456998084608 00000100:00100000:0.0:1776178457.953089:0:8231:0:(nrs_fifo.c:179:nrs_fifo_req_get()) NRS start fifo request from 12345-0@lo, seq: 587 00000100:00100000:0.0:1776178457.953097:0:8231:0:(service.c:2324:ptlrpc_server_handle_request()) Handling RPC req@00000000fed3e037 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_create00:lustre-MDT0001-mdtlov_UUID+5:3354:x1862456998084608:12345-0@lo:13:osp-pre-1-1.0 00002000:00000024:0.0:1776178457.953144:0:8231:0:(ofd_obd.c:740:ofd_statfs()) blocks cached 229376 granted 491746368 pending 0 free 3923689472 avail 3692167168 00000100:00100000:0.0:1776178457.953210:0:8231:0:(service.c:2372:ptlrpc_server_handle_request()) Handled RPC req@00000000fed3e037 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_create00:lustre-MDT0001-mdtlov_UUID+5:3354:x1862456998084608:12345-0@lo:13:osp-pre-1-1.0 Request processed in 112us (391us total) trans 0 rc 0/0 00000100:00100000:0.0:1776178457.953218:0:8231:0:(nrs_fifo.c:241:nrs_fifo_req_stop()) NRS stop fifo request from 12345-0@lo, seq: 587 00000100:00100000:3.0:1776178457.953320:0:3354:0:(client.c:2215:ptlrpc_check_set()) Completed RPC req@0000000047a42b33 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_01:lustre-MDT0001-mdtlov_UUID:3354:1862456998084608:0@lo:13:osp-pre-1-1.0 00010000:00010000:1.0:1776178458.081872:0:9225:0:(ldlm_lockd.c:2792:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:2.0:1776178458.464612:0:3353:0:(ptlrpcd.c:415:ptlrpcd_check()) transfer 1 async RPCs [1->0] 00000100:00100000:2.0:1776178458.464644:0:3353:0:(client.c:1733:ptlrpc_send_new_req()) Sending RPC req@0000000040899e7d pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_00:lustre-MDT0001-mdtlov_UUID:3353:1862456998084672:0@lo:13:osp-pre-0-1.0 00000100:00100000:2.0:1776178458.464731:0:3353:0:(events.c:361:request_in_callback()) peer: 12345-0@lo (source: 12345-0@lo) 00000100:00100000:1.0:1776178458.464841:0:9554:0:(service.c:2152:ptlrpc_server_handle_req_in()) got req x1862456998084672 00000100:00100000:1.0:1776178458.464903:0:9554:0:(nrs_fifo.c:179:nrs_fifo_req_get()) NRS start fifo request from 12345-0@lo, seq: 588 00000100:00100000:1.0:1776178458.464909:0:9554:0:(service.c:2324:ptlrpc_server_handle_request()) Handling RPC req@00000000597c3414 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_create00:lustre-MDT0001-mdtlov_UUID+5:3353:x1862456998084672:12345-0@lo:13:osp-pre-0-1.0 00002000:00000024:1.0:1776178458.464951:0:9554:0:(ofd_obd.c:740:ofd_statfs()) blocks cached 315392 granted 484883776 pending 0 free 3923689472 avail 3692167168 00000100:00100000:1.0:1776178458.465009:0:9554:0:(service.c:2372:ptlrpc_server_handle_request()) Handled RPC req@00000000597c3414 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_create00:lustre-MDT0001-mdtlov_UUID+5:3353:x1862456998084672:12345-0@lo:13:osp-pre-0-1.0 Request processed in 98us (278us total) trans 0 rc 0/0 00000100:00100000:1.0:1776178458.465017:0:9554:0:(nrs_fifo.c:241:nrs_fifo_req_stop()) NRS stop fifo request from 12345-0@lo, seq: 588 00000100:00100000:2.0:1776178458.465235:0:3353:0:(client.c:2215:ptlrpc_check_set()) Completed RPC req@0000000040899e7d pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_00:lustre-MDT0001-mdtlov_UUID:3353:1862456998084672:0@lo:13:osp-pre-0-1.0 00010000:00010000:3.0:1776178459.104336:0:6244:0:(ldlm_lockd.c:2792:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:3.0:1776178459.954964:0:3348:0:(events.c:361:request_in_callback()) peer: 12345-192.168.204.3@tcp (source: 12345-192.168.204.3@tcp) 00000100:00100000:3.0:1776178459.955012:0:3348:0:(events.c:361:request_in_callback()) peer: 12345-192.168.204.3@tcp (source: 12345-192.168.204.3@tcp) 00000100:00100000:3.0:1776178459.955051:0:3348:0:(events.c:361:request_in_callback()) peer: 12345-192.168.204.3@tcp (source: 12345-192.168.204.3@tcp) 00000100:00100000:1.0:1776178459.955128:0:20137:0:(service.c:2152:ptlrpc_server_handle_req_in()) got req x1862456979124992 00000100:00100000:3.0:1776178459.955152:0:3348:0:(events.c:361:request_in_callback()) peer: 12345-192.168.204.3@tcp (source: 12345-192.168.204.3@tcp) 00000100:00100000:1.0:1776178459.955190:0:20137:0:(nrs_fifo.c:179:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.204.3@tcp, seq: 255 00000100:00100000:3.0:1776178459.955195:0:3348:0:(events.c:361:request_in_callback()) peer: 12345-192.168.204.3@tcp (source: 12345-192.168.204.3@tcp) 00000100:00100000:1.0:1776178459.955202:0:20137:0:(service.c:2324:ptlrpc_server_handle_request()) Handling RPC req@0000000076ad720a pname:cluuid+ref:pid:xid:nid:opc:job ll_mgs_0000:5b5c66b1-1a2f-4651-8548-0c75b9a38a94+9:2272:x1862456979124992:12345-192.168.204.3@tcp:400:kworker/u8:3.0 00000100:00100000:3.0:1776178459.955233:0:6255:0:(service.c:2152:ptlrpc_server_handle_req_in()) got req x1862456979125120 00000100:00100000:3.0:1776178459.955273:0:6255:0:(service.c:2152:ptlrpc_server_handle_req_in()) got req x1862456979125056 00000100:00100000:1.0:1776178459.955291:0:20137:0:(service.c:2372:ptlrpc_server_handle_request()) Handled RPC req@0000000076ad720a pname:cluuid+ref:pid:xid:nid:opc:job ll_mgs_0000:5b5c66b1-1a2f-4651-8548-0c75b9a38a94+9:2272:x1862456979124992:12345-192.168.204.3@tcp:400:kworker/u8:3.0 Request processed in 90us (346us total) trans 0 rc 0/0 00000100:00100000:3.0:1776178459.955295:0:6255:0:(nrs_fifo.c:179:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.204.3@tcp, seq: 1078 00000100:00100000:3.0:1776178459.955304:0:6255:0:(service.c:2324:ptlrpc_server_handle_request()) Handling RPC req@000000007d6ef583 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_000:b50963bd-8eba-4432-9a54-8888d00e3fa1+6:2272:x1862456979125120:12345-192.168.204.3@tcp:400:kworker/u8:3.0 00000100:00100000:1.0:1776178459.955309:0:20137:0:(nrs_fifo.c:241:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.204.3@tcp, seq: 255 00000100:00100000:3.0:1776178459.955345:0:6255:0:(service.c:2372:ptlrpc_server_handle_request()) Handled RPC req@000000007d6ef583 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_000:b50963bd-8eba-4432-9a54-8888d00e3fa1+6:2272:x1862456979125120:12345-192.168.204.3@tcp:400:kworker/u8:3.0 Request processed in 41us (293us total) trans 0 rc 0/0 00000100:00100000:3.0:1776178459.955353:0:6255:0:(nrs_fifo.c:241:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.204.3@tcp, seq: 1078 00000100:00100000:3.0:1776178459.955367:0:6255:0:(nrs_fifo.c:179:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.204.3@tcp, seq: 6824 00000100:00100000:3.0:1776178459.955370:0:6255:0:(service.c:2324:ptlrpc_server_handle_request()) Handling RPC req@00000000af9e3283 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_000:0+-99:2269:x1862456979125056:12345-192.168.204.3@tcp:38:kworker/u8:3.0 00010000:02000000:3.0:1776178459.955385:0:6255:0:(ldlm_lib.c:1139:target_handle_connect()) lustre-MDT0000: Not available for connect from 192.168.204.3@tcp (stopping) 00000100:00100000:3.0:1776178459.955410:0:6255:0:(service.c:2372:ptlrpc_server_handle_request()) Handled RPC req@00000000af9e3283 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_000:0+-99:2269:x1862456979125056:12345-192.168.204.3@tcp:38:kworker/u8:3.0 Request processed in 39us (224us total) trans 0 rc -19/-19 00000100:00100000:3.0:1776178459.955417:0:6255:0:(nrs_fifo.c:241:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.204.3@tcp, seq: 6824 00000100:00100000:3.0:1776178459.955493:0:20759:0:(service.c:2152:ptlrpc_server_handle_req_in()) got req x1862456979125184 00000100:00100000:3.0:1776178459.955521:0:20759:0:(service.c:2152:ptlrpc_server_handle_req_in()) got req x1862456979125248 00000100:00100000:3.0:1776178459.955540:0:20759:0:(nrs_fifo.c:179:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.204.3@tcp, seq: 711 00000100:00100000:3.0:1776178459.955544:0:20759:0:(service.c:2324:ptlrpc_server_handle_request()) Handling RPC req@0000000047a42b33 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_010:b50963bd-8eba-4432-9a54-8888d00e3fa1+8:2272:x1862456979125184:12345-192.168.204.3@tcp:400:kworker/u8:3.0 00000100:00100000:3.0:1776178459.955581:0:20759:0:(service.c:2372:ptlrpc_server_handle_request()) Handled RPC req@0000000047a42b33 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_010:b50963bd-8eba-4432-9a54-8888d00e3fa1+8:2272:x1862456979125184:12345-192.168.204.3@tcp:400:kworker/u8:3.0 Request processed in 35us (568us total) trans 0 rc 0/0 00000100:00100000:0.0:1776178459.957414:0:20762:0:(nrs_fifo.c:179:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.204.3@tcp, seq: 712 00000100:00100000:0.0:1776178459.957430:0:20762:0:(service.c:2324:ptlrpc_server_handle_request()) Handling RPC req@000000009de51106 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_013:b50963bd-8eba-4432-9a54-8888d00e3fa1+9:2272:x1862456979125248:12345-192.168.204.3@tcp:400:kworker/u8:3.0 00000100:00100000:0.0:1776178459.957503:0:20762:0:(service.c:2372:ptlrpc_server_handle_request()) Handled RPC req@000000009de51106 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_013:b50963bd-8eba-4432-9a54-8888d00e3fa1+9:2272:x1862456979125248:12345-192.168.204.3@tcp:400:kworker/u8:3.0 Request processed in 77us (2352us total) trans 0 rc 0/0 00000100:00100000:0.0:1776178459.957517:0:20762:0:(nrs_fifo.c:241:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.204.3@tcp, seq: 712 00000100:00100000:3.0:1776178459.959648:0:20759:0:(nrs_fifo.c:241:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.204.3@tcp, seq: 711 00010000:00010000:2.0:1776178460.128887:0:6245:0:(ldlm_lockd.c:2792:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00010000:00010000:1.0:1776178461.159522:0:9224:0:(ldlm_lockd.c:2792:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00080000:0.0:1776178461.536368:0:36:0:(pinger.c:251:ptlrpc_pinger_process_import()) lustre-MDT0000-lwp-OST0000_UUID->lustre-MDT0000_UUID: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00080000:0.0:1776178461.536382:0:36:0:(recover.c:57:ptlrpc_initiate_recovery()) lustre-MDT0000_UUID: starting recovery 00000100:00080000:0.0:1776178461.536387:0:36:0:(import.c:86:import_set_state_nolock()) 0000000047601a98 lustre-MDT0000_UUID: changing import state from DISCONN to CONNECTING 00000100:00080000:0.0:1776178461.536399:0:36:0:(import.c:534:import_select_connection()) lustre-MDT0000-lwp-OST0000: connect to NID 0@lo last attempt 1163 00000100:00080000:0.0:1776178461.536498:0:36:0:(import.c:581:import_select_connection()) lustre-MDT0000-lwp-OST0000: tried all connections, increasing latency to 21s 00000100:00080000:0.0:1776178461.536509:0:36:0:(import.c:615:import_select_connection()) lustre-MDT0000-lwp-OST0000: import 0000000047601a98 using connection 192.168.204.103@tcp/0@lo 00000100:00100000:0.0:1776178461.536541:0:36:0:(import.c:807:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5) req@00000000ab295186 x1862456998084800/t0(0) o38->lustre-MDT0000-lwp-OST0000@0@lo:12/10 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' 00000100:00100000:3.0:1776178461.536642:0:3352:0:(client.c:1733:ptlrpc_send_new_req()) Sending RPC req@00000000ab295186 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:lustre-MDT0000-lwp-OST0000_UUID:3352:1862456998084800:0@lo:38:kworker/u8:1.0 00000100:00100000:3.0:1776178461.536739:0:3352:0:(events.c:361:request_in_callback()) peer: 12345-0@lo (source: 12345-0@lo) 00000100:00100000:1.0:1776178461.536852:0:3355:0:(client.c:1733:ptlrpc_send_new_req()) Sending RPC req@000000006ce0dfc8 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_02:ac61f839-0e8e-4cac-9c59-08bcfbbad330:3355:1862456998084736:0@lo:400:kworker/u8:1.0 00000100:00100000:1.0:1776178461.536901:0:3355:0:(events.c:361:request_in_callback()) peer: 12345-0@lo (source: 12345-0@lo) 00000100:00100000:1.0:1776178461.536962:0:20140:0:(service.c:2152:ptlrpc_server_handle_req_in()) got req x1862456998084736 00000100:00100000:1.0:1776178461.537016:0:20140:0:(nrs_fifo.c:179:nrs_fifo_req_get()) NRS start fifo request from 12345-0@lo, seq: 256 00000100:00100000:1.0:1776178461.537023:0:20140:0:(service.c:2324:ptlrpc_server_handle_request()) Handling RPC req@00000000d4927be7 pname:cluuid+ref:pid:xid:nid:opc:job ll_mgs_0002:ac61f839-0e8e-4cac-9c59-08bcfbbad330+18:3355:x1862456998084736:12345-0@lo:400:kworker/u8:1.0 00000100:00100000:1.0:1776178461.537123:0:20140:0:(service.c:2372:ptlrpc_server_handle_request()) Handled RPC req@00000000d4927be7 pname:cluuid+ref:pid:xid:nid:opc:job ll_mgs_0002:ac61f839-0e8e-4cac-9c59-08bcfbbad330+18:3355:x1862456998084736:12345-0@lo:400:kworker/u8:1.0 Request processed in 99us (221us total) trans 0 rc 0/0 00000100:00100000:1.0:1776178461.537132:0:20140:0:(nrs_fifo.c:241:nrs_fifo_req_stop()) NRS stop fifo request from 12345-0@lo, seq: 256 00000100:00100000:1.0:1776178461.537203:0:3355:0:(client.c:2215:ptlrpc_check_set()) Completed RPC req@000000006ce0dfc8 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_02:ac61f839-0e8e-4cac-9c59-08bcfbbad330:3355:1862456998084736:0@lo:400:kworker/u8:1.0 00000100:00080000:0.0:1776178461.537423:0:36:0:(pinger.c:251:ptlrpc_pinger_process_import()) lustre-MDT0000-lwp-OST0001_UUID->lustre-MDT0000_UUID: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00080000:0.0:1776178461.537429:0:36:0:(recover.c:57:ptlrpc_initiate_recovery()) lustre-MDT0000_UUID: starting recovery 00000100:00080000:0.0:1776178461.537433:0:36:0:(import.c:86:import_set_state_nolock()) 00000000dfc8db13 lustre-MDT0000_UUID: changing import state from DISCONN to CONNECTING 00000100:00080000:0.0:1776178461.537439:0:36:0:(import.c:534:import_select_connection()) lustre-MDT0000-lwp-OST0001: connect to NID 0@lo last attempt 1163 00000100:00080000:0.0:1776178461.537442:0:36:0:(import.c:581:import_select_connection()) lustre-MDT0000-lwp-OST0001: tried all connections, increasing latency to 21s 00000100:00080000:0.0:1776178461.537449:0:36:0:(import.c:615:import_select_connection()) lustre-MDT0000-lwp-OST0001: import 00000000dfc8db13 using connection 192.168.204.103@tcp/0@lo 00000100:00100000:0.0:1776178461.537470:0:36:0:(import.c:807:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5) req@000000000ff6c1d2 x1862456998084928/t0(0) o38->lustre-MDT0000-lwp-OST0001@0@lo:12/10 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' 00000100:00100000:2.0:1776178461.537476:0:3356:0:(client.c:1733:ptlrpc_send_new_req()) Sending RPC req@00000000c59ba3a2 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_03:lustre-MDT0001-lwp-OST0000_UUID:3356:1862456998084864:0@lo:400:kworker/u8:1.0 00000100:00100000:2.0:1776178461.537591:0:3356:0:(events.c:361:request_in_callback()) peer: 12345-0@lo (source: 12345-0@lo) 00000100:00100000:1.0:1776178461.537686:0:11281:0:(service.c:2152:ptlrpc_server_handle_req_in()) got req x1862456998084800 00000100:00100000:2.0:1776178461.537698:0:8250:0:(service.c:2152:ptlrpc_server_handle_req_in()) got req x1862456998084864 00000100:00100000:1.0:1776178461.537725:0:11281:0:(nrs_fifo.c:179:nrs_fifo_req_get()) NRS start fifo request from 12345-0@lo, seq: 6825 00000100:00100000:1.0:1776178461.537732:0:11281:0:(service.c:2324:ptlrpc_server_handle_request()) Handling RPC req@000000003334dea7 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_004:0+-99:3352:x1862456998084800:12345-0@lo:38:kworker/u8:1.0 00010000:02000000:1.0:1776178461.537750:0:11281:0:(ldlm_lib.c:1139:target_handle_connect()) lustre-MDT0000: Not available for connect from 0@lo (stopping) 00000100:00100000:2.0:1776178461.537768:0:8250:0:(nrs_fifo.c:179:nrs_fifo_req_get()) NRS start fifo request from 12345-0@lo, seq: 1079 00000100:00100000:2.0:1776178461.537776:0:8250:0:(service.c:2324:ptlrpc_server_handle_request()) Handling RPC req@000000007e1b5a24 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_003:lustre-MDT0001-lwp-OST0000_UUID+5:3356:x1862456998084864:12345-0@lo:400:kworker/u8:1.0 00000100:00100000:2.0:1776178461.537850:0:8250:0:(service.c:2372:ptlrpc_server_handle_request()) Handled RPC req@000000007e1b5a24 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_003:lustre-MDT0001-lwp-OST0000_UUID+5:3356:x1862456998084864:12345-0@lo:400:kworker/u8:1.0 Request processed in 75us (261us total) trans 0 rc 0/0 00000100:00100000:2.0:1776178461.537858:0:8250:0:(nrs_fifo.c:241:nrs_fifo_req_stop()) NRS stop fifo request from 12345-0@lo, seq: 1079 00000100:00100000:2.0:1776178461.537897:0:3356:0:(client.c:2215:ptlrpc_check_set()) Completed RPC req@00000000c59ba3a2 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_03:lustre-MDT0001-lwp-OST0000_UUID:3356:1862456998084864:0@lo:400:kworker/u8:1.0 00000100:00100000:3.0:1776178461.538669:0:3352:0:(client.c:1733:ptlrpc_send_new_req()) Sending RPC req@000000000ff6c1d2 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:lustre-MDT0000-lwp-OST0001_UUID:3352:1862456998084928:0@lo:38:kworker/u8:1.0 00000100:00100000:3.0:1776178461.538725:0:3352:0:(events.c:361:request_in_callback()) peer: 12345-0@lo (source: 12345-0@lo) 00000100:00100000:2.0:1776178461.538846:0:6255:0:(service.c:2152:ptlrpc_server_handle_req_in()) got req x1862456998084928 00000100:00080000:0.0:1776178461.538868:0:36:0:(pinger.c:251:ptlrpc_pinger_process_import()) lustre-MDT0001-mdtlov_UUID->lustre-MDT0000_UUID: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00080000:0.0:1776178461.538876:0:36:0:(recover.c:57:ptlrpc_initiate_recovery()) lustre-MDT0000_UUID: starting recovery 00000100:00080000:0.0:1776178461.538880:0:36:0:(import.c:86:import_set_state_nolock()) 00000000ba4f1821 lustre-MDT0000_UUID: changing import state from DISCONN to CONNECTING 00000100:00080000:0.0:1776178461.538888:0:36:0:(import.c:534:import_select_connection()) lustre-MDT0000-osp-MDT0001: connect to NID 0@lo last attempt 1163 00000100:00080000:0.0:1776178461.538891:0:36:0:(import.c:581:import_select_connection()) lustre-MDT0000-osp-MDT0001: tried all connections, increasing latency to 21s 00000100:00080000:0.0:1776178461.538899:0:36:0:(import.c:615:import_select_connection()) lustre-MDT0000-osp-MDT0001: import 00000000ba4f1821 using connection 192.168.204.103@tcp/0@lo 00000100:00100000:2.0:1776178461.538903:0:6255:0:(nrs_fifo.c:179:nrs_fifo_req_get()) NRS start fifo request from 12345-0@lo, seq: 6826 00000100:00100000:2.0:1776178461.538912:0:6255:0:(service.c:2324:ptlrpc_server_handle_request()) Handling RPC req@0000000018082495 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_000:0+-99:3352:x1862456998084928:12345-0@lo:38:kworker/u8:1.0 00000100:00100000:0.0:1776178461.538926:0:36:0:(import.c:807:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5) req@00000000eeeaf4ad x1862456998085056/t0(0) o38->lustre-MDT0000-osp-MDT0001@0@lo:24/4 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' 00010000:02000000:2.0:1776178461.538937:0:6255:0:(ldlm_lib.c:1139:target_handle_connect()) lustre-MDT0000: Not available for connect from 0@lo (stopping) 00000100:00080000:0.0:1776178461.538974:0:36:0:(pinger.c:251:ptlrpc_pinger_process_import()) lustre-MDT0000-lwp-MDT0001_UUID->lustre-MDT0000_UUID: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00080000:0.0:1776178461.538991:0:36:0:(recover.c:57:ptlrpc_initiate_recovery()) lustre-MDT0000_UUID: starting recovery 00000100:00080000:0.0:1776178461.538992:0:36:0:(import.c:86:import_set_state_nolock()) 00000000a7c55b1f lustre-MDT0000_UUID: changing import state from DISCONN to CONNECTING 00000100:00080000:0.0:1776178461.538997:0:36:0:(import.c:534:import_select_connection()) lustre-MDT0000-lwp-MDT0001: connect to NID 0@lo last attempt 1163 00000100:00080000:0.0:1776178461.538999:0:36:0:(import.c:581:import_select_connection()) lustre-MDT0000-lwp-MDT0001: tried all connections, increasing latency to 21s 00000100:00080000:0.0:1776178461.539003:0:36:0:(import.c:615:import_select_connection()) lustre-MDT0000-lwp-MDT0001: import 00000000a7c55b1f using connection 192.168.204.103@tcp/0@lo 00000100:00100000:0.0:1776178461.539016:0:36:0:(import.c:807:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5) req@0000000008c9fb19 x1862456998085248/t0(0) o38->lustre-MDT0000-lwp-MDT0001@0@lo:12/10 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' 00000100:00100000:0.0:1776178461.539053:0:3354:0:(client.c:1733:ptlrpc_send_new_req()) Sending RPC req@00000000d17c6101 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_01:lustre-MDT0001-mdtlov_UUID:3354:1862456998085120:0@lo:400:kworker/u8:1.0 00000100:00100000:0.0:1776178461.539125:0:3354:0:(events.c:361:request_in_callback()) peer: 12345-0@lo (source: 12345-0@lo) 00000100:00100000:0.0:1776178461.539163:0:3354:0:(ptlrpcd.c:415:ptlrpcd_check()) transfer 1 async RPCs [0->1] 00000100:00100000:0.0:1776178461.539170:0:3354:0:(client.c:1733:ptlrpc_send_new_req()) Sending RPC req@000000006459e586 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_01:lustre-MDT0001-lwp-OST0001_UUID:3354:1862456998084992:0@lo:400:kworker/u8:1.0 00000100:00100000:0.0:1776178461.539196:0:3354:0:(events.c:361:request_in_callback()) peer: 12345-0@lo (source: 12345-0@lo) 00000100:00100000:0.0:1776178461.539235:0:20759:0:(service.c:2152:ptlrpc_server_handle_req_in()) got req x1862456998085120 00000100:00100000:0.0:1776178461.539707:0:20759:0:(nrs_fifo.c:179:nrs_fifo_req_get()) NRS start fifo request from 12345-0@lo, seq: 713 00000100:00100000:0.0:1776178461.539713:0:20759:0:(service.c:2324:ptlrpc_server_handle_request()) Handling RPC req@00000000d9d2b484 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_010:lustre-MDT0001-mdtlov_UUID+5:3354:x1862456998085120:12345-0@lo:400:kworker/u8:1.0 00000100:00100000:0.0:1776178461.539773:0:20759:0:(service.c:2372:ptlrpc_server_handle_request()) Handled RPC req@00000000d9d2b484 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_010:lustre-MDT0001-mdtlov_UUID+5:3354:x1862456998085120:12345-0@lo:400:kworker/u8:1.0 Request processed in 59us (648us total) trans 0 rc 0/0 00000100:00100000:0.0:1776178461.539780:0:20759:0:(nrs_fifo.c:241:nrs_fifo_req_stop()) NRS stop fifo request from 12345-0@lo, seq: 713 00000100:00100000:0.0:1776178461.539847:0:3354:0:(client.c:2215:ptlrpc_check_set()) Completed RPC req@00000000d17c6101 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_01:lustre-MDT0001-mdtlov_UUID:3354:1862456998085120:0@lo:400:kworker/u8:1.0 00000100:00100000:0.0:1776178461.539879:0:3355:0:(client.c:1733:ptlrpc_send_new_req()) Sending RPC req@000000005f1d3350 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_02:lustre-MDT0001-mdtlov_UUID:3355:1862456998085184:0@lo:400:kworker/u8:1.0 00000100:00100000:0.0:1776178461.539905:0:3355:0:(events.c:361:request_in_callback()) peer: 12345-0@lo (source: 12345-0@lo) 00000100:00100000:0.0:1776178461.539935:0:20755:0:(service.c:2152:ptlrpc_server_handle_req_in()) got req x1862456998085184 00000100:00100000:0.0:1776178461.539959:0:20755:0:(nrs_fifo.c:179:nrs_fifo_req_get()) NRS start fifo request from 12345-0@lo, seq: 714 00000100:00100000:0.0:1776178461.539962:0:20755:0:(service.c:2324:ptlrpc_server_handle_request()) Handling RPC req@00000000d17c6101 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_006:lustre-MDT0001-mdtlov_UUID+5:3355:x1862456998085184:12345-0@lo:400:kworker/u8:1.0 00000100:00100000:0.0:1776178461.539998:0:20755:0:(service.c:2372:ptlrpc_server_handle_request()) Handled RPC req@00000000d17c6101 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_006:lustre-MDT0001-mdtlov_UUID+5:3355:x1862456998085184:12345-0@lo:400:kworker/u8:1.0 Request processed in 35us (93us total) trans 0 rc 0/0 00000100:00100000:0.0:1776178461.540004:0:20755:0:(nrs_fifo.c:241:nrs_fifo_req_stop()) NRS stop fifo request from 12345-0@lo, seq: 714 00000100:00100000:0.0:1776178461.540029:0:3355:0:(client.c:2215:ptlrpc_check_set()) Completed RPC req@000000005f1d3350 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_02:lustre-MDT0001-mdtlov_UUID:3355:1862456998085184:0@lo:400:kworker/u8:1.0 00000100:00100000:2.0:1776178461.540265:0:6255:0:(service.c:2372:ptlrpc_server_handle_request()) Handled RPC req@0000000018082495 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_000:0+-99:3352:x1862456998084928:12345-0@lo:38:kworker/u8:1.0 Request processed in 1347us (1534us total) trans 0 rc -19/-19 00000100:00100000:2.0:1776178461.540281:0:6255:0:(nrs_fifo.c:241:nrs_fifo_req_stop()) NRS stop fifo request from 12345-0@lo, seq: 6826 00000100:00100000:2.0:1776178461.540317:0:6255:0:(service.c:2152:ptlrpc_server_handle_req_in()) got req x1862456998084992 00000100:00100000:2.0:1776178461.540367:0:6255:0:(nrs_fifo.c:179:nrs_fifo_req_get()) NRS start fifo request from 12345-0@lo, seq: 1080 00000100:00100000:2.0:1776178461.540372:0:6255:0:(service.c:2324:ptlrpc_server_handle_request()) Handling RPC req@00000000c59ba3a2 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_000:lustre-MDT0001-lwp-OST0001_UUID+5:3354:x1862456998084992:12345-0@lo:400:kworker/u8:1.0 00000100:00100000:2.0:1776178461.540435:0:6255:0:(service.c:2372:ptlrpc_server_handle_request()) Handled RPC req@00000000c59ba3a2 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_000:lustre-MDT0001-lwp-OST0001_UUID+5:3354:x1862456998084992:12345-0@lo:400:kworker/u8:1.0 Request processed in 60us (1235us total) trans 0 rc 0/0 00000100:00100000:2.0:1776178461.540444:0:6255:0:(nrs_fifo.c:241:nrs_fifo_req_stop()) NRS stop fifo request from 12345-0@lo, seq: 1080 00000100:00100000:0.0:1776178461.540512:0:3354:0:(client.c:2215:ptlrpc_check_set()) Completed RPC req@000000006459e586 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_01:lustre-MDT0001-lwp-OST0001_UUID:3354:1862456998084992:0@lo:400:kworker/u8:1.0 00000100:00100000:1.0:1776178461.540700:0:11281:0:(service.c:2372:ptlrpc_server_handle_request()) Handled RPC req@000000003334dea7 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_004:0+-99:3352:x1862456998084800:12345-0@lo:38:kworker/u8:1.0 Request processed in 2967us (3961us total) trans 0 rc -19/-19 00000100:00100000:1.0:1776178461.540711:0:11281:0:(nrs_fifo.c:241:nrs_fifo_req_stop()) NRS stop fifo request from 12345-0@lo, seq: 6825 00000100:00080000:3.0:1776178461.543944:0:3352:0:(import.c:86:import_set_state_nolock()) 0000000047601a98 lustre-MDT0000_UUID: changing import state from CONNECTING to DISCONN 00000100:00080000:3.0:1776178461.543962:0:3352:0:(import.c:1429:ptlrpc_connect_interpret()) recovery of lustre-MDT0000_UUID on 192.168.204.103@tcp failed (-19) 00000100:00100000:3.0:1776178461.543975:0:3352:0:(client.c:2215:ptlrpc_check_set()) Completed RPC req@00000000ab295186 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:lustre-MDT0000-lwp-OST0000_UUID:3352:1862456998084800:0@lo:38:kworker/u8:1.0 00000100:00100000:3.0:1776178461.544030:0:3352:0:(client.c:1733:ptlrpc_send_new_req()) Sending RPC req@00000000eeeaf4ad pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:lustre-MDT0001-mdtlov_UUID:3352:1862456998085056:0@lo:38:kworker/u8:1.0 00000100:00100000:3.0:1776178461.544147:0:3352:0:(events.c:361:request_in_callback()) peer: 12345-0@lo (source: 12345-0@lo) 00000100:00100000:3.0:1776178461.544192:0:3352:0:(client.c:1733:ptlrpc_send_new_req()) Sending RPC req@0000000008c9fb19 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:lustre-MDT0000-lwp-MDT0001_UUID:3352:1862456998085248:0@lo:38:kworker/u8:1.0 00000100:00100000:3.0:1776178461.544223:0:3352:0:(events.c:361:request_in_callback()) peer: 12345-0@lo (source: 12345-0@lo) 00000100:00080000:3.0:1776178461.544275:0:3352:0:(import.c:86:import_set_state_nolock()) 00000000dfc8db13 lustre-MDT0000_UUID: changing import state from CONNECTING to DISCONN 00000100:00080000:3.0:1776178461.544280:0:3352:0:(import.c:1429:ptlrpc_connect_interpret()) recovery of lustre-MDT0000_UUID on 192.168.204.103@tcp failed (-19) 00000100:00100000:3.0:1776178461.544291:0:3352:0:(client.c:2215:ptlrpc_check_set()) Completed RPC req@000000000ff6c1d2 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:lustre-MDT0000-lwp-OST0001_UUID:3352:1862456998084928:0@lo:38:kworker/u8:1.0 00000100:00100000:1.0:1776178461.544538:0:14128:0:(service.c:2152:ptlrpc_server_handle_req_in()) got req x1862456998085056 00000100:00100000:1.0:1776178461.544608:0:14128:0:(nrs_fifo.c:179:nrs_fifo_req_get()) NRS start fifo request from 12345-0@lo, seq: 758 00000100:00100000:1.0:1776178461.544616:0:14128:0:(service.c:2324:ptlrpc_server_handle_request()) Handling RPC req@00000000638772fd pname:cluuid+ref:pid:xid:nid:opc:job mdt_out00_002:0+-99:3352:x1862456998085056:12345-0@lo:38:kworker/u8:1.0 00010000:02000000:1.0:1776178461.544637:0:14128:0:(ldlm_lib.c:1139:target_handle_connect()) lustre-MDT0000: Not available for connect from 0@lo (stopping) 00000100:00100000:1.0:1776178461.544695:0:14128:0:(service.c:2372:ptlrpc_server_handle_request()) Handled RPC req@00000000638772fd pname:cluuid+ref:pid:xid:nid:opc:job mdt_out00_002:0+-99:3352:x1862456998085056:12345-0@lo:38:kworker/u8:1.0 Request processed in 79us (550us total) trans 0 rc -19/-19 00000100:00100000:1.0:1776178461.544703:0:14128:0:(nrs_fifo.c:241:nrs_fifo_req_stop()) NRS stop fifo request from 12345-0@lo, seq: 758 00000100:00100000:2.0:1776178461.545154:0:11281:0:(service.c:2152:ptlrpc_server_handle_req_in()) got req x1862456998085248 00000100:00100000:2.0:1776178461.545243:0:11281:0:(nrs_fifo.c:179:nrs_fifo_req_get()) NRS start fifo request from 12345-0@lo, seq: 6827 00000100:00100000:2.0:1776178461.545250:0:11281:0:(service.c:2324:ptlrpc_server_handle_request()) Handling RPC req@00000000544c54c4 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_004:0+-99:3352:x1862456998085248:12345-0@lo:38:kworker/u8:1.0 00010000:02000000:2.0:1776178461.545270:0:11281:0:(ldlm_lib.c:1139:target_handle_connect()) lustre-MDT0000: Not available for connect from 0@lo (stopping) 00000100:00100000:2.0:1776178461.545364:0:11281:0:(service.c:2372:ptlrpc_server_handle_request()) Handled RPC req@00000000544c54c4 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_004:0+-99:3352:x1862456998085248:12345-0@lo:38:kworker/u8:1.0 Request processed in 113us (1140us total) trans 0 rc -19/-19 00000100:00100000:2.0:1776178461.545371:0:11281:0:(nrs_fifo.c:241:nrs_fifo_req_stop()) NRS stop fifo request from 12345-0@lo, seq: 6827 00000100:00080000:0.0:1776178461.547717:0:3352:0:(import.c:86:import_set_state_nolock()) 00000000ba4f1821 lustre-MDT0000_UUID: changing import state from CONNECTING to DISCONN 00000100:00080000:0.0:1776178461.547734:0:3352:0:(import.c:1429:ptlrpc_connect_interpret()) recovery of lustre-MDT0000_UUID on 192.168.204.103@tcp failed (-19) 00000100:00100000:0.0:1776178461.547747:0:3352:0:(client.c:2215:ptlrpc_check_set()) Completed RPC req@00000000eeeaf4ad pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:lustre-MDT0001-mdtlov_UUID:3352:1862456998085056:0@lo:38:kworker/u8:1.0 00000100:00080000:0.0:1776178461.547772:0:3352:0:(import.c:86:import_set_state_nolock()) 00000000a7c55b1f lustre-MDT0000_UUID: changing import state from CONNECTING to DISCONN 00000100:00080000:0.0:1776178461.547776:0:3352:0:(import.c:1429:ptlrpc_connect_interpret()) recovery of lustre-MDT0000_UUID on 192.168.204.103@tcp failed (-19) 00000100:00100000:0.0:1776178461.547779:0:3352:0:(client.c:2215:ptlrpc_check_set()) Completed RPC req@0000000008c9fb19 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:lustre-MDT0000-lwp-MDT0001_UUID:3352:1862456998085248:0@lo:38:kworker/u8:1.0 00010000:00010000:1.0:1776178462.176366:0:9225:0:(ldlm_lockd.c:2792:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:0.0:1776178463.076855:0:3356:0:(client.c:1733:ptlrpc_send_new_req()) Sending RPC req@00000000d0706d7c pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_03:lustre-MDT0001-mdtlov_UUID:3356:1862456998085312:0@lo:13:osp-pre-1-1.0 00000100:00100000:0.0:1776178463.076946:0:3356:0:(events.c:361:request_in_callback()) peer: 12345-0@lo (source: 12345-0@lo) 00000100:00100000:3.0:1776178463.077209:0:8231:0:(service.c:2152:ptlrpc_server_handle_req_in()) got req x1862456998085312 00000100:00100000:3.0:1776178463.077266:0:8231:0:(nrs_fifo.c:179:nrs_fifo_req_get()) NRS start fifo request from 12345-0@lo, seq: 589 00000100:00100000:3.0:1776178463.077273:0:8231:0:(service.c:2324:ptlrpc_server_handle_request()) Handling RPC req@0000000008c9fb19 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_create00:lustre-MDT0001-mdtlov_UUID+5:3356:x1862456998085312:12345-0@lo:13:osp-pre-1-1.0 00002000:00000024:3.0:1776178463.077311:0:8231:0:(ofd_obd.c:740:ofd_statfs()) blocks cached 229376 granted 491746368 pending 0 free 3923689472 avail 3692167168 00000100:00100000:3.0:1776178463.077367:0:8231:0:(service.c:2372:ptlrpc_server_handle_request()) Handled RPC req@0000000008c9fb19 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_create00:lustre-MDT0001-mdtlov_UUID+5:3356:x1862456998085312:12345-0@lo:13:osp-pre-1-1.0 Request processed in 93us (422us total) trans 0 rc 0/0 00000100:00100000:3.0:1776178463.077374:0:8231:0:(nrs_fifo.c:241:nrs_fifo_req_stop()) NRS stop fifo request from 12345-0@lo, seq: 589 00000100:00100000:0.0:1776178463.077460:0:3356:0:(client.c:2215:ptlrpc_check_set()) Completed RPC req@00000000d0706d7c pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_03:lustre-MDT0001-mdtlov_UUID:3356:1862456998085312:0@lo:13:osp-pre-1-1.0 00010000:00010000:3.0:1776178463.200595:0:6244:0:(ldlm_lockd.c:2792:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:0.0:1776178463.584331:0:3353:0:(client.c:1733:ptlrpc_send_new_req()) Sending RPC req@00000000eeeaf4ad pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_00:lustre-MDT0001-mdtlov_UUID:3353:1862456998085376:0@lo:13:osp-pre-0-1.0 00000100:00100000:0.0:1776178463.584407:0:3353:0:(events.c:361:request_in_callback()) peer: 12345-0@lo (source: 12345-0@lo) 00000100:00100000:2.0:1776178463.584839:0:9554:0:(service.c:2152:ptlrpc_server_handle_req_in()) got req x1862456998085376 00000100:00100000:2.0:1776178463.585663:0:9554:0:(nrs_fifo.c:179:nrs_fifo_req_get()) NRS start fifo request from 12345-0@lo, seq: 590 00000100:00100000:2.0:1776178463.585674:0:9554:0:(service.c:2324:ptlrpc_server_handle_request()) Handling RPC req@000000006459e586 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_create00:lustre-MDT0001-mdtlov_UUID+5:3353:x1862456998085376:12345-0@lo:13:osp-pre-0-1.0 00002000:00000024:2.0:1776178463.585713:0:9554:0:(ofd_obd.c:740:ofd_statfs()) blocks cached 315392 granted 484883776 pending 0 free 3923689472 avail 3692167168 00000100:00100000:2.0:1776178463.585778:0:9554:0:(service.c:2372:ptlrpc_server_handle_request()) Handled RPC req@000000006459e586 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_create00:lustre-MDT0001-mdtlov_UUID+5:3353:x1862456998085376:12345-0@lo:13:osp-pre-0-1.0 Request processed in 104us (1371us total) trans 0 rc 0/0 00000100:00100000:2.0:1776178463.585785:0:9554:0:(nrs_fifo.c:241:nrs_fifo_req_stop()) NRS stop fifo request from 12345-0@lo, seq: 590 00000100:00100000:0.0:1776178463.585874:0:3353:0:(client.c:2215:ptlrpc_check_set()) Completed RPC req@00000000eeeaf4ad pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_00:lustre-MDT0001-mdtlov_UUID:3353:1862456998085376:0@lo:13:osp-pre-0-1.0 00010000:00010000:2.0:1776178464.224786:0:6245:0:(ldlm_lockd.c:2792:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:3.0:1776178465.060437:0:3348:0:(events.c:361:request_in_callback()) peer: 12345-192.168.204.3@tcp (source: 12345-192.168.204.3@tcp) 00000100:00100000:2.0:1776178465.060654:0:20137:0:(service.c:2152:ptlrpc_server_handle_req_in()) got req x1862456979125312 00000100:00100000:2.0:1776178465.060737:0:20137:0:(nrs_fifo.c:179:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.204.3@tcp, seq: 257 00000100:00100000:2.0:1776178465.060745:0:20137:0:(service.c:2324:ptlrpc_server_handle_request()) Handling RPC req@00000000286b1e8d pname:cluuid+ref:pid:xid:nid:opc:job ll_mgs_0000:5b5c66b1-1a2f-4651-8548-0c75b9a38a94+9:2271:x1862456979125312:12345-192.168.204.3@tcp:400:kworker/u8:4.0 00000100:00100000:2.0:1776178465.060845:0:20137:0:(service.c:2372:ptlrpc_server_handle_request()) Handled RPC req@00000000286b1e8d pname:cluuid+ref:pid:xid:nid:opc:job ll_mgs_0000:5b5c66b1-1a2f-4651-8548-0c75b9a38a94+9:2271:x1862456979125312:12345-192.168.204.3@tcp:400:kworker/u8:4.0 Request processed in 100us (415us total) trans 0 rc 0/0 00000100:00100000:2.0:1776178465.060853:0:20137:0:(nrs_fifo.c:241:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.204.3@tcp, seq: 257 00000100:00100000:0.0:1776178465.068931:0:3347:0:(events.c:361:request_in_callback()) peer: 12345-192.168.204.3@tcp (source: 12345-192.168.204.3@tcp) 00000100:00100000:0.0:1776178465.069042:0:3347:0:(events.c:361:request_in_callback()) peer: 12345-192.168.204.3@tcp (source: 12345-192.168.204.3@tcp) 00000100:00100000:0.0:1776178465.069115:0:3347:0:(events.c:361:request_in_callback()) peer: 12345-192.168.204.3@tcp (source: 12345-192.168.204.3@tcp) 00000100:00100000:0.0:1776178465.069192:0:3347:0:(events.c:361:request_in_callback()) peer: 12345-192.168.204.3@tcp (source: 12345-192.168.204.3@tcp) 00000100:00100000:2.0:1776178465.069261:0:20759:0:(service.c:2152:ptlrpc_server_handle_req_in()) got req x1862456979125504 00000100:00100000:2.0:1776178465.069328:0:20759:0:(service.c:2152:ptlrpc_server_handle_req_in()) got req x1862456979125568 00000100:00100000:2.0:1776178465.069353:0:20759:0:(nrs_fifo.c:179:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.204.3@tcp, seq: 715 00000100:00100000:2.0:1776178465.069360:0:20759:0:(service.c:2324:ptlrpc_server_handle_request()) Handling RPC req@000000005f1d3350 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_010:b50963bd-8eba-4432-9a54-8888d00e3fa1+8:2270:x1862456979125504:12345-192.168.204.3@tcp:400:kworker/u8:4.0 00000100:00100000:2.0:1776178465.069440:0:20759:0:(service.c:2372:ptlrpc_server_handle_request()) Handled RPC req@000000005f1d3350 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_010:b50963bd-8eba-4432-9a54-8888d00e3fa1+8:2270:x1862456979125504:12345-192.168.204.3@tcp:400:kworker/u8:4.0 Request processed in 80us (399us total) trans 0 rc 0/0 00000100:00100000:2.0:1776178465.069448:0:20759:0:(nrs_fifo.c:241:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.204.3@tcp, seq: 715 00000100:00100000:2.0:1776178465.069479:0:20759:0:(nrs_fifo.c:179:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.204.3@tcp, seq: 716 00000100:00100000:2.0:1776178465.069484:0:20759:0:(service.c:2324:ptlrpc_server_handle_request()) Handling RPC req@000000006ce0dfc8 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_010:b50963bd-8eba-4432-9a54-8888d00e3fa1+9:2270:x1862456979125568:12345-192.168.204.3@tcp:400:kworker/u8:4.0 00000100:00100000:2.0:1776178465.069507:0:20759:0:(service.c:2372:ptlrpc_server_handle_request()) Handled RPC req@000000006ce0dfc8 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_010:b50963bd-8eba-4432-9a54-8888d00e3fa1+9:2270:x1862456979125568:12345-192.168.204.3@tcp:400:kworker/u8:4.0 Request processed in 23us (393us total) trans 0 rc 0/0 00000100:00100000:2.0:1776178465.069513:0:20759:0:(nrs_fifo.c:241:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.204.3@tcp, seq: 716 00000100:00100000:1.0:1776178465.073791:0:6256:0:(service.c:2152:ptlrpc_server_handle_req_in()) got req x1862456979125440 00000100:00100000:1.0:1776178465.073953:0:6256:0:(service.c:2152:ptlrpc_server_handle_req_in()) got req x1862456979125376 00000100:00100000:2.0:1776178465.077990:0:11281:0:(nrs_fifo.c:179:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.204.3@tcp, seq: 1081 00000100:00100000:2.0:1776178465.078004:0:11281:0:(service.c:2324:ptlrpc_server_handle_request()) Handling RPC req@00000000eeeaf4ad pname:cluuid+ref:pid:xid:nid:opc:job mdt00_004:b50963bd-8eba-4432-9a54-8888d00e3fa1+6:2270:x1862456979125440:12345-192.168.204.3@tcp:400:kworker/u8:4.0 00000100:00100000:2.0:1776178465.078142:0:11281:0:(service.c:2372:ptlrpc_server_handle_request()) Handled RPC req@00000000eeeaf4ad pname:cluuid+ref:pid:xid:nid:opc:job mdt00_004:b50963bd-8eba-4432-9a54-8888d00e3fa1+6:2270:x1862456979125440:12345-192.168.204.3@tcp:400:kworker/u8:4.0 Request processed in 139us (9219us total) trans 0 rc 0/0 00000100:00100000:2.0:1776178465.078150:0:11281:0:(nrs_fifo.c:241:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.204.3@tcp, seq: 1081 00000100:00100000:2.0:1776178465.078183:0:11281:0:(nrs_fifo.c:179:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.204.3@tcp, seq: 6828 00000100:00100000:2.0:1776178465.078188:0:11281:0:(service.c:2324:ptlrpc_server_handle_request()) Handling RPC req@0000000040899e7d pname:cluuid+ref:pid:xid:nid:opc:job mdt00_004:0+-99:2269:x1862456979125376:12345-192.168.204.3@tcp:38:kworker/u8:4.0 00010000:02000000:2.0:1776178465.078203:0:11281:0:(ldlm_lib.c:1139:target_handle_connect()) lustre-MDT0000: Not available for connect from 192.168.204.3@tcp (stopping) 00000100:00100000:2.0:1776178465.078231:0:11281:0:(service.c:2372:ptlrpc_server_handle_request()) Handled RPC req@0000000040899e7d pname:cluuid+ref:pid:xid:nid:opc:job mdt00_004:0+-99:2269:x1862456979125376:12345-192.168.204.3@tcp:38:kworker/u8:4.0 Request processed in 42us (9039us total) trans 0 rc -19/-19 00000100:00100000:2.0:1776178465.078236:0:11281:0:(nrs_fifo.c:241:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.204.3@tcp, seq: 6828 00010000:00010000:1.0:1776178465.248443:0:9224:0:(ldlm_lockd.c:2792:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00010000:00010000:1.0:1776178466.272384:0:9225:0:(ldlm_lockd.c:2792:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00080000:3.0:1776178466.656401:0:37:0:(pinger.c:251:ptlrpc_pinger_process_import()) lustre-MDT0000-lwp-OST0000_UUID->lustre-MDT0000_UUID: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00080000:3.0:1776178466.656411:0:37:0:(recover.c:57:ptlrpc_initiate_recovery()) lustre-MDT0000_UUID: starting recovery 00000100:00080000:3.0:1776178466.656415:0:37:0:(import.c:86:import_set_state_nolock()) 0000000047601a98 lustre-MDT0000_UUID: changing import state from DISCONN to CONNECTING 00000100:00080000:3.0:1776178466.656428:0:37:0:(import.c:534:import_select_connection()) lustre-MDT0000-lwp-OST0000: connect to NID 0@lo last attempt 1168 00000100:00080000:3.0:1776178466.656432:0:37:0:(import.c:581:import_select_connection()) lustre-MDT0000-lwp-OST0000: tried all connections, increasing latency to 21s 00000100:00080000:3.0:1776178466.656440:0:37:0:(import.c:615:import_select_connection()) lustre-MDT0000-lwp-OST0000: import 0000000047601a98 using connection 192.168.204.103@tcp/0@lo 00000100:00100000:3.0:1776178466.656469:0:37:0:(import.c:807:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5) req@0000000021b0d516 x1862456998085504/t0(0) o38->lustre-MDT0000-lwp-OST0000@0@lo:12/10 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' 00000100:00100000:1.0:1776178466.656553:0:3352:0:(client.c:1733:ptlrpc_send_new_req()) Sending RPC req@0000000021b0d516 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:lustre-MDT0000-lwp-OST0000_UUID:3352:1862456998085504:0@lo:38:kworker/u8:2.0 00000100:00100000:1.0:1776178466.656634:0:3352:0:(events.c:361:request_in_callback()) peer: 12345-0@lo (source: 12345-0@lo) 00000100:00100000:2.0:1776178466.656740:0:3354:0:(client.c:1733:ptlrpc_send_new_req()) Sending RPC req@00000000167dc455 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_01:ac61f839-0e8e-4cac-9c59-08bcfbbad330:3354:1862456998085440:0@lo:400:kworker/u8:2.0 00000100:00100000:2.0:1776178466.656793:0:3354:0:(events.c:361:request_in_callback()) peer: 12345-0@lo (source: 12345-0@lo) 00000100:00100000:2.0:1776178466.656847:0:11281:0:(service.c:2152:ptlrpc_server_handle_req_in()) got req x1862456998085504 00000100:00100000:2.0:1776178466.656890:0:11281:0:(nrs_fifo.c:179:nrs_fifo_req_get()) NRS start fifo request from 12345-0@lo, seq: 6829 00000100:00100000:2.0:1776178466.656897:0:11281:0:(service.c:2324:ptlrpc_server_handle_request()) Handling RPC req@00000000f9929c4b pname:cluuid+ref:pid:xid:nid:opc:job mdt00_004:0+-99:3352:x1862456998085504:12345-0@lo:38:kworker/u8:2.0 00010000:02000000:2.0:1776178466.656920:0:11281:0:(ldlm_lib.c:1139:target_handle_connect()) lustre-MDT0000: Not available for connect from 0@lo (stopping) 00000100:00080000:3.0:1776178466.657183:0:37:0:(pinger.c:251:ptlrpc_pinger_process_import()) lustre-MDT0000-lwp-OST0001_UUID->lustre-MDT0000_UUID: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00080000:3.0:1776178466.657188:0:37:0:(recover.c:57:ptlrpc_initiate_recovery()) lustre-MDT0000_UUID: starting recovery 00000100:00080000:3.0:1776178466.657190:0:37:0:(import.c:86:import_set_state_nolock()) 00000000dfc8db13 lustre-MDT0000_UUID: changing import state from DISCONN to CONNECTING 00000100:00080000:3.0:1776178466.657195:0:37:0:(import.c:534:import_select_connection()) lustre-MDT0000-lwp-OST0001: connect to NID 0@lo last attempt 1168 00000100:00080000:3.0:1776178466.657197:0:37:0:(import.c:581:import_select_connection()) lustre-MDT0000-lwp-OST0001: tried all connections, increasing latency to 21s 00000100:00080000:3.0:1776178466.657201:0:37:0:(import.c:615:import_select_connection()) lustre-MDT0000-lwp-OST0001: import 00000000dfc8db13 using connection 192.168.204.103@tcp/0@lo 00000100:00100000:3.0:1776178466.657215:0:37:0:(import.c:807:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5) req@00000000bdad24ac x1862456998085632/t0(0) o38->lustre-MDT0000-lwp-OST0001@0@lo:12/10 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' 00000100:00100000:0.0:1776178466.657368:0:3356:0:(ptlrpcd.c:415:ptlrpcd_check()) transfer 1 async RPCs [2->3] 00000100:00100000:0.0:1776178466.657378:0:3356:0:(client.c:1733:ptlrpc_send_new_req()) Sending RPC req@00000000044bd507 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_03:lustre-MDT0001-lwp-OST0000_UUID:3356:1862456998085568:0@lo:400:kworker/u8:2.0 00000100:00100000:0.0:1776178466.657409:0:3356:0:(events.c:361:request_in_callback()) peer: 12345-0@lo (source: 12345-0@lo) 00000100:00100000:0.0:1776178466.657452:0:6256:0:(service.c:2152:ptlrpc_server_handle_req_in()) got req x1862456998085568 00000100:00100000:0.0:1776178466.657486:0:6256:0:(nrs_fifo.c:179:nrs_fifo_req_get()) NRS start fifo request from 12345-0@lo, seq: 1082 00000100:00100000:0.0:1776178466.657490:0:6256:0:(service.c:2324:ptlrpc_server_handle_request()) Handling RPC req@000000000ff6c1d2 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_001:lustre-MDT0001-lwp-OST0000_UUID+5:3356:x1862456998085568:12345-0@lo:400:kworker/u8:2.0 00000100:00100000:0.0:1776178466.657532:0:6256:0:(service.c:2372:ptlrpc_server_handle_request()) Handled RPC req@000000000ff6c1d2 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_001:lustre-MDT0001-lwp-OST0000_UUID+5:3356:x1862456998085568:12345-0@lo:400:kworker/u8:2.0 Request processed in 41us (123us total) trans 0 rc 0/0 00000100:00100000:0.0:1776178466.657538:0:6256:0:(nrs_fifo.c:241:nrs_fifo_req_stop()) NRS stop fifo request from 12345-0@lo, seq: 1082 00000100:00100000:0.0:1776178466.657565:0:20140:0:(service.c:2152:ptlrpc_server_handle_req_in()) got req x1862456998085440 00000100:00100000:0.0:1776178466.657590:0:20140:0:(nrs_fifo.c:179:nrs_fifo_req_get()) NRS start fifo request from 12345-0@lo, seq: 258 00000100:00100000:0.0:1776178466.657594:0:20140:0:(service.c:2324:ptlrpc_server_handle_request()) Handling RPC req@00000000fd0a07ee pname:cluuid+ref:pid:xid:nid:opc:job ll_mgs_0002:ac61f839-0e8e-4cac-9c59-08bcfbbad330+18:3354:x1862456998085440:12345-0@lo:400:kworker/u8:2.0 00000100:00100000:0.0:1776178466.657626:0:20140:0:(service.c:2372:ptlrpc_server_handle_request()) Handled RPC req@00000000fd0a07ee pname:cluuid+ref:pid:xid:nid:opc:job ll_mgs_0002:ac61f839-0e8e-4cac-9c59-08bcfbbad330+18:3354:x1862456998085440:12345-0@lo:400:kworker/u8:2.0 Request processed in 31us (833us total) trans 0 rc 0/0 00000100:00100000:0.0:1776178466.657633:0:20140:0:(nrs_fifo.c:241:nrs_fifo_req_stop()) NRS stop fifo request from 12345-0@lo, seq: 258 00000100:00100000:0.0:1776178466.657687:0:3356:0:(client.c:2215:ptlrpc_check_set()) Completed RPC req@00000000044bd507 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_03:lustre-MDT0001-lwp-OST0000_UUID:3356:1862456998085568:0@lo:400:kworker/u8:2.0 00000100:00100000:0.0:1776178466.657759:0:3354:0:(client.c:2215:ptlrpc_check_set()) Completed RPC req@00000000167dc455 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_01:ac61f839-0e8e-4cac-9c59-08bcfbbad330:3354:1862456998085440:0@lo:400:kworker/u8:2.0 00000100:00100000:1.0:1776178466.660113:0:3352:0:(client.c:1733:ptlrpc_send_new_req()) Sending RPC req@00000000bdad24ac pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:lustre-MDT0000-lwp-OST0001_UUID:3352:1862456998085632:0@lo:38:kworker/u8:2.0 00000100:00100000:1.0:1776178466.660215:0:3352:0:(events.c:361:request_in_callback()) peer: 12345-0@lo (source: 12345-0@lo) 00000100:00100000:0.0:1776178466.660344:0:6255:0:(service.c:2152:ptlrpc_server_handle_req_in()) got req x1862456998085632 00000100:00100000:0.0:1776178466.660393:0:6255:0:(nrs_fifo.c:179:nrs_fifo_req_get()) NRS start fifo request from 12345-0@lo, seq: 6830 00000100:00100000:0.0:1776178466.660400:0:6255:0:(service.c:2324:ptlrpc_server_handle_request()) Handling RPC req@00000000a62daf9a pname:cluuid+ref:pid:xid:nid:opc:job mdt00_000:0+-99:3352:x1862456998085632:12345-0@lo:38:kworker/u8:2.0 00010000:02000000:0.0:1776178466.660417:0:6255:0:(ldlm_lib.c:1139:target_handle_connect()) lustre-MDT0000: Not available for connect from 0@lo (stopping) 00000100:00100000:0.0:1776178466.660460:0:6255:0:(service.c:2372:ptlrpc_server_handle_request()) Handled RPC req@00000000a62daf9a pname:cluuid+ref:pid:xid:nid:opc:job mdt00_000:0+-99:3352:x1862456998085632:12345-0@lo:38:kworker/u8:2.0 Request processed in 60us (247us total) trans 0 rc -19/-19 00000100:00100000:0.0:1776178466.660467:0:6255:0:(nrs_fifo.c:241:nrs_fifo_req_stop()) NRS stop fifo request from 12345-0@lo, seq: 6830 00000100:00100000:2.0:1776178466.660640:0:11281:0:(service.c:2372:ptlrpc_server_handle_request()) Handled RPC req@00000000f9929c4b pname:cluuid+ref:pid:xid:nid:opc:job mdt00_004:0+-99:3352:x1862456998085504:12345-0@lo:38:kworker/u8:2.0 Request processed in 3742us (4007us total) trans 0 rc -19/-19 00000100:00100000:2.0:1776178466.660659:0:11281:0:(nrs_fifo.c:241:nrs_fifo_req_stop()) NRS stop fifo request from 12345-0@lo, seq: 6829 00000100:00080000:3.0:1776178466.663188:0:37:0:(pinger.c:251:ptlrpc_pinger_process_import()) lustre-MDT0001-mdtlov_UUID->lustre-MDT0000_UUID: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00080000:3.0:1776178466.663200:0:37:0:(recover.c:57:ptlrpc_initiate_recovery()) lustre-MDT0000_UUID: starting recovery 00000100:00080000:3.0:1776178466.663205:0:37:0:(import.c:86:import_set_state_nolock()) 00000000ba4f1821 lustre-MDT0000_UUID: changing import state from DISCONN to CONNECTING 00000100:00080000:3.0:1776178466.663217:0:37:0:(import.c:534:import_select_connection()) lustre-MDT0000-osp-MDT0001: connect to NID 0@lo last attempt 1168 00000100:00080000:3.0:1776178466.663221:0:37:0:(import.c:581:import_select_connection()) lustre-MDT0000-osp-MDT0001: tried all connections, increasing latency to 21s 00000100:00080000:3.0:1776178466.663230:0:37:0:(import.c:615:import_select_connection()) lustre-MDT0000-osp-MDT0001: import 00000000ba4f1821 using connection 192.168.204.103@tcp/0@lo 00000100:00100000:3.0:1776178466.663262:0:37:0:(import.c:807:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5) req@0000000038c09127 x1862456998085760/t0(0) o38->lustre-MDT0000-osp-MDT0001@0@lo:24/4 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' 00000100:00100000:0.0:1776178466.663448:0:3356:0:(client.c:1733:ptlrpc_send_new_req()) Sending RPC req@000000006c7327ca pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_03:lustre-MDT0001-lwp-OST0001_UUID:3356:1862456998085696:0@lo:400:kworker/u8:2.0 00000100:00100000:0.0:1776178466.663527:0:3356:0:(events.c:361:request_in_callback()) peer: 12345-0@lo (source: 12345-0@lo) 00000100:00080000:3.0:1776178466.663626:0:37:0:(pinger.c:251:ptlrpc_pinger_process_import()) lustre-MDT0000-lwp-MDT0001_UUID->lustre-MDT0000_UUID: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00080000:3.0:1776178466.663630:0:37:0:(recover.c:57:ptlrpc_initiate_recovery()) lustre-MDT0000_UUID: starting recovery 00000100:00080000:3.0:1776178466.663632:0:37:0:(import.c:86:import_set_state_nolock()) 00000000a7c55b1f lustre-MDT0000_UUID: changing import state from DISCONN to CONNECTING 00000100:00080000:3.0:1776178466.663636:0:37:0:(import.c:534:import_select_connection()) lustre-MDT0000-lwp-MDT0001: connect to NID 0@lo last attempt 1168 00000100:00080000:3.0:1776178466.663638:0:37:0:(import.c:581:import_select_connection()) lustre-MDT0000-lwp-MDT0001: tried all connections, increasing latency to 21s 00000100:00080000:3.0:1776178466.663642:0:37:0:(import.c:615:import_select_connection()) lustre-MDT0000-lwp-MDT0001: import 00000000a7c55b1f using connection 192.168.204.103@tcp/0@lo 00000100:00100000:0.0:1776178466.663760:0:3354:0:(client.c:1733:ptlrpc_send_new_req()) Sending RPC req@00000000167dc455 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_01:lustre-MDT0001-mdtlov_UUID:3354:1862456998085888:0@lo:400:kworker/u8:2.0 00000100:00100000:0.0:1776178466.663794:0:3354:0:(events.c:361:request_in_callback()) peer: 12345-0@lo (source: 12345-0@lo) 00000100:00100000:0.0:1776178466.663814:0:3354:0:(ptlrpcd.c:415:ptlrpcd_check()) transfer 1 async RPCs [0->1] 00000100:00100000:0.0:1776178466.663820:0:3354:0:(client.c:1733:ptlrpc_send_new_req()) Sending RPC req@00000000bcefda18 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_01:lustre-MDT0001-mdtlov_UUID:3354:1862456998085824:0@lo:400:kworker/u8:2.0 00000100:00100000:0.0:1776178466.663842:0:3354:0:(events.c:361:request_in_callback()) peer: 12345-0@lo (source: 12345-0@lo) 00000100:00100000:3.0:1776178466.663962:0:37:0:(import.c:807:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5) req@00000000044bd507 x1862456998085952/t0(0) o38->lustre-MDT0000-lwp-MDT0001@0@lo:12/10 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' 00000100:00100000:3.0:1776178466.664012:0:20755:0:(service.c:2152:ptlrpc_server_handle_req_in()) got req x1862456998085888 00000100:00100000:3.0:1776178466.664107:0:20755:0:(service.c:2152:ptlrpc_server_handle_req_in()) got req x1862456998085824 00000100:00100000:3.0:1776178466.664131:0:20755:0:(nrs_fifo.c:179:nrs_fifo_req_get()) NRS start fifo request from 12345-0@lo, seq: 717 00000100:00100000:3.0:1776178466.664139:0:20755:0:(service.c:2324:ptlrpc_server_handle_request()) Handling RPC req@0000000042de09e9 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_006:lustre-MDT0001-mdtlov_UUID+5:3354:x1862456998085888:12345-0@lo:400:kworker/u8:2.0 00000100:00100000:3.0:1776178466.664211:0:20755:0:(service.c:2372:ptlrpc_server_handle_request()) Handled RPC req@0000000042de09e9 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_006:lustre-MDT0001-mdtlov_UUID+5:3354:x1862456998085888:12345-0@lo:400:kworker/u8:2.0 Request processed in 71us (416us total) trans 0 rc 0/0 00000100:00100000:3.0:1776178466.664219:0:20755:0:(nrs_fifo.c:241:nrs_fifo_req_stop()) NRS stop fifo request from 12345-0@lo, seq: 717 00000100:00100000:3.0:1776178466.664240:0:20755:0:(nrs_fifo.c:179:nrs_fifo_req_get()) NRS start fifo request from 12345-0@lo, seq: 718 00000100:00100000:3.0:1776178466.664244:0:20755:0:(service.c:2324:ptlrpc_server_handle_request()) Handling RPC req@0000000029e5c385 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_006:lustre-MDT0001-mdtlov_UUID+5:3354:x1862456998085824:12345-0@lo:400:kworker/u8:2.0 00000100:00100000:3.0:1776178466.664271:0:20755:0:(service.c:2372:ptlrpc_server_handle_request()) Handled RPC req@0000000029e5c385 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_006:lustre-MDT0001-mdtlov_UUID+5:3354:x1862456998085824:12345-0@lo:400:kworker/u8:2.0 Request processed in 26us (428us total) trans 0 rc 0/0 00000100:00100000:3.0:1776178466.664277:0:20755:0:(nrs_fifo.c:241:nrs_fifo_req_stop()) NRS stop fifo request from 12345-0@lo, seq: 718 00000100:00100000:3.0:1776178466.664397:0:3354:0:(client.c:2215:ptlrpc_check_set()) Completed RPC req@00000000bcefda18 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_01:lustre-MDT0001-mdtlov_UUID:3354:1862456998085824:0@lo:400:kworker/u8:2.0 00000100:00100000:3.0:1776178466.664407:0:3354:0:(client.c:2215:ptlrpc_check_set()) Completed RPC req@00000000167dc455 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_01:lustre-MDT0001-mdtlov_UUID:3354:1862456998085888:0@lo:400:kworker/u8:2.0 00000100:00100000:2.0:1776178466.664610:0:11281:0:(service.c:2152:ptlrpc_server_handle_req_in()) got req x1862456998085696 00000100:00100000:2.0:1776178466.664661:0:11281:0:(nrs_fifo.c:179:nrs_fifo_req_get()) NRS start fifo request from 12345-0@lo, seq: 1083 00000100:00100000:2.0:1776178466.664665:0:11281:0:(service.c:2324:ptlrpc_server_handle_request()) Handling RPC req@00000000ab295186 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_004:lustre-MDT0001-lwp-OST0001_UUID+5:3356:x1862456998085696:12345-0@lo:400:kworker/u8:2.0 00000100:00100000:2.0:1776178466.664717:0:11281:0:(service.c:2372:ptlrpc_server_handle_request()) Handled RPC req@00000000ab295186 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_004:lustre-MDT0001-lwp-OST0001_UUID+5:3356:x1862456998085696:12345-0@lo:400:kworker/u8:2.0 Request processed in 51us (1191us total) trans 0 rc 0/0 00000100:00100000:2.0:1776178466.664724:0:11281:0:(nrs_fifo.c:241:nrs_fifo_req_stop()) NRS stop fifo request from 12345-0@lo, seq: 1083 00000100:00100000:2.0:1776178466.665447:0:3356:0:(client.c:2215:ptlrpc_check_set()) Completed RPC req@000000006c7327ca pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_03:lustre-MDT0001-lwp-OST0001_UUID:3356:1862456998085696:0@lo:400:kworker/u8:2.0 00000100:00080000:1.0:1776178466.666469:0:3352:0:(import.c:86:import_set_state_nolock()) 0000000047601a98 lustre-MDT0000_UUID: changing import state from CONNECTING to DISCONN 00000100:00080000:1.0:1776178466.666479:0:3352:0:(import.c:1429:ptlrpc_connect_interpret()) recovery of lustre-MDT0000_UUID on 192.168.204.103@tcp failed (-19) 00000100:00100000:1.0:1776178466.666486:0:3352:0:(client.c:2215:ptlrpc_check_set()) Completed RPC req@0000000021b0d516 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:lustre-MDT0000-lwp-OST0000_UUID:3352:1862456998085504:0@lo:38:kworker/u8:2.0 00000100:00100000:1.0:1776178466.666515:0:3352:0:(client.c:1733:ptlrpc_send_new_req()) Sending RPC req@0000000038c09127 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:lustre-MDT0001-mdtlov_UUID:3352:1862456998085760:0@lo:38:kworker/u8:2.0 00000100:00100000:1.0:1776178466.666578:0:3352:0:(events.c:361:request_in_callback()) peer: 12345-0@lo (source: 12345-0@lo) 00000100:00100000:1.0:1776178466.666616:0:3352:0:(client.c:1733:ptlrpc_send_new_req()) Sending RPC req@00000000044bd507 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:lustre-MDT0000-lwp-MDT0001_UUID:3352:1862456998085952:0@lo:38:kworker/u8:2.0 00000100:00100000:1.0:1776178466.667165:0:3352:0:(events.c:361:request_in_callback()) peer: 12345-0@lo (source: 12345-0@lo) 00000100:00080000:1.0:1776178466.667210:0:3352:0:(import.c:86:import_set_state_nolock()) 00000000dfc8db13 lustre-MDT0000_UUID: changing import state from CONNECTING to DISCONN 00000100:00080000:1.0:1776178466.667215:0:3352:0:(import.c:1429:ptlrpc_connect_interpret()) recovery of lustre-MDT0000_UUID on 192.168.204.103@tcp failed (-19) 00000100:00100000:1.0:1776178466.667219:0:3352:0:(client.c:2215:ptlrpc_check_set()) Completed RPC req@00000000bdad24ac pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:lustre-MDT0000-lwp-OST0001_UUID:3352:1862456998085632:0@lo:38:kworker/u8:2.0 00000100:00100000:0.0:1776178466.667348:0:8250:0:(service.c:2152:ptlrpc_server_handle_req_in()) got req x1862456998085952 00000100:00100000:0.0:1776178466.667408:0:8250:0:(nrs_fifo.c:179:nrs_fifo_req_get()) NRS start fifo request from 12345-0@lo, seq: 6831 00000100:00100000:0.0:1776178466.667416:0:8250:0:(service.c:2324:ptlrpc_server_handle_request()) Handling RPC req@000000006a209b80 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_003:0+-99:3352:x1862456998085952:12345-0@lo:38:kworker/u8:2.0 00010000:02000000:0.0:1776178466.667437:0:8250:0:(ldlm_lib.c:1139:target_handle_connect()) lustre-MDT0000: Not available for connect from 0@lo (stopping) 00000100:00100000:0.0:1776178466.667501:0:8250:0:(service.c:2372:ptlrpc_server_handle_request()) Handled RPC req@000000006a209b80 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_003:0+-99:3352:x1862456998085952:12345-0@lo:38:kworker/u8:2.0 Request processed in 84us (854us total) trans 0 rc -19/-19 00000100:00100000:0.0:1776178466.667508:0:8250:0:(nrs_fifo.c:241:nrs_fifo_req_stop()) NRS stop fifo request from 12345-0@lo, seq: 6831 00000100:00080000:1.0:1776178466.667969:0:3352:0:(import.c:86:import_set_state_nolock()) 00000000a7c55b1f lustre-MDT0000_UUID: changing import state from CONNECTING to DISCONN 00000100:00080000:1.0:1776178466.667974:0:3352:0:(import.c:1429:ptlrpc_connect_interpret()) recovery of lustre-MDT0000_UUID on 192.168.204.103@tcp failed (-19) 00000100:00100000:1.0:1776178466.667978:0:3352:0:(client.c:2215:ptlrpc_check_set()) Completed RPC req@00000000044bd507 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:lustre-MDT0000-lwp-MDT0001_UUID:3352:1862456998085952:0@lo:38:kworker/u8:2.0 00000100:00100000:3.0:1776178466.668057:0:15213:0:(service.c:2152:ptlrpc_server_handle_req_in()) got req x1862456998085760 00000100:00100000:3.0:1776178466.668117:0:15213:0:(nrs_fifo.c:179:nrs_fifo_req_get()) NRS start fifo request from 12345-0@lo, seq: 759 00000100:00100000:3.0:1776178466.668121:0:15213:0:(service.c:2324:ptlrpc_server_handle_request()) Handling RPC req@00000000d3c6acc9 pname:cluuid+ref:pid:xid:nid:opc:job mdt_out00_003:0+-99:3352:x1862456998085760:12345-0@lo:38:kworker/u8:2.0 00010000:02000000:3.0:1776178466.668130:0:15213:0:(ldlm_lib.c:1139:target_handle_connect()) lustre-MDT0000: Not available for connect from 0@lo (stopping) 00000100:00100000:3.0:1776178466.668168:0:15213:0:(service.c:2372:ptlrpc_server_handle_request()) Handled RPC req@00000000d3c6acc9 pname:cluuid+ref:pid:xid:nid:opc:job mdt_out00_003:0+-99:3352:x1862456998085760:12345-0@lo:38:kworker/u8:2.0 Request processed in 47us (1591us total) trans 0 rc -19/-19 00000100:00100000:3.0:1776178466.668175:0:15213:0:(nrs_fifo.c:241:nrs_fifo_req_stop()) NRS stop fifo request from 12345-0@lo, seq: 759 00000100:00080000:0.0:1776178466.668592:0:3352:0:(import.c:86:import_set_state_nolock()) 00000000ba4f1821 lustre-MDT0000_UUID: changing import state from CONNECTING to DISCONN 00000100:00080000:0.0:1776178466.668598:0:3352:0:(import.c:1429:ptlrpc_connect_interpret()) recovery of lustre-MDT0000_UUID on 192.168.204.103@tcp failed (-19) 00000100:00100000:0.0:1776178466.668603:0:3352:0:(client.c:2215:ptlrpc_check_set()) Completed RPC req@0000000038c09127 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:lustre-MDT0001-mdtlov_UUID:3352:1862456998085760:0@lo:38:kworker/u8:2.0 00010000:00010000:3.0:1776178467.296391:0:6244:0:(ldlm_lockd.c:2792:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:2.0:1776178468.192407:0:3355:0:(client.c:1733:ptlrpc_send_new_req()) Sending RPC req@00000000b8464782 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_02:lustre-MDT0001-mdtlov_UUID:3355:1862456998086016:0@lo:13:osp-pre-1-1.0 00000100:00100000:2.0:1776178468.192482:0:3355:0:(events.c:361:request_in_callback()) peer: 12345-0@lo (source: 12345-0@lo) 00000100:00100000:1.0:1776178468.194864:0:9554:0:(service.c:2152:ptlrpc_server_handle_req_in()) got req x1862456998086016 00000100:00100000:1.0:1776178468.194959:0:9554:0:(nrs_fifo.c:179:nrs_fifo_req_get()) NRS start fifo request from 12345-0@lo, seq: 591 00000100:00100000:1.0:1776178468.194966:0:9554:0:(service.c:2324:ptlrpc_server_handle_request()) Handling RPC req@000000001c8035b1 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_create00:lustre-MDT0001-mdtlov_UUID+5:3355:x1862456998086016:12345-0@lo:13:osp-pre-1-1.0 00002000:00000024:1.0:1776178468.195012:0:9554:0:(ofd_obd.c:740:ofd_statfs()) blocks cached 229376 granted 491746368 pending 0 free 3923689472 avail 3692167168 00000100:00100000:1.0:1776178468.195151:0:9554:0:(service.c:2372:ptlrpc_server_handle_request()) Handled RPC req@000000001c8035b1 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_create00:lustre-MDT0001-mdtlov_UUID+5:3355:x1862456998086016:12345-0@lo:13:osp-pre-1-1.0 Request processed in 184us (2669us total) trans 0 rc 0/0 00000100:00100000:1.0:1776178468.195162:0:9554:0:(nrs_fifo.c:241:nrs_fifo_req_stop()) NRS stop fifo request from 12345-0@lo, seq: 591 00000100:00100000:2.0:1776178468.195266:0:3355:0:(client.c:2215:ptlrpc_check_set()) Completed RPC req@00000000b8464782 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_02:lustre-MDT0001-mdtlov_UUID:3355:1862456998086016:0@lo:13:osp-pre-1-1.0 00010000:00010000:2.0:1776178468.320541:0:6245:0:(ldlm_lockd.c:2792:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:3.0:1776178468.704435:0:3355:0:(ptlrpcd.c:415:ptlrpcd_check()) transfer 1 async RPCs [3->2] 00000100:00100000:3.0:1776178468.704456:0:3355:0:(client.c:1733:ptlrpc_send_new_req()) Sending RPC req@000000006a3d91b0 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_02:lustre-MDT0001-mdtlov_UUID:3355:1862456998086080:0@lo:13:osp-pre-0-1.0 00000100:00100000:3.0:1776178468.704535:0:3355:0:(events.c:361:request_in_callback()) peer: 12345-0@lo (source: 12345-0@lo) 00000100:00100000:0.0:1776178468.704714:0:8231:0:(service.c:2152:ptlrpc_server_handle_req_in()) got req x1862456998086080 00000100:00100000:0.0:1776178468.704771:0:8231:0:(nrs_fifo.c:179:nrs_fifo_req_get()) NRS start fifo request from 12345-0@lo, seq: 592 00000100:00100000:0.0:1776178468.704777:0:8231:0:(service.c:2324:ptlrpc_server_handle_request()) Handling RPC req@0000000023562990 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_create00:lustre-MDT0001-mdtlov_UUID+5:3355:x1862456998086080:12345-0@lo:13:osp-pre-0-1.0 00002000:00000024:0.0:1776178468.704805:0:8231:0:(ofd_obd.c:740:ofd_statfs()) blocks cached 315392 granted 484883776 pending 0 free 3923689472 avail 3692167168 00000100:00100000:0.0:1776178468.704848:0:8231:0:(service.c:2372:ptlrpc_server_handle_request()) Handled RPC req@0000000023562990 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_create00:lustre-MDT0001-mdtlov_UUID+5:3355:x1862456998086080:12345-0@lo:13:osp-pre-0-1.0 Request processed in 71us (315us total) trans 0 rc 0/0 00000100:00100000:0.0:1776178468.704858:0:8231:0:(nrs_fifo.c:241:nrs_fifo_req_stop()) NRS stop fifo request from 12345-0@lo, seq: 592 00000100:00100000:3.0:1776178468.704940:0:3355:0:(client.c:2215:ptlrpc_check_set()) Completed RPC req@000000006a3d91b0 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_02:lustre-MDT0001-mdtlov_UUID:3355:1862456998086080:0@lo:13:osp-pre-0-1.0 00010000:00010000:1.0:1776178469.344492:0:9224:0:(ldlm_lockd.c:2792:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:3.0:1776178470.178506:0:3348:0:(events.c:361:request_in_callback()) peer: 12345-192.168.204.3@tcp (source: 12345-192.168.204.3@tcp) 00000100:00100000:3.0:1776178470.178554:0:3348:0:(events.c:361:request_in_callback()) peer: 12345-192.168.204.3@tcp (source: 12345-192.168.204.3@tcp) 00000100:00100000:3.0:1776178470.178613:0:3348:0:(events.c:361:request_in_callback()) peer: 12345-192.168.204.3@tcp (source: 12345-192.168.204.3@tcp) 00000100:00100000:3.0:1776178470.178667:0:3348:0:(events.c:361:request_in_callback()) peer: 12345-192.168.204.3@tcp (source: 12345-192.168.204.3@tcp) 00000100:00100000:2.0:1776178470.178782:0:20759:0:(service.c:2152:ptlrpc_server_handle_req_in()) got req x1862456979125888 00000100:00100000:2.0:1776178470.178846:0:20759:0:(service.c:2152:ptlrpc_server_handle_req_in()) got req x1862456979125824 00000100:00100000:2.0:1776178470.178873:0:20759:0:(nrs_fifo.c:179:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.204.3@tcp, seq: 719 00000100:00100000:2.0:1776178470.178881:0:20759:0:(service.c:2324:ptlrpc_server_handle_request()) Handling RPC req@00000000167dc455 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_010:b50963bd-8eba-4432-9a54-8888d00e3fa1+9:2273:x1862456979125888:12345-192.168.204.3@tcp:400:kworker/u8:4.0 00000100:00100000:2.0:1776178470.178974:0:20759:0:(service.c:2372:ptlrpc_server_handle_request()) Handled RPC req@00000000167dc455 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_010:b50963bd-8eba-4432-9a54-8888d00e3fa1+9:2273:x1862456979125888:12345-192.168.204.3@tcp:400:kworker/u8:4.0 Request processed in 93us (419us total) trans 0 rc 0/0 00000100:00100000:2.0:1776178470.178981:0:20759:0:(nrs_fifo.c:241:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.204.3@tcp, seq: 719 00000100:00100000:2.0:1776178470.179000:0:20759:0:(nrs_fifo.c:179:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.204.3@tcp, seq: 720 00000100:00100000:2.0:1776178470.179004:0:20759:0:(service.c:2324:ptlrpc_server_handle_request()) Handling RPC req@00000000bcefda18 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_010:b50963bd-8eba-4432-9a54-8888d00e3fa1+8:2272:x1862456979125824:12345-192.168.204.3@tcp:400:kworker/u8:4.0 00000100:00100000:2.0:1776178470.179027:0:20759:0:(service.c:2372:ptlrpc_server_handle_request()) Handled RPC req@00000000bcefda18 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_010:b50963bd-8eba-4432-9a54-8888d00e3fa1+8:2272:x1862456979125824:12345-192.168.204.3@tcp:400:kworker/u8:4.0 Request processed in 22us (361us total) trans 0 rc 0/0 00000100:00100000:2.0:1776178470.179033:0:20759:0:(nrs_fifo.c:241:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.204.3@tcp, seq: 720 00000100:00100000:2.0:1776178470.179212:0:11281:0:(service.c:2152:ptlrpc_server_handle_req_in()) got req x1862456979125760 00000100:00100000:2.0:1776178470.179242:0:11281:0:(nrs_fifo.c:179:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.204.3@tcp, seq: 1084 00000100:00100000:2.0:1776178470.179246:0:11281:0:(service.c:2324:ptlrpc_server_handle_request()) Handling RPC req@00000000638772fd pname:cluuid+ref:pid:xid:nid:opc:job mdt00_004:b50963bd-8eba-4432-9a54-8888d00e3fa1+6:2273:x1862456979125760:12345-192.168.204.3@tcp:400:kworker/u8:4.0 00000100:00100000:2.0:1776178470.179274:0:11281:0:(service.c:2372:ptlrpc_server_handle_request()) Handled RPC req@00000000638772fd pname:cluuid+ref:pid:xid:nid:opc:job mdt00_004:b50963bd-8eba-4432-9a54-8888d00e3fa1+6:2273:x1862456979125760:12345-192.168.204.3@tcp:400:kworker/u8:4.0 Request processed in 27us (778us total) trans 0 rc 0/0 00000100:00100000:2.0:1776178470.179279:0:11281:0:(nrs_fifo.c:241:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.204.3@tcp, seq: 1084 00000100:00100000:1.0:1776178470.181267:0:20137:0:(service.c:2152:ptlrpc_server_handle_req_in()) got req x1862456979125632 00000100:00100000:1.0:1776178470.181337:0:20137:0:(nrs_fifo.c:179:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.204.3@tcp, seq: 259 00000100:00100000:1.0:1776178470.181343:0:20137:0:(service.c:2324:ptlrpc_server_handle_request()) Handling RPC req@00000000f240c829 pname:cluuid+ref:pid:xid:nid:opc:job ll_mgs_0000:5b5c66b1-1a2f-4651-8548-0c75b9a38a94+9:2272:x1862456979125632:12345-192.168.204.3@tcp:400:kworker/u8:4.0 00000100:00100000:1.0:1776178470.181422:0:20137:0:(service.c:2372:ptlrpc_server_handle_request()) Handled RPC req@00000000f240c829 pname:cluuid+ref:pid:xid:nid:opc:job ll_mgs_0000:5b5c66b1-1a2f-4651-8548-0c75b9a38a94+9:2272:x1862456979125632:12345-192.168.204.3@tcp:400:kworker/u8:4.0 Request processed in 78us (2808us total) trans 0 rc 0/0 00000100:00100000:1.0:1776178470.181430:0:20137:0:(nrs_fifo.c:241:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.204.3@tcp, seq: 259 00000100:00100000:2.0:1776178470.186287:0:3347:0:(events.c:361:request_in_callback()) peer: 12345-192.168.204.3@tcp (source: 12345-192.168.204.3@tcp) 00000100:00100000:0.0:1776178470.186507:0:8250:0:(service.c:2152:ptlrpc_server_handle_req_in()) got req x1862456979125696 00000100:00100000:0.0:1776178470.186569:0:8250:0:(nrs_fifo.c:179:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.204.3@tcp, seq: 6832 00000100:00100000:0.0:1776178470.186576:0:8250:0:(service.c:2324:ptlrpc_server_handle_request()) Handling RPC req@00000000865e4f65 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_003:0+-99:2269:x1862456979125696:12345-192.168.204.3@tcp:38:kworker/u8:4.0 00010000:02000000:0.0:1776178470.186594:0:8250:0:(ldlm_lib.c:1139:target_handle_connect()) lustre-MDT0000: Not available for connect from 192.168.204.3@tcp (stopping) 00000100:00100000:0.0:1776178470.186869:0:8250:0:(service.c:2372:ptlrpc_server_handle_request()) Handled RPC req@00000000865e4f65 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_003:0+-99:2269:x1862456979125696:12345-192.168.204.3@tcp:38:kworker/u8:4.0 Request processed in 292us (592us total) trans 0 rc -19/-19 00000100:00100000:0.0:1776178470.186878:0:8250:0:(nrs_fifo.c:241:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.204.3@tcp, seq: 6832 00010000:00010000:1.0:1776178470.369187:0:9225:0:(ldlm_lockd.c:2792:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00010000:00010000:3.0:1776178471.392389:0:6244:0:(ldlm_lockd.c:2792:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00080000:1.0:1776178471.776415:0:36:0:(pinger.c:251:ptlrpc_pinger_process_import()) lustre-MDT0000-lwp-OST0000_UUID->lustre-MDT0000_UUID: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00080000:1.0:1776178471.776423:0:36:0:(recover.c:57:ptlrpc_initiate_recovery()) lustre-MDT0000_UUID: starting recovery 00000100:00080000:1.0:1776178471.776427:0:36:0:(import.c:86:import_set_state_nolock()) 0000000047601a98 lustre-MDT0000_UUID: changing import state from DISCONN to CONNECTING 00000100:00080000:1.0:1776178471.776436:0:36:0:(import.c:534:import_select_connection()) lustre-MDT0000-lwp-OST0000: connect to NID 0@lo last attempt 1173 00000100:00080000:1.0:1776178471.776440:0:36:0:(import.c:581:import_select_connection()) lustre-MDT0000-lwp-OST0000: tried all connections, increasing latency to 21s 00000100:00100000:0.0:1776178471.776536:0:3353:0:(client.c:1733:ptlrpc_send_new_req()) Sending RPC req@00000000ba273b2a pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_00:ac61f839-0e8e-4cac-9c59-08bcfbbad330:3353:1862456998086144:0@lo:400:kworker/u8:1.0 00000100:00100000:0.0:1776178471.776622:0:3353:0:(events.c:361:request_in_callback()) peer: 12345-0@lo (source: 12345-0@lo) 00000100:00100000:2.0:1776178471.776756:0:20137:0:(service.c:2152:ptlrpc_server_handle_req_in()) got req x1862456998086144 00000100:00100000:2.0:1776178471.776807:0:20137:0:(nrs_fifo.c:179:nrs_fifo_req_get()) NRS start fifo request from 12345-0@lo, seq: 260 00000100:00100000:2.0:1776178471.776814:0:20137:0:(service.c:2324:ptlrpc_server_handle_request()) Handling RPC req@0000000091f9a08d pname:cluuid+ref:pid:xid:nid:opc:job ll_mgs_0000:ac61f839-0e8e-4cac-9c59-08bcfbbad330+18:3353:x1862456998086144:12345-0@lo:400:kworker/u8:1.0 00000100:00100000:2.0:1776178471.776867:0:20137:0:(service.c:2372:ptlrpc_server_handle_request()) Handled RPC req@0000000091f9a08d pname:cluuid+ref:pid:xid:nid:opc:job ll_mgs_0000:ac61f839-0e8e-4cac-9c59-08bcfbbad330+18:3353:x1862456998086144:12345-0@lo:400:kworker/u8:1.0 Request processed in 53us (246us total) trans 0 rc 0/0 00000100:00100000:2.0:1776178471.776874:0:20137:0:(nrs_fifo.c:241:nrs_fifo_req_stop()) NRS stop fifo request from 12345-0@lo, seq: 260 00000100:00100000:2.0:1776178471.776918:0:3353:0:(client.c:2215:ptlrpc_check_set()) Completed RPC req@00000000ba273b2a pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_00:ac61f839-0e8e-4cac-9c59-08bcfbbad330:3353:1862456998086144:0@lo:400:kworker/u8:1.0 00000100:00080000:1.0:1776178471.776966:0:36:0:(import.c:615:import_select_connection()) lustre-MDT0000-lwp-OST0000: import 0000000047601a98 using connection 192.168.204.103@tcp/0@lo 00000100:00100000:1.0:1776178471.776993:0:36:0:(import.c:807:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5) req@0000000017d54e92 x1862456998086208/t0(0) o38->lustre-MDT0000-lwp-OST0000@0@lo:12/10 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' 00000100:00100000:2.0:1776178471.777041:0:3352:0:(client.c:1733:ptlrpc_send_new_req()) Sending RPC req@0000000017d54e92 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:lustre-MDT0000-lwp-OST0000_UUID:3352:1862456998086208:0@lo:38:kworker/u8:1.0 00000100:00100000:2.0:1776178471.777096:0:3352:0:(events.c:361:request_in_callback()) peer: 12345-0@lo (source: 12345-0@lo) 00000100:00080000:1.0:1776178471.777210:0:36:0:(pinger.c:251:ptlrpc_pinger_process_import()) lustre-MDT0000-lwp-OST0001_UUID->lustre-MDT0000_UUID: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00080000:1.0:1776178471.777214:0:36:0:(recover.c:57:ptlrpc_initiate_recovery()) lustre-MDT0000_UUID: starting recovery 00000100:00080000:1.0:1776178471.777216:0:36:0:(import.c:86:import_set_state_nolock()) 00000000dfc8db13 lustre-MDT0000_UUID: changing import state from DISCONN to CONNECTING 00000100:00080000:1.0:1776178471.777220:0:36:0:(import.c:534:import_select_connection()) lustre-MDT0000-lwp-OST0001: connect to NID 0@lo last attempt 1173 00000100:00080000:1.0:1776178471.777222:0:36:0:(import.c:581:import_select_connection()) lustre-MDT0000-lwp-OST0001: tried all connections, increasing latency to 21s 00000100:00080000:1.0:1776178471.777226:0:36:0:(import.c:615:import_select_connection()) lustre-MDT0000-lwp-OST0001: import 00000000dfc8db13 using connection 192.168.204.103@tcp/0@lo 00000100:00100000:1.0:1776178471.777237:0:36:0:(import.c:807:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5) req@00000000d2194868 x1862456998086336/t0(0) o38->lustre-MDT0000-lwp-OST0001@0@lo:12/10 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' 00000100:00100000:0.0:1776178471.777277:0:3353:0:(ptlrpcd.c:415:ptlrpcd_check()) transfer 1 async RPCs [1->0] 00000100:00100000:0.0:1776178471.777285:0:3353:0:(client.c:1733:ptlrpc_send_new_req()) Sending RPC req@000000006466ee18 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_00:lustre-MDT0001-lwp-OST0000_UUID:3353:1862456998086272:0@lo:400:kworker/u8:1.0 00000100:00100000:0.0:1776178471.777311:0:3353:0:(events.c:361:request_in_callback()) peer: 12345-0@lo (source: 12345-0@lo) 00000100:00100000:0.0:1776178471.777351:0:8250:0:(service.c:2152:ptlrpc_server_handle_req_in()) got req x1862456998086208 00000100:00100000:0.0:1776178471.777376:0:8250:0:(service.c:2152:ptlrpc_server_handle_req_in()) got req x1862456998086272 00000100:00100000:0.0:1776178471.777396:0:8250:0:(nrs_fifo.c:179:nrs_fifo_req_get()) NRS start fifo request from 12345-0@lo, seq: 1085 00000100:00100000:0.0:1776178471.777399:0:8250:0:(service.c:2324:ptlrpc_server_handle_request()) Handling RPC req@0000000075189f40 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_003:lustre-MDT0001-lwp-OST0000_UUID+5:3353:x1862456998086272:12345-0@lo:400:kworker/u8:1.0 00000100:00100000:0.0:1776178471.777434:0:8250:0:(service.c:2372:ptlrpc_server_handle_request()) Handled RPC req@0000000075189f40 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_003:lustre-MDT0001-lwp-OST0000_UUID+5:3353:x1862456998086272:12345-0@lo:400:kworker/u8:1.0 Request processed in 34us (123us total) trans 0 rc 0/0 00000100:00100000:0.0:1776178471.777440:0:8250:0:(nrs_fifo.c:241:nrs_fifo_req_stop()) NRS stop fifo request from 12345-0@lo, seq: 1085 00000100:00100000:0.0:1776178471.777464:0:8250:0:(nrs_fifo.c:179:nrs_fifo_req_get()) NRS start fifo request from 12345-0@lo, seq: 6833 00000100:00100000:0.0:1776178471.777467:0:8250:0:(service.c:2324:ptlrpc_server_handle_request()) Handling RPC req@0000000071333ecf pname:cluuid+ref:pid:xid:nid:opc:job mdt00_003:0+-99:3352:x1862456998086208:12345-0@lo:38:kworker/u8:1.0 00010000:02000000:0.0:1776178471.777479:0:8250:0:(ldlm_lib.c:1139:target_handle_connect()) lustre-MDT0000: Not available for connect from 0@lo (stopping) 00000100:00100000:3.0:1776178471.781424:0:3353:0:(client.c:2215:ptlrpc_check_set()) Completed RPC req@000000006466ee18 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_00:lustre-MDT0001-lwp-OST0000_UUID:3353:1862456998086272:0@lo:400:kworker/u8:1.0 00000100:00100000:2.0:1776178471.781517:0:3352:0:(client.c:1733:ptlrpc_send_new_req()) Sending RPC req@00000000d2194868 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:lustre-MDT0000-lwp-OST0001_UUID:3352:1862456998086336:0@lo:38:kworker/u8:1.0 00000100:00100000:2.0:1776178471.781590:0:3352:0:(events.c:361:request_in_callback()) peer: 12345-0@lo (source: 12345-0@lo) 00000100:00080000:1.0:1776178471.781725:0:36:0:(pinger.c:251:ptlrpc_pinger_process_import()) lustre-MDT0001-mdtlov_UUID->lustre-MDT0000_UUID: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00080000:1.0:1776178471.781731:0:36:0:(recover.c:57:ptlrpc_initiate_recovery()) lustre-MDT0000_UUID: starting recovery 00000100:00080000:1.0:1776178471.781734:0:36:0:(import.c:86:import_set_state_nolock()) 00000000ba4f1821 lustre-MDT0000_UUID: changing import state from DISCONN to CONNECTING 00000100:00080000:1.0:1776178471.781740:0:36:0:(import.c:534:import_select_connection()) lustre-MDT0000-osp-MDT0001: connect to NID 0@lo last attempt 1173 00000100:00080000:1.0:1776178471.781742:0:36:0:(import.c:581:import_select_connection()) lustre-MDT0000-osp-MDT0001: tried all connections, increasing latency to 21s 00000100:00080000:1.0:1776178471.781747:0:36:0:(import.c:615:import_select_connection()) lustre-MDT0000-osp-MDT0001: import 00000000ba4f1821 using connection 192.168.204.103@tcp/0@lo 00000100:00100000:1.0:1776178471.781769:0:36:0:(import.c:807:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5) req@000000006466ee18 x1862456998086464/t0(0) o38->lustre-MDT0000-osp-MDT0001@0@lo:24/4 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' 00000100:00080000:1.0:1776178471.781801:0:36:0:(pinger.c:251:ptlrpc_pinger_process_import()) lustre-MDT0000-lwp-MDT0001_UUID->lustre-MDT0000_UUID: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00080000:1.0:1776178471.781805:0:36:0:(recover.c:57:ptlrpc_initiate_recovery()) lustre-MDT0000_UUID: starting recovery 00000100:00080000:1.0:1776178471.781806:0:36:0:(import.c:86:import_set_state_nolock()) 00000000a7c55b1f lustre-MDT0000_UUID: changing import state from DISCONN to CONNECTING 00000100:00080000:1.0:1776178471.781809:0:36:0:(import.c:534:import_select_connection()) lustre-MDT0000-lwp-MDT0001: connect to NID 0@lo last attempt 1173 00000100:00080000:1.0:1776178471.781811:0:36:0:(import.c:581:import_select_connection()) lustre-MDT0000-lwp-MDT0001: tried all connections, increasing latency to 21s 00000100:00080000:1.0:1776178471.781814:0:36:0:(import.c:615:import_select_connection()) lustre-MDT0000-lwp-MDT0001: import 00000000a7c55b1f using connection 192.168.204.103@tcp/0@lo 00000100:00100000:1.0:1776178471.781827:0:36:0:(import.c:807:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5) req@000000001f977915 x1862456998086656/t0(0) o38->lustre-MDT0000-lwp-MDT0001@0@lo:12/10 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' 00000100:00100000:1.0:1776178471.781856:0:3355:0:(client.c:1733:ptlrpc_send_new_req()) Sending RPC req@00000000cb1b73cb pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_02:lustre-MDT0001-lwp-OST0001_UUID:3355:1862456998086400:0@lo:400:kworker/u8:1.0 00000100:00100000:1.0:1776178471.781888:0:3354:0:(ptlrpcd.c:415:ptlrpcd_check()) transfer 1 async RPCs [0->1] 00000100:00100000:1.0:1776178471.781897:0:3354:0:(client.c:1733:ptlrpc_send_new_req()) Sending RPC req@00000000c1934814 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_01:lustre-MDT0001-mdtlov_UUID:3354:1862456998086592:0@lo:400:kworker/u8:1.0 00000100:00100000:1.0:1776178471.781922:0:3354:0:(events.c:361:request_in_callback()) peer: 12345-0@lo (source: 12345-0@lo) 00000100:00100000:1.0:1776178471.781965:0:14440:0:(service.c:2152:ptlrpc_server_handle_req_in()) got req x1862456998086592 00000100:00100000:1.0:1776178471.782009:0:14440:0:(nrs_fifo.c:179:nrs_fifo_req_get()) NRS start fifo request from 12345-0@lo, seq: 721 00000100:00100000:1.0:1776178471.782015:0:14440:0:(service.c:2324:ptlrpc_server_handle_request()) Handling RPC req@00000000b0409689 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_004:lustre-MDT0001-mdtlov_UUID+5:3354:x1862456998086592:12345-0@lo:400:kworker/u8:1.0 00000100:00100000:1.0:1776178471.782074:0:14440:0:(service.c:2372:ptlrpc_server_handle_request()) Handled RPC req@00000000b0409689 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_004:lustre-MDT0001-mdtlov_UUID+5:3354:x1862456998086592:12345-0@lo:400:kworker/u8:1.0 Request processed in 43us (136us total) trans 0 rc 0/0 00000100:00100000:1.0:1776178471.782081:0:14440:0:(nrs_fifo.c:241:nrs_fifo_req_stop()) NRS stop fifo request from 12345-0@lo, seq: 721 00000100:00100000:1.0:1776178471.782114:0:3354:0:(client.c:2215:ptlrpc_check_set()) Completed RPC req@00000000c1934814 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_01:lustre-MDT0001-mdtlov_UUID:3354:1862456998086592:0@lo:400:kworker/u8:1.0 00000100:00100000:1.0:1776178471.782141:0:3356:0:(client.c:1733:ptlrpc_send_new_req()) Sending RPC req@00000000ba273b2a pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_03:lustre-MDT0001-mdtlov_UUID:3356:1862456998086528:0@lo:400:kworker/u8:1.0 00000100:00100000:1.0:1776178471.782164:0:3356:0:(events.c:361:request_in_callback()) peer: 12345-0@lo (source: 12345-0@lo) 00000100:00100000:1.0:1776178471.782189:0:20762:0:(service.c:2152:ptlrpc_server_handle_req_in()) got req x1862456998086528 00000100:00100000:1.0:1776178471.782209:0:20762:0:(nrs_fifo.c:179:nrs_fifo_req_get()) NRS start fifo request from 12345-0@lo, seq: 722 00000100:00100000:1.0:1776178471.782212:0:20762:0:(service.c:2324:ptlrpc_server_handle_request()) Handling RPC req@000000008c3fbdb6 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_013:lustre-MDT0001-mdtlov_UUID+5:3356:x1862456998086528:12345-0@lo:400:kworker/u8:1.0 00000100:00100000:1.0:1776178471.782242:0:20762:0:(service.c:2372:ptlrpc_server_handle_request()) Handled RPC req@000000008c3fbdb6 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_013:lustre-MDT0001-mdtlov_UUID+5:3356:x1862456998086528:12345-0@lo:400:kworker/u8:1.0 Request processed in 29us (78us total) trans 0 rc 0/0 00000100:00100000:1.0:1776178471.782247:0:20762:0:(nrs_fifo.c:241:nrs_fifo_req_stop()) NRS stop fifo request from 12345-0@lo, seq: 722 00000100:00100000:1.0:1776178471.782267:0:3356:0:(client.c:2215:ptlrpc_check_set()) Completed RPC req@00000000ba273b2a pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_03:lustre-MDT0001-mdtlov_UUID:3356:1862456998086528:0@lo:400:kworker/u8:1.0 00000100:00100000:1.0:1776178471.782380:0:3355:0:(events.c:361:request_in_callback()) peer: 12345-0@lo (source: 12345-0@lo) 00000100:00100000:0.0:1776178471.782452:0:8250:0:(service.c:2372:ptlrpc_server_handle_request()) Handled RPC req@0000000071333ecf pname:cluuid+ref:pid:xid:nid:opc:job mdt00_003:0+-99:3352:x1862456998086208:12345-0@lo:38:kworker/u8:1.0 Request processed in 4983us (5356us total) trans 0 rc -19/-19 00000100:00100000:0.0:1776178471.782464:0:8250:0:(nrs_fifo.c:241:nrs_fifo_req_stop()) NRS stop fifo request from 12345-0@lo, seq: 6833 00000100:00080000:2.0:1776178471.786185:0:3352:0:(import.c:86:import_set_state_nolock()) 0000000047601a98 lustre-MDT0000_UUID: changing import state from CONNECTING to DISCONN 00000100:00080000:2.0:1776178471.786199:0:3352:0:(import.c:1429:ptlrpc_connect_interpret()) recovery of lustre-MDT0000_UUID on 192.168.204.103@tcp failed (-19) 00000100:00100000:2.0:1776178471.786208:0:3352:0:(client.c:2215:ptlrpc_check_set()) Completed RPC req@0000000017d54e92 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:lustre-MDT0000-lwp-OST0000_UUID:3352:1862456998086208:0@lo:38:kworker/u8:1.0 00000100:00100000:2.0:1776178471.786248:0:3352:0:(client.c:1733:ptlrpc_send_new_req()) Sending RPC req@000000006466ee18 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:lustre-MDT0001-mdtlov_UUID:3352:1862456998086464:0@lo:38:kworker/u8:1.0 00000100:00100000:2.0:1776178471.786369:0:3352:0:(events.c:361:request_in_callback()) peer: 12345-0@lo (source: 12345-0@lo) 00000100:00100000:2.0:1776178471.786405:0:3352:0:(client.c:1733:ptlrpc_send_new_req()) Sending RPC req@000000001f977915 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:lustre-MDT0000-lwp-MDT0001_UUID:3352:1862456998086656:0@lo:38:kworker/u8:1.0 00000100:00100000:2.0:1776178471.786470:0:14128:0:(service.c:2152:ptlrpc_server_handle_req_in()) got req x1862456998086464 00000100:00100000:2.0:1776178471.786522:0:14128:0:(nrs_fifo.c:179:nrs_fifo_req_get()) NRS start fifo request from 12345-0@lo, seq: 760 00000100:00100000:2.0:1776178471.786528:0:14128:0:(service.c:2324:ptlrpc_server_handle_request()) Handling RPC req@0000000045560455 pname:cluuid+ref:pid:xid:nid:opc:job mdt_out00_002:0+-99:3352:x1862456998086464:12345-0@lo:38:kworker/u8:1.0 00010000:02000000:2.0:1776178471.786545:0:14128:0:(ldlm_lib.c:1139:target_handle_connect()) lustre-MDT0000: Not available for connect from 0@lo (stopping) 00000100:00100000:2.0:1776178471.786584:0:14128:0:(service.c:2372:ptlrpc_server_handle_request()) Handled RPC req@0000000045560455 pname:cluuid+ref:pid:xid:nid:opc:job mdt_out00_002:0+-99:3352:x1862456998086464:12345-0@lo:38:kworker/u8:1.0 Request processed in 55us (216us total) trans 0 rc -19/-19 00000100:00100000:2.0:1776178471.786590:0:14128:0:(nrs_fifo.c:241:nrs_fifo_req_stop()) NRS stop fifo request from 12345-0@lo, seq: 760 00000100:00100000:0.0:1776178471.787192:0:8250:0:(service.c:2152:ptlrpc_server_handle_req_in()) got req x1862456998086336 00000100:00100000:0.0:1776178471.787220:0:8250:0:(service.c:2152:ptlrpc_server_handle_req_in()) got req x1862456998086400 00000100:00100000:0.0:1776178471.787255:0:6257:0:(nrs_fifo.c:179:nrs_fifo_req_get()) NRS start fifo request from 12345-0@lo, seq: 1086 00000100:00100000:0.0:1776178471.787259:0:6257:0:(service.c:2324:ptlrpc_server_handle_request()) Handling RPC req@00000000ef862c6d pname:cluuid+ref:pid:xid:nid:opc:job mdt00_002:lustre-MDT0001-lwp-OST0001_UUID+5:3355:x1862456998086400:12345-0@lo:400:kworker/u8:1.0 00000100:00100000:0.0:1776178471.787311:0:6257:0:(service.c:2372:ptlrpc_server_handle_request()) Handled RPC req@00000000ef862c6d pname:cluuid+ref:pid:xid:nid:opc:job mdt00_002:lustre-MDT0001-lwp-OST0001_UUID+5:3355:x1862456998086400:12345-0@lo:400:kworker/u8:1.0 Request processed in 50us (4931us total) trans 0 rc 0/0 00000100:00100000:0.0:1776178471.787317:0:6257:0:(nrs_fifo.c:241:nrs_fifo_req_stop()) NRS stop fifo request from 12345-0@lo, seq: 1086 00000100:00100000:0.0:1776178471.787333:0:6257:0:(nrs_fifo.c:179:nrs_fifo_req_get()) NRS start fifo request from 12345-0@lo, seq: 6834 00000100:00100000:0.0:1776178471.787336:0:6257:0:(service.c:2324:ptlrpc_server_handle_request()) Handling RPC req@000000007cbab86b pname:cluuid+ref:pid:xid:nid:opc:job mdt00_002:0+-99:3352:x1862456998086336:12345-0@lo:38:kworker/u8:1.0 00010000:02000000:0.0:1776178471.787345:0:6257:0:(ldlm_lib.c:1139:target_handle_connect()) lustre-MDT0000: Not available for connect from 0@lo (stopping) 00000100:00100000:0.0:1776178471.787369:0:6257:0:(service.c:2372:ptlrpc_server_handle_request()) Handled RPC req@000000007cbab86b pname:cluuid+ref:pid:xid:nid:opc:job mdt00_002:0+-99:3352:x1862456998086336:12345-0@lo:38:kworker/u8:1.0 Request processed in 32us (5781us total) trans 0 rc -19/-19 00000100:00100000:0.0:1776178471.787374:0:6257:0:(nrs_fifo.c:241:nrs_fifo_req_stop()) NRS stop fifo request from 12345-0@lo, seq: 6834 00000100:00100000:0.0:1776178471.787412:0:3355:0:(client.c:2215:ptlrpc_check_set()) Completed RPC req@00000000cb1b73cb pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_02:lustre-MDT0001-lwp-OST0001_UUID:3355:1862456998086400:0@lo:400:kworker/u8:1.0 00000100:00100000:1.0:1776178471.787984:0:3352:0:(events.c:361:request_in_callback()) peer: 12345-0@lo (source: 12345-0@lo) 00000100:00080000:1.0:1776178471.788033:0:3352:0:(import.c:86:import_set_state_nolock()) 00000000dfc8db13 lustre-MDT0000_UUID: changing import state from CONNECTING to DISCONN 00000100:00080000:1.0:1776178471.788038:0:3352:0:(import.c:1429:ptlrpc_connect_interpret()) recovery of lustre-MDT0000_UUID on 192.168.204.103@tcp failed (-19) 00000100:00100000:1.0:1776178471.788042:0:3352:0:(client.c:2215:ptlrpc_check_set()) Completed RPC req@00000000d2194868 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:lustre-MDT0000-lwp-OST0001_UUID:3352:1862456998086336:0@lo:38:kworker/u8:1.0 00000100:00080000:1.0:1776178471.788089:0:3352:0:(import.c:86:import_set_state_nolock()) 00000000ba4f1821 lustre-MDT0000_UUID: changing import state from CONNECTING to DISCONN 00000100:00080000:1.0:1776178471.788093:0:3352:0:(import.c:1429:ptlrpc_connect_interpret()) recovery of lustre-MDT0000_UUID on 192.168.204.103@tcp failed (-19) 00000100:00100000:1.0:1776178471.788095:0:3352:0:(client.c:2215:ptlrpc_check_set()) Completed RPC req@000000006466ee18 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:lustre-MDT0001-mdtlov_UUID:3352:1862456998086464:0@lo:38:kworker/u8:1.0 00000100:00100000:0.0:1776178471.788171:0:11281:0:(service.c:2152:ptlrpc_server_handle_req_in()) got req x1862456998086656 00000100:00100000:0.0:1776178471.788207:0:11281:0:(nrs_fifo.c:179:nrs_fifo_req_get()) NRS start fifo request from 12345-0@lo, seq: 6835 00000100:00100000:0.0:1776178471.788212:0:11281:0:(service.c:2324:ptlrpc_server_handle_request()) Handling RPC req@00000000dd4b6b2f pname:cluuid+ref:pid:xid:nid:opc:job mdt00_004:0+-99:3352:x1862456998086656:12345-0@lo:38:kworker/u8:1.0 00010000:02000000:0.0:1776178471.788224:0:11281:0:(ldlm_lib.c:1139:target_handle_connect()) lustre-MDT0000: Not available for connect from 0@lo (stopping) 00000100:00100000:0.0:1776178471.788268:0:11281:0:(service.c:2372:ptlrpc_server_handle_request()) Handled RPC req@00000000dd4b6b2f pname:cluuid+ref:pid:xid:nid:opc:job mdt00_004:0+-99:3352:x1862456998086656:12345-0@lo:38:kworker/u8:1.0 Request processed in 55us (286us total) trans 0 rc -19/-19 00000100:00100000:0.0:1776178471.788274:0:11281:0:(nrs_fifo.c:241:nrs_fifo_req_stop()) NRS stop fifo request from 12345-0@lo, seq: 6835 00000100:00080000:0.0:1776178471.788298:0:3352:0:(import.c:86:import_set_state_nolock()) 00000000a7c55b1f lustre-MDT0000_UUID: changing import state from CONNECTING to DISCONN 00000100:00080000:0.0:1776178471.788301:0:3352:0:(import.c:1429:ptlrpc_connect_interpret()) recovery of lustre-MDT0000_UUID on 192.168.204.103@tcp failed (-19) 00000100:00100000:0.0:1776178471.788304:0:3352:0:(client.c:2215:ptlrpc_check_set()) Completed RPC req@000000001f977915 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:lustre-MDT0000-lwp-MDT0001_UUID:3352:1862456998086656:0@lo:38:kworker/u8:1.0 00010000:00010000:2.0:1776178472.417945:0:6245:0:(ldlm_lockd.c:2792:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:2.0:1776178473.312958:0:3353:0:(ptlrpcd.c:415:ptlrpcd_check()) transfer 1 async RPCs [1->0] 00000100:00100000:2.0:1776178473.312987:0:3353:0:(client.c:1733:ptlrpc_send_new_req()) Sending RPC req@00000000d0706d7c pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_00:lustre-MDT0001-mdtlov_UUID:3353:1862456998086720:0@lo:13:osp-pre-1-1.0 00000100:00100000:2.0:1776178473.313103:0:3353:0:(events.c:361:request_in_callback()) peer: 12345-0@lo (source: 12345-0@lo) 00000100:00100000:0.0:1776178473.313444:0:9554:0:(service.c:2152:ptlrpc_server_handle_req_in()) got req x1862456998086720 00000100:00100000:0.0:1776178473.313509:0:9554:0:(nrs_fifo.c:179:nrs_fifo_req_get()) NRS start fifo request from 12345-0@lo, seq: 593 00000100:00100000:0.0:1776178473.313518:0:9554:0:(service.c:2324:ptlrpc_server_handle_request()) Handling RPC req@0000000046115d91 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_create00:lustre-MDT0001-mdtlov_UUID+5:3353:x1862456998086720:12345-0@lo:13:osp-pre-1-1.0 00002000:00000024:0.0:1776178473.313557:0:9554:0:(ofd_obd.c:740:ofd_statfs()) blocks cached 229376 granted 491746368 pending 0 free 3923689472 avail 3692167168 00000100:00100000:0.0:1776178473.313620:0:9554:0:(service.c:2372:ptlrpc_server_handle_request()) Handled RPC req@0000000046115d91 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_create00:lustre-MDT0001-mdtlov_UUID+5:3353:x1862456998086720:12345-0@lo:13:osp-pre-1-1.0 Request processed in 103us (518us total) trans 0 rc 0/0 00000100:00100000:0.0:1776178473.313628:0:9554:0:(nrs_fifo.c:241:nrs_fifo_req_stop()) NRS stop fifo request from 12345-0@lo, seq: 593 00000100:00100000:2.0:1776178473.313821:0:3353:0:(client.c:2215:ptlrpc_check_set()) Completed RPC req@00000000d0706d7c pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_00:lustre-MDT0001-mdtlov_UUID:3353:1862456998086720:0@lo:13:osp-pre-1-1.0 00010000:00010000:1.0:1776178473.443650:0:9224:0:(ldlm_lockd.c:2792:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:2.0:1776178473.824691:0:3356:0:(ptlrpcd.c:415:ptlrpcd_check()) transfer 1 async RPCs [2->3] 00000100:00100000:2.0:1776178473.824737:0:3356:0:(client.c:1733:ptlrpc_send_new_req()) Sending RPC req@00000000d0d837c5 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_03:lustre-MDT0001-mdtlov_UUID:3356:1862456998086784:0@lo:13:osp-pre-0-1.0 00000100:00100000:2.0:1776178473.824832:0:3356:0:(events.c:361:request_in_callback()) peer: 12345-0@lo (source: 12345-0@lo) 00000100:00100000:3.0:1776178473.825188:0:8231:0:(service.c:2152:ptlrpc_server_handle_req_in()) got req x1862456998086784 00000100:00100000:3.0:1776178473.825243:0:8231:0:(nrs_fifo.c:179:nrs_fifo_req_get()) NRS start fifo request from 12345-0@lo, seq: 594 00000100:00100000:3.0:1776178473.825249:0:8231:0:(service.c:2324:ptlrpc_server_handle_request()) Handling RPC req@000000008c593d89 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_create00:lustre-MDT0001-mdtlov_UUID+5:3356:x1862456998086784:12345-0@lo:13:osp-pre-0-1.0 00002000:00000024:3.0:1776178473.825283:0:8231:0:(ofd_obd.c:740:ofd_statfs()) blocks cached 315392 granted 484883776 pending 0 free 3923689472 avail 3692167168 00000100:00100000:3.0:1776178473.825338:0:8231:0:(service.c:2372:ptlrpc_server_handle_request()) Handled RPC req@000000008c593d89 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_create00:lustre-MDT0001-mdtlov_UUID+5:3356:x1862456998086784:12345-0@lo:13:osp-pre-0-1.0 Request processed in 88us (508us total) trans 0 rc 0/0 00000100:00100000:3.0:1776178473.825353:0:8231:0:(nrs_fifo.c:241:nrs_fifo_req_stop()) NRS stop fifo request from 12345-0@lo, seq: 594 00000100:00100000:2.0:1776178473.825710:0:3356:0:(client.c:2215:ptlrpc_check_set()) Completed RPC req@00000000d0d837c5 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_03:lustre-MDT0001-mdtlov_UUID:3356:1862456998086784:0@lo:13:osp-pre-0-1.0 00010000:00010000:1.0:1776178474.465464:0:9225:0:(ldlm_lockd.c:2792:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:3.0:1776178475.297873:0:3348:0:(events.c:361:request_in_callback()) peer: 12345-192.168.204.3@tcp (source: 12345-192.168.204.3@tcp) 00000100:00100000:3.0:1776178475.297927:0:3348:0:(events.c:361:request_in_callback()) peer: 12345-192.168.204.3@tcp (source: 12345-192.168.204.3@tcp) 00000100:00100000:0.0:1776178475.297974:0:20140:0:(service.c:2152:ptlrpc_server_handle_req_in()) got req x1862456979125952 00000100:00100000:0.0:1776178475.298048:0:20140:0:(nrs_fifo.c:179:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.204.3@tcp, seq: 261 00000100:00100000:0.0:1776178475.298055:0:20140:0:(service.c:2324:ptlrpc_server_handle_request()) Handling RPC req@00000000c0367ec3 pname:cluuid+ref:pid:xid:nid:opc:job ll_mgs_0002:5b5c66b1-1a2f-4651-8548-0c75b9a38a94+9:2272:x1862456979125952:12345-192.168.204.3@tcp:400:kworker/u8:3.0 00000100:00100000:3.0:1776178475.298146:0:3348:0:(events.c:361:request_in_callback()) peer: 12345-192.168.204.3@tcp (source: 12345-192.168.204.3@tcp) 00000100:00100000:3.0:1776178475.298175:0:3348:0:(events.c:361:request_in_callback()) peer: 12345-192.168.204.3@tcp (source: 12345-192.168.204.3@tcp) 00000100:00100000:3.0:1776178475.298210:0:20762:0:(service.c:2152:ptlrpc_server_handle_req_in()) got req x1862456979126144 00000100:00100000:2.0:1776178475.298221:0:14440:0:(service.c:2152:ptlrpc_server_handle_req_in()) got req x1862456979126208 00000100:00100000:2.0:1776178475.298259:0:14440:0:(nrs_fifo.c:179:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.204.3@tcp, seq: 723 00000100:00100000:2.0:1776178475.298263:0:14440:0:(service.c:2324:ptlrpc_server_handle_request()) Handling RPC req@0000000038c09127 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_004:b50963bd-8eba-4432-9a54-8888d00e3fa1+9:2272:x1862456979126208:12345-192.168.204.3@tcp:400:kworker/u8:3.0 00000100:00100000:3.0:1776178475.298323:0:20762:0:(nrs_fifo.c:179:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.204.3@tcp, seq: 724 00000100:00100000:3.0:1776178475.298329:0:20762:0:(service.c:2324:ptlrpc_server_handle_request()) Handling RPC req@00000000d0706d7c pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_013:b50963bd-8eba-4432-9a54-8888d00e3fa1+8:2272:x1862456979126144:12345-192.168.204.3@tcp:400:kworker/u8:3.0 00000100:00100000:2.0:1776178475.298337:0:14440:0:(service.c:2372:ptlrpc_server_handle_request()) Handled RPC req@0000000038c09127 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_004:b50963bd-8eba-4432-9a54-8888d00e3fa1+9:2272:x1862456979126208:12345-192.168.204.3@tcp:400:kworker/u8:3.0 Request processed in 71us (160us total) trans 0 rc 0/0 00000100:00100000:2.0:1776178475.298346:0:14440:0:(nrs_fifo.c:241:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.204.3@tcp, seq: 723 00000100:00100000:3.0:1776178475.298365:0:20762:0:(service.c:2372:ptlrpc_server_handle_request()) Handled RPC req@00000000d0706d7c pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_013:b50963bd-8eba-4432-9a54-8888d00e3fa1+8:2272:x1862456979126144:12345-192.168.204.3@tcp:400:kworker/u8:3.0 Request processed in 33us (436us total) trans 0 rc 0/0 00000100:00100000:3.0:1776178475.298373:0:20762:0:(nrs_fifo.c:241:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.204.3@tcp, seq: 724 00000100:00100000:3.0:1776178475.298418:0:6256:0:(service.c:2152:ptlrpc_server_handle_req_in()) got req x1862456979126080 00000100:00100000:3.0:1776178475.298820:0:6256:0:(nrs_fifo.c:179:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.204.3@tcp, seq: 1087 00000100:00100000:2.0:1776178475.299481:0:3347:0:(events.c:361:request_in_callback()) peer: 12345-192.168.204.3@tcp (source: 12345-192.168.204.3@tcp) 00000100:00100000:2.0:1776178475.300506:0:8250:0:(service.c:2152:ptlrpc_server_handle_req_in()) got req x1862456979126016 00000100:00100000:0.0:1776178475.300590:0:20140:0:(service.c:2372:ptlrpc_server_handle_request()) Handled RPC req@00000000c0367ec3 pname:cluuid+ref:pid:xid:nid:opc:job ll_mgs_0002:5b5c66b1-1a2f-4651-8548-0c75b9a38a94+9:2272:x1862456979125952:12345-192.168.204.3@tcp:400:kworker/u8:3.0 Request processed in 843us (1036us total) trans 0 rc 0/0 00000100:00100000:0.0:1776178475.300603:0:20140:0:(nrs_fifo.c:241:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.204.3@tcp, seq: 261 00000100:00100000:3.0:1776178475.301341:0:6256:0:(service.c:2324:ptlrpc_server_handle_request()) Handling RPC req@00000000b8464782 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_001:b50963bd-8eba-4432-9a54-8888d00e3fa1+6:2272:x1862456979126080:12345-192.168.204.3@tcp:400:kworker/u8:3.0 00000100:00100000:3.0:1776178475.301989:0:6256:0:(service.c:2372:ptlrpc_server_handle_request()) Handled RPC req@00000000b8464782 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_001:b50963bd-8eba-4432-9a54-8888d00e3fa1+6:2272:x1862456979126080:12345-192.168.204.3@tcp:400:kworker/u8:3.0 Request processed in 3158us (3840us total) trans 0 rc 0/0 00000100:00100000:3.0:1776178475.302005:0:6256:0:(nrs_fifo.c:241:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.204.3@tcp, seq: 1087 00000100:00100000:2.0:1776178475.311617:0:8250:0:(nrs_fifo.c:179:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.204.3@tcp, seq: 6836 00000100:00100000:2.0:1776178475.311633:0:8250:0:(service.c:2324:ptlrpc_server_handle_request()) Handling RPC req@00000000a604eea4 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_003:0+-99:2269:x1862456979126016:12345-192.168.204.3@tcp:38:kworker/u8:3.0 00010000:02000000:2.0:1776178475.311657:0:8250:0:(ldlm_lib.c:1139:target_handle_connect()) lustre-MDT0000: Not available for connect from 192.168.204.3@tcp (stopping) 00000100:00100000:2.0:1776178475.311774:0:8250:0:(service.c:2372:ptlrpc_server_handle_request()) Handled RPC req@00000000a604eea4 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_003:0+-99:2269:x1862456979126016:12345-192.168.204.3@tcp:38:kworker/u8:3.0 Request processed in 141us (13322us total) trans 0 rc -19/-19 00000100:00100000:2.0:1776178475.311782:0:8250:0:(nrs_fifo.c:241:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.204.3@tcp, seq: 6836 00010000:00010000:3.0:1776178475.490882:0:6244:0:(ldlm_lockd.c:2792:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00010000:00010000:2.0:1776178476.512427:0:6245:0:(ldlm_lockd.c:2792:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00080000:0.0:1776178476.898434:0:36:0:(pinger.c:251:ptlrpc_pinger_process_import()) lustre-MDT0000-lwp-OST0000_UUID->lustre-MDT0000_UUID: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00080000:0.0:1776178476.898445:0:36:0:(recover.c:57:ptlrpc_initiate_recovery()) lustre-MDT0000_UUID: starting recovery 00000100:00080000:0.0:1776178476.898449:0:36:0:(import.c:86:import_set_state_nolock()) 0000000047601a98 lustre-MDT0000_UUID: changing import state from DISCONN to CONNECTING 00000100:00080000:0.0:1776178476.898459:0:36:0:(import.c:534:import_select_connection()) lustre-MDT0000-lwp-OST0000: connect to NID 0@lo last attempt 1179 00000100:00080000:0.0:1776178476.898463:0:36:0:(import.c:581:import_select_connection()) lustre-MDT0000-lwp-OST0000: tried all connections, increasing latency to 21s 00000100:00100000:2.0:1776178476.898621:0:3356:0:(client.c:1733:ptlrpc_send_new_req()) Sending RPC req@00000000e8195050 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_03:ac61f839-0e8e-4cac-9c59-08bcfbbad330:3356:1862456998086848:0@lo:400:kworker/u8:1.0 00000100:00100000:2.0:1776178476.898731:0:3356:0:(events.c:361:request_in_callback()) peer: 12345-0@lo (source: 12345-0@lo) 00000100:00100000:3.0:1776178476.898848:0:20140:0:(service.c:2152:ptlrpc_server_handle_req_in()) got req x1862456998086848 00000100:00100000:3.0:1776178476.898908:0:20140:0:(nrs_fifo.c:179:nrs_fifo_req_get()) NRS start fifo request from 12345-0@lo, seq: 262 00000100:00100000:3.0:1776178476.898916:0:20140:0:(service.c:2324:ptlrpc_server_handle_request()) Handling RPC req@00000000ecf3e0a2 pname:cluuid+ref:pid:xid:nid:opc:job ll_mgs_0002:ac61f839-0e8e-4cac-9c59-08bcfbbad330+18:3356:x1862456998086848:12345-0@lo:400:kworker/u8:1.0 00000100:00100000:3.0:1776178476.899012:0:20140:0:(service.c:2372:ptlrpc_server_handle_request()) Handled RPC req@00000000ecf3e0a2 pname:cluuid+ref:pid:xid:nid:opc:job ll_mgs_0002:ac61f839-0e8e-4cac-9c59-08bcfbbad330+18:3356:x1862456998086848:12345-0@lo:400:kworker/u8:1.0 Request processed in 95us (282us total) trans 0 rc 0/0 00000100:00100000:3.0:1776178476.899020:0:20140:0:(nrs_fifo.c:241:nrs_fifo_req_stop()) NRS stop fifo request from 12345-0@lo, seq: 262 00000100:00100000:2.0:1776178476.899103:0:3356:0:(client.c:2215:ptlrpc_check_set()) Completed RPC req@00000000e8195050 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_03:ac61f839-0e8e-4cac-9c59-08bcfbbad330:3356:1862456998086848:0@lo:400:kworker/u8:1.0 00000100:00080000:0.0:1776178476.899169:0:36:0:(import.c:615:import_select_connection()) lustre-MDT0000-lwp-OST0000: import 0000000047601a98 using connection 192.168.204.103@tcp/0@lo 00000100:00100000:0.0:1776178476.899198:0:36:0:(import.c:807:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5) req@0000000037aace32 x1862456998086912/t0(0) o38->lustre-MDT0000-lwp-OST0000@0@lo:12/10 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' 00000100:00100000:1.0:1776178476.899363:0:3352:0:(client.c:1733:ptlrpc_send_new_req()) Sending RPC req@0000000037aace32 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:lustre-MDT0000-lwp-OST0000_UUID:3352:1862456998086912:0@lo:38:kworker/u8:1.0 00000100:00100000:1.0:1776178476.899405:0:3352:0:(events.c:361:request_in_callback()) peer: 12345-0@lo (source: 12345-0@lo) 00000100:00100000:2.0:1776178476.899496:0:8250:0:(service.c:2152:ptlrpc_server_handle_req_in()) got req x1862456998086912 00000100:00100000:2.0:1776178476.899545:0:8250:0:(nrs_fifo.c:179:nrs_fifo_req_get()) NRS start fifo request from 12345-0@lo, seq: 6837 00000100:00100000:2.0:1776178476.899550:0:8250:0:(service.c:2324:ptlrpc_server_handle_request()) Handling RPC req@0000000047c15ac0 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_003:0+-99:3352:x1862456998086912:12345-0@lo:38:kworker/u8:1.0 00010000:02000000:2.0:1776178476.899568:0:8250:0:(ldlm_lib.c:1139:target_handle_connect()) lustre-MDT0000: Not available for connect from 0@lo (stopping) 00000100:00100000:2.0:1776178476.899611:0:8250:0:(service.c:2372:ptlrpc_server_handle_request()) Handled RPC req@0000000047c15ac0 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_003:0+-99:3352:x1862456998086912:12345-0@lo:38:kworker/u8:1.0 Request processed in 60us (205us total) trans 0 rc -19/-19 00000100:00100000:2.0:1776178476.899618:0:8250:0:(nrs_fifo.c:241:nrs_fifo_req_stop()) NRS stop fifo request from 12345-0@lo, seq: 6837 00000100:00080000:1.0:1776178476.899779:0:3352:0:(import.c:86:import_set_state_nolock()) 0000000047601a98 lustre-MDT0000_UUID: changing import state from CONNECTING to DISCONN 00000100:00080000:1.0:1776178476.899783:0:3352:0:(import.c:1429:ptlrpc_connect_interpret()) recovery of lustre-MDT0000_UUID on 192.168.204.103@tcp failed (-19) 00000100:00100000:1.0:1776178476.899787:0:3352:0:(client.c:2215:ptlrpc_check_set()) Completed RPC req@0000000037aace32 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:lustre-MDT0000-lwp-OST0000_UUID:3352:1862456998086912:0@lo:38:kworker/u8:1.0 00000100:00080000:0.0:1776178476.900191:0:36:0:(pinger.c:251:ptlrpc_pinger_process_import()) lustre-MDT0000-lwp-OST0001_UUID->lustre-MDT0000_UUID: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00080000:0.0:1776178476.900199:0:36:0:(recover.c:57:ptlrpc_initiate_recovery()) lustre-MDT0000_UUID: starting recovery 00000100:00080000:0.0:1776178476.900202:0:36:0:(import.c:86:import_set_state_nolock()) 00000000dfc8db13 lustre-MDT0000_UUID: changing import state from DISCONN to CONNECTING 00000100:00080000:0.0:1776178476.900207:0:36:0:(import.c:534:import_select_connection()) lustre-MDT0000-lwp-OST0001: connect to NID 0@lo last attempt 1179 00000100:00080000:0.0:1776178476.900213:0:36:0:(import.c:581:import_select_connection()) lustre-MDT0000-lwp-OST0001: tried all connections, increasing latency to 21s 00000100:00080000:0.0:1776178476.900218:0:36:0:(import.c:615:import_select_connection()) lustre-MDT0000-lwp-OST0001: import 00000000dfc8db13 using connection 192.168.204.103@tcp/0@lo 00000100:00100000:1.0:1776178476.900352:0:3354:0:(ptlrpcd.c:415:ptlrpcd_check()) transfer 1 async RPCs [0->1] 00000100:00100000:1.0:1776178476.900367:0:3354:0:(client.c:1733:ptlrpc_send_new_req()) Sending RPC req@00000000e8195050 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_01:lustre-MDT0001-lwp-OST0000_UUID:3354:1862456998086976:0@lo:400:kworker/u8:1.0 00000100:00100000:1.0:1776178476.900412:0:3354:0:(events.c:361:request_in_callback()) peer: 12345-0@lo (source: 12345-0@lo) 00000100:00100000:3.0:1776178476.900513:0:6256:0:(service.c:2152:ptlrpc_server_handle_req_in()) got req x1862456998086976 00000100:00100000:3.0:1776178476.900562:0:6256:0:(nrs_fifo.c:179:nrs_fifo_req_get()) NRS start fifo request from 12345-0@lo, seq: 1088 00000100:00100000:3.0:1776178476.900566:0:6256:0:(service.c:2324:ptlrpc_server_handle_request()) Handling RPC req@000000001f977915 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_001:lustre-MDT0001-lwp-OST0000_UUID+5:3354:x1862456998086976:12345-0@lo:400:kworker/u8:1.0 00000100:00100000:3.0:1776178476.900624:0:6256:0:(service.c:2372:ptlrpc_server_handle_request()) Handled RPC req@000000001f977915 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_001:lustre-MDT0001-lwp-OST0000_UUID+5:3354:x1862456998086976:12345-0@lo:400:kworker/u8:1.0 Request processed in 58us (214us total) trans 0 rc 0/0 00000100:00100000:3.0:1776178476.900630:0:6256:0:(nrs_fifo.c:241:nrs_fifo_req_stop()) NRS stop fifo request from 12345-0@lo, seq: 1088 00000100:00100000:1.0:1776178476.900890:0:3354:0:(client.c:2215:ptlrpc_check_set()) Completed RPC req@00000000e8195050 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_01:lustre-MDT0001-lwp-OST0000_UUID:3354:1862456998086976:0@lo:400:kworker/u8:1.0 00000100:00100000:0.0:1776178476.900970:0:36:0:(import.c:807:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5) req@00000000d0d837c5 x1862456998087040/t0(0) o38->lustre-MDT0000-lwp-OST0001@0@lo:12/10 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' 00000100:00100000:1.0:1776178476.901022:0:3352:0:(client.c:1733:ptlrpc_send_new_req()) Sending RPC req@00000000d0d837c5 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:lustre-MDT0000-lwp-OST0001_UUID:3352:1862456998087040:0@lo:38:kworker/u8:1.0 00000100:00100000:1.0:1776178476.901053:0:3352:0:(events.c:361:request_in_callback()) peer: 12345-0@lo (source: 12345-0@lo) 00000100:00100000:3.0:1776178476.901135:0:8250:0:(service.c:2152:ptlrpc_server_handle_req_in()) got req x1862456998087040 00000100:00100000:3.0:1776178476.901159:0:8250:0:(nrs_fifo.c:179:nrs_fifo_req_get()) NRS start fifo request from 12345-0@lo, seq: 6838 00000100:00100000:3.0:1776178476.901163:0:8250:0:(service.c:2324:ptlrpc_server_handle_request()) Handling RPC req@000000006466ee18 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_003:0+-99:3352:x1862456998087040:12345-0@lo:38:kworker/u8:1.0 00010000:02000000:3.0:1776178476.901177:0:8250:0:(ldlm_lib.c:1139:target_handle_connect()) lustre-MDT0000: Not available for connect from 0@lo (stopping) 00000100:00080000:0.0:1776178476.901267:0:36:0:(pinger.c:251:ptlrpc_pinger_process_import()) lustre-MDT0001-mdtlov_UUID->lustre-MDT0000_UUID: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00080000:0.0:1776178476.901271:0:36:0:(recover.c:57:ptlrpc_initiate_recovery()) lustre-MDT0000_UUID: starting recovery 00000100:00080000:0.0:1776178476.901274:0:36:0:(import.c:86:import_set_state_nolock()) 00000000ba4f1821 lustre-MDT0000_UUID: changing import state from DISCONN to CONNECTING 00000100:00080000:0.0:1776178476.901279:0:36:0:(import.c:534:import_select_connection()) lustre-MDT0000-osp-MDT0001: connect to NID 0@lo last attempt 1179 00000100:00080000:0.0:1776178476.901281:0:36:0:(import.c:581:import_select_connection()) lustre-MDT0000-osp-MDT0001: tried all connections, increasing latency to 21s 00000100:00080000:0.0:1776178476.901286:0:36:0:(import.c:615:import_select_connection()) lustre-MDT0000-osp-MDT0001: import 00000000ba4f1821 using connection 192.168.204.103@tcp/0@lo 00000100:00100000:0.0:1776178476.901300:0:36:0:(import.c:807:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5) req@0000000004dd3b9b x1862456998087168/t0(0) o38->lustre-MDT0000-osp-MDT0001@0@lo:24/4 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' 00000100:00100000:2.0:1776178476.901359:0:3353:0:(ptlrpcd.c:415:ptlrpcd_check()) transfer 1 async RPCs [1->0] 00000100:00100000:2.0:1776178476.901369:0:3353:0:(client.c:1733:ptlrpc_send_new_req()) Sending RPC req@000000006a3d91b0 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_00:lustre-MDT0001-lwp-OST0001_UUID:3353:1862456998087104:0@lo:400:kworker/u8:1.0 00000100:00100000:2.0:1776178476.901399:0:3353:0:(events.c:361:request_in_callback()) peer: 12345-0@lo (source: 12345-0@lo) 00000100:00100000:2.0:1776178476.901437:0:6256:0:(service.c:2152:ptlrpc_server_handle_req_in()) got req x1862456998087104 00000100:00100000:2.0:1776178476.901463:0:6256:0:(nrs_fifo.c:179:nrs_fifo_req_get()) NRS start fifo request from 12345-0@lo, seq: 1089 00000100:00100000:2.0:1776178476.901466:0:6256:0:(service.c:2324:ptlrpc_server_handle_request()) Handling RPC req@00000000a92aab25 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_001:lustre-MDT0001-lwp-OST0001_UUID+5:3353:x1862456998087104:12345-0@lo:400:kworker/u8:1.0 00000100:00100000:2.0:1776178476.901522:0:6256:0:(service.c:2372:ptlrpc_server_handle_request()) Handled RPC req@00000000a92aab25 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_001:lustre-MDT0001-lwp-OST0001_UUID+5:3353:x1862456998087104:12345-0@lo:400:kworker/u8:1.0 Request processed in 55us (122us total) trans 0 rc 0/0 00000100:00100000:2.0:1776178476.901528:0:6256:0:(nrs_fifo.c:241:nrs_fifo_req_stop()) NRS stop fifo request from 12345-0@lo, seq: 1089 00000100:00100000:2.0:1776178476.901554:0:3353:0:(client.c:2215:ptlrpc_check_set()) Completed RPC req@000000006a3d91b0 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_00:lustre-MDT0001-lwp-OST0001_UUID:3353:1862456998087104:0@lo:400:kworker/u8:1.0 00000100:00100000:1.0:1776178476.916411:0:3352:0:(client.c:1733:ptlrpc_send_new_req()) Sending RPC req@0000000004dd3b9b pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:lustre-MDT0001-mdtlov_UUID:3352:1862456998087168:0@lo:38:kworker/u8:1.0 00000100:00100000:1.0:1776178476.916553:0:3352:0:(events.c:361:request_in_callback()) peer: 12345-0@lo (source: 12345-0@lo) 00000100:00100000:2.0:1776178476.916722:0:15213:0:(service.c:2152:ptlrpc_server_handle_req_in()) got req x1862456998087168 00000100:00100000:2.0:1776178476.916794:0:15213:0:(nrs_fifo.c:179:nrs_fifo_req_get()) NRS start fifo request from 12345-0@lo, seq: 761 00000100:00100000:2.0:1776178476.916801:0:15213:0:(service.c:2324:ptlrpc_server_handle_request()) Handling RPC req@00000000d2194868 pname:cluuid+ref:pid:xid:nid:opc:job mdt_out00_003:0+-99:3352:x1862456998087168:12345-0@lo:38:kworker/u8:1.0 00010000:02000000:2.0:1776178476.916819:0:15213:0:(ldlm_lib.c:1139:target_handle_connect()) lustre-MDT0000: Not available for connect from 0@lo (stopping) 00000100:00100000:2.0:1776178476.916868:0:15213:0:(service.c:2372:ptlrpc_server_handle_request()) Handled RPC req@00000000d2194868 pname:cluuid+ref:pid:xid:nid:opc:job mdt_out00_003:0+-99:3352:x1862456998087168:12345-0@lo:38:kworker/u8:1.0 Request processed in 67us (318us total) trans 0 rc -19/-19 00000100:00100000:2.0:1776178476.916879:0:15213:0:(nrs_fifo.c:241:nrs_fifo_req_stop()) NRS stop fifo request from 12345-0@lo, seq: 761 00000100:00100000:3.0:1776178476.922362:0:8250:0:(service.c:2372:ptlrpc_server_handle_request()) Handled RPC req@000000006466ee18 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_003:0+-99:3352:x1862456998087040:12345-0@lo:38:kworker/u8:1.0 Request processed in 21185us (21296us total) trans 0 rc -19/-19 00000100:00100000:3.0:1776178476.922382:0:8250:0:(nrs_fifo.c:241:nrs_fifo_req_stop()) NRS stop fifo request from 12345-0@lo, seq: 6838 00000100:00080000:0.0:1776178476.937760:0:36:0:(pinger.c:251:ptlrpc_pinger_process_import()) lustre-MDT0000-lwp-MDT0001_UUID->lustre-MDT0000_UUID: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00080000:0.0:1776178476.937769:0:36:0:(recover.c:57:ptlrpc_initiate_recovery()) lustre-MDT0000_UUID: starting recovery 00000100:00080000:0.0:1776178476.937773:0:36:0:(import.c:86:import_set_state_nolock()) 00000000a7c55b1f lustre-MDT0000_UUID: changing import state from DISCONN to CONNECTING 00000100:00080000:0.0:1776178476.937783:0:36:0:(import.c:534:import_select_connection()) lustre-MDT0000-lwp-MDT0001: connect to NID 0@lo last attempt 1179 00000100:00080000:0.0:1776178476.937787:0:36:0:(import.c:581:import_select_connection()) lustre-MDT0000-lwp-MDT0001: tried all connections, increasing latency to 21s 00000100:00080000:0.0:1776178476.937795:0:36:0:(import.c:615:import_select_connection()) lustre-MDT0000-lwp-MDT0001: import 00000000a7c55b1f using connection 192.168.204.103@tcp/0@lo 00000100:00100000:0.0:1776178476.937816:0:36:0:(import.c:807:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5) req@000000006a3d91b0 x1862456998087360/t0(0) o38->lustre-MDT0000-lwp-MDT0001@0@lo:12/10 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' 00000100:00100000:2.0:1776178476.937959:0:3355:0:(client.c:1733:ptlrpc_send_new_req()) Sending RPC req@000000002300ff10 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_02:lustre-MDT0001-mdtlov_UUID:3355:1862456998087232:0@lo:400:kworker/u8:1.0 00000100:00100000:2.0:1776178476.938041:0:3355:0:(events.c:361:request_in_callback()) peer: 12345-0@lo (source: 12345-0@lo) 00000100:00100000:2.0:1776178476.938113:0:3355:0:(ptlrpcd.c:415:ptlrpcd_check()) transfer 1 async RPCs [3->2] 00000100:00100000:2.0:1776178476.938118:0:3355:0:(client.c:1733:ptlrpc_send_new_req()) Sending RPC req@000000007216887e pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_02:lustre-MDT0001-mdtlov_UUID:3355:1862456998087296:0@lo:400:kworker/u8:1.0 00000100:00100000:2.0:1776178476.938139:0:3355:0:(events.c:361:request_in_callback()) peer: 12345-0@lo (source: 12345-0@lo) 00000100:00080000:1.0:1776178476.938760:0:3352:0:(import.c:86:import_set_state_nolock()) 00000000dfc8db13 lustre-MDT0000_UUID: changing import state from CONNECTING to DISCONN 00000100:00080000:1.0:1776178476.938767:0:3352:0:(import.c:1429:ptlrpc_connect_interpret()) recovery of lustre-MDT0000_UUID on 192.168.204.103@tcp failed (-19) 00000100:00100000:1.0:1776178476.938772:0:3352:0:(client.c:2215:ptlrpc_check_set()) Completed RPC req@00000000d0d837c5 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:lustre-MDT0000-lwp-OST0001_UUID:3352:1862456998087040:0@lo:38:kworker/u8:1.0 00000100:00100000:1.0:1776178476.938794:0:3352:0:(client.c:1733:ptlrpc_send_new_req()) Sending RPC req@000000006a3d91b0 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:lustre-MDT0000-lwp-MDT0001_UUID:3352:1862456998087360:0@lo:38:kworker/u8:1.0 00000100:00100000:1.0:1776178476.938825:0:3352:0:(events.c:361:request_in_callback()) peer: 12345-0@lo (source: 12345-0@lo) 00000100:00080000:1.0:1776178476.938854:0:3352:0:(import.c:86:import_set_state_nolock()) 00000000ba4f1821 lustre-MDT0000_UUID: changing import state from CONNECTING to DISCONN 00000100:00080000:1.0:1776178476.938858:0:3352:0:(import.c:1429:ptlrpc_connect_interpret()) recovery of lustre-MDT0000_UUID on 192.168.204.103@tcp failed (-19) 00000100:00100000:1.0:1776178476.938861:0:3352:0:(client.c:2215:ptlrpc_check_set()) Completed RPC req@0000000004dd3b9b pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:lustre-MDT0001-mdtlov_UUID:3352:1862456998087168:0@lo:38:kworker/u8:1.0 00000100:00100000:1.0:1776178476.938894:0:8250:0:(service.c:2152:ptlrpc_server_handle_req_in()) got req x1862456998087360 00000100:00100000:1.0:1776178476.938934:0:8250:0:(nrs_fifo.c:179:nrs_fifo_req_get()) NRS start fifo request from 12345-0@lo, seq: 6839 00000100:00100000:1.0:1776178476.938940:0:8250:0:(service.c:2324:ptlrpc_server_handle_request()) Handling RPC req@00000000cb1b73cb pname:cluuid+ref:pid:xid:nid:opc:job mdt00_003:0+-99:3352:x1862456998087360:12345-0@lo:38:kworker/u8:1.0 00010000:02000000:1.0:1776178476.938955:0:8250:0:(ldlm_lib.c:1139:target_handle_connect()) lustre-MDT0000: Not available for connect from 0@lo (stopping) 00000100:00100000:1.0:1776178476.938997:0:8250:0:(service.c:2372:ptlrpc_server_handle_request()) Handled RPC req@00000000cb1b73cb pname:cluuid+ref:pid:xid:nid:opc:job mdt00_003:0+-99:3352:x1862456998087360:12345-0@lo:38:kworker/u8:1.0 Request processed in 56us (171us total) trans 0 rc -19/-19 00000100:00100000:1.0:1776178476.939004:0:8250:0:(nrs_fifo.c:241:nrs_fifo_req_stop()) NRS stop fifo request from 12345-0@lo, seq: 6839 00000100:00080000:1.0:1776178476.939031:0:3352:0:(import.c:86:import_set_state_nolock()) 00000000a7c55b1f lustre-MDT0000_UUID: changing import state from CONNECTING to DISCONN 00000100:00080000:1.0:1776178476.939034:0:3352:0:(import.c:1429:ptlrpc_connect_interpret()) recovery of lustre-MDT0000_UUID on 192.168.204.103@tcp failed (-19) 00000100:00100000:1.0:1776178476.939037:0:3352:0:(client.c:2215:ptlrpc_check_set()) Completed RPC req@000000006a3d91b0 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:lustre-MDT0000-lwp-MDT0001_UUID:3352:1862456998087360:0@lo:38:kworker/u8:1.0 00000100:00100000:0.0:1776178476.939316:0:20759:0:(service.c:2152:ptlrpc_server_handle_req_in()) got req x1862456998087232 00000100:00100000:0.0:1776178476.939368:0:20759:0:(service.c:2152:ptlrpc_server_handle_req_in()) got req x1862456998087296 00000100:00100000:0.0:1776178476.939397:0:20759:0:(nrs_fifo.c:179:nrs_fifo_req_get()) NRS start fifo request from 12345-0@lo, seq: 725 00000100:00100000:0.0:1776178476.939401:0:20759:0:(service.c:2324:ptlrpc_server_handle_request()) Handling RPC req@00000000eafab700 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_010:lustre-MDT0001-mdtlov_UUID+5:3355:x1862456998087232:12345-0@lo:400:kworker/u8:1.0 00000100:00100000:0.0:1776178476.939458:0:20759:0:(service.c:2372:ptlrpc_server_handle_request()) Handled RPC req@00000000eafab700 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_010:lustre-MDT0001-mdtlov_UUID+5:3355:x1862456998087232:12345-0@lo:400:kworker/u8:1.0 Request processed in 56us (1419us total) trans 0 rc 0/0 00000100:00100000:0.0:1776178476.939478:0:20759:0:(nrs_fifo.c:241:nrs_fifo_req_stop()) NRS stop fifo request from 12345-0@lo, seq: 725 00000100:00100000:0.0:1776178476.939492:0:20759:0:(nrs_fifo.c:179:nrs_fifo_req_get()) NRS start fifo request from 12345-0@lo, seq: 726 00000100:00100000:0.0:1776178476.939495:0:20759:0:(service.c:2324:ptlrpc_server_handle_request()) Handling RPC req@0000000083bc4c0f pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_010:lustre-MDT0001-mdtlov_UUID+5:3355:x1862456998087296:12345-0@lo:400:kworker/u8:1.0 00000100:00100000:0.0:1776178476.939531:0:20759:0:(service.c:2372:ptlrpc_server_handle_request()) Handled RPC req@0000000083bc4c0f pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_010:lustre-MDT0001-mdtlov_UUID+5:3355:x1862456998087296:12345-0@lo:400:kworker/u8:1.0 Request processed in 35us (1393us total) trans 0 rc 0/0 00000100:00100000:0.0:1776178476.939537:0:20759:0:(nrs_fifo.c:241:nrs_fifo_req_stop()) NRS stop fifo request from 12345-0@lo, seq: 726 00000100:00100000:0.0:1776178476.939568:0:3355:0:(client.c:2215:ptlrpc_check_set()) Completed RPC req@000000007216887e pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_02:lustre-MDT0001-mdtlov_UUID:3355:1862456998087296:0@lo:400:kworker/u8:1.0 00000100:00100000:0.0:1776178476.939580:0:3355:0:(client.c:2215:ptlrpc_check_set()) Completed RPC req@000000002300ff10 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_02:lustre-MDT0001-mdtlov_UUID:3355:1862456998087232:0@lo:400:kworker/u8:1.0 00010000:00010000:1.0:1776178477.540311:0:9224:0:(ldlm_lockd.c:2792:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:2.0:1776178478.432422:0:3353:0:(client.c:1733:ptlrpc_send_new_req()) Sending RPC req@00000000044bd507 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_00:lustre-MDT0001-mdtlov_UUID:3353:1862456998087424:0@lo:13:osp-pre-1-1.0 00000100:00100000:2.0:1776178478.432501:0:3353:0:(events.c:361:request_in_callback()) peer: 12345-0@lo (source: 12345-0@lo) 00000100:00100000:0.0:1776178478.432693:0:9554:0:(service.c:2152:ptlrpc_server_handle_req_in()) got req x1862456998087424 00000100:00100000:0.0:1776178478.432760:0:9554:0:(nrs_fifo.c:179:nrs_fifo_req_get()) NRS start fifo request from 12345-0@lo, seq: 595 00000100:00100000:0.0:1776178478.432765:0:9554:0:(service.c:2324:ptlrpc_server_handle_request()) Handling RPC req@000000008aab9f17 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_create00:lustre-MDT0001-mdtlov_UUID+5:3353:x1862456998087424:12345-0@lo:13:osp-pre-1-1.0 00002000:00000024:0.0:1776178478.432795:0:9554:0:(ofd_obd.c:740:ofd_statfs()) blocks cached 229376 granted 491746368 pending 0 free 3923689472 avail 3692167168 00000100:00100000:0.0:1776178478.432845:0:9554:0:(service.c:2372:ptlrpc_server_handle_request()) Handled RPC req@000000008aab9f17 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_create00:lustre-MDT0001-mdtlov_UUID+5:3353:x1862456998087424:12345-0@lo:13:osp-pre-1-1.0 Request processed in 79us (345us total) trans 0 rc 0/0 00000100:00100000:0.0:1776178478.432853:0:9554:0:(nrs_fifo.c:241:nrs_fifo_req_stop()) NRS stop fifo request from 12345-0@lo, seq: 595 00000100:00100000:2.0:1776178478.432939:0:3353:0:(client.c:2215:ptlrpc_check_set()) Completed RPC req@00000000044bd507 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_00:lustre-MDT0001-mdtlov_UUID:3353:1862456998087424:0@lo:13:osp-pre-1-1.0 00010000:00010000:1.0:1776178478.561331:0:9225:0:(ldlm_lockd.c:2792:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:1.0:1776178478.945479:0:3354:0:(client.c:1733:ptlrpc_send_new_req()) Sending RPC req@000000002300ff10 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_01:lustre-MDT0001-mdtlov_UUID:3354:1862456998087488:0@lo:13:osp-pre-0-1.0 00000100:00100000:1.0:1776178478.945574:0:3354:0:(events.c:361:request_in_callback()) peer: 12345-0@lo (source: 12345-0@lo) 00000100:00100000:3.0:1776178478.945721:0:8231:0:(service.c:2152:ptlrpc_server_handle_req_in()) got req x1862456998087488 00000100:00100000:3.0:1776178478.945782:0:8231:0:(nrs_fifo.c:179:nrs_fifo_req_get()) NRS start fifo request from 12345-0@lo, seq: 596 00000100:00100000:3.0:1776178478.945789:0:8231:0:(service.c:2324:ptlrpc_server_handle_request()) Handling RPC req@0000000017d54e92 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_create00:lustre-MDT0001-mdtlov_UUID+5:3354:x1862456998087488:12345-0@lo:13:osp-pre-0-1.0 00002000:00000024:3.0:1776178478.945820:0:8231:0:(ofd_obd.c:740:ofd_statfs()) blocks cached 315392 granted 484883776 pending 0 free 3923689472 avail 3692167168 00000100:00100000:3.0:1776178478.945872:0:8231:0:(service.c:2372:ptlrpc_server_handle_request()) Handled RPC req@0000000017d54e92 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_create00:lustre-MDT0001-mdtlov_UUID+5:3354:x1862456998087488:12345-0@lo:13:osp-pre-0-1.0 Request processed in 82us (300us total) trans 0 rc 0/0 00000100:00100000:3.0:1776178478.945879:0:8231:0:(nrs_fifo.c:241:nrs_fifo_req_stop()) NRS stop fifo request from 12345-0@lo, seq: 596 00000100:00100000:1.0:1776178478.945970:0:3354:0:(client.c:2215:ptlrpc_check_set()) Completed RPC req@000000002300ff10 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_01:lustre-MDT0001-mdtlov_UUID:3354:1862456998087488:0@lo:13:osp-pre-0-1.0 00010000:00010000:3.0:1776178479.589515:0:6244:0:(ldlm_lockd.c:2792:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:2.0:1776178480.418157:0:3348:0:(events.c:361:request_in_callback()) peer: 12345-192.168.204.3@tcp (source: 12345-192.168.204.3@tcp) 00000100:00100000:2.0:1776178480.418207:0:3348:0:(events.c:361:request_in_callback()) peer: 12345-192.168.204.3@tcp (source: 12345-192.168.204.3@tcp) 00000100:00100000:2.0:1776178480.418241:0:3348:0:(events.c:361:request_in_callback()) peer: 12345-192.168.204.3@tcp (source: 12345-192.168.204.3@tcp) 00000100:00100000:3.0:1776178480.418323:0:20762:0:(service.c:2152:ptlrpc_server_handle_req_in()) got req x1862456979126464 00000100:00100000:0.0:1776178480.418327:0:20140:0:(service.c:2152:ptlrpc_server_handle_req_in()) got req x1862456979126272 00000100:00100000:3.0:1776178480.418381:0:20762:0:(nrs_fifo.c:179:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.204.3@tcp, seq: 727 00000100:00100000:0.0:1776178480.418382:0:20140:0:(nrs_fifo.c:179:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.204.3@tcp, seq: 263 00000100:00100000:3.0:1776178480.418393:0:20762:0:(service.c:2324:ptlrpc_server_handle_request()) Handling RPC req@00000000889a4355 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_013:b50963bd-8eba-4432-9a54-8888d00e3fa1+8:2272:x1862456979126464:12345-192.168.204.3@tcp:400:kworker/u8:4.0 00000100:00100000:0.0:1776178480.418401:0:20140:0:(service.c:2324:ptlrpc_server_handle_request()) Handling RPC req@000000003960fbcb pname:cluuid+ref:pid:xid:nid:opc:job ll_mgs_0002:5b5c66b1-1a2f-4651-8548-0c75b9a38a94+9:2272:x1862456979126272:12345-192.168.204.3@tcp:400:kworker/u8:4.0 00000100:00100000:3.0:1776178480.418489:0:20762:0:(service.c:2372:ptlrpc_server_handle_request()) Handled RPC req@00000000889a4355 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_013:b50963bd-8eba-4432-9a54-8888d00e3fa1+8:2272:x1862456979126464:12345-192.168.204.3@tcp:400:kworker/u8:4.0 Request processed in 98us (247us total) trans 0 rc 0/0 00000100:00100000:1.0:1776178480.418493:0:6255:0:(service.c:2152:ptlrpc_server_handle_req_in()) got req x1862456979126400 00000100:00100000:3.0:1776178480.418499:0:20762:0:(nrs_fifo.c:241:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.204.3@tcp, seq: 727 00000100:00100000:1.0:1776178480.418529:0:6255:0:(nrs_fifo.c:179:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.204.3@tcp, seq: 1090 00000100:00100000:1.0:1776178480.418534:0:6255:0:(service.c:2324:ptlrpc_server_handle_request()) Handling RPC req@000000003a3139fc pname:cluuid+ref:pid:xid:nid:opc:job mdt00_000:b50963bd-8eba-4432-9a54-8888d00e3fa1+6:2273:x1862456979126400:12345-192.168.204.3@tcp:400:kworker/u8:4.0 00000100:00100000:0.0:1776178480.426221:0:20140:0:(service.c:2372:ptlrpc_server_handle_request()) Handled RPC req@000000003960fbcb pname:cluuid+ref:pid:xid:nid:opc:job ll_mgs_0002:5b5c66b1-1a2f-4651-8548-0c75b9a38a94+9:2272:x1862456979126272:12345-192.168.204.3@tcp:400:kworker/u8:4.0 Request processed in 7813us (8055us total) trans 0 rc 0/0 00000100:00100000:0.0:1776178480.426239:0:20140:0:(nrs_fifo.c:241:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.204.3@tcp, seq: 263 00000100:00100000:1.0:1776178480.428827:0:6255:0:(service.c:2372:ptlrpc_server_handle_request()) Handled RPC req@000000003a3139fc pname:cluuid+ref:pid:xid:nid:opc:job mdt00_000:b50963bd-8eba-4432-9a54-8888d00e3fa1+6:2273:x1862456979126400:12345-192.168.204.3@tcp:400:kworker/u8:4.0 Request processed in 10278us (10606us total) trans 0 rc 0/0 00000100:00100000:1.0:1776178480.428848:0:6255:0:(nrs_fifo.c:241:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.204.3@tcp, seq: 1090 00000100:00100000:2.0:1776178480.431717:0:3347:0:(events.c:361:request_in_callback()) peer: 12345-192.168.204.3@tcp (source: 12345-192.168.204.3@tcp) 00000100:00100000:2.0:1776178480.431856:0:3347:0:(events.c:361:request_in_callback()) peer: 12345-192.168.204.3@tcp (source: 12345-192.168.204.3@tcp) 00000100:00100000:3.0:1776178480.432045:0:6255:0:(service.c:2152:ptlrpc_server_handle_req_in()) got req x1862456979126336 00000100:00100000:1.0:1776178480.432099:0:14440:0:(service.c:2152:ptlrpc_server_handle_req_in()) got req x1862456979126528 00000100:00100000:1.0:1776178480.432151:0:14440:0:(nrs_fifo.c:179:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.204.3@tcp, seq: 728 00000100:00100000:3.0:1776178480.432157:0:6255:0:(nrs_fifo.c:179:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.204.3@tcp, seq: 6840 00000100:00100000:1.0:1776178480.432162:0:14440:0:(service.c:2324:ptlrpc_server_handle_request()) Handling RPC req@0000000044f82452 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_004:b50963bd-8eba-4432-9a54-8888d00e3fa1+9:2273:x1862456979126528:12345-192.168.204.3@tcp:400:kworker/u8:4.0 00000100:00100000:3.0:1776178480.432168:0:6255:0:(service.c:2324:ptlrpc_server_handle_request()) Handling RPC req@0000000065dbeee0 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_000:0+-99:2269:x1862456979126336:12345-192.168.204.3@tcp:38:kworker/u8:4.0 00010000:02000000:3.0:1776178480.432186:0:6255:0:(ldlm_lib.c:1139:target_handle_connect()) lustre-MDT0000: Not available for connect from 192.168.204.3@tcp (stopping) 00000100:00100000:3.0:1776178480.432242:0:6255:0:(service.c:2372:ptlrpc_server_handle_request()) Handled RPC req@0000000065dbeee0 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_000:0+-99:2269:x1862456979126336:12345-192.168.204.3@tcp:38:kworker/u8:4.0 Request processed in 76us (386us total) trans 0 rc -19/-19 00000100:00100000:3.0:1776178480.432250:0:6255:0:(nrs_fifo.c:241:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.204.3@tcp, seq: 6840 00000100:00100000:1.0:1776178480.434296:0:14440:0:(service.c:2372:ptlrpc_server_handle_request()) Handled RPC req@0000000044f82452 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_004:b50963bd-8eba-4432-9a54-8888d00e3fa1+9:2273:x1862456979126528:12345-192.168.204.3@tcp:400:kworker/u8:4.0 Request processed in 2128us (2579us total) trans 0 rc 0/0 00000100:00100000:1.0:1776178480.434310:0:14440:0:(nrs_fifo.c:241:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.204.3@tcp, seq: 728 00010000:00010000:2.0:1776178480.608448:0:6245:0:(ldlm_lockd.c:2792:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00010000:00010000:1.0:1776178481.632310:0:9224:0:(ldlm_lockd.c:2792:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00080000:2.0:1776178482.017276:0:37:0:(pinger.c:251:ptlrpc_pinger_process_import()) lustre-MDT0000-lwp-OST0000_UUID->lustre-MDT0000_UUID: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00080000:2.0:1776178482.017285:0:37:0:(recover.c:57:ptlrpc_initiate_recovery()) lustre-MDT0000_UUID: starting recovery 00000100:00080000:2.0:1776178482.017288:0:37:0:(import.c:86:import_set_state_nolock()) 0000000047601a98 lustre-MDT0000_UUID: changing import state from DISCONN to CONNECTING 00000100:00080000:2.0:1776178482.017297:0:37:0:(import.c:534:import_select_connection()) lustre-MDT0000-lwp-OST0000: connect to NID 0@lo last attempt 1184 00000100:00080000:2.0:1776178482.017301:0:37:0:(import.c:581:import_select_connection()) lustre-MDT0000-lwp-OST0000: tried all connections, increasing latency to 21s 00000100:00080000:2.0:1776178482.017307:0:37:0:(import.c:615:import_select_connection()) lustre-MDT0000-lwp-OST0000: import 0000000047601a98 using connection 192.168.204.103@tcp/0@lo 00000100:00100000:2.0:1776178482.017341:0:37:0:(import.c:807:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5) req@00000000664eb80b x1862456998087616/t0(0) o38->lustre-MDT0000-lwp-OST0000@0@lo:12/10 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' 00000100:00080000:2.0:1776178482.017376:0:37:0:(pinger.c:251:ptlrpc_pinger_process_import()) lustre-MDT0000-lwp-OST0001_UUID->lustre-MDT0000_UUID: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00080000:2.0:1776178482.017379:0:37:0:(recover.c:57:ptlrpc_initiate_recovery()) lustre-MDT0000_UUID: starting recovery 00000100:00080000:2.0:1776178482.017382:0:37:0:(import.c:86:import_set_state_nolock()) 00000000dfc8db13 lustre-MDT0000_UUID: changing import state from DISCONN to CONNECTING 00000100:00080000:2.0:1776178482.017387:0:37:0:(import.c:534:import_select_connection()) lustre-MDT0000-lwp-OST0001: connect to NID 0@lo last attempt 1184 00000100:00080000:2.0:1776178482.017389:0:37:0:(import.c:581:import_select_connection()) lustre-MDT0000-lwp-OST0001: tried all connections, increasing latency to 21s 00000100:00080000:2.0:1776178482.017393:0:37:0:(import.c:615:import_select_connection()) lustre-MDT0000-lwp-OST0001: import 00000000dfc8db13 using connection 192.168.204.103@tcp/0@lo 00000100:00100000:2.0:1776178482.017413:0:37:0:(import.c:807:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5) req@00000000d384e871 x1862456998087744/t0(0) o38->lustre-MDT0000-lwp-OST0001@0@lo:12/10 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' 00000100:00080000:2.0:1776178482.017450:0:37:0:(pinger.c:251:ptlrpc_pinger_process_import()) lustre-MDT0001-mdtlov_UUID->lustre-MDT0000_UUID: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00080000:2.0:1776178482.017453:0:37:0:(recover.c:57:ptlrpc_initiate_recovery()) lustre-MDT0000_UUID: starting recovery 00000100:00080000:2.0:1776178482.017455:0:37:0:(import.c:86:import_set_state_nolock()) 00000000ba4f1821 lustre-MDT0000_UUID: changing import state from DISCONN to CONNECTING 00000100:00080000:2.0:1776178482.017459:0:37:0:(import.c:534:import_select_connection()) lustre-MDT0000-osp-MDT0001: connect to NID 0@lo last attempt 1184 00000100:00080000:2.0:1776178482.017461:0:37:0:(import.c:581:import_select_connection()) lustre-MDT0000-osp-MDT0001: tried all connections, increasing latency to 21s 00000100:00080000:2.0:1776178482.017464:0:37:0:(import.c:615:import_select_connection()) lustre-MDT0000-osp-MDT0001: import 00000000ba4f1821 using connection 192.168.204.103@tcp/0@lo 00000100:00100000:3.0:1776178482.017472:0:3356:0:(client.c:1733:ptlrpc_send_new_req()) Sending RPC req@0000000028ade006 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_03:lustre-MDT0001-lwp-OST0000_UUID:3356:1862456998087680:0@lo:400:kworker/u8:2.0 00000100:00100000:2.0:1776178482.017474:0:37:0:(import.c:807:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5) req@00000000165897bf x1862456998087872/t0(0) o38->lustre-MDT0000-osp-MDT0001@0@lo:24/4 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' 00000100:00080000:2.0:1776178482.017490:0:37:0:(pinger.c:251:ptlrpc_pinger_process_import()) lustre-MDT0000-lwp-MDT0001_UUID->lustre-MDT0000_UUID: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00080000:2.0:1776178482.017494:0:37:0:(recover.c:57:ptlrpc_initiate_recovery()) lustre-MDT0000_UUID: starting recovery 00000100:00080000:2.0:1776178482.017495:0:37:0:(import.c:86:import_set_state_nolock()) 00000000a7c55b1f lustre-MDT0000_UUID: changing import state from DISCONN to CONNECTING 00000100:00080000:2.0:1776178482.017497:0:37:0:(import.c:534:import_select_connection()) lustre-MDT0000-lwp-MDT0001: connect to NID 0@lo last attempt 1184 00000100:00080000:2.0:1776178482.017499:0:37:0:(import.c:581:import_select_connection()) lustre-MDT0000-lwp-MDT0001: tried all connections, increasing latency to 21s 00000100:00080000:2.0:1776178482.017501:0:37:0:(import.c:615:import_select_connection()) lustre-MDT0000-lwp-MDT0001: import 00000000a7c55b1f using connection 192.168.204.103@tcp/0@lo 00000100:00100000:2.0:1776178482.017508:0:37:0:(import.c:807:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5) req@000000007d60c0e9 x1862456998088064/t0(0) o38->lustre-MDT0000-lwp-MDT0001@0@lo:12/10 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' 00000100:00100000:2.0:1776178482.017532:0:3353:0:(client.c:1733:ptlrpc_send_new_req()) Sending RPC req@000000006c748d80 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_00:lustre-MDT0001-lwp-OST0001_UUID:3353:1862456998087808:0@lo:400:kworker/u8:2.0 00000100:00100000:3.0:1776178482.017542:0:3356:0:(events.c:361:request_in_callback()) peer: 12345-0@lo (source: 12345-0@lo) 00000100:00100000:2.0:1776178482.017555:0:3353:0:(events.c:361:request_in_callback()) peer: 12345-0@lo (source: 12345-0@lo) 00000100:00100000:2.0:1776178482.017577:0:3353:0:(ptlrpcd.c:415:ptlrpcd_check()) transfer 1 async RPCs [1->0] 00000100:00100000:3.0:1776178482.017577:0:3356:0:(ptlrpcd.c:415:ptlrpcd_check()) transfer 2 async RPCs [2->3] 00000100:00100000:2.0:1776178482.017582:0:3353:0:(client.c:1733:ptlrpc_send_new_req()) Sending RPC req@0000000092f98018 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_00:lustre-MDT0001-mdtlov_UUID:3353:1862456998087936:0@lo:400:kworker/u8:2.0 00000100:00100000:3.0:1776178482.017582:0:3356:0:(client.c:1733:ptlrpc_send_new_req()) Sending RPC req@00000000d28be450 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_03:ac61f839-0e8e-4cac-9c59-08bcfbbad330:3356:1862456998087552:0@lo:400:kworker/u8:2.0 00000100:00100000:0.0:1776178482.017673:0:8250:0:(service.c:2152:ptlrpc_server_handle_req_in()) got req x1862456998087680 00000100:00100000:0.0:1776178482.017724:0:8250:0:(service.c:2152:ptlrpc_server_handle_req_in()) got req x1862456998087808 00000100:00100000:0.0:1776178482.017744:0:8250:0:(nrs_fifo.c:179:nrs_fifo_req_get()) NRS start fifo request from 12345-0@lo, seq: 1091 00000100:00100000:0.0:1776178482.017750:0:8250:0:(service.c:2324:ptlrpc_server_handle_request()) Handling RPC req@00000000bdad24ac pname:cluuid+ref:pid:xid:nid:opc:job mdt00_003:lustre-MDT0001-lwp-OST0000_UUID+5:3356:x1862456998087680:12345-0@lo:400:kworker/u8:2.0 00000100:00100000:1.0:1776178482.017826:0:3352:0:(client.c:1733:ptlrpc_send_new_req()) Sending RPC req@00000000664eb80b pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:lustre-MDT0000-lwp-OST0000_UUID:3352:1862456998087616:0@lo:38:kworker/u8:2.0 00000100:00100000:2.0:1776178482.017857:0:3353:0:(events.c:361:request_in_callback()) peer: 12345-0@lo (source: 12345-0@lo) 00000100:00100000:2.0:1776178482.017887:0:6256:0:(nrs_fifo.c:179:nrs_fifo_req_get()) NRS start fifo request from 12345-0@lo, seq: 1092 00000100:00100000:2.0:1776178482.017891:0:6256:0:(service.c:2324:ptlrpc_server_handle_request()) Handling RPC req@00000000a6227918 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_001:lustre-MDT0001-lwp-OST0001_UUID+5:3353:x1862456998087808:12345-0@lo:400:kworker/u8:2.0 00000100:00100000:2.0:1776178482.017924:0:6256:0:(service.c:2372:ptlrpc_server_handle_request()) Handled RPC req@00000000a6227918 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_001:lustre-MDT0001-lwp-OST0001_UUID+5:3353:x1862456998087808:12345-0@lo:400:kworker/u8:2.0 Request processed in 31us (368us total) trans 0 rc 0/0 00000100:00100000:2.0:1776178482.017929:0:6256:0:(nrs_fifo.c:241:nrs_fifo_req_stop()) NRS stop fifo request from 12345-0@lo, seq: 1092 00000100:00100000:2.0:1776178482.017970:0:3353:0:(client.c:2215:ptlrpc_check_set()) Completed RPC req@000000006c748d80 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_00:lustre-MDT0001-lwp-OST0001_UUID:3353:1862456998087808:0@lo:400:kworker/u8:2.0 00000100:00100000:2.0:1776178482.018012:0:14440:0:(service.c:2152:ptlrpc_server_handle_req_in()) got req x1862456998087936 00000100:00100000:2.0:1776178482.018048:0:14440:0:(nrs_fifo.c:179:nrs_fifo_req_get()) NRS start fifo request from 12345-0@lo, seq: 729 00000100:00100000:2.0:1776178482.018052:0:14440:0:(service.c:2324:ptlrpc_server_handle_request()) Handling RPC req@0000000022d4dc29 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_004:lustre-MDT0001-mdtlov_UUID+5:3353:x1862456998087936:12345-0@lo:400:kworker/u8:2.0 00000100:00100000:2.0:1776178482.018109:0:14440:0:(service.c:2372:ptlrpc_server_handle_request()) Handled RPC req@0000000022d4dc29 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_004:lustre-MDT0001-mdtlov_UUID+5:3353:x1862456998087936:12345-0@lo:400:kworker/u8:2.0 Request processed in 56us (252us total) trans 0 rc 0/0 00000100:00100000:2.0:1776178482.018114:0:14440:0:(nrs_fifo.c:241:nrs_fifo_req_stop()) NRS stop fifo request from 12345-0@lo, seq: 729 00000100:00100000:2.0:1776178482.018135:0:3353:0:(client.c:2215:ptlrpc_check_set()) Completed RPC req@0000000092f98018 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_00:lustre-MDT0001-mdtlov_UUID:3353:1862456998087936:0@lo:400:kworker/u8:2.0 00000100:00100000:3.0:1776178482.021620:0:3356:0:(events.c:361:request_in_callback()) peer: 12345-0@lo (source: 12345-0@lo) 00000100:00100000:3.0:1776178482.021687:0:3356:0:(client.c:1733:ptlrpc_send_new_req()) Sending RPC req@0000000056ddcb09 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_03:lustre-MDT0001-mdtlov_UUID:3356:1862456998088000:0@lo:400:kworker/u8:2.0 00000100:00100000:3.0:1776178482.021946:0:3356:0:(events.c:361:request_in_callback()) peer: 12345-0@lo (source: 12345-0@lo) 00000100:00100000:2.0:1776178482.022093:0:20140:0:(service.c:2152:ptlrpc_server_handle_req_in()) got req x1862456998087552 00000100:00100000:2.0:1776178482.022139:0:20140:0:(nrs_fifo.c:179:nrs_fifo_req_get()) NRS start fifo request from 12345-0@lo, seq: 264 00000100:00100000:2.0:1776178482.022145:0:20140:0:(service.c:2324:ptlrpc_server_handle_request()) Handling RPC req@00000000f1dccdd0 pname:cluuid+ref:pid:xid:nid:opc:job ll_mgs_0002:ac61f839-0e8e-4cac-9c59-08bcfbbad330+18:3356:x1862456998087552:12345-0@lo:400:kworker/u8:2.0 00000100:00100000:2.0:1776178482.022196:0:20140:0:(service.c:2372:ptlrpc_server_handle_request()) Handled RPC req@00000000f1dccdd0 pname:cluuid+ref:pid:xid:nid:opc:job ll_mgs_0002:ac61f839-0e8e-4cac-9c59-08bcfbbad330+18:3356:x1862456998087552:12345-0@lo:400:kworker/u8:2.0 Request processed in 50us (584us total) trans 0 rc 0/0 00000100:00100000:2.0:1776178482.022203:0:20140:0:(nrs_fifo.c:241:nrs_fifo_req_stop()) NRS stop fifo request from 12345-0@lo, seq: 264 00000100:00100000:2.0:1776178482.022237:0:20762:0:(service.c:2152:ptlrpc_server_handle_req_in()) got req x1862456998088000 00000100:00100000:2.0:1776178482.022262:0:20762:0:(nrs_fifo.c:179:nrs_fifo_req_get()) NRS start fifo request from 12345-0@lo, seq: 730 00000100:00100000:2.0:1776178482.022266:0:20762:0:(service.c:2324:ptlrpc_server_handle_request()) Handling RPC req@0000000021b0d516 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_013:lustre-MDT0001-mdtlov_UUID+5:3356:x1862456998088000:12345-0@lo:400:kworker/u8:2.0 00000100:00100000:2.0:1776178482.022294:0:20762:0:(service.c:2372:ptlrpc_server_handle_request()) Handled RPC req@0000000021b0d516 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_013:lustre-MDT0001-mdtlov_UUID+5:3356:x1862456998088000:12345-0@lo:400:kworker/u8:2.0 Request processed in 28us (349us total) trans 0 rc 0/0 00000100:00100000:2.0:1776178482.022300:0:20762:0:(nrs_fifo.c:241:nrs_fifo_req_stop()) NRS stop fifo request from 12345-0@lo, seq: 730 00000100:00100000:3.0:1776178482.022384:0:3356:0:(client.c:2215:ptlrpc_check_set()) Completed RPC req@00000000d28be450 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_03:ac61f839-0e8e-4cac-9c59-08bcfbbad330:3356:1862456998087552:0@lo:400:kworker/u8:2.0 00000100:00100000:3.0:1776178482.022397:0:3356:0:(client.c:2215:ptlrpc_check_set()) Completed RPC req@0000000056ddcb09 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_03:lustre-MDT0001-mdtlov_UUID:3356:1862456998088000:0@lo:400:kworker/u8:2.0 00000100:00100000:0.0:1776178482.030584:0:8250:0:(service.c:2372:ptlrpc_server_handle_request()) Handled RPC req@00000000bdad24ac pname:cluuid+ref:pid:xid:nid:opc:job mdt00_003:lustre-MDT0001-lwp-OST0000_UUID+5:3356:x1862456998087680:12345-0@lo:400:kworker/u8:2.0 Request processed in 12824us (13033us total) trans 0 rc 0/0 00000100:00100000:0.0:1776178482.030601:0:8250:0:(nrs_fifo.c:241:nrs_fifo_req_stop()) NRS stop fifo request from 12345-0@lo, seq: 1091 00000100:00100000:1.0:1776178482.030750:0:3352:0:(events.c:361:request_in_callback()) peer: 12345-0@lo (source: 12345-0@lo) 00000100:00100000:1.0:1776178482.030786:0:3352:0:(client.c:1733:ptlrpc_send_new_req()) Sending RPC req@00000000d384e871 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:lustre-MDT0000-lwp-OST0001_UUID:3352:1862456998087744:0@lo:38:kworker/u8:2.0 00000100:00100000:1.0:1776178482.030813:0:3352:0:(events.c:361:request_in_callback()) peer: 12345-0@lo (source: 12345-0@lo) 00000100:00100000:0.0:1776178482.031079:0:8250:0:(service.c:2152:ptlrpc_server_handle_req_in()) got req x1862456998087616 00000100:00100000:0.0:1776178482.031113:0:8250:0:(service.c:2152:ptlrpc_server_handle_req_in()) got req x1862456998087744 00000100:00100000:0.0:1776178482.031130:0:8250:0:(nrs_fifo.c:179:nrs_fifo_req_get()) NRS start fifo request from 12345-0@lo, seq: 6841 00000100:00100000:0.0:1776178482.031137:0:8250:0:(service.c:2324:ptlrpc_server_handle_request()) Handling RPC req@00000000ba273b2a pname:cluuid+ref:pid:xid:nid:opc:job mdt00_003:0+-99:3352:x1862456998087616:12345-0@lo:38:kworker/u8:2.0 00010000:02000000:0.0:1776178482.031156:0:8250:0:(ldlm_lib.c:1139:target_handle_connect()) lustre-MDT0000: Not available for connect from 0@lo (stopping) 00000100:00100000:0.0:1776178482.031195:0:8250:0:(service.c:2372:ptlrpc_server_handle_request()) Handled RPC req@00000000ba273b2a pname:cluuid+ref:pid:xid:nid:opc:job mdt00_003:0+-99:3352:x1862456998087616:12345-0@lo:38:kworker/u8:2.0 Request processed in 58us (445us total) trans 0 rc -19/-19 00000100:00100000:0.0:1776178482.031214:0:8250:0:(nrs_fifo.c:241:nrs_fifo_req_stop()) NRS stop fifo request from 12345-0@lo, seq: 6841 00000100:00100000:0.0:1776178482.031225:0:8250:0:(nrs_fifo.c:179:nrs_fifo_req_get()) NRS start fifo request from 12345-0@lo, seq: 6842 00000100:00100000:0.0:1776178482.031229:0:8250:0:(service.c:2324:ptlrpc_server_handle_request()) Handling RPC req@00000000c1934814 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_003:0+-99:3352:x1862456998087744:12345-0@lo:38:kworker/u8:2.0 00010000:02000000:0.0:1776178482.031235:0:8250:0:(ldlm_lib.c:1139:target_handle_connect()) lustre-MDT0000: Not available for connect from 0@lo (stopping) 00000100:00100000:0.0:1776178482.031255:0:8250:0:(service.c:2372:ptlrpc_server_handle_request()) Handled RPC req@00000000c1934814 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_003:0+-99:3352:x1862456998087744:12345-0@lo:38:kworker/u8:2.0 Request processed in 25us (441us total) trans 0 rc -19/-19 00000100:00100000:0.0:1776178482.031260:0:8250:0:(nrs_fifo.c:241:nrs_fifo_req_stop()) NRS stop fifo request from 12345-0@lo, seq: 6842 00000100:00100000:1.0:1776178482.031717:0:3352:0:(client.c:1733:ptlrpc_send_new_req()) Sending RPC req@00000000165897bf pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:lustre-MDT0001-mdtlov_UUID:3352:1862456998087872:0@lo:38:kworker/u8:2.0 00000100:00100000:1.0:1776178482.031753:0:3352:0:(events.c:361:request_in_callback()) peer: 12345-0@lo (source: 12345-0@lo) 00000100:00100000:1.0:1776178482.031772:0:3352:0:(client.c:1733:ptlrpc_send_new_req()) Sending RPC req@000000007d60c0e9 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:lustre-MDT0000-lwp-MDT0001_UUID:3352:1862456998088064:0@lo:38:kworker/u8:2.0 00000100:00100000:1.0:1776178482.032012:0:3352:0:(events.c:361:request_in_callback()) peer: 12345-0@lo (source: 12345-0@lo) 00000100:00080000:1.0:1776178482.032078:0:3352:0:(import.c:86:import_set_state_nolock()) 0000000047601a98 lustre-MDT0000_UUID: changing import state from CONNECTING to DISCONN 00000100:00080000:1.0:1776178482.032083:0:3352:0:(import.c:1429:ptlrpc_connect_interpret()) recovery of lustre-MDT0000_UUID on 192.168.204.103@tcp failed (-19) 00000100:00100000:1.0:1776178482.032087:0:3352:0:(client.c:2215:ptlrpc_check_set()) Completed RPC req@00000000664eb80b pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:lustre-MDT0000-lwp-OST0000_UUID:3352:1862456998087616:0@lo:38:kworker/u8:2.0 00000100:00080000:1.0:1776178482.032097:0:3352:0:(import.c:86:import_set_state_nolock()) 00000000dfc8db13 lustre-MDT0000_UUID: changing import state from CONNECTING to DISCONN 00000100:00080000:1.0:1776178482.032100:0:3352:0:(import.c:1429:ptlrpc_connect_interpret()) recovery of lustre-MDT0000_UUID on 192.168.204.103@tcp failed (-19) 00000100:00100000:1.0:1776178482.032102:0:3352:0:(client.c:2215:ptlrpc_check_set()) Completed RPC req@00000000d384e871 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:lustre-MDT0000-lwp-OST0001_UUID:3352:1862456998087744:0@lo:38:kworker/u8:2.0 00000100:00100000:2.0:1776178482.032381:0:11281:0:(service.c:2152:ptlrpc_server_handle_req_in()) got req x1862456998088064 00000100:00100000:2.0:1776178482.032423:0:11281:0:(nrs_fifo.c:179:nrs_fifo_req_get()) NRS start fifo request from 12345-0@lo, seq: 6843 00000100:00100000:2.0:1776178482.032428:0:11281:0:(service.c:2324:ptlrpc_server_handle_request()) Handling RPC req@00000000c38303b3 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_004:0+-99:3352:x1862456998088064:12345-0@lo:38:kworker/u8:2.0 00010000:02000000:2.0:1776178482.032438:0:11281:0:(ldlm_lib.c:1139:target_handle_connect()) lustre-MDT0000: Not available for connect from 0@lo (stopping) 00000100:00100000:2.0:1776178482.032471:0:11281:0:(service.c:2372:ptlrpc_server_handle_request()) Handled RPC req@00000000c38303b3 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_004:0+-99:3352:x1862456998088064:12345-0@lo:38:kworker/u8:2.0 Request processed in 42us (459us total) trans 0 rc -19/-19 00000100:00100000:2.0:1776178482.032476:0:11281:0:(nrs_fifo.c:241:nrs_fifo_req_stop()) NRS stop fifo request from 12345-0@lo, seq: 6843 00000100:00100000:0.0:1776178482.032619:0:14128:0:(service.c:2152:ptlrpc_server_handle_req_in()) got req x1862456998087872 00000100:00100000:0.0:1776178482.032647:0:14128:0:(nrs_fifo.c:179:nrs_fifo_req_get()) NRS start fifo request from 12345-0@lo, seq: 762 00000100:00100000:0.0:1776178482.032650:0:14128:0:(service.c:2324:ptlrpc_server_handle_request()) Handling RPC req@000000008a6fdb8d pname:cluuid+ref:pid:xid:nid:opc:job mdt_out00_002:0+-99:3352:x1862456998087872:12345-0@lo:38:kworker/u8:2.0 00010000:02000000:0.0:1776178482.032657:0:14128:0:(ldlm_lib.c:1139:target_handle_connect()) lustre-MDT0000: Not available for connect from 0@lo (stopping) 00000100:00100000:0.0:1776178482.032683:0:14128:0:(service.c:2372:ptlrpc_server_handle_request()) Handled RPC req@000000008a6fdb8d pname:cluuid+ref:pid:xid:nid:opc:job mdt_out00_002:0+-99:3352:x1862456998087872:12345-0@lo:38:kworker/u8:2.0 Request processed in 32us (930us total) trans 0 rc -19/-19 00000100:00100000:0.0:1776178482.032689:0:14128:0:(nrs_fifo.c:241:nrs_fifo_req_stop()) NRS stop fifo request from 12345-0@lo, seq: 762 00000100:00080000:1.0:1776178482.032962:0:3352:0:(import.c:86:import_set_state_nolock()) 00000000ba4f1821 lustre-MDT0000_UUID: changing import state from CONNECTING to DISCONN 00000100:00080000:1.0:1776178482.032966:0:3352:0:(import.c:1429:ptlrpc_connect_interpret()) recovery of lustre-MDT0000_UUID on 192.168.204.103@tcp failed (-19) 00000100:00100000:1.0:1776178482.032970:0:3352:0:(client.c:2215:ptlrpc_check_set()) Completed RPC req@00000000165897bf pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:lustre-MDT0001-mdtlov_UUID:3352:1862456998087872:0@lo:38:kworker/u8:2.0 00000100:00080000:1.0:1776178482.032978:0:3352:0:(import.c:86:import_set_state_nolock()) 00000000a7c55b1f lustre-MDT0000_UUID: changing import state from CONNECTING to DISCONN 00000100:00080000:1.0:1776178482.032981:0:3352:0:(import.c:1429:ptlrpc_connect_interpret()) recovery of lustre-MDT0000_UUID on 192.168.204.103@tcp failed (-19) 00000100:00100000:1.0:1776178482.032983:0:3352:0:(client.c:2215:ptlrpc_check_set()) Completed RPC req@000000007d60c0e9 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:lustre-MDT0000-lwp-MDT0001_UUID:3352:1862456998088064:0@lo:38:kworker/u8:2.0 00000100:00100000:3.0:1776178482.035030:0:3356:0:(client.c:2215:ptlrpc_check_set()) Completed RPC req@0000000028ade006 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_03:lustre-MDT0001-lwp-OST0000_UUID:3356:1862456998087680:0@lo:400:kworker/u8:2.0 00010000:00010000:1.0:1776178482.657206:0:9225:0:(ldlm_lockd.c:2792:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:2.0:1776178483.552655:0:3355:0:(ptlrpcd.c:415:ptlrpcd_check()) transfer 1 async RPCs [3->2] 00000100:00100000:2.0:1776178483.552679:0:3355:0:(client.c:1733:ptlrpc_send_new_req()) Sending RPC req@000000006c7327ca pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_02:lustre-MDT0001-mdtlov_UUID:3355:1862456998088128:0@lo:13:osp-pre-1-1.0 00000100:00100000:2.0:1776178483.552747:0:3355:0:(events.c:361:request_in_callback()) peer: 12345-0@lo (source: 12345-0@lo) 00000100:00100000:1.0:1776178483.552858:0:9554:0:(service.c:2152:ptlrpc_server_handle_req_in()) got req x1862456998088128 00000100:00100000:1.0:1776178483.552907:0:9554:0:(nrs_fifo.c:179:nrs_fifo_req_get()) NRS start fifo request from 12345-0@lo, seq: 597 00000100:00100000:1.0:1776178483.552913:0:9554:0:(service.c:2324:ptlrpc_server_handle_request()) Handling RPC req@0000000039ff8335 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_create00:lustre-MDT0001-mdtlov_UUID+5:3355:x1862456998088128:12345-0@lo:13:osp-pre-1-1.0 00002000:00000024:1.0:1776178483.552941:0:9554:0:(ofd_obd.c:740:ofd_statfs()) blocks cached 229376 granted 491746368 pending 0 free 3923689472 avail 3692167168 00000100:00100000:1.0:1776178483.552990:0:9554:0:(service.c:2372:ptlrpc_server_handle_request()) Handled RPC req@0000000039ff8335 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_create00:lustre-MDT0001-mdtlov_UUID+5:3355:x1862456998088128:12345-0@lo:13:osp-pre-1-1.0 Request processed in 76us (243us total) trans 0 rc 0/0 00000100:00100000:1.0:1776178483.553007:0:9554:0:(nrs_fifo.c:241:nrs_fifo_req_stop()) NRS stop fifo request from 12345-0@lo, seq: 597 00000100:00100000:2.0:1776178483.553106:0:3355:0:(client.c:2215:ptlrpc_check_set()) Completed RPC req@000000006c7327ca pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_02:lustre-MDT0001-mdtlov_UUID:3355:1862456998088128:0@lo:13:osp-pre-1-1.0 00010000:00010000:3.0:1776178483.683782:0:6244:0:(ldlm_lockd.c:2792:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:2.0:1776178484.065245:0:3353:0:(client.c:1733:ptlrpc_send_new_req()) Sending RPC req@000000007216887e pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_00:lustre-MDT0001-mdtlov_UUID:3353:1862456998088192:0@lo:13:osp-pre-0-1.0 00000100:00100000:2.0:1776178484.065342:0:3353:0:(events.c:361:request_in_callback()) peer: 12345-0@lo (source: 12345-0@lo) 00000100:00100000:3.0:1776178484.065615:0:8231:0:(service.c:2152:ptlrpc_server_handle_req_in()) got req x1862456998088192 00000100:00100000:3.0:1776178484.065686:0:8231:0:(nrs_fifo.c:179:nrs_fifo_req_get()) NRS start fifo request from 12345-0@lo, seq: 598 00000100:00100000:3.0:1776178484.065693:0:8231:0:(service.c:2324:ptlrpc_server_handle_request()) Handling RPC req@000000007d60c0e9 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_create00:lustre-MDT0001-mdtlov_UUID+5:3353:x1862456998088192:12345-0@lo:13:osp-pre-0-1.0 00002000:00000024:3.0:1776178484.065731:0:8231:0:(ofd_obd.c:740:ofd_statfs()) blocks cached 315392 granted 484883776 pending 0 free 3923689472 avail 3692167168 00000100:00100000:3.0:1776178484.075723:0:8231:0:(service.c:2372:ptlrpc_server_handle_request()) Handled RPC req@000000007d60c0e9 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_create00:lustre-MDT0001-mdtlov_UUID+5:3353:x1862456998088192:12345-0@lo:13:osp-pre-0-1.0 Request processed in 10018us (10370us total) trans 0 rc 0/0 00000100:00100000:3.0:1776178484.075749:0:8231:0:(nrs_fifo.c:241:nrs_fifo_req_stop()) NRS stop fifo request from 12345-0@lo, seq: 598 00000100:00100000:2.0:1776178484.084284:0:3353:0:(client.c:2215:ptlrpc_check_set()) Completed RPC req@000000007216887e pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_00:lustre-MDT0001-mdtlov_UUID:3353:1862456998088192:0@lo:13:osp-pre-0-1.0 00010000:00010000:2.0:1776178484.720420:0:6245:0:(ldlm_lockd.c:2792:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:3.0:1776178485.537298:0:3346:0:(events.c:361:request_in_callback()) peer: 12345-192.168.204.3@tcp (source: 12345-192.168.204.3@tcp) 00000100:00100000:3.0:1776178485.537365:0:3346:0:(events.c:361:request_in_callback()) peer: 12345-192.168.204.3@tcp (source: 12345-192.168.204.3@tcp) 00000100:00100000:3.0:1776178485.537399:0:3346:0:(events.c:361:request_in_callback()) peer: 12345-192.168.204.3@tcp (source: 12345-192.168.204.3@tcp) 00000100:00100000:3.0:1776178485.537429:0:3346:0:(events.c:361:request_in_callback()) peer: 12345-192.168.204.3@tcp (source: 12345-192.168.204.3@tcp) 00000100:00100000:3.0:1776178485.537455:0:3346:0:(events.c:361:request_in_callback()) peer: 12345-192.168.204.3@tcp (source: 12345-192.168.204.3@tcp) 00000100:00100000:3.0:1776178485.537489:0:20762:0:(service.c:2152:ptlrpc_server_handle_req_in()) got req x1862456979126784 00000100:00100000:3.0:1776178485.537530:0:20762:0:(service.c:2152:ptlrpc_server_handle_req_in()) got req x1862456979126848 00000100:00100000:3.0:1776178485.537547:0:20762:0:(nrs_fifo.c:179:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.204.3@tcp, seq: 731 00000100:00100000:3.0:1776178485.537555:0:20762:0:(service.c:2324:ptlrpc_server_handle_request()) Handling RPC req@00000000044bd507 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_013:b50963bd-8eba-4432-9a54-8888d00e3fa1+8:2272:x1862456979126784:12345-192.168.204.3@tcp:400:kworker/u8:3.0 00000100:00100000:3.0:1776178485.537655:0:20762:0:(service.c:2372:ptlrpc_server_handle_request()) Handled RPC req@00000000044bd507 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_013:b50963bd-8eba-4432-9a54-8888d00e3fa1+8:2272:x1862456979126784:12345-192.168.204.3@tcp:400:kworker/u8:3.0 Request processed in 92us (249us total) trans 0 rc 0/0 00000100:00100000:3.0:1776178485.537663:0:20762:0:(nrs_fifo.c:241:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.204.3@tcp, seq: 731 00000100:00100000:3.0:1776178485.537681:0:20762:0:(nrs_fifo.c:179:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.204.3@tcp, seq: 732 00000100:00100000:3.0:1776178485.537685:0:20762:0:(service.c:2324:ptlrpc_server_handle_request()) Handling RPC req@0000000028ade006 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_013:b50963bd-8eba-4432-9a54-8888d00e3fa1+9:2273:x1862456979126848:12345-192.168.204.3@tcp:400:kworker/u8:3.0 00000100:00100000:3.0:1776178485.537708:0:20762:0:(service.c:2372:ptlrpc_server_handle_request()) Handled RPC req@0000000028ade006 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_013:b50963bd-8eba-4432-9a54-8888d00e3fa1+9:2273:x1862456979126848:12345-192.168.204.3@tcp:400:kworker/u8:3.0 Request processed in 23us (280us total) trans 0 rc 0/0 00000100:00100000:3.0:1776178485.537714:0:20762:0:(nrs_fifo.c:241:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.204.3@tcp, seq: 732 00000100:00100000:2.0:1776178485.537836:0:20137:0:(service.c:2152:ptlrpc_server_handle_req_in()) got req x1862456979126592 00000100:00100000:2.0:1776178485.537874:0:20137:0:(nrs_fifo.c:179:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.204.3@tcp, seq: 265 00000100:00100000:2.0:1776178485.537878:0:20137:0:(service.c:2324:ptlrpc_server_handle_request()) Handling RPC req@00000000322375a6 pname:cluuid+ref:pid:xid:nid:opc:job ll_mgs_0000:5b5c66b1-1a2f-4651-8548-0c75b9a38a94+9:2272:x1862456979126592:12345-192.168.204.3@tcp:400:kworker/u8:3.0 00000100:00100000:2.0:1776178485.537905:0:20137:0:(service.c:2372:ptlrpc_server_handle_request()) Handled RPC req@00000000322375a6 pname:cluuid+ref:pid:xid:nid:opc:job ll_mgs_0000:5b5c66b1-1a2f-4651-8548-0c75b9a38a94+9:2272:x1862456979126592:12345-192.168.204.3@tcp:400:kworker/u8:3.0 Request processed in 27us (619us total) trans 0 rc 0/0 00000100:00100000:2.0:1776178485.537911:0:20137:0:(nrs_fifo.c:241:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.204.3@tcp, seq: 265 00000100:00100000:1.0:1776178485.538291:0:6256:0:(service.c:2152:ptlrpc_server_handle_req_in()) got req x1862456979126720 00000100:00100000:1.0:1776178485.538330:0:6256:0:(service.c:2152:ptlrpc_server_handle_req_in()) got req x1862456979126656 00000100:00100000:1.0:1776178485.538358:0:6256:0:(nrs_fifo.c:179:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.204.3@tcp, seq: 1093 00000100:00100000:1.0:1776178485.538362:0:6256:0:(service.c:2324:ptlrpc_server_handle_request()) Handling RPC req@000000006c7327ca pname:cluuid+ref:pid:xid:nid:opc:job mdt00_001:b50963bd-8eba-4432-9a54-8888d00e3fa1+6:2273:x1862456979126720:12345-192.168.204.3@tcp:400:kworker/u8:3.0 00000100:00100000:1.0:1776178485.538395:0:6256:0:(service.c:2372:ptlrpc_server_handle_request()) Handled RPC req@000000006c7327ca pname:cluuid+ref:pid:xid:nid:opc:job mdt00_001:b50963bd-8eba-4432-9a54-8888d00e3fa1+6:2273:x1862456979126720:12345-192.168.204.3@tcp:400:kworker/u8:3.0 Request processed in 32us (1031us total) trans 0 rc 0/0 00000100:00100000:1.0:1776178485.538402:0:6256:0:(nrs_fifo.c:241:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.204.3@tcp, seq: 1093 00000100:00100000:1.0:1776178485.538414:0:6256:0:(nrs_fifo.c:179:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.204.3@tcp, seq: 6844 00000100:00100000:1.0:1776178485.538418:0:6256:0:(service.c:2324:ptlrpc_server_handle_request()) Handling RPC req@00000000d384e871 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_001:0+-99:2269:x1862456979126656:12345-192.168.204.3@tcp:38:kworker/u8:3.0 00010000:02000000:1.0:1776178485.538433:0:6256:0:(ldlm_lib.c:1139:target_handle_connect()) lustre-MDT0000: Not available for connect from 192.168.204.3@tcp (stopping) 00000100:00100000:1.0:1776178485.538458:0:6256:0:(service.c:2372:ptlrpc_server_handle_request()) Handled RPC req@00000000d384e871 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_001:0+-99:2269:x1862456979126656:12345-192.168.204.3@tcp:38:kworker/u8:3.0 Request processed in 39us (1003us total) trans 0 rc -19/-19 00000100:00100000:1.0:1776178485.538464:0:6256:0:(nrs_fifo.c:241:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.204.3@tcp, seq: 6844 00000020:02000400:3.0:1776178485.600216:0:22200:0:(genops.c:1786:obd_exports_barrier()) lustre-MDT0000 is waiting for obd_unlinked_exports more than 128 seconds. The obd refcount = 4. Is it stuck? 00000020:02000400:3.0:1776178485.613963:0:22200:0:(genops.c:1752:print_export_data()) lustre-MDT0000: UNLINKED 00000000ca22e26f b50963bd-8eba-4432-9a54-8888d00e3fa1 1 (0 0 0) 1 0 1 0: 0000000000000000 8589934704 stale:0 00010000:00010000:1.0:1776178485.728304:0:9224:0:(ldlm_lockd.c:2792:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00010000:00010000:1.0:1776178486.752351:0:9225:0:(ldlm_lockd.c:2792:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00080000:3.0:1776178487.137160:0:36:0:(pinger.c:251:ptlrpc_pinger_process_import()) lustre-MDT0000-lwp-OST0000_UUID->lustre-MDT0000_UUID: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00080000:3.0:1776178487.137169:0:36:0:(recover.c:57:ptlrpc_initiate_recovery()) lustre-MDT0000_UUID: starting recovery 00000100:00080000:3.0:1776178487.137173:0:36:0:(import.c:86:import_set_state_nolock()) 0000000047601a98 lustre-MDT0000_UUID: changing import state from DISCONN to CONNECTING 00000100:00080000:3.0:1776178487.137182:0:36:0:(import.c:534:import_select_connection()) lustre-MDT0000-lwp-OST0000: connect to NID 0@lo last attempt 1189 00000100:00080000:3.0:1776178487.137187:0:36:0:(import.c:581:import_select_connection()) lustre-MDT0000-lwp-OST0000: tried all connections, increasing latency to 21s 00000100:00080000:3.0:1776178487.137195:0:36:0:(import.c:615:import_select_connection()) lustre-MDT0000-lwp-OST0000: import 0000000047601a98 using connection 192.168.204.103@tcp/0@lo 00000100:00100000:3.0:1776178487.137219:0:36:0:(import.c:807:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5) req@00000000d28be450 x1862456998088320/t0(0) o38->lustre-MDT0000-lwp-OST0000@0@lo:12/10 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' 00000100:00080000:3.0:1776178487.137253:0:36:0:(pinger.c:251:ptlrpc_pinger_process_import()) lustre-MDT0000-lwp-OST0001_UUID->lustre-MDT0000_UUID: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00080000:3.0:1776178487.137256:0:36:0:(recover.c:57:ptlrpc_initiate_recovery()) lustre-MDT0000_UUID: starting recovery 00000100:00080000:3.0:1776178487.137258:0:36:0:(import.c:86:import_set_state_nolock()) 00000000dfc8db13 lustre-MDT0000_UUID: changing import state from DISCONN to CONNECTING 00000100:00080000:3.0:1776178487.137261:0:36:0:(import.c:534:import_select_connection()) lustre-MDT0000-lwp-OST0001: connect to NID 0@lo last attempt 1189 00000100:00080000:3.0:1776178487.137264:0:36:0:(import.c:581:import_select_connection()) lustre-MDT0000-lwp-OST0001: tried all connections, increasing latency to 21s 00000100:00080000:3.0:1776178487.137267:0:36:0:(import.c:615:import_select_connection()) lustre-MDT0000-lwp-OST0001: import 00000000dfc8db13 using connection 192.168.204.103@tcp/0@lo 00000100:00100000:3.0:1776178487.137277:0:36:0:(import.c:807:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5) req@00000000b7440871 x1862456998088448/t0(0) o38->lustre-MDT0000-lwp-OST0001@0@lo:12/10 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' 00000100:00080000:3.0:1776178487.137290:0:36:0:(pinger.c:251:ptlrpc_pinger_process_import()) lustre-MDT0001-mdtlov_UUID->lustre-MDT0000_UUID: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00080000:3.0:1776178487.137292:0:36:0:(recover.c:57:ptlrpc_initiate_recovery()) lustre-MDT0000_UUID: starting recovery 00000100:00080000:3.0:1776178487.137294:0:36:0:(import.c:86:import_set_state_nolock()) 00000000ba4f1821 lustre-MDT0000_UUID: changing import state from DISCONN to CONNECTING 00000100:00080000:3.0:1776178487.137296:0:36:0:(import.c:534:import_select_connection()) lustre-MDT0000-osp-MDT0001: connect to NID 0@lo last attempt 1189 00000100:00080000:3.0:1776178487.137298:0:36:0:(import.c:581:import_select_connection()) lustre-MDT0000-osp-MDT0001: tried all connections, increasing latency to 21s 00000100:00080000:3.0:1776178487.137301:0:36:0:(import.c:615:import_select_connection()) lustre-MDT0000-osp-MDT0001: import 00000000ba4f1821 using connection 192.168.204.103@tcp/0@lo 00000100:00100000:3.0:1776178487.137321:0:36:0:(import.c:807:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5) req@0000000037059d8f x1862456998088576/t0(0) o38->lustre-MDT0000-osp-MDT0001@0@lo:24/4 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' 00000100:00080000:3.0:1776178487.137344:0:36:0:(pinger.c:251:ptlrpc_pinger_process_import()) lustre-MDT0000-lwp-MDT0001_UUID->lustre-MDT0000_UUID: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00080000:3.0:1776178487.137347:0:36:0:(recover.c:57:ptlrpc_initiate_recovery()) lustre-MDT0000_UUID: starting recovery 00000100:00080000:3.0:1776178487.137348:0:36:0:(import.c:86:import_set_state_nolock()) 00000000a7c55b1f lustre-MDT0000_UUID: changing import state from DISCONN to CONNECTING 00000100:00080000:3.0:1776178487.137351:0:36:0:(import.c:534:import_select_connection()) lustre-MDT0000-lwp-MDT0001: connect to NID 0@lo last attempt 1189 00000100:00080000:3.0:1776178487.137353:0:36:0:(import.c:581:import_select_connection()) lustre-MDT0000-lwp-MDT0001: tried all connections, increasing latency to 21s 00000100:00080000:3.0:1776178487.137355:0:36:0:(import.c:615:import_select_connection()) lustre-MDT0000-lwp-MDT0001: import 00000000a7c55b1f using connection 192.168.204.103@tcp/0@lo 00000100:00100000:3.0:1776178487.137364:0:36:0:(import.c:807:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5) req@000000005927b2c3 x1862456998088768/t0(0) o38->lustre-MDT0000-lwp-MDT0001@0@lo:12/10 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' 00000100:00100000:3.0:1776178487.137417:0:3352:0:(client.c:1733:ptlrpc_send_new_req()) Sending RPC req@00000000d28be450 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:lustre-MDT0000-lwp-OST0000_UUID:3352:1862456998088320:0@lo:38:kworker/u8:1.0 00000100:00100000:3.0:1776178487.137485:0:3352:0:(events.c:361:request_in_callback()) peer: 12345-0@lo (source: 12345-0@lo) 00000100:00100000:3.0:1776178487.137511:0:3352:0:(client.c:1733:ptlrpc_send_new_req()) Sending RPC req@00000000b7440871 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:lustre-MDT0000-lwp-OST0001_UUID:3352:1862456998088448:0@lo:38:kworker/u8:1.0 00000100:00100000:3.0:1776178487.137531:0:3352:0:(events.c:361:request_in_callback()) peer: 12345-0@lo (source: 12345-0@lo) 00000100:00100000:3.0:1776178487.137541:0:3352:0:(client.c:1733:ptlrpc_send_new_req()) Sending RPC req@0000000037059d8f pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:lustre-MDT0001-mdtlov_UUID:3352:1862456998088576:0@lo:38:kworker/u8:1.0 00000100:00100000:3.0:1776178487.137564:0:3352:0:(events.c:361:request_in_callback()) peer: 12345-0@lo (source: 12345-0@lo) 00000100:00100000:3.0:1776178487.137576:0:3352:0:(client.c:1733:ptlrpc_send_new_req()) Sending RPC req@000000005927b2c3 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:lustre-MDT0000-lwp-MDT0001_UUID:3352:1862456998088768:0@lo:38:kworker/u8:1.0 00000100:00100000:3.0:1776178487.137593:0:3352:0:(events.c:361:request_in_callback()) peer: 12345-0@lo (source: 12345-0@lo) 00000100:00100000:2.0:1776178487.137679:0:3355:0:(client.c:1733:ptlrpc_send_new_req()) Sending RPC req@00000000d7e84c97 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_02:lustre-MDT0001-lwp-OST0000_UUID:3355:1862456998088384:0@lo:400:kworker/u8:1.0 00000100:00100000:2.0:1776178487.137704:0:3355:0:(events.c:361:request_in_callback()) peer: 12345-0@lo (source: 12345-0@lo) 00000100:00100000:2.0:1776178487.137729:0:3355:0:(ptlrpcd.c:415:ptlrpcd_check()) transfer 1 async RPCs [3->2] 00000100:00100000:2.0:1776178487.137734:0:3355:0:(client.c:1733:ptlrpc_send_new_req()) Sending RPC req@00000000758405de pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_02:lustre-MDT0001-lwp-OST0001_UUID:3355:1862456998088512:0@lo:400:kworker/u8:1.0 00000100:00100000:2.0:1776178487.137765:0:3355:0:(events.c:361:request_in_callback()) peer: 12345-0@lo (source: 12345-0@lo) 00000100:00100000:2.0:1776178487.137795:0:15213:0:(service.c:2152:ptlrpc_server_handle_req_in()) got req x1862456998088576 00000100:00100000:2.0:1776178487.137824:0:15213:0:(nrs_fifo.c:179:nrs_fifo_req_get()) NRS start fifo request from 12345-0@lo, seq: 763 00000100:00100000:2.0:1776178487.137830:0:15213:0:(service.c:2324:ptlrpc_server_handle_request()) Handling RPC req@00000000b8f6795f pname:cluuid+ref:pid:xid:nid:opc:job mdt_out00_003:0+-99:3352:x1862456998088576:12345-0@lo:38:kworker/u8:1.0 00010000:02000000:2.0:1776178487.137843:0:15213:0:(ldlm_lib.c:1139:target_handle_connect()) lustre-MDT0000: Not available for connect from 0@lo (stopping) 00000100:00100000:2.0:1776178487.137879:0:15213:0:(service.c:2372:ptlrpc_server_handle_request()) Handled RPC req@00000000b8f6795f pname:cluuid+ref:pid:xid:nid:opc:job mdt_out00_003:0+-99:3352:x1862456998088576:12345-0@lo:38:kworker/u8:1.0 Request processed in 48us (314us total) trans 0 rc -19/-19 00000100:00100000:2.0:1776178487.137885:0:15213:0:(nrs_fifo.c:241:nrs_fifo_req_stop()) NRS stop fifo request from 12345-0@lo, seq: 763 00000100:00100000:2.0:1776178487.137908:0:11281:0:(service.c:2152:ptlrpc_server_handle_req_in()) got req x1862456998088320 00000100:00100000:2.0:1776178487.137919:0:11281:0:(service.c:2152:ptlrpc_server_handle_req_in()) got req x1862456998088448 00000100:00100000:2.0:1776178487.137926:0:11281:0:(service.c:2152:ptlrpc_server_handle_req_in()) got req x1862456998088768 00000100:00100000:2.0:1776178487.137932:0:11281:0:(service.c:2152:ptlrpc_server_handle_req_in()) got req x1862456998088384 00000100:00100000:2.0:1776178487.137950:0:11281:0:(service.c:2152:ptlrpc_server_handle_req_in()) got req x1862456998088512 00000100:00100000:2.0:1776178487.137960:0:11281:0:(nrs_fifo.c:179:nrs_fifo_req_get()) NRS start fifo request from 12345-0@lo, seq: 1094 00000100:00100000:2.0:1776178487.137963:0:11281:0:(service.c:2324:ptlrpc_server_handle_request()) Handling RPC req@0000000056ddcb09 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_004:lustre-MDT0001-lwp-OST0000_UUID+5:3355:x1862456998088384:12345-0@lo:400:kworker/u8:1.0 00000100:00100000:2.0:1776178487.138002:0:11281:0:(service.c:2372:ptlrpc_server_handle_request()) Handled RPC req@0000000056ddcb09 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_004:lustre-MDT0001-lwp-OST0000_UUID+5:3355:x1862456998088384:12345-0@lo:400:kworker/u8:1.0 Request processed in 39us (298us total) trans 0 rc 0/0 00000100:00100000:2.0:1776178487.138007:0:11281:0:(nrs_fifo.c:241:nrs_fifo_req_stop()) NRS stop fifo request from 12345-0@lo, seq: 1094 00000100:00100000:2.0:1776178487.138018:0:11281:0:(nrs_fifo.c:179:nrs_fifo_req_get()) NRS start fifo request from 12345-0@lo, seq: 1095 00000100:00100000:2.0:1776178487.138020:0:11281:0:(service.c:2324:ptlrpc_server_handle_request()) Handling RPC req@00000000165897bf pname:cluuid+ref:pid:xid:nid:opc:job mdt00_004:lustre-MDT0001-lwp-OST0001_UUID+5:3355:x1862456998088512:12345-0@lo:400:kworker/u8:1.0 00000100:00100000:2.0:1776178487.138040:0:11281:0:(service.c:2372:ptlrpc_server_handle_request()) Handled RPC req@00000000165897bf pname:cluuid+ref:pid:xid:nid:opc:job mdt00_004:lustre-MDT0001-lwp-OST0001_UUID+5:3355:x1862456998088512:12345-0@lo:400:kworker/u8:1.0 Request processed in 19us (276us total) trans 0 rc 0/0 00000100:00100000:2.0:1776178487.138044:0:11281:0:(nrs_fifo.c:241:nrs_fifo_req_stop()) NRS stop fifo request from 12345-0@lo, seq: 1095 00000100:00100000:2.0:1776178487.138054:0:11281:0:(nrs_fifo.c:179:nrs_fifo_req_get()) NRS start fifo request from 12345-0@lo, seq: 6845 00000100:00100000:2.0:1776178487.138057:0:11281:0:(service.c:2324:ptlrpc_server_handle_request()) Handling RPC req@00000000a8bae24c pname:cluuid+ref:pid:xid:nid:opc:job mdt00_004:0+-99:3352:x1862456998088320:12345-0@lo:38:kworker/u8:1.0 00010000:02000000:2.0:1776178487.138087:0:11281:0:(ldlm_lib.c:1139:target_handle_connect()) lustre-MDT0000: Not available for connect from 0@lo (stopping) 00000100:00100000:2.0:1776178487.138107:0:11281:0:(service.c:2372:ptlrpc_server_handle_request()) Handled RPC req@00000000a8bae24c pname:cluuid+ref:pid:xid:nid:opc:job mdt00_004:0+-99:3352:x1862456998088320:12345-0@lo:38:kworker/u8:1.0 Request processed in 50us (623us total) trans 0 rc -19/-19 00000100:00100000:2.0:1776178487.138112:0:11281:0:(nrs_fifo.c:241:nrs_fifo_req_stop()) NRS stop fifo request from 12345-0@lo, seq: 6845 00000100:00100000:2.0:1776178487.138120:0:11281:0:(nrs_fifo.c:179:nrs_fifo_req_get()) NRS start fifo request from 12345-0@lo, seq: 6846 00000100:00100000:2.0:1776178487.138123:0:11281:0:(service.c:2324:ptlrpc_server_handle_request()) Handling RPC req@00000000d8af564d pname:cluuid+ref:pid:xid:nid:opc:job mdt00_004:0+-99:3352:x1862456998088448:12345-0@lo:38:kworker/u8:1.0 00010000:02000000:2.0:1776178487.138127:0:11281:0:(ldlm_lib.c:1139:target_handle_connect()) lustre-MDT0000: Not available for connect from 0@lo (stopping) 00000100:00100000:2.0:1776178487.138142:0:11281:0:(service.c:2372:ptlrpc_server_handle_request()) Handled RPC req@00000000d8af564d pname:cluuid+ref:pid:xid:nid:opc:job mdt00_004:0+-99:3352:x1862456998088448:12345-0@lo:38:kworker/u8:1.0 Request processed in 18us (611us total) trans 0 rc -19/-19 00000100:00100000:2.0:1776178487.138147:0:11281:0:(nrs_fifo.c:241:nrs_fifo_req_stop()) NRS stop fifo request from 12345-0@lo, seq: 6846 00000100:00100000:2.0:1776178487.138154:0:11281:0:(nrs_fifo.c:179:nrs_fifo_req_get()) NRS start fifo request from 12345-0@lo, seq: 6847 00000100:00100000:2.0:1776178487.138157:0:11281:0:(service.c:2324:ptlrpc_server_handle_request()) Handling RPC req@00000000f88b2465 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_004:0+-99:3352:x1862456998088768:12345-0@lo:38:kworker/u8:1.0 00010000:02000000:2.0:1776178487.138161:0:11281:0:(ldlm_lib.c:1139:target_handle_connect()) lustre-MDT0000: Not available for connect from 0@lo (stopping) 00000100:00100000:2.0:1776178487.138179:0:11281:0:(service.c:2372:ptlrpc_server_handle_request()) Handled RPC req@00000000f88b2465 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_004:0+-99:3352:x1862456998088768:12345-0@lo:38:kworker/u8:1.0 Request processed in 21us (585us total) trans 0 rc -19/-19 00000100:00100000:2.0:1776178487.138182:0:11281:0:(nrs_fifo.c:241:nrs_fifo_req_stop()) NRS stop fifo request from 12345-0@lo, seq: 6847 00000100:00100000:2.0:1776178487.138211:0:3355:0:(client.c:2215:ptlrpc_check_set()) Completed RPC req@00000000758405de pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_02:lustre-MDT0001-lwp-OST0001_UUID:3355:1862456998088512:0@lo:400:kworker/u8:1.0 00000100:00100000:2.0:1776178487.138219:0:3355:0:(client.c:2215:ptlrpc_check_set()) Completed RPC req@00000000d7e84c97 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_02:lustre-MDT0001-lwp-OST0000_UUID:3355:1862456998088384:0@lo:400:kworker/u8:1.0 00000100:00080000:3.0:1776178487.138380:0:3352:0:(import.c:86:import_set_state_nolock()) 0000000047601a98 lustre-MDT0000_UUID: changing import state from CONNECTING to DISCONN 00000100:00080000:3.0:1776178487.138383:0:3352:0:(import.c:1429:ptlrpc_connect_interpret()) recovery of lustre-MDT0000_UUID on 192.168.204.103@tcp failed (-19) 00000100:00100000:3.0:1776178487.138386:0:3352:0:(client.c:2215:ptlrpc_check_set()) Completed RPC req@00000000d28be450 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:lustre-MDT0000-lwp-OST0000_UUID:3352:1862456998088320:0@lo:38:kworker/u8:1.0 00000100:00080000:3.0:1776178487.138393:0:3352:0:(import.c:86:import_set_state_nolock()) 00000000dfc8db13 lustre-MDT0000_UUID: changing import state from CONNECTING to DISCONN 00000100:00080000:3.0:1776178487.138395:0:3352:0:(import.c:1429:ptlrpc_connect_interpret()) recovery of lustre-MDT0000_UUID on 192.168.204.103@tcp failed (-19) 00000100:00100000:3.0:1776178487.138397:0:3352:0:(client.c:2215:ptlrpc_check_set()) Completed RPC req@00000000b7440871 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:lustre-MDT0000-lwp-OST0001_UUID:3352:1862456998088448:0@lo:38:kworker/u8:1.0 00000100:00080000:3.0:1776178487.138403:0:3352:0:(import.c:86:import_set_state_nolock()) 00000000ba4f1821 lustre-MDT0000_UUID: changing import state from CONNECTING to DISCONN 00000100:00080000:3.0:1776178487.138405:0:3352:0:(import.c:1429:ptlrpc_connect_interpret()) recovery of lustre-MDT0000_UUID on 192.168.204.103@tcp failed (-19) 00000100:00100000:3.0:1776178487.138406:0:3352:0:(client.c:2215:ptlrpc_check_set()) Completed RPC req@0000000037059d8f pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:lustre-MDT0001-mdtlov_UUID:3352:1862456998088576:0@lo:38:kworker/u8:1.0 00000100:00080000:3.0:1776178487.138413:0:3352:0:(import.c:86:import_set_state_nolock()) 00000000a7c55b1f lustre-MDT0000_UUID: changing import state from CONNECTING to DISCONN 00000100:00080000:3.0:1776178487.138415:0:3352:0:(import.c:1429:ptlrpc_connect_interpret()) recovery of lustre-MDT0000_UUID on 192.168.204.103@tcp failed (-19) 00000100:00100000:3.0:1776178487.138417:0:3352:0:(client.c:2215:ptlrpc_check_set()) Completed RPC req@000000005927b2c3 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:lustre-MDT0000-lwp-MDT0001_UUID:3352:1862456998088768:0@lo:38:kworker/u8:1.0 00000100:00100000:0.0:1776178487.138498:0:3353:0:(client.c:1733:ptlrpc_send_new_req()) Sending RPC req@00000000d4b41d38 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_00:lustre-MDT0001-mdtlov_UUID:3353:1862456998088640:0@lo:400:kworker/u8:1.0 00000100:00100000:0.0:1776178487.138533:0:3353:0:(events.c:361:request_in_callback()) peer: 12345-0@lo (source: 12345-0@lo) 00000100:00100000:0.0:1776178487.138564:0:3353:0:(ptlrpcd.c:415:ptlrpcd_check()) transfer 2 async RPCs [1->0] 00000100:00100000:0.0:1776178487.138569:0:3353:0:(client.c:1733:ptlrpc_send_new_req()) Sending RPC req@00000000664eb80b pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_00:ac61f839-0e8e-4cac-9c59-08bcfbbad330:3353:1862456998088256:0@lo:400:kworker/u8:1.0 00000100:00100000:0.0:1776178487.138595:0:3353:0:(events.c:361:request_in_callback()) peer: 12345-0@lo (source: 12345-0@lo) 00000100:00100000:0.0:1776178487.138616:0:3353:0:(client.c:1733:ptlrpc_send_new_req()) Sending RPC req@00000000a30eb964 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_00:lustre-MDT0001-mdtlov_UUID:3353:1862456998088704:0@lo:400:kworker/u8:1.0 00000100:00100000:0.0:1776178487.138636:0:3353:0:(events.c:361:request_in_callback()) peer: 12345-0@lo (source: 12345-0@lo) 00000100:00100000:0.0:1776178487.138667:0:14440:0:(service.c:2152:ptlrpc_server_handle_req_in()) got req x1862456998088640 00000100:00100000:0.0:1776178487.138695:0:14440:0:(service.c:2152:ptlrpc_server_handle_req_in()) got req x1862456998088704 00000100:00100000:0.0:1776178487.138711:0:14440:0:(nrs_fifo.c:179:nrs_fifo_req_get()) NRS start fifo request from 12345-0@lo, seq: 733 00000100:00100000:0.0:1776178487.138715:0:14440:0:(service.c:2324:ptlrpc_server_handle_request()) Handling RPC req@00000000e8195050 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_004:lustre-MDT0001-mdtlov_UUID+5:3353:x1862456998088640:12345-0@lo:400:kworker/u8:1.0 00000100:00100000:0.0:1776178487.138747:0:14440:0:(service.c:2372:ptlrpc_server_handle_request()) Handled RPC req@00000000e8195050 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_004:lustre-MDT0001-mdtlov_UUID+5:3353:x1862456998088640:12345-0@lo:400:kworker/u8:1.0 Request processed in 30us (214us total) trans 0 rc 0/0 00000100:00100000:3.0:1776178487.138841:0:20140:0:(service.c:2152:ptlrpc_server_handle_req_in()) got req x1862456998088256 00000100:00100000:3.0:1776178487.138868:0:20140:0:(nrs_fifo.c:179:nrs_fifo_req_get()) NRS start fifo request from 12345-0@lo, seq: 266 00000100:00100000:3.0:1776178487.138872:0:20140:0:(service.c:2324:ptlrpc_server_handle_request()) Handling RPC req@00000000f0ff59c2 pname:cluuid+ref:pid:xid:nid:opc:job ll_mgs_0002:ac61f839-0e8e-4cac-9c59-08bcfbbad330+18:3353:x1862456998088256:12345-0@lo:400:kworker/u8:1.0 00000100:00100000:3.0:1776178487.138901:0:20140:0:(service.c:2372:ptlrpc_server_handle_request()) Handled RPC req@00000000f0ff59c2 pname:cluuid+ref:pid:xid:nid:opc:job ll_mgs_0002:ac61f839-0e8e-4cac-9c59-08bcfbbad330+18:3353:x1862456998088256:12345-0@lo:400:kworker/u8:1.0 Request processed in 28us (306us total) trans 0 rc 0/0 00000100:00100000:3.0:1776178487.138907:0:20140:0:(nrs_fifo.c:241:nrs_fifo_req_stop()) NRS stop fifo request from 12345-0@lo, seq: 266 00000100:00100000:3.0:1776178487.138974:0:20762:0:(nrs_fifo.c:179:nrs_fifo_req_get()) NRS start fifo request from 12345-0@lo, seq: 734 00000100:00100000:3.0:1776178487.138978:0:20762:0:(service.c:2324:ptlrpc_server_handle_request()) Handling RPC req@0000000037aace32 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_013:lustre-MDT0001-mdtlov_UUID+5:3353:x1862456998088704:12345-0@lo:400:kworker/u8:1.0 00000100:00100000:3.0:1776178487.139007:0:20762:0:(service.c:2372:ptlrpc_server_handle_request()) Handled RPC req@0000000037aace32 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_013:lustre-MDT0001-mdtlov_UUID+5:3353:x1862456998088704:12345-0@lo:400:kworker/u8:1.0 Request processed in 28us (370us total) trans 0 rc 0/0 00000100:00100000:3.0:1776178487.139013:0:20762:0:(nrs_fifo.c:241:nrs_fifo_req_stop()) NRS stop fifo request from 12345-0@lo, seq: 734 00000100:00100000:0.0:1776178487.140018:0:14440:0:(nrs_fifo.c:241:nrs_fifo_req_stop()) NRS stop fifo request from 12345-0@lo, seq: 733 00000100:00100000:0.0:1776178487.140077:0:3353:0:(client.c:2215:ptlrpc_check_set()) Completed RPC req@00000000664eb80b pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_00:ac61f839-0e8e-4cac-9c59-08bcfbbad330:3353:1862456998088256:0@lo:400:kworker/u8:1.0 00000100:00100000:0.0:1776178487.140089:0:3353:0:(client.c:2215:ptlrpc_check_set()) Completed RPC req@00000000a30eb964 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_00:lustre-MDT0001-mdtlov_UUID:3353:1862456998088704:0@lo:400:kworker/u8:1.0 00000100:00100000:0.0:1776178487.140098:0:3353:0:(client.c:2215:ptlrpc_check_set()) Completed RPC req@00000000d4b41d38 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_00:lustre-MDT0001-mdtlov_UUID:3353:1862456998088640:0@lo:400:kworker/u8:1.0 00010000:00010000:3.0:1776178487.776520:0:6244:0:(ldlm_lockd.c:2792:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:2.0:1776178488.672471:0:3355:0:(client.c:1733:ptlrpc_send_new_req()) Sending RPC req@00000000cc7f07e3 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_02:lustre-MDT0001-mdtlov_UUID:3355:1862456998088832:0@lo:13:osp-pre-1-1.0 00000100:00100000:2.0:1776178488.672565:0:3355:0:(events.c:361:request_in_callback()) peer: 12345-0@lo (source: 12345-0@lo) 00000100:00100000:0.0:1776178488.672662:0:8231:0:(service.c:2152:ptlrpc_server_handle_req_in()) got req x1862456998088832 00000100:00100000:0.0:1776178488.672737:0:8231:0:(nrs_fifo.c:179:nrs_fifo_req_get()) NRS start fifo request from 12345-0@lo, seq: 599 00000100:00100000:0.0:1776178488.672743:0:8231:0:(service.c:2324:ptlrpc_server_handle_request()) Handling RPC req@0000000092f98018 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_create00:lustre-MDT0001-mdtlov_UUID+5:3355:x1862456998088832:12345-0@lo:13:osp-pre-1-1.0 00002000:00000024:0.0:1776178488.672776:0:8231:0:(ofd_obd.c:740:ofd_statfs()) blocks cached 229376 granted 491746368 pending 0 free 3923689472 avail 3692167168 00000100:00100000:0.0:1776178488.673356:0:8231:0:(service.c:2372:ptlrpc_server_handle_request()) Handled RPC req@0000000092f98018 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_create00:lustre-MDT0001-mdtlov_UUID+5:3355:x1862456998088832:12345-0@lo:13:osp-pre-1-1.0 Request processed in 611us (791us total) trans 0 rc 0/0 00000100:00100000:0.0:1776178488.673364:0:8231:0:(nrs_fifo.c:241:nrs_fifo_req_stop()) NRS stop fifo request from 12345-0@lo, seq: 599 00000100:00100000:2.0:1776178488.673468:0:3355:0:(client.c:2215:ptlrpc_check_set()) Completed RPC req@00000000cc7f07e3 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_02:lustre-MDT0001-mdtlov_UUID:3355:1862456998088832:0@lo:13:osp-pre-1-1.0 00010000:00010000:2.0:1776178488.800572:0:6245:0:(ldlm_lockd.c:2792:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:1.0:1776178489.187788:0:3356:0:(client.c:1733:ptlrpc_send_new_req()) Sending RPC req@000000007216887e pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_03:lustre-MDT0001-mdtlov_UUID:3356:1862456998088896:0@lo:13:osp-pre-0-1.0 00000100:00100000:1.0:1776178489.187878:0:3356:0:(events.c:361:request_in_callback()) peer: 12345-0@lo (source: 12345-0@lo) 00000100:00100000:0.0:1776178489.188003:0:8231:0:(service.c:2152:ptlrpc_server_handle_req_in()) got req x1862456998088896 00000100:00100000:0.0:1776178489.189516:0:8231:0:(nrs_fifo.c:179:nrs_fifo_req_get()) NRS start fifo request from 12345-0@lo, seq: 600 00000100:00100000:0.0:1776178489.189526:0:8231:0:(service.c:2324:ptlrpc_server_handle_request()) Handling RPC req@00000000dd52d0d0 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_create00:lustre-MDT0001-mdtlov_UUID+5:3356:x1862456998088896:12345-0@lo:13:osp-pre-0-1.0 00002000:00000024:0.0:1776178489.189562:0:8231:0:(ofd_obd.c:740:ofd_statfs()) blocks cached 315392 granted 484883776 pending 0 free 3923689472 avail 3692167168 00000100:00100000:0.0:1776178489.191577:0:8231:0:(service.c:2372:ptlrpc_server_handle_request()) Handled RPC req@00000000dd52d0d0 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_create00:lustre-MDT0001-mdtlov_UUID+5:3356:x1862456998088896:12345-0@lo:13:osp-pre-0-1.0 Request processed in 2042us (3691us total) trans 0 rc 0/0 00000100:00100000:0.0:1776178489.191592:0:8231:0:(nrs_fifo.c:241:nrs_fifo_req_stop()) NRS stop fifo request from 12345-0@lo, seq: 600 00000100:00100000:3.0:1776178489.193663:0:3356:0:(client.c:2215:ptlrpc_check_set()) Completed RPC req@000000007216887e pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_03:lustre-MDT0001-mdtlov_UUID:3356:1862456998088896:0@lo:13:osp-pre-0-1.0 00010000:00010000:1.0:1776178489.828291:0:9224:0:(ldlm_lockd.c:2792:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:3.0:1776178490.661844:0:3346:0:(events.c:361:request_in_callback()) peer: 12345-192.168.204.3@tcp (source: 12345-192.168.204.3@tcp) 00000100:00100000:3.0:1776178490.662855:0:3346:0:(events.c:361:request_in_callback()) peer: 12345-192.168.204.3@tcp (source: 12345-192.168.204.3@tcp) 00000100:00100000:3.0:1776178490.674991:0:3346:0:(events.c:361:request_in_callback()) peer: 12345-192.168.204.3@tcp (source: 12345-192.168.204.3@tcp) 00000100:00100000:3.0:1776178490.675030:0:3346:0:(events.c:361:request_in_callback()) peer: 12345-192.168.204.3@tcp (source: 12345-192.168.204.3@tcp) 00000100:00100000:3.0:1776178490.675054:0:3346:0:(events.c:361:request_in_callback()) peer: 12345-192.168.204.3@tcp (source: 12345-192.168.204.3@tcp) 00000100:00100000:3.0:1776178490.675107:0:20137:0:(service.c:2152:ptlrpc_server_handle_req_in()) got req x1862456979126912 00000100:00100000:3.0:1776178490.675147:0:20137:0:(nrs_fifo.c:179:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.204.3@tcp, seq: 267 00000100:00100000:3.0:1776178490.675154:0:20137:0:(service.c:2324:ptlrpc_server_handle_request()) Handling RPC req@00000000ba1c7e2e pname:cluuid+ref:pid:xid:nid:opc:job ll_mgs_0000:5b5c66b1-1a2f-4651-8548-0c75b9a38a94+9:2273:x1862456979126912:12345-192.168.204.3@tcp:400:kworker/u8:3.0 00000100:00100000:3.0:1776178490.675219:0:20137:0:(service.c:2372:ptlrpc_server_handle_request()) Handled RPC req@00000000ba1c7e2e pname:cluuid+ref:pid:xid:nid:opc:job ll_mgs_0000:5b5c66b1-1a2f-4651-8548-0c75b9a38a94+9:2273:x1862456979126912:12345-192.168.204.3@tcp:400:kworker/u8:3.0 Request processed in 64us (188us total) trans 0 rc 0/0 00000100:00100000:3.0:1776178490.675227:0:20137:0:(nrs_fifo.c:241:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.204.3@tcp, seq: 267 00000100:00100000:3.0:1776178490.675436:0:14440:0:(service.c:2152:ptlrpc_server_handle_req_in()) got req x1862456979127168 00000100:00100000:3.0:1776178490.675462:0:14440:0:(service.c:2152:ptlrpc_server_handle_req_in()) got req x1862456979127104 00000100:00100000:3.0:1776178490.675479:0:14440:0:(nrs_fifo.c:179:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.204.3@tcp, seq: 735 00000100:00100000:3.0:1776178490.675484:0:14440:0:(service.c:2324:ptlrpc_server_handle_request()) Handling RPC req@00000000a30eb964 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_004:b50963bd-8eba-4432-9a54-8888d00e3fa1+9:2273:x1862456979127168:12345-192.168.204.3@tcp:400:kworker/u8:3.0 00000100:00100000:3.0:1776178490.675513:0:14440:0:(service.c:2372:ptlrpc_server_handle_request()) Handled RPC req@00000000a30eb964 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_004:b50963bd-8eba-4432-9a54-8888d00e3fa1+9:2273:x1862456979127168:12345-192.168.204.3@tcp:400:kworker/u8:3.0 Request processed in 29us (531us total) trans 0 rc 0/0 00000100:00100000:3.0:1776178490.675519:0:14440:0:(nrs_fifo.c:241:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.204.3@tcp, seq: 735 00000100:00100000:3.0:1776178490.675530:0:14440:0:(nrs_fifo.c:179:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.204.3@tcp, seq: 736 00000100:00100000:3.0:1776178490.675547:0:14440:0:(service.c:2324:ptlrpc_server_handle_request()) Handling RPC req@000000005927b2c3 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_004:b50963bd-8eba-4432-9a54-8888d00e3fa1+8:2273:x1862456979127104:12345-192.168.204.3@tcp:400:kworker/u8:3.0 00000100:00100000:3.0:1776178490.675574:0:14440:0:(service.c:2372:ptlrpc_server_handle_request()) Handled RPC req@000000005927b2c3 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_004:b50963bd-8eba-4432-9a54-8888d00e3fa1+8:2273:x1862456979127104:12345-192.168.204.3@tcp:400:kworker/u8:3.0 Request processed in 26us (519us total) trans 0 rc 0/0 00000100:00100000:3.0:1776178490.675580:0:14440:0:(nrs_fifo.c:241:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.204.3@tcp, seq: 736 00000100:00100000:0.0:1776178490.686080:0:8250:0:(service.c:2152:ptlrpc_server_handle_req_in()) got req x1862456979126976 00000100:00100000:0.0:1776178490.686138:0:8250:0:(service.c:2152:ptlrpc_server_handle_req_in()) got req x1862456979127040 00000100:00100000:0.0:1776178490.686178:0:8250:0:(nrs_fifo.c:179:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.204.3@tcp, seq: 1096 00000100:00100000:0.0:1776178490.686185:0:8250:0:(service.c:2324:ptlrpc_server_handle_request()) Handling RPC req@00000000cc7f07e3 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_003:b50963bd-8eba-4432-9a54-8888d00e3fa1+6:2273:x1862456979127040:12345-192.168.204.3@tcp:400:kworker/u8:3.0 00000100:00100000:0.0:1776178490.686297:0:8250:0:(service.c:2372:ptlrpc_server_handle_request()) Handled RPC req@00000000cc7f07e3 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_003:b50963bd-8eba-4432-9a54-8888d00e3fa1+6:2273:x1862456979127040:12345-192.168.204.3@tcp:400:kworker/u8:3.0 Request processed in 111us (23445us total) trans 0 rc 0/0 00000100:00100000:0.0:1776178490.686306:0:8250:0:(nrs_fifo.c:241:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.204.3@tcp, seq: 1096 00000100:00100000:0.0:1776178490.686325:0:8250:0:(nrs_fifo.c:179:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.204.3@tcp, seq: 6848 00000100:00100000:0.0:1776178490.686329:0:8250:0:(service.c:2324:ptlrpc_server_handle_request()) Handling RPC req@0000000098aa1ab6 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_003:0+-99:2269:x1862456979126976:12345-192.168.204.3@tcp:38:kworker/u8:3.0 00010000:02000000:0.0:1776178490.686344:0:8250:0:(ldlm_lib.c:1139:target_handle_connect()) lustre-MDT0000: Not available for connect from 192.168.204.3@tcp (stopping) 00000100:00100000:0.0:1776178490.689224:0:8250:0:(service.c:2372:ptlrpc_server_handle_request()) Handled RPC req@0000000098aa1ab6 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_003:0+-99:2269:x1862456979126976:12345-192.168.204.3@tcp:38:kworker/u8:3.0 Request processed in 2887us (27381us total) trans 0 rc -19/-19 00000100:00100000:0.0:1776178490.689249:0:8250:0:(nrs_fifo.c:241:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.204.3@tcp, seq: 6848 00010000:00010000:1.0:1776178490.851309:0:9225:0:(ldlm_lockd.c:2792:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00010000:00010000:3.0:1776178491.883573:0:6244:0:(ldlm_lockd.c:2792:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00080000:2.0:1776178492.256728:0:37:0:(pinger.c:251:ptlrpc_pinger_process_import()) lustre-MDT0000-lwp-OST0000_UUID->lustre-MDT0000_UUID: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00080000:2.0:1776178492.256736:0:37:0:(recover.c:57:ptlrpc_initiate_recovery()) lustre-MDT0000_UUID: starting recovery 00000100:00080000:2.0:1776178492.256741:0:37:0:(import.c:86:import_set_state_nolock()) 0000000047601a98 lustre-MDT0000_UUID: changing import state from DISCONN to CONNECTING 00000100:00080000:2.0:1776178492.256750:0:37:0:(import.c:534:import_select_connection()) lustre-MDT0000-lwp-OST0000: connect to NID 0@lo last attempt 1194 00000100:00080000:2.0:1776178492.256754:0:37:0:(import.c:581:import_select_connection()) lustre-MDT0000-lwp-OST0000: tried all connections, increasing latency to 21s 00000100:00080000:2.0:1776178492.256761:0:37:0:(import.c:615:import_select_connection()) lustre-MDT0000-lwp-OST0000: import 0000000047601a98 using connection 192.168.204.103@tcp/0@lo 00000100:00100000:2.0:1776178492.256787:0:37:0:(import.c:807:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5) req@00000000b7440871 x1862456998089024/t0(0) o38->lustre-MDT0000-lwp-OST0000@0@lo:12/10 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' 00000100:00080000:2.0:1776178492.256810:0:37:0:(pinger.c:251:ptlrpc_pinger_process_import()) lustre-MDT0000-lwp-OST0001_UUID->lustre-MDT0000_UUID: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00080000:2.0:1776178492.256812:0:37:0:(recover.c:57:ptlrpc_initiate_recovery()) lustre-MDT0000_UUID: starting recovery 00000100:00080000:2.0:1776178492.256814:0:37:0:(import.c:86:import_set_state_nolock()) 00000000dfc8db13 lustre-MDT0000_UUID: changing import state from DISCONN to CONNECTING 00000100:00080000:2.0:1776178492.256817:0:37:0:(import.c:534:import_select_connection()) lustre-MDT0000-lwp-OST0001: connect to NID 0@lo last attempt 1194 00000100:00080000:2.0:1776178492.256818:0:37:0:(import.c:581:import_select_connection()) lustre-MDT0000-lwp-OST0001: tried all connections, increasing latency to 21s 00000100:00080000:2.0:1776178492.256821:0:37:0:(import.c:615:import_select_connection()) lustre-MDT0000-lwp-OST0001: import 00000000dfc8db13 using connection 192.168.204.103@tcp/0@lo 00000100:00100000:2.0:1776178492.256829:0:37:0:(import.c:807:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5) req@00000000758405de x1862456998089152/t0(0) o38->lustre-MDT0000-lwp-OST0001@0@lo:12/10 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' 00000100:00080000:2.0:1776178492.256853:0:37:0:(pinger.c:251:ptlrpc_pinger_process_import()) lustre-MDT0001-mdtlov_UUID->lustre-MDT0000_UUID: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00080000:2.0:1776178492.256856:0:37:0:(recover.c:57:ptlrpc_initiate_recovery()) lustre-MDT0000_UUID: starting recovery 00000100:00080000:2.0:1776178492.256858:0:37:0:(import.c:86:import_set_state_nolock()) 00000000ba4f1821 lustre-MDT0000_UUID: changing import state from DISCONN to CONNECTING 00000100:00080000:2.0:1776178492.256860:0:37:0:(import.c:534:import_select_connection()) lustre-MDT0000-osp-MDT0001: connect to NID 0@lo last attempt 1194 00000100:00080000:2.0:1776178492.256862:0:37:0:(import.c:581:import_select_connection()) lustre-MDT0000-osp-MDT0001: tried all connections, increasing latency to 21s 00000100:00080000:2.0:1776178492.256865:0:37:0:(import.c:615:import_select_connection()) lustre-MDT0000-osp-MDT0001: import 00000000ba4f1821 using connection 192.168.204.103@tcp/0@lo 00000100:00100000:2.0:1776178492.256875:0:37:0:(import.c:807:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5) req@00000000f97fd38e x1862456998089280/t0(0) o38->lustre-MDT0000-osp-MDT0001@0@lo:24/4 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' 00000100:00080000:2.0:1776178492.256891:0:37:0:(pinger.c:251:ptlrpc_pinger_process_import()) lustre-MDT0000-lwp-MDT0001_UUID->lustre-MDT0000_UUID: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00080000:2.0:1776178492.256894:0:37:0:(recover.c:57:ptlrpc_initiate_recovery()) lustre-MDT0000_UUID: starting recovery 00000100:00080000:2.0:1776178492.256895:0:37:0:(import.c:86:import_set_state_nolock()) 00000000a7c55b1f lustre-MDT0000_UUID: changing import state from DISCONN to CONNECTING 00000100:00080000:2.0:1776178492.256898:0:37:0:(import.c:534:import_select_connection()) lustre-MDT0000-lwp-MDT0001: connect to NID 0@lo last attempt 1194 00000100:00080000:2.0:1776178492.256900:0:37:0:(import.c:581:import_select_connection()) lustre-MDT0000-lwp-MDT0001: tried all connections, increasing latency to 21s 00000100:00080000:2.0:1776178492.256903:0:37:0:(import.c:615:import_select_connection()) lustre-MDT0000-lwp-MDT0001: import 00000000a7c55b1f using connection 192.168.204.103@tcp/0@lo 00000100:00100000:2.0:1776178492.256924:0:37:0:(import.c:807:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5) req@000000008b978a8a x1862456998089472/t0(0) o38->lustre-MDT0000-lwp-MDT0001@0@lo:12/10 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' 00000100:00100000:1.0:1776178492.257251:0:3354:0:(client.c:1733:ptlrpc_send_new_req()) Sending RPC req@00000000d7e84c97 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_01:lustre-MDT0001-lwp-OST0000_UUID:3354:1862456998089088:0@lo:400:kworker/u8:2.0 00000100:00100000:1.0:1776178492.257334:0:3354:0:(events.c:361:request_in_callback()) peer: 12345-0@lo (source: 12345-0@lo) 00000100:00100000:1.0:1776178492.257368:0:3354:0:(ptlrpcd.c:415:ptlrpcd_check()) transfer 2 async RPCs [0->1] 00000100:00100000:1.0:1776178492.257373:0:3354:0:(client.c:1733:ptlrpc_send_new_req()) Sending RPC req@000000006c748d80 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_01:ac61f839-0e8e-4cac-9c59-08bcfbbad330:3354:1862456998088960:0@lo:400:kworker/u8:2.0 00000100:00100000:1.0:1776178492.257406:0:3354:0:(events.c:361:request_in_callback()) peer: 12345-0@lo (source: 12345-0@lo) 00000100:00100000:1.0:1776178492.257422:0:3354:0:(client.c:1733:ptlrpc_send_new_req()) Sending RPC req@000000007309261c pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_01:lustre-MDT0001-mdtlov_UUID:3354:1862456998089408:0@lo:400:kworker/u8:2.0 00000100:00100000:1.0:1776178492.257462:0:3354:0:(events.c:361:request_in_callback()) peer: 12345-0@lo (source: 12345-0@lo) 00000100:00100000:1.0:1776178492.257501:0:6255:0:(service.c:2152:ptlrpc_server_handle_req_in()) got req x1862456998089088 00000100:00100000:1.0:1776178492.257550:0:6255:0:(nrs_fifo.c:179:nrs_fifo_req_get()) NRS start fifo request from 12345-0@lo, seq: 1097 00000100:00100000:1.0:1776178492.257555:0:6255:0:(service.c:2324:ptlrpc_server_handle_request()) Handling RPC req@000000000daa3c9a pname:cluuid+ref:pid:xid:nid:opc:job mdt00_000:lustre-MDT0001-lwp-OST0000_UUID+5:3354:x1862456998089088:12345-0@lo:400:kworker/u8:2.0 00000100:00100000:1.0:1776178492.257604:0:6255:0:(service.c:2372:ptlrpc_server_handle_request()) Handled RPC req@000000000daa3c9a pname:cluuid+ref:pid:xid:nid:opc:job mdt00_000:lustre-MDT0001-lwp-OST0000_UUID+5:3354:x1862456998089088:12345-0@lo:400:kworker/u8:2.0 Request processed in 48us (272us total) trans 0 rc 0/0 00000100:00100000:1.0:1776178492.257612:0:6255:0:(nrs_fifo.c:241:nrs_fifo_req_stop()) NRS stop fifo request from 12345-0@lo, seq: 1097 00000100:00100000:1.0:1776178492.257638:0:20759:0:(service.c:2152:ptlrpc_server_handle_req_in()) got req x1862456998089408 00000100:00100000:1.0:1776178492.257663:0:20759:0:(nrs_fifo.c:179:nrs_fifo_req_get()) NRS start fifo request from 12345-0@lo, seq: 737 00000100:00100000:1.0:1776178492.257666:0:20759:0:(service.c:2324:ptlrpc_server_handle_request()) Handling RPC req@0000000087a31eb2 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_010:lustre-MDT0001-mdtlov_UUID+5:3354:x1862456998089408:12345-0@lo:400:kworker/u8:2.0 00000100:00100000:1.0:1776178492.257693:0:20759:0:(service.c:2372:ptlrpc_server_handle_request()) Handled RPC req@0000000087a31eb2 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_010:lustre-MDT0001-mdtlov_UUID+5:3354:x1862456998089408:12345-0@lo:400:kworker/u8:2.0 Request processed in 26us (231us total) trans 0 rc 0/0 00000100:00100000:1.0:1776178492.257699:0:20759:0:(nrs_fifo.c:241:nrs_fifo_req_stop()) NRS stop fifo request from 12345-0@lo, seq: 737 00000100:00100000:1.0:1776178492.257720:0:20140:0:(service.c:2152:ptlrpc_server_handle_req_in()) got req x1862456998088960 00000100:00100000:1.0:1776178492.257744:0:20140:0:(nrs_fifo.c:179:nrs_fifo_req_get()) NRS start fifo request from 12345-0@lo, seq: 268 00000100:00100000:1.0:1776178492.257748:0:20140:0:(service.c:2324:ptlrpc_server_handle_request()) Handling RPC req@000000005db97027 pname:cluuid+ref:pid:xid:nid:opc:job ll_mgs_0002:ac61f839-0e8e-4cac-9c59-08bcfbbad330+18:3354:x1862456998088960:12345-0@lo:400:kworker/u8:2.0 00000100:00100000:1.0:1776178492.257775:0:20140:0:(service.c:2372:ptlrpc_server_handle_request()) Handled RPC req@000000005db97027 pname:cluuid+ref:pid:xid:nid:opc:job ll_mgs_0002:ac61f839-0e8e-4cac-9c59-08bcfbbad330+18:3354:x1862456998088960:12345-0@lo:400:kworker/u8:2.0 Request processed in 26us (369us total) trans 0 rc 0/0 00000100:00100000:1.0:1776178492.257780:0:20140:0:(nrs_fifo.c:241:nrs_fifo_req_stop()) NRS stop fifo request from 12345-0@lo, seq: 268 00000100:00100000:1.0:1776178492.257817:0:3354:0:(client.c:2215:ptlrpc_check_set()) Completed RPC req@000000006c748d80 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_01:ac61f839-0e8e-4cac-9c59-08bcfbbad330:3354:1862456998088960:0@lo:400:kworker/u8:2.0 00000100:00100000:1.0:1776178492.257828:0:3354:0:(client.c:2215:ptlrpc_check_set()) Completed RPC req@000000007309261c pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_01:lustre-MDT0001-mdtlov_UUID:3354:1862456998089408:0@lo:400:kworker/u8:2.0 00000100:00100000:1.0:1776178492.257837:0:3354:0:(client.c:2215:ptlrpc_check_set()) Completed RPC req@00000000d7e84c97 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_01:lustre-MDT0001-lwp-OST0000_UUID:3354:1862456998089088:0@lo:400:kworker/u8:2.0 00000100:00100000:1.0:1776178492.257880:0:3356:0:(client.c:1733:ptlrpc_send_new_req()) Sending RPC req@0000000057946847 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_03:lustre-MDT0001-mdtlov_UUID:3356:1862456998089344:0@lo:400:kworker/u8:2.0 00000100:00100000:1.0:1776178492.257902:0:3356:0:(events.c:361:request_in_callback()) peer: 12345-0@lo (source: 12345-0@lo) 00000100:00100000:1.0:1776178492.257917:0:3356:0:(ptlrpcd.c:415:ptlrpcd_check()) transfer 1 async RPCs [2->3] 00000100:00100000:1.0:1776178492.257922:0:3356:0:(client.c:1733:ptlrpc_send_new_req()) Sending RPC req@0000000007e413b7 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_03:lustre-MDT0001-lwp-OST0001_UUID:3356:1862456998089216:0@lo:400:kworker/u8:2.0 00000100:00100000:1.0:1776178492.257940:0:3356:0:(events.c:361:request_in_callback()) peer: 12345-0@lo (source: 12345-0@lo) 00000100:00100000:1.0:1776178492.257961:0:3352:0:(client.c:1733:ptlrpc_send_new_req()) Sending RPC req@00000000b7440871 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:lustre-MDT0000-lwp-OST0000_UUID:3352:1862456998089024:0@lo:38:kworker/u8:2.0 00000100:00100000:1.0:1776178492.257985:0:3352:0:(events.c:361:request_in_callback()) peer: 12345-0@lo (source: 12345-0@lo) 00000100:00100000:1.0:1776178492.257997:0:3352:0:(client.c:1733:ptlrpc_send_new_req()) Sending RPC req@00000000758405de pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:lustre-MDT0000-lwp-OST0001_UUID:3352:1862456998089152:0@lo:38:kworker/u8:2.0 00000100:00100000:1.0:1776178492.258017:0:3352:0:(events.c:361:request_in_callback()) peer: 12345-0@lo (source: 12345-0@lo) 00000100:00100000:1.0:1776178492.258030:0:3352:0:(client.c:1733:ptlrpc_send_new_req()) Sending RPC req@00000000f97fd38e pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:lustre-MDT0001-mdtlov_UUID:3352:1862456998089280:0@lo:38:kworker/u8:2.0 00000100:00100000:1.0:1776178492.258050:0:3352:0:(events.c:361:request_in_callback()) peer: 12345-0@lo (source: 12345-0@lo) 00000100:00100000:1.0:1776178492.258098:0:3352:0:(client.c:1733:ptlrpc_send_new_req()) Sending RPC req@000000008b978a8a pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:lustre-MDT0000-lwp-MDT0001_UUID:3352:1862456998089472:0@lo:38:kworker/u8:2.0 00000100:00100000:1.0:1776178492.258502:0:3352:0:(events.c:361:request_in_callback()) peer: 12345-0@lo (source: 12345-0@lo) 00000100:00100000:1.0:1776178492.258541:0:20762:0:(service.c:2152:ptlrpc_server_handle_req_in()) got req x1862456998089344 00000100:00100000:1.0:1776178492.258565:0:20762:0:(nrs_fifo.c:179:nrs_fifo_req_get()) NRS start fifo request from 12345-0@lo, seq: 738 00000100:00100000:1.0:1776178492.258568:0:20762:0:(service.c:2324:ptlrpc_server_handle_request()) Handling RPC req@00000000d7e84c97 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_013:lustre-MDT0001-mdtlov_UUID+5:3356:x1862456998089344:12345-0@lo:400:kworker/u8:2.0 00000100:00100000:1.0:1776178492.258603:0:20762:0:(service.c:2372:ptlrpc_server_handle_request()) Handled RPC req@00000000d7e84c97 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_013:lustre-MDT0001-mdtlov_UUID+5:3356:x1862456998089344:12345-0@lo:400:kworker/u8:2.0 Request processed in 33us (700us total) trans 0 rc 0/0 00000100:00100000:1.0:1776178492.258608:0:20762:0:(nrs_fifo.c:241:nrs_fifo_req_stop()) NRS stop fifo request from 12345-0@lo, seq: 738 00000100:00100000:1.0:1776178492.258626:0:8250:0:(service.c:2152:ptlrpc_server_handle_req_in()) got req x1862456998089216 00000100:00100000:1.0:1776178492.258648:0:8250:0:(service.c:2152:ptlrpc_server_handle_req_in()) got req x1862456998089024 00000100:00100000:1.0:1776178492.258656:0:8250:0:(service.c:2152:ptlrpc_server_handle_req_in()) got req x1862456998089152 00000100:00100000:1.0:1776178492.258663:0:8250:0:(service.c:2152:ptlrpc_server_handle_req_in()) got req x1862456998089472 00000100:00100000:1.0:1776178492.258670:0:8250:0:(nrs_fifo.c:179:nrs_fifo_req_get()) NRS start fifo request from 12345-0@lo, seq: 1098 00000100:00100000:1.0:1776178492.258673:0:8250:0:(service.c:2324:ptlrpc_server_handle_request()) Handling RPC req@000000006c748d80 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_003:lustre-MDT0001-lwp-OST0001_UUID+5:3356:x1862456998089216:12345-0@lo:400:kworker/u8:2.0 00000100:00100000:1.0:1776178492.258699:0:8250:0:(service.c:2372:ptlrpc_server_handle_request()) Handled RPC req@000000006c748d80 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_003:lustre-MDT0001-lwp-OST0001_UUID+5:3356:x1862456998089216:12345-0@lo:400:kworker/u8:2.0 Request processed in 24us (758us total) trans 0 rc 0/0 00000100:00100000:1.0:1776178492.258705:0:8250:0:(nrs_fifo.c:241:nrs_fifo_req_stop()) NRS stop fifo request from 12345-0@lo, seq: 1098 00000100:00100000:1.0:1776178492.258715:0:8250:0:(nrs_fifo.c:179:nrs_fifo_req_get()) NRS start fifo request from 12345-0@lo, seq: 6849 00000100:00100000:1.0:1776178492.258717:0:8250:0:(service.c:2324:ptlrpc_server_handle_request()) Handling RPC req@000000005459e48f pname:cluuid+ref:pid:xid:nid:opc:job mdt00_003:0+-99:3352:x1862456998089024:12345-0@lo:38:kworker/u8:2.0 00010000:02000000:1.0:1776178492.258728:0:8250:0:(ldlm_lib.c:1139:target_handle_connect()) lustre-MDT0000: Not available for connect from 0@lo (stopping) 00000100:00100000:1.0:1776178492.258754:0:8250:0:(service.c:2372:ptlrpc_server_handle_request()) Handled RPC req@000000005459e48f pname:cluuid+ref:pid:xid:nid:opc:job mdt00_003:0+-99:3352:x1862456998089024:12345-0@lo:38:kworker/u8:2.0 Request processed in 36us (769us total) trans 0 rc -19/-19 00000100:00100000:1.0:1776178492.258758:0:8250:0:(nrs_fifo.c:241:nrs_fifo_req_stop()) NRS stop fifo request from 12345-0@lo, seq: 6849 00000100:00100000:1.0:1776178492.258767:0:8250:0:(nrs_fifo.c:179:nrs_fifo_req_get()) NRS start fifo request from 12345-0@lo, seq: 6850 00000100:00100000:1.0:1776178492.258769:0:8250:0:(service.c:2324:ptlrpc_server_handle_request()) Handling RPC req@0000000096181210 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_003:0+-99:3352:x1862456998089152:12345-0@lo:38:kworker/u8:2.0 00010000:02000000:1.0:1776178492.258775:0:8250:0:(ldlm_lib.c:1139:target_handle_connect()) lustre-MDT0000: Not available for connect from 0@lo (stopping) 00000100:00100000:1.0:1776178492.258792:0:8250:0:(service.c:2372:ptlrpc_server_handle_request()) Handled RPC req@0000000096181210 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_003:0+-99:3352:x1862456998089152:12345-0@lo:38:kworker/u8:2.0 Request processed in 21us (775us total) trans 0 rc -19/-19 00000100:00100000:1.0:1776178492.258797:0:8250:0:(nrs_fifo.c:241:nrs_fifo_req_stop()) NRS stop fifo request from 12345-0@lo, seq: 6850 00000100:00100000:1.0:1776178492.258806:0:8250:0:(nrs_fifo.c:179:nrs_fifo_req_get()) NRS start fifo request from 12345-0@lo, seq: 6851 00000100:00100000:1.0:1776178492.258809:0:8250:0:(service.c:2324:ptlrpc_server_handle_request()) Handling RPC req@0000000078c1d864 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_003:0+-99:3352:x1862456998089472:12345-0@lo:38:kworker/u8:2.0 00010000:02000000:1.0:1776178492.258815:0:8250:0:(ldlm_lib.c:1139:target_handle_connect()) lustre-MDT0000: Not available for connect from 0@lo (stopping) 00000100:00100000:1.0:1776178492.258831:0:8250:0:(service.c:2372:ptlrpc_server_handle_request()) Handled RPC req@0000000078c1d864 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_003:0+-99:3352:x1862456998089472:12345-0@lo:38:kworker/u8:2.0 Request processed in 21us (330us total) trans 0 rc -19/-19 00000100:00100000:1.0:1776178492.258835:0:8250:0:(nrs_fifo.c:241:nrs_fifo_req_stop()) NRS stop fifo request from 12345-0@lo, seq: 6851 00000100:00100000:1.0:1776178492.258862:0:3356:0:(client.c:2215:ptlrpc_check_set()) Completed RPC req@0000000007e413b7 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_03:lustre-MDT0001-lwp-OST0001_UUID:3356:1862456998089216:0@lo:400:kworker/u8:2.0 00000100:00100000:1.0:1776178492.258869:0:3356:0:(client.c:2215:ptlrpc_check_set()) Completed RPC req@0000000057946847 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_03:lustre-MDT0001-mdtlov_UUID:3356:1862456998089344:0@lo:400:kworker/u8:2.0 00000100:00080000:1.0:1776178492.258891:0:3352:0:(import.c:86:import_set_state_nolock()) 0000000047601a98 lustre-MDT0000_UUID: changing import state from CONNECTING to DISCONN 00000100:00080000:1.0:1776178492.258894:0:3352:0:(import.c:1429:ptlrpc_connect_interpret()) recovery of lustre-MDT0000_UUID on 192.168.204.103@tcp failed (-19) 00000100:00100000:1.0:1776178492.258897:0:3352:0:(client.c:2215:ptlrpc_check_set()) Completed RPC req@00000000b7440871 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:lustre-MDT0000-lwp-OST0000_UUID:3352:1862456998089024:0@lo:38:kworker/u8:2.0 00000100:00080000:1.0:1776178492.258902:0:3352:0:(import.c:86:import_set_state_nolock()) 00000000dfc8db13 lustre-MDT0000_UUID: changing import state from CONNECTING to DISCONN 00000100:00080000:1.0:1776178492.258905:0:3352:0:(import.c:1429:ptlrpc_connect_interpret()) recovery of lustre-MDT0000_UUID on 192.168.204.103@tcp failed (-19) 00000100:00100000:1.0:1776178492.258907:0:3352:0:(client.c:2215:ptlrpc_check_set()) Completed RPC req@00000000758405de pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:lustre-MDT0000-lwp-OST0001_UUID:3352:1862456998089152:0@lo:38:kworker/u8:2.0 00000100:00080000:1.0:1776178492.258912:0:3352:0:(import.c:86:import_set_state_nolock()) 00000000a7c55b1f lustre-MDT0000_UUID: changing import state from CONNECTING to DISCONN 00000100:00100000:2.0:1776178492.259281:0:14128:0:(service.c:2152:ptlrpc_server_handle_req_in()) got req x1862456998089280 00000100:00100000:2.0:1776178492.259305:0:14128:0:(nrs_fifo.c:179:nrs_fifo_req_get()) NRS start fifo request from 12345-0@lo, seq: 764 00000100:00100000:2.0:1776178492.259309:0:14128:0:(service.c:2324:ptlrpc_server_handle_request()) Handling RPC req@00000000ee45921e pname:cluuid+ref:pid:xid:nid:opc:job mdt_out00_002:0+-99:3352:x1862456998089280:12345-0@lo:38:kworker/u8:2.0 00010000:02000000:2.0:1776178492.259316:0:14128:0:(ldlm_lib.c:1139:target_handle_connect()) lustre-MDT0000: Not available for connect from 0@lo (stopping) 00000100:00100000:2.0:1776178492.259357:0:14128:0:(service.c:2372:ptlrpc_server_handle_request()) Handled RPC req@00000000ee45921e pname:cluuid+ref:pid:xid:nid:opc:job mdt_out00_002:0+-99:3352:x1862456998089280:12345-0@lo:38:kworker/u8:2.0 Request processed in 47us (1305us total) trans 0 rc -19/-19 00000100:00100000:2.0:1776178492.259362:0:14128:0:(nrs_fifo.c:241:nrs_fifo_req_stop()) NRS stop fifo request from 12345-0@lo, seq: 764 00000100:00080000:1.0:1776178492.260967:0:3352:0:(import.c:1429:ptlrpc_connect_interpret()) recovery of lustre-MDT0000_UUID on 192.168.204.103@tcp failed (-19) 00000100:00100000:1.0:1776178492.260976:0:3352:0:(client.c:2215:ptlrpc_check_set()) Completed RPC req@000000008b978a8a pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:lustre-MDT0000-lwp-MDT0001_UUID:3352:1862456998089472:0@lo:38:kworker/u8:2.0 00000100:00080000:1.0:1776178492.261024:0:3352:0:(import.c:86:import_set_state_nolock()) 00000000ba4f1821 lustre-MDT0000_UUID: changing import state from CONNECTING to DISCONN 00000100:00080000:1.0:1776178492.261027:0:3352:0:(import.c:1429:ptlrpc_connect_interpret()) recovery of lustre-MDT0000_UUID on 192.168.204.103@tcp failed (-19) 00000100:00100000:1.0:1776178492.261029:0:3352:0:(client.c:2215:ptlrpc_check_set()) Completed RPC req@00000000f97fd38e pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:lustre-MDT0001-mdtlov_UUID:3352:1862456998089280:0@lo:38:kworker/u8:2.0 00010000:00010000:2.0:1776178492.899963:0:6245:0:(ldlm_lockd.c:2792:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:2.0:1776178493.792748:0:3353:0:(ptlrpcd.c:415:ptlrpcd_check()) transfer 1 async RPCs [1->0] 00000100:00100000:2.0:1776178493.792771:0:3353:0:(client.c:1733:ptlrpc_send_new_req()) Sending RPC req@0000000037059d8f pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_00:lustre-MDT0001-mdtlov_UUID:3353:1862456998089536:0@lo:13:osp-pre-1-1.0 00000100:00100000:2.0:1776178493.792842:0:3353:0:(events.c:361:request_in_callback()) peer: 12345-0@lo (source: 12345-0@lo) 00000100:00100000:0.0:1776178493.792957:0:8231:0:(service.c:2152:ptlrpc_server_handle_req_in()) got req x1862456998089536 00000100:00100000:0.0:1776178493.793030:0:8231:0:(nrs_fifo.c:179:nrs_fifo_req_get()) NRS start fifo request from 12345-0@lo, seq: 601 00000100:00100000:0.0:1776178493.793036:0:8231:0:(service.c:2324:ptlrpc_server_handle_request()) Handling RPC req@000000001ee743a7 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_create00:lustre-MDT0001-mdtlov_UUID+5:3353:x1862456998089536:12345-0@lo:13:osp-pre-1-1.0 00002000:00000024:0.0:1776178493.793100:0:8231:0:(ofd_obd.c:740:ofd_statfs()) blocks cached 229376 granted 491746368 pending 0 free 3923689472 avail 3692167168 00000100:00100000:0.0:1776178493.793160:0:8231:0:(service.c:2372:ptlrpc_server_handle_request()) Handled RPC req@000000001ee743a7 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_create00:lustre-MDT0001-mdtlov_UUID+5:3353:x1862456998089536:12345-0@lo:13:osp-pre-1-1.0 Request processed in 122us (318us total) trans 0 rc 0/0 00000100:00100000:0.0:1776178493.793171:0:8231:0:(nrs_fifo.c:241:nrs_fifo_req_stop()) NRS stop fifo request from 12345-0@lo, seq: 601 00000100:00100000:3.0:1776178493.794754:0:3353:0:(client.c:2215:ptlrpc_check_set()) Completed RPC req@0000000037059d8f pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_00:lustre-MDT0001-mdtlov_UUID:3353:1862456998089536:0@lo:13:osp-pre-1-1.0 00010000:00010000:1.0:1776178493.925661:0:9224:0:(ldlm_lockd.c:2792:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:2.0:1776178494.307596:0:3355:0:(client.c:1733:ptlrpc_send_new_req()) Sending RPC req@000000002300ff10 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_02:lustre-MDT0001-mdtlov_UUID:3355:1862456998089600:0@lo:13:osp-pre-0-1.0 00000100:00100000:2.0:1776178494.307680:0:3355:0:(events.c:361:request_in_callback()) peer: 12345-0@lo (source: 12345-0@lo) 00000100:00100000:3.0:1776178494.308202:0:9554:0:(service.c:2152:ptlrpc_server_handle_req_in()) got req x1862456998089600 00000100:00100000:3.0:1776178494.308260:0:9554:0:(nrs_fifo.c:179:nrs_fifo_req_get()) NRS start fifo request from 12345-0@lo, seq: 602 00000100:00100000:3.0:1776178494.308266:0:9554:0:(service.c:2324:ptlrpc_server_handle_request()) Handling RPC req@00000000f97fd38e pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_create00:lustre-MDT0001-mdtlov_UUID+5:3355:x1862456998089600:12345-0@lo:13:osp-pre-0-1.0 00002000:00000024:3.0:1776178494.308296:0:9554:0:(ofd_obd.c:740:ofd_statfs()) blocks cached 315392 granted 484883776 pending 0 free 3923689472 avail 3692167168 00000100:00100000:3.0:1776178494.308356:0:9554:0:(service.c:2372:ptlrpc_server_handle_request()) Handled RPC req@00000000f97fd38e pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_create00:lustre-MDT0001-mdtlov_UUID+5:3355:x1862456998089600:12345-0@lo:13:osp-pre-0-1.0 Request processed in 80us (668us total) trans 0 rc 0/0 00000100:00100000:3.0:1776178494.308364:0:9554:0:(nrs_fifo.c:241:nrs_fifo_req_stop()) NRS stop fifo request from 12345-0@lo, seq: 602 00000100:00100000:2.0:1776178494.308541:0:3355:0:(client.c:2215:ptlrpc_check_set()) Completed RPC req@000000002300ff10 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_02:lustre-MDT0001-mdtlov_UUID:3355:1862456998089600:0@lo:13:osp-pre-0-1.0 00010000:00010000:1.0:1776178494.944453:0:9225:0:(ldlm_lockd.c:2792:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:2.0:1776178495.779806:0:3347:0:(events.c:361:request_in_callback()) peer: 12345-192.168.204.3@tcp (source: 12345-192.168.204.3@tcp) 00000100:00100000:3.0:1776178495.780183:0:6257:0:(service.c:2152:ptlrpc_server_handle_req_in()) got req x1862456979127296 00000100:00100000:3.0:1776178495.780289:0:6257:0:(nrs_fifo.c:179:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.204.3@tcp, seq: 6852 00000100:00100000:3.0:1776178495.780298:0:6257:0:(service.c:2324:ptlrpc_server_handle_request()) Handling RPC req@000000008b978a8a pname:cluuid+ref:pid:xid:nid:opc:job mdt00_002:0+-99:2269:x1862456979127296:12345-192.168.204.3@tcp:38:kworker/u8:4.0 00010000:02000000:3.0:1776178495.780328:0:6257:0:(ldlm_lib.c:1139:target_handle_connect()) lustre-MDT0000: Not available for connect from 192.168.204.3@tcp (stopping) 00000100:00100000:3.0:1776178495.780563:0:6257:0:(service.c:2372:ptlrpc_server_handle_request()) Handled RPC req@000000008b978a8a pname:cluuid+ref:pid:xid:nid:opc:job mdt00_002:0+-99:2269:x1862456979127296:12345-192.168.204.3@tcp:38:kworker/u8:4.0 Request processed in 264us (770us total) trans 0 rc -19/-19 00000100:00100000:3.0:1776178495.780572:0:6257:0:(nrs_fifo.c:241:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.204.3@tcp, seq: 6852 00000100:00100000:1.0:1776178495.791027:0:3346:0:(events.c:361:request_in_callback()) peer: 12345-192.168.204.3@tcp (source: 12345-192.168.204.3@tcp) 00000100:00100000:1.0:1776178495.791119:0:3346:0:(events.c:361:request_in_callback()) peer: 12345-192.168.204.3@tcp (source: 12345-192.168.204.3@tcp) 00000100:00100000:1.0:1776178495.791171:0:3346:0:(events.c:361:request_in_callback()) peer: 12345-192.168.204.3@tcp (source: 12345-192.168.204.3@tcp) 00000100:00100000:1.0:1776178495.791296:0:3346:0:(events.c:361:request_in_callback()) peer: 12345-192.168.204.3@tcp (source: 12345-192.168.204.3@tcp) 00000100:00100000:1.0:1776178495.791365:0:20137:0:(service.c:2152:ptlrpc_server_handle_req_in()) got req x1862456979127232 00000100:00100000:1.0:1776178495.791456:0:20137:0:(nrs_fifo.c:179:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.204.3@tcp, seq: 269 00000100:00100000:1.0:1776178495.791466:0:20137:0:(service.c:2324:ptlrpc_server_handle_request()) Handling RPC req@0000000017512ce0 pname:cluuid+ref:pid:xid:nid:opc:job ll_mgs_0000:5b5c66b1-1a2f-4651-8548-0c75b9a38a94+9:2270:x1862456979127232:12345-192.168.204.3@tcp:400:kworker/u8:4.0 00000100:00100000:1.0:1776178495.792766:0:20137:0:(service.c:2372:ptlrpc_server_handle_request()) Handled RPC req@0000000017512ce0 pname:cluuid+ref:pid:xid:nid:opc:job ll_mgs_0000:5b5c66b1-1a2f-4651-8548-0c75b9a38a94+9:2270:x1862456979127232:12345-192.168.204.3@tcp:400:kworker/u8:4.0 Request processed in 1292us (1587us total) trans 0 rc 0/0 00000100:00100000:1.0:1776178495.792783:0:20137:0:(nrs_fifo.c:241:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.204.3@tcp, seq: 269 00000100:00100000:0.0:1776178495.792799:0:20759:0:(service.c:2152:ptlrpc_server_handle_req_in()) got req x1862456979127488 00000100:00100000:0.0:1776178495.792885:0:20759:0:(service.c:2152:ptlrpc_server_handle_req_in()) got req x1862456979127424 00000100:00100000:0.0:1776178495.792925:0:20759:0:(nrs_fifo.c:179:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.204.3@tcp, seq: 739 00000100:00100000:0.0:1776178495.792934:0:20759:0:(service.c:2324:ptlrpc_server_handle_request()) Handling RPC req@00000000b7440871 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_010:b50963bd-8eba-4432-9a54-8888d00e3fa1+9:2270:x1862456979127488:12345-192.168.204.3@tcp:400:kworker/u8:4.0 00000100:00100000:1.0:1776178495.793125:0:20762:0:(nrs_fifo.c:179:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.204.3@tcp, seq: 740 00000100:00100000:1.0:1776178495.793136:0:20762:0:(service.c:2324:ptlrpc_server_handle_request()) Handling RPC req@0000000056e345df pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_013:b50963bd-8eba-4432-9a54-8888d00e3fa1+8:2270:x1862456979127424:12345-192.168.204.3@tcp:400:kworker/u8:4.0 00000100:00100000:1.0:1776178495.793213:0:20762:0:(service.c:2372:ptlrpc_server_handle_request()) Handled RPC req@0000000056e345df pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_013:b50963bd-8eba-4432-9a54-8888d00e3fa1+8:2270:x1862456979127424:12345-192.168.204.3@tcp:400:kworker/u8:4.0 Request processed in 78us (1918us total) trans 0 rc 0/0 00000100:00100000:1.0:1776178495.793221:0:20762:0:(nrs_fifo.c:241:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.204.3@tcp, seq: 740 00000100:00100000:0.0:1776178495.794270:0:20759:0:(service.c:2372:ptlrpc_server_handle_request()) Handled RPC req@00000000b7440871 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_010:b50963bd-8eba-4432-9a54-8888d00e3fa1+9:2270:x1862456979127488:12345-192.168.204.3@tcp:400:kworker/u8:4.0 Request processed in 1329us (3154us total) trans 0 rc 0/0 00000100:00100000:0.0:1776178495.794283:0:20759:0:(nrs_fifo.c:241:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.204.3@tcp, seq: 739 00000100:00100000:2.0:1776178495.806001:0:6257:0:(service.c:2152:ptlrpc_server_handle_req_in()) got req x1862456979127360 00000100:00100000:2.0:1776178495.806132:0:6257:0:(nrs_fifo.c:179:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.204.3@tcp, seq: 1099 00000100:00100000:2.0:1776178495.806149:0:6257:0:(service.c:2324:ptlrpc_server_handle_request()) Handling RPC req@00000000758405de pname:cluuid+ref:pid:xid:nid:opc:job mdt00_002:b50963bd-8eba-4432-9a54-8888d00e3fa1+6:2270:x1862456979127360:12345-192.168.204.3@tcp:400:kworker/u8:4.0 00000100:00100000:2.0:1776178495.806299:0:6257:0:(service.c:2372:ptlrpc_server_handle_request()) Handled RPC req@00000000758405de pname:cluuid+ref:pid:xid:nid:opc:job mdt00_002:b50963bd-8eba-4432-9a54-8888d00e3fa1+6:2270:x1862456979127360:12345-192.168.204.3@tcp:400:kworker/u8:4.0 Request processed in 150us (15289us total) trans 0 rc 0/0 00000100:00100000:2.0:1776178495.806845:0:6257:0:(nrs_fifo.c:241:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.204.3@tcp, seq: 1099 00010000:00010000:3.0:1776178495.970147:0:6244:0:(ldlm_lockd.c:2792:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00010000:00010000:2.0:1776178496.992473:0:6245:0:(ldlm_lockd.c:2792:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00080000:1.0:1776178497.379082:0:36:0:(pinger.c:251:ptlrpc_pinger_process_import()) lustre-MDT0000-lwp-OST0000_UUID->lustre-MDT0000_UUID: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00080000:1.0:1776178497.379098:0:36:0:(recover.c:57:ptlrpc_initiate_recovery()) lustre-MDT0000_UUID: starting recovery 00000100:00080000:1.0:1776178497.379104:0:36:0:(import.c:86:import_set_state_nolock()) 0000000047601a98 lustre-MDT0000_UUID: changing import state from DISCONN to CONNECTING 00000100:00080000:1.0:1776178497.379128:0:36:0:(import.c:534:import_select_connection()) lustre-MDT0000-lwp-OST0000: connect to NID 0@lo last attempt 1199 00000100:00080000:1.0:1776178497.379133:0:36:0:(import.c:581:import_select_connection()) lustre-MDT0000-lwp-OST0000: tried all connections, increasing latency to 21s 00000100:00080000:1.0:1776178497.379143:0:36:0:(import.c:615:import_select_connection()) lustre-MDT0000-lwp-OST0000: import 0000000047601a98 using connection 192.168.204.103@tcp/0@lo 00000100:00100000:1.0:1776178497.379178:0:36:0:(import.c:807:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5) req@000000003095df64 x1862456998089728/t0(0) o38->lustre-MDT0000-lwp-OST0000@0@lo:12/10 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' 00000100:00080000:1.0:1776178497.379228:0:36:0:(pinger.c:251:ptlrpc_pinger_process_import()) lustre-MDT0000-lwp-OST0001_UUID->lustre-MDT0000_UUID: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00080000:1.0:1776178497.379233:0:36:0:(recover.c:57:ptlrpc_initiate_recovery()) lustre-MDT0000_UUID: starting recovery 00000100:00080000:1.0:1776178497.379235:0:36:0:(import.c:86:import_set_state_nolock()) 00000000dfc8db13 lustre-MDT0000_UUID: changing import state from DISCONN to CONNECTING 00000100:00080000:1.0:1776178497.379240:0:36:0:(import.c:534:import_select_connection()) lustre-MDT0000-lwp-OST0001: connect to NID 0@lo last attempt 1199 00000100:00080000:1.0:1776178497.379242:0:36:0:(import.c:581:import_select_connection()) lustre-MDT0000-lwp-OST0001: tried all connections, increasing latency to 21s 00000100:00080000:1.0:1776178497.379246:0:36:0:(import.c:615:import_select_connection()) lustre-MDT0000-lwp-OST0001: import 00000000dfc8db13 using connection 192.168.204.103@tcp/0@lo 00000100:00100000:1.0:1776178497.379262:0:36:0:(import.c:807:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5) req@000000009497b17e x1862456998089856/t0(0) o38->lustre-MDT0000-lwp-OST0001@0@lo:12/10 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' 00000100:00080000:1.0:1776178497.379277:0:36:0:(pinger.c:251:ptlrpc_pinger_process_import()) lustre-MDT0001-mdtlov_UUID->lustre-MDT0000_UUID: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00080000:1.0:1776178497.379281:0:36:0:(recover.c:57:ptlrpc_initiate_recovery()) lustre-MDT0000_UUID: starting recovery 00000100:00080000:1.0:1776178497.379282:0:36:0:(import.c:86:import_set_state_nolock()) 00000000ba4f1821 lustre-MDT0000_UUID: changing import state from DISCONN to CONNECTING 00000100:00080000:1.0:1776178497.379285:0:36:0:(import.c:534:import_select_connection()) lustre-MDT0000-osp-MDT0001: connect to NID 0@lo last attempt 1199 00000100:00080000:1.0:1776178497.379287:0:36:0:(import.c:581:import_select_connection()) lustre-MDT0000-osp-MDT0001: tried all connections, increasing latency to 21s 00000100:00080000:1.0:1776178497.379292:0:36:0:(import.c:615:import_select_connection()) lustre-MDT0000-osp-MDT0001: import 00000000ba4f1821 using connection 192.168.204.103@tcp/0@lo 00000100:00100000:1.0:1776178497.379303:0:36:0:(import.c:807:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5) req@000000000489d393 x1862456998089984/t0(0) o38->lustre-MDT0000-osp-MDT0001@0@lo:24/4 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' 00000100:00080000:1.0:1776178497.379324:0:36:0:(pinger.c:251:ptlrpc_pinger_process_import()) lustre-MDT0000-lwp-MDT0001_UUID->lustre-MDT0000_UUID: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00080000:1.0:1776178497.379328:0:36:0:(recover.c:57:ptlrpc_initiate_recovery()) lustre-MDT0000_UUID: starting recovery 00000100:00080000:1.0:1776178497.379329:0:36:0:(import.c:86:import_set_state_nolock()) 00000000a7c55b1f lustre-MDT0000_UUID: changing import state from DISCONN to CONNECTING 00000100:00080000:1.0:1776178497.379333:0:36:0:(import.c:534:import_select_connection()) lustre-MDT0000-lwp-MDT0001: connect to NID 0@lo last attempt 1199 00000100:00080000:1.0:1776178497.379335:0:36:0:(import.c:581:import_select_connection()) lustre-MDT0000-lwp-MDT0001: tried all connections, increasing latency to 21s 00000100:00080000:1.0:1776178497.379339:0:36:0:(import.c:615:import_select_connection()) lustre-MDT0000-lwp-MDT0001: import 00000000a7c55b1f using connection 192.168.204.103@tcp/0@lo 00000100:00100000:1.0:1776178497.379348:0:36:0:(import.c:807:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5) req@000000004547bc6e x1862456998090176/t0(0) o38->lustre-MDT0000-lwp-MDT0001@0@lo:12/10 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' 00000100:00100000:1.0:1776178497.379414:0:3354:0:(client.c:1733:ptlrpc_send_new_req()) Sending RPC req@000000001d2ef500 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_01:lustre-MDT0001-lwp-OST0001_UUID:3354:1862456998089920:0@lo:400:kworker/u8:1.0 00000100:00100000:1.0:1776178497.379914:0:3354:0:(events.c:361:request_in_callback()) peer: 12345-0@lo (source: 12345-0@lo) 00000100:00100000:1.0:1776178497.379963:0:3354:0:(ptlrpcd.c:415:ptlrpcd_check()) transfer 1 async RPCs [0->1] 00000100:00100000:1.0:1776178497.379970:0:3354:0:(client.c:1733:ptlrpc_send_new_req()) Sending RPC req@000000009b5eeec2 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_01:lustre-MDT0001-lwp-OST0000_UUID:3354:1862456998089792:0@lo:400:kworker/u8:1.0 00000100:00100000:1.0:1776178497.379996:0:3354:0:(events.c:361:request_in_callback()) peer: 12345-0@lo (source: 12345-0@lo) 00000100:00100000:3.0:1776178497.380233:0:3355:0:(client.c:1733:ptlrpc_send_new_req()) Sending RPC req@000000005f1c2da4 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_02:lustre-MDT0001-mdtlov_UUID:3355:1862456998090048:0@lo:400:kworker/u8:1.0 00000100:00100000:3.0:1776178497.380277:0:3355:0:(events.c:361:request_in_callback()) peer: 12345-0@lo (source: 12345-0@lo) 00000100:00100000:3.0:1776178497.380308:0:3355:0:(ptlrpcd.c:415:ptlrpcd_check()) transfer 2 async RPCs [3->2] 00000100:00100000:3.0:1776178497.380314:0:3355:0:(client.c:1733:ptlrpc_send_new_req()) Sending RPC req@000000006b9d8ccb pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_02:ac61f839-0e8e-4cac-9c59-08bcfbbad330:3355:1862456998089664:0@lo:400:kworker/u8:1.0 00000100:00100000:3.0:1776178497.380348:0:3355:0:(events.c:361:request_in_callback()) peer: 12345-0@lo (source: 12345-0@lo) 00000100:00100000:3.0:1776178497.380370:0:3355:0:(client.c:1733:ptlrpc_send_new_req()) Sending RPC req@00000000ad37fc5d pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_02:lustre-MDT0001-mdtlov_UUID:3355:1862456998090112:0@lo:400:kworker/u8:1.0 00000100:00100000:3.0:1776178497.380394:0:3355:0:(events.c:361:request_in_callback()) peer: 12345-0@lo (source: 12345-0@lo) 00000100:00100000:3.0:1776178497.380444:0:20140:0:(service.c:2152:ptlrpc_server_handle_req_in()) got req x1862456998089664 00000100:00100000:3.0:1776178497.380682:0:20140:0:(nrs_fifo.c:179:nrs_fifo_req_get()) NRS start fifo request from 12345-0@lo, seq: 270 00000100:00100000:3.0:1776178497.380690:0:20140:0:(service.c:2324:ptlrpc_server_handle_request()) Handling RPC req@000000004d415026 pname:cluuid+ref:pid:xid:nid:opc:job ll_mgs_0002:ac61f839-0e8e-4cac-9c59-08bcfbbad330+18:3355:x1862456998089664:12345-0@lo:400:kworker/u8:1.0 00000100:00100000:3.0:1776178497.380751:0:20140:0:(service.c:2372:ptlrpc_server_handle_request()) Handled RPC req@000000004d415026 pname:cluuid+ref:pid:xid:nid:opc:job ll_mgs_0002:ac61f839-0e8e-4cac-9c59-08bcfbbad330+18:3355:x1862456998089664:12345-0@lo:400:kworker/u8:1.0 Request processed in 60us (402us total) trans 0 rc 0/0 00000100:00100000:3.0:1776178497.380758:0:20140:0:(nrs_fifo.c:241:nrs_fifo_req_stop()) NRS stop fifo request from 12345-0@lo, seq: 270 00000100:00100000:3.0:1776178497.380813:0:3355:0:(client.c:2215:ptlrpc_check_set()) Completed RPC req@000000006b9d8ccb pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_02:ac61f839-0e8e-4cac-9c59-08bcfbbad330:3355:1862456998089664:0@lo:400:kworker/u8:1.0 00000100:00100000:0.0:1776178497.381091:0:3352:0:(client.c:1733:ptlrpc_send_new_req()) Sending RPC req@000000003095df64 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:lustre-MDT0000-lwp-OST0000_UUID:3352:1862456998089728:0@lo:38:kworker/u8:1.0 00000100:00100000:0.0:1776178497.381138:0:3352:0:(events.c:361:request_in_callback()) peer: 12345-0@lo (source: 12345-0@lo) 00000100:00100000:0.0:1776178497.381164:0:3352:0:(client.c:1733:ptlrpc_send_new_req()) Sending RPC req@000000009497b17e pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:lustre-MDT0000-lwp-OST0001_UUID:3352:1862456998089856:0@lo:38:kworker/u8:1.0 00000100:00100000:0.0:1776178497.381193:0:3352:0:(events.c:361:request_in_callback()) peer: 12345-0@lo (source: 12345-0@lo) 00000100:00100000:0.0:1776178497.381206:0:3352:0:(client.c:1733:ptlrpc_send_new_req()) Sending RPC req@000000000489d393 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:lustre-MDT0001-mdtlov_UUID:3352:1862456998089984:0@lo:38:kworker/u8:1.0 00000100:00100000:0.0:1776178497.381231:0:3352:0:(events.c:361:request_in_callback()) peer: 12345-0@lo (source: 12345-0@lo) 00000100:00100000:0.0:1776178497.381250:0:3352:0:(client.c:1733:ptlrpc_send_new_req()) Sending RPC req@000000004547bc6e pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:lustre-MDT0000-lwp-MDT0001_UUID:3352:1862456998090176:0@lo:38:kworker/u8:1.0 00000100:00100000:0.0:1776178497.381269:0:3352:0:(events.c:361:request_in_callback()) peer: 12345-0@lo (source: 12345-0@lo) 00000100:00100000:0.0:1776178497.381306:0:6256:0:(service.c:2152:ptlrpc_server_handle_req_in()) got req x1862456998089920 00000100:00100000:0.0:1776178497.381331:0:6256:0:(service.c:2152:ptlrpc_server_handle_req_in()) got req x1862456998089792 00000100:00100000:0.0:1776178497.381340:0:6256:0:(service.c:2152:ptlrpc_server_handle_req_in()) got req x1862456998089728 00000100:00100000:0.0:1776178497.381348:0:6256:0:(service.c:2152:ptlrpc_server_handle_req_in()) got req x1862456998089856 00000100:00100000:0.0:1776178497.381354:0:6256:0:(service.c:2152:ptlrpc_server_handle_req_in()) got req x1862456998090176 00000100:00100000:0.0:1776178497.381364:0:6256:0:(nrs_fifo.c:179:nrs_fifo_req_get()) NRS start fifo request from 12345-0@lo, seq: 1100 00000100:00100000:0.0:1776178497.381367:0:6256:0:(service.c:2324:ptlrpc_server_handle_request()) Handling RPC req@00000000049f8cf3 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_001:lustre-MDT0001-lwp-OST0001_UUID+5:3354:x1862456998089920:12345-0@lo:400:kworker/u8:1.0 00000100:00100000:0.0:1776178497.381406:0:6256:0:(service.c:2372:ptlrpc_server_handle_request()) Handled RPC req@00000000049f8cf3 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_001:lustre-MDT0001-lwp-OST0001_UUID+5:3354:x1862456998089920:12345-0@lo:400:kworker/u8:1.0 Request processed in 38us (1496us total) trans 0 rc 0/0 00000100:00100000:0.0:1776178497.381415:0:6256:0:(nrs_fifo.c:241:nrs_fifo_req_stop()) NRS stop fifo request from 12345-0@lo, seq: 1100 00000100:00100000:0.0:1776178497.381427:0:6256:0:(nrs_fifo.c:179:nrs_fifo_req_get()) NRS start fifo request from 12345-0@lo, seq: 1101 00000100:00100000:0.0:1776178497.381431:0:6256:0:(service.c:2324:ptlrpc_server_handle_request()) Handling RPC req@000000006f0aa048 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_001:lustre-MDT0001-lwp-OST0000_UUID+5:3354:x1862456998089792:12345-0@lo:400:kworker/u8:1.0 00000100:00100000:0.0:1776178497.381451:0:6256:0:(service.c:2372:ptlrpc_server_handle_request()) Handled RPC req@000000006f0aa048 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_001:lustre-MDT0001-lwp-OST0000_UUID+5:3354:x1862456998089792:12345-0@lo:400:kworker/u8:1.0 Request processed in 20us (1456us total) trans 0 rc 0/0 00000100:00100000:0.0:1776178497.381456:0:6256:0:(nrs_fifo.c:241:nrs_fifo_req_stop()) NRS stop fifo request from 12345-0@lo, seq: 1101 00000100:00100000:0.0:1776178497.381466:0:6256:0:(nrs_fifo.c:179:nrs_fifo_req_get()) NRS start fifo request from 12345-0@lo, seq: 6853 00000100:00100000:0.0:1776178497.381470:0:6256:0:(service.c:2324:ptlrpc_server_handle_request()) Handling RPC req@000000006a3d91b0 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_001:0+-99:3352:x1862456998089728:12345-0@lo:38:kworker/u8:1.0 00010000:02000000:0.0:1776178497.381487:0:6256:0:(ldlm_lib.c:1139:target_handle_connect()) lustre-MDT0000: Not available for connect from 0@lo (stopping) 00000100:00100000:2.0:1776178497.382211:0:6257:0:(nrs_fifo.c:179:nrs_fifo_req_get()) NRS start fifo request from 12345-0@lo, seq: 6854 00000100:00100000:2.0:1776178497.382218:0:6257:0:(service.c:2324:ptlrpc_server_handle_request()) Handling RPC req@0000000004dd3b9b pname:cluuid+ref:pid:xid:nid:opc:job mdt00_002:0+-99:3352:x1862456998089856:12345-0@lo:38:kworker/u8:1.0 00010000:02000000:2.0:1776178497.382230:0:6257:0:(ldlm_lib.c:1139:target_handle_connect()) lustre-MDT0000: Not available for connect from 0@lo (stopping) 00000100:00100000:2.0:1776178497.382286:0:6257:0:(service.c:2372:ptlrpc_server_handle_request()) Handled RPC req@0000000004dd3b9b pname:cluuid+ref:pid:xid:nid:opc:job mdt00_002:0+-99:3352:x1862456998089856:12345-0@lo:38:kworker/u8:1.0 Request processed in 66us (1091us total) trans 0 rc -19/-19 00000100:00100000:2.0:1776178497.382291:0:6257:0:(nrs_fifo.c:241:nrs_fifo_req_stop()) NRS stop fifo request from 12345-0@lo, seq: 6854 00000100:00100000:2.0:1776178497.382308:0:6257:0:(nrs_fifo.c:179:nrs_fifo_req_get()) NRS start fifo request from 12345-0@lo, seq: 6855 00000100:00100000:2.0:1776178497.382312:0:6257:0:(service.c:2324:ptlrpc_server_handle_request()) Handling RPC req@000000002300ff10 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_002:0+-99:3352:x1862456998090176:12345-0@lo:38:kworker/u8:1.0 00010000:02000000:2.0:1776178497.382318:0:6257:0:(ldlm_lib.c:1139:target_handle_connect()) lustre-MDT0000: Not available for connect from 0@lo (stopping) 00000100:00100000:2.0:1776178497.382340:0:6257:0:(service.c:2372:ptlrpc_server_handle_request()) Handled RPC req@000000002300ff10 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_002:0+-99:3352:x1862456998090176:12345-0@lo:38:kworker/u8:1.0 Request processed in 28us (1070us total) trans 0 rc -19/-19 00000100:00100000:2.0:1776178497.382346:0:6257:0:(nrs_fifo.c:241:nrs_fifo_req_stop()) NRS stop fifo request from 12345-0@lo, seq: 6855 00000100:00100000:2.0:1776178497.382371:0:20755:0:(service.c:2152:ptlrpc_server_handle_req_in()) got req x1862456998090048 00000100:00100000:2.0:1776178497.382406:0:20755:0:(service.c:2152:ptlrpc_server_handle_req_in()) got req x1862456998090112 00000100:00100000:2.0:1776178497.382426:0:20755:0:(nrs_fifo.c:179:nrs_fifo_req_get()) NRS start fifo request from 12345-0@lo, seq: 741 00000100:00100000:2.0:1776178497.382429:0:20755:0:(service.c:2324:ptlrpc_server_handle_request()) Handling RPC req@0000000037059d8f pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_006:lustre-MDT0001-mdtlov_UUID+5:3355:x1862456998090048:12345-0@lo:400:kworker/u8:1.0 00000100:00100000:2.0:1776178497.382467:0:20755:0:(service.c:2372:ptlrpc_server_handle_request()) Handled RPC req@0000000037059d8f pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_006:lustre-MDT0001-mdtlov_UUID+5:3355:x1862456998090048:12345-0@lo:400:kworker/u8:1.0 Request processed in 37us (2190us total) trans 0 rc 0/0 00000100:00100000:2.0:1776178497.382472:0:20755:0:(nrs_fifo.c:241:nrs_fifo_req_stop()) NRS stop fifo request from 12345-0@lo, seq: 741 00000100:00100000:2.0:1776178497.382840:0:20755:0:(nrs_fifo.c:179:nrs_fifo_req_get()) NRS start fifo request from 12345-0@lo, seq: 742 00000100:00100000:2.0:1776178497.382845:0:20755:0:(service.c:2324:ptlrpc_server_handle_request()) Handling RPC req@00000000664eb80b pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_006:lustre-MDT0001-mdtlov_UUID+5:3355:x1862456998090112:12345-0@lo:400:kworker/u8:1.0 00000100:00100000:2.0:1776178497.382879:0:20755:0:(service.c:2372:ptlrpc_server_handle_request()) Handled RPC req@00000000664eb80b pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_006:lustre-MDT0001-mdtlov_UUID+5:3355:x1862456998090112:12345-0@lo:400:kworker/u8:1.0 Request processed in 33us (2484us total) trans 0 rc 0/0 00000100:00100000:2.0:1776178497.382884:0:20755:0:(nrs_fifo.c:241:nrs_fifo_req_stop()) NRS stop fifo request from 12345-0@lo, seq: 742 00000100:00080000:2.0:1776178497.384334:0:3352:0:(import.c:86:import_set_state_nolock()) 00000000dfc8db13 lustre-MDT0000_UUID: changing import state from CONNECTING to DISCONN 00000100:00080000:2.0:1776178497.384348:0:3352:0:(import.c:1429:ptlrpc_connect_interpret()) recovery of lustre-MDT0000_UUID on 192.168.204.103@tcp failed (-19) 00000100:00100000:2.0:1776178497.384360:0:3352:0:(client.c:2215:ptlrpc_check_set()) Completed RPC req@000000009497b17e pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:lustre-MDT0000-lwp-OST0001_UUID:3352:1862456998089856:0@lo:38:kworker/u8:1.0 00000100:00080000:2.0:1776178497.384399:0:3352:0:(import.c:86:import_set_state_nolock()) 00000000a7c55b1f lustre-MDT0000_UUID: changing import state from CONNECTING to DISCONN 00000100:00080000:2.0:1776178497.384404:0:3352:0:(import.c:1429:ptlrpc_connect_interpret()) recovery of lustre-MDT0000_UUID on 192.168.204.103@tcp failed (-19) 00000100:00100000:2.0:1776178497.384408:0:3352:0:(client.c:2215:ptlrpc_check_set()) Completed RPC req@000000004547bc6e pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:lustre-MDT0000-lwp-MDT0001_UUID:3352:1862456998090176:0@lo:38:kworker/u8:1.0 00000100:00100000:3.0:1776178497.384702:0:3355:0:(client.c:2215:ptlrpc_check_set()) Completed RPC req@00000000ad37fc5d pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_02:lustre-MDT0001-mdtlov_UUID:3355:1862456998090112:0@lo:400:kworker/u8:1.0 00000100:00100000:3.0:1776178497.384724:0:3355:0:(client.c:2215:ptlrpc_check_set()) Completed RPC req@000000005f1c2da4 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_02:lustre-MDT0001-mdtlov_UUID:3355:1862456998090048:0@lo:400:kworker/u8:1.0 00000100:00100000:1.0:1776178497.384738:0:3354:0:(client.c:2215:ptlrpc_check_set()) Completed RPC req@000000009b5eeec2 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_01:lustre-MDT0001-lwp-OST0000_UUID:3354:1862456998089792:0@lo:400:kworker/u8:1.0 00000100:00100000:1.0:1776178497.384758:0:3354:0:(client.c:2215:ptlrpc_check_set()) Completed RPC req@000000001d2ef500 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_01:lustre-MDT0001-lwp-OST0001_UUID:3354:1862456998089920:0@lo:400:kworker/u8:1.0 00000100:00100000:2.0:1776178497.386029:0:15213:0:(service.c:2152:ptlrpc_server_handle_req_in()) got req x1862456998089984 00000100:00100000:2.0:1776178497.386167:0:15213:0:(nrs_fifo.c:179:nrs_fifo_req_get()) NRS start fifo request from 12345-0@lo, seq: 765 00000100:00100000:2.0:1776178497.386176:0:15213:0:(service.c:2324:ptlrpc_server_handle_request()) Handling RPC req@00000000d0d837c5 pname:cluuid+ref:pid:xid:nid:opc:job mdt_out00_003:0+-99:3352:x1862456998089984:12345-0@lo:38:kworker/u8:1.0 00010000:02000000:2.0:1776178497.386212:0:15213:0:(ldlm_lib.c:1139:target_handle_connect()) lustre-MDT0000: Not available for connect from 0@lo (stopping) 00000100:00100000:2.0:1776178497.388227:0:15213:0:(service.c:2372:ptlrpc_server_handle_request()) Handled RPC req@00000000d0d837c5 pname:cluuid+ref:pid:xid:nid:opc:job mdt_out00_003:0+-99:3352:x1862456998089984:12345-0@lo:38:kworker/u8:1.0 Request processed in 2031us (6974us total) trans 0 rc -19/-19 00000100:00100000:2.0:1776178497.388257:0:15213:0:(nrs_fifo.c:241:nrs_fifo_req_stop()) NRS stop fifo request from 12345-0@lo, seq: 765 00000100:00080000:1.0:1776178497.389005:0:3352:0:(import.c:86:import_set_state_nolock()) 00000000ba4f1821 lustre-MDT0000_UUID: changing import state from CONNECTING to DISCONN 00000100:00080000:1.0:1776178497.389013:0:3352:0:(import.c:1429:ptlrpc_connect_interpret()) recovery of lustre-MDT0000_UUID on 192.168.204.103@tcp failed (-19) 00000100:00100000:1.0:1776178497.389021:0:3352:0:(client.c:2215:ptlrpc_check_set()) Completed RPC req@000000000489d393 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:lustre-MDT0001-mdtlov_UUID:3352:1862456998089984:0@lo:38:kworker/u8:1.0 00000100:00100000:0.0:1776178497.408403:0:6256:0:(service.c:2372:ptlrpc_server_handle_request()) Handled RPC req@000000006a3d91b0 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_001:0+-99:3352:x1862456998089728:12345-0@lo:38:kworker/u8:1.0 Request processed in 26920us (27252us total) trans 0 rc -19/-19 00000100:00100000:0.0:1776178497.408423:0:6256:0:(nrs_fifo.c:241:nrs_fifo_req_stop()) NRS stop fifo request from 12345-0@lo, seq: 6853 00000100:00080000:2.0:1776178497.408518:0:3352:0:(import.c:86:import_set_state_nolock()) 0000000047601a98 lustre-MDT0000_UUID: changing import state from CONNECTING to DISCONN 00000100:00080000:2.0:1776178497.408528:0:3352:0:(import.c:1429:ptlrpc_connect_interpret()) recovery of lustre-MDT0000_UUID on 192.168.204.103@tcp failed (-19) 00000100:00100000:2.0:1776178497.408538:0:3352:0:(client.c:2215:ptlrpc_check_set()) Completed RPC req@000000003095df64 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:lustre-MDT0000-lwp-OST0000_UUID:3352:1862456998089728:0@lo:38:kworker/u8:1.0 00010000:00010000:1.0:1776178498.017995:0:9224:0:(ldlm_lockd.c:2792:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:0.0:1776178498.912364:0:3353:0:(client.c:1733:ptlrpc_send_new_req()) Sending RPC req@00000000d28be450 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_00:lustre-MDT0001-mdtlov_UUID:3353:1862456998090240:0@lo:13:osp-pre-1-1.0 00000100:00100000:0.0:1776178498.912461:0:3353:0:(events.c:361:request_in_callback()) peer: 12345-0@lo (source: 12345-0@lo) 00000100:00100000:3.0:1776178498.912664:0:8231:0:(service.c:2152:ptlrpc_server_handle_req_in()) got req x1862456998090240 00000100:00100000:3.0:1776178498.912730:0:8231:0:(nrs_fifo.c:179:nrs_fifo_req_get()) NRS start fifo request from 12345-0@lo, seq: 603 00000100:00100000:3.0:1776178498.912738:0:8231:0:(service.c:2324:ptlrpc_server_handle_request()) Handling RPC req@000000007216887e pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_create00:lustre-MDT0001-mdtlov_UUID+5:3353:x1862456998090240:12345-0@lo:13:osp-pre-1-1.0 00002000:00000024:3.0:1776178498.912776:0:8231:0:(ofd_obd.c:740:ofd_statfs()) blocks cached 229376 granted 491746368 pending 0 free 3923689472 avail 3692167168 00000100:00100000:3.0:1776178498.912837:0:8231:0:(service.c:2372:ptlrpc_server_handle_request()) Handled RPC req@000000007216887e pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_create00:lustre-MDT0001-mdtlov_UUID+5:3353:x1862456998090240:12345-0@lo:13:osp-pre-1-1.0 Request processed in 99us (388us total) trans 0 rc 0/0 00000100:00100000:3.0:1776178498.912844:0:8231:0:(nrs_fifo.c:241:nrs_fifo_req_stop()) NRS stop fifo request from 12345-0@lo, seq: 603 00000100:00100000:0.0:1776178498.913284:0:3353:0:(client.c:2215:ptlrpc_check_set()) Completed RPC req@00000000d28be450 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_00:lustre-MDT0001-mdtlov_UUID:3353:1862456998090240:0@lo:13:osp-pre-1-1.0 00010000:00010000:1.0:1776178499.045982:0:9225:0:(ldlm_lockd.c:2792:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:0.0:1776178499.426592:0:3353:0:(ptlrpcd.c:415:ptlrpcd_check()) transfer 1 async RPCs [1->0] 00000100:00100000:0.0:1776178499.426619:0:3353:0:(client.c:1733:ptlrpc_send_new_req()) Sending RPC req@00000000d4b41d38 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_00:lustre-MDT0001-mdtlov_UUID:3353:1862456998090304:0@lo:13:osp-pre-0-1.0 00000100:00100000:0.0:1776178499.426712:0:3353:0:(events.c:361:request_in_callback()) peer: 12345-0@lo (source: 12345-0@lo) 00000100:00100000:0.0:1776178499.426766:0:9554:0:(service.c:2152:ptlrpc_server_handle_req_in()) got req x1862456998090304 00000100:00100000:0.0:1776178499.426813:0:9554:0:(nrs_fifo.c:179:nrs_fifo_req_get()) NRS start fifo request from 12345-0@lo, seq: 604 00000100:00100000:0.0:1776178499.426820:0:9554:0:(service.c:2324:ptlrpc_server_handle_request()) Handling RPC req@00000000606788fe pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_create00:lustre-MDT0001-mdtlov_UUID+5:3353:x1862456998090304:12345-0@lo:13:osp-pre-0-1.0 00002000:00000024:0.0:1776178499.426846:0:9554:0:(ofd_obd.c:740:ofd_statfs()) blocks cached 315392 granted 484883776 pending 0 free 3923689472 avail 3692167168 00000100:00100000:0.0:1776178499.426883:0:9554:0:(service.c:2372:ptlrpc_server_handle_request()) Handled RPC req@00000000606788fe pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_create00:lustre-MDT0001-mdtlov_UUID+5:3353:x1862456998090304:12345-0@lo:13:osp-pre-0-1.0 Request processed in 63us (173us total) trans 0 rc 0/0 00000100:00100000:0.0:1776178499.426895:0:9554:0:(nrs_fifo.c:241:nrs_fifo_req_stop()) NRS stop fifo request from 12345-0@lo, seq: 604 00000100:00100000:0.0:1776178499.426941:0:3353:0:(client.c:2215:ptlrpc_check_set()) Completed RPC req@00000000d4b41d38 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_00:lustre-MDT0001-mdtlov_UUID:3353:1862456998090304:0@lo:13:osp-pre-0-1.0 00010000:00010000:3.0:1776178500.064470:0:6244:0:(ldlm_lockd.c:2792:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:1.0:1776178500.906210:0:3347:0:(events.c:361:request_in_callback()) peer: 12345-192.168.204.3@tcp (source: 12345-192.168.204.3@tcp) 00000100:00100000:1.0:1776178500.906275:0:3347:0:(events.c:361:request_in_callback()) peer: 12345-192.168.204.3@tcp (source: 12345-192.168.204.3@tcp) 00000100:00100000:1.0:1776178500.906390:0:3347:0:(events.c:361:request_in_callback()) peer: 12345-192.168.204.3@tcp (source: 12345-192.168.204.3@tcp) 00000100:00100000:1.0:1776178500.906421:0:3347:0:(events.c:361:request_in_callback()) peer: 12345-192.168.204.3@tcp (source: 12345-192.168.204.3@tcp) 00000100:00100000:0.0:1776178500.906515:0:6256:0:(service.c:2152:ptlrpc_server_handle_req_in()) got req x1862456979127680 00000100:00100000:0.0:1776178500.906609:0:6256:0:(nrs_fifo.c:179:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.204.3@tcp, seq: 1102 00000100:00100000:0.0:1776178500.906619:0:6256:0:(service.c:2324:ptlrpc_server_handle_request()) Handling RPC req@0000000055f08c84 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_001:b50963bd-8eba-4432-9a54-8888d00e3fa1+6:2272:x1862456979127680:12345-192.168.204.3@tcp:400:kworker/u8:4.0 00000100:00100000:0.0:1776178500.906732:0:6256:0:(service.c:2372:ptlrpc_server_handle_request()) Handled RPC req@0000000055f08c84 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_001:b50963bd-8eba-4432-9a54-8888d00e3fa1+6:2272:x1862456979127680:12345-192.168.204.3@tcp:400:kworker/u8:4.0 Request processed in 113us (343us total) trans 0 rc 0/0 00000100:00100000:0.0:1776178500.906741:0:6256:0:(nrs_fifo.c:241:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.204.3@tcp, seq: 1102 00000100:00100000:2.0:1776178500.909959:0:20137:0:(service.c:2152:ptlrpc_server_handle_req_in()) got req x1862456979127552 00000100:00100000:2.0:1776178500.910120:0:20137:0:(nrs_fifo.c:179:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.204.3@tcp, seq: 271 00000100:00100000:2.0:1776178500.910130:0:20137:0:(service.c:2324:ptlrpc_server_handle_request()) Handling RPC req@00000000d1b75234 pname:cluuid+ref:pid:xid:nid:opc:job ll_mgs_0000:5b5c66b1-1a2f-4651-8548-0c75b9a38a94+9:2272:x1862456979127552:12345-192.168.204.3@tcp:400:kworker/u8:4.0 00000100:00100000:2.0:1776178500.910259:0:20137:0:(service.c:2372:ptlrpc_server_handle_request()) Handled RPC req@00000000d1b75234 pname:cluuid+ref:pid:xid:nid:opc:job ll_mgs_0000:5b5c66b1-1a2f-4651-8548-0c75b9a38a94+9:2272:x1862456979127552:12345-192.168.204.3@tcp:400:kworker/u8:4.0 Request processed in 130us (4061us total) trans 0 rc 0/0 00000100:00100000:2.0:1776178500.910268:0:20137:0:(nrs_fifo.c:241:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.204.3@tcp, seq: 271 00000100:00100000:1.0:1776178500.915382:0:3347:0:(events.c:361:request_in_callback()) peer: 12345-192.168.204.3@tcp (source: 12345-192.168.204.3@tcp) 00000100:00100000:3.0:1776178500.915520:0:14440:0:(service.c:2152:ptlrpc_server_handle_req_in()) got req x1862456979127744 00000100:00100000:0.0:1776178500.919558:0:20759:0:(service.c:2152:ptlrpc_server_handle_req_in()) got req x1862456979127808 00000100:00100000:0.0:1776178500.919684:0:20759:0:(nrs_fifo.c:179:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.204.3@tcp, seq: 743 00000100:00100000:0.0:1776178500.919694:0:20759:0:(service.c:2324:ptlrpc_server_handle_request()) Handling RPC req@0000000034d11618 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_010:b50963bd-8eba-4432-9a54-8888d00e3fa1+9:2272:x1862456979127808:12345-192.168.204.3@tcp:400:kworker/u8:4.0 00000100:00100000:0.0:1776178500.919908:0:20759:0:(service.c:2372:ptlrpc_server_handle_request()) Handled RPC req@0000000034d11618 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_010:b50963bd-8eba-4432-9a54-8888d00e3fa1+9:2272:x1862456979127808:12345-192.168.204.3@tcp:400:kworker/u8:4.0 Request processed in 214us (13486us total) trans 0 rc 0/0 00000100:00100000:0.0:1776178500.919917:0:20759:0:(nrs_fifo.c:241:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.204.3@tcp, seq: 743 00000100:00100000:2.0:1776178500.920392:0:8250:0:(service.c:2152:ptlrpc_server_handle_req_in()) got req x1862456979127616 00000100:00100000:2.0:1776178500.920427:0:8250:0:(nrs_fifo.c:179:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.204.3@tcp, seq: 6856 00000100:00100000:2.0:1776178500.920432:0:8250:0:(service.c:2324:ptlrpc_server_handle_request()) Handling RPC req@00000000ac4ee3ce pname:cluuid+ref:pid:xid:nid:opc:job mdt00_003:0+-99:2269:x1862456979127616:12345-192.168.204.3@tcp:38:kworker/u8:4.0 00010000:02000000:2.0:1776178500.920455:0:8250:0:(ldlm_lib.c:1139:target_handle_connect()) lustre-MDT0000: Not available for connect from 192.168.204.3@tcp (stopping) 00000100:00100000:2.0:1776178500.920506:0:8250:0:(service.c:2372:ptlrpc_server_handle_request()) Handled RPC req@00000000ac4ee3ce pname:cluuid+ref:pid:xid:nid:opc:job mdt00_003:0+-99:2269:x1862456979127616:12345-192.168.204.3@tcp:38:kworker/u8:4.0 Request processed in 72us (5141us total) trans 0 rc -19/-19 00000100:00100000:2.0:1776178500.920513:0:8250:0:(nrs_fifo.c:241:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.204.3@tcp, seq: 6856 00000100:00100000:3.0:1776178500.920561:0:14440:0:(nrs_fifo.c:179:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.204.3@tcp, seq: 744 00000100:00100000:3.0:1776178500.920567:0:14440:0:(service.c:2324:ptlrpc_server_handle_request()) Handling RPC req@0000000072c0bce9 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_004:b50963bd-8eba-4432-9a54-8888d00e3fa1+8:2272:x1862456979127744:12345-192.168.204.3@tcp:400:kworker/u8:4.0 00000100:00100000:3.0:1776178500.920603:0:14440:0:(service.c:2372:ptlrpc_server_handle_request()) Handled RPC req@0000000072c0bce9 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_004:b50963bd-8eba-4432-9a54-8888d00e3fa1+8:2272:x1862456979127744:12345-192.168.204.3@tcp:400:kworker/u8:4.0 Request processed in 36us (14329us total) trans 0 rc 0/0 00000100:00100000:3.0:1776178500.920615:0:14440:0:(nrs_fifo.c:241:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.204.3@tcp, seq: 744 00010000:00010000:2.0:1776178501.090705:0:6245:0:(ldlm_lockd.c:2792:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00010000:00010000:1.0:1776178502.112368:0:9224:0:(ldlm_lockd.c:2792:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:3.0:1776178502.497031:0:3355:0:(client.c:1733:ptlrpc_send_new_req()) Sending RPC req@0000000057946847 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_02:ac61f839-0e8e-4cac-9c59-08bcfbbad330:3355:1862456998090368:0@lo:400:kworker/u8:2.0 00000100:00100000:3.0:1776178502.497327:0:3355:0:(events.c:361:request_in_callback()) peer: 12345-0@lo (source: 12345-0@lo) 00000100:00080000:2.0:1776178502.497422:0:37:0:(pinger.c:251:ptlrpc_pinger_process_import()) lustre-MDT0000-lwp-OST0000_UUID->lustre-MDT0000_UUID: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00080000:2.0:1776178502.497429:0:37:0:(recover.c:57:ptlrpc_initiate_recovery()) lustre-MDT0000_UUID: starting recovery 00000100:00080000:2.0:1776178502.497441:0:37:0:(import.c:86:import_set_state_nolock()) 0000000047601a98 lustre-MDT0000_UUID: changing import state from DISCONN to CONNECTING 00000100:00080000:2.0:1776178502.497448:0:37:0:(import.c:534:import_select_connection()) lustre-MDT0000-lwp-OST0000: connect to NID 0@lo last attempt 1204 00000100:00080000:2.0:1776178502.497451:0:37:0:(import.c:581:import_select_connection()) lustre-MDT0000-lwp-OST0000: tried all connections, increasing latency to 21s 00000100:00080000:2.0:1776178502.497458:0:37:0:(import.c:615:import_select_connection()) lustre-MDT0000-lwp-OST0000: import 0000000047601a98 using connection 192.168.204.103@tcp/0@lo 00000100:00100000:2.0:1776178502.497483:0:37:0:(import.c:807:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5) req@0000000007e413b7 x1862456998090432/t0(0) o38->lustre-MDT0000-lwp-OST0000@0@lo:12/10 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' 00000100:00100000:0.0:1776178502.497570:0:20140:0:(service.c:2152:ptlrpc_server_handle_req_in()) got req x1862456998090368 00000100:00100000:0.0:1776178502.497623:0:20140:0:(nrs_fifo.c:179:nrs_fifo_req_get()) NRS start fifo request from 12345-0@lo, seq: 272 00000100:00100000:0.0:1776178502.497629:0:20140:0:(service.c:2324:ptlrpc_server_handle_request()) Handling RPC req@00000000e22b0439 pname:cluuid+ref:pid:xid:nid:opc:job ll_mgs_0002:ac61f839-0e8e-4cac-9c59-08bcfbbad330+18:3355:x1862456998090368:12345-0@lo:400:kworker/u8:2.0 00000100:00100000:1.0:1776178502.497832:0:3352:0:(client.c:1733:ptlrpc_send_new_req()) Sending RPC req@0000000007e413b7 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:lustre-MDT0000-lwp-OST0000_UUID:3352:1862456998090432:0@lo:38:kworker/u8:2.0 00000100:00100000:1.0:1776178502.497872:0:3352:0:(events.c:361:request_in_callback()) peer: 12345-0@lo (source: 12345-0@lo) 00000100:00100000:1.0:1776178502.497908:0:3356:0:(client.c:1733:ptlrpc_send_new_req()) Sending RPC req@000000007309261c pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_03:lustre-MDT0001-lwp-OST0000_UUID:3356:1862456998090496:0@lo:400:kworker/u8:2.0 00000100:00100000:1.0:1776178502.497931:0:3356:0:(events.c:361:request_in_callback()) peer: 12345-0@lo (source: 12345-0@lo) 00000100:00100000:1.0:1776178502.497964:0:6256:0:(service.c:2152:ptlrpc_server_handle_req_in()) got req x1862456998090432 00000100:00100000:1.0:1776178502.497991:0:6255:0:(service.c:2152:ptlrpc_server_handle_req_in()) got req x1862456998090496 00000100:00100000:1.0:1776178502.498029:0:6255:0:(nrs_fifo.c:179:nrs_fifo_req_get()) NRS start fifo request from 12345-0@lo, seq: 1103 00000100:00100000:1.0:1776178502.498033:0:6255:0:(service.c:2324:ptlrpc_server_handle_request()) Handling RPC req@00000000752f12be pname:cluuid+ref:pid:xid:nid:opc:job mdt00_000:lustre-MDT0001-lwp-OST0000_UUID+5:3356:x1862456998090496:12345-0@lo:400:kworker/u8:2.0 00000100:00100000:1.0:1776178502.498102:0:6255:0:(service.c:2372:ptlrpc_server_handle_request()) Handled RPC req@00000000752f12be pname:cluuid+ref:pid:xid:nid:opc:job mdt00_000:lustre-MDT0001-lwp-OST0000_UUID+5:3356:x1862456998090496:12345-0@lo:400:kworker/u8:2.0 Request processed in 67us (170us total) trans 0 rc 0/0 00000100:00100000:1.0:1776178502.498110:0:6255:0:(nrs_fifo.c:241:nrs_fifo_req_stop()) NRS stop fifo request from 12345-0@lo, seq: 1103 00000100:00100000:1.0:1776178502.498127:0:6255:0:(nrs_fifo.c:179:nrs_fifo_req_get()) NRS start fifo request from 12345-0@lo, seq: 6857 00000100:00100000:1.0:1776178502.498131:0:6255:0:(service.c:2324:ptlrpc_server_handle_request()) Handling RPC req@00000000b70c4784 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_000:0+-99:3352:x1862456998090432:12345-0@lo:38:kworker/u8:2.0 00010000:02000000:1.0:1776178502.498154:0:6255:0:(ldlm_lib.c:1139:target_handle_connect()) lustre-MDT0000: Not available for connect from 0@lo (stopping) 00000100:00100000:1.0:1776178502.498186:0:6255:0:(service.c:2372:ptlrpc_server_handle_request()) Handled RPC req@00000000b70c4784 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_000:0+-99:3352:x1862456998090432:12345-0@lo:38:kworker/u8:2.0 Request processed in 54us (314us total) trans 0 rc -19/-19 00000100:00100000:1.0:1776178502.498192:0:6255:0:(nrs_fifo.c:241:nrs_fifo_req_stop()) NRS stop fifo request from 12345-0@lo, seq: 6857 00000100:00100000:1.0:1776178502.498236:0:3356:0:(client.c:2215:ptlrpc_check_set()) Completed RPC req@000000007309261c pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_03:lustre-MDT0001-lwp-OST0000_UUID:3356:1862456998090496:0@lo:400:kworker/u8:2.0 00000100:00080000:1.0:1776178502.498263:0:3352:0:(import.c:86:import_set_state_nolock()) 0000000047601a98 lustre-MDT0000_UUID: changing import state from CONNECTING to DISCONN 00000100:00080000:1.0:1776178502.498267:0:3352:0:(import.c:1429:ptlrpc_connect_interpret()) recovery of lustre-MDT0000_UUID on 192.168.204.103@tcp failed (-19) 00000100:00100000:1.0:1776178502.498270:0:3352:0:(client.c:2215:ptlrpc_check_set()) Completed RPC req@0000000007e413b7 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:lustre-MDT0000-lwp-OST0000_UUID:3352:1862456998090432:0@lo:38:kworker/u8:2.0 00000100:00080000:2.0:1776178502.504457:0:37:0:(pinger.c:251:ptlrpc_pinger_process_import()) lustre-MDT0000-lwp-OST0001_UUID->lustre-MDT0000_UUID: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00080000:2.0:1776178502.504474:0:37:0:(recover.c:57:ptlrpc_initiate_recovery()) lustre-MDT0000_UUID: starting recovery 00000100:00080000:2.0:1776178502.504479:0:37:0:(import.c:86:import_set_state_nolock()) 00000000dfc8db13 lustre-MDT0000_UUID: changing import state from DISCONN to CONNECTING 00000100:00080000:2.0:1776178502.504490:0:37:0:(import.c:534:import_select_connection()) lustre-MDT0000-lwp-OST0001: connect to NID 0@lo last attempt 1204 00000100:00080000:2.0:1776178502.504495:0:37:0:(import.c:581:import_select_connection()) lustre-MDT0000-lwp-OST0001: tried all connections, increasing latency to 21s 00000100:00080000:2.0:1776178502.504503:0:37:0:(import.c:615:import_select_connection()) lustre-MDT0000-lwp-OST0001: import 00000000dfc8db13 using connection 192.168.204.103@tcp/0@lo 00000100:00100000:2.0:1776178502.504554:0:37:0:(import.c:807:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5) req@0000000007e413b7 x1862456998090560/t0(0) o38->lustre-MDT0000-lwp-OST0001@0@lo:12/10 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' 00000100:00080000:2.0:1776178502.504626:0:37:0:(pinger.c:251:ptlrpc_pinger_process_import()) lustre-MDT0001-mdtlov_UUID->lustre-MDT0000_UUID: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00080000:2.0:1776178502.504630:0:37:0:(recover.c:57:ptlrpc_initiate_recovery()) lustre-MDT0000_UUID: starting recovery 00000100:00080000:2.0:1776178502.504631:0:37:0:(import.c:86:import_set_state_nolock()) 00000000ba4f1821 lustre-MDT0000_UUID: changing import state from DISCONN to CONNECTING 00000100:00080000:2.0:1776178502.504634:0:37:0:(import.c:534:import_select_connection()) lustre-MDT0000-osp-MDT0001: connect to NID 0@lo last attempt 1204 00000100:00080000:2.0:1776178502.504636:0:37:0:(import.c:581:import_select_connection()) lustre-MDT0000-osp-MDT0001: tried all connections, increasing latency to 21s 00000100:00080000:2.0:1776178502.504639:0:37:0:(import.c:615:import_select_connection()) lustre-MDT0000-osp-MDT0001: import 00000000ba4f1821 using connection 192.168.204.103@tcp/0@lo 00000100:00100000:2.0:1776178502.504655:0:37:0:(import.c:807:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5) req@000000005f1c2da4 x1862456998090688/t0(0) o38->lustre-MDT0000-osp-MDT0001@0@lo:24/4 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' 00000100:00080000:2.0:1776178502.504683:0:37:0:(pinger.c:251:ptlrpc_pinger_process_import()) lustre-MDT0000-lwp-MDT0001_UUID->lustre-MDT0000_UUID: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00080000:2.0:1776178502.504686:0:37:0:(recover.c:57:ptlrpc_initiate_recovery()) lustre-MDT0000_UUID: starting recovery 00000100:00080000:2.0:1776178502.504687:0:37:0:(import.c:86:import_set_state_nolock()) 00000000a7c55b1f lustre-MDT0000_UUID: changing import state from DISCONN to CONNECTING 00000100:00080000:2.0:1776178502.504690:0:37:0:(import.c:534:import_select_connection()) lustre-MDT0000-lwp-MDT0001: connect to NID 0@lo last attempt 1204 00000100:00080000:2.0:1776178502.504692:0:37:0:(import.c:581:import_select_connection()) lustre-MDT0000-lwp-MDT0001: tried all connections, increasing latency to 21s 00000100:00080000:2.0:1776178502.504695:0:37:0:(import.c:615:import_select_connection()) lustre-MDT0000-lwp-MDT0001: import 00000000a7c55b1f using connection 192.168.204.103@tcp/0@lo 00000100:00100000:2.0:1776178502.504705:0:37:0:(import.c:807:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5) req@000000000489d393 x1862456998090880/t0(0) o38->lustre-MDT0000-lwp-MDT0001@0@lo:12/10 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' 00000100:00100000:2.0:1776178502.504754:0:3354:0:(client.c:1733:ptlrpc_send_new_req()) Sending RPC req@00000000ad37fc5d pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_01:lustre-MDT0001-mdtlov_UUID:3354:1862456998090752:0@lo:400:kworker/u8:2.0 00000100:00100000:2.0:1776178502.504838:0:3354:0:(events.c:361:request_in_callback()) peer: 12345-0@lo (source: 12345-0@lo) 00000100:00100000:2.0:1776178502.504878:0:3354:0:(ptlrpcd.c:415:ptlrpcd_check()) transfer 1 async RPCs [0->1] 00000100:00100000:2.0:1776178502.504884:0:3354:0:(client.c:1733:ptlrpc_send_new_req()) Sending RPC req@000000007309261c pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_01:lustre-MDT0001-lwp-OST0001_UUID:3354:1862456998090624:0@lo:400:kworker/u8:2.0 00000100:00100000:2.0:1776178502.504910:0:3354:0:(events.c:361:request_in_callback()) peer: 12345-0@lo (source: 12345-0@lo) 00000100:00100000:2.0:1776178502.504948:0:11281:0:(service.c:2152:ptlrpc_server_handle_req_in()) got req x1862456998090624 00000100:00100000:2.0:1776178502.505000:0:11281:0:(nrs_fifo.c:179:nrs_fifo_req_get()) NRS start fifo request from 12345-0@lo, seq: 1104 00000100:00100000:2.0:1776178502.505006:0:11281:0:(service.c:2324:ptlrpc_server_handle_request()) Handling RPC req@000000009497b17e pname:cluuid+ref:pid:xid:nid:opc:job mdt00_004:lustre-MDT0001-lwp-OST0001_UUID+5:3354:x1862456998090624:12345-0@lo:400:kworker/u8:2.0 00000100:00100000:2.0:1776178502.505091:0:11281:0:(service.c:2372:ptlrpc_server_handle_request()) Handled RPC req@000000009497b17e pname:cluuid+ref:pid:xid:nid:opc:job mdt00_004:lustre-MDT0001-lwp-OST0001_UUID+5:3354:x1862456998090624:12345-0@lo:400:kworker/u8:2.0 Request processed in 85us (181us total) trans 0 rc 0/0 00000100:00100000:2.0:1776178502.505099:0:11281:0:(nrs_fifo.c:241:nrs_fifo_req_stop()) NRS stop fifo request from 12345-0@lo, seq: 1104 00000100:00100000:2.0:1776178502.505155:0:3354:0:(client.c:2215:ptlrpc_check_set()) Completed RPC req@000000007309261c pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_01:lustre-MDT0001-lwp-OST0001_UUID:3354:1862456998090624:0@lo:400:kworker/u8:2.0 00000100:00100000:2.0:1776178502.505187:0:20762:0:(service.c:2152:ptlrpc_server_handle_req_in()) got req x1862456998090752 00000100:00100000:2.0:1776178502.505212:0:20762:0:(nrs_fifo.c:179:nrs_fifo_req_get()) NRS start fifo request from 12345-0@lo, seq: 745 00000100:00100000:2.0:1776178502.505216:0:20762:0:(service.c:2324:ptlrpc_server_handle_request()) Handling RPC req@000000001d2ef500 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_013:lustre-MDT0001-mdtlov_UUID+5:3354:x1862456998090752:12345-0@lo:400:kworker/u8:2.0 00000100:00100000:2.0:1776178502.505251:0:20762:0:(service.c:2372:ptlrpc_server_handle_request()) Handled RPC req@000000001d2ef500 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_013:lustre-MDT0001-mdtlov_UUID+5:3354:x1862456998090752:12345-0@lo:400:kworker/u8:2.0 Request processed in 35us (415us total) trans 0 rc 0/0 00000100:00100000:2.0:1776178502.505256:0:20762:0:(nrs_fifo.c:241:nrs_fifo_req_stop()) NRS stop fifo request from 12345-0@lo, seq: 745 00000100:00100000:2.0:1776178502.505277:0:3354:0:(client.c:2215:ptlrpc_check_set()) Completed RPC req@00000000ad37fc5d pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_01:lustre-MDT0001-mdtlov_UUID:3354:1862456998090752:0@lo:400:kworker/u8:2.0 00000100:00100000:2.0:1776178502.505292:0:3356:0:(ptlrpcd.c:415:ptlrpcd_check()) transfer 1 async RPCs [2->3] 00000100:00100000:2.0:1776178502.505298:0:3356:0:(client.c:1733:ptlrpc_send_new_req()) Sending RPC req@000000004547bc6e pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_03:lustre-MDT0001-mdtlov_UUID:3356:1862456998090816:0@lo:400:kworker/u8:2.0 00000100:00100000:2.0:1776178502.505319:0:3356:0:(events.c:361:request_in_callback()) peer: 12345-0@lo (source: 12345-0@lo) 00000100:00100000:2.0:1776178502.505336:0:20762:0:(service.c:2152:ptlrpc_server_handle_req_in()) got req x1862456998090816 00000100:00100000:2.0:1776178502.505358:0:20762:0:(nrs_fifo.c:179:nrs_fifo_req_get()) NRS start fifo request from 12345-0@lo, seq: 746 00000100:00100000:2.0:1776178502.505361:0:20762:0:(service.c:2324:ptlrpc_server_handle_request()) Handling RPC req@00000000ad37fc5d pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_013:lustre-MDT0001-mdtlov_UUID+5:3356:x1862456998090816:12345-0@lo:400:kworker/u8:2.0 00000100:00100000:2.0:1776178502.505387:0:20762:0:(service.c:2372:ptlrpc_server_handle_request()) Handled RPC req@00000000ad37fc5d pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_013:lustre-MDT0001-mdtlov_UUID+5:3356:x1862456998090816:12345-0@lo:400:kworker/u8:2.0 Request processed in 26us (68us total) trans 0 rc 0/0 00000100:00100000:2.0:1776178502.505393:0:20762:0:(nrs_fifo.c:241:nrs_fifo_req_stop()) NRS stop fifo request from 12345-0@lo, seq: 746 00000100:00100000:2.0:1776178502.505411:0:3356:0:(client.c:2215:ptlrpc_check_set()) Completed RPC req@000000004547bc6e pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_03:lustre-MDT0001-mdtlov_UUID:3356:1862456998090816:0@lo:400:kworker/u8:2.0 00000100:00100000:1.0:1776178502.506294:0:3352:0:(client.c:1733:ptlrpc_send_new_req()) Sending RPC req@0000000007e413b7 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:lustre-MDT0000-lwp-OST0001_UUID:3352:1862456998090560:0@lo:38:kworker/u8:2.0 00000100:00100000:1.0:1776178502.506340:0:3352:0:(events.c:361:request_in_callback()) peer: 12345-0@lo (source: 12345-0@lo) 00000100:00100000:1.0:1776178502.506371:0:3352:0:(client.c:1733:ptlrpc_send_new_req()) Sending RPC req@000000005f1c2da4 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:lustre-MDT0001-mdtlov_UUID:3352:1862456998090688:0@lo:38:kworker/u8:2.0 00000100:00100000:1.0:1776178502.506393:0:3352:0:(events.c:361:request_in_callback()) peer: 12345-0@lo (source: 12345-0@lo) 00000100:00100000:1.0:1776178502.506408:0:3352:0:(client.c:1733:ptlrpc_send_new_req()) Sending RPC req@000000000489d393 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:lustre-MDT0000-lwp-MDT0001_UUID:3352:1862456998090880:0@lo:38:kworker/u8:2.0 00000100:00100000:1.0:1776178502.506426:0:3352:0:(events.c:361:request_in_callback()) peer: 12345-0@lo (source: 12345-0@lo) 00000100:00100000:2.0:1776178502.507119:0:15213:0:(service.c:2152:ptlrpc_server_handle_req_in()) got req x1862456998090688 00000100:00100000:2.0:1776178502.507265:0:15213:0:(nrs_fifo.c:179:nrs_fifo_req_get()) NRS start fifo request from 12345-0@lo, seq: 766 00000100:00100000:2.0:1776178502.507271:0:15213:0:(service.c:2324:ptlrpc_server_handle_request()) Handling RPC req@00000000eee9e8f3 pname:cluuid+ref:pid:xid:nid:opc:job mdt_out00_003:0+-99:3352:x1862456998090688:12345-0@lo:38:kworker/u8:2.0 00010000:02000000:2.0:1776178502.507288:0:15213:0:(ldlm_lib.c:1139:target_handle_connect()) lustre-MDT0000: Not available for connect from 0@lo (stopping) 00000100:00100000:2.0:1776178502.507342:0:15213:0:(service.c:2372:ptlrpc_server_handle_request()) Handled RPC req@00000000eee9e8f3 pname:cluuid+ref:pid:xid:nid:opc:job mdt_out00_003:0+-99:3352:x1862456998090688:12345-0@lo:38:kworker/u8:2.0 Request processed in 70us (948us total) trans 0 rc -19/-19 00000100:00100000:2.0:1776178502.507348:0:15213:0:(nrs_fifo.c:241:nrs_fifo_req_stop()) NRS stop fifo request from 12345-0@lo, seq: 766 00000100:00100000:2.0:1776178502.507373:0:11281:0:(service.c:2152:ptlrpc_server_handle_req_in()) got req x1862456998090560 00000100:00100000:2.0:1776178502.507391:0:11281:0:(service.c:2152:ptlrpc_server_handle_req_in()) got req x1862456998090880 00000100:00100000:2.0:1776178502.507404:0:11281:0:(nrs_fifo.c:179:nrs_fifo_req_get()) NRS start fifo request from 12345-0@lo, seq: 6858 00000100:00100000:2.0:1776178502.507407:0:11281:0:(service.c:2324:ptlrpc_server_handle_request()) Handling RPC req@000000005586abe3 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_004:0+-99:3352:x1862456998090560:12345-0@lo:38:kworker/u8:2.0 00010000:02000000:2.0:1776178502.507416:0:11281:0:(ldlm_lib.c:1139:target_handle_connect()) lustre-MDT0000: Not available for connect from 0@lo (stopping) 00000100:00100000:1.0:1776178502.507591:0:6255:0:(nrs_fifo.c:179:nrs_fifo_req_get()) NRS start fifo request from 12345-0@lo, seq: 6859 00000100:00100000:1.0:1776178502.507595:0:6255:0:(service.c:2324:ptlrpc_server_handle_request()) Handling RPC req@000000002f3c2153 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_000:0+-99:3352:x1862456998090880:12345-0@lo:38:kworker/u8:2.0 00010000:02000000:1.0:1776178502.507602:0:6255:0:(ldlm_lib.c:1139:target_handle_connect()) lustre-MDT0000: Not available for connect from 0@lo (stopping) 00000100:00100000:1.0:1776178502.513485:0:6255:0:(service.c:2372:ptlrpc_server_handle_request()) Handled RPC req@000000002f3c2153 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_000:0+-99:3352:x1862456998090880:12345-0@lo:38:kworker/u8:2.0 Request processed in 5874us (7044us total) trans 0 rc -19/-19 00000100:00100000:1.0:1776178502.513508:0:6255:0:(nrs_fifo.c:241:nrs_fifo_req_stop()) NRS stop fifo request from 12345-0@lo, seq: 6859 00000100:00080000:1.0:1776178502.513605:0:3352:0:(import.c:86:import_set_state_nolock()) 00000000ba4f1821 lustre-MDT0000_UUID: changing import state from CONNECTING to DISCONN 00000100:00080000:1.0:1776178502.513610:0:3352:0:(import.c:1429:ptlrpc_connect_interpret()) recovery of lustre-MDT0000_UUID on 192.168.204.103@tcp failed (-19) 00000100:00100000:1.0:1776178502.513615:0:3352:0:(client.c:2215:ptlrpc_check_set()) Completed RPC req@000000005f1c2da4 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:lustre-MDT0001-mdtlov_UUID:3352:1862456998090688:0@lo:38:kworker/u8:2.0 00000100:00080000:1.0:1776178502.513628:0:3352:0:(import.c:86:import_set_state_nolock()) 00000000a7c55b1f lustre-MDT0000_UUID: changing import state from CONNECTING to DISCONN 00000100:00080000:1.0:1776178502.513632:0:3352:0:(import.c:1429:ptlrpc_connect_interpret()) recovery of lustre-MDT0000_UUID on 192.168.204.103@tcp failed (-19) 00000100:00100000:1.0:1776178502.513634:0:3352:0:(client.c:2215:ptlrpc_check_set()) Completed RPC req@000000000489d393 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:lustre-MDT0000-lwp-MDT0001_UUID:3352:1862456998090880:0@lo:38:kworker/u8:2.0 00000100:00100000:2.0:1776178502.527371:0:11281:0:(service.c:2372:ptlrpc_server_handle_request()) Handled RPC req@000000005586abe3 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_004:0+-99:3352:x1862456998090560:12345-0@lo:38:kworker/u8:2.0 Request processed in 19951us (21019us total) trans 0 rc -19/-19 00000100:00100000:2.0:1776178502.527389:0:11281:0:(nrs_fifo.c:241:nrs_fifo_req_stop()) NRS stop fifo request from 12345-0@lo, seq: 6858 00000100:00080000:1.0:1776178502.527513:0:3352:0:(import.c:86:import_set_state_nolock()) 00000000dfc8db13 lustre-MDT0000_UUID: changing import state from CONNECTING to DISCONN 00000100:00080000:1.0:1776178502.527518:0:3352:0:(import.c:1429:ptlrpc_connect_interpret()) recovery of lustre-MDT0000_UUID on 192.168.204.103@tcp failed (-19) 00000100:00100000:1.0:1776178502.527523:0:3352:0:(client.c:2215:ptlrpc_check_set()) Completed RPC req@0000000007e413b7 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:lustre-MDT0000-lwp-OST0001_UUID:3352:1862456998090560:0@lo:38:kworker/u8:2.0 00000100:00100000:3.0:1776178502.533319:0:3355:0:(client.c:2215:ptlrpc_check_set()) Completed RPC req@0000000057946847 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_02:ac61f839-0e8e-4cac-9c59-08bcfbbad330:3355:1862456998090368:0@lo:400:kworker/u8:2.0 00000100:00100000:0.0:1776178502.533432:0:20140:0:(service.c:2372:ptlrpc_server_handle_request()) Handled RPC req@00000000e22b0439 pname:cluuid+ref:pid:xid:nid:opc:job ll_mgs_0002:ac61f839-0e8e-4cac-9c59-08bcfbbad330+18:3355:x1862456998090368:12345-0@lo:400:kworker/u8:2.0 Request processed in 35798us (36104us total) trans 0 rc 0/0 00000100:00100000:0.0:1776178502.533448:0:20140:0:(nrs_fifo.c:241:nrs_fifo_req_stop()) NRS stop fifo request from 12345-0@lo, seq: 272 00010000:00010000:1.0:1776178503.140641:0:9225:0:(ldlm_lockd.c:2792:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:0.0:1776178504.035053:0:3355:0:(ptlrpcd.c:415:ptlrpcd_check()) transfer 1 async RPCs [3->2] 00000100:00100000:0.0:1776178504.035118:0:3355:0:(client.c:1733:ptlrpc_send_new_req()) Sending RPC req@00000000d28be450 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_02:lustre-MDT0001-mdtlov_UUID:3355:1862456998090944:0@lo:13:osp-pre-1-1.0 00000100:00100000:0.0:1776178504.035193:0:3355:0:(events.c:361:request_in_callback()) peer: 12345-0@lo (source: 12345-0@lo) 00000100:00100000:3.0:1776178504.035295:0:8231:0:(service.c:2152:ptlrpc_server_handle_req_in()) got req x1862456998090944 00000100:00100000:3.0:1776178504.035363:0:8231:0:(nrs_fifo.c:179:nrs_fifo_req_get()) NRS start fifo request from 12345-0@lo, seq: 605 00000100:00100000:3.0:1776178504.035369:0:8231:0:(service.c:2324:ptlrpc_server_handle_request()) Handling RPC req@0000000056a48d5b pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_create00:lustre-MDT0001-mdtlov_UUID+5:3355:x1862456998090944:12345-0@lo:13:osp-pre-1-1.0 00002000:00000024:3.0:1776178504.035401:0:8231:0:(ofd_obd.c:740:ofd_statfs()) blocks cached 229376 granted 491746368 pending 0 free 3923689472 avail 3692167168 00000100:00100000:3.0:1776178504.035460:0:8231:0:(service.c:2372:ptlrpc_server_handle_request()) Handled RPC req@0000000056a48d5b pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_create00:lustre-MDT0001-mdtlov_UUID+5:3355:x1862456998090944:12345-0@lo:13:osp-pre-1-1.0 Request processed in 90us (268us total) trans 0 rc 0/0 00000100:00100000:3.0:1776178504.035468:0:8231:0:(nrs_fifo.c:241:nrs_fifo_req_stop()) NRS stop fifo request from 12345-0@lo, seq: 605 00000100:00100000:1.0:1776178504.035584:0:3355:0:(client.c:2215:ptlrpc_check_set()) Completed RPC req@00000000d28be450 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_02:lustre-MDT0001-mdtlov_UUID:3355:1862456998090944:0@lo:13:osp-pre-1-1.0 00010000:00010000:3.0:1776178504.161467:0:6244:0:(ldlm_lockd.c:2792:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:1.0:1776178504.544401:0:3354:0:(ptlrpcd.c:415:ptlrpcd_check()) transfer 1 async RPCs [0->1] 00000100:00100000:1.0:1776178504.544437:0:3354:0:(client.c:1733:ptlrpc_send_new_req()) Sending RPC req@000000007c827612 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_01:lustre-MDT0001-mdtlov_UUID:3354:1862456998091008:0@lo:13:osp-pre-0-1.0 00000100:00100000:1.0:1776178504.544509:0:3354:0:(events.c:361:request_in_callback()) peer: 12345-0@lo (source: 12345-0@lo) 00000100:00100000:0.0:1776178504.544671:0:9554:0:(service.c:2152:ptlrpc_server_handle_req_in()) got req x1862456998091008 00000100:00100000:0.0:1776178504.544729:0:9554:0:(nrs_fifo.c:179:nrs_fifo_req_get()) NRS start fifo request from 12345-0@lo, seq: 606 00000100:00100000:0.0:1776178504.544735:0:9554:0:(service.c:2324:ptlrpc_server_handle_request()) Handling RPC req@00000000dc0a388a pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_create00:lustre-MDT0001-mdtlov_UUID+5:3354:x1862456998091008:12345-0@lo:13:osp-pre-0-1.0 00002000:00000024:0.0:1776178504.544770:0:9554:0:(ofd_obd.c:740:ofd_statfs()) blocks cached 315392 granted 484883776 pending 0 free 3923689472 avail 3692167168 00000100:00100000:0.0:1776178504.544819:0:9554:0:(service.c:2372:ptlrpc_server_handle_request()) Handled RPC req@00000000dc0a388a pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_create00:lustre-MDT0001-mdtlov_UUID+5:3354:x1862456998091008:12345-0@lo:13:osp-pre-0-1.0 Request processed in 82us (311us total) trans 0 rc 0/0 00000100:00100000:0.0:1776178504.544826:0:9554:0:(nrs_fifo.c:241:nrs_fifo_req_stop()) NRS stop fifo request from 12345-0@lo, seq: 606 00000100:00100000:1.0:1776178504.544914:0:3354:0:(client.c:2215:ptlrpc_check_set()) Completed RPC req@000000007c827612 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_01:lustre-MDT0001-mdtlov_UUID:3354:1862456998091008:0@lo:13:osp-pre-0-1.0 00010000:00010000:2.0:1776178505.190047:0:6245:0:(ldlm_lockd.c:2792:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:1.0:1776178506.018211:0:3347:0:(events.c:361:request_in_callback()) peer: 12345-192.168.204.3@tcp (source: 12345-192.168.204.3@tcp) 00000100:00100000:2.0:1776178506.018354:0:11281:0:(service.c:2152:ptlrpc_server_handle_req_in()) got req x1862456979127936 00000100:00100000:2.0:1776178506.018413:0:11281:0:(nrs_fifo.c:179:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.204.3@tcp, seq: 6860 00000100:00100000:2.0:1776178506.018420:0:11281:0:(service.c:2324:ptlrpc_server_handle_request()) Handling RPC req@000000009c8ee576 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_004:0+-99:2269:x1862456979127936:12345-192.168.204.3@tcp:38:kworker/u8:3.0 00010000:02000000:2.0:1776178506.018439:0:11281:0:(ldlm_lib.c:1139:target_handle_connect()) lustre-MDT0000: Not available for connect from 192.168.204.3@tcp (stopping) 00000100:00100000:2.0:1776178506.018509:0:11281:0:(service.c:2372:ptlrpc_server_handle_request()) Handled RPC req@000000009c8ee576 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_004:0+-99:2269:x1862456979127936:12345-192.168.204.3@tcp:38:kworker/u8:3.0 Request processed in 88us (311us total) trans 0 rc -19/-19 00000100:00100000:2.0:1776178506.018516:0:11281:0:(nrs_fifo.c:241:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.204.3@tcp, seq: 6860 00000100:00100000:0.0:1776178506.027706:0:3346:0:(events.c:361:request_in_callback()) peer: 12345-192.168.204.3@tcp (source: 12345-192.168.204.3@tcp) 00000100:00100000:3.0:1776178506.027883:0:20140:0:(service.c:2152:ptlrpc_server_handle_req_in()) got req x1862456979127872 00000100:00100000:3.0:1776178506.027965:0:20140:0:(nrs_fifo.c:179:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.204.3@tcp, seq: 273 00000100:00100000:3.0:1776178506.027973:0:20140:0:(service.c:2324:ptlrpc_server_handle_request()) Handling RPC req@00000000b8c10765 pname:cluuid+ref:pid:xid:nid:opc:job ll_mgs_0002:5b5c66b1-1a2f-4651-8548-0c75b9a38a94+9:2271:x1862456979127872:12345-192.168.204.3@tcp:400:kworker/u8:3.0 00000100:00100000:3.0:1776178506.028093:0:20140:0:(service.c:2372:ptlrpc_server_handle_request()) Handled RPC req@00000000b8c10765 pname:cluuid+ref:pid:xid:nid:opc:job ll_mgs_0002:5b5c66b1-1a2f-4651-8548-0c75b9a38a94+9:2271:x1862456979127872:12345-192.168.204.3@tcp:400:kworker/u8:3.0 Request processed in 119us (403us total) trans 0 rc 0/0 00000100:00100000:3.0:1776178506.028101:0:20140:0:(nrs_fifo.c:241:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.204.3@tcp, seq: 273 00000100:00100000:3.0:1776178506.028164:0:3347:0:(events.c:361:request_in_callback()) peer: 12345-192.168.204.3@tcp (source: 12345-192.168.204.3@tcp) 00000100:00100000:3.0:1776178506.037688:0:3347:0:(events.c:361:request_in_callback()) peer: 12345-192.168.204.3@tcp (source: 12345-192.168.204.3@tcp) 00000100:00100000:3.0:1776178506.037800:0:3347:0:(events.c:361:request_in_callback()) peer: 12345-192.168.204.3@tcp (source: 12345-192.168.204.3@tcp) 00000100:00100000:0.0:1776178506.037861:0:8250:0:(service.c:2152:ptlrpc_server_handle_req_in()) got req x1862456979128000 00000100:00100000:0.0:1776178506.037933:0:8250:0:(nrs_fifo.c:179:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.204.3@tcp, seq: 1105 00000100:00100000:0.0:1776178506.037954:0:8250:0:(service.c:2324:ptlrpc_server_handle_request()) Handling RPC req@000000006b9d8ccb pname:cluuid+ref:pid:xid:nid:opc:job mdt00_003:b50963bd-8eba-4432-9a54-8888d00e3fa1+6:2271:x1862456979128000:12345-192.168.204.3@tcp:400:kworker/u8:3.0 00000100:00100000:0.0:1776178506.038104:0:8250:0:(service.c:2372:ptlrpc_server_handle_request()) Handled RPC req@000000006b9d8ccb pname:cluuid+ref:pid:xid:nid:opc:job mdt00_003:b50963bd-8eba-4432-9a54-8888d00e3fa1+6:2271:x1862456979128000:12345-192.168.204.3@tcp:400:kworker/u8:3.0 Request processed in 162us (435us total) trans 0 rc 0/0 00000100:00100000:0.0:1776178506.038130:0:8250:0:(nrs_fifo.c:241:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.204.3@tcp, seq: 1105 00000100:00100000:2.0:1776178506.043184:0:14440:0:(service.c:2152:ptlrpc_server_handle_req_in()) got req x1862456979128064 00000100:00100000:2.0:1776178506.043306:0:14440:0:(service.c:2152:ptlrpc_server_handle_req_in()) got req x1862456979128128 00000100:00100000:2.0:1776178506.043344:0:14440:0:(nrs_fifo.c:179:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.204.3@tcp, seq: 747 00000100:00100000:2.0:1776178506.043351:0:14440:0:(service.c:2324:ptlrpc_server_handle_request()) Handling RPC req@00000000d28be450 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_004:b50963bd-8eba-4432-9a54-8888d00e3fa1+8:2271:x1862456979128064:12345-192.168.204.3@tcp:400:kworker/u8:3.0 00000100:00100000:2.0:1776178506.044516:0:14440:0:(service.c:2372:ptlrpc_server_handle_request()) Handled RPC req@00000000d28be450 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_004:b50963bd-8eba-4432-9a54-8888d00e3fa1+8:2271:x1862456979128064:12345-192.168.204.3@tcp:400:kworker/u8:3.0 Request processed in 1158us (16344us total) trans 0 rc 0/0 00000100:00100000:2.0:1776178506.044533:0:14440:0:(nrs_fifo.c:241:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.204.3@tcp, seq: 747 00000100:00100000:2.0:1776178506.044574:0:14440:0:(nrs_fifo.c:179:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.204.3@tcp, seq: 748 00000100:00100000:2.0:1776178506.044580:0:14440:0:(service.c:2324:ptlrpc_server_handle_request()) Handling RPC req@000000009d0d1846 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_004:b50963bd-8eba-4432-9a54-8888d00e3fa1+9:2271:x1862456979128128:12345-192.168.204.3@tcp:400:kworker/u8:3.0 00000100:00100000:2.0:1776178506.044629:0:14440:0:(service.c:2372:ptlrpc_server_handle_request()) Handled RPC req@000000009d0d1846 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_004:b50963bd-8eba-4432-9a54-8888d00e3fa1+9:2271:x1862456979128128:12345-192.168.204.3@tcp:400:kworker/u8:3.0 Request processed in 50us (6831us total) trans 0 rc 0/0 00000100:00100000:2.0:1776178506.044635:0:14440:0:(nrs_fifo.c:241:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.204.3@tcp, seq: 748 00010000:00010000:1.0:1776178506.210922:0:9224:0:(ldlm_lockd.c:2792:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00010000:00010000:1.0:1776178507.239223:0:9225:0:(ldlm_lockd.c:2792:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00080000:1.0:1776178507.616734:0:36:0:(pinger.c:251:ptlrpc_pinger_process_import()) lustre-MDT0000-lwp-OST0000_UUID->lustre-MDT0000_UUID: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00080000:1.0:1776178507.616745:0:36:0:(recover.c:57:ptlrpc_initiate_recovery()) lustre-MDT0000_UUID: starting recovery 00000100:00080000:1.0:1776178507.616749:0:36:0:(import.c:86:import_set_state_nolock()) 0000000047601a98 lustre-MDT0000_UUID: changing import state from DISCONN to CONNECTING 00000100:00100000:3.0:1776178507.616936:0:3354:0:(client.c:1733:ptlrpc_send_new_req()) Sending RPC req@00000000ae7e263a pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_01:ac61f839-0e8e-4cac-9c59-08bcfbbad330:3354:1862456998091072:0@lo:400:kworker/u8:1.0 00000100:00100000:3.0:1776178507.617042:0:3354:0:(events.c:361:request_in_callback()) peer: 12345-0@lo (source: 12345-0@lo) 00000100:00100000:2.0:1776178507.617118:0:20140:0:(service.c:2152:ptlrpc_server_handle_req_in()) got req x1862456998091072 00000100:00100000:2.0:1776178507.617191:0:20140:0:(nrs_fifo.c:179:nrs_fifo_req_get()) NRS start fifo request from 12345-0@lo, seq: 274 00000100:00100000:2.0:1776178507.617199:0:20140:0:(service.c:2324:ptlrpc_server_handle_request()) Handling RPC req@00000000b59e728a pname:cluuid+ref:pid:xid:nid:opc:job ll_mgs_0002:ac61f839-0e8e-4cac-9c59-08bcfbbad330+18:3354:x1862456998091072:12345-0@lo:400:kworker/u8:1.0 00000100:00100000:2.0:1776178507.617287:0:20140:0:(service.c:2372:ptlrpc_server_handle_request()) Handled RPC req@00000000b59e728a pname:cluuid+ref:pid:xid:nid:opc:job ll_mgs_0002:ac61f839-0e8e-4cac-9c59-08bcfbbad330+18:3354:x1862456998091072:12345-0@lo:400:kworker/u8:1.0 Request processed in 87us (248us total) trans 0 rc 0/0 00000100:00100000:2.0:1776178507.617296:0:20140:0:(nrs_fifo.c:241:nrs_fifo_req_stop()) NRS stop fifo request from 12345-0@lo, seq: 274 00000100:00100000:3.0:1776178507.617369:0:3354:0:(client.c:2215:ptlrpc_check_set()) Completed RPC req@00000000ae7e263a pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_01:ac61f839-0e8e-4cac-9c59-08bcfbbad330:3354:1862456998091072:0@lo:400:kworker/u8:1.0 00000100:00080000:1.0:1776178507.617719:0:36:0:(import.c:534:import_select_connection()) lustre-MDT0000-lwp-OST0000: connect to NID 0@lo last attempt 1209 00000100:00080000:1.0:1776178507.617722:0:36:0:(import.c:581:import_select_connection()) lustre-MDT0000-lwp-OST0000: tried all connections, increasing latency to 21s 00000100:00080000:1.0:1776178507.617727:0:36:0:(import.c:615:import_select_connection()) lustre-MDT0000-lwp-OST0000: import 0000000047601a98 using connection 192.168.204.103@tcp/0@lo 00000100:00100000:1.0:1776178507.617767:0:36:0:(import.c:807:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5) req@0000000054386dd3 x1862456998091136/t0(0) o38->lustre-MDT0000-lwp-OST0000@0@lo:12/10 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' 00000100:00080000:1.0:1776178507.617822:0:36:0:(pinger.c:251:ptlrpc_pinger_process_import()) lustre-MDT0000-lwp-OST0001_UUID->lustre-MDT0000_UUID: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00080000:1.0:1776178507.617827:0:36:0:(recover.c:57:ptlrpc_initiate_recovery()) lustre-MDT0000_UUID: starting recovery 00000100:00080000:1.0:1776178507.617829:0:36:0:(import.c:86:import_set_state_nolock()) 00000000dfc8db13 lustre-MDT0000_UUID: changing import state from DISCONN to CONNECTING 00000100:00080000:1.0:1776178507.617832:0:36:0:(import.c:534:import_select_connection()) lustre-MDT0000-lwp-OST0001: connect to NID 0@lo last attempt 1209 00000100:00080000:1.0:1776178507.617834:0:36:0:(import.c:581:import_select_connection()) lustre-MDT0000-lwp-OST0001: tried all connections, increasing latency to 21s 00000100:00080000:1.0:1776178507.617838:0:36:0:(import.c:615:import_select_connection()) lustre-MDT0000-lwp-OST0001: import 00000000dfc8db13 using connection 192.168.204.103@tcp/0@lo 00000100:00100000:3.0:1776178507.618086:0:3355:0:(client.c:1733:ptlrpc_send_new_req()) Sending RPC req@0000000072a0bf14 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_02:lustre-MDT0001-lwp-OST0000_UUID:3355:1862456998091200:0@lo:400:kworker/u8:1.0 00000100:00100000:3.0:1776178507.618133:0:3355:0:(events.c:361:request_in_callback()) peer: 12345-0@lo (source: 12345-0@lo) 00000100:00100000:0.0:1776178507.618584:0:8250:0:(service.c:2152:ptlrpc_server_handle_req_in()) got req x1862456998091200 00000100:00100000:0.0:1776178507.618651:0:8250:0:(nrs_fifo.c:179:nrs_fifo_req_get()) NRS start fifo request from 12345-0@lo, seq: 1106 00000100:00100000:0.0:1776178507.618656:0:8250:0:(service.c:2324:ptlrpc_server_handle_request()) Handling RPC req@0000000024f67795 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_003:lustre-MDT0001-lwp-OST0000_UUID+5:3355:x1862456998091200:12345-0@lo:400:kworker/u8:1.0 00000100:00100000:0.0:1776178507.618727:0:8250:0:(service.c:2372:ptlrpc_server_handle_request()) Handled RPC req@0000000024f67795 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_003:lustre-MDT0001-lwp-OST0000_UUID+5:3355:x1862456998091200:12345-0@lo:400:kworker/u8:1.0 Request processed in 70us (594us total) trans 0 rc 0/0 00000100:00100000:0.0:1776178507.618735:0:8250:0:(nrs_fifo.c:241:nrs_fifo_req_stop()) NRS stop fifo request from 12345-0@lo, seq: 1106 00000100:00100000:0.0:1776178507.618788:0:3355:0:(client.c:2215:ptlrpc_check_set()) Completed RPC req@0000000072a0bf14 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_02:lustre-MDT0001-lwp-OST0000_UUID:3355:1862456998091200:0@lo:400:kworker/u8:1.0 00000100:00100000:2.0:1776178507.619127:0:3352:0:(client.c:1733:ptlrpc_send_new_req()) Sending RPC req@0000000054386dd3 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:lustre-MDT0000-lwp-OST0000_UUID:3352:1862456998091136:0@lo:38:kworker/u8:1.0 00000100:00100000:2.0:1776178507.619170:0:3352:0:(events.c:361:request_in_callback()) peer: 12345-0@lo (source: 12345-0@lo) 00000100:00100000:1.0:1776178507.619272:0:36:0:(import.c:807:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5) req@000000000a224757 x1862456998091264/t0(0) o38->lustre-MDT0000-lwp-OST0001@0@lo:12/10 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' 00000100:00080000:1.0:1776178507.619301:0:36:0:(pinger.c:251:ptlrpc_pinger_process_import()) lustre-MDT0001-mdtlov_UUID->lustre-MDT0000_UUID: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00080000:1.0:1776178507.619305:0:36:0:(recover.c:57:ptlrpc_initiate_recovery()) lustre-MDT0000_UUID: starting recovery 00000100:00080000:1.0:1776178507.619309:0:36:0:(import.c:86:import_set_state_nolock()) 00000000ba4f1821 lustre-MDT0000_UUID: changing import state from DISCONN to CONNECTING 00000100:00080000:1.0:1776178507.619313:0:36:0:(import.c:534:import_select_connection()) lustre-MDT0000-osp-MDT0001: connect to NID 0@lo last attempt 1209 00000100:00080000:1.0:1776178507.619316:0:36:0:(import.c:581:import_select_connection()) lustre-MDT0000-osp-MDT0001: tried all connections, increasing latency to 21s 00000100:00080000:1.0:1776178507.619322:0:36:0:(import.c:615:import_select_connection()) lustre-MDT0000-osp-MDT0001: import 00000000ba4f1821 using connection 192.168.204.103@tcp/0@lo 00000100:00100000:1.0:1776178507.619350:0:36:0:(import.c:807:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5) req@00000000d596a45c x1862456998091392/t0(0) o38->lustre-MDT0000-osp-MDT0001@0@lo:24/4 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' 00000100:00080000:1.0:1776178507.619384:0:36:0:(pinger.c:251:ptlrpc_pinger_process_import()) lustre-MDT0000-lwp-MDT0001_UUID->lustre-MDT0000_UUID: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00080000:1.0:1776178507.619387:0:36:0:(recover.c:57:ptlrpc_initiate_recovery()) lustre-MDT0000_UUID: starting recovery 00000100:00080000:1.0:1776178507.619389:0:36:0:(import.c:86:import_set_state_nolock()) 00000000a7c55b1f lustre-MDT0000_UUID: changing import state from DISCONN to CONNECTING 00000100:00080000:1.0:1776178507.619393:0:36:0:(import.c:534:import_select_connection()) lustre-MDT0000-lwp-MDT0001: connect to NID 0@lo last attempt 1209 00000100:00080000:1.0:1776178507.619395:0:36:0:(import.c:581:import_select_connection()) lustre-MDT0000-lwp-MDT0001: tried all connections, increasing latency to 21s 00000100:00080000:1.0:1776178507.619398:0:36:0:(import.c:615:import_select_connection()) lustre-MDT0000-lwp-MDT0001: import 00000000a7c55b1f using connection 192.168.204.103@tcp/0@lo 00000100:00100000:1.0:1776178507.619409:0:36:0:(import.c:807:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5) req@000000002917d432 x1862456998091584/t0(0) o38->lustre-MDT0000-lwp-MDT0001@0@lo:12/10 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' 00000100:00100000:1.0:1776178507.619453:0:3354:0:(client.c:1733:ptlrpc_send_new_req()) Sending RPC req@0000000012d4bd5e pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_01:lustre-MDT0001-mdtlov_UUID:3354:1862456998091520:0@lo:400:kworker/u8:1.0 00000100:00100000:1.0:1776178507.619678:0:3354:0:(events.c:361:request_in_callback()) peer: 12345-0@lo (source: 12345-0@lo) 00000100:00100000:1.0:1776178507.619705:0:3354:0:(ptlrpcd.c:415:ptlrpcd_check()) transfer 1 async RPCs [0->1] 00000100:00100000:1.0:1776178507.619711:0:3354:0:(client.c:1733:ptlrpc_send_new_req()) Sending RPC req@00000000582bb477 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_01:lustre-MDT0001-mdtlov_UUID:3354:1862456998091456:0@lo:400:kworker/u8:1.0 00000100:00100000:1.0:1776178507.619735:0:3354:0:(events.c:361:request_in_callback()) peer: 12345-0@lo (source: 12345-0@lo) 00000100:00100000:1.0:1776178507.619781:0:20759:0:(service.c:2152:ptlrpc_server_handle_req_in()) got req x1862456998091520 00000100:00100000:1.0:1776178507.619823:0:20759:0:(service.c:2152:ptlrpc_server_handle_req_in()) got req x1862456998091456 00000100:00100000:1.0:1776178507.620007:0:20759:0:(nrs_fifo.c:179:nrs_fifo_req_get()) NRS start fifo request from 12345-0@lo, seq: 749 00000100:00100000:1.0:1776178507.620013:0:20759:0:(service.c:2324:ptlrpc_server_handle_request()) Handling RPC req@0000000028dd2aee pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_010:lustre-MDT0001-mdtlov_UUID+5:3354:x1862456998091520:12345-0@lo:400:kworker/u8:1.0 00000100:00100000:1.0:1776178507.620109:0:20759:0:(service.c:2372:ptlrpc_server_handle_request()) Handled RPC req@0000000028dd2aee pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_010:lustre-MDT0001-mdtlov_UUID+5:3354:x1862456998091520:12345-0@lo:400:kworker/u8:1.0 Request processed in 96us (431us total) trans 0 rc 0/0 00000100:00100000:1.0:1776178507.620117:0:20759:0:(nrs_fifo.c:241:nrs_fifo_req_stop()) NRS stop fifo request from 12345-0@lo, seq: 749 00000100:00100000:1.0:1776178507.620141:0:20759:0:(nrs_fifo.c:179:nrs_fifo_req_get()) NRS start fifo request from 12345-0@lo, seq: 750 00000100:00100000:1.0:1776178507.620145:0:20759:0:(service.c:2324:ptlrpc_server_handle_request()) Handling RPC req@00000000b319fd93 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_010:lustre-MDT0001-mdtlov_UUID+5:3354:x1862456998091456:12345-0@lo:400:kworker/u8:1.0 00000100:00100000:0.0:1776178507.620171:0:3355:0:(ptlrpcd.c:415:ptlrpcd_check()) transfer 1 async RPCs [3->2] 00000100:00100000:0.0:1776178507.620186:0:3355:0:(client.c:1733:ptlrpc_send_new_req()) Sending RPC req@00000000ccad874e pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_02:lustre-MDT0001-lwp-OST0001_UUID:3355:1862456998091328:0@lo:400:kworker/u8:1.0 00000100:00100000:2.0:1776178507.620211:0:3352:0:(client.c:1733:ptlrpc_send_new_req()) Sending RPC req@000000000a224757 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:lustre-MDT0000-lwp-OST0001_UUID:3352:1862456998091264:0@lo:38:kworker/u8:1.0 00000100:00100000:0.0:1776178507.620229:0:3355:0:(events.c:361:request_in_callback()) peer: 12345-0@lo (source: 12345-0@lo) 00000100:00100000:1.0:1776178507.620664:0:20759:0:(service.c:2372:ptlrpc_server_handle_request()) Handled RPC req@00000000b319fd93 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_010:lustre-MDT0001-mdtlov_UUID+5:3354:x1862456998091456:12345-0@lo:400:kworker/u8:1.0 Request processed in 517us (928us total) trans 0 rc 0/0 00000100:00100000:1.0:1776178507.620672:0:20759:0:(nrs_fifo.c:241:nrs_fifo_req_stop()) NRS stop fifo request from 12345-0@lo, seq: 750 00000100:00100000:1.0:1776178507.620732:0:3354:0:(client.c:2215:ptlrpc_check_set()) Completed RPC req@00000000582bb477 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_01:lustre-MDT0001-mdtlov_UUID:3354:1862456998091456:0@lo:400:kworker/u8:1.0 00000100:00100000:1.0:1776178507.620746:0:3354:0:(client.c:2215:ptlrpc_check_set()) Completed RPC req@0000000012d4bd5e pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_01:lustre-MDT0001-mdtlov_UUID:3354:1862456998091520:0@lo:400:kworker/u8:1.0 00000100:00100000:2.0:1776178507.621459:0:3352:0:(events.c:361:request_in_callback()) peer: 12345-0@lo (source: 12345-0@lo) 00000100:00100000:2.0:1776178507.621487:0:3352:0:(client.c:1733:ptlrpc_send_new_req()) Sending RPC req@00000000d596a45c pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:lustre-MDT0001-mdtlov_UUID:3352:1862456998091392:0@lo:38:kworker/u8:1.0 00000100:00100000:2.0:1776178507.621538:0:3352:0:(events.c:361:request_in_callback()) peer: 12345-0@lo (source: 12345-0@lo) 00000100:00100000:2.0:1776178507.621556:0:3352:0:(client.c:1733:ptlrpc_send_new_req()) Sending RPC req@000000002917d432 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:lustre-MDT0000-lwp-MDT0001_UUID:3352:1862456998091584:0@lo:38:kworker/u8:1.0 00000100:00100000:2.0:1776178507.621588:0:3352:0:(events.c:361:request_in_callback()) peer: 12345-0@lo (source: 12345-0@lo) 00000100:00100000:0.0:1776178507.621696:0:6255:0:(service.c:2152:ptlrpc_server_handle_req_in()) got req x1862456998091136 00000100:00100000:0.0:1776178507.621743:0:6255:0:(service.c:2152:ptlrpc_server_handle_req_in()) got req x1862456998091328 00000100:00100000:1.0:1776178507.622025:0:6256:0:(service.c:2152:ptlrpc_server_handle_req_in()) got req x1862456998091264 00000100:00100000:1.0:1776178507.622041:0:6256:0:(service.c:2152:ptlrpc_server_handle_req_in()) got req x1862456998091584 00000100:00100000:1.0:1776178507.622052:0:6256:0:(nrs_fifo.c:179:nrs_fifo_req_get()) NRS start fifo request from 12345-0@lo, seq: 1107 00000100:00100000:1.0:1776178507.622058:0:6256:0:(service.c:2324:ptlrpc_server_handle_request()) Handling RPC req@00000000a343c008 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_001:lustre-MDT0001-lwp-OST0001_UUID+5:3355:x1862456998091328:12345-0@lo:400:kworker/u8:1.0 00000100:00100000:1.0:1776178507.622145:0:6256:0:(service.c:2372:ptlrpc_server_handle_request()) Handled RPC req@00000000a343c008 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_001:lustre-MDT0001-lwp-OST0001_UUID+5:3355:x1862456998091328:12345-0@lo:400:kworker/u8:1.0 Request processed in 86us (1917us total) trans 0 rc 0/0 00000100:00100000:1.0:1776178507.622152:0:6256:0:(nrs_fifo.c:241:nrs_fifo_req_stop()) NRS stop fifo request from 12345-0@lo, seq: 1107 00000100:00100000:1.0:1776178507.622172:0:6256:0:(nrs_fifo.c:179:nrs_fifo_req_get()) NRS start fifo request from 12345-0@lo, seq: 6861 00000100:00100000:1.0:1776178507.622175:0:6256:0:(service.c:2324:ptlrpc_server_handle_request()) Handling RPC req@000000004547bc6e pname:cluuid+ref:pid:xid:nid:opc:job mdt00_001:0+-99:3352:x1862456998091136:12345-0@lo:38:kworker/u8:1.0 00010000:02000000:1.0:1776178507.622192:0:6256:0:(ldlm_lib.c:1139:target_handle_connect()) lustre-MDT0000: Not available for connect from 0@lo (stopping) 00000100:00100000:1.0:1776178507.622231:0:6256:0:(service.c:2372:ptlrpc_server_handle_request()) Handled RPC req@000000004547bc6e pname:cluuid+ref:pid:xid:nid:opc:job mdt00_001:0+-99:3352:x1862456998091136:12345-0@lo:38:kworker/u8:1.0 Request processed in 54us (3061us total) trans 0 rc -19/-19 00000100:00100000:1.0:1776178507.622238:0:6256:0:(nrs_fifo.c:241:nrs_fifo_req_stop()) NRS stop fifo request from 12345-0@lo, seq: 6861 00000100:00100000:1.0:1776178507.622258:0:6256:0:(nrs_fifo.c:179:nrs_fifo_req_get()) NRS start fifo request from 12345-0@lo, seq: 6862 00000100:00100000:1.0:1776178507.622263:0:6256:0:(service.c:2324:ptlrpc_server_handle_request()) Handling RPC req@000000005f1c2da4 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_001:0+-99:3352:x1862456998091264:12345-0@lo:38:kworker/u8:1.0 00010000:02000000:1.0:1776178507.622269:0:6256:0:(ldlm_lib.c:1139:target_handle_connect()) lustre-MDT0000: Not available for connect from 0@lo (stopping) 00000100:00100000:1.0:1776178507.622294:0:6256:0:(service.c:2372:ptlrpc_server_handle_request()) Handled RPC req@000000005f1c2da4 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_001:0+-99:3352:x1862456998091264:12345-0@lo:38:kworker/u8:1.0 Request processed in 31us (843us total) trans 0 rc -19/-19 00000100:00100000:1.0:1776178507.622303:0:6256:0:(nrs_fifo.c:241:nrs_fifo_req_stop()) NRS stop fifo request from 12345-0@lo, seq: 6862 00000100:00100000:1.0:1776178507.622314:0:6256:0:(nrs_fifo.c:179:nrs_fifo_req_get()) NRS start fifo request from 12345-0@lo, seq: 6863 00000100:00100000:1.0:1776178507.622317:0:6256:0:(service.c:2324:ptlrpc_server_handle_request()) Handling RPC req@0000000007e413b7 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_001:0+-99:3352:x1862456998091584:12345-0@lo:38:kworker/u8:1.0 00010000:02000000:1.0:1776178507.622322:0:6256:0:(ldlm_lib.c:1139:target_handle_connect()) lustre-MDT0000: Not available for connect from 0@lo (stopping) 00000100:00100000:1.0:1776178507.622351:0:6256:0:(service.c:2372:ptlrpc_server_handle_request()) Handled RPC req@0000000007e413b7 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_001:0+-99:3352:x1862456998091584:12345-0@lo:38:kworker/u8:1.0 Request processed in 33us (764us total) trans 0 rc -19/-19 00000100:00100000:1.0:1776178507.622356:0:6256:0:(nrs_fifo.c:241:nrs_fifo_req_stop()) NRS stop fifo request from 12345-0@lo, seq: 6863 00000100:00100000:1.0:1776178507.622379:0:14128:0:(service.c:2152:ptlrpc_server_handle_req_in()) got req x1862456998091392 00000100:00100000:1.0:1776178507.622400:0:14128:0:(nrs_fifo.c:179:nrs_fifo_req_get()) NRS start fifo request from 12345-0@lo, seq: 767 00000100:00100000:1.0:1776178507.622404:0:14128:0:(service.c:2324:ptlrpc_server_handle_request()) Handling RPC req@0000000057946847 pname:cluuid+ref:pid:xid:nid:opc:job mdt_out00_002:0+-99:3352:x1862456998091392:12345-0@lo:38:kworker/u8:1.0 00010000:02000000:1.0:1776178507.622411:0:14128:0:(ldlm_lib.c:1139:target_handle_connect()) lustre-MDT0000: Not available for connect from 0@lo (stopping) 00000100:00100000:1.0:1776178507.622434:0:14128:0:(service.c:2372:ptlrpc_server_handle_request()) Handled RPC req@0000000057946847 pname:cluuid+ref:pid:xid:nid:opc:job mdt_out00_002:0+-99:3352:x1862456998091392:12345-0@lo:38:kworker/u8:1.0 Request processed in 30us (897us total) trans 0 rc -19/-19 00000100:00100000:1.0:1776178507.622440:0:14128:0:(nrs_fifo.c:241:nrs_fifo_req_stop()) NRS stop fifo request from 12345-0@lo, seq: 767 00000100:00100000:1.0:1776178507.622507:0:3355:0:(client.c:2215:ptlrpc_check_set()) Completed RPC req@00000000ccad874e pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_02:lustre-MDT0001-lwp-OST0001_UUID:3355:1862456998091328:0@lo:400:kworker/u8:1.0 00000100:00080000:1.0:1776178507.622558:0:3352:0:(import.c:86:import_set_state_nolock()) 00000000dfc8db13 lustre-MDT0000_UUID: changing import state from CONNECTING to DISCONN 00000100:00080000:1.0:1776178507.622563:0:3352:0:(import.c:1429:ptlrpc_connect_interpret()) recovery of lustre-MDT0000_UUID on 192.168.204.103@tcp failed (-19) 00000100:00100000:1.0:1776178507.622567:0:3352:0:(client.c:2215:ptlrpc_check_set()) Completed RPC req@000000000a224757 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:lustre-MDT0000-lwp-OST0001_UUID:3352:1862456998091264:0@lo:38:kworker/u8:1.0 00000100:00080000:1.0:1776178507.622575:0:3352:0:(import.c:86:import_set_state_nolock()) 00000000ba4f1821 lustre-MDT0000_UUID: changing import state from CONNECTING to DISCONN 00000100:00080000:1.0:1776178507.622578:0:3352:0:(import.c:1429:ptlrpc_connect_interpret()) recovery of lustre-MDT0000_UUID on 192.168.204.103@tcp failed (-19) 00000100:00100000:1.0:1776178507.622581:0:3352:0:(client.c:2215:ptlrpc_check_set()) Completed RPC req@00000000d596a45c pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:lustre-MDT0001-mdtlov_UUID:3352:1862456998091392:0@lo:38:kworker/u8:1.0 00000100:00080000:1.0:1776178507.622589:0:3352:0:(import.c:86:import_set_state_nolock()) 00000000a7c55b1f lustre-MDT0000_UUID: changing import state from CONNECTING to DISCONN 00000100:00080000:1.0:1776178507.622592:0:3352:0:(import.c:1429:ptlrpc_connect_interpret()) recovery of lustre-MDT0000_UUID on 192.168.204.103@tcp failed (-19) 00000100:00100000:1.0:1776178507.622594:0:3352:0:(client.c:2215:ptlrpc_check_set()) Completed RPC req@000000002917d432 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:lustre-MDT0000-lwp-MDT0001_UUID:3352:1862456998091584:0@lo:38:kworker/u8:1.0 00000100:00080000:1.0:1776178507.622603:0:3352:0:(import.c:86:import_set_state_nolock()) 0000000047601a98 lustre-MDT0000_UUID: changing import state from CONNECTING to DISCONN 00000100:00080000:1.0:1776178507.622607:0:3352:0:(import.c:1429:ptlrpc_connect_interpret()) recovery of lustre-MDT0000_UUID on 192.168.204.103@tcp failed (-19) 00000100:00100000:1.0:1776178507.622609:0:3352:0:(client.c:2215:ptlrpc_check_set()) Completed RPC req@0000000054386dd3 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:lustre-MDT0000-lwp-OST0000_UUID:3352:1862456998091136:0@lo:38:kworker/u8:1.0 00010000:00010000:3.0:1776178508.258680:0:6244:0:(ldlm_lockd.c:2792:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:1.0:1776178509.152452:0:3355:0:(client.c:1733:ptlrpc_send_new_req()) Sending RPC req@00000000f9429027 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_02:lustre-MDT0001-mdtlov_UUID:3355:1862456998091648:0@lo:13:osp-pre-1-1.0 00000100:00100000:1.0:1776178509.152583:0:3355:0:(events.c:361:request_in_callback()) peer: 12345-0@lo (source: 12345-0@lo) 00000100:00100000:3.0:1776178509.152795:0:8231:0:(service.c:2152:ptlrpc_server_handle_req_in()) got req x1862456998091648 00000100:00100000:3.0:1776178509.152870:0:8231:0:(nrs_fifo.c:179:nrs_fifo_req_get()) NRS start fifo request from 12345-0@lo, seq: 607 00000100:00100000:3.0:1776178509.152879:0:8231:0:(service.c:2324:ptlrpc_server_handle_request()) Handling RPC req@0000000054386dd3 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_create00:lustre-MDT0001-mdtlov_UUID+5:3355:x1862456998091648:12345-0@lo:13:osp-pre-1-1.0 00002000:00000024:3.0:1776178509.152930:0:8231:0:(ofd_obd.c:740:ofd_statfs()) blocks cached 229376 granted 491746368 pending 0 free 3923689472 avail 3692167168 00000100:00100000:3.0:1776178509.153241:0:8231:0:(service.c:2372:ptlrpc_server_handle_request()) Handled RPC req@0000000054386dd3 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_create00:lustre-MDT0001-mdtlov_UUID+5:3355:x1862456998091648:12345-0@lo:13:osp-pre-1-1.0 Request processed in 361us (659us total) trans 0 rc 0/0 00000100:00100000:3.0:1776178509.153251:0:8231:0:(nrs_fifo.c:241:nrs_fifo_req_stop()) NRS stop fifo request from 12345-0@lo, seq: 607 00000100:00100000:1.0:1776178509.154575:0:3355:0:(client.c:2215:ptlrpc_check_set()) Completed RPC req@00000000f9429027 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_02:lustre-MDT0001-mdtlov_UUID:3355:1862456998091648:0@lo:13:osp-pre-1-1.0 00010000:00010000:2.0:1776178509.280601:0:6245:0:(ldlm_lockd.c:2792:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:1.0:1776178509.664469:0:3355:0:(ptlrpcd.c:415:ptlrpcd_check()) transfer 1 async RPCs [3->2] 00000100:00100000:1.0:1776178509.664513:0:3355:0:(client.c:1733:ptlrpc_send_new_req()) Sending RPC req@0000000045ff6e42 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_02:lustre-MDT0001-mdtlov_UUID:3355:1862456998091712:0@lo:13:osp-pre-0-1.0 00000100:00100000:1.0:1776178509.664635:0:3355:0:(events.c:361:request_in_callback()) peer: 12345-0@lo (source: 12345-0@lo) 00000100:00100000:3.0:1776178509.664788:0:8231:0:(service.c:2152:ptlrpc_server_handle_req_in()) got req x1862456998091712 00000100:00100000:3.0:1776178509.664881:0:8231:0:(nrs_fifo.c:179:nrs_fifo_req_get()) NRS start fifo request from 12345-0@lo, seq: 608 00000100:00100000:3.0:1776178509.664888:0:8231:0:(service.c:2324:ptlrpc_server_handle_request()) Handling RPC req@000000002917d432 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_create00:lustre-MDT0001-mdtlov_UUID+5:3355:x1862456998091712:12345-0@lo:13:osp-pre-0-1.0 00002000:00000024:3.0:1776178509.664934:0:8231:0:(ofd_obd.c:740:ofd_statfs()) blocks cached 315392 granted 484883776 pending 0 free 3923689472 avail 3692167168 00000100:00100000:3.0:1776178509.664991:0:8231:0:(service.c:2372:ptlrpc_server_handle_request()) Handled RPC req@000000002917d432 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_create00:lustre-MDT0001-mdtlov_UUID+5:3355:x1862456998091712:12345-0@lo:13:osp-pre-0-1.0 Request processed in 103us (359us total) trans 0 rc 0/0 00000100:00100000:3.0:1776178509.664998:0:8231:0:(nrs_fifo.c:241:nrs_fifo_req_stop()) NRS stop fifo request from 12345-0@lo, seq: 608 00000100:00100000:1.0:1776178509.665168:0:3355:0:(client.c:2215:ptlrpc_check_set()) Completed RPC req@0000000045ff6e42 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_02:lustre-MDT0001-mdtlov_UUID:3355:1862456998091712:0@lo:13:osp-pre-0-1.0 00010000:00010000:1.0:1776178510.306500:0:9224:0:(ldlm_lockd.c:2792:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:3.0:1776178511.140448:0:3347:0:(events.c:361:request_in_callback()) peer: 12345-192.168.204.3@tcp (source: 12345-192.168.204.3@tcp) 00000100:00100000:3.0:1776178511.140496:0:3347:0:(events.c:361:request_in_callback()) peer: 12345-192.168.204.3@tcp (source: 12345-192.168.204.3@tcp) 00000100:00100000:3.0:1776178511.140554:0:3347:0:(events.c:361:request_in_callback()) peer: 12345-192.168.204.3@tcp (source: 12345-192.168.204.3@tcp) 00000100:00100000:1.0:1776178511.140587:0:8250:0:(service.c:2152:ptlrpc_server_handle_req_in()) got req x1862456979128320 00000100:00100000:3.0:1776178511.140599:0:3347:0:(events.c:361:request_in_callback()) peer: 12345-192.168.204.3@tcp (source: 12345-192.168.204.3@tcp) 00000100:00100000:2.0:1776178511.140642:0:20759:0:(service.c:2152:ptlrpc_server_handle_req_in()) got req x1862456979128448 00000100:00100000:3.0:1776178511.140643:0:3347:0:(events.c:361:request_in_callback()) peer: 12345-192.168.204.3@tcp (source: 12345-192.168.204.3@tcp) 00000100:00100000:2.0:1776178511.140711:0:20759:0:(service.c:2152:ptlrpc_server_handle_req_in()) got req x1862456979128384 00000100:00100000:0.0:1776178511.140777:0:20140:0:(service.c:2152:ptlrpc_server_handle_req_in()) got req x1862456979128192 00000100:00100000:0.0:1776178511.140856:0:20140:0:(nrs_fifo.c:179:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.204.3@tcp, seq: 275 00000100:00100000:0.0:1776178511.140863:0:20140:0:(service.c:2324:ptlrpc_server_handle_request()) Handling RPC req@00000000dd6bfaa2 pname:cluuid+ref:pid:xid:nid:opc:job ll_mgs_0002:5b5c66b1-1a2f-4651-8548-0c75b9a38a94+9:2273:x1862456979128192:12345-192.168.204.3@tcp:400:kworker/u8:4.0 00000100:00100000:1.0:1776178511.140898:0:8250:0:(service.c:2152:ptlrpc_server_handle_req_in()) got req x1862456979128256 00000100:00100000:1.0:1776178511.140949:0:8250:0:(nrs_fifo.c:179:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.204.3@tcp, seq: 1108 00000100:00100000:1.0:1776178511.140958:0:8250:0:(service.c:2324:ptlrpc_server_handle_request()) Handling RPC req@0000000004d186b9 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_003:b50963bd-8eba-4432-9a54-8888d00e3fa1+6:2273:x1862456979128320:12345-192.168.204.3@tcp:400:kworker/u8:4.0 00000100:00100000:0.0:1776178511.140972:0:20140:0:(service.c:2372:ptlrpc_server_handle_request()) Handled RPC req@00000000dd6bfaa2 pname:cluuid+ref:pid:xid:nid:opc:job ll_mgs_0002:5b5c66b1-1a2f-4651-8548-0c75b9a38a94+9:2273:x1862456979128192:12345-192.168.204.3@tcp:400:kworker/u8:4.0 Request processed in 100us (412us total) trans 0 rc 0/0 00000100:00100000:0.0:1776178511.140984:0:20140:0:(nrs_fifo.c:241:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.204.3@tcp, seq: 275 00000100:00100000:0.0:1776178511.141044:0:6255:0:(nrs_fifo.c:179:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.204.3@tcp, seq: 6864 00000100:00100000:1.0:1776178511.141051:0:8250:0:(service.c:2372:ptlrpc_server_handle_request()) Handled RPC req@0000000004d186b9 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_003:b50963bd-8eba-4432-9a54-8888d00e3fa1+6:2273:x1862456979128320:12345-192.168.204.3@tcp:400:kworker/u8:4.0 Request processed in 88us (606us total) trans 0 rc 0/0 00000100:00100000:0.0:1776178511.141101:0:6255:0:(service.c:2324:ptlrpc_server_handle_request()) Handling RPC req@000000001516b27f pname:cluuid+ref:pid:xid:nid:opc:job mdt00_000:0+-99:2269:x1862456979128256:12345-192.168.204.3@tcp:38:kworker/u8:4.0 00010000:02000000:0.0:1776178511.141125:0:6255:0:(ldlm_lib.c:1139:target_handle_connect()) lustre-MDT0000: Not available for connect from 192.168.204.3@tcp (stopping) 00000100:00100000:1.0:1776178511.141126:0:8250:0:(nrs_fifo.c:241:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.204.3@tcp, seq: 1108 00000100:00100000:0.0:1776178511.141169:0:6255:0:(service.c:2372:ptlrpc_server_handle_request()) Handled RPC req@000000001516b27f pname:cluuid+ref:pid:xid:nid:opc:job mdt00_000:0+-99:2269:x1862456979128256:12345-192.168.204.3@tcp:38:kworker/u8:4.0 Request processed in 110us (524us total) trans 0 rc -19/-19 00000100:00100000:0.0:1776178511.141178:0:6255:0:(nrs_fifo.c:241:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.204.3@tcp, seq: 6864 00000100:00100000:3.0:1776178511.141636:0:14440:0:(nrs_fifo.c:179:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.204.3@tcp, seq: 751 00000100:00100000:3.0:1776178511.141643:0:14440:0:(service.c:2324:ptlrpc_server_handle_request()) Handling RPC req@000000003c23bd8e pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_004:b50963bd-8eba-4432-9a54-8888d00e3fa1+9:2273:x1862456979128448:12345-192.168.204.3@tcp:400:kworker/u8:4.0 00000100:00100000:3.0:1776178511.141701:0:14440:0:(service.c:2372:ptlrpc_server_handle_request()) Handled RPC req@000000003c23bd8e pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_004:b50963bd-8eba-4432-9a54-8888d00e3fa1+9:2273:x1862456979128448:12345-192.168.204.3@tcp:400:kworker/u8:4.0 Request processed in 59us (1206us total) trans 0 rc 0/0 00000100:00100000:3.0:1776178511.141709:0:14440:0:(nrs_fifo.c:241:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.204.3@tcp, seq: 751 00000100:00100000:3.0:1776178511.141730:0:14440:0:(nrs_fifo.c:179:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.204.3@tcp, seq: 752 00000100:00100000:3.0:1776178511.141735:0:14440:0:(service.c:2324:ptlrpc_server_handle_request()) Handling RPC req@00000000a9c256f6 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_004:b50963bd-8eba-4432-9a54-8888d00e3fa1+8:2273:x1862456979128384:12345-192.168.204.3@tcp:400:kworker/u8:4.0 00000100:00100000:3.0:1776178511.141760:0:14440:0:(service.c:2372:ptlrpc_server_handle_request()) Handled RPC req@00000000a9c256f6 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_004:b50963bd-8eba-4432-9a54-8888d00e3fa1+8:2273:x1862456979128384:12345-192.168.204.3@tcp:400:kworker/u8:4.0 Request processed in 25us (1164us total) trans 0 rc 0/0 00000100:00100000:3.0:1776178511.141767:0:14440:0:(nrs_fifo.c:241:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.204.3@tcp, seq: 752 00010000:00010000:1.0:1776178511.328375:0:9225:0:(ldlm_lockd.c:2792:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00010000:00010000:3.0:1776178512.352916:0:6244:0:(ldlm_lockd.c:2792:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00080000:3.0:1776178512.741300:0:36:0:(pinger.c:251:ptlrpc_pinger_process_import()) lustre-MDT0000-lwp-OST0000_UUID->lustre-MDT0000_UUID: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00080000:3.0:1776178512.741312:0:36:0:(recover.c:57:ptlrpc_initiate_recovery()) lustre-MDT0000_UUID: starting recovery 00000100:00080000:3.0:1776178512.741317:0:36:0:(import.c:86:import_set_state_nolock()) 0000000047601a98 lustre-MDT0000_UUID: changing import state from DISCONN to CONNECTING 00000100:00080000:3.0:1776178512.741330:0:36:0:(import.c:534:import_select_connection()) lustre-MDT0000-lwp-OST0000: connect to NID 0@lo last attempt 1214 00000100:00080000:3.0:1776178512.741336:0:36:0:(import.c:581:import_select_connection()) lustre-MDT0000-lwp-OST0000: tried all connections, increasing latency to 21s 00000100:00080000:3.0:1776178512.741346:0:36:0:(import.c:615:import_select_connection()) lustre-MDT0000-lwp-OST0000: import 0000000047601a98 using connection 192.168.204.103@tcp/0@lo 00000100:00100000:3.0:1776178512.741376:0:36:0:(import.c:807:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5) req@00000000a55cac67 x1862456998091840/t0(0) o38->lustre-MDT0000-lwp-OST0000@0@lo:12/10 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' 00000100:00080000:3.0:1776178512.741420:0:36:0:(pinger.c:251:ptlrpc_pinger_process_import()) lustre-MDT0000-lwp-OST0001_UUID->lustre-MDT0000_UUID: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00080000:3.0:1776178512.741424:0:36:0:(recover.c:57:ptlrpc_initiate_recovery()) lustre-MDT0000_UUID: starting recovery 00000100:00080000:3.0:1776178512.741425:0:36:0:(import.c:86:import_set_state_nolock()) 00000000dfc8db13 lustre-MDT0000_UUID: changing import state from DISCONN to CONNECTING 00000100:00080000:3.0:1776178512.741429:0:36:0:(import.c:534:import_select_connection()) lustre-MDT0000-lwp-OST0001: connect to NID 0@lo last attempt 1214 00000100:00080000:3.0:1776178512.741431:0:36:0:(import.c:581:import_select_connection()) lustre-MDT0000-lwp-OST0001: tried all connections, increasing latency to 21s 00000100:00080000:3.0:1776178512.741435:0:36:0:(import.c:615:import_select_connection()) lustre-MDT0000-lwp-OST0001: import 00000000dfc8db13 using connection 192.168.204.103@tcp/0@lo 00000100:00100000:3.0:1776178512.741445:0:36:0:(import.c:807:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5) req@000000008584c293 x1862456998091968/t0(0) o38->lustre-MDT0000-lwp-OST0001@0@lo:12/10 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' 00000100:00080000:3.0:1776178512.741473:0:36:0:(pinger.c:251:ptlrpc_pinger_process_import()) lustre-MDT0001-mdtlov_UUID->lustre-MDT0000_UUID: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00080000:3.0:1776178512.741478:0:36:0:(recover.c:57:ptlrpc_initiate_recovery()) lustre-MDT0000_UUID: starting recovery 00000100:00080000:3.0:1776178512.741480:0:36:0:(import.c:86:import_set_state_nolock()) 00000000ba4f1821 lustre-MDT0000_UUID: changing import state from DISCONN to CONNECTING 00000100:00080000:3.0:1776178512.741483:0:36:0:(import.c:534:import_select_connection()) lustre-MDT0000-osp-MDT0001: connect to NID 0@lo last attempt 1214 00000100:00080000:3.0:1776178512.741485:0:36:0:(import.c:581:import_select_connection()) lustre-MDT0000-osp-MDT0001: tried all connections, increasing latency to 21s 00000100:00080000:3.0:1776178512.741489:0:36:0:(import.c:615:import_select_connection()) lustre-MDT0000-osp-MDT0001: import 00000000ba4f1821 using connection 192.168.204.103@tcp/0@lo 00000100:00100000:3.0:1776178512.741501:0:36:0:(import.c:807:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5) req@0000000067f71a07 x1862456998092096/t0(0) o38->lustre-MDT0000-osp-MDT0001@0@lo:24/4 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' 00000100:00100000:2.0:1776178512.741520:0:3352:0:(client.c:1733:ptlrpc_send_new_req()) Sending RPC req@00000000a55cac67 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:lustre-MDT0000-lwp-OST0000_UUID:3352:1862456998091840:0@lo:38:kworker/u8:1.0 00000100:00080000:3.0:1776178512.741526:0:36:0:(pinger.c:251:ptlrpc_pinger_process_import()) lustre-MDT0000-lwp-MDT0001_UUID->lustre-MDT0000_UUID: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00080000:3.0:1776178512.741530:0:36:0:(recover.c:57:ptlrpc_initiate_recovery()) lustre-MDT0000_UUID: starting recovery 00000100:00080000:3.0:1776178512.741531:0:36:0:(import.c:86:import_set_state_nolock()) 00000000a7c55b1f lustre-MDT0000_UUID: changing import state from DISCONN to CONNECTING 00000100:00080000:3.0:1776178512.741534:0:36:0:(import.c:534:import_select_connection()) lustre-MDT0000-lwp-MDT0001: connect to NID 0@lo last attempt 1214 00000100:00080000:3.0:1776178512.741537:0:36:0:(import.c:581:import_select_connection()) lustre-MDT0000-lwp-MDT0001: tried all connections, increasing latency to 21s 00000100:00080000:3.0:1776178512.741540:0:36:0:(import.c:615:import_select_connection()) lustre-MDT0000-lwp-MDT0001: import 00000000a7c55b1f using connection 192.168.204.103@tcp/0@lo 00000100:00100000:3.0:1776178512.741551:0:36:0:(import.c:807:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5) req@00000000d4b43163 x1862456998092288/t0(0) o38->lustre-MDT0000-lwp-MDT0001@0@lo:12/10 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' 00000100:00100000:3.0:1776178512.741616:0:3356:0:(client.c:1733:ptlrpc_send_new_req()) Sending RPC req@0000000098006deb pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_03:lustre-MDT0001-mdtlov_UUID:3356:1862456998092160:0@lo:400:kworker/u8:1.0 00000100:00100000:1.0:1776178512.741701:0:3354:0:(client.c:1733:ptlrpc_send_new_req()) Sending RPC req@000000006154cc2b pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_01:lustre-MDT0001-lwp-OST0000_UUID:3354:1862456998091904:0@lo:400:kworker/u8:1.0 00000100:00100000:2.0:1776178512.741745:0:3352:0:(events.c:361:request_in_callback()) peer: 12345-0@lo (source: 12345-0@lo) 00000100:00100000:0.0:1776178512.741912:0:3353:0:(client.c:1733:ptlrpc_send_new_req()) Sending RPC req@00000000f52236ad pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_00:ac61f839-0e8e-4cac-9c59-08bcfbbad330:3353:1862456998091776:0@lo:400:kworker/u8:1.0 00000100:00100000:3.0:1776178512.741951:0:3356:0:(events.c:361:request_in_callback()) peer: 12345-0@lo (source: 12345-0@lo) 00000100:00100000:1.0:1776178512.745419:0:3355:0:(client.c:1733:ptlrpc_send_new_req()) Sending RPC req@00000000bea25c66 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_02:lustre-MDT0001-lwp-OST0001_UUID:3355:1862456998092032:0@lo:400:kworker/u8:1.0 00000100:00100000:1.0:1776178512.745504:0:6257:0:(service.c:2152:ptlrpc_server_handle_req_in()) got req x1862456998091840 00000100:00100000:2.0:1776178512.746006:0:3352:0:(client.c:1733:ptlrpc_send_new_req()) Sending RPC req@000000008584c293 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:lustre-MDT0000-lwp-OST0001_UUID:3352:1862456998091968:0@lo:38:kworker/u8:1.0 00000100:00100000:2.0:1776178512.746110:0:3352:0:(events.c:361:request_in_callback()) peer: 12345-0@lo (source: 12345-0@lo) 00000100:00100000:2.0:1776178512.746144:0:3352:0:(client.c:1733:ptlrpc_send_new_req()) Sending RPC req@0000000067f71a07 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:lustre-MDT0001-mdtlov_UUID:3352:1862456998092096:0@lo:38:kworker/u8:1.0 00000100:00100000:0.0:1776178512.746187:0:6255:0:(service.c:2152:ptlrpc_server_handle_req_in()) got req x1862456998091968 00000100:00100000:0.0:1776178512.746222:0:6255:0:(nrs_fifo.c:179:nrs_fifo_req_get()) NRS start fifo request from 12345-0@lo, seq: 6865 00000100:00100000:0.0:1776178512.746232:0:6255:0:(service.c:2324:ptlrpc_server_handle_request()) Handling RPC req@000000007309261c pname:cluuid+ref:pid:xid:nid:opc:job mdt00_000:0+-99:3352:x1862456998091840:12345-0@lo:38:kworker/u8:1.0 00000100:00100000:2.0:1776178512.746488:0:3352:0:(events.c:361:request_in_callback()) peer: 12345-0@lo (source: 12345-0@lo) 00000100:00100000:2.0:1776178512.746515:0:3352:0:(client.c:1733:ptlrpc_send_new_req()) Sending RPC req@00000000d4b43163 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:lustre-MDT0000-lwp-MDT0001_UUID:3352:1862456998092288:0@lo:38:kworker/u8:1.0 00010000:02000000:0.0:1776178512.746524:0:6255:0:(ldlm_lib.c:1139:target_handle_connect()) lustre-MDT0000: Not available for connect from 0@lo (stopping) 00000100:00100000:2.0:1776178512.746547:0:3352:0:(events.c:361:request_in_callback()) peer: 12345-0@lo (source: 12345-0@lo) 00000100:00100000:1.0:1776178512.746735:0:6257:0:(service.c:2152:ptlrpc_server_handle_req_in()) got req x1862456998092288 00000100:00100000:1.0:1776178512.746888:0:6257:0:(nrs_fifo.c:179:nrs_fifo_req_get()) NRS start fifo request from 12345-0@lo, seq: 6866 00000100:00100000:1.0:1776178512.746893:0:6257:0:(service.c:2324:ptlrpc_server_handle_request()) Handling RPC req@00000000b069e162 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_002:0+-99:3352:x1862456998091968:12345-0@lo:38:kworker/u8:1.0 00010000:02000000:1.0:1776178512.746902:0:6257:0:(ldlm_lib.c:1139:target_handle_connect()) lustre-MDT0000: Not available for connect from 0@lo (stopping) 00000100:00100000:1.0:1776178512.746957:0:6257:0:(service.c:2372:ptlrpc_server_handle_request()) Handled RPC req@00000000b069e162 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_002:0+-99:3352:x1862456998091968:12345-0@lo:38:kworker/u8:1.0 Request processed in 63us (850us total) trans 0 rc -19/-19 00000100:00100000:1.0:1776178512.746965:0:6257:0:(nrs_fifo.c:241:nrs_fifo_req_stop()) NRS stop fifo request from 12345-0@lo, seq: 6866 00000100:00100000:1.0:1776178512.746991:0:6257:0:(nrs_fifo.c:179:nrs_fifo_req_get()) NRS start fifo request from 12345-0@lo, seq: 6867 00000100:00100000:1.0:1776178512.746995:0:6257:0:(service.c:2324:ptlrpc_server_handle_request()) Handling RPC req@00000000722afaa5 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_002:0+-99:3352:x1862456998092288:12345-0@lo:38:kworker/u8:1.0 00010000:02000000:1.0:1776178512.747002:0:6257:0:(ldlm_lib.c:1139:target_handle_connect()) lustre-MDT0000: Not available for connect from 0@lo (stopping) 00000100:00080000:2.0:1776178512.747055:0:3352:0:(import.c:86:import_set_state_nolock()) 00000000dfc8db13 lustre-MDT0000_UUID: changing import state from CONNECTING to DISCONN 00000100:00080000:2.0:1776178512.747084:0:3352:0:(import.c:1429:ptlrpc_connect_interpret()) recovery of lustre-MDT0000_UUID on 192.168.204.103@tcp failed (-19) 00000100:00100000:2.0:1776178512.747092:0:3352:0:(client.c:2215:ptlrpc_check_set()) Completed RPC req@000000008584c293 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:lustre-MDT0000-lwp-OST0001_UUID:3352:1862456998091968:0@lo:38:kworker/u8:1.0 00000100:00100000:0.0:1776178512.747138:0:6255:0:(service.c:2372:ptlrpc_server_handle_request()) Handled RPC req@000000007309261c pname:cluuid+ref:pid:xid:nid:opc:job mdt00_000:0+-99:3352:x1862456998091840:12345-0@lo:38:kworker/u8:1.0 Request processed in 905us (5485us total) trans 0 rc -19/-19 00000100:00080000:2.0:1776178512.747140:0:3352:0:(import.c:86:import_set_state_nolock()) 0000000047601a98 lustre-MDT0000_UUID: changing import state from CONNECTING to DISCONN 00000100:00100000:0.0:1776178512.747148:0:6255:0:(nrs_fifo.c:241:nrs_fifo_req_stop()) NRS stop fifo request from 12345-0@lo, seq: 6865 00000100:00080000:2.0:1776178512.747148:0:3352:0:(import.c:1429:ptlrpc_connect_interpret()) recovery of lustre-MDT0000_UUID on 192.168.204.103@tcp failed (-19) 00000100:00100000:1.0:1776178512.747154:0:6257:0:(service.c:2372:ptlrpc_server_handle_request()) Handled RPC req@00000000722afaa5 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_002:0+-99:3352:x1862456998092288:12345-0@lo:38:kworker/u8:1.0 Request processed in 157us (607us total) trans 0 rc -19/-19 00000100:00100000:2.0:1776178512.747156:0:3352:0:(client.c:2215:ptlrpc_check_set()) Completed RPC req@00000000a55cac67 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:lustre-MDT0000-lwp-OST0000_UUID:3352:1862456998091840:0@lo:38:kworker/u8:1.0 00000100:00100000:1.0:1776178512.747164:0:6257:0:(nrs_fifo.c:241:nrs_fifo_req_stop()) NRS stop fifo request from 12345-0@lo, seq: 6867 00000100:00080000:2.0:1776178512.747182:0:3352:0:(import.c:86:import_set_state_nolock()) 00000000a7c55b1f lustre-MDT0000_UUID: changing import state from CONNECTING to DISCONN 00000100:00080000:2.0:1776178512.747187:0:3352:0:(import.c:1429:ptlrpc_connect_interpret()) recovery of lustre-MDT0000_UUID on 192.168.204.103@tcp failed (-19) 00000100:00100000:2.0:1776178512.747191:0:3352:0:(client.c:2215:ptlrpc_check_set()) Completed RPC req@00000000d4b43163 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:lustre-MDT0000-lwp-MDT0001_UUID:3352:1862456998092288:0@lo:38:kworker/u8:1.0 00000100:00100000:1.0:1776178512.747209:0:15213:0:(service.c:2152:ptlrpc_server_handle_req_in()) got req x1862456998092096 00000100:00100000:1.0:1776178512.747238:0:15213:0:(nrs_fifo.c:179:nrs_fifo_req_get()) NRS start fifo request from 12345-0@lo, seq: 768 00000100:00100000:1.0:1776178512.747243:0:15213:0:(service.c:2324:ptlrpc_server_handle_request()) Handling RPC req@0000000091f8240c pname:cluuid+ref:pid:xid:nid:opc:job mdt_out00_003:0+-99:3352:x1862456998092096:12345-0@lo:38:kworker/u8:1.0 00010000:02000000:1.0:1776178512.747253:0:15213:0:(ldlm_lib.c:1139:target_handle_connect()) lustre-MDT0000: Not available for connect from 0@lo (stopping) 00000100:00100000:2.0:1776178512.747256:0:20755:0:(service.c:2152:ptlrpc_server_handle_req_in()) got req x1862456998092160 00000100:00100000:1.0:1776178512.747295:0:15213:0:(service.c:2372:ptlrpc_server_handle_request()) Handled RPC req@0000000091f8240c pname:cluuid+ref:pid:xid:nid:opc:job mdt_out00_003:0+-99:3352:x1862456998092096:12345-0@lo:38:kworker/u8:1.0 Request processed in 51us (809us total) trans 0 rc -19/-19 00000100:00100000:1.0:1776178512.747303:0:15213:0:(nrs_fifo.c:241:nrs_fifo_req_stop()) NRS stop fifo request from 12345-0@lo, seq: 768 00000100:00100000:2.0:1776178512.747320:0:20755:0:(nrs_fifo.c:179:nrs_fifo_req_get()) NRS start fifo request from 12345-0@lo, seq: 753 00000100:00100000:2.0:1776178512.747326:0:20755:0:(service.c:2324:ptlrpc_server_handle_request()) Handling RPC req@000000005fb6b792 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_006:lustre-MDT0001-mdtlov_UUID+5:3356:x1862456998092160:12345-0@lo:400:kworker/u8:1.0 00000100:00100000:1.0:1776178512.747340:0:3355:0:(events.c:361:request_in_callback()) peer: 12345-0@lo (source: 12345-0@lo) 00000100:00100000:1.0:1776178512.747390:0:8250:0:(service.c:2152:ptlrpc_server_handle_req_in()) got req x1862456998092032 00000100:00100000:2.0:1776178512.747395:0:20755:0:(service.c:2372:ptlrpc_server_handle_request()) Handled RPC req@000000005fb6b792 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_006:lustre-MDT0001-mdtlov_UUID+5:3356:x1862456998092160:12345-0@lo:400:kworker/u8:1.0 Request processed in 68us (5444us total) trans 0 rc 0/0 00000100:00100000:2.0:1776178512.747403:0:20755:0:(nrs_fifo.c:241:nrs_fifo_req_stop()) NRS stop fifo request from 12345-0@lo, seq: 753 00000100:00100000:1.0:1776178512.747425:0:8250:0:(nrs_fifo.c:179:nrs_fifo_req_get()) NRS start fifo request from 12345-0@lo, seq: 1109 00000100:00100000:1.0:1776178512.747431:0:8250:0:(service.c:2324:ptlrpc_server_handle_request()) Handling RPC req@00000000d596a45c pname:cluuid+ref:pid:xid:nid:opc:job mdt00_003:lustre-MDT0001-lwp-OST0001_UUID+5:3355:x1862456998092032:12345-0@lo:400:kworker/u8:1.0 00000100:00100000:1.0:1776178512.747484:0:8250:0:(service.c:2372:ptlrpc_server_handle_request()) Handled RPC req@00000000d596a45c pname:cluuid+ref:pid:xid:nid:opc:job mdt00_003:lustre-MDT0001-lwp-OST0001_UUID+5:3355:x1862456998092032:12345-0@lo:400:kworker/u8:1.0 Request processed in 52us (145us total) trans 0 rc 0/0 00000100:00100000:1.0:1776178512.747490:0:8250:0:(nrs_fifo.c:241:nrs_fifo_req_stop()) NRS stop fifo request from 12345-0@lo, seq: 1109 00000100:00100000:1.0:1776178512.747534:0:3355:0:(client.c:2215:ptlrpc_check_set()) Completed RPC req@00000000bea25c66 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_02:lustre-MDT0001-lwp-OST0001_UUID:3355:1862456998092032:0@lo:400:kworker/u8:1.0 00000100:00100000:1.0:1776178512.747563:0:3354:0:(events.c:361:request_in_callback()) peer: 12345-0@lo (source: 12345-0@lo) 00000100:00100000:3.0:1776178512.748111:0:3356:0:(client.c:2215:ptlrpc_check_set()) Completed RPC req@0000000098006deb pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_03:lustre-MDT0001-mdtlov_UUID:3356:1862456998092160:0@lo:400:kworker/u8:1.0 00000100:00100000:3.0:1776178512.748144:0:11281:0:(service.c:2152:ptlrpc_server_handle_req_in()) got req x1862456998091904 00000100:00100000:3.0:1776178512.748177:0:11281:0:(nrs_fifo.c:179:nrs_fifo_req_get()) NRS start fifo request from 12345-0@lo, seq: 1110 00000100:00100000:3.0:1776178512.748181:0:11281:0:(service.c:2324:ptlrpc_server_handle_request()) Handling RPC req@000000000a224757 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_004:lustre-MDT0001-lwp-OST0000_UUID+5:3354:x1862456998091904:12345-0@lo:400:kworker/u8:1.0 00000100:00100000:3.0:1776178512.748232:0:11281:0:(service.c:2372:ptlrpc_server_handle_request()) Handled RPC req@000000000a224757 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_004:lustre-MDT0001-lwp-OST0000_UUID+5:3354:x1862456998091904:12345-0@lo:400:kworker/u8:1.0 Request processed in 50us (670us total) trans 0 rc 0/0 00000100:00100000:0.0:1776178512.748471:0:3353:0:(events.c:361:request_in_callback()) peer: 12345-0@lo (source: 12345-0@lo) 00000100:00100000:0.0:1776178512.748496:0:3353:0:(client.c:1733:ptlrpc_send_new_req()) Sending RPC req@000000003ecedfd0 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_00:lustre-MDT0001-mdtlov_UUID:3353:1862456998092224:0@lo:400:kworker/u8:1.0 00000100:00100000:0.0:1776178512.748531:0:3353:0:(events.c:361:request_in_callback()) peer: 12345-0@lo (source: 12345-0@lo) 00000100:00080000:0.0:1776178512.749179:0:3352:0:(import.c:86:import_set_state_nolock()) 00000000ba4f1821 lustre-MDT0000_UUID: changing import state from CONNECTING to DISCONN 00000100:00080000:0.0:1776178512.749188:0:3352:0:(import.c:1429:ptlrpc_connect_interpret()) recovery of lustre-MDT0000_UUID on 192.168.204.103@tcp failed (-19) 00000100:00100000:0.0:1776178512.749195:0:3352:0:(client.c:2215:ptlrpc_check_set()) Completed RPC req@0000000067f71a07 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:lustre-MDT0001-mdtlov_UUID:3352:1862456998092096:0@lo:38:kworker/u8:1.0 00000100:00100000:3.0:1776178512.749356:0:11281:0:(nrs_fifo.c:241:nrs_fifo_req_stop()) NRS stop fifo request from 12345-0@lo, seq: 1110 00000100:00100000:0.0:1776178512.749421:0:3354:0:(client.c:2215:ptlrpc_check_set()) Completed RPC req@000000006154cc2b pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_01:lustre-MDT0001-lwp-OST0000_UUID:3354:1862456998091904:0@lo:400:kworker/u8:1.0 00000100:00100000:1.0:1776178512.749561:0:20137:0:(service.c:2152:ptlrpc_server_handle_req_in()) got req x1862456998091776 00000100:00100000:1.0:1776178512.749882:0:20137:0:(nrs_fifo.c:179:nrs_fifo_req_get()) NRS start fifo request from 12345-0@lo, seq: 276 00000100:00100000:1.0:1776178512.749888:0:20137:0:(service.c:2324:ptlrpc_server_handle_request()) Handling RPC req@00000000ddaf121b pname:cluuid+ref:pid:xid:nid:opc:job ll_mgs_0000:ac61f839-0e8e-4cac-9c59-08bcfbbad330+18:3353:x1862456998091776:12345-0@lo:400:kworker/u8:1.0 00000100:00100000:1.0:1776178512.749971:0:20137:0:(service.c:2372:ptlrpc_server_handle_request()) Handled RPC req@00000000ddaf121b pname:cluuid+ref:pid:xid:nid:opc:job ll_mgs_0000:ac61f839-0e8e-4cac-9c59-08bcfbbad330+18:3353:x1862456998091776:12345-0@lo:400:kworker/u8:1.0 Request processed in 83us (1501us total) trans 0 rc 0/0 00000100:00100000:1.0:1776178512.749978:0:20137:0:(nrs_fifo.c:241:nrs_fifo_req_stop()) NRS stop fifo request from 12345-0@lo, seq: 276 00000100:00100000:3.0:1776178512.753253:0:14440:0:(service.c:2152:ptlrpc_server_handle_req_in()) got req x1862456998092224 00000100:00100000:3.0:1776178512.753357:0:14440:0:(nrs_fifo.c:179:nrs_fifo_req_get()) NRS start fifo request from 12345-0@lo, seq: 754 00000100:00100000:3.0:1776178512.753366:0:14440:0:(service.c:2324:ptlrpc_server_handle_request()) Handling RPC req@0000000022264af5 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_004:lustre-MDT0001-mdtlov_UUID+5:3353:x1862456998092224:12345-0@lo:400:kworker/u8:1.0 00000100:00100000:3.0:1776178512.753478:0:14440:0:(service.c:2372:ptlrpc_server_handle_request()) Handled RPC req@0000000022264af5 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_004:lustre-MDT0001-mdtlov_UUID+5:3353:x1862456998092224:12345-0@lo:400:kworker/u8:1.0 Request processed in 112us (4945us total) trans 0 rc 0/0 00000100:00100000:3.0:1776178512.753484:0:14440:0:(nrs_fifo.c:241:nrs_fifo_req_stop()) NRS stop fifo request from 12345-0@lo, seq: 754 00000100:00100000:0.0:1776178512.753945:0:3353:0:(client.c:2215:ptlrpc_check_set()) Completed RPC req@00000000f52236ad pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_00:ac61f839-0e8e-4cac-9c59-08bcfbbad330:3353:1862456998091776:0@lo:400:kworker/u8:1.0 00000100:00100000:0.0:1776178512.753967:0:3353:0:(client.c:2215:ptlrpc_check_set()) Completed RPC req@000000003ecedfd0 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_00:lustre-MDT0001-mdtlov_UUID:3353:1862456998092224:0@lo:400:kworker/u8:1.0 00010000:00010000:2.0:1776178513.376586:0:6245:0:(ldlm_lockd.c:2792:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:1.0:1776178514.274207:0:3353:0:(ptlrpcd.c:415:ptlrpcd_check()) transfer 1 async RPCs [1->0] 00000100:00100000:1.0:1776178514.274233:0:3353:0:(client.c:1733:ptlrpc_send_new_req()) Sending RPC req@0000000098006deb pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_00:lustre-MDT0001-mdtlov_UUID:3353:1862456998092352:0@lo:13:osp-pre-1-1.0 00000100:00100000:1.0:1776178514.274308:0:3353:0:(events.c:361:request_in_callback()) peer: 12345-0@lo (source: 12345-0@lo) 00000100:00100000:2.0:1776178514.274691:0:9554:0:(service.c:2152:ptlrpc_server_handle_req_in()) got req x1862456998092352 00000100:00100000:1.0:1776178514.274914:0:8231:0:(nrs_fifo.c:179:nrs_fifo_req_get()) NRS start fifo request from 12345-0@lo, seq: 609 00000100:00100000:1.0:1776178514.274923:0:8231:0:(service.c:2324:ptlrpc_server_handle_request()) Handling RPC req@00000000ccad874e pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_create00:lustre-MDT0001-mdtlov_UUID+5:3353:x1862456998092352:12345-0@lo:13:osp-pre-1-1.0 00002000:00000024:1.0:1776178514.274972:0:8231:0:(ofd_obd.c:740:ofd_statfs()) blocks cached 229376 granted 491746368 pending 0 free 3923689472 avail 3692167168 00000100:00100000:1.0:1776178514.275094:0:8231:0:(service.c:2372:ptlrpc_server_handle_request()) Handled RPC req@00000000ccad874e pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_create00:lustre-MDT0001-mdtlov_UUID+5:3353:x1862456998092352:12345-0@lo:13:osp-pre-1-1.0 Request processed in 170us (786us total) trans 0 rc 0/0 00000100:00100000:1.0:1776178514.275101:0:8231:0:(nrs_fifo.c:241:nrs_fifo_req_stop()) NRS stop fifo request from 12345-0@lo, seq: 609 00000100:00100000:3.0:1776178514.275714:0:3353:0:(client.c:2215:ptlrpc_check_set()) Completed RPC req@0000000098006deb pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_00:lustre-MDT0001-mdtlov_UUID:3353:1862456998092352:0@lo:13:osp-pre-1-1.0 00010000:00010000:1.0:1776178514.403154:0:9224:0:(ldlm_lockd.c:2792:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:3.0:1776178514.786393:0:3356:0:(ptlrpcd.c:415:ptlrpcd_check()) transfer 1 async RPCs [2->3] 00000100:00100000:3.0:1776178514.786427:0:3356:0:(client.c:1733:ptlrpc_send_new_req()) Sending RPC req@00000000a101cbf3 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_03:lustre-MDT0001-mdtlov_UUID:3356:1862456998092416:0@lo:13:osp-pre-0-1.0 00000100:00100000:3.0:1776178514.786552:0:3356:0:(events.c:361:request_in_callback()) peer: 12345-0@lo (source: 12345-0@lo) 00000100:00100000:2.0:1776178514.786717:0:9554:0:(service.c:2152:ptlrpc_server_handle_req_in()) got req x1862456998092416 00000100:00100000:2.0:1776178514.786794:0:9554:0:(nrs_fifo.c:179:nrs_fifo_req_get()) NRS start fifo request from 12345-0@lo, seq: 610 00000100:00100000:2.0:1776178514.786801:0:9554:0:(service.c:2324:ptlrpc_server_handle_request()) Handling RPC req@0000000098006deb pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_create00:lustre-MDT0001-mdtlov_UUID+5:3356:x1862456998092416:12345-0@lo:13:osp-pre-0-1.0 00002000:00000024:2.0:1776178514.786838:0:9554:0:(ofd_obd.c:740:ofd_statfs()) blocks cached 315392 granted 484883776 pending 0 free 3923689472 avail 3692167168 00000100:00100000:2.0:1776178514.786895:0:9554:0:(service.c:2372:ptlrpc_server_handle_request()) Handled RPC req@0000000098006deb pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_create00:lustre-MDT0001-mdtlov_UUID+5:3356:x1862456998092416:12345-0@lo:13:osp-pre-0-1.0 Request processed in 94us (345us total) trans 0 rc 0/0 00000100:00100000:2.0:1776178514.786904:0:9554:0:(nrs_fifo.c:241:nrs_fifo_req_stop()) NRS stop fifo request from 12345-0@lo, seq: 610 00000100:00100000:0.0:1776178514.787018:0:3356:0:(client.c:2215:ptlrpc_check_set()) Completed RPC req@00000000a101cbf3 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_03:lustre-MDT0001-mdtlov_UUID:3356:1862456998092416:0@lo:13:osp-pre-0-1.0 00010000:00010000:1.0:1776178515.428466:0:9225:0:(ldlm_lockd.c:2792:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:3.0:1776178516.259281:0:3347:0:(events.c:361:request_in_callback()) peer: 12345-192.168.204.3@tcp (source: 12345-192.168.204.3@tcp) 00000100:00100000:0.0:1776178516.259785:0:11281:0:(service.c:2152:ptlrpc_server_handle_req_in()) got req x1862456979128576 00000100:00100000:0.0:1776178516.259853:0:11281:0:(nrs_fifo.c:179:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.204.3@tcp, seq: 6868 00000100:00100000:0.0:1776178516.259864:0:11281:0:(service.c:2324:ptlrpc_server_handle_request()) Handling RPC req@00000000034df97a pname:cluuid+ref:pid:xid:nid:opc:job mdt00_004:0+-99:2269:x1862456979128576:12345-192.168.204.3@tcp:38:kworker/u8:3.0 00010000:02000000:0.0:1776178516.259890:0:11281:0:(ldlm_lib.c:1139:target_handle_connect()) lustre-MDT0000: Not available for connect from 192.168.204.3@tcp (stopping) 00000100:00100000:0.0:1776178516.260652:0:11281:0:(service.c:2372:ptlrpc_server_handle_request()) Handled RPC req@00000000034df97a pname:cluuid+ref:pid:xid:nid:opc:job mdt00_004:0+-99:2269:x1862456979128576:12345-192.168.204.3@tcp:38:kworker/u8:3.0 Request processed in 786us (1381us total) trans 0 rc -19/-19 00000100:00100000:0.0:1776178516.260664:0:11281:0:(nrs_fifo.c:241:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.204.3@tcp, seq: 6868 00000100:00100000:1.0:1776178516.269614:0:3348:0:(events.c:361:request_in_callback()) peer: 12345-192.168.204.3@tcp (source: 12345-192.168.204.3@tcp) 00000100:00100000:0.0:1776178516.269745:0:20140:0:(service.c:2152:ptlrpc_server_handle_req_in()) got req x1862456979128512 00000100:00100000:0.0:1776178516.269825:0:20140:0:(nrs_fifo.c:179:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.204.3@tcp, seq: 277 00000100:00100000:0.0:1776178516.269833:0:20140:0:(service.c:2324:ptlrpc_server_handle_request()) Handling RPC req@000000003ef1d595 pname:cluuid+ref:pid:xid:nid:opc:job ll_mgs_0002:5b5c66b1-1a2f-4651-8548-0c75b9a38a94+9:2272:x1862456979128512:12345-192.168.204.3@tcp:400:kworker/u8:3.0 00000100:00100000:0.0:1776178516.269933:0:20140:0:(service.c:2372:ptlrpc_server_handle_request()) Handled RPC req@000000003ef1d595 pname:cluuid+ref:pid:xid:nid:opc:job ll_mgs_0002:5b5c66b1-1a2f-4651-8548-0c75b9a38a94+9:2272:x1862456979128512:12345-192.168.204.3@tcp:400:kworker/u8:3.0 Request processed in 100us (328us total) trans 0 rc 0/0 00000100:00100000:0.0:1776178516.269941:0:20140:0:(nrs_fifo.c:241:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.204.3@tcp, seq: 277 00000100:00100000:3.0:1776178516.275625:0:3346:0:(events.c:361:request_in_callback()) peer: 12345-192.168.204.3@tcp (source: 12345-192.168.204.3@tcp) 00000100:00100000:3.0:1776178516.275672:0:3346:0:(events.c:361:request_in_callback()) peer: 12345-192.168.204.3@tcp (source: 12345-192.168.204.3@tcp) 00000100:00100000:3.0:1776178516.276101:0:3346:0:(events.c:361:request_in_callback()) peer: 12345-192.168.204.3@tcp (source: 12345-192.168.204.3@tcp) 00000100:00100000:2.0:1776178516.276441:0:20762:0:(service.c:2152:ptlrpc_server_handle_req_in()) got req x1862456979128704 00000100:00100000:2.0:1776178516.276512:0:20762:0:(service.c:2152:ptlrpc_server_handle_req_in()) got req x1862456979128768 00000100:00100000:2.0:1776178516.276538:0:20762:0:(nrs_fifo.c:179:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.204.3@tcp, seq: 755 00000100:00100000:2.0:1776178516.276545:0:20762:0:(service.c:2324:ptlrpc_server_handle_request()) Handling RPC req@00000000c3bac325 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_013:b50963bd-8eba-4432-9a54-8888d00e3fa1+8:2272:x1862456979128704:12345-192.168.204.3@tcp:400:kworker/u8:3.0 00000100:00100000:2.0:1776178516.276635:0:20762:0:(service.c:2372:ptlrpc_server_handle_request()) Handled RPC req@00000000c3bac325 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_013:b50963bd-8eba-4432-9a54-8888d00e3fa1+8:2272:x1862456979128704:12345-192.168.204.3@tcp:400:kworker/u8:3.0 Request processed in 90us (1021us total) trans 0 rc 0/0 00000100:00100000:2.0:1776178516.276643:0:20762:0:(nrs_fifo.c:241:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.204.3@tcp, seq: 755 00000100:00100000:2.0:1776178516.276660:0:20762:0:(nrs_fifo.c:179:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.204.3@tcp, seq: 756 00000100:00100000:2.0:1776178516.276663:0:20762:0:(service.c:2324:ptlrpc_server_handle_request()) Handling RPC req@00000000f5405d7f pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_013:b50963bd-8eba-4432-9a54-8888d00e3fa1+9:2272:x1862456979128768:12345-192.168.204.3@tcp:400:kworker/u8:3.0 00000100:00100000:2.0:1776178516.276897:0:20762:0:(service.c:2372:ptlrpc_server_handle_request()) Handled RPC req@00000000f5405d7f pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_013:b50963bd-8eba-4432-9a54-8888d00e3fa1+9:2272:x1862456979128768:12345-192.168.204.3@tcp:400:kworker/u8:3.0 Request processed in 232us (838us total) trans 0 rc 0/0 00000100:00100000:2.0:1776178516.276905:0:20762:0:(nrs_fifo.c:241:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.204.3@tcp, seq: 756 00000100:00100000:1.0:1776178516.277373:0:11281:0:(service.c:2152:ptlrpc_server_handle_req_in()) got req x1862456979128640 00000100:00100000:1.0:1776178516.277420:0:11281:0:(nrs_fifo.c:179:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.204.3@tcp, seq: 1111 00000100:00100000:1.0:1776178516.277425:0:11281:0:(service.c:2324:ptlrpc_server_handle_request()) Handling RPC req@00000000d8a65f8a pname:cluuid+ref:pid:xid:nid:opc:job mdt00_004:b50963bd-8eba-4432-9a54-8888d00e3fa1+6:2272:x1862456979128640:12345-192.168.204.3@tcp:400:kworker/u8:3.0 00000100:00100000:1.0:1776178516.277480:0:11281:0:(service.c:2372:ptlrpc_server_handle_request()) Handled RPC req@00000000d8a65f8a pname:cluuid+ref:pid:xid:nid:opc:job mdt00_004:b50963bd-8eba-4432-9a54-8888d00e3fa1+6:2272:x1862456979128640:12345-192.168.204.3@tcp:400:kworker/u8:3.0 Request processed in 55us (1808us total) trans 0 rc 0/0 00000100:00100000:1.0:1776178516.277489:0:11281:0:(nrs_fifo.c:241:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.204.3@tcp, seq: 1111 00010000:00010000:3.0:1776178516.452202:0:6244:0:(ldlm_lockd.c:2792:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00010000:00010000:2.0:1776178517.477008:0:6245:0:(ldlm_lockd.c:2792:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00080000:1.0:1776178517.857813:0:37:0:(pinger.c:251:ptlrpc_pinger_process_import()) lustre-MDT0000-lwp-OST0000_UUID->lustre-MDT0000_UUID: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00080000:1.0:1776178517.857821:0:37:0:(recover.c:57:ptlrpc_initiate_recovery()) lustre-MDT0000_UUID: starting recovery 00000100:00080000:1.0:1776178517.857825:0:37:0:(import.c:86:import_set_state_nolock()) 0000000047601a98 lustre-MDT0000_UUID: changing import state from DISCONN to CONNECTING 00000100:00080000:1.0:1776178517.857833:0:37:0:(import.c:534:import_select_connection()) lustre-MDT0000-lwp-OST0000: connect to NID 0@lo last attempt 1220 00000100:00080000:1.0:1776178517.857836:0:37:0:(import.c:581:import_select_connection()) lustre-MDT0000-lwp-OST0000: tried all connections, increasing latency to 21s 00000100:00080000:1.0:1776178517.857842:0:37:0:(import.c:615:import_select_connection()) lustre-MDT0000-lwp-OST0000: import 0000000047601a98 using connection 192.168.204.103@tcp/0@lo 00000100:00100000:1.0:1776178517.857866:0:37:0:(import.c:807:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5) req@00000000582bb477 x1862456998092544/t0(0) o38->lustre-MDT0000-lwp-OST0000@0@lo:12/10 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' 00000100:00080000:1.0:1776178517.857911:0:37:0:(pinger.c:251:ptlrpc_pinger_process_import()) lustre-MDT0000-lwp-OST0001_UUID->lustre-MDT0000_UUID: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00080000:1.0:1776178517.857915:0:37:0:(recover.c:57:ptlrpc_initiate_recovery()) lustre-MDT0000_UUID: starting recovery 00000100:00080000:1.0:1776178517.857917:0:37:0:(import.c:86:import_set_state_nolock()) 00000000dfc8db13 lustre-MDT0000_UUID: changing import state from DISCONN to CONNECTING 00000100:00080000:1.0:1776178517.857920:0:37:0:(import.c:534:import_select_connection()) lustre-MDT0000-lwp-OST0001: connect to NID 0@lo last attempt 1220 00000100:00080000:1.0:1776178517.857922:0:37:0:(import.c:581:import_select_connection()) lustre-MDT0000-lwp-OST0001: tried all connections, increasing latency to 21s 00000100:00080000:1.0:1776178517.857926:0:37:0:(import.c:615:import_select_connection()) lustre-MDT0000-lwp-OST0001: import 00000000dfc8db13 using connection 192.168.204.103@tcp/0@lo 00000100:00100000:1.0:1776178517.857936:0:37:0:(import.c:807:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5) req@0000000009dce6fb x1862456998092672/t0(0) o38->lustre-MDT0000-lwp-OST0001@0@lo:12/10 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' 00000100:00080000:1.0:1776178517.857950:0:37:0:(pinger.c:251:ptlrpc_pinger_process_import()) lustre-MDT0001-mdtlov_UUID->lustre-MDT0000_UUID: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00080000:1.0:1776178517.857953:0:37:0:(recover.c:57:ptlrpc_initiate_recovery()) lustre-MDT0000_UUID: starting recovery 00000100:00080000:1.0:1776178517.857954:0:37:0:(import.c:86:import_set_state_nolock()) 00000000ba4f1821 lustre-MDT0000_UUID: changing import state from DISCONN to CONNECTING 00000100:00080000:1.0:1776178517.857957:0:37:0:(import.c:534:import_select_connection()) lustre-MDT0000-osp-MDT0001: connect to NID 0@lo last attempt 1220 00000100:00080000:1.0:1776178517.857959:0:37:0:(import.c:581:import_select_connection()) lustre-MDT0000-osp-MDT0001: tried all connections, increasing latency to 21s 00000100:00080000:1.0:1776178517.857962:0:37:0:(import.c:615:import_select_connection()) lustre-MDT0000-osp-MDT0001: import 00000000ba4f1821 using connection 192.168.204.103@tcp/0@lo 00000100:00100000:1.0:1776178517.857972:0:37:0:(import.c:807:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5) req@0000000072c0bce9 x1862456998092800/t0(0) o38->lustre-MDT0000-osp-MDT0001@0@lo:24/4 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' 00000100:00080000:1.0:1776178517.857988:0:37:0:(pinger.c:251:ptlrpc_pinger_process_import()) lustre-MDT0000-lwp-MDT0001_UUID->lustre-MDT0000_UUID: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00080000:1.0:1776178517.857992:0:37:0:(recover.c:57:ptlrpc_initiate_recovery()) lustre-MDT0000_UUID: starting recovery 00000100:00080000:1.0:1776178517.857993:0:37:0:(import.c:86:import_set_state_nolock()) 00000000a7c55b1f lustre-MDT0000_UUID: changing import state from DISCONN to CONNECTING 00000100:00080000:1.0:1776178517.857997:0:37:0:(import.c:534:import_select_connection()) lustre-MDT0000-lwp-MDT0001: connect to NID 0@lo last attempt 1220 00000100:00080000:1.0:1776178517.857998:0:37:0:(import.c:581:import_select_connection()) lustre-MDT0000-lwp-MDT0001: tried all connections, increasing latency to 21s 00000100:00080000:1.0:1776178517.858001:0:37:0:(import.c:615:import_select_connection()) lustre-MDT0000-lwp-MDT0001: import 00000000a7c55b1f using connection 192.168.204.103@tcp/0@lo 00000100:00100000:1.0:1776178517.858010:0:37:0:(import.c:807:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5) req@0000000072a0bf14 x1862456998092992/t0(0) o38->lustre-MDT0000-lwp-MDT0001@0@lo:12/10 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' 00000100:00100000:1.0:1776178517.858040:0:3354:0:(client.c:1733:ptlrpc_send_new_req()) Sending RPC req@0000000034d11618 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_01:lustre-MDT0001-lwp-OST0001_UUID:3354:1862456998092736:0@lo:400:kworker/u8:2.0 00000100:00100000:1.0:1776178517.858135:0:3354:0:(events.c:361:request_in_callback()) peer: 12345-0@lo (source: 12345-0@lo) 00000100:00100000:1.0:1776178517.858172:0:3354:0:(ptlrpcd.c:415:ptlrpcd_check()) transfer 1 async RPCs [0->1] 00000100:00100000:1.0:1776178517.858178:0:3354:0:(client.c:1733:ptlrpc_send_new_req()) Sending RPC req@0000000054bbc2c8 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_01:lustre-MDT0001-lwp-OST0000_UUID:3354:1862456998092608:0@lo:400:kworker/u8:2.0 00000100:00100000:1.0:1776178517.858200:0:3354:0:(events.c:361:request_in_callback()) peer: 12345-0@lo (source: 12345-0@lo) 00000100:00100000:1.0:1776178517.858232:0:11281:0:(service.c:2152:ptlrpc_server_handle_req_in()) got req x1862456998092736 00000100:00100000:1.0:1776178517.858297:0:11281:0:(service.c:2152:ptlrpc_server_handle_req_in()) got req x1862456998092608 00000100:00100000:1.0:1776178517.858316:0:11281:0:(nrs_fifo.c:179:nrs_fifo_req_get()) NRS start fifo request from 12345-0@lo, seq: 1112 00000100:00100000:1.0:1776178517.858322:0:11281:0:(service.c:2324:ptlrpc_server_handle_request()) Handling RPC req@00000000ae7e263a pname:cluuid+ref:pid:xid:nid:opc:job mdt00_004:lustre-MDT0001-lwp-OST0001_UUID+5:3354:x1862456998092736:12345-0@lo:400:kworker/u8:2.0 00000100:00100000:1.0:1776178517.858398:0:11281:0:(service.c:2372:ptlrpc_server_handle_request()) Handled RPC req@00000000ae7e263a pname:cluuid+ref:pid:xid:nid:opc:job mdt00_004:lustre-MDT0001-lwp-OST0001_UUID+5:3354:x1862456998092736:12345-0@lo:400:kworker/u8:2.0 Request processed in 75us (264us total) trans 0 rc 0/0 00000100:00100000:1.0:1776178517.858411:0:11281:0:(nrs_fifo.c:241:nrs_fifo_req_stop()) NRS stop fifo request from 12345-0@lo, seq: 1112 00000100:00100000:1.0:1776178517.858430:0:11281:0:(nrs_fifo.c:179:nrs_fifo_req_get()) NRS start fifo request from 12345-0@lo, seq: 1113 00000100:00100000:1.0:1776178517.858433:0:11281:0:(service.c:2324:ptlrpc_server_handle_request()) Handling RPC req@000000001d2ef500 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_004:lustre-MDT0001-lwp-OST0000_UUID+5:3354:x1862456998092608:12345-0@lo:400:kworker/u8:2.0 00000100:00100000:1.0:1776178517.858458:0:11281:0:(service.c:2372:ptlrpc_server_handle_request()) Handled RPC req@000000001d2ef500 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_004:lustre-MDT0001-lwp-OST0000_UUID+5:3354:x1862456998092608:12345-0@lo:400:kworker/u8:2.0 Request processed in 25us (259us total) trans 0 rc 0/0 00000100:00100000:1.0:1776178517.858464:0:11281:0:(nrs_fifo.c:241:nrs_fifo_req_stop()) NRS stop fifo request from 12345-0@lo, seq: 1113 00000100:00100000:1.0:1776178517.858508:0:3354:0:(client.c:2215:ptlrpc_check_set()) Completed RPC req@0000000054bbc2c8 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_01:lustre-MDT0001-lwp-OST0000_UUID:3354:1862456998092608:0@lo:400:kworker/u8:2.0 00000100:00100000:1.0:1776178517.858519:0:3354:0:(client.c:2215:ptlrpc_check_set()) Completed RPC req@0000000034d11618 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_01:lustre-MDT0001-lwp-OST0001_UUID:3354:1862456998092736:0@lo:400:kworker/u8:2.0 00000100:00100000:2.0:1776178517.858855:0:3355:0:(client.c:1733:ptlrpc_send_new_req()) Sending RPC req@0000000028dd2aee pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_02:lustre-MDT0001-mdtlov_UUID:3355:1862456998092864:0@lo:400:kworker/u8:2.0 00000100:00100000:2.0:1776178517.858891:0:3355:0:(events.c:361:request_in_callback()) peer: 12345-0@lo (source: 12345-0@lo) 00000100:00100000:2.0:1776178517.858922:0:3355:0:(ptlrpcd.c:415:ptlrpcd_check()) transfer 2 async RPCs [3->2] 00000100:00100000:2.0:1776178517.858928:0:3355:0:(client.c:1733:ptlrpc_send_new_req()) Sending RPC req@0000000012d4bd5e pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_02:ac61f839-0e8e-4cac-9c59-08bcfbbad330:3355:1862456998092480:0@lo:400:kworker/u8:2.0 00000100:00100000:2.0:1776178517.858954:0:3355:0:(events.c:361:request_in_callback()) peer: 12345-0@lo (source: 12345-0@lo) 00000100:00100000:2.0:1776178517.858971:0:3355:0:(client.c:1733:ptlrpc_send_new_req()) Sending RPC req@00000000b319fd93 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_02:lustre-MDT0001-mdtlov_UUID:3355:1862456998092928:0@lo:400:kworker/u8:2.0 00000100:00100000:2.0:1776178517.858990:0:3355:0:(events.c:361:request_in_callback()) peer: 12345-0@lo (source: 12345-0@lo) 00000100:00100000:2.0:1776178517.859018:0:20137:0:(service.c:2152:ptlrpc_server_handle_req_in()) got req x1862456998092480 00000100:00100000:2.0:1776178517.859050:0:20137:0:(nrs_fifo.c:179:nrs_fifo_req_get()) NRS start fifo request from 12345-0@lo, seq: 278 00000100:00100000:2.0:1776178517.859055:0:20137:0:(service.c:2324:ptlrpc_server_handle_request()) Handling RPC req@00000000b4b6b848 pname:cluuid+ref:pid:xid:nid:opc:job ll_mgs_0000:ac61f839-0e8e-4cac-9c59-08bcfbbad330+18:3355:x1862456998092480:12345-0@lo:400:kworker/u8:2.0 00000100:00100000:2.0:1776178517.859124:0:20137:0:(service.c:2372:ptlrpc_server_handle_request()) Handled RPC req@00000000b4b6b848 pname:cluuid+ref:pid:xid:nid:opc:job ll_mgs_0000:ac61f839-0e8e-4cac-9c59-08bcfbbad330+18:3355:x1862456998092480:12345-0@lo:400:kworker/u8:2.0 Request processed in 67us (168us total) trans 0 rc 0/0 00000100:00100000:2.0:1776178517.859131:0:20137:0:(nrs_fifo.c:241:nrs_fifo_req_stop()) NRS stop fifo request from 12345-0@lo, seq: 278 00000100:00100000:2.0:1776178517.859175:0:3355:0:(client.c:2215:ptlrpc_check_set()) Completed RPC req@0000000012d4bd5e pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_02:ac61f839-0e8e-4cac-9c59-08bcfbbad330:3355:1862456998092480:0@lo:400:kworker/u8:2.0 00000100:00100000:3.0:1776178517.859352:0:3352:0:(client.c:1733:ptlrpc_send_new_req()) Sending RPC req@00000000582bb477 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:lustre-MDT0000-lwp-OST0000_UUID:3352:1862456998092544:0@lo:38:kworker/u8:2.0 00000100:00100000:3.0:1776178517.859394:0:3352:0:(events.c:361:request_in_callback()) peer: 12345-0@lo (source: 12345-0@lo) 00000100:00100000:3.0:1776178517.859573:0:3352:0:(client.c:1733:ptlrpc_send_new_req()) Sending RPC req@0000000009dce6fb pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:lustre-MDT0000-lwp-OST0001_UUID:3352:1862456998092672:0@lo:38:kworker/u8:2.0 00000100:00100000:3.0:1776178517.859596:0:3352:0:(events.c:361:request_in_callback()) peer: 12345-0@lo (source: 12345-0@lo) 00000100:00100000:3.0:1776178517.859612:0:3352:0:(client.c:1733:ptlrpc_send_new_req()) Sending RPC req@0000000072c0bce9 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:lustre-MDT0001-mdtlov_UUID:3352:1862456998092800:0@lo:38:kworker/u8:2.0 00000100:00100000:3.0:1776178517.859632:0:3352:0:(events.c:361:request_in_callback()) peer: 12345-0@lo (source: 12345-0@lo) 00000100:00100000:3.0:1776178517.859650:0:3352:0:(client.c:1733:ptlrpc_send_new_req()) Sending RPC req@0000000072a0bf14 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:lustre-MDT0000-lwp-MDT0001_UUID:3352:1862456998092992:0@lo:38:kworker/u8:2.0 00000100:00100000:3.0:1776178517.859673:0:3352:0:(events.c:361:request_in_callback()) peer: 12345-0@lo (source: 12345-0@lo) 00000100:00100000:3.0:1776178517.859706:0:14128:0:(service.c:2152:ptlrpc_server_handle_req_in()) got req x1862456998092800 00000100:00100000:3.0:1776178517.859745:0:14128:0:(nrs_fifo.c:179:nrs_fifo_req_get()) NRS start fifo request from 12345-0@lo, seq: 769 00000100:00100000:3.0:1776178517.859750:0:14128:0:(service.c:2324:ptlrpc_server_handle_request()) Handling RPC req@00000000fc42e675 pname:cluuid+ref:pid:xid:nid:opc:job mdt_out00_002:0+-99:3352:x1862456998092800:12345-0@lo:38:kworker/u8:2.0 00010000:02000000:3.0:1776178517.859773:0:14128:0:(ldlm_lib.c:1139:target_handle_connect()) lustre-MDT0000: Not available for connect from 0@lo (stopping) 00000100:00100000:3.0:1776178517.859809:0:14128:0:(service.c:2372:ptlrpc_server_handle_request()) Handled RPC req@00000000fc42e675 pname:cluuid+ref:pid:xid:nid:opc:job mdt_out00_002:0+-99:3352:x1862456998092800:12345-0@lo:38:kworker/u8:2.0 Request processed in 59us (177us total) trans 0 rc -19/-19 00000100:00100000:3.0:1776178517.859816:0:14128:0:(nrs_fifo.c:241:nrs_fifo_req_stop()) NRS stop fifo request from 12345-0@lo, seq: 769 00000100:00100000:3.0:1776178517.859841:0:6256:0:(service.c:2152:ptlrpc_server_handle_req_in()) got req x1862456998092544 00000100:00100000:3.0:1776178517.859859:0:6256:0:(service.c:2152:ptlrpc_server_handle_req_in()) got req x1862456998092672 00000100:00100000:3.0:1776178517.859867:0:6256:0:(service.c:2152:ptlrpc_server_handle_req_in()) got req x1862456998092992 00000100:00100000:3.0:1776178517.859876:0:6256:0:(nrs_fifo.c:179:nrs_fifo_req_get()) NRS start fifo request from 12345-0@lo, seq: 6869 00000100:00100000:3.0:1776178517.859879:0:6256:0:(service.c:2324:ptlrpc_server_handle_request()) Handling RPC req@00000000767e3c38 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_001:0+-99:3352:x1862456998092544:12345-0@lo:38:kworker/u8:2.0 00010000:02000000:3.0:1776178517.859885:0:6256:0:(ldlm_lib.c:1139:target_handle_connect()) lustre-MDT0000: Not available for connect from 0@lo (stopping) 00000100:00100000:3.0:1776178517.859909:0:6256:0:(service.c:2372:ptlrpc_server_handle_request()) Handled RPC req@00000000767e3c38 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_001:0+-99:3352:x1862456998092544:12345-0@lo:38:kworker/u8:2.0 Request processed in 29us (515us total) trans 0 rc -19/-19 00000100:00100000:3.0:1776178517.859915:0:6256:0:(nrs_fifo.c:241:nrs_fifo_req_stop()) NRS stop fifo request from 12345-0@lo, seq: 6869 00000100:00100000:3.0:1776178517.859925:0:6256:0:(nrs_fifo.c:179:nrs_fifo_req_get()) NRS start fifo request from 12345-0@lo, seq: 6870 00000100:00100000:3.0:1776178517.859928:0:6256:0:(service.c:2324:ptlrpc_server_handle_request()) Handling RPC req@00000000c6619661 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_001:0+-99:3352:x1862456998092672:12345-0@lo:38:kworker/u8:2.0 00010000:02000000:3.0:1776178517.859933:0:6256:0:(ldlm_lib.c:1139:target_handle_connect()) lustre-MDT0000: Not available for connect from 0@lo (stopping) 00000100:00100000:3.0:1776178517.859953:0:6256:0:(service.c:2372:ptlrpc_server_handle_request()) Handled RPC req@00000000c6619661 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_001:0+-99:3352:x1862456998092672:12345-0@lo:38:kworker/u8:2.0 Request processed in 24us (356us total) trans 0 rc -19/-19 00000100:00100000:3.0:1776178517.859957:0:6256:0:(nrs_fifo.c:241:nrs_fifo_req_stop()) NRS stop fifo request from 12345-0@lo, seq: 6870 00000100:00100000:3.0:1776178517.859965:0:6256:0:(nrs_fifo.c:179:nrs_fifo_req_get()) NRS start fifo request from 12345-0@lo, seq: 6871 00000100:00100000:3.0:1776178517.859968:0:6256:0:(service.c:2324:ptlrpc_server_handle_request()) Handling RPC req@0000000070d0c29c pname:cluuid+ref:pid:xid:nid:opc:job mdt00_001:0+-99:3352:x1862456998092992:12345-0@lo:38:kworker/u8:2.0 00010000:02000000:3.0:1776178517.859974:0:6256:0:(ldlm_lib.c:1139:target_handle_connect()) lustre-MDT0000: Not available for connect from 0@lo (stopping) 00000100:00100000:3.0:1776178517.859990:0:6256:0:(service.c:2372:ptlrpc_server_handle_request()) Handled RPC req@0000000070d0c29c pname:cluuid+ref:pid:xid:nid:opc:job mdt00_001:0+-99:3352:x1862456998092992:12345-0@lo:38:kworker/u8:2.0 Request processed in 21us (318us total) trans 0 rc -19/-19 00000100:00100000:3.0:1776178517.859996:0:6256:0:(nrs_fifo.c:241:nrs_fifo_req_stop()) NRS stop fifo request from 12345-0@lo, seq: 6871 00000100:00080000:3.0:1776178517.860023:0:3352:0:(import.c:86:import_set_state_nolock()) 0000000047601a98 lustre-MDT0000_UUID: changing import state from CONNECTING to DISCONN 00000100:00080000:3.0:1776178517.860027:0:3352:0:(import.c:1429:ptlrpc_connect_interpret()) recovery of lustre-MDT0000_UUID on 192.168.204.103@tcp failed (-19) 00000100:00100000:3.0:1776178517.860031:0:3352:0:(client.c:2215:ptlrpc_check_set()) Completed RPC req@00000000582bb477 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:lustre-MDT0000-lwp-OST0000_UUID:3352:1862456998092544:0@lo:38:kworker/u8:2.0 00000100:00080000:3.0:1776178517.860039:0:3352:0:(import.c:86:import_set_state_nolock()) 00000000dfc8db13 lustre-MDT0000_UUID: changing import state from CONNECTING to DISCONN 00000100:00080000:3.0:1776178517.860042:0:3352:0:(import.c:1429:ptlrpc_connect_interpret()) recovery of lustre-MDT0000_UUID on 192.168.204.103@tcp failed (-19) 00000100:00100000:3.0:1776178517.860044:0:3352:0:(client.c:2215:ptlrpc_check_set()) Completed RPC req@0000000009dce6fb pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:lustre-MDT0000-lwp-OST0001_UUID:3352:1862456998092672:0@lo:38:kworker/u8:2.0 00000100:00080000:3.0:1776178517.860050:0:3352:0:(import.c:86:import_set_state_nolock()) 00000000ba4f1821 lustre-MDT0000_UUID: changing import state from CONNECTING to DISCONN 00000100:00080000:3.0:1776178517.860052:0:3352:0:(import.c:1429:ptlrpc_connect_interpret()) recovery of lustre-MDT0000_UUID on 192.168.204.103@tcp failed (-19) 00000100:00100000:3.0:1776178517.860054:0:3352:0:(client.c:2215:ptlrpc_check_set()) Completed RPC req@0000000072c0bce9 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:lustre-MDT0001-mdtlov_UUID:3352:1862456998092800:0@lo:38:kworker/u8:2.0 00000100:00080000:3.0:1776178517.860085:0:3352:0:(import.c:86:import_set_state_nolock()) 00000000a7c55b1f lustre-MDT0000_UUID: changing import state from CONNECTING to DISCONN 00000100:00080000:3.0:1776178517.860088:0:3352:0:(import.c:1429:ptlrpc_connect_interpret()) recovery of lustre-MDT0000_UUID on 192.168.204.103@tcp failed (-19) 00000100:00100000:3.0:1776178517.860091:0:3352:0:(client.c:2215:ptlrpc_check_set()) Completed RPC req@0000000072a0bf14 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:lustre-MDT0000-lwp-MDT0001_UUID:3352:1862456998092992:0@lo:38:kworker/u8:2.0 00000100:00100000:0.0:1776178517.860576:0:14440:0:(service.c:2152:ptlrpc_server_handle_req_in()) got req x1862456998092864 00000100:00100000:0.0:1776178517.860614:0:14440:0:(service.c:2152:ptlrpc_server_handle_req_in()) got req x1862456998092928 00000100:00100000:0.0:1776178517.860639:0:14440:0:(nrs_fifo.c:179:nrs_fifo_req_get()) NRS start fifo request from 12345-0@lo, seq: 757 00000100:00100000:0.0:1776178517.860643:0:14440:0:(service.c:2324:ptlrpc_server_handle_request()) Handling RPC req@00000000abb6cbab pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_004:lustre-MDT0001-mdtlov_UUID+5:3355:x1862456998092864:12345-0@lo:400:kworker/u8:2.0 00000100:00100000:0.0:1776178517.860686:0:14440:0:(service.c:2372:ptlrpc_server_handle_request()) Handled RPC req@00000000abb6cbab pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_004:lustre-MDT0001-mdtlov_UUID+5:3355:x1862456998092864:12345-0@lo:400:kworker/u8:2.0 Request processed in 42us (1795us total) trans 0 rc 0/0 00000100:00100000:0.0:1776178517.860692:0:14440:0:(nrs_fifo.c:241:nrs_fifo_req_stop()) NRS stop fifo request from 12345-0@lo, seq: 757 00000100:00100000:0.0:1776178517.860704:0:14440:0:(nrs_fifo.c:179:nrs_fifo_req_get()) NRS start fifo request from 12345-0@lo, seq: 758 00000100:00100000:0.0:1776178517.860707:0:14440:0:(service.c:2324:ptlrpc_server_handle_request()) Handling RPC req@00000000715d380c pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_004:lustre-MDT0001-mdtlov_UUID+5:3355:x1862456998092928:12345-0@lo:400:kworker/u8:2.0 00000100:00100000:0.0:1776178517.860743:0:14440:0:(service.c:2372:ptlrpc_server_handle_request()) Handled RPC req@00000000715d380c pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_004:lustre-MDT0001-mdtlov_UUID+5:3355:x1862456998092928:12345-0@lo:400:kworker/u8:2.0 Request processed in 35us (1752us total) trans 0 rc 0/0 00000100:00100000:0.0:1776178517.860748:0:14440:0:(nrs_fifo.c:241:nrs_fifo_req_stop()) NRS stop fifo request from 12345-0@lo, seq: 758 00000100:00100000:2.0:1776178517.861737:0:3355:0:(client.c:2215:ptlrpc_check_set()) Completed RPC req@00000000b319fd93 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_02:lustre-MDT0001-mdtlov_UUID:3355:1862456998092928:0@lo:400:kworker/u8:2.0 00000100:00100000:2.0:1776178517.861753:0:3355:0:(client.c:2215:ptlrpc_check_set()) Completed RPC req@0000000028dd2aee pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_02:lustre-MDT0001-mdtlov_UUID:3355:1862456998092864:0@lo:400:kworker/u8:2.0 00010000:00010000:1.0:1776178518.496502:0:9224:0:(ldlm_lockd.c:2792:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:1.0:1776178519.392451:0:3354:0:(ptlrpcd.c:415:ptlrpcd_check()) transfer 1 async RPCs [0->1] 00000100:00100000:1.0:1776178519.392622:0:3354:0:(client.c:1733:ptlrpc_send_new_req()) Sending RPC req@00000000240cb7a8 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_01:lustre-MDT0001-mdtlov_UUID:3354:1862456998093056:0@lo:13:osp-pre-1-1.0 00000100:00100000:1.0:1776178519.392712:0:3354:0:(events.c:361:request_in_callback()) peer: 12345-0@lo (source: 12345-0@lo) 00000100:00100000:3.0:1776178519.392831:0:8231:0:(service.c:2152:ptlrpc_server_handle_req_in()) got req x1862456998093056 00000100:00100000:3.0:1776178519.392897:0:8231:0:(nrs_fifo.c:179:nrs_fifo_req_get()) NRS start fifo request from 12345-0@lo, seq: 611 00000100:00100000:3.0:1776178519.392902:0:8231:0:(service.c:2324:ptlrpc_server_handle_request()) Handling RPC req@000000000489d393 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_create00:lustre-MDT0001-mdtlov_UUID+5:3354:x1862456998093056:12345-0@lo:13:osp-pre-1-1.0 00002000:00000024:3.0:1776178519.392935:0:8231:0:(ofd_obd.c:740:ofd_statfs()) blocks cached 229376 granted 491746368 pending 0 free 3923689472 avail 3692167168 00000100:00100000:3.0:1776178519.392989:0:8231:0:(service.c:2372:ptlrpc_server_handle_request()) Handled RPC req@000000000489d393 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_create00:lustre-MDT0001-mdtlov_UUID+5:3354:x1862456998093056:12345-0@lo:13:osp-pre-1-1.0 Request processed in 86us (279us total) trans 0 rc 0/0 00000100:00100000:3.0:1776178519.392997:0:8231:0:(nrs_fifo.c:241:nrs_fifo_req_stop()) NRS stop fifo request from 12345-0@lo, seq: 611 00000100:00100000:1.0:1776178519.393324:0:3354:0:(client.c:2215:ptlrpc_check_set()) Completed RPC req@00000000240cb7a8 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_01:lustre-MDT0001-mdtlov_UUID:3354:1862456998093056:0@lo:13:osp-pre-1-1.0 00010000:00010000:1.0:1776178519.526819:0:9225:0:(ldlm_lockd.c:2792:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:1.0:1776178519.905353:0:3354:0:(client.c:1733:ptlrpc_send_new_req()) Sending RPC req@00000000a101cbf3 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_01:lustre-MDT0001-mdtlov_UUID:3354:1862456998093120:0@lo:13:osp-pre-0-1.0 00000100:00100000:1.0:1776178519.905439:0:3354:0:(events.c:361:request_in_callback()) peer: 12345-0@lo (source: 12345-0@lo) 00000100:00100000:2.0:1776178519.907223:0:9554:0:(service.c:2152:ptlrpc_server_handle_req_in()) got req x1862456998093120 00000100:00100000:2.0:1776178519.907306:0:9554:0:(nrs_fifo.c:179:nrs_fifo_req_get()) NRS start fifo request from 12345-0@lo, seq: 612 00000100:00100000:2.0:1776178519.907312:0:9554:0:(service.c:2324:ptlrpc_server_handle_request()) Handling RPC req@0000000028dd2aee pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_create00:lustre-MDT0001-mdtlov_UUID+5:3354:x1862456998093120:12345-0@lo:13:osp-pre-0-1.0 00002000:00000024:2.0:1776178519.907354:0:9554:0:(ofd_obd.c:740:ofd_statfs()) blocks cached 315392 granted 484883776 pending 0 free 3923689472 avail 3692167168 00000100:00100000:2.0:1776178519.907447:0:9554:0:(service.c:2372:ptlrpc_server_handle_request()) Handled RPC req@0000000028dd2aee pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_create00:lustre-MDT0001-mdtlov_UUID+5:3354:x1862456998093120:12345-0@lo:13:osp-pre-0-1.0 Request processed in 134us (2009us total) trans 0 rc 0/0 00000100:00100000:2.0:1776178519.907454:0:9554:0:(nrs_fifo.c:241:nrs_fifo_req_stop()) NRS stop fifo request from 12345-0@lo, seq: 612 00000100:00100000:1.0:1776178519.907695:0:3354:0:(client.c:2215:ptlrpc_check_set()) Completed RPC req@00000000a101cbf3 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_01:lustre-MDT0001-mdtlov_UUID:3354:1862456998093120:0@lo:13:osp-pre-0-1.0 00010000:00010000:3.0:1776178520.547426:0:6244:0:(ldlm_lockd.c:2792:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:3.0:1776178521.382237:0:3346:0:(events.c:361:request_in_callback()) peer: 12345-192.168.204.3@tcp (source: 12345-192.168.204.3@tcp) 00000100:00100000:3.0:1776178521.382526:0:3346:0:(events.c:361:request_in_callback()) peer: 12345-192.168.204.3@tcp (source: 12345-192.168.204.3@tcp) 00000100:00100000:0.0:1776178521.382535:0:20140:0:(service.c:2152:ptlrpc_server_handle_req_in()) got req x1862456979128832 00000100:00100000:0.0:1776178521.382616:0:20140:0:(nrs_fifo.c:179:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.204.3@tcp, seq: 279 00000100:00100000:3.0:1776178521.382623:0:3346:0:(events.c:361:request_in_callback()) peer: 12345-192.168.204.3@tcp (source: 12345-192.168.204.3@tcp) 00000100:00100000:0.0:1776178521.382627:0:20140:0:(service.c:2324:ptlrpc_server_handle_request()) Handling RPC req@000000001333a9de pname:cluuid+ref:pid:xid:nid:opc:job ll_mgs_0002:5b5c66b1-1a2f-4651-8548-0c75b9a38a94+9:2273:x1862456979128832:12345-192.168.204.3@tcp:400:kworker/u8:4.0 00000100:00100000:2.0:1776178521.382645:0:20755:0:(service.c:2152:ptlrpc_server_handle_req_in()) got req x1862456979129024 00000100:00100000:3.0:1776178521.382681:0:8250:0:(service.c:2152:ptlrpc_server_handle_req_in()) got req x1862456979128960 00000100:00100000:0.0:1776178521.382734:0:20140:0:(service.c:2372:ptlrpc_server_handle_request()) Handled RPC req@000000001333a9de pname:cluuid+ref:pid:xid:nid:opc:job ll_mgs_0002:5b5c66b1-1a2f-4651-8548-0c75b9a38a94+9:2273:x1862456979128832:12345-192.168.204.3@tcp:400:kworker/u8:4.0 Request processed in 106us (502us total) trans 0 rc 0/0 00000100:00100000:2.0:1776178521.382749:0:20755:0:(nrs_fifo.c:179:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.204.3@tcp, seq: 759 00000100:00100000:3.0:1776178521.382751:0:8250:0:(nrs_fifo.c:179:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.204.3@tcp, seq: 1114 00000100:00100000:0.0:1776178521.382759:0:20140:0:(nrs_fifo.c:241:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.204.3@tcp, seq: 279 00000100:00100000:2.0:1776178521.382766:0:20755:0:(service.c:2324:ptlrpc_server_handle_request()) Handling RPC req@00000000f285e30c pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_006:b50963bd-8eba-4432-9a54-8888d00e3fa1+8:2272:x1862456979129024:12345-192.168.204.3@tcp:400:kworker/u8:4.0 00000100:00100000:3.0:1776178521.382770:0:8250:0:(service.c:2324:ptlrpc_server_handle_request()) Handling RPC req@00000000240cb7a8 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_003:b50963bd-8eba-4432-9a54-8888d00e3fa1+6:2272:x1862456979128960:12345-192.168.204.3@tcp:400:kworker/u8:4.0 00000100:00100000:2.0:1776178521.382840:0:20755:0:(service.c:2372:ptlrpc_server_handle_request()) Handled RPC req@00000000f285e30c pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_006:b50963bd-8eba-4432-9a54-8888d00e3fa1+8:2272:x1862456979129024:12345-192.168.204.3@tcp:400:kworker/u8:4.0 Request processed in 80us (314us total) trans 0 rc 0/0 00000100:00100000:2.0:1776178521.382848:0:20755:0:(nrs_fifo.c:241:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.204.3@tcp, seq: 759 00000100:00100000:3.0:1776178521.384778:0:8250:0:(service.c:2372:ptlrpc_server_handle_request()) Handled RPC req@00000000240cb7a8 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_003:b50963bd-8eba-4432-9a54-8888d00e3fa1+6:2272:x1862456979128960:12345-192.168.204.3@tcp:400:kworker/u8:4.0 Request processed in 2004us (2148us total) trans 0 rc 0/0 00000100:00100000:3.0:1776178521.384798:0:8250:0:(nrs_fifo.c:241:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.204.3@tcp, seq: 1114 00000100:00100000:2.0:1776178521.385236:0:3347:0:(events.c:361:request_in_callback()) peer: 12345-192.168.204.3@tcp (source: 12345-192.168.204.3@tcp) 00000100:00100000:3.0:1776178521.385786:0:8250:0:(service.c:2152:ptlrpc_server_handle_req_in()) got req x1862456979128896 00000100:00100000:3.0:1776178521.385990:0:8250:0:(nrs_fifo.c:179:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.204.3@tcp, seq: 6872 00000100:00100000:3.0:1776178521.386001:0:8250:0:(service.c:2324:ptlrpc_server_handle_request()) Handling RPC req@00000000c21d7f55 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_003:0+-99:2269:x1862456979128896:12345-192.168.204.3@tcp:38:kworker/u8:4.0 00010000:02000000:3.0:1776178521.386027:0:8250:0:(ldlm_lib.c:1139:target_handle_connect()) lustre-MDT0000: Not available for connect from 192.168.204.3@tcp (stopping) 00000100:00100000:3.0:1776178521.389184:0:8250:0:(service.c:2372:ptlrpc_server_handle_request()) Handled RPC req@00000000c21d7f55 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_003:0+-99:2269:x1862456979128896:12345-192.168.204.3@tcp:38:kworker/u8:4.0 Request processed in 3169us (3940us total) trans 0 rc -19/-19 00000100:00100000:3.0:1776178521.389410:0:8250:0:(nrs_fifo.c:241:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.204.3@tcp, seq: 6872 00000100:00100000:1.0:1776178521.393644:0:3348:0:(events.c:361:request_in_callback()) peer: 12345-192.168.204.3@tcp (source: 12345-192.168.204.3@tcp) 00000100:00100000:2.0:1776178521.395001:0:20759:0:(service.c:2152:ptlrpc_server_handle_req_in()) got req x1862456979129088 00000100:00100000:2.0:1776178521.395099:0:20759:0:(nrs_fifo.c:179:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.204.3@tcp, seq: 760 00000100:00100000:2.0:1776178521.395108:0:20759:0:(service.c:2324:ptlrpc_server_handle_request()) Handling RPC req@00000000b319fd93 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_010:b50963bd-8eba-4432-9a54-8888d00e3fa1+9:2273:x1862456979129088:12345-192.168.204.3@tcp:400:kworker/u8:4.0 00000100:00100000:2.0:1776178521.395184:0:20759:0:(service.c:2372:ptlrpc_server_handle_request()) Handled RPC req@00000000b319fd93 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_010:b50963bd-8eba-4432-9a54-8888d00e3fa1+9:2273:x1862456979129088:12345-192.168.204.3@tcp:400:kworker/u8:4.0 Request processed in 77us (1553us total) trans 0 rc 0/0 00000100:00100000:2.0:1776178521.395192:0:20759:0:(nrs_fifo.c:241:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.204.3@tcp, seq: 760 00010000:00010000:2.0:1776178521.568618:0:6245:0:(ldlm_lockd.c:2792:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00010000:00010000:1.0:1776178522.592378:0:9224:0:(ldlm_lockd.c:2792:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00080000:2.0:1776178522.977450:0:37:0:(pinger.c:251:ptlrpc_pinger_process_import()) lustre-MDT0000-lwp-OST0000_UUID->lustre-MDT0000_UUID: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00080000:2.0:1776178522.977462:0:37:0:(recover.c:57:ptlrpc_initiate_recovery()) lustre-MDT0000_UUID: starting recovery 00000100:00080000:2.0:1776178522.977478:0:37:0:(import.c:86:import_set_state_nolock()) 0000000047601a98 lustre-MDT0000_UUID: changing import state from DISCONN to CONNECTING 00000100:00080000:2.0:1776178522.977492:0:37:0:(import.c:534:import_select_connection()) lustre-MDT0000-lwp-OST0000: connect to NID 0@lo last attempt 1225 00000100:00080000:2.0:1776178522.977497:0:37:0:(import.c:581:import_select_connection()) lustre-MDT0000-lwp-OST0000: tried all connections, increasing latency to 21s 00000100:00080000:2.0:1776178522.977505:0:37:0:(import.c:615:import_select_connection()) lustre-MDT0000-lwp-OST0000: import 0000000047601a98 using connection 192.168.204.103@tcp/0@lo 00000100:00100000:2.0:1776178522.977542:0:37:0:(import.c:807:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5) req@000000004f638b57 x1862456998093248/t0(0) o38->lustre-MDT0000-lwp-OST0000@0@lo:12/10 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' 00000100:00080000:2.0:1776178522.977581:0:37:0:(pinger.c:251:ptlrpc_pinger_process_import()) lustre-MDT0000-lwp-OST0001_UUID->lustre-MDT0000_UUID: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00080000:2.0:1776178522.977584:0:37:0:(recover.c:57:ptlrpc_initiate_recovery()) lustre-MDT0000_UUID: starting recovery 00000100:00080000:2.0:1776178522.977585:0:37:0:(import.c:86:import_set_state_nolock()) 00000000dfc8db13 lustre-MDT0000_UUID: changing import state from DISCONN to CONNECTING 00000100:00080000:2.0:1776178522.977589:0:37:0:(import.c:534:import_select_connection()) lustre-MDT0000-lwp-OST0001: connect to NID 0@lo last attempt 1225 00000100:00080000:2.0:1776178522.977591:0:37:0:(import.c:581:import_select_connection()) lustre-MDT0000-lwp-OST0001: tried all connections, increasing latency to 21s 00000100:00080000:2.0:1776178522.977594:0:37:0:(import.c:615:import_select_connection()) lustre-MDT0000-lwp-OST0001: import 00000000dfc8db13 using connection 192.168.204.103@tcp/0@lo 00000100:00100000:2.0:1776178522.977603:0:37:0:(import.c:807:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5) req@000000009e484c71 x1862456998093376/t0(0) o38->lustre-MDT0000-lwp-OST0001@0@lo:12/10 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' 00000100:00080000:2.0:1776178522.977634:0:37:0:(pinger.c:251:ptlrpc_pinger_process_import()) lustre-MDT0001-mdtlov_UUID->lustre-MDT0000_UUID: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00080000:2.0:1776178522.977637:0:37:0:(recover.c:57:ptlrpc_initiate_recovery()) lustre-MDT0000_UUID: starting recovery 00000100:00080000:2.0:1776178522.977639:0:37:0:(import.c:86:import_set_state_nolock()) 00000000ba4f1821 lustre-MDT0000_UUID: changing import state from DISCONN to CONNECTING 00000100:00080000:2.0:1776178522.977642:0:37:0:(import.c:534:import_select_connection()) lustre-MDT0000-osp-MDT0001: connect to NID 0@lo last attempt 1225 00000100:00080000:2.0:1776178522.977644:0:37:0:(import.c:581:import_select_connection()) lustre-MDT0000-osp-MDT0001: tried all connections, increasing latency to 21s 00000100:00080000:2.0:1776178522.977647:0:37:0:(import.c:615:import_select_connection()) lustre-MDT0000-osp-MDT0001: import 00000000ba4f1821 using connection 192.168.204.103@tcp/0@lo 00000100:00100000:2.0:1776178522.977659:0:37:0:(import.c:807:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5) req@0000000013ad9ac7 x1862456998093504/t0(0) o38->lustre-MDT0000-osp-MDT0001@0@lo:24/4 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' 00000100:00080000:2.0:1776178522.977679:0:37:0:(pinger.c:251:ptlrpc_pinger_process_import()) lustre-MDT0000-lwp-MDT0001_UUID->lustre-MDT0000_UUID: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00080000:2.0:1776178522.977685:0:37:0:(recover.c:57:ptlrpc_initiate_recovery()) lustre-MDT0000_UUID: starting recovery 00000100:00080000:2.0:1776178522.977686:0:37:0:(import.c:86:import_set_state_nolock()) 00000000a7c55b1f lustre-MDT0000_UUID: changing import state from DISCONN to CONNECTING 00000100:00080000:2.0:1776178522.977689:0:37:0:(import.c:534:import_select_connection()) lustre-MDT0000-lwp-MDT0001: connect to NID 0@lo last attempt 1225 00000100:00080000:2.0:1776178522.977691:0:37:0:(import.c:581:import_select_connection()) lustre-MDT0000-lwp-MDT0001: tried all connections, increasing latency to 21s 00000100:00080000:2.0:1776178522.977694:0:37:0:(import.c:615:import_select_connection()) lustre-MDT0000-lwp-MDT0001: import 00000000a7c55b1f using connection 192.168.204.103@tcp/0@lo 00000100:00100000:2.0:1776178522.977706:0:37:0:(import.c:807:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5) req@0000000065c78381 x1862456998093696/t0(0) o38->lustre-MDT0000-lwp-MDT0001@0@lo:12/10 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' 00000100:00100000:3.0:1776178522.977813:0:3356:0:(client.c:1733:ptlrpc_send_new_req()) Sending RPC req@0000000070829000 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_03:lustre-MDT0001-lwp-OST0000_UUID:3356:1862456998093312:0@lo:400:kworker/u8:2.0 00000100:00100000:3.0:1776178522.977925:0:3356:0:(events.c:361:request_in_callback()) peer: 12345-0@lo (source: 12345-0@lo) 00000100:00100000:3.0:1776178522.977985:0:3356:0:(ptlrpcd.c:415:ptlrpcd_check()) transfer 2 async RPCs [2->3] 00000100:00100000:3.0:1776178522.977991:0:3356:0:(client.c:1733:ptlrpc_send_new_req()) Sending RPC req@0000000082764007 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_03:ac61f839-0e8e-4cac-9c59-08bcfbbad330:3356:1862456998093184:0@lo:400:kworker/u8:2.0 00000100:00100000:3.0:1776178522.978033:0:3356:0:(events.c:361:request_in_callback()) peer: 12345-0@lo (source: 12345-0@lo) 00000100:00100000:3.0:1776178522.978051:0:3356:0:(client.c:1733:ptlrpc_send_new_req()) Sending RPC req@0000000080e59d72 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_03:lustre-MDT0001-mdtlov_UUID:3356:1862456998093632:0@lo:400:kworker/u8:2.0 00000100:00100000:3.0:1776178522.978136:0:3356:0:(events.c:361:request_in_callback()) peer: 12345-0@lo (source: 12345-0@lo) 00000100:00100000:3.0:1776178522.978169:0:3353:0:(client.c:1733:ptlrpc_send_new_req()) Sending RPC req@0000000012d88c00 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_00:lustre-MDT0001-lwp-OST0001_UUID:3353:1862456998093440:0@lo:400:kworker/u8:2.0 00000100:00100000:3.0:1776178522.978198:0:3353:0:(events.c:361:request_in_callback()) peer: 12345-0@lo (source: 12345-0@lo) 00000100:00100000:3.0:1776178522.978215:0:3353:0:(ptlrpcd.c:415:ptlrpcd_check()) transfer 1 async RPCs [1->0] 00000100:00100000:3.0:1776178522.978220:0:3353:0:(client.c:1733:ptlrpc_send_new_req()) Sending RPC req@0000000048564d2a pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_00:lustre-MDT0001-mdtlov_UUID:3353:1862456998093568:0@lo:400:kworker/u8:2.0 00000100:00100000:3.0:1776178522.978243:0:3353:0:(events.c:361:request_in_callback()) peer: 12345-0@lo (source: 12345-0@lo) 00000100:00100000:3.0:1776178522.978286:0:20755:0:(service.c:2152:ptlrpc_server_handle_req_in()) got req x1862456998093632 00000100:00100000:3.0:1776178522.978414:0:20755:0:(service.c:2152:ptlrpc_server_handle_req_in()) got req x1862456998093568 00000100:00100000:3.0:1776178522.978447:0:20755:0:(nrs_fifo.c:179:nrs_fifo_req_get()) NRS start fifo request from 12345-0@lo, seq: 761 00000100:00100000:3.0:1776178522.978456:0:20755:0:(service.c:2324:ptlrpc_server_handle_request()) Handling RPC req@00000000f52236ad pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_006:lustre-MDT0001-mdtlov_UUID+5:3356:x1862456998093632:12345-0@lo:400:kworker/u8:2.0 00000100:00100000:3.0:1776178522.978524:0:20755:0:(service.c:2372:ptlrpc_server_handle_request()) Handled RPC req@00000000f52236ad pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_006:lustre-MDT0001-mdtlov_UUID+5:3356:x1862456998093632:12345-0@lo:400:kworker/u8:2.0 Request processed in 68us (387us total) trans 0 rc 0/0 00000100:00100000:3.0:1776178522.978532:0:20755:0:(nrs_fifo.c:241:nrs_fifo_req_stop()) NRS stop fifo request from 12345-0@lo, seq: 761 00000100:00100000:0.0:1776178522.978536:0:6257:0:(service.c:2152:ptlrpc_server_handle_req_in()) got req x1862456998093312 00000100:00100000:3.0:1776178522.978553:0:20755:0:(nrs_fifo.c:179:nrs_fifo_req_get()) NRS start fifo request from 12345-0@lo, seq: 762 00000100:00100000:3.0:1776178522.978559:0:20755:0:(service.c:2324:ptlrpc_server_handle_request()) Handling RPC req@0000000067f71a07 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_006:lustre-MDT0001-mdtlov_UUID+5:3353:x1862456998093568:12345-0@lo:400:kworker/u8:2.0 00000100:00100000:0.0:1776178522.978579:0:6257:0:(service.c:2152:ptlrpc_server_handle_req_in()) got req x1862456998093440 00000100:00100000:3.0:1776178522.978603:0:20755:0:(service.c:2372:ptlrpc_server_handle_request()) Handled RPC req@0000000067f71a07 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_006:lustre-MDT0001-mdtlov_UUID+5:3353:x1862456998093568:12345-0@lo:400:kworker/u8:2.0 Request processed in 43us (358us total) trans 0 rc 0/0 00000100:00100000:0.0:1776178522.978608:0:6257:0:(nrs_fifo.c:179:nrs_fifo_req_get()) NRS start fifo request from 12345-0@lo, seq: 1115 00000100:00100000:3.0:1776178522.978616:0:20755:0:(nrs_fifo.c:241:nrs_fifo_req_stop()) NRS stop fifo request from 12345-0@lo, seq: 762 00000100:00100000:0.0:1776178522.978632:0:6257:0:(service.c:2324:ptlrpc_server_handle_request()) Handling RPC req@000000003ecedfd0 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_002:lustre-MDT0001-lwp-OST0000_UUID+5:3356:x1862456998093312:12345-0@lo:400:kworker/u8:2.0 00000100:00100000:0.0:1776178522.978681:0:6257:0:(service.c:2372:ptlrpc_server_handle_request()) Handled RPC req@000000003ecedfd0 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_002:lustre-MDT0001-lwp-OST0000_UUID+5:3356:x1862456998093312:12345-0@lo:400:kworker/u8:2.0 Request processed in 66us (759us total) trans 0 rc 0/0 00000100:00100000:0.0:1776178522.978689:0:6257:0:(nrs_fifo.c:241:nrs_fifo_req_stop()) NRS stop fifo request from 12345-0@lo, seq: 1115 00000100:00100000:3.0:1776178522.978704:0:3353:0:(client.c:2215:ptlrpc_check_set()) Completed RPC req@0000000048564d2a pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_00:lustre-MDT0001-mdtlov_UUID:3353:1862456998093568:0@lo:400:kworker/u8:2.0 00000100:00100000:0.0:1776178522.978705:0:6257:0:(nrs_fifo.c:179:nrs_fifo_req_get()) NRS start fifo request from 12345-0@lo, seq: 1116 00000100:00100000:0.0:1776178522.978714:0:6257:0:(service.c:2324:ptlrpc_server_handle_request()) Handling RPC req@000000006154cc2b pname:cluuid+ref:pid:xid:nid:opc:job mdt00_002:lustre-MDT0001-lwp-OST0001_UUID+5:3353:x1862456998093440:12345-0@lo:400:kworker/u8:2.0 00000100:00100000:3.0:1776178522.978755:0:3356:0:(client.c:2215:ptlrpc_check_set()) Completed RPC req@0000000080e59d72 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_03:lustre-MDT0001-mdtlov_UUID:3356:1862456998093632:0@lo:400:kworker/u8:2.0 00000100:00100000:3.0:1776178522.978770:0:3356:0:(client.c:2215:ptlrpc_check_set()) Completed RPC req@0000000070829000 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_03:lustre-MDT0001-lwp-OST0000_UUID:3356:1862456998093312:0@lo:400:kworker/u8:2.0 00000100:00100000:0.0:1776178522.978775:0:6257:0:(service.c:2372:ptlrpc_server_handle_request()) Handled RPC req@000000006154cc2b pname:cluuid+ref:pid:xid:nid:opc:job mdt00_002:lustre-MDT0001-lwp-OST0001_UUID+5:3353:x1862456998093440:12345-0@lo:400:kworker/u8:2.0 Request processed in 59us (574us total) trans 0 rc 0/0 00000100:00100000:0.0:1776178522.978784:0:6257:0:(nrs_fifo.c:241:nrs_fifo_req_stop()) NRS stop fifo request from 12345-0@lo, seq: 1116 00000100:00100000:1.0:1776178522.978869:0:3352:0:(client.c:1733:ptlrpc_send_new_req()) Sending RPC req@000000004f638b57 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:lustre-MDT0000-lwp-OST0000_UUID:3352:1862456998093248:0@lo:38:kworker/u8:2.0 00000100:00100000:1.0:1776178522.978905:0:3352:0:(events.c:361:request_in_callback()) peer: 12345-0@lo (source: 12345-0@lo) 00000100:00100000:0.0:1776178522.979251:0:20137:0:(service.c:2152:ptlrpc_server_handle_req_in()) got req x1862456998093184 00000100:00100000:0.0:1776178522.979298:0:20137:0:(nrs_fifo.c:179:nrs_fifo_req_get()) NRS start fifo request from 12345-0@lo, seq: 280 00000100:00100000:0.0:1776178522.979302:0:20137:0:(service.c:2324:ptlrpc_server_handle_request()) Handling RPC req@00000000b4857168 pname:cluuid+ref:pid:xid:nid:opc:job ll_mgs_0000:ac61f839-0e8e-4cac-9c59-08bcfbbad330+18:3356:x1862456998093184:12345-0@lo:400:kworker/u8:2.0 00000100:00100000:0.0:1776178522.979344:0:20137:0:(service.c:2372:ptlrpc_server_handle_request()) Handled RPC req@00000000b4857168 pname:cluuid+ref:pid:xid:nid:opc:job ll_mgs_0000:ac61f839-0e8e-4cac-9c59-08bcfbbad330+18:3356:x1862456998093184:12345-0@lo:400:kworker/u8:2.0 Request processed in 41us (1311us total) trans 0 rc 0/0 00000100:00100000:0.0:1776178522.979351:0:20137:0:(nrs_fifo.c:241:nrs_fifo_req_stop()) NRS stop fifo request from 12345-0@lo, seq: 280 00000100:00100000:0.0:1776178522.979406:0:3356:0:(client.c:2215:ptlrpc_check_set()) Completed RPC req@0000000082764007 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_03:ac61f839-0e8e-4cac-9c59-08bcfbbad330:3356:1862456998093184:0@lo:400:kworker/u8:2.0 00000100:00100000:0.0:1776178522.979444:0:3353:0:(client.c:2215:ptlrpc_check_set()) Completed RPC req@0000000012d88c00 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_00:lustre-MDT0001-lwp-OST0001_UUID:3353:1862456998093440:0@lo:400:kworker/u8:2.0 00000100:00100000:1.0:1776178522.980822:0:3352:0:(client.c:1733:ptlrpc_send_new_req()) Sending RPC req@000000009e484c71 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:lustre-MDT0000-lwp-OST0001_UUID:3352:1862456998093376:0@lo:38:kworker/u8:2.0 00000100:00100000:1.0:1776178522.980890:0:3352:0:(events.c:361:request_in_callback()) peer: 12345-0@lo (source: 12345-0@lo) 00000100:00100000:2.0:1776178522.990000:0:8250:0:(service.c:2152:ptlrpc_server_handle_req_in()) got req x1862456998093248 00000100:00100000:2.0:1776178522.990242:0:8250:0:(service.c:2152:ptlrpc_server_handle_req_in()) got req x1862456998093376 00000100:00100000:2.0:1776178522.990267:0:8250:0:(nrs_fifo.c:179:nrs_fifo_req_get()) NRS start fifo request from 12345-0@lo, seq: 6873 00000100:00100000:2.0:1776178522.990275:0:8250:0:(service.c:2324:ptlrpc_server_handle_request()) Handling RPC req@0000000072a0bf14 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_003:0+-99:3352:x1862456998093248:12345-0@lo:38:kworker/u8:2.0 00010000:02000000:2.0:1776178522.990300:0:8250:0:(ldlm_lib.c:1139:target_handle_connect()) lustre-MDT0000: Not available for connect from 0@lo (stopping) 00000100:00100000:2.0:1776178522.990419:0:8250:0:(service.c:2372:ptlrpc_server_handle_request()) Handled RPC req@0000000072a0bf14 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_003:0+-99:3352:x1862456998093248:12345-0@lo:38:kworker/u8:2.0 Request processed in 144us (11513us total) trans 0 rc -19/-19 00000100:00100000:2.0:1776178522.990426:0:8250:0:(nrs_fifo.c:241:nrs_fifo_req_stop()) NRS stop fifo request from 12345-0@lo, seq: 6873 00000100:00100000:2.0:1776178522.990456:0:8250:0:(nrs_fifo.c:179:nrs_fifo_req_get()) NRS start fifo request from 12345-0@lo, seq: 6874 00000100:00100000:2.0:1776178522.990460:0:8250:0:(service.c:2324:ptlrpc_server_handle_request()) Handling RPC req@0000000072c0bce9 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_003:0+-99:3352:x1862456998093376:12345-0@lo:38:kworker/u8:2.0 00010000:02000000:2.0:1776178522.990466:0:8250:0:(ldlm_lib.c:1139:target_handle_connect()) lustre-MDT0000: Not available for connect from 0@lo (stopping) 00000100:00100000:2.0:1776178522.990488:0:8250:0:(service.c:2372:ptlrpc_server_handle_request()) Handled RPC req@0000000072c0bce9 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_003:0+-99:3352:x1862456998093376:12345-0@lo:38:kworker/u8:2.0 Request processed in 27us (9599us total) trans 0 rc -19/-19 00000100:00100000:2.0:1776178522.990493:0:8250:0:(nrs_fifo.c:241:nrs_fifo_req_stop()) NRS stop fifo request from 12345-0@lo, seq: 6874 00000100:00080000:1.0:1776178522.993753:0:3352:0:(import.c:86:import_set_state_nolock()) 00000000dfc8db13 lustre-MDT0000_UUID: changing import state from CONNECTING to DISCONN 00000100:00080000:1.0:1776178522.993771:0:3352:0:(import.c:1429:ptlrpc_connect_interpret()) recovery of lustre-MDT0000_UUID on 192.168.204.103@tcp failed (-19) 00000100:00100000:1.0:1776178522.993784:0:3352:0:(client.c:2215:ptlrpc_check_set()) Completed RPC req@000000009e484c71 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:lustre-MDT0000-lwp-OST0001_UUID:3352:1862456998093376:0@lo:38:kworker/u8:2.0 00000100:00100000:1.0:1776178522.993799:0:3352:0:(client.c:1733:ptlrpc_send_new_req()) Sending RPC req@0000000013ad9ac7 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:lustre-MDT0001-mdtlov_UUID:3352:1862456998093504:0@lo:38:kworker/u8:2.0 00000100:00100000:1.0:1776178522.993875:0:3352:0:(events.c:361:request_in_callback()) peer: 12345-0@lo (source: 12345-0@lo) 00000100:00100000:1.0:1776178522.993910:0:3352:0:(client.c:1733:ptlrpc_send_new_req()) Sending RPC req@0000000065c78381 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:lustre-MDT0000-lwp-MDT0001_UUID:3352:1862456998093696:0@lo:38:kworker/u8:2.0 00000100:00100000:1.0:1776178522.993940:0:3352:0:(events.c:361:request_in_callback()) peer: 12345-0@lo (source: 12345-0@lo) 00000100:00080000:1.0:1776178522.994011:0:3352:0:(import.c:86:import_set_state_nolock()) 0000000047601a98 lustre-MDT0000_UUID: changing import state from CONNECTING to DISCONN 00000100:00080000:1.0:1776178522.994015:0:3352:0:(import.c:1429:ptlrpc_connect_interpret()) recovery of lustre-MDT0000_UUID on 192.168.204.103@tcp failed (-19) 00000100:00100000:1.0:1776178522.994019:0:3352:0:(client.c:2215:ptlrpc_check_set()) Completed RPC req@000000004f638b57 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:lustre-MDT0000-lwp-OST0000_UUID:3352:1862456998093248:0@lo:38:kworker/u8:2.0 00000100:00100000:2.0:1776178522.994676:0:6257:0:(service.c:2152:ptlrpc_server_handle_req_in()) got req x1862456998093696 00000100:00100000:2.0:1776178522.994741:0:6257:0:(nrs_fifo.c:179:nrs_fifo_req_get()) NRS start fifo request from 12345-0@lo, seq: 6875 00000100:00100000:2.0:1776178522.994749:0:6257:0:(service.c:2324:ptlrpc_server_handle_request()) Handling RPC req@00000000582bb477 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_002:0+-99:3352:x1862456998093696:12345-0@lo:38:kworker/u8:2.0 00010000:02000000:2.0:1776178522.994771:0:6257:0:(ldlm_lib.c:1139:target_handle_connect()) lustre-MDT0000: Not available for connect from 0@lo (stopping) 00000100:00100000:2.0:1776178522.994834:0:6257:0:(service.c:2372:ptlrpc_server_handle_request()) Handled RPC req@00000000582bb477 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_002:0+-99:3352:x1862456998093696:12345-0@lo:38:kworker/u8:2.0 Request processed in 84us (893us total) trans 0 rc -19/-19 00000100:00100000:2.0:1776178522.994844:0:6257:0:(nrs_fifo.c:241:nrs_fifo_req_stop()) NRS stop fifo request from 12345-0@lo, seq: 6875 00000100:00080000:1.0:1776178522.994971:0:3352:0:(import.c:86:import_set_state_nolock()) 00000000a7c55b1f lustre-MDT0000_UUID: changing import state from CONNECTING to DISCONN 00000100:00080000:1.0:1776178522.994976:0:3352:0:(import.c:1429:ptlrpc_connect_interpret()) recovery of lustre-MDT0000_UUID on 192.168.204.103@tcp failed (-19) 00000100:00100000:1.0:1776178522.994993:0:3352:0:(client.c:2215:ptlrpc_check_set()) Completed RPC req@0000000065c78381 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:lustre-MDT0000-lwp-MDT0001_UUID:3352:1862456998093696:0@lo:38:kworker/u8:2.0 00000100:00100000:3.0:1776178522.995148:0:15213:0:(service.c:2152:ptlrpc_server_handle_req_in()) got req x1862456998093504 00000100:00100000:3.0:1776178522.995186:0:15213:0:(nrs_fifo.c:179:nrs_fifo_req_get()) NRS start fifo request from 12345-0@lo, seq: 770 00000100:00100000:3.0:1776178522.995190:0:15213:0:(service.c:2324:ptlrpc_server_handle_request()) Handling RPC req@0000000009dce6fb pname:cluuid+ref:pid:xid:nid:opc:job mdt_out00_003:0+-99:3352:x1862456998093504:12345-0@lo:38:kworker/u8:2.0 00010000:02000000:3.0:1776178522.995199:0:15213:0:(ldlm_lib.c:1139:target_handle_connect()) lustre-MDT0000: Not available for connect from 0@lo (stopping) 00000100:00100000:3.0:1776178522.995245:0:15213:0:(service.c:2372:ptlrpc_server_handle_request()) Handled RPC req@0000000009dce6fb pname:cluuid+ref:pid:xid:nid:opc:job mdt_out00_003:0+-99:3352:x1862456998093504:12345-0@lo:38:kworker/u8:2.0 Request processed in 53us (1371us total) trans 0 rc -19/-19 00000100:00100000:3.0:1776178522.995251:0:15213:0:(nrs_fifo.c:241:nrs_fifo_req_stop()) NRS stop fifo request from 12345-0@lo, seq: 770 00000100:00080000:2.0:1776178522.995353:0:3352:0:(import.c:86:import_set_state_nolock()) 00000000ba4f1821 lustre-MDT0000_UUID: changing import state from CONNECTING to DISCONN 00000100:00080000:2.0:1776178522.995357:0:3352:0:(import.c:1429:ptlrpc_connect_interpret()) recovery of lustre-MDT0000_UUID on 192.168.204.103@tcp failed (-19) 00000100:00100000:2.0:1776178522.995361:0:3352:0:(client.c:2215:ptlrpc_check_set()) Completed RPC req@0000000013ad9ac7 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:lustre-MDT0001-mdtlov_UUID:3352:1862456998093504:0@lo:38:kworker/u8:2.0 00010000:00010000:1.0:1776178523.623034:0:9225:0:(ldlm_lockd.c:2792:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:0.0:1776178524.516430:0:3356:0:(client.c:1733:ptlrpc_send_new_req()) Sending RPC req@00000000bea25c66 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_03:lustre-MDT0001-mdtlov_UUID:3356:1862456998093760:0@lo:13:osp-pre-1-1.0 00000100:00100000:0.0:1776178524.516533:0:3356:0:(events.c:361:request_in_callback()) peer: 12345-0@lo (source: 12345-0@lo) 00000100:00100000:3.0:1776178524.516681:0:8231:0:(service.c:2152:ptlrpc_server_handle_req_in()) got req x1862456998093760 00000100:00100000:3.0:1776178524.516750:0:8231:0:(nrs_fifo.c:179:nrs_fifo_req_get()) NRS start fifo request from 12345-0@lo, seq: 613 00000100:00100000:3.0:1776178524.516757:0:8231:0:(service.c:2324:ptlrpc_server_handle_request()) Handling RPC req@000000001c9b18be pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_create00:lustre-MDT0001-mdtlov_UUID+5:3356:x1862456998093760:12345-0@lo:13:osp-pre-1-1.0 00002000:00000024:3.0:1776178524.516806:0:8231:0:(ofd_obd.c:740:ofd_statfs()) blocks cached 229376 granted 491746368 pending 0 free 3923689472 avail 3692167168 00000100:00100000:3.0:1776178524.516882:0:8231:0:(service.c:2372:ptlrpc_server_handle_request()) Handled RPC req@000000001c9b18be pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_create00:lustre-MDT0001-mdtlov_UUID+5:3356:x1862456998093760:12345-0@lo:13:osp-pre-1-1.0 Request processed in 123us (349us total) trans 0 rc 0/0 00000100:00100000:3.0:1776178524.516892:0:8231:0:(nrs_fifo.c:241:nrs_fifo_req_stop()) NRS stop fifo request from 12345-0@lo, seq: 613 00000100:00100000:1.0:1776178524.517007:0:3356:0:(client.c:2215:ptlrpc_check_set()) Completed RPC req@00000000bea25c66 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_03:lustre-MDT0001-mdtlov_UUID:3356:1862456998093760:0@lo:13:osp-pre-1-1.0 00010000:00010000:3.0:1776178524.641956:0:6244:0:(ldlm_lockd.c:2792:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:3.0:1776178525.024604:0:3353:0:(client.c:1733:ptlrpc_send_new_req()) Sending RPC req@00000000d22cceea pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_00:lustre-MDT0001-mdtlov_UUID:3353:1862456998093824:0@lo:13:osp-pre-0-1.0 00000100:00100000:3.0:1776178525.024710:0:3353:0:(events.c:361:request_in_callback()) peer: 12345-0@lo (source: 12345-0@lo) 00000100:00100000:0.0:1776178525.024889:0:9554:0:(service.c:2152:ptlrpc_server_handle_req_in()) got req x1862456998093824 00000100:00100000:0.0:1776178525.024965:0:9554:0:(nrs_fifo.c:179:nrs_fifo_req_get()) NRS start fifo request from 12345-0@lo, seq: 614 00000100:00100000:0.0:1776178525.024973:0:9554:0:(service.c:2324:ptlrpc_server_handle_request()) Handling RPC req@000000009d0d1846 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_create00:lustre-MDT0001-mdtlov_UUID+5:3353:x1862456998093824:12345-0@lo:13:osp-pre-0-1.0 00002000:00000024:0.0:1776178525.025030:0:9554:0:(ofd_obd.c:740:ofd_statfs()) blocks cached 315392 granted 484883776 pending 0 free 3923689472 avail 3692167168 00000100:00100000:0.0:1776178525.025146:0:9554:0:(service.c:2372:ptlrpc_server_handle_request()) Handled RPC req@000000009d0d1846 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_create00:lustre-MDT0001-mdtlov_UUID+5:3353:x1862456998093824:12345-0@lo:13:osp-pre-0-1.0 Request processed in 172us (437us total) trans 0 rc 0/0 00000100:00100000:0.0:1776178525.025153:0:9554:0:(nrs_fifo.c:241:nrs_fifo_req_stop()) NRS stop fifo request from 12345-0@lo, seq: 614 00000100:00100000:1.0:1776178525.025334:0:3353:0:(client.c:2215:ptlrpc_check_set()) Completed RPC req@00000000d22cceea pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_00:lustre-MDT0001-mdtlov_UUID:3353:1862456998093824:0@lo:13:osp-pre-0-1.0 00010000:00010000:2.0:1776178525.667886:0:6245:0:(ldlm_lockd.c:2792:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:2.0:1776178526.504769:0:3347:0:(events.c:361:request_in_callback()) peer: 12345-192.168.204.3@tcp (source: 12345-192.168.204.3@tcp) 00000100:00100000:2.0:1776178526.504829:0:3347:0:(events.c:361:request_in_callback()) peer: 12345-192.168.204.3@tcp (source: 12345-192.168.204.3@tcp) 00000100:00100000:2.0:1776178526.504866:0:3347:0:(events.c:361:request_in_callback()) peer: 12345-192.168.204.3@tcp (source: 12345-192.168.204.3@tcp) 00000100:00100000:2.0:1776178526.504913:0:3347:0:(events.c:361:request_in_callback()) peer: 12345-192.168.204.3@tcp (source: 12345-192.168.204.3@tcp) 00000100:00100000:1.0:1776178526.505082:0:6256:0:(service.c:2152:ptlrpc_server_handle_req_in()) got req x1862456979129280 00000100:00100000:1.0:1776178526.505156:0:6256:0:(nrs_fifo.c:179:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.204.3@tcp, seq: 1117 00000100:00100000:1.0:1776178526.505165:0:6256:0:(service.c:2324:ptlrpc_server_handle_request()) Handling RPC req@0000000037b0061a pname:cluuid+ref:pid:xid:nid:opc:job mdt00_001:b50963bd-8eba-4432-9a54-8888d00e3fa1+6:2271:x1862456979129280:12345-192.168.204.3@tcp:400:kworker/u8:4.0 00000100:00100000:1.0:1776178526.505285:0:6256:0:(service.c:2372:ptlrpc_server_handle_request()) Handled RPC req@0000000037b0061a pname:cluuid+ref:pid:xid:nid:opc:job mdt00_001:b50963bd-8eba-4432-9a54-8888d00e3fa1+6:2271:x1862456979129280:12345-192.168.204.3@tcp:400:kworker/u8:4.0 Request processed in 120us (419us total) trans 0 rc 0/0 00000100:00100000:1.0:1776178526.505295:0:6256:0:(nrs_fifo.c:241:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.204.3@tcp, seq: 1117 00000100:00100000:3.0:1776178526.505736:0:14440:0:(service.c:2152:ptlrpc_server_handle_req_in()) got req x1862456979129344 00000100:00100000:3.0:1776178526.505780:0:14440:0:(service.c:2152:ptlrpc_server_handle_req_in()) got req x1862456979129408 00000100:00100000:3.0:1776178526.505807:0:14440:0:(nrs_fifo.c:179:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.204.3@tcp, seq: 763 00000100:00100000:3.0:1776178526.505811:0:14440:0:(service.c:2324:ptlrpc_server_handle_request()) Handling RPC req@0000000013ad9ac7 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_004:b50963bd-8eba-4432-9a54-8888d00e3fa1+8:2271:x1862456979129344:12345-192.168.204.3@tcp:400:kworker/u8:4.0 00000100:00100000:3.0:1776178526.505853:0:14440:0:(service.c:2372:ptlrpc_server_handle_request()) Handled RPC req@0000000013ad9ac7 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_004:b50963bd-8eba-4432-9a54-8888d00e3fa1+8:2271:x1862456979129344:12345-192.168.204.3@tcp:400:kworker/u8:4.0 Request processed in 41us (1024us total) trans 0 rc 0/0 00000100:00100000:3.0:1776178526.505860:0:14440:0:(nrs_fifo.c:241:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.204.3@tcp, seq: 763 00000100:00100000:3.0:1776178526.505872:0:14440:0:(nrs_fifo.c:179:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.204.3@tcp, seq: 764 00000100:00100000:3.0:1776178526.505876:0:14440:0:(service.c:2324:ptlrpc_server_handle_request()) Handling RPC req@000000009dd1dd0f pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_004:b50963bd-8eba-4432-9a54-8888d00e3fa1+9:2271:x1862456979129408:12345-192.168.204.3@tcp:400:kworker/u8:4.0 00000100:00100000:3.0:1776178526.505896:0:14440:0:(service.c:2372:ptlrpc_server_handle_request()) Handled RPC req@000000009dd1dd0f pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_004:b50963bd-8eba-4432-9a54-8888d00e3fa1+9:2271:x1862456979129408:12345-192.168.204.3@tcp:400:kworker/u8:4.0 Request processed in 19us (984us total) trans 0 rc 0/0 00000100:00100000:3.0:1776178526.505902:0:14440:0:(nrs_fifo.c:241:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.204.3@tcp, seq: 764 00000100:00100000:0.0:1776178526.506028:0:20140:0:(service.c:2152:ptlrpc_server_handle_req_in()) got req x1862456979129152 00000100:00100000:0.0:1776178526.506238:0:20140:0:(nrs_fifo.c:179:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.204.3@tcp, seq: 281 00000100:00100000:0.0:1776178526.506244:0:20140:0:(service.c:2324:ptlrpc_server_handle_request()) Handling RPC req@0000000088793b53 pname:cluuid+ref:pid:xid:nid:opc:job ll_mgs_0002:5b5c66b1-1a2f-4651-8548-0c75b9a38a94+9:2271:x1862456979129152:12345-192.168.204.3@tcp:400:kworker/u8:4.0 00000100:00100000:0.0:1776178526.506278:0:20140:0:(service.c:2372:ptlrpc_server_handle_request()) Handled RPC req@0000000088793b53 pname:cluuid+ref:pid:xid:nid:opc:job ll_mgs_0002:5b5c66b1-1a2f-4651-8548-0c75b9a38a94+9:2271:x1862456979129152:12345-192.168.204.3@tcp:400:kworker/u8:4.0 Request processed in 34us (1526us total) trans 0 rc 0/0 00000100:00100000:0.0:1776178526.506285:0:20140:0:(nrs_fifo.c:241:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.204.3@tcp, seq: 281 00000100:00100000:2.0:1776178526.514548:0:3347:0:(events.c:361:request_in_callback()) peer: 12345-192.168.204.3@tcp (source: 12345-192.168.204.3@tcp) 00000100:00100000:0.0:1776178526.514789:0:6257:0:(service.c:2152:ptlrpc_server_handle_req_in()) got req x1862456979129216 00000100:00100000:0.0:1776178526.514864:0:6257:0:(nrs_fifo.c:179:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.204.3@tcp, seq: 6876 00000100:00100000:0.0:1776178526.514874:0:6257:0:(service.c:2324:ptlrpc_server_handle_request()) Handling RPC req@00000000f20bd8f9 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_002:0+-99:2269:x1862456979129216:12345-192.168.204.3@tcp:38:kworker/u8:4.0 00010000:02000000:0.0:1776178526.514902:0:6257:0:(ldlm_lib.c:1139:target_handle_connect()) lustre-MDT0000: Not available for connect from 192.168.204.3@tcp (stopping) 00000100:00100000:0.0:1776178526.515257:0:6257:0:(service.c:2372:ptlrpc_server_handle_request()) Handled RPC req@00000000f20bd8f9 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_002:0+-99:2269:x1862456979129216:12345-192.168.204.3@tcp:38:kworker/u8:4.0 Request processed in 383us (726us total) trans 0 rc -19/-19 00000100:00100000:0.0:1776178526.515267:0:6257:0:(nrs_fifo.c:241:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.204.3@tcp, seq: 6876 00010000:00010000:1.0:1776178526.688367:0:9224:0:(ldlm_lockd.c:2792:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00010000:00010000:1.0:1776178527.718543:0:9225:0:(ldlm_lockd.c:2792:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00080000:2.0:1776178528.096311:0:37:0:(pinger.c:251:ptlrpc_pinger_process_import()) lustre-MDT0000-lwp-OST0000_UUID->lustre-MDT0000_UUID: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00080000:2.0:1776178528.096321:0:37:0:(recover.c:57:ptlrpc_initiate_recovery()) lustre-MDT0000_UUID: starting recovery 00000100:00080000:2.0:1776178528.096326:0:37:0:(import.c:86:import_set_state_nolock()) 0000000047601a98 lustre-MDT0000_UUID: changing import state from DISCONN to CONNECTING 00000100:00080000:2.0:1776178528.096337:0:37:0:(import.c:534:import_select_connection()) lustre-MDT0000-lwp-OST0000: connect to NID 0@lo last attempt 1230 00000100:00080000:2.0:1776178528.096342:0:37:0:(import.c:581:import_select_connection()) lustre-MDT0000-lwp-OST0000: tried all connections, increasing latency to 21s 00000100:00080000:2.0:1776178528.096350:0:37:0:(import.c:615:import_select_connection()) lustre-MDT0000-lwp-OST0000: import 0000000047601a98 using connection 192.168.204.103@tcp/0@lo 00000100:00100000:2.0:1776178528.096397:0:37:0:(import.c:807:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5) req@0000000098682d9e x1862456998093952/t0(0) o38->lustre-MDT0000-lwp-OST0000@0@lo:12/10 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' 00000100:00080000:2.0:1776178528.096452:0:37:0:(pinger.c:251:ptlrpc_pinger_process_import()) lustre-MDT0000-lwp-OST0001_UUID->lustre-MDT0000_UUID: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00080000:2.0:1776178528.096456:0:37:0:(recover.c:57:ptlrpc_initiate_recovery()) lustre-MDT0000_UUID: starting recovery 00000100:00080000:2.0:1776178528.096457:0:37:0:(import.c:86:import_set_state_nolock()) 00000000dfc8db13 lustre-MDT0000_UUID: changing import state from DISCONN to CONNECTING 00000100:00080000:2.0:1776178528.096461:0:37:0:(import.c:534:import_select_connection()) lustre-MDT0000-lwp-OST0001: connect to NID 0@lo last attempt 1230 00000100:00080000:2.0:1776178528.096463:0:37:0:(import.c:581:import_select_connection()) lustre-MDT0000-lwp-OST0001: tried all connections, increasing latency to 21s 00000100:00080000:2.0:1776178528.096467:0:37:0:(import.c:615:import_select_connection()) lustre-MDT0000-lwp-OST0001: import 00000000dfc8db13 using connection 192.168.204.103@tcp/0@lo 00000100:00100000:2.0:1776178528.096479:0:37:0:(import.c:807:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5) req@00000000ef4da5b8 x1862456998094080/t0(0) o38->lustre-MDT0000-lwp-OST0001@0@lo:12/10 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' 00000100:00080000:2.0:1776178528.096497:0:37:0:(pinger.c:251:ptlrpc_pinger_process_import()) lustre-MDT0001-mdtlov_UUID->lustre-MDT0000_UUID: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00080000:2.0:1776178528.096501:0:37:0:(recover.c:57:ptlrpc_initiate_recovery()) lustre-MDT0000_UUID: starting recovery 00000100:00080000:2.0:1776178528.096502:0:37:0:(import.c:86:import_set_state_nolock()) 00000000ba4f1821 lustre-MDT0000_UUID: changing import state from DISCONN to CONNECTING 00000100:00080000:2.0:1776178528.096506:0:37:0:(import.c:534:import_select_connection()) lustre-MDT0000-osp-MDT0001: connect to NID 0@lo last attempt 1230 00000100:00080000:2.0:1776178528.096507:0:37:0:(import.c:581:import_select_connection()) lustre-MDT0000-osp-MDT0001: tried all connections, increasing latency to 21s 00000100:00080000:2.0:1776178528.096512:0:37:0:(import.c:615:import_select_connection()) lustre-MDT0000-osp-MDT0001: import 00000000ba4f1821 using connection 192.168.204.103@tcp/0@lo 00000100:00100000:2.0:1776178528.096539:0:37:0:(import.c:807:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5) req@00000000b766fbe9 x1862456998094208/t0(0) o38->lustre-MDT0000-osp-MDT0001@0@lo:24/4 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' 00000100:00080000:2.0:1776178528.096562:0:37:0:(pinger.c:251:ptlrpc_pinger_process_import()) lustre-MDT0000-lwp-MDT0001_UUID->lustre-MDT0000_UUID: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00080000:2.0:1776178528.096566:0:37:0:(recover.c:57:ptlrpc_initiate_recovery()) lustre-MDT0000_UUID: starting recovery 00000100:00080000:2.0:1776178528.096568:0:37:0:(import.c:86:import_set_state_nolock()) 00000000a7c55b1f lustre-MDT0000_UUID: changing import state from DISCONN to CONNECTING 00000100:00080000:2.0:1776178528.096572:0:37:0:(import.c:534:import_select_connection()) lustre-MDT0000-lwp-MDT0001: connect to NID 0@lo last attempt 1230 00000100:00080000:2.0:1776178528.096574:0:37:0:(import.c:581:import_select_connection()) lustre-MDT0000-lwp-MDT0001: tried all connections, increasing latency to 21s 00000100:00080000:2.0:1776178528.096578:0:37:0:(import.c:615:import_select_connection()) lustre-MDT0000-lwp-MDT0001: import 00000000a7c55b1f using connection 192.168.204.103@tcp/0@lo 00000100:00100000:2.0:1776178528.096588:0:37:0:(import.c:807:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5) req@0000000065c78381 x1862456998094400/t0(0) o38->lustre-MDT0000-lwp-MDT0001@0@lo:12/10 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' 00000100:00100000:1.0:1776178528.096595:0:3354:0:(client.c:1733:ptlrpc_send_new_req()) Sending RPC req@000000002db08e42 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_01:ac61f839-0e8e-4cac-9c59-08bcfbbad330:3354:1862456998093888:0@lo:400:kworker/u8:2.0 00000100:00100000:2.0:1776178528.096670:0:3355:0:(client.c:1733:ptlrpc_send_new_req()) Sending RPC req@0000000021958fc7 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_02:lustre-MDT0001-lwp-OST0000_UUID:3355:1862456998094016:0@lo:400:kworker/u8:2.0 00000100:00100000:1.0:1776178528.096706:0:3354:0:(events.c:361:request_in_callback()) peer: 12345-0@lo (source: 12345-0@lo) 00000100:00100000:3.0:1776178528.096754:0:3353:0:(client.c:1733:ptlrpc_send_new_req()) Sending RPC req@00000000598050cc pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_00:lustre-MDT0001-mdtlov_UUID:3353:1862456998094272:0@lo:400:kworker/u8:2.0 00000100:00100000:2.0:1776178528.097045:0:3355:0:(events.c:361:request_in_callback()) peer: 12345-0@lo (source: 12345-0@lo) 00000100:00100000:1.0:1776178528.097127:0:3354:0:(client.c:1733:ptlrpc_send_new_req()) Sending RPC req@000000009ff44a0a pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_01:lustre-MDT0001-mdtlov_UUID:3354:1862456998094336:0@lo:400:kworker/u8:2.0 00000100:00100000:1.0:1776178528.097169:0:3354:0:(events.c:361:request_in_callback()) peer: 12345-0@lo (source: 12345-0@lo) 00000100:00100000:1.0:1776178528.097216:0:3356:0:(client.c:1733:ptlrpc_send_new_req()) Sending RPC req@000000007d1fa4e3 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_03:lustre-MDT0001-lwp-OST0001_UUID:3356:1862456998094144:0@lo:400:kworker/u8:2.0 00000100:00100000:1.0:1776178528.097270:0:20137:0:(service.c:2152:ptlrpc_server_handle_req_in()) got req x1862456998093888 00000100:00100000:1.0:1776178528.097339:0:20137:0:(nrs_fifo.c:179:nrs_fifo_req_get()) NRS start fifo request from 12345-0@lo, seq: 282 00000100:00100000:1.0:1776178528.097347:0:20137:0:(service.c:2324:ptlrpc_server_handle_request()) Handling RPC req@000000004c7a25f9 pname:cluuid+ref:pid:xid:nid:opc:job ll_mgs_0000:ac61f839-0e8e-4cac-9c59-08bcfbbad330+18:3354:x1862456998093888:12345-0@lo:400:kworker/u8:2.0 00000100:00100000:1.0:1776178528.097428:0:20137:0:(service.c:2372:ptlrpc_server_handle_request()) Handled RPC req@000000004c7a25f9 pname:cluuid+ref:pid:xid:nid:opc:job ll_mgs_0000:ac61f839-0e8e-4cac-9c59-08bcfbbad330+18:3354:x1862456998093888:12345-0@lo:400:kworker/u8:2.0 Request processed in 81us (724us total) trans 0 rc 0/0 00000100:00100000:1.0:1776178528.097437:0:20137:0:(nrs_fifo.c:241:nrs_fifo_req_stop()) NRS stop fifo request from 12345-0@lo, seq: 282 00000100:00100000:1.0:1776178528.097487:0:20755:0:(service.c:2152:ptlrpc_server_handle_req_in()) got req x1862456998094336 00000100:00100000:1.0:1776178528.097536:0:20755:0:(nrs_fifo.c:179:nrs_fifo_req_get()) NRS start fifo request from 12345-0@lo, seq: 765 00000100:00100000:1.0:1776178528.097539:0:20755:0:(service.c:2324:ptlrpc_server_handle_request()) Handling RPC req@0000000012d4bd5e pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_006:lustre-MDT0001-mdtlov_UUID+5:3354:x1862456998094336:12345-0@lo:400:kworker/u8:2.0 00000100:00100000:1.0:1776178528.097574:0:20755:0:(service.c:2372:ptlrpc_server_handle_request()) Handled RPC req@0000000012d4bd5e pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_006:lustre-MDT0001-mdtlov_UUID+5:3354:x1862456998094336:12345-0@lo:400:kworker/u8:2.0 Request processed in 34us (405us total) trans 0 rc 0/0 00000100:00100000:1.0:1776178528.097581:0:20755:0:(nrs_fifo.c:241:nrs_fifo_req_stop()) NRS stop fifo request from 12345-0@lo, seq: 765 00000100:00100000:1.0:1776178528.097636:0:3354:0:(client.c:2215:ptlrpc_check_set()) Completed RPC req@000000002db08e42 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_01:ac61f839-0e8e-4cac-9c59-08bcfbbad330:3354:1862456998093888:0@lo:400:kworker/u8:2.0 00000100:00100000:1.0:1776178528.097656:0:3354:0:(client.c:2215:ptlrpc_check_set()) Completed RPC req@000000009ff44a0a pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_01:lustre-MDT0001-mdtlov_UUID:3354:1862456998094336:0@lo:400:kworker/u8:2.0 00000100:00100000:3.0:1776178528.100881:0:3353:0:(events.c:361:request_in_callback()) peer: 12345-0@lo (source: 12345-0@lo) 00000100:00100000:2.0:1776178528.101848:0:6257:0:(service.c:2152:ptlrpc_server_handle_req_in()) got req x1862456998094016 00000100:00100000:2.0:1776178528.102562:0:6257:0:(nrs_fifo.c:179:nrs_fifo_req_get()) NRS start fifo request from 12345-0@lo, seq: 1118 00000100:00100000:2.0:1776178528.102572:0:6257:0:(service.c:2324:ptlrpc_server_handle_request()) Handling RPC req@000000004f638b57 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_002:lustre-MDT0001-lwp-OST0000_UUID+5:3355:x1862456998094016:12345-0@lo:400:kworker/u8:2.0 00000100:00100000:2.0:1776178528.102667:0:6257:0:(service.c:2372:ptlrpc_server_handle_request()) Handled RPC req@000000004f638b57 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_002:lustre-MDT0001-lwp-OST0000_UUID+5:3355:x1862456998094016:12345-0@lo:400:kworker/u8:2.0 Request processed in 97us (5623us total) trans 0 rc 0/0 00000100:00100000:2.0:1776178528.102676:0:6257:0:(nrs_fifo.c:241:nrs_fifo_req_stop()) NRS stop fifo request from 12345-0@lo, seq: 1118 00000100:00100000:2.0:1776178528.102734:0:3355:0:(client.c:2215:ptlrpc_check_set()) Completed RPC req@0000000021958fc7 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_02:lustre-MDT0001-lwp-OST0000_UUID:3355:1862456998094016:0@lo:400:kworker/u8:2.0 00000100:00100000:0.0:1776178528.102861:0:3352:0:(client.c:1733:ptlrpc_send_new_req()) Sending RPC req@0000000098682d9e pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:lustre-MDT0000-lwp-OST0000_UUID:3352:1862456998093952:0@lo:38:kworker/u8:2.0 00000100:00100000:0.0:1776178528.103268:0:3352:0:(events.c:361:request_in_callback()) peer: 12345-0@lo (source: 12345-0@lo) 00000100:00100000:0.0:1776178528.103293:0:3352:0:(client.c:1733:ptlrpc_send_new_req()) Sending RPC req@00000000ef4da5b8 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:lustre-MDT0000-lwp-OST0001_UUID:3352:1862456998094080:0@lo:38:kworker/u8:2.0 00000100:00100000:0.0:1776178528.103317:0:3352:0:(events.c:361:request_in_callback()) peer: 12345-0@lo (source: 12345-0@lo) 00000100:00100000:0.0:1776178528.103346:0:8250:0:(service.c:2152:ptlrpc_server_handle_req_in()) got req x1862456998093952 00000100:00100000:0.0:1776178528.103378:0:8250:0:(service.c:2152:ptlrpc_server_handle_req_in()) got req x1862456998094080 00000100:00100000:0.0:1776178528.103396:0:8250:0:(nrs_fifo.c:179:nrs_fifo_req_get()) NRS start fifo request from 12345-0@lo, seq: 6877 00000100:00100000:0.0:1776178528.103401:0:8250:0:(service.c:2324:ptlrpc_server_handle_request()) Handling RPC req@00000000c31dc70e pname:cluuid+ref:pid:xid:nid:opc:job mdt00_003:0+-99:3352:x1862456998093952:12345-0@lo:38:kworker/u8:2.0 00000100:00100000:1.0:1776178528.103408:0:3356:0:(events.c:361:request_in_callback()) peer: 12345-0@lo (source: 12345-0@lo) 00010000:02000000:0.0:1776178528.103421:0:8250:0:(ldlm_lib.c:1139:target_handle_connect()) lustre-MDT0000: Not available for connect from 0@lo (stopping) 00000100:00100000:1.0:1776178528.103461:0:20759:0:(service.c:2152:ptlrpc_server_handle_req_in()) got req x1862456998094272 00000100:00100000:0.0:1776178528.103463:0:8250:0:(service.c:2372:ptlrpc_server_handle_request()) Handled RPC req@00000000c31dc70e pname:cluuid+ref:pid:xid:nid:opc:job mdt00_003:0+-99:3352:x1862456998093952:12345-0@lo:38:kworker/u8:2.0 Request processed in 61us (195us total) trans 0 rc -19/-19 00000100:00100000:0.0:1776178528.103471:0:8250:0:(nrs_fifo.c:241:nrs_fifo_req_stop()) NRS stop fifo request from 12345-0@lo, seq: 6877 00000100:00100000:0.0:1776178528.103487:0:8250:0:(service.c:2152:ptlrpc_server_handle_req_in()) got req x1862456998094144 00000100:00100000:1.0:1776178528.103508:0:20759:0:(nrs_fifo.c:179:nrs_fifo_req_get()) NRS start fifo request from 12345-0@lo, seq: 766 00000100:00100000:0.0:1776178528.103511:0:8250:0:(nrs_fifo.c:179:nrs_fifo_req_get()) NRS start fifo request from 12345-0@lo, seq: 1119 00000100:00100000:1.0:1776178528.103515:0:20759:0:(service.c:2324:ptlrpc_server_handle_request()) Handling RPC req@000000003c23bd8e pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_010:lustre-MDT0001-mdtlov_UUID+5:3353:x1862456998094272:12345-0@lo:400:kworker/u8:2.0 00000100:00100000:0.0:1776178528.103521:0:8250:0:(service.c:2324:ptlrpc_server_handle_request()) Handling RPC req@0000000034d11618 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_003:lustre-MDT0001-lwp-OST0001_UUID+5:3356:x1862456998094144:12345-0@lo:400:kworker/u8:2.0 00000100:00100000:0.0:1776178528.103572:0:8250:0:(service.c:2372:ptlrpc_server_handle_request()) Handled RPC req@0000000034d11618 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_003:lustre-MDT0001-lwp-OST0001_UUID+5:3356:x1862456998094144:12345-0@lo:400:kworker/u8:2.0 Request processed in 54us (166us total) trans 0 rc 0/0 00000100:00100000:0.0:1776178528.103578:0:8250:0:(nrs_fifo.c:241:nrs_fifo_req_stop()) NRS stop fifo request from 12345-0@lo, seq: 1119 00000100:00100000:0.0:1776178528.103590:0:8250:0:(nrs_fifo.c:179:nrs_fifo_req_get()) NRS start fifo request from 12345-0@lo, seq: 6878 00000100:00100000:0.0:1776178528.103593:0:8250:0:(service.c:2324:ptlrpc_server_handle_request()) Handling RPC req@00000000ecea139c pname:cluuid+ref:pid:xid:nid:opc:job mdt00_003:0+-99:3352:x1862456998094080:12345-0@lo:38:kworker/u8:2.0 00010000:02000000:0.0:1776178528.103600:0:8250:0:(ldlm_lib.c:1139:target_handle_connect()) lustre-MDT0000: Not available for connect from 0@lo (stopping) 00000100:00100000:0.0:1776178528.103622:0:8250:0:(service.c:2372:ptlrpc_server_handle_request()) Handled RPC req@00000000ecea139c pname:cluuid+ref:pid:xid:nid:opc:job mdt00_003:0+-99:3352:x1862456998094080:12345-0@lo:38:kworker/u8:2.0 Request processed in 28us (304us total) trans 0 rc -19/-19 00000100:00100000:0.0:1776178528.103628:0:8250:0:(nrs_fifo.c:241:nrs_fifo_req_stop()) NRS stop fifo request from 12345-0@lo, seq: 6878 00000100:00100000:0.0:1776178528.103672:0:3356:0:(client.c:2215:ptlrpc_check_set()) Completed RPC req@000000007d1fa4e3 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_03:lustre-MDT0001-lwp-OST0001_UUID:3356:1862456998094144:0@lo:400:kworker/u8:2.0 00000100:00100000:0.0:1776178528.103702:0:3352:0:(client.c:1733:ptlrpc_send_new_req()) Sending RPC req@00000000b766fbe9 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:lustre-MDT0001-mdtlov_UUID:3352:1862456998094208:0@lo:38:kworker/u8:2.0 00000100:00100000:0.0:1776178528.103734:0:3352:0:(events.c:361:request_in_callback()) peer: 12345-0@lo (source: 12345-0@lo) 00000100:00100000:1.0:1776178528.103845:0:20759:0:(service.c:2372:ptlrpc_server_handle_request()) Handled RPC req@000000003c23bd8e pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_010:lustre-MDT0001-mdtlov_UUID+5:3353:x1862456998094272:12345-0@lo:400:kworker/u8:2.0 Request processed in 330us (2970us total) trans 0 rc 0/0 00000100:00100000:1.0:1776178528.103852:0:20759:0:(nrs_fifo.c:241:nrs_fifo_req_stop()) NRS stop fifo request from 12345-0@lo, seq: 766 00000100:00100000:1.0:1776178528.103878:0:14128:0:(service.c:2152:ptlrpc_server_handle_req_in()) got req x1862456998094208 00000100:00100000:1.0:1776178528.103900:0:14128:0:(nrs_fifo.c:179:nrs_fifo_req_get()) NRS start fifo request from 12345-0@lo, seq: 771 00000100:00100000:1.0:1776178528.103904:0:14128:0:(service.c:2324:ptlrpc_server_handle_request()) Handling RPC req@000000007cee9cf6 pname:cluuid+ref:pid:xid:nid:opc:job mdt_out00_002:0+-99:3352:x1862456998094208:12345-0@lo:38:kworker/u8:2.0 00010000:02000000:1.0:1776178528.103912:0:14128:0:(ldlm_lib.c:1139:target_handle_connect()) lustre-MDT0000: Not available for connect from 0@lo (stopping) 00000100:00100000:1.0:1776178528.103939:0:14128:0:(service.c:2372:ptlrpc_server_handle_request()) Handled RPC req@000000007cee9cf6 pname:cluuid+ref:pid:xid:nid:opc:job mdt_out00_002:0+-99:3352:x1862456998094208:12345-0@lo:38:kworker/u8:2.0 Request processed in 34us (203us total) trans 0 rc -19/-19 00000100:00100000:1.0:1776178528.103946:0:14128:0:(nrs_fifo.c:241:nrs_fifo_req_stop()) NRS stop fifo request from 12345-0@lo, seq: 771 00000100:00100000:3.0:1776178528.104283:0:3353:0:(client.c:2215:ptlrpc_check_set()) Completed RPC req@00000000598050cc pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_00:lustre-MDT0001-mdtlov_UUID:3353:1862456998094272:0@lo:400:kworker/u8:2.0 00000100:00080000:0.0:1776178528.104358:0:3352:0:(import.c:86:import_set_state_nolock()) 00000000ba4f1821 lustre-MDT0000_UUID: changing import state from CONNECTING to DISCONN 00000100:00080000:0.0:1776178528.104364:0:3352:0:(import.c:1429:ptlrpc_connect_interpret()) recovery of lustre-MDT0000_UUID on 192.168.204.103@tcp failed (-19) 00000100:00100000:0.0:1776178528.104367:0:3352:0:(client.c:2215:ptlrpc_check_set()) Completed RPC req@00000000b766fbe9 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:lustre-MDT0001-mdtlov_UUID:3352:1862456998094208:0@lo:38:kworker/u8:2.0 00000100:00100000:0.0:1776178528.104374:0:3352:0:(client.c:1733:ptlrpc_send_new_req()) Sending RPC req@0000000065c78381 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:lustre-MDT0000-lwp-MDT0001_UUID:3352:1862456998094400:0@lo:38:kworker/u8:2.0 00000100:00100000:0.0:1776178528.104402:0:3352:0:(events.c:361:request_in_callback()) peer: 12345-0@lo (source: 12345-0@lo) 00000100:00080000:0.0:1776178528.104659:0:3352:0:(import.c:86:import_set_state_nolock()) 0000000047601a98 lustre-MDT0000_UUID: changing import state from CONNECTING to DISCONN 00000100:00080000:0.0:1776178528.104664:0:3352:0:(import.c:1429:ptlrpc_connect_interpret()) recovery of lustre-MDT0000_UUID on 192.168.204.103@tcp failed (-19) 00000100:00100000:0.0:1776178528.104667:0:3352:0:(client.c:2215:ptlrpc_check_set()) Completed RPC req@0000000098682d9e pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:lustre-MDT0000-lwp-OST0000_UUID:3352:1862456998093952:0@lo:38:kworker/u8:2.0 00000100:00080000:0.0:1776178528.104677:0:3352:0:(import.c:86:import_set_state_nolock()) 00000000dfc8db13 lustre-MDT0000_UUID: changing import state from CONNECTING to DISCONN 00000100:00080000:0.0:1776178528.104681:0:3352:0:(import.c:1429:ptlrpc_connect_interpret()) recovery of lustre-MDT0000_UUID on 192.168.204.103@tcp failed (-19) 00000100:00100000:0.0:1776178528.104683:0:3352:0:(client.c:2215:ptlrpc_check_set()) Completed RPC req@00000000ef4da5b8 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:lustre-MDT0000-lwp-OST0001_UUID:3352:1862456998094080:0@lo:38:kworker/u8:2.0 00000100:00100000:3.0:1776178528.107712:0:6255:0:(service.c:2152:ptlrpc_server_handle_req_in()) got req x1862456998094400 00000100:00100000:3.0:1776178528.107807:0:6255:0:(nrs_fifo.c:179:nrs_fifo_req_get()) NRS start fifo request from 12345-0@lo, seq: 6879 00000100:00100000:3.0:1776178528.107815:0:6255:0:(service.c:2324:ptlrpc_server_handle_request()) Handling RPC req@00000000d1a7070f pname:cluuid+ref:pid:xid:nid:opc:job mdt00_000:0+-99:3352:x1862456998094400:12345-0@lo:38:kworker/u8:2.0 00010000:02000000:3.0:1776178528.107851:0:6255:0:(ldlm_lib.c:1139:target_handle_connect()) lustre-MDT0000: Not available for connect from 0@lo (stopping) 00000100:00100000:3.0:1776178528.109468:0:6255:0:(service.c:2372:ptlrpc_server_handle_request()) Handled RPC req@00000000d1a7070f pname:cluuid+ref:pid:xid:nid:opc:job mdt00_000:0+-99:3352:x1862456998094400:12345-0@lo:38:kworker/u8:2.0 Request processed in 1643us (5056us total) trans 0 rc -19/-19 00000100:00100000:3.0:1776178528.109489:0:6255:0:(nrs_fifo.c:241:nrs_fifo_req_stop()) NRS stop fifo request from 12345-0@lo, seq: 6879 00000100:00080000:0.0:1776178528.109649:0:3352:0:(import.c:86:import_set_state_nolock()) 00000000a7c55b1f lustre-MDT0000_UUID: changing import state from CONNECTING to DISCONN 00000100:00080000:0.0:1776178528.109656:0:3352:0:(import.c:1429:ptlrpc_connect_interpret()) recovery of lustre-MDT0000_UUID on 192.168.204.103@tcp failed (-19) 00000100:00100000:0.0:1776178528.109662:0:3352:0:(client.c:2215:ptlrpc_check_set()) Completed RPC req@0000000065c78381 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:lustre-MDT0000-lwp-MDT0001_UUID:3352:1862456998094400:0@lo:38:kworker/u8:2.0 00010000:00010000:3.0:1776178528.736998:0:6244:0:(ldlm_lockd.c:2792:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:2.0:1776178529.633180:0:3355:0:(client.c:1733:ptlrpc_send_new_req()) Sending RPC req@00000000a9c256f6 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_02:lustre-MDT0001-mdtlov_UUID:3355:1862456998094464:0@lo:13:osp-pre-1-1.0 00000100:00100000:2.0:1776178529.633285:0:3355:0:(events.c:361:request_in_callback()) peer: 12345-0@lo (source: 12345-0@lo) 00000100:00100000:3.0:1776178529.633608:0:8231:0:(service.c:2152:ptlrpc_server_handle_req_in()) got req x1862456998094464 00000100:00100000:3.0:1776178529.633817:0:8231:0:(nrs_fifo.c:179:nrs_fifo_req_get()) NRS start fifo request from 12345-0@lo, seq: 615 00000100:00100000:3.0:1776178529.633834:0:8231:0:(service.c:2324:ptlrpc_server_handle_request()) Handling RPC req@0000000021958fc7 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_create00:lustre-MDT0001-mdtlov_UUID+5:3355:x1862456998094464:12345-0@lo:13:osp-pre-1-1.0 00002000:00000024:3.0:1776178529.633881:0:8231:0:(ofd_obd.c:740:ofd_statfs()) blocks cached 229376 granted 491746368 pending 0 free 3923689472 avail 3692167168 00000100:00100000:3.0:1776178529.633945:0:8231:0:(service.c:2372:ptlrpc_server_handle_request()) Handled RPC req@0000000021958fc7 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_create00:lustre-MDT0001-mdtlov_UUID+5:3355:x1862456998094464:12345-0@lo:13:osp-pre-1-1.0 Request processed in 111us (666us total) trans 0 rc 0/0 00000100:00100000:3.0:1776178529.633952:0:8231:0:(nrs_fifo.c:241:nrs_fifo_req_stop()) NRS stop fifo request from 12345-0@lo, seq: 615 00000100:00100000:2.0:1776178529.634055:0:3355:0:(client.c:2215:ptlrpc_check_set()) Completed RPC req@00000000a9c256f6 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_02:lustre-MDT0001-mdtlov_UUID:3355:1862456998094464:0@lo:13:osp-pre-1-1.0 00010000:00010000:2.0:1776178529.764784:0:6245:0:(ldlm_lockd.c:2792:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:2.0:1776178530.144549:0:3355:0:(ptlrpcd.c:415:ptlrpcd_check()) transfer 1 async RPCs [3->2] 00000100:00100000:2.0:1776178530.144584:0:3355:0:(client.c:1733:ptlrpc_send_new_req()) Sending RPC req@00000000eec4b6d0 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_02:lustre-MDT0001-mdtlov_UUID:3355:1862456998094528:0@lo:13:osp-pre-0-1.0 00000100:00100000:2.0:1776178530.144682:0:3355:0:(events.c:361:request_in_callback()) peer: 12345-0@lo (source: 12345-0@lo) 00000100:00100000:3.0:1776178530.144782:0:8231:0:(service.c:2152:ptlrpc_server_handle_req_in()) got req x1862456998094528 00000100:00100000:3.0:1776178530.144852:0:8231:0:(nrs_fifo.c:179:nrs_fifo_req_get()) NRS start fifo request from 12345-0@lo, seq: 616 00000100:00100000:3.0:1776178530.144861:0:8231:0:(service.c:2324:ptlrpc_server_handle_request()) Handling RPC req@000000009e484c71 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_create00:lustre-MDT0001-mdtlov_UUID+5:3355:x1862456998094528:12345-0@lo:13:osp-pre-0-1.0 00002000:00000024:3.0:1776178530.144905:0:8231:0:(ofd_obd.c:740:ofd_statfs()) blocks cached 315392 granted 484883776 pending 0 free 3923689472 avail 3692167168 00000100:00100000:3.0:1776178530.144968:0:8231:0:(service.c:2372:ptlrpc_server_handle_request()) Handled RPC req@000000009e484c71 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_create00:lustre-MDT0001-mdtlov_UUID+5:3355:x1862456998094528:12345-0@lo:13:osp-pre-0-1.0 Request processed in 107us (288us total) trans 0 rc 0/0 00000100:00100000:3.0:1776178530.144977:0:8231:0:(nrs_fifo.c:241:nrs_fifo_req_stop()) NRS stop fifo request from 12345-0@lo, seq: 616 00000100:00100000:2.0:1776178530.145129:0:3355:0:(client.c:2215:ptlrpc_check_set()) Completed RPC req@00000000eec4b6d0 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_02:lustre-MDT0001-mdtlov_UUID:3355:1862456998094528:0@lo:13:osp-pre-0-1.0 00010000:00010000:1.0:1776178530.795448:0:9224:0:(ldlm_lockd.c:2792:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:0.0:1776178531.618704:0:3346:0:(events.c:361:request_in_callback()) peer: 12345-192.168.204.3@tcp (source: 12345-192.168.204.3@tcp) 00000100:00100000:1.0:1776178531.619159:0:20140:0:(service.c:2152:ptlrpc_server_handle_req_in()) got req x1862456979129472 00000100:00100000:1.0:1776178531.619226:0:20140:0:(nrs_fifo.c:179:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.204.3@tcp, seq: 283 00000100:00100000:1.0:1776178531.619233:0:20140:0:(service.c:2324:ptlrpc_server_handle_request()) Handling RPC req@00000000ae8faa77 pname:cluuid+ref:pid:xid:nid:opc:job ll_mgs_0002:5b5c66b1-1a2f-4651-8548-0c75b9a38a94+9:2272:x1862456979129472:12345-192.168.204.3@tcp:400:kworker/u8:4.0 00000100:00100000:1.0:1776178531.619335:0:20140:0:(service.c:2372:ptlrpc_server_handle_request()) Handled RPC req@00000000ae8faa77 pname:cluuid+ref:pid:xid:nid:opc:job ll_mgs_0002:5b5c66b1-1a2f-4651-8548-0c75b9a38a94+9:2272:x1862456979129472:12345-192.168.204.3@tcp:400:kworker/u8:4.0 Request processed in 102us (641us total) trans 0 rc 0/0 00000100:00100000:1.0:1776178531.619343:0:20140:0:(nrs_fifo.c:241:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.204.3@tcp, seq: 283 00000100:00100000:2.0:1776178531.624046:0:3348:0:(events.c:361:request_in_callback()) peer: 12345-192.168.204.3@tcp (source: 12345-192.168.204.3@tcp) 00000100:00100000:2.0:1776178531.624145:0:3348:0:(events.c:361:request_in_callback()) peer: 12345-192.168.204.3@tcp (source: 12345-192.168.204.3@tcp) 00000100:00100000:2.0:1776178531.624183:0:3348:0:(events.c:361:request_in_callback()) peer: 12345-192.168.204.3@tcp (source: 12345-192.168.204.3@tcp) 00000100:00100000:0.0:1776178531.624293:0:6257:0:(service.c:2152:ptlrpc_server_handle_req_in()) got req x1862456979129600 00000100:00100000:0.0:1776178531.624360:0:6257:0:(nrs_fifo.c:179:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.204.3@tcp, seq: 1120 00000100:00100000:0.0:1776178531.624366:0:6257:0:(service.c:2324:ptlrpc_server_handle_request()) Handling RPC req@0000000012d88c00 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_002:b50963bd-8eba-4432-9a54-8888d00e3fa1+6:2273:x1862456979129600:12345-192.168.204.3@tcp:400:kworker/u8:4.0 00000100:00100000:0.0:1776178531.624445:0:6257:0:(service.c:2372:ptlrpc_server_handle_request()) Handled RPC req@0000000012d88c00 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_002:b50963bd-8eba-4432-9a54-8888d00e3fa1+6:2273:x1862456979129600:12345-192.168.204.3@tcp:400:kworker/u8:4.0 Request processed in 78us (411us total) trans 0 rc 0/0 00000100:00100000:0.0:1776178531.624452:0:6257:0:(nrs_fifo.c:241:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.204.3@tcp, seq: 1120 00000100:00100000:1.0:1776178531.624659:0:20755:0:(service.c:2152:ptlrpc_server_handle_req_in()) got req x1862456979129728 00000100:00100000:1.0:1776178531.624689:0:20755:0:(service.c:2152:ptlrpc_server_handle_req_in()) got req x1862456979129664 00000100:00100000:1.0:1776178531.624708:0:20755:0:(nrs_fifo.c:179:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.204.3@tcp, seq: 767 00000100:00100000:1.0:1776178531.624712:0:20755:0:(service.c:2324:ptlrpc_server_handle_request()) Handling RPC req@0000000082764007 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_006:b50963bd-8eba-4432-9a54-8888d00e3fa1+9:2273:x1862456979129728:12345-192.168.204.3@tcp:400:kworker/u8:4.0 00000100:00100000:1.0:1776178531.624748:0:20755:0:(service.c:2372:ptlrpc_server_handle_request()) Handled RPC req@0000000082764007 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_006:b50963bd-8eba-4432-9a54-8888d00e3fa1+9:2273:x1862456979129728:12345-192.168.204.3@tcp:400:kworker/u8:4.0 Request processed in 35us (604us total) trans 0 rc 0/0 00000100:00100000:1.0:1776178531.624755:0:20755:0:(nrs_fifo.c:241:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.204.3@tcp, seq: 767 00000100:00100000:1.0:1776178531.624768:0:20755:0:(nrs_fifo.c:179:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.204.3@tcp, seq: 768 00000100:00100000:1.0:1776178531.624772:0:20755:0:(service.c:2324:ptlrpc_server_handle_request()) Handling RPC req@0000000070829000 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_006:b50963bd-8eba-4432-9a54-8888d00e3fa1+8:2273:x1862456979129664:12345-192.168.204.3@tcp:400:kworker/u8:4.0 00000100:00100000:1.0:1776178531.624796:0:20755:0:(service.c:2372:ptlrpc_server_handle_request()) Handled RPC req@0000000070829000 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_006:b50963bd-8eba-4432-9a54-8888d00e3fa1+8:2273:x1862456979129664:12345-192.168.204.3@tcp:400:kworker/u8:4.0 Request processed in 23us (614us total) trans 0 rc 0/0 00000100:00100000:1.0:1776178531.624802:0:20755:0:(nrs_fifo.c:241:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.204.3@tcp, seq: 768 00000100:00100000:3.0:1776178531.625946:0:3347:0:(events.c:361:request_in_callback()) peer: 12345-192.168.204.3@tcp (source: 12345-192.168.204.3@tcp) 00000100:00100000:1.0:1776178531.626578:0:6255:0:(service.c:2152:ptlrpc_server_handle_req_in()) got req x1862456979129536 00000100:00100000:1.0:1776178531.626630:0:6255:0:(nrs_fifo.c:179:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.204.3@tcp, seq: 6880 00000100:00100000:1.0:1776178531.626637:0:6255:0:(service.c:2324:ptlrpc_server_handle_request()) Handling RPC req@00000000d4b43163 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_000:0+-99:2269:x1862456979129536:12345-192.168.204.3@tcp:38:kworker/u8:4.0 00010000:02000000:1.0:1776178531.626658:0:6255:0:(ldlm_lib.c:1139:target_handle_connect()) lustre-MDT0000: Not available for connect from 192.168.204.3@tcp (stopping) 00000100:00100000:1.0:1776178531.626757:0:6255:0:(service.c:2372:ptlrpc_server_handle_request()) Handled RPC req@00000000d4b43163 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_000:0+-99:2269:x1862456979129536:12345-192.168.204.3@tcp:38:kworker/u8:4.0 Request processed in 120us (818us total) trans 0 rc -19/-19 00000100:00100000:1.0:1776178531.627908:0:6255:0:(nrs_fifo.c:241:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.204.3@tcp, seq: 6880 00010000:00010000:1.0:1776178531.808595:0:9225:0:(ldlm_lockd.c:2792:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00010000:00010000:3.0:1776178532.833278:0:6244:0:(ldlm_lockd.c:2792:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00080000:0.0:1776178533.216536:0:36:0:(pinger.c:251:ptlrpc_pinger_process_import()) lustre-MDT0000-lwp-OST0000_UUID->lustre-MDT0000_UUID: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00080000:0.0:1776178533.216547:0:36:0:(recover.c:57:ptlrpc_initiate_recovery()) lustre-MDT0000_UUID: starting recovery 00000100:00080000:0.0:1776178533.216550:0:36:0:(import.c:86:import_set_state_nolock()) 0000000047601a98 lustre-MDT0000_UUID: changing import state from DISCONN to CONNECTING 00000100:00080000:0.0:1776178533.216560:0:36:0:(import.c:534:import_select_connection()) lustre-MDT0000-lwp-OST0000: connect to NID 0@lo last attempt 1235 00000100:00080000:0.0:1776178533.216563:0:36:0:(import.c:581:import_select_connection()) lustre-MDT0000-lwp-OST0000: tried all connections, increasing latency to 21s 00000100:00080000:0.0:1776178533.216571:0:36:0:(import.c:615:import_select_connection()) lustre-MDT0000-lwp-OST0000: import 0000000047601a98 using connection 192.168.204.103@tcp/0@lo 00000100:00100000:0.0:1776178533.216597:0:36:0:(import.c:807:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5) req@0000000020763766 x1862456998094656/t0(0) o38->lustre-MDT0000-lwp-OST0000@0@lo:12/10 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' 00000100:00100000:2.0:1776178533.216665:0:3352:0:(client.c:1733:ptlrpc_send_new_req()) Sending RPC req@0000000020763766 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:lustre-MDT0000-lwp-OST0000_UUID:3352:1862456998094656:0@lo:38:kworker/u8:1.0 00000100:00100000:2.0:1776178533.216749:0:3352:0:(events.c:361:request_in_callback()) peer: 12345-0@lo (source: 12345-0@lo) 00000100:00100000:1.0:1776178533.216830:0:3354:0:(ptlrpcd.c:415:ptlrpcd_check()) transfer 1 async RPCs [0->1] 00000100:00100000:1.0:1776178533.216858:0:3354:0:(client.c:1733:ptlrpc_send_new_req()) Sending RPC req@0000000014e690d4 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_01:ac61f839-0e8e-4cac-9c59-08bcfbbad330:3354:1862456998094592:0@lo:400:kworker/u8:1.0 00000100:00100000:1.0:1776178533.216900:0:3354:0:(events.c:361:request_in_callback()) peer: 12345-0@lo (source: 12345-0@lo) 00000100:00100000:1.0:1776178533.216948:0:6255:0:(service.c:2152:ptlrpc_server_handle_req_in()) got req x1862456998094656 00000100:00100000:1.0:1776178533.216989:0:6255:0:(nrs_fifo.c:179:nrs_fifo_req_get()) NRS start fifo request from 12345-0@lo, seq: 6881 00000100:00100000:1.0:1776178533.216995:0:6255:0:(service.c:2324:ptlrpc_server_handle_request()) Handling RPC req@0000000080e59d72 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_000:0+-99:3352:x1862456998094656:12345-0@lo:38:kworker/u8:1.0 00010000:02000000:1.0:1776178533.217011:0:6255:0:(ldlm_lib.c:1139:target_handle_connect()) lustre-MDT0000: Not available for connect from 0@lo (stopping) 00000100:00100000:3.0:1776178533.217123:0:20137:0:(service.c:2152:ptlrpc_server_handle_req_in()) got req x1862456998094592 00000100:00100000:3.0:1776178533.217155:0:20137:0:(nrs_fifo.c:179:nrs_fifo_req_get()) NRS start fifo request from 12345-0@lo, seq: 284 00000100:00100000:3.0:1776178533.217158:0:20137:0:(service.c:2324:ptlrpc_server_handle_request()) Handling RPC req@000000002efeec88 pname:cluuid+ref:pid:xid:nid:opc:job ll_mgs_0000:ac61f839-0e8e-4cac-9c59-08bcfbbad330+18:3354:x1862456998094592:12345-0@lo:400:kworker/u8:1.0 00000100:00100000:3.0:1776178533.217201:0:20137:0:(service.c:2372:ptlrpc_server_handle_request()) Handled RPC req@000000002efeec88 pname:cluuid+ref:pid:xid:nid:opc:job ll_mgs_0000:ac61f839-0e8e-4cac-9c59-08bcfbbad330+18:3354:x1862456998094592:12345-0@lo:400:kworker/u8:1.0 Request processed in 41us (301us total) trans 0 rc 0/0 00000100:00100000:3.0:1776178533.217209:0:20137:0:(nrs_fifo.c:241:nrs_fifo_req_stop()) NRS stop fifo request from 12345-0@lo, seq: 284 00000100:00080000:0.0:1776178533.219595:0:36:0:(pinger.c:251:ptlrpc_pinger_process_import()) lustre-MDT0000-lwp-OST0001_UUID->lustre-MDT0000_UUID: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00080000:0.0:1776178533.219602:0:36:0:(recover.c:57:ptlrpc_initiate_recovery()) lustre-MDT0000_UUID: starting recovery 00000100:00080000:0.0:1776178533.219607:0:36:0:(import.c:86:import_set_state_nolock()) 00000000dfc8db13 lustre-MDT0000_UUID: changing import state from DISCONN to CONNECTING 00000100:00080000:0.0:1776178533.219616:0:36:0:(import.c:534:import_select_connection()) lustre-MDT0000-lwp-OST0001: connect to NID 0@lo last attempt 1235 00000100:00080000:0.0:1776178533.219621:0:36:0:(import.c:581:import_select_connection()) lustre-MDT0000-lwp-OST0001: tried all connections, increasing latency to 21s 00000100:00080000:0.0:1776178533.219628:0:36:0:(import.c:615:import_select_connection()) lustre-MDT0000-lwp-OST0001: import 00000000dfc8db13 using connection 192.168.204.103@tcp/0@lo 00000100:00100000:0.0:1776178533.219653:0:36:0:(import.c:807:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5) req@000000009b04a931 x1862456998094784/t0(0) o38->lustre-MDT0000-lwp-OST0001@0@lo:12/10 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' 00000100:00100000:3.0:1776178533.219905:0:3353:0:(ptlrpcd.c:415:ptlrpcd_check()) transfer 1 async RPCs [1->0] 00000100:00100000:3.0:1776178533.219926:0:3353:0:(client.c:1733:ptlrpc_send_new_req()) Sending RPC req@0000000001246e58 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_00:lustre-MDT0001-lwp-OST0000_UUID:3353:1862456998094720:0@lo:400:kworker/u8:1.0 00000100:00100000:3.0:1776178533.219989:0:3353:0:(events.c:361:request_in_callback()) peer: 12345-0@lo (source: 12345-0@lo) 00000100:00100000:3.0:1776178533.220047:0:6257:0:(service.c:2152:ptlrpc_server_handle_req_in()) got req x1862456998094720 00000100:00100000:3.0:1776178533.220116:0:6257:0:(nrs_fifo.c:179:nrs_fifo_req_get()) NRS start fifo request from 12345-0@lo, seq: 1121 00000100:00100000:3.0:1776178533.220121:0:6257:0:(service.c:2324:ptlrpc_server_handle_request()) Handling RPC req@00000000a55cac67 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_002:lustre-MDT0001-lwp-OST0000_UUID+5:3353:x1862456998094720:12345-0@lo:400:kworker/u8:1.0 00000100:00100000:3.0:1776178533.220174:0:6257:0:(service.c:2372:ptlrpc_server_handle_request()) Handled RPC req@00000000a55cac67 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_002:lustre-MDT0001-lwp-OST0000_UUID+5:3353:x1862456998094720:12345-0@lo:400:kworker/u8:1.0 Request processed in 53us (186us total) trans 0 rc 0/0 00000100:00100000:3.0:1776178533.220182:0:6257:0:(nrs_fifo.c:241:nrs_fifo_req_stop()) NRS stop fifo request from 12345-0@lo, seq: 1121 00000100:00100000:3.0:1776178533.220239:0:3353:0:(client.c:2215:ptlrpc_check_set()) Completed RPC req@0000000001246e58 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_00:lustre-MDT0001-lwp-OST0000_UUID:3353:1862456998094720:0@lo:400:kworker/u8:1.0 00000100:00100000:3.0:1776178533.220296:0:3354:0:(client.c:2215:ptlrpc_check_set()) Completed RPC req@0000000014e690d4 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_01:ac61f839-0e8e-4cac-9c59-08bcfbbad330:3354:1862456998094592:0@lo:400:kworker/u8:1.0 00000100:00100000:2.0:1776178533.220715:0:3352:0:(client.c:1733:ptlrpc_send_new_req()) Sending RPC req@000000009b04a931 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:lustre-MDT0000-lwp-OST0001_UUID:3352:1862456998094784:0@lo:38:kworker/u8:1.0 00000100:00100000:2.0:1776178533.220753:0:3352:0:(events.c:361:request_in_callback()) peer: 12345-0@lo (source: 12345-0@lo) 00000100:00100000:1.0:1776178533.221347:0:6255:0:(service.c:2372:ptlrpc_server_handle_request()) Handled RPC req@0000000080e59d72 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_000:0+-99:3352:x1862456998094656:12345-0@lo:38:kworker/u8:1.0 Request processed in 4349us (4597us total) trans 0 rc -19/-19 00000100:00100000:1.0:1776178533.221357:0:6255:0:(nrs_fifo.c:241:nrs_fifo_req_stop()) NRS stop fifo request from 12345-0@lo, seq: 6881 00000100:00100000:1.0:1776178533.221381:0:6255:0:(service.c:2152:ptlrpc_server_handle_req_in()) got req x1862456998094784 00000100:00080000:0.0:1776178533.221488:0:36:0:(pinger.c:251:ptlrpc_pinger_process_import()) lustre-MDT0001-mdtlov_UUID->lustre-MDT0000_UUID: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00080000:0.0:1776178533.221493:0:36:0:(recover.c:57:ptlrpc_initiate_recovery()) lustre-MDT0000_UUID: starting recovery 00000100:00080000:0.0:1776178533.221497:0:36:0:(import.c:86:import_set_state_nolock()) 00000000ba4f1821 lustre-MDT0000_UUID: changing import state from DISCONN to CONNECTING 00000100:00080000:0.0:1776178533.221504:0:36:0:(import.c:534:import_select_connection()) lustre-MDT0000-osp-MDT0001: connect to NID 0@lo last attempt 1235 00000100:00080000:0.0:1776178533.221508:0:36:0:(import.c:581:import_select_connection()) lustre-MDT0000-osp-MDT0001: tried all connections, increasing latency to 21s 00000100:00080000:0.0:1776178533.221513:0:36:0:(import.c:615:import_select_connection()) lustre-MDT0000-osp-MDT0001: import 00000000ba4f1821 using connection 192.168.204.103@tcp/0@lo 00000100:00100000:0.0:1776178533.221534:0:36:0:(import.c:807:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5) req@00000000949e3fea x1862456998094912/t0(0) o38->lustre-MDT0000-osp-MDT0001@0@lo:24/4 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' 00000100:00080000:0.0:1776178533.221567:0:36:0:(pinger.c:251:ptlrpc_pinger_process_import()) lustre-MDT0000-lwp-MDT0001_UUID->lustre-MDT0000_UUID: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00080000:0.0:1776178533.221570:0:36:0:(recover.c:57:ptlrpc_initiate_recovery()) lustre-MDT0000_UUID: starting recovery 00000100:00080000:0.0:1776178533.221572:0:36:0:(import.c:86:import_set_state_nolock()) 00000000a7c55b1f lustre-MDT0000_UUID: changing import state from DISCONN to CONNECTING 00000100:00080000:0.0:1776178533.221575:0:36:0:(import.c:534:import_select_connection()) lustre-MDT0000-lwp-MDT0001: connect to NID 0@lo last attempt 1235 00000100:00080000:0.0:1776178533.221577:0:36:0:(import.c:581:import_select_connection()) lustre-MDT0000-lwp-MDT0001: tried all connections, increasing latency to 21s 00000100:00080000:0.0:1776178533.221580:0:36:0:(import.c:615:import_select_connection()) lustre-MDT0000-lwp-MDT0001: import 00000000a7c55b1f using connection 192.168.204.103@tcp/0@lo 00000100:00100000:0.0:1776178533.221590:0:36:0:(import.c:807:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5) req@00000000a979415d x1862456998095104/t0(0) o38->lustre-MDT0000-lwp-MDT0001@0@lo:12/10 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' 00000100:00100000:0.0:1776178533.221617:0:3356:0:(client.c:1733:ptlrpc_send_new_req()) Sending RPC req@0000000002da67e2 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_03:lustre-MDT0001-mdtlov_UUID:3356:1862456998094976:0@lo:400:kworker/u8:1.0 00000100:00100000:0.0:1776178533.221655:0:3356:0:(events.c:361:request_in_callback()) peer: 12345-0@lo (source: 12345-0@lo) 00000100:00100000:0.0:1776178533.221677:0:3356:0:(ptlrpcd.c:415:ptlrpcd_check()) transfer 1 async RPCs [2->3] 00000100:00100000:0.0:1776178533.221683:0:3356:0:(client.c:1733:ptlrpc_send_new_req()) Sending RPC req@00000000c7e83057 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_03:lustre-MDT0001-lwp-OST0001_UUID:3356:1862456998094848:0@lo:400:kworker/u8:1.0 00000100:00100000:0.0:1776178533.221706:0:3356:0:(events.c:361:request_in_callback()) peer: 12345-0@lo (source: 12345-0@lo) 00000100:00080000:2.0:1776178533.226093:0:3352:0:(import.c:86:import_set_state_nolock()) 0000000047601a98 lustre-MDT0000_UUID: changing import state from CONNECTING to DISCONN 00000100:00080000:2.0:1776178533.226105:0:3352:0:(import.c:1429:ptlrpc_connect_interpret()) recovery of lustre-MDT0000_UUID on 192.168.204.103@tcp failed (-19) 00000100:00100000:2.0:1776178533.226114:0:3352:0:(client.c:2215:ptlrpc_check_set()) Completed RPC req@0000000020763766 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:lustre-MDT0000-lwp-OST0000_UUID:3352:1862456998094656:0@lo:38:kworker/u8:1.0 00000100:00100000:2.0:1776178533.226152:0:3352:0:(client.c:1733:ptlrpc_send_new_req()) Sending RPC req@00000000949e3fea pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:lustre-MDT0001-mdtlov_UUID:3352:1862456998094912:0@lo:38:kworker/u8:1.0 00000100:00100000:2.0:1776178533.226229:0:3352:0:(events.c:361:request_in_callback()) peer: 12345-0@lo (source: 12345-0@lo) 00000100:00100000:2.0:1776178533.226270:0:3352:0:(client.c:1733:ptlrpc_send_new_req()) Sending RPC req@00000000a979415d pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:lustre-MDT0000-lwp-MDT0001_UUID:3352:1862456998095104:0@lo:38:kworker/u8:1.0 00000100:00100000:2.0:1776178533.226321:0:14440:0:(service.c:2152:ptlrpc_server_handle_req_in()) got req x1862456998094976 00000100:00100000:2.0:1776178533.226374:0:14440:0:(nrs_fifo.c:179:nrs_fifo_req_get()) NRS start fifo request from 12345-0@lo, seq: 769 00000100:00100000:2.0:1776178533.226382:0:14440:0:(service.c:2324:ptlrpc_server_handle_request()) Handling RPC req@00000000e1865444 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_004:lustre-MDT0001-mdtlov_UUID+5:3356:x1862456998094976:12345-0@lo:400:kworker/u8:1.0 00000100:00100000:2.0:1776178533.226456:0:14440:0:(service.c:2372:ptlrpc_server_handle_request()) Handled RPC req@00000000e1865444 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_004:lustre-MDT0001-mdtlov_UUID+5:3356:x1862456998094976:12345-0@lo:400:kworker/u8:1.0 Request processed in 73us (4801us total) trans 0 rc 0/0 00000100:00100000:2.0:1776178533.226463:0:14440:0:(nrs_fifo.c:241:nrs_fifo_req_stop()) NRS stop fifo request from 12345-0@lo, seq: 769 00000100:00100000:2.0:1776178533.226494:0:15213:0:(service.c:2152:ptlrpc_server_handle_req_in()) got req x1862456998094912 00000100:00100000:2.0:1776178533.226519:0:15213:0:(nrs_fifo.c:179:nrs_fifo_req_get()) NRS start fifo request from 12345-0@lo, seq: 772 00000100:00100000:2.0:1776178533.226523:0:15213:0:(service.c:2324:ptlrpc_server_handle_request()) Handling RPC req@0000000065c78381 pname:cluuid+ref:pid:xid:nid:opc:job mdt_out00_003:0+-99:3352:x1862456998094912:12345-0@lo:38:kworker/u8:1.0 00010000:02000000:2.0:1776178533.226539:0:15213:0:(ldlm_lib.c:1139:target_handle_connect()) lustre-MDT0000: Not available for connect from 0@lo (stopping) 00000100:00100000:2.0:1776178533.226578:0:15213:0:(service.c:2372:ptlrpc_server_handle_request()) Handled RPC req@0000000065c78381 pname:cluuid+ref:pid:xid:nid:opc:job mdt_out00_003:0+-99:3352:x1862456998094912:12345-0@lo:38:kworker/u8:1.0 Request processed in 54us (350us total) trans 0 rc -19/-19 00000100:00100000:2.0:1776178533.226584:0:15213:0:(nrs_fifo.c:241:nrs_fifo_req_stop()) NRS stop fifo request from 12345-0@lo, seq: 772 00000100:00100000:2.0:1776178533.226637:0:3354:0:(ptlrpcd.c:415:ptlrpcd_check()) transfer 1 async RPCs [0->1] 00000100:00100000:2.0:1776178533.226649:0:3354:0:(client.c:1733:ptlrpc_send_new_req()) Sending RPC req@00000000702f2be6 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_01:lustre-MDT0001-mdtlov_UUID:3354:1862456998095040:0@lo:400:kworker/u8:1.0 00000100:00100000:2.0:1776178533.226679:0:3354:0:(events.c:361:request_in_callback()) peer: 12345-0@lo (source: 12345-0@lo) 00000100:00100000:2.0:1776178533.226711:0:20759:0:(service.c:2152:ptlrpc_server_handle_req_in()) got req x1862456998095040 00000100:00100000:2.0:1776178533.226739:0:20759:0:(nrs_fifo.c:179:nrs_fifo_req_get()) NRS start fifo request from 12345-0@lo, seq: 770 00000100:00100000:2.0:1776178533.226743:0:20759:0:(service.c:2324:ptlrpc_server_handle_request()) Handling RPC req@00000000ef4da5b8 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_010:lustre-MDT0001-mdtlov_UUID+5:3354:x1862456998095040:12345-0@lo:400:kworker/u8:1.0 00000100:00100000:2.0:1776178533.226777:0:20759:0:(service.c:2372:ptlrpc_server_handle_request()) Handled RPC req@00000000ef4da5b8 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_010:lustre-MDT0001-mdtlov_UUID+5:3354:x1862456998095040:12345-0@lo:400:kworker/u8:1.0 Request processed in 33us (98us total) trans 0 rc 0/0 00000100:00100000:2.0:1776178533.226783:0:20759:0:(nrs_fifo.c:241:nrs_fifo_req_stop()) NRS stop fifo request from 12345-0@lo, seq: 770 00000100:00100000:2.0:1776178533.226812:0:3354:0:(client.c:2215:ptlrpc_check_set()) Completed RPC req@00000000702f2be6 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_01:lustre-MDT0001-mdtlov_UUID:3354:1862456998095040:0@lo:400:kworker/u8:1.0 00000100:00100000:0.0:1776178533.228673:0:3356:0:(client.c:2215:ptlrpc_check_set()) Completed RPC req@0000000002da67e2 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_03:lustre-MDT0001-mdtlov_UUID:3356:1862456998094976:0@lo:400:kworker/u8:1.0 00000100:00100000:0.0:1776178533.228723:0:3352:0:(events.c:361:request_in_callback()) peer: 12345-0@lo (source: 12345-0@lo) 00000100:00100000:3.0:1776178533.228798:0:6256:0:(service.c:2152:ptlrpc_server_handle_req_in()) got req x1862456998094848 00000100:00100000:3.0:1776178533.228851:0:6256:0:(nrs_fifo.c:179:nrs_fifo_req_get()) NRS start fifo request from 12345-0@lo, seq: 1122 00000100:00100000:3.0:1776178533.228856:0:6256:0:(service.c:2324:ptlrpc_server_handle_request()) Handling RPC req@00000000f871d6a5 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_001:lustre-MDT0001-lwp-OST0001_UUID+5:3356:x1862456998094848:12345-0@lo:400:kworker/u8:1.0 00000100:00100000:3.0:1776178533.228932:0:6256:0:(service.c:2372:ptlrpc_server_handle_request()) Handled RPC req@00000000f871d6a5 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_001:lustre-MDT0001-lwp-OST0001_UUID+5:3356:x1862456998094848:12345-0@lo:400:kworker/u8:1.0 Request processed in 75us (7225us total) trans 0 rc 0/0 00000100:00100000:3.0:1776178533.228940:0:6256:0:(nrs_fifo.c:241:nrs_fifo_req_stop()) NRS stop fifo request from 12345-0@lo, seq: 1122 00000100:00100000:3.0:1776178533.228956:0:6256:0:(nrs_fifo.c:179:nrs_fifo_req_get()) NRS start fifo request from 12345-0@lo, seq: 6882 00000100:00100000:3.0:1776178533.228960:0:6256:0:(service.c:2324:ptlrpc_server_handle_request()) Handling RPC req@0000000048564d2a pname:cluuid+ref:pid:xid:nid:opc:job mdt00_001:0+-99:3352:x1862456998094784:12345-0@lo:38:kworker/u8:1.0 00010000:02000000:3.0:1776178533.228976:0:6256:0:(ldlm_lib.c:1139:target_handle_connect()) lustre-MDT0000: Not available for connect from 0@lo (stopping) 00000100:00100000:3.0:1776178533.229004:0:6256:0:(service.c:2372:ptlrpc_server_handle_request()) Handled RPC req@0000000048564d2a pname:cluuid+ref:pid:xid:nid:opc:job mdt00_001:0+-99:3352:x1862456998094784:12345-0@lo:38:kworker/u8:1.0 Request processed in 44us (8252us total) trans 0 rc -19/-19 00000100:00100000:3.0:1776178533.229009:0:6256:0:(nrs_fifo.c:241:nrs_fifo_req_stop()) NRS stop fifo request from 12345-0@lo, seq: 6882 00000100:00100000:1.0:1776178533.241801:0:3356:0:(client.c:2215:ptlrpc_check_set()) Completed RPC req@00000000c7e83057 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_03:lustre-MDT0001-lwp-OST0001_UUID:3356:1862456998094848:0@lo:400:kworker/u8:1.0 00000100:00080000:0.0:1776178533.242260:0:3352:0:(import.c:86:import_set_state_nolock()) 00000000dfc8db13 lustre-MDT0000_UUID: changing import state from CONNECTING to DISCONN 00000100:00080000:0.0:1776178533.242266:0:3352:0:(import.c:1429:ptlrpc_connect_interpret()) recovery of lustre-MDT0000_UUID on 192.168.204.103@tcp failed (-19) 00000100:00100000:0.0:1776178533.242269:0:3352:0:(client.c:2215:ptlrpc_check_set()) Completed RPC req@000000009b04a931 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:lustre-MDT0000-lwp-OST0001_UUID:3352:1862456998094784:0@lo:38:kworker/u8:1.0 00000100:00080000:0.0:1776178533.242289:0:3352:0:(import.c:86:import_set_state_nolock()) 00000000ba4f1821 lustre-MDT0000_UUID: changing import state from CONNECTING to DISCONN 00000100:00080000:0.0:1776178533.242292:0:3352:0:(import.c:1429:ptlrpc_connect_interpret()) recovery of lustre-MDT0000_UUID on 192.168.204.103@tcp failed (-19) 00000100:00100000:0.0:1776178533.242295:0:3352:0:(client.c:2215:ptlrpc_check_set()) Completed RPC req@00000000949e3fea pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:lustre-MDT0001-mdtlov_UUID:3352:1862456998094912:0@lo:38:kworker/u8:1.0 00000100:00100000:3.0:1776178533.242651:0:6255:0:(service.c:2152:ptlrpc_server_handle_req_in()) got req x1862456998095104 00000100:00100000:3.0:1776178533.242700:0:6255:0:(nrs_fifo.c:179:nrs_fifo_req_get()) NRS start fifo request from 12345-0@lo, seq: 6883 00000100:00100000:3.0:1776178533.242706:0:6255:0:(service.c:2324:ptlrpc_server_handle_request()) Handling RPC req@0000000002da67e2 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_000:0+-99:3352:x1862456998095104:12345-0@lo:38:kworker/u8:1.0 00010000:02000000:3.0:1776178533.242724:0:6255:0:(ldlm_lib.c:1139:target_handle_connect()) lustre-MDT0000: Not available for connect from 0@lo (stopping) 00000100:00100000:3.0:1776178533.242793:0:6255:0:(service.c:2372:ptlrpc_server_handle_request()) Handled RPC req@0000000002da67e2 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_000:0+-99:3352:x1862456998095104:12345-0@lo:38:kworker/u8:1.0 Request processed in 86us (14070us total) trans 0 rc -19/-19 00000100:00100000:3.0:1776178533.242801:0:6255:0:(nrs_fifo.c:241:nrs_fifo_req_stop()) NRS stop fifo request from 12345-0@lo, seq: 6883 00000100:00080000:0.0:1776178533.242941:0:3352:0:(import.c:86:import_set_state_nolock()) 00000000a7c55b1f lustre-MDT0000_UUID: changing import state from CONNECTING to DISCONN 00000100:00080000:0.0:1776178533.242945:0:3352:0:(import.c:1429:ptlrpc_connect_interpret()) recovery of lustre-MDT0000_UUID on 192.168.204.103@tcp failed (-19) 00000100:00100000:0.0:1776178533.242948:0:3352:0:(client.c:2215:ptlrpc_check_set()) Completed RPC req@00000000a979415d pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:lustre-MDT0000-lwp-MDT0001_UUID:3352:1862456998095104:0@lo:38:kworker/u8:1.0 00010000:00010000:2.0:1776178533.856919:0:6245:0:(ldlm_lockd.c:2792:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:2.0:1776178534.752791:0:3354:0:(client.c:1733:ptlrpc_send_new_req()) Sending RPC req@000000008584c293 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_01:lustre-MDT0001-mdtlov_UUID:3354:1862456998095168:0@lo:13:osp-pre-1-1.0 00000100:00100000:2.0:1776178534.752881:0:3354:0:(events.c:361:request_in_callback()) peer: 12345-0@lo (source: 12345-0@lo) 00000100:00100000:1.0:1776178534.753013:0:9554:0:(service.c:2152:ptlrpc_server_handle_req_in()) got req x1862456998095168 00000100:00100000:1.0:1776178534.753101:0:9554:0:(nrs_fifo.c:179:nrs_fifo_req_get()) NRS start fifo request from 12345-0@lo, seq: 617 00000100:00100000:1.0:1776178534.753107:0:9554:0:(service.c:2324:ptlrpc_server_handle_request()) Handling RPC req@0000000098682d9e pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_create00:lustre-MDT0001-mdtlov_UUID+5:3354:x1862456998095168:12345-0@lo:13:osp-pre-1-1.0 00002000:00000024:1.0:1776178534.753141:0:9554:0:(ofd_obd.c:740:ofd_statfs()) blocks cached 229376 granted 491746368 pending 0 free 3923689472 avail 3692167168 00000100:00100000:1.0:1776178534.753194:0:9554:0:(service.c:2372:ptlrpc_server_handle_request()) Handled RPC req@0000000098682d9e pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_create00:lustre-MDT0001-mdtlov_UUID+5:3354:x1862456998095168:12345-0@lo:13:osp-pre-1-1.0 Request processed in 86us (315us total) trans 0 rc 0/0 00000100:00100000:1.0:1776178534.753201:0:9554:0:(nrs_fifo.c:241:nrs_fifo_req_stop()) NRS stop fifo request from 12345-0@lo, seq: 617 00000100:00100000:2.0:1776178534.753296:0:3354:0:(client.c:2215:ptlrpc_check_set()) Completed RPC req@000000008584c293 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_01:lustre-MDT0001-mdtlov_UUID:3354:1862456998095168:0@lo:13:osp-pre-1-1.0 00010000:00010000:1.0:1776178534.880365:0:9224:0:(ldlm_lockd.c:2792:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:1.0:1776178535.264525:0:3356:0:(ptlrpcd.c:415:ptlrpcd_check()) transfer 1 async RPCs [2->3] 00000100:00100000:1.0:1776178535.264559:0:3356:0:(client.c:1733:ptlrpc_send_new_req()) Sending RPC req@00000000a979415d pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_03:lustre-MDT0001-mdtlov_UUID:3356:1862456998095232:0@lo:13:osp-pre-0-1.0 00000100:00100000:1.0:1776178535.264674:0:3356:0:(events.c:361:request_in_callback()) peer: 12345-0@lo (source: 12345-0@lo) 00000100:00100000:3.0:1776178535.264964:0:8231:0:(service.c:2152:ptlrpc_server_handle_req_in()) got req x1862456998095232 00000100:00100000:3.0:1776178535.265025:0:8231:0:(nrs_fifo.c:179:nrs_fifo_req_get()) NRS start fifo request from 12345-0@lo, seq: 618 00000100:00100000:3.0:1776178535.265033:0:8231:0:(service.c:2324:ptlrpc_server_handle_request()) Handling RPC req@0000000054bbc2c8 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_create00:lustre-MDT0001-mdtlov_UUID+5:3356:x1862456998095232:12345-0@lo:13:osp-pre-0-1.0 00002000:00000024:3.0:1776178535.265104:0:8231:0:(ofd_obd.c:740:ofd_statfs()) blocks cached 315392 granted 484883776 pending 0 free 3923689472 avail 3692167168 00000100:00100000:3.0:1776178535.265155:0:8231:0:(service.c:2372:ptlrpc_server_handle_request()) Handled RPC req@0000000054bbc2c8 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_create00:lustre-MDT0001-mdtlov_UUID+5:3356:x1862456998095232:12345-0@lo:13:osp-pre-0-1.0 Request processed in 122us (484us total) trans 0 rc 0/0 00000100:00100000:3.0:1776178535.265163:0:8231:0:(nrs_fifo.c:241:nrs_fifo_req_stop()) NRS stop fifo request from 12345-0@lo, seq: 618 00000100:00100000:0.0:1776178535.269806:0:3356:0:(client.c:2215:ptlrpc_check_set()) Completed RPC req@00000000a979415d pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_03:lustre-MDT0001-mdtlov_UUID:3356:1862456998095232:0@lo:13:osp-pre-0-1.0 00010000:00010000:1.0:1776178535.904335:0:9225:0:(ldlm_lockd.c:2792:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:2.0:1776178536.755329:0:3348:0:(events.c:361:request_in_callback()) peer: 12345-192.168.204.3@tcp (source: 12345-192.168.204.3@tcp) 00000100:00100000:2.0:1776178536.755443:0:3348:0:(events.c:361:request_in_callback()) peer: 12345-192.168.204.3@tcp (source: 12345-192.168.204.3@tcp) 00000100:00100000:2.0:1776178536.755478:0:3348:0:(events.c:361:request_in_callback()) peer: 12345-192.168.204.3@tcp (source: 12345-192.168.204.3@tcp) 00000100:00100000:3.0:1776178536.755886:0:20140:0:(service.c:2152:ptlrpc_server_handle_req_in()) got req x1862456979129792 00000100:00100000:3.0:1776178536.756329:0:20140:0:(nrs_fifo.c:179:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.204.3@tcp, seq: 285 00000100:00100000:3.0:1776178536.756338:0:20140:0:(service.c:2324:ptlrpc_server_handle_request()) Handling RPC req@0000000021ee30f5 pname:cluuid+ref:pid:xid:nid:opc:job ll_mgs_0002:5b5c66b1-1a2f-4651-8548-0c75b9a38a94+9:2272:x1862456979129792:12345-192.168.204.3@tcp:400:kworker/u8:4.0 00000100:00100000:3.0:1776178536.756463:0:20140:0:(service.c:2372:ptlrpc_server_handle_request()) Handled RPC req@0000000021ee30f5 pname:cluuid+ref:pid:xid:nid:opc:job ll_mgs_0002:5b5c66b1-1a2f-4651-8548-0c75b9a38a94+9:2272:x1862456979129792:12345-192.168.204.3@tcp:400:kworker/u8:4.0 Request processed in 125us (1020us total) trans 0 rc 0/0 00000100:00100000:0.0:1776178536.759290:0:11281:0:(service.c:2152:ptlrpc_server_handle_req_in()) got req x1862456979129856 00000100:00100000:0.0:1776178536.759392:0:11281:0:(nrs_fifo.c:179:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.204.3@tcp, seq: 6884 00000100:00100000:0.0:1776178536.759403:0:11281:0:(service.c:2324:ptlrpc_server_handle_request()) Handling RPC req@00000000b766fbe9 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_004:0+-99:2269:x1862456979129856:12345-192.168.204.3@tcp:38:kworker/u8:4.0 00010000:02000000:0.0:1776178536.759422:0:11281:0:(ldlm_lib.c:1139:target_handle_connect()) lustre-MDT0000: Not available for connect from 192.168.204.3@tcp (stopping) 00000100:00100000:0.0:1776178536.759495:0:11281:0:(service.c:2372:ptlrpc_server_handle_request()) Handled RPC req@00000000b766fbe9 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_004:0+-99:2269:x1862456979129856:12345-192.168.204.3@tcp:38:kworker/u8:4.0 Request processed in 94us (4176us total) trans 0 rc -19/-19 00000100:00100000:0.0:1776178536.759503:0:11281:0:(nrs_fifo.c:241:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.204.3@tcp, seq: 6884 00000100:00100000:1.0:1776178536.762292:0:14440:0:(service.c:2152:ptlrpc_server_handle_req_in()) got req x1862456979129984 00000100:00100000:1.0:1776178536.762408:0:14440:0:(nrs_fifo.c:179:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.204.3@tcp, seq: 771 00000100:00100000:1.0:1776178536.762415:0:14440:0:(service.c:2324:ptlrpc_server_handle_request()) Handling RPC req@00000000598050cc pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_004:b50963bd-8eba-4432-9a54-8888d00e3fa1+8:2272:x1862456979129984:12345-192.168.204.3@tcp:400:kworker/u8:4.0 00000100:00100000:1.0:1776178536.762498:0:14440:0:(service.c:2372:ptlrpc_server_handle_request()) Handled RPC req@00000000598050cc pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_004:b50963bd-8eba-4432-9a54-8888d00e3fa1+8:2272:x1862456979129984:12345-192.168.204.3@tcp:400:kworker/u8:4.0 Request processed in 83us (7020us total) trans 0 rc 0/0 00000100:00100000:1.0:1776178536.762506:0:14440:0:(nrs_fifo.c:241:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.204.3@tcp, seq: 771 00000100:00100000:1.0:1776178536.762602:0:3346:0:(events.c:361:request_in_callback()) peer: 12345-192.168.204.3@tcp (source: 12345-192.168.204.3@tcp) 00000100:00100000:1.0:1776178536.762686:0:3346:0:(events.c:361:request_in_callback()) peer: 12345-192.168.204.3@tcp (source: 12345-192.168.204.3@tcp) 00000100:00100000:1.0:1776178536.762779:0:14440:0:(service.c:2152:ptlrpc_server_handle_req_in()) got req x1862456979130048 00000100:00100000:1.0:1776178536.762806:0:14440:0:(nrs_fifo.c:179:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.204.3@tcp, seq: 772 00000100:00100000:1.0:1776178536.762810:0:14440:0:(service.c:2324:ptlrpc_server_handle_request()) Handling RPC req@00000000f064e279 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_004:b50963bd-8eba-4432-9a54-8888d00e3fa1+9:2273:x1862456979130048:12345-192.168.204.3@tcp:400:kworker/u8:4.0 00000100:00100000:1.0:1776178536.762842:0:14440:0:(service.c:2372:ptlrpc_server_handle_request()) Handled RPC req@00000000f064e279 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_004:b50963bd-8eba-4432-9a54-8888d00e3fa1+9:2273:x1862456979130048:12345-192.168.204.3@tcp:400:kworker/u8:4.0 Request processed in 31us (156us total) trans 0 rc 0/0 00000100:00100000:1.0:1776178536.762848:0:14440:0:(nrs_fifo.c:241:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.204.3@tcp, seq: 772 00000100:00100000:3.0:1776178536.764052:0:20140:0:(nrs_fifo.c:241:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.204.3@tcp, seq: 285 00000100:00100000:0.0:1776178536.764294:0:6257:0:(service.c:2152:ptlrpc_server_handle_req_in()) got req x1862456979129920 00000100:00100000:1.0:1776178536.764542:0:6257:0:(nrs_fifo.c:179:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.204.3@tcp, seq: 1123 00000100:00100000:1.0:1776178536.764551:0:6257:0:(service.c:2324:ptlrpc_server_handle_request()) Handling RPC req@00000000e837a7cd pname:cluuid+ref:pid:xid:nid:opc:job mdt00_002:b50963bd-8eba-4432-9a54-8888d00e3fa1+6:2273:x1862456979129920:12345-192.168.204.3@tcp:400:kworker/u8:4.0 00000100:00100000:1.0:1776178536.764690:0:6257:0:(service.c:2372:ptlrpc_server_handle_request()) Handled RPC req@00000000e837a7cd pname:cluuid+ref:pid:xid:nid:opc:job mdt00_002:b50963bd-8eba-4432-9a54-8888d00e3fa1+6:2273:x1862456979129920:12345-192.168.204.3@tcp:400:kworker/u8:4.0 Request processed in 139us (2088us total) trans 0 rc 0/0 00000100:00100000:1.0:1776178536.764698:0:6257:0:(nrs_fifo.c:241:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.204.3@tcp, seq: 1123 00010000:00010000:3.0:1776178536.928719:0:6244:0:(ldlm_lockd.c:2792:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00010000:00010000:2.0:1776178537.959121:0:6245:0:(ldlm_lockd.c:2792:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00080000:2.0:1776178538.336385:0:37:0:(pinger.c:251:ptlrpc_pinger_process_import()) lustre-MDT0000-lwp-OST0000_UUID->lustre-MDT0000_UUID: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00080000:2.0:1776178538.336397:0:37:0:(recover.c:57:ptlrpc_initiate_recovery()) lustre-MDT0000_UUID: starting recovery 00000100:00080000:2.0:1776178538.336401:0:37:0:(import.c:86:import_set_state_nolock()) 0000000047601a98 lustre-MDT0000_UUID: changing import state from DISCONN to CONNECTING 00000100:00080000:2.0:1776178538.336421:0:37:0:(import.c:534:import_select_connection()) lustre-MDT0000-lwp-OST0000: connect to NID 0@lo last attempt 1240 00000100:00080000:2.0:1776178538.336425:0:37:0:(import.c:581:import_select_connection()) lustre-MDT0000-lwp-OST0000: tried all connections, increasing latency to 21s 00000100:00080000:2.0:1776178538.336433:0:37:0:(import.c:615:import_select_connection()) lustre-MDT0000-lwp-OST0000: import 0000000047601a98 using connection 192.168.204.103@tcp/0@lo 00000100:00100000:2.0:1776178538.336469:0:37:0:(import.c:807:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5) req@000000009ff44a0a x1862456998095360/t0(0) o38->lustre-MDT0000-lwp-OST0000@0@lo:12/10 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' 00000100:00080000:2.0:1776178538.336527:0:37:0:(pinger.c:251:ptlrpc_pinger_process_import()) lustre-MDT0000-lwp-OST0001_UUID->lustre-MDT0000_UUID: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00080000:2.0:1776178538.336531:0:37:0:(recover.c:57:ptlrpc_initiate_recovery()) lustre-MDT0000_UUID: starting recovery 00000100:00080000:2.0:1776178538.336533:0:37:0:(import.c:86:import_set_state_nolock()) 00000000dfc8db13 lustre-MDT0000_UUID: changing import state from DISCONN to CONNECTING 00000100:00080000:2.0:1776178538.336536:0:37:0:(import.c:534:import_select_connection()) lustre-MDT0000-lwp-OST0001: connect to NID 0@lo last attempt 1240 00000100:00080000:2.0:1776178538.336538:0:37:0:(import.c:581:import_select_connection()) lustre-MDT0000-lwp-OST0001: tried all connections, increasing latency to 21s 00000100:00080000:2.0:1776178538.336541:0:37:0:(import.c:615:import_select_connection()) lustre-MDT0000-lwp-OST0001: import 00000000dfc8db13 using connection 192.168.204.103@tcp/0@lo 00000100:00100000:2.0:1776178538.336554:0:37:0:(import.c:807:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5) req@00000000ddf29755 x1862456998095488/t0(0) o38->lustre-MDT0000-lwp-OST0001@0@lo:12/10 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' 00000100:00100000:1.0:1776178538.336576:0:3352:0:(client.c:1733:ptlrpc_send_new_req()) Sending RPC req@000000009ff44a0a pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:lustre-MDT0000-lwp-OST0000_UUID:3352:1862456998095360:0@lo:38:kworker/u8:2.0 00000100:00080000:2.0:1776178538.336582:0:37:0:(pinger.c:251:ptlrpc_pinger_process_import()) lustre-MDT0001-mdtlov_UUID->lustre-MDT0000_UUID: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00080000:2.0:1776178538.336587:0:37:0:(recover.c:57:ptlrpc_initiate_recovery()) lustre-MDT0000_UUID: starting recovery 00000100:00080000:2.0:1776178538.336589:0:37:0:(import.c:86:import_set_state_nolock()) 00000000ba4f1821 lustre-MDT0000_UUID: changing import state from DISCONN to CONNECTING 00000100:00080000:2.0:1776178538.336592:0:37:0:(import.c:534:import_select_connection()) lustre-MDT0000-osp-MDT0001: connect to NID 0@lo last attempt 1240 00000100:00080000:2.0:1776178538.336594:0:37:0:(import.c:581:import_select_connection()) lustre-MDT0000-osp-MDT0001: tried all connections, increasing latency to 21s 00000100:00080000:2.0:1776178538.336598:0:37:0:(import.c:615:import_select_connection()) lustre-MDT0000-osp-MDT0001: import 00000000ba4f1821 using connection 192.168.204.103@tcp/0@lo 00000100:00100000:2.0:1776178538.336614:0:37:0:(import.c:807:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5) req@0000000068adf51c x1862456998095616/t0(0) o38->lustre-MDT0000-osp-MDT0001@0@lo:24/4 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' 00000100:00080000:2.0:1776178538.336639:0:37:0:(pinger.c:251:ptlrpc_pinger_process_import()) lustre-MDT0000-lwp-MDT0001_UUID->lustre-MDT0000_UUID: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00080000:2.0:1776178538.336642:0:37:0:(recover.c:57:ptlrpc_initiate_recovery()) lustre-MDT0000_UUID: starting recovery 00000100:00080000:2.0:1776178538.336643:0:37:0:(import.c:86:import_set_state_nolock()) 00000000a7c55b1f lustre-MDT0000_UUID: changing import state from DISCONN to CONNECTING 00000100:00080000:2.0:1776178538.336647:0:37:0:(import.c:534:import_select_connection()) lustre-MDT0000-lwp-MDT0001: connect to NID 0@lo last attempt 1240 00000100:00080000:2.0:1776178538.336648:0:37:0:(import.c:581:import_select_connection()) lustre-MDT0000-lwp-MDT0001: tried all connections, increasing latency to 21s 00000100:00080000:2.0:1776178538.336652:0:37:0:(import.c:615:import_select_connection()) lustre-MDT0000-lwp-MDT0001: import 00000000a7c55b1f using connection 192.168.204.103@tcp/0@lo 00000100:00100000:2.0:1776178538.336662:0:37:0:(import.c:807:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5) req@000000001f0b1867 x1862456998095808/t0(0) o38->lustre-MDT0000-lwp-MDT0001@0@lo:12/10 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' 00000100:00100000:2.0:1776178538.336693:0:3353:0:(client.c:1733:ptlrpc_send_new_req()) Sending RPC req@000000002db08e42 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_00:lustre-MDT0001-lwp-OST0000_UUID:3353:1862456998095424:0@lo:400:kworker/u8:2.0 00000100:00100000:0.0:1776178538.336990:0:3356:0:(client.c:1733:ptlrpc_send_new_req()) Sending RPC req@000000007d1fa4e3 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_03:ac61f839-0e8e-4cac-9c59-08bcfbbad330:3356:1862456998095296:0@lo:400:kworker/u8:2.0 00000100:00100000:3.0:1776178538.337048:0:3355:0:(client.c:1733:ptlrpc_send_new_req()) Sending RPC req@0000000053e142bc pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_02:lustre-MDT0001-mdtlov_UUID:3355:1862456998095680:0@lo:400:kworker/u8:2.0 00000100:00100000:2.0:1776178538.337340:0:3353:0:(events.c:361:request_in_callback()) peer: 12345-0@lo (source: 12345-0@lo) 00000100:00100000:2.0:1776178538.337391:0:3353:0:(ptlrpcd.c:415:ptlrpcd_check()) transfer 1 async RPCs [1->0] 00000100:00100000:2.0:1776178538.337397:0:3353:0:(client.c:1733:ptlrpc_send_new_req()) Sending RPC req@000000001554c6b4 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_00:lustre-MDT0001-lwp-OST0001_UUID:3353:1862456998095552:0@lo:400:kworker/u8:2.0 00000100:00100000:2.0:1776178538.337421:0:3353:0:(events.c:361:request_in_callback()) peer: 12345-0@lo (source: 12345-0@lo) 00000100:00100000:1.0:1776178538.337461:0:3352:0:(events.c:361:request_in_callback()) peer: 12345-0@lo (source: 12345-0@lo) 00000100:00100000:2.0:1776178538.337470:0:11281:0:(service.c:2152:ptlrpc_server_handle_req_in()) got req x1862456998095424 00000100:00100000:1.0:1776178538.337498:0:3352:0:(client.c:1733:ptlrpc_send_new_req()) Sending RPC req@00000000ddf29755 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:lustre-MDT0000-lwp-OST0001_UUID:3352:1862456998095488:0@lo:38:kworker/u8:2.0 00000100:00100000:0.0:1776178538.337500:0:3356:0:(events.c:361:request_in_callback()) peer: 12345-0@lo (source: 12345-0@lo) 00000100:00100000:3.0:1776178538.337508:0:3355:0:(events.c:361:request_in_callback()) peer: 12345-0@lo (source: 12345-0@lo) 00000100:00100000:2.0:1776178538.337529:0:11281:0:(service.c:2152:ptlrpc_server_handle_req_in()) got req x1862456998095552 00000100:00100000:0.0:1776178538.337535:0:3356:0:(client.c:1733:ptlrpc_send_new_req()) Sending RPC req@000000005fdea934 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_03:lustre-MDT0001-mdtlov_UUID:3356:1862456998095744:0@lo:400:kworker/u8:2.0 00000100:00100000:1.0:1776178538.337554:0:3352:0:(events.c:361:request_in_callback()) peer: 12345-0@lo (source: 12345-0@lo) 00000100:00100000:2.0:1776178538.337556:0:11281:0:(service.c:2152:ptlrpc_server_handle_req_in()) got req x1862456998095360 00000100:00100000:3.0:1776178538.337575:0:6255:0:(service.c:2152:ptlrpc_server_handle_req_in()) got req x1862456998095488 00000100:00100000:2.0:1776178538.337580:0:11281:0:(nrs_fifo.c:179:nrs_fifo_req_get()) NRS start fifo request from 12345-0@lo, seq: 1124 00000100:00100000:2.0:1776178538.337587:0:11281:0:(service.c:2324:ptlrpc_server_handle_request()) Handling RPC req@000000007886b426 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_004:lustre-MDT0001-lwp-OST0000_UUID+5:3353:x1862456998095424:12345-0@lo:400:kworker/u8:2.0 00000100:00100000:3.0:1776178538.337599:0:6255:0:(nrs_fifo.c:179:nrs_fifo_req_get()) NRS start fifo request from 12345-0@lo, seq: 1125 00000100:00100000:3.0:1776178538.337605:0:6255:0:(service.c:2324:ptlrpc_server_handle_request()) Handling RPC req@0000000013603266 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_000:lustre-MDT0001-lwp-OST0001_UUID+5:3353:x1862456998095552:12345-0@lo:400:kworker/u8:2.0 00000100:00100000:1.0:1776178538.338687:0:3352:0:(client.c:1733:ptlrpc_send_new_req()) Sending RPC req@0000000068adf51c pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:lustre-MDT0001-mdtlov_UUID:3352:1862456998095616:0@lo:38:kworker/u8:2.0 00000100:00100000:1.0:1776178538.338727:0:3352:0:(events.c:361:request_in_callback()) peer: 12345-0@lo (source: 12345-0@lo) 00000100:00100000:1.0:1776178538.338751:0:3352:0:(client.c:1733:ptlrpc_send_new_req()) Sending RPC req@000000001f0b1867 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:lustre-MDT0000-lwp-MDT0001_UUID:3352:1862456998095808:0@lo:38:kworker/u8:2.0 00000100:00100000:1.0:1776178538.338773:0:3352:0:(events.c:361:request_in_callback()) peer: 12345-0@lo (source: 12345-0@lo) 00000100:00100000:1.0:1776178538.338804:0:6257:0:(service.c:2152:ptlrpc_server_handle_req_in()) got req x1862456998095808 00000100:00100000:1.0:1776178538.338826:0:6257:0:(nrs_fifo.c:179:nrs_fifo_req_get()) NRS start fifo request from 12345-0@lo, seq: 6885 00000100:00100000:1.0:1776178538.338831:0:6257:0:(service.c:2324:ptlrpc_server_handle_request()) Handling RPC req@00000000002be227 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_002:0+-99:3352:x1862456998095360:12345-0@lo:38:kworker/u8:2.0 00010000:02000000:1.0:1776178538.338848:0:6257:0:(ldlm_lib.c:1139:target_handle_connect()) lustre-MDT0000: Not available for connect from 0@lo (stopping) 00000100:00100000:1.0:1776178538.338899:0:6257:0:(service.c:2372:ptlrpc_server_handle_request()) Handled RPC req@00000000002be227 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_002:0+-99:3352:x1862456998095360:12345-0@lo:38:kworker/u8:2.0 Request processed in 67us (1441us total) trans 0 rc -19/-19 00000100:00100000:1.0:1776178538.338906:0:6257:0:(nrs_fifo.c:241:nrs_fifo_req_stop()) NRS stop fifo request from 12345-0@lo, seq: 6885 00000100:00100000:1.0:1776178538.338926:0:6257:0:(nrs_fifo.c:179:nrs_fifo_req_get()) NRS start fifo request from 12345-0@lo, seq: 6886 00000100:00100000:1.0:1776178538.338929:0:6257:0:(service.c:2324:ptlrpc_server_handle_request()) Handling RPC req@0000000095214b8e pname:cluuid+ref:pid:xid:nid:opc:job mdt00_002:0+-99:3352:x1862456998095488:12345-0@lo:38:kworker/u8:2.0 00010000:02000000:1.0:1776178538.338936:0:6257:0:(ldlm_lib.c:1139:target_handle_connect()) lustre-MDT0000: Not available for connect from 0@lo (stopping) 00000100:00100000:1.0:1776178538.338958:0:6257:0:(service.c:2372:ptlrpc_server_handle_request()) Handled RPC req@0000000095214b8e pname:cluuid+ref:pid:xid:nid:opc:job mdt00_002:0+-99:3352:x1862456998095488:12345-0@lo:38:kworker/u8:2.0 Request processed in 28us (1407us total) trans 0 rc -19/-19 00000100:00100000:1.0:1776178538.338964:0:6257:0:(nrs_fifo.c:241:nrs_fifo_req_stop()) NRS stop fifo request from 12345-0@lo, seq: 6886 00000100:00100000:1.0:1776178538.338988:0:6257:0:(nrs_fifo.c:179:nrs_fifo_req_get()) NRS start fifo request from 12345-0@lo, seq: 6887 00000100:00100000:1.0:1776178538.338992:0:6257:0:(service.c:2324:ptlrpc_server_handle_request()) Handling RPC req@000000003095df64 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_002:0+-99:3352:x1862456998095808:12345-0@lo:38:kworker/u8:2.0 00010000:02000000:1.0:1776178538.338997:0:6257:0:(ldlm_lib.c:1139:target_handle_connect()) lustre-MDT0000: Not available for connect from 0@lo (stopping) 00000100:00100000:1.0:1776178538.339031:0:6257:0:(service.c:2372:ptlrpc_server_handle_request()) Handled RPC req@000000003095df64 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_002:0+-99:3352:x1862456998095808:12345-0@lo:38:kworker/u8:2.0 Request processed in 38us (257us total) trans 0 rc -19/-19 00000100:00100000:1.0:1776178538.339037:0:6257:0:(nrs_fifo.c:241:nrs_fifo_req_stop()) NRS stop fifo request from 12345-0@lo, seq: 6887 00000100:00100000:1.0:1776178538.339058:0:14128:0:(service.c:2152:ptlrpc_server_handle_req_in()) got req x1862456998095616 00000100:00100000:1.0:1776178538.339123:0:14128:0:(nrs_fifo.c:179:nrs_fifo_req_get()) NRS start fifo request from 12345-0@lo, seq: 773 00000100:00100000:1.0:1776178538.339128:0:14128:0:(service.c:2324:ptlrpc_server_handle_request()) Handling RPC req@00000000829797f1 pname:cluuid+ref:pid:xid:nid:opc:job mdt_out00_002:0+-99:3352:x1862456998095616:12345-0@lo:38:kworker/u8:2.0 00010000:02000000:1.0:1776178538.339137:0:14128:0:(ldlm_lib.c:1139:target_handle_connect()) lustre-MDT0000: Not available for connect from 0@lo (stopping) 00000100:00100000:3.0:1776178538.339196:0:6255:0:(service.c:2372:ptlrpc_server_handle_request()) Handled RPC req@0000000013603266 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_000:lustre-MDT0001-lwp-OST0001_UUID+5:3353:x1862456998095552:12345-0@lo:400:kworker/u8:2.0 Request processed in 1589us (1772us total) trans 0 rc 0/0 00000100:00100000:2.0:1776178538.339197:0:11281:0:(service.c:2372:ptlrpc_server_handle_request()) Handled RPC req@000000007886b426 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_004:lustre-MDT0001-lwp-OST0000_UUID+5:3353:x1862456998095424:12345-0@lo:400:kworker/u8:2.0 Request processed in 1609us (1873us total) trans 0 rc 0/0 00000100:00100000:3.0:1776178538.339209:0:6255:0:(nrs_fifo.c:241:nrs_fifo_req_stop()) NRS stop fifo request from 12345-0@lo, seq: 1125 00000100:00100000:2.0:1776178538.339216:0:11281:0:(nrs_fifo.c:241:nrs_fifo_req_stop()) NRS stop fifo request from 12345-0@lo, seq: 1124 00000100:00100000:3.0:1776178538.339246:0:20140:0:(service.c:2152:ptlrpc_server_handle_req_in()) got req x1862456998095296 00000100:00100000:3.0:1776178538.339283:0:20140:0:(nrs_fifo.c:179:nrs_fifo_req_get()) NRS start fifo request from 12345-0@lo, seq: 286 00000100:00100000:3.0:1776178538.339288:0:20140:0:(service.c:2324:ptlrpc_server_handle_request()) Handling RPC req@000000006efdb84e pname:cluuid+ref:pid:xid:nid:opc:job ll_mgs_0002:ac61f839-0e8e-4cac-9c59-08bcfbbad330+18:3356:x1862456998095296:12345-0@lo:400:kworker/u8:2.0 00000100:00100000:2.0:1776178538.339289:0:20762:0:(service.c:2152:ptlrpc_server_handle_req_in()) got req x1862456998095680 00000100:00100000:2.0:1776178538.339318:0:20762:0:(nrs_fifo.c:179:nrs_fifo_req_get()) NRS start fifo request from 12345-0@lo, seq: 773 00000100:00100000:2.0:1776178538.339324:0:20762:0:(service.c:2324:ptlrpc_server_handle_request()) Handling RPC req@00000000f9429027 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_013:lustre-MDT0001-mdtlov_UUID+5:3355:x1862456998095680:12345-0@lo:400:kworker/u8:2.0 00000100:00100000:3.0:1776178538.339343:0:20140:0:(service.c:2372:ptlrpc_server_handle_request()) Handled RPC req@000000006efdb84e pname:cluuid+ref:pid:xid:nid:opc:job ll_mgs_0002:ac61f839-0e8e-4cac-9c59-08bcfbbad330+18:3356:x1862456998095296:12345-0@lo:400:kworker/u8:2.0 Request processed in 53us (1846us total) trans 0 rc 0/0 00000100:00100000:3.0:1776178538.339351:0:20140:0:(nrs_fifo.c:241:nrs_fifo_req_stop()) NRS stop fifo request from 12345-0@lo, seq: 286 00000100:00100000:2.0:1776178538.339387:0:20762:0:(service.c:2372:ptlrpc_server_handle_request()) Handled RPC req@00000000f9429027 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_013:lustre-MDT0001-mdtlov_UUID+5:3355:x1862456998095680:12345-0@lo:400:kworker/u8:2.0 Request processed in 62us (1881us total) trans 0 rc 0/0 00000100:00100000:2.0:1776178538.339394:0:20762:0:(nrs_fifo.c:241:nrs_fifo_req_stop()) NRS stop fifo request from 12345-0@lo, seq: 773 00000100:00100000:3.0:1776178538.339425:0:3353:0:(client.c:2215:ptlrpc_check_set()) Completed RPC req@000000001554c6b4 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_00:lustre-MDT0001-lwp-OST0001_UUID:3353:1862456998095552:0@lo:400:kworker/u8:2.0 00000100:00100000:3.0:1776178538.339444:0:3353:0:(client.c:2215:ptlrpc_check_set()) Completed RPC req@000000002db08e42 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_00:lustre-MDT0001-lwp-OST0000_UUID:3353:1862456998095424:0@lo:400:kworker/u8:2.0 00000100:00100000:2.0:1776178538.339456:0:3355:0:(client.c:2215:ptlrpc_check_set()) Completed RPC req@0000000053e142bc pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_02:lustre-MDT0001-mdtlov_UUID:3355:1862456998095680:0@lo:400:kworker/u8:2.0 00000100:00080000:3.0:1776178538.339522:0:3352:0:(import.c:86:import_set_state_nolock()) 00000000ba4f1821 lustre-MDT0000_UUID: changing import state from CONNECTING to DISCONN 00000100:00080000:3.0:1776178538.339527:0:3352:0:(import.c:1429:ptlrpc_connect_interpret()) recovery of lustre-MDT0000_UUID on 192.168.204.103@tcp failed (-19) 00000100:00100000:3.0:1776178538.339531:0:3352:0:(client.c:2215:ptlrpc_check_set()) Completed RPC req@0000000068adf51c pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:lustre-MDT0001-mdtlov_UUID:3352:1862456998095616:0@lo:38:kworker/u8:2.0 00000100:00080000:3.0:1776178538.339541:0:3352:0:(import.c:86:import_set_state_nolock()) 00000000a7c55b1f lustre-MDT0000_UUID: changing import state from CONNECTING to DISCONN 00000100:00080000:3.0:1776178538.339544:0:3352:0:(import.c:1429:ptlrpc_connect_interpret()) recovery of lustre-MDT0000_UUID on 192.168.204.103@tcp failed (-19) 00000100:00100000:3.0:1776178538.339546:0:3352:0:(client.c:2215:ptlrpc_check_set()) Completed RPC req@000000001f0b1867 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:lustre-MDT0000-lwp-MDT0001_UUID:3352:1862456998095808:0@lo:38:kworker/u8:2.0 00000100:00080000:3.0:1776178538.339557:0:3352:0:(import.c:86:import_set_state_nolock()) 0000000047601a98 lustre-MDT0000_UUID: changing import state from CONNECTING to DISCONN 00000100:00080000:3.0:1776178538.339560:0:3352:0:(import.c:1429:ptlrpc_connect_interpret()) recovery of lustre-MDT0000_UUID on 192.168.204.103@tcp failed (-19) 00000100:00100000:3.0:1776178538.339563:0:3352:0:(client.c:2215:ptlrpc_check_set()) Completed RPC req@000000009ff44a0a pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:lustre-MDT0000-lwp-OST0000_UUID:3352:1862456998095360:0@lo:38:kworker/u8:2.0 00000100:00080000:3.0:1776178538.339571:0:3352:0:(import.c:86:import_set_state_nolock()) 00000000dfc8db13 lustre-MDT0000_UUID: changing import state from CONNECTING to DISCONN 00000100:00080000:3.0:1776178538.339573:0:3352:0:(import.c:1429:ptlrpc_connect_interpret()) recovery of lustre-MDT0000_UUID on 192.168.204.103@tcp failed (-19) 00000100:00100000:3.0:1776178538.339576:0:3352:0:(client.c:2215:ptlrpc_check_set()) Completed RPC req@00000000ddf29755 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:lustre-MDT0000-lwp-OST0001_UUID:3352:1862456998095488:0@lo:38:kworker/u8:2.0 00000100:00100000:1.0:1776178538.339675:0:14128:0:(service.c:2372:ptlrpc_server_handle_request()) Handled RPC req@00000000829797f1 pname:cluuid+ref:pid:xid:nid:opc:job mdt_out00_002:0+-99:3352:x1862456998095616:12345-0@lo:38:kworker/u8:2.0 Request processed in 545us (947us total) trans 0 rc -19/-19 00000100:00100000:0.0:1776178538.339679:0:3356:0:(events.c:361:request_in_callback()) peer: 12345-0@lo (source: 12345-0@lo) 00000100:00100000:1.0:1776178538.339685:0:14128:0:(nrs_fifo.c:241:nrs_fifo_req_stop()) NRS stop fifo request from 12345-0@lo, seq: 773 00000100:00100000:0.0:1776178538.339716:0:3356:0:(client.c:2215:ptlrpc_check_set()) Completed RPC req@000000007d1fa4e3 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_03:ac61f839-0e8e-4cac-9c59-08bcfbbad330:3356:1862456998095296:0@lo:400:kworker/u8:2.0 00000100:00100000:3.0:1776178538.339744:0:20759:0:(service.c:2152:ptlrpc_server_handle_req_in()) got req x1862456998095744 00000100:00100000:3.0:1776178538.339767:0:20759:0:(nrs_fifo.c:179:nrs_fifo_req_get()) NRS start fifo request from 12345-0@lo, seq: 774 00000100:00100000:3.0:1776178538.339771:0:20759:0:(service.c:2324:ptlrpc_server_handle_request()) Handling RPC req@00000000a979415d pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_010:lustre-MDT0001-mdtlov_UUID+5:3356:x1862456998095744:12345-0@lo:400:kworker/u8:2.0 00000100:00100000:3.0:1776178538.339813:0:20759:0:(service.c:2372:ptlrpc_server_handle_request()) Handled RPC req@00000000a979415d pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_010:lustre-MDT0001-mdtlov_UUID+5:3356:x1862456998095744:12345-0@lo:400:kworker/u8:2.0 Request processed in 40us (142us total) trans 0 rc 0/0 00000100:00100000:3.0:1776178538.339820:0:20759:0:(nrs_fifo.c:241:nrs_fifo_req_stop()) NRS stop fifo request from 12345-0@lo, seq: 774 00000100:00100000:0.0:1776178538.342796:0:3356:0:(client.c:2215:ptlrpc_check_set()) Completed RPC req@000000005fdea934 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_03:lustre-MDT0001-mdtlov_UUID:3356:1862456998095744:0@lo:400:kworker/u8:2.0 00010000:00010000:1.0:1776178538.977965:0:9224:0:(ldlm_lockd.c:2792:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:2.0:1776178539.872661:0:3353:0:(client.c:1733:ptlrpc_send_new_req()) Sending RPC req@000000008584c293 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_00:lustre-MDT0001-mdtlov_UUID:3353:1862456998095872:0@lo:13:osp-pre-1-1.0 00000100:00100000:2.0:1776178539.872834:0:3353:0:(events.c:361:request_in_callback()) peer: 12345-0@lo (source: 12345-0@lo) 00000100:00100000:1.0:1776178539.873363:0:9554:0:(service.c:2152:ptlrpc_server_handle_req_in()) got req x1862456998095872 00000100:00100000:1.0:1776178539.873451:0:9554:0:(nrs_fifo.c:179:nrs_fifo_req_get()) NRS start fifo request from 12345-0@lo, seq: 619 00000100:00100000:1.0:1776178539.873462:0:9554:0:(service.c:2324:ptlrpc_server_handle_request()) Handling RPC req@00000000500c0ae6 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_create00:lustre-MDT0001-mdtlov_UUID+5:3353:x1862456998095872:12345-0@lo:13:osp-pre-1-1.0 00002000:00000024:1.0:1776178539.873511:0:9554:0:(ofd_obd.c:740:ofd_statfs()) blocks cached 229376 granted 491746368 pending 0 free 3923689472 avail 3692167168 00000100:00100000:1.0:1776178539.873581:0:9554:0:(service.c:2372:ptlrpc_server_handle_request()) Handled RPC req@00000000500c0ae6 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_create00:lustre-MDT0001-mdtlov_UUID+5:3353:x1862456998095872:12345-0@lo:13:osp-pre-1-1.0 Request processed in 121us (750us total) trans 0 rc 0/0 00000100:00100000:1.0:1776178539.873589:0:9554:0:(nrs_fifo.c:241:nrs_fifo_req_stop()) NRS stop fifo request from 12345-0@lo, seq: 619 00000100:00100000:2.0:1776178539.873867:0:3353:0:(client.c:2215:ptlrpc_check_set()) Completed RPC req@000000008584c293 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_00:lustre-MDT0001-mdtlov_UUID:3353:1862456998095872:0@lo:13:osp-pre-1-1.0 00010000:00010000:1.0:1776178540.000646:0:9225:0:(ldlm_lockd.c:2792:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:2.0:1776178540.384830:0:3354:0:(client.c:1733:ptlrpc_send_new_req()) Sending RPC req@00000000949e3fea pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_01:lustre-MDT0001-mdtlov_UUID:3354:1862456998095936:0@lo:13:osp-pre-0-1.0 00000100:00100000:2.0:1776178540.384948:0:3354:0:(events.c:361:request_in_callback()) peer: 12345-0@lo (source: 12345-0@lo) 00000100:00100000:3.0:1776178540.385008:0:8231:0:(service.c:2152:ptlrpc_server_handle_req_in()) got req x1862456998095936 00000100:00100000:3.0:1776178540.385099:0:8231:0:(nrs_fifo.c:179:nrs_fifo_req_get()) NRS start fifo request from 12345-0@lo, seq: 620 00000100:00100000:3.0:1776178540.385107:0:8231:0:(service.c:2324:ptlrpc_server_handle_request()) Handling RPC req@000000005ff04c99 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_create00:lustre-MDT0001-mdtlov_UUID+5:3354:x1862456998095936:12345-0@lo:13:osp-pre-0-1.0 00002000:00000024:3.0:1776178540.385145:0:8231:0:(ofd_obd.c:740:ofd_statfs()) blocks cached 315392 granted 484883776 pending 0 free 3923689472 avail 3692167168 00000100:00100000:3.0:1776178540.385206:0:8231:0:(service.c:2372:ptlrpc_server_handle_request()) Handled RPC req@000000005ff04c99 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_create00:lustre-MDT0001-mdtlov_UUID+5:3354:x1862456998095936:12345-0@lo:13:osp-pre-0-1.0 Request processed in 99us (260us total) trans 0 rc 0/0 00000100:00100000:3.0:1776178540.385214:0:8231:0:(nrs_fifo.c:241:nrs_fifo_req_stop()) NRS stop fifo request from 12345-0@lo, seq: 620 00000100:00100000:2.0:1776178540.386518:0:3354:0:(client.c:2215:ptlrpc_check_set()) Completed RPC req@00000000949e3fea pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_01:lustre-MDT0001-mdtlov_UUID:3354:1862456998095936:0@lo:13:osp-pre-0-1.0 00010000:00010000:3.0:1776178541.030693:0:6244:0:(ldlm_lockd.c:2792:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:1.0:1776178541.871110:0:3347:0:(events.c:361:request_in_callback()) peer: 12345-192.168.204.3@tcp (source: 12345-192.168.204.3@tcp) 00000100:00100000:1.0:1776178541.871165:0:3347:0:(events.c:361:request_in_callback()) peer: 12345-192.168.204.3@tcp (source: 12345-192.168.204.3@tcp) 00000100:00100000:1.0:1776178541.871217:0:3347:0:(events.c:361:request_in_callback()) peer: 12345-192.168.204.3@tcp (source: 12345-192.168.204.3@tcp) 00000100:00100000:3.0:1776178541.871515:0:8250:0:(service.c:2152:ptlrpc_server_handle_req_in()) got req x1862456979130240 00000100:00100000:1.0:1776178541.871607:0:3347:0:(events.c:361:request_in_callback()) peer: 12345-192.168.204.3@tcp (source: 12345-192.168.204.3@tcp) 00000100:00100000:1.0:1776178541.871642:0:20137:0:(service.c:2152:ptlrpc_server_handle_req_in()) got req x1862456979130112 00000100:00100000:1.0:1776178541.871973:0:20137:0:(nrs_fifo.c:179:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.204.3@tcp, seq: 287 00000100:00100000:1.0:1776178541.871982:0:20137:0:(service.c:2324:ptlrpc_server_handle_request()) Handling RPC req@0000000038b01669 pname:cluuid+ref:pid:xid:nid:opc:job ll_mgs_0000:5b5c66b1-1a2f-4651-8548-0c75b9a38a94+9:2271:x1862456979130112:12345-192.168.204.3@tcp:400:kworker/u8:3.0 00000100:00100000:1.0:1776178541.872100:0:20137:0:(service.c:2372:ptlrpc_server_handle_request()) Handled RPC req@0000000038b01669 pname:cluuid+ref:pid:xid:nid:opc:job ll_mgs_0000:5b5c66b1-1a2f-4651-8548-0c75b9a38a94+9:2271:x1862456979130112:12345-192.168.204.3@tcp:400:kworker/u8:3.0 Request processed in 118us (883us total) trans 0 rc 0/0 00000100:00100000:1.0:1776178541.872110:0:20137:0:(nrs_fifo.c:241:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.204.3@tcp, seq: 287 00000100:00100000:1.0:1776178541.872153:0:20762:0:(service.c:2152:ptlrpc_server_handle_req_in()) got req x1862456979130304 00000100:00100000:1.0:1776178541.872185:0:20762:0:(nrs_fifo.c:179:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.204.3@tcp, seq: 775 00000100:00100000:1.0:1776178541.872189:0:20762:0:(service.c:2324:ptlrpc_server_handle_request()) Handling RPC req@000000009c55a259 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_013:b50963bd-8eba-4432-9a54-8888d00e3fa1+8:2271:x1862456979130304:12345-192.168.204.3@tcp:400:kworker/u8:3.0 00000100:00100000:1.0:1776178541.872221:0:20762:0:(service.c:2372:ptlrpc_server_handle_request()) Handled RPC req@000000009c55a259 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_013:b50963bd-8eba-4432-9a54-8888d00e3fa1+8:2271:x1862456979130304:12345-192.168.204.3@tcp:400:kworker/u8:3.0 Request processed in 31us (615us total) trans 0 rc 0/0 00000100:00100000:1.0:1776178541.872228:0:20762:0:(nrs_fifo.c:241:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.204.3@tcp, seq: 775 00000100:00100000:1.0:1776178541.874635:0:8250:0:(service.c:2152:ptlrpc_server_handle_req_in()) got req x1862456979130176 00000100:00100000:0.0:1776178541.874866:0:6256:0:(nrs_fifo.c:179:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.204.3@tcp, seq: 1126 00000100:00100000:0.0:1776178541.874879:0:6256:0:(service.c:2324:ptlrpc_server_handle_request()) Handling RPC req@000000009b5eeec2 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_001:b50963bd-8eba-4432-9a54-8888d00e3fa1+6:2270:x1862456979130240:12345-192.168.204.3@tcp:400:kworker/u8:3.0 00000100:00100000:0.0:1776178541.875006:0:6256:0:(service.c:2372:ptlrpc_server_handle_request()) Handled RPC req@000000009b5eeec2 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_001:b50963bd-8eba-4432-9a54-8888d00e3fa1+6:2270:x1862456979130240:12345-192.168.204.3@tcp:400:kworker/u8:3.0 Request processed in 129us (3950us total) trans 0 rc 0/0 00000100:00100000:0.0:1776178541.875029:0:6256:0:(nrs_fifo.c:241:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.204.3@tcp, seq: 1126 00000100:00100000:0.0:1776178541.875057:0:6256:0:(nrs_fifo.c:179:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.204.3@tcp, seq: 6888 00000100:00100000:0.0:1776178541.875080:0:6256:0:(service.c:2324:ptlrpc_server_handle_request()) Handling RPC req@00000000ddc1dc02 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_001:0+-99:2269:x1862456979130176:12345-192.168.204.3@tcp:38:kworker/u8:3.0 00010000:02000000:0.0:1776178541.875099:0:6256:0:(ldlm_lib.c:1139:target_handle_connect()) lustre-MDT0000: Not available for connect from 192.168.204.3@tcp (stopping) 00000100:00100000:0.0:1776178541.875132:0:6256:0:(service.c:2372:ptlrpc_server_handle_request()) Handled RPC req@00000000ddc1dc02 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_001:0+-99:2269:x1862456979130176:12345-192.168.204.3@tcp:38:kworker/u8:3.0 Request processed in 51us (3967us total) trans 0 rc -19/-19 00000100:00100000:0.0:1776178541.875139:0:6256:0:(nrs_fifo.c:241:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.204.3@tcp, seq: 6888 00000100:00100000:3.0:1776178541.875987:0:3347:0:(events.c:361:request_in_callback()) peer: 12345-192.168.204.3@tcp (source: 12345-192.168.204.3@tcp) 00000100:00100000:0.0:1776178541.876190:0:20759:0:(service.c:2152:ptlrpc_server_handle_req_in()) got req x1862456979130368 00000100:00100000:0.0:1776178541.876245:0:20759:0:(nrs_fifo.c:179:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.204.3@tcp, seq: 776 00000100:00100000:0.0:1776178541.876252:0:20759:0:(service.c:2324:ptlrpc_server_handle_request()) Handling RPC req@0000000098006deb pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_010:b50963bd-8eba-4432-9a54-8888d00e3fa1+9:2270:x1862456979130368:12345-192.168.204.3@tcp:400:kworker/u8:3.0 00000100:00100000:0.0:1776178541.876935:0:20759:0:(service.c:2372:ptlrpc_server_handle_request()) Handled RPC req@0000000098006deb pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_010:b50963bd-8eba-4432-9a54-8888d00e3fa1+9:2270:x1862456979130368:12345-192.168.204.3@tcp:400:kworker/u8:3.0 Request processed in 681us (951us total) trans 0 rc 0/0 00000100:00100000:0.0:1776178541.876945:0:20759:0:(nrs_fifo.c:241:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.204.3@tcp, seq: 776 00010000:00010000:2.0:1776178542.052002:0:6245:0:(ldlm_lockd.c:2792:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00010000:00010000:1.0:1776178543.072533:0:9224:0:(ldlm_lockd.c:2792:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00080000:0.0:1776178543.456522:0:36:0:(pinger.c:251:ptlrpc_pinger_process_import()) lustre-MDT0000-lwp-OST0000_UUID->lustre-MDT0000_UUID: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00080000:0.0:1776178543.456532:0:36:0:(recover.c:57:ptlrpc_initiate_recovery()) lustre-MDT0000_UUID: starting recovery 00000100:00080000:0.0:1776178543.456537:0:36:0:(import.c:86:import_set_state_nolock()) 0000000047601a98 lustre-MDT0000_UUID: changing import state from DISCONN to CONNECTING 00000100:00080000:0.0:1776178543.456548:0:36:0:(import.c:534:import_select_connection()) lustre-MDT0000-lwp-OST0000: connect to NID 0@lo last attempt 1245 00000100:00080000:0.0:1776178543.456553:0:36:0:(import.c:581:import_select_connection()) lustre-MDT0000-lwp-OST0000: tried all connections, increasing latency to 21s 00000100:00080000:0.0:1776178543.456562:0:36:0:(import.c:615:import_select_connection()) lustre-MDT0000-lwp-OST0000: import 0000000047601a98 using connection 192.168.204.103@tcp/0@lo 00000100:00100000:1.0:1776178543.456567:0:3356:0:(ptlrpcd.c:415:ptlrpcd_check()) transfer 1 async RPCs [2->3] 00000100:00100000:0.0:1776178543.456590:0:36:0:(import.c:807:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5) req@00000000fd32fba5 x1862456998096064/t0(0) o38->lustre-MDT0000-lwp-OST0000@0@lo:12/10 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' 00000100:00100000:1.0:1776178543.456594:0:3356:0:(client.c:1733:ptlrpc_send_new_req()) Sending RPC req@000000009b04a931 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_03:ac61f839-0e8e-4cac-9c59-08bcfbbad330:3356:1862456998096000:0@lo:400:kworker/u8:1.0 00000100:00080000:0.0:1776178543.456637:0:36:0:(pinger.c:251:ptlrpc_pinger_process_import()) lustre-MDT0000-lwp-OST0001_UUID->lustre-MDT0000_UUID: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00080000:0.0:1776178543.456641:0:36:0:(recover.c:57:ptlrpc_initiate_recovery()) lustre-MDT0000_UUID: starting recovery 00000100:00080000:0.0:1776178543.456642:0:36:0:(import.c:86:import_set_state_nolock()) 00000000dfc8db13 lustre-MDT0000_UUID: changing import state from DISCONN to CONNECTING 00000100:00080000:0.0:1776178543.456648:0:36:0:(import.c:534:import_select_connection()) lustre-MDT0000-lwp-OST0001: connect to NID 0@lo last attempt 1245 00000100:00080000:0.0:1776178543.456650:0:36:0:(import.c:581:import_select_connection()) lustre-MDT0000-lwp-OST0001: tried all connections, increasing latency to 21s 00000100:00080000:0.0:1776178543.456654:0:36:0:(import.c:615:import_select_connection()) lustre-MDT0000-lwp-OST0001: import 00000000dfc8db13 using connection 192.168.204.103@tcp/0@lo 00000100:00100000:0.0:1776178543.456666:0:36:0:(import.c:807:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5) req@00000000606788fe x1862456998096192/t0(0) o38->lustre-MDT0000-lwp-OST0001@0@lo:12/10 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' 00000100:00080000:0.0:1776178543.456694:0:36:0:(pinger.c:251:ptlrpc_pinger_process_import()) lustre-MDT0001-mdtlov_UUID->lustre-MDT0000_UUID: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00080000:0.0:1776178543.456698:0:36:0:(recover.c:57:ptlrpc_initiate_recovery()) lustre-MDT0000_UUID: starting recovery 00000100:00080000:0.0:1776178543.456699:0:36:0:(import.c:86:import_set_state_nolock()) 00000000ba4f1821 lustre-MDT0000_UUID: changing import state from DISCONN to CONNECTING 00000100:00080000:0.0:1776178543.456703:0:36:0:(import.c:534:import_select_connection()) lustre-MDT0000-osp-MDT0001: connect to NID 0@lo last attempt 1245 00000100:00080000:0.0:1776178543.456705:0:36:0:(import.c:581:import_select_connection()) lustre-MDT0000-osp-MDT0001: tried all connections, increasing latency to 21s 00000100:00100000:1.0:1776178543.456709:0:3356:0:(events.c:361:request_in_callback()) peer: 12345-0@lo (source: 12345-0@lo) 00000100:00080000:0.0:1776178543.456712:0:36:0:(import.c:615:import_select_connection()) lustre-MDT0000-osp-MDT0001: import 00000000ba4f1821 using connection 192.168.204.103@tcp/0@lo 00000100:00100000:0.0:1776178543.456727:0:36:0:(import.c:807:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5) req@000000001c9b18be x1862456998096320/t0(0) o38->lustre-MDT0000-osp-MDT0001@0@lo:24/4 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' 00000100:00100000:1.0:1776178543.456746:0:3356:0:(client.c:1733:ptlrpc_send_new_req()) Sending RPC req@00000000949e3fea pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_03:lustre-MDT0001-lwp-OST0000_UUID:3356:1862456998096128:0@lo:400:kworker/u8:1.0 00000100:00080000:0.0:1776178543.456755:0:36:0:(pinger.c:251:ptlrpc_pinger_process_import()) lustre-MDT0000-lwp-MDT0001_UUID->lustre-MDT0000_UUID: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00080000:0.0:1776178543.456760:0:36:0:(recover.c:57:ptlrpc_initiate_recovery()) lustre-MDT0000_UUID: starting recovery 00000100:00080000:0.0:1776178543.456761:0:36:0:(import.c:86:import_set_state_nolock()) 00000000a7c55b1f lustre-MDT0000_UUID: changing import state from DISCONN to CONNECTING 00000100:00080000:0.0:1776178543.456765:0:36:0:(import.c:534:import_select_connection()) lustre-MDT0000-lwp-MDT0001: connect to NID 0@lo last attempt 1245 00000100:00080000:0.0:1776178543.456768:0:36:0:(import.c:581:import_select_connection()) lustre-MDT0000-lwp-MDT0001: tried all connections, increasing latency to 21s 00000100:00080000:0.0:1776178543.456771:0:36:0:(import.c:615:import_select_connection()) lustre-MDT0000-lwp-MDT0001: import 00000000a7c55b1f using connection 192.168.204.103@tcp/0@lo 00000100:00100000:1.0:1776178543.456782:0:3356:0:(events.c:361:request_in_callback()) peer: 12345-0@lo (source: 12345-0@lo) 00000100:00100000:0.0:1776178543.456785:0:36:0:(import.c:807:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5) req@0000000020763766 x1862456998096512/t0(0) o38->lustre-MDT0000-lwp-MDT0001@0@lo:12/10 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' 00000100:00100000:1.0:1776178543.456801:0:3356:0:(ptlrpcd.c:415:ptlrpcd_check()) transfer 1 async RPCs [2->3] 00000100:00100000:1.0:1776178543.456809:0:3356:0:(client.c:1733:ptlrpc_send_new_req()) Sending RPC req@00000000702f2be6 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_03:lustre-MDT0001-mdtlov_UUID:3356:1862456998096448:0@lo:400:kworker/u8:1.0 00000100:00100000:1.0:1776178543.456834:0:3356:0:(events.c:361:request_in_callback()) peer: 12345-0@lo (source: 12345-0@lo) 00000100:00100000:0.0:1776178543.456855:0:20140:0:(service.c:2152:ptlrpc_server_handle_req_in()) got req x1862456998096000 00000100:00100000:1.0:1776178543.456870:0:20759:0:(service.c:2152:ptlrpc_server_handle_req_in()) got req x1862456998096448 00000100:00100000:1.0:1776178543.456911:0:20759:0:(nrs_fifo.c:179:nrs_fifo_req_get()) NRS start fifo request from 12345-0@lo, seq: 777 00000100:00100000:0.0:1776178543.456916:0:20140:0:(nrs_fifo.c:179:nrs_fifo_req_get()) NRS start fifo request from 12345-0@lo, seq: 288 00000100:00100000:1.0:1776178543.456921:0:20759:0:(service.c:2324:ptlrpc_server_handle_request()) Handling RPC req@0000000072865dff pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_010:lustre-MDT0001-mdtlov_UUID+5:3356:x1862456998096448:12345-0@lo:400:kworker/u8:1.0 00000100:00100000:0.0:1776178543.456925:0:20140:0:(service.c:2324:ptlrpc_server_handle_request()) Handling RPC req@000000003524ab73 pname:cluuid+ref:pid:xid:nid:opc:job ll_mgs_0002:ac61f839-0e8e-4cac-9c59-08bcfbbad330+18:3356:x1862456998096000:12345-0@lo:400:kworker/u8:1.0 00000100:00100000:1.0:1776178543.456983:0:20759:0:(service.c:2372:ptlrpc_server_handle_request()) Handled RPC req@0000000072865dff pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_010:lustre-MDT0001-mdtlov_UUID+5:3356:x1862456998096448:12345-0@lo:400:kworker/u8:1.0 Request processed in 63us (148us total) trans 0 rc 0/0 00000100:00100000:1.0:1776178543.456994:0:20759:0:(nrs_fifo.c:241:nrs_fifo_req_stop()) NRS stop fifo request from 12345-0@lo, seq: 777 00000100:00100000:0.0:1776178543.456995:0:20140:0:(service.c:2372:ptlrpc_server_handle_request()) Handled RPC req@000000003524ab73 pname:cluuid+ref:pid:xid:nid:opc:job ll_mgs_0002:ac61f839-0e8e-4cac-9c59-08bcfbbad330+18:3356:x1862456998096000:12345-0@lo:400:kworker/u8:1.0 Request processed in 69us (287us total) trans 0 rc 0/0 00000100:00100000:0.0:1776178543.457004:0:20140:0:(nrs_fifo.c:241:nrs_fifo_req_stop()) NRS stop fifo request from 12345-0@lo, seq: 288 00000100:00100000:1.0:1776178543.457047:0:3356:0:(client.c:2215:ptlrpc_check_set()) Completed RPC req@00000000702f2be6 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_03:lustre-MDT0001-mdtlov_UUID:3356:1862456998096448:0@lo:400:kworker/u8:1.0 00000100:00100000:3.0:1776178543.457137:0:3352:0:(client.c:1733:ptlrpc_send_new_req()) Sending RPC req@00000000fd32fba5 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:lustre-MDT0000-lwp-OST0000_UUID:3352:1862456998096064:0@lo:38:kworker/u8:1.0 00000100:00100000:3.0:1776178543.457178:0:3352:0:(events.c:361:request_in_callback()) peer: 12345-0@lo (source: 12345-0@lo) 00000100:00100000:3.0:1776178543.457211:0:3352:0:(client.c:1733:ptlrpc_send_new_req()) Sending RPC req@00000000606788fe pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:lustre-MDT0000-lwp-OST0001_UUID:3352:1862456998096192:0@lo:38:kworker/u8:1.0 00000100:00100000:3.0:1776178543.457247:0:3352:0:(events.c:361:request_in_callback()) peer: 12345-0@lo (source: 12345-0@lo) 00000100:00100000:3.0:1776178543.457264:0:3352:0:(client.c:1733:ptlrpc_send_new_req()) Sending RPC req@000000001c9b18be pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:lustre-MDT0001-mdtlov_UUID:3352:1862456998096320:0@lo:38:kworker/u8:1.0 00000100:00100000:3.0:1776178543.457289:0:3352:0:(events.c:361:request_in_callback()) peer: 12345-0@lo (source: 12345-0@lo) 00000100:00100000:3.0:1776178543.457305:0:3352:0:(client.c:1733:ptlrpc_send_new_req()) Sending RPC req@0000000020763766 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:lustre-MDT0000-lwp-MDT0001_UUID:3352:1862456998096512:0@lo:38:kworker/u8:1.0 00000100:00100000:3.0:1776178543.457328:0:3352:0:(events.c:361:request_in_callback()) peer: 12345-0@lo (source: 12345-0@lo) 00000100:00100000:3.0:1776178543.457357:0:3353:0:(client.c:1733:ptlrpc_send_new_req()) Sending RPC req@0000000056a48d5b pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_00:lustre-MDT0001-lwp-OST0001_UUID:3353:1862456998096256:0@lo:400:kworker/u8:1.0 00000100:00100000:3.0:1776178543.457383:0:3353:0:(events.c:361:request_in_callback()) peer: 12345-0@lo (source: 12345-0@lo) 00000100:00100000:3.0:1776178543.457401:0:3353:0:(ptlrpcd.c:415:ptlrpcd_check()) transfer 1 async RPCs [1->0] 00000100:00100000:3.0:1776178543.457406:0:3353:0:(client.c:1733:ptlrpc_send_new_req()) Sending RPC req@00000000c7e83057 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_00:lustre-MDT0001-mdtlov_UUID:3353:1862456998096384:0@lo:400:kworker/u8:1.0 00000100:00100000:3.0:1776178543.457430:0:3353:0:(events.c:361:request_in_callback()) peer: 12345-0@lo (source: 12345-0@lo) 00000100:00100000:3.0:1776178543.457474:0:20762:0:(service.c:2152:ptlrpc_server_handle_req_in()) got req x1862456998096384 00000100:00100000:3.0:1776178543.457517:0:20762:0:(nrs_fifo.c:179:nrs_fifo_req_get()) NRS start fifo request from 12345-0@lo, seq: 778 00000100:00100000:3.0:1776178543.457521:0:20762:0:(service.c:2324:ptlrpc_server_handle_request()) Handling RPC req@000000008516f059 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_013:lustre-MDT0001-mdtlov_UUID+5:3353:x1862456998096384:12345-0@lo:400:kworker/u8:1.0 00000100:00100000:3.0:1776178543.457556:0:20762:0:(service.c:2372:ptlrpc_server_handle_request()) Handled RPC req@000000008516f059 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_013:lustre-MDT0001-mdtlov_UUID+5:3353:x1862456998096384:12345-0@lo:400:kworker/u8:1.0 Request processed in 34us (125us total) trans 0 rc 0/0 00000100:00100000:3.0:1776178543.457562:0:20762:0:(nrs_fifo.c:241:nrs_fifo_req_stop()) NRS stop fifo request from 12345-0@lo, seq: 778 00000100:00100000:3.0:1776178543.457584:0:6256:0:(service.c:2152:ptlrpc_server_handle_req_in()) got req x1862456998096128 00000100:00100000:3.0:1776178543.457604:0:6256:0:(service.c:2152:ptlrpc_server_handle_req_in()) got req x1862456998096064 00000100:00100000:3.0:1776178543.457612:0:6256:0:(service.c:2152:ptlrpc_server_handle_req_in()) got req x1862456998096192 00000100:00100000:3.0:1776178543.457619:0:6256:0:(service.c:2152:ptlrpc_server_handle_req_in()) got req x1862456998096512 00000100:00100000:3.0:1776178543.457626:0:6256:0:(service.c:2152:ptlrpc_server_handle_req_in()) got req x1862456998096256 00000100:00100000:3.0:1776178543.457636:0:6256:0:(nrs_fifo.c:179:nrs_fifo_req_get()) NRS start fifo request from 12345-0@lo, seq: 1127 00000100:00100000:3.0:1776178543.457640:0:6256:0:(service.c:2324:ptlrpc_server_handle_request()) Handling RPC req@000000009d5112f6 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_001:lustre-MDT0001-lwp-OST0000_UUID+5:3356:x1862456998096128:12345-0@lo:400:kworker/u8:1.0 00000100:00100000:3.0:1776178543.457671:0:6256:0:(service.c:2372:ptlrpc_server_handle_request()) Handled RPC req@000000009d5112f6 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_001:lustre-MDT0001-lwp-OST0000_UUID+5:3356:x1862456998096128:12345-0@lo:400:kworker/u8:1.0 Request processed in 30us (891us total) trans 0 rc 0/0 00000100:00100000:3.0:1776178543.457677:0:6256:0:(nrs_fifo.c:241:nrs_fifo_req_stop()) NRS stop fifo request from 12345-0@lo, seq: 1127 00000100:00100000:3.0:1776178543.457688:0:6256:0:(nrs_fifo.c:179:nrs_fifo_req_get()) NRS start fifo request from 12345-0@lo, seq: 1128 00000100:00100000:3.0:1776178543.457691:0:6256:0:(service.c:2324:ptlrpc_server_handle_request()) Handling RPC req@0000000004c8f27c pname:cluuid+ref:pid:xid:nid:opc:job mdt00_001:lustre-MDT0001-lwp-OST0001_UUID+5:3353:x1862456998096256:12345-0@lo:400:kworker/u8:1.0 00000100:00100000:3.0:1776178543.457715:0:6256:0:(service.c:2372:ptlrpc_server_handle_request()) Handled RPC req@0000000004c8f27c pname:cluuid+ref:pid:xid:nid:opc:job mdt00_001:lustre-MDT0001-lwp-OST0001_UUID+5:3353:x1862456998096256:12345-0@lo:400:kworker/u8:1.0 Request processed in 23us (333us total) trans 0 rc 0/0 00000100:00100000:3.0:1776178543.457721:0:6256:0:(nrs_fifo.c:241:nrs_fifo_req_stop()) NRS stop fifo request from 12345-0@lo, seq: 1128 00000100:00100000:3.0:1776178543.457733:0:6256:0:(nrs_fifo.c:179:nrs_fifo_req_get()) NRS start fifo request from 12345-0@lo, seq: 6889 00000100:00100000:3.0:1776178543.457737:0:6256:0:(service.c:2324:ptlrpc_server_handle_request()) Handling RPC req@000000009d0d1846 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_001:0+-99:3352:x1862456998096064:12345-0@lo:38:kworker/u8:1.0 00010000:02000000:3.0:1776178543.457750:0:6256:0:(ldlm_lib.c:1139:target_handle_connect()) lustre-MDT0000: Not available for connect from 0@lo (stopping) 00000100:00100000:3.0:1776178543.457784:0:6256:0:(service.c:2372:ptlrpc_server_handle_request()) Handled RPC req@000000009d0d1846 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_001:0+-99:3352:x1862456998096064:12345-0@lo:38:kworker/u8:1.0 Request processed in 46us (606us total) trans 0 rc -19/-19 00000100:00100000:3.0:1776178543.457791:0:6256:0:(nrs_fifo.c:241:nrs_fifo_req_stop()) NRS stop fifo request from 12345-0@lo, seq: 6889 00000100:00100000:3.0:1776178543.457802:0:6256:0:(nrs_fifo.c:179:nrs_fifo_req_get()) NRS start fifo request from 12345-0@lo, seq: 6890 00000100:00100000:3.0:1776178543.457806:0:6256:0:(service.c:2324:ptlrpc_server_handle_request()) Handling RPC req@00000000a9c256f6 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_001:0+-99:3352:x1862456998096192:12345-0@lo:38:kworker/u8:1.0 00010000:02000000:3.0:1776178543.457813:0:6256:0:(ldlm_lib.c:1139:target_handle_connect()) lustre-MDT0000: Not available for connect from 0@lo (stopping) 00000100:00100000:1.0:1776178543.457818:0:3356:0:(client.c:2215:ptlrpc_check_set()) Completed RPC req@000000009b04a931 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_03:ac61f839-0e8e-4cac-9c59-08bcfbbad330:3356:1862456998096000:0@lo:400:kworker/u8:1.0 00000100:00100000:3.0:1776178543.457847:0:6256:0:(service.c:2372:ptlrpc_server_handle_request()) Handled RPC req@00000000a9c256f6 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_001:0+-99:3352:x1862456998096192:12345-0@lo:38:kworker/u8:1.0 Request processed in 39us (598us total) trans 0 rc -19/-19 00000100:00100000:3.0:1776178543.457854:0:6256:0:(nrs_fifo.c:241:nrs_fifo_req_stop()) NRS stop fifo request from 12345-0@lo, seq: 6890 00000100:00100000:1.0:1776178543.457859:0:3356:0:(client.c:2215:ptlrpc_check_set()) Completed RPC req@00000000949e3fea pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_03:lustre-MDT0001-lwp-OST0000_UUID:3356:1862456998096128:0@lo:400:kworker/u8:1.0 00000100:00100000:3.0:1776178543.457868:0:6256:0:(nrs_fifo.c:179:nrs_fifo_req_get()) NRS start fifo request from 12345-0@lo, seq: 6891 00000100:00100000:3.0:1776178543.457873:0:6256:0:(service.c:2324:ptlrpc_server_handle_request()) Handling RPC req@000000008584c293 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_001:0+-99:3352:x1862456998096512:12345-0@lo:38:kworker/u8:1.0 00010000:02000000:3.0:1776178543.457881:0:6256:0:(ldlm_lib.c:1139:target_handle_connect()) lustre-MDT0000: Not available for connect from 0@lo (stopping) 00000100:00100000:3.0:1776178543.457905:0:6256:0:(service.c:2372:ptlrpc_server_handle_request()) Handled RPC req@000000008584c293 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_001:0+-99:3352:x1862456998096512:12345-0@lo:38:kworker/u8:1.0 Request processed in 31us (576us total) trans 0 rc -19/-19 00000100:00100000:3.0:1776178543.457913:0:6256:0:(nrs_fifo.c:241:nrs_fifo_req_stop()) NRS stop fifo request from 12345-0@lo, seq: 6891 00000100:00100000:1.0:1776178543.457916:0:14128:0:(service.c:2152:ptlrpc_server_handle_req_in()) got req x1862456998096320 00000100:00100000:3.0:1776178543.457942:0:3353:0:(client.c:2215:ptlrpc_check_set()) Completed RPC req@00000000c7e83057 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_00:lustre-MDT0001-mdtlov_UUID:3353:1862456998096384:0@lo:400:kworker/u8:1.0 00000100:00100000:1.0:1776178543.457944:0:14128:0:(nrs_fifo.c:179:nrs_fifo_req_get()) NRS start fifo request from 12345-0@lo, seq: 774 00000100:00100000:1.0:1776178543.457953:0:14128:0:(service.c:2324:ptlrpc_server_handle_request()) Handling RPC req@00000000bea25c66 pname:cluuid+ref:pid:xid:nid:opc:job mdt_out00_002:0+-99:3352:x1862456998096320:12345-0@lo:38:kworker/u8:1.0 00000100:00100000:3.0:1776178543.457957:0:3353:0:(client.c:2215:ptlrpc_check_set()) Completed RPC req@0000000056a48d5b pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_00:lustre-MDT0001-lwp-OST0001_UUID:3353:1862456998096256:0@lo:400:kworker/u8:1.0 00010000:02000000:1.0:1776178543.457965:0:14128:0:(ldlm_lib.c:1139:target_handle_connect()) lustre-MDT0000: Not available for connect from 0@lo (stopping) 00000100:00100000:1.0:1776178543.458000:0:14128:0:(service.c:2372:ptlrpc_server_handle_request()) Handled RPC req@00000000bea25c66 pname:cluuid+ref:pid:xid:nid:opc:job mdt_out00_002:0+-99:3352:x1862456998096320:12345-0@lo:38:kworker/u8:1.0 Request processed in 47us (710us total) trans 0 rc -19/-19 00000100:00080000:3.0:1776178543.458000:0:3352:0:(import.c:86:import_set_state_nolock()) 0000000047601a98 lustre-MDT0000_UUID: changing import state from CONNECTING to DISCONN 00000100:00080000:3.0:1776178543.458007:0:3352:0:(import.c:1429:ptlrpc_connect_interpret()) recovery of lustre-MDT0000_UUID on 192.168.204.103@tcp failed (-19) 00000100:00100000:1.0:1776178543.458011:0:14128:0:(nrs_fifo.c:241:nrs_fifo_req_stop()) NRS stop fifo request from 12345-0@lo, seq: 774 00000100:00100000:3.0:1776178543.458014:0:3352:0:(client.c:2215:ptlrpc_check_set()) Completed RPC req@00000000fd32fba5 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:lustre-MDT0000-lwp-OST0000_UUID:3352:1862456998096064:0@lo:38:kworker/u8:1.0 00000100:00080000:3.0:1776178543.458028:0:3352:0:(import.c:86:import_set_state_nolock()) 00000000dfc8db13 lustre-MDT0000_UUID: changing import state from CONNECTING to DISCONN 00000100:00080000:3.0:1776178543.458030:0:3352:0:(import.c:1429:ptlrpc_connect_interpret()) recovery of lustre-MDT0000_UUID on 192.168.204.103@tcp failed (-19) 00000100:00100000:3.0:1776178543.458034:0:3352:0:(client.c:2215:ptlrpc_check_set()) Completed RPC req@00000000606788fe pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:lustre-MDT0000-lwp-OST0001_UUID:3352:1862456998096192:0@lo:38:kworker/u8:1.0 00000100:00080000:3.0:1776178543.458042:0:3352:0:(import.c:86:import_set_state_nolock()) 00000000ba4f1821 lustre-MDT0000_UUID: changing import state from CONNECTING to DISCONN 00000100:00080000:3.0:1776178543.458045:0:3352:0:(import.c:1429:ptlrpc_connect_interpret()) recovery of lustre-MDT0000_UUID on 192.168.204.103@tcp failed (-19) 00000100:00100000:3.0:1776178543.458047:0:3352:0:(client.c:2215:ptlrpc_check_set()) Completed RPC req@000000001c9b18be pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:lustre-MDT0001-mdtlov_UUID:3352:1862456998096320:0@lo:38:kworker/u8:1.0 00000100:00080000:3.0:1776178543.458053:0:3352:0:(import.c:86:import_set_state_nolock()) 00000000a7c55b1f lustre-MDT0000_UUID: changing import state from CONNECTING to DISCONN 00000100:00080000:3.0:1776178543.458056:0:3352:0:(import.c:1429:ptlrpc_connect_interpret()) recovery of lustre-MDT0000_UUID on 192.168.204.103@tcp failed (-19) 00000100:00100000:3.0:1776178543.458101:0:3352:0:(client.c:2215:ptlrpc_check_set()) Completed RPC req@0000000020763766 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:lustre-MDT0000-lwp-MDT0001_UUID:3352:1862456998096512:0@lo:38:kworker/u8:1.0 00010000:00010000:1.0:1776178544.097649:0:9225:0:(ldlm_lockd.c:2792:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:2.0:1776178544.992365:0:3355:0:(ptlrpcd.c:415:ptlrpcd_check()) transfer 1 async RPCs [3->2] 00000100:00100000:2.0:1776178544.992388:0:3355:0:(client.c:1733:ptlrpc_send_new_req()) Sending RPC req@00000000be94e5d7 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_02:lustre-MDT0001-mdtlov_UUID:3355:1862456998096576:0@lo:13:osp-pre-1-1.0 00000100:00100000:2.0:1776178544.992456:0:3355:0:(events.c:361:request_in_callback()) peer: 12345-0@lo (source: 12345-0@lo) 00000100:00100000:0.0:1776178544.992848:0:9554:0:(service.c:2152:ptlrpc_server_handle_req_in()) got req x1862456998096576 00000100:00100000:0.0:1776178544.992899:0:9554:0:(nrs_fifo.c:179:nrs_fifo_req_get()) NRS start fifo request from 12345-0@lo, seq: 621 00000100:00100000:0.0:1776178544.992905:0:9554:0:(service.c:2324:ptlrpc_server_handle_request()) Handling RPC req@00000000a13f5de6 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_create00:lustre-MDT0001-mdtlov_UUID+5:3355:x1862456998096576:12345-0@lo:13:osp-pre-1-1.0 00002000:00000024:0.0:1776178544.992932:0:9554:0:(ofd_obd.c:740:ofd_statfs()) blocks cached 229376 granted 491746368 pending 0 free 3923689472 avail 3692167168 00000100:00100000:0.0:1776178544.992979:0:9554:0:(service.c:2372:ptlrpc_server_handle_request()) Handled RPC req@00000000a13f5de6 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_create00:lustre-MDT0001-mdtlov_UUID+5:3355:x1862456998096576:12345-0@lo:13:osp-pre-1-1.0 Request processed in 74us (524us total) trans 0 rc 0/0 00000100:00100000:0.0:1776178544.992985:0:9554:0:(nrs_fifo.c:241:nrs_fifo_req_stop()) NRS stop fifo request from 12345-0@lo, seq: 621 00000100:00100000:2.0:1776178544.993237:0:3355:0:(client.c:2215:ptlrpc_check_set()) Completed RPC req@00000000be94e5d7 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_02:lustre-MDT0001-mdtlov_UUID:3355:1862456998096576:0@lo:13:osp-pre-1-1.0 00010000:00010000:3.0:1776178545.122267:0:6244:0:(ldlm_lockd.c:2792:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:3.0:1776178545.505592:0:3353:0:(client.c:1733:ptlrpc_send_new_req()) Sending RPC req@0000000014e690d4 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_00:lustre-MDT0001-mdtlov_UUID:3353:1862456998096640:0@lo:13:osp-pre-0-1.0 00000100:00100000:3.0:1776178545.505709:0:3353:0:(events.c:361:request_in_callback()) peer: 12345-0@lo (source: 12345-0@lo) 00000100:00100000:1.0:1776178545.505899:0:8231:0:(service.c:2152:ptlrpc_server_handle_req_in()) got req x1862456998096640 00000100:00100000:1.0:1776178545.505981:0:8231:0:(nrs_fifo.c:179:nrs_fifo_req_get()) NRS start fifo request from 12345-0@lo, seq: 622 00000100:00100000:1.0:1776178545.505990:0:8231:0:(service.c:2324:ptlrpc_server_handle_request()) Handling RPC req@00000000be94e5d7 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_create00:lustre-MDT0001-mdtlov_UUID+5:3353:x1862456998096640:12345-0@lo:13:osp-pre-0-1.0 00002000:00000024:1.0:1776178545.506050:0:8231:0:(ofd_obd.c:740:ofd_statfs()) blocks cached 315392 granted 484883776 pending 0 free 3923689472 avail 3692167168 00000100:00100000:1.0:1776178545.506206:0:8231:0:(service.c:2372:ptlrpc_server_handle_request()) Handled RPC req@00000000be94e5d7 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_create00:lustre-MDT0001-mdtlov_UUID+5:3353:x1862456998096640:12345-0@lo:13:osp-pre-0-1.0 Request processed in 214us (498us total) trans 0 rc 0/0 00000100:00100000:1.0:1776178545.506215:0:8231:0:(nrs_fifo.c:241:nrs_fifo_req_stop()) NRS stop fifo request from 12345-0@lo, seq: 622 00000100:00100000:3.0:1776178545.507372:0:3353:0:(client.c:2215:ptlrpc_check_set()) Completed RPC req@0000000014e690d4 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_00:lustre-MDT0001-mdtlov_UUID:3353:1862456998096640:0@lo:13:osp-pre-0-1.0 00010000:00010000:2.0:1776178546.145928:0:6245:0:(ldlm_lockd.c:2792:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:3.0:1776178546.977934:0:3346:0:(events.c:361:request_in_callback()) peer: 12345-192.168.204.3@tcp (source: 12345-192.168.204.3@tcp) 00000100:00100000:2.0:1776178546.978227:0:11281:0:(service.c:2152:ptlrpc_server_handle_req_in()) got req x1862456979130496 00000100:00100000:2.0:1776178546.978280:0:11281:0:(nrs_fifo.c:179:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.204.3@tcp, seq: 6892 00000100:00100000:2.0:1776178546.978287:0:11281:0:(service.c:2324:ptlrpc_server_handle_request()) Handling RPC req@0000000013f6d767 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_004:0+-99:2269:x1862456979130496:12345-192.168.204.3@tcp:38:kworker/u8:3.0 00000100:00100000:3.0:1776178546.978452:0:3347:0:(events.c:361:request_in_callback()) peer: 12345-192.168.204.3@tcp (source: 12345-192.168.204.3@tcp) 00000100:00100000:3.0:1776178546.978487:0:3347:0:(events.c:361:request_in_callback()) peer: 12345-192.168.204.3@tcp (source: 12345-192.168.204.3@tcp) 00000100:00100000:3.0:1776178546.978526:0:3347:0:(events.c:361:request_in_callback()) peer: 12345-192.168.204.3@tcp (source: 12345-192.168.204.3@tcp) 00000100:00100000:0.0:1776178546.978577:0:8250:0:(service.c:2152:ptlrpc_server_handle_req_in()) got req x1862456979130560 00000100:00100000:3.0:1776178546.978598:0:3347:0:(events.c:361:request_in_callback()) peer: 12345-192.168.204.3@tcp (source: 12345-192.168.204.3@tcp) 00000100:00100000:0.0:1776178546.978631:0:8250:0:(nrs_fifo.c:179:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.204.3@tcp, seq: 1129 00000100:00100000:0.0:1776178546.978647:0:8250:0:(service.c:2324:ptlrpc_server_handle_request()) Handling RPC req@00000000a1262b14 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_003:b50963bd-8eba-4432-9a54-8888d00e3fa1+6:2270:x1862456979130560:12345-192.168.204.3@tcp:400:kworker/u8:3.0 00000100:00100000:3.0:1776178546.978657:0:20140:0:(service.c:2152:ptlrpc_server_handle_req_in()) got req x1862456979130432 00000100:00100000:3.0:1776178546.978687:0:20140:0:(nrs_fifo.c:179:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.204.3@tcp, seq: 289 00000100:00100000:3.0:1776178546.978692:0:20140:0:(service.c:2324:ptlrpc_server_handle_request()) Handling RPC req@0000000048947fd8 pname:cluuid+ref:pid:xid:nid:opc:job ll_mgs_0002:5b5c66b1-1a2f-4651-8548-0c75b9a38a94+9:2270:x1862456979130432:12345-192.168.204.3@tcp:400:kworker/u8:3.0 00000100:00100000:0.0:1776178546.978726:0:8250:0:(service.c:2372:ptlrpc_server_handle_request()) Handled RPC req@00000000a1262b14 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_003:b50963bd-8eba-4432-9a54-8888d00e3fa1+6:2270:x1862456979130560:12345-192.168.204.3@tcp:400:kworker/u8:3.0 Request processed in 78us (274us total) trans 0 rc 0/0 00000100:00100000:3.0:1776178546.978729:0:20140:0:(service.c:2372:ptlrpc_server_handle_request()) Handled RPC req@0000000048947fd8 pname:cluuid+ref:pid:xid:nid:opc:job ll_mgs_0002:5b5c66b1-1a2f-4651-8548-0c75b9a38a94+9:2270:x1862456979130432:12345-192.168.204.3@tcp:400:kworker/u8:3.0 Request processed in 35us (201us total) trans 0 rc 0/0 00000100:00100000:0.0:1776178546.978740:0:8250:0:(nrs_fifo.c:241:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.204.3@tcp, seq: 1129 00000100:00100000:3.0:1776178546.978744:0:20140:0:(nrs_fifo.c:241:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.204.3@tcp, seq: 289 00000100:00100000:1.0:1776178546.978773:0:14440:0:(service.c:2152:ptlrpc_server_handle_req_in()) got req x1862456979130688 00000100:00100000:1.0:1776178546.978809:0:14440:0:(service.c:2152:ptlrpc_server_handle_req_in()) got req x1862456979130624 00000100:00100000:1.0:1776178546.978829:0:14440:0:(nrs_fifo.c:179:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.204.3@tcp, seq: 779 00000100:00100000:1.0:1776178546.978833:0:14440:0:(service.c:2324:ptlrpc_server_handle_request()) Handling RPC req@0000000046115d91 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_004:b50963bd-8eba-4432-9a54-8888d00e3fa1+9:2270:x1862456979130688:12345-192.168.204.3@tcp:400:kworker/u8:3.0 00000100:00100000:1.0:1776178546.978864:0:14440:0:(service.c:2372:ptlrpc_server_handle_request()) Handled RPC req@0000000046115d91 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_004:b50963bd-8eba-4432-9a54-8888d00e3fa1+9:2270:x1862456979130688:12345-192.168.204.3@tcp:400:kworker/u8:3.0 Request processed in 30us (377us total) trans 0 rc 0/0 00000100:00100000:1.0:1776178546.978871:0:14440:0:(nrs_fifo.c:241:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.204.3@tcp, seq: 779 00000100:00100000:1.0:1776178546.978906:0:14440:0:(nrs_fifo.c:179:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.204.3@tcp, seq: 780 00000100:00100000:1.0:1776178546.978913:0:14440:0:(service.c:2324:ptlrpc_server_handle_request()) Handling RPC req@000000008c593d89 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_004:b50963bd-8eba-4432-9a54-8888d00e3fa1+8:2270:x1862456979130624:12345-192.168.204.3@tcp:400:kworker/u8:3.0 00000100:00100000:1.0:1776178546.978953:0:14440:0:(service.c:2372:ptlrpc_server_handle_request()) Handled RPC req@000000008c593d89 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_004:b50963bd-8eba-4432-9a54-8888d00e3fa1+8:2270:x1862456979130624:12345-192.168.204.3@tcp:400:kworker/u8:3.0 Request processed in 39us (355us total) trans 0 rc 0/0 00000100:00100000:1.0:1776178546.978961:0:14440:0:(nrs_fifo.c:241:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.204.3@tcp, seq: 780 00010000:02000000:2.0:1776178546.979013:0:11281:0:(ldlm_lib.c:1139:target_handle_connect()) lustre-MDT0000: Not available for connect from 192.168.204.3@tcp (stopping) 00000100:00100000:2.0:1776178546.979047:0:11281:0:(service.c:2372:ptlrpc_server_handle_request()) Handled RPC req@0000000013f6d767 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_004:0+-99:2269:x1862456979130496:12345-192.168.204.3@tcp:38:kworker/u8:3.0 Request processed in 760us (1123us total) trans 0 rc -19/-19 00000100:00100000:2.0:1776178546.979057:0:11281:0:(nrs_fifo.c:241:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.204.3@tcp, seq: 6892 00010000:00010000:1.0:1776178547.168532:0:9224:0:(ldlm_lockd.c:2792:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00010000:00010000:1.0:1776178548.192368:0:9225:0:(ldlm_lockd.c:2792:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00080000:3.0:1776178548.576566:0:36:0:(pinger.c:251:ptlrpc_pinger_process_import()) lustre-MDT0000-lwp-OST0000_UUID->lustre-MDT0000_UUID: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00080000:3.0:1776178548.576575:0:36:0:(recover.c:57:ptlrpc_initiate_recovery()) lustre-MDT0000_UUID: starting recovery 00000100:00080000:3.0:1776178548.576579:0:36:0:(import.c:86:import_set_state_nolock()) 0000000047601a98 lustre-MDT0000_UUID: changing import state from DISCONN to CONNECTING 00000100:00080000:3.0:1776178548.576587:0:36:0:(import.c:534:import_select_connection()) lustre-MDT0000-lwp-OST0000: connect to NID 0@lo last attempt 1250 00000100:00080000:3.0:1776178548.576590:0:36:0:(import.c:581:import_select_connection()) lustre-MDT0000-lwp-OST0000: tried all connections, increasing latency to 21s 00000100:00080000:3.0:1776178548.576597:0:36:0:(import.c:615:import_select_connection()) lustre-MDT0000-lwp-OST0000: import 0000000047601a98 using connection 192.168.204.103@tcp/0@lo 00000100:00100000:3.0:1776178548.576626:0:36:0:(import.c:807:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5) req@000000002973935f x1862456998096768/t0(0) o38->lustre-MDT0000-lwp-OST0000@0@lo:12/10 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' 00000100:00080000:3.0:1776178548.576679:0:36:0:(pinger.c:251:ptlrpc_pinger_process_import()) lustre-MDT0000-lwp-OST0001_UUID->lustre-MDT0000_UUID: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00080000:3.0:1776178548.576682:0:36:0:(recover.c:57:ptlrpc_initiate_recovery()) lustre-MDT0000_UUID: starting recovery 00000100:00080000:3.0:1776178548.576683:0:36:0:(import.c:86:import_set_state_nolock()) 00000000dfc8db13 lustre-MDT0000_UUID: changing import state from DISCONN to CONNECTING 00000100:00080000:3.0:1776178548.576686:0:36:0:(import.c:534:import_select_connection()) lustre-MDT0000-lwp-OST0001: connect to NID 0@lo last attempt 1250 00000100:00080000:3.0:1776178548.576689:0:36:0:(import.c:581:import_select_connection()) lustre-MDT0000-lwp-OST0001: tried all connections, increasing latency to 21s 00000100:00080000:3.0:1776178548.576692:0:36:0:(import.c:615:import_select_connection()) lustre-MDT0000-lwp-OST0001: import 00000000dfc8db13 using connection 192.168.204.103@tcp/0@lo 00000100:00100000:3.0:1776178548.576702:0:36:0:(import.c:807:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5) req@000000007d5bffd0 x1862456998096896/t0(0) o38->lustre-MDT0000-lwp-OST0001@0@lo:12/10 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' 00000100:00080000:3.0:1776178548.576716:0:36:0:(pinger.c:251:ptlrpc_pinger_process_import()) lustre-MDT0001-mdtlov_UUID->lustre-MDT0000_UUID: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00080000:3.0:1776178548.576719:0:36:0:(recover.c:57:ptlrpc_initiate_recovery()) lustre-MDT0000_UUID: starting recovery 00000100:00080000:3.0:1776178548.576720:0:36:0:(import.c:86:import_set_state_nolock()) 00000000ba4f1821 lustre-MDT0000_UUID: changing import state from DISCONN to CONNECTING 00000100:00080000:3.0:1776178548.576724:0:36:0:(import.c:534:import_select_connection()) lustre-MDT0000-osp-MDT0001: connect to NID 0@lo last attempt 1250 00000100:00080000:3.0:1776178548.576725:0:36:0:(import.c:581:import_select_connection()) lustre-MDT0000-osp-MDT0001: tried all connections, increasing latency to 21s 00000100:00080000:3.0:1776178548.576728:0:36:0:(import.c:615:import_select_connection()) lustre-MDT0000-osp-MDT0001: import 00000000ba4f1821 using connection 192.168.204.103@tcp/0@lo 00000100:00100000:3.0:1776178548.576739:0:36:0:(import.c:807:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5) req@0000000017d54e92 x1862456998097024/t0(0) o38->lustre-MDT0000-osp-MDT0001@0@lo:24/4 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' 00000100:00080000:3.0:1776178548.576760:0:36:0:(pinger.c:251:ptlrpc_pinger_process_import()) lustre-MDT0000-lwp-MDT0001_UUID->lustre-MDT0000_UUID: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00080000:3.0:1776178548.576763:0:36:0:(recover.c:57:ptlrpc_initiate_recovery()) lustre-MDT0000_UUID: starting recovery 00000100:00080000:3.0:1776178548.576764:0:36:0:(import.c:86:import_set_state_nolock()) 00000000a7c55b1f lustre-MDT0000_UUID: changing import state from DISCONN to CONNECTING 00000100:00080000:3.0:1776178548.576767:0:36:0:(import.c:534:import_select_connection()) lustre-MDT0000-lwp-MDT0001: connect to NID 0@lo last attempt 1250 00000100:00080000:3.0:1776178548.576769:0:36:0:(import.c:581:import_select_connection()) lustre-MDT0000-lwp-MDT0001: tried all connections, increasing latency to 21s 00000100:00080000:3.0:1776178548.576772:0:36:0:(import.c:615:import_select_connection()) lustre-MDT0000-lwp-MDT0001: import 00000000a7c55b1f using connection 192.168.204.103@tcp/0@lo 00000100:00100000:3.0:1776178548.576780:0:36:0:(import.c:807:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5) req@00000000dd52d0d0 x1862456998097216/t0(0) o38->lustre-MDT0000-lwp-MDT0001@0@lo:12/10 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' 00000100:00100000:3.0:1776178548.576807:0:3355:0:(client.c:1733:ptlrpc_send_new_req()) Sending RPC req@00000000812a7ebe pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_02:lustre-MDT0001-lwp-OST0000_UUID:3355:1862456998096832:0@lo:400:kworker/u8:1.0 00000100:00100000:3.0:1776178548.576886:0:3355:0:(events.c:361:request_in_callback()) peer: 12345-0@lo (source: 12345-0@lo) 00000100:00100000:3.0:1776178548.576927:0:3355:0:(ptlrpcd.c:415:ptlrpcd_check()) transfer 1 async RPCs [3->2] 00000100:00100000:3.0:1776178548.576933:0:3355:0:(client.c:1733:ptlrpc_send_new_req()) Sending RPC req@00000000fed3e037 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_02:lustre-MDT0001-lwp-OST0001_UUID:3355:1862456998096960:0@lo:400:kworker/u8:1.0 00000100:00100000:3.0:1776178548.576969:0:3355:0:(events.c:361:request_in_callback()) peer: 12345-0@lo (source: 12345-0@lo) 00000100:00100000:3.0:1776178548.577010:0:11281:0:(service.c:2152:ptlrpc_server_handle_req_in()) got req x1862456998096832 00000100:00100000:3.0:1776178548.577056:0:11281:0:(service.c:2152:ptlrpc_server_handle_req_in()) got req x1862456998096960 00000100:00100000:3.0:1776178548.577110:0:11281:0:(nrs_fifo.c:179:nrs_fifo_req_get()) NRS start fifo request from 12345-0@lo, seq: 1130 00000100:00100000:3.0:1776178548.577115:0:11281:0:(service.c:2324:ptlrpc_server_handle_request()) Handling RPC req@0000000087a31eb2 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_004:lustre-MDT0001-lwp-OST0000_UUID+5:3355:x1862456998096832:12345-0@lo:400:kworker/u8:1.0 00000100:00100000:3.0:1776178548.577179:0:11281:0:(service.c:2372:ptlrpc_server_handle_request()) Handled RPC req@0000000087a31eb2 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_004:lustre-MDT0001-lwp-OST0000_UUID+5:3355:x1862456998096832:12345-0@lo:400:kworker/u8:1.0 Request processed in 63us (294us total) trans 0 rc 0/0 00000100:00100000:3.0:1776178548.577187:0:11281:0:(nrs_fifo.c:241:nrs_fifo_req_stop()) NRS stop fifo request from 12345-0@lo, seq: 1130 00000100:00100000:3.0:1776178548.577205:0:11281:0:(nrs_fifo.c:179:nrs_fifo_req_get()) NRS start fifo request from 12345-0@lo, seq: 1131 00000100:00100000:3.0:1776178548.577208:0:11281:0:(service.c:2324:ptlrpc_server_handle_request()) Handling RPC req@000000004e1a7b1b pname:cluuid+ref:pid:xid:nid:opc:job mdt00_004:lustre-MDT0001-lwp-OST0001_UUID+5:3355:x1862456998096960:12345-0@lo:400:kworker/u8:1.0 00000100:00100000:3.0:1776178548.577234:0:11281:0:(service.c:2372:ptlrpc_server_handle_request()) Handled RPC req@000000004e1a7b1b pname:cluuid+ref:pid:xid:nid:opc:job mdt00_004:lustre-MDT0001-lwp-OST0001_UUID+5:3355:x1862456998096960:12345-0@lo:400:kworker/u8:1.0 Request processed in 24us (265us total) trans 0 rc 0/0 00000100:00100000:3.0:1776178548.577239:0:11281:0:(nrs_fifo.c:241:nrs_fifo_req_stop()) NRS stop fifo request from 12345-0@lo, seq: 1131 00000100:00100000:3.0:1776178548.577279:0:3355:0:(client.c:2215:ptlrpc_check_set()) Completed RPC req@00000000fed3e037 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_02:lustre-MDT0001-lwp-OST0001_UUID:3355:1862456998096960:0@lo:400:kworker/u8:1.0 00000100:00100000:3.0:1776178548.577288:0:3355:0:(client.c:2215:ptlrpc_check_set()) Completed RPC req@00000000812a7ebe pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_02:lustre-MDT0001-lwp-OST0000_UUID:3355:1862456998096832:0@lo:400:kworker/u8:1.0 00000100:00100000:0.0:1776178548.577370:0:3353:0:(client.c:1733:ptlrpc_send_new_req()) Sending RPC req@00000000239d7b9a pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_00:lustre-MDT0001-mdtlov_UUID:3353:1862456998097088:0@lo:400:kworker/u8:1.0 00000100:00100000:0.0:1776178548.577401:0:3353:0:(events.c:361:request_in_callback()) peer: 12345-0@lo (source: 12345-0@lo) 00000100:00100000:0.0:1776178548.577423:0:3353:0:(ptlrpcd.c:415:ptlrpcd_check()) transfer 2 async RPCs [1->0] 00000100:00100000:0.0:1776178548.577428:0:3353:0:(client.c:1733:ptlrpc_send_new_req()) Sending RPC req@000000003316d46e pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_00:ac61f839-0e8e-4cac-9c59-08bcfbbad330:3353:1862456998096704:0@lo:400:kworker/u8:1.0 00000100:00100000:0.0:1776178548.577452:0:3353:0:(events.c:361:request_in_callback()) peer: 12345-0@lo (source: 12345-0@lo) 00000100:00100000:0.0:1776178548.577466:0:3353:0:(client.c:1733:ptlrpc_send_new_req()) Sending RPC req@00000000f62fe06f pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_00:lustre-MDT0001-mdtlov_UUID:3353:1862456998097152:0@lo:400:kworker/u8:1.0 00000100:00100000:0.0:1776178548.577485:0:3353:0:(events.c:361:request_in_callback()) peer: 12345-0@lo (source: 12345-0@lo) 00000100:00100000:0.0:1776178548.577509:0:20762:0:(service.c:2152:ptlrpc_server_handle_req_in()) got req x1862456998097088 00000100:00100000:0.0:1776178548.577533:0:20762:0:(service.c:2152:ptlrpc_server_handle_req_in()) got req x1862456998097152 00000100:00100000:0.0:1776178548.577550:0:20762:0:(nrs_fifo.c:179:nrs_fifo_req_get()) NRS start fifo request from 12345-0@lo, seq: 781 00000100:00100000:0.0:1776178548.577554:0:20762:0:(service.c:2324:ptlrpc_server_handle_request()) Handling RPC req@0000000001246e58 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_013:lustre-MDT0001-mdtlov_UUID+5:3353:x1862456998097088:12345-0@lo:400:kworker/u8:1.0 00000100:00100000:0.0:1776178548.577586:0:20762:0:(service.c:2372:ptlrpc_server_handle_request()) Handled RPC req@0000000001246e58 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_013:lustre-MDT0001-mdtlov_UUID+5:3353:x1862456998097088:12345-0@lo:400:kworker/u8:1.0 Request processed in 31us (185us total) trans 0 rc 0/0 00000100:00100000:0.0:1776178548.577592:0:20762:0:(nrs_fifo.c:241:nrs_fifo_req_stop()) NRS stop fifo request from 12345-0@lo, seq: 781 00000100:00100000:0.0:1776178548.577603:0:20762:0:(nrs_fifo.c:179:nrs_fifo_req_get()) NRS start fifo request from 12345-0@lo, seq: 782 00000100:00100000:0.0:1776178548.577606:0:20762:0:(service.c:2324:ptlrpc_server_handle_request()) Handling RPC req@00000000eec4b6d0 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_013:lustre-MDT0001-mdtlov_UUID+5:3353:x1862456998097152:12345-0@lo:400:kworker/u8:1.0 00000100:00100000:0.0:1776178548.577629:0:20762:0:(service.c:2372:ptlrpc_server_handle_request()) Handled RPC req@00000000eec4b6d0 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_013:lustre-MDT0001-mdtlov_UUID+5:3353:x1862456998097152:12345-0@lo:400:kworker/u8:1.0 Request processed in 22us (144us total) trans 0 rc 0/0 00000100:00100000:0.0:1776178548.577634:0:20762:0:(nrs_fifo.c:241:nrs_fifo_req_stop()) NRS stop fifo request from 12345-0@lo, seq: 782 00000100:00100000:0.0:1776178548.577657:0:20137:0:(service.c:2152:ptlrpc_server_handle_req_in()) got req x1862456998096704 00000100:00100000:0.0:1776178548.577680:0:20137:0:(nrs_fifo.c:179:nrs_fifo_req_get()) NRS start fifo request from 12345-0@lo, seq: 290 00000100:00100000:0.0:1776178548.577683:0:20137:0:(service.c:2324:ptlrpc_server_handle_request()) Handling RPC req@00000000ed9d6990 pname:cluuid+ref:pid:xid:nid:opc:job ll_mgs_0000:ac61f839-0e8e-4cac-9c59-08bcfbbad330+18:3353:x1862456998096704:12345-0@lo:400:kworker/u8:1.0 00000100:00100000:0.0:1776178548.577708:0:20137:0:(service.c:2372:ptlrpc_server_handle_request()) Handled RPC req@00000000ed9d6990 pname:cluuid+ref:pid:xid:nid:opc:job ll_mgs_0000:ac61f839-0e8e-4cac-9c59-08bcfbbad330+18:3353:x1862456998096704:12345-0@lo:400:kworker/u8:1.0 Request processed in 24us (255us total) trans 0 rc 0/0 00000100:00100000:0.0:1776178548.577713:0:20137:0:(nrs_fifo.c:241:nrs_fifo_req_stop()) NRS stop fifo request from 12345-0@lo, seq: 290 00000100:00100000:0.0:1776178548.577745:0:3353:0:(client.c:2215:ptlrpc_check_set()) Completed RPC req@000000003316d46e pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_00:ac61f839-0e8e-4cac-9c59-08bcfbbad330:3353:1862456998096704:0@lo:400:kworker/u8:1.0 00000100:00100000:0.0:1776178548.577753:0:3353:0:(client.c:2215:ptlrpc_check_set()) Completed RPC req@00000000f62fe06f pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_00:lustre-MDT0001-mdtlov_UUID:3353:1862456998097152:0@lo:400:kworker/u8:1.0 00000100:00100000:0.0:1776178548.577761:0:3353:0:(client.c:2215:ptlrpc_check_set()) Completed RPC req@00000000239d7b9a pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_00:lustre-MDT0001-mdtlov_UUID:3353:1862456998097088:0@lo:400:kworker/u8:1.0 00000100:00100000:1.0:1776178548.578091:0:3352:0:(client.c:1733:ptlrpc_send_new_req()) Sending RPC req@000000002973935f pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:lustre-MDT0000-lwp-OST0000_UUID:3352:1862456998096768:0@lo:38:kworker/u8:1.0 00000100:00100000:1.0:1776178548.578123:0:3352:0:(events.c:361:request_in_callback()) peer: 12345-0@lo (source: 12345-0@lo) 00000100:00100000:1.0:1776178548.578146:0:3352:0:(client.c:1733:ptlrpc_send_new_req()) Sending RPC req@000000007d5bffd0 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:lustre-MDT0000-lwp-OST0001_UUID:3352:1862456998096896:0@lo:38:kworker/u8:1.0 00000100:00100000:1.0:1776178548.578165:0:3352:0:(events.c:361:request_in_callback()) peer: 12345-0@lo (source: 12345-0@lo) 00000100:00100000:1.0:1776178548.578178:0:3352:0:(client.c:1733:ptlrpc_send_new_req()) Sending RPC req@0000000017d54e92 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:lustre-MDT0001-mdtlov_UUID:3352:1862456998097024:0@lo:38:kworker/u8:1.0 00000100:00100000:1.0:1776178548.578199:0:3352:0:(events.c:361:request_in_callback()) peer: 12345-0@lo (source: 12345-0@lo) 00000100:00100000:1.0:1776178548.578213:0:3352:0:(client.c:1733:ptlrpc_send_new_req()) Sending RPC req@00000000dd52d0d0 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:lustre-MDT0000-lwp-MDT0001_UUID:3352:1862456998097216:0@lo:38:kworker/u8:1.0 00000100:00100000:1.0:1776178548.578232:0:3352:0:(events.c:361:request_in_callback()) peer: 12345-0@lo (source: 12345-0@lo) 00000100:00100000:1.0:1776178548.578262:0:6255:0:(service.c:2152:ptlrpc_server_handle_req_in()) got req x1862456998096768 00000100:00100000:1.0:1776178548.578281:0:6255:0:(service.c:2152:ptlrpc_server_handle_req_in()) got req x1862456998096896 00000100:00100000:1.0:1776178548.578288:0:6255:0:(service.c:2152:ptlrpc_server_handle_req_in()) got req x1862456998097216 00000100:00100000:1.0:1776178548.578297:0:6255:0:(nrs_fifo.c:179:nrs_fifo_req_get()) NRS start fifo request from 12345-0@lo, seq: 6893 00000100:00100000:1.0:1776178548.578300:0:6255:0:(service.c:2324:ptlrpc_server_handle_request()) Handling RPC req@00000000dd1281fe pname:cluuid+ref:pid:xid:nid:opc:job mdt00_000:0+-99:3352:x1862456998096768:12345-0@lo:38:kworker/u8:1.0 00010000:02000000:1.0:1776178548.578312:0:6255:0:(ldlm_lib.c:1139:target_handle_connect()) lustre-MDT0000: Not available for connect from 0@lo (stopping) 00000100:00100000:1.0:1776178548.578345:0:6255:0:(service.c:2372:ptlrpc_server_handle_request()) Handled RPC req@00000000dd1281fe pname:cluuid+ref:pid:xid:nid:opc:job mdt00_000:0+-99:3352:x1862456998096768:12345-0@lo:38:kworker/u8:1.0 Request processed in 44us (222us total) trans 0 rc -19/-19 00000100:00100000:1.0:1776178548.578350:0:6255:0:(nrs_fifo.c:241:nrs_fifo_req_stop()) NRS stop fifo request from 12345-0@lo, seq: 6893 00000100:00100000:1.0:1776178548.578360:0:6255:0:(nrs_fifo.c:179:nrs_fifo_req_get()) NRS start fifo request from 12345-0@lo, seq: 6894 00000100:00100000:1.0:1776178548.578362:0:6255:0:(service.c:2324:ptlrpc_server_handle_request()) Handling RPC req@00000000b5702a7a pname:cluuid+ref:pid:xid:nid:opc:job mdt00_000:0+-99:3352:x1862456998096896:12345-0@lo:38:kworker/u8:1.0 00010000:02000000:1.0:1776178548.578367:0:6255:0:(ldlm_lib.c:1139:target_handle_connect()) lustre-MDT0000: Not available for connect from 0@lo (stopping) 00000100:00100000:1.0:1776178548.578384:0:6255:0:(service.c:2372:ptlrpc_server_handle_request()) Handled RPC req@00000000b5702a7a pname:cluuid+ref:pid:xid:nid:opc:job mdt00_000:0+-99:3352:x1862456998096896:12345-0@lo:38:kworker/u8:1.0 Request processed in 21us (219us total) trans 0 rc -19/-19 00000100:00100000:1.0:1776178548.578388:0:6255:0:(nrs_fifo.c:241:nrs_fifo_req_stop()) NRS stop fifo request from 12345-0@lo, seq: 6894 00000100:00100000:1.0:1776178548.578397:0:6255:0:(nrs_fifo.c:179:nrs_fifo_req_get()) NRS start fifo request from 12345-0@lo, seq: 6895 00000100:00100000:1.0:1776178548.578400:0:6255:0:(service.c:2324:ptlrpc_server_handle_request()) Handling RPC req@00000000ecb64602 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_000:0+-99:3352:x1862456998097216:12345-0@lo:38:kworker/u8:1.0 00010000:02000000:1.0:1776178548.578406:0:6255:0:(ldlm_lib.c:1139:target_handle_connect()) lustre-MDT0000: Not available for connect from 0@lo (stopping) 00000100:00100000:1.0:1776178548.578422:0:6255:0:(service.c:2372:ptlrpc_server_handle_request()) Handled RPC req@00000000ecb64602 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_000:0+-99:3352:x1862456998097216:12345-0@lo:38:kworker/u8:1.0 Request processed in 21us (191us total) trans 0 rc -19/-19 00000100:00100000:1.0:1776178548.578427:0:6255:0:(nrs_fifo.c:241:nrs_fifo_req_stop()) NRS stop fifo request from 12345-0@lo, seq: 6895 00000100:00100000:1.0:1776178548.578443:0:14128:0:(service.c:2152:ptlrpc_server_handle_req_in()) got req x1862456998097024 00000100:00100000:1.0:1776178548.578460:0:14128:0:(nrs_fifo.c:179:nrs_fifo_req_get()) NRS start fifo request from 12345-0@lo, seq: 775 00000100:00100000:1.0:1776178548.578463:0:14128:0:(service.c:2324:ptlrpc_server_handle_request()) Handling RPC req@00000000a59bfc89 pname:cluuid+ref:pid:xid:nid:opc:job mdt_out00_002:0+-99:3352:x1862456998097024:12345-0@lo:38:kworker/u8:1.0 00010000:02000000:1.0:1776178548.578468:0:14128:0:(ldlm_lib.c:1139:target_handle_connect()) lustre-MDT0000: Not available for connect from 0@lo (stopping) 00000100:00100000:1.0:1776178548.578488:0:14128:0:(service.c:2372:ptlrpc_server_handle_request()) Handled RPC req@00000000a59bfc89 pname:cluuid+ref:pid:xid:nid:opc:job mdt_out00_002:0+-99:3352:x1862456998097024:12345-0@lo:38:kworker/u8:1.0 Request processed in 25us (289us total) trans 0 rc -19/-19 00000100:00100000:1.0:1776178548.578493:0:14128:0:(nrs_fifo.c:241:nrs_fifo_req_stop()) NRS stop fifo request from 12345-0@lo, seq: 775 00000100:00080000:1.0:1776178548.578542:0:3352:0:(import.c:86:import_set_state_nolock()) 0000000047601a98 lustre-MDT0000_UUID: changing import state from CONNECTING to DISCONN 00000100:00080000:1.0:1776178548.578546:0:3352:0:(import.c:1429:ptlrpc_connect_interpret()) recovery of lustre-MDT0000_UUID on 192.168.204.103@tcp failed (-19) 00000100:00100000:1.0:1776178548.578549:0:3352:0:(client.c:2215:ptlrpc_check_set()) Completed RPC req@000000002973935f pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:lustre-MDT0000-lwp-OST0000_UUID:3352:1862456998096768:0@lo:38:kworker/u8:1.0 00000100:00080000:1.0:1776178548.578556:0:3352:0:(import.c:86:import_set_state_nolock()) 00000000dfc8db13 lustre-MDT0000_UUID: changing import state from CONNECTING to DISCONN 00000100:00080000:1.0:1776178548.578558:0:3352:0:(import.c:1429:ptlrpc_connect_interpret()) recovery of lustre-MDT0000_UUID on 192.168.204.103@tcp failed (-19) 00000100:00100000:1.0:1776178548.578560:0:3352:0:(client.c:2215:ptlrpc_check_set()) Completed RPC req@000000007d5bffd0 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:lustre-MDT0000-lwp-OST0001_UUID:3352:1862456998096896:0@lo:38:kworker/u8:1.0 00000100:00080000:1.0:1776178548.578566:0:3352:0:(import.c:86:import_set_state_nolock()) 00000000ba4f1821 lustre-MDT0000_UUID: changing import state from CONNECTING to DISCONN 00000100:00080000:1.0:1776178548.578569:0:3352:0:(import.c:1429:ptlrpc_connect_interpret()) recovery of lustre-MDT0000_UUID on 192.168.204.103@tcp failed (-19) 00000100:00100000:1.0:1776178548.578571:0:3352:0:(client.c:2215:ptlrpc_check_set()) Completed RPC req@0000000017d54e92 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:lustre-MDT0001-mdtlov_UUID:3352:1862456998097024:0@lo:38:kworker/u8:1.0 00000100:00080000:1.0:1776178548.578577:0:3352:0:(import.c:86:import_set_state_nolock()) 00000000a7c55b1f lustre-MDT0000_UUID: changing import state from CONNECTING to DISCONN 00000100:00080000:1.0:1776178548.578579:0:3352:0:(import.c:1429:ptlrpc_connect_interpret()) recovery of lustre-MDT0000_UUID on 192.168.204.103@tcp failed (-19) 00000100:00100000:1.0:1776178548.578581:0:3352:0:(client.c:2215:ptlrpc_check_set()) Completed RPC req@00000000dd52d0d0 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:lustre-MDT0000-lwp-MDT0001_UUID:3352:1862456998097216:0@lo:38:kworker/u8:1.0 00010000:00010000:3.0:1776178549.231331:0:6244:0:(ldlm_lockd.c:2792:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:1.0:1776178550.113232:0:3356:0:(ptlrpcd.c:415:ptlrpcd_check()) transfer 1 async RPCs [2->3] 00000100:00100000:1.0:1776178550.113267:0:3356:0:(client.c:1733:ptlrpc_send_new_req()) Sending RPC req@0000000018bd7474 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_03:lustre-MDT0001-mdtlov_UUID:3356:1862456998097280:0@lo:13:osp-pre-1-1.0 00000100:00100000:1.0:1776178550.113362:0:3356:0:(events.c:361:request_in_callback()) peer: 12345-0@lo (source: 12345-0@lo) 00000100:00100000:2.0:1776178550.113571:0:9554:0:(service.c:2152:ptlrpc_server_handle_req_in()) got req x1862456998097280 00000100:00100000:2.0:1776178550.113638:0:9554:0:(nrs_fifo.c:179:nrs_fifo_req_get()) NRS start fifo request from 12345-0@lo, seq: 623 00000100:00100000:2.0:1776178550.113645:0:9554:0:(service.c:2324:ptlrpc_server_handle_request()) Handling RPC req@000000009e7a8bb5 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_create00:lustre-MDT0001-mdtlov_UUID+5:3356:x1862456998097280:12345-0@lo:13:osp-pre-1-1.0 00002000:00000024:2.0:1776178550.113687:0:9554:0:(ofd_obd.c:740:ofd_statfs()) blocks cached 229376 granted 491746368 pending 0 free 3923689472 avail 3692167168 00000100:00100000:2.0:1776178550.113751:0:9554:0:(service.c:2372:ptlrpc_server_handle_request()) Handled RPC req@000000009e7a8bb5 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_create00:lustre-MDT0001-mdtlov_UUID+5:3356:x1862456998097280:12345-0@lo:13:osp-pre-1-1.0 Request processed in 105us (390us total) trans 0 rc 0/0 00000100:00100000:2.0:1776178550.113760:0:9554:0:(nrs_fifo.c:241:nrs_fifo_req_stop()) NRS stop fifo request from 12345-0@lo, seq: 623 00000100:00100000:3.0:1776178550.113875:0:3356:0:(client.c:2215:ptlrpc_check_set()) Completed RPC req@0000000018bd7474 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_03:lustre-MDT0001-mdtlov_UUID:3356:1862456998097280:0@lo:13:osp-pre-1-1.0 00010000:00010000:2.0:1776178550.240931:0:6245:0:(ldlm_lockd.c:2792:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:3.0:1776178550.626086:0:3356:0:(client.c:1733:ptlrpc_send_new_req()) Sending RPC req@00000000d22cceea pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_03:lustre-MDT0001-mdtlov_UUID:3356:1862456998097344:0@lo:13:osp-pre-0-1.0 00000100:00100000:3.0:1776178550.626173:0:3356:0:(events.c:361:request_in_callback()) peer: 12345-0@lo (source: 12345-0@lo) 00000100:00100000:1.0:1776178550.626228:0:8231:0:(service.c:2152:ptlrpc_server_handle_req_in()) got req x1862456998097344 00000100:00100000:1.0:1776178550.626328:0:8231:0:(nrs_fifo.c:179:nrs_fifo_req_get()) NRS start fifo request from 12345-0@lo, seq: 624 00000100:00100000:1.0:1776178550.626337:0:8231:0:(service.c:2324:ptlrpc_server_handle_request()) Handling RPC req@0000000018bd7474 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_create00:lustre-MDT0001-mdtlov_UUID+5:3356:x1862456998097344:12345-0@lo:13:osp-pre-0-1.0 00002000:00000024:1.0:1776178550.626393:0:8231:0:(ofd_obd.c:740:ofd_statfs()) blocks cached 315392 granted 484883776 pending 0 free 3923689472 avail 3692167168 00000100:00100000:1.0:1776178550.626491:0:8231:0:(service.c:2372:ptlrpc_server_handle_request()) Handled RPC req@0000000018bd7474 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_create00:lustre-MDT0001-mdtlov_UUID+5:3356:x1862456998097344:12345-0@lo:13:osp-pre-0-1.0 Request processed in 154us (319us total) trans 0 rc 0/0 00000100:00100000:1.0:1776178550.626501:0:8231:0:(nrs_fifo.c:241:nrs_fifo_req_stop()) NRS stop fifo request from 12345-0@lo, seq: 624 00000100:00100000:3.0:1776178550.626631:0:3356:0:(client.c:2215:ptlrpc_check_set()) Completed RPC req@00000000d22cceea pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_03:lustre-MDT0001-mdtlov_UUID:3356:1862456998097344:0@lo:13:osp-pre-0-1.0 00010000:00010000:1.0:1776178551.264667:0:9224:0:(ldlm_lockd.c:2792:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:3.0:1776178552.108376:0:3346:0:(events.c:361:request_in_callback()) peer: 12345-192.168.204.3@tcp (source: 12345-192.168.204.3@tcp) 00000100:00100000:0.0:1776178552.111150:0:20140:0:(service.c:2152:ptlrpc_server_handle_req_in()) got req x1862456979130752 00000100:00100000:0.0:1776178552.111233:0:20140:0:(nrs_fifo.c:179:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.204.3@tcp, seq: 291 00000100:00100000:0.0:1776178552.111243:0:20140:0:(service.c:2324:ptlrpc_server_handle_request()) Handling RPC req@00000000b4107e97 pname:cluuid+ref:pid:xid:nid:opc:job ll_mgs_0002:5b5c66b1-1a2f-4651-8548-0c75b9a38a94+9:2272:x1862456979130752:12345-192.168.204.3@tcp:400:kworker/u8:4.0 00000100:00100000:0.0:1776178552.111341:0:20140:0:(service.c:2372:ptlrpc_server_handle_request()) Handled RPC req@00000000b4107e97 pname:cluuid+ref:pid:xid:nid:opc:job ll_mgs_0002:5b5c66b1-1a2f-4651-8548-0c75b9a38a94+9:2272:x1862456979130752:12345-192.168.204.3@tcp:400:kworker/u8:4.0 Request processed in 99us (2973us total) trans 0 rc 0/0 00000100:00100000:0.0:1776178552.111349:0:20140:0:(nrs_fifo.c:241:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.204.3@tcp, seq: 291 00000100:00100000:3.0:1776178552.112167:0:3346:0:(events.c:361:request_in_callback()) peer: 12345-192.168.204.3@tcp (source: 12345-192.168.204.3@tcp) 00000100:00100000:3.0:1776178552.112195:0:3346:0:(events.c:361:request_in_callback()) peer: 12345-192.168.204.3@tcp (source: 12345-192.168.204.3@tcp) 00000100:00100000:3.0:1776178552.112217:0:3346:0:(events.c:361:request_in_callback()) peer: 12345-192.168.204.3@tcp (source: 12345-192.168.204.3@tcp) 00000100:00100000:3.0:1776178552.112236:0:3346:0:(events.c:361:request_in_callback()) peer: 12345-192.168.204.3@tcp (source: 12345-192.168.204.3@tcp) 00000100:00100000:1.0:1776178552.113121:0:11281:0:(service.c:2152:ptlrpc_server_handle_req_in()) got req x1862456979130816 00000100:00100000:1.0:1776178552.113185:0:11281:0:(service.c:2152:ptlrpc_server_handle_req_in()) got req x1862456979130880 00000100:00100000:1.0:1776178552.113232:0:11281:0:(nrs_fifo.c:179:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.204.3@tcp, seq: 1132 00000100:00100000:1.0:1776178552.113240:0:11281:0:(service.c:2324:ptlrpc_server_handle_request()) Handling RPC req@00000000f351bcc3 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_004:b50963bd-8eba-4432-9a54-8888d00e3fa1+6:2273:x1862456979130880:12345-192.168.204.3@tcp:400:kworker/u8:4.0 00000100:00100000:1.0:1776178552.113405:0:11281:0:(service.c:2372:ptlrpc_server_handle_request()) Handled RPC req@00000000f351bcc3 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_004:b50963bd-8eba-4432-9a54-8888d00e3fa1+6:2273:x1862456979130880:12345-192.168.204.3@tcp:400:kworker/u8:4.0 Request processed in 164us (1208us total) trans 0 rc 0/0 00000100:00100000:1.0:1776178552.113413:0:11281:0:(nrs_fifo.c:241:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.204.3@tcp, seq: 1132 00000100:00100000:1.0:1776178552.113433:0:11281:0:(nrs_fifo.c:179:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.204.3@tcp, seq: 6896 00000100:00100000:1.0:1776178552.113439:0:11281:0:(service.c:2324:ptlrpc_server_handle_request()) Handling RPC req@00000000803faf52 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_004:0+-99:2269:x1862456979130816:12345-192.168.204.3@tcp:38:kworker/u8:4.0 00010000:02000000:1.0:1776178552.113457:0:11281:0:(ldlm_lib.c:1139:target_handle_connect()) lustre-MDT0000: Not available for connect from 192.168.204.3@tcp (stopping) 00000100:00100000:0.0:1776178552.114502:0:14440:0:(service.c:2152:ptlrpc_server_handle_req_in()) got req x1862456979131008 00000100:00100000:1.0:1776178552.115487:0:11281:0:(service.c:2372:ptlrpc_server_handle_request()) Handled RPC req@00000000803faf52 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_004:0+-99:2269:x1862456979130816:12345-192.168.204.3@tcp:38:kworker/u8:4.0 Request processed in 2029us (3304us total) trans 0 rc -19/-19 00000100:00100000:1.0:1776178552.115503:0:11281:0:(nrs_fifo.c:241:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.204.3@tcp, seq: 6896 00000100:00100000:0.0:1776178552.117192:0:14440:0:(service.c:2152:ptlrpc_server_handle_req_in()) got req x1862456979130944 00000100:00100000:0.0:1776178552.133677:0:14440:0:(nrs_fifo.c:179:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.204.3@tcp, seq: 783 00000100:00100000:0.0:1776178552.133696:0:14440:0:(service.c:2324:ptlrpc_server_handle_request()) Handling RPC req@0000000047a42b33 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_004:b50963bd-8eba-4432-9a54-8888d00e3fa1+9:2273:x1862456979131008:12345-192.168.204.3@tcp:400:kworker/u8:4.0 00000100:00100000:0.0:1776178552.133847:0:14440:0:(service.c:2372:ptlrpc_server_handle_request()) Handled RPC req@0000000047a42b33 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_004:b50963bd-8eba-4432-9a54-8888d00e3fa1+9:2273:x1862456979131008:12345-192.168.204.3@tcp:400:kworker/u8:4.0 Request processed in 155us (21629us total) trans 0 rc 0/0 00000100:00100000:0.0:1776178552.133857:0:14440:0:(nrs_fifo.c:241:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.204.3@tcp, seq: 783 00000100:00100000:0.0:1776178552.133889:0:14440:0:(nrs_fifo.c:179:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.204.3@tcp, seq: 784 00000100:00100000:0.0:1776178552.133895:0:14440:0:(service.c:2324:ptlrpc_server_handle_request()) Handling RPC req@0000000056e345df pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_004:b50963bd-8eba-4432-9a54-8888d00e3fa1+8:2272:x1862456979130944:12345-192.168.204.3@tcp:400:kworker/u8:4.0 00000100:00100000:0.0:1776178552.133935:0:14440:0:(service.c:2372:ptlrpc_server_handle_request()) Handled RPC req@0000000056e345df pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_004:b50963bd-8eba-4432-9a54-8888d00e3fa1+8:2272:x1862456979130944:12345-192.168.204.3@tcp:400:kworker/u8:4.0 Request processed in 40us (21699us total) trans 0 rc 0/0 00000100:00100000:0.0:1776178552.133942:0:14440:0:(nrs_fifo.c:241:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.204.3@tcp, seq: 784 00010000:00010000:1.0:1776178552.290010:0:9225:0:(ldlm_lockd.c:2792:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00010000:00010000:2.0:1776178553.333609:0:6244:0:(ldlm_lockd.c:2792:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00080000:3.0:1776178553.696338:0:37:0:(pinger.c:251:ptlrpc_pinger_process_import()) lustre-MDT0000-lwp-OST0000_UUID->lustre-MDT0000_UUID: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00080000:3.0:1776178553.696347:0:37:0:(recover.c:57:ptlrpc_initiate_recovery()) lustre-MDT0000_UUID: starting recovery 00000100:00080000:3.0:1776178553.696350:0:37:0:(import.c:86:import_set_state_nolock()) 0000000047601a98 lustre-MDT0000_UUID: changing import state from DISCONN to CONNECTING 00000100:00080000:3.0:1776178553.696357:0:37:0:(import.c:534:import_select_connection()) lustre-MDT0000-lwp-OST0000: connect to NID 0@lo last attempt 1255 00000100:00080000:3.0:1776178553.696360:0:37:0:(import.c:581:import_select_connection()) lustre-MDT0000-lwp-OST0000: tried all connections, increasing latency to 21s 00000100:00080000:3.0:1776178553.696366:0:37:0:(import.c:615:import_select_connection()) lustre-MDT0000-lwp-OST0000: import 0000000047601a98 using connection 192.168.204.103@tcp/0@lo 00000100:00100000:3.0:1776178553.696387:0:37:0:(import.c:807:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5) req@000000007d5bffd0 x1862456998097472/t0(0) o38->lustre-MDT0000-lwp-OST0000@0@lo:12/10 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' 00000100:00080000:3.0:1776178553.696420:0:37:0:(pinger.c:251:ptlrpc_pinger_process_import()) lustre-MDT0000-lwp-OST0001_UUID->lustre-MDT0000_UUID: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00080000:3.0:1776178553.696424:0:37:0:(recover.c:57:ptlrpc_initiate_recovery()) lustre-MDT0000_UUID: starting recovery 00000100:00080000:3.0:1776178553.696425:0:37:0:(import.c:86:import_set_state_nolock()) 00000000dfc8db13 lustre-MDT0000_UUID: changing import state from DISCONN to CONNECTING 00000100:00080000:3.0:1776178553.696428:0:37:0:(import.c:534:import_select_connection()) lustre-MDT0000-lwp-OST0001: connect to NID 0@lo last attempt 1255 00000100:00080000:3.0:1776178553.696430:0:37:0:(import.c:581:import_select_connection()) lustre-MDT0000-lwp-OST0001: tried all connections, increasing latency to 21s 00000100:00080000:3.0:1776178553.696434:0:37:0:(import.c:615:import_select_connection()) lustre-MDT0000-lwp-OST0001: import 00000000dfc8db13 using connection 192.168.204.103@tcp/0@lo 00000100:00100000:3.0:1776178553.696443:0:37:0:(import.c:807:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5) req@000000003316d46e x1862456998097600/t0(0) o38->lustre-MDT0000-lwp-OST0001@0@lo:12/10 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' 00000100:00080000:3.0:1776178553.696480:0:37:0:(pinger.c:251:ptlrpc_pinger_process_import()) lustre-MDT0001-mdtlov_UUID->lustre-MDT0000_UUID: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00080000:3.0:1776178553.696484:0:37:0:(recover.c:57:ptlrpc_initiate_recovery()) lustre-MDT0000_UUID: starting recovery 00000100:00080000:3.0:1776178553.696485:0:37:0:(import.c:86:import_set_state_nolock()) 00000000ba4f1821 lustre-MDT0000_UUID: changing import state from DISCONN to CONNECTING 00000100:00080000:3.0:1776178553.696488:0:37:0:(import.c:534:import_select_connection()) lustre-MDT0000-osp-MDT0001: connect to NID 0@lo last attempt 1255 00000100:00080000:3.0:1776178553.696490:0:37:0:(import.c:581:import_select_connection()) lustre-MDT0000-osp-MDT0001: tried all connections, increasing latency to 21s 00000100:00080000:3.0:1776178553.696493:0:37:0:(import.c:615:import_select_connection()) lustre-MDT0000-osp-MDT0001: import 00000000ba4f1821 using connection 192.168.204.103@tcp/0@lo 00000100:00100000:3.0:1776178553.696509:0:37:0:(import.c:807:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5) req@00000000fed3e037 x1862456998097728/t0(0) o38->lustre-MDT0000-osp-MDT0001@0@lo:24/4 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' 00000100:00080000:3.0:1776178553.696548:0:37:0:(pinger.c:251:ptlrpc_pinger_process_import()) lustre-MDT0000-lwp-MDT0001_UUID->lustre-MDT0000_UUID: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00080000:3.0:1776178553.696551:0:37:0:(recover.c:57:ptlrpc_initiate_recovery()) lustre-MDT0000_UUID: starting recovery 00000100:00080000:3.0:1776178553.696553:0:37:0:(import.c:86:import_set_state_nolock()) 00000000a7c55b1f lustre-MDT0000_UUID: changing import state from DISCONN to CONNECTING 00000100:00080000:3.0:1776178553.696556:0:37:0:(import.c:534:import_select_connection()) lustre-MDT0000-lwp-MDT0001: connect to NID 0@lo last attempt 1255 00000100:00080000:3.0:1776178553.696557:0:37:0:(import.c:581:import_select_connection()) lustre-MDT0000-lwp-MDT0001: tried all connections, increasing latency to 21s 00000100:00080000:3.0:1776178553.696560:0:37:0:(import.c:615:import_select_connection()) lustre-MDT0000-lwp-MDT0001: import 00000000a7c55b1f using connection 192.168.204.103@tcp/0@lo 00000100:00100000:3.0:1776178553.696568:0:37:0:(import.c:807:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5) req@00000000d9d2b484 x1862456998097920/t0(0) o38->lustre-MDT0000-lwp-MDT0001@0@lo:12/10 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' 00000100:00100000:3.0:1776178553.696600:0:3356:0:(client.c:1733:ptlrpc_send_new_req()) Sending RPC req@0000000008c9fb19 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_03:lustre-MDT0001-mdtlov_UUID:3356:1862456998097792:0@lo:400:kworker/u8:2.0 00000100:00100000:3.0:1776178553.696663:0:3356:0:(events.c:361:request_in_callback()) peer: 12345-0@lo (source: 12345-0@lo) 00000100:00100000:3.0:1776178553.696696:0:3356:0:(ptlrpcd.c:415:ptlrpcd_check()) transfer 1 async RPCs [2->3] 00000100:00100000:3.0:1776178553.696702:0:3356:0:(client.c:1733:ptlrpc_send_new_req()) Sending RPC req@00000000812a7ebe pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_03:lustre-MDT0001-lwp-OST0001_UUID:3356:1862456998097664:0@lo:400:kworker/u8:2.0 00000100:00100000:3.0:1776178553.696727:0:3356:0:(events.c:361:request_in_callback()) peer: 12345-0@lo (source: 12345-0@lo) 00000100:00100000:2.0:1776178553.697727:0:3354:0:(client.c:1733:ptlrpc_send_new_req()) Sending RPC req@000000002973935f pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_01:lustre-MDT0001-lwp-OST0000_UUID:3354:1862456998097536:0@lo:400:kworker/u8:2.0 00000100:00100000:2.0:1776178553.697769:0:3354:0:(events.c:361:request_in_callback()) peer: 12345-0@lo (source: 12345-0@lo) 00000100:00100000:2.0:1776178553.697803:0:3354:0:(ptlrpcd.c:415:ptlrpcd_check()) transfer 2 async RPCs [0->1] 00000100:00100000:2.0:1776178553.697809:0:3354:0:(client.c:1733:ptlrpc_send_new_req()) Sending RPC req@0000000017d54e92 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_01:ac61f839-0e8e-4cac-9c59-08bcfbbad330:3354:1862456998097408:0@lo:400:kworker/u8:2.0 00000100:00100000:2.0:1776178553.697837:0:3354:0:(events.c:361:request_in_callback()) peer: 12345-0@lo (source: 12345-0@lo) 00000100:00100000:2.0:1776178553.697850:0:3354:0:(client.c:1733:ptlrpc_send_new_req()) Sending RPC req@000000006459e586 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_01:lustre-MDT0001-mdtlov_UUID:3354:1862456998097856:0@lo:400:kworker/u8:2.0 00000100:00100000:2.0:1776178553.697885:0:3354:0:(events.c:361:request_in_callback()) peer: 12345-0@lo (source: 12345-0@lo) 00000100:00100000:2.0:1776178553.697924:0:6257:0:(service.c:2152:ptlrpc_server_handle_req_in()) got req x1862456998097664 00000100:00100000:3.0:1776178553.698229:0:11281:0:(nrs_fifo.c:179:nrs_fifo_req_get()) NRS start fifo request from 12345-0@lo, seq: 1133 00000100:00100000:3.0:1776178553.698236:0:11281:0:(service.c:2324:ptlrpc_server_handle_request()) Handling RPC req@000000005f1d3350 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_004:lustre-MDT0001-lwp-OST0001_UUID+5:3356:x1862456998097664:12345-0@lo:400:kworker/u8:2.0 00000100:00100000:3.0:1776178553.698308:0:11281:0:(service.c:2372:ptlrpc_server_handle_request()) Handled RPC req@000000005f1d3350 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_004:lustre-MDT0001-lwp-OST0001_UUID+5:3356:x1862456998097664:12345-0@lo:400:kworker/u8:2.0 Request processed in 71us (1580us total) trans 0 rc 0/0 00000100:00100000:3.0:1776178553.698315:0:11281:0:(nrs_fifo.c:241:nrs_fifo_req_stop()) NRS stop fifo request from 12345-0@lo, seq: 1133 00000100:00100000:3.0:1776178553.698357:0:3356:0:(client.c:2215:ptlrpc_check_set()) Completed RPC req@00000000812a7ebe pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_03:lustre-MDT0001-lwp-OST0001_UUID:3356:1862456998097664:0@lo:400:kworker/u8:2.0 00000100:00100000:3.0:1776178553.698799:0:20762:0:(service.c:2152:ptlrpc_server_handle_req_in()) got req x1862456998097792 00000100:00100000:3.0:1776178553.698825:0:20762:0:(service.c:2152:ptlrpc_server_handle_req_in()) got req x1862456998097856 00000100:00100000:3.0:1776178553.698842:0:20762:0:(nrs_fifo.c:179:nrs_fifo_req_get()) NRS start fifo request from 12345-0@lo, seq: 785 00000100:00100000:3.0:1776178553.698845:0:20762:0:(service.c:2324:ptlrpc_server_handle_request()) Handling RPC req@00000000d17c6101 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_013:lustre-MDT0001-mdtlov_UUID+5:3356:x1862456998097792:12345-0@lo:400:kworker/u8:2.0 00000100:00100000:3.0:1776178553.699139:0:20762:0:(service.c:2372:ptlrpc_server_handle_request()) Handled RPC req@00000000d17c6101 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_013:lustre-MDT0001-mdtlov_UUID+5:3356:x1862456998097792:12345-0@lo:400:kworker/u8:2.0 Request processed in 292us (2476us total) trans 0 rc 0/0 00000100:00100000:3.0:1776178553.699146:0:20762:0:(nrs_fifo.c:241:nrs_fifo_req_stop()) NRS stop fifo request from 12345-0@lo, seq: 785 00000100:00100000:3.0:1776178553.699159:0:20762:0:(nrs_fifo.c:179:nrs_fifo_req_get()) NRS start fifo request from 12345-0@lo, seq: 786 00000100:00100000:3.0:1776178553.699162:0:20762:0:(service.c:2324:ptlrpc_server_handle_request()) Handling RPC req@000000001c8035b1 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_013:lustre-MDT0001-mdtlov_UUID+5:3354:x1862456998097856:12345-0@lo:400:kworker/u8:2.0 00000100:00100000:3.0:1776178553.699189:0:20762:0:(service.c:2372:ptlrpc_server_handle_request()) Handled RPC req@000000001c8035b1 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_013:lustre-MDT0001-mdtlov_UUID+5:3354:x1862456998097856:12345-0@lo:400:kworker/u8:2.0 Request processed in 27us (1305us total) trans 0 rc 0/0 00000100:00100000:3.0:1776178553.699194:0:20762:0:(nrs_fifo.c:241:nrs_fifo_req_stop()) NRS stop fifo request from 12345-0@lo, seq: 786 00000100:00100000:3.0:1776178553.699220:0:3356:0:(client.c:2215:ptlrpc_check_set()) Completed RPC req@0000000008c9fb19 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_03:lustre-MDT0001-mdtlov_UUID:3356:1862456998097792:0@lo:400:kworker/u8:2.0 00000100:00100000:3.0:1776178553.699245:0:3354:0:(client.c:2215:ptlrpc_check_set()) Completed RPC req@000000006459e586 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_01:lustre-MDT0001-mdtlov_UUID:3354:1862456998097856:0@lo:400:kworker/u8:2.0 00000100:00100000:2.0:1776178553.699336:0:6257:0:(service.c:2152:ptlrpc_server_handle_req_in()) got req x1862456998097536 00000100:00100000:2.0:1776178553.699368:0:6257:0:(nrs_fifo.c:179:nrs_fifo_req_get()) NRS start fifo request from 12345-0@lo, seq: 1134 00000100:00100000:2.0:1776178553.699372:0:6257:0:(service.c:2324:ptlrpc_server_handle_request()) Handling RPC req@00000000597c3414 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_002:lustre-MDT0001-lwp-OST0000_UUID+5:3354:x1862456998097536:12345-0@lo:400:kworker/u8:2.0 00000100:00100000:2.0:1776178553.699406:0:6257:0:(service.c:2372:ptlrpc_server_handle_request()) Handled RPC req@00000000597c3414 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_002:lustre-MDT0001-lwp-OST0000_UUID+5:3354:x1862456998097536:12345-0@lo:400:kworker/u8:2.0 Request processed in 34us (1638us total) trans 0 rc 0/0 00000100:00100000:2.0:1776178553.699411:0:6257:0:(nrs_fifo.c:241:nrs_fifo_req_stop()) NRS stop fifo request from 12345-0@lo, seq: 1134 00000100:00100000:2.0:1776178553.699433:0:3354:0:(client.c:2215:ptlrpc_check_set()) Completed RPC req@000000002973935f pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_01:lustre-MDT0001-lwp-OST0000_UUID:3354:1862456998097536:0@lo:400:kworker/u8:2.0 00000100:00100000:0.0:1776178553.701025:0:20137:0:(service.c:2152:ptlrpc_server_handle_req_in()) got req x1862456998097408 00000100:00100000:0.0:1776178553.701266:0:20137:0:(nrs_fifo.c:179:nrs_fifo_req_get()) NRS start fifo request from 12345-0@lo, seq: 292 00000100:00100000:0.0:1776178553.701273:0:20137:0:(service.c:2324:ptlrpc_server_handle_request()) Handling RPC req@00000000b42c55dd pname:cluuid+ref:pid:xid:nid:opc:job ll_mgs_0000:ac61f839-0e8e-4cac-9c59-08bcfbbad330+18:3354:x1862456998097408:12345-0@lo:400:kworker/u8:2.0 00000100:00100000:0.0:1776178553.701355:0:20137:0:(service.c:2372:ptlrpc_server_handle_request()) Handled RPC req@00000000b42c55dd pname:cluuid+ref:pid:xid:nid:opc:job ll_mgs_0000:ac61f839-0e8e-4cac-9c59-08bcfbbad330+18:3354:x1862456998097408:12345-0@lo:400:kworker/u8:2.0 Request processed in 82us (3517us total) trans 0 rc 0/0 00000100:00100000:0.0:1776178553.701362:0:20137:0:(nrs_fifo.c:241:nrs_fifo_req_stop()) NRS stop fifo request from 12345-0@lo, seq: 292 00000100:00100000:0.0:1776178553.701417:0:3354:0:(client.c:2215:ptlrpc_check_set()) Completed RPC req@0000000017d54e92 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_01:ac61f839-0e8e-4cac-9c59-08bcfbbad330:3354:1862456998097408:0@lo:400:kworker/u8:2.0 00000100:00100000:1.0:1776178553.701928:0:3352:0:(client.c:1733:ptlrpc_send_new_req()) Sending RPC req@000000007d5bffd0 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:lustre-MDT0000-lwp-OST0000_UUID:3352:1862456998097472:0@lo:38:kworker/u8:2.0 00000100:00100000:1.0:1776178553.701979:0:3352:0:(events.c:361:request_in_callback()) peer: 12345-0@lo (source: 12345-0@lo) 00000100:00100000:1.0:1776178553.702006:0:3352:0:(client.c:1733:ptlrpc_send_new_req()) Sending RPC req@000000003316d46e pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:lustre-MDT0000-lwp-OST0001_UUID:3352:1862456998097600:0@lo:38:kworker/u8:2.0 00000100:00100000:1.0:1776178553.702026:0:3352:0:(events.c:361:request_in_callback()) peer: 12345-0@lo (source: 12345-0@lo) 00000100:00100000:1.0:1776178553.702055:0:3352:0:(client.c:1733:ptlrpc_send_new_req()) Sending RPC req@00000000fed3e037 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:lustre-MDT0001-mdtlov_UUID:3352:1862456998097728:0@lo:38:kworker/u8:2.0 00000100:00100000:1.0:1776178553.702109:0:3352:0:(events.c:361:request_in_callback()) peer: 12345-0@lo (source: 12345-0@lo) 00000100:00100000:1.0:1776178553.702124:0:3352:0:(client.c:1733:ptlrpc_send_new_req()) Sending RPC req@00000000d9d2b484 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:lustre-MDT0000-lwp-MDT0001_UUID:3352:1862456998097920:0@lo:38:kworker/u8:2.0 00000100:00100000:1.0:1776178553.702146:0:3352:0:(events.c:361:request_in_callback()) peer: 12345-0@lo (source: 12345-0@lo) 00000100:00100000:1.0:1776178553.702200:0:15213:0:(service.c:2152:ptlrpc_server_handle_req_in()) got req x1862456998097728 00000100:00100000:1.0:1776178553.702223:0:15213:0:(nrs_fifo.c:179:nrs_fifo_req_get()) NRS start fifo request from 12345-0@lo, seq: 776 00000100:00100000:1.0:1776178553.702227:0:15213:0:(service.c:2324:ptlrpc_server_handle_request()) Handling RPC req@000000005927b2c3 pname:cluuid+ref:pid:xid:nid:opc:job mdt_out00_003:0+-99:3352:x1862456998097728:12345-0@lo:38:kworker/u8:2.0 00010000:02000000:1.0:1776178553.702240:0:15213:0:(ldlm_lib.c:1139:target_handle_connect()) lustre-MDT0000: Not available for connect from 0@lo (stopping) 00000100:00100000:1.0:1776178553.702274:0:15213:0:(service.c:2372:ptlrpc_server_handle_request()) Handled RPC req@000000005927b2c3 pname:cluuid+ref:pid:xid:nid:opc:job mdt_out00_003:0+-99:3352:x1862456998097728:12345-0@lo:38:kworker/u8:2.0 Request processed in 47us (165us total) trans 0 rc -19/-19 00000100:00100000:1.0:1776178553.702279:0:15213:0:(nrs_fifo.c:241:nrs_fifo_req_stop()) NRS stop fifo request from 12345-0@lo, seq: 776 00000100:00100000:1.0:1776178553.702298:0:8250:0:(service.c:2152:ptlrpc_server_handle_req_in()) got req x1862456998097472 00000100:00100000:1.0:1776178553.702314:0:8250:0:(service.c:2152:ptlrpc_server_handle_req_in()) got req x1862456998097600 00000100:00100000:1.0:1776178553.702321:0:8250:0:(service.c:2152:ptlrpc_server_handle_req_in()) got req x1862456998097920 00000100:00100000:1.0:1776178553.702328:0:8250:0:(nrs_fifo.c:179:nrs_fifo_req_get()) NRS start fifo request from 12345-0@lo, seq: 6897 00000100:00100000:1.0:1776178553.702331:0:8250:0:(service.c:2324:ptlrpc_server_handle_request()) Handling RPC req@0000000047458d9f pname:cluuid+ref:pid:xid:nid:opc:job mdt00_003:0+-99:3352:x1862456998097472:12345-0@lo:38:kworker/u8:2.0 00010000:02000000:1.0:1776178553.702338:0:8250:0:(ldlm_lib.c:1139:target_handle_connect()) lustre-MDT0000: Not available for connect from 0@lo (stopping) 00000100:00100000:1.0:1776178553.702359:0:8250:0:(service.c:2372:ptlrpc_server_handle_request()) Handled RPC req@0000000047458d9f pname:cluuid+ref:pid:xid:nid:opc:job mdt00_003:0+-99:3352:x1862456998097472:12345-0@lo:38:kworker/u8:2.0 Request processed in 27us (381us total) trans 0 rc -19/-19 00000100:00100000:1.0:1776178553.702364:0:8250:0:(nrs_fifo.c:241:nrs_fifo_req_stop()) NRS stop fifo request from 12345-0@lo, seq: 6897 00000100:00100000:1.0:1776178553.702374:0:8250:0:(nrs_fifo.c:179:nrs_fifo_req_get()) NRS start fifo request from 12345-0@lo, seq: 6898 00000100:00100000:1.0:1776178553.702377:0:8250:0:(service.c:2324:ptlrpc_server_handle_request()) Handling RPC req@00000000a30eb964 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_003:0+-99:3352:x1862456998097600:12345-0@lo:38:kworker/u8:2.0 00010000:02000000:1.0:1776178553.702381:0:8250:0:(ldlm_lib.c:1139:target_handle_connect()) lustre-MDT0000: Not available for connect from 0@lo (stopping) 00000100:00100000:1.0:1776178553.702397:0:8250:0:(service.c:2372:ptlrpc_server_handle_request()) Handled RPC req@00000000a30eb964 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_003:0+-99:3352:x1862456998097600:12345-0@lo:38:kworker/u8:2.0 Request processed in 20us (371us total) trans 0 rc -19/-19 00000100:00100000:1.0:1776178553.702405:0:8250:0:(nrs_fifo.c:241:nrs_fifo_req_stop()) NRS stop fifo request from 12345-0@lo, seq: 6898 00000100:00100000:1.0:1776178553.702414:0:8250:0:(nrs_fifo.c:179:nrs_fifo_req_get()) NRS start fifo request from 12345-0@lo, seq: 6899 00000100:00100000:1.0:1776178553.702417:0:8250:0:(service.c:2324:ptlrpc_server_handle_request()) Handling RPC req@0000000037059d8f pname:cluuid+ref:pid:xid:nid:opc:job mdt00_003:0+-99:3352:x1862456998097920:12345-0@lo:38:kworker/u8:2.0 00010000:02000000:1.0:1776178553.702422:0:8250:0:(ldlm_lib.c:1139:target_handle_connect()) lustre-MDT0000: Not available for connect from 0@lo (stopping) 00000100:00100000:1.0:1776178553.702439:0:8250:0:(service.c:2372:ptlrpc_server_handle_request()) Handled RPC req@0000000037059d8f pname:cluuid+ref:pid:xid:nid:opc:job mdt00_003:0+-99:3352:x1862456998097920:12345-0@lo:38:kworker/u8:2.0 Request processed in 21us (293us total) trans 0 rc -19/-19 00000100:00100000:1.0:1776178553.702444:0:8250:0:(nrs_fifo.c:241:nrs_fifo_req_stop()) NRS stop fifo request from 12345-0@lo, seq: 6899 00000100:00080000:2.0:1776178553.702740:0:3352:0:(import.c:86:import_set_state_nolock()) 0000000047601a98 lustre-MDT0000_UUID: changing import state from CONNECTING to DISCONN 00000100:00080000:2.0:1776178553.702745:0:3352:0:(import.c:1429:ptlrpc_connect_interpret()) recovery of lustre-MDT0000_UUID on 192.168.204.103@tcp failed (-19) 00000100:00100000:2.0:1776178553.702750:0:3352:0:(client.c:2215:ptlrpc_check_set()) Completed RPC req@000000007d5bffd0 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:lustre-MDT0000-lwp-OST0000_UUID:3352:1862456998097472:0@lo:38:kworker/u8:2.0 00000100:00080000:2.0:1776178553.702759:0:3352:0:(import.c:86:import_set_state_nolock()) 00000000dfc8db13 lustre-MDT0000_UUID: changing import state from CONNECTING to DISCONN 00000100:00080000:2.0:1776178553.702761:0:3352:0:(import.c:1429:ptlrpc_connect_interpret()) recovery of lustre-MDT0000_UUID on 192.168.204.103@tcp failed (-19) 00000100:00100000:2.0:1776178553.702763:0:3352:0:(client.c:2215:ptlrpc_check_set()) Completed RPC req@000000003316d46e pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:lustre-MDT0000-lwp-OST0001_UUID:3352:1862456998097600:0@lo:38:kworker/u8:2.0 00000100:00080000:2.0:1776178553.702771:0:3352:0:(import.c:86:import_set_state_nolock()) 00000000ba4f1821 lustre-MDT0000_UUID: changing import state from CONNECTING to DISCONN 00000100:00080000:2.0:1776178553.702773:0:3352:0:(import.c:1429:ptlrpc_connect_interpret()) recovery of lustre-MDT0000_UUID on 192.168.204.103@tcp failed (-19) 00000100:00100000:2.0:1776178553.702775:0:3352:0:(client.c:2215:ptlrpc_check_set()) Completed RPC req@00000000fed3e037 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:lustre-MDT0001-mdtlov_UUID:3352:1862456998097728:0@lo:38:kworker/u8:2.0 00000100:00080000:2.0:1776178553.702782:0:3352:0:(import.c:86:import_set_state_nolock()) 00000000a7c55b1f lustre-MDT0000_UUID: changing import state from CONNECTING to DISCONN 00000100:00080000:2.0:1776178553.702784:0:3352:0:(import.c:1429:ptlrpc_connect_interpret()) recovery of lustre-MDT0000_UUID on 192.168.204.103@tcp failed (-19) 00000100:00100000:2.0:1776178553.702786:0:3352:0:(client.c:2215:ptlrpc_check_set()) Completed RPC req@00000000d9d2b484 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:lustre-MDT0000-lwp-MDT0001_UUID:3352:1862456998097920:0@lo:38:kworker/u8:2.0 00010000:00010000:2.0:1776178554.336699:0:6245:0:(ldlm_lockd.c:2792:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:1.0:1776178555.232472:0:3353:0:(ptlrpcd.c:415:ptlrpcd_check()) transfer 1 async RPCs [1->0] 00000100:00100000:1.0:1776178555.234688:0:3353:0:(client.c:1733:ptlrpc_send_new_req()) Sending RPC req@000000006459e586 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_00:lustre-MDT0001-mdtlov_UUID:3353:1862456998097984:0@lo:13:osp-pre-1-1.0 00000100:00100000:1.0:1776178555.234860:0:3353:0:(events.c:361:request_in_callback()) peer: 12345-0@lo (source: 12345-0@lo) 00000100:00100000:2.0:1776178555.236348:0:9554:0:(service.c:2152:ptlrpc_server_handle_req_in()) got req x1862456998097984 00000100:00100000:2.0:1776178555.236490:0:9554:0:(nrs_fifo.c:179:nrs_fifo_req_get()) NRS start fifo request from 12345-0@lo, seq: 625 00000100:00100000:2.0:1776178555.236502:0:9554:0:(service.c:2324:ptlrpc_server_handle_request()) Handling RPC req@0000000052775143 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_create00:lustre-MDT0001-mdtlov_UUID+5:3353:x1862456998097984:12345-0@lo:13:osp-pre-1-1.0 00002000:00000024:2.0:1776178555.236550:0:9554:0:(ofd_obd.c:740:ofd_statfs()) blocks cached 229376 granted 491746368 pending 0 free 3923689472 avail 3692167168 00000100:00100000:2.0:1776178555.236645:0:9554:0:(service.c:2372:ptlrpc_server_handle_request()) Handled RPC req@0000000052775143 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_create00:lustre-MDT0001-mdtlov_UUID+5:3353:x1862456998097984:12345-0@lo:13:osp-pre-1-1.0 Request processed in 145us (1787us total) trans 0 rc 0/0 00000100:00100000:2.0:1776178555.236652:0:9554:0:(nrs_fifo.c:241:nrs_fifo_req_stop()) NRS stop fifo request from 12345-0@lo, seq: 625 00000100:00100000:0.0:1776178555.236756:0:3353:0:(client.c:2215:ptlrpc_check_set()) Completed RPC req@000000006459e586 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_00:lustre-MDT0001-mdtlov_UUID:3353:1862456998097984:0@lo:13:osp-pre-1-1.0 00010000:00010000:1.0:1776178555.363303:0:9224:0:(ldlm_lockd.c:2792:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:3.0:1776178555.744813:0:3356:0:(ptlrpcd.c:415:ptlrpcd_check()) transfer 1 async RPCs [2->3] 00000100:00100000:3.0:1776178555.746816:0:3356:0:(client.c:1733:ptlrpc_send_new_req()) Sending RPC req@00000000a101cbf3 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_03:lustre-MDT0001-mdtlov_UUID:3356:1862456998098048:0@lo:13:osp-pre-0-1.0 00000100:00100000:3.0:1776178555.747802:0:3356:0:(events.c:361:request_in_callback()) peer: 12345-0@lo (source: 12345-0@lo) 00000100:00100000:2.0:1776178555.748735:0:9554:0:(service.c:2152:ptlrpc_server_handle_req_in()) got req x1862456998098048 00000100:00100000:2.0:1776178555.748836:0:9554:0:(nrs_fifo.c:179:nrs_fifo_req_get()) NRS start fifo request from 12345-0@lo, seq: 626 00000100:00100000:2.0:1776178555.748845:0:9554:0:(service.c:2324:ptlrpc_server_handle_request()) Handling RPC req@0000000008c9fb19 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_create00:lustre-MDT0001-mdtlov_UUID+5:3356:x1862456998098048:12345-0@lo:13:osp-pre-0-1.0 00002000:00000024:2.0:1776178555.748891:0:9554:0:(ofd_obd.c:740:ofd_statfs()) blocks cached 315392 granted 484883776 pending 0 free 3923689472 avail 3692167168 00000100:00100000:2.0:1776178555.750032:0:9554:0:(service.c:2372:ptlrpc_server_handle_request()) Handled RPC req@0000000008c9fb19 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_create00:lustre-MDT0001-mdtlov_UUID+5:3356:x1862456998098048:12345-0@lo:13:osp-pre-0-1.0 Request processed in 1183us (2230us total) trans 0 rc 0/0 00000100:00100000:2.0:1776178555.750051:0:9554:0:(nrs_fifo.c:241:nrs_fifo_req_stop()) NRS stop fifo request from 12345-0@lo, seq: 626 00000100:00100000:3.0:1776178555.750818:0:3356:0:(client.c:2215:ptlrpc_check_set()) Completed RPC req@00000000a101cbf3 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_03:lustre-MDT0001-mdtlov_UUID:3356:1862456998098048:0@lo:13:osp-pre-0-1.0 00010000:00010000:1.0:1776178556.384369:0:9225:0:(ldlm_lockd.c:2792:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:2.0:1776178557.218869:0:3348:0:(events.c:361:request_in_callback()) peer: 12345-192.168.204.3@tcp (source: 12345-192.168.204.3@tcp) 00000100:00100000:3.0:1776178557.219787:0:6256:0:(service.c:2152:ptlrpc_server_handle_req_in()) got req x1862456979131136 00000100:00100000:3.0:1776178557.219842:0:6256:0:(nrs_fifo.c:179:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.204.3@tcp, seq: 6900 00000100:00100000:3.0:1776178557.219848:0:6256:0:(service.c:2324:ptlrpc_server_handle_request()) Handling RPC req@00000000500c0ae6 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_001:0+-99:2269:x1862456979131136:12345-192.168.204.3@tcp:38:kworker/u8:3.0 00010000:02000000:3.0:1776178557.219865:0:6256:0:(ldlm_lib.c:1139:target_handle_connect()) lustre-MDT0000: Not available for connect from 192.168.204.3@tcp (stopping) 00000100:00100000:3.0:1776178557.219998:0:6256:0:(service.c:2372:ptlrpc_server_handle_request()) Handled RPC req@00000000500c0ae6 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_001:0+-99:2269:x1862456979131136:12345-192.168.204.3@tcp:38:kworker/u8:3.0 Request processed in 148us (1137us total) trans 0 rc -19/-19 00000100:00100000:3.0:1776178557.220006:0:6256:0:(nrs_fifo.c:241:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.204.3@tcp, seq: 6900 00000100:00100000:1.0:1776178557.232139:0:3346:0:(events.c:361:request_in_callback()) peer: 12345-192.168.204.3@tcp (source: 12345-192.168.204.3@tcp) 00000100:00100000:1.0:1776178557.232231:0:3346:0:(events.c:361:request_in_callback()) peer: 12345-192.168.204.3@tcp (source: 12345-192.168.204.3@tcp) 00000100:00100000:1.0:1776178557.232422:0:3346:0:(events.c:361:request_in_callback()) peer: 12345-192.168.204.3@tcp (source: 12345-192.168.204.3@tcp) 00000100:00100000:1.0:1776178557.232448:0:3346:0:(events.c:361:request_in_callback()) peer: 12345-192.168.204.3@tcp (source: 12345-192.168.204.3@tcp) 00000100:00100000:2.0:1776178557.232512:0:20140:0:(service.c:2152:ptlrpc_server_handle_req_in()) got req x1862456979131072 00000100:00100000:2.0:1776178557.232563:0:20140:0:(nrs_fifo.c:179:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.204.3@tcp, seq: 293 00000100:00100000:2.0:1776178557.232570:0:20140:0:(service.c:2324:ptlrpc_server_handle_request()) Handling RPC req@00000000e31b009d pname:cluuid+ref:pid:xid:nid:opc:job ll_mgs_0002:5b5c66b1-1a2f-4651-8548-0c75b9a38a94+9:2271:x1862456979131072:12345-192.168.204.3@tcp:400:kworker/u8:3.0 00000100:00100000:2.0:1776178557.232648:0:20140:0:(service.c:2372:ptlrpc_server_handle_request()) Handled RPC req@00000000e31b009d pname:cluuid+ref:pid:xid:nid:opc:job ll_mgs_0002:5b5c66b1-1a2f-4651-8548-0c75b9a38a94+9:2271:x1862456979131072:12345-192.168.204.3@tcp:400:kworker/u8:3.0 Request processed in 78us (518us total) trans 0 rc 0/0 00000100:00100000:2.0:1776178557.232658:0:20140:0:(nrs_fifo.c:241:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.204.3@tcp, seq: 293 00000100:00100000:2.0:1776178557.232733:0:6257:0:(service.c:2152:ptlrpc_server_handle_req_in()) got req x1862456979131200 00000100:00100000:2.0:1776178557.232761:0:6257:0:(nrs_fifo.c:179:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.204.3@tcp, seq: 1135 00000100:00100000:2.0:1776178557.232765:0:6257:0:(service.c:2324:ptlrpc_server_handle_request()) Handling RPC req@00000000c8bb8784 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_002:b50963bd-8eba-4432-9a54-8888d00e3fa1+6:2270:x1862456979131200:12345-192.168.204.3@tcp:400:kworker/u8:3.0 00000100:00100000:2.0:1776178557.232792:0:6257:0:(service.c:2372:ptlrpc_server_handle_request()) Handled RPC req@00000000c8bb8784 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_002:b50963bd-8eba-4432-9a54-8888d00e3fa1+6:2270:x1862456979131200:12345-192.168.204.3@tcp:400:kworker/u8:3.0 Request processed in 26us (372us total) trans 0 rc 0/0 00000100:00100000:2.0:1776178557.232798:0:6257:0:(nrs_fifo.c:241:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.204.3@tcp, seq: 1135 00000100:00100000:0.0:1776178557.242573:0:20755:0:(service.c:2152:ptlrpc_server_handle_req_in()) got req x1862456979131264 00000100:00100000:0.0:1776178557.242651:0:20755:0:(service.c:2152:ptlrpc_server_handle_req_in()) got req x1862456979131328 00000100:00100000:0.0:1776178557.243373:0:20755:0:(nrs_fifo.c:179:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.204.3@tcp, seq: 787 00000100:00100000:0.0:1776178557.243383:0:20755:0:(service.c:2324:ptlrpc_server_handle_request()) Handling RPC req@000000001ef63da8 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_006:b50963bd-8eba-4432-9a54-8888d00e3fa1+8:2271:x1862456979131264:12345-192.168.204.3@tcp:400:kworker/u8:3.0 00000100:00100000:0.0:1776178557.243491:0:20755:0:(service.c:2372:ptlrpc_server_handle_request()) Handled RPC req@000000001ef63da8 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_006:b50963bd-8eba-4432-9a54-8888d00e3fa1+8:2271:x1862456979131264:12345-192.168.204.3@tcp:400:kworker/u8:3.0 Request processed in 107us (11259us total) trans 0 rc 0/0 00000100:00100000:2.0:1776178557.253649:0:20759:0:(nrs_fifo.c:179:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.204.3@tcp, seq: 788 00000100:00100000:2.0:1776178557.253665:0:20759:0:(service.c:2324:ptlrpc_server_handle_request()) Handling RPC req@0000000013888310 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_010:b50963bd-8eba-4432-9a54-8888d00e3fa1+9:2270:x1862456979131328:12345-192.168.204.3@tcp:400:kworker/u8:3.0 00000100:00100000:2.0:1776178557.256718:0:20759:0:(service.c:2372:ptlrpc_server_handle_request()) Handled RPC req@0000000013888310 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_010:b50963bd-8eba-4432-9a54-8888d00e3fa1+9:2270:x1862456979131328:12345-192.168.204.3@tcp:400:kworker/u8:3.0 Request processed in 3048us (24261us total) trans 0 rc 0/0 00000100:00100000:2.0:1776178557.256735:0:20759:0:(nrs_fifo.c:241:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.204.3@tcp, seq: 788 00000100:00100000:0.0:1776178557.257783:0:20755:0:(nrs_fifo.c:241:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.204.3@tcp, seq: 787 00010000:00010000:2.0:1776178557.409401:0:6244:0:(ldlm_lockd.c:2792:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00010000:00010000:1.0:1776178558.432646:0:6245:0:(ldlm_lockd.c:2792:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00080000:2.0:1776178558.818704:0:36:0:(pinger.c:251:ptlrpc_pinger_process_import()) lustre-MDT0000-lwp-OST0000_UUID->lustre-MDT0000_UUID: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00080000:2.0:1776178558.818712:0:36:0:(recover.c:57:ptlrpc_initiate_recovery()) lustre-MDT0000_UUID: starting recovery 00000100:00080000:2.0:1776178558.818716:0:36:0:(import.c:86:import_set_state_nolock()) 0000000047601a98 lustre-MDT0000_UUID: changing import state from DISCONN to CONNECTING 00000100:00080000:2.0:1776178558.818724:0:36:0:(import.c:534:import_select_connection()) lustre-MDT0000-lwp-OST0000: connect to NID 0@lo last attempt 1261 00000100:00080000:2.0:1776178558.818728:0:36:0:(import.c:581:import_select_connection()) lustre-MDT0000-lwp-OST0000: tried all connections, increasing latency to 21s 00000100:00080000:2.0:1776178558.818734:0:36:0:(import.c:615:import_select_connection()) lustre-MDT0000-lwp-OST0000: import 0000000047601a98 using connection 192.168.204.103@tcp/0@lo 00000100:00100000:2.0:1776178558.818753:0:36:0:(import.c:807:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5) req@000000008f48ae7c x1862456998098176/t0(0) o38->lustre-MDT0000-lwp-OST0000@0@lo:12/10 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' 00000100:00080000:2.0:1776178558.818804:0:36:0:(pinger.c:251:ptlrpc_pinger_process_import()) lustre-MDT0000-lwp-OST0001_UUID->lustre-MDT0000_UUID: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00080000:2.0:1776178558.818808:0:36:0:(recover.c:57:ptlrpc_initiate_recovery()) lustre-MDT0000_UUID: starting recovery 00000100:00080000:2.0:1776178558.818809:0:36:0:(import.c:86:import_set_state_nolock()) 00000000dfc8db13 lustre-MDT0000_UUID: changing import state from DISCONN to CONNECTING 00000100:00080000:2.0:1776178558.818813:0:36:0:(import.c:534:import_select_connection()) lustre-MDT0000-lwp-OST0001: connect to NID 0@lo last attempt 1261 00000100:00080000:2.0:1776178558.818814:0:36:0:(import.c:581:import_select_connection()) lustre-MDT0000-lwp-OST0001: tried all connections, increasing latency to 21s 00000100:00080000:2.0:1776178558.818818:0:36:0:(import.c:615:import_select_connection()) lustre-MDT0000-lwp-OST0001: import 00000000dfc8db13 using connection 192.168.204.103@tcp/0@lo 00000100:00100000:2.0:1776178558.818826:0:36:0:(import.c:807:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5) req@00000000103d2c47 x1862456998098304/t0(0) o38->lustre-MDT0000-lwp-OST0001@0@lo:12/10 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' 00000100:00080000:2.0:1776178558.818842:0:36:0:(pinger.c:251:ptlrpc_pinger_process_import()) lustre-MDT0001-mdtlov_UUID->lustre-MDT0000_UUID: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00080000:2.0:1776178558.818845:0:36:0:(recover.c:57:ptlrpc_initiate_recovery()) lustre-MDT0000_UUID: starting recovery 00000100:00080000:2.0:1776178558.818846:0:36:0:(import.c:86:import_set_state_nolock()) 00000000ba4f1821 lustre-MDT0000_UUID: changing import state from DISCONN to CONNECTING 00000100:00080000:2.0:1776178558.818849:0:36:0:(import.c:534:import_select_connection()) lustre-MDT0000-osp-MDT0001: connect to NID 0@lo last attempt 1261 00000100:00080000:2.0:1776178558.818850:0:36:0:(import.c:581:import_select_connection()) lustre-MDT0000-osp-MDT0001: tried all connections, increasing latency to 21s 00000100:00080000:2.0:1776178558.818853:0:36:0:(import.c:615:import_select_connection()) lustre-MDT0000-osp-MDT0001: import 00000000ba4f1821 using connection 192.168.204.103@tcp/0@lo 00000100:00100000:2.0:1776178558.818861:0:36:0:(import.c:807:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5) req@0000000044f82452 x1862456998098432/t0(0) o38->lustre-MDT0000-osp-MDT0001@0@lo:24/4 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' 00000100:00080000:2.0:1776178558.818878:0:36:0:(pinger.c:251:ptlrpc_pinger_process_import()) lustre-MDT0000-lwp-MDT0001_UUID->lustre-MDT0000_UUID: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00080000:2.0:1776178558.818881:0:36:0:(recover.c:57:ptlrpc_initiate_recovery()) lustre-MDT0000_UUID: starting recovery 00000100:00080000:2.0:1776178558.818882:0:36:0:(import.c:86:import_set_state_nolock()) 00000000a7c55b1f lustre-MDT0000_UUID: changing import state from DISCONN to CONNECTING 00000100:00080000:2.0:1776178558.818885:0:36:0:(import.c:534:import_select_connection()) lustre-MDT0000-lwp-MDT0001: connect to NID 0@lo last attempt 1261 00000100:00080000:2.0:1776178558.818887:0:36:0:(import.c:581:import_select_connection()) lustre-MDT0000-lwp-MDT0001: tried all connections, increasing latency to 21s 00000100:00080000:2.0:1776178558.818890:0:36:0:(import.c:615:import_select_connection()) lustre-MDT0000-lwp-MDT0001: import 00000000a7c55b1f using connection 192.168.204.103@tcp/0@lo 00000100:00100000:2.0:1776178558.818897:0:36:0:(import.c:807:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5) req@00000000d28be450 x1862456998098624/t0(0) o38->lustre-MDT0000-lwp-MDT0001@0@lo:12/10 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' 00000100:00100000:2.0:1776178558.818925:0:3354:0:(client.c:1733:ptlrpc_send_new_req()) Sending RPC req@00000000889a4355 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_01:lustre-MDT0001-lwp-OST0001_UUID:3354:1862456998098368:0@lo:400:kworker/u8:1.0 00000100:00100000:2.0:1776178558.819270:0:3354:0:(events.c:361:request_in_callback()) peer: 12345-0@lo (source: 12345-0@lo) 00000100:00100000:2.0:1776178558.819304:0:3354:0:(ptlrpcd.c:415:ptlrpcd_check()) transfer 1 async RPCs [0->1] 00000100:00100000:2.0:1776178558.819310:0:3354:0:(client.c:1733:ptlrpc_send_new_req()) Sending RPC req@00000000a430b18f pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_01:lustre-MDT0001-lwp-OST0000_UUID:3354:1862456998098240:0@lo:400:kworker/u8:1.0 00000100:00100000:2.0:1776178558.819332:0:3354:0:(events.c:361:request_in_callback()) peer: 12345-0@lo (source: 12345-0@lo) 00000100:00100000:2.0:1776178558.819364:0:6256:0:(service.c:2152:ptlrpc_server_handle_req_in()) got req x1862456998098368 00000100:00100000:2.0:1776178558.819405:0:6256:0:(service.c:2152:ptlrpc_server_handle_req_in()) got req x1862456998098240 00000100:00100000:2.0:1776178558.819425:0:6256:0:(nrs_fifo.c:179:nrs_fifo_req_get()) NRS start fifo request from 12345-0@lo, seq: 1136 00000100:00100000:2.0:1776178558.819431:0:6256:0:(service.c:2324:ptlrpc_server_handle_request()) Handling RPC req@000000000e0e1967 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_001:lustre-MDT0001-lwp-OST0001_UUID+5:3354:x1862456998098368:12345-0@lo:400:kworker/u8:1.0 00000100:00100000:2.0:1776178558.819480:0:6256:0:(service.c:2372:ptlrpc_server_handle_request()) Handled RPC req@000000000e0e1967 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_001:lustre-MDT0001-lwp-OST0001_UUID+5:3354:x1862456998098368:12345-0@lo:400:kworker/u8:1.0 Request processed in 48us (212us total) trans 0 rc 0/0 00000100:00100000:2.0:1776178558.819487:0:6256:0:(nrs_fifo.c:241:nrs_fifo_req_stop()) NRS stop fifo request from 12345-0@lo, seq: 1136 00000100:00100000:2.0:1776178558.819502:0:6256:0:(nrs_fifo.c:179:nrs_fifo_req_get()) NRS start fifo request from 12345-0@lo, seq: 1137 00000100:00100000:2.0:1776178558.819506:0:6256:0:(service.c:2324:ptlrpc_server_handle_request()) Handling RPC req@000000006b0ae3d8 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_001:lustre-MDT0001-lwp-OST0000_UUID+5:3354:x1862456998098240:12345-0@lo:400:kworker/u8:1.0 00000100:00100000:2.0:1776178558.819526:0:6256:0:(service.c:2372:ptlrpc_server_handle_request()) Handled RPC req@000000006b0ae3d8 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_001:lustre-MDT0001-lwp-OST0000_UUID+5:3354:x1862456998098240:12345-0@lo:400:kworker/u8:1.0 Request processed in 20us (195us total) trans 0 rc 0/0 00000100:00100000:2.0:1776178558.819532:0:6256:0:(nrs_fifo.c:241:nrs_fifo_req_stop()) NRS stop fifo request from 12345-0@lo, seq: 1137 00000100:00100000:2.0:1776178558.819566:0:3354:0:(client.c:2215:ptlrpc_check_set()) Completed RPC req@00000000a430b18f pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_01:lustre-MDT0001-lwp-OST0000_UUID:3354:1862456998098240:0@lo:400:kworker/u8:1.0 00000100:00100000:2.0:1776178558.819574:0:3354:0:(client.c:2215:ptlrpc_check_set()) Completed RPC req@00000000889a4355 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_01:lustre-MDT0001-lwp-OST0001_UUID:3354:1862456998098368:0@lo:400:kworker/u8:1.0 00000100:00100000:3.0:1776178558.819956:0:3356:0:(client.c:1733:ptlrpc_send_new_req()) Sending RPC req@0000000067c1ae4f pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_03:ac61f839-0e8e-4cac-9c59-08bcfbbad330:3356:1862456998098112:0@lo:400:kworker/u8:1.0 00000100:00100000:3.0:1776178558.819993:0:3356:0:(events.c:361:request_in_callback()) peer: 12345-0@lo (source: 12345-0@lo) 00000100:00100000:3.0:1776178558.820018:0:3356:0:(client.c:1733:ptlrpc_send_new_req()) Sending RPC req@00000000664eb80b pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_03:lustre-MDT0001-mdtlov_UUID:3356:1862456998098560:0@lo:400:kworker/u8:1.0 00000100:00100000:3.0:1776178558.820042:0:3356:0:(events.c:361:request_in_callback()) peer: 12345-0@lo (source: 12345-0@lo) 00000100:00100000:3.0:1776178558.820090:0:3356:0:(ptlrpcd.c:415:ptlrpcd_check()) transfer 1 async RPCs [2->3] 00000100:00100000:3.0:1776178558.820097:0:3356:0:(client.c:1733:ptlrpc_send_new_req()) Sending RPC req@0000000028ade006 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_03:lustre-MDT0001-mdtlov_UUID:3356:1862456998098496:0@lo:400:kworker/u8:1.0 00000100:00100000:3.0:1776178558.820118:0:3356:0:(events.c:361:request_in_callback()) peer: 12345-0@lo (source: 12345-0@lo) 00000100:00100000:3.0:1776178558.820143:0:20755:0:(service.c:2152:ptlrpc_server_handle_req_in()) got req x1862456998098560 00000100:00100000:3.0:1776178558.820166:0:20755:0:(service.c:2152:ptlrpc_server_handle_req_in()) got req x1862456998098496 00000100:00100000:3.0:1776178558.820179:0:20755:0:(nrs_fifo.c:179:nrs_fifo_req_get()) NRS start fifo request from 12345-0@lo, seq: 789 00000100:00100000:3.0:1776178558.820183:0:20755:0:(service.c:2324:ptlrpc_server_handle_request()) Handling RPC req@000000006ce0dfc8 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_006:lustre-MDT0001-mdtlov_UUID+5:3356:x1862456998098560:12345-0@lo:400:kworker/u8:1.0 00000100:00100000:3.0:1776178558.820214:0:20755:0:(service.c:2372:ptlrpc_server_handle_request()) Handled RPC req@000000006ce0dfc8 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_006:lustre-MDT0001-mdtlov_UUID+5:3356:x1862456998098560:12345-0@lo:400:kworker/u8:1.0 Request processed in 30us (172us total) trans 0 rc 0/0 00000100:00100000:3.0:1776178558.820220:0:20755:0:(nrs_fifo.c:241:nrs_fifo_req_stop()) NRS stop fifo request from 12345-0@lo, seq: 789 00000100:00100000:3.0:1776178558.820232:0:20755:0:(nrs_fifo.c:179:nrs_fifo_req_get()) NRS start fifo request from 12345-0@lo, seq: 790 00000100:00100000:3.0:1776178558.820235:0:20755:0:(service.c:2324:ptlrpc_server_handle_request()) Handling RPC req@00000000d7e84c97 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_006:lustre-MDT0001-mdtlov_UUID+5:3356:x1862456998098496:12345-0@lo:400:kworker/u8:1.0 00000100:00100000:3.0:1776178558.820257:0:20755:0:(service.c:2372:ptlrpc_server_handle_request()) Handled RPC req@00000000d7e84c97 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_006:lustre-MDT0001-mdtlov_UUID+5:3356:x1862456998098496:12345-0@lo:400:kworker/u8:1.0 Request processed in 21us (139us total) trans 0 rc 0/0 00000100:00100000:3.0:1776178558.820263:0:20755:0:(nrs_fifo.c:241:nrs_fifo_req_stop()) NRS stop fifo request from 12345-0@lo, seq: 790 00000100:00100000:3.0:1776178558.820291:0:3356:0:(client.c:2215:ptlrpc_check_set()) Completed RPC req@0000000028ade006 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_03:lustre-MDT0001-mdtlov_UUID:3356:1862456998098496:0@lo:400:kworker/u8:1.0 00000100:00100000:3.0:1776178558.820300:0:3356:0:(client.c:2215:ptlrpc_check_set()) Completed RPC req@00000000664eb80b pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_03:lustre-MDT0001-mdtlov_UUID:3356:1862456998098560:0@lo:400:kworker/u8:1.0 00000100:00100000:0.0:1776178558.821110:0:3352:0:(client.c:1733:ptlrpc_send_new_req()) Sending RPC req@000000008f48ae7c pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:lustre-MDT0000-lwp-OST0000_UUID:3352:1862456998098176:0@lo:38:kworker/u8:1.0 00000100:00100000:0.0:1776178558.821150:0:3352:0:(events.c:361:request_in_callback()) peer: 12345-0@lo (source: 12345-0@lo) 00000100:00100000:0.0:1776178558.821175:0:3352:0:(client.c:1733:ptlrpc_send_new_req()) Sending RPC req@00000000103d2c47 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:lustre-MDT0000-lwp-OST0001_UUID:3352:1862456998098304:0@lo:38:kworker/u8:1.0 00000100:00100000:0.0:1776178558.821193:0:3352:0:(events.c:361:request_in_callback()) peer: 12345-0@lo (source: 12345-0@lo) 00000100:00100000:0.0:1776178558.821202:0:3352:0:(client.c:1733:ptlrpc_send_new_req()) Sending RPC req@0000000044f82452 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:lustre-MDT0001-mdtlov_UUID:3352:1862456998098432:0@lo:38:kworker/u8:1.0 00000100:00100000:0.0:1776178558.821220:0:3352:0:(events.c:361:request_in_callback()) peer: 12345-0@lo (source: 12345-0@lo) 00000100:00100000:0.0:1776178558.821232:0:3352:0:(client.c:1733:ptlrpc_send_new_req()) Sending RPC req@00000000d28be450 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:lustre-MDT0000-lwp-MDT0001_UUID:3352:1862456998098624:0@lo:38:kworker/u8:1.0 00000100:00100000:0.0:1776178558.821250:0:3352:0:(events.c:361:request_in_callback()) peer: 12345-0@lo (source: 12345-0@lo) 00000100:00100000:1.0:1776178558.821415:0:11281:0:(service.c:2152:ptlrpc_server_handle_req_in()) got req x1862456998098176 00000100:00100000:1.0:1776178558.821437:0:11281:0:(service.c:2152:ptlrpc_server_handle_req_in()) got req x1862456998098304 00000100:00100000:1.0:1776178558.821444:0:11281:0:(service.c:2152:ptlrpc_server_handle_req_in()) got req x1862456998098624 00000100:00100000:1.0:1776178558.821453:0:11281:0:(nrs_fifo.c:179:nrs_fifo_req_get()) NRS start fifo request from 12345-0@lo, seq: 6901 00000100:00100000:1.0:1776178558.821456:0:11281:0:(service.c:2324:ptlrpc_server_handle_request()) Handling RPC req@0000000045ff6e42 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_004:0+-99:3352:x1862456998098176:12345-0@lo:38:kworker/u8:1.0 00010000:02000000:1.0:1776178558.821470:0:11281:0:(ldlm_lib.c:1139:target_handle_connect()) lustre-MDT0000: Not available for connect from 0@lo (stopping) 00000100:00100000:1.0:1776178558.821510:0:11281:0:(service.c:2372:ptlrpc_server_handle_request()) Handled RPC req@0000000045ff6e42 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_004:0+-99:3352:x1862456998098176:12345-0@lo:38:kworker/u8:1.0 Request processed in 53us (361us total) trans 0 rc -19/-19 00000100:00100000:1.0:1776178558.821516:0:11281:0:(nrs_fifo.c:241:nrs_fifo_req_stop()) NRS stop fifo request from 12345-0@lo, seq: 6901 00000100:00100000:1.0:1776178558.821527:0:11281:0:(nrs_fifo.c:179:nrs_fifo_req_get()) NRS start fifo request from 12345-0@lo, seq: 6902 00000100:00100000:1.0:1776178558.821530:0:11281:0:(service.c:2324:ptlrpc_server_handle_request()) Handling RPC req@000000007c827612 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_004:0+-99:3352:x1862456998098304:12345-0@lo:38:kworker/u8:1.0 00010000:02000000:1.0:1776178558.821535:0:11281:0:(ldlm_lib.c:1139:target_handle_connect()) lustre-MDT0000: Not available for connect from 0@lo (stopping) 00000100:00100000:1.0:1776178558.821550:0:11281:0:(service.c:2372:ptlrpc_server_handle_request()) Handled RPC req@000000007c827612 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_004:0+-99:3352:x1862456998098304:12345-0@lo:38:kworker/u8:1.0 Request processed in 20us (357us total) trans 0 rc -19/-19 00000100:00100000:1.0:1776178558.821555:0:11281:0:(nrs_fifo.c:241:nrs_fifo_req_stop()) NRS stop fifo request from 12345-0@lo, seq: 6902 00000100:00100000:1.0:1776178558.821562:0:11281:0:(nrs_fifo.c:179:nrs_fifo_req_get()) NRS start fifo request from 12345-0@lo, seq: 6903 00000100:00100000:1.0:1776178558.821565:0:11281:0:(service.c:2324:ptlrpc_server_handle_request()) Handling RPC req@00000000d22cceea pname:cluuid+ref:pid:xid:nid:opc:job mdt00_004:0+-99:3352:x1862456998098624:12345-0@lo:38:kworker/u8:1.0 00010000:02000000:1.0:1776178558.821569:0:11281:0:(ldlm_lib.c:1139:target_handle_connect()) lustre-MDT0000: Not available for connect from 0@lo (stopping) 00000100:00100000:1.0:1776178558.821581:0:11281:0:(service.c:2372:ptlrpc_server_handle_request()) Handled RPC req@00000000d22cceea pname:cluuid+ref:pid:xid:nid:opc:job mdt00_004:0+-99:3352:x1862456998098624:12345-0@lo:38:kworker/u8:1.0 Request processed in 16us (331us total) trans 0 rc -19/-19 00000100:00100000:1.0:1776178558.821586:0:11281:0:(nrs_fifo.c:241:nrs_fifo_req_stop()) NRS stop fifo request from 12345-0@lo, seq: 6903 00000100:00100000:1.0:1776178558.821603:0:14128:0:(service.c:2152:ptlrpc_server_handle_req_in()) got req x1862456998098432 00000100:00100000:1.0:1776178558.821617:0:14128:0:(nrs_fifo.c:179:nrs_fifo_req_get()) NRS start fifo request from 12345-0@lo, seq: 777 00000100:00100000:1.0:1776178558.821620:0:14128:0:(service.c:2324:ptlrpc_server_handle_request()) Handling RPC req@00000000a101cbf3 pname:cluuid+ref:pid:xid:nid:opc:job mdt_out00_002:0+-99:3352:x1862456998098432:12345-0@lo:38:kworker/u8:1.0 00010000:02000000:1.0:1776178558.821626:0:14128:0:(ldlm_lib.c:1139:target_handle_connect()) lustre-MDT0000: Not available for connect from 0@lo (stopping) 00000100:00100000:1.0:1776178558.821642:0:14128:0:(service.c:2372:ptlrpc_server_handle_request()) Handled RPC req@00000000a101cbf3 pname:cluuid+ref:pid:xid:nid:opc:job mdt_out00_002:0+-99:3352:x1862456998098432:12345-0@lo:38:kworker/u8:1.0 Request processed in 21us (421us total) trans 0 rc -19/-19 00000100:00100000:1.0:1776178558.821646:0:14128:0:(nrs_fifo.c:241:nrs_fifo_req_stop()) NRS stop fifo request from 12345-0@lo, seq: 777 00000100:00100000:2.0:1776178558.821995:0:20137:0:(service.c:2152:ptlrpc_server_handle_req_in()) got req x1862456998098112 00000100:00100000:2.0:1776178558.822032:0:20137:0:(nrs_fifo.c:179:nrs_fifo_req_get()) NRS start fifo request from 12345-0@lo, seq: 294 00000100:00100000:2.0:1776178558.822035:0:20137:0:(service.c:2324:ptlrpc_server_handle_request()) Handling RPC req@0000000035060500 pname:cluuid+ref:pid:xid:nid:opc:job ll_mgs_0000:ac61f839-0e8e-4cac-9c59-08bcfbbad330+18:3356:x1862456998098112:12345-0@lo:400:kworker/u8:1.0 00000100:00100000:2.0:1776178558.822965:0:20137:0:(service.c:2372:ptlrpc_server_handle_request()) Handled RPC req@0000000035060500 pname:cluuid+ref:pid:xid:nid:opc:job ll_mgs_0000:ac61f839-0e8e-4cac-9c59-08bcfbbad330+18:3356:x1862456998098112:12345-0@lo:400:kworker/u8:1.0 Request processed in 925us (2969us total) trans 0 rc 0/0 00000100:00100000:2.0:1776178558.822977:0:20137:0:(nrs_fifo.c:241:nrs_fifo_req_stop()) NRS stop fifo request from 12345-0@lo, seq: 294 00000100:00080000:0.0:1776178558.823246:0:3352:0:(import.c:86:import_set_state_nolock()) 0000000047601a98 lustre-MDT0000_UUID: changing import state from CONNECTING to DISCONN 00000100:00080000:0.0:1776178558.823252:0:3352:0:(import.c:1429:ptlrpc_connect_interpret()) recovery of lustre-MDT0000_UUID on 192.168.204.103@tcp failed (-19) 00000100:00100000:0.0:1776178558.823256:0:3352:0:(client.c:2215:ptlrpc_check_set()) Completed RPC req@000000008f48ae7c pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:lustre-MDT0000-lwp-OST0000_UUID:3352:1862456998098176:0@lo:38:kworker/u8:1.0 00000100:00080000:0.0:1776178558.823267:0:3352:0:(import.c:86:import_set_state_nolock()) 00000000dfc8db13 lustre-MDT0000_UUID: changing import state from CONNECTING to DISCONN 00000100:00080000:0.0:1776178558.823270:0:3352:0:(import.c:1429:ptlrpc_connect_interpret()) recovery of lustre-MDT0000_UUID on 192.168.204.103@tcp failed (-19) 00000100:00100000:0.0:1776178558.823272:0:3352:0:(client.c:2215:ptlrpc_check_set()) Completed RPC req@00000000103d2c47 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:lustre-MDT0000-lwp-OST0001_UUID:3352:1862456998098304:0@lo:38:kworker/u8:1.0 00000100:00080000:0.0:1776178558.823281:0:3352:0:(import.c:86:import_set_state_nolock()) 00000000ba4f1821 lustre-MDT0000_UUID: changing import state from CONNECTING to DISCONN 00000100:00080000:0.0:1776178558.823283:0:3352:0:(import.c:1429:ptlrpc_connect_interpret()) recovery of lustre-MDT0000_UUID on 192.168.204.103@tcp failed (-19) 00000100:00100000:0.0:1776178558.823286:0:3352:0:(client.c:2215:ptlrpc_check_set()) Completed RPC req@0000000044f82452 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:lustre-MDT0001-mdtlov_UUID:3352:1862456998098432:0@lo:38:kworker/u8:1.0 00000100:00080000:0.0:1776178558.823294:0:3352:0:(import.c:86:import_set_state_nolock()) 00000000a7c55b1f lustre-MDT0000_UUID: changing import state from CONNECTING to DISCONN 00000100:00080000:0.0:1776178558.823296:0:3352:0:(import.c:1429:ptlrpc_connect_interpret()) recovery of lustre-MDT0000_UUID on 192.168.204.103@tcp failed (-19) 00000100:00100000:0.0:1776178558.823299:0:3352:0:(client.c:2215:ptlrpc_check_set()) Completed RPC req@00000000d28be450 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:lustre-MDT0000-lwp-MDT0001_UUID:3352:1862456998098624:0@lo:38:kworker/u8:1.0 00000100:00100000:1.0:1776178558.823949:0:3356:0:(client.c:2215:ptlrpc_check_set()) Completed RPC req@0000000067c1ae4f pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_03:ac61f839-0e8e-4cac-9c59-08bcfbbad330:3356:1862456998098112:0@lo:400:kworker/u8:1.0 00010000:00010000:1.0:1776178559.457508:0:9224:0:(ldlm_lockd.c:2792:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:2.0:1776178560.354227:0:3354:0:(ptlrpcd.c:415:ptlrpcd_check()) transfer 1 async RPCs [0->1] 00000100:00100000:2.0:1776178560.354250:0:3354:0:(client.c:1733:ptlrpc_send_new_req()) Sending RPC req@00000000b7440871 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_01:lustre-MDT0001-mdtlov_UUID:3354:1862456998098688:0@lo:13:osp-pre-1-1.0 00000100:00100000:2.0:1776178560.354332:0:3354:0:(events.c:361:request_in_callback()) peer: 12345-0@lo (source: 12345-0@lo) 00000100:00100000:3.0:1776178560.354423:0:9554:0:(service.c:2152:ptlrpc_server_handle_req_in()) got req x1862456998098688 00000100:00100000:3.0:1776178560.354480:0:9554:0:(nrs_fifo.c:179:nrs_fifo_req_get()) NRS start fifo request from 12345-0@lo, seq: 627 00000100:00100000:3.0:1776178560.354486:0:9554:0:(service.c:2324:ptlrpc_server_handle_request()) Handling RPC req@00000000380b40c4 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_create00:lustre-MDT0001-mdtlov_UUID+5:3354:x1862456998098688:12345-0@lo:13:osp-pre-1-1.0 00002000:00000024:3.0:1776178560.354512:0:9554:0:(ofd_obd.c:740:ofd_statfs()) blocks cached 229376 granted 491746368 pending 0 free 3923689472 avail 3692167168 00000100:00100000:3.0:1776178560.354791:0:9554:0:(service.c:2372:ptlrpc_server_handle_request()) Handled RPC req@00000000380b40c4 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_create00:lustre-MDT0001-mdtlov_UUID+5:3354:x1862456998098688:12345-0@lo:13:osp-pre-1-1.0 Request processed in 304us (458us total) trans 0 rc 0/0 00000100:00100000:3.0:1776178560.354803:0:9554:0:(nrs_fifo.c:241:nrs_fifo_req_stop()) NRS stop fifo request from 12345-0@lo, seq: 627 00000100:00100000:2.0:1776178560.354949:0:3354:0:(client.c:2215:ptlrpc_check_set()) Completed RPC req@00000000b7440871 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_01:lustre-MDT0001-mdtlov_UUID:3354:1862456998098688:0@lo:13:osp-pre-1-1.0 00010000:00010000:1.0:1776178560.481600:0:9225:0:(ldlm_lockd.c:2792:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:1.0:1776178560.864471:0:3353:0:(ptlrpcd.c:415:ptlrpcd_check()) transfer 1 async RPCs [1->0] 00000100:00100000:1.0:1776178560.870758:0:3353:0:(client.c:1733:ptlrpc_send_new_req()) Sending RPC req@0000000014e690d4 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_00:lustre-MDT0001-mdtlov_UUID:3353:1862456998098752:0@lo:13:osp-pre-0-1.0 00000100:00100000:1.0:1776178560.870873:0:3353:0:(events.c:361:request_in_callback()) peer: 12345-0@lo (source: 12345-0@lo) 00000100:00100000:3.0:1776178560.871008:0:9554:0:(service.c:2152:ptlrpc_server_handle_req_in()) got req x1862456998098752 00000100:00100000:1.0:1776178560.871409:0:8231:0:(nrs_fifo.c:179:nrs_fifo_req_get()) NRS start fifo request from 12345-0@lo, seq: 628 00000100:00100000:1.0:1776178560.871421:0:8231:0:(service.c:2324:ptlrpc_server_handle_request()) Handling RPC req@0000000028dd2f9a pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_create00:lustre-MDT0001-mdtlov_UUID+5:3353:x1862456998098752:12345-0@lo:13:osp-pre-0-1.0 00002000:00000024:1.0:1776178560.871491:0:8231:0:(ofd_obd.c:740:ofd_statfs()) blocks cached 315392 granted 484883776 pending 0 free 3923689472 avail 3692167168 00000100:00100000:1.0:1776178560.871591:0:8231:0:(service.c:2372:ptlrpc_server_handle_request()) Handled RPC req@0000000028dd2f9a pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_create00:lustre-MDT0001-mdtlov_UUID+5:3353:x1862456998098752:12345-0@lo:13:osp-pre-0-1.0 Request processed in 170us (719us total) trans 0 rc 0/0 00000100:00100000:1.0:1776178560.871599:0:8231:0:(nrs_fifo.c:241:nrs_fifo_req_stop()) NRS stop fifo request from 12345-0@lo, seq: 628 00000100:00100000:0.0:1776178560.871944:0:3353:0:(client.c:2215:ptlrpc_check_set()) Completed RPC req@0000000014e690d4 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_00:lustre-MDT0001-mdtlov_UUID:3353:1862456998098752:0@lo:13:osp-pre-0-1.0 00010000:00010000:2.0:1776178561.504623:0:6244:0:(ldlm_lockd.c:2792:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:3.0:1776178562.338973:0:3348:0:(events.c:361:request_in_callback()) peer: 12345-192.168.204.3@tcp (source: 12345-192.168.204.3@tcp) 00000100:00100000:3.0:1776178562.339019:0:3348:0:(events.c:361:request_in_callback()) peer: 12345-192.168.204.3@tcp (source: 12345-192.168.204.3@tcp) 00000100:00100000:3.0:1776178562.339056:0:3348:0:(events.c:361:request_in_callback()) peer: 12345-192.168.204.3@tcp (source: 12345-192.168.204.3@tcp) 00000100:00100000:3.0:1776178562.339136:0:3348:0:(events.c:361:request_in_callback()) peer: 12345-192.168.204.3@tcp (source: 12345-192.168.204.3@tcp) 00000100:00100000:0.0:1776178562.339148:0:20759:0:(service.c:2152:ptlrpc_server_handle_req_in()) got req x1862456979131584 00000100:00100000:3.0:1776178562.339171:0:3348:0:(events.c:361:request_in_callback()) peer: 12345-192.168.204.3@tcp (source: 12345-192.168.204.3@tcp) 00000100:00100000:3.0:1776178562.339209:0:14440:0:(service.c:2152:ptlrpc_server_handle_req_in()) got req x1862456979131648 00000100:00100000:0.0:1776178562.339262:0:20759:0:(nrs_fifo.c:179:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.204.3@tcp, seq: 791 00000100:00100000:3.0:1776178562.339280:0:14440:0:(nrs_fifo.c:179:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.204.3@tcp, seq: 792 00000100:00100000:0.0:1776178562.339281:0:20759:0:(service.c:2324:ptlrpc_server_handle_request()) Handling RPC req@00000000b7440871 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_010:b50963bd-8eba-4432-9a54-8888d00e3fa1+8:2272:x1862456979131584:12345-192.168.204.3@tcp:400:kworker/u8:4.0 00000100:00100000:3.0:1776178562.339295:0:14440:0:(service.c:2324:ptlrpc_server_handle_request()) Handling RPC req@00000000d9d2b484 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_004:b50963bd-8eba-4432-9a54-8888d00e3fa1+9:2272:x1862456979131648:12345-192.168.204.3@tcp:400:kworker/u8:4.0 00000100:00100000:3.0:1776178562.339389:0:14440:0:(service.c:2372:ptlrpc_server_handle_request()) Handled RPC req@00000000d9d2b484 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_004:b50963bd-8eba-4432-9a54-8888d00e3fa1+9:2272:x1862456979131648:12345-192.168.204.3@tcp:400:kworker/u8:4.0 Request processed in 99us (254us total) trans 0 rc 0/0 00000100:00100000:3.0:1776178562.339399:0:14440:0:(nrs_fifo.c:241:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.204.3@tcp, seq: 792 00000100:00100000:2.0:1776178562.339426:0:20140:0:(service.c:2152:ptlrpc_server_handle_req_in()) got req x1862456979131392 00000100:00100000:3.0:1776178562.339431:0:6257:0:(service.c:2152:ptlrpc_server_handle_req_in()) got req x1862456979131520 00000100:00100000:2.0:1776178562.339468:0:20140:0:(nrs_fifo.c:179:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.204.3@tcp, seq: 295 00000100:00100000:3.0:1776178562.339468:0:6257:0:(service.c:2152:ptlrpc_server_handle_req_in()) got req x1862456979131456 00000100:00100000:2.0:1776178562.339475:0:20140:0:(service.c:2324:ptlrpc_server_handle_request()) Handling RPC req@00000000f1e6b6f9 pname:cluuid+ref:pid:xid:nid:opc:job ll_mgs_0002:5b5c66b1-1a2f-4651-8548-0c75b9a38a94+9:2272:x1862456979131392:12345-192.168.204.3@tcp:400:kworker/u8:4.0 00000100:00100000:3.0:1776178562.339486:0:6257:0:(nrs_fifo.c:179:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.204.3@tcp, seq: 1138 00000100:00100000:3.0:1776178562.339492:0:6257:0:(service.c:2324:ptlrpc_server_handle_request()) Handling RPC req@000000006459e586 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_002:b50963bd-8eba-4432-9a54-8888d00e3fa1+6:2272:x1862456979131520:12345-192.168.204.3@tcp:400:kworker/u8:4.0 00000100:00100000:0.0:1776178562.339597:0:20759:0:(service.c:2372:ptlrpc_server_handle_request()) Handled RPC req@00000000b7440871 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_010:b50963bd-8eba-4432-9a54-8888d00e3fa1+8:2272:x1862456979131584:12345-192.168.204.3@tcp:400:kworker/u8:4.0 Request processed in 318us (575us total) trans 0 rc 0/0 00000100:00100000:0.0:1776178562.339607:0:20759:0:(nrs_fifo.c:241:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.204.3@tcp, seq: 791 00000100:00100000:1.0:1776178562.340046:0:8250:0:(nrs_fifo.c:179:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.204.3@tcp, seq: 6904 00000100:00100000:1.0:1776178562.340055:0:8250:0:(service.c:2324:ptlrpc_server_handle_request()) Handling RPC req@00000000fed3e037 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_003:0+-99:2269:x1862456979131456:12345-192.168.204.3@tcp:38:kworker/u8:4.0 00010000:02000000:1.0:1776178562.340110:0:8250:0:(ldlm_lib.c:1139:target_handle_connect()) lustre-MDT0000: Not available for connect from 192.168.204.3@tcp (stopping) 00000100:00100000:1.0:1776178562.340171:0:8250:0:(service.c:2372:ptlrpc_server_handle_request()) Handled RPC req@00000000fed3e037 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_003:0+-99:2269:x1862456979131456:12345-192.168.204.3@tcp:38:kworker/u8:4.0 Request processed in 116us (1003us total) trans 0 rc -19/-19 00000100:00100000:1.0:1776178562.340179:0:8250:0:(nrs_fifo.c:241:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.204.3@tcp, seq: 6904 00000100:00100000:2.0:1776178562.340965:0:20140:0:(service.c:2372:ptlrpc_server_handle_request()) Handled RPC req@00000000f1e6b6f9 pname:cluuid+ref:pid:xid:nid:opc:job ll_mgs_0002:5b5c66b1-1a2f-4651-8548-0c75b9a38a94+9:2272:x1862456979131392:12345-192.168.204.3@tcp:400:kworker/u8:4.0 Request processed in 1487us (1999us total) trans 0 rc 0/0 00000100:00100000:2.0:1776178562.340977:0:20140:0:(nrs_fifo.c:241:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.204.3@tcp, seq: 295 00000100:00100000:3.0:1776178562.341275:0:6257:0:(service.c:2372:ptlrpc_server_handle_request()) Handled RPC req@000000006459e586 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_002:b50963bd-8eba-4432-9a54-8888d00e3fa1+6:2272:x1862456979131520:12345-192.168.204.3@tcp:400:kworker/u8:4.0 Request processed in 1780us (2216us total) trans 0 rc 0/0 00000100:00100000:3.0:1776178562.341285:0:6257:0:(nrs_fifo.c:241:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.204.3@tcp, seq: 1138 00010000:00010000:1.0:1776178562.528395:0:6245:0:(ldlm_lockd.c:2792:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00010000:00010000:1.0:1776178563.554139:0:9224:0:(ldlm_lockd.c:2792:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:1.0:1776178563.938044:0:3355:0:(client.c:1733:ptlrpc_send_new_req()) Sending RPC req@0000000014e690d4 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_02:ac61f839-0e8e-4cac-9c59-08bcfbbad330:3355:1862456998098816:0@lo:400:kworker/u8:2.0 00000100:00100000:1.0:1776178563.938303:0:3355:0:(events.c:361:request_in_callback()) peer: 12345-0@lo (source: 12345-0@lo) 00000100:00080000:0.0:1776178563.938486:0:37:0:(pinger.c:251:ptlrpc_pinger_process_import()) lustre-MDT0000-lwp-OST0000_UUID->lustre-MDT0000_UUID: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00080000:0.0:1776178563.938495:0:37:0:(recover.c:57:ptlrpc_initiate_recovery()) lustre-MDT0000_UUID: starting recovery 00000100:00080000:0.0:1776178563.938501:0:37:0:(import.c:86:import_set_state_nolock()) 0000000047601a98 lustre-MDT0000_UUID: changing import state from DISCONN to CONNECTING 00000100:00080000:0.0:1776178563.938508:0:37:0:(import.c:534:import_select_connection()) lustre-MDT0000-lwp-OST0000: connect to NID 0@lo last attempt 1266 00000100:00080000:0.0:1776178563.938512:0:37:0:(import.c:581:import_select_connection()) lustre-MDT0000-lwp-OST0000: tried all connections, increasing latency to 21s 00000100:00080000:0.0:1776178563.938532:0:37:0:(import.c:615:import_select_connection()) lustre-MDT0000-lwp-OST0000: import 0000000047601a98 using connection 192.168.204.103@tcp/0@lo 00000100:00100000:3.0:1776178563.938572:0:20140:0:(service.c:2152:ptlrpc_server_handle_req_in()) got req x1862456998098816 00000100:00100000:0.0:1776178563.938576:0:37:0:(import.c:807:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5) req@0000000020763766 x1862456998098880/t0(0) o38->lustre-MDT0000-lwp-OST0000@0@lo:12/10 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' 00000100:00100000:3.0:1776178563.938642:0:20140:0:(nrs_fifo.c:179:nrs_fifo_req_get()) NRS start fifo request from 12345-0@lo, seq: 296 00000100:00100000:3.0:1776178563.938649:0:20140:0:(service.c:2324:ptlrpc_server_handle_request()) Handling RPC req@000000001091921c pname:cluuid+ref:pid:xid:nid:opc:job ll_mgs_0002:ac61f839-0e8e-4cac-9c59-08bcfbbad330+18:3355:x1862456998098816:12345-0@lo:400:kworker/u8:2.0 00000100:00100000:2.0:1776178563.938810:0:3352:0:(client.c:1733:ptlrpc_send_new_req()) Sending RPC req@0000000020763766 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:lustre-MDT0000-lwp-OST0000_UUID:3352:1862456998098880:0@lo:38:kworker/u8:2.0 00000100:00100000:2.0:1776178563.938867:0:3352:0:(events.c:361:request_in_callback()) peer: 12345-0@lo (source: 12345-0@lo) 00000100:00100000:2.0:1776178563.939011:0:6257:0:(service.c:2152:ptlrpc_server_handle_req_in()) got req x1862456998098880 00000100:00100000:2.0:1776178563.939039:0:6257:0:(nrs_fifo.c:179:nrs_fifo_req_get()) NRS start fifo request from 12345-0@lo, seq: 6905 00000100:00100000:2.0:1776178563.939042:0:6257:0:(service.c:2324:ptlrpc_server_handle_request()) Handling RPC req@000000007d5cfd93 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_002:0+-99:3352:x1862456998098880:12345-0@lo:38:kworker/u8:2.0 00010000:02000000:2.0:1776178563.939057:0:6257:0:(ldlm_lib.c:1139:target_handle_connect()) lustre-MDT0000: Not available for connect from 0@lo (stopping) 00000100:00100000:2.0:1776178563.939118:0:6257:0:(service.c:2372:ptlrpc_server_handle_request()) Handled RPC req@000000007d5cfd93 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_002:0+-99:3352:x1862456998098880:12345-0@lo:38:kworker/u8:2.0 Request processed in 73us (259us total) trans 0 rc -19/-19 00000100:00100000:2.0:1776178563.939125:0:6257:0:(nrs_fifo.c:241:nrs_fifo_req_stop()) NRS stop fifo request from 12345-0@lo, seq: 6905 00000100:00080000:2.0:1776178563.939171:0:3352:0:(import.c:86:import_set_state_nolock()) 0000000047601a98 lustre-MDT0000_UUID: changing import state from CONNECTING to DISCONN 00000100:00080000:2.0:1776178563.939175:0:3352:0:(import.c:1429:ptlrpc_connect_interpret()) recovery of lustre-MDT0000_UUID on 192.168.204.103@tcp failed (-19) 00000100:00100000:2.0:1776178563.939180:0:3352:0:(client.c:2215:ptlrpc_check_set()) Completed RPC req@0000000020763766 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:lustre-MDT0000-lwp-OST0000_UUID:3352:1862456998098880:0@lo:38:kworker/u8:2.0 00000100:00100000:2.0:1776178563.939229:0:3356:0:(client.c:1733:ptlrpc_send_new_req()) Sending RPC req@000000001c9b18be pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_03:lustre-MDT0001-lwp-OST0000_UUID:3356:1862456998098944:0@lo:400:kworker/u8:2.0 00000100:00100000:2.0:1776178563.939256:0:3356:0:(events.c:361:request_in_callback()) peer: 12345-0@lo (source: 12345-0@lo) 00000100:00100000:2.0:1776178563.939283:0:6257:0:(service.c:2152:ptlrpc_server_handle_req_in()) got req x1862456998098944 00000100:00100000:2.0:1776178563.939309:0:6257:0:(nrs_fifo.c:179:nrs_fifo_req_get()) NRS start fifo request from 12345-0@lo, seq: 1139 00000100:00100000:2.0:1776178563.939313:0:6257:0:(service.c:2324:ptlrpc_server_handle_request()) Handling RPC req@00000000d28be450 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_002:lustre-MDT0001-lwp-OST0000_UUID+5:3356:x1862456998098944:12345-0@lo:400:kworker/u8:2.0 00000100:00100000:2.0:1776178563.939350:0:6257:0:(service.c:2372:ptlrpc_server_handle_request()) Handled RPC req@00000000d28be450 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_002:lustre-MDT0001-lwp-OST0000_UUID+5:3356:x1862456998098944:12345-0@lo:400:kworker/u8:2.0 Request processed in 36us (94us total) trans 0 rc 0/0 00000100:00100000:2.0:1776178563.939357:0:6257:0:(nrs_fifo.c:241:nrs_fifo_req_stop()) NRS stop fifo request from 12345-0@lo, seq: 1139 00000100:00100000:2.0:1776178563.939383:0:3356:0:(client.c:2215:ptlrpc_check_set()) Completed RPC req@000000001c9b18be pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_03:lustre-MDT0001-lwp-OST0000_UUID:3356:1862456998098944:0@lo:400:kworker/u8:2.0 00000100:00080000:0.0:1776178563.940213:0:37:0:(pinger.c:251:ptlrpc_pinger_process_import()) lustre-MDT0000-lwp-OST0001_UUID->lustre-MDT0000_UUID: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00080000:0.0:1776178563.940223:0:37:0:(recover.c:57:ptlrpc_initiate_recovery()) lustre-MDT0000_UUID: starting recovery 00000100:00080000:0.0:1776178563.940226:0:37:0:(import.c:86:import_set_state_nolock()) 00000000dfc8db13 lustre-MDT0000_UUID: changing import state from DISCONN to CONNECTING 00000100:00080000:0.0:1776178563.940233:0:37:0:(import.c:534:import_select_connection()) lustre-MDT0000-lwp-OST0001: connect to NID 0@lo last attempt 1266 00000100:00080000:0.0:1776178563.940236:0:37:0:(import.c:581:import_select_connection()) lustre-MDT0000-lwp-OST0001: tried all connections, increasing latency to 21s 00000100:00080000:0.0:1776178563.940242:0:37:0:(import.c:615:import_select_connection()) lustre-MDT0000-lwp-OST0001: import 00000000dfc8db13 using connection 192.168.204.103@tcp/0@lo 00000100:00100000:0.0:1776178563.940279:0:37:0:(import.c:807:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5) req@00000000606788fe x1862456998099008/t0(0) o38->lustre-MDT0000-lwp-OST0001@0@lo:12/10 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' 00000100:00080000:0.0:1776178563.940348:0:37:0:(pinger.c:251:ptlrpc_pinger_process_import()) lustre-MDT0001-mdtlov_UUID->lustre-MDT0000_UUID: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00080000:0.0:1776178563.940352:0:37:0:(recover.c:57:ptlrpc_initiate_recovery()) lustre-MDT0000_UUID: starting recovery 00000100:00080000:0.0:1776178563.940354:0:37:0:(import.c:86:import_set_state_nolock()) 00000000ba4f1821 lustre-MDT0000_UUID: changing import state from DISCONN to CONNECTING 00000100:00080000:0.0:1776178563.940358:0:37:0:(import.c:534:import_select_connection()) lustre-MDT0000-osp-MDT0001: connect to NID 0@lo last attempt 1266 00000100:00080000:0.0:1776178563.940360:0:37:0:(import.c:581:import_select_connection()) lustre-MDT0000-osp-MDT0001: tried all connections, increasing latency to 21s 00000100:00080000:0.0:1776178563.940364:0:37:0:(import.c:615:import_select_connection()) lustre-MDT0000-osp-MDT0001: import 00000000ba4f1821 using connection 192.168.204.103@tcp/0@lo 00000100:00100000:0.0:1776178563.940377:0:37:0:(import.c:807:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5) req@0000000056a48d5b x1862456998099136/t0(0) o38->lustre-MDT0000-osp-MDT0001@0@lo:24/4 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' 00000100:00080000:0.0:1776178563.940405:0:37:0:(pinger.c:251:ptlrpc_pinger_process_import()) lustre-MDT0000-lwp-MDT0001_UUID->lustre-MDT0000_UUID: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00080000:0.0:1776178563.940408:0:37:0:(recover.c:57:ptlrpc_initiate_recovery()) lustre-MDT0000_UUID: starting recovery 00000100:00080000:0.0:1776178563.940410:0:37:0:(import.c:86:import_set_state_nolock()) 00000000a7c55b1f lustre-MDT0000_UUID: changing import state from DISCONN to CONNECTING 00000100:00080000:0.0:1776178563.940413:0:37:0:(import.c:534:import_select_connection()) lustre-MDT0000-lwp-MDT0001: connect to NID 0@lo last attempt 1266 00000100:00080000:0.0:1776178563.940415:0:37:0:(import.c:581:import_select_connection()) lustre-MDT0000-lwp-MDT0001: tried all connections, increasing latency to 21s 00000100:00080000:0.0:1776178563.940419:0:37:0:(import.c:615:import_select_connection()) lustre-MDT0000-lwp-MDT0001: import 00000000a7c55b1f using connection 192.168.204.103@tcp/0@lo 00000100:00100000:0.0:1776178563.940429:0:37:0:(import.c:807:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5) req@000000009b04a931 x1862456998099328/t0(0) o38->lustre-MDT0000-lwp-MDT0001@0@lo:12/10 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' 00000100:00100000:0.0:1776178563.940468:0:3353:0:(client.c:1733:ptlrpc_send_new_req()) Sending RPC req@00000000fd32fba5 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_00:lustre-MDT0001-lwp-OST0001_UUID:3353:1862456998099072:0@lo:400:kworker/u8:2.0 00000100:00100000:0.0:1776178563.940529:0:3353:0:(events.c:361:request_in_callback()) peer: 12345-0@lo (source: 12345-0@lo) 00000100:00100000:0.0:1776178563.940554:0:3353:0:(ptlrpcd.c:415:ptlrpcd_check()) transfer 1 async RPCs [1->0] 00000100:00100000:0.0:1776178563.940560:0:3353:0:(client.c:1733:ptlrpc_send_new_req()) Sending RPC req@00000000c7e83057 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_00:lustre-MDT0001-mdtlov_UUID:3353:1862456998099200:0@lo:400:kworker/u8:2.0 00000100:00100000:0.0:1776178563.940588:0:3353:0:(events.c:361:request_in_callback()) peer: 12345-0@lo (source: 12345-0@lo) 00000100:00100000:0.0:1776178563.940621:0:20762:0:(service.c:2152:ptlrpc_server_handle_req_in()) got req x1862456998099200 00000100:00100000:0.0:1776178563.940663:0:20762:0:(nrs_fifo.c:179:nrs_fifo_req_get()) NRS start fifo request from 12345-0@lo, seq: 793 00000100:00100000:0.0:1776178563.940669:0:20762:0:(service.c:2324:ptlrpc_server_handle_request()) Handling RPC req@000000001c9b18be pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_013:lustre-MDT0001-mdtlov_UUID+5:3353:x1862456998099200:12345-0@lo:400:kworker/u8:2.0 00000100:00100000:2.0:1776178563.940866:0:3352:0:(client.c:1733:ptlrpc_send_new_req()) Sending RPC req@00000000606788fe pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:lustre-MDT0000-lwp-OST0001_UUID:3352:1862456998099008:0@lo:38:kworker/u8:2.0 00000100:00100000:2.0:1776178563.940894:0:3352:0:(events.c:361:request_in_callback()) peer: 12345-0@lo (source: 12345-0@lo) 00000100:00100000:2.0:1776178563.940910:0:3352:0:(client.c:1733:ptlrpc_send_new_req()) Sending RPC req@0000000056a48d5b pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:lustre-MDT0001-mdtlov_UUID:3352:1862456998099136:0@lo:38:kworker/u8:2.0 00000100:00100000:2.0:1776178563.940933:0:3352:0:(events.c:361:request_in_callback()) peer: 12345-0@lo (source: 12345-0@lo) 00000100:00100000:2.0:1776178563.940949:0:3352:0:(client.c:1733:ptlrpc_send_new_req()) Sending RPC req@000000009b04a931 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:lustre-MDT0000-lwp-MDT0001_UUID:3352:1862456998099328:0@lo:38:kworker/u8:2.0 00000100:00100000:2.0:1776178563.940969:0:3352:0:(events.c:361:request_in_callback()) peer: 12345-0@lo (source: 12345-0@lo) 00000100:00100000:2.0:1776178563.940991:0:3356:0:(ptlrpcd.c:415:ptlrpcd_check()) transfer 1 async RPCs [2->3] 00000100:00100000:2.0:1776178563.940998:0:3356:0:(client.c:1733:ptlrpc_send_new_req()) Sending RPC req@00000000949e3fea pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_03:lustre-MDT0001-mdtlov_UUID:3356:1862456998099264:0@lo:400:kworker/u8:2.0 00000100:00100000:2.0:1776178563.941020:0:3356:0:(events.c:361:request_in_callback()) peer: 12345-0@lo (source: 12345-0@lo) 00000100:00100000:2.0:1776178563.941047:0:8250:0:(service.c:2152:ptlrpc_server_handle_req_in()) got req x1862456998099072 00000100:00100000:2.0:1776178563.941102:0:8250:0:(service.c:2152:ptlrpc_server_handle_req_in()) got req x1862456998099008 00000100:00100000:2.0:1776178563.941119:0:8250:0:(service.c:2152:ptlrpc_server_handle_req_in()) got req x1862456998099328 00000100:00100000:2.0:1776178563.941129:0:8250:0:(nrs_fifo.c:179:nrs_fifo_req_get()) NRS start fifo request from 12345-0@lo, seq: 1140 00000100:00100000:2.0:1776178563.941133:0:8250:0:(service.c:2324:ptlrpc_server_handle_request()) Handling RPC req@00000000702f2be6 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_003:lustre-MDT0001-lwp-OST0001_UUID+5:3353:x1862456998099072:12345-0@lo:400:kworker/u8:2.0 00000100:00100000:2.0:1776178563.941186:0:8250:0:(service.c:2372:ptlrpc_server_handle_request()) Handled RPC req@00000000702f2be6 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_003:lustre-MDT0001-lwp-OST0001_UUID+5:3353:x1862456998099072:12345-0@lo:400:kworker/u8:2.0 Request processed in 52us (668us total) trans 0 rc 0/0 00000100:00100000:2.0:1776178563.941194:0:8250:0:(nrs_fifo.c:241:nrs_fifo_req_stop()) NRS stop fifo request from 12345-0@lo, seq: 1140 00000100:00100000:2.0:1776178563.941211:0:8250:0:(nrs_fifo.c:179:nrs_fifo_req_get()) NRS start fifo request from 12345-0@lo, seq: 6906 00000100:00100000:2.0:1776178563.941215:0:8250:0:(service.c:2324:ptlrpc_server_handle_request()) Handling RPC req@0000000044f82452 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_003:0+-99:3352:x1862456998099008:12345-0@lo:38:kworker/u8:2.0 00010000:02000000:2.0:1776178563.941228:0:8250:0:(ldlm_lib.c:1139:target_handle_connect()) lustre-MDT0000: Not available for connect from 0@lo (stopping) 00000100:00100000:2.0:1776178563.941258:0:8250:0:(service.c:2372:ptlrpc_server_handle_request()) Handled RPC req@0000000044f82452 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_003:0+-99:3352:x1862456998099008:12345-0@lo:38:kworker/u8:2.0 Request processed in 42us (364us total) trans 0 rc -19/-19 00000100:00100000:2.0:1776178563.941264:0:8250:0:(nrs_fifo.c:241:nrs_fifo_req_stop()) NRS stop fifo request from 12345-0@lo, seq: 6906 00000100:00100000:2.0:1776178563.941275:0:8250:0:(nrs_fifo.c:179:nrs_fifo_req_get()) NRS start fifo request from 12345-0@lo, seq: 6907 00000100:00100000:2.0:1776178563.941279:0:8250:0:(service.c:2324:ptlrpc_server_handle_request()) Handling RPC req@0000000028ade006 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_003:0+-99:3352:x1862456998099328:12345-0@lo:38:kworker/u8:2.0 00010000:02000000:2.0:1776178563.941285:0:8250:0:(ldlm_lib.c:1139:target_handle_connect()) lustre-MDT0000: Not available for connect from 0@lo (stopping) 00000100:00100000:2.0:1776178563.941305:0:8250:0:(service.c:2372:ptlrpc_server_handle_request()) Handled RPC req@0000000028ade006 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_003:0+-99:3352:x1862456998099328:12345-0@lo:38:kworker/u8:2.0 Request processed in 26us (336us total) trans 0 rc -19/-19 00000100:00100000:2.0:1776178563.941310:0:8250:0:(nrs_fifo.c:241:nrs_fifo_req_stop()) NRS stop fifo request from 12345-0@lo, seq: 6907 00000100:00080000:2.0:1776178563.941347:0:3352:0:(import.c:86:import_set_state_nolock()) 00000000dfc8db13 lustre-MDT0000_UUID: changing import state from CONNECTING to DISCONN 00000100:00080000:2.0:1776178563.941351:0:3352:0:(import.c:1429:ptlrpc_connect_interpret()) recovery of lustre-MDT0000_UUID on 192.168.204.103@tcp failed (-19) 00000100:00100000:2.0:1776178563.941356:0:3352:0:(client.c:2215:ptlrpc_check_set()) Completed RPC req@00000000606788fe pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:lustre-MDT0000-lwp-OST0001_UUID:3352:1862456998099008:0@lo:38:kworker/u8:2.0 00000100:00080000:2.0:1776178563.941364:0:3352:0:(import.c:86:import_set_state_nolock()) 00000000a7c55b1f lustre-MDT0000_UUID: changing import state from CONNECTING to DISCONN 00000100:00080000:2.0:1776178563.941367:0:3352:0:(import.c:1429:ptlrpc_connect_interpret()) recovery of lustre-MDT0000_UUID on 192.168.204.103@tcp failed (-19) 00000100:00100000:2.0:1776178563.941369:0:3352:0:(client.c:2215:ptlrpc_check_set()) Completed RPC req@000000009b04a931 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:lustre-MDT0000-lwp-MDT0001_UUID:3352:1862456998099328:0@lo:38:kworker/u8:2.0 00000100:00100000:2.0:1776178563.941429:0:20759:0:(service.c:2152:ptlrpc_server_handle_req_in()) got req x1862456998099264 00000100:00100000:2.0:1776178563.941458:0:20759:0:(nrs_fifo.c:179:nrs_fifo_req_get()) NRS start fifo request from 12345-0@lo, seq: 794 00000100:00100000:2.0:1776178563.941462:0:20759:0:(service.c:2324:ptlrpc_server_handle_request()) Handling RPC req@00000000889a4355 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_010:lustre-MDT0001-mdtlov_UUID+5:3356:x1862456998099264:12345-0@lo:400:kworker/u8:2.0 00000100:00100000:2.0:1776178563.941497:0:20759:0:(service.c:2372:ptlrpc_server_handle_request()) Handled RPC req@00000000889a4355 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_010:lustre-MDT0001-mdtlov_UUID+5:3356:x1862456998099264:12345-0@lo:400:kworker/u8:2.0 Request processed in 34us (476us total) trans 0 rc 0/0 00000100:00100000:2.0:1776178563.941507:0:20759:0:(nrs_fifo.c:241:nrs_fifo_req_stop()) NRS stop fifo request from 12345-0@lo, seq: 794 00000100:00100000:2.0:1776178563.941560:0:3356:0:(client.c:2215:ptlrpc_check_set()) Completed RPC req@00000000949e3fea pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_03:lustre-MDT0001-mdtlov_UUID:3356:1862456998099264:0@lo:400:kworker/u8:2.0 00000100:00100000:2.0:1776178563.941615:0:15213:0:(service.c:2152:ptlrpc_server_handle_req_in()) got req x1862456998099136 00000100:00100000:2.0:1776178563.941636:0:15213:0:(nrs_fifo.c:179:nrs_fifo_req_get()) NRS start fifo request from 12345-0@lo, seq: 778 00000100:00100000:3.0:1776178563.941640:0:20140:0:(service.c:2372:ptlrpc_server_handle_request()) Handled RPC req@000000001091921c pname:cluuid+ref:pid:xid:nid:opc:job ll_mgs_0002:ac61f839-0e8e-4cac-9c59-08bcfbbad330+18:3355:x1862456998098816:12345-0@lo:400:kworker/u8:2.0 Request processed in 2987us (3436us total) trans 0 rc 0/0 00000100:00100000:2.0:1776178563.941642:0:15213:0:(service.c:2324:ptlrpc_server_handle_request()) Handling RPC req@00000000664eb80b pname:cluuid+ref:pid:xid:nid:opc:job mdt_out00_003:0+-99:3352:x1862456998099136:12345-0@lo:38:kworker/u8:2.0 00010000:02000000:2.0:1776178563.941656:0:15213:0:(ldlm_lib.c:1139:target_handle_connect()) lustre-MDT0000: Not available for connect from 0@lo (stopping) 00000100:00100000:3.0:1776178563.941662:0:20140:0:(nrs_fifo.c:241:nrs_fifo_req_stop()) NRS stop fifo request from 12345-0@lo, seq: 296 00000100:00100000:3.0:1776178563.941714:0:3353:0:(client.c:2215:ptlrpc_check_set()) Completed RPC req@00000000fd32fba5 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_00:lustre-MDT0001-lwp-OST0001_UUID:3353:1862456998099072:0@lo:400:kworker/u8:2.0 00000100:00100000:2.0:1776178563.941965:0:15213:0:(service.c:2372:ptlrpc_server_handle_request()) Handled RPC req@00000000664eb80b pname:cluuid+ref:pid:xid:nid:opc:job mdt_out00_003:0+-99:3352:x1862456998099136:12345-0@lo:38:kworker/u8:2.0 Request processed in 324us (1031us total) trans 0 rc -19/-19 00000100:00100000:2.0:1776178563.941974:0:15213:0:(nrs_fifo.c:241:nrs_fifo_req_stop()) NRS stop fifo request from 12345-0@lo, seq: 778 00000100:00080000:3.0:1776178563.941996:0:3352:0:(import.c:86:import_set_state_nolock()) 00000000ba4f1821 lustre-MDT0000_UUID: changing import state from CONNECTING to DISCONN 00000100:00080000:3.0:1776178563.942001:0:3352:0:(import.c:1429:ptlrpc_connect_interpret()) recovery of lustre-MDT0000_UUID on 192.168.204.103@tcp failed (-19) 00000100:00100000:3.0:1776178563.942006:0:3352:0:(client.c:2215:ptlrpc_check_set()) Completed RPC req@0000000056a48d5b pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:lustre-MDT0001-mdtlov_UUID:3352:1862456998099136:0@lo:38:kworker/u8:2.0 00000100:00100000:0.0:1776178563.942113:0:20762:0:(service.c:2372:ptlrpc_server_handle_request()) Handled RPC req@000000001c9b18be pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_013:lustre-MDT0001-mdtlov_UUID+5:3353:x1862456998099200:12345-0@lo:400:kworker/u8:2.0 Request processed in 1444us (1524us total) trans 0 rc 0/0 00000100:00100000:0.0:1776178563.942121:0:20762:0:(nrs_fifo.c:241:nrs_fifo_req_stop()) NRS stop fifo request from 12345-0@lo, seq: 793 00000100:00100000:3.0:1776178563.942261:0:3353:0:(client.c:2215:ptlrpc_check_set()) Completed RPC req@00000000c7e83057 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_00:lustre-MDT0001-mdtlov_UUID:3353:1862456998099200:0@lo:400:kworker/u8:2.0 00000100:00100000:1.0:1776178563.951809:0:3355:0:(client.c:2215:ptlrpc_check_set()) Completed RPC req@0000000014e690d4 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_02:ac61f839-0e8e-4cac-9c59-08bcfbbad330:3355:1862456998098816:0@lo:400:kworker/u8:2.0 00010000:00010000:1.0:1776178564.576636:0:9225:0:(ldlm_lockd.c:2792:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:2.0:1776178565.475681:0:3356:0:(client.c:1733:ptlrpc_send_new_req()) Sending RPC req@000000003316d46e pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_03:lustre-MDT0001-mdtlov_UUID:3356:1862456998099392:0@lo:13:osp-pre-1-1.0 00000100:00100000:2.0:1776178565.475782:0:3356:0:(events.c:361:request_in_callback()) peer: 12345-0@lo (source: 12345-0@lo) 00000100:00100000:0.0:1776178565.475932:0:9554:0:(service.c:2152:ptlrpc_server_handle_req_in()) got req x1862456998099392 00000100:00100000:0.0:1776178565.475998:0:9554:0:(nrs_fifo.c:179:nrs_fifo_req_get()) NRS start fifo request from 12345-0@lo, seq: 629 00000100:00100000:0.0:1776178565.476005:0:9554:0:(service.c:2324:ptlrpc_server_handle_request()) Handling RPC req@0000000067c1ae4f pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_create00:lustre-MDT0001-mdtlov_UUID+5:3356:x1862456998099392:12345-0@lo:13:osp-pre-1-1.0 00002000:00000024:0.0:1776178565.476038:0:9554:0:(ofd_obd.c:740:ofd_statfs()) blocks cached 229376 granted 491746368 pending 0 free 3923689472 avail 3692167168 00000100:00100000:0.0:1776178565.476137:0:9554:0:(service.c:2372:ptlrpc_server_handle_request()) Handled RPC req@0000000067c1ae4f pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_create00:lustre-MDT0001-mdtlov_UUID+5:3356:x1862456998099392:12345-0@lo:13:osp-pre-1-1.0 Request processed in 130us (356us total) trans 0 rc 0/0 00000100:00100000:0.0:1776178565.476145:0:9554:0:(nrs_fifo.c:241:nrs_fifo_req_stop()) NRS stop fifo request from 12345-0@lo, seq: 629 00000100:00100000:2.0:1776178565.476277:0:3356:0:(client.c:2215:ptlrpc_check_set()) Completed RPC req@000000003316d46e pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_03:lustre-MDT0001-mdtlov_UUID:3356:1862456998099392:0@lo:13:osp-pre-1-1.0 00010000:00010000:2.0:1776178565.600541:0:6244:0:(ldlm_lockd.c:2792:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:3.0:1776178565.984778:0:3353:0:(client.c:1733:ptlrpc_send_new_req()) Sending RPC req@0000000020763766 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_00:lustre-MDT0001-mdtlov_UUID:3353:1862456998099456:0@lo:13:osp-pre-0-1.0 00000100:00100000:3.0:1776178565.984861:0:3353:0:(events.c:361:request_in_callback()) peer: 12345-0@lo (source: 12345-0@lo) 00000100:00100000:1.0:1776178565.988452:0:8231:0:(service.c:2152:ptlrpc_server_handle_req_in()) got req x1862456998099456 00000100:00100000:1.0:1776178565.988556:0:8231:0:(nrs_fifo.c:179:nrs_fifo_req_get()) NRS start fifo request from 12345-0@lo, seq: 630 00000100:00100000:1.0:1776178565.988563:0:8231:0:(service.c:2324:ptlrpc_server_handle_request()) Handling RPC req@000000007d5bffd0 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_create00:lustre-MDT0001-mdtlov_UUID+5:3353:x1862456998099456:12345-0@lo:13:osp-pre-0-1.0 00002000:00000024:1.0:1776178565.988610:0:8231:0:(ofd_obd.c:740:ofd_statfs()) blocks cached 315392 granted 484883776 pending 0 free 3923689472 avail 3692167168 00000100:00100000:1.0:1776178565.988720:0:8231:0:(service.c:2372:ptlrpc_server_handle_request()) Handled RPC req@000000007d5bffd0 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_create00:lustre-MDT0001-mdtlov_UUID+5:3353:x1862456998099456:12345-0@lo:13:osp-pre-0-1.0 Request processed in 157us (3860us total) trans 0 rc 0/0 00000100:00100000:1.0:1776178565.988729:0:8231:0:(nrs_fifo.c:241:nrs_fifo_req_stop()) NRS stop fifo request from 12345-0@lo, seq: 630 00000100:00100000:3.0:1776178565.989561:0:3353:0:(client.c:2215:ptlrpc_check_set()) Completed RPC req@0000000020763766 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_00:lustre-MDT0001-mdtlov_UUID:3353:1862456998099456:0@lo:13:osp-pre-0-1.0 00010000:00010000:1.0:1776178566.625283:0:6245:0:(ldlm_lockd.c:2792:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:0.0:1776178567.457460:0:3346:0:(events.c:361:request_in_callback()) peer: 12345-192.168.204.3@tcp (source: 12345-192.168.204.3@tcp) 00000100:00100000:0.0:1776178567.457506:0:3346:0:(events.c:361:request_in_callback()) peer: 12345-192.168.204.3@tcp (source: 12345-192.168.204.3@tcp) 00000100:00100000:0.0:1776178567.457542:0:3346:0:(events.c:361:request_in_callback()) peer: 12345-192.168.204.3@tcp (source: 12345-192.168.204.3@tcp) 00000100:00100000:0.0:1776178567.457578:0:3346:0:(events.c:361:request_in_callback()) peer: 12345-192.168.204.3@tcp (source: 12345-192.168.204.3@tcp) 00000100:00100000:0.0:1776178567.457602:0:3346:0:(events.c:361:request_in_callback()) peer: 12345-192.168.204.3@tcp (source: 12345-192.168.204.3@tcp) 00000100:00100000:1.0:1776178567.457920:0:20762:0:(service.c:2152:ptlrpc_server_handle_req_in()) got req x1862456979131904 00000100:00100000:1.0:1776178567.457976:0:20762:0:(service.c:2152:ptlrpc_server_handle_req_in()) got req x1862456979131968 00000100:00100000:1.0:1776178567.457997:0:20762:0:(nrs_fifo.c:179:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.204.3@tcp, seq: 795 00000100:00100000:1.0:1776178567.458003:0:20762:0:(service.c:2324:ptlrpc_server_handle_request()) Handling RPC req@0000000014e690d4 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_013:b50963bd-8eba-4432-9a54-8888d00e3fa1+8:2271:x1862456979131904:12345-192.168.204.3@tcp:400:kworker/u8:3.0 00000100:00100000:1.0:1776178567.458107:0:20762:0:(service.c:2372:ptlrpc_server_handle_request()) Handled RPC req@0000000014e690d4 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_013:b50963bd-8eba-4432-9a54-8888d00e3fa1+8:2271:x1862456979131904:12345-192.168.204.3@tcp:400:kworker/u8:3.0 Request processed in 103us (565us total) trans 0 rc 0/0 00000100:00100000:1.0:1776178567.458115:0:20762:0:(nrs_fifo.c:241:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.204.3@tcp, seq: 795 00000100:00100000:1.0:1776178567.458146:0:20762:0:(nrs_fifo.c:179:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.204.3@tcp, seq: 796 00000100:00100000:1.0:1776178567.458152:0:20762:0:(service.c:2324:ptlrpc_server_handle_request()) Handling RPC req@0000000056a48d5b pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_013:b50963bd-8eba-4432-9a54-8888d00e3fa1+9:2271:x1862456979131968:12345-192.168.204.3@tcp:400:kworker/u8:3.0 00000100:00100000:1.0:1776178567.458176:0:20762:0:(service.c:2372:ptlrpc_server_handle_request()) Handled RPC req@0000000056a48d5b pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_013:b50963bd-8eba-4432-9a54-8888d00e3fa1+9:2271:x1862456979131968:12345-192.168.204.3@tcp:400:kworker/u8:3.0 Request processed in 24us (575us total) trans 0 rc 0/0 00000100:00100000:1.0:1776178567.458181:0:20762:0:(nrs_fifo.c:241:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.204.3@tcp, seq: 796 00000100:00100000:2.0:1776178567.458369:0:11281:0:(service.c:2152:ptlrpc_server_handle_req_in()) got req x1862456979131776 00000100:00100000:2.0:1776178567.458398:0:11281:0:(service.c:2152:ptlrpc_server_handle_req_in()) got req x1862456979131840 00000100:00100000:2.0:1776178567.458418:0:11281:0:(nrs_fifo.c:179:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.204.3@tcp, seq: 1141 00000100:00100000:2.0:1776178567.458422:0:11281:0:(service.c:2324:ptlrpc_server_handle_request()) Handling RPC req@00000000c7e83057 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_004:b50963bd-8eba-4432-9a54-8888d00e3fa1+6:2271:x1862456979131840:12345-192.168.204.3@tcp:400:kworker/u8:3.0 00000100:00100000:2.0:1776178567.458449:0:11281:0:(service.c:2372:ptlrpc_server_handle_request()) Handled RPC req@00000000c7e83057 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_004:b50963bd-8eba-4432-9a54-8888d00e3fa1+6:2271:x1862456979131840:12345-192.168.204.3@tcp:400:kworker/u8:3.0 Request processed in 27us (873us total) trans 0 rc 0/0 00000100:00100000:2.0:1776178567.458455:0:11281:0:(nrs_fifo.c:241:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.204.3@tcp, seq: 1141 00000100:00100000:2.0:1776178567.458467:0:11281:0:(nrs_fifo.c:179:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.204.3@tcp, seq: 6908 00000100:00100000:2.0:1776178567.458471:0:11281:0:(service.c:2324:ptlrpc_server_handle_request()) Handling RPC req@0000000020763766 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_004:0+-99:2269:x1862456979131776:12345-192.168.204.3@tcp:38:kworker/u8:3.0 00010000:02000000:2.0:1776178567.458483:0:11281:0:(ldlm_lib.c:1139:target_handle_connect()) lustre-MDT0000: Not available for connect from 192.168.204.3@tcp (stopping) 00000100:00100000:2.0:1776178567.458504:0:11281:0:(service.c:2372:ptlrpc_server_handle_request()) Handled RPC req@0000000020763766 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_004:0+-99:2269:x1862456979131776:12345-192.168.204.3@tcp:38:kworker/u8:3.0 Request processed in 32us (999us total) trans 0 rc -19/-19 00000100:00100000:2.0:1776178567.458509:0:11281:0:(nrs_fifo.c:241:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.204.3@tcp, seq: 6908 00000100:00100000:3.0:1776178567.459095:0:20140:0:(service.c:2152:ptlrpc_server_handle_req_in()) got req x1862456979131712 00000100:00100000:3.0:1776178567.459143:0:20140:0:(nrs_fifo.c:179:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.204.3@tcp, seq: 297 00000100:00100000:3.0:1776178567.459147:0:20140:0:(service.c:2324:ptlrpc_server_handle_request()) Handling RPC req@000000005b4b43d0 pname:cluuid+ref:pid:xid:nid:opc:job ll_mgs_0002:5b5c66b1-1a2f-4651-8548-0c75b9a38a94+9:2271:x1862456979131712:12345-192.168.204.3@tcp:400:kworker/u8:3.0 00000100:00100000:3.0:1776178567.459188:0:20140:0:(service.c:2372:ptlrpc_server_handle_request()) Handled RPC req@000000005b4b43d0 pname:cluuid+ref:pid:xid:nid:opc:job ll_mgs_0002:5b5c66b1-1a2f-4651-8548-0c75b9a38a94+9:2271:x1862456979131712:12345-192.168.204.3@tcp:400:kworker/u8:3.0 Request processed in 41us (1739us total) trans 0 rc 0/0 00000100:00100000:3.0:1776178567.459194:0:20140:0:(nrs_fifo.c:241:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.204.3@tcp, seq: 297 00010000:00010000:1.0:1776178567.650144:0:9224:0:(ldlm_lockd.c:2792:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00010000:00010000:1.0:1776178568.672905:0:9225:0:(ldlm_lockd.c:2792:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:3.0:1776178569.056464:0:3353:0:(ptlrpcd.c:415:ptlrpcd_check()) transfer 1 async RPCs [1->0] 00000100:00080000:0.0:1776178569.056489:0:37:0:(pinger.c:251:ptlrpc_pinger_process_import()) lustre-MDT0000-lwp-OST0000_UUID->lustre-MDT0000_UUID: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00080000:0.0:1776178569.056496:0:37:0:(recover.c:57:ptlrpc_initiate_recovery()) lustre-MDT0000_UUID: starting recovery 00000100:00080000:0.0:1776178569.056501:0:37:0:(import.c:86:import_set_state_nolock()) 0000000047601a98 lustre-MDT0000_UUID: changing import state from DISCONN to CONNECTING 00000100:00080000:0.0:1776178569.056510:0:37:0:(import.c:534:import_select_connection()) lustre-MDT0000-lwp-OST0000: connect to NID 0@lo last attempt 1271 00000100:00080000:0.0:1776178569.056514:0:37:0:(import.c:581:import_select_connection()) lustre-MDT0000-lwp-OST0000: tried all connections, increasing latency to 21s 00000100:00080000:0.0:1776178569.056521:0:37:0:(import.c:615:import_select_connection()) lustre-MDT0000-lwp-OST0000: import 0000000047601a98 using connection 192.168.204.103@tcp/0@lo 00000100:00100000:0.0:1776178569.056547:0:37:0:(import.c:807:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5) req@00000000949e3fea x1862456998099584/t0(0) o38->lustre-MDT0000-lwp-OST0000@0@lo:12/10 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' 00000100:00100000:1.0:1776178569.056633:0:3352:0:(client.c:1733:ptlrpc_send_new_req()) Sending RPC req@00000000949e3fea pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:lustre-MDT0000-lwp-OST0000_UUID:3352:1862456998099584:0@lo:38:kworker/u8:2.0 00000100:00100000:1.0:1776178569.056709:0:3352:0:(events.c:361:request_in_callback()) peer: 12345-0@lo (source: 12345-0@lo) 00000100:00080000:0.0:1776178569.056795:0:37:0:(pinger.c:251:ptlrpc_pinger_process_import()) lustre-MDT0000-lwp-OST0001_UUID->lustre-MDT0000_UUID: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00080000:0.0:1776178569.056799:0:37:0:(recover.c:57:ptlrpc_initiate_recovery()) lustre-MDT0000_UUID: starting recovery 00000100:00080000:0.0:1776178569.056800:0:37:0:(import.c:86:import_set_state_nolock()) 00000000dfc8db13 lustre-MDT0000_UUID: changing import state from DISCONN to CONNECTING 00000100:00080000:0.0:1776178569.056804:0:37:0:(import.c:534:import_select_connection()) lustre-MDT0000-lwp-OST0001: connect to NID 0@lo last attempt 1271 00000100:00080000:0.0:1776178569.056807:0:37:0:(import.c:581:import_select_connection()) lustre-MDT0000-lwp-OST0001: tried all connections, increasing latency to 21s 00000100:00080000:0.0:1776178569.056811:0:37:0:(import.c:615:import_select_connection()) lustre-MDT0000-lwp-OST0001: import 00000000dfc8db13 using connection 192.168.204.103@tcp/0@lo 00000100:00100000:0.0:1776178569.056822:0:37:0:(import.c:807:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5) req@00000000606788fe x1862456998099712/t0(0) o38->lustre-MDT0000-lwp-OST0001@0@lo:12/10 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' 00000100:00100000:3.0:1776178569.056860:0:3353:0:(client.c:1733:ptlrpc_send_new_req()) Sending RPC req@00000000fd32fba5 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_00:ac61f839-0e8e-4cac-9c59-08bcfbbad330:3353:1862456998099520:0@lo:400:kworker/u8:2.0 00000100:00100000:3.0:1776178569.056975:0:3353:0:(events.c:361:request_in_callback()) peer: 12345-0@lo (source: 12345-0@lo) 00000100:00100000:3.0:1776178569.057018:0:3355:0:(client.c:1733:ptlrpc_send_new_req()) Sending RPC req@000000009b04a931 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_02:lustre-MDT0001-lwp-OST0000_UUID:3355:1862456998099648:0@lo:400:kworker/u8:2.0 00000100:00100000:3.0:1776178569.057043:0:3355:0:(events.c:361:request_in_callback()) peer: 12345-0@lo (source: 12345-0@lo) 00000100:00100000:3.0:1776178569.057106:0:6257:0:(service.c:2152:ptlrpc_server_handle_req_in()) got req x1862456998099584 00000100:00100000:3.0:1776178569.057152:0:6257:0:(service.c:2152:ptlrpc_server_handle_req_in()) got req x1862456998099648 00000100:00100000:3.0:1776178569.057181:0:6257:0:(nrs_fifo.c:179:nrs_fifo_req_get()) NRS start fifo request from 12345-0@lo, seq: 1142 00000100:00100000:3.0:1776178569.057187:0:6257:0:(service.c:2324:ptlrpc_server_handle_request()) Handling RPC req@0000000017d54e92 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_002:lustre-MDT0001-lwp-OST0000_UUID+5:3355:x1862456998099648:12345-0@lo:400:kworker/u8:2.0 00000100:00100000:3.0:1776178569.057237:0:6257:0:(service.c:2372:ptlrpc_server_handle_request()) Handled RPC req@0000000017d54e92 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_002:lustre-MDT0001-lwp-OST0000_UUID+5:3355:x1862456998099648:12345-0@lo:400:kworker/u8:2.0 Request processed in 50us (195us total) trans 0 rc 0/0 00000100:00100000:3.0:1776178569.057243:0:6257:0:(nrs_fifo.c:241:nrs_fifo_req_stop()) NRS stop fifo request from 12345-0@lo, seq: 1142 00000100:00100000:3.0:1776178569.057258:0:6257:0:(nrs_fifo.c:179:nrs_fifo_req_get()) NRS start fifo request from 12345-0@lo, seq: 6909 00000100:00100000:3.0:1776178569.057261:0:6257:0:(service.c:2324:ptlrpc_server_handle_request()) Handling RPC req@00000000ad37fc5d pname:cluuid+ref:pid:xid:nid:opc:job mdt00_002:0+-99:3352:x1862456998099584:12345-0@lo:38:kworker/u8:2.0 00010000:02000000:3.0:1776178569.057276:0:6257:0:(ldlm_lib.c:1139:target_handle_connect()) lustre-MDT0000: Not available for connect from 0@lo (stopping) 00000100:00100000:1.0:1776178569.057349:0:3352:0:(client.c:1733:ptlrpc_send_new_req()) Sending RPC req@00000000606788fe pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:lustre-MDT0000-lwp-OST0001_UUID:3352:1862456998099712:0@lo:38:kworker/u8:2.0 00000100:00100000:1.0:1776178569.057378:0:3352:0:(events.c:361:request_in_callback()) peer: 12345-0@lo (source: 12345-0@lo) 00000100:00080000:0.0:1776178569.057566:0:37:0:(pinger.c:251:ptlrpc_pinger_process_import()) lustre-MDT0001-mdtlov_UUID->lustre-MDT0000_UUID: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00080000:0.0:1776178569.057570:0:37:0:(recover.c:57:ptlrpc_initiate_recovery()) lustre-MDT0000_UUID: starting recovery 00000100:00080000:0.0:1776178569.057572:0:37:0:(import.c:86:import_set_state_nolock()) 00000000ba4f1821 lustre-MDT0000_UUID: changing import state from DISCONN to CONNECTING 00000100:00080000:0.0:1776178569.057576:0:37:0:(import.c:534:import_select_connection()) lustre-MDT0000-osp-MDT0001: connect to NID 0@lo last attempt 1271 00000100:00080000:0.0:1776178569.057579:0:37:0:(import.c:581:import_select_connection()) lustre-MDT0000-osp-MDT0001: tried all connections, increasing latency to 21s 00000100:00080000:0.0:1776178569.057583:0:37:0:(import.c:615:import_select_connection()) lustre-MDT0000-osp-MDT0001: import 00000000ba4f1821 using connection 192.168.204.103@tcp/0@lo 00000100:00100000:0.0:1776178569.057594:0:37:0:(import.c:807:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5) req@00000000239d7b9a x1862456998099840/t0(0) o38->lustre-MDT0000-osp-MDT0001@0@lo:24/4 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' 00000100:00080000:0.0:1776178569.057622:0:37:0:(pinger.c:251:ptlrpc_pinger_process_import()) lustre-MDT0000-lwp-MDT0001_UUID->lustre-MDT0000_UUID: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00080000:0.0:1776178569.057625:0:37:0:(recover.c:57:ptlrpc_initiate_recovery()) lustre-MDT0000_UUID: starting recovery 00000100:00080000:0.0:1776178569.057626:0:37:0:(import.c:86:import_set_state_nolock()) 00000000a7c55b1f lustre-MDT0000_UUID: changing import state from DISCONN to CONNECTING 00000100:00080000:0.0:1776178569.057629:0:37:0:(import.c:534:import_select_connection()) lustre-MDT0000-lwp-MDT0001: connect to NID 0@lo last attempt 1271 00000100:00080000:0.0:1776178569.057631:0:37:0:(import.c:581:import_select_connection()) lustre-MDT0000-lwp-MDT0001: tried all connections, increasing latency to 21s 00000100:00080000:0.0:1776178569.057634:0:37:0:(import.c:615:import_select_connection()) lustre-MDT0000-lwp-MDT0001: import 00000000a7c55b1f using connection 192.168.204.103@tcp/0@lo 00000100:00100000:0.0:1776178569.057644:0:37:0:(import.c:807:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5) req@00000000dab5abff x1862456998100032/t0(0) o38->lustre-MDT0000-lwp-MDT0001@0@lo:12/10 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' 00000100:00100000:0.0:1776178569.057669:0:3353:0:(client.c:1733:ptlrpc_send_new_req()) Sending RPC req@00000000f62fe06f pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_00:lustre-MDT0001-mdtlov_UUID:3353:1862456998099904:0@lo:400:kworker/u8:2.0 00000100:00100000:0.0:1776178569.057701:0:3353:0:(events.c:361:request_in_callback()) peer: 12345-0@lo (source: 12345-0@lo) 00000100:00100000:0.0:1776178569.057721:0:3353:0:(ptlrpcd.c:415:ptlrpcd_check()) transfer 1 async RPCs [1->0] 00000100:00100000:0.0:1776178569.057728:0:3353:0:(client.c:1733:ptlrpc_send_new_req()) Sending RPC req@00000000d4b41d38 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_00:lustre-MDT0001-mdtlov_UUID:3353:1862456998099968:0@lo:400:kworker/u8:2.0 00000100:00100000:0.0:1776178569.057764:0:3353:0:(events.c:361:request_in_callback()) peer: 12345-0@lo (source: 12345-0@lo) 00000100:00100000:2.0:1776178569.057765:0:6255:0:(service.c:2152:ptlrpc_server_handle_req_in()) got req x1862456998099712 00000100:00100000:2.0:1776178569.057789:0:6255:0:(nrs_fifo.c:179:nrs_fifo_req_get()) NRS start fifo request from 12345-0@lo, seq: 6910 00000100:00100000:2.0:1776178569.057794:0:6255:0:(service.c:2324:ptlrpc_server_handle_request()) Handling RPC req@000000009de51106 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_000:0+-99:3352:x1862456998099712:12345-0@lo:38:kworker/u8:2.0 00000100:00100000:0.0:1776178569.057801:0:20755:0:(service.c:2152:ptlrpc_server_handle_req_in()) got req x1862456998099904 00010000:02000000:2.0:1776178569.057804:0:6255:0:(ldlm_lib.c:1139:target_handle_connect()) lustre-MDT0000: Not available for connect from 0@lo (stopping) 00000100:00100000:2.0:1776178569.057835:0:6255:0:(service.c:2372:ptlrpc_server_handle_request()) Handled RPC req@000000009de51106 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_000:0+-99:3352:x1862456998099712:12345-0@lo:38:kworker/u8:2.0 Request processed in 41us (457us total) trans 0 rc -19/-19 00000100:00100000:2.0:1776178569.057842:0:6255:0:(nrs_fifo.c:241:nrs_fifo_req_stop()) NRS stop fifo request from 12345-0@lo, seq: 6910 00000100:00100000:0.0:1776178569.057843:0:20755:0:(service.c:2152:ptlrpc_server_handle_req_in()) got req x1862456998099968 00000100:00100000:0.0:1776178569.057870:0:20755:0:(nrs_fifo.c:179:nrs_fifo_req_get()) NRS start fifo request from 12345-0@lo, seq: 797 00000100:00100000:2.0:1776178569.057877:0:14440:0:(nrs_fifo.c:179:nrs_fifo_req_get()) NRS start fifo request from 12345-0@lo, seq: 798 00000100:00100000:2.0:1776178569.057882:0:14440:0:(service.c:2324:ptlrpc_server_handle_request()) Handling RPC req@00000000eafab700 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_004:lustre-MDT0001-mdtlov_UUID+5:3353:x1862456998099968:12345-0@lo:400:kworker/u8:2.0 00000100:00100000:2.0:1776178569.057936:0:14440:0:(service.c:2372:ptlrpc_server_handle_request()) Handled RPC req@00000000eafab700 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_004:lustre-MDT0001-mdtlov_UUID+5:3353:x1862456998099968:12345-0@lo:400:kworker/u8:2.0 Request processed in 53us (172us total) trans 0 rc 0/0 00000100:00100000:2.0:1776178569.057941:0:14440:0:(nrs_fifo.c:241:nrs_fifo_req_stop()) NRS stop fifo request from 12345-0@lo, seq: 798 00000100:00100000:2.0:1776178569.057963:0:20137:0:(service.c:2152:ptlrpc_server_handle_req_in()) got req x1862456998099520 00000100:00100000:2.0:1776178569.058000:0:20137:0:(nrs_fifo.c:179:nrs_fifo_req_get()) NRS start fifo request from 12345-0@lo, seq: 298 00000100:00100000:2.0:1776178569.058003:0:20137:0:(service.c:2324:ptlrpc_server_handle_request()) Handling RPC req@0000000020ee145c pname:cluuid+ref:pid:xid:nid:opc:job ll_mgs_0000:ac61f839-0e8e-4cac-9c59-08bcfbbad330+18:3353:x1862456998099520:12345-0@lo:400:kworker/u8:2.0 00000100:00100000:0.0:1776178569.058030:0:20755:0:(service.c:2324:ptlrpc_server_handle_request()) Handling RPC req@000000008aab9f17 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_006:lustre-MDT0001-mdtlov_UUID+5:3353:x1862456998099904:12345-0@lo:400:kworker/u8:2.0 00000100:00100000:2.0:1776178569.058033:0:20137:0:(service.c:2372:ptlrpc_server_handle_request()) Handled RPC req@0000000020ee145c pname:cluuid+ref:pid:xid:nid:opc:job ll_mgs_0000:ac61f839-0e8e-4cac-9c59-08bcfbbad330+18:3353:x1862456998099520:12345-0@lo:400:kworker/u8:2.0 Request processed in 27us (1057us total) trans 0 rc 0/0 00000100:00100000:2.0:1776178569.058040:0:20137:0:(nrs_fifo.c:241:nrs_fifo_req_stop()) NRS stop fifo request from 12345-0@lo, seq: 298 00000100:00100000:0.0:1776178569.058098:0:20755:0:(service.c:2372:ptlrpc_server_handle_request()) Handled RPC req@000000008aab9f17 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_006:lustre-MDT0001-mdtlov_UUID+5:3353:x1862456998099904:12345-0@lo:400:kworker/u8:2.0 Request processed in 222us (396us total) trans 0 rc 0/0 00000100:00100000:0.0:1776178569.058106:0:20755:0:(nrs_fifo.c:241:nrs_fifo_req_stop()) NRS stop fifo request from 12345-0@lo, seq: 797 00000100:00100000:2.0:1776178569.058114:0:3356:0:(client.c:1733:ptlrpc_send_new_req()) Sending RPC req@00000000dd52d0d0 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_03:lustre-MDT0001-lwp-OST0001_UUID:3356:1862456998099776:0@lo:400:kworker/u8:2.0 00000100:00100000:2.0:1776178569.058156:0:3356:0:(events.c:361:request_in_callback()) peer: 12345-0@lo (source: 12345-0@lo) 00000100:00100000:0.0:1776178569.058164:0:3353:0:(client.c:2215:ptlrpc_check_set()) Completed RPC req@00000000d4b41d38 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_00:lustre-MDT0001-mdtlov_UUID:3353:1862456998099968:0@lo:400:kworker/u8:2.0 00000100:00100000:0.0:1776178569.058181:0:3353:0:(client.c:2215:ptlrpc_check_set()) Completed RPC req@00000000f62fe06f pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_00:lustre-MDT0001-mdtlov_UUID:3353:1862456998099904:0@lo:400:kworker/u8:2.0 00000100:00100000:0.0:1776178569.058194:0:3353:0:(client.c:2215:ptlrpc_check_set()) Completed RPC req@00000000fd32fba5 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_00:ac61f839-0e8e-4cac-9c59-08bcfbbad330:3353:1862456998099520:0@lo:400:kworker/u8:2.0 00000100:00100000:2.0:1776178569.058198:0:8250:0:(service.c:2152:ptlrpc_server_handle_req_in()) got req x1862456998099776 00000100:00100000:2.0:1776178569.058224:0:8250:0:(nrs_fifo.c:179:nrs_fifo_req_get()) NRS start fifo request from 12345-0@lo, seq: 1143 00000100:00100000:2.0:1776178569.058229:0:8250:0:(service.c:2324:ptlrpc_server_handle_request()) Handling RPC req@00000000103d2c47 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_003:lustre-MDT0001-lwp-OST0001_UUID+5:3356:x1862456998099776:12345-0@lo:400:kworker/u8:2.0 00000100:00100000:2.0:1776178569.058263:0:8250:0:(service.c:2372:ptlrpc_server_handle_request()) Handled RPC req@00000000103d2c47 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_003:lustre-MDT0001-lwp-OST0001_UUID+5:3356:x1862456998099776:12345-0@lo:400:kworker/u8:2.0 Request processed in 33us (107us total) trans 0 rc 0/0 00000100:00100000:2.0:1776178569.058270:0:8250:0:(nrs_fifo.c:241:nrs_fifo_req_stop()) NRS stop fifo request from 12345-0@lo, seq: 1143 00000100:00100000:0.0:1776178569.058296:0:3356:0:(client.c:2215:ptlrpc_check_set()) Completed RPC req@00000000dd52d0d0 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_03:lustre-MDT0001-lwp-OST0001_UUID:3356:1862456998099776:0@lo:400:kworker/u8:2.0 00000100:00100000:2.0:1776178569.058334:0:3355:0:(client.c:2215:ptlrpc_check_set()) Completed RPC req@000000009b04a931 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_02:lustre-MDT0001-lwp-OST0000_UUID:3355:1862456998099648:0@lo:400:kworker/u8:2.0 00000100:00100000:3.0:1776178569.058424:0:6257:0:(service.c:2372:ptlrpc_server_handle_request()) Handled RPC req@00000000ad37fc5d pname:cluuid+ref:pid:xid:nid:opc:job mdt00_002:0+-99:3352:x1862456998099584:12345-0@lo:38:kworker/u8:2.0 Request processed in 1160us (1715us total) trans 0 rc -19/-19 00000100:00100000:3.0:1776178569.058431:0:6257:0:(nrs_fifo.c:241:nrs_fifo_req_stop()) NRS stop fifo request from 12345-0@lo, seq: 6909 00000100:00080000:1.0:1776178569.058536:0:3352:0:(import.c:86:import_set_state_nolock()) 0000000047601a98 lustre-MDT0000_UUID: changing import state from CONNECTING to DISCONN 00000100:00080000:1.0:1776178569.058541:0:3352:0:(import.c:1429:ptlrpc_connect_interpret()) recovery of lustre-MDT0000_UUID on 192.168.204.103@tcp failed (-19) 00000100:00100000:1.0:1776178569.058545:0:3352:0:(client.c:2215:ptlrpc_check_set()) Completed RPC req@00000000949e3fea pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:lustre-MDT0000-lwp-OST0000_UUID:3352:1862456998099584:0@lo:38:kworker/u8:2.0 00000100:00100000:1.0:1776178569.058560:0:3352:0:(client.c:1733:ptlrpc_send_new_req()) Sending RPC req@00000000239d7b9a pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:lustre-MDT0001-mdtlov_UUID:3352:1862456998099840:0@lo:38:kworker/u8:2.0 00000100:00100000:1.0:1776178569.058589:0:3352:0:(events.c:361:request_in_callback()) peer: 12345-0@lo (source: 12345-0@lo) 00000100:00100000:1.0:1776178569.058611:0:3352:0:(client.c:1733:ptlrpc_send_new_req()) Sending RPC req@00000000dab5abff pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:lustre-MDT0000-lwp-MDT0001_UUID:3352:1862456998100032:0@lo:38:kworker/u8:2.0 00000100:00100000:1.0:1776178569.058630:0:3352:0:(events.c:361:request_in_callback()) peer: 12345-0@lo (source: 12345-0@lo) 00000100:00080000:1.0:1776178569.058692:0:3352:0:(import.c:86:import_set_state_nolock()) 00000000dfc8db13 lustre-MDT0000_UUID: changing import state from CONNECTING to DISCONN 00000100:00080000:1.0:1776178569.058696:0:3352:0:(import.c:1429:ptlrpc_connect_interpret()) recovery of lustre-MDT0000_UUID on 192.168.204.103@tcp failed (-19) 00000100:00100000:1.0:1776178569.058699:0:3352:0:(client.c:2215:ptlrpc_check_set()) Completed RPC req@00000000606788fe pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:lustre-MDT0000-lwp-OST0001_UUID:3352:1862456998099712:0@lo:38:kworker/u8:2.0 00000100:00100000:3.0:1776178569.058766:0:6257:0:(service.c:2152:ptlrpc_server_handle_req_in()) got req x1862456998100032 00000100:00100000:3.0:1776178569.058833:0:6257:0:(nrs_fifo.c:179:nrs_fifo_req_get()) NRS start fifo request from 12345-0@lo, seq: 6911 00000100:00100000:3.0:1776178569.058837:0:6257:0:(service.c:2324:ptlrpc_server_handle_request()) Handling RPC req@00000000b0409689 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_002:0+-99:3352:x1862456998100032:12345-0@lo:38:kworker/u8:2.0 00010000:02000000:3.0:1776178569.058848:0:6257:0:(ldlm_lib.c:1139:target_handle_connect()) lustre-MDT0000: Not available for connect from 0@lo (stopping) 00000100:00100000:3.0:1776178569.058894:0:6257:0:(service.c:2372:ptlrpc_server_handle_request()) Handled RPC req@00000000b0409689 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_002:0+-99:3352:x1862456998100032:12345-0@lo:38:kworker/u8:2.0 Request processed in 56us (264us total) trans 0 rc -19/-19 00000100:00100000:3.0:1776178569.058900:0:6257:0:(nrs_fifo.c:241:nrs_fifo_req_stop()) NRS stop fifo request from 12345-0@lo, seq: 6911 00000100:00080000:1.0:1776178569.059088:0:3352:0:(import.c:86:import_set_state_nolock()) 00000000a7c55b1f lustre-MDT0000_UUID: changing import state from CONNECTING to DISCONN 00000100:00080000:1.0:1776178569.059093:0:3352:0:(import.c:1429:ptlrpc_connect_interpret()) recovery of lustre-MDT0000_UUID on 192.168.204.103@tcp failed (-19) 00000100:00100000:1.0:1776178569.059096:0:3352:0:(client.c:2215:ptlrpc_check_set()) Completed RPC req@00000000dab5abff pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:lustre-MDT0000-lwp-MDT0001_UUID:3352:1862456998100032:0@lo:38:kworker/u8:2.0 00000100:00100000:2.0:1776178569.059928:0:15213:0:(service.c:2152:ptlrpc_server_handle_req_in()) got req x1862456998099840 00000100:00100000:2.0:1776178569.059977:0:15213:0:(nrs_fifo.c:179:nrs_fifo_req_get()) NRS start fifo request from 12345-0@lo, seq: 779 00000100:00100000:2.0:1776178569.059981:0:15213:0:(service.c:2324:ptlrpc_server_handle_request()) Handling RPC req@00000000ec65370c pname:cluuid+ref:pid:xid:nid:opc:job mdt_out00_003:0+-99:3352:x1862456998099840:12345-0@lo:38:kworker/u8:2.0 00010000:02000000:2.0:1776178569.060104:0:15213:0:(ldlm_lib.c:1139:target_handle_connect()) lustre-MDT0000: Not available for connect from 0@lo (stopping) 00000100:00100000:2.0:1776178569.060172:0:15213:0:(service.c:2372:ptlrpc_server_handle_request()) Handled RPC req@00000000ec65370c pname:cluuid+ref:pid:xid:nid:opc:job mdt_out00_003:0+-99:3352:x1862456998099840:12345-0@lo:38:kworker/u8:2.0 Request processed in 189us (1582us total) trans 0 rc -19/-19 00000100:00100000:2.0:1776178569.060179:0:15213:0:(nrs_fifo.c:241:nrs_fifo_req_stop()) NRS stop fifo request from 12345-0@lo, seq: 779 00000100:00080000:1.0:1776178569.060237:0:3352:0:(import.c:86:import_set_state_nolock()) 00000000ba4f1821 lustre-MDT0000_UUID: changing import state from CONNECTING to DISCONN 00000100:00080000:1.0:1776178569.060242:0:3352:0:(import.c:1429:ptlrpc_connect_interpret()) recovery of lustre-MDT0000_UUID on 192.168.204.103@tcp failed (-19) 00000100:00100000:1.0:1776178569.060246:0:3352:0:(client.c:2215:ptlrpc_check_set()) Completed RPC req@00000000239d7b9a pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:lustre-MDT0001-mdtlov_UUID:3352:1862456998099840:0@lo:38:kworker/u8:2.0 00010000:00010000:2.0:1776178569.696892:0:6244:0:(ldlm_lockd.c:2792:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:2.0:1776178570.593227:0:3355:0:(client.c:1733:ptlrpc_send_new_req()) Sending RPC req@000000002973935f pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_02:lustre-MDT0001-mdtlov_UUID:3355:1862456998100096:0@lo:13:osp-pre-1-1.0 00000100:00100000:2.0:1776178570.593297:0:3355:0:(events.c:361:request_in_callback()) peer: 12345-0@lo (source: 12345-0@lo) 00000100:00100000:1.0:1776178570.593504:0:9554:0:(service.c:2152:ptlrpc_server_handle_req_in()) got req x1862456998100096 00000100:00100000:1.0:1776178570.593553:0:9554:0:(nrs_fifo.c:179:nrs_fifo_req_get()) NRS start fifo request from 12345-0@lo, seq: 631 00000100:00100000:1.0:1776178570.593559:0:9554:0:(service.c:2324:ptlrpc_server_handle_request()) Handling RPC req@000000008f48ae7c pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_create00:lustre-MDT0001-mdtlov_UUID+5:3355:x1862456998100096:12345-0@lo:13:osp-pre-1-1.0 00002000:00000024:1.0:1776178570.593587:0:9554:0:(ofd_obd.c:740:ofd_statfs()) blocks cached 229376 granted 491746368 pending 0 free 3923689472 avail 3692167168 00000100:00100000:1.0:1776178570.593648:0:9554:0:(service.c:2372:ptlrpc_server_handle_request()) Handled RPC req@000000008f48ae7c pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_create00:lustre-MDT0001-mdtlov_UUID+5:3355:x1862456998100096:12345-0@lo:13:osp-pre-1-1.0 Request processed in 88us (352us total) trans 0 rc 0/0 00000100:00100000:1.0:1776178570.593655:0:9554:0:(nrs_fifo.c:241:nrs_fifo_req_stop()) NRS stop fifo request from 12345-0@lo, seq: 631 00000100:00100000:2.0:1776178570.593927:0:3355:0:(client.c:2215:ptlrpc_check_set()) Completed RPC req@000000002973935f pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_02:lustre-MDT0001-mdtlov_UUID:3355:1862456998100096:0@lo:13:osp-pre-1-1.0 00010000:00010000:1.0:1776178570.723331:0:6245:0:(ldlm_lockd.c:2792:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:3.0:1776178571.104842:0:3355:0:(ptlrpcd.c:415:ptlrpcd_check()) transfer 1 async RPCs [3->2] 00000100:00100000:3.0:1776178571.104867:0:3355:0:(client.c:1733:ptlrpc_send_new_req()) Sending RPC req@0000000083bc4c0f pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_02:lustre-MDT0001-mdtlov_UUID:3355:1862456998100160:0@lo:13:osp-pre-0-1.0 00000100:00100000:3.0:1776178571.104933:0:3355:0:(events.c:361:request_in_callback()) peer: 12345-0@lo (source: 12345-0@lo) 00000100:00100000:1.0:1776178571.105041:0:9554:0:(service.c:2152:ptlrpc_server_handle_req_in()) got req x1862456998100160 00000100:00100000:1.0:1776178571.105114:0:9554:0:(nrs_fifo.c:179:nrs_fifo_req_get()) NRS start fifo request from 12345-0@lo, seq: 632 00000100:00100000:1.0:1776178571.105120:0:9554:0:(service.c:2324:ptlrpc_server_handle_request()) Handling RPC req@00000000812a7ebe pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_create00:lustre-MDT0001-mdtlov_UUID+5:3355:x1862456998100160:12345-0@lo:13:osp-pre-0-1.0 00002000:00000024:1.0:1776178571.105149:0:9554:0:(ofd_obd.c:740:ofd_statfs()) blocks cached 315392 granted 484883776 pending 0 free 3923689472 avail 3692167168 00000100:00100000:1.0:1776178571.105207:0:9554:0:(service.c:2372:ptlrpc_server_handle_request()) Handled RPC req@00000000812a7ebe pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_create00:lustre-MDT0001-mdtlov_UUID+5:3355:x1862456998100160:12345-0@lo:13:osp-pre-0-1.0 Request processed in 87us (274us total) trans 0 rc 0/0 00000100:00100000:1.0:1776178571.105215:0:9554:0:(nrs_fifo.c:241:nrs_fifo_req_stop()) NRS stop fifo request from 12345-0@lo, seq: 632 00000100:00100000:0.0:1776178571.106431:0:3355:0:(client.c:2215:ptlrpc_check_set()) Completed RPC req@0000000083bc4c0f pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_02:lustre-MDT0001-mdtlov_UUID:3355:1862456998100160:0@lo:13:osp-pre-0-1.0 00010000:00010000:1.0:1776178571.744479:0:9224:0:(ldlm_lockd.c:2792:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:0.0:1776178572.586421:0:3346:0:(events.c:361:request_in_callback()) peer: 12345-192.168.204.3@tcp (source: 12345-192.168.204.3@tcp) 00000100:00100000:0.0:1776178572.586479:0:3346:0:(events.c:361:request_in_callback()) peer: 12345-192.168.204.3@tcp (source: 12345-192.168.204.3@tcp) 00000100:00100000:0.0:1776178572.586531:0:3346:0:(events.c:361:request_in_callback()) peer: 12345-192.168.204.3@tcp (source: 12345-192.168.204.3@tcp) 00000100:00100000:0.0:1776178572.586561:0:3346:0:(events.c:361:request_in_callback()) peer: 12345-192.168.204.3@tcp (source: 12345-192.168.204.3@tcp) 00000100:00100000:0.0:1776178572.586598:0:3346:0:(events.c:361:request_in_callback()) peer: 12345-192.168.204.3@tcp (source: 12345-192.168.204.3@tcp) 00000100:00100000:0.0:1776178572.586644:0:20755:0:(service.c:2152:ptlrpc_server_handle_req_in()) got req x1862456979132288 00000100:00100000:0.0:1776178572.586719:0:20755:0:(service.c:2152:ptlrpc_server_handle_req_in()) got req x1862456979132224 00000100:00100000:0.0:1776178572.586743:0:20755:0:(nrs_fifo.c:179:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.204.3@tcp, seq: 799 00000100:00100000:0.0:1776178572.586751:0:20755:0:(service.c:2324:ptlrpc_server_handle_request()) Handling RPC req@00000000dab5abff pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_006:b50963bd-8eba-4432-9a54-8888d00e3fa1+9:2270:x1862456979132288:12345-192.168.204.3@tcp:400:kworker/u8:3.0 00000100:00100000:0.0:1776178572.586848:0:20755:0:(service.c:2372:ptlrpc_server_handle_request()) Handled RPC req@00000000dab5abff pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_006:b50963bd-8eba-4432-9a54-8888d00e3fa1+9:2270:x1862456979132288:12345-192.168.204.3@tcp:400:kworker/u8:3.0 Request processed in 96us (369us total) trans 0 rc 0/0 00000100:00100000:0.0:1776178572.586856:0:20755:0:(nrs_fifo.c:241:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.204.3@tcp, seq: 799 00000100:00100000:0.0:1776178572.586892:0:20755:0:(nrs_fifo.c:179:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.204.3@tcp, seq: 800 00000100:00100000:0.0:1776178572.586896:0:20755:0:(service.c:2324:ptlrpc_server_handle_request()) Handling RPC req@00000000606788fe pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_006:b50963bd-8eba-4432-9a54-8888d00e3fa1+8:2270:x1862456979132224:12345-192.168.204.3@tcp:400:kworker/u8:3.0 00000100:00100000:1.0:1776178572.587201:0:11281:0:(service.c:2152:ptlrpc_server_handle_req_in()) got req x1862456979132160 00000100:00100000:1.0:1776178572.587238:0:11281:0:(service.c:2152:ptlrpc_server_handle_req_in()) got req x1862456979132096 00000100:00100000:1.0:1776178572.587255:0:11281:0:(nrs_fifo.c:179:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.204.3@tcp, seq: 1144 00000100:00100000:1.0:1776178572.587270:0:11281:0:(service.c:2324:ptlrpc_server_handle_request()) Handling RPC req@0000000083bc4c0f pname:cluuid+ref:pid:xid:nid:opc:job mdt00_004:b50963bd-8eba-4432-9a54-8888d00e3fa1+6:2270:x1862456979132160:12345-192.168.204.3@tcp:400:kworker/u8:3.0 00000100:00100000:1.0:1776178572.587309:0:11281:0:(service.c:2372:ptlrpc_server_handle_request()) Handled RPC req@0000000083bc4c0f pname:cluuid+ref:pid:xid:nid:opc:job mdt00_004:b50963bd-8eba-4432-9a54-8888d00e3fa1+6:2270:x1862456979132160:12345-192.168.204.3@tcp:400:kworker/u8:3.0 Request processed in 49us (901us total) trans 0 rc 0/0 00000100:00100000:1.0:1776178572.587316:0:11281:0:(nrs_fifo.c:241:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.204.3@tcp, seq: 1144 00000100:00100000:1.0:1776178572.587332:0:11281:0:(nrs_fifo.c:179:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.204.3@tcp, seq: 6912 00000100:00100000:1.0:1776178572.587336:0:11281:0:(service.c:2324:ptlrpc_server_handle_request()) Handling RPC req@00000000949e3fea pname:cluuid+ref:pid:xid:nid:opc:job mdt00_004:0+-99:2269:x1862456979132096:12345-192.168.204.3@tcp:38:kworker/u8:3.0 00010000:02000000:1.0:1776178572.587354:0:11281:0:(ldlm_lib.c:1139:target_handle_connect()) lustre-MDT0000: Not available for connect from 192.168.204.3@tcp (stopping) 00000100:00100000:1.0:1776178572.587384:0:11281:0:(service.c:2372:ptlrpc_server_handle_request()) Handled RPC req@00000000949e3fea pname:cluuid+ref:pid:xid:nid:opc:job mdt00_004:0+-99:2269:x1862456979132096:12345-192.168.204.3@tcp:38:kworker/u8:3.0 Request processed in 47us (785us total) trans 0 rc -19/-19 00000100:00100000:1.0:1776178572.587390:0:11281:0:(nrs_fifo.c:241:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.204.3@tcp, seq: 6912 00000100:00100000:2.0:1776178572.587776:0:20140:0:(service.c:2152:ptlrpc_server_handle_req_in()) got req x1862456979132032 00000100:00100000:2.0:1776178572.587817:0:20140:0:(nrs_fifo.c:179:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.204.3@tcp, seq: 299 00000100:00100000:2.0:1776178572.587821:0:20140:0:(service.c:2324:ptlrpc_server_handle_request()) Handling RPC req@00000000f8a3917d pname:cluuid+ref:pid:xid:nid:opc:job ll_mgs_0002:5b5c66b1-1a2f-4651-8548-0c75b9a38a94+9:2270:x1862456979132032:12345-192.168.204.3@tcp:400:kworker/u8:3.0 00000100:00100000:2.0:1776178572.587862:0:20140:0:(service.c:2372:ptlrpc_server_handle_request()) Handled RPC req@00000000f8a3917d pname:cluuid+ref:pid:xid:nid:opc:job ll_mgs_0002:5b5c66b1-1a2f-4651-8548-0c75b9a38a94+9:2270:x1862456979132032:12345-192.168.204.3@tcp:400:kworker/u8:3.0 Request processed in 39us (1330us total) trans 0 rc 0/0 00000100:00100000:2.0:1776178572.587881:0:20140:0:(nrs_fifo.c:241:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.204.3@tcp, seq: 299 00000100:00100000:0.0:1776178572.589462:0:20755:0:(service.c:2372:ptlrpc_server_handle_request()) Handled RPC req@00000000606788fe pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_006:b50963bd-8eba-4432-9a54-8888d00e3fa1+8:2270:x1862456979132224:12345-192.168.204.3@tcp:400:kworker/u8:3.0 Request processed in 2557us (2893us total) trans 0 rc 0/0 00000100:00100000:0.0:1776178572.589478:0:20755:0:(nrs_fifo.c:241:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.204.3@tcp, seq: 800 00010000:00010000:1.0:1776178572.769336:0:9225:0:(ldlm_lockd.c:2792:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00010000:00010000:2.0:1776178573.793943:0:6244:0:(ldlm_lockd.c:2792:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:0.0:1776178574.176472:0:3353:0:(client.c:1733:ptlrpc_send_new_req()) Sending RPC req@000000008c3fbdb6 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_00:ac61f839-0e8e-4cac-9c59-08bcfbbad330:3353:1862456998100224:0@lo:400:kworker/u8:2.0 00000100:00100000:0.0:1776178574.176608:0:3353:0:(events.c:361:request_in_callback()) peer: 12345-0@lo (source: 12345-0@lo) 00000100:00080000:1.0:1776178574.176778:0:37:0:(pinger.c:251:ptlrpc_pinger_process_import()) lustre-MDT0000-lwp-OST0000_UUID->lustre-MDT0000_UUID: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00080000:1.0:1776178574.176785:0:37:0:(recover.c:57:ptlrpc_initiate_recovery()) lustre-MDT0000_UUID: starting recovery 00000100:00080000:1.0:1776178574.176790:0:37:0:(import.c:86:import_set_state_nolock()) 0000000047601a98 lustre-MDT0000_UUID: changing import state from DISCONN to CONNECTING 00000100:00080000:1.0:1776178574.176796:0:37:0:(import.c:534:import_select_connection()) lustre-MDT0000-lwp-OST0000: connect to NID 0@lo last attempt 1276 00000100:00080000:1.0:1776178574.176800:0:37:0:(import.c:581:import_select_connection()) lustre-MDT0000-lwp-OST0000: tried all connections, increasing latency to 21s 00000100:00080000:1.0:1776178574.176807:0:37:0:(import.c:615:import_select_connection()) lustre-MDT0000-lwp-OST0000: import 0000000047601a98 using connection 192.168.204.103@tcp/0@lo 00000100:00100000:1.0:1776178574.176844:0:37:0:(import.c:807:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5) req@0000000049b1b339 x1862456998100288/t0(0) o38->lustre-MDT0000-lwp-OST0000@0@lo:12/10 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' 00000100:00100000:3.0:1776178574.176951:0:3354:0:(client.c:1733:ptlrpc_send_new_req()) Sending RPC req@0000000039ff8335 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_01:lustre-MDT0001-lwp-OST0000_UUID:3354:1862456998100352:0@lo:400:kworker/u8:2.0 00000100:00100000:3.0:1776178574.177031:0:3354:0:(events.c:361:request_in_callback()) peer: 12345-0@lo (source: 12345-0@lo) 00000100:00100000:3.0:1776178574.177156:0:6257:0:(service.c:2152:ptlrpc_server_handle_req_in()) got req x1862456998100352 00000100:00100000:3.0:1776178574.177224:0:6257:0:(nrs_fifo.c:179:nrs_fifo_req_get()) NRS start fifo request from 12345-0@lo, seq: 1145 00000100:00100000:3.0:1776178574.177230:0:6257:0:(service.c:2324:ptlrpc_server_handle_request()) Handling RPC req@000000002973935f pname:cluuid+ref:pid:xid:nid:opc:job mdt00_002:lustre-MDT0001-lwp-OST0000_UUID+5:3354:x1862456998100352:12345-0@lo:400:kworker/u8:2.0 00000100:00100000:3.0:1776178574.177291:0:6257:0:(service.c:2372:ptlrpc_server_handle_request()) Handled RPC req@000000002973935f pname:cluuid+ref:pid:xid:nid:opc:job mdt00_002:lustre-MDT0001-lwp-OST0000_UUID+5:3354:x1862456998100352:12345-0@lo:400:kworker/u8:2.0 Request processed in 60us (263us total) trans 0 rc 0/0 00000100:00100000:3.0:1776178574.177298:0:6257:0:(nrs_fifo.c:241:nrs_fifo_req_stop()) NRS stop fifo request from 12345-0@lo, seq: 1145 00000100:00100000:3.0:1776178574.177348:0:3354:0:(client.c:2215:ptlrpc_check_set()) Completed RPC req@0000000039ff8335 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_01:lustre-MDT0001-lwp-OST0000_UUID:3354:1862456998100352:0@lo:400:kworker/u8:2.0 00000100:00100000:3.0:1776178574.177433:0:3352:0:(client.c:1733:ptlrpc_send_new_req()) Sending RPC req@0000000049b1b339 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:lustre-MDT0000-lwp-OST0000_UUID:3352:1862456998100288:0@lo:38:kworker/u8:2.0 00000100:00100000:3.0:1776178574.177477:0:3352:0:(events.c:361:request_in_callback()) peer: 12345-0@lo (source: 12345-0@lo) 00000100:00100000:3.0:1776178574.177523:0:6256:0:(service.c:2152:ptlrpc_server_handle_req_in()) got req x1862456998100288 00000100:00100000:3.0:1776178574.177556:0:6256:0:(nrs_fifo.c:179:nrs_fifo_req_get()) NRS start fifo request from 12345-0@lo, seq: 6913 00000100:00100000:3.0:1776178574.177563:0:6256:0:(service.c:2324:ptlrpc_server_handle_request()) Handling RPC req@000000003316d46e pname:cluuid+ref:pid:xid:nid:opc:job mdt00_001:0+-99:3352:x1862456998100288:12345-0@lo:38:kworker/u8:2.0 00010000:02000000:3.0:1776178574.177588:0:6256:0:(ldlm_lib.c:1139:target_handle_connect()) lustre-MDT0000: Not available for connect from 0@lo (stopping) 00000100:00100000:3.0:1776178574.177659:0:6256:0:(service.c:2372:ptlrpc_server_handle_request()) Handled RPC req@000000003316d46e pname:cluuid+ref:pid:xid:nid:opc:job mdt00_001:0+-99:3352:x1862456998100288:12345-0@lo:38:kworker/u8:2.0 Request processed in 78us (165us total) trans 0 rc -19/-19 00000100:00100000:3.0:1776178574.177669:0:6256:0:(nrs_fifo.c:241:nrs_fifo_req_stop()) NRS stop fifo request from 12345-0@lo, seq: 6913 00000100:00080000:3.0:1776178574.177701:0:3352:0:(import.c:86:import_set_state_nolock()) 0000000047601a98 lustre-MDT0000_UUID: changing import state from CONNECTING to DISCONN 00000100:00080000:3.0:1776178574.177706:0:3352:0:(import.c:1429:ptlrpc_connect_interpret()) recovery of lustre-MDT0000_UUID on 192.168.204.103@tcp failed (-19) 00000100:00100000:3.0:1776178574.177709:0:3352:0:(client.c:2215:ptlrpc_check_set()) Completed RPC req@0000000049b1b339 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:lustre-MDT0000-lwp-OST0000_UUID:3352:1862456998100288:0@lo:38:kworker/u8:2.0 00000100:00100000:2.0:1776178574.177860:0:20137:0:(service.c:2152:ptlrpc_server_handle_req_in()) got req x1862456998100224 00000100:00100000:2.0:1776178574.178001:0:20137:0:(nrs_fifo.c:179:nrs_fifo_req_get()) NRS start fifo request from 12345-0@lo, seq: 300 00000100:00100000:2.0:1776178574.178009:0:20137:0:(service.c:2324:ptlrpc_server_handle_request()) Handling RPC req@000000006fd17377 pname:cluuid+ref:pid:xid:nid:opc:job ll_mgs_0000:ac61f839-0e8e-4cac-9c59-08bcfbbad330+18:3353:x1862456998100224:12345-0@lo:400:kworker/u8:2.0 00000100:00100000:2.0:1776178574.178107:0:20137:0:(service.c:2372:ptlrpc_server_handle_request()) Handled RPC req@000000006fd17377 pname:cluuid+ref:pid:xid:nid:opc:job ll_mgs_0000:ac61f839-0e8e-4cac-9c59-08bcfbbad330+18:3353:x1862456998100224:12345-0@lo:400:kworker/u8:2.0 Request processed in 98us (1502us total) trans 0 rc 0/0 00000100:00100000:2.0:1776178574.178114:0:20137:0:(nrs_fifo.c:241:nrs_fifo_req_stop()) NRS stop fifo request from 12345-0@lo, seq: 300 00000100:00080000:1.0:1776178574.178179:0:37:0:(pinger.c:251:ptlrpc_pinger_process_import()) lustre-MDT0000-lwp-OST0001_UUID->lustre-MDT0000_UUID: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00080000:1.0:1776178574.178186:0:37:0:(recover.c:57:ptlrpc_initiate_recovery()) lustre-MDT0000_UUID: starting recovery 00000100:00080000:1.0:1776178574.178189:0:37:0:(import.c:86:import_set_state_nolock()) 00000000dfc8db13 lustre-MDT0000_UUID: changing import state from DISCONN to CONNECTING 00000100:00080000:1.0:1776178574.178194:0:37:0:(import.c:534:import_select_connection()) lustre-MDT0000-lwp-OST0001: connect to NID 0@lo last attempt 1276 00000100:00080000:1.0:1776178574.178197:0:37:0:(import.c:581:import_select_connection()) lustre-MDT0000-lwp-OST0001: tried all connections, increasing latency to 21s 00000100:00080000:1.0:1776178574.178201:0:37:0:(import.c:615:import_select_connection()) lustre-MDT0000-lwp-OST0001: import 00000000dfc8db13 using connection 192.168.204.103@tcp/0@lo 00000100:00100000:1.0:1776178574.178228:0:37:0:(import.c:807:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5) req@000000007d60c0e9 x1862456998100416/t0(0) o38->lustre-MDT0000-lwp-OST0001@0@lo:12/10 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' 00000100:00080000:1.0:1776178574.178284:0:37:0:(pinger.c:251:ptlrpc_pinger_process_import()) lustre-MDT0001-mdtlov_UUID->lustre-MDT0000_UUID: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00080000:1.0:1776178574.178290:0:37:0:(recover.c:57:ptlrpc_initiate_recovery()) lustre-MDT0000_UUID: starting recovery 00000100:00080000:1.0:1776178574.178292:0:37:0:(import.c:86:import_set_state_nolock()) 00000000ba4f1821 lustre-MDT0000_UUID: changing import state from DISCONN to CONNECTING 00000100:00080000:1.0:1776178574.178297:0:37:0:(import.c:534:import_select_connection()) lustre-MDT0000-osp-MDT0001: connect to NID 0@lo last attempt 1276 00000100:00080000:1.0:1776178574.178299:0:37:0:(import.c:581:import_select_connection()) lustre-MDT0000-osp-MDT0001: tried all connections, increasing latency to 21s 00000100:00080000:1.0:1776178574.178305:0:37:0:(import.c:615:import_select_connection()) lustre-MDT0000-osp-MDT0001: import 00000000ba4f1821 using connection 192.168.204.103@tcp/0@lo 00000100:00100000:1.0:1776178574.178324:0:37:0:(import.c:807:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5) req@0000000049b1b339 x1862456998100544/t0(0) o38->lustre-MDT0000-osp-MDT0001@0@lo:24/4 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' 00000100:00080000:1.0:1776178574.178371:0:37:0:(pinger.c:251:ptlrpc_pinger_process_import()) lustre-MDT0000-lwp-MDT0001_UUID->lustre-MDT0000_UUID: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00080000:1.0:1776178574.178376:0:37:0:(recover.c:57:ptlrpc_initiate_recovery()) lustre-MDT0000_UUID: starting recovery 00000100:00080000:1.0:1776178574.178378:0:37:0:(import.c:86:import_set_state_nolock()) 00000000a7c55b1f lustre-MDT0000_UUID: changing import state from DISCONN to CONNECTING 00000100:00080000:1.0:1776178574.178383:0:37:0:(import.c:534:import_select_connection()) lustre-MDT0000-lwp-MDT0001: connect to NID 0@lo last attempt 1276 00000100:00080000:1.0:1776178574.178386:0:37:0:(import.c:581:import_select_connection()) lustre-MDT0000-lwp-MDT0001: tried all connections, increasing latency to 21s 00000100:00080000:1.0:1776178574.178391:0:37:0:(import.c:615:import_select_connection()) lustre-MDT0000-lwp-MDT0001: import 00000000a7c55b1f using connection 192.168.204.103@tcp/0@lo 00000100:00100000:1.0:1776178574.178408:0:37:0:(import.c:807:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5) req@00000000167dc455 x1862456998100736/t0(0) o38->lustre-MDT0000-lwp-MDT0001@0@lo:12/10 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' 00000100:00100000:1.0:1776178574.178441:0:3354:0:(ptlrpcd.c:415:ptlrpcd_check()) transfer 1 async RPCs [0->1] 00000100:00100000:1.0:1776178574.178457:0:3354:0:(client.c:1733:ptlrpc_send_new_req()) Sending RPC req@0000000023562990 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_01:lustre-MDT0001-mdtlov_UUID:3354:1862456998100672:0@lo:400:kworker/u8:2.0 00000100:00100000:1.0:1776178574.178519:0:3354:0:(events.c:361:request_in_callback()) peer: 12345-0@lo (source: 12345-0@lo) 00000100:00100000:1.0:1776178574.178580:0:3356:0:(client.c:1733:ptlrpc_send_new_req()) Sending RPC req@0000000039ff8335 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_03:lustre-MDT0001-mdtlov_UUID:3356:1862456998100608:0@lo:400:kworker/u8:2.0 00000100:00100000:1.0:1776178574.178619:0:3356:0:(events.c:361:request_in_callback()) peer: 12345-0@lo (source: 12345-0@lo) 00000100:00100000:1.0:1776178574.178665:0:3356:0:(ptlrpcd.c:415:ptlrpcd_check()) transfer 1 async RPCs [2->3] 00000100:00100000:1.0:1776178574.178676:0:3356:0:(client.c:1733:ptlrpc_send_new_req()) Sending RPC req@0000000022d4dc29 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_03:lustre-MDT0001-lwp-OST0001_UUID:3356:1862456998100480:0@lo:400:kworker/u8:2.0 00000100:00100000:1.0:1776178574.178706:0:3356:0:(events.c:361:request_in_callback()) peer: 12345-0@lo (source: 12345-0@lo) 00000100:00100000:1.0:1776178574.178732:0:6257:0:(service.c:2152:ptlrpc_server_handle_req_in()) got req x1862456998100480 00000100:00100000:1.0:1776178574.178763:0:6257:0:(nrs_fifo.c:179:nrs_fifo_req_get()) NRS start fifo request from 12345-0@lo, seq: 1146 00000100:00100000:1.0:1776178574.178767:0:6257:0:(service.c:2324:ptlrpc_server_handle_request()) Handling RPC req@00000000d0706d7c pname:cluuid+ref:pid:xid:nid:opc:job mdt00_002:lustre-MDT0001-lwp-OST0001_UUID+5:3356:x1862456998100480:12345-0@lo:400:kworker/u8:2.0 00000100:00100000:1.0:1776178574.178805:0:6257:0:(service.c:2372:ptlrpc_server_handle_request()) Handled RPC req@00000000d0706d7c pname:cluuid+ref:pid:xid:nid:opc:job mdt00_002:lustre-MDT0001-lwp-OST0001_UUID+5:3356:x1862456998100480:12345-0@lo:400:kworker/u8:2.0 Request processed in 37us (99us total) trans 0 rc 0/0 00000100:00100000:1.0:1776178574.178811:0:6257:0:(nrs_fifo.c:241:nrs_fifo_req_stop()) NRS stop fifo request from 12345-0@lo, seq: 1146 00000100:00100000:1.0:1776178574.178845:0:3356:0:(client.c:2215:ptlrpc_check_set()) Completed RPC req@0000000022d4dc29 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_03:lustre-MDT0001-lwp-OST0001_UUID:3356:1862456998100480:0@lo:400:kworker/u8:2.0 00000100:00100000:1.0:1776178574.178898:0:14440:0:(service.c:2152:ptlrpc_server_handle_req_in()) got req x1862456998100672 00000100:00100000:1.0:1776178574.178941:0:14440:0:(service.c:2152:ptlrpc_server_handle_req_in()) got req x1862456998100608 00000100:00100000:1.0:1776178574.178975:0:14440:0:(nrs_fifo.c:179:nrs_fifo_req_get()) NRS start fifo request from 12345-0@lo, seq: 801 00000100:00100000:1.0:1776178574.178983:0:14440:0:(service.c:2324:ptlrpc_server_handle_request()) Handling RPC req@00000000bcefda18 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_004:lustre-MDT0001-mdtlov_UUID+5:3354:x1862456998100672:12345-0@lo:400:kworker/u8:2.0 00000100:00100000:1.0:1776178574.179048:0:14440:0:(service.c:2372:ptlrpc_server_handle_request()) Handled RPC req@00000000bcefda18 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_004:lustre-MDT0001-mdtlov_UUID+5:3354:x1862456998100672:12345-0@lo:400:kworker/u8:2.0 Request processed in 64us (529us total) trans 0 rc 0/0 00000100:00100000:1.0:1776178574.179094:0:14440:0:(nrs_fifo.c:241:nrs_fifo_req_stop()) NRS stop fifo request from 12345-0@lo, seq: 801 00000100:00100000:1.0:1776178574.179106:0:14440:0:(nrs_fifo.c:179:nrs_fifo_req_get()) NRS start fifo request from 12345-0@lo, seq: 802 00000100:00100000:1.0:1776178574.179109:0:14440:0:(service.c:2324:ptlrpc_server_handle_request()) Handling RPC req@0000000038c09127 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_004:lustre-MDT0001-mdtlov_UUID+5:3356:x1862456998100608:12345-0@lo:400:kworker/u8:2.0 00000100:00100000:1.0:1776178574.179137:0:14440:0:(service.c:2372:ptlrpc_server_handle_request()) Handled RPC req@0000000038c09127 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_004:lustre-MDT0001-mdtlov_UUID+5:3356:x1862456998100608:12345-0@lo:400:kworker/u8:2.0 Request processed in 27us (520us total) trans 0 rc 0/0 00000100:00100000:1.0:1776178574.179143:0:14440:0:(nrs_fifo.c:241:nrs_fifo_req_stop()) NRS stop fifo request from 12345-0@lo, seq: 802 00000100:00100000:1.0:1776178574.179178:0:3356:0:(client.c:2215:ptlrpc_check_set()) Completed RPC req@0000000039ff8335 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_03:lustre-MDT0001-mdtlov_UUID:3356:1862456998100608:0@lo:400:kworker/u8:2.0 00000100:00100000:1.0:1776178574.179217:0:3354:0:(client.c:2215:ptlrpc_check_set()) Completed RPC req@0000000023562990 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_01:lustre-MDT0001-mdtlov_UUID:3354:1862456998100672:0@lo:400:kworker/u8:2.0 00000100:00100000:1.0:1776178574.179276:0:3352:0:(client.c:1733:ptlrpc_send_new_req()) Sending RPC req@000000007d60c0e9 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:lustre-MDT0000-lwp-OST0001_UUID:3352:1862456998100416:0@lo:38:kworker/u8:2.0 00000100:00100000:1.0:1776178574.179321:0:3352:0:(events.c:361:request_in_callback()) peer: 12345-0@lo (source: 12345-0@lo) 00000100:00100000:1.0:1776178574.179347:0:3352:0:(client.c:1733:ptlrpc_send_new_req()) Sending RPC req@0000000049b1b339 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:lustre-MDT0001-mdtlov_UUID:3352:1862456998100544:0@lo:38:kworker/u8:2.0 00000100:00100000:1.0:1776178574.179387:0:3352:0:(events.c:361:request_in_callback()) peer: 12345-0@lo (source: 12345-0@lo) 00000100:00100000:1.0:1776178574.179414:0:3352:0:(client.c:1733:ptlrpc_send_new_req()) Sending RPC req@00000000167dc455 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:lustre-MDT0000-lwp-MDT0001_UUID:3352:1862456998100736:0@lo:38:kworker/u8:2.0 00000100:00100000:1.0:1776178574.179453:0:3352:0:(events.c:361:request_in_callback()) peer: 12345-0@lo (source: 12345-0@lo) 00000100:00100000:1.0:1776178574.179493:0:6256:0:(service.c:2152:ptlrpc_server_handle_req_in()) got req x1862456998100416 00000100:00100000:1.0:1776178574.179527:0:6256:0:(service.c:2152:ptlrpc_server_handle_req_in()) got req x1862456998100736 00000100:00100000:1.0:1776178574.179552:0:6256:0:(nrs_fifo.c:179:nrs_fifo_req_get()) NRS start fifo request from 12345-0@lo, seq: 6914 00000100:00100000:1.0:1776178574.179558:0:6256:0:(service.c:2324:ptlrpc_server_handle_request()) Handling RPC req@0000000023562990 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_001:0+-99:3352:x1862456998100416:12345-0@lo:38:kworker/u8:2.0 00010000:02000000:1.0:1776178574.179578:0:6256:0:(ldlm_lib.c:1139:target_handle_connect()) lustre-MDT0000: Not available for connect from 0@lo (stopping) 00000100:00100000:1.0:1776178574.179631:0:6256:0:(service.c:2372:ptlrpc_server_handle_request()) Handled RPC req@0000000023562990 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_001:0+-99:3352:x1862456998100416:12345-0@lo:38:kworker/u8:2.0 Request processed in 71us (310us total) trans 0 rc -19/-19 00000100:00100000:1.0:1776178574.179640:0:6256:0:(nrs_fifo.c:241:nrs_fifo_req_stop()) NRS stop fifo request from 12345-0@lo, seq: 6914 00000100:00100000:1.0:1776178574.179676:0:6256:0:(nrs_fifo.c:179:nrs_fifo_req_get()) NRS start fifo request from 12345-0@lo, seq: 6915 00000100:00100000:1.0:1776178574.179682:0:6256:0:(service.c:2324:ptlrpc_server_handle_request()) Handling RPC req@00000000044bd507 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_001:0+-99:3352:x1862456998100736:12345-0@lo:38:kworker/u8:2.0 00010000:02000000:1.0:1776178574.179691:0:6256:0:(ldlm_lib.c:1139:target_handle_connect()) lustre-MDT0000: Not available for connect from 0@lo (stopping) 00000100:00100000:1.0:1776178574.179712:0:6256:0:(service.c:2372:ptlrpc_server_handle_request()) Handled RPC req@00000000044bd507 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_001:0+-99:3352:x1862456998100736:12345-0@lo:38:kworker/u8:2.0 Request processed in 30us (260us total) trans 0 rc -19/-19 00000100:00100000:1.0:1776178574.179717:0:6256:0:(nrs_fifo.c:241:nrs_fifo_req_stop()) NRS stop fifo request from 12345-0@lo, seq: 6915 00000100:00080000:1.0:1776178574.179743:0:3352:0:(import.c:86:import_set_state_nolock()) 00000000dfc8db13 lustre-MDT0000_UUID: changing import state from CONNECTING to DISCONN 00000100:00080000:1.0:1776178574.179747:0:3352:0:(import.c:1429:ptlrpc_connect_interpret()) recovery of lustre-MDT0000_UUID on 192.168.204.103@tcp failed (-19) 00000100:00100000:1.0:1776178574.179750:0:3352:0:(client.c:2215:ptlrpc_check_set()) Completed RPC req@000000007d60c0e9 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:lustre-MDT0000-lwp-OST0001_UUID:3352:1862456998100416:0@lo:38:kworker/u8:2.0 00000100:00080000:1.0:1776178574.179758:0:3352:0:(import.c:86:import_set_state_nolock()) 00000000a7c55b1f lustre-MDT0000_UUID: changing import state from CONNECTING to DISCONN 00000100:00080000:1.0:1776178574.179760:0:3352:0:(import.c:1429:ptlrpc_connect_interpret()) recovery of lustre-MDT0000_UUID on 192.168.204.103@tcp failed (-19) 00000100:00100000:1.0:1776178574.179763:0:3352:0:(client.c:2215:ptlrpc_check_set()) Completed RPC req@00000000167dc455 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:lustre-MDT0000-lwp-MDT0001_UUID:3352:1862456998100736:0@lo:38:kworker/u8:2.0 00000100:00100000:1.0:1776178574.179799:0:15213:0:(service.c:2152:ptlrpc_server_handle_req_in()) got req x1862456998100544 00000100:00100000:1.0:1776178574.179822:0:15213:0:(nrs_fifo.c:179:nrs_fifo_req_get()) NRS start fifo request from 12345-0@lo, seq: 780 00000100:00100000:1.0:1776178574.179825:0:15213:0:(service.c:2324:ptlrpc_server_handle_request()) Handling RPC req@0000000021b0d516 pname:cluuid+ref:pid:xid:nid:opc:job mdt_out00_003:0+-99:3352:x1862456998100544:12345-0@lo:38:kworker/u8:2.0 00010000:02000000:1.0:1776178574.179833:0:15213:0:(ldlm_lib.c:1139:target_handle_connect()) lustre-MDT0000: Not available for connect from 0@lo (stopping) 00000100:00100000:1.0:1776178574.179864:0:15213:0:(service.c:2372:ptlrpc_server_handle_request()) Handled RPC req@0000000021b0d516 pname:cluuid+ref:pid:xid:nid:opc:job mdt_out00_003:0+-99:3352:x1862456998100544:12345-0@lo:38:kworker/u8:2.0 Request processed in 37us (478us total) trans 0 rc -19/-19 00000100:00100000:1.0:1776178574.179869:0:15213:0:(nrs_fifo.c:241:nrs_fifo_req_stop()) NRS stop fifo request from 12345-0@lo, seq: 780 00000100:00080000:1.0:1776178574.179907:0:3352:0:(import.c:86:import_set_state_nolock()) 00000000ba4f1821 lustre-MDT0000_UUID: changing import state from CONNECTING to DISCONN 00000100:00080000:1.0:1776178574.179910:0:3352:0:(import.c:1429:ptlrpc_connect_interpret()) recovery of lustre-MDT0000_UUID on 192.168.204.103@tcp failed (-19) 00000100:00100000:1.0:1776178574.179913:0:3352:0:(client.c:2215:ptlrpc_check_set()) Completed RPC req@0000000049b1b339 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:lustre-MDT0001-mdtlov_UUID:3352:1862456998100544:0@lo:38:kworker/u8:2.0 00000100:00100000:0.0:1776178574.180055:0:3353:0:(client.c:2215:ptlrpc_check_set()) Completed RPC req@000000008c3fbdb6 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_00:ac61f839-0e8e-4cac-9c59-08bcfbbad330:3353:1862456998100224:0@lo:400:kworker/u8:2.0 00010000:00010000:1.0:1776178574.816323:0:6245:0:(ldlm_lockd.c:2792:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:0.0:1776178575.712479:0:3353:0:(ptlrpcd.c:415:ptlrpcd_check()) transfer 1 async RPCs [1->0] 00000100:00100000:0.0:1776178575.712511:0:3353:0:(client.c:1733:ptlrpc_send_new_req()) Sending RPC req@000000007216887e pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_00:lustre-MDT0001-mdtlov_UUID:3353:1862456998100800:0@lo:13:osp-pre-1-1.0 00000100:00100000:0.0:1776178575.712596:0:3353:0:(events.c:361:request_in_callback()) peer: 12345-0@lo (source: 12345-0@lo) 00000100:00100000:3.0:1776178575.712705:0:8231:0:(service.c:2152:ptlrpc_server_handle_req_in()) got req x1862456998100800 00000100:00100000:3.0:1776178575.712771:0:8231:0:(nrs_fifo.c:179:nrs_fifo_req_get()) NRS start fifo request from 12345-0@lo, seq: 633 00000100:00100000:3.0:1776178575.712778:0:8231:0:(service.c:2324:ptlrpc_server_handle_request()) Handling RPC req@000000009b04a931 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_create00:lustre-MDT0001-mdtlov_UUID+5:3353:x1862456998100800:12345-0@lo:13:osp-pre-1-1.0 00002000:00000024:3.0:1776178575.712826:0:8231:0:(ofd_obd.c:740:ofd_statfs()) blocks cached 229376 granted 491746368 pending 0 free 3923689472 avail 3692167168 00000100:00100000:3.0:1776178575.712881:0:8231:0:(service.c:2372:ptlrpc_server_handle_request()) Handled RPC req@000000009b04a931 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_create00:lustre-MDT0001-mdtlov_UUID+5:3353:x1862456998100800:12345-0@lo:13:osp-pre-1-1.0 Request processed in 103us (286us total) trans 0 rc 0/0 00000100:00100000:3.0:1776178575.712889:0:8231:0:(nrs_fifo.c:241:nrs_fifo_req_stop()) NRS stop fifo request from 12345-0@lo, seq: 633 00000100:00100000:2.0:1776178575.713114:0:3353:0:(client.c:2215:ptlrpc_check_set()) Completed RPC req@000000007216887e pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_00:lustre-MDT0001-mdtlov_UUID:3353:1862456998100800:0@lo:13:osp-pre-1-1.0 00010000:00010000:1.0:1776178575.841039:0:9224:0:(ldlm_lockd.c:2792:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:2.0:1776178576.224463:0:3355:0:(client.c:1733:ptlrpc_send_new_req()) Sending RPC req@00000000fd32fba5 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_02:lustre-MDT0001-mdtlov_UUID:3355:1862456998100864:0@lo:13:osp-pre-0-1.0 00000100:00100000:2.0:1776178576.224549:0:3355:0:(events.c:361:request_in_callback()) peer: 12345-0@lo (source: 12345-0@lo) 00000100:00100000:0.0:1776178576.224728:0:9554:0:(service.c:2152:ptlrpc_server_handle_req_in()) got req x1862456998100864 00000100:00100000:0.0:1776178576.224783:0:9554:0:(nrs_fifo.c:179:nrs_fifo_req_get()) NRS start fifo request from 12345-0@lo, seq: 634 00000100:00100000:0.0:1776178576.224788:0:9554:0:(service.c:2324:ptlrpc_server_handle_request()) Handling RPC req@00000000a430b18f pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_create00:lustre-MDT0001-mdtlov_UUID+5:3355:x1862456998100864:12345-0@lo:13:osp-pre-0-1.0 00002000:00000024:0.0:1776178576.224821:0:9554:0:(ofd_obd.c:740:ofd_statfs()) blocks cached 315392 granted 484883776 pending 0 free 3923689472 avail 3692167168 00000100:00100000:0.0:1776178576.224876:0:9554:0:(service.c:2372:ptlrpc_server_handle_request()) Handled RPC req@00000000a430b18f pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_create00:lustre-MDT0001-mdtlov_UUID+5:3355:x1862456998100864:12345-0@lo:13:osp-pre-0-1.0 Request processed in 86us (339us total) trans 0 rc 0/0 00000100:00100000:0.0:1776178576.224884:0:9554:0:(nrs_fifo.c:241:nrs_fifo_req_stop()) NRS stop fifo request from 12345-0@lo, seq: 634 00000100:00100000:2.0:1776178576.225141:0:3355:0:(client.c:2215:ptlrpc_check_set()) Completed RPC req@00000000fd32fba5 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_02:lustre-MDT0001-mdtlov_UUID:3355:1862456998100864:0@lo:13:osp-pre-0-1.0 00010000:00010000:1.0:1776178576.870131:0:9225:0:(ldlm_lockd.c:2792:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:1.0:1776178577.698414:0:3348:0:(events.c:361:request_in_callback()) peer: 12345-192.168.204.3@tcp (source: 12345-192.168.204.3@tcp) 00000100:00100000:2.0:1776178577.698551:0:8250:0:(service.c:2152:ptlrpc_server_handle_req_in()) got req x1862456979132416 00000100:00100000:2.0:1776178577.698632:0:8250:0:(nrs_fifo.c:179:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.204.3@tcp, seq: 6916 00000100:00100000:2.0:1776178577.698643:0:8250:0:(service.c:2324:ptlrpc_server_handle_request()) Handling RPC req@00000000167dc455 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_003:0+-99:2269:x1862456979132416:12345-192.168.204.3@tcp:38:kworker/u8:4.0 00010000:02000000:2.0:1776178577.698673:0:8250:0:(ldlm_lib.c:1139:target_handle_connect()) lustre-MDT0000: Not available for connect from 192.168.204.3@tcp (stopping) 00000100:00100000:2.0:1776178577.699908:0:8250:0:(service.c:2372:ptlrpc_server_handle_request()) Handled RPC req@00000000167dc455 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_003:0+-99:2269:x1862456979132416:12345-192.168.204.3@tcp:38:kworker/u8:4.0 Request processed in 1259us (1501us total) trans 0 rc -19/-19 00000100:00100000:2.0:1776178577.699923:0:8250:0:(nrs_fifo.c:241:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.204.3@tcp, seq: 6916 00000100:00100000:1.0:1776178577.794389:0:3346:0:(events.c:361:request_in_callback()) peer: 12345-192.168.204.3@tcp (source: 12345-192.168.204.3@tcp) 00000100:00100000:1.0:1776178577.794449:0:3346:0:(events.c:361:request_in_callback()) peer: 12345-192.168.204.3@tcp (source: 12345-192.168.204.3@tcp) 00000100:00100000:1.0:1776178577.794487:0:3346:0:(events.c:361:request_in_callback()) peer: 12345-192.168.204.3@tcp (source: 12345-192.168.204.3@tcp) 00000100:00100000:1.0:1776178577.794530:0:3346:0:(events.c:361:request_in_callback()) peer: 12345-192.168.204.3@tcp (source: 12345-192.168.204.3@tcp) 00000100:00100000:1.0:1776178577.794599:0:20759:0:(service.c:2152:ptlrpc_server_handle_req_in()) got req x1862456979132544 00000100:00100000:1.0:1776178577.794655:0:20759:0:(service.c:2152:ptlrpc_server_handle_req_in()) got req x1862456979132608 00000100:00100000:1.0:1776178577.794676:0:20759:0:(nrs_fifo.c:179:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.204.3@tcp, seq: 803 00000100:00100000:1.0:1776178577.794685:0:20759:0:(service.c:2324:ptlrpc_server_handle_request()) Handling RPC req@0000000049b1b339 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_010:b50963bd-8eba-4432-9a54-8888d00e3fa1+8:2272:x1862456979132544:12345-192.168.204.3@tcp:400:kworker/u8:4.0 00000100:00100000:1.0:1776178577.794782:0:20759:0:(service.c:2372:ptlrpc_server_handle_request()) Handled RPC req@0000000049b1b339 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_010:b50963bd-8eba-4432-9a54-8888d00e3fa1+8:2272:x1862456979132544:12345-192.168.204.3@tcp:400:kworker/u8:4.0 Request processed in 98us (333us total) trans 0 rc 0/0 00000100:00100000:1.0:1776178577.794790:0:20759:0:(nrs_fifo.c:241:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.204.3@tcp, seq: 803 00000100:00100000:1.0:1776178577.794807:0:20759:0:(nrs_fifo.c:179:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.204.3@tcp, seq: 804 00000100:00100000:1.0:1776178577.794812:0:20759:0:(service.c:2324:ptlrpc_server_handle_request()) Handling RPC req@0000000039ff8335 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_010:b50963bd-8eba-4432-9a54-8888d00e3fa1+9:2272:x1862456979132608:12345-192.168.204.3@tcp:400:kworker/u8:4.0 00000100:00100000:2.0:1776178577.797417:0:20137:0:(service.c:2152:ptlrpc_server_handle_req_in()) got req x1862456979132352 00000100:00100000:2.0:1776178577.797502:0:20137:0:(nrs_fifo.c:179:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.204.3@tcp, seq: 301 00000100:00100000:2.0:1776178577.797509:0:20137:0:(service.c:2324:ptlrpc_server_handle_request()) Handling RPC req@00000000eb350ccb pname:cluuid+ref:pid:xid:nid:opc:job ll_mgs_0000:5b5c66b1-1a2f-4651-8548-0c75b9a38a94+9:2272:x1862456979132352:12345-192.168.204.3@tcp:400:kworker/u8:4.0 00000100:00100000:1.0:1776178577.797664:0:20759:0:(service.c:2372:ptlrpc_server_handle_request()) Handled RPC req@0000000039ff8335 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_010:b50963bd-8eba-4432-9a54-8888d00e3fa1+9:2272:x1862456979132608:12345-192.168.204.3@tcp:400:kworker/u8:4.0 Request processed in 2849us (3133us total) trans 0 rc 0/0 00000100:00100000:1.0:1776178577.797676:0:20759:0:(nrs_fifo.c:241:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.204.3@tcp, seq: 804 00000100:00100000:3.0:1776178577.801270:0:8250:0:(service.c:2152:ptlrpc_server_handle_req_in()) got req x1862456979132480 00000100:00100000:3.0:1776178577.801364:0:8250:0:(nrs_fifo.c:179:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.204.3@tcp, seq: 1147 00000100:00100000:3.0:1776178577.801373:0:8250:0:(service.c:2324:ptlrpc_server_handle_request()) Handling RPC req@000000007d60c0e9 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_003:b50963bd-8eba-4432-9a54-8888d00e3fa1+6:2272:x1862456979132480:12345-192.168.204.3@tcp:400:kworker/u8:4.0 00000100:00100000:2.0:1776178577.816704:0:20137:0:(service.c:2372:ptlrpc_server_handle_request()) Handled RPC req@00000000eb350ccb pname:cluuid+ref:pid:xid:nid:opc:job ll_mgs_0000:5b5c66b1-1a2f-4651-8548-0c75b9a38a94+9:2272:x1862456979132352:12345-192.168.204.3@tcp:400:kworker/u8:4.0 Request processed in 19180us (22312us total) trans 0 rc 0/0 00000100:00100000:2.0:1776178577.816725:0:20137:0:(nrs_fifo.c:241:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.204.3@tcp, seq: 301 00000100:00100000:3.0:1776178577.817511:0:8250:0:(service.c:2372:ptlrpc_server_handle_request()) Handled RPC req@000000007d60c0e9 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_003:b50963bd-8eba-4432-9a54-8888d00e3fa1+6:2272:x1862456979132480:12345-192.168.204.3@tcp:400:kworker/u8:4.0 Request processed in 16133us (23020us total) trans 0 rc 0/0 00000100:00100000:3.0:1776178577.817524:0:8250:0:(nrs_fifo.c:241:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.204.3@tcp, seq: 1147 00010000:00010000:2.0:1776178577.909294:0:6244:0:(ldlm_lockd.c:2792:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00010000:00010000:1.0:1776178578.912296:0:6245:0:(ldlm_lockd.c:2792:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:2.0:1776178579.298977:0:3356:0:(client.c:1733:ptlrpc_send_new_req()) Sending RPC req@0000000022d4dc29 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_03:ac61f839-0e8e-4cac-9c59-08bcfbbad330:3356:1862456998100928:0@lo:400:kworker/u8:2.0 00000100:00100000:2.0:1776178579.299152:0:3356:0:(events.c:361:request_in_callback()) peer: 12345-0@lo (source: 12345-0@lo) 00000100:00100000:3.0:1776178579.303666:0:20137:0:(service.c:2152:ptlrpc_server_handle_req_in()) got req x1862456998100928 00000100:00100000:3.0:1776178579.303755:0:20137:0:(nrs_fifo.c:179:nrs_fifo_req_get()) NRS start fifo request from 12345-0@lo, seq: 302 00000100:00100000:3.0:1776178579.303763:0:20137:0:(service.c:2324:ptlrpc_server_handle_request()) Handling RPC req@0000000048e8901d pname:cluuid+ref:pid:xid:nid:opc:job ll_mgs_0000:ac61f839-0e8e-4cac-9c59-08bcfbbad330+18:3356:x1862456998100928:12345-0@lo:400:kworker/u8:2.0 00000100:00100000:3.0:1776178579.303891:0:20137:0:(service.c:2372:ptlrpc_server_handle_request()) Handled RPC req@0000000048e8901d pname:cluuid+ref:pid:xid:nid:opc:job ll_mgs_0000:ac61f839-0e8e-4cac-9c59-08bcfbbad330+18:3356:x1862456998100928:12345-0@lo:400:kworker/u8:2.0 Request processed in 128us (4741us total) trans 0 rc 0/0 00000100:00100000:3.0:1776178579.303899:0:20137:0:(nrs_fifo.c:241:nrs_fifo_req_stop()) NRS stop fifo request from 12345-0@lo, seq: 302 00000100:00100000:2.0:1776178579.305759:0:3356:0:(client.c:2215:ptlrpc_check_set()) Completed RPC req@0000000022d4dc29 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_03:ac61f839-0e8e-4cac-9c59-08bcfbbad330:3356:1862456998100928:0@lo:400:kworker/u8:2.0 00000100:00080000:1.0:1776178579.310018:0:37:0:(pinger.c:251:ptlrpc_pinger_process_import()) lustre-MDT0000-lwp-OST0000_UUID->lustre-MDT0000_UUID: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00080000:1.0:1776178579.310039:0:37:0:(recover.c:57:ptlrpc_initiate_recovery()) lustre-MDT0000_UUID: starting recovery 00000100:00080000:1.0:1776178579.310044:0:37:0:(import.c:86:import_set_state_nolock()) 0000000047601a98 lustre-MDT0000_UUID: changing import state from DISCONN to CONNECTING 00000100:00080000:1.0:1776178579.310057:0:37:0:(import.c:534:import_select_connection()) lustre-MDT0000-lwp-OST0000: connect to NID 0@lo last attempt 1281 00000100:00080000:1.0:1776178579.310091:0:37:0:(import.c:581:import_select_connection()) lustre-MDT0000-lwp-OST0000: tried all connections, increasing latency to 21s 00000100:00080000:1.0:1776178579.310102:0:37:0:(import.c:615:import_select_connection()) lustre-MDT0000-lwp-OST0000: import 0000000047601a98 using connection 192.168.204.103@tcp/0@lo 00000100:00100000:1.0:1776178579.310173:0:37:0:(import.c:807:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5) req@0000000022d4dc29 x1862456998100992/t0(0) o38->lustre-MDT0000-lwp-OST0000@0@lo:12/10 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' 00000100:00080000:1.0:1776178579.310232:0:37:0:(pinger.c:251:ptlrpc_pinger_process_import()) lustre-MDT0000-lwp-OST0001_UUID->lustre-MDT0000_UUID: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00080000:1.0:1776178579.310236:0:37:0:(recover.c:57:ptlrpc_initiate_recovery()) lustre-MDT0000_UUID: starting recovery 00000100:00080000:1.0:1776178579.310237:0:37:0:(import.c:86:import_set_state_nolock()) 00000000dfc8db13 lustre-MDT0000_UUID: changing import state from DISCONN to CONNECTING 00000100:00080000:1.0:1776178579.310241:0:37:0:(import.c:534:import_select_connection()) lustre-MDT0000-lwp-OST0001: connect to NID 0@lo last attempt 1281 00000100:00080000:1.0:1776178579.310243:0:37:0:(import.c:581:import_select_connection()) lustre-MDT0000-lwp-OST0001: tried all connections, increasing latency to 21s 00000100:00080000:1.0:1776178579.310246:0:37:0:(import.c:615:import_select_connection()) lustre-MDT0000-lwp-OST0001: import 00000000dfc8db13 using connection 192.168.204.103@tcp/0@lo 00000100:00100000:1.0:1776178579.310258:0:37:0:(import.c:807:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5) req@0000000092f98018 x1862456998101120/t0(0) o38->lustre-MDT0000-lwp-OST0001@0@lo:12/10 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' 00000100:00080000:1.0:1776178579.310273:0:37:0:(pinger.c:251:ptlrpc_pinger_process_import()) lustre-MDT0001-mdtlov_UUID->lustre-MDT0000_UUID: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00080000:1.0:1776178579.310276:0:37:0:(recover.c:57:ptlrpc_initiate_recovery()) lustre-MDT0000_UUID: starting recovery 00000100:00080000:1.0:1776178579.310277:0:37:0:(import.c:86:import_set_state_nolock()) 00000000ba4f1821 lustre-MDT0000_UUID: changing import state from DISCONN to CONNECTING 00000100:00080000:1.0:1776178579.310280:0:37:0:(import.c:534:import_select_connection()) lustre-MDT0000-osp-MDT0001: connect to NID 0@lo last attempt 1281 00000100:00080000:1.0:1776178579.310283:0:37:0:(import.c:581:import_select_connection()) lustre-MDT0000-osp-MDT0001: tried all connections, increasing latency to 21s 00000100:00080000:1.0:1776178579.310286:0:37:0:(import.c:615:import_select_connection()) lustre-MDT0000-osp-MDT0001: import 00000000ba4f1821 using connection 192.168.204.103@tcp/0@lo 00000100:00100000:1.0:1776178579.310296:0:37:0:(import.c:807:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5) req@00000000f97fd38e x1862456998101248/t0(0) o38->lustre-MDT0000-osp-MDT0001@0@lo:24/4 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' 00000100:00100000:2.0:1776178579.310437:0:3352:0:(client.c:1733:ptlrpc_send_new_req()) Sending RPC req@0000000022d4dc29 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:lustre-MDT0000-lwp-OST0000_UUID:3352:1862456998100992:0@lo:38:kworker/u8:2.0 00000100:00100000:2.0:1776178579.310537:0:3352:0:(events.c:361:request_in_callback()) peer: 12345-0@lo (source: 12345-0@lo) 00000100:00100000:2.0:1776178579.310590:0:3352:0:(client.c:1733:ptlrpc_send_new_req()) Sending RPC req@0000000092f98018 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:lustre-MDT0000-lwp-OST0001_UUID:3352:1862456998101120:0@lo:38:kworker/u8:2.0 00000100:00100000:2.0:1776178579.310615:0:3352:0:(events.c:361:request_in_callback()) peer: 12345-0@lo (source: 12345-0@lo) 00000100:00100000:2.0:1776178579.310678:0:8250:0:(service.c:2152:ptlrpc_server_handle_req_in()) got req x1862456998100992 00000100:00100000:2.0:1776178579.310725:0:8250:0:(service.c:2152:ptlrpc_server_handle_req_in()) got req x1862456998101120 00000100:00100000:2.0:1776178579.310845:0:8250:0:(nrs_fifo.c:179:nrs_fifo_req_get()) NRS start fifo request from 12345-0@lo, seq: 6917 00000100:00100000:2.0:1776178579.310853:0:8250:0:(service.c:2324:ptlrpc_server_handle_request()) Handling RPC req@000000006c57be21 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_003:0+-99:3352:x1862456998100992:12345-0@lo:38:kworker/u8:2.0 00000100:00100000:0.0:1776178579.311137:0:6257:0:(nrs_fifo.c:179:nrs_fifo_req_get()) NRS start fifo request from 12345-0@lo, seq: 6918 00000100:00100000:0.0:1776178579.311144:0:6257:0:(service.c:2324:ptlrpc_server_handle_request()) Handling RPC req@000000000489d393 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_002:0+-99:3352:x1862456998101120:12345-0@lo:38:kworker/u8:2.0 00010000:02000000:0.0:1776178579.311164:0:6257:0:(ldlm_lib.c:1139:target_handle_connect()) lustre-MDT0000: Not available for connect from 0@lo (stopping) 00000100:00100000:0.0:1776178579.311224:0:6257:0:(service.c:2372:ptlrpc_server_handle_request()) Handled RPC req@000000000489d393 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_002:0+-99:3352:x1862456998101120:12345-0@lo:38:kworker/u8:2.0 Request processed in 79us (608us total) trans 0 rc -19/-19 00000100:00100000:0.0:1776178579.311232:0:6257:0:(nrs_fifo.c:241:nrs_fifo_req_stop()) NRS stop fifo request from 12345-0@lo, seq: 6918 00000100:00100000:0.0:1776178579.311262:0:3354:0:(client.c:1733:ptlrpc_send_new_req()) Sending RPC req@000000001ee743a7 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_01:lustre-MDT0001-lwp-OST0001_UUID:3354:1862456998101184:0@lo:400:kworker/u8:2.0 00000100:00100000:0.0:1776178579.311291:0:3354:0:(events.c:361:request_in_callback()) peer: 12345-0@lo (source: 12345-0@lo) 00000100:00100000:3.0:1776178579.312374:0:3353:0:(client.c:1733:ptlrpc_send_new_req()) Sending RPC req@0000000063c20b43 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_00:lustre-MDT0001-lwp-OST0000_UUID:3353:1862456998101056:0@lo:400:kworker/u8:2.0 00000100:00080000:3.0:1776178579.312536:0:3352:0:(import.c:86:import_set_state_nolock()) 00000000dfc8db13 lustre-MDT0000_UUID: changing import state from CONNECTING to DISCONN 00000100:00080000:3.0:1776178579.312540:0:3352:0:(import.c:1429:ptlrpc_connect_interpret()) recovery of lustre-MDT0000_UUID on 192.168.204.103@tcp failed (-19) 00000100:00100000:3.0:1776178579.312545:0:3352:0:(client.c:2215:ptlrpc_check_set()) Completed RPC req@0000000092f98018 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:lustre-MDT0000-lwp-OST0001_UUID:3352:1862456998101120:0@lo:38:kworker/u8:2.0 00000100:00080000:1.0:1776178579.313372:0:37:0:(pinger.c:251:ptlrpc_pinger_process_import()) lustre-MDT0000-lwp-MDT0001_UUID->lustre-MDT0000_UUID: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00080000:1.0:1776178579.313378:0:37:0:(recover.c:57:ptlrpc_initiate_recovery()) lustre-MDT0000_UUID: starting recovery 00000100:00080000:1.0:1776178579.313382:0:37:0:(import.c:86:import_set_state_nolock()) 00000000a7c55b1f lustre-MDT0000_UUID: changing import state from DISCONN to CONNECTING 00000100:00080000:1.0:1776178579.313388:0:37:0:(import.c:534:import_select_connection()) lustre-MDT0000-lwp-MDT0001: connect to NID 0@lo last attempt 1281 00000100:00080000:1.0:1776178579.313391:0:37:0:(import.c:581:import_select_connection()) lustre-MDT0000-lwp-MDT0001: tried all connections, increasing latency to 21s 00000100:00080000:1.0:1776178579.313398:0:37:0:(import.c:615:import_select_connection()) lustre-MDT0000-lwp-MDT0001: import 00000000a7c55b1f using connection 192.168.204.103@tcp/0@lo 00000100:00100000:1.0:1776178579.313422:0:37:0:(import.c:807:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5) req@000000009e484c71 x1862456998101440/t0(0) o38->lustre-MDT0000-lwp-MDT0001@0@lo:12/10 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' 00010000:02000000:2.0:1776178579.314778:0:8250:0:(ldlm_lib.c:1139:target_handle_connect()) lustre-MDT0000: Not available for connect from 0@lo (stopping) 00000100:00100000:2.0:1776178579.316108:0:8250:0:(service.c:2372:ptlrpc_server_handle_request()) Handled RPC req@000000006c57be21 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_003:0+-99:3352:x1862456998100992:12345-0@lo:38:kworker/u8:2.0 Request processed in 5242us (5560us total) trans 0 rc -19/-19 00000100:00100000:2.0:1776178579.316133:0:8250:0:(nrs_fifo.c:241:nrs_fifo_req_stop()) NRS stop fifo request from 12345-0@lo, seq: 6917 00000100:00100000:3.0:1776178579.316297:0:3353:0:(events.c:361:request_in_callback()) peer: 12345-0@lo (source: 12345-0@lo) 00000100:00100000:2.0:1776178579.316383:0:8250:0:(service.c:2152:ptlrpc_server_handle_req_in()) got req x1862456998101184 00000100:00100000:1.0:1776178579.316567:0:11281:0:(nrs_fifo.c:179:nrs_fifo_req_get()) NRS start fifo request from 12345-0@lo, seq: 1148 00000100:00100000:1.0:1776178579.316578:0:11281:0:(service.c:2324:ptlrpc_server_handle_request()) Handling RPC req@00000000fd32fba5 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_004:lustre-MDT0001-lwp-OST0001_UUID+5:3354:x1862456998101184:12345-0@lo:400:kworker/u8:2.0 00000100:00100000:1.0:1776178579.316673:0:11281:0:(service.c:2372:ptlrpc_server_handle_request()) Handled RPC req@00000000fd32fba5 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_004:lustre-MDT0001-lwp-OST0001_UUID+5:3354:x1862456998101184:12345-0@lo:400:kworker/u8:2.0 Request processed in 96us (5381us total) trans 0 rc 0/0 00000100:00100000:1.0:1776178579.316680:0:11281:0:(nrs_fifo.c:241:nrs_fifo_req_stop()) NRS stop fifo request from 12345-0@lo, seq: 1148 00000100:00100000:1.0:1776178579.316723:0:3355:0:(client.c:1733:ptlrpc_send_new_req()) Sending RPC req@00000000ddf29755 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_02:lustre-MDT0001-mdtlov_UUID:3355:1862456998101312:0@lo:400:kworker/u8:2.0 00000100:00100000:1.0:1776178579.316885:0:3355:0:(events.c:361:request_in_callback()) peer: 12345-0@lo (source: 12345-0@lo) 00000100:00100000:1.0:1776178579.316908:0:3355:0:(ptlrpcd.c:415:ptlrpcd_check()) transfer 1 async RPCs [3->2] 00000100:00100000:1.0:1776178579.316916:0:3355:0:(client.c:1733:ptlrpc_send_new_req()) Sending RPC req@0000000021958fc7 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_02:lustre-MDT0001-mdtlov_UUID:3355:1862456998101376:0@lo:400:kworker/u8:2.0 00000100:00100000:1.0:1776178579.316939:0:3355:0:(events.c:361:request_in_callback()) peer: 12345-0@lo (source: 12345-0@lo) 00000100:00100000:1.0:1776178579.316970:0:20759:0:(service.c:2152:ptlrpc_server_handle_req_in()) got req x1862456998101312 00000100:00100000:1.0:1776178579.316999:0:20759:0:(service.c:2152:ptlrpc_server_handle_req_in()) got req x1862456998101376 00000100:00100000:1.0:1776178579.317019:0:20759:0:(nrs_fifo.c:179:nrs_fifo_req_get()) NRS start fifo request from 12345-0@lo, seq: 805 00000100:00100000:1.0:1776178579.317023:0:20759:0:(service.c:2324:ptlrpc_server_handle_request()) Handling RPC req@0000000098682d9e pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_010:lustre-MDT0001-mdtlov_UUID+5:3355:x1862456998101312:12345-0@lo:400:kworker/u8:2.0 00000100:00100000:1.0:1776178579.317057:0:20759:0:(service.c:2372:ptlrpc_server_handle_request()) Handled RPC req@0000000098682d9e pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_010:lustre-MDT0001-mdtlov_UUID+5:3355:x1862456998101312:12345-0@lo:400:kworker/u8:2.0 Request processed in 33us (174us total) trans 0 rc 0/0 00000100:00100000:1.0:1776178579.317087:0:20759:0:(nrs_fifo.c:241:nrs_fifo_req_stop()) NRS stop fifo request from 12345-0@lo, seq: 805 00000100:00100000:1.0:1776178579.317099:0:20759:0:(nrs_fifo.c:179:nrs_fifo_req_get()) NRS start fifo request from 12345-0@lo, seq: 806 00000100:00100000:1.0:1776178579.317102:0:20759:0:(service.c:2324:ptlrpc_server_handle_request()) Handling RPC req@000000007d1fa4e3 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_010:lustre-MDT0001-mdtlov_UUID+5:3355:x1862456998101376:12345-0@lo:400:kworker/u8:2.0 00000100:00100000:1.0:1776178579.317126:0:20759:0:(service.c:2372:ptlrpc_server_handle_request()) Handled RPC req@000000007d1fa4e3 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_010:lustre-MDT0001-mdtlov_UUID+5:3355:x1862456998101376:12345-0@lo:400:kworker/u8:2.0 Request processed in 23us (186us total) trans 0 rc 0/0 00000100:00100000:1.0:1776178579.317132:0:20759:0:(nrs_fifo.c:241:nrs_fifo_req_stop()) NRS stop fifo request from 12345-0@lo, seq: 806 00000100:00100000:1.0:1776178579.317182:0:3355:0:(client.c:2215:ptlrpc_check_set()) Completed RPC req@0000000021958fc7 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_02:lustre-MDT0001-mdtlov_UUID:3355:1862456998101376:0@lo:400:kworker/u8:2.0 00000100:00100000:1.0:1776178579.317192:0:3355:0:(client.c:2215:ptlrpc_check_set()) Completed RPC req@00000000ddf29755 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_02:lustre-MDT0001-mdtlov_UUID:3355:1862456998101312:0@lo:400:kworker/u8:2.0 00000100:00100000:1.0:1776178579.317227:0:3354:0:(client.c:2215:ptlrpc_check_set()) Completed RPC req@000000001ee743a7 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_01:lustre-MDT0001-lwp-OST0001_UUID:3354:1862456998101184:0@lo:400:kworker/u8:2.0 00000100:00100000:2.0:1776178579.317728:0:3352:0:(client.c:1733:ptlrpc_send_new_req()) Sending RPC req@00000000f97fd38e pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:lustre-MDT0001-mdtlov_UUID:3352:1862456998101248:0@lo:38:kworker/u8:2.0 00000100:00100000:2.0:1776178579.317936:0:3352:0:(events.c:361:request_in_callback()) peer: 12345-0@lo (source: 12345-0@lo) 00000100:00100000:2.0:1776178579.318001:0:3352:0:(client.c:1733:ptlrpc_send_new_req()) Sending RPC req@000000009e484c71 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:lustre-MDT0000-lwp-MDT0001_UUID:3352:1862456998101440:0@lo:38:kworker/u8:2.0 00000100:00100000:1.0:1776178579.318100:0:14128:0:(service.c:2152:ptlrpc_server_handle_req_in()) got req x1862456998101248 00000100:00100000:2.0:1776178579.318257:0:14128:0:(nrs_fifo.c:179:nrs_fifo_req_get()) NRS start fifo request from 12345-0@lo, seq: 781 00000100:00100000:2.0:1776178579.318263:0:14128:0:(service.c:2324:ptlrpc_server_handle_request()) Handling RPC req@00000000dc0a388a pname:cluuid+ref:pid:xid:nid:opc:job mdt_out00_002:0+-99:3352:x1862456998101248:12345-0@lo:38:kworker/u8:2.0 00010000:02000000:2.0:1776178579.318289:0:14128:0:(ldlm_lib.c:1139:target_handle_connect()) lustre-MDT0000: Not available for connect from 0@lo (stopping) 00000100:00100000:2.0:1776178579.318500:0:14128:0:(service.c:2372:ptlrpc_server_handle_request()) Handled RPC req@00000000dc0a388a pname:cluuid+ref:pid:xid:nid:opc:job mdt_out00_002:0+-99:3352:x1862456998101248:12345-0@lo:38:kworker/u8:2.0 Request processed in 236us (563us total) trans 0 rc -19/-19 00000100:00100000:2.0:1776178579.318507:0:14128:0:(nrs_fifo.c:241:nrs_fifo_req_stop()) NRS stop fifo request from 12345-0@lo, seq: 781 00000100:00100000:1.0:1776178579.325226:0:3352:0:(events.c:361:request_in_callback()) peer: 12345-0@lo (source: 12345-0@lo) 00000100:00080000:1.0:1776178579.325340:0:3352:0:(import.c:86:import_set_state_nolock()) 0000000047601a98 lustre-MDT0000_UUID: changing import state from CONNECTING to DISCONN 00000100:00080000:1.0:1776178579.325350:0:3352:0:(import.c:1429:ptlrpc_connect_interpret()) recovery of lustre-MDT0000_UUID on 192.168.204.103@tcp failed (-19) 00000100:00100000:1.0:1776178579.325357:0:3352:0:(client.c:2215:ptlrpc_check_set()) Completed RPC req@0000000022d4dc29 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:lustre-MDT0000-lwp-OST0000_UUID:3352:1862456998100992:0@lo:38:kworker/u8:2.0 00000100:00080000:1.0:1776178579.325406:0:3352:0:(import.c:86:import_set_state_nolock()) 00000000ba4f1821 lustre-MDT0000_UUID: changing import state from CONNECTING to DISCONN 00000100:00080000:1.0:1776178579.325410:0:3352:0:(import.c:1429:ptlrpc_connect_interpret()) recovery of lustre-MDT0000_UUID on 192.168.204.103@tcp failed (-19) 00000100:00100000:1.0:1776178579.325413:0:3352:0:(client.c:2215:ptlrpc_check_set()) Completed RPC req@00000000f97fd38e pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:lustre-MDT0001-mdtlov_UUID:3352:1862456998101248:0@lo:38:kworker/u8:2.0 00000100:00100000:2.0:1776178579.325542:0:8250:0:(service.c:2152:ptlrpc_server_handle_req_in()) got req x1862456998101056 00000100:00100000:2.0:1776178579.325630:0:8250:0:(service.c:2152:ptlrpc_server_handle_req_in()) got req x1862456998101440 00000100:00100000:2.0:1776178579.325780:0:8250:0:(nrs_fifo.c:179:nrs_fifo_req_get()) NRS start fifo request from 12345-0@lo, seq: 1149 00000100:00100000:2.0:1776178579.325790:0:8250:0:(service.c:2324:ptlrpc_server_handle_request()) Handling RPC req@0000000042de09e9 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_003:lustre-MDT0001-lwp-OST0000_UUID+5:3353:x1862456998101056:12345-0@lo:400:kworker/u8:2.0 00000100:00100000:2.0:1776178579.325927:0:8250:0:(service.c:2372:ptlrpc_server_handle_request()) Handled RPC req@0000000042de09e9 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_003:lustre-MDT0001-lwp-OST0000_UUID+5:3353:x1862456998101056:12345-0@lo:400:kworker/u8:2.0 Request processed in 137us (9641us total) trans 0 rc 0/0 00000100:00100000:2.0:1776178579.325937:0:8250:0:(nrs_fifo.c:241:nrs_fifo_req_stop()) NRS stop fifo request from 12345-0@lo, seq: 1149 00000100:00100000:2.0:1776178579.325961:0:8250:0:(nrs_fifo.c:179:nrs_fifo_req_get()) NRS start fifo request from 12345-0@lo, seq: 6919 00000100:00100000:2.0:1776178579.325965:0:8250:0:(service.c:2324:ptlrpc_server_handle_request()) Handling RPC req@0000000021958fc7 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_003:0+-99:3352:x1862456998101440:12345-0@lo:38:kworker/u8:2.0 00010000:02000000:2.0:1776178579.325985:0:8250:0:(ldlm_lib.c:1139:target_handle_connect()) lustre-MDT0000: Not available for connect from 0@lo (stopping) 00000100:00100000:2.0:1776178579.326022:0:8250:0:(service.c:2372:ptlrpc_server_handle_request()) Handled RPC req@0000000021958fc7 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_003:0+-99:3352:x1862456998101440:12345-0@lo:38:kworker/u8:2.0 Request processed in 56us (811us total) trans 0 rc -19/-19 00000100:00100000:2.0:1776178579.326030:0:8250:0:(nrs_fifo.c:241:nrs_fifo_req_stop()) NRS stop fifo request from 12345-0@lo, seq: 6919 00000100:00080000:0.0:1776178579.326449:0:3352:0:(import.c:86:import_set_state_nolock()) 00000000a7c55b1f lustre-MDT0000_UUID: changing import state from CONNECTING to DISCONN 00000100:00080000:0.0:1776178579.326455:0:3352:0:(import.c:1429:ptlrpc_connect_interpret()) recovery of lustre-MDT0000_UUID on 192.168.204.103@tcp failed (-19) 00000100:00100000:0.0:1776178579.326460:0:3352:0:(client.c:2215:ptlrpc_check_set()) Completed RPC req@000000009e484c71 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:lustre-MDT0000-lwp-MDT0001_UUID:3352:1862456998101440:0@lo:38:kworker/u8:2.0 00000100:00100000:2.0:1776178579.334857:0:3353:0:(client.c:2215:ptlrpc_check_set()) Completed RPC req@0000000063c20b43 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_00:lustre-MDT0001-lwp-OST0000_UUID:3353:1862456998101056:0@lo:400:kworker/u8:2.0 00010000:00010000:1.0:1776178579.937820:0:9224:0:(ldlm_lockd.c:2792:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:1.0:1776178580.832472:0:3354:0:(ptlrpcd.c:415:ptlrpcd_check()) transfer 1 async RPCs [0->1] 00000100:00100000:1.0:1776178580.832501:0:3354:0:(client.c:1733:ptlrpc_send_new_req()) Sending RPC req@0000000029e5c385 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_01:lustre-MDT0001-mdtlov_UUID:3354:1862456998101504:0@lo:13:osp-pre-1-1.0 00000100:00100000:1.0:1776178580.832581:0:3354:0:(events.c:361:request_in_callback()) peer: 12345-0@lo (source: 12345-0@lo) 00000100:00100000:3.0:1776178580.832673:0:8231:0:(service.c:2152:ptlrpc_server_handle_req_in()) got req x1862456998101504 00000100:00100000:0.0:1776178580.832838:0:9554:0:(nrs_fifo.c:179:nrs_fifo_req_get()) NRS start fifo request from 12345-0@lo, seq: 635 00000100:00100000:0.0:1776178580.832845:0:9554:0:(service.c:2324:ptlrpc_server_handle_request()) Handling RPC req@000000009ff44a0a pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_create00:lustre-MDT0001-mdtlov_UUID+5:3354:x1862456998101504:12345-0@lo:13:osp-pre-1-1.0 00002000:00000024:0.0:1776178580.832876:0:9554:0:(ofd_obd.c:740:ofd_statfs()) blocks cached 229376 granted 491746368 pending 0 free 3923689472 avail 3692167168 00000100:00100000:0.0:1776178580.832928:0:9554:0:(service.c:2372:ptlrpc_server_handle_request()) Handled RPC req@000000009ff44a0a pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_create00:lustre-MDT0001-mdtlov_UUID+5:3354:x1862456998101504:12345-0@lo:13:osp-pre-1-1.0 Request processed in 82us (349us total) trans 0 rc 0/0 00000100:00100000:0.0:1776178580.832936:0:9554:0:(nrs_fifo.c:241:nrs_fifo_req_stop()) NRS stop fifo request from 12345-0@lo, seq: 635 00000100:00100000:1.0:1776178580.833397:0:3354:0:(client.c:2215:ptlrpc_check_set()) Completed RPC req@0000000029e5c385 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_01:lustre-MDT0001-mdtlov_UUID:3354:1862456998101504:0@lo:13:osp-pre-1-1.0 00010000:00010000:1.0:1776178580.961457:0:9225:0:(ldlm_lockd.c:2792:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:2.0:1776178581.344351:0:3353:0:(ptlrpcd.c:415:ptlrpcd_check()) transfer 1 async RPCs [1->0] 00000100:00100000:2.0:1776178581.344374:0:3353:0:(client.c:1733:ptlrpc_send_new_req()) Sending RPC req@000000008c3fbdb6 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_00:lustre-MDT0001-mdtlov_UUID:3353:1862456998101568:0@lo:13:osp-pre-0-1.0 00000100:00100000:2.0:1776178581.344446:0:3353:0:(events.c:361:request_in_callback()) peer: 12345-0@lo (source: 12345-0@lo) 00000100:00100000:3.0:1776178581.344728:0:8231:0:(service.c:2152:ptlrpc_server_handle_req_in()) got req x1862456998101568 00000100:00100000:3.0:1776178581.344793:0:8231:0:(nrs_fifo.c:179:nrs_fifo_req_get()) NRS start fifo request from 12345-0@lo, seq: 636 00000100:00100000:3.0:1776178581.344798:0:8231:0:(service.c:2324:ptlrpc_server_handle_request()) Handling RPC req@0000000054386dd3 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_create00:lustre-MDT0001-mdtlov_UUID+5:3353:x1862456998101568:12345-0@lo:13:osp-pre-0-1.0 00002000:00000024:3.0:1776178581.344834:0:8231:0:(ofd_obd.c:740:ofd_statfs()) blocks cached 315392 granted 484883776 pending 0 free 3923689472 avail 3692167168 00000100:00100000:3.0:1776178581.344875:0:8231:0:(service.c:2372:ptlrpc_server_handle_request()) Handled RPC req@0000000054386dd3 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_create00:lustre-MDT0001-mdtlov_UUID+5:3353:x1862456998101568:12345-0@lo:13:osp-pre-0-1.0 Request processed in 77us (431us total) trans 0 rc 0/0 00000100:00100000:3.0:1776178581.344883:0:8231:0:(nrs_fifo.c:241:nrs_fifo_req_stop()) NRS stop fifo request from 12345-0@lo, seq: 636 00000100:00100000:2.0:1776178581.344979:0:3353:0:(client.c:2215:ptlrpc_check_set()) Completed RPC req@000000008c3fbdb6 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_00:lustre-MDT0001-mdtlov_UUID:3353:1862456998101568:0@lo:13:osp-pre-0-1.0 00010000:00010000:2.0:1776178581.987903:0:6244:0:(ldlm_lockd.c:2792:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:1.0:1776178582.820876:0:3346:0:(events.c:361:request_in_callback()) peer: 12345-192.168.204.3@tcp (source: 12345-192.168.204.3@tcp) 00000100:00100000:1.0:1776178582.820929:0:3346:0:(events.c:361:request_in_callback()) peer: 12345-192.168.204.3@tcp (source: 12345-192.168.204.3@tcp) 00000100:00100000:0.0:1776178582.821039:0:8250:0:(service.c:2152:ptlrpc_server_handle_req_in()) got req x1862456979132736 00000100:00100000:0.0:1776178582.821136:0:8250:0:(nrs_fifo.c:179:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.204.3@tcp, seq: 6920 00000100:00100000:0.0:1776178582.821143:0:8250:0:(service.c:2324:ptlrpc_server_handle_request()) Handling RPC req@000000002db08e42 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_003:0+-99:2269:x1862456979132736:12345-192.168.204.3@tcp:38:kworker/u8:3.0 00010000:02000000:0.0:1776178582.821161:0:8250:0:(ldlm_lib.c:1139:target_handle_connect()) lustre-MDT0000: Not available for connect from 192.168.204.3@tcp (stopping) 00000100:00100000:0.0:1776178582.821223:0:8250:0:(service.c:2372:ptlrpc_server_handle_request()) Handled RPC req@000000002db08e42 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_003:0+-99:2269:x1862456979132736:12345-192.168.204.3@tcp:38:kworker/u8:3.0 Request processed in 78us (293us total) trans 0 rc -19/-19 00000100:00100000:0.0:1776178582.821240:0:8250:0:(nrs_fifo.c:241:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.204.3@tcp, seq: 6920 00000100:00100000:3.0:1776178582.823845:0:20140:0:(service.c:2152:ptlrpc_server_handle_req_in()) got req x1862456979132672 00000100:00100000:3.0:1776178582.823931:0:20140:0:(nrs_fifo.c:179:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.204.3@tcp, seq: 303 00000100:00100000:3.0:1776178582.823943:0:20140:0:(service.c:2324:ptlrpc_server_handle_request()) Handling RPC req@00000000c201d06e pname:cluuid+ref:pid:xid:nid:opc:job ll_mgs_0002:5b5c66b1-1a2f-4651-8548-0c75b9a38a94+9:2271:x1862456979132672:12345-192.168.204.3@tcp:400:kworker/u8:3.0 00000100:00100000:3.0:1776178582.824332:0:20140:0:(service.c:2372:ptlrpc_server_handle_request()) Handled RPC req@00000000c201d06e pname:cluuid+ref:pid:xid:nid:opc:job ll_mgs_0002:5b5c66b1-1a2f-4651-8548-0c75b9a38a94+9:2271:x1862456979132672:12345-192.168.204.3@tcp:400:kworker/u8:3.0 Request processed in 390us (3465us total) trans 0 rc 0/0 00000100:00100000:3.0:1776178582.824346:0:20140:0:(nrs_fifo.c:241:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.204.3@tcp, seq: 303 00000100:00100000:1.0:1776178582.833286:0:3346:0:(events.c:361:request_in_callback()) peer: 12345-192.168.204.3@tcp (source: 12345-192.168.204.3@tcp) 00000100:00100000:1.0:1776178582.833340:0:3346:0:(events.c:361:request_in_callback()) peer: 12345-192.168.204.3@tcp (source: 12345-192.168.204.3@tcp) 00000100:00100000:1.0:1776178582.833377:0:3346:0:(events.c:361:request_in_callback()) peer: 12345-192.168.204.3@tcp (source: 12345-192.168.204.3@tcp) 00000100:00100000:0.0:1776178582.833475:0:20755:0:(service.c:2152:ptlrpc_server_handle_req_in()) got req x1862456979132864 00000100:00100000:0.0:1776178582.833541:0:20755:0:(service.c:2152:ptlrpc_server_handle_req_in()) got req x1862456979132928 00000100:00100000:0.0:1776178582.833564:0:20755:0:(nrs_fifo.c:179:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.204.3@tcp, seq: 807 00000100:00100000:0.0:1776178582.833570:0:20755:0:(service.c:2324:ptlrpc_server_handle_request()) Handling RPC req@0000000022d4dc29 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_006:b50963bd-8eba-4432-9a54-8888d00e3fa1+8:2271:x1862456979132864:12345-192.168.204.3@tcp:400:kworker/u8:3.0 00000100:00100000:0.0:1776178582.833643:0:20755:0:(service.c:2372:ptlrpc_server_handle_request()) Handled RPC req@0000000022d4dc29 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_006:b50963bd-8eba-4432-9a54-8888d00e3fa1+8:2271:x1862456979132864:12345-192.168.204.3@tcp:400:kworker/u8:3.0 Request processed in 73us (303us total) trans 0 rc 0/0 00000100:00100000:0.0:1776178582.833651:0:20755:0:(nrs_fifo.c:241:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.204.3@tcp, seq: 807 00000100:00100000:0.0:1776178582.833668:0:20755:0:(nrs_fifo.c:179:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.204.3@tcp, seq: 808 00000100:00100000:0.0:1776178582.833682:0:20755:0:(service.c:2324:ptlrpc_server_handle_request()) Handling RPC req@00000000f97fd38e pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_006:b50963bd-8eba-4432-9a54-8888d00e3fa1+9:2270:x1862456979132928:12345-192.168.204.3@tcp:400:kworker/u8:3.0 00000100:00100000:0.0:1776178582.833717:0:20755:0:(service.c:2372:ptlrpc_server_handle_request()) Handled RPC req@00000000f97fd38e pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_006:b50963bd-8eba-4432-9a54-8888d00e3fa1+9:2270:x1862456979132928:12345-192.168.204.3@tcp:400:kworker/u8:3.0 Request processed in 24us (329us total) trans 0 rc 0/0 00000100:00100000:0.0:1776178582.833722:0:20755:0:(nrs_fifo.c:241:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.204.3@tcp, seq: 808 00000100:00100000:2.0:1776178582.836226:0:6256:0:(service.c:2152:ptlrpc_server_handle_req_in()) got req x1862456979132800 00000100:00100000:2.0:1776178582.836310:0:6256:0:(nrs_fifo.c:179:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.204.3@tcp, seq: 1150 00000100:00100000:2.0:1776178582.836316:0:6256:0:(service.c:2324:ptlrpc_server_handle_request()) Handling RPC req@000000009e484c71 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_001:b50963bd-8eba-4432-9a54-8888d00e3fa1+6:2270:x1862456979132800:12345-192.168.204.3@tcp:400:kworker/u8:3.0 00000100:00100000:2.0:1776178582.836403:0:6256:0:(service.c:2372:ptlrpc_server_handle_request()) Handled RPC req@000000009e484c71 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_001:b50963bd-8eba-4432-9a54-8888d00e3fa1+6:2270:x1862456979132800:12345-192.168.204.3@tcp:400:kworker/u8:3.0 Request processed in 85us (3125us total) trans 0 rc 0/0 00000100:00100000:2.0:1776178582.836411:0:6256:0:(nrs_fifo.c:241:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.204.3@tcp, seq: 1150 00010000:00010000:1.0:1776178583.008305:0:6245:0:(ldlm_lockd.c:2792:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00010000:00010000:1.0:1776178584.037377:0:9224:0:(ldlm_lockd.c:2792:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:1.0:1776178584.416372:0:3355:0:(client.c:1733:ptlrpc_send_new_req()) Sending RPC req@000000008c3fbdb6 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_02:ac61f839-0e8e-4cac-9c59-08bcfbbad330:3355:1862456998101632:0@lo:400:kworker/u8:2.0 00000100:00100000:1.0:1776178584.416468:0:3355:0:(events.c:361:request_in_callback()) peer: 12345-0@lo (source: 12345-0@lo) 00000100:00100000:2.0:1776178584.416557:0:20137:0:(service.c:2152:ptlrpc_server_handle_req_in()) got req x1862456998101632 00000100:00100000:2.0:1776178584.416682:0:20137:0:(nrs_fifo.c:179:nrs_fifo_req_get()) NRS start fifo request from 12345-0@lo, seq: 304 00000100:00100000:2.0:1776178584.416689:0:20137:0:(service.c:2324:ptlrpc_server_handle_request()) Handling RPC req@000000009c2c404e pname:cluuid+ref:pid:xid:nid:opc:job ll_mgs_0000:ac61f839-0e8e-4cac-9c59-08bcfbbad330+18:3355:x1862456998101632:12345-0@lo:400:kworker/u8:2.0 00000100:00080000:0.0:1776178584.416710:0:37:0:(pinger.c:251:ptlrpc_pinger_process_import()) lustre-MDT0000-lwp-OST0000_UUID->lustre-MDT0000_UUID: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00080000:0.0:1776178584.416715:0:37:0:(recover.c:57:ptlrpc_initiate_recovery()) lustre-MDT0000_UUID: starting recovery 00000100:00080000:0.0:1776178584.416718:0:37:0:(import.c:86:import_set_state_nolock()) 0000000047601a98 lustre-MDT0000_UUID: changing import state from DISCONN to CONNECTING 00000100:00080000:0.0:1776178584.416723:0:37:0:(import.c:534:import_select_connection()) lustre-MDT0000-lwp-OST0000: connect to NID 0@lo last attempt 1286 00000100:00080000:0.0:1776178584.416725:0:37:0:(import.c:581:import_select_connection()) lustre-MDT0000-lwp-OST0000: tried all connections, increasing latency to 21s 00000100:00080000:0.0:1776178584.416729:0:37:0:(import.c:615:import_select_connection()) lustre-MDT0000-lwp-OST0000: import 0000000047601a98 using connection 192.168.204.103@tcp/0@lo 00000100:00100000:0.0:1776178584.416750:0:37:0:(import.c:807:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5) req@00000000239d7b9a x1862456998101696/t0(0) o38->lustre-MDT0000-lwp-OST0000@0@lo:12/10 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' 00000100:00100000:3.0:1776178584.416784:0:3352:0:(client.c:1733:ptlrpc_send_new_req()) Sending RPC req@00000000239d7b9a pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:lustre-MDT0000-lwp-OST0000_UUID:3352:1862456998101696:0@lo:38:kworker/u8:2.0 00000100:00100000:3.0:1776178584.416820:0:3352:0:(events.c:361:request_in_callback()) peer: 12345-0@lo (source: 12345-0@lo) 00000100:00100000:3.0:1776178584.416865:0:8250:0:(service.c:2152:ptlrpc_server_handle_req_in()) got req x1862456998101696 00000100:00100000:3.0:1776178584.416896:0:8250:0:(nrs_fifo.c:179:nrs_fifo_req_get()) NRS start fifo request from 12345-0@lo, seq: 6921 00000100:00100000:3.0:1776178584.416903:0:8250:0:(service.c:2324:ptlrpc_server_handle_request()) Handling RPC req@0000000037aace32 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_003:0+-99:3352:x1862456998101696:12345-0@lo:38:kworker/u8:2.0 00010000:02000000:3.0:1776178584.416918:0:8250:0:(ldlm_lib.c:1139:target_handle_connect()) lustre-MDT0000: Not available for connect from 0@lo (stopping) 00000100:00100000:3.0:1776178584.416953:0:8250:0:(service.c:2372:ptlrpc_server_handle_request()) Handled RPC req@0000000037aace32 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_003:0+-99:3352:x1862456998101696:12345-0@lo:38:kworker/u8:2.0 Request processed in 46us (129us total) trans 0 rc -19/-19 00000100:00100000:3.0:1776178584.416959:0:8250:0:(nrs_fifo.c:241:nrs_fifo_req_stop()) NRS stop fifo request from 12345-0@lo, seq: 6921 00000100:00080000:3.0:1776178584.416993:0:3352:0:(import.c:86:import_set_state_nolock()) 0000000047601a98 lustre-MDT0000_UUID: changing import state from CONNECTING to DISCONN 00000100:00080000:3.0:1776178584.416998:0:3352:0:(import.c:1429:ptlrpc_connect_interpret()) recovery of lustre-MDT0000_UUID on 192.168.204.103@tcp failed (-19) 00000100:00100000:3.0:1776178584.417002:0:3352:0:(client.c:2215:ptlrpc_check_set()) Completed RPC req@00000000239d7b9a pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:lustre-MDT0000-lwp-OST0000_UUID:3352:1862456998101696:0@lo:38:kworker/u8:2.0 00000100:00100000:3.0:1776178584.417034:0:3356:0:(client.c:1733:ptlrpc_send_new_req()) Sending RPC req@00000000dd52d0d0 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_03:lustre-MDT0001-lwp-OST0000_UUID:3356:1862456998101760:0@lo:400:kworker/u8:2.0 00000100:00100000:3.0:1776178584.417085:0:3356:0:(events.c:361:request_in_callback()) peer: 12345-0@lo (source: 12345-0@lo) 00000100:00100000:3.0:1776178584.417107:0:8250:0:(service.c:2152:ptlrpc_server_handle_req_in()) got req x1862456998101760 00000100:00100000:3.0:1776178584.417136:0:8250:0:(nrs_fifo.c:179:nrs_fifo_req_get()) NRS start fifo request from 12345-0@lo, seq: 1151 00000100:00100000:3.0:1776178584.417139:0:8250:0:(service.c:2324:ptlrpc_server_handle_request()) Handling RPC req@00000000e8195050 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_003:lustre-MDT0001-lwp-OST0000_UUID+5:3356:x1862456998101760:12345-0@lo:400:kworker/u8:2.0 00000100:00100000:3.0:1776178584.417176:0:8250:0:(service.c:2372:ptlrpc_server_handle_request()) Handled RPC req@00000000e8195050 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_003:lustre-MDT0001-lwp-OST0000_UUID+5:3356:x1862456998101760:12345-0@lo:400:kworker/u8:2.0 Request processed in 36us (117us total) trans 0 rc 0/0 00000100:00100000:3.0:1776178584.417184:0:8250:0:(nrs_fifo.c:241:nrs_fifo_req_stop()) NRS stop fifo request from 12345-0@lo, seq: 1151 00000100:00100000:3.0:1776178584.417209:0:3356:0:(client.c:2215:ptlrpc_check_set()) Completed RPC req@00000000dd52d0d0 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_03:lustre-MDT0001-lwp-OST0000_UUID:3356:1862456998101760:0@lo:400:kworker/u8:2.0 00000100:00080000:0.0:1776178584.425566:0:37:0:(pinger.c:251:ptlrpc_pinger_process_import()) lustre-MDT0000-lwp-OST0001_UUID->lustre-MDT0000_UUID: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00080000:0.0:1776178584.426527:0:37:0:(recover.c:57:ptlrpc_initiate_recovery()) lustre-MDT0000_UUID: starting recovery 00000100:00080000:0.0:1776178584.426531:0:37:0:(import.c:86:import_set_state_nolock()) 00000000dfc8db13 lustre-MDT0000_UUID: changing import state from DISCONN to CONNECTING 00000100:00080000:0.0:1776178584.426579:0:37:0:(import.c:534:import_select_connection()) lustre-MDT0000-lwp-OST0001: connect to NID 0@lo last attempt 1286 00000100:00080000:0.0:1776178584.426582:0:37:0:(import.c:581:import_select_connection()) lustre-MDT0000-lwp-OST0001: tried all connections, increasing latency to 21s 00000100:00080000:0.0:1776178584.426590:0:37:0:(import.c:615:import_select_connection()) lustre-MDT0000-lwp-OST0001: import 00000000dfc8db13 using connection 192.168.204.103@tcp/0@lo 00000100:00100000:0.0:1776178584.426710:0:37:0:(import.c:807:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5) req@00000000f62fe06f x1862456998101824/t0(0) o38->lustre-MDT0000-lwp-OST0001@0@lo:12/10 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' 00000100:00080000:0.0:1776178584.426782:0:37:0:(pinger.c:251:ptlrpc_pinger_process_import()) lustre-MDT0001-mdtlov_UUID->lustre-MDT0000_UUID: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00080000:0.0:1776178584.426786:0:37:0:(recover.c:57:ptlrpc_initiate_recovery()) lustre-MDT0000_UUID: starting recovery 00000100:00080000:0.0:1776178584.426787:0:37:0:(import.c:86:import_set_state_nolock()) 00000000ba4f1821 lustre-MDT0000_UUID: changing import state from DISCONN to CONNECTING 00000100:00080000:0.0:1776178584.426790:0:37:0:(import.c:534:import_select_connection()) lustre-MDT0000-osp-MDT0001: connect to NID 0@lo last attempt 1286 00000100:00080000:0.0:1776178584.426792:0:37:0:(import.c:581:import_select_connection()) lustre-MDT0000-osp-MDT0001: tried all connections, increasing latency to 21s 00000100:00080000:0.0:1776178584.426796:0:37:0:(import.c:615:import_select_connection()) lustre-MDT0000-osp-MDT0001: import 00000000ba4f1821 using connection 192.168.204.103@tcp/0@lo 00000100:00100000:0.0:1776178584.426803:0:3354:0:(ptlrpcd.c:415:ptlrpcd_check()) transfer 1 async RPCs [0->1] 00000100:00100000:0.0:1776178584.426822:0:3354:0:(client.c:1733:ptlrpc_send_new_req()) Sending RPC req@00000000dd52d0d0 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_01:lustre-MDT0001-lwp-OST0001_UUID:3354:1862456998101888:0@lo:400:kworker/u8:2.0 00000100:00100000:2.0:1776178584.427992:0:20137:0:(service.c:2372:ptlrpc_server_handle_request()) Handled RPC req@000000009c2c404e pname:cluuid+ref:pid:xid:nid:opc:job ll_mgs_0000:ac61f839-0e8e-4cac-9c59-08bcfbbad330+18:3355:x1862456998101632:12345-0@lo:400:kworker/u8:2.0 Request processed in 11296us (11518us total) trans 0 rc 0/0 00000100:00100000:2.0:1776178584.428011:0:20137:0:(nrs_fifo.c:241:nrs_fifo_req_stop()) NRS stop fifo request from 12345-0@lo, seq: 304 00000100:00100000:2.0:1776178584.428130:0:37:0:(import.c:807:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5) req@000000002917d432 x1862456998101952/t0(0) o38->lustre-MDT0000-osp-MDT0001@0@lo:24/4 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' 00000100:00080000:2.0:1776178584.428167:0:37:0:(pinger.c:251:ptlrpc_pinger_process_import()) lustre-MDT0000-lwp-MDT0001_UUID->lustre-MDT0000_UUID: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00080000:2.0:1776178584.428172:0:37:0:(recover.c:57:ptlrpc_initiate_recovery()) lustre-MDT0000_UUID: starting recovery 00000100:00080000:2.0:1776178584.428175:0:37:0:(import.c:86:import_set_state_nolock()) 00000000a7c55b1f lustre-MDT0000_UUID: changing import state from DISCONN to CONNECTING 00000100:00080000:2.0:1776178584.428179:0:37:0:(import.c:534:import_select_connection()) lustre-MDT0000-lwp-MDT0001: connect to NID 0@lo last attempt 1286 00000100:00080000:2.0:1776178584.428182:0:37:0:(import.c:581:import_select_connection()) lustre-MDT0000-lwp-MDT0001: tried all connections, increasing latency to 21s 00000100:00080000:2.0:1776178584.428187:0:37:0:(import.c:615:import_select_connection()) lustre-MDT0000-lwp-MDT0001: import 00000000a7c55b1f using connection 192.168.204.103@tcp/0@lo 00000100:00100000:2.0:1776178584.428197:0:37:0:(import.c:807:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5) req@0000000054bbc2c8 x1862456998102144/t0(0) o38->lustre-MDT0000-lwp-MDT0001@0@lo:12/10 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' 00000100:00100000:0.0:1776178584.428478:0:3354:0:(events.c:361:request_in_callback()) peer: 12345-0@lo (source: 12345-0@lo) 00000100:00100000:0.0:1776178584.428529:0:3354:0:(client.c:1733:ptlrpc_send_new_req()) Sending RPC req@00000000ccad874e pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_01:lustre-MDT0001-mdtlov_UUID:3354:1862456998102016:0@lo:400:kworker/u8:2.0 00000100:00100000:0.0:1776178584.428561:0:3354:0:(events.c:361:request_in_callback()) peer: 12345-0@lo (source: 12345-0@lo) 00000100:00100000:0.0:1776178584.428610:0:20759:0:(service.c:2152:ptlrpc_server_handle_req_in()) got req x1862456998102016 00000100:00100000:2.0:1776178584.428748:0:8250:0:(service.c:2152:ptlrpc_server_handle_req_in()) got req x1862456998101888 00000100:00100000:2.0:1776178584.428777:0:8250:0:(nrs_fifo.c:179:nrs_fifo_req_get()) NRS start fifo request from 12345-0@lo, seq: 1152 00000100:00100000:2.0:1776178584.428783:0:8250:0:(service.c:2324:ptlrpc_server_handle_request()) Handling RPC req@00000000239d7b9a pname:cluuid+ref:pid:xid:nid:opc:job mdt00_003:lustre-MDT0001-lwp-OST0001_UUID+5:3354:x1862456998101888:12345-0@lo:400:kworker/u8:2.0 00000100:00100000:0.0:1776178584.428855:0:20759:0:(nrs_fifo.c:179:nrs_fifo_req_get()) NRS start fifo request from 12345-0@lo, seq: 809 00000100:00100000:0.0:1776178584.428860:0:20759:0:(service.c:2324:ptlrpc_server_handle_request()) Handling RPC req@00000000d4b41d38 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_010:lustre-MDT0001-mdtlov_UUID+5:3354:x1862456998102016:12345-0@lo:400:kworker/u8:2.0 00000100:00100000:2.0:1776178584.429086:0:8250:0:(service.c:2372:ptlrpc_server_handle_request()) Handled RPC req@00000000239d7b9a pname:cluuid+ref:pid:xid:nid:opc:job mdt00_003:lustre-MDT0001-lwp-OST0001_UUID+5:3354:x1862456998101888:12345-0@lo:400:kworker/u8:2.0 Request processed in 301us (608us total) trans 0 rc 0/0 00000100:00100000:2.0:1776178584.429093:0:8250:0:(nrs_fifo.c:241:nrs_fifo_req_stop()) NRS stop fifo request from 12345-0@lo, seq: 1152 00000100:00100000:0.0:1776178584.429220:0:20759:0:(service.c:2372:ptlrpc_server_handle_request()) Handled RPC req@00000000d4b41d38 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_010:lustre-MDT0001-mdtlov_UUID+5:3354:x1862456998102016:12345-0@lo:400:kworker/u8:2.0 Request processed in 359us (658us total) trans 0 rc 0/0 00000100:00100000:0.0:1776178584.429227:0:20759:0:(nrs_fifo.c:241:nrs_fifo_req_stop()) NRS stop fifo request from 12345-0@lo, seq: 809 00000100:00100000:3.0:1776178584.429353:0:3356:0:(ptlrpcd.c:415:ptlrpcd_check()) transfer 1 async RPCs [2->3] 00000100:00100000:3.0:1776178584.429369:0:3356:0:(client.c:1733:ptlrpc_send_new_req()) Sending RPC req@0000000028dd2aee pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_03:lustre-MDT0001-mdtlov_UUID:3356:1862456998102080:0@lo:400:kworker/u8:2.0 00000100:00100000:0.0:1776178584.429378:0:3354:0:(client.c:2215:ptlrpc_check_set()) Completed RPC req@00000000ccad874e pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_01:lustre-MDT0001-mdtlov_UUID:3354:1862456998102016:0@lo:400:kworker/u8:2.0 00000100:00100000:0.0:1776178584.429396:0:3354:0:(client.c:2215:ptlrpc_check_set()) Completed RPC req@00000000dd52d0d0 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_01:lustre-MDT0001-lwp-OST0001_UUID:3354:1862456998101888:0@lo:400:kworker/u8:2.0 00000100:00100000:3.0:1776178584.429410:0:3356:0:(events.c:361:request_in_callback()) peer: 12345-0@lo (source: 12345-0@lo) 00000100:00100000:2.0:1776178584.429488:0:20759:0:(service.c:2152:ptlrpc_server_handle_req_in()) got req x1862456998102080 00000100:00100000:2.0:1776178584.429518:0:20759:0:(nrs_fifo.c:179:nrs_fifo_req_get()) NRS start fifo request from 12345-0@lo, seq: 810 00000100:00100000:2.0:1776178584.429521:0:20759:0:(service.c:2324:ptlrpc_server_handle_request()) Handling RPC req@0000000029e5c385 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_010:lustre-MDT0001-mdtlov_UUID+5:3356:x1862456998102080:12345-0@lo:400:kworker/u8:2.0 00000100:00100000:2.0:1776178584.429553:0:20759:0:(service.c:2372:ptlrpc_server_handle_request()) Handled RPC req@0000000029e5c385 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_010:lustre-MDT0001-mdtlov_UUID+5:3356:x1862456998102080:12345-0@lo:400:kworker/u8:2.0 Request processed in 31us (144us total) trans 0 rc 0/0 00000100:00100000:2.0:1776178584.429558:0:20759:0:(nrs_fifo.c:241:nrs_fifo_req_stop()) NRS stop fifo request from 12345-0@lo, seq: 810 00000100:00100000:0.0:1776178584.429599:0:3355:0:(client.c:2215:ptlrpc_check_set()) Completed RPC req@000000008c3fbdb6 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_02:ac61f839-0e8e-4cac-9c59-08bcfbbad330:3355:1862456998101632:0@lo:400:kworker/u8:2.0 00000100:00100000:3.0:1776178584.430345:0:3356:0:(client.c:2215:ptlrpc_check_set()) Completed RPC req@0000000028dd2aee pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_03:lustre-MDT0001-mdtlov_UUID:3356:1862456998102080:0@lo:400:kworker/u8:2.0 00000100:00100000:1.0:1776178584.432173:0:3352:0:(client.c:1733:ptlrpc_send_new_req()) Sending RPC req@00000000f62fe06f pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:lustre-MDT0000-lwp-OST0001_UUID:3352:1862456998101824:0@lo:38:kworker/u8:2.0 00000100:00100000:1.0:1776178584.432273:0:3352:0:(events.c:361:request_in_callback()) peer: 12345-0@lo (source: 12345-0@lo) 00000100:00100000:1.0:1776178584.432312:0:3352:0:(client.c:1733:ptlrpc_send_new_req()) Sending RPC req@000000002917d432 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:lustre-MDT0001-mdtlov_UUID:3352:1862456998101952:0@lo:38:kworker/u8:2.0 00000100:00100000:1.0:1776178584.432339:0:3352:0:(events.c:361:request_in_callback()) peer: 12345-0@lo (source: 12345-0@lo) 00000100:00100000:1.0:1776178584.432435:0:3352:0:(client.c:1733:ptlrpc_send_new_req()) Sending RPC req@0000000054bbc2c8 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:lustre-MDT0000-lwp-MDT0001_UUID:3352:1862456998102144:0@lo:38:kworker/u8:2.0 00000100:00100000:1.0:1776178584.432473:0:3352:0:(events.c:361:request_in_callback()) peer: 12345-0@lo (source: 12345-0@lo) 00000100:00100000:3.0:1776178584.433154:0:6256:0:(service.c:2152:ptlrpc_server_handle_req_in()) got req x1862456998101824 00000100:00100000:3.0:1776178584.433193:0:6256:0:(service.c:2152:ptlrpc_server_handle_req_in()) got req x1862456998102144 00000100:00100000:3.0:1776178584.433213:0:6256:0:(nrs_fifo.c:179:nrs_fifo_req_get()) NRS start fifo request from 12345-0@lo, seq: 6922 00000100:00100000:3.0:1776178584.433217:0:6256:0:(service.c:2324:ptlrpc_server_handle_request()) Handling RPC req@000000001ee743a7 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_001:0+-99:3352:x1862456998101824:12345-0@lo:38:kworker/u8:2.0 00010000:02000000:3.0:1776178584.433233:0:6256:0:(ldlm_lib.c:1139:target_handle_connect()) lustre-MDT0000: Not available for connect from 0@lo (stopping) 00000100:00100000:3.0:1776178584.433277:0:6256:0:(service.c:2372:ptlrpc_server_handle_request()) Handled RPC req@000000001ee743a7 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_001:0+-99:3352:x1862456998101824:12345-0@lo:38:kworker/u8:2.0 Request processed in 59us (1006us total) trans 0 rc -19/-19 00000100:00100000:3.0:1776178584.433284:0:6256:0:(nrs_fifo.c:241:nrs_fifo_req_stop()) NRS stop fifo request from 12345-0@lo, seq: 6922 00000100:00100000:3.0:1776178584.433298:0:6256:0:(nrs_fifo.c:179:nrs_fifo_req_get()) NRS start fifo request from 12345-0@lo, seq: 6923 00000100:00100000:3.0:1776178584.433301:0:6256:0:(service.c:2324:ptlrpc_server_handle_request()) Handling RPC req@000000008d6accd5 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_001:0+-99:3352:x1862456998102144:12345-0@lo:38:kworker/u8:2.0 00010000:02000000:3.0:1776178584.433308:0:6256:0:(ldlm_lib.c:1139:target_handle_connect()) lustre-MDT0000: Not available for connect from 0@lo (stopping) 00000100:00100000:3.0:1776178584.433328:0:6256:0:(service.c:2372:ptlrpc_server_handle_request()) Handled RPC req@000000008d6accd5 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_001:0+-99:3352:x1862456998102144:12345-0@lo:38:kworker/u8:2.0 Request processed in 26us (855us total) trans 0 rc -19/-19 00000100:00100000:3.0:1776178584.433333:0:6256:0:(nrs_fifo.c:241:nrs_fifo_req_stop()) NRS stop fifo request from 12345-0@lo, seq: 6923 00000100:00080000:1.0:1776178584.433697:0:3352:0:(import.c:86:import_set_state_nolock()) 00000000dfc8db13 lustre-MDT0000_UUID: changing import state from CONNECTING to DISCONN 00000100:00080000:1.0:1776178584.433702:0:3352:0:(import.c:1429:ptlrpc_connect_interpret()) recovery of lustre-MDT0000_UUID on 192.168.204.103@tcp failed (-19) 00000100:00100000:1.0:1776178584.433706:0:3352:0:(client.c:2215:ptlrpc_check_set()) Completed RPC req@00000000f62fe06f pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:lustre-MDT0000-lwp-OST0001_UUID:3352:1862456998101824:0@lo:38:kworker/u8:2.0 00000100:00080000:1.0:1776178584.433714:0:3352:0:(import.c:86:import_set_state_nolock()) 00000000a7c55b1f lustre-MDT0000_UUID: changing import state from CONNECTING to DISCONN 00000100:00080000:1.0:1776178584.433716:0:3352:0:(import.c:1429:ptlrpc_connect_interpret()) recovery of lustre-MDT0000_UUID on 192.168.204.103@tcp failed (-19) 00000100:00100000:1.0:1776178584.433719:0:3352:0:(client.c:2215:ptlrpc_check_set()) Completed RPC req@0000000054bbc2c8 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:lustre-MDT0000-lwp-MDT0001_UUID:3352:1862456998102144:0@lo:38:kworker/u8:2.0 00000100:00100000:2.0:1776178584.435349:0:14128:0:(service.c:2152:ptlrpc_server_handle_req_in()) got req x1862456998101952 00000100:00100000:3.0:1776178584.440736:0:14128:0:(nrs_fifo.c:179:nrs_fifo_req_get()) NRS start fifo request from 12345-0@lo, seq: 782 00000100:00100000:3.0:1776178584.440755:0:14128:0:(service.c:2324:ptlrpc_server_handle_request()) Handling RPC req@00000000ddf29755 pname:cluuid+ref:pid:xid:nid:opc:job mdt_out00_002:0+-99:3352:x1862456998101952:12345-0@lo:38:kworker/u8:2.0 00010000:02000000:3.0:1776178584.440783:0:14128:0:(ldlm_lib.c:1139:target_handle_connect()) lustre-MDT0000: Not available for connect from 0@lo (stopping) 00000100:00100000:3.0:1776178584.440892:0:14128:0:(service.c:2372:ptlrpc_server_handle_request()) Handled RPC req@00000000ddf29755 pname:cluuid+ref:pid:xid:nid:opc:job mdt_out00_002:0+-99:3352:x1862456998101952:12345-0@lo:38:kworker/u8:2.0 Request processed in 146us (8551us total) trans 0 rc -19/-19 00000100:00100000:3.0:1776178584.440900:0:14128:0:(nrs_fifo.c:241:nrs_fifo_req_stop()) NRS stop fifo request from 12345-0@lo, seq: 782 00000100:00080000:2.0:1776178584.441006:0:3352:0:(import.c:86:import_set_state_nolock()) 00000000ba4f1821 lustre-MDT0000_UUID: changing import state from CONNECTING to DISCONN 00000100:00080000:2.0:1776178584.441012:0:3352:0:(import.c:1429:ptlrpc_connect_interpret()) recovery of lustre-MDT0000_UUID on 192.168.204.103@tcp failed (-19) 00000100:00100000:2.0:1776178584.441017:0:3352:0:(client.c:2215:ptlrpc_check_set()) Completed RPC req@000000002917d432 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:lustre-MDT0001-mdtlov_UUID:3352:1862456998101952:0@lo:38:kworker/u8:2.0 00010000:00010000:1.0:1776178585.056606:0:9225:0:(ldlm_lockd.c:2792:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:1.0:1776178585.952321:0:3355:0:(ptlrpcd.c:415:ptlrpcd_check()) transfer 1 async RPCs [3->2] 00000100:00100000:1.0:1776178585.952405:0:3355:0:(client.c:1733:ptlrpc_send_new_req()) Sending RPC req@000000007216887e pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_02:lustre-MDT0001-mdtlov_UUID:3355:1862456998102208:0@lo:13:osp-pre-1-1.0 00000100:00100000:1.0:1776178585.952543:0:3355:0:(events.c:361:request_in_callback()) peer: 12345-0@lo (source: 12345-0@lo) 00000100:00100000:2.0:1776178585.952627:0:9554:0:(service.c:2152:ptlrpc_server_handle_req_in()) got req x1862456998102208 00000100:00100000:2.0:1776178585.952706:0:9554:0:(nrs_fifo.c:179:nrs_fifo_req_get()) NRS start fifo request from 12345-0@lo, seq: 637 00000100:00100000:2.0:1776178585.952713:0:9554:0:(service.c:2324:ptlrpc_server_handle_request()) Handling RPC req@00000000b9a3856b pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_create00:lustre-MDT0001-mdtlov_UUID+5:3355:x1862456998102208:12345-0@lo:13:osp-pre-1-1.0 00002000:00000024:2.0:1776178585.952743:0:9554:0:(ofd_obd.c:740:ofd_statfs()) blocks cached 229376 granted 491746368 pending 0 free 3923689472 avail 3692167168 00000100:00100000:2.0:1776178585.952787:0:9554:0:(service.c:2372:ptlrpc_server_handle_request()) Handled RPC req@00000000b9a3856b pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_create00:lustre-MDT0001-mdtlov_UUID+5:3355:x1862456998102208:12345-0@lo:13:osp-pre-1-1.0 Request processed in 74us (246us total) trans 0 rc 0/0 00000100:00100000:2.0:1776178585.952794:0:9554:0:(nrs_fifo.c:241:nrs_fifo_req_stop()) NRS stop fifo request from 12345-0@lo, seq: 637 00000100:00100000:1.0:1776178585.953028:0:3355:0:(client.c:2215:ptlrpc_check_set()) Completed RPC req@000000007216887e pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_02:lustre-MDT0001-mdtlov_UUID:3355:1862456998102208:0@lo:13:osp-pre-1-1.0 00010000:00010000:2.0:1776178586.081287:0:6244:0:(ldlm_lockd.c:2792:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:1.0:1776178586.464473:0:3354:0:(ptlrpcd.c:415:ptlrpcd_check()) transfer 1 async RPCs [0->1] 00000100:00100000:1.0:1776178586.464498:0:3354:0:(client.c:1733:ptlrpc_send_new_req()) Sending RPC req@000000008c3fbdb6 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_01:lustre-MDT0001-mdtlov_UUID:3354:1862456998102272:0@lo:13:osp-pre-0-1.0 00000100:00100000:1.0:1776178586.464572:0:3354:0:(events.c:361:request_in_callback()) peer: 12345-0@lo (source: 12345-0@lo) 00000100:00100000:2.0:1776178586.469010:0:9554:0:(service.c:2152:ptlrpc_server_handle_req_in()) got req x1862456998102272 00000100:00100000:2.0:1776178586.469123:0:9554:0:(nrs_fifo.c:179:nrs_fifo_req_get()) NRS start fifo request from 12345-0@lo, seq: 638 00000100:00100000:2.0:1776178586.469131:0:9554:0:(service.c:2324:ptlrpc_server_handle_request()) Handling RPC req@00000000599b0cf2 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_create00:lustre-MDT0001-mdtlov_UUID+5:3354:x1862456998102272:12345-0@lo:13:osp-pre-0-1.0 00002000:00000024:2.0:1776178586.469167:0:9554:0:(ofd_obd.c:740:ofd_statfs()) blocks cached 315392 granted 484883776 pending 0 free 3923689472 avail 3692167168 00000100:00100000:2.0:1776178586.469274:0:9554:0:(service.c:2372:ptlrpc_server_handle_request()) Handled RPC req@00000000599b0cf2 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_create00:lustre-MDT0001-mdtlov_UUID+5:3354:x1862456998102272:12345-0@lo:13:osp-pre-0-1.0 Request processed in 143us (4703us total) trans 0 rc 0/0 00000100:00100000:2.0:1776178586.469281:0:9554:0:(nrs_fifo.c:241:nrs_fifo_req_stop()) NRS stop fifo request from 12345-0@lo, seq: 638 00000100:00100000:1.0:1776178586.469535:0:3354:0:(client.c:2215:ptlrpc_check_set()) Completed RPC req@000000008c3fbdb6 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_01:lustre-MDT0001-mdtlov_UUID:3354:1862456998102272:0@lo:13:osp-pre-0-1.0 00010000:00010000:1.0:1776178587.105455:0:6245:0:(ldlm_lockd.c:2792:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:1.0:1776178587.964960:0:3346:0:(events.c:361:request_in_callback()) peer: 12345-192.168.204.3@tcp (source: 12345-192.168.204.3@tcp) 00000100:00100000:1.0:1776178587.965017:0:3346:0:(events.c:361:request_in_callback()) peer: 12345-192.168.204.3@tcp (source: 12345-192.168.204.3@tcp) 00000100:00100000:1.0:1776178587.965050:0:3346:0:(events.c:361:request_in_callback()) peer: 12345-192.168.204.3@tcp (source: 12345-192.168.204.3@tcp) 00000100:00100000:1.0:1776178587.965120:0:3346:0:(events.c:361:request_in_callback()) peer: 12345-192.168.204.3@tcp (source: 12345-192.168.204.3@tcp) 00000100:00100000:1.0:1776178587.965147:0:3346:0:(events.c:361:request_in_callback()) peer: 12345-192.168.204.3@tcp (source: 12345-192.168.204.3@tcp) 00000100:00100000:2.0:1776178587.965249:0:20762:0:(service.c:2152:ptlrpc_server_handle_req_in()) got req x1862456979133184 00000100:00100000:2.0:1776178587.965303:0:20762:0:(service.c:2152:ptlrpc_server_handle_req_in()) got req x1862456979133248 00000100:00100000:2.0:1776178587.965324:0:20762:0:(nrs_fifo.c:179:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.204.3@tcp, seq: 811 00000100:00100000:2.0:1776178587.965330:0:20762:0:(service.c:2324:ptlrpc_server_handle_request()) Handling RPC req@000000001f0b1867 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_013:b50963bd-8eba-4432-9a54-8888d00e3fa1+8:2271:x1862456979133184:12345-192.168.204.3@tcp:400:kworker/u8:3.0 00000100:00100000:2.0:1776178587.965410:0:20762:0:(service.c:2372:ptlrpc_server_handle_request()) Handled RPC req@000000001f0b1867 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_013:b50963bd-8eba-4432-9a54-8888d00e3fa1+8:2271:x1862456979133184:12345-192.168.204.3@tcp:400:kworker/u8:3.0 Request processed in 79us (360us total) trans 0 rc 0/0 00000100:00100000:2.0:1776178587.965417:0:20762:0:(nrs_fifo.c:241:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.204.3@tcp, seq: 811 00000100:00100000:2.0:1776178587.965433:0:20762:0:(nrs_fifo.c:179:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.204.3@tcp, seq: 812 00000100:00100000:2.0:1776178587.965437:0:20762:0:(service.c:2324:ptlrpc_server_handle_request()) Handling RPC req@0000000053e142bc pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_013:b50963bd-8eba-4432-9a54-8888d00e3fa1+9:2270:x1862456979133248:12345-192.168.204.3@tcp:400:kworker/u8:3.0 00000100:00100000:2.0:1776178587.965462:0:20762:0:(service.c:2372:ptlrpc_server_handle_request()) Handled RPC req@0000000053e142bc pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_013:b50963bd-8eba-4432-9a54-8888d00e3fa1+9:2270:x1862456979133248:12345-192.168.204.3@tcp:400:kworker/u8:3.0 Request processed in 24us (316us total) trans 0 rc 0/0 00000100:00100000:2.0:1776178587.965468:0:20762:0:(nrs_fifo.c:241:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.204.3@tcp, seq: 812 00000100:00100000:3.0:1776178587.966100:0:6257:0:(service.c:2152:ptlrpc_server_handle_req_in()) got req x1862456979133120 00000100:00100000:3.0:1776178587.966128:0:6257:0:(service.c:2152:ptlrpc_server_handle_req_in()) got req x1862456979133056 00000100:00100000:3.0:1776178587.966154:0:6257:0:(nrs_fifo.c:179:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.204.3@tcp, seq: 1153 00000100:00100000:3.0:1776178587.966158:0:6257:0:(service.c:2324:ptlrpc_server_handle_request()) Handling RPC req@000000005fdea934 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_002:b50963bd-8eba-4432-9a54-8888d00e3fa1+6:2270:x1862456979133120:12345-192.168.204.3@tcp:400:kworker/u8:3.0 00000100:00100000:3.0:1776178587.966187:0:6257:0:(service.c:2372:ptlrpc_server_handle_request()) Handled RPC req@000000005fdea934 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_002:b50963bd-8eba-4432-9a54-8888d00e3fa1+6:2270:x1862456979133120:12345-192.168.204.3@tcp:400:kworker/u8:3.0 Request processed in 28us (1237us total) trans 0 rc 0/0 00000100:00100000:3.0:1776178587.966192:0:6257:0:(nrs_fifo.c:241:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.204.3@tcp, seq: 1153 00000100:00100000:3.0:1776178587.966204:0:6257:0:(nrs_fifo.c:179:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.204.3@tcp, seq: 6924 00000100:00100000:3.0:1776178587.966208:0:6257:0:(service.c:2324:ptlrpc_server_handle_request()) Handling RPC req@0000000068adf51c pname:cluuid+ref:pid:xid:nid:opc:job mdt00_002:0+-99:2269:x1862456979133056:12345-192.168.204.3@tcp:38:kworker/u8:3.0 00010000:02000000:3.0:1776178587.966221:0:6257:0:(ldlm_lib.c:1139:target_handle_connect()) lustre-MDT0000: Not available for connect from 192.168.204.3@tcp (stopping) 00000100:00100000:3.0:1776178587.966615:0:6257:0:(service.c:2372:ptlrpc_server_handle_request()) Handled RPC req@0000000068adf51c pname:cluuid+ref:pid:xid:nid:opc:job mdt00_002:0+-99:2269:x1862456979133056:12345-192.168.204.3@tcp:38:kworker/u8:3.0 Request processed in 280us (1368us total) trans 0 rc -19/-19 00000100:00100000:3.0:1776178587.966623:0:6257:0:(nrs_fifo.c:241:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.204.3@tcp, seq: 6924 00000100:00100000:0.0:1776178587.969285:0:20137:0:(service.c:2152:ptlrpc_server_handle_req_in()) got req x1862456979132992 00000100:00100000:0.0:1776178587.969358:0:20137:0:(nrs_fifo.c:179:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.204.3@tcp, seq: 305 00000100:00100000:0.0:1776178587.969364:0:20137:0:(service.c:2324:ptlrpc_server_handle_request()) Handling RPC req@000000005f72129d pname:cluuid+ref:pid:xid:nid:opc:job ll_mgs_0000:5b5c66b1-1a2f-4651-8548-0c75b9a38a94+9:2271:x1862456979132992:12345-192.168.204.3@tcp:400:kworker/u8:3.0 00000100:00100000:0.0:1776178587.969457:0:20137:0:(service.c:2372:ptlrpc_server_handle_request()) Handled RPC req@000000005f72129d pname:cluuid+ref:pid:xid:nid:opc:job ll_mgs_0000:5b5c66b1-1a2f-4651-8548-0c75b9a38a94+9:2271:x1862456979132992:12345-192.168.204.3@tcp:400:kworker/u8:3.0 Request processed in 91us (4439us total) trans 0 rc 0/0 00000100:00100000:0.0:1776178587.969465:0:20137:0:(nrs_fifo.c:241:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.204.3@tcp, seq: 305 00010000:00010000:1.0:1776178588.129096:0:9224:0:(ldlm_lockd.c:2792:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00010000:00010000:1.0:1776178589.153435:0:9225:0:(ldlm_lockd.c:2792:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:0.0:1776178589.539326:0:3353:0:(ptlrpcd.c:415:ptlrpcd_check()) transfer 1 async RPCs [1->0] 00000100:00080000:3.0:1776178589.540268:0:37:0:(pinger.c:251:ptlrpc_pinger_process_import()) lustre-MDT0000-lwp-OST0000_UUID->lustre-MDT0000_UUID: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00080000:3.0:1776178589.540276:0:37:0:(recover.c:57:ptlrpc_initiate_recovery()) lustre-MDT0000_UUID: starting recovery 00000100:00080000:3.0:1776178589.540280:0:37:0:(import.c:86:import_set_state_nolock()) 0000000047601a98 lustre-MDT0000_UUID: changing import state from DISCONN to CONNECTING 00000100:00080000:3.0:1776178589.540288:0:37:0:(import.c:534:import_select_connection()) lustre-MDT0000-lwp-OST0000: connect to NID 0@lo last attempt 1291 00000100:00080000:3.0:1776178589.540292:0:37:0:(import.c:581:import_select_connection()) lustre-MDT0000-lwp-OST0000: tried all connections, increasing latency to 21s 00000100:00080000:3.0:1776178589.540298:0:37:0:(import.c:615:import_select_connection()) lustre-MDT0000-lwp-OST0000: import 0000000047601a98 using connection 192.168.204.103@tcp/0@lo 00000100:00100000:3.0:1776178589.540328:0:37:0:(import.c:807:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5) req@0000000063c20b43 x1862456998102400/t0(0) o38->lustre-MDT0000-lwp-OST0000@0@lo:12/10 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' 00000100:00080000:3.0:1776178589.540388:0:37:0:(pinger.c:251:ptlrpc_pinger_process_import()) lustre-MDT0000-lwp-OST0001_UUID->lustre-MDT0000_UUID: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00080000:3.0:1776178589.540391:0:37:0:(recover.c:57:ptlrpc_initiate_recovery()) lustre-MDT0000_UUID: starting recovery 00000100:00080000:3.0:1776178589.540393:0:37:0:(import.c:86:import_set_state_nolock()) 00000000dfc8db13 lustre-MDT0000_UUID: changing import state from DISCONN to CONNECTING 00000100:00080000:3.0:1776178589.540396:0:37:0:(import.c:534:import_select_connection()) lustre-MDT0000-lwp-OST0001: connect to NID 0@lo last attempt 1291 00000100:00080000:3.0:1776178589.540397:0:37:0:(import.c:581:import_select_connection()) lustre-MDT0000-lwp-OST0001: tried all connections, increasing latency to 21s 00000100:00080000:3.0:1776178589.540401:0:37:0:(import.c:615:import_select_connection()) lustre-MDT0000-lwp-OST0001: import 00000000dfc8db13 using connection 192.168.204.103@tcp/0@lo 00000100:00100000:3.0:1776178589.540436:0:37:0:(import.c:807:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5) req@00000000faeeda53 x1862456998102528/t0(0) o38->lustre-MDT0000-lwp-OST0001@0@lo:12/10 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' 00000100:00080000:3.0:1776178589.540451:0:37:0:(pinger.c:251:ptlrpc_pinger_process_import()) lustre-MDT0001-mdtlov_UUID->lustre-MDT0000_UUID: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00080000:3.0:1776178589.540454:0:37:0:(recover.c:57:ptlrpc_initiate_recovery()) lustre-MDT0000_UUID: starting recovery 00000100:00080000:3.0:1776178589.540455:0:37:0:(import.c:86:import_set_state_nolock()) 00000000ba4f1821 lustre-MDT0000_UUID: changing import state from DISCONN to CONNECTING 00000100:00080000:3.0:1776178589.540458:0:37:0:(import.c:534:import_select_connection()) lustre-MDT0000-osp-MDT0001: connect to NID 0@lo last attempt 1291 00000100:00080000:3.0:1776178589.540459:0:37:0:(import.c:581:import_select_connection()) lustre-MDT0000-osp-MDT0001: tried all connections, increasing latency to 21s 00000100:00080000:3.0:1776178589.540463:0:37:0:(import.c:615:import_select_connection()) lustre-MDT0000-osp-MDT0001: import 00000000ba4f1821 using connection 192.168.204.103@tcp/0@lo 00000100:00100000:3.0:1776178589.540474:0:37:0:(import.c:807:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5) req@00000000c91f3ff6 x1862456998102656/t0(0) o38->lustre-MDT0000-osp-MDT0001@0@lo:24/4 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' 00000100:00100000:1.0:1776178589.540543:0:3355:0:(client.c:1733:ptlrpc_send_new_req()) Sending RPC req@0000000092f98018 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_02:lustre-MDT0001-lwp-OST0000_UUID:3355:1862456998102464:0@lo:400:kworker/u8:2.0 00000100:00100000:1.0:1776178589.540612:0:3355:0:(events.c:361:request_in_callback()) peer: 12345-0@lo (source: 12345-0@lo) 00000100:00100000:1.0:1776178589.540650:0:3355:0:(ptlrpcd.c:415:ptlrpcd_check()) transfer 1 async RPCs [3->2] 00000100:00100000:1.0:1776178589.540655:0:3355:0:(client.c:1733:ptlrpc_send_new_req()) Sending RPC req@000000009015f30b pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_02:lustre-MDT0001-lwp-OST0001_UUID:3355:1862456998102592:0@lo:400:kworker/u8:2.0 00000100:00100000:1.0:1776178589.540804:0:3355:0:(events.c:361:request_in_callback()) peer: 12345-0@lo (source: 12345-0@lo) 00000100:00100000:1.0:1776178589.540861:0:8250:0:(service.c:2152:ptlrpc_server_handle_req_in()) got req x1862456998102464 00000100:00100000:1.0:1776178589.540906:0:8250:0:(service.c:2152:ptlrpc_server_handle_req_in()) got req x1862456998102592 00000100:00100000:1.0:1776178589.540922:0:8250:0:(nrs_fifo.c:179:nrs_fifo_req_get()) NRS start fifo request from 12345-0@lo, seq: 1154 00000100:00100000:1.0:1776178589.540927:0:8250:0:(service.c:2324:ptlrpc_server_handle_request()) Handling RPC req@000000001554c6b4 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_003:lustre-MDT0001-lwp-OST0000_UUID+5:3355:x1862456998102464:12345-0@lo:400:kworker/u8:2.0 00000100:00100000:1.0:1776178589.540972:0:8250:0:(service.c:2372:ptlrpc_server_handle_request()) Handled RPC req@000000001554c6b4 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_003:lustre-MDT0001-lwp-OST0000_UUID+5:3355:x1862456998102464:12345-0@lo:400:kworker/u8:2.0 Request processed in 44us (361us total) trans 0 rc 0/0 00000100:00100000:1.0:1776178589.540982:0:8250:0:(nrs_fifo.c:241:nrs_fifo_req_stop()) NRS stop fifo request from 12345-0@lo, seq: 1154 00000100:00100000:1.0:1776178589.540997:0:8250:0:(nrs_fifo.c:179:nrs_fifo_req_get()) NRS start fifo request from 12345-0@lo, seq: 1155 00000100:00100000:1.0:1776178589.541001:0:8250:0:(service.c:2324:ptlrpc_server_handle_request()) Handling RPC req@000000008a15a61d pname:cluuid+ref:pid:xid:nid:opc:job mdt00_003:lustre-MDT0001-lwp-OST0001_UUID+5:3355:x1862456998102592:12345-0@lo:400:kworker/u8:2.0 00000100:00100000:1.0:1776178589.541021:0:8250:0:(service.c:2372:ptlrpc_server_handle_request()) Handled RPC req@000000008a15a61d pname:cluuid+ref:pid:xid:nid:opc:job mdt00_003:lustre-MDT0001-lwp-OST0001_UUID+5:3355:x1862456998102592:12345-0@lo:400:kworker/u8:2.0 Request processed in 20us (218us total) trans 0 rc 0/0 00000100:00100000:1.0:1776178589.541026:0:8250:0:(nrs_fifo.c:241:nrs_fifo_req_stop()) NRS stop fifo request from 12345-0@lo, seq: 1155 00000100:00100000:1.0:1776178589.541093:0:3355:0:(client.c:2215:ptlrpc_check_set()) Completed RPC req@000000009015f30b pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_02:lustre-MDT0001-lwp-OST0001_UUID:3355:1862456998102592:0@lo:400:kworker/u8:2.0 00000100:00100000:1.0:1776178589.541107:0:3355:0:(client.c:2215:ptlrpc_check_set()) Completed RPC req@0000000092f98018 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_02:lustre-MDT0001-lwp-OST0000_UUID:3355:1862456998102464:0@lo:400:kworker/u8:2.0 00000100:00100000:2.0:1776178589.541344:0:3352:0:(client.c:1733:ptlrpc_send_new_req()) Sending RPC req@0000000063c20b43 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:lustre-MDT0000-lwp-OST0000_UUID:3352:1862456998102400:0@lo:38:kworker/u8:2.0 00000100:00100000:2.0:1776178589.541389:0:3352:0:(events.c:361:request_in_callback()) peer: 12345-0@lo (source: 12345-0@lo) 00000100:00100000:2.0:1776178589.541412:0:3352:0:(client.c:1733:ptlrpc_send_new_req()) Sending RPC req@00000000faeeda53 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:lustre-MDT0000-lwp-OST0001_UUID:3352:1862456998102528:0@lo:38:kworker/u8:2.0 00000100:00100000:2.0:1776178589.541431:0:3352:0:(events.c:361:request_in_callback()) peer: 12345-0@lo (source: 12345-0@lo) 00000100:00100000:2.0:1776178589.541444:0:3352:0:(client.c:1733:ptlrpc_send_new_req()) Sending RPC req@00000000c91f3ff6 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:lustre-MDT0001-mdtlov_UUID:3352:1862456998102656:0@lo:38:kworker/u8:2.0 00000100:00100000:2.0:1776178589.541465:0:3352:0:(events.c:361:request_in_callback()) peer: 12345-0@lo (source: 12345-0@lo) 00000100:00100000:2.0:1776178589.541491:0:11281:0:(service.c:2152:ptlrpc_server_handle_req_in()) got req x1862456998102400 00000100:00100000:2.0:1776178589.541510:0:11281:0:(service.c:2152:ptlrpc_server_handle_req_in()) got req x1862456998102528 00000100:00100000:2.0:1776178589.541522:0:11281:0:(nrs_fifo.c:179:nrs_fifo_req_get()) NRS start fifo request from 12345-0@lo, seq: 6925 00000100:00100000:2.0:1776178589.541526:0:11281:0:(service.c:2324:ptlrpc_server_handle_request()) Handling RPC req@000000002917d432 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_004:0+-99:3352:x1862456998102400:12345-0@lo:38:kworker/u8:2.0 00010000:02000000:2.0:1776178589.541538:0:11281:0:(ldlm_lib.c:1139:target_handle_connect()) lustre-MDT0000: Not available for connect from 0@lo (stopping) 00000100:00100000:2.0:1776178589.541568:0:11281:0:(service.c:2372:ptlrpc_server_handle_request()) Handled RPC req@000000002917d432 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_004:0+-99:3352:x1862456998102400:12345-0@lo:38:kworker/u8:2.0 Request processed in 42us (180us total) trans 0 rc -19/-19 00000100:00100000:2.0:1776178589.541574:0:11281:0:(nrs_fifo.c:241:nrs_fifo_req_stop()) NRS stop fifo request from 12345-0@lo, seq: 6925 00000100:00100000:2.0:1776178589.541586:0:11281:0:(nrs_fifo.c:179:nrs_fifo_req_get()) NRS start fifo request from 12345-0@lo, seq: 6926 00000100:00100000:2.0:1776178589.541588:0:11281:0:(service.c:2324:ptlrpc_server_handle_request()) Handling RPC req@0000000054bbc2c8 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_004:0+-99:3352:x1862456998102528:12345-0@lo:38:kworker/u8:2.0 00010000:02000000:2.0:1776178589.541594:0:11281:0:(ldlm_lib.c:1139:target_handle_connect()) lustre-MDT0000: Not available for connect from 0@lo (stopping) 00000100:00100000:2.0:1776178589.541611:0:11281:0:(service.c:2372:ptlrpc_server_handle_request()) Handled RPC req@0000000054bbc2c8 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_004:0+-99:3352:x1862456998102528:12345-0@lo:38:kworker/u8:2.0 Request processed in 22us (180us total) trans 0 rc -19/-19 00000100:00100000:2.0:1776178589.541616:0:11281:0:(nrs_fifo.c:241:nrs_fifo_req_stop()) NRS stop fifo request from 12345-0@lo, seq: 6926 00000100:00080000:2.0:1776178589.541640:0:3352:0:(import.c:86:import_set_state_nolock()) 0000000047601a98 lustre-MDT0000_UUID: changing import state from CONNECTING to DISCONN 00000100:00080000:2.0:1776178589.541644:0:3352:0:(import.c:1429:ptlrpc_connect_interpret()) recovery of lustre-MDT0000_UUID on 192.168.204.103@tcp failed (-19) 00000100:00100000:2.0:1776178589.541647:0:3352:0:(client.c:2215:ptlrpc_check_set()) Completed RPC req@0000000063c20b43 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:lustre-MDT0000-lwp-OST0000_UUID:3352:1862456998102400:0@lo:38:kworker/u8:2.0 00000100:00080000:2.0:1776178589.541654:0:3352:0:(import.c:86:import_set_state_nolock()) 00000000dfc8db13 lustre-MDT0000_UUID: changing import state from CONNECTING to DISCONN 00000100:00080000:2.0:1776178589.541656:0:3352:0:(import.c:1429:ptlrpc_connect_interpret()) recovery of lustre-MDT0000_UUID on 192.168.204.103@tcp failed (-19) 00000100:00100000:2.0:1776178589.541670:0:3352:0:(client.c:2215:ptlrpc_check_set()) Completed RPC req@00000000faeeda53 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:lustre-MDT0000-lwp-OST0001_UUID:3352:1862456998102528:0@lo:38:kworker/u8:2.0 00000100:00100000:2.0:1776178589.541722:0:14128:0:(service.c:2152:ptlrpc_server_handle_req_in()) got req x1862456998102656 00000100:00100000:2.0:1776178589.541743:0:14128:0:(nrs_fifo.c:179:nrs_fifo_req_get()) NRS start fifo request from 12345-0@lo, seq: 783 00000100:00100000:2.0:1776178589.541747:0:14128:0:(service.c:2324:ptlrpc_server_handle_request()) Handling RPC req@0000000028dd2aee pname:cluuid+ref:pid:xid:nid:opc:job mdt_out00_002:0+-99:3352:x1862456998102656:12345-0@lo:38:kworker/u8:2.0 00010000:02000000:2.0:1776178589.544290:0:14128:0:(ldlm_lib.c:1139:target_handle_connect()) lustre-MDT0000: Not available for connect from 0@lo (stopping) 00000100:00100000:2.0:1776178589.544398:0:14128:0:(service.c:2372:ptlrpc_server_handle_request()) Handled RPC req@0000000028dd2aee pname:cluuid+ref:pid:xid:nid:opc:job mdt_out00_002:0+-99:3352:x1862456998102656:12345-0@lo:38:kworker/u8:2.0 Request processed in 2647us (2929us total) trans 0 rc -19/-19 00000100:00100000:2.0:1776178589.544410:0:14128:0:(nrs_fifo.c:241:nrs_fifo_req_stop()) NRS stop fifo request from 12345-0@lo, seq: 783 00000100:00080000:1.0:1776178589.545118:0:3352:0:(import.c:86:import_set_state_nolock()) 00000000ba4f1821 lustre-MDT0000_UUID: changing import state from CONNECTING to DISCONN 00000100:00080000:1.0:1776178589.545125:0:3352:0:(import.c:1429:ptlrpc_connect_interpret()) recovery of lustre-MDT0000_UUID on 192.168.204.103@tcp failed (-19) 00000100:00100000:1.0:1776178589.545139:0:3352:0:(client.c:2215:ptlrpc_check_set()) Completed RPC req@00000000c91f3ff6 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:lustre-MDT0001-mdtlov_UUID:3352:1862456998102656:0@lo:38:kworker/u8:2.0 00000100:00100000:0.0:1776178589.545528:0:3353:0:(client.c:1733:ptlrpc_send_new_req()) Sending RPC req@0000000009435d5b pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_00:lustre-MDT0001-mdtlov_UUID:3353:1862456998102720:0@lo:400:kworker/u8:2.0 00000100:00100000:0.0:1776178589.545589:0:3353:0:(events.c:361:request_in_callback()) peer: 12345-0@lo (source: 12345-0@lo) 00000100:00100000:0.0:1776178589.545614:0:3353:0:(client.c:1733:ptlrpc_send_new_req()) Sending RPC req@000000007216887e pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_00:ac61f839-0e8e-4cac-9c59-08bcfbbad330:3353:1862456998102336:0@lo:400:kworker/u8:2.0 00000100:00100000:0.0:1776178589.545641:0:3353:0:(events.c:361:request_in_callback()) peer: 12345-0@lo (source: 12345-0@lo) 00000100:00100000:1.0:1776178589.546047:0:20755:0:(service.c:2152:ptlrpc_server_handle_req_in()) got req x1862456998102720 00000100:00100000:1.0:1776178589.546128:0:20755:0:(nrs_fifo.c:179:nrs_fifo_req_get()) NRS start fifo request from 12345-0@lo, seq: 813 00000100:00100000:1.0:1776178589.546144:0:20755:0:(service.c:2324:ptlrpc_server_handle_request()) Handling RPC req@000000008c3fbdb6 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_006:lustre-MDT0001-mdtlov_UUID+5:3353:x1862456998102720:12345-0@lo:400:kworker/u8:2.0 00000100:00100000:1.0:1776178589.546203:0:20755:0:(service.c:2372:ptlrpc_server_handle_request()) Handled RPC req@000000008c3fbdb6 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_006:lustre-MDT0001-mdtlov_UUID+5:3353:x1862456998102720:12345-0@lo:400:kworker/u8:2.0 Request processed in 59us (614us total) trans 0 rc 0/0 00000100:00100000:1.0:1776178589.546209:0:20755:0:(nrs_fifo.c:241:nrs_fifo_req_stop()) NRS stop fifo request from 12345-0@lo, seq: 813 00000100:00100000:0.0:1776178589.551519:0:3353:0:(client.c:2215:ptlrpc_check_set()) Completed RPC req@0000000009435d5b pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_00:lustre-MDT0001-mdtlov_UUID:3353:1862456998102720:0@lo:400:kworker/u8:2.0 00000100:00100000:2.0:1776178589.552831:0:20140:0:(service.c:2152:ptlrpc_server_handle_req_in()) got req x1862456998102336 00000100:00100000:2.0:1776178589.552892:0:20140:0:(nrs_fifo.c:179:nrs_fifo_req_get()) NRS start fifo request from 12345-0@lo, seq: 306 00000100:00100000:2.0:1776178589.552899:0:20140:0:(service.c:2324:ptlrpc_server_handle_request()) Handling RPC req@00000000a6b47274 pname:cluuid+ref:pid:xid:nid:opc:job ll_mgs_0002:ac61f839-0e8e-4cac-9c59-08bcfbbad330+18:3353:x1862456998102336:12345-0@lo:400:kworker/u8:2.0 00000100:00100000:2.0:1776178589.552993:0:20140:0:(service.c:2372:ptlrpc_server_handle_request()) Handled RPC req@00000000a6b47274 pname:cluuid+ref:pid:xid:nid:opc:job ll_mgs_0002:ac61f839-0e8e-4cac-9c59-08bcfbbad330+18:3353:x1862456998102336:12345-0@lo:400:kworker/u8:2.0 Request processed in 93us (7351us total) trans 0 rc 0/0 00000100:00100000:2.0:1776178589.553000:0:20140:0:(nrs_fifo.c:241:nrs_fifo_req_stop()) NRS stop fifo request from 12345-0@lo, seq: 306 00000100:00100000:0.0:1776178589.554347:0:3353:0:(client.c:2215:ptlrpc_check_set()) Completed RPC req@000000007216887e pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_00:ac61f839-0e8e-4cac-9c59-08bcfbbad330:3353:1862456998102336:0@lo:400:kworker/u8:2.0 00000100:00080000:3.0:1776178589.563624:0:37:0:(pinger.c:251:ptlrpc_pinger_process_import()) lustre-MDT0000-lwp-MDT0001_UUID->lustre-MDT0000_UUID: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00080000:3.0:1776178589.563632:0:37:0:(recover.c:57:ptlrpc_initiate_recovery()) lustre-MDT0000_UUID: starting recovery 00000100:00080000:3.0:1776178589.563635:0:37:0:(import.c:86:import_set_state_nolock()) 00000000a7c55b1f lustre-MDT0000_UUID: changing import state from DISCONN to CONNECTING 00000100:00080000:3.0:1776178589.563643:0:37:0:(import.c:534:import_select_connection()) lustre-MDT0000-lwp-MDT0001: connect to NID 0@lo last attempt 1291 00000100:00080000:3.0:1776178589.563646:0:37:0:(import.c:581:import_select_connection()) lustre-MDT0000-lwp-MDT0001: tried all connections, increasing latency to 21s 00000100:00080000:3.0:1776178589.563652:0:37:0:(import.c:615:import_select_connection()) lustre-MDT0000-lwp-MDT0001: import 00000000a7c55b1f using connection 192.168.204.103@tcp/0@lo 00000100:00100000:3.0:1776178589.563672:0:37:0:(import.c:807:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5) req@0000000081aacc4a x1862456998102848/t0(0) o38->lustre-MDT0000-lwp-MDT0001@0@lo:12/10 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' 00000100:00100000:2.0:1776178589.563894:0:3352:0:(client.c:1733:ptlrpc_send_new_req()) Sending RPC req@0000000081aacc4a pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:lustre-MDT0000-lwp-MDT0001_UUID:3352:1862456998102848:0@lo:38:kworker/u8:2.0 00000100:00100000:2.0:1776178589.563966:0:3352:0:(events.c:361:request_in_callback()) peer: 12345-0@lo (source: 12345-0@lo) 00000100:00100000:3.0:1776178589.565814:0:6256:0:(service.c:2152:ptlrpc_server_handle_req_in()) got req x1862456998102848 00000100:00100000:3.0:1776178589.565874:0:6256:0:(nrs_fifo.c:179:nrs_fifo_req_get()) NRS start fifo request from 12345-0@lo, seq: 6927 00000100:00100000:3.0:1776178589.565881:0:6256:0:(service.c:2324:ptlrpc_server_handle_request()) Handling RPC req@00000000ccad874e pname:cluuid+ref:pid:xid:nid:opc:job mdt00_001:0+-99:3352:x1862456998102848:12345-0@lo:38:kworker/u8:2.0 00010000:02000000:3.0:1776178589.565896:0:6256:0:(ldlm_lib.c:1139:target_handle_connect()) lustre-MDT0000: Not available for connect from 0@lo (stopping) 00000100:00100000:3.0:1776178589.565971:0:6256:0:(service.c:2372:ptlrpc_server_handle_request()) Handled RPC req@00000000ccad874e pname:cluuid+ref:pid:xid:nid:opc:job mdt00_001:0+-99:3352:x1862456998102848:12345-0@lo:38:kworker/u8:2.0 Request processed in 87us (2002us total) trans 0 rc -19/-19 00000100:00100000:3.0:1776178589.565977:0:6256:0:(nrs_fifo.c:241:nrs_fifo_req_stop()) NRS stop fifo request from 12345-0@lo, seq: 6927 00000100:00080000:2.0:1776178589.566859:0:3352:0:(import.c:86:import_set_state_nolock()) 00000000a7c55b1f lustre-MDT0000_UUID: changing import state from CONNECTING to DISCONN 00000100:00080000:2.0:1776178589.566866:0:3352:0:(import.c:1429:ptlrpc_connect_interpret()) recovery of lustre-MDT0000_UUID on 192.168.204.103@tcp failed (-19) 00000100:00100000:2.0:1776178589.566871:0:3352:0:(client.c:2215:ptlrpc_check_set()) Completed RPC req@0000000081aacc4a pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:lustre-MDT0000-lwp-MDT0001_UUID:3352:1862456998102848:0@lo:38:kworker/u8:2.0 00000100:00100000:1.0:1776178589.567030:0:3354:0:(client.c:1733:ptlrpc_send_new_req()) Sending RPC req@00000000ad99db59 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_01:lustre-MDT0001-mdtlov_UUID:3354:1862456998102784:0@lo:400:kworker/u8:2.0 00000100:00100000:1.0:1776178589.567128:0:3354:0:(events.c:361:request_in_callback()) peer: 12345-0@lo (source: 12345-0@lo) 00000100:00100000:2.0:1776178589.567805:0:14440:0:(service.c:2152:ptlrpc_server_handle_req_in()) got req x1862456998102784 00000100:00100000:2.0:1776178589.567857:0:14440:0:(nrs_fifo.c:179:nrs_fifo_req_get()) NRS start fifo request from 12345-0@lo, seq: 814 00000100:00100000:2.0:1776178589.567862:0:14440:0:(service.c:2324:ptlrpc_server_handle_request()) Handling RPC req@00000000e4ab0682 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_004:lustre-MDT0001-mdtlov_UUID+5:3354:x1862456998102784:12345-0@lo:400:kworker/u8:2.0 00000100:00100000:2.0:1776178589.567931:0:14440:0:(service.c:2372:ptlrpc_server_handle_request()) Handled RPC req@00000000e4ab0682 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_004:lustre-MDT0001-mdtlov_UUID+5:3354:x1862456998102784:12345-0@lo:400:kworker/u8:2.0 Request processed in 68us (805us total) trans 0 rc 0/0 00000100:00100000:2.0:1776178589.567938:0:14440:0:(nrs_fifo.c:241:nrs_fifo_req_stop()) NRS stop fifo request from 12345-0@lo, seq: 814 00000100:00100000:3.0:1776178589.569990:0:3354:0:(client.c:2215:ptlrpc_check_set()) Completed RPC req@00000000ad99db59 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_01:lustre-MDT0001-mdtlov_UUID:3354:1862456998102784:0@lo:400:kworker/u8:2.0 00010000:00010000:2.0:1776178590.176880:0:6244:0:(ldlm_lockd.c:2792:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:1.0:1776178591.073080:0:3355:0:(client.c:1733:ptlrpc_send_new_req()) Sending RPC req@00000000ad99db59 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_02:lustre-MDT0001-mdtlov_UUID:3355:1862456998102912:0@lo:13:osp-pre-1-1.0 00000100:00100000:1.0:1776178591.073157:0:3355:0:(events.c:361:request_in_callback()) peer: 12345-0@lo (source: 12345-0@lo) 00000100:00100000:0.0:1776178591.073468:0:8231:0:(service.c:2152:ptlrpc_server_handle_req_in()) got req x1862456998102912 00000100:00100000:0.0:1776178591.073554:0:8231:0:(nrs_fifo.c:179:nrs_fifo_req_get()) NRS start fifo request from 12345-0@lo, seq: 639 00000100:00100000:0.0:1776178591.073560:0:8231:0:(service.c:2324:ptlrpc_server_handle_request()) Handling RPC req@00000000e4dc0a67 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_create00:lustre-MDT0001-mdtlov_UUID+5:3355:x1862456998102912:12345-0@lo:13:osp-pre-1-1.0 00002000:00000024:0.0:1776178591.073591:0:8231:0:(ofd_obd.c:740:ofd_statfs()) blocks cached 229376 granted 491746368 pending 0 free 3923689472 avail 3692167168 00000100:00100000:0.0:1776178591.073742:0:8231:0:(service.c:2372:ptlrpc_server_handle_request()) Handled RPC req@00000000e4dc0a67 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_create00:lustre-MDT0001-mdtlov_UUID+5:3355:x1862456998102912:12345-0@lo:13:osp-pre-1-1.0 Request processed in 181us (585us total) trans 0 rc 0/0 00000100:00100000:0.0:1776178591.073750:0:8231:0:(nrs_fifo.c:241:nrs_fifo_req_stop()) NRS stop fifo request from 12345-0@lo, seq: 639 00000100:00100000:1.0:1776178591.073833:0:3355:0:(client.c:2215:ptlrpc_check_set()) Completed RPC req@00000000ad99db59 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_02:lustre-MDT0001-mdtlov_UUID:3355:1862456998102912:0@lo:13:osp-pre-1-1.0 00010000:00010000:1.0:1776178591.200303:0:6245:0:(ldlm_lockd.c:2792:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:3.0:1776178591.584357:0:3356:0:(client.c:1733:ptlrpc_send_new_req()) Sending RPC req@000000007216887e pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_03:lustre-MDT0001-mdtlov_UUID:3356:1862456998102976:0@lo:13:osp-pre-0-1.0 00000100:00100000:3.0:1776178591.584447:0:3356:0:(events.c:361:request_in_callback()) peer: 12345-0@lo (source: 12345-0@lo) 00000100:00100000:1.0:1776178591.585015:0:8231:0:(service.c:2152:ptlrpc_server_handle_req_in()) got req x1862456998102976 00000100:00100000:1.0:1776178591.585138:0:8231:0:(nrs_fifo.c:179:nrs_fifo_req_get()) NRS start fifo request from 12345-0@lo, seq: 640 00000100:00100000:1.0:1776178591.585144:0:8231:0:(service.c:2324:ptlrpc_server_handle_request()) Handling RPC req@00000000b206023d pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_create00:lustre-MDT0001-mdtlov_UUID+5:3356:x1862456998102976:12345-0@lo:13:osp-pre-0-1.0 00002000:00000024:1.0:1776178591.585174:0:8231:0:(ofd_obd.c:740:ofd_statfs()) blocks cached 315392 granted 484883776 pending 0 free 3923689472 avail 3692167168 00000100:00100000:1.0:1776178591.585271:0:8231:0:(service.c:2372:ptlrpc_server_handle_request()) Handled RPC req@00000000b206023d pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_create00:lustre-MDT0001-mdtlov_UUID+5:3356:x1862456998102976:12345-0@lo:13:osp-pre-0-1.0 Request processed in 126us (826us total) trans 0 rc 0/0 00000100:00100000:1.0:1776178591.585278:0:8231:0:(nrs_fifo.c:241:nrs_fifo_req_stop()) NRS stop fifo request from 12345-0@lo, seq: 640 00000100:00100000:3.0:1776178591.585349:0:3356:0:(client.c:2215:ptlrpc_check_set()) Completed RPC req@000000007216887e pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_03:lustre-MDT0001-mdtlov_UUID:3356:1862456998102976:0@lo:13:osp-pre-0-1.0 00010000:00010000:1.0:1776178592.225125:0:9224:0:(ldlm_lockd.c:2792:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:3.0:1776178593.058538:0:3347:0:(events.c:361:request_in_callback()) peer: 12345-192.168.204.3@tcp (source: 12345-192.168.204.3@tcp) 00000100:00100000:3.0:1776178593.058600:0:3347:0:(events.c:361:request_in_callback()) peer: 12345-192.168.204.3@tcp (source: 12345-192.168.204.3@tcp) 00000100:00100000:3.0:1776178593.058633:0:3347:0:(events.c:361:request_in_callback()) peer: 12345-192.168.204.3@tcp (source: 12345-192.168.204.3@tcp) 00000100:00100000:3.0:1776178593.058662:0:3347:0:(events.c:361:request_in_callback()) peer: 12345-192.168.204.3@tcp (source: 12345-192.168.204.3@tcp) 00000100:00100000:3.0:1776178593.058743:0:3347:0:(events.c:361:request_in_callback()) peer: 12345-192.168.204.3@tcp (source: 12345-192.168.204.3@tcp) 00000100:00100000:3.0:1776178593.058783:0:6256:0:(service.c:2152:ptlrpc_server_handle_req_in()) got req x1862456979133376 00000100:00100000:3.0:1776178593.058817:0:6256:0:(service.c:2152:ptlrpc_server_handle_req_in()) got req x1862456979133440 00000100:00100000:3.0:1776178593.058848:0:6256:0:(nrs_fifo.c:179:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.204.3@tcp, seq: 1156 00000100:00100000:3.0:1776178593.058856:0:6256:0:(service.c:2324:ptlrpc_server_handle_request()) Handling RPC req@0000000051c85b9b pname:cluuid+ref:pid:xid:nid:opc:job mdt00_001:b50963bd-8eba-4432-9a54-8888d00e3fa1+6:2271:x1862456979133440:12345-192.168.204.3@tcp:400:kworker/u8:3.0 00000100:00100000:3.0:1776178593.058928:0:6256:0:(service.c:2372:ptlrpc_server_handle_request()) Handled RPC req@0000000051c85b9b pname:cluuid+ref:pid:xid:nid:opc:job mdt00_001:b50963bd-8eba-4432-9a54-8888d00e3fa1+6:2271:x1862456979133440:12345-192.168.204.3@tcp:400:kworker/u8:3.0 Request processed in 72us (265us total) trans 0 rc 0/0 00000100:00100000:3.0:1776178593.058936:0:6256:0:(nrs_fifo.c:241:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.204.3@tcp, seq: 1156 00000100:00100000:3.0:1776178593.058953:0:6256:0:(nrs_fifo.c:179:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.204.3@tcp, seq: 6928 00000100:00100000:3.0:1776178593.058957:0:6256:0:(service.c:2324:ptlrpc_server_handle_request()) Handling RPC req@00000000dc38eb5b pname:cluuid+ref:pid:xid:nid:opc:job mdt00_001:0+-99:2269:x1862456979133376:12345-192.168.204.3@tcp:38:kworker/u8:3.0 00010000:02000000:3.0:1776178593.058971:0:6256:0:(ldlm_lib.c:1139:target_handle_connect()) lustre-MDT0000: Not available for connect from 192.168.204.3@tcp (stopping) 00000100:00100000:3.0:1776178593.058997:0:6256:0:(service.c:2372:ptlrpc_server_handle_request()) Handled RPC req@00000000dc38eb5b pname:cluuid+ref:pid:xid:nid:opc:job mdt00_001:0+-99:2269:x1862456979133376:12345-192.168.204.3@tcp:38:kworker/u8:3.0 Request processed in 39us (468us total) trans 0 rc -19/-19 00000100:00100000:3.0:1776178593.059004:0:6256:0:(nrs_fifo.c:241:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.204.3@tcp, seq: 6928 00000100:00100000:3.0:1776178593.059024:0:14440:0:(service.c:2152:ptlrpc_server_handle_req_in()) got req x1862456979133504 00000100:00100000:3.0:1776178593.059045:0:14440:0:(service.c:2152:ptlrpc_server_handle_req_in()) got req x1862456979133568 00000100:00100000:3.0:1776178593.059098:0:14440:0:(nrs_fifo.c:179:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.204.3@tcp, seq: 815 00000100:00100000:3.0:1776178593.059102:0:14440:0:(service.c:2324:ptlrpc_server_handle_request()) Handling RPC req@0000000045cb5035 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_004:b50963bd-8eba-4432-9a54-8888d00e3fa1+8:2271:x1862456979133504:12345-192.168.204.3@tcp:400:kworker/u8:3.0 00000100:00100000:3.0:1776178593.059141:0:14440:0:(service.c:2372:ptlrpc_server_handle_request()) Handled RPC req@0000000045cb5035 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_004:b50963bd-8eba-4432-9a54-8888d00e3fa1+8:2271:x1862456979133504:12345-192.168.204.3@tcp:400:kworker/u8:3.0 Request processed in 37us (508us total) trans 0 rc 0/0 00000100:00100000:3.0:1776178593.059147:0:14440:0:(nrs_fifo.c:241:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.204.3@tcp, seq: 815 00000100:00100000:3.0:1776178593.059159:0:14440:0:(nrs_fifo.c:179:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.204.3@tcp, seq: 816 00000100:00100000:3.0:1776178593.059163:0:14440:0:(service.c:2324:ptlrpc_server_handle_request()) Handling RPC req@000000006d8fce24 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_004:b50963bd-8eba-4432-9a54-8888d00e3fa1+9:2271:x1862456979133568:12345-192.168.204.3@tcp:400:kworker/u8:3.0 00000100:00100000:3.0:1776178593.059185:0:14440:0:(service.c:2372:ptlrpc_server_handle_request()) Handled RPC req@000000006d8fce24 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_004:b50963bd-8eba-4432-9a54-8888d00e3fa1+9:2271:x1862456979133568:12345-192.168.204.3@tcp:400:kworker/u8:3.0 Request processed in 22us (455us total) trans 0 rc 0/0 00000100:00100000:3.0:1776178593.059191:0:14440:0:(nrs_fifo.c:241:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.204.3@tcp, seq: 816 00000100:00100000:0.0:1776178593.059498:0:20137:0:(service.c:2152:ptlrpc_server_handle_req_in()) got req x1862456979133312 00000100:00100000:0.0:1776178593.059534:0:20137:0:(nrs_fifo.c:179:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.204.3@tcp, seq: 307 00000100:00100000:0.0:1776178593.059538:0:20137:0:(service.c:2324:ptlrpc_server_handle_request()) Handling RPC req@0000000080051315 pname:cluuid+ref:pid:xid:nid:opc:job ll_mgs_0000:5b5c66b1-1a2f-4651-8548-0c75b9a38a94+9:2271:x1862456979133312:12345-192.168.204.3@tcp:400:kworker/u8:3.0 00000100:00100000:0.0:1776178593.059579:0:20137:0:(service.c:2372:ptlrpc_server_handle_request()) Handled RPC req@0000000080051315 pname:cluuid+ref:pid:xid:nid:opc:job ll_mgs_0000:5b5c66b1-1a2f-4651-8548-0c75b9a38a94+9:2271:x1862456979133312:12345-192.168.204.3@tcp:400:kworker/u8:3.0 Request processed in 39us (979us total) trans 0 rc 0/0 00000100:00100000:0.0:1776178593.059586:0:20137:0:(nrs_fifo.c:241:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.204.3@tcp, seq: 307 00010000:00010000:1.0:1776178593.248250:0:9225:0:(ldlm_lockd.c:2792:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00010000:00010000:2.0:1776178594.414961:0:6244:0:(ldlm_lockd.c:2792:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00080000:1.0:1776178594.666920:0:36:0:(pinger.c:251:ptlrpc_pinger_process_import()) lustre-MDT0000-lwp-OST0000_UUID->lustre-MDT0000_UUID: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00080000:1.0:1776178594.666932:0:36:0:(recover.c:57:ptlrpc_initiate_recovery()) lustre-MDT0000_UUID: starting recovery 00000100:00080000:1.0:1776178594.666936:0:36:0:(import.c:86:import_set_state_nolock()) 0000000047601a98 lustre-MDT0000_UUID: changing import state from DISCONN to CONNECTING 00000100:00080000:1.0:1776178594.666945:0:36:0:(import.c:534:import_select_connection()) lustre-MDT0000-lwp-OST0000: connect to NID 0@lo last attempt 1296 00000100:00080000:1.0:1776178594.666949:0:36:0:(import.c:581:import_select_connection()) lustre-MDT0000-lwp-OST0000: tried all connections, increasing latency to 21s 00000100:00100000:0.0:1776178594.666955:0:3353:0:(client.c:1733:ptlrpc_send_new_req()) Sending RPC req@00000000fa20ba2c pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_00:ac61f839-0e8e-4cac-9c59-08bcfbbad330:3353:1862456998103040:0@lo:400:kworker/u8:1.0 00000100:00080000:1.0:1776178594.666956:0:36:0:(import.c:615:import_select_connection()) lustre-MDT0000-lwp-OST0000: import 0000000047601a98 using connection 192.168.204.103@tcp/0@lo 00000100:00100000:1.0:1776178594.666984:0:36:0:(import.c:807:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5) req@00000000238121c2 x1862456998103104/t0(0) o38->lustre-MDT0000-lwp-OST0000@0@lo:12/10 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' 00000100:00080000:1.0:1776178594.667008:0:36:0:(pinger.c:251:ptlrpc_pinger_process_import()) lustre-MDT0000-lwp-OST0001_UUID->lustre-MDT0000_UUID: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00080000:1.0:1776178594.667012:0:36:0:(recover.c:57:ptlrpc_initiate_recovery()) lustre-MDT0000_UUID: starting recovery 00000100:00080000:1.0:1776178594.667013:0:36:0:(import.c:86:import_set_state_nolock()) 00000000dfc8db13 lustre-MDT0000_UUID: changing import state from DISCONN to CONNECTING 00000100:00080000:1.0:1776178594.667016:0:36:0:(import.c:534:import_select_connection()) lustre-MDT0000-lwp-OST0001: connect to NID 0@lo last attempt 1296 00000100:00080000:1.0:1776178594.667018:0:36:0:(import.c:581:import_select_connection()) lustre-MDT0000-lwp-OST0001: tried all connections, increasing latency to 21s 00000100:00080000:1.0:1776178594.667021:0:36:0:(import.c:615:import_select_connection()) lustre-MDT0000-lwp-OST0001: import 00000000dfc8db13 using connection 192.168.204.103@tcp/0@lo 00000100:00100000:1.0:1776178594.667029:0:36:0:(import.c:807:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5) req@00000000907210d1 x1862456998103232/t0(0) o38->lustre-MDT0000-lwp-OST0001@0@lo:12/10 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' 00000100:00100000:0.0:1776178594.667121:0:3353:0:(events.c:361:request_in_callback()) peer: 12345-0@lo (source: 12345-0@lo) 00000100:00100000:2.0:1776178594.667190:0:3352:0:(client.c:1733:ptlrpc_send_new_req()) Sending RPC req@00000000238121c2 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:lustre-MDT0000-lwp-OST0000_UUID:3352:1862456998103104:0@lo:38:kworker/u8:1.0 00000100:00100000:2.0:1776178594.667268:0:3352:0:(events.c:361:request_in_callback()) peer: 12345-0@lo (source: 12345-0@lo) 00000100:00100000:2.0:1776178594.667299:0:3352:0:(client.c:1733:ptlrpc_send_new_req()) Sending RPC req@00000000907210d1 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:lustre-MDT0000-lwp-OST0001_UUID:3352:1862456998103232:0@lo:38:kworker/u8:1.0 00000100:00100000:2.0:1776178594.667323:0:3352:0:(events.c:361:request_in_callback()) peer: 12345-0@lo (source: 12345-0@lo) 00000100:00100000:2.0:1776178594.667367:0:3355:0:(client.c:1733:ptlrpc_send_new_req()) Sending RPC req@00000000b1db1835 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_02:lustre-MDT0001-lwp-OST0001_UUID:3355:1862456998103296:0@lo:400:kworker/u8:1.0 00000100:00100000:2.0:1776178594.667390:0:3355:0:(events.c:361:request_in_callback()) peer: 12345-0@lo (source: 12345-0@lo) 00000100:00100000:3.0:1776178594.669132:0:3354:0:(client.c:1733:ptlrpc_send_new_req()) Sending RPC req@000000003777ab0e pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_01:lustre-MDT0001-lwp-OST0000_UUID:3354:1862456998103168:0@lo:400:kworker/u8:1.0 00000100:00080000:1.0:1776178594.670376:0:36:0:(pinger.c:251:ptlrpc_pinger_process_import()) lustre-MDT0001-mdtlov_UUID->lustre-MDT0000_UUID: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00080000:1.0:1776178594.670383:0:36:0:(recover.c:57:ptlrpc_initiate_recovery()) lustre-MDT0000_UUID: starting recovery 00000100:00080000:1.0:1776178594.670387:0:36:0:(import.c:86:import_set_state_nolock()) 00000000ba4f1821 lustre-MDT0000_UUID: changing import state from DISCONN to CONNECTING 00000100:00080000:1.0:1776178594.670392:0:36:0:(import.c:534:import_select_connection()) lustre-MDT0000-osp-MDT0001: connect to NID 0@lo last attempt 1296 00000100:00080000:1.0:1776178594.670394:0:36:0:(import.c:581:import_select_connection()) lustre-MDT0000-osp-MDT0001: tried all connections, increasing latency to 21s 00000100:00080000:1.0:1776178594.670400:0:36:0:(import.c:615:import_select_connection()) lustre-MDT0000-osp-MDT0001: import 00000000ba4f1821 using connection 192.168.204.103@tcp/0@lo 00000100:00100000:1.0:1776178594.670435:0:36:0:(import.c:807:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5) req@000000006c0c7de5 x1862456998103360/t0(0) o38->lustre-MDT0000-osp-MDT0001@0@lo:24/4 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' 00000100:00080000:1.0:1776178594.670479:0:36:0:(pinger.c:251:ptlrpc_pinger_process_import()) lustre-MDT0000-lwp-MDT0001_UUID->lustre-MDT0000_UUID: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00080000:1.0:1776178594.670483:0:36:0:(recover.c:57:ptlrpc_initiate_recovery()) lustre-MDT0000_UUID: starting recovery 00000100:00080000:1.0:1776178594.670484:0:36:0:(import.c:86:import_set_state_nolock()) 00000000a7c55b1f lustre-MDT0000_UUID: changing import state from DISCONN to CONNECTING 00000100:00080000:1.0:1776178594.670488:0:36:0:(import.c:534:import_select_connection()) lustre-MDT0000-lwp-MDT0001: connect to NID 0@lo last attempt 1296 00000100:00080000:1.0:1776178594.670490:0:36:0:(import.c:581:import_select_connection()) lustre-MDT0000-lwp-MDT0001: tried all connections, increasing latency to 21s 00000100:00080000:1.0:1776178594.670493:0:36:0:(import.c:615:import_select_connection()) lustre-MDT0000-lwp-MDT0001: import 00000000a7c55b1f using connection 192.168.204.103@tcp/0@lo 00000100:00100000:1.0:1776178594.670505:0:36:0:(import.c:807:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5) req@00000000e964920a x1862456998103552/t0(0) o38->lustre-MDT0000-lwp-MDT0001@0@lo:12/10 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' 00000100:00100000:0.0:1776178594.670539:0:3353:0:(client.c:1733:ptlrpc_send_new_req()) Sending RPC req@0000000011f8e42c pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_00:lustre-MDT0001-mdtlov_UUID:3353:1862456998103488:0@lo:400:kworker/u8:1.0 00000100:00100000:0.0:1776178594.670583:0:3353:0:(events.c:361:request_in_callback()) peer: 12345-0@lo (source: 12345-0@lo) 00000100:00100000:0.0:1776178594.670623:0:20762:0:(service.c:2152:ptlrpc_server_handle_req_in()) got req x1862456998103488 00000100:00100000:0.0:1776178594.670659:0:20762:0:(nrs_fifo.c:179:nrs_fifo_req_get()) NRS start fifo request from 12345-0@lo, seq: 817 00000100:00100000:0.0:1776178594.670665:0:20762:0:(service.c:2324:ptlrpc_server_handle_request()) Handling RPC req@000000007216887e pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_013:lustre-MDT0001-mdtlov_UUID+5:3353:x1862456998103488:12345-0@lo:400:kworker/u8:1.0 00000100:00100000:0.0:1776178594.670724:0:20762:0:(service.c:2372:ptlrpc_server_handle_request()) Handled RPC req@000000007216887e pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_013:lustre-MDT0001-mdtlov_UUID+5:3353:x1862456998103488:12345-0@lo:400:kworker/u8:1.0 Request processed in 58us (141us total) trans 0 rc 0/0 00000100:00100000:0.0:1776178594.670731:0:20762:0:(nrs_fifo.c:241:nrs_fifo_req_stop()) NRS stop fifo request from 12345-0@lo, seq: 817 00000100:00100000:0.0:1776178594.670771:0:3353:0:(client.c:2215:ptlrpc_check_set()) Completed RPC req@0000000011f8e42c pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_00:lustre-MDT0001-mdtlov_UUID:3353:1862456998103488:0@lo:400:kworker/u8:1.0 00000100:00100000:0.0:1776178594.670803:0:3352:0:(client.c:1733:ptlrpc_send_new_req()) Sending RPC req@000000006c0c7de5 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:lustre-MDT0001-mdtlov_UUID:3352:1862456998103360:0@lo:38:kworker/u8:1.0 00000100:00100000:0.0:1776178594.670828:0:3352:0:(events.c:361:request_in_callback()) peer: 12345-0@lo (source: 12345-0@lo) 00000100:00100000:0.0:1776178594.670842:0:3352:0:(client.c:1733:ptlrpc_send_new_req()) Sending RPC req@00000000e964920a pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:lustre-MDT0000-lwp-MDT0001_UUID:3352:1862456998103552:0@lo:38:kworker/u8:1.0 00000100:00100000:0.0:1776178594.670882:0:14128:0:(service.c:2152:ptlrpc_server_handle_req_in()) got req x1862456998103360 00000100:00100000:0.0:1776178594.670900:0:14128:0:(nrs_fifo.c:179:nrs_fifo_req_get()) NRS start fifo request from 12345-0@lo, seq: 784 00000100:00100000:0.0:1776178594.670904:0:14128:0:(service.c:2324:ptlrpc_server_handle_request()) Handling RPC req@00000000f62fe06f pname:cluuid+ref:pid:xid:nid:opc:job mdt_out00_002:0+-99:3352:x1862456998103360:12345-0@lo:38:kworker/u8:1.0 00010000:02000000:0.0:1776178594.670916:0:14128:0:(ldlm_lib.c:1139:target_handle_connect()) lustre-MDT0000: Not available for connect from 0@lo (stopping) 00000100:00100000:0.0:1776178594.670943:0:14128:0:(service.c:2372:ptlrpc_server_handle_request()) Handled RPC req@00000000f62fe06f pname:cluuid+ref:pid:xid:nid:opc:job mdt_out00_002:0+-99:3352:x1862456998103360:12345-0@lo:38:kworker/u8:1.0 Request processed in 39us (115us total) trans 0 rc -19/-19 00000100:00100000:0.0:1776178594.670949:0:14128:0:(nrs_fifo.c:241:nrs_fifo_req_stop()) NRS stop fifo request from 12345-0@lo, seq: 784 00000100:00100000:2.0:1776178594.671031:0:3355:0:(ptlrpcd.c:415:ptlrpcd_check()) transfer 1 async RPCs [3->2] 00000100:00100000:2.0:1776178594.671037:0:3355:0:(client.c:1733:ptlrpc_send_new_req()) Sending RPC req@00000000ae814754 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_02:lustre-MDT0001-mdtlov_UUID:3355:1862456998103424:0@lo:400:kworker/u8:1.0 00000100:00100000:2.0:1776178594.671071:0:3355:0:(events.c:361:request_in_callback()) peer: 12345-0@lo (source: 12345-0@lo) 00000100:00100000:2.0:1776178594.671094:0:20759:0:(service.c:2152:ptlrpc_server_handle_req_in()) got req x1862456998103424 00000100:00100000:2.0:1776178594.671113:0:20759:0:(nrs_fifo.c:179:nrs_fifo_req_get()) NRS start fifo request from 12345-0@lo, seq: 818 00000100:00100000:2.0:1776178594.671116:0:20759:0:(service.c:2324:ptlrpc_server_handle_request()) Handling RPC req@00000000d23b79a7 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_010:lustre-MDT0001-mdtlov_UUID+5:3355:x1862456998103424:12345-0@lo:400:kworker/u8:1.0 00000100:00100000:1.0:1776178594.671568:0:3352:0:(events.c:361:request_in_callback()) peer: 12345-0@lo (source: 12345-0@lo) 00000100:00100000:2.0:1776178594.671582:0:20759:0:(service.c:2372:ptlrpc_server_handle_request()) Handled RPC req@00000000d23b79a7 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_010:lustre-MDT0001-mdtlov_UUID+5:3355:x1862456998103424:12345-0@lo:400:kworker/u8:1.0 Request processed in 460us (518us total) trans 0 rc 0/0 00000100:00100000:2.0:1776178594.671590:0:20759:0:(nrs_fifo.c:241:nrs_fifo_req_stop()) NRS stop fifo request from 12345-0@lo, seq: 818 00000100:00100000:2.0:1776178594.671622:0:3355:0:(client.c:2215:ptlrpc_check_set()) Completed RPC req@00000000ae814754 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_02:lustre-MDT0001-mdtlov_UUID:3355:1862456998103424:0@lo:400:kworker/u8:1.0 00000100:00100000:2.0:1776178594.671648:0:20140:0:(service.c:2152:ptlrpc_server_handle_req_in()) got req x1862456998103040 00000100:00100000:2.0:1776178594.671671:0:20140:0:(nrs_fifo.c:179:nrs_fifo_req_get()) NRS start fifo request from 12345-0@lo, seq: 308 00000100:00100000:2.0:1776178594.671675:0:20140:0:(service.c:2324:ptlrpc_server_handle_request()) Handling RPC req@000000008f0263f6 pname:cluuid+ref:pid:xid:nid:opc:job ll_mgs_0002:ac61f839-0e8e-4cac-9c59-08bcfbbad330+18:3353:x1862456998103040:12345-0@lo:400:kworker/u8:1.0 00000100:00100000:2.0:1776178594.671719:0:20140:0:(service.c:2372:ptlrpc_server_handle_request()) Handled RPC req@000000008f0263f6 pname:cluuid+ref:pid:xid:nid:opc:job ll_mgs_0002:ac61f839-0e8e-4cac-9c59-08bcfbbad330+18:3353:x1862456998103040:12345-0@lo:400:kworker/u8:1.0 Request processed in 43us (4599us total) trans 0 rc 0/0 00000100:00100000:2.0:1776178594.671726:0:20140:0:(nrs_fifo.c:241:nrs_fifo_req_stop()) NRS stop fifo request from 12345-0@lo, seq: 308 00000100:00100000:2.0:1776178594.671758:0:3353:0:(client.c:2215:ptlrpc_check_set()) Completed RPC req@00000000fa20ba2c pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_00:ac61f839-0e8e-4cac-9c59-08bcfbbad330:3353:1862456998103040:0@lo:400:kworker/u8:1.0 00000100:00080000:1.0:1776178594.675542:0:3352:0:(import.c:86:import_set_state_nolock()) 00000000ba4f1821 lustre-MDT0000_UUID: changing import state from CONNECTING to DISCONN 00000100:00080000:1.0:1776178594.675554:0:3352:0:(import.c:1429:ptlrpc_connect_interpret()) recovery of lustre-MDT0000_UUID on 192.168.204.103@tcp failed (-19) 00000100:00100000:1.0:1776178594.675561:0:3352:0:(client.c:2215:ptlrpc_check_set()) Completed RPC req@000000006c0c7de5 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:lustre-MDT0001-mdtlov_UUID:3352:1862456998103360:0@lo:38:kworker/u8:1.0 00000100:00100000:0.0:1776178594.675703:0:6257:0:(service.c:2152:ptlrpc_server_handle_req_in()) got req x1862456998103104 00000100:00100000:0.0:1776178594.675742:0:6257:0:(service.c:2152:ptlrpc_server_handle_req_in()) got req x1862456998103232 00000100:00100000:0.0:1776178594.675751:0:6257:0:(service.c:2152:ptlrpc_server_handle_req_in()) got req x1862456998103296 00000100:00100000:0.0:1776178594.675768:0:6257:0:(service.c:2152:ptlrpc_server_handle_req_in()) got req x1862456998103552 00000100:00100000:0.0:1776178594.675780:0:6257:0:(nrs_fifo.c:179:nrs_fifo_req_get()) NRS start fifo request from 12345-0@lo, seq: 1157 00000100:00100000:0.0:1776178594.675786:0:6257:0:(service.c:2324:ptlrpc_server_handle_request()) Handling RPC req@000000006b092368 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_002:lustre-MDT0001-lwp-OST0001_UUID+5:3355:x1862456998103296:12345-0@lo:400:kworker/u8:1.0 00000100:00100000:0.0:1776178594.675970:0:6257:0:(service.c:2372:ptlrpc_server_handle_request()) Handled RPC req@000000006b092368 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_002:lustre-MDT0001-lwp-OST0001_UUID+5:3355:x1862456998103296:12345-0@lo:400:kworker/u8:1.0 Request processed in 183us (8579us total) trans 0 rc 0/0 00000100:00100000:0.0:1776178594.675978:0:6257:0:(nrs_fifo.c:241:nrs_fifo_req_stop()) NRS stop fifo request from 12345-0@lo, seq: 1157 00000100:00100000:0.0:1776178594.676011:0:6257:0:(nrs_fifo.c:179:nrs_fifo_req_get()) NRS start fifo request from 12345-0@lo, seq: 6929 00000100:00100000:0.0:1776178594.676015:0:6257:0:(service.c:2324:ptlrpc_server_handle_request()) Handling RPC req@00000000b6e1d5c6 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_002:0+-99:3352:x1862456998103104:12345-0@lo:38:kworker/u8:1.0 00010000:02000000:0.0:1776178594.676031:0:6257:0:(ldlm_lib.c:1139:target_handle_connect()) lustre-MDT0000: Not available for connect from 0@lo (stopping) 00000100:00100000:0.0:1776178594.676078:0:6257:0:(service.c:2372:ptlrpc_server_handle_request()) Handled RPC req@00000000b6e1d5c6 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_002:0+-99:3352:x1862456998103104:12345-0@lo:38:kworker/u8:1.0 Request processed in 62us (8810us total) trans 0 rc -19/-19 00000100:00100000:0.0:1776178594.676083:0:6257:0:(nrs_fifo.c:241:nrs_fifo_req_stop()) NRS stop fifo request from 12345-0@lo, seq: 6929 00000100:00100000:0.0:1776178594.676094:0:6257:0:(nrs_fifo.c:179:nrs_fifo_req_get()) NRS start fifo request from 12345-0@lo, seq: 6930 00000100:00100000:0.0:1776178594.676097:0:6257:0:(service.c:2324:ptlrpc_server_handle_request()) Handling RPC req@0000000065f62d8c pname:cluuid+ref:pid:xid:nid:opc:job mdt00_002:0+-99:3352:x1862456998103232:12345-0@lo:38:kworker/u8:1.0 00010000:02000000:0.0:1776178594.676102:0:6257:0:(ldlm_lib.c:1139:target_handle_connect()) lustre-MDT0000: Not available for connect from 0@lo (stopping) 00000100:00100000:0.0:1776178594.676121:0:6257:0:(service.c:2372:ptlrpc_server_handle_request()) Handled RPC req@0000000065f62d8c pname:cluuid+ref:pid:xid:nid:opc:job mdt00_002:0+-99:3352:x1862456998103232:12345-0@lo:38:kworker/u8:1.0 Request processed in 24us (8798us total) trans 0 rc -19/-19 00000100:00100000:0.0:1776178594.676127:0:6257:0:(nrs_fifo.c:241:nrs_fifo_req_stop()) NRS stop fifo request from 12345-0@lo, seq: 6930 00000100:00100000:0.0:1776178594.676135:0:6257:0:(nrs_fifo.c:179:nrs_fifo_req_get()) NRS start fifo request from 12345-0@lo, seq: 6931 00000100:00100000:0.0:1776178594.676138:0:6257:0:(service.c:2324:ptlrpc_server_handle_request()) Handling RPC req@0000000075284cd7 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_002:0+-99:3352:x1862456998103552:12345-0@lo:38:kworker/u8:1.0 00010000:02000000:0.0:1776178594.676329:0:6257:0:(ldlm_lib.c:1139:target_handle_connect()) lustre-MDT0000: Not available for connect from 0@lo (stopping) 00000100:00100000:0.0:1776178594.676368:0:6257:0:(service.c:2372:ptlrpc_server_handle_request()) Handled RPC req@0000000075284cd7 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_002:0+-99:3352:x1862456998103552:12345-0@lo:38:kworker/u8:1.0 Request processed in 229us (4802us total) trans 0 rc -19/-19 00000100:00100000:0.0:1776178594.676374:0:6257:0:(nrs_fifo.c:241:nrs_fifo_req_stop()) NRS stop fifo request from 12345-0@lo, seq: 6931 00000100:00100000:0.0:1776178594.676421:0:3354:0:(events.c:361:request_in_callback()) peer: 12345-0@lo (source: 12345-0@lo) 00000100:00100000:0.0:1776178594.676450:0:6257:0:(service.c:2152:ptlrpc_server_handle_req_in()) got req x1862456998103168 00000100:00100000:0.0:1776178594.676464:0:6257:0:(nrs_fifo.c:179:nrs_fifo_req_get()) NRS start fifo request from 12345-0@lo, seq: 1158 00000100:00100000:0.0:1776178594.676468:0:6257:0:(service.c:2324:ptlrpc_server_handle_request()) Handling RPC req@00000000dd52d0d0 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_002:lustre-MDT0001-lwp-OST0000_UUID+5:3354:x1862456998103168:12345-0@lo:400:kworker/u8:1.0 00000100:00100000:0.0:1776178594.676495:0:6257:0:(service.c:2372:ptlrpc_server_handle_request()) Handled RPC req@00000000dd52d0d0 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_002:lustre-MDT0001-lwp-OST0000_UUID+5:3354:x1862456998103168:12345-0@lo:400:kworker/u8:1.0 Request processed in 27us (75us total) trans 0 rc 0/0 00000100:00100000:0.0:1776178594.676500:0:6257:0:(nrs_fifo.c:241:nrs_fifo_req_stop()) NRS stop fifo request from 12345-0@lo, seq: 1158 00000100:00100000:0.0:1776178594.676524:0:3354:0:(client.c:2215:ptlrpc_check_set()) Completed RPC req@000000003777ab0e pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_01:lustre-MDT0001-lwp-OST0000_UUID:3354:1862456998103168:0@lo:400:kworker/u8:1.0 00000100:00080000:0.0:1776178594.676563:0:3352:0:(import.c:86:import_set_state_nolock()) 00000000a7c55b1f lustre-MDT0000_UUID: changing import state from CONNECTING to DISCONN 00000100:00080000:0.0:1776178594.676567:0:3352:0:(import.c:1429:ptlrpc_connect_interpret()) recovery of lustre-MDT0000_UUID on 192.168.204.103@tcp failed (-19) 00000100:00100000:0.0:1776178594.676571:0:3352:0:(client.c:2215:ptlrpc_check_set()) Completed RPC req@00000000e964920a pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:lustre-MDT0000-lwp-MDT0001_UUID:3352:1862456998103552:0@lo:38:kworker/u8:1.0 00000100:00080000:0.0:1776178594.676580:0:3352:0:(import.c:86:import_set_state_nolock()) 0000000047601a98 lustre-MDT0000_UUID: changing import state from CONNECTING to DISCONN 00000100:00080000:0.0:1776178594.676582:0:3352:0:(import.c:1429:ptlrpc_connect_interpret()) recovery of lustre-MDT0000_UUID on 192.168.204.103@tcp failed (-19) 00000100:00100000:0.0:1776178594.676585:0:3352:0:(client.c:2215:ptlrpc_check_set()) Completed RPC req@00000000238121c2 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:lustre-MDT0000-lwp-OST0000_UUID:3352:1862456998103104:0@lo:38:kworker/u8:1.0 00000100:00080000:0.0:1776178594.676593:0:3352:0:(import.c:86:import_set_state_nolock()) 00000000dfc8db13 lustre-MDT0000_UUID: changing import state from CONNECTING to DISCONN 00000100:00080000:0.0:1776178594.676596:0:3352:0:(import.c:1429:ptlrpc_connect_interpret()) recovery of lustre-MDT0000_UUID on 192.168.204.103@tcp failed (-19) 00000100:00100000:0.0:1776178594.676598:0:3352:0:(client.c:2215:ptlrpc_check_set()) Completed RPC req@00000000907210d1 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:lustre-MDT0000-lwp-OST0001_UUID:3352:1862456998103232:0@lo:38:kworker/u8:1.0 00000100:00100000:3.0:1776178594.677640:0:3355:0:(client.c:2215:ptlrpc_check_set()) Completed RPC req@00000000b1db1835 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_02:lustre-MDT0001-lwp-OST0001_UUID:3355:1862456998103296:0@lo:400:kworker/u8:1.0 00010000:00010000:1.0:1776178595.296442:0:6245:0:(ldlm_lockd.c:2792:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:0.0:1776178596.194125:0:3354:0:(client.c:1733:ptlrpc_send_new_req()) Sending RPC req@00000000a942a398 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_01:lustre-MDT0001-mdtlov_UUID:3354:1862456998103616:0@lo:13:osp-pre-1-1.0 00000100:00100000:0.0:1776178596.194221:0:3354:0:(events.c:361:request_in_callback()) peer: 12345-0@lo (source: 12345-0@lo) 00000100:00100000:1.0:1776178596.194352:0:8231:0:(service.c:2152:ptlrpc_server_handle_req_in()) got req x1862456998103616 00000100:00100000:1.0:1776178596.194418:0:8231:0:(nrs_fifo.c:179:nrs_fifo_req_get()) NRS start fifo request from 12345-0@lo, seq: 641 00000100:00100000:1.0:1776178596.194424:0:8231:0:(service.c:2324:ptlrpc_server_handle_request()) Handling RPC req@000000005a0e2b58 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_create00:lustre-MDT0001-mdtlov_UUID+5:3354:x1862456998103616:12345-0@lo:13:osp-pre-1-1.0 00002000:00000024:1.0:1776178596.194458:0:8231:0:(ofd_obd.c:740:ofd_statfs()) blocks cached 229376 granted 491746368 pending 0 free 3923689472 avail 3692167168 00000100:00100000:1.0:1776178596.194515:0:8231:0:(service.c:2372:ptlrpc_server_handle_request()) Handled RPC req@000000005a0e2b58 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_create00:lustre-MDT0001-mdtlov_UUID+5:3354:x1862456998103616:12345-0@lo:13:osp-pre-1-1.0 Request processed in 90us (295us total) trans 0 rc 0/0 00000100:00100000:1.0:1776178596.194530:0:8231:0:(nrs_fifo.c:241:nrs_fifo_req_stop()) NRS stop fifo request from 12345-0@lo, seq: 641 00000100:00100000:3.0:1776178596.194610:0:3354:0:(client.c:2215:ptlrpc_check_set()) Completed RPC req@00000000a942a398 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_01:lustre-MDT0001-mdtlov_UUID:3354:1862456998103616:0@lo:13:osp-pre-1-1.0 00010000:00010000:1.0:1776178596.321574:0:9224:0:(ldlm_lockd.c:2792:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:3.0:1776178596.716137:0:3355:0:(client.c:1733:ptlrpc_send_new_req()) Sending RPC req@000000005b8f544b pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_02:lustre-MDT0001-mdtlov_UUID:3355:1862456998103680:0@lo:13:osp-pre-0-1.0 00000100:00100000:3.0:1776178596.716235:0:3355:0:(events.c:361:request_in_callback()) peer: 12345-0@lo (source: 12345-0@lo) 00000100:00100000:2.0:1776178596.717129:0:9554:0:(service.c:2152:ptlrpc_server_handle_req_in()) got req x1862456998103680 00000100:00100000:2.0:1776178596.717625:0:9554:0:(nrs_fifo.c:179:nrs_fifo_req_get()) NRS start fifo request from 12345-0@lo, seq: 642 00000100:00100000:2.0:1776178596.717632:0:9554:0:(service.c:2324:ptlrpc_server_handle_request()) Handling RPC req@00000000a942a398 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_create00:lustre-MDT0001-mdtlov_UUID+5:3355:x1862456998103680:12345-0@lo:13:osp-pre-0-1.0 00002000:00000024:2.0:1776178596.717665:0:9554:0:(ofd_obd.c:740:ofd_statfs()) blocks cached 315392 granted 484883776 pending 0 free 3923689472 avail 3692167168 00000100:00100000:2.0:1776178596.718321:0:9554:0:(service.c:2372:ptlrpc_server_handle_request()) Handled RPC req@00000000a942a398 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_create00:lustre-MDT0001-mdtlov_UUID+5:3355:x1862456998103680:12345-0@lo:13:osp-pre-0-1.0 Request processed in 684us (2083us total) trans 0 rc 0/0 00000100:00100000:2.0:1776178596.718332:0:9554:0:(nrs_fifo.c:241:nrs_fifo_req_stop()) NRS stop fifo request from 12345-0@lo, seq: 642 00000100:00100000:3.0:1776178596.719080:0:3355:0:(client.c:2215:ptlrpc_check_set()) Completed RPC req@000000005b8f544b pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_02:lustre-MDT0001-mdtlov_UUID:3355:1862456998103680:0@lo:13:osp-pre-0-1.0 00010000:00010000:1.0:1776178597.345138:0:9225:0:(ldlm_lockd.c:2792:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:3.0:1776178598.185341:0:3347:0:(events.c:361:request_in_callback()) peer: 12345-192.168.204.3@tcp (source: 12345-192.168.204.3@tcp) 00000100:00100000:3.0:1776178598.185393:0:3347:0:(events.c:361:request_in_callback()) peer: 12345-192.168.204.3@tcp (source: 12345-192.168.204.3@tcp) 00000100:00100000:3.0:1776178598.185436:0:3347:0:(events.c:361:request_in_callback()) peer: 12345-192.168.204.3@tcp (source: 12345-192.168.204.3@tcp) 00000100:00100000:3.0:1776178598.185456:0:3347:0:(events.c:361:request_in_callback()) peer: 12345-192.168.204.3@tcp (source: 12345-192.168.204.3@tcp) 00000100:00100000:3.0:1776178598.185497:0:3347:0:(events.c:361:request_in_callback()) peer: 12345-192.168.204.3@tcp (source: 12345-192.168.204.3@tcp) 00000100:00100000:3.0:1776178598.185535:0:20759:0:(service.c:2152:ptlrpc_server_handle_req_in()) got req x1862456979133888 00000100:00100000:3.0:1776178598.185582:0:20759:0:(service.c:2152:ptlrpc_server_handle_req_in()) got req x1862456979133824 00000100:00100000:3.0:1776178598.185606:0:20759:0:(nrs_fifo.c:179:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.204.3@tcp, seq: 819 00000100:00100000:3.0:1776178598.185614:0:20759:0:(service.c:2324:ptlrpc_server_handle_request()) Handling RPC req@0000000025b3b8e0 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_010:b50963bd-8eba-4432-9a54-8888d00e3fa1+9:2270:x1862456979133888:12345-192.168.204.3@tcp:400:kworker/u8:3.0 00000100:00100000:2.0:1776178598.185822:0:20755:0:(nrs_fifo.c:179:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.204.3@tcp, seq: 820 00000100:00100000:2.0:1776178598.185828:0:20755:0:(service.c:2324:ptlrpc_server_handle_request()) Handling RPC req@000000008f94e9dd pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_006:b50963bd-8eba-4432-9a54-8888d00e3fa1+8:2270:x1862456979133824:12345-192.168.204.3@tcp:400:kworker/u8:3.0 00000100:00100000:3.0:1776178598.185997:0:20759:0:(service.c:2372:ptlrpc_server_handle_request()) Handled RPC req@0000000025b3b8e0 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_010:b50963bd-8eba-4432-9a54-8888d00e3fa1+9:2270:x1862456979133888:12345-192.168.204.3@tcp:400:kworker/u8:3.0 Request processed in 383us (604us total) trans 0 rc 0/0 00000100:00100000:0.0:1776178598.186000:0:20137:0:(service.c:2152:ptlrpc_server_handle_req_in()) got req x1862456979133632 00000100:00100000:3.0:1776178598.186008:0:20759:0:(nrs_fifo.c:241:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.204.3@tcp, seq: 819 00000100:00100000:0.0:1776178598.186041:0:20137:0:(nrs_fifo.c:179:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.204.3@tcp, seq: 309 00000100:00100000:0.0:1776178598.186047:0:20137:0:(service.c:2324:ptlrpc_server_handle_request()) Handling RPC req@000000000084a3e4 pname:cluuid+ref:pid:xid:nid:opc:job ll_mgs_0000:5b5c66b1-1a2f-4651-8548-0c75b9a38a94+9:2270:x1862456979133632:12345-192.168.204.3@tcp:400:kworker/u8:3.0 00000100:00100000:3.0:1776178598.186057:0:6255:0:(service.c:2152:ptlrpc_server_handle_req_in()) got req x1862456979133760 00000100:00100000:1.0:1776178598.186151:0:6256:0:(service.c:2152:ptlrpc_server_handle_req_in()) got req x1862456979133696 00000100:00100000:1.0:1776178598.186178:0:6256:0:(nrs_fifo.c:179:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.204.3@tcp, seq: 6932 00000100:00100000:1.0:1776178598.186182:0:6256:0:(service.c:2324:ptlrpc_server_handle_request()) Handling RPC req@0000000088512a81 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_001:0+-99:2269:x1862456979133696:12345-192.168.204.3@tcp:38:kworker/u8:3.0 00010000:02000000:1.0:1776178598.186205:0:6256:0:(ldlm_lib.c:1139:target_handle_connect()) lustre-MDT0000: Not available for connect from 192.168.204.3@tcp (stopping) 00000100:00100000:3.0:1776178598.186257:0:6255:0:(nrs_fifo.c:179:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.204.3@tcp, seq: 1159 00000100:00100000:3.0:1776178598.186261:0:6255:0:(service.c:2324:ptlrpc_server_handle_request()) Handling RPC req@0000000086665d0f pname:cluuid+ref:pid:xid:nid:opc:job mdt00_000:b50963bd-8eba-4432-9a54-8888d00e3fa1+6:2270:x1862456979133760:12345-192.168.204.3@tcp:400:kworker/u8:3.0 00000100:00100000:2.0:1776178598.191763:0:20755:0:(service.c:2372:ptlrpc_server_handle_request()) Handled RPC req@000000008f94e9dd pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_006:b50963bd-8eba-4432-9a54-8888d00e3fa1+8:2270:x1862456979133824:12345-192.168.204.3@tcp:400:kworker/u8:3.0 Request processed in 5925us (6297us total) trans 0 rc 0/0 00000100:00100000:2.0:1776178598.191782:0:20755:0:(nrs_fifo.c:241:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.204.3@tcp, seq: 820 00000100:00100000:0.0:1776178598.195092:0:20137:0:(service.c:2372:ptlrpc_server_handle_request()) Handled RPC req@000000000084a3e4 pname:cluuid+ref:pid:xid:nid:opc:job ll_mgs_0000:5b5c66b1-1a2f-4651-8548-0c75b9a38a94+9:2270:x1862456979133632:12345-192.168.204.3@tcp:400:kworker/u8:3.0 Request processed in 9035us (9646us total) trans 0 rc 0/0 00000100:00100000:0.0:1776178598.195107:0:20137:0:(nrs_fifo.c:241:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.204.3@tcp, seq: 309 00000100:00100000:1.0:1776178598.209393:0:6256:0:(service.c:2372:ptlrpc_server_handle_request()) Handled RPC req@0000000088512a81 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_001:0+-99:2269:x1862456979133696:12345-192.168.204.3@tcp:38:kworker/u8:3.0 Request processed in 23201us (23887us total) trans 0 rc -19/-19 00000100:00100000:1.0:1776178598.209412:0:6256:0:(nrs_fifo.c:241:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.204.3@tcp, seq: 6932 00000100:00100000:3.0:1776178598.210947:0:6255:0:(service.c:2372:ptlrpc_server_handle_request()) Handled RPC req@0000000086665d0f pname:cluuid+ref:pid:xid:nid:opc:job mdt00_000:b50963bd-8eba-4432-9a54-8888d00e3fa1+6:2270:x1862456979133760:12345-192.168.204.3@tcp:400:kworker/u8:3.0 Request processed in 24676us (25608us total) trans 0 rc 0/0 00000100:00100000:3.0:1776178598.210964:0:6255:0:(nrs_fifo.c:241:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.204.3@tcp, seq: 1159 00010000:00010000:2.0:1776178598.368827:0:6244:0:(ldlm_lockd.c:2792:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00010000:00010000:1.0:1776178599.392609:0:6245:0:(ldlm_lockd.c:2792:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:3.0:1776178599.776526:0:3355:0:(ptlrpcd.c:415:ptlrpcd_check()) transfer 1 async RPCs [3->2] 00000100:00080000:1.0:1776178599.778483:0:36:0:(pinger.c:251:ptlrpc_pinger_process_import()) lustre-MDT0000-lwp-OST0000_UUID->lustre-MDT0000_UUID: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00080000:1.0:1776178599.778494:0:36:0:(recover.c:57:ptlrpc_initiate_recovery()) lustre-MDT0000_UUID: starting recovery 00000100:00080000:1.0:1776178599.778498:0:36:0:(import.c:86:import_set_state_nolock()) 0000000047601a98 lustre-MDT0000_UUID: changing import state from DISCONN to CONNECTING 00000100:00080000:1.0:1776178599.778510:0:36:0:(import.c:534:import_select_connection()) lustre-MDT0000-lwp-OST0000: connect to NID 0@lo last attempt 1301 00000100:00080000:1.0:1776178599.778515:0:36:0:(import.c:581:import_select_connection()) lustre-MDT0000-lwp-OST0000: tried all connections, increasing latency to 21s 00000100:00080000:1.0:1776178599.778523:0:36:0:(import.c:615:import_select_connection()) lustre-MDT0000-lwp-OST0000: import 0000000047601a98 using connection 192.168.204.103@tcp/0@lo 00000100:00100000:1.0:1776178599.778565:0:36:0:(import.c:807:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5) req@0000000008dc7ab9 x1862456998103808/t0(0) o38->lustre-MDT0000-lwp-OST0000@0@lo:12/10 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' 00000100:00080000:1.0:1776178599.778614:0:36:0:(pinger.c:251:ptlrpc_pinger_process_import()) lustre-MDT0000-lwp-OST0001_UUID->lustre-MDT0000_UUID: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00080000:1.0:1776178599.778617:0:36:0:(recover.c:57:ptlrpc_initiate_recovery()) lustre-MDT0000_UUID: starting recovery 00000100:00080000:1.0:1776178599.778619:0:36:0:(import.c:86:import_set_state_nolock()) 00000000dfc8db13 lustre-MDT0000_UUID: changing import state from DISCONN to CONNECTING 00000100:00080000:1.0:1776178599.778622:0:36:0:(import.c:534:import_select_connection()) lustre-MDT0000-lwp-OST0001: connect to NID 0@lo last attempt 1301 00000100:00080000:1.0:1776178599.778624:0:36:0:(import.c:581:import_select_connection()) lustre-MDT0000-lwp-OST0001: tried all connections, increasing latency to 21s 00000100:00080000:1.0:1776178599.778629:0:36:0:(import.c:615:import_select_connection()) lustre-MDT0000-lwp-OST0001: import 00000000dfc8db13 using connection 192.168.204.103@tcp/0@lo 00000100:00100000:1.0:1776178599.778640:0:36:0:(import.c:807:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5) req@000000006159eedd x1862456998103936/t0(0) o38->lustre-MDT0000-lwp-OST0001@0@lo:12/10 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' 00000100:00080000:1.0:1776178599.778655:0:36:0:(pinger.c:251:ptlrpc_pinger_process_import()) lustre-MDT0001-mdtlov_UUID->lustre-MDT0000_UUID: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00080000:1.0:1776178599.778657:0:36:0:(recover.c:57:ptlrpc_initiate_recovery()) lustre-MDT0000_UUID: starting recovery 00000100:00080000:1.0:1776178599.778659:0:36:0:(import.c:86:import_set_state_nolock()) 00000000ba4f1821 lustre-MDT0000_UUID: changing import state from DISCONN to CONNECTING 00000100:00080000:1.0:1776178599.778662:0:36:0:(import.c:534:import_select_connection()) lustre-MDT0000-osp-MDT0001: connect to NID 0@lo last attempt 1301 00000100:00080000:1.0:1776178599.778664:0:36:0:(import.c:581:import_select_connection()) lustre-MDT0000-osp-MDT0001: tried all connections, increasing latency to 21s 00000100:00080000:1.0:1776178599.778667:0:36:0:(import.c:615:import_select_connection()) lustre-MDT0000-osp-MDT0001: import 00000000ba4f1821 using connection 192.168.204.103@tcp/0@lo 00000100:00100000:1.0:1776178599.778678:0:36:0:(import.c:807:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5) req@000000002c128264 x1862456998104064/t0(0) o38->lustre-MDT0000-osp-MDT0001@0@lo:24/4 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' 00000100:00100000:2.0:1776178599.778783:0:3353:0:(client.c:1733:ptlrpc_send_new_req()) Sending RPC req@00000000c598e4ab pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_00:lustre-MDT0001-lwp-OST0000_UUID:3353:1862456998103872:0@lo:400:kworker/u8:1.0 00000100:00100000:2.0:1776178599.778851:0:3353:0:(events.c:361:request_in_callback()) peer: 12345-0@lo (source: 12345-0@lo) 00000100:00100000:2.0:1776178599.778888:0:3353:0:(ptlrpcd.c:415:ptlrpcd_check()) transfer 1 async RPCs [1->0] 00000100:00100000:2.0:1776178599.778894:0:3353:0:(client.c:1733:ptlrpc_send_new_req()) Sending RPC req@000000007cfd1c43 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_00:lustre-MDT0001-lwp-OST0001_UUID:3353:1862456998104000:0@lo:400:kworker/u8:1.0 00000100:00100000:2.0:1776178599.778917:0:3353:0:(events.c:361:request_in_callback()) peer: 12345-0@lo (source: 12345-0@lo) 00000100:00100000:2.0:1776178599.778951:0:6255:0:(service.c:2152:ptlrpc_server_handle_req_in()) got req x1862456998103872 00000100:00100000:2.0:1776178599.778996:0:11281:0:(service.c:2152:ptlrpc_server_handle_req_in()) got req x1862456998104000 00000100:00100000:2.0:1776178599.779017:0:11281:0:(nrs_fifo.c:179:nrs_fifo_req_get()) NRS start fifo request from 12345-0@lo, seq: 1160 00000100:00100000:2.0:1776178599.779023:0:11281:0:(service.c:2324:ptlrpc_server_handle_request()) Handling RPC req@00000000c6b32ddb pname:cluuid+ref:pid:xid:nid:opc:job mdt00_004:lustre-MDT0001-lwp-OST0000_UUID+5:3353:x1862456998103872:12345-0@lo:400:kworker/u8:1.0 00000100:00100000:2.0:1776178599.779094:0:11281:0:(service.c:2372:ptlrpc_server_handle_request()) Handled RPC req@00000000c6b32ddb pname:cluuid+ref:pid:xid:nid:opc:job mdt00_004:lustre-MDT0001-lwp-OST0000_UUID+5:3353:x1862456998103872:12345-0@lo:400:kworker/u8:1.0 Request processed in 70us (244us total) trans 0 rc 0/0 00000100:00100000:2.0:1776178599.779101:0:11281:0:(nrs_fifo.c:241:nrs_fifo_req_stop()) NRS stop fifo request from 12345-0@lo, seq: 1160 00000100:00100000:2.0:1776178599.779119:0:11281:0:(nrs_fifo.c:179:nrs_fifo_req_get()) NRS start fifo request from 12345-0@lo, seq: 1161 00000100:00100000:2.0:1776178599.779122:0:11281:0:(service.c:2324:ptlrpc_server_handle_request()) Handling RPC req@000000009cee1481 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_004:lustre-MDT0001-lwp-OST0001_UUID+5:3353:x1862456998104000:12345-0@lo:400:kworker/u8:1.0 00000100:00100000:2.0:1776178599.779144:0:11281:0:(service.c:2372:ptlrpc_server_handle_request()) Handled RPC req@000000009cee1481 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_004:lustre-MDT0001-lwp-OST0001_UUID+5:3353:x1862456998104000:12345-0@lo:400:kworker/u8:1.0 Request processed in 22us (228us total) trans 0 rc 0/0 00000100:00100000:2.0:1776178599.779149:0:11281:0:(nrs_fifo.c:241:nrs_fifo_req_stop()) NRS stop fifo request from 12345-0@lo, seq: 1161 00000100:00100000:2.0:1776178599.779650:0:3353:0:(client.c:2215:ptlrpc_check_set()) Completed RPC req@000000007cfd1c43 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_00:lustre-MDT0001-lwp-OST0001_UUID:3353:1862456998104000:0@lo:400:kworker/u8:1.0 00000100:00100000:2.0:1776178599.779663:0:3353:0:(client.c:2215:ptlrpc_check_set()) Completed RPC req@00000000c598e4ab pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_00:lustre-MDT0001-lwp-OST0000_UUID:3353:1862456998103872:0@lo:400:kworker/u8:1.0 00000100:00100000:0.0:1776178599.780172:0:3352:0:(client.c:1733:ptlrpc_send_new_req()) Sending RPC req@0000000008dc7ab9 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:lustre-MDT0000-lwp-OST0000_UUID:3352:1862456998103808:0@lo:38:kworker/u8:1.0 00000100:00100000:0.0:1776178599.780227:0:3352:0:(events.c:361:request_in_callback()) peer: 12345-0@lo (source: 12345-0@lo) 00000100:00100000:0.0:1776178599.780251:0:3352:0:(client.c:1733:ptlrpc_send_new_req()) Sending RPC req@000000006159eedd pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:lustre-MDT0000-lwp-OST0001_UUID:3352:1862456998103936:0@lo:38:kworker/u8:1.0 00000100:00100000:0.0:1776178599.780274:0:3352:0:(events.c:361:request_in_callback()) peer: 12345-0@lo (source: 12345-0@lo) 00000100:00100000:0.0:1776178599.780289:0:3352:0:(client.c:1733:ptlrpc_send_new_req()) Sending RPC req@000000002c128264 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:lustre-MDT0001-mdtlov_UUID:3352:1862456998104064:0@lo:38:kworker/u8:1.0 00000100:00100000:0.0:1776178599.780314:0:3352:0:(events.c:361:request_in_callback()) peer: 12345-0@lo (source: 12345-0@lo) 00000100:00100000:0.0:1776178599.780348:0:8250:0:(service.c:2152:ptlrpc_server_handle_req_in()) got req x1862456998103808 00000100:00100000:0.0:1776178599.780371:0:8250:0:(service.c:2152:ptlrpc_server_handle_req_in()) got req x1862456998103936 00000100:00100000:0.0:1776178599.780387:0:8250:0:(nrs_fifo.c:179:nrs_fifo_req_get()) NRS start fifo request from 12345-0@lo, seq: 6933 00000100:00100000:0.0:1776178599.780391:0:8250:0:(service.c:2324:ptlrpc_server_handle_request()) Handling RPC req@00000000c63df573 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_003:0+-99:3352:x1862456998103808:12345-0@lo:38:kworker/u8:1.0 00010000:02000000:0.0:1776178599.780405:0:8250:0:(ldlm_lib.c:1139:target_handle_connect()) lustre-MDT0000: Not available for connect from 0@lo (stopping) 00000100:00100000:0.0:1776178599.780441:0:8250:0:(service.c:2372:ptlrpc_server_handle_request()) Handled RPC req@00000000c63df573 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_003:0+-99:3352:x1862456998103808:12345-0@lo:38:kworker/u8:1.0 Request processed in 49us (214us total) trans 0 rc -19/-19 00000100:00100000:0.0:1776178599.780448:0:8250:0:(nrs_fifo.c:241:nrs_fifo_req_stop()) NRS stop fifo request from 12345-0@lo, seq: 6933 00000100:00100000:0.0:1776178599.780477:0:8250:0:(nrs_fifo.c:179:nrs_fifo_req_get()) NRS start fifo request from 12345-0@lo, seq: 6934 00000100:00100000:0.0:1776178599.780481:0:8250:0:(service.c:2324:ptlrpc_server_handle_request()) Handling RPC req@000000003a8c2e67 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_003:0+-99:3352:x1862456998103936:12345-0@lo:38:kworker/u8:1.0 00010000:02000000:0.0:1776178599.780487:0:8250:0:(ldlm_lib.c:1139:target_handle_connect()) lustre-MDT0000: Not available for connect from 0@lo (stopping) 00000100:00100000:0.0:1776178599.780511:0:8250:0:(service.c:2372:ptlrpc_server_handle_request()) Handled RPC req@000000003a8c2e67 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_003:0+-99:3352:x1862456998103936:12345-0@lo:38:kworker/u8:1.0 Request processed in 29us (237us total) trans 0 rc -19/-19 00000100:00100000:0.0:1776178599.780517:0:8250:0:(nrs_fifo.c:241:nrs_fifo_req_stop()) NRS stop fifo request from 12345-0@lo, seq: 6934 00000100:00100000:0.0:1776178599.780536:0:15213:0:(service.c:2152:ptlrpc_server_handle_req_in()) got req x1862456998104064 00000100:00100000:0.0:1776178599.780555:0:15213:0:(nrs_fifo.c:179:nrs_fifo_req_get()) NRS start fifo request from 12345-0@lo, seq: 785 00000100:00100000:0.0:1776178599.780559:0:15213:0:(service.c:2324:ptlrpc_server_handle_request()) Handling RPC req@00000000ea616553 pname:cluuid+ref:pid:xid:nid:opc:job mdt_out00_003:0+-99:3352:x1862456998104064:12345-0@lo:38:kworker/u8:1.0 00010000:02000000:0.0:1776178599.780566:0:15213:0:(ldlm_lib.c:1139:target_handle_connect()) lustre-MDT0000: Not available for connect from 0@lo (stopping) 00000100:00100000:0.0:1776178599.780603:0:15213:0:(service.c:2372:ptlrpc_server_handle_request()) Handled RPC req@00000000ea616553 pname:cluuid+ref:pid:xid:nid:opc:job mdt_out00_003:0+-99:3352:x1862456998104064:12345-0@lo:38:kworker/u8:1.0 Request processed in 43us (289us total) trans 0 rc -19/-19 00000100:00100000:0.0:1776178599.780609:0:15213:0:(nrs_fifo.c:241:nrs_fifo_req_stop()) NRS stop fifo request from 12345-0@lo, seq: 785 00000100:00080000:0.0:1776178599.780644:0:3352:0:(import.c:86:import_set_state_nolock()) 0000000047601a98 lustre-MDT0000_UUID: changing import state from CONNECTING to DISCONN 00000100:00080000:0.0:1776178599.780648:0:3352:0:(import.c:1429:ptlrpc_connect_interpret()) recovery of lustre-MDT0000_UUID on 192.168.204.103@tcp failed (-19) 00000100:00100000:0.0:1776178599.780652:0:3352:0:(client.c:2215:ptlrpc_check_set()) Completed RPC req@0000000008dc7ab9 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:lustre-MDT0000-lwp-OST0000_UUID:3352:1862456998103808:0@lo:38:kworker/u8:1.0 00000100:00080000:0.0:1776178599.780660:0:3352:0:(import.c:86:import_set_state_nolock()) 00000000dfc8db13 lustre-MDT0000_UUID: changing import state from CONNECTING to DISCONN 00000100:00080000:0.0:1776178599.780662:0:3352:0:(import.c:1429:ptlrpc_connect_interpret()) recovery of lustre-MDT0000_UUID on 192.168.204.103@tcp failed (-19) 00000100:00100000:0.0:1776178599.780664:0:3352:0:(client.c:2215:ptlrpc_check_set()) Completed RPC req@000000006159eedd pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:lustre-MDT0000-lwp-OST0001_UUID:3352:1862456998103936:0@lo:38:kworker/u8:1.0 00000100:00080000:0.0:1776178599.780671:0:3352:0:(import.c:86:import_set_state_nolock()) 00000000ba4f1821 lustre-MDT0000_UUID: changing import state from CONNECTING to DISCONN 00000100:00080000:0.0:1776178599.780674:0:3352:0:(import.c:1429:ptlrpc_connect_interpret()) recovery of lustre-MDT0000_UUID on 192.168.204.103@tcp failed (-19) 00000100:00100000:0.0:1776178599.780676:0:3352:0:(client.c:2215:ptlrpc_check_set()) Completed RPC req@000000002c128264 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:lustre-MDT0001-mdtlov_UUID:3352:1862456998104064:0@lo:38:kworker/u8:1.0 00000100:00100000:2.0:1776178599.782130:0:3355:0:(client.c:1733:ptlrpc_send_new_req()) Sending RPC req@00000000ce34953a pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_02:lustre-MDT0001-mdtlov_UUID:3355:1862456998104128:0@lo:400:kworker/u8:1.0 00000100:00100000:2.0:1776178599.782198:0:3355:0:(events.c:361:request_in_callback()) peer: 12345-0@lo (source: 12345-0@lo) 00000100:00100000:2.0:1776178599.782240:0:20755:0:(service.c:2152:ptlrpc_server_handle_req_in()) got req x1862456998104128 00000100:00100000:2.0:1776178599.782284:0:20755:0:(nrs_fifo.c:179:nrs_fifo_req_get()) NRS start fifo request from 12345-0@lo, seq: 821 00000100:00100000:2.0:1776178599.782289:0:20755:0:(service.c:2324:ptlrpc_server_handle_request()) Handling RPC req@00000000ec38a60b pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_006:lustre-MDT0001-mdtlov_UUID+5:3355:x1862456998104128:12345-0@lo:400:kworker/u8:1.0 00000100:00100000:2.0:1776178599.782339:0:20755:0:(service.c:2372:ptlrpc_server_handle_request()) Handled RPC req@00000000ec38a60b pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_006:lustre-MDT0001-mdtlov_UUID+5:3355:x1862456998104128:12345-0@lo:400:kworker/u8:1.0 Request processed in 50us (142us total) trans 0 rc 0/0 00000100:00100000:2.0:1776178599.782347:0:20755:0:(nrs_fifo.c:241:nrs_fifo_req_stop()) NRS stop fifo request from 12345-0@lo, seq: 821 00000100:00100000:2.0:1776178599.782369:0:3355:0:(client.c:1733:ptlrpc_send_new_req()) Sending RPC req@000000006c0c7de5 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_02:ac61f839-0e8e-4cac-9c59-08bcfbbad330:3355:1862456998103744:0@lo:400:kworker/u8:1.0 00000100:00100000:2.0:1776178599.782401:0:3355:0:(events.c:361:request_in_callback()) peer: 12345-0@lo (source: 12345-0@lo) 00000100:00100000:2.0:1776178599.782428:0:3355:0:(client.c:2215:ptlrpc_check_set()) Completed RPC req@00000000ce34953a pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_02:lustre-MDT0001-mdtlov_UUID:3355:1862456998104128:0@lo:400:kworker/u8:1.0 00000100:00100000:0.0:1776178599.783266:0:20137:0:(service.c:2152:ptlrpc_server_handle_req_in()) got req x1862456998103744 00000100:00100000:0.0:1776178599.783304:0:20137:0:(nrs_fifo.c:179:nrs_fifo_req_get()) NRS start fifo request from 12345-0@lo, seq: 310 00000100:00100000:0.0:1776178599.783308:0:20137:0:(service.c:2324:ptlrpc_server_handle_request()) Handling RPC req@00000000ff2b550f pname:cluuid+ref:pid:xid:nid:opc:job ll_mgs_0000:ac61f839-0e8e-4cac-9c59-08bcfbbad330+18:3355:x1862456998103744:12345-0@lo:400:kworker/u8:1.0 00000100:00100000:0.0:1776178599.783358:0:20137:0:(service.c:2372:ptlrpc_server_handle_request()) Handled RPC req@00000000ff2b550f pname:cluuid+ref:pid:xid:nid:opc:job ll_mgs_0000:ac61f839-0e8e-4cac-9c59-08bcfbbad330+18:3355:x1862456998103744:12345-0@lo:400:kworker/u8:1.0 Request processed in 49us (958us total) trans 0 rc 0/0 00000100:00100000:0.0:1776178599.783365:0:20137:0:(nrs_fifo.c:241:nrs_fifo_req_stop()) NRS stop fifo request from 12345-0@lo, seq: 310 00000100:00080000:1.0:1776178599.784183:0:36:0:(pinger.c:251:ptlrpc_pinger_process_import()) lustre-MDT0000-lwp-MDT0001_UUID->lustre-MDT0000_UUID: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00080000:1.0:1776178599.784189:0:36:0:(recover.c:57:ptlrpc_initiate_recovery()) lustre-MDT0000_UUID: starting recovery 00000100:00080000:1.0:1776178599.784192:0:36:0:(import.c:86:import_set_state_nolock()) 00000000a7c55b1f lustre-MDT0000_UUID: changing import state from DISCONN to CONNECTING 00000100:00080000:1.0:1776178599.784197:0:36:0:(import.c:534:import_select_connection()) lustre-MDT0000-lwp-MDT0001: connect to NID 0@lo last attempt 1301 00000100:00080000:1.0:1776178599.784199:0:36:0:(import.c:581:import_select_connection()) lustre-MDT0000-lwp-MDT0001: tried all connections, increasing latency to 21s 00000100:00080000:1.0:1776178599.784203:0:36:0:(import.c:615:import_select_connection()) lustre-MDT0000-lwp-MDT0001: import 00000000a7c55b1f using connection 192.168.204.103@tcp/0@lo 00000100:00100000:1.0:1776178599.784223:0:36:0:(import.c:807:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5) req@000000000eda5b77 x1862456998104256/t0(0) o38->lustre-MDT0000-lwp-MDT0001@0@lo:12/10 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' 00000100:00100000:3.0:1776178599.784371:0:3355:0:(client.c:2215:ptlrpc_check_set()) Completed RPC req@000000006c0c7de5 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_02:ac61f839-0e8e-4cac-9c59-08bcfbbad330:3355:1862456998103744:0@lo:400:kworker/u8:1.0 00000100:00100000:3.0:1776178599.784388:0:3355:0:(ptlrpcd.c:415:ptlrpcd_check()) transfer 1 async RPCs [3->2] 00000100:00100000:3.0:1776178599.784395:0:3355:0:(client.c:1733:ptlrpc_send_new_req()) Sending RPC req@00000000c989e78f pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_02:lustre-MDT0001-mdtlov_UUID:3355:1862456998104192:0@lo:400:kworker/u8:1.0 00000100:00100000:3.0:1776178599.784431:0:3355:0:(events.c:361:request_in_callback()) peer: 12345-0@lo (source: 12345-0@lo) 00000100:00100000:3.0:1776178599.784482:0:14440:0:(service.c:2152:ptlrpc_server_handle_req_in()) got req x1862456998104192 00000100:00100000:3.0:1776178599.784509:0:14440:0:(nrs_fifo.c:179:nrs_fifo_req_get()) NRS start fifo request from 12345-0@lo, seq: 822 00000100:00100000:3.0:1776178599.784513:0:14440:0:(service.c:2324:ptlrpc_server_handle_request()) Handling RPC req@00000000494227fb pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_004:lustre-MDT0001-mdtlov_UUID+5:3355:x1862456998104192:12345-0@lo:400:kworker/u8:1.0 00000100:00100000:3.0:1776178599.784547:0:14440:0:(service.c:2372:ptlrpc_server_handle_request()) Handled RPC req@00000000494227fb pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_004:lustre-MDT0001-mdtlov_UUID+5:3355:x1862456998104192:12345-0@lo:400:kworker/u8:1.0 Request processed in 33us (117us total) trans 0 rc 0/0 00000100:00100000:3.0:1776178599.784553:0:14440:0:(nrs_fifo.c:241:nrs_fifo_req_stop()) NRS stop fifo request from 12345-0@lo, seq: 822 00000100:00100000:3.0:1776178599.784775:0:3355:0:(client.c:2215:ptlrpc_check_set()) Completed RPC req@00000000c989e78f pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_02:lustre-MDT0001-mdtlov_UUID:3355:1862456998104192:0@lo:400:kworker/u8:1.0 00000100:00100000:0.0:1776178599.785143:0:3352:0:(client.c:1733:ptlrpc_send_new_req()) Sending RPC req@000000000eda5b77 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:lustre-MDT0000-lwp-MDT0001_UUID:3352:1862456998104256:0@lo:38:kworker/u8:1.0 00000100:00100000:0.0:1776178599.785190:0:3352:0:(events.c:361:request_in_callback()) peer: 12345-0@lo (source: 12345-0@lo) 00000100:00100000:3.0:1776178599.785761:0:6256:0:(service.c:2152:ptlrpc_server_handle_req_in()) got req x1862456998104256 00000100:00100000:3.0:1776178599.785795:0:6256:0:(nrs_fifo.c:179:nrs_fifo_req_get()) NRS start fifo request from 12345-0@lo, seq: 6935 00000100:00100000:3.0:1776178599.785799:0:6256:0:(service.c:2324:ptlrpc_server_handle_request()) Handling RPC req@00000000cd795392 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_001:0+-99:3352:x1862456998104256:12345-0@lo:38:kworker/u8:1.0 00010000:02000000:3.0:1776178599.785815:0:6256:0:(ldlm_lib.c:1139:target_handle_connect()) lustre-MDT0000: Not available for connect from 0@lo (stopping) 00000100:00100000:3.0:1776178599.785849:0:6256:0:(service.c:2372:ptlrpc_server_handle_request()) Handled RPC req@00000000cd795392 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_001:0+-99:3352:x1862456998104256:12345-0@lo:38:kworker/u8:1.0 Request processed in 49us (659us total) trans 0 rc -19/-19 00000100:00100000:3.0:1776178599.785855:0:6256:0:(nrs_fifo.c:241:nrs_fifo_req_stop()) NRS stop fifo request from 12345-0@lo, seq: 6935 00000100:00080000:0.0:1776178599.785927:0:3352:0:(import.c:86:import_set_state_nolock()) 00000000a7c55b1f lustre-MDT0000_UUID: changing import state from CONNECTING to DISCONN 00000100:00080000:0.0:1776178599.785931:0:3352:0:(import.c:1429:ptlrpc_connect_interpret()) recovery of lustre-MDT0000_UUID on 192.168.204.103@tcp failed (-19) 00000100:00100000:0.0:1776178599.785935:0:3352:0:(client.c:2215:ptlrpc_check_set()) Completed RPC req@000000000eda5b77 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:lustre-MDT0000-lwp-MDT0001_UUID:3352:1862456998104256:0@lo:38:kworker/u8:1.0 00010000:00010000:1.0:1776178600.417934:0:9224:0:(ldlm_lockd.c:2792:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:2.0:1776178601.323601:0:3353:0:(client.c:1733:ptlrpc_send_new_req()) Sending RPC req@0000000045cb5035 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_00:lustre-MDT0001-mdtlov_UUID:3353:1862456998104320:0@lo:13:osp-pre-1-1.0 00000100:00100000:2.0:1776178601.323687:0:3353:0:(events.c:361:request_in_callback()) peer: 12345-0@lo (source: 12345-0@lo) 00000100:00100000:1.0:1776178601.323832:0:9554:0:(service.c:2152:ptlrpc_server_handle_req_in()) got req x1862456998104320 00000100:00100000:1.0:1776178601.324206:0:9554:0:(nrs_fifo.c:179:nrs_fifo_req_get()) NRS start fifo request from 12345-0@lo, seq: 643 00000100:00100000:1.0:1776178601.324212:0:9554:0:(service.c:2324:ptlrpc_server_handle_request()) Handling RPC req@00000000bcb535b2 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_create00:lustre-MDT0001-mdtlov_UUID+5:3353:x1862456998104320:12345-0@lo:13:osp-pre-1-1.0 00002000:00000024:1.0:1776178601.324245:0:9554:0:(ofd_obd.c:740:ofd_statfs()) blocks cached 229376 granted 491746368 pending 0 free 3923689472 avail 3692167168 00000100:00100000:1.0:1776178601.324358:0:9554:0:(service.c:2372:ptlrpc_server_handle_request()) Handled RPC req@00000000bcb535b2 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_create00:lustre-MDT0001-mdtlov_UUID+5:3353:x1862456998104320:12345-0@lo:13:osp-pre-1-1.0 Request processed in 144us (672us total) trans 0 rc 0/0 00000100:00100000:1.0:1776178601.324370:0:9554:0:(nrs_fifo.c:241:nrs_fifo_req_stop()) NRS stop fifo request from 12345-0@lo, seq: 643 00000100:00100000:3.0:1776178601.324448:0:3353:0:(client.c:2215:ptlrpc_check_set()) Completed RPC req@0000000045cb5035 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_00:lustre-MDT0001-mdtlov_UUID:3353:1862456998104320:0@lo:13:osp-pre-1-1.0 00010000:00010000:1.0:1776178601.440327:0:9225:0:(ldlm_lockd.c:2792:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:2.0:1776178601.824559:0:3354:0:(client.c:1733:ptlrpc_send_new_req()) Sending RPC req@00000000bdb7858e pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_01:lustre-MDT0001-mdtlov_UUID:3354:1862456998104384:0@lo:13:osp-pre-0-1.0 00000100:00100000:2.0:1776178601.824642:0:3354:0:(events.c:361:request_in_callback()) peer: 12345-0@lo (source: 12345-0@lo) 00000100:00100000:1.0:1776178601.824751:0:9554:0:(service.c:2152:ptlrpc_server_handle_req_in()) got req x1862456998104384 00000100:00100000:1.0:1776178601.824801:0:9554:0:(nrs_fifo.c:179:nrs_fifo_req_get()) NRS start fifo request from 12345-0@lo, seq: 644 00000100:00100000:1.0:1776178601.824806:0:9554:0:(service.c:2324:ptlrpc_server_handle_request()) Handling RPC req@0000000049854387 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_create00:lustre-MDT0001-mdtlov_UUID+5:3354:x1862456998104384:12345-0@lo:13:osp-pre-0-1.0 00002000:00000024:1.0:1776178601.824833:0:9554:0:(ofd_obd.c:740:ofd_statfs()) blocks cached 315392 granted 484883776 pending 0 free 3923689472 avail 3692167168 00000100:00100000:1.0:1776178601.824884:0:9554:0:(service.c:2372:ptlrpc_server_handle_request()) Handled RPC req@0000000049854387 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_create00:lustre-MDT0001-mdtlov_UUID+5:3354:x1862456998104384:12345-0@lo:13:osp-pre-0-1.0 Request processed in 77us (243us total) trans 0 rc 0/0 00000100:00100000:1.0:1776178601.824891:0:9554:0:(nrs_fifo.c:241:nrs_fifo_req_stop()) NRS stop fifo request from 12345-0@lo, seq: 644 00000100:00100000:0.0:1776178601.825109:0:3354:0:(client.c:2215:ptlrpc_check_set()) Completed RPC req@00000000bdb7858e pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_01:lustre-MDT0001-mdtlov_UUID:3354:1862456998104384:0@lo:13:osp-pre-0-1.0 00010000:00010000:2.0:1776178602.464327:0:6244:0:(ldlm_lockd.c:2792:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:2.0:1776178603.302532:0:3348:0:(events.c:361:request_in_callback()) peer: 12345-192.168.204.3@tcp (source: 12345-192.168.204.3@tcp) 00000100:00100000:3.0:1776178603.306935:0:20140:0:(service.c:2152:ptlrpc_server_handle_req_in()) got req x1862456979133952 00000100:00100000:3.0:1776178603.307014:0:20140:0:(nrs_fifo.c:179:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.204.3@tcp, seq: 311 00000100:00100000:3.0:1776178603.307021:0:20140:0:(service.c:2324:ptlrpc_server_handle_request()) Handling RPC req@00000000fb48a5c8 pname:cluuid+ref:pid:xid:nid:opc:job ll_mgs_0002:5b5c66b1-1a2f-4651-8548-0c75b9a38a94+9:2272:x1862456979133952:12345-192.168.204.3@tcp:400:kworker/u8:4.0 00000100:00100000:3.0:1776178603.307171:0:20140:0:(service.c:2372:ptlrpc_server_handle_request()) Handled RPC req@00000000fb48a5c8 pname:cluuid+ref:pid:xid:nid:opc:job ll_mgs_0002:5b5c66b1-1a2f-4651-8548-0c75b9a38a94+9:2272:x1862456979133952:12345-192.168.204.3@tcp:400:kworker/u8:4.0 Request processed in 148us (4649us total) trans 0 rc 0/0 00000100:00100000:3.0:1776178603.307179:0:20140:0:(nrs_fifo.c:241:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.204.3@tcp, seq: 311 00000100:00100000:2.0:1776178603.359643:0:3346:0:(events.c:361:request_in_callback()) peer: 12345-192.168.204.3@tcp (source: 12345-192.168.204.3@tcp) 00000100:00100000:2.0:1776178603.359707:0:3346:0:(events.c:361:request_in_callback()) peer: 12345-192.168.204.3@tcp (source: 12345-192.168.204.3@tcp) 00000100:00100000:2.0:1776178603.359770:0:3346:0:(events.c:361:request_in_callback()) peer: 12345-192.168.204.3@tcp (source: 12345-192.168.204.3@tcp) 00000100:00100000:2.0:1776178603.359826:0:3346:0:(events.c:361:request_in_callback()) peer: 12345-192.168.204.3@tcp (source: 12345-192.168.204.3@tcp) 00000100:00100000:1.0:1776178603.360566:0:14440:0:(service.c:2152:ptlrpc_server_handle_req_in()) got req x1862456979134208 00000100:00100000:1.0:1776178603.360653:0:14440:0:(service.c:2152:ptlrpc_server_handle_req_in()) got req x1862456979134144 00000100:00100000:1.0:1776178603.360681:0:14440:0:(nrs_fifo.c:179:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.204.3@tcp, seq: 823 00000100:00100000:1.0:1776178603.360689:0:14440:0:(service.c:2324:ptlrpc_server_handle_request()) Handling RPC req@000000000f170d35 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_004:b50963bd-8eba-4432-9a54-8888d00e3fa1+9:2273:x1862456979134208:12345-192.168.204.3@tcp:400:kworker/u8:4.0 00000100:00100000:1.0:1776178603.360818:0:14440:0:(service.c:2372:ptlrpc_server_handle_request()) Handled RPC req@000000000f170d35 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_004:b50963bd-8eba-4432-9a54-8888d00e3fa1+9:2273:x1862456979134208:12345-192.168.204.3@tcp:400:kworker/u8:4.0 Request processed in 129us (1111us total) trans 0 rc 0/0 00000100:00100000:1.0:1776178603.360827:0:14440:0:(nrs_fifo.c:241:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.204.3@tcp, seq: 823 00000100:00100000:1.0:1776178603.360858:0:14440:0:(nrs_fifo.c:179:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.204.3@tcp, seq: 824 00000100:00100000:1.0:1776178603.360862:0:14440:0:(service.c:2324:ptlrpc_server_handle_request()) Handling RPC req@00000000c01dcf7c pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_004:b50963bd-8eba-4432-9a54-8888d00e3fa1+8:2273:x1862456979134144:12345-192.168.204.3@tcp:400:kworker/u8:4.0 00000100:00100000:1.0:1776178603.361305:0:14440:0:(service.c:2372:ptlrpc_server_handle_request()) Handled RPC req@00000000c01dcf7c pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_004:b50963bd-8eba-4432-9a54-8888d00e3fa1+8:2273:x1862456979134144:12345-192.168.204.3@tcp:400:kworker/u8:4.0 Request processed in 440us (1534us total) trans 0 rc 0/0 00000100:00100000:1.0:1776178603.361312:0:14440:0:(nrs_fifo.c:241:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.204.3@tcp, seq: 824 00000100:00100000:3.0:1776178603.361597:0:6256:0:(service.c:2152:ptlrpc_server_handle_req_in()) got req x1862456979134080 00000100:00100000:3.0:1776178603.361682:0:6256:0:(service.c:2152:ptlrpc_server_handle_req_in()) got req x1862456979134016 00000100:00100000:3.0:1776178603.361699:0:6256:0:(nrs_fifo.c:179:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.204.3@tcp, seq: 1162 00000100:00100000:3.0:1776178603.361704:0:6256:0:(service.c:2324:ptlrpc_server_handle_request()) Handling RPC req@00000000606ce17f pname:cluuid+ref:pid:xid:nid:opc:job mdt00_001:b50963bd-8eba-4432-9a54-8888d00e3fa1+6:2273:x1862456979134080:12345-192.168.204.3@tcp:400:kworker/u8:4.0 00000100:00100000:3.0:1776178603.362344:0:6256:0:(service.c:2372:ptlrpc_server_handle_request()) Handled RPC req@00000000606ce17f pname:cluuid+ref:pid:xid:nid:opc:job mdt00_001:b50963bd-8eba-4432-9a54-8888d00e3fa1+6:2273:x1862456979134080:12345-192.168.204.3@tcp:400:kworker/u8:4.0 Request processed in 635us (2713us total) trans 0 rc 0/0 00000100:00100000:3.0:1776178603.362355:0:6256:0:(nrs_fifo.c:241:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.204.3@tcp, seq: 1162 00000100:00100000:3.0:1776178603.362471:0:6256:0:(nrs_fifo.c:179:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.204.3@tcp, seq: 6936 00000100:00100000:3.0:1776178603.362476:0:6256:0:(service.c:2324:ptlrpc_server_handle_request()) Handling RPC req@0000000011d4114f pname:cluuid+ref:pid:xid:nid:opc:job mdt00_001:0+-99:2269:x1862456979134016:12345-192.168.204.3@tcp:38:kworker/u8:4.0 00010000:02000000:3.0:1776178603.362501:0:6256:0:(ldlm_lib.c:1139:target_handle_connect()) lustre-MDT0000: Not available for connect from 192.168.204.3@tcp (stopping) 00000100:00100000:3.0:1776178603.365531:0:6256:0:(service.c:2372:ptlrpc_server_handle_request()) Handled RPC req@0000000011d4114f pname:cluuid+ref:pid:xid:nid:opc:job mdt00_001:0+-99:2269:x1862456979134016:12345-192.168.204.3@tcp:38:kworker/u8:4.0 Request processed in 3041us (5692us total) trans 0 rc -19/-19 00000100:00100000:3.0:1776178603.365553:0:6256:0:(nrs_fifo.c:241:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.204.3@tcp, seq: 6936 00010000:00010000:1.0:1776178603.491026:0:6245:0:(ldlm_lockd.c:2792:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00010000:00010000:1.0:1776178604.512408:0:9224:0:(ldlm_lockd.c:2792:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00080000:0.0:1776178604.896410:0:22249:0:(pinger.c:251:ptlrpc_pinger_process_import()) lustre-MDT0000-lwp-OST0000_UUID->lustre-MDT0000_UUID: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00080000:0.0:1776178604.896419:0:22249:0:(recover.c:57:ptlrpc_initiate_recovery()) lustre-MDT0000_UUID: starting recovery 00000100:00080000:0.0:1776178604.896423:0:22249:0:(import.c:86:import_set_state_nolock()) 0000000047601a98 lustre-MDT0000_UUID: changing import state from DISCONN to CONNECTING 00000100:00080000:0.0:1776178604.896433:0:22249:0:(import.c:534:import_select_connection()) lustre-MDT0000-lwp-OST0000: connect to NID 0@lo last attempt 1307 00000100:00080000:0.0:1776178604.896437:0:22249:0:(import.c:581:import_select_connection()) lustre-MDT0000-lwp-OST0000: tried all connections, increasing latency to 21s 00000100:00080000:0.0:1776178604.896444:0:22249:0:(import.c:615:import_select_connection()) lustre-MDT0000-lwp-OST0000: import 0000000047601a98 using connection 192.168.204.103@tcp/0@lo 00000100:00100000:0.0:1776178604.896468:0:22249:0:(import.c:807:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5) req@00000000f247d7cd x1862456998104512/t0(0) o38->lustre-MDT0000-lwp-OST0000@0@lo:12/10 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' 00000100:00100000:3.0:1776178604.896610:0:3355:0:(client.c:1733:ptlrpc_send_new_req()) Sending RPC req@00000000bdb7858e pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_02:ac61f839-0e8e-4cac-9c59-08bcfbbad330:3355:1862456998104448:0@lo:400:kworker/u8:0.0 00000100:00100000:2.0:1776178604.896671:0:3356:0:(client.c:1733:ptlrpc_send_new_req()) Sending RPC req@0000000086df2fc5 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_03:lustre-MDT0001-lwp-OST0000_UUID:3356:1862456998104576:0@lo:400:kworker/u8:0.0 00000100:00100000:1.0:1776178604.896724:0:3352:0:(client.c:1733:ptlrpc_send_new_req()) Sending RPC req@00000000f247d7cd pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:lustre-MDT0000-lwp-OST0000_UUID:3352:1862456998104512:0@lo:38:kworker/u8:0.0 00000100:00080000:0.0:1776178604.897625:0:22249:0:(pinger.c:251:ptlrpc_pinger_process_import()) lustre-MDT0000-lwp-OST0001_UUID->lustre-MDT0000_UUID: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00080000:0.0:1776178604.897633:0:22249:0:(recover.c:57:ptlrpc_initiate_recovery()) lustre-MDT0000_UUID: starting recovery 00000100:00080000:0.0:1776178604.897636:0:22249:0:(import.c:86:import_set_state_nolock()) 00000000dfc8db13 lustre-MDT0000_UUID: changing import state from DISCONN to CONNECTING 00000100:00080000:0.0:1776178604.897642:0:22249:0:(import.c:534:import_select_connection()) lustre-MDT0000-lwp-OST0001: connect to NID 0@lo last attempt 1307 00000100:00080000:0.0:1776178604.897644:0:22249:0:(import.c:581:import_select_connection()) lustre-MDT0000-lwp-OST0001: tried all connections, increasing latency to 21s 00000100:00080000:0.0:1776178604.897649:0:22249:0:(import.c:615:import_select_connection()) lustre-MDT0000-lwp-OST0001: import 00000000dfc8db13 using connection 192.168.204.103@tcp/0@lo 00000100:00100000:0.0:1776178604.897676:0:22249:0:(import.c:807:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5) req@00000000dd5bd4cd x1862456998104640/t0(0) o38->lustre-MDT0000-lwp-OST0001@0@lo:12/10 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' 00000100:00080000:0.0:1776178604.897709:0:22249:0:(pinger.c:251:ptlrpc_pinger_process_import()) lustre-MDT0001-mdtlov_UUID->lustre-MDT0000_UUID: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00080000:0.0:1776178604.897713:0:22249:0:(recover.c:57:ptlrpc_initiate_recovery()) lustre-MDT0000_UUID: starting recovery 00000100:00080000:0.0:1776178604.897714:0:22249:0:(import.c:86:import_set_state_nolock()) 00000000ba4f1821 lustre-MDT0000_UUID: changing import state from DISCONN to CONNECTING 00000100:00080000:0.0:1776178604.897717:0:22249:0:(import.c:534:import_select_connection()) lustre-MDT0000-osp-MDT0001: connect to NID 0@lo last attempt 1307 00000100:00080000:0.0:1776178604.897719:0:22249:0:(import.c:581:import_select_connection()) lustre-MDT0000-osp-MDT0001: tried all connections, increasing latency to 21s 00000100:00080000:0.0:1776178604.897722:0:22249:0:(import.c:615:import_select_connection()) lustre-MDT0000-osp-MDT0001: import 00000000ba4f1821 using connection 192.168.204.103@tcp/0@lo 00000100:00100000:0.0:1776178604.897731:0:22249:0:(import.c:807:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5) req@00000000950bb9b2 x1862456998104768/t0(0) o38->lustre-MDT0000-osp-MDT0001@0@lo:24/4 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' 00000100:00080000:0.0:1776178604.897749:0:22249:0:(pinger.c:251:ptlrpc_pinger_process_import()) lustre-MDT0000-lwp-MDT0001_UUID->lustre-MDT0000_UUID: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00080000:0.0:1776178604.897751:0:22249:0:(recover.c:57:ptlrpc_initiate_recovery()) lustre-MDT0000_UUID: starting recovery 00000100:00080000:0.0:1776178604.897752:0:22249:0:(import.c:86:import_set_state_nolock()) 00000000a7c55b1f lustre-MDT0000_UUID: changing import state from DISCONN to CONNECTING 00000100:00080000:0.0:1776178604.897755:0:22249:0:(import.c:534:import_select_connection()) lustre-MDT0000-lwp-MDT0001: connect to NID 0@lo last attempt 1307 00000100:00080000:0.0:1776178604.897757:0:22249:0:(import.c:581:import_select_connection()) lustre-MDT0000-lwp-MDT0001: tried all connections, increasing latency to 21s 00000100:00080000:0.0:1776178604.897759:0:22249:0:(import.c:615:import_select_connection()) lustre-MDT0000-lwp-MDT0001: import 00000000a7c55b1f using connection 192.168.204.103@tcp/0@lo 00000100:00100000:0.0:1776178604.897766:0:22249:0:(import.c:807:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5) req@00000000fccd82c9 x1862456998104960/t0(0) o38->lustre-MDT0000-lwp-MDT0001@0@lo:12/10 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' 00000100:00100000:0.0:1776178604.897801:0:3354:0:(client.c:1733:ptlrpc_send_new_req()) Sending RPC req@00000000d1597b46 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_01:lustre-MDT0001-mdtlov_UUID:3354:1862456998104832:0@lo:400:kworker/u8:0.0 00000100:00100000:2.0:1776178604.907571:0:3356:0:(events.c:361:request_in_callback()) peer: 12345-0@lo (source: 12345-0@lo) 00000100:00100000:2.0:1776178604.907642:0:3356:0:(ptlrpcd.c:415:ptlrpcd_check()) transfer 1 async RPCs [2->3] 00000100:00100000:2.0:1776178604.907666:0:3356:0:(client.c:1733:ptlrpc_send_new_req()) Sending RPC req@00000000f6259053 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_03:lustre-MDT0001-mdtlov_UUID:3356:1862456998104896:0@lo:400:kworker/u8:0.0 00000100:00100000:2.0:1776178604.907705:0:3356:0:(events.c:361:request_in_callback()) peer: 12345-0@lo (source: 12345-0@lo) 00000100:00100000:2.0:1776178604.907757:0:20759:0:(service.c:2152:ptlrpc_server_handle_req_in()) got req x1862456998104896 00000100:00100000:2.0:1776178604.908596:0:20759:0:(nrs_fifo.c:179:nrs_fifo_req_get()) NRS start fifo request from 12345-0@lo, seq: 825 00000100:00100000:2.0:1776178604.908605:0:20759:0:(service.c:2324:ptlrpc_server_handle_request()) Handling RPC req@00000000e2bf2394 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_010:lustre-MDT0001-mdtlov_UUID+5:3356:x1862456998104896:12345-0@lo:400:kworker/u8:0.0 00000100:00100000:2.0:1776178604.908752:0:20759:0:(service.c:2372:ptlrpc_server_handle_request()) Handled RPC req@00000000e2bf2394 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_010:lustre-MDT0001-mdtlov_UUID+5:3356:x1862456998104896:12345-0@lo:400:kworker/u8:0.0 Request processed in 147us (1046us total) trans 0 rc 0/0 00000100:00100000:2.0:1776178604.908768:0:20759:0:(nrs_fifo.c:241:nrs_fifo_req_stop()) NRS stop fifo request from 12345-0@lo, seq: 825 00000100:00100000:2.0:1776178604.908901:0:3356:0:(client.c:2215:ptlrpc_check_set()) Completed RPC req@00000000f6259053 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_03:lustre-MDT0001-mdtlov_UUID:3356:1862456998104896:0@lo:400:kworker/u8:0.0 00000100:00100000:2.0:1776178604.908933:0:6256:0:(service.c:2152:ptlrpc_server_handle_req_in()) got req x1862456998104576 00000100:00100000:2.0:1776178604.908964:0:6256:0:(nrs_fifo.c:179:nrs_fifo_req_get()) NRS start fifo request from 12345-0@lo, seq: 1163 00000100:00100000:2.0:1776178604.908967:0:6256:0:(service.c:2324:ptlrpc_server_handle_request()) Handling RPC req@000000005831fa36 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_001:lustre-MDT0001-lwp-OST0000_UUID+5:3356:x1862456998104576:12345-0@lo:400:kworker/u8:0.0 00000100:00100000:2.0:1776178604.909001:0:6256:0:(service.c:2372:ptlrpc_server_handle_request()) Handled RPC req@000000005831fa36 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_001:lustre-MDT0001-lwp-OST0000_UUID+5:3356:x1862456998104576:12345-0@lo:400:kworker/u8:0.0 Request processed in 33us (1441us total) trans 0 rc 0/0 00000100:00100000:2.0:1776178604.909006:0:6256:0:(nrs_fifo.c:241:nrs_fifo_req_stop()) NRS stop fifo request from 12345-0@lo, seq: 1163 00000100:00100000:2.0:1776178604.909026:0:3356:0:(client.c:2215:ptlrpc_check_set()) Completed RPC req@0000000086df2fc5 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_03:lustre-MDT0001-lwp-OST0000_UUID:3356:1862456998104576:0@lo:400:kworker/u8:0.0 00000100:00100000:2.0:1776178604.909588:0:3353:0:(client.c:1733:ptlrpc_send_new_req()) Sending RPC req@00000000afa36b49 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_00:lustre-MDT0001-lwp-OST0001_UUID:3353:1862456998104704:0@lo:400:kworker/u8:0.0 00000100:00100000:2.0:1776178604.909638:0:3353:0:(events.c:361:request_in_callback()) peer: 12345-0@lo (source: 12345-0@lo) 00000100:00100000:2.0:1776178604.910178:0:6256:0:(service.c:2152:ptlrpc_server_handle_req_in()) got req x1862456998104704 00000100:00100000:2.0:1776178604.910226:0:6256:0:(nrs_fifo.c:179:nrs_fifo_req_get()) NRS start fifo request from 12345-0@lo, seq: 1164 00000100:00100000:2.0:1776178604.910230:0:6256:0:(service.c:2324:ptlrpc_server_handle_request()) Handling RPC req@000000006890aba2 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_001:lustre-MDT0001-lwp-OST0001_UUID+5:3353:x1862456998104704:12345-0@lo:400:kworker/u8:0.0 00000100:00100000:2.0:1776178604.910275:0:6256:0:(service.c:2372:ptlrpc_server_handle_request()) Handled RPC req@000000006890aba2 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_001:lustre-MDT0001-lwp-OST0001_UUID+5:3353:x1862456998104704:12345-0@lo:400:kworker/u8:0.0 Request processed in 44us (638us total) trans 0 rc 0/0 00000100:00100000:2.0:1776178604.910280:0:6256:0:(nrs_fifo.c:241:nrs_fifo_req_stop()) NRS stop fifo request from 12345-0@lo, seq: 1164 00000100:00100000:2.0:1776178604.910308:0:3353:0:(client.c:2215:ptlrpc_check_set()) Completed RPC req@00000000afa36b49 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_00:lustre-MDT0001-lwp-OST0001_UUID:3353:1862456998104704:0@lo:400:kworker/u8:0.0 00000100:00100000:1.0:1776178604.929278:0:3352:0:(events.c:361:request_in_callback()) peer: 12345-0@lo (source: 12345-0@lo) 00000100:00100000:1.0:1776178604.929399:0:6257:0:(service.c:2152:ptlrpc_server_handle_req_in()) got req x1862456998104512 00000100:00100000:1.0:1776178604.929452:0:6257:0:(nrs_fifo.c:179:nrs_fifo_req_get()) NRS start fifo request from 12345-0@lo, seq: 6937 00000100:00100000:1.0:1776178604.929461:0:6257:0:(service.c:2324:ptlrpc_server_handle_request()) Handling RPC req@0000000051ff1425 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_002:0+-99:3352:x1862456998104512:12345-0@lo:38:kworker/u8:0.0 00010000:02000000:1.0:1776178604.929487:0:6257:0:(ldlm_lib.c:1139:target_handle_connect()) lustre-MDT0000: Not available for connect from 0@lo (stopping) 00000100:00100000:1.0:1776178604.929549:0:6257:0:(service.c:2372:ptlrpc_server_handle_request()) Handled RPC req@0000000051ff1425 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_002:0+-99:3352:x1862456998104512:12345-0@lo:38:kworker/u8:0.0 Request processed in 87us (285us total) trans 0 rc -19/-19 00000100:00100000:1.0:1776178604.929557:0:6257:0:(nrs_fifo.c:241:nrs_fifo_req_stop()) NRS stop fifo request from 12345-0@lo, seq: 6937 00000100:00100000:1.0:1776178604.929606:0:3352:0:(client.c:1733:ptlrpc_send_new_req()) Sending RPC req@00000000dd5bd4cd pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:lustre-MDT0000-lwp-OST0001_UUID:3352:1862456998104640:0@lo:38:kworker/u8:0.0 00000100:00100000:1.0:1776178604.929644:0:3352:0:(events.c:361:request_in_callback()) peer: 12345-0@lo (source: 12345-0@lo) 00000100:00100000:1.0:1776178604.929666:0:11281:0:(service.c:2152:ptlrpc_server_handle_req_in()) got req x1862456998104640 00000100:00100000:1.0:1776178604.929684:0:11281:0:(nrs_fifo.c:179:nrs_fifo_req_get()) NRS start fifo request from 12345-0@lo, seq: 6938 00000100:00100000:1.0:1776178604.929687:0:11281:0:(service.c:2324:ptlrpc_server_handle_request()) Handling RPC req@000000008c0fa959 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_004:0+-99:3352:x1862456998104640:12345-0@lo:38:kworker/u8:0.0 00010000:02000000:1.0:1776178604.929694:0:11281:0:(ldlm_lib.c:1139:target_handle_connect()) lustre-MDT0000: Not available for connect from 0@lo (stopping) 00000100:00100000:1.0:1776178604.929717:0:11281:0:(service.c:2372:ptlrpc_server_handle_request()) Handled RPC req@000000008c0fa959 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_004:0+-99:3352:x1862456998104640:12345-0@lo:38:kworker/u8:0.0 Request processed in 28us (73us total) trans 0 rc -19/-19 00000100:00100000:1.0:1776178604.929722:0:11281:0:(nrs_fifo.c:241:nrs_fifo_req_stop()) NRS stop fifo request from 12345-0@lo, seq: 6938 00000100:00100000:1.0:1776178604.929736:0:3352:0:(client.c:1733:ptlrpc_send_new_req()) Sending RPC req@00000000950bb9b2 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:lustre-MDT0001-mdtlov_UUID:3352:1862456998104768:0@lo:38:kworker/u8:0.0 00000100:00100000:1.0:1776178604.929765:0:3352:0:(events.c:361:request_in_callback()) peer: 12345-0@lo (source: 12345-0@lo) 00000100:00100000:1.0:1776178604.929785:0:3352:0:(client.c:1733:ptlrpc_send_new_req()) Sending RPC req@00000000fccd82c9 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:lustre-MDT0000-lwp-MDT0001_UUID:3352:1862456998104960:0@lo:38:kworker/u8:0.0 00000100:00100000:1.0:1776178604.929807:0:3352:0:(events.c:361:request_in_callback()) peer: 12345-0@lo (source: 12345-0@lo) 00000100:00100000:1.0:1776178604.929820:0:6257:0:(service.c:2152:ptlrpc_server_handle_req_in()) got req x1862456998104960 00000100:00100000:1.0:1776178604.929833:0:6257:0:(nrs_fifo.c:179:nrs_fifo_req_get()) NRS start fifo request from 12345-0@lo, seq: 6939 00000100:00100000:1.0:1776178604.929836:0:6257:0:(service.c:2324:ptlrpc_server_handle_request()) Handling RPC req@00000000589b0247 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_002:0+-99:3352:x1862456998104960:12345-0@lo:38:kworker/u8:0.0 00010000:02000000:1.0:1776178604.929842:0:6257:0:(ldlm_lib.c:1139:target_handle_connect()) lustre-MDT0000: Not available for connect from 0@lo (stopping) 00000100:00100000:1.0:1776178604.929861:0:6257:0:(service.c:2372:ptlrpc_server_handle_request()) Handled RPC req@00000000589b0247 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_002:0+-99:3352:x1862456998104960:12345-0@lo:38:kworker/u8:0.0 Request processed in 24us (54us total) trans 0 rc -19/-19 00000100:00100000:1.0:1776178604.929869:0:6257:0:(nrs_fifo.c:241:nrs_fifo_req_stop()) NRS stop fifo request from 12345-0@lo, seq: 6939 00000100:00080000:1.0:1776178604.929907:0:3352:0:(import.c:86:import_set_state_nolock()) 0000000047601a98 lustre-MDT0000_UUID: changing import state from CONNECTING to DISCONN 00000100:00080000:1.0:1776178604.929912:0:3352:0:(import.c:1429:ptlrpc_connect_interpret()) recovery of lustre-MDT0000_UUID on 192.168.204.103@tcp failed (-19) 00000100:00100000:1.0:1776178604.929916:0:3352:0:(client.c:2215:ptlrpc_check_set()) Completed RPC req@00000000f247d7cd pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:lustre-MDT0000-lwp-OST0000_UUID:3352:1862456998104512:0@lo:38:kworker/u8:0.0 00000100:00080000:1.0:1776178604.929947:0:3352:0:(import.c:86:import_set_state_nolock()) 00000000dfc8db13 lustre-MDT0000_UUID: changing import state from CONNECTING to DISCONN 00000100:00080000:1.0:1776178604.929950:0:3352:0:(import.c:1429:ptlrpc_connect_interpret()) recovery of lustre-MDT0000_UUID on 192.168.204.103@tcp failed (-19) 00000100:00100000:1.0:1776178604.929953:0:3352:0:(client.c:2215:ptlrpc_check_set()) Completed RPC req@00000000dd5bd4cd pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:lustre-MDT0000-lwp-OST0001_UUID:3352:1862456998104640:0@lo:38:kworker/u8:0.0 00000100:00080000:1.0:1776178604.929960:0:3352:0:(import.c:86:import_set_state_nolock()) 00000000a7c55b1f lustre-MDT0000_UUID: changing import state from CONNECTING to DISCONN 00000100:00080000:1.0:1776178604.929963:0:3352:0:(import.c:1429:ptlrpc_connect_interpret()) recovery of lustre-MDT0000_UUID on 192.168.204.103@tcp failed (-19) 00000100:00100000:1.0:1776178604.929965:0:3352:0:(client.c:2215:ptlrpc_check_set()) Completed RPC req@00000000fccd82c9 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:lustre-MDT0000-lwp-MDT0001_UUID:3352:1862456998104960:0@lo:38:kworker/u8:0.0 00000100:00100000:1.0:1776178604.930005:0:14128:0:(service.c:2152:ptlrpc_server_handle_req_in()) got req x1862456998104768 00000100:00100000:1.0:1776178604.930023:0:14128:0:(nrs_fifo.c:179:nrs_fifo_req_get()) NRS start fifo request from 12345-0@lo, seq: 786 00000100:00100000:1.0:1776178604.930027:0:14128:0:(service.c:2324:ptlrpc_server_handle_request()) Handling RPC req@000000001ff6e514 pname:cluuid+ref:pid:xid:nid:opc:job mdt_out00_002:0+-99:3352:x1862456998104768:12345-0@lo:38:kworker/u8:0.0 00010000:02000000:1.0:1776178604.930034:0:14128:0:(ldlm_lib.c:1139:target_handle_connect()) lustre-MDT0000: Not available for connect from 0@lo (stopping) 00000100:00100000:1.0:1776178604.930244:0:14128:0:(service.c:2372:ptlrpc_server_handle_request()) Handled RPC req@000000001ff6e514 pname:cluuid+ref:pid:xid:nid:opc:job mdt_out00_002:0+-99:3352:x1862456998104768:12345-0@lo:38:kworker/u8:0.0 Request processed in 216us (478us total) trans 0 rc -19/-19 00000100:00100000:1.0:1776178604.930251:0:14128:0:(nrs_fifo.c:241:nrs_fifo_req_stop()) NRS stop fifo request from 12345-0@lo, seq: 786 00000100:00080000:1.0:1776178604.930275:0:3352:0:(import.c:86:import_set_state_nolock()) 00000000ba4f1821 lustre-MDT0000_UUID: changing import state from CONNECTING to DISCONN 00000100:00080000:1.0:1776178604.930278:0:3352:0:(import.c:1429:ptlrpc_connect_interpret()) recovery of lustre-MDT0000_UUID on 192.168.204.103@tcp failed (-19) 00000100:00100000:1.0:1776178604.930280:0:3352:0:(client.c:2215:ptlrpc_check_set()) Completed RPC req@00000000950bb9b2 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:lustre-MDT0001-mdtlov_UUID:3352:1862456998104768:0@lo:38:kworker/u8:0.0 00000100:00100000:3.0:1776178604.938728:0:3355:0:(events.c:361:request_in_callback()) peer: 12345-0@lo (source: 12345-0@lo) 00000100:00100000:0.0:1776178604.939312:0:3354:0:(events.c:361:request_in_callback()) peer: 12345-0@lo (source: 12345-0@lo) 00000100:00100000:3.0:1776178604.939508:0:20762:0:(service.c:2152:ptlrpc_server_handle_req_in()) got req x1862456998104832 00000100:00100000:3.0:1776178604.939600:0:20762:0:(nrs_fifo.c:179:nrs_fifo_req_get()) NRS start fifo request from 12345-0@lo, seq: 826 00000100:00100000:3.0:1776178604.939610:0:20762:0:(service.c:2324:ptlrpc_server_handle_request()) Handling RPC req@00000000ff7622e8 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_013:lustre-MDT0001-mdtlov_UUID+5:3354:x1862456998104832:12345-0@lo:400:kworker/u8:0.0 00000100:00100000:3.0:1776178604.939686:0:20762:0:(service.c:2372:ptlrpc_server_handle_request()) Handled RPC req@00000000ff7622e8 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_013:lustre-MDT0001-mdtlov_UUID+5:3354:x1862456998104832:12345-0@lo:400:kworker/u8:0.0 Request processed in 76us (375us total) trans 0 rc 0/0 00000100:00100000:3.0:1776178604.939694:0:20762:0:(nrs_fifo.c:241:nrs_fifo_req_stop()) NRS stop fifo request from 12345-0@lo, seq: 826 00000100:00100000:1.0:1776178604.940046:0:3354:0:(client.c:2215:ptlrpc_check_set()) Completed RPC req@00000000d1597b46 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_01:lustre-MDT0001-mdtlov_UUID:3354:1862456998104832:0@lo:400:kworker/u8:0.0 00000100:00100000:2.0:1776178604.946522:0:20137:0:(service.c:2152:ptlrpc_server_handle_req_in()) got req x1862456998104448 00000100:00100000:2.0:1776178604.946634:0:20137:0:(nrs_fifo.c:179:nrs_fifo_req_get()) NRS start fifo request from 12345-0@lo, seq: 312 00000100:00100000:2.0:1776178604.946643:0:20137:0:(service.c:2324:ptlrpc_server_handle_request()) Handling RPC req@0000000060db901b pname:cluuid+ref:pid:xid:nid:opc:job ll_mgs_0000:ac61f839-0e8e-4cac-9c59-08bcfbbad330+18:3355:x1862456998104448:12345-0@lo:400:kworker/u8:0.0 00000100:00100000:2.0:1776178604.946769:0:20137:0:(service.c:2372:ptlrpc_server_handle_request()) Handled RPC req@0000000060db901b pname:cluuid+ref:pid:xid:nid:opc:job ll_mgs_0000:ac61f839-0e8e-4cac-9c59-08bcfbbad330+18:3355:x1862456998104448:12345-0@lo:400:kworker/u8:0.0 Request processed in 126us (8056us total) trans 0 rc 0/0 00000100:00100000:2.0:1776178604.946778:0:20137:0:(nrs_fifo.c:241:nrs_fifo_req_stop()) NRS stop fifo request from 12345-0@lo, seq: 312 00000100:00100000:0.0:1776178604.947114:0:3355:0:(client.c:2215:ptlrpc_check_set()) Completed RPC req@00000000bdb7858e pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_02:ac61f839-0e8e-4cac-9c59-08bcfbbad330:3355:1862456998104448:0@lo:400:kworker/u8:0.0 00010000:00010000:1.0:1776178605.536790:0:9225:0:(ldlm_lockd.c:2792:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:0.0:1776178606.432486:0:3355:0:(ptlrpcd.c:415:ptlrpcd_check()) transfer 1 async RPCs [3->2] 00000100:00100000:0.0:1776178606.432517:0:3355:0:(client.c:1733:ptlrpc_send_new_req()) Sending RPC req@0000000045cb5035 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_02:lustre-MDT0001-mdtlov_UUID:3355:1862456998105024:0@lo:13:osp-pre-1-1.0 00000100:00100000:0.0:1776178606.432611:0:3355:0:(events.c:361:request_in_callback()) peer: 12345-0@lo (source: 12345-0@lo) 00000100:00100000:1.0:1776178606.432726:0:8231:0:(service.c:2152:ptlrpc_server_handle_req_in()) got req x1862456998105024 00000100:00100000:1.0:1776178606.432795:0:8231:0:(nrs_fifo.c:179:nrs_fifo_req_get()) NRS start fifo request from 12345-0@lo, seq: 645 00000100:00100000:1.0:1776178606.432802:0:8231:0:(service.c:2324:ptlrpc_server_handle_request()) Handling RPC req@00000000bdb7858e pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_create00:lustre-MDT0001-mdtlov_UUID+5:3355:x1862456998105024:12345-0@lo:13:osp-pre-1-1.0 00002000:00000024:1.0:1776178606.432843:0:8231:0:(ofd_obd.c:740:ofd_statfs()) blocks cached 229376 granted 491746368 pending 0 free 3923689472 avail 3692167168 00000100:00100000:1.0:1776178606.432907:0:8231:0:(service.c:2372:ptlrpc_server_handle_request()) Handled RPC req@00000000bdb7858e pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_create00:lustre-MDT0001-mdtlov_UUID+5:3355:x1862456998105024:12345-0@lo:13:osp-pre-1-1.0 Request processed in 104us (297us total) trans 0 rc 0/0 00000100:00100000:1.0:1776178606.432915:0:8231:0:(nrs_fifo.c:241:nrs_fifo_req_stop()) NRS stop fifo request from 12345-0@lo, seq: 645 00000100:00100000:3.0:1776178606.433024:0:3355:0:(client.c:2215:ptlrpc_check_set()) Completed RPC req@0000000045cb5035 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_02:lustre-MDT0001-mdtlov_UUID:3355:1862456998105024:0@lo:13:osp-pre-1-1.0 00010000:00010000:2.0:1776178606.560457:0:6244:0:(ldlm_lockd.c:2792:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:2.0:1776178606.950391:0:3353:0:(client.c:1733:ptlrpc_send_new_req()) Sending RPC req@00000000fd453e99 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_00:lustre-MDT0001-mdtlov_UUID:3353:1862456998105088:0@lo:13:osp-pre-0-1.0 00000100:00100000:2.0:1776178606.950493:0:3353:0:(events.c:361:request_in_callback()) peer: 12345-0@lo (source: 12345-0@lo) 00000100:00100000:0.0:1776178606.950633:0:9554:0:(service.c:2152:ptlrpc_server_handle_req_in()) got req x1862456998105088 00000100:00100000:0.0:1776178606.950702:0:9554:0:(nrs_fifo.c:179:nrs_fifo_req_get()) NRS start fifo request from 12345-0@lo, seq: 646 00000100:00100000:0.0:1776178606.950709:0:9554:0:(service.c:2324:ptlrpc_server_handle_request()) Handling RPC req@00000000c71dd77c pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_create00:lustre-MDT0001-mdtlov_UUID+5:3353:x1862456998105088:12345-0@lo:13:osp-pre-0-1.0 00002000:00000024:0.0:1776178606.950751:0:9554:0:(ofd_obd.c:740:ofd_statfs()) blocks cached 315392 granted 484883776 pending 0 free 3923689472 avail 3692167168 00000100:00100000:0.0:1776178606.950809:0:9554:0:(service.c:2372:ptlrpc_server_handle_request()) Handled RPC req@00000000c71dd77c pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_create00:lustre-MDT0001-mdtlov_UUID+5:3353:x1862456998105088:12345-0@lo:13:osp-pre-0-1.0 Request processed in 99us (318us total) trans 0 rc 0/0 00000100:00100000:0.0:1776178606.950817:0:9554:0:(nrs_fifo.c:241:nrs_fifo_req_stop()) NRS stop fifo request from 12345-0@lo, seq: 646 00000100:00100000:2.0:1776178606.950992:0:3353:0:(client.c:2215:ptlrpc_check_set()) Completed RPC req@00000000fd453e99 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_00:lustre-MDT0001-mdtlov_UUID:3353:1862456998105088:0@lo:13:osp-pre-0-1.0 00010000:00010000:1.0:1776178607.584334:0:6245:0:(ldlm_lockd.c:2792:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:0.0:1776178608.427258:0:3348:0:(events.c:361:request_in_callback()) peer: 12345-192.168.204.3@tcp (source: 12345-192.168.204.3@tcp) 00000100:00100000:0.0:1776178608.427349:0:3348:0:(events.c:361:request_in_callback()) peer: 12345-192.168.204.3@tcp (source: 12345-192.168.204.3@tcp) 00000100:00100000:0.0:1776178608.427398:0:3348:0:(events.c:361:request_in_callback()) peer: 12345-192.168.204.3@tcp (source: 12345-192.168.204.3@tcp) 00000100:00100000:0.0:1776178608.427430:0:3348:0:(events.c:361:request_in_callback()) peer: 12345-192.168.204.3@tcp (source: 12345-192.168.204.3@tcp) 00000100:00100000:3.0:1776178608.427496:0:20762:0:(service.c:2152:ptlrpc_server_handle_req_in()) got req x1862456979134528 00000100:00100000:0.0:1776178608.427544:0:3348:0:(events.c:361:request_in_callback()) peer: 12345-192.168.204.3@tcp (source: 12345-192.168.204.3@tcp) 00000100:00100000:3.0:1776178608.427578:0:20762:0:(service.c:2152:ptlrpc_server_handle_req_in()) got req x1862456979134464 00000100:00100000:3.0:1776178608.427604:0:20762:0:(nrs_fifo.c:179:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.204.3@tcp, seq: 827 00000100:00100000:3.0:1776178608.427613:0:20762:0:(service.c:2324:ptlrpc_server_handle_request()) Handling RPC req@000000000b651128 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_013:b50963bd-8eba-4432-9a54-8888d00e3fa1+9:2273:x1862456979134528:12345-192.168.204.3@tcp:400:kworker/u8:4.0 00000100:00100000:1.0:1776178608.427658:0:11281:0:(service.c:2152:ptlrpc_server_handle_req_in()) got req x1862456979134400 00000100:00100000:1.0:1776178608.427696:0:11281:0:(service.c:2152:ptlrpc_server_handle_req_in()) got req x1862456979134336 00000100:00100000:3.0:1776178608.427717:0:20762:0:(service.c:2372:ptlrpc_server_handle_request()) Handled RPC req@000000000b651128 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_013:b50963bd-8eba-4432-9a54-8888d00e3fa1+9:2273:x1862456979134528:12345-192.168.204.3@tcp:400:kworker/u8:4.0 Request processed in 104us (369us total) trans 0 rc 0/0 00000100:00100000:1.0:1776178608.427723:0:11281:0:(nrs_fifo.c:179:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.204.3@tcp, seq: 1165 00000100:00100000:3.0:1776178608.427733:0:20762:0:(nrs_fifo.c:241:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.204.3@tcp, seq: 827 00000100:00100000:1.0:1776178608.427734:0:11281:0:(service.c:2324:ptlrpc_server_handle_request()) Handling RPC req@00000000bbf26718 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_004:b50963bd-8eba-4432-9a54-8888d00e3fa1+6:2273:x1862456979134400:12345-192.168.204.3@tcp:400:kworker/u8:4.0 00000100:00100000:3.0:1776178608.427757:0:20762:0:(nrs_fifo.c:179:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.204.3@tcp, seq: 828 00000100:00100000:3.0:1776178608.427763:0:20762:0:(service.c:2324:ptlrpc_server_handle_request()) Handling RPC req@000000005d5717f1 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_013:b50963bd-8eba-4432-9a54-8888d00e3fa1+8:2273:x1862456979134464:12345-192.168.204.3@tcp:400:kworker/u8:4.0 00000100:00100000:1.0:1776178608.427774:0:11281:0:(service.c:2372:ptlrpc_server_handle_request()) Handled RPC req@00000000bbf26718 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_004:b50963bd-8eba-4432-9a54-8888d00e3fa1+6:2273:x1862456979134400:12345-192.168.204.3@tcp:400:kworker/u8:4.0 Request processed in 41us (541us total) trans 0 rc 0/0 00000100:00100000:1.0:1776178608.427782:0:11281:0:(nrs_fifo.c:241:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.204.3@tcp, seq: 1165 00000100:00100000:3.0:1776178608.427797:0:20762:0:(service.c:2372:ptlrpc_server_handle_request()) Handled RPC req@000000005d5717f1 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_013:b50963bd-8eba-4432-9a54-8888d00e3fa1+8:2273:x1862456979134464:12345-192.168.204.3@tcp:400:kworker/u8:4.0 Request processed in 32us (365us total) trans 0 rc 0/0 00000100:00100000:1.0:1776178608.427803:0:11281:0:(nrs_fifo.c:179:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.204.3@tcp, seq: 6940 00000100:00100000:3.0:1776178608.427812:0:20762:0:(nrs_fifo.c:241:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.204.3@tcp, seq: 828 00000100:00100000:1.0:1776178608.427813:0:11281:0:(service.c:2324:ptlrpc_server_handle_request()) Handling RPC req@00000000eb611cec pname:cluuid+ref:pid:xid:nid:opc:job mdt00_004:0+-99:2269:x1862456979134336:12345-192.168.204.3@tcp:38:kworker/u8:4.0 00010000:02000000:1.0:1776178608.427835:0:11281:0:(ldlm_lib.c:1139:target_handle_connect()) lustre-MDT0000: Not available for connect from 192.168.204.3@tcp (stopping) 00000100:00100000:1.0:1776178608.427871:0:11281:0:(service.c:2372:ptlrpc_server_handle_request()) Handled RPC req@00000000eb611cec pname:cluuid+ref:pid:xid:nid:opc:job mdt00_004:0+-99:2269:x1862456979134336:12345-192.168.204.3@tcp:38:kworker/u8:4.0 Request processed in 60us (328us total) trans 0 rc -19/-19 00000100:00100000:1.0:1776178608.427879:0:11281:0:(nrs_fifo.c:241:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.204.3@tcp, seq: 6940 00000100:00100000:0.0:1776178608.428444:0:20140:0:(service.c:2152:ptlrpc_server_handle_req_in()) got req x1862456979134272 00000100:00100000:0.0:1776178608.428493:0:20140:0:(nrs_fifo.c:179:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.204.3@tcp, seq: 313 00000100:00100000:0.0:1776178608.428498:0:20140:0:(service.c:2324:ptlrpc_server_handle_request()) Handling RPC req@000000003e5a23bc pname:cluuid+ref:pid:xid:nid:opc:job ll_mgs_0002:5b5c66b1-1a2f-4651-8548-0c75b9a38a94+9:2273:x1862456979134272:12345-192.168.204.3@tcp:400:kworker/u8:4.0 00000100:00100000:0.0:1776178608.428540:0:20140:0:(service.c:2372:ptlrpc_server_handle_request()) Handled RPC req@000000003e5a23bc pname:cluuid+ref:pid:xid:nid:opc:job ll_mgs_0002:5b5c66b1-1a2f-4651-8548-0c75b9a38a94+9:2273:x1862456979134272:12345-192.168.204.3@tcp:400:kworker/u8:4.0 Request processed in 41us (1141us total) trans 0 rc 0/0 00000100:00100000:0.0:1776178608.428547:0:20140:0:(nrs_fifo.c:241:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.204.3@tcp, seq: 313 00010000:00010000:1.0:1776178608.613396:0:9224:0:(ldlm_lockd.c:2792:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00010000:00010000:1.0:1776178609.632621:0:9225:0:(ldlm_lockd.c:2792:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00080000:3.0:1776178610.020553:0:22249:0:(pinger.c:251:ptlrpc_pinger_process_import()) lustre-MDT0000-lwp-OST0000_UUID->lustre-MDT0000_UUID: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00080000:3.0:1776178610.020568:0:22249:0:(recover.c:57:ptlrpc_initiate_recovery()) lustre-MDT0000_UUID: starting recovery 00000100:00080000:3.0:1776178610.020573:0:22249:0:(import.c:86:import_set_state_nolock()) 0000000047601a98 lustre-MDT0000_UUID: changing import state from DISCONN to CONNECTING 00000100:00080000:3.0:1776178610.020587:0:22249:0:(import.c:534:import_select_connection()) lustre-MDT0000-lwp-OST0000: connect to NID 0@lo last attempt 1312 00000100:00080000:3.0:1776178610.020593:0:22249:0:(import.c:581:import_select_connection()) lustre-MDT0000-lwp-OST0000: tried all connections, increasing latency to 21s 00000100:00080000:3.0:1776178610.020601:0:22249:0:(import.c:615:import_select_connection()) lustre-MDT0000-lwp-OST0000: import 0000000047601a98 using connection 192.168.204.103@tcp/0@lo 00000100:00100000:3.0:1776178610.020653:0:22249:0:(import.c:807:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5) req@000000006d8fce24 x1862456998105216/t0(0) o38->lustre-MDT0000-lwp-OST0000@0@lo:12/10 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' 00000100:00100000:0.0:1776178610.020765:0:3352:0:(client.c:1733:ptlrpc_send_new_req()) Sending RPC req@000000006d8fce24 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:lustre-MDT0000-lwp-OST0000_UUID:3352:1862456998105216:0@lo:38:kworker/u8:0.0 00000100:00100000:0.0:1776178610.020862:0:3352:0:(events.c:361:request_in_callback()) peer: 12345-0@lo (source: 12345-0@lo) 00000100:00100000:1.0:1776178610.020979:0:3354:0:(client.c:1733:ptlrpc_send_new_req()) Sending RPC req@0000000045cb5035 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_01:ac61f839-0e8e-4cac-9c59-08bcfbbad330:3354:1862456998105152:0@lo:400:kworker/u8:0.0 00000100:00080000:3.0:1776178610.021132:0:22249:0:(pinger.c:251:ptlrpc_pinger_process_import()) lustre-MDT0000-lwp-OST0001_UUID->lustre-MDT0000_UUID: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00080000:3.0:1776178610.021136:0:22249:0:(recover.c:57:ptlrpc_initiate_recovery()) lustre-MDT0000_UUID: starting recovery 00000100:00080000:3.0:1776178610.021139:0:22249:0:(import.c:86:import_set_state_nolock()) 00000000dfc8db13 lustre-MDT0000_UUID: changing import state from DISCONN to CONNECTING 00000100:00080000:3.0:1776178610.021143:0:22249:0:(import.c:534:import_select_connection()) lustre-MDT0000-lwp-OST0001: connect to NID 0@lo last attempt 1312 00000100:00080000:3.0:1776178610.021145:0:22249:0:(import.c:581:import_select_connection()) lustre-MDT0000-lwp-OST0001: tried all connections, increasing latency to 21s 00000100:00080000:3.0:1776178610.021150:0:22249:0:(import.c:615:import_select_connection()) lustre-MDT0000-lwp-OST0001: import 00000000dfc8db13 using connection 192.168.204.103@tcp/0@lo 00000100:00100000:3.0:1776178610.021163:0:22249:0:(import.c:807:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5) req@000000008f94e9dd x1862456998105344/t0(0) o38->lustre-MDT0000-lwp-OST0001@0@lo:12/10 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' 00000100:00100000:1.0:1776178610.021377:0:3354:0:(events.c:361:request_in_callback()) peer: 12345-0@lo (source: 12345-0@lo) 00000100:00100000:1.0:1776178610.021460:0:6256:0:(service.c:2152:ptlrpc_server_handle_req_in()) got req x1862456998105216 00000100:00100000:1.0:1776178610.021506:0:6256:0:(nrs_fifo.c:179:nrs_fifo_req_get()) NRS start fifo request from 12345-0@lo, seq: 6941 00000100:00100000:1.0:1776178610.021514:0:6256:0:(service.c:2324:ptlrpc_server_handle_request()) Handling RPC req@00000000d04e7a02 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_001:0+-99:3352:x1862456998105216:12345-0@lo:38:kworker/u8:0.0 00010000:02000000:1.0:1776178610.021534:0:6256:0:(ldlm_lib.c:1139:target_handle_connect()) lustre-MDT0000: Not available for connect from 0@lo (stopping) 00000100:00100000:2.0:1776178610.021693:0:3356:0:(ptlrpcd.c:415:ptlrpcd_check()) transfer 1 async RPCs [2->3] 00000100:00100000:2.0:1776178610.021705:0:3356:0:(client.c:1733:ptlrpc_send_new_req()) Sending RPC req@0000000025b3b8e0 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_03:lustre-MDT0001-lwp-OST0000_UUID:3356:1862456998105280:0@lo:400:kworker/u8:0.0 00000100:00100000:2.0:1776178610.021739:0:3356:0:(events.c:361:request_in_callback()) peer: 12345-0@lo (source: 12345-0@lo) 00000100:00100000:2.0:1776178610.021769:0:6257:0:(service.c:2152:ptlrpc_server_handle_req_in()) got req x1862456998105280 00000100:00100000:2.0:1776178610.021811:0:6257:0:(nrs_fifo.c:179:nrs_fifo_req_get()) NRS start fifo request from 12345-0@lo, seq: 1166 00000100:00100000:2.0:1776178610.021815:0:6257:0:(service.c:2324:ptlrpc_server_handle_request()) Handling RPC req@00000000fd453e99 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_002:lustre-MDT0001-lwp-OST0000_UUID+5:3356:x1862456998105280:12345-0@lo:400:kworker/u8:0.0 00000100:00100000:2.0:1776178610.021866:0:6257:0:(service.c:2372:ptlrpc_server_handle_request()) Handled RPC req@00000000fd453e99 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_002:lustre-MDT0001-lwp-OST0000_UUID+5:3356:x1862456998105280:12345-0@lo:400:kworker/u8:0.0 Request processed in 50us (127us total) trans 0 rc 0/0 00000100:00100000:2.0:1776178610.021873:0:6257:0:(nrs_fifo.c:241:nrs_fifo_req_stop()) NRS stop fifo request from 12345-0@lo, seq: 1166 00000100:00100000:2.0:1776178610.021926:0:3356:0:(client.c:2215:ptlrpc_check_set()) Completed RPC req@0000000025b3b8e0 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_03:lustre-MDT0001-lwp-OST0000_UUID:3356:1862456998105280:0@lo:400:kworker/u8:0.0 00000100:00100000:2.0:1776178610.021983:0:20137:0:(service.c:2152:ptlrpc_server_handle_req_in()) got req x1862456998105152 00000100:00100000:2.0:1776178610.022138:0:20137:0:(nrs_fifo.c:179:nrs_fifo_req_get()) NRS start fifo request from 12345-0@lo, seq: 314 00000100:00100000:2.0:1776178610.022143:0:20137:0:(service.c:2324:ptlrpc_server_handle_request()) Handling RPC req@00000000fc8eef33 pname:cluuid+ref:pid:xid:nid:opc:job ll_mgs_0000:ac61f839-0e8e-4cac-9c59-08bcfbbad330+18:3354:x1862456998105152:12345-0@lo:400:kworker/u8:0.0 00000100:00100000:2.0:1776178610.022182:0:20137:0:(service.c:2372:ptlrpc_server_handle_request()) Handled RPC req@00000000fc8eef33 pname:cluuid+ref:pid:xid:nid:opc:job ll_mgs_0000:ac61f839-0e8e-4cac-9c59-08bcfbbad330+18:3354:x1862456998105152:12345-0@lo:400:kworker/u8:0.0 Request processed in 38us (805us total) trans 0 rc 0/0 00000100:00100000:2.0:1776178610.022188:0:20137:0:(nrs_fifo.c:241:nrs_fifo_req_stop()) NRS stop fifo request from 12345-0@lo, seq: 314 00000100:00100000:2.0:1776178610.022576:0:3354:0:(client.c:2215:ptlrpc_check_set()) Completed RPC req@0000000045cb5035 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_01:ac61f839-0e8e-4cac-9c59-08bcfbbad330:3354:1862456998105152:0@lo:400:kworker/u8:0.0 00000100:00100000:0.0:1776178610.022796:0:3352:0:(client.c:1733:ptlrpc_send_new_req()) Sending RPC req@000000008f94e9dd pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:lustre-MDT0000-lwp-OST0001_UUID:3352:1862456998105344:0@lo:38:kworker/u8:0.0 00000100:00100000:0.0:1776178610.022842:0:3352:0:(events.c:361:request_in_callback()) peer: 12345-0@lo (source: 12345-0@lo) 00000100:00080000:3.0:1776178610.023098:0:22249:0:(pinger.c:251:ptlrpc_pinger_process_import()) lustre-MDT0001-mdtlov_UUID->lustre-MDT0000_UUID: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00080000:3.0:1776178610.023105:0:22249:0:(recover.c:57:ptlrpc_initiate_recovery()) lustre-MDT0000_UUID: starting recovery 00000100:00080000:3.0:1776178610.023109:0:22249:0:(import.c:86:import_set_state_nolock()) 00000000ba4f1821 lustre-MDT0000_UUID: changing import state from DISCONN to CONNECTING 00000100:00080000:3.0:1776178610.023114:0:22249:0:(import.c:534:import_select_connection()) lustre-MDT0000-osp-MDT0001: connect to NID 0@lo last attempt 1312 00000100:00080000:3.0:1776178610.023117:0:22249:0:(import.c:581:import_select_connection()) lustre-MDT0000-osp-MDT0001: tried all connections, increasing latency to 21s 00000100:00080000:3.0:1776178610.023122:0:22249:0:(import.c:615:import_select_connection()) lustre-MDT0000-osp-MDT0001: import 00000000ba4f1821 using connection 192.168.204.103@tcp/0@lo 00000100:00100000:3.0:1776178610.023146:0:22249:0:(import.c:807:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5) req@0000000037d88937 x1862456998105472/t0(0) o38->lustre-MDT0000-osp-MDT0001@0@lo:24/4 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' 00000100:00080000:3.0:1776178610.023191:0:22249:0:(pinger.c:251:ptlrpc_pinger_process_import()) lustre-MDT0000-lwp-MDT0001_UUID->lustre-MDT0000_UUID: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00080000:3.0:1776178610.023195:0:22249:0:(recover.c:57:ptlrpc_initiate_recovery()) lustre-MDT0000_UUID: starting recovery 00000100:00080000:3.0:1776178610.023196:0:22249:0:(import.c:86:import_set_state_nolock()) 00000000a7c55b1f lustre-MDT0000_UUID: changing import state from DISCONN to CONNECTING 00000100:00080000:3.0:1776178610.023200:0:22249:0:(import.c:534:import_select_connection()) lustre-MDT0000-lwp-MDT0001: connect to NID 0@lo last attempt 1312 00000100:00080000:3.0:1776178610.023202:0:22249:0:(import.c:581:import_select_connection()) lustre-MDT0000-lwp-MDT0001: tried all connections, increasing latency to 21s 00000100:00080000:3.0:1776178610.023206:0:22249:0:(import.c:615:import_select_connection()) lustre-MDT0000-lwp-MDT0001: import 00000000a7c55b1f using connection 192.168.204.103@tcp/0@lo 00000100:00100000:3.0:1776178610.023216:0:22249:0:(import.c:807:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5) req@00000000f8a7b10a x1862456998105664/t0(0) o38->lustre-MDT0000-lwp-MDT0001@0@lo:12/10 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' 00000100:00100000:3.0:1776178610.023423:0:3353:0:(client.c:1733:ptlrpc_send_new_req()) Sending RPC req@00000000964f7a65 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_00:lustre-MDT0001-mdtlov_UUID:3353:1862456998105536:0@lo:400:kworker/u8:0.0 00000100:00100000:3.0:1776178610.023462:0:3353:0:(events.c:361:request_in_callback()) peer: 12345-0@lo (source: 12345-0@lo) 00000100:00100000:3.0:1776178610.023488:0:3353:0:(ptlrpcd.c:415:ptlrpcd_check()) transfer 1 async RPCs [1->0] 00000100:00100000:3.0:1776178610.023493:0:3353:0:(client.c:1733:ptlrpc_send_new_req()) Sending RPC req@00000000f1ed5d03 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_00:lustre-MDT0001-mdtlov_UUID:3353:1862456998105600:0@lo:400:kworker/u8:0.0 00000100:00100000:3.0:1776178610.023515:0:3353:0:(events.c:361:request_in_callback()) peer: 12345-0@lo (source: 12345-0@lo) 00000100:00100000:3.0:1776178610.023550:0:20759:0:(service.c:2152:ptlrpc_server_handle_req_in()) got req x1862456998105536 00000100:00100000:3.0:1776178610.023586:0:20759:0:(service.c:2152:ptlrpc_server_handle_req_in()) got req x1862456998105600 00000100:00100000:3.0:1776178610.023605:0:20759:0:(nrs_fifo.c:179:nrs_fifo_req_get()) NRS start fifo request from 12345-0@lo, seq: 829 00000100:00100000:3.0:1776178610.023610:0:20759:0:(service.c:2324:ptlrpc_server_handle_request()) Handling RPC req@00000000de8350d4 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_010:lustre-MDT0001-mdtlov_UUID+5:3353:x1862456998105536:12345-0@lo:400:kworker/u8:0.0 00000100:00100000:3.0:1776178610.023658:0:20759:0:(service.c:2372:ptlrpc_server_handle_request()) Handled RPC req@00000000de8350d4 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_010:lustre-MDT0001-mdtlov_UUID+5:3353:x1862456998105536:12345-0@lo:400:kworker/u8:0.0 Request processed in 48us (196us total) trans 0 rc 0/0 00000100:00100000:3.0:1776178610.023665:0:20759:0:(nrs_fifo.c:241:nrs_fifo_req_stop()) NRS stop fifo request from 12345-0@lo, seq: 829 00000100:00100000:3.0:1776178610.023679:0:20759:0:(nrs_fifo.c:179:nrs_fifo_req_get()) NRS start fifo request from 12345-0@lo, seq: 830 00000100:00100000:3.0:1776178610.023683:0:20759:0:(service.c:2324:ptlrpc_server_handle_request()) Handling RPC req@000000005c567288 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_010:lustre-MDT0001-mdtlov_UUID+5:3353:x1862456998105600:12345-0@lo:400:kworker/u8:0.0 00000100:00100000:3.0:1776178610.023706:0:20759:0:(service.c:2372:ptlrpc_server_handle_request()) Handled RPC req@000000005c567288 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_010:lustre-MDT0001-mdtlov_UUID+5:3353:x1862456998105600:12345-0@lo:400:kworker/u8:0.0 Request processed in 23us (191us total) trans 0 rc 0/0 00000100:00100000:3.0:1776178610.023711:0:20759:0:(nrs_fifo.c:241:nrs_fifo_req_stop()) NRS stop fifo request from 12345-0@lo, seq: 830 00000100:00100000:3.0:1776178610.023750:0:3353:0:(client.c:2215:ptlrpc_check_set()) Completed RPC req@00000000f1ed5d03 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_00:lustre-MDT0001-mdtlov_UUID:3353:1862456998105600:0@lo:400:kworker/u8:0.0 00000100:00100000:3.0:1776178610.023761:0:3353:0:(client.c:2215:ptlrpc_check_set()) Completed RPC req@00000000964f7a65 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_00:lustre-MDT0001-mdtlov_UUID:3353:1862456998105536:0@lo:400:kworker/u8:0.0 00000100:00100000:3.0:1776178610.023782:0:3356:0:(client.c:1733:ptlrpc_send_new_req()) Sending RPC req@00000000be147540 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_03:lustre-MDT0001-lwp-OST0001_UUID:3356:1862456998105408:0@lo:400:kworker/u8:0.0 00000100:00100000:3.0:1776178610.023806:0:3356:0:(events.c:361:request_in_callback()) peer: 12345-0@lo (source: 12345-0@lo) 00000100:00100000:1.0:1776178610.024116:0:6256:0:(service.c:2372:ptlrpc_server_handle_request()) Handled RPC req@00000000d04e7a02 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_001:0+-99:3352:x1862456998105216:12345-0@lo:38:kworker/u8:0.0 Request processed in 2600us (3254us total) trans 0 rc -19/-19 00000100:00100000:1.0:1776178610.024126:0:6256:0:(nrs_fifo.c:241:nrs_fifo_req_stop()) NRS stop fifo request from 12345-0@lo, seq: 6941 00000100:00080000:0.0:1776178610.024698:0:3352:0:(import.c:86:import_set_state_nolock()) 0000000047601a98 lustre-MDT0000_UUID: changing import state from CONNECTING to DISCONN 00000100:00080000:0.0:1776178610.024705:0:3352:0:(import.c:1429:ptlrpc_connect_interpret()) recovery of lustre-MDT0000_UUID on 192.168.204.103@tcp failed (-19) 00000100:00100000:0.0:1776178610.024709:0:3352:0:(client.c:2215:ptlrpc_check_set()) Completed RPC req@000000006d8fce24 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:lustre-MDT0000-lwp-OST0000_UUID:3352:1862456998105216:0@lo:38:kworker/u8:0.0 00000100:00100000:0.0:1776178610.024730:0:3352:0:(client.c:1733:ptlrpc_send_new_req()) Sending RPC req@0000000037d88937 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:lustre-MDT0001-mdtlov_UUID:3352:1862456998105472:0@lo:38:kworker/u8:0.0 00000100:00100000:0.0:1776178610.024770:0:3352:0:(events.c:361:request_in_callback()) peer: 12345-0@lo (source: 12345-0@lo) 00000100:00100000:0.0:1776178610.024791:0:3352:0:(client.c:1733:ptlrpc_send_new_req()) Sending RPC req@00000000f8a7b10a pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:lustre-MDT0000-lwp-MDT0001_UUID:3352:1862456998105664:0@lo:38:kworker/u8:0.0 00000100:00100000:0.0:1776178610.024815:0:3352:0:(events.c:361:request_in_callback()) peer: 12345-0@lo (source: 12345-0@lo) 00000100:00100000:2.0:1776178610.024880:0:6255:0:(service.c:2152:ptlrpc_server_handle_req_in()) got req x1862456998105344 00000100:00100000:2.0:1776178610.024916:0:6255:0:(service.c:2152:ptlrpc_server_handle_req_in()) got req x1862456998105408 00000100:00100000:2.0:1776178610.024943:0:6255:0:(nrs_fifo.c:179:nrs_fifo_req_get()) NRS start fifo request from 12345-0@lo, seq: 1167 00000100:00100000:2.0:1776178610.024946:0:6255:0:(service.c:2324:ptlrpc_server_handle_request()) Handling RPC req@00000000964f7a65 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_000:lustre-MDT0001-lwp-OST0001_UUID+5:3356:x1862456998105408:12345-0@lo:400:kworker/u8:0.0 00000100:00100000:2.0:1776178610.025121:0:6255:0:(service.c:2372:ptlrpc_server_handle_request()) Handled RPC req@00000000964f7a65 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_000:lustre-MDT0001-lwp-OST0001_UUID+5:3356:x1862456998105408:12345-0@lo:400:kworker/u8:0.0 Request processed in 173us (1314us total) trans 0 rc 0/0 00000100:00100000:2.0:1776178610.025128:0:6255:0:(nrs_fifo.c:241:nrs_fifo_req_stop()) NRS stop fifo request from 12345-0@lo, seq: 1167 00000100:00100000:2.0:1776178610.025145:0:6255:0:(nrs_fifo.c:179:nrs_fifo_req_get()) NRS start fifo request from 12345-0@lo, seq: 6942 00000100:00100000:2.0:1776178610.025149:0:6255:0:(service.c:2324:ptlrpc_server_handle_request()) Handling RPC req@00000000e2b892b4 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_000:0+-99:3352:x1862456998105344:12345-0@lo:38:kworker/u8:0.0 00010000:02000000:2.0:1776178610.025174:0:6255:0:(ldlm_lib.c:1139:target_handle_connect()) lustre-MDT0000: Not available for connect from 0@lo (stopping) 00000100:00100000:2.0:1776178610.025202:0:6255:0:(service.c:2372:ptlrpc_server_handle_request()) Handled RPC req@00000000e2b892b4 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_000:0+-99:3352:x1862456998105344:12345-0@lo:38:kworker/u8:0.0 Request processed in 52us (2361us total) trans 0 rc -19/-19 00000100:00100000:2.0:1776178610.025211:0:6255:0:(nrs_fifo.c:241:nrs_fifo_req_stop()) NRS stop fifo request from 12345-0@lo, seq: 6942 00000100:00100000:2.0:1776178610.025485:0:3356:0:(client.c:2215:ptlrpc_check_set()) Completed RPC req@00000000be147540 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_03:lustre-MDT0001-lwp-OST0001_UUID:3356:1862456998105408:0@lo:400:kworker/u8:0.0 00000100:00100000:2.0:1776178610.025527:0:15213:0:(service.c:2152:ptlrpc_server_handle_req_in()) got req x1862456998105472 00000100:00100000:2.0:1776178610.025550:0:15213:0:(nrs_fifo.c:179:nrs_fifo_req_get()) NRS start fifo request from 12345-0@lo, seq: 787 00000100:00100000:2.0:1776178610.025554:0:15213:0:(service.c:2324:ptlrpc_server_handle_request()) Handling RPC req@000000009be9ae8d pname:cluuid+ref:pid:xid:nid:opc:job mdt_out00_003:0+-99:3352:x1862456998105472:12345-0@lo:38:kworker/u8:0.0 00010000:02000000:2.0:1776178610.025562:0:15213:0:(ldlm_lib.c:1139:target_handle_connect()) lustre-MDT0000: Not available for connect from 0@lo (stopping) 00000100:00100000:2.0:1776178610.025590:0:15213:0:(service.c:2372:ptlrpc_server_handle_request()) Handled RPC req@000000009be9ae8d pname:cluuid+ref:pid:xid:nid:opc:job mdt_out00_003:0+-99:3352:x1862456998105472:12345-0@lo:38:kworker/u8:0.0 Request processed in 34us (819us total) trans 0 rc -19/-19 00000100:00100000:2.0:1776178610.025595:0:15213:0:(nrs_fifo.c:241:nrs_fifo_req_stop()) NRS stop fifo request from 12345-0@lo, seq: 787 00000100:00080000:0.0:1776178610.026352:0:3352:0:(import.c:86:import_set_state_nolock()) 00000000dfc8db13 lustre-MDT0000_UUID: changing import state from CONNECTING to DISCONN 00000100:00080000:0.0:1776178610.026358:0:3352:0:(import.c:1429:ptlrpc_connect_interpret()) recovery of lustre-MDT0000_UUID on 192.168.204.103@tcp failed (-19) 00000100:00100000:0.0:1776178610.026363:0:3352:0:(client.c:2215:ptlrpc_check_set()) Completed RPC req@000000008f94e9dd pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:lustre-MDT0000-lwp-OST0001_UUID:3352:1862456998105344:0@lo:38:kworker/u8:0.0 00000100:00080000:0.0:1776178610.026388:0:3352:0:(import.c:86:import_set_state_nolock()) 00000000ba4f1821 lustre-MDT0000_UUID: changing import state from CONNECTING to DISCONN 00000100:00080000:0.0:1776178610.026392:0:3352:0:(import.c:1429:ptlrpc_connect_interpret()) recovery of lustre-MDT0000_UUID on 192.168.204.103@tcp failed (-19) 00000100:00100000:0.0:1776178610.026395:0:3352:0:(client.c:2215:ptlrpc_check_set()) Completed RPC req@0000000037d88937 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:lustre-MDT0001-mdtlov_UUID:3352:1862456998105472:0@lo:38:kworker/u8:0.0 00000100:00100000:3.0:1776178610.026445:0:11281:0:(service.c:2152:ptlrpc_server_handle_req_in()) got req x1862456998105664 00000100:00100000:3.0:1776178610.026520:0:11281:0:(nrs_fifo.c:179:nrs_fifo_req_get()) NRS start fifo request from 12345-0@lo, seq: 6943 00000100:00100000:3.0:1776178610.026524:0:11281:0:(service.c:2324:ptlrpc_server_handle_request()) Handling RPC req@00000000f18ffde7 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_004:0+-99:3352:x1862456998105664:12345-0@lo:38:kworker/u8:0.0 00010000:02000000:3.0:1776178610.026536:0:11281:0:(ldlm_lib.c:1139:target_handle_connect()) lustre-MDT0000: Not available for connect from 0@lo (stopping) 00000100:00100000:3.0:1776178610.026584:0:11281:0:(service.c:2372:ptlrpc_server_handle_request()) Handled RPC req@00000000f18ffde7 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_004:0+-99:3352:x1862456998105664:12345-0@lo:38:kworker/u8:0.0 Request processed in 58us (1768us total) trans 0 rc -19/-19 00000100:00100000:3.0:1776178610.026590:0:11281:0:(nrs_fifo.c:241:nrs_fifo_req_stop()) NRS stop fifo request from 12345-0@lo, seq: 6943 00000100:00080000:0.0:1776178610.026641:0:3352:0:(import.c:86:import_set_state_nolock()) 00000000a7c55b1f lustre-MDT0000_UUID: changing import state from CONNECTING to DISCONN 00000100:00080000:0.0:1776178610.026645:0:3352:0:(import.c:1429:ptlrpc_connect_interpret()) recovery of lustre-MDT0000_UUID on 192.168.204.103@tcp failed (-19) 00000100:00100000:0.0:1776178610.026649:0:3352:0:(client.c:2215:ptlrpc_check_set()) Completed RPC req@00000000f8a7b10a pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:lustre-MDT0000-lwp-MDT0001_UUID:3352:1862456998105664:0@lo:38:kworker/u8:0.0 00010000:00010000:2.0:1776178610.656512:0:6244:0:(ldlm_lockd.c:2792:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:0.0:1776178611.552479:0:3356:0:(ptlrpcd.c:415:ptlrpcd_check()) transfer 1 async RPCs [2->3] 00000100:00100000:0.0:1776178611.552508:0:3356:0:(client.c:1733:ptlrpc_send_new_req()) Sending RPC req@00000000f1ed5d03 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_03:lustre-MDT0001-mdtlov_UUID:3356:1862456998105728:0@lo:13:osp-pre-1-1.0 00000100:00100000:0.0:1776178611.552592:0:3356:0:(events.c:361:request_in_callback()) peer: 12345-0@lo (source: 12345-0@lo) 00000100:00100000:1.0:1776178611.552702:0:8231:0:(service.c:2152:ptlrpc_server_handle_req_in()) got req x1862456998105728 00000100:00100000:1.0:1776178611.552772:0:8231:0:(nrs_fifo.c:179:nrs_fifo_req_get()) NRS start fifo request from 12345-0@lo, seq: 647 00000100:00100000:1.0:1776178611.552779:0:8231:0:(service.c:2324:ptlrpc_server_handle_request()) Handling RPC req@00000000714adc4d pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_create00:lustre-MDT0001-mdtlov_UUID+5:3356:x1862456998105728:12345-0@lo:13:osp-pre-1-1.0 00002000:00000024:1.0:1776178611.552816:0:8231:0:(ofd_obd.c:740:ofd_statfs()) blocks cached 229376 granted 491746368 pending 0 free 3923689472 avail 3692167168 00000100:00100000:1.0:1776178611.552871:0:8231:0:(service.c:2372:ptlrpc_server_handle_request()) Handled RPC req@00000000714adc4d pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_create00:lustre-MDT0001-mdtlov_UUID+5:3356:x1862456998105728:12345-0@lo:13:osp-pre-1-1.0 Request processed in 92us (281us total) trans 0 rc 0/0 00000100:00100000:1.0:1776178611.552879:0:8231:0:(nrs_fifo.c:241:nrs_fifo_req_stop()) NRS stop fifo request from 12345-0@lo, seq: 647 00000100:00100000:3.0:1776178611.552972:0:3356:0:(client.c:2215:ptlrpc_check_set()) Completed RPC req@00000000f1ed5d03 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_03:lustre-MDT0001-mdtlov_UUID:3356:1862456998105728:0@lo:13:osp-pre-1-1.0 00010000:00010000:1.0:1776178611.681400:0:6245:0:(ldlm_lockd.c:2792:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:3.0:1776178612.065000:0:3356:0:(client.c:1733:ptlrpc_send_new_req()) Sending RPC req@00000000936848a2 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_03:lustre-MDT0001-mdtlov_UUID:3356:1862456998105792:0@lo:13:osp-pre-0-1.0 00000100:00100000:3.0:1776178612.065177:0:3356:0:(events.c:361:request_in_callback()) peer: 12345-0@lo (source: 12345-0@lo) 00000100:00100000:0.0:1776178612.065743:0:9554:0:(service.c:2152:ptlrpc_server_handle_req_in()) got req x1862456998105792 00000100:00100000:0.0:1776178612.068824:0:9554:0:(nrs_fifo.c:179:nrs_fifo_req_get()) NRS start fifo request from 12345-0@lo, seq: 648 00000100:00100000:0.0:1776178612.068838:0:9554:0:(service.c:2324:ptlrpc_server_handle_request()) Handling RPC req@00000000f1ed5d03 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_create00:lustre-MDT0001-mdtlov_UUID+5:3356:x1862456998105792:12345-0@lo:13:osp-pre-0-1.0 00002000:00000024:0.0:1776178612.068890:0:9554:0:(ofd_obd.c:740:ofd_statfs()) blocks cached 315392 granted 484883776 pending 0 free 3923689472 avail 3692167168 00000100:00100000:0.0:1776178612.069020:0:9554:0:(service.c:2372:ptlrpc_server_handle_request()) Handled RPC req@00000000f1ed5d03 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_create00:lustre-MDT0001-mdtlov_UUID+5:3356:x1862456998105792:12345-0@lo:13:osp-pre-0-1.0 Request processed in 185us (3845us total) trans 0 rc 0/0 00000100:00100000:0.0:1776178612.069030:0:9554:0:(nrs_fifo.c:241:nrs_fifo_req_stop()) NRS stop fifo request from 12345-0@lo, seq: 648 00000100:00100000:3.0:1776178612.069178:0:3356:0:(client.c:2215:ptlrpc_check_set()) Completed RPC req@00000000936848a2 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_03:lustre-MDT0001-mdtlov_UUID:3356:1862456998105792:0@lo:13:osp-pre-0-1.0 00010000:00010000:1.0:1776178612.706752:0:9224:0:(ldlm_lockd.c:2792:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:0.0:1776178613.537782:0:3348:0:(events.c:361:request_in_callback()) peer: 12345-192.168.204.3@tcp (source: 12345-192.168.204.3@tcp) 00000100:00100000:3.0:1776178613.537874:0:11281:0:(service.c:2152:ptlrpc_server_handle_req_in()) got req x1862456979134656 00000100:00100000:3.0:1776178613.537998:0:11281:0:(nrs_fifo.c:179:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.204.3@tcp, seq: 6944 00000100:00100000:3.0:1776178613.538005:0:11281:0:(service.c:2324:ptlrpc_server_handle_request()) Handling RPC req@00000000dac56e3f pname:cluuid+ref:pid:xid:nid:opc:job mdt00_004:0+-99:2269:x1862456979134656:12345-192.168.204.3@tcp:38:kworker/u8:3.0 00010000:02000000:3.0:1776178613.538023:0:11281:0:(ldlm_lib.c:1139:target_handle_connect()) lustre-MDT0000: Not available for connect from 192.168.204.3@tcp (stopping) 00000100:00100000:3.0:1776178613.538123:0:11281:0:(service.c:2372:ptlrpc_server_handle_request()) Handled RPC req@00000000dac56e3f pname:cluuid+ref:pid:xid:nid:opc:job mdt00_004:0+-99:2269:x1862456979134656:12345-192.168.204.3@tcp:38:kworker/u8:3.0 Request processed in 117us (351us total) trans 0 rc -19/-19 00000100:00100000:3.0:1776178613.538131:0:11281:0:(nrs_fifo.c:241:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.204.3@tcp, seq: 6944 00000100:00100000:2.0:1776178613.672406:0:3346:0:(events.c:361:request_in_callback()) peer: 12345-192.168.204.3@tcp (source: 12345-192.168.204.3@tcp) 00000100:00100000:2.0:1776178613.672461:0:3346:0:(events.c:361:request_in_callback()) peer: 12345-192.168.204.3@tcp (source: 12345-192.168.204.3@tcp) 00000100:00100000:2.0:1776178613.672494:0:3346:0:(events.c:361:request_in_callback()) peer: 12345-192.168.204.3@tcp (source: 12345-192.168.204.3@tcp) 00000100:00100000:2.0:1776178613.672531:0:3346:0:(events.c:361:request_in_callback()) peer: 12345-192.168.204.3@tcp (source: 12345-192.168.204.3@tcp) 00000100:00100000:2.0:1776178613.672570:0:20755:0:(service.c:2152:ptlrpc_server_handle_req_in()) got req x1862456979134784 00000100:00100000:2.0:1776178613.672617:0:20755:0:(service.c:2152:ptlrpc_server_handle_req_in()) got req x1862456979134848 00000100:00100000:2.0:1776178613.672641:0:20755:0:(nrs_fifo.c:179:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.204.3@tcp, seq: 831 00000100:00100000:2.0:1776178613.672648:0:20755:0:(service.c:2324:ptlrpc_server_handle_request()) Handling RPC req@0000000066eec801 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_006:b50963bd-8eba-4432-9a54-8888d00e3fa1+8:2271:x1862456979134784:12345-192.168.204.3@tcp:400:kworker/u8:3.0 00000100:00100000:2.0:1776178613.672715:0:20755:0:(service.c:2372:ptlrpc_server_handle_request()) Handled RPC req@0000000066eec801 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_006:b50963bd-8eba-4432-9a54-8888d00e3fa1+8:2271:x1862456979134784:12345-192.168.204.3@tcp:400:kworker/u8:3.0 Request processed in 67us (254us total) trans 0 rc 0/0 00000100:00100000:2.0:1776178613.672737:0:20755:0:(nrs_fifo.c:241:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.204.3@tcp, seq: 831 00000100:00100000:2.0:1776178613.672755:0:20755:0:(nrs_fifo.c:179:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.204.3@tcp, seq: 832 00000100:00100000:2.0:1776178613.672759:0:20755:0:(service.c:2324:ptlrpc_server_handle_request()) Handling RPC req@000000004f0c951d pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_006:b50963bd-8eba-4432-9a54-8888d00e3fa1+9:2271:x1862456979134848:12345-192.168.204.3@tcp:400:kworker/u8:3.0 00000100:00100000:2.0:1776178613.672783:0:20755:0:(service.c:2372:ptlrpc_server_handle_request()) Handled RPC req@000000004f0c951d pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_006:b50963bd-8eba-4432-9a54-8888d00e3fa1+9:2271:x1862456979134848:12345-192.168.204.3@tcp:400:kworker/u8:3.0 Request processed in 23us (252us total) trans 0 rc 0/0 00000100:00100000:2.0:1776178613.672793:0:20755:0:(nrs_fifo.c:241:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.204.3@tcp, seq: 832 00000100:00100000:1.0:1776178613.672960:0:11281:0:(service.c:2152:ptlrpc_server_handle_req_in()) got req x1862456979134720 00000100:00100000:1.0:1776178613.673006:0:11281:0:(nrs_fifo.c:179:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.204.3@tcp, seq: 1168 00000100:00100000:1.0:1776178613.673010:0:11281:0:(service.c:2324:ptlrpc_server_handle_request()) Handling RPC req@00000000fc9499fa pname:cluuid+ref:pid:xid:nid:opc:job mdt00_004:b50963bd-8eba-4432-9a54-8888d00e3fa1+6:2271:x1862456979134720:12345-192.168.204.3@tcp:400:kworker/u8:3.0 00000100:00100000:1.0:1776178613.673039:0:11281:0:(service.c:2372:ptlrpc_server_handle_request()) Handled RPC req@00000000fc9499fa pname:cluuid+ref:pid:xid:nid:opc:job mdt00_004:b50963bd-8eba-4432-9a54-8888d00e3fa1+6:2271:x1862456979134720:12345-192.168.204.3@tcp:400:kworker/u8:3.0 Request processed in 28us (546us total) trans 0 rc 0/0 00000100:00100000:1.0:1776178613.673045:0:11281:0:(nrs_fifo.c:241:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.204.3@tcp, seq: 1168 00000100:00100000:0.0:1776178613.675790:0:20140:0:(service.c:2152:ptlrpc_server_handle_req_in()) got req x1862456979134592 00000100:00100000:0.0:1776178613.675853:0:20140:0:(nrs_fifo.c:179:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.204.3@tcp, seq: 315 00000100:00100000:0.0:1776178613.675867:0:20140:0:(service.c:2324:ptlrpc_server_handle_request()) Handling RPC req@0000000044c7032a pname:cluuid+ref:pid:xid:nid:opc:job ll_mgs_0002:5b5c66b1-1a2f-4651-8548-0c75b9a38a94+9:2271:x1862456979134592:12345-192.168.204.3@tcp:400:kworker/u8:3.0 00000100:00100000:0.0:1776178613.675940:0:20140:0:(service.c:2372:ptlrpc_server_handle_request()) Handled RPC req@0000000044c7032a pname:cluuid+ref:pid:xid:nid:opc:job ll_mgs_0002:5b5c66b1-1a2f-4651-8548-0c75b9a38a94+9:2271:x1862456979134592:12345-192.168.204.3@tcp:400:kworker/u8:3.0 Request processed in 80us (3543us total) trans 0 rc 0/0 00000100:00100000:0.0:1776178613.675948:0:20140:0:(nrs_fifo.c:241:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.204.3@tcp, seq: 315 00010000:00010000:1.0:1776178613.728410:0:9225:0:(ldlm_lockd.c:2792:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00010000:00010000:2.0:1776178614.756812:0:6244:0:(ldlm_lockd.c:2792:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00080000:1.0:1776178615.138988:0:36:0:(pinger.c:251:ptlrpc_pinger_process_import()) lustre-MDT0000-lwp-OST0000_UUID->lustre-MDT0000_UUID: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00080000:1.0:1776178615.138999:0:36:0:(recover.c:57:ptlrpc_initiate_recovery()) lustre-MDT0000_UUID: starting recovery 00000100:00080000:1.0:1776178615.139003:0:36:0:(import.c:86:import_set_state_nolock()) 0000000047601a98 lustre-MDT0000_UUID: changing import state from DISCONN to CONNECTING 00000100:00080000:1.0:1776178615.139014:0:36:0:(import.c:534:import_select_connection()) lustre-MDT0000-lwp-OST0000: connect to NID 0@lo last attempt 1317 00000100:00080000:1.0:1776178615.139019:0:36:0:(import.c:581:import_select_connection()) lustre-MDT0000-lwp-OST0000: tried all connections, increasing latency to 21s 00000100:00080000:1.0:1776178615.139028:0:36:0:(import.c:615:import_select_connection()) lustre-MDT0000-lwp-OST0000: import 0000000047601a98 using connection 192.168.204.103@tcp/0@lo 00000100:00100000:1.0:1776178615.139054:0:36:0:(import.c:807:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5) req@000000000eda5b77 x1862456998105920/t0(0) o38->lustre-MDT0000-lwp-OST0000@0@lo:12/10 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' 00000100:00080000:1.0:1776178615.139126:0:36:0:(pinger.c:251:ptlrpc_pinger_process_import()) lustre-MDT0000-lwp-OST0001_UUID->lustre-MDT0000_UUID: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00080000:1.0:1776178615.139130:0:36:0:(recover.c:57:ptlrpc_initiate_recovery()) lustre-MDT0000_UUID: starting recovery 00000100:00080000:1.0:1776178615.139132:0:36:0:(import.c:86:import_set_state_nolock()) 00000000dfc8db13 lustre-MDT0000_UUID: changing import state from DISCONN to CONNECTING 00000100:00080000:1.0:1776178615.139135:0:36:0:(import.c:534:import_select_connection()) lustre-MDT0000-lwp-OST0001: connect to NID 0@lo last attempt 1317 00000100:00080000:1.0:1776178615.139137:0:36:0:(import.c:581:import_select_connection()) lustre-MDT0000-lwp-OST0001: tried all connections, increasing latency to 21s 00000100:00080000:1.0:1776178615.139141:0:36:0:(import.c:615:import_select_connection()) lustre-MDT0000-lwp-OST0001: import 00000000dfc8db13 using connection 192.168.204.103@tcp/0@lo 00000100:00100000:1.0:1776178615.139151:0:36:0:(import.c:807:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5) req@00000000e4ab0682 x1862456998106048/t0(0) o38->lustre-MDT0000-lwp-OST0001@0@lo:12/10 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' 00000100:00080000:1.0:1776178615.139176:0:36:0:(pinger.c:251:ptlrpc_pinger_process_import()) lustre-MDT0001-mdtlov_UUID->lustre-MDT0000_UUID: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00080000:1.0:1776178615.139179:0:36:0:(recover.c:57:ptlrpc_initiate_recovery()) lustre-MDT0000_UUID: starting recovery 00000100:00080000:1.0:1776178615.139180:0:36:0:(import.c:86:import_set_state_nolock()) 00000000ba4f1821 lustre-MDT0000_UUID: changing import state from DISCONN to CONNECTING 00000100:00080000:1.0:1776178615.139183:0:36:0:(import.c:534:import_select_connection()) lustre-MDT0000-osp-MDT0001: connect to NID 0@lo last attempt 1317 00000100:00080000:1.0:1776178615.139185:0:36:0:(import.c:581:import_select_connection()) lustre-MDT0000-osp-MDT0001: tried all connections, increasing latency to 21s 00000100:00080000:1.0:1776178615.139188:0:36:0:(import.c:615:import_select_connection()) lustre-MDT0000-osp-MDT0001: import 00000000ba4f1821 using connection 192.168.204.103@tcp/0@lo 00000100:00100000:1.0:1776178615.139199:0:36:0:(import.c:807:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5) req@00000000ce34953a x1862456998106176/t0(0) o38->lustre-MDT0000-osp-MDT0001@0@lo:24/4 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' 00000100:00080000:1.0:1776178615.139218:0:36:0:(pinger.c:251:ptlrpc_pinger_process_import()) lustre-MDT0000-lwp-MDT0001_UUID->lustre-MDT0000_UUID: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00080000:1.0:1776178615.139221:0:36:0:(recover.c:57:ptlrpc_initiate_recovery()) lustre-MDT0000_UUID: starting recovery 00000100:00080000:1.0:1776178615.139222:0:36:0:(import.c:86:import_set_state_nolock()) 00000000a7c55b1f lustre-MDT0000_UUID: changing import state from DISCONN to CONNECTING 00000100:00080000:1.0:1776178615.139225:0:36:0:(import.c:534:import_select_connection()) lustre-MDT0000-lwp-MDT0001: connect to NID 0@lo last attempt 1317 00000100:00080000:1.0:1776178615.139227:0:36:0:(import.c:581:import_select_connection()) lustre-MDT0000-lwp-MDT0001: tried all connections, increasing latency to 21s 00000100:00080000:1.0:1776178615.139230:0:36:0:(import.c:615:import_select_connection()) lustre-MDT0000-lwp-MDT0001: import 00000000a7c55b1f using connection 192.168.204.103@tcp/0@lo 00000100:00100000:1.0:1776178615.139240:0:36:0:(import.c:807:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5) req@0000000008dc7ab9 x1862456998106368/t0(0) o38->lustre-MDT0000-lwp-MDT0001@0@lo:12/10 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' 00000100:00100000:1.0:1776178615.139286:0:3355:0:(client.c:1733:ptlrpc_send_new_req()) Sending RPC req@000000006c0c7de5 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_02:lustre-MDT0001-lwp-OST0001_UUID:3355:1862456998106112:0@lo:400:kworker/u8:1.0 00000100:00100000:1.0:1776178615.139354:0:3355:0:(events.c:361:request_in_callback()) peer: 12345-0@lo (source: 12345-0@lo) 00000100:00100000:1.0:1776178615.139386:0:3355:0:(ptlrpcd.c:415:ptlrpcd_check()) transfer 1 async RPCs [3->2] 00000100:00100000:1.0:1776178615.139392:0:3355:0:(client.c:1733:ptlrpc_send_new_req()) Sending RPC req@000000002c128264 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_02:lustre-MDT0001-mdtlov_UUID:3355:1862456998106240:0@lo:400:kworker/u8:1.0 00000100:00100000:1.0:1776178615.139419:0:3355:0:(events.c:361:request_in_callback()) peer: 12345-0@lo (source: 12345-0@lo) 00000100:00100000:0.0:1776178615.139586:0:3352:0:(client.c:1733:ptlrpc_send_new_req()) Sending RPC req@000000000eda5b77 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:lustre-MDT0000-lwp-OST0000_UUID:3352:1862456998105920:0@lo:38:kworker/u8:1.0 00000100:00100000:0.0:1776178615.139615:0:3352:0:(events.c:361:request_in_callback()) peer: 12345-0@lo (source: 12345-0@lo) 00000100:00100000:0.0:1776178615.139637:0:3352:0:(client.c:1733:ptlrpc_send_new_req()) Sending RPC req@00000000e4ab0682 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:lustre-MDT0000-lwp-OST0001_UUID:3352:1862456998106048:0@lo:38:kworker/u8:1.0 00000100:00100000:0.0:1776178615.139659:0:3352:0:(events.c:361:request_in_callback()) peer: 12345-0@lo (source: 12345-0@lo) 00000100:00100000:0.0:1776178615.139671:0:3352:0:(client.c:1733:ptlrpc_send_new_req()) Sending RPC req@00000000ce34953a pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:lustre-MDT0001-mdtlov_UUID:3352:1862456998106176:0@lo:38:kworker/u8:1.0 00000100:00100000:0.0:1776178615.139703:0:3352:0:(events.c:361:request_in_callback()) peer: 12345-0@lo (source: 12345-0@lo) 00000100:00100000:0.0:1776178615.139716:0:3352:0:(client.c:1733:ptlrpc_send_new_req()) Sending RPC req@0000000008dc7ab9 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:lustre-MDT0000-lwp-MDT0001_UUID:3352:1862456998106368:0@lo:38:kworker/u8:1.0 00000100:00100000:0.0:1776178615.139734:0:3352:0:(events.c:361:request_in_callback()) peer: 12345-0@lo (source: 12345-0@lo) 00000100:00100000:2.0:1776178615.139949:0:3354:0:(client.c:1733:ptlrpc_send_new_req()) Sending RPC req@00000000c989e78f pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_01:lustre-MDT0001-lwp-OST0000_UUID:3354:1862456998105984:0@lo:400:kworker/u8:1.0 00000100:00100000:2.0:1776178615.139978:0:3354:0:(events.c:361:request_in_callback()) peer: 12345-0@lo (source: 12345-0@lo) 00000100:00100000:2.0:1776178615.140005:0:3354:0:(ptlrpcd.c:415:ptlrpcd_check()) transfer 2 async RPCs [0->1] 00000100:00100000:2.0:1776178615.140010:0:3354:0:(client.c:1733:ptlrpc_send_new_req()) Sending RPC req@00000000cc329471 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_01:ac61f839-0e8e-4cac-9c59-08bcfbbad330:3354:1862456998105856:0@lo:400:kworker/u8:1.0 00000100:00100000:2.0:1776178615.140039:0:3354:0:(events.c:361:request_in_callback()) peer: 12345-0@lo (source: 12345-0@lo) 00000100:00100000:2.0:1776178615.140055:0:3354:0:(client.c:1733:ptlrpc_send_new_req()) Sending RPC req@000000006159eedd pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_01:lustre-MDT0001-mdtlov_UUID:3354:1862456998106304:0@lo:400:kworker/u8:1.0 00000100:00100000:2.0:1776178615.140105:0:3354:0:(events.c:361:request_in_callback()) peer: 12345-0@lo (source: 12345-0@lo) 00000100:00100000:2.0:1776178615.140143:0:8250:0:(service.c:2152:ptlrpc_server_handle_req_in()) got req x1862456998106112 00000100:00100000:2.0:1776178615.140174:0:8250:0:(service.c:2152:ptlrpc_server_handle_req_in()) got req x1862456998105920 00000100:00100000:2.0:1776178615.140181:0:8250:0:(service.c:2152:ptlrpc_server_handle_req_in()) got req x1862456998106048 00000100:00100000:2.0:1776178615.140188:0:8250:0:(service.c:2152:ptlrpc_server_handle_req_in()) got req x1862456998106368 00000100:00100000:2.0:1776178615.140197:0:8250:0:(service.c:2152:ptlrpc_server_handle_req_in()) got req x1862456998105984 00000100:00100000:2.0:1776178615.140208:0:8250:0:(nrs_fifo.c:179:nrs_fifo_req_get()) NRS start fifo request from 12345-0@lo, seq: 1169 00000100:00100000:2.0:1776178615.140213:0:8250:0:(service.c:2324:ptlrpc_server_handle_request()) Handling RPC req@00000000c598e4ab pname:cluuid+ref:pid:xid:nid:opc:job mdt00_003:lustre-MDT0001-lwp-OST0001_UUID+5:3355:x1862456998106112:12345-0@lo:400:kworker/u8:1.0 00000100:00100000:2.0:1776178615.140258:0:8250:0:(service.c:2372:ptlrpc_server_handle_request()) Handled RPC req@00000000c598e4ab pname:cluuid+ref:pid:xid:nid:opc:job mdt00_003:lustre-MDT0001-lwp-OST0001_UUID+5:3355:x1862456998106112:12345-0@lo:400:kworker/u8:1.0 Request processed in 45us (905us total) trans 0 rc 0/0 00000100:00100000:2.0:1776178615.140277:0:8250:0:(nrs_fifo.c:241:nrs_fifo_req_stop()) NRS stop fifo request from 12345-0@lo, seq: 1169 00000100:00100000:2.0:1776178615.140292:0:8250:0:(nrs_fifo.c:179:nrs_fifo_req_get()) NRS start fifo request from 12345-0@lo, seq: 1170 00000100:00100000:2.0:1776178615.140295:0:8250:0:(service.c:2324:ptlrpc_server_handle_request()) Handling RPC req@00000000093435de pname:cluuid+ref:pid:xid:nid:opc:job mdt00_003:lustre-MDT0001-lwp-OST0000_UUID+5:3354:x1862456998105984:12345-0@lo:400:kworker/u8:1.0 00000100:00100000:2.0:1776178615.140321:0:8250:0:(service.c:2372:ptlrpc_server_handle_request()) Handled RPC req@00000000093435de pname:cluuid+ref:pid:xid:nid:opc:job mdt00_003:lustre-MDT0001-lwp-OST0000_UUID+5:3354:x1862456998105984:12345-0@lo:400:kworker/u8:1.0 Request processed in 25us (344us total) trans 0 rc 0/0 00000100:00100000:2.0:1776178615.140326:0:8250:0:(nrs_fifo.c:241:nrs_fifo_req_stop()) NRS stop fifo request from 12345-0@lo, seq: 1170 00000100:00100000:2.0:1776178615.140336:0:8250:0:(nrs_fifo.c:179:nrs_fifo_req_get()) NRS start fifo request from 12345-0@lo, seq: 6945 00000100:00100000:2.0:1776178615.140340:0:8250:0:(service.c:2324:ptlrpc_server_handle_request()) Handling RPC req@00000000d1597b46 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_003:0+-99:3352:x1862456998105920:12345-0@lo:38:kworker/u8:1.0 00010000:02000000:2.0:1776178615.140352:0:8250:0:(ldlm_lib.c:1139:target_handle_connect()) lustre-MDT0000: Not available for connect from 0@lo (stopping) 00000100:00100000:2.0:1776178615.140376:0:8250:0:(service.c:2372:ptlrpc_server_handle_request()) Handled RPC req@00000000d1597b46 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_003:0+-99:3352:x1862456998105920:12345-0@lo:38:kworker/u8:1.0 Request processed in 35us (761us total) trans 0 rc -19/-19 00000100:00100000:2.0:1776178615.140381:0:8250:0:(nrs_fifo.c:241:nrs_fifo_req_stop()) NRS stop fifo request from 12345-0@lo, seq: 6945 00000100:00100000:2.0:1776178615.140390:0:8250:0:(nrs_fifo.c:179:nrs_fifo_req_get()) NRS start fifo request from 12345-0@lo, seq: 6946 00000100:00100000:2.0:1776178615.140393:0:8250:0:(service.c:2324:ptlrpc_server_handle_request()) Handling RPC req@000000000ec316da pname:cluuid+ref:pid:xid:nid:opc:job mdt00_003:0+-99:3352:x1862456998106048:12345-0@lo:38:kworker/u8:1.0 00010000:02000000:2.0:1776178615.140399:0:8250:0:(ldlm_lib.c:1139:target_handle_connect()) lustre-MDT0000: Not available for connect from 0@lo (stopping) 00000100:00100000:2.0:1776178615.140419:0:8250:0:(service.c:2372:ptlrpc_server_handle_request()) Handled RPC req@000000000ec316da pname:cluuid+ref:pid:xid:nid:opc:job mdt00_003:0+-99:3352:x1862456998106048:12345-0@lo:38:kworker/u8:1.0 Request processed in 24us (759us total) trans 0 rc -19/-19 00000100:00100000:2.0:1776178615.140424:0:8250:0:(nrs_fifo.c:241:nrs_fifo_req_stop()) NRS stop fifo request from 12345-0@lo, seq: 6946 00000100:00100000:2.0:1776178615.141578:0:8250:0:(nrs_fifo.c:179:nrs_fifo_req_get()) NRS start fifo request from 12345-0@lo, seq: 6947 00000100:00100000:2.0:1776178615.141585:0:8250:0:(service.c:2324:ptlrpc_server_handle_request()) Handling RPC req@00000000fccd82c9 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_003:0+-99:3352:x1862456998106368:12345-0@lo:38:kworker/u8:1.0 00010000:02000000:2.0:1776178615.141596:0:8250:0:(ldlm_lib.c:1139:target_handle_connect()) lustre-MDT0000: Not available for connect from 0@lo (stopping) 00000100:00100000:2.0:1776178615.141633:0:8250:0:(service.c:2372:ptlrpc_server_handle_request()) Handled RPC req@00000000fccd82c9 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_003:0+-99:3352:x1862456998106368:12345-0@lo:38:kworker/u8:1.0 Request processed in 47us (1898us total) trans 0 rc -19/-19 00000100:00100000:2.0:1776178615.141639:0:8250:0:(nrs_fifo.c:241:nrs_fifo_req_stop()) NRS stop fifo request from 12345-0@lo, seq: 6947 00000100:00100000:2.0:1776178615.141668:0:14128:0:(service.c:2152:ptlrpc_server_handle_req_in()) got req x1862456998106176 00000100:00100000:2.0:1776178615.141695:0:14128:0:(nrs_fifo.c:179:nrs_fifo_req_get()) NRS start fifo request from 12345-0@lo, seq: 788 00000100:00100000:2.0:1776178615.141699:0:14128:0:(service.c:2324:ptlrpc_server_handle_request()) Handling RPC req@00000000950bb9b2 pname:cluuid+ref:pid:xid:nid:opc:job mdt_out00_002:0+-99:3352:x1862456998106176:12345-0@lo:38:kworker/u8:1.0 00010000:02000000:2.0:1776178615.141706:0:14128:0:(ldlm_lib.c:1139:target_handle_connect()) lustre-MDT0000: Not available for connect from 0@lo (stopping) 00000100:00100000:2.0:1776178615.141729:0:14128:0:(service.c:2372:ptlrpc_server_handle_request()) Handled RPC req@00000000950bb9b2 pname:cluuid+ref:pid:xid:nid:opc:job mdt_out00_002:0+-99:3352:x1862456998106176:12345-0@lo:38:kworker/u8:1.0 Request processed in 30us (2026us total) trans 0 rc -19/-19 00000100:00100000:2.0:1776178615.141734:0:14128:0:(nrs_fifo.c:241:nrs_fifo_req_stop()) NRS stop fifo request from 12345-0@lo, seq: 788 00000100:00100000:2.0:1776178615.141760:0:20759:0:(service.c:2152:ptlrpc_server_handle_req_in()) got req x1862456998106240 00000100:00100000:2.0:1776178615.141784:0:20759:0:(service.c:2152:ptlrpc_server_handle_req_in()) got req x1862456998106304 00000100:00100000:2.0:1776178615.141800:0:20759:0:(nrs_fifo.c:179:nrs_fifo_req_get()) NRS start fifo request from 12345-0@lo, seq: 833 00000100:00100000:2.0:1776178615.141804:0:20759:0:(service.c:2324:ptlrpc_server_handle_request()) Handling RPC req@000000007cfd1c43 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_010:lustre-MDT0001-mdtlov_UUID+5:3355:x1862456998106240:12345-0@lo:400:kworker/u8:1.0 00000100:00100000:2.0:1776178615.141839:0:20759:0:(service.c:2372:ptlrpc_server_handle_request()) Handled RPC req@000000007cfd1c43 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_010:lustre-MDT0001-mdtlov_UUID+5:3355:x1862456998106240:12345-0@lo:400:kworker/u8:1.0 Request processed in 34us (2420us total) trans 0 rc 0/0 00000100:00100000:2.0:1776178615.141845:0:20759:0:(nrs_fifo.c:241:nrs_fifo_req_stop()) NRS stop fifo request from 12345-0@lo, seq: 833 00000100:00100000:2.0:1776178615.141856:0:20759:0:(nrs_fifo.c:179:nrs_fifo_req_get()) NRS start fifo request from 12345-0@lo, seq: 834 00000100:00100000:2.0:1776178615.141859:0:20759:0:(service.c:2324:ptlrpc_server_handle_request()) Handling RPC req@0000000010282d62 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_010:lustre-MDT0001-mdtlov_UUID+5:3354:x1862456998106304:12345-0@lo:400:kworker/u8:1.0 00000100:00100000:2.0:1776178615.141897:0:20759:0:(service.c:2372:ptlrpc_server_handle_request()) Handled RPC req@0000000010282d62 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_010:lustre-MDT0001-mdtlov_UUID+5:3354:x1862456998106304:12345-0@lo:400:kworker/u8:1.0 Request processed in 37us (1792us total) trans 0 rc 0/0 00000100:00100000:2.0:1776178615.141903:0:20759:0:(nrs_fifo.c:241:nrs_fifo_req_stop()) NRS stop fifo request from 12345-0@lo, seq: 834 00000100:00100000:2.0:1776178615.141925:0:20137:0:(service.c:2152:ptlrpc_server_handle_req_in()) got req x1862456998105856 00000100:00100000:2.0:1776178615.141947:0:20137:0:(nrs_fifo.c:179:nrs_fifo_req_get()) NRS start fifo request from 12345-0@lo, seq: 316 00000100:00100000:2.0:1776178615.141951:0:20137:0:(service.c:2324:ptlrpc_server_handle_request()) Handling RPC req@00000000c6a76119 pname:cluuid+ref:pid:xid:nid:opc:job ll_mgs_0000:ac61f839-0e8e-4cac-9c59-08bcfbbad330+18:3354:x1862456998105856:12345-0@lo:400:kworker/u8:1.0 00000100:00100000:2.0:1776178615.141976:0:20137:0:(service.c:2372:ptlrpc_server_handle_request()) Handled RPC req@00000000c6a76119 pname:cluuid+ref:pid:xid:nid:opc:job ll_mgs_0000:ac61f839-0e8e-4cac-9c59-08bcfbbad330+18:3354:x1862456998105856:12345-0@lo:400:kworker/u8:1.0 Request processed in 25us (1937us total) trans 0 rc 0/0 00000100:00100000:2.0:1776178615.141982:0:20137:0:(nrs_fifo.c:241:nrs_fifo_req_stop()) NRS stop fifo request from 12345-0@lo, seq: 316 00000100:00100000:2.0:1776178615.142025:0:3354:0:(client.c:2215:ptlrpc_check_set()) Completed RPC req@00000000cc329471 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_01:ac61f839-0e8e-4cac-9c59-08bcfbbad330:3354:1862456998105856:0@lo:400:kworker/u8:1.0 00000100:00100000:2.0:1776178615.142038:0:3354:0:(client.c:2215:ptlrpc_check_set()) Completed RPC req@000000006159eedd pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_01:lustre-MDT0001-mdtlov_UUID:3354:1862456998106304:0@lo:400:kworker/u8:1.0 00000100:00100000:2.0:1776178615.142048:0:3354:0:(client.c:2215:ptlrpc_check_set()) Completed RPC req@00000000c989e78f pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_01:lustre-MDT0001-lwp-OST0000_UUID:3354:1862456998105984:0@lo:400:kworker/u8:1.0 00000100:00100000:2.0:1776178615.142109:0:3355:0:(client.c:2215:ptlrpc_check_set()) Completed RPC req@000000002c128264 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_02:lustre-MDT0001-mdtlov_UUID:3355:1862456998106240:0@lo:400:kworker/u8:1.0 00000100:00100000:2.0:1776178615.142119:0:3355:0:(client.c:2215:ptlrpc_check_set()) Completed RPC req@000000006c0c7de5 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_02:lustre-MDT0001-lwp-OST0001_UUID:3355:1862456998106112:0@lo:400:kworker/u8:1.0 00000100:00080000:0.0:1776178615.142898:0:3352:0:(import.c:86:import_set_state_nolock()) 0000000047601a98 lustre-MDT0000_UUID: changing import state from CONNECTING to DISCONN 00000100:00080000:0.0:1776178615.142904:0:3352:0:(import.c:1429:ptlrpc_connect_interpret()) recovery of lustre-MDT0000_UUID on 192.168.204.103@tcp failed (-19) 00000100:00100000:0.0:1776178615.142910:0:3352:0:(client.c:2215:ptlrpc_check_set()) Completed RPC req@000000000eda5b77 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:lustre-MDT0000-lwp-OST0000_UUID:3352:1862456998105920:0@lo:38:kworker/u8:1.0 00000100:00080000:0.0:1776178615.142918:0:3352:0:(import.c:86:import_set_state_nolock()) 00000000dfc8db13 lustre-MDT0000_UUID: changing import state from CONNECTING to DISCONN 00000100:00080000:0.0:1776178615.142920:0:3352:0:(import.c:1429:ptlrpc_connect_interpret()) recovery of lustre-MDT0000_UUID on 192.168.204.103@tcp failed (-19) 00000100:00100000:0.0:1776178615.142922:0:3352:0:(client.c:2215:ptlrpc_check_set()) Completed RPC req@00000000e4ab0682 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:lustre-MDT0000-lwp-OST0001_UUID:3352:1862456998106048:0@lo:38:kworker/u8:1.0 00000100:00080000:0.0:1776178615.142930:0:3352:0:(import.c:86:import_set_state_nolock()) 00000000ba4f1821 lustre-MDT0000_UUID: changing import state from CONNECTING to DISCONN 00000100:00080000:0.0:1776178615.142932:0:3352:0:(import.c:1429:ptlrpc_connect_interpret()) recovery of lustre-MDT0000_UUID on 192.168.204.103@tcp failed (-19) 00000100:00100000:0.0:1776178615.142935:0:3352:0:(client.c:2215:ptlrpc_check_set()) Completed RPC req@00000000ce34953a pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:lustre-MDT0001-mdtlov_UUID:3352:1862456998106176:0@lo:38:kworker/u8:1.0 00000100:00080000:0.0:1776178615.142942:0:3352:0:(import.c:86:import_set_state_nolock()) 00000000a7c55b1f lustre-MDT0000_UUID: changing import state from CONNECTING to DISCONN 00000100:00080000:0.0:1776178615.142945:0:3352:0:(import.c:1429:ptlrpc_connect_interpret()) recovery of lustre-MDT0000_UUID on 192.168.204.103@tcp failed (-19) 00000100:00100000:0.0:1776178615.142947:0:3352:0:(client.c:2215:ptlrpc_check_set()) Completed RPC req@0000000008dc7ab9 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:lustre-MDT0000-lwp-MDT0001_UUID:3352:1862456998106368:0@lo:38:kworker/u8:1.0 00010000:00010000:1.0:1776178615.794096:0:6245:0:(ldlm_lockd.c:2792:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:2.0:1776178616.672388:0:3354:0:(client.c:1733:ptlrpc_send_new_req()) Sending RPC req@0000000098623c2d pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_01:lustre-MDT0001-mdtlov_UUID:3354:1862456998106432:0@lo:13:osp-pre-1-1.0 00000100:00100000:2.0:1776178616.672462:0:3354:0:(events.c:361:request_in_callback()) peer: 12345-0@lo (source: 12345-0@lo) 00000100:00100000:1.0:1776178616.675226:0:9554:0:(service.c:2152:ptlrpc_server_handle_req_in()) got req x1862456998106432 00000100:00100000:1.0:1776178616.675303:0:9554:0:(nrs_fifo.c:179:nrs_fifo_req_get()) NRS start fifo request from 12345-0@lo, seq: 649 00000100:00100000:1.0:1776178616.675311:0:9554:0:(service.c:2324:ptlrpc_server_handle_request()) Handling RPC req@000000006b092368 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_create00:lustre-MDT0001-mdtlov_UUID+5:3354:x1862456998106432:12345-0@lo:13:osp-pre-1-1.0 00002000:00000024:1.0:1776178616.675348:0:9554:0:(ofd_obd.c:740:ofd_statfs()) blocks cached 229376 granted 491746368 pending 0 free 3923689472 avail 3692167168 00000100:00100000:1.0:1776178616.675430:0:9554:0:(service.c:2372:ptlrpc_server_handle_request()) Handled RPC req@000000006b092368 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_create00:lustre-MDT0001-mdtlov_UUID+5:3354:x1862456998106432:12345-0@lo:13:osp-pre-1-1.0 Request processed in 118us (2968us total) trans 0 rc 0/0 00000100:00100000:1.0:1776178616.675437:0:9554:0:(nrs_fifo.c:241:nrs_fifo_req_stop()) NRS stop fifo request from 12345-0@lo, seq: 649 00000100:00100000:2.0:1776178616.675772:0:3354:0:(client.c:2215:ptlrpc_check_set()) Completed RPC req@0000000098623c2d pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_01:lustre-MDT0001-mdtlov_UUID:3354:1862456998106432:0@lo:13:osp-pre-1-1.0 00010000:00010000:1.0:1776178616.802130:0:9224:0:(ldlm_lockd.c:2792:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:3.0:1776178617.185713:0:3356:0:(ptlrpcd.c:415:ptlrpcd_check()) transfer 1 async RPCs [2->3] 00000100:00100000:3.0:1776178617.185737:0:3356:0:(client.c:1733:ptlrpc_send_new_req()) Sending RPC req@00000000b6e1d5c6 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_03:lustre-MDT0001-mdtlov_UUID:3356:1862456998106496:0@lo:13:osp-pre-0-1.0 00000100:00100000:3.0:1776178617.185814:0:3356:0:(events.c:361:request_in_callback()) peer: 12345-0@lo (source: 12345-0@lo) 00000100:00100000:0.0:1776178617.186117:0:9554:0:(service.c:2152:ptlrpc_server_handle_req_in()) got req x1862456998106496 00000100:00100000:0.0:1776178617.186461:0:9554:0:(nrs_fifo.c:179:nrs_fifo_req_get()) NRS start fifo request from 12345-0@lo, seq: 650 00000100:00100000:0.0:1776178617.186468:0:9554:0:(service.c:2324:ptlrpc_server_handle_request()) Handling RPC req@000000003b0b9a0e pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_create00:lustre-MDT0001-mdtlov_UUID+5:3356:x1862456998106496:12345-0@lo:13:osp-pre-0-1.0 00002000:00000024:0.0:1776178617.186505:0:9554:0:(ofd_obd.c:740:ofd_statfs()) blocks cached 315392 granted 484883776 pending 0 free 3923689472 avail 3692167168 00000100:00100000:0.0:1776178617.186572:0:9554:0:(service.c:2372:ptlrpc_server_handle_request()) Handled RPC req@000000003b0b9a0e pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_create00:lustre-MDT0001-mdtlov_UUID+5:3356:x1862456998106496:12345-0@lo:13:osp-pre-0-1.0 Request processed in 103us (759us total) trans 0 rc 0/0 00000100:00100000:0.0:1776178617.186579:0:9554:0:(nrs_fifo.c:241:nrs_fifo_req_stop()) NRS stop fifo request from 12345-0@lo, seq: 650 00000100:00100000:3.0:1776178617.186770:0:3356:0:(client.c:2215:ptlrpc_check_set()) Completed RPC req@00000000b6e1d5c6 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_03:lustre-MDT0001-mdtlov_UUID:3356:1862456998106496:0@lo:13:osp-pre-0-1.0 00010000:00010000:1.0:1776178617.825872:0:9225:0:(ldlm_lockd.c:2792:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:0.0:1776178618.685333:0:3347:0:(events.c:361:request_in_callback()) peer: 12345-192.168.204.3@tcp (source: 12345-192.168.204.3@tcp) 00000100:00100000:0.0:1776178618.685432:0:3347:0:(events.c:361:request_in_callback()) peer: 12345-192.168.204.3@tcp (source: 12345-192.168.204.3@tcp) 00000100:00100000:0.0:1776178618.685474:0:3347:0:(events.c:361:request_in_callback()) peer: 12345-192.168.204.3@tcp (source: 12345-192.168.204.3@tcp) 00000100:00100000:0.0:1776178618.685507:0:3347:0:(events.c:361:request_in_callback()) peer: 12345-192.168.204.3@tcp (source: 12345-192.168.204.3@tcp) 00000100:00100000:0.0:1776178618.685538:0:3347:0:(events.c:361:request_in_callback()) peer: 12345-192.168.204.3@tcp (source: 12345-192.168.204.3@tcp) 00000100:00100000:0.0:1776178618.685583:0:6255:0:(service.c:2152:ptlrpc_server_handle_req_in()) got req x1862456979134976 00000100:00100000:0.0:1776178618.685620:0:6255:0:(service.c:2152:ptlrpc_server_handle_req_in()) got req x1862456979135040 00000100:00100000:0.0:1776178618.685648:0:6255:0:(nrs_fifo.c:179:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.204.3@tcp, seq: 1171 00000100:00100000:0.0:1776178618.685656:0:6255:0:(service.c:2324:ptlrpc_server_handle_request()) Handling RPC req@00000000afa36b49 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_000:b50963bd-8eba-4432-9a54-8888d00e3fa1+6:2272:x1862456979135040:12345-192.168.204.3@tcp:400:kworker/u8:4.0 00000100:00100000:0.0:1776178618.685732:0:6255:0:(service.c:2372:ptlrpc_server_handle_request()) Handled RPC req@00000000afa36b49 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_000:b50963bd-8eba-4432-9a54-8888d00e3fa1+6:2272:x1862456979135040:12345-192.168.204.3@tcp:400:kworker/u8:4.0 Request processed in 77us (225us total) trans 0 rc 0/0 00000100:00100000:0.0:1776178618.685740:0:6255:0:(nrs_fifo.c:241:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.204.3@tcp, seq: 1171 00000100:00100000:0.0:1776178618.685758:0:6255:0:(nrs_fifo.c:179:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.204.3@tcp, seq: 6948 00000100:00100000:0.0:1776178618.685763:0:6255:0:(service.c:2324:ptlrpc_server_handle_request()) Handling RPC req@00000000dd5bd4cd pname:cluuid+ref:pid:xid:nid:opc:job mdt00_000:0+-99:2269:x1862456979134976:12345-192.168.204.3@tcp:38:kworker/u8:4.0 00010000:02000000:0.0:1776178618.685776:0:6255:0:(ldlm_lib.c:1139:target_handle_connect()) lustre-MDT0000: Not available for connect from 192.168.204.3@tcp (stopping) 00000100:00100000:0.0:1776178618.685816:0:6255:0:(service.c:2372:ptlrpc_server_handle_request()) Handled RPC req@00000000dd5bd4cd pname:cluuid+ref:pid:xid:nid:opc:job mdt00_000:0+-99:2269:x1862456979134976:12345-192.168.204.3@tcp:38:kworker/u8:4.0 Request processed in 52us (493us total) trans 0 rc -19/-19 00000100:00100000:0.0:1776178618.685823:0:6255:0:(nrs_fifo.c:241:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.204.3@tcp, seq: 6948 00000100:00100000:0.0:1776178618.685842:0:20755:0:(service.c:2152:ptlrpc_server_handle_req_in()) got req x1862456979135104 00000100:00100000:0.0:1776178618.685866:0:20755:0:(service.c:2152:ptlrpc_server_handle_req_in()) got req x1862456979135168 00000100:00100000:0.0:1776178618.685883:0:20755:0:(nrs_fifo.c:179:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.204.3@tcp, seq: 835 00000100:00100000:0.0:1776178618.685887:0:20755:0:(service.c:2324:ptlrpc_server_handle_request()) Handling RPC req@00000000f247d7cd pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_006:b50963bd-8eba-4432-9a54-8888d00e3fa1+8:2272:x1862456979135104:12345-192.168.204.3@tcp:400:kworker/u8:4.0 00000100:00100000:0.0:1776178618.685914:0:20755:0:(service.c:2372:ptlrpc_server_handle_request()) Handled RPC req@00000000f247d7cd pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_006:b50963bd-8eba-4432-9a54-8888d00e3fa1+8:2272:x1862456979135104:12345-192.168.204.3@tcp:400:kworker/u8:4.0 Request processed in 26us (441us total) trans 0 rc 0/0 00000100:00100000:0.0:1776178618.685921:0:20755:0:(nrs_fifo.c:241:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.204.3@tcp, seq: 835 00000100:00100000:0.0:1776178618.685932:0:20755:0:(nrs_fifo.c:179:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.204.3@tcp, seq: 836 00000100:00100000:0.0:1776178618.685935:0:20755:0:(service.c:2324:ptlrpc_server_handle_request()) Handling RPC req@0000000086df2fc5 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_006:b50963bd-8eba-4432-9a54-8888d00e3fa1+9:2272:x1862456979135168:12345-192.168.204.3@tcp:400:kworker/u8:4.0 00000100:00100000:0.0:1776178618.685958:0:20755:0:(service.c:2372:ptlrpc_server_handle_request()) Handled RPC req@0000000086df2fc5 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_006:b50963bd-8eba-4432-9a54-8888d00e3fa1+9:2272:x1862456979135168:12345-192.168.204.3@tcp:400:kworker/u8:4.0 Request processed in 21us (419us total) trans 0 rc 0/0 00000100:00100000:0.0:1776178618.685965:0:20755:0:(nrs_fifo.c:241:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.204.3@tcp, seq: 836 00000100:00100000:2.0:1776178618.686188:0:20140:0:(service.c:2152:ptlrpc_server_handle_req_in()) got req x1862456979134912 00000100:00100000:2.0:1776178618.686231:0:20140:0:(nrs_fifo.c:179:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.204.3@tcp, seq: 317 00000100:00100000:2.0:1776178618.686235:0:20140:0:(service.c:2324:ptlrpc_server_handle_request()) Handling RPC req@00000000a2768ffd pname:cluuid+ref:pid:xid:nid:opc:job ll_mgs_0002:5b5c66b1-1a2f-4651-8548-0c75b9a38a94+9:2272:x1862456979134912:12345-192.168.204.3@tcp:400:kworker/u8:4.0 00000100:00100000:2.0:1776178618.686268:0:20140:0:(service.c:2372:ptlrpc_server_handle_request()) Handled RPC req@00000000a2768ffd pname:cluuid+ref:pid:xid:nid:opc:job ll_mgs_0002:5b5c66b1-1a2f-4651-8548-0c75b9a38a94+9:2272:x1862456979134912:12345-192.168.204.3@tcp:400:kworker/u8:4.0 Request processed in 32us (836us total) trans 0 rc 0/0 00000100:00100000:2.0:1776178618.686274:0:20140:0:(nrs_fifo.c:241:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.204.3@tcp, seq: 317 00010000:00010000:2.0:1776178618.850591:0:6244:0:(ldlm_lockd.c:2792:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00010000:00010000:1.0:1776178619.872647:0:6245:0:(ldlm_lockd.c:2792:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00080000:2.0:1776178620.256884:0:36:0:(pinger.c:251:ptlrpc_pinger_process_import()) lustre-MDT0000-lwp-OST0000_UUID->lustre-MDT0000_UUID: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00080000:2.0:1776178620.256892:0:36:0:(recover.c:57:ptlrpc_initiate_recovery()) lustre-MDT0000_UUID: starting recovery 00000100:00080000:2.0:1776178620.256895:0:36:0:(import.c:86:import_set_state_nolock()) 0000000047601a98 lustre-MDT0000_UUID: changing import state from DISCONN to CONNECTING 00000100:00080000:2.0:1776178620.256904:0:36:0:(import.c:534:import_select_connection()) lustre-MDT0000-lwp-OST0000: connect to NID 0@lo last attempt 1322 00000100:00080000:2.0:1776178620.256907:0:36:0:(import.c:581:import_select_connection()) lustre-MDT0000-lwp-OST0000: tried all connections, increasing latency to 21s 00000100:00080000:2.0:1776178620.256914:0:36:0:(import.c:615:import_select_connection()) lustre-MDT0000-lwp-OST0000: import 0000000047601a98 using connection 192.168.204.103@tcp/0@lo 00000100:00100000:2.0:1776178620.256936:0:36:0:(import.c:807:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5) req@00000000c6b32ddb x1862456998106624/t0(0) o38->lustre-MDT0000-lwp-OST0000@0@lo:12/10 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' 00000100:00100000:1.0:1776178620.256977:0:3355:0:(ptlrpcd.c:415:ptlrpcd_check()) transfer 1 async RPCs [3->2] 00000100:00080000:2.0:1776178620.256980:0:36:0:(pinger.c:251:ptlrpc_pinger_process_import()) lustre-MDT0000-lwp-OST0001_UUID->lustre-MDT0000_UUID: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00080000:2.0:1776178620.256985:0:36:0:(recover.c:57:ptlrpc_initiate_recovery()) lustre-MDT0000_UUID: starting recovery 00000100:00080000:2.0:1776178620.256986:0:36:0:(import.c:86:import_set_state_nolock()) 00000000dfc8db13 lustre-MDT0000_UUID: changing import state from DISCONN to CONNECTING 00000100:00080000:2.0:1776178620.256990:0:36:0:(import.c:534:import_select_connection()) lustre-MDT0000-lwp-OST0001: connect to NID 0@lo last attempt 1322 00000100:00080000:2.0:1776178620.256993:0:36:0:(import.c:581:import_select_connection()) lustre-MDT0000-lwp-OST0001: tried all connections, increasing latency to 21s 00000100:00080000:2.0:1776178620.256997:0:36:0:(import.c:615:import_select_connection()) lustre-MDT0000-lwp-OST0001: import 00000000dfc8db13 using connection 192.168.204.103@tcp/0@lo 00000100:00100000:1.0:1776178620.257002:0:3355:0:(client.c:1733:ptlrpc_send_new_req()) Sending RPC req@0000000065f62d8c pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_02:ac61f839-0e8e-4cac-9c59-08bcfbbad330:3355:1862456998106560:0@lo:400:kworker/u8:1.0 00000100:00100000:2.0:1776178620.257009:0:36:0:(import.c:807:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5) req@00000000606ce17f x1862456998106752/t0(0) o38->lustre-MDT0000-lwp-OST0001@0@lo:12/10 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' 00000100:00080000:2.0:1776178620.257038:0:36:0:(pinger.c:251:ptlrpc_pinger_process_import()) lustre-MDT0001-mdtlov_UUID->lustre-MDT0000_UUID: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00080000:2.0:1776178620.257043:0:36:0:(recover.c:57:ptlrpc_initiate_recovery()) lustre-MDT0000_UUID: starting recovery 00000100:00080000:2.0:1776178620.257045:0:36:0:(import.c:86:import_set_state_nolock()) 00000000ba4f1821 lustre-MDT0000_UUID: changing import state from DISCONN to CONNECTING 00000100:00080000:2.0:1776178620.257048:0:36:0:(import.c:534:import_select_connection()) lustre-MDT0000-osp-MDT0001: connect to NID 0@lo last attempt 1322 00000100:00080000:2.0:1776178620.257050:0:36:0:(import.c:581:import_select_connection()) lustre-MDT0000-osp-MDT0001: tried all connections, increasing latency to 21s 00000100:00080000:2.0:1776178620.257053:0:36:0:(import.c:615:import_select_connection()) lustre-MDT0000-osp-MDT0001: import 00000000ba4f1821 using connection 192.168.204.103@tcp/0@lo 00000100:00100000:2.0:1776178620.257091:0:36:0:(import.c:807:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5) req@000000005831fa36 x1862456998106880/t0(0) o38->lustre-MDT0000-osp-MDT0001@0@lo:24/4 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' 00000100:00100000:1.0:1776178620.257107:0:3355:0:(events.c:361:request_in_callback()) peer: 12345-0@lo (source: 12345-0@lo) 00000100:00080000:2.0:1776178620.257115:0:36:0:(pinger.c:251:ptlrpc_pinger_process_import()) lustre-MDT0000-lwp-MDT0001_UUID->lustre-MDT0000_UUID: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00080000:2.0:1776178620.257118:0:36:0:(recover.c:57:ptlrpc_initiate_recovery()) lustre-MDT0000_UUID: starting recovery 00000100:00080000:2.0:1776178620.257120:0:36:0:(import.c:86:import_set_state_nolock()) 00000000a7c55b1f lustre-MDT0000_UUID: changing import state from DISCONN to CONNECTING 00000100:00080000:2.0:1776178620.257124:0:36:0:(import.c:534:import_select_connection()) lustre-MDT0000-lwp-MDT0001: connect to NID 0@lo last attempt 1322 00000100:00080000:2.0:1776178620.257127:0:36:0:(import.c:581:import_select_connection()) lustre-MDT0000-lwp-MDT0001: tried all connections, increasing latency to 21s 00000100:00080000:2.0:1776178620.257130:0:36:0:(import.c:615:import_select_connection()) lustre-MDT0000-lwp-MDT0001: import 00000000a7c55b1f using connection 192.168.204.103@tcp/0@lo 00000100:00100000:1.0:1776178620.257137:0:3355:0:(client.c:1733:ptlrpc_send_new_req()) Sending RPC req@000000006890aba2 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_02:lustre-MDT0001-mdtlov_UUID:3355:1862456998106944:0@lo:400:kworker/u8:1.0 00000100:00100000:2.0:1776178620.257142:0:36:0:(import.c:807:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5) req@00000000fc9499fa x1862456998107072/t0(0) o38->lustre-MDT0000-lwp-MDT0001@0@lo:12/10 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' 00000100:00100000:1.0:1776178620.257169:0:3355:0:(events.c:361:request_in_callback()) peer: 12345-0@lo (source: 12345-0@lo) 00000100:00100000:2.0:1776178620.257171:0:3354:0:(client.c:1733:ptlrpc_send_new_req()) Sending RPC req@0000000011d4114f pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_01:lustre-MDT0001-lwp-OST0001_UUID:3354:1862456998106816:0@lo:400:kworker/u8:1.0 00000100:00100000:1.0:1776178620.257194:0:3355:0:(ptlrpcd.c:415:ptlrpcd_check()) transfer 1 async RPCs [3->2] 00000100:00100000:1.0:1776178620.257201:0:3355:0:(client.c:1733:ptlrpc_send_new_req()) Sending RPC req@00000000fd453e99 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_02:lustre-MDT0001-mdtlov_UUID:3355:1862456998107008:0@lo:400:kworker/u8:1.0 00000100:00100000:2.0:1776178620.257211:0:3354:0:(events.c:361:request_in_callback()) peer: 12345-0@lo (source: 12345-0@lo) 00000100:00100000:3.0:1776178620.257302:0:20762:0:(service.c:2152:ptlrpc_server_handle_req_in()) got req x1862456998106944 00000100:00100000:3.0:1776178620.257344:0:20762:0:(nrs_fifo.c:179:nrs_fifo_req_get()) NRS start fifo request from 12345-0@lo, seq: 837 00000100:00100000:3.0:1776178620.257350:0:20762:0:(service.c:2324:ptlrpc_server_handle_request()) Handling RPC req@00000000b1db1835 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_013:lustre-MDT0001-mdtlov_UUID+5:3355:x1862456998106944:12345-0@lo:400:kworker/u8:1.0 00000100:00100000:0.0:1776178620.257542:0:3352:0:(client.c:1733:ptlrpc_send_new_req()) Sending RPC req@00000000c6b32ddb pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:lustre-MDT0000-lwp-OST0000_UUID:3352:1862456998106624:0@lo:38:kworker/u8:1.0 00000100:00100000:1.0:1776178620.261859:0:3355:0:(events.c:361:request_in_callback()) peer: 12345-0@lo (source: 12345-0@lo) 00000100:00100000:1.0:1776178620.261954:0:11281:0:(service.c:2152:ptlrpc_server_handle_req_in()) got req x1862456998106816 00000100:00100000:1.0:1776178620.262010:0:11281:0:(nrs_fifo.c:179:nrs_fifo_req_get()) NRS start fifo request from 12345-0@lo, seq: 1172 00000100:00100000:1.0:1776178620.262017:0:11281:0:(service.c:2324:ptlrpc_server_handle_request()) Handling RPC req@00000000ec65a22d pname:cluuid+ref:pid:xid:nid:opc:job mdt00_004:lustre-MDT0001-lwp-OST0001_UUID+5:3354:x1862456998106816:12345-0@lo:400:kworker/u8:1.0 00000100:00100000:1.0:1776178620.262335:0:11281:0:(service.c:2372:ptlrpc_server_handle_request()) Handled RPC req@00000000ec65a22d pname:cluuid+ref:pid:xid:nid:opc:job mdt00_004:lustre-MDT0001-lwp-OST0001_UUID+5:3354:x1862456998106816:12345-0@lo:400:kworker/u8:1.0 Request processed in 316us (5125us total) trans 0 rc 0/0 00000100:00100000:1.0:1776178620.262502:0:11281:0:(nrs_fifo.c:241:nrs_fifo_req_stop()) NRS stop fifo request from 12345-0@lo, seq: 1172 00000100:00100000:1.0:1776178620.262563:0:20137:0:(service.c:2152:ptlrpc_server_handle_req_in()) got req x1862456998106560 00000100:00100000:1.0:1776178620.262761:0:20137:0:(nrs_fifo.c:179:nrs_fifo_req_get()) NRS start fifo request from 12345-0@lo, seq: 318 00000100:00100000:1.0:1776178620.262775:0:20137:0:(service.c:2324:ptlrpc_server_handle_request()) Handling RPC req@00000000887443f4 pname:cluuid+ref:pid:xid:nid:opc:job ll_mgs_0000:ac61f839-0e8e-4cac-9c59-08bcfbbad330+18:3355:x1862456998106560:12345-0@lo:400:kworker/u8:1.0 00000100:00100000:1.0:1776178620.262819:0:20137:0:(service.c:2372:ptlrpc_server_handle_request()) Handled RPC req@00000000887443f4 pname:cluuid+ref:pid:xid:nid:opc:job ll_mgs_0000:ac61f839-0e8e-4cac-9c59-08bcfbbad330+18:3355:x1862456998106560:12345-0@lo:400:kworker/u8:1.0 Request processed in 51us (5714us total) trans 0 rc 0/0 00000100:00100000:1.0:1776178620.262825:0:20137:0:(nrs_fifo.c:241:nrs_fifo_req_stop()) NRS stop fifo request from 12345-0@lo, seq: 318 00000100:00100000:1.0:1776178620.262859:0:3353:0:(client.c:1733:ptlrpc_send_new_req()) Sending RPC req@000000009cee1481 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_00:lustre-MDT0001-lwp-OST0000_UUID:3353:1862456998106688:0@lo:400:kworker/u8:1.0 00000100:00100000:1.0:1776178620.262890:0:3353:0:(events.c:361:request_in_callback()) peer: 12345-0@lo (source: 12345-0@lo) 00000100:00100000:1.0:1776178620.262917:0:6257:0:(service.c:2152:ptlrpc_server_handle_req_in()) got req x1862456998106688 00000100:00100000:1.0:1776178620.262940:0:6257:0:(nrs_fifo.c:179:nrs_fifo_req_get()) NRS start fifo request from 12345-0@lo, seq: 1173 00000100:00100000:1.0:1776178620.262944:0:6257:0:(service.c:2324:ptlrpc_server_handle_request()) Handling RPC req@00000000238121c2 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_002:lustre-MDT0001-lwp-OST0000_UUID+5:3353:x1862456998106688:12345-0@lo:400:kworker/u8:1.0 00000100:00100000:1.0:1776178620.262975:0:6257:0:(service.c:2372:ptlrpc_server_handle_request()) Handled RPC req@00000000238121c2 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_002:lustre-MDT0001-lwp-OST0000_UUID+5:3353:x1862456998106688:12345-0@lo:400:kworker/u8:1.0 Request processed in 31us (86us total) trans 0 rc 0/0 00000100:00100000:1.0:1776178620.262981:0:6257:0:(nrs_fifo.c:241:nrs_fifo_req_stop()) NRS stop fifo request from 12345-0@lo, seq: 1173 00000100:00100000:1.0:1776178620.263016:0:3353:0:(client.c:2215:ptlrpc_check_set()) Completed RPC req@000000009cee1481 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_00:lustre-MDT0001-lwp-OST0000_UUID:3353:1862456998106688:0@lo:400:kworker/u8:1.0 00000100:00100000:1.0:1776178620.263048:0:3355:0:(client.c:2215:ptlrpc_check_set()) Completed RPC req@0000000065f62d8c pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_02:ac61f839-0e8e-4cac-9c59-08bcfbbad330:3355:1862456998106560:0@lo:400:kworker/u8:1.0 00000100:00100000:1.0:1776178620.263261:0:14440:0:(service.c:2152:ptlrpc_server_handle_req_in()) got req x1862456998107008 00000100:00100000:1.0:1776178620.263287:0:14440:0:(nrs_fifo.c:179:nrs_fifo_req_get()) NRS start fifo request from 12345-0@lo, seq: 838 00000100:00100000:1.0:1776178620.263291:0:14440:0:(service.c:2324:ptlrpc_server_handle_request()) Handling RPC req@00000000907210d1 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_004:lustre-MDT0001-mdtlov_UUID+5:3355:x1862456998107008:12345-0@lo:400:kworker/u8:1.0 00000100:00100000:1.0:1776178620.263327:0:14440:0:(service.c:2372:ptlrpc_server_handle_request()) Handled RPC req@00000000907210d1 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_004:lustre-MDT0001-mdtlov_UUID+5:3355:x1862456998107008:12345-0@lo:400:kworker/u8:1.0 Request processed in 35us (1475us total) trans 0 rc 0/0 00000100:00100000:1.0:1776178620.263333:0:14440:0:(nrs_fifo.c:241:nrs_fifo_req_stop()) NRS stop fifo request from 12345-0@lo, seq: 838 00000100:00100000:1.0:1776178620.263845:0:3355:0:(client.c:2215:ptlrpc_check_set()) Completed RPC req@00000000fd453e99 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_02:lustre-MDT0001-mdtlov_UUID:3355:1862456998107008:0@lo:400:kworker/u8:1.0 00000100:00100000:3.0:1776178620.268514:0:20762:0:(service.c:2372:ptlrpc_server_handle_request()) Handled RPC req@00000000b1db1835 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_013:lustre-MDT0001-mdtlov_UUID+5:3355:x1862456998106944:12345-0@lo:400:kworker/u8:1.0 Request processed in 11153us (11336us total) trans 0 rc 0/0 00000100:00100000:3.0:1776178620.268531:0:20762:0:(nrs_fifo.c:241:nrs_fifo_req_stop()) NRS stop fifo request from 12345-0@lo, seq: 837 00000100:00100000:1.0:1776178620.268542:0:3355:0:(client.c:2215:ptlrpc_check_set()) Completed RPC req@000000006890aba2 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_02:lustre-MDT0001-mdtlov_UUID:3355:1862456998106944:0@lo:400:kworker/u8:1.0 00000100:00100000:2.0:1776178620.280222:0:3354:0:(client.c:2215:ptlrpc_check_set()) Completed RPC req@0000000011d4114f pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_01:lustre-MDT0001-lwp-OST0001_UUID:3354:1862456998106816:0@lo:400:kworker/u8:1.0 00000100:00100000:0.0:1776178620.281132:0:3352:0:(events.c:361:request_in_callback()) peer: 12345-0@lo (source: 12345-0@lo) 00000100:00100000:0.0:1776178620.281173:0:3352:0:(client.c:1733:ptlrpc_send_new_req()) Sending RPC req@00000000606ce17f pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:lustre-MDT0000-lwp-OST0001_UUID:3352:1862456998106752:0@lo:38:kworker/u8:1.0 00000100:00100000:0.0:1776178620.281213:0:3352:0:(events.c:361:request_in_callback()) peer: 12345-0@lo (source: 12345-0@lo) 00000100:00100000:0.0:1776178620.281373:0:3352:0:(client.c:1733:ptlrpc_send_new_req()) Sending RPC req@000000005831fa36 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:lustre-MDT0001-mdtlov_UUID:3352:1862456998106880:0@lo:38:kworker/u8:1.0 00000100:00100000:0.0:1776178620.281402:0:3352:0:(events.c:361:request_in_callback()) peer: 12345-0@lo (source: 12345-0@lo) 00000100:00100000:0.0:1776178620.281436:0:15213:0:(service.c:2152:ptlrpc_server_handle_req_in()) got req x1862456998106880 00000100:00100000:0.0:1776178620.281491:0:15213:0:(nrs_fifo.c:179:nrs_fifo_req_get()) NRS start fifo request from 12345-0@lo, seq: 789 00000100:00100000:0.0:1776178620.281497:0:15213:0:(service.c:2324:ptlrpc_server_handle_request()) Handling RPC req@00000000c63df573 pname:cluuid+ref:pid:xid:nid:opc:job mdt_out00_003:0+-99:3352:x1862456998106880:12345-0@lo:38:kworker/u8:1.0 00010000:02000000:0.0:1776178620.281511:0:15213:0:(ldlm_lib.c:1139:target_handle_connect()) lustre-MDT0000: Not available for connect from 0@lo (stopping) 00000100:00100000:0.0:1776178620.281548:0:15213:0:(service.c:2372:ptlrpc_server_handle_request()) Handled RPC req@00000000c63df573 pname:cluuid+ref:pid:xid:nid:opc:job mdt_out00_003:0+-99:3352:x1862456998106880:12345-0@lo:38:kworker/u8:1.0 Request processed in 50us (146us total) trans 0 rc -19/-19 00000100:00100000:0.0:1776178620.281555:0:15213:0:(nrs_fifo.c:241:nrs_fifo_req_stop()) NRS stop fifo request from 12345-0@lo, seq: 789 00000100:00100000:0.0:1776178620.281574:0:3352:0:(client.c:1733:ptlrpc_send_new_req()) Sending RPC req@00000000fc9499fa pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:lustre-MDT0000-lwp-MDT0001_UUID:3352:1862456998107072:0@lo:38:kworker/u8:1.0 00000100:00100000:0.0:1776178620.281596:0:3352:0:(events.c:361:request_in_callback()) peer: 12345-0@lo (source: 12345-0@lo) 00000100:00080000:0.0:1776178620.320985:0:3352:0:(import.c:86:import_set_state_nolock()) 00000000ba4f1821 lustre-MDT0000_UUID: changing import state from CONNECTING to DISCONN 00000100:00080000:0.0:1776178620.320996:0:3352:0:(import.c:1429:ptlrpc_connect_interpret()) recovery of lustre-MDT0000_UUID on 192.168.204.103@tcp failed (-19) 00000100:00100000:0.0:1776178620.321003:0:3352:0:(client.c:2215:ptlrpc_check_set()) Completed RPC req@000000005831fa36 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:lustre-MDT0001-mdtlov_UUID:3352:1862456998106880:0@lo:38:kworker/u8:1.0 00000100:00100000:1.0:1776178620.322163:0:11281:0:(service.c:2152:ptlrpc_server_handle_req_in()) got req x1862456998106624 00000100:00100000:1.0:1776178620.322209:0:11281:0:(service.c:2152:ptlrpc_server_handle_req_in()) got req x1862456998106752 00000100:00100000:1.0:1776178620.322226:0:11281:0:(service.c:2152:ptlrpc_server_handle_req_in()) got req x1862456998107072 00000100:00100000:3.0:1776178620.322246:0:6257:0:(nrs_fifo.c:179:nrs_fifo_req_get()) NRS start fifo request from 12345-0@lo, seq: 6949 00000100:00100000:3.0:1776178620.322256:0:6257:0:(service.c:2324:ptlrpc_server_handle_request()) Handling RPC req@00000000f6259053 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_002:0+-99:3352:x1862456998106624:12345-0@lo:38:kworker/u8:1.0 00010000:02000000:3.0:1776178620.322272:0:6257:0:(ldlm_lib.c:1139:target_handle_connect()) lustre-MDT0000: Not available for connect from 0@lo (stopping) 00000100:00100000:3.0:1776178620.322348:0:6257:0:(service.c:2372:ptlrpc_server_handle_request()) Handled RPC req@00000000f6259053 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_002:0+-99:3352:x1862456998106624:12345-0@lo:38:kworker/u8:1.0 Request processed in 94us (41221us total) trans 0 rc -19/-19 00000100:00100000:1.0:1776178620.331941:0:11281:0:(nrs_fifo.c:179:nrs_fifo_req_get()) NRS start fifo request from 12345-0@lo, seq: 6950 00000100:00100000:1.0:1776178620.331954:0:11281:0:(service.c:2324:ptlrpc_server_handle_request()) Handling RPC req@000000005b8f544b pname:cluuid+ref:pid:xid:nid:opc:job mdt00_004:0+-99:3352:x1862456998106752:12345-0@lo:38:kworker/u8:1.0 00010000:02000000:1.0:1776178620.331974:0:11281:0:(ldlm_lib.c:1139:target_handle_connect()) lustre-MDT0000: Not available for connect from 0@lo (stopping) 00000100:00100000:1.0:1776178620.332053:0:11281:0:(service.c:2372:ptlrpc_server_handle_request()) Handled RPC req@000000005b8f544b pname:cluuid+ref:pid:xid:nid:opc:job mdt00_004:0+-99:3352:x1862456998106752:12345-0@lo:38:kworker/u8:1.0 Request processed in 101us (50840us total) trans 0 rc -19/-19 00000100:00100000:1.0:1776178620.332083:0:11281:0:(nrs_fifo.c:241:nrs_fifo_req_stop()) NRS stop fifo request from 12345-0@lo, seq: 6950 00000100:00100000:1.0:1776178620.332112:0:11281:0:(nrs_fifo.c:179:nrs_fifo_req_get()) NRS start fifo request from 12345-0@lo, seq: 6951 00000100:00100000:1.0:1776178620.332115:0:11281:0:(service.c:2324:ptlrpc_server_handle_request()) Handling RPC req@000000003a8c2e67 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_004:0+-99:3352:x1862456998107072:12345-0@lo:38:kworker/u8:1.0 00010000:02000000:1.0:1776178620.332122:0:11281:0:(ldlm_lib.c:1139:target_handle_connect()) lustre-MDT0000: Not available for connect from 0@lo (stopping) 00000100:00100000:1.0:1776178620.332141:0:11281:0:(service.c:2372:ptlrpc_server_handle_request()) Handled RPC req@000000003a8c2e67 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_004:0+-99:3352:x1862456998107072:12345-0@lo:38:kworker/u8:1.0 Request processed in 25us (50545us total) trans 0 rc -19/-19 00000100:00100000:1.0:1776178620.332147:0:11281:0:(nrs_fifo.c:241:nrs_fifo_req_stop()) NRS stop fifo request from 12345-0@lo, seq: 6951 00000100:00080000:1.0:1776178620.332276:0:3352:0:(import.c:86:import_set_state_nolock()) 0000000047601a98 lustre-MDT0000_UUID: changing import state from CONNECTING to DISCONN 00000100:00080000:1.0:1776178620.332280:0:3352:0:(import.c:1429:ptlrpc_connect_interpret()) recovery of lustre-MDT0000_UUID on 192.168.204.103@tcp failed (-19) 00000100:00100000:1.0:1776178620.332284:0:3352:0:(client.c:2215:ptlrpc_check_set()) Completed RPC req@00000000c6b32ddb pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:lustre-MDT0000-lwp-OST0000_UUID:3352:1862456998106624:0@lo:38:kworker/u8:1.0 00000100:00080000:1.0:1776178620.332295:0:3352:0:(import.c:86:import_set_state_nolock()) 00000000dfc8db13 lustre-MDT0000_UUID: changing import state from CONNECTING to DISCONN 00000100:00080000:1.0:1776178620.332297:0:3352:0:(import.c:1429:ptlrpc_connect_interpret()) recovery of lustre-MDT0000_UUID on 192.168.204.103@tcp failed (-19) 00000100:00100000:1.0:1776178620.332300:0:3352:0:(client.c:2215:ptlrpc_check_set()) Completed RPC req@00000000606ce17f pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:lustre-MDT0000-lwp-OST0001_UUID:3352:1862456998106752:0@lo:38:kworker/u8:1.0 00000100:00080000:1.0:1776178620.332307:0:3352:0:(import.c:86:import_set_state_nolock()) 00000000a7c55b1f lustre-MDT0000_UUID: changing import state from CONNECTING to DISCONN 00000100:00080000:1.0:1776178620.332310:0:3352:0:(import.c:1429:ptlrpc_connect_interpret()) recovery of lustre-MDT0000_UUID on 192.168.204.103@tcp failed (-19) 00000100:00100000:1.0:1776178620.332312:0:3352:0:(client.c:2215:ptlrpc_check_set()) Completed RPC req@00000000fc9499fa pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:lustre-MDT0000-lwp-MDT0001_UUID:3352:1862456998107072:0@lo:38:kworker/u8:1.0 00000100:00100000:3.0:1776178620.342206:0:6257:0:(nrs_fifo.c:241:nrs_fifo_req_stop()) NRS stop fifo request from 12345-0@lo, seq: 6949 00010000:00010000:1.0:1776178620.896797:0:9224:0:(ldlm_lockd.c:2792:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:2.0:1776178621.794627:0:3354:0:(ptlrpcd.c:415:ptlrpcd_check()) transfer 1 async RPCs [0->1] 00000100:00100000:2.0:1776178621.794654:0:3354:0:(client.c:1733:ptlrpc_send_new_req()) Sending RPC req@0000000092e0e13d pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_01:lustre-MDT0001-mdtlov_UUID:3354:1862456998107136:0@lo:13:osp-pre-1-1.0 00000100:00100000:2.0:1776178621.794723:0:3354:0:(events.c:361:request_in_callback()) peer: 12345-0@lo (source: 12345-0@lo) 00000100:00100000:3.0:1776178621.794886:0:8231:0:(service.c:2152:ptlrpc_server_handle_req_in()) got req x1862456998107136 00000100:00100000:3.0:1776178621.794937:0:8231:0:(nrs_fifo.c:179:nrs_fifo_req_get()) NRS start fifo request from 12345-0@lo, seq: 651 00000100:00100000:3.0:1776178621.794942:0:8231:0:(service.c:2324:ptlrpc_server_handle_request()) Handling RPC req@0000000011d4114f pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_create00:lustre-MDT0001-mdtlov_UUID+5:3354:x1862456998107136:12345-0@lo:13:osp-pre-1-1.0 00002000:00000024:3.0:1776178621.794970:0:8231:0:(ofd_obd.c:740:ofd_statfs()) blocks cached 229376 granted 491746368 pending 0 free 3923689472 avail 3692167168 00000100:00100000:3.0:1776178621.795016:0:8231:0:(service.c:2372:ptlrpc_server_handle_request()) Handled RPC req@0000000011d4114f pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_create00:lustre-MDT0001-mdtlov_UUID+5:3354:x1862456998107136:12345-0@lo:13:osp-pre-1-1.0 Request processed in 74us (295us total) trans 0 rc 0/0 00000100:00100000:3.0:1776178621.795023:0:8231:0:(nrs_fifo.c:241:nrs_fifo_req_stop()) NRS stop fifo request from 12345-0@lo, seq: 651 00000100:00100000:2.0:1776178621.797697:0:3354:0:(client.c:2215:ptlrpc_check_set()) Completed RPC req@0000000092e0e13d pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_01:lustre-MDT0001-mdtlov_UUID:3354:1862456998107136:0@lo:13:osp-pre-1-1.0 00010000:00010000:1.0:1776178621.920427:0:9225:0:(ldlm_lockd.c:2792:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:0.0:1776178622.304565:0:3354:0:(client.c:1733:ptlrpc_send_new_req()) Sending RPC req@00000000de4e29fd pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_01:lustre-MDT0001-mdtlov_UUID:3354:1862456998107200:0@lo:13:osp-pre-0-1.0 00000100:00100000:0.0:1776178622.304644:0:3354:0:(events.c:361:request_in_callback()) peer: 12345-0@lo (source: 12345-0@lo) 00000100:00100000:1.0:1776178622.304769:0:9554:0:(service.c:2152:ptlrpc_server_handle_req_in()) got req x1862456998107200 00000100:00100000:1.0:1776178622.304820:0:9554:0:(nrs_fifo.c:179:nrs_fifo_req_get()) NRS start fifo request from 12345-0@lo, seq: 652 00000100:00100000:1.0:1776178622.304826:0:9554:0:(service.c:2324:ptlrpc_server_handle_request()) Handling RPC req@00000000cd795392 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_create00:lustre-MDT0001-mdtlov_UUID+5:3354:x1862456998107200:12345-0@lo:13:osp-pre-0-1.0 00002000:00000024:1.0:1776178622.304858:0:9554:0:(ofd_obd.c:740:ofd_statfs()) blocks cached 315392 granted 484883776 pending 0 free 3923689472 avail 3692167168 00000100:00100000:1.0:1776178622.304908:0:9554:0:(service.c:2372:ptlrpc_server_handle_request()) Handled RPC req@00000000cd795392 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_create00:lustre-MDT0001-mdtlov_UUID+5:3354:x1862456998107200:12345-0@lo:13:osp-pre-0-1.0 Request processed in 81us (266us total) trans 0 rc 0/0 00000100:00100000:1.0:1776178622.304916:0:9554:0:(nrs_fifo.c:241:nrs_fifo_req_stop()) NRS stop fifo request from 12345-0@lo, seq: 652 00000100:00100000:0.0:1776178622.305213:0:3354:0:(client.c:2215:ptlrpc_check_set()) Completed RPC req@00000000de4e29fd pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_01:lustre-MDT0001-mdtlov_UUID:3354:1862456998107200:0@lo:13:osp-pre-0-1.0 00010000:00010000:2.0:1776178622.945859:0:6244:0:(ldlm_lockd.c:2792:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:0.0:1776178623.780983:0:3347:0:(events.c:361:request_in_callback()) peer: 12345-192.168.204.3@tcp (source: 12345-192.168.204.3@tcp) 00000100:00100000:0.0:1776178623.781046:0:3347:0:(events.c:361:request_in_callback()) peer: 12345-192.168.204.3@tcp (source: 12345-192.168.204.3@tcp) 00000100:00100000:0.0:1776178623.781135:0:3347:0:(events.c:361:request_in_callback()) peer: 12345-192.168.204.3@tcp (source: 12345-192.168.204.3@tcp) 00000100:00100000:3.0:1776178623.781168:0:20762:0:(service.c:2152:ptlrpc_server_handle_req_in()) got req x1862456979135488 00000100:00100000:2.0:1776178623.781195:0:6256:0:(service.c:2152:ptlrpc_server_handle_req_in()) got req x1862456979135360 00000100:00100000:2.0:1776178623.781248:0:6256:0:(nrs_fifo.c:179:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.204.3@tcp, seq: 1174 00000100:00100000:3.0:1776178623.781248:0:20762:0:(nrs_fifo.c:179:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.204.3@tcp, seq: 839 00000100:00100000:2.0:1776178623.781259:0:6256:0:(service.c:2324:ptlrpc_server_handle_request()) Handling RPC req@00000000bbf26718 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_001:b50963bd-8eba-4432-9a54-8888d00e3fa1+6:2273:x1862456979135360:12345-192.168.204.3@tcp:400:kworker/u8:4.0 00000100:00100000:3.0:1776178623.781262:0:20762:0:(service.c:2324:ptlrpc_server_handle_request()) Handling RPC req@00000000eb611cec pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_013:b50963bd-8eba-4432-9a54-8888d00e3fa1+9:2273:x1862456979135488:12345-192.168.204.3@tcp:400:kworker/u8:4.0 00000100:00100000:3.0:1776178623.781448:0:20762:0:(service.c:2372:ptlrpc_server_handle_request()) Handled RPC req@00000000eb611cec pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_013:b50963bd-8eba-4432-9a54-8888d00e3fa1+9:2273:x1862456979135488:12345-192.168.204.3@tcp:400:kworker/u8:4.0 Request processed in 190us (402us total) trans 0 rc 0/0 00000100:00100000:3.0:1776178623.781456:0:20762:0:(nrs_fifo.c:241:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.204.3@tcp, seq: 839 00000100:00100000:3.0:1776178623.782496:0:20140:0:(service.c:2152:ptlrpc_server_handle_req_in()) got req x1862456979135232 00000100:00100000:3.0:1776178623.782549:0:20140:0:(nrs_fifo.c:179:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.204.3@tcp, seq: 319 00000100:00100000:3.0:1776178623.782555:0:20140:0:(service.c:2324:ptlrpc_server_handle_request()) Handling RPC req@0000000076ad720a pname:cluuid+ref:pid:xid:nid:opc:job ll_mgs_0002:5b5c66b1-1a2f-4651-8548-0c75b9a38a94+9:2273:x1862456979135232:12345-192.168.204.3@tcp:400:kworker/u8:4.0 00000100:00100000:2.0:1776178623.783832:0:6256:0:(service.c:2372:ptlrpc_server_handle_request()) Handled RPC req@00000000bbf26718 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_001:b50963bd-8eba-4432-9a54-8888d00e3fa1+6:2273:x1862456979135360:12345-192.168.204.3@tcp:400:kworker/u8:4.0 Request processed in 2567us (2851us total) trans 0 rc 0/0 00000100:00100000:2.0:1776178623.783848:0:6256:0:(nrs_fifo.c:241:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.204.3@tcp, seq: 1174 00000100:00100000:3.0:1776178623.790603:0:20140:0:(service.c:2372:ptlrpc_server_handle_request()) Handled RPC req@0000000076ad720a pname:cluuid+ref:pid:xid:nid:opc:job ll_mgs_0002:5b5c66b1-1a2f-4651-8548-0c75b9a38a94+9:2273:x1862456979135232:12345-192.168.204.3@tcp:400:kworker/u8:4.0 Request processed in 8042us (9474us total) trans 0 rc 0/0 00000100:00100000:3.0:1776178623.790621:0:20140:0:(nrs_fifo.c:241:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.204.3@tcp, seq: 319 00000100:00100000:0.0:1776178623.790732:0:3347:0:(events.c:361:request_in_callback()) peer: 12345-192.168.204.3@tcp (source: 12345-192.168.204.3@tcp) 00000100:00100000:1.0:1776178623.790812:0:6256:0:(service.c:2152:ptlrpc_server_handle_req_in()) got req x1862456979135296 00000100:00100000:1.0:1776178623.790873:0:6256:0:(nrs_fifo.c:179:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.204.3@tcp, seq: 6952 00000100:00100000:1.0:1776178623.790880:0:6256:0:(service.c:2324:ptlrpc_server_handle_request()) Handling RPC req@00000000e2b892b4 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_001:0+-99:2269:x1862456979135296:12345-192.168.204.3@tcp:38:kworker/u8:4.0 00010000:02000000:1.0:1776178623.790898:0:6256:0:(ldlm_lib.c:1139:target_handle_connect()) lustre-MDT0000: Not available for connect from 192.168.204.3@tcp (stopping) 00000100:00100000:0.0:1776178623.800989:0:3347:0:(events.c:361:request_in_callback()) peer: 12345-192.168.204.3@tcp (source: 12345-192.168.204.3@tcp) 00000100:00100000:3.0:1776178623.801816:0:20762:0:(service.c:2152:ptlrpc_server_handle_req_in()) got req x1862456979135424 00000100:00100000:3.0:1776178623.801890:0:20759:0:(nrs_fifo.c:179:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.204.3@tcp, seq: 840 00000100:00100000:3.0:1776178623.801897:0:20759:0:(service.c:2324:ptlrpc_server_handle_request()) Handling RPC req@00000000d04e7a02 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_010:b50963bd-8eba-4432-9a54-8888d00e3fa1+8:2273:x1862456979135424:12345-192.168.204.3@tcp:400:kworker/u8:4.0 00000100:00100000:3.0:1776178623.801967:0:20759:0:(service.c:2372:ptlrpc_server_handle_request()) Handled RPC req@00000000d04e7a02 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_010:b50963bd-8eba-4432-9a54-8888d00e3fa1+8:2273:x1862456979135424:12345-192.168.204.3@tcp:400:kworker/u8:4.0 Request processed in 68us (985us total) trans 0 rc 0/0 00000100:00100000:3.0:1776178623.801974:0:20759:0:(nrs_fifo.c:241:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.204.3@tcp, seq: 840 00000100:00100000:1.0:1776178623.814804:0:6256:0:(service.c:2372:ptlrpc_server_handle_request()) Handled RPC req@00000000e2b892b4 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_001:0+-99:2269:x1862456979135296:12345-192.168.204.3@tcp:38:kworker/u8:4.0 Request processed in 23914us (24662us total) trans 0 rc -19/-19 00000100:00100000:1.0:1776178623.814822:0:6256:0:(nrs_fifo.c:241:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.204.3@tcp, seq: 6952 00010000:00010000:1.0:1776178623.969308:0:6245:0:(ldlm_lockd.c:2792:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00010000:00010000:1.0:1776178624.992562:0:9224:0:(ldlm_lockd.c:2792:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:1.0:1776178625.386253:0:3355:0:(client.c:1733:ptlrpc_send_new_req()) Sending RPC req@00000000f18ffde7 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_02:ac61f839-0e8e-4cac-9c59-08bcfbbad330:3355:1862456998107264:0@lo:400:kworker/u8:0.0 00000100:00100000:1.0:1776178625.386333:0:3355:0:(events.c:361:request_in_callback()) peer: 12345-0@lo (source: 12345-0@lo) 00000100:00080000:0.0:1776178625.388584:0:22249:0:(pinger.c:251:ptlrpc_pinger_process_import()) lustre-MDT0000-lwp-OST0000_UUID->lustre-MDT0000_UUID: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00080000:0.0:1776178625.388596:0:22249:0:(recover.c:57:ptlrpc_initiate_recovery()) lustre-MDT0000_UUID: starting recovery 00000100:00080000:0.0:1776178625.388599:0:22249:0:(import.c:86:import_set_state_nolock()) 0000000047601a98 lustre-MDT0000_UUID: changing import state from DISCONN to CONNECTING 00000100:00080000:0.0:1776178625.388608:0:22249:0:(import.c:534:import_select_connection()) lustre-MDT0000-lwp-OST0000: connect to NID 0@lo last attempt 1327 00000100:00080000:0.0:1776178625.388611:0:22249:0:(import.c:581:import_select_connection()) lustre-MDT0000-lwp-OST0000: tried all connections, increasing latency to 21s 00000100:00080000:0.0:1776178625.388617:0:22249:0:(import.c:615:import_select_connection()) lustre-MDT0000-lwp-OST0000: import 0000000047601a98 using connection 192.168.204.103@tcp/0@lo 00000100:00100000:0.0:1776178625.388655:0:22249:0:(import.c:807:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5) req@00000000dac56e3f x1862456998107328/t0(0) o38->lustre-MDT0000-lwp-OST0000@0@lo:12/10 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' 00000100:00100000:3.0:1776178625.388945:0:20140:0:(service.c:2152:ptlrpc_server_handle_req_in()) got req x1862456998107264 00000100:00100000:3.0:1776178625.388999:0:20140:0:(nrs_fifo.c:179:nrs_fifo_req_get()) NRS start fifo request from 12345-0@lo, seq: 320 00000100:00100000:3.0:1776178625.389006:0:20140:0:(service.c:2324:ptlrpc_server_handle_request()) Handling RPC req@00000000d4927be7 pname:cluuid+ref:pid:xid:nid:opc:job ll_mgs_0002:ac61f839-0e8e-4cac-9c59-08bcfbbad330+18:3355:x1862456998107264:12345-0@lo:400:kworker/u8:0.0 00000100:00100000:3.0:1776178625.389139:0:20140:0:(service.c:2372:ptlrpc_server_handle_request()) Handled RPC req@00000000d4927be7 pname:cluuid+ref:pid:xid:nid:opc:job ll_mgs_0002:ac61f839-0e8e-4cac-9c59-08bcfbbad330+18:3355:x1862456998107264:12345-0@lo:400:kworker/u8:0.0 Request processed in 133us (2808us total) trans 0 rc 0/0 00000100:00100000:2.0:1776178625.389148:0:3352:0:(client.c:1733:ptlrpc_send_new_req()) Sending RPC req@00000000dac56e3f pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:lustre-MDT0000-lwp-OST0000_UUID:3352:1862456998107328:0@lo:38:kworker/u8:0.0 00000100:00100000:3.0:1776178625.389148:0:20140:0:(nrs_fifo.c:241:nrs_fifo_req_stop()) NRS stop fifo request from 12345-0@lo, seq: 320 00000100:00100000:2.0:1776178625.389190:0:3352:0:(events.c:361:request_in_callback()) peer: 12345-0@lo (source: 12345-0@lo) 00000100:00100000:1.0:1776178625.389221:0:3355:0:(client.c:2215:ptlrpc_check_set()) Completed RPC req@00000000f18ffde7 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_02:ac61f839-0e8e-4cac-9c59-08bcfbbad330:3355:1862456998107264:0@lo:400:kworker/u8:0.0 00000100:00100000:1.0:1776178625.389241:0:3355:0:(ptlrpcd.c:415:ptlrpcd_check()) transfer 1 async RPCs [3->2] 00000100:00100000:1.0:1776178625.389248:0:3355:0:(client.c:1733:ptlrpc_send_new_req()) Sending RPC req@00000000d1597b46 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_02:lustre-MDT0001-lwp-OST0000_UUID:3355:1862456998107392:0@lo:400:kworker/u8:0.0 00000100:00080000:0.0:1776178625.389251:0:22249:0:(pinger.c:251:ptlrpc_pinger_process_import()) lustre-MDT0000-lwp-OST0001_UUID->lustre-MDT0000_UUID: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00080000:0.0:1776178625.389256:0:22249:0:(recover.c:57:ptlrpc_initiate_recovery()) lustre-MDT0000_UUID: starting recovery 00000100:00080000:0.0:1776178625.389258:0:22249:0:(import.c:86:import_set_state_nolock()) 00000000dfc8db13 lustre-MDT0000_UUID: changing import state from DISCONN to CONNECTING 00000100:00080000:0.0:1776178625.389263:0:22249:0:(import.c:534:import_select_connection()) lustre-MDT0000-lwp-OST0001: connect to NID 0@lo last attempt 1327 00000100:00080000:0.0:1776178625.389266:0:22249:0:(import.c:581:import_select_connection()) lustre-MDT0000-lwp-OST0001: tried all connections, increasing latency to 21s 00000100:00080000:0.0:1776178625.389270:0:22249:0:(import.c:615:import_select_connection()) lustre-MDT0000-lwp-OST0001: import 00000000dfc8db13 using connection 192.168.204.103@tcp/0@lo 00000100:00100000:1.0:1776178625.389276:0:3355:0:(events.c:361:request_in_callback()) peer: 12345-0@lo (source: 12345-0@lo) 00000100:00100000:0.0:1776178625.389285:0:22249:0:(import.c:807:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5) req@00000000f18ffde7 x1862456998107456/t0(0) o38->lustre-MDT0000-lwp-OST0001@0@lo:12/10 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' 00000100:00080000:0.0:1776178625.389315:0:22249:0:(pinger.c:251:ptlrpc_pinger_process_import()) lustre-MDT0001-mdtlov_UUID->lustre-MDT0000_UUID: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00080000:0.0:1776178625.389318:0:22249:0:(recover.c:57:ptlrpc_initiate_recovery()) lustre-MDT0000_UUID: starting recovery 00000100:00080000:0.0:1776178625.389319:0:22249:0:(import.c:86:import_set_state_nolock()) 00000000ba4f1821 lustre-MDT0000_UUID: changing import state from DISCONN to CONNECTING 00000100:00080000:0.0:1776178625.389322:0:22249:0:(import.c:534:import_select_connection()) lustre-MDT0000-osp-MDT0001: connect to NID 0@lo last attempt 1327 00000100:00080000:0.0:1776178625.389324:0:22249:0:(import.c:581:import_select_connection()) lustre-MDT0000-osp-MDT0001: tried all connections, increasing latency to 21s 00000100:00080000:0.0:1776178625.389327:0:22249:0:(import.c:615:import_select_connection()) lustre-MDT0000-osp-MDT0001: import 00000000ba4f1821 using connection 192.168.204.103@tcp/0@lo 00000100:00100000:0.0:1776178625.389336:0:22249:0:(import.c:807:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5) req@00000000722afaa5 x1862456998107584/t0(0) o38->lustre-MDT0000-osp-MDT0001@0@lo:24/4 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' 00000100:00080000:0.0:1776178625.389355:0:22249:0:(pinger.c:251:ptlrpc_pinger_process_import()) lustre-MDT0000-lwp-MDT0001_UUID->lustre-MDT0000_UUID: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00080000:0.0:1776178625.389359:0:22249:0:(recover.c:57:ptlrpc_initiate_recovery()) lustre-MDT0000_UUID: starting recovery 00000100:00080000:0.0:1776178625.389360:0:22249:0:(import.c:86:import_set_state_nolock()) 00000000a7c55b1f lustre-MDT0000_UUID: changing import state from DISCONN to CONNECTING 00000100:00080000:0.0:1776178625.389363:0:22249:0:(import.c:534:import_select_connection()) lustre-MDT0000-lwp-MDT0001: connect to NID 0@lo last attempt 1327 00000100:00080000:0.0:1776178625.389365:0:22249:0:(import.c:581:import_select_connection()) lustre-MDT0000-lwp-MDT0001: tried all connections, increasing latency to 21s 00000100:00100000:2.0:1776178625.389368:0:3352:0:(client.c:1733:ptlrpc_send_new_req()) Sending RPC req@00000000f18ffde7 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:lustre-MDT0000-lwp-OST0001_UUID:3352:1862456998107456:0@lo:38:kworker/u8:0.0 00000100:00080000:0.0:1776178625.389370:0:22249:0:(import.c:615:import_select_connection()) lustre-MDT0000-lwp-MDT0001: import 00000000a7c55b1f using connection 192.168.204.103@tcp/0@lo 00000100:00100000:0.0:1776178625.389385:0:22249:0:(import.c:807:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5) req@00000000f20bd8f9 x1862456998107776/t0(0) o38->lustre-MDT0000-lwp-MDT0001@0@lo:12/10 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' 00000100:00100000:2.0:1776178625.389400:0:3356:0:(ptlrpcd.c:415:ptlrpcd_check()) transfer 1 async RPCs [2->3] 00000100:00100000:2.0:1776178625.389409:0:3356:0:(client.c:1733:ptlrpc_send_new_req()) Sending RPC req@0000000037b0061a pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_03:lustre-MDT0001-mdtlov_UUID:3356:1862456998107712:0@lo:400:kworker/u8:0.0 00000100:00100000:0.0:1776178625.389415:0:3353:0:(client.c:1733:ptlrpc_send_new_req()) Sending RPC req@00000000b069e162 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_00:lustre-MDT0001-lwp-OST0001_UUID:3353:1862456998107520:0@lo:400:kworker/u8:0.0 00000100:00100000:2.0:1776178625.389437:0:3356:0:(events.c:361:request_in_callback()) peer: 12345-0@lo (source: 12345-0@lo) 00000100:00100000:0.0:1776178625.389459:0:3354:0:(client.c:1733:ptlrpc_send_new_req()) Sending RPC req@00000000c21d7f55 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_01:lustre-MDT0001-mdtlov_UUID:3354:1862456998107648:0@lo:400:kworker/u8:0.0 00000100:00100000:2.0:1776178625.389471:0:20762:0:(service.c:2152:ptlrpc_server_handle_req_in()) got req x1862456998107712 00000100:00100000:0.0:1776178625.389491:0:3354:0:(events.c:361:request_in_callback()) peer: 12345-0@lo (source: 12345-0@lo) 00000100:00100000:2.0:1776178625.389508:0:20759:0:(service.c:2152:ptlrpc_server_handle_req_in()) got req x1862456998107648 00000100:00100000:2.0:1776178625.389533:0:20759:0:(nrs_fifo.c:179:nrs_fifo_req_get()) NRS start fifo request from 12345-0@lo, seq: 841 00000100:00100000:2.0:1776178625.389537:0:20759:0:(service.c:2324:ptlrpc_server_handle_request()) Handling RPC req@00000000fc9499fa pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_010:lustre-MDT0001-mdtlov_UUID+5:3356:x1862456998107712:12345-0@lo:400:kworker/u8:0.0 00000100:00100000:0.0:1776178625.389552:0:20755:0:(nrs_fifo.c:179:nrs_fifo_req_get()) NRS start fifo request from 12345-0@lo, seq: 842 00000100:00100000:0.0:1776178625.389558:0:20755:0:(service.c:2324:ptlrpc_server_handle_request()) Handling RPC req@000000004f638b57 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_006:lustre-MDT0001-mdtlov_UUID+5:3354:x1862456998107648:12345-0@lo:400:kworker/u8:0.0 00000100:00100000:2.0:1776178625.389575:0:20759:0:(service.c:2372:ptlrpc_server_handle_request()) Handled RPC req@00000000fc9499fa pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_010:lustre-MDT0001-mdtlov_UUID+5:3356:x1862456998107712:12345-0@lo:400:kworker/u8:0.0 Request processed in 36us (138us total) trans 0 rc 0/0 00000100:00100000:2.0:1776178625.389583:0:20759:0:(nrs_fifo.c:241:nrs_fifo_req_stop()) NRS stop fifo request from 12345-0@lo, seq: 841 00000100:00100000:0.0:1776178625.389601:0:20755:0:(service.c:2372:ptlrpc_server_handle_request()) Handled RPC req@000000004f638b57 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_006:lustre-MDT0001-mdtlov_UUID+5:3354:x1862456998107648:12345-0@lo:400:kworker/u8:0.0 Request processed in 42us (111us total) trans 0 rc 0/0 00000100:00100000:0.0:1776178625.389608:0:20755:0:(nrs_fifo.c:241:nrs_fifo_req_stop()) NRS stop fifo request from 12345-0@lo, seq: 842 00000100:00100000:2.0:1776178625.389616:0:3356:0:(client.c:2215:ptlrpc_check_set()) Completed RPC req@0000000037b0061a pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_03:lustre-MDT0001-mdtlov_UUID:3356:1862456998107712:0@lo:400:kworker/u8:0.0 00000100:00100000:0.0:1776178625.389643:0:3354:0:(client.c:2215:ptlrpc_check_set()) Completed RPC req@00000000c21d7f55 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_01:lustre-MDT0001-mdtlov_UUID:3354:1862456998107648:0@lo:400:kworker/u8:0.0 00000100:00100000:2.0:1776178625.397096:0:6255:0:(service.c:2152:ptlrpc_server_handle_req_in()) got req x1862456998107328 00000100:00100000:2.0:1776178625.397156:0:6255:0:(service.c:2152:ptlrpc_server_handle_req_in()) got req x1862456998107392 00000100:00100000:2.0:1776178625.397190:0:6255:0:(nrs_fifo.c:179:nrs_fifo_req_get()) NRS start fifo request from 12345-0@lo, seq: 1175 00000100:00100000:2.0:1776178625.397196:0:6255:0:(service.c:2324:ptlrpc_server_handle_request()) Handling RPC req@00000000e964920a pname:cluuid+ref:pid:xid:nid:opc:job mdt00_000:lustre-MDT0001-lwp-OST0000_UUID+5:3355:x1862456998107392:12345-0@lo:400:kworker/u8:0.0 00000100:00100000:1.0:1776178625.397279:0:3352:0:(events.c:361:request_in_callback()) peer: 12345-0@lo (source: 12345-0@lo) 00000100:00100000:2.0:1776178625.397286:0:6255:0:(service.c:2372:ptlrpc_server_handle_request()) Handled RPC req@00000000e964920a pname:cluuid+ref:pid:xid:nid:opc:job mdt00_000:lustre-MDT0001-lwp-OST0000_UUID+5:3355:x1862456998107392:12345-0@lo:400:kworker/u8:0.0 Request processed in 88us (8010us total) trans 0 rc 0/0 00000100:00100000:2.0:1776178625.397295:0:6255:0:(nrs_fifo.c:241:nrs_fifo_req_stop()) NRS stop fifo request from 12345-0@lo, seq: 1175 00000100:00100000:1.0:1776178625.397313:0:3352:0:(client.c:1733:ptlrpc_send_new_req()) Sending RPC req@00000000722afaa5 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:lustre-MDT0001-mdtlov_UUID:3352:1862456998107584:0@lo:38:kworker/u8:0.0 00000100:00100000:2.0:1776178625.397315:0:6255:0:(service.c:2152:ptlrpc_server_handle_req_in()) got req x1862456998107456 00000100:00100000:2.0:1776178625.397328:0:6255:0:(nrs_fifo.c:179:nrs_fifo_req_get()) NRS start fifo request from 12345-0@lo, seq: 6953 00000100:00100000:2.0:1776178625.397332:0:6255:0:(service.c:2324:ptlrpc_server_handle_request()) Handling RPC req@00000000de4e29fd pname:cluuid+ref:pid:xid:nid:opc:job mdt00_000:0+-99:3352:x1862456998107328:12345-0@lo:38:kworker/u8:0.0 00010000:02000000:2.0:1776178625.397346:0:6255:0:(ldlm_lib.c:1139:target_handle_connect()) lustre-MDT0000: Not available for connect from 0@lo (stopping) 00000100:00100000:1.0:1776178625.397361:0:3352:0:(events.c:361:request_in_callback()) peer: 12345-0@lo (source: 12345-0@lo) 00000100:00100000:2.0:1776178625.397387:0:6255:0:(service.c:2372:ptlrpc_server_handle_request()) Handled RPC req@00000000de4e29fd pname:cluuid+ref:pid:xid:nid:opc:job mdt00_000:0+-99:3352:x1862456998107328:12345-0@lo:38:kworker/u8:0.0 Request processed in 54us (8197us total) trans 0 rc -19/-19 00000100:00100000:2.0:1776178625.397393:0:6255:0:(nrs_fifo.c:241:nrs_fifo_req_stop()) NRS stop fifo request from 12345-0@lo, seq: 6953 00000100:00080000:1.0:1776178625.397404:0:3352:0:(import.c:86:import_set_state_nolock()) 0000000047601a98 lustre-MDT0000_UUID: changing import state from CONNECTING to DISCONN 00000100:00100000:2.0:1776178625.397406:0:6255:0:(nrs_fifo.c:179:nrs_fifo_req_get()) NRS start fifo request from 12345-0@lo, seq: 6954 00000100:00080000:1.0:1776178625.397412:0:3352:0:(import.c:1429:ptlrpc_connect_interpret()) recovery of lustre-MDT0000_UUID on 192.168.204.103@tcp failed (-19) 00000100:00100000:2.0:1776178625.397414:0:6255:0:(service.c:2324:ptlrpc_server_handle_request()) Handling RPC req@000000003777ab0e pname:cluuid+ref:pid:xid:nid:opc:job mdt00_000:0+-99:3352:x1862456998107456:12345-0@lo:38:kworker/u8:0.0 00000100:00100000:1.0:1776178625.397419:0:3352:0:(client.c:2215:ptlrpc_check_set()) Completed RPC req@00000000dac56e3f pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:lustre-MDT0000-lwp-OST0000_UUID:3352:1862456998107328:0@lo:38:kworker/u8:0.0 00010000:02000000:2.0:1776178625.397426:0:6255:0:(ldlm_lib.c:1139:target_handle_connect()) lustre-MDT0000: Not available for connect from 0@lo (stopping) 00000100:00100000:2.0:1776178625.397449:0:6255:0:(service.c:2372:ptlrpc_server_handle_request()) Handled RPC req@000000003777ab0e pname:cluuid+ref:pid:xid:nid:opc:job mdt00_000:0+-99:3352:x1862456998107456:12345-0@lo:38:kworker/u8:0.0 Request processed in 35us (175us total) trans 0 rc -19/-19 00000100:00100000:2.0:1776178625.397456:0:6255:0:(nrs_fifo.c:241:nrs_fifo_req_stop()) NRS stop fifo request from 12345-0@lo, seq: 6954 00000100:00100000:1.0:1776178625.397501:0:3352:0:(client.c:1733:ptlrpc_send_new_req()) Sending RPC req@00000000f20bd8f9 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:lustre-MDT0000-lwp-MDT0001_UUID:3352:1862456998107776:0@lo:38:kworker/u8:0.0 00000100:00100000:1.0:1776178625.397531:0:3352:0:(events.c:361:request_in_callback()) peer: 12345-0@lo (source: 12345-0@lo) 00000100:00080000:1.0:1776178625.397561:0:3352:0:(import.c:86:import_set_state_nolock()) 00000000dfc8db13 lustre-MDT0000_UUID: changing import state from CONNECTING to DISCONN 00000100:00080000:1.0:1776178625.397565:0:3352:0:(import.c:1429:ptlrpc_connect_interpret()) recovery of lustre-MDT0000_UUID on 192.168.204.103@tcp failed (-19) 00000100:00100000:0.0:1776178625.397568:0:11281:0:(service.c:2152:ptlrpc_server_handle_req_in()) got req x1862456998107776 00000100:00100000:1.0:1776178625.397569:0:3352:0:(client.c:2215:ptlrpc_check_set()) Completed RPC req@00000000f18ffde7 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:lustre-MDT0000-lwp-OST0001_UUID:3352:1862456998107456:0@lo:38:kworker/u8:0.0 00000100:00100000:1.0:1776178625.397598:0:14128:0:(service.c:2152:ptlrpc_server_handle_req_in()) got req x1862456998107584 00000100:00100000:1.0:1776178625.397623:0:14128:0:(nrs_fifo.c:179:nrs_fifo_req_get()) NRS start fifo request from 12345-0@lo, seq: 790 00000100:00100000:1.0:1776178625.397626:0:14128:0:(service.c:2324:ptlrpc_server_handle_request()) Handling RPC req@00000000fa20ba2c pname:cluuid+ref:pid:xid:nid:opc:job mdt_out00_002:0+-99:3352:x1862456998107584:12345-0@lo:38:kworker/u8:0.0 00010000:02000000:1.0:1776178625.397634:0:14128:0:(ldlm_lib.c:1139:target_handle_connect()) lustre-MDT0000: Not available for connect from 0@lo (stopping) 00000100:00100000:1.0:1776178625.397665:0:14128:0:(service.c:2372:ptlrpc_server_handle_request()) Handled RPC req@00000000fa20ba2c pname:cluuid+ref:pid:xid:nid:opc:job mdt_out00_002:0+-99:3352:x1862456998107584:12345-0@lo:38:kworker/u8:0.0 Request processed in 37us (306us total) trans 0 rc -19/-19 00000100:00100000:1.0:1776178625.397687:0:14128:0:(nrs_fifo.c:241:nrs_fifo_req_stop()) NRS stop fifo request from 12345-0@lo, seq: 790 00000100:00080000:1.0:1776178625.397717:0:3352:0:(import.c:86:import_set_state_nolock()) 00000000ba4f1821 lustre-MDT0000_UUID: changing import state from CONNECTING to DISCONN 00000100:00080000:1.0:1776178625.397720:0:3352:0:(import.c:1429:ptlrpc_connect_interpret()) recovery of lustre-MDT0000_UUID on 192.168.204.103@tcp failed (-19) 00000100:00100000:1.0:1776178625.397722:0:3352:0:(client.c:2215:ptlrpc_check_set()) Completed RPC req@00000000722afaa5 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:lustre-MDT0001-mdtlov_UUID:3352:1862456998107584:0@lo:38:kworker/u8:0.0 00000100:00100000:1.0:1776178625.397756:0:3355:0:(client.c:2215:ptlrpc_check_set()) Completed RPC req@00000000d1597b46 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_02:lustre-MDT0001-lwp-OST0000_UUID:3355:1862456998107392:0@lo:400:kworker/u8:0.0 00000100:00100000:1.0:1776178625.397779:0:3353:0:(events.c:361:request_in_callback()) peer: 12345-0@lo (source: 12345-0@lo) 00000100:00100000:0.0:1776178625.403084:0:11281:0:(service.c:2152:ptlrpc_server_handle_req_in()) got req x1862456998107520 00000100:00100000:2.0:1776178625.403113:0:6256:0:(nrs_fifo.c:179:nrs_fifo_req_get()) NRS start fifo request from 12345-0@lo, seq: 6955 00000100:00100000:2.0:1776178625.403123:0:6256:0:(service.c:2324:ptlrpc_server_handle_request()) Handling RPC req@00000000ae814754 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_001:0+-99:3352:x1862456998107776:12345-0@lo:38:kworker/u8:0.0 00010000:02000000:2.0:1776178625.403142:0:6256:0:(ldlm_lib.c:1139:target_handle_connect()) lustre-MDT0000: Not available for connect from 0@lo (stopping) 00000100:00100000:0.0:1776178625.403146:0:11281:0:(nrs_fifo.c:179:nrs_fifo_req_get()) NRS start fifo request from 12345-0@lo, seq: 1176 00000100:00100000:0.0:1776178625.403153:0:11281:0:(service.c:2324:ptlrpc_server_handle_request()) Handling RPC req@0000000011f8e42c pname:cluuid+ref:pid:xid:nid:opc:job mdt00_004:lustre-MDT0001-lwp-OST0001_UUID+5:3353:x1862456998107520:12345-0@lo:400:kworker/u8:0.0 00000100:00100000:0.0:1776178625.403301:0:11281:0:(service.c:2372:ptlrpc_server_handle_request()) Handled RPC req@0000000011f8e42c pname:cluuid+ref:pid:xid:nid:opc:job mdt00_004:lustre-MDT0001-lwp-OST0001_UUID+5:3353:x1862456998107520:12345-0@lo:400:kworker/u8:0.0 Request processed in 147us (5520us total) trans 0 rc 0/0 00000100:00100000:0.0:1776178625.403309:0:11281:0:(nrs_fifo.c:241:nrs_fifo_req_stop()) NRS stop fifo request from 12345-0@lo, seq: 1176 00000100:00100000:0.0:1776178625.406798:0:3353:0:(client.c:2215:ptlrpc_check_set()) Completed RPC req@00000000b069e162 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_00:lustre-MDT0001-lwp-OST0001_UUID:3353:1862456998107520:0@lo:400:kworker/u8:0.0 00000100:00100000:2.0:1776178625.406928:0:6256:0:(service.c:2372:ptlrpc_server_handle_request()) Handled RPC req@00000000ae814754 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_001:0+-99:3352:x1862456998107776:12345-0@lo:38:kworker/u8:0.0 Request processed in 3804us (9395us total) trans 0 rc -19/-19 00000100:00100000:2.0:1776178625.406941:0:6256:0:(nrs_fifo.c:241:nrs_fifo_req_stop()) NRS stop fifo request from 12345-0@lo, seq: 6955 00000100:00080000:1.0:1776178625.407125:0:3352:0:(import.c:86:import_set_state_nolock()) 00000000a7c55b1f lustre-MDT0000_UUID: changing import state from CONNECTING to DISCONN 00000100:00080000:1.0:1776178625.407130:0:3352:0:(import.c:1429:ptlrpc_connect_interpret()) recovery of lustre-MDT0000_UUID on 192.168.204.103@tcp failed (-19) 00000100:00100000:1.0:1776178625.407133:0:3352:0:(client.c:2215:ptlrpc_check_set()) Completed RPC req@00000000f20bd8f9 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:lustre-MDT0000-lwp-MDT0001_UUID:3352:1862456998107776:0@lo:38:kworker/u8:0.0 00010000:00010000:1.0:1776178626.016507:0:9225:0:(ldlm_lockd.c:2792:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:2.0:1776178626.912396:0:3356:0:(client.c:1733:ptlrpc_send_new_req()) Sending RPC req@0000000092e0e13d pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_03:lustre-MDT0001-mdtlov_UUID:3356:1862456998107840:0@lo:13:osp-pre-1-1.0 00000100:00100000:2.0:1776178626.912478:0:3356:0:(events.c:361:request_in_callback()) peer: 12345-0@lo (source: 12345-0@lo) 00000100:00100000:0.0:1776178626.912588:0:8231:0:(service.c:2152:ptlrpc_server_handle_req_in()) got req x1862456998107840 00000100:00100000:0.0:1776178626.912652:0:8231:0:(nrs_fifo.c:179:nrs_fifo_req_get()) NRS start fifo request from 12345-0@lo, seq: 653 00000100:00100000:0.0:1776178626.912657:0:8231:0:(service.c:2324:ptlrpc_server_handle_request()) Handling RPC req@00000000606ce17f pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_create00:lustre-MDT0001-mdtlov_UUID+5:3356:x1862456998107840:12345-0@lo:13:osp-pre-1-1.0 00002000:00000024:0.0:1776178626.912690:0:8231:0:(ofd_obd.c:740:ofd_statfs()) blocks cached 229376 granted 491746368 pending 0 free 3923689472 avail 3692167168 00000100:00100000:0.0:1776178626.913084:0:8231:0:(service.c:2372:ptlrpc_server_handle_request()) Handled RPC req@00000000606ce17f pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_create00:lustre-MDT0001-mdtlov_UUID+5:3356:x1862456998107840:12345-0@lo:13:osp-pre-1-1.0 Request processed in 425us (606us total) trans 0 rc 0/0 00000100:00100000:0.0:1776178626.913096:0:8231:0:(nrs_fifo.c:241:nrs_fifo_req_stop()) NRS stop fifo request from 12345-0@lo, seq: 653 00000100:00100000:2.0:1776178626.913175:0:3356:0:(client.c:2215:ptlrpc_check_set()) Completed RPC req@0000000092e0e13d pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_03:lustre-MDT0001-mdtlov_UUID:3356:1862456998107840:0@lo:13:osp-pre-1-1.0 00010000:00010000:2.0:1776178627.044288:0:6244:0:(ldlm_lockd.c:2792:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:0.0:1776178627.428548:0:3353:0:(client.c:1733:ptlrpc_send_new_req()) Sending RPC req@00000000c6b32ddb pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_00:lustre-MDT0001-mdtlov_UUID:3353:1862456998107904:0@lo:13:osp-pre-0-1.0 00000100:00100000:0.0:1776178627.428625:0:3353:0:(events.c:361:request_in_callback()) peer: 12345-0@lo (source: 12345-0@lo) 00000100:00100000:3.0:1776178627.430007:0:8231:0:(service.c:2152:ptlrpc_server_handle_req_in()) got req x1862456998107904 00000100:00100000:3.0:1776178627.430137:0:8231:0:(nrs_fifo.c:179:nrs_fifo_req_get()) NRS start fifo request from 12345-0@lo, seq: 654 00000100:00100000:3.0:1776178627.430144:0:8231:0:(service.c:2324:ptlrpc_server_handle_request()) Handling RPC req@00000000b069e162 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_create00:lustre-MDT0001-mdtlov_UUID+5:3353:x1862456998107904:12345-0@lo:13:osp-pre-0-1.0 00002000:00000024:3.0:1776178627.430182:0:8231:0:(ofd_obd.c:740:ofd_statfs()) blocks cached 315392 granted 484883776 pending 0 free 3923689472 avail 3692167168 00000100:00100000:3.0:1776178627.430260:0:8231:0:(service.c:2372:ptlrpc_server_handle_request()) Handled RPC req@00000000b069e162 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_create00:lustre-MDT0001-mdtlov_UUID+5:3353:x1862456998107904:12345-0@lo:13:osp-pre-0-1.0 Request processed in 115us (1635us total) trans 0 rc 0/0 00000100:00100000:3.0:1776178627.430266:0:8231:0:(nrs_fifo.c:241:nrs_fifo_req_stop()) NRS stop fifo request from 12345-0@lo, seq: 654 00000100:00100000:0.0:1776178627.430409:0:3353:0:(client.c:2215:ptlrpc_check_set()) Completed RPC req@00000000c6b32ddb pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_00:lustre-MDT0001-mdtlov_UUID:3353:1862456998107904:0@lo:13:osp-pre-0-1.0 00010000:00010000:1.0:1776178628.064305:0:6245:0:(ldlm_lockd.c:2792:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:0.0:1776178628.906241:0:3347:0:(events.c:361:request_in_callback()) peer: 12345-192.168.204.3@tcp (source: 12345-192.168.204.3@tcp) 00000100:00100000:0.0:1776178628.906281:0:3347:0:(events.c:361:request_in_callback()) peer: 12345-192.168.204.3@tcp (source: 12345-192.168.204.3@tcp) 00000100:00100000:0.0:1776178628.906311:0:3347:0:(events.c:361:request_in_callback()) peer: 12345-192.168.204.3@tcp (source: 12345-192.168.204.3@tcp) 00000100:00100000:0.0:1776178628.906348:0:3347:0:(events.c:361:request_in_callback()) peer: 12345-192.168.204.3@tcp (source: 12345-192.168.204.3@tcp) 00000100:00100000:0.0:1776178628.906391:0:20762:0:(service.c:2152:ptlrpc_server_handle_req_in()) got req x1862456979135744 00000100:00100000:1.0:1776178628.906396:0:14440:0:(service.c:2152:ptlrpc_server_handle_req_in()) got req x1862456979135808 00000100:00100000:2.0:1776178628.906472:0:6256:0:(service.c:2152:ptlrpc_server_handle_req_in()) got req x1862456979135680 00000100:00100000:2.0:1776178628.906510:0:6256:0:(nrs_fifo.c:179:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.204.3@tcp, seq: 1177 00000100:00100000:2.0:1776178628.906516:0:6256:0:(service.c:2324:ptlrpc_server_handle_request()) Handling RPC req@00000000c21d7f55 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_001:b50963bd-8eba-4432-9a54-8888d00e3fa1+6:2273:x1862456979135680:12345-192.168.204.3@tcp:400:kworker/u8:4.0 00000100:00100000:2.0:1776178628.906590:0:6256:0:(service.c:2372:ptlrpc_server_handle_request()) Handled RPC req@00000000c21d7f55 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_001:b50963bd-8eba-4432-9a54-8888d00e3fa1+6:2273:x1862456979135680:12345-192.168.204.3@tcp:400:kworker/u8:4.0 Request processed in 72us (308us total) trans 0 rc 0/0 00000100:00100000:2.0:1776178628.906597:0:6256:0:(nrs_fifo.c:241:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.204.3@tcp, seq: 1177 00000100:00100000:0.0:1776178628.907285:0:20762:0:(nrs_fifo.c:179:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.204.3@tcp, seq: 843 00000100:00100000:0.0:1776178628.907292:0:20762:0:(service.c:2324:ptlrpc_server_handle_request()) Handling RPC req@0000000012d88c00 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_013:b50963bd-8eba-4432-9a54-8888d00e3fa1+8:2272:x1862456979135744:12345-192.168.204.3@tcp:400:kworker/u8:4.0 00000100:00100000:0.0:1776178628.907329:0:20762:0:(service.c:2372:ptlrpc_server_handle_request()) Handled RPC req@0000000012d88c00 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_013:b50963bd-8eba-4432-9a54-8888d00e3fa1+8:2272:x1862456979135744:12345-192.168.204.3@tcp:400:kworker/u8:4.0 Request processed in 37us (1018us total) trans 0 rc 0/0 00000100:00100000:0.0:1776178628.907335:0:20762:0:(nrs_fifo.c:241:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.204.3@tcp, seq: 843 00000100:00100000:3.0:1776178628.909399:0:20140:0:(service.c:2152:ptlrpc_server_handle_req_in()) got req x1862456979135552 00000100:00100000:1.0:1776178628.909406:0:14440:0:(nrs_fifo.c:179:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.204.3@tcp, seq: 844 00000100:00100000:1.0:1776178628.909415:0:14440:0:(service.c:2324:ptlrpc_server_handle_request()) Handling RPC req@0000000080e59d72 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_004:b50963bd-8eba-4432-9a54-8888d00e3fa1+9:2273:x1862456979135808:12345-192.168.204.3@tcp:400:kworker/u8:4.0 00000100:00100000:3.0:1776178628.909451:0:20140:0:(nrs_fifo.c:179:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.204.3@tcp, seq: 321 00000100:00100000:3.0:1776178628.909458:0:20140:0:(service.c:2324:ptlrpc_server_handle_request()) Handling RPC req@00000000286b1e8d pname:cluuid+ref:pid:xid:nid:opc:job ll_mgs_0002:5b5c66b1-1a2f-4651-8548-0c75b9a38a94+9:2272:x1862456979135552:12345-192.168.204.3@tcp:400:kworker/u8:4.0 00000100:00100000:1.0:1776178628.909486:0:14440:0:(service.c:2372:ptlrpc_server_handle_request()) Handled RPC req@0000000080e59d72 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_004:b50963bd-8eba-4432-9a54-8888d00e3fa1+9:2273:x1862456979135808:12345-192.168.204.3@tcp:400:kworker/u8:4.0 Request processed in 70us (3136us total) trans 0 rc 0/0 00000100:00100000:3.0:1776178628.909494:0:20140:0:(service.c:2372:ptlrpc_server_handle_request()) Handled RPC req@00000000286b1e8d pname:cluuid+ref:pid:xid:nid:opc:job ll_mgs_0002:5b5c66b1-1a2f-4651-8548-0c75b9a38a94+9:2272:x1862456979135552:12345-192.168.204.3@tcp:400:kworker/u8:4.0 Request processed in 35us (3260us total) trans 0 rc 0/0 00000100:00100000:1.0:1776178628.909496:0:14440:0:(nrs_fifo.c:241:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.204.3@tcp, seq: 844 00000100:00100000:3.0:1776178628.909506:0:20140:0:(nrs_fifo.c:241:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.204.3@tcp, seq: 321 00000100:00100000:1.0:1776178628.910030:0:3348:0:(events.c:361:request_in_callback()) peer: 12345-192.168.204.3@tcp (source: 12345-192.168.204.3@tcp) 00000100:00100000:3.0:1776178628.910506:0:6255:0:(service.c:2152:ptlrpc_server_handle_req_in()) got req x1862456979135616 00000100:00100000:3.0:1776178628.910676:0:6255:0:(nrs_fifo.c:179:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.204.3@tcp, seq: 6956 00000100:00100000:3.0:1776178628.910681:0:6255:0:(service.c:2324:ptlrpc_server_handle_request()) Handling RPC req@0000000008dc7ab9 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_000:0+-99:2269:x1862456979135616:12345-192.168.204.3@tcp:38:kworker/u8:4.0 00010000:02000000:3.0:1776178628.910698:0:6255:0:(ldlm_lib.c:1139:target_handle_connect()) lustre-MDT0000: Not available for connect from 192.168.204.3@tcp (stopping) 00000100:00100000:3.0:1776178628.910752:0:6255:0:(service.c:2372:ptlrpc_server_handle_request()) Handled RPC req@0000000008dc7ab9 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_000:0+-99:2269:x1862456979135616:12345-192.168.204.3@tcp:38:kworker/u8:4.0 Request processed in 70us (1188us total) trans 0 rc -19/-19 00000100:00100000:3.0:1776178628.910759:0:6255:0:(nrs_fifo.c:241:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.204.3@tcp, seq: 6956 00010000:00010000:1.0:1776178629.091375:0:9224:0:(ldlm_lockd.c:2792:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00010000:00010000:1.0:1776178630.112299:0:9225:0:(ldlm_lockd.c:2792:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00080000:2.0:1776178630.497680:0:36:0:(pinger.c:251:ptlrpc_pinger_process_import()) lustre-MDT0000-lwp-OST0000_UUID->lustre-MDT0000_UUID: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00080000:2.0:1776178630.497689:0:36:0:(recover.c:57:ptlrpc_initiate_recovery()) lustre-MDT0000_UUID: starting recovery 00000100:00080000:2.0:1776178630.497692:0:36:0:(import.c:86:import_set_state_nolock()) 0000000047601a98 lustre-MDT0000_UUID: changing import state from DISCONN to CONNECTING 00000100:00080000:2.0:1776178630.497702:0:36:0:(import.c:534:import_select_connection()) lustre-MDT0000-lwp-OST0000: connect to NID 0@lo last attempt 1332 00000100:00080000:2.0:1776178630.497705:0:36:0:(import.c:581:import_select_connection()) lustre-MDT0000-lwp-OST0000: tried all connections, increasing latency to 21s 00000100:00080000:2.0:1776178630.497712:0:36:0:(import.c:615:import_select_connection()) lustre-MDT0000-lwp-OST0000: import 0000000047601a98 using connection 192.168.204.103@tcp/0@lo 00000100:00100000:2.0:1776178630.497735:0:36:0:(import.c:807:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5) req@000000006890aba2 x1862456998108032/t0(0) o38->lustre-MDT0000-lwp-OST0000@0@lo:12/10 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' 00000100:00080000:2.0:1776178630.497781:0:36:0:(pinger.c:251:ptlrpc_pinger_process_import()) lustre-MDT0000-lwp-OST0001_UUID->lustre-MDT0000_UUID: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00080000:2.0:1776178630.497785:0:36:0:(recover.c:57:ptlrpc_initiate_recovery()) lustre-MDT0000_UUID: starting recovery 00000100:00080000:2.0:1776178630.497786:0:36:0:(import.c:86:import_set_state_nolock()) 00000000dfc8db13 lustre-MDT0000_UUID: changing import state from DISCONN to CONNECTING 00000100:00080000:2.0:1776178630.497790:0:36:0:(import.c:534:import_select_connection()) lustre-MDT0000-lwp-OST0001: connect to NID 0@lo last attempt 1332 00000100:00080000:2.0:1776178630.497792:0:36:0:(import.c:581:import_select_connection()) lustre-MDT0000-lwp-OST0001: tried all connections, increasing latency to 21s 00000100:00080000:2.0:1776178630.497796:0:36:0:(import.c:615:import_select_connection()) lustre-MDT0000-lwp-OST0001: import 00000000dfc8db13 using connection 192.168.204.103@tcp/0@lo 00000100:00100000:2.0:1776178630.497804:0:36:0:(import.c:807:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5) req@0000000065f62d8c x1862456998108160/t0(0) o38->lustre-MDT0000-lwp-OST0001@0@lo:12/10 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' 00000100:00080000:2.0:1776178630.497819:0:36:0:(pinger.c:251:ptlrpc_pinger_process_import()) lustre-MDT0001-mdtlov_UUID->lustre-MDT0000_UUID: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00080000:2.0:1776178630.497822:0:36:0:(recover.c:57:ptlrpc_initiate_recovery()) lustre-MDT0000_UUID: starting recovery 00000100:00080000:2.0:1776178630.497823:0:36:0:(import.c:86:import_set_state_nolock()) 00000000ba4f1821 lustre-MDT0000_UUID: changing import state from DISCONN to CONNECTING 00000100:00080000:2.0:1776178630.497835:0:36:0:(import.c:534:import_select_connection()) lustre-MDT0000-osp-MDT0001: connect to NID 0@lo last attempt 1332 00000100:00080000:2.0:1776178630.497838:0:36:0:(import.c:581:import_select_connection()) lustre-MDT0000-osp-MDT0001: tried all connections, increasing latency to 21s 00000100:00080000:2.0:1776178630.497841:0:36:0:(import.c:615:import_select_connection()) lustre-MDT0000-osp-MDT0001: import 00000000ba4f1821 using connection 192.168.204.103@tcp/0@lo 00000100:00100000:2.0:1776178630.497852:0:36:0:(import.c:807:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5) req@00000000b6e1d5c6 x1862456998108288/t0(0) o38->lustre-MDT0000-osp-MDT0001@0@lo:24/4 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' 00000100:00080000:2.0:1776178630.497871:0:36:0:(pinger.c:251:ptlrpc_pinger_process_import()) lustre-MDT0000-lwp-MDT0001_UUID->lustre-MDT0000_UUID: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00080000:2.0:1776178630.497874:0:36:0:(recover.c:57:ptlrpc_initiate_recovery()) lustre-MDT0000_UUID: starting recovery 00000100:00080000:2.0:1776178630.497875:0:36:0:(import.c:86:import_set_state_nolock()) 00000000a7c55b1f lustre-MDT0000_UUID: changing import state from DISCONN to CONNECTING 00000100:00080000:2.0:1776178630.497885:0:36:0:(import.c:534:import_select_connection()) lustre-MDT0000-lwp-MDT0001: connect to NID 0@lo last attempt 1332 00000100:00080000:2.0:1776178630.497887:0:36:0:(import.c:581:import_select_connection()) lustre-MDT0000-lwp-MDT0001: tried all connections, increasing latency to 21s 00000100:00080000:2.0:1776178630.497890:0:36:0:(import.c:615:import_select_connection()) lustre-MDT0000-lwp-MDT0001: import 00000000a7c55b1f using connection 192.168.204.103@tcp/0@lo 00000100:00100000:2.0:1776178630.497899:0:36:0:(import.c:807:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5) req@00000000ec38a60b x1862456998108480/t0(0) o38->lustre-MDT0000-lwp-MDT0001@0@lo:12/10 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' 00000100:00100000:2.0:1776178630.497929:0:3356:0:(client.c:1733:ptlrpc_send_new_req()) Sending RPC req@000000009cee1481 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_03:lustre-MDT0001-lwp-OST0001_UUID:3356:1862456998108224:0@lo:400:kworker/u8:1.0 00000100:00100000:2.0:1776178630.497994:0:3356:0:(events.c:361:request_in_callback()) peer: 12345-0@lo (source: 12345-0@lo) 00000100:00100000:2.0:1776178630.498031:0:3356:0:(ptlrpcd.c:415:ptlrpcd_check()) transfer 1 async RPCs [2->3] 00000100:00100000:2.0:1776178630.498037:0:3356:0:(client.c:1733:ptlrpc_send_new_req()) Sending RPC req@00000000fd453e99 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_03:lustre-MDT0001-lwp-OST0000_UUID:3356:1862456998108096:0@lo:400:kworker/u8:1.0 00000100:00100000:2.0:1776178630.498085:0:3356:0:(events.c:361:request_in_callback()) peer: 12345-0@lo (source: 12345-0@lo) 00000100:00100000:2.0:1776178630.498123:0:6255:0:(service.c:2152:ptlrpc_server_handle_req_in()) got req x1862456998108224 00000100:00100000:0.0:1776178630.498142:0:3353:0:(client.c:1733:ptlrpc_send_new_req()) Sending RPC req@00000000936848a2 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_00:lustre-MDT0001-mdtlov_UUID:3353:1862456998108352:0@lo:400:kworker/u8:1.0 00000100:00100000:2.0:1776178630.498172:0:6255:0:(service.c:2152:ptlrpc_server_handle_req_in()) got req x1862456998108096 00000100:00100000:0.0:1776178630.498184:0:3353:0:(events.c:361:request_in_callback()) peer: 12345-0@lo (source: 12345-0@lo) 00000100:00100000:2.0:1776178630.498199:0:6255:0:(nrs_fifo.c:179:nrs_fifo_req_get()) NRS start fifo request from 12345-0@lo, seq: 1178 00000100:00100000:2.0:1776178630.498206:0:6255:0:(service.c:2324:ptlrpc_server_handle_request()) Handling RPC req@00000000c6b32ddb pname:cluuid+ref:pid:xid:nid:opc:job mdt00_000:lustre-MDT0001-lwp-OST0001_UUID+5:3356:x1862456998108224:12345-0@lo:400:kworker/u8:1.0 00000100:00100000:0.0:1776178630.498212:0:3353:0:(ptlrpcd.c:415:ptlrpcd_check()) transfer 2 async RPCs [1->0] 00000100:00100000:0.0:1776178630.498220:0:3353:0:(client.c:1733:ptlrpc_send_new_req()) Sending RPC req@000000005831fa36 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_00:ac61f839-0e8e-4cac-9c59-08bcfbbad330:3353:1862456998107968:0@lo:400:kworker/u8:1.0 00000100:00100000:2.0:1776178630.498262:0:6255:0:(service.c:2372:ptlrpc_server_handle_request()) Handled RPC req@00000000c6b32ddb pname:cluuid+ref:pid:xid:nid:opc:job mdt00_000:lustre-MDT0001-lwp-OST0001_UUID+5:3356:x1862456998108224:12345-0@lo:400:kworker/u8:1.0 Request processed in 56us (269us total) trans 0 rc 0/0 00000100:00100000:2.0:1776178630.498270:0:6255:0:(nrs_fifo.c:241:nrs_fifo_req_stop()) NRS stop fifo request from 12345-0@lo, seq: 1178 00000100:00100000:2.0:1776178630.498289:0:6255:0:(nrs_fifo.c:179:nrs_fifo_req_get()) NRS start fifo request from 12345-0@lo, seq: 1179 00000100:00100000:2.0:1776178630.498293:0:6255:0:(service.c:2324:ptlrpc_server_handle_request()) Handling RPC req@0000000023da22ef pname:cluuid+ref:pid:xid:nid:opc:job mdt00_000:lustre-MDT0001-lwp-OST0000_UUID+5:3356:x1862456998108096:12345-0@lo:400:kworker/u8:1.0 00000100:00100000:2.0:1776178630.498317:0:6255:0:(service.c:2372:ptlrpc_server_handle_request()) Handled RPC req@0000000023da22ef pname:cluuid+ref:pid:xid:nid:opc:job mdt00_000:lustre-MDT0001-lwp-OST0000_UUID+5:3356:x1862456998108096:12345-0@lo:400:kworker/u8:1.0 Request processed in 24us (233us total) trans 0 rc 0/0 00000100:00100000:3.0:1776178630.498805:0:3352:0:(client.c:1733:ptlrpc_send_new_req()) Sending RPC req@000000006890aba2 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:lustre-MDT0000-lwp-OST0000_UUID:3352:1862456998108032:0@lo:38:kworker/u8:1.0 00000100:00100000:0.0:1776178630.498864:0:3353:0:(events.c:361:request_in_callback()) peer: 12345-0@lo (source: 12345-0@lo) 00000100:00100000:2.0:1776178630.498902:0:6255:0:(nrs_fifo.c:241:nrs_fifo_req_stop()) NRS stop fifo request from 12345-0@lo, seq: 1179 00000100:00100000:1.0:1776178630.498977:0:20762:0:(service.c:2152:ptlrpc_server_handle_req_in()) got req x1862456998108352 00000100:00100000:1.0:1776178630.499008:0:20762:0:(nrs_fifo.c:179:nrs_fifo_req_get()) NRS start fifo request from 12345-0@lo, seq: 845 00000100:00100000:1.0:1776178630.499012:0:20762:0:(service.c:2324:ptlrpc_server_handle_request()) Handling RPC req@0000000048564d2a pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_013:lustre-MDT0001-mdtlov_UUID+5:3353:x1862456998108352:12345-0@lo:400:kworker/u8:1.0 00000100:00100000:3.0:1776178630.505943:0:3352:0:(events.c:361:request_in_callback()) peer: 12345-0@lo (source: 12345-0@lo) 00000100:00100000:3.0:1776178630.505994:0:3352:0:(client.c:1733:ptlrpc_send_new_req()) Sending RPC req@0000000065f62d8c pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:lustre-MDT0000-lwp-OST0001_UUID:3352:1862456998108160:0@lo:38:kworker/u8:1.0 00000100:00100000:3.0:1776178630.506028:0:3352:0:(events.c:361:request_in_callback()) peer: 12345-0@lo (source: 12345-0@lo) 00000100:00100000:3.0:1776178630.506045:0:3352:0:(client.c:1733:ptlrpc_send_new_req()) Sending RPC req@00000000b6e1d5c6 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:lustre-MDT0001-mdtlov_UUID:3352:1862456998108288:0@lo:38:kworker/u8:1.0 00000100:00100000:3.0:1776178630.506086:0:3352:0:(events.c:361:request_in_callback()) peer: 12345-0@lo (source: 12345-0@lo) 00000100:00100000:3.0:1776178630.506104:0:3352:0:(client.c:1733:ptlrpc_send_new_req()) Sending RPC req@00000000ec38a60b pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:lustre-MDT0000-lwp-MDT0001_UUID:3352:1862456998108480:0@lo:38:kworker/u8:1.0 00000100:00100000:3.0:1776178630.506125:0:3352:0:(events.c:361:request_in_callback()) peer: 12345-0@lo (source: 12345-0@lo) 00000100:00100000:3.0:1776178630.506185:0:15213:0:(service.c:2152:ptlrpc_server_handle_req_in()) got req x1862456998108288 00000100:00100000:3.0:1776178630.506225:0:15213:0:(nrs_fifo.c:179:nrs_fifo_req_get()) NRS start fifo request from 12345-0@lo, seq: 791 00000100:00100000:3.0:1776178630.506230:0:15213:0:(service.c:2324:ptlrpc_server_handle_request()) Handling RPC req@00000000dc38eb5b pname:cluuid+ref:pid:xid:nid:opc:job mdt_out00_003:0+-99:3352:x1862456998108288:12345-0@lo:38:kworker/u8:1.0 00010000:02000000:3.0:1776178630.506246:0:15213:0:(ldlm_lib.c:1139:target_handle_connect()) lustre-MDT0000: Not available for connect from 0@lo (stopping) 00000100:00100000:3.0:1776178630.506289:0:15213:0:(service.c:2372:ptlrpc_server_handle_request()) Handled RPC req@00000000dc38eb5b pname:cluuid+ref:pid:xid:nid:opc:job mdt_out00_003:0+-99:3352:x1862456998108288:12345-0@lo:38:kworker/u8:1.0 Request processed in 58us (202us total) trans 0 rc -19/-19 00000100:00100000:3.0:1776178630.506300:0:15213:0:(nrs_fifo.c:241:nrs_fifo_req_stop()) NRS stop fifo request from 12345-0@lo, seq: 791 00000100:00100000:3.0:1776178630.506328:0:6257:0:(service.c:2152:ptlrpc_server_handle_req_in()) got req x1862456998108032 00000100:00100000:3.0:1776178630.506347:0:6257:0:(service.c:2152:ptlrpc_server_handle_req_in()) got req x1862456998108160 00000100:00100000:3.0:1776178630.506359:0:6257:0:(service.c:2152:ptlrpc_server_handle_req_in()) got req x1862456998108480 00000100:00100000:3.0:1776178630.506367:0:6257:0:(nrs_fifo.c:179:nrs_fifo_req_get()) NRS start fifo request from 12345-0@lo, seq: 6957 00000100:00100000:3.0:1776178630.506370:0:6257:0:(service.c:2324:ptlrpc_server_handle_request()) Handling RPC req@0000000098623c2d pname:cluuid+ref:pid:xid:nid:opc:job mdt00_002:0+-99:3352:x1862456998108032:12345-0@lo:38:kworker/u8:1.0 00010000:02000000:3.0:1776178630.506376:0:6257:0:(ldlm_lib.c:1139:target_handle_connect()) lustre-MDT0000: Not available for connect from 0@lo (stopping) 00000100:00100000:3.0:1776178630.506399:0:6257:0:(service.c:2372:ptlrpc_server_handle_request()) Handled RPC req@0000000098623c2d pname:cluuid+ref:pid:xid:nid:opc:job mdt00_002:0+-99:3352:x1862456998108032:12345-0@lo:38:kworker/u8:1.0 Request processed in 29us (464us total) trans 0 rc -19/-19 00000100:00100000:3.0:1776178630.506405:0:6257:0:(nrs_fifo.c:241:nrs_fifo_req_stop()) NRS stop fifo request from 12345-0@lo, seq: 6957 00000100:00100000:3.0:1776178630.506415:0:6257:0:(nrs_fifo.c:179:nrs_fifo_req_get()) NRS start fifo request from 12345-0@lo, seq: 6958 00000100:00100000:3.0:1776178630.506418:0:6257:0:(service.c:2324:ptlrpc_server_handle_request()) Handling RPC req@0000000051c85b9b pname:cluuid+ref:pid:xid:nid:opc:job mdt00_002:0+-99:3352:x1862456998108160:12345-0@lo:38:kworker/u8:1.0 00010000:02000000:3.0:1776178630.506424:0:6257:0:(ldlm_lib.c:1139:target_handle_connect()) lustre-MDT0000: Not available for connect from 0@lo (stopping) 00000100:00100000:3.0:1776178630.506443:0:6257:0:(service.c:2372:ptlrpc_server_handle_request()) Handled RPC req@0000000051c85b9b pname:cluuid+ref:pid:xid:nid:opc:job mdt00_002:0+-99:3352:x1862456998108160:12345-0@lo:38:kworker/u8:1.0 Request processed in 24us (416us total) trans 0 rc -19/-19 00000100:00100000:3.0:1776178630.506449:0:6257:0:(nrs_fifo.c:241:nrs_fifo_req_stop()) NRS stop fifo request from 12345-0@lo, seq: 6958 00000100:00100000:3.0:1776178630.506459:0:6257:0:(nrs_fifo.c:179:nrs_fifo_req_get()) NRS start fifo request from 12345-0@lo, seq: 6959 00000100:00100000:3.0:1776178630.506462:0:6257:0:(service.c:2324:ptlrpc_server_handle_request()) Handling RPC req@0000000088512a81 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_002:0+-99:3352:x1862456998108480:12345-0@lo:38:kworker/u8:1.0 00010000:02000000:3.0:1776178630.506467:0:6257:0:(ldlm_lib.c:1139:target_handle_connect()) lustre-MDT0000: Not available for connect from 0@lo (stopping) 00000100:00100000:3.0:1776178630.506485:0:6257:0:(service.c:2372:ptlrpc_server_handle_request()) Handled RPC req@0000000088512a81 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_002:0+-99:3352:x1862456998108480:12345-0@lo:38:kworker/u8:1.0 Request processed in 22us (360us total) trans 0 rc -19/-19 00000100:00100000:3.0:1776178630.506490:0:6257:0:(nrs_fifo.c:241:nrs_fifo_req_stop()) NRS stop fifo request from 12345-0@lo, seq: 6959 00000100:00080000:3.0:1776178630.506673:0:3352:0:(import.c:86:import_set_state_nolock()) 0000000047601a98 lustre-MDT0000_UUID: changing import state from CONNECTING to DISCONN 00000100:00080000:3.0:1776178630.506677:0:3352:0:(import.c:1429:ptlrpc_connect_interpret()) recovery of lustre-MDT0000_UUID on 192.168.204.103@tcp failed (-19) 00000100:00100000:3.0:1776178630.506681:0:3352:0:(client.c:2215:ptlrpc_check_set()) Completed RPC req@000000006890aba2 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:lustre-MDT0000-lwp-OST0000_UUID:3352:1862456998108032:0@lo:38:kworker/u8:1.0 00000100:00080000:3.0:1776178630.506689:0:3352:0:(import.c:86:import_set_state_nolock()) 00000000dfc8db13 lustre-MDT0000_UUID: changing import state from CONNECTING to DISCONN 00000100:00080000:3.0:1776178630.506691:0:3352:0:(import.c:1429:ptlrpc_connect_interpret()) recovery of lustre-MDT0000_UUID on 192.168.204.103@tcp failed (-19) 00000100:00100000:3.0:1776178630.506694:0:3352:0:(client.c:2215:ptlrpc_check_set()) Completed RPC req@0000000065f62d8c pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:lustre-MDT0000-lwp-OST0001_UUID:3352:1862456998108160:0@lo:38:kworker/u8:1.0 00000100:00080000:3.0:1776178630.506700:0:3352:0:(import.c:86:import_set_state_nolock()) 00000000ba4f1821 lustre-MDT0000_UUID: changing import state from CONNECTING to DISCONN 00000100:00080000:3.0:1776178630.506702:0:3352:0:(import.c:1429:ptlrpc_connect_interpret()) recovery of lustre-MDT0000_UUID on 192.168.204.103@tcp failed (-19) 00000100:00100000:3.0:1776178630.506704:0:3352:0:(client.c:2215:ptlrpc_check_set()) Completed RPC req@00000000b6e1d5c6 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:lustre-MDT0001-mdtlov_UUID:3352:1862456998108288:0@lo:38:kworker/u8:1.0 00000100:00080000:3.0:1776178630.506711:0:3352:0:(import.c:86:import_set_state_nolock()) 00000000a7c55b1f lustre-MDT0000_UUID: changing import state from CONNECTING to DISCONN 00000100:00080000:3.0:1776178630.506713:0:3352:0:(import.c:1429:ptlrpc_connect_interpret()) recovery of lustre-MDT0000_UUID on 192.168.204.103@tcp failed (-19) 00000100:00100000:3.0:1776178630.506715:0:3352:0:(client.c:2215:ptlrpc_check_set()) Completed RPC req@00000000ec38a60b pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:lustre-MDT0000-lwp-MDT0001_UUID:3352:1862456998108480:0@lo:38:kworker/u8:1.0 00000100:00100000:3.0:1776178630.506916:0:20137:0:(service.c:2152:ptlrpc_server_handle_req_in()) got req x1862456998107968 00000100:00100000:3.0:1776178630.506946:0:20137:0:(nrs_fifo.c:179:nrs_fifo_req_get()) NRS start fifo request from 12345-0@lo, seq: 322 00000100:00100000:3.0:1776178630.506950:0:20137:0:(service.c:2324:ptlrpc_server_handle_request()) Handling RPC req@00000000fd0a07ee pname:cluuid+ref:pid:xid:nid:opc:job ll_mgs_0000:ac61f839-0e8e-4cac-9c59-08bcfbbad330+18:3353:x1862456998107968:12345-0@lo:400:kworker/u8:1.0 00000100:00100000:3.0:1776178630.506990:0:20137:0:(service.c:2372:ptlrpc_server_handle_request()) Handled RPC req@00000000fd0a07ee pname:cluuid+ref:pid:xid:nid:opc:job ll_mgs_0000:ac61f839-0e8e-4cac-9c59-08bcfbbad330+18:3353:x1862456998107968:12345-0@lo:400:kworker/u8:1.0 Request processed in 39us (8128us total) trans 0 rc 0/0 00000100:00100000:3.0:1776178630.506995:0:20137:0:(nrs_fifo.c:241:nrs_fifo_req_stop()) NRS stop fifo request from 12345-0@lo, seq: 322 00000100:00100000:0.0:1776178630.512839:0:3353:0:(client.c:2215:ptlrpc_check_set()) Completed RPC req@000000005831fa36 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_00:ac61f839-0e8e-4cac-9c59-08bcfbbad330:3353:1862456998107968:0@lo:400:kworker/u8:1.0 00000100:00100000:0.0:1776178630.512864:0:3353:0:(client.c:1733:ptlrpc_send_new_req()) Sending RPC req@00000000d23b79a7 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_00:lustre-MDT0001-mdtlov_UUID:3353:1862456998108416:0@lo:400:kworker/u8:1.0 00000100:00100000:0.0:1776178630.512933:0:3353:0:(events.c:361:request_in_callback()) peer: 12345-0@lo (source: 12345-0@lo) 00000100:00100000:3.0:1776178630.512989:0:20755:0:(service.c:2152:ptlrpc_server_handle_req_in()) got req x1862456998108416 00000100:00100000:3.0:1776178630.513074:0:20755:0:(nrs_fifo.c:179:nrs_fifo_req_get()) NRS start fifo request from 12345-0@lo, seq: 846 00000100:00100000:3.0:1776178630.513081:0:20755:0:(service.c:2324:ptlrpc_server_handle_request()) Handling RPC req@00000000b766fbe9 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_006:lustre-MDT0001-mdtlov_UUID+5:3353:x1862456998108416:12345-0@lo:400:kworker/u8:1.0 00000100:00100000:3.0:1776178630.513144:0:20755:0:(service.c:2372:ptlrpc_server_handle_request()) Handled RPC req@00000000b766fbe9 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_006:lustre-MDT0001-mdtlov_UUID+5:3353:x1862456998108416:12345-0@lo:400:kworker/u8:1.0 Request processed in 63us (212us total) trans 0 rc 0/0 00000100:00100000:3.0:1776178630.513151:0:20755:0:(nrs_fifo.c:241:nrs_fifo_req_stop()) NRS stop fifo request from 12345-0@lo, seq: 846 00000100:00100000:0.0:1776178630.513586:0:3353:0:(client.c:2215:ptlrpc_check_set()) Completed RPC req@00000000d23b79a7 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_00:lustre-MDT0001-mdtlov_UUID:3353:1862456998108416:0@lo:400:kworker/u8:1.0 00000100:00100000:2.0:1776178630.513636:0:3356:0:(client.c:2215:ptlrpc_check_set()) Completed RPC req@00000000fd453e99 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_03:lustre-MDT0001-lwp-OST0000_UUID:3356:1862456998108096:0@lo:400:kworker/u8:1.0 00000100:00100000:2.0:1776178630.513650:0:3356:0:(client.c:2215:ptlrpc_check_set()) Completed RPC req@000000009cee1481 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_03:lustre-MDT0001-lwp-OST0001_UUID:3356:1862456998108224:0@lo:400:kworker/u8:1.0 00000100:00100000:1.0:1776178630.513934:0:20762:0:(service.c:2372:ptlrpc_server_handle_request()) Handled RPC req@0000000048564d2a pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_013:lustre-MDT0001-mdtlov_UUID+5:3353:x1862456998108352:12345-0@lo:400:kworker/u8:1.0 Request processed in 14920us (15750us total) trans 0 rc 0/0 00000100:00100000:1.0:1776178630.513942:0:20762:0:(nrs_fifo.c:241:nrs_fifo_req_stop()) NRS stop fifo request from 12345-0@lo, seq: 845 00000100:00100000:0.0:1776178630.514052:0:3353:0:(client.c:2215:ptlrpc_check_set()) Completed RPC req@00000000936848a2 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_00:lustre-MDT0001-mdtlov_UUID:3353:1862456998108352:0@lo:400:kworker/u8:1.0 00010000:00010000:2.0:1776178631.137579:0:6244:0:(ldlm_lockd.c:2792:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:1.0:1776178632.036098:0:3355:0:(client.c:1733:ptlrpc_send_new_req()) Sending RPC req@0000000086665d0f pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_02:lustre-MDT0001-mdtlov_UUID:3355:1862456998108544:0@lo:13:osp-pre-1-1.0 00000100:00100000:1.0:1776178632.036183:0:3355:0:(events.c:361:request_in_callback()) peer: 12345-0@lo (source: 12345-0@lo) 00000100:00100000:0.0:1776178632.037010:0:9554:0:(service.c:2152:ptlrpc_server_handle_req_in()) got req x1862456998108544 00000100:00100000:0.0:1776178632.037094:0:9554:0:(nrs_fifo.c:179:nrs_fifo_req_get()) NRS start fifo request from 12345-0@lo, seq: 655 00000100:00100000:0.0:1776178632.037101:0:9554:0:(service.c:2324:ptlrpc_server_handle_request()) Handling RPC req@00000000ce34953a pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_create00:lustre-MDT0001-mdtlov_UUID+5:3355:x1862456998108544:12345-0@lo:13:osp-pre-1-1.0 00002000:00000024:0.0:1776178632.037130:0:9554:0:(ofd_obd.c:740:ofd_statfs()) blocks cached 229376 granted 491746368 pending 0 free 3923689472 avail 3692167168 00000100:00100000:0.0:1776178632.037192:0:9554:0:(service.c:2372:ptlrpc_server_handle_request()) Handled RPC req@00000000ce34953a pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_create00:lustre-MDT0001-mdtlov_UUID+5:3355:x1862456998108544:12345-0@lo:13:osp-pre-1-1.0 Request processed in 90us (1011us total) trans 0 rc 0/0 00000100:00100000:0.0:1776178632.037198:0:9554:0:(nrs_fifo.c:241:nrs_fifo_req_stop()) NRS stop fifo request from 12345-0@lo, seq: 655 00000100:00100000:1.0:1776178632.039450:0:3355:0:(client.c:2215:ptlrpc_check_set()) Completed RPC req@0000000086665d0f pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_02:lustre-MDT0001-mdtlov_UUID:3355:1862456998108544:0@lo:13:osp-pre-1-1.0 00010000:00010000:1.0:1776178632.160644:0:6245:0:(ldlm_lockd.c:2792:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:0.0:1776178632.547966:0:3356:0:(client.c:1733:ptlrpc_send_new_req()) Sending RPC req@00000000efb13deb pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_03:lustre-MDT0001-mdtlov_UUID:3356:1862456998108608:0@lo:13:osp-pre-0-1.0 00000100:00100000:0.0:1776178632.548045:0:3356:0:(events.c:361:request_in_callback()) peer: 12345-0@lo (source: 12345-0@lo) 00000100:00100000:3.0:1776178632.548834:0:8231:0:(service.c:2152:ptlrpc_server_handle_req_in()) got req x1862456998108608 00000100:00100000:3.0:1776178632.548897:0:8231:0:(nrs_fifo.c:179:nrs_fifo_req_get()) NRS start fifo request from 12345-0@lo, seq: 656 00000100:00100000:3.0:1776178632.548902:0:8231:0:(service.c:2324:ptlrpc_server_handle_request()) Handling RPC req@0000000021958fc7 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_create00:lustre-MDT0001-mdtlov_UUID+5:3356:x1862456998108608:12345-0@lo:13:osp-pre-0-1.0 00002000:00000024:3.0:1776178632.548933:0:8231:0:(ofd_obd.c:740:ofd_statfs()) blocks cached 315392 granted 484883776 pending 0 free 3923689472 avail 3692167168 00000100:00100000:3.0:1776178632.548991:0:8231:0:(service.c:2372:ptlrpc_server_handle_request()) Handled RPC req@0000000021958fc7 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_create00:lustre-MDT0001-mdtlov_UUID+5:3356:x1862456998108608:12345-0@lo:13:osp-pre-0-1.0 Request processed in 88us (948us total) trans 0 rc 0/0 00000100:00100000:3.0:1776178632.548998:0:8231:0:(nrs_fifo.c:241:nrs_fifo_req_stop()) NRS stop fifo request from 12345-0@lo, seq: 656 00000100:00100000:2.0:1776178632.549120:0:3356:0:(client.c:2215:ptlrpc_check_set()) Completed RPC req@00000000efb13deb pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_03:lustre-MDT0001-mdtlov_UUID:3356:1862456998108608:0@lo:13:osp-pre-0-1.0 00010000:00010000:1.0:1776178633.184740:0:9224:0:(ldlm_lockd.c:2792:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:1.0:1776178634.021218:0:3348:0:(events.c:361:request_in_callback()) peer: 12345-192.168.204.3@tcp (source: 12345-192.168.204.3@tcp) 00000100:00100000:1.0:1776178634.021462:0:3348:0:(events.c:361:request_in_callback()) peer: 12345-192.168.204.3@tcp (source: 12345-192.168.204.3@tcp) 00000100:00100000:1.0:1776178634.021522:0:3348:0:(events.c:361:request_in_callback()) peer: 12345-192.168.204.3@tcp (source: 12345-192.168.204.3@tcp) 00000100:00100000:1.0:1776178634.021553:0:3348:0:(events.c:361:request_in_callback()) peer: 12345-192.168.204.3@tcp (source: 12345-192.168.204.3@tcp) 00000100:00100000:3.0:1776178634.021602:0:6256:0:(service.c:2152:ptlrpc_server_handle_req_in()) got req x1862456979136000 00000100:00100000:1.0:1776178634.021637:0:3348:0:(events.c:361:request_in_callback()) peer: 12345-192.168.204.3@tcp (source: 12345-192.168.204.3@tcp) 00000100:00100000:0.0:1776178634.021683:0:20140:0:(service.c:2152:ptlrpc_server_handle_req_in()) got req x1862456979135872 00000100:00100000:0.0:1776178634.021717:0:20140:0:(nrs_fifo.c:179:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.204.3@tcp, seq: 323 00000100:00100000:0.0:1776178634.021724:0:20140:0:(service.c:2324:ptlrpc_server_handle_request()) Handling RPC req@00000000f240c829 pname:cluuid+ref:pid:xid:nid:opc:job ll_mgs_0002:5b5c66b1-1a2f-4651-8548-0c75b9a38a94+9:2272:x1862456979135872:12345-192.168.204.3@tcp:400:kworker/u8:3.0 00000100:00100000:0.0:1776178634.021790:0:20140:0:(service.c:2372:ptlrpc_server_handle_request()) Handled RPC req@00000000f240c829 pname:cluuid+ref:pid:xid:nid:opc:job ll_mgs_0002:5b5c66b1-1a2f-4651-8548-0c75b9a38a94+9:2272:x1862456979135872:12345-192.168.204.3@tcp:400:kworker/u8:3.0 Request processed in 66us (581us total) trans 0 rc 0/0 00000100:00100000:0.0:1776178634.021798:0:20140:0:(nrs_fifo.c:241:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.204.3@tcp, seq: 323 00000100:00100000:0.0:1776178634.021834:0:14440:0:(service.c:2152:ptlrpc_server_handle_req_in()) got req x1862456979136064 00000100:00100000:0.0:1776178634.021856:0:14440:0:(service.c:2152:ptlrpc_server_handle_req_in()) got req x1862456979136128 00000100:00100000:0.0:1776178634.021872:0:14440:0:(nrs_fifo.c:179:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.204.3@tcp, seq: 847 00000100:00100000:0.0:1776178634.021876:0:14440:0:(service.c:2324:ptlrpc_server_handle_request()) Handling RPC req@00000000e4ab0682 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_004:b50963bd-8eba-4432-9a54-8888d00e3fa1+8:2272:x1862456979136064:12345-192.168.204.3@tcp:400:kworker/u8:3.0 00000100:00100000:0.0:1776178634.021903:0:14440:0:(service.c:2372:ptlrpc_server_handle_request()) Handled RPC req@00000000e4ab0682 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_004:b50963bd-8eba-4432-9a54-8888d00e3fa1+8:2272:x1862456979136064:12345-192.168.204.3@tcp:400:kworker/u8:3.0 Request processed in 26us (442us total) trans 0 rc 0/0 00000100:00100000:0.0:1776178634.021909:0:14440:0:(nrs_fifo.c:241:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.204.3@tcp, seq: 847 00000100:00100000:0.0:1776178634.021921:0:14440:0:(nrs_fifo.c:179:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.204.3@tcp, seq: 848 00000100:00100000:0.0:1776178634.021926:0:14440:0:(service.c:2324:ptlrpc_server_handle_request()) Handling RPC req@000000006c0c7de5 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_004:b50963bd-8eba-4432-9a54-8888d00e3fa1+9:2273:x1862456979136128:12345-192.168.204.3@tcp:400:kworker/u8:3.0 00000100:00100000:0.0:1776178634.021963:0:14440:0:(service.c:2372:ptlrpc_server_handle_request()) Handled RPC req@000000006c0c7de5 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_004:b50963bd-8eba-4432-9a54-8888d00e3fa1+9:2273:x1862456979136128:12345-192.168.204.3@tcp:400:kworker/u8:3.0 Request processed in 37us (411us total) trans 0 rc 0/0 00000100:00100000:0.0:1776178634.021970:0:14440:0:(nrs_fifo.c:241:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.204.3@tcp, seq: 848 00000100:00100000:3.0:1776178634.023350:0:6256:0:(service.c:2152:ptlrpc_server_handle_req_in()) got req x1862456979135936 00000100:00100000:3.0:1776178634.023392:0:6256:0:(nrs_fifo.c:179:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.204.3@tcp, seq: 1180 00000100:00100000:3.0:1776178634.023399:0:6256:0:(service.c:2324:ptlrpc_server_handle_request()) Handling RPC req@000000000eda5b77 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_001:b50963bd-8eba-4432-9a54-8888d00e3fa1+6:2273:x1862456979136000:12345-192.168.204.3@tcp:400:kworker/u8:3.0 00000100:00100000:3.0:1776178634.023471:0:6256:0:(service.c:2372:ptlrpc_server_handle_request()) Handled RPC req@000000000eda5b77 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_001:b50963bd-8eba-4432-9a54-8888d00e3fa1+6:2273:x1862456979136000:12345-192.168.204.3@tcp:400:kworker/u8:3.0 Request processed in 63us (1940us total) trans 0 rc 0/0 00000100:00100000:3.0:1776178634.023478:0:6256:0:(nrs_fifo.c:241:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.204.3@tcp, seq: 1180 00000100:00100000:3.0:1776178634.023492:0:6256:0:(nrs_fifo.c:179:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.204.3@tcp, seq: 6960 00000100:00100000:3.0:1776178634.023495:0:6256:0:(service.c:2324:ptlrpc_server_handle_request()) Handling RPC req@000000002c128264 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_001:0+-99:2269:x1862456979135936:12345-192.168.204.3@tcp:38:kworker/u8:3.0 00010000:02000000:3.0:1776178634.023517:0:6256:0:(ldlm_lib.c:1139:target_handle_connect()) lustre-MDT0000: Not available for connect from 192.168.204.3@tcp (stopping) 00000100:00100000:3.0:1776178634.023544:0:6256:0:(service.c:2372:ptlrpc_server_handle_request()) Handled RPC req@000000002c128264 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_001:0+-99:2269:x1862456979135936:12345-192.168.204.3@tcp:38:kworker/u8:3.0 Request processed in 47us (1908us total) trans 0 rc -19/-19 00000100:00100000:3.0:1776178634.023550:0:6256:0:(nrs_fifo.c:241:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.204.3@tcp, seq: 6960 00010000:00010000:1.0:1776178634.217329:0:9225:0:(ldlm_lockd.c:2792:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00010000:00010000:2.0:1776178635.232680:0:6244:0:(ldlm_lockd.c:2792:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00080000:2.0:1776178635.617195:0:36:0:(pinger.c:251:ptlrpc_pinger_process_import()) lustre-MDT0000-lwp-OST0000_UUID->lustre-MDT0000_UUID: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00080000:2.0:1776178635.617201:0:36:0:(recover.c:57:ptlrpc_initiate_recovery()) lustre-MDT0000_UUID: starting recovery 00000100:00080000:2.0:1776178635.617205:0:36:0:(import.c:86:import_set_state_nolock()) 0000000047601a98 lustre-MDT0000_UUID: changing import state from DISCONN to CONNECTING 00000100:00080000:2.0:1776178635.617212:0:36:0:(import.c:534:import_select_connection()) lustre-MDT0000-lwp-OST0000: connect to NID 0@lo last attempt 1337 00000100:00080000:2.0:1776178635.617215:0:36:0:(import.c:581:import_select_connection()) lustre-MDT0000-lwp-OST0000: tried all connections, increasing latency to 21s 00000100:00080000:2.0:1776178635.617221:0:36:0:(import.c:615:import_select_connection()) lustre-MDT0000-lwp-OST0000: import 0000000047601a98 using connection 192.168.204.103@tcp/0@lo 00000100:00100000:2.0:1776178635.617239:0:36:0:(import.c:807:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5) req@0000000027e376cf x1862456998108736/t0(0) o38->lustre-MDT0000-lwp-OST0000@0@lo:12/10 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' 00000100:00080000:2.0:1776178635.617262:0:36:0:(pinger.c:251:ptlrpc_pinger_process_import()) lustre-MDT0000-lwp-OST0001_UUID->lustre-MDT0000_UUID: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00080000:2.0:1776178635.617265:0:36:0:(recover.c:57:ptlrpc_initiate_recovery()) lustre-MDT0000_UUID: starting recovery 00000100:00080000:2.0:1776178635.617266:0:36:0:(import.c:86:import_set_state_nolock()) 00000000dfc8db13 lustre-MDT0000_UUID: changing import state from DISCONN to CONNECTING 00000100:00080000:2.0:1776178635.617269:0:36:0:(import.c:534:import_select_connection()) lustre-MDT0000-lwp-OST0001: connect to NID 0@lo last attempt 1337 00000100:00080000:2.0:1776178635.617271:0:36:0:(import.c:581:import_select_connection()) lustre-MDT0000-lwp-OST0001: tried all connections, increasing latency to 21s 00000100:00100000:1.0:1776178635.617350:0:3354:0:(client.c:1733:ptlrpc_send_new_req()) Sending RPC req@0000000024989815 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_01:lustre-MDT0001-lwp-OST0000_UUID:3354:1862456998108800:0@lo:400:kworker/u8:1.0 00000100:00100000:1.0:1776178635.617410:0:3354:0:(events.c:361:request_in_callback()) peer: 12345-0@lo (source: 12345-0@lo) 00000100:00100000:1.0:1776178635.617449:0:3354:0:(ptlrpcd.c:415:ptlrpcd_check()) transfer 1 async RPCs [0->1] 00000100:00100000:1.0:1776178635.617454:0:3354:0:(client.c:1733:ptlrpc_send_new_req()) Sending RPC req@00000000efb13deb pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_01:ac61f839-0e8e-4cac-9c59-08bcfbbad330:3354:1862456998108672:0@lo:400:kworker/u8:1.0 00000100:00100000:1.0:1776178635.617492:0:3354:0:(events.c:361:request_in_callback()) peer: 12345-0@lo (source: 12345-0@lo) 00000100:00100000:1.0:1776178635.617524:0:8250:0:(service.c:2152:ptlrpc_server_handle_req_in()) got req x1862456998108800 00000100:00100000:1.0:1776178635.617560:0:8250:0:(nrs_fifo.c:179:nrs_fifo_req_get()) NRS start fifo request from 12345-0@lo, seq: 1181 00000100:00100000:1.0:1776178635.617565:0:8250:0:(service.c:2324:ptlrpc_server_handle_request()) Handling RPC req@00000000c989e78f pname:cluuid+ref:pid:xid:nid:opc:job mdt00_003:lustre-MDT0001-lwp-OST0000_UUID+5:3354:x1862456998108800:12345-0@lo:400:kworker/u8:1.0 00000100:00100000:1.0:1776178635.617605:0:8250:0:(service.c:2372:ptlrpc_server_handle_request()) Handled RPC req@00000000c989e78f pname:cluuid+ref:pid:xid:nid:opc:job mdt00_003:lustre-MDT0001-lwp-OST0000_UUID+5:3354:x1862456998108800:12345-0@lo:400:kworker/u8:1.0 Request processed in 40us (197us total) trans 0 rc 0/0 00000100:00100000:1.0:1776178635.617611:0:8250:0:(nrs_fifo.c:241:nrs_fifo_req_stop()) NRS stop fifo request from 12345-0@lo, seq: 1181 00000100:00100000:1.0:1776178635.617631:0:20137:0:(service.c:2152:ptlrpc_server_handle_req_in()) got req x1862456998108672 00000100:00100000:1.0:1776178635.617652:0:20137:0:(nrs_fifo.c:179:nrs_fifo_req_get()) NRS start fifo request from 12345-0@lo, seq: 324 00000100:00100000:1.0:1776178635.617655:0:20137:0:(service.c:2324:ptlrpc_server_handle_request()) Handling RPC req@0000000091f9a08d pname:cluuid+ref:pid:xid:nid:opc:job ll_mgs_0000:ac61f839-0e8e-4cac-9c59-08bcfbbad330+18:3354:x1862456998108672:12345-0@lo:400:kworker/u8:1.0 00000100:00100000:1.0:1776178635.617675:0:20137:0:(service.c:2372:ptlrpc_server_handle_request()) Handled RPC req@0000000091f9a08d pname:cluuid+ref:pid:xid:nid:opc:job ll_mgs_0000:ac61f839-0e8e-4cac-9c59-08bcfbbad330+18:3354:x1862456998108672:12345-0@lo:400:kworker/u8:1.0 Request processed in 20us (184us total) trans 0 rc 0/0 00000100:00100000:1.0:1776178635.617680:0:20137:0:(nrs_fifo.c:241:nrs_fifo_req_stop()) NRS stop fifo request from 12345-0@lo, seq: 324 00000100:00100000:1.0:1776178635.617718:0:3354:0:(client.c:2215:ptlrpc_check_set()) Completed RPC req@00000000efb13deb pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_01:ac61f839-0e8e-4cac-9c59-08bcfbbad330:3354:1862456998108672:0@lo:400:kworker/u8:1.0 00000100:00100000:1.0:1776178635.617728:0:3354:0:(client.c:2215:ptlrpc_check_set()) Completed RPC req@0000000024989815 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_01:lustre-MDT0001-lwp-OST0000_UUID:3354:1862456998108800:0@lo:400:kworker/u8:1.0 00000100:00100000:1.0:1776178635.617757:0:3352:0:(client.c:1733:ptlrpc_send_new_req()) Sending RPC req@0000000027e376cf pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:lustre-MDT0000-lwp-OST0000_UUID:3352:1862456998108736:0@lo:38:kworker/u8:1.0 00000100:00100000:1.0:1776178635.617777:0:3352:0:(events.c:361:request_in_callback()) peer: 12345-0@lo (source: 12345-0@lo) 00000100:00100000:1.0:1776178635.617796:0:6255:0:(service.c:2152:ptlrpc_server_handle_req_in()) got req x1862456998108736 00000100:00100000:1.0:1776178635.617809:0:6255:0:(nrs_fifo.c:179:nrs_fifo_req_get()) NRS start fifo request from 12345-0@lo, seq: 6961 00000100:00100000:1.0:1776178635.617813:0:6255:0:(service.c:2324:ptlrpc_server_handle_request()) Handling RPC req@000000006159eedd pname:cluuid+ref:pid:xid:nid:opc:job mdt00_000:0+-99:3352:x1862456998108736:12345-0@lo:38:kworker/u8:1.0 00010000:02000000:1.0:1776178635.617824:0:6255:0:(ldlm_lib.c:1139:target_handle_connect()) lustre-MDT0000: Not available for connect from 0@lo (stopping) 00000100:00100000:1.0:1776178635.617847:0:6255:0:(service.c:2372:ptlrpc_server_handle_request()) Handled RPC req@000000006159eedd pname:cluuid+ref:pid:xid:nid:opc:job mdt00_000:0+-99:3352:x1862456998108736:12345-0@lo:38:kworker/u8:1.0 Request processed in 34us (71us total) trans 0 rc -19/-19 00000100:00100000:1.0:1776178635.617852:0:6255:0:(nrs_fifo.c:241:nrs_fifo_req_stop()) NRS stop fifo request from 12345-0@lo, seq: 6961 00000100:00080000:1.0:1776178635.617868:0:3352:0:(import.c:86:import_set_state_nolock()) 0000000047601a98 lustre-MDT0000_UUID: changing import state from CONNECTING to DISCONN 00000100:00080000:1.0:1776178635.617871:0:3352:0:(import.c:1429:ptlrpc_connect_interpret()) recovery of lustre-MDT0000_UUID on 192.168.204.103@tcp failed (-19) 00000100:00100000:1.0:1776178635.617873:0:3352:0:(client.c:2215:ptlrpc_check_set()) Completed RPC req@0000000027e376cf pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:lustre-MDT0000-lwp-OST0000_UUID:3352:1862456998108736:0@lo:38:kworker/u8:1.0 00000100:00080000:2.0:1776178635.618043:0:36:0:(import.c:615:import_select_connection()) lustre-MDT0000-lwp-OST0001: import 00000000dfc8db13 using connection 192.168.204.103@tcp/0@lo 00000100:00100000:2.0:1776178635.618058:0:36:0:(import.c:807:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5) req@00000000eb400454 x1862456998108864/t0(0) o38->lustre-MDT0000-lwp-OST0001@0@lo:12/10 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' 00000100:00080000:2.0:1776178635.618110:0:36:0:(pinger.c:251:ptlrpc_pinger_process_import()) lustre-MDT0001-mdtlov_UUID->lustre-MDT0000_UUID: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00080000:2.0:1776178635.618115:0:36:0:(recover.c:57:ptlrpc_initiate_recovery()) lustre-MDT0000_UUID: starting recovery 00000100:00080000:2.0:1776178635.618117:0:36:0:(import.c:86:import_set_state_nolock()) 00000000ba4f1821 lustre-MDT0000_UUID: changing import state from DISCONN to CONNECTING 00000100:00080000:2.0:1776178635.618121:0:36:0:(import.c:534:import_select_connection()) lustre-MDT0000-osp-MDT0001: connect to NID 0@lo last attempt 1337 00000100:00080000:2.0:1776178635.618122:0:36:0:(import.c:581:import_select_connection()) lustre-MDT0000-osp-MDT0001: tried all connections, increasing latency to 21s 00000100:00080000:2.0:1776178635.618126:0:36:0:(import.c:615:import_select_connection()) lustre-MDT0000-osp-MDT0001: import 00000000ba4f1821 using connection 192.168.204.103@tcp/0@lo 00000100:00100000:2.0:1776178635.618134:0:36:0:(import.c:807:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5) req@00000000e2b18a6d x1862456998108992/t0(0) o38->lustre-MDT0000-osp-MDT0001@0@lo:24/4 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' 00000100:00080000:2.0:1776178635.618162:0:36:0:(pinger.c:251:ptlrpc_pinger_process_import()) lustre-MDT0000-lwp-MDT0001_UUID->lustre-MDT0000_UUID: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00080000:2.0:1776178635.618165:0:36:0:(recover.c:57:ptlrpc_initiate_recovery()) lustre-MDT0000_UUID: starting recovery 00000100:00080000:2.0:1776178635.618166:0:36:0:(import.c:86:import_set_state_nolock()) 00000000a7c55b1f lustre-MDT0000_UUID: changing import state from DISCONN to CONNECTING 00000100:00080000:2.0:1776178635.618168:0:36:0:(import.c:534:import_select_connection()) lustre-MDT0000-lwp-MDT0001: connect to NID 0@lo last attempt 1337 00000100:00080000:2.0:1776178635.618170:0:36:0:(import.c:581:import_select_connection()) lustre-MDT0000-lwp-MDT0001: tried all connections, increasing latency to 21s 00000100:00080000:2.0:1776178635.618172:0:36:0:(import.c:615:import_select_connection()) lustre-MDT0000-lwp-MDT0001: import 00000000a7c55b1f using connection 192.168.204.103@tcp/0@lo 00000100:00100000:2.0:1776178635.618178:0:36:0:(import.c:807:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5) req@000000004420855e x1862456998109184/t0(0) o38->lustre-MDT0000-lwp-MDT0001@0@lo:12/10 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' 00000100:00100000:2.0:1776178635.618196:0:3355:0:(client.c:1733:ptlrpc_send_new_req()) Sending RPC req@00000000f0f1167e pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_02:lustre-MDT0001-lwp-OST0001_UUID:3355:1862456998108928:0@lo:400:kworker/u8:1.0 00000100:00100000:2.0:1776178635.618220:0:3355:0:(events.c:361:request_in_callback()) peer: 12345-0@lo (source: 12345-0@lo) 00000100:00100000:2.0:1776178635.618235:0:3355:0:(ptlrpcd.c:415:ptlrpcd_check()) transfer 1 async RPCs [3->2] 00000100:00100000:2.0:1776178635.618238:0:3355:0:(client.c:1733:ptlrpc_send_new_req()) Sending RPC req@0000000078ae567e pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_02:lustre-MDT0001-mdtlov_UUID:3355:1862456998109056:0@lo:400:kworker/u8:1.0 00000100:00100000:2.0:1776178635.618257:0:3355:0:(events.c:361:request_in_callback()) peer: 12345-0@lo (source: 12345-0@lo) 00000100:00100000:2.0:1776178635.618351:0:20762:0:(service.c:2152:ptlrpc_server_handle_req_in()) got req x1862456998109056 00000100:00100000:2.0:1776178635.618375:0:20762:0:(nrs_fifo.c:179:nrs_fifo_req_get()) NRS start fifo request from 12345-0@lo, seq: 849 00000100:00100000:2.0:1776178635.618379:0:20762:0:(service.c:2324:ptlrpc_server_handle_request()) Handling RPC req@0000000026195e8c pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_013:lustre-MDT0001-mdtlov_UUID+5:3355:x1862456998109056:12345-0@lo:400:kworker/u8:1.0 00000100:00100000:2.0:1776178635.618412:0:20762:0:(service.c:2372:ptlrpc_server_handle_request()) Handled RPC req@0000000026195e8c pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_013:lustre-MDT0001-mdtlov_UUID+5:3355:x1862456998109056:12345-0@lo:400:kworker/u8:1.0 Request processed in 32us (154us total) trans 0 rc 0/0 00000100:00100000:2.0:1776178635.618420:0:20762:0:(nrs_fifo.c:241:nrs_fifo_req_stop()) NRS stop fifo request from 12345-0@lo, seq: 849 00000100:00100000:1.0:1776178635.618613:0:3352:0:(client.c:1733:ptlrpc_send_new_req()) Sending RPC req@00000000eb400454 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:lustre-MDT0000-lwp-OST0001_UUID:3352:1862456998108864:0@lo:38:kworker/u8:1.0 00000100:00100000:1.0:1776178635.618641:0:3352:0:(events.c:361:request_in_callback()) peer: 12345-0@lo (source: 12345-0@lo) 00000100:00100000:1.0:1776178635.618652:0:3352:0:(client.c:1733:ptlrpc_send_new_req()) Sending RPC req@00000000e2b18a6d pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:lustre-MDT0001-mdtlov_UUID:3352:1862456998108992:0@lo:38:kworker/u8:1.0 00000100:00100000:1.0:1776178635.618670:0:3352:0:(events.c:361:request_in_callback()) peer: 12345-0@lo (source: 12345-0@lo) 00000100:00100000:1.0:1776178635.618684:0:3352:0:(client.c:1733:ptlrpc_send_new_req()) Sending RPC req@000000004420855e pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:lustre-MDT0000-lwp-MDT0001_UUID:3352:1862456998109184:0@lo:38:kworker/u8:1.0 00000100:00100000:1.0:1776178635.618702:0:3352:0:(events.c:361:request_in_callback()) peer: 12345-0@lo (source: 12345-0@lo) 00000100:00100000:1.0:1776178635.618729:0:14128:0:(service.c:2152:ptlrpc_server_handle_req_in()) got req x1862456998108992 00000100:00100000:1.0:1776178635.618747:0:14128:0:(nrs_fifo.c:179:nrs_fifo_req_get()) NRS start fifo request from 12345-0@lo, seq: 792 00000100:00100000:1.0:1776178635.618750:0:14128:0:(service.c:2324:ptlrpc_server_handle_request()) Handling RPC req@000000008a15a61d pname:cluuid+ref:pid:xid:nid:opc:job mdt_out00_002:0+-99:3352:x1862456998108992:12345-0@lo:38:kworker/u8:1.0 00010000:02000000:1.0:1776178635.618758:0:14128:0:(ldlm_lib.c:1139:target_handle_connect()) lustre-MDT0000: Not available for connect from 0@lo (stopping) 00000100:00100000:1.0:1776178635.618786:0:14128:0:(service.c:2372:ptlrpc_server_handle_request()) Handled RPC req@000000008a15a61d pname:cluuid+ref:pid:xid:nid:opc:job mdt_out00_002:0+-99:3352:x1862456998108992:12345-0@lo:38:kworker/u8:1.0 Request processed in 34us (116us total) trans 0 rc -19/-19 00000100:00100000:1.0:1776178635.618791:0:14128:0:(nrs_fifo.c:241:nrs_fifo_req_stop()) NRS stop fifo request from 12345-0@lo, seq: 792 00000100:00100000:1.0:1776178635.618806:0:11281:0:(service.c:2152:ptlrpc_server_handle_req_in()) got req x1862456998108928 00000100:00100000:1.0:1776178635.618825:0:11281:0:(service.c:2152:ptlrpc_server_handle_req_in()) got req x1862456998108864 00000100:00100000:1.0:1776178635.618835:0:11281:0:(service.c:2152:ptlrpc_server_handle_req_in()) got req x1862456998109184 00000100:00100000:1.0:1776178635.618843:0:11281:0:(nrs_fifo.c:179:nrs_fifo_req_get()) NRS start fifo request from 12345-0@lo, seq: 1182 00000100:00100000:1.0:1776178635.618846:0:11281:0:(service.c:2324:ptlrpc_server_handle_request()) Handling RPC req@000000001982f085 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_004:lustre-MDT0001-lwp-OST0001_UUID+5:3355:x1862456998108928:12345-0@lo:400:kworker/u8:1.0 00000100:00100000:1.0:1776178635.618871:0:11281:0:(service.c:2372:ptlrpc_server_handle_request()) Handled RPC req@000000001982f085 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_004:lustre-MDT0001-lwp-OST0001_UUID+5:3355:x1862456998108928:12345-0@lo:400:kworker/u8:1.0 Request processed in 24us (650us total) trans 0 rc 0/0 00000100:00100000:1.0:1776178635.618876:0:11281:0:(nrs_fifo.c:241:nrs_fifo_req_stop()) NRS stop fifo request from 12345-0@lo, seq: 1182 00000100:00100000:1.0:1776178635.618886:0:11281:0:(nrs_fifo.c:179:nrs_fifo_req_get()) NRS start fifo request from 12345-0@lo, seq: 6962 00000100:00100000:1.0:1776178635.618889:0:11281:0:(service.c:2324:ptlrpc_server_handle_request()) Handling RPC req@00000000cc329471 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_004:0+-99:3352:x1862456998108864:12345-0@lo:38:kworker/u8:1.0 00010000:02000000:1.0:1776178635.618894:0:11281:0:(ldlm_lib.c:1139:target_handle_connect()) lustre-MDT0000: Not available for connect from 0@lo (stopping) 00000100:00100000:1.0:1776178635.618911:0:11281:0:(service.c:2372:ptlrpc_server_handle_request()) Handled RPC req@00000000cc329471 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_004:0+-99:3352:x1862456998108864:12345-0@lo:38:kworker/u8:1.0 Request processed in 21us (270us total) trans 0 rc -19/-19 00000100:00100000:1.0:1776178635.618915:0:11281:0:(nrs_fifo.c:241:nrs_fifo_req_stop()) NRS stop fifo request from 12345-0@lo, seq: 6962 00000100:00100000:1.0:1776178635.618922:0:11281:0:(nrs_fifo.c:179:nrs_fifo_req_get()) NRS start fifo request from 12345-0@lo, seq: 6963 00000100:00100000:1.0:1776178635.618925:0:11281:0:(service.c:2324:ptlrpc_server_handle_request()) Handling RPC req@0000000075284cd7 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_004:0+-99:3352:x1862456998109184:12345-0@lo:38:kworker/u8:1.0 00010000:02000000:1.0:1776178635.618930:0:11281:0:(ldlm_lib.c:1139:target_handle_connect()) lustre-MDT0000: Not available for connect from 0@lo (stopping) 00000100:00100000:1.0:1776178635.618943:0:11281:0:(service.c:2372:ptlrpc_server_handle_request()) Handled RPC req@0000000075284cd7 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_004:0+-99:3352:x1862456998109184:12345-0@lo:38:kworker/u8:1.0 Request processed in 18us (242us total) trans 0 rc -19/-19 00000100:00100000:1.0:1776178635.618948:0:11281:0:(nrs_fifo.c:241:nrs_fifo_req_stop()) NRS stop fifo request from 12345-0@lo, seq: 6963 00000100:00080000:1.0:1776178635.618970:0:3352:0:(import.c:86:import_set_state_nolock()) 00000000dfc8db13 lustre-MDT0000_UUID: changing import state from CONNECTING to DISCONN 00000100:00080000:1.0:1776178635.618973:0:3352:0:(import.c:1429:ptlrpc_connect_interpret()) recovery of lustre-MDT0000_UUID on 192.168.204.103@tcp failed (-19) 00000100:00100000:1.0:1776178635.618976:0:3352:0:(client.c:2215:ptlrpc_check_set()) Completed RPC req@00000000eb400454 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:lustre-MDT0000-lwp-OST0001_UUID:3352:1862456998108864:0@lo:38:kworker/u8:1.0 00000100:00080000:1.0:1776178635.618982:0:3352:0:(import.c:86:import_set_state_nolock()) 00000000ba4f1821 lustre-MDT0000_UUID: changing import state from CONNECTING to DISCONN 00000100:00080000:1.0:1776178635.618984:0:3352:0:(import.c:1429:ptlrpc_connect_interpret()) recovery of lustre-MDT0000_UUID on 192.168.204.103@tcp failed (-19) 00000100:00100000:1.0:1776178635.618987:0:3352:0:(client.c:2215:ptlrpc_check_set()) Completed RPC req@00000000e2b18a6d pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:lustre-MDT0001-mdtlov_UUID:3352:1862456998108992:0@lo:38:kworker/u8:1.0 00000100:00080000:1.0:1776178635.618993:0:3352:0:(import.c:86:import_set_state_nolock()) 00000000a7c55b1f lustre-MDT0000_UUID: changing import state from CONNECTING to DISCONN 00000100:00080000:1.0:1776178635.618995:0:3352:0:(import.c:1429:ptlrpc_connect_interpret()) recovery of lustre-MDT0000_UUID on 192.168.204.103@tcp failed (-19) 00000100:00100000:1.0:1776178635.618998:0:3352:0:(client.c:2215:ptlrpc_check_set()) Completed RPC req@000000004420855e pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:lustre-MDT0000-lwp-MDT0001_UUID:3352:1862456998109184:0@lo:38:kworker/u8:1.0 00000100:00100000:0.0:1776178635.620150:0:3354:0:(ptlrpcd.c:415:ptlrpcd_check()) transfer 1 async RPCs [0->1] 00000100:00100000:0.0:1776178635.620170:0:3354:0:(client.c:1733:ptlrpc_send_new_req()) Sending RPC req@00000000456acdaf pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_01:lustre-MDT0001-mdtlov_UUID:3354:1862456998109120:0@lo:400:kworker/u8:1.0 00000100:00100000:0.0:1776178635.620217:0:3354:0:(events.c:361:request_in_callback()) peer: 12345-0@lo (source: 12345-0@lo) 00000100:00100000:3.0:1776178635.620445:0:3355:0:(client.c:2215:ptlrpc_check_set()) Completed RPC req@0000000078ae567e pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_02:lustre-MDT0001-mdtlov_UUID:3355:1862456998109056:0@lo:400:kworker/u8:1.0 00000100:00100000:3.0:1776178635.620461:0:3355:0:(client.c:2215:ptlrpc_check_set()) Completed RPC req@00000000f0f1167e pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_02:lustre-MDT0001-lwp-OST0001_UUID:3355:1862456998108928:0@lo:400:kworker/u8:1.0 00000100:00100000:2.0:1776178635.628390:0:20759:0:(service.c:2152:ptlrpc_server_handle_req_in()) got req x1862456998109120 00000100:00100000:2.0:1776178635.628461:0:20759:0:(nrs_fifo.c:179:nrs_fifo_req_get()) NRS start fifo request from 12345-0@lo, seq: 850 00000100:00100000:2.0:1776178635.628467:0:20759:0:(service.c:2324:ptlrpc_server_handle_request()) Handling RPC req@000000002db08e42 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_010:lustre-MDT0001-mdtlov_UUID+5:3354:x1862456998109120:12345-0@lo:400:kworker/u8:1.0 00000100:00100000:2.0:1776178635.628575:0:20759:0:(service.c:2372:ptlrpc_server_handle_request()) Handled RPC req@000000002db08e42 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_010:lustre-MDT0001-mdtlov_UUID+5:3354:x1862456998109120:12345-0@lo:400:kworker/u8:1.0 Request processed in 108us (8358us total) trans 0 rc 0/0 00000100:00100000:2.0:1776178635.628583:0:20759:0:(nrs_fifo.c:241:nrs_fifo_req_stop()) NRS stop fifo request from 12345-0@lo, seq: 850 00000100:00100000:1.0:1776178635.628680:0:3354:0:(client.c:2215:ptlrpc_check_set()) Completed RPC req@00000000456acdaf pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_01:lustre-MDT0001-mdtlov_UUID:3354:1862456998109120:0@lo:400:kworker/u8:1.0 00010000:00010000:1.0:1776178636.256474:0:6245:0:(ldlm_lockd.c:2792:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:0.0:1776178637.152587:0:3353:0:(ptlrpcd.c:415:ptlrpcd_check()) transfer 1 async RPCs [1->0] 00000100:00100000:0.0:1776178637.152612:0:3353:0:(client.c:1733:ptlrpc_send_new_req()) Sending RPC req@00000000964f7a65 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_00:lustre-MDT0001-mdtlov_UUID:3353:1862456998109248:0@lo:13:osp-pre-1-1.0 00000100:00100000:0.0:1776178637.152685:0:3353:0:(events.c:361:request_in_callback()) peer: 12345-0@lo (source: 12345-0@lo) 00000100:00100000:2.0:1776178637.152798:0:9554:0:(service.c:2152:ptlrpc_server_handle_req_in()) got req x1862456998109248 00000100:00100000:2.0:1776178637.152848:0:9554:0:(nrs_fifo.c:179:nrs_fifo_req_get()) NRS start fifo request from 12345-0@lo, seq: 657 00000100:00100000:2.0:1776178637.152854:0:9554:0:(service.c:2324:ptlrpc_server_handle_request()) Handling RPC req@000000009e484c71 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_create00:lustre-MDT0001-mdtlov_UUID+5:3353:x1862456998109248:12345-0@lo:13:osp-pre-1-1.0 00002000:00000024:2.0:1776178637.152884:0:9554:0:(ofd_obd.c:740:ofd_statfs()) blocks cached 229376 granted 491746368 pending 0 free 3923689472 avail 3692167168 00000100:00100000:2.0:1776178637.152929:0:9554:0:(service.c:2372:ptlrpc_server_handle_request()) Handled RPC req@000000009e484c71 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_create00:lustre-MDT0001-mdtlov_UUID+5:3353:x1862456998109248:12345-0@lo:13:osp-pre-1-1.0 Request processed in 75us (245us total) trans 0 rc 0/0 00000100:00100000:2.0:1776178637.152935:0:9554:0:(nrs_fifo.c:241:nrs_fifo_req_stop()) NRS stop fifo request from 12345-0@lo, seq: 657 00000100:00100000:2.0:1776178637.152977:0:3353:0:(client.c:2215:ptlrpc_check_set()) Completed RPC req@00000000964f7a65 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_00:lustre-MDT0001-mdtlov_UUID:3353:1862456998109248:0@lo:13:osp-pre-1-1.0 00010000:00010000:1.0:1776178637.281470:0:9224:0:(ldlm_lockd.c:2792:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:3.0:1776178637.664577:0:3355:0:(client.c:1733:ptlrpc_send_new_req()) Sending RPC req@0000000058cd86cb pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_02:lustre-MDT0001-mdtlov_UUID:3355:1862456998109312:0@lo:13:osp-pre-0-1.0 00000100:00100000:3.0:1776178637.664646:0:3355:0:(events.c:361:request_in_callback()) peer: 12345-0@lo (source: 12345-0@lo) 00000100:00100000:1.0:1776178637.664725:0:8231:0:(service.c:2152:ptlrpc_server_handle_req_in()) got req x1862456998109312 00000100:00100000:1.0:1776178637.664779:0:8231:0:(nrs_fifo.c:179:nrs_fifo_req_get()) NRS start fifo request from 12345-0@lo, seq: 658 00000100:00100000:1.0:1776178637.664785:0:8231:0:(service.c:2324:ptlrpc_server_handle_request()) Handling RPC req@00000000f8a7b10a pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_create00:lustre-MDT0001-mdtlov_UUID+5:3355:x1862456998109312:12345-0@lo:13:osp-pre-0-1.0 00002000:00000024:1.0:1776178637.664813:0:8231:0:(ofd_obd.c:740:ofd_statfs()) blocks cached 315392 granted 484883776 pending 0 free 3923689472 avail 3692167168 00000100:00100000:1.0:1776178637.664871:0:8231:0:(service.c:2372:ptlrpc_server_handle_request()) Handled RPC req@00000000f8a7b10a pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_create00:lustre-MDT0001-mdtlov_UUID+5:3355:x1862456998109312:12345-0@lo:13:osp-pre-0-1.0 Request processed in 85us (224us total) trans 0 rc 0/0 00000100:00100000:1.0:1776178637.664879:0:8231:0:(nrs_fifo.c:241:nrs_fifo_req_stop()) NRS stop fifo request from 12345-0@lo, seq: 658 00000100:00100000:3.0:1776178637.664909:0:3355:0:(client.c:2215:ptlrpc_check_set()) Completed RPC req@0000000058cd86cb pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_02:lustre-MDT0001-mdtlov_UUID:3355:1862456998109312:0@lo:13:osp-pre-0-1.0 00010000:00010000:1.0:1776178638.304727:0:9225:0:(ldlm_lockd.c:2792:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:2.0:1776178639.141283:0:3347:0:(events.c:361:request_in_callback()) peer: 12345-192.168.204.3@tcp (source: 12345-192.168.204.3@tcp) 00000100:00100000:2.0:1776178639.141331:0:3347:0:(events.c:361:request_in_callback()) peer: 12345-192.168.204.3@tcp (source: 12345-192.168.204.3@tcp) 00000100:00100000:2.0:1776178639.141360:0:3347:0:(events.c:361:request_in_callback()) peer: 12345-192.168.204.3@tcp (source: 12345-192.168.204.3@tcp) 00000100:00100000:2.0:1776178639.141455:0:3347:0:(events.c:361:request_in_callback()) peer: 12345-192.168.204.3@tcp (source: 12345-192.168.204.3@tcp) 00000100:00100000:2.0:1776178639.141486:0:3347:0:(events.c:361:request_in_callback()) peer: 12345-192.168.204.3@tcp (source: 12345-192.168.204.3@tcp) 00000100:00100000:2.0:1776178639.141518:0:20762:0:(service.c:2152:ptlrpc_server_handle_req_in()) got req x1862456979136384 00000100:00100000:2.0:1776178639.141565:0:20762:0:(service.c:2152:ptlrpc_server_handle_req_in()) got req x1862456979136448 00000100:00100000:2.0:1776178639.141588:0:20762:0:(nrs_fifo.c:179:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.204.3@tcp, seq: 851 00000100:00100000:2.0:1776178639.141594:0:20762:0:(service.c:2324:ptlrpc_server_handle_request()) Handling RPC req@00000000ed90ff75 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_013:b50963bd-8eba-4432-9a54-8888d00e3fa1+8:2272:x1862456979136384:12345-192.168.204.3@tcp:400:kworker/u8:4.0 00000100:00100000:2.0:1776178639.141671:0:20762:0:(service.c:2372:ptlrpc_server_handle_request()) Handled RPC req@00000000ed90ff75 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_013:b50963bd-8eba-4432-9a54-8888d00e3fa1+8:2272:x1862456979136384:12345-192.168.204.3@tcp:400:kworker/u8:4.0 Request processed in 76us (217us total) trans 0 rc 0/0 00000100:00100000:2.0:1776178639.141680:0:20762:0:(nrs_fifo.c:241:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.204.3@tcp, seq: 851 00000100:00100000:2.0:1776178639.141696:0:20762:0:(nrs_fifo.c:179:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.204.3@tcp, seq: 852 00000100:00100000:2.0:1776178639.141700:0:20762:0:(service.c:2324:ptlrpc_server_handle_request()) Handling RPC req@00000000a6227918 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_013:b50963bd-8eba-4432-9a54-8888d00e3fa1+9:2273:x1862456979136448:12345-192.168.204.3@tcp:400:kworker/u8:4.0 00000100:00100000:2.0:1776178639.141720:0:20762:0:(service.c:2372:ptlrpc_server_handle_request()) Handled RPC req@00000000a6227918 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_013:b50963bd-8eba-4432-9a54-8888d00e3fa1+9:2273:x1862456979136448:12345-192.168.204.3@tcp:400:kworker/u8:4.0 Request processed in 20us (235us total) trans 0 rc 0/0 00000100:00100000:2.0:1776178639.141726:0:20762:0:(nrs_fifo.c:241:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.204.3@tcp, seq: 852 00000100:00100000:1.0:1776178639.141821:0:20140:0:(service.c:2152:ptlrpc_server_handle_req_in()) got req x1862456979136192 00000100:00100000:1.0:1776178639.141855:0:20140:0:(nrs_fifo.c:179:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.204.3@tcp, seq: 325 00000100:00100000:1.0:1776178639.141859:0:20140:0:(service.c:2324:ptlrpc_server_handle_request()) Handling RPC req@00000000c0367ec3 pname:cluuid+ref:pid:xid:nid:opc:job ll_mgs_0002:5b5c66b1-1a2f-4651-8548-0c75b9a38a94+9:2272:x1862456979136192:12345-192.168.204.3@tcp:400:kworker/u8:4.0 00000100:00100000:1.0:1776178639.141883:0:20140:0:(service.c:2372:ptlrpc_server_handle_request()) Handled RPC req@00000000c0367ec3 pname:cluuid+ref:pid:xid:nid:opc:job ll_mgs_0002:5b5c66b1-1a2f-4651-8548-0c75b9a38a94+9:2272:x1862456979136192:12345-192.168.204.3@tcp:400:kworker/u8:4.0 Request processed in 23us (608us total) trans 0 rc 0/0 00000100:00100000:1.0:1776178639.141888:0:20140:0:(nrs_fifo.c:241:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.204.3@tcp, seq: 325 00000100:00100000:0.0:1776178639.147276:0:6256:0:(service.c:2152:ptlrpc_server_handle_req_in()) got req x1862456979136256 00000100:00100000:0.0:1776178639.147337:0:6256:0:(service.c:2152:ptlrpc_server_handle_req_in()) got req x1862456979136320 00000100:00100000:0.0:1776178639.147387:0:6256:0:(nrs_fifo.c:179:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.204.3@tcp, seq: 1183 00000100:00100000:0.0:1776178639.147394:0:6256:0:(service.c:2324:ptlrpc_server_handle_request()) Handling RPC req@00000000699e66b5 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_001:b50963bd-8eba-4432-9a54-8888d00e3fa1+6:2273:x1862456979136320:12345-192.168.204.3@tcp:400:kworker/u8:4.0 00000100:00100000:0.0:1776178639.147511:0:6256:0:(service.c:2372:ptlrpc_server_handle_request()) Handled RPC req@00000000699e66b5 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_001:b50963bd-8eba-4432-9a54-8888d00e3fa1+6:2273:x1862456979136320:12345-192.168.204.3@tcp:400:kworker/u8:4.0 Request processed in 116us (6150us total) trans 0 rc 0/0 00000100:00100000:0.0:1776178639.147520:0:6256:0:(nrs_fifo.c:241:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.204.3@tcp, seq: 1183 00000100:00100000:0.0:1776178639.147552:0:6256:0:(nrs_fifo.c:179:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.204.3@tcp, seq: 6964 00000100:00100000:0.0:1776178639.147568:0:6256:0:(service.c:2324:ptlrpc_server_handle_request()) Handling RPC req@0000000021d5d4e8 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_001:0+-99:2269:x1862456979136256:12345-192.168.204.3@tcp:38:kworker/u8:4.0 00010000:02000000:0.0:1776178639.147582:0:6256:0:(ldlm_lib.c:1139:target_handle_connect()) lustre-MDT0000: Not available for connect from 192.168.204.3@tcp (stopping) 00000100:00100000:0.0:1776178639.147616:0:6256:0:(service.c:2372:ptlrpc_server_handle_request()) Handled RPC req@0000000021d5d4e8 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_001:0+-99:2269:x1862456979136256:12345-192.168.204.3@tcp:38:kworker/u8:4.0 Request processed in 59us (6285us total) trans 0 rc -19/-19 00000100:00100000:0.0:1776178639.147623:0:6256:0:(nrs_fifo.c:241:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.204.3@tcp, seq: 6964 00010000:00010000:2.0:1776178639.328318:0:6244:0:(ldlm_lockd.c:2792:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00010000:00010000:1.0:1776178640.352952:0:6245:0:(ldlm_lockd.c:2792:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00080000:0.0:1776178640.736409:0:36:0:(pinger.c:251:ptlrpc_pinger_process_import()) lustre-MDT0000-lwp-OST0000_UUID->lustre-MDT0000_UUID: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00080000:0.0:1776178640.736417:0:36:0:(recover.c:57:ptlrpc_initiate_recovery()) lustre-MDT0000_UUID: starting recovery 00000100:00080000:0.0:1776178640.736421:0:36:0:(import.c:86:import_set_state_nolock()) 0000000047601a98 lustre-MDT0000_UUID: changing import state from DISCONN to CONNECTING 00000100:00080000:0.0:1776178640.736429:0:36:0:(import.c:534:import_select_connection()) lustre-MDT0000-lwp-OST0000: connect to NID 0@lo last attempt 1342 00000100:00080000:0.0:1776178640.736432:0:36:0:(import.c:581:import_select_connection()) lustre-MDT0000-lwp-OST0000: tried all connections, increasing latency to 21s 00000100:00080000:0.0:1776178640.736438:0:36:0:(import.c:615:import_select_connection()) lustre-MDT0000-lwp-OST0000: import 0000000047601a98 using connection 192.168.204.103@tcp/0@lo 00000100:00100000:0.0:1776178640.736461:0:36:0:(import.c:807:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5) req@00000000715d380c x1862456998109440/t0(0) o38->lustre-MDT0000-lwp-OST0000@0@lo:12/10 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' 00000100:00080000:0.0:1776178640.736504:0:36:0:(pinger.c:251:ptlrpc_pinger_process_import()) lustre-MDT0000-lwp-OST0001_UUID->lustre-MDT0000_UUID: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00080000:0.0:1776178640.736507:0:36:0:(recover.c:57:ptlrpc_initiate_recovery()) lustre-MDT0000_UUID: starting recovery 00000100:00080000:0.0:1776178640.736508:0:36:0:(import.c:86:import_set_state_nolock()) 00000000dfc8db13 lustre-MDT0000_UUID: changing import state from DISCONN to CONNECTING 00000100:00080000:0.0:1776178640.736511:0:36:0:(import.c:534:import_select_connection()) lustre-MDT0000-lwp-OST0001: connect to NID 0@lo last attempt 1342 00000100:00080000:0.0:1776178640.736513:0:36:0:(import.c:581:import_select_connection()) lustre-MDT0000-lwp-OST0001: tried all connections, increasing latency to 21s 00000100:00080000:0.0:1776178640.736517:0:36:0:(import.c:615:import_select_connection()) lustre-MDT0000-lwp-OST0001: import 00000000dfc8db13 using connection 192.168.204.103@tcp/0@lo 00000100:00100000:0.0:1776178640.736525:0:36:0:(import.c:807:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5) req@000000009dd1dd0f x1862456998109568/t0(0) o38->lustre-MDT0000-lwp-OST0001@0@lo:12/10 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' 00000100:00080000:0.0:1776178640.736539:0:36:0:(pinger.c:251:ptlrpc_pinger_process_import()) lustre-MDT0001-mdtlov_UUID->lustre-MDT0000_UUID: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00080000:0.0:1776178640.736541:0:36:0:(recover.c:57:ptlrpc_initiate_recovery()) lustre-MDT0000_UUID: starting recovery 00000100:00080000:0.0:1776178640.736542:0:36:0:(import.c:86:import_set_state_nolock()) 00000000ba4f1821 lustre-MDT0000_UUID: changing import state from DISCONN to CONNECTING 00000100:00080000:0.0:1776178640.736545:0:36:0:(import.c:534:import_select_connection()) lustre-MDT0000-osp-MDT0001: connect to NID 0@lo last attempt 1342 00000100:00080000:0.0:1776178640.736547:0:36:0:(import.c:581:import_select_connection()) lustre-MDT0000-osp-MDT0001: tried all connections, increasing latency to 21s 00000100:00080000:0.0:1776178640.736549:0:36:0:(import.c:615:import_select_connection()) lustre-MDT0000-osp-MDT0001: import 00000000ba4f1821 using connection 192.168.204.103@tcp/0@lo 00000100:00100000:0.0:1776178640.736560:0:36:0:(import.c:807:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5) req@0000000070829000 x1862456998109696/t0(0) o38->lustre-MDT0000-osp-MDT0001@0@lo:24/4 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' 00000100:00080000:0.0:1776178640.736578:0:36:0:(pinger.c:251:ptlrpc_pinger_process_import()) lustre-MDT0000-lwp-MDT0001_UUID->lustre-MDT0000_UUID: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00080000:0.0:1776178640.736581:0:36:0:(recover.c:57:ptlrpc_initiate_recovery()) lustre-MDT0000_UUID: starting recovery 00000100:00080000:0.0:1776178640.736582:0:36:0:(import.c:86:import_set_state_nolock()) 00000000a7c55b1f lustre-MDT0000_UUID: changing import state from DISCONN to CONNECTING 00000100:00080000:0.0:1776178640.736584:0:36:0:(import.c:534:import_select_connection()) lustre-MDT0000-lwp-MDT0001: connect to NID 0@lo last attempt 1342 00000100:00080000:0.0:1776178640.736586:0:36:0:(import.c:581:import_select_connection()) lustre-MDT0000-lwp-MDT0001: tried all connections, increasing latency to 21s 00000100:00080000:0.0:1776178640.736589:0:36:0:(import.c:615:import_select_connection()) lustre-MDT0000-lwp-MDT0001: import 00000000a7c55b1f using connection 192.168.204.103@tcp/0@lo 00000100:00100000:0.0:1776178640.736596:0:36:0:(import.c:807:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5) req@0000000098682d9e x1862456998109888/t0(0) o38->lustre-MDT0000-lwp-MDT0001@0@lo:12/10 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' 00000100:00100000:1.0:1776178640.736837:0:3352:0:(client.c:1733:ptlrpc_send_new_req()) Sending RPC req@00000000715d380c pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:lustre-MDT0000-lwp-OST0000_UUID:3352:1862456998109440:0@lo:38:kworker/u8:1.0 00000100:00100000:1.0:1776178640.736903:0:3352:0:(events.c:361:request_in_callback()) peer: 12345-0@lo (source: 12345-0@lo) 00000100:00100000:1.0:1776178640.736938:0:3352:0:(client.c:1733:ptlrpc_send_new_req()) Sending RPC req@000000009dd1dd0f pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:lustre-MDT0000-lwp-OST0001_UUID:3352:1862456998109568:0@lo:38:kworker/u8:1.0 00000100:00100000:1.0:1776178640.736960:0:3352:0:(events.c:361:request_in_callback()) peer: 12345-0@lo (source: 12345-0@lo) 00000100:00100000:1.0:1776178640.736974:0:3352:0:(client.c:1733:ptlrpc_send_new_req()) Sending RPC req@0000000070829000 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:lustre-MDT0001-mdtlov_UUID:3352:1862456998109696:0@lo:38:kworker/u8:1.0 00000100:00100000:1.0:1776178640.736998:0:3352:0:(events.c:361:request_in_callback()) peer: 12345-0@lo (source: 12345-0@lo) 00000100:00100000:1.0:1776178640.737015:0:3352:0:(client.c:1733:ptlrpc_send_new_req()) Sending RPC req@0000000098682d9e pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:lustre-MDT0000-lwp-MDT0001_UUID:3352:1862456998109888:0@lo:38:kworker/u8:1.0 00000100:00100000:1.0:1776178640.737036:0:3352:0:(events.c:361:request_in_callback()) peer: 12345-0@lo (source: 12345-0@lo) 00000100:00100000:1.0:1776178640.737093:0:3354:0:(client.c:1733:ptlrpc_send_new_req()) Sending RPC req@0000000082764007 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_01:lustre-MDT0001-lwp-OST0001_UUID:3354:1862456998109632:0@lo:400:kworker/u8:1.0 00000100:00100000:1.0:1776178640.737220:0:3354:0:(events.c:361:request_in_callback()) peer: 12345-0@lo (source: 12345-0@lo) 00000100:00100000:1.0:1776178640.737242:0:3354:0:(ptlrpcd.c:415:ptlrpcd_check()) transfer 1 async RPCs [0->1] 00000100:00100000:1.0:1776178640.737248:0:3354:0:(client.c:1733:ptlrpc_send_new_req()) Sending RPC req@0000000013ad9ac7 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_01:lustre-MDT0001-lwp-OST0000_UUID:3354:1862456998109504:0@lo:400:kworker/u8:1.0 00000100:00100000:1.0:1776178640.737270:0:3354:0:(events.c:361:request_in_callback()) peer: 12345-0@lo (source: 12345-0@lo) 00000100:00100000:1.0:1776178640.737304:0:15213:0:(service.c:2152:ptlrpc_server_handle_req_in()) got req x1862456998109696 00000100:00100000:1.0:1776178640.737409:0:15213:0:(nrs_fifo.c:179:nrs_fifo_req_get()) NRS start fifo request from 12345-0@lo, seq: 793 00000100:00100000:1.0:1776178640.737417:0:15213:0:(service.c:2324:ptlrpc_server_handle_request()) Handling RPC req@00000000589b0247 pname:cluuid+ref:pid:xid:nid:opc:job mdt_out00_003:0+-99:3352:x1862456998109696:12345-0@lo:38:kworker/u8:1.0 00010000:02000000:1.0:1776178640.737431:0:15213:0:(ldlm_lib.c:1139:target_handle_connect()) lustre-MDT0000: Not available for connect from 0@lo (stopping) 00000100:00100000:1.0:1776178640.737555:0:15213:0:(service.c:2372:ptlrpc_server_handle_request()) Handled RPC req@00000000589b0247 pname:cluuid+ref:pid:xid:nid:opc:job mdt_out00_003:0+-99:3352:x1862456998109696:12345-0@lo:38:kworker/u8:1.0 Request processed in 137us (555us total) trans 0 rc -19/-19 00000100:00100000:1.0:1776178640.737563:0:15213:0:(nrs_fifo.c:241:nrs_fifo_req_stop()) NRS stop fifo request from 12345-0@lo, seq: 793 00000100:00100000:1.0:1776178640.737588:0:8250:0:(service.c:2152:ptlrpc_server_handle_req_in()) got req x1862456998109440 00000100:00100000:1.0:1776178640.737602:0:8250:0:(service.c:2152:ptlrpc_server_handle_req_in()) got req x1862456998109568 00000100:00100000:1.0:1776178640.737609:0:8250:0:(service.c:2152:ptlrpc_server_handle_req_in()) got req x1862456998109888 00000100:00100000:1.0:1776178640.737616:0:8250:0:(service.c:2152:ptlrpc_server_handle_req_in()) got req x1862456998109632 00000100:00100000:1.0:1776178640.737633:0:8250:0:(service.c:2152:ptlrpc_server_handle_req_in()) got req x1862456998109504 00000100:00100000:1.0:1776178640.737646:0:8250:0:(nrs_fifo.c:179:nrs_fifo_req_get()) NRS start fifo request from 12345-0@lo, seq: 1184 00000100:00100000:1.0:1776178640.737649:0:8250:0:(service.c:2324:ptlrpc_server_handle_request()) Handling RPC req@0000000027e376cf pname:cluuid+ref:pid:xid:nid:opc:job mdt00_003:lustre-MDT0001-lwp-OST0001_UUID+5:3354:x1862456998109632:12345-0@lo:400:kworker/u8:1.0 00000100:00100000:1.0:1776178640.737691:0:8250:0:(service.c:2372:ptlrpc_server_handle_request()) Handled RPC req@0000000027e376cf pname:cluuid+ref:pid:xid:nid:opc:job mdt00_003:lustre-MDT0001-lwp-OST0001_UUID+5:3354:x1862456998109632:12345-0@lo:400:kworker/u8:1.0 Request processed in 41us (472us total) trans 0 rc 0/0 00000100:00100000:1.0:1776178640.737702:0:8250:0:(nrs_fifo.c:241:nrs_fifo_req_stop()) NRS stop fifo request from 12345-0@lo, seq: 1184 00000100:00100000:1.0:1776178640.737715:0:8250:0:(nrs_fifo.c:179:nrs_fifo_req_get()) NRS start fifo request from 12345-0@lo, seq: 1185 00000100:00100000:1.0:1776178640.737718:0:8250:0:(service.c:2324:ptlrpc_server_handle_request()) Handling RPC req@00000000efb13deb pname:cluuid+ref:pid:xid:nid:opc:job mdt00_003:lustre-MDT0001-lwp-OST0000_UUID+5:3354:x1862456998109504:12345-0@lo:400:kworker/u8:1.0 00000100:00100000:1.0:1776178640.737743:0:8250:0:(service.c:2372:ptlrpc_server_handle_request()) Handled RPC req@00000000efb13deb pname:cluuid+ref:pid:xid:nid:opc:job mdt00_003:lustre-MDT0001-lwp-OST0000_UUID+5:3354:x1862456998109504:12345-0@lo:400:kworker/u8:1.0 Request processed in 24us (473us total) trans 0 rc 0/0 00000100:00100000:1.0:1776178640.737748:0:8250:0:(nrs_fifo.c:241:nrs_fifo_req_stop()) NRS stop fifo request from 12345-0@lo, seq: 1185 00000100:00100000:1.0:1776178640.737759:0:8250:0:(nrs_fifo.c:179:nrs_fifo_req_get()) NRS start fifo request from 12345-0@lo, seq: 6965 00000100:00100000:1.0:1776178640.737762:0:8250:0:(service.c:2324:ptlrpc_server_handle_request()) Handling RPC req@0000000051ff1425 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_003:0+-99:3352:x1862456998109440:12345-0@lo:38:kworker/u8:1.0 00010000:02000000:1.0:1776178640.737768:0:8250:0:(ldlm_lib.c:1139:target_handle_connect()) lustre-MDT0000: Not available for connect from 0@lo (stopping) 00000100:00100000:1.0:1776178640.737789:0:8250:0:(service.c:2372:ptlrpc_server_handle_request()) Handled RPC req@0000000051ff1425 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_003:0+-99:3352:x1862456998109440:12345-0@lo:38:kworker/u8:1.0 Request processed in 25us (888us total) trans 0 rc -19/-19 00000100:00100000:1.0:1776178640.737794:0:8250:0:(nrs_fifo.c:241:nrs_fifo_req_stop()) NRS stop fifo request from 12345-0@lo, seq: 6965 00000100:00100000:1.0:1776178640.737805:0:8250:0:(nrs_fifo.c:179:nrs_fifo_req_get()) NRS start fifo request from 12345-0@lo, seq: 6966 00000100:00100000:1.0:1776178640.737808:0:8250:0:(service.c:2324:ptlrpc_server_handle_request()) Handling RPC req@000000008c0fa959 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_003:0+-99:3352:x1862456998109568:12345-0@lo:38:kworker/u8:1.0 00010000:02000000:1.0:1776178640.737813:0:8250:0:(ldlm_lib.c:1139:target_handle_connect()) lustre-MDT0000: Not available for connect from 0@lo (stopping) 00000100:00100000:1.0:1776178640.737835:0:8250:0:(service.c:2372:ptlrpc_server_handle_request()) Handled RPC req@000000008c0fa959 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_003:0+-99:3352:x1862456998109568:12345-0@lo:38:kworker/u8:1.0 Request processed in 26us (874us total) trans 0 rc -19/-19 00000100:00100000:1.0:1776178640.737839:0:8250:0:(nrs_fifo.c:241:nrs_fifo_req_stop()) NRS stop fifo request from 12345-0@lo, seq: 6966 00000100:00100000:1.0:1776178640.737848:0:8250:0:(nrs_fifo.c:179:nrs_fifo_req_get()) NRS start fifo request from 12345-0@lo, seq: 6967 00000100:00100000:1.0:1776178640.737852:0:8250:0:(service.c:2324:ptlrpc_server_handle_request()) Handling RPC req@00000000c598e4ab pname:cluuid+ref:pid:xid:nid:opc:job mdt00_003:0+-99:3352:x1862456998109888:12345-0@lo:38:kworker/u8:1.0 00010000:02000000:1.0:1776178640.737858:0:8250:0:(ldlm_lib.c:1139:target_handle_connect()) lustre-MDT0000: Not available for connect from 0@lo (stopping) 00000100:00100000:1.0:1776178640.737877:0:8250:0:(service.c:2372:ptlrpc_server_handle_request()) Handled RPC req@00000000c598e4ab pname:cluuid+ref:pid:xid:nid:opc:job mdt00_003:0+-99:3352:x1862456998109888:12345-0@lo:38:kworker/u8:1.0 Request processed in 24us (840us total) trans 0 rc -19/-19 00000100:00100000:1.0:1776178640.737882:0:8250:0:(nrs_fifo.c:241:nrs_fifo_req_stop()) NRS stop fifo request from 12345-0@lo, seq: 6967 00000100:00100000:1.0:1776178640.737926:0:3354:0:(client.c:2215:ptlrpc_check_set()) Completed RPC req@0000000013ad9ac7 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_01:lustre-MDT0001-lwp-OST0000_UUID:3354:1862456998109504:0@lo:400:kworker/u8:1.0 00000100:00100000:1.0:1776178640.737937:0:3354:0:(client.c:2215:ptlrpc_check_set()) Completed RPC req@0000000082764007 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_01:lustre-MDT0001-lwp-OST0001_UUID:3354:1862456998109632:0@lo:400:kworker/u8:1.0 00000100:00080000:1.0:1776178640.737967:0:3352:0:(import.c:86:import_set_state_nolock()) 0000000047601a98 lustre-MDT0000_UUID: changing import state from CONNECTING to DISCONN 00000100:00080000:1.0:1776178640.737970:0:3352:0:(import.c:1429:ptlrpc_connect_interpret()) recovery of lustre-MDT0000_UUID on 192.168.204.103@tcp failed (-19) 00000100:00100000:1.0:1776178640.737974:0:3352:0:(client.c:2215:ptlrpc_check_set()) Completed RPC req@00000000715d380c pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:lustre-MDT0000-lwp-OST0000_UUID:3352:1862456998109440:0@lo:38:kworker/u8:1.0 00000100:00080000:1.0:1776178640.737981:0:3352:0:(import.c:86:import_set_state_nolock()) 00000000dfc8db13 lustre-MDT0000_UUID: changing import state from CONNECTING to DISCONN 00000100:00080000:1.0:1776178640.737984:0:3352:0:(import.c:1429:ptlrpc_connect_interpret()) recovery of lustre-MDT0000_UUID on 192.168.204.103@tcp failed (-19) 00000100:00100000:1.0:1776178640.737986:0:3352:0:(client.c:2215:ptlrpc_check_set()) Completed RPC req@000000009dd1dd0f pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:lustre-MDT0000-lwp-OST0001_UUID:3352:1862456998109568:0@lo:38:kworker/u8:1.0 00000100:00080000:1.0:1776178640.737993:0:3352:0:(import.c:86:import_set_state_nolock()) 00000000ba4f1821 lustre-MDT0000_UUID: changing import state from CONNECTING to DISCONN 00000100:00080000:1.0:1776178640.737995:0:3352:0:(import.c:1429:ptlrpc_connect_interpret()) recovery of lustre-MDT0000_UUID on 192.168.204.103@tcp failed (-19) 00000100:00100000:1.0:1776178640.737997:0:3352:0:(client.c:2215:ptlrpc_check_set()) Completed RPC req@0000000070829000 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:lustre-MDT0001-mdtlov_UUID:3352:1862456998109696:0@lo:38:kworker/u8:1.0 00000100:00080000:1.0:1776178640.738003:0:3352:0:(import.c:86:import_set_state_nolock()) 00000000a7c55b1f lustre-MDT0000_UUID: changing import state from CONNECTING to DISCONN 00000100:00080000:1.0:1776178640.738006:0:3352:0:(import.c:1429:ptlrpc_connect_interpret()) recovery of lustre-MDT0000_UUID on 192.168.204.103@tcp failed (-19) 00000100:00100000:1.0:1776178640.738008:0:3352:0:(client.c:2215:ptlrpc_check_set()) Completed RPC req@0000000098682d9e pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:lustre-MDT0000-lwp-MDT0001_UUID:3352:1862456998109888:0@lo:38:kworker/u8:1.0 00000100:00100000:2.0:1776178640.738136:0:3356:0:(client.c:1733:ptlrpc_send_new_req()) Sending RPC req@00000000abb6cbab pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_03:ac61f839-0e8e-4cac-9c59-08bcfbbad330:3356:1862456998109376:0@lo:400:kworker/u8:1.0 00000100:00100000:2.0:1776178640.738171:0:3356:0:(events.c:361:request_in_callback()) peer: 12345-0@lo (source: 12345-0@lo) 00000100:00100000:2.0:1776178640.738196:0:3356:0:(client.c:1733:ptlrpc_send_new_req()) Sending RPC req@00000000598050cc pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_03:lustre-MDT0001-mdtlov_UUID:3356:1862456998109824:0@lo:400:kworker/u8:1.0 00000100:00100000:2.0:1776178640.738220:0:3356:0:(events.c:361:request_in_callback()) peer: 12345-0@lo (source: 12345-0@lo) 00000100:00100000:2.0:1776178640.738236:0:3356:0:(ptlrpcd.c:415:ptlrpcd_check()) transfer 1 async RPCs [2->3] 00000100:00100000:2.0:1776178640.738241:0:3356:0:(client.c:1733:ptlrpc_send_new_req()) Sending RPC req@00000000ef4da5b8 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_03:lustre-MDT0001-mdtlov_UUID:3356:1862456998109760:0@lo:400:kworker/u8:1.0 00000100:00100000:2.0:1776178640.738261:0:3356:0:(events.c:361:request_in_callback()) peer: 12345-0@lo (source: 12345-0@lo) 00000100:00100000:2.0:1776178640.738282:0:14440:0:(service.c:2152:ptlrpc_server_handle_req_in()) got req x1862456998109824 00000100:00100000:2.0:1776178640.738308:0:14440:0:(service.c:2152:ptlrpc_server_handle_req_in()) got req x1862456998109760 00000100:00100000:2.0:1776178640.738323:0:14440:0:(nrs_fifo.c:179:nrs_fifo_req_get()) NRS start fifo request from 12345-0@lo, seq: 853 00000100:00100000:2.0:1776178640.738326:0:14440:0:(service.c:2324:ptlrpc_server_handle_request()) Handling RPC req@0000000056ddcb09 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_004:lustre-MDT0001-mdtlov_UUID+5:3356:x1862456998109824:12345-0@lo:400:kworker/u8:1.0 00000100:00100000:2.0:1776178640.738371:0:14440:0:(service.c:2372:ptlrpc_server_handle_request()) Handled RPC req@0000000056ddcb09 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_004:lustre-MDT0001-mdtlov_UUID+5:3356:x1862456998109824:12345-0@lo:400:kworker/u8:1.0 Request processed in 43us (150us total) trans 0 rc 0/0 00000100:00100000:2.0:1776178640.738388:0:14440:0:(nrs_fifo.c:241:nrs_fifo_req_stop()) NRS stop fifo request from 12345-0@lo, seq: 853 00000100:00100000:2.0:1776178640.738401:0:14440:0:(nrs_fifo.c:179:nrs_fifo_req_get()) NRS start fifo request from 12345-0@lo, seq: 854 00000100:00100000:2.0:1776178640.738404:0:14440:0:(service.c:2324:ptlrpc_server_handle_request()) Handling RPC req@000000007d60c0e9 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_004:lustre-MDT0001-mdtlov_UUID+5:3356:x1862456998109760:12345-0@lo:400:kworker/u8:1.0 00000100:00100000:2.0:1776178640.738429:0:14440:0:(service.c:2372:ptlrpc_server_handle_request()) Handled RPC req@000000007d60c0e9 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_004:lustre-MDT0001-mdtlov_UUID+5:3356:x1862456998109760:12345-0@lo:400:kworker/u8:1.0 Request processed in 24us (167us total) trans 0 rc 0/0 00000100:00100000:2.0:1776178640.738435:0:14440:0:(nrs_fifo.c:241:nrs_fifo_req_stop()) NRS stop fifo request from 12345-0@lo, seq: 854 00000100:00100000:2.0:1776178640.738467:0:3356:0:(client.c:2215:ptlrpc_check_set()) Completed RPC req@00000000ef4da5b8 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_03:lustre-MDT0001-mdtlov_UUID:3356:1862456998109760:0@lo:400:kworker/u8:1.0 00000100:00100000:2.0:1776178640.738486:0:3356:0:(client.c:2215:ptlrpc_check_set()) Completed RPC req@00000000598050cc pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_03:lustre-MDT0001-mdtlov_UUID:3356:1862456998109824:0@lo:400:kworker/u8:1.0 00000100:00100000:1.0:1776178640.738909:0:20137:0:(service.c:2152:ptlrpc_server_handle_req_in()) got req x1862456998109376 00000100:00100000:1.0:1776178640.738949:0:20137:0:(nrs_fifo.c:179:nrs_fifo_req_get()) NRS start fifo request from 12345-0@lo, seq: 326 00000100:00100000:1.0:1776178640.738953:0:20137:0:(service.c:2324:ptlrpc_server_handle_request()) Handling RPC req@00000000ecf3e0a2 pname:cluuid+ref:pid:xid:nid:opc:job ll_mgs_0000:ac61f839-0e8e-4cac-9c59-08bcfbbad330+18:3356:x1862456998109376:12345-0@lo:400:kworker/u8:1.0 00000100:00100000:1.0:1776178640.739177:0:20137:0:(service.c:2372:ptlrpc_server_handle_request()) Handled RPC req@00000000ecf3e0a2 pname:cluuid+ref:pid:xid:nid:opc:job ll_mgs_0000:ac61f839-0e8e-4cac-9c59-08bcfbbad330+18:3356:x1862456998109376:12345-0@lo:400:kworker/u8:1.0 Request processed in 222us (1004us total) trans 0 rc 0/0 00000100:00100000:1.0:1776178640.739185:0:20137:0:(nrs_fifo.c:241:nrs_fifo_req_stop()) NRS stop fifo request from 12345-0@lo, seq: 326 00000100:00100000:3.0:1776178640.740036:0:3356:0:(client.c:2215:ptlrpc_check_set()) Completed RPC req@00000000abb6cbab pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_03:ac61f839-0e8e-4cac-9c59-08bcfbbad330:3356:1862456998109376:0@lo:400:kworker/u8:1.0 00010000:00010000:1.0:1776178641.380598:0:9224:0:(ldlm_lockd.c:2792:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:2.0:1776178642.273089:0:3353:0:(client.c:1733:ptlrpc_send_new_req()) Sending RPC req@0000000037d88937 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_00:lustre-MDT0001-mdtlov_UUID:3353:1862456998109952:0@lo:13:osp-pre-1-1.0 00000100:00100000:2.0:1776178642.273234:0:3353:0:(events.c:361:request_in_callback()) peer: 12345-0@lo (source: 12345-0@lo) 00000100:00100000:0.0:1776178642.273380:0:9554:0:(service.c:2152:ptlrpc_server_handle_req_in()) got req x1862456998109952 00000100:00100000:0.0:1776178642.273433:0:9554:0:(nrs_fifo.c:179:nrs_fifo_req_get()) NRS start fifo request from 12345-0@lo, seq: 659 00000100:00100000:0.0:1776178642.273440:0:9554:0:(service.c:2324:ptlrpc_server_handle_request()) Handling RPC req@0000000049b1b339 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_create00:lustre-MDT0001-mdtlov_UUID+5:3353:x1862456998109952:12345-0@lo:13:osp-pre-1-1.0 00002000:00000024:0.0:1776178642.273480:0:9554:0:(ofd_obd.c:740:ofd_statfs()) blocks cached 229376 granted 491746368 pending 0 free 3923689472 avail 3692167168 00000100:00100000:0.0:1776178642.273533:0:9554:0:(service.c:2372:ptlrpc_server_handle_request()) Handled RPC req@0000000049b1b339 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_create00:lustre-MDT0001-mdtlov_UUID+5:3353:x1862456998109952:12345-0@lo:13:osp-pre-1-1.0 Request processed in 92us (299us total) trans 0 rc 0/0 00000100:00100000:0.0:1776178642.273540:0:9554:0:(nrs_fifo.c:241:nrs_fifo_req_stop()) NRS stop fifo request from 12345-0@lo, seq: 659 00000100:00100000:3.0:1776178642.273697:0:3353:0:(client.c:2215:ptlrpc_check_set()) Completed RPC req@0000000037d88937 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_00:lustre-MDT0001-mdtlov_UUID:3353:1862456998109952:0@lo:13:osp-pre-1-1.0 00010000:00010000:1.0:1776178642.403024:0:9225:0:(ldlm_lockd.c:2792:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:1.0:1776178642.784385:0:3354:0:(client.c:1733:ptlrpc_send_new_req()) Sending RPC req@0000000039ff8335 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_01:lustre-MDT0001-mdtlov_UUID:3354:1862456998110016:0@lo:13:osp-pre-0-1.0 00000100:00100000:1.0:1776178642.784462:0:3354:0:(events.c:361:request_in_callback()) peer: 12345-0@lo (source: 12345-0@lo) 00000100:00100000:2.0:1776178642.785195:0:8231:0:(service.c:2152:ptlrpc_server_handle_req_in()) got req x1862456998110016 00000100:00100000:2.0:1776178642.785250:0:8231:0:(nrs_fifo.c:179:nrs_fifo_req_get()) NRS start fifo request from 12345-0@lo, seq: 660 00000100:00100000:2.0:1776178642.785257:0:8231:0:(service.c:2324:ptlrpc_server_handle_request()) Handling RPC req@00000000d1597b46 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_create00:lustre-MDT0001-mdtlov_UUID+5:3354:x1862456998110016:12345-0@lo:13:osp-pre-0-1.0 00002000:00000024:2.0:1776178642.785294:0:8231:0:(ofd_obd.c:740:ofd_statfs()) blocks cached 315392 granted 484883776 pending 0 free 3923689472 avail 3692167168 00000100:00100000:2.0:1776178642.785359:0:8231:0:(service.c:2372:ptlrpc_server_handle_request()) Handled RPC req@00000000d1597b46 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_create00:lustre-MDT0001-mdtlov_UUID+5:3354:x1862456998110016:12345-0@lo:13:osp-pre-0-1.0 Request processed in 102us (898us total) trans 0 rc 0/0 00000100:00100000:2.0:1776178642.785367:0:8231:0:(nrs_fifo.c:241:nrs_fifo_req_stop()) NRS stop fifo request from 12345-0@lo, seq: 660 00000100:00100000:0.0:1776178642.785678:0:3354:0:(client.c:2215:ptlrpc_check_set()) Completed RPC req@0000000039ff8335 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_01:lustre-MDT0001-mdtlov_UUID:3354:1862456998110016:0@lo:13:osp-pre-0-1.0 00010000:00010000:2.0:1776178643.424985:0:6244:0:(ldlm_lockd.c:2792:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:2.0:1776178644.261299:0:3347:0:(events.c:361:request_in_callback()) peer: 12345-192.168.204.3@tcp (source: 12345-192.168.204.3@tcp) 00000100:00100000:2.0:1776178644.261353:0:3347:0:(events.c:361:request_in_callback()) peer: 12345-192.168.204.3@tcp (source: 12345-192.168.204.3@tcp) 00000100:00100000:2.0:1776178644.261383:0:3347:0:(events.c:361:request_in_callback()) peer: 12345-192.168.204.3@tcp (source: 12345-192.168.204.3@tcp) 00000100:00100000:2.0:1776178644.261414:0:3347:0:(events.c:361:request_in_callback()) peer: 12345-192.168.204.3@tcp (source: 12345-192.168.204.3@tcp) 00000100:00100000:2.0:1776178644.261477:0:3347:0:(events.c:361:request_in_callback()) peer: 12345-192.168.204.3@tcp (source: 12345-192.168.204.3@tcp) 00000100:00100000:2.0:1776178644.261513:0:6257:0:(service.c:2152:ptlrpc_server_handle_req_in()) got req x1862456979136640 00000100:00100000:0.0:1776178644.261563:0:6255:0:(service.c:2152:ptlrpc_server_handle_req_in()) got req x1862456979136576 00000100:00100000:2.0:1776178644.261564:0:6257:0:(nrs_fifo.c:179:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.204.3@tcp, seq: 1186 00000100:00100000:2.0:1776178644.261575:0:6257:0:(service.c:2324:ptlrpc_server_handle_request()) Handling RPC req@0000000039ff8335 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_002:b50963bd-8eba-4432-9a54-8888d00e3fa1+6:2272:x1862456979136640:12345-192.168.204.3@tcp:400:kworker/u8:4.0 00000100:00100000:0.0:1776178644.261624:0:6255:0:(nrs_fifo.c:179:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.204.3@tcp, seq: 6968 00000100:00100000:0.0:1776178644.261632:0:6255:0:(service.c:2324:ptlrpc_server_handle_request()) Handling RPC req@00000000456acdaf pname:cluuid+ref:pid:xid:nid:opc:job mdt00_000:0+-99:2269:x1862456979136576:12345-192.168.204.3@tcp:38:kworker/u8:4.0 00010000:02000000:0.0:1776178644.261655:0:6255:0:(ldlm_lib.c:1139:target_handle_connect()) lustre-MDT0000: Not available for connect from 192.168.204.3@tcp (stopping) 00000100:00100000:2.0:1776178644.261661:0:6257:0:(service.c:2372:ptlrpc_server_handle_request()) Handled RPC req@0000000039ff8335 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_002:b50963bd-8eba-4432-9a54-8888d00e3fa1+6:2272:x1862456979136640:12345-192.168.204.3@tcp:400:kworker/u8:4.0 Request processed in 85us (276us total) trans 0 rc 0/0 00000100:00100000:0.0:1776178644.261728:0:6255:0:(service.c:2372:ptlrpc_server_handle_request()) Handled RPC req@00000000456acdaf pname:cluuid+ref:pid:xid:nid:opc:job mdt00_000:0+-99:2269:x1862456979136576:12345-192.168.204.3@tcp:38:kworker/u8:4.0 Request processed in 89us (244us total) trans 0 rc -19/-19 00000100:00100000:1.0:1776178644.261851:0:20140:0:(service.c:2152:ptlrpc_server_handle_req_in()) got req x1862456979136512 00000100:00100000:1.0:1776178644.261879:0:20140:0:(nrs_fifo.c:179:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.204.3@tcp, seq: 327 00000100:00100000:1.0:1776178644.261884:0:20140:0:(service.c:2324:ptlrpc_server_handle_request()) Handling RPC req@000000003960fbcb pname:cluuid+ref:pid:xid:nid:opc:job ll_mgs_0002:5b5c66b1-1a2f-4651-8548-0c75b9a38a94+9:2272:x1862456979136512:12345-192.168.204.3@tcp:400:kworker/u8:4.0 00000100:00100000:1.0:1776178644.261916:0:20140:0:(service.c:2372:ptlrpc_server_handle_request()) Handled RPC req@000000003960fbcb pname:cluuid+ref:pid:xid:nid:opc:job ll_mgs_0002:5b5c66b1-1a2f-4651-8548-0c75b9a38a94+9:2272:x1862456979136512:12345-192.168.204.3@tcp:400:kworker/u8:4.0 Request processed in 32us (624us total) trans 0 rc 0/0 00000100:00100000:1.0:1776178644.261923:0:20140:0:(nrs_fifo.c:241:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.204.3@tcp, seq: 327 00000100:00100000:3.0:1776178644.262119:0:20759:0:(service.c:2152:ptlrpc_server_handle_req_in()) got req x1862456979136704 00000100:00100000:3.0:1776178644.262151:0:20759:0:(service.c:2152:ptlrpc_server_handle_req_in()) got req x1862456979136768 00000100:00100000:3.0:1776178644.262168:0:20759:0:(nrs_fifo.c:179:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.204.3@tcp, seq: 855 00000100:00100000:3.0:1776178644.262172:0:20759:0:(service.c:2324:ptlrpc_server_handle_request()) Handling RPC req@0000000022d4dc29 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_010:b50963bd-8eba-4432-9a54-8888d00e3fa1+8:2272:x1862456979136704:12345-192.168.204.3@tcp:400:kworker/u8:4.0 00000100:00100000:3.0:1776178644.262202:0:20759:0:(service.c:2372:ptlrpc_server_handle_request()) Handled RPC req@0000000022d4dc29 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_010:b50963bd-8eba-4432-9a54-8888d00e3fa1+8:2272:x1862456979136704:12345-192.168.204.3@tcp:400:kworker/u8:4.0 Request processed in 28us (849us total) trans 0 rc 0/0 00000100:00100000:3.0:1776178644.262208:0:20759:0:(nrs_fifo.c:241:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.204.3@tcp, seq: 855 00000100:00100000:3.0:1776178644.262218:0:20759:0:(nrs_fifo.c:179:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.204.3@tcp, seq: 856 00000100:00100000:3.0:1776178644.262222:0:20759:0:(service.c:2324:ptlrpc_server_handle_request()) Handling RPC req@0000000058cd86cb pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_010:b50963bd-8eba-4432-9a54-8888d00e3fa1+9:2272:x1862456979136768:12345-192.168.204.3@tcp:400:kworker/u8:4.0 00000100:00100000:3.0:1776178644.262239:0:20759:0:(service.c:2372:ptlrpc_server_handle_request()) Handled RPC req@0000000058cd86cb pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_010:b50963bd-8eba-4432-9a54-8888d00e3fa1+9:2272:x1862456979136768:12345-192.168.204.3@tcp:400:kworker/u8:4.0 Request processed in 17us (826us total) trans 0 rc 0/0 00000100:00100000:3.0:1776178644.262244:0:20759:0:(nrs_fifo.c:241:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.204.3@tcp, seq: 856 00000100:00100000:2.0:1776178644.264614:0:6257:0:(nrs_fifo.c:241:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.204.3@tcp, seq: 1186 00000100:00100000:0.0:1776178644.265129:0:6255:0:(nrs_fifo.c:241:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.204.3@tcp, seq: 6968 00010000:00010000:1.0:1776178644.449712:0:6245:0:(ldlm_lockd.c:2792:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00010000:00010000:2.0:1776178645.472659:0:9224:0:(ldlm_lockd.c:2792:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:2.0:1776178645.856293:0:3356:0:(ptlrpcd.c:415:ptlrpcd_check()) transfer 1 async RPCs [2->3] 00000100:00080000:0.0:1776178645.859367:0:36:0:(pinger.c:251:ptlrpc_pinger_process_import()) lustre-MDT0000-lwp-OST0000_UUID->lustre-MDT0000_UUID: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00080000:0.0:1776178645.859378:0:36:0:(recover.c:57:ptlrpc_initiate_recovery()) lustre-MDT0000_UUID: starting recovery 00000100:00080000:0.0:1776178645.859382:0:36:0:(import.c:86:import_set_state_nolock()) 0000000047601a98 lustre-MDT0000_UUID: changing import state from DISCONN to CONNECTING 00000100:00080000:0.0:1776178645.859389:0:36:0:(import.c:534:import_select_connection()) lustre-MDT0000-lwp-OST0000: connect to NID 0@lo last attempt 1348 00000100:00080000:0.0:1776178645.859393:0:36:0:(import.c:581:import_select_connection()) lustre-MDT0000-lwp-OST0000: tried all connections, increasing latency to 21s 00000100:00080000:0.0:1776178645.859399:0:36:0:(import.c:615:import_select_connection()) lustre-MDT0000-lwp-OST0000: import 0000000047601a98 using connection 192.168.204.103@tcp/0@lo 00000100:00100000:0.0:1776178645.859440:0:36:0:(import.c:807:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5) req@00000000abb6cbab x1862456998110144/t0(0) o38->lustre-MDT0000-lwp-OST0000@0@lo:12/10 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' 00000100:00100000:1.0:1776178645.859552:0:3352:0:(client.c:1733:ptlrpc_send_new_req()) Sending RPC req@00000000abb6cbab pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:lustre-MDT0000-lwp-OST0000_UUID:3352:1862456998110144:0@lo:38:kworker/u8:1.0 00000100:00100000:1.0:1776178645.859622:0:3352:0:(events.c:361:request_in_callback()) peer: 12345-0@lo (source: 12345-0@lo) 00000100:00100000:3.0:1776178645.859955:0:6255:0:(service.c:2152:ptlrpc_server_handle_req_in()) got req x1862456998110144 00000100:00100000:3.0:1776178645.860005:0:6255:0:(nrs_fifo.c:179:nrs_fifo_req_get()) NRS start fifo request from 12345-0@lo, seq: 6969 00000100:00100000:3.0:1776178645.860011:0:6255:0:(service.c:2324:ptlrpc_server_handle_request()) Handling RPC req@00000000f18ffde7 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_000:0+-99:3352:x1862456998110144:12345-0@lo:38:kworker/u8:1.0 00010000:02000000:3.0:1776178645.860025:0:6255:0:(ldlm_lib.c:1139:target_handle_connect()) lustre-MDT0000: Not available for connect from 0@lo (stopping) 00000100:00100000:3.0:1776178645.860093:0:6255:0:(service.c:2372:ptlrpc_server_handle_request()) Handled RPC req@00000000f18ffde7 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_000:0+-99:3352:x1862456998110144:12345-0@lo:38:kworker/u8:1.0 Request processed in 80us (470us total) trans 0 rc -19/-19 00000100:00100000:3.0:1776178645.860100:0:6255:0:(nrs_fifo.c:241:nrs_fifo_req_stop()) NRS stop fifo request from 12345-0@lo, seq: 6969 00000100:00080000:1.0:1776178645.860234:0:3352:0:(import.c:86:import_set_state_nolock()) 0000000047601a98 lustre-MDT0000_UUID: changing import state from CONNECTING to DISCONN 00000100:00080000:1.0:1776178645.860238:0:3352:0:(import.c:1429:ptlrpc_connect_interpret()) recovery of lustre-MDT0000_UUID on 192.168.204.103@tcp failed (-19) 00000100:00100000:1.0:1776178645.860242:0:3352:0:(client.c:2215:ptlrpc_check_set()) Completed RPC req@00000000abb6cbab pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:lustre-MDT0000-lwp-OST0000_UUID:3352:1862456998110144:0@lo:38:kworker/u8:1.0 00000100:00100000:2.0:1776178645.860423:0:3356:0:(client.c:1733:ptlrpc_send_new_req()) Sending RPC req@00000000598050cc pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_03:lustre-MDT0001-lwp-OST0000_UUID:3356:1862456998110208:0@lo:400:kworker/u8:1.0 00000100:00100000:2.0:1776178645.860459:0:3356:0:(events.c:361:request_in_callback()) peer: 12345-0@lo (source: 12345-0@lo) 00000100:00100000:2.0:1776178645.860550:0:3356:0:(client.c:1733:ptlrpc_send_new_req()) Sending RPC req@000000007d1fa4e3 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_03:ac61f839-0e8e-4cac-9c59-08bcfbbad330:3356:1862456998110080:0@lo:400:kworker/u8:1.0 00000100:00100000:2.0:1776178645.860591:0:3356:0:(events.c:361:request_in_callback()) peer: 12345-0@lo (source: 12345-0@lo) 00000100:00100000:1.0:1776178645.860744:0:20137:0:(service.c:2152:ptlrpc_server_handle_req_in()) got req x1862456998110080 00000100:00100000:1.0:1776178645.860785:0:20137:0:(nrs_fifo.c:179:nrs_fifo_req_get()) NRS start fifo request from 12345-0@lo, seq: 328 00000100:00100000:1.0:1776178645.860789:0:20137:0:(service.c:2324:ptlrpc_server_handle_request()) Handling RPC req@00000000f1dccdd0 pname:cluuid+ref:pid:xid:nid:opc:job ll_mgs_0000:ac61f839-0e8e-4cac-9c59-08bcfbbad330+18:3356:x1862456998110080:12345-0@lo:400:kworker/u8:1.0 00000100:00100000:1.0:1776178645.860831:0:20137:0:(service.c:2372:ptlrpc_server_handle_request()) Handled RPC req@00000000f1dccdd0 pname:cluuid+ref:pid:xid:nid:opc:job ll_mgs_0000:ac61f839-0e8e-4cac-9c59-08bcfbbad330+18:3356:x1862456998110080:12345-0@lo:400:kworker/u8:1.0 Request processed in 41us (240us total) trans 0 rc 0/0 00000100:00100000:1.0:1776178645.860837:0:20137:0:(nrs_fifo.c:241:nrs_fifo_req_stop()) NRS stop fifo request from 12345-0@lo, seq: 328 00000100:00080000:0.0:1776178645.860919:0:36:0:(pinger.c:251:ptlrpc_pinger_process_import()) lustre-MDT0000-lwp-OST0001_UUID->lustre-MDT0000_UUID: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00080000:0.0:1776178645.860924:0:36:0:(recover.c:57:ptlrpc_initiate_recovery()) lustre-MDT0000_UUID: starting recovery 00000100:00080000:0.0:1776178645.860926:0:36:0:(import.c:86:import_set_state_nolock()) 00000000dfc8db13 lustre-MDT0000_UUID: changing import state from DISCONN to CONNECTING 00000100:00080000:0.0:1776178645.860930:0:36:0:(import.c:534:import_select_connection()) lustre-MDT0000-lwp-OST0001: connect to NID 0@lo last attempt 1348 00000100:00080000:0.0:1776178645.860933:0:36:0:(import.c:581:import_select_connection()) lustre-MDT0000-lwp-OST0001: tried all connections, increasing latency to 21s 00000100:00080000:0.0:1776178645.860936:0:36:0:(import.c:615:import_select_connection()) lustre-MDT0000-lwp-OST0001: import 00000000dfc8db13 using connection 192.168.204.103@tcp/0@lo 00000100:00100000:0.0:1776178645.861043:0:36:0:(import.c:807:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5) req@00000000ef4da5b8 x1862456998110272/t0(0) o38->lustre-MDT0000-lwp-OST0001@0@lo:12/10 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' 00000100:00080000:0.0:1776178645.861100:0:36:0:(pinger.c:251:ptlrpc_pinger_process_import()) lustre-MDT0001-mdtlov_UUID->lustre-MDT0000_UUID: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00080000:0.0:1776178645.861104:0:36:0:(recover.c:57:ptlrpc_initiate_recovery()) lustre-MDT0000_UUID: starting recovery 00000100:00080000:0.0:1776178645.861106:0:36:0:(import.c:86:import_set_state_nolock()) 00000000ba4f1821 lustre-MDT0000_UUID: changing import state from DISCONN to CONNECTING 00000100:00080000:0.0:1776178645.861109:0:36:0:(import.c:534:import_select_connection()) lustre-MDT0000-osp-MDT0001: connect to NID 0@lo last attempt 1348 00000100:00080000:0.0:1776178645.861111:0:36:0:(import.c:581:import_select_connection()) lustre-MDT0000-osp-MDT0001: tried all connections, increasing latency to 21s 00000100:00080000:0.0:1776178645.861114:0:36:0:(import.c:615:import_select_connection()) lustre-MDT0000-osp-MDT0001: import 00000000ba4f1821 using connection 192.168.204.103@tcp/0@lo 00000100:00100000:0.0:1776178645.861126:0:36:0:(import.c:807:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5) req@0000000070829000 x1862456998110400/t0(0) o38->lustre-MDT0000-osp-MDT0001@0@lo:24/4 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' 00000100:00080000:0.0:1776178645.861149:0:36:0:(pinger.c:251:ptlrpc_pinger_process_import()) lustre-MDT0000-lwp-MDT0001_UUID->lustre-MDT0000_UUID: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00080000:0.0:1776178645.861152:0:36:0:(recover.c:57:ptlrpc_initiate_recovery()) lustre-MDT0000_UUID: starting recovery 00000100:00080000:0.0:1776178645.861154:0:36:0:(import.c:86:import_set_state_nolock()) 00000000a7c55b1f lustre-MDT0000_UUID: changing import state from DISCONN to CONNECTING 00000100:00080000:0.0:1776178645.861157:0:36:0:(import.c:534:import_select_connection()) lustre-MDT0000-lwp-MDT0001: connect to NID 0@lo last attempt 1348 00000100:00080000:0.0:1776178645.861158:0:36:0:(import.c:581:import_select_connection()) lustre-MDT0000-lwp-MDT0001: tried all connections, increasing latency to 21s 00000100:00080000:0.0:1776178645.861161:0:36:0:(import.c:615:import_select_connection()) lustre-MDT0000-lwp-MDT0001: import 00000000a7c55b1f using connection 192.168.204.103@tcp/0@lo 00000100:00100000:0.0:1776178645.861170:0:36:0:(import.c:807:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5) req@0000000082764007 x1862456998110592/t0(0) o38->lustre-MDT0000-lwp-MDT0001@0@lo:12/10 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' 00000100:00100000:0.0:1776178645.861192:0:3354:0:(client.c:1733:ptlrpc_send_new_req()) Sending RPC req@000000009dd1dd0f pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_01:lustre-MDT0001-mdtlov_UUID:3354:1862456998110464:0@lo:400:kworker/u8:1.0 00000100:00100000:0.0:1776178645.861223:0:3354:0:(events.c:361:request_in_callback()) peer: 12345-0@lo (source: 12345-0@lo) 00000100:00100000:0.0:1776178645.861244:0:3354:0:(ptlrpcd.c:415:ptlrpcd_check()) transfer 1 async RPCs [0->1] 00000100:00100000:0.0:1776178645.861249:0:3354:0:(client.c:1733:ptlrpc_send_new_req()) Sending RPC req@0000000098682d9e pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_01:lustre-MDT0001-lwp-OST0001_UUID:3354:1862456998110336:0@lo:400:kworker/u8:1.0 00000100:00100000:0.0:1776178645.861270:0:3354:0:(events.c:361:request_in_callback()) peer: 12345-0@lo (source: 12345-0@lo) 00000100:00100000:1.0:1776178645.866304:0:3352:0:(client.c:1733:ptlrpc_send_new_req()) Sending RPC req@00000000ef4da5b8 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:lustre-MDT0000-lwp-OST0001_UUID:3352:1862456998110272:0@lo:38:kworker/u8:1.0 00000100:00100000:1.0:1776178645.866415:0:3355:0:(client.c:1733:ptlrpc_send_new_req()) Sending RPC req@00000000715d380c pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_02:lustre-MDT0001-mdtlov_UUID:3355:1862456998110528:0@lo:400:kworker/u8:1.0 00000100:00100000:1.0:1776178645.866444:0:3355:0:(events.c:361:request_in_callback()) peer: 12345-0@lo (source: 12345-0@lo) 00000100:00100000:1.0:1776178645.866513:0:20755:0:(service.c:2152:ptlrpc_server_handle_req_in()) got req x1862456998110464 00000100:00100000:1.0:1776178645.866547:0:20755:0:(service.c:2152:ptlrpc_server_handle_req_in()) got req x1862456998110528 00000100:00100000:1.0:1776178645.866564:0:20755:0:(nrs_fifo.c:179:nrs_fifo_req_get()) NRS start fifo request from 12345-0@lo, seq: 857 00000100:00100000:1.0:1776178645.866569:0:20755:0:(service.c:2324:ptlrpc_server_handle_request()) Handling RPC req@0000000013ad9ac7 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_006:lustre-MDT0001-mdtlov_UUID+5:3354:x1862456998110464:12345-0@lo:400:kworker/u8:1.0 00000100:00100000:1.0:1776178645.866612:0:20755:0:(service.c:2372:ptlrpc_server_handle_request()) Handled RPC req@0000000013ad9ac7 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_006:lustre-MDT0001-mdtlov_UUID+5:3354:x1862456998110464:12345-0@lo:400:kworker/u8:1.0 Request processed in 43us (5390us total) trans 0 rc 0/0 00000100:00100000:1.0:1776178645.866619:0:20755:0:(nrs_fifo.c:241:nrs_fifo_req_stop()) NRS stop fifo request from 12345-0@lo, seq: 857 00000100:00100000:1.0:1776178645.866632:0:20755:0:(nrs_fifo.c:179:nrs_fifo_req_get()) NRS start fifo request from 12345-0@lo, seq: 858 00000100:00100000:1.0:1776178645.866635:0:20755:0:(service.c:2324:ptlrpc_server_handle_request()) Handling RPC req@00000000dac56e3f pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_006:lustre-MDT0001-mdtlov_UUID+5:3355:x1862456998110528:12345-0@lo:400:kworker/u8:1.0 00000100:00100000:1.0:1776178645.866655:0:20755:0:(service.c:2372:ptlrpc_server_handle_request()) Handled RPC req@00000000dac56e3f pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_006:lustre-MDT0001-mdtlov_UUID+5:3355:x1862456998110528:12345-0@lo:400:kworker/u8:1.0 Request processed in 19us (213us total) trans 0 rc 0/0 00000100:00100000:1.0:1776178645.866660:0:20755:0:(nrs_fifo.c:241:nrs_fifo_req_stop()) NRS stop fifo request from 12345-0@lo, seq: 858 00000100:00100000:1.0:1776178645.866689:0:3355:0:(client.c:2215:ptlrpc_check_set()) Completed RPC req@00000000715d380c pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_02:lustre-MDT0001-mdtlov_UUID:3355:1862456998110528:0@lo:400:kworker/u8:1.0 00000100:00100000:2.0:1776178645.866945:0:3356:0:(client.c:2215:ptlrpc_check_set()) Completed RPC req@000000007d1fa4e3 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_03:ac61f839-0e8e-4cac-9c59-08bcfbbad330:3356:1862456998110080:0@lo:400:kworker/u8:1.0 00000100:00100000:3.0:1776178645.868701:0:6257:0:(service.c:2152:ptlrpc_server_handle_req_in()) got req x1862456998110208 00000100:00100000:3.0:1776178645.868773:0:6256:0:(service.c:2152:ptlrpc_server_handle_req_in()) got req x1862456998110336 00000100:00100000:3.0:1776178645.868799:0:6256:0:(nrs_fifo.c:179:nrs_fifo_req_get()) NRS start fifo request from 12345-0@lo, seq: 1187 00000100:00100000:3.0:1776178645.868804:0:6256:0:(service.c:2324:ptlrpc_server_handle_request()) Handling RPC req@00000000f0f1167e pname:cluuid+ref:pid:xid:nid:opc:job mdt00_001:lustre-MDT0001-lwp-OST0000_UUID+5:3356:x1862456998110208:12345-0@lo:400:kworker/u8:1.0 00000100:00100000:1.0:1776178645.869079:0:3352:0:(events.c:361:request_in_callback()) peer: 12345-0@lo (source: 12345-0@lo) 00000100:00100000:1.0:1776178645.869114:0:3352:0:(client.c:1733:ptlrpc_send_new_req()) Sending RPC req@0000000070829000 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:lustre-MDT0001-mdtlov_UUID:3352:1862456998110400:0@lo:38:kworker/u8:1.0 00000100:00100000:1.0:1776178645.869162:0:3352:0:(events.c:361:request_in_callback()) peer: 12345-0@lo (source: 12345-0@lo) 00000100:00100000:1.0:1776178645.869176:0:3352:0:(client.c:1733:ptlrpc_send_new_req()) Sending RPC req@0000000082764007 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:lustre-MDT0000-lwp-MDT0001_UUID:3352:1862456998110592:0@lo:38:kworker/u8:1.0 00000100:00100000:1.0:1776178645.869199:0:3352:0:(events.c:361:request_in_callback()) peer: 12345-0@lo (source: 12345-0@lo) 00000100:00100000:1.0:1776178645.869235:0:8250:0:(service.c:2152:ptlrpc_server_handle_req_in()) got req x1862456998110272 00000100:00100000:1.0:1776178645.869250:0:8250:0:(service.c:2152:ptlrpc_server_handle_req_in()) got req x1862456998110592 00000100:00100000:1.0:1776178645.869258:0:8250:0:(nrs_fifo.c:179:nrs_fifo_req_get()) NRS start fifo request from 12345-0@lo, seq: 1188 00000100:00100000:1.0:1776178645.869262:0:8250:0:(service.c:2324:ptlrpc_server_handle_request()) Handling RPC req@00000000f20bd8f9 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_003:lustre-MDT0001-lwp-OST0001_UUID+5:3354:x1862456998110336:12345-0@lo:400:kworker/u8:1.0 00000100:00100000:1.0:1776178645.869305:0:8250:0:(service.c:2372:ptlrpc_server_handle_request()) Handled RPC req@00000000f20bd8f9 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_003:lustre-MDT0001-lwp-OST0001_UUID+5:3354:x1862456998110336:12345-0@lo:400:kworker/u8:1.0 Request processed in 42us (8035us total) trans 0 rc 0/0 00000100:00100000:1.0:1776178645.869312:0:8250:0:(nrs_fifo.c:241:nrs_fifo_req_stop()) NRS stop fifo request from 12345-0@lo, seq: 1188 00000100:00100000:1.0:1776178645.869328:0:8250:0:(nrs_fifo.c:179:nrs_fifo_req_get()) NRS start fifo request from 12345-0@lo, seq: 6970 00000100:00100000:1.0:1776178645.869332:0:8250:0:(service.c:2324:ptlrpc_server_handle_request()) Handling RPC req@0000000025f8393e pname:cluuid+ref:pid:xid:nid:opc:job mdt00_003:0+-99:3352:x1862456998110272:12345-0@lo:38:kworker/u8:1.0 00010000:02000000:1.0:1776178645.869344:0:8250:0:(ldlm_lib.c:1139:target_handle_connect()) lustre-MDT0000: Not available for connect from 0@lo (stopping) 00000100:00100000:1.0:1776178645.869375:0:8250:0:(service.c:2372:ptlrpc_server_handle_request()) Handled RPC req@0000000025f8393e pname:cluuid+ref:pid:xid:nid:opc:job mdt00_003:0+-99:3352:x1862456998110272:12345-0@lo:38:kworker/u8:1.0 Request processed in 41us (297us total) trans 0 rc -19/-19 00000100:00100000:1.0:1776178645.869382:0:8250:0:(nrs_fifo.c:241:nrs_fifo_req_stop()) NRS stop fifo request from 12345-0@lo, seq: 6970 00000100:00100000:1.0:1776178645.869392:0:8250:0:(nrs_fifo.c:179:nrs_fifo_req_get()) NRS start fifo request from 12345-0@lo, seq: 6971 00000100:00100000:1.0:1776178645.869395:0:8250:0:(service.c:2324:ptlrpc_server_handle_request()) Handling RPC req@000000002dd26ce5 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_003:0+-99:3352:x1862456998110592:12345-0@lo:38:kworker/u8:1.0 00010000:02000000:1.0:1776178645.869402:0:8250:0:(ldlm_lib.c:1139:target_handle_connect()) lustre-MDT0000: Not available for connect from 0@lo (stopping) 00000100:00100000:1.0:1776178645.869432:0:8250:0:(service.c:2372:ptlrpc_server_handle_request()) Handled RPC req@000000002dd26ce5 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_003:0+-99:3352:x1862456998110592:12345-0@lo:38:kworker/u8:1.0 Request processed in 34us (231us total) trans 0 rc -19/-19 00000100:00100000:1.0:1776178645.869437:0:8250:0:(nrs_fifo.c:241:nrs_fifo_req_stop()) NRS stop fifo request from 12345-0@lo, seq: 6971 00000100:00100000:1.0:1776178645.869455:0:14128:0:(service.c:2152:ptlrpc_server_handle_req_in()) got req x1862456998110400 00000100:00100000:1.0:1776178645.869470:0:14128:0:(nrs_fifo.c:179:nrs_fifo_req_get()) NRS start fifo request from 12345-0@lo, seq: 794 00000100:00100000:1.0:1776178645.869473:0:14128:0:(service.c:2324:ptlrpc_server_handle_request()) Handling RPC req@00000000505e62f8 pname:cluuid+ref:pid:xid:nid:opc:job mdt_out00_002:0+-99:3352:x1862456998110400:12345-0@lo:38:kworker/u8:1.0 00010000:02000000:1.0:1776178645.869479:0:14128:0:(ldlm_lib.c:1139:target_handle_connect()) lustre-MDT0000: Not available for connect from 0@lo (stopping) 00000100:00100000:1.0:1776178645.869501:0:14128:0:(service.c:2372:ptlrpc_server_handle_request()) Handled RPC req@00000000505e62f8 pname:cluuid+ref:pid:xid:nid:opc:job mdt_out00_002:0+-99:3352:x1862456998110400:12345-0@lo:38:kworker/u8:1.0 Request processed in 27us (339us total) trans 0 rc -19/-19 00000100:00100000:1.0:1776178645.869507:0:14128:0:(nrs_fifo.c:241:nrs_fifo_req_stop()) NRS stop fifo request from 12345-0@lo, seq: 794 00000100:00080000:1.0:1776178645.869541:0:3352:0:(import.c:86:import_set_state_nolock()) 00000000dfc8db13 lustre-MDT0000_UUID: changing import state from CONNECTING to DISCONN 00000100:00080000:1.0:1776178645.869545:0:3352:0:(import.c:1429:ptlrpc_connect_interpret()) recovery of lustre-MDT0000_UUID on 192.168.204.103@tcp failed (-19) 00000100:00100000:1.0:1776178645.869549:0:3352:0:(client.c:2215:ptlrpc_check_set()) Completed RPC req@00000000ef4da5b8 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:lustre-MDT0000-lwp-OST0001_UUID:3352:1862456998110272:0@lo:38:kworker/u8:1.0 00000100:00080000:1.0:1776178645.869558:0:3352:0:(import.c:86:import_set_state_nolock()) 00000000ba4f1821 lustre-MDT0000_UUID: changing import state from CONNECTING to DISCONN 00000100:00080000:1.0:1776178645.869560:0:3352:0:(import.c:1429:ptlrpc_connect_interpret()) recovery of lustre-MDT0000_UUID on 192.168.204.103@tcp failed (-19) 00000100:00100000:1.0:1776178645.869562:0:3352:0:(client.c:2215:ptlrpc_check_set()) Completed RPC req@0000000070829000 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:lustre-MDT0001-mdtlov_UUID:3352:1862456998110400:0@lo:38:kworker/u8:1.0 00000100:00080000:1.0:1776178645.869567:0:3352:0:(import.c:86:import_set_state_nolock()) 00000000a7c55b1f lustre-MDT0000_UUID: changing import state from CONNECTING to DISCONN 00000100:00080000:1.0:1776178645.869570:0:3352:0:(import.c:1429:ptlrpc_connect_interpret()) recovery of lustre-MDT0000_UUID on 192.168.204.103@tcp failed (-19) 00000100:00100000:1.0:1776178645.869572:0:3352:0:(client.c:2215:ptlrpc_check_set()) Completed RPC req@0000000082764007 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:lustre-MDT0000-lwp-MDT0001_UUID:3352:1862456998110592:0@lo:38:kworker/u8:1.0 00000100:00100000:0.0:1776178645.870890:0:3354:0:(client.c:2215:ptlrpc_check_set()) Completed RPC req@0000000098682d9e pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_01:lustre-MDT0001-lwp-OST0001_UUID:3354:1862456998110336:0@lo:400:kworker/u8:1.0 00000100:00100000:0.0:1776178645.870909:0:3354:0:(client.c:2215:ptlrpc_check_set()) Completed RPC req@000000009dd1dd0f pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_01:lustre-MDT0001-mdtlov_UUID:3354:1862456998110464:0@lo:400:kworker/u8:1.0 00000100:00100000:3.0:1776178645.871987:0:6256:0:(service.c:2372:ptlrpc_server_handle_request()) Handled RPC req@00000000f0f1167e pname:cluuid+ref:pid:xid:nid:opc:job mdt00_001:lustre-MDT0001-lwp-OST0000_UUID+5:3356:x1862456998110208:12345-0@lo:400:kworker/u8:1.0 Request processed in 3176us (11522us total) trans 0 rc 0/0 00000100:00100000:3.0:1776178645.872006:0:6256:0:(nrs_fifo.c:241:nrs_fifo_req_stop()) NRS stop fifo request from 12345-0@lo, seq: 1187 00000100:00100000:2.0:1776178645.872177:0:3356:0:(client.c:2215:ptlrpc_check_set()) Completed RPC req@00000000598050cc pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_03:lustre-MDT0001-lwp-OST0000_UUID:3356:1862456998110208:0@lo:400:kworker/u8:1.0 00010000:00010000:1.0:1776178646.497819:0:9225:0:(ldlm_lockd.c:2792:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:2.0:1776178647.392341:0:3356:0:(client.c:1733:ptlrpc_send_new_req()) Sending RPC req@0000000037d88937 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_03:lustre-MDT0001-mdtlov_UUID:3356:1862456998110656:0@lo:13:osp-pre-1-1.0 00000100:00100000:2.0:1776178647.392415:0:3356:0:(events.c:361:request_in_callback()) peer: 12345-0@lo (source: 12345-0@lo) 00000100:00100000:1.0:1776178647.392603:0:9554:0:(service.c:2152:ptlrpc_server_handle_req_in()) got req x1862456998110656 00000100:00100000:1.0:1776178647.392661:0:9554:0:(nrs_fifo.c:179:nrs_fifo_req_get()) NRS start fifo request from 12345-0@lo, seq: 661 00000100:00100000:1.0:1776178647.392667:0:9554:0:(service.c:2324:ptlrpc_server_handle_request()) Handling RPC req@0000000078ae567e pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_create00:lustre-MDT0001-mdtlov_UUID+5:3356:x1862456998110656:12345-0@lo:13:osp-pre-1-1.0 00002000:00000024:1.0:1776178647.392698:0:9554:0:(ofd_obd.c:740:ofd_statfs()) blocks cached 229376 granted 491746368 pending 0 free 3923689472 avail 3692167168 00000100:00100000:1.0:1776178647.392752:0:9554:0:(service.c:2372:ptlrpc_server_handle_request()) Handled RPC req@0000000078ae567e pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_create00:lustre-MDT0001-mdtlov_UUID+5:3356:x1862456998110656:12345-0@lo:13:osp-pre-1-1.0 Request processed in 84us (338us total) trans 0 rc 0/0 00000100:00100000:1.0:1776178647.392759:0:9554:0:(nrs_fifo.c:241:nrs_fifo_req_stop()) NRS stop fifo request from 12345-0@lo, seq: 661 00000100:00100000:3.0:1776178647.392850:0:3356:0:(client.c:2215:ptlrpc_check_set()) Completed RPC req@0000000037d88937 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_03:lustre-MDT0001-mdtlov_UUID:3356:1862456998110656:0@lo:13:osp-pre-1-1.0 00010000:00010000:2.0:1776178647.520451:0:6244:0:(ldlm_lockd.c:2792:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:0.0:1776178647.904306:0:3354:0:(ptlrpcd.c:415:ptlrpcd_check()) transfer 1 async RPCs [0->1] 00000100:00100000:0.0:1776178647.904330:0:3354:0:(client.c:1733:ptlrpc_send_new_req()) Sending RPC req@000000004420855e pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_01:lustre-MDT0001-mdtlov_UUID:3354:1862456998110720:0@lo:13:osp-pre-0-1.0 00000100:00100000:0.0:1776178647.904394:0:3354:0:(events.c:361:request_in_callback()) peer: 12345-0@lo (source: 12345-0@lo) 00000100:00100000:2.0:1776178647.904528:0:8231:0:(service.c:2152:ptlrpc_server_handle_req_in()) got req x1862456998110720 00000100:00100000:2.0:1776178647.904588:0:8231:0:(nrs_fifo.c:179:nrs_fifo_req_get()) NRS start fifo request from 12345-0@lo, seq: 662 00000100:00100000:2.0:1776178647.904594:0:8231:0:(service.c:2324:ptlrpc_server_handle_request()) Handling RPC req@000000009dd1dd0f pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_create00:lustre-MDT0001-mdtlov_UUID+5:3354:x1862456998110720:12345-0@lo:13:osp-pre-0-1.0 00002000:00000024:2.0:1776178647.904621:0:8231:0:(ofd_obd.c:740:ofd_statfs()) blocks cached 315392 granted 484883776 pending 0 free 3923689472 avail 3692167168 00000100:00100000:2.0:1776178647.904676:0:8231:0:(service.c:2372:ptlrpc_server_handle_request()) Handled RPC req@000000009dd1dd0f pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_create00:lustre-MDT0001-mdtlov_UUID+5:3354:x1862456998110720:12345-0@lo:13:osp-pre-0-1.0 Request processed in 81us (282us total) trans 0 rc 0/0 00000100:00100000:2.0:1776178647.904683:0:8231:0:(nrs_fifo.c:241:nrs_fifo_req_stop()) NRS stop fifo request from 12345-0@lo, seq: 662 00000100:00100000:3.0:1776178647.904901:0:3354:0:(client.c:2215:ptlrpc_check_set()) Completed RPC req@000000004420855e pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_01:lustre-MDT0001-mdtlov_UUID:3354:1862456998110720:0@lo:13:osp-pre-0-1.0 00010000:00010000:1.0:1776178648.545033:0:6245:0:(ldlm_lockd.c:2792:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:1.0:1776178649.377550:0:3348:0:(events.c:361:request_in_callback()) peer: 12345-192.168.204.3@tcp (source: 12345-192.168.204.3@tcp) 00000100:00100000:1.0:1776178649.377609:0:3348:0:(events.c:361:request_in_callback()) peer: 12345-192.168.204.3@tcp (source: 12345-192.168.204.3@tcp) 00000100:00100000:1.0:1776178649.377637:0:3348:0:(events.c:361:request_in_callback()) peer: 12345-192.168.204.3@tcp (source: 12345-192.168.204.3@tcp) 00000100:00100000:2.0:1776178649.377802:0:20140:0:(service.c:2152:ptlrpc_server_handle_req_in()) got req x1862456979136832 00000100:00100000:2.0:1776178649.377854:0:20140:0:(nrs_fifo.c:179:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.204.3@tcp, seq: 329 00000100:00100000:2.0:1776178649.377861:0:20140:0:(service.c:2324:ptlrpc_server_handle_request()) Handling RPC req@00000000322375a6 pname:cluuid+ref:pid:xid:nid:opc:job ll_mgs_0002:5b5c66b1-1a2f-4651-8548-0c75b9a38a94+9:2271:x1862456979136832:12345-192.168.204.3@tcp:400:kworker/u8:3.0 00000100:00100000:2.0:1776178649.377947:0:20140:0:(service.c:2372:ptlrpc_server_handle_request()) Handled RPC req@00000000322375a6 pname:cluuid+ref:pid:xid:nid:opc:job ll_mgs_0002:5b5c66b1-1a2f-4651-8548-0c75b9a38a94+9:2271:x1862456979136832:12345-192.168.204.3@tcp:400:kworker/u8:3.0 Request processed in 85us (406us total) trans 0 rc 0/0 00000100:00100000:2.0:1776178649.377955:0:20140:0:(nrs_fifo.c:241:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.204.3@tcp, seq: 329 00000100:00100000:1.0:1776178649.378245:0:3348:0:(events.c:361:request_in_callback()) peer: 12345-192.168.204.3@tcp (source: 12345-192.168.204.3@tcp) 00000100:00100000:2.0:1776178649.378613:0:20762:0:(service.c:2152:ptlrpc_server_handle_req_in()) got req x1862456979137024 00000100:00100000:2.0:1776178649.378651:0:20762:0:(nrs_fifo.c:179:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.204.3@tcp, seq: 859 00000100:00100000:2.0:1776178649.378656:0:20762:0:(service.c:2324:ptlrpc_server_handle_request()) Handling RPC req@0000000063c20b43 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_013:b50963bd-8eba-4432-9a54-8888d00e3fa1+8:2271:x1862456979137024:12345-192.168.204.3@tcp:400:kworker/u8:3.0 00000100:00100000:2.0:1776178649.378689:0:20762:0:(service.c:2372:ptlrpc_server_handle_request()) Handled RPC req@0000000063c20b43 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_013:b50963bd-8eba-4432-9a54-8888d00e3fa1+8:2271:x1862456979137024:12345-192.168.204.3@tcp:400:kworker/u8:3.0 Request processed in 32us (445us total) trans 0 rc 0/0 00000100:00100000:2.0:1776178649.378695:0:20762:0:(nrs_fifo.c:241:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.204.3@tcp, seq: 859 00000100:00100000:0.0:1776178649.381678:0:6257:0:(service.c:2152:ptlrpc_server_handle_req_in()) got req x1862456979136960 00000100:00100000:0.0:1776178649.381800:0:6257:0:(service.c:2152:ptlrpc_server_handle_req_in()) got req x1862456979136896 00000100:00100000:0.0:1776178649.381829:0:6257:0:(nrs_fifo.c:179:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.204.3@tcp, seq: 1189 00000100:00100000:0.0:1776178649.381835:0:6257:0:(service.c:2324:ptlrpc_server_handle_request()) Handling RPC req@00000000165897bf pname:cluuid+ref:pid:xid:nid:opc:job mdt00_002:b50963bd-8eba-4432-9a54-8888d00e3fa1+6:2270:x1862456979136960:12345-192.168.204.3@tcp:400:kworker/u8:3.0 00000100:00100000:0.0:1776178649.381930:0:6257:0:(service.c:2372:ptlrpc_server_handle_request()) Handled RPC req@00000000165897bf pname:cluuid+ref:pid:xid:nid:opc:job mdt00_002:b50963bd-8eba-4432-9a54-8888d00e3fa1+6:2270:x1862456979136960:12345-192.168.204.3@tcp:400:kworker/u8:3.0 Request processed in 94us (4322us total) trans 0 rc 0/0 00000100:00100000:0.0:1776178649.381938:0:6257:0:(nrs_fifo.c:241:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.204.3@tcp, seq: 1189 00000100:00100000:0.0:1776178649.381958:0:6257:0:(nrs_fifo.c:179:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.204.3@tcp, seq: 6972 00000100:00100000:0.0:1776178649.381962:0:6257:0:(service.c:2324:ptlrpc_server_handle_request()) Handling RPC req@000000006c748d80 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_002:0+-99:2269:x1862456979136896:12345-192.168.204.3@tcp:38:kworker/u8:3.0 00010000:02000000:0.0:1776178649.381976:0:6257:0:(ldlm_lib.c:1139:target_handle_connect()) lustre-MDT0000: Not available for connect from 192.168.204.3@tcp (stopping) 00000100:00100000:0.0:1776178649.382005:0:6257:0:(service.c:2372:ptlrpc_server_handle_request()) Handled RPC req@000000006c748d80 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_002:0+-99:2269:x1862456979136896:12345-192.168.204.3@tcp:38:kworker/u8:3.0 Request processed in 41us (4367us total) trans 0 rc -19/-19 00000100:00100000:0.0:1776178649.382010:0:6257:0:(nrs_fifo.c:241:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.204.3@tcp, seq: 6972 00000100:00100000:1.0:1776178649.392507:0:3348:0:(events.c:361:request_in_callback()) peer: 12345-192.168.204.3@tcp (source: 12345-192.168.204.3@tcp) 00000100:00100000:3.0:1776178649.393330:0:14440:0:(service.c:2152:ptlrpc_server_handle_req_in()) got req x1862456979137088 00000100:00100000:3.0:1776178649.393428:0:14440:0:(nrs_fifo.c:179:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.204.3@tcp, seq: 860 00000100:00100000:3.0:1776178649.393438:0:14440:0:(service.c:2324:ptlrpc_server_handle_request()) Handling RPC req@0000000092f98018 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_004:b50963bd-8eba-4432-9a54-8888d00e3fa1+9:2270:x1862456979137088:12345-192.168.204.3@tcp:400:kworker/u8:3.0 00000100:00100000:3.0:1776178649.393559:0:14440:0:(service.c:2372:ptlrpc_server_handle_request()) Handled RPC req@0000000092f98018 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_004:b50963bd-8eba-4432-9a54-8888d00e3fa1+9:2270:x1862456979137088:12345-192.168.204.3@tcp:400:kworker/u8:3.0 Request processed in 121us (1062us total) trans 0 rc 0/0 00000100:00100000:3.0:1776178649.393568:0:14440:0:(nrs_fifo.c:241:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.204.3@tcp, seq: 860 00010000:00010000:2.0:1776178649.573449:0:9224:0:(ldlm_lockd.c:2792:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00010000:00010000:1.0:1776178650.592418:0:9225:0:(ldlm_lockd.c:2792:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00080000:0.0:1776178650.980190:0:36:0:(pinger.c:251:ptlrpc_pinger_process_import()) lustre-MDT0000-lwp-OST0000_UUID->lustre-MDT0000_UUID: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00080000:0.0:1776178650.980199:0:36:0:(recover.c:57:ptlrpc_initiate_recovery()) lustre-MDT0000_UUID: starting recovery 00000100:00080000:0.0:1776178650.980202:0:36:0:(import.c:86:import_set_state_nolock()) 0000000047601a98 lustre-MDT0000_UUID: changing import state from DISCONN to CONNECTING 00000100:00080000:0.0:1776178650.980211:0:36:0:(import.c:534:import_select_connection()) lustre-MDT0000-lwp-OST0000: connect to NID 0@lo last attempt 1353 00000100:00080000:0.0:1776178650.980215:0:36:0:(import.c:581:import_select_connection()) lustre-MDT0000-lwp-OST0000: tried all connections, increasing latency to 21s 00000100:00080000:0.0:1776178650.980220:0:36:0:(import.c:615:import_select_connection()) lustre-MDT0000-lwp-OST0000: import 0000000047601a98 using connection 192.168.204.103@tcp/0@lo 00000100:00100000:0.0:1776178650.980243:0:36:0:(import.c:807:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5) req@00000000722afaa5 x1862456998110848/t0(0) o38->lustre-MDT0000-lwp-OST0000@0@lo:12/10 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' 00000100:00080000:0.0:1776178650.980292:0:36:0:(pinger.c:251:ptlrpc_pinger_process_import()) lustre-MDT0000-lwp-OST0001_UUID->lustre-MDT0000_UUID: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00080000:0.0:1776178650.980295:0:36:0:(recover.c:57:ptlrpc_initiate_recovery()) lustre-MDT0000_UUID: starting recovery 00000100:00080000:0.0:1776178650.980297:0:36:0:(import.c:86:import_set_state_nolock()) 00000000dfc8db13 lustre-MDT0000_UUID: changing import state from DISCONN to CONNECTING 00000100:00080000:0.0:1776178650.980300:0:36:0:(import.c:534:import_select_connection()) lustre-MDT0000-lwp-OST0001: connect to NID 0@lo last attempt 1353 00000100:00080000:0.0:1776178650.980302:0:36:0:(import.c:581:import_select_connection()) lustre-MDT0000-lwp-OST0001: tried all connections, increasing latency to 21s 00000100:00080000:0.0:1776178650.980306:0:36:0:(import.c:615:import_select_connection()) lustre-MDT0000-lwp-OST0001: import 00000000dfc8db13 using connection 192.168.204.103@tcp/0@lo 00000100:00100000:0.0:1776178650.980316:0:36:0:(import.c:807:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5) req@00000000598050cc x1862456998110976/t0(0) o38->lustre-MDT0000-lwp-OST0001@0@lo:12/10 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' 00000100:00080000:0.0:1776178650.980329:0:36:0:(pinger.c:251:ptlrpc_pinger_process_import()) lustre-MDT0001-mdtlov_UUID->lustre-MDT0000_UUID: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00080000:0.0:1776178650.980331:0:36:0:(recover.c:57:ptlrpc_initiate_recovery()) lustre-MDT0000_UUID: starting recovery 00000100:00080000:0.0:1776178650.980332:0:36:0:(import.c:86:import_set_state_nolock()) 00000000ba4f1821 lustre-MDT0000_UUID: changing import state from DISCONN to CONNECTING 00000100:00080000:0.0:1776178650.980335:0:36:0:(import.c:534:import_select_connection()) lustre-MDT0000-osp-MDT0001: connect to NID 0@lo last attempt 1353 00000100:00080000:0.0:1776178650.980337:0:36:0:(import.c:581:import_select_connection()) lustre-MDT0000-osp-MDT0001: tried all connections, increasing latency to 21s 00000100:00080000:0.0:1776178650.980340:0:36:0:(import.c:615:import_select_connection()) lustre-MDT0000-osp-MDT0001: import 00000000ba4f1821 using connection 192.168.204.103@tcp/0@lo 00000100:00100000:0.0:1776178650.980349:0:36:0:(import.c:807:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5) req@0000000070829000 x1862456998111104/t0(0) o38->lustre-MDT0000-osp-MDT0001@0@lo:24/4 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' 00000100:00080000:0.0:1776178650.980365:0:36:0:(pinger.c:251:ptlrpc_pinger_process_import()) lustre-MDT0000-lwp-MDT0001_UUID->lustre-MDT0000_UUID: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00080000:0.0:1776178650.980368:0:36:0:(recover.c:57:ptlrpc_initiate_recovery()) lustre-MDT0000_UUID: starting recovery 00000100:00080000:0.0:1776178650.980370:0:36:0:(import.c:86:import_set_state_nolock()) 00000000a7c55b1f lustre-MDT0000_UUID: changing import state from DISCONN to CONNECTING 00000100:00080000:0.0:1776178650.980373:0:36:0:(import.c:534:import_select_connection()) lustre-MDT0000-lwp-MDT0001: connect to NID 0@lo last attempt 1353 00000100:00080000:0.0:1776178650.980374:0:36:0:(import.c:581:import_select_connection()) lustre-MDT0000-lwp-MDT0001: tried all connections, increasing latency to 21s 00000100:00080000:0.0:1776178650.980377:0:36:0:(import.c:615:import_select_connection()) lustre-MDT0000-lwp-MDT0001: import 00000000a7c55b1f using connection 192.168.204.103@tcp/0@lo 00000100:00100000:0.0:1776178650.980385:0:36:0:(import.c:807:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5) req@00000000715d380c x1862456998111296/t0(0) o38->lustre-MDT0000-lwp-MDT0001@0@lo:12/10 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' 00000100:00100000:0.0:1776178650.980414:0:3356:0:(client.c:1733:ptlrpc_send_new_req()) Sending RPC req@0000000082764007 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_03:lustre-MDT0001-lwp-OST0001_UUID:3356:1862456998111040:0@lo:400:kworker/u8:1.0 00000100:00100000:0.0:1776178650.980476:0:3356:0:(events.c:361:request_in_callback()) peer: 12345-0@lo (source: 12345-0@lo) 00000100:00100000:0.0:1776178650.980511:0:3356:0:(ptlrpcd.c:415:ptlrpcd_check()) transfer 1 async RPCs [2->3] 00000100:00100000:0.0:1776178650.980517:0:3356:0:(client.c:1733:ptlrpc_send_new_req()) Sending RPC req@0000000037b0061a pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_03:lustre-MDT0001-lwp-OST0000_UUID:3356:1862456998110912:0@lo:400:kworker/u8:1.0 00000100:00100000:0.0:1776178650.980549:0:3356:0:(events.c:361:request_in_callback()) peer: 12345-0@lo (source: 12345-0@lo) 00000100:00100000:3.0:1776178650.980822:0:6256:0:(service.c:2152:ptlrpc_server_handle_req_in()) got req x1862456998111040 00000100:00100000:3.0:1776178650.980871:0:6256:0:(service.c:2152:ptlrpc_server_handle_req_in()) got req x1862456998110912 00000100:00100000:3.0:1776178650.980890:0:6256:0:(nrs_fifo.c:179:nrs_fifo_req_get()) NRS start fifo request from 12345-0@lo, seq: 1190 00000100:00100000:3.0:1776178650.980896:0:6256:0:(service.c:2324:ptlrpc_server_handle_request()) Handling RPC req@00000000abb6cbab pname:cluuid+ref:pid:xid:nid:opc:job mdt00_001:lustre-MDT0001-lwp-OST0001_UUID+5:3356:x1862456998111040:12345-0@lo:400:kworker/u8:1.0 00000100:00100000:3.0:1776178650.980943:0:6256:0:(service.c:2372:ptlrpc_server_handle_request()) Handled RPC req@00000000abb6cbab pname:cluuid+ref:pid:xid:nid:opc:job mdt00_001:lustre-MDT0001-lwp-OST0001_UUID+5:3356:x1862456998111040:12345-0@lo:400:kworker/u8:1.0 Request processed in 46us (467us total) trans 0 rc 0/0 00000100:00100000:3.0:1776178650.980950:0:6256:0:(nrs_fifo.c:241:nrs_fifo_req_stop()) NRS stop fifo request from 12345-0@lo, seq: 1190 00000100:00100000:3.0:1776178650.980965:0:6256:0:(nrs_fifo.c:179:nrs_fifo_req_get()) NRS start fifo request from 12345-0@lo, seq: 1191 00000100:00100000:3.0:1776178650.980968:0:6256:0:(service.c:2324:ptlrpc_server_handle_request()) Handling RPC req@00000000fb022b32 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_001:lustre-MDT0001-lwp-OST0000_UUID+5:3356:x1862456998110912:12345-0@lo:400:kworker/u8:1.0 00000100:00100000:3.0:1776178650.980992:0:6256:0:(service.c:2372:ptlrpc_server_handle_request()) Handled RPC req@00000000fb022b32 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_001:lustre-MDT0001-lwp-OST0000_UUID+5:3356:x1862456998110912:12345-0@lo:400:kworker/u8:1.0 Request processed in 24us (444us total) trans 0 rc 0/0 00000100:00100000:3.0:1776178650.980997:0:6256:0:(nrs_fifo.c:241:nrs_fifo_req_stop()) NRS stop fifo request from 12345-0@lo, seq: 1191 00000100:00100000:3.0:1776178650.981016:0:3354:0:(client.c:1733:ptlrpc_send_new_req()) Sending RPC req@0000000098682d9e pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_01:ac61f839-0e8e-4cac-9c59-08bcfbbad330:3354:1862456998110784:0@lo:400:kworker/u8:1.0 00000100:00100000:3.0:1776178650.981045:0:3354:0:(events.c:361:request_in_callback()) peer: 12345-0@lo (source: 12345-0@lo) 00000100:00100000:3.0:1776178650.981080:0:3354:0:(client.c:1733:ptlrpc_send_new_req()) Sending RPC req@000000007d1fa4e3 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_01:lustre-MDT0001-mdtlov_UUID:3354:1862456998111232:0@lo:400:kworker/u8:1.0 00000100:00100000:3.0:1776178650.981115:0:3354:0:(events.c:361:request_in_callback()) peer: 12345-0@lo (source: 12345-0@lo) 00000100:00100000:3.0:1776178650.981130:0:3354:0:(ptlrpcd.c:415:ptlrpcd_check()) transfer 1 async RPCs [0->1] 00000100:00100000:3.0:1776178650.981135:0:3354:0:(client.c:1733:ptlrpc_send_new_req()) Sending RPC req@00000000ef4da5b8 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_01:lustre-MDT0001-mdtlov_UUID:3354:1862456998111168:0@lo:400:kworker/u8:1.0 00000100:00100000:3.0:1776178650.981155:0:3354:0:(events.c:361:request_in_callback()) peer: 12345-0@lo (source: 12345-0@lo) 00000100:00100000:3.0:1776178650.981191:0:14440:0:(service.c:2152:ptlrpc_server_handle_req_in()) got req x1862456998111232 00000100:00100000:3.0:1776178650.981213:0:14440:0:(service.c:2152:ptlrpc_server_handle_req_in()) got req x1862456998111168 00000100:00100000:3.0:1776178650.981226:0:14440:0:(nrs_fifo.c:179:nrs_fifo_req_get()) NRS start fifo request from 12345-0@lo, seq: 861 00000100:00100000:3.0:1776178650.981229:0:14440:0:(service.c:2324:ptlrpc_server_handle_request()) Handling RPC req@0000000037d88937 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_004:lustre-MDT0001-mdtlov_UUID+5:3354:x1862456998111232:12345-0@lo:400:kworker/u8:1.0 00000100:00100000:3.0:1776178650.981260:0:14440:0:(service.c:2372:ptlrpc_server_handle_request()) Handled RPC req@0000000037d88937 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_004:lustre-MDT0001-mdtlov_UUID+5:3354:x1862456998111232:12345-0@lo:400:kworker/u8:1.0 Request processed in 30us (145us total) trans 0 rc 0/0 00000100:00100000:3.0:1776178650.981266:0:14440:0:(nrs_fifo.c:241:nrs_fifo_req_stop()) NRS stop fifo request from 12345-0@lo, seq: 861 00000100:00100000:3.0:1776178650.981277:0:14440:0:(nrs_fifo.c:179:nrs_fifo_req_get()) NRS start fifo request from 12345-0@lo, seq: 862 00000100:00100000:3.0:1776178650.981280:0:14440:0:(service.c:2324:ptlrpc_server_handle_request()) Handling RPC req@000000008f94e9dd pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_004:lustre-MDT0001-mdtlov_UUID+5:3354:x1862456998111168:12345-0@lo:400:kworker/u8:1.0 00000100:00100000:3.0:1776178650.981302:0:14440:0:(service.c:2372:ptlrpc_server_handle_request()) Handled RPC req@000000008f94e9dd pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_004:lustre-MDT0001-mdtlov_UUID+5:3354:x1862456998111168:12345-0@lo:400:kworker/u8:1.0 Request processed in 22us (147us total) trans 0 rc 0/0 00000100:00100000:3.0:1776178650.981307:0:14440:0:(nrs_fifo.c:241:nrs_fifo_req_stop()) NRS stop fifo request from 12345-0@lo, seq: 862 00000100:00100000:3.0:1776178650.981338:0:3354:0:(client.c:2215:ptlrpc_check_set()) Completed RPC req@00000000ef4da5b8 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_01:lustre-MDT0001-mdtlov_UUID:3354:1862456998111168:0@lo:400:kworker/u8:1.0 00000100:00100000:3.0:1776178650.981349:0:3354:0:(client.c:2215:ptlrpc_check_set()) Completed RPC req@000000007d1fa4e3 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_01:lustre-MDT0001-mdtlov_UUID:3354:1862456998111232:0@lo:400:kworker/u8:1.0 00000100:00100000:2.0:1776178650.981549:0:3352:0:(client.c:1733:ptlrpc_send_new_req()) Sending RPC req@00000000722afaa5 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:lustre-MDT0000-lwp-OST0000_UUID:3352:1862456998110848:0@lo:38:kworker/u8:1.0 00000100:00100000:2.0:1776178650.981578:0:3352:0:(events.c:361:request_in_callback()) peer: 12345-0@lo (source: 12345-0@lo) 00000100:00100000:2.0:1776178650.981601:0:3352:0:(client.c:1733:ptlrpc_send_new_req()) Sending RPC req@00000000598050cc pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:lustre-MDT0000-lwp-OST0001_UUID:3352:1862456998110976:0@lo:38:kworker/u8:1.0 00000100:00100000:2.0:1776178650.981622:0:3352:0:(events.c:361:request_in_callback()) peer: 12345-0@lo (source: 12345-0@lo) 00000100:00100000:2.0:1776178650.981636:0:3352:0:(client.c:1733:ptlrpc_send_new_req()) Sending RPC req@0000000070829000 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:lustre-MDT0001-mdtlov_UUID:3352:1862456998111104:0@lo:38:kworker/u8:1.0 00000100:00100000:2.0:1776178650.981668:0:3352:0:(events.c:361:request_in_callback()) peer: 12345-0@lo (source: 12345-0@lo) 00000100:00100000:2.0:1776178650.981683:0:3352:0:(client.c:1733:ptlrpc_send_new_req()) Sending RPC req@00000000715d380c pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:lustre-MDT0000-lwp-MDT0001_UUID:3352:1862456998111296:0@lo:38:kworker/u8:1.0 00000100:00100000:2.0:1776178650.981700:0:3352:0:(events.c:361:request_in_callback()) peer: 12345-0@lo (source: 12345-0@lo) 00000100:00100000:2.0:1776178650.981722:0:8250:0:(service.c:2152:ptlrpc_server_handle_req_in()) got req x1862456998110848 00000100:00100000:2.0:1776178650.981736:0:8250:0:(service.c:2152:ptlrpc_server_handle_req_in()) got req x1862456998110976 00000100:00100000:2.0:1776178650.981744:0:8250:0:(service.c:2152:ptlrpc_server_handle_req_in()) got req x1862456998111296 00000100:00100000:2.0:1776178650.981752:0:8250:0:(nrs_fifo.c:179:nrs_fifo_req_get()) NRS start fifo request from 12345-0@lo, seq: 6973 00000100:00100000:2.0:1776178650.981755:0:8250:0:(service.c:2324:ptlrpc_server_handle_request()) Handling RPC req@00000000e2b18a6d pname:cluuid+ref:pid:xid:nid:opc:job mdt00_003:0+-99:3352:x1862456998110848:12345-0@lo:38:kworker/u8:1.0 00010000:02000000:2.0:1776178650.981768:0:8250:0:(ldlm_lib.c:1139:target_handle_connect()) lustre-MDT0000: Not available for connect from 0@lo (stopping) 00000100:00100000:2.0:1776178650.981799:0:8250:0:(service.c:2372:ptlrpc_server_handle_request()) Handled RPC req@00000000e2b18a6d pname:cluuid+ref:pid:xid:nid:opc:job mdt00_003:0+-99:3352:x1862456998110848:12345-0@lo:38:kworker/u8:1.0 Request processed in 43us (221us total) trans 0 rc -19/-19 00000100:00100000:2.0:1776178650.981805:0:8250:0:(nrs_fifo.c:241:nrs_fifo_req_stop()) NRS stop fifo request from 12345-0@lo, seq: 6973 00000100:00100000:2.0:1776178650.981817:0:8250:0:(nrs_fifo.c:179:nrs_fifo_req_get()) NRS start fifo request from 12345-0@lo, seq: 6974 00000100:00100000:2.0:1776178650.981820:0:8250:0:(service.c:2324:ptlrpc_server_handle_request()) Handling RPC req@00000000eb400454 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_003:0+-99:3352:x1862456998110976:12345-0@lo:38:kworker/u8:1.0 00010000:02000000:2.0:1776178650.981826:0:8250:0:(ldlm_lib.c:1139:target_handle_connect()) lustre-MDT0000: Not available for connect from 0@lo (stopping) 00000100:00100000:2.0:1776178650.981846:0:8250:0:(service.c:2372:ptlrpc_server_handle_request()) Handled RPC req@00000000eb400454 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_003:0+-99:3352:x1862456998110976:12345-0@lo:38:kworker/u8:1.0 Request processed in 25us (224us total) trans 0 rc -19/-19 00000100:00100000:2.0:1776178650.981851:0:8250:0:(nrs_fifo.c:241:nrs_fifo_req_stop()) NRS stop fifo request from 12345-0@lo, seq: 6974 00000100:00100000:2.0:1776178650.981862:0:8250:0:(nrs_fifo.c:179:nrs_fifo_req_get()) NRS start fifo request from 12345-0@lo, seq: 6975 00000100:00100000:2.0:1776178650.981864:0:8250:0:(service.c:2324:ptlrpc_server_handle_request()) Handling RPC req@000000004420855e pname:cluuid+ref:pid:xid:nid:opc:job mdt00_003:0+-99:3352:x1862456998111296:12345-0@lo:38:kworker/u8:1.0 00010000:02000000:2.0:1776178650.981870:0:8250:0:(ldlm_lib.c:1139:target_handle_connect()) lustre-MDT0000: Not available for connect from 0@lo (stopping) 00000100:00100000:2.0:1776178650.981889:0:8250:0:(service.c:2372:ptlrpc_server_handle_request()) Handled RPC req@000000004420855e pname:cluuid+ref:pid:xid:nid:opc:job mdt00_003:0+-99:3352:x1862456998111296:12345-0@lo:38:kworker/u8:1.0 Request processed in 23us (188us total) trans 0 rc -19/-19 00000100:00100000:2.0:1776178650.981894:0:8250:0:(nrs_fifo.c:241:nrs_fifo_req_stop()) NRS stop fifo request from 12345-0@lo, seq: 6975 00000100:00100000:2.0:1776178650.981912:0:15213:0:(service.c:2152:ptlrpc_server_handle_req_in()) got req x1862456998111104 00000100:00100000:2.0:1776178650.981932:0:15213:0:(nrs_fifo.c:179:nrs_fifo_req_get()) NRS start fifo request from 12345-0@lo, seq: 795 00000100:00100000:2.0:1776178650.981935:0:15213:0:(service.c:2324:ptlrpc_server_handle_request()) Handling RPC req@0000000024989815 pname:cluuid+ref:pid:xid:nid:opc:job mdt_out00_003:0+-99:3352:x1862456998111104:12345-0@lo:38:kworker/u8:1.0 00010000:02000000:2.0:1776178650.981941:0:15213:0:(ldlm_lib.c:1139:target_handle_connect()) lustre-MDT0000: Not available for connect from 0@lo (stopping) 00000100:00100000:2.0:1776178650.981962:0:15213:0:(service.c:2372:ptlrpc_server_handle_request()) Handled RPC req@0000000024989815 pname:cluuid+ref:pid:xid:nid:opc:job mdt_out00_003:0+-99:3352:x1862456998111104:12345-0@lo:38:kworker/u8:1.0 Request processed in 26us (294us total) trans 0 rc -19/-19 00000100:00100000:2.0:1776178650.981967:0:15213:0:(nrs_fifo.c:241:nrs_fifo_req_stop()) NRS stop fifo request from 12345-0@lo, seq: 795 00000100:00080000:2.0:1776178650.981995:0:3352:0:(import.c:86:import_set_state_nolock()) 0000000047601a98 lustre-MDT0000_UUID: changing import state from CONNECTING to DISCONN 00000100:00080000:2.0:1776178650.981999:0:3352:0:(import.c:1429:ptlrpc_connect_interpret()) recovery of lustre-MDT0000_UUID on 192.168.204.103@tcp failed (-19) 00000100:00100000:2.0:1776178650.982002:0:3352:0:(client.c:2215:ptlrpc_check_set()) Completed RPC req@00000000722afaa5 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:lustre-MDT0000-lwp-OST0000_UUID:3352:1862456998110848:0@lo:38:kworker/u8:1.0 00000100:00080000:2.0:1776178650.982009:0:3352:0:(import.c:86:import_set_state_nolock()) 00000000dfc8db13 lustre-MDT0000_UUID: changing import state from CONNECTING to DISCONN 00000100:00080000:2.0:1776178650.982012:0:3352:0:(import.c:1429:ptlrpc_connect_interpret()) recovery of lustre-MDT0000_UUID on 192.168.204.103@tcp failed (-19) 00000100:00100000:2.0:1776178650.982014:0:3352:0:(client.c:2215:ptlrpc_check_set()) Completed RPC req@00000000598050cc pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:lustre-MDT0000-lwp-OST0001_UUID:3352:1862456998110976:0@lo:38:kworker/u8:1.0 00000100:00080000:2.0:1776178650.982020:0:3352:0:(import.c:86:import_set_state_nolock()) 00000000ba4f1821 lustre-MDT0000_UUID: changing import state from CONNECTING to DISCONN 00000100:00080000:2.0:1776178650.982023:0:3352:0:(import.c:1429:ptlrpc_connect_interpret()) recovery of lustre-MDT0000_UUID on 192.168.204.103@tcp failed (-19) 00000100:00100000:2.0:1776178650.982025:0:3352:0:(client.c:2215:ptlrpc_check_set()) Completed RPC req@0000000070829000 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:lustre-MDT0001-mdtlov_UUID:3352:1862456998111104:0@lo:38:kworker/u8:1.0 00000100:00080000:2.0:1776178650.982031:0:3352:0:(import.c:86:import_set_state_nolock()) 00000000a7c55b1f lustre-MDT0000_UUID: changing import state from CONNECTING to DISCONN 00000100:00080000:2.0:1776178650.982034:0:3352:0:(import.c:1429:ptlrpc_connect_interpret()) recovery of lustre-MDT0000_UUID on 192.168.204.103@tcp failed (-19) 00000100:00100000:2.0:1776178650.982036:0:3352:0:(client.c:2215:ptlrpc_check_set()) Completed RPC req@00000000715d380c pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:lustre-MDT0000-lwp-MDT0001_UUID:3352:1862456998111296:0@lo:38:kworker/u8:1.0 00000100:00100000:1.0:1776178650.982299:0:20137:0:(service.c:2152:ptlrpc_server_handle_req_in()) got req x1862456998110784 00000100:00100000:1.0:1776178650.982335:0:20137:0:(nrs_fifo.c:179:nrs_fifo_req_get()) NRS start fifo request from 12345-0@lo, seq: 330 00000100:00100000:1.0:1776178650.982340:0:20137:0:(service.c:2324:ptlrpc_server_handle_request()) Handling RPC req@00000000f0ff59c2 pname:cluuid+ref:pid:xid:nid:opc:job ll_mgs_0000:ac61f839-0e8e-4cac-9c59-08bcfbbad330+18:3354:x1862456998110784:12345-0@lo:400:kworker/u8:1.0 00000100:00100000:1.0:1776178650.982380:0:20137:0:(service.c:2372:ptlrpc_server_handle_request()) Handled RPC req@00000000f0ff59c2 pname:cluuid+ref:pid:xid:nid:opc:job ll_mgs_0000:ac61f839-0e8e-4cac-9c59-08bcfbbad330+18:3354:x1862456998110784:12345-0@lo:400:kworker/u8:1.0 Request processed in 39us (1335us total) trans 0 rc 0/0 00000100:00100000:1.0:1776178650.982386:0:20137:0:(nrs_fifo.c:241:nrs_fifo_req_stop()) NRS stop fifo request from 12345-0@lo, seq: 330 00000100:00100000:1.0:1776178650.982426:0:3354:0:(client.c:2215:ptlrpc_check_set()) Completed RPC req@0000000098682d9e pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_01:ac61f839-0e8e-4cac-9c59-08bcfbbad330:3354:1862456998110784:0@lo:400:kworker/u8:1.0 00000100:00100000:0.0:1776178650.982516:0:3356:0:(client.c:2215:ptlrpc_check_set()) Completed RPC req@0000000037b0061a pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_03:lustre-MDT0001-lwp-OST0000_UUID:3356:1862456998110912:0@lo:400:kworker/u8:1.0 00000100:00100000:0.0:1776178650.982539:0:3356:0:(client.c:2215:ptlrpc_check_set()) Completed RPC req@0000000082764007 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_03:lustre-MDT0001-lwp-OST0001_UUID:3356:1862456998111040:0@lo:400:kworker/u8:1.0 00010000:00010000:2.0:1776178651.617331:0:6244:0:(ldlm_lockd.c:2792:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:1.0:1776178652.512430:0:3356:0:(ptlrpcd.c:415:ptlrpcd_check()) transfer 1 async RPCs [2->3] 00000100:00100000:1.0:1776178652.512453:0:3356:0:(client.c:1733:ptlrpc_send_new_req()) Sending RPC req@00000000be147540 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_03:lustre-MDT0001-mdtlov_UUID:3356:1862456998111360:0@lo:13:osp-pre-1-1.0 00000100:00100000:1.0:1776178652.512526:0:3356:0:(events.c:361:request_in_callback()) peer: 12345-0@lo (source: 12345-0@lo) 00000100:00100000:2.0:1776178652.512665:0:9554:0:(service.c:2152:ptlrpc_server_handle_req_in()) got req x1862456998111360 00000100:00100000:2.0:1776178652.512715:0:9554:0:(nrs_fifo.c:179:nrs_fifo_req_get()) NRS start fifo request from 12345-0@lo, seq: 663 00000100:00100000:2.0:1776178652.512721:0:9554:0:(service.c:2324:ptlrpc_server_handle_request()) Handling RPC req@0000000081d550e6 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_create00:lustre-MDT0001-mdtlov_UUID+5:3356:x1862456998111360:12345-0@lo:13:osp-pre-1-1.0 00002000:00000024:2.0:1776178652.512749:0:9554:0:(ofd_obd.c:740:ofd_statfs()) blocks cached 229376 granted 491746368 pending 0 free 3923689472 avail 3692167168 00000100:00100000:2.0:1776178652.513268:0:9554:0:(service.c:2372:ptlrpc_server_handle_request()) Handled RPC req@0000000081d550e6 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_create00:lustre-MDT0001-mdtlov_UUID+5:3356:x1862456998111360:12345-0@lo:13:osp-pre-1-1.0 Request processed in 545us (742us total) trans 0 rc 0/0 00000100:00100000:2.0:1776178652.513277:0:9554:0:(nrs_fifo.c:241:nrs_fifo_req_stop()) NRS stop fifo request from 12345-0@lo, seq: 663 00000100:00100000:0.0:1776178652.517424:0:3356:0:(client.c:2215:ptlrpc_check_set()) Completed RPC req@00000000be147540 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_03:lustre-MDT0001-mdtlov_UUID:3356:1862456998111360:0@lo:13:osp-pre-1-1.0 00010000:00010000:1.0:1776178652.641243:0:6245:0:(ldlm_lockd.c:2792:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:1.0:1776178653.024364:0:3355:0:(ptlrpcd.c:415:ptlrpcd_check()) transfer 1 async RPCs [3->2] 00000100:00100000:1.0:1776178653.024387:0:3355:0:(client.c:1733:ptlrpc_send_new_req()) Sending RPC req@0000000007d49440 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_02:lustre-MDT0001-mdtlov_UUID:3355:1862456998111424:0@lo:13:osp-pre-0-1.0 00000100:00100000:1.0:1776178653.024457:0:3355:0:(events.c:361:request_in_callback()) peer: 12345-0@lo (source: 12345-0@lo) 00000100:00100000:2.0:1776178653.024657:0:9554:0:(service.c:2152:ptlrpc_server_handle_req_in()) got req x1862456998111424 00000100:00100000:2.0:1776178653.024748:0:9554:0:(nrs_fifo.c:179:nrs_fifo_req_get()) NRS start fifo request from 12345-0@lo, seq: 664 00000100:00100000:2.0:1776178653.024755:0:9554:0:(service.c:2324:ptlrpc_server_handle_request()) Handling RPC req@00000000d4b41d38 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_create00:lustre-MDT0001-mdtlov_UUID+5:3355:x1862456998111424:12345-0@lo:13:osp-pre-0-1.0 00002000:00000024:2.0:1776178653.024784:0:9554:0:(ofd_obd.c:740:ofd_statfs()) blocks cached 315392 granted 484883776 pending 0 free 3923689472 avail 3692167168 00000100:00100000:2.0:1776178653.024841:0:9554:0:(service.c:2372:ptlrpc_server_handle_request()) Handled RPC req@00000000d4b41d38 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_create00:lustre-MDT0001-mdtlov_UUID+5:3355:x1862456998111424:12345-0@lo:13:osp-pre-0-1.0 Request processed in 86us (385us total) trans 0 rc 0/0 00000100:00100000:2.0:1776178653.024849:0:9554:0:(nrs_fifo.c:241:nrs_fifo_req_stop()) NRS stop fifo request from 12345-0@lo, seq: 664 00000100:00100000:0.0:1776178653.024933:0:3355:0:(client.c:2215:ptlrpc_check_set()) Completed RPC req@0000000007d49440 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_02:lustre-MDT0001-mdtlov_UUID:3355:1862456998111424:0@lo:13:osp-pre-0-1.0 00010000:00010000:2.0:1776178653.664381:0:9224:0:(ldlm_lockd.c:2792:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:0.0:1776178654.502043:0:3347:0:(events.c:361:request_in_callback()) peer: 12345-192.168.204.3@tcp (source: 12345-192.168.204.3@tcp) 00000100:00100000:0.0:1776178654.502132:0:3347:0:(events.c:361:request_in_callback()) peer: 12345-192.168.204.3@tcp (source: 12345-192.168.204.3@tcp) 00000100:00100000:0.0:1776178654.502173:0:3347:0:(events.c:361:request_in_callback()) peer: 12345-192.168.204.3@tcp (source: 12345-192.168.204.3@tcp) 00000100:00100000:0.0:1776178654.502203:0:3347:0:(events.c:361:request_in_callback()) peer: 12345-192.168.204.3@tcp (source: 12345-192.168.204.3@tcp) 00000100:00100000:0.0:1776178654.502274:0:3347:0:(events.c:361:request_in_callback()) peer: 12345-192.168.204.3@tcp (source: 12345-192.168.204.3@tcp) 00000100:00100000:0.0:1776178654.502310:0:6257:0:(service.c:2152:ptlrpc_server_handle_req_in()) got req x1862456979137216 00000100:00100000:0.0:1776178654.502347:0:6257:0:(service.c:2152:ptlrpc_server_handle_req_in()) got req x1862456979137280 00000100:00100000:0.0:1776178654.502371:0:6257:0:(nrs_fifo.c:179:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.204.3@tcp, seq: 1192 00000100:00100000:0.0:1776178654.502379:0:6257:0:(service.c:2324:ptlrpc_server_handle_request()) Handling RPC req@00000000f45ed991 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_002:b50963bd-8eba-4432-9a54-8888d00e3fa1+6:2273:x1862456979137280:12345-192.168.204.3@tcp:400:kworker/u8:4.0 00000100:00100000:0.0:1776178654.502459:0:6257:0:(service.c:2372:ptlrpc_server_handle_request()) Handled RPC req@00000000f45ed991 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_002:b50963bd-8eba-4432-9a54-8888d00e3fa1+6:2273:x1862456979137280:12345-192.168.204.3@tcp:400:kworker/u8:4.0 Request processed in 79us (256us total) trans 0 rc 0/0 00000100:00100000:0.0:1776178654.502467:0:6257:0:(nrs_fifo.c:241:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.204.3@tcp, seq: 1192 00000100:00100000:0.0:1776178654.502485:0:6257:0:(nrs_fifo.c:179:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.204.3@tcp, seq: 6976 00000100:00100000:0.0:1776178654.502489:0:6257:0:(service.c:2324:ptlrpc_server_handle_request()) Handling RPC req@00000000d81deb92 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_002:0+-99:2269:x1862456979137216:12345-192.168.204.3@tcp:38:kworker/u8:4.0 00010000:02000000:0.0:1776178654.502502:0:6257:0:(ldlm_lib.c:1139:target_handle_connect()) lustre-MDT0000: Not available for connect from 192.168.204.3@tcp (stopping) 00000100:00100000:0.0:1776178654.502526:0:6257:0:(service.c:2372:ptlrpc_server_handle_request()) Handled RPC req@00000000d81deb92 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_002:0+-99:2269:x1862456979137216:12345-192.168.204.3@tcp:38:kworker/u8:4.0 Request processed in 36us (353us total) trans 0 rc -19/-19 00000100:00100000:0.0:1776178654.502532:0:6257:0:(nrs_fifo.c:241:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.204.3@tcp, seq: 6976 00000100:00100000:2.0:1776178654.506646:0:20140:0:(service.c:2152:ptlrpc_server_handle_req_in()) got req x1862456979137152 00000100:00100000:2.0:1776178654.506921:0:20140:0:(nrs_fifo.c:179:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.204.3@tcp, seq: 331 00000100:00100000:2.0:1776178654.506927:0:20140:0:(service.c:2324:ptlrpc_server_handle_request()) Handling RPC req@00000000ba1c7e2e pname:cluuid+ref:pid:xid:nid:opc:job ll_mgs_0002:5b5c66b1-1a2f-4651-8548-0c75b9a38a94+9:2272:x1862456979137152:12345-192.168.204.3@tcp:400:kworker/u8:4.0 00000100:00100000:2.0:1776178654.507036:0:20140:0:(service.c:2372:ptlrpc_server_handle_request()) Handled RPC req@00000000ba1c7e2e pname:cluuid+ref:pid:xid:nid:opc:job ll_mgs_0002:5b5c66b1-1a2f-4651-8548-0c75b9a38a94+9:2272:x1862456979137152:12345-192.168.204.3@tcp:400:kworker/u8:4.0 Request processed in 108us (5002us total) trans 0 rc 0/0 00000100:00100000:2.0:1776178654.507048:0:20140:0:(nrs_fifo.c:241:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.204.3@tcp, seq: 331 00000100:00100000:1.0:1776178654.507426:0:20762:0:(service.c:2152:ptlrpc_server_handle_req_in()) got req x1862456979137344 00000100:00100000:1.0:1776178654.507468:0:20762:0:(service.c:2152:ptlrpc_server_handle_req_in()) got req x1862456979137408 00000100:00100000:1.0:1776178654.507491:0:20762:0:(nrs_fifo.c:179:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.204.3@tcp, seq: 863 00000100:00100000:1.0:1776178654.507495:0:20762:0:(service.c:2324:ptlrpc_server_handle_request()) Handling RPC req@00000000196ec025 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_013:b50963bd-8eba-4432-9a54-8888d00e3fa1+8:2272:x1862456979137344:12345-192.168.204.3@tcp:400:kworker/u8:4.0 00000100:00100000:1.0:1776178654.507529:0:20762:0:(service.c:2372:ptlrpc_server_handle_request()) Handled RPC req@00000000196ec025 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_013:b50963bd-8eba-4432-9a54-8888d00e3fa1+8:2272:x1862456979137344:12345-192.168.204.3@tcp:400:kworker/u8:4.0 Request processed in 32us (5397us total) trans 0 rc 0/0 00000100:00100000:1.0:1776178654.507536:0:20762:0:(nrs_fifo.c:241:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.204.3@tcp, seq: 863 00000100:00100000:1.0:1776178654.507550:0:20762:0:(nrs_fifo.c:179:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.204.3@tcp, seq: 864 00000100:00100000:1.0:1776178654.507570:0:20762:0:(service.c:2324:ptlrpc_server_handle_request()) Handling RPC req@000000009a62f69c pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_013:b50963bd-8eba-4432-9a54-8888d00e3fa1+9:2273:x1862456979137408:12345-192.168.204.3@tcp:400:kworker/u8:4.0 00000100:00100000:1.0:1776178654.507593:0:20762:0:(service.c:2372:ptlrpc_server_handle_request()) Handled RPC req@000000009a62f69c pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_013:b50963bd-8eba-4432-9a54-8888d00e3fa1+9:2273:x1862456979137408:12345-192.168.204.3@tcp:400:kworker/u8:4.0 Request processed in 39us (5319us total) trans 0 rc 0/0 00000100:00100000:1.0:1776178654.507598:0:20762:0:(nrs_fifo.c:241:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.204.3@tcp, seq: 864 00010000:00010000:1.0:1776178654.688354:0:9225:0:(ldlm_lockd.c:2792:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00010000:00010000:2.0:1776178655.713020:0:6244:0:(ldlm_lockd.c:2792:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00080000:0.0:1776178656.096281:0:36:0:(pinger.c:251:ptlrpc_pinger_process_import()) lustre-MDT0000-lwp-OST0000_UUID->lustre-MDT0000_UUID: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00080000:0.0:1776178656.096291:0:36:0:(recover.c:57:ptlrpc_initiate_recovery()) lustre-MDT0000_UUID: starting recovery 00000100:00080000:0.0:1776178656.096295:0:36:0:(import.c:86:import_set_state_nolock()) 0000000047601a98 lustre-MDT0000_UUID: changing import state from DISCONN to CONNECTING 00000100:00080000:0.0:1776178656.096304:0:36:0:(import.c:534:import_select_connection()) lustre-MDT0000-lwp-OST0000: connect to NID 0@lo last attempt 1358 00000100:00080000:0.0:1776178656.096308:0:36:0:(import.c:581:import_select_connection()) lustre-MDT0000-lwp-OST0000: tried all connections, increasing latency to 21s 00000100:00080000:0.0:1776178656.096315:0:36:0:(import.c:615:import_select_connection()) lustre-MDT0000-lwp-OST0000: import 0000000047601a98 using connection 192.168.204.103@tcp/0@lo 00000100:00100000:0.0:1776178656.096342:0:36:0:(import.c:807:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5) req@00000000034781c2 x1862456998111552/t0(0) o38->lustre-MDT0000-lwp-OST0000@0@lo:12/10 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' 00000100:00080000:0.0:1776178656.096376:0:36:0:(pinger.c:251:ptlrpc_pinger_process_import()) lustre-MDT0000-lwp-OST0001_UUID->lustre-MDT0000_UUID: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00080000:0.0:1776178656.096379:0:36:0:(recover.c:57:ptlrpc_initiate_recovery()) lustre-MDT0000_UUID: starting recovery 00000100:00080000:0.0:1776178656.096381:0:36:0:(import.c:86:import_set_state_nolock()) 00000000dfc8db13 lustre-MDT0000_UUID: changing import state from DISCONN to CONNECTING 00000100:00080000:0.0:1776178656.096384:0:36:0:(import.c:534:import_select_connection()) lustre-MDT0000-lwp-OST0001: connect to NID 0@lo last attempt 1358 00000100:00080000:0.0:1776178656.096386:0:36:0:(import.c:581:import_select_connection()) lustre-MDT0000-lwp-OST0001: tried all connections, increasing latency to 21s 00000100:00080000:0.0:1776178656.096389:0:36:0:(import.c:615:import_select_connection()) lustre-MDT0000-lwp-OST0001: import 00000000dfc8db13 using connection 192.168.204.103@tcp/0@lo 00000100:00100000:0.0:1776178656.096399:0:36:0:(import.c:807:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5) req@00000000c37c187e x1862456998111680/t0(0) o38->lustre-MDT0000-lwp-OST0001@0@lo:12/10 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' 00000100:00100000:3.0:1776178656.096424:0:3354:0:(client.c:1733:ptlrpc_send_new_req()) Sending RPC req@000000000bdf54a0 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_01:lustre-MDT0001-lwp-OST0000_UUID:3354:1862456998111616:0@lo:400:kworker/u8:1.0 00000100:00080000:0.0:1776178656.096427:0:36:0:(pinger.c:251:ptlrpc_pinger_process_import()) lustre-MDT0001-mdtlov_UUID->lustre-MDT0000_UUID: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00080000:0.0:1776178656.096430:0:36:0:(recover.c:57:ptlrpc_initiate_recovery()) lustre-MDT0000_UUID: starting recovery 00000100:00080000:0.0:1776178656.096431:0:36:0:(import.c:86:import_set_state_nolock()) 00000000ba4f1821 lustre-MDT0000_UUID: changing import state from DISCONN to CONNECTING 00000100:00080000:0.0:1776178656.096434:0:36:0:(import.c:534:import_select_connection()) lustre-MDT0000-osp-MDT0001: connect to NID 0@lo last attempt 1358 00000100:00080000:0.0:1776178656.096436:0:36:0:(import.c:581:import_select_connection()) lustre-MDT0000-osp-MDT0001: tried all connections, increasing latency to 21s 00000100:00080000:0.0:1776178656.096438:0:36:0:(import.c:615:import_select_connection()) lustre-MDT0000-osp-MDT0001: import 00000000ba4f1821 using connection 192.168.204.103@tcp/0@lo 00000100:00100000:0.0:1776178656.096447:0:36:0:(import.c:807:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5) req@0000000038b19922 x1862456998111808/t0(0) o38->lustre-MDT0000-osp-MDT0001@0@lo:24/4 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' 00000100:00080000:0.0:1776178656.096462:0:36:0:(pinger.c:251:ptlrpc_pinger_process_import()) lustre-MDT0000-lwp-MDT0001_UUID->lustre-MDT0000_UUID: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00080000:0.0:1776178656.096465:0:36:0:(recover.c:57:ptlrpc_initiate_recovery()) lustre-MDT0000_UUID: starting recovery 00000100:00080000:0.0:1776178656.096465:0:36:0:(import.c:86:import_set_state_nolock()) 00000000a7c55b1f lustre-MDT0000_UUID: changing import state from DISCONN to CONNECTING 00000100:00080000:0.0:1776178656.096468:0:36:0:(import.c:534:import_select_connection()) lustre-MDT0000-lwp-MDT0001: connect to NID 0@lo last attempt 1358 00000100:00080000:0.0:1776178656.096469:0:36:0:(import.c:581:import_select_connection()) lustre-MDT0000-lwp-MDT0001: tried all connections, increasing latency to 21s 00000100:00080000:0.0:1776178656.096472:0:36:0:(import.c:615:import_select_connection()) lustre-MDT0000-lwp-MDT0001: import 00000000a7c55b1f using connection 192.168.204.103@tcp/0@lo 00000100:00100000:0.0:1776178656.096479:0:36:0:(import.c:807:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5) req@000000000d203d1d x1862456998112000/t0(0) o38->lustre-MDT0000-lwp-MDT0001@0@lo:12/10 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' 00000100:00100000:3.0:1776178656.096488:0:3354:0:(events.c:361:request_in_callback()) peer: 12345-0@lo (source: 12345-0@lo) 00000100:00100000:0.0:1776178656.096500:0:3355:0:(client.c:1733:ptlrpc_send_new_req()) Sending RPC req@00000000ac4116de pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_02:lustre-MDT0001-lwp-OST0001_UUID:3355:1862456998111744:0@lo:400:kworker/u8:1.0 00000100:00100000:3.0:1776178656.096520:0:3354:0:(ptlrpcd.c:415:ptlrpcd_check()) transfer 2 async RPCs [0->1] 00000100:00100000:0.0:1776178656.096524:0:3355:0:(events.c:361:request_in_callback()) peer: 12345-0@lo (source: 12345-0@lo) 00000100:00100000:3.0:1776178656.096525:0:3354:0:(client.c:1733:ptlrpc_send_new_req()) Sending RPC req@00000000c237605d pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_01:ac61f839-0e8e-4cac-9c59-08bcfbbad330:3354:1862456998111488:0@lo:400:kworker/u8:1.0 00000100:00100000:0.0:1776178656.096541:0:3355:0:(ptlrpcd.c:415:ptlrpcd_check()) transfer 1 async RPCs [3->2] 00000100:00100000:0.0:1776178656.096545:0:3355:0:(client.c:1733:ptlrpc_send_new_req()) Sending RPC req@00000000ffd05959 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_02:lustre-MDT0001-mdtlov_UUID:3355:1862456998111872:0@lo:400:kworker/u8:1.0 00000100:00100000:2.0:1776178656.096618:0:6255:0:(service.c:2152:ptlrpc_server_handle_req_in()) got req x1862456998111616 00000100:00100000:1.0:1776178656.096624:0:3352:0:(client.c:1733:ptlrpc_send_new_req()) Sending RPC req@00000000034781c2 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:lustre-MDT0000-lwp-OST0000_UUID:3352:1862456998111552:0@lo:38:kworker/u8:1.0 00000100:00100000:2.0:1776178656.096674:0:6255:0:(service.c:2152:ptlrpc_server_handle_req_in()) got req x1862456998111744 00000100:00100000:2.0:1776178656.096693:0:6255:0:(nrs_fifo.c:179:nrs_fifo_req_get()) NRS start fifo request from 12345-0@lo, seq: 1193 00000100:00100000:2.0:1776178656.096698:0:6255:0:(service.c:2324:ptlrpc_server_handle_request()) Handling RPC req@000000006d8fce24 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_000:lustre-MDT0001-lwp-OST0000_UUID+5:3354:x1862456998111616:12345-0@lo:400:kworker/u8:1.0 00000100:00100000:0.0:1776178656.098433:0:3355:0:(events.c:361:request_in_callback()) peer: 12345-0@lo (source: 12345-0@lo) 00000100:00100000:0.0:1776178656.098510:0:20759:0:(service.c:2152:ptlrpc_server_handle_req_in()) got req x1862456998111872 00000100:00100000:0.0:1776178656.098566:0:20759:0:(nrs_fifo.c:179:nrs_fifo_req_get()) NRS start fifo request from 12345-0@lo, seq: 865 00000100:00100000:0.0:1776178656.098573:0:20759:0:(service.c:2324:ptlrpc_server_handle_request()) Handling RPC req@0000000007e413b7 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_010:lustre-MDT0001-mdtlov_UUID+5:3355:x1862456998111872:12345-0@lo:400:kworker/u8:1.0 00000100:00100000:0.0:1776178656.098632:0:20759:0:(service.c:2372:ptlrpc_server_handle_request()) Handled RPC req@0000000007e413b7 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_010:lustre-MDT0001-mdtlov_UUID+5:3355:x1862456998111872:12345-0@lo:400:kworker/u8:1.0 Request processed in 59us (205us total) trans 0 rc 0/0 00000100:00100000:0.0:1776178656.098640:0:20759:0:(nrs_fifo.c:241:nrs_fifo_req_stop()) NRS stop fifo request from 12345-0@lo, seq: 865 00000100:00100000:0.0:1776178656.098688:0:3355:0:(client.c:2215:ptlrpc_check_set()) Completed RPC req@00000000ffd05959 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_02:lustre-MDT0001-mdtlov_UUID:3355:1862456998111872:0@lo:400:kworker/u8:1.0 00000100:00100000:0.0:1776178656.098753:0:6257:0:(nrs_fifo.c:179:nrs_fifo_req_get()) NRS start fifo request from 12345-0@lo, seq: 1194 00000100:00100000:0.0:1776178656.098758:0:6257:0:(service.c:2324:ptlrpc_server_handle_request()) Handling RPC req@000000008b978a8a pname:cluuid+ref:pid:xid:nid:opc:job mdt00_002:lustre-MDT0001-lwp-OST0001_UUID+5:3355:x1862456998111744:12345-0@lo:400:kworker/u8:1.0 00000100:00100000:0.0:1776178656.098797:0:6257:0:(service.c:2372:ptlrpc_server_handle_request()) Handled RPC req@000000008b978a8a pname:cluuid+ref:pid:xid:nid:opc:job mdt00_002:lustre-MDT0001-lwp-OST0001_UUID+5:3355:x1862456998111744:12345-0@lo:400:kworker/u8:1.0 Request processed in 39us (2272us total) trans 0 rc 0/0 00000100:00100000:0.0:1776178656.098803:0:6257:0:(nrs_fifo.c:241:nrs_fifo_req_stop()) NRS stop fifo request from 12345-0@lo, seq: 1194 00000100:00100000:0.0:1776178656.098837:0:3355:0:(client.c:2215:ptlrpc_check_set()) Completed RPC req@00000000ac4116de pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_02:lustre-MDT0001-lwp-OST0001_UUID:3355:1862456998111744:0@lo:400:kworker/u8:1.0 00000100:00100000:3.0:1776178656.100767:0:3354:0:(events.c:361:request_in_callback()) peer: 12345-0@lo (source: 12345-0@lo) 00000100:00100000:3.0:1776178656.100809:0:3354:0:(client.c:1733:ptlrpc_send_new_req()) Sending RPC req@00000000e82039bb pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_01:lustre-MDT0001-mdtlov_UUID:3354:1862456998111936:0@lo:400:kworker/u8:1.0 00000100:00100000:3.0:1776178656.100962:0:3354:0:(events.c:361:request_in_callback()) peer: 12345-0@lo (source: 12345-0@lo) 00000100:00100000:3.0:1776178656.101011:0:20755:0:(service.c:2152:ptlrpc_server_handle_req_in()) got req x1862456998111936 00000100:00100000:3.0:1776178656.101050:0:20755:0:(nrs_fifo.c:179:nrs_fifo_req_get()) NRS start fifo request from 12345-0@lo, seq: 866 00000100:00100000:3.0:1776178656.101056:0:20755:0:(service.c:2324:ptlrpc_server_handle_request()) Handling RPC req@0000000045cb5035 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_006:lustre-MDT0001-mdtlov_UUID+5:3354:x1862456998111936:12345-0@lo:400:kworker/u8:1.0 00000100:00100000:3.0:1776178656.101136:0:20755:0:(service.c:2372:ptlrpc_server_handle_request()) Handled RPC req@0000000045cb5035 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_006:lustre-MDT0001-mdtlov_UUID+5:3354:x1862456998111936:12345-0@lo:400:kworker/u8:1.0 Request processed in 79us (174us total) trans 0 rc 0/0 00000100:00100000:3.0:1776178656.101145:0:20755:0:(nrs_fifo.c:241:nrs_fifo_req_stop()) NRS stop fifo request from 12345-0@lo, seq: 866 00000100:00100000:3.0:1776178656.101192:0:3354:0:(client.c:2215:ptlrpc_check_set()) Completed RPC req@00000000e82039bb pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_01:lustre-MDT0001-mdtlov_UUID:3354:1862456998111936:0@lo:400:kworker/u8:1.0 00000100:00100000:0.0:1776178656.101283:0:20137:0:(service.c:2152:ptlrpc_server_handle_req_in()) got req x1862456998111488 00000100:00100000:0.0:1776178656.101322:0:20137:0:(nrs_fifo.c:179:nrs_fifo_req_get()) NRS start fifo request from 12345-0@lo, seq: 332 00000100:00100000:0.0:1776178656.101325:0:20137:0:(service.c:2324:ptlrpc_server_handle_request()) Handling RPC req@000000005db97027 pname:cluuid+ref:pid:xid:nid:opc:job ll_mgs_0000:ac61f839-0e8e-4cac-9c59-08bcfbbad330+18:3354:x1862456998111488:12345-0@lo:400:kworker/u8:1.0 00000100:00100000:0.0:1776178656.101364:0:20137:0:(service.c:2372:ptlrpc_server_handle_request()) Handled RPC req@000000005db97027 pname:cluuid+ref:pid:xid:nid:opc:job ll_mgs_0000:ac61f839-0e8e-4cac-9c59-08bcfbbad330+18:3354:x1862456998111488:12345-0@lo:400:kworker/u8:1.0 Request processed in 38us (607us total) trans 0 rc 0/0 00000100:00100000:0.0:1776178656.101370:0:20137:0:(nrs_fifo.c:241:nrs_fifo_req_stop()) NRS stop fifo request from 12345-0@lo, seq: 332 00000100:00100000:3.0:1776178656.101449:0:3354:0:(client.c:2215:ptlrpc_check_set()) Completed RPC req@00000000c237605d pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_01:ac61f839-0e8e-4cac-9c59-08bcfbbad330:3354:1862456998111488:0@lo:400:kworker/u8:1.0 00000100:00100000:1.0:1776178656.109300:0:3352:0:(events.c:361:request_in_callback()) peer: 12345-0@lo (source: 12345-0@lo) 00000100:00100000:1.0:1776178656.109357:0:3352:0:(client.c:1733:ptlrpc_send_new_req()) Sending RPC req@00000000c37c187e pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:lustre-MDT0000-lwp-OST0001_UUID:3352:1862456998111680:0@lo:38:kworker/u8:1.0 00000100:00100000:1.0:1776178656.109392:0:3352:0:(events.c:361:request_in_callback()) peer: 12345-0@lo (source: 12345-0@lo) 00000100:00100000:1.0:1776178656.109421:0:3352:0:(client.c:1733:ptlrpc_send_new_req()) Sending RPC req@0000000038b19922 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:lustre-MDT0001-mdtlov_UUID:3352:1862456998111808:0@lo:38:kworker/u8:1.0 00000100:00100000:1.0:1776178656.109445:0:3352:0:(events.c:361:request_in_callback()) peer: 12345-0@lo (source: 12345-0@lo) 00000100:00100000:1.0:1776178656.109465:0:3352:0:(client.c:1733:ptlrpc_send_new_req()) Sending RPC req@000000000d203d1d pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:lustre-MDT0000-lwp-MDT0001_UUID:3352:1862456998112000:0@lo:38:kworker/u8:1.0 00000100:00100000:1.0:1776178656.109486:0:3352:0:(events.c:361:request_in_callback()) peer: 12345-0@lo (source: 12345-0@lo) 00000100:00100000:1.0:1776178656.109537:0:14128:0:(service.c:2152:ptlrpc_server_handle_req_in()) got req x1862456998111808 00000100:00100000:1.0:1776178656.109575:0:14128:0:(nrs_fifo.c:179:nrs_fifo_req_get()) NRS start fifo request from 12345-0@lo, seq: 796 00000100:00100000:1.0:1776178656.109582:0:14128:0:(service.c:2324:ptlrpc_server_handle_request()) Handling RPC req@00000000f6af7e5b pname:cluuid+ref:pid:xid:nid:opc:job mdt_out00_002:0+-99:3352:x1862456998111808:12345-0@lo:38:kworker/u8:1.0 00010000:02000000:1.0:1776178656.109601:0:14128:0:(ldlm_lib.c:1139:target_handle_connect()) lustre-MDT0000: Not available for connect from 0@lo (stopping) 00000100:00100000:1.0:1776178656.109646:0:14128:0:(service.c:2372:ptlrpc_server_handle_request()) Handled RPC req@00000000f6af7e5b pname:cluuid+ref:pid:xid:nid:opc:job mdt_out00_002:0+-99:3352:x1862456998111808:12345-0@lo:38:kworker/u8:1.0 Request processed in 63us (199us total) trans 0 rc -19/-19 00000100:00100000:1.0:1776178656.109652:0:14128:0:(nrs_fifo.c:241:nrs_fifo_req_stop()) NRS stop fifo request from 12345-0@lo, seq: 796 00000100:00080000:1.0:1776178656.109702:0:3352:0:(import.c:86:import_set_state_nolock()) 00000000ba4f1821 lustre-MDT0000_UUID: changing import state from CONNECTING to DISCONN 00000100:00080000:1.0:1776178656.109706:0:3352:0:(import.c:1429:ptlrpc_connect_interpret()) recovery of lustre-MDT0000_UUID on 192.168.204.103@tcp failed (-19) 00000100:00100000:1.0:1776178656.109711:0:3352:0:(client.c:2215:ptlrpc_check_set()) Completed RPC req@0000000038b19922 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:lustre-MDT0001-mdtlov_UUID:3352:1862456998111808:0@lo:38:kworker/u8:1.0 00000100:00100000:0.0:1776178656.109804:0:6256:0:(service.c:2152:ptlrpc_server_handle_req_in()) got req x1862456998111552 00000100:00100000:0.0:1776178656.109833:0:6256:0:(service.c:2152:ptlrpc_server_handle_req_in()) got req x1862456998111680 00000100:00100000:0.0:1776178656.109841:0:6256:0:(service.c:2152:ptlrpc_server_handle_req_in()) got req x1862456998112000 00000100:00100000:0.0:1776178656.109850:0:6256:0:(nrs_fifo.c:179:nrs_fifo_req_get()) NRS start fifo request from 12345-0@lo, seq: 6977 00000100:00100000:0.0:1776178656.109853:0:6256:0:(service.c:2324:ptlrpc_server_handle_request()) Handling RPC req@0000000015909b2d pname:cluuid+ref:pid:xid:nid:opc:job mdt00_001:0+-99:3352:x1862456998111552:12345-0@lo:38:kworker/u8:1.0 00010000:02000000:0.0:1776178656.109861:0:6256:0:(ldlm_lib.c:1139:target_handle_connect()) lustre-MDT0000: Not available for connect from 0@lo (stopping) 00000100:00100000:0.0:1776178656.109894:0:6256:0:(service.c:2372:ptlrpc_server_handle_request()) Handled RPC req@0000000015909b2d pname:cluuid+ref:pid:xid:nid:opc:job mdt00_001:0+-99:3352:x1862456998111552:12345-0@lo:38:kworker/u8:1.0 Request processed in 40us (603us total) trans 0 rc -19/-19 00000100:00100000:0.0:1776178656.109900:0:6256:0:(nrs_fifo.c:241:nrs_fifo_req_stop()) NRS stop fifo request from 12345-0@lo, seq: 6977 00000100:00100000:0.0:1776178656.109911:0:6256:0:(nrs_fifo.c:179:nrs_fifo_req_get()) NRS start fifo request from 12345-0@lo, seq: 6978 00000100:00100000:0.0:1776178656.109914:0:6256:0:(service.c:2324:ptlrpc_server_handle_request()) Handling RPC req@00000000a2f460e7 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_001:0+-99:3352:x1862456998111680:12345-0@lo:38:kworker/u8:1.0 00010000:02000000:0.0:1776178656.109921:0:6256:0:(ldlm_lib.c:1139:target_handle_connect()) lustre-MDT0000: Not available for connect from 0@lo (stopping) 00000100:00100000:0.0:1776178656.109942:0:6256:0:(service.c:2372:ptlrpc_server_handle_request()) Handled RPC req@00000000a2f460e7 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_001:0+-99:3352:x1862456998111680:12345-0@lo:38:kworker/u8:1.0 Request processed in 27us (551us total) trans 0 rc -19/-19 00000100:00100000:0.0:1776178656.109947:0:6256:0:(nrs_fifo.c:241:nrs_fifo_req_stop()) NRS stop fifo request from 12345-0@lo, seq: 6978 00000100:00100000:0.0:1776178656.109956:0:6256:0:(nrs_fifo.c:179:nrs_fifo_req_get()) NRS start fifo request from 12345-0@lo, seq: 6979 00000100:00100000:0.0:1776178656.109958:0:6256:0:(service.c:2324:ptlrpc_server_handle_request()) Handling RPC req@00000000124cea36 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_001:0+-99:3352:x1862456998112000:12345-0@lo:38:kworker/u8:1.0 00010000:02000000:0.0:1776178656.109963:0:6256:0:(ldlm_lib.c:1139:target_handle_connect()) lustre-MDT0000: Not available for connect from 0@lo (stopping) 00000100:00100000:0.0:1776178656.109981:0:6256:0:(service.c:2372:ptlrpc_server_handle_request()) Handled RPC req@00000000124cea36 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_001:0+-99:3352:x1862456998112000:12345-0@lo:38:kworker/u8:1.0 Request processed in 22us (495us total) trans 0 rc -19/-19 00000100:00100000:0.0:1776178656.109986:0:6256:0:(nrs_fifo.c:241:nrs_fifo_req_stop()) NRS stop fifo request from 12345-0@lo, seq: 6979 00000100:00080000:1.0:1776178656.110182:0:3352:0:(import.c:86:import_set_state_nolock()) 0000000047601a98 lustre-MDT0000_UUID: changing import state from CONNECTING to DISCONN 00000100:00080000:1.0:1776178656.110187:0:3352:0:(import.c:1429:ptlrpc_connect_interpret()) recovery of lustre-MDT0000_UUID on 192.168.204.103@tcp failed (-19) 00000100:00100000:1.0:1776178656.110298:0:3352:0:(client.c:2215:ptlrpc_check_set()) Completed RPC req@00000000034781c2 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:lustre-MDT0000-lwp-OST0000_UUID:3352:1862456998111552:0@lo:38:kworker/u8:1.0 00000100:00080000:1.0:1776178656.110312:0:3352:0:(import.c:86:import_set_state_nolock()) 00000000dfc8db13 lustre-MDT0000_UUID: changing import state from CONNECTING to DISCONN 00000100:00080000:1.0:1776178656.110315:0:3352:0:(import.c:1429:ptlrpc_connect_interpret()) recovery of lustre-MDT0000_UUID on 192.168.204.103@tcp failed (-19) 00000100:00100000:1.0:1776178656.110318:0:3352:0:(client.c:2215:ptlrpc_check_set()) Completed RPC req@00000000c37c187e pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:lustre-MDT0000-lwp-OST0001_UUID:3352:1862456998111680:0@lo:38:kworker/u8:1.0 00000100:00080000:1.0:1776178656.110325:0:3352:0:(import.c:86:import_set_state_nolock()) 00000000a7c55b1f lustre-MDT0000_UUID: changing import state from CONNECTING to DISCONN 00000100:00080000:1.0:1776178656.110328:0:3352:0:(import.c:1429:ptlrpc_connect_interpret()) recovery of lustre-MDT0000_UUID on 192.168.204.103@tcp failed (-19) 00000100:00100000:1.0:1776178656.110331:0:3352:0:(client.c:2215:ptlrpc_check_set()) Completed RPC req@000000000d203d1d pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:lustre-MDT0000-lwp-MDT0001_UUID:3352:1862456998112000:0@lo:38:kworker/u8:1.0 00000100:00100000:2.0:1776178656.112471:0:6255:0:(service.c:2372:ptlrpc_server_handle_request()) Handled RPC req@000000006d8fce24 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_000:lustre-MDT0001-lwp-OST0000_UUID+5:3354:x1862456998111616:12345-0@lo:400:kworker/u8:1.0 Request processed in 15764us (15975us total) trans 0 rc 0/0 00000100:00100000:2.0:1776178656.112483:0:6255:0:(nrs_fifo.c:241:nrs_fifo_req_stop()) NRS stop fifo request from 12345-0@lo, seq: 1193 00000100:00100000:3.0:1776178656.114889:0:3354:0:(client.c:2215:ptlrpc_check_set()) Completed RPC req@000000000bdf54a0 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_01:lustre-MDT0001-lwp-OST0000_UUID:3354:1862456998111616:0@lo:400:kworker/u8:1.0 00010000:00010000:1.0:1776178656.736241:0:6245:0:(ldlm_lockd.c:2792:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:1.0:1776178657.632834:0:3353:0:(ptlrpcd.c:415:ptlrpcd_check()) transfer 1 async RPCs [1->0] 00000100:00100000:1.0:1776178657.632858:0:3353:0:(client.c:1733:ptlrpc_send_new_req()) Sending RPC req@0000000025b3b8e0 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_00:lustre-MDT0001-mdtlov_UUID:3353:1862456998112064:0@lo:13:osp-pre-1-1.0 00000100:00100000:1.0:1776178657.632926:0:3353:0:(events.c:361:request_in_callback()) peer: 12345-0@lo (source: 12345-0@lo) 00000100:00100000:2.0:1776178657.633023:0:9554:0:(service.c:2152:ptlrpc_server_handle_req_in()) got req x1862456998112064 00000100:00100000:2.0:1776178657.633101:0:9554:0:(nrs_fifo.c:179:nrs_fifo_req_get()) NRS start fifo request from 12345-0@lo, seq: 665 00000100:00100000:2.0:1776178657.633107:0:9554:0:(service.c:2324:ptlrpc_server_handle_request()) Handling RPC req@00000000a9cfe89c pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_create00:lustre-MDT0001-mdtlov_UUID+5:3353:x1862456998112064:12345-0@lo:13:osp-pre-1-1.0 00002000:00000024:2.0:1776178657.633138:0:9554:0:(ofd_obd.c:740:ofd_statfs()) blocks cached 229376 granted 491746368 pending 0 free 3923689472 avail 3692167168 00000100:00100000:2.0:1776178657.633196:0:9554:0:(service.c:2372:ptlrpc_server_handle_request()) Handled RPC req@00000000a9cfe89c pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_create00:lustre-MDT0001-mdtlov_UUID+5:3353:x1862456998112064:12345-0@lo:13:osp-pre-1-1.0 Request processed in 77us (260us total) trans 0 rc 0/0 00000100:00100000:2.0:1776178657.633204:0:9554:0:(nrs_fifo.c:241:nrs_fifo_req_stop()) NRS stop fifo request from 12345-0@lo, seq: 665 00000100:00100000:3.0:1776178657.633416:0:3353:0:(client.c:2215:ptlrpc_check_set()) Completed RPC req@0000000025b3b8e0 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_00:lustre-MDT0001-mdtlov_UUID:3353:1862456998112064:0@lo:13:osp-pre-1-1.0 00010000:00010000:2.0:1776178657.760549:0:9224:0:(ldlm_lockd.c:2792:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:3.0:1776178658.144984:0:3356:0:(ptlrpcd.c:415:ptlrpcd_check()) transfer 1 async RPCs [2->3] 00000100:00100000:3.0:1776178658.145012:0:3356:0:(client.c:1733:ptlrpc_send_new_req()) Sending RPC req@00000000e787b61f pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_03:lustre-MDT0001-mdtlov_UUID:3356:1862456998112128:0@lo:13:osp-pre-0-1.0 00000100:00100000:3.0:1776178658.145118:0:3356:0:(events.c:361:request_in_callback()) peer: 12345-0@lo (source: 12345-0@lo) 00000100:00100000:1.0:1776178658.145185:0:8231:0:(service.c:2152:ptlrpc_server_handle_req_in()) got req x1862456998112128 00000100:00100000:1.0:1776178658.145242:0:8231:0:(nrs_fifo.c:179:nrs_fifo_req_get()) NRS start fifo request from 12345-0@lo, seq: 666 00000100:00100000:1.0:1776178658.145248:0:8231:0:(service.c:2324:ptlrpc_server_handle_request()) Handling RPC req@0000000025b3b8e0 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_create00:lustre-MDT0001-mdtlov_UUID+5:3356:x1862456998112128:12345-0@lo:13:osp-pre-0-1.0 00002000:00000024:1.0:1776178658.145282:0:8231:0:(ofd_obd.c:740:ofd_statfs()) blocks cached 315392 granted 484883776 pending 0 free 3923689472 avail 3692167168 00000100:00100000:1.0:1776178658.145343:0:8231:0:(service.c:2372:ptlrpc_server_handle_request()) Handled RPC req@0000000025b3b8e0 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_create00:lustre-MDT0001-mdtlov_UUID+5:3356:x1862456998112128:12345-0@lo:13:osp-pre-0-1.0 Request processed in 94us (226us total) trans 0 rc 0/0 00000100:00100000:1.0:1776178658.145351:0:8231:0:(nrs_fifo.c:241:nrs_fifo_req_stop()) NRS stop fifo request from 12345-0@lo, seq: 666 00000100:00100000:3.0:1776178658.145407:0:3356:0:(client.c:2215:ptlrpc_check_set()) Completed RPC req@00000000e787b61f pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_03:lustre-MDT0001-mdtlov_UUID:3356:1862456998112128:0@lo:13:osp-pre-0-1.0 00010000:00010000:1.0:1776178658.785928:0:9225:0:(ldlm_lockd.c:2792:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:1.0:1776178659.617755:0:3346:0:(events.c:361:request_in_callback()) peer: 12345-192.168.204.3@tcp (source: 12345-192.168.204.3@tcp) 00000100:00100000:2.0:1776178659.618189:0:6255:0:(service.c:2152:ptlrpc_server_handle_req_in()) got req x1862456979137536 00000100:00100000:2.0:1776178659.618249:0:6255:0:(nrs_fifo.c:179:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.204.3@tcp, seq: 6980 00000100:00100000:2.0:1776178659.618257:0:6255:0:(service.c:2324:ptlrpc_server_handle_request()) Handling RPC req@00000000f141ac55 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_000:0+-99:2269:x1862456979137536:12345-192.168.204.3@tcp:38:kworker/u8:4.0 00010000:02000000:2.0:1776178659.618276:0:6255:0:(ldlm_lib.c:1139:target_handle_connect()) lustre-MDT0000: Not available for connect from 192.168.204.3@tcp (stopping) 00000100:00100000:0.0:1776178659.618352:0:3347:0:(events.c:361:request_in_callback()) peer: 12345-192.168.204.3@tcp (source: 12345-192.168.204.3@tcp) 00000100:00100000:2.0:1776178659.618354:0:6255:0:(service.c:2372:ptlrpc_server_handle_request()) Handled RPC req@00000000f141ac55 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_000:0+-99:2269:x1862456979137536:12345-192.168.204.3@tcp:38:kworker/u8:4.0 Request processed in 95us (604us total) trans 0 rc -19/-19 00000100:00100000:2.0:1776178659.618365:0:6255:0:(nrs_fifo.c:241:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.204.3@tcp, seq: 6980 00000100:00100000:2.0:1776178659.618387:0:6255:0:(service.c:2152:ptlrpc_server_handle_req_in()) got req x1862456979137600 00000100:00100000:2.0:1776178659.618440:0:6255:0:(nrs_fifo.c:179:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.204.3@tcp, seq: 1195 00000100:00100000:2.0:1776178659.618446:0:6255:0:(service.c:2324:ptlrpc_server_handle_request()) Handling RPC req@00000000ac4116de pname:cluuid+ref:pid:xid:nid:opc:job mdt00_000:b50963bd-8eba-4432-9a54-8888d00e3fa1+6:2273:x1862456979137600:12345-192.168.204.3@tcp:400:kworker/u8:4.0 00000100:00100000:2.0:1776178659.618486:0:6255:0:(service.c:2372:ptlrpc_server_handle_request()) Handled RPC req@00000000ac4116de pname:cluuid+ref:pid:xid:nid:opc:job mdt00_000:b50963bd-8eba-4432-9a54-8888d00e3fa1+6:2273:x1862456979137600:12345-192.168.204.3@tcp:400:kworker/u8:4.0 Request processed in 40us (142us total) trans 0 rc 0/0 00000100:00100000:2.0:1776178659.618493:0:6255:0:(nrs_fifo.c:241:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.204.3@tcp, seq: 1195 00000100:00100000:0.0:1776178659.618548:0:3347:0:(events.c:361:request_in_callback()) peer: 12345-192.168.204.3@tcp (source: 12345-192.168.204.3@tcp) 00000100:00100000:0.0:1776178659.618632:0:3347:0:(events.c:361:request_in_callback()) peer: 12345-192.168.204.3@tcp (source: 12345-192.168.204.3@tcp) 00000100:00100000:0.0:1776178659.618887:0:3347:0:(events.c:361:request_in_callback()) peer: 12345-192.168.204.3@tcp (source: 12345-192.168.204.3@tcp) 00000100:00100000:0.0:1776178659.618931:0:20755:0:(service.c:2152:ptlrpc_server_handle_req_in()) got req x1862456979137728 00000100:00100000:0.0:1776178659.618977:0:20755:0:(service.c:2152:ptlrpc_server_handle_req_in()) got req x1862456979137664 00000100:00100000:0.0:1776178659.619144:0:20755:0:(nrs_fifo.c:179:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.204.3@tcp, seq: 867 00000100:00100000:0.0:1776178659.619151:0:20755:0:(service.c:2324:ptlrpc_server_handle_request()) Handling RPC req@00000000ffd05959 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_006:b50963bd-8eba-4432-9a54-8888d00e3fa1+9:2273:x1862456979137728:12345-192.168.204.3@tcp:400:kworker/u8:4.0 00000100:00100000:0.0:1776178659.619216:0:20755:0:(service.c:2372:ptlrpc_server_handle_request()) Handled RPC req@00000000ffd05959 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_006:b50963bd-8eba-4432-9a54-8888d00e3fa1+9:2273:x1862456979137728:12345-192.168.204.3@tcp:400:kworker/u8:4.0 Request processed in 65us (671us total) trans 0 rc 0/0 00000100:00100000:0.0:1776178659.619224:0:20755:0:(nrs_fifo.c:241:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.204.3@tcp, seq: 867 00000100:00100000:0.0:1776178659.619241:0:20755:0:(nrs_fifo.c:179:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.204.3@tcp, seq: 868 00000100:00100000:0.0:1776178659.619245:0:20755:0:(service.c:2324:ptlrpc_server_handle_request()) Handling RPC req@000000007309261c pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_006:b50963bd-8eba-4432-9a54-8888d00e3fa1+8:2273:x1862456979137664:12345-192.168.204.3@tcp:400:kworker/u8:4.0 00000100:00100000:0.0:1776178659.619271:0:20755:0:(service.c:2372:ptlrpc_server_handle_request()) Handled RPC req@000000007309261c pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_006:b50963bd-8eba-4432-9a54-8888d00e3fa1+8:2273:x1862456979137664:12345-192.168.204.3@tcp:400:kworker/u8:4.0 Request processed in 25us (385us total) trans 0 rc 0/0 00000100:00100000:0.0:1776178659.619281:0:20755:0:(nrs_fifo.c:241:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.204.3@tcp, seq: 868 00000100:00100000:1.0:1776178659.619444:0:20140:0:(service.c:2152:ptlrpc_server_handle_req_in()) got req x1862456979137472 00000100:00100000:1.0:1776178659.619499:0:20140:0:(nrs_fifo.c:179:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.204.3@tcp, seq: 333 00000100:00100000:1.0:1776178659.619504:0:20140:0:(service.c:2324:ptlrpc_server_handle_request()) Handling RPC req@0000000017512ce0 pname:cluuid+ref:pid:xid:nid:opc:job ll_mgs_0002:5b5c66b1-1a2f-4651-8548-0c75b9a38a94+9:2273:x1862456979137472:12345-192.168.204.3@tcp:400:kworker/u8:4.0 00000100:00100000:1.0:1776178659.619545:0:20140:0:(service.c:2372:ptlrpc_server_handle_request()) Handled RPC req@0000000017512ce0 pname:cluuid+ref:pid:xid:nid:opc:job ll_mgs_0002:5b5c66b1-1a2f-4651-8548-0c75b9a38a94+9:2273:x1862456979137472:12345-192.168.204.3@tcp:400:kworker/u8:4.0 Request processed in 40us (914us total) trans 0 rc 0/0 00000100:00100000:1.0:1776178659.619551:0:20140:0:(nrs_fifo.c:241:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.204.3@tcp, seq: 333 00010000:00010000:2.0:1776178659.808540:0:6244:0:(ldlm_lockd.c:2792:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00010000:00010000:1.0:1776178660.832556:0:6245:0:(ldlm_lockd.c:2792:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00080000:1.0:1776178661.217035:0:22249:0:(pinger.c:251:ptlrpc_pinger_process_import()) lustre-MDT0000-lwp-OST0000_UUID->lustre-MDT0000_UUID: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00080000:1.0:1776178661.217046:0:22249:0:(recover.c:57:ptlrpc_initiate_recovery()) lustre-MDT0000_UUID: starting recovery 00000100:00080000:1.0:1776178661.217050:0:22249:0:(import.c:86:import_set_state_nolock()) 0000000047601a98 lustre-MDT0000_UUID: changing import state from DISCONN to CONNECTING 00000100:00080000:1.0:1776178661.217114:0:22249:0:(import.c:534:import_select_connection()) lustre-MDT0000-lwp-OST0000: connect to NID 0@lo last attempt 1363 00000100:00080000:1.0:1776178661.217118:0:22249:0:(import.c:581:import_select_connection()) lustre-MDT0000-lwp-OST0000: tried all connections, increasing latency to 21s 00000100:00080000:1.0:1776178661.217127:0:22249:0:(import.c:615:import_select_connection()) lustre-MDT0000-lwp-OST0000: import 0000000047601a98 using connection 192.168.204.103@tcp/0@lo 00000100:00100000:1.0:1776178661.217248:0:22249:0:(import.c:807:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5) req@000000005b125149 x1862456998112256/t0(0) o38->lustre-MDT0000-lwp-OST0000@0@lo:12/10 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' 00000100:00080000:1.0:1776178661.217294:0:22249:0:(pinger.c:251:ptlrpc_pinger_process_import()) lustre-MDT0000-lwp-OST0001_UUID->lustre-MDT0000_UUID: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00080000:1.0:1776178661.217298:0:22249:0:(recover.c:57:ptlrpc_initiate_recovery()) lustre-MDT0000_UUID: starting recovery 00000100:00080000:1.0:1776178661.217299:0:22249:0:(import.c:86:import_set_state_nolock()) 00000000dfc8db13 lustre-MDT0000_UUID: changing import state from DISCONN to CONNECTING 00000100:00080000:1.0:1776178661.217303:0:22249:0:(import.c:534:import_select_connection()) lustre-MDT0000-lwp-OST0001: connect to NID 0@lo last attempt 1363 00000100:00080000:1.0:1776178661.217306:0:22249:0:(import.c:581:import_select_connection()) lustre-MDT0000-lwp-OST0001: tried all connections, increasing latency to 21s 00000100:00080000:1.0:1776178661.217310:0:22249:0:(import.c:615:import_select_connection()) lustre-MDT0000-lwp-OST0001: import 00000000dfc8db13 using connection 192.168.204.103@tcp/0@lo 00000100:00100000:1.0:1776178661.217322:0:22249:0:(import.c:807:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5) req@00000000a23c68f9 x1862456998112384/t0(0) o38->lustre-MDT0000-lwp-OST0001@0@lo:12/10 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' 00000100:00080000:1.0:1776178661.217426:0:22249:0:(pinger.c:251:ptlrpc_pinger_process_import()) lustre-MDT0001-mdtlov_UUID->lustre-MDT0000_UUID: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00080000:1.0:1776178661.217431:0:22249:0:(recover.c:57:ptlrpc_initiate_recovery()) lustre-MDT0000_UUID: starting recovery 00000100:00080000:1.0:1776178661.217432:0:22249:0:(import.c:86:import_set_state_nolock()) 00000000ba4f1821 lustre-MDT0000_UUID: changing import state from DISCONN to CONNECTING 00000100:00080000:1.0:1776178661.217436:0:22249:0:(import.c:534:import_select_connection()) lustre-MDT0000-osp-MDT0001: connect to NID 0@lo last attempt 1363 00000100:00080000:1.0:1776178661.217437:0:22249:0:(import.c:581:import_select_connection()) lustre-MDT0000-osp-MDT0001: tried all connections, increasing latency to 21s 00000100:00080000:1.0:1776178661.217441:0:22249:0:(import.c:615:import_select_connection()) lustre-MDT0000-osp-MDT0001: import 00000000ba4f1821 using connection 192.168.204.103@tcp/0@lo 00000100:00100000:1.0:1776178661.217454:0:22249:0:(import.c:807:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5) req@000000009126d8c6 x1862456998112512/t0(0) o38->lustre-MDT0000-osp-MDT0001@0@lo:24/4 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' 00000100:00080000:1.0:1776178661.217475:0:22249:0:(pinger.c:251:ptlrpc_pinger_process_import()) lustre-MDT0000-lwp-MDT0001_UUID->lustre-MDT0000_UUID: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00080000:1.0:1776178661.217479:0:22249:0:(recover.c:57:ptlrpc_initiate_recovery()) lustre-MDT0000_UUID: starting recovery 00000100:00080000:1.0:1776178661.217480:0:22249:0:(import.c:86:import_set_state_nolock()) 00000000a7c55b1f lustre-MDT0000_UUID: changing import state from DISCONN to CONNECTING 00000100:00080000:1.0:1776178661.217483:0:22249:0:(import.c:534:import_select_connection()) lustre-MDT0000-lwp-MDT0001: connect to NID 0@lo last attempt 1363 00000100:00080000:1.0:1776178661.217485:0:22249:0:(import.c:581:import_select_connection()) lustre-MDT0000-lwp-MDT0001: tried all connections, increasing latency to 21s 00000100:00080000:1.0:1776178661.217488:0:22249:0:(import.c:615:import_select_connection()) lustre-MDT0000-lwp-MDT0001: import 00000000a7c55b1f using connection 192.168.204.103@tcp/0@lo 00000100:00100000:1.0:1776178661.217497:0:22249:0:(import.c:807:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5) req@00000000bfb13eb6 x1862456998112704/t0(0) o38->lustre-MDT0000-lwp-MDT0001@0@lo:12/10 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' 00000100:00100000:1.0:1776178661.217547:0:3354:0:(client.c:1733:ptlrpc_send_new_req()) Sending RPC req@000000005accf721 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_01:lustre-MDT0001-lwp-OST0001_UUID:3354:1862456998112448:0@lo:400:kworker/u8:0.0 00000100:00100000:1.0:1776178661.217724:0:3354:0:(events.c:361:request_in_callback()) peer: 12345-0@lo (source: 12345-0@lo) 00000100:00100000:1.0:1776178661.217772:0:3354:0:(ptlrpcd.c:415:ptlrpcd_check()) transfer 1 async RPCs [0->1] 00000100:00100000:1.0:1776178661.217779:0:3354:0:(client.c:1733:ptlrpc_send_new_req()) Sending RPC req@00000000b821049d pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_01:lustre-MDT0001-lwp-OST0000_UUID:3354:1862456998112320:0@lo:400:kworker/u8:0.0 00000100:00100000:1.0:1776178661.217803:0:3354:0:(events.c:361:request_in_callback()) peer: 12345-0@lo (source: 12345-0@lo) 00000100:00100000:1.0:1776178661.217852:0:8250:0:(service.c:2152:ptlrpc_server_handle_req_in()) got req x1862456998112448 00000100:00100000:1.0:1776178661.217912:0:8250:0:(service.c:2152:ptlrpc_server_handle_req_in()) got req x1862456998112320 00000100:00100000:1.0:1776178661.217940:0:8250:0:(nrs_fifo.c:179:nrs_fifo_req_get()) NRS start fifo request from 12345-0@lo, seq: 1196 00000100:00100000:1.0:1776178661.217947:0:8250:0:(service.c:2324:ptlrpc_server_handle_request()) Handling RPC req@0000000078c1d864 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_003:lustre-MDT0001-lwp-OST0001_UUID+5:3354:x1862456998112448:12345-0@lo:400:kworker/u8:0.0 00000100:00100000:1.0:1776178661.218018:0:8250:0:(service.c:2372:ptlrpc_server_handle_request()) Handled RPC req@0000000078c1d864 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_003:lustre-MDT0001-lwp-OST0001_UUID+5:3354:x1862456998112448:12345-0@lo:400:kworker/u8:0.0 Request processed in 70us (296us total) trans 0 rc 0/0 00000100:00100000:1.0:1776178661.218028:0:8250:0:(nrs_fifo.c:241:nrs_fifo_req_stop()) NRS stop fifo request from 12345-0@lo, seq: 1196 00000100:00100000:1.0:1776178661.218051:0:8250:0:(nrs_fifo.c:179:nrs_fifo_req_get()) NRS start fifo request from 12345-0@lo, seq: 1197 00000100:00100000:1.0:1776178661.218055:0:8250:0:(service.c:2324:ptlrpc_server_handle_request()) Handling RPC req@000000008d6accd5 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_003:lustre-MDT0001-lwp-OST0000_UUID+5:3354:x1862456998112320:12345-0@lo:400:kworker/u8:0.0 00000100:00100000:1.0:1776178661.218114:0:8250:0:(service.c:2372:ptlrpc_server_handle_request()) Handled RPC req@000000008d6accd5 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_003:lustre-MDT0001-lwp-OST0000_UUID+5:3354:x1862456998112320:12345-0@lo:400:kworker/u8:0.0 Request processed in 58us (311us total) trans 0 rc 0/0 00000100:00100000:1.0:1776178661.218120:0:8250:0:(nrs_fifo.c:241:nrs_fifo_req_stop()) NRS stop fifo request from 12345-0@lo, seq: 1197 00000100:00100000:3.0:1776178661.218289:0:3356:0:(client.c:1733:ptlrpc_send_new_req()) Sending RPC req@00000000ae838296 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_03:ac61f839-0e8e-4cac-9c59-08bcfbbad330:3356:1862456998112192:0@lo:400:kworker/u8:0.0 00000100:00100000:3.0:1776178661.218344:0:3356:0:(events.c:361:request_in_callback()) peer: 12345-0@lo (source: 12345-0@lo) 00000100:00100000:3.0:1776178661.218366:0:3356:0:(client.c:1733:ptlrpc_send_new_req()) Sending RPC req@000000006247494d pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_03:lustre-MDT0001-mdtlov_UUID:3356:1862456998112640:0@lo:400:kworker/u8:0.0 00000100:00100000:3.0:1776178661.218392:0:3356:0:(events.c:361:request_in_callback()) peer: 12345-0@lo (source: 12345-0@lo) 00000100:00100000:3.0:1776178661.218413:0:3356:0:(ptlrpcd.c:415:ptlrpcd_check()) transfer 1 async RPCs [2->3] 00000100:00100000:3.0:1776178661.218419:0:3356:0:(client.c:1733:ptlrpc_send_new_req()) Sending RPC req@000000008c4bfd71 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_03:lustre-MDT0001-mdtlov_UUID:3356:1862456998112576:0@lo:400:kworker/u8:0.0 00000100:00100000:3.0:1776178661.218440:0:3356:0:(events.c:361:request_in_callback()) peer: 12345-0@lo (source: 12345-0@lo) 00000100:00100000:3.0:1776178661.218474:0:20137:0:(service.c:2152:ptlrpc_server_handle_req_in()) got req x1862456998112192 00000100:00100000:3.0:1776178661.218509:0:20137:0:(nrs_fifo.c:179:nrs_fifo_req_get()) NRS start fifo request from 12345-0@lo, seq: 334 00000100:00100000:3.0:1776178661.218513:0:20137:0:(service.c:2324:ptlrpc_server_handle_request()) Handling RPC req@000000004d415026 pname:cluuid+ref:pid:xid:nid:opc:job ll_mgs_0000:ac61f839-0e8e-4cac-9c59-08bcfbbad330+18:3356:x1862456998112192:12345-0@lo:400:kworker/u8:0.0 00000100:00100000:3.0:1776178661.218547:0:20137:0:(service.c:2372:ptlrpc_server_handle_request()) Handled RPC req@000000004d415026 pname:cluuid+ref:pid:xid:nid:opc:job ll_mgs_0000:ac61f839-0e8e-4cac-9c59-08bcfbbad330+18:3356:x1862456998112192:12345-0@lo:400:kworker/u8:0.0 Request processed in 33us (203us total) trans 0 rc 0/0 00000100:00100000:3.0:1776178661.218553:0:20137:0:(nrs_fifo.c:241:nrs_fifo_req_stop()) NRS stop fifo request from 12345-0@lo, seq: 334 00000100:00100000:3.0:1776178661.218585:0:20759:0:(service.c:2152:ptlrpc_server_handle_req_in()) got req x1862456998112640 00000100:00100000:3.0:1776178661.218626:0:20759:0:(service.c:2152:ptlrpc_server_handle_req_in()) got req x1862456998112576 00000100:00100000:3.0:1776178661.218643:0:20759:0:(nrs_fifo.c:179:nrs_fifo_req_get()) NRS start fifo request from 12345-0@lo, seq: 869 00000100:00100000:3.0:1776178661.218646:0:20759:0:(service.c:2324:ptlrpc_server_handle_request()) Handling RPC req@00000000ad99db59 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_010:lustre-MDT0001-mdtlov_UUID+5:3356:x1862456998112640:12345-0@lo:400:kworker/u8:0.0 00000100:00100000:3.0:1776178661.218676:0:20759:0:(service.c:2372:ptlrpc_server_handle_request()) Handled RPC req@00000000ad99db59 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_010:lustre-MDT0001-mdtlov_UUID+5:3356:x1862456998112640:12345-0@lo:400:kworker/u8:0.0 Request processed in 29us (284us total) trans 0 rc 0/0 00000100:00100000:3.0:1776178661.218682:0:20759:0:(nrs_fifo.c:241:nrs_fifo_req_stop()) NRS stop fifo request from 12345-0@lo, seq: 869 00000100:00100000:3.0:1776178661.218693:0:20759:0:(nrs_fifo.c:179:nrs_fifo_req_get()) NRS start fifo request from 12345-0@lo, seq: 870 00000100:00100000:3.0:1776178661.218696:0:20759:0:(service.c:2324:ptlrpc_server_handle_request()) Handling RPC req@0000000081aacc4a pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_010:lustre-MDT0001-mdtlov_UUID+5:3356:x1862456998112576:12345-0@lo:400:kworker/u8:0.0 00000100:00100000:3.0:1776178661.218717:0:20759:0:(service.c:2372:ptlrpc_server_handle_request()) Handled RPC req@0000000081aacc4a pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_010:lustre-MDT0001-mdtlov_UUID+5:3356:x1862456998112576:12345-0@lo:400:kworker/u8:0.0 Request processed in 20us (277us total) trans 0 rc 0/0 00000100:00100000:3.0:1776178661.218722:0:20759:0:(nrs_fifo.c:241:nrs_fifo_req_stop()) NRS stop fifo request from 12345-0@lo, seq: 870 00000100:00100000:3.0:1776178661.218767:0:3356:0:(client.c:2215:ptlrpc_check_set()) Completed RPC req@000000008c4bfd71 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_03:lustre-MDT0001-mdtlov_UUID:3356:1862456998112576:0@lo:400:kworker/u8:0.0 00000100:00100000:3.0:1776178661.218776:0:3356:0:(client.c:2215:ptlrpc_check_set()) Completed RPC req@00000000ae838296 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_03:ac61f839-0e8e-4cac-9c59-08bcfbbad330:3356:1862456998112192:0@lo:400:kworker/u8:0.0 00000100:00100000:3.0:1776178661.218784:0:3356:0:(client.c:2215:ptlrpc_check_set()) Completed RPC req@000000006247494d pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_03:lustre-MDT0001-mdtlov_UUID:3356:1862456998112640:0@lo:400:kworker/u8:0.0 00000100:00100000:2.0:1776178661.219114:0:3352:0:(client.c:1733:ptlrpc_send_new_req()) Sending RPC req@000000005b125149 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:lustre-MDT0000-lwp-OST0000_UUID:3352:1862456998112256:0@lo:38:kworker/u8:0.0 00000100:00100000:2.0:1776178661.219168:0:3352:0:(events.c:361:request_in_callback()) peer: 12345-0@lo (source: 12345-0@lo) 00000100:00100000:2.0:1776178661.219193:0:3352:0:(client.c:1733:ptlrpc_send_new_req()) Sending RPC req@00000000a23c68f9 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:lustre-MDT0000-lwp-OST0001_UUID:3352:1862456998112384:0@lo:38:kworker/u8:0.0 00000100:00100000:2.0:1776178661.219222:0:3352:0:(events.c:361:request_in_callback()) peer: 12345-0@lo (source: 12345-0@lo) 00000100:00100000:2.0:1776178661.219242:0:3352:0:(client.c:1733:ptlrpc_send_new_req()) Sending RPC req@000000009126d8c6 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:lustre-MDT0001-mdtlov_UUID:3352:1862456998112512:0@lo:38:kworker/u8:0.0 00000100:00100000:2.0:1776178661.219282:0:3352:0:(events.c:361:request_in_callback()) peer: 12345-0@lo (source: 12345-0@lo) 00000100:00100000:2.0:1776178661.219300:0:3352:0:(client.c:1733:ptlrpc_send_new_req()) Sending RPC req@00000000bfb13eb6 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:lustre-MDT0000-lwp-MDT0001_UUID:3352:1862456998112704:0@lo:38:kworker/u8:0.0 00000100:00100000:2.0:1776178661.219323:0:3352:0:(events.c:361:request_in_callback()) peer: 12345-0@lo (source: 12345-0@lo) 00000100:00100000:2.0:1776178661.219373:0:15213:0:(service.c:2152:ptlrpc_server_handle_req_in()) got req x1862456998112512 00000100:00100000:2.0:1776178661.219405:0:15213:0:(nrs_fifo.c:179:nrs_fifo_req_get()) NRS start fifo request from 12345-0@lo, seq: 797 00000100:00100000:2.0:1776178661.219409:0:15213:0:(service.c:2324:ptlrpc_server_handle_request()) Handling RPC req@0000000084c00ed0 pname:cluuid+ref:pid:xid:nid:opc:job mdt_out00_003:0+-99:3352:x1862456998112512:12345-0@lo:38:kworker/u8:0.0 00010000:02000000:2.0:1776178661.219426:0:15213:0:(ldlm_lib.c:1139:target_handle_connect()) lustre-MDT0000: Not available for connect from 0@lo (stopping) 00000100:00100000:2.0:1776178661.219464:0:15213:0:(service.c:2372:ptlrpc_server_handle_request()) Handled RPC req@0000000084c00ed0 pname:cluuid+ref:pid:xid:nid:opc:job mdt_out00_003:0+-99:3352:x1862456998112512:12345-0@lo:38:kworker/u8:0.0 Request processed in 54us (181us total) trans 0 rc -19/-19 00000100:00100000:2.0:1776178661.219470:0:15213:0:(nrs_fifo.c:241:nrs_fifo_req_stop()) NRS stop fifo request from 12345-0@lo, seq: 797 00000100:00100000:2.0:1776178661.219499:0:6255:0:(service.c:2152:ptlrpc_server_handle_req_in()) got req x1862456998112256 00000100:00100000:2.0:1776178661.219521:0:6255:0:(service.c:2152:ptlrpc_server_handle_req_in()) got req x1862456998112384 00000100:00100000:2.0:1776178661.219533:0:6255:0:(service.c:2152:ptlrpc_server_handle_req_in()) got req x1862456998112704 00000100:00100000:2.0:1776178661.219543:0:6255:0:(nrs_fifo.c:179:nrs_fifo_req_get()) NRS start fifo request from 12345-0@lo, seq: 6981 00000100:00100000:2.0:1776178661.219547:0:6255:0:(service.c:2324:ptlrpc_server_handle_request()) Handling RPC req@0000000007fa878c pname:cluuid+ref:pid:xid:nid:opc:job mdt00_000:0+-99:3352:x1862456998112256:12345-0@lo:38:kworker/u8:0.0 00010000:02000000:2.0:1776178661.219556:0:6255:0:(ldlm_lib.c:1139:target_handle_connect()) lustre-MDT0000: Not available for connect from 0@lo (stopping) 00000100:00100000:2.0:1776178661.219582:0:6255:0:(service.c:2372:ptlrpc_server_handle_request()) Handled RPC req@0000000007fa878c pname:cluuid+ref:pid:xid:nid:opc:job mdt00_000:0+-99:3352:x1862456998112256:12345-0@lo:38:kworker/u8:0.0 Request processed in 33us (414us total) trans 0 rc -19/-19 00000100:00100000:2.0:1776178661.219588:0:6255:0:(nrs_fifo.c:241:nrs_fifo_req_stop()) NRS stop fifo request from 12345-0@lo, seq: 6981 00000100:00100000:2.0:1776178661.219614:0:6255:0:(nrs_fifo.c:179:nrs_fifo_req_get()) NRS start fifo request from 12345-0@lo, seq: 6982 00000100:00100000:2.0:1776178661.219617:0:6255:0:(service.c:2324:ptlrpc_server_handle_request()) Handling RPC req@000000004eda7977 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_000:0+-99:3352:x1862456998112384:12345-0@lo:38:kworker/u8:0.0 00010000:02000000:2.0:1776178661.219623:0:6255:0:(ldlm_lib.c:1139:target_handle_connect()) lustre-MDT0000: Not available for connect from 0@lo (stopping) 00000100:00100000:2.0:1776178661.219644:0:6255:0:(service.c:2372:ptlrpc_server_handle_request()) Handled RPC req@000000004eda7977 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_000:0+-99:3352:x1862456998112384:12345-0@lo:38:kworker/u8:0.0 Request processed in 26us (422us total) trans 0 rc -19/-19 00000100:00100000:2.0:1776178661.219650:0:6255:0:(nrs_fifo.c:241:nrs_fifo_req_stop()) NRS stop fifo request from 12345-0@lo, seq: 6982 00000100:00100000:2.0:1776178661.219661:0:6255:0:(nrs_fifo.c:179:nrs_fifo_req_get()) NRS start fifo request from 12345-0@lo, seq: 6983 00000100:00100000:2.0:1776178661.219663:0:6255:0:(service.c:2324:ptlrpc_server_handle_request()) Handling RPC req@00000000a2082164 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_000:0+-99:3352:x1862456998112704:12345-0@lo:38:kworker/u8:0.0 00010000:02000000:2.0:1776178661.219669:0:6255:0:(ldlm_lib.c:1139:target_handle_connect()) lustre-MDT0000: Not available for connect from 0@lo (stopping) 00000100:00100000:2.0:1776178661.219688:0:6255:0:(service.c:2372:ptlrpc_server_handle_request()) Handled RPC req@00000000a2082164 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_000:0+-99:3352:x1862456998112704:12345-0@lo:38:kworker/u8:0.0 Request processed in 23us (365us total) trans 0 rc -19/-19 00000100:00100000:2.0:1776178661.219694:0:6255:0:(nrs_fifo.c:241:nrs_fifo_req_stop()) NRS stop fifo request from 12345-0@lo, seq: 6983 00000100:00080000:2.0:1776178661.219726:0:3352:0:(import.c:86:import_set_state_nolock()) 0000000047601a98 lustre-MDT0000_UUID: changing import state from CONNECTING to DISCONN 00000100:00080000:2.0:1776178661.219730:0:3352:0:(import.c:1429:ptlrpc_connect_interpret()) recovery of lustre-MDT0000_UUID on 192.168.204.103@tcp failed (-19) 00000100:00100000:2.0:1776178661.219734:0:3352:0:(client.c:2215:ptlrpc_check_set()) Completed RPC req@000000005b125149 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:lustre-MDT0000-lwp-OST0000_UUID:3352:1862456998112256:0@lo:38:kworker/u8:0.0 00000100:00080000:2.0:1776178661.219743:0:3352:0:(import.c:86:import_set_state_nolock()) 00000000dfc8db13 lustre-MDT0000_UUID: changing import state from CONNECTING to DISCONN 00000100:00080000:2.0:1776178661.219746:0:3352:0:(import.c:1429:ptlrpc_connect_interpret()) recovery of lustre-MDT0000_UUID on 192.168.204.103@tcp failed (-19) 00000100:00100000:2.0:1776178661.219748:0:3352:0:(client.c:2215:ptlrpc_check_set()) Completed RPC req@00000000a23c68f9 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:lustre-MDT0000-lwp-OST0001_UUID:3352:1862456998112384:0@lo:38:kworker/u8:0.0 00000100:00080000:2.0:1776178661.219756:0:3352:0:(import.c:86:import_set_state_nolock()) 00000000ba4f1821 lustre-MDT0000_UUID: changing import state from CONNECTING to DISCONN 00000100:00080000:2.0:1776178661.219759:0:3352:0:(import.c:1429:ptlrpc_connect_interpret()) recovery of lustre-MDT0000_UUID on 192.168.204.103@tcp failed (-19) 00000100:00100000:2.0:1776178661.219761:0:3352:0:(client.c:2215:ptlrpc_check_set()) Completed RPC req@000000009126d8c6 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:lustre-MDT0001-mdtlov_UUID:3352:1862456998112512:0@lo:38:kworker/u8:0.0 00000100:00080000:2.0:1776178661.219769:0:3352:0:(import.c:86:import_set_state_nolock()) 00000000a7c55b1f lustre-MDT0000_UUID: changing import state from CONNECTING to DISCONN 00000100:00080000:2.0:1776178661.219771:0:3352:0:(import.c:1429:ptlrpc_connect_interpret()) recovery of lustre-MDT0000_UUID on 192.168.204.103@tcp failed (-19) 00000100:00100000:2.0:1776178661.219773:0:3352:0:(client.c:2215:ptlrpc_check_set()) Completed RPC req@00000000bfb13eb6 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:lustre-MDT0000-lwp-MDT0001_UUID:3352:1862456998112704:0@lo:38:kworker/u8:0.0 00000100:00100000:1.0:1776178661.222834:0:3354:0:(client.c:2215:ptlrpc_check_set()) Completed RPC req@00000000b821049d pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_01:lustre-MDT0001-lwp-OST0000_UUID:3354:1862456998112320:0@lo:400:kworker/u8:0.0 00000100:00100000:1.0:1776178661.222864:0:3354:0:(client.c:2215:ptlrpc_check_set()) Completed RPC req@000000005accf721 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_01:lustre-MDT0001-lwp-OST0001_UUID:3354:1862456998112448:0@lo:400:kworker/u8:0.0 00010000:00010000:2.0:1776178661.856416:0:9224:0:(ldlm_lockd.c:2792:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:1.0:1776178662.752438:0:3354:0:(ptlrpcd.c:415:ptlrpcd_check()) transfer 1 async RPCs [0->1] 00000100:00100000:1.0:1776178662.752459:0:3354:0:(client.c:1733:ptlrpc_send_new_req()) Sending RPC req@0000000009435d5b pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_01:lustre-MDT0001-mdtlov_UUID:3354:1862456998112768:0@lo:13:osp-pre-1-1.0 00000100:00100000:1.0:1776178662.752523:0:3354:0:(events.c:361:request_in_callback()) peer: 12345-0@lo (source: 12345-0@lo) 00000100:00100000:1.0:1776178662.752577:0:9554:0:(service.c:2152:ptlrpc_server_handle_req_in()) got req x1862456998112768 00000100:00100000:1.0:1776178662.752622:0:9554:0:(nrs_fifo.c:179:nrs_fifo_req_get()) NRS start fifo request from 12345-0@lo, seq: 667 00000100:00100000:1.0:1776178662.752627:0:9554:0:(service.c:2324:ptlrpc_server_handle_request()) Handling RPC req@000000005accf721 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_create00:lustre-MDT0001-mdtlov_UUID+5:3354:x1862456998112768:12345-0@lo:13:osp-pre-1-1.0 00002000:00000024:1.0:1776178662.752656:0:9554:0:(ofd_obd.c:740:ofd_statfs()) blocks cached 229376 granted 491746368 pending 0 free 3923689472 avail 3692167168 00000100:00100000:1.0:1776178662.752700:0:9554:0:(service.c:2372:ptlrpc_server_handle_request()) Handled RPC req@000000005accf721 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_create00:lustre-MDT0001-mdtlov_UUID+5:3354:x1862456998112768:12345-0@lo:13:osp-pre-1-1.0 Request processed in 72us (178us total) trans 0 rc 0/0 00000100:00100000:1.0:1776178662.752706:0:9554:0:(nrs_fifo.c:241:nrs_fifo_req_stop()) NRS stop fifo request from 12345-0@lo, seq: 667 00000100:00100000:1.0:1776178662.752747:0:3354:0:(client.c:2215:ptlrpc_check_set()) Completed RPC req@0000000009435d5b pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_01:lustre-MDT0001-mdtlov_UUID:3354:1862456998112768:0@lo:13:osp-pre-1-1.0 00010000:00010000:1.0:1776178662.889280:0:9225:0:(ldlm_lockd.c:2792:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:1.0:1776178663.264359:0:3354:0:(client.c:1733:ptlrpc_send_new_req()) Sending RPC req@00000000c324c57b pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_01:lustre-MDT0001-mdtlov_UUID:3354:1862456998112832:0@lo:13:osp-pre-0-1.0 00000100:00100000:1.0:1776178663.264428:0:3354:0:(events.c:361:request_in_callback()) peer: 12345-0@lo (source: 12345-0@lo) 00000100:00100000:2.0:1776178663.264573:0:8231:0:(service.c:2152:ptlrpc_server_handle_req_in()) got req x1862456998112832 00000100:00100000:2.0:1776178663.264619:0:8231:0:(nrs_fifo.c:179:nrs_fifo_req_get()) NRS start fifo request from 12345-0@lo, seq: 668 00000100:00100000:2.0:1776178663.264625:0:8231:0:(service.c:2324:ptlrpc_server_handle_request()) Handling RPC req@00000000b821049d pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_create00:lustre-MDT0001-mdtlov_UUID+5:3354:x1862456998112832:12345-0@lo:13:osp-pre-0-1.0 00002000:00000024:2.0:1776178663.264656:0:8231:0:(ofd_obd.c:740:ofd_statfs()) blocks cached 315392 granted 484883776 pending 0 free 3923689472 avail 3692167168 00000100:00100000:2.0:1776178663.264706:0:8231:0:(service.c:2372:ptlrpc_server_handle_request()) Handled RPC req@00000000b821049d pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_create00:lustre-MDT0001-mdtlov_UUID+5:3354:x1862456998112832:12345-0@lo:13:osp-pre-0-1.0 Request processed in 79us (279us total) trans 0 rc 0/0 00000100:00100000:2.0:1776178663.264713:0:8231:0:(nrs_fifo.c:241:nrs_fifo_req_stop()) NRS stop fifo request from 12345-0@lo, seq: 668 00000100:00100000:3.0:1776178663.264909:0:3354:0:(client.c:2215:ptlrpc_check_set()) Completed RPC req@00000000c324c57b pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_01:lustre-MDT0001-mdtlov_UUID:3354:1862456998112832:0@lo:13:osp-pre-0-1.0 00010000:00010000:2.0:1776178663.913021:0:6244:0:(ldlm_lockd.c:2792:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:0.0:1776178664.738188:0:3347:0:(events.c:361:request_in_callback()) peer: 12345-192.168.204.3@tcp (source: 12345-192.168.204.3@tcp) 00000100:00100000:0.0:1776178664.738242:0:3347:0:(events.c:361:request_in_callback()) peer: 12345-192.168.204.3@tcp (source: 12345-192.168.204.3@tcp) 00000100:00100000:0.0:1776178664.738282:0:3347:0:(events.c:361:request_in_callback()) peer: 12345-192.168.204.3@tcp (source: 12345-192.168.204.3@tcp) 00000100:00100000:0.0:1776178664.738327:0:3347:0:(events.c:361:request_in_callback()) peer: 12345-192.168.204.3@tcp (source: 12345-192.168.204.3@tcp) 00000100:00100000:0.0:1776178664.738374:0:14440:0:(service.c:2152:ptlrpc_server_handle_req_in()) got req x1862456979138048 00000100:00100000:0.0:1776178664.738421:0:14440:0:(service.c:2152:ptlrpc_server_handle_req_in()) got req x1862456979137984 00000100:00100000:0.0:1776178664.738441:0:14440:0:(nrs_fifo.c:179:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.204.3@tcp, seq: 871 00000100:00100000:0.0:1776178664.738447:0:14440:0:(service.c:2324:ptlrpc_server_handle_request()) Handling RPC req@00000000f97fd38e pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_004:b50963bd-8eba-4432-9a54-8888d00e3fa1+9:2270:x1862456979138048:12345-192.168.204.3@tcp:400:kworker/u8:3.0 00000100:00100000:0.0:1776178664.738513:0:14440:0:(service.c:2372:ptlrpc_server_handle_request()) Handled RPC req@00000000f97fd38e pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_004:b50963bd-8eba-4432-9a54-8888d00e3fa1+9:2270:x1862456979138048:12345-192.168.204.3@tcp:400:kworker/u8:3.0 Request processed in 65us (271us total) trans 0 rc 0/0 00000100:00100000:0.0:1776178664.738520:0:14440:0:(nrs_fifo.c:241:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.204.3@tcp, seq: 871 00000100:00100000:0.0:1776178664.738536:0:14440:0:(nrs_fifo.c:179:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.204.3@tcp, seq: 872 00000100:00100000:0.0:1776178664.738539:0:14440:0:(service.c:2324:ptlrpc_server_handle_request()) Handling RPC req@000000000bdf54a0 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_004:b50963bd-8eba-4432-9a54-8888d00e3fa1+8:2270:x1862456979137984:12345-192.168.204.3@tcp:400:kworker/u8:3.0 00000100:00100000:0.0:1776178664.738565:0:14440:0:(service.c:2372:ptlrpc_server_handle_request()) Handled RPC req@000000000bdf54a0 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_004:b50963bd-8eba-4432-9a54-8888d00e3fa1+8:2270:x1862456979137984:12345-192.168.204.3@tcp:400:kworker/u8:3.0 Request processed in 25us (239us total) trans 0 rc 0/0 00000100:00100000:0.0:1776178664.738571:0:14440:0:(nrs_fifo.c:241:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.204.3@tcp, seq: 872 00000100:00100000:3.0:1776178664.738987:0:20140:0:(service.c:2152:ptlrpc_server_handle_req_in()) got req x1862456979137792 00000100:00100000:3.0:1776178664.739024:0:20140:0:(nrs_fifo.c:179:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.204.3@tcp, seq: 335 00000100:00100000:3.0:1776178664.739028:0:20140:0:(service.c:2324:ptlrpc_server_handle_request()) Handling RPC req@00000000d1b75234 pname:cluuid+ref:pid:xid:nid:opc:job ll_mgs_0002:5b5c66b1-1a2f-4651-8548-0c75b9a38a94+9:2270:x1862456979137792:12345-192.168.204.3@tcp:400:kworker/u8:3.0 00000100:00100000:3.0:1776178664.739089:0:20140:0:(service.c:2372:ptlrpc_server_handle_request()) Handled RPC req@00000000d1b75234 pname:cluuid+ref:pid:xid:nid:opc:job ll_mgs_0002:5b5c66b1-1a2f-4651-8548-0c75b9a38a94+9:2270:x1862456979137792:12345-192.168.204.3@tcp:400:kworker/u8:3.0 Request processed in 59us (806us total) trans 0 rc 0/0 00000100:00100000:3.0:1776178664.739095:0:20140:0:(nrs_fifo.c:241:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.204.3@tcp, seq: 335 00000100:00100000:3.0:1776178664.739586:0:3347:0:(events.c:361:request_in_callback()) peer: 12345-192.168.204.3@tcp (source: 12345-192.168.204.3@tcp) 00000100:00100000:2.0:1776178664.739881:0:11281:0:(service.c:2152:ptlrpc_server_handle_req_in()) got req x1862456979137920 00000100:00100000:2.0:1776178664.739932:0:11281:0:(service.c:2152:ptlrpc_server_handle_req_in()) got req x1862456979137856 00000100:00100000:2.0:1776178664.739953:0:11281:0:(nrs_fifo.c:179:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.204.3@tcp, seq: 1198 00000100:00100000:2.0:1776178664.739957:0:11281:0:(service.c:2324:ptlrpc_server_handle_request()) Handling RPC req@000000001ee743a7 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_004:b50963bd-8eba-4432-9a54-8888d00e3fa1+6:2270:x1862456979137920:12345-192.168.204.3@tcp:400:kworker/u8:3.0 00000100:00100000:2.0:1776178664.740004:0:11281:0:(service.c:2372:ptlrpc_server_handle_request()) Handled RPC req@000000001ee743a7 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_004:b50963bd-8eba-4432-9a54-8888d00e3fa1+6:2270:x1862456979137920:12345-192.168.204.3@tcp:400:kworker/u8:3.0 Request processed in 46us (1825us total) trans 0 rc 0/0 00000100:00100000:2.0:1776178664.740010:0:11281:0:(nrs_fifo.c:241:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.204.3@tcp, seq: 1198 00000100:00100000:2.0:1776178664.740024:0:11281:0:(nrs_fifo.c:179:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.204.3@tcp, seq: 6984 00000100:00100000:2.0:1776178664.740027:0:11281:0:(service.c:2324:ptlrpc_server_handle_request()) Handling RPC req@00000000c91f3ff6 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_004:0+-99:2269:x1862456979137856:12345-192.168.204.3@tcp:38:kworker/u8:3.0 00010000:02000000:2.0:1776178664.740039:0:11281:0:(ldlm_lib.c:1139:target_handle_connect()) lustre-MDT0000: Not available for connect from 192.168.204.3@tcp (stopping) 00000100:00100000:2.0:1776178664.740098:0:11281:0:(service.c:2372:ptlrpc_server_handle_request()) Handled RPC req@00000000c91f3ff6 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_004:0+-99:2269:x1862456979137856:12345-192.168.204.3@tcp:38:kworker/u8:3.0 Request processed in 69us (514us total) trans 0 rc -19/-19 00000100:00100000:2.0:1776178664.740105:0:11281:0:(nrs_fifo.c:241:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.204.3@tcp, seq: 6984 00010000:00010000:1.0:1776178664.928419:0:6245:0:(ldlm_lockd.c:2792:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00010000:00010000:2.0:1776178665.952260:0:9224:0:(ldlm_lockd.c:2792:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00080000:3.0:1776178666.336609:0:36:0:(pinger.c:251:ptlrpc_pinger_process_import()) lustre-MDT0000-lwp-OST0000_UUID->lustre-MDT0000_UUID: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00100000:1.0:1776178666.336718:0:3356:0:(ptlrpcd.c:415:ptlrpcd_check()) transfer 1 async RPCs [2->3] 00000100:00100000:1.0:1776178666.336739:0:3356:0:(client.c:1733:ptlrpc_send_new_req()) Sending RPC req@00000000faeeda53 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_03:ac61f839-0e8e-4cac-9c59-08bcfbbad330:3356:1862456998112896:0@lo:400:kworker/u8:1.0 00000100:00100000:1.0:1776178666.336809:0:3356:0:(events.c:361:request_in_callback()) peer: 12345-0@lo (source: 12345-0@lo) 00000100:00100000:2.0:1776178666.336925:0:20137:0:(service.c:2152:ptlrpc_server_handle_req_in()) got req x1862456998112896 00000100:00100000:2.0:1776178666.336978:0:20137:0:(nrs_fifo.c:179:nrs_fifo_req_get()) NRS start fifo request from 12345-0@lo, seq: 336 00000100:00100000:2.0:1776178666.336984:0:20137:0:(service.c:2324:ptlrpc_server_handle_request()) Handling RPC req@00000000e22b0439 pname:cluuid+ref:pid:xid:nid:opc:job ll_mgs_0000:ac61f839-0e8e-4cac-9c59-08bcfbbad330+18:3356:x1862456998112896:12345-0@lo:400:kworker/u8:1.0 00000100:00100000:2.0:1776178666.337032:0:20137:0:(service.c:2372:ptlrpc_server_handle_request()) Handled RPC req@00000000e22b0439 pname:cluuid+ref:pid:xid:nid:opc:job ll_mgs_0000:ac61f839-0e8e-4cac-9c59-08bcfbbad330+18:3356:x1862456998112896:12345-0@lo:400:kworker/u8:1.0 Request processed in 47us (223us total) trans 0 rc 0/0 00000100:00100000:2.0:1776178666.337039:0:20137:0:(nrs_fifo.c:241:nrs_fifo_req_stop()) NRS stop fifo request from 12345-0@lo, seq: 336 00000100:00100000:1.0:1776178666.337154:0:3356:0:(client.c:2215:ptlrpc_check_set()) Completed RPC req@00000000faeeda53 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_03:ac61f839-0e8e-4cac-9c59-08bcfbbad330:3356:1862456998112896:0@lo:400:kworker/u8:1.0 00000100:00080000:3.0:1776178666.337294:0:36:0:(recover.c:57:ptlrpc_initiate_recovery()) lustre-MDT0000_UUID: starting recovery 00000100:00080000:3.0:1776178666.337350:0:36:0:(import.c:86:import_set_state_nolock()) 0000000047601a98 lustre-MDT0000_UUID: changing import state from DISCONN to CONNECTING 00000100:00080000:3.0:1776178666.337354:0:36:0:(import.c:534:import_select_connection()) lustre-MDT0000-lwp-OST0000: connect to NID 0@lo last attempt 1368 00000100:00080000:3.0:1776178666.337357:0:36:0:(import.c:581:import_select_connection()) lustre-MDT0000-lwp-OST0000: tried all connections, increasing latency to 21s 00000100:00080000:3.0:1776178666.337360:0:36:0:(import.c:615:import_select_connection()) lustre-MDT0000-lwp-OST0000: import 0000000047601a98 using connection 192.168.204.103@tcp/0@lo 00000100:00100000:3.0:1776178666.337474:0:36:0:(import.c:807:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5) req@000000009015f30b x1862456998112960/t0(0) o38->lustre-MDT0000-lwp-OST0000@0@lo:12/10 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' 00000100:00080000:3.0:1776178666.337516:0:36:0:(pinger.c:251:ptlrpc_pinger_process_import()) lustre-MDT0000-lwp-OST0001_UUID->lustre-MDT0000_UUID: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00080000:3.0:1776178666.337519:0:36:0:(recover.c:57:ptlrpc_initiate_recovery()) lustre-MDT0000_UUID: starting recovery 00000100:00080000:3.0:1776178666.337521:0:36:0:(import.c:86:import_set_state_nolock()) 00000000dfc8db13 lustre-MDT0000_UUID: changing import state from DISCONN to CONNECTING 00000100:00080000:3.0:1776178666.337524:0:36:0:(import.c:534:import_select_connection()) lustre-MDT0000-lwp-OST0001: connect to NID 0@lo last attempt 1368 00000100:00080000:3.0:1776178666.337526:0:36:0:(import.c:581:import_select_connection()) lustre-MDT0000-lwp-OST0001: tried all connections, increasing latency to 21s 00000100:00080000:3.0:1776178666.337530:0:36:0:(import.c:615:import_select_connection()) lustre-MDT0000-lwp-OST0001: import 00000000dfc8db13 using connection 192.168.204.103@tcp/0@lo 00000100:00100000:3.0:1776178666.337541:0:36:0:(import.c:807:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5) req@0000000009435d5b x1862456998113088/t0(0) o38->lustre-MDT0000-lwp-OST0001@0@lo:12/10 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' 00000100:00080000:3.0:1776178666.337564:0:36:0:(pinger.c:251:ptlrpc_pinger_process_import()) lustre-MDT0001-mdtlov_UUID->lustre-MDT0000_UUID: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00080000:3.0:1776178666.337567:0:36:0:(recover.c:57:ptlrpc_initiate_recovery()) lustre-MDT0000_UUID: starting recovery 00000100:00080000:3.0:1776178666.337569:0:36:0:(import.c:86:import_set_state_nolock()) 00000000ba4f1821 lustre-MDT0000_UUID: changing import state from DISCONN to CONNECTING 00000100:00080000:3.0:1776178666.337572:0:36:0:(import.c:534:import_select_connection()) lustre-MDT0000-osp-MDT0001: connect to NID 0@lo last attempt 1368 00000100:00080000:3.0:1776178666.337573:0:36:0:(import.c:581:import_select_connection()) lustre-MDT0000-osp-MDT0001: tried all connections, increasing latency to 21s 00000100:00080000:3.0:1776178666.337578:0:36:0:(import.c:615:import_select_connection()) lustre-MDT0000-osp-MDT0001: import 00000000ba4f1821 using connection 192.168.204.103@tcp/0@lo 00000100:00100000:3.0:1776178666.337587:0:36:0:(import.c:807:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5) req@00000000964f7a65 x1862456998113216/t0(0) o38->lustre-MDT0000-osp-MDT0001@0@lo:24/4 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' 00000100:00080000:3.0:1776178666.337604:0:36:0:(pinger.c:251:ptlrpc_pinger_process_import()) lustre-MDT0000-lwp-MDT0001_UUID->lustre-MDT0000_UUID: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00080000:3.0:1776178666.337607:0:36:0:(recover.c:57:ptlrpc_initiate_recovery()) lustre-MDT0000_UUID: starting recovery 00000100:00080000:3.0:1776178666.337608:0:36:0:(import.c:86:import_set_state_nolock()) 00000000a7c55b1f lustre-MDT0000_UUID: changing import state from DISCONN to CONNECTING 00000100:00080000:3.0:1776178666.337610:0:36:0:(import.c:534:import_select_connection()) lustre-MDT0000-lwp-MDT0001: connect to NID 0@lo last attempt 1368 00000100:00080000:3.0:1776178666.337612:0:36:0:(import.c:581:import_select_connection()) lustre-MDT0000-lwp-MDT0001: tried all connections, increasing latency to 21s 00000100:00080000:3.0:1776178666.337615:0:36:0:(import.c:615:import_select_connection()) lustre-MDT0000-lwp-MDT0001: import 00000000a7c55b1f using connection 192.168.204.103@tcp/0@lo 00000100:00100000:3.0:1776178666.337622:0:36:0:(import.c:807:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5) req@0000000082764007 x1862456998113408/t0(0) o38->lustre-MDT0000-lwp-MDT0001@0@lo:12/10 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' 00000100:00100000:3.0:1776178666.337643:0:3354:0:(client.c:1733:ptlrpc_send_new_req()) Sending RPC req@0000000086665d0f pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_01:lustre-MDT0001-mdtlov_UUID:3354:1862456998113280:0@lo:400:kworker/u8:1.0 00000100:00100000:3.0:1776178666.337675:0:3354:0:(events.c:361:request_in_callback()) peer: 12345-0@lo (source: 12345-0@lo) 00000100:00100000:3.0:1776178666.337693:0:3354:0:(ptlrpcd.c:415:ptlrpcd_check()) transfer 1 async RPCs [0->1] 00000100:00100000:3.0:1776178666.337698:0:3354:0:(client.c:1733:ptlrpc_send_new_req()) Sending RPC req@00000000be147540 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_01:lustre-MDT0001-lwp-OST0001_UUID:3354:1862456998113152:0@lo:400:kworker/u8:1.0 00000100:00100000:3.0:1776178666.337723:0:3354:0:(events.c:361:request_in_callback()) peer: 12345-0@lo (source: 12345-0@lo) 00000100:00100000:1.0:1776178666.337799:0:6256:0:(service.c:2152:ptlrpc_server_handle_req_in()) got req x1862456998113152 00000100:00100000:1.0:1776178666.337834:0:6256:0:(nrs_fifo.c:179:nrs_fifo_req_get()) NRS start fifo request from 12345-0@lo, seq: 1199 00000100:00100000:1.0:1776178666.337839:0:6256:0:(service.c:2324:ptlrpc_server_handle_request()) Handling RPC req@0000000098682d9e pname:cluuid+ref:pid:xid:nid:opc:job mdt00_001:lustre-MDT0001-lwp-OST0001_UUID+5:3354:x1862456998113152:12345-0@lo:400:kworker/u8:1.0 00000100:00100000:1.0:1776178666.337878:0:6256:0:(service.c:2372:ptlrpc_server_handle_request()) Handled RPC req@0000000098682d9e pname:cluuid+ref:pid:xid:nid:opc:job mdt00_001:lustre-MDT0001-lwp-OST0001_UUID+5:3354:x1862456998113152:12345-0@lo:400:kworker/u8:1.0 Request processed in 38us (154us total) trans 0 rc 0/0 00000100:00100000:1.0:1776178666.337883:0:6256:0:(nrs_fifo.c:241:nrs_fifo_req_stop()) NRS stop fifo request from 12345-0@lo, seq: 1199 00000100:00100000:1.0:1776178666.337924:0:3354:0:(client.c:2215:ptlrpc_check_set()) Completed RPC req@00000000be147540 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_01:lustre-MDT0001-lwp-OST0001_UUID:3354:1862456998113152:0@lo:400:kworker/u8:1.0 00000100:00100000:1.0:1776178666.337939:0:3356:0:(client.c:1733:ptlrpc_send_new_req()) Sending RPC req@00000000faeeda53 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_03:lustre-MDT0001-lwp-OST0000_UUID:3356:1862456998113024:0@lo:400:kworker/u8:1.0 00000100:00100000:1.0:1776178666.337961:0:3356:0:(events.c:361:request_in_callback()) peer: 12345-0@lo (source: 12345-0@lo) 00000100:00100000:1.0:1776178666.337974:0:3356:0:(ptlrpcd.c:415:ptlrpcd_check()) transfer 1 async RPCs [2->3] 00000100:00100000:1.0:1776178666.337978:0:3356:0:(client.c:1733:ptlrpc_send_new_req()) Sending RPC req@0000000092e0e13d pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_03:lustre-MDT0001-mdtlov_UUID:3356:1862456998113344:0@lo:400:kworker/u8:1.0 00000100:00100000:1.0:1776178666.337996:0:3356:0:(events.c:361:request_in_callback()) peer: 12345-0@lo (source: 12345-0@lo) 00000100:00100000:1.0:1776178666.338020:0:20759:0:(service.c:2152:ptlrpc_server_handle_req_in()) got req x1862456998113280 00000100:00100000:1.0:1776178666.338041:0:20759:0:(service.c:2152:ptlrpc_server_handle_req_in()) got req x1862456998113344 00000100:00100000:1.0:1776178666.338055:0:20759:0:(nrs_fifo.c:179:nrs_fifo_req_get()) NRS start fifo request from 12345-0@lo, seq: 873 00000100:00100000:1.0:1776178666.338077:0:20759:0:(service.c:2324:ptlrpc_server_handle_request()) Handling RPC req@0000000037b0061a pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_010:lustre-MDT0001-mdtlov_UUID+5:3354:x1862456998113280:12345-0@lo:400:kworker/u8:1.0 00000100:00100000:1.0:1776178666.338110:0:20759:0:(service.c:2372:ptlrpc_server_handle_request()) Handled RPC req@0000000037b0061a pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_010:lustre-MDT0001-mdtlov_UUID+5:3354:x1862456998113280:12345-0@lo:400:kworker/u8:1.0 Request processed in 50us (435us total) trans 0 rc 0/0 00000100:00100000:1.0:1776178666.338116:0:20759:0:(nrs_fifo.c:241:nrs_fifo_req_stop()) NRS stop fifo request from 12345-0@lo, seq: 873 00000100:00100000:1.0:1776178666.338126:0:20759:0:(nrs_fifo.c:179:nrs_fifo_req_get()) NRS start fifo request from 12345-0@lo, seq: 874 00000100:00100000:1.0:1776178666.338130:0:20759:0:(service.c:2324:ptlrpc_server_handle_request()) Handling RPC req@0000000068adf51c pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_010:lustre-MDT0001-mdtlov_UUID+5:3356:x1862456998113344:12345-0@lo:400:kworker/u8:1.0 00000100:00100000:1.0:1776178666.338155:0:20759:0:(service.c:2372:ptlrpc_server_handle_request()) Handled RPC req@0000000068adf51c pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_010:lustre-MDT0001-mdtlov_UUID+5:3356:x1862456998113344:12345-0@lo:400:kworker/u8:1.0 Request processed in 24us (158us total) trans 0 rc 0/0 00000100:00100000:1.0:1776178666.338160:0:20759:0:(nrs_fifo.c:241:nrs_fifo_req_stop()) NRS stop fifo request from 12345-0@lo, seq: 874 00000100:00100000:1.0:1776178666.338175:0:6256:0:(service.c:2152:ptlrpc_server_handle_req_in()) got req x1862456998113024 00000100:00100000:1.0:1776178666.338192:0:6256:0:(nrs_fifo.c:179:nrs_fifo_req_get()) NRS start fifo request from 12345-0@lo, seq: 1200 00000100:00100000:1.0:1776178666.338195:0:6256:0:(service.c:2324:ptlrpc_server_handle_request()) Handling RPC req@000000005fdea934 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_001:lustre-MDT0001-lwp-OST0000_UUID+5:3356:x1862456998113024:12345-0@lo:400:kworker/u8:1.0 00000100:00100000:1.0:1776178666.338232:0:6256:0:(service.c:2372:ptlrpc_server_handle_request()) Handled RPC req@000000005fdea934 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_001:lustre-MDT0001-lwp-OST0000_UUID+5:3356:x1862456998113024:12345-0@lo:400:kworker/u8:1.0 Request processed in 35us (271us total) trans 0 rc 0/0 00000100:00100000:1.0:1776178666.338237:0:6256:0:(nrs_fifo.c:241:nrs_fifo_req_stop()) NRS stop fifo request from 12345-0@lo, seq: 1200 00000100:00100000:1.0:1776178666.338260:0:3354:0:(client.c:2215:ptlrpc_check_set()) Completed RPC req@0000000086665d0f pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_01:lustre-MDT0001-mdtlov_UUID:3354:1862456998113280:0@lo:400:kworker/u8:1.0 00000100:00100000:1.0:1776178666.338280:0:3356:0:(client.c:2215:ptlrpc_check_set()) Completed RPC req@0000000092e0e13d pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_03:lustre-MDT0001-mdtlov_UUID:3356:1862456998113344:0@lo:400:kworker/u8:1.0 00000100:00100000:1.0:1776178666.338288:0:3356:0:(client.c:2215:ptlrpc_check_set()) Completed RPC req@00000000faeeda53 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_03:lustre-MDT0001-lwp-OST0000_UUID:3356:1862456998113024:0@lo:400:kworker/u8:1.0 00000100:00100000:2.0:1776178666.338684:0:3352:0:(client.c:1733:ptlrpc_send_new_req()) Sending RPC req@000000009015f30b pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:lustre-MDT0000-lwp-OST0000_UUID:3352:1862456998112960:0@lo:38:kworker/u8:1.0 00000100:00100000:2.0:1776178666.338715:0:3352:0:(events.c:361:request_in_callback()) peer: 12345-0@lo (source: 12345-0@lo) 00000100:00100000:2.0:1776178666.338737:0:3352:0:(client.c:1733:ptlrpc_send_new_req()) Sending RPC req@0000000009435d5b pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:lustre-MDT0000-lwp-OST0001_UUID:3352:1862456998113088:0@lo:38:kworker/u8:1.0 00000100:00100000:2.0:1776178666.338758:0:3352:0:(events.c:361:request_in_callback()) peer: 12345-0@lo (source: 12345-0@lo) 00000100:00100000:2.0:1776178666.338777:0:3352:0:(client.c:1733:ptlrpc_send_new_req()) Sending RPC req@00000000964f7a65 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:lustre-MDT0001-mdtlov_UUID:3352:1862456998113216:0@lo:38:kworker/u8:1.0 00000100:00100000:2.0:1776178666.338798:0:3352:0:(events.c:361:request_in_callback()) peer: 12345-0@lo (source: 12345-0@lo) 00000100:00100000:2.0:1776178666.338820:0:3352:0:(client.c:1733:ptlrpc_send_new_req()) Sending RPC req@0000000082764007 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:lustre-MDT0000-lwp-MDT0001_UUID:3352:1862456998113408:0@lo:38:kworker/u8:1.0 00000100:00100000:2.0:1776178666.338840:0:3352:0:(events.c:361:request_in_callback()) peer: 12345-0@lo (source: 12345-0@lo) 00000100:00100000:2.0:1776178666.338873:0:6257:0:(service.c:2152:ptlrpc_server_handle_req_in()) got req x1862456998112960 00000100:00100000:2.0:1776178666.338902:0:6257:0:(service.c:2152:ptlrpc_server_handle_req_in()) got req x1862456998113088 00000100:00100000:2.0:1776178666.338911:0:6257:0:(service.c:2152:ptlrpc_server_handle_req_in()) got req x1862456998113408 00000100:00100000:2.0:1776178666.338919:0:6257:0:(nrs_fifo.c:179:nrs_fifo_req_get()) NRS start fifo request from 12345-0@lo, seq: 6985 00000100:00100000:2.0:1776178666.338923:0:6257:0:(service.c:2324:ptlrpc_server_handle_request()) Handling RPC req@0000000089dbd0da pname:cluuid+ref:pid:xid:nid:opc:job mdt00_002:0+-99:3352:x1862456998112960:12345-0@lo:38:kworker/u8:1.0 00010000:02000000:2.0:1776178666.338935:0:6257:0:(ldlm_lib.c:1139:target_handle_connect()) lustre-MDT0000: Not available for connect from 0@lo (stopping) 00000100:00100000:2.0:1776178666.338966:0:6257:0:(service.c:2372:ptlrpc_server_handle_request()) Handled RPC req@0000000089dbd0da pname:cluuid+ref:pid:xid:nid:opc:job mdt00_002:0+-99:3352:x1862456998112960:12345-0@lo:38:kworker/u8:1.0 Request processed in 42us (251us total) trans 0 rc -19/-19 00000100:00100000:2.0:1776178666.338971:0:6257:0:(nrs_fifo.c:241:nrs_fifo_req_stop()) NRS stop fifo request from 12345-0@lo, seq: 6985 00000100:00100000:2.0:1776178666.338983:0:6257:0:(nrs_fifo.c:179:nrs_fifo_req_get()) NRS start fifo request from 12345-0@lo, seq: 6986 00000100:00100000:2.0:1776178666.338986:0:6257:0:(service.c:2324:ptlrpc_server_handle_request()) Handling RPC req@00000000542c8034 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_002:0+-99:3352:x1862456998113088:12345-0@lo:38:kworker/u8:1.0 00010000:02000000:2.0:1776178666.338992:0:6257:0:(ldlm_lib.c:1139:target_handle_connect()) lustre-MDT0000: Not available for connect from 0@lo (stopping) 00000100:00100000:2.0:1776178666.339010:0:6257:0:(service.c:2372:ptlrpc_server_handle_request()) Handled RPC req@00000000542c8034 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_002:0+-99:3352:x1862456998113088:12345-0@lo:38:kworker/u8:1.0 Request processed in 23us (252us total) trans 0 rc -19/-19 00000100:00100000:2.0:1776178666.339014:0:6257:0:(nrs_fifo.c:241:nrs_fifo_req_stop()) NRS stop fifo request from 12345-0@lo, seq: 6986 00000100:00100000:2.0:1776178666.339024:0:6257:0:(nrs_fifo.c:179:nrs_fifo_req_get()) NRS start fifo request from 12345-0@lo, seq: 6987 00000100:00100000:2.0:1776178666.339027:0:6257:0:(service.c:2324:ptlrpc_server_handle_request()) Handling RPC req@0000000014250674 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_002:0+-99:3352:x1862456998113408:12345-0@lo:38:kworker/u8:1.0 00010000:02000000:2.0:1776178666.339031:0:6257:0:(ldlm_lib.c:1139:target_handle_connect()) lustre-MDT0000: Not available for connect from 0@lo (stopping) 00000100:00100000:2.0:1776178666.339049:0:6257:0:(service.c:2372:ptlrpc_server_handle_request()) Handled RPC req@0000000014250674 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_002:0+-99:3352:x1862456998113408:12345-0@lo:38:kworker/u8:1.0 Request processed in 22us (209us total) trans 0 rc -19/-19 00000100:00100000:2.0:1776178666.339054:0:6257:0:(nrs_fifo.c:241:nrs_fifo_req_stop()) NRS stop fifo request from 12345-0@lo, seq: 6987 00000100:00080000:2.0:1776178666.339103:0:3352:0:(import.c:86:import_set_state_nolock()) 0000000047601a98 lustre-MDT0000_UUID: changing import state from CONNECTING to DISCONN 00000100:00080000:2.0:1776178666.339107:0:3352:0:(import.c:1429:ptlrpc_connect_interpret()) recovery of lustre-MDT0000_UUID on 192.168.204.103@tcp failed (-19) 00000100:00100000:2.0:1776178666.339110:0:3352:0:(client.c:2215:ptlrpc_check_set()) Completed RPC req@000000009015f30b pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:lustre-MDT0000-lwp-OST0000_UUID:3352:1862456998112960:0@lo:38:kworker/u8:1.0 00000100:00080000:2.0:1776178666.339117:0:3352:0:(import.c:86:import_set_state_nolock()) 00000000dfc8db13 lustre-MDT0000_UUID: changing import state from CONNECTING to DISCONN 00000100:00080000:2.0:1776178666.339119:0:3352:0:(import.c:1429:ptlrpc_connect_interpret()) recovery of lustre-MDT0000_UUID on 192.168.204.103@tcp failed (-19) 00000100:00100000:2.0:1776178666.339121:0:3352:0:(client.c:2215:ptlrpc_check_set()) Completed RPC req@0000000009435d5b pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:lustre-MDT0000-lwp-OST0001_UUID:3352:1862456998113088:0@lo:38:kworker/u8:1.0 00000100:00080000:2.0:1776178666.339127:0:3352:0:(import.c:86:import_set_state_nolock()) 00000000a7c55b1f lustre-MDT0000_UUID: changing import state from CONNECTING to DISCONN 00000100:00080000:2.0:1776178666.339129:0:3352:0:(import.c:1429:ptlrpc_connect_interpret()) recovery of lustre-MDT0000_UUID on 192.168.204.103@tcp failed (-19) 00000100:00100000:2.0:1776178666.339131:0:3352:0:(client.c:2215:ptlrpc_check_set()) Completed RPC req@0000000082764007 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:lustre-MDT0000-lwp-MDT0001_UUID:3352:1862456998113408:0@lo:38:kworker/u8:1.0 00000100:00100000:0.0:1776178666.339198:0:14128:0:(service.c:2152:ptlrpc_server_handle_req_in()) got req x1862456998113216 00000100:00100000:0.0:1776178666.339238:0:14128:0:(nrs_fifo.c:179:nrs_fifo_req_get()) NRS start fifo request from 12345-0@lo, seq: 798 00000100:00100000:0.0:1776178666.339241:0:14128:0:(service.c:2324:ptlrpc_server_handle_request()) Handling RPC req@000000001671af6a pname:cluuid+ref:pid:xid:nid:opc:job mdt_out00_002:0+-99:3352:x1862456998113216:12345-0@lo:38:kworker/u8:1.0 00010000:02000000:0.0:1776178666.339248:0:14128:0:(ldlm_lib.c:1139:target_handle_connect()) lustre-MDT0000: Not available for connect from 0@lo (stopping) 00000100:00100000:0.0:1776178666.339282:0:14128:0:(service.c:2372:ptlrpc_server_handle_request()) Handled RPC req@000000001671af6a pname:cluuid+ref:pid:xid:nid:opc:job mdt_out00_002:0+-99:3352:x1862456998113216:12345-0@lo:38:kworker/u8:1.0 Request processed in 39us (484us total) trans 0 rc -19/-19 00000100:00100000:0.0:1776178666.339288:0:14128:0:(nrs_fifo.c:241:nrs_fifo_req_stop()) NRS stop fifo request from 12345-0@lo, seq: 798 00000100:00080000:0.0:1776178666.339506:0:3352:0:(import.c:86:import_set_state_nolock()) 00000000ba4f1821 lustre-MDT0000_UUID: changing import state from CONNECTING to DISCONN 00000100:00080000:0.0:1776178666.339511:0:3352:0:(import.c:1429:ptlrpc_connect_interpret()) recovery of lustre-MDT0000_UUID on 192.168.204.103@tcp failed (-19) 00000100:00100000:0.0:1776178666.339514:0:3352:0:(client.c:2215:ptlrpc_check_set()) Completed RPC req@00000000964f7a65 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:lustre-MDT0001-mdtlov_UUID:3352:1862456998113216:0@lo:38:kworker/u8:1.0 00010000:00010000:1.0:1776178666.977054:0:9225:0:(ldlm_lockd.c:2792:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:0.0:1776178667.872338:0:3355:0:(ptlrpcd.c:415:ptlrpcd_check()) transfer 1 async RPCs [3->2] 00000100:00100000:0.0:1776178667.872361:0:3355:0:(client.c:1733:ptlrpc_send_new_req()) Sending RPC req@00000000715d380c pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_02:lustre-MDT0001-mdtlov_UUID:3355:1862456998113472:0@lo:13:osp-pre-1-1.0 00000100:00100000:0.0:1776178667.872429:0:3355:0:(events.c:361:request_in_callback()) peer: 12345-0@lo (source: 12345-0@lo) 00000100:00100000:2.0:1776178667.872552:0:9554:0:(service.c:2152:ptlrpc_server_handle_req_in()) got req x1862456998113472 00000100:00100000:2.0:1776178667.872604:0:9554:0:(nrs_fifo.c:179:nrs_fifo_req_get()) NRS start fifo request from 12345-0@lo, seq: 669 00000100:00100000:2.0:1776178667.872610:0:9554:0:(service.c:2324:ptlrpc_server_handle_request()) Handling RPC req@000000000d203d1d pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_create00:lustre-MDT0001-mdtlov_UUID+5:3355:x1862456998113472:12345-0@lo:13:osp-pre-1-1.0 00002000:00000024:2.0:1776178667.872639:0:9554:0:(ofd_obd.c:740:ofd_statfs()) blocks cached 229376 granted 491746368 pending 0 free 3923689472 avail 3692167168 00000100:00100000:2.0:1776178667.872690:0:9554:0:(service.c:2372:ptlrpc_server_handle_request()) Handled RPC req@000000000d203d1d pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_create00:lustre-MDT0001-mdtlov_UUID+5:3355:x1862456998113472:12345-0@lo:13:osp-pre-1-1.0 Request processed in 79us (261us total) trans 0 rc 0/0 00000100:00100000:2.0:1776178667.872697:0:9554:0:(nrs_fifo.c:241:nrs_fifo_req_stop()) NRS stop fifo request from 12345-0@lo, seq: 669 00000100:00100000:3.0:1776178667.872755:0:3355:0:(client.c:2215:ptlrpc_check_set()) Completed RPC req@00000000715d380c pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_02:lustre-MDT0001-mdtlov_UUID:3355:1862456998113472:0@lo:13:osp-pre-1-1.0 00010000:00010000:2.0:1776178668.000276:0:6244:0:(ldlm_lockd.c:2792:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:1.0:1776178668.384315:0:3354:0:(ptlrpcd.c:415:ptlrpcd_check()) transfer 1 async RPCs [0->1] 00000100:00100000:1.0:1776178668.384339:0:3354:0:(client.c:1733:ptlrpc_send_new_req()) Sending RPC req@0000000039f3bccf pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_01:lustre-MDT0001-mdtlov_UUID:3354:1862456998113536:0@lo:13:osp-pre-0-1.0 00000100:00100000:1.0:1776178668.384415:0:3354:0:(events.c:361:request_in_callback()) peer: 12345-0@lo (source: 12345-0@lo) 00000100:00100000:2.0:1776178668.384475:0:9554:0:(service.c:2152:ptlrpc_server_handle_req_in()) got req x1862456998113536 00000100:00100000:2.0:1776178668.384528:0:9554:0:(nrs_fifo.c:179:nrs_fifo_req_get()) NRS start fifo request from 12345-0@lo, seq: 670 00000100:00100000:2.0:1776178668.384534:0:9554:0:(service.c:2324:ptlrpc_server_handle_request()) Handling RPC req@000000001554c6b4 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_create00:lustre-MDT0001-mdtlov_UUID+5:3354:x1862456998113536:12345-0@lo:13:osp-pre-0-1.0 00002000:00000024:2.0:1776178668.384565:0:9554:0:(ofd_obd.c:740:ofd_statfs()) blocks cached 315392 granted 484883776 pending 0 free 3923689472 avail 3692167168 00000100:00100000:2.0:1776178668.384618:0:9554:0:(service.c:2372:ptlrpc_server_handle_request()) Handled RPC req@000000001554c6b4 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_create00:lustre-MDT0001-mdtlov_UUID+5:3354:x1862456998113536:12345-0@lo:13:osp-pre-0-1.0 Request processed in 83us (204us total) trans 0 rc 0/0 00000100:00100000:2.0:1776178668.384625:0:9554:0:(nrs_fifo.c:241:nrs_fifo_req_stop()) NRS stop fifo request from 12345-0@lo, seq: 670 00000100:00100000:1.0:1776178668.384769:0:3354:0:(client.c:2215:ptlrpc_check_set()) Completed RPC req@0000000039f3bccf pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_01:lustre-MDT0001-mdtlov_UUID:3354:1862456998113536:0@lo:13:osp-pre-0-1.0 00010000:00010000:1.0:1776178669.029917:0:6245:0:(ldlm_lockd.c:2792:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:3.0:1776178669.857771:0:3347:0:(events.c:361:request_in_callback()) peer: 12345-192.168.204.3@tcp (source: 12345-192.168.204.3@tcp) 00000100:00100000:3.0:1776178669.857831:0:3347:0:(events.c:361:request_in_callback()) peer: 12345-192.168.204.3@tcp (source: 12345-192.168.204.3@tcp) 00000100:00100000:3.0:1776178669.857873:0:3347:0:(events.c:361:request_in_callback()) peer: 12345-192.168.204.3@tcp (source: 12345-192.168.204.3@tcp) 00000100:00100000:3.0:1776178669.857916:0:3347:0:(events.c:361:request_in_callback()) peer: 12345-192.168.204.3@tcp (source: 12345-192.168.204.3@tcp) 00000100:00100000:2.0:1776178669.857924:0:20755:0:(service.c:2152:ptlrpc_server_handle_req_in()) got req x1862456979138368 00000100:00100000:3.0:1776178669.857954:0:3347:0:(events.c:361:request_in_callback()) peer: 12345-192.168.204.3@tcp (source: 12345-192.168.204.3@tcp) 00000100:00100000:0.0:1776178669.858210:0:20140:0:(service.c:2152:ptlrpc_server_handle_req_in()) got req x1862456979138112 00000100:00100000:0.0:1776178669.858251:0:20140:0:(nrs_fifo.c:179:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.204.3@tcp, seq: 337 00000100:00100000:0.0:1776178669.858259:0:20140:0:(service.c:2324:ptlrpc_server_handle_request()) Handling RPC req@00000000b8c10765 pname:cluuid+ref:pid:xid:nid:opc:job ll_mgs_0002:5b5c66b1-1a2f-4651-8548-0c75b9a38a94+9:2270:x1862456979138112:12345-192.168.204.3@tcp:400:kworker/u8:2.0 00000100:00100000:0.0:1776178669.858341:0:20140:0:(service.c:2372:ptlrpc_server_handle_request()) Handled RPC req@00000000b8c10765 pname:cluuid+ref:pid:xid:nid:opc:job ll_mgs_0002:5b5c66b1-1a2f-4651-8548-0c75b9a38a94+9:2270:x1862456979138112:12345-192.168.204.3@tcp:400:kworker/u8:2.0 Request processed in 82us (469us total) trans 0 rc 0/0 00000100:00100000:0.0:1776178669.858349:0:20140:0:(nrs_fifo.c:241:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.204.3@tcp, seq: 337 00000100:00100000:0.0:1776178669.858389:0:14440:0:(service.c:2152:ptlrpc_server_handle_req_in()) got req x1862456979138304 00000100:00100000:0.0:1776178669.858415:0:14440:0:(nrs_fifo.c:179:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.204.3@tcp, seq: 875 00000100:00100000:0.0:1776178669.858419:0:14440:0:(service.c:2324:ptlrpc_server_handle_request()) Handling RPC req@0000000070829000 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_004:b50963bd-8eba-4432-9a54-8888d00e3fa1+9:2270:x1862456979138368:12345-192.168.204.3@tcp:400:kworker/u8:2.0 00000100:00100000:0.0:1776178669.858448:0:14440:0:(service.c:2372:ptlrpc_server_handle_request()) Handled RPC req@0000000070829000 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_004:b50963bd-8eba-4432-9a54-8888d00e3fa1+9:2270:x1862456979138368:12345-192.168.204.3@tcp:400:kworker/u8:2.0 Request processed in 29us (627us total) trans 0 rc 0/0 00000100:00100000:0.0:1776178669.858455:0:14440:0:(nrs_fifo.c:241:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.204.3@tcp, seq: 875 00000100:00100000:0.0:1776178669.858470:0:14440:0:(nrs_fifo.c:179:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.204.3@tcp, seq: 876 00000100:00100000:0.0:1776178669.858474:0:14440:0:(service.c:2324:ptlrpc_server_handle_request()) Handling RPC req@00000000722afaa5 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_004:b50963bd-8eba-4432-9a54-8888d00e3fa1+8:2270:x1862456979138304:12345-192.168.204.3@tcp:400:kworker/u8:2.0 00000100:00100000:0.0:1776178669.858499:0:14440:0:(service.c:2372:ptlrpc_server_handle_request()) Handled RPC req@00000000722afaa5 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_004:b50963bd-8eba-4432-9a54-8888d00e3fa1+8:2270:x1862456979138304:12345-192.168.204.3@tcp:400:kworker/u8:2.0 Request processed in 25us (546us total) trans 0 rc 0/0 00000100:00100000:0.0:1776178669.858506:0:14440:0:(nrs_fifo.c:241:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.204.3@tcp, seq: 876 00000100:00100000:1.0:1776178669.858591:0:6256:0:(service.c:2152:ptlrpc_server_handle_req_in()) got req x1862456979138240 00000100:00100000:1.0:1776178669.858619:0:6256:0:(service.c:2152:ptlrpc_server_handle_req_in()) got req x1862456979138176 00000100:00100000:1.0:1776178669.858635:0:6256:0:(nrs_fifo.c:179:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.204.3@tcp, seq: 1201 00000100:00100000:1.0:1776178669.858652:0:6256:0:(service.c:2324:ptlrpc_server_handle_request()) Handling RPC req@00000000715d380c pname:cluuid+ref:pid:xid:nid:opc:job mdt00_001:b50963bd-8eba-4432-9a54-8888d00e3fa1+6:2270:x1862456979138240:12345-192.168.204.3@tcp:400:kworker/u8:2.0 00000100:00100000:1.0:1776178669.858681:0:6256:0:(service.c:2372:ptlrpc_server_handle_request()) Handled RPC req@00000000715d380c pname:cluuid+ref:pid:xid:nid:opc:job mdt00_001:b50963bd-8eba-4432-9a54-8888d00e3fa1+6:2270:x1862456979138240:12345-192.168.204.3@tcp:400:kworker/u8:2.0 Request processed in 29us (922us total) trans 0 rc 0/0 00000100:00100000:1.0:1776178669.858687:0:6256:0:(nrs_fifo.c:241:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.204.3@tcp, seq: 1201 00000100:00100000:1.0:1776178669.858699:0:6256:0:(nrs_fifo.c:179:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.204.3@tcp, seq: 6988 00000100:00100000:1.0:1776178669.858703:0:6256:0:(service.c:2324:ptlrpc_server_handle_request()) Handling RPC req@00000000598050cc pname:cluuid+ref:pid:xid:nid:opc:job mdt00_001:0+-99:2269:x1862456979138176:12345-192.168.204.3@tcp:38:kworker/u8:2.0 00010000:02000000:1.0:1776178669.858716:0:6256:0:(ldlm_lib.c:1139:target_handle_connect()) lustre-MDT0000: Not available for connect from 192.168.204.3@tcp (stopping) 00000100:00100000:1.0:1776178669.858740:0:6256:0:(service.c:2372:ptlrpc_server_handle_request()) Handled RPC req@00000000598050cc pname:cluuid+ref:pid:xid:nid:opc:job mdt00_001:0+-99:2269:x1862456979138176:12345-192.168.204.3@tcp:38:kworker/u8:2.0 Request processed in 36us (827us total) trans 0 rc -19/-19 00000100:00100000:1.0:1776178669.858746:0:6256:0:(nrs_fifo.c:241:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.204.3@tcp, seq: 6988 00010000:00010000:2.0:1776178670.051804:0:9224:0:(ldlm_lockd.c:2792:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00010000:00010000:1.0:1776178671.074428:0:9225:0:(ldlm_lockd.c:2792:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00080000:0.0:1776178671.461377:0:36:0:(pinger.c:251:ptlrpc_pinger_process_import()) lustre-MDT0000-lwp-OST0000_UUID->lustre-MDT0000_UUID: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00080000:0.0:1776178671.461384:0:36:0:(recover.c:57:ptlrpc_initiate_recovery()) lustre-MDT0000_UUID: starting recovery 00000100:00080000:0.0:1776178671.461388:0:36:0:(import.c:86:import_set_state_nolock()) 0000000047601a98 lustre-MDT0000_UUID: changing import state from DISCONN to CONNECTING 00000100:00080000:0.0:1776178671.461395:0:36:0:(import.c:534:import_select_connection()) lustre-MDT0000-lwp-OST0000: connect to NID 0@lo last attempt 1373 00000100:00080000:0.0:1776178671.461399:0:36:0:(import.c:581:import_select_connection()) lustre-MDT0000-lwp-OST0000: tried all connections, increasing latency to 21s 00000100:00080000:0.0:1776178671.461404:0:36:0:(import.c:615:import_select_connection()) lustre-MDT0000-lwp-OST0000: import 0000000047601a98 using connection 192.168.204.103@tcp/0@lo 00000100:00100000:0.0:1776178671.461423:0:36:0:(import.c:807:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5) req@00000000034781c2 x1862456998113664/t0(0) o38->lustre-MDT0000-lwp-OST0000@0@lo:12/10 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' 00000100:00080000:0.0:1776178671.461451:0:36:0:(pinger.c:251:ptlrpc_pinger_process_import()) lustre-MDT0000-lwp-OST0001_UUID->lustre-MDT0000_UUID: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00080000:0.0:1776178671.461454:0:36:0:(recover.c:57:ptlrpc_initiate_recovery()) lustre-MDT0000_UUID: starting recovery 00000100:00080000:0.0:1776178671.461455:0:36:0:(import.c:86:import_set_state_nolock()) 00000000dfc8db13 lustre-MDT0000_UUID: changing import state from DISCONN to CONNECTING 00000100:00080000:0.0:1776178671.461458:0:36:0:(import.c:534:import_select_connection()) lustre-MDT0000-lwp-OST0001: connect to NID 0@lo last attempt 1373 00000100:00080000:0.0:1776178671.461460:0:36:0:(import.c:581:import_select_connection()) lustre-MDT0000-lwp-OST0001: tried all connections, increasing latency to 21s 00000100:00080000:0.0:1776178671.461462:0:36:0:(import.c:615:import_select_connection()) lustre-MDT0000-lwp-OST0001: import 00000000dfc8db13 using connection 192.168.204.103@tcp/0@lo 00000100:00100000:0.0:1776178671.461470:0:36:0:(import.c:807:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5) req@00000000c237605d x1862456998113792/t0(0) o38->lustre-MDT0000-lwp-OST0001@0@lo:12/10 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' 00000100:00080000:0.0:1776178671.461481:0:36:0:(pinger.c:251:ptlrpc_pinger_process_import()) lustre-MDT0001-mdtlov_UUID->lustre-MDT0000_UUID: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00080000:0.0:1776178671.461484:0:36:0:(recover.c:57:ptlrpc_initiate_recovery()) lustre-MDT0000_UUID: starting recovery 00000100:00080000:0.0:1776178671.461485:0:36:0:(import.c:86:import_set_state_nolock()) 00000000ba4f1821 lustre-MDT0000_UUID: changing import state from DISCONN to CONNECTING 00000100:00080000:0.0:1776178671.461487:0:36:0:(import.c:534:import_select_connection()) lustre-MDT0000-osp-MDT0001: connect to NID 0@lo last attempt 1373 00000100:00080000:0.0:1776178671.461489:0:36:0:(import.c:581:import_select_connection()) lustre-MDT0000-osp-MDT0001: tried all connections, increasing latency to 21s 00000100:00080000:0.0:1776178671.461492:0:36:0:(import.c:615:import_select_connection()) lustre-MDT0000-osp-MDT0001: import 00000000ba4f1821 using connection 192.168.204.103@tcp/0@lo 00000100:00100000:0.0:1776178671.461502:0:36:0:(import.c:807:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5) req@0000000076452107 x1862456998113920/t0(0) o38->lustre-MDT0000-osp-MDT0001@0@lo:24/4 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' 00000100:00080000:0.0:1776178671.461515:0:36:0:(pinger.c:251:ptlrpc_pinger_process_import()) lustre-MDT0000-lwp-MDT0001_UUID->lustre-MDT0000_UUID: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00080000:0.0:1776178671.461518:0:36:0:(recover.c:57:ptlrpc_initiate_recovery()) lustre-MDT0000_UUID: starting recovery 00000100:00080000:0.0:1776178671.461519:0:36:0:(import.c:86:import_set_state_nolock()) 00000000a7c55b1f lustre-MDT0000_UUID: changing import state from DISCONN to CONNECTING 00000100:00080000:0.0:1776178671.461521:0:36:0:(import.c:534:import_select_connection()) lustre-MDT0000-lwp-MDT0001: connect to NID 0@lo last attempt 1373 00000100:00080000:0.0:1776178671.461523:0:36:0:(import.c:581:import_select_connection()) lustre-MDT0000-lwp-MDT0001: tried all connections, increasing latency to 21s 00000100:00080000:0.0:1776178671.461525:0:36:0:(import.c:615:import_select_connection()) lustre-MDT0000-lwp-MDT0001: import 00000000a7c55b1f using connection 192.168.204.103@tcp/0@lo 00000100:00100000:0.0:1776178671.461531:0:36:0:(import.c:807:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5) req@00000000a08c40f7 x1862456998114112/t0(0) o38->lustre-MDT0000-lwp-MDT0001@0@lo:12/10 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' 00000100:00100000:0.0:1776178671.461554:0:3353:0:(client.c:1733:ptlrpc_send_new_req()) Sending RPC req@0000000035819321 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_00:lustre-MDT0001-mdtlov_UUID:3353:1862456998113984:0@lo:400:kworker/u8:1.0 00000100:00100000:0.0:1776178671.461609:0:3353:0:(events.c:361:request_in_callback()) peer: 12345-0@lo (source: 12345-0@lo) 00000100:00100000:0.0:1776178671.461638:0:3353:0:(ptlrpcd.c:415:ptlrpcd_check()) transfer 2 async RPCs [1->0] 00000100:00100000:0.0:1776178671.461642:0:3353:0:(client.c:1733:ptlrpc_send_new_req()) Sending RPC req@00000000c37c187e pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_00:ac61f839-0e8e-4cac-9c59-08bcfbbad330:3353:1862456998113600:0@lo:400:kworker/u8:1.0 00000100:00100000:0.0:1776178671.461682:0:3353:0:(events.c:361:request_in_callback()) peer: 12345-0@lo (source: 12345-0@lo) 00000100:00100000:0.0:1776178671.461697:0:3353:0:(client.c:1733:ptlrpc_send_new_req()) Sending RPC req@0000000012b22334 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_00:lustre-MDT0001-mdtlov_UUID:3353:1862456998114048:0@lo:400:kworker/u8:1.0 00000100:00100000:0.0:1776178671.461718:0:3353:0:(events.c:361:request_in_callback()) peer: 12345-0@lo (source: 12345-0@lo) 00000100:00100000:0.0:1776178671.461745:0:3352:0:(client.c:1733:ptlrpc_send_new_req()) Sending RPC req@00000000034781c2 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:lustre-MDT0000-lwp-OST0000_UUID:3352:1862456998113664:0@lo:38:kworker/u8:1.0 00000100:00100000:0.0:1776178671.461770:0:3352:0:(events.c:361:request_in_callback()) peer: 12345-0@lo (source: 12345-0@lo) 00000100:00100000:0.0:1776178671.461783:0:3352:0:(client.c:1733:ptlrpc_send_new_req()) Sending RPC req@00000000c237605d pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:lustre-MDT0000-lwp-OST0001_UUID:3352:1862456998113792:0@lo:38:kworker/u8:1.0 00000100:00100000:0.0:1776178671.461814:0:3352:0:(events.c:361:request_in_callback()) peer: 12345-0@lo (source: 12345-0@lo) 00000100:00100000:0.0:1776178671.461827:0:3352:0:(client.c:1733:ptlrpc_send_new_req()) Sending RPC req@0000000076452107 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:lustre-MDT0001-mdtlov_UUID:3352:1862456998113920:0@lo:38:kworker/u8:1.0 00000100:00100000:0.0:1776178671.461849:0:3352:0:(events.c:361:request_in_callback()) peer: 12345-0@lo (source: 12345-0@lo) 00000100:00100000:0.0:1776178671.461860:0:3352:0:(client.c:1733:ptlrpc_send_new_req()) Sending RPC req@00000000a08c40f7 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:lustre-MDT0000-lwp-MDT0001_UUID:3352:1862456998114112:0@lo:38:kworker/u8:1.0 00000100:00100000:0.0:1776178671.461877:0:3352:0:(events.c:361:request_in_callback()) peer: 12345-0@lo (source: 12345-0@lo) 00000100:00100000:0.0:1776178671.461899:0:3355:0:(client.c:1733:ptlrpc_send_new_req()) Sending RPC req@0000000038b19922 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_02:lustre-MDT0001-lwp-OST0000_UUID:3355:1862456998113728:0@lo:400:kworker/u8:1.0 00000100:00100000:0.0:1776178671.461924:0:3355:0:(events.c:361:request_in_callback()) peer: 12345-0@lo (source: 12345-0@lo) 00000100:00100000:0.0:1776178671.461940:0:3355:0:(ptlrpcd.c:415:ptlrpcd_check()) transfer 1 async RPCs [3->2] 00000100:00100000:0.0:1776178671.461945:0:3355:0:(client.c:1733:ptlrpc_send_new_req()) Sending RPC req@00000000e82039bb pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_02:lustre-MDT0001-lwp-OST0001_UUID:3355:1862456998113856:0@lo:400:kworker/u8:1.0 00000100:00100000:0.0:1776178671.461965:0:3355:0:(events.c:361:request_in_callback()) peer: 12345-0@lo (source: 12345-0@lo) 00000100:00100000:0.0:1776178671.461995:0:20137:0:(service.c:2152:ptlrpc_server_handle_req_in()) got req x1862456998113600 00000100:00100000:0.0:1776178671.462035:0:20137:0:(nrs_fifo.c:179:nrs_fifo_req_get()) NRS start fifo request from 12345-0@lo, seq: 338 00000100:00100000:0.0:1776178671.462040:0:20137:0:(service.c:2324:ptlrpc_server_handle_request()) Handling RPC req@00000000b59e728a pname:cluuid+ref:pid:xid:nid:opc:job ll_mgs_0000:ac61f839-0e8e-4cac-9c59-08bcfbbad330+18:3353:x1862456998113600:12345-0@lo:400:kworker/u8:1.0 00000100:00100000:0.0:1776178671.462127:0:20137:0:(service.c:2372:ptlrpc_server_handle_request()) Handled RPC req@00000000b59e728a pname:cluuid+ref:pid:xid:nid:opc:job ll_mgs_0000:ac61f839-0e8e-4cac-9c59-08bcfbbad330+18:3353:x1862456998113600:12345-0@lo:400:kworker/u8:1.0 Request processed in 86us (445us total) trans 0 rc 0/0 00000100:00100000:0.0:1776178671.462135:0:20137:0:(nrs_fifo.c:241:nrs_fifo_req_stop()) NRS stop fifo request from 12345-0@lo, seq: 338 00000100:00100000:0.0:1776178671.462164:0:20762:0:(service.c:2152:ptlrpc_server_handle_req_in()) got req x1862456998113984 00000100:00100000:0.0:1776178671.462186:0:20762:0:(service.c:2152:ptlrpc_server_handle_req_in()) got req x1862456998114048 00000100:00100000:0.0:1776178671.462200:0:20762:0:(nrs_fifo.c:179:nrs_fifo_req_get()) NRS start fifo request from 12345-0@lo, seq: 877 00000100:00100000:1.0:1776178671.462202:0:6256:0:(service.c:2152:ptlrpc_server_handle_req_in()) got req x1862456998113664 00000100:00100000:0.0:1776178671.462204:0:20762:0:(service.c:2324:ptlrpc_server_handle_request()) Handling RPC req@00000000ea88a302 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_013:lustre-MDT0001-mdtlov_UUID+5:3353:x1862456998113984:12345-0@lo:400:kworker/u8:1.0 00000100:00100000:1.0:1776178671.462218:0:6256:0:(service.c:2152:ptlrpc_server_handle_req_in()) got req x1862456998113792 00000100:00100000:1.0:1776178671.462225:0:6256:0:(service.c:2152:ptlrpc_server_handle_req_in()) got req x1862456998114112 00000100:00100000:1.0:1776178671.462231:0:6256:0:(service.c:2152:ptlrpc_server_handle_req_in()) got req x1862456998113728 00000100:00100000:0.0:1776178671.462233:0:20762:0:(service.c:2372:ptlrpc_server_handle_request()) Handled RPC req@00000000ea88a302 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_013:lustre-MDT0001-mdtlov_UUID+5:3353:x1862456998113984:12345-0@lo:400:kworker/u8:1.0 Request processed in 29us (625us total) trans 0 rc 0/0 00000100:00100000:0.0:1776178671.462238:0:20762:0:(nrs_fifo.c:241:nrs_fifo_req_stop()) NRS stop fifo request from 12345-0@lo, seq: 877 00000100:00100000:1.0:1776178671.462243:0:6256:0:(service.c:2152:ptlrpc_server_handle_req_in()) got req x1862456998113856 00000100:00100000:0.0:1776178671.462247:0:20762:0:(nrs_fifo.c:179:nrs_fifo_req_get()) NRS start fifo request from 12345-0@lo, seq: 878 00000100:00100000:0.0:1776178671.462250:0:20762:0:(service.c:2324:ptlrpc_server_handle_request()) Handling RPC req@00000000fd2b8392 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_013:lustre-MDT0001-mdtlov_UUID+5:3353:x1862456998114048:12345-0@lo:400:kworker/u8:1.0 00000100:00100000:1.0:1776178671.462253:0:6256:0:(nrs_fifo.c:179:nrs_fifo_req_get()) NRS start fifo request from 12345-0@lo, seq: 1202 00000100:00100000:1.0:1776178671.462256:0:6256:0:(service.c:2324:ptlrpc_server_handle_request()) Handling RPC req@000000000a0451ff pname:cluuid+ref:pid:xid:nid:opc:job mdt00_001:lustre-MDT0001-lwp-OST0000_UUID+5:3355:x1862456998113728:12345-0@lo:400:kworker/u8:1.0 00000100:00100000:0.0:1776178671.462271:0:20762:0:(service.c:2372:ptlrpc_server_handle_request()) Handled RPC req@00000000fd2b8392 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_013:lustre-MDT0001-mdtlov_UUID+5:3353:x1862456998114048:12345-0@lo:400:kworker/u8:1.0 Request processed in 20us (553us total) trans 0 rc 0/0 00000100:00100000:0.0:1776178671.462277:0:20762:0:(nrs_fifo.c:241:nrs_fifo_req_stop()) NRS stop fifo request from 12345-0@lo, seq: 878 00000100:00100000:1.0:1776178671.462284:0:6256:0:(service.c:2372:ptlrpc_server_handle_request()) Handled RPC req@000000000a0451ff pname:cluuid+ref:pid:xid:nid:opc:job mdt00_001:lustre-MDT0001-lwp-OST0000_UUID+5:3355:x1862456998113728:12345-0@lo:400:kworker/u8:1.0 Request processed in 27us (360us total) trans 0 rc 0/0 00000100:00100000:1.0:1776178671.462290:0:6256:0:(nrs_fifo.c:241:nrs_fifo_req_stop()) NRS stop fifo request from 12345-0@lo, seq: 1202 00000100:00100000:1.0:1776178671.462300:0:6256:0:(nrs_fifo.c:179:nrs_fifo_req_get()) NRS start fifo request from 12345-0@lo, seq: 1203 00000100:00100000:1.0:1776178671.462303:0:6256:0:(service.c:2324:ptlrpc_server_handle_request()) Handling RPC req@00000000be2fd98d pname:cluuid+ref:pid:xid:nid:opc:job mdt00_001:lustre-MDT0001-lwp-OST0001_UUID+5:3355:x1862456998113856:12345-0@lo:400:kworker/u8:1.0 00000100:00100000:2.0:1776178671.462393:0:11281:0:(nrs_fifo.c:179:nrs_fifo_req_get()) NRS start fifo request from 12345-0@lo, seq: 6989 00000100:00100000:2.0:1776178671.462397:0:11281:0:(service.c:2324:ptlrpc_server_handle_request()) Handling RPC req@00000000e933e46b pname:cluuid+ref:pid:xid:nid:opc:job mdt00_004:0+-99:3352:x1862456998113664:12345-0@lo:38:kworker/u8:1.0 00010000:02000000:2.0:1776178671.462410:0:11281:0:(ldlm_lib.c:1139:target_handle_connect()) lustre-MDT0000: Not available for connect from 0@lo (stopping) 00000100:00100000:0.0:1776178671.463146:0:3353:0:(client.c:2215:ptlrpc_check_set()) Completed RPC req@00000000c37c187e pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_00:ac61f839-0e8e-4cac-9c59-08bcfbbad330:3353:1862456998113600:0@lo:400:kworker/u8:1.0 00000100:00100000:3.0:1776178671.467406:0:6257:0:(nrs_fifo.c:179:nrs_fifo_req_get()) NRS start fifo request from 12345-0@lo, seq: 6990 00000100:00100000:1.0:1776178671.469165:0:6256:0:(service.c:2372:ptlrpc_server_handle_request()) Handled RPC req@00000000be2fd98d pname:cluuid+ref:pid:xid:nid:opc:job mdt00_001:lustre-MDT0001-lwp-OST0001_UUID+5:3355:x1862456998113856:12345-0@lo:400:kworker/u8:1.0 Request processed in 6852us (7191us total) trans 0 rc 0/0 00000100:00100000:3.0:1776178671.469418:0:6257:0:(service.c:2324:ptlrpc_server_handle_request()) Handling RPC req@0000000079529182 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_002:0+-99:3352:x1862456998113792:12345-0@lo:38:kworker/u8:1.0 00010000:02000000:3.0:1776178671.469443:0:6257:0:(ldlm_lib.c:1139:target_handle_connect()) lustre-MDT0000: Not available for connect from 0@lo (stopping) 00000100:00100000:3.0:1776178671.469528:0:6257:0:(service.c:2372:ptlrpc_server_handle_request()) Handled RPC req@0000000079529182 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_002:0+-99:3352:x1862456998113792:12345-0@lo:38:kworker/u8:1.0 Request processed in 111us (7713us total) trans 0 rc -19/-19 00000100:00100000:2.0:1776178671.469606:0:11281:0:(service.c:2372:ptlrpc_server_handle_request()) Handled RPC req@00000000e933e46b pname:cluuid+ref:pid:xid:nid:opc:job mdt00_004:0+-99:3352:x1862456998113664:12345-0@lo:38:kworker/u8:1.0 Request processed in 7207us (7835us total) trans 0 rc -19/-19 00000100:00100000:1.0:1776178671.473072:0:6256:0:(nrs_fifo.c:241:nrs_fifo_req_stop()) NRS stop fifo request from 12345-0@lo, seq: 1203 00000100:00100000:1.0:1776178671.473263:0:8250:0:(nrs_fifo.c:179:nrs_fifo_req_get()) NRS start fifo request from 12345-0@lo, seq: 6991 00000100:00100000:1.0:1776178671.473270:0:8250:0:(service.c:2324:ptlrpc_server_handle_request()) Handling RPC req@0000000029ef3988 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_003:0+-99:3352:x1862456998114112:12345-0@lo:38:kworker/u8:1.0 00010000:02000000:1.0:1776178671.473289:0:8250:0:(ldlm_lib.c:1139:target_handle_connect()) lustre-MDT0000: Not available for connect from 0@lo (stopping) 00000100:00100000:1.0:1776178671.473362:0:8250:0:(service.c:2372:ptlrpc_server_handle_request()) Handled RPC req@0000000029ef3988 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_003:0+-99:3352:x1862456998114112:12345-0@lo:38:kworker/u8:1.0 Request processed in 92us (11484us total) trans 0 rc -19/-19 00000100:00100000:1.0:1776178671.473368:0:8250:0:(nrs_fifo.c:241:nrs_fifo_req_stop()) NRS stop fifo request from 12345-0@lo, seq: 6991 00000100:00100000:1.0:1776178671.473449:0:3355:0:(client.c:2215:ptlrpc_check_set()) Completed RPC req@00000000e82039bb pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_02:lustre-MDT0001-lwp-OST0001_UUID:3355:1862456998113856:0@lo:400:kworker/u8:1.0 00000100:00100000:1.0:1776178671.473462:0:3355:0:(client.c:2215:ptlrpc_check_set()) Completed RPC req@0000000038b19922 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_02:lustre-MDT0001-lwp-OST0000_UUID:3355:1862456998113728:0@lo:400:kworker/u8:1.0 00000100:00080000:1.0:1776178671.473523:0:3352:0:(import.c:86:import_set_state_nolock()) 0000000047601a98 lustre-MDT0000_UUID: changing import state from CONNECTING to DISCONN 00000100:00080000:1.0:1776178671.473528:0:3352:0:(import.c:1429:ptlrpc_connect_interpret()) recovery of lustre-MDT0000_UUID on 192.168.204.103@tcp failed (-19) 00000100:00100000:1.0:1776178671.473531:0:3352:0:(client.c:2215:ptlrpc_check_set()) Completed RPC req@00000000034781c2 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:lustre-MDT0000-lwp-OST0000_UUID:3352:1862456998113664:0@lo:38:kworker/u8:1.0 00000100:00080000:1.0:1776178671.473542:0:3352:0:(import.c:86:import_set_state_nolock()) 00000000dfc8db13 lustre-MDT0000_UUID: changing import state from CONNECTING to DISCONN 00000100:00080000:1.0:1776178671.473544:0:3352:0:(import.c:1429:ptlrpc_connect_interpret()) recovery of lustre-MDT0000_UUID on 192.168.204.103@tcp failed (-19) 00000100:00100000:1.0:1776178671.473546:0:3352:0:(client.c:2215:ptlrpc_check_set()) Completed RPC req@00000000c237605d pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:lustre-MDT0000-lwp-OST0001_UUID:3352:1862456998113792:0@lo:38:kworker/u8:1.0 00000100:00080000:1.0:1776178671.473553:0:3352:0:(import.c:86:import_set_state_nolock()) 00000000a7c55b1f lustre-MDT0000_UUID: changing import state from CONNECTING to DISCONN 00000100:00080000:1.0:1776178671.473556:0:3352:0:(import.c:1429:ptlrpc_connect_interpret()) recovery of lustre-MDT0000_UUID on 192.168.204.103@tcp failed (-19) 00000100:00100000:1.0:1776178671.473557:0:3352:0:(client.c:2215:ptlrpc_check_set()) Completed RPC req@00000000a08c40f7 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:lustre-MDT0000-lwp-MDT0001_UUID:3352:1862456998114112:0@lo:38:kworker/u8:1.0 00000100:00100000:0.0:1776178671.473606:0:3353:0:(client.c:2215:ptlrpc_check_set()) Completed RPC req@0000000012b22334 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_00:lustre-MDT0001-mdtlov_UUID:3353:1862456998114048:0@lo:400:kworker/u8:1.0 00000100:00100000:0.0:1776178671.473630:0:3353:0:(client.c:2215:ptlrpc_check_set()) Completed RPC req@0000000035819321 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_00:lustre-MDT0001-mdtlov_UUID:3353:1862456998113984:0@lo:400:kworker/u8:1.0 00000100:00100000:1.0:1776178671.473662:0:15213:0:(service.c:2152:ptlrpc_server_handle_req_in()) got req x1862456998113920 00000100:00100000:1.0:1776178671.473702:0:15213:0:(nrs_fifo.c:179:nrs_fifo_req_get()) NRS start fifo request from 12345-0@lo, seq: 799 00000100:00100000:1.0:1776178671.473705:0:15213:0:(service.c:2324:ptlrpc_server_handle_request()) Handling RPC req@00000000542ce52f pname:cluuid+ref:pid:xid:nid:opc:job mdt_out00_003:0+-99:3352:x1862456998113920:12345-0@lo:38:kworker/u8:1.0 00010000:02000000:1.0:1776178671.473714:0:15213:0:(ldlm_lib.c:1139:target_handle_connect()) lustre-MDT0000: Not available for connect from 0@lo (stopping) 00000100:00100000:1.0:1776178671.473747:0:15213:0:(service.c:2372:ptlrpc_server_handle_request()) Handled RPC req@00000000542ce52f pname:cluuid+ref:pid:xid:nid:opc:job mdt_out00_003:0+-99:3352:x1862456998113920:12345-0@lo:38:kworker/u8:1.0 Request processed in 41us (11897us total) trans 0 rc -19/-19 00000100:00100000:1.0:1776178671.473753:0:15213:0:(nrs_fifo.c:241:nrs_fifo_req_stop()) NRS stop fifo request from 12345-0@lo, seq: 799 00000100:00080000:1.0:1776178671.473775:0:3352:0:(import.c:86:import_set_state_nolock()) 00000000ba4f1821 lustre-MDT0000_UUID: changing import state from CONNECTING to DISCONN 00000100:00080000:1.0:1776178671.473799:0:3352:0:(import.c:1429:ptlrpc_connect_interpret()) recovery of lustre-MDT0000_UUID on 192.168.204.103@tcp failed (-19) 00000100:00100000:1.0:1776178671.473802:0:3352:0:(client.c:2215:ptlrpc_check_set()) Completed RPC req@0000000076452107 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:lustre-MDT0001-mdtlov_UUID:3352:1862456998113920:0@lo:38:kworker/u8:1.0 00000100:00100000:3.0:1776178671.478565:0:6257:0:(nrs_fifo.c:241:nrs_fifo_req_stop()) NRS stop fifo request from 12345-0@lo, seq: 6990 00000100:00100000:2.0:1776178671.483136:0:11281:0:(nrs_fifo.c:241:nrs_fifo_req_stop()) NRS stop fifo request from 12345-0@lo, seq: 6989 00010000:00010000:2.0:1776178672.096775:0:6244:0:(ldlm_lockd.c:2792:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:1.0:1776178672.992842:0:3355:0:(client.c:1733:ptlrpc_send_new_req()) Sending RPC req@000000007d1fa4e3 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_02:lustre-MDT0001-mdtlov_UUID:3355:1862456998114176:0@lo:13:osp-pre-1-1.0 00000100:00100000:1.0:1776178672.992914:0:3355:0:(events.c:361:request_in_callback()) peer: 12345-0@lo (source: 12345-0@lo) 00000100:00100000:2.0:1776178672.993054:0:8231:0:(service.c:2152:ptlrpc_server_handle_req_in()) got req x1862456998114176 00000100:00100000:2.0:1776178672.993148:0:8231:0:(nrs_fifo.c:179:nrs_fifo_req_get()) NRS start fifo request from 12345-0@lo, seq: 671 00000100:00100000:2.0:1776178672.993153:0:8231:0:(service.c:2324:ptlrpc_server_handle_request()) Handling RPC req@000000001f0b1867 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_create00:lustre-MDT0001-mdtlov_UUID+5:3355:x1862456998114176:12345-0@lo:13:osp-pre-1-1.0 00002000:00000024:2.0:1776178672.993187:0:8231:0:(ofd_obd.c:740:ofd_statfs()) blocks cached 229376 granted 491746368 pending 0 free 3923689472 avail 3692167168 00000100:00100000:2.0:1776178672.993243:0:8231:0:(service.c:2372:ptlrpc_server_handle_request()) Handled RPC req@000000001f0b1867 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_create00:lustre-MDT0001-mdtlov_UUID+5:3355:x1862456998114176:12345-0@lo:13:osp-pre-1-1.0 Request processed in 89us (330us total) trans 0 rc 0/0 00000100:00100000:2.0:1776178672.993251:0:8231:0:(nrs_fifo.c:241:nrs_fifo_req_stop()) NRS stop fifo request from 12345-0@lo, seq: 671 00000100:00100000:0.0:1776178672.993542:0:3355:0:(client.c:2215:ptlrpc_check_set()) Completed RPC req@000000007d1fa4e3 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_02:lustre-MDT0001-mdtlov_UUID:3355:1862456998114176:0@lo:13:osp-pre-1-1.0 00010000:00010000:1.0:1776178673.129342:0:6245:0:(ldlm_lockd.c:2792:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:1.0:1776178673.504419:0:3355:0:(ptlrpcd.c:415:ptlrpcd_check()) transfer 1 async RPCs [3->2] 00000100:00100000:1.0:1776178673.504444:0:3355:0:(client.c:1733:ptlrpc_send_new_req()) Sending RPC req@00000000952003a4 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_02:lustre-MDT0001-mdtlov_UUID:3355:1862456998114240:0@lo:13:osp-pre-0-1.0 00000100:00100000:1.0:1776178673.504522:0:3355:0:(events.c:361:request_in_callback()) peer: 12345-0@lo (source: 12345-0@lo) 00000100:00100000:2.0:1776178673.504677:0:9554:0:(service.c:2152:ptlrpc_server_handle_req_in()) got req x1862456998114240 00000100:00100000:2.0:1776178673.504743:0:9554:0:(nrs_fifo.c:179:nrs_fifo_req_get()) NRS start fifo request from 12345-0@lo, seq: 672 00000100:00100000:2.0:1776178673.504762:0:9554:0:(service.c:2324:ptlrpc_server_handle_request()) Handling RPC req@0000000053e142bc pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_create00:lustre-MDT0001-mdtlov_UUID+5:3355:x1862456998114240:12345-0@lo:13:osp-pre-0-1.0 00002000:00000024:2.0:1776178673.504794:0:9554:0:(ofd_obd.c:740:ofd_statfs()) blocks cached 315392 granted 484883776 pending 0 free 3923689472 avail 3692167168 00000100:00100000:2.0:1776178673.504845:0:9554:0:(service.c:2372:ptlrpc_server_handle_request()) Handled RPC req@0000000053e142bc pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_create00:lustre-MDT0001-mdtlov_UUID+5:3355:x1862456998114240:12345-0@lo:13:osp-pre-0-1.0 Request processed in 94us (325us total) trans 0 rc 0/0 00000100:00100000:2.0:1776178673.504852:0:9554:0:(nrs_fifo.c:241:nrs_fifo_req_stop()) NRS stop fifo request from 12345-0@lo, seq: 672 00000100:00100000:1.0:1776178673.504999:0:3355:0:(client.c:2215:ptlrpc_check_set()) Completed RPC req@00000000952003a4 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_02:lustre-MDT0001-mdtlov_UUID:3355:1862456998114240:0@lo:13:osp-pre-0-1.0 00010000:00010000:2.0:1776178674.144459:0:9224:0:(ldlm_lockd.c:2792:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:3.0:1776178674.980827:0:3348:0:(events.c:361:request_in_callback()) peer: 12345-192.168.204.3@tcp (source: 12345-192.168.204.3@tcp) 00000100:00100000:3.0:1776178674.980883:0:3348:0:(events.c:361:request_in_callback()) peer: 12345-192.168.204.3@tcp (source: 12345-192.168.204.3@tcp) 00000100:00100000:3.0:1776178674.980922:0:3348:0:(events.c:361:request_in_callback()) peer: 12345-192.168.204.3@tcp (source: 12345-192.168.204.3@tcp) 00000100:00100000:3.0:1776178674.980955:0:3348:0:(events.c:361:request_in_callback()) peer: 12345-192.168.204.3@tcp (source: 12345-192.168.204.3@tcp) 00000100:00100000:0.0:1776178674.980980:0:20140:0:(service.c:2152:ptlrpc_server_handle_req_in()) got req x1862456979138432 00000100:00100000:3.0:1776178674.980999:0:3348:0:(events.c:361:request_in_callback()) peer: 12345-192.168.204.3@tcp (source: 12345-192.168.204.3@tcp) 00000100:00100000:3.0:1776178674.981027:0:6257:0:(service.c:2152:ptlrpc_server_handle_req_in()) got req x1862456979138496 00000100:00100000:0.0:1776178674.981031:0:20140:0:(nrs_fifo.c:179:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.204.3@tcp, seq: 339 00000100:00100000:0.0:1776178674.981038:0:20140:0:(service.c:2324:ptlrpc_server_handle_request()) Handling RPC req@00000000dd6bfaa2 pname:cluuid+ref:pid:xid:nid:opc:job ll_mgs_0002:5b5c66b1-1a2f-4651-8548-0c75b9a38a94+9:2272:x1862456979138432:12345-192.168.204.3@tcp:400:kworker/u8:3.0 00000100:00100000:0.0:1776178674.981137:0:20140:0:(service.c:2372:ptlrpc_server_handle_request()) Handled RPC req@00000000dd6bfaa2 pname:cluuid+ref:pid:xid:nid:opc:job ll_mgs_0002:5b5c66b1-1a2f-4651-8548-0c75b9a38a94+9:2272:x1862456979138432:12345-192.168.204.3@tcp:400:kworker/u8:3.0 Request processed in 89us (245us total) trans 0 rc 0/0 00000100:00100000:0.0:1776178674.981212:0:20140:0:(nrs_fifo.c:241:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.204.3@tcp, seq: 339 00000100:00100000:1.0:1776178674.981782:0:20759:0:(service.c:2152:ptlrpc_server_handle_req_in()) got req x1862456979138624 00000100:00100000:1.0:1776178674.981823:0:20759:0:(service.c:2152:ptlrpc_server_handle_req_in()) got req x1862456979138688 00000100:00100000:1.0:1776178674.981843:0:20759:0:(nrs_fifo.c:179:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.204.3@tcp, seq: 879 00000100:00100000:1.0:1776178674.981848:0:20759:0:(service.c:2324:ptlrpc_server_handle_request()) Handling RPC req@000000007d1fa4e3 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_010:b50963bd-8eba-4432-9a54-8888d00e3fa1+8:2272:x1862456979138624:12345-192.168.204.3@tcp:400:kworker/u8:3.0 00000100:00100000:1.0:1776178674.982097:0:20759:0:(service.c:2372:ptlrpc_server_handle_request()) Handled RPC req@000000007d1fa4e3 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_010:b50963bd-8eba-4432-9a54-8888d00e3fa1+8:2272:x1862456979138624:12345-192.168.204.3@tcp:400:kworker/u8:3.0 Request processed in 247us (1175us total) trans 0 rc 0/0 00000100:00100000:1.0:1776178674.982105:0:20759:0:(nrs_fifo.c:241:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.204.3@tcp, seq: 879 00000100:00100000:1.0:1776178674.982119:0:20759:0:(nrs_fifo.c:179:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.204.3@tcp, seq: 880 00000100:00100000:1.0:1776178674.982123:0:20759:0:(service.c:2324:ptlrpc_server_handle_request()) Handling RPC req@00000000936848a2 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_010:b50963bd-8eba-4432-9a54-8888d00e3fa1+9:2272:x1862456979138688:12345-192.168.204.3@tcp:400:kworker/u8:3.0 00000100:00100000:1.0:1776178674.982349:0:20759:0:(service.c:2372:ptlrpc_server_handle_request()) Handled RPC req@00000000936848a2 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_010:b50963bd-8eba-4432-9a54-8888d00e3fa1+9:2272:x1862456979138688:12345-192.168.204.3@tcp:400:kworker/u8:3.0 Request processed in 226us (1351us total) trans 0 rc 0/0 00000100:00100000:1.0:1776178674.982356:0:20759:0:(nrs_fifo.c:241:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.204.3@tcp, seq: 880 00000100:00100000:3.0:1776178674.982701:0:6257:0:(service.c:2152:ptlrpc_server_handle_req_in()) got req x1862456979138560 00000100:00100000:3.0:1776178674.982745:0:6257:0:(nrs_fifo.c:179:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.204.3@tcp, seq: 1204 00000100:00100000:3.0:1776178674.982750:0:6257:0:(service.c:2324:ptlrpc_server_handle_request()) Handling RPC req@0000000082764007 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_002:b50963bd-8eba-4432-9a54-8888d00e3fa1+6:2272:x1862456979138560:12345-192.168.204.3@tcp:400:kworker/u8:3.0 00000100:00100000:3.0:1776178674.982789:0:6257:0:(service.c:2372:ptlrpc_server_handle_request()) Handled RPC req@0000000082764007 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_002:b50963bd-8eba-4432-9a54-8888d00e3fa1+6:2272:x1862456979138560:12345-192.168.204.3@tcp:400:kworker/u8:3.0 Request processed in 39us (1835us total) trans 0 rc 0/0 00000100:00100000:3.0:1776178674.982797:0:6257:0:(nrs_fifo.c:241:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.204.3@tcp, seq: 1204 00000100:00100000:3.0:1776178674.982809:0:6257:0:(nrs_fifo.c:179:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.204.3@tcp, seq: 6992 00000100:00100000:3.0:1776178674.982813:0:6257:0:(service.c:2324:ptlrpc_server_handle_request()) Handling RPC req@00000000ef4da5b8 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_002:0+-99:2269:x1862456979138496:12345-192.168.204.3@tcp:38:kworker/u8:3.0 00010000:02000000:3.0:1776178674.982826:0:6257:0:(ldlm_lib.c:1139:target_handle_connect()) lustre-MDT0000: Not available for connect from 192.168.204.3@tcp (stopping) 00000100:00100000:3.0:1776178674.982852:0:6257:0:(service.c:2372:ptlrpc_server_handle_request()) Handled RPC req@00000000ef4da5b8 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_002:0+-99:2269:x1862456979138496:12345-192.168.204.3@tcp:38:kworker/u8:3.0 Request processed in 37us (2033us total) trans 0 rc -19/-19 00000100:00100000:3.0:1776178674.982859:0:6257:0:(nrs_fifo.c:241:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.204.3@tcp, seq: 6992 00010000:00010000:1.0:1776178675.168325:0:9225:0:(ldlm_lockd.c:2792:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00010000:00010000:2.0:1776178676.192321:0:6244:0:(ldlm_lockd.c:2792:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:3.0:1776178676.576389:0:3354:0:(ptlrpcd.c:415:ptlrpcd_check()) transfer 1 async RPCs [0->1] 00000100:00080000:0.0:1776178676.578210:0:36:0:(pinger.c:251:ptlrpc_pinger_process_import()) lustre-MDT0000-lwp-OST0000_UUID->lustre-MDT0000_UUID: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00080000:0.0:1776178676.578220:0:36:0:(recover.c:57:ptlrpc_initiate_recovery()) lustre-MDT0000_UUID: starting recovery 00000100:00080000:0.0:1776178676.578224:0:36:0:(import.c:86:import_set_state_nolock()) 0000000047601a98 lustre-MDT0000_UUID: changing import state from DISCONN to CONNECTING 00000100:00080000:0.0:1776178676.578232:0:36:0:(import.c:534:import_select_connection()) lustre-MDT0000-lwp-OST0000: connect to NID 0@lo last attempt 1378 00000100:00080000:0.0:1776178676.578236:0:36:0:(import.c:581:import_select_connection()) lustre-MDT0000-lwp-OST0000: tried all connections, increasing latency to 21s 00000100:00080000:0.0:1776178676.578242:0:36:0:(import.c:615:import_select_connection()) lustre-MDT0000-lwp-OST0000: import 0000000047601a98 using connection 192.168.204.103@tcp/0@lo 00000100:00100000:0.0:1776178676.578278:0:36:0:(import.c:807:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5) req@0000000012b22334 x1862456998114368/t0(0) o38->lustre-MDT0000-lwp-OST0000@0@lo:12/10 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' 00000100:00100000:1.0:1776178676.578368:0:3352:0:(client.c:1733:ptlrpc_send_new_req()) Sending RPC req@0000000012b22334 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:lustre-MDT0000-lwp-OST0000_UUID:3352:1862456998114368:0@lo:38:kworker/u8:1.0 00000100:00100000:1.0:1776178676.578454:0:3352:0:(events.c:361:request_in_callback()) peer: 12345-0@lo (source: 12345-0@lo) 00000100:00100000:2.0:1776178676.578856:0:6255:0:(service.c:2152:ptlrpc_server_handle_req_in()) got req x1862456998114368 00000100:00100000:2.0:1776178676.578898:0:6255:0:(nrs_fifo.c:179:nrs_fifo_req_get()) NRS start fifo request from 12345-0@lo, seq: 6993 00000100:00100000:2.0:1776178676.578903:0:6255:0:(service.c:2324:ptlrpc_server_handle_request()) Handling RPC req@00000000bfb13eb6 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_000:0+-99:3352:x1862456998114368:12345-0@lo:38:kworker/u8:1.0 00010000:02000000:2.0:1776178676.578916:0:6255:0:(ldlm_lib.c:1139:target_handle_connect()) lustre-MDT0000: Not available for connect from 0@lo (stopping) 00000100:00100000:2.0:1776178676.578956:0:6255:0:(service.c:2372:ptlrpc_server_handle_request()) Handled RPC req@00000000bfb13eb6 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_000:0+-99:3352:x1862456998114368:12345-0@lo:38:kworker/u8:1.0 Request processed in 52us (512us total) trans 0 rc -19/-19 00000100:00100000:2.0:1776178676.578963:0:6255:0:(nrs_fifo.c:241:nrs_fifo_req_stop()) NRS stop fifo request from 12345-0@lo, seq: 6993 00000100:00080000:1.0:1776178676.579043:0:3352:0:(import.c:86:import_set_state_nolock()) 0000000047601a98 lustre-MDT0000_UUID: changing import state from CONNECTING to DISCONN 00000100:00080000:1.0:1776178676.579047:0:3352:0:(import.c:1429:ptlrpc_connect_interpret()) recovery of lustre-MDT0000_UUID on 192.168.204.103@tcp failed (-19) 00000100:00100000:1.0:1776178676.579051:0:3352:0:(client.c:2215:ptlrpc_check_set()) Completed RPC req@0000000012b22334 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:lustre-MDT0000-lwp-OST0000_UUID:3352:1862456998114368:0@lo:38:kworker/u8:1.0 00000100:00100000:3.0:1776178676.583095:0:3354:0:(client.c:1733:ptlrpc_send_new_req()) Sending RPC req@000000004828c19d pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_01:lustre-MDT0001-lwp-OST0000_UUID:3354:1862456998114432:0@lo:400:kworker/u8:1.0 00000100:00100000:3.0:1776178676.583175:0:3354:0:(events.c:361:request_in_callback()) peer: 12345-0@lo (source: 12345-0@lo) 00000100:00100000:3.0:1776178676.583221:0:3354:0:(client.c:1733:ptlrpc_send_new_req()) Sending RPC req@0000000035819321 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_01:ac61f839-0e8e-4cac-9c59-08bcfbbad330:3354:1862456998114304:0@lo:400:kworker/u8:1.0 00000100:00100000:3.0:1776178676.583249:0:3354:0:(events.c:361:request_in_callback()) peer: 12345-0@lo (source: 12345-0@lo) 00000100:00100000:2.0:1776178676.583531:0:20137:0:(service.c:2152:ptlrpc_server_handle_req_in()) got req x1862456998114304 00000100:00100000:2.0:1776178676.583592:0:20137:0:(nrs_fifo.c:179:nrs_fifo_req_get()) NRS start fifo request from 12345-0@lo, seq: 340 00000100:00100000:2.0:1776178676.583598:0:20137:0:(service.c:2324:ptlrpc_server_handle_request()) Handling RPC req@00000000ddaf121b pname:cluuid+ref:pid:xid:nid:opc:job ll_mgs_0000:ac61f839-0e8e-4cac-9c59-08bcfbbad330+18:3354:x1862456998114304:12345-0@lo:400:kworker/u8:1.0 00000100:00100000:2.0:1776178676.583659:0:20137:0:(service.c:2372:ptlrpc_server_handle_request()) Handled RPC req@00000000ddaf121b pname:cluuid+ref:pid:xid:nid:opc:job ll_mgs_0000:ac61f839-0e8e-4cac-9c59-08bcfbbad330+18:3354:x1862456998114304:12345-0@lo:400:kworker/u8:1.0 Request processed in 61us (410us total) trans 0 rc 0/0 00000100:00100000:2.0:1776178676.583667:0:20137:0:(nrs_fifo.c:241:nrs_fifo_req_stop()) NRS stop fifo request from 12345-0@lo, seq: 340 00000100:00100000:1.0:1776178676.584568:0:6256:0:(service.c:2152:ptlrpc_server_handle_req_in()) got req x1862456998114432 00000100:00100000:1.0:1776178676.584629:0:6256:0:(nrs_fifo.c:179:nrs_fifo_req_get()) NRS start fifo request from 12345-0@lo, seq: 1205 00000100:00100000:1.0:1776178676.584633:0:6256:0:(service.c:2324:ptlrpc_server_handle_request()) Handling RPC req@000000009cee1481 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_001:lustre-MDT0001-lwp-OST0000_UUID+5:3354:x1862456998114432:12345-0@lo:400:kworker/u8:1.0 00000100:00100000:1.0:1776178676.584682:0:6256:0:(service.c:2372:ptlrpc_server_handle_request()) Handled RPC req@000000009cee1481 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_001:lustre-MDT0001-lwp-OST0000_UUID+5:3354:x1862456998114432:12345-0@lo:400:kworker/u8:1.0 Request processed in 49us (1510us total) trans 0 rc 0/0 00000100:00100000:1.0:1776178676.584688:0:6256:0:(nrs_fifo.c:241:nrs_fifo_req_stop()) NRS stop fifo request from 12345-0@lo, seq: 1205 00000100:00100000:3.0:1776178676.587846:0:3354:0:(client.c:2215:ptlrpc_check_set()) Completed RPC req@000000004828c19d pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_01:lustre-MDT0001-lwp-OST0000_UUID:3354:1862456998114432:0@lo:400:kworker/u8:1.0 00000100:00100000:3.0:1776178676.587864:0:3354:0:(client.c:2215:ptlrpc_check_set()) Completed RPC req@0000000035819321 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_01:ac61f839-0e8e-4cac-9c59-08bcfbbad330:3354:1862456998114304:0@lo:400:kworker/u8:1.0 00000100:00080000:0.0:1776178676.590637:0:36:0:(pinger.c:251:ptlrpc_pinger_process_import()) lustre-MDT0000-lwp-OST0001_UUID->lustre-MDT0000_UUID: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00080000:0.0:1776178676.590649:0:36:0:(recover.c:57:ptlrpc_initiate_recovery()) lustre-MDT0000_UUID: starting recovery 00000100:00080000:0.0:1776178676.590653:0:36:0:(import.c:86:import_set_state_nolock()) 00000000dfc8db13 lustre-MDT0000_UUID: changing import state from DISCONN to CONNECTING 00000100:00080000:0.0:1776178676.590661:0:36:0:(import.c:534:import_select_connection()) lustre-MDT0000-lwp-OST0001: connect to NID 0@lo last attempt 1378 00000100:00080000:0.0:1776178676.590666:0:36:0:(import.c:581:import_select_connection()) lustre-MDT0000-lwp-OST0001: tried all connections, increasing latency to 21s 00000100:00080000:0.0:1776178676.590674:0:36:0:(import.c:615:import_select_connection()) lustre-MDT0000-lwp-OST0001: import 00000000dfc8db13 using connection 192.168.204.103@tcp/0@lo 00000100:00100000:0.0:1776178676.590977:0:36:0:(import.c:807:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5) req@000000004ce46e20 x1862456998114496/t0(0) o38->lustre-MDT0000-lwp-OST0001@0@lo:12/10 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' 00000100:00100000:2.0:1776178676.591181:0:3355:0:(client.c:1733:ptlrpc_send_new_req()) Sending RPC req@000000009c4a59b3 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_02:lustre-MDT0001-lwp-OST0001_UUID:3355:1862456998114560:0@lo:400:kworker/u8:1.0 00000100:00100000:2.0:1776178676.591244:0:3355:0:(events.c:361:request_in_callback()) peer: 12345-0@lo (source: 12345-0@lo) 00000100:00080000:0.0:1776178676.591301:0:36:0:(pinger.c:251:ptlrpc_pinger_process_import()) lustre-MDT0001-mdtlov_UUID->lustre-MDT0000_UUID: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00080000:0.0:1776178676.591305:0:36:0:(recover.c:57:ptlrpc_initiate_recovery()) lustre-MDT0000_UUID: starting recovery 00000100:00080000:0.0:1776178676.591307:0:36:0:(import.c:86:import_set_state_nolock()) 00000000ba4f1821 lustre-MDT0000_UUID: changing import state from DISCONN to CONNECTING 00000100:00080000:0.0:1776178676.591311:0:36:0:(import.c:534:import_select_connection()) lustre-MDT0000-osp-MDT0001: connect to NID 0@lo last attempt 1378 00000100:00080000:0.0:1776178676.591313:0:36:0:(import.c:581:import_select_connection()) lustre-MDT0000-osp-MDT0001: tried all connections, increasing latency to 21s 00000100:00080000:0.0:1776178676.591320:0:36:0:(import.c:615:import_select_connection()) lustre-MDT0000-osp-MDT0001: import 00000000ba4f1821 using connection 192.168.204.103@tcp/0@lo 00000100:00100000:0.0:1776178676.591337:0:36:0:(import.c:807:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5) req@0000000075189f40 x1862456998114624/t0(0) o38->lustre-MDT0000-osp-MDT0001@0@lo:24/4 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' 00000100:00100000:1.0:1776178676.591400:0:3352:0:(client.c:1733:ptlrpc_send_new_req()) Sending RPC req@000000004ce46e20 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:lustre-MDT0000-lwp-OST0001_UUID:3352:1862456998114496:0@lo:38:kworker/u8:1.0 00000100:00100000:1.0:1776178676.591429:0:3352:0:(events.c:361:request_in_callback()) peer: 12345-0@lo (source: 12345-0@lo) 00000100:00100000:1.0:1776178676.591445:0:3352:0:(client.c:1733:ptlrpc_send_new_req()) Sending RPC req@0000000075189f40 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:lustre-MDT0001-mdtlov_UUID:3352:1862456998114624:0@lo:38:kworker/u8:1.0 00000100:00100000:1.0:1776178676.591468:0:3352:0:(events.c:361:request_in_callback()) peer: 12345-0@lo (source: 12345-0@lo) 00000100:00100000:1.0:1776178676.591513:0:6256:0:(service.c:2152:ptlrpc_server_handle_req_in()) got req x1862456998114560 00000100:00100000:1.0:1776178676.591555:0:6256:0:(service.c:2152:ptlrpc_server_handle_req_in()) got req x1862456998114496 00000100:00100000:1.0:1776178676.591572:0:6256:0:(nrs_fifo.c:179:nrs_fifo_req_get()) NRS start fifo request from 12345-0@lo, seq: 1206 00000100:00100000:1.0:1776178676.591578:0:6256:0:(service.c:2324:ptlrpc_server_handle_request()) Handling RPC req@000000006342fd4b pname:cluuid+ref:pid:xid:nid:opc:job mdt00_001:lustre-MDT0001-lwp-OST0001_UUID+5:3355:x1862456998114560:12345-0@lo:400:kworker/u8:1.0 00000100:00100000:3.0:1776178676.591672:0:6255:0:(nrs_fifo.c:179:nrs_fifo_req_get()) NRS start fifo request from 12345-0@lo, seq: 6994 00000100:00100000:3.0:1776178676.591677:0:6255:0:(service.c:2324:ptlrpc_server_handle_request()) Handling RPC req@000000009126d8c6 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_000:0+-99:3352:x1862456998114496:12345-0@lo:38:kworker/u8:1.0 00010000:02000000:3.0:1776178676.591690:0:6255:0:(ldlm_lib.c:1139:target_handle_connect()) lustre-MDT0000: Not available for connect from 0@lo (stopping) 00000100:00100000:3.0:1776178676.591744:0:6255:0:(service.c:2372:ptlrpc_server_handle_request()) Handled RPC req@000000009126d8c6 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_000:0+-99:3352:x1862456998114496:12345-0@lo:38:kworker/u8:1.0 Request processed in 66us (314us total) trans 0 rc -19/-19 00000100:00100000:3.0:1776178676.591751:0:6255:0:(nrs_fifo.c:241:nrs_fifo_req_stop()) NRS stop fifo request from 12345-0@lo, seq: 6994 00000100:00100000:3.0:1776178676.591806:0:14128:0:(service.c:2152:ptlrpc_server_handle_req_in()) got req x1862456998114624 00000100:00100000:3.0:1776178676.591824:0:14128:0:(nrs_fifo.c:179:nrs_fifo_req_get()) NRS start fifo request from 12345-0@lo, seq: 800 00000100:00100000:3.0:1776178676.591827:0:14128:0:(service.c:2324:ptlrpc_server_handle_request()) Handling RPC req@00000000a23c68f9 pname:cluuid+ref:pid:xid:nid:opc:job mdt_out00_002:0+-99:3352:x1862456998114624:12345-0@lo:38:kworker/u8:1.0 00010000:02000000:3.0:1776178676.591832:0:14128:0:(ldlm_lib.c:1139:target_handle_connect()) lustre-MDT0000: Not available for connect from 0@lo (stopping) 00000100:00100000:3.0:1776178676.591853:0:14128:0:(service.c:2372:ptlrpc_server_handle_request()) Handled RPC req@00000000a23c68f9 pname:cluuid+ref:pid:xid:nid:opc:job mdt_out00_002:0+-99:3352:x1862456998114624:12345-0@lo:38:kworker/u8:1.0 Request processed in 25us (385us total) trans 0 rc -19/-19 00000100:00100000:3.0:1776178676.591857:0:14128:0:(nrs_fifo.c:241:nrs_fifo_req_stop()) NRS stop fifo request from 12345-0@lo, seq: 800 00000100:00100000:3.0:1776178676.591873:0:3356:0:(client.c:1733:ptlrpc_send_new_req()) Sending RPC req@0000000004dd3b9b pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_03:lustre-MDT0001-mdtlov_UUID:3356:1862456998114688:0@lo:400:kworker/u8:1.0 00000100:00100000:3.0:1776178676.591895:0:3356:0:(events.c:361:request_in_callback()) peer: 12345-0@lo (source: 12345-0@lo) 00000100:00100000:3.0:1776178676.591946:0:20755:0:(service.c:2152:ptlrpc_server_handle_req_in()) got req x1862456998114688 00000100:00100000:3.0:1776178676.591967:0:20755:0:(nrs_fifo.c:179:nrs_fifo_req_get()) NRS start fifo request from 12345-0@lo, seq: 881 00000100:00100000:3.0:1776178676.591970:0:20755:0:(service.c:2324:ptlrpc_server_handle_request()) Handling RPC req@00000000fd453e99 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_006:lustre-MDT0001-mdtlov_UUID+5:3356:x1862456998114688:12345-0@lo:400:kworker/u8:1.0 00000100:00100000:3.0:1776178676.591997:0:20755:0:(service.c:2372:ptlrpc_server_handle_request()) Handled RPC req@00000000fd453e99 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_006:lustre-MDT0001-mdtlov_UUID+5:3356:x1862456998114688:12345-0@lo:400:kworker/u8:1.0 Request processed in 26us (102us total) trans 0 rc 0/0 00000100:00100000:3.0:1776178676.592014:0:20755:0:(nrs_fifo.c:241:nrs_fifo_req_stop()) NRS stop fifo request from 12345-0@lo, seq: 881 00000100:00100000:3.0:1776178676.592043:0:3356:0:(client.c:2215:ptlrpc_check_set()) Completed RPC req@0000000004dd3b9b pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_03:lustre-MDT0001-mdtlov_UUID:3356:1862456998114688:0@lo:400:kworker/u8:1.0 00000100:00080000:3.0:1776178676.592096:0:3352:0:(import.c:86:import_set_state_nolock()) 00000000dfc8db13 lustre-MDT0000_UUID: changing import state from CONNECTING to DISCONN 00000100:00080000:3.0:1776178676.592100:0:3352:0:(import.c:1429:ptlrpc_connect_interpret()) recovery of lustre-MDT0000_UUID on 192.168.204.103@tcp failed (-19) 00000100:00100000:3.0:1776178676.592102:0:3352:0:(client.c:2215:ptlrpc_check_set()) Completed RPC req@000000004ce46e20 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:lustre-MDT0000-lwp-OST0001_UUID:3352:1862456998114496:0@lo:38:kworker/u8:1.0 00000100:00080000:3.0:1776178676.592108:0:3352:0:(import.c:86:import_set_state_nolock()) 00000000ba4f1821 lustre-MDT0000_UUID: changing import state from CONNECTING to DISCONN 00000100:00080000:3.0:1776178676.592110:0:3352:0:(import.c:1429:ptlrpc_connect_interpret()) recovery of lustre-MDT0000_UUID on 192.168.204.103@tcp failed (-19) 00000100:00100000:3.0:1776178676.592112:0:3352:0:(client.c:2215:ptlrpc_check_set()) Completed RPC req@0000000075189f40 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:lustre-MDT0001-mdtlov_UUID:3352:1862456998114624:0@lo:38:kworker/u8:1.0 00000100:00080000:0.0:1776178676.603940:0:36:0:(pinger.c:251:ptlrpc_pinger_process_import()) lustre-MDT0000-lwp-MDT0001_UUID->lustre-MDT0000_UUID: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00080000:0.0:1776178676.603948:0:36:0:(recover.c:57:ptlrpc_initiate_recovery()) lustre-MDT0000_UUID: starting recovery 00000100:00080000:0.0:1776178676.603951:0:36:0:(import.c:86:import_set_state_nolock()) 00000000a7c55b1f lustre-MDT0000_UUID: changing import state from DISCONN to CONNECTING 00000100:00080000:0.0:1776178676.603958:0:36:0:(import.c:534:import_select_connection()) lustre-MDT0000-lwp-MDT0001: connect to NID 0@lo last attempt 1378 00000100:00080000:0.0:1776178676.603962:0:36:0:(import.c:581:import_select_connection()) lustre-MDT0000-lwp-MDT0001: tried all connections, increasing latency to 21s 00000100:00080000:0.0:1776178676.603967:0:36:0:(import.c:615:import_select_connection()) lustre-MDT0000-lwp-MDT0001: import 00000000a7c55b1f using connection 192.168.204.103@tcp/0@lo 00000100:00100000:0.0:1776178676.603988:0:36:0:(import.c:807:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5) req@000000006a3d91b0 x1862456998114816/t0(0) o38->lustre-MDT0000-lwp-MDT0001@0@lo:12/10 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' 00000100:00100000:0.0:1776178676.604039:0:3354:0:(ptlrpcd.c:415:ptlrpcd_check()) transfer 1 async RPCs [0->1] 00000100:00100000:0.0:1776178676.604055:0:3354:0:(client.c:1733:ptlrpc_send_new_req()) Sending RPC req@000000002300ff10 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_01:lustre-MDT0001-mdtlov_UUID:3354:1862456998114752:0@lo:400:kworker/u8:1.0 00000100:00100000:0.0:1776178676.604130:0:3354:0:(events.c:361:request_in_callback()) peer: 12345-0@lo (source: 12345-0@lo) 00000100:00100000:0.0:1776178676.604168:0:3352:0:(client.c:1733:ptlrpc_send_new_req()) Sending RPC req@000000006a3d91b0 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:lustre-MDT0000-lwp-MDT0001_UUID:3352:1862456998114816:0@lo:38:kworker/u8:1.0 00000100:00100000:0.0:1776178676.604192:0:3352:0:(events.c:361:request_in_callback()) peer: 12345-0@lo (source: 12345-0@lo) 00000100:00100000:0.0:1776178676.604223:0:20762:0:(service.c:2152:ptlrpc_server_handle_req_in()) got req x1862456998114752 00000100:00100000:0.0:1776178676.604259:0:20762:0:(nrs_fifo.c:179:nrs_fifo_req_get()) NRS start fifo request from 12345-0@lo, seq: 882 00000100:00100000:0.0:1776178676.604265:0:20762:0:(service.c:2324:ptlrpc_server_handle_request()) Handling RPC req@0000000042de09e9 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_013:lustre-MDT0001-mdtlov_UUID+5:3354:x1862456998114752:12345-0@lo:400:kworker/u8:1.0 00000100:00100000:0.0:1776178676.604305:0:20762:0:(service.c:2372:ptlrpc_server_handle_request()) Handled RPC req@0000000042de09e9 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_013:lustre-MDT0001-mdtlov_UUID+5:3354:x1862456998114752:12345-0@lo:400:kworker/u8:1.0 Request processed in 40us (177us total) trans 0 rc 0/0 00000100:00100000:0.0:1776178676.604312:0:20762:0:(nrs_fifo.c:241:nrs_fifo_req_stop()) NRS stop fifo request from 12345-0@lo, seq: 882 00000100:00100000:0.0:1776178676.604334:0:11281:0:(service.c:2152:ptlrpc_server_handle_req_in()) got req x1862456998114816 00000100:00100000:0.0:1776178676.604350:0:11281:0:(nrs_fifo.c:179:nrs_fifo_req_get()) NRS start fifo request from 12345-0@lo, seq: 6995 00000100:00100000:0.0:1776178676.604353:0:11281:0:(service.c:2324:ptlrpc_server_handle_request()) Handling RPC req@0000000037aace32 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_004:0+-99:3352:x1862456998114816:12345-0@lo:38:kworker/u8:1.0 00010000:02000000:0.0:1776178676.604365:0:11281:0:(ldlm_lib.c:1139:target_handle_connect()) lustre-MDT0000: Not available for connect from 0@lo (stopping) 00000100:00100000:0.0:1776178676.604391:0:11281:0:(service.c:2372:ptlrpc_server_handle_request()) Handled RPC req@0000000037aace32 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_004:0+-99:3352:x1862456998114816:12345-0@lo:38:kworker/u8:1.0 Request processed in 37us (199us total) trans 0 rc -19/-19 00000100:00100000:0.0:1776178676.604396:0:11281:0:(nrs_fifo.c:241:nrs_fifo_req_stop()) NRS stop fifo request from 12345-0@lo, seq: 6995 00000100:00100000:0.0:1776178676.604428:0:3354:0:(client.c:2215:ptlrpc_check_set()) Completed RPC req@000000002300ff10 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_01:lustre-MDT0001-mdtlov_UUID:3354:1862456998114752:0@lo:400:kworker/u8:1.0 00000100:00080000:0.0:1776178676.604452:0:3352:0:(import.c:86:import_set_state_nolock()) 00000000a7c55b1f lustre-MDT0000_UUID: changing import state from CONNECTING to DISCONN 00000100:00080000:0.0:1776178676.604455:0:3352:0:(import.c:1429:ptlrpc_connect_interpret()) recovery of lustre-MDT0000_UUID on 192.168.204.103@tcp failed (-19) 00000100:00100000:0.0:1776178676.604458:0:3352:0:(client.c:2215:ptlrpc_check_set()) Completed RPC req@000000006a3d91b0 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:lustre-MDT0000-lwp-MDT0001_UUID:3352:1862456998114816:0@lo:38:kworker/u8:1.0 00000100:00100000:2.0:1776178676.610859:0:3355:0:(client.c:2215:ptlrpc_check_set()) Completed RPC req@000000009c4a59b3 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_02:lustre-MDT0001-lwp-OST0001_UUID:3355:1862456998114560:0@lo:400:kworker/u8:1.0 00000100:00100000:1.0:1776178676.614179:0:6256:0:(service.c:2372:ptlrpc_server_handle_request()) Handled RPC req@000000006342fd4b pname:cluuid+ref:pid:xid:nid:opc:job mdt00_001:lustre-MDT0001-lwp-OST0001_UUID+5:3355:x1862456998114560:12345-0@lo:400:kworker/u8:1.0 Request processed in 22591us (22926us total) trans 0 rc 0/0 00000100:00100000:1.0:1776178676.614196:0:6256:0:(nrs_fifo.c:241:nrs_fifo_req_stop()) NRS stop fifo request from 12345-0@lo, seq: 1206 00010000:00010000:1.0:1776178677.216302:0:6245:0:(ldlm_lockd.c:2792:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:2.0:1776178678.112494:0:3353:0:(ptlrpcd.c:415:ptlrpcd_check()) transfer 1 async RPCs [1->0] 00000100:00100000:2.0:1776178678.112526:0:3353:0:(client.c:1733:ptlrpc_send_new_req()) Sending RPC req@00000000d23b79a7 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_00:lustre-MDT0001-mdtlov_UUID:3353:1862456998114880:0@lo:13:osp-pre-1-1.0 00000100:00100000:2.0:1776178678.112605:0:3353:0:(events.c:361:request_in_callback()) peer: 12345-0@lo (source: 12345-0@lo) 00000100:00100000:1.0:1776178678.112849:0:8231:0:(service.c:2152:ptlrpc_server_handle_req_in()) got req x1862456998114880 00000100:00100000:1.0:1776178678.112898:0:8231:0:(nrs_fifo.c:179:nrs_fifo_req_get()) NRS start fifo request from 12345-0@lo, seq: 673 00000100:00100000:1.0:1776178678.112904:0:8231:0:(service.c:2324:ptlrpc_server_handle_request()) Handling RPC req@0000000028422d92 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_create00:lustre-MDT0001-mdtlov_UUID+5:3353:x1862456998114880:12345-0@lo:13:osp-pre-1-1.0 00002000:00000024:1.0:1776178678.112952:0:8231:0:(ofd_obd.c:740:ofd_statfs()) blocks cached 229376 granted 491746368 pending 0 free 3923689472 avail 3692167168 00000100:00100000:1.0:1776178678.113012:0:8231:0:(service.c:2372:ptlrpc_server_handle_request()) Handled RPC req@0000000028422d92 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_create00:lustre-MDT0001-mdtlov_UUID+5:3353:x1862456998114880:12345-0@lo:13:osp-pre-1-1.0 Request processed in 107us (407us total) trans 0 rc 0/0 00000100:00100000:1.0:1776178678.113019:0:8231:0:(nrs_fifo.c:241:nrs_fifo_req_stop()) NRS stop fifo request from 12345-0@lo, seq: 673 00000100:00100000:3.0:1776178678.114124:0:3353:0:(client.c:2215:ptlrpc_check_set()) Completed RPC req@00000000d23b79a7 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_00:lustre-MDT0001-mdtlov_UUID:3353:1862456998114880:0@lo:13:osp-pre-1-1.0 00010000:00010000:2.0:1776178678.241231:0:9224:0:(ldlm_lockd.c:2792:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:2.0:1776178678.624365:0:3355:0:(client.c:1733:ptlrpc_send_new_req()) Sending RPC req@00000000ef862c6d pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_02:lustre-MDT0001-mdtlov_UUID:3355:1862456998114944:0@lo:13:osp-pre-0-1.0 00000100:00100000:2.0:1776178678.624436:0:3355:0:(events.c:361:request_in_callback()) peer: 12345-0@lo (source: 12345-0@lo) 00000100:00100000:2.0:1776178678.624487:0:9554:0:(service.c:2152:ptlrpc_server_handle_req_in()) got req x1862456998114944 00000100:00100000:2.0:1776178678.624591:0:9554:0:(nrs_fifo.c:179:nrs_fifo_req_get()) NRS start fifo request from 12345-0@lo, seq: 674 00000100:00100000:2.0:1776178678.624598:0:9554:0:(service.c:2324:ptlrpc_server_handle_request()) Handling RPC req@00000000dd4b6b2f pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_create00:lustre-MDT0001-mdtlov_UUID+5:3355:x1862456998114944:12345-0@lo:13:osp-pre-0-1.0 00002000:00000024:2.0:1776178678.624629:0:9554:0:(ofd_obd.c:740:ofd_statfs()) blocks cached 315392 granted 484883776 pending 0 free 3923689472 avail 3692167168 00000100:00100000:2.0:1776178678.624687:0:9554:0:(service.c:2372:ptlrpc_server_handle_request()) Handled RPC req@00000000dd4b6b2f pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_create00:lustre-MDT0001-mdtlov_UUID+5:3355:x1862456998114944:12345-0@lo:13:osp-pre-0-1.0 Request processed in 89us (253us total) trans 0 rc 0/0 00000100:00100000:2.0:1776178678.624694:0:9554:0:(nrs_fifo.c:241:nrs_fifo_req_stop()) NRS stop fifo request from 12345-0@lo, seq: 674 00000100:00100000:3.0:1776178678.628111:0:3355:0:(client.c:2215:ptlrpc_check_set()) Completed RPC req@00000000ef862c6d pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_02:lustre-MDT0001-mdtlov_UUID:3355:1862456998114944:0@lo:13:osp-pre-0-1.0 00010000:00010000:1.0:1776178679.264596:0:9225:0:(ldlm_lockd.c:2792:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:3.0:1776178680.103872:0:3347:0:(events.c:361:request_in_callback()) peer: 12345-192.168.204.3@tcp (source: 12345-192.168.204.3@tcp) 00000100:00100000:3.0:1776178680.103924:0:3347:0:(events.c:361:request_in_callback()) peer: 12345-192.168.204.3@tcp (source: 12345-192.168.204.3@tcp) 00000100:00100000:3.0:1776178680.103965:0:3347:0:(events.c:361:request_in_callback()) peer: 12345-192.168.204.3@tcp (source: 12345-192.168.204.3@tcp) 00000100:00100000:2.0:1776178680.104019:0:6256:0:(service.c:2152:ptlrpc_server_handle_req_in()) got req x1862456979138816 00000100:00100000:2.0:1776178680.104104:0:6256:0:(service.c:2152:ptlrpc_server_handle_req_in()) got req x1862456979138880 00000100:00100000:2.0:1776178680.104144:0:6256:0:(nrs_fifo.c:179:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.204.3@tcp, seq: 1207 00000100:00100000:2.0:1776178680.104151:0:6256:0:(service.c:2324:ptlrpc_server_handle_request()) Handling RPC req@000000005831fa36 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_001:b50963bd-8eba-4432-9a54-8888d00e3fa1+6:2272:x1862456979138880:12345-192.168.204.3@tcp:400:kworker/u8:3.0 00000100:00100000:2.0:1776178680.104235:0:6256:0:(service.c:2372:ptlrpc_server_handle_request()) Handled RPC req@000000005831fa36 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_001:b50963bd-8eba-4432-9a54-8888d00e3fa1+6:2272:x1862456979138880:12345-192.168.204.3@tcp:400:kworker/u8:3.0 Request processed in 83us (270us total) trans 0 rc 0/0 00000100:00100000:2.0:1776178680.104243:0:6256:0:(nrs_fifo.c:241:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.204.3@tcp, seq: 1207 00000100:00100000:2.0:1776178680.104260:0:6256:0:(nrs_fifo.c:179:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.204.3@tcp, seq: 6996 00000100:00100000:2.0:1776178680.104264:0:6256:0:(service.c:2324:ptlrpc_server_handle_request()) Handling RPC req@00000000d23b79a7 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_001:0+-99:2269:x1862456979138816:12345-192.168.204.3@tcp:38:kworker/u8:3.0 00010000:02000000:2.0:1776178680.104278:0:6256:0:(ldlm_lib.c:1139:target_handle_connect()) lustre-MDT0000: Not available for connect from 192.168.204.3@tcp (stopping) 00000100:00100000:2.0:1776178680.104303:0:6256:0:(service.c:2372:ptlrpc_server_handle_request()) Handled RPC req@00000000d23b79a7 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_001:0+-99:2269:x1862456979138816:12345-192.168.204.3@tcp:38:kworker/u8:3.0 Request processed in 38us (379us total) trans 0 rc -19/-19 00000100:00100000:2.0:1776178680.104309:0:6256:0:(nrs_fifo.c:241:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.204.3@tcp, seq: 6996 00000100:00100000:1.0:1776178680.104751:0:20140:0:(service.c:2152:ptlrpc_server_handle_req_in()) got req x1862456979138752 00000100:00100000:1.0:1776178680.104788:0:20140:0:(nrs_fifo.c:179:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.204.3@tcp, seq: 341 00000100:00100000:1.0:1776178680.104793:0:20140:0:(service.c:2324:ptlrpc_server_handle_request()) Handling RPC req@000000003ef1d595 pname:cluuid+ref:pid:xid:nid:opc:job ll_mgs_0002:5b5c66b1-1a2f-4651-8548-0c75b9a38a94+9:2272:x1862456979138752:12345-192.168.204.3@tcp:400:kworker/u8:3.0 00000100:00100000:1.0:1776178680.104834:0:20140:0:(service.c:2372:ptlrpc_server_handle_request()) Handled RPC req@000000003ef1d595 pname:cluuid+ref:pid:xid:nid:opc:job ll_mgs_0002:5b5c66b1-1a2f-4651-8548-0c75b9a38a94+9:2272:x1862456979138752:12345-192.168.204.3@tcp:400:kworker/u8:3.0 Request processed in 40us (984us total) trans 0 rc 0/0 00000100:00100000:1.0:1776178680.104840:0:20140:0:(nrs_fifo.c:241:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.204.3@tcp, seq: 341 00000100:00100000:0.0:1776178680.105809:0:3347:0:(events.c:361:request_in_callback()) peer: 12345-192.168.204.3@tcp (source: 12345-192.168.204.3@tcp) 00000100:00100000:0.0:1776178680.105859:0:3347:0:(events.c:361:request_in_callback()) peer: 12345-192.168.204.3@tcp (source: 12345-192.168.204.3@tcp) 00000100:00100000:1.0:1776178680.105942:0:20762:0:(service.c:2152:ptlrpc_server_handle_req_in()) got req x1862456979139008 00000100:00100000:1.0:1776178680.105990:0:20762:0:(service.c:2152:ptlrpc_server_handle_req_in()) got req x1862456979138944 00000100:00100000:1.0:1776178680.106134:0:20762:0:(nrs_fifo.c:179:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.204.3@tcp, seq: 883 00000100:00100000:1.0:1776178680.106139:0:20762:0:(service.c:2324:ptlrpc_server_handle_request()) Handling RPC req@000000006a3d91b0 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_013:b50963bd-8eba-4432-9a54-8888d00e3fa1+9:2273:x1862456979139008:12345-192.168.204.3@tcp:400:kworker/u8:3.0 00000100:00100000:1.0:1776178680.106203:0:20762:0:(service.c:2372:ptlrpc_server_handle_request()) Handled RPC req@000000006a3d91b0 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_013:b50963bd-8eba-4432-9a54-8888d00e3fa1+9:2273:x1862456979139008:12345-192.168.204.3@tcp:400:kworker/u8:3.0 Request processed in 62us (403us total) trans 0 rc 0/0 00000100:00100000:1.0:1776178680.106226:0:20762:0:(nrs_fifo.c:241:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.204.3@tcp, seq: 883 00000100:00100000:1.0:1776178680.106248:0:20762:0:(nrs_fifo.c:179:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.204.3@tcp, seq: 884 00000100:00100000:1.0:1776178680.106251:0:20762:0:(service.c:2324:ptlrpc_server_handle_request()) Handling RPC req@000000002300ff10 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_013:b50963bd-8eba-4432-9a54-8888d00e3fa1+8:2273:x1862456979138944:12345-192.168.204.3@tcp:400:kworker/u8:3.0 00000100:00100000:1.0:1776178680.106276:0:20762:0:(service.c:2372:ptlrpc_server_handle_request()) Handled RPC req@000000002300ff10 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_013:b50963bd-8eba-4432-9a54-8888d00e3fa1+8:2273:x1862456979138944:12345-192.168.204.3@tcp:400:kworker/u8:3.0 Request processed in 23us (418us total) trans 0 rc 0/0 00000100:00100000:1.0:1776178680.106282:0:20762:0:(nrs_fifo.c:241:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.204.3@tcp, seq: 884 00010000:00010000:2.0:1776178680.295111:0:6244:0:(ldlm_lockd.c:2792:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00010000:00010000:1.0:1776178681.314347:0:6245:0:(ldlm_lockd.c:2792:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00080000:1.0:1776178681.696422:0:36:0:(pinger.c:251:ptlrpc_pinger_process_import()) lustre-MDT0000-lwp-OST0000_UUID->lustre-MDT0000_UUID: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00080000:1.0:1776178681.696431:0:36:0:(recover.c:57:ptlrpc_initiate_recovery()) lustre-MDT0000_UUID: starting recovery 00000100:00080000:1.0:1776178681.696435:0:36:0:(import.c:86:import_set_state_nolock()) 0000000047601a98 lustre-MDT0000_UUID: changing import state from DISCONN to CONNECTING 00000100:00080000:1.0:1776178681.696452:0:36:0:(import.c:534:import_select_connection()) lustre-MDT0000-lwp-OST0000: connect to NID 0@lo last attempt 1383 00000100:00080000:1.0:1776178681.696456:0:36:0:(import.c:581:import_select_connection()) lustre-MDT0000-lwp-OST0000: tried all connections, increasing latency to 21s 00000100:00080000:1.0:1776178681.696462:0:36:0:(import.c:615:import_select_connection()) lustre-MDT0000-lwp-OST0000: import 0000000047601a98 using connection 192.168.204.103@tcp/0@lo 00000100:00100000:1.0:1776178681.696490:0:36:0:(import.c:807:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5) req@000000006247494d x1862456998115072/t0(0) o38->lustre-MDT0000-lwp-OST0000@0@lo:12/10 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' 00000100:00100000:0.0:1776178681.696555:0:3352:0:(client.c:1733:ptlrpc_send_new_req()) Sending RPC req@000000006247494d pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:lustre-MDT0000-lwp-OST0000_UUID:3352:1862456998115072:0@lo:38:kworker/u8:1.0 00000100:00100000:3.0:1776178681.696590:0:3356:0:(client.c:1733:ptlrpc_send_new_req()) Sending RPC req@000000005b125149 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_03:ac61f839-0e8e-4cac-9c59-08bcfbbad330:3356:1862456998115008:0@lo:400:kworker/u8:1.0 00000100:00080000:1.0:1776178681.696683:0:36:0:(pinger.c:251:ptlrpc_pinger_process_import()) lustre-MDT0000-lwp-OST0001_UUID->lustre-MDT0000_UUID: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00080000:1.0:1776178681.696687:0:36:0:(recover.c:57:ptlrpc_initiate_recovery()) lustre-MDT0000_UUID: starting recovery 00000100:00080000:1.0:1776178681.696688:0:36:0:(import.c:86:import_set_state_nolock()) 00000000dfc8db13 lustre-MDT0000_UUID: changing import state from DISCONN to CONNECTING 00000100:00080000:1.0:1776178681.696693:0:36:0:(import.c:534:import_select_connection()) lustre-MDT0000-lwp-OST0001: connect to NID 0@lo last attempt 1383 00000100:00080000:1.0:1776178681.696695:0:36:0:(import.c:581:import_select_connection()) lustre-MDT0000-lwp-OST0001: tried all connections, increasing latency to 21s 00000100:00080000:1.0:1776178681.696698:0:36:0:(import.c:615:import_select_connection()) lustre-MDT0000-lwp-OST0001: import 00000000dfc8db13 using connection 192.168.204.103@tcp/0@lo 00000100:00100000:1.0:1776178681.696721:0:36:0:(import.c:807:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5) req@000000008c4bfd71 x1862456998115200/t0(0) o38->lustre-MDT0000-lwp-OST0001@0@lo:12/10 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' 00000100:00080000:1.0:1776178681.696745:0:36:0:(pinger.c:251:ptlrpc_pinger_process_import()) lustre-MDT0001-mdtlov_UUID->lustre-MDT0000_UUID: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00080000:1.0:1776178681.696748:0:36:0:(recover.c:57:ptlrpc_initiate_recovery()) lustre-MDT0000_UUID: starting recovery 00000100:00080000:1.0:1776178681.696749:0:36:0:(import.c:86:import_set_state_nolock()) 00000000ba4f1821 lustre-MDT0000_UUID: changing import state from DISCONN to CONNECTING 00000100:00080000:1.0:1776178681.696753:0:36:0:(import.c:534:import_select_connection()) lustre-MDT0000-osp-MDT0001: connect to NID 0@lo last attempt 1383 00000100:00080000:1.0:1776178681.696754:0:36:0:(import.c:581:import_select_connection()) lustre-MDT0000-osp-MDT0001: tried all connections, increasing latency to 21s 00000100:00080000:1.0:1776178681.696758:0:36:0:(import.c:615:import_select_connection()) lustre-MDT0000-osp-MDT0001: import 00000000ba4f1821 using connection 192.168.204.103@tcp/0@lo 00000100:00100000:1.0:1776178681.696767:0:36:0:(import.c:807:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5) req@00000000c237605d x1862456998115328/t0(0) o38->lustre-MDT0000-osp-MDT0001@0@lo:24/4 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' 00000100:00080000:1.0:1776178681.696785:0:36:0:(pinger.c:251:ptlrpc_pinger_process_import()) lustre-MDT0000-lwp-MDT0001_UUID->lustre-MDT0000_UUID: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00080000:1.0:1776178681.696790:0:36:0:(recover.c:57:ptlrpc_initiate_recovery()) lustre-MDT0000_UUID: starting recovery 00000100:00080000:1.0:1776178681.696791:0:36:0:(import.c:86:import_set_state_nolock()) 00000000a7c55b1f lustre-MDT0000_UUID: changing import state from DISCONN to CONNECTING 00000100:00080000:1.0:1776178681.696793:0:36:0:(import.c:534:import_select_connection()) lustre-MDT0000-lwp-MDT0001: connect to NID 0@lo last attempt 1383 00000100:00080000:1.0:1776178681.696795:0:36:0:(import.c:581:import_select_connection()) lustre-MDT0000-lwp-MDT0001: tried all connections, increasing latency to 21s 00000100:00080000:1.0:1776178681.696798:0:36:0:(import.c:615:import_select_connection()) lustre-MDT0000-lwp-MDT0001: import 00000000a7c55b1f using connection 192.168.204.103@tcp/0@lo 00000100:00100000:1.0:1776178681.696805:0:36:0:(import.c:807:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5) req@00000000e82039bb x1862456998115520/t0(0) o38->lustre-MDT0000-lwp-MDT0001@0@lo:12/10 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' 00000100:00100000:1.0:1776178681.696827:0:3353:0:(client.c:1733:ptlrpc_send_new_req()) Sending RPC req@00000000ae838296 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_00:lustre-MDT0001-lwp-OST0000_UUID:3353:1862456998115136:0@lo:400:kworker/u8:1.0 00000100:00100000:2.0:1776178681.696878:0:3355:0:(client.c:1733:ptlrpc_send_new_req()) Sending RPC req@00000000034781c2 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_02:lustre-MDT0001-mdtlov_UUID:3355:1862456998115392:0@lo:400:kworker/u8:1.0 00000100:00100000:3.0:1776178681.700086:0:3356:0:(events.c:361:request_in_callback()) peer: 12345-0@lo (source: 12345-0@lo) 00000100:00100000:0.0:1776178681.700092:0:3352:0:(events.c:361:request_in_callback()) peer: 12345-0@lo (source: 12345-0@lo) 00000100:00100000:0.0:1776178681.703106:0:3352:0:(client.c:1733:ptlrpc_send_new_req()) Sending RPC req@000000008c4bfd71 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:lustre-MDT0000-lwp-OST0001_UUID:3352:1862456998115200:0@lo:38:kworker/u8:1.0 00000100:00100000:0.0:1776178681.703188:0:3352:0:(events.c:361:request_in_callback()) peer: 12345-0@lo (source: 12345-0@lo) 00000100:00100000:0.0:1776178681.703309:0:3352:0:(client.c:1733:ptlrpc_send_new_req()) Sending RPC req@00000000c237605d pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:lustre-MDT0001-mdtlov_UUID:3352:1862456998115328:0@lo:38:kworker/u8:1.0 00000100:00100000:0.0:1776178681.703339:0:3352:0:(events.c:361:request_in_callback()) peer: 12345-0@lo (source: 12345-0@lo) 00000100:00100000:0.0:1776178681.703359:0:3352:0:(client.c:1733:ptlrpc_send_new_req()) Sending RPC req@00000000e82039bb pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:lustre-MDT0000-lwp-MDT0001_UUID:3352:1862456998115520:0@lo:38:kworker/u8:1.0 00000100:00100000:0.0:1776178681.703380:0:3352:0:(events.c:361:request_in_callback()) peer: 12345-0@lo (source: 12345-0@lo) 00000100:00100000:0.0:1776178681.703433:0:11281:0:(service.c:2152:ptlrpc_server_handle_req_in()) got req x1862456998115072 00000100:00100000:2.0:1776178681.703467:0:3355:0:(events.c:361:request_in_callback()) peer: 12345-0@lo (source: 12345-0@lo) 00000100:00100000:0.0:1776178681.703470:0:11281:0:(service.c:2152:ptlrpc_server_handle_req_in()) got req x1862456998115200 00000100:00100000:0.0:1776178681.703492:0:11281:0:(service.c:2152:ptlrpc_server_handle_req_in()) got req x1862456998115520 00000100:00100000:2.0:1776178681.703498:0:3355:0:(ptlrpcd.c:415:ptlrpcd_check()) transfer 1 async RPCs [3->2] 00000100:00100000:0.0:1776178681.703508:0:11281:0:(nrs_fifo.c:179:nrs_fifo_req_get()) NRS start fifo request from 12345-0@lo, seq: 6997 00000100:00100000:2.0:1776178681.703509:0:3355:0:(client.c:1733:ptlrpc_send_new_req()) Sending RPC req@0000000038b19922 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_02:lustre-MDT0001-mdtlov_UUID:3355:1862456998115456:0@lo:400:kworker/u8:1.0 00000100:00100000:0.0:1776178681.703516:0:11281:0:(service.c:2324:ptlrpc_server_handle_request()) Handling RPC req@00000000e8195050 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_004:0+-99:3352:x1862456998115072:12345-0@lo:38:kworker/u8:1.0 00000100:00100000:2.0:1776178681.703538:0:3355:0:(events.c:361:request_in_callback()) peer: 12345-0@lo (source: 12345-0@lo) 00000100:00100000:2.0:1776178681.703569:0:6256:0:(nrs_fifo.c:179:nrs_fifo_req_get()) NRS start fifo request from 12345-0@lo, seq: 6998 00000100:00100000:2.0:1776178681.703574:0:6256:0:(service.c:2324:ptlrpc_server_handle_request()) Handling RPC req@0000000029e5c385 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_001:0+-99:3352:x1862456998115200:12345-0@lo:38:kworker/u8:1.0 00010000:02000000:2.0:1776178681.703585:0:6256:0:(ldlm_lib.c:1139:target_handle_connect()) lustre-MDT0000: Not available for connect from 0@lo (stopping) 00000100:00100000:2.0:1776178681.703638:0:6256:0:(service.c:2372:ptlrpc_server_handle_request()) Handled RPC req@0000000029e5c385 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_001:0+-99:3352:x1862456998115200:12345-0@lo:38:kworker/u8:1.0 Request processed in 63us (451us total) trans 0 rc -19/-19 00000100:00100000:2.0:1776178681.703645:0:6256:0:(nrs_fifo.c:241:nrs_fifo_req_stop()) NRS stop fifo request from 12345-0@lo, seq: 6998 00000100:00100000:2.0:1776178681.703673:0:6256:0:(nrs_fifo.c:179:nrs_fifo_req_get()) NRS start fifo request from 12345-0@lo, seq: 6999 00000100:00100000:2.0:1776178681.703688:0:6256:0:(service.c:2324:ptlrpc_server_handle_request()) Handling RPC req@000000009c4a59b3 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_001:0+-99:3352:x1862456998115520:12345-0@lo:38:kworker/u8:1.0 00010000:02000000:2.0:1776178681.703696:0:6256:0:(ldlm_lib.c:1139:target_handle_connect()) lustre-MDT0000: Not available for connect from 0@lo (stopping) 00000100:00100000:2.0:1776178681.703718:0:6256:0:(service.c:2372:ptlrpc_server_handle_request()) Handled RPC req@000000009c4a59b3 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_001:0+-99:3352:x1862456998115520:12345-0@lo:38:kworker/u8:1.0 Request processed in 41us (338us total) trans 0 rc -19/-19 00010000:02000000:0.0:1776178681.703723:0:11281:0:(ldlm_lib.c:1139:target_handle_connect()) lustre-MDT0000: Not available for connect from 0@lo (stopping) 00000100:00100000:2.0:1776178681.703727:0:6256:0:(nrs_fifo.c:241:nrs_fifo_req_stop()) NRS stop fifo request from 12345-0@lo, seq: 6999 00000100:00100000:2.0:1776178681.703749:0:20762:0:(service.c:2152:ptlrpc_server_handle_req_in()) got req x1862456998115392 00000100:00100000:0.0:1776178681.703759:0:11281:0:(service.c:2372:ptlrpc_server_handle_request()) Handled RPC req@00000000e8195050 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_004:0+-99:3352:x1862456998115072:12345-0@lo:38:kworker/u8:1.0 Request processed in 243us (3670us total) trans 0 rc -19/-19 00000100:00100000:0.0:1776178681.703770:0:11281:0:(nrs_fifo.c:241:nrs_fifo_req_stop()) NRS stop fifo request from 12345-0@lo, seq: 6997 00000100:00100000:2.0:1776178681.703786:0:20762:0:(service.c:2152:ptlrpc_server_handle_req_in()) got req x1862456998115456 00000100:00100000:2.0:1776178681.703810:0:20762:0:(nrs_fifo.c:179:nrs_fifo_req_get()) NRS start fifo request from 12345-0@lo, seq: 885 00000100:00100000:2.0:1776178681.703815:0:20762:0:(service.c:2324:ptlrpc_server_handle_request()) Handling RPC req@0000000047c15ac0 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_013:lustre-MDT0001-mdtlov_UUID+5:3355:x1862456998115392:12345-0@lo:400:kworker/u8:1.0 00000100:00080000:0.0:1776178681.703816:0:3352:0:(import.c:86:import_set_state_nolock()) 00000000dfc8db13 lustre-MDT0000_UUID: changing import state from CONNECTING to DISCONN 00000100:00080000:0.0:1776178681.703822:0:3352:0:(import.c:1429:ptlrpc_connect_interpret()) recovery of lustre-MDT0000_UUID on 192.168.204.103@tcp failed (-19) 00000100:00100000:0.0:1776178681.703827:0:3352:0:(client.c:2215:ptlrpc_check_set()) Completed RPC req@000000008c4bfd71 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:lustre-MDT0000-lwp-OST0001_UUID:3352:1862456998115200:0@lo:38:kworker/u8:1.0 00000100:00080000:0.0:1776178681.703837:0:3352:0:(import.c:86:import_set_state_nolock()) 00000000a7c55b1f lustre-MDT0000_UUID: changing import state from CONNECTING to DISCONN 00000100:00080000:0.0:1776178681.703840:0:3352:0:(import.c:1429:ptlrpc_connect_interpret()) recovery of lustre-MDT0000_UUID on 192.168.204.103@tcp failed (-19) 00000100:00100000:0.0:1776178681.703843:0:3352:0:(client.c:2215:ptlrpc_check_set()) Completed RPC req@00000000e82039bb pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:lustre-MDT0000-lwp-MDT0001_UUID:3352:1862456998115520:0@lo:38:kworker/u8:1.0 00000100:00080000:0.0:1776178681.703853:0:3352:0:(import.c:86:import_set_state_nolock()) 0000000047601a98 lustre-MDT0000_UUID: changing import state from CONNECTING to DISCONN 00000100:00080000:0.0:1776178681.703856:0:3352:0:(import.c:1429:ptlrpc_connect_interpret()) recovery of lustre-MDT0000_UUID on 192.168.204.103@tcp failed (-19) 00000100:00100000:0.0:1776178681.703859:0:3352:0:(client.c:2215:ptlrpc_check_set()) Completed RPC req@000000006247494d pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:lustre-MDT0000-lwp-OST0000_UUID:3352:1862456998115072:0@lo:38:kworker/u8:1.0 00000100:00100000:2.0:1776178681.703869:0:20762:0:(service.c:2372:ptlrpc_server_handle_request()) Handled RPC req@0000000047c15ac0 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_013:lustre-MDT0001-mdtlov_UUID+5:3355:x1862456998115392:12345-0@lo:400:kworker/u8:1.0 Request processed in 52us (404us total) trans 0 rc 0/0 00000100:00100000:2.0:1776178681.703877:0:20762:0:(nrs_fifo.c:241:nrs_fifo_req_stop()) NRS stop fifo request from 12345-0@lo, seq: 885 00000100:00100000:2.0:1776178681.703892:0:20762:0:(nrs_fifo.c:179:nrs_fifo_req_get()) NRS start fifo request from 12345-0@lo, seq: 886 00000100:00100000:2.0:1776178681.703896:0:20762:0:(service.c:2324:ptlrpc_server_handle_request()) Handling RPC req@000000001f977915 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_013:lustre-MDT0001-mdtlov_UUID+5:3355:x1862456998115456:12345-0@lo:400:kworker/u8:1.0 00000100:00100000:0.0:1776178681.703911:0:3354:0:(client.c:1733:ptlrpc_send_new_req()) Sending RPC req@00000000c37c187e pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_01:lustre-MDT0001-lwp-OST0001_UUID:3354:1862456998115264:0@lo:400:kworker/u8:1.0 00000100:00100000:2.0:1776178681.703924:0:20762:0:(service.c:2372:ptlrpc_server_handle_request()) Handled RPC req@000000001f977915 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_013:lustre-MDT0001-mdtlov_UUID+5:3355:x1862456998115456:12345-0@lo:400:kworker/u8:1.0 Request processed in 26us (386us total) trans 0 rc 0/0 00000100:00100000:2.0:1776178681.703930:0:20762:0:(nrs_fifo.c:241:nrs_fifo_req_stop()) NRS stop fifo request from 12345-0@lo, seq: 886 00000100:00100000:2.0:1776178681.703962:0:3355:0:(client.c:2215:ptlrpc_check_set()) Completed RPC req@0000000038b19922 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_02:lustre-MDT0001-mdtlov_UUID:3355:1862456998115456:0@lo:400:kworker/u8:1.0 00000100:00100000:2.0:1776178681.703970:0:3355:0:(client.c:2215:ptlrpc_check_set()) Completed RPC req@00000000034781c2 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_02:lustre-MDT0001-mdtlov_UUID:3355:1862456998115392:0@lo:400:kworker/u8:1.0 00000100:00100000:1.0:1776178681.704028:0:3353:0:(events.c:361:request_in_callback()) peer: 12345-0@lo (source: 12345-0@lo) 00000100:00100000:2.0:1776178681.704030:0:20137:0:(service.c:2152:ptlrpc_server_handle_req_in()) got req x1862456998115008 00000100:00100000:1.0:1776178681.704086:0:8250:0:(service.c:2152:ptlrpc_server_handle_req_in()) got req x1862456998115136 00000100:00100000:2.0:1776178681.704088:0:20137:0:(nrs_fifo.c:179:nrs_fifo_req_get()) NRS start fifo request from 12345-0@lo, seq: 342 00000100:00100000:2.0:1776178681.704093:0:20137:0:(service.c:2324:ptlrpc_server_handle_request()) Handling RPC req@00000000b4b6b848 pname:cluuid+ref:pid:xid:nid:opc:job ll_mgs_0000:ac61f839-0e8e-4cac-9c59-08bcfbbad330+18:3356:x1862456998115008:12345-0@lo:400:kworker/u8:1.0 00000100:00100000:1.0:1776178681.704117:0:8250:0:(nrs_fifo.c:179:nrs_fifo_req_get()) NRS start fifo request from 12345-0@lo, seq: 1208 00000100:00100000:1.0:1776178681.704123:0:8250:0:(service.c:2324:ptlrpc_server_handle_request()) Handling RPC req@00000000034781c2 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_003:lustre-MDT0001-lwp-OST0000_UUID+5:3353:x1862456998115136:12345-0@lo:400:kworker/u8:1.0 00000100:00100000:2.0:1776178681.704132:0:20137:0:(service.c:2372:ptlrpc_server_handle_request()) Handled RPC req@00000000b4b6b848 pname:cluuid+ref:pid:xid:nid:opc:job ll_mgs_0000:ac61f839-0e8e-4cac-9c59-08bcfbbad330+18:3356:x1862456998115008:12345-0@lo:400:kworker/u8:1.0 Request processed in 37us (4054us total) trans 0 rc 0/0 00000100:00100000:2.0:1776178681.704139:0:20137:0:(nrs_fifo.c:241:nrs_fifo_req_stop()) NRS stop fifo request from 12345-0@lo, seq: 342 00000100:00100000:1.0:1776178681.704165:0:8250:0:(service.c:2372:ptlrpc_server_handle_request()) Handled RPC req@00000000034781c2 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_003:lustre-MDT0001-lwp-OST0000_UUID+5:3353:x1862456998115136:12345-0@lo:400:kworker/u8:1.0 Request processed in 42us (139us total) trans 0 rc 0/0 00000100:00100000:1.0:1776178681.704173:0:8250:0:(nrs_fifo.c:241:nrs_fifo_req_stop()) NRS stop fifo request from 12345-0@lo, seq: 1208 00000100:00100000:2.0:1776178681.704190:0:15213:0:(service.c:2152:ptlrpc_server_handle_req_in()) got req x1862456998115328 00000100:00100000:2.0:1776178681.704212:0:15213:0:(nrs_fifo.c:179:nrs_fifo_req_get()) NRS start fifo request from 12345-0@lo, seq: 801 00000100:00100000:2.0:1776178681.704216:0:15213:0:(service.c:2324:ptlrpc_server_handle_request()) Handling RPC req@000000007216887e pname:cluuid+ref:pid:xid:nid:opc:job mdt_out00_003:0+-99:3352:x1862456998115328:12345-0@lo:38:kworker/u8:1.0 00010000:02000000:2.0:1776178681.704224:0:15213:0:(ldlm_lib.c:1139:target_handle_connect()) lustre-MDT0000: Not available for connect from 0@lo (stopping) 00000100:00100000:2.0:1776178681.704255:0:15213:0:(service.c:2372:ptlrpc_server_handle_request()) Handled RPC req@000000007216887e pname:cluuid+ref:pid:xid:nid:opc:job mdt_out00_003:0+-99:3352:x1862456998115328:12345-0@lo:38:kworker/u8:1.0 Request processed in 38us (916us total) trans 0 rc -19/-19 00000100:00100000:2.0:1776178681.704261:0:15213:0:(nrs_fifo.c:241:nrs_fifo_req_stop()) NRS stop fifo request from 12345-0@lo, seq: 801 00000100:00100000:1.0:1776178681.704306:0:3353:0:(client.c:2215:ptlrpc_check_set()) Completed RPC req@00000000ae838296 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_00:lustre-MDT0001-lwp-OST0000_UUID:3353:1862456998115136:0@lo:400:kworker/u8:1.0 00000100:00080000:2.0:1776178681.704308:0:3352:0:(import.c:86:import_set_state_nolock()) 00000000ba4f1821 lustre-MDT0000_UUID: changing import state from CONNECTING to DISCONN 00000100:00080000:2.0:1776178681.704314:0:3352:0:(import.c:1429:ptlrpc_connect_interpret()) recovery of lustre-MDT0000_UUID on 192.168.204.103@tcp failed (-19) 00000100:00100000:2.0:1776178681.704318:0:3352:0:(client.c:2215:ptlrpc_check_set()) Completed RPC req@00000000c237605d pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:lustre-MDT0001-mdtlov_UUID:3352:1862456998115328:0@lo:38:kworker/u8:1.0 00000100:00100000:0.0:1776178681.704397:0:3354:0:(events.c:361:request_in_callback()) peer: 12345-0@lo (source: 12345-0@lo) 00000100:00100000:3.0:1776178681.704999:0:3356:0:(client.c:2215:ptlrpc_check_set()) Completed RPC req@000000005b125149 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_03:ac61f839-0e8e-4cac-9c59-08bcfbbad330:3356:1862456998115008:0@lo:400:kworker/u8:1.0 00000100:00100000:1.0:1776178681.705315:0:6257:0:(service.c:2152:ptlrpc_server_handle_req_in()) got req x1862456998115264 00000100:00100000:1.0:1776178681.705363:0:6257:0:(nrs_fifo.c:179:nrs_fifo_req_get()) NRS start fifo request from 12345-0@lo, seq: 1209 00000100:00100000:1.0:1776178681.705368:0:6257:0:(service.c:2324:ptlrpc_server_handle_request()) Handling RPC req@0000000075189f40 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_002:lustre-MDT0001-lwp-OST0001_UUID+5:3354:x1862456998115264:12345-0@lo:400:kworker/u8:1.0 00000100:00100000:1.0:1776178681.705416:0:6257:0:(service.c:2372:ptlrpc_server_handle_request()) Handled RPC req@0000000075189f40 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_002:lustre-MDT0001-lwp-OST0001_UUID+5:3354:x1862456998115264:12345-0@lo:400:kworker/u8:1.0 Request processed in 47us (1019us total) trans 0 rc 0/0 00000100:00100000:1.0:1776178681.705422:0:6257:0:(nrs_fifo.c:241:nrs_fifo_req_stop()) NRS stop fifo request from 12345-0@lo, seq: 1209 00000100:00100000:3.0:1776178681.706963:0:3354:0:(client.c:2215:ptlrpc_check_set()) Completed RPC req@00000000c37c187e pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_01:lustre-MDT0001-lwp-OST0001_UUID:3354:1862456998115264:0@lo:400:kworker/u8:1.0 00010000:00010000:2.0:1776178682.341414:0:9224:0:(ldlm_lockd.c:2792:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:1.0:1776178683.232331:0:3353:0:(client.c:1733:ptlrpc_send_new_req()) Sending RPC req@00000000ec38a60b pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_00:lustre-MDT0001-mdtlov_UUID:3353:1862456998115584:0@lo:13:osp-pre-1-1.0 00000100:00100000:1.0:1776178683.232404:0:3353:0:(events.c:361:request_in_callback()) peer: 12345-0@lo (source: 12345-0@lo) 00000100:00100000:2.0:1776178683.232688:0:9554:0:(service.c:2152:ptlrpc_server_handle_req_in()) got req x1862456998115584 00000100:00100000:2.0:1776178683.232744:0:9554:0:(nrs_fifo.c:179:nrs_fifo_req_get()) NRS start fifo request from 12345-0@lo, seq: 675 00000100:00100000:2.0:1776178683.232749:0:9554:0:(service.c:2324:ptlrpc_server_handle_request()) Handling RPC req@0000000038b19922 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_create00:lustre-MDT0001-mdtlov_UUID+5:3353:x1862456998115584:12345-0@lo:13:osp-pre-1-1.0 00002000:00000024:2.0:1776178683.232780:0:9554:0:(ofd_obd.c:740:ofd_statfs()) blocks cached 229376 granted 491746368 pending 0 free 3923689472 avail 3692167168 00000100:00100000:2.0:1776178683.232834:0:9554:0:(service.c:2372:ptlrpc_server_handle_request()) Handled RPC req@0000000038b19922 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_create00:lustre-MDT0001-mdtlov_UUID+5:3353:x1862456998115584:12345-0@lo:13:osp-pre-1-1.0 Request processed in 83us (430us total) trans 0 rc 0/0 00000100:00100000:2.0:1776178683.232840:0:9554:0:(nrs_fifo.c:241:nrs_fifo_req_stop()) NRS stop fifo request from 12345-0@lo, seq: 675 00000100:00100000:3.0:1776178683.233019:0:3353:0:(client.c:2215:ptlrpc_check_set()) Completed RPC req@00000000ec38a60b pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_00:lustre-MDT0001-mdtlov_UUID:3353:1862456998115584:0@lo:13:osp-pre-1-1.0 00010000:00010000:1.0:1776178683.360579:0:9225:0:(ldlm_lockd.c:2792:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:3.0:1776178683.744442:0:3353:0:(ptlrpcd.c:415:ptlrpcd_check()) transfer 1 async RPCs [1->0] 00000100:00100000:3.0:1776178683.744465:0:3353:0:(client.c:1733:ptlrpc_send_new_req()) Sending RPC req@00000000b0409689 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_00:lustre-MDT0001-mdtlov_UUID:3353:1862456998115648:0@lo:13:osp-pre-0-1.0 00000100:00100000:3.0:1776178683.744532:0:3353:0:(events.c:361:request_in_callback()) peer: 12345-0@lo (source: 12345-0@lo) 00000100:00100000:1.0:1776178683.744644:0:8231:0:(service.c:2152:ptlrpc_server_handle_req_in()) got req x1862456998115648 00000100:00100000:1.0:1776178683.744693:0:8231:0:(nrs_fifo.c:179:nrs_fifo_req_get()) NRS start fifo request from 12345-0@lo, seq: 676 00000100:00100000:1.0:1776178683.744698:0:8231:0:(service.c:2324:ptlrpc_server_handle_request()) Handling RPC req@00000000ec38a60b pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_create00:lustre-MDT0001-mdtlov_UUID+5:3353:x1862456998115648:12345-0@lo:13:osp-pre-0-1.0 00002000:00000024:1.0:1776178683.744725:0:8231:0:(ofd_obd.c:740:ofd_statfs()) blocks cached 315392 granted 484883776 pending 0 free 3923689472 avail 3692167168 00000100:00100000:1.0:1776178683.744772:0:8231:0:(service.c:2372:ptlrpc_server_handle_request()) Handled RPC req@00000000ec38a60b pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_create00:lustre-MDT0001-mdtlov_UUID+5:3353:x1862456998115648:12345-0@lo:13:osp-pre-0-1.0 Request processed in 73us (241us total) trans 0 rc 0/0 00000100:00100000:1.0:1776178683.744779:0:8231:0:(nrs_fifo.c:241:nrs_fifo_req_stop()) NRS stop fifo request from 12345-0@lo, seq: 676 00000100:00100000:3.0:1776178683.744859:0:3353:0:(client.c:2215:ptlrpc_check_set()) Completed RPC req@00000000b0409689 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_00:lustre-MDT0001-mdtlov_UUID:3353:1862456998115648:0@lo:13:osp-pre-0-1.0 00010000:00010000:2.0:1776178684.386083:0:6244:0:(ldlm_lockd.c:2792:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:0.0:1776178685.217932:0:3348:0:(events.c:361:request_in_callback()) peer: 12345-192.168.204.3@tcp (source: 12345-192.168.204.3@tcp) 00000100:00100000:0.0:1776178685.218056:0:3348:0:(events.c:361:request_in_callback()) peer: 12345-192.168.204.3@tcp (source: 12345-192.168.204.3@tcp) 00000100:00100000:2.0:1776178685.218447:0:20140:0:(service.c:2152:ptlrpc_server_handle_req_in()) got req x1862456979139072 00000100:00100000:2.0:1776178685.218513:0:20140:0:(nrs_fifo.c:179:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.204.3@tcp, seq: 343 00000100:00100000:2.0:1776178685.218520:0:20140:0:(service.c:2324:ptlrpc_server_handle_request()) Handling RPC req@000000001333a9de pname:cluuid+ref:pid:xid:nid:opc:job ll_mgs_0002:5b5c66b1-1a2f-4651-8548-0c75b9a38a94+9:2273:x1862456979139072:12345-192.168.204.3@tcp:400:kworker/u8:3.0 00000100:00100000:2.0:1776178685.218642:0:20140:0:(service.c:2372:ptlrpc_server_handle_request()) Handled RPC req@000000001333a9de pname:cluuid+ref:pid:xid:nid:opc:job ll_mgs_0002:5b5c66b1-1a2f-4651-8548-0c75b9a38a94+9:2273:x1862456979139072:12345-192.168.204.3@tcp:400:kworker/u8:3.0 Request processed in 121us (724us total) trans 0 rc 0/0 00000100:00100000:2.0:1776178685.218652:0:20140:0:(nrs_fifo.c:241:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.204.3@tcp, seq: 343 00000100:00100000:1.0:1776178685.218878:0:8250:0:(service.c:2152:ptlrpc_server_handle_req_in()) got req x1862456979139136 00000100:00100000:1.0:1776178685.218925:0:8250:0:(nrs_fifo.c:179:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.204.3@tcp, seq: 7000 00000100:00100000:1.0:1776178685.218930:0:8250:0:(service.c:2324:ptlrpc_server_handle_request()) Handling RPC req@000000004ce46e20 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_003:0+-99:2269:x1862456979139136:12345-192.168.204.3@tcp:38:kworker/u8:3.0 00010000:02000000:1.0:1776178685.218950:0:8250:0:(ldlm_lib.c:1139:target_handle_connect()) lustre-MDT0000: Not available for connect from 192.168.204.3@tcp (stopping) 00000100:00100000:1.0:1776178685.218991:0:8250:0:(service.c:2372:ptlrpc_server_handle_request()) Handled RPC req@000000004ce46e20 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_003:0+-99:2269:x1862456979139136:12345-192.168.204.3@tcp:38:kworker/u8:3.0 Request processed in 61us (936us total) trans 0 rc -19/-19 00000100:00100000:1.0:1776178685.218998:0:8250:0:(nrs_fifo.c:241:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.204.3@tcp, seq: 7000 00000100:00100000:2.0:1776178685.219857:0:3346:0:(events.c:361:request_in_callback()) peer: 12345-192.168.204.3@tcp (source: 12345-192.168.204.3@tcp) 00000100:00100000:2.0:1776178685.219895:0:3346:0:(events.c:361:request_in_callback()) peer: 12345-192.168.204.3@tcp (source: 12345-192.168.204.3@tcp) 00000100:00100000:2.0:1776178685.219922:0:3346:0:(events.c:361:request_in_callback()) peer: 12345-192.168.204.3@tcp (source: 12345-192.168.204.3@tcp) 00000100:00100000:3.0:1776178685.220050:0:20755:0:(service.c:2152:ptlrpc_server_handle_req_in()) got req x1862456979139264 00000100:00100000:3.0:1776178685.220150:0:20755:0:(service.c:2152:ptlrpc_server_handle_req_in()) got req x1862456979139328 00000100:00100000:3.0:1776178685.220173:0:20755:0:(nrs_fifo.c:179:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.204.3@tcp, seq: 887 00000100:00100000:3.0:1776178685.220179:0:20755:0:(service.c:2324:ptlrpc_server_handle_request()) Handling RPC req@00000000b0409689 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_006:b50963bd-8eba-4432-9a54-8888d00e3fa1+8:2272:x1862456979139264:12345-192.168.204.3@tcp:400:kworker/u8:3.0 00000100:00100000:3.0:1776178685.220241:0:20755:0:(service.c:2372:ptlrpc_server_handle_request()) Handled RPC req@00000000b0409689 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_006:b50963bd-8eba-4432-9a54-8888d00e3fa1+8:2272:x1862456979139264:12345-192.168.204.3@tcp:400:kworker/u8:3.0 Request processed in 61us (345us total) trans 0 rc 0/0 00000100:00100000:3.0:1776178685.220249:0:20755:0:(nrs_fifo.c:241:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.204.3@tcp, seq: 887 00000100:00100000:3.0:1776178685.220264:0:20755:0:(nrs_fifo.c:179:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.204.3@tcp, seq: 888 00000100:00100000:3.0:1776178685.220268:0:20755:0:(service.c:2324:ptlrpc_server_handle_request()) Handling RPC req@00000000ef862c6d pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_006:b50963bd-8eba-4432-9a54-8888d00e3fa1+9:2272:x1862456979139328:12345-192.168.204.3@tcp:400:kworker/u8:3.0 00000100:00100000:3.0:1776178685.220291:0:20755:0:(service.c:2372:ptlrpc_server_handle_request()) Handled RPC req@00000000ef862c6d pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_006:b50963bd-8eba-4432-9a54-8888d00e3fa1+9:2272:x1862456979139328:12345-192.168.204.3@tcp:400:kworker/u8:3.0 Request processed in 22us (369us total) trans 0 rc 0/0 00000100:00100000:3.0:1776178685.220297:0:20755:0:(nrs_fifo.c:241:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.204.3@tcp, seq: 888 00000100:00100000:1.0:1776178685.220763:0:6257:0:(service.c:2152:ptlrpc_server_handle_req_in()) got req x1862456979139200 00000100:00100000:1.0:1776178685.220807:0:6257:0:(nrs_fifo.c:179:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.204.3@tcp, seq: 1210 00000100:00100000:1.0:1776178685.220812:0:6257:0:(service.c:2324:ptlrpc_server_handle_request()) Handling RPC req@000000008c3fbdb6 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_002:b50963bd-8eba-4432-9a54-8888d00e3fa1+6:2273:x1862456979139200:12345-192.168.204.3@tcp:400:kworker/u8:3.0 00000100:00100000:1.0:1776178685.220865:0:6257:0:(service.c:2372:ptlrpc_server_handle_request()) Handled RPC req@000000008c3fbdb6 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_002:b50963bd-8eba-4432-9a54-8888d00e3fa1+6:2273:x1862456979139200:12345-192.168.204.3@tcp:400:kworker/u8:3.0 Request processed in 52us (1013us total) trans 0 rc 0/0 00000100:00100000:1.0:1776178685.220871:0:6257:0:(nrs_fifo.c:241:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.204.3@tcp, seq: 1210 00010000:00010000:1.0:1776178685.409076:0:6245:0:(ldlm_lockd.c:2792:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00010000:00010000:2.0:1776178686.432388:0:9224:0:(ldlm_lockd.c:2792:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00080000:1.0:1776178686.816271:0:36:0:(pinger.c:251:ptlrpc_pinger_process_import()) lustre-MDT0000-lwp-OST0000_UUID->lustre-MDT0000_UUID: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00080000:1.0:1776178686.816280:0:36:0:(recover.c:57:ptlrpc_initiate_recovery()) lustre-MDT0000_UUID: starting recovery 00000100:00080000:1.0:1776178686.816284:0:36:0:(import.c:86:import_set_state_nolock()) 0000000047601a98 lustre-MDT0000_UUID: changing import state from DISCONN to CONNECTING 00000100:00080000:1.0:1776178686.816295:0:36:0:(import.c:534:import_select_connection()) lustre-MDT0000-lwp-OST0000: connect to NID 0@lo last attempt 1389 00000100:00080000:1.0:1776178686.816298:0:36:0:(import.c:581:import_select_connection()) lustre-MDT0000-lwp-OST0000: tried all connections, increasing latency to 21s 00000100:00080000:1.0:1776178686.816306:0:36:0:(import.c:615:import_select_connection()) lustre-MDT0000-lwp-OST0000: import 0000000047601a98 using connection 192.168.204.103@tcp/0@lo 00000100:00100000:1.0:1776178686.816337:0:36:0:(import.c:807:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5) req@00000000c37c187e x1862456998115776/t0(0) o38->lustre-MDT0000-lwp-OST0000@0@lo:12/10 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' 00000100:00080000:1.0:1776178686.816361:0:36:0:(pinger.c:251:ptlrpc_pinger_process_import()) lustre-MDT0000-lwp-OST0001_UUID->lustre-MDT0000_UUID: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00080000:1.0:1776178686.816364:0:36:0:(recover.c:57:ptlrpc_initiate_recovery()) lustre-MDT0000_UUID: starting recovery 00000100:00080000:1.0:1776178686.816366:0:36:0:(import.c:86:import_set_state_nolock()) 00000000dfc8db13 lustre-MDT0000_UUID: changing import state from DISCONN to CONNECTING 00000100:00080000:1.0:1776178686.816369:0:36:0:(import.c:534:import_select_connection()) lustre-MDT0000-lwp-OST0001: connect to NID 0@lo last attempt 1389 00000100:00080000:1.0:1776178686.816371:0:36:0:(import.c:581:import_select_connection()) lustre-MDT0000-lwp-OST0001: tried all connections, increasing latency to 21s 00000100:00080000:1.0:1776178686.816374:0:36:0:(import.c:615:import_select_connection()) lustre-MDT0000-lwp-OST0001: import 00000000dfc8db13 using connection 192.168.204.103@tcp/0@lo 00000100:00100000:1.0:1776178686.816384:0:36:0:(import.c:807:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5) req@00000000964f7a65 x1862456998115904/t0(0) o38->lustre-MDT0000-lwp-OST0001@0@lo:12/10 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' 00000100:00080000:1.0:1776178686.816413:0:36:0:(pinger.c:251:ptlrpc_pinger_process_import()) lustre-MDT0001-mdtlov_UUID->lustre-MDT0000_UUID: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00080000:1.0:1776178686.816416:0:36:0:(recover.c:57:ptlrpc_initiate_recovery()) lustre-MDT0000_UUID: starting recovery 00000100:00080000:1.0:1776178686.816418:0:36:0:(import.c:86:import_set_state_nolock()) 00000000ba4f1821 lustre-MDT0000_UUID: changing import state from DISCONN to CONNECTING 00000100:00080000:1.0:1776178686.816422:0:36:0:(import.c:534:import_select_connection()) lustre-MDT0000-osp-MDT0001: connect to NID 0@lo last attempt 1389 00000100:00080000:1.0:1776178686.816423:0:36:0:(import.c:581:import_select_connection()) lustre-MDT0000-osp-MDT0001: tried all connections, increasing latency to 21s 00000100:00080000:1.0:1776178686.816426:0:36:0:(import.c:615:import_select_connection()) lustre-MDT0000-osp-MDT0001: import 00000000ba4f1821 using connection 192.168.204.103@tcp/0@lo 00000100:00100000:1.0:1776178686.816436:0:36:0:(import.c:807:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5) req@000000009015f30b x1862456998116032/t0(0) o38->lustre-MDT0000-osp-MDT0001@0@lo:24/4 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' 00000100:00080000:1.0:1776178686.816453:0:36:0:(pinger.c:251:ptlrpc_pinger_process_import()) lustre-MDT0000-lwp-MDT0001_UUID->lustre-MDT0000_UUID: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00080000:1.0:1776178686.816455:0:36:0:(recover.c:57:ptlrpc_initiate_recovery()) lustre-MDT0000_UUID: starting recovery 00000100:00080000:1.0:1776178686.816456:0:36:0:(import.c:86:import_set_state_nolock()) 00000000a7c55b1f lustre-MDT0000_UUID: changing import state from DISCONN to CONNECTING 00000100:00080000:1.0:1776178686.816459:0:36:0:(import.c:534:import_select_connection()) lustre-MDT0000-lwp-MDT0001: connect to NID 0@lo last attempt 1389 00000100:00080000:1.0:1776178686.816461:0:36:0:(import.c:581:import_select_connection()) lustre-MDT0000-lwp-MDT0001: tried all connections, increasing latency to 21s 00000100:00080000:1.0:1776178686.816464:0:36:0:(import.c:615:import_select_connection()) lustre-MDT0000-lwp-MDT0001: import 00000000a7c55b1f using connection 192.168.204.103@tcp/0@lo 00000100:00100000:1.0:1776178686.816472:0:36:0:(import.c:807:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5) req@0000000086665d0f x1862456998116224/t0(0) o38->lustre-MDT0000-lwp-MDT0001@0@lo:12/10 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' 00000100:00100000:1.0:1776178686.816524:0:3354:0:(client.c:1733:ptlrpc_send_new_req()) Sending RPC req@00000000faeeda53 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_01:lustre-MDT0001-mdtlov_UUID:3354:1862456998116096:0@lo:400:kworker/u8:1.0 00000100:00100000:1.0:1776178686.816608:0:3354:0:(events.c:361:request_in_callback()) peer: 12345-0@lo (source: 12345-0@lo) 00000100:00100000:1.0:1776178686.816641:0:3354:0:(ptlrpcd.c:415:ptlrpcd_check()) transfer 1 async RPCs [0->1] 00000100:00100000:1.0:1776178686.816646:0:3354:0:(client.c:1733:ptlrpc_send_new_req()) Sending RPC req@0000000009435d5b pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_01:lustre-MDT0001-lwp-OST0001_UUID:3354:1862456998115968:0@lo:400:kworker/u8:1.0 00000100:00100000:1.0:1776178686.816670:0:3354:0:(events.c:361:request_in_callback()) peer: 12345-0@lo (source: 12345-0@lo) 00000100:00100000:2.0:1776178686.817128:0:3355:0:(client.c:1733:ptlrpc_send_new_req()) Sending RPC req@00000000e82039bb pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_02:ac61f839-0e8e-4cac-9c59-08bcfbbad330:3355:1862456998115712:0@lo:400:kworker/u8:1.0 00000100:00100000:2.0:1776178686.817168:0:3355:0:(events.c:361:request_in_callback()) peer: 12345-0@lo (source: 12345-0@lo) 00000100:00100000:2.0:1776178686.817184:0:3355:0:(client.c:1733:ptlrpc_send_new_req()) Sending RPC req@0000000092e0e13d pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_02:lustre-MDT0001-mdtlov_UUID:3355:1862456998116160:0@lo:400:kworker/u8:1.0 00000100:00100000:2.0:1776178686.817205:0:3355:0:(events.c:361:request_in_callback()) peer: 12345-0@lo (source: 12345-0@lo) 00000100:00100000:2.0:1776178686.817225:0:3355:0:(ptlrpcd.c:415:ptlrpcd_check()) transfer 1 async RPCs [3->2] 00000100:00100000:2.0:1776178686.817231:0:3355:0:(client.c:1733:ptlrpc_send_new_req()) Sending RPC req@00000000c237605d pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_02:lustre-MDT0001-lwp-OST0000_UUID:3355:1862456998115840:0@lo:400:kworker/u8:1.0 00000100:00100000:2.0:1776178686.817265:0:3355:0:(events.c:361:request_in_callback()) peer: 12345-0@lo (source: 12345-0@lo) 00000100:00100000:2.0:1776178686.817292:0:3352:0:(client.c:1733:ptlrpc_send_new_req()) Sending RPC req@00000000c37c187e pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:lustre-MDT0000-lwp-OST0000_UUID:3352:1862456998115776:0@lo:38:kworker/u8:1.0 00000100:00100000:2.0:1776178686.817380:0:3352:0:(events.c:361:request_in_callback()) peer: 12345-0@lo (source: 12345-0@lo) 00000100:00100000:2.0:1776178686.817394:0:3352:0:(client.c:1733:ptlrpc_send_new_req()) Sending RPC req@00000000964f7a65 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:lustre-MDT0000-lwp-OST0001_UUID:3352:1862456998115904:0@lo:38:kworker/u8:1.0 00000100:00100000:2.0:1776178686.817412:0:3352:0:(events.c:361:request_in_callback()) peer: 12345-0@lo (source: 12345-0@lo) 00000100:00100000:2.0:1776178686.817423:0:3352:0:(client.c:1733:ptlrpc_send_new_req()) Sending RPC req@000000009015f30b pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:lustre-MDT0001-mdtlov_UUID:3352:1862456998116032:0@lo:38:kworker/u8:1.0 00000100:00100000:2.0:1776178686.817442:0:3352:0:(events.c:361:request_in_callback()) peer: 12345-0@lo (source: 12345-0@lo) 00000100:00100000:2.0:1776178686.817453:0:3352:0:(client.c:1733:ptlrpc_send_new_req()) Sending RPC req@0000000086665d0f pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:lustre-MDT0000-lwp-MDT0001_UUID:3352:1862456998116224:0@lo:38:kworker/u8:1.0 00000100:00100000:2.0:1776178686.817473:0:3352:0:(events.c:361:request_in_callback()) peer: 12345-0@lo (source: 12345-0@lo) 00000100:00100000:2.0:1776178686.817519:0:14440:0:(service.c:2152:ptlrpc_server_handle_req_in()) got req x1862456998116096 00000100:00100000:2.0:1776178686.817564:0:14440:0:(service.c:2152:ptlrpc_server_handle_req_in()) got req x1862456998116160 00000100:00100000:2.0:1776178686.817584:0:14440:0:(nrs_fifo.c:179:nrs_fifo_req_get()) NRS start fifo request from 12345-0@lo, seq: 889 00000100:00100000:2.0:1776178686.817590:0:14440:0:(service.c:2324:ptlrpc_server_handle_request()) Handling RPC req@00000000be147540 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_004:lustre-MDT0001-mdtlov_UUID+5:3354:x1862456998116096:12345-0@lo:400:kworker/u8:1.0 00000100:00100000:2.0:1776178686.817643:0:14440:0:(service.c:2372:ptlrpc_server_handle_request()) Handled RPC req@00000000be147540 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_004:lustre-MDT0001-mdtlov_UUID+5:3354:x1862456998116096:12345-0@lo:400:kworker/u8:1.0 Request processed in 52us (1049us total) trans 0 rc 0/0 00000100:00100000:2.0:1776178686.817650:0:14440:0:(nrs_fifo.c:241:nrs_fifo_req_stop()) NRS stop fifo request from 12345-0@lo, seq: 889 00000100:00100000:2.0:1776178686.817667:0:14440:0:(nrs_fifo.c:179:nrs_fifo_req_get()) NRS start fifo request from 12345-0@lo, seq: 890 00000100:00100000:2.0:1776178686.817670:0:14440:0:(service.c:2324:ptlrpc_server_handle_request()) Handling RPC req@00000000952003a4 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_004:lustre-MDT0001-mdtlov_UUID+5:3355:x1862456998116160:12345-0@lo:400:kworker/u8:1.0 00000100:00100000:1.0:1776178686.818218:0:20137:0:(service.c:2152:ptlrpc_server_handle_req_in()) got req x1862456998115712 00000100:00100000:1.0:1776178686.818255:0:20137:0:(nrs_fifo.c:179:nrs_fifo_req_get()) NRS start fifo request from 12345-0@lo, seq: 344 00000100:00100000:1.0:1776178686.818260:0:20137:0:(service.c:2324:ptlrpc_server_handle_request()) Handling RPC req@00000000b4857168 pname:cluuid+ref:pid:xid:nid:opc:job ll_mgs_0000:ac61f839-0e8e-4cac-9c59-08bcfbbad330+18:3355:x1862456998115712:12345-0@lo:400:kworker/u8:1.0 00000100:00100000:2.0:1776178686.818347:0:14440:0:(service.c:2372:ptlrpc_server_handle_request()) Handled RPC req@00000000952003a4 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_004:lustre-MDT0001-mdtlov_UUID+5:3355:x1862456998116160:12345-0@lo:400:kworker/u8:1.0 Request processed in 675us (1140us total) trans 0 rc 0/0 00000100:00100000:2.0:1776178686.818355:0:14440:0:(nrs_fifo.c:241:nrs_fifo_req_stop()) NRS stop fifo request from 12345-0@lo, seq: 890 00000100:00100000:2.0:1776178686.818379:0:6257:0:(service.c:2152:ptlrpc_server_handle_req_in()) got req x1862456998115968 00000100:00100000:2.0:1776178686.818398:0:6257:0:(service.c:2152:ptlrpc_server_handle_req_in()) got req x1862456998115840 00000100:00100000:2.0:1776178686.818410:0:6257:0:(service.c:2152:ptlrpc_server_handle_req_in()) got req x1862456998115776 00000100:00100000:2.0:1776178686.818419:0:6257:0:(service.c:2152:ptlrpc_server_handle_req_in()) got req x1862456998115904 00000100:00100000:2.0:1776178686.818426:0:6257:0:(service.c:2152:ptlrpc_server_handle_req_in()) got req x1862456998116224 00000100:00100000:2.0:1776178686.818436:0:6257:0:(nrs_fifo.c:179:nrs_fifo_req_get()) NRS start fifo request from 12345-0@lo, seq: 1211 00000100:00100000:2.0:1776178686.818440:0:6257:0:(service.c:2324:ptlrpc_server_handle_request()) Handling RPC req@0000000070c16828 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_002:lustre-MDT0001-lwp-OST0001_UUID+5:3354:x1862456998115968:12345-0@lo:400:kworker/u8:1.0 00000100:00100000:2.0:1776178686.818473:0:6257:0:(service.c:2372:ptlrpc_server_handle_request()) Handled RPC req@0000000070c16828 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_002:lustre-MDT0001-lwp-OST0001_UUID+5:3354:x1862456998115968:12345-0@lo:400:kworker/u8:1.0 Request processed in 32us (1802us total) trans 0 rc 0/0 00000100:00100000:2.0:1776178686.818482:0:6257:0:(nrs_fifo.c:241:nrs_fifo_req_stop()) NRS stop fifo request from 12345-0@lo, seq: 1211 00000100:00100000:2.0:1776178686.818493:0:6257:0:(nrs_fifo.c:179:nrs_fifo_req_get()) NRS start fifo request from 12345-0@lo, seq: 1212 00000100:00100000:2.0:1776178686.818509:0:6257:0:(service.c:2324:ptlrpc_server_handle_request()) Handling RPC req@0000000039f3bccf pname:cluuid+ref:pid:xid:nid:opc:job mdt00_002:lustre-MDT0001-lwp-OST0000_UUID+5:3355:x1862456998115840:12345-0@lo:400:kworker/u8:1.0 00000100:00100000:3.0:1776178686.818609:0:8250:0:(nrs_fifo.c:179:nrs_fifo_req_get()) NRS start fifo request from 12345-0@lo, seq: 7001 00000100:00100000:3.0:1776178686.818614:0:8250:0:(service.c:2324:ptlrpc_server_handle_request()) Handling RPC req@00000000c324c57b pname:cluuid+ref:pid:xid:nid:opc:job mdt00_003:0+-99:3352:x1862456998115776:12345-0@lo:38:kworker/u8:1.0 00010000:02000000:3.0:1776178686.818629:0:8250:0:(ldlm_lib.c:1139:target_handle_connect()) lustre-MDT0000: Not available for connect from 0@lo (stopping) 00000100:00100000:3.0:1776178686.818673:0:8250:0:(service.c:2372:ptlrpc_server_handle_request()) Handled RPC req@00000000c324c57b pname:cluuid+ref:pid:xid:nid:opc:job mdt00_003:0+-99:3352:x1862456998115776:12345-0@lo:38:kworker/u8:1.0 Request processed in 57us (1292us total) trans 0 rc -19/-19 00000100:00100000:3.0:1776178686.818679:0:8250:0:(nrs_fifo.c:241:nrs_fifo_req_stop()) NRS stop fifo request from 12345-0@lo, seq: 7001 00000100:00100000:3.0:1776178686.818694:0:8250:0:(nrs_fifo.c:179:nrs_fifo_req_get()) NRS start fifo request from 12345-0@lo, seq: 7002 00000100:00100000:3.0:1776178686.818696:0:8250:0:(service.c:2324:ptlrpc_server_handle_request()) Handling RPC req@00000000e787b61f pname:cluuid+ref:pid:xid:nid:opc:job mdt00_003:0+-99:3352:x1862456998115904:12345-0@lo:38:kworker/u8:1.0 00010000:02000000:3.0:1776178686.818702:0:8250:0:(ldlm_lib.c:1139:target_handle_connect()) lustre-MDT0000: Not available for connect from 0@lo (stopping) 00000100:00100000:3.0:1776178686.818724:0:8250:0:(service.c:2372:ptlrpc_server_handle_request()) Handled RPC req@00000000e787b61f pname:cluuid+ref:pid:xid:nid:opc:job mdt00_003:0+-99:3352:x1862456998115904:12345-0@lo:38:kworker/u8:1.0 Request processed in 27us (1312us total) trans 0 rc -19/-19 00000100:00100000:3.0:1776178686.818730:0:8250:0:(nrs_fifo.c:241:nrs_fifo_req_stop()) NRS stop fifo request from 12345-0@lo, seq: 7002 00000100:00100000:3.0:1776178686.818741:0:8250:0:(nrs_fifo.c:179:nrs_fifo_req_get()) NRS start fifo request from 12345-0@lo, seq: 7003 00000100:00100000:3.0:1776178686.818743:0:8250:0:(service.c:2324:ptlrpc_server_handle_request()) Handling RPC req@00000000028917ab pname:cluuid+ref:pid:xid:nid:opc:job mdt00_003:0+-99:3352:x1862456998116224:12345-0@lo:38:kworker/u8:1.0 00010000:02000000:3.0:1776178686.818749:0:8250:0:(ldlm_lib.c:1139:target_handle_connect()) lustre-MDT0000: Not available for connect from 0@lo (stopping) 00000100:00100000:3.0:1776178686.818768:0:8250:0:(service.c:2372:ptlrpc_server_handle_request()) Handled RPC req@00000000028917ab pname:cluuid+ref:pid:xid:nid:opc:job mdt00_003:0+-99:3352:x1862456998116224:12345-0@lo:38:kworker/u8:1.0 Request processed in 23us (1295us total) trans 0 rc -19/-19 00000100:00100000:3.0:1776178686.818773:0:8250:0:(nrs_fifo.c:241:nrs_fifo_req_stop()) NRS stop fifo request from 12345-0@lo, seq: 7003 00000100:00080000:3.0:1776178686.818811:0:3352:0:(import.c:86:import_set_state_nolock()) 0000000047601a98 lustre-MDT0000_UUID: changing import state from CONNECTING to DISCONN 00000100:00080000:3.0:1776178686.818815:0:3352:0:(import.c:1429:ptlrpc_connect_interpret()) recovery of lustre-MDT0000_UUID on 192.168.204.103@tcp failed (-19) 00000100:00100000:3.0:1776178686.818819:0:3352:0:(client.c:2215:ptlrpc_check_set()) Completed RPC req@00000000c37c187e pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:lustre-MDT0000-lwp-OST0000_UUID:3352:1862456998115776:0@lo:38:kworker/u8:1.0 00000100:00080000:3.0:1776178686.818828:0:3352:0:(import.c:86:import_set_state_nolock()) 00000000dfc8db13 lustre-MDT0000_UUID: changing import state from CONNECTING to DISCONN 00000100:00080000:3.0:1776178686.818830:0:3352:0:(import.c:1429:ptlrpc_connect_interpret()) recovery of lustre-MDT0000_UUID on 192.168.204.103@tcp failed (-19) 00000100:00100000:3.0:1776178686.818833:0:3352:0:(client.c:2215:ptlrpc_check_set()) Completed RPC req@00000000964f7a65 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:lustre-MDT0000-lwp-OST0001_UUID:3352:1862456998115904:0@lo:38:kworker/u8:1.0 00000100:00080000:3.0:1776178686.818840:0:3352:0:(import.c:86:import_set_state_nolock()) 00000000a7c55b1f lustre-MDT0000_UUID: changing import state from CONNECTING to DISCONN 00000100:00080000:3.0:1776178686.818843:0:3352:0:(import.c:1429:ptlrpc_connect_interpret()) recovery of lustre-MDT0000_UUID on 192.168.204.103@tcp failed (-19) 00000100:00100000:3.0:1776178686.818845:0:3352:0:(client.c:2215:ptlrpc_check_set()) Completed RPC req@0000000086665d0f pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:lustre-MDT0000-lwp-MDT0001_UUID:3352:1862456998116224:0@lo:38:kworker/u8:1.0 00000100:00100000:0.0:1776178686.819730:0:14128:0:(service.c:2152:ptlrpc_server_handle_req_in()) got req x1862456998116032 00000100:00100000:0.0:1776178686.819771:0:14128:0:(nrs_fifo.c:179:nrs_fifo_req_get()) NRS start fifo request from 12345-0@lo, seq: 802 00000100:00100000:0.0:1776178686.819776:0:14128:0:(service.c:2324:ptlrpc_server_handle_request()) Handling RPC req@0000000007d49440 pname:cluuid+ref:pid:xid:nid:opc:job mdt_out00_002:0+-99:3352:x1862456998116032:12345-0@lo:38:kworker/u8:1.0 00010000:02000000:0.0:1776178686.819788:0:14128:0:(ldlm_lib.c:1139:target_handle_connect()) lustre-MDT0000: Not available for connect from 0@lo (stopping) 00000100:00100000:0.0:1776178686.819845:0:14128:0:(service.c:2372:ptlrpc_server_handle_request()) Handled RPC req@0000000007d49440 pname:cluuid+ref:pid:xid:nid:opc:job mdt_out00_002:0+-99:3352:x1862456998116032:12345-0@lo:38:kworker/u8:1.0 Request processed in 68us (2401us total) trans 0 rc -19/-19 00000100:00100000:0.0:1776178686.819851:0:14128:0:(nrs_fifo.c:241:nrs_fifo_req_stop()) NRS stop fifo request from 12345-0@lo, seq: 802 00000100:00080000:3.0:1776178686.820635:0:3352:0:(import.c:86:import_set_state_nolock()) 00000000ba4f1821 lustre-MDT0000_UUID: changing import state from CONNECTING to DISCONN 00000100:00080000:3.0:1776178686.820641:0:3352:0:(import.c:1429:ptlrpc_connect_interpret()) recovery of lustre-MDT0000_UUID on 192.168.204.103@tcp failed (-19) 00000100:00100000:3.0:1776178686.820646:0:3352:0:(client.c:2215:ptlrpc_check_set()) Completed RPC req@000000009015f30b pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:lustre-MDT0001-mdtlov_UUID:3352:1862456998116032:0@lo:38:kworker/u8:1.0 00000100:00100000:3.0:1776178686.820704:0:3354:0:(client.c:2215:ptlrpc_check_set()) Completed RPC req@0000000009435d5b pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_01:lustre-MDT0001-lwp-OST0001_UUID:3354:1862456998115968:0@lo:400:kworker/u8:1.0 00000100:00100000:3.0:1776178686.820715:0:3354:0:(client.c:2215:ptlrpc_check_set()) Completed RPC req@00000000faeeda53 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_01:lustre-MDT0001-mdtlov_UUID:3354:1862456998116096:0@lo:400:kworker/u8:1.0 00000100:00100000:1.0:1776178686.825106:0:20137:0:(service.c:2372:ptlrpc_server_handle_request()) Handled RPC req@00000000b4857168 pname:cluuid+ref:pid:xid:nid:opc:job ll_mgs_0000:ac61f839-0e8e-4cac-9c59-08bcfbbad330+18:3355:x1862456998115712:12345-0@lo:400:kworker/u8:1.0 Request processed in 6833us (7926us total) trans 0 rc 0/0 00000100:00100000:2.0:1776178686.825106:0:6257:0:(service.c:2372:ptlrpc_server_handle_request()) Handled RPC req@0000000039f3bccf pname:cluuid+ref:pid:xid:nid:opc:job mdt00_002:lustre-MDT0001-lwp-OST0000_UUID+5:3355:x1862456998115840:12345-0@lo:400:kworker/u8:1.0 Request processed in 6596us (7842us total) trans 0 rc 0/0 00000100:00100000:1.0:1776178686.825131:0:20137:0:(nrs_fifo.c:241:nrs_fifo_req_stop()) NRS stop fifo request from 12345-0@lo, seq: 344 00000100:00100000:2.0:1776178686.825131:0:6257:0:(nrs_fifo.c:241:nrs_fifo_req_stop()) NRS stop fifo request from 12345-0@lo, seq: 1212 00000100:00100000:0.0:1776178686.825355:0:3355:0:(client.c:2215:ptlrpc_check_set()) Completed RPC req@00000000c237605d pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_02:lustre-MDT0001-lwp-OST0000_UUID:3355:1862456998115840:0@lo:400:kworker/u8:1.0 00000100:00100000:0.0:1776178686.825369:0:3355:0:(client.c:2215:ptlrpc_check_set()) Completed RPC req@00000000e82039bb pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_02:ac61f839-0e8e-4cac-9c59-08bcfbbad330:3355:1862456998115712:0@lo:400:kworker/u8:1.0 00000100:00100000:0.0:1776178686.825674:0:3355:0:(client.c:2215:ptlrpc_check_set()) Completed RPC req@0000000092e0e13d pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_02:lustre-MDT0001-mdtlov_UUID:3355:1862456998116160:0@lo:400:kworker/u8:1.0 00010000:00010000:1.0:1776178687.456363:0:9225:0:(ldlm_lockd.c:2792:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:2.0:1776178688.354378:0:3355:0:(ptlrpcd.c:415:ptlrpcd_check()) transfer 1 async RPCs [3->2] 00000100:00100000:2.0:1776178688.354402:0:3355:0:(client.c:1733:ptlrpc_send_new_req()) Sending RPC req@00000000b6e1d5c6 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_02:lustre-MDT0001-mdtlov_UUID:3355:1862456998116288:0@lo:13:osp-pre-1-1.0 00000100:00100000:2.0:1776178688.354480:0:3355:0:(events.c:361:request_in_callback()) peer: 12345-0@lo (source: 12345-0@lo) 00000100:00100000:3.0:1776178688.360524:0:9554:0:(service.c:2152:ptlrpc_server_handle_req_in()) got req x1862456998116288 00000100:00100000:3.0:1776178688.360686:0:9554:0:(nrs_fifo.c:179:nrs_fifo_req_get()) NRS start fifo request from 12345-0@lo, seq: 677 00000100:00100000:3.0:1776178688.360694:0:9554:0:(service.c:2324:ptlrpc_server_handle_request()) Handling RPC req@00000000050847c1 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_create00:lustre-MDT0001-mdtlov_UUID+5:3355:x1862456998116288:12345-0@lo:13:osp-pre-1-1.0 00002000:00000024:3.0:1776178688.360732:0:9554:0:(ofd_obd.c:740:ofd_statfs()) blocks cached 229376 granted 491746368 pending 0 free 3923689472 avail 3692167168 00000100:00100000:3.0:1776178688.360819:0:9554:0:(service.c:2372:ptlrpc_server_handle_request()) Handled RPC req@00000000050847c1 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_create00:lustre-MDT0001-mdtlov_UUID+5:3355:x1862456998116288:12345-0@lo:13:osp-pre-1-1.0 Request processed in 125us (6340us total) trans 0 rc 0/0 00000100:00100000:3.0:1776178688.360826:0:9554:0:(nrs_fifo.c:241:nrs_fifo_req_stop()) NRS stop fifo request from 12345-0@lo, seq: 677 00000100:00100000:2.0:1776178688.360919:0:3355:0:(client.c:2215:ptlrpc_check_set()) Completed RPC req@00000000b6e1d5c6 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_02:lustre-MDT0001-mdtlov_UUID:3355:1862456998116288:0@lo:13:osp-pre-1-1.0 00010000:00010000:2.0:1776178688.480894:0:6244:0:(ldlm_lockd.c:2792:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:3.0:1776178688.864624:0:3353:0:(client.c:1733:ptlrpc_send_new_req()) Sending RPC req@000000009751a398 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_00:lustre-MDT0001-mdtlov_UUID:3353:1862456998116352:0@lo:13:osp-pre-0-1.0 00000100:00100000:3.0:1776178688.864695:0:3353:0:(events.c:361:request_in_callback()) peer: 12345-0@lo (source: 12345-0@lo) 00000100:00100000:2.0:1776178688.865054:0:9554:0:(service.c:2152:ptlrpc_server_handle_req_in()) got req x1862456998116352 00000100:00100000:2.0:1776178688.865125:0:9554:0:(nrs_fifo.c:179:nrs_fifo_req_get()) NRS start fifo request from 12345-0@lo, seq: 678 00000100:00100000:2.0:1776178688.865131:0:9554:0:(service.c:2324:ptlrpc_server_handle_request()) Handling RPC req@0000000065f62d8c pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_create00:lustre-MDT0001-mdtlov_UUID+5:3353:x1862456998116352:12345-0@lo:13:osp-pre-0-1.0 00002000:00000024:2.0:1776178688.865159:0:9554:0:(ofd_obd.c:740:ofd_statfs()) blocks cached 315392 granted 484883776 pending 0 free 3923689472 avail 3692167168 00000100:00100000:2.0:1776178688.865214:0:9554:0:(service.c:2372:ptlrpc_server_handle_request()) Handled RPC req@0000000065f62d8c pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_create00:lustre-MDT0001-mdtlov_UUID+5:3353:x1862456998116352:12345-0@lo:13:osp-pre-0-1.0 Request processed in 74us (511us total) trans 0 rc 0/0 00000100:00100000:2.0:1776178688.865221:0:9554:0:(nrs_fifo.c:241:nrs_fifo_req_stop()) NRS stop fifo request from 12345-0@lo, seq: 678 00000100:00100000:3.0:1776178688.865304:0:3353:0:(client.c:2215:ptlrpc_check_set()) Completed RPC req@000000009751a398 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_00:lustre-MDT0001-mdtlov_UUID:3353:1862456998116352:0@lo:13:osp-pre-0-1.0 00010000:00010000:1.0:1776178689.504329:0:6245:0:(ldlm_lockd.c:2792:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:3.0:1776178690.344352:0:3346:0:(events.c:361:request_in_callback()) peer: 12345-192.168.204.3@tcp (source: 12345-192.168.204.3@tcp) 00000100:00100000:3.0:1776178690.344399:0:3346:0:(events.c:361:request_in_callback()) peer: 12345-192.168.204.3@tcp (source: 12345-192.168.204.3@tcp) 00000100:00100000:3.0:1776178690.344451:0:3346:0:(events.c:361:request_in_callback()) peer: 12345-192.168.204.3@tcp (source: 12345-192.168.204.3@tcp) 00000100:00100000:3.0:1776178690.344485:0:3346:0:(events.c:361:request_in_callback()) peer: 12345-192.168.204.3@tcp (source: 12345-192.168.204.3@tcp) 00000100:00100000:2.0:1776178690.344520:0:6257:0:(service.c:2152:ptlrpc_server_handle_req_in()) got req x1862456979139520 00000100:00100000:0.0:1776178690.344533:0:20759:0:(service.c:2152:ptlrpc_server_handle_req_in()) got req x1862456979139584 00000100:00100000:3.0:1776178690.344552:0:3346:0:(events.c:361:request_in_callback()) peer: 12345-192.168.204.3@tcp (source: 12345-192.168.204.3@tcp) 00000100:00100000:3.0:1776178690.344595:0:20762:0:(service.c:2152:ptlrpc_server_handle_req_in()) got req x1862456979139648 00000100:00100000:3.0:1776178690.344627:0:20762:0:(nrs_fifo.c:179:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.204.3@tcp, seq: 891 00000100:00100000:3.0:1776178690.344634:0:20762:0:(service.c:2324:ptlrpc_server_handle_request()) Handling RPC req@00000000b6e1d5c6 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_013:b50963bd-8eba-4432-9a54-8888d00e3fa1+9:2273:x1862456979139648:12345-192.168.204.3@tcp:400:kworker/u8:3.0 00000100:00100000:3.0:1776178690.344702:0:20762:0:(service.c:2372:ptlrpc_server_handle_request()) Handled RPC req@00000000b6e1d5c6 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_013:b50963bd-8eba-4432-9a54-8888d00e3fa1+9:2273:x1862456979139648:12345-192.168.204.3@tcp:400:kworker/u8:3.0 Request processed in 68us (217us total) trans 0 rc 0/0 00000100:00100000:3.0:1776178690.344710:0:20762:0:(nrs_fifo.c:241:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.204.3@tcp, seq: 891 00000100:00100000:0.0:1776178690.345232:0:20759:0:(nrs_fifo.c:179:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.204.3@tcp, seq: 892 00000100:00100000:0.0:1776178690.345237:0:20759:0:(service.c:2324:ptlrpc_server_handle_request()) Handling RPC req@0000000034bc0662 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_010:b50963bd-8eba-4432-9a54-8888d00e3fa1+8:2273:x1862456979139584:12345-192.168.204.3@tcp:400:kworker/u8:3.0 00000100:00100000:0.0:1776178690.345271:0:20759:0:(service.c:2372:ptlrpc_server_handle_request()) Handled RPC req@0000000034bc0662 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_010:b50963bd-8eba-4432-9a54-8888d00e3fa1+8:2273:x1862456979139584:12345-192.168.204.3@tcp:400:kworker/u8:3.0 Request processed in 33us (821us total) trans 0 rc 0/0 00000100:00100000:0.0:1776178690.345277:0:20759:0:(nrs_fifo.c:241:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.204.3@tcp, seq: 892 00000100:00100000:1.0:1776178690.345563:0:20137:0:(service.c:2152:ptlrpc_server_handle_req_in()) got req x1862456979139392 00000100:00100000:2.0:1776178690.345591:0:6257:0:(service.c:2152:ptlrpc_server_handle_req_in()) got req x1862456979139456 00000100:00100000:1.0:1776178690.345599:0:20137:0:(nrs_fifo.c:179:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.204.3@tcp, seq: 345 00000100:00100000:1.0:1776178690.345605:0:20137:0:(service.c:2324:ptlrpc_server_handle_request()) Handling RPC req@0000000088793b53 pname:cluuid+ref:pid:xid:nid:opc:job ll_mgs_0000:5b5c66b1-1a2f-4651-8548-0c75b9a38a94+9:2273:x1862456979139392:12345-192.168.204.3@tcp:400:kworker/u8:3.0 00000100:00100000:2.0:1776178690.345625:0:6257:0:(nrs_fifo.c:179:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.204.3@tcp, seq: 1213 00000100:00100000:2.0:1776178690.345632:0:6257:0:(service.c:2324:ptlrpc_server_handle_request()) Handling RPC req@000000006890aba2 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_002:b50963bd-8eba-4432-9a54-8888d00e3fa1+6:2273:x1862456979139520:12345-192.168.204.3@tcp:400:kworker/u8:3.0 00000100:00100000:1.0:1776178690.345638:0:20137:0:(service.c:2372:ptlrpc_server_handle_request()) Handled RPC req@0000000088793b53 pname:cluuid+ref:pid:xid:nid:opc:job ll_mgs_0000:5b5c66b1-1a2f-4651-8548-0c75b9a38a94+9:2273:x1862456979139392:12345-192.168.204.3@tcp:400:kworker/u8:3.0 Request processed in 30us (1291us total) trans 0 rc 0/0 00000100:00100000:1.0:1776178690.345646:0:20137:0:(nrs_fifo.c:241:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.204.3@tcp, seq: 345 00000100:00100000:3.0:1776178690.345664:0:11281:0:(nrs_fifo.c:179:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.204.3@tcp, seq: 7004 00000100:00100000:3.0:1776178690.345670:0:11281:0:(service.c:2324:ptlrpc_server_handle_request()) Handling RPC req@0000000092e0e13d pname:cluuid+ref:pid:xid:nid:opc:job mdt00_004:0+-99:2269:x1862456979139456:12345-192.168.204.3@tcp:38:kworker/u8:3.0 00000100:00100000:2.0:1776178690.345680:0:6257:0:(service.c:2372:ptlrpc_server_handle_request()) Handled RPC req@000000006890aba2 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_002:b50963bd-8eba-4432-9a54-8888d00e3fa1+6:2273:x1862456979139520:12345-192.168.204.3@tcp:400:kworker/u8:3.0 Request processed in 47us (1280us total) trans 0 rc 0/0 00010000:02000000:3.0:1776178690.345683:0:11281:0:(ldlm_lib.c:1139:target_handle_connect()) lustre-MDT0000: Not available for connect from 192.168.204.3@tcp (stopping) 00000100:00100000:2.0:1776178690.345693:0:6257:0:(nrs_fifo.c:241:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.204.3@tcp, seq: 1213 00000100:00100000:3.0:1776178690.345712:0:11281:0:(service.c:2372:ptlrpc_server_handle_request()) Handled RPC req@0000000092e0e13d pname:cluuid+ref:pid:xid:nid:opc:job mdt00_004:0+-99:2269:x1862456979139456:12345-192.168.204.3@tcp:38:kworker/u8:3.0 Request processed in 41us (1160us total) trans 0 rc -19/-19 00000100:00100000:3.0:1776178690.345719:0:11281:0:(nrs_fifo.c:241:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.204.3@tcp, seq: 7004 00010000:00010000:2.0:1776178690.531050:0:9224:0:(ldlm_lockd.c:2792:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00010000:00010000:1.0:1776178691.554092:0:9225:0:(ldlm_lockd.c:2792:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00080000:0.0:1776178691.936444:0:22249:0:(pinger.c:251:ptlrpc_pinger_process_import()) lustre-MDT0000-lwp-OST0000_UUID->lustre-MDT0000_UUID: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00080000:0.0:1776178691.936453:0:22249:0:(recover.c:57:ptlrpc_initiate_recovery()) lustre-MDT0000_UUID: starting recovery 00000100:00080000:0.0:1776178691.936456:0:22249:0:(import.c:86:import_set_state_nolock()) 0000000047601a98 lustre-MDT0000_UUID: changing import state from DISCONN to CONNECTING 00000100:00080000:0.0:1776178691.936465:0:22249:0:(import.c:534:import_select_connection()) lustre-MDT0000-lwp-OST0000: connect to NID 0@lo last attempt 1394 00000100:00080000:0.0:1776178691.936469:0:22249:0:(import.c:581:import_select_connection()) lustre-MDT0000-lwp-OST0000: tried all connections, increasing latency to 21s 00000100:00080000:0.0:1776178691.936475:0:22249:0:(import.c:615:import_select_connection()) lustre-MDT0000-lwp-OST0000: import 0000000047601a98 using connection 192.168.204.103@tcp/0@lo 00000100:00100000:0.0:1776178691.936502:0:22249:0:(import.c:807:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5) req@00000000fd2b8392 x1862456998116480/t0(0) o38->lustre-MDT0000-lwp-OST0000@0@lo:12/10 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' 00000100:00080000:0.0:1776178691.936553:0:22249:0:(pinger.c:251:ptlrpc_pinger_process_import()) lustre-MDT0000-lwp-OST0001_UUID->lustre-MDT0000_UUID: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00080000:0.0:1776178691.936557:0:22249:0:(recover.c:57:ptlrpc_initiate_recovery()) lustre-MDT0000_UUID: starting recovery 00000100:00080000:0.0:1776178691.936558:0:22249:0:(import.c:86:import_set_state_nolock()) 00000000dfc8db13 lustre-MDT0000_UUID: changing import state from DISCONN to CONNECTING 00000100:00080000:0.0:1776178691.936561:0:22249:0:(import.c:534:import_select_connection()) lustre-MDT0000-lwp-OST0001: connect to NID 0@lo last attempt 1394 00000100:00080000:0.0:1776178691.936563:0:22249:0:(import.c:581:import_select_connection()) lustre-MDT0000-lwp-OST0001: tried all connections, increasing latency to 21s 00000100:00080000:0.0:1776178691.936566:0:22249:0:(import.c:615:import_select_connection()) lustre-MDT0000-lwp-OST0001: import 00000000dfc8db13 using connection 192.168.204.103@tcp/0@lo 00000100:00100000:0.0:1776178691.936575:0:22249:0:(import.c:807:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5) req@000000006a3d91b0 x1862456998116608/t0(0) o38->lustre-MDT0000-lwp-OST0001@0@lo:12/10 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' 00000100:00080000:0.0:1776178691.936589:0:22249:0:(pinger.c:251:ptlrpc_pinger_process_import()) lustre-MDT0001-mdtlov_UUID->lustre-MDT0000_UUID: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00080000:0.0:1776178691.936592:0:22249:0:(recover.c:57:ptlrpc_initiate_recovery()) lustre-MDT0000_UUID: starting recovery 00000100:00080000:0.0:1776178691.936593:0:22249:0:(import.c:86:import_set_state_nolock()) 00000000ba4f1821 lustre-MDT0000_UUID: changing import state from DISCONN to CONNECTING 00000100:00080000:0.0:1776178691.936596:0:22249:0:(import.c:534:import_select_connection()) lustre-MDT0000-osp-MDT0001: connect to NID 0@lo last attempt 1394 00000100:00080000:0.0:1776178691.936598:0:22249:0:(import.c:581:import_select_connection()) lustre-MDT0000-osp-MDT0001: tried all connections, increasing latency to 21s 00000100:00080000:0.0:1776178691.936601:0:22249:0:(import.c:615:import_select_connection()) lustre-MDT0000-osp-MDT0001: import 00000000ba4f1821 using connection 192.168.204.103@tcp/0@lo 00000100:00100000:0.0:1776178691.936610:0:22249:0:(import.c:807:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5) req@0000000004dd3b9b x1862456998116736/t0(0) o38->lustre-MDT0000-osp-MDT0001@0@lo:24/4 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' 00000100:00080000:0.0:1776178691.936629:0:22249:0:(pinger.c:251:ptlrpc_pinger_process_import()) lustre-MDT0000-lwp-MDT0001_UUID->lustre-MDT0000_UUID: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00080000:0.0:1776178691.936632:0:22249:0:(recover.c:57:ptlrpc_initiate_recovery()) lustre-MDT0000_UUID: starting recovery 00000100:00080000:0.0:1776178691.936633:0:22249:0:(import.c:86:import_set_state_nolock()) 00000000a7c55b1f lustre-MDT0000_UUID: changing import state from DISCONN to CONNECTING 00000100:00080000:0.0:1776178691.936637:0:22249:0:(import.c:534:import_select_connection()) lustre-MDT0000-lwp-MDT0001: connect to NID 0@lo last attempt 1394 00000100:00080000:0.0:1776178691.936638:0:22249:0:(import.c:581:import_select_connection()) lustre-MDT0000-lwp-MDT0001: tried all connections, increasing latency to 21s 00000100:00080000:0.0:1776178691.936641:0:22249:0:(import.c:615:import_select_connection()) lustre-MDT0000-lwp-MDT0001: import 00000000a7c55b1f using connection 192.168.204.103@tcp/0@lo 00000100:00100000:0.0:1776178691.936649:0:22249:0:(import.c:807:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5) req@0000000012b22334 x1862456998116928/t0(0) o38->lustre-MDT0000-lwp-MDT0001@0@lo:12/10 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' 00000100:00100000:0.0:1776178691.936692:0:3356:0:(client.c:1733:ptlrpc_send_new_req()) Sending RPC req@000000002300ff10 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_03:lustre-MDT0001-lwp-OST0001_UUID:3356:1862456998116672:0@lo:400:kworker/u8:0.0 00000100:00100000:0.0:1776178691.936758:0:3356:0:(events.c:361:request_in_callback()) peer: 12345-0@lo (source: 12345-0@lo) 00000100:00100000:0.0:1776178691.936794:0:3356:0:(ptlrpcd.c:415:ptlrpcd_check()) transfer 1 async RPCs [2->3] 00000100:00100000:0.0:1776178691.936799:0:3356:0:(client.c:1733:ptlrpc_send_new_req()) Sending RPC req@0000000042de09e9 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_03:lustre-MDT0001-lwp-OST0000_UUID:3356:1862456998116544:0@lo:400:kworker/u8:0.0 00000100:00100000:0.0:1776178691.936821:0:3356:0:(events.c:361:request_in_callback()) peer: 12345-0@lo (source: 12345-0@lo) 00000100:00100000:0.0:1776178691.936856:0:6255:0:(service.c:2152:ptlrpc_server_handle_req_in()) got req x1862456998116672 00000100:00100000:2.0:1776178691.936863:0:3352:0:(client.c:1733:ptlrpc_send_new_req()) Sending RPC req@00000000fd2b8392 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:lustre-MDT0000-lwp-OST0000_UUID:3352:1862456998116480:0@lo:38:kworker/u8:0.0 00000100:00100000:0.0:1776178691.936899:0:6255:0:(service.c:2152:ptlrpc_server_handle_req_in()) got req x1862456998116544 00000100:00100000:2.0:1776178691.936899:0:3352:0:(events.c:361:request_in_callback()) peer: 12345-0@lo (source: 12345-0@lo) 00000100:00100000:0.0:1776178691.936921:0:6255:0:(service.c:2152:ptlrpc_server_handle_req_in()) got req x1862456998116480 00000100:00100000:2.0:1776178691.936922:0:3352:0:(client.c:1733:ptlrpc_send_new_req()) Sending RPC req@000000006a3d91b0 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:lustre-MDT0000-lwp-OST0001_UUID:3352:1862456998116608:0@lo:38:kworker/u8:0.0 00000100:00100000:0.0:1776178691.936938:0:6255:0:(nrs_fifo.c:179:nrs_fifo_req_get()) NRS start fifo request from 12345-0@lo, seq: 1214 00000100:00100000:0.0:1776178691.936944:0:6255:0:(service.c:2324:ptlrpc_server_handle_request()) Handling RPC req@0000000076452107 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_000:lustre-MDT0001-lwp-OST0001_UUID+5:3356:x1862456998116672:12345-0@lo:400:kworker/u8:0.0 00000100:00100000:2.0:1776178691.936951:0:3352:0:(events.c:361:request_in_callback()) peer: 12345-0@lo (source: 12345-0@lo) 00000100:00100000:2.0:1776178691.936960:0:3352:0:(client.c:1733:ptlrpc_send_new_req()) Sending RPC req@0000000004dd3b9b pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:lustre-MDT0001-mdtlov_UUID:3352:1862456998116736:0@lo:38:kworker/u8:0.0 00000100:00100000:2.0:1776178691.937000:0:3352:0:(events.c:361:request_in_callback()) peer: 12345-0@lo (source: 12345-0@lo) 00000100:00100000:0.0:1776178691.937008:0:6255:0:(service.c:2372:ptlrpc_server_handle_request()) Handled RPC req@0000000076452107 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_000:lustre-MDT0001-lwp-OST0001_UUID+5:3356:x1862456998116672:12345-0@lo:400:kworker/u8:0.0 Request processed in 63us (250us total) trans 0 rc 0/0 00000100:00100000:0.0:1776178691.937018:0:6255:0:(nrs_fifo.c:241:nrs_fifo_req_stop()) NRS stop fifo request from 12345-0@lo, seq: 1214 00000100:00100000:2.0:1776178691.937018:0:3352:0:(client.c:1733:ptlrpc_send_new_req()) Sending RPC req@0000000012b22334 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:lustre-MDT0000-lwp-MDT0001_UUID:3352:1862456998116928:0@lo:38:kworker/u8:0.0 00000100:00100000:0.0:1776178691.937039:0:6255:0:(service.c:2152:ptlrpc_server_handle_req_in()) got req x1862456998116608 00000100:00100000:2.0:1776178691.937045:0:3352:0:(events.c:361:request_in_callback()) peer: 12345-0@lo (source: 12345-0@lo) 00000100:00100000:0.0:1776178691.937056:0:6255:0:(service.c:2152:ptlrpc_server_handle_req_in()) got req x1862456998116928 00000100:00100000:2.0:1776178691.937108:0:6257:0:(nrs_fifo.c:179:nrs_fifo_req_get()) NRS start fifo request from 12345-0@lo, seq: 1215 00000100:00100000:0.0:1776178691.937115:0:6255:0:(nrs_fifo.c:179:nrs_fifo_req_get()) NRS start fifo request from 12345-0@lo, seq: 7005 00000100:00100000:2.0:1776178691.937117:0:6257:0:(service.c:2324:ptlrpc_server_handle_request()) Handling RPC req@00000000a08c40f7 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_002:lustre-MDT0001-lwp-OST0000_UUID+5:3356:x1862456998116544:12345-0@lo:400:kworker/u8:0.0 00000100:00100000:0.0:1776178691.937123:0:6255:0:(service.c:2324:ptlrpc_server_handle_request()) Handling RPC req@00000000ead84f6b pname:cluuid+ref:pid:xid:nid:opc:job mdt00_000:0+-99:3352:x1862456998116480:12345-0@lo:38:kworker/u8:0.0 00010000:02000000:0.0:1776178691.937136:0:6255:0:(ldlm_lib.c:1139:target_handle_connect()) lustre-MDT0000: Not available for connect from 0@lo (stopping) 00000100:00100000:2.0:1776178691.937151:0:6257:0:(service.c:2372:ptlrpc_server_handle_request()) Handled RPC req@00000000a08c40f7 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_002:lustre-MDT0001-lwp-OST0000_UUID+5:3356:x1862456998116544:12345-0@lo:400:kworker/u8:0.0 Request processed in 36us (329us total) trans 0 rc 0/0 00000100:00100000:2.0:1776178691.937158:0:6257:0:(nrs_fifo.c:241:nrs_fifo_req_stop()) NRS stop fifo request from 12345-0@lo, seq: 1215 00000100:00100000:0.0:1776178691.937177:0:6255:0:(service.c:2372:ptlrpc_server_handle_request()) Handled RPC req@00000000ead84f6b pname:cluuid+ref:pid:xid:nid:opc:job mdt00_000:0+-99:3352:x1862456998116480:12345-0@lo:38:kworker/u8:0.0 Request processed in 53us (277us total) trans 0 rc -19/-19 00000100:00100000:2.0:1776178691.937178:0:6257:0:(nrs_fifo.c:179:nrs_fifo_req_get()) NRS start fifo request from 12345-0@lo, seq: 7006 00000100:00100000:2.0:1776178691.937185:0:6257:0:(service.c:2324:ptlrpc_server_handle_request()) Handling RPC req@00000000314f395b pname:cluuid+ref:pid:xid:nid:opc:job mdt00_002:0+-99:3352:x1862456998116608:12345-0@lo:38:kworker/u8:0.0 00000100:00100000:0.0:1776178691.937189:0:6255:0:(nrs_fifo.c:241:nrs_fifo_req_stop()) NRS stop fifo request from 12345-0@lo, seq: 7005 00010000:02000000:2.0:1776178691.937197:0:6257:0:(ldlm_lib.c:1139:target_handle_connect()) lustre-MDT0000: Not available for connect from 0@lo (stopping) 00000100:00100000:0.0:1776178691.937206:0:6255:0:(nrs_fifo.c:179:nrs_fifo_req_get()) NRS start fifo request from 12345-0@lo, seq: 7007 00000100:00100000:0.0:1776178691.937211:0:6255:0:(service.c:2324:ptlrpc_server_handle_request()) Handling RPC req@00000000033734be pname:cluuid+ref:pid:xid:nid:opc:job mdt00_000:0+-99:3352:x1862456998116928:12345-0@lo:38:kworker/u8:0.0 00010000:02000000:0.0:1776178691.937219:0:6255:0:(ldlm_lib.c:1139:target_handle_connect()) lustre-MDT0000: Not available for connect from 0@lo (stopping) 00000100:00100000:2.0:1776178691.937226:0:6257:0:(service.c:2372:ptlrpc_server_handle_request()) Handled RPC req@00000000314f395b pname:cluuid+ref:pid:xid:nid:opc:job mdt00_002:0+-99:3352:x1862456998116608:12345-0@lo:38:kworker/u8:0.0 Request processed in 40us (277us total) trans 0 rc -19/-19 00000100:00100000:2.0:1776178691.937233:0:6257:0:(nrs_fifo.c:241:nrs_fifo_req_stop()) NRS stop fifo request from 12345-0@lo, seq: 7006 00000100:00100000:0.0:1776178691.937251:0:6255:0:(service.c:2372:ptlrpc_server_handle_request()) Handled RPC req@00000000033734be pname:cluuid+ref:pid:xid:nid:opc:job mdt00_000:0+-99:3352:x1862456998116928:12345-0@lo:38:kworker/u8:0.0 Request processed in 39us (205us total) trans 0 rc -19/-19 00000100:00100000:0.0:1776178691.937259:0:6255:0:(nrs_fifo.c:241:nrs_fifo_req_stop()) NRS stop fifo request from 12345-0@lo, seq: 7007 00000100:00080000:2.0:1776178691.937273:0:3352:0:(import.c:86:import_set_state_nolock()) 0000000047601a98 lustre-MDT0000_UUID: changing import state from CONNECTING to DISCONN 00000100:00080000:2.0:1776178691.937278:0:3352:0:(import.c:1429:ptlrpc_connect_interpret()) recovery of lustre-MDT0000_UUID on 192.168.204.103@tcp failed (-19) 00000100:00100000:2.0:1776178691.937283:0:3352:0:(client.c:2215:ptlrpc_check_set()) Completed RPC req@00000000fd2b8392 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:lustre-MDT0000-lwp-OST0000_UUID:3352:1862456998116480:0@lo:38:kworker/u8:0.0 00000100:00100000:0.0:1776178691.937295:0:3356:0:(client.c:2215:ptlrpc_check_set()) Completed RPC req@0000000042de09e9 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_03:lustre-MDT0001-lwp-OST0000_UUID:3356:1862456998116544:0@lo:400:kworker/u8:0.0 00000100:00080000:2.0:1776178691.937296:0:3352:0:(import.c:86:import_set_state_nolock()) 00000000dfc8db13 lustre-MDT0000_UUID: changing import state from CONNECTING to DISCONN 00000100:00080000:2.0:1776178691.937300:0:3352:0:(import.c:1429:ptlrpc_connect_interpret()) recovery of lustre-MDT0000_UUID on 192.168.204.103@tcp failed (-19) 00000100:00100000:2.0:1776178691.937306:0:3352:0:(client.c:2215:ptlrpc_check_set()) Completed RPC req@000000006a3d91b0 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:lustre-MDT0000-lwp-OST0001_UUID:3352:1862456998116608:0@lo:38:kworker/u8:0.0 00000100:00100000:0.0:1776178691.937314:0:3356:0:(client.c:2215:ptlrpc_check_set()) Completed RPC req@000000002300ff10 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_03:lustre-MDT0001-lwp-OST0001_UUID:3356:1862456998116672:0@lo:400:kworker/u8:0.0 00000100:00080000:2.0:1776178691.937321:0:3352:0:(import.c:86:import_set_state_nolock()) 00000000a7c55b1f lustre-MDT0000_UUID: changing import state from CONNECTING to DISCONN 00000100:00080000:2.0:1776178691.937325:0:3352:0:(import.c:1429:ptlrpc_connect_interpret()) recovery of lustre-MDT0000_UUID on 192.168.204.103@tcp failed (-19) 00000100:00100000:2.0:1776178691.937330:0:3352:0:(client.c:2215:ptlrpc_check_set()) Completed RPC req@0000000012b22334 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:lustre-MDT0000-lwp-MDT0001_UUID:3352:1862456998116928:0@lo:38:kworker/u8:0.0 00000100:00100000:1.0:1776178691.937791:0:3354:0:(client.c:1733:ptlrpc_send_new_req()) Sending RPC req@00000000ea88a302 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_01:ac61f839-0e8e-4cac-9c59-08bcfbbad330:3354:1862456998116416:0@lo:400:kworker/u8:0.0 00000100:00100000:1.0:1776178691.937835:0:3354:0:(events.c:361:request_in_callback()) peer: 12345-0@lo (source: 12345-0@lo) 00000100:00100000:1.0:1776178691.937868:0:3354:0:(client.c:1733:ptlrpc_send_new_req()) Sending RPC req@000000004828c19d pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_01:lustre-MDT0001-mdtlov_UUID:3354:1862456998116864:0@lo:400:kworker/u8:0.0 00000100:00100000:1.0:1776178691.937898:0:3354:0:(events.c:361:request_in_callback()) peer: 12345-0@lo (source: 12345-0@lo) 00000100:00100000:3.0:1776178691.937903:0:15213:0:(service.c:2152:ptlrpc_server_handle_req_in()) got req x1862456998116736 00000100:00100000:1.0:1776178691.937928:0:3354:0:(ptlrpcd.c:415:ptlrpcd_check()) transfer 1 async RPCs [0->1] 00000100:00100000:3.0:1776178691.937934:0:15213:0:(nrs_fifo.c:179:nrs_fifo_req_get()) NRS start fifo request from 12345-0@lo, seq: 803 00000100:00100000:1.0:1776178691.937936:0:3354:0:(client.c:1733:ptlrpc_send_new_req()) Sending RPC req@0000000035819321 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_01:lustre-MDT0001-mdtlov_UUID:3354:1862456998116800:0@lo:400:kworker/u8:0.0 00000100:00100000:3.0:1776178691.937942:0:15213:0:(service.c:2324:ptlrpc_server_handle_request()) Handling RPC req@000000001f88d899 pname:cluuid+ref:pid:xid:nid:opc:job mdt_out00_003:0+-99:3352:x1862456998116736:12345-0@lo:38:kworker/u8:0.0 00010000:02000000:3.0:1776178691.937953:0:15213:0:(ldlm_lib.c:1139:target_handle_connect()) lustre-MDT0000: Not available for connect from 0@lo (stopping) 00000100:00100000:1.0:1776178691.937964:0:3354:0:(events.c:361:request_in_callback()) peer: 12345-0@lo (source: 12345-0@lo) 00000100:00100000:3.0:1776178691.937996:0:15213:0:(service.c:2372:ptlrpc_server_handle_request()) Handled RPC req@000000001f88d899 pname:cluuid+ref:pid:xid:nid:opc:job mdt_out00_003:0+-99:3352:x1862456998116736:12345-0@lo:38:kworker/u8:0.0 Request processed in 55us (996us total) trans 0 rc -19/-19 00000100:00100000:1.0:1776178691.938004:0:20759:0:(service.c:2152:ptlrpc_server_handle_req_in()) got req x1862456998116864 00000100:00100000:3.0:1776178691.938004:0:15213:0:(nrs_fifo.c:241:nrs_fifo_req_stop()) NRS stop fifo request from 12345-0@lo, seq: 803 00000100:00100000:1.0:1776178691.938040:0:20759:0:(service.c:2152:ptlrpc_server_handle_req_in()) got req x1862456998116800 00000100:00100000:3.0:1776178691.938043:0:20762:0:(nrs_fifo.c:179:nrs_fifo_req_get()) NRS start fifo request from 12345-0@lo, seq: 893 00000100:00100000:3.0:1776178691.938049:0:20762:0:(service.c:2324:ptlrpc_server_handle_request()) Handling RPC req@0000000010d80ea5 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_013:lustre-MDT0001-mdtlov_UUID+5:3354:x1862456998116864:12345-0@lo:400:kworker/u8:0.0 00000100:00100000:1.0:1776178691.938081:0:20759:0:(nrs_fifo.c:179:nrs_fifo_req_get()) NRS start fifo request from 12345-0@lo, seq: 894 00000100:00100000:1.0:1776178691.938086:0:20759:0:(service.c:2324:ptlrpc_server_handle_request()) Handling RPC req@000000001b4f0008 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_010:lustre-MDT0001-mdtlov_UUID+5:3354:x1862456998116800:12345-0@lo:400:kworker/u8:0.0 00000100:00100000:3.0:1776178691.938132:0:20762:0:(service.c:2372:ptlrpc_server_handle_request()) Handled RPC req@0000000010d80ea5 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_013:lustre-MDT0001-mdtlov_UUID+5:3354:x1862456998116864:12345-0@lo:400:kworker/u8:0.0 Request processed in 82us (233us total) trans 0 rc 0/0 00000100:00100000:1.0:1776178691.938134:0:20759:0:(service.c:2372:ptlrpc_server_handle_request()) Handled RPC req@000000001b4f0008 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_010:lustre-MDT0001-mdtlov_UUID+5:3354:x1862456998116800:12345-0@lo:400:kworker/u8:0.0 Request processed in 46us (169us total) trans 0 rc 0/0 00000100:00100000:3.0:1776178691.938147:0:20762:0:(nrs_fifo.c:241:nrs_fifo_req_stop()) NRS stop fifo request from 12345-0@lo, seq: 893 00000100:00100000:3.0:1776178691.938179:0:3354:0:(client.c:2215:ptlrpc_check_set()) Completed RPC req@0000000035819321 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_01:lustre-MDT0001-mdtlov_UUID:3354:1862456998116800:0@lo:400:kworker/u8:0.0 00000100:00100000:3.0:1776178691.938191:0:3354:0:(client.c:2215:ptlrpc_check_set()) Completed RPC req@000000004828c19d pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_01:lustre-MDT0001-mdtlov_UUID:3354:1862456998116864:0@lo:400:kworker/u8:0.0 00000100:00100000:0.0:1776178691.938197:0:20140:0:(service.c:2152:ptlrpc_server_handle_req_in()) got req x1862456998116416 00000100:00100000:0.0:1776178691.938230:0:20140:0:(nrs_fifo.c:179:nrs_fifo_req_get()) NRS start fifo request from 12345-0@lo, seq: 346 00000100:00100000:0.0:1776178691.938235:0:20140:0:(service.c:2324:ptlrpc_server_handle_request()) Handling RPC req@000000004c7a25f9 pname:cluuid+ref:pid:xid:nid:opc:job ll_mgs_0002:ac61f839-0e8e-4cac-9c59-08bcfbbad330+18:3354:x1862456998116416:12345-0@lo:400:kworker/u8:0.0 00000100:00100000:0.0:1776178691.938286:0:20140:0:(service.c:2372:ptlrpc_server_handle_request()) Handled RPC req@000000004c7a25f9 pname:cluuid+ref:pid:xid:nid:opc:job ll_mgs_0002:ac61f839-0e8e-4cac-9c59-08bcfbbad330+18:3354:x1862456998116416:12345-0@lo:400:kworker/u8:0.0 Request processed in 51us (451us total) trans 0 rc 0/0 00000100:00100000:0.0:1776178691.938294:0:20140:0:(nrs_fifo.c:241:nrs_fifo_req_stop()) NRS stop fifo request from 12345-0@lo, seq: 346 00000100:00080000:2.0:1776178691.938321:0:3352:0:(import.c:86:import_set_state_nolock()) 00000000ba4f1821 lustre-MDT0000_UUID: changing import state from CONNECTING to DISCONN 00000100:00080000:2.0:1776178691.938326:0:3352:0:(import.c:1429:ptlrpc_connect_interpret()) recovery of lustre-MDT0000_UUID on 192.168.204.103@tcp failed (-19) 00000100:00100000:2.0:1776178691.938331:0:3352:0:(client.c:2215:ptlrpc_check_set()) Completed RPC req@0000000004dd3b9b pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:lustre-MDT0001-mdtlov_UUID:3352:1862456998116736:0@lo:38:kworker/u8:0.0 00000100:00100000:1.0:1776178691.938996:0:20759:0:(nrs_fifo.c:241:nrs_fifo_req_stop()) NRS stop fifo request from 12345-0@lo, seq: 894 00000100:00100000:3.0:1776178691.941574:0:3354:0:(client.c:2215:ptlrpc_check_set()) Completed RPC req@00000000ea88a302 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_01:ac61f839-0e8e-4cac-9c59-08bcfbbad330:3354:1862456998116416:0@lo:400:kworker/u8:0.0 00010000:00010000:2.0:1776178692.576428:0:6244:0:(ldlm_lockd.c:2792:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:2.0:1776178693.474748:0:3355:0:(client.c:1733:ptlrpc_send_new_req()) Sending RPC req@00000000faeeda53 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_02:lustre-MDT0001-mdtlov_UUID:3355:1862456998116992:0@lo:13:osp-pre-1-1.0 00000100:00100000:2.0:1776178693.474823:0:3355:0:(events.c:361:request_in_callback()) peer: 12345-0@lo (source: 12345-0@lo) 00000100:00100000:1.0:1776178693.475247:0:8231:0:(service.c:2152:ptlrpc_server_handle_req_in()) got req x1862456998116992 00000100:00100000:1.0:1776178693.475299:0:8231:0:(nrs_fifo.c:179:nrs_fifo_req_get()) NRS start fifo request from 12345-0@lo, seq: 679 00000100:00100000:1.0:1776178693.475305:0:8231:0:(service.c:2324:ptlrpc_server_handle_request()) Handling RPC req@00000000a0db51c1 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_create00:lustre-MDT0001-mdtlov_UUID+5:3355:x1862456998116992:12345-0@lo:13:osp-pre-1-1.0 00002000:00000024:1.0:1776178693.475339:0:8231:0:(ofd_obd.c:740:ofd_statfs()) blocks cached 229376 granted 491746368 pending 0 free 3923689472 avail 3692167168 00000100:00100000:1.0:1776178693.475402:0:8231:0:(service.c:2372:ptlrpc_server_handle_request()) Handled RPC req@00000000a0db51c1 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_create00:lustre-MDT0001-mdtlov_UUID+5:3355:x1862456998116992:12345-0@lo:13:osp-pre-1-1.0 Request processed in 96us (580us total) trans 0 rc 0/0 00000100:00100000:1.0:1776178693.475409:0:8231:0:(nrs_fifo.c:241:nrs_fifo_req_stop()) NRS stop fifo request from 12345-0@lo, seq: 679 00000100:00100000:3.0:1776178693.475505:0:3355:0:(client.c:2215:ptlrpc_check_set()) Completed RPC req@00000000faeeda53 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_02:lustre-MDT0001-mdtlov_UUID:3355:1862456998116992:0@lo:13:osp-pre-1-1.0 00010000:00010000:1.0:1776178693.600360:0:6245:0:(ldlm_lockd.c:2792:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:0.0:1776178693.984351:0:3356:0:(client.c:1733:ptlrpc_send_new_req()) Sending RPC req@0000000050bb92d0 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_03:lustre-MDT0001-mdtlov_UUID:3356:1862456998117056:0@lo:13:osp-pre-0-1.0 00000100:00100000:0.0:1776178693.984456:0:3356:0:(events.c:361:request_in_callback()) peer: 12345-0@lo (source: 12345-0@lo) 00000100:00100000:3.0:1776178693.984615:0:9554:0:(service.c:2152:ptlrpc_server_handle_req_in()) got req x1862456998117056 00000100:00100000:3.0:1776178693.984673:0:9554:0:(nrs_fifo.c:179:nrs_fifo_req_get()) NRS start fifo request from 12345-0@lo, seq: 680 00000100:00100000:3.0:1776178693.984679:0:9554:0:(service.c:2324:ptlrpc_server_handle_request()) Handling RPC req@000000002300ff10 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_create00:lustre-MDT0001-mdtlov_UUID+5:3356:x1862456998117056:12345-0@lo:13:osp-pre-0-1.0 00002000:00000024:3.0:1776178693.984710:0:9554:0:(ofd_obd.c:740:ofd_statfs()) blocks cached 315392 granted 484883776 pending 0 free 3923689472 avail 3692167168 00000100:00100000:3.0:1776178693.988530:0:9554:0:(service.c:2372:ptlrpc_server_handle_request()) Handled RPC req@000000002300ff10 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_create00:lustre-MDT0001-mdtlov_UUID+5:3356:x1862456998117056:12345-0@lo:13:osp-pre-0-1.0 Request processed in 3840us (4067us total) trans 0 rc 0/0 00000100:00100000:3.0:1776178693.988547:0:9554:0:(nrs_fifo.c:241:nrs_fifo_req_stop()) NRS stop fifo request from 12345-0@lo, seq: 680 00000100:00100000:0.0:1776178693.989537:0:3356:0:(client.c:2215:ptlrpc_check_set()) Completed RPC req@0000000050bb92d0 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_03:lustre-MDT0001-mdtlov_UUID:3356:1862456998117056:0@lo:13:osp-pre-0-1.0 00010000:00010000:2.0:1776178694.624862:0:9224:0:(ldlm_lockd.c:2792:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:3.0:1776178695.458241:0:3348:0:(events.c:361:request_in_callback()) peer: 12345-192.168.204.3@tcp (source: 12345-192.168.204.3@tcp) 00000100:00100000:3.0:1776178695.458281:0:3348:0:(events.c:361:request_in_callback()) peer: 12345-192.168.204.3@tcp (source: 12345-192.168.204.3@tcp) 00000100:00100000:3.0:1776178695.458312:0:3348:0:(events.c:361:request_in_callback()) peer: 12345-192.168.204.3@tcp (source: 12345-192.168.204.3@tcp) 00000100:00100000:3.0:1776178695.458346:0:3348:0:(events.c:361:request_in_callback()) peer: 12345-192.168.204.3@tcp (source: 12345-192.168.204.3@tcp) 00000100:00100000:3.0:1776178695.458368:0:3348:0:(events.c:361:request_in_callback()) peer: 12345-192.168.204.3@tcp (source: 12345-192.168.204.3@tcp) 00000100:00100000:3.0:1776178695.458411:0:20140:0:(service.c:2152:ptlrpc_server_handle_req_in()) got req x1862456979139712 00000100:00100000:3.0:1776178695.458458:0:20140:0:(nrs_fifo.c:179:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.204.3@tcp, seq: 347 00000100:00100000:3.0:1776178695.458464:0:20140:0:(service.c:2324:ptlrpc_server_handle_request()) Handling RPC req@00000000ae8faa77 pname:cluuid+ref:pid:xid:nid:opc:job ll_mgs_0002:5b5c66b1-1a2f-4651-8548-0c75b9a38a94+9:2270:x1862456979139712:12345-192.168.204.3@tcp:400:kworker/u8:3.0 00000100:00100000:3.0:1776178695.458525:0:20140:0:(service.c:2372:ptlrpc_server_handle_request()) Handled RPC req@00000000ae8faa77 pname:cluuid+ref:pid:xid:nid:opc:job ll_mgs_0002:5b5c66b1-1a2f-4651-8548-0c75b9a38a94+9:2270:x1862456979139712:12345-192.168.204.3@tcp:400:kworker/u8:3.0 Request processed in 60us (213us total) trans 0 rc 0/0 00000100:00100000:3.0:1776178695.458533:0:20140:0:(nrs_fifo.c:241:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.204.3@tcp, seq: 347 00000100:00100000:3.0:1776178695.458565:0:20755:0:(service.c:2152:ptlrpc_server_handle_req_in()) got req x1862456979139968 00000100:00100000:3.0:1776178695.458588:0:20755:0:(service.c:2152:ptlrpc_server_handle_req_in()) got req x1862456979139904 00000100:00100000:3.0:1776178695.458603:0:20755:0:(nrs_fifo.c:179:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.204.3@tcp, seq: 895 00000100:00100000:3.0:1776178695.458607:0:20755:0:(service.c:2324:ptlrpc_server_handle_request()) Handling RPC req@0000000009435d5b pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_006:b50963bd-8eba-4432-9a54-8888d00e3fa1+9:2270:x1862456979139968:12345-192.168.204.3@tcp:400:kworker/u8:3.0 00000100:00100000:3.0:1776178695.458629:0:20755:0:(service.c:2372:ptlrpc_server_handle_request()) Handled RPC req@0000000009435d5b pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_006:b50963bd-8eba-4432-9a54-8888d00e3fa1+9:2270:x1862456979139968:12345-192.168.204.3@tcp:400:kworker/u8:3.0 Request processed in 22us (349us total) trans 0 rc 0/0 00000100:00100000:3.0:1776178695.458634:0:20755:0:(nrs_fifo.c:241:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.204.3@tcp, seq: 895 00000100:00100000:3.0:1776178695.458643:0:20755:0:(nrs_fifo.c:179:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.204.3@tcp, seq: 896 00000100:00100000:3.0:1776178695.458647:0:20755:0:(service.c:2324:ptlrpc_server_handle_request()) Handling RPC req@000000009015f30b pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_006:b50963bd-8eba-4432-9a54-8888d00e3fa1+8:2270:x1862456979139904:12345-192.168.204.3@tcp:400:kworker/u8:3.0 00000100:00100000:3.0:1776178695.458663:0:20755:0:(service.c:2372:ptlrpc_server_handle_request()) Handled RPC req@000000009015f30b pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_006:b50963bd-8eba-4432-9a54-8888d00e3fa1+8:2270:x1862456979139904:12345-192.168.204.3@tcp:400:kworker/u8:3.0 Request processed in 16us (319us total) trans 0 rc 0/0 00000100:00100000:3.0:1776178695.458668:0:20755:0:(nrs_fifo.c:241:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.204.3@tcp, seq: 896 00000100:00100000:3.0:1776178695.458682:0:11281:0:(service.c:2152:ptlrpc_server_handle_req_in()) got req x1862456979139840 00000100:00100000:3.0:1776178695.458700:0:11281:0:(service.c:2152:ptlrpc_server_handle_req_in()) got req x1862456979139776 00000100:00100000:3.0:1776178695.458715:0:11281:0:(nrs_fifo.c:179:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.204.3@tcp, seq: 1216 00000100:00100000:3.0:1776178695.458718:0:11281:0:(service.c:2324:ptlrpc_server_handle_request()) Handling RPC req@00000000faeeda53 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_004:b50963bd-8eba-4432-9a54-8888d00e3fa1+6:2270:x1862456979139840:12345-192.168.204.3@tcp:400:kworker/u8:3.0 00000100:00100000:3.0:1776178695.458742:0:11281:0:(service.c:2372:ptlrpc_server_handle_request()) Handled RPC req@00000000faeeda53 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_004:b50963bd-8eba-4432-9a54-8888d00e3fa1+6:2270:x1862456979139840:12345-192.168.204.3@tcp:400:kworker/u8:3.0 Request processed in 23us (509us total) trans 0 rc 0/0 00000100:00100000:3.0:1776178695.458748:0:11281:0:(nrs_fifo.c:241:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.204.3@tcp, seq: 1216 00000100:00100000:3.0:1776178695.458760:0:11281:0:(nrs_fifo.c:179:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.204.3@tcp, seq: 7008 00000100:00100000:3.0:1776178695.458764:0:11281:0:(service.c:2324:ptlrpc_server_handle_request()) Handling RPC req@0000000086665d0f pname:cluuid+ref:pid:xid:nid:opc:job mdt00_004:0+-99:2269:x1862456979139776:12345-192.168.204.3@tcp:38:kworker/u8:3.0 00010000:02000000:3.0:1776178695.458777:0:11281:0:(ldlm_lib.c:1139:target_handle_connect()) lustre-MDT0000: Not available for connect from 192.168.204.3@tcp (stopping) 00000100:00100000:3.0:1776178695.458802:0:11281:0:(service.c:2372:ptlrpc_server_handle_request()) Handled RPC req@0000000086665d0f pname:cluuid+ref:pid:xid:nid:opc:job mdt00_004:0+-99:2269:x1862456979139776:12345-192.168.204.3@tcp:38:kworker/u8:3.0 Request processed in 36us (433us total) trans 0 rc -19/-19 00000100:00100000:3.0:1776178695.458807:0:11281:0:(nrs_fifo.c:241:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.204.3@tcp, seq: 7008 00010000:00010000:1.0:1776178695.648304:0:9225:0:(ldlm_lockd.c:2792:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00010000:00010000:2.0:1776178696.681667:0:6244:0:(ldlm_lockd.c:2792:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00080000:0.0:1776178697.057996:0:36:0:(pinger.c:251:ptlrpc_pinger_process_import()) lustre-MDT0000-lwp-OST0000_UUID->lustre-MDT0000_UUID: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00080000:0.0:1776178697.058006:0:36:0:(recover.c:57:ptlrpc_initiate_recovery()) lustre-MDT0000_UUID: starting recovery 00000100:00080000:0.0:1776178697.058010:0:36:0:(import.c:86:import_set_state_nolock()) 0000000047601a98 lustre-MDT0000_UUID: changing import state from DISCONN to CONNECTING 00000100:00080000:0.0:1776178697.058020:0:36:0:(import.c:534:import_select_connection()) lustre-MDT0000-lwp-OST0000: connect to NID 0@lo last attempt 1399 00000100:00080000:0.0:1776178697.058025:0:36:0:(import.c:581:import_select_connection()) lustre-MDT0000-lwp-OST0000: tried all connections, increasing latency to 21s 00000100:00080000:0.0:1776178697.058033:0:36:0:(import.c:615:import_select_connection()) lustre-MDT0000-lwp-OST0000: import 0000000047601a98 using connection 192.168.204.103@tcp/0@lo 00000100:00100000:0.0:1776178697.058087:0:36:0:(import.c:807:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5) req@00000000ea88a302 x1862456998117184/t0(0) o38->lustre-MDT0000-lwp-OST0000@0@lo:12/10 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' 00000100:00080000:0.0:1776178697.058122:0:36:0:(pinger.c:251:ptlrpc_pinger_process_import()) lustre-MDT0000-lwp-OST0001_UUID->lustre-MDT0000_UUID: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00080000:0.0:1776178697.058126:0:36:0:(recover.c:57:ptlrpc_initiate_recovery()) lustre-MDT0000_UUID: starting recovery 00000100:00080000:0.0:1776178697.058127:0:36:0:(import.c:86:import_set_state_nolock()) 00000000dfc8db13 lustre-MDT0000_UUID: changing import state from DISCONN to CONNECTING 00000100:00080000:0.0:1776178697.058131:0:36:0:(import.c:534:import_select_connection()) lustre-MDT0000-lwp-OST0001: connect to NID 0@lo last attempt 1399 00000100:00080000:0.0:1776178697.058133:0:36:0:(import.c:581:import_select_connection()) lustre-MDT0000-lwp-OST0001: tried all connections, increasing latency to 21s 00000100:00080000:0.0:1776178697.058136:0:36:0:(import.c:615:import_select_connection()) lustre-MDT0000-lwp-OST0001: import 00000000dfc8db13 using connection 192.168.204.103@tcp/0@lo 00000100:00100000:0.0:1776178697.058148:0:36:0:(import.c:807:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5) req@000000004828c19d x1862456998117312/t0(0) o38->lustre-MDT0000-lwp-OST0001@0@lo:12/10 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' 00000100:00080000:0.0:1776178697.058180:0:36:0:(pinger.c:251:ptlrpc_pinger_process_import()) lustre-MDT0001-mdtlov_UUID->lustre-MDT0000_UUID: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00080000:0.0:1776178697.058184:0:36:0:(recover.c:57:ptlrpc_initiate_recovery()) lustre-MDT0000_UUID: starting recovery 00000100:00080000:0.0:1776178697.058185:0:36:0:(import.c:86:import_set_state_nolock()) 00000000ba4f1821 lustre-MDT0000_UUID: changing import state from DISCONN to CONNECTING 00000100:00080000:0.0:1776178697.058188:0:36:0:(import.c:534:import_select_connection()) lustre-MDT0000-osp-MDT0001: connect to NID 0@lo last attempt 1399 00000100:00080000:0.0:1776178697.058190:0:36:0:(import.c:581:import_select_connection()) lustre-MDT0000-osp-MDT0001: tried all connections, increasing latency to 21s 00000100:00080000:0.0:1776178697.058194:0:36:0:(import.c:615:import_select_connection()) lustre-MDT0000-osp-MDT0001: import 00000000ba4f1821 using connection 192.168.204.103@tcp/0@lo 00000100:00100000:0.0:1776178697.058206:0:36:0:(import.c:807:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5) req@0000000012b22334 x1862456998117440/t0(0) o38->lustre-MDT0000-osp-MDT0001@0@lo:24/4 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' 00000100:00080000:0.0:1776178697.058226:0:36:0:(pinger.c:251:ptlrpc_pinger_process_import()) lustre-MDT0000-lwp-MDT0001_UUID->lustre-MDT0000_UUID: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00080000:0.0:1776178697.058229:0:36:0:(recover.c:57:ptlrpc_initiate_recovery()) lustre-MDT0000_UUID: starting recovery 00000100:00080000:0.0:1776178697.058231:0:36:0:(import.c:86:import_set_state_nolock()) 00000000a7c55b1f lustre-MDT0000_UUID: changing import state from DISCONN to CONNECTING 00000100:00080000:0.0:1776178697.058234:0:36:0:(import.c:534:import_select_connection()) lustre-MDT0000-lwp-MDT0001: connect to NID 0@lo last attempt 1399 00000100:00080000:0.0:1776178697.058236:0:36:0:(import.c:581:import_select_connection()) lustre-MDT0000-lwp-MDT0001: tried all connections, increasing latency to 21s 00000100:00080000:0.0:1776178697.058239:0:36:0:(import.c:615:import_select_connection()) lustre-MDT0000-lwp-MDT0001: import 00000000a7c55b1f using connection 192.168.204.103@tcp/0@lo 00000100:00100000:0.0:1776178697.058250:0:36:0:(import.c:807:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5) req@000000007cfd1c43 x1862456998117632/t0(0) o38->lustre-MDT0000-lwp-MDT0001@0@lo:12/10 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' 00000100:00100000:0.0:1776178697.058299:0:3355:0:(client.c:1733:ptlrpc_send_new_req()) Sending RPC req@0000000035819321 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_02:lustre-MDT0001-lwp-OST0001_UUID:3355:1862456998117376:0@lo:400:kworker/u8:1.0 00000100:00100000:2.0:1776178697.059316:0:3352:0:(client.c:1733:ptlrpc_send_new_req()) Sending RPC req@00000000ea88a302 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:lustre-MDT0000-lwp-OST0000_UUID:3352:1862456998117184:0@lo:38:kworker/u8:1.0 00000100:00100000:1.0:1776178697.059411:0:3354:0:(client.c:1733:ptlrpc_send_new_req()) Sending RPC req@0000000004dd3b9b pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_01:lustre-MDT0001-lwp-OST0000_UUID:3354:1862456998117248:0@lo:400:kworker/u8:1.0 00000100:00100000:3.0:1776178697.060095:0:3353:0:(client.c:1733:ptlrpc_send_new_req()) Sending RPC req@0000000042de09e9 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_00:ac61f839-0e8e-4cac-9c59-08bcfbbad330:3353:1862456998117120:0@lo:400:kworker/u8:1.0 00000100:00100000:2.0:1776178697.074912:0:3352:0:(events.c:361:request_in_callback()) peer: 12345-0@lo (source: 12345-0@lo) 00000100:00100000:2.0:1776178697.074970:0:3352:0:(client.c:1733:ptlrpc_send_new_req()) Sending RPC req@000000004828c19d pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:lustre-MDT0000-lwp-OST0001_UUID:3352:1862456998117312:0@lo:38:kworker/u8:1.0 00000100:00100000:2.0:1776178697.075029:0:3352:0:(events.c:361:request_in_callback()) peer: 12345-0@lo (source: 12345-0@lo) 00000100:00100000:2.0:1776178697.075052:0:3352:0:(client.c:1733:ptlrpc_send_new_req()) Sending RPC req@0000000012b22334 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:lustre-MDT0001-mdtlov_UUID:3352:1862456998117440:0@lo:38:kworker/u8:1.0 00000100:00100000:2.0:1776178697.075108:0:3352:0:(events.c:361:request_in_callback()) peer: 12345-0@lo (source: 12345-0@lo) 00000100:00100000:2.0:1776178697.075126:0:3352:0:(client.c:1733:ptlrpc_send_new_req()) Sending RPC req@000000007cfd1c43 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:lustre-MDT0000-lwp-MDT0001_UUID:3352:1862456998117632:0@lo:38:kworker/u8:1.0 00000100:00100000:2.0:1776178697.075150:0:3352:0:(events.c:361:request_in_callback()) peer: 12345-0@lo (source: 12345-0@lo) 00000100:00100000:2.0:1776178697.075222:0:8250:0:(service.c:2152:ptlrpc_server_handle_req_in()) got req x1862456998117184 00000100:00100000:2.0:1776178697.075265:0:8250:0:(service.c:2152:ptlrpc_server_handle_req_in()) got req x1862456998117312 00000100:00100000:2.0:1776178697.075277:0:8250:0:(service.c:2152:ptlrpc_server_handle_req_in()) got req x1862456998117632 00000100:00100000:2.0:1776178697.075291:0:8250:0:(nrs_fifo.c:179:nrs_fifo_req_get()) NRS start fifo request from 12345-0@lo, seq: 7009 00000100:00100000:2.0:1776178697.075298:0:8250:0:(service.c:2324:ptlrpc_server_handle_request()) Handling RPC req@000000000354b8ca pname:cluuid+ref:pid:xid:nid:opc:job mdt00_003:0+-99:3352:x1862456998117184:12345-0@lo:38:kworker/u8:1.0 00010000:02000000:2.0:1776178697.075329:0:8250:0:(ldlm_lib.c:1139:target_handle_connect()) lustre-MDT0000: Not available for connect from 0@lo (stopping) 00000100:00100000:2.0:1776178697.075383:0:8250:0:(service.c:2372:ptlrpc_server_handle_request()) Handled RPC req@000000000354b8ca pname:cluuid+ref:pid:xid:nid:opc:job mdt00_003:0+-99:3352:x1862456998117184:12345-0@lo:38:kworker/u8:1.0 Request processed in 85us (483us total) trans 0 rc -19/-19 00000100:00100000:2.0:1776178697.075391:0:8250:0:(nrs_fifo.c:241:nrs_fifo_req_stop()) NRS stop fifo request from 12345-0@lo, seq: 7009 00000100:00100000:2.0:1776178697.075410:0:8250:0:(nrs_fifo.c:179:nrs_fifo_req_get()) NRS start fifo request from 12345-0@lo, seq: 7010 00000100:00100000:2.0:1776178697.075413:0:8250:0:(service.c:2324:ptlrpc_server_handle_request()) Handling RPC req@00000000df769e31 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_003:0+-99:3352:x1862456998117312:12345-0@lo:38:kworker/u8:1.0 00010000:02000000:2.0:1776178697.075419:0:8250:0:(ldlm_lib.c:1139:target_handle_connect()) lustre-MDT0000: Not available for connect from 0@lo (stopping) 00000100:00100000:2.0:1776178697.075441:0:8250:0:(service.c:2372:ptlrpc_server_handle_request()) Handled RPC req@00000000df769e31 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_003:0+-99:3352:x1862456998117312:12345-0@lo:38:kworker/u8:1.0 Request processed in 26us (413us total) trans 0 rc -19/-19 00000100:00100000:2.0:1776178697.075446:0:8250:0:(nrs_fifo.c:241:nrs_fifo_req_stop()) NRS stop fifo request from 12345-0@lo, seq: 7010 00000100:00100000:2.0:1776178697.075455:0:8250:0:(nrs_fifo.c:179:nrs_fifo_req_get()) NRS start fifo request from 12345-0@lo, seq: 7011 00000100:00100000:2.0:1776178697.075458:0:8250:0:(service.c:2324:ptlrpc_server_handle_request()) Handling RPC req@00000000b5990455 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_003:0+-99:3352:x1862456998117632:12345-0@lo:38:kworker/u8:1.0 00010000:02000000:2.0:1776178697.075463:0:8250:0:(ldlm_lib.c:1139:target_handle_connect()) lustre-MDT0000: Not available for connect from 0@lo (stopping) 00000100:00100000:2.0:1776178697.075483:0:8250:0:(service.c:2372:ptlrpc_server_handle_request()) Handled RPC req@00000000b5990455 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_003:0+-99:3352:x1862456998117632:12345-0@lo:38:kworker/u8:1.0 Request processed in 24us (333us total) trans 0 rc -19/-19 00000100:00100000:2.0:1776178697.075488:0:8250:0:(nrs_fifo.c:241:nrs_fifo_req_stop()) NRS stop fifo request from 12345-0@lo, seq: 7011 00000100:00080000:2.0:1776178697.075541:0:3352:0:(import.c:86:import_set_state_nolock()) 0000000047601a98 lustre-MDT0000_UUID: changing import state from CONNECTING to DISCONN 00000100:00080000:2.0:1776178697.075546:0:3352:0:(import.c:1429:ptlrpc_connect_interpret()) recovery of lustre-MDT0000_UUID on 192.168.204.103@tcp failed (-19) 00000100:00100000:2.0:1776178697.075550:0:3352:0:(client.c:2215:ptlrpc_check_set()) Completed RPC req@00000000ea88a302 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:lustre-MDT0000-lwp-OST0000_UUID:3352:1862456998117184:0@lo:38:kworker/u8:1.0 00000100:00080000:2.0:1776178697.075560:0:3352:0:(import.c:86:import_set_state_nolock()) 00000000dfc8db13 lustre-MDT0000_UUID: changing import state from CONNECTING to DISCONN 00000100:00080000:2.0:1776178697.075563:0:3352:0:(import.c:1429:ptlrpc_connect_interpret()) recovery of lustre-MDT0000_UUID on 192.168.204.103@tcp failed (-19) 00000100:00100000:2.0:1776178697.075565:0:3352:0:(client.c:2215:ptlrpc_check_set()) Completed RPC req@000000004828c19d pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:lustre-MDT0000-lwp-OST0001_UUID:3352:1862456998117312:0@lo:38:kworker/u8:1.0 00000100:00080000:2.0:1776178697.075572:0:3352:0:(import.c:86:import_set_state_nolock()) 00000000a7c55b1f lustre-MDT0000_UUID: changing import state from CONNECTING to DISCONN 00000100:00080000:2.0:1776178697.075574:0:3352:0:(import.c:1429:ptlrpc_connect_interpret()) recovery of lustre-MDT0000_UUID on 192.168.204.103@tcp failed (-19) 00000100:00100000:2.0:1776178697.075576:0:3352:0:(client.c:2215:ptlrpc_check_set()) Completed RPC req@000000007cfd1c43 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:lustre-MDT0000-lwp-MDT0001_UUID:3352:1862456998117632:0@lo:38:kworker/u8:1.0 00000100:00100000:2.0:1776178697.075627:0:14128:0:(service.c:2152:ptlrpc_server_handle_req_in()) got req x1862456998117440 00000100:00100000:2.0:1776178697.075652:0:14128:0:(nrs_fifo.c:179:nrs_fifo_req_get()) NRS start fifo request from 12345-0@lo, seq: 804 00000100:00100000:2.0:1776178697.075655:0:14128:0:(service.c:2324:ptlrpc_server_handle_request()) Handling RPC req@00000000fdb309c4 pname:cluuid+ref:pid:xid:nid:opc:job mdt_out00_002:0+-99:3352:x1862456998117440:12345-0@lo:38:kworker/u8:1.0 00010000:02000000:2.0:1776178697.075662:0:14128:0:(ldlm_lib.c:1139:target_handle_connect()) lustre-MDT0000: Not available for connect from 0@lo (stopping) 00000100:00100000:2.0:1776178697.075692:0:14128:0:(service.c:2372:ptlrpc_server_handle_request()) Handled RPC req@00000000fdb309c4 pname:cluuid+ref:pid:xid:nid:opc:job mdt_out00_002:0+-99:3352:x1862456998117440:12345-0@lo:38:kworker/u8:1.0 Request processed in 35us (584us total) trans 0 rc -19/-19 00000100:00100000:2.0:1776178697.075698:0:14128:0:(nrs_fifo.c:241:nrs_fifo_req_stop()) NRS stop fifo request from 12345-0@lo, seq: 804 00000100:00080000:2.0:1776178697.075722:0:3352:0:(import.c:86:import_set_state_nolock()) 00000000ba4f1821 lustre-MDT0000_UUID: changing import state from CONNECTING to DISCONN 00000100:00080000:2.0:1776178697.075725:0:3352:0:(import.c:1429:ptlrpc_connect_interpret()) recovery of lustre-MDT0000_UUID on 192.168.204.103@tcp failed (-19) 00000100:00100000:2.0:1776178697.075728:0:3352:0:(client.c:2215:ptlrpc_check_set()) Completed RPC req@0000000012b22334 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:lustre-MDT0001-mdtlov_UUID:3352:1862456998117440:0@lo:38:kworker/u8:1.0 00000100:00100000:2.0:1776178697.075760:0:3356:0:(client.c:1733:ptlrpc_send_new_req()) Sending RPC req@000000006a3d91b0 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_03:lustre-MDT0001-mdtlov_UUID:3356:1862456998117504:0@lo:400:kworker/u8:1.0 00000100:00100000:2.0:1776178697.075790:0:3356:0:(events.c:361:request_in_callback()) peer: 12345-0@lo (source: 12345-0@lo) 00000100:00100000:1.0:1776178697.076608:0:3354:0:(events.c:361:request_in_callback()) peer: 12345-0@lo (source: 12345-0@lo) 00000100:00100000:1.0:1776178697.076680:0:20759:0:(service.c:2152:ptlrpc_server_handle_req_in()) got req x1862456998117504 00000100:00100000:1.0:1776178697.076725:0:20759:0:(nrs_fifo.c:179:nrs_fifo_req_get()) NRS start fifo request from 12345-0@lo, seq: 897 00000100:00100000:1.0:1776178697.076729:0:20759:0:(service.c:2324:ptlrpc_server_handle_request()) Handling RPC req@00000000b2b2b654 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_010:lustre-MDT0001-mdtlov_UUID+5:3356:x1862456998117504:12345-0@lo:400:kworker/u8:1.0 00000100:00100000:1.0:1776178697.076786:0:20759:0:(service.c:2372:ptlrpc_server_handle_request()) Handled RPC req@00000000b2b2b654 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_010:lustre-MDT0001-mdtlov_UUID+5:3356:x1862456998117504:12345-0@lo:400:kworker/u8:1.0 Request processed in 56us (996us total) trans 0 rc 0/0 00000100:00100000:1.0:1776178697.076795:0:20759:0:(nrs_fifo.c:241:nrs_fifo_req_stop()) NRS stop fifo request from 12345-0@lo, seq: 897 00000100:00100000:1.0:1776178697.076838:0:3356:0:(client.c:2215:ptlrpc_check_set()) Completed RPC req@000000006a3d91b0 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_03:lustre-MDT0001-mdtlov_UUID:3356:1862456998117504:0@lo:400:kworker/u8:1.0 00000100:00100000:2.0:1776178697.078746:0:6255:0:(service.c:2152:ptlrpc_server_handle_req_in()) got req x1862456998117248 00000100:00100000:2.0:1776178697.078839:0:6255:0:(nrs_fifo.c:179:nrs_fifo_req_get()) NRS start fifo request from 12345-0@lo, seq: 1217 00000100:00100000:2.0:1776178697.078846:0:6255:0:(service.c:2324:ptlrpc_server_handle_request()) Handling RPC req@000000001e2473b2 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_000:lustre-MDT0001-lwp-OST0000_UUID+5:3354:x1862456998117248:12345-0@lo:400:kworker/u8:1.0 00000100:00100000:2.0:1776178697.078959:0:6255:0:(service.c:2372:ptlrpc_server_handle_request()) Handled RPC req@000000001e2473b2 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_000:lustre-MDT0001-lwp-OST0000_UUID+5:3354:x1862456998117248:12345-0@lo:400:kworker/u8:1.0 Request processed in 112us (2354us total) trans 0 rc 0/0 00000100:00100000:2.0:1776178697.078966:0:6255:0:(nrs_fifo.c:241:nrs_fifo_req_stop()) NRS stop fifo request from 12345-0@lo, seq: 1217 00000100:00100000:1.0:1776178697.081747:0:3354:0:(client.c:2215:ptlrpc_check_set()) Completed RPC req@0000000004dd3b9b pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_01:lustre-MDT0001-lwp-OST0000_UUID:3354:1862456998117248:0@lo:400:kworker/u8:1.0 00000100:00100000:0.0:1776178697.086288:0:3355:0:(events.c:361:request_in_callback()) peer: 12345-0@lo (source: 12345-0@lo) 00000100:00100000:1.0:1776178697.086647:0:11281:0:(service.c:2152:ptlrpc_server_handle_req_in()) got req x1862456998117376 00000100:00100000:1.0:1776178697.086724:0:11281:0:(nrs_fifo.c:179:nrs_fifo_req_get()) NRS start fifo request from 12345-0@lo, seq: 1218 00000100:00100000:1.0:1776178697.086743:0:11281:0:(service.c:2324:ptlrpc_server_handle_request()) Handling RPC req@00000000907210d1 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_004:lustre-MDT0001-lwp-OST0001_UUID+5:3355:x1862456998117376:12345-0@lo:400:kworker/u8:1.0 00000100:00100000:1.0:1776178697.086836:0:11281:0:(service.c:2372:ptlrpc_server_handle_request()) Handled RPC req@00000000907210d1 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_004:lustre-MDT0001-lwp-OST0001_UUID+5:3355:x1862456998117376:12345-0@lo:400:kworker/u8:1.0 Request processed in 104us (560us total) trans 0 rc 0/0 00000100:00100000:1.0:1776178697.086845:0:11281:0:(nrs_fifo.c:241:nrs_fifo_req_stop()) NRS stop fifo request from 12345-0@lo, seq: 1218 00000100:00100000:3.0:1776178697.087086:0:3353:0:(events.c:361:request_in_callback()) peer: 12345-0@lo (source: 12345-0@lo) 00000100:00100000:3.0:1776178697.087116:0:3353:0:(client.c:1733:ptlrpc_send_new_req()) Sending RPC req@00000000fd2b8392 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_00:lustre-MDT0001-mdtlov_UUID:3353:1862456998117568:0@lo:400:kworker/u8:1.0 00000100:00100000:3.0:1776178697.087153:0:3353:0:(events.c:361:request_in_callback()) peer: 12345-0@lo (source: 12345-0@lo) 00000100:00100000:1.0:1776178697.087198:0:20137:0:(service.c:2152:ptlrpc_server_handle_req_in()) got req x1862456998117120 00000100:00100000:3.0:1776178697.087210:0:20762:0:(service.c:2152:ptlrpc_server_handle_req_in()) got req x1862456998117568 00000100:00100000:1.0:1776178697.087237:0:20137:0:(nrs_fifo.c:179:nrs_fifo_req_get()) NRS start fifo request from 12345-0@lo, seq: 348 00000100:00100000:1.0:1776178697.087242:0:20137:0:(service.c:2324:ptlrpc_server_handle_request()) Handling RPC req@000000002efeec88 pname:cluuid+ref:pid:xid:nid:opc:job ll_mgs_0000:ac61f839-0e8e-4cac-9c59-08bcfbbad330+18:3353:x1862456998117120:12345-0@lo:400:kworker/u8:1.0 00000100:00100000:3.0:1776178697.087250:0:20762:0:(nrs_fifo.c:179:nrs_fifo_req_get()) NRS start fifo request from 12345-0@lo, seq: 898 00000100:00100000:3.0:1776178697.087256:0:20762:0:(service.c:2324:ptlrpc_server_handle_request()) Handling RPC req@00000000964f7a65 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_013:lustre-MDT0001-mdtlov_UUID+5:3353:x1862456998117568:12345-0@lo:400:kworker/u8:1.0 00000100:00100000:1.0:1776178697.087287:0:20137:0:(service.c:2372:ptlrpc_server_handle_request()) Handled RPC req@000000002efeec88 pname:cluuid+ref:pid:xid:nid:opc:job ll_mgs_0000:ac61f839-0e8e-4cac-9c59-08bcfbbad330+18:3353:x1862456998117120:12345-0@lo:400:kworker/u8:1.0 Request processed in 43us (201us total) trans 0 rc 0/0 00000100:00100000:1.0:1776178697.087296:0:20137:0:(nrs_fifo.c:241:nrs_fifo_req_stop()) NRS stop fifo request from 12345-0@lo, seq: 348 00000100:00100000:3.0:1776178697.087322:0:20762:0:(service.c:2372:ptlrpc_server_handle_request()) Handled RPC req@00000000964f7a65 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_013:lustre-MDT0001-mdtlov_UUID+5:3353:x1862456998117568:12345-0@lo:400:kworker/u8:1.0 Request processed in 43us (146us total) trans 0 rc 0/0 00000100:00100000:3.0:1776178697.087331:0:20762:0:(nrs_fifo.c:241:nrs_fifo_req_stop()) NRS stop fifo request from 12345-0@lo, seq: 898 00000100:00100000:3.0:1776178697.087374:0:3353:0:(client.c:2215:ptlrpc_check_set()) Completed RPC req@0000000042de09e9 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_00:ac61f839-0e8e-4cac-9c59-08bcfbbad330:3353:1862456998117120:0@lo:400:kworker/u8:1.0 00000100:00100000:3.0:1776178697.087389:0:3353:0:(client.c:2215:ptlrpc_check_set()) Completed RPC req@00000000fd2b8392 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_00:lustre-MDT0001-mdtlov_UUID:3353:1862456998117568:0@lo:400:kworker/u8:1.0 00000100:00100000:0.0:1776178697.094737:0:3355:0:(client.c:2215:ptlrpc_check_set()) Completed RPC req@0000000035819321 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_02:lustre-MDT0001-lwp-OST0001_UUID:3355:1862456998117376:0@lo:400:kworker/u8:1.0 00010000:00010000:1.0:1776178697.696396:0:6245:0:(ldlm_lockd.c:2792:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:1.0:1776178698.592785:0:3354:0:(client.c:1733:ptlrpc_send_new_req()) Sending RPC req@00000000ad228ff6 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_01:lustre-MDT0001-mdtlov_UUID:3354:1862456998117696:0@lo:13:osp-pre-1-1.0 00000100:00100000:1.0:1776178698.592901:0:3354:0:(events.c:361:request_in_callback()) peer: 12345-0@lo (source: 12345-0@lo) 00000100:00100000:3.0:1776178698.593009:0:9554:0:(service.c:2152:ptlrpc_server_handle_req_in()) got req x1862456998117696 00000100:00100000:3.0:1776178698.593086:0:9554:0:(nrs_fifo.c:179:nrs_fifo_req_get()) NRS start fifo request from 12345-0@lo, seq: 681 00000100:00100000:3.0:1776178698.593093:0:9554:0:(service.c:2324:ptlrpc_server_handle_request()) Handling RPC req@00000000ee27102e pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_create00:lustre-MDT0001-mdtlov_UUID+5:3354:x1862456998117696:12345-0@lo:13:osp-pre-1-1.0 00002000:00000024:3.0:1776178698.593128:0:9554:0:(ofd_obd.c:740:ofd_statfs()) blocks cached 229376 granted 491746368 pending 0 free 3923689472 avail 3692167168 00000100:00100000:3.0:1776178698.593186:0:9554:0:(service.c:2372:ptlrpc_server_handle_request()) Handled RPC req@00000000ee27102e pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_create00:lustre-MDT0001-mdtlov_UUID+5:3354:x1862456998117696:12345-0@lo:13:osp-pre-1-1.0 Request processed in 93us (288us total) trans 0 rc 0/0 00000100:00100000:3.0:1776178698.593198:0:9554:0:(nrs_fifo.c:241:nrs_fifo_req_stop()) NRS stop fifo request from 12345-0@lo, seq: 681 00000100:00100000:0.0:1776178698.593287:0:3354:0:(client.c:2215:ptlrpc_check_set()) Completed RPC req@00000000ad228ff6 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_01:lustre-MDT0001-mdtlov_UUID:3354:1862456998117696:0@lo:13:osp-pre-1-1.0 00010000:00010000:2.0:1776178698.720608:0:9224:0:(ldlm_lockd.c:2792:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:0.0:1776178699.105987:0:3355:0:(client.c:1733:ptlrpc_send_new_req()) Sending RPC req@000000004547bc6e pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_02:lustre-MDT0001-mdtlov_UUID:3355:1862456998117760:0@lo:13:osp-pre-0-1.0 00000100:00100000:0.0:1776178699.106101:0:3355:0:(events.c:361:request_in_callback()) peer: 12345-0@lo (source: 12345-0@lo) 00000100:00100000:3.0:1776178699.106275:0:9554:0:(service.c:2152:ptlrpc_server_handle_req_in()) got req x1862456998117760 00000100:00100000:3.0:1776178699.106342:0:9554:0:(nrs_fifo.c:179:nrs_fifo_req_get()) NRS start fifo request from 12345-0@lo, seq: 682 00000100:00100000:3.0:1776178699.106350:0:9554:0:(service.c:2324:ptlrpc_server_handle_request()) Handling RPC req@00000000b1db1835 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_create00:lustre-MDT0001-mdtlov_UUID+5:3355:x1862456998117760:12345-0@lo:13:osp-pre-0-1.0 00002000:00000024:3.0:1776178699.106391:0:9554:0:(ofd_obd.c:740:ofd_statfs()) blocks cached 315392 granted 484883776 pending 0 free 3923689472 avail 3692167168 00000100:00100000:3.0:1776178699.106530:0:9554:0:(service.c:2372:ptlrpc_server_handle_request()) Handled RPC req@00000000b1db1835 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_create00:lustre-MDT0001-mdtlov_UUID+5:3355:x1862456998117760:12345-0@lo:13:osp-pre-0-1.0 Request processed in 179us (430us total) trans 0 rc 0/0 00000100:00100000:3.0:1776178699.106550:0:9554:0:(nrs_fifo.c:241:nrs_fifo_req_stop()) NRS stop fifo request from 12345-0@lo, seq: 682 00000100:00100000:0.0:1776178699.110014:0:3355:0:(client.c:2215:ptlrpc_check_set()) Completed RPC req@000000004547bc6e pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_02:lustre-MDT0001-mdtlov_UUID:3355:1862456998117760:0@lo:13:osp-pre-0-1.0 00010000:00010000:1.0:1776178699.744521:0:9225:0:(ldlm_lockd.c:2792:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:1.0:1776178700.581814:0:3346:0:(events.c:361:request_in_callback()) peer: 12345-192.168.204.3@tcp (source: 12345-192.168.204.3@tcp) 00000100:00100000:1.0:1776178700.581920:0:3346:0:(events.c:361:request_in_callback()) peer: 12345-192.168.204.3@tcp (source: 12345-192.168.204.3@tcp) 00000100:00100000:1.0:1776178700.581964:0:3346:0:(events.c:361:request_in_callback()) peer: 12345-192.168.204.3@tcp (source: 12345-192.168.204.3@tcp) 00000100:00100000:1.0:1776178700.581993:0:3346:0:(events.c:361:request_in_callback()) peer: 12345-192.168.204.3@tcp (source: 12345-192.168.204.3@tcp) 00000100:00100000:1.0:1776178700.582028:0:3346:0:(events.c:361:request_in_callback()) peer: 12345-192.168.204.3@tcp (source: 12345-192.168.204.3@tcp) 00000100:00100000:1.0:1776178700.582099:0:20759:0:(service.c:2152:ptlrpc_server_handle_req_in()) got req x1862456979140288 00000100:00100000:1.0:1776178700.582148:0:20759:0:(service.c:2152:ptlrpc_server_handle_req_in()) got req x1862456979140224 00000100:00100000:1.0:1776178700.582168:0:20759:0:(nrs_fifo.c:179:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.204.3@tcp, seq: 899 00000100:00100000:1.0:1776178700.582175:0:20759:0:(service.c:2324:ptlrpc_server_handle_request()) Handling RPC req@00000000a0f2a112 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_010:b50963bd-8eba-4432-9a54-8888d00e3fa1+9:2270:x1862456979140288:12345-192.168.204.3@tcp:400:kworker/u8:2.0 00000100:00100000:1.0:1776178700.582266:0:20759:0:(service.c:2372:ptlrpc_server_handle_request()) Handled RPC req@00000000a0f2a112 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_010:b50963bd-8eba-4432-9a54-8888d00e3fa1+9:2270:x1862456979140288:12345-192.168.204.3@tcp:400:kworker/u8:2.0 Request processed in 84us (266us total) trans 0 rc 0/0 00000100:00100000:1.0:1776178700.582274:0:20759:0:(nrs_fifo.c:241:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.204.3@tcp, seq: 899 00000100:00100000:2.0:1776178700.582286:0:6255:0:(service.c:2152:ptlrpc_server_handle_req_in()) got req x1862456979140096 00000100:00100000:1.0:1776178700.582290:0:20759:0:(nrs_fifo.c:179:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.204.3@tcp, seq: 900 00000100:00100000:1.0:1776178700.582297:0:20759:0:(service.c:2324:ptlrpc_server_handle_request()) Handling RPC req@00000000bacf691a pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_010:b50963bd-8eba-4432-9a54-8888d00e3fa1+8:2270:x1862456979140224:12345-192.168.204.3@tcp:400:kworker/u8:2.0 00000100:00100000:2.0:1776178700.582313:0:6255:0:(service.c:2152:ptlrpc_server_handle_req_in()) got req x1862456979140160 00000100:00100000:1.0:1776178700.582320:0:20759:0:(service.c:2372:ptlrpc_server_handle_request()) Handled RPC req@00000000bacf691a pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_010:b50963bd-8eba-4432-9a54-8888d00e3fa1+8:2270:x1862456979140224:12345-192.168.204.3@tcp:400:kworker/u8:2.0 Request processed in 22us (293us total) trans 0 rc 0/0 00000100:00100000:1.0:1776178700.582327:0:20759:0:(nrs_fifo.c:241:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.204.3@tcp, seq: 900 00000100:00100000:2.0:1776178700.582336:0:6255:0:(nrs_fifo.c:179:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.204.3@tcp, seq: 1219 00000100:00100000:2.0:1776178700.582353:0:6255:0:(service.c:2324:ptlrpc_server_handle_request()) Handling RPC req@000000000154171f pname:cluuid+ref:pid:xid:nid:opc:job mdt00_000:b50963bd-8eba-4432-9a54-8888d00e3fa1+6:2270:x1862456979140160:12345-192.168.204.3@tcp:400:kworker/u8:2.0 00000100:00100000:2.0:1776178700.582383:0:6255:0:(service.c:2372:ptlrpc_server_handle_request()) Handled RPC req@000000000154171f pname:cluuid+ref:pid:xid:nid:opc:job mdt00_000:b50963bd-8eba-4432-9a54-8888d00e3fa1+6:2270:x1862456979140160:12345-192.168.204.3@tcp:400:kworker/u8:2.0 Request processed in 42us (419us total) trans 0 rc 0/0 00000100:00100000:2.0:1776178700.582392:0:6255:0:(nrs_fifo.c:241:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.204.3@tcp, seq: 1219 00000100:00100000:2.0:1776178700.582406:0:6255:0:(nrs_fifo.c:179:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.204.3@tcp, seq: 7012 00000100:00100000:2.0:1776178700.582409:0:6255:0:(service.c:2324:ptlrpc_server_handle_request()) Handling RPC req@00000000d2922afa pname:cluuid+ref:pid:xid:nid:opc:job mdt00_000:0+-99:2269:x1862456979140096:12345-192.168.204.3@tcp:38:kworker/u8:2.0 00010000:02000000:2.0:1776178700.582422:0:6255:0:(ldlm_lib.c:1139:target_handle_connect()) lustre-MDT0000: Not available for connect from 192.168.204.3@tcp (stopping) 00000100:00100000:2.0:1776178700.582447:0:6255:0:(service.c:2372:ptlrpc_server_handle_request()) Handled RPC req@00000000d2922afa pname:cluuid+ref:pid:xid:nid:opc:job mdt00_000:0+-99:2269:x1862456979140096:12345-192.168.204.3@tcp:38:kworker/u8:2.0 Request processed in 37us (529us total) trans 0 rc -19/-19 00000100:00100000:2.0:1776178700.582453:0:6255:0:(nrs_fifo.c:241:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.204.3@tcp, seq: 7012 00000100:00100000:3.0:1776178700.583099:0:20140:0:(service.c:2152:ptlrpc_server_handle_req_in()) got req x1862456979140032 00000100:00100000:3.0:1776178700.583138:0:20140:0:(nrs_fifo.c:179:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.204.3@tcp, seq: 349 00000100:00100000:3.0:1776178700.583142:0:20140:0:(service.c:2324:ptlrpc_server_handle_request()) Handling RPC req@0000000021ee30f5 pname:cluuid+ref:pid:xid:nid:opc:job ll_mgs_0002:5b5c66b1-1a2f-4651-8548-0c75b9a38a94+9:2271:x1862456979140032:12345-192.168.204.3@tcp:400:kworker/u8:2.0 00000100:00100000:3.0:1776178700.583183:0:20140:0:(service.c:2372:ptlrpc_server_handle_request()) Handled RPC req@0000000021ee30f5 pname:cluuid+ref:pid:xid:nid:opc:job ll_mgs_0002:5b5c66b1-1a2f-4651-8548-0c75b9a38a94+9:2271:x1862456979140032:12345-192.168.204.3@tcp:400:kworker/u8:2.0 Request processed in 39us (1378us total) trans 0 rc 0/0 00000100:00100000:3.0:1776178700.583189:0:20140:0:(nrs_fifo.c:241:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.204.3@tcp, seq: 349 00010000:00010000:2.0:1776178700.768868:0:6244:0:(ldlm_lockd.c:2792:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00010000:00010000:1.0:1776178701.794382:0:6245:0:(ldlm_lockd.c:2792:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00080000:3.0:1776178702.176306:0:22249:0:(pinger.c:251:ptlrpc_pinger_process_import()) lustre-MDT0000-lwp-OST0000_UUID->lustre-MDT0000_UUID: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00080000:3.0:1776178702.176373:0:22249:0:(recover.c:57:ptlrpc_initiate_recovery()) lustre-MDT0000_UUID: starting recovery 00000100:00080000:3.0:1776178702.176377:0:22249:0:(import.c:86:import_set_state_nolock()) 0000000047601a98 lustre-MDT0000_UUID: changing import state from DISCONN to CONNECTING 00000100:00080000:3.0:1776178702.176384:0:22249:0:(import.c:534:import_select_connection()) lustre-MDT0000-lwp-OST0000: connect to NID 0@lo last attempt 1404 00000100:00080000:3.0:1776178702.176387:0:22249:0:(import.c:581:import_select_connection()) lustre-MDT0000-lwp-OST0000: tried all connections, increasing latency to 21s 00000100:00080000:3.0:1776178702.176393:0:22249:0:(import.c:615:import_select_connection()) lustre-MDT0000-lwp-OST0000: import 0000000047601a98 using connection 192.168.204.103@tcp/0@lo 00000100:00100000:3.0:1776178702.176418:0:22249:0:(import.c:807:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5) req@000000005c567288 x1862456998117888/t0(0) o38->lustre-MDT0000-lwp-OST0000@0@lo:12/10 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' 00000100:00080000:3.0:1776178702.176460:0:22249:0:(pinger.c:251:ptlrpc_pinger_process_import()) lustre-MDT0000-lwp-OST0001_UUID->lustre-MDT0000_UUID: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00080000:3.0:1776178702.176464:0:22249:0:(recover.c:57:ptlrpc_initiate_recovery()) lustre-MDT0000_UUID: starting recovery 00000100:00080000:3.0:1776178702.176466:0:22249:0:(import.c:86:import_set_state_nolock()) 00000000dfc8db13 lustre-MDT0000_UUID: changing import state from DISCONN to CONNECTING 00000100:00080000:3.0:1776178702.176469:0:22249:0:(import.c:534:import_select_connection()) lustre-MDT0000-lwp-OST0001: connect to NID 0@lo last attempt 1404 00000100:00080000:3.0:1776178702.176471:0:22249:0:(import.c:581:import_select_connection()) lustre-MDT0000-lwp-OST0001: tried all connections, increasing latency to 21s 00000100:00080000:3.0:1776178702.176474:0:22249:0:(import.c:615:import_select_connection()) lustre-MDT0000-lwp-OST0001: import 00000000dfc8db13 using connection 192.168.204.103@tcp/0@lo 00000100:00100000:3.0:1776178702.176483:0:22249:0:(import.c:807:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5) req@000000008f94e9dd x1862456998118016/t0(0) o38->lustre-MDT0000-lwp-OST0001@0@lo:12/10 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' 00000100:00080000:3.0:1776178702.176498:0:22249:0:(pinger.c:251:ptlrpc_pinger_process_import()) lustre-MDT0001-mdtlov_UUID->lustre-MDT0000_UUID: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00080000:3.0:1776178702.176500:0:22249:0:(recover.c:57:ptlrpc_initiate_recovery()) lustre-MDT0000_UUID: starting recovery 00000100:00080000:3.0:1776178702.176501:0:22249:0:(import.c:86:import_set_state_nolock()) 00000000ba4f1821 lustre-MDT0000_UUID: changing import state from DISCONN to CONNECTING 00000100:00080000:3.0:1776178702.176504:0:22249:0:(import.c:534:import_select_connection()) lustre-MDT0000-osp-MDT0001: connect to NID 0@lo last attempt 1404 00000100:00080000:3.0:1776178702.176505:0:22249:0:(import.c:581:import_select_connection()) lustre-MDT0000-osp-MDT0001: tried all connections, increasing latency to 21s 00000100:00080000:3.0:1776178702.176508:0:22249:0:(import.c:615:import_select_connection()) lustre-MDT0000-osp-MDT0001: import 00000000ba4f1821 using connection 192.168.204.103@tcp/0@lo 00000100:00100000:3.0:1776178702.176517:0:22249:0:(import.c:807:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5) req@00000000ad99db59 x1862456998118144/t0(0) o38->lustre-MDT0000-osp-MDT0001@0@lo:24/4 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' 00000100:00080000:3.0:1776178702.176533:0:22249:0:(pinger.c:251:ptlrpc_pinger_process_import()) lustre-MDT0000-lwp-MDT0001_UUID->lustre-MDT0000_UUID: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00080000:3.0:1776178702.176536:0:22249:0:(recover.c:57:ptlrpc_initiate_recovery()) lustre-MDT0000_UUID: starting recovery 00000100:00080000:3.0:1776178702.176538:0:22249:0:(import.c:86:import_set_state_nolock()) 00000000a7c55b1f lustre-MDT0000_UUID: changing import state from DISCONN to CONNECTING 00000100:00080000:3.0:1776178702.176541:0:22249:0:(import.c:534:import_select_connection()) lustre-MDT0000-lwp-MDT0001: connect to NID 0@lo last attempt 1404 00000100:00080000:3.0:1776178702.176544:0:22249:0:(import.c:581:import_select_connection()) lustre-MDT0000-lwp-MDT0001: tried all connections, increasing latency to 21s 00000100:00080000:3.0:1776178702.176547:0:22249:0:(import.c:615:import_select_connection()) lustre-MDT0000-lwp-MDT0001: import 00000000a7c55b1f using connection 192.168.204.103@tcp/0@lo 00000100:00100000:3.0:1776178702.176558:0:22249:0:(import.c:807:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5) req@00000000196ec025 x1862456998118336/t0(0) o38->lustre-MDT0000-lwp-MDT0001@0@lo:12/10 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' 00000100:00100000:2.0:1776178702.176559:0:3352:0:(client.c:1733:ptlrpc_send_new_req()) Sending RPC req@000000005c567288 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:lustre-MDT0000-lwp-OST0000_UUID:3352:1862456998117888:0@lo:38:kworker/u8:0.0 00000100:00100000:3.0:1776178702.176593:0:3353:0:(client.c:1733:ptlrpc_send_new_req()) Sending RPC req@0000000037d88937 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_00:lustre-MDT0001-lwp-OST0000_UUID:3353:1862456998117952:0@lo:400:kworker/u8:0.0 00000100:00100000:2.0:1776178702.176624:0:3352:0:(events.c:361:request_in_callback()) peer: 12345-0@lo (source: 12345-0@lo) 00000100:00100000:0.0:1776178702.176671:0:3355:0:(client.c:1733:ptlrpc_send_new_req()) Sending RPC req@0000000081aacc4a pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_02:lustre-MDT0001-mdtlov_UUID:3355:1862456998118208:0@lo:400:kworker/u8:0.0 00000100:00100000:1.0:1776178702.176702:0:3356:0:(client.c:1733:ptlrpc_send_new_req()) Sending RPC req@00000000de8350d4 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_03:ac61f839-0e8e-4cac-9c59-08bcfbbad330:3356:1862456998117824:0@lo:400:kworker/u8:0.0 00000100:00100000:2.0:1776178702.180887:0:3352:0:(client.c:1733:ptlrpc_send_new_req()) Sending RPC req@000000008f94e9dd pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:lustre-MDT0000-lwp-OST0001_UUID:3352:1862456998118016:0@lo:38:kworker/u8:0.0 00000100:00100000:2.0:1776178702.180946:0:3352:0:(events.c:361:request_in_callback()) peer: 12345-0@lo (source: 12345-0@lo) 00000100:00100000:2.0:1776178702.180968:0:3352:0:(client.c:1733:ptlrpc_send_new_req()) Sending RPC req@00000000ad99db59 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:lustre-MDT0001-mdtlov_UUID:3352:1862456998118144:0@lo:38:kworker/u8:0.0 00000100:00100000:2.0:1776178702.180997:0:3352:0:(events.c:361:request_in_callback()) peer: 12345-0@lo (source: 12345-0@lo) 00000100:00100000:2.0:1776178702.181022:0:3352:0:(client.c:1733:ptlrpc_send_new_req()) Sending RPC req@00000000196ec025 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:lustre-MDT0000-lwp-MDT0001_UUID:3352:1862456998118336:0@lo:38:kworker/u8:0.0 00000100:00100000:2.0:1776178702.181043:0:3352:0:(events.c:361:request_in_callback()) peer: 12345-0@lo (source: 12345-0@lo) 00000100:00100000:2.0:1776178702.181099:0:6257:0:(service.c:2152:ptlrpc_server_handle_req_in()) got req x1862456998117888 00000100:00100000:2.0:1776178702.181129:0:6257:0:(service.c:2152:ptlrpc_server_handle_req_in()) got req x1862456998118016 00000100:00100000:2.0:1776178702.181140:0:6257:0:(service.c:2152:ptlrpc_server_handle_req_in()) got req x1862456998118336 00000100:00100000:2.0:1776178702.181151:0:6257:0:(nrs_fifo.c:179:nrs_fifo_req_get()) NRS start fifo request from 12345-0@lo, seq: 7013 00000100:00100000:2.0:1776178702.181157:0:6257:0:(service.c:2324:ptlrpc_server_handle_request()) Handling RPC req@000000005f1c2da4 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_002:0+-99:3352:x1862456998117888:12345-0@lo:38:kworker/u8:0.0 00010000:02000000:2.0:1776178702.181170:0:6257:0:(ldlm_lib.c:1139:target_handle_connect()) lustre-MDT0000: Not available for connect from 0@lo (stopping) 00000100:00100000:2.0:1776178702.181217:0:6257:0:(service.c:2372:ptlrpc_server_handle_request()) Handled RPC req@000000005f1c2da4 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_002:0+-99:3352:x1862456998117888:12345-0@lo:38:kworker/u8:0.0 Request processed in 59us (4594us total) trans 0 rc -19/-19 00000100:00100000:2.0:1776178702.181224:0:6257:0:(nrs_fifo.c:241:nrs_fifo_req_stop()) NRS stop fifo request from 12345-0@lo, seq: 7013 00000100:00100000:2.0:1776178702.181239:0:6257:0:(nrs_fifo.c:179:nrs_fifo_req_get()) NRS start fifo request from 12345-0@lo, seq: 7014 00000100:00100000:2.0:1776178702.181241:0:6257:0:(service.c:2324:ptlrpc_server_handle_request()) Handling RPC req@00000000c8bb8784 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_002:0+-99:3352:x1862456998118016:12345-0@lo:38:kworker/u8:0.0 00010000:02000000:2.0:1776178702.181247:0:6257:0:(ldlm_lib.c:1139:target_handle_connect()) lustre-MDT0000: Not available for connect from 0@lo (stopping) 00000100:00100000:2.0:1776178702.181269:0:6257:0:(service.c:2372:ptlrpc_server_handle_request()) Handled RPC req@00000000c8bb8784 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_002:0+-99:3352:x1862456998118016:12345-0@lo:38:kworker/u8:0.0 Request processed in 26us (324us total) trans 0 rc -19/-19 00000100:00100000:2.0:1776178702.181274:0:6257:0:(nrs_fifo.c:241:nrs_fifo_req_stop()) NRS stop fifo request from 12345-0@lo, seq: 7014 00000100:00100000:2.0:1776178702.181283:0:6257:0:(nrs_fifo.c:179:nrs_fifo_req_get()) NRS start fifo request from 12345-0@lo, seq: 7015 00000100:00100000:2.0:1776178702.181286:0:6257:0:(service.c:2324:ptlrpc_server_handle_request()) Handling RPC req@00000000ad37fc5d pname:cluuid+ref:pid:xid:nid:opc:job mdt00_002:0+-99:3352:x1862456998118336:12345-0@lo:38:kworker/u8:0.0 00010000:02000000:2.0:1776178702.181292:0:6257:0:(ldlm_lib.c:1139:target_handle_connect()) lustre-MDT0000: Not available for connect from 0@lo (stopping) 00000100:00100000:2.0:1776178702.181312:0:6257:0:(service.c:2372:ptlrpc_server_handle_request()) Handled RPC req@00000000ad37fc5d pname:cluuid+ref:pid:xid:nid:opc:job mdt00_002:0+-99:3352:x1862456998118336:12345-0@lo:38:kworker/u8:0.0 Request processed in 24us (268us total) trans 0 rc -19/-19 00000100:00100000:2.0:1776178702.181318:0:6257:0:(nrs_fifo.c:241:nrs_fifo_req_stop()) NRS stop fifo request from 12345-0@lo, seq: 7015 00000100:00100000:2.0:1776178702.181334:0:15213:0:(service.c:2152:ptlrpc_server_handle_req_in()) got req x1862456998118144 00000100:00100000:2.0:1776178702.181350:0:15213:0:(nrs_fifo.c:179:nrs_fifo_req_get()) NRS start fifo request from 12345-0@lo, seq: 805 00000100:00100000:2.0:1776178702.181354:0:15213:0:(service.c:2324:ptlrpc_server_handle_request()) Handling RPC req@00000000038ab950 pname:cluuid+ref:pid:xid:nid:opc:job mdt_out00_003:0+-99:3352:x1862456998118144:12345-0@lo:38:kworker/u8:0.0 00010000:02000000:2.0:1776178702.181360:0:15213:0:(ldlm_lib.c:1139:target_handle_connect()) lustre-MDT0000: Not available for connect from 0@lo (stopping) 00000100:00100000:2.0:1776178702.181381:0:15213:0:(service.c:2372:ptlrpc_server_handle_request()) Handled RPC req@00000000038ab950 pname:cluuid+ref:pid:xid:nid:opc:job mdt_out00_003:0+-99:3352:x1862456998118144:12345-0@lo:38:kworker/u8:0.0 Request processed in 26us (384us total) trans 0 rc -19/-19 00000100:00100000:2.0:1776178702.181386:0:15213:0:(nrs_fifo.c:241:nrs_fifo_req_stop()) NRS stop fifo request from 12345-0@lo, seq: 805 00000100:00100000:3.0:1776178702.181426:0:3353:0:(events.c:361:request_in_callback()) peer: 12345-0@lo (source: 12345-0@lo) 00000100:00100000:3.0:1776178702.181455:0:3353:0:(ptlrpcd.c:415:ptlrpcd_check()) transfer 1 async RPCs [1->0] 00000100:00100000:3.0:1776178702.181463:0:3353:0:(client.c:1733:ptlrpc_send_new_req()) Sending RPC req@0000000045cb5035 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_00:lustre-MDT0001-lwp-OST0001_UUID:3353:1862456998118080:0@lo:400:kworker/u8:0.0 00000100:00100000:3.0:1776178702.181490:0:3353:0:(events.c:361:request_in_callback()) peer: 12345-0@lo (source: 12345-0@lo) 00000100:00100000:3.0:1776178702.181526:0:8250:0:(service.c:2152:ptlrpc_server_handle_req_in()) got req x1862456998117952 00000100:00100000:3.0:1776178702.181549:0:8250:0:(service.c:2152:ptlrpc_server_handle_req_in()) got req x1862456998118080 00000100:00100000:3.0:1776178702.181560:0:8250:0:(nrs_fifo.c:179:nrs_fifo_req_get()) NRS start fifo request from 12345-0@lo, seq: 1220 00000100:00100000:3.0:1776178702.181564:0:8250:0:(service.c:2324:ptlrpc_server_handle_request()) Handling RPC req@000000009a62f69c pname:cluuid+ref:pid:xid:nid:opc:job mdt00_003:lustre-MDT0001-lwp-OST0000_UUID+5:3353:x1862456998117952:12345-0@lo:400:kworker/u8:0.0 00000100:00100000:3.0:1776178702.181606:0:8250:0:(service.c:2372:ptlrpc_server_handle_request()) Handled RPC req@000000009a62f69c pname:cluuid+ref:pid:xid:nid:opc:job mdt00_003:lustre-MDT0001-lwp-OST0000_UUID+5:3353:x1862456998117952:12345-0@lo:400:kworker/u8:0.0 Request processed in 42us (184us total) trans 0 rc 0/0 00000100:00100000:3.0:1776178702.181613:0:8250:0:(nrs_fifo.c:241:nrs_fifo_req_stop()) NRS stop fifo request from 12345-0@lo, seq: 1220 00000100:00100000:3.0:1776178702.181626:0:8250:0:(nrs_fifo.c:179:nrs_fifo_req_get()) NRS start fifo request from 12345-0@lo, seq: 1221 00000100:00100000:3.0:1776178702.181692:0:8250:0:(service.c:2324:ptlrpc_server_handle_request()) Handling RPC req@0000000007e413b7 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_003:lustre-MDT0001-lwp-OST0001_UUID+5:3353:x1862456998118080:12345-0@lo:400:kworker/u8:0.0 00000100:00100000:0.0:1776178702.181709:0:3355:0:(events.c:361:request_in_callback()) peer: 12345-0@lo (source: 12345-0@lo) 00000100:00100000:3.0:1776178702.181727:0:8250:0:(service.c:2372:ptlrpc_server_handle_request()) Handled RPC req@0000000007e413b7 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_003:lustre-MDT0001-lwp-OST0001_UUID+5:3353:x1862456998118080:12345-0@lo:400:kworker/u8:0.0 Request processed in 34us (236us total) trans 0 rc 0/0 00000100:00100000:3.0:1776178702.181735:0:8250:0:(nrs_fifo.c:241:nrs_fifo_req_stop()) NRS stop fifo request from 12345-0@lo, seq: 1221 00000100:00100000:0.0:1776178702.181765:0:20762:0:(service.c:2152:ptlrpc_server_handle_req_in()) got req x1862456998118208 00000100:00100000:0.0:1776178702.181795:0:20762:0:(nrs_fifo.c:179:nrs_fifo_req_get()) NRS start fifo request from 12345-0@lo, seq: 901 00000100:00100000:0.0:1776178702.181799:0:20762:0:(service.c:2324:ptlrpc_server_handle_request()) Handling RPC req@00000000e4ab0682 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_013:lustre-MDT0001-mdtlov_UUID+5:3355:x1862456998118208:12345-0@lo:400:kworker/u8:0.0 00000100:00100000:0.0:1776178702.181832:0:20762:0:(service.c:2372:ptlrpc_server_handle_request()) Handled RPC req@00000000e4ab0682 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_013:lustre-MDT0001-mdtlov_UUID+5:3355:x1862456998118208:12345-0@lo:400:kworker/u8:0.0 Request processed in 32us (125us total) trans 0 rc 0/0 00000100:00100000:0.0:1776178702.181837:0:20762:0:(nrs_fifo.c:241:nrs_fifo_req_stop()) NRS stop fifo request from 12345-0@lo, seq: 901 00000100:00100000:0.0:1776178702.181870:0:3355:0:(client.c:2215:ptlrpc_check_set()) Completed RPC req@0000000081aacc4a pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_02:lustre-MDT0001-mdtlov_UUID:3355:1862456998118208:0@lo:400:kworker/u8:0.0 00000100:00080000:0.0:1776178702.181918:0:3352:0:(import.c:86:import_set_state_nolock()) 00000000a7c55b1f lustre-MDT0000_UUID: changing import state from CONNECTING to DISCONN 00000100:00080000:0.0:1776178702.181922:0:3352:0:(import.c:1429:ptlrpc_connect_interpret()) recovery of lustre-MDT0000_UUID on 192.168.204.103@tcp failed (-19) 00000100:00100000:0.0:1776178702.181925:0:3352:0:(client.c:2215:ptlrpc_check_set()) Completed RPC req@00000000196ec025 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:lustre-MDT0000-lwp-MDT0001_UUID:3352:1862456998118336:0@lo:38:kworker/u8:0.0 00000100:00080000:0.0:1776178702.181933:0:3352:0:(import.c:86:import_set_state_nolock()) 0000000047601a98 lustre-MDT0000_UUID: changing import state from CONNECTING to DISCONN 00000100:00080000:0.0:1776178702.181936:0:3352:0:(import.c:1429:ptlrpc_connect_interpret()) recovery of lustre-MDT0000_UUID on 192.168.204.103@tcp failed (-19) 00000100:00100000:0.0:1776178702.181938:0:3352:0:(client.c:2215:ptlrpc_check_set()) Completed RPC req@000000005c567288 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:lustre-MDT0000-lwp-OST0000_UUID:3352:1862456998117888:0@lo:38:kworker/u8:0.0 00000100:00080000:0.0:1776178702.181945:0:3352:0:(import.c:86:import_set_state_nolock()) 00000000dfc8db13 lustre-MDT0000_UUID: changing import state from CONNECTING to DISCONN 00000100:00080000:0.0:1776178702.181947:0:3352:0:(import.c:1429:ptlrpc_connect_interpret()) recovery of lustre-MDT0000_UUID on 192.168.204.103@tcp failed (-19) 00000100:00100000:0.0:1776178702.181949:0:3352:0:(client.c:2215:ptlrpc_check_set()) Completed RPC req@000000008f94e9dd pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:lustre-MDT0000-lwp-OST0001_UUID:3352:1862456998118016:0@lo:38:kworker/u8:0.0 00000100:00080000:0.0:1776178702.181955:0:3352:0:(import.c:86:import_set_state_nolock()) 00000000ba4f1821 lustre-MDT0000_UUID: changing import state from CONNECTING to DISCONN 00000100:00080000:0.0:1776178702.181957:0:3352:0:(import.c:1429:ptlrpc_connect_interpret()) recovery of lustre-MDT0000_UUID on 192.168.204.103@tcp failed (-19) 00000100:00100000:0.0:1776178702.181959:0:3352:0:(client.c:2215:ptlrpc_check_set()) Completed RPC req@00000000ad99db59 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:lustre-MDT0001-mdtlov_UUID:3352:1862456998118144:0@lo:38:kworker/u8:0.0 00000100:00100000:3.0:1776178702.183098:0:3353:0:(client.c:2215:ptlrpc_check_set()) Completed RPC req@0000000045cb5035 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_00:lustre-MDT0001-lwp-OST0001_UUID:3353:1862456998118080:0@lo:400:kworker/u8:0.0 00000100:00100000:3.0:1776178702.183443:0:3353:0:(client.c:2215:ptlrpc_check_set()) Completed RPC req@0000000037d88937 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_00:lustre-MDT0001-lwp-OST0000_UUID:3353:1862456998117952:0@lo:400:kworker/u8:0.0 00000100:00100000:1.0:1776178702.191417:0:3356:0:(events.c:361:request_in_callback()) peer: 12345-0@lo (source: 12345-0@lo) 00000100:00100000:1.0:1776178702.191470:0:3356:0:(client.c:1733:ptlrpc_send_new_req()) Sending RPC req@00000000be147540 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_03:lustre-MDT0001-mdtlov_UUID:3356:1862456998118272:0@lo:400:kworker/u8:0.0 00000100:00100000:1.0:1776178702.191518:0:3356:0:(events.c:361:request_in_callback()) peer: 12345-0@lo (source: 12345-0@lo) 00000100:00100000:0.0:1776178702.191870:0:20755:0:(service.c:2152:ptlrpc_server_handle_req_in()) got req x1862456998118272 00000100:00100000:0.0:1776178702.191923:0:20755:0:(nrs_fifo.c:179:nrs_fifo_req_get()) NRS start fifo request from 12345-0@lo, seq: 902 00000100:00100000:0.0:1776178702.191930:0:20755:0:(service.c:2324:ptlrpc_server_handle_request()) Handling RPC req@00000000e178f924 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_006:lustre-MDT0001-mdtlov_UUID+5:3356:x1862456998118272:12345-0@lo:400:kworker/u8:0.0 00000100:00100000:0.0:1776178702.191995:0:20755:0:(service.c:2372:ptlrpc_server_handle_request()) Handled RPC req@00000000e178f924 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_006:lustre-MDT0001-mdtlov_UUID+5:3356:x1862456998118272:12345-0@lo:400:kworker/u8:0.0 Request processed in 64us (477us total) trans 0 rc 0/0 00000100:00100000:0.0:1776178702.192002:0:20755:0:(nrs_fifo.c:241:nrs_fifo_req_stop()) NRS stop fifo request from 12345-0@lo, seq: 902 00000100:00100000:2.0:1776178702.192495:0:3356:0:(client.c:2215:ptlrpc_check_set()) Completed RPC req@00000000be147540 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_03:lustre-MDT0001-mdtlov_UUID:3356:1862456998118272:0@lo:400:kworker/u8:0.0 00000100:00100000:3.0:1776178702.196771:0:20140:0:(service.c:2152:ptlrpc_server_handle_req_in()) got req x1862456998117824 00000100:00100000:3.0:1776178702.196833:0:20140:0:(nrs_fifo.c:179:nrs_fifo_req_get()) NRS start fifo request from 12345-0@lo, seq: 350 00000100:00100000:3.0:1776178702.196839:0:20140:0:(service.c:2324:ptlrpc_server_handle_request()) Handling RPC req@000000006efdb84e pname:cluuid+ref:pid:xid:nid:opc:job ll_mgs_0002:ac61f839-0e8e-4cac-9c59-08bcfbbad330+18:3356:x1862456998117824:12345-0@lo:400:kworker/u8:0.0 00000100:00100000:3.0:1776178702.196932:0:20140:0:(service.c:2372:ptlrpc_server_handle_request()) Handled RPC req@000000006efdb84e pname:cluuid+ref:pid:xid:nid:opc:job ll_mgs_0002:ac61f839-0e8e-4cac-9c59-08bcfbbad330+18:3356:x1862456998117824:12345-0@lo:400:kworker/u8:0.0 Request processed in 92us (5523us total) trans 0 rc 0/0 00000100:00100000:3.0:1776178702.196940:0:20140:0:(nrs_fifo.c:241:nrs_fifo_req_stop()) NRS stop fifo request from 12345-0@lo, seq: 350 00000100:00100000:2.0:1776178702.197107:0:3356:0:(client.c:2215:ptlrpc_check_set()) Completed RPC req@00000000de8350d4 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_03:ac61f839-0e8e-4cac-9c59-08bcfbbad330:3356:1862456998117824:0@lo:400:kworker/u8:0.0 00010000:00010000:2.0:1776178702.816281:0:9224:0:(ldlm_lockd.c:2792:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:3.0:1776178703.716718:0:3353:0:(client.c:1733:ptlrpc_send_new_req()) Sending RPC req@000000001f88d899 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_00:lustre-MDT0001-mdtlov_UUID:3353:1862456998118400:0@lo:13:osp-pre-1-1.0 00000100:00100000:3.0:1776178703.716798:0:3353:0:(events.c:361:request_in_callback()) peer: 12345-0@lo (source: 12345-0@lo) 00000100:00100000:2.0:1776178703.717367:0:8231:0:(service.c:2152:ptlrpc_server_handle_req_in()) got req x1862456998118400 00000100:00100000:2.0:1776178703.717427:0:8231:0:(nrs_fifo.c:179:nrs_fifo_req_get()) NRS start fifo request from 12345-0@lo, seq: 683 00000100:00100000:2.0:1776178703.717433:0:8231:0:(service.c:2324:ptlrpc_server_handle_request()) Handling RPC req@00000000ffd05959 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_create00:lustre-MDT0001-mdtlov_UUID+5:3353:x1862456998118400:12345-0@lo:13:osp-pre-1-1.0 00002000:00000024:2.0:1776178703.717464:0:8231:0:(ofd_obd.c:740:ofd_statfs()) blocks cached 229376 granted 491746368 pending 0 free 3923689472 avail 3692167168 00000100:00100000:2.0:1776178703.717533:0:8231:0:(service.c:2372:ptlrpc_server_handle_request()) Handled RPC req@00000000ffd05959 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_create00:lustre-MDT0001-mdtlov_UUID+5:3353:x1862456998118400:12345-0@lo:13:osp-pre-1-1.0 Request processed in 98us (735us total) trans 0 rc 0/0 00000100:00100000:2.0:1776178703.717541:0:8231:0:(nrs_fifo.c:241:nrs_fifo_req_stop()) NRS stop fifo request from 12345-0@lo, seq: 683 00000100:00100000:0.0:1776178703.719104:0:3353:0:(client.c:2215:ptlrpc_check_set()) Completed RPC req@000000001f88d899 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_00:lustre-MDT0001-mdtlov_UUID:3353:1862456998118400:0@lo:13:osp-pre-1-1.0 00010000:00010000:1.0:1776178703.840396:0:9225:0:(ldlm_lockd.c:2792:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:3.0:1776178704.225192:0:3353:0:(ptlrpcd.c:415:ptlrpcd_check()) transfer 1 async RPCs [1->0] 00000100:00100000:3.0:1776178704.225217:0:3353:0:(client.c:1733:ptlrpc_send_new_req()) Sending RPC req@00000000fdb309c4 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_00:lustre-MDT0001-mdtlov_UUID:3353:1862456998118464:0@lo:13:osp-pre-0-1.0 00000100:00100000:3.0:1776178704.225285:0:3353:0:(events.c:361:request_in_callback()) peer: 12345-0@lo (source: 12345-0@lo) 00000100:00100000:2.0:1776178704.225532:0:9554:0:(service.c:2152:ptlrpc_server_handle_req_in()) got req x1862456998118464 00000100:00100000:2.0:1776178704.225578:0:9554:0:(nrs_fifo.c:179:nrs_fifo_req_get()) NRS start fifo request from 12345-0@lo, seq: 684 00000100:00100000:2.0:1776178704.225584:0:9554:0:(service.c:2324:ptlrpc_server_handle_request()) Handling RPC req@000000007309261c pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_create00:lustre-MDT0001-mdtlov_UUID+5:3353:x1862456998118464:12345-0@lo:13:osp-pre-0-1.0 00002000:00000024:2.0:1776178704.225616:0:9554:0:(ofd_obd.c:740:ofd_statfs()) blocks cached 315392 granted 484883776 pending 0 free 3923689472 avail 3692167168 00000100:00100000:2.0:1776178704.225666:0:9554:0:(service.c:2372:ptlrpc_server_handle_request()) Handled RPC req@000000007309261c pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_create00:lustre-MDT0001-mdtlov_UUID+5:3353:x1862456998118464:12345-0@lo:13:osp-pre-0-1.0 Request processed in 81us (382us total) trans 0 rc 0/0 00000100:00100000:2.0:1776178704.225673:0:9554:0:(nrs_fifo.c:241:nrs_fifo_req_stop()) NRS stop fifo request from 12345-0@lo, seq: 684 00000100:00100000:3.0:1776178704.225759:0:3353:0:(client.c:2215:ptlrpc_check_set()) Completed RPC req@00000000fdb309c4 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_00:lustre-MDT0001-mdtlov_UUID:3353:1862456998118464:0@lo:13:osp-pre-0-1.0 00010000:00010000:2.0:1776178704.864937:0:6244:0:(ldlm_lockd.c:2792:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:1.0:1776178705.713193:0:3346:0:(events.c:361:request_in_callback()) peer: 12345-192.168.204.3@tcp (source: 12345-192.168.204.3@tcp) 00000100:00100000:1.0:1776178705.713237:0:3346:0:(events.c:361:request_in_callback()) peer: 12345-192.168.204.3@tcp (source: 12345-192.168.204.3@tcp) 00000100:00100000:1.0:1776178705.713269:0:3346:0:(events.c:361:request_in_callback()) peer: 12345-192.168.204.3@tcp (source: 12345-192.168.204.3@tcp) 00000100:00100000:1.0:1776178705.713298:0:3346:0:(events.c:361:request_in_callback()) peer: 12345-192.168.204.3@tcp (source: 12345-192.168.204.3@tcp) 00000100:00100000:1.0:1776178705.713323:0:3346:0:(events.c:361:request_in_callback()) peer: 12345-192.168.204.3@tcp (source: 12345-192.168.204.3@tcp) 00000100:00100000:1.0:1776178705.713361:0:11281:0:(service.c:2152:ptlrpc_server_handle_req_in()) got req x1862456979140416 00000100:00100000:1.0:1776178705.713400:0:11281:0:(service.c:2152:ptlrpc_server_handle_req_in()) got req x1862456979140480 00000100:00100000:1.0:1776178705.713429:0:11281:0:(nrs_fifo.c:179:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.204.3@tcp, seq: 1222 00000100:00100000:1.0:1776178705.713436:0:11281:0:(service.c:2324:ptlrpc_server_handle_request()) Handling RPC req@00000000a33eecf4 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_004:b50963bd-8eba-4432-9a54-8888d00e3fa1+6:2273:x1862456979140480:12345-192.168.204.3@tcp:400:kworker/u8:3.0 00000100:00100000:1.0:1776178705.713526:0:11281:0:(service.c:2372:ptlrpc_server_handle_request()) Handled RPC req@00000000a33eecf4 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_004:b50963bd-8eba-4432-9a54-8888d00e3fa1+6:2273:x1862456979140480:12345-192.168.204.3@tcp:400:kworker/u8:3.0 Request processed in 90us (289us total) trans 0 rc 0/0 00000100:00100000:1.0:1776178705.713534:0:11281:0:(nrs_fifo.c:241:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.204.3@tcp, seq: 1222 00000100:00100000:1.0:1776178705.713551:0:11281:0:(nrs_fifo.c:179:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.204.3@tcp, seq: 7016 00000100:00100000:1.0:1776178705.713554:0:11281:0:(service.c:2324:ptlrpc_server_handle_request()) Handling RPC req@000000008245b0ee pname:cluuid+ref:pid:xid:nid:opc:job mdt00_004:0+-99:2269:x1862456979140416:12345-192.168.204.3@tcp:38:kworker/u8:3.0 00010000:02000000:1.0:1776178705.713567:0:11281:0:(ldlm_lib.c:1139:target_handle_connect()) lustre-MDT0000: Not available for connect from 192.168.204.3@tcp (stopping) 00000100:00100000:1.0:1776178705.713865:0:11281:0:(service.c:2372:ptlrpc_server_handle_request()) Handled RPC req@000000008245b0ee pname:cluuid+ref:pid:xid:nid:opc:job mdt00_004:0+-99:2269:x1862456979140416:12345-192.168.204.3@tcp:38:kworker/u8:3.0 Request processed in 308us (683us total) trans 0 rc -19/-19 00000100:00100000:1.0:1776178705.713874:0:11281:0:(nrs_fifo.c:241:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.204.3@tcp, seq: 7016 00000100:00100000:1.0:1776178705.713897:0:20762:0:(service.c:2152:ptlrpc_server_handle_req_in()) got req x1862456979140608 00000100:00100000:1.0:1776178705.713920:0:20762:0:(service.c:2152:ptlrpc_server_handle_req_in()) got req x1862456979140544 00000100:00100000:1.0:1776178705.713937:0:20762:0:(nrs_fifo.c:179:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.204.3@tcp, seq: 903 00000100:00100000:1.0:1776178705.713941:0:20762:0:(service.c:2324:ptlrpc_server_handle_request()) Handling RPC req@00000000a66f2317 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_013:b50963bd-8eba-4432-9a54-8888d00e3fa1+9:2273:x1862456979140608:12345-192.168.204.3@tcp:400:kworker/u8:3.0 00000100:00100000:1.0:1776178705.713967:0:20762:0:(service.c:2372:ptlrpc_server_handle_request()) Handled RPC req@00000000a66f2317 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_013:b50963bd-8eba-4432-9a54-8888d00e3fa1+9:2273:x1862456979140608:12345-192.168.204.3@tcp:400:kworker/u8:3.0 Request processed in 25us (699us total) trans 0 rc 0/0 00000100:00100000:1.0:1776178705.713973:0:20762:0:(nrs_fifo.c:241:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.204.3@tcp, seq: 903 00000100:00100000:1.0:1776178705.713984:0:20762:0:(nrs_fifo.c:179:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.204.3@tcp, seq: 904 00000100:00100000:1.0:1776178705.713988:0:20762:0:(service.c:2324:ptlrpc_server_handle_request()) Handling RPC req@00000000d94d52a5 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_013:b50963bd-8eba-4432-9a54-8888d00e3fa1+8:2272:x1862456979140544:12345-192.168.204.3@tcp:400:kworker/u8:3.0 00000100:00100000:1.0:1776178705.714008:0:20762:0:(service.c:2372:ptlrpc_server_handle_request()) Handled RPC req@00000000d94d52a5 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_013:b50963bd-8eba-4432-9a54-8888d00e3fa1+8:2272:x1862456979140544:12345-192.168.204.3@tcp:400:kworker/u8:3.0 Request processed in 20us (686us total) trans 0 rc 0/0 00000100:00100000:1.0:1776178705.714014:0:20762:0:(nrs_fifo.c:241:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.204.3@tcp, seq: 904 00000100:00100000:1.0:1776178705.714044:0:20137:0:(service.c:2152:ptlrpc_server_handle_req_in()) got req x1862456979140352 00000100:00100000:1.0:1776178705.714087:0:20137:0:(nrs_fifo.c:179:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.204.3@tcp, seq: 351 00000100:00100000:1.0:1776178705.714091:0:20137:0:(service.c:2324:ptlrpc_server_handle_request()) Handling RPC req@0000000038b01669 pname:cluuid+ref:pid:xid:nid:opc:job ll_mgs_0000:5b5c66b1-1a2f-4651-8548-0c75b9a38a94+9:2272:x1862456979140352:12345-192.168.204.3@tcp:400:kworker/u8:3.0 00000100:00100000:1.0:1776178705.714113:0:20137:0:(service.c:2372:ptlrpc_server_handle_request()) Handled RPC req@0000000038b01669 pname:cluuid+ref:pid:xid:nid:opc:job ll_mgs_0000:5b5c66b1-1a2f-4651-8548-0c75b9a38a94+9:2272:x1862456979140352:12345-192.168.204.3@tcp:400:kworker/u8:3.0 Request processed in 22us (816us total) trans 0 rc 0/0 00000100:00100000:1.0:1776178705.714119:0:20137:0:(nrs_fifo.c:241:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.204.3@tcp, seq: 351 00010000:00010000:1.0:1776178705.890597:0:6245:0:(ldlm_lockd.c:2792:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00010000:00010000:2.0:1776178706.912825:0:9224:0:(ldlm_lockd.c:2792:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00080000:0.0:1776178707.296642:0:22249:0:(pinger.c:251:ptlrpc_pinger_process_import()) lustre-MDT0000-lwp-OST0000_UUID->lustre-MDT0000_UUID: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00080000:0.0:1776178707.296650:0:22249:0:(recover.c:57:ptlrpc_initiate_recovery()) lustre-MDT0000_UUID: starting recovery 00000100:00080000:0.0:1776178707.296653:0:22249:0:(import.c:86:import_set_state_nolock()) 0000000047601a98 lustre-MDT0000_UUID: changing import state from DISCONN to CONNECTING 00000100:00080000:0.0:1776178707.296661:0:22249:0:(import.c:534:import_select_connection()) lustre-MDT0000-lwp-OST0000: connect to NID 0@lo last attempt 1409 00000100:00080000:0.0:1776178707.296665:0:22249:0:(import.c:581:import_select_connection()) lustre-MDT0000-lwp-OST0000: tried all connections, increasing latency to 21s 00000100:00080000:0.0:1776178707.296672:0:22249:0:(import.c:615:import_select_connection()) lustre-MDT0000-lwp-OST0000: import 0000000047601a98 using connection 192.168.204.103@tcp/0@lo 00000100:00100000:0.0:1776178707.296695:0:22249:0:(import.c:807:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5) req@000000001ff6e514 x1862456998118592/t0(0) o38->lustre-MDT0000-lwp-OST0000@0@lo:12/10 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' 00000100:00080000:0.0:1776178707.296727:0:22249:0:(pinger.c:251:ptlrpc_pinger_process_import()) lustre-MDT0000-lwp-OST0001_UUID->lustre-MDT0000_UUID: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00080000:0.0:1776178707.296730:0:22249:0:(recover.c:57:ptlrpc_initiate_recovery()) lustre-MDT0000_UUID: starting recovery 00000100:00080000:0.0:1776178707.296731:0:22249:0:(import.c:86:import_set_state_nolock()) 00000000dfc8db13 lustre-MDT0000_UUID: changing import state from DISCONN to CONNECTING 00000100:00080000:0.0:1776178707.296734:0:22249:0:(import.c:534:import_select_connection()) lustre-MDT0000-lwp-OST0001: connect to NID 0@lo last attempt 1409 00000100:00080000:0.0:1776178707.296736:0:22249:0:(import.c:581:import_select_connection()) lustre-MDT0000-lwp-OST0001: tried all connections, increasing latency to 21s 00000100:00080000:0.0:1776178707.296738:0:22249:0:(import.c:615:import_select_connection()) lustre-MDT0000-lwp-OST0001: import 00000000dfc8db13 using connection 192.168.204.103@tcp/0@lo 00000100:00100000:0.0:1776178707.296747:0:22249:0:(import.c:807:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5) req@000000008a15a61d x1862456998118720/t0(0) o38->lustre-MDT0000-lwp-OST0001@0@lo:12/10 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' 00000100:00080000:0.0:1776178707.296772:0:22249:0:(pinger.c:251:ptlrpc_pinger_process_import()) lustre-MDT0001-mdtlov_UUID->lustre-MDT0000_UUID: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00080000:0.0:1776178707.296776:0:22249:0:(recover.c:57:ptlrpc_initiate_recovery()) lustre-MDT0000_UUID: starting recovery 00000100:00080000:0.0:1776178707.296777:0:22249:0:(import.c:86:import_set_state_nolock()) 00000000ba4f1821 lustre-MDT0000_UUID: changing import state from DISCONN to CONNECTING 00000100:00080000:0.0:1776178707.296779:0:22249:0:(import.c:534:import_select_connection()) lustre-MDT0000-osp-MDT0001: connect to NID 0@lo last attempt 1409 00000100:00080000:0.0:1776178707.296781:0:22249:0:(import.c:581:import_select_connection()) lustre-MDT0000-osp-MDT0001: tried all connections, increasing latency to 21s 00000100:00080000:0.0:1776178707.296783:0:22249:0:(import.c:615:import_select_connection()) lustre-MDT0000-osp-MDT0001: import 00000000ba4f1821 using connection 192.168.204.103@tcp/0@lo 00000100:00100000:0.0:1776178707.296792:0:22249:0:(import.c:807:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5) req@000000007cfd1c43 x1862456998118848/t0(0) o38->lustre-MDT0000-osp-MDT0001@0@lo:24/4 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' 00000100:00080000:0.0:1776178707.296811:0:22249:0:(pinger.c:251:ptlrpc_pinger_process_import()) lustre-MDT0000-lwp-MDT0001_UUID->lustre-MDT0000_UUID: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00080000:0.0:1776178707.296814:0:22249:0:(recover.c:57:ptlrpc_initiate_recovery()) lustre-MDT0000_UUID: starting recovery 00000100:00080000:0.0:1776178707.296816:0:22249:0:(import.c:86:import_set_state_nolock()) 00000000a7c55b1f lustre-MDT0000_UUID: changing import state from DISCONN to CONNECTING 00000100:00080000:0.0:1776178707.296818:0:22249:0:(import.c:534:import_select_connection()) lustre-MDT0000-lwp-MDT0001: connect to NID 0@lo last attempt 1409 00000100:00080000:0.0:1776178707.296821:0:22249:0:(import.c:581:import_select_connection()) lustre-MDT0000-lwp-MDT0001: tried all connections, increasing latency to 21s 00000100:00080000:0.0:1776178707.296824:0:22249:0:(import.c:615:import_select_connection()) lustre-MDT0000-lwp-MDT0001: import 00000000a7c55b1f using connection 192.168.204.103@tcp/0@lo 00000100:00100000:0.0:1776178707.296833:0:22249:0:(import.c:807:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5) req@00000000dc38eb5b x1862456998119040/t0(0) o38->lustre-MDT0000-lwp-MDT0001@0@lo:12/10 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' 00000100:00100000:0.0:1776178707.296899:0:3354:0:(client.c:1733:ptlrpc_send_new_req()) Sending RPC req@00000000de8350d4 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_01:lustre-MDT0001-mdtlov_UUID:3354:1862456998118912:0@lo:400:kworker/u8:0.0 00000100:00100000:1.0:1776178707.297092:0:3356:0:(client.c:1733:ptlrpc_send_new_req()) Sending RPC req@00000000fa20ba2c pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_03:lustre-MDT0001-lwp-OST0000_UUID:3356:1862456998118656:0@lo:400:kworker/u8:0.0 00000100:00100000:1.0:1776178707.297149:0:3356:0:(events.c:361:request_in_callback()) peer: 12345-0@lo (source: 12345-0@lo) 00000100:00100000:1.0:1776178707.297182:0:3356:0:(ptlrpcd.c:415:ptlrpcd_check()) transfer 2 async RPCs [2->3] 00000100:00100000:1.0:1776178707.297187:0:3356:0:(client.c:1733:ptlrpc_send_new_req()) Sending RPC req@000000006c0c7de5 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_03:ac61f839-0e8e-4cac-9c59-08bcfbbad330:3356:1862456998118528:0@lo:400:kworker/u8:0.0 00000100:00100000:1.0:1776178707.297228:0:3356:0:(events.c:361:request_in_callback()) peer: 12345-0@lo (source: 12345-0@lo) 00000100:00100000:1.0:1776178707.297246:0:3356:0:(client.c:1733:ptlrpc_send_new_req()) Sending RPC req@00000000be147540 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_03:lustre-MDT0001-mdtlov_UUID:3356:1862456998118976:0@lo:400:kworker/u8:0.0 00000100:00100000:1.0:1776178707.297415:0:6257:0:(service.c:2152:ptlrpc_server_handle_req_in()) got req x1862456998118656 00000100:00100000:1.0:1776178707.297455:0:6257:0:(nrs_fifo.c:179:nrs_fifo_req_get()) NRS start fifo request from 12345-0@lo, seq: 1223 00000100:00100000:1.0:1776178707.297461:0:6257:0:(service.c:2324:ptlrpc_server_handle_request()) Handling RPC req@00000000aba13576 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_002:lustre-MDT0001-lwp-OST0000_UUID+5:3356:x1862456998118656:12345-0@lo:400:kworker/u8:0.0 00000100:00100000:1.0:1776178707.297509:0:6257:0:(service.c:2372:ptlrpc_server_handle_request()) Handled RPC req@00000000aba13576 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_002:lustre-MDT0001-lwp-OST0000_UUID+5:3356:x1862456998118656:12345-0@lo:400:kworker/u8:0.0 Request processed in 48us (362us total) trans 0 rc 0/0 00000100:00100000:1.0:1776178707.297516:0:6257:0:(nrs_fifo.c:241:nrs_fifo_req_stop()) NRS stop fifo request from 12345-0@lo, seq: 1223 00000100:00100000:1.0:1776178707.297538:0:20140:0:(service.c:2152:ptlrpc_server_handle_req_in()) got req x1862456998118528 00000100:00100000:1.0:1776178707.297558:0:20140:0:(nrs_fifo.c:179:nrs_fifo_req_get()) NRS start fifo request from 12345-0@lo, seq: 352 00000100:00100000:1.0:1776178707.297562:0:20140:0:(service.c:2324:ptlrpc_server_handle_request()) Handling RPC req@000000003524ab73 pname:cluuid+ref:pid:xid:nid:opc:job ll_mgs_0002:ac61f839-0e8e-4cac-9c59-08bcfbbad330+18:3356:x1862456998118528:12345-0@lo:400:kworker/u8:0.0 00000100:00100000:1.0:1776178707.297587:0:20140:0:(service.c:2372:ptlrpc_server_handle_request()) Handled RPC req@000000003524ab73 pname:cluuid+ref:pid:xid:nid:opc:job ll_mgs_0002:ac61f839-0e8e-4cac-9c59-08bcfbbad330+18:3356:x1862456998118528:12345-0@lo:400:kworker/u8:0.0 Request processed in 24us (359us total) trans 0 rc 0/0 00000100:00100000:1.0:1776178707.297592:0:20140:0:(nrs_fifo.c:241:nrs_fifo_req_stop()) NRS stop fifo request from 12345-0@lo, seq: 352 00000100:00100000:1.0:1776178707.297760:0:3353:0:(client.c:1733:ptlrpc_send_new_req()) Sending RPC req@00000000589b0247 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_00:lustre-MDT0001-lwp-OST0001_UUID:3353:1862456998118784:0@lo:400:kworker/u8:0.0 00000100:00100000:1.0:1776178707.297785:0:3353:0:(events.c:361:request_in_callback()) peer: 12345-0@lo (source: 12345-0@lo) 00000100:00100000:1.0:1776178707.297806:0:6257:0:(service.c:2152:ptlrpc_server_handle_req_in()) got req x1862456998118784 00000100:00100000:1.0:1776178707.297825:0:6257:0:(nrs_fifo.c:179:nrs_fifo_req_get()) NRS start fifo request from 12345-0@lo, seq: 1224 00000100:00100000:1.0:1776178707.297828:0:6257:0:(service.c:2324:ptlrpc_server_handle_request()) Handling RPC req@00000000d1e0f01c pname:cluuid+ref:pid:xid:nid:opc:job mdt00_002:lustre-MDT0001-lwp-OST0001_UUID+5:3353:x1862456998118784:12345-0@lo:400:kworker/u8:0.0 00000100:00100000:1.0:1776178707.297853:0:6257:0:(service.c:2372:ptlrpc_server_handle_request()) Handled RPC req@00000000d1e0f01c pname:cluuid+ref:pid:xid:nid:opc:job mdt00_002:lustre-MDT0001-lwp-OST0001_UUID+5:3353:x1862456998118784:12345-0@lo:400:kworker/u8:0.0 Request processed in 24us (69us total) trans 0 rc 0/0 00000100:00100000:1.0:1776178707.297859:0:6257:0:(nrs_fifo.c:241:nrs_fifo_req_stop()) NRS stop fifo request from 12345-0@lo, seq: 1224 00000100:00100000:1.0:1776178707.297890:0:3353:0:(client.c:2215:ptlrpc_check_set()) Completed RPC req@00000000589b0247 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_00:lustre-MDT0001-lwp-OST0001_UUID:3353:1862456998118784:0@lo:400:kworker/u8:0.0 00000100:00100000:3.0:1776178707.298767:0:3352:0:(client.c:1733:ptlrpc_send_new_req()) Sending RPC req@000000001ff6e514 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:lustre-MDT0000-lwp-OST0000_UUID:3352:1862456998118592:0@lo:38:kworker/u8:0.0 00000100:00100000:3.0:1776178707.298805:0:3352:0:(events.c:361:request_in_callback()) peer: 12345-0@lo (source: 12345-0@lo) 00000100:00100000:3.0:1776178707.298823:0:3352:0:(client.c:1733:ptlrpc_send_new_req()) Sending RPC req@000000008a15a61d pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:lustre-MDT0000-lwp-OST0001_UUID:3352:1862456998118720:0@lo:38:kworker/u8:0.0 00000100:00100000:3.0:1776178707.298844:0:3352:0:(events.c:361:request_in_callback()) peer: 12345-0@lo (source: 12345-0@lo) 00000100:00100000:3.0:1776178707.298858:0:3352:0:(client.c:1733:ptlrpc_send_new_req()) Sending RPC req@000000007cfd1c43 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:lustre-MDT0001-mdtlov_UUID:3352:1862456998118848:0@lo:38:kworker/u8:0.0 00000100:00100000:3.0:1776178707.298880:0:3352:0:(events.c:361:request_in_callback()) peer: 12345-0@lo (source: 12345-0@lo) 00000100:00100000:3.0:1776178707.298893:0:3352:0:(client.c:1733:ptlrpc_send_new_req()) Sending RPC req@00000000dc38eb5b pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:lustre-MDT0000-lwp-MDT0001_UUID:3352:1862456998119040:0@lo:38:kworker/u8:0.0 00000100:00100000:3.0:1776178707.298916:0:3352:0:(events.c:361:request_in_callback()) peer: 12345-0@lo (source: 12345-0@lo) 00000100:00100000:3.0:1776178707.298954:0:6255:0:(service.c:2152:ptlrpc_server_handle_req_in()) got req x1862456998118592 00000100:00100000:3.0:1776178707.298986:0:6255:0:(service.c:2152:ptlrpc_server_handle_req_in()) got req x1862456998118720 00000100:00100000:3.0:1776178707.298994:0:6255:0:(service.c:2152:ptlrpc_server_handle_req_in()) got req x1862456998119040 00000100:00100000:3.0:1776178707.299004:0:6255:0:(nrs_fifo.c:179:nrs_fifo_req_get()) NRS start fifo request from 12345-0@lo, seq: 7017 00000100:00100000:3.0:1776178707.299007:0:6255:0:(service.c:2324:ptlrpc_server_handle_request()) Handling RPC req@00000000f97fd38e pname:cluuid+ref:pid:xid:nid:opc:job mdt00_000:0+-99:3352:x1862456998118592:12345-0@lo:38:kworker/u8:0.0 00010000:02000000:3.0:1776178707.299020:0:6255:0:(ldlm_lib.c:1139:target_handle_connect()) lustre-MDT0000: Not available for connect from 0@lo (stopping) 00000100:00100000:3.0:1776178707.299056:0:6255:0:(service.c:2372:ptlrpc_server_handle_request()) Handled RPC req@00000000f97fd38e pname:cluuid+ref:pid:xid:nid:opc:job mdt00_000:0+-99:3352:x1862456998118592:12345-0@lo:38:kworker/u8:0.0 Request processed in 47us (251us total) trans 0 rc -19/-19 00000100:00100000:3.0:1776178707.299086:0:6255:0:(nrs_fifo.c:241:nrs_fifo_req_stop()) NRS stop fifo request from 12345-0@lo, seq: 7017 00000100:00100000:3.0:1776178707.299100:0:6255:0:(nrs_fifo.c:179:nrs_fifo_req_get()) NRS start fifo request from 12345-0@lo, seq: 7018 00000100:00100000:3.0:1776178707.299103:0:6255:0:(service.c:2324:ptlrpc_server_handle_request()) Handling RPC req@000000000bdf54a0 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_000:0+-99:3352:x1862456998118720:12345-0@lo:38:kworker/u8:0.0 00010000:02000000:3.0:1776178707.299121:0:6255:0:(ldlm_lib.c:1139:target_handle_connect()) lustre-MDT0000: Not available for connect from 0@lo (stopping) 00000100:00100000:3.0:1776178707.299143:0:6255:0:(service.c:2372:ptlrpc_server_handle_request()) Handled RPC req@000000000bdf54a0 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_000:0+-99:3352:x1862456998118720:12345-0@lo:38:kworker/u8:0.0 Request processed in 39us (298us total) trans 0 rc -19/-19 00000100:00100000:3.0:1776178707.299149:0:6255:0:(nrs_fifo.c:241:nrs_fifo_req_stop()) NRS stop fifo request from 12345-0@lo, seq: 7018 00000100:00100000:3.0:1776178707.299159:0:6255:0:(nrs_fifo.c:179:nrs_fifo_req_get()) NRS start fifo request from 12345-0@lo, seq: 7019 00000100:00100000:3.0:1776178707.299161:0:6255:0:(service.c:2324:ptlrpc_server_handle_request()) Handling RPC req@00000000c237605d pname:cluuid+ref:pid:xid:nid:opc:job mdt00_000:0+-99:3352:x1862456998119040:12345-0@lo:38:kworker/u8:0.0 00010000:02000000:3.0:1776178707.299166:0:6255:0:(ldlm_lib.c:1139:target_handle_connect()) lustre-MDT0000: Not available for connect from 0@lo (stopping) 00000100:00100000:3.0:1776178707.299184:0:6255:0:(service.c:2372:ptlrpc_server_handle_request()) Handled RPC req@00000000c237605d pname:cluuid+ref:pid:xid:nid:opc:job mdt00_000:0+-99:3352:x1862456998119040:12345-0@lo:38:kworker/u8:0.0 Request processed in 22us (268us total) trans 0 rc -19/-19 00000100:00100000:3.0:1776178707.299188:0:6255:0:(nrs_fifo.c:241:nrs_fifo_req_stop()) NRS stop fifo request from 12345-0@lo, seq: 7019 00000100:00080000:3.0:1776178707.299246:0:3352:0:(import.c:86:import_set_state_nolock()) 0000000047601a98 lustre-MDT0000_UUID: changing import state from CONNECTING to DISCONN 00000100:00080000:3.0:1776178707.299250:0:3352:0:(import.c:1429:ptlrpc_connect_interpret()) recovery of lustre-MDT0000_UUID on 192.168.204.103@tcp failed (-19) 00000100:00100000:3.0:1776178707.299254:0:3352:0:(client.c:2215:ptlrpc_check_set()) Completed RPC req@000000001ff6e514 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:lustre-MDT0000-lwp-OST0000_UUID:3352:1862456998118592:0@lo:38:kworker/u8:0.0 00000100:00080000:3.0:1776178707.299263:0:3352:0:(import.c:86:import_set_state_nolock()) 00000000dfc8db13 lustre-MDT0000_UUID: changing import state from CONNECTING to DISCONN 00000100:00080000:3.0:1776178707.299265:0:3352:0:(import.c:1429:ptlrpc_connect_interpret()) recovery of lustre-MDT0000_UUID on 192.168.204.103@tcp failed (-19) 00000100:00100000:3.0:1776178707.299267:0:3352:0:(client.c:2215:ptlrpc_check_set()) Completed RPC req@000000008a15a61d pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:lustre-MDT0000-lwp-OST0001_UUID:3352:1862456998118720:0@lo:38:kworker/u8:0.0 00000100:00080000:3.0:1776178707.299273:0:3352:0:(import.c:86:import_set_state_nolock()) 00000000a7c55b1f lustre-MDT0000_UUID: changing import state from CONNECTING to DISCONN 00000100:00080000:3.0:1776178707.299275:0:3352:0:(import.c:1429:ptlrpc_connect_interpret()) recovery of lustre-MDT0000_UUID on 192.168.204.103@tcp failed (-19) 00000100:00100000:3.0:1776178707.299277:0:3352:0:(client.c:2215:ptlrpc_check_set()) Completed RPC req@00000000dc38eb5b pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:lustre-MDT0000-lwp-MDT0001_UUID:3352:1862456998119040:0@lo:38:kworker/u8:0.0 00000100:00100000:2.0:1776178707.299972:0:15213:0:(service.c:2152:ptlrpc_server_handle_req_in()) got req x1862456998118848 00000100:00100000:2.0:1776178707.300005:0:15213:0:(nrs_fifo.c:179:nrs_fifo_req_get()) NRS start fifo request from 12345-0@lo, seq: 806 00000100:00100000:2.0:1776178707.300009:0:15213:0:(service.c:2324:ptlrpc_server_handle_request()) Handling RPC req@00000000e82039bb pname:cluuid+ref:pid:xid:nid:opc:job mdt_out00_003:0+-99:3352:x1862456998118848:12345-0@lo:38:kworker/u8:0.0 00010000:02000000:2.0:1776178707.300019:0:15213:0:(ldlm_lib.c:1139:target_handle_connect()) lustre-MDT0000: Not available for connect from 0@lo (stopping) 00000100:00100000:2.0:1776178707.300088:0:15213:0:(service.c:2372:ptlrpc_server_handle_request()) Handled RPC req@00000000e82039bb pname:cluuid+ref:pid:xid:nid:opc:job mdt_out00_003:0+-99:3352:x1862456998118848:12345-0@lo:38:kworker/u8:0.0 Request processed in 77us (1207us total) trans 0 rc -19/-19 00000100:00100000:2.0:1776178707.300094:0:15213:0:(nrs_fifo.c:241:nrs_fifo_req_stop()) NRS stop fifo request from 12345-0@lo, seq: 806 00000100:00080000:2.0:1776178707.300134:0:3352:0:(import.c:86:import_set_state_nolock()) 00000000ba4f1821 lustre-MDT0000_UUID: changing import state from CONNECTING to DISCONN 00000100:00080000:2.0:1776178707.300137:0:3352:0:(import.c:1429:ptlrpc_connect_interpret()) recovery of lustre-MDT0000_UUID on 192.168.204.103@tcp failed (-19) 00000100:00100000:2.0:1776178707.300140:0:3352:0:(client.c:2215:ptlrpc_check_set()) Completed RPC req@000000007cfd1c43 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:lustre-MDT0001-mdtlov_UUID:3352:1862456998118848:0@lo:38:kworker/u8:0.0 00000100:00100000:0.0:1776178707.300833:0:3354:0:(events.c:361:request_in_callback()) peer: 12345-0@lo (source: 12345-0@lo) 00000100:00100000:0.0:1776178707.300886:0:3356:0:(events.c:361:request_in_callback()) peer: 12345-0@lo (source: 12345-0@lo) 00000100:00100000:0.0:1776178707.300916:0:3356:0:(client.c:2215:ptlrpc_check_set()) Completed RPC req@00000000fa20ba2c pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_03:lustre-MDT0001-lwp-OST0000_UUID:3356:1862456998118656:0@lo:400:kworker/u8:0.0 00000100:00100000:0.0:1776178707.300933:0:3356:0:(client.c:2215:ptlrpc_check_set()) Completed RPC req@000000006c0c7de5 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_03:ac61f839-0e8e-4cac-9c59-08bcfbbad330:3356:1862456998118528:0@lo:400:kworker/u8:0.0 00000100:00100000:3.0:1776178707.301604:0:20759:0:(service.c:2152:ptlrpc_server_handle_req_in()) got req x1862456998118912 00000100:00100000:3.0:1776178707.301645:0:20759:0:(service.c:2152:ptlrpc_server_handle_req_in()) got req x1862456998118976 00000100:00100000:3.0:1776178707.301664:0:20759:0:(nrs_fifo.c:179:nrs_fifo_req_get()) NRS start fifo request from 12345-0@lo, seq: 905 00000100:00100000:3.0:1776178707.301668:0:20759:0:(service.c:2324:ptlrpc_server_handle_request()) Handling RPC req@0000000037d88937 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_010:lustre-MDT0001-mdtlov_UUID+5:3354:x1862456998118912:12345-0@lo:400:kworker/u8:0.0 00000100:00100000:3.0:1776178707.301725:0:20759:0:(service.c:2372:ptlrpc_server_handle_request()) Handled RPC req@0000000037d88937 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_010:lustre-MDT0001-mdtlov_UUID+5:3354:x1862456998118912:12345-0@lo:400:kworker/u8:0.0 Request processed in 56us (893us total) trans 0 rc 0/0 00000100:00100000:3.0:1776178707.301732:0:20759:0:(nrs_fifo.c:241:nrs_fifo_req_stop()) NRS stop fifo request from 12345-0@lo, seq: 905 00000100:00100000:3.0:1776178707.301748:0:20759:0:(nrs_fifo.c:179:nrs_fifo_req_get()) NRS start fifo request from 12345-0@lo, seq: 906 00000100:00100000:3.0:1776178707.301752:0:20759:0:(service.c:2324:ptlrpc_server_handle_request()) Handling RPC req@0000000045cb5035 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_010:lustre-MDT0001-mdtlov_UUID+5:3356:x1862456998118976:12345-0@lo:400:kworker/u8:0.0 00000100:00100000:3.0:1776178707.301784:0:20759:0:(service.c:2372:ptlrpc_server_handle_request()) Handled RPC req@0000000045cb5035 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_010:lustre-MDT0001-mdtlov_UUID+5:3356:x1862456998118976:12345-0@lo:400:kworker/u8:0.0 Request processed in 31us (898us total) trans 0 rc 0/0 00000100:00100000:3.0:1776178707.301790:0:20759:0:(nrs_fifo.c:241:nrs_fifo_req_stop()) NRS stop fifo request from 12345-0@lo, seq: 906 00000100:00100000:3.0:1776178707.301831:0:3356:0:(client.c:2215:ptlrpc_check_set()) Completed RPC req@00000000be147540 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_03:lustre-MDT0001-mdtlov_UUID:3356:1862456998118976:0@lo:400:kworker/u8:0.0 00000100:00100000:0.0:1776178707.301911:0:3354:0:(client.c:2215:ptlrpc_check_set()) Completed RPC req@00000000de8350d4 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_01:lustre-MDT0001-mdtlov_UUID:3354:1862456998118912:0@lo:400:kworker/u8:0.0 00010000:00010000:1.0:1776178707.938260:0:9225:0:(ldlm_lockd.c:2792:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:0.0:1776178708.832526:0:3355:0:(ptlrpcd.c:415:ptlrpcd_check()) transfer 1 async RPCs [3->2] 00000100:00100000:0.0:1776178708.832551:0:3355:0:(client.c:1733:ptlrpc_send_new_req()) Sending RPC req@0000000013888310 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_02:lustre-MDT0001-mdtlov_UUID:3355:1862456998119104:0@lo:13:osp-pre-1-1.0 00000100:00100000:0.0:1776178708.832621:0:3355:0:(events.c:361:request_in_callback()) peer: 12345-0@lo (source: 12345-0@lo) 00000100:00100000:2.0:1776178708.832870:0:8231:0:(service.c:2152:ptlrpc_server_handle_req_in()) got req x1862456998119104 00000100:00100000:2.0:1776178708.832918:0:8231:0:(nrs_fifo.c:179:nrs_fifo_req_get()) NRS start fifo request from 12345-0@lo, seq: 685 00000100:00100000:2.0:1776178708.832924:0:8231:0:(service.c:2324:ptlrpc_server_handle_request()) Handling RPC req@00000000de8350d4 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_create00:lustre-MDT0001-mdtlov_UUID+5:3355:x1862456998119104:12345-0@lo:13:osp-pre-1-1.0 00002000:00000024:2.0:1776178708.832954:0:8231:0:(ofd_obd.c:740:ofd_statfs()) blocks cached 229376 granted 491746368 pending 0 free 3923689472 avail 3692167168 00000100:00100000:2.0:1776178708.833015:0:8231:0:(service.c:2372:ptlrpc_server_handle_request()) Handled RPC req@00000000de8350d4 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_create00:lustre-MDT0001-mdtlov_UUID+5:3355:x1862456998119104:12345-0@lo:13:osp-pre-1-1.0 Request processed in 83us (388us total) trans 0 rc 0/0 00000100:00100000:2.0:1776178708.833022:0:8231:0:(nrs_fifo.c:241:nrs_fifo_req_stop()) NRS stop fifo request from 12345-0@lo, seq: 685 00000100:00100000:0.0:1776178708.833270:0:3355:0:(client.c:2215:ptlrpc_check_set()) Completed RPC req@0000000013888310 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_02:lustre-MDT0001-mdtlov_UUID:3355:1862456998119104:0@lo:13:osp-pre-1-1.0 00010000:00010000:2.0:1776178708.960330:0:6244:0:(ldlm_lockd.c:2792:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:1.0:1776178709.344531:0:3353:0:(client.c:1733:ptlrpc_send_new_req()) Sending RPC req@000000001ef63da8 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_00:lustre-MDT0001-mdtlov_UUID:3353:1862456998119168:0@lo:13:osp-pre-0-1.0 00000100:00100000:1.0:1776178709.344617:0:3353:0:(events.c:361:request_in_callback()) peer: 12345-0@lo (source: 12345-0@lo) 00000100:00100000:3.0:1776178709.344867:0:9554:0:(service.c:2152:ptlrpc_server_handle_req_in()) got req x1862456998119168 00000100:00100000:3.0:1776178709.344921:0:9554:0:(nrs_fifo.c:179:nrs_fifo_req_get()) NRS start fifo request from 12345-0@lo, seq: 686 00000100:00100000:3.0:1776178709.344927:0:9554:0:(service.c:2324:ptlrpc_server_handle_request()) Handling RPC req@000000009220b2b0 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_create00:lustre-MDT0001-mdtlov_UUID+5:3353:x1862456998119168:12345-0@lo:13:osp-pre-0-1.0 00002000:00000024:3.0:1776178709.344956:0:9554:0:(ofd_obd.c:740:ofd_statfs()) blocks cached 315392 granted 484883776 pending 0 free 3923689472 avail 3692167168 00000100:00100000:3.0:1776178709.345004:0:9554:0:(service.c:2372:ptlrpc_server_handle_request()) Handled RPC req@000000009220b2b0 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_create00:lustre-MDT0001-mdtlov_UUID+5:3353:x1862456998119168:12345-0@lo:13:osp-pre-0-1.0 Request processed in 76us (388us total) trans 0 rc 0/0 00000100:00100000:3.0:1776178709.345012:0:9554:0:(nrs_fifo.c:241:nrs_fifo_req_stop()) NRS stop fifo request from 12345-0@lo, seq: 686 00000100:00100000:1.0:1776178709.345160:0:3353:0:(client.c:2215:ptlrpc_check_set()) Completed RPC req@000000001ef63da8 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_00:lustre-MDT0001-mdtlov_UUID:3353:1862456998119168:0@lo:13:osp-pre-0-1.0 00010000:00010000:1.0:1776178709.987387:0:6245:0:(ldlm_lockd.c:2792:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:1.0:1776178710.822697:0:3347:0:(events.c:361:request_in_callback()) peer: 12345-192.168.204.3@tcp (source: 12345-192.168.204.3@tcp) 00000100:00100000:1.0:1776178710.822751:0:3347:0:(events.c:361:request_in_callback()) peer: 12345-192.168.204.3@tcp (source: 12345-192.168.204.3@tcp) 00000100:00100000:1.0:1776178710.822806:0:3347:0:(events.c:361:request_in_callback()) peer: 12345-192.168.204.3@tcp (source: 12345-192.168.204.3@tcp) 00000100:00100000:1.0:1776178710.822839:0:3347:0:(events.c:361:request_in_callback()) peer: 12345-192.168.204.3@tcp (source: 12345-192.168.204.3@tcp) 00000100:00100000:1.0:1776178710.822863:0:3347:0:(events.c:361:request_in_callback()) peer: 12345-192.168.204.3@tcp (source: 12345-192.168.204.3@tcp) 00000100:00100000:3.0:1776178710.823135:0:6257:0:(service.c:2152:ptlrpc_server_handle_req_in()) got req x1862456979140736 00000100:00100000:3.0:1776178710.823180:0:6257:0:(service.c:2152:ptlrpc_server_handle_req_in()) got req x1862456979140800 00000100:00100000:3.0:1776178710.823208:0:6257:0:(nrs_fifo.c:179:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.204.3@tcp, seq: 1225 00000100:00100000:3.0:1776178710.823214:0:6257:0:(service.c:2324:ptlrpc_server_handle_request()) Handling RPC req@000000007e1b5a24 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_002:b50963bd-8eba-4432-9a54-8888d00e3fa1+6:2271:x1862456979140800:12345-192.168.204.3@tcp:400:kworker/u8:2.0 00000100:00100000:3.0:1776178710.823283:0:6257:0:(service.c:2372:ptlrpc_server_handle_request()) Handled RPC req@000000007e1b5a24 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_002:b50963bd-8eba-4432-9a54-8888d00e3fa1+6:2271:x1862456979140800:12345-192.168.204.3@tcp:400:kworker/u8:2.0 Request processed in 68us (443us total) trans 0 rc 0/0 00000100:00100000:3.0:1776178710.823290:0:6257:0:(nrs_fifo.c:241:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.204.3@tcp, seq: 1225 00000100:00100000:3.0:1776178710.823305:0:6257:0:(nrs_fifo.c:179:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.204.3@tcp, seq: 7020 00000100:00100000:3.0:1776178710.823308:0:6257:0:(service.c:2324:ptlrpc_server_handle_request()) Handling RPC req@000000006e2391ef pname:cluuid+ref:pid:xid:nid:opc:job mdt00_002:0+-99:2269:x1862456979140736:12345-192.168.204.3@tcp:38:kworker/u8:2.0 00010000:02000000:3.0:1776178710.823321:0:6257:0:(ldlm_lib.c:1139:target_handle_connect()) lustre-MDT0000: Not available for connect from 192.168.204.3@tcp (stopping) 00000100:00100000:3.0:1776178710.823347:0:6257:0:(service.c:2372:ptlrpc_server_handle_request()) Handled RPC req@000000006e2391ef pname:cluuid+ref:pid:xid:nid:opc:job mdt00_002:0+-99:2269:x1862456979140736:12345-192.168.204.3@tcp:38:kworker/u8:2.0 Request processed in 37us (659us total) trans 0 rc -19/-19 00000100:00100000:3.0:1776178710.823353:0:6257:0:(nrs_fifo.c:241:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.204.3@tcp, seq: 7020 00000100:00100000:3.0:1776178710.823376:0:20755:0:(service.c:2152:ptlrpc_server_handle_req_in()) got req x1862456979140864 00000100:00100000:3.0:1776178710.823398:0:20755:0:(service.c:2152:ptlrpc_server_handle_req_in()) got req x1862456979140928 00000100:00100000:3.0:1776178710.823413:0:20755:0:(nrs_fifo.c:179:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.204.3@tcp, seq: 907 00000100:00100000:3.0:1776178710.823417:0:20755:0:(service.c:2324:ptlrpc_server_handle_request()) Handling RPC req@000000007cfde41a pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_006:b50963bd-8eba-4432-9a54-8888d00e3fa1+8:2271:x1862456979140864:12345-192.168.204.3@tcp:400:kworker/u8:2.0 00000100:00100000:3.0:1776178710.823444:0:20755:0:(service.c:2372:ptlrpc_server_handle_request()) Handled RPC req@000000007cfde41a pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_006:b50963bd-8eba-4432-9a54-8888d00e3fa1+8:2271:x1862456979140864:12345-192.168.204.3@tcp:400:kworker/u8:2.0 Request processed in 25us (638us total) trans 0 rc 0/0 00000100:00100000:3.0:1776178710.823449:0:20755:0:(nrs_fifo.c:241:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.204.3@tcp, seq: 907 00000100:00100000:3.0:1776178710.823460:0:20755:0:(nrs_fifo.c:179:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.204.3@tcp, seq: 908 00000100:00100000:3.0:1776178710.823464:0:20755:0:(service.c:2324:ptlrpc_server_handle_request()) Handling RPC req@00000000865e4f65 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_006:b50963bd-8eba-4432-9a54-8888d00e3fa1+9:2271:x1862456979140928:12345-192.168.204.3@tcp:400:kworker/u8:2.0 00000100:00100000:3.0:1776178710.823502:0:20755:0:(service.c:2372:ptlrpc_server_handle_request()) Handled RPC req@00000000865e4f65 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_006:b50963bd-8eba-4432-9a54-8888d00e3fa1+9:2271:x1862456979140928:12345-192.168.204.3@tcp:400:kworker/u8:2.0 Request processed in 37us (639us total) trans 0 rc 0/0 00000100:00100000:3.0:1776178710.823508:0:20755:0:(nrs_fifo.c:241:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.204.3@tcp, seq: 908 00000100:00100000:2.0:1776178710.826729:0:20137:0:(service.c:2152:ptlrpc_server_handle_req_in()) got req x1862456979140672 00000100:00100000:2.0:1776178710.826800:0:20137:0:(nrs_fifo.c:179:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.204.3@tcp, seq: 353 00000100:00100000:2.0:1776178710.826807:0:20137:0:(service.c:2324:ptlrpc_server_handle_request()) Handling RPC req@0000000048947fd8 pname:cluuid+ref:pid:xid:nid:opc:job ll_mgs_0000:5b5c66b1-1a2f-4651-8548-0c75b9a38a94+9:2271:x1862456979140672:12345-192.168.204.3@tcp:400:kworker/u8:2.0 00000100:00100000:2.0:1776178710.826891:0:20137:0:(service.c:2372:ptlrpc_server_handle_request()) Handled RPC req@0000000048947fd8 pname:cluuid+ref:pid:xid:nid:opc:job ll_mgs_0000:5b5c66b1-1a2f-4651-8548-0c75b9a38a94+9:2271:x1862456979140672:12345-192.168.204.3@tcp:400:kworker/u8:2.0 Request processed in 83us (4140us total) trans 0 rc 0/0 00000100:00100000:2.0:1776178710.826898:0:20137:0:(nrs_fifo.c:241:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.204.3@tcp, seq: 353 00010000:00010000:2.0:1776178711.009121:0:9224:0:(ldlm_lockd.c:2792:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00010000:00010000:1.0:1776178712.033581:0:9225:0:(ldlm_lockd.c:2792:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00080000:3.0:1776178712.416757:0:36:0:(pinger.c:251:ptlrpc_pinger_process_import()) lustre-MDT0000-lwp-OST0000_UUID->lustre-MDT0000_UUID: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00080000:3.0:1776178712.416765:0:36:0:(recover.c:57:ptlrpc_initiate_recovery()) lustre-MDT0000_UUID: starting recovery 00000100:00080000:3.0:1776178712.416768:0:36:0:(import.c:86:import_set_state_nolock()) 0000000047601a98 lustre-MDT0000_UUID: changing import state from DISCONN to CONNECTING 00000100:00080000:3.0:1776178712.416776:0:36:0:(import.c:534:import_select_connection()) lustre-MDT0000-lwp-OST0000: connect to NID 0@lo last attempt 1414 00000100:00080000:3.0:1776178712.416780:0:36:0:(import.c:581:import_select_connection()) lustre-MDT0000-lwp-OST0000: tried all connections, increasing latency to 21s 00000100:00080000:3.0:1776178712.416785:0:36:0:(import.c:615:import_select_connection()) lustre-MDT0000-lwp-OST0000: import 0000000047601a98 using connection 192.168.204.103@tcp/0@lo 00000100:00100000:3.0:1776178712.416807:0:36:0:(import.c:807:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5) req@0000000057946847 x1862456998119296/t0(0) o38->lustre-MDT0000-lwp-OST0000@0@lo:12/10 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' 00000100:00080000:3.0:1776178712.416845:0:36:0:(pinger.c:251:ptlrpc_pinger_process_import()) lustre-MDT0000-lwp-OST0001_UUID->lustre-MDT0000_UUID: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00080000:3.0:1776178712.416849:0:36:0:(recover.c:57:ptlrpc_initiate_recovery()) lustre-MDT0000_UUID: starting recovery 00000100:00080000:3.0:1776178712.416850:0:36:0:(import.c:86:import_set_state_nolock()) 00000000dfc8db13 lustre-MDT0000_UUID: changing import state from DISCONN to CONNECTING 00000100:00080000:3.0:1776178712.416853:0:36:0:(import.c:534:import_select_connection()) lustre-MDT0000-lwp-OST0001: connect to NID 0@lo last attempt 1414 00000100:00080000:3.0:1776178712.416855:0:36:0:(import.c:581:import_select_connection()) lustre-MDT0000-lwp-OST0001: tried all connections, increasing latency to 21s 00000100:00080000:3.0:1776178712.416858:0:36:0:(import.c:615:import_select_connection()) lustre-MDT0000-lwp-OST0001: import 00000000dfc8db13 using connection 192.168.204.103@tcp/0@lo 00000100:00100000:3.0:1776178712.416870:0:36:0:(import.c:807:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5) req@000000004c44fad7 x1862456998119424/t0(0) o38->lustre-MDT0000-lwp-OST0001@0@lo:12/10 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' 00000100:00080000:3.0:1776178712.416883:0:36:0:(pinger.c:251:ptlrpc_pinger_process_import()) lustre-MDT0001-mdtlov_UUID->lustre-MDT0000_UUID: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00080000:3.0:1776178712.416886:0:36:0:(recover.c:57:ptlrpc_initiate_recovery()) lustre-MDT0000_UUID: starting recovery 00000100:00080000:3.0:1776178712.416887:0:36:0:(import.c:86:import_set_state_nolock()) 00000000ba4f1821 lustre-MDT0000_UUID: changing import state from DISCONN to CONNECTING 00000100:00080000:3.0:1776178712.416890:0:36:0:(import.c:534:import_select_connection()) lustre-MDT0000-osp-MDT0001: connect to NID 0@lo last attempt 1414 00000100:00080000:3.0:1776178712.416892:0:36:0:(import.c:581:import_select_connection()) lustre-MDT0000-osp-MDT0001: tried all connections, increasing latency to 21s 00000100:00080000:3.0:1776178712.416894:0:36:0:(import.c:615:import_select_connection()) lustre-MDT0000-osp-MDT0001: import 00000000ba4f1821 using connection 192.168.204.103@tcp/0@lo 00000100:00100000:3.0:1776178712.416903:0:36:0:(import.c:807:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5) req@000000000cde9837 x1862456998119552/t0(0) o38->lustre-MDT0000-osp-MDT0001@0@lo:24/4 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' 00000100:00080000:3.0:1776178712.416919:0:36:0:(pinger.c:251:ptlrpc_pinger_process_import()) lustre-MDT0000-lwp-MDT0001_UUID->lustre-MDT0000_UUID: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00080000:3.0:1776178712.416922:0:36:0:(recover.c:57:ptlrpc_initiate_recovery()) lustre-MDT0000_UUID: starting recovery 00000100:00080000:3.0:1776178712.416923:0:36:0:(import.c:86:import_set_state_nolock()) 00000000a7c55b1f lustre-MDT0000_UUID: changing import state from DISCONN to CONNECTING 00000100:00080000:3.0:1776178712.416926:0:36:0:(import.c:534:import_select_connection()) lustre-MDT0000-lwp-MDT0001: connect to NID 0@lo last attempt 1414 00000100:00080000:3.0:1776178712.416928:0:36:0:(import.c:581:import_select_connection()) lustre-MDT0000-lwp-MDT0001: tried all connections, increasing latency to 21s 00000100:00080000:3.0:1776178712.416930:0:36:0:(import.c:615:import_select_connection()) lustre-MDT0000-lwp-MDT0001: import 00000000a7c55b1f using connection 192.168.204.103@tcp/0@lo 00000100:00100000:3.0:1776178712.416939:0:36:0:(import.c:807:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5) req@000000006165f052 x1862456998119744/t0(0) o38->lustre-MDT0000-lwp-MDT0001@0@lo:12/10 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' 00000100:00100000:3.0:1776178712.416967:0:3356:0:(client.c:1733:ptlrpc_send_new_req()) Sending RPC req@0000000053c46e8d pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_03:lustre-MDT0001-lwp-OST0001_UUID:3356:1862456998119488:0@lo:400:kworker/u8:1.0 00000100:00100000:3.0:1776178712.417044:0:3356:0:(events.c:361:request_in_callback()) peer: 12345-0@lo (source: 12345-0@lo) 00000100:00100000:3.0:1776178712.417113:0:3356:0:(ptlrpcd.c:415:ptlrpcd_check()) transfer 1 async RPCs [2->3] 00000100:00100000:3.0:1776178712.417118:0:3356:0:(client.c:1733:ptlrpc_send_new_req()) Sending RPC req@00000000196ec025 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_03:lustre-MDT0001-lwp-OST0000_UUID:3356:1862456998119360:0@lo:400:kworker/u8:1.0 00000100:00100000:3.0:1776178712.417141:0:3356:0:(events.c:361:request_in_callback()) peer: 12345-0@lo (source: 12345-0@lo) 00000100:00100000:3.0:1776178712.417173:0:11281:0:(service.c:2152:ptlrpc_server_handle_req_in()) got req x1862456998119488 00000100:00100000:3.0:1776178712.417227:0:11281:0:(service.c:2152:ptlrpc_server_handle_req_in()) got req x1862456998119360 00000100:00100000:3.0:1776178712.417247:0:11281:0:(nrs_fifo.c:179:nrs_fifo_req_get()) NRS start fifo request from 12345-0@lo, seq: 1226 00000100:00100000:3.0:1776178712.417252:0:11281:0:(service.c:2324:ptlrpc_server_handle_request()) Handling RPC req@0000000016c7a5ff pname:cluuid+ref:pid:xid:nid:opc:job mdt00_004:lustre-MDT0001-lwp-OST0001_UUID+5:3356:x1862456998119488:12345-0@lo:400:kworker/u8:1.0 00000100:00100000:3.0:1776178712.417296:0:11281:0:(service.c:2372:ptlrpc_server_handle_request()) Handled RPC req@0000000016c7a5ff pname:cluuid+ref:pid:xid:nid:opc:job mdt00_004:lustre-MDT0001-lwp-OST0001_UUID+5:3356:x1862456998119488:12345-0@lo:400:kworker/u8:1.0 Request processed in 43us (254us total) trans 0 rc 0/0 00000100:00100000:3.0:1776178712.417303:0:11281:0:(nrs_fifo.c:241:nrs_fifo_req_stop()) NRS stop fifo request from 12345-0@lo, seq: 1226 00000100:00100000:3.0:1776178712.417319:0:11281:0:(nrs_fifo.c:179:nrs_fifo_req_get()) NRS start fifo request from 12345-0@lo, seq: 1227 00000100:00100000:3.0:1776178712.417322:0:11281:0:(service.c:2324:ptlrpc_server_handle_request()) Handling RPC req@00000000b30b9cdd pname:cluuid+ref:pid:xid:nid:opc:job mdt00_004:lustre-MDT0001-lwp-OST0000_UUID+5:3356:x1862456998119360:12345-0@lo:400:kworker/u8:1.0 00000100:00100000:3.0:1776178712.417345:0:11281:0:(service.c:2372:ptlrpc_server_handle_request()) Handled RPC req@00000000b30b9cdd pname:cluuid+ref:pid:xid:nid:opc:job mdt00_004:lustre-MDT0001-lwp-OST0000_UUID+5:3356:x1862456998119360:12345-0@lo:400:kworker/u8:1.0 Request processed in 22us (204us total) trans 0 rc 0/0 00000100:00100000:3.0:1776178712.417350:0:11281:0:(nrs_fifo.c:241:nrs_fifo_req_stop()) NRS stop fifo request from 12345-0@lo, seq: 1227 00000100:00100000:3.0:1776178712.417396:0:3356:0:(client.c:2215:ptlrpc_check_set()) Completed RPC req@00000000196ec025 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_03:lustre-MDT0001-lwp-OST0000_UUID:3356:1862456998119360:0@lo:400:kworker/u8:1.0 00000100:00100000:3.0:1776178712.417405:0:3356:0:(client.c:2215:ptlrpc_check_set()) Completed RPC req@0000000053c46e8d pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_03:lustre-MDT0001-lwp-OST0001_UUID:3356:1862456998119488:0@lo:400:kworker/u8:1.0 00000100:00100000:2.0:1776178712.417647:0:3352:0:(client.c:1733:ptlrpc_send_new_req()) Sending RPC req@0000000057946847 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:lustre-MDT0000-lwp-OST0000_UUID:3352:1862456998119296:0@lo:38:kworker/u8:1.0 00000100:00100000:2.0:1776178712.417686:0:3352:0:(events.c:361:request_in_callback()) peer: 12345-0@lo (source: 12345-0@lo) 00000100:00100000:2.0:1776178712.417706:0:3352:0:(client.c:1733:ptlrpc_send_new_req()) Sending RPC req@000000004c44fad7 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:lustre-MDT0000-lwp-OST0001_UUID:3352:1862456998119424:0@lo:38:kworker/u8:1.0 00000100:00100000:2.0:1776178712.417728:0:3352:0:(events.c:361:request_in_callback()) peer: 12345-0@lo (source: 12345-0@lo) 00000100:00100000:2.0:1776178712.417738:0:3352:0:(client.c:1733:ptlrpc_send_new_req()) Sending RPC req@000000000cde9837 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:lustre-MDT0001-mdtlov_UUID:3352:1862456998119552:0@lo:38:kworker/u8:1.0 00000100:00100000:2.0:1776178712.417760:0:3352:0:(events.c:361:request_in_callback()) peer: 12345-0@lo (source: 12345-0@lo) 00000100:00100000:2.0:1776178712.417773:0:3352:0:(client.c:1733:ptlrpc_send_new_req()) Sending RPC req@000000006165f052 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:lustre-MDT0000-lwp-MDT0001_UUID:3352:1862456998119744:0@lo:38:kworker/u8:1.0 00000100:00100000:2.0:1776178712.417793:0:3352:0:(events.c:361:request_in_callback()) peer: 12345-0@lo (source: 12345-0@lo) 00000100:00100000:1.0:1776178712.417921:0:3353:0:(client.c:1733:ptlrpc_send_new_req()) Sending RPC req@00000000f4172b4c pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_00:lustre-MDT0001-mdtlov_UUID:3353:1862456998119616:0@lo:400:kworker/u8:1.0 00000100:00100000:1.0:1776178712.417952:0:3353:0:(events.c:361:request_in_callback()) peer: 12345-0@lo (source: 12345-0@lo) 00000100:00100000:1.0:1776178712.417980:0:3353:0:(ptlrpcd.c:415:ptlrpcd_check()) transfer 2 async RPCs [1->0] 00000100:00100000:1.0:1776178712.417986:0:3353:0:(client.c:1733:ptlrpc_send_new_req()) Sending RPC req@00000000c37c187e pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_00:ac61f839-0e8e-4cac-9c59-08bcfbbad330:3353:1862456998119232:0@lo:400:kworker/u8:1.0 00000100:00100000:1.0:1776178712.418019:0:3353:0:(events.c:361:request_in_callback()) peer: 12345-0@lo (source: 12345-0@lo) 00000100:00100000:1.0:1776178712.418033:0:3353:0:(client.c:1733:ptlrpc_send_new_req()) Sending RPC req@00000000ba9bc15d pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_00:lustre-MDT0001-mdtlov_UUID:3353:1862456998119680:0@lo:400:kworker/u8:1.0 00000100:00100000:1.0:1776178712.418053:0:3353:0:(events.c:361:request_in_callback()) peer: 12345-0@lo (source: 12345-0@lo) 00000100:00100000:2.0:1776178712.418388:0:20762:0:(service.c:2152:ptlrpc_server_handle_req_in()) got req x1862456998119616 00000100:00100000:2.0:1776178712.418427:0:20762:0:(service.c:2152:ptlrpc_server_handle_req_in()) got req x1862456998119680 00000100:00100000:2.0:1776178712.418460:0:20762:0:(nrs_fifo.c:179:nrs_fifo_req_get()) NRS start fifo request from 12345-0@lo, seq: 909 00000100:00100000:2.0:1776178712.418464:0:20762:0:(service.c:2324:ptlrpc_server_handle_request()) Handling RPC req@0000000071333ecf pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_013:lustre-MDT0001-mdtlov_UUID+5:3353:x1862456998119616:12345-0@lo:400:kworker/u8:1.0 00000100:00100000:2.0:1776178712.418507:0:20762:0:(service.c:2372:ptlrpc_server_handle_request()) Handled RPC req@0000000071333ecf pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_013:lustre-MDT0001-mdtlov_UUID+5:3353:x1862456998119616:12345-0@lo:400:kworker/u8:1.0 Request processed in 42us (555us total) trans 0 rc 0/0 00000100:00100000:2.0:1776178712.418515:0:20762:0:(nrs_fifo.c:241:nrs_fifo_req_stop()) NRS stop fifo request from 12345-0@lo, seq: 909 00000100:00100000:2.0:1776178712.418529:0:20762:0:(nrs_fifo.c:179:nrs_fifo_req_get()) NRS start fifo request from 12345-0@lo, seq: 910 00000100:00100000:2.0:1776178712.418532:0:20762:0:(service.c:2324:ptlrpc_server_handle_request()) Handling RPC req@00000000597c3414 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_013:lustre-MDT0001-mdtlov_UUID+5:3353:x1862456998119680:12345-0@lo:400:kworker/u8:1.0 00000100:00100000:2.0:1776178712.418556:0:20762:0:(service.c:2372:ptlrpc_server_handle_request()) Handled RPC req@00000000597c3414 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_013:lustre-MDT0001-mdtlov_UUID+5:3353:x1862456998119680:12345-0@lo:400:kworker/u8:1.0 Request processed in 23us (503us total) trans 0 rc 0/0 00000100:00100000:2.0:1776178712.418561:0:20762:0:(nrs_fifo.c:241:nrs_fifo_req_stop()) NRS stop fifo request from 12345-0@lo, seq: 910 00000100:00100000:2.0:1776178712.418589:0:3353:0:(client.c:2215:ptlrpc_check_set()) Completed RPC req@00000000ba9bc15d pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_00:lustre-MDT0001-mdtlov_UUID:3353:1862456998119680:0@lo:400:kworker/u8:1.0 00000100:00100000:2.0:1776178712.418597:0:3353:0:(client.c:2215:ptlrpc_check_set()) Completed RPC req@00000000f4172b4c pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_00:lustre-MDT0001-mdtlov_UUID:3353:1862456998119616:0@lo:400:kworker/u8:1.0 00000100:00100000:3.0:1776178712.419558:0:14128:0:(service.c:2152:ptlrpc_server_handle_req_in()) got req x1862456998119552 00000100:00100000:3.0:1776178712.419604:0:14128:0:(nrs_fifo.c:179:nrs_fifo_req_get()) NRS start fifo request from 12345-0@lo, seq: 807 00000100:00100000:3.0:1776178712.419609:0:14128:0:(service.c:2324:ptlrpc_server_handle_request()) Handling RPC req@00000000fdb309c4 pname:cluuid+ref:pid:xid:nid:opc:job mdt_out00_002:0+-99:3352:x1862456998119552:12345-0@lo:38:kworker/u8:1.0 00010000:02000000:3.0:1776178712.419623:0:14128:0:(ldlm_lib.c:1139:target_handle_connect()) lustre-MDT0000: Not available for connect from 0@lo (stopping) 00000100:00100000:3.0:1776178712.420141:0:14128:0:(service.c:2372:ptlrpc_server_handle_request()) Handled RPC req@00000000fdb309c4 pname:cluuid+ref:pid:xid:nid:opc:job mdt_out00_002:0+-99:3352:x1862456998119552:12345-0@lo:38:kworker/u8:1.0 Request processed in 529us (2378us total) trans 0 rc -19/-19 00000100:00100000:3.0:1776178712.420148:0:14128:0:(nrs_fifo.c:241:nrs_fifo_req_stop()) NRS stop fifo request from 12345-0@lo, seq: 807 00000100:00100000:3.0:1776178712.420179:0:6256:0:(service.c:2152:ptlrpc_server_handle_req_in()) got req x1862456998119296 00000100:00100000:3.0:1776178712.420208:0:6256:0:(service.c:2152:ptlrpc_server_handle_req_in()) got req x1862456998119424 00000100:00100000:3.0:1776178712.420238:0:6256:0:(service.c:2152:ptlrpc_server_handle_req_in()) got req x1862456998119744 00000100:00100000:3.0:1776178712.420250:0:6256:0:(nrs_fifo.c:179:nrs_fifo_req_get()) NRS start fifo request from 12345-0@lo, seq: 7021 00000100:00100000:3.0:1776178712.420253:0:6256:0:(service.c:2324:ptlrpc_server_handle_request()) Handling RPC req@000000001ef63da8 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_001:0+-99:3352:x1862456998119296:12345-0@lo:38:kworker/u8:1.0 00010000:02000000:3.0:1776178712.420262:0:6256:0:(ldlm_lib.c:1139:target_handle_connect()) lustre-MDT0000: Not available for connect from 0@lo (stopping) 00000100:00100000:3.0:1776178712.420287:0:6256:0:(service.c:2372:ptlrpc_server_handle_request()) Handled RPC req@000000001ef63da8 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_001:0+-99:3352:x1862456998119296:12345-0@lo:38:kworker/u8:1.0 Request processed in 33us (2601us total) trans 0 rc -19/-19 00000100:00100000:3.0:1776178712.420297:0:6256:0:(nrs_fifo.c:241:nrs_fifo_req_stop()) NRS stop fifo request from 12345-0@lo, seq: 7021 00000100:00100000:3.0:1776178712.420309:0:6256:0:(nrs_fifo.c:179:nrs_fifo_req_get()) NRS start fifo request from 12345-0@lo, seq: 7022 00000100:00100000:3.0:1776178712.420312:0:6256:0:(service.c:2324:ptlrpc_server_handle_request()) Handling RPC req@0000000013888310 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_001:0+-99:3352:x1862456998119424:12345-0@lo:38:kworker/u8:1.0 00010000:02000000:3.0:1776178712.420318:0:6256:0:(ldlm_lib.c:1139:target_handle_connect()) lustre-MDT0000: Not available for connect from 0@lo (stopping) 00000100:00100000:3.0:1776178712.420340:0:6256:0:(service.c:2372:ptlrpc_server_handle_request()) Handled RPC req@0000000013888310 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_001:0+-99:3352:x1862456998119424:12345-0@lo:38:kworker/u8:1.0 Request processed in 28us (2612us total) trans 0 rc -19/-19 00000100:00100000:3.0:1776178712.420345:0:6256:0:(nrs_fifo.c:241:nrs_fifo_req_stop()) NRS stop fifo request from 12345-0@lo, seq: 7022 00000100:00100000:3.0:1776178712.420356:0:6256:0:(nrs_fifo.c:179:nrs_fifo_req_get()) NRS start fifo request from 12345-0@lo, seq: 7023 00000100:00100000:3.0:1776178712.420360:0:6256:0:(service.c:2324:ptlrpc_server_handle_request()) Handling RPC req@000000001f88d899 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_001:0+-99:3352:x1862456998119744:12345-0@lo:38:kworker/u8:1.0 00010000:02000000:3.0:1776178712.420365:0:6256:0:(ldlm_lib.c:1139:target_handle_connect()) lustre-MDT0000: Not available for connect from 0@lo (stopping) 00000100:00100000:3.0:1776178712.420383:0:6256:0:(service.c:2372:ptlrpc_server_handle_request()) Handled RPC req@000000001f88d899 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_001:0+-99:3352:x1862456998119744:12345-0@lo:38:kworker/u8:1.0 Request processed in 23us (2591us total) trans 0 rc -19/-19 00000100:00100000:3.0:1776178712.420387:0:6256:0:(nrs_fifo.c:241:nrs_fifo_req_stop()) NRS stop fifo request from 12345-0@lo, seq: 7023 00000100:00100000:3.0:1776178712.420415:0:20140:0:(service.c:2152:ptlrpc_server_handle_req_in()) got req x1862456998119232 00000100:00100000:3.0:1776178712.420439:0:20140:0:(nrs_fifo.c:179:nrs_fifo_req_get()) NRS start fifo request from 12345-0@lo, seq: 354 00000100:00100000:3.0:1776178712.420442:0:20140:0:(service.c:2324:ptlrpc_server_handle_request()) Handling RPC req@00000000ed9d6990 pname:cluuid+ref:pid:xid:nid:opc:job ll_mgs_0002:ac61f839-0e8e-4cac-9c59-08bcfbbad330+18:3353:x1862456998119232:12345-0@lo:400:kworker/u8:1.0 00000100:00100000:3.0:1776178712.420954:0:20140:0:(service.c:2372:ptlrpc_server_handle_request()) Handled RPC req@00000000ed9d6990 pname:cluuid+ref:pid:xid:nid:opc:job ll_mgs_0002:ac61f839-0e8e-4cac-9c59-08bcfbbad330+18:3353:x1862456998119232:12345-0@lo:400:kworker/u8:1.0 Request processed in 509us (2942us total) trans 0 rc 0/0 00000100:00100000:3.0:1776178712.420962:0:20140:0:(nrs_fifo.c:241:nrs_fifo_req_stop()) NRS stop fifo request from 12345-0@lo, seq: 354 00000100:00080000:0.0:1776178712.421088:0:3352:0:(import.c:86:import_set_state_nolock()) 0000000047601a98 lustre-MDT0000_UUID: changing import state from CONNECTING to DISCONN 00000100:00080000:0.0:1776178712.421093:0:3352:0:(import.c:1429:ptlrpc_connect_interpret()) recovery of lustre-MDT0000_UUID on 192.168.204.103@tcp failed (-19) 00000100:00100000:0.0:1776178712.421098:0:3352:0:(client.c:2215:ptlrpc_check_set()) Completed RPC req@0000000057946847 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:lustre-MDT0000-lwp-OST0000_UUID:3352:1862456998119296:0@lo:38:kworker/u8:1.0 00000100:00080000:0.0:1776178712.421110:0:3352:0:(import.c:86:import_set_state_nolock()) 00000000dfc8db13 lustre-MDT0000_UUID: changing import state from CONNECTING to DISCONN 00000100:00080000:0.0:1776178712.421112:0:3352:0:(import.c:1429:ptlrpc_connect_interpret()) recovery of lustre-MDT0000_UUID on 192.168.204.103@tcp failed (-19) 00000100:00100000:0.0:1776178712.421115:0:3352:0:(client.c:2215:ptlrpc_check_set()) Completed RPC req@000000004c44fad7 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:lustre-MDT0000-lwp-OST0001_UUID:3352:1862456998119424:0@lo:38:kworker/u8:1.0 00000100:00080000:0.0:1776178712.421122:0:3352:0:(import.c:86:import_set_state_nolock()) 00000000ba4f1821 lustre-MDT0000_UUID: changing import state from CONNECTING to DISCONN 00000100:00080000:0.0:1776178712.421125:0:3352:0:(import.c:1429:ptlrpc_connect_interpret()) recovery of lustre-MDT0000_UUID on 192.168.204.103@tcp failed (-19) 00000100:00100000:0.0:1776178712.421127:0:3352:0:(client.c:2215:ptlrpc_check_set()) Completed RPC req@000000000cde9837 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:lustre-MDT0001-mdtlov_UUID:3352:1862456998119552:0@lo:38:kworker/u8:1.0 00000100:00080000:0.0:1776178712.421136:0:3352:0:(import.c:86:import_set_state_nolock()) 00000000a7c55b1f lustre-MDT0000_UUID: changing import state from CONNECTING to DISCONN 00000100:00080000:0.0:1776178712.421138:0:3352:0:(import.c:1429:ptlrpc_connect_interpret()) recovery of lustre-MDT0000_UUID on 192.168.204.103@tcp failed (-19) 00000100:00100000:0.0:1776178712.421141:0:3352:0:(client.c:2215:ptlrpc_check_set()) Completed RPC req@000000006165f052 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:lustre-MDT0000-lwp-MDT0001_UUID:3352:1862456998119744:0@lo:38:kworker/u8:1.0 00000100:00100000:1.0:1776178712.421427:0:3353:0:(client.c:2215:ptlrpc_check_set()) Completed RPC req@00000000c37c187e pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_00:ac61f839-0e8e-4cac-9c59-08bcfbbad330:3353:1862456998119232:0@lo:400:kworker/u8:1.0 00010000:00010000:2.0:1776178713.056455:0:6244:0:(ldlm_lockd.c:2792:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:0.0:1776178713.953534:0:3355:0:(client.c:1733:ptlrpc_send_new_req()) Sending RPC req@000000004547bc6e pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_02:lustre-MDT0001-mdtlov_UUID:3355:1862456998119808:0@lo:13:osp-pre-1-1.0 00000100:00100000:0.0:1776178713.953610:0:3355:0:(events.c:361:request_in_callback()) peer: 12345-0@lo (source: 12345-0@lo) 00000100:00100000:2.0:1776178713.953973:0:8231:0:(service.c:2152:ptlrpc_server_handle_req_in()) got req x1862456998119808 00000100:00100000:2.0:1776178713.954037:0:8231:0:(nrs_fifo.c:179:nrs_fifo_req_get()) NRS start fifo request from 12345-0@lo, seq: 687 00000100:00100000:2.0:1776178713.954044:0:8231:0:(service.c:2324:ptlrpc_server_handle_request()) Handling RPC req@00000000ad99db59 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_create00:lustre-MDT0001-mdtlov_UUID+5:3355:x1862456998119808:12345-0@lo:13:osp-pre-1-1.0 00002000:00000024:2.0:1776178713.954103:0:8231:0:(ofd_obd.c:740:ofd_statfs()) blocks cached 229376 granted 491746368 pending 0 free 3923689472 avail 3692167168 00000100:00100000:2.0:1776178713.954158:0:8231:0:(service.c:2372:ptlrpc_server_handle_request()) Handled RPC req@00000000ad99db59 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_create00:lustre-MDT0001-mdtlov_UUID+5:3355:x1862456998119808:12345-0@lo:13:osp-pre-1-1.0 Request processed in 113us (549us total) trans 0 rc 0/0 00000100:00100000:2.0:1776178713.954165:0:8231:0:(nrs_fifo.c:241:nrs_fifo_req_stop()) NRS stop fifo request from 12345-0@lo, seq: 687 00000100:00100000:1.0:1776178713.954249:0:3355:0:(client.c:2215:ptlrpc_check_set()) Completed RPC req@000000004547bc6e pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_02:lustre-MDT0001-mdtlov_UUID:3355:1862456998119808:0@lo:13:osp-pre-1-1.0 00010000:00010000:1.0:1776178714.080327:0:6245:0:(ldlm_lockd.c:2792:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:1.0:1776178714.468492:0:3355:0:(ptlrpcd.c:415:ptlrpcd_check()) transfer 1 async RPCs [3->2] 00000100:00100000:1.0:1776178714.468527:0:3355:0:(client.c:1733:ptlrpc_send_new_req()) Sending RPC req@00000000ad228ff6 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_02:lustre-MDT0001-mdtlov_UUID:3355:1862456998119872:0@lo:13:osp-pre-0-1.0 00000100:00100000:1.0:1776178714.468627:0:3355:0:(events.c:361:request_in_callback()) peer: 12345-0@lo (source: 12345-0@lo) 00000100:00100000:2.0:1776178714.468773:0:9554:0:(service.c:2152:ptlrpc_server_handle_req_in()) got req x1862456998119872 00000100:00100000:2.0:1776178714.468830:0:9554:0:(nrs_fifo.c:179:nrs_fifo_req_get()) NRS start fifo request from 12345-0@lo, seq: 688 00000100:00100000:2.0:1776178714.468836:0:9554:0:(service.c:2324:ptlrpc_server_handle_request()) Handling RPC req@00000000500c0ae6 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_create00:lustre-MDT0001-mdtlov_UUID+5:3355:x1862456998119872:12345-0@lo:13:osp-pre-0-1.0 00002000:00000024:2.0:1776178714.468868:0:9554:0:(ofd_obd.c:740:ofd_statfs()) blocks cached 315392 granted 484883776 pending 0 free 3923689472 avail 3692167168 00000100:00100000:2.0:1776178714.468918:0:9554:0:(service.c:2372:ptlrpc_server_handle_request()) Handled RPC req@00000000500c0ae6 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_create00:lustre-MDT0001-mdtlov_UUID+5:3355:x1862456998119872:12345-0@lo:13:osp-pre-0-1.0 Request processed in 80us (292us total) trans 0 rc 0/0 00000100:00100000:2.0:1776178714.468927:0:9554:0:(nrs_fifo.c:241:nrs_fifo_req_stop()) NRS stop fifo request from 12345-0@lo, seq: 688 00000100:00100000:2.0:1776178714.468990:0:3355:0:(client.c:2215:ptlrpc_check_set()) Completed RPC req@00000000ad228ff6 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_02:lustre-MDT0001-mdtlov_UUID:3355:1862456998119872:0@lo:13:osp-pre-0-1.0 00010000:00010000:2.0:1776178715.104452:0:9224:0:(ldlm_lockd.c:2792:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:1.0:1776178715.941671:0:3347:0:(events.c:361:request_in_callback()) peer: 12345-192.168.204.3@tcp (source: 12345-192.168.204.3@tcp) 00000100:00100000:1.0:1776178715.941731:0:3347:0:(events.c:361:request_in_callback()) peer: 12345-192.168.204.3@tcp (source: 12345-192.168.204.3@tcp) 00000100:00100000:1.0:1776178715.941774:0:3347:0:(events.c:361:request_in_callback()) peer: 12345-192.168.204.3@tcp (source: 12345-192.168.204.3@tcp) 00000100:00100000:1.0:1776178715.941811:0:3347:0:(events.c:361:request_in_callback()) peer: 12345-192.168.204.3@tcp (source: 12345-192.168.204.3@tcp) 00000100:00100000:2.0:1776178715.941854:0:20137:0:(service.c:2152:ptlrpc_server_handle_req_in()) got req x1862456979140992 00000100:00100000:2.0:1776178715.941912:0:20137:0:(nrs_fifo.c:179:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.204.3@tcp, seq: 355 00000100:00100000:2.0:1776178715.941920:0:20137:0:(service.c:2324:ptlrpc_server_handle_request()) Handling RPC req@00000000b4107e97 pname:cluuid+ref:pid:xid:nid:opc:job ll_mgs_0000:5b5c66b1-1a2f-4651-8548-0c75b9a38a94+9:2273:x1862456979140992:12345-192.168.204.3@tcp:400:kworker/u8:2.0 00000100:00100000:2.0:1776178715.943476:0:20137:0:(service.c:2372:ptlrpc_server_handle_request()) Handled RPC req@00000000b4107e97 pname:cluuid+ref:pid:xid:nid:opc:job ll_mgs_0000:5b5c66b1-1a2f-4651-8548-0c75b9a38a94+9:2273:x1862456979140992:12345-192.168.204.3@tcp:400:kworker/u8:2.0 Request processed in 1548us (1693us total) trans 0 rc 0/0 00000100:00100000:2.0:1776178715.943490:0:20137:0:(nrs_fifo.c:241:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.204.3@tcp, seq: 355 00000100:00100000:2.0:1776178715.943573:0:6257:0:(service.c:2152:ptlrpc_server_handle_req_in()) got req x1862456979141120 00000100:00100000:2.0:1776178715.943612:0:6257:0:(nrs_fifo.c:179:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.204.3@tcp, seq: 1228 00000100:00100000:2.0:1776178715.943617:0:6257:0:(service.c:2324:ptlrpc_server_handle_request()) Handling RPC req@000000001c8035b1 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_002:b50963bd-8eba-4432-9a54-8888d00e3fa1+6:2273:x1862456979141120:12345-192.168.204.3@tcp:400:kworker/u8:2.0 00000100:00100000:2.0:1776178715.943669:0:6257:0:(service.c:2372:ptlrpc_server_handle_request()) Handled RPC req@000000001c8035b1 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_002:b50963bd-8eba-4432-9a54-8888d00e3fa1+6:2273:x1862456979141120:12345-192.168.204.3@tcp:400:kworker/u8:2.0 Request processed in 51us (2008us total) trans 0 rc 0/0 00000100:00100000:2.0:1776178715.943675:0:6257:0:(nrs_fifo.c:241:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.204.3@tcp, seq: 1228 00000100:00100000:2.0:1776178715.947377:0:20755:0:(service.c:2152:ptlrpc_server_handle_req_in()) got req x1862456979141248 00000100:00100000:2.0:1776178715.947447:0:20755:0:(service.c:2152:ptlrpc_server_handle_req_in()) got req x1862456979141184 00000100:00100000:0.0:1776178715.947604:0:20759:0:(nrs_fifo.c:179:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.204.3@tcp, seq: 911 00000100:00100000:0.0:1776178715.947613:0:20759:0:(service.c:2324:ptlrpc_server_handle_request()) Handling RPC req@000000006c0c7de5 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_010:b50963bd-8eba-4432-9a54-8888d00e3fa1+9:2273:x1862456979141248:12345-192.168.204.3@tcp:400:kworker/u8:2.0 00000100:00100000:0.0:1776178715.947686:0:20759:0:(service.c:2372:ptlrpc_server_handle_request()) Handled RPC req@000000006c0c7de5 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_010:b50963bd-8eba-4432-9a54-8888d00e3fa1+9:2273:x1862456979141248:12345-192.168.204.3@tcp:400:kworker/u8:2.0 Request processed in 73us (5955us total) trans 0 rc 0/0 00000100:00100000:0.0:1776178715.947694:0:20759:0:(nrs_fifo.c:241:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.204.3@tcp, seq: 911 00000100:00100000:0.0:1776178715.947713:0:20759:0:(nrs_fifo.c:179:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.204.3@tcp, seq: 912 00000100:00100000:0.0:1776178715.947717:0:20759:0:(service.c:2324:ptlrpc_server_handle_request()) Handling RPC req@00000000fa20ba2c pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_010:b50963bd-8eba-4432-9a54-8888d00e3fa1+8:2273:x1862456979141184:12345-192.168.204.3@tcp:400:kworker/u8:2.0 00000100:00100000:0.0:1776178715.947735:0:20759:0:(service.c:2372:ptlrpc_server_handle_request()) Handled RPC req@00000000fa20ba2c pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_010:b50963bd-8eba-4432-9a54-8888d00e3fa1+8:2273:x1862456979141184:12345-192.168.204.3@tcp:400:kworker/u8:2.0 Request processed in 18us (5926us total) trans 0 rc 0/0 00000100:00100000:0.0:1776178715.947741:0:20759:0:(nrs_fifo.c:241:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.204.3@tcp, seq: 912 00000100:00100000:3.0:1776178715.951899:0:3346:0:(events.c:361:request_in_callback()) peer: 12345-192.168.204.3@tcp (source: 12345-192.168.204.3@tcp) 00000100:00100000:0.0:1776178715.952109:0:11281:0:(service.c:2152:ptlrpc_server_handle_req_in()) got req x1862456979141056 00000100:00100000:0.0:1776178715.952166:0:11281:0:(nrs_fifo.c:179:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.204.3@tcp, seq: 7024 00000100:00100000:0.0:1776178715.952172:0:11281:0:(service.c:2324:ptlrpc_server_handle_request()) Handling RPC req@0000000053c46e8d pname:cluuid+ref:pid:xid:nid:opc:job mdt00_004:0+-99:2269:x1862456979141056:12345-192.168.204.3@tcp:38:kworker/u8:2.0 00010000:02000000:0.0:1776178715.952191:0:11281:0:(ldlm_lib.c:1139:target_handle_connect()) lustre-MDT0000: Not available for connect from 192.168.204.3@tcp (stopping) 00000100:00100000:0.0:1776178715.952268:0:11281:0:(service.c:2372:ptlrpc_server_handle_request()) Handled RPC req@0000000053c46e8d pname:cluuid+ref:pid:xid:nid:opc:job mdt00_004:0+-99:2269:x1862456979141056:12345-192.168.204.3@tcp:38:kworker/u8:2.0 Request processed in 94us (377us total) trans 0 rc -19/-19 00000100:00100000:0.0:1776178715.952275:0:11281:0:(nrs_fifo.c:241:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.204.3@tcp, seq: 7024 00010000:00010000:1.0:1776178716.128833:0:9225:0:(ldlm_lockd.c:2792:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00010000:00010000:3.0:1776178717.152331:0:6244:0:(ldlm_lockd.c:2792:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00080000:3.0:1776178717.541221:0:22249:0:(pinger.c:251:ptlrpc_pinger_process_import()) lustre-MDT0000-lwp-OST0000_UUID->lustre-MDT0000_UUID: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00080000:3.0:1776178717.541229:0:22249:0:(recover.c:57:ptlrpc_initiate_recovery()) lustre-MDT0000_UUID: starting recovery 00000100:00080000:3.0:1776178717.541232:0:22249:0:(import.c:86:import_set_state_nolock()) 0000000047601a98 lustre-MDT0000_UUID: changing import state from DISCONN to CONNECTING 00000100:00080000:3.0:1776178717.541240:0:22249:0:(import.c:534:import_select_connection()) lustre-MDT0000-lwp-OST0000: connect to NID 0@lo last attempt 1419 00000100:00080000:3.0:1776178717.541245:0:22249:0:(import.c:581:import_select_connection()) lustre-MDT0000-lwp-OST0000: tried all connections, increasing latency to 21s 00000100:00080000:3.0:1776178717.541251:0:22249:0:(import.c:615:import_select_connection()) lustre-MDT0000-lwp-OST0000: import 0000000047601a98 using connection 192.168.204.103@tcp/0@lo 00000100:00100000:3.0:1776178717.541271:0:22249:0:(import.c:807:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5) req@000000008a1e777f x1862456998120000/t0(0) o38->lustre-MDT0000-lwp-OST0000@0@lo:12/10 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' 00000100:00080000:3.0:1776178717.541312:0:22249:0:(pinger.c:251:ptlrpc_pinger_process_import()) lustre-MDT0000-lwp-OST0001_UUID->lustre-MDT0000_UUID: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00080000:3.0:1776178717.541315:0:22249:0:(recover.c:57:ptlrpc_initiate_recovery()) lustre-MDT0000_UUID: starting recovery 00000100:00080000:3.0:1776178717.541317:0:22249:0:(import.c:86:import_set_state_nolock()) 00000000dfc8db13 lustre-MDT0000_UUID: changing import state from DISCONN to CONNECTING 00000100:00080000:3.0:1776178717.541320:0:22249:0:(import.c:534:import_select_connection()) lustre-MDT0000-lwp-OST0001: connect to NID 0@lo last attempt 1419 00000100:00080000:3.0:1776178717.541322:0:22249:0:(import.c:581:import_select_connection()) lustre-MDT0000-lwp-OST0001: tried all connections, increasing latency to 21s 00000100:00080000:3.0:1776178717.541325:0:22249:0:(import.c:615:import_select_connection()) lustre-MDT0000-lwp-OST0001: import 00000000dfc8db13 using connection 192.168.204.103@tcp/0@lo 00000100:00100000:3.0:1776178717.541335:0:22249:0:(import.c:807:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5) req@00000000c59ba3a2 x1862456998120128/t0(0) o38->lustre-MDT0000-lwp-OST0001@0@lo:12/10 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' 00000100:00080000:3.0:1776178717.541361:0:22249:0:(pinger.c:251:ptlrpc_pinger_process_import()) lustre-MDT0001-mdtlov_UUID->lustre-MDT0000_UUID: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00080000:3.0:1776178717.541364:0:22249:0:(recover.c:57:ptlrpc_initiate_recovery()) lustre-MDT0000_UUID: starting recovery 00000100:00080000:3.0:1776178717.541365:0:22249:0:(import.c:86:import_set_state_nolock()) 00000000ba4f1821 lustre-MDT0000_UUID: changing import state from DISCONN to CONNECTING 00000100:00080000:3.0:1776178717.541368:0:22249:0:(import.c:534:import_select_connection()) lustre-MDT0000-osp-MDT0001: connect to NID 0@lo last attempt 1419 00000100:00080000:3.0:1776178717.541369:0:22249:0:(import.c:581:import_select_connection()) lustre-MDT0000-osp-MDT0001: tried all connections, increasing latency to 21s 00000100:00080000:3.0:1776178717.541372:0:22249:0:(import.c:615:import_select_connection()) lustre-MDT0000-osp-MDT0001: import 00000000ba4f1821 using connection 192.168.204.103@tcp/0@lo 00000100:00100000:3.0:1776178717.541384:0:22249:0:(import.c:807:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5) req@0000000040899e7d x1862456998120256/t0(0) o38->lustre-MDT0000-osp-MDT0001@0@lo:24/4 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' 00000100:00080000:3.0:1776178717.541402:0:22249:0:(pinger.c:251:ptlrpc_pinger_process_import()) lustre-MDT0000-lwp-MDT0001_UUID->lustre-MDT0000_UUID: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00080000:3.0:1776178717.541405:0:22249:0:(recover.c:57:ptlrpc_initiate_recovery()) lustre-MDT0000_UUID: starting recovery 00000100:00080000:3.0:1776178717.541406:0:22249:0:(import.c:86:import_set_state_nolock()) 00000000a7c55b1f lustre-MDT0000_UUID: changing import state from DISCONN to CONNECTING 00000100:00080000:3.0:1776178717.541408:0:22249:0:(import.c:534:import_select_connection()) lustre-MDT0000-lwp-MDT0001: connect to NID 0@lo last attempt 1419 00000100:00080000:3.0:1776178717.541410:0:22249:0:(import.c:581:import_select_connection()) lustre-MDT0000-lwp-MDT0001: tried all connections, increasing latency to 21s 00000100:00080000:3.0:1776178717.541413:0:22249:0:(import.c:615:import_select_connection()) lustre-MDT0000-lwp-MDT0001: import 00000000a7c55b1f using connection 192.168.204.103@tcp/0@lo 00000100:00100000:3.0:1776178717.541423:0:22249:0:(import.c:807:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5) req@00000000758405de x1862456998120448/t0(0) o38->lustre-MDT0000-lwp-MDT0001@0@lo:12/10 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' 00000100:00100000:3.0:1776178717.541452:0:3355:0:(client.c:1733:ptlrpc_send_new_req()) Sending RPC req@00000000eeeaf4ad pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_02:lustre-MDT0001-lwp-OST0001_UUID:3355:1862456998120192:0@lo:400:kworker/u8:0.0 00000100:00100000:3.0:1776178717.541512:0:3355:0:(events.c:361:request_in_callback()) peer: 12345-0@lo (source: 12345-0@lo) 00000100:00100000:3.0:1776178717.541545:0:3355:0:(ptlrpcd.c:415:ptlrpcd_check()) transfer 1 async RPCs [3->2] 00000100:00100000:3.0:1776178717.541550:0:3355:0:(client.c:1733:ptlrpc_send_new_req()) Sending RPC req@000000000ff6c1d2 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_02:lustre-MDT0001-mdtlov_UUID:3355:1862456998120320:0@lo:400:kworker/u8:0.0 00000100:00100000:3.0:1776178717.541573:0:3355:0:(events.c:361:request_in_callback()) peer: 12345-0@lo (source: 12345-0@lo) 00000100:00100000:3.0:1776178717.541606:0:6257:0:(service.c:2152:ptlrpc_server_handle_req_in()) got req x1862456998120192 00000100:00100000:3.0:1776178717.541645:0:6257:0:(nrs_fifo.c:179:nrs_fifo_req_get()) NRS start fifo request from 12345-0@lo, seq: 1229 00000100:00100000:3.0:1776178717.541651:0:6257:0:(service.c:2324:ptlrpc_server_handle_request()) Handling RPC req@000000005f1d3350 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_002:lustre-MDT0001-lwp-OST0001_UUID+5:3355:x1862456998120192:12345-0@lo:400:kworker/u8:0.0 00000100:00100000:3.0:1776178717.541696:0:6257:0:(service.c:2372:ptlrpc_server_handle_request()) Handled RPC req@000000005f1d3350 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_002:lustre-MDT0001-lwp-OST0001_UUID+5:3355:x1862456998120192:12345-0@lo:400:kworker/u8:0.0 Request processed in 44us (185us total) trans 0 rc 0/0 00000100:00100000:3.0:1776178717.541703:0:6257:0:(nrs_fifo.c:241:nrs_fifo_req_stop()) NRS stop fifo request from 12345-0@lo, seq: 1229 00000100:00100000:3.0:1776178717.541741:0:3355:0:(client.c:2215:ptlrpc_check_set()) Completed RPC req@00000000eeeaf4ad pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_02:lustre-MDT0001-lwp-OST0001_UUID:3355:1862456998120192:0@lo:400:kworker/u8:0.0 00000100:00100000:3.0:1776178717.541766:0:20755:0:(service.c:2152:ptlrpc_server_handle_req_in()) got req x1862456998120320 00000100:00100000:3.0:1776178717.541788:0:20755:0:(nrs_fifo.c:179:nrs_fifo_req_get()) NRS start fifo request from 12345-0@lo, seq: 913 00000100:00100000:3.0:1776178717.541792:0:20755:0:(service.c:2324:ptlrpc_server_handle_request()) Handling RPC req@000000006459e586 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_006:lustre-MDT0001-mdtlov_UUID+5:3355:x1862456998120320:12345-0@lo:400:kworker/u8:0.0 00000100:00100000:3.0:1776178717.541823:0:20755:0:(service.c:2372:ptlrpc_server_handle_request()) Handled RPC req@000000006459e586 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_006:lustre-MDT0001-mdtlov_UUID+5:3355:x1862456998120320:12345-0@lo:400:kworker/u8:0.0 Request processed in 30us (250us total) trans 0 rc 0/0 00000100:00100000:3.0:1776178717.541829:0:20755:0:(nrs_fifo.c:241:nrs_fifo_req_stop()) NRS stop fifo request from 12345-0@lo, seq: 913 00000100:00100000:3.0:1776178717.541851:0:3355:0:(client.c:2215:ptlrpc_check_set()) Completed RPC req@000000000ff6c1d2 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_02:lustre-MDT0001-mdtlov_UUID:3355:1862456998120320:0@lo:400:kworker/u8:0.0 00000100:00100000:2.0:1776178717.542244:0:3352:0:(client.c:1733:ptlrpc_send_new_req()) Sending RPC req@000000008a1e777f pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:lustre-MDT0000-lwp-OST0000_UUID:3352:1862456998120000:0@lo:38:kworker/u8:0.0 00000100:00100000:2.0:1776178717.542277:0:3352:0:(events.c:361:request_in_callback()) peer: 12345-0@lo (source: 12345-0@lo) 00000100:00100000:2.0:1776178717.542296:0:3352:0:(client.c:1733:ptlrpc_send_new_req()) Sending RPC req@00000000c59ba3a2 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:lustre-MDT0000-lwp-OST0001_UUID:3352:1862456998120128:0@lo:38:kworker/u8:0.0 00000100:00100000:2.0:1776178717.542325:0:3352:0:(events.c:361:request_in_callback()) peer: 12345-0@lo (source: 12345-0@lo) 00000100:00100000:2.0:1776178717.542339:0:3352:0:(client.c:1733:ptlrpc_send_new_req()) Sending RPC req@0000000040899e7d pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:lustre-MDT0001-mdtlov_UUID:3352:1862456998120256:0@lo:38:kworker/u8:0.0 00000100:00100000:2.0:1776178717.542359:0:3352:0:(events.c:361:request_in_callback()) peer: 12345-0@lo (source: 12345-0@lo) 00000100:00100000:2.0:1776178717.542371:0:3352:0:(client.c:1733:ptlrpc_send_new_req()) Sending RPC req@00000000758405de pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:lustre-MDT0000-lwp-MDT0001_UUID:3352:1862456998120448:0@lo:38:kworker/u8:0.0 00000100:00100000:2.0:1776178717.542390:0:3352:0:(events.c:361:request_in_callback()) peer: 12345-0@lo (source: 12345-0@lo) 00000100:00100000:2.0:1776178717.542422:0:8250:0:(service.c:2152:ptlrpc_server_handle_req_in()) got req x1862456998120000 00000100:00100000:2.0:1776178717.542443:0:8250:0:(service.c:2152:ptlrpc_server_handle_req_in()) got req x1862456998120128 00000100:00100000:2.0:1776178717.542451:0:8250:0:(service.c:2152:ptlrpc_server_handle_req_in()) got req x1862456998120448 00000100:00100000:2.0:1776178717.542460:0:8250:0:(nrs_fifo.c:179:nrs_fifo_req_get()) NRS start fifo request from 12345-0@lo, seq: 7025 00000100:00100000:2.0:1776178717.542464:0:8250:0:(service.c:2324:ptlrpc_server_handle_request()) Handling RPC req@00000000ad228ff6 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_003:0+-99:3352:x1862456998120000:12345-0@lo:38:kworker/u8:0.0 00010000:02000000:2.0:1776178717.542477:0:8250:0:(ldlm_lib.c:1139:target_handle_connect()) lustre-MDT0000: Not available for connect from 0@lo (stopping) 00000100:00100000:2.0:1776178717.542508:0:8250:0:(service.c:2372:ptlrpc_server_handle_request()) Handled RPC req@00000000ad228ff6 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_003:0+-99:3352:x1862456998120000:12345-0@lo:38:kworker/u8:0.0 Request processed in 42us (231us total) trans 0 rc -19/-19 00000100:00100000:2.0:1776178717.542514:0:8250:0:(nrs_fifo.c:241:nrs_fifo_req_stop()) NRS stop fifo request from 12345-0@lo, seq: 7025 00000100:00100000:2.0:1776178717.542527:0:8250:0:(nrs_fifo.c:179:nrs_fifo_req_get()) NRS start fifo request from 12345-0@lo, seq: 7026 00000100:00100000:2.0:1776178717.542530:0:8250:0:(service.c:2324:ptlrpc_server_handle_request()) Handling RPC req@0000000050bb92d0 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_003:0+-99:3352:x1862456998120128:12345-0@lo:38:kworker/u8:0.0 00010000:02000000:2.0:1776178717.542535:0:8250:0:(ldlm_lib.c:1139:target_handle_connect()) lustre-MDT0000: Not available for connect from 0@lo (stopping) 00000100:00100000:2.0:1776178717.542553:0:8250:0:(service.c:2372:ptlrpc_server_handle_request()) Handled RPC req@0000000050bb92d0 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_003:0+-99:3352:x1862456998120128:12345-0@lo:38:kworker/u8:0.0 Request processed in 23us (229us total) trans 0 rc -19/-19 00000100:00100000:2.0:1776178717.542558:0:8250:0:(nrs_fifo.c:241:nrs_fifo_req_stop()) NRS stop fifo request from 12345-0@lo, seq: 7026 00000100:00100000:2.0:1776178717.542566:0:8250:0:(nrs_fifo.c:179:nrs_fifo_req_get()) NRS start fifo request from 12345-0@lo, seq: 7027 00000100:00100000:2.0:1776178717.542569:0:8250:0:(service.c:2324:ptlrpc_server_handle_request()) Handling RPC req@000000004547bc6e pname:cluuid+ref:pid:xid:nid:opc:job mdt00_003:0+-99:3352:x1862456998120448:12345-0@lo:38:kworker/u8:0.0 00010000:02000000:2.0:1776178717.542574:0:8250:0:(ldlm_lib.c:1139:target_handle_connect()) lustre-MDT0000: Not available for connect from 0@lo (stopping) 00000100:00100000:2.0:1776178717.542589:0:8250:0:(service.c:2372:ptlrpc_server_handle_request()) Handled RPC req@000000004547bc6e pname:cluuid+ref:pid:xid:nid:opc:job mdt00_003:0+-99:3352:x1862456998120448:12345-0@lo:38:kworker/u8:0.0 Request processed in 19us (199us total) trans 0 rc -19/-19 00000100:00100000:2.0:1776178717.542594:0:8250:0:(nrs_fifo.c:241:nrs_fifo_req_stop()) NRS stop fifo request from 12345-0@lo, seq: 7027 00000100:00100000:2.0:1776178717.542609:0:15213:0:(service.c:2152:ptlrpc_server_handle_req_in()) got req x1862456998120256 00000100:00100000:2.0:1776178717.542627:0:15213:0:(nrs_fifo.c:179:nrs_fifo_req_get()) NRS start fifo request from 12345-0@lo, seq: 808 00000100:00100000:2.0:1776178717.542630:0:15213:0:(service.c:2324:ptlrpc_server_handle_request()) Handling RPC req@000000009751a398 pname:cluuid+ref:pid:xid:nid:opc:job mdt_out00_003:0+-99:3352:x1862456998120256:12345-0@lo:38:kworker/u8:0.0 00010000:02000000:2.0:1776178717.542636:0:15213:0:(ldlm_lib.c:1139:target_handle_connect()) lustre-MDT0000: Not available for connect from 0@lo (stopping) 00000100:00100000:2.0:1776178717.542655:0:15213:0:(service.c:2372:ptlrpc_server_handle_request()) Handled RPC req@000000009751a398 pname:cluuid+ref:pid:xid:nid:opc:job mdt_out00_003:0+-99:3352:x1862456998120256:12345-0@lo:38:kworker/u8:0.0 Request processed in 24us (296us total) trans 0 rc -19/-19 00000100:00100000:2.0:1776178717.542659:0:15213:0:(nrs_fifo.c:241:nrs_fifo_req_stop()) NRS stop fifo request from 12345-0@lo, seq: 808 00000100:00080000:2.0:1776178717.542690:0:3352:0:(import.c:86:import_set_state_nolock()) 0000000047601a98 lustre-MDT0000_UUID: changing import state from CONNECTING to DISCONN 00000100:00080000:2.0:1776178717.542693:0:3352:0:(import.c:1429:ptlrpc_connect_interpret()) recovery of lustre-MDT0000_UUID on 192.168.204.103@tcp failed (-19) 00000100:00100000:2.0:1776178717.542697:0:3352:0:(client.c:2215:ptlrpc_check_set()) Completed RPC req@000000008a1e777f pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:lustre-MDT0000-lwp-OST0000_UUID:3352:1862456998120000:0@lo:38:kworker/u8:0.0 00000100:00080000:2.0:1776178717.542705:0:3352:0:(import.c:86:import_set_state_nolock()) 00000000dfc8db13 lustre-MDT0000_UUID: changing import state from CONNECTING to DISCONN 00000100:00080000:2.0:1776178717.542707:0:3352:0:(import.c:1429:ptlrpc_connect_interpret()) recovery of lustre-MDT0000_UUID on 192.168.204.103@tcp failed (-19) 00000100:00100000:2.0:1776178717.542708:0:3352:0:(client.c:2215:ptlrpc_check_set()) Completed RPC req@00000000c59ba3a2 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:lustre-MDT0000-lwp-OST0001_UUID:3352:1862456998120128:0@lo:38:kworker/u8:0.0 00000100:00080000:2.0:1776178717.542714:0:3352:0:(import.c:86:import_set_state_nolock()) 00000000ba4f1821 lustre-MDT0000_UUID: changing import state from CONNECTING to DISCONN 00000100:00080000:2.0:1776178717.542717:0:3352:0:(import.c:1429:ptlrpc_connect_interpret()) recovery of lustre-MDT0000_UUID on 192.168.204.103@tcp failed (-19) 00000100:00100000:2.0:1776178717.542719:0:3352:0:(client.c:2215:ptlrpc_check_set()) Completed RPC req@0000000040899e7d pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:lustre-MDT0001-mdtlov_UUID:3352:1862456998120256:0@lo:38:kworker/u8:0.0 00000100:00080000:2.0:1776178717.542724:0:3352:0:(import.c:86:import_set_state_nolock()) 00000000a7c55b1f lustre-MDT0000_UUID: changing import state from CONNECTING to DISCONN 00000100:00080000:2.0:1776178717.542726:0:3352:0:(import.c:1429:ptlrpc_connect_interpret()) recovery of lustre-MDT0000_UUID on 192.168.204.103@tcp failed (-19) 00000100:00100000:2.0:1776178717.542728:0:3352:0:(client.c:2215:ptlrpc_check_set()) Completed RPC req@00000000758405de pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:lustre-MDT0000-lwp-MDT0001_UUID:3352:1862456998120448:0@lo:38:kworker/u8:0.0 00000100:00100000:1.0:1776178717.543248:0:3353:0:(client.c:1733:ptlrpc_send_new_req()) Sending RPC req@000000001a990b02 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_00:ac61f839-0e8e-4cac-9c59-08bcfbbad330:3353:1862456998119936:0@lo:400:kworker/u8:0.0 00000100:00100000:1.0:1776178717.543286:0:3353:0:(events.c:361:request_in_callback()) peer: 12345-0@lo (source: 12345-0@lo) 00000100:00100000:1.0:1776178717.543320:0:3353:0:(client.c:1733:ptlrpc_send_new_req()) Sending RPC req@00000000ab295186 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_00:lustre-MDT0001-mdtlov_UUID:3353:1862456998120384:0@lo:400:kworker/u8:0.0 00000100:00100000:1.0:1776178717.543341:0:3353:0:(events.c:361:request_in_callback()) peer: 12345-0@lo (source: 12345-0@lo) 00000100:00100000:1.0:1776178717.543358:0:3353:0:(ptlrpcd.c:415:ptlrpcd_check()) transfer 1 async RPCs [1->0] 00000100:00100000:1.0:1776178717.543364:0:3353:0:(client.c:1733:ptlrpc_send_new_req()) Sending RPC req@00000000854f8d52 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_00:lustre-MDT0001-lwp-OST0000_UUID:3353:1862456998120064:0@lo:400:kworker/u8:0.0 00000100:00100000:1.0:1776178717.543384:0:3353:0:(events.c:361:request_in_callback()) peer: 12345-0@lo (source: 12345-0@lo) 00000100:00100000:1.0:1776178717.543410:0:6256:0:(service.c:2152:ptlrpc_server_handle_req_in()) got req x1862456998120064 00000100:00100000:1.0:1776178717.543439:0:6256:0:(nrs_fifo.c:179:nrs_fifo_req_get()) NRS start fifo request from 12345-0@lo, seq: 1230 00000100:00100000:1.0:1776178717.543443:0:6256:0:(service.c:2324:ptlrpc_server_handle_request()) Handling RPC req@000000008a15a61d pname:cluuid+ref:pid:xid:nid:opc:job mdt00_001:lustre-MDT0001-lwp-OST0000_UUID+5:3353:x1862456998120064:12345-0@lo:400:kworker/u8:0.0 00000100:00100000:1.0:1776178717.543479:0:6256:0:(service.c:2372:ptlrpc_server_handle_request()) Handled RPC req@000000008a15a61d pname:cluuid+ref:pid:xid:nid:opc:job mdt00_001:lustre-MDT0001-lwp-OST0000_UUID+5:3353:x1862456998120064:12345-0@lo:400:kworker/u8:0.0 Request processed in 35us (95us total) trans 0 rc 0/0 00000100:00100000:1.0:1776178717.543485:0:6256:0:(nrs_fifo.c:241:nrs_fifo_req_stop()) NRS stop fifo request from 12345-0@lo, seq: 1230 00000100:00100000:1.0:1776178717.543503:0:20762:0:(service.c:2152:ptlrpc_server_handle_req_in()) got req x1862456998120384 00000100:00100000:1.0:1776178717.543522:0:20762:0:(nrs_fifo.c:179:nrs_fifo_req_get()) NRS start fifo request from 12345-0@lo, seq: 914 00000100:00100000:1.0:1776178717.543527:0:20762:0:(service.c:2324:ptlrpc_server_handle_request()) Handling RPC req@000000007cfd1c43 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_013:lustre-MDT0001-mdtlov_UUID+5:3353:x1862456998120384:12345-0@lo:400:kworker/u8:0.0 00000100:00100000:1.0:1776178717.543550:0:20762:0:(service.c:2372:ptlrpc_server_handle_request()) Handled RPC req@000000007cfd1c43 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_013:lustre-MDT0001-mdtlov_UUID+5:3353:x1862456998120384:12345-0@lo:400:kworker/u8:0.0 Request processed in 23us (210us total) trans 0 rc 0/0 00000100:00100000:1.0:1776178717.543556:0:20762:0:(nrs_fifo.c:241:nrs_fifo_req_stop()) NRS stop fifo request from 12345-0@lo, seq: 914 00000100:00100000:1.0:1776178717.543579:0:3353:0:(client.c:2215:ptlrpc_check_set()) Completed RPC req@00000000854f8d52 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_00:lustre-MDT0001-lwp-OST0000_UUID:3353:1862456998120064:0@lo:400:kworker/u8:0.0 00000100:00100000:1.0:1776178717.543587:0:3353:0:(client.c:2215:ptlrpc_check_set()) Completed RPC req@00000000ab295186 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_00:lustre-MDT0001-mdtlov_UUID:3353:1862456998120384:0@lo:400:kworker/u8:0.0 00000100:00100000:2.0:1776178717.543923:0:20140:0:(service.c:2152:ptlrpc_server_handle_req_in()) got req x1862456998119936 00000100:00100000:2.0:1776178717.543958:0:20140:0:(nrs_fifo.c:179:nrs_fifo_req_get()) NRS start fifo request from 12345-0@lo, seq: 356 00000100:00100000:2.0:1776178717.543962:0:20140:0:(service.c:2324:ptlrpc_server_handle_request()) Handling RPC req@00000000b42c55dd pname:cluuid+ref:pid:xid:nid:opc:job ll_mgs_0002:ac61f839-0e8e-4cac-9c59-08bcfbbad330+18:3353:x1862456998119936:12345-0@lo:400:kworker/u8:0.0 00000100:00100000:2.0:1776178717.544002:0:20140:0:(service.c:2372:ptlrpc_server_handle_request()) Handled RPC req@00000000b42c55dd pname:cluuid+ref:pid:xid:nid:opc:job ll_mgs_0002:ac61f839-0e8e-4cac-9c59-08bcfbbad330+18:3353:x1862456998119936:12345-0@lo:400:kworker/u8:0.0 Request processed in 39us (716us total) trans 0 rc 0/0 00000100:00100000:2.0:1776178717.544008:0:20140:0:(nrs_fifo.c:241:nrs_fifo_req_stop()) NRS stop fifo request from 12345-0@lo, seq: 356 00000100:00100000:1.0:1776178717.544164:0:3353:0:(client.c:2215:ptlrpc_check_set()) Completed RPC req@000000001a990b02 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_00:ac61f839-0e8e-4cac-9c59-08bcfbbad330:3353:1862456998119936:0@lo:400:kworker/u8:0.0 00010000:00010000:1.0:1776178718.180430:0:6245:0:(ldlm_lockd.c:2792:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:1.0:1776178719.072734:0:3353:0:(ptlrpcd.c:415:ptlrpcd_check()) transfer 1 async RPCs [1->0] 00000100:00100000:1.0:1776178719.072757:0:3353:0:(client.c:1733:ptlrpc_send_new_req()) Sending RPC req@00000000d0d837c5 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_00:lustre-MDT0001-mdtlov_UUID:3353:1862456998120512:0@lo:13:osp-pre-1-1.0 00000100:00100000:1.0:1776178719.072833:0:3353:0:(events.c:361:request_in_callback()) peer: 12345-0@lo (source: 12345-0@lo) 00000100:00100000:2.0:1776178719.073138:0:8231:0:(service.c:2152:ptlrpc_server_handle_req_in()) got req x1862456998120512 00000100:00100000:2.0:1776178719.073195:0:8231:0:(nrs_fifo.c:179:nrs_fifo_req_get()) NRS start fifo request from 12345-0@lo, seq: 689 00000100:00100000:2.0:1776178719.073201:0:8231:0:(service.c:2324:ptlrpc_server_handle_request()) Handling RPC req@00000000087015c3 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_create00:lustre-MDT0001-mdtlov_UUID+5:3353:x1862456998120512:12345-0@lo:13:osp-pre-1-1.0 00002000:00000024:2.0:1776178719.073232:0:8231:0:(ofd_obd.c:740:ofd_statfs()) blocks cached 229376 granted 491746368 pending 0 free 3923689472 avail 3692167168 00000100:00100000:2.0:1776178719.073326:0:8231:0:(service.c:2372:ptlrpc_server_handle_request()) Handled RPC req@00000000087015c3 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_create00:lustre-MDT0001-mdtlov_UUID+5:3353:x1862456998120512:12345-0@lo:13:osp-pre-1-1.0 Request processed in 124us (494us total) trans 0 rc 0/0 00000100:00100000:2.0:1776178719.073337:0:8231:0:(nrs_fifo.c:241:nrs_fifo_req_stop()) NRS stop fifo request from 12345-0@lo, seq: 689 00000100:00100000:1.0:1776178719.079122:0:3353:0:(client.c:2215:ptlrpc_check_set()) Completed RPC req@00000000d0d837c5 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_00:lustre-MDT0001-mdtlov_UUID:3353:1862456998120512:0@lo:13:osp-pre-1-1.0 00010000:00010000:2.0:1776178719.200739:0:9224:0:(ldlm_lockd.c:2792:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:3.0:1776178719.584379:0:3355:0:(client.c:1733:ptlrpc_send_new_req()) Sending RPC req@000000002300ff10 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_02:lustre-MDT0001-mdtlov_UUID:3355:1862456998120576:0@lo:13:osp-pre-0-1.0 00000100:00100000:3.0:1776178719.584460:0:3355:0:(events.c:361:request_in_callback()) peer: 12345-0@lo (source: 12345-0@lo) 00000100:00100000:1.0:1776178719.584563:0:8231:0:(service.c:2152:ptlrpc_server_handle_req_in()) got req x1862456998120576 00000100:00100000:1.0:1776178719.584618:0:8231:0:(nrs_fifo.c:179:nrs_fifo_req_get()) NRS start fifo request from 12345-0@lo, seq: 690 00000100:00100000:1.0:1776178719.584624:0:8231:0:(service.c:2324:ptlrpc_server_handle_request()) Handling RPC req@000000000ff6c1d2 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_create00:lustre-MDT0001-mdtlov_UUID+5:3355:x1862456998120576:12345-0@lo:13:osp-pre-0-1.0 00002000:00000024:1.0:1776178719.584657:0:8231:0:(ofd_obd.c:740:ofd_statfs()) blocks cached 315392 granted 484883776 pending 0 free 3923689472 avail 3692167168 00000100:00100000:1.0:1776178719.591144:0:8231:0:(service.c:2372:ptlrpc_server_handle_request()) Handled RPC req@000000000ff6c1d2 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_create00:lustre-MDT0001-mdtlov_UUID+5:3355:x1862456998120576:12345-0@lo:13:osp-pre-0-1.0 Request processed in 6508us (6673us total) trans 0 rc 0/0 00000100:00100000:1.0:1776178719.591163:0:8231:0:(nrs_fifo.c:241:nrs_fifo_req_stop()) NRS stop fifo request from 12345-0@lo, seq: 690 00000100:00100000:3.0:1776178719.596001:0:3355:0:(client.c:2215:ptlrpc_check_set()) Completed RPC req@000000002300ff10 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_02:lustre-MDT0001-mdtlov_UUID:3355:1862456998120576:0@lo:13:osp-pre-0-1.0 00010000:00010000:1.0:1776178720.225306:0:9225:0:(ldlm_lockd.c:2792:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:1.0:1776178721.061946:0:3347:0:(events.c:361:request_in_callback()) peer: 12345-192.168.204.3@tcp (source: 12345-192.168.204.3@tcp) 00000100:00100000:1.0:1776178721.061993:0:3347:0:(events.c:361:request_in_callback()) peer: 12345-192.168.204.3@tcp (source: 12345-192.168.204.3@tcp) 00000100:00100000:1.0:1776178721.062029:0:3347:0:(events.c:361:request_in_callback()) peer: 12345-192.168.204.3@tcp (source: 12345-192.168.204.3@tcp) 00000100:00100000:1.0:1776178721.062101:0:3347:0:(events.c:361:request_in_callback()) peer: 12345-192.168.204.3@tcp (source: 12345-192.168.204.3@tcp) 00000100:00100000:3.0:1776178721.062111:0:6257:0:(service.c:2152:ptlrpc_server_handle_req_in()) got req x1862456979141376 00000100:00100000:1.0:1776178721.062129:0:3347:0:(events.c:361:request_in_callback()) peer: 12345-192.168.204.3@tcp (source: 12345-192.168.204.3@tcp) 00000100:00100000:3.0:1776178721.062162:0:6257:0:(service.c:2152:ptlrpc_server_handle_req_in()) got req x1862456979141440 00000100:00100000:0.0:1776178721.062207:0:20137:0:(service.c:2152:ptlrpc_server_handle_req_in()) got req x1862456979141312 00000100:00100000:2.0:1776178721.062240:0:20755:0:(service.c:2152:ptlrpc_server_handle_req_in()) got req x1862456979141504 00000100:00100000:0.0:1776178721.062244:0:20137:0:(nrs_fifo.c:179:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.204.3@tcp, seq: 357 00000100:00100000:0.0:1776178721.062252:0:20137:0:(service.c:2324:ptlrpc_server_handle_request()) Handling RPC req@00000000e31b009d pname:cluuid+ref:pid:xid:nid:opc:job ll_mgs_0000:5b5c66b1-1a2f-4651-8548-0c75b9a38a94+9:2272:x1862456979141312:12345-192.168.204.3@tcp:400:kworker/u8:2.0 00000100:00100000:2.0:1776178721.062273:0:20755:0:(service.c:2152:ptlrpc_server_handle_req_in()) got req x1862456979141568 00000100:00100000:2.0:1776178721.062293:0:20755:0:(nrs_fifo.c:179:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.204.3@tcp, seq: 915 00000100:00100000:2.0:1776178721.062298:0:20755:0:(service.c:2324:ptlrpc_server_handle_request()) Handling RPC req@00000000589b0247 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_006:b50963bd-8eba-4432-9a54-8888d00e3fa1+8:2272:x1862456979141504:12345-192.168.204.3@tcp:400:kworker/u8:2.0 00000100:00100000:0.0:1776178721.062331:0:20137:0:(service.c:2372:ptlrpc_server_handle_request()) Handled RPC req@00000000e31b009d pname:cluuid+ref:pid:xid:nid:opc:job ll_mgs_0000:5b5c66b1-1a2f-4651-8548-0c75b9a38a94+9:2272:x1862456979141312:12345-192.168.204.3@tcp:400:kworker/u8:2.0 Request processed in 78us (337us total) trans 0 rc 0/0 00000100:00100000:0.0:1776178721.062338:0:20137:0:(nrs_fifo.c:241:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.204.3@tcp, seq: 357 00000100:00100000:2.0:1776178721.062350:0:20755:0:(service.c:2372:ptlrpc_server_handle_request()) Handled RPC req@00000000589b0247 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_006:b50963bd-8eba-4432-9a54-8888d00e3fa1+8:2272:x1862456979141504:12345-192.168.204.3@tcp:400:kworker/u8:2.0 Request processed in 50us (320us total) trans 0 rc 0/0 00000100:00100000:2.0:1776178721.062359:0:20755:0:(nrs_fifo.c:241:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.204.3@tcp, seq: 915 00000100:00100000:2.0:1776178721.062375:0:20755:0:(nrs_fifo.c:179:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.204.3@tcp, seq: 916 00000100:00100000:2.0:1776178721.062379:0:20755:0:(service.c:2324:ptlrpc_server_handle_request()) Handling RPC req@00000000ce34953a pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_006:b50963bd-8eba-4432-9a54-8888d00e3fa1+9:2272:x1862456979141568:12345-192.168.204.3@tcp:400:kworker/u8:2.0 00000100:00100000:2.0:1776178721.062401:0:20755:0:(service.c:2372:ptlrpc_server_handle_request()) Handled RPC req@00000000ce34953a pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_006:b50963bd-8eba-4432-9a54-8888d00e3fa1+9:2272:x1862456979141568:12345-192.168.204.3@tcp:400:kworker/u8:2.0 Request processed in 21us (273us total) trans 0 rc 0/0 00000100:00100000:2.0:1776178721.062407:0:20755:0:(nrs_fifo.c:241:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.204.3@tcp, seq: 916 00000100:00100000:1.0:1776178721.063017:0:6255:0:(nrs_fifo.c:179:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.204.3@tcp, seq: 1231 00000100:00100000:1.0:1776178721.063025:0:6255:0:(service.c:2324:ptlrpc_server_handle_request()) Handling RPC req@00000000e4dc0a67 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_000:b50963bd-8eba-4432-9a54-8888d00e3fa1+6:2272:x1862456979141440:12345-192.168.204.3@tcp:400:kworker/u8:2.0 00000100:00100000:1.0:1776178721.063095:0:6255:0:(service.c:2372:ptlrpc_server_handle_request()) Handled RPC req@00000000e4dc0a67 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_000:b50963bd-8eba-4432-9a54-8888d00e3fa1+6:2272:x1862456979141440:12345-192.168.204.3@tcp:400:kworker/u8:2.0 Request processed in 70us (995us total) trans 0 rc 0/0 00000100:00100000:1.0:1776178721.063102:0:6255:0:(nrs_fifo.c:241:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.204.3@tcp, seq: 1231 00000100:00100000:1.0:1776178721.063117:0:6255:0:(nrs_fifo.c:179:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.204.3@tcp, seq: 7028 00000100:00100000:1.0:1776178721.063121:0:6255:0:(service.c:2324:ptlrpc_server_handle_request()) Handling RPC req@000000001ff6e514 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_000:0+-99:2269:x1862456979141376:12345-192.168.204.3@tcp:38:kworker/u8:2.0 00010000:02000000:1.0:1776178721.063134:0:6255:0:(ldlm_lib.c:1139:target_handle_connect()) lustre-MDT0000: Not available for connect from 192.168.204.3@tcp (stopping) 00000100:00100000:1.0:1776178721.063159:0:6255:0:(service.c:2372:ptlrpc_server_handle_request()) Handled RPC req@000000001ff6e514 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_000:0+-99:2269:x1862456979141376:12345-192.168.204.3@tcp:38:kworker/u8:2.0 Request processed in 37us (1222us total) trans 0 rc -19/-19 00000100:00100000:1.0:1776178721.063165:0:6255:0:(nrs_fifo.c:241:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.204.3@tcp, seq: 7028 00010000:00010000:3.0:1776178721.251551:0:6244:0:(ldlm_lockd.c:2792:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00010000:00010000:1.0:1776178722.272317:0:6245:0:(ldlm_lockd.c:2792:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00080000:3.0:1776178722.656322:0:22249:0:(pinger.c:251:ptlrpc_pinger_process_import()) lustre-MDT0000-lwp-OST0000_UUID->lustre-MDT0000_UUID: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00080000:3.0:1776178722.656331:0:22249:0:(recover.c:57:ptlrpc_initiate_recovery()) lustre-MDT0000_UUID: starting recovery 00000100:00080000:3.0:1776178722.656334:0:22249:0:(import.c:86:import_set_state_nolock()) 0000000047601a98 lustre-MDT0000_UUID: changing import state from DISCONN to CONNECTING 00000100:00080000:3.0:1776178722.656342:0:22249:0:(import.c:534:import_select_connection()) lustre-MDT0000-lwp-OST0000: connect to NID 0@lo last attempt 1424 00000100:00080000:3.0:1776178722.656346:0:22249:0:(import.c:581:import_select_connection()) lustre-MDT0000-lwp-OST0000: tried all connections, increasing latency to 21s 00000100:00080000:3.0:1776178722.656352:0:22249:0:(import.c:615:import_select_connection()) lustre-MDT0000-lwp-OST0000: import 0000000047601a98 using connection 192.168.204.103@tcp/0@lo 00000100:00100000:3.0:1776178722.656374:0:22249:0:(import.c:807:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5) req@00000000d17c6101 x1862456998120704/t0(0) o38->lustre-MDT0000-lwp-OST0000@0@lo:12/10 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' 00000100:00080000:3.0:1776178722.656416:0:22249:0:(pinger.c:251:ptlrpc_pinger_process_import()) lustre-MDT0000-lwp-OST0001_UUID->lustre-MDT0000_UUID: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00080000:3.0:1776178722.656419:0:22249:0:(recover.c:57:ptlrpc_initiate_recovery()) lustre-MDT0000_UUID: starting recovery 00000100:00080000:3.0:1776178722.656421:0:22249:0:(import.c:86:import_set_state_nolock()) 00000000dfc8db13 lustre-MDT0000_UUID: changing import state from DISCONN to CONNECTING 00000100:00080000:3.0:1776178722.656424:0:22249:0:(import.c:534:import_select_connection()) lustre-MDT0000-lwp-OST0001: connect to NID 0@lo last attempt 1424 00000100:00080000:3.0:1776178722.656426:0:22249:0:(import.c:581:import_select_connection()) lustre-MDT0000-lwp-OST0001: tried all connections, increasing latency to 21s 00000100:00080000:3.0:1776178722.656430:0:22249:0:(import.c:615:import_select_connection()) lustre-MDT0000-lwp-OST0001: import 00000000dfc8db13 using connection 192.168.204.103@tcp/0@lo 00000100:00100000:3.0:1776178722.656440:0:22249:0:(import.c:807:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5) req@00000000d7e84c97 x1862456998120832/t0(0) o38->lustre-MDT0000-lwp-OST0001@0@lo:12/10 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' 00000100:00080000:3.0:1776178722.656452:0:22249:0:(pinger.c:251:ptlrpc_pinger_process_import()) lustre-MDT0001-mdtlov_UUID->lustre-MDT0000_UUID: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00080000:3.0:1776178722.656455:0:22249:0:(recover.c:57:ptlrpc_initiate_recovery()) lustre-MDT0000_UUID: starting recovery 00000100:00080000:3.0:1776178722.656456:0:22249:0:(import.c:86:import_set_state_nolock()) 00000000ba4f1821 lustre-MDT0000_UUID: changing import state from DISCONN to CONNECTING 00000100:00080000:3.0:1776178722.656459:0:22249:0:(import.c:534:import_select_connection()) lustre-MDT0000-osp-MDT0001: connect to NID 0@lo last attempt 1424 00000100:00080000:3.0:1776178722.656461:0:22249:0:(import.c:581:import_select_connection()) lustre-MDT0000-osp-MDT0001: tried all connections, increasing latency to 21s 00000100:00080000:3.0:1776178722.656464:0:22249:0:(import.c:615:import_select_connection()) lustre-MDT0000-osp-MDT0001: import 00000000ba4f1821 using connection 192.168.204.103@tcp/0@lo 00000100:00100000:2.0:1776178722.656473:0:3352:0:(client.c:1733:ptlrpc_send_new_req()) Sending RPC req@00000000d17c6101 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:lustre-MDT0000-lwp-OST0000_UUID:3352:1862456998120704:0@lo:38:kworker/u8:0.0 00000100:00100000:3.0:1776178722.656473:0:22249:0:(import.c:807:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5) req@00000000b7440871 x1862456998120960/t0(0) o38->lustre-MDT0000-osp-MDT0001@0@lo:24/4 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' 00000100:00080000:3.0:1776178722.656494:0:22249:0:(pinger.c:251:ptlrpc_pinger_process_import()) lustre-MDT0000-lwp-MDT0001_UUID->lustre-MDT0000_UUID: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00080000:3.0:1776178722.656496:0:22249:0:(recover.c:57:ptlrpc_initiate_recovery()) lustre-MDT0000_UUID: starting recovery 00000100:00080000:3.0:1776178722.656497:0:22249:0:(import.c:86:import_set_state_nolock()) 00000000a7c55b1f lustre-MDT0000_UUID: changing import state from DISCONN to CONNECTING 00000100:00080000:3.0:1776178722.656499:0:22249:0:(import.c:534:import_select_connection()) lustre-MDT0000-lwp-MDT0001: connect to NID 0@lo last attempt 1424 00000100:00080000:3.0:1776178722.656501:0:22249:0:(import.c:581:import_select_connection()) lustre-MDT0000-lwp-MDT0001: tried all connections, increasing latency to 21s 00000100:00080000:3.0:1776178722.656503:0:22249:0:(import.c:615:import_select_connection()) lustre-MDT0000-lwp-MDT0001: import 00000000a7c55b1f using connection 192.168.204.103@tcp/0@lo 00000100:00100000:3.0:1776178722.656509:0:22249:0:(import.c:807:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5) req@00000000ab295186 x1862456998121152/t0(0) o38->lustre-MDT0000-lwp-MDT0001@0@lo:12/10 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' 00000100:00100000:2.0:1776178722.656543:0:3352:0:(events.c:361:request_in_callback()) peer: 12345-0@lo (source: 12345-0@lo) 00000100:00100000:3.0:1776178722.656546:0:3354:0:(client.c:1733:ptlrpc_send_new_req()) Sending RPC req@00000000d9d2b484 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_01:lustre-MDT0001-lwp-OST0001_UUID:3354:1862456998120896:0@lo:400:kworker/u8:0.0 00000100:00100000:2.0:1776178722.656570:0:3352:0:(client.c:1733:ptlrpc_send_new_req()) Sending RPC req@00000000d7e84c97 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:lustre-MDT0000-lwp-OST0001_UUID:3352:1862456998120832:0@lo:38:kworker/u8:0.0 00000100:00100000:3.0:1776178722.656573:0:3354:0:(events.c:361:request_in_callback()) peer: 12345-0@lo (source: 12345-0@lo) 00000100:00100000:2.0:1776178722.656589:0:3352:0:(events.c:361:request_in_callback()) peer: 12345-0@lo (source: 12345-0@lo) 00000100:00100000:3.0:1776178722.656595:0:3354:0:(ptlrpcd.c:415:ptlrpcd_check()) transfer 1 async RPCs [0->1] 00000100:00100000:3.0:1776178722.656600:0:3354:0:(client.c:1733:ptlrpc_send_new_req()) Sending RPC req@000000006ce0dfc8 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_01:lustre-MDT0001-lwp-OST0000_UUID:3354:1862456998120768:0@lo:400:kworker/u8:0.0 00000100:00100000:2.0:1776178722.656602:0:3352:0:(client.c:1733:ptlrpc_send_new_req()) Sending RPC req@00000000b7440871 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:lustre-MDT0001-mdtlov_UUID:3352:1862456998120960:0@lo:38:kworker/u8:0.0 00000100:00100000:3.0:1776178722.656619:0:3354:0:(events.c:361:request_in_callback()) peer: 12345-0@lo (source: 12345-0@lo) 00000100:00100000:2.0:1776178722.656620:0:3352:0:(events.c:361:request_in_callback()) peer: 12345-0@lo (source: 12345-0@lo) 00000100:00100000:2.0:1776178722.656631:0:3352:0:(client.c:1733:ptlrpc_send_new_req()) Sending RPC req@00000000ab295186 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:lustre-MDT0000-lwp-MDT0001_UUID:3352:1862456998121152:0@lo:38:kworker/u8:0.0 00000100:00100000:2.0:1776178722.656647:0:3352:0:(events.c:361:request_in_callback()) peer: 12345-0@lo (source: 12345-0@lo) 00000100:00100000:2.0:1776178722.656676:0:11281:0:(service.c:2152:ptlrpc_server_handle_req_in()) got req x1862456998120704 00000100:00100000:2.0:1776178722.656701:0:11281:0:(service.c:2152:ptlrpc_server_handle_req_in()) got req x1862456998120896 00000100:00100000:2.0:1776178722.656718:0:11281:0:(service.c:2152:ptlrpc_server_handle_req_in()) got req x1862456998120832 00000100:00100000:2.0:1776178722.656725:0:11281:0:(service.c:2152:ptlrpc_server_handle_req_in()) got req x1862456998120768 00000100:00100000:2.0:1776178722.656733:0:11281:0:(service.c:2152:ptlrpc_server_handle_req_in()) got req x1862456998121152 00000100:00100000:2.0:1776178722.656743:0:11281:0:(nrs_fifo.c:179:nrs_fifo_req_get()) NRS start fifo request from 12345-0@lo, seq: 1232 00000100:00100000:2.0:1776178722.656748:0:11281:0:(service.c:2324:ptlrpc_server_handle_request()) Handling RPC req@00000000854f8d52 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_004:lustre-MDT0001-lwp-OST0001_UUID+5:3354:x1862456998120896:12345-0@lo:400:kworker/u8:0.0 00000100:00100000:2.0:1776178722.656794:0:11281:0:(service.c:2372:ptlrpc_server_handle_request()) Handled RPC req@00000000854f8d52 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_004:lustre-MDT0001-lwp-OST0001_UUID+5:3354:x1862456998120896:12345-0@lo:400:kworker/u8:0.0 Request processed in 45us (221us total) trans 0 rc 0/0 00000100:00100000:2.0:1776178722.656801:0:11281:0:(nrs_fifo.c:241:nrs_fifo_req_stop()) NRS stop fifo request from 12345-0@lo, seq: 1232 00000100:00100000:3.0:1776178722.656809:0:6257:0:(nrs_fifo.c:179:nrs_fifo_req_get()) NRS start fifo request from 12345-0@lo, seq: 1233 00000100:00100000:3.0:1776178722.656822:0:6257:0:(service.c:2324:ptlrpc_server_handle_request()) Handling RPC req@00000000758405de pname:cluuid+ref:pid:xid:nid:opc:job mdt00_002:lustre-MDT0001-lwp-OST0000_UUID+5:3354:x1862456998120768:12345-0@lo:400:kworker/u8:0.0 00000100:00100000:2.0:1776178722.656827:0:11281:0:(nrs_fifo.c:179:nrs_fifo_req_get()) NRS start fifo request from 12345-0@lo, seq: 7029 00000100:00100000:2.0:1776178722.656830:0:11281:0:(service.c:2324:ptlrpc_server_handle_request()) Handling RPC req@00000000542ce52f pname:cluuid+ref:pid:xid:nid:opc:job mdt00_004:0+-99:3352:x1862456998120704:12345-0@lo:38:kworker/u8:0.0 00010000:02000000:2.0:1776178722.656840:0:11281:0:(ldlm_lib.c:1139:target_handle_connect()) lustre-MDT0000: Not available for connect from 0@lo (stopping) 00000100:00100000:3.0:1776178722.656851:0:6257:0:(service.c:2372:ptlrpc_server_handle_request()) Handled RPC req@00000000758405de pname:cluuid+ref:pid:xid:nid:opc:job mdt00_002:lustre-MDT0001-lwp-OST0000_UUID+5:3354:x1862456998120768:12345-0@lo:400:kworker/u8:0.0 Request processed in 38us (232us total) trans 0 rc 0/0 00000100:00100000:3.0:1776178722.656857:0:6257:0:(nrs_fifo.c:241:nrs_fifo_req_stop()) NRS stop fifo request from 12345-0@lo, seq: 1233 00000100:00100000:2.0:1776178722.656865:0:11281:0:(service.c:2372:ptlrpc_server_handle_request()) Handled RPC req@00000000542ce52f pname:cluuid+ref:pid:xid:nid:opc:job mdt00_004:0+-99:3352:x1862456998120704:12345-0@lo:38:kworker/u8:0.0 Request processed in 35us (325us total) trans 0 rc -19/-19 00000100:00100000:3.0:1776178722.656870:0:6257:0:(nrs_fifo.c:179:nrs_fifo_req_get()) NRS start fifo request from 12345-0@lo, seq: 7030 00000100:00100000:2.0:1776178722.656872:0:11281:0:(nrs_fifo.c:241:nrs_fifo_req_stop()) NRS stop fifo request from 12345-0@lo, seq: 7029 00000100:00100000:3.0:1776178722.656873:0:6257:0:(service.c:2324:ptlrpc_server_handle_request()) Handling RPC req@000000007216887e pname:cluuid+ref:pid:xid:nid:opc:job mdt00_002:0+-99:3352:x1862456998120832:12345-0@lo:38:kworker/u8:0.0 00010000:02000000:3.0:1776178722.656879:0:6257:0:(ldlm_lib.c:1139:target_handle_connect()) lustre-MDT0000: Not available for connect from 0@lo (stopping) 00000100:00100000:2.0:1776178722.656880:0:11281:0:(nrs_fifo.c:179:nrs_fifo_req_get()) NRS start fifo request from 12345-0@lo, seq: 7031 00000100:00100000:2.0:1776178722.656883:0:11281:0:(service.c:2324:ptlrpc_server_handle_request()) Handling RPC req@00000000fd2b8392 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_004:0+-99:3352:x1862456998121152:12345-0@lo:38:kworker/u8:0.0 00010000:02000000:2.0:1776178722.656887:0:11281:0:(ldlm_lib.c:1139:target_handle_connect()) lustre-MDT0000: Not available for connect from 0@lo (stopping) 00000100:00100000:3.0:1776178722.656897:0:6257:0:(service.c:2372:ptlrpc_server_handle_request()) Handled RPC req@000000007216887e pname:cluuid+ref:pid:xid:nid:opc:job mdt00_002:0+-99:3352:x1862456998120832:12345-0@lo:38:kworker/u8:0.0 Request processed in 24us (308us total) trans 0 rc -19/-19 00000100:00100000:2.0:1776178722.656902:0:11281:0:(service.c:2372:ptlrpc_server_handle_request()) Handled RPC req@00000000fd2b8392 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_004:0+-99:3352:x1862456998121152:12345-0@lo:38:kworker/u8:0.0 Request processed in 19us (255us total) trans 0 rc -19/-19 00000100:00100000:3.0:1776178722.656906:0:6257:0:(nrs_fifo.c:241:nrs_fifo_req_stop()) NRS stop fifo request from 12345-0@lo, seq: 7030 00000100:00100000:0.0:1776178722.656960:0:3356:0:(client.c:1733:ptlrpc_send_new_req()) Sending RPC req@00000000eeeaf4ad pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_03:ac61f839-0e8e-4cac-9c59-08bcfbbad330:3356:1862456998120640:0@lo:400:kworker/u8:0.0 00000100:00100000:0.0:1776178722.656992:0:3356:0:(events.c:361:request_in_callback()) peer: 12345-0@lo (source: 12345-0@lo) 00000100:00100000:0.0:1776178722.657009:0:3356:0:(client.c:1733:ptlrpc_send_new_req()) Sending RPC req@000000001a990b02 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_03:lustre-MDT0001-mdtlov_UUID:3356:1862456998121088:0@lo:400:kworker/u8:0.0 00000100:00100000:0.0:1776178722.657032:0:3356:0:(events.c:361:request_in_callback()) peer: 12345-0@lo (source: 12345-0@lo) 00000100:00100000:0.0:1776178722.657048:0:3356:0:(ptlrpcd.c:415:ptlrpcd_check()) transfer 1 async RPCs [2->3] 00000100:00100000:0.0:1776178722.657053:0:3356:0:(client.c:1733:ptlrpc_send_new_req()) Sending RPC req@0000000008c9fb19 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_03:lustre-MDT0001-mdtlov_UUID:3356:1862456998121024:0@lo:400:kworker/u8:0.0 00000100:00100000:0.0:1776178722.657094:0:3356:0:(events.c:361:request_in_callback()) peer: 12345-0@lo (source: 12345-0@lo) 00000100:00100000:0.0:1776178722.657220:0:20762:0:(service.c:2152:ptlrpc_server_handle_req_in()) got req x1862456998121088 00000100:00100000:0.0:1776178722.657245:0:20762:0:(service.c:2152:ptlrpc_server_handle_req_in()) got req x1862456998121024 00000100:00100000:0.0:1776178722.657263:0:20762:0:(nrs_fifo.c:179:nrs_fifo_req_get()) NRS start fifo request from 12345-0@lo, seq: 917 00000100:00100000:0.0:1776178722.657267:0:20762:0:(service.c:2324:ptlrpc_server_handle_request()) Handling RPC req@000000008f94e9dd pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_013:lustre-MDT0001-mdtlov_UUID+5:3356:x1862456998121088:12345-0@lo:400:kworker/u8:0.0 00000100:00100000:0.0:1776178722.657300:0:20762:0:(service.c:2372:ptlrpc_server_handle_request()) Handled RPC req@000000008f94e9dd pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_013:lustre-MDT0001-mdtlov_UUID+5:3356:x1862456998121088:12345-0@lo:400:kworker/u8:0.0 Request processed in 32us (267us total) trans 0 rc 0/0 00000100:00100000:0.0:1776178722.657305:0:20762:0:(nrs_fifo.c:241:nrs_fifo_req_stop()) NRS stop fifo request from 12345-0@lo, seq: 917 00000100:00100000:0.0:1776178722.657317:0:20762:0:(nrs_fifo.c:179:nrs_fifo_req_get()) NRS start fifo request from 12345-0@lo, seq: 918 00000100:00100000:0.0:1776178722.657321:0:20762:0:(service.c:2324:ptlrpc_server_handle_request()) Handling RPC req@000000005c567288 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_013:lustre-MDT0001-mdtlov_UUID+5:3356:x1862456998121024:12345-0@lo:400:kworker/u8:0.0 00000100:00100000:0.0:1776178722.657345:0:20762:0:(service.c:2372:ptlrpc_server_handle_request()) Handled RPC req@000000005c567288 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_013:lustre-MDT0001-mdtlov_UUID+5:3356:x1862456998121024:12345-0@lo:400:kworker/u8:0.0 Request processed in 23us (251us total) trans 0 rc 0/0 00000100:00100000:0.0:1776178722.657350:0:20762:0:(nrs_fifo.c:241:nrs_fifo_req_stop()) NRS stop fifo request from 12345-0@lo, seq: 918 00000100:00100000:0.0:1776178722.657367:0:20140:0:(service.c:2152:ptlrpc_server_handle_req_in()) got req x1862456998120640 00000100:00100000:0.0:1776178722.657388:0:20140:0:(nrs_fifo.c:179:nrs_fifo_req_get()) NRS start fifo request from 12345-0@lo, seq: 358 00000100:00100000:0.0:1776178722.657392:0:20140:0:(service.c:2324:ptlrpc_server_handle_request()) Handling RPC req@0000000035060500 pname:cluuid+ref:pid:xid:nid:opc:job ll_mgs_0002:ac61f839-0e8e-4cac-9c59-08bcfbbad330+18:3356:x1862456998120640:12345-0@lo:400:kworker/u8:0.0 00000100:00100000:0.0:1776178722.657415:0:20140:0:(service.c:2372:ptlrpc_server_handle_request()) Handled RPC req@0000000035060500 pname:cluuid+ref:pid:xid:nid:opc:job ll_mgs_0002:ac61f839-0e8e-4cac-9c59-08bcfbbad330+18:3356:x1862456998120640:12345-0@lo:400:kworker/u8:0.0 Request processed in 22us (423us total) trans 0 rc 0/0 00000100:00100000:0.0:1776178722.657420:0:20140:0:(nrs_fifo.c:241:nrs_fifo_req_stop()) NRS stop fifo request from 12345-0@lo, seq: 358 00000100:00100000:0.0:1776178722.657449:0:14128:0:(service.c:2152:ptlrpc_server_handle_req_in()) got req x1862456998120960 00000100:00100000:0.0:1776178722.657468:0:14128:0:(nrs_fifo.c:179:nrs_fifo_req_get()) NRS start fifo request from 12345-0@lo, seq: 809 00000100:00100000:0.0:1776178722.657471:0:14128:0:(service.c:2324:ptlrpc_server_handle_request()) Handling RPC req@0000000035819321 pname:cluuid+ref:pid:xid:nid:opc:job mdt_out00_002:0+-99:3352:x1862456998120960:12345-0@lo:38:kworker/u8:0.0 00010000:02000000:0.0:1776178722.657478:0:14128:0:(ldlm_lib.c:1139:target_handle_connect()) lustre-MDT0000: Not available for connect from 0@lo (stopping) 00000100:00100000:0.0:1776178722.657500:0:14128:0:(service.c:2372:ptlrpc_server_handle_request()) Handled RPC req@0000000035819321 pname:cluuid+ref:pid:xid:nid:opc:job mdt_out00_002:0+-99:3352:x1862456998120960:12345-0@lo:38:kworker/u8:0.0 Request processed in 28us (879us total) trans 0 rc -19/-19 00000100:00100000:0.0:1776178722.657504:0:14128:0:(nrs_fifo.c:241:nrs_fifo_req_stop()) NRS stop fifo request from 12345-0@lo, seq: 809 00000100:00100000:0.0:1776178722.657546:0:3356:0:(client.c:2215:ptlrpc_check_set()) Completed RPC req@0000000008c9fb19 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_03:lustre-MDT0001-mdtlov_UUID:3356:1862456998121024:0@lo:400:kworker/u8:0.0 00000100:00100000:0.0:1776178722.657555:0:3356:0:(client.c:2215:ptlrpc_check_set()) Completed RPC req@00000000eeeaf4ad pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_03:ac61f839-0e8e-4cac-9c59-08bcfbbad330:3356:1862456998120640:0@lo:400:kworker/u8:0.0 00000100:00100000:0.0:1776178722.657562:0:3356:0:(client.c:2215:ptlrpc_check_set()) Completed RPC req@000000001a990b02 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_03:lustre-MDT0001-mdtlov_UUID:3356:1862456998121088:0@lo:400:kworker/u8:0.0 00000100:00100000:3.0:1776178722.657773:0:3354:0:(client.c:2215:ptlrpc_check_set()) Completed RPC req@000000006ce0dfc8 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_01:lustre-MDT0001-lwp-OST0000_UUID:3354:1862456998120768:0@lo:400:kworker/u8:0.0 00000100:00100000:3.0:1776178722.657783:0:3354:0:(client.c:2215:ptlrpc_check_set()) Completed RPC req@00000000d9d2b484 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_01:lustre-MDT0001-lwp-OST0001_UUID:3354:1862456998120896:0@lo:400:kworker/u8:0.0 00000100:00080000:3.0:1776178722.657835:0:3352:0:(import.c:86:import_set_state_nolock()) 00000000ba4f1821 lustre-MDT0000_UUID: changing import state from CONNECTING to DISCONN 00000100:00080000:3.0:1776178722.657839:0:3352:0:(import.c:1429:ptlrpc_connect_interpret()) recovery of lustre-MDT0000_UUID on 192.168.204.103@tcp failed (-19) 00000100:00100000:3.0:1776178722.657842:0:3352:0:(client.c:2215:ptlrpc_check_set()) Completed RPC req@00000000b7440871 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:lustre-MDT0001-mdtlov_UUID:3352:1862456998120960:0@lo:38:kworker/u8:0.0 00000100:00080000:3.0:1776178722.657851:0:3352:0:(import.c:86:import_set_state_nolock()) 00000000a7c55b1f lustre-MDT0000_UUID: changing import state from CONNECTING to DISCONN 00000100:00080000:3.0:1776178722.657853:0:3352:0:(import.c:1429:ptlrpc_connect_interpret()) recovery of lustre-MDT0000_UUID on 192.168.204.103@tcp failed (-19) 00000100:00100000:3.0:1776178722.657856:0:3352:0:(client.c:2215:ptlrpc_check_set()) Completed RPC req@00000000ab295186 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:lustre-MDT0000-lwp-MDT0001_UUID:3352:1862456998121152:0@lo:38:kworker/u8:0.0 00000100:00080000:3.0:1776178722.657863:0:3352:0:(import.c:86:import_set_state_nolock()) 0000000047601a98 lustre-MDT0000_UUID: changing import state from CONNECTING to DISCONN 00000100:00080000:3.0:1776178722.657866:0:3352:0:(import.c:1429:ptlrpc_connect_interpret()) recovery of lustre-MDT0000_UUID on 192.168.204.103@tcp failed (-19) 00000100:00100000:3.0:1776178722.657868:0:3352:0:(client.c:2215:ptlrpc_check_set()) Completed RPC req@00000000d17c6101 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:lustre-MDT0000-lwp-OST0000_UUID:3352:1862456998120704:0@lo:38:kworker/u8:0.0 00000100:00080000:3.0:1776178722.657875:0:3352:0:(import.c:86:import_set_state_nolock()) 00000000dfc8db13 lustre-MDT0000_UUID: changing import state from CONNECTING to DISCONN 00000100:00080000:3.0:1776178722.657877:0:3352:0:(import.c:1429:ptlrpc_connect_interpret()) recovery of lustre-MDT0000_UUID on 192.168.204.103@tcp failed (-19) 00000100:00100000:3.0:1776178722.657879:0:3352:0:(client.c:2215:ptlrpc_check_set()) Completed RPC req@00000000d7e84c97 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:lustre-MDT0000-lwp-OST0001_UUID:3352:1862456998120832:0@lo:38:kworker/u8:0.0 00000100:00100000:2.0:1776178722.661576:0:11281:0:(nrs_fifo.c:241:nrs_fifo_req_stop()) NRS stop fifo request from 12345-0@lo, seq: 7031 00010000:00010000:2.0:1776178723.296454:0:9224:0:(ldlm_lockd.c:2792:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:1.0:1776178724.192518:0:3353:0:(client.c:1733:ptlrpc_send_new_req()) Sending RPC req@0000000042de09e9 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_00:lustre-MDT0001-mdtlov_UUID:3353:1862456998121216:0@lo:13:osp-pre-1-1.0 00000100:00100000:1.0:1776178724.192600:0:3353:0:(events.c:361:request_in_callback()) peer: 12345-0@lo (source: 12345-0@lo) 00000100:00100000:2.0:1776178724.192702:0:8231:0:(service.c:2152:ptlrpc_server_handle_req_in()) got req x1862456998121216 00000100:00100000:2.0:1776178724.192755:0:8231:0:(nrs_fifo.c:179:nrs_fifo_req_get()) NRS start fifo request from 12345-0@lo, seq: 691 00000100:00100000:2.0:1776178724.192761:0:8231:0:(service.c:2324:ptlrpc_server_handle_request()) Handling RPC req@00000000b1db1835 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_create00:lustre-MDT0001-mdtlov_UUID+5:3353:x1862456998121216:12345-0@lo:13:osp-pre-1-1.0 00002000:00000024:2.0:1776178724.192801:0:8231:0:(ofd_obd.c:740:ofd_statfs()) blocks cached 229376 granted 491746368 pending 0 free 3923689472 avail 3692167168 00000100:00100000:2.0:1776178724.192845:0:8231:0:(service.c:2372:ptlrpc_server_handle_request()) Handled RPC req@00000000b1db1835 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_create00:lustre-MDT0001-mdtlov_UUID+5:3353:x1862456998121216:12345-0@lo:13:osp-pre-1-1.0 Request processed in 84us (247us total) trans 0 rc 0/0 00000100:00100000:2.0:1776178724.192853:0:8231:0:(nrs_fifo.c:241:nrs_fifo_req_stop()) NRS stop fifo request from 12345-0@lo, seq: 691 00000100:00100000:1.0:1776178724.192985:0:3353:0:(client.c:2215:ptlrpc_check_set()) Completed RPC req@0000000042de09e9 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_00:lustre-MDT0001-mdtlov_UUID:3353:1862456998121216:0@lo:13:osp-pre-1-1.0 00010000:00010000:1.0:1776178724.320315:0:9225:0:(ldlm_lockd.c:2792:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:1.0:1776178724.707113:0:3353:0:(ptlrpcd.c:415:ptlrpcd_check()) transfer 1 async RPCs [1->0] 00000100:00100000:1.0:1776178724.707132:0:3353:0:(client.c:1733:ptlrpc_send_new_req()) Sending RPC req@0000000004dd3b9b pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_00:lustre-MDT0001-mdtlov_UUID:3353:1862456998121280:0@lo:13:osp-pre-0-1.0 00000100:00100000:1.0:1776178724.707193:0:3353:0:(events.c:361:request_in_callback()) peer: 12345-0@lo (source: 12345-0@lo) 00000100:00100000:0.0:1776178724.707529:0:9554:0:(service.c:2152:ptlrpc_server_handle_req_in()) got req x1862456998121280 00000100:00100000:0.0:1776178724.709464:0:9554:0:(nrs_fifo.c:179:nrs_fifo_req_get()) NRS start fifo request from 12345-0@lo, seq: 692 00000100:00100000:0.0:1776178724.709477:0:9554:0:(service.c:2324:ptlrpc_server_handle_request()) Handling RPC req@00000000664eb80b pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_create00:lustre-MDT0001-mdtlov_UUID+5:3353:x1862456998121280:12345-0@lo:13:osp-pre-0-1.0 00002000:00000024:0.0:1776178724.709519:0:9554:0:(ofd_obd.c:740:ofd_statfs()) blocks cached 315392 granted 484883776 pending 0 free 3923689472 avail 3692167168 00000100:00100000:0.0:1776178724.709611:0:9554:0:(service.c:2372:ptlrpc_server_handle_request()) Handled RPC req@00000000664eb80b pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_create00:lustre-MDT0001-mdtlov_UUID+5:3353:x1862456998121280:12345-0@lo:13:osp-pre-0-1.0 Request processed in 135us (2416us total) trans 0 rc 0/0 00000100:00100000:0.0:1776178724.709618:0:9554:0:(nrs_fifo.c:241:nrs_fifo_req_stop()) NRS stop fifo request from 12345-0@lo, seq: 692 00000100:00100000:1.0:1776178724.709787:0:3353:0:(client.c:2215:ptlrpc_check_set()) Completed RPC req@0000000004dd3b9b pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_00:lustre-MDT0001-mdtlov_UUID:3353:1862456998121280:0@lo:13:osp-pre-0-1.0 00010000:00010000:3.0:1776178725.348301:0:6244:0:(ldlm_lockd.c:2792:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:3.0:1776178726.187108:0:3347:0:(events.c:361:request_in_callback()) peer: 12345-192.168.204.3@tcp (source: 12345-192.168.204.3@tcp) 00000100:00100000:3.0:1776178726.187151:0:3347:0:(events.c:361:request_in_callback()) peer: 12345-192.168.204.3@tcp (source: 12345-192.168.204.3@tcp) 00000100:00100000:3.0:1776178726.187184:0:3347:0:(events.c:361:request_in_callback()) peer: 12345-192.168.204.3@tcp (source: 12345-192.168.204.3@tcp) 00000100:00100000:2.0:1776178726.187202:0:11281:0:(service.c:2152:ptlrpc_server_handle_req_in()) got req x1862456979141760 00000100:00100000:3.0:1776178726.187207:0:3347:0:(events.c:361:request_in_callback()) peer: 12345-192.168.204.3@tcp (source: 12345-192.168.204.3@tcp) 00000100:00100000:3.0:1776178726.187240:0:3347:0:(events.c:361:request_in_callback()) peer: 12345-192.168.204.3@tcp (source: 12345-192.168.204.3@tcp) 00000100:00100000:2.0:1776178726.187253:0:11281:0:(service.c:2152:ptlrpc_server_handle_req_in()) got req x1862456979141696 00000100:00100000:3.0:1776178726.187272:0:20759:0:(service.c:2152:ptlrpc_server_handle_req_in()) got req x1862456979141888 00000100:00100000:2.0:1776178726.187276:0:11281:0:(nrs_fifo.c:179:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.204.3@tcp, seq: 1234 00000100:00100000:2.0:1776178726.187283:0:11281:0:(service.c:2324:ptlrpc_server_handle_request()) Handling RPC req@00000000d7e84c97 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_004:b50963bd-8eba-4432-9a54-8888d00e3fa1+6:2270:x1862456979141760:12345-192.168.204.3@tcp:400:kworker/u8:3.0 00000100:00100000:3.0:1776178726.187303:0:20759:0:(service.c:2152:ptlrpc_server_handle_req_in()) got req x1862456979141824 00000100:00100000:3.0:1776178726.187324:0:20759:0:(nrs_fifo.c:179:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.204.3@tcp, seq: 919 00000100:00100000:3.0:1776178726.187328:0:20759:0:(service.c:2324:ptlrpc_server_handle_request()) Handling RPC req@00000000d17c6101 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_010:b50963bd-8eba-4432-9a54-8888d00e3fa1+9:2270:x1862456979141888:12345-192.168.204.3@tcp:400:kworker/u8:3.0 00000100:00100000:2.0:1776178726.187356:0:11281:0:(service.c:2372:ptlrpc_server_handle_request()) Handled RPC req@00000000d7e84c97 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_004:b50963bd-8eba-4432-9a54-8888d00e3fa1+6:2270:x1862456979141760:12345-192.168.204.3@tcp:400:kworker/u8:3.0 Request processed in 73us (304us total) trans 0 rc 0/0 00000100:00100000:2.0:1776178726.187365:0:11281:0:(nrs_fifo.c:241:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.204.3@tcp, seq: 1234 00000100:00100000:3.0:1776178726.187369:0:20759:0:(service.c:2372:ptlrpc_server_handle_request()) Handled RPC req@00000000d17c6101 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_010:b50963bd-8eba-4432-9a54-8888d00e3fa1+9:2270:x1862456979141888:12345-192.168.204.3@tcp:400:kworker/u8:3.0 Request processed in 38us (216us total) trans 0 rc 0/0 00000100:00100000:3.0:1776178726.187378:0:20759:0:(nrs_fifo.c:241:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.204.3@tcp, seq: 919 00000100:00100000:3.0:1776178726.187394:0:20759:0:(nrs_fifo.c:179:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.204.3@tcp, seq: 920 00000100:00100000:2.0:1776178726.187398:0:11281:0:(nrs_fifo.c:179:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.204.3@tcp, seq: 7032 00000100:00100000:3.0:1776178726.187401:0:20759:0:(service.c:2324:ptlrpc_server_handle_request()) Handling RPC req@00000000ab295186 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_010:b50963bd-8eba-4432-9a54-8888d00e3fa1+8:2270:x1862456979141824:12345-192.168.204.3@tcp:400:kworker/u8:3.0 00000100:00100000:2.0:1776178726.187408:0:11281:0:(service.c:2324:ptlrpc_server_handle_request()) Handling RPC req@00000000b7440871 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_004:0+-99:2269:x1862456979141696:12345-192.168.204.3@tcp:38:kworker/u8:3.0 00010000:02000000:2.0:1776178726.187422:0:11281:0:(ldlm_lib.c:1139:target_handle_connect()) lustre-MDT0000: Not available for connect from 192.168.204.3@tcp (stopping) 00000100:00100000:3.0:1776178726.187428:0:20759:0:(service.c:2372:ptlrpc_server_handle_request()) Handled RPC req@00000000ab295186 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_010:b50963bd-8eba-4432-9a54-8888d00e3fa1+8:2270:x1862456979141824:12345-192.168.204.3@tcp:400:kworker/u8:3.0 Request processed in 28us (222us total) trans 0 rc 0/0 00000100:00100000:3.0:1776178726.187436:0:20759:0:(nrs_fifo.c:241:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.204.3@tcp, seq: 920 00000100:00100000:2.0:1776178726.187454:0:11281:0:(service.c:2372:ptlrpc_server_handle_request()) Handled RPC req@00000000b7440871 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_004:0+-99:2269:x1862456979141696:12345-192.168.204.3@tcp:38:kworker/u8:3.0 Request processed in 46us (213us total) trans 0 rc -19/-19 00000100:00100000:2.0:1776178726.187461:0:11281:0:(nrs_fifo.c:241:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.204.3@tcp, seq: 7032 00000100:00100000:0.0:1776178726.188533:0:20137:0:(service.c:2152:ptlrpc_server_handle_req_in()) got req x1862456979141632 00000100:00100000:0.0:1776178726.188601:0:20137:0:(nrs_fifo.c:179:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.204.3@tcp, seq: 359 00000100:00100000:0.0:1776178726.188606:0:20137:0:(service.c:2324:ptlrpc_server_handle_request()) Handling RPC req@00000000f1e6b6f9 pname:cluuid+ref:pid:xid:nid:opc:job ll_mgs_0000:5b5c66b1-1a2f-4651-8548-0c75b9a38a94+9:2270:x1862456979141632:12345-192.168.204.3@tcp:400:kworker/u8:3.0 00000100:00100000:0.0:1776178726.188665:0:20137:0:(service.c:2372:ptlrpc_server_handle_request()) Handled RPC req@00000000f1e6b6f9 pname:cluuid+ref:pid:xid:nid:opc:job ll_mgs_0000:5b5c66b1-1a2f-4651-8548-0c75b9a38a94+9:2270:x1862456979141632:12345-192.168.204.3@tcp:400:kworker/u8:3.0 Request processed in 58us (1483us total) trans 0 rc 0/0 00000100:00100000:0.0:1776178726.188673:0:20137:0:(nrs_fifo.c:241:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.204.3@tcp, seq: 359 00010000:00010000:1.0:1776178726.368940:0:6245:0:(ldlm_lockd.c:2792:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00010000:00010000:2.0:1776178727.394091:0:9224:0:(ldlm_lockd.c:2792:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00080000:1.0:1776178727.780203:0:22249:0:(pinger.c:251:ptlrpc_pinger_process_import()) lustre-MDT0000-lwp-OST0000_UUID->lustre-MDT0000_UUID: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00080000:1.0:1776178727.780212:0:22249:0:(recover.c:57:ptlrpc_initiate_recovery()) lustre-MDT0000_UUID: starting recovery 00000100:00080000:1.0:1776178727.780215:0:22249:0:(import.c:86:import_set_state_nolock()) 0000000047601a98 lustre-MDT0000_UUID: changing import state from DISCONN to CONNECTING 00000100:00080000:1.0:1776178727.780225:0:22249:0:(import.c:534:import_select_connection()) lustre-MDT0000-lwp-OST0000: connect to NID 0@lo last attempt 1429 00000100:00080000:1.0:1776178727.780229:0:22249:0:(import.c:581:import_select_connection()) lustre-MDT0000-lwp-OST0000: tried all connections, increasing latency to 21s 00000100:00080000:1.0:1776178727.780235:0:22249:0:(import.c:615:import_select_connection()) lustre-MDT0000-lwp-OST0000: import 0000000047601a98 using connection 192.168.204.103@tcp/0@lo 00000100:00100000:1.0:1776178727.780255:0:22249:0:(import.c:807:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5) req@00000000b85458e9 x1862456998121408/t0(0) o38->lustre-MDT0000-lwp-OST0000@0@lo:12/10 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' 00000100:00080000:1.0:1776178727.780289:0:22249:0:(pinger.c:251:ptlrpc_pinger_process_import()) lustre-MDT0000-lwp-OST0001_UUID->lustre-MDT0000_UUID: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00080000:1.0:1776178727.780291:0:22249:0:(recover.c:57:ptlrpc_initiate_recovery()) lustre-MDT0000_UUID: starting recovery 00000100:00080000:1.0:1776178727.780293:0:22249:0:(import.c:86:import_set_state_nolock()) 00000000dfc8db13 lustre-MDT0000_UUID: changing import state from DISCONN to CONNECTING 00000100:00080000:1.0:1776178727.780295:0:22249:0:(import.c:534:import_select_connection()) lustre-MDT0000-lwp-OST0001: connect to NID 0@lo last attempt 1429 00000100:00080000:1.0:1776178727.780297:0:22249:0:(import.c:581:import_select_connection()) lustre-MDT0000-lwp-OST0001: tried all connections, increasing latency to 21s 00000100:00080000:1.0:1776178727.780300:0:22249:0:(import.c:615:import_select_connection()) lustre-MDT0000-lwp-OST0001: import 00000000dfc8db13 using connection 192.168.204.103@tcp/0@lo 00000100:00100000:2.0:1776178727.780353:0:3352:0:(client.c:1733:ptlrpc_send_new_req()) Sending RPC req@00000000b85458e9 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:lustre-MDT0000-lwp-OST0000_UUID:3352:1862456998121408:0@lo:38:kworker/u8:0.0 00000100:00100000:1.0:1776178727.780378:0:22249:0:(import.c:807:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5) req@00000000f433e404 x1862456998121536/t0(0) o38->lustre-MDT0000-lwp-OST0001@0@lo:12/10 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' 00000100:00080000:1.0:1776178727.780411:0:22249:0:(pinger.c:251:ptlrpc_pinger_process_import()) lustre-MDT0001-mdtlov_UUID->lustre-MDT0000_UUID: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00080000:1.0:1776178727.780415:0:22249:0:(recover.c:57:ptlrpc_initiate_recovery()) lustre-MDT0000_UUID: starting recovery 00000100:00080000:1.0:1776178727.780417:0:22249:0:(import.c:86:import_set_state_nolock()) 00000000ba4f1821 lustre-MDT0000_UUID: changing import state from DISCONN to CONNECTING 00000100:00080000:1.0:1776178727.780420:0:22249:0:(import.c:534:import_select_connection()) lustre-MDT0000-osp-MDT0001: connect to NID 0@lo last attempt 1429 00000100:00080000:1.0:1776178727.780422:0:22249:0:(import.c:581:import_select_connection()) lustre-MDT0000-osp-MDT0001: tried all connections, increasing latency to 21s 00000100:00080000:1.0:1776178727.780425:0:22249:0:(import.c:615:import_select_connection()) lustre-MDT0000-osp-MDT0001: import 00000000ba4f1821 using connection 192.168.204.103@tcp/0@lo 00000100:00100000:2.0:1776178727.780435:0:3352:0:(events.c:361:request_in_callback()) peer: 12345-0@lo (source: 12345-0@lo) 00000100:00100000:1.0:1776178727.780442:0:22249:0:(import.c:807:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5) req@00000000fab910b9 x1862456998121664/t0(0) o38->lustre-MDT0000-osp-MDT0001@0@lo:24/4 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' 00000100:00080000:1.0:1776178727.780465:0:22249:0:(pinger.c:251:ptlrpc_pinger_process_import()) lustre-MDT0000-lwp-MDT0001_UUID->lustre-MDT0000_UUID: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00080000:1.0:1776178727.780469:0:22249:0:(recover.c:57:ptlrpc_initiate_recovery()) lustre-MDT0000_UUID: starting recovery 00000100:00080000:1.0:1776178727.780470:0:22249:0:(import.c:86:import_set_state_nolock()) 00000000a7c55b1f lustre-MDT0000_UUID: changing import state from DISCONN to CONNECTING 00000100:00080000:1.0:1776178727.780473:0:22249:0:(import.c:534:import_select_connection()) lustre-MDT0000-lwp-MDT0001: connect to NID 0@lo last attempt 1429 00000100:00080000:1.0:1776178727.780475:0:22249:0:(import.c:581:import_select_connection()) lustre-MDT0000-lwp-MDT0001: tried all connections, increasing latency to 21s 00000100:00080000:1.0:1776178727.780478:0:22249:0:(import.c:615:import_select_connection()) lustre-MDT0000-lwp-MDT0001: import 00000000a7c55b1f using connection 192.168.204.103@tcp/0@lo 00000100:00100000:2.0:1776178727.780482:0:3352:0:(client.c:1733:ptlrpc_send_new_req()) Sending RPC req@00000000f433e404 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:lustre-MDT0000-lwp-OST0001_UUID:3352:1862456998121536:0@lo:38:kworker/u8:0.0 00000100:00100000:1.0:1776178727.780505:0:22249:0:(import.c:807:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5) req@00000000a0c64d21 x1862456998121856/t0(0) o38->lustre-MDT0000-lwp-MDT0001@0@lo:12/10 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' 00000100:00100000:2.0:1776178727.780513:0:3352:0:(events.c:361:request_in_callback()) peer: 12345-0@lo (source: 12345-0@lo) 00000100:00100000:2.0:1776178727.780533:0:3352:0:(client.c:1733:ptlrpc_send_new_req()) Sending RPC req@00000000fab910b9 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:lustre-MDT0001-mdtlov_UUID:3352:1862456998121664:0@lo:38:kworker/u8:0.0 00000100:00100000:1.0:1776178727.780545:0:3353:0:(client.c:1733:ptlrpc_send_new_req()) Sending RPC req@0000000015f283a6 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_00:lustre-MDT0001-lwp-OST0001_UUID:3353:1862456998121600:0@lo:400:kworker/u8:0.0 00000100:00100000:2.0:1776178727.780562:0:3352:0:(events.c:361:request_in_callback()) peer: 12345-0@lo (source: 12345-0@lo) 00000100:00100000:1.0:1776178727.780587:0:3353:0:(events.c:361:request_in_callback()) peer: 12345-0@lo (source: 12345-0@lo) 00000100:00100000:2.0:1776178727.780589:0:3352:0:(client.c:1733:ptlrpc_send_new_req()) Sending RPC req@00000000a0c64d21 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:lustre-MDT0000-lwp-MDT0001_UUID:3352:1862456998121856:0@lo:38:kworker/u8:0.0 00000100:00100000:1.0:1776178727.780613:0:3353:0:(ptlrpcd.c:415:ptlrpcd_check()) transfer 1 async RPCs [1->0] 00000100:00100000:1.0:1776178727.780621:0:3353:0:(client.c:1733:ptlrpc_send_new_req()) Sending RPC req@00000000ca8b81ba pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_00:lustre-MDT0001-mdtlov_UUID:3353:1862456998121728:0@lo:400:kworker/u8:0.0 00000100:00100000:2.0:1776178727.780633:0:3352:0:(events.c:361:request_in_callback()) peer: 12345-0@lo (source: 12345-0@lo) 00000100:00100000:1.0:1776178727.780659:0:3353:0:(events.c:361:request_in_callback()) peer: 12345-0@lo (source: 12345-0@lo) 00000100:00100000:2.0:1776178727.780690:0:8250:0:(service.c:2152:ptlrpc_server_handle_req_in()) got req x1862456998121408 00000100:00100000:1.0:1776178727.780703:0:6255:0:(service.c:2152:ptlrpc_server_handle_req_in()) got req x1862456998121536 00000100:00100000:2.0:1776178727.780739:0:8250:0:(service.c:2152:ptlrpc_server_handle_req_in()) got req x1862456998121600 00000100:00100000:1.0:1776178727.780741:0:6255:0:(service.c:2152:ptlrpc_server_handle_req_in()) got req x1862456998121856 00000100:00100000:1.0:1776178727.780758:0:6255:0:(nrs_fifo.c:179:nrs_fifo_req_get()) NRS start fifo request from 12345-0@lo, seq: 7033 00000100:00100000:1.0:1776178727.780764:0:6255:0:(service.c:2324:ptlrpc_server_handle_request()) Handling RPC req@000000006a3d91b0 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_000:0+-99:3352:x1862456998121408:12345-0@lo:38:kworker/u8:0.0 00000100:00100000:2.0:1776178727.780770:0:8250:0:(nrs_fifo.c:179:nrs_fifo_req_get()) NRS start fifo request from 12345-0@lo, seq: 1235 00000100:00100000:2.0:1776178727.780775:0:8250:0:(service.c:2324:ptlrpc_server_handle_request()) Handling RPC req@000000000b5ad4fd pname:cluuid+ref:pid:xid:nid:opc:job mdt00_003:lustre-MDT0001-lwp-OST0001_UUID+5:3353:x1862456998121600:12345-0@lo:400:kworker/u8:0.0 00010000:02000000:1.0:1776178727.780778:0:6255:0:(ldlm_lib.c:1139:target_handle_connect()) lustre-MDT0000: Not available for connect from 0@lo (stopping) 00000100:00100000:1.0:1776178727.780822:0:6255:0:(service.c:2372:ptlrpc_server_handle_request()) Handled RPC req@000000006a3d91b0 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_000:0+-99:3352:x1862456998121408:12345-0@lo:38:kworker/u8:0.0 Request processed in 58us (390us total) trans 0 rc -19/-19 00000100:00100000:2.0:1776178727.780830:0:8250:0:(service.c:2372:ptlrpc_server_handle_request()) Handled RPC req@000000000b5ad4fd pname:cluuid+ref:pid:xid:nid:opc:job mdt00_003:lustre-MDT0001-lwp-OST0001_UUID+5:3353:x1862456998121600:12345-0@lo:400:kworker/u8:0.0 Request processed in 52us (243us total) trans 0 rc 0/0 00000100:00100000:1.0:1776178727.780832:0:6255:0:(nrs_fifo.c:241:nrs_fifo_req_stop()) NRS stop fifo request from 12345-0@lo, seq: 7033 00000100:00100000:2.0:1776178727.780840:0:8250:0:(nrs_fifo.c:241:nrs_fifo_req_stop()) NRS stop fifo request from 12345-0@lo, seq: 1235 00000100:00100000:1.0:1776178727.780856:0:6255:0:(nrs_fifo.c:179:nrs_fifo_req_get()) NRS start fifo request from 12345-0@lo, seq: 7034 00000100:00100000:1.0:1776178727.780865:0:6255:0:(service.c:2324:ptlrpc_server_handle_request()) Handling RPC req@0000000012b22334 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_000:0+-99:3352:x1862456998121536:12345-0@lo:38:kworker/u8:0.0 00000100:00100000:2.0:1776178727.780865:0:8250:0:(nrs_fifo.c:179:nrs_fifo_req_get()) NRS start fifo request from 12345-0@lo, seq: 7035 00000100:00100000:2.0:1776178727.780873:0:8250:0:(service.c:2324:ptlrpc_server_handle_request()) Handling RPC req@00000000ea88a302 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_003:0+-99:3352:x1862456998121856:12345-0@lo:38:kworker/u8:0.0 00010000:02000000:1.0:1776178727.780876:0:6255:0:(ldlm_lib.c:1139:target_handle_connect()) lustre-MDT0000: Not available for connect from 0@lo (stopping) 00010000:02000000:2.0:1776178727.780885:0:8250:0:(ldlm_lib.c:1139:target_handle_connect()) lustre-MDT0000: Not available for connect from 0@lo (stopping) 00000100:00100000:1.0:1776178727.780905:0:6255:0:(service.c:2372:ptlrpc_server_handle_request()) Handled RPC req@0000000012b22334 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_000:0+-99:3352:x1862456998121536:12345-0@lo:38:kworker/u8:0.0 Request processed in 43us (395us total) trans 0 rc -19/-19 00000100:00100000:1.0:1776178727.780914:0:6255:0:(nrs_fifo.c:241:nrs_fifo_req_stop()) NRS stop fifo request from 12345-0@lo, seq: 7034 00000100:00100000:2.0:1776178727.780924:0:8250:0:(service.c:2372:ptlrpc_server_handle_request()) Handled RPC req@00000000ea88a302 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_003:0+-99:3352:x1862456998121856:12345-0@lo:38:kworker/u8:0.0 Request processed in 50us (290us total) trans 0 rc -19/-19 00000100:00100000:2.0:1776178727.780932:0:8250:0:(nrs_fifo.c:241:nrs_fifo_req_stop()) NRS stop fifo request from 12345-0@lo, seq: 7035 00000100:00100000:1.0:1776178727.780945:0:14440:0:(service.c:2152:ptlrpc_server_handle_req_in()) got req x1862456998121728 00000100:00100000:2.0:1776178727.780959:0:15213:0:(service.c:2152:ptlrpc_server_handle_req_in()) got req x1862456998121664 00000100:00100000:1.0:1776178727.780985:0:14440:0:(nrs_fifo.c:179:nrs_fifo_req_get()) NRS start fifo request from 12345-0@lo, seq: 921 00000100:00100000:2.0:1776178727.780990:0:15213:0:(nrs_fifo.c:179:nrs_fifo_req_get()) NRS start fifo request from 12345-0@lo, seq: 810 00000100:00100000:1.0:1776178727.780993:0:14440:0:(service.c:2324:ptlrpc_server_handle_request()) Handling RPC req@0000000083b11d73 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_004:lustre-MDT0001-mdtlov_UUID+5:3353:x1862456998121728:12345-0@lo:400:kworker/u8:0.0 00000100:00100000:2.0:1776178727.780999:0:15213:0:(service.c:2324:ptlrpc_server_handle_request()) Handling RPC req@000000004828c19d pname:cluuid+ref:pid:xid:nid:opc:job mdt_out00_003:0+-99:3352:x1862456998121664:12345-0@lo:38:kworker/u8:0.0 00010000:02000000:2.0:1776178727.781009:0:15213:0:(ldlm_lib.c:1139:target_handle_connect()) lustre-MDT0000: Not available for connect from 0@lo (stopping) 00000100:00100000:1.0:1776178727.781034:0:14440:0:(service.c:2372:ptlrpc_server_handle_request()) Handled RPC req@0000000083b11d73 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_004:lustre-MDT0001-mdtlov_UUID+5:3353:x1862456998121728:12345-0@lo:400:kworker/u8:0.0 Request processed in 42us (375us total) trans 0 rc 0/0 00000100:00100000:1.0:1776178727.781042:0:14440:0:(nrs_fifo.c:241:nrs_fifo_req_stop()) NRS stop fifo request from 12345-0@lo, seq: 921 00000100:00100000:2.0:1776178727.781093:0:15213:0:(service.c:2372:ptlrpc_server_handle_request()) Handled RPC req@000000004828c19d pname:cluuid+ref:pid:xid:nid:opc:job mdt_out00_003:0+-99:3352:x1862456998121664:12345-0@lo:38:kworker/u8:0.0 Request processed in 60us (497us total) trans 0 rc -19/-19 00000100:00100000:2.0:1776178727.781101:0:15213:0:(nrs_fifo.c:241:nrs_fifo_req_stop()) NRS stop fifo request from 12345-0@lo, seq: 810 00000100:00100000:1.0:1776178727.781113:0:3353:0:(client.c:2215:ptlrpc_check_set()) Completed RPC req@00000000ca8b81ba pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_00:lustre-MDT0001-mdtlov_UUID:3353:1862456998121728:0@lo:400:kworker/u8:0.0 00000100:00100000:1.0:1776178727.781129:0:3353:0:(client.c:2215:ptlrpc_check_set()) Completed RPC req@0000000015f283a6 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_00:lustre-MDT0001-lwp-OST0001_UUID:3353:1862456998121600:0@lo:400:kworker/u8:0.0 00000100:00080000:2.0:1776178727.781146:0:3352:0:(import.c:86:import_set_state_nolock()) 00000000a7c55b1f lustre-MDT0000_UUID: changing import state from CONNECTING to DISCONN 00000100:00080000:2.0:1776178727.781151:0:3352:0:(import.c:1429:ptlrpc_connect_interpret()) recovery of lustre-MDT0000_UUID on 192.168.204.103@tcp failed (-19) 00000100:00100000:2.0:1776178727.781155:0:3352:0:(client.c:2215:ptlrpc_check_set()) Completed RPC req@00000000a0c64d21 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:lustre-MDT0000-lwp-MDT0001_UUID:3352:1862456998121856:0@lo:38:kworker/u8:0.0 00000100:00080000:2.0:1776178727.781163:0:3352:0:(import.c:86:import_set_state_nolock()) 00000000dfc8db13 lustre-MDT0000_UUID: changing import state from CONNECTING to DISCONN 00000100:00080000:2.0:1776178727.781166:0:3352:0:(import.c:1429:ptlrpc_connect_interpret()) recovery of lustre-MDT0000_UUID on 192.168.204.103@tcp failed (-19) 00000100:00100000:2.0:1776178727.781168:0:3352:0:(client.c:2215:ptlrpc_check_set()) Completed RPC req@00000000f433e404 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:lustre-MDT0000-lwp-OST0001_UUID:3352:1862456998121536:0@lo:38:kworker/u8:0.0 00000100:00080000:2.0:1776178727.781175:0:3352:0:(import.c:86:import_set_state_nolock()) 00000000ba4f1821 lustre-MDT0000_UUID: changing import state from CONNECTING to DISCONN 00000100:00080000:2.0:1776178727.781177:0:3352:0:(import.c:1429:ptlrpc_connect_interpret()) recovery of lustre-MDT0000_UUID on 192.168.204.103@tcp failed (-19) 00000100:00100000:2.0:1776178727.781179:0:3352:0:(client.c:2215:ptlrpc_check_set()) Completed RPC req@00000000fab910b9 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:lustre-MDT0001-mdtlov_UUID:3352:1862456998121664:0@lo:38:kworker/u8:0.0 00000100:00080000:2.0:1776178727.781187:0:3352:0:(import.c:86:import_set_state_nolock()) 0000000047601a98 lustre-MDT0000_UUID: changing import state from CONNECTING to DISCONN 00000100:00080000:2.0:1776178727.781189:0:3352:0:(import.c:1429:ptlrpc_connect_interpret()) recovery of lustre-MDT0000_UUID on 192.168.204.103@tcp failed (-19) 00000100:00100000:2.0:1776178727.781190:0:3352:0:(client.c:2215:ptlrpc_check_set()) Completed RPC req@00000000b85458e9 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:lustre-MDT0000-lwp-OST0000_UUID:3352:1862456998121408:0@lo:38:kworker/u8:0.0 00000100:00100000:3.0:1776178727.783578:0:3355:0:(client.c:1733:ptlrpc_send_new_req()) Sending RPC req@000000002c6408bc pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_02:ac61f839-0e8e-4cac-9c59-08bcfbbad330:3355:1862456998121344:0@lo:400:kworker/u8:0.0 00000100:00100000:3.0:1776178727.783676:0:3355:0:(events.c:361:request_in_callback()) peer: 12345-0@lo (source: 12345-0@lo) 00000100:00100000:3.0:1776178727.783712:0:3355:0:(client.c:1733:ptlrpc_send_new_req()) Sending RPC req@00000000b6612f78 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_02:lustre-MDT0001-mdtlov_UUID:3355:1862456998121792:0@lo:400:kworker/u8:0.0 00000100:00100000:3.0:1776178727.783747:0:3355:0:(events.c:361:request_in_callback()) peer: 12345-0@lo (source: 12345-0@lo) 00000100:00100000:3.0:1776178727.783775:0:3355:0:(ptlrpcd.c:415:ptlrpcd_check()) transfer 1 async RPCs [3->2] 00000100:00100000:3.0:1776178727.783782:0:3355:0:(client.c:1733:ptlrpc_send_new_req()) Sending RPC req@000000007d766b2a pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_02:lustre-MDT0001-lwp-OST0000_UUID:3355:1862456998121472:0@lo:400:kworker/u8:0.0 00000100:00100000:1.0:1776178727.783807:0:20762:0:(service.c:2152:ptlrpc_server_handle_req_in()) got req x1862456998121792 00000100:00100000:3.0:1776178727.783811:0:3355:0:(events.c:361:request_in_callback()) peer: 12345-0@lo (source: 12345-0@lo) 00000100:00100000:1.0:1776178727.783850:0:20762:0:(nrs_fifo.c:179:nrs_fifo_req_get()) NRS start fifo request from 12345-0@lo, seq: 922 00000100:00100000:1.0:1776178727.783855:0:20762:0:(service.c:2324:ptlrpc_server_handle_request()) Handling RPC req@00000000d9d2b484 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_013:lustre-MDT0001-mdtlov_UUID+5:3355:x1862456998121792:12345-0@lo:400:kworker/u8:0.0 00000100:00100000:1.0:1776178727.783905:0:20762:0:(service.c:2372:ptlrpc_server_handle_request()) Handled RPC req@00000000d9d2b484 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_013:lustre-MDT0001-mdtlov_UUID+5:3355:x1862456998121792:12345-0@lo:400:kworker/u8:0.0 Request processed in 50us (158us total) trans 0 rc 0/0 00000100:00100000:1.0:1776178727.783910:0:20762:0:(nrs_fifo.c:241:nrs_fifo_req_stop()) NRS stop fifo request from 12345-0@lo, seq: 922 00000100:00100000:3.0:1776178727.784140:0:3355:0:(client.c:2215:ptlrpc_check_set()) Completed RPC req@00000000b6612f78 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_02:lustre-MDT0001-mdtlov_UUID:3355:1862456998121792:0@lo:400:kworker/u8:0.0 00000100:00100000:2.0:1776178727.784216:0:20140:0:(service.c:2152:ptlrpc_server_handle_req_in()) got req x1862456998121344 00000100:00100000:2.0:1776178727.784253:0:20140:0:(nrs_fifo.c:179:nrs_fifo_req_get()) NRS start fifo request from 12345-0@lo, seq: 360 00000100:00100000:2.0:1776178727.784257:0:20140:0:(service.c:2324:ptlrpc_server_handle_request()) Handling RPC req@000000001091921c pname:cluuid+ref:pid:xid:nid:opc:job ll_mgs_0002:ac61f839-0e8e-4cac-9c59-08bcfbbad330+18:3355:x1862456998121344:12345-0@lo:400:kworker/u8:0.0 00000100:00100000:2.0:1776178727.784303:0:20140:0:(service.c:2372:ptlrpc_server_handle_request()) Handled RPC req@000000001091921c pname:cluuid+ref:pid:xid:nid:opc:job ll_mgs_0002:ac61f839-0e8e-4cac-9c59-08bcfbbad330+18:3355:x1862456998121344:12345-0@lo:400:kworker/u8:0.0 Request processed in 46us (629us total) trans 0 rc 0/0 00000100:00100000:2.0:1776178727.784310:0:20140:0:(nrs_fifo.c:241:nrs_fifo_req_stop()) NRS stop fifo request from 12345-0@lo, seq: 360 00000100:00100000:2.0:1776178727.784341:0:11281:0:(service.c:2152:ptlrpc_server_handle_req_in()) got req x1862456998121472 00000100:00100000:2.0:1776178727.784379:0:11281:0:(nrs_fifo.c:179:nrs_fifo_req_get()) NRS start fifo request from 12345-0@lo, seq: 1236 00000100:00100000:2.0:1776178727.784383:0:11281:0:(service.c:2324:ptlrpc_server_handle_request()) Handling RPC req@000000006ce0dfc8 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_004:lustre-MDT0001-lwp-OST0000_UUID+5:3355:x1862456998121472:12345-0@lo:400:kworker/u8:0.0 00000100:00100000:2.0:1776178727.784412:0:11281:0:(service.c:2372:ptlrpc_server_handle_request()) Handled RPC req@000000006ce0dfc8 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_004:lustre-MDT0001-lwp-OST0000_UUID+5:3355:x1862456998121472:12345-0@lo:400:kworker/u8:0.0 Request processed in 28us (603us total) trans 0 rc 0/0 00000100:00100000:2.0:1776178727.784418:0:11281:0:(nrs_fifo.c:241:nrs_fifo_req_stop()) NRS stop fifo request from 12345-0@lo, seq: 1236 00000100:00100000:3.0:1776178727.784568:0:3355:0:(client.c:2215:ptlrpc_check_set()) Completed RPC req@000000007d766b2a pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_02:lustre-MDT0001-lwp-OST0000_UUID:3355:1862456998121472:0@lo:400:kworker/u8:0.0 00000100:00100000:3.0:1776178727.784576:0:3355:0:(client.c:2215:ptlrpc_check_set()) Completed RPC req@000000002c6408bc pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_02:ac61f839-0e8e-4cac-9c59-08bcfbbad330:3355:1862456998121344:0@lo:400:kworker/u8:0.0 00010000:00010000:1.0:1776178728.416407:0:9225:0:(ldlm_lockd.c:2792:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:1.0:1776178729.312453:0:3355:0:(ptlrpcd.c:415:ptlrpcd_check()) transfer 1 async RPCs [3->2] 00000100:00100000:1.0:1776178729.312478:0:3355:0:(client.c:1733:ptlrpc_send_new_req()) Sending RPC req@0000000004dd3b9b pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_02:lustre-MDT0001-mdtlov_UUID:3355:1862456998121920:0@lo:13:osp-pre-1-1.0 00000100:00100000:1.0:1776178729.312547:0:3355:0:(events.c:361:request_in_callback()) peer: 12345-0@lo (source: 12345-0@lo) 00000100:00100000:2.0:1776178729.312709:0:9554:0:(service.c:2152:ptlrpc_server_handle_req_in()) got req x1862456998121920 00000100:00100000:2.0:1776178729.312759:0:9554:0:(nrs_fifo.c:179:nrs_fifo_req_get()) NRS start fifo request from 12345-0@lo, seq: 693 00000100:00100000:2.0:1776178729.312765:0:9554:0:(service.c:2324:ptlrpc_server_handle_request()) Handling RPC req@0000000015f283a6 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_create00:lustre-MDT0001-mdtlov_UUID+5:3355:x1862456998121920:12345-0@lo:13:osp-pre-1-1.0 00002000:00000024:2.0:1776178729.312801:0:9554:0:(ofd_obd.c:740:ofd_statfs()) blocks cached 229376 granted 491746368 pending 0 free 3923689472 avail 3692167168 00000100:00100000:2.0:1776178729.312840:0:9554:0:(service.c:2372:ptlrpc_server_handle_request()) Handled RPC req@0000000015f283a6 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_create00:lustre-MDT0001-mdtlov_UUID+5:3355:x1862456998121920:12345-0@lo:13:osp-pre-1-1.0 Request processed in 74us (294us total) trans 0 rc 0/0 00000100:00100000:2.0:1776178729.312846:0:9554:0:(nrs_fifo.c:241:nrs_fifo_req_stop()) NRS stop fifo request from 12345-0@lo, seq: 693 00000100:00100000:1.0:1776178729.314135:0:3355:0:(client.c:2215:ptlrpc_check_set()) Completed RPC req@0000000004dd3b9b pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_02:lustre-MDT0001-mdtlov_UUID:3355:1862456998121920:0@lo:13:osp-pre-1-1.0 00010000:00010000:3.0:1776178729.440478:0:6244:0:(ldlm_lockd.c:2792:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:3.0:1776178729.826663:0:3354:0:(ptlrpcd.c:415:ptlrpcd_check()) transfer 1 async RPCs [0->1] 00000100:00100000:3.0:1776178729.826746:0:3354:0:(client.c:1733:ptlrpc_send_new_req()) Sending RPC req@0000000042de09e9 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_01:lustre-MDT0001-mdtlov_UUID:3354:1862456998121984:0@lo:13:osp-pre-0-1.0 00000100:00100000:3.0:1776178729.826963:0:3354:0:(events.c:361:request_in_callback()) peer: 12345-0@lo (source: 12345-0@lo) 00000100:00100000:1.0:1776178729.827088:0:8231:0:(service.c:2152:ptlrpc_server_handle_req_in()) got req x1862456998121984 00000100:00100000:1.0:1776178729.827138:0:8231:0:(nrs_fifo.c:179:nrs_fifo_req_get()) NRS start fifo request from 12345-0@lo, seq: 694 00000100:00100000:1.0:1776178729.827144:0:8231:0:(service.c:2324:ptlrpc_server_handle_request()) Handling RPC req@0000000040899e7d pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_create00:lustre-MDT0001-mdtlov_UUID+5:3354:x1862456998121984:12345-0@lo:13:osp-pre-0-1.0 00002000:00000024:1.0:1776178729.827172:0:8231:0:(ofd_obd.c:740:ofd_statfs()) blocks cached 315392 granted 484883776 pending 0 free 3923689472 avail 3692167168 00000100:00100000:1.0:1776178729.827219:0:8231:0:(service.c:2372:ptlrpc_server_handle_request()) Handled RPC req@0000000040899e7d pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_create00:lustre-MDT0001-mdtlov_UUID+5:3354:x1862456998121984:12345-0@lo:13:osp-pre-0-1.0 Request processed in 74us (258us total) trans 0 rc 0/0 00000100:00100000:1.0:1776178729.827226:0:8231:0:(nrs_fifo.c:241:nrs_fifo_req_stop()) NRS stop fifo request from 12345-0@lo, seq: 694 00000100:00100000:3.0:1776178729.827386:0:3354:0:(client.c:2215:ptlrpc_check_set()) Completed RPC req@0000000042de09e9 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_01:lustre-MDT0001-mdtlov_UUID:3354:1862456998121984:0@lo:13:osp-pre-0-1.0 00010000:00010000:1.0:1776178730.464283:0:6245:0:(ldlm_lockd.c:2792:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:3.0:1776178731.297608:0:3347:0:(events.c:361:request_in_callback()) peer: 12345-192.168.204.3@tcp (source: 12345-192.168.204.3@tcp) 00000100:00100000:3.0:1776178731.297670:0:3347:0:(events.c:361:request_in_callback()) peer: 12345-192.168.204.3@tcp (source: 12345-192.168.204.3@tcp) 00000100:00100000:2.0:1776178731.297792:0:20137:0:(service.c:2152:ptlrpc_server_handle_req_in()) got req x1862456979141952 00000100:00100000:2.0:1776178731.297849:0:20137:0:(nrs_fifo.c:179:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.204.3@tcp, seq: 361 00000100:00100000:2.0:1776178731.297856:0:20137:0:(service.c:2324:ptlrpc_server_handle_request()) Handling RPC req@000000005b4b43d0 pname:cluuid+ref:pid:xid:nid:opc:job ll_mgs_0000:5b5c66b1-1a2f-4651-8548-0c75b9a38a94+9:2270:x1862456979141952:12345-192.168.204.3@tcp:400:kworker/u8:3.0 00000100:00100000:2.0:1776178731.297926:0:20137:0:(service.c:2372:ptlrpc_server_handle_request()) Handled RPC req@000000005b4b43d0 pname:cluuid+ref:pid:xid:nid:opc:job ll_mgs_0000:5b5c66b1-1a2f-4651-8548-0c75b9a38a94+9:2270:x1862456979141952:12345-192.168.204.3@tcp:400:kworker/u8:3.0 Request processed in 69us (327us total) trans 0 rc 0/0 00000100:00100000:2.0:1776178731.297933:0:20137:0:(nrs_fifo.c:241:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.204.3@tcp, seq: 361 00000100:00100000:3.0:1776178731.298252:0:3347:0:(events.c:361:request_in_callback()) peer: 12345-192.168.204.3@tcp (source: 12345-192.168.204.3@tcp) 00000100:00100000:3.0:1776178731.298497:0:3347:0:(events.c:361:request_in_callback()) peer: 12345-192.168.204.3@tcp (source: 12345-192.168.204.3@tcp) 00000100:00100000:2.0:1776178731.298539:0:6257:0:(service.c:2152:ptlrpc_server_handle_req_in()) got req x1862456979142080 00000100:00100000:2.0:1776178731.298582:0:6257:0:(nrs_fifo.c:179:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.204.3@tcp, seq: 1237 00000100:00100000:2.0:1776178731.298587:0:6257:0:(service.c:2324:ptlrpc_server_handle_request()) Handling RPC req@000000008a1e777f pname:cluuid+ref:pid:xid:nid:opc:job mdt00_002:b50963bd-8eba-4432-9a54-8888d00e3fa1+6:2270:x1862456979142080:12345-192.168.204.3@tcp:400:kworker/u8:3.0 00000100:00100000:2.0:1776178731.298628:0:6257:0:(service.c:2372:ptlrpc_server_handle_request()) Handled RPC req@000000008a1e777f pname:cluuid+ref:pid:xid:nid:opc:job mdt00_002:b50963bd-8eba-4432-9a54-8888d00e3fa1+6:2270:x1862456979142080:12345-192.168.204.3@tcp:400:kworker/u8:3.0 Request processed in 39us (377us total) trans 0 rc 0/0 00000100:00100000:2.0:1776178731.298634:0:6257:0:(nrs_fifo.c:241:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.204.3@tcp, seq: 1237 00000100:00100000:3.0:1776178731.303672:0:3347:0:(events.c:361:request_in_callback()) peer: 12345-192.168.204.3@tcp (source: 12345-192.168.204.3@tcp) 00000100:00100000:2.0:1776178731.303794:0:11281:0:(service.c:2152:ptlrpc_server_handle_req_in()) got req x1862456979142016 00000100:00100000:2.0:1776178731.303840:0:11281:0:(nrs_fifo.c:179:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.204.3@tcp, seq: 7036 00000100:00100000:2.0:1776178731.303847:0:11281:0:(service.c:2324:ptlrpc_server_handle_request()) Handling RPC req@000000000cde9837 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_004:0+-99:2269:x1862456979142016:12345-192.168.204.3@tcp:38:kworker/u8:3.0 00010000:02000000:2.0:1776178731.303865:0:11281:0:(ldlm_lib.c:1139:target_handle_connect()) lustre-MDT0000: Not available for connect from 192.168.204.3@tcp (stopping) 00000100:00100000:2.0:1776178731.303923:0:11281:0:(service.c:2372:ptlrpc_server_handle_request()) Handled RPC req@000000000cde9837 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_004:0+-99:2269:x1862456979142016:12345-192.168.204.3@tcp:38:kworker/u8:3.0 Request processed in 76us (258us total) trans 0 rc -19/-19 00000100:00100000:2.0:1776178731.303930:0:11281:0:(nrs_fifo.c:241:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.204.3@tcp, seq: 7036 00000100:00100000:1.0:1776178731.304501:0:14440:0:(service.c:2152:ptlrpc_server_handle_req_in()) got req x1862456979142144 00000100:00100000:1.0:1776178731.304547:0:14440:0:(service.c:2152:ptlrpc_server_handle_req_in()) got req x1862456979142208 00000100:00100000:1.0:1776178731.304569:0:14440:0:(nrs_fifo.c:179:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.204.3@tcp, seq: 923 00000100:00100000:1.0:1776178731.304575:0:14440:0:(service.c:2324:ptlrpc_server_handle_request()) Handling RPC req@00000000c59ba3a2 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_004:b50963bd-8eba-4432-9a54-8888d00e3fa1+8:2271:x1862456979142144:12345-192.168.204.3@tcp:400:kworker/u8:3.0 00000100:00100000:1.0:1776178731.304616:0:14440:0:(service.c:2372:ptlrpc_server_handle_request()) Handled RPC req@00000000c59ba3a2 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_004:b50963bd-8eba-4432-9a54-8888d00e3fa1+8:2271:x1862456979142144:12345-192.168.204.3@tcp:400:kworker/u8:3.0 Request processed in 41us (6946us total) trans 0 rc 0/0 00000100:00100000:1.0:1776178731.304623:0:14440:0:(nrs_fifo.c:241:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.204.3@tcp, seq: 923 00000100:00100000:1.0:1776178731.304636:0:14440:0:(nrs_fifo.c:179:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.204.3@tcp, seq: 924 00000100:00100000:1.0:1776178731.304639:0:14440:0:(service.c:2324:ptlrpc_server_handle_request()) Handling RPC req@000000006165f052 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_004:b50963bd-8eba-4432-9a54-8888d00e3fa1+9:2270:x1862456979142208:12345-192.168.204.3@tcp:400:kworker/u8:3.0 00000100:00100000:1.0:1776178731.304671:0:14440:0:(service.c:2372:ptlrpc_server_handle_request()) Handled RPC req@000000006165f052 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_004:b50963bd-8eba-4432-9a54-8888d00e3fa1+9:2270:x1862456979142208:12345-192.168.204.3@tcp:400:kworker/u8:3.0 Request processed in 30us (6175us total) trans 0 rc 0/0 00000100:00100000:1.0:1776178731.304690:0:14440:0:(nrs_fifo.c:241:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.204.3@tcp, seq: 924 00010000:00010000:2.0:1776178731.488315:0:9224:0:(ldlm_lockd.c:2792:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00010000:00010000:1.0:1776178732.512430:0:9225:0:(ldlm_lockd.c:2792:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00080000:0.0:1776178732.896323:0:36:0:(pinger.c:251:ptlrpc_pinger_process_import()) lustre-MDT0000-lwp-OST0000_UUID->lustre-MDT0000_UUID: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00080000:0.0:1776178732.896332:0:36:0:(recover.c:57:ptlrpc_initiate_recovery()) lustre-MDT0000_UUID: starting recovery 00000100:00080000:0.0:1776178732.896335:0:36:0:(import.c:86:import_set_state_nolock()) 0000000047601a98 lustre-MDT0000_UUID: changing import state from DISCONN to CONNECTING 00000100:00080000:0.0:1776178732.896343:0:36:0:(import.c:534:import_select_connection()) lustre-MDT0000-lwp-OST0000: connect to NID 0@lo last attempt 1435 00000100:00080000:0.0:1776178732.896346:0:36:0:(import.c:581:import_select_connection()) lustre-MDT0000-lwp-OST0000: tried all connections, increasing latency to 21s 00000100:00080000:0.0:1776178732.896352:0:36:0:(import.c:615:import_select_connection()) lustre-MDT0000-lwp-OST0000: import 0000000047601a98 using connection 192.168.204.103@tcp/0@lo 00000100:00100000:0.0:1776178732.896372:0:36:0:(import.c:807:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5) req@00000000b206023d x1862456998122112/t0(0) o38->lustre-MDT0000-lwp-OST0000@0@lo:12/10 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' 00000100:00080000:0.0:1776178732.896419:0:36:0:(pinger.c:251:ptlrpc_pinger_process_import()) lustre-MDT0000-lwp-OST0001_UUID->lustre-MDT0000_UUID: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00080000:0.0:1776178732.896422:0:36:0:(recover.c:57:ptlrpc_initiate_recovery()) lustre-MDT0000_UUID: starting recovery 00000100:00080000:0.0:1776178732.896424:0:36:0:(import.c:86:import_set_state_nolock()) 00000000dfc8db13 lustre-MDT0000_UUID: changing import state from DISCONN to CONNECTING 00000100:00080000:0.0:1776178732.896427:0:36:0:(import.c:534:import_select_connection()) lustre-MDT0000-lwp-OST0001: connect to NID 0@lo last attempt 1435 00000100:00080000:0.0:1776178732.896428:0:36:0:(import.c:581:import_select_connection()) lustre-MDT0000-lwp-OST0001: tried all connections, increasing latency to 21s 00000100:00080000:0.0:1776178732.896431:0:36:0:(import.c:615:import_select_connection()) lustre-MDT0000-lwp-OST0001: import 00000000dfc8db13 using connection 192.168.204.103@tcp/0@lo 00000100:00100000:0.0:1776178732.896440:0:36:0:(import.c:807:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5) req@00000000f1ed5d03 x1862456998122240/t0(0) o38->lustre-MDT0000-lwp-OST0001@0@lo:12/10 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' 00000100:00080000:0.0:1776178732.896453:0:36:0:(pinger.c:251:ptlrpc_pinger_process_import()) lustre-MDT0001-mdtlov_UUID->lustre-MDT0000_UUID: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00080000:0.0:1776178732.896456:0:36:0:(recover.c:57:ptlrpc_initiate_recovery()) lustre-MDT0000_UUID: starting recovery 00000100:00080000:0.0:1776178732.896458:0:36:0:(import.c:86:import_set_state_nolock()) 00000000ba4f1821 lustre-MDT0000_UUID: changing import state from DISCONN to CONNECTING 00000100:00080000:0.0:1776178732.896462:0:36:0:(import.c:534:import_select_connection()) lustre-MDT0000-osp-MDT0001: connect to NID 0@lo last attempt 1435 00000100:00080000:0.0:1776178732.896464:0:36:0:(import.c:581:import_select_connection()) lustre-MDT0000-osp-MDT0001: tried all connections, increasing latency to 21s 00000100:00080000:0.0:1776178732.896468:0:36:0:(import.c:615:import_select_connection()) lustre-MDT0000-osp-MDT0001: import 00000000ba4f1821 using connection 192.168.204.103@tcp/0@lo 00000100:00100000:0.0:1776178732.896478:0:36:0:(import.c:807:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5) req@00000000f8a7b10a x1862456998122368/t0(0) o38->lustre-MDT0000-osp-MDT0001@0@lo:24/4 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' 00000100:00100000:2.0:1776178732.896483:0:3352:0:(client.c:1733:ptlrpc_send_new_req()) Sending RPC req@00000000b206023d pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:lustre-MDT0000-lwp-OST0000_UUID:3352:1862456998122112:0@lo:38:kworker/u8:1.0 00000100:00080000:0.0:1776178732.896502:0:36:0:(pinger.c:251:ptlrpc_pinger_process_import()) lustre-MDT0000-lwp-MDT0001_UUID->lustre-MDT0000_UUID: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00080000:0.0:1776178732.896506:0:36:0:(recover.c:57:ptlrpc_initiate_recovery()) lustre-MDT0000_UUID: starting recovery 00000100:00080000:0.0:1776178732.896507:0:36:0:(import.c:86:import_set_state_nolock()) 00000000a7c55b1f lustre-MDT0000_UUID: changing import state from DISCONN to CONNECTING 00000100:00080000:0.0:1776178732.896510:0:36:0:(import.c:534:import_select_connection()) lustre-MDT0000-lwp-MDT0001: connect to NID 0@lo last attempt 1435 00000100:00080000:0.0:1776178732.896512:0:36:0:(import.c:581:import_select_connection()) lustre-MDT0000-lwp-MDT0001: tried all connections, increasing latency to 21s 00000100:00080000:0.0:1776178732.896515:0:36:0:(import.c:615:import_select_connection()) lustre-MDT0000-lwp-MDT0001: import 00000000a7c55b1f using connection 192.168.204.103@tcp/0@lo 00000100:00100000:0.0:1776178732.896525:0:36:0:(import.c:807:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5) req@00000000ad99db59 x1862456998122560/t0(0) o38->lustre-MDT0000-lwp-MDT0001@0@lo:12/10 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' 00000100:00100000:0.0:1776178732.896552:0:3356:0:(client.c:1733:ptlrpc_send_new_req()) Sending RPC req@000000003b0b9a0e pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_03:lustre-MDT0001-lwp-OST0001_UUID:3356:1862456998122304:0@lo:400:kworker/u8:1.0 00000100:00100000:2.0:1776178732.896559:0:3352:0:(events.c:361:request_in_callback()) peer: 12345-0@lo (source: 12345-0@lo) 00000100:00100000:3.0:1776178732.896646:0:3354:0:(client.c:1733:ptlrpc_send_new_req()) Sending RPC req@0000000081aacc4a pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_01:ac61f839-0e8e-4cac-9c59-08bcfbbad330:3354:1862456998122048:0@lo:400:kworker/u8:1.0 00000100:00100000:1.0:1776178732.896701:0:3353:0:(client.c:1733:ptlrpc_send_new_req()) Sending RPC req@0000000025b3b8e0 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_00:lustre-MDT0001-mdtlov_UUID:3353:1862456998122432:0@lo:400:kworker/u8:1.0 00000100:00100000:0.0:1776178732.903573:0:3356:0:(events.c:361:request_in_callback()) peer: 12345-0@lo (source: 12345-0@lo) 00000100:00100000:0.0:1776178732.903646:0:3356:0:(ptlrpcd.c:415:ptlrpcd_check()) transfer 1 async RPCs [2->3] 00000100:00100000:0.0:1776178732.903659:0:3356:0:(client.c:1733:ptlrpc_send_new_req()) Sending RPC req@00000000a942a398 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_03:lustre-MDT0001-lwp-OST0000_UUID:3356:1862456998122176:0@lo:400:kworker/u8:1.0 00000100:00100000:0.0:1776178732.903701:0:3356:0:(events.c:361:request_in_callback()) peer: 12345-0@lo (source: 12345-0@lo) 00000100:00100000:0.0:1776178732.903753:0:8250:0:(service.c:2152:ptlrpc_server_handle_req_in()) got req x1862456998122112 00000100:00100000:2.0:1776178732.903758:0:11281:0:(service.c:2152:ptlrpc_server_handle_req_in()) got req x1862456998122304 00000100:00100000:3.0:1776178732.906754:0:3354:0:(events.c:361:request_in_callback()) peer: 12345-0@lo (source: 12345-0@lo) 00000100:00100000:3.0:1776178732.906804:0:3354:0:(client.c:1733:ptlrpc_send_new_req()) Sending RPC req@00000000de8350d4 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_01:lustre-MDT0001-mdtlov_UUID:3354:1862456998122496:0@lo:400:kworker/u8:1.0 00000100:00100000:3.0:1776178732.906848:0:3354:0:(events.c:361:request_in_callback()) peer: 12345-0@lo (source: 12345-0@lo) 00000100:00100000:3.0:1776178732.906906:0:20762:0:(service.c:2152:ptlrpc_server_handle_req_in()) got req x1862456998122496 00000100:00100000:3.0:1776178732.906947:0:20762:0:(nrs_fifo.c:179:nrs_fifo_req_get()) NRS start fifo request from 12345-0@lo, seq: 925 00000100:00100000:3.0:1776178732.906953:0:20762:0:(service.c:2324:ptlrpc_server_handle_request()) Handling RPC req@000000004c44fad7 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_013:lustre-MDT0001-mdtlov_UUID+5:3354:x1862456998122496:12345-0@lo:400:kworker/u8:1.0 00000100:00100000:3.0:1776178732.907003:0:20762:0:(service.c:2372:ptlrpc_server_handle_request()) Handled RPC req@000000004c44fad7 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_013:lustre-MDT0001-mdtlov_UUID+5:3354:x1862456998122496:12345-0@lo:400:kworker/u8:1.0 Request processed in 49us (154us total) trans 0 rc 0/0 00000100:00100000:3.0:1776178732.907010:0:20762:0:(nrs_fifo.c:241:nrs_fifo_req_stop()) NRS stop fifo request from 12345-0@lo, seq: 925 00000100:00100000:3.0:1776178732.907184:0:20140:0:(service.c:2152:ptlrpc_server_handle_req_in()) got req x1862456998122048 00000100:00100000:3.0:1776178732.907210:0:20140:0:(nrs_fifo.c:179:nrs_fifo_req_get()) NRS start fifo request from 12345-0@lo, seq: 362 00000100:00100000:3.0:1776178732.907213:0:20140:0:(service.c:2324:ptlrpc_server_handle_request()) Handling RPC req@0000000020ee145c pname:cluuid+ref:pid:xid:nid:opc:job ll_mgs_0002:ac61f839-0e8e-4cac-9c59-08bcfbbad330+18:3354:x1862456998122048:12345-0@lo:400:kworker/u8:1.0 00000100:00100000:3.0:1776178732.907249:0:20140:0:(service.c:2372:ptlrpc_server_handle_request()) Handled RPC req@0000000020ee145c pname:cluuid+ref:pid:xid:nid:opc:job ll_mgs_0002:ac61f839-0e8e-4cac-9c59-08bcfbbad330+18:3354:x1862456998122048:12345-0@lo:400:kworker/u8:1.0 Request processed in 35us (504us total) trans 0 rc 0/0 00000100:00100000:3.0:1776178732.907256:0:20140:0:(nrs_fifo.c:241:nrs_fifo_req_stop()) NRS stop fifo request from 12345-0@lo, seq: 362 00000100:00100000:3.0:1776178732.907299:0:3354:0:(client.c:2215:ptlrpc_check_set()) Completed RPC req@0000000081aacc4a pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_01:ac61f839-0e8e-4cac-9c59-08bcfbbad330:3354:1862456998122048:0@lo:400:kworker/u8:1.0 00000100:00100000:3.0:1776178732.907312:0:3354:0:(client.c:2215:ptlrpc_check_set()) Completed RPC req@00000000de8350d4 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_01:lustre-MDT0001-mdtlov_UUID:3354:1862456998122496:0@lo:400:kworker/u8:1.0 00000100:00100000:3.0:1776178732.907347:0:3352:0:(client.c:1733:ptlrpc_send_new_req()) Sending RPC req@00000000f1ed5d03 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:lustre-MDT0000-lwp-OST0001_UUID:3352:1862456998122240:0@lo:38:kworker/u8:1.0 00000100:00100000:3.0:1776178732.907373:0:3352:0:(events.c:361:request_in_callback()) peer: 12345-0@lo (source: 12345-0@lo) 00000100:00100000:3.0:1776178732.915535:0:3352:0:(client.c:1733:ptlrpc_send_new_req()) Sending RPC req@00000000f8a7b10a pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:lustre-MDT0001-mdtlov_UUID:3352:1862456998122368:0@lo:38:kworker/u8:1.0 00000100:00100000:3.0:1776178732.915602:0:3352:0:(events.c:361:request_in_callback()) peer: 12345-0@lo (source: 12345-0@lo) 00000100:00100000:3.0:1776178732.915639:0:3352:0:(client.c:1733:ptlrpc_send_new_req()) Sending RPC req@00000000ad99db59 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:lustre-MDT0000-lwp-MDT0001_UUID:3352:1862456998122560:0@lo:38:kworker/u8:1.0 00000100:00100000:3.0:1776178732.915665:0:3352:0:(events.c:361:request_in_callback()) peer: 12345-0@lo (source: 12345-0@lo) 00000100:00100000:0.0:1776178732.919200:0:8250:0:(service.c:2152:ptlrpc_server_handle_req_in()) got req x1862456998122176 00000100:00100000:0.0:1776178732.919714:0:8250:0:(service.c:2152:ptlrpc_server_handle_req_in()) got req x1862456998122240 00000100:00100000:0.0:1776178732.919740:0:8250:0:(nrs_fifo.c:179:nrs_fifo_req_get()) NRS start fifo request from 12345-0@lo, seq: 1238 00000100:00100000:0.0:1776178732.919746:0:8250:0:(service.c:2324:ptlrpc_server_handle_request()) Handling RPC req@00000000be147540 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_003:lustre-MDT0001-lwp-OST0001_UUID+5:3356:x1862456998122304:12345-0@lo:400:kworker/u8:1.0 00000100:00100000:0.0:1776178732.919848:0:8250:0:(service.c:2372:ptlrpc_server_handle_request()) Handled RPC req@00000000be147540 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_003:lustre-MDT0001-lwp-OST0001_UUID+5:3356:x1862456998122304:12345-0@lo:400:kworker/u8:1.0 Request processed in 102us (16283us total) trans 0 rc 0/0 00000100:00100000:0.0:1776178732.919855:0:8250:0:(nrs_fifo.c:241:nrs_fifo_req_stop()) NRS stop fifo request from 12345-0@lo, seq: 1238 00000100:00100000:0.0:1776178732.919877:0:8250:0:(nrs_fifo.c:179:nrs_fifo_req_get()) NRS start fifo request from 12345-0@lo, seq: 1239 00000100:00100000:0.0:1776178732.919879:0:8250:0:(service.c:2324:ptlrpc_server_handle_request()) Handling RPC req@00000000dc38eb5b pname:cluuid+ref:pid:xid:nid:opc:job mdt00_003:lustre-MDT0001-lwp-OST0000_UUID+5:3356:x1862456998122176:12345-0@lo:400:kworker/u8:1.0 00000100:00100000:0.0:1776178732.919906:0:8250:0:(service.c:2372:ptlrpc_server_handle_request()) Handled RPC req@00000000dc38eb5b pname:cluuid+ref:pid:xid:nid:opc:job mdt00_003:lustre-MDT0001-lwp-OST0000_UUID+5:3356:x1862456998122176:12345-0@lo:400:kworker/u8:1.0 Request processed in 26us (16205us total) trans 0 rc 0/0 00000100:00100000:0.0:1776178732.919912:0:8250:0:(nrs_fifo.c:241:nrs_fifo_req_stop()) NRS stop fifo request from 12345-0@lo, seq: 1239 00000100:00100000:0.0:1776178732.919924:0:8250:0:(nrs_fifo.c:179:nrs_fifo_req_get()) NRS start fifo request from 12345-0@lo, seq: 7037 00000100:00100000:0.0:1776178732.919928:0:8250:0:(service.c:2324:ptlrpc_server_handle_request()) Handling RPC req@000000002300ff10 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_003:0+-99:3352:x1862456998122112:12345-0@lo:38:kworker/u8:1.0 00010000:02000000:0.0:1776178732.919941:0:8250:0:(ldlm_lib.c:1139:target_handle_connect()) lustre-MDT0000: Not available for connect from 0@lo (stopping) 00000100:00100000:0.0:1776178732.919968:0:8250:0:(service.c:2372:ptlrpc_server_handle_request()) Handled RPC req@000000002300ff10 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_003:0+-99:3352:x1862456998122112:12345-0@lo:38:kworker/u8:1.0 Request processed in 39us (23415us total) trans 0 rc -19/-19 00000100:00100000:0.0:1776178732.919973:0:8250:0:(nrs_fifo.c:241:nrs_fifo_req_stop()) NRS stop fifo request from 12345-0@lo, seq: 7037 00000100:00100000:0.0:1776178732.919983:0:8250:0:(nrs_fifo.c:179:nrs_fifo_req_get()) NRS start fifo request from 12345-0@lo, seq: 7038 00000100:00100000:0.0:1776178732.919986:0:8250:0:(service.c:2324:ptlrpc_server_handle_request()) Handling RPC req@00000000f4172b4c pname:cluuid+ref:pid:xid:nid:opc:job mdt00_003:0+-99:3352:x1862456998122240:12345-0@lo:38:kworker/u8:1.0 00010000:02000000:0.0:1776178732.919992:0:8250:0:(ldlm_lib.c:1139:target_handle_connect()) lustre-MDT0000: Not available for connect from 0@lo (stopping) 00000100:00100000:0.0:1776178732.920012:0:8250:0:(service.c:2372:ptlrpc_server_handle_request()) Handled RPC req@00000000f4172b4c pname:cluuid+ref:pid:xid:nid:opc:job mdt00_003:0+-99:3352:x1862456998122240:12345-0@lo:38:kworker/u8:1.0 Request processed in 25us (12639us total) trans 0 rc -19/-19 00000100:00100000:2.0:1776178732.921182:0:14128:0:(service.c:2152:ptlrpc_server_handle_req_in()) got req x1862456998122368 00000100:00100000:2.0:1776178732.921221:0:14128:0:(nrs_fifo.c:179:nrs_fifo_req_get()) NRS start fifo request from 12345-0@lo, seq: 811 00000100:00100000:2.0:1776178732.921226:0:14128:0:(service.c:2324:ptlrpc_server_handle_request()) Handling RPC req@00000000ba9bc15d pname:cluuid+ref:pid:xid:nid:opc:job mdt_out00_002:0+-99:3352:x1862456998122368:12345-0@lo:38:kworker/u8:1.0 00000100:00100000:1.0:1776178732.921233:0:3353:0:(events.c:361:request_in_callback()) peer: 12345-0@lo (source: 12345-0@lo) 00010000:02000000:2.0:1776178732.921245:0:14128:0:(ldlm_lib.c:1139:target_handle_connect()) lustre-MDT0000: Not available for connect from 0@lo (stopping) 00000100:00100000:2.0:1776178732.921289:0:14128:0:(service.c:2372:ptlrpc_server_handle_request()) Handled RPC req@00000000ba9bc15d pname:cluuid+ref:pid:xid:nid:opc:job mdt_out00_002:0+-99:3352:x1862456998122368:12345-0@lo:38:kworker/u8:1.0 Request processed in 63us (5688us total) trans 0 rc -19/-19 00000100:00100000:2.0:1776178732.921297:0:14128:0:(nrs_fifo.c:241:nrs_fifo_req_stop()) NRS stop fifo request from 12345-0@lo, seq: 811 00000100:00100000:2.0:1776178732.921336:0:20755:0:(service.c:2152:ptlrpc_server_handle_req_in()) got req x1862456998122432 00000100:00100000:2.0:1776178732.921360:0:20755:0:(nrs_fifo.c:179:nrs_fifo_req_get()) NRS start fifo request from 12345-0@lo, seq: 926 00000100:00100000:2.0:1776178732.921364:0:20755:0:(service.c:2324:ptlrpc_server_handle_request()) Handling RPC req@00000000ca8b81ba pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_006:lustre-MDT0001-mdtlov_UUID+5:3353:x1862456998122432:12345-0@lo:400:kworker/u8:1.0 00000100:00100000:1.0:1776178732.921570:0:3353:0:(client.c:2215:ptlrpc_check_set()) Completed RPC req@0000000025b3b8e0 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_00:lustre-MDT0001-mdtlov_UUID:3353:1862456998122432:0@lo:400:kworker/u8:1.0 00000100:00100000:2.0:1776178732.926096:0:20755:0:(service.c:2372:ptlrpc_server_handle_request()) Handled RPC req@00000000ca8b81ba pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_006:lustre-MDT0001-mdtlov_UUID+5:3353:x1862456998122432:12345-0@lo:400:kworker/u8:1.0 Request processed in 4722us (4860us total) trans 0 rc 0/0 00000100:00100000:2.0:1776178732.926113:0:20755:0:(nrs_fifo.c:241:nrs_fifo_req_stop()) NRS stop fifo request from 12345-0@lo, seq: 926 00000100:00100000:0.0:1776178732.927177:0:8250:0:(nrs_fifo.c:241:nrs_fifo_req_stop()) NRS stop fifo request from 12345-0@lo, seq: 7038 00000100:00100000:1.0:1776178732.927221:0:3356:0:(client.c:2215:ptlrpc_check_set()) Completed RPC req@00000000a942a398 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_03:lustre-MDT0001-lwp-OST0000_UUID:3356:1862456998122176:0@lo:400:kworker/u8:1.0 00000100:00100000:1.0:1776178732.927240:0:3356:0:(client.c:2215:ptlrpc_check_set()) Completed RPC req@000000003b0b9a0e pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_03:lustre-MDT0001-lwp-OST0001_UUID:3356:1862456998122304:0@lo:400:kworker/u8:1.0 00000100:00080000:3.0:1776178732.927458:0:3352:0:(import.c:86:import_set_state_nolock()) 0000000047601a98 lustre-MDT0000_UUID: changing import state from CONNECTING to DISCONN 00000100:00080000:3.0:1776178732.927464:0:3352:0:(import.c:1429:ptlrpc_connect_interpret()) recovery of lustre-MDT0000_UUID on 192.168.204.103@tcp failed (-19) 00000100:00100000:3.0:1776178732.927468:0:3352:0:(client.c:2215:ptlrpc_check_set()) Completed RPC req@00000000b206023d pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:lustre-MDT0000-lwp-OST0000_UUID:3352:1862456998122112:0@lo:38:kworker/u8:1.0 00000100:00080000:3.0:1776178732.927478:0:3352:0:(import.c:86:import_set_state_nolock()) 00000000dfc8db13 lustre-MDT0000_UUID: changing import state from CONNECTING to DISCONN 00000100:00080000:3.0:1776178732.927480:0:3352:0:(import.c:1429:ptlrpc_connect_interpret()) recovery of lustre-MDT0000_UUID on 192.168.204.103@tcp failed (-19) 00000100:00100000:3.0:1776178732.927482:0:3352:0:(client.c:2215:ptlrpc_check_set()) Completed RPC req@00000000f1ed5d03 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:lustre-MDT0000-lwp-OST0001_UUID:3352:1862456998122240:0@lo:38:kworker/u8:1.0 00000100:00080000:3.0:1776178732.927502:0:3352:0:(import.c:86:import_set_state_nolock()) 00000000ba4f1821 lustre-MDT0000_UUID: changing import state from CONNECTING to DISCONN 00000100:00080000:3.0:1776178732.927505:0:3352:0:(import.c:1429:ptlrpc_connect_interpret()) recovery of lustre-MDT0000_UUID on 192.168.204.103@tcp failed (-19) 00000100:00100000:3.0:1776178732.927507:0:3352:0:(client.c:2215:ptlrpc_check_set()) Completed RPC req@00000000f8a7b10a pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:lustre-MDT0001-mdtlov_UUID:3352:1862456998122368:0@lo:38:kworker/u8:1.0 00000100:00100000:0.0:1776178732.927607:0:8250:0:(service.c:2152:ptlrpc_server_handle_req_in()) got req x1862456998122560 00000100:00100000:0.0:1776178732.927658:0:8250:0:(nrs_fifo.c:179:nrs_fifo_req_get()) NRS start fifo request from 12345-0@lo, seq: 7039 00000100:00100000:0.0:1776178732.927663:0:8250:0:(service.c:2324:ptlrpc_server_handle_request()) Handling RPC req@000000001a990b02 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_003:0+-99:3352:x1862456998122560:12345-0@lo:38:kworker/u8:1.0 00010000:02000000:0.0:1776178732.927675:0:8250:0:(ldlm_lib.c:1139:target_handle_connect()) lustre-MDT0000: Not available for connect from 0@lo (stopping) 00000100:00100000:0.0:1776178732.927741:0:8250:0:(service.c:2372:ptlrpc_server_handle_request()) Handled RPC req@000000001a990b02 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_003:0+-99:3352:x1862456998122560:12345-0@lo:38:kworker/u8:1.0 Request processed in 77us (12075us total) trans 0 rc -19/-19 00000100:00100000:0.0:1776178732.927746:0:8250:0:(nrs_fifo.c:241:nrs_fifo_req_stop()) NRS stop fifo request from 12345-0@lo, seq: 7039 00000100:00080000:3.0:1776178732.927835:0:3352:0:(import.c:86:import_set_state_nolock()) 00000000a7c55b1f lustre-MDT0000_UUID: changing import state from CONNECTING to DISCONN 00000100:00080000:3.0:1776178732.927839:0:3352:0:(import.c:1429:ptlrpc_connect_interpret()) recovery of lustre-MDT0000_UUID on 192.168.204.103@tcp failed (-19) 00000100:00100000:3.0:1776178732.927842:0:3352:0:(client.c:2215:ptlrpc_check_set()) Completed RPC req@00000000ad99db59 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:lustre-MDT0000-lwp-MDT0001_UUID:3352:1862456998122560:0@lo:38:kworker/u8:1.0 00010000:00010000:2.0:1776178733.537361:0:6244:0:(ldlm_lockd.c:2792:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:0.0:1776178734.433094:0:3356:0:(ptlrpcd.c:415:ptlrpcd_check()) transfer 1 async RPCs [2->3] 00000100:00100000:0.0:1776178734.433118:0:3356:0:(client.c:1733:ptlrpc_send_new_req()) Sending RPC req@00000000d0d837c5 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_03:lustre-MDT0001-mdtlov_UUID:3356:1862456998122624:0@lo:13:osp-pre-1-1.0 00000100:00100000:0.0:1776178734.433188:0:3356:0:(events.c:361:request_in_callback()) peer: 12345-0@lo (source: 12345-0@lo) 00000100:00100000:2.0:1776178734.433290:0:9554:0:(service.c:2152:ptlrpc_server_handle_req_in()) got req x1862456998122624 00000100:00100000:2.0:1776178734.433364:0:9554:0:(nrs_fifo.c:179:nrs_fifo_req_get()) NRS start fifo request from 12345-0@lo, seq: 695 00000100:00100000:2.0:1776178734.433369:0:9554:0:(service.c:2324:ptlrpc_server_handle_request()) Handling RPC req@00000000ad99db59 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_create00:lustre-MDT0001-mdtlov_UUID+5:3356:x1862456998122624:12345-0@lo:13:osp-pre-1-1.0 00002000:00000024:2.0:1776178734.433399:0:9554:0:(ofd_obd.c:740:ofd_statfs()) blocks cached 229376 granted 491746368 pending 0 free 3923689472 avail 3692167168 00000100:00100000:2.0:1776178734.433455:0:9554:0:(service.c:2372:ptlrpc_server_handle_request()) Handled RPC req@00000000ad99db59 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_create00:lustre-MDT0001-mdtlov_UUID+5:3356:x1862456998122624:12345-0@lo:13:osp-pre-1-1.0 Request processed in 84us (267us total) trans 0 rc 0/0 00000100:00100000:2.0:1776178734.433461:0:9554:0:(nrs_fifo.c:241:nrs_fifo_req_stop()) NRS stop fifo request from 12345-0@lo, seq: 695 00000100:00100000:0.0:1776178734.433541:0:3356:0:(client.c:2215:ptlrpc_check_set()) Completed RPC req@00000000d0d837c5 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_03:lustre-MDT0001-mdtlov_UUID:3356:1862456998122624:0@lo:13:osp-pre-1-1.0 00010000:00010000:1.0:1776178734.561645:0:6245:0:(ldlm_lockd.c:2792:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:0.0:1776178734.944382:0:3356:0:(client.c:1733:ptlrpc_send_new_req()) Sending RPC req@00000000d0d837c5 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_03:lustre-MDT0001-mdtlov_UUID:3356:1862456998122688:0@lo:13:osp-pre-0-1.0 00000100:00100000:0.0:1776178734.944457:0:3356:0:(events.c:361:request_in_callback()) peer: 12345-0@lo (source: 12345-0@lo) 00000100:00100000:3.0:1776178734.944625:0:8231:0:(service.c:2152:ptlrpc_server_handle_req_in()) got req x1862456998122688 00000100:00100000:3.0:1776178734.944679:0:8231:0:(nrs_fifo.c:179:nrs_fifo_req_get()) NRS start fifo request from 12345-0@lo, seq: 696 00000100:00100000:3.0:1776178734.944685:0:8231:0:(service.c:2324:ptlrpc_server_handle_request()) Handling RPC req@00000000f8a7b10a pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_create00:lustre-MDT0001-mdtlov_UUID+5:3356:x1862456998122688:12345-0@lo:13:osp-pre-0-1.0 00002000:00000024:3.0:1776178734.944713:0:8231:0:(ofd_obd.c:740:ofd_statfs()) blocks cached 315392 granted 484883776 pending 0 free 3923689472 avail 3692167168 00000100:00100000:3.0:1776178734.944936:0:8231:0:(service.c:2372:ptlrpc_server_handle_request()) Handled RPC req@00000000f8a7b10a pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_create00:lustre-MDT0001-mdtlov_UUID+5:3356:x1862456998122688:12345-0@lo:13:osp-pre-0-1.0 Request processed in 249us (479us total) trans 0 rc 0/0 00000100:00100000:3.0:1776178734.944944:0:8231:0:(nrs_fifo.c:241:nrs_fifo_req_stop()) NRS stop fifo request from 12345-0@lo, seq: 696 00000100:00100000:0.0:1776178734.945676:0:3356:0:(client.c:2215:ptlrpc_check_set()) Completed RPC req@00000000d0d837c5 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_03:lustre-MDT0001-mdtlov_UUID:3356:1862456998122688:0@lo:13:osp-pre-0-1.0 00010000:00010000:2.0:1776178735.586670:0:9224:0:(ldlm_lockd.c:2792:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:3.0:1776178736.417405:0:3347:0:(events.c:361:request_in_callback()) peer: 12345-192.168.204.3@tcp (source: 12345-192.168.204.3@tcp) 00000100:00100000:3.0:1776178736.417536:0:3347:0:(events.c:361:request_in_callback()) peer: 12345-192.168.204.3@tcp (source: 12345-192.168.204.3@tcp) 00000100:00100000:3.0:1776178736.417582:0:3347:0:(events.c:361:request_in_callback()) peer: 12345-192.168.204.3@tcp (source: 12345-192.168.204.3@tcp) 00000100:00100000:3.0:1776178736.417629:0:3347:0:(events.c:361:request_in_callback()) peer: 12345-192.168.204.3@tcp (source: 12345-192.168.204.3@tcp) 00000100:00100000:1.0:1776178736.417697:0:20762:0:(service.c:2152:ptlrpc_server_handle_req_in()) got req x1862456979142464 00000100:00100000:1.0:1776178736.417746:0:20762:0:(nrs_fifo.c:179:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.204.3@tcp, seq: 927 00000100:00100000:1.0:1776178736.417754:0:20762:0:(service.c:2324:ptlrpc_server_handle_request()) Handling RPC req@0000000008c9fb19 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_013:b50963bd-8eba-4432-9a54-8888d00e3fa1+8:2271:x1862456979142464:12345-192.168.204.3@tcp:400:kworker/u8:2.0 00000100:00100000:1.0:1776178736.417820:0:20762:0:(service.c:2372:ptlrpc_server_handle_request()) Handled RPC req@0000000008c9fb19 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_013:b50963bd-8eba-4432-9a54-8888d00e3fa1+8:2271:x1862456979142464:12345-192.168.204.3@tcp:400:kworker/u8:2.0 Request processed in 66us (239us total) trans 0 rc 0/0 00000100:00100000:1.0:1776178736.417828:0:20762:0:(nrs_fifo.c:241:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.204.3@tcp, seq: 927 00000100:00100000:2.0:1776178736.417942:0:20137:0:(service.c:2152:ptlrpc_server_handle_req_in()) got req x1862456979142272 00000100:00100000:2.0:1776178736.417979:0:20137:0:(nrs_fifo.c:179:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.204.3@tcp, seq: 363 00000100:00100000:2.0:1776178736.417984:0:20137:0:(service.c:2324:ptlrpc_server_handle_request()) Handling RPC req@00000000f8a3917d pname:cluuid+ref:pid:xid:nid:opc:job ll_mgs_0000:5b5c66b1-1a2f-4651-8548-0c75b9a38a94+9:2271:x1862456979142272:12345-192.168.204.3@tcp:400:kworker/u8:2.0 00000100:00100000:2.0:1776178736.418012:0:20137:0:(service.c:2372:ptlrpc_server_handle_request()) Handled RPC req@00000000f8a3917d pname:cluuid+ref:pid:xid:nid:opc:job ll_mgs_0000:5b5c66b1-1a2f-4651-8548-0c75b9a38a94+9:2271:x1862456979142272:12345-192.168.204.3@tcp:400:kworker/u8:2.0 Request processed in 28us (478us total) trans 0 rc 0/0 00000100:00100000:2.0:1776178736.418019:0:20137:0:(nrs_fifo.c:241:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.204.3@tcp, seq: 363 00000100:00100000:3.0:1776178736.419280:0:3347:0:(events.c:361:request_in_callback()) peer: 12345-192.168.204.3@tcp (source: 12345-192.168.204.3@tcp) 00000100:00100000:0.0:1776178736.419292:0:6257:0:(service.c:2152:ptlrpc_server_handle_req_in()) got req x1862456979142336 00000100:00100000:1.0:1776178736.419356:0:20755:0:(service.c:2152:ptlrpc_server_handle_req_in()) got req x1862456979142528 00000100:00100000:1.0:1776178736.419390:0:20755:0:(nrs_fifo.c:179:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.204.3@tcp, seq: 928 00000100:00100000:1.0:1776178736.419395:0:20755:0:(service.c:2324:ptlrpc_server_handle_request()) Handling RPC req@0000000038b19922 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_006:b50963bd-8eba-4432-9a54-8888d00e3fa1+9:2270:x1862456979142528:12345-192.168.204.3@tcp:400:kworker/u8:2.0 00000100:00100000:1.0:1776178736.419498:0:20755:0:(service.c:2372:ptlrpc_server_handle_request()) Handled RPC req@0000000038b19922 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_006:b50963bd-8eba-4432-9a54-8888d00e3fa1+9:2270:x1862456979142528:12345-192.168.204.3@tcp:400:kworker/u8:2.0 Request processed in 102us (223us total) trans 0 rc 0/0 00000100:00100000:1.0:1776178736.419514:0:20755:0:(nrs_fifo.c:241:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.204.3@tcp, seq: 928 00000100:00100000:2.0:1776178736.419989:0:8250:0:(service.c:2152:ptlrpc_server_handle_req_in()) got req x1862456979142400 00000100:00100000:2.0:1776178736.420078:0:8250:0:(nrs_fifo.c:179:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.204.3@tcp, seq: 1240 00000100:00100000:2.0:1776178736.420082:0:8250:0:(service.c:2324:ptlrpc_server_handle_request()) Handling RPC req@000000000d203d1d pname:cluuid+ref:pid:xid:nid:opc:job mdt00_003:b50963bd-8eba-4432-9a54-8888d00e3fa1+6:2270:x1862456979142400:12345-192.168.204.3@tcp:400:kworker/u8:2.0 00000100:00100000:2.0:1776178736.420125:0:8250:0:(service.c:2372:ptlrpc_server_handle_request()) Handled RPC req@000000000d203d1d pname:cluuid+ref:pid:xid:nid:opc:job mdt00_003:b50963bd-8eba-4432-9a54-8888d00e3fa1+6:2270:x1862456979142400:12345-192.168.204.3@tcp:400:kworker/u8:2.0 Request processed in 41us (2496us total) trans 0 rc 0/0 00000100:00100000:2.0:1776178736.420131:0:8250:0:(nrs_fifo.c:241:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.204.3@tcp, seq: 1240 00000100:00100000:2.0:1776178736.420144:0:8250:0:(nrs_fifo.c:179:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.204.3@tcp, seq: 7040 00000100:00100000:2.0:1776178736.420148:0:8250:0:(service.c:2324:ptlrpc_server_handle_request()) Handling RPC req@00000000eeeaf4ad pname:cluuid+ref:pid:xid:nid:opc:job mdt00_003:0+-99:2269:x1862456979142336:12345-192.168.204.3@tcp:38:kworker/u8:2.0 00010000:02000000:2.0:1776178736.420163:0:8250:0:(ldlm_lib.c:1139:target_handle_connect()) lustre-MDT0000: Not available for connect from 192.168.204.3@tcp (stopping) 00000100:00100000:2.0:1776178736.420188:0:8250:0:(service.c:2372:ptlrpc_server_handle_request()) Handled RPC req@00000000eeeaf4ad pname:cluuid+ref:pid:xid:nid:opc:job mdt00_003:0+-99:2269:x1862456979142336:12345-192.168.204.3@tcp:38:kworker/u8:2.0 Request processed in 39us (2791us total) trans 0 rc -19/-19 00000100:00100000:2.0:1776178736.420194:0:8250:0:(nrs_fifo.c:241:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.204.3@tcp, seq: 7040 00010000:00010000:1.0:1776178736.610461:0:9225:0:(ldlm_lockd.c:2792:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00010000:00010000:2.0:1776178737.632897:0:6244:0:(ldlm_lockd.c:2792:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00080000:0.0:1776178738.016271:0:36:0:(pinger.c:251:ptlrpc_pinger_process_import()) lustre-MDT0000-lwp-OST0000_UUID->lustre-MDT0000_UUID: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00080000:0.0:1776178738.016279:0:36:0:(recover.c:57:ptlrpc_initiate_recovery()) lustre-MDT0000_UUID: starting recovery 00000100:00080000:0.0:1776178738.016282:0:36:0:(import.c:86:import_set_state_nolock()) 0000000047601a98 lustre-MDT0000_UUID: changing import state from DISCONN to CONNECTING 00000100:00080000:0.0:1776178738.016291:0:36:0:(import.c:534:import_select_connection()) lustre-MDT0000-lwp-OST0000: connect to NID 0@lo last attempt 1440 00000100:00080000:0.0:1776178738.016296:0:36:0:(import.c:581:import_select_connection()) lustre-MDT0000-lwp-OST0000: tried all connections, increasing latency to 6s 00000100:00080000:0.0:1776178738.016303:0:36:0:(import.c:615:import_select_connection()) lustre-MDT0000-lwp-OST0000: import 0000000047601a98 using connection 192.168.204.103@tcp/0@lo 00000100:00100000:0.0:1776178738.016328:0:36:0:(import.c:807:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5) req@00000000b206023d x1862456998122816/t0(0) o38->lustre-MDT0000-lwp-OST0000@0@lo:12/10 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' 00000100:00080000:0.0:1776178738.016361:0:36:0:(pinger.c:251:ptlrpc_pinger_process_import()) lustre-MDT0000-lwp-OST0001_UUID->lustre-MDT0000_UUID: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00080000:0.0:1776178738.016364:0:36:0:(recover.c:57:ptlrpc_initiate_recovery()) lustre-MDT0000_UUID: starting recovery 00000100:00080000:0.0:1776178738.016366:0:36:0:(import.c:86:import_set_state_nolock()) 00000000dfc8db13 lustre-MDT0000_UUID: changing import state from DISCONN to CONNECTING 00000100:00080000:0.0:1776178738.016368:0:36:0:(import.c:534:import_select_connection()) lustre-MDT0000-lwp-OST0001: connect to NID 0@lo last attempt 1440 00000100:00080000:0.0:1776178738.016370:0:36:0:(import.c:581:import_select_connection()) lustre-MDT0000-lwp-OST0001: tried all connections, increasing latency to 6s 00000100:00080000:0.0:1776178738.016373:0:36:0:(import.c:615:import_select_connection()) lustre-MDT0000-lwp-OST0001: import 00000000dfc8db13 using connection 192.168.204.103@tcp/0@lo 00000100:00100000:0.0:1776178738.016382:0:36:0:(import.c:807:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5) req@00000000a942a398 x1862456998122944/t0(0) o38->lustre-MDT0000-lwp-OST0001@0@lo:12/10 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' 00000100:00080000:0.0:1776178738.016410:0:36:0:(pinger.c:251:ptlrpc_pinger_process_import()) lustre-MDT0001-mdtlov_UUID->lustre-MDT0000_UUID: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00080000:0.0:1776178738.016413:0:36:0:(recover.c:57:ptlrpc_initiate_recovery()) lustre-MDT0000_UUID: starting recovery 00000100:00080000:0.0:1776178738.016414:0:36:0:(import.c:86:import_set_state_nolock()) 00000000ba4f1821 lustre-MDT0000_UUID: changing import state from DISCONN to CONNECTING 00000100:00080000:0.0:1776178738.016417:0:36:0:(import.c:534:import_select_connection()) lustre-MDT0000-osp-MDT0001: connect to NID 0@lo last attempt 1440 00000100:00080000:0.0:1776178738.016419:0:36:0:(import.c:581:import_select_connection()) lustre-MDT0000-osp-MDT0001: tried all connections, increasing latency to 6s 00000100:00080000:0.0:1776178738.016422:0:36:0:(import.c:615:import_select_connection()) lustre-MDT0000-osp-MDT0001: import 00000000ba4f1821 using connection 192.168.204.103@tcp/0@lo 00000100:00100000:0.0:1776178738.016433:0:36:0:(import.c:807:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5) req@00000000de8350d4 x1862456998123072/t0(0) o38->lustre-MDT0000-osp-MDT0001@0@lo:24/4 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' 00000100:00080000:0.0:1776178738.016452:0:36:0:(pinger.c:251:ptlrpc_pinger_process_import()) lustre-MDT0000-lwp-MDT0001_UUID->lustre-MDT0000_UUID: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00080000:0.0:1776178738.016455:0:36:0:(recover.c:57:ptlrpc_initiate_recovery()) lustre-MDT0000_UUID: starting recovery 00000100:00080000:0.0:1776178738.016456:0:36:0:(import.c:86:import_set_state_nolock()) 00000000a7c55b1f lustre-MDT0000_UUID: changing import state from DISCONN to CONNECTING 00000100:00080000:0.0:1776178738.016459:0:36:0:(import.c:534:import_select_connection()) lustre-MDT0000-lwp-MDT0001: connect to NID 0@lo last attempt 1440 00000100:00080000:0.0:1776178738.016460:0:36:0:(import.c:581:import_select_connection()) lustre-MDT0000-lwp-MDT0001: tried all connections, increasing latency to 6s 00000100:00080000:0.0:1776178738.016463:0:36:0:(import.c:615:import_select_connection()) lustre-MDT0000-lwp-MDT0001: import 00000000a7c55b1f using connection 192.168.204.103@tcp/0@lo 00000100:00100000:0.0:1776178738.016471:0:36:0:(import.c:807:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5) req@00000000e1d307fb x1862456998123264/t0(0) o38->lustre-MDT0000-lwp-MDT0001@0@lo:12/10 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' 00000100:00100000:0.0:1776178738.016504:0:3355:0:(client.c:1733:ptlrpc_send_new_req()) Sending RPC req@0000000025b3b8e0 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_02:lustre-MDT0001-lwp-OST0001_UUID:3355:1862456998123008:0@lo:400:kworker/u8:1.0 00000100:00100000:0.0:1776178738.016578:0:3355:0:(events.c:361:request_in_callback()) peer: 12345-0@lo (source: 12345-0@lo) 00000100:00100000:0.0:1776178738.016614:0:3355:0:(ptlrpcd.c:415:ptlrpcd_check()) transfer 1 async RPCs [3->2] 00000100:00100000:0.0:1776178738.016618:0:3355:0:(client.c:1733:ptlrpc_send_new_req()) Sending RPC req@0000000081aacc4a pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_02:lustre-MDT0001-mdtlov_UUID:3355:1862456998123136:0@lo:400:kworker/u8:1.0 00000100:00100000:0.0:1776178738.016644:0:3355:0:(events.c:361:request_in_callback()) peer: 12345-0@lo (source: 12345-0@lo) 00000100:00100000:0.0:1776178738.016697:0:6256:0:(service.c:2152:ptlrpc_server_handle_req_in()) got req x1862456998123008 00000100:00100000:0.0:1776178738.016743:0:6256:0:(nrs_fifo.c:179:nrs_fifo_req_get()) NRS start fifo request from 12345-0@lo, seq: 1241 00000100:00100000:0.0:1776178738.016749:0:6256:0:(service.c:2324:ptlrpc_server_handle_request()) Handling RPC req@000000009d7a00b9 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_001:lustre-MDT0001-lwp-OST0001_UUID+5:3355:x1862456998123008:12345-0@lo:400:kworker/u8:1.0 00000100:00100000:0.0:1776178738.016794:0:6256:0:(service.c:2372:ptlrpc_server_handle_request()) Handled RPC req@000000009d7a00b9 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_001:lustre-MDT0001-lwp-OST0001_UUID+5:3355:x1862456998123008:12345-0@lo:400:kworker/u8:1.0 Request processed in 44us (218us total) trans 0 rc 0/0 00000100:00100000:0.0:1776178738.016801:0:6256:0:(nrs_fifo.c:241:nrs_fifo_req_stop()) NRS stop fifo request from 12345-0@lo, seq: 1241 00000100:00100000:0.0:1776178738.016841:0:3355:0:(client.c:2215:ptlrpc_check_set()) Completed RPC req@0000000025b3b8e0 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_02:lustre-MDT0001-lwp-OST0001_UUID:3355:1862456998123008:0@lo:400:kworker/u8:1.0 00000100:00100000:2.0:1776178738.016935:0:3353:0:(client.c:1733:ptlrpc_send_new_req()) Sending RPC req@00000000f1ed5d03 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_00:ac61f839-0e8e-4cac-9c59-08bcfbbad330:3353:1862456998122752:0@lo:400:kworker/u8:1.0 00000100:00100000:2.0:1776178738.016968:0:3353:0:(events.c:361:request_in_callback()) peer: 12345-0@lo (source: 12345-0@lo) 00000100:00100000:2.0:1776178738.016994:0:3353:0:(client.c:1733:ptlrpc_send_new_req()) Sending RPC req@0000000039cf69e0 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_00:lustre-MDT0001-mdtlov_UUID:3353:1862456998123200:0@lo:400:kworker/u8:1.0 00000100:00100000:2.0:1776178738.017016:0:3353:0:(events.c:361:request_in_callback()) peer: 12345-0@lo (source: 12345-0@lo) 00000100:00100000:2.0:1776178738.017031:0:3353:0:(ptlrpcd.c:415:ptlrpcd_check()) transfer 1 async RPCs [1->0] 00000100:00100000:2.0:1776178738.017036:0:3353:0:(client.c:1733:ptlrpc_send_new_req()) Sending RPC req@000000003b0b9a0e pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_00:lustre-MDT0001-lwp-OST0000_UUID:3353:1862456998122880:0@lo:400:kworker/u8:1.0 00000100:00100000:2.0:1776178738.017055:0:3353:0:(events.c:361:request_in_callback()) peer: 12345-0@lo (source: 12345-0@lo) 00000100:00100000:2.0:1776178738.017105:0:20755:0:(service.c:2152:ptlrpc_server_handle_req_in()) got req x1862456998123136 00000100:00100000:2.0:1776178738.017132:0:20755:0:(service.c:2152:ptlrpc_server_handle_req_in()) got req x1862456998123200 00000100:00100000:2.0:1776178738.017146:0:20755:0:(nrs_fifo.c:179:nrs_fifo_req_get()) NRS start fifo request from 12345-0@lo, seq: 929 00000100:00100000:2.0:1776178738.017150:0:20755:0:(service.c:2324:ptlrpc_server_handle_request()) Handling RPC req@00000000f44a2faf pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_006:lustre-MDT0001-mdtlov_UUID+5:3355:x1862456998123136:12345-0@lo:400:kworker/u8:1.0 00000100:00100000:2.0:1776178738.017181:0:20755:0:(service.c:2372:ptlrpc_server_handle_request()) Handled RPC req@00000000f44a2faf pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_006:lustre-MDT0001-mdtlov_UUID+5:3355:x1862456998123136:12345-0@lo:400:kworker/u8:1.0 Request processed in 30us (536us total) trans 0 rc 0/0 00000100:00100000:2.0:1776178738.017189:0:20755:0:(nrs_fifo.c:241:nrs_fifo_req_stop()) NRS stop fifo request from 12345-0@lo, seq: 929 00000100:00100000:2.0:1776178738.017200:0:20755:0:(nrs_fifo.c:179:nrs_fifo_req_get()) NRS start fifo request from 12345-0@lo, seq: 930 00000100:00100000:2.0:1776178738.017204:0:20755:0:(service.c:2324:ptlrpc_server_handle_request()) Handling RPC req@0000000042de09e9 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_006:lustre-MDT0001-mdtlov_UUID+5:3353:x1862456998123200:12345-0@lo:400:kworker/u8:1.0 00000100:00100000:2.0:1776178738.017232:0:20755:0:(service.c:2372:ptlrpc_server_handle_request()) Handled RPC req@0000000042de09e9 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_006:lustre-MDT0001-mdtlov_UUID+5:3353:x1862456998123200:12345-0@lo:400:kworker/u8:1.0 Request processed in 27us (216us total) trans 0 rc 0/0 00000100:00100000:2.0:1776178738.017236:0:20755:0:(nrs_fifo.c:241:nrs_fifo_req_stop()) NRS stop fifo request from 12345-0@lo, seq: 930 00000100:00100000:2.0:1776178738.017261:0:3353:0:(client.c:2215:ptlrpc_check_set()) Completed RPC req@0000000039cf69e0 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_00:lustre-MDT0001-mdtlov_UUID:3353:1862456998123200:0@lo:400:kworker/u8:1.0 00000100:00100000:0.0:1776178738.017480:0:20140:0:(service.c:2152:ptlrpc_server_handle_req_in()) got req x1862456998122752 00000100:00100000:0.0:1776178738.017514:0:20140:0:(nrs_fifo.c:179:nrs_fifo_req_get()) NRS start fifo request from 12345-0@lo, seq: 364 00000100:00100000:0.0:1776178738.017517:0:20140:0:(service.c:2324:ptlrpc_server_handle_request()) Handling RPC req@000000006fd17377 pname:cluuid+ref:pid:xid:nid:opc:job ll_mgs_0002:ac61f839-0e8e-4cac-9c59-08bcfbbad330+18:3353:x1862456998122752:12345-0@lo:400:kworker/u8:1.0 00000100:00100000:0.0:1776178738.017566:0:20140:0:(service.c:2372:ptlrpc_server_handle_request()) Handled RPC req@000000006fd17377 pname:cluuid+ref:pid:xid:nid:opc:job ll_mgs_0002:ac61f839-0e8e-4cac-9c59-08bcfbbad330+18:3353:x1862456998122752:12345-0@lo:400:kworker/u8:1.0 Request processed in 48us (598us total) trans 0 rc 0/0 00000100:00100000:0.0:1776178738.017572:0:20140:0:(nrs_fifo.c:241:nrs_fifo_req_stop()) NRS stop fifo request from 12345-0@lo, seq: 364 00000100:00100000:0.0:1776178738.017603:0:3353:0:(client.c:2215:ptlrpc_check_set()) Completed RPC req@00000000f1ed5d03 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_00:ac61f839-0e8e-4cac-9c59-08bcfbbad330:3353:1862456998122752:0@lo:400:kworker/u8:1.0 00000100:00100000:0.0:1776178738.017623:0:3355:0:(client.c:2215:ptlrpc_check_set()) Completed RPC req@0000000081aacc4a pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_02:lustre-MDT0001-mdtlov_UUID:3355:1862456998123136:0@lo:400:kworker/u8:1.0 00000100:00100000:0.0:1776178738.017641:0:6257:0:(service.c:2152:ptlrpc_server_handle_req_in()) got req x1862456998122880 00000100:00100000:0.0:1776178738.017824:0:6257:0:(nrs_fifo.c:179:nrs_fifo_req_get()) NRS start fifo request from 12345-0@lo, seq: 1242 00000100:00100000:0.0:1776178738.017828:0:6257:0:(service.c:2324:ptlrpc_server_handle_request()) Handling RPC req@0000000004dd3b9b pname:cluuid+ref:pid:xid:nid:opc:job mdt00_002:lustre-MDT0001-lwp-OST0000_UUID+5:3353:x1862456998122880:12345-0@lo:400:kworker/u8:1.0 00000100:00100000:0.0:1776178738.017863:0:6257:0:(service.c:2372:ptlrpc_server_handle_request()) Handled RPC req@0000000004dd3b9b pname:cluuid+ref:pid:xid:nid:opc:job mdt00_002:lustre-MDT0001-lwp-OST0000_UUID+5:3353:x1862456998122880:12345-0@lo:400:kworker/u8:1.0 Request processed in 34us (808us total) trans 0 rc 0/0 00000100:00100000:0.0:1776178738.017868:0:6257:0:(nrs_fifo.c:241:nrs_fifo_req_stop()) NRS stop fifo request from 12345-0@lo, seq: 1242 00000100:00100000:0.0:1776178738.017892:0:3353:0:(client.c:2215:ptlrpc_check_set()) Completed RPC req@000000003b0b9a0e pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_00:lustre-MDT0001-lwp-OST0000_UUID:3353:1862456998122880:0@lo:400:kworker/u8:1.0 00000100:00100000:3.0:1776178738.018094:0:3352:0:(client.c:1733:ptlrpc_send_new_req()) Sending RPC req@00000000b206023d pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:lustre-MDT0000-lwp-OST0000_UUID:3352:1862456998122816:0@lo:38:kworker/u8:1.0 00000100:00100000:3.0:1776178738.018128:0:3352:0:(events.c:361:request_in_callback()) peer: 12345-0@lo (source: 12345-0@lo) 00000100:00100000:3.0:1776178738.018150:0:3352:0:(client.c:1733:ptlrpc_send_new_req()) Sending RPC req@00000000a942a398 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:lustre-MDT0000-lwp-OST0001_UUID:3352:1862456998122944:0@lo:38:kworker/u8:1.0 00000100:00100000:3.0:1776178738.018170:0:3352:0:(events.c:361:request_in_callback()) peer: 12345-0@lo (source: 12345-0@lo) 00000100:00100000:3.0:1776178738.018187:0:3352:0:(client.c:1733:ptlrpc_send_new_req()) Sending RPC req@00000000de8350d4 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:lustre-MDT0001-mdtlov_UUID:3352:1862456998123072:0@lo:38:kworker/u8:1.0 00000100:00100000:3.0:1776178738.018207:0:3352:0:(events.c:361:request_in_callback()) peer: 12345-0@lo (source: 12345-0@lo) 00000100:00100000:3.0:1776178738.018221:0:3352:0:(client.c:1733:ptlrpc_send_new_req()) Sending RPC req@00000000e1d307fb pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:lustre-MDT0000-lwp-MDT0001_UUID:3352:1862456998123264:0@lo:38:kworker/u8:1.0 00000100:00100000:3.0:1776178738.018238:0:3352:0:(events.c:361:request_in_callback()) peer: 12345-0@lo (source: 12345-0@lo) 00000100:00100000:3.0:1776178738.018263:0:15213:0:(service.c:2152:ptlrpc_server_handle_req_in()) got req x1862456998123072 00000100:00100000:3.0:1776178738.018281:0:15213:0:(nrs_fifo.c:179:nrs_fifo_req_get()) NRS start fifo request from 12345-0@lo, seq: 812 00000100:00100000:3.0:1776178738.018285:0:15213:0:(service.c:2324:ptlrpc_server_handle_request()) Handling RPC req@00000000c37c187e pname:cluuid+ref:pid:xid:nid:opc:job mdt_out00_003:0+-99:3352:x1862456998123072:12345-0@lo:38:kworker/u8:1.0 00010000:02000000:3.0:1776178738.018298:0:15213:0:(ldlm_lib.c:1139:target_handle_connect()) lustre-MDT0000: Not available for connect from 0@lo (stopping) 00000100:00100000:3.0:1776178738.018327:0:15213:0:(service.c:2372:ptlrpc_server_handle_request()) Handled RPC req@00000000c37c187e pname:cluuid+ref:pid:xid:nid:opc:job mdt_out00_003:0+-99:3352:x1862456998123072:12345-0@lo:38:kworker/u8:1.0 Request processed in 41us (119us total) trans 0 rc -19/-19 00000100:00100000:3.0:1776178738.018333:0:15213:0:(nrs_fifo.c:241:nrs_fifo_req_stop()) NRS stop fifo request from 12345-0@lo, seq: 812 00000100:00080000:3.0:1776178738.018367:0:3352:0:(import.c:86:import_set_state_nolock()) 00000000ba4f1821 lustre-MDT0000_UUID: changing import state from CONNECTING to DISCONN 00000100:00080000:3.0:1776178738.018371:0:3352:0:(import.c:1429:ptlrpc_connect_interpret()) recovery of lustre-MDT0000_UUID on 192.168.204.103@tcp failed (-19) 00000100:00100000:3.0:1776178738.018374:0:3352:0:(client.c:2215:ptlrpc_check_set()) Completed RPC req@00000000de8350d4 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:lustre-MDT0001-mdtlov_UUID:3352:1862456998123072:0@lo:38:kworker/u8:1.0 00000100:00100000:2.0:1776178738.018444:0:8250:0:(service.c:2152:ptlrpc_server_handle_req_in()) got req x1862456998122816 00000100:00100000:2.0:1776178738.018468:0:8250:0:(service.c:2152:ptlrpc_server_handle_req_in()) got req x1862456998122944 00000100:00100000:2.0:1776178738.018482:0:8250:0:(service.c:2152:ptlrpc_server_handle_req_in()) got req x1862456998123264 00000100:00100000:2.0:1776178738.018492:0:8250:0:(nrs_fifo.c:179:nrs_fifo_req_get()) NRS start fifo request from 12345-0@lo, seq: 7041 00000100:00100000:2.0:1776178738.018496:0:8250:0:(service.c:2324:ptlrpc_server_handle_request()) Handling RPC req@00000000ffd05959 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_003:0+-99:3352:x1862456998122816:12345-0@lo:38:kworker/u8:1.0 00010000:02000000:2.0:1776178738.018503:0:8250:0:(ldlm_lib.c:1139:target_handle_connect()) lustre-MDT0000: Not available for connect from 0@lo (stopping) 00000100:00100000:2.0:1776178738.018541:0:8250:0:(service.c:2372:ptlrpc_server_handle_request()) Handled RPC req@00000000ffd05959 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_003:0+-99:3352:x1862456998122816:12345-0@lo:38:kworker/u8:1.0 Request processed in 44us (413us total) trans 0 rc -19/-19 00000100:00100000:2.0:1776178738.018547:0:8250:0:(nrs_fifo.c:241:nrs_fifo_req_stop()) NRS stop fifo request from 12345-0@lo, seq: 7041 00000100:00100000:2.0:1776178738.018557:0:8250:0:(nrs_fifo.c:179:nrs_fifo_req_get()) NRS start fifo request from 12345-0@lo, seq: 7042 00000100:00100000:2.0:1776178738.018560:0:8250:0:(service.c:2324:ptlrpc_server_handle_request()) Handling RPC req@000000007309261c pname:cluuid+ref:pid:xid:nid:opc:job mdt00_003:0+-99:3352:x1862456998122944:12345-0@lo:38:kworker/u8:1.0 00010000:02000000:2.0:1776178738.018566:0:8250:0:(ldlm_lib.c:1139:target_handle_connect()) lustre-MDT0000: Not available for connect from 0@lo (stopping) 00000100:00100000:2.0:1776178738.018585:0:8250:0:(service.c:2372:ptlrpc_server_handle_request()) Handled RPC req@000000007309261c pname:cluuid+ref:pid:xid:nid:opc:job mdt00_003:0+-99:3352:x1862456998122944:12345-0@lo:38:kworker/u8:1.0 Request processed in 23us (415us total) trans 0 rc -19/-19 00000100:00100000:2.0:1776178738.018589:0:8250:0:(nrs_fifo.c:241:nrs_fifo_req_stop()) NRS stop fifo request from 12345-0@lo, seq: 7042 00000100:00100000:2.0:1776178738.018598:0:8250:0:(nrs_fifo.c:179:nrs_fifo_req_get()) NRS start fifo request from 12345-0@lo, seq: 7043 00000100:00100000:2.0:1776178738.018600:0:8250:0:(service.c:2324:ptlrpc_server_handle_request()) Handling RPC req@0000000057946847 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_003:0+-99:3352:x1862456998123264:12345-0@lo:38:kworker/u8:1.0 00010000:02000000:2.0:1776178738.018605:0:8250:0:(ldlm_lib.c:1139:target_handle_connect()) lustre-MDT0000: Not available for connect from 0@lo (stopping) 00000100:00100000:2.0:1776178738.018622:0:8250:0:(service.c:2372:ptlrpc_server_handle_request()) Handled RPC req@0000000057946847 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_003:0+-99:3352:x1862456998123264:12345-0@lo:38:kworker/u8:1.0 Request processed in 20us (384us total) trans 0 rc -19/-19 00000100:00100000:2.0:1776178738.018627:0:8250:0:(nrs_fifo.c:241:nrs_fifo_req_stop()) NRS stop fifo request from 12345-0@lo, seq: 7043 00000100:00080000:3.0:1776178738.020532:0:3352:0:(import.c:86:import_set_state_nolock()) 0000000047601a98 lustre-MDT0000_UUID: changing import state from CONNECTING to DISCONN 00000100:00080000:3.0:1776178738.020542:0:3352:0:(import.c:1429:ptlrpc_connect_interpret()) recovery of lustre-MDT0000_UUID on 192.168.204.103@tcp failed (-19) 00000100:00100000:3.0:1776178738.020549:0:3352:0:(client.c:2215:ptlrpc_check_set()) Completed RPC req@00000000b206023d pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:lustre-MDT0000-lwp-OST0000_UUID:3352:1862456998122816:0@lo:38:kworker/u8:1.0 00000100:00080000:3.0:1776178738.020568:0:3352:0:(import.c:86:import_set_state_nolock()) 00000000dfc8db13 lustre-MDT0000_UUID: changing import state from CONNECTING to DISCONN 00000100:00080000:3.0:1776178738.020571:0:3352:0:(import.c:1429:ptlrpc_connect_interpret()) recovery of lustre-MDT0000_UUID on 192.168.204.103@tcp failed (-19) 00000100:00100000:3.0:1776178738.020573:0:3352:0:(client.c:2215:ptlrpc_check_set()) Completed RPC req@00000000a942a398 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:lustre-MDT0000-lwp-OST0001_UUID:3352:1862456998122944:0@lo:38:kworker/u8:1.0 00000100:00080000:3.0:1776178738.020581:0:3352:0:(import.c:86:import_set_state_nolock()) 00000000a7c55b1f lustre-MDT0000_UUID: changing import state from CONNECTING to DISCONN 00000100:00080000:3.0:1776178738.020587:0:3352:0:(import.c:1429:ptlrpc_connect_interpret()) recovery of lustre-MDT0000_UUID on 192.168.204.103@tcp failed (-19) 00000100:00100000:3.0:1776178738.020590:0:3352:0:(client.c:2215:ptlrpc_check_set()) Completed RPC req@00000000e1d307fb pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:lustre-MDT0000-lwp-MDT0001_UUID:3352:1862456998123264:0@lo:38:kworker/u8:1.0 00010000:00010000:1.0:1776178738.656867:0:6245:0:(ldlm_lockd.c:2792:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:0.0:1776178739.552928:0:3353:0:(ptlrpcd.c:415:ptlrpcd_check()) transfer 1 async RPCs [1->0] 00000100:00100000:0.0:1776178739.552948:0:3353:0:(client.c:1733:ptlrpc_send_new_req()) Sending RPC req@00000000d0d837c5 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_00:lustre-MDT0001-mdtlov_UUID:3353:1862456998123328:0@lo:13:osp-pre-1-1.0 00000100:00100000:0.0:1776178739.553016:0:3353:0:(events.c:361:request_in_callback()) peer: 12345-0@lo (source: 12345-0@lo) 00000100:00100000:1.0:1776178739.557790:0:8231:0:(service.c:2152:ptlrpc_server_handle_req_in()) got req x1862456998123328 00000100:00100000:1.0:1776178739.557860:0:8231:0:(nrs_fifo.c:179:nrs_fifo_req_get()) NRS start fifo request from 12345-0@lo, seq: 697 00000100:00100000:1.0:1776178739.557867:0:8231:0:(service.c:2324:ptlrpc_server_handle_request()) Handling RPC req@000000003b436fee pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_create00:lustre-MDT0001-mdtlov_UUID+5:3353:x1862456998123328:12345-0@lo:13:osp-pre-1-1.0 00002000:00000024:1.0:1776178739.557906:0:8231:0:(ofd_obd.c:740:ofd_statfs()) blocks cached 229376 granted 491746368 pending 0 free 3923689472 avail 3692167168 00000100:00100000:1.0:1776178739.557988:0:8231:0:(service.c:2372:ptlrpc_server_handle_request()) Handled RPC req@000000003b436fee pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_create00:lustre-MDT0001-mdtlov_UUID+5:3353:x1862456998123328:12345-0@lo:13:osp-pre-1-1.0 Request processed in 120us (4972us total) trans 0 rc 0/0 00000100:00100000:1.0:1776178739.557995:0:8231:0:(nrs_fifo.c:241:nrs_fifo_req_stop()) NRS stop fifo request from 12345-0@lo, seq: 697 00000100:00100000:2.0:1776178739.558112:0:3353:0:(client.c:2215:ptlrpc_check_set()) Completed RPC req@00000000d0d837c5 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_00:lustre-MDT0001-mdtlov_UUID:3353:1862456998123328:0@lo:13:osp-pre-1-1.0 00010000:00010000:2.0:1776178739.680367:0:9224:0:(ldlm_lockd.c:2792:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:3.0:1776178740.064399:0:3356:0:(ptlrpcd.c:415:ptlrpcd_check()) transfer 1 async RPCs [2->3] 00000100:00100000:3.0:1776178740.064424:0:3356:0:(client.c:1733:ptlrpc_send_new_req()) Sending RPC req@00000000d0d837c5 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_03:lustre-MDT0001-mdtlov_UUID:3356:1862456998123392:0@lo:13:osp-pre-0-1.0 00000100:00100000:3.0:1776178740.064502:0:3356:0:(events.c:361:request_in_callback()) peer: 12345-0@lo (source: 12345-0@lo) 00000100:00100000:2.0:1776178740.064588:0:9554:0:(service.c:2152:ptlrpc_server_handle_req_in()) got req x1862456998123392 00000100:00100000:2.0:1776178740.064637:0:9554:0:(nrs_fifo.c:179:nrs_fifo_req_get()) NRS start fifo request from 12345-0@lo, seq: 698 00000100:00100000:2.0:1776178740.064643:0:9554:0:(service.c:2324:ptlrpc_server_handle_request()) Handling RPC req@00000000196ec025 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_create00:lustre-MDT0001-mdtlov_UUID+5:3356:x1862456998123392:12345-0@lo:13:osp-pre-0-1.0 00002000:00000024:2.0:1776178740.064671:0:9554:0:(ofd_obd.c:740:ofd_statfs()) blocks cached 315392 granted 484883776 pending 0 free 3923689472 avail 3692167168 00000100:00100000:2.0:1776178740.064722:0:9554:0:(service.c:2372:ptlrpc_server_handle_request()) Handled RPC req@00000000196ec025 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_create00:lustre-MDT0001-mdtlov_UUID+5:3356:x1862456998123392:12345-0@lo:13:osp-pre-0-1.0 Request processed in 77us (220us total) trans 0 rc 0/0 00000100:00100000:2.0:1776178740.064730:0:9554:0:(nrs_fifo.c:241:nrs_fifo_req_stop()) NRS stop fifo request from 12345-0@lo, seq: 698 00000100:00100000:3.0:1776178740.064814:0:3356:0:(client.c:2215:ptlrpc_check_set()) Completed RPC req@00000000d0d837c5 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_03:lustre-MDT0001-mdtlov_UUID:3356:1862456998123392:0@lo:13:osp-pre-0-1.0 00010000:00010000:1.0:1776178740.707228:0:9225:0:(ldlm_lockd.c:2792:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:3.0:1776178741.539052:0:3347:0:(events.c:361:request_in_callback()) peer: 12345-192.168.204.3@tcp (source: 12345-192.168.204.3@tcp) 00000100:00100000:3.0:1776178741.539181:0:3347:0:(events.c:361:request_in_callback()) peer: 12345-192.168.204.3@tcp (source: 12345-192.168.204.3@tcp) 00000100:00100000:3.0:1776178741.539221:0:3347:0:(events.c:361:request_in_callback()) peer: 12345-192.168.204.3@tcp (source: 12345-192.168.204.3@tcp) 00000100:00100000:1.0:1776178741.540130:0:6255:0:(service.c:2152:ptlrpc_server_handle_req_in()) got req x1862456979142656 00000100:00100000:2.0:1776178741.540270:0:20137:0:(service.c:2152:ptlrpc_server_handle_req_in()) got req x1862456979142592 00000100:00100000:2.0:1776178741.540310:0:20137:0:(nrs_fifo.c:179:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.204.3@tcp, seq: 365 00000100:00100000:2.0:1776178741.540315:0:20137:0:(service.c:2324:ptlrpc_server_handle_request()) Handling RPC req@00000000eb350ccb pname:cluuid+ref:pid:xid:nid:opc:job ll_mgs_0000:5b5c66b1-1a2f-4651-8548-0c75b9a38a94+9:2272:x1862456979142592:12345-192.168.204.3@tcp:400:kworker/u8:4.0 00000100:00100000:2.0:1776178741.540387:0:20137:0:(service.c:2372:ptlrpc_server_handle_request()) Handled RPC req@00000000eb350ccb pname:cluuid+ref:pid:xid:nid:opc:job ll_mgs_0000:5b5c66b1-1a2f-4651-8548-0c75b9a38a94+9:2272:x1862456979142592:12345-192.168.204.3@tcp:400:kworker/u8:4.0 Request processed in 71us (1345us total) trans 0 rc 0/0 00000100:00100000:2.0:1776178741.540395:0:20137:0:(nrs_fifo.c:241:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.204.3@tcp, seq: 365 00000100:00100000:1.0:1776178741.542833:0:6255:0:(service.c:2152:ptlrpc_server_handle_req_in()) got req x1862456979142720 00000100:00100000:2.0:1776178741.542986:0:3346:0:(events.c:361:request_in_callback()) peer: 12345-192.168.204.3@tcp (source: 12345-192.168.204.3@tcp) 00000100:00100000:2.0:1776178741.543150:0:3346:0:(events.c:361:request_in_callback()) peer: 12345-192.168.204.3@tcp (source: 12345-192.168.204.3@tcp) 00000100:00100000:2.0:1776178741.543198:0:20762:0:(service.c:2152:ptlrpc_server_handle_req_in()) got req x1862456979142848 00000100:00100000:2.0:1776178741.543223:0:20762:0:(service.c:2152:ptlrpc_server_handle_req_in()) got req x1862456979142784 00000100:00100000:2.0:1776178741.543239:0:20762:0:(nrs_fifo.c:179:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.204.3@tcp, seq: 931 00000100:00100000:2.0:1776178741.543246:0:20762:0:(service.c:2324:ptlrpc_server_handle_request()) Handling RPC req@00000000d0d837c5 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_013:b50963bd-8eba-4432-9a54-8888d00e3fa1+9:2273:x1862456979142848:12345-192.168.204.3@tcp:400:kworker/u8:4.0 00000100:00100000:2.0:1776178741.543307:0:20762:0:(service.c:2372:ptlrpc_server_handle_request()) Handled RPC req@00000000d0d837c5 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_013:b50963bd-8eba-4432-9a54-8888d00e3fa1+9:2273:x1862456979142848:12345-192.168.204.3@tcp:400:kworker/u8:4.0 Request processed in 61us (325us total) trans 0 rc 0/0 00000100:00100000:2.0:1776178741.543318:0:20762:0:(nrs_fifo.c:241:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.204.3@tcp, seq: 931 00000100:00100000:2.0:1776178741.543333:0:20762:0:(nrs_fifo.c:179:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.204.3@tcp, seq: 932 00000100:00100000:2.0:1776178741.543337:0:20762:0:(service.c:2324:ptlrpc_server_handle_request()) Handling RPC req@0000000081d550e6 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_013:b50963bd-8eba-4432-9a54-8888d00e3fa1+8:2272:x1862456979142784:12345-192.168.204.3@tcp:400:kworker/u8:4.0 00000100:00100000:2.0:1776178741.543356:0:20762:0:(service.c:2372:ptlrpc_server_handle_request()) Handled RPC req@0000000081d550e6 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_013:b50963bd-8eba-4432-9a54-8888d00e3fa1+8:2272:x1862456979142784:12345-192.168.204.3@tcp:400:kworker/u8:4.0 Request processed in 18us (208us total) trans 0 rc 0/0 00000100:00100000:2.0:1776178741.543362:0:20762:0:(nrs_fifo.c:241:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.204.3@tcp, seq: 932 00000100:00100000:3.0:1776178741.546025:0:6256:0:(nrs_fifo.c:179:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.204.3@tcp, seq: 1243 00000100:00100000:3.0:1776178741.546039:0:6256:0:(service.c:2324:ptlrpc_server_handle_request()) Handling RPC req@00000000b63b7b84 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_001:b50963bd-8eba-4432-9a54-8888d00e3fa1+6:2272:x1862456979142720:12345-192.168.204.3@tcp:400:kworker/u8:4.0 00000100:00100000:3.0:1776178741.546267:0:6256:0:(service.c:2372:ptlrpc_server_handle_request()) Handled RPC req@00000000b63b7b84 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_001:b50963bd-8eba-4432-9a54-8888d00e3fa1+6:2272:x1862456979142720:12345-192.168.204.3@tcp:400:kworker/u8:4.0 Request processed in 229us (7044us total) trans 0 rc 0/0 00000100:00100000:3.0:1776178741.546275:0:6256:0:(nrs_fifo.c:241:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.204.3@tcp, seq: 1243 00000100:00100000:3.0:1776178741.546309:0:6256:0:(nrs_fifo.c:179:nrs_fifo_req_get()) NRS start fifo request from 12345-192.168.204.3@tcp, seq: 7044 00000100:00100000:3.0:1776178741.546313:0:6256:0:(service.c:2324:ptlrpc_server_handle_request()) Handling RPC req@0000000052a7c870 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_001:0+-99:2269:x1862456979142656:12345-192.168.204.3@tcp:38:kworker/u8:4.0 00010000:02000000:3.0:1776178741.546329:0:6256:0:(ldlm_lib.c:1139:target_handle_connect()) lustre-MDT0000: Not available for connect from 192.168.204.3@tcp (stopping) 00000100:00100000:3.0:1776178741.546370:0:6256:0:(service.c:2372:ptlrpc_server_handle_request()) Handled RPC req@0000000052a7c870 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_001:0+-99:2269:x1862456979142656:12345-192.168.204.3@tcp:38:kworker/u8:4.0 Request processed in 56us (7190us total) trans 0 rc -19/-19 00000100:00100000:3.0:1776178741.546376:0:6256:0:(nrs_fifo.c:241:nrs_fifo_req_stop()) NRS stop fifo request from 12345-192.168.204.3@tcp, seq: 7044 00010000:00010000:2.0:1776178741.732614:0:6244:0:(ldlm_lockd.c:2792:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00010000:00010000:1.0:1776178742.756674:0:6245:0:(ldlm_lockd.c:2792:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00080000:3.0:1776178743.143144:0:22249:0:(pinger.c:251:ptlrpc_pinger_process_import()) lustre-MDT0000-lwp-OST0000_UUID->lustre-MDT0000_UUID: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00080000:3.0:1776178743.143152:0:22249:0:(recover.c:57:ptlrpc_initiate_recovery()) lustre-MDT0000_UUID: starting recovery 00000100:00080000:3.0:1776178743.143156:0:22249:0:(import.c:86:import_set_state_nolock()) 0000000047601a98 lustre-MDT0000_UUID: changing import state from DISCONN to CONNECTING 00000100:00080000:3.0:1776178743.143164:0:22249:0:(import.c:534:import_select_connection()) lustre-MDT0000-lwp-OST0000: connect to NID 0@lo last attempt 1445 00000100:00080000:3.0:1776178743.143168:0:22249:0:(import.c:581:import_select_connection()) lustre-MDT0000-lwp-OST0000: tried all connections, increasing latency to 11s 00000100:00080000:3.0:1776178743.143174:0:22249:0:(import.c:615:import_select_connection()) lustre-MDT0000-lwp-OST0000: import 0000000047601a98 using connection 192.168.204.103@tcp/0@lo 00000100:00100000:3.0:1776178743.143196:0:22249:0:(import.c:807:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5) req@0000000037059d8f x1862456998123520/t0(0) o38->lustre-MDT0000-lwp-OST0000@0@lo:12/10 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' 00000100:00080000:3.0:1776178743.143229:0:22249:0:(pinger.c:251:ptlrpc_pinger_process_import()) lustre-MDT0000-lwp-OST0001_UUID->lustre-MDT0000_UUID: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00080000:3.0:1776178743.143232:0:22249:0:(recover.c:57:ptlrpc_initiate_recovery()) lustre-MDT0000_UUID: starting recovery 00000100:00080000:3.0:1776178743.143234:0:22249:0:(import.c:86:import_set_state_nolock()) 00000000dfc8db13 lustre-MDT0000_UUID: changing import state from DISCONN to CONNECTING 00000100:00080000:3.0:1776178743.143236:0:22249:0:(import.c:534:import_select_connection()) lustre-MDT0000-lwp-OST0001: connect to NID 0@lo last attempt 1445 00000100:00080000:3.0:1776178743.143238:0:22249:0:(import.c:581:import_select_connection()) lustre-MDT0000-lwp-OST0001: tried all connections, increasing latency to 11s 00000100:00080000:3.0:1776178743.143241:0:22249:0:(import.c:615:import_select_connection()) lustre-MDT0000-lwp-OST0001: import 00000000dfc8db13 using connection 192.168.204.103@tcp/0@lo 00000100:00100000:3.0:1776178743.143250:0:22249:0:(import.c:807:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5) req@0000000098e424d0 x1862456998123648/t0(0) o38->lustre-MDT0000-lwp-OST0001@0@lo:12/10 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' 00000100:00080000:3.0:1776178743.143262:0:22249:0:(pinger.c:251:ptlrpc_pinger_process_import()) lustre-MDT0001-mdtlov_UUID->lustre-MDT0000_UUID: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00080000:3.0:1776178743.143265:0:22249:0:(recover.c:57:ptlrpc_initiate_recovery()) lustre-MDT0000_UUID: starting recovery 00000100:00080000:3.0:1776178743.143266:0:22249:0:(import.c:86:import_set_state_nolock()) 00000000ba4f1821 lustre-MDT0000_UUID: changing import state from DISCONN to CONNECTING 00000100:00080000:3.0:1776178743.143269:0:22249:0:(import.c:534:import_select_connection()) lustre-MDT0000-osp-MDT0001: connect to NID 0@lo last attempt 1445 00000100:00080000:3.0:1776178743.143271:0:22249:0:(import.c:581:import_select_connection()) lustre-MDT0000-osp-MDT0001: tried all connections, increasing latency to 11s 00000100:00080000:3.0:1776178743.143274:0:22249:0:(import.c:615:import_select_connection()) lustre-MDT0000-osp-MDT0001: import 00000000ba4f1821 using connection 192.168.204.103@tcp/0@lo 00000100:00100000:3.0:1776178743.143283:0:22249:0:(import.c:807:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5) req@000000001f101832 x1862456998123776/t0(0) o38->lustre-MDT0000-osp-MDT0001@0@lo:24/4 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' 00000100:00080000:3.0:1776178743.143302:0:22249:0:(pinger.c:251:ptlrpc_pinger_process_import()) lustre-MDT0000-lwp-MDT0001_UUID->lustre-MDT0000_UUID: level DISCONN/3 force 0 force_next 0 deactive 0 pingable 1 suppress 0 00000100:00080000:3.0:1776178743.143304:0:22249:0:(recover.c:57:ptlrpc_initiate_recovery()) lustre-MDT0000_UUID: starting recovery 00000100:00080000:3.0:1776178743.143306:0:22249:0:(import.c:86:import_set_state_nolock()) 00000000a7c55b1f lustre-MDT0000_UUID: changing import state from DISCONN to CONNECTING 00000100:00080000:3.0:1776178743.143309:0:22249:0:(import.c:534:import_select_connection()) lustre-MDT0000-lwp-MDT0001: connect to NID 0@lo last attempt 1445 00000100:00080000:3.0:1776178743.143310:0:22249:0:(import.c:581:import_select_connection()) lustre-MDT0000-lwp-MDT0001: tried all connections, increasing latency to 11s 00000100:00080000:3.0:1776178743.143313:0:22249:0:(import.c:615:import_select_connection()) lustre-MDT0000-lwp-MDT0001: import 00000000a7c55b1f using connection 192.168.204.103@tcp/0@lo 00000100:00100000:3.0:1776178743.143320:0:22249:0:(import.c:807:ptlrpc_connect_import_locked()) @@@ (re)connect request (timeout 5) req@000000007d5cfd93 x1862456998123968/t0(0) o38->lustre-MDT0000-lwp-MDT0001@0@lo:12/10 lens 520/544 e 0 to 0 dl 0 ref 1 fl New:NQU/0/ffffffff rc 0/-1 job:'' 00000100:00100000:3.0:1776178743.143502:0:3352:0:(client.c:1733:ptlrpc_send_new_req()) Sending RPC req@0000000037059d8f pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:lustre-MDT0000-lwp-OST0000_UUID:3352:1862456998123520:0@lo:38:kworker/u8:0.0 00000100:00100000:3.0:1776178743.143568:0:3352:0:(events.c:361:request_in_callback()) peer: 12345-0@lo (source: 12345-0@lo) 00000100:00100000:3.0:1776178743.143601:0:3352:0:(client.c:1733:ptlrpc_send_new_req()) Sending RPC req@0000000098e424d0 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:lustre-MDT0000-lwp-OST0001_UUID:3352:1862456998123648:0@lo:38:kworker/u8:0.0 00000100:00100000:3.0:1776178743.143624:0:3352:0:(events.c:361:request_in_callback()) peer: 12345-0@lo (source: 12345-0@lo) 00000100:00100000:3.0:1776178743.143634:0:3352:0:(client.c:1733:ptlrpc_send_new_req()) Sending RPC req@000000001f101832 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:lustre-MDT0001-mdtlov_UUID:3352:1862456998123776:0@lo:38:kworker/u8:0.0 00000100:00100000:3.0:1776178743.143657:0:3352:0:(events.c:361:request_in_callback()) peer: 12345-0@lo (source: 12345-0@lo) 00000100:00100000:3.0:1776178743.143669:0:3352:0:(client.c:1733:ptlrpc_send_new_req()) Sending RPC req@000000007d5cfd93 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:lustre-MDT0000-lwp-MDT0001_UUID:3352:1862456998123968:0@lo:38:kworker/u8:0.0 00000100:00100000:3.0:1776178743.143688:0:3352:0:(events.c:361:request_in_callback()) peer: 12345-0@lo (source: 12345-0@lo) 00000100:00100000:3.0:1776178743.143717:0:3354:0:(client.c:1733:ptlrpc_send_new_req()) Sending RPC req@0000000090a7aa50 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_01:lustre-MDT0001-lwp-OST0001_UUID:3354:1862456998123712:0@lo:400:kworker/u8:0.0 00000100:00100000:3.0:1776178743.143738:0:3354:0:(events.c:361:request_in_callback()) peer: 12345-0@lo (source: 12345-0@lo) 00000100:00100000:3.0:1776178743.143763:0:3354:0:(ptlrpcd.c:415:ptlrpcd_check()) transfer 1 async RPCs [0->1] 00000100:00100000:3.0:1776178743.143775:0:3354:0:(client.c:1733:ptlrpc_send_new_req()) Sending RPC req@00000000380b40c4 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_01:lustre-MDT0001-lwp-OST0000_UUID:3354:1862456998123584:0@lo:400:kworker/u8:0.0 00000100:00100000:3.0:1776178743.143797:0:3354:0:(events.c:361:request_in_callback()) peer: 12345-0@lo (source: 12345-0@lo) 00000100:00100000:3.0:1776178743.143827:0:6256:0:(service.c:2152:ptlrpc_server_handle_req_in()) got req x1862456998123520 00000100:00100000:3.0:1776178743.143853:0:6256:0:(service.c:2152:ptlrpc_server_handle_req_in()) got req x1862456998123648 00000100:00100000:3.0:1776178743.143860:0:6256:0:(service.c:2152:ptlrpc_server_handle_req_in()) got req x1862456998123968 00000100:00100000:3.0:1776178743.143865:0:6256:0:(service.c:2152:ptlrpc_server_handle_req_in()) got req x1862456998123712 00000100:00100000:3.0:1776178743.143882:0:6256:0:(service.c:2152:ptlrpc_server_handle_req_in()) got req x1862456998123584 00000100:00100000:3.0:1776178743.143893:0:6256:0:(nrs_fifo.c:179:nrs_fifo_req_get()) NRS start fifo request from 12345-0@lo, seq: 1244 00000100:00100000:3.0:1776178743.143898:0:6256:0:(service.c:2324:ptlrpc_server_handle_request()) Handling RPC req@000000009de51106 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_001:lustre-MDT0001-lwp-OST0001_UUID+5:3354:x1862456998123712:12345-0@lo:400:kworker/u8:0.0 00000100:00100000:3.0:1776178743.143943:0:6256:0:(service.c:2372:ptlrpc_server_handle_request()) Handled RPC req@000000009de51106 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_001:lustre-MDT0001-lwp-OST0001_UUID+5:3354:x1862456998123712:12345-0@lo:400:kworker/u8:0.0 Request processed in 44us (204us total) trans 0 rc 0/0 00000100:00100000:3.0:1776178743.143949:0:6256:0:(nrs_fifo.c:241:nrs_fifo_req_stop()) NRS stop fifo request from 12345-0@lo, seq: 1244 00000100:00100000:3.0:1776178743.143965:0:6256:0:(nrs_fifo.c:179:nrs_fifo_req_get()) NRS start fifo request from 12345-0@lo, seq: 1245 00000100:00100000:3.0:1776178743.143968:0:6256:0:(service.c:2324:ptlrpc_server_handle_request()) Handling RPC req@0000000067c1ae4f pname:cluuid+ref:pid:xid:nid:opc:job mdt00_001:lustre-MDT0001-lwp-OST0000_UUID+5:3354:x1862456998123584:12345-0@lo:400:kworker/u8:0.0 00000100:00100000:3.0:1776178743.143992:0:6256:0:(service.c:2372:ptlrpc_server_handle_request()) Handled RPC req@0000000067c1ae4f pname:cluuid+ref:pid:xid:nid:opc:job mdt00_001:lustre-MDT0001-lwp-OST0000_UUID+5:3354:x1862456998123584:12345-0@lo:400:kworker/u8:0.0 Request processed in 23us (195us total) trans 0 rc 0/0 00000100:00100000:3.0:1776178743.143998:0:6256:0:(nrs_fifo.c:241:nrs_fifo_req_stop()) NRS stop fifo request from 12345-0@lo, seq: 1245 00000100:00100000:3.0:1776178743.144009:0:6256:0:(nrs_fifo.c:179:nrs_fifo_req_get()) NRS start fifo request from 12345-0@lo, seq: 7045 00000100:00100000:3.0:1776178743.144012:0:6256:0:(service.c:2324:ptlrpc_server_handle_request()) Handling RPC req@0000000015095c30 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_001:0+-99:3352:x1862456998123520:12345-0@lo:38:kworker/u8:0.0 00010000:02000000:3.0:1776178743.144024:0:6256:0:(ldlm_lib.c:1139:target_handle_connect()) lustre-MDT0000: Not available for connect from 0@lo (stopping) 00000100:00100000:3.0:1776178743.144054:0:6256:0:(service.c:2372:ptlrpc_server_handle_request()) Handled RPC req@0000000015095c30 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_001:0+-99:3352:x1862456998123520:12345-0@lo:38:kworker/u8:0.0 Request processed in 41us (488us total) trans 0 rc -19/-19 00000100:00100000:3.0:1776178743.144085:0:6256:0:(nrs_fifo.c:241:nrs_fifo_req_stop()) NRS stop fifo request from 12345-0@lo, seq: 7045 00000100:00100000:3.0:1776178743.144097:0:6256:0:(nrs_fifo.c:179:nrs_fifo_req_get()) NRS start fifo request from 12345-0@lo, seq: 7046 00000100:00100000:3.0:1776178743.144101:0:6256:0:(service.c:2324:ptlrpc_server_handle_request()) Handling RPC req@00000000c63a98c1 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_001:0+-99:3352:x1862456998123648:12345-0@lo:38:kworker/u8:0.0 00010000:02000000:3.0:1776178743.144107:0:6256:0:(ldlm_lib.c:1139:target_handle_connect()) lustre-MDT0000: Not available for connect from 0@lo (stopping) 00000100:00100000:3.0:1776178743.144128:0:6256:0:(service.c:2372:ptlrpc_server_handle_request()) Handled RPC req@00000000c63a98c1 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_001:0+-99:3352:x1862456998123648:12345-0@lo:38:kworker/u8:0.0 Request processed in 27us (504us total) trans 0 rc -19/-19 00000100:00100000:3.0:1776178743.144133:0:6256:0:(nrs_fifo.c:241:nrs_fifo_req_stop()) NRS stop fifo request from 12345-0@lo, seq: 7046 00000100:00100000:3.0:1776178743.144143:0:6256:0:(nrs_fifo.c:179:nrs_fifo_req_get()) NRS start fifo request from 12345-0@lo, seq: 7047 00000100:00100000:3.0:1776178743.144146:0:6256:0:(service.c:2324:ptlrpc_server_handle_request()) Handling RPC req@000000003334dea7 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_001:0+-99:3352:x1862456998123968:12345-0@lo:38:kworker/u8:0.0 00010000:02000000:3.0:1776178743.144150:0:6256:0:(ldlm_lib.c:1139:target_handle_connect()) lustre-MDT0000: Not available for connect from 0@lo (stopping) 00000100:00100000:3.0:1776178743.144168:0:6256:0:(service.c:2372:ptlrpc_server_handle_request()) Handled RPC req@000000003334dea7 pname:cluuid+ref:pid:xid:nid:opc:job mdt00_001:0+-99:3352:x1862456998123968:12345-0@lo:38:kworker/u8:0.0 Request processed in 21us (480us total) trans 0 rc -19/-19 00000100:00100000:3.0:1776178743.144173:0:6256:0:(nrs_fifo.c:241:nrs_fifo_req_stop()) NRS stop fifo request from 12345-0@lo, seq: 7047 00000100:00100000:3.0:1776178743.144206:0:3354:0:(client.c:2215:ptlrpc_check_set()) Completed RPC req@00000000380b40c4 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_01:lustre-MDT0001-lwp-OST0000_UUID:3354:1862456998123584:0@lo:400:kworker/u8:0.0 00000100:00100000:3.0:1776178743.144216:0:3354:0:(client.c:2215:ptlrpc_check_set()) Completed RPC req@0000000090a7aa50 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_01:lustre-MDT0001-lwp-OST0001_UUID:3354:1862456998123712:0@lo:400:kworker/u8:0.0 00000100:00080000:3.0:1776178743.144243:0:3352:0:(import.c:86:import_set_state_nolock()) 0000000047601a98 lustre-MDT0000_UUID: changing import state from CONNECTING to DISCONN 00000100:00080000:3.0:1776178743.144247:0:3352:0:(import.c:1429:ptlrpc_connect_interpret()) recovery of lustre-MDT0000_UUID on 192.168.204.103@tcp failed (-19) 00000100:00100000:3.0:1776178743.144249:0:3352:0:(client.c:2215:ptlrpc_check_set()) Completed RPC req@0000000037059d8f pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:lustre-MDT0000-lwp-OST0000_UUID:3352:1862456998123520:0@lo:38:kworker/u8:0.0 00000100:00080000:3.0:1776178743.144256:0:3352:0:(import.c:86:import_set_state_nolock()) 00000000dfc8db13 lustre-MDT0000_UUID: changing import state from CONNECTING to DISCONN 00000100:00080000:3.0:1776178743.144257:0:3352:0:(import.c:1429:ptlrpc_connect_interpret()) recovery of lustre-MDT0000_UUID on 192.168.204.103@tcp failed (-19) 00000100:00100000:3.0:1776178743.144259:0:3352:0:(client.c:2215:ptlrpc_check_set()) Completed RPC req@0000000098e424d0 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:lustre-MDT0000-lwp-OST0001_UUID:3352:1862456998123648:0@lo:38:kworker/u8:0.0 00000100:00080000:3.0:1776178743.144265:0:3352:0:(import.c:86:import_set_state_nolock()) 00000000a7c55b1f lustre-MDT0000_UUID: changing import state from CONNECTING to DISCONN 00000100:00080000:3.0:1776178743.144267:0:3352:0:(import.c:1429:ptlrpc_connect_interpret()) recovery of lustre-MDT0000_UUID on 192.168.204.103@tcp failed (-19) 00000100:00100000:3.0:1776178743.144269:0:3352:0:(client.c:2215:ptlrpc_check_set()) Completed RPC req@000000007d5cfd93 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:lustre-MDT0000-lwp-MDT0001_UUID:3352:1862456998123968:0@lo:38:kworker/u8:0.0 00000100:00100000:2.0:1776178743.144323:0:3355:0:(client.c:1733:ptlrpc_send_new_req()) Sending RPC req@000000000e0e1967 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_02:lustre-MDT0001-mdtlov_UUID:3355:1862456998123840:0@lo:400:kworker/u8:0.0 00000100:00100000:2.0:1776178743.144357:0:3355:0:(events.c:361:request_in_callback()) peer: 12345-0@lo (source: 12345-0@lo) 00000100:00100000:2.0:1776178743.144385:0:3355:0:(ptlrpcd.c:415:ptlrpcd_check()) transfer 2 async RPCs [3->2] 00000100:00100000:2.0:1776178743.144391:0:3355:0:(client.c:1733:ptlrpc_send_new_req()) Sending RPC req@00000000bd795500 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_02:ac61f839-0e8e-4cac-9c59-08bcfbbad330:3355:1862456998123456:0@lo:400:kworker/u8:0.0 00000100:00100000:2.0:1776178743.144417:0:3355:0:(events.c:361:request_in_callback()) peer: 12345-0@lo (source: 12345-0@lo) 00000100:00100000:2.0:1776178743.144444:0:3355:0:(client.c:1733:ptlrpc_send_new_req()) Sending RPC req@000000006b0ae3d8 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_02:lustre-MDT0001-mdtlov_UUID:3355:1862456998123904:0@lo:400:kworker/u8:0.0 00000100:00100000:2.0:1776178743.144463:0:3355:0:(events.c:361:request_in_callback()) peer: 12345-0@lo (source: 12345-0@lo) 00000100:00100000:2.0:1776178743.144494:0:14128:0:(service.c:2152:ptlrpc_server_handle_req_in()) got req x1862456998123776 00000100:00100000:2.0:1776178743.144519:0:14128:0:(nrs_fifo.c:179:nrs_fifo_req_get()) NRS start fifo request from 12345-0@lo, seq: 813 00000100:00100000:2.0:1776178743.144522:0:14128:0:(service.c:2324:ptlrpc_server_handle_request()) Handling RPC req@00000000af9e3283 pname:cluuid+ref:pid:xid:nid:opc:job mdt_out00_002:0+-99:3352:x1862456998123776:12345-0@lo:38:kworker/u8:0.0 00010000:02000000:2.0:1776178743.144529:0:14128:0:(ldlm_lib.c:1139:target_handle_connect()) lustre-MDT0000: Not available for connect from 0@lo (stopping) 00000100:00100000:2.0:1776178743.144558:0:14128:0:(service.c:2372:ptlrpc_server_handle_request()) Handled RPC req@00000000af9e3283 pname:cluuid+ref:pid:xid:nid:opc:job mdt_out00_002:0+-99:3352:x1862456998123776:12345-0@lo:38:kworker/u8:0.0 Request processed in 34us (901us total) trans 0 rc -19/-19 00000100:00100000:2.0:1776178743.144564:0:14128:0:(nrs_fifo.c:241:nrs_fifo_req_stop()) NRS stop fifo request from 12345-0@lo, seq: 813 00000100:00100000:2.0:1776178743.144589:0:20755:0:(service.c:2152:ptlrpc_server_handle_req_in()) got req x1862456998123840 00000100:00100000:2.0:1776178743.144608:0:20755:0:(service.c:2152:ptlrpc_server_handle_req_in()) got req x1862456998123904 00000100:00100000:2.0:1776178743.144620:0:20755:0:(nrs_fifo.c:179:nrs_fifo_req_get()) NRS start fifo request from 12345-0@lo, seq: 933 00000100:00100000:2.0:1776178743.144624:0:20755:0:(service.c:2324:ptlrpc_server_handle_request()) Handling RPC req@00000000d4b41d38 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_006:lustre-MDT0001-mdtlov_UUID+5:3355:x1862456998123840:12345-0@lo:400:kworker/u8:0.0 00000100:00100000:2.0:1776178743.144651:0:20755:0:(service.c:2372:ptlrpc_server_handle_request()) Handled RPC req@00000000d4b41d38 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_006:lustre-MDT0001-mdtlov_UUID+5:3355:x1862456998123840:12345-0@lo:400:kworker/u8:0.0 Request processed in 27us (294us total) trans 0 rc 0/0 00000100:00100000:2.0:1776178743.144657:0:20755:0:(nrs_fifo.c:241:nrs_fifo_req_stop()) NRS stop fifo request from 12345-0@lo, seq: 933 00000100:00100000:2.0:1776178743.144666:0:20755:0:(nrs_fifo.c:179:nrs_fifo_req_get()) NRS start fifo request from 12345-0@lo, seq: 934 00000100:00100000:2.0:1776178743.144669:0:20755:0:(service.c:2324:ptlrpc_server_handle_request()) Handling RPC req@0000000052775143 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_006:lustre-MDT0001-mdtlov_UUID+5:3355:x1862456998123904:12345-0@lo:400:kworker/u8:0.0 00000100:00100000:2.0:1776178743.144688:0:20755:0:(service.c:2372:ptlrpc_server_handle_request()) Handled RPC req@0000000052775143 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost00_006:lustre-MDT0001-mdtlov_UUID+5:3355:x1862456998123904:12345-0@lo:400:kworker/u8:0.0 Request processed in 19us (225us total) trans 0 rc 0/0 00000100:00100000:2.0:1776178743.144693:0:20755:0:(nrs_fifo.c:241:nrs_fifo_req_stop()) NRS stop fifo request from 12345-0@lo, seq: 934 00000100:00100000:2.0:1776178743.144723:0:3355:0:(client.c:2215:ptlrpc_check_set()) Completed RPC req@000000006b0ae3d8 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_02:lustre-MDT0001-mdtlov_UUID:3355:1862456998123904:0@lo:400:kworker/u8:0.0 00000100:00100000:2.0:1776178743.144732:0:3355:0:(client.c:2215:ptlrpc_check_set()) Completed RPC req@000000000e0e1967 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_02:lustre-MDT0001-mdtlov_UUID:3355:1862456998123840:0@lo:400:kworker/u8:0.0 00000100:00080000:3.0:1776178743.144835:0:3352:0:(import.c:86:import_set_state_nolock()) 00000000ba4f1821 lustre-MDT0000_UUID: changing import state from CONNECTING to DISCONN 00000100:00080000:3.0:1776178743.144839:0:3352:0:(import.c:1429:ptlrpc_connect_interpret()) recovery of lustre-MDT0000_UUID on 192.168.204.103@tcp failed (-19) 00000100:00100000:3.0:1776178743.144843:0:3352:0:(client.c:2215:ptlrpc_check_set()) Completed RPC req@000000001f101832 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_rcv:lustre-MDT0001-mdtlov_UUID:3352:1862456998123776:0@lo:38:kworker/u8:0.0 00000100:00100000:0.0:1776178743.145352:0:20140:0:(service.c:2152:ptlrpc_server_handle_req_in()) got req x1862456998123456 00000100:00100000:0.0:1776178743.145386:0:20140:0:(nrs_fifo.c:179:nrs_fifo_req_get()) NRS start fifo request from 12345-0@lo, seq: 366 00000100:00100000:0.0:1776178743.145390:0:20140:0:(service.c:2324:ptlrpc_server_handle_request()) Handling RPC req@0000000048e8901d pname:cluuid+ref:pid:xid:nid:opc:job ll_mgs_0002:ac61f839-0e8e-4cac-9c59-08bcfbbad330+18:3355:x1862456998123456:12345-0@lo:400:kworker/u8:0.0 00000100:00100000:0.0:1776178743.153013:0:20140:0:(service.c:2372:ptlrpc_server_handle_request()) Handled RPC req@0000000048e8901d pname:cluuid+ref:pid:xid:nid:opc:job ll_mgs_0002:ac61f839-0e8e-4cac-9c59-08bcfbbad330+18:3355:x1862456998123456:12345-0@lo:400:kworker/u8:0.0 Request processed in 7613us (8586us total) trans 0 rc 0/0 00000100:00100000:2.0:1776178743.153025:0:3355:0:(client.c:2215:ptlrpc_check_set()) Completed RPC req@00000000bd795500 pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_02:ac61f839-0e8e-4cac-9c59-08bcfbbad330:3355:1862456998123456:0@lo:400:kworker/u8:0.0 00000100:00100000:0.0:1776178743.153033:0:20140:0:(nrs_fifo.c:241:nrs_fifo_req_stop()) NRS stop fifo request from 12345-0@lo, seq: 366 00000020:02000400:3.0:1776178743.649192:0:22200:0:(genops.c:1786:obd_exports_barrier()) lustre-MDT0000 is waiting for obd_unlinked_exports more than 256 seconds. The obd refcount = 4. Is it stuck? 00000020:02000400:3.0:1776178743.654193:0:22200:0:(genops.c:1752:print_export_data()) lustre-MDT0000: UNLINKED 00000000ca22e26f b50963bd-8eba-4432-9a54-8888d00e3fa1 1 (0 0 0) 1 0 1 0: 0000000000000000 8589934704 stale:0 00010000:00010000:2.0:1776178743.780077:0:9224:0:(ldlm_lockd.c:2792:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) 00000100:00100000:0.0:1776178744.676663:0:3354:0:(ptlrpcd.c:415:ptlrpcd_check()) transfer 1 async RPCs [0->1] 00000100:00100000:0.0:1776178744.676688:0:3354:0:(client.c:1733:ptlrpc_send_new_req()) Sending RPC req@0000000028dd2f9a pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_01:lustre-MDT0001-mdtlov_UUID:3354:1862456998124032:0@lo:13:osp-pre-1-1.0 00000100:00100000:0.0:1776178744.676756:0:3354:0:(events.c:361:request_in_callback()) peer: 12345-0@lo (source: 12345-0@lo) 00000100:00100000:2.0:1776178744.676854:0:9554:0:(service.c:2152:ptlrpc_server_handle_req_in()) got req x1862456998124032 00000100:00100000:2.0:1776178744.676905:0:9554:0:(nrs_fifo.c:179:nrs_fifo_req_get()) NRS start fifo request from 12345-0@lo, seq: 699 00000100:00100000:2.0:1776178744.676910:0:9554:0:(service.c:2324:ptlrpc_server_handle_request()) Handling RPC req@00000000775bd6f2 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_create00:lustre-MDT0001-mdtlov_UUID+5:3354:x1862456998124032:12345-0@lo:13:osp-pre-1-1.0 00002000:00000024:2.0:1776178744.676938:0:9554:0:(ofd_obd.c:740:ofd_statfs()) blocks cached 229376 granted 491746368 pending 0 free 3923689472 avail 3692167168 00000100:00100000:2.0:1776178744.676980:0:9554:0:(service.c:2372:ptlrpc_server_handle_request()) Handled RPC req@00000000775bd6f2 pname:cluuid+ref:pid:xid:nid:opc:job ll_ost_create00:lustre-MDT0001-mdtlov_UUID+5:3354:x1862456998124032:12345-0@lo:13:osp-pre-1-1.0 Request processed in 69us (225us total) trans 0 rc 0/0 00000100:00100000:2.0:1776178744.676987:0:9554:0:(nrs_fifo.c:241:nrs_fifo_req_stop()) NRS stop fifo request from 12345-0@lo, seq: 699 00000100:00100000:2.0:1776178744.677031:0:3354:0:(client.c:2215:ptlrpc_check_set()) Completed RPC req@0000000028dd2f9a pname:cluuid:pid:xid:nid:opc:job ptlrpcd_00_01:lustre-MDT0001-mdtlov_UUID:3354:1862456998124032:0@lo:13:osp-pre-1-1.0 00010000:00010000:1.0:1776178744.802532:0:9225:0:(ldlm_lockd.c:2792:ldlm_bl_get_work()) No blwi found in queue (no bl locks in queue) Debug log: 31551 lines, 31551 kept, 0 dropped, 0 bad.